2024-04-16T16:38:39.480 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-16T16:38:39.480 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_mchangir@teuthology --archive /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438 --verbose --name mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/orig.config.yaml 2024-04-16T16:38:39.539 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:38:39.630 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438 branch: wip-mchangir-qa-debug-resource-temporarily-unavailable-issue description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: mchangir@redhat.com first_in_suite: false job_id: '7658438' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-qa-debug-resource-temporarily-unavailable-issue ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 9b42bfbeda298874861ece590aa565cf64ac8c45 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9b42bfbeda298874861ece590aa565cf64ac8c45 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: qa-debug-resource-temporarily-unavailable-issue sha1: b137e4a675398e295567f9652210865a4d573fac owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2381 sha1: 9b42bfbeda298874861ece590aa565cf64ac8c45 sleep_before_teardown: 0 suite: fs suite_branch: qa-debug-resource-temporarily-unavailable-issue suite_path: /home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa suite_relpath: qa suite_repo: https://github.com/mchangir/ceph.git suite_sha1: b137e4a675398e295567f9652210865a4d573fac syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi018.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAo4EezMFHe1z901+6BtI0fVVwcpdTxG4yIKOw6LLTIs95phj17UsQ982onIZxnxa5uJO7iw3QvIgJ8j3M2GjEU= smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDhzcHsZsvFDokU439Oqm3OxXlUxW+WgmFwyGYD6esZF/hUhBBQBW9Y1ABceWKyM8NjhjqzMDiJg06elct/2WpE= smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiririN9/EWV7rRFNXeAQeezT+rGESyk0NhmvQT5TpQZQGQsT3MwV83KrND2u57or8cyOpX2NQsWXfkK1+y25I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-16_16:32:10 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-16T16:38:39.630 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa; will attempt to use it 2024-04-16T16:38:39.631 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks 2024-04-16T16:38:39.631 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-16T16:38:39.633 INFO:teuthology.task.internal:Checking packages... 2024-04-16T16:38:39.770 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9b42bfbeda298874861ece590aa565cf64ac8c45' 2024-04-16T16:38:39.770 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-16T16:38:39.770 INFO:teuthology.packaging:ref: None 2024-04-16T16:38:39.770 INFO:teuthology.packaging:tag: None 2024-04-16T16:38:39.770 INFO:teuthology.packaging:branch: wip-mchangir-qa-debug-resource-temporarily-unavailable-issue 2024-04-16T16:38:39.770 INFO:teuthology.packaging:sha1: 9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:38:39.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue 2024-04-16T16:38:39.983 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3101.g9b42bfbe 2024-04-16T16:38:39.984 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-16T16:38:39.992 INFO:teuthology.task.internal:no buildpackages task found 2024-04-16T16:38:39.992 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-16T16:38:40.050 INFO:teuthology.task.internal:Saving configuration 2024-04-16T16:38:40.071 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-16T16:38:40.085 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T16:38:40.108 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.017147', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAo4EezMFHe1z901+6BtI0fVVwcpdTxG4yIKOw6LLTIs95phj17UsQ982onIZxnxa5uJO7iw3QvIgJ8j3M2GjEU='} 2024-04-16T16:38:40.132 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.015479', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDhzcHsZsvFDokU439Oqm3OxXlUxW+WgmFwyGYD6esZF/hUhBBQBW9Y1ABceWKyM8NjhjqzMDiJg06elct/2WpE='} 2024-04-16T16:38:40.156 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.018500', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiririN9/EWV7rRFNXeAQeezT+rGESyk0NhmvQT5TpQZQGQsT3MwV83KrND2u57or8cyOpX2NQsWXfkK1+y25I='} 2024-04-16T16:38:40.156 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-16T16:38:40.168 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-16T16:38:40.168 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-16T16:38:40.168 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-16T16:38:40.168 INFO:teuthology.run_tasks:Running task console_log... 2024-04-16T16:38:40.276 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6a6b31f0d0>, signals=[15]) 2024-04-16T16:38:40.276 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-16T16:38:40.282 INFO:teuthology.task.internal:Opening connections... 2024-04-16T16:38:40.283 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T16:38:40.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:40.391 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-04-16T16:38:40.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:40.493 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-04-16T16:38:40.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:40.594 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-16T16:38:40.602 DEBUG:teuthology.orchestra.run.smithi018:> uname -m 2024-04-16T16:38:40.621 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2024-04-16T16:38:40.621 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="9" 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="9" 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:40.679 INFO:teuthology.orchestra.run.smithi018.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:40.680 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:40.680 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:40.680 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:40.680 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:40.680 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:40.680 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2024-04-16T16:38:40.697 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-04-16T16:38:40.715 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-04-16T16:38:40.715 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:40.773 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:40.774 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-04-16T16:38:40.789 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-04-16T16:38:40.807 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-04-16T16:38:40.807 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-04-16T16:38:40.865 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-04-16T16:38:40.865 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-04-16T16:38:40.865 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-04-16T16:38:40.865 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:40.865 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-04-16T16:38:40.865 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:40.865 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:40.865 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:40.866 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:40.866 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:40.866 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:40.866 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:40.866 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:40.866 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:40.866 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-04-16T16:38:40.881 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-16T16:38:40.892 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-16T16:38:40.897 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-16T16:38:40.898 DEBUG:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2024-04-16T16:38:40.900 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-04-16T16:38:40.903 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-04-16T16:38:40.922 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-16T16:38:40.928 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-16T16:38:40.928 DEBUG:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2024-04-16T16:38:40.958 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-04-16T16:38:40.960 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-04-16T16:38:40.985 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-16T16:38:41.091 INFO:teuthology.run_tasks:Running task kernel... 2024-04-16T16:38:41.136 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-16T16:38:41.137 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-16T16:38:41.137 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-16T16:38:41.137 DEBUG:teuthology.orchestra.run.smithi018:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T16:38:41.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:38:41.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-16T16:38:41.396 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-16T16:38:41.396 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-16T16:38:41.397 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-04-16T16:38:41.414 INFO:teuthology.orchestra.run.smithi018.stdout:5.14.0-361.el9.x86_64 2024-04-16T16:38:41.415 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-16T16:38:41.415 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-16T16:38:41.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum upgrade -y linux-firmware 2024-04-16T16:38:43.642 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-04-16T16:38:47.518 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - BaseOS 753 kB/s | 2.0 MB 00:02 2024-04-16T16:38:49.666 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.0 MB 00:01 2024-04-16T16:38:52.959 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 19 MB 00:02 2024-04-16T16:38:57.068 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-16T16:38:58.535 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-04-16T16:38:59.994 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-16T16:39:05.228 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-16T16:39:07.781 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-16T16:39:07.792 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-16T16:39:07.792 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-16T16:39:07.792 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-16T16:39:07.792 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2024-04-16T16:39:07.792 INFO:teuthology.orchestra.run.smithi018.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-16T16:39:07.792 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:39:07.792 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-16T16:39:07.792 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-16T16:39:07.793 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 1 Package 2024-04-16T16:39:07.793 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:39:07.793 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 375 M 2024-04-16T16:39:07.793 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-16T16:39:15.329 INFO:teuthology.orchestra.run.smithi018.stdout:linux-firmware-20240219-143.el9.noarch.rpm 50 MB/s | 375 MB 00:07 2024-04-16T16:39:15.329 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:39:15.330 INFO:teuthology.orchestra.run.smithi018.stdout:Total 50 MB/s | 375 MB 00:07 2024-04-16T16:39:15.330 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-16T16:39:15.416 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-16T16:39:15.416 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-16T16:39:17.938 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-16T16:39:17.938 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-16T16:39:24.479 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-16T16:39:24.702 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-16T16:39:24.827 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-16T16:39:26.154 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-16T16:39:26.154 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-16T16:39:26.541 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-16T16:39:26.542 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:39:26.542 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2024-04-16T16:39:26.542 INFO:teuthology.orchestra.run.smithi018.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-16T16:39:26.542 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:39:26.542 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-16T16:39:26.828 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-16T16:39:26.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-16T16:39:26.972 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-16T16:39:26.972 DEBUG:teuthology.orchestra.run.smithi018:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-16T16:39:26.975 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi018.front.sepia.ceph.com to complete... 2024-04-16T16:39:41.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16 16:39:41 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-16T16:39:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:FINISHED --2024-04-16 16:39:41-- 2024-04-16T16:39:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:Total wall clock time: 15s 2024-04-16T16:39:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:Downloaded: 1 files, 341M in 14s (24.2 MB/s) 2024-04-16T16:39:41.525 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-16T16:39:41.525 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-16T16:39:43.682 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying... ######################################## 2024-04-16T16:39:43.736 INFO:teuthology.orchestra.run.smithi018.stdout:Preparing... ######################################## 2024-04-16T16:39:43.820 INFO:teuthology.orchestra.run.smithi018.stdout:Updating / installing... 2024-04-16T16:39:52.129 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-16T16:40:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-16T16:40:32.298 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi018.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-16T16:40:32.298 DEBUG:teuthology.orchestra.run.smithi018:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-16T16:40:32.372 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-16T16:40:32.373 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-16T16:40:32.374 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-16T16:40:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-16T16:40:32.376 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-16T16:40:32.377 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-16T16:40:32.378 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-16T16:40:32.379 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-16T16:40:32.380 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-16T16:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-16T16:40:32.382 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-16T16:40:32.383 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-16T16:40:32.384 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-16T16:40:32.385 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-16T16:40:32.386 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-16T16:40:32.387 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-16T16:40:32.388 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-16T16:40:32.389 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-16T16:40:32.390 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-16T16:40:32.391 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-16T16:40:32.392 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-16T16:40:32.393 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-16T16:40:32.394 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-16T16:40:32.395 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-16T16:40:32.396 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-16T16:40:32.397 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-16T16:40:32.398 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-16T16:40:32.399 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-16T16:40:32.400 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-16T16:40:32.401 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-16T16:40:32.402 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-16T16:40:32.403 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-16T16:40:32.404 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-16T16:40:32.405 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-16T16:40:32.406 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-16T16:40:32.407 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-16T16:40:32.408 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-16T16:40:32.409 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-16T16:40:32.410 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-16T16:40:32.411 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-16T16:40:32.412 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-16T16:40:32.413 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-16T16:40:32.414 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-16T16:40:32.415 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-16T16:40:32.416 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-16T16:40:32.417 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-16T16:40:32.418 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-16T16:40:32.419 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-16T16:40:32.420 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-16T16:40:32.421 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-16T16:40:32.422 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-16T16:40:32.423 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-16T16:40:32.424 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-16T16:40:32.425 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-16T16:40:32.426 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-16T16:40:32.427 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-16T16:40:32.428 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-16T16:40:32.429 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-16T16:40:32.430 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-16T16:40:32.431 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-16T16:40:32.432 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-16T16:40:32.433 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-16T16:40:32.434 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-16T16:40:32.435 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-16T16:40:32.436 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-16T16:40:32.437 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-16T16:40:32.438 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-16T16:40:32.439 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-16T16:40:32.440 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-16T16:40:32.441 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-16T16:40:32.442 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-16T16:40:32.443 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-16T16:40:32.444 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-16T16:40:32.445 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-16T16:40:32.446 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-16T16:40:32.447 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-16T16:40:32.448 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-16T16:40:32.449 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-16T16:40:32.450 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-16T16:40:32.451 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-16T16:40:32.452 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-16T16:40:32.453 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-16T16:40:32.454 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-16T16:40:32.455 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-16T16:40:32.456 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-16T16:40:32.457 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-16T16:40:32.458 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-16T16:40:32.459 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-16T16:40:32.460 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-16T16:40:32.461 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-16T16:40:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-16T16:40:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-16T16:40:32.464 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-16T16:40:32.465 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-16T16:40:32.466 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-16T16:40:32.467 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-16T16:40:32.468 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-16T16:40:32.469 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-16T16:40:32.470 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-16T16:40:32.471 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-16T16:40:32.472 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-16T16:40:32.473 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-16T16:40:32.474 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-16T16:40:32.475 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-16T16:40:32.476 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-16T16:40:32.477 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-16T16:40:32.478 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-16T16:40:32.479 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-16T16:40:32.480 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-16T16:40:32.481 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-16T16:40:32.482 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-16T16:40:32.483 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-16T16:40:32.484 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-16T16:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-16T16:40:32.486 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-16T16:40:32.487 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-16T16:40:32.488 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-16T16:40:32.489 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-16T16:40:32.490 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-16T16:40:32.491 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-16T16:40:32.492 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-16T16:40:32.496 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:40:32.496 DEBUG:teuthology.orchestra.run.smithi018:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-16T16:40:32.520 INFO:teuthology.orchestra.run.smithi018.stdout:postinstall scriptlet (using /bin/sh): 2024-04-16T16:40:32.520 INFO:teuthology.orchestra.run.smithi018.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-16T16:40:32.520 INFO:teuthology.orchestra.run.smithi018.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout:fi 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout:for file in vmlinuz System.map config; do 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout: fi 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout:done 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout:fi 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout:fi 2024-04-16T16:40:32.521 INFO:teuthology.orchestra.run.smithi018.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-16T16:40:32.522 INFO:teuthology.orchestra.run.smithi018.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-16T16:40:32.522 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:40:32.522 INFO:teuthology.orchestra.run.smithi018.stdout:fi 2024-04-16T16:40:32.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -qi grub2-tools 2024-04-16T16:40:32.604 INFO:teuthology.orchestra.run.smithi018.stdout:Name : grub2-tools 2024-04-16T16:40:32.604 INFO:teuthology.orchestra.run.smithi018.stdout:Epoch : 1 2024-04-16T16:40:32.604 INFO:teuthology.orchestra.run.smithi018.stdout:Version : 2.06 2024-04-16T16:40:32.604 INFO:teuthology.orchestra.run.smithi018.stdout:Release : 68.el9 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Architecture: x86_64 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Group : Unspecified 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Size : 8264254 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:License : GPLv3+ 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Packager : builder@centos.org 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Vendor : CentOS 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Summary : Support tools for GRUB. 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:Description : 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:40:32.605 INFO:teuthology.orchestra.run.smithi018.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-16T16:40:32.606 INFO:teuthology.orchestra.run.smithi018.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-16T16:40:32.606 INFO:teuthology.orchestra.run.smithi018.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-16T16:40:32.606 INFO:teuthology.orchestra.run.smithi018.stdout:hardware devices. 2024-04-16T16:40:32.606 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:40:32.606 INFO:teuthology.orchestra.run.smithi018.stdout:This subpackage provides tools for support of all platforms. 2024-04-16T16:40:32.607 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-16T16:40:32.607 INFO:teuthology.task.kernel:Updating grub on smithi018 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:40:32.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-16T16:40:33.200 INFO:teuthology.orchestra.run.smithi018.stderr:Generating grub configuration file ... 2024-04-16T16:40:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-16T16:40:34.637 INFO:teuthology.orchestra.run.smithi018.stderr:done 2024-04-16T16:40:34.639 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2024-04-16T16:40:34.654 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.eWODBLimaQ 2024-04-16T16:40:34.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eWODBLimaQ 2024-04-16T16:40:34.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.eWODBLimaQ 2024-04-16T16:40:34.880 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.eWODBLimaQ is 6KB 2024-04-16T16:40:34.928 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.eWODBLimaQ 2024-04-16T16:40:34.943 DEBUG:teuthology.orchestra.run.smithi018:> sudo /bin/ls /boot/loader/entries || true 2024-04-16T16:40:35.011 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-16T16:40:35.011 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-16T16:40:35.011 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-16T16:40:35.011 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-16T16:40:35.012 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:40:35.169 DEBUG:teuthology.orchestra.run.smithi018:> sudo shutdown -r now 2024-04-16T16:41:05.172 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-16T16:41:05.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-16T16:41:05.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:41:23.743 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.18 2024-04-16T16:41:32.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-16T16:41:32.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:42:32.746 DEBUG:teuthology.orchestra.remote:timed out 2024-04-16T16:42:44.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-16T16:42:44.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:42:45.236 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-04-16T16:42:45.912 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-16T16:42:45.912 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-16T16:42:45.912 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-04-16T16:42:45.965 INFO:teuthology.orchestra.run.smithi018.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:42:45.965 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:42:45.965 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T16:42:45.965 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-16T16:42:46.966 DEBUG:teuthology.parallel:result is None 2024-04-16T16:42:46.967 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-16T16:42:46.975 INFO:teuthology.task.internal:Creating test directory... 2024-04-16T16:42:46.975 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T16:42:46.983 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T16:42:46.986 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T16:42:47.035 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-16T16:42:47.042 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-16T16:42:47.048 INFO:teuthology.task.internal:Creating archive directory... 2024-04-16T16:42:47.048 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T16:42:47.056 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T16:42:47.058 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T16:42:47.121 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-16T16:42:47.128 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-16T16:42:47.128 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T16:42:47.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:42:47.145 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T16:42:47.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:42:47.162 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T16:42:47.170 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T16:42:47.188 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T16:42:47.250 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:42:47.272 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:42:47.278 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:42:47.300 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:42:47.315 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:42:47.388 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:42:47.393 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-16T16:42:47.400 INFO:teuthology.task.internal:Configuring sudo... 2024-04-16T16:42:47.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T16:42:47.406 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T16:42:47.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T16:42:47.500 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-16T16:42:47.509 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-16T16:42:47.510 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T16:42:47.516 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T16:42:47.519 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T16:42:47.567 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:42:47.647 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:42:47.756 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:42:47.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:42:47.943 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:42:47.943 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T16:42:48.033 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:42:48.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:42:48.140 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:42:48.225 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:42:48.273 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:42:48.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T16:42:48.344 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:42:48.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:42:48.452 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:42:48.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:42:48.544 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:42:48.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T16:42:48.616 DEBUG:teuthology.orchestra.run.smithi018:> sudo service rsyslog restart 2024-04-16T16:42:48.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-04-16T16:42:48.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-04-16T16:42:48.671 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T16:42:48.698 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T16:42:48.750 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T16:42:49.276 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-16T16:42:49.284 INFO:teuthology.task.internal:Starting timer... 2024-04-16T16:42:49.284 INFO:teuthology.run_tasks:Running task pcp... 2024-04-16T16:42:49.294 INFO:teuthology.run_tasks:Running task selinux... 2024-04-16T16:42:49.303 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-16T16:42:49.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo service auditd rotate 2024-04-16T16:42:49.424 INFO:teuthology.orchestra.run.smithi018.stdout:Rotating logs: 2024-04-16T16:42:49.429 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-04-16T16:42:49.489 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-04-16T16:42:49.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-04-16T16:42:49.558 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-04-16T16:42:49.560 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-16T16:42:49.560 DEBUG:teuthology.orchestra.run.smithi018:> /usr/sbin/getenforce 2024-04-16T16:42:49.616 INFO:teuthology.orchestra.run.smithi018.stdout:Permissive 2024-04-16T16:42:49.616 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-04-16T16:42:49.649 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-04-16T16:42:49.650 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-04-16T16:42:49.680 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-04-16T16:42:49.680 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi018.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi043.front.sepia.ceph.com': 'permissive'} 2024-04-16T16:42:49.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T16:42:49.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:42:49.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T16:42:49.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:42:49.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T16:42:49.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:42:49.833 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-16T16:42:49.833 DEBUG:teuthology.orchestra.run.smithi018:> sudo /usr/sbin/setenforce permissive 2024-04-16T16:42:49.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-04-16T16:42:49.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-04-16T16:42:50.016 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-16T16:42:50.463 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-16T16:42:50.468 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-16T16:42:50.688 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-16T16:42:50.792 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-16T16:42:50.793 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi018.front.sepia.ceph.com,smithi022.front.sepia.ceph.com,smithi043.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-16T16:48:59.618 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi018.front.sepia.ceph.com'), Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi043.front.sepia.ceph.com')] 2024-04-16T16:48:59.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-16T16:48:59.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:48:59.770 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-04-16T16:48:59.932 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-16T16:48:59.932 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-16T16:48:59.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:49:00.036 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-04-16T16:49:00.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-16T16:49:00.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-16T16:49:00.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:49:00.216 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-04-16T16:49:00.287 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-16T16:49:00.287 INFO:teuthology.run_tasks:Running task clock... 2024-04-16T16:49:00.301 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-16T16:49:00.301 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T16:49:00.301 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T16:49:00.309 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T16:49:00.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T16:49:00.314 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T16:49:00.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T16:49:00.349 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T16:49:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T16:49:00.370 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T16:49:00.385 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T16:49:00.408 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-04-16T16:49:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T16:49:00.422 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-04-16T16:49:00.426 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-04-16T16:49:00.440 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-04-16T16:49:00.447 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T16:49:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T16:49:00.467 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T16:49:00.469 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T16:49:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T16:49:00.518 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-04-16T16:49:00.519 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T16:49:00.519 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-04-16T16:49:00.519 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.519 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.519 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.519 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-16T16:49:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T16:49:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-16T16:49:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: ntpd: command not found 2024-04-16T16:49:00.609 INFO:teuthology.orchestra.run.smithi018.stdout:506 Cannot talk to daemon 2024-04-16T16:49:00.657 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T16:49:00.706 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T16:49:00.859 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-04-16T16:49:00.868 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T16:49:00.868 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-04-16T16:49:00.868 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.868 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.868 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.868 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:00.870 INFO:teuthology.run_tasks:Running task install... 2024-04-16T16:49:00.878 DEBUG:teuthology.task.install:project ceph 2024-04-16T16:49:00.879 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45'}} 2024-04-16T16:49:00.879 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45'} 2024-04-16T16:49:00.879 INFO:teuthology.task.install:Using flavor: default 2024-04-16T16:49:00.885 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-16T16:49:00.885 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-16T16:49:00.886 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'tag': None, 'wait_for_package': False} 2024-04-16T16:49:00.886 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:49:00.889 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'tag': None, 'wait_for_package': False} 2024-04-16T16:49:00.889 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:49:00.890 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'tag': None, 'wait_for_package': False} 2024-04-16T16:49:00.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:49:01.046 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/ 2024-04-16T16:49:01.046 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.g9b42bfbe 2024-04-16T16:49:01.050 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/ 2024-04-16T16:49:01.050 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.g9b42bfbe 2024-04-16T16:49:01.056 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/ 2024-04-16T16:49:01.056 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.g9b42bfbe 2024-04-16T16:49:01.163 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T16:49:01.163 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:49:01.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T16:49:01.166 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T16:49:01.166 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:49:01.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T16:49:01.201 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T16:49:01.201 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b42bfbeda298874861ece590aa565cf64ac8c45/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T16:49:01.274 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T16:49:01.274 DEBUG:teuthology.orchestra.run.smithi018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b42bfbeda298874861ece590aa565cf64ac8c45/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T16:49:01.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T16:49:01.336 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T16:49:01.336 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:49:01.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T16:49:01.366 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T16:49:01.369 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T16:49:01.369 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b42bfbeda298874861ece590aa565cf64ac8c45/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T16:49:01.396 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-04-16T16:49:01.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-04-16T16:49:01.419 DEBUG:teuthology.orchestra.run.smithi018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T16:49:01.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T16:49:01.530 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T16:49:01.560 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-04-16T16:49:01.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-04-16T16:49:01.562 DEBUG:teuthology.orchestra.run.smithi018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T16:49:01.647 INFO:teuthology.orchestra.run.smithi043.stdout:74 files removed 2024-04-16T16:49:01.661 INFO:teuthology.orchestra.run.smithi018.stdout:check_obsoletes = 1 2024-04-16T16:49:01.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum clean all 2024-04-16T16:49:01.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T16:49:01.809 INFO:teuthology.orchestra.run.smithi022.stdout:74 files removed 2024-04-16T16:49:01.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T16:49:02.024 INFO:teuthology.orchestra.run.smithi018.stdout:74 files removed 2024-04-16T16:49:02.065 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T16:49:02.528 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 154 kB/s | 83 kB 00:00 2024-04-16T16:49:02.563 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-04-16T16:49:02.928 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-16T16:49:02.947 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-16T16:49:02.998 INFO:teuthology.orchestra.run.smithi018.stdout:ceph packages for x86_64 206 kB/s | 83 kB 00:00 2024-04-16T16:49:03.295 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-16T16:49:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-16T16:49:03.532 INFO:teuthology.orchestra.run.smithi018.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-16T16:49:04.124 INFO:teuthology.orchestra.run.smithi018.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-04-16T16:49:04.175 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - AppStream 8.5 MB/s | 6.7 MB 00:00 2024-04-16T16:49:04.206 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 7.6 MB/s | 6.7 MB 00:00 2024-04-16T16:49:05.161 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - AppStream 6.9 MB/s | 6.7 MB 00:00 2024-04-16T16:49:05.967 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - BaseOS 3.2 MB/s | 2.0 MB 00:00 2024-04-16T16:49:06.023 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-04-16T16:49:06.956 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-16T16:49:07.253 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - BaseOS 2.7 MB/s | 2.0 MB 00:00 2024-04-16T16:49:08.265 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-16T16:49:08.416 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.0 MB 00:02 2024-04-16T16:49:09.023 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-16T16:49:10.350 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-16T16:49:12.733 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-04-16T16:49:13.632 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-04-16T16:49:14.626 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-16T16:49:15.208 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-16T16:49:16.202 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-16T16:49:17.428 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-16T16:49:17.468 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 21 MB 00:02 2024-04-16T16:49:17.583 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-04-16T16:49:20.400 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-16T16:49:22.443 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-16T16:49:22.734 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-16T16:49:22.803 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-16T16:49:25.531 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:49:25.532 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:49:25.544 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T16:49:25.545 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T16:49:25.546 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T16:49:25.547 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T16:49:25.549 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T16:49:25.552 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T16:49:25.553 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T16:49:25.616 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:49:25.617 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:49:25.630 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T16:49:25.631 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T16:49:25.632 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T16:49:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T16:49:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T16:49:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T16:49:25.638 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-16T16:49:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T16:49:25.656 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-04-16T16:49:25.656 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-04-16T16:49:25.656 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-04-16T16:49:25.656 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-04-16T16:49:25.656 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T16:49:25.656 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 6.6 k 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 890 k 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 145 k 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 1.5 M 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 142 k 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 3.5 M 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 7.4 M 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 49 k 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 11 M 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 49 M 2024-04-16T16:49:25.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 273 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 762 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 225 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 27 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 18 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 32 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 747 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 127 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T16:49:25.658 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 162 k 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 320 k 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 298 k 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 99 k 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 85 k 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:49:25.659 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 170 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T16:49:25.660 INFO:teuthology.orchestra.run.smithi043.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.4 M 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T16:49:25.661 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 21 M 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 34 k 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 2.4 M 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 249 k 2024-04-16T16:49:25.662 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 4.7 M 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 17 M 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 16 k 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 25 k 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T16:49:25.663 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 163 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 501 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T16:49:25.664 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T16:49:25.665 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 45 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 135 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T16:49:25.666 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T16:49:25.667 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T16:49:25.668 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T16:49:25.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T16:49:25.670 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-04-16T16:49:25.671 INFO:teuthology.orchestra.run.smithi043.stdout:Install 150 Packages 2024-04-16T16:49:25.672 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 24 Packages 2024-04-16T16:49:25.672 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:49:25.672 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 229 M 2024-04-16T16:49:25.672 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-16T16:49:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 6.6 k 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 890 k 2024-04-16T16:49:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 145 k 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 1.5 M 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 142 k 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 3.5 M 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 7.4 M 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 49 k 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 11 M 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 49 M 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 273 k 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 762 k 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 225 k 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 27 k 2024-04-16T16:49:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 18 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 32 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 747 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 127 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T16:49:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 162 k 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 320 k 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 298 k 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 99 k 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 85 k 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 170 k 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T16:49:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T16:49:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.4 M 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-04-16T16:49:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 21 M 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 34 k 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 2.4 M 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 249 k 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 4.7 M 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 17 M 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 16 k 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 25 k 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T16:49:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 163 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T16:49:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 501 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T16:49:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 45 k 2024-04-16T16:49:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 135 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T16:49:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T16:49:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T16:49:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T16:49:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T16:49:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Install 150 Packages 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 24 Packages 2024-04-16T16:49:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:49:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 229 M 2024-04-16T16:49:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-04-16T16:49:26.596 INFO:teuthology.orchestra.run.smithi022.stdout:(1/174): ceph-19.0.0-3101.g9b42bfbe.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-16T16:49:26.927 INFO:teuthology.orchestra.run.smithi043.stdout:(1/174): ceph-19.0.0-3101.g9b42bfbe.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-16T16:49:27.252 INFO:teuthology.orchestra.run.smithi043.stdout:(2/174): ceph-fuse-19.0.0-3101.g9b42bfbe.el9.x8 2.7 MB/s | 890 kB 00:00 2024-04-16T16:49:27.254 INFO:teuthology.orchestra.run.smithi022.stdout:(2/174): ceph-fuse-19.0.0-3101.g9b42bfbe.el9.x8 1.3 MB/s | 890 kB 00:00 2024-04-16T16:49:27.344 INFO:teuthology.orchestra.run.smithi043.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 1.5 MB/s | 145 kB 00:00 2024-04-16T16:49:27.570 INFO:teuthology.orchestra.run.smithi043.stdout:(4/174): ceph-mds-19.0.0-3101.g9b42bfbe.el9.x86 10 MB/s | 2.4 MB 00:00 2024-04-16T16:49:27.770 INFO:teuthology.orchestra.run.smithi043.stdout:(5/174): ceph-mgr-19.0.0-3101.g9b42bfbe.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-04-16T16:49:27.878 INFO:teuthology.orchestra.run.smithi043.stdout:(6/174): ceph-base-19.0.0-3101.g9b42bfbe.el9.x8 4.1 MB/s | 5.4 MB 00:01 2024-04-16T16:49:28.196 INFO:teuthology.orchestra.run.smithi043.stdout:(7/174): ceph-mon-19.0.0-3101.g9b42bfbe.el9.x86 11 MB/s | 4.7 MB 00:00 2024-04-16T16:49:28.696 INFO:teuthology.orchestra.run.smithi022.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 100 kB/s | 145 kB 00:01 2024-04-16T16:49:28.784 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 9.5 kB/s | 1.7 kB 00:00 2024-04-16T16:49:29.072 INFO:teuthology.orchestra.run.smithi022.stdout:(4/174): ceph-common-19.0.0-3101.g9b42bfbe.el9. 7.8 MB/s | 21 MB 00:02 2024-04-16T16:49:29.263 INFO:teuthology.orchestra.run.smithi022.stdout:(5/174): ceph-mgr-19.0.0-3101.g9b42bfbe.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-16T16:49:29.762 INFO:teuthology.orchestra.run.smithi043.stdout:(8/174): ceph-radosgw-19.0.0-3101.g9b42bfbe.el9 6.8 MB/s | 11 MB 00:01 2024-04-16T16:49:29.771 INFO:teuthology.orchestra.run.smithi022.stdout:(6/174): ceph-mon-19.0.0-3101.g9b42bfbe.el9.x86 9.2 MB/s | 4.7 MB 00:00 2024-04-16T16:49:30.079 INFO:teuthology.orchestra.run.smithi043.stdout:(9/174): ceph-osd-19.0.0-3101.g9b42bfbe.el9.x86 7.6 MB/s | 17 MB 00:02 2024-04-16T16:49:30.096 INFO:teuthology.orchestra.run.smithi043.stdout:(10/174): ceph-selinux-19.0.0-3101.g9b42bfbe.el 75 kB/s | 25 kB 00:00 2024-04-16T16:49:30.371 INFO:teuthology.orchestra.run.smithi043.stdout:(11/174): cephfs-mirror-19.0.0-3101.g9b42bfbe.e 817 kB/s | 225 kB 00:00 2024-04-16T16:49:30.480 INFO:teuthology.orchestra.run.smithi043.stdout:(12/174): cephfs-shell-19.0.0-3101.g9b42bfbe.el 250 kB/s | 27 kB 00:00 2024-04-16T16:49:30.663 INFO:teuthology.orchestra.run.smithi022.stdout:(7/174): ceph-base-19.0.0-3101.g9b42bfbe.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-16T16:49:30.688 INFO:teuthology.orchestra.run.smithi043.stdout:(13/174): libcephfs-devel-19.0.0-3101.g9b42bfbe 152 kB/s | 32 kB 00:00 2024-04-16T16:49:30.880 INFO:teuthology.orchestra.run.smithi043.stdout:(14/174): libcephfs2-19.0.0-3101.g9b42bfbe.el9. 3.8 MB/s | 747 kB 00:00 2024-04-16T16:49:30.980 INFO:teuthology.orchestra.run.smithi022.stdout:(8/174): ceph-osd-19.0.0-3101.g9b42bfbe.el9.x86 14 MB/s | 17 MB 00:01 2024-04-16T16:49:30.989 INFO:teuthology.orchestra.run.smithi043.stdout:(15/174): libcephsqlite-19.0.0-3101.g9b42bfbe.e 1.5 MB/s | 163 kB 00:00 2024-04-16T16:49:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:(9/174): ceph-selinux-19.0.0-3101.g9b42bfbe.el9 506 kB/s | 25 kB 00:00 2024-04-16T16:49:31.097 INFO:teuthology.orchestra.run.smithi043.stdout:(16/174): librados-devel-19.0.0-3101.g9b42bfbe. 1.1 MB/s | 127 kB 00:00 2024-04-16T16:49:31.234 INFO:teuthology.orchestra.run.smithi043.stdout:(17/174): libradosstriper1-19.0.0-3101.g9b42bfb 3.6 MB/s | 501 kB 00:00 2024-04-16T16:49:31.331 INFO:teuthology.orchestra.run.smithi022.stdout:(10/174): ceph-mds-19.0.0-3101.g9b42bfbe.el9.x8 917 kB/s | 2.4 MB 00:02 2024-04-16T16:49:31.690 INFO:teuthology.orchestra.run.smithi022.stdout:(11/174): cephfs-mirror-19.0.0-3101.g9b42bfbe.e 627 kB/s | 225 kB 00:00 2024-04-16T16:49:31.806 INFO:teuthology.orchestra.run.smithi022.stdout:(12/174): cephfs-shell-19.0.0-3101.g9b42bfbe.el 232 kB/s | 27 kB 00:00 2024-04-16T16:49:31.914 INFO:teuthology.orchestra.run.smithi018.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:49:31.916 INFO:teuthology.orchestra.run.smithi018.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:49:31.932 INFO:teuthology.orchestra.run.smithi022.stdout:(13/174): libcephfs-devel-19.0.0-3101.g9b42bfbe 254 kB/s | 32 kB 00:00 2024-04-16T16:49:31.940 INFO:teuthology.orchestra.run.smithi018.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T16:49:31.941 INFO:teuthology.orchestra.run.smithi018.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T16:49:31.943 INFO:teuthology.orchestra.run.smithi018.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T16:49:31.944 INFO:teuthology.orchestra.run.smithi018.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T16:49:31.947 INFO:teuthology.orchestra.run.smithi018.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T16:49:31.951 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T16:49:31.952 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T16:49:32.046 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-16T16:49:32.065 INFO:teuthology.orchestra.run.smithi018.stdout:============================================================================================ 2024-04-16T16:49:32.065 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-16T16:49:32.065 INFO:teuthology.orchestra.run.smithi018.stdout:============================================================================================ 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 6.6 k 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 890 k 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 145 k 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 1.5 M 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 142 k 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 3.5 M 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 7.4 M 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 49 k 2024-04-16T16:49:32.066 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 11 M 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 49 M 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 273 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 762 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 225 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-shell x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 27 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 18 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 32 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 747 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 127 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T16:49:32.067 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 162 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 320 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 298 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 99 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 85 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 170 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2024-04-16T16:49:32.068 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T16:49:32.069 INFO:teuthology.orchestra.run.smithi018.stdout: librados2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.4 M 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T16:49:32.070 INFO:teuthology.orchestra.run.smithi018.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 21 M 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 34 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 2.4 M 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 249 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 4.7 M 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 17 M 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 16 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 25 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T16:49:32.071 INFO:teuthology.orchestra.run.smithi018.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 163 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 501 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T16:49:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T16:49:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 45 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 135 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T16:49:32.074 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T16:49:32.075 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T16:49:32.076 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T16:49:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout:Installing weak dependencies: 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout:============================================================================================ 2024-04-16T16:49:32.078 INFO:teuthology.orchestra.run.smithi018.stdout:Install 150 Packages 2024-04-16T16:49:32.079 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 24 Packages 2024-04-16T16:49:32.079 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:49:32.079 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 229 M 2024-04-16T16:49:32.079 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-16T16:49:32.160 INFO:teuthology.orchestra.run.smithi043.stdout:(18/174): librgw2-19.0.0-3101.g9b42bfbe.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-04-16T16:49:32.310 INFO:teuthology.orchestra.run.smithi043.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.g9b 301 kB/s | 45 kB 00:00 2024-04-16T16:49:32.419 INFO:teuthology.orchestra.run.smithi043.stdout:(20/174): python3-ceph-common-19.0.0-3101.g9b42 1.2 MB/s | 135 kB 00:00 2024-04-16T16:49:32.536 INFO:teuthology.orchestra.run.smithi043.stdout:(21/174): python3-cephfs-19.0.0-3101.g9b42bfbe. 1.4 MB/s | 162 kB 00:00 2024-04-16T16:49:32.653 INFO:teuthology.orchestra.run.smithi043.stdout:(22/174): python3-rados-19.0.0-3101.g9b42bfbe.e 2.7 MB/s | 320 kB 00:00 2024-04-16T16:49:32.761 INFO:teuthology.orchestra.run.smithi043.stdout:(23/174): python3-rbd-19.0.0-3101.g9b42bfbe.el9 2.7 MB/s | 298 kB 00:00 2024-04-16T16:49:32.798 INFO:teuthology.orchestra.run.smithi022.stdout:(14/174): libcephfs2-19.0.0-3101.g9b42bfbe.el9. 862 kB/s | 747 kB 00:00 2024-04-16T16:49:32.879 INFO:teuthology.orchestra.run.smithi043.stdout:(24/174): python3-rgw-19.0.0-3101.g9b42bfbe.el9 849 kB/s | 99 kB 00:00 2024-04-16T16:49:32.979 INFO:teuthology.orchestra.run.smithi043.stdout:(25/174): rbd-fuse-19.0.0-3101.g9b42bfbe.el9.x8 855 kB/s | 85 kB 00:00 2024-04-16T16:49:33.032 INFO:teuthology.orchestra.run.smithi022.stdout:(15/174): libcephsqlite-19.0.0-3101.g9b42bfbe.e 701 kB/s | 163 kB 00:00 2024-04-16T16:49:33.232 INFO:teuthology.orchestra.run.smithi022.stdout:(16/174): librados-devel-19.0.0-3101.g9b42bfbe. 635 kB/s | 127 kB 00:00 2024-04-16T16:49:33.494 INFO:teuthology.orchestra.run.smithi018.stdout:(1/174): ceph-19.0.0-3101.g9b42bfbe.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-16T16:49:33.529 INFO:teuthology.orchestra.run.smithi043.stdout:(26/174): rbd-mirror-19.0.0-3101.g9b42bfbe.el9. 5.6 MB/s | 3.1 MB 00:00 2024-04-16T16:49:33.671 INFO:teuthology.orchestra.run.smithi043.stdout:(27/174): rbd-nbd-19.0.0-3101.g9b42bfbe.el9.x86 1.2 MB/s | 170 kB 00:00 2024-04-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi022.stdout:(17/174): libradosstriper1-19.0.0-3101.g9b42bfb 987 kB/s | 501 kB 00:00 2024-04-16T16:49:33.771 INFO:teuthology.orchestra.run.smithi043.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 342 kB/s | 34 kB 00:00 2024-04-16T16:49:34.469 INFO:teuthology.orchestra.run.smithi018.stdout:(2/174): ceph-fuse-19.0.0-3101.g9b42bfbe.el9.x8 913 kB/s | 890 kB 00:00 2024-04-16T16:49:34.638 INFO:teuthology.orchestra.run.smithi043.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.g9b42bfb 164 kB/s | 142 kB 00:00 2024-04-16T16:49:34.661 INFO:teuthology.orchestra.run.smithi018.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 759 kB/s | 145 kB 00:00 2024-04-16T16:49:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:(4/174): ceph-base-19.0.0-3101.g9b42bfbe.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-04-16T16:49:35.470 INFO:teuthology.orchestra.run.smithi018.stdout:(5/174): ceph-mgr-19.0.0-3101.g9b42bfbe.el9.x86 3.0 MB/s | 1.5 MB 00:00 2024-04-16T16:49:35.789 INFO:teuthology.orchestra.run.smithi043.stdout:(30/174): ceph-test-19.0.0-3101.g9b42bfbe.el9.x 8.6 MB/s | 49 MB 00:05 2024-04-16T16:49:35.839 INFO:teuthology.orchestra.run.smithi043.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.g9b42b 2.9 MB/s | 3.5 MB 00:01 2024-04-16T16:49:36.008 INFO:teuthology.orchestra.run.smithi022.stdout:(18/174): ceph-radosgw-19.0.0-3101.g9b42bfbe.el 2.0 MB/s | 11 MB 00:05 2024-04-16T16:49:36.147 INFO:teuthology.orchestra.run.smithi043.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.g9b 807 kB/s | 249 kB 00:00 2024-04-16T16:49:36.206 INFO:teuthology.orchestra.run.smithi043.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.g9b42bfbe.e 849 kB/s | 49 kB 00:00 2024-04-16T16:49:36.265 INFO:teuthology.orchestra.run.smithi043.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.g9 281 kB/s | 16 kB 00:00 2024-04-16T16:49:36.348 INFO:teuthology.orchestra.run.smithi043.stdout:(35/174): ceph-volume-19.0.0-3101.g9b42bfbe.el9 3.2 MB/s | 273 kB 00:00 2024-04-16T16:49:36.440 INFO:teuthology.orchestra.run.smithi043.stdout:(36/174): cephadm-19.0.0-3101.g9b42bfbe.el9.noa 8.1 MB/s | 762 kB 00:00 2024-04-16T16:49:36.441 INFO:teuthology.orchestra.run.smithi022.stdout:(19/174): ceph-test-19.0.0-3101.g9b42bfbe.el9.x 9.1 MB/s | 49 MB 00:05 2024-04-16T16:49:36.445 INFO:teuthology.orchestra.run.smithi018.stdout:(6/174): ceph-mon-19.0.0-3101.g9b42bfbe.el9.x86 4.8 MB/s | 4.7 MB 00:00 2024-04-16T16:49:36.466 INFO:teuthology.orchestra.run.smithi022.stdout:(20/174): python3-ceph-argparse-19.0.0-3101.g9b 98 kB/s | 45 kB 00:00 2024-04-16T16:49:36.490 INFO:teuthology.orchestra.run.smithi043.stdout:(37/174): cephfs-top-19.0.0-3101.g9b42bfbe.el9. 364 kB/s | 18 kB 00:00 2024-04-16T16:49:36.500 INFO:teuthology.orchestra.run.smithi022.stdout:(21/174): python3-ceph-common-19.0.0-3101.g9b42 2.3 MB/s | 135 kB 00:00 2024-04-16T16:49:36.512 INFO:teuthology.orchestra.run.smithi018.stdout:(7/174): ceph-mds-19.0.0-3101.g9b42bfbe.el9.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-16T16:49:36.550 INFO:teuthology.orchestra.run.smithi022.stdout:(22/174): python3-rados-19.0.0-3101.g9b42bfbe.e 6.3 MB/s | 320 kB 00:00 2024-04-16T16:49:36.617 INFO:teuthology.orchestra.run.smithi022.stdout:(23/174): python3-rbd-19.0.0-3101.g9b42bfbe.el9 4.4 MB/s | 298 kB 00:00 2024-04-16T16:49:36.642 INFO:teuthology.orchestra.run.smithi022.stdout:(24/174): python3-cephfs-19.0.0-3101.g9b42bfbe. 920 kB/s | 162 kB 00:00 2024-04-16T16:49:36.668 INFO:teuthology.orchestra.run.smithi022.stdout:(25/174): python3-rgw-19.0.0-3101.g9b42bfbe.el9 1.9 MB/s | 99 kB 00:00 2024-04-16T16:49:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:(26/174): rbd-fuse-19.0.0-3101.g9b42bfbe.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-16T16:49:36.799 INFO:teuthology.orchestra.run.smithi043.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 662 kB 00:00 2024-04-16T16:49:36.918 INFO:teuthology.orchestra.run.smithi022.stdout:(27/174): rbd-nbd-19.0.0-3101.g9b42bfbe.el9.x86 891 kB/s | 170 kB 00:00 2024-04-16T16:49:36.974 INFO:teuthology.orchestra.run.smithi043.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 6.3 MB/s | 7.4 MB 00:01 2024-04-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 920 kB 00:00 2024-04-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi022.stdout:(28/174): rbd-mirror-19.0.0-3101.g9b42bfbe.el9. 8.8 MB/s | 3.1 MB 00:00 2024-04-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi043.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-04-16T16:49:37.043 INFO:teuthology.orchestra.run.smithi022.stdout:(29/174): ceph-grafana-dashboards-19.0.0-3101.g 273 kB/s | 34 kB 00:00 2024-04-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi043.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi022.stdout:(30/174): ceph-mgr-cephadm-19.0.0-3101.g9b42bfb 2.8 MB/s | 142 kB 00:00 2024-04-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi043.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 640 kB/s | 32 kB 00:00 2024-04-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi043.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi043.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 114 kB/s | 17 kB 00:00 2024-04-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi043.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 240 kB/s | 10 kB 00:00 2024-04-16T16:49:37.293 INFO:teuthology.orchestra.run.smithi043.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi043.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-04-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi043.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi043.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2024-04-16T16:49:37.394 INFO:teuthology.orchestra.run.smithi043.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 46 kB 00:00 2024-04-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi043.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-16T16:49:37.445 INFO:teuthology.orchestra.run.smithi043.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi043.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-04-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi043.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 824 kB/s | 34 kB 00:00 2024-04-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi022.stdout:(31/174): librgw2-19.0.0-3101.g9b42bfbe.el9.x86 1.4 MB/s | 5.4 MB 00:03 2024-04-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi043.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-16T16:49:37.688 INFO:teuthology.orchestra.run.smithi043.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.5 MB 00:00 2024-04-16T16:49:37.721 INFO:teuthology.orchestra.run.smithi043.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 817 kB 00:00 2024-04-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi043.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-04-16T16:49:37.794 INFO:teuthology.orchestra.run.smithi022.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi022.stdout:(33/174): ceph-mgr-modules-core-19.0.0-3101.g9b 905 kB/s | 249 kB 00:00 2024-04-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi022.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.g9 396 kB/s | 16 kB 00:00 2024-04-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi022.stdout:(35/174): ceph-mgr-rook-19.0.0-3101.g9b42bfbe.e 454 kB/s | 49 kB 00:00 2024-04-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi022.stdout:(36/174): ceph-volume-19.0.0-3101.g9b42bfbe.el9 4.0 MB/s | 273 kB 00:00 2024-04-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi043.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.7 MB 00:00 2024-04-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi022.stdout:(37/174): cephfs-top-19.0.0-3101.g9b42bfbe.el9. 437 kB/s | 18 kB 00:00 2024-04-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi043.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-04-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi043.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-16T16:49:38.206 INFO:teuthology.orchestra.run.smithi043.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-04-16T16:49:38.231 INFO:teuthology.orchestra.run.smithi043.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 112 kB/s | 24 kB 00:00 2024-04-16T16:49:38.257 INFO:teuthology.orchestra.run.smithi043.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-04-16T16:49:38.262 INFO:teuthology.orchestra.run.smithi022.stdout:(38/174): cephadm-19.0.0-3101.g9b42bfbe.el9.noa 2.1 MB/s | 762 kB 00:00 2024-04-16T16:49:38.282 INFO:teuthology.orchestra.run.smithi043.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi022.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2024-04-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi043.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 620 kB/s | 32 kB 00:00 2024-04-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi022.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 7.2 MB/s | 920 kB 00:00 2024-04-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi022.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 847 kB/s | 106 kB 00:00 2024-04-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi043.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-04-16T16:49:38.480 INFO:teuthology.orchestra.run.smithi022.stdout:(42/174): ceph-mgr-dashboard-19.0.0-3101.g9b42b 2.5 MB/s | 3.5 MB 00:01 2024-04-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi043.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-04-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi022.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 307 kB 00:00 2024-04-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 251 kB/s | 32 kB 00:00 2024-04-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi043.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 69 kB 00:00 2024-04-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi043.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-04-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi043.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-04-16T16:49:38.626 INFO:teuthology.orchestra.run.smithi043.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-16T16:49:38.660 INFO:teuthology.orchestra.run.smithi043.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 211 kB 00:00 2024-04-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi043.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 942 kB/s | 47 kB 00:00 2024-04-16T16:49:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi043.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-04-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi022.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 61 kB/s | 10 kB 00:00 2024-04-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:(47/174): flexiblas-openblas-openmp-3.0.4-8.el9 86 kB/s | 17 kB 00:00 2024-04-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi022.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.8 MB/s | 159 kB 00:00 2024-04-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi022.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 525 kB/s | 44 kB 00:00 2024-04-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi022.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-04-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi022.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 578 kB 00:00 2024-04-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi022.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 370 kB/s | 46 kB 00:00 2024-04-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi043.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 172 kB/s | 39 kB 00:00 2024-04-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi022.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-04-16T16:49:38.976 INFO:teuthology.orchestra.run.smithi022.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.7 MB/s | 217 kB 00:00 2024-04-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi022.stdout:(55/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-04-16T16:49:39.026 INFO:teuthology.orchestra.run.smithi022.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 409 kB/s | 34 kB 00:00 2024-04-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi018.stdout:(8/174): ceph-common-19.0.0-3101.g9b42bfbe.el9. 3.5 MB/s | 21 MB 00:06 2024-04-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi043.stdout:(79/174): ceph-common-19.0.0-3101.g9b42bfbe.el9 1.7 MB/s | 21 MB 00:12 2024-04-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi018.stdout:(9/174): ceph-selinux-19.0.0-3101.g9b42bfbe.el9 306 kB/s | 25 kB 00:00 2024-04-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi022.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi022.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 817 kB 00:00 2024-04-16T16:49:39.278 INFO:teuthology.orchestra.run.smithi022.stdout:(59/174): postgresql-13.14-1.el9.x86_64.rpm 6.1 MB/s | 1.5 MB 00:00 2024-04-16T16:49:39.345 INFO:teuthology.orchestra.run.smithi043.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-16T16:49:39.361 INFO:teuthology.orchestra.run.smithi043.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 93 kB/s | 41 kB 00:00 2024-04-16T16:49:39.387 INFO:teuthology.orchestra.run.smithi043.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-04-16T16:49:39.403 INFO:teuthology.orchestra.run.smithi043.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 848 kB/s | 35 kB 00:00 2024-04-16T16:49:39.428 INFO:teuthology.orchestra.run.smithi022.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 609 kB/s | 132 kB 00:00 2024-04-16T16:49:39.443 INFO:teuthology.orchestra.run.smithi043.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 572 kB/s | 46 kB 00:00 2024-04-16T16:49:39.462 INFO:teuthology.orchestra.run.smithi018.stdout:(10/174): ceph-osd-19.0.0-3101.g9b42bfbe.el9.x8 5.6 MB/s | 17 MB 00:03 2024-04-16T16:49:39.471 INFO:teuthology.orchestra.run.smithi043.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-04-16T16:49:39.496 INFO:teuthology.orchestra.run.smithi043.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-16T16:49:39.514 INFO:teuthology.orchestra.run.smithi043.stdout:(87/174): uuid-1.6.2-55.el9.x86_64.rpm 822 kB/s | 56 kB 00:00 2024-04-16T16:49:39.530 INFO:teuthology.orchestra.run.smithi043.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 751 kB/s | 44 kB 00:00 2024-04-16T16:49:39.555 INFO:teuthology.orchestra.run.smithi043.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-04-16T16:49:39.578 INFO:teuthology.orchestra.run.smithi022.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.7 MB 00:00 2024-04-16T16:49:39.580 INFO:teuthology.orchestra.run.smithi043.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-04-16T16:49:39.596 INFO:teuthology.orchestra.run.smithi018.stdout:(11/174): cephfs-mirror-19.0.0-3101.g9b42bfbe.e 1.7 MB/s | 225 kB 00:00 2024-04-16T16:49:39.603 INFO:teuthology.orchestra.run.smithi022.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 452 kB/s | 78 kB 00:00 2024-04-16T16:49:39.622 INFO:teuthology.orchestra.run.smithi043.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-04-16T16:49:39.637 INFO:teuthology.orchestra.run.smithi022.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-04-16T16:49:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-04-16T16:49:39.673 INFO:teuthology.orchestra.run.smithi043.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-04-16T16:49:39.796 INFO:teuthology.orchestra.run.smithi018.stdout:(12/174): cephfs-shell-19.0.0-3101.g9b42bfbe.el 136 kB/s | 27 kB 00:00 2024-04-16T16:49:39.829 INFO:teuthology.orchestra.run.smithi022.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-04-16T16:49:39.849 INFO:teuthology.orchestra.run.smithi043.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 830 kB/s | 187 kB 00:00 2024-04-16T16:49:39.854 INFO:teuthology.orchestra.run.smithi022.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 98 kB/s | 24 kB 00:00 2024-04-16T16:49:39.879 INFO:teuthology.orchestra.run.smithi022.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 76 kB/s | 18 kB 00:00 2024-04-16T16:49:39.888 INFO:teuthology.orchestra.run.smithi018.stdout:(13/174): libcephfs-devel-19.0.0-3101.g9b42bfbe 350 kB/s | 32 kB 00:00 2024-04-16T16:49:39.913 INFO:teuthology.orchestra.run.smithi022.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-04-16T16:49:39.938 INFO:teuthology.orchestra.run.smithi022.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 543 kB/s | 32 kB 00:00 2024-04-16T16:49:39.982 INFO:teuthology.orchestra.run.smithi043.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 607 kB/s | 187 kB 00:00 2024-04-16T16:49:40.007 INFO:teuthology.orchestra.run.smithi043.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-16T16:49:40.074 INFO:teuthology.orchestra.run.smithi043.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-04-16T16:49:40.088 INFO:teuthology.orchestra.run.smithi022.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-04-16T16:49:40.099 INFO:teuthology.orchestra.run.smithi043.stdout:(98/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-16T16:49:40.113 INFO:teuthology.orchestra.run.smithi018.stdout:(14/174): libcephfs2-19.0.0-3101.g9b42bfbe.el9. 3.3 MB/s | 747 kB 00:00 2024-04-16T16:49:40.114 INFO:teuthology.orchestra.run.smithi022.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-04-16T16:49:40.139 INFO:teuthology.orchestra.run.smithi022.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 883 kB/s | 174 kB 00:00 2024-04-16T16:49:40.150 INFO:teuthology.orchestra.run.smithi043.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 151 kB/s | 21 kB 00:00 2024-04-16T16:49:40.164 INFO:teuthology.orchestra.run.smithi022.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 924 kB/s | 69 kB 00:00 2024-04-16T16:49:40.189 INFO:teuthology.orchestra.run.smithi022.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 826 kB/s | 41 kB 00:00 2024-04-16T16:49:40.208 INFO:teuthology.orchestra.run.smithi043.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 190 kB/s | 25 kB 00:00 2024-04-16T16:49:40.214 INFO:teuthology.orchestra.run.smithi022.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-16T16:49:40.222 INFO:teuthology.orchestra.run.smithi018.stdout:(15/174): libcephsqlite-19.0.0-3101.g9b42bfbe.e 1.5 MB/s | 163 kB 00:00 2024-04-16T16:49:40.240 INFO:teuthology.orchestra.run.smithi022.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-16T16:49:40.265 INFO:teuthology.orchestra.run.smithi022.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-16T16:49:40.282 INFO:teuthology.orchestra.run.smithi022.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 711 kB/s | 47 kB 00:00 2024-04-16T16:49:40.299 INFO:teuthology.orchestra.run.smithi022.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 730 kB/s | 43 kB 00:00 2024-04-16T16:49:40.305 INFO:teuthology.orchestra.run.smithi018.stdout:(16/174): librados-devel-19.0.0-3101.g9b42bfbe. 1.5 MB/s | 127 kB 00:00 2024-04-16T16:49:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.1 MB/s | 245 kB 00:00 2024-04-16T16:49:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-16T16:49:40.466 INFO:teuthology.orchestra.run.smithi022.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 210 kB/s | 39 kB 00:00 2024-04-16T16:49:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-16T16:49:40.531 INFO:teuthology.orchestra.run.smithi018.stdout:(17/174): libradosstriper1-19.0.0-3101.g9b42bfb 2.2 MB/s | 501 kB 00:00 2024-04-16T16:49:40.535 INFO:teuthology.orchestra.run.smithi043.stdout:(104/174): python3-jinja2-2.11.3-5.el9.noarch.r 761 kB/s | 248 kB 00:00 2024-04-16T16:49:40.635 INFO:teuthology.orchestra.run.smithi043.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-04-16T16:49:40.716 INFO:teuthology.orchestra.run.smithi022.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 98 kB/s | 41 kB 00:00 2024-04-16T16:49:40.776 INFO:teuthology.orchestra.run.smithi043.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-04-16T16:49:40.818 INFO:teuthology.orchestra.run.smithi043.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 471 kB/s | 19 kB 00:00 2024-04-16T16:49:40.925 INFO:teuthology.orchestra.run.smithi022.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-16T16:49:40.950 INFO:teuthology.orchestra.run.smithi022.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 455 kB/s | 220 kB 00:00 2024-04-16T16:49:40.975 INFO:teuthology.orchestra.run.smithi022.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 135 kB/s | 35 kB 00:00 2024-04-16T16:49:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 623 kB/s | 46 kB 00:00 2024-04-16T16:49:41.026 INFO:teuthology.orchestra.run.smithi022.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-16T16:49:41.051 INFO:teuthology.orchestra.run.smithi022.stdout:(86/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-16T16:49:41.069 INFO:teuthology.orchestra.run.smithi043.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-16T16:49:41.078 INFO:teuthology.orchestra.run.smithi022.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 831 kB/s | 63 kB 00:00 2024-04-16T16:49:41.101 INFO:teuthology.orchestra.run.smithi022.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 590 kB/s | 44 kB 00:00 2024-04-16T16:49:41.110 INFO:teuthology.orchestra.run.smithi043.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 628 kB/s | 26 kB 00:00 2024-04-16T16:49:41.118 INFO:teuthology.orchestra.run.smithi022.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2024-04-16T16:49:41.143 INFO:teuthology.orchestra.run.smithi022.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-04-16T16:49:41.169 INFO:teuthology.orchestra.run.smithi043.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-04-16T16:49:41.177 INFO:teuthology.orchestra.run.smithi022.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-04-16T16:49:41.224 INFO:teuthology.orchestra.run.smithi022.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 103 kB 00:00 2024-04-16T16:49:41.244 INFO:teuthology.orchestra.run.smithi022.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-04-16T16:49:41.270 INFO:teuthology.orchestra.run.smithi022.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-04-16T16:49:41.277 INFO:teuthology.orchestra.run.smithi043.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 622 kB/s | 67 kB 00:00 2024-04-16T16:49:41.361 INFO:teuthology.orchestra.run.smithi043.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-16T16:49:41.395 INFO:teuthology.orchestra.run.smithi022.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-04-16T16:49:41.420 INFO:teuthology.orchestra.run.smithi022.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-16T16:49:41.445 INFO:teuthology.orchestra.run.smithi022.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-16T16:49:41.479 INFO:teuthology.orchestra.run.smithi022.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-16T16:49:41.529 INFO:teuthology.orchestra.run.smithi022.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-04-16T16:49:41.546 INFO:teuthology.orchestra.run.smithi022.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 254 kB/s | 25 kB 00:00 2024-04-16T16:49:41.628 INFO:teuthology.orchestra.run.smithi043.stdout:(113/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.7 MB/s | 4.1 MB 00:01 2024-04-16T16:49:41.646 INFO:teuthology.orchestra.run.smithi022.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.4 MB/s | 248 kB 00:00 2024-04-16T16:49:41.671 INFO:teuthology.orchestra.run.smithi022.stdout:(102/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.9 MB/s | 245 kB 00:00 2024-04-16T16:49:41.696 INFO:teuthology.orchestra.run.smithi022.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi018.stdout:(18/174): ceph-radosgw-19.0.0-3101.g9b42bfbe.el 2.0 MB/s | 11 MB 00:05 2024-04-16T16:49:41.772 INFO:teuthology.orchestra.run.smithi022.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 14 MB/s | 4.1 MB 00:00 2024-04-16T16:49:41.814 INFO:teuthology.orchestra.run.smithi018.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.g9b 612 kB/s | 45 kB 00:00 2024-04-16T16:49:41.880 INFO:teuthology.orchestra.run.smithi022.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 107 kB/s | 22 kB 00:00 2024-04-16T16:49:41.912 INFO:teuthology.orchestra.run.smithi043.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.5 MB/s | 838 kB 00:00 2024-04-16T16:49:41.939 INFO:teuthology.orchestra.run.smithi018.stdout:(20/174): python3-ceph-common-19.0.0-3101.g9b42 1.1 MB/s | 135 kB 00:00 2024-04-16T16:49:41.954 INFO:teuthology.orchestra.run.smithi043.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-16T16:49:41.981 INFO:teuthology.orchestra.run.smithi043.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-04-16T16:49:41.998 INFO:teuthology.orchestra.run.smithi043.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 734 kB/s | 32 kB 00:00 2024-04-16T16:49:42.015 INFO:teuthology.orchestra.run.smithi043.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 449 kB/s | 15 kB 00:00 2024-04-16T16:49:42.101 INFO:teuthology.orchestra.run.smithi018.stdout:(21/174): librgw2-19.0.0-3101.g9b42bfbe.el9.x86 3.4 MB/s | 5.4 MB 00:01 2024-04-16T16:49:42.132 INFO:teuthology.orchestra.run.smithi018.stdout:(22/174): python3-cephfs-19.0.0-3101.g9b42bfbe. 846 kB/s | 162 kB 00:00 2024-04-16T16:49:42.224 INFO:teuthology.orchestra.run.smithi043.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-04-16T16:49:42.232 INFO:teuthology.orchestra.run.smithi018.stdout:(23/174): python3-rados-19.0.0-3101.g9b42bfbe.e 2.4 MB/s | 320 kB 00:00 2024-04-16T16:49:42.299 INFO:teuthology.orchestra.run.smithi018.stdout:(24/174): python3-rgw-19.0.0-3101.g9b42bfbe.el9 1.5 MB/s | 99 kB 00:00 2024-04-16T16:49:42.339 INFO:teuthology.orchestra.run.smithi022.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 967 kB/s | 548 kB 00:00 2024-04-16T16:49:42.340 INFO:teuthology.orchestra.run.smithi043.stdout:(120/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-04-16T16:49:42.349 INFO:teuthology.orchestra.run.smithi018.stdout:(25/174): python3-rbd-19.0.0-3101.g9b42bfbe.el9 1.3 MB/s | 298 kB 00:00 2024-04-16T16:49:42.366 INFO:teuthology.orchestra.run.smithi043.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-04-16T16:49:42.372 INFO:teuthology.orchestra.run.smithi022.stdout:(107/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-04-16T16:49:42.374 INFO:teuthology.orchestra.run.smithi018.stdout:(26/174): rbd-fuse-19.0.0-3101.g9b42bfbe.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-16T16:49:42.408 INFO:teuthology.orchestra.run.smithi043.stdout:(122/174): python3-cheroot-8.6.0-4.el9.noarch.r 419 kB/s | 172 kB 00:00 2024-04-16T16:49:42.439 INFO:teuthology.orchestra.run.smithi022.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-16T16:49:42.458 INFO:teuthology.orchestra.run.smithi018.stdout:(27/174): rbd-nbd-19.0.0-3101.g9b42bfbe.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-16T16:49:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-16T16:49:42.473 INFO:teuthology.orchestra.run.smithi022.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 522 kB/s | 309 kB 00:00 2024-04-16T16:49:42.483 INFO:teuthology.orchestra.run.smithi043.stdout:(124/174): python3-colorama-0.4.4-12.el9.noarch 248 kB/s | 35 kB 00:00 2024-04-16T16:49:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 322 kB/s | 11 kB 00:00 2024-04-16T16:49:42.525 INFO:teuthology.orchestra.run.smithi018.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 520 kB/s | 34 kB 00:00 2024-04-16T16:49:42.532 INFO:teuthology.orchestra.run.smithi022.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-16T16:49:42.551 INFO:teuthology.orchestra.run.smithi043.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-16T16:49:42.574 INFO:teuthology.orchestra.run.smithi022.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-04-16T16:49:42.575 INFO:teuthology.orchestra.run.smithi043.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-16T16:49:42.592 INFO:teuthology.orchestra.run.smithi018.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.g9b42bfb 2.1 MB/s | 142 kB 00:00 2024-04-16T16:49:42.609 INFO:teuthology.orchestra.run.smithi043.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-04-16T16:49:42.632 INFO:teuthology.orchestra.run.smithi022.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-04-16T16:49:42.654 INFO:teuthology.orchestra.run.smithi043.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 248 kB/s | 19 kB 00:00 2024-04-16T16:49:42.685 INFO:teuthology.orchestra.run.smithi043.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-16T16:49:42.757 INFO:teuthology.orchestra.run.smithi022.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 824 kB/s | 151 kB 00:00 2024-04-16T16:49:42.824 INFO:teuthology.orchestra.run.smithi022.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-04-16T16:49:42.852 INFO:teuthology.orchestra.run.smithi043.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 278 kB/s | 46 kB 00:00 2024-04-16T16:49:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-16T16:49:42.924 INFO:teuthology.orchestra.run.smithi022.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-04-16T16:49:42.977 INFO:teuthology.orchestra.run.smithi043.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 992 kB/s | 58 kB 00:00 2024-04-16T16:49:43.011 INFO:teuthology.orchestra.run.smithi043.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-04-16T16:49:43.041 INFO:teuthology.orchestra.run.smithi022.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-04-16T16:49:43.136 INFO:teuthology.orchestra.run.smithi043.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-16T16:49:43.150 INFO:teuthology.orchestra.run.smithi022.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-16T16:49:43.161 INFO:teuthology.orchestra.run.smithi043.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-04-16T16:49:43.187 INFO:teuthology.orchestra.run.smithi043.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-16T16:49:43.229 INFO:teuthology.orchestra.run.smithi043.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-04-16T16:49:43.250 INFO:teuthology.orchestra.run.smithi022.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-16T16:49:43.275 INFO:teuthology.orchestra.run.smithi022.stdout:(119/174): python3-asyncssh-2.13.2-5.el9.noarch 1.0 MB/s | 548 kB 00:00 2024-04-16T16:49:43.320 INFO:teuthology.orchestra.run.smithi043.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-16T16:49:43.346 INFO:teuthology.orchestra.run.smithi043.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 194 kB/s | 31 kB 00:00 2024-04-16T16:49:43.359 INFO:teuthology.orchestra.run.smithi022.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-16T16:49:43.379 INFO:teuthology.orchestra.run.smithi043.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-16T16:49:43.404 INFO:teuthology.orchestra.run.smithi043.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-04-16T16:49:43.447 INFO:teuthology.orchestra.run.smithi043.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 763 kB/s | 51 kB 00:00 2024-04-16T16:49:43.460 INFO:teuthology.orchestra.run.smithi022.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 354 kB/s | 35 kB 00:00 2024-04-16T16:49:43.489 INFO:teuthology.orchestra.run.smithi043.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-16T16:49:43.522 INFO:teuthology.orchestra.run.smithi043.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-16T16:49:43.526 INFO:teuthology.orchestra.run.smithi018.stdout:(30/174): ceph-mgr-dashboard-19.0.0-3101.g9b42b 3.8 MB/s | 3.5 MB 00:00 2024-04-16T16:49:43.568 INFO:teuthology.orchestra.run.smithi022.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-16T16:49:43.589 INFO:teuthology.orchestra.run.smithi043.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-16T16:49:43.593 INFO:teuthology.orchestra.run.smithi022.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 954 kB/s | 303 kB 00:00 2024-04-16T16:49:43.615 INFO:teuthology.orchestra.run.smithi043.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-16T16:49:43.694 INFO:teuthology.orchestra.run.smithi022.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-16T16:49:43.723 INFO:teuthology.orchestra.run.smithi043.stdout:(148/174): python3-grpcio-1.46.7-10.el9.x86_64. 1.5 MB/s | 2.0 MB 00:01 2024-04-16T16:49:43.802 INFO:teuthology.orchestra.run.smithi022.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-16T16:49:43.840 INFO:teuthology.orchestra.run.smithi043.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 765 kB/s | 192 kB 00:00 2024-04-16T16:49:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2024-04-16T16:49:44.067 INFO:teuthology.orchestra.run.smithi018.stdout:(31/174): rbd-mirror-19.0.0-3101.g9b42bfbe.el9. 1.8 MB/s | 3.1 MB 00:01 2024-04-16T16:49:44.069 INFO:teuthology.orchestra.run.smithi022.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-04-16T16:49:44.257 INFO:teuthology.orchestra.run.smithi043.stdout:(150/174): librbd1-19.0.0-3101.g9b42bfbe.el9.x8 7.5 MB/s | 3.1 MB 00:00 2024-04-16T16:49:44.301 INFO:teuthology.orchestra.run.smithi018.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.g9b 1.0 MB/s | 249 kB 00:00 2024-04-16T16:49:44.385 INFO:teuthology.orchestra.run.smithi018.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.g9b42bfbe.e 598 kB/s | 49 kB 00:00 2024-04-16T16:49:44.468 INFO:teuthology.orchestra.run.smithi018.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.g9 199 kB/s | 16 kB 00:00 2024-04-16T16:49:44.566 INFO:teuthology.orchestra.run.smithi043.stdout:(151/174): thrift-0.15.0-2.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-04-16T16:49:44.628 INFO:teuthology.orchestra.run.smithi022.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 1.9 MB/s | 2.0 MB 00:01 2024-04-16T16:49:44.649 INFO:teuthology.orchestra.run.smithi043.stdout:(152/174): NetworkManager-1.47.4-1.el9.x86_64.r 5.9 MB/s | 2.3 MB 00:00 2024-04-16T16:49:44.653 INFO:teuthology.orchestra.run.smithi022.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 33 kB/s | 19 kB 00:00 2024-04-16T16:49:44.702 INFO:teuthology.orchestra.run.smithi018.stdout:(35/174): ceph-volume-19.0.0-3101.g9b42bfbe.el9 1.1 MB/s | 273 kB 00:00 2024-04-16T16:49:44.745 INFO:teuthology.orchestra.run.smithi022.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-04-16T16:49:44.758 INFO:teuthology.orchestra.run.smithi043.stdout:(153/174): NetworkManager-libnm-1.47.4-1.el9.x8 9.6 MB/s | 1.8 MB 00:00 2024-04-16T16:49:44.783 INFO:teuthology.orchestra.run.smithi043.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 311 kB/s | 41 kB 00:00 2024-04-16T16:49:44.862 INFO:teuthology.orchestra.run.smithi022.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-04-16T16:49:44.963 INFO:teuthology.orchestra.run.smithi022.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 790 kB/s | 79 kB 00:00 2024-04-16T16:49:44.966 INFO:teuthology.orchestra.run.smithi043.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 1.3 MB/s | 248 kB 00:00 2024-04-16T16:49:44.992 INFO:teuthology.orchestra.run.smithi043.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.5 MB/s | 321 kB 00:00 2024-04-16T16:49:45.061 INFO:teuthology.orchestra.run.smithi018.stdout:(36/174): cephadm-19.0.0-3101.g9b42bfbe.el9.noa 2.1 MB/s | 762 kB 00:00 2024-04-16T16:49:45.063 INFO:teuthology.orchestra.run.smithi022.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 578 kB/s | 58 kB 00:00 2024-04-16T16:49:45.067 INFO:teuthology.orchestra.run.smithi043.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 718 kB/s | 71 kB 00:00 2024-04-16T16:49:45.092 INFO:teuthology.orchestra.run.smithi043.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 364 kB/s | 36 kB 00:00 2024-04-16T16:49:45.119 INFO:teuthology.orchestra.run.smithi018.stdout:(37/174): cephfs-top-19.0.0-3101.g9b42bfbe.el9. 318 kB/s | 18 kB 00:00 2024-04-16T16:49:45.126 INFO:teuthology.orchestra.run.smithi043.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-04-16T16:49:45.155 INFO:teuthology.orchestra.run.smithi022.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-16T16:49:45.168 INFO:teuthology.orchestra.run.smithi043.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 555 kB/s | 23 kB 00:00 2024-04-16T16:49:45.193 INFO:teuthology.orchestra.run.smithi043.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-04-16T16:49:45.227 INFO:teuthology.orchestra.run.smithi043.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-16T16:49:45.247 INFO:teuthology.orchestra.run.smithi022.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-16T16:49:45.252 INFO:teuthology.orchestra.run.smithi043.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-04-16T16:49:45.320 INFO:teuthology.orchestra.run.smithi018.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.3 MB/s | 662 kB 00:00 2024-04-16T16:49:45.327 INFO:teuthology.orchestra.run.smithi043.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-04-16T16:49:45.330 INFO:teuthology.orchestra.run.smithi022.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-16T16:49:45.352 INFO:teuthology.orchestra.run.smithi043.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 636 kB/s | 64 kB 00:00 2024-04-16T16:49:45.378 INFO:teuthology.orchestra.run.smithi043.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-04-16T16:49:45.378 INFO:teuthology.orchestra.run.smithi018.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 920 kB 00:00 2024-04-16T16:49:45.412 INFO:teuthology.orchestra.run.smithi018.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.2 MB/s | 106 kB 00:00 2024-04-16T16:49:45.414 INFO:teuthology.orchestra.run.smithi022.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 339 kB/s | 28 kB 00:00 2024-04-16T16:49:45.454 INFO:teuthology.orchestra.run.smithi018.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 307 kB 00:00 2024-04-16T16:49:45.470 INFO:teuthology.orchestra.run.smithi043.stdout:(167/174): librados2-19.0.0-3101.g9b42bfbe.el9. 1.9 MB/s | 3.4 MB 00:01 2024-04-16T16:49:45.479 INFO:teuthology.orchestra.run.smithi018.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 32 kB 00:00 2024-04-16T16:49:45.498 INFO:teuthology.orchestra.run.smithi022.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 370 kB/s | 31 kB 00:00 2024-04-16T16:49:45.503 INFO:teuthology.orchestra.run.smithi043.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 8.1 MB/s | 1.2 MB 00:00 2024-04-16T16:49:45.531 INFO:teuthology.orchestra.run.smithi022.stdout:(139/174): python3-kubernetes-26.1.0-2.el9.noar 1.2 MB/s | 1.0 MB 00:00 2024-04-16T16:49:45.590 INFO:teuthology.orchestra.run.smithi022.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-16T16:49:45.596 INFO:teuthology.orchestra.run.smithi018.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-16T16:49:45.615 INFO:teuthology.orchestra.run.smithi022.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 706 kB/s | 59 kB 00:00 2024-04-16T16:49:45.646 INFO:teuthology.orchestra.run.smithi018.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 349 kB/s | 17 kB 00:00 2024-04-16T16:49:45.672 INFO:teuthology.orchestra.run.smithi018.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 419 kB/s | 10 kB 00:00 2024-04-16T16:49:45.674 INFO:teuthology.orchestra.run.smithi022.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-04-16T16:49:45.704 INFO:teuthology.orchestra.run.smithi043.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.8 MB/s | 2.3 MB 00:00 2024-04-16T16:49:45.707 INFO:teuthology.orchestra.run.smithi022.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-04-16T16:49:45.714 INFO:teuthology.orchestra.run.smithi018.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2024-04-16T16:49:45.729 INFO:teuthology.orchestra.run.smithi043.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.0 MB/s | 465 kB 00:00 2024-04-16T16:49:45.747 INFO:teuthology.orchestra.run.smithi018.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-16T16:49:45.766 INFO:teuthology.orchestra.run.smithi022.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-16T16:49:45.779 INFO:teuthology.orchestra.run.smithi043.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.4 MB/s | 2.2 MB 00:00 2024-04-16T16:49:45.797 INFO:teuthology.orchestra.run.smithi018.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-16T16:49:45.799 INFO:teuthology.orchestra.run.smithi022.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 976 kB/s | 90 kB 00:00 2024-04-16T16:49:45.804 INFO:teuthology.orchestra.run.smithi043.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 310 kB/s | 31 kB 00:00 2024-04-16T16:49:45.848 INFO:teuthology.orchestra.run.smithi018.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-04-16T16:49:45.866 INFO:teuthology.orchestra.run.smithi022.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-16T16:49:45.871 INFO:teuthology.orchestra.run.smithi043.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-04-16T16:49:45.883 INFO:teuthology.orchestra.run.smithi022.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-16T16:49:45.896 INFO:teuthology.orchestra.run.smithi043.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 637 kB/s | 74 kB 00:00 2024-04-16T16:49:45.901 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:49:45.901 INFO:teuthology.orchestra.run.smithi043.stdout:Total 11 MB/s | 229 MB 00:20 2024-04-16T16:49:45.967 INFO:teuthology.orchestra.run.smithi022.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-16T16:49:45.990 INFO:teuthology.orchestra.run.smithi018.stdout:(50/174): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-04-16T16:49:46.015 INFO:teuthology.orchestra.run.smithi018.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 279 kB/s | 46 kB 00:00 2024-04-16T16:49:46.043 INFO:teuthology.orchestra.run.smithi022.stdout:(149/174): libarrow-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 4.4 MB 00:03 2024-04-16T16:49:46.048 INFO:teuthology.orchestra.run.smithi018.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-04-16T16:49:46.082 INFO:teuthology.orchestra.run.smithi018.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-04-16T16:49:46.124 INFO:teuthology.orchestra.run.smithi018.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2024-04-16T16:49:46.149 INFO:teuthology.orchestra.run.smithi018.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 534 kB/s | 34 kB 00:00 2024-04-16T16:49:46.293 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-16T16:49:46.299 INFO:teuthology.orchestra.run.smithi018.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-16T16:49:46.350 INFO:teuthology.orchestra.run.smithi018.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 8.7 MB/s | 1.5 MB 00:00 2024-04-16T16:49:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 8.1 MB/s | 817 kB 00:00 2024-04-16T16:49:46.425 INFO:teuthology.orchestra.run.smithi018.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 132 kB 00:00 2024-04-16T16:49:46.443 INFO:teuthology.orchestra.run.smithi022.stdout:(150/174): librados2-19.0.0-3101.g9b42bfbe.el9. 7.1 MB/s | 3.4 MB 00:00 2024-04-16T16:49:46.584 INFO:teuthology.orchestra.run.smithi018.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-04-16T16:49:46.589 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-16T16:49:46.590 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-16T16:49:46.618 INFO:teuthology.orchestra.run.smithi018.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-04-16T16:49:46.651 INFO:teuthology.orchestra.run.smithi022.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-04-16T16:49:46.668 INFO:teuthology.orchestra.run.smithi018.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-04-16T16:49:46.811 INFO:teuthology.orchestra.run.smithi022.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 11 MB/s | 1.8 MB 00:00 2024-04-16T16:49:46.860 INFO:teuthology.orchestra.run.smithi022.stdout:(153/174): librbd1-19.0.0-3101.g9b42bfbe.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-04-16T16:49:46.877 INFO:teuthology.orchestra.run.smithi018.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-04-16T16:49:46.885 INFO:teuthology.orchestra.run.smithi022.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 542 kB/s | 41 kB 00:00 2024-04-16T16:49:46.910 INFO:teuthology.orchestra.run.smithi022.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.8 MB/s | 248 kB 00:00 2024-04-16T16:49:46.926 INFO:teuthology.orchestra.run.smithi018.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 103 kB/s | 24 kB 00:00 2024-04-16T16:49:46.936 INFO:teuthology.orchestra.run.smithi022.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-16T16:49:46.977 INFO:teuthology.orchestra.run.smithi018.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 369 kB/s | 18 kB 00:00 2024-04-16T16:49:46.978 INFO:teuthology.orchestra.run.smithi022.stdout:(157/174): thrift-0.15.0-2.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-04-16T16:49:47.002 INFO:teuthology.orchestra.run.smithi018.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-16T16:49:47.028 INFO:teuthology.orchestra.run.smithi018.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 657 kB/s | 32 kB 00:00 2024-04-16T16:49:47.036 INFO:teuthology.orchestra.run.smithi022.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 571 kB/s | 71 kB 00:00 2024-04-16T16:49:47.062 INFO:teuthology.orchestra.run.smithi022.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 290 kB/s | 36 kB 00:00 2024-04-16T16:49:47.104 INFO:teuthology.orchestra.run.smithi022.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 256 kB 00:00 2024-04-16T16:49:47.129 INFO:teuthology.orchestra.run.smithi022.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 344 kB/s | 23 kB 00:00 2024-04-16T16:49:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:(162/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.1 MB/s | 192 kB 00:00 2024-04-16T16:49:47.178 INFO:teuthology.orchestra.run.smithi018.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-04-16T16:49:47.179 INFO:teuthology.orchestra.run.smithi022.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-04-16T16:49:47.205 INFO:teuthology.orchestra.run.smithi022.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-16T16:49:47.211 INFO:teuthology.orchestra.run.smithi018.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-04-16T16:49:47.222 INFO:teuthology.orchestra.run.smithi022.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 155 kB 00:00 2024-04-16T16:49:47.238 INFO:teuthology.orchestra.run.smithi022.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-04-16T16:49:47.245 INFO:teuthology.orchestra.run.smithi018.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.7 MB/s | 174 kB 00:00 2024-04-16T16:49:47.255 INFO:teuthology.orchestra.run.smithi022.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 568 kB/s | 29 kB 00:00 2024-04-16T16:49:47.270 INFO:teuthology.orchestra.run.smithi018.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-04-16T16:49:47.296 INFO:teuthology.orchestra.run.smithi018.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 41 kB 00:00 2024-04-16T16:49:47.314 INFO:teuthology.orchestra.run.smithi022.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-04-16T16:49:47.337 INFO:teuthology.orchestra.run.smithi018.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-04-16T16:49:47.363 INFO:teuthology.orchestra.run.smithi018.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-04-16T16:49:47.364 INFO:teuthology.orchestra.run.smithi022.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-04-16T16:49:47.388 INFO:teuthology.orchestra.run.smithi018.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.3 MB/s | 211 kB 00:00 2024-04-16T16:49:47.406 INFO:teuthology.orchestra.run.smithi022.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 15 MB/s | 2.3 MB 00:00 2024-04-16T16:49:47.413 INFO:teuthology.orchestra.run.smithi018.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 999 kB/s | 47 kB 00:00 2024-04-16T16:49:47.423 INFO:teuthology.orchestra.run.smithi022.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 533 kB/s | 31 kB 00:00 2024-04-16T16:49:47.447 INFO:teuthology.orchestra.run.smithi018.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 748 kB/s | 43 kB 00:00 2024-04-16T16:49:47.465 INFO:teuthology.orchestra.run.smithi022.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.5 MB/s | 2.2 MB 00:00 2024-04-16T16:49:47.507 INFO:teuthology.orchestra.run.smithi022.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2024-04-16T16:49:47.532 INFO:teuthology.orchestra.run.smithi022.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 684 kB/s | 74 kB 00:00 2024-04-16T16:49:47.536 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:49:47.537 INFO:teuthology.orchestra.run.smithi022.stdout:Total 11 MB/s | 229 MB 00:21 2024-04-16T16:49:47.639 INFO:teuthology.orchestra.run.smithi018.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 227 kB/s | 39 kB 00:00 2024-04-16T16:49:47.930 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-04-16T16:49:47.998 INFO:teuthology.orchestra.run.smithi018.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-04-16T16:49:48.023 INFO:teuthology.orchestra.run.smithi018.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 110 kB/s | 41 kB 00:00 2024-04-16T16:49:48.073 INFO:teuthology.orchestra.run.smithi018.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-04-16T16:49:48.098 INFO:teuthology.orchestra.run.smithi018.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 467 kB/s | 35 kB 00:00 2024-04-16T16:49:48.140 INFO:teuthology.orchestra.run.smithi018.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 706 kB/s | 46 kB 00:00 2024-04-16T16:49:48.174 INFO:teuthology.orchestra.run.smithi018.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-16T16:49:48.199 INFO:teuthology.orchestra.run.smithi018.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-04-16T16:49:48.224 INFO:teuthology.orchestra.run.smithi018.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-04-16T16:49:48.229 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-04-16T16:49:48.229 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-04-16T16:49:48.275 INFO:teuthology.orchestra.run.smithi018.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 597 kB/s | 44 kB 00:00 2024-04-16T16:49:48.300 INFO:teuthology.orchestra.run.smithi018.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-16T16:49:48.333 INFO:teuthology.orchestra.run.smithi018.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-04-16T16:49:48.392 INFO:teuthology.orchestra.run.smithi018.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-04-16T16:49:48.417 INFO:teuthology.orchestra.run.smithi018.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-04-16T16:49:48.429 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-16T16:49:48.429 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-16T16:49:48.442 INFO:teuthology.orchestra.run.smithi018.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-04-16T16:49:48.476 INFO:teuthology.orchestra.run.smithi018.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-04-16T16:49:48.610 INFO:teuthology.orchestra.run.smithi018.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-16T16:49:48.686 INFO:teuthology.orchestra.run.smithi018.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-16T16:49:48.719 INFO:teuthology.orchestra.run.smithi018.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-16T16:49:48.753 INFO:teuthology.orchestra.run.smithi018.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-16T16:49:48.778 INFO:teuthology.orchestra.run.smithi018.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 371 kB/s | 21 kB 00:00 2024-04-16T16:49:48.803 INFO:teuthology.orchestra.run.smithi018.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 516 kB/s | 25 kB 00:00 2024-04-16T16:49:48.853 INFO:teuthology.orchestra.run.smithi018.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 4.9 MB/s | 245 kB 00:00 2024-04-16T16:49:48.912 INFO:teuthology.orchestra.run.smithi018.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 4.2 MB/s | 248 kB 00:00 2024-04-16T16:49:48.954 INFO:teuthology.orchestra.run.smithi018.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-16T16:49:49.054 INFO:teuthology.orchestra.run.smithi018.stdout:(103/174): openssl-devel-3.0.7-27.el9.x86_64.rp 15 MB/s | 4.1 MB 00:00 2024-04-16T16:49:49.079 INFO:teuthology.orchestra.run.smithi018.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-04-16T16:49:49.138 INFO:teuthology.orchestra.run.smithi018.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-16T16:49:49.322 INFO:teuthology.orchestra.run.smithi018.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-16T16:49:49.355 INFO:teuthology.orchestra.run.smithi018.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-16T16:49:49.381 INFO:teuthology.orchestra.run.smithi018.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 337 kB/s | 19 kB 00:00 2024-04-16T16:49:49.431 INFO:teuthology.orchestra.run.smithi018.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 530 kB/s | 26 kB 00:00 2024-04-16T16:49:49.481 INFO:teuthology.orchestra.run.smithi018.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 995 kB/s | 49 kB 00:00 2024-04-16T16:49:49.548 INFO:teuthology.orchestra.run.smithi018.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-16T16:49:49.623 INFO:teuthology.orchestra.run.smithi018.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-16T16:49:49.824 INFO:teuthology.orchestra.run.smithi018.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-04-16T16:49:49.949 INFO:teuthology.orchestra.run.smithi018.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-16T16:49:49.999 INFO:teuthology.orchestra.run.smithi018.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 887 kB/s | 43 kB 00:00 2024-04-16T16:49:50.049 INFO:teuthology.orchestra.run.smithi018.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 656 kB/s | 32 kB 00:00 2024-04-16T16:49:50.081 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-04-16T16:49:50.081 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-04-16T16:49:50.099 INFO:teuthology.orchestra.run.smithi018.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 306 kB/s | 15 kB 00:00 2024-04-16T16:49:50.200 INFO:teuthology.orchestra.run.smithi018.stdout:(118/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-04-16T16:49:50.233 INFO:teuthology.orchestra.run.smithi018.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-16T16:49:50.284 INFO:teuthology.orchestra.run.smithi018.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-16T16:49:50.317 INFO:teuthology.orchestra.run.smithi018.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-04-16T16:49:50.351 INFO:teuthology.orchestra.run.smithi018.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 535 kB/s | 35 kB 00:00 2024-04-16T16:49:50.401 INFO:teuthology.orchestra.run.smithi018.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-16T16:49:50.452 INFO:teuthology.orchestra.run.smithi018.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.9 MB/s | 144 kB 00:00 2024-04-16T16:49:50.511 INFO:teuthology.orchestra.run.smithi018.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-04-16T16:49:50.553 INFO:teuthology.orchestra.run.smithi018.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 434 kB/s | 18 kB 00:00 2024-04-16T16:49:50.594 INFO:teuthology.orchestra.run.smithi018.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 569 kB/s | 23 kB 00:00 2024-04-16T16:49:50.636 INFO:teuthology.orchestra.run.smithi018.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 476 kB/s | 19 kB 00:00 2024-04-16T16:49:50.703 INFO:teuthology.orchestra.run.smithi018.stdout:(129/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-04-16T16:49:50.720 INFO:teuthology.orchestra.run.smithi018.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 238 kB/s | 20 kB 00:00 2024-04-16T16:49:50.762 INFO:teuthology.orchestra.run.smithi018.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-16T16:49:50.837 INFO:teuthology.orchestra.run.smithi018.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-16T16:49:50.887 INFO:teuthology.orchestra.run.smithi018.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-16T16:49:50.929 INFO:teuthology.orchestra.run.smithi018.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-04-16T16:49:50.971 INFO:teuthology.orchestra.run.smithi018.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 399 kB/s | 16 kB 00:00 2024-04-16T16:49:51.005 INFO:teuthology.orchestra.run.smithi018.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-16T16:49:51.048 INFO:teuthology.orchestra.run.smithi018.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-16T16:49:51.073 INFO:teuthology.orchestra.run.smithi018.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-04-16T16:49:51.099 INFO:teuthology.orchestra.run.smithi018.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 624 kB/s | 31 kB 00:00 2024-04-16T16:49:51.132 INFO:teuthology.orchestra.run.smithi018.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-16T16:49:51.158 INFO:teuthology.orchestra.run.smithi018.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-16T16:49:51.183 INFO:teuthology.orchestra.run.smithi018.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 721 kB/s | 36 kB 00:00 2024-04-16T16:49:51.208 INFO:teuthology.orchestra.run.smithi018.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-16T16:49:51.726 INFO:teuthology.orchestra.run.smithi018.stdout:(144/174): ceph-test-19.0.0-3101.g9b42bfbe.el9. 3.9 MB/s | 49 MB 00:12 2024-04-16T16:49:51.751 INFO:teuthology.orchestra.run.smithi018.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 405 kB/s | 230 kB 00:00 2024-04-16T16:49:51.776 INFO:teuthology.orchestra.run.smithi018.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 158 kB/s | 90 kB 00:00 2024-04-16T16:49:51.843 INFO:teuthology.orchestra.run.smithi018.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-16T16:49:51.868 INFO:teuthology.orchestra.run.smithi018.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2024-04-16T16:49:51.893 INFO:teuthology.orchestra.run.smithi018.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-16T16:49:52.100 INFO:teuthology.orchestra.run.smithi018.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.2 MB/s | 1.6 MB 00:00 2024-04-16T16:49:52.219 INFO:teuthology.orchestra.run.smithi018.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 19 MB/s | 2.3 MB 00:00 2024-04-16T16:49:52.319 INFO:teuthology.orchestra.run.smithi018.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-04-16T16:49:52.353 INFO:teuthology.orchestra.run.smithi018.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-16T16:49:52.395 INFO:teuthology.orchestra.run.smithi018.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.9 MB/s | 248 kB 00:00 2024-04-16T16:49:52.478 INFO:teuthology.orchestra.run.smithi018.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-04-16T16:49:52.553 INFO:teuthology.orchestra.run.smithi018.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 965 kB/s | 71 kB 00:00 2024-04-16T16:49:52.595 INFO:teuthology.orchestra.run.smithi018.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 902 kB/s | 36 kB 00:00 2024-04-16T16:49:52.662 INFO:teuthology.orchestra.run.smithi018.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.9 MB/s | 192 kB 00:00 2024-04-16T16:49:52.721 INFO:teuthology.orchestra.run.smithi018.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.4 MB/s | 256 kB 00:00 2024-04-16T16:49:52.754 INFO:teuthology.orchestra.run.smithi018.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 707 kB/s | 23 kB 00:00 2024-04-16T16:49:52.796 INFO:teuthology.orchestra.run.smithi018.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 108 kB 00:00 2024-04-16T16:49:52.897 INFO:teuthology.orchestra.run.smithi018.stdout:(162/174): librbd1-19.0.0-3101.g9b42bfbe.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-04-16T16:49:52.930 INFO:teuthology.orchestra.run.smithi018.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-04-16T16:49:52.964 INFO:teuthology.orchestra.run.smithi018.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.9 MB/s | 64 kB 00:00 2024-04-16T16:49:52.989 INFO:teuthology.orchestra.run.smithi018.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 136 kB 00:00 2024-04-16T16:49:53.015 INFO:teuthology.orchestra.run.smithi018.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 580 kB/s | 29 kB 00:00 2024-04-16T16:49:53.107 INFO:teuthology.orchestra.run.smithi018.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-04-16T16:49:53.182 INFO:teuthology.orchestra.run.smithi018.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 13 MB/s | 2.2 MB 00:00 2024-04-16T16:49:53.249 INFO:teuthology.orchestra.run.smithi018.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.9 MB/s | 465 kB 00:00 2024-04-16T16:49:53.332 INFO:teuthology.orchestra.run.smithi018.stdout:(170/174): librados2-19.0.0-3101.g9b42bfbe.el9. 2.3 MB/s | 3.4 MB 00:01 2024-04-16T16:49:53.377 INFO:teuthology.orchestra.run.smithi018.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 245 kB/s | 31 kB 00:00 2024-04-16T16:49:53.450 INFO:teuthology.orchestra.run.smithi018.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.7 MB/s | 2.3 MB 00:00 2024-04-16T16:49:53.483 INFO:teuthology.orchestra.run.smithi018.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 707 kB/s | 74 kB 00:00 2024-04-16T16:49:53.600 INFO:teuthology.orchestra.run.smithi018.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.0 MB/s | 1.1 MB 00:00 2024-04-16T16:49:53.609 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:49:53.609 INFO:teuthology.orchestra.run.smithi018.stdout:Total 11 MB/s | 229 MB 00:21 2024-04-16T16:49:54.788 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-16T16:49:55.184 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-16T16:49:55.184 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-16T16:49:55.406 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-16T16:49:55.525 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T16:49:55.559 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T16:49:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:49:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:49:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T16:49:55.703 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T16:49:55.736 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T16:49:55.761 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T16:49:55.804 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T16:49:56.157 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T16:49:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:49:56.327 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:49:56.446 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:49:56.475 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:49:56.516 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T16:49:56.699 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T16:49:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T16:49:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:49:56.864 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:49:56.893 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:49:56.941 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:49:56.980 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T16:49:57.003 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:49:57.019 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:49:57.212 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:49:57.250 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:49:57.274 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T16:49:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-04-16T16:49:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T16:49:57.468 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:49:57.520 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:49:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:49:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T16:49:57.654 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:49:57.660 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-16T16:49:57.662 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-16T16:49:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T16:49:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:49:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:49:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T16:49:57.882 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:49:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T16:49:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T16:49:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T16:49:57.999 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T16:49:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T16:49:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T16:49:58.126 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T16:49:58.153 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T16:49:58.189 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T16:49:58.266 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T16:49:58.325 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T16:49:58.363 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T16:49:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T16:49:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:49:58.439 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T16:49:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T16:49:58.523 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T16:49:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:49:58.555 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T16:49:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:49:58.590 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T16:49:58.648 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:49:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T16:49:58.676 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:49:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 39/198 2024-04-16T16:49:58.711 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T16:49:58.740 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 40/198 2024-04-16T16:49:58.768 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 41/198 2024-04-16T16:49:58.840 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T16:49:58.883 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T16:49:58.897 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T16:49:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T16:49:58.960 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T16:49:58.980 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:49:59.068 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:49:59.097 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T16:49:59.097 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:49:59.145 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:49:59.162 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T16:49:59.184 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T16:49:59.207 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:49:59.224 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:49:59.366 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:49:59.397 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:49:59.428 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T16:49:59.537 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T16:49:59.639 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:49:59.682 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:49:59.735 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T16:49:59.763 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:49:59.815 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T16:49:59.817 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:49:59.897 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T16:49:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:49:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:49:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T16:50:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T16:50:00.049 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:50:00.069 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 53/198 2024-04-16T16:50:00.098 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T16:50:00.159 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T16:50:00.256 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T16:50:00.297 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T16:50:00.326 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T16:50:00.336 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T16:50:00.360 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T16:50:00.436 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T16:50:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T16:50:00.490 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T16:50:00.529 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T16:50:00.550 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T16:50:00.595 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T16:50:00.602 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T16:50:00.642 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T16:50:00.686 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T16:50:00.718 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T16:50:00.746 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T16:50:00.819 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T16:50:00.851 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 39/198 2024-04-16T16:50:00.887 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 40/198 2024-04-16T16:50:00.896 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T16:50:00.906 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 41/198 2024-04-16T16:50:00.980 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T16:50:01.021 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T16:50:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T16:50:01.236 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T16:50:01.245 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T16:50:01.281 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T16:50:01.300 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T16:50:01.317 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T16:50:01.365 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T16:50:01.400 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T16:50:01.444 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T16:50:01.486 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T16:50:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T16:50:01.675 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T16:50:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T16:50:01.747 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T16:50:01.844 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T16:50:01.862 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T16:50:01.878 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T16:50:01.927 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T16:50:01.947 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T16:50:01.952 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T16:50:01.977 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T16:50:02.023 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T16:50:02.053 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T16:50:02.072 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:50:02.082 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:50:02.112 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T16:50:02.153 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T16:50:02.184 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T16:50:02.190 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 53/198 2024-04-16T16:50:02.220 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T16:50:02.307 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T16:50:02.353 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T16:50:02.410 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T16:50:02.440 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T16:50:02.482 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T16:50:02.596 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T16:50:02.665 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T16:50:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T16:50:02.709 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T16:50:02.723 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T16:50:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T16:50:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T16:50:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T16:50:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:50:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T16:50:03.019 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T16:50:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T16:50:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T16:50:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T16:50:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T16:50:03.198 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T16:50:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T16:50:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T16:50:03.252 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T16:50:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T16:50:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T16:50:03.364 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T16:50:03.401 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T16:50:03.448 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T16:50:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T16:50:03.483 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T16:50:03.519 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T16:50:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T16:50:03.561 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T16:50:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T16:50:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T16:50:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T16:50:03.704 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T16:50:03.750 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T16:50:03.780 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T16:50:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T16:50:03.822 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T16:50:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T16:50:03.919 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T16:50:03.954 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T16:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T16:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T16:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T16:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T16:50:04.243 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T16:50:04.354 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T16:50:04.405 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T16:50:04.462 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T16:50:04.526 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T16:50:04.637 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T16:50:04.676 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T16:50:04.710 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T16:50:04.721 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T16:50:04.759 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T16:50:04.766 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T16:50:04.851 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T16:50:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T16:50:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T16:50:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T16:50:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T16:50:04.996 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-16T16:50:05.010 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:50:05.041 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T16:50:05.072 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T16:50:05.072 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T16:50:05.099 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T16:50:05.100 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T16:50:05.112 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T16:50:05.152 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T16:50:05.160 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T16:50:05.181 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T16:50:05.181 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T16:50:05.262 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T16:50:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:50:05.292 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T16:50:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:50:05.325 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T16:50:05.367 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T16:50:05.401 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T16:50:05.411 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T16:50:05.495 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T16:50:05.521 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T16:50:05.580 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T16:50:05.584 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T16:50:05.625 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T16:50:05.634 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T16:50:05.712 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T16:50:05.727 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T16:50:05.823 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T16:50:05.862 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T16:50:05.875 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T16:50:05.977 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T16:50:06.161 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T16:50:06.232 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:50:06.401 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:50:06.433 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:50:06.560 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:50:06.598 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:50:06.646 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T16:50:06.664 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T16:50:06.696 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T16:50:06.721 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T16:50:06.770 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T16:50:06.863 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T16:50:06.897 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T16:50:06.974 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T16:50:07.037 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T16:50:07.065 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:50:07.082 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T16:50:07.120 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T16:50:07.200 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T16:50:07.230 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:50:07.257 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:50:07.272 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T16:50:07.351 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T16:50:07.358 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:50:07.382 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T16:50:07.410 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T16:50:07.412 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T16:50:07.440 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T16:50:07.463 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:50:07.537 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:50:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:50:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:50:07.825 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:50:07.868 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T16:50:07.878 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T16:50:07.886 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:50:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 125/198 2024-04-16T16:50:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T16:50:08.014 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T16:50:08.184 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T16:50:08.342 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:50:08.430 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:50:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T16:50:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:50:08.569 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:50:08.590 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 129/198 2024-04-16T16:50:08.627 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:50:08.678 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:50:09.015 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:50:09.162 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T16:50:09.296 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T16:50:09.321 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T16:50:09.334 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T16:50:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T16:50:09.412 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T16:50:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T16:50:09.443 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T16:50:09.474 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T16:50:09.502 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T16:50:09.546 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T16:50:09.621 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T16:50:09.677 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T16:50:09.786 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T16:50:09.832 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T16:50:09.848 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T16:50:09.856 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:50:09.915 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T16:50:09.931 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:50:09.962 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 125/198 2024-04-16T16:50:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T16:50:09.998 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T16:50:10.034 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T16:50:10.166 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T16:50:10.206 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 39/198 2024-04-16T16:50:10.264 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 40/198 2024-04-16T16:50:10.296 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 41/198 2024-04-16T16:50:10.410 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T16:50:10.475 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T16:50:10.506 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T16:50:10.552 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:50:10.598 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T16:50:10.606 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:50:10.637 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 129/198 2024-04-16T16:50:10.785 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T16:50:10.852 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T16:50:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:50:11.573 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T16:50:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:50:11.639 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T16:50:11.639 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T16:50:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:50:11.675 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T16:50:11.792 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T16:50:11.849 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:50:11.877 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:50:11.922 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T16:50:11.981 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T16:50:12.030 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 53/198 2024-04-16T16:50:12.081 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T16:50:12.496 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T16:50:12.762 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T16:50:12.856 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T16:50:12.926 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T16:50:13.046 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:13.083 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:13.143 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T16:50:13.192 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T16:50:13.274 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T16:50:13.377 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T16:50:13.417 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T16:50:13.457 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T16:50:13.516 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T16:50:13.554 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T16:50:13.616 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T16:50:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T16:50:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:50:13.751 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:50:13.752 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T16:50:13.752 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T16:50:13.752 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:13.779 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:50:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T16:50:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T16:50:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T16:50:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T16:50:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T16:50:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T16:50:14.295 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T16:50:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T16:50:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T16:50:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T16:50:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T16:50:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T16:50:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T16:50:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T16:50:15.014 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T16:50:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T16:50:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T16:50:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T16:50:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T16:50:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T16:50:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:50:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T16:50:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T16:50:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T16:50:15.846 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T16:50:15.895 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T16:50:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T16:50:16.074 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T16:50:16.118 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T16:50:16.183 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T16:50:16.229 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T16:50:16.348 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T16:50:16.464 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T16:50:16.526 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T16:50:16.561 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T16:50:16.761 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T16:50:16.827 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T16:50:17.828 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T16:50:17.884 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T16:50:17.917 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T16:50:17.984 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T16:50:18.099 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T16:50:18.143 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T16:50:18.389 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T16:50:18.423 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T16:50:18.519 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T16:50:19.361 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T16:50:19.508 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T16:50:21.146 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T16:50:21.237 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T16:50:21.274 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T16:50:21.303 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T16:50:21.331 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T16:50:21.698 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T16:50:21.735 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:50:21.847 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:50:21.883 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 125/198 2024-04-16T16:50:21.943 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T16:50:22.522 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T16:50:22.554 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:50:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:50:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 129/198 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-04-16T16:50:23.836 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:24.013 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T16:50:24.029 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:25.480 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:25.520 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:25.571 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:50:25.604 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:50:25.605 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:25.605 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T16:50:25.605 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:50:25.605 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:50:25.605 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-04-16T16:50:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:25.840 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:50:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:50:25.864 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:25.864 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T16:50:25.864 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:50:25.864 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:50:25.864 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:50:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:50:26.018 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:26.018 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T16:50:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:26.203 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:50:26.248 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:50:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:50:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:50:26.365 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:26.366 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T16:50:26.366 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:50:26.366 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:50:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:26.425 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:50:26.425 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T16:50:26.425 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T16:50:26.426 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T16:50:26.465 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:50:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:50:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:50:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 139/198 2024-04-16T16:50:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 140/198 2024-04-16T16:50:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:50:27.546 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:50:27.573 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:50:27.573 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:27.573 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T16:50:27.573 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:50:27.573 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:50:27.573 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:27.805 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:50:27.829 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:50:27.829 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:27.829 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T16:50:27.829 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:50:27.829 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:50:27.829 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:28.121 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:50:28.160 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:50:28.297 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:50:28.321 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:50:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T16:50:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:50:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:50:28.321 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:28.424 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T16:50:28.815 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:50:28.847 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:50:28.880 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 139/198 2024-04-16T16:50:28.935 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 140/198 2024-04-16T16:50:28.962 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:50:29.177 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:50:29.210 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:50:29.315 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:50:29.480 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 144/198 2024-04-16T16:50:29.512 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:50:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:50:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T16:50:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:50:29.537 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:50:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:50:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:50:29.646 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 147/198 2024-04-16T16:50:30.067 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:50:30.106 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:50:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T16:50:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:50:30.672 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:50:30.672 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:30.672 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T16:50:30.672 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:50:30.672 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:50:30.672 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:50:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:50:30.756 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:30.756 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T16:50:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:50:30.832 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:50:30.832 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:30.832 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T16:50:30.832 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:50:30.832 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:50:30.833 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:31.128 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:50:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:50:31.159 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:50:31.183 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:50:31.183 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:31.183 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T16:50:31.183 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:50:31.183 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:50:31.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:50:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 144/198 2024-04-16T16:50:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:50:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:50:31.491 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:31.492 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T16:50:31.492 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:50:31.492 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:50:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:50:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:50:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 147/198 2024-04-16T16:50:32.576 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T16:50:32.602 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:50:32.627 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:50:32.627 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:32.627 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T16:50:32.627 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:50:32.627 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:50:32.627 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:32.662 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:50:32.685 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:50:32.686 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:32.686 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T16:50:32.686 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:32.737 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:50:32.761 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:50:32.761 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:32.761 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T16:50:32.761 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:50:32.762 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:50:32.762 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:33.088 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:50:33.113 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:50:33.114 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:33.114 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T16:50:33.114 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:50:33.114 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:50:33.114 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:36.131 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:50:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 154/198 2024-04-16T16:50:36.202 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 155/198 2024-04-16T16:50:36.229 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T16:50:36.436 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 157/198 2024-04-16T16:50:36.649 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T16:50:36.694 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 159/198 2024-04-16T16:50:36.732 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T16:50:36.785 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T16:50:36.820 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T16:50:36.852 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 163/198 2024-04-16T16:50:36.901 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T16:50:36.931 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T16:50:36.977 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T16:50:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T16:50:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T16:50:37.606 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:37.644 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:37.664 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:37.985 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:50:38.026 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 154/198 2024-04-16T16:50:38.056 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 155/198 2024-04-16T16:50:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T16:50:38.274 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 157/198 2024-04-16T16:50:38.404 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T16:50:38.458 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 159/198 2024-04-16T16:50:38.495 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T16:50:38.548 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T16:50:38.559 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 170/198 2024-04-16T16:50:38.584 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T16:50:38.609 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 163/198 2024-04-16T16:50:38.641 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T16:50:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T16:50:38.671 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T16:50:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T16:50:38.725 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T16:50:38.772 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T16:50:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T16:50:38.809 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T16:50:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T16:50:38.833 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T16:50:38.867 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T16:50:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T16:50:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:50:38.982 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:50:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T16:50:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:50:39.096 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:50:39.123 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T16:50:39.155 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T16:50:39.181 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T16:50:39.206 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T16:50:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T16:50:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T16:50:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T16:50:39.344 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T16:50:39.358 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:39.370 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T16:50:39.395 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:50:39.408 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:39.420 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T16:50:39.427 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:39.427 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:39.430 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:39.606 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:39.607 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:50:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:50:39.612 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /sys 2024-04-16T16:50:39.613 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /proc 2024-04-16T16:50:39.613 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /mnt 2024-04-16T16:50:39.613 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /var/tmp 2024-04-16T16:50:39.613 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /home 2024-04-16T16:50:39.613 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /root 2024-04-16T16:50:39.613 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /tmp 2024-04-16T16:50:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:39.698 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:50:39.729 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:50:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T16:50:39.782 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T16:50:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T16:50:39.840 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:50:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:50:39.890 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:50:39.890 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:39.891 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T16:50:39.891 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:40.319 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 170/198 2024-04-16T16:50:40.395 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T16:50:40.429 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T16:50:40.521 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T16:50:40.557 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T16:50:40.582 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T16:50:40.608 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T16:50:40.634 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T16:50:40.635 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:50:40.722 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:50:40.752 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T16:50:40.752 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:50:40.836 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:50:40.863 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T16:50:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T16:50:40.921 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T16:50:40.946 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T16:50:40.980 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T16:50:41.028 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T16:50:41.059 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T16:50:41.084 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T16:50:41.110 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T16:50:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:50:41.161 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T16:50:41.168 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:41.168 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:41.355 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:41.355 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:50:41.446 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:50:41.487 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:50:41.513 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T16:50:41.532 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:50:41.539 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T16:50:41.564 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T16:50:41.614 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:50:41.614 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:41.614 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T16:50:41.614 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:50:41.614 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:50:41.614 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:41.631 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:50:41.866 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:50:41.939 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:50:41.939 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:41.939 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T16:50:41.939 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:50:41.939 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:50:41.939 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:42.450 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:50:42.522 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:50:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T16:50:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:50:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:50:42.523 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:42.652 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T16:50:43.238 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:50:43.262 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:50:43.314 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 139/198 2024-04-16T16:50:43.420 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 140/198 2024-04-16T16:50:43.449 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:50:44.802 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:50:44.843 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:50:45.350 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 1/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 3/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 4/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 5/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 6/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 7/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 8/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 9/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:50:45.351 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 13/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 14/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 15/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 17/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 18/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 19/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 20/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 21/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 22/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 23/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 24/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 25/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 26/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 27/198 2024-04-16T16:50:45.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 28/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 29/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 30/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 31/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 32/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 33/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 34/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 35/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 36/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 37/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 38/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 39/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T16:50:45.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T16:50:45.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T16:50:45.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T16:50:45.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T16:50:45.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T16:50:45.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T16:50:45.359 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T16:50:45.360 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T16:50:45.361 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T16:50:45.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:50:45.363 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T16:50:45.364 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T16:50:45.364 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T16:50:46.000 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:50:46.026 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:50:46.178 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:50:46.357 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 144/198 2024-04-16T16:50:46.386 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:50:46.469 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:50:46.469 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:46.469 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T16:50:46.469 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:50:46.469 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:50:46.470 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:46.520 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:50:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:50:46.624 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 147/198 2024-04-16T16:50:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:50:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 1/198 2024-04-16T16:50:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2/198 2024-04-16T16:50:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 3/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 4/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 5/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 6/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 7/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 8/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 9/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 13/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 14/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 15/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:50:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 17/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 18/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 19/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 20/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 21/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 22/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 23/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 24/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 25/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 26/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 27/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 28/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 29/198 2024-04-16T16:50:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 30/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 31/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 32/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 33/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 34/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 35/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 36/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 37/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 38/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 39/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T16:50:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T16:50:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T16:50:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T16:50:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T16:50:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T16:50:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T16:50:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T16:50:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T16:50:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T16:50:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T16:50:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T16:50:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T16:50:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T16:50:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T16:50:47.700 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T16:50:47.739 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:50:47.841 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:50:47.841 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:47.841 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T16:50:47.841 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:50:47.841 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:50:47.841 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:47.898 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:50:47.974 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:50:47.974 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:47.974 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T16:50:47.974 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:48.041 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:50:48.120 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:50:48.120 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:48.120 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T16:50:48.120 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:50:48.121 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:50:48.121 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:48.516 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:50:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T16:50:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:48.612 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-04-16T16:50:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T16:50:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T16:50:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T16:50:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-04-16T16:50:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: jq-1.6-16.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T16:50:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T16:50:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T16:50:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T16:50:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T16:50:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T16:50:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T16:50:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T16:50:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T16:50:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T16:50:48.627 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:50:48.627 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:50:48.627 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T16:50:48.627 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:50:48.627 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:50:48.628 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:50:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T16:50:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:50:48.628 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-16T16:50:49.081 DEBUG:teuthology.parallel:result is None 2024-04-16T16:50:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T16:50:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T16:50:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T16:50:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T16:50:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: jq-1.6-16.el9.x86_64 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T16:50:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T16:50:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T16:50:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T16:50:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T16:50:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T16:50:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T16:50:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T16:50:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T16:50:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T16:50:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T16:50:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T16:50:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T16:50:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T16:50:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T16:50:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T16:50:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T16:50:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T16:50:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T16:50:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T16:50:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T16:50:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T16:50:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T16:50:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T16:50:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T16:50:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T16:50:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T16:50:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T16:50:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T16:50:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T16:50:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T16:50:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T16:50:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T16:50:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T16:50:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T16:50:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T16:50:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T16:50:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T16:50:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T16:50:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T16:50:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T16:50:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T16:50:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T16:50:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T16:50:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T16:50:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T16:50:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:50:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T16:50:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T16:50:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T16:50:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T16:50:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T16:50:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T16:50:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T16:50:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-16T16:50:50.051 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-04-16T16:50:50.456 DEBUG:teuthology.parallel:result is None 2024-04-16T16:50:53.857 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:50:53.902 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 154/198 2024-04-16T16:50:53.937 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 155/198 2024-04-16T16:50:53.976 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T16:50:54.296 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 157/198 2024-04-16T16:50:54.423 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T16:50:54.474 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 159/198 2024-04-16T16:50:54.538 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T16:50:54.598 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T16:50:54.645 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T16:50:54.680 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 163/198 2024-04-16T16:50:54.725 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T16:50:54.776 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T16:50:54.864 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T16:50:54.970 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T16:50:55.008 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T16:50:56.037 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:56.080 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:56.139 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:50:57.550 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 170/198 2024-04-16T16:50:57.652 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T16:50:57.697 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T16:50:57.824 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T16:50:57.866 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T16:50:57.913 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T16:50:57.948 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T16:50:57.990 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T16:50:57.991 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:50:58.135 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:50:58.185 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T16:50:58.186 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:50:58.315 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:50:58.360 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T16:50:58.404 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T16:50:58.438 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T16:50:58.470 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T16:50:58.541 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T16:50:58.633 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T16:50:58.684 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T16:50:58.717 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T16:50:58.759 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T16:50:58.792 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:50:58.826 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T16:50:58.863 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:58.864 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:59.201 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:50:59.201 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:50:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:50:59.400 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:50:59.435 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T16:50:59.474 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T16:50:59.514 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T16:50:59.638 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:50:59.897 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:50:59.897 INFO:teuthology.orchestra.run.smithi018.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-16T16:50:59.897 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:51:05.236 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 1/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 3/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 4/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 5/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 6/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 7/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 8/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 9/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:51:05.237 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 13/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 14/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 15/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 17/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 18/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 19/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 20/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 21/198 2024-04-16T16:51:05.238 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 22/198 2024-04-16T16:51:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 23/198 2024-04-16T16:51:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 24/198 2024-04-16T16:51:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 25/198 2024-04-16T16:51:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 26/198 2024-04-16T16:51:05.239 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 27/198 2024-04-16T16:51:05.255 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 28/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 29/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 30/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 31/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 32/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 33/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 34/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 35/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 36/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 37/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 38/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 39/198 2024-04-16T16:51:05.256 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T16:51:05.258 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T16:51:05.259 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T16:51:05.260 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T16:51:05.261 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T16:51:05.262 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T16:51:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T16:51:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T16:51:05.266 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T16:51:05.267 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T16:51:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:51:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T16:51:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T16:51:05.268 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T16:51:07.894 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T16:51:07.894 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:51:07.894 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2024-04-16T16:51:07.894 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T16:51:07.894 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T16:51:07.894 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T16:51:07.894 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.895 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.896 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.897 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: jq-1.6-16.el9.x86_64 2024-04-16T16:51:07.898 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T16:51:07.899 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T16:51:07.900 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T16:51:07.901 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T16:51:07.902 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T16:51:07.903 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T16:51:07.904 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.905 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:51:07.906 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-16T16:51:08.584 DEBUG:teuthology.parallel:result is None 2024-04-16T16:51:08.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:51:08.847 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T16:51:08.900 INFO:teuthology.orchestra.run.smithi018.stdout:19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:51:08.901 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:51:08.901 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.g9b42bfbe is installed. 2024-04-16T16:51:08.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:51:09.050 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T16:51:09.076 INFO:teuthology.orchestra.run.smithi022.stdout:19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:51:09.077 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:51:09.077 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.g9b42bfbe is installed. 2024-04-16T16:51:09.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:51:09.217 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T16:51:09.243 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:51:09.243 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:51:09.243 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.g9b42bfbe is installed. 2024-04-16T16:51:09.245 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-16T16:51:09.245 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:51:09.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T16:51:09.333 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:51:09.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T16:51:09.372 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:51:09.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T16:51:09.412 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-16T16:51:09.412 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:51:09.412 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T16:51:09.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T16:51:09.614 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:51:09.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T16:51:09.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T16:51:09.724 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:51:09.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T16:51:09.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T16:51:09.836 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-16T16:51:09.836 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:51:09.836 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T16:51:09.909 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T16:51:10.025 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:51:10.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T16:51:10.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T16:51:10.128 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:51:10.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T16:51:10.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T16:51:10.229 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-16T16:51:10.230 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:51:10.230 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T16:51:10.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T16:51:10.419 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:51:10.420 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T16:51:10.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T16:51:10.521 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:51:10.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T16:51:10.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T16:51:10.623 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-16T16:51:10.732 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'subvols': {'create': 2}} 2024-04-16T16:51:10.732 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:51:10.733 INFO:tasks.cephadm:Cluster fsid is 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:51:10.733 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-16T16:51:10.733 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.18', 'mon.b': '172.21.15.22', 'mon.c': '172.21.15.43'} 2024-04-16T16:51:10.733 INFO:tasks.cephadm:First mon is mon.a on smithi018 2024-04-16T16:51:10.733 INFO:tasks.cephadm:First mgr is x 2024-04-16T16:51:10.733 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-16T16:51:10.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo hostname $(hostname -s) 2024-04-16T16:51:10.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2024-04-16T16:51:10.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-04-16T16:51:10.878 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-16T16:51:10.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:51:11.096 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78612/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3101-g9b42bfbe', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3101.g9b42bfbe'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 07:47:33.284700', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-qa-debug-resource-temporarily-unavailable-issue', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-16T16:51:11.239 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-mchangir-qa-debug-resource-temporarily-unavailable-issue, sha1 9b42bfbeda298874861ece590aa565cf64ac8c45 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:51:11.241 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/x86_64/flavors/default/cephadm 2024-04-16T16:51:11.241 DEBUG:teuthology.orchestra.run.smithi018:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:11.894 INFO:teuthology.orchestra.run.smithi018.stdout:-rw-r--r--. 1 ubuntu ubuntu 781394 Apr 16 16:51 /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:11.894 DEBUG:teuthology.orchestra.run.smithi022:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:12.334 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-r--r--. 1 ubuntu ubuntu 781394 Apr 16 16:51 /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:12.334 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:12.935 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 781394 Apr 16 16:51 /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:12.936 DEBUG:teuthology.orchestra.run.smithi018:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:13.000 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:13.020 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T16:51:13.055 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 on all hosts... 2024-04-16T16:51:13.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 pull 2024-04-16T16:51:13.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 pull 2024-04-16T16:51:13.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 pull 2024-04-16T16:51:13.889 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:51:13.909 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:51:14.357 INFO:teuthology.orchestra.run.smithi018.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:51:46.936 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-16T16:51:46.937 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev)", 2024-04-16T16:51:46.937 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907", 2024-04-16T16:51:46.937 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2024-04-16T16:51:46.937 INFO:teuthology.orchestra.run.smithi022.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2" 2024-04-16T16:51:46.937 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-16T16:51:46.937 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-16T16:51:47.712 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-16T16:51:47.712 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev)", 2024-04-16T16:51:47.712 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907", 2024-04-16T16:51:47.713 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-04-16T16:51:47.713 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2" 2024-04-16T16:51:47.713 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-16T16:51:47.713 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-16T16:51:53.508 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T16:51:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev)", 2024-04-16T16:51:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "image_id": "e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907", 2024-04-16T16:51:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "repo_digests": [ 2024-04-16T16:51:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2" 2024-04-16T16:51:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T16:51:53.508 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T16:51:53.563 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /etc/ceph 2024-04-16T16:51:53.669 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2024-04-16T16:51:53.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-04-16T16:51:53.748 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 777 /etc/ceph 2024-04-16T16:51:53.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2024-04-16T16:51:53.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-04-16T16:51:53.926 INFO:tasks.cephadm:Writing seed config... 2024-04-16T16:51:53.927 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-16T16:51:53.927 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-16T16:51:53.927 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-16T16:51:53.927 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-16T16:51:53.927 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-16T16:51:53.927 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-16T16:51:53.927 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-16T16:51:53.927 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-16T16:51:53.928 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-16T16:51:53.929 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-16T16:51:53.930 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-16T16:51:53.930 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-16T16:51:53.930 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-16T16:51:53.930 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-16T16:51:53.930 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-16T16:51:53.930 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-16T16:51:53.931 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:51:53.931 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-16T16:51:53.990 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 872930f8-fc11-11ee-bc90-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-16T16:51:53.990 DEBUG:teuthology.orchestra.run.smithi018:mon.a> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a.service 2024-04-16T16:51:54.037 DEBUG:teuthology.orchestra.run.smithi018:mgr.x> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.x.service 2024-04-16T16:51:54.084 INFO:tasks.cephadm:Bootstrapping... 2024-04-16T16:51:54.084 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 -v bootstrap --fsid 872930f8-fc11-11ee-bc90-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.18 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-16T16:51:54.367 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:51:54.367 INFO:teuthology.orchestra.run.smithi018.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45', '-v', 'bootstrap', '--fsid', '872930f8-fc11-11ee-bc90-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.18', '--skip-admin-label'] 2024-04-16T16:51:54.367 INFO:teuthology.orchestra.run.smithi018.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-16T16:51:54.367 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying podman|docker is present... 2024-04-16T16:51:54.429 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-16T16:51:54.429 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying lvm2 is present... 2024-04-16T16:51:54.429 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying time synchronization is in place... 2024-04-16T16:51:54.445 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-16T16:51:54.445 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-16T16:51:54.460 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-16T16:51:54.461 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-16T16:51:54.478 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout enabled 2024-04-16T16:51:54.498 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout active 2024-04-16T16:51:54.498 INFO:teuthology.orchestra.run.smithi018.stdout:Unit chronyd.service is enabled and running 2024-04-16T16:51:54.499 INFO:teuthology.orchestra.run.smithi018.stdout:Repeating the final host check... 2024-04-16T16:51:54.554 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-16T16:51:54.554 INFO:teuthology.orchestra.run.smithi018.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-16T16:51:54.554 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl is present 2024-04-16T16:51:54.555 INFO:teuthology.orchestra.run.smithi018.stdout:lvcreate is present 2024-04-16T16:51:54.572 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-16T16:51:54.572 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-16T16:51:54.588 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-16T16:51:54.589 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-16T16:51:54.610 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout enabled 2024-04-16T16:51:54.628 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout active 2024-04-16T16:51:54.628 INFO:teuthology.orchestra.run.smithi018.stdout:Unit chronyd.service is enabled and running 2024-04-16T16:51:54.628 INFO:teuthology.orchestra.run.smithi018.stdout:Host looks OK 2024-04-16T16:51:54.628 INFO:teuthology.orchestra.run.smithi018.stdout:Cluster fsid: 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:51:54.629 INFO:teuthology.orchestra.run.smithi018.stdout:Acquiring lock 140613816898464 on /run/cephadm/872930f8-fc11-11ee-bc90-c7b262605968.lock 2024-04-16T16:51:54.629 INFO:teuthology.orchestra.run.smithi018.stdout:Lock 140613816898464 acquired on /run/cephadm/872930f8-fc11-11ee-bc90-c7b262605968.lock 2024-04-16T16:51:54.629 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying IP 172.21.15.18 port 3300 ... 2024-04-16T16:51:54.630 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying IP 172.21.15.18 port 6789 ... 2024-04-16T16:51:54.630 INFO:teuthology.orchestra.run.smithi018.stdout:Base mon IP(s) is [172.21.15.18:3300, 172.21.15.18:6789], mon addrv is [v2:172.21.15.18:3300,v1:172.21.15.18:6789] 2024-04-16T16:51:54.639 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.18 metric 100 2024-04-16T16:51:54.639 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.18 metric 100 2024-04-16T16:51:54.646 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-16T16:51:54.646 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-16T16:51:54.647 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-16T16:51:54.654 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-16T16:51:54.654 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-16T16:51:54.654 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-16T16:51:54.654 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-16T16:51:54.654 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1362/64 scope link noprefixroute 2024-04-16T16:51:54.654 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-16T16:51:54.655 INFO:teuthology.orchestra.run.smithi018.stdout:Mon IP `172.21.15.18` is in CIDR network `172.21.0.0/20` 2024-04-16T16:51:54.655 INFO:teuthology.orchestra.run.smithi018.stdout:Mon IP `172.21.15.18` is in CIDR network `172.21.0.0/20` 2024-04-16T16:51:54.655 INFO:teuthology.orchestra.run.smithi018.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-16T16:51:54.656 INFO:teuthology.orchestra.run.smithi018.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-16T16:51:54.657 INFO:teuthology.orchestra.run.smithi018.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:51:55.349 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 2024-04-16T16:51:55.350 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:51:55.350 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Getting image source signatures 2024-04-16T16:51:55.350 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying blob sha256:9793b5b5582798bde394f4383d142e9f248a25e4ebfd72ee05ea9ec6251a877d 2024-04-16T16:51:55.350 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying blob sha256:1b020bf69d7c5884a90622708ad98de4ba7dfcb01b3fd49d68b46566b30aa33a 2024-04-16T16:51:55.350 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying config sha256:e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 2024-04-16T16:51:55.350 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-16T16:51:56.557 INFO:teuthology.orchestra.run.smithi018.stdout:ceph: stdout ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T16:51:56.557 INFO:teuthology.orchestra.run.smithi018.stdout:Ceph version: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T16:51:56.557 INFO:teuthology.orchestra.run.smithi018.stdout:Extracting ceph user uid/gid from container image... 2024-04-16T16:52:06.718 INFO:teuthology.orchestra.run.smithi018.stdout:stat: stdout 167 167 2024-04-16T16:52:06.719 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial keys... 2024-04-16T16:52:07.884 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQC3rB5mf1U0DBAAuOOxw6GbutGYPnqV+3weFw== 2024-04-16T16:52:09.112 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQC4rB5m0tdgHBAA6P+LbRu6R3RHxSGZY+g8iw== 2024-04-16T16:52:10.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQC5rB5mCRmNIhAAcdja8cgNPZGhqaDnPuCzlQ== 2024-04-16T16:52:10.213 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial monmap... 2024-04-16T16:52:12.626 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-16T16:52:12.626 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-16T16:52:12.626 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:12.626 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-16T16:52:12.626 INFO:teuthology.orchestra.run.smithi018.stdout:monmaptool for a [v2:172.21.15.18:3300,v1:172.21.15.18:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-16T16:52:12.627 INFO:teuthology.orchestra.run.smithi018.stdout:setting min_mon_release = quincy 2024-04-16T16:52:12.627 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: set fsid to 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:12.627 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-16T16:52:12.627 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:52:12.627 INFO:teuthology.orchestra.run.smithi018.stdout:Creating mon... 2024-04-16T16:52:13.811 INFO:teuthology.orchestra.run.smithi018.stdout:create mon.a on 2024-04-16T16:52:14.222 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-16T16:52:14.472 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-16T16:52:14.775 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-872930f8-fc11-11ee-bc90-c7b262605968.target → /etc/systemd/system/ceph-872930f8-fc11-11ee-bc90-c7b262605968.target. 2024-04-16T16:52:14.775 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-872930f8-fc11-11ee-bc90-c7b262605968.target → /etc/systemd/system/ceph-872930f8-fc11-11ee-bc90-c7b262605968.target. 2024-04-16T16:52:15.180 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a 2024-04-16T16:52:15.180 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to reset failed state of unit ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a.service: Unit ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a.service not loaded. 2024-04-16T16:52:15.469 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-872930f8-fc11-11ee-bc90-c7b262605968.target.wants/ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a.service → /etc/systemd/system/ceph-872930f8-fc11-11ee-bc90-c7b262605968@.service. 2024-04-16T16:52:16.135 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:15 smithi018 podman[31387]: 2024-04-16 16:52:15.780725442 +0000 UTC m=+0.036911472 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:16.135 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:16 smithi018 podman[31387]: 2024-04-16 16:52:16.132838111 +0000 UTC m=+0.389024218 container init 6582cb72f79f35c2909ac8e7c2a6a93440c6b9623a439c9ba86a1fd377b28c9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-16T16:52:16.257 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T16:52:16.257 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-04-16T16:52:16.274 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T16:52:16.274 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-16T16:52:16.274 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-04-16T16:52:16.275 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to enable service . firewalld.service is not available 2024-04-16T16:52:16.275 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mon to start... 2024-04-16T16:52:16.276 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mon... 2024-04-16T16:52:16.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:16 smithi018 podman[31387]: 2024-04-16 16:52:16.144281857 +0000 UTC m=+0.400467927 container start 6582cb72f79f35c2909ac8e7c2a6a93440c6b9623a439c9ba86a1fd377b28c9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T16:52:16.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:16 smithi018 bash[31387]: 6582cb72f79f35c2909ac8e7c2a6a93440c6b9623a439c9ba86a1fd377b28c9e 2024-04-16T16:52:16.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:16 smithi018 ceph-mon[31406]: mkfs 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:16.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:16 smithi018 systemd[1]: Started Ceph mon.a for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:52:16.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:16 smithi018 ceph-mon[31406]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T16:52:17.351 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: monmap epoch 1 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: last_changed 2024-04-16T16:52:10.664893+0000 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: min_mon_release 19 (squid) 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: election_strategy: 1 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: fsmap 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: osdmap e1: 0 total, 0 up, 0 in 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: mgrmap e1: no daemons active 2024-04-16T16:52:17.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:17 smithi018 ceph-mon[31406]: from='client.? 172.21.15.18:0/2963504174' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-16T16:52:17.632 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout cluster: 2024-04-16T16:52:17.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout id: 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout services: 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.767575s) 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout data: 2024-04-16T16:52:17.634 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-16T16:52:17.640 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout pgs: 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.897+0000 7ff0bb200700 1 Processor -- start 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.897+0000 7ff0bb200700 1 -- start start 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.897+0000 7ff0bb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b4108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.897+0000 7ff0bb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4109040 con 0x7ff0b4108650 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.898+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b4108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.898+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b4108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35520/0 (socket says 172.21.15.18:35520) 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.898+0000 7ff0b3e00700 1 -- 172.21.15.18:0/3934963135 learned_addr learned my addr 172.21.15.18:0/3934963135 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.899+0000 7ff0b3e00700 1 -- 172.21.15.18:0/3934963135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b4109800 con 0x7ff0b4108650 2024-04-16T16:52:17.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.899+0000 7ff0b3e00700 1 --2- 172.21.15.18:0/3934963135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b4108a70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009e80 tx=0x7ff0a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb0388957e1101a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.899+0000 7ff0b3400700 1 -- 172.21.15.18:0/3934963135 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a4017070 con 0x7ff0b4108650 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.899+0000 7ff0b3400700 1 -- 172.21.15.18:0/3934963135 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a4003a20 con 0x7ff0b4108650 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.899+0000 7ff0b3400700 1 -- 172.21.15.18:0/3934963135 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a4003d20 con 0x7ff0b4108650 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.900+0000 7ff0bb200700 1 -- 172.21.15.18:0/3934963135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 msgr2=0x7ff0b4108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.900+0000 7ff0bb200700 1 --2- 172.21.15.18:0/3934963135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b4108a70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009e80 tx=0x7ff0a400b330 comp rx=0 tx=0).stop 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.900+0000 7ff0bb200700 1 -- 172.21.15.18:0/3934963135 shutdown_connections 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.900+0000 7ff0bb200700 1 --2- 172.21.15.18:0/3934963135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b4108a70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.900+0000 7ff0bb200700 1 -- 172.21.15.18:0/3934963135 >> 172.21.15.18:0/3934963135 conn(0x7ff0b4103c30 msgr2=0x7ff0b4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.900+0000 7ff0bb200700 1 -- 172.21.15.18:0/3934963135 shutdown_connections 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.900+0000 7ff0bb200700 1 -- 172.21.15.18:0/3934963135 wait complete. 2024-04-16T16:52:17.642 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.901+0000 7ff0bb200700 1 Processor -- start 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.902+0000 7ff0bb200700 1 -- start start 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.902+0000 7ff0bb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.902+0000 7ff0bb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4109d30 con 0x7ff0b4108650 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.902+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.902+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35532/0 (socket says 172.21.15.18:35532) 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.902+0000 7ff0b3e00700 1 -- 172.21.15.18:0/2963504174 learned_addr learned my addr 172.21.15.18:0/2963504174 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.903+0000 7ff0b3e00700 1 -- 172.21.15.18:0/2963504174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b419e520 con 0x7ff0b4108650 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.903+0000 7ff0b3e00700 1 --2- 172.21.15.18:0/2963504174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b419dfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4004000 tx=0x7ff0a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.903+0000 7ff0b1600700 1 -- 172.21.15.18:0/2963504174 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a4022020 con 0x7ff0b4108650 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.903+0000 7ff0b1600700 1 -- 172.21.15.18:0/2963504174 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a4018040 con 0x7ff0b4108650 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.903+0000 7ff0b1600700 1 -- 172.21.15.18:0/2963504174 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a4017070 con 0x7ff0b4108650 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.903+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b419e7b0 con 0x7ff0b4108650 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.903+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b41a14a0 con 0x7ff0b4108650 2024-04-16T16:52:17.643 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.904+0000 7ff0b1600700 1 -- 172.21.15.18:0/2963504174 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0a4025020 con 0x7ff0b4108650 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.904+0000 7ff0b1600700 1 -- 172.21.15.18:0/2963504174 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff0a400f050 con 0x7ff0b4108650 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.904+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b4064150 con 0x7ff0b4108650 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.908+0000 7ff0b1600700 1 -- 172.21.15.18:0/2963504174 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff0a402c030 con 0x7ff0b4108650 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.990+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff0b402e1a0 con 0x7ff0b4108650 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.990+0000 7ff0b1600700 1 -- 172.21.15.18:0/2963504174 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ff0a401e480 con 0x7ff0b4108650 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.992+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 msgr2=0x7ff0b419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.992+0000 7ff0bb200700 1 --2- 172.21.15.18:0/2963504174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b419dfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4004000 tx=0x7ff0a4004ad0 comp rx=0 tx=0).stop 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.993+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 shutdown_connections 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.993+0000 7ff0bb200700 1 --2- 172.21.15.18:0/2963504174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0b4108650 0x7ff0b419dfe0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.993+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 >> 172.21.15.18:0/2963504174 conn(0x7ff0b4103c30 msgr2=0x7ff0b4194df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.993+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 shutdown_connections 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:16.993+0000 7ff0bb200700 1 -- 172.21.15.18:0/2963504174 wait complete. 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:mon is available 2024-04-16T16:52:17.644 INFO:teuthology.orchestra.run.smithi018.stdout:Assimilating anything we can from ceph.conf... 2024-04-16T16:52:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:18 smithi018 ceph-mon[31406]: from='client.? 172.21.15.18:0/3494482426' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-16T16:52:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:18 smithi018 ceph-mon[31406]: from='client.? 172.21.15.18:0/3494482426' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-16T16:52:19.039 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:19.041 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [global] 2024-04-16T16:52:19.041 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout fsid = 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:19.041 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.18:3300,v1:172.21.15.18:6789] 2024-04-16T16:52:19.041 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-16T16:52:19.041 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-16T16:52:19.041 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-16T16:52:19.041 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-16T16:52:19.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:19.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-16T16:52:19.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-16T16:52:19.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:19.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [osd] 2024-04-16T16:52:19.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-16T16:52:19.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.260+0000 7f1a12200700 1 Processor -- start 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.260+0000 7f1a12200700 1 -- start start 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.260+0000 7f1a12200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.260+0000 7f1a12200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c07c810 con 0x7f1a0c07de20 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.261+0000 7f1a0b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.261+0000 7f1a0b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35536/0 (socket says 172.21.15.18:35536) 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.261+0000 7f1a0b400700 1 -- 172.21.15.18:0/4224447159 learned_addr learned my addr 172.21.15.18:0/4224447159 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.262+0000 7f1a0b400700 1 -- 172.21.15.18:0/4224447159 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c07c990 con 0x7f1a0c07de20 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.262+0000 7f1a0b400700 1 --2- 172.21.15.18:0/4224447159 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c07c2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0098a0 tx=0x7f19fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7f40cf4ef64ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.262+0000 7f1a0aa00700 1 -- 172.21.15.18:0/4224447159 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19fc004160 con 0x7f1a0c07de20 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.262+0000 7f1a0aa00700 1 -- 172.21.15.18:0/4224447159 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc004300 con 0x7f1a0c07de20 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.262+0000 7f1a0aa00700 1 -- 172.21.15.18:0/4224447159 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19fc0036a0 con 0x7f1a0c07de20 2024-04-16T16:52:19.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.263+0000 7f1a12200700 1 -- 172.21.15.18:0/4224447159 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 msgr2=0x7f1a0c07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.263+0000 7f1a12200700 1 --2- 172.21.15.18:0/4224447159 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c07c2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0098a0 tx=0x7f19fc009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.263+0000 7f1a12200700 1 -- 172.21.15.18:0/4224447159 shutdown_connections 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.263+0000 7f1a12200700 1 --2- 172.21.15.18:0/4224447159 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c07c2d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.263+0000 7f1a12200700 1 -- 172.21.15.18:0/4224447159 >> 172.21.15.18:0/4224447159 conn(0x7f1a0c103c60 msgr2=0x7f1a0c106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.263+0000 7f1a12200700 1 -- 172.21.15.18:0/4224447159 shutdown_connections 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.263+0000 7f1a12200700 1 -- 172.21.15.18:0/4224447159 wait complete. 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.264+0000 7f1a12200700 1 Processor -- start 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.264+0000 7f1a12200700 1 -- start start 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.265+0000 7f1a12200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.265+0000 7f1a12200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c07cec0 con 0x7f1a0c07de20 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.265+0000 7f1a0b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.265+0000 7f1a0b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35552/0 (socket says 172.21.15.18:35552) 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.265+0000 7f1a0b400700 1 -- 172.21.15.18:0/3494482426 learned_addr learned my addr 172.21.15.18:0/3494482426 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.265+0000 7f1a0b400700 1 -- 172.21.15.18:0/3494482426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c19e4d0 con 0x7f1a0c07de20 2024-04-16T16:52:19.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.266+0000 7f1a0b400700 1 --2- 172.21.15.18:0/3494482426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c19df90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f19fc000c00 tx=0x7f19fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.266+0000 7f1a08c00700 1 -- 172.21.15.18:0/3494482426 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19fc003db0 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.266+0000 7f1a08c00700 1 -- 172.21.15.18:0/3494482426 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc011630 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.266+0000 7f1a08c00700 1 -- 172.21.15.18:0/3494482426 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19fc011950 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.266+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c19e760 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.266+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c1a1450 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.266+0000 7f1a08c00700 1 -- 172.21.15.18:0/3494482426 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f19fc01a460 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.266+0000 7f1a08c00700 1 -- 172.21.15.18:0/3494482426 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f19fc01b780 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.267+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0c064150 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.270+0000 7f1a08c00700 1 -- 172.21.15.18:0/3494482426 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f19fc01ba20 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.349+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1a0c02e2e0 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.362+0000 7f1a08c00700 1 -- 172.21.15.18:0/3494482426 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f19fc01bc00 con 0x7f1a0c07de20 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.364+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 msgr2=0x7f1a0c19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.364+0000 7f1a12200700 1 --2- 172.21.15.18:0/3494482426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c19df90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f19fc000c00 tx=0x7f19fc004750 comp rx=0 tx=0).stop 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.364+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 shutdown_connections 2024-04-16T16:52:19.050 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.364+0000 7f1a12200700 1 --2- 172.21.15.18:0/3494482426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a0c07de20 0x7f1a0c19df90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:19.051 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.364+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 >> 172.21.15.18:0/3494482426 conn(0x7f1a0c103c60 msgr2=0x7f1a0c105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:19.051 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.365+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 shutdown_connections 2024-04-16T16:52:19.051 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:18.365+0000 7f1a12200700 1 -- 172.21.15.18:0/3494482426 wait complete. 2024-04-16T16:52:19.051 INFO:teuthology.orchestra.run.smithi018.stdout:Generating new minimal ceph.conf... 2024-04-16T16:52:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:19 smithi018 ceph-mon[31406]: from='client.? 172.21.15.18:0/3268262986' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:52:20.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.639+0000 7f4117a00700 1 Processor -- start 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.640+0000 7f4117a00700 1 -- start start 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.640+0000 7f4117a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f4110108a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.640+0000 7f4117a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4110109030 con 0x7f4110108640 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.641+0000 7f4115000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f4110108a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.641+0000 7f4115000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f4110108a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35564/0 (socket says 172.21.15.18:35564) 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.641+0000 7f4115000700 1 -- 172.21.15.18:0/898021815 learned_addr learned my addr 172.21.15.18:0/898021815 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.641+0000 7f4115000700 1 -- 172.21.15.18:0/898021815 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41101097f0 con 0x7f4110108640 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.642+0000 7f4115000700 1 --2- 172.21.15.18:0/898021815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f4110108a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41040098a0 tx=0x7f4104009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc92929583776d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.642+0000 7f410f400700 1 -- 172.21.15.18:0/898021815 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4104004160 con 0x7f4110108640 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.642+0000 7f410f400700 1 -- 172.21.15.18:0/898021815 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4104004300 con 0x7f4110108640 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.642+0000 7f410f400700 1 -- 172.21.15.18:0/898021815 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4104011b50 con 0x7f4110108640 2024-04-16T16:52:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.643+0000 7f4117a00700 1 -- 172.21.15.18:0/898021815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 msgr2=0x7f4110108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.643+0000 7f4117a00700 1 --2- 172.21.15.18:0/898021815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f4110108a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41040098a0 tx=0x7f4104009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.643+0000 7f4117a00700 1 -- 172.21.15.18:0/898021815 shutdown_connections 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.643+0000 7f4117a00700 1 --2- 172.21.15.18:0/898021815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f4110108a60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.643+0000 7f4117a00700 1 -- 172.21.15.18:0/898021815 >> 172.21.15.18:0/898021815 conn(0x7f4110103c60 msgr2=0x7f4110106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.643+0000 7f4117a00700 1 -- 172.21.15.18:0/898021815 shutdown_connections 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.644+0000 7f4117a00700 1 -- 172.21.15.18:0/898021815 wait complete. 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.645+0000 7f4117a00700 1 Processor -- start 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.645+0000 7f4117a00700 1 -- start start 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.645+0000 7f4117a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f411019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.645+0000 7f4117a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f411010cba0 con 0x7f4110108640 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.645+0000 7f4115000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f411019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.645+0000 7f4115000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f411019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35568/0 (socket says 172.21.15.18:35568) 2024-04-16T16:52:20.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.645+0000 7f4115000700 1 -- 172.21.15.18:0/3268262986 learned_addr learned my addr 172.21.15.18:0/3268262986 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.646+0000 7f4115000700 1 -- 172.21.15.18:0/3268262986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411019ed50 con 0x7f4110108640 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.646+0000 7f4115000700 1 --2- 172.21.15.18:0/3268262986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f411019e810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4104000c00 tx=0x7f4104011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.646+0000 7f410d600700 1 -- 172.21.15.18:0/3268262986 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4104003b20 con 0x7f4110108640 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.646+0000 7f410d600700 1 -- 172.21.15.18:0/3268262986 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4104003cc0 con 0x7f4110108640 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.646+0000 7f410d600700 1 -- 172.21.15.18:0/3268262986 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4104024910 con 0x7f4110108640 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.646+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411019efe0 con 0x7f4110108640 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.647+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41101a1cd0 con 0x7f4110108640 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.647+0000 7f410d600700 1 -- 172.21.15.18:0/3268262986 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4104022020 con 0x7f4110108640 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.647+0000 7f410d600700 1 -- 172.21.15.18:0/3268262986 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4104011e00 con 0x7f4110108640 2024-04-16T16:52:20.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.648+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4110064150 con 0x7f4110108640 2024-04-16T16:52:20.383 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.651+0000 7f410d600700 1 -- 172.21.15.18:0/3268262986 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f410402ddf0 con 0x7f4110108640 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.729+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f41101a1ea0 con 0x7f4110108640 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.730+0000 7f410d600700 1 -- 172.21.15.18:0/3268262986 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f4104032030 con 0x7f4110108640 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.731+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 msgr2=0x7f411019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.731+0000 7f4117a00700 1 --2- 172.21.15.18:0/3268262986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f411019e810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4104000c00 tx=0x7f4104011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.731+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 shutdown_connections 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.731+0000 7f4117a00700 1 --2- 172.21.15.18:0/3268262986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4110108640 0x7f411019e810 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.731+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 >> 172.21.15.18:0/3268262986 conn(0x7f4110103c60 msgr2=0x7f4110104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.732+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 shutdown_connections 2024-04-16T16:52:20.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:19.732+0000 7f4117a00700 1 -- 172.21.15.18:0/3268262986 wait complete. 2024-04-16T16:52:20.400 INFO:teuthology.orchestra.run.smithi018.stdout:Restarting the monitor... 2024-04-16T16:52:20.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:20 smithi018 systemd[1]: Stopping Ceph mon.a for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:52:21.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a[31402]: 2024-04-16T16:52:20.893+0000 7fb4b5400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T16:52:21.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a[31402]: 2024-04-16T16:52:20.893+0000 7fb4b5400700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-16T16:52:21.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:20 smithi018 podman[31650]: 2024-04-16 16:52:20.955851347 +0000 UTC m=+0.185973918 container died 6582cb72f79f35c2909ac8e7c2a6a93440c6b9623a439c9ba86a1fd377b28c9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5) 2024-04-16T16:52:21.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:21 smithi018 podman[31650]: 2024-04-16 16:52:21.368807948 +0000 UTC m=+0.598930526 container cleanup 6582cb72f79f35c2909ac8e7c2a6a93440c6b9623a439c9ba86a1fd377b28c9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T16:52:21.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:21 smithi018 bash[31650]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a 2024-04-16T16:52:21.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:21 smithi018 podman[31664]: 2024-04-16 16:52:21.611124325 +0000 UTC m=+0.634566539 container remove 6582cb72f79f35c2909ac8e7c2a6a93440c6b9623a439c9ba86a1fd377b28c9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-16T16:52:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-16T16:52:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 systemd[1]: Stopped Ceph mon.a for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:52:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a.service: Consumed 2.000s CPU time. 2024-04-16T16:52:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 systemd[1]: Starting Ceph mon.a for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:52:22.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 podman[31757]: 2024-04-16 16:52:22.426004313 +0000 UTC m=+0.033173392 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:22.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 podman[31757]: 2024-04-16 16:52:22.645416381 +0000 UTC m=+0.252585504 container create 387238bc74ca4531be73c90bd119996ff34b8b94cee7b79c277dd8f8d969fe49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-16T16:52:22.982 INFO:teuthology.orchestra.run.smithi018.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 podman[31757]: 2024-04-16 16:52:22.890560331 +0000 UTC m=+0.497729474 container init 387238bc74ca4531be73c90bd119996ff34b8b94cee7b79c277dd8f8d969fe49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 podman[31757]: 2024-04-16 16:52:22.901287936 +0000 UTC m=+0.508457046 container start 387238bc74ca4531be73c90bd119996ff34b8b94cee7b79c277dd8f8d969fe49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev), process ceph-mon, pid 2 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: pidfile_write: ignore empty --pid-file 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: load: jerasure load: lrc 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: RocksDB version: 7.9.2 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Git sha 0 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Compile date 2024-04-16 06:46:19 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: DB SUMMARY 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: DB Session ID: 1XW03DDFCN0RJCMS9QRO 2024-04-16T16:52:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: CURRENT file: CURRENT 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106238 ; 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.error_if_exists: 0 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.create_if_missing: 0 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.paranoid_checks: 1 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.env: 0x55e9214c2cc0 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.info_log: 0x55e94b067c60 2024-04-16T16:52:23.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T16:52:23.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.statistics: (nil) 2024-04-16T16:52:23.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.use_fsync: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_log_file_size: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.allow_fallocate: 1 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.use_direct_reads: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.db_log_dir: 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.wal_dir: 2024-04-16T16:52:23.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.write_buffer_manager: 0x55e94b0361e0 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T16:52:23.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.unordered_write: 0 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.row_cache: None 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.wal_filter: None 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.two_write_queues: 0 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.wal_compression: 0 2024-04-16T16:52:23.202 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.atomic_flush: 0 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.log_readahead_size: 0 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_background_jobs: 2 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_background_compactions: -1 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_subcompactions: 1 2024-04-16T16:52:23.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_open_files: -1 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_background_flushes: -1 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Compression algorithms supported: 2024-04-16T16:52:23.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: kZSTD supported: 0 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: kXpressCompression supported: 0 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: kZlibCompression supported: 1 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: kSnappyCompression supported: 1 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: kLZ4Compression supported: 1 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: kBZip2Compression supported: 0 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.merge_operator: 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_filter: None 2024-04-16T16:52:23.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e94b067e20) 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: cache_index_and_filter_blocks: 1 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: pin_top_level_index_and_filter: 1 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_type: 0 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: data_block_index_type: 0 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_shortening: 1 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T16:52:23.206 INFO:journalctl@ceph.mon.a.smithi018.stdout: checksum: 4 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: no_block_cache: 0 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache: 0x55e94b069090 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_name: BinnedLRUCache 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_options: 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: capacity : 536870912 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: num_shard_bits : 4 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: strict_capacity_limit : 0 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: high_pri_pool_ratio: 0.000 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_compressed: (nil) 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: persistent_cache: (nil) 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_size: 4096 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_size_deviation: 10 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_restart_interval: 16 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_block_restart_interval: 1 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: metadata_block_size: 4096 2024-04-16T16:52:23.207 INFO:journalctl@ceph.mon.a.smithi018.stdout: partition_filters: 0 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: use_delta_encoding: 1 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: filter_policy: bloomfilter 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: whole_key_filtering: 1 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: verify_compression: 0 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: read_amp_bytes_per_bit: 0 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: format_version: 5 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: enable_index_compression: 1 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_align: 0 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: max_auto_readahead_size: 262144 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: prepopulate_block_cache: 0 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: initial_auto_readahead_size: 8192 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T16:52:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression: NoCompression 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.num_levels: 7 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:52:23.209 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T16:52:23.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T16:52:23.211 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.inplace_update_support: 0 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T16:52:23.212 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.bloom_locality: 0 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.max_successive_merges: 0 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.ttl: 2592000 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T16:52:23.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T16:52:23.214 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T16:52:23.214 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.enable_blob_files: false 2024-04-16T16:52:23.214 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.min_blob_size: 0 2024-04-16T16:52:23.214 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T16:52:23.214 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T16:52:23.214 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T16:52:23.214 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b54f4375-c506-4e1a-beb8-5c488412a72c 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286342959938, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286342961689, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 99193, "index_size": 305, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14439, "raw_average_key_size": 49, "raw_value_size": 91324, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713286342, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b54f4375-c506-4e1a-beb8-5c488412a72c", "db_session_id": "1XW03DDFCN0RJCMS9QRO", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286342961844, "job": 1, "event": "recovery_finished"} 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e94b156000 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: DB pointer 0x55e94b140000 2024-04-16T16:52:23.215 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** DB Stats ** 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** Compaction Stats [default] ** 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: L0 2/0 100.79 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 88.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Sum 2/0 100.79 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 88.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:52:23.216 INFO:journalctl@ceph.mon.a.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 88.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** Compaction Stats [default] ** 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 88.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T16:52:23.217 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative compaction: 0.00 GB write, 14.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval compaction: 0.00 GB write, 14.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: Block cache BinnedLRUCache@0x55e94b069090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: starting mon.a rank 0 at public addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] at bind addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: mon.a@-1(???) e1 preinit fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: mon.a@-1(???).mds e1 new map 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: mon.a@-1(???).mds e1 print_map 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: e1 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: legacy client fscid: -1 2024-04-16T16:52:23.218 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout: No filesystems configured 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 ceph-mon[31776]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 bash[31757]: 387238bc74ca4531be73c90bd119996ff34b8b94cee7b79c277dd8f8d969fe49 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:22 smithi018 systemd[1]: Started Ceph mon.a for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: monmap epoch 1 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: last_changed 2024-04-16T16:52:10.664893+0000 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: min_mon_release 19 (squid) 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: election_strategy: 1 2024-04-16T16:52:23.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:52:23.220 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: fsmap 2024-04-16T16:52:23.220 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: osdmap e1: 0 total, 0 up, 0 in 2024-04-16T16:52:23.220 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:23 smithi018 ceph-mon[31776]: mgrmap e1: no daemons active 2024-04-16T16:52:24.323 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.580+0000 7fd7dd600700 1 Processor -- start 2024-04-16T16:52:24.323 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.581+0000 7fd7dd600700 1 -- start start 2024-04-16T16:52:24.323 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.581+0000 7fd7dd600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d807c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:24.323 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.581+0000 7fd7dd600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d807dff0 con 0x7fd7d807c310 2024-04-16T16:52:24.323 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.582+0000 7fd7d6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d807c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:24.323 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.582+0000 7fd7d6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d807c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35578/0 (socket says 172.21.15.18:35578) 2024-04-16T16:52:24.323 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.582+0000 7fd7d6a00700 1 -- 172.21.15.18:0/2686056561 learned_addr learned my addr 172.21.15.18:0/2686056561 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.583+0000 7fd7d6a00700 1 -- 172.21.15.18:0/2686056561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d807cc70 con 0x7fd7d807c310 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.583+0000 7fd7d6a00700 1 --2- 172.21.15.18:0/2686056561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d807c730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80098a0 tx=0x7fd7c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a867ab45ba9ecc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.583+0000 7fd7d6000700 1 -- 172.21.15.18:0/2686056561 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7c8004160 con 0x7fd7d807c310 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.584+0000 7fd7d6000700 1 -- 172.21.15.18:0/2686056561 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd7c8004300 con 0x7fd7d807c310 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.584+0000 7fd7d6000700 1 -- 172.21.15.18:0/2686056561 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7c8011b50 con 0x7fd7d807c310 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.584+0000 7fd7dd600700 1 -- 172.21.15.18:0/2686056561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 msgr2=0x7fd7d807c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.584+0000 7fd7dd600700 1 --2- 172.21.15.18:0/2686056561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d807c730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80098a0 tx=0x7fd7c8009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.584+0000 7fd7dd600700 1 -- 172.21.15.18:0/2686056561 shutdown_connections 2024-04-16T16:52:24.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.584+0000 7fd7dd600700 1 --2- 172.21.15.18:0/2686056561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d807c730 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.585+0000 7fd7dd600700 1 -- 172.21.15.18:0/2686056561 >> 172.21.15.18:0/2686056561 conn(0x7fd7d8103cd0 msgr2=0x7fd7d81060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.585+0000 7fd7dd600700 1 -- 172.21.15.18:0/2686056561 shutdown_connections 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.585+0000 7fd7dd600700 1 -- 172.21.15.18:0/2686056561 wait complete. 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.586+0000 7fd7dd600700 1 Processor -- start 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.586+0000 7fd7dd600700 1 -- start start 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.586+0000 7fd7dd600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.586+0000 7fd7dd600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d810ed60 con 0x7fd7d807c310 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.587+0000 7fd7d6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.587+0000 7fd7d6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35586/0 (socket says 172.21.15.18:35586) 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.587+0000 7fd7d6a00700 1 -- 172.21.15.18:0/1497391553 learned_addr learned my addr 172.21.15.18:0/1497391553 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.587+0000 7fd7d6a00700 1 -- 172.21.15.18:0/1497391553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d819ee80 con 0x7fd7d807c310 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.588+0000 7fd7d6a00700 1 --2- 172.21.15.18:0/1497391553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d819e940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8011b60 tx=0x7fd7c8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.588+0000 7fd7cf400700 1 -- 172.21.15.18:0/1497391553 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7c8003970 con 0x7fd7d807c310 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.588+0000 7fd7cf400700 1 -- 172.21.15.18:0/1497391553 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd7c8003b10 con 0x7fd7d807c310 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.588+0000 7fd7cf400700 1 -- 172.21.15.18:0/1497391553 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7c8024610 con 0x7fd7d807c310 2024-04-16T16:52:24.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.588+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d819f110 con 0x7fd7d807c310 2024-04-16T16:52:24.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.588+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d81a1e00 con 0x7fd7d807c310 2024-04-16T16:52:24.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.589+0000 7fd7cf400700 1 -- 172.21.15.18:0/1497391553 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd7c8024ca0 con 0x7fd7d807c310 2024-04-16T16:52:24.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.589+0000 7fd7cf400700 1 -- 172.21.15.18:0/1497391553 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd7c8011e00 con 0x7fd7d807c310 2024-04-16T16:52:24.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.589+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d8064150 con 0x7fd7d807c310 2024-04-16T16:52:24.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.593+0000 7fd7cf400700 1 -- 172.21.15.18:0/1497391553 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd7c801a9f0 con 0x7fd7d807c310 2024-04-16T16:52:24.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.672+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd7d81a1fd0 con 0x7fd7d807c310 2024-04-16T16:52:24.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.675+0000 7fd7cf400700 1 -- 172.21.15.18:0/1497391553 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fd7c801a430 con 0x7fd7d807c310 2024-04-16T16:52:24.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.678+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 msgr2=0x7fd7d819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:24.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.678+0000 7fd7dd600700 1 --2- 172.21.15.18:0/1497391553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d819e940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8011b60 tx=0x7fd7c8011c40 comp rx=0 tx=0).stop 2024-04-16T16:52:24.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.678+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 shutdown_connections 2024-04-16T16:52:24.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.678+0000 7fd7dd600700 1 --2- 172.21.15.18:0/1497391553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7d807c310 0x7fd7d819e940 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:24.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.678+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 >> 172.21.15.18:0/1497391553 conn(0x7fd7d8103cd0 msgr2=0x7fd7d8104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:24.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.678+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 shutdown_connections 2024-04-16T16:52:24.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:23.678+0000 7fd7dd600700 1 -- 172.21.15.18:0/1497391553 wait complete. 2024-04-16T16:52:24.331 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-16T16:52:24.353 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-16T16:52:24.353 INFO:teuthology.orchestra.run.smithi018.stdout:Creating mgr... 2024-04-16T16:52:24.355 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-16T16:52:24.356 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-16T16:52:24.764 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.x 2024-04-16T16:52:24.764 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to reset failed state of unit ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.x.service: Unit ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.x.service not loaded. 2024-04-16T16:52:24.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:24 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1497391553' entity='client.admin' 2024-04-16T16:52:25.013 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-872930f8-fc11-11ee-bc90-c7b262605968.target.wants/ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-872930f8-fc11-11ee-bc90-c7b262605968@.service. 2024-04-16T16:52:25.401 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:25 smithi018 systemd[1]: Starting Ceph mgr.x for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:52:25.654 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:25 smithi018 podman[31998]: 2024-04-16 16:52:25.303421379 +0000 UTC m=+0.033458728 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:25.654 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:25 smithi018 podman[31998]: 2024-04-16 16:52:25.413562417 +0000 UTC m=+0.143599803 container create 29fb07aec5fd4486c0d03ff8bf8b85a785bd74f5adf57ada3d7853d11517497c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-16T16:52:25.654 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:25 smithi018 podman[31998]: 2024-04-16 16:52:25.642716318 +0000 UTC m=+0.372753710 container init 29fb07aec5fd4486c0d03ff8bf8b85a785bd74f5adf57ada3d7853d11517497c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-16T16:52:25.655 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:25 smithi018 podman[31998]: 2024-04-16 16:52:25.653248421 +0000 UTC m=+0.383285801 container start 29fb07aec5fd4486c0d03ff8bf8b85a785bd74f5adf57ada3d7853d11517497c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS) 2024-04-16T16:52:25.802 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T16:52:25.802 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-04-16T16:52:25.822 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T16:52:25.822 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-16T16:52:25.822 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-04-16T16:52:25.822 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to enable service . firewalld.service is not available 2024-04-16T16:52:25.846 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T16:52:25.847 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-04-16T16:52:25.867 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T16:52:25.868 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-16T16:52:25.868 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-04-16T16:52:25.869 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-16T16:52:25.869 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr to start... 2024-04-16T16:52:25.869 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr... 2024-04-16T16:52:25.945 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:25 smithi018 bash[31998]: 29fb07aec5fd4486c0d03ff8bf8b85a785bd74f5adf57ada3d7853d11517497c 2024-04-16T16:52:25.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:25 smithi018 systemd[1]: Started Ceph mgr.x for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:52:26.563 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:26 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:26.446+0000 7f1f852e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:52:26.870 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:26 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1453483135' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T16:52:26.870 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:26 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:26.576+0000 7f1f852e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:52:27.171 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:27.171 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:52:27.172 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T16:52:27.172 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T16:52:27.172 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T16:52:27.173 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T16:52:27.173 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T16:52:27.173 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:27.173 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T16:52:27.173 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T16:52:27.174 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-04-16T16:52:27.174 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:27.174 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T16:52:27.174 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T16:52:27.174 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:27.174 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-16T16:52:27.174 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T16:52:27.175 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:27.175 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T16:52:27.175 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T16:52:27.175 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:27.175 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T16:52:27.176 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:27.176 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T16:52:27.176 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T16:52:27.176 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T16:52:27.176 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T16:52:27.181 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:52:27.182 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T16:52:16.225944+0000", 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.467+0000 7f4fe3200700 1 Processor -- start 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.468+0000 7f4fe3200700 1 -- start start 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.468+0000 7f4fe3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe41088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.468+0000 7f4fe3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4108e80 con 0x7f4fe4108490 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.469+0000 7f4fe1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe41088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.469+0000 7f4fe1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe41088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35620/0 (socket says 172.21.15.18:35620) 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.469+0000 7f4fe1e00700 1 -- 172.21.15.18:0/4243111255 learned_addr learned my addr 172.21.15.18:0/4243111255 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:27.183 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.469+0000 7f4fe1e00700 1 -- 172.21.15.18:0/4243111255 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe4109640 con 0x7f4fe4108490 2024-04-16T16:52:27.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.470+0000 7f4fe1e00700 1 --2- 172.21.15.18:0/4243111255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe41088b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40098a0 tx=0x7f4fd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faeefd4ed4c1731 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:27.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.470+0000 7f4fe0a00700 1 -- 172.21.15.18:0/4243111255 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd4004160 con 0x7f4fe4108490 2024-04-16T16:52:27.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.470+0000 7f4fe0a00700 1 -- 172.21.15.18:0/4243111255 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4fd4004300 con 0x7f4fe4108490 2024-04-16T16:52:27.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.470+0000 7f4fe0a00700 1 -- 172.21.15.18:0/4243111255 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd4011b50 con 0x7f4fe4108490 2024-04-16T16:52:27.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.471+0000 7f4fe3200700 1 -- 172.21.15.18:0/4243111255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 msgr2=0x7f4fe41088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:27.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.471+0000 7f4fe3200700 1 --2- 172.21.15.18:0/4243111255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe41088b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40098a0 tx=0x7f4fd4009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:27.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.471+0000 7f4fe3200700 1 -- 172.21.15.18:0/4243111255 shutdown_connections 2024-04-16T16:52:27.184 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.471+0000 7f4fe3200700 1 --2- 172.21.15.18:0/4243111255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe41088b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.471+0000 7f4fe3200700 1 -- 172.21.15.18:0/4243111255 >> 172.21.15.18:0/4243111255 conn(0x7f4fe4103ab0 msgr2=0x7f4fe4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.471+0000 7f4fe3200700 1 -- 172.21.15.18:0/4243111255 shutdown_connections 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.471+0000 7f4fe3200700 1 -- 172.21.15.18:0/4243111255 wait complete. 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.472+0000 7f4fe3200700 1 Processor -- start 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.472+0000 7f4fe3200700 1 -- start start 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.473+0000 7f4fe3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.473+0000 7f4fe3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe410c9f0 con 0x7f4fe4108490 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.473+0000 7f4fe1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.473+0000 7f4fe1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35622/0 (socket says 172.21.15.18:35622) 2024-04-16T16:52:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.473+0000 7f4fe1e00700 1 -- 172.21.15.18:0/1453483135 learned_addr learned my addr 172.21.15.18:0/1453483135 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.473+0000 7f4fe1e00700 1 -- 172.21.15.18:0/1453483135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe419c950 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.474+0000 7f4fe1e00700 1 --2- 172.21.15.18:0/1453483135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe419c410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.474+0000 7f4fdaa00700 1 -- 172.21.15.18:0/1453483135 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd4003b20 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.474+0000 7f4fdaa00700 1 -- 172.21.15.18:0/1453483135 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4fd4003cc0 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.474+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe419cbe0 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.474+0000 7f4fdaa00700 1 -- 172.21.15.18:0/1453483135 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd4024910 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.474+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe419f8d0 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.475+0000 7f4fdaa00700 1 -- 172.21.15.18:0/1453483135 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4fd4022020 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.475+0000 7f4fdaa00700 1 -- 172.21.15.18:0/1453483135 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4fd4011e00 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.475+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe4069c40 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.478+0000 7f4fdaa00700 1 -- 172.21.15.18:0/1453483135 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4fd4042e50 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.559+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4fe419faa0 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.560+0000 7f4fdaa00700 1 -- 172.21.15.18:0/1453483135 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4fd4031020 con 0x7f4fe4108490 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.561+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 msgr2=0x7f4fe419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.561+0000 7f4fe3200700 1 --2- 172.21.15.18:0/1453483135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe419c410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd4011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.561+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 shutdown_connections 2024-04-16T16:52:27.189 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.561+0000 7f4fe3200700 1 --2- 172.21.15.18:0/1453483135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe4108490 0x7f4fe419c410 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:27.189 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.561+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 >> 172.21.15.18:0/1453483135 conn(0x7f4fe4103ab0 msgr2=0x7f4fe41045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:27.189 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.561+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 shutdown_connections 2024-04-16T16:52:27.189 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:26.562+0000 7f4fe3200700 1 -- 172.21.15.18:0/1453483135 wait complete. 2024-04-16T16:52:27.189 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (1/15)... 2024-04-16T16:52:27.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:27.428+0000 7f1f852e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:52:27.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:27.554+0000 7f1f852e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:52:28.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:27.737+0000 7f1f852e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:52:28.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:27.865+0000 7f1f852e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:52:28.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:27.979+0000 7f1f852e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:52:28.589 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:28.324+0000 7f1f852e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:52:28.590 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:28.469+0000 7f1f852e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:52:28.940 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:28.587+0000 7f1f852e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:52:28.940 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:28.715+0000 7f1f852e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:52:29.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:28.938+0000 7f1f852e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:52:30.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:29 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/224132925' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T16:52:30.579 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:30.352+0000 7f1f852e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:52:30.579 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:30.531+0000 7f1f852e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:52:30.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:30.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:52:30.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T16:52:30.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T16:52:30.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:30.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T16:52:30.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T16:52:30.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:30.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T16:52:16.225944+0000", 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.773+0000 7f1a19200700 1 Processor -- start 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.774+0000 7f1a19200700 1 -- start start 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.774+0000 7f1a19200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1410b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.774+0000 7f1a19200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1407bf20 con 0x7f1a14109190 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.775+0000 7f1a11e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1410b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:30.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.775+0000 7f1a11e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1410b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38554/0 (socket says 172.21.15.18:38554) 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.775+0000 7f1a11e00700 1 -- 172.21.15.18:0/3476974176 learned_addr learned my addr 172.21.15.18:0/3476974176 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.775+0000 7f1a11e00700 1 -- 172.21.15.18:0/3476974176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1410bac0 con 0x7f1a14109190 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.776+0000 7f1a11e00700 1 --2- 172.21.15.18:0/3476974176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1410b580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a040098a0 tx=0x7f1a04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad2af0a907c697b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.776+0000 7f1a10a00700 1 -- 172.21.15.18:0/3476974176 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a04004160 con 0x7f1a14109190 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.776+0000 7f1a10a00700 1 -- 172.21.15.18:0/3476974176 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a04004300 con 0x7f1a14109190 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.776+0000 7f1a10a00700 1 -- 172.21.15.18:0/3476974176 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a04011b50 con 0x7f1a14109190 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.777+0000 7f1a19200700 1 -- 172.21.15.18:0/3476974176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 msgr2=0x7f1a1410b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.777+0000 7f1a19200700 1 --2- 172.21.15.18:0/3476974176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1410b580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a040098a0 tx=0x7f1a04009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.777+0000 7f1a19200700 1 -- 172.21.15.18:0/3476974176 shutdown_connections 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.777+0000 7f1a19200700 1 --2- 172.21.15.18:0/3476974176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1410b580 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.777+0000 7f1a19200700 1 -- 172.21.15.18:0/3476974176 >> 172.21.15.18:0/3476974176 conn(0x7f1a14102b60 msgr2=0x7f1a14104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.777+0000 7f1a19200700 1 -- 172.21.15.18:0/3476974176 shutdown_connections 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.777+0000 7f1a19200700 1 -- 172.21.15.18:0/3476974176 wait complete. 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.779+0000 7f1a19200700 1 Processor -- start 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.779+0000 7f1a19200700 1 -- start start 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.779+0000 7f1a19200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:30.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.779+0000 7f1a19200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1410cb50 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.780+0000 7f1a11e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.780+0000 7f1a11e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38568/0 (socket says 172.21.15.18:38568) 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.780+0000 7f1a11e00700 1 -- 172.21.15.18:0/224132925 learned_addr learned my addr 172.21.15.18:0/224132925 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.780+0000 7f1a11e00700 1 -- 172.21.15.18:0/224132925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1419cc30 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.780+0000 7f1a11e00700 1 --2- 172.21.15.18:0/224132925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1419c6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a04011b60 tx=0x7f1a04011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.781+0000 7f1a0aa00700 1 -- 172.21.15.18:0/224132925 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a04003b20 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.781+0000 7f1a0aa00700 1 -- 172.21.15.18:0/224132925 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a04003cc0 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.781+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1419cec0 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.781+0000 7f1a0aa00700 1 -- 172.21.15.18:0/224132925 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a04024840 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.781+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1419fbb0 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.781+0000 7f1a0aa00700 1 -- 172.21.15.18:0/224132925 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a04022020 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.781+0000 7f1a0aa00700 1 -- 172.21.15.18:0/224132925 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1a04011e00 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.782+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a14064150 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.785+0000 7f1a0aa00700 1 -- 172.21.15.18:0/224132925 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1a04043e50 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.865+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1a141a00a0 con 0x7f1a14109190 2024-04-16T16:52:30.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.866+0000 7f1a0aa00700 1 -- 172.21.15.18:0/224132925 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1a04032030 con 0x7f1a14109190 2024-04-16T16:52:30.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.867+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 msgr2=0x7f1a1419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:30.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.867+0000 7f1a19200700 1 --2- 172.21.15.18:0/224132925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1419c6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a04011b60 tx=0x7f1a04011c40 comp rx=0 tx=0).stop 2024-04-16T16:52:30.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.867+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 shutdown_connections 2024-04-16T16:52:30.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.867+0000 7f1a19200700 1 --2- 172.21.15.18:0/224132925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a14109190 0x7f1a1419c6f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:30.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.867+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 >> 172.21.15.18:0/224132925 conn(0x7f1a14102b60 msgr2=0x7f1a14103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:30.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.867+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 shutdown_connections 2024-04-16T16:52:30.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:29.867+0000 7f1a19200700 1 -- 172.21.15.18:0/224132925 wait complete. 2024-04-16T16:52:30.606 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (2/15)... 2024-04-16T16:52:31.415 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:31.028+0000 7f1f852e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:52:31.416 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:31.151+0000 7f1f852e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:52:31.667 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:31.413+0000 7f1f852e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:52:31.667 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:31.665+0000 7f1f852e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:52:32.381 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:32 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:32.054+0000 7f1f852e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:52:32.381 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:32 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:32.379+0000 7f1f852e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:52:33.056 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:32 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:32.717+0000 7f1f852e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:52:33.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:33 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1535514128' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T16:52:33.967 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:33 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:33.823+0000 7f1f852e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:52:33.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:33.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:52:33.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T16:52:33.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T16:52:33.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T16:52:33.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T16:52:33.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T16:52:33.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:33.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:33.986 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T16:52:33.987 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T16:52:33.987 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:33.987 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T16:52:33.987 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:33.987 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T16:52:33.987 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T16:52:33.990 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T16:52:33.990 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:33.991 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T16:52:16.225944+0000", 2024-04-16T16:52:33.992 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.210+0000 7fe7cdc00700 1 Processor -- start 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.210+0000 7fe7cdc00700 1 -- start start 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.211+0000 7fe7cdc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c810a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.211+0000 7fe7cdc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81082b0 con 0x7fe7c8109d20 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.211+0000 7fe7c6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c810a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.212+0000 7fe7c6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c810a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38582/0 (socket says 172.21.15.18:38582) 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.212+0000 7fe7c6a00700 1 -- 172.21.15.18:0/605227478 learned_addr learned my addr 172.21.15.18:0/605227478 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.212+0000 7fe7c6a00700 1 -- 172.21.15.18:0/605227478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c8108430 con 0x7fe7c8109d20 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.212+0000 7fe7c6a00700 1 --2- 172.21.15.18:0/605227478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c810a140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80098a0 tx=0x7fe7b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc1e768c97b74d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.213+0000 7fe7c5600700 1 -- 172.21.15.18:0/605227478 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7b8004160 con 0x7fe7c8109d20 2024-04-16T16:52:33.998 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.213+0000 7fe7c5600700 1 -- 172.21.15.18:0/605227478 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7b8004300 con 0x7fe7c8109d20 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.213+0000 7fe7c5600700 1 -- 172.21.15.18:0/605227478 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7b8011b50 con 0x7fe7c8109d20 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.213+0000 7fe7cdc00700 1 -- 172.21.15.18:0/605227478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 msgr2=0x7fe7c810a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.213+0000 7fe7cdc00700 1 --2- 172.21.15.18:0/605227478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c810a140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80098a0 tx=0x7fe7b8009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.214+0000 7fe7cdc00700 1 -- 172.21.15.18:0/605227478 shutdown_connections 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.214+0000 7fe7cdc00700 1 --2- 172.21.15.18:0/605227478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c810a140 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.214+0000 7fe7cdc00700 1 -- 172.21.15.18:0/605227478 >> 172.21.15.18:0/605227478 conn(0x7fe7c8103cd0 msgr2=0x7fe7c81060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.214+0000 7fe7cdc00700 1 -- 172.21.15.18:0/605227478 shutdown_connections 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.214+0000 7fe7cdc00700 1 -- 172.21.15.18:0/605227478 wait complete. 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.215+0000 7fe7cdc00700 1 Processor -- start 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.215+0000 7fe7cdc00700 1 -- start start 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.215+0000 7fe7cdc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.215+0000 7fe7cdc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810cbf0 con 0x7fe7c8109d20 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.216+0000 7fe7c6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.216+0000 7fe7c6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38588/0 (socket says 172.21.15.18:38588) 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.216+0000 7fe7c6a00700 1 -- 172.21.15.18:0/1535514128 learned_addr learned my addr 172.21.15.18:0/1535514128 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.216+0000 7fe7c6a00700 1 -- 172.21.15.18:0/1535514128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c819eee0 con 0x7fe7c8109d20 2024-04-16T16:52:33.999 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.216+0000 7fe7c6a00700 1 --2- 172.21.15.18:0/1535514128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c819e9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8011850 tx=0x7fe7b8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.217+0000 7fe7bf400700 1 -- 172.21.15.18:0/1535514128 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7b8003af0 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.217+0000 7fe7bf400700 1 -- 172.21.15.18:0/1535514128 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7b8020070 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.217+0000 7fe7bf400700 1 -- 172.21.15.18:0/1535514128 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7b80195a0 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.217+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c819f170 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.217+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a1e60 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.217+0000 7fe7bf400700 1 -- 172.21.15.18:0/1535514128 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe7b8019740 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.217+0000 7fe7bf400700 1 -- 172.21.15.18:0/1535514128 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe7b8011e00 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.218+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c8064150 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.222+0000 7fe7bf400700 1 -- 172.21.15.18:0/1535514128 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe7b8003c90 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.304+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe7c81a22d0 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.304+0000 7fe7bf400700 1 -- 172.21.15.18:0/1535514128 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fe7b8025460 con 0x7fe7c8109d20 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.306+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 msgr2=0x7fe7c819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.306+0000 7fe7cdc00700 1 --2- 172.21.15.18:0/1535514128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c819e9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8011850 tx=0x7fe7b8011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.306+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 shutdown_connections 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.306+0000 7fe7cdc00700 1 --2- 172.21.15.18:0/1535514128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c8109d20 0x7fe7c819e9a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.306+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 >> 172.21.15.18:0/1535514128 conn(0x7fe7c8103cd0 msgr2=0x7fe7c81047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:34.000 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.306+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 shutdown_connections 2024-04-16T16:52:34.001 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:33.306+0000 7fe7cdc00700 1 -- 172.21.15.18:0/1535514128 wait complete. 2024-04-16T16:52:34.001 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (3/15)... 2024-04-16T16:52:34.687 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:34.550+0000 7f1f852e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:52:34.687 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:34.685+0000 7f1f852e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:52:35.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:34.976+0000 7f1f852e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:52:35.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: Activating manager daemon x 2024-04-16T16:52:35.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: mgrmap e2: x(active, starting, since 0.0064129s) 2024-04-16T16:52:35.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T16:52:35.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: Manager daemon x is now available 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' 2024-04-16T16:52:35.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:35 smithi018 ceph-mon[31776]: from='mgr.14100 172.21.15.18:0/2317645168' entity='mgr.x' 2024-04-16T16:52:37.284 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:36 smithi018 ceph-mon[31776]: mgrmap e3: x(active, since 1.01945s) 2024-04-16T16:52:37.284 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:36 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/437883470' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T16:52:37.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:37.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:52:37.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T16:52:37.600 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T16:52:37.601 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T16:52:37.602 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T16:52:37.603 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:52:37.604 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T16:52:16.225944+0000", 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.649+0000 7f11b4a00700 1 Processor -- start 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.649+0000 7f11b4a00700 1 -- start start 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.649+0000 7f11b4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b010aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.649+0000 7f11b4a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11b010b090 con 0x7f11b010a6a0 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.650+0000 7f11af400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b010aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.650+0000 7f11af400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b010aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38662/0 (socket says 172.21.15.18:38662) 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.650+0000 7f11af400700 1 -- 172.21.15.18:0/3408121626 learned_addr learned my addr 172.21.15.18:0/3408121626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.650+0000 7f11af400700 1 -- 172.21.15.18:0/3408121626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b010b850 con 0x7f11b010a6a0 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.650+0000 7f11af400700 1 --2- 172.21.15.18:0/3408121626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b010aac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f11a00098a0 tx=0x7f11a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4accec1ffc11f493 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.651+0000 7f11aea00700 1 -- 172.21.15.18:0/3408121626 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a0004160 con 0x7f11b010a6a0 2024-04-16T16:52:37.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.651+0000 7f11aea00700 1 -- 172.21.15.18:0/3408121626 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11a0004300 con 0x7f11b010a6a0 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.651+0000 7f11aea00700 1 -- 172.21.15.18:0/3408121626 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a0011b50 con 0x7f11b010a6a0 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.651+0000 7f11b4a00700 1 -- 172.21.15.18:0/3408121626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 msgr2=0x7f11b010aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.651+0000 7f11b4a00700 1 --2- 172.21.15.18:0/3408121626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b010aac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f11a00098a0 tx=0x7f11a0009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.652+0000 7f11b4a00700 1 -- 172.21.15.18:0/3408121626 shutdown_connections 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.652+0000 7f11b4a00700 1 --2- 172.21.15.18:0/3408121626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b010aac0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.652+0000 7f11b4a00700 1 -- 172.21.15.18:0/3408121626 >> 172.21.15.18:0/3408121626 conn(0x7f11b007dc20 msgr2=0x7f11b01084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.652+0000 7f11b4a00700 1 -- 172.21.15.18:0/3408121626 shutdown_connections 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.652+0000 7f11b4a00700 1 -- 172.21.15.18:0/3408121626 wait complete. 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.653+0000 7f11b4a00700 1 Processor -- start 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.653+0000 7f11b4a00700 1 -- start start 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.653+0000 7f11b4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.653+0000 7f11b4a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11b010ec00 con 0x7f11b010a6a0 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.654+0000 7f11af400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.654+0000 7f11af400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38676/0 (socket says 172.21.15.18:38676) 2024-04-16T16:52:37.606 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.654+0000 7f11af400700 1 -- 172.21.15.18:0/437883470 learned_addr learned my addr 172.21.15.18:0/437883470 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.654+0000 7f11af400700 1 -- 172.21.15.18:0/437883470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b019ebb0 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.654+0000 7f11af400700 1 --2- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b019e670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11a0000c00 tx=0x7f11a0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.655+0000 7f11acc00700 1 -- 172.21.15.18:0/437883470 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a0003b20 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.655+0000 7f11acc00700 1 -- 172.21.15.18:0/437883470 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11a0003cc0 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.655+0000 7f11acc00700 1 -- 172.21.15.18:0/437883470 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a0024910 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.655+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b019ee40 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.655+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b01a1b30 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.656+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b0069c40 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.656+0000 7f11acc00700 1 -- 172.21.15.18:0/437883470 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f11a0022020 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.656+0000 7f11acc00700 1 --2- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1194039490 0x7f119403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.656+0000 7f11acc00700 1 -- 172.21.15.18:0/437883470 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11a0011e00 con 0x7f11b010a6a0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.657+0000 7f11a7e00700 1 --2- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1194039490 0x7f119403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:37.607 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.658+0000 7f11a7e00700 1 --2- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1194039490 0x7f119403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1198006fd0 tx=0x7f1198006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.663+0000 7f11acc00700 1 -- 172.21.15.18:0/437883470 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11a001ad80 con 0x7f11b010a6a0 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.981+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f11b01a1d00 con 0x7f11b010a6a0 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.981+0000 7f11acc00700 1 -- 172.21.15.18:0/437883470 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f11b01a1d00 con 0x7f11b010a6a0 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.984+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1194039490 msgr2=0x7f119403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.984+0000 7f11b4a00700 1 --2- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1194039490 0x7f119403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1198006fd0 tx=0x7f1198006e40 comp rx=0 tx=0).stop 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.984+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 msgr2=0x7f11b019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.984+0000 7f11b4a00700 1 --2- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b019e670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11a0000c00 tx=0x7f11a0011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.984+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 shutdown_connections 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.984+0000 7f11b4a00700 1 --2- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1194039490 0x7f119403b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.984+0000 7f11b4a00700 1 --2- 172.21.15.18:0/437883470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b010a6a0 0x7f11b019e670 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.984+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 >> 172.21.15.18:0/437883470 conn(0x7f11b007dc20 msgr2=0x7f11b0107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.985+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 shutdown_connections 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:36.985+0000 7f11b4a00700 1 -- 172.21.15.18:0/437883470 wait complete. 2024-04-16T16:52:37.608 INFO:teuthology.orchestra.run.smithi018.stdout:mgr is available 2024-04-16T16:52:38.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:37 smithi018 ceph-mon[31776]: mgrmap e4: x(active, since 2s) 2024-04-16T16:52:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:38 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1736196158' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-16T16:52:39.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:39.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [global] 2024-04-16T16:52:39.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout fsid = 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:39.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-16T16:52:39.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-16T16:52:39.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-16T16:52:39.327 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [osd] 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.236+0000 7fb770c00700 1 Processor -- start 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.237+0000 7fb770c00700 1 -- start start 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.237+0000 7fb770c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.237+0000 7fb770c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c07c830 con 0x7fb76c07de40 2024-04-16T16:52:39.328 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.238+0000 7fb76b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.238+0000 7fb76b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49694/0 (socket says 172.21.15.18:49694) 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.238+0000 7fb76b400700 1 -- 172.21.15.18:0/498361685 learned_addr learned my addr 172.21.15.18:0/498361685 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.238+0000 7fb76b400700 1 -- 172.21.15.18:0/498361685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c07c9b0 con 0x7fb76c07de40 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.239+0000 7fb76b400700 1 --2- 172.21.15.18:0/498361685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c07c2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098a0 tx=0x7fb75c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea7611c43146b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.239+0000 7fb76aa00700 1 -- 172.21.15.18:0/498361685 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c004160 con 0x7fb76c07de40 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.239+0000 7fb76aa00700 1 -- 172.21.15.18:0/498361685 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb75c004300 con 0x7fb76c07de40 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.239+0000 7fb76aa00700 1 -- 172.21.15.18:0/498361685 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c011b50 con 0x7fb76c07de40 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.239+0000 7fb770c00700 1 -- 172.21.15.18:0/498361685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 msgr2=0x7fb76c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.239+0000 7fb770c00700 1 --2- 172.21.15.18:0/498361685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c07c2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098a0 tx=0x7fb75c009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.240+0000 7fb770c00700 1 -- 172.21.15.18:0/498361685 shutdown_connections 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.240+0000 7fb770c00700 1 --2- 172.21.15.18:0/498361685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c07c2f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.240+0000 7fb770c00700 1 -- 172.21.15.18:0/498361685 >> 172.21.15.18:0/498361685 conn(0x7fb76c103ab0 msgr2=0x7fb76c105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.240+0000 7fb770c00700 1 -- 172.21.15.18:0/498361685 shutdown_connections 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.240+0000 7fb770c00700 1 -- 172.21.15.18:0/498361685 wait complete. 2024-04-16T16:52:39.329 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.241+0000 7fb770c00700 1 Processor -- start 2024-04-16T16:52:39.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.241+0000 7fb770c00700 1 -- start start 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.241+0000 7fb770c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.241+0000 7fb770c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c10ea60 con 0x7fb76c07de40 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.242+0000 7fb76b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.242+0000 7fb76b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49698/0 (socket says 172.21.15.18:49698) 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.242+0000 7fb76b400700 1 -- 172.21.15.18:0/1736196158 learned_addr learned my addr 172.21.15.18:0/1736196158 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.242+0000 7fb76b400700 1 -- 172.21.15.18:0/1736196158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c19eb30 con 0x7fb76c07de40 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.242+0000 7fb76b400700 1 --2- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c19e5f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb75c011b60 tx=0x7fb75c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.243+0000 7fb768c00700 1 -- 172.21.15.18:0/1736196158 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c003b20 con 0x7fb76c07de40 2024-04-16T16:52:39.333 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.243+0000 7fb768c00700 1 -- 172.21.15.18:0/1736196158 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb75c003cc0 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.243+0000 7fb768c00700 1 -- 172.21.15.18:0/1736196158 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c024840 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.243+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c19edc0 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.243+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c1a1ab0 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.244+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c069c40 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.244+0000 7fb768c00700 1 -- 172.21.15.18:0/1736196158 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fb75c022020 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.244+0000 7fb768c00700 1 --2- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7fb7500395b0 0x7fb75003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.245+0000 7fb768c00700 1 -- 172.21.15.18:0/1736196158 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb75c011e00 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.245+0000 7fb762a00700 1 --2- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7fb7500395b0 0x7fb75003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.246+0000 7fb762a00700 1 --2- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7fb7500395b0 0x7fb75003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb754006fd0 tx=0x7fb754006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.250+0000 7fb768c00700 1 -- 172.21.15.18:0/1736196158 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb75c00b3e0 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.478+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb76c1a1c80 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.480+0000 7fb768c00700 1 -- 172.21.15.18:0/1736196158 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fb76c1a1c80 con 0x7fb76c07de40 2024-04-16T16:52:39.334 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.483+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7fb7500395b0 msgr2=0x7fb75003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.483+0000 7fb770c00700 1 --2- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7fb7500395b0 0x7fb75003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb754006fd0 tx=0x7fb754006e40 comp rx=0 tx=0).stop 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.483+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 msgr2=0x7fb76c19e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.483+0000 7fb770c00700 1 --2- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c19e5f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb75c011b60 tx=0x7fb75c011c40 comp rx=0 tx=0).stop 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.483+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 shutdown_connections 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.483+0000 7fb770c00700 1 --2- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7fb7500395b0 0x7fb75003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.483+0000 7fb770c00700 1 --2- 172.21.15.18:0/1736196158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c07de40 0x7fb76c19e5f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.483+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 >> 172.21.15.18:0/1736196158 conn(0x7fb76c103ab0 msgr2=0x7fb76c1045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.484+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 shutdown_connections 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:38.484+0000 7fb770c00700 1 -- 172.21.15.18:0/1736196158 wait complete. 2024-04-16T16:52:39.335 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling cephadm module... 2024-04-16T16:52:40.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:40 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3412001579' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-16T16:52:40.638 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: ignoring --setuser ceph since I am not root 2024-04-16T16:52:40.638 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: ignoring --setgroup ceph since I am not root 2024-04-16T16:52:40.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.919+0000 7f3767600700 1 Processor -- start 2024-04-16T16:52:40.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.919+0000 7f3767600700 1 -- start start 2024-04-16T16:52:40.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.920+0000 7f3767600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f3760108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:40.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.920+0000 7f3767600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3760109070 con 0x7f3760108680 2024-04-16T16:52:40.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.920+0000 7f3764c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f3760108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:40.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.920+0000 7f3764c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f3760108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49702/0 (socket says 172.21.15.18:49702) 2024-04-16T16:52:40.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.920+0000 7f3764c00700 1 -- 172.21.15.18:0/2188713187 learned_addr learned my addr 172.21.15.18:0/2188713187 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.921+0000 7f3764c00700 1 -- 172.21.15.18:0/2188713187 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3760109830 con 0x7f3760108680 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.921+0000 7f3764c00700 1 --2- 172.21.15.18:0/2188713187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f3760108aa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37540098a0 tx=0x7f3754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c67b712a8abbe69 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.922+0000 7f375f400700 1 -- 172.21.15.18:0/2188713187 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3754004160 con 0x7f3760108680 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.922+0000 7f375f400700 1 -- 172.21.15.18:0/2188713187 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3754004300 con 0x7f3760108680 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.922+0000 7f375f400700 1 -- 172.21.15.18:0/2188713187 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3754011b50 con 0x7f3760108680 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.922+0000 7f3767600700 1 -- 172.21.15.18:0/2188713187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 msgr2=0x7f3760108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.922+0000 7f3767600700 1 --2- 172.21.15.18:0/2188713187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f3760108aa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37540098a0 tx=0x7f3754009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.922+0000 7f3767600700 1 -- 172.21.15.18:0/2188713187 shutdown_connections 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.922+0000 7f3767600700 1 --2- 172.21.15.18:0/2188713187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f3760108aa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.923+0000 7f3767600700 1 -- 172.21.15.18:0/2188713187 >> 172.21.15.18:0/2188713187 conn(0x7f3760103d00 msgr2=0x7f3760106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.923+0000 7f3767600700 1 -- 172.21.15.18:0/2188713187 shutdown_connections 2024-04-16T16:52:40.935 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.923+0000 7f3767600700 1 -- 172.21.15.18:0/2188713187 wait complete. 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.924+0000 7f3767600700 1 Processor -- start 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.924+0000 7f3767600700 1 -- start start 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.924+0000 7f3767600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f376019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.924+0000 7f3767600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f376010cbe0 con 0x7f3760108680 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.924+0000 7f3764c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f376019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.925+0000 7f3764c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f376019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49708/0 (socket says 172.21.15.18:49708) 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.925+0000 7f3764c00700 1 -- 172.21.15.18:0/3412001579 learned_addr learned my addr 172.21.15.18:0/3412001579 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.925+0000 7f3764c00700 1 -- 172.21.15.18:0/3412001579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376019cb60 con 0x7f3760108680 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.925+0000 7f3764c00700 1 --2- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f376019c620 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3754011850 tx=0x7f3754011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.925+0000 7f375d600700 1 -- 172.21.15.18:0/3412001579 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3754003af0 con 0x7f3760108680 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.925+0000 7f375d600700 1 -- 172.21.15.18:0/3412001579 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3754020070 con 0x7f3760108680 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.926+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376019cdf0 con 0x7f3760108680 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.926+0000 7f375d600700 1 -- 172.21.15.18:0/3412001579 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37540195a0 con 0x7f3760108680 2024-04-16T16:52:40.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.926+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376019fae0 con 0x7f3760108680 2024-04-16T16:52:40.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.927+0000 7f375d600700 1 -- 172.21.15.18:0/3412001579 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f375401a430 con 0x7f3760108680 2024-04-16T16:52:40.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.927+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3760064150 con 0x7f3760108680 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.927+0000 7f375d600700 1 --2- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f37480395b0 0x7f374803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.927+0000 7f375d600700 1 -- 172.21.15.18:0/3412001579 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3754011e00 con 0x7f3760108680 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.928+0000 7f375fe00700 1 --2- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f37480395b0 0x7f374803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.928+0000 7f375fe00700 1 --2- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f37480395b0 0x7f374803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f374c006fd0 tx=0x7f374c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:39.933+0000 7f375d600700 1 -- 172.21.15.18:0/3412001579 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f375400e260 con 0x7f3760108680 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.227+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f376019fcb0 con 0x7f3760108680 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.281+0000 7f375d600700 1 -- 172.21.15.18:0/3412001579 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f376019fcb0 con 0x7f3760108680 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f375d600700 1 -- 172.21.15.18:0/3412001579 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f375401a720 con 0x7f3760108680 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f37480395b0 msgr2=0x7f374803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f3767600700 1 --2- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f37480395b0 0x7f374803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f374c006fd0 tx=0x7f374c006e40 comp rx=0 tx=0).stop 2024-04-16T16:52:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 msgr2=0x7f376019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:40.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f3767600700 1 --2- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f376019c620 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3754011850 tx=0x7f3754011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:40.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 shutdown_connections 2024-04-16T16:52:40.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f3767600700 1 --2- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f37480395b0 0x7f374803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:40.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f3767600700 1 --2- 172.21.15.18:0/3412001579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3760108680 0x7f376019c620 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3754011850 tx=0x7f3754011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:40.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.285+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 >> 172.21.15.18:0/3412001579 conn(0x7f3760103d00 msgr2=0x7f3760104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:40.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.288+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 shutdown_connections 2024-04-16T16:52:40.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:40.288+0000 7f3767600700 1 -- 172.21.15.18:0/3412001579 wait complete. 2024-04-16T16:52:41.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:41.096+0000 7fea75620200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:52:41.478 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:41 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3412001579' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-16T16:52:41.478 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:41 smithi018 ceph-mon[31776]: mgrmap e5: x(active, since 5s) 2024-04-16T16:52:41.478 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:41.229+0000 7fea75620200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:52:42.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:42 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/571786134' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-16T16:52:42.381 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:42.069+0000 7fea75620200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:52:42.382 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:42.196+0000 7fea75620200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:52:42.382 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:42.379+0000 7fea75620200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:52:42.391 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:52:42.392 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-16T16:52:42.392 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T16:52:42.392 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-16T16:52:42.392 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-16T16:52:42.393 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:52:42.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.476+0000 7f1feaa00700 1 Processor -- start 2024-04-16T16:52:42.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.476+0000 7f1feaa00700 1 -- start start 2024-04-16T16:52:42.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.477+0000 7f1feaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:42.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.477+0000 7f1feaa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe407c810 con 0x7f1fe407de20 2024-04-16T16:52:42.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.477+0000 7f1fe9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:42.396 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.478+0000 7f1fe9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49730/0 (socket says 172.21.15.18:49730) 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.478+0000 7f1fe9600700 1 -- 172.21.15.18:0/2405214493 learned_addr learned my addr 172.21.15.18:0/2405214493 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.478+0000 7f1fe9600700 1 -- 172.21.15.18:0/2405214493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe407c990 con 0x7f1fe407de20 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.478+0000 7f1fe9600700 1 --2- 172.21.15.18:0/2405214493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe407c2d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098a0 tx=0x7f1fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba6a1cb22f90710 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.479+0000 7f1fe3e00700 1 -- 172.21.15.18:0/2405214493 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd8004160 con 0x7f1fe407de20 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.479+0000 7f1fe3e00700 1 -- 172.21.15.18:0/2405214493 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1fd8004300 con 0x7f1fe407de20 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.479+0000 7f1fe3e00700 1 -- 172.21.15.18:0/2405214493 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd8011b50 con 0x7f1fe407de20 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.479+0000 7f1feaa00700 1 -- 172.21.15.18:0/2405214493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 msgr2=0x7f1fe407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.479+0000 7f1feaa00700 1 --2- 172.21.15.18:0/2405214493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe407c2d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098a0 tx=0x7f1fd8009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.480+0000 7f1feaa00700 1 -- 172.21.15.18:0/2405214493 shutdown_connections 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.480+0000 7f1feaa00700 1 --2- 172.21.15.18:0/2405214493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe407c2d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.480+0000 7f1feaa00700 1 -- 172.21.15.18:0/2405214493 >> 172.21.15.18:0/2405214493 conn(0x7f1fe4103ab0 msgr2=0x7f1fe4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.480+0000 7f1feaa00700 1 -- 172.21.15.18:0/2405214493 shutdown_connections 2024-04-16T16:52:42.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.480+0000 7f1feaa00700 1 -- 172.21.15.18:0/2405214493 wait complete. 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.481+0000 7f1feaa00700 1 Processor -- start 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.481+0000 7f1feaa00700 1 -- start start 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.482+0000 7f1feaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.482+0000 7f1feaa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410ea80 con 0x7f1fe407de20 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.482+0000 7f1fe9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.482+0000 7f1fe9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49738/0 (socket says 172.21.15.18:49738) 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.482+0000 7f1fe9600700 1 -- 172.21.15.18:0/571786134 learned_addr learned my addr 172.21.15.18:0/571786134 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.482+0000 7f1fe9600700 1 -- 172.21.15.18:0/571786134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe419eb10 con 0x7f1fe407de20 2024-04-16T16:52:42.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.483+0000 7f1fe9600700 1 --2- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe419e5d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8011b60 tx=0x7f1fd8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:42.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.483+0000 7f1fe2000700 1 -- 172.21.15.18:0/571786134 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd8003b20 con 0x7f1fe407de20 2024-04-16T16:52:42.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.483+0000 7f1fe2000700 1 -- 172.21.15.18:0/571786134 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1fd8003cc0 con 0x7f1fe407de20 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.483+0000 7f1fe2000700 1 -- 172.21.15.18:0/571786134 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd8024840 con 0x7f1fe407de20 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.483+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe419eda0 con 0x7f1fe407de20 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.483+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41a1a90 con 0x7f1fe407de20 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.484+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe4069c40 con 0x7f1fe407de20 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.484+0000 7f1fe2000700 1 -- 172.21.15.18:0/571786134 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f1fd8022020 con 0x7f1fe407de20 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.485+0000 7f1fe2000700 1 --2- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1fcc039660 0x7f1fcc03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.485+0000 7f1fe2000700 1 -- 172.21.15.18:0/571786134 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1fd8011e00 con 0x7f1fe407de20 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.485+0000 7f1fe8c00700 1 -- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1fcc039660 msgr2=0x7f1fcc03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1116733165 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.485+0000 7f1fe8c00700 1 --2- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1fcc039660 0x7f1fcc03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.490+0000 7f1fe2000700 1 -- 172.21.15.18:0/571786134 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fd800b3e0 con 0x7f1fe407de20 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.686+0000 7f1fe8c00700 1 -- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1fcc039660 msgr2=0x7f1fcc03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1116733165 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.686+0000 7f1fe8c00700 1 --2- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1fcc039660 0x7f1fcc03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T16:52:42.402 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.779+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1fe41a1c60 con 0x7f1fe407de20 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.779+0000 7f1fe2000700 1 -- 172.21.15.18:0/571786134 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1fe41a1c60 con 0x7f1fe407de20 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1fcc039660 msgr2=0x7f1fcc03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 --2- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1fcc039660 0x7f1fcc03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 msgr2=0x7f1fe419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 --2- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe419e5d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8011b60 tx=0x7f1fd8011c40 comp rx=0 tx=0).stop 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 shutdown_connections 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 --2- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7f1fcc039660 0x7f1fcc03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 --2- 172.21.15.18:0/571786134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe407de20 0x7f1fe419e5d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 >> 172.21.15.18:0/571786134 conn(0x7f1fe4103ab0 msgr2=0x7f1fe41045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 shutdown_connections 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:41.783+0000 7f1feaa00700 1 -- 172.21.15.18:0/571786134 wait complete. 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for the mgr to restart... 2024-04-16T16:52:42.403 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr epoch 5... 2024-04-16T16:52:42.632 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:42.515+0000 7fea75620200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:52:42.632 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:42.630+0000 7fea75620200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:52:43.247 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:42.983+0000 7fea75620200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:52:43.247 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:43.127+0000 7fea75620200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:52:43.247 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:43.245+0000 7fea75620200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:52:43.602 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:43.375+0000 7fea75620200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:52:43.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:43.601+0000 7fea75620200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:52:45.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:44.927+0000 7fea75620200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:52:45.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:45.106+0000 7fea75620200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:52:45.924 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:45.535+0000 7fea75620200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:52:45.925 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:45.659+0000 7fea75620200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:52:46.180 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:45.923+0000 7fea75620200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:52:46.445 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:46.179+0000 7fea75620200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:52:46.896 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:46.570+0000 7fea75620200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:52:46.896 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:46.895+0000 7fea75620200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:52:47.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:47 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:47.214+0000 7fea75620200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:52:48.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:48 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:48.282+0000 7fea75620200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:52:49.432 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:49.003+0000 7fea75620200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:52:49.432 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:49.137+0000 7fea75620200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:52:49.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:52:49.430+0000 7fea75620200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:52:49.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: Active manager daemon x restarted 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: Activating manager daemon x 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: osdmap e2: 0 total, 0 up, 0 in 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: mgrmap e6: x(active, starting, since 0.00871062s) 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T16:52:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:49 smithi018 ceph-mon[31776]: Manager daemon x is now available 2024-04-16T16:52:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:50 smithi018 ceph-mon[31776]: Found migration_current of "None". Setting to last migration. 2024-04-16T16:52:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:50 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:52:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:50 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:52:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:50 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:52:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:50 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:52:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:50 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T16:52:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:50 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T16:52:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:50 smithi018 ceph-mon[31776]: mgrmap e7: x(active, since 1.01798s) 2024-04-16T16:52:51.198 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:52:51.198 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-16T16:52:51.199 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-16T16:52:51.199 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:52:51.202 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.993+0000 7ff371c00700 1 Processor -- start 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.993+0000 7ff371c00700 1 -- start start 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.993+0000 7ff371c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.993+0000 7ff371c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c07c810 con 0x7ff36c07de20 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.993+0000 7ff36be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.993+0000 7ff36be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49754/0 (socket says 172.21.15.18:49754) 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.993+0000 7ff36be00700 1 -- 172.21.15.18:0/3039006338 learned_addr learned my addr 172.21.15.18:0/3039006338 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.994+0000 7ff36be00700 1 -- 172.21.15.18:0/3039006338 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36c07c990 con 0x7ff36c07de20 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.994+0000 7ff36be00700 1 --2- 172.21.15.18:0/3039006338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c07c2d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0098a0 tx=0x7ff35c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4ff659582e5a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.994+0000 7ff36aa00700 1 -- 172.21.15.18:0/3039006338 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff35c004160 con 0x7ff36c07de20 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.994+0000 7ff36aa00700 1 -- 172.21.15.18:0/3039006338 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff35c004300 con 0x7ff36c07de20 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.994+0000 7ff36aa00700 1 -- 172.21.15.18:0/3039006338 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff35c011b50 con 0x7ff36c07de20 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.995+0000 7ff371c00700 1 -- 172.21.15.18:0/3039006338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 msgr2=0x7ff36c07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:51.203 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.995+0000 7ff371c00700 1 --2- 172.21.15.18:0/3039006338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c07c2d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0098a0 tx=0x7ff35c009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.995+0000 7ff371c00700 1 -- 172.21.15.18:0/3039006338 shutdown_connections 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.995+0000 7ff371c00700 1 --2- 172.21.15.18:0/3039006338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c07c2d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.995+0000 7ff371c00700 1 -- 172.21.15.18:0/3039006338 >> 172.21.15.18:0/3039006338 conn(0x7ff36c103ab0 msgr2=0x7ff36c105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.995+0000 7ff371c00700 1 -- 172.21.15.18:0/3039006338 shutdown_connections 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.995+0000 7ff371c00700 1 -- 172.21.15.18:0/3039006338 wait complete. 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.996+0000 7ff371c00700 1 Processor -- start 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.997+0000 7ff371c00700 1 -- start start 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.997+0000 7ff371c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.997+0000 7ff371c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c10ea80 con 0x7ff36c07de20 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.997+0000 7ff36be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.997+0000 7ff36be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49756/0 (socket says 172.21.15.18:49756) 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.997+0000 7ff36be00700 1 -- 172.21.15.18:0/4006301496 learned_addr learned my addr 172.21.15.18:0/4006301496 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.998+0000 7ff36be00700 1 -- 172.21.15.18:0/4006301496 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36c19eb20 con 0x7ff36c07de20 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.998+0000 7ff36be00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c19e5e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff35c011b60 tx=0x7ff35c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:51.204 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.998+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff35c003b20 con 0x7ff36c07de20 2024-04-16T16:52:51.205 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.998+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff35c003cc0 con 0x7ff36c07de20 2024-04-16T16:52:51.209 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.998+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff35c024840 con 0x7ff36c07de20 2024-04-16T16:52:51.209 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.998+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36c19edb0 con 0x7ff36c07de20 2024-04-16T16:52:51.209 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:42.998+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff36c1a1aa0 con 0x7ff36c07de20 2024-04-16T16:52:51.209 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.000+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff35c022020 con 0x7ff36c07de20 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.000+0000 7ff368c00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.000+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 --> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff35003c180 con 0x7ff3500395b0 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.000+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff35c011e00 con 0x7ff36c07de20 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.000+0000 7ff36b400700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 msgr2=0x7ff35003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1116733165 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.000+0000 7ff36b400700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.201+0000 7ff36b400700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 msgr2=0x7ff35003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1116733165 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.201+0000 7ff36b400700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.603+0000 7ff36b400700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 msgr2=0x7ff35003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1116733165 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:43.603+0000 7ff36b400700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:44.404+0000 7ff36b400700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 msgr2=0x7ff35003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1116733165 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:44.404+0000 7ff36b400700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T16:52:51.210 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:46.006+0000 7ff36b400700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 msgr2=0x7ff35003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1116733165 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:46.006+0000 7ff36b400700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:49.209+0000 7ff36b400700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 msgr2=0x7ff35003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1116733165 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:49.209+0000 7ff36b400700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:49.438+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7ff35c02d910 con 0x7ff36c07de20 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:49.438+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 msgr2=0x7ff35003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:49.438+0000 7ff368c00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.444+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7ff35c04ef50 con 0x7ff36c07de20 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.444+0000 7ff368c00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7ff35003cea0 0x7ff35003f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.444+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff35c001d30 con 0x7ff35003cea0 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.446+0000 7ff36b400700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7ff35003cea0 0x7ff35003f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.451+0000 7ff36b400700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7ff35003cea0 0x7ff35003f290 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff358003a10 tx=0x7ff358009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.453+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mgr.14120 v2:172.21.15.18:6800/1759186913 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff35c001d30 con 0x7ff35003cea0 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.462+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff36c069c40 con 0x7ff35003cea0 2024-04-16T16:52:51.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.462+0000 7ff368c00700 1 -- 172.21.15.18:0/4006301496 <== mgr.14120 v2:172.21.15.18:6800/1759186913 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff36c069c40 con 0x7ff35003cea0 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7ff35003cea0 msgr2=0x7ff35003f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7ff35003cea0 0x7ff35003f290 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff358003a10 tx=0x7ff358009290 comp rx=0 tx=0).stop 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 msgr2=0x7ff36c19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c19e5e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff35c011b60 tx=0x7ff35c011c40 comp rx=0 tx=0).stop 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 shutdown_connections 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7ff35003cea0 0x7ff35003f290 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff36c07de20 0x7ff36c19e5e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 --2- 172.21.15.18:0/4006301496 >> [v2:172.21.15.18:6800/1116733165,v1:172.21.15.18:6801/1116733165] conn(0x7ff3500395b0 0x7ff35003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 >> 172.21.15.18:0/4006301496 conn(0x7ff36c103ab0 msgr2=0x7ff36c104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 shutdown_connections 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:50.463+0000 7ff371c00700 1 -- 172.21.15.18:0/4006301496 wait complete. 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:mgr epoch 5 is available 2024-04-16T16:52:51.212 INFO:teuthology.orchestra.run.smithi018.stdout:Setting orchestrator backend to cephadm... 2024-04-16T16:52:52.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:52 smithi018 ceph-mon[31776]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-16T16:52:52.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:52 smithi018 ceph-mon[31776]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-16T16:52:52.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:52 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:52:52.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:52 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:52:52.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:52 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:52:52.400 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:52 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:52:52.400 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:52 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.800+0000 7f064da00700 1 Processor -- start 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.800+0000 7f064da00700 1 -- start start 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.800+0000 7f064da00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.800+0000 7f064da00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f064807c900 con 0x7f064807de80 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.801+0000 7f0647e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.801+0000 7f0647e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35172/0 (socket says 172.21.15.18:35172) 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.801+0000 7f0647e00700 1 -- 172.21.15.18:0/762370436 learned_addr learned my addr 172.21.15.18:0/762370436 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.801+0000 7f0647e00700 1 -- 172.21.15.18:0/762370436 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064807ca80 con 0x7f064807de80 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.802+0000 7f0647e00700 1 --2- 172.21.15.18:0/762370436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064807c330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06380098a0 tx=0x7f0638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90dce65ee9f9cad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.802+0000 7f0646a00700 1 -- 172.21.15.18:0/762370436 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0638004160 con 0x7f064807de80 2024-04-16T16:52:52.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.802+0000 7f0646a00700 1 -- 172.21.15.18:0/762370436 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0638004300 con 0x7f064807de80 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.802+0000 7f0646a00700 1 -- 172.21.15.18:0/762370436 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0638011b50 con 0x7f064807de80 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.803+0000 7f064da00700 1 -- 172.21.15.18:0/762370436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 msgr2=0x7f064807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.803+0000 7f064da00700 1 --2- 172.21.15.18:0/762370436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064807c330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06380098a0 tx=0x7f0638009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.803+0000 7f064da00700 1 -- 172.21.15.18:0/762370436 shutdown_connections 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.803+0000 7f064da00700 1 --2- 172.21.15.18:0/762370436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064807c330 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.803+0000 7f064da00700 1 -- 172.21.15.18:0/762370436 >> 172.21.15.18:0/762370436 conn(0x7f0648103ab0 msgr2=0x7f0648105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.803+0000 7f064da00700 1 -- 172.21.15.18:0/762370436 shutdown_connections 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.803+0000 7f064da00700 1 -- 172.21.15.18:0/762370436 wait complete. 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.804+0000 7f064da00700 1 Processor -- start 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.804+0000 7f064da00700 1 -- start start 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.804+0000 7f064da00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.804+0000 7f064da00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f064810ea20 con 0x7f064807de80 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.805+0000 7f0647e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.805+0000 7f0647e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35188/0 (socket says 172.21.15.18:35188) 2024-04-16T16:52:52.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.805+0000 7f0647e00700 1 -- 172.21.15.18:0/2893247704 learned_addr learned my addr 172.21.15.18:0/2893247704 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.805+0000 7f0647e00700 1 -- 172.21.15.18:0/2893247704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064819eb10 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.805+0000 7f0647e00700 1 --2- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064819e5d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0638011b60 tx=0x7f0638011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.806+0000 7f0644c00700 1 -- 172.21.15.18:0/2893247704 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0638003b20 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.806+0000 7f0644c00700 1 -- 172.21.15.18:0/2893247704 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0638003cc0 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.806+0000 7f0644c00700 1 -- 172.21.15.18:0/2893247704 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0638024840 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.806+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064819eda0 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.806+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06481a1a90 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.807+0000 7f0644c00700 1 -- 172.21.15.18:0/2893247704 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f0638022020 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.807+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0648069c40 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.807+0000 7f0644c00700 1 --2- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f062c0394e0 0x7f062c03b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.807+0000 7f0644c00700 1 -- 172.21.15.18:0/2893247704 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0638011e00 con 0x7f064807de80 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.808+0000 7f0647400700 1 --2- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f062c0394e0 0x7f062c03b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.809+0000 7f0647400700 1 --2- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f062c0394e0 0x7f062c03b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0634006fd0 tx=0x7f0634006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:52.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:51.814+0000 7f0644c00700 1 -- 172.21.15.18:0/2893247704 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f063800b3e0 con 0x7f064807de80 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.063+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0648104d00 con 0x7f062c0394e0 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.075+0000 7f0644c00700 1 -- 172.21.15.18:0/2893247704 <== mgr.14120 v2:172.21.15.18:6800/1759186913 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0648104d00 con 0x7f062c0394e0 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.078+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f062c0394e0 msgr2=0x7f062c03b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.078+0000 7f064da00700 1 --2- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f062c0394e0 0x7f062c03b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0634006fd0 tx=0x7f0634006e40 comp rx=0 tx=0).stop 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.078+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 msgr2=0x7f064819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.078+0000 7f064da00700 1 --2- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064819e5d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0638011b60 tx=0x7f0638011c40 comp rx=0 tx=0).stop 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.079+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 shutdown_connections 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.079+0000 7f064da00700 1 --2- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f062c0394e0 0x7f062c03b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.079+0000 7f064da00700 1 --2- 172.21.15.18:0/2893247704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064807de80 0x7f064819e5d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.079+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 >> 172.21.15.18:0/2893247704 conn(0x7f0648103ab0 msgr2=0x7f06481045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.079+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 shutdown_connections 2024-04-16T16:52:52.679 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:52.079+0000 7f064da00700 1 -- 172.21.15.18:0/2893247704 wait complete. 2024-04-16T16:52:53.495 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:53 smithi018 ceph-mon[31776]: [16/Apr/2024:16:52:51] ENGINE Bus STARTING 2024-04-16T16:52:53.495 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:53 smithi018 ceph-mon[31776]: [16/Apr/2024:16:52:51] ENGINE Serving on https://172.21.15.18:7150 2024-04-16T16:52:53.495 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:53 smithi018 ceph-mon[31776]: [16/Apr/2024:16:52:51] ENGINE Serving on http://172.21.15.18:8765 2024-04-16T16:52:53.495 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:53 smithi018 ceph-mon[31776]: [16/Apr/2024:16:52:51] ENGINE Bus STARTED 2024-04-16T16:52:53.495 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:53 smithi018 ceph-mon[31776]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:52:53.495 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:53 smithi018 ceph-mon[31776]: mgrmap e8: x(active, since 2s) 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.231+0000 7f9409600700 1 Processor -- start 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.231+0000 7f9409600700 1 -- start start 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.231+0000 7f9409600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940410acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.231+0000 7f9409600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f940410b2a0 con 0x7f940410a8b0 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.232+0000 7f9402a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940410acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.232+0000 7f9402a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940410acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35202/0 (socket says 172.21.15.18:35202) 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.232+0000 7f9402a00700 1 -- 172.21.15.18:0/1402344746 learned_addr learned my addr 172.21.15.18:0/1402344746 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.233+0000 7f9402a00700 1 -- 172.21.15.18:0/1402344746 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940410ba60 con 0x7f940410a8b0 2024-04-16T16:52:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.233+0000 7f9402a00700 1 --2- 172.21.15.18:0/1402344746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940410acd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098a0 tx=0x7f93f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcd0e9597c07bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.233+0000 7f9401600700 1 -- 172.21.15.18:0/1402344746 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f4004160 con 0x7f940410a8b0 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.233+0000 7f9401600700 1 -- 172.21.15.18:0/1402344746 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93f4004300 con 0x7f940410a8b0 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.233+0000 7f9401600700 1 -- 172.21.15.18:0/1402344746 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f4011b50 con 0x7f940410a8b0 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.234+0000 7f9409600700 1 -- 172.21.15.18:0/1402344746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 msgr2=0x7f940410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.234+0000 7f9409600700 1 --2- 172.21.15.18:0/1402344746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940410acd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098a0 tx=0x7f93f4009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.234+0000 7f9409600700 1 -- 172.21.15.18:0/1402344746 shutdown_connections 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.234+0000 7f9409600700 1 --2- 172.21.15.18:0/1402344746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940410acd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.234+0000 7f9409600700 1 -- 172.21.15.18:0/1402344746 >> 172.21.15.18:0/1402344746 conn(0x7f940407ddf0 msgr2=0x7f94041084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.234+0000 7f9409600700 1 -- 172.21.15.18:0/1402344746 shutdown_connections 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.235+0000 7f9409600700 1 -- 172.21.15.18:0/1402344746 wait complete. 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.236+0000 7f9409600700 1 Processor -- start 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.236+0000 7f9409600700 1 -- start start 2024-04-16T16:52:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.236+0000 7f9409600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.236+0000 7f9409600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f940410ee10 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.236+0000 7f9402a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.237+0000 7f9402a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35206/0 (socket says 172.21.15.18:35206) 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.237+0000 7f9402a00700 1 -- 172.21.15.18:0/1647439546 learned_addr learned my addr 172.21.15.18:0/1647439546 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.237+0000 7f9402a00700 1 -- 172.21.15.18:0/1647439546 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940419edc0 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.237+0000 7f9402a00700 1 --2- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940419e880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93f4011850 tx=0x7f93f4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.237+0000 7f93fb400700 1 -- 172.21.15.18:0/1647439546 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f4003af0 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.237+0000 7f93fb400700 1 -- 172.21.15.18:0/1647439546 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93f4020070 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.237+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940419f050 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.238+0000 7f93fb400700 1 -- 172.21.15.18:0/1647439546 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f40195a0 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.238+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94041a1d40 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.239+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9404069c40 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.239+0000 7f93fb400700 1 -- 172.21.15.18:0/1647439546 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f93f4019870 con 0x7f940410a8b0 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.240+0000 7f93fb400700 1 --2- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f93e8039600 0x7f93e803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:54.179 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.240+0000 7f93fb400700 1 -- 172.21.15.18:0/1647439546 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f93f4011e00 con 0x7f940410a8b0 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.240+0000 7f9402000700 1 --2- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f93e8039600 0x7f93e803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.241+0000 7f9402000700 1 --2- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f93e8039600 0x7f93e803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006fd0 tx=0x7f93f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.245+0000 7f93fb400700 1 -- 172.21.15.18:0/1647439546 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93f400e260 con 0x7f940410a8b0 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.473+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9404108370 con 0x7f93e8039600 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.475+0000 7f93fb400700 1 -- 172.21.15.18:0/1647439546 <== mgr.14120 v2:172.21.15.18:6800/1759186913 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9404108370 con 0x7f93e8039600 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.477+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f93e8039600 msgr2=0x7f93e803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.477+0000 7f9409600700 1 --2- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f93e8039600 0x7f93e803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006fd0 tx=0x7f93f0006e40 comp rx=0 tx=0).stop 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.477+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 msgr2=0x7f940419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.477+0000 7f9409600700 1 --2- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940419e880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93f4011850 tx=0x7f93f4011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.477+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 shutdown_connections 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.477+0000 7f9409600700 1 --2- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f93e8039600 0x7f93e803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.477+0000 7f9409600700 1 --2- 172.21.15.18:0/1647439546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f940410a8b0 0x7f940419e880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.477+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 >> 172.21.15.18:0/1647439546 conn(0x7f940407ddf0 msgr2=0x7f9404107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.478+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 shutdown_connections 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:53.478+0000 7f9409600700 1 -- 172.21.15.18:0/1647439546 wait complete. 2024-04-16T16:52:54.180 INFO:teuthology.orchestra.run.smithi018.stdout:Generating ssh key... 2024-04-16T16:52:54.763 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:54 smithi018 ceph-mon[31776]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:52:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-mon[31776]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:52:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-mon[31776]: Generating ssh key... 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: Generating public/private rsa key pair. 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: Your identification has been saved in /tmp/tmpu7t0do7_/key. 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: Your public key has been saved in /tmp/tmpu7t0do7_/key.pub. 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: The key fingerprint is: 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: SHA256:1RvqbjCUK1FbDzNwHRVQsQEiHnkuBQWzPrIEbDnvpy0 ceph-872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: The key's randomart image is: 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: +---[RSA 3072]----+ 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | B*+.+**o | 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | . . .o*Bo. o | 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | * .oB.=o . | 2024-04-16T16:52:55.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | . +..=....o | 2024-04-16T16:52:55.948 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | +oSo. . | 2024-04-16T16:52:55.948 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | o.o+o | 2024-04-16T16:52:55.948 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | o..o. | 2024-04-16T16:52:55.948 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | E+ .. | 2024-04-16T16:52:55.948 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: | ..... | 2024-04-16T16:52:55.948 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:52:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: +----[SHA256]-----+ 2024-04-16T16:52:56.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.761+0000 7f9d08000700 1 Processor -- start 2024-04-16T16:52:56.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.761+0000 7f9d08000700 1 -- start start 2024-04-16T16:52:56.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.762+0000 7f9d08000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0010ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:56.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.762+0000 7f9d08000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010b260 con 0x7f9d0010a870 2024-04-16T16:52:56.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.762+0000 7f9d05600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0010ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:56.335 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.762+0000 7f9d05600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0010ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35222/0 (socket says 172.21.15.18:35222) 2024-04-16T16:52:56.336 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.762+0000 7f9d05600700 1 -- 172.21.15.18:0/473519984 learned_addr learned my addr 172.21.15.18:0/473519984 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:56.336 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.763+0000 7f9d05600700 1 -- 172.21.15.18:0/473519984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0010ba20 con 0x7f9d0010a870 2024-04-16T16:52:56.336 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.763+0000 7f9d05600700 1 --2- 172.21.15.18:0/473519984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0010ac90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40098a0 tx=0x7f9cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c78721f9d6e3a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:56.337 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.763+0000 7f9cffe00700 1 -- 172.21.15.18:0/473519984 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cf4004160 con 0x7f9d0010a870 2024-04-16T16:52:56.337 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.763+0000 7f9cffe00700 1 -- 172.21.15.18:0/473519984 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9cf4004300 con 0x7f9d0010a870 2024-04-16T16:52:56.337 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.763+0000 7f9cffe00700 1 -- 172.21.15.18:0/473519984 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cf4011b50 con 0x7f9d0010a870 2024-04-16T16:52:56.337 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.764+0000 7f9d08000700 1 -- 172.21.15.18:0/473519984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 msgr2=0x7f9d0010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:56.337 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.764+0000 7f9d08000700 1 --2- 172.21.15.18:0/473519984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0010ac90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40098a0 tx=0x7f9cf4009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:56.338 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.764+0000 7f9d08000700 1 -- 172.21.15.18:0/473519984 shutdown_connections 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.764+0000 7f9d08000700 1 --2- 172.21.15.18:0/473519984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0010ac90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.764+0000 7f9d08000700 1 -- 172.21.15.18:0/473519984 >> 172.21.15.18:0/473519984 conn(0x7f9d0007de20 msgr2=0x7f9d001084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.764+0000 7f9d08000700 1 -- 172.21.15.18:0/473519984 shutdown_connections 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.764+0000 7f9d08000700 1 -- 172.21.15.18:0/473519984 wait complete. 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.765+0000 7f9d08000700 1 Processor -- start 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.765+0000 7f9d08000700 1 -- start start 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.766+0000 7f9d08000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.766+0000 7f9d08000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010edd0 con 0x7f9d0010a870 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.766+0000 7f9d05600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.766+0000 7f9d05600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35238/0 (socket says 172.21.15.18:35238) 2024-04-16T16:52:56.339 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.766+0000 7f9d05600700 1 -- 172.21.15.18:0/688272068 learned_addr learned my addr 172.21.15.18:0/688272068 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:56.340 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.767+0000 7f9d05600700 1 -- 172.21.15.18:0/688272068 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0019eda0 con 0x7f9d0010a870 2024-04-16T16:52:56.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.767+0000 7f9d05600700 1 --2- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0019e860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4011850 tx=0x7f9cf4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:56.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.767+0000 7f9cfe000700 1 -- 172.21.15.18:0/688272068 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cf4003af0 con 0x7f9d0010a870 2024-04-16T16:52:56.342 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.767+0000 7f9cfe000700 1 -- 172.21.15.18:0/688272068 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9cf4020070 con 0x7f9d0010a870 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.767+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0019f030 con 0x7f9d0010a870 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.767+0000 7f9cfe000700 1 -- 172.21.15.18:0/688272068 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cf40195a0 con 0x7f9d0010a870 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.767+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d001a1d20 con 0x7f9d0010a870 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.768+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d00064150 con 0x7f9d0010a870 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.769+0000 7f9cfe000700 1 -- 172.21.15.18:0/688272068 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9cf401a430 con 0x7f9d0010a870 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.769+0000 7f9cfe000700 1 --2- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f9ce8039600 0x7f9ce803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.769+0000 7f9cfe000700 1 -- 172.21.15.18:0/688272068 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9cf4011e00 con 0x7f9d0010a870 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.769+0000 7f9d04c00700 1 --2- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f9ce8039600 0x7f9ce803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.770+0000 7f9d04c00700 1 --2- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f9ce8039600 0x7f9ce803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006fd0 tx=0x7f9cec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:54.775+0000 7f9cfe000700 1 -- 172.21.15.18:0/688272068 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cf400e260 con 0x7f9d0010a870 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.007+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f9d00108360 con 0x7f9ce8039600 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.685+0000 7f9cfe000700 1 -- 172.21.15.18:0/688272068 <== mgr.14120 v2:172.21.15.18:6800/1759186913 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d00108360 con 0x7f9ce8039600 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f9ce8039600 msgr2=0x7f9ce803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:56.343 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 --2- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f9ce8039600 0x7f9ce803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006fd0 tx=0x7f9cec006e40 comp rx=0 tx=0).stop 2024-04-16T16:52:56.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 msgr2=0x7f9d0019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:56.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 --2- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0019e860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4011850 tx=0x7f9cf4011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:56.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 shutdown_connections 2024-04-16T16:52:56.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 --2- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f9ce8039600 0x7f9ce803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:56.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 --2- 172.21.15.18:0/688272068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d0010a870 0x7f9d0019e860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:56.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 >> 172.21.15.18:0/688272068 conn(0x7f9d0007de20 msgr2=0x7f9d00107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:56.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 shutdown_connections 2024-04-16T16:52:56.344 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:55.688+0000 7f9d08000700 1 -- 172.21.15.18:0/688272068 wait complete. 2024-04-16T16:52:56.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:56 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:52:56.927 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:56 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:52:57.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:57 smithi018 ceph-mon[31776]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:52:57.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOBALsi02soSat7yWqMzlbYlQGnjU6USLx/9UX7Uc6sjpggY2j38KNo0+L1xF9IEy61Fxk0b+aTVcLQonH0LJKBCSR/PfiEDJlAoqInP/JdBY6MXLZUrQXZ4Gtvjvha275BHRgM0nNKzz9ncwVjHPQaoLb5GBPrHE0ISOV1QKH47DZD8/5KMdg8PKpbjDkIOM2G/lSoy3UtyZ7kIdbTn2qj78eZ88feuDn8KWbddlb/lNO/2lsJkRXVG15YwW9EJ0V7kgaooF/aSzv2q7Z+WnvTq8J9KdMiSg1BcT68uj++nl+V0D3BZGOAto9L1XzJFsrPKAsnchOIXys3lOieeTESHMqZJAPWUjJNYfVhgcIWPB7ndUAlJZfmbPcAS7agDEkCNF5eQoC9OIIefD4uTlKI0Or6Uvdiq0Cl0EBedI4H9WiXayWeR40inxUcEaknHYl61+nBaI5jRgYhdrTMXl9AjQB3MoskkC46iTJTg8QEa8R245IeV/fAoa0hT+yekk= ceph-872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:57.861 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.925+0000 7f508aa00700 1 Processor -- start 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.925+0000 7f508aa00700 1 -- start start 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.925+0000 7f508aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f508410f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.925+0000 7f508aa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f508410fab0 con 0x7f508410f0c0 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.926+0000 7f5083e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f508410f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.926+0000 7f5083e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f508410f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50408/0 (socket says 172.21.15.18:50408) 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.926+0000 7f5083e00700 1 -- 172.21.15.18:0/1638272795 learned_addr learned my addr 172.21.15.18:0/1638272795 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.926+0000 7f5083e00700 1 -- 172.21.15.18:0/1638272795 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508410fdf0 con 0x7f508410f0c0 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.927+0000 7f5083e00700 1 --2- 172.21.15.18:0/1638272795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f508410f4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f50740098a0 tx=0x7f5074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e478c8bb59a5738b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.927+0000 7f5082a00700 1 -- 172.21.15.18:0/1638272795 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5074004160 con 0x7f508410f0c0 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.927+0000 7f5082a00700 1 -- 172.21.15.18:0/1638272795 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5074004300 con 0x7f508410f0c0 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.927+0000 7f5082a00700 1 -- 172.21.15.18:0/1638272795 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5074011b50 con 0x7f508410f0c0 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.928+0000 7f508aa00700 1 -- 172.21.15.18:0/1638272795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 msgr2=0x7f508410f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.928+0000 7f508aa00700 1 --2- 172.21.15.18:0/1638272795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f508410f4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f50740098a0 tx=0x7f5074009bb0 comp rx=0 tx=0).stop 2024-04-16T16:52:57.862 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.928+0000 7f508aa00700 1 -- 172.21.15.18:0/1638272795 shutdown_connections 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.928+0000 7f508aa00700 1 --2- 172.21.15.18:0/1638272795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f508410f4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.928+0000 7f508aa00700 1 -- 172.21.15.18:0/1638272795 >> 172.21.15.18:0/1638272795 conn(0x7f508407de20 msgr2=0x7f5084080260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.928+0000 7f508aa00700 1 -- 172.21.15.18:0/1638272795 shutdown_connections 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.928+0000 7f508aa00700 1 -- 172.21.15.18:0/1638272795 wait complete. 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.929+0000 7f508aa00700 1 Processor -- start 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.929+0000 7f508aa00700 1 -- start start 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.929+0000 7f508aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f50841a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.929+0000 7f508aa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50841131a0 con 0x7f508410f0c0 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.930+0000 7f5083e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f50841a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.930+0000 7f5083e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f50841a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50424/0 (socket says 172.21.15.18:50424) 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.930+0000 7f5083e00700 1 -- 172.21.15.18:0/3335642123 learned_addr learned my addr 172.21.15.18:0/3335642123 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.930+0000 7f5083e00700 1 -- 172.21.15.18:0/3335642123 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841a31c0 con 0x7f508410f0c0 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.930+0000 7f5083e00700 1 --2- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f50841a2c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5074011850 tx=0x7f5074011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.931+0000 7f5080c00700 1 -- 172.21.15.18:0/3335642123 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5074003af0 con 0x7f508410f0c0 2024-04-16T16:52:57.863 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.931+0000 7f5080c00700 1 -- 172.21.15.18:0/3335642123 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5074020070 con 0x7f508410f0c0 2024-04-16T16:52:57.864 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.931+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841a3450 con 0x7f508410f0c0 2024-04-16T16:52:57.864 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.931+0000 7f5080c00700 1 -- 172.21.15.18:0/3335642123 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50740195a0 con 0x7f508410f0c0 2024-04-16T16:52:57.864 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.931+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841a6140 con 0x7f508410f0c0 2024-04-16T16:52:57.864 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.932+0000 7f5080c00700 1 -- 172.21.15.18:0/3335642123 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5074019850 con 0x7f508410f0c0 2024-04-16T16:52:57.866 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.932+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084064150 con 0x7f508410f0c0 2024-04-16T16:52:57.866 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.933+0000 7f5080c00700 1 --2- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f5068039600 0x7f506803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:52:57.866 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.933+0000 7f5080c00700 1 -- 172.21.15.18:0/3335642123 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5074011e00 con 0x7f508410f0c0 2024-04-16T16:52:57.866 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.933+0000 7f5083400700 1 --2- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f5068039600 0x7f506803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.934+0000 7f5083400700 1 --2- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f5068039600 0x7f506803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5070006fd0 tx=0x7f5070006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:56.938+0000 7f5080c00700 1 -- 172.21.15.18:0/3335642123 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f507400e260 con 0x7f508410f0c0 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.168+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f5084080160 con 0x7f5068039600 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.170+0000 7f5080c00700 1 -- 172.21.15.18:0/3335642123 <== mgr.14120 v2:172.21.15.18:6800/1759186913 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f5084080160 con 0x7f5068039600 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.172+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f5068039600 msgr2=0x7f506803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.172+0000 7f508aa00700 1 --2- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f5068039600 0x7f506803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5070006fd0 tx=0x7f5070006e40 comp rx=0 tx=0).stop 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.172+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 msgr2=0x7f50841a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.172+0000 7f508aa00700 1 --2- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f50841a2c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5074011850 tx=0x7f5074011c10 comp rx=0 tx=0).stop 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.172+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 shutdown_connections 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.173+0000 7f508aa00700 1 --2- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f5068039600 0x7f506803bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.173+0000 7f508aa00700 1 --2- 172.21.15.18:0/3335642123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f508410f0c0 0x7f50841a2c80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.173+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 >> 172.21.15.18:0/3335642123 conn(0x7f508407de20 msgr2=0x7f508407fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.173+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 shutdown_connections 2024-04-16T16:52:57.867 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:57.173+0000 7f508aa00700 1 -- 172.21.15.18:0/3335642123 wait complete. 2024-04-16T16:52:57.868 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-16T16:52:57.868 INFO:teuthology.orchestra.run.smithi018.stdout:Adding key to root@localhost authorized_keys... 2024-04-16T16:52:57.868 INFO:teuthology.orchestra.run.smithi018.stdout:Adding host smithi018... 2024-04-16T16:52:59.893 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:52:59 smithi018 ceph-mon[31776]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi018", "addr": "172.21.15.18", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:53:00.671 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:00 smithi018 ceph-mon[31776]: Deploying cephadm binary to smithi018 2024-04-16T16:53:06.296 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Added host 'smithi018' with addr '172.21.15.18' 2024-04-16T16:53:06.296 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.461+0000 7f80d0200700 1 Processor -- start 2024-04-16T16:53:06.297 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.461+0000 7f80d0200700 1 -- start start 2024-04-16T16:53:06.297 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.462+0000 7f80d0200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c810c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:06.297 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.462+0000 7f80d0200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80c810a530 con 0x7f80c810bfa0 2024-04-16T16:53:06.297 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.462+0000 7f80cd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c810c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:06.297 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.463+0000 7f80cd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c810c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50426/0 (socket says 172.21.15.18:50426) 2024-04-16T16:53:06.297 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.463+0000 7f80cd800700 1 -- 172.21.15.18:0/2061131559 learned_addr learned my addr 172.21.15.18:0/2061131559 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:06.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.463+0000 7f80cd800700 1 -- 172.21.15.18:0/2061131559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c810a6b0 con 0x7f80c810bfa0 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.463+0000 7f80cd800700 1 --2- 172.21.15.18:0/2061131559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c810c3c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009e80 tx=0x7f80bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4b2fc97b33c3b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.464+0000 7f80c7e00700 1 -- 172.21.15.18:0/2061131559 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80bc017070 con 0x7f80c810bfa0 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.464+0000 7f80c7e00700 1 -- 172.21.15.18:0/2061131559 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80bc00b8c0 con 0x7f80c810bfa0 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.464+0000 7f80c7e00700 1 -- 172.21.15.18:0/2061131559 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80bc0045c0 con 0x7f80c810bfa0 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.464+0000 7f80d0200700 1 -- 172.21.15.18:0/2061131559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 msgr2=0x7f80c810c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.464+0000 7f80d0200700 1 --2- 172.21.15.18:0/2061131559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c810c3c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009e80 tx=0x7f80bc00b330 comp rx=0 tx=0).stop 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.464+0000 7f80d0200700 1 -- 172.21.15.18:0/2061131559 shutdown_connections 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.465+0000 7f80d0200700 1 --2- 172.21.15.18:0/2061131559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c810c3c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.465+0000 7f80d0200700 1 -- 172.21.15.18:0/2061131559 >> 172.21.15.18:0/2061131559 conn(0x7f80c807dd90 msgr2=0x7f80c81084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.465+0000 7f80d0200700 1 -- 172.21.15.18:0/2061131559 shutdown_connections 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.465+0000 7f80d0200700 1 -- 172.21.15.18:0/2061131559 wait complete. 2024-04-16T16:53:06.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.466+0000 7f80d0200700 1 Processor -- start 2024-04-16T16:53:06.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.466+0000 7f80d0200700 1 -- start start 2024-04-16T16:53:06.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.466+0000 7f80d0200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:06.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.466+0000 7f80d0200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80c810ee70 con 0x7f80c810bfa0 2024-04-16T16:53:06.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.467+0000 7f80cd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:06.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.467+0000 7f80cd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50436/0 (socket says 172.21.15.18:50436) 2024-04-16T16:53:06.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.467+0000 7f80cd800700 1 -- 172.21.15.18:0/2605618137 learned_addr learned my addr 172.21.15.18:0/2605618137 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:06.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.467+0000 7f80cd800700 1 -- 172.21.15.18:0/2605618137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c819ee10 con 0x7f80c810bfa0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.467+0000 7f80cd800700 1 --2- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c819e8d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009530 tx=0x7f80bc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.468+0000 7f80c6000700 1 -- 172.21.15.18:0/2605618137 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80bc020020 con 0x7f80c810bfa0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.468+0000 7f80c6000700 1 -- 172.21.15.18:0/2605618137 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80bc003b50 con 0x7f80c810bfa0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.468+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c819f0a0 con 0x7f80c810bfa0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.468+0000 7f80c6000700 1 -- 172.21.15.18:0/2605618137 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80bc017040 con 0x7f80c810bfa0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.468+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c81a1d90 con 0x7f80c810bfa0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.469+0000 7f80c6000700 1 -- 172.21.15.18:0/2605618137 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f80bc023020 con 0x7f80c810bfa0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.469+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c810eff0 con 0x7f80c810bfa0 2024-04-16T16:53:06.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.469+0000 7f80c6000700 1 --2- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f80b0039600 0x7f80b003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.470+0000 7f80c6000700 1 -- 172.21.15.18:0/2605618137 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f80bc00f050 con 0x7f80c810bfa0 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.470+0000 7f80cce00700 1 --2- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f80b0039600 0x7f80b003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.471+0000 7f80cce00700 1 --2- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f80b0039600 0x7f80b003bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f80b4006fd0 tx=0x7f80b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.476+0000 7f80c6000700 1 -- 172.21.15.18:0/2605618137 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80bc02a020 con 0x7f80c810bfa0 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:52:58.709+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi018", "addr": "172.21.15.18", "target": ["mon-mgr", ""]}) -- 0x7f80c8108330 con 0x7f80b0039600 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.479+0000 7f80c6000700 1 -- 172.21.15.18:0/2605618137 <== mgr.14120 v2:172.21.15.18:6800/1759186913 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f80c8108330 con 0x7f80b0039600 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f80b0039600 msgr2=0x7f80b003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 --2- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f80b0039600 0x7f80b003bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f80b4006fd0 tx=0x7f80b4006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 msgr2=0x7f80c819e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 --2- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c819e8d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009530 tx=0x7f80bc00bd50 comp rx=0 tx=0).stop 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 shutdown_connections 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 --2- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f80b0039600 0x7f80b003bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 --2- 172.21.15.18:0/2605618137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80c810bfa0 0x7f80c819e8d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 >> 172.21.15.18:0/2605618137 conn(0x7f80c807dd90 msgr2=0x7f80c8107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:06.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 shutdown_connections 2024-04-16T16:53:06.306 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:05.483+0000 7f80d0200700 1 -- 172.21.15.18:0/2605618137 wait complete. 2024-04-16T16:53:06.306 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying unmanaged mon service... 2024-04-16T16:53:06.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:06 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:53:06.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:06 smithi018 ceph-mon[31776]: Added host smithi018 2024-04-16T16:53:06.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:06 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:53:07.869 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-16T16:53:07.869 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.909+0000 7f8bb2a00700 1 Processor -- start 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.909+0000 7f8bb2a00700 1 -- start start 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.910+0000 7f8bb2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac07c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.910+0000 7f8bb2a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac07dff0 con 0x7f8bac07c310 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.910+0000 7f8babe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac07c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.911+0000 7f8babe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac07c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43546/0 (socket says 172.21.15.18:43546) 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.911+0000 7f8babe00700 1 -- 172.21.15.18:0/702438503 learned_addr learned my addr 172.21.15.18:0/702438503 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.911+0000 7f8babe00700 1 -- 172.21.15.18:0/702438503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac07cc70 con 0x7f8bac07c310 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.911+0000 7f8babe00700 1 --2- 172.21.15.18:0/702438503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac07c730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0098a0 tx=0x7f8b9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8809c6ae53b667f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.912+0000 7f8bab400700 1 -- 172.21.15.18:0/702438503 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b9c004160 con 0x7f8bac07c310 2024-04-16T16:53:07.870 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.912+0000 7f8bab400700 1 -- 172.21.15.18:0/702438503 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b9c004300 con 0x7f8bac07c310 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.912+0000 7f8bab400700 1 -- 172.21.15.18:0/702438503 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b9c011b50 con 0x7f8bac07c310 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.912+0000 7f8bb2a00700 1 -- 172.21.15.18:0/702438503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 msgr2=0x7f8bac07c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.912+0000 7f8bb2a00700 1 --2- 172.21.15.18:0/702438503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac07c730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0098a0 tx=0x7f8b9c009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.913+0000 7f8bb2a00700 1 -- 172.21.15.18:0/702438503 shutdown_connections 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.913+0000 7f8bb2a00700 1 --2- 172.21.15.18:0/702438503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac07c730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.913+0000 7f8bb2a00700 1 -- 172.21.15.18:0/702438503 >> 172.21.15.18:0/702438503 conn(0x7f8bac103cd0 msgr2=0x7f8bac1060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.913+0000 7f8bb2a00700 1 -- 172.21.15.18:0/702438503 shutdown_connections 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.913+0000 7f8bb2a00700 1 -- 172.21.15.18:0/702438503 wait complete. 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.914+0000 7f8bb2a00700 1 Processor -- start 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.914+0000 7f8bb2a00700 1 -- start start 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.914+0000 7f8bb2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.914+0000 7f8bb2a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac10ed60 con 0x7f8bac07c310 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.915+0000 7f8babe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:07.871 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.915+0000 7f8babe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43550/0 (socket says 172.21.15.18:43550) 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.915+0000 7f8babe00700 1 -- 172.21.15.18:0/3499786887 learned_addr learned my addr 172.21.15.18:0/3499786887 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.915+0000 7f8babe00700 1 -- 172.21.15.18:0/3499786887 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac19ee80 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.915+0000 7f8babe00700 1 --2- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac19e940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c011b60 tx=0x7f8b9c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.916+0000 7f8ba9600700 1 -- 172.21.15.18:0/3499786887 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b9c003970 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.916+0000 7f8ba9600700 1 -- 172.21.15.18:0/3499786887 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b9c003b10 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.916+0000 7f8ba9600700 1 -- 172.21.15.18:0/3499786887 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b9c024610 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.916+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bac19f110 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.916+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bac1a1e00 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.917+0000 7f8ba9600700 1 -- 172.21.15.18:0/3499786887 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8b9c01a9f0 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.917+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bac064150 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.918+0000 7f8ba9600700 1 --2- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f8b90039600 0x7f8b9003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.918+0000 7f8ba9600700 1 -- 172.21.15.18:0/3499786887 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8b9c011e00 con 0x7f8bac07c310 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.918+0000 7f8ba3400700 1 --2- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f8b90039600 0x7f8b9003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.919+0000 7f8ba3400700 1 --2- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f8b90039600 0x7f8b9003bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8b94006fd0 tx=0x7f8b94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:07.872 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:06.923+0000 7f8ba9600700 1 -- 172.21.15.18:0/3499786887 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b9c016260 con 0x7f8bac07c310 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.171+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8bac104f30 con 0x7f8b90039600 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.180+0000 7f8ba9600700 1 -- 172.21.15.18:0/3499786887 <== mgr.14120 v2:172.21.15.18:6800/1759186913 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8bac104f30 con 0x7f8b90039600 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.182+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f8b90039600 msgr2=0x7f8b9003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.182+0000 7f8bb2a00700 1 --2- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f8b90039600 0x7f8b9003bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8b94006fd0 tx=0x7f8b94006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.182+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 msgr2=0x7f8bac19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.182+0000 7f8bb2a00700 1 --2- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac19e940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c011b60 tx=0x7f8b9c011c40 comp rx=0 tx=0).stop 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.183+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 shutdown_connections 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.183+0000 7f8bb2a00700 1 --2- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f8b90039600 0x7f8b9003bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.183+0000 7f8bb2a00700 1 --2- 172.21.15.18:0/3499786887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bac07c310 0x7f8bac19e940 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.183+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 >> 172.21.15.18:0/3499786887 conn(0x7f8bac103cd0 msgr2=0x7f8bac104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.183+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 shutdown_connections 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:07.183+0000 7f8bb2a00700 1 -- 172.21.15.18:0/3499786887 wait complete. 2024-04-16T16:53:07.873 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying unmanaged mgr service... 2024-04-16T16:53:08.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:08 smithi018 ceph-mon[31776]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:53:08.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:08 smithi018 ceph-mon[31776]: Saving service mon spec with placement count:5 2024-04-16T16:53:08.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:08 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:53:10.068 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-16T16:53:10.068 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.815+0000 7f220f200700 1 Processor -- start 2024-04-16T16:53:10.068 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.815+0000 7f220f200700 1 -- start start 2024-04-16T16:53:10.068 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.815+0000 7f220f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220810a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.815+0000 7f220f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f22081082b0 con 0x7f2208109d20 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.816+0000 7f2207e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220810a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.816+0000 7f2207e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220810a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43564/0 (socket says 172.21.15.18:43564) 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.816+0000 7f2207e00700 1 -- 172.21.15.18:0/3150241651 learned_addr learned my addr 172.21.15.18:0/3150241651 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.817+0000 7f2207e00700 1 -- 172.21.15.18:0/3150241651 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2208108430 con 0x7f2208109d20 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.817+0000 7f2207e00700 1 --2- 172.21.15.18:0/3150241651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220810a140 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009e80 tx=0x7f21f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c60aa78cb689c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.817+0000 7f2207400700 1 -- 172.21.15.18:0/3150241651 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f8017070 con 0x7f2208109d20 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.817+0000 7f2207400700 1 -- 172.21.15.18:0/3150241651 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21f800b8c0 con 0x7f2208109d20 2024-04-16T16:53:10.070 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.817+0000 7f2207400700 1 -- 172.21.15.18:0/3150241651 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f80045c0 con 0x7f2208109d20 2024-04-16T16:53:10.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.818+0000 7f220f200700 1 -- 172.21.15.18:0/3150241651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 msgr2=0x7f220810a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:10.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.818+0000 7f220f200700 1 --2- 172.21.15.18:0/3150241651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220810a140 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009e80 tx=0x7f21f800b330 comp rx=0 tx=0).stop 2024-04-16T16:53:10.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.818+0000 7f220f200700 1 -- 172.21.15.18:0/3150241651 shutdown_connections 2024-04-16T16:53:10.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.818+0000 7f220f200700 1 --2- 172.21.15.18:0/3150241651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220810a140 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:10.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.818+0000 7f220f200700 1 -- 172.21.15.18:0/3150241651 >> 172.21.15.18:0/3150241651 conn(0x7f2208103cd0 msgr2=0x7f22081060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:10.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.818+0000 7f220f200700 1 -- 172.21.15.18:0/3150241651 shutdown_connections 2024-04-16T16:53:10.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.819+0000 7f220f200700 1 -- 172.21.15.18:0/3150241651 wait complete. 2024-04-16T16:53:10.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.820+0000 7f220f200700 1 Processor -- start 2024-04-16T16:53:10.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.820+0000 7f220f200700 1 -- start start 2024-04-16T16:53:10.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.820+0000 7f220f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:10.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.820+0000 7f220f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f220810cbf0 con 0x7f2208109d20 2024-04-16T16:53:10.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.820+0000 7f2207e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:10.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.821+0000 7f2207e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43580/0 (socket says 172.21.15.18:43580) 2024-04-16T16:53:10.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.821+0000 7f2207e00700 1 -- 172.21.15.18:0/2448164001 learned_addr learned my addr 172.21.15.18:0/2448164001 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.821+0000 7f2207e00700 1 -- 172.21.15.18:0/2448164001 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220819cc80 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.821+0000 7f2207e00700 1 --2- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220819c740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f21f8004210 tx=0x7f21f800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.821+0000 7f2205600700 1 -- 172.21.15.18:0/2448164001 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f8020020 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.822+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220819cf10 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.822+0000 7f2205600700 1 -- 172.21.15.18:0/2448164001 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21f8003b50 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.822+0000 7f2205600700 1 -- 172.21.15.18:0/2448164001 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f8017040 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.822+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220819fc00 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.823+0000 7f2205600700 1 -- 172.21.15.18:0/2448164001 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f21f8023020 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.823+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2208064150 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.823+0000 7f2205600700 1 --2- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f21ec0395b0 0x7f21ec03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.823+0000 7f2205600700 1 -- 172.21.15.18:0/2448164001 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f21f800f050 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.824+0000 7f21ff400700 1 --2- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f21ec0395b0 0x7f21ec03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.825+0000 7f21ff400700 1 --2- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f21ec0395b0 0x7f21ec03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f21f0006fd0 tx=0x7f21f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:08.830+0000 7f2205600700 1 -- 172.21.15.18:0/2448164001 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21f802a020 con 0x7f2208109d20 2024-04-16T16:53:10.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.087+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2208104eb0 con 0x7f21ec0395b0 2024-04-16T16:53:10.076 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.096+0000 7f2205600700 1 -- 172.21.15.18:0/2448164001 <== mgr.14120 v2:172.21.15.18:6800/1759186913 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2208104eb0 con 0x7f21ec0395b0 2024-04-16T16:53:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.099+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f21ec0395b0 msgr2=0x7f21ec03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.099+0000 7f220f200700 1 --2- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f21ec0395b0 0x7f21ec03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f21f0006fd0 tx=0x7f21f0006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.099+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 msgr2=0x7f220819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.099+0000 7f220f200700 1 --2- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220819c740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f21f8004210 tx=0x7f21f800bd50 comp rx=0 tx=0).stop 2024-04-16T16:53:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.100+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 shutdown_connections 2024-04-16T16:53:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.100+0000 7f220f200700 1 --2- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f21ec0395b0 0x7f21ec03ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.100+0000 7f220f200700 1 --2- 172.21.15.18:0/2448164001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2208109d20 0x7f220819c740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:10.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.100+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 >> 172.21.15.18:0/2448164001 conn(0x7f2208103cd0 msgr2=0x7f22081047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:10.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.100+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 shutdown_connections 2024-04-16T16:53:10.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:09.100+0000 7f220f200700 1 -- 172.21.15.18:0/2448164001 wait complete. 2024-04-16T16:53:10.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:10 smithi018 ceph-mon[31776]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:53:10.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:10 smithi018 ceph-mon[31776]: Saving service mgr spec with placement count:2 2024-04-16T16:53:10.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:10 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:53:10.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:10 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:53:11.647 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:11 smithi018 ceph-mon[31776]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:11.647 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:11 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:53:11.648 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:11 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2764927973' entity='client.admin' 2024-04-16T16:53:11.664 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.688+0000 7fda1c800700 1 Processor -- start 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.688+0000 7fda1c800700 1 -- start start 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.689+0000 7fda1c800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1410a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.689+0000 7fda1c800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fda14108350 con 0x7fda14109dc0 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.689+0000 7fda19e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1410a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.690+0000 7fda19e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1410a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43584/0 (socket says 172.21.15.18:43584) 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.690+0000 7fda19e00700 1 -- 172.21.15.18:0/683482497 learned_addr learned my addr 172.21.15.18:0/683482497 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.690+0000 7fda19e00700 1 -- 172.21.15.18:0/683482497 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda141084d0 con 0x7fda14109dc0 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.690+0000 7fda19e00700 1 --2- 172.21.15.18:0/683482497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1410a1e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda080098a0 tx=0x7fda08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4187746cc1bc76ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:11.665 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.691+0000 7fda18a00700 1 -- 172.21.15.18:0/683482497 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda08004160 con 0x7fda14109dc0 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.691+0000 7fda18a00700 1 -- 172.21.15.18:0/683482497 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda08004300 con 0x7fda14109dc0 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.691+0000 7fda18a00700 1 -- 172.21.15.18:0/683482497 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda08011b20 con 0x7fda14109dc0 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.691+0000 7fda1c800700 1 -- 172.21.15.18:0/683482497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 msgr2=0x7fda1410a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.691+0000 7fda1c800700 1 --2- 172.21.15.18:0/683482497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1410a1e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda080098a0 tx=0x7fda08009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.692+0000 7fda1c800700 1 -- 172.21.15.18:0/683482497 shutdown_connections 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.692+0000 7fda1c800700 1 --2- 172.21.15.18:0/683482497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1410a1e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.692+0000 7fda1c800700 1 -- 172.21.15.18:0/683482497 >> 172.21.15.18:0/683482497 conn(0x7fda14103d30 msgr2=0x7fda14106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.692+0000 7fda1c800700 1 -- 172.21.15.18:0/683482497 shutdown_connections 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.692+0000 7fda1c800700 1 -- 172.21.15.18:0/683482497 wait complete. 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.693+0000 7fda1c800700 1 Processor -- start 2024-04-16T16:53:11.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.693+0000 7fda1c800700 1 -- start start 2024-04-16T16:53:11.667 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.693+0000 7fda1c800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:11.667 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.693+0000 7fda1c800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fda1410cc90 con 0x7fda14109dc0 2024-04-16T16:53:11.667 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.694+0000 7fda19e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:11.667 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.694+0000 7fda19e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43600/0 (socket says 172.21.15.18:43600) 2024-04-16T16:53:11.667 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.694+0000 7fda19e00700 1 -- 172.21.15.18:0/2764927973 learned_addr learned my addr 172.21.15.18:0/2764927973 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:11.667 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.694+0000 7fda19e00700 1 -- 172.21.15.18:0/2764927973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1419ee80 con 0x7fda14109dc0 2024-04-16T16:53:11.667 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.694+0000 7fda19e00700 1 --2- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1419e940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda08000c00 tx=0x7fda08011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.695+0000 7fda0ea00700 1 -- 172.21.15.18:0/2764927973 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda08003b60 con 0x7fda14109dc0 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.695+0000 7fda0ea00700 1 -- 172.21.15.18:0/2764927973 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda0801a910 con 0x7fda14109dc0 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.695+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1419f110 con 0x7fda14109dc0 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.695+0000 7fda0ea00700 1 -- 172.21.15.18:0/2764927973 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda08024930 con 0x7fda14109dc0 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.695+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1419f570 con 0x7fda14109dc0 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.696+0000 7fda0ea00700 1 -- 172.21.15.18:0/2764927973 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fda08022020 con 0x7fda14109dc0 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.696+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1410ce10 con 0x7fda14109dc0 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.697+0000 7fda0ea00700 1 --2- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7fd9fc0395b0 0x7fd9fc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:11.668 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.697+0000 7fda0ea00700 1 -- 172.21.15.18:0/2764927973 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fda0804d8f0 con 0x7fda14109dc0 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.697+0000 7fda19400700 1 --2- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7fd9fc0395b0 0x7fd9fc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.698+0000 7fda19400700 1 --2- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7fd9fc0395b0 0x7fd9fc03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fda04006fd0 tx=0x7fda04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.702+0000 7fda0ea00700 1 -- 172.21.15.18:0/2764927973 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda0800b3e0 con 0x7fda14109dc0 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.934+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fda14064150 con 0x7fda14109dc0 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.940+0000 7fda0ea00700 1 -- 172.21.15.18:0/2764927973 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fda0801aac0 con 0x7fda14109dc0 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.942+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7fd9fc0395b0 msgr2=0x7fd9fc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.943+0000 7fda1c800700 1 --2- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7fd9fc0395b0 0x7fd9fc03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fda04006fd0 tx=0x7fda04006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.943+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 msgr2=0x7fda1419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:11.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.943+0000 7fda1c800700 1 --2- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1419e940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda08000c00 tx=0x7fda08011c10 comp rx=0 tx=0).stop 2024-04-16T16:53:11.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.943+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 shutdown_connections 2024-04-16T16:53:11.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.943+0000 7fda1c800700 1 --2- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7fd9fc0395b0 0x7fd9fc03ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:11.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.943+0000 7fda1c800700 1 --2- 172.21.15.18:0/2764927973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda14109dc0 0x7fda1419e940 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:11.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.943+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 >> 172.21.15.18:0/2764927973 conn(0x7fda14103d30 msgr2=0x7fda14104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:11.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.943+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 shutdown_connections 2024-04-16T16:53:11.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:10.944+0000 7fda1c800700 1 -- 172.21.15.18:0/2764927973 wait complete. 2024-04-16T16:53:13.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:13 smithi018 ceph-mon[31776]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:13.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:13 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1820505441' entity='client.admin' 2024-04-16T16:53:13.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:13 smithi018 ceph-mon[31776]: from='mgr.14120 172.21.15.18:0/1621546613' entity='mgr.x' 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.253+0000 7f4fa7e00700 1 Processor -- start 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.253+0000 7f4fa7e00700 1 -- start start 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.253+0000 7f4fa7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa010a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.253+0000 7f4fa7e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01082b0 con 0x7f4fa0109d20 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.254+0000 7f4fa5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa010a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.254+0000 7f4fa5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa010a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43608/0 (socket says 172.21.15.18:43608) 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.254+0000 7f4fa5400700 1 -- 172.21.15.18:0/2795135782 learned_addr learned my addr 172.21.15.18:0/2795135782 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.254+0000 7f4fa5400700 1 -- 172.21.15.18:0/2795135782 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa0108430 con 0x7f4fa0109d20 2024-04-16T16:53:13.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.255+0000 7f4fa5400700 1 --2- 172.21.15.18:0/2795135782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa010a140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f940098a0 tx=0x7f4f94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f50ae05b69d7e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.255+0000 7f4f9fe00700 1 -- 172.21.15.18:0/2795135782 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f94004160 con 0x7f4fa0109d20 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.255+0000 7f4f9fe00700 1 -- 172.21.15.18:0/2795135782 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f94004300 con 0x7f4fa0109d20 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.255+0000 7f4f9fe00700 1 -- 172.21.15.18:0/2795135782 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f94011b20 con 0x7f4fa0109d20 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.256+0000 7f4fa7e00700 1 -- 172.21.15.18:0/2795135782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 msgr2=0x7f4fa010a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.256+0000 7f4fa7e00700 1 --2- 172.21.15.18:0/2795135782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa010a140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f940098a0 tx=0x7f4f94009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.256+0000 7f4fa7e00700 1 -- 172.21.15.18:0/2795135782 shutdown_connections 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.256+0000 7f4fa7e00700 1 --2- 172.21.15.18:0/2795135782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa010a140 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.256+0000 7f4fa7e00700 1 -- 172.21.15.18:0/2795135782 >> 172.21.15.18:0/2795135782 conn(0x7f4fa0103cd0 msgr2=0x7f4fa01060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.256+0000 7f4fa7e00700 1 -- 172.21.15.18:0/2795135782 shutdown_connections 2024-04-16T16:53:13.491 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.256+0000 7f4fa7e00700 1 -- 172.21.15.18:0/2795135782 wait complete. 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.257+0000 7f4fa7e00700 1 Processor -- start 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.257+0000 7f4fa7e00700 1 -- start start 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.257+0000 7f4fa7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.257+0000 7f4fa7e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa010cbf0 con 0x7f4fa0109d20 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.258+0000 7f4fa5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.258+0000 7f4fa5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43616/0 (socket says 172.21.15.18:43616) 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.258+0000 7f4fa5400700 1 -- 172.21.15.18:0/1820505441 learned_addr learned my addr 172.21.15.18:0/1820505441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.258+0000 7f4fa5400700 1 -- 172.21.15.18:0/1820505441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa019ee70 con 0x7f4fa0109d20 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.258+0000 7f4fa5400700 1 --2- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa019e930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f94000c00 tx=0x7f4f94011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.259+0000 7f4f9e000700 1 -- 172.21.15.18:0/1820505441 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f94003b60 con 0x7f4fa0109d20 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.259+0000 7f4f9e000700 1 -- 172.21.15.18:0/1820505441 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f9401a910 con 0x7f4fa0109d20 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.259+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa019f100 con 0x7f4fa0109d20 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.259+0000 7f4f9e000700 1 -- 172.21.15.18:0/1820505441 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f94024930 con 0x7f4fa0109d20 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.259+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa019f560 con 0x7f4fa0109d20 2024-04-16T16:53:13.492 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.260+0000 7f4f9e000700 1 -- 172.21.15.18:0/1820505441 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4f94022020 con 0x7f4fa0109d20 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.260+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa0064150 con 0x7f4fa0109d20 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.261+0000 7f4f9e000700 1 --2- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f4f88039660 0x7f4f8803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.261+0000 7f4f9e000700 1 -- 172.21.15.18:0/1820505441 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f9404d930 con 0x7f4fa0109d20 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.261+0000 7f4fa4a00700 1 --2- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f4f88039660 0x7f4f8803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.262+0000 7f4fa4a00700 1 --2- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f4f88039660 0x7f4f8803bb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006fd0 tx=0x7f4f8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.266+0000 7f4f9e000700 1 -- 172.21.15.18:0/1820505441 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f9400f2e0 con 0x7f4fa0109d20 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.496+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4fa01a1df0 con 0x7f4fa0109d20 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.502+0000 7f4f9e000700 1 -- 172.21.15.18:0/1820505441 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4fa01a1df0 con 0x7f4fa0109d20 2024-04-16T16:53:13.493 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.505+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f4f88039660 msgr2=0x7f4f8803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:13.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.505+0000 7f4fa7e00700 1 --2- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f4f88039660 0x7f4f8803bb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006fd0 tx=0x7f4f8c006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:13.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.505+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 msgr2=0x7f4fa019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:13.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.505+0000 7f4fa7e00700 1 --2- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa019e930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f94000c00 tx=0x7f4f94011c10 comp rx=0 tx=0).stop 2024-04-16T16:53:13.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.505+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 shutdown_connections 2024-04-16T16:53:13.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.505+0000 7f4fa7e00700 1 --2- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f4f88039660 0x7f4f8803bb20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:13.494 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.505+0000 7f4fa7e00700 1 --2- 172.21.15.18:0/1820505441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fa0109d20 0x7f4fa019e930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:13.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.505+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 >> 172.21.15.18:0/1820505441 conn(0x7f4fa0103cd0 msgr2=0x7f4fa0104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:13.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.506+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 shutdown_connections 2024-04-16T16:53:13.495 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:12.506+0000 7f4fa7e00700 1 -- 172.21.15.18:0/1820505441 wait complete. 2024-04-16T16:53:13.495 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling the dashboard module... 2024-04-16T16:53:15.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:15 smithi018 ceph-mon[31776]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:15.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:15 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3581226598' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-16T16:53:15.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:15 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: ignoring --setuser ceph since I am not root 2024-04-16T16:53:15.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:15 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: ignoring --setgroup ceph since I am not root 2024-04-16T16:53:16.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.087+0000 7f7710600700 1 Processor -- start 2024-04-16T16:53:16.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.088+0000 7f7710600700 1 -- start start 2024-04-16T16:53:16.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.088+0000 7f7710600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770807c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:16.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.088+0000 7f7710600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f770807dff0 con 0x7f770807c310 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.088+0000 7f770dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770807c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.089+0000 7f770dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770807c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43626/0 (socket says 172.21.15.18:43626) 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.089+0000 7f770dc00700 1 -- 172.21.15.18:0/572838468 learned_addr learned my addr 172.21.15.18:0/572838468 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.089+0000 7f770dc00700 1 -- 172.21.15.18:0/572838468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770807cc70 con 0x7f770807c310 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.089+0000 7f770dc00700 1 --2- 172.21.15.18:0/572838468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770807c730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0098a0 tx=0x7f76fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4ddab5056d3c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.089+0000 7f7707e00700 1 -- 172.21.15.18:0/572838468 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc004160 con 0x7f770807c310 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.090+0000 7f7707e00700 1 -- 172.21.15.18:0/572838468 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc004300 con 0x7f770807c310 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.090+0000 7f7707e00700 1 -- 172.21.15.18:0/572838468 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc011b20 con 0x7f770807c310 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.090+0000 7f7710600700 1 -- 172.21.15.18:0/572838468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 msgr2=0x7f770807c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.090+0000 7f7710600700 1 --2- 172.21.15.18:0/572838468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770807c730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0098a0 tx=0x7f76fc009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:16.078 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.091+0000 7f7710600700 1 -- 172.21.15.18:0/572838468 shutdown_connections 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.091+0000 7f7710600700 1 --2- 172.21.15.18:0/572838468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770807c730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.091+0000 7f7710600700 1 -- 172.21.15.18:0/572838468 >> 172.21.15.18:0/572838468 conn(0x7f7708103cd0 msgr2=0x7f77081060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.091+0000 7f7710600700 1 -- 172.21.15.18:0/572838468 shutdown_connections 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.091+0000 7f7710600700 1 -- 172.21.15.18:0/572838468 wait complete. 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.092+0000 7f7710600700 1 Processor -- start 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.092+0000 7f7710600700 1 -- start start 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.093+0000 7f7710600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.093+0000 7f7710600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f770810ed50 con 0x7f770807c310 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.093+0000 7f770dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.093+0000 7f770dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43636/0 (socket says 172.21.15.18:43636) 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.093+0000 7f770dc00700 1 -- 172.21.15.18:0/3581226598 learned_addr learned my addr 172.21.15.18:0/3581226598 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:16.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.093+0000 7f770dc00700 1 -- 172.21.15.18:0/3581226598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770819edf0 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.094+0000 7f770dc00700 1 --2- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770819e8b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76fc011b60 tx=0x7f76fc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.094+0000 7f7706000700 1 -- 172.21.15.18:0/3581226598 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc003b60 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.094+0000 7f7706000700 1 -- 172.21.15.18:0/3581226598 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc01a910 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.094+0000 7f7706000700 1 -- 172.21.15.18:0/3581226598 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc024930 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.094+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770819f080 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.094+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770819f4e0 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.095+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7708064150 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.096+0000 7f7706000700 1 -- 172.21.15.18:0/3581226598 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f76fc022020 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.096+0000 7f7706000700 1 --2- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f76f0039600 0x7f76f003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.096+0000 7f7706000700 1 -- 172.21.15.18:0/3581226598 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f76fc04d960 con 0x7f770807c310 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.096+0000 7f770d200700 1 --2- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f76f0039600 0x7f76f003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.097+0000 7f770d200700 1 --2- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f76f0039600 0x7f76f003bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f76f4006fd0 tx=0x7f76f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:16.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.102+0000 7f7706000700 1 -- 172.21.15.18:0/3581226598 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76fc00b3e0 con 0x7f770807c310 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:14.422+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f77081a1e30 con 0x7f770807c310 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.383+0000 7f7706000700 1 -- 172.21.15.18:0/3581226598 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f76fc04d2b0 con 0x7f770807c310 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.387+0000 7f7706000700 1 -- 172.21.15.18:0/3581226598 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f76fc024ad0 con 0x7f770807c310 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.389+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f76f0039600 msgr2=0x7f76f003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.389+0000 7f7710600700 1 --2- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f76f0039600 0x7f76f003bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f76f4006fd0 tx=0x7f76f4006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.389+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 msgr2=0x7f770819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.389+0000 7f7710600700 1 --2- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770819e8b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76fc011b60 tx=0x7f76fc011c40 comp rx=0 tx=0).stop 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.389+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 shutdown_connections 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.389+0000 7f7710600700 1 --2- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f76f0039600 0x7f76f003bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.389+0000 7f7710600700 1 --2- 172.21.15.18:0/3581226598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f770807c310 0x7f770819e8b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.389+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 >> 172.21.15.18:0/3581226598 conn(0x7f7708103cd0 msgr2=0x7f7708104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.390+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 shutdown_connections 2024-04-16T16:53:16.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:15.391+0000 7f7710600700 1 -- 172.21.15.18:0/3581226598 wait complete. 2024-04-16T16:53:16.391 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:16 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:16.262+0000 7f38c0598200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:53:16.391 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:16 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:16.389+0000 7f38c0598200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:53:16.688 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:16 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3581226598' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-16T16:53:16.688 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:16 smithi018 ceph-mon[31776]: mgrmap e9: x(active, since 25s) 2024-04-16T16:53:17.356 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:17.227+0000 7f38c0598200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:53:17.659 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:17 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2433295233' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-16T16:53:17.659 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:17.363+0000 7f38c0598200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:53:17.659 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:17.570+0000 7f38c0598200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:53:17.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:53:17.683 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-16T16:53:17.683 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T16:53:17.683 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-16T16:53:17.683 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-16T16:53:17.683 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:53:17.683 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.686+0000 7f443d400700 1 Processor -- start 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.686+0000 7f443d400700 1 -- start start 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.687+0000 7f443d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443807c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.687+0000 7f443d400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f443807dff0 con 0x7f443807c310 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.687+0000 7f4436800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443807c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.688+0000 7f4436800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443807c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43670/0 (socket says 172.21.15.18:43670) 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.688+0000 7f4436800700 1 -- 172.21.15.18:0/2436396655 learned_addr learned my addr 172.21.15.18:0/2436396655 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.688+0000 7f4436800700 1 -- 172.21.15.18:0/2436396655 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443807cc70 con 0x7f443807c310 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.688+0000 7f4436800700 1 --2- 172.21.15.18:0/2436396655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443807c730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f44280098a0 tx=0x7f4428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf7184ad0dccca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.689+0000 7f4435400700 1 -- 172.21.15.18:0/2436396655 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4428004160 con 0x7f443807c310 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.689+0000 7f4435400700 1 -- 172.21.15.18:0/2436396655 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4428004300 con 0x7f443807c310 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.689+0000 7f4435400700 1 -- 172.21.15.18:0/2436396655 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4428011b20 con 0x7f443807c310 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.689+0000 7f443d400700 1 -- 172.21.15.18:0/2436396655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 msgr2=0x7f443807c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.689+0000 7f443d400700 1 --2- 172.21.15.18:0/2436396655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443807c730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f44280098a0 tx=0x7f4428009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:17.684 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.690+0000 7f443d400700 1 -- 172.21.15.18:0/2436396655 shutdown_connections 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.690+0000 7f443d400700 1 --2- 172.21.15.18:0/2436396655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443807c730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.690+0000 7f443d400700 1 -- 172.21.15.18:0/2436396655 >> 172.21.15.18:0/2436396655 conn(0x7f4438103cd0 msgr2=0x7f44381060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.690+0000 7f443d400700 1 -- 172.21.15.18:0/2436396655 shutdown_connections 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.690+0000 7f443d400700 1 -- 172.21.15.18:0/2436396655 wait complete. 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.691+0000 7f443d400700 1 Processor -- start 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.691+0000 7f443d400700 1 -- start start 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.691+0000 7f443d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.691+0000 7f443d400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f443810ed50 con 0x7f443807c310 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.691+0000 7f4436800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.691+0000 7f4436800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43686/0 (socket says 172.21.15.18:43686) 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.692+0000 7f4436800700 1 -- 172.21.15.18:0/2433295233 learned_addr learned my addr 172.21.15.18:0/2433295233 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.692+0000 7f4436800700 1 -- 172.21.15.18:0/2433295233 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443819edf0 con 0x7f443807c310 2024-04-16T16:53:17.685 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.692+0000 7f4436800700 1 --2- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443819e8b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4428011b60 tx=0x7f4428011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.692+0000 7f442f400700 1 -- 172.21.15.18:0/2433295233 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4428003b60 con 0x7f443807c310 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.693+0000 7f442f400700 1 -- 172.21.15.18:0/2433295233 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f442801a910 con 0x7f443807c310 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.693+0000 7f442f400700 1 -- 172.21.15.18:0/2433295233 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4428024930 con 0x7f443807c310 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.693+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443819f080 con 0x7f443807c310 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.693+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443819f4e0 con 0x7f443807c310 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.694+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438064150 con 0x7f443807c310 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.694+0000 7f442f400700 1 -- 172.21.15.18:0/2433295233 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f4428022020 con 0x7f443807c310 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.695+0000 7f442f400700 1 --2- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f441c039660 0x7f441c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.695+0000 7f442f400700 1 -- 172.21.15.18:0/2433295233 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f442804d9e0 con 0x7f443807c310 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.695+0000 7f4435e00700 1 -- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f441c039660 msgr2=0x7f441c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1759186913 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.695+0000 7f4435e00700 1 --2- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f441c039660 0x7f441c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:53:17.686 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.700+0000 7f442f400700 1 -- 172.21.15.18:0/2433295233 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f442800b3e0 con 0x7f443807c310 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.896+0000 7f4435e00700 1 -- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f441c039660 msgr2=0x7f441c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1759186913 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.896+0000 7f4435e00700 1 --2- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f441c039660 0x7f441c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.988+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f44381a1d70 con 0x7f443807c310 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.989+0000 7f442f400700 1 -- 172.21.15.18:0/2433295233 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f44381a1d70 con 0x7f443807c310 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.991+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f441c039660 msgr2=0x7f441c03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.991+0000 7f443d400700 1 --2- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f441c039660 0x7f441c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.991+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 msgr2=0x7f443819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.991+0000 7f443d400700 1 --2- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443819e8b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4428011b60 tx=0x7f4428011c40 comp rx=0 tx=0).stop 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.991+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 shutdown_connections 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.991+0000 7f443d400700 1 --2- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f441c039660 0x7f441c03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.991+0000 7f443d400700 1 --2- 172.21.15.18:0/2433295233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f443807c310 0x7f443819e8b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.991+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 >> 172.21.15.18:0/2433295233 conn(0x7f4438103cd0 msgr2=0x7f4438104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.992+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 shutdown_connections 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:16.992+0000 7f443d400700 1 -- 172.21.15.18:0/2433295233 wait complete. 2024-04-16T16:53:17.687 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for the mgr to restart... 2024-04-16T16:53:17.688 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr epoch 9... 2024-04-16T16:53:17.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:17.713+0000 7f38c0598200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:53:17.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:17.833+0000 7f38c0598200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:53:18.269 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:18.184+0000 7f38c0598200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:53:18.583 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:18.334+0000 7f38c0598200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:53:18.583 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:18.452+0000 7f38c0598200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:53:18.583 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:18.580+0000 7f38c0598200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:53:18.881 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:18.807+0000 7f38c0598200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:53:20.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:20.144+0000 7f38c0598200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:53:20.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:20.325+0000 7f38c0598200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:53:21.154 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:20.757+0000 7f38c0598200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:53:21.154 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:20.882+0000 7f38c0598200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:53:21.419 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:21.151+0000 7f38c0598200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:53:21.420 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:21.416+0000 7f38c0598200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:53:22.162 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:21.827+0000 7f38c0598200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:53:22.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:22.160+0000 7f38c0598200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:53:22.760 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:22.483+0000 7f38c0598200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:53:23.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:23 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:23.567+0000 7f38c0598200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:53:24.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:24.292+0000 7f38c0598200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:53:24.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:24.428+0000 7f38c0598200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:53:24.728+0000 7f38c0598200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: Active manager daemon x restarted 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: Activating manager daemon x 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: osdmap e3: 0 total, 0 up, 0 in 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: mgrmap e10: x(active, starting, since 0.00760673s) 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T16:53:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T16:53:25.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T16:53:25.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:24 smithi018 ceph-mon[31776]: Manager daemon x is now available 2024-04-16T16:53:26.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:53:26.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T16:53:26.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T16:53:26.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:25 smithi018 ceph-mon[31776]: mgrmap e11: x(active, since 1.00995s) 2024-04-16T16:53:26.572 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.267+0000 7f19c3e00700 1 Processor -- start 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.267+0000 7f19c3e00700 1 -- start start 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.267+0000 7f19c3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.267+0000 7f19c3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f19c407c830 con 0x7f19c407de40 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.268+0000 7f19c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.268+0000 7f19c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60032/0 (socket says 172.21.15.18:60032) 2024-04-16T16:53:26.573 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.268+0000 7f19c2a00700 1 -- 172.21.15.18:0/3743549547 learned_addr learned my addr 172.21.15.18:0/3743549547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.269+0000 7f19c2a00700 1 -- 172.21.15.18:0/3743549547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c407c9b0 con 0x7f19c407de40 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.269+0000 7f19c2a00700 1 --2- 172.21.15.18:0/3743549547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c407c2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19b40098a0 tx=0x7f19b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12948c9d6312b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.269+0000 7f19c1600700 1 -- 172.21.15.18:0/3743549547 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19b4004160 con 0x7f19c407de40 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.269+0000 7f19c1600700 1 -- 172.21.15.18:0/3743549547 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b4004300 con 0x7f19c407de40 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.269+0000 7f19c1600700 1 -- 172.21.15.18:0/3743549547 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19b4011b20 con 0x7f19c407de40 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.270+0000 7f19c3e00700 1 -- 172.21.15.18:0/3743549547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 msgr2=0x7f19c407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.270+0000 7f19c3e00700 1 --2- 172.21.15.18:0/3743549547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c407c2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19b40098a0 tx=0x7f19b4009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.270+0000 7f19c3e00700 1 -- 172.21.15.18:0/3743549547 shutdown_connections 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.270+0000 7f19c3e00700 1 --2- 172.21.15.18:0/3743549547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c407c2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:26.574 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.270+0000 7f19c3e00700 1 -- 172.21.15.18:0/3743549547 >> 172.21.15.18:0/3743549547 conn(0x7f19c4103ab0 msgr2=0x7f19c4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.270+0000 7f19c3e00700 1 -- 172.21.15.18:0/3743549547 shutdown_connections 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.270+0000 7f19c3e00700 1 -- 172.21.15.18:0/3743549547 wait complete. 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.271+0000 7f19c3e00700 1 Processor -- start 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.272+0000 7f19c3e00700 1 -- start start 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.272+0000 7f19c3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.272+0000 7f19c3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f19c410ea50 con 0x7f19c407de40 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.272+0000 7f19c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.272+0000 7f19c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60046/0 (socket says 172.21.15.18:60046) 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.272+0000 7f19c2a00700 1 -- 172.21.15.18:0/2025281456 learned_addr learned my addr 172.21.15.18:0/2025281456 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.273+0000 7f19c2a00700 1 -- 172.21.15.18:0/2025281456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c419eac0 con 0x7f19c407de40 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.273+0000 7f19c2a00700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c419e580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f19b40117e0 tx=0x7f19b4011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.273+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19b401a430 con 0x7f19c407de40 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.273+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b401aab0 con 0x7f19c407de40 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.273+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c419ed50 con 0x7f19c407de40 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.273+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19b4024a60 con 0x7f19c407de40 2024-04-16T16:53:26.575 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.274+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c419f1b0 con 0x7f19c407de40 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.275+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f19b4024d10 con 0x7f19c407de40 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.275+0000 7f19bb400700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.275+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 --> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19a803c1d0 con 0x7f19a8039600 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.275+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f19b404dee0 con 0x7f19c407de40 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.275+0000 7f19c2000700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 msgr2=0x7f19a803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1759186913 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.276+0000 7f19c2000700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.477+0000 7f19c2000700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 msgr2=0x7f19a803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1759186913 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.477+0000 7f19c2000700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.878+0000 7f19c2000700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 msgr2=0x7f19a803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1759186913 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:18.878+0000 7f19c2000700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:19.679+0000 7f19c2000700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 msgr2=0x7f19a803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1759186913 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:19.679+0000 7f19c2000700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:21.281+0000 7f19c2000700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 msgr2=0x7f19a803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1759186913 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:21.281+0000 7f19c2000700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-16T16:53:26.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:24.485+0000 7f19c2000700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 msgr2=0x7f19a803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1759186913 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:24.485+0000 7f19c2000700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:24.736+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f19b4022020 con 0x7f19c407de40 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:24.736+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 msgr2=0x7f19a803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:24.736+0000 7f19bb400700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.739+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f19b404f3c0 con 0x7f19c407de40 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.739+0000 7f19bb400700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19a803cfe0 0x7f19a803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.739+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19b402be50 con 0x7f19a803cfe0 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.741+0000 7f19c2000700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19a803cfe0 0x7f19a803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.742+0000 7f19c2000700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19a803cfe0 0x7f19a803f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19b0003a10 tx=0x7f19b00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.743+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f19b402be50 con 0x7f19a803cfe0 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.751+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f19c4069c40 con 0x7f19a803cfe0 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.751+0000 7f19bb400700 1 -- 172.21.15.18:0/2025281456 <== mgr.14152 v2:172.21.15.18:6800/2462414912 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f19c4069c40 con 0x7f19a803cfe0 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19a803cfe0 msgr2=0x7f19a803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19a803cfe0 0x7f19a803f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19b0003a10 tx=0x7f19b00092b0 comp rx=0 tx=0).stop 2024-04-16T16:53:26.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 msgr2=0x7f19c419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c419e580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f19b40117e0 tx=0x7f19b4011ce0 comp rx=0 tx=0).stop 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 shutdown_connections 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19c407de40 0x7f19c419e580 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19a803cfe0 0x7f19a803f3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 --2- 172.21.15.18:0/2025281456 >> [v2:172.21.15.18:6800/1759186913,v1:172.21.15.18:6801/1759186913] conn(0x7f19a8039600 0x7f19a803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 >> 172.21.15.18:0/2025281456 conn(0x7f19c4103ab0 msgr2=0x7f19c4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 shutdown_connections 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:25.752+0000 7f19c3e00700 1 -- 172.21.15.18:0/2025281456 wait complete. 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:mgr epoch 9 is available 2024-04-16T16:53:26.578 INFO:teuthology.orchestra.run.smithi018.stdout:Generating a dashboard self-signed certificate... 2024-04-16T16:53:27.035 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:26 smithi018 ceph-mon[31776]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-16T16:53:27.035 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:26 smithi018 ceph-mon[31776]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-16T16:53:28.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:27 smithi018 ceph-mon[31776]: mgrmap e12: x(active, since 2s) 2024-04-16T16:53:28.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:27 smithi018 ceph-mon[31776]: [16/Apr/2024:16:53:26] ENGINE Bus STARTING 2024-04-16T16:53:28.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:27 smithi018 ceph-mon[31776]: [16/Apr/2024:16:53:27] ENGINE Serving on https://172.21.15.18:7150 2024-04-16T16:53:28.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:27 smithi018 ceph-mon[31776]: [16/Apr/2024:16:53:27] ENGINE Serving on http://172.21.15.18:8765 2024-04-16T16:53:28.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:27 smithi018 ceph-mon[31776]: [16/Apr/2024:16:53:27] ENGINE Bus STARTED 2024-04-16T16:53:28.054 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:27 smithi018 ceph-mon[31776]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:53:28.054 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:27 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:28.054 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:27 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:28.669 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-16T16:53:28.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.183+0000 7f0b08c00700 1 Processor -- start 2024-04-16T16:53:28.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.183+0000 7f0b08c00700 1 -- start start 2024-04-16T16:53:28.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.184+0000 7f0b08c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0407c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:28.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.184+0000 7f0b08c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0407dff0 con 0x7f0b0407c310 2024-04-16T16:53:28.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.184+0000 7f0b01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0407c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:28.670 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.184+0000 7f0b01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0407c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43446/0 (socket says 172.21.15.18:43446) 2024-04-16T16:53:28.671 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.184+0000 7f0b01e00700 1 -- 172.21.15.18:0/1302898271 learned_addr learned my addr 172.21.15.18:0/1302898271 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:28.671 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.185+0000 7f0b01e00700 1 -- 172.21.15.18:0/1302898271 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0407cc70 con 0x7f0b0407c310 2024-04-16T16:53:28.671 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.185+0000 7f0b01e00700 1 --2- 172.21.15.18:0/1302898271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0407c730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0af40098a0 tx=0x7f0af4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e221e8c2b7ec89d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:28.675 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.185+0000 7f0b00a00700 1 -- 172.21.15.18:0/1302898271 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af4004160 con 0x7f0b0407c310 2024-04-16T16:53:28.675 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.185+0000 7f0b00a00700 1 -- 172.21.15.18:0/1302898271 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af4004300 con 0x7f0b0407c310 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.185+0000 7f0b00a00700 1 -- 172.21.15.18:0/1302898271 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af4011b20 con 0x7f0b0407c310 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.186+0000 7f0b08c00700 1 -- 172.21.15.18:0/1302898271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 msgr2=0x7f0b0407c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.186+0000 7f0b08c00700 1 --2- 172.21.15.18:0/1302898271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0407c730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0af40098a0 tx=0x7f0af4009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.186+0000 7f0b08c00700 1 -- 172.21.15.18:0/1302898271 shutdown_connections 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.186+0000 7f0b08c00700 1 --2- 172.21.15.18:0/1302898271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0407c730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.186+0000 7f0b08c00700 1 -- 172.21.15.18:0/1302898271 >> 172.21.15.18:0/1302898271 conn(0x7f0b04103cd0 msgr2=0x7f0b041060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.186+0000 7f0b08c00700 1 -- 172.21.15.18:0/1302898271 shutdown_connections 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.186+0000 7f0b08c00700 1 -- 172.21.15.18:0/1302898271 wait complete. 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.187+0000 7f0b08c00700 1 Processor -- start 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.188+0000 7f0b08c00700 1 -- start start 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.188+0000 7f0b08c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.188+0000 7f0b08c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410ed50 con 0x7f0b0407c310 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.188+0000 7f0b01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:28.676 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.188+0000 7f0b01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43450/0 (socket says 172.21.15.18:43450) 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.188+0000 7f0b01e00700 1 -- 172.21.15.18:0/98481069 learned_addr learned my addr 172.21.15.18:0/98481069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.189+0000 7f0b01e00700 1 -- 172.21.15.18:0/98481069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0419edf0 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.189+0000 7f0b01e00700 1 --2- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0419e8b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0af4011b60 tx=0x7f0af4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.189+0000 7f0afaa00700 1 -- 172.21.15.18:0/98481069 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af4003b60 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.189+0000 7f0afaa00700 1 -- 172.21.15.18:0/98481069 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af401a910 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.189+0000 7f0afaa00700 1 -- 172.21.15.18:0/98481069 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af4024930 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.189+0000 7f0b08c00700 1 -- 172.21.15.18:0/98481069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0419f080 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.189+0000 7f0b08c00700 1 -- 172.21.15.18:0/98481069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0419f4e0 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.191+0000 7f0afaa00700 1 -- 172.21.15.18:0/98481069 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0af4022020 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.191+0000 7f0aefe00700 1 -- 172.21.15.18:0/98481069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0adc005330 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.192+0000 7f0afaa00700 1 --2- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0ae8039660 0x7f0ae803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.192+0000 7f0afaa00700 1 -- 172.21.15.18:0/98481069 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0af404da70 con 0x7f0b0407c310 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.192+0000 7f0b01400700 1 --2- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0ae8039660 0x7f0ae803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.193+0000 7f0b01400700 1 --2- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0ae8039660 0x7f0ae803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0af0006fd0 tx=0x7f0af0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:28.677 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.198+0000 7f0afaa00700 1 -- 172.21.15.18:0/98481069 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0af400b3e0 con 0x7f0b0407c310 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.425+0000 7f0aefe00700 1 -- 172.21.15.18:0/98481069 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0adc000bc0 con 0x7f0ae8039660 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.601+0000 7f0afaa00700 1 -- 172.21.15.18:0/98481069 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0adc000bc0 con 0x7f0ae8039660 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 -- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0ae8039660 msgr2=0x7f0ae803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 --2- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0ae8039660 0x7f0ae803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0af0006fd0 tx=0x7f0af0006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 -- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 msgr2=0x7f0b0419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 --2- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0419e8b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0af4011b60 tx=0x7f0af4011c40 comp rx=0 tx=0).stop 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 -- 172.21.15.18:0/98481069 shutdown_connections 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 --2- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0ae8039660 0x7f0ae803bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 --2- 172.21.15.18:0/98481069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b0407c310 0x7f0b0419e8b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 -- 172.21.15.18:0/98481069 >> 172.21.15.18:0/98481069 conn(0x7f0b04103cd0 msgr2=0x7f0b04104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 -- 172.21.15.18:0/98481069 shutdown_connections 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:27.604+0000 7f0aefe00700 1 -- 172.21.15.18:0/98481069 wait complete. 2024-04-16T16:53:28.678 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial admin user... 2024-04-16T16:53:29.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:29 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:30.481 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PuUyzTO3hPl48t4w167SS.0Fl6RVJPKgjmh8sQi6u4IBZ9q/bfGaO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713286409, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-16T16:53:30.481 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.272+0000 7f2c5f000700 1 Processor -- start 2024-04-16T16:53:30.481 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.272+0000 7f2c5f000700 1 -- start start 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.272+0000 7f2c5f000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c5810b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.272+0000 7f2c5f000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5807be40 con 0x7f2c58109140 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.273+0000 7f2c57e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c5810b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.273+0000 7f2c57e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c5810b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43460/0 (socket says 172.21.15.18:43460) 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.273+0000 7f2c57e00700 1 -- 172.21.15.18:0/2440771273 learned_addr learned my addr 172.21.15.18:0/2440771273 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.273+0000 7f2c57e00700 1 -- 172.21.15.18:0/2440771273 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5810ba70 con 0x7f2c58109140 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.274+0000 7f2c57e00700 1 --2- 172.21.15.18:0/2440771273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c5810b530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009e80 tx=0x7f2c4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8463eb7f6666630d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.274+0000 7f2c57400700 1 -- 172.21.15.18:0/2440771273 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48017070 con 0x7f2c58109140 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.274+0000 7f2c57400700 1 -- 172.21.15.18:0/2440771273 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c4800b8c0 con 0x7f2c58109140 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.274+0000 7f2c57400700 1 -- 172.21.15.18:0/2440771273 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c480045c0 con 0x7f2c58109140 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.275+0000 7f2c5f000700 1 -- 172.21.15.18:0/2440771273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 msgr2=0x7f2c5810b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.275+0000 7f2c5f000700 1 --2- 172.21.15.18:0/2440771273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c5810b530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009e80 tx=0x7f2c4800b330 comp rx=0 tx=0).stop 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.275+0000 7f2c5f000700 1 -- 172.21.15.18:0/2440771273 shutdown_connections 2024-04-16T16:53:30.482 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.275+0000 7f2c5f000700 1 --2- 172.21.15.18:0/2440771273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c5810b530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.275+0000 7f2c5f000700 1 -- 172.21.15.18:0/2440771273 >> 172.21.15.18:0/2440771273 conn(0x7f2c58102b10 msgr2=0x7f2c58104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.275+0000 7f2c5f000700 1 -- 172.21.15.18:0/2440771273 shutdown_connections 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.275+0000 7f2c5f000700 1 -- 172.21.15.18:0/2440771273 wait complete. 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.276+0000 7f2c5f000700 1 Processor -- start 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.276+0000 7f2c5f000700 1 -- start start 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.277+0000 7f2c5f000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c581089c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.277+0000 7f2c5f000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5810ca70 con 0x7f2c58109140 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.277+0000 7f2c57e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c581089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.277+0000 7f2c57e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c581089c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43470/0 (socket says 172.21.15.18:43470) 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.277+0000 7f2c57e00700 1 -- 172.21.15.18:0/1163555686 learned_addr learned my addr 172.21.15.18:0/1163555686 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.277+0000 7f2c57e00700 1 -- 172.21.15.18:0/1163555686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c58108f00 con 0x7f2c58109140 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.278+0000 7f2c57e00700 1 --2- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c581089c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c48004350 tx=0x7f2c4800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.278+0000 7f2c55600700 1 -- 172.21.15.18:0/1163555686 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48020020 con 0x7f2c58109140 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.278+0000 7f2c55600700 1 -- 172.21.15.18:0/1163555686 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c48003d90 con 0x7f2c58109140 2024-04-16T16:53:30.483 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.278+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c581070f0 con 0x7f2c58109140 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.278+0000 7f2c55600700 1 -- 172.21.15.18:0/1163555686 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48017040 con 0x7f2c58109140 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.278+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c58107630 con 0x7f2c58109140 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.280+0000 7f2c55600700 1 -- 172.21.15.18:0/1163555686 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f2c48023020 con 0x7f2c58109140 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.280+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c58064150 con 0x7f2c58109140 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.280+0000 7f2c55600700 1 --2- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f2c3c039610 0x7f2c3c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.280+0000 7f2c55600700 1 -- 172.21.15.18:0/1163555686 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2c4800f050 con 0x7f2c58109140 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.280+0000 7f2c4f400700 1 --2- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f2c3c039610 0x7f2c3c03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.281+0000 7f2c4f400700 1 --2- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f2c3c039610 0x7f2c3c03bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c40006fd0 tx=0x7f2c40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.286+0000 7f2c55600700 1 -- 172.21.15.18:0/1163555686 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c4802a020 con 0x7f2c58109140 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.542+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2c58107800 con 0x7f2c3c039610 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.796+0000 7f2c55600700 1 -- 172.21.15.18:0/1163555686 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2c58107800 con 0x7f2c3c039610 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f2c3c039610 msgr2=0x7f2c3c03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 --2- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f2c3c039610 0x7f2c3c03bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c40006fd0 tx=0x7f2c40006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:30.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 msgr2=0x7f2c581089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:30.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 --2- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c581089c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c48004350 tx=0x7f2c4800bd50 comp rx=0 tx=0).stop 2024-04-16T16:53:30.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 shutdown_connections 2024-04-16T16:53:30.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 --2- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f2c3c039610 0x7f2c3c03bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:30.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 --2- 172.21.15.18:0/1163555686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c58109140 0x7f2c581089c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:30.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 >> 172.21.15.18:0/1163555686 conn(0x7f2c58102b10 msgr2=0x7f2c58103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:30.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.799+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 shutdown_connections 2024-04-16T16:53:30.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:29.800+0000 7f2c5f000700 1 -- 172.21.15.18:0/1163555686 wait complete. 2024-04-16T16:53:30.485 INFO:teuthology.orchestra.run.smithi018.stdout:Fetching dashboard port number... 2024-04-16T16:53:31.094 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:30 smithi018 ceph-mon[31776]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:53:31.094 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:30 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:31 smithi018 ceph-mon[31776]: mgrmap e13: x(active, since 6s) 2024-04-16T16:53:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:31 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:31 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3046014959' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-16T16:53:32.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 8443 2024-04-16T16:53:32.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.258+0000 7f8972e00700 1 Processor -- start 2024-04-16T16:53:32.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.258+0000 7f8972e00700 1 -- start start 2024-04-16T16:53:32.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.258+0000 7f8972e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c10b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:32.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.258+0000 7f8972e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f896c07be60 con 0x7f896c109120 2024-04-16T16:53:32.211 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.259+0000 7f896be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c10b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.259+0000 7f896be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c10b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43478/0 (socket says 172.21.15.18:43478) 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.259+0000 7f896be00700 1 -- 172.21.15.18:0/4073317636 learned_addr learned my addr 172.21.15.18:0/4073317636 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.259+0000 7f896be00700 1 -- 172.21.15.18:0/4073317636 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c10ba50 con 0x7f896c109120 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.260+0000 7f896be00700 1 --2- 172.21.15.18:0/4073317636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c10b510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f895c0098a0 tx=0x7f895c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22159c72694e3536 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.260+0000 7f896aa00700 1 -- 172.21.15.18:0/4073317636 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f895c004160 con 0x7f896c109120 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.260+0000 7f896aa00700 1 -- 172.21.15.18:0/4073317636 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c004300 con 0x7f896c109120 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.260+0000 7f896aa00700 1 -- 172.21.15.18:0/4073317636 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f895c011b20 con 0x7f896c109120 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.261+0000 7f8972e00700 1 -- 172.21.15.18:0/4073317636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 msgr2=0x7f896c10b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.261+0000 7f8972e00700 1 --2- 172.21.15.18:0/4073317636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c10b510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f895c0098a0 tx=0x7f895c009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.261+0000 7f8972e00700 1 -- 172.21.15.18:0/4073317636 shutdown_connections 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.261+0000 7f8972e00700 1 --2- 172.21.15.18:0/4073317636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c10b510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.261+0000 7f8972e00700 1 -- 172.21.15.18:0/4073317636 >> 172.21.15.18:0/4073317636 conn(0x7f896c102b10 msgr2=0x7f896c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.261+0000 7f8972e00700 1 -- 172.21.15.18:0/4073317636 shutdown_connections 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.261+0000 7f8972e00700 1 -- 172.21.15.18:0/4073317636 wait complete. 2024-04-16T16:53:32.212 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.262+0000 7f8972e00700 1 Processor -- start 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.262+0000 7f8972e00700 1 -- start start 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.263+0000 7f8972e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.263+0000 7f8972e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f896c10ca60 con 0x7f896c109120 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.263+0000 7f896be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.263+0000 7f896be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43488/0 (socket says 172.21.15.18:43488) 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.263+0000 7f896be00700 1 -- 172.21.15.18:0/3046014959 learned_addr learned my addr 172.21.15.18:0/3046014959 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.263+0000 7f896be00700 1 -- 172.21.15.18:0/3046014959 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c19ed50 con 0x7f896c109120 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.264+0000 7f896be00700 1 --2- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c19e810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f895c011b60 tx=0x7f895c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.264+0000 7f8968c00700 1 -- 172.21.15.18:0/3046014959 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f895c003b60 con 0x7f896c109120 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.264+0000 7f8968c00700 1 -- 172.21.15.18:0/3046014959 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c01a910 con 0x7f896c109120 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.264+0000 7f8968c00700 1 -- 172.21.15.18:0/3046014959 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f895c024930 con 0x7f896c109120 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.264+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c19efe0 con 0x7f896c109120 2024-04-16T16:53:32.213 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.264+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c19f440 con 0x7f896c109120 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.266+0000 7f8968c00700 1 -- 172.21.15.18:0/3046014959 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f895c022020 con 0x7f896c109120 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.266+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896c064150 con 0x7f896c109120 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.266+0000 7f8968c00700 1 --2- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8950039750 0x7f895003bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.266+0000 7f8968c00700 1 -- 172.21.15.18:0/3046014959 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f895c04dc10 con 0x7f896c109120 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.267+0000 7f896b400700 1 --2- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8950039750 0x7f895003bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.267+0000 7f896b400700 1 --2- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8950039750 0x7f895003bc10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8958006fd0 tx=0x7f8958006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.272+0000 7f8968c00700 1 -- 172.21.15.18:0/3046014959 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f895c01a4e0 con 0x7f896c109120 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.505+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f896c1a1e10 con 0x7f896c109120 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.506+0000 7f8968c00700 1 -- 172.21.15.18:0/3046014959 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f895c02d3f0 con 0x7f896c109120 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.508+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8950039750 msgr2=0x7f895003bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 --2- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8950039750 0x7f895003bc10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8958006fd0 tx=0x7f8958006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 msgr2=0x7f896c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 --2- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c19e810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f895c011b60 tx=0x7f895c011c40 comp rx=0 tx=0).stop 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 shutdown_connections 2024-04-16T16:53:32.214 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 --2- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8950039750 0x7f895003bc10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:32.215 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 --2- 172.21.15.18:0/3046014959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f896c109120 0x7f896c19e810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:32.215 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 >> 172.21.15.18:0/3046014959 conn(0x7f896c102b10 msgr2=0x7f896c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:32.215 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 shutdown_connections 2024-04-16T16:53:32.215 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:31.509+0000 7f8972e00700 1 -- 172.21.15.18:0/3046014959 wait complete. 2024-04-16T16:53:32.234 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T16:53:32.234 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-04-16T16:53:32.252 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T16:53:32.252 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-16T16:53:32.252 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-04-16T16:53:32.252 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-16T16:53:32.254 INFO:teuthology.orchestra.run.smithi018.stdout:Ceph Dashboard is now available at: 2024-04-16T16:53:32.254 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:32.255 INFO:teuthology.orchestra.run.smithi018.stdout: URL: https://smithi018.front.sepia.ceph.com:8443/ 2024-04-16T16:53:32.255 INFO:teuthology.orchestra.run.smithi018.stdout: User: admin 2024-04-16T16:53:32.255 INFO:teuthology.orchestra.run.smithi018.stdout: Password: 46yn3oy1xg 2024-04-16T16:53:32.255 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:32.255 INFO:teuthology.orchestra.run.smithi018.stdout:Saving cluster configuration to /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config directory 2024-04-16T16:53:32.256 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling autotune for osd_memory_target 2024-04-16T16:53:33.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.987+0000 7febf0600700 1 Processor -- start 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.987+0000 7febf0600700 1 -- start start 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.988+0000 7febf0600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe8108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.988+0000 7febf0600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7febe8109040 con 0x7febe8108650 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.988+0000 7febedc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe8108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.988+0000 7febedc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe8108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43502/0 (socket says 172.21.15.18:43502) 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.988+0000 7febedc00700 1 -- 172.21.15.18:0/3088020028 learned_addr learned my addr 172.21.15.18:0/3088020028 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.989+0000 7febedc00700 1 -- 172.21.15.18:0/3088020028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe8109800 con 0x7febe8108650 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.989+0000 7febedc00700 1 --2- 172.21.15.18:0/3088020028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe8108a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7febdc0098a0 tx=0x7febdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55848690092bf3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.989+0000 7febe7e00700 1 -- 172.21.15.18:0/3088020028 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febdc004160 con 0x7febe8108650 2024-04-16T16:53:33.817 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.989+0000 7febe7e00700 1 -- 172.21.15.18:0/3088020028 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febdc004300 con 0x7febe8108650 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.989+0000 7febe7e00700 1 -- 172.21.15.18:0/3088020028 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febdc011b20 con 0x7febe8108650 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.990+0000 7febf0600700 1 -- 172.21.15.18:0/3088020028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 msgr2=0x7febe8108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.990+0000 7febf0600700 1 --2- 172.21.15.18:0/3088020028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe8108a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7febdc0098a0 tx=0x7febdc009bb0 comp rx=0 tx=0).stop 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.990+0000 7febf0600700 1 -- 172.21.15.18:0/3088020028 shutdown_connections 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.990+0000 7febf0600700 1 --2- 172.21.15.18:0/3088020028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe8108a70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.990+0000 7febf0600700 1 -- 172.21.15.18:0/3088020028 >> 172.21.15.18:0/3088020028 conn(0x7febe8103c30 msgr2=0x7febe8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.990+0000 7febf0600700 1 -- 172.21.15.18:0/3088020028 shutdown_connections 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.990+0000 7febf0600700 1 -- 172.21.15.18:0/3088020028 wait complete. 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.991+0000 7febf0600700 1 Processor -- start 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.991+0000 7febf0600700 1 -- start start 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.991+0000 7febf0600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.991+0000 7febf0600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7febe810cbb0 con 0x7febe8108650 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.992+0000 7febedc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:33.818 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.992+0000 7febedc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43518/0 (socket says 172.21.15.18:43518) 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.992+0000 7febedc00700 1 -- 172.21.15.18:0/2007552677 learned_addr learned my addr 172.21.15.18:0/2007552677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.992+0000 7febedc00700 1 -- 172.21.15.18:0/2007552677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe819ed80 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.992+0000 7febedc00700 1 --2- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe819e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febdc000c00 tx=0x7febdc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.993+0000 7febe6000700 1 -- 172.21.15.18:0/2007552677 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febdc003b60 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.993+0000 7febe6000700 1 -- 172.21.15.18:0/2007552677 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febdc01a910 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.993+0000 7febe6000700 1 -- 172.21.15.18:0/2007552677 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febdc024930 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.993+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe819f010 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.993+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe819f470 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.994+0000 7febe6000700 1 -- 172.21.15.18:0/2007552677 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7febdc022020 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.994+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe8064150 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.994+0000 7febe6000700 1 --2- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7febd0039750 0x7febd003bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.994+0000 7febe6000700 1 -- 172.21.15.18:0/2007552677 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7febdc04db30 con 0x7febe8108650 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.995+0000 7febed200700 1 --2- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7febd0039750 0x7febd003bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:32.995+0000 7febed200700 1 --2- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7febd0039750 0x7febd003bc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7febd4006fd0 tx=0x7febd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.000+0000 7febe6000700 1 -- 172.21.15.18:0/2007552677 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febdc02d910 con 0x7febe8108650 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.233+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7febe81a1e40 con 0x7febe8108650 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.234+0000 7febe6000700 1 -- 172.21.15.18:0/2007552677 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7febdc02d3f0 con 0x7febe8108650 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7febd0039750 msgr2=0x7febd003bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 --2- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7febd0039750 0x7febd003bc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7febd4006fd0 tx=0x7febd4006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 msgr2=0x7febe819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 --2- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe819e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febdc000c00 tx=0x7febdc011c10 comp rx=0 tx=0).stop 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 shutdown_connections 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 --2- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7febd0039750 0x7febd003bc10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 --2- 172.21.15.18:0/2007552677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febe8108650 0x7febe819e840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 >> 172.21.15.18:0/2007552677 conn(0x7febe8103c30 msgr2=0x7febe8104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.237+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 shutdown_connections 2024-04-16T16:53:33.820 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:33.238+0000 7febf0600700 1 -- 172.21.15.18:0/2007552677 wait complete. 2024-04-16T16:53:35.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.787+0000 7f6c21600700 1 Processor -- start 2024-04-16T16:53:35.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.787+0000 7f6c21600700 1 -- start start 2024-04-16T16:53:35.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.788+0000 7f6c21600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c10b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:35.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.788+0000 7f6c21600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c10ba20 con 0x7f6c1c1090f0 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.788+0000 7f6c1aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c10b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.788+0000 7f6c1aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c10b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43532/0 (socket says 172.21.15.18:43532) 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.788+0000 7f6c1aa00700 1 -- 172.21.15.18:0/822110818 learned_addr learned my addr 172.21.15.18:0/822110818 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.789+0000 7f6c1aa00700 1 -- 172.21.15.18:0/822110818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c10bba0 con 0x7f6c1c1090f0 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.789+0000 7f6c1aa00700 1 --2- 172.21.15.18:0/822110818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c10b4e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009e80 tx=0x7f6c0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7898a10576862009 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.789+0000 7f6c1a000700 1 -- 172.21.15.18:0/822110818 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c017070 con 0x7f6c1c1090f0 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.790+0000 7f6c1a000700 1 -- 172.21.15.18:0/822110818 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c00b8c0 con 0x7f6c1c1090f0 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.790+0000 7f6c1a000700 1 -- 172.21.15.18:0/822110818 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c0045c0 con 0x7f6c1c1090f0 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.790+0000 7f6c21600700 1 -- 172.21.15.18:0/822110818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 msgr2=0x7f6c1c10b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:35.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.790+0000 7f6c21600700 1 --2- 172.21.15.18:0/822110818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c10b4e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009e80 tx=0x7f6c0c00b330 comp rx=0 tx=0).stop 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.790+0000 7f6c21600700 1 -- 172.21.15.18:0/822110818 shutdown_connections 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.790+0000 7f6c21600700 1 --2- 172.21.15.18:0/822110818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c10b4e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.790+0000 7f6c21600700 1 -- 172.21.15.18:0/822110818 >> 172.21.15.18:0/822110818 conn(0x7f6c1c102b60 msgr2=0x7f6c1c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.791+0000 7f6c21600700 1 -- 172.21.15.18:0/822110818 shutdown_connections 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.791+0000 7f6c21600700 1 -- 172.21.15.18:0/822110818 wait complete. 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.792+0000 7f6c21600700 1 Processor -- start 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.792+0000 7f6c21600700 1 -- start start 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.792+0000 7f6c21600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.792+0000 7f6c21600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c10cba0 con 0x7f6c1c1090f0 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.792+0000 7f6c1aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.793+0000 7f6c1aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43548/0 (socket says 172.21.15.18:43548) 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.793+0000 7f6c1aa00700 1 -- 172.21.15.18:0/2107283590 learned_addr learned my addr 172.21.15.18:0/2107283590 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:35.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.793+0000 7f6c1aa00700 1 -- 172.21.15.18:0/2107283590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c19cc00 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.793+0000 7f6c1aa00700 1 --2- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c19c6c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009fd0 tx=0x7f6c0c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.794+0000 7f6c13e00700 1 -- 172.21.15.18:0/2107283590 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c020020 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.794+0000 7f6c13e00700 1 -- 172.21.15.18:0/2107283590 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c00bba0 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.794+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c19ce90 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.794+0000 7f6c13e00700 1 -- 172.21.15.18:0/2107283590 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c017040 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.794+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c19fb80 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.795+0000 7f6c13e00700 1 -- 172.21.15.18:0/2107283590 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6c0c023020 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.795+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1c064150 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.795+0000 7f6c13e00700 1 --2- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c00039700 0x7f6c0003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.796+0000 7f6c13e00700 1 -- 172.21.15.18:0/2107283590 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c0c00f050 con 0x7f6c1c1090f0 2024-04-16T16:53:35.775 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.796+0000 7f6c12000700 1 --2- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c00039700 0x7f6c0003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.797+0000 7f6c12000700 1 --2- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c00039700 0x7f6c0003bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c04006fd0 tx=0x7f6c04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:34.801+0000 7f6c13e00700 1 -- 172.21.15.18:0/2107283590 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c0c02a020 con 0x7f6c1c1090f0 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.151+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f6c1c19fd50 con 0x7f6c1c1090f0 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.155+0000 7f6c13e00700 1 -- 172.21.15.18:0/2107283590 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f6c0c01bbb0 con 0x7f6c1c1090f0 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.157+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c00039700 msgr2=0x7f6c0003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.157+0000 7f6c21600700 1 --2- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c00039700 0x7f6c0003bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c04006fd0 tx=0x7f6c04006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.157+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 msgr2=0x7f6c1c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.157+0000 7f6c21600700 1 --2- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c19c6c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009fd0 tx=0x7f6c0c003d00 comp rx=0 tx=0).stop 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.158+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 shutdown_connections 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.158+0000 7f6c21600700 1 --2- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c00039700 0x7f6c0003bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.158+0000 7f6c21600700 1 --2- 172.21.15.18:0/2107283590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c1c1090f0 0x7f6c1c19c6c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:35.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.158+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 >> 172.21.15.18:0/2107283590 conn(0x7f6c1c102b60 msgr2=0x7f6c1c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.158+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 shutdown_connections 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-16T16:53:35.158+0000 7f6c21600700 1 -- 172.21.15.18:0/2107283590 wait complete. 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout:Or, if you are only running a single cluster on this host: 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:35.777 INFO:teuthology.orchestra.run.smithi018.stdout: ceph telemetry on 2024-04-16T16:53:35.778 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:35.778 INFO:teuthology.orchestra.run.smithi018.stdout:For more information see: 2024-04-16T16:53:35.778 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:35.778 INFO:teuthology.orchestra.run.smithi018.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-16T16:53:35.778 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:35.778 INFO:teuthology.orchestra.run.smithi018.stdout:Bootstrap complete. 2024-04-16T16:53:35.885 INFO:tasks.cephadm:Fetching config... 2024-04-16T16:53:35.885 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:53:35.886 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-16T16:53:35.938 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-16T16:53:35.939 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:53:35.939 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-16T16:53:35.980 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-16T16:53:35.981 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:53:35.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-16T16:53:36.102 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-16T16:53:36.103 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:53:36.103 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-16T16:53:36.201 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-16T16:53:36.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOBALsi02soSat7yWqMzlbYlQGnjU6USLx/9UX7Uc6sjpggY2j38KNo0+L1xF9IEy61Fxk0b+aTVcLQonH0LJKBCSR/PfiEDJlAoqInP/JdBY6MXLZUrQXZ4Gtvjvha275BHRgM0nNKzz9ncwVjHPQaoLb5GBPrHE0ISOV1QKH47DZD8/5KMdg8PKpbjDkIOM2G/lSoy3UtyZ7kIdbTn2qj78eZ88feuDn8KWbddlb/lNO/2lsJkRXVG15YwW9EJ0V7kgaooF/aSzv2q7Z+WnvTq8J9KdMiSg1BcT68uj++nl+V0D3BZGOAto9L1XzJFsrPKAsnchOIXys3lOieeTESHMqZJAPWUjJNYfVhgcIWPB7ndUAlJZfmbPcAS7agDEkCNF5eQoC9OIIefD4uTlKI0Or6Uvdiq0Cl0EBedI4H9WiXayWeR40inxUcEaknHYl61+nBaI5jRgYhdrTMXl9AjQB3MoskkC46iTJTg8QEa8R245IeV/fAoa0hT+yekk= ceph-872930f8-fc11-11ee-bc90-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T16:53:36.328 INFO:teuthology.orchestra.run.smithi018.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOBALsi02soSat7yWqMzlbYlQGnjU6USLx/9UX7Uc6sjpggY2j38KNo0+L1xF9IEy61Fxk0b+aTVcLQonH0LJKBCSR/PfiEDJlAoqInP/JdBY6MXLZUrQXZ4Gtvjvha275BHRgM0nNKzz9ncwVjHPQaoLb5GBPrHE0ISOV1QKH47DZD8/5KMdg8PKpbjDkIOM2G/lSoy3UtyZ7kIdbTn2qj78eZ88feuDn8KWbddlb/lNO/2lsJkRXVG15YwW9EJ0V7kgaooF/aSzv2q7Z+WnvTq8J9KdMiSg1BcT68uj++nl+V0D3BZGOAto9L1XzJFsrPKAsnchOIXys3lOieeTESHMqZJAPWUjJNYfVhgcIWPB7ndUAlJZfmbPcAS7agDEkCNF5eQoC9OIIefD4uTlKI0Or6Uvdiq0Cl0EBedI4H9WiXayWeR40inxUcEaknHYl61+nBaI5jRgYhdrTMXl9AjQB3MoskkC46iTJTg8QEa8R245IeV/fAoa0hT+yekk= ceph-872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:53:36.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:36 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2107283590' entity='client.admin' 2024-04-16T16:53:36.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOBALsi02soSat7yWqMzlbYlQGnjU6USLx/9UX7Uc6sjpggY2j38KNo0+L1xF9IEy61Fxk0b+aTVcLQonH0LJKBCSR/PfiEDJlAoqInP/JdBY6MXLZUrQXZ4Gtvjvha275BHRgM0nNKzz9ncwVjHPQaoLb5GBPrHE0ISOV1QKH47DZD8/5KMdg8PKpbjDkIOM2G/lSoy3UtyZ7kIdbTn2qj78eZ88feuDn8KWbddlb/lNO/2lsJkRXVG15YwW9EJ0V7kgaooF/aSzv2q7Z+WnvTq8J9KdMiSg1BcT68uj++nl+V0D3BZGOAto9L1XzJFsrPKAsnchOIXys3lOieeTESHMqZJAPWUjJNYfVhgcIWPB7ndUAlJZfmbPcAS7agDEkCNF5eQoC9OIIefD4uTlKI0Or6Uvdiq0Cl0EBedI4H9WiXayWeR40inxUcEaknHYl61+nBaI5jRgYhdrTMXl9AjQB3MoskkC46iTJTg8QEa8R245IeV/fAoa0hT+yekk= ceph-872930f8-fc11-11ee-bc90-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T16:53:36.435 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOBALsi02soSat7yWqMzlbYlQGnjU6USLx/9UX7Uc6sjpggY2j38KNo0+L1xF9IEy61Fxk0b+aTVcLQonH0LJKBCSR/PfiEDJlAoqInP/JdBY6MXLZUrQXZ4Gtvjvha275BHRgM0nNKzz9ncwVjHPQaoLb5GBPrHE0ISOV1QKH47DZD8/5KMdg8PKpbjDkIOM2G/lSoy3UtyZ7kIdbTn2qj78eZ88feuDn8KWbddlb/lNO/2lsJkRXVG15YwW9EJ0V7kgaooF/aSzv2q7Z+WnvTq8J9KdMiSg1BcT68uj++nl+V0D3BZGOAto9L1XzJFsrPKAsnchOIXys3lOieeTESHMqZJAPWUjJNYfVhgcIWPB7ndUAlJZfmbPcAS7agDEkCNF5eQoC9OIIefD4uTlKI0Or6Uvdiq0Cl0EBedI4H9WiXayWeR40inxUcEaknHYl61+nBaI5jRgYhdrTMXl9AjQB3MoskkC46iTJTg8QEa8R245IeV/fAoa0hT+yekk= ceph-872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:53:36.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOBALsi02soSat7yWqMzlbYlQGnjU6USLx/9UX7Uc6sjpggY2j38KNo0+L1xF9IEy61Fxk0b+aTVcLQonH0LJKBCSR/PfiEDJlAoqInP/JdBY6MXLZUrQXZ4Gtvjvha275BHRgM0nNKzz9ncwVjHPQaoLb5GBPrHE0ISOV1QKH47DZD8/5KMdg8PKpbjDkIOM2G/lSoy3UtyZ7kIdbTn2qj78eZ88feuDn8KWbddlb/lNO/2lsJkRXVG15YwW9EJ0V7kgaooF/aSzv2q7Z+WnvTq8J9KdMiSg1BcT68uj++nl+V0D3BZGOAto9L1XzJFsrPKAsnchOIXys3lOieeTESHMqZJAPWUjJNYfVhgcIWPB7ndUAlJZfmbPcAS7agDEkCNF5eQoC9OIIefD4uTlKI0Or6Uvdiq0Cl0EBedI4H9WiXayWeR40inxUcEaknHYl61+nBaI5jRgYhdrTMXl9AjQB3MoskkC46iTJTg8QEa8R245IeV/fAoa0hT+yekk= ceph-872930f8-fc11-11ee-bc90-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T16:53:36.510 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOBALsi02soSat7yWqMzlbYlQGnjU6USLx/9UX7Uc6sjpggY2j38KNo0+L1xF9IEy61Fxk0b+aTVcLQonH0LJKBCSR/PfiEDJlAoqInP/JdBY6MXLZUrQXZ4Gtvjvha275BHRgM0nNKzz9ncwVjHPQaoLb5GBPrHE0ISOV1QKH47DZD8/5KMdg8PKpbjDkIOM2G/lSoy3UtyZ7kIdbTn2qj78eZ88feuDn8KWbddlb/lNO/2lsJkRXVG15YwW9EJ0V7kgaooF/aSzv2q7Z+WnvTq8J9KdMiSg1BcT68uj++nl+V0D3BZGOAto9L1XzJFsrPKAsnchOIXys3lOieeTESHMqZJAPWUjJNYfVhgcIWPB7ndUAlJZfmbPcAS7agDEkCNF5eQoC9OIIefD4uTlKI0Or6Uvdiq0Cl0EBedI4H9WiXayWeR40inxUcEaknHYl61+nBaI5jRgYhdrTMXl9AjQB3MoskkC46iTJTg8QEa8R245IeV/fAoa0hT+yekk= ceph-872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:53:36.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-16T16:53:36.965 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:53:39.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.196+0000 7f0088000700 1 -- 172.21.15.18:0/4125766015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 msgr2=0x7f0080104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.196+0000 7f0088000700 1 --2- 172.21.15.18:0/4125766015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 0x7f0080104080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0074009910 tx=0x7f0074009c20 comp rx=0 tx=0).stop 2024-04-16T16:53:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.196+0000 7f0088000700 1 -- 172.21.15.18:0/4125766015 shutdown_connections 2024-04-16T16:53:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.196+0000 7f0088000700 1 --2- 172.21.15.18:0/4125766015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 0x7f0080104080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.196+0000 7f0088000700 1 -- 172.21.15.18:0/4125766015 >> 172.21.15.18:0/4125766015 conn(0x7f00800fdb40 msgr2=0x7f00800fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.196+0000 7f0088000700 1 -- 172.21.15.18:0/4125766015 shutdown_connections 2024-04-16T16:53:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.196+0000 7f0088000700 1 -- 172.21.15.18:0/4125766015 wait complete. 2024-04-16T16:53:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.198+0000 7f0088000700 1 Processor -- start 2024-04-16T16:53:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.198+0000 7f0088000700 1 -- start start 2024-04-16T16:53:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.198+0000 7f0088000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 0x7f0080199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.198+0000 7f0088000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0080108720 con 0x7f0080103c60 2024-04-16T16:53:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.199+0000 7f0085600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 0x7f0080199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.199+0000 7f0085600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 0x7f0080199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40932/0 (socket says 172.21.15.18:40932) 2024-04-16T16:53:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.199+0000 7f0085600700 1 -- 172.21.15.18:0/3717967353 learned_addr learned my addr 172.21.15.18:0/3717967353 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:39.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.199+0000 7f0085600700 1 -- 172.21.15.18:0/3717967353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008019a520 con 0x7f0080103c60 2024-04-16T16:53:39.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.199+0000 7f0085600700 1 --2- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 0x7f0080199fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0074005850 tx=0x7f0074005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:39.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.200+0000 7f007e000700 1 -- 172.21.15.18:0/3717967353 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0074026070 con 0x7f0080103c60 2024-04-16T16:53:39.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.200+0000 7f007e000700 1 -- 172.21.15.18:0/3717967353 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00740044a0 con 0x7f0080103c60 2024-04-16T16:53:39.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.200+0000 7f007e000700 1 -- 172.21.15.18:0/3717967353 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0074003ac0 con 0x7f0080103c60 2024-04-16T16:53:39.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.200+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008019a7b0 con 0x7f0080103c60 2024-04-16T16:53:39.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.200+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008019d4a0 con 0x7f0080103c60 2024-04-16T16:53:39.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.201+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0080108ea0 con 0x7f0080103c60 2024-04-16T16:53:39.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.201+0000 7f007e000700 1 -- 172.21.15.18:0/3717967353 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0074029070 con 0x7f0080103c60 2024-04-16T16:53:39.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.202+0000 7f007e000700 1 --2- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00680396b0 0x7f006803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:39.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.202+0000 7f007e000700 1 -- 172.21.15.18:0/3717967353 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f007401e070 con 0x7f0080103c60 2024-04-16T16:53:39.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.202+0000 7f0084c00700 1 --2- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00680396b0 0x7f006803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:39.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.203+0000 7f0084c00700 1 --2- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00680396b0 0x7f006803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f006c006fd0 tx=0x7f006c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:39.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.208+0000 7f007e000700 1 -- 172.21.15.18:0/3717967353 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0074013070 con 0x7f0080103c60 2024-04-16T16:53:39.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.444+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f008002dfe0 con 0x7f0080103c60 2024-04-16T16:53:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.449+0000 7f007e000700 1 -- 172.21.15.18:0/3717967353 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f00740300c0 con 0x7f0080103c60 2024-04-16T16:53:39.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.452+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00680396b0 msgr2=0x7f006803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:39.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.452+0000 7f0088000700 1 --2- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00680396b0 0x7f006803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f006c006fd0 tx=0x7f006c006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:39.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.452+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 msgr2=0x7f0080199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:39.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.452+0000 7f0088000700 1 --2- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 0x7f0080199fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0074005850 tx=0x7f0074005030 comp rx=0 tx=0).stop 2024-04-16T16:53:39.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.453+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 shutdown_connections 2024-04-16T16:53:39.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.453+0000 7f0088000700 1 --2- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00680396b0 0x7f006803bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:39.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.453+0000 7f0088000700 1 --2- 172.21.15.18:0/3717967353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0080103c60 0x7f0080199fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:39.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.453+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 >> 172.21.15.18:0/3717967353 conn(0x7f00800fdb40 msgr2=0x7f00800fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:39.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.453+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 shutdown_connections 2024-04-16T16:53:39.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:39.453+0000 7f0088000700 1 -- 172.21.15.18:0/3717967353 wait complete. 2024-04-16T16:53:40.170 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-16T16:53:40.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-16T16:53:40.633 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:53:40.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:40 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3717967353' entity='client.admin' 2024-04-16T16:53:40.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:40.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:40.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:53:40.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:40.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:53:40.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.574+0000 7f889a000700 1 -- 172.21.15.18:0/741565453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 msgr2=0x7f88940778c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.574+0000 7f889a000700 1 --2- 172.21.15.18:0/741565453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 0x7f88940778c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8884009910 tx=0x7f8884009c20 comp rx=0 tx=0).stop 2024-04-16T16:53:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.574+0000 7f889a000700 1 -- 172.21.15.18:0/741565453 shutdown_connections 2024-04-16T16:53:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.574+0000 7f889a000700 1 --2- 172.21.15.18:0/741565453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 0x7f88940778c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.574+0000 7f889a000700 1 -- 172.21.15.18:0/741565453 >> 172.21.15.18:0/741565453 conn(0x7f88940ffd20 msgr2=0x7f8894102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.574+0000 7f889a000700 1 -- 172.21.15.18:0/741565453 shutdown_connections 2024-04-16T16:53:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.574+0000 7f889a000700 1 -- 172.21.15.18:0/741565453 wait complete. 2024-04-16T16:53:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.575+0000 7f889a000700 1 Processor -- start 2024-04-16T16:53:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.575+0000 7f889a000700 1 -- start start 2024-04-16T16:53:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.576+0000 7f889a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 0x7f889419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:42.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.576+0000 7f889a000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f889410ec70 con 0x7f88940774a0 2024-04-16T16:53:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.576+0000 7f8893400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 0x7f889419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.576+0000 7f8893400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 0x7f889419e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40950/0 (socket says 172.21.15.18:40950) 2024-04-16T16:53:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.576+0000 7f8893400700 1 -- 172.21.15.18:0/3765684553 learned_addr learned my addr 172.21.15.18:0/3765684553 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.576+0000 7f8893400700 1 -- 172.21.15.18:0/3765684553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889419e750 con 0x7f88940774a0 2024-04-16T16:53:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.577+0000 7f8893400700 1 --2- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 0x7f889419e210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8884005950 tx=0x7f8884005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.577+0000 7f8890c00700 1 -- 172.21.15.18:0/3765684553 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8884026070 con 0x7f88940774a0 2024-04-16T16:53:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.577+0000 7f8890c00700 1 -- 172.21.15.18:0/3765684553 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888400bc30 con 0x7f88940774a0 2024-04-16T16:53:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.577+0000 7f8890c00700 1 -- 172.21.15.18:0/3765684553 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8884003b30 con 0x7f88940774a0 2024-04-16T16:53:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.577+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889419e9e0 con 0x7f88940774a0 2024-04-16T16:53:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.577+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88941a16d0 con 0x7f88940774a0 2024-04-16T16:53:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.578+0000 7f8890c00700 1 -- 172.21.15.18:0/3765684553 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8884029070 con 0x7f88940774a0 2024-04-16T16:53:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.578+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889410f380 con 0x7f88940774a0 2024-04-16T16:53:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.579+0000 7f8890c00700 1 --2- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8878039700 0x7f887803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.579+0000 7f8890c00700 1 -- 172.21.15.18:0/3765684553 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f888401e070 con 0x7f88940774a0 2024-04-16T16:53:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.579+0000 7f888aa00700 1 --2- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8878039700 0x7f887803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.580+0000 7f888aa00700 1 --2- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8878039700 0x7f887803bbc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f887c00ad30 tx=0x7f887c0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.585+0000 7f8890c00700 1 -- 172.21.15.18:0/3765684553 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8884013070 con 0x7f88940774a0 2024-04-16T16:53:42.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.822+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f889402e0f0 con 0x7f8878039700 2024-04-16T16:53:42.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.828+0000 7f8890c00700 1 -- 172.21.15.18:0/3765684553 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f889402e0f0 con 0x7f8878039700 2024-04-16T16:53:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8878039700 msgr2=0x7f887803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 --2- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8878039700 0x7f887803bbc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f887c00ad30 tx=0x7f887c0093f0 comp rx=0 tx=0).stop 2024-04-16T16:53:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 msgr2=0x7f889419e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 --2- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 0x7f889419e210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8884005950 tx=0x7f8884005030 comp rx=0 tx=0).stop 2024-04-16T16:53:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 shutdown_connections 2024-04-16T16:53:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 --2- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8878039700 0x7f887803bbc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 --2- 172.21.15.18:0/3765684553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88940774a0 0x7f889419e210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 >> 172.21.15.18:0/3765684553 conn(0x7f88940ffd20 msgr2=0x7f8894100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 shutdown_connections 2024-04-16T16:53:42.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:42.831+0000 7f889a000700 1 -- 172.21.15.18:0/3765684553 wait complete. 2024-04-16T16:53:43.534 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi022 2024-04-16T16:53:43.534 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:53:43.534 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/etc/ceph/ceph.conf 2024-04-16T16:53:43.557 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:53:43.558 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:53:43.617 INFO:tasks.cephadm:Adding host smithi022 to orchestrator... 2024-04-16T16:53:43.617 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch host add smithi022 2024-04-16T16:53:44.054 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:53:44.184 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:43 smithi018 ceph-mon[31776]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:53:44.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:43 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:44.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:43 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:53:44.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:43 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:53:44.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:43 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:53:44.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:43 smithi018 ceph-mon[31776]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-16T16:53:45.191 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:44 smithi018 ceph-mon[31776]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:53:45.191 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:44 smithi018 ceph-mon[31776]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:53:45.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.980+0000 7f773a600700 1 -- 172.21.15.18:0/1411743169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 msgr2=0x7f77341089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:45.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.980+0000 7f773a600700 1 --2- 172.21.15.18:0/1411743169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 0x7f77341089b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7724009910 tx=0x7f7724009c20 comp rx=0 tx=0).stop 2024-04-16T16:53:45.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.981+0000 7f773a600700 1 -- 172.21.15.18:0/1411743169 shutdown_connections 2024-04-16T16:53:45.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.981+0000 7f773a600700 1 --2- 172.21.15.18:0/1411743169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 0x7f77341089b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:45.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.981+0000 7f773a600700 1 -- 172.21.15.18:0/1411743169 >> 172.21.15.18:0/1411743169 conn(0x7f77340fe030 msgr2=0x7f7734100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:45.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.981+0000 7f773a600700 1 -- 172.21.15.18:0/1411743169 shutdown_connections 2024-04-16T16:53:45.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.981+0000 7f773a600700 1 -- 172.21.15.18:0/1411743169 wait complete. 2024-04-16T16:53:45.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.982+0000 7f773a600700 1 Processor -- start 2024-04-16T16:53:45.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.982+0000 7f773a600700 1 -- start start 2024-04-16T16:53:45.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.983+0000 7f773a600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 0x7f773419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:45.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.983+0000 7f773a600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f773410cb90 con 0x7f77341085d0 2024-04-16T16:53:45.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.983+0000 7f7733400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 0x7f773419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:45.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.983+0000 7f7733400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 0x7f773419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40972/0 (socket says 172.21.15.18:40972) 2024-04-16T16:53:45.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.983+0000 7f7733400700 1 -- 172.21.15.18:0/3345074292 learned_addr learned my addr 172.21.15.18:0/3345074292 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:45.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.984+0000 7f7733400700 1 -- 172.21.15.18:0/3345074292 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773419a6c0 con 0x7f77341085d0 2024-04-16T16:53:45.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.984+0000 7f7733400700 1 --2- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 0x7f773419a180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7724006010 tx=0x7f7724005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:45.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.984+0000 7f772be00700 1 -- 172.21.15.18:0/3345074292 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7724026070 con 0x7f77341085d0 2024-04-16T16:53:45.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.984+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773419a950 con 0x7f77341085d0 2024-04-16T16:53:45.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.984+0000 7f772be00700 1 -- 172.21.15.18:0/3345074292 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f772400bc30 con 0x7f77341085d0 2024-04-16T16:53:45.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.984+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773419ea50 con 0x7f77341085d0 2024-04-16T16:53:45.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.985+0000 7f772be00700 1 -- 172.21.15.18:0/3345074292 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7724030060 con 0x7f77341085d0 2024-04-16T16:53:45.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.986+0000 7f772be00700 1 -- 172.21.15.18:0/3345074292 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7724029070 con 0x7f77341085d0 2024-04-16T16:53:45.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.985+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7734103c40 con 0x7f77341085d0 2024-04-16T16:53:45.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.986+0000 7f772be00700 1 --2- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77180396b0 0x7f771803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:45.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.986+0000 7f772be00700 1 -- 172.21.15.18:0/3345074292 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f772401e070 con 0x7f77341085d0 2024-04-16T16:53:45.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.986+0000 7f7732a00700 1 --2- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77180396b0 0x7f771803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:45.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.987+0000 7f7732a00700 1 --2- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77180396b0 0x7f771803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7720006fd0 tx=0x7f7720006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:45.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:45.992+0000 7f772be00700 1 -- 172.21.15.18:0/3345074292 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7724013070 con 0x7f77341085d0 2024-04-16T16:53:46.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:45 smithi018 ceph-mon[31776]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:46.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:45 smithi018 ceph-mon[31776]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:53:46.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:46.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:46.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:46.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:46.226+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi022", "target": ["mon-mgr", ""]}) -- 0x7f773419b4e0 con 0x7f77180396b0 2024-04-16T16:53:47.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:46 smithi018 ceph-mon[31776]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:53:48.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:47 smithi018 ceph-mon[31776]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:48.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:47 smithi018 ceph-mon[31776]: Deploying cephadm binary to smithi022 2024-04-16T16:53:50.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:49 smithi018 ceph-mon[31776]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:51.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.298+0000 7f772be00700 1 -- 172.21.15.18:0/3345074292 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f773419b4e0 con 0x7f77180396b0 2024-04-16T16:53:51.300 INFO:teuthology.orchestra.run.smithi018.stdout:Added host 'smithi022' with addr '172.21.15.22' 2024-04-16T16:53:51.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.301+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77180396b0 msgr2=0x7f771803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:51.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.301+0000 7f773a600700 1 --2- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77180396b0 0x7f771803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7720006fd0 tx=0x7f7720006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:51.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.301+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 msgr2=0x7f773419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:51.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.301+0000 7f773a600700 1 --2- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 0x7f773419a180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7724006010 tx=0x7f7724005030 comp rx=0 tx=0).stop 2024-04-16T16:53:51.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.301+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 shutdown_connections 2024-04-16T16:53:51.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.301+0000 7f773a600700 1 --2- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77180396b0 0x7f771803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:51.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.301+0000 7f773a600700 1 --2- 172.21.15.18:0/3345074292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77341085d0 0x7f773419a180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:51.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.301+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 >> 172.21.15.18:0/3345074292 conn(0x7f77340fe030 msgr2=0x7f7734076710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:51.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.302+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 shutdown_connections 2024-04-16T16:53:51.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:51.302+0000 7f773a600700 1 -- 172.21.15.18:0/3345074292 wait complete. 2024-04-16T16:53:51.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch host ls --format=json 2024-04-16T16:53:52.408 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:53:52.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:52 smithi018 ceph-mon[31776]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:52.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:52.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:52 smithi018 ceph-mon[31776]: Added host smithi022 2024-04-16T16:53:52.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:53:52.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:53.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.321+0000 7f562f000700 1 -- 172.21.15.18:0/2166652121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 msgr2=0x7f5628075300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.321+0000 7f562f000700 1 --2- 172.21.15.18:0/2166652121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 0x7f5628075300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f561c009910 tx=0x7f561c009c20 comp rx=0 tx=0).stop 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.321+0000 7f562f000700 1 -- 172.21.15.18:0/2166652121 shutdown_connections 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.321+0000 7f562f000700 1 --2- 172.21.15.18:0/2166652121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 0x7f5628075300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.321+0000 7f562f000700 1 -- 172.21.15.18:0/2166652121 >> 172.21.15.18:0/2166652121 conn(0x7f56280fde50 msgr2=0x7f5628100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.321+0000 7f562f000700 1 -- 172.21.15.18:0/2166652121 shutdown_connections 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.321+0000 7f562f000700 1 -- 172.21.15.18:0/2166652121 wait complete. 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.322+0000 7f562f000700 1 Processor -- start 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.323+0000 7f562f000700 1 -- start start 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.323+0000 7f562f000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 0x7f562819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:54.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.323+0000 7f562f000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f562810ebf0 con 0x7f5628074f20 2024-04-16T16:53:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.323+0000 7f562dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 0x7f562819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.324+0000 7f562dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 0x7f562819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40642/0 (socket says 172.21.15.18:40642) 2024-04-16T16:53:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.324+0000 7f562dc00700 1 -- 172.21.15.18:0/3615331387 learned_addr learned my addr 172.21.15.18:0/3615331387 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.324+0000 7f562dc00700 1 -- 172.21.15.18:0/3615331387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562819a630 con 0x7f5628074f20 2024-04-16T16:53:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.324+0000 7f562dc00700 1 --2- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 0x7f562819a0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f561c006010 tx=0x7f561c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.324+0000 7f5626000700 1 -- 172.21.15.18:0/3615331387 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f561c026070 con 0x7f5628074f20 2024-04-16T16:53:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.325+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562819a8c0 con 0x7f5628074f20 2024-04-16T16:53:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.325+0000 7f5626000700 1 -- 172.21.15.18:0/3615331387 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c00bc30 con 0x7f5628074f20 2024-04-16T16:53:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.325+0000 7f5626000700 1 -- 172.21.15.18:0/3615331387 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f561c003ab0 con 0x7f5628074f20 2024-04-16T16:53:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.325+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562819ad20 con 0x7f5628074f20 2024-04-16T16:53:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.326+0000 7f5626000700 1 -- 172.21.15.18:0/3615331387 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f561c029070 con 0x7f5628074f20 2024-04-16T16:53:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.326+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56280765d0 con 0x7f5628074f20 2024-04-16T16:53:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.326+0000 7f5626000700 1 --2- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f56100396b0 0x7f561003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.327+0000 7f5626000700 1 -- 172.21.15.18:0/3615331387 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f561c01e070 con 0x7f5628074f20 2024-04-16T16:53:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.327+0000 7f562d200700 1 --2- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f56100396b0 0x7f561003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:54.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.328+0000 7f562d200700 1 --2- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f56100396b0 0x7f561003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5614006fd0 tx=0x7f5614006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:54.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.333+0000 7f5626000700 1 -- 172.21.15.18:0/3615331387 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f561c013070 con 0x7f5628074f20 2024-04-16T16:53:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.574+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f562819b450 con 0x7f56100396b0 2024-04-16T16:53:54.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.576+0000 7f5626000700 1 -- 172.21.15.18:0/3615331387 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f562819b450 con 0x7f56100396b0 2024-04-16T16:53:54.577 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:53:54.577 INFO:teuthology.orchestra.run.smithi018.stdout:[{"addr": "172.21.15.18", "hostname": "smithi018", "labels": [], "status": ""}, {"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}] 2024-04-16T16:53:54.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.579+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f56100396b0 msgr2=0x7f561003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:54.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.579+0000 7f562f000700 1 --2- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f56100396b0 0x7f561003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5614006fd0 tx=0x7f5614006e40 comp rx=0 tx=0).stop 2024-04-16T16:53:54.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.579+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 msgr2=0x7f562819a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:54.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.579+0000 7f562f000700 1 --2- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 0x7f562819a0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f561c006010 tx=0x7f561c005050 comp rx=0 tx=0).stop 2024-04-16T16:53:54.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.579+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 shutdown_connections 2024-04-16T16:53:54.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.579+0000 7f562f000700 1 --2- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f56100396b0 0x7f561003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:54.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.579+0000 7f562f000700 1 --2- 172.21.15.18:0/3615331387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5628074f20 0x7f562819a0f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:54.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.579+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 >> 172.21.15.18:0/3615331387 conn(0x7f56280fde50 msgr2=0x7f56281065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:54.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.580+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 shutdown_connections 2024-04-16T16:53:54.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:54.580+0000 7f562f000700 1 -- 172.21.15.18:0/3615331387 wait complete. 2024-04-16T16:53:54.913 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:54 smithi018 ceph-mon[31776]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:54.913 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:55.322 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi043 2024-04-16T16:53:55.322 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:53:55.323 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/etc/ceph/ceph.conf 2024-04-16T16:53:55.341 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:53:55.341 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:53:55.400 INFO:tasks.cephadm:Adding host smithi043 to orchestrator... 2024-04-16T16:53:55.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch host add smithi043 2024-04-16T16:53:55.863 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:53:56.691 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:56 smithi018 ceph-mon[31776]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:53:56.691 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:56 smithi018 ceph-mon[31776]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:57.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.853+0000 7f5af7200700 1 -- 172.21.15.18:0/681714062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 msgr2=0x7f5af8102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:53:57.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.853+0000 7f5af7200700 1 --2- 172.21.15.18:0/681714062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 0x7f5af8102980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009910 tx=0x7f5ae8009c20 comp rx=0 tx=0).stop 2024-04-16T16:53:57.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.854+0000 7f5af7200700 1 -- 172.21.15.18:0/681714062 shutdown_connections 2024-04-16T16:53:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.854+0000 7f5af7200700 1 --2- 172.21.15.18:0/681714062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 0x7f5af8102980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:53:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.854+0000 7f5af7200700 1 -- 172.21.15.18:0/681714062 >> 172.21.15.18:0/681714062 conn(0x7f5af80fde50 msgr2=0x7f5af8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:53:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.854+0000 7f5af7200700 1 -- 172.21.15.18:0/681714062 shutdown_connections 2024-04-16T16:53:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.854+0000 7f5af7200700 1 -- 172.21.15.18:0/681714062 wait complete. 2024-04-16T16:53:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.855+0000 7f5af7200700 1 Processor -- start 2024-04-16T16:53:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.855+0000 7f5af7200700 1 -- start start 2024-04-16T16:53:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.855+0000 7f5af7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 0x7f5af819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.855+0000 7f5af7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5af810c970 con 0x7f5af81025a0 2024-04-16T16:53:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.856+0000 7f5af5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 0x7f5af819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.856+0000 7f5af5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 0x7f5af819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60764/0 (socket says 172.21.15.18:60764) 2024-04-16T16:53:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.856+0000 7f5af5e00700 1 -- 172.21.15.18:0/1550189231 learned_addr learned my addr 172.21.15.18:0/1550189231 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:53:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.856+0000 7f5af5e00700 1 -- 172.21.15.18:0/1550189231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af819a5f0 con 0x7f5af81025a0 2024-04-16T16:53:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.856+0000 7f5af5e00700 1 --2- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 0x7f5af819a0b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8006010 tx=0x7f5ae8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:57.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.857+0000 7f5aef400700 1 -- 172.21.15.18:0/1550189231 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae8026070 con 0x7f5af81025a0 2024-04-16T16:53:57.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.857+0000 7f5aef400700 1 -- 172.21.15.18:0/1550189231 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae800bbd0 con 0x7f5af81025a0 2024-04-16T16:53:57.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.857+0000 7f5aef400700 1 -- 172.21.15.18:0/1550189231 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae8003910 con 0x7f5af81025a0 2024-04-16T16:53:57.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.857+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af819a8e0 con 0x7f5af81025a0 2024-04-16T16:53:57.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.857+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af819e980 con 0x7f5af81025a0 2024-04-16T16:53:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.858+0000 7f5aef400700 1 -- 172.21.15.18:0/1550189231 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5ae8029070 con 0x7f5af81025a0 2024-04-16T16:53:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.858+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af8103a80 con 0x7f5af81025a0 2024-04-16T16:53:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.859+0000 7f5aef400700 1 --2- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5adc039650 0x7f5adc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:53:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.859+0000 7f5aef400700 1 -- 172.21.15.18:0/1550189231 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5ae801e070 con 0x7f5af81025a0 2024-04-16T16:53:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.859+0000 7f5aed400700 1 --2- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5adc039650 0x7f5adc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:53:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.860+0000 7f5aed400700 1 --2- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5adc039650 0x7f5adc03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0006fd0 tx=0x7f5ae0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:53:57.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:57.864+0000 7f5aef400700 1 -- 172.21.15.18:0/1550189231 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ae8013070 con 0x7f5af81025a0 2024-04-16T16:53:58.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:53:58.104+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "target": ["mon-mgr", ""]}) -- 0x7f5af819b410 con 0x7f5adc039650 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:53:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:58 smithi018 ceph-mon[31776]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T16:53:59.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:59 smithi018 ceph-mon[31776]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:53:59.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:59 smithi018 ceph-mon[31776]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:53:59.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:59 smithi018 ceph-mon[31776]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:53:59.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:59 smithi018 ceph-mon[31776]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:53:59.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:59.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:53:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:53:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:00.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:00 smithi018 ceph-mon[31776]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:00.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:00 smithi018 ceph-mon[31776]: Deploying cephadm binary to smithi043 2024-04-16T16:54:02.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:02 smithi018 ceph-mon[31776]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:03.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.195+0000 7f5aef400700 1 -- 172.21.15.18:0/1550189231 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5af819b410 con 0x7f5adc039650 2024-04-16T16:54:03.197 INFO:teuthology.orchestra.run.smithi018.stdout:Added host 'smithi043' with addr '172.21.15.43' 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5adc039650 msgr2=0x7f5adc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 --2- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5adc039650 0x7f5adc03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0006fd0 tx=0x7f5ae0006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 msgr2=0x7f5af819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 --2- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 0x7f5af819a0b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8006010 tx=0x7f5ae8005030 comp rx=0 tx=0).stop 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 shutdown_connections 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 --2- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5adc039650 0x7f5adc03bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 --2- 172.21.15.18:0/1550189231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af81025a0 0x7f5af819a0b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 >> 172.21.15.18:0/1550189231 conn(0x7f5af80fde50 msgr2=0x7f5af8074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 shutdown_connections 2024-04-16T16:54:03.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:03.199+0000 7f5af7200700 1 -- 172.21.15.18:0/1550189231 wait complete. 2024-04-16T16:54:03.870 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch host ls --format=json 2024-04-16T16:54:04.335 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:54:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:04 smithi018 ceph-mon[31776]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:04 smithi018 ceph-mon[31776]: Added host smithi043 2024-04-16T16:54:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.353+0000 7f9e8d000700 1 -- 172.21.15.18:0/2072677196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 msgr2=0x7f9e88102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.353+0000 7f9e8d000700 1 --2- 172.21.15.18:0/2072677196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 0x7f9e88102980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800b0a0 tx=0x7f9e7800b3b0 comp rx=0 tx=0).stop 2024-04-16T16:54:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.353+0000 7f9e8d000700 1 -- 172.21.15.18:0/2072677196 shutdown_connections 2024-04-16T16:54:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.353+0000 7f9e8d000700 1 --2- 172.21.15.18:0/2072677196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 0x7f9e88102980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.353+0000 7f9e8d000700 1 -- 172.21.15.18:0/2072677196 >> 172.21.15.18:0/2072677196 conn(0x7f9e880fde50 msgr2=0x7f9e88100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.353+0000 7f9e8d000700 1 -- 172.21.15.18:0/2072677196 shutdown_connections 2024-04-16T16:54:06.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.353+0000 7f9e8d000700 1 -- 172.21.15.18:0/2072677196 wait complete. 2024-04-16T16:54:06.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.354+0000 7f9e8d000700 1 Processor -- start 2024-04-16T16:54:06.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.354+0000 7f9e8d000700 1 -- start start 2024-04-16T16:54:06.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.355+0000 7f9e8d000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 0x7f9e88199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:06.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.355+0000 7f9e8d000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8810c970 con 0x7f9e881025a0 2024-04-16T16:54:06.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.355+0000 7f9e87400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 0x7f9e88199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:06.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.355+0000 7f9e87400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 0x7f9e88199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60774/0 (socket says 172.21.15.18:60774) 2024-04-16T16:54:06.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.355+0000 7f9e87400700 1 -- 172.21.15.18:0/2530207095 learned_addr learned my addr 172.21.15.18:0/2530207095 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:54:06.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.355+0000 7f9e87400700 1 -- 172.21.15.18:0/2530207095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8819a530 con 0x7f9e881025a0 2024-04-16T16:54:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.356+0000 7f9e87400700 1 --2- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 0x7f9e88199ff0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e780093e0 tx=0x7f9e78003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.356+0000 7f9e84c00700 1 -- 172.21.15.18:0/2530207095 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e78017040 con 0x7f9e881025a0 2024-04-16T16:54:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.356+0000 7f9e84c00700 1 -- 172.21.15.18:0/2530207095 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e78022070 con 0x7f9e881025a0 2024-04-16T16:54:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.356+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8819a7c0 con 0x7f9e881025a0 2024-04-16T16:54:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.356+0000 7f9e84c00700 1 -- 172.21.15.18:0/2530207095 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e7801d440 con 0x7f9e881025a0 2024-04-16T16:54:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.356+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8819e8c0 con 0x7f9e881025a0 2024-04-16T16:54:06.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.357+0000 7f9e84c00700 1 -- 172.21.15.18:0/2530207095 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9e78004000 con 0x7f9e881025a0 2024-04-16T16:54:06.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.358+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e88103a80 con 0x7f9e881025a0 2024-04-16T16:54:06.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.358+0000 7f9e84c00700 1 --2- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e6c039750 0x7f9e6c03bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:06.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.358+0000 7f9e84c00700 1 -- 172.21.15.18:0/2530207095 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e7800f050 con 0x7f9e881025a0 2024-04-16T16:54:06.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.358+0000 7f9e7ea00700 1 --2- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e6c039750 0x7f9e6c03bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:06.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.359+0000 7f9e7ea00700 1 --2- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e6c039750 0x7f9e6c03bc10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006fd0 tx=0x7f9e70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:06.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.364+0000 7f9e84c00700 1 -- 172.21.15.18:0/2530207095 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e78007e20 con 0x7f9e881025a0 2024-04-16T16:54:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:06 smithi018 ceph-mon[31776]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:06 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:06.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.605+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e8819b350 con 0x7f9e6c039750 2024-04-16T16:54:06.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.607+0000 7f9e84c00700 1 -- 172.21.15.18:0/2530207095 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f9e8819b350 con 0x7f9e6c039750 2024-04-16T16:54:06.608 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T16:54:06.608 INFO:teuthology.orchestra.run.smithi018.stdout:[{"addr": "172.21.15.18", "hostname": "smithi018", "labels": [], "status": ""}, {"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}] 2024-04-16T16:54:06.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.610+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e6c039750 msgr2=0x7f9e6c03bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:06.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.610+0000 7f9e8d000700 1 --2- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e6c039750 0x7f9e6c03bc10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006fd0 tx=0x7f9e70006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:06.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.610+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 msgr2=0x7f9e88199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:06.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.610+0000 7f9e8d000700 1 --2- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 0x7f9e88199ff0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e780093e0 tx=0x7f9e78003ce0 comp rx=0 tx=0).stop 2024-04-16T16:54:06.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.610+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 shutdown_connections 2024-04-16T16:54:06.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.610+0000 7f9e8d000700 1 --2- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e6c039750 0x7f9e6c03bc10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:06.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.610+0000 7f9e8d000700 1 --2- 172.21.15.18:0/2530207095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e881025a0 0x7f9e88199ff0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:06.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.611+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 >> 172.21.15.18:0/2530207095 conn(0x7f9e880fde50 msgr2=0x7f9e88105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:06.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.611+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 shutdown_connections 2024-04-16T16:54:06.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:06.611+0000 7f9e8d000700 1 -- 172.21.15.18:0/2530207095 wait complete. 2024-04-16T16:54:07.344 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-16T16:54:07.345 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd crush tunables default 2024-04-16T16:54:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:54:08.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:08 smithi018 ceph-mon[31776]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:54:08.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:08 smithi018 ceph-mon[31776]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.813+0000 7feaf2c00700 1 -- 172.21.15.18:0/3241288381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 msgr2=0x7feaec10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.813+0000 7feaf2c00700 1 --2- 172.21.15.18:0/3241288381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 0x7feaec10de80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feadc009910 tx=0x7feadc009c20 comp rx=0 tx=0).stop 2024-04-16T16:54:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.814+0000 7feaf2c00700 1 -- 172.21.15.18:0/3241288381 shutdown_connections 2024-04-16T16:54:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.814+0000 7feaf2c00700 1 --2- 172.21.15.18:0/3241288381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 0x7feaec10de80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.814+0000 7feaf2c00700 1 -- 172.21.15.18:0/3241288381 >> 172.21.15.18:0/3241288381 conn(0x7feaec078290 msgr2=0x7feaec07a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.814+0000 7feaf2c00700 1 -- 172.21.15.18:0/3241288381 shutdown_connections 2024-04-16T16:54:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.814+0000 7feaf2c00700 1 -- 172.21.15.18:0/3241288381 wait complete. 2024-04-16T16:54:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.815+0000 7feaf2c00700 1 Processor -- start 2024-04-16T16:54:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.815+0000 7feaf2c00700 1 -- start start 2024-04-16T16:54:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.815+0000 7feaf2c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 0x7feaec19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.815+0000 7feaf2c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feaec113160 con 0x7feaec10daa0 2024-04-16T16:54:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.816+0000 7feaebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 0x7feaec19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.816+0000 7feaebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 0x7feaec19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34234/0 (socket says 172.21.15.18:34234) 2024-04-16T16:54:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.816+0000 7feaebe00700 1 -- 172.21.15.18:0/827336876 learned_addr learned my addr 172.21.15.18:0/827336876 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:54:09.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.816+0000 7feaebe00700 1 -- 172.21.15.18:0/827336876 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec19eb40 con 0x7feaec10daa0 2024-04-16T16:54:09.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.816+0000 7feaebe00700 1 --2- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 0x7feaec19e600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feadc006010 tx=0x7feadc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:09.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.817+0000 7feae8c00700 1 -- 172.21.15.18:0/827336876 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feadc026070 con 0x7feaec10daa0 2024-04-16T16:54:09.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.817+0000 7feae8c00700 1 -- 172.21.15.18:0/827336876 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feadc00bbd0 con 0x7feaec10daa0 2024-04-16T16:54:09.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.817+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec19ee30 con 0x7feaec10daa0 2024-04-16T16:54:09.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.817+0000 7feae8c00700 1 -- 172.21.15.18:0/827336876 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feadc0039d0 con 0x7feaec10daa0 2024-04-16T16:54:09.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.817+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec1a2ed0 con 0x7feaec10daa0 2024-04-16T16:54:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.818+0000 7feae8c00700 1 -- 172.21.15.18:0/827336876 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7feadc029070 con 0x7feaec10daa0 2024-04-16T16:54:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.818+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaec10a1f0 con 0x7feaec10daa0 2024-04-16T16:54:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.819+0000 7feae8c00700 1 --2- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fead0039650 0x7fead003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.819+0000 7feae8c00700 1 -- 172.21.15.18:0/827336876 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feadc01e070 con 0x7feaec10daa0 2024-04-16T16:54:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.819+0000 7feaeb400700 1 --2- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fead0039650 0x7fead003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.820+0000 7feaeb400700 1 --2- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fead0039650 0x7fead003bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fead8006fd0 tx=0x7fead8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:09.826+0000 7feae8c00700 1 -- 172.21.15.18:0/827336876 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feadc013070 con 0x7feaec10daa0 2024-04-16T16:54:10.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.061+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7feaec19f960 con 0x7feaec10daa0 2024-04-16T16:54:10.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.357+0000 7feae8c00700 1 -- 172.21.15.18:0/827336876 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7feadc00de00 con 0x7feaec10daa0 2024-04-16T16:54:10.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fead0039650 msgr2=0x7fead003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:10.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 --2- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fead0039650 0x7fead003bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fead8006fd0 tx=0x7fead8006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:10.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 msgr2=0x7feaec19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:10.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 --2- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 0x7feaec19e600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feadc006010 tx=0x7feadc005030 comp rx=0 tx=0).stop 2024-04-16T16:54:10.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 shutdown_connections 2024-04-16T16:54:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 --2- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fead0039650 0x7fead003bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 --2- 172.21.15.18:0/827336876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec10daa0 0x7feaec19e600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 >> 172.21.15.18:0/827336876 conn(0x7feaec078290 msgr2=0x7feaec1104c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 shutdown_connections 2024-04-16T16:54:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:10.361+0000 7feaf2c00700 1 -- 172.21.15.18:0/827336876 wait complete. 2024-04-16T16:54:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:adjusted tunables profile to default 2024-04-16T16:54:10.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:10.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:10.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:10.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:10.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:10.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:54:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T16:54:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:10 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/827336876' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T16:54:11.107 INFO:tasks.cephadm:Adding mon.a on smithi018 2024-04-16T16:54:11.107 INFO:tasks.cephadm:Adding mon.b on smithi022 2024-04-16T16:54:11.107 INFO:tasks.cephadm:Adding mon.c on smithi043 2024-04-16T16:54:11.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch apply mon '3;smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c' 2024-04-16T16:54:11.349 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:11 smithi018 ceph-mon[31776]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:11 smithi018 ceph-mon[31776]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:11 smithi018 ceph-mon[31776]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:54:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:11 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/827336876' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T16:54:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:11 smithi018 ceph-mon[31776]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:11.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:11.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:11.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:12 smithi018 ceph-mon[31776]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:12.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.879+0000 7f376f7fe700 1 -- 172.21.15.43:0/2932476029 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3764005590 con 0x7f3770107480 2024-04-16T16:54:12.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 -- 172.21.15.43:0/2932476029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 msgr2=0x7f3770107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:12.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 --2- 172.21.15.43:0/2932476029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 0x7f3770107860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3764009910 tx=0x7f3764009c20 comp rx=0 tx=0).stop 2024-04-16T16:54:12.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 -- 172.21.15.43:0/2932476029 shutdown_connections 2024-04-16T16:54:12.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 --2- 172.21.15.43:0/2932476029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 0x7f3770107860 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:12.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 -- 172.21.15.43:0/2932476029 >> 172.21.15.43:0/2932476029 conn(0x7f3770076f50 msgr2=0x7f3770077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:12.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 -- 172.21.15.43:0/2932476029 shutdown_connections 2024-04-16T16:54:12.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 -- 172.21.15.43:0/2932476029 wait complete. 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 Processor -- start 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.880+0000 7f3777230700 1 -- start start 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.881+0000 7f3777230700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 0x7f377010a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.881+0000 7f3777230700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f377010cc60 con 0x7f3770107480 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.881+0000 7f3774fcc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 0x7f377010a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.881+0000 7f3774fcc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 0x7f377010a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.43:45528/0 (socket says 172.21.15.43:45528) 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.881+0000 7f3774fcc700 1 -- 172.21.15.43:0/2551478307 learned_addr learned my addr 172.21.15.43:0/2551478307 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.882+0000 7f3774fcc700 1 -- 172.21.15.43:0/2551478307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377010c890 con 0x7f3770107480 2024-04-16T16:54:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.882+0000 7f3774fcc700 1 --2- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 0x7f377010a9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3764005850 tx=0x7f3764005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:12.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.882+0000 7f376dffb700 1 -- 172.21.15.43:0/2551478307 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3764026070 con 0x7f3770107480 2024-04-16T16:54:12.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.882+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377010af40 con 0x7f3770107480 2024-04-16T16:54:12.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.883+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377010b400 con 0x7f3770107480 2024-04-16T16:54:12.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.883+0000 7f376dffb700 1 -- 172.21.15.43:0/2551478307 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f376400bbd0 con 0x7f3770107480 2024-04-16T16:54:12.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.883+0000 7f376dffb700 1 -- 172.21.15.43:0/2551478307 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3764003970 con 0x7f3770107480 2024-04-16T16:54:12.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.883+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3754005330 con 0x7f3770107480 2024-04-16T16:54:12.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.884+0000 7f376dffb700 1 -- 172.21.15.43:0/2551478307 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3764029070 con 0x7f3770107480 2024-04-16T16:54:12.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.884+0000 7f376dffb700 1 --2- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37600396b0 0x7f376003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:12.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.884+0000 7f376dffb700 1 -- 172.21.15.43:0/2551478307 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f376401e070 con 0x7f3770107480 2024-04-16T16:54:12.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.885+0000 7f376ffff700 1 --2- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37600396b0 0x7f376003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:12.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.886+0000 7f376ffff700 1 --2- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37600396b0 0x7f376003bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f375c006fd0 tx=0x7f375c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:12.899+0000 7f376dffb700 1 -- 172.21.15.43:0/2551478307 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3764013070 con 0x7f3770107480 2024-04-16T16:54:13.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.119+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c", "target": ["mon-mgr", ""]}) -- 0x7f3754000c70 con 0x7f37600396b0 2024-04-16T16:54:13.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.134+0000 7f376dffb700 1 -- 172.21.15.43:0/2551478307 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3754000c70 con 0x7f37600396b0 2024-04-16T16:54:13.137 INFO:teuthology.orchestra.run.smithi043.stdout:Scheduled mon update... 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37600396b0 msgr2=0x7f376003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 --2- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37600396b0 0x7f376003bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f375c006fd0 tx=0x7f375c006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 msgr2=0x7f377010a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 --2- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 0x7f377010a9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3764005850 tx=0x7f3764005030 comp rx=0 tx=0).stop 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 shutdown_connections 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 --2- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37600396b0 0x7f376003bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 --2- 172.21.15.43:0/2551478307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770107480 0x7f377010a9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 >> 172.21.15.43:0/2551478307 conn(0x7f3770076f50 msgr2=0x7f3770102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.136+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 shutdown_connections 2024-04-16T16:54:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:13.137+0000 7f3777230700 1 -- 172.21.15.43:0/2551478307 wait complete. 2024-04-16T16:54:13.865 DEBUG:teuthology.orchestra.run.smithi022:mon.b> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.b.service 2024-04-16T16:54:13.872 DEBUG:teuthology.orchestra.run.smithi043:mon.c> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.c.service 2024-04-16T16:54:13.876 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T16:54:13.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph mon dump -f json 2024-04-16T16:54:14.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c;count:3 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:14 smithi018 ceph-mon[31776]: Deploying daemon mon.c on smithi043 2024-04-16T16:54:15.012 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:16.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:16 smithi018 ceph-mon[31776]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.340+0000 7fdb65914700 1 -- 172.21.15.43:0/4241312167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 msgr2=0x7fdb60103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.340+0000 7fdb65914700 1 --2- 172.21.15.43:0/4241312167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 0x7fdb60103b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009910 tx=0x7fdb50009c20 comp rx=0 tx=0).stop 2024-04-16T16:54:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.340+0000 7fdb65914700 1 -- 172.21.15.43:0/4241312167 shutdown_connections 2024-04-16T16:54:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.340+0000 7fdb65914700 1 --2- 172.21.15.43:0/4241312167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 0x7fdb60103b00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.340+0000 7fdb65914700 1 -- 172.21.15.43:0/4241312167 >> 172.21.15.43:0/4241312167 conn(0x7fdb600fcef0 msgr2=0x7fdb600ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.340+0000 7fdb65914700 1 -- 172.21.15.43:0/4241312167 shutdown_connections 2024-04-16T16:54:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.340+0000 7fdb65914700 1 -- 172.21.15.43:0/4241312167 wait complete. 2024-04-16T16:54:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.340+0000 7fdb65914700 1 Processor -- start 2024-04-16T16:54:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.341+0000 7fdb65914700 1 -- start start 2024-04-16T16:54:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.341+0000 7fdb65914700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 0x7fdb60198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.341+0000 7fdb65914700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6010cba0 con 0x7fdb60103720 2024-04-16T16:54:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.341+0000 7fdb5effd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 0x7fdb60198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.341+0000 7fdb5effd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 0x7fdb60198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.43:45552/0 (socket says 172.21.15.43:45552) 2024-04-16T16:54:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.342+0000 7fdb5effd700 1 -- 172.21.15.43:0/3024700037 learned_addr learned my addr 172.21.15.43:0/3024700037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T16:54:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.342+0000 7fdb5effd700 1 -- 172.21.15.43:0/3024700037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb601985a0 con 0x7fdb60103720 2024-04-16T16:54:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.343+0000 7fdb5effd700 1 --2- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 0x7fdb60198060 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb50006010 tx=0x7fdb50005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.343+0000 7fdb64912700 1 -- 172.21.15.43:0/3024700037 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb50026070 con 0x7fdb60103720 2024-04-16T16:54:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.343+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb60198830 con 0x7fdb60103720 2024-04-16T16:54:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.343+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb60198c90 con 0x7fdb60103720 2024-04-16T16:54:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.343+0000 7fdb64912700 1 -- 172.21.15.43:0/3024700037 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5000bc30 con 0x7fdb60103720 2024-04-16T16:54:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.343+0000 7fdb64912700 1 -- 172.21.15.43:0/3024700037 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb50003ab0 con 0x7fdb60103720 2024-04-16T16:54:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.344+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb6006b180 con 0x7fdb60103720 2024-04-16T16:54:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.345+0000 7fdb64912700 1 -- 172.21.15.43:0/3024700037 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdb50029070 con 0x7fdb60103720 2024-04-16T16:54:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.345+0000 7fdb64912700 1 --2- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdb48039650 0x7fdb4803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.345+0000 7fdb64912700 1 -- 172.21.15.43:0/3024700037 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdb5001e070 con 0x7fdb60103720 2024-04-16T16:54:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.346+0000 7fdb5e7fc700 1 --2- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdb48039650 0x7fdb4803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.347+0000 7fdb5e7fc700 1 --2- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdb48039650 0x7fdb4803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb54006fd0 tx=0x7fdb54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.361+0000 7fdb64912700 1 -- 172.21.15.43:0/3024700037 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb50030020 con 0x7fdb60103720 2024-04-16T16:54:17.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.693+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdb601993c0 con 0x7fdb60103720 2024-04-16T16:54:17.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.694+0000 7fdb64912700 1 -- 172.21.15.43:0/3024700037 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fdb50013070 con 0x7fdb60103720 2024-04-16T16:54:17.696 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:54:17.696 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","modified":"2024-04-16T16:52:10.664893Z","created":"2024-04-16T16:52:10.664893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-16T16:54:17.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdb48039650 msgr2=0x7fdb4803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 --2- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdb48039650 0x7fdb4803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb54006fd0 tx=0x7fdb54006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 msgr2=0x7fdb60198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 --2- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 0x7fdb60198060 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb50006010 tx=0x7fdb50005050 comp rx=0 tx=0).stop 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 shutdown_connections 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 --2- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdb48039650 0x7fdb4803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 --2- 172.21.15.43:0/3024700037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb60103720 0x7fdb60198060 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 >> 172.21.15.43:0/3024700037 conn(0x7fdb600fcef0 msgr2=0x7fdb601044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.696+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 shutdown_connections 2024-04-16T16:54:17.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:17.697+0000 7fdb65914700 1 -- 172.21.15.43:0/3024700037 wait complete. 2024-04-16T16:54:17.700 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-04-16T16:54:17.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:17 smithi043 podman[36870]: 2024-04-16 16:54:17.908926674 +0000 UTC m=+0.245380682 container create 986fbbd02f667355d38ac3e619e74c759457b68a591beb00f0f110b92d970ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-c, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-16T16:54:18.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 podman[36870]: 2024-04-16 16:54:18.111460837 +0000 UTC m=+0.447914846 container init 986fbbd02f667355d38ac3e619e74c759457b68a591beb00f0f110b92d970ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-16T16:54:18.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 podman[36870]: 2024-04-16 16:54:18.114811232 +0000 UTC m=+0.451265233 container start 986fbbd02f667355d38ac3e619e74c759457b68a591beb00f0f110b92d970ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-c, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-16T16:54:18.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T16:54:18.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev), process ceph-mon, pid 2 2024-04-16T16:54:18.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pidfile_write: ignore empty --pid-file 2024-04-16T16:54:18.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: load: jerasure load: lrc 2024-04-16T16:54:18.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: RocksDB version: 7.9.2 2024-04-16T16:54:18.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Git sha 0 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Compile date 2024-04-16 06:46:19 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: DB SUMMARY 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: DB Session ID: 2CGSEAKA58I9238M607H 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: CURRENT file: CURRENT 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.error_if_exists: 0 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.create_if_missing: 0 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.paranoid_checks: 1 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T16:54:18.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.env: 0x565369316cc0 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.info_log: 0x56536adbdc20 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.statistics: (nil) 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.use_fsync: 0 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_log_file_size: 0 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.allow_fallocate: 1 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T16:54:18.181 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.use_direct_reads: 0 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.db_log_dir: 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.wal_dir: 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.write_buffer_manager: 0x56536ad8c1e0 2024-04-16T16:54:18.182 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.unordered_write: 0 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T16:54:18.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.row_cache: None 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.wal_filter: None 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.two_write_queues: 0 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.wal_compression: 0 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.atomic_flush: 0 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T16:54:18.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.log_readahead_size: 0 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_background_jobs: 2 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_background_compactions: -1 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_subcompactions: 1 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T16:54:18.186 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_open_files: -1 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_background_flushes: -1 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Compression algorithms supported: 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: kZSTD supported: 0 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: kXpressCompression supported: 0 2024-04-16T16:54:18.187 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: kZlibCompression supported: 1 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: kSnappyCompression supported: 1 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: kLZ4Compression supported: 1 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: kBZip2Compression supported: 0 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.merge_operator: 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_filter: None 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T16:54:18.188 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56536adbdde0) 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_type: 0 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_index_type: 0 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_shortening: 1 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: checksum: 4 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: no_block_cache: 0 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache: 0x56536adbf090 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_options: 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: capacity : 536870912 2024-04-16T16:54:18.189 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_shard_bits : 4 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: strict_capacity_limit : 0 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_compressed: (nil) 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: persistent_cache: (nil) 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size: 4096 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size_deviation: 10 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_restart_interval: 16 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_block_restart_interval: 1 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: metadata_block_size: 4096 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: partition_filters: 0 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: use_delta_encoding: 1 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: filter_policy: bloomfilter 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: whole_key_filtering: 1 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: verify_compression: 0 2024-04-16T16:54:18.190 INFO:journalctl@ceph.mon.c.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout: format_version: 5 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_index_compression: 1 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_align: 0 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout: max_auto_readahead_size: 262144 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout: prepopulate_block_cache: 0 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout: initial_auto_readahead_size: 8192 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression: NoCompression 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.num_levels: 7 2024-04-16T16:54:18.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T16:54:18.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T16:54:18.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T16:54:18.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.inplace_update_support: 0 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.bloom_locality: 0 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.max_successive_merges: 0 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T16:54:18.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.ttl: 2592000 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.enable_blob_files: false 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.min_blob_size: 0 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T16:54:18.196 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 93c93593-e220-42c2-a49f-2643ecfcf56d 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286458145212, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286458145816, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713286458, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "93c93593-e220-42c2-a49f-2643ecfcf56d", "db_session_id": "2CGSEAKA58I9238M607H", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286458145934, "job": 1, "event": "recovery_finished"} 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56536aeac000 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: DB pointer 0x56536ae96000 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** DB Stats ** 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:54:18.197 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-04-16T16:54:18.198 INFO:journalctl@ceph.mon.c.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache BinnedLRUCache@0x56536adbf090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.5e-05 secs_since: 0 2024-04-16T16:54:18.199 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: using public_addr v2:172.21.15.43:0/0 -> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: starting mon.c rank -1 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(???) e0 preinit fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).mds e1 new map 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout: e1 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout: legacy client fscid: -1 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout: No filesystems configured 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-16T16:54:18.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Deploying cephadm binary to smithi043 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Added host smithi043 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/827336876' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/827336876' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:18.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c;count:3 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: Deploying daemon mon.c on smithi043 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/3024700037' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:18.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 ceph-mon[36901]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-16T16:54:18.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:18 smithi018 ceph-mon[31776]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:18.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:18 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/3024700037' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:18.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 bash[36870]: 986fbbd02f667355d38ac3e619e74c759457b68a591beb00f0f110b92d970ccd 2024-04-16T16:54:18.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:18 smithi043 systemd[1]: Started Ceph mon.c for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:54:19.819 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T16:54:19.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph mon dump -f json 2024-04-16T16:54:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:54:23.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: mon.a calling monitor election 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: mon.c calling monitor election 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: monmap epoch 2 2024-04-16T16:54:23.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: last_changed 2024-04-16T16:54:18.184557+0000 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: min_mon_release 19 (squid) 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: election_strategy: 1 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: fsmap 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: mgrmap e13: x(active, since 58s) 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: overall HEALTH_OK 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:23.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:23.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:54:23.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: mon.a calling monitor election 2024-04-16T16:54:23.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:23.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: mon.c calling monitor election 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: monmap epoch 2 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: last_changed 2024-04-16T16:54:18.184557+0000 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: min_mon_release 19 (squid) 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: election_strategy: 1 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: fsmap 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:23.746 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: mgrmap e13: x(active, since 58s) 2024-04-16T16:54:23.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: overall HEALTH_OK 2024-04-16T16:54:23.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:23.747 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:24.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:24 smithi018 ceph-mon[31776]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:24.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:24.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:24.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:24.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:24 smithi018 ceph-mon[31776]: Deploying daemon mon.b on smithi022 2024-04-16T16:54:24.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:24.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.552+0000 7fe448517700 1 -- 172.21.15.43:0/183041048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe42c0035c0 msgr2=0x7fe42c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.552+0000 7fe448517700 1 --2- 172.21.15.43:0/183041048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe42c0035c0 0x7fe42c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe440078c20 tx=0x7fe43000feb0 comp rx=0 tx=0).stop 2024-04-16T16:54:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.552+0000 7fe448517700 1 -- 172.21.15.43:0/183041048 shutdown_connections 2024-04-16T16:54:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.552+0000 7fe448517700 1 --2- 172.21.15.43:0/183041048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe42c0035c0 0x7fe42c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.552+0000 7fe448517700 1 -- 172.21.15.43:0/183041048 >> 172.21.15.43:0/183041048 conn(0x7fe440077970 msgr2=0x7fe440077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.552+0000 7fe448517700 1 -- 172.21.15.43:0/183041048 shutdown_connections 2024-04-16T16:54:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.552+0000 7fe448517700 1 -- 172.21.15.43:0/183041048 wait complete. 2024-04-16T16:54:24.554 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:24 smithi043 ceph-mon[36901]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:24.554 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:24.554 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:24.555 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:24.555 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:24 smithi043 ceph-mon[36901]: Deploying daemon mon.b on smithi022 2024-04-16T16:54:24.555 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.553+0000 7fe448517700 1 Processor -- start 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.553+0000 7fe448517700 1 -- start start 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe448517700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 0x7fe4400feb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe448517700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4400ff060 0x7fe4400ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe448517700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401095c0 con 0x7fe4400ff060 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe448517700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe440109440 con 0x7fe42c0035c0 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe4462b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 0x7fe4400feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe4462b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 0x7fe4400feb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33484/0 (socket says 172.21.15.43:33484) 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe4462b3700 1 -- 172.21.15.43:0/3254134820 learned_addr learned my addr 172.21.15.43:0/3254134820 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe4462b3700 1 -- 172.21.15.43:0/3254134820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 msgr2=0x7fe4400feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe4462b3700 1 -- 172.21.15.43:0/3254134820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 msgr2=0x7fe4400feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T16:54:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe4462b3700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 0x7fe4400feb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T16:54:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe4462b3700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 0x7fe4400feb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:54:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.554+0000 7fe445ab2700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4400ff060 0x7fe4400ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.555+0000 7fe445ab2700 1 -- 172.21.15.43:0/3254134820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 msgr2=0x7fe4400feb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:54:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.555+0000 7fe445ab2700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 0x7fe4400feb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.555+0000 7fe445ab2700 1 -- 172.21.15.43:0/3254134820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe440100a40 con 0x7fe4400ff060 2024-04-16T16:54:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.555+0000 7fe445ab2700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4400ff060 0x7fe4400ff4e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00b4d0 tx=0x7fe43c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.556+0000 7fe4377fe700 1 -- 172.21.15.43:0/3254134820 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe43c011630 con 0x7fe4400ff060 2024-04-16T16:54:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.556+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4401a4430 con 0x7fe4400ff060 2024-04-16T16:54:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.556+0000 7fe4377fe700 1 -- 172.21.15.43:0/3254134820 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43c011cb0 con 0x7fe4400ff060 2024-04-16T16:54:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.556+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4401a4970 con 0x7fe4400ff060 2024-04-16T16:54:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.556+0000 7fe4377fe700 1 -- 172.21.15.43:0/3254134820 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe43c010e60 con 0x7fe4400ff060 2024-04-16T16:54:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.557+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe440102de0 con 0x7fe4400ff060 2024-04-16T16:54:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.557+0000 7fe4377fe700 1 -- 172.21.15.43:0/3254134820 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe43c018020 con 0x7fe4400ff060 2024-04-16T16:54:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.558+0000 7fe4377fe700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe424039390 0x7fe42403b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.558+0000 7fe4377fe700 1 -- 172.21.15.43:0/3254134820 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe43c04c3b0 con 0x7fe4400ff060 2024-04-16T16:54:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.558+0000 7fe4462b3700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe424039390 0x7fe42403b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:24.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.559+0000 7fe4462b3700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe424039390 0x7fe42403b850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe43000fa60 tx=0x7fe430005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:24.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.573+0000 7fe4377fe700 1 -- 172.21.15.43:0/3254134820 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe43c01f020 con 0x7fe4400ff060 2024-04-16T16:54:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.868+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe44002dfe0 con 0x7fe4400ff060 2024-04-16T16:54:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.869+0000 7fe4377fe700 1 -- 172.21.15.43:0/3254134820 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fe43c01e1b0 con 0x7fe4400ff060 2024-04-16T16:54:24.871 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:54:24.871 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","modified":"2024-04-16T16:54:18.184557Z","created":"2024-04-16T16:52:10.664893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-16T16:54:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe424039390 msgr2=0x7fe42403b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe424039390 0x7fe42403b850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe43000fa60 tx=0x7fe430005c20 comp rx=0 tx=0).stop 2024-04-16T16:54:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4400ff060 msgr2=0x7fe4400ff4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4400ff060 0x7fe4400ff4e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00b4d0 tx=0x7fe43c00b890 comp rx=0 tx=0).stop 2024-04-16T16:54:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 shutdown_connections 2024-04-16T16:54:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe424039390 0x7fe42403b850 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe42c0035c0 0x7fe4400feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 --2- 172.21.15.43:0/3254134820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4400ff060 0x7fe4400ff4e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.871+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 >> 172.21.15.43:0/3254134820 conn(0x7fe440077970 msgr2=0x7fe440101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.872+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 shutdown_connections 2024-04-16T16:54:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:24.872+0000 7fe448517700 1 -- 172.21.15.43:0/3254134820 wait complete. 2024-04-16T16:54:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 2 2024-04-16T16:54:25.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:25 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/3254134820' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:25.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 16:54:25 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T16:54:25.185+0000 7f3875800700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-16T16:54:25.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:25 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/3254134820' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:26.487 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T16:54:26.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph mon dump -f json 2024-04-16T16:54:26.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:26 smithi018 ceph-mon[31776]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:54:26.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:26 smithi043 ceph-mon[36901]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 podman[36005]: 2024-04-16 16:54:27.397910488 +0000 UTC m=+0.334093495 container start a4591baa9f1d26bebcf50b1b60b30ce803393b55e62764d0ca2a03b402c84af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-16T16:54:27.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T16:54:27.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev), process ceph-mon, pid 2 2024-04-16T16:54:27.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pidfile_write: ignore empty --pid-file 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: load: jerasure load: lrc 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: RocksDB version: 7.9.2 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Git sha 0 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Compile date 2024-04-16 06:46:19 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: DB SUMMARY 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: DB Session ID: 5ADTI90B3542ICEMWJ5X 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: CURRENT file: CURRENT 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.error_if_exists: 0 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.create_if_missing: 0 2024-04-16T16:54:27.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.paranoid_checks: 1 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.env: 0x55c1c710bcc0 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.info_log: 0x55c1c8e9bc20 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.statistics: (nil) 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.use_fsync: 0 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_log_file_size: 0 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T16:54:27.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.allow_fallocate: 1 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.use_direct_reads: 0 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.db_log_dir: 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.wal_dir: 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T16:54:27.678 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.write_buffer_manager: 0x55c1c8e6a1e0 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.unordered_write: 0 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T16:54:27.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.row_cache: None 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.wal_filter: None 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.two_write_queues: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.wal_compression: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.atomic_flush: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.log_readahead_size: 0 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T16:54:27.680 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T16:54:27.681 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T16:54:27.681 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T16:54:27.681 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T16:54:27.681 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T16:54:27.681 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T16:54:27.682 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_background_jobs: 2 2024-04-16T16:54:27.682 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_background_compactions: -1 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_subcompactions: 1 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_open_files: -1 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T16:54:27.683 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_background_flushes: -1 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Compression algorithms supported: 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: kZSTD supported: 0 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: kXpressCompression supported: 0 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: kZlibCompression supported: 1 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: kSnappyCompression supported: 1 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: kLZ4Compression supported: 1 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: kBZip2Compression supported: 0 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.merge_operator: 2024-04-16T16:54:27.684 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_filter: None 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c1c8e9bde0) 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: index_type: 0 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: data_block_index_type: 0 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: index_shortening: 1 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: checksum: 4 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: no_block_cache: 0 2024-04-16T16:54:27.685 INFO:journalctl@ceph.mon.b.smithi022.stdout: block_cache: 0x55c1c8e9d090 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: block_cache_options: 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: capacity : 536870912 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: num_shard_bits : 4 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: strict_capacity_limit : 0 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: block_cache_compressed: (nil) 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: persistent_cache: (nil) 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: block_size: 4096 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: block_size_deviation: 10 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: block_restart_interval: 16 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: index_block_restart_interval: 1 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: metadata_block_size: 4096 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: partition_filters: 0 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: use_delta_encoding: 1 2024-04-16T16:54:27.686 INFO:journalctl@ceph.mon.b.smithi022.stdout: filter_policy: bloomfilter 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: whole_key_filtering: 1 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: verify_compression: 0 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: format_version: 5 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: enable_index_compression: 1 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: block_align: 0 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: max_auto_readahead_size: 262144 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: prepopulate_block_cache: 0 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: initial_auto_readahead_size: 8192 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression: NoCompression 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T16:54:27.687 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.num_levels: 7 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T16:54:27.688 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T16:54:27.689 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T16:54:27.690 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.inplace_update_support: 0 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.bloom_locality: 0 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.max_successive_merges: 0 2024-04-16T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.ttl: 2592000 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.enable_blob_files: false 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.min_blob_size: 0 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T16:54:27.692 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 500c5b8d-12f5-4ea0-a6e6-540b9b890649 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286467428505, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286467428950, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713286467, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "500c5b8d-12f5-4ea0-a6e6-540b9b890649", "db_session_id": "5ADTI90B3542ICEMWJ5X", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286467429040, "job": 1, "event": "recovery_finished"} 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c1c8f8a000 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: DB pointer 0x55c1c8f74000 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout: ** DB Stats ** 2024-04-16T16:54:27.693 INFO:journalctl@ceph.mon.b.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: ** Compaction Stats [default] ** 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: 2024-04-16T16:54:27.694 INFO:journalctl@ceph.mon.b.smithi022.stdout: ** Compaction Stats [default] ** 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T16:54:27.695 INFO:journalctl@ceph.mon.b.smithi022.stdout: Block cache BinnedLRUCache@0x55c1c8e9d090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: using public_addr v2:172.21.15.22:0/0 -> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: starting mon.b rank -1 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(???) e0 preinit fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).mds e1 new map 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: e1 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: legacy client fscid: -1 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout: No filesystems configured 2024-04-16T16:54:27.696 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.698 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Deploying cephadm binary to smithi043 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Added host smithi043 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.699 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/827336876' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/827336876' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.700 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c;count:3 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Deploying daemon mon.c on smithi043 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/3024700037' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.a calling monitor election 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:27.701 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.c calling monitor election 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: monmap epoch 2 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: last_changed 2024-04-16T16:54:18.184557+0000 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: min_mon_release 19 (squid) 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: election_strategy: 1 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:27.702 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: fsmap 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mgrmap e13: x(active, since 58s) 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: overall HEALTH_OK 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: Deploying daemon mon.b on smithi022 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/3254134820' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 ceph-mon[36023]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-16T16:54:27.703 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 bash[36005]: a4591baa9f1d26bebcf50b1b60b30ce803393b55e62764d0ca2a03b402c84af9 2024-04-16T16:54:27.704 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:27 smithi022 systemd[1]: Started Ceph mon.b for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.705+0000 7fd212728700 1 -- 172.21.15.43:0/2383056874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 msgr2=0x7fd1f4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.705+0000 7fd212728700 1 --2- 172.21.15.43:0/2383056874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 0x7fd1f4005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd20c101430 tx=0x7fd1fc00fe60 comp rx=0 tx=0).stop 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.706+0000 7fd212728700 1 -- 172.21.15.43:0/2383056874 shutdown_connections 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.706+0000 7fd212728700 1 --2- 172.21.15.43:0/2383056874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 0x7fd1f4005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.706+0000 7fd212728700 1 -- 172.21.15.43:0/2383056874 >> 172.21.15.43:0/2383056874 conn(0x7fd20c0fd030 msgr2=0x7fd20c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.706+0000 7fd212728700 1 -- 172.21.15.43:0/2383056874 shutdown_connections 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.706+0000 7fd212728700 1 -- 172.21.15.43:0/2383056874 wait complete. 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.707+0000 7fd212728700 1 Processor -- start 2024-04-16T16:54:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.707+0000 7fd212728700 1 -- start start 2024-04-16T16:54:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd212728700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 0x7fd20c10a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd212728700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 0x7fd20c10b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd212728700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 0x7fd20c1a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd212728700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c109550 con 0x7fd1f40035c0 2024-04-16T16:54:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd212728700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c1093d0 con 0x7fd20c1a43e0 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd212728700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c1096d0 con 0x7fd20c10af20 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd20b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 0x7fd20c10b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd20b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 0x7fd20c10b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42692/0 (socket says 172.21.15.43:42692) 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd20b7fe700 1 -- 172.21.15.43:0/1238342273 learned_addr learned my addr 172.21.15.43:0/1238342273 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd210cc5700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 0x7fd20c1a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd20b7fe700 1 -- 172.21.15.43:0/1238342273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 msgr2=0x7fd20c10b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd20b7fe700 1 -- 172.21.15.43:0/1238342273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 msgr2=0x7fd20c10b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd20b7fe700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 0x7fd20c10b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd20b7fe700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 0x7fd20c10b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd20bfff700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 0x7fd20c10a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd210cc5700 1 -- 172.21.15.43:0/1238342273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 msgr2=0x7fd20c1a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd210cc5700 1 -- 172.21.15.43:0/1238342273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 msgr2=0x7fd20c1a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd210cc5700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 0x7fd20c1a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T16:54:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.708+0000 7fd210cc5700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 0x7fd20c1a67d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd20bfff700 1 -- 172.21.15.43:0/1238342273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 msgr2=0x7fd20c10b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd20bfff700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 0x7fd20c10b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd20bfff700 1 -- 172.21.15.43:0/1238342273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 msgr2=0x7fd20c1a67d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd20bfff700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 0x7fd20c1a67d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd20bfff700 1 -- 172.21.15.43:0/1238342273 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c1a6f00 con 0x7fd1f40035c0 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd20bfff700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 0x7fd20c10a9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd20c101430 tx=0x7fd1fc014fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd2097fa700 1 -- 172.21.15.43:0/1238342273 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc022040 con 0x7fd1f40035c0 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd2097fa700 1 -- 172.21.15.43:0/1238342273 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1fc02c070 con 0x7fd1f40035c0 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.709+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c1a7190 con 0x7fd1f40035c0 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.710+0000 7fd2097fa700 1 -- 172.21.15.43:0/1238342273 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0042b0 con 0x7fd1f40035c0 2024-04-16T16:54:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.710+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c1a7720 con 0x7fd1f40035c0 2024-04-16T16:54:32.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.711+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c064150 con 0x7fd1f40035c0 2024-04-16T16:54:32.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.711+0000 7fd2097fa700 1 -- 172.21.15.43:0/1238342273 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd1fc043910 con 0x7fd1f40035c0 2024-04-16T16:54:32.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.711+0000 7fd2097fa700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1f8039ae0 0x7fd1f803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:32.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.711+0000 7fd2097fa700 1 -- 172.21.15.43:0/1238342273 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd1fc04ef90 con 0x7fd1f40035c0 2024-04-16T16:54:32.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.712+0000 7fd20b7fe700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1f8039ae0 0x7fd1f803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:32.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.713+0000 7fd20b7fe700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1f8039ae0 0x7fd1f803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd200009180 tx=0x7fd200008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:32.720+0000 7fd2097fa700 1 -- 172.21.15.43:0/1238342273 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1fc004450 con 0x7fd1f40035c0 2024-04-16T16:54:32.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: mon.c calling monitor election 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: mon.a calling monitor election 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: monmap epoch 3 2024-04-16T16:54:32.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: last_changed 2024-04-16T16:54:27.452328+0000 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: min_mon_release 19 (squid) 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: election_strategy: 1 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: 2: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.b 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: fsmap 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: mgrmap e13: x(active, since 67s) 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: overall HEALTH_OK 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:32.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: mon.c calling monitor election 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: mon.a calling monitor election 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:33.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: monmap epoch 3 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: last_changed 2024-04-16T16:54:27.452328+0000 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: min_mon_release 19 (squid) 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: election_strategy: 1 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: 2: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.b 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: fsmap 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: mgrmap e13: x(active, since 67s) 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: overall HEALTH_OK 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:33.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.024+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd20c1a7ac0 con 0x7fd1f40035c0 2024-04-16T16:54:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.024+0000 7fd2097fa700 1 -- 172.21.15.43:0/1238342273 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fd1fc012ca0 con 0x7fd1f40035c0 2024-04-16T16:54:33.026 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-16T16:54:33.026 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","modified":"2024-04-16T16:54:27.452328Z","created":"2024-04-16T16:52:10.664893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1f8039ae0 msgr2=0x7fd1f803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1f8039ae0 0x7fd1f803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd200009180 tx=0x7fd200008040 comp rx=0 tx=0).stop 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 msgr2=0x7fd20c10a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 0x7fd20c10a9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd20c101430 tx=0x7fd1fc014fc0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 shutdown_connections 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1f8039ae0 0x7fd1f803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd20c1a43e0 0x7fd20c1a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1f40035c0 0x7fd20c10a9e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 --2- 172.21.15.43:0/1238342273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20c10af20 0x7fd20c10b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 >> 172.21.15.43:0/1238342273 conn(0x7fd20c0fd030 msgr2=0x7fd20c06b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 shutdown_connections 2024-04-16T16:54:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:33.027+0000 7fd212728700 1 -- 172.21.15.43:0/1238342273 wait complete. 2024-04-16T16:54:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 3 2024-04-16T16:54:33.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:33.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:33.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:33 smithi043 ceph-mon[36901]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:33.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:33.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:33 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/1238342273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:33.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:33.666 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-16T16:54:33.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph config generate-minimal-conf 2024-04-16T16:54:33.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:33.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:33.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:33 smithi018 ceph-mon[31776]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:33 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/1238342273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:34.123 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: mon.c calling monitor election 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: mon.a calling monitor election 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: monmap epoch 3 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: last_changed 2024-04-16T16:54:27.452328+0000 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: min_mon_release 19 (squid) 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: election_strategy: 1 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: 2: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.b 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: fsmap 2024-04-16T16:54:34.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: mgrmap e13: x(active, since 67s) 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: overall HEALTH_OK 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/1238342273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:54:34.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:35.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: mon.b calling monitor election 2024-04-16T16:54:35.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-16T16:54:35.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: mon.b calling monitor election 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: mon.c calling monitor election 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: mon.a calling monitor election 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: monmap epoch 3 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: last_changed 2024-04-16T16:54:27.452328+0000 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: min_mon_release 19 (squid) 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: election_strategy: 1 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: 2: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.b 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: fsmap 2024-04-16T16:54:35.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:35.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: mgrmap e13: x(active, since 69s) 2024-04-16T16:54:35.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: overall HEALTH_OK 2024-04-16T16:54:35.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:35 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: mon.b calling monitor election 2024-04-16T16:54:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-16T16:54:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T16:54:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: mon.b calling monitor election 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: mon.c calling monitor election 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: mon.a calling monitor election 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: monmap epoch 3 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: last_changed 2024-04-16T16:54:27.452328+0000 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: min_mon_release 19 (squid) 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: election_strategy: 1 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: 2: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.b 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: fsmap 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: mgrmap e13: x(active, since 69s) 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: overall HEALTH_OK 2024-04-16T16:54:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:35 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: mon.b calling monitor election 2024-04-16T16:54:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-16T16:54:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T16:54:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T16:54:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: mon.b calling monitor election 2024-04-16T16:54:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: mon.c calling monitor election 2024-04-16T16:54:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: mon.a calling monitor election 2024-04-16T16:54:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: monmap epoch 3 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: last_changed 2024-04-16T16:54:27.452328+0000 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: created 2024-04-16T16:52:10.664893+0000 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: min_mon_release 19 (squid) 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: election_strategy: 1 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.c 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: 2: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.b 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: fsmap 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: mgrmap e13: x(active, since 69s) 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: overall HEALTH_OK 2024-04-16T16:54:35.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:35.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:35 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.217+0000 7f7482e00700 1 -- 172.21.15.18:0/17092221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f747c108580 msgr2=0x7f747c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:36.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.217+0000 7f7482e00700 1 --2- 172.21.15.18:0/17092221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f747c108580 0x7f747c108960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f746c009910 tx=0x7f746c009c20 comp rx=0 tx=0).stop 2024-04-16T16:54:36.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.217+0000 7f7482e00700 1 -- 172.21.15.18:0/17092221 shutdown_connections 2024-04-16T16:54:36.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.217+0000 7f7482e00700 1 --2- 172.21.15.18:0/17092221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f747c108580 0x7f747c108960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.217+0000 7f7482e00700 1 -- 172.21.15.18:0/17092221 >> 172.21.15.18:0/17092221 conn(0x7f747c076f50 msgr2=0x7f747c077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:36.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.217+0000 7f7482e00700 1 -- 172.21.15.18:0/17092221 shutdown_connections 2024-04-16T16:54:36.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.217+0000 7f7482e00700 1 -- 172.21.15.18:0/17092221 wait complete. 2024-04-16T16:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.219+0000 7f7482e00700 1 Processor -- start 2024-04-16T16:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.219+0000 7f7482e00700 1 -- start start 2024-04-16T16:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.219+0000 7f7482e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f747c108580 0x7f747c101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.219+0000 7f7482e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c101a50 0x7f747c101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.219+0000 7f7482e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f747c1a43f0 0x7f747c1a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.220+0000 7f7482e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f747c10ca70 con 0x7f747c1a43f0 2024-04-16T16:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.220+0000 7f7482e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f747c10c8f0 con 0x7f747c108580 2024-04-16T16:54:36.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.220+0000 7f7482e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f747c10cbf0 con 0x7f747c101a50 2024-04-16T16:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.220+0000 7f747be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f747c108580 0x7f747c101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.220+0000 7f747be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f747c108580 0x7f747c101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55458/0 (socket says 172.21.15.18:55458) 2024-04-16T16:54:36.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.220+0000 7f747be00700 1 -- 172.21.15.18:0/2862304773 learned_addr learned my addr 172.21.15.18:0/2862304773 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.220+0000 7f7480e00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f747c1a43f0 0x7f747c1a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.220+0000 7f747b400700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c101a50 0x7f747c101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f747be00700 1 -- 172.21.15.18:0/2862304773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c101a50 msgr2=0x7f747c101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f747be00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c101a50 0x7f747c101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f747be00700 1 -- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f747c1a43f0 msgr2=0x7f747c1a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f747be00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f747c1a43f0 0x7f747c1a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f747be00700 1 -- 172.21.15.18:0/2862304773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c1a6f10 con 0x7f747c108580 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f747be00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f747c108580 0x7f747c101510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f746c00bdc0 tx=0x7f746c00bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:36.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f7478c00700 1 -- 172.21.15.18:0/2862304773 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c026070 con 0x7f747c108580 2024-04-16T16:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f7478c00700 1 -- 172.21.15.18:0/2862304773 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f746c005da0 con 0x7f747c108580 2024-04-16T16:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747c1a7200 con 0x7f747c108580 2024-04-16T16:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.221+0000 7f7478c00700 1 -- 172.21.15.18:0/2862304773 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c0038e0 con 0x7f747c108580 2024-04-16T16:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.222+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747c1a7740 con 0x7f747c108580 2024-04-16T16:54:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.223+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f747c064150 con 0x7f747c108580 2024-04-16T16:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.223+0000 7f7478c00700 1 -- 172.21.15.18:0/2862304773 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f746c029070 con 0x7f747c108580 2024-04-16T16:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.224+0000 7f7478c00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7460039ae0 0x7f746003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.224+0000 7f7478c00700 1 -- 172.21.15.18:0/2862304773 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f746c04d460 con 0x7f747c108580 2024-04-16T16:54:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.224+0000 7f747b400700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7460039ae0 0x7f746003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:36.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.225+0000 7f747b400700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7460039ae0 0x7f746003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7468009ad0 tx=0x7f7468009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:36.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.229+0000 7f7478c00700 1 -- 172.21.15.18:0/2862304773 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f746c013070 con 0x7f747c108580 2024-04-16T16:54:36.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.473+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f747c1a7a60 con 0x7f747c108580 2024-04-16T16:54:36.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.473+0000 7f7478c00700 1 -- 172.21.15.18:0/2862304773 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f746c030090 con 0x7f747c108580 2024-04-16T16:54:36.475 INFO:teuthology.orchestra.run.smithi018.stdout:# minimal ceph.conf for 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:36.475 INFO:teuthology.orchestra.run.smithi018.stdout:[global] 2024-04-16T16:54:36.475 INFO:teuthology.orchestra.run.smithi018.stdout: fsid = 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:36.475 INFO:teuthology.orchestra.run.smithi018.stdout: mon_host = [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] 2024-04-16T16:54:36.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7460039ae0 msgr2=0x7f746003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:36.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7460039ae0 0x7f746003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7468009ad0 tx=0x7f7468009210 comp rx=0 tx=0).stop 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f747c108580 msgr2=0x7f747c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f747c108580 0x7f747c101510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f746c00bdc0 tx=0x7f746c00bdf0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 shutdown_connections 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7460039ae0 0x7f746003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f747c108580 0x7f747c101510 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f747c1a43f0 0x7f747c1a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 --2- 172.21.15.18:0/2862304773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c101a50 0x7f747c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 >> 172.21.15.18:0/2862304773 conn(0x7f747c076f50 msgr2=0x7f747c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 shutdown_connections 2024-04-16T16:54:36.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:36.477+0000 7f7482e00700 1 -- 172.21.15.18:0/2862304773 wait complete. 2024-04-16T16:54:36.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:36.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:36.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:36.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:36.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:36.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:36.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:37.219 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-16T16:54:37.220 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:54:37.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T16:54:37.313 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:54:37.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:37.439 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:54:37.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T16:54:37.479 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:54:37.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:37.553 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:54:37.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T16:54:37.592 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:54:37.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:37.668 INFO:tasks.cephadm:Adding mgr.x on smithi018 2024-04-16T16:54:37.668 INFO:tasks.cephadm:Adding mgr.y on smithi022 2024-04-16T16:54:37.668 INFO:tasks.cephadm:Adding mgr.z on smithi043 2024-04-16T16:54:37.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch apply mgr '3;smithi018=x;smithi022=y;smithi043=z' 2024-04-16T16:54:37.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:37 smithi018 ceph-mon[31776]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T16:54:37.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:37 smithi018 ceph-mon[31776]: Reconfiguring daemon mon.a on smithi018 2024-04-16T16:54:37.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:37 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2862304773' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:37.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:37 smithi043 ceph-mon[36901]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T16:54:37.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:37 smithi043 ceph-mon[36901]: Reconfiguring daemon mon.a on smithi018 2024-04-16T16:54:37.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:37 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2862304773' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:37.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:37 smithi022 ceph-mon[36023]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T16:54:37.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:37 smithi022 ceph-mon[36023]: Reconfiguring daemon mon.a on smithi018 2024-04-16T16:54:37.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:37 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2862304773' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:54:38.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:38 smithi018 ceph-mon[31776]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:38.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:38 smithi043 ceph-mon[36901]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:38.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:38 smithi022 ceph-mon[36023]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 -- 172.21.15.43:0/3609505479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308102680 msgr2=0x7f4308102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 --2- 172.21.15.43:0/3609505479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308102680 0x7f4308102aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4304009910 tx=0x7f4304009c20 comp rx=0 tx=0).stop 2024-04-16T16:54:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 -- 172.21.15.43:0/3609505479 shutdown_connections 2024-04-16T16:54:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 --2- 172.21.15.43:0/3609505479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308102680 0x7f4308102aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 -- 172.21.15.43:0/3609505479 >> 172.21.15.43:0/3609505479 conn(0x7f43080fdc20 msgr2=0x7f4308100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 -- 172.21.15.43:0/3609505479 shutdown_connections 2024-04-16T16:54:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 -- 172.21.15.43:0/3609505479 wait complete. 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 Processor -- start 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.519+0000 7f430fa31700 1 -- start start 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430fa31700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308102680 0x7f4308076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430fa31700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4308074ee0 0x7f4308075360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430fa31700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43080758a0 0x7f43081a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430fa31700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4308108710 con 0x7f4308102680 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430fa31700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4308108590 con 0x7f43080758a0 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430fa31700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4308108890 con 0x7f4308074ee0 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4308074ee0 0x7f4308075360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4308074ee0 0x7f4308075360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42702/0 (socket says 172.21.15.43:42702) 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 -- 172.21.15.43:0/1873930474 learned_addr learned my addr 172.21.15.43:0/1873930474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430dfce700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43080758a0 0x7f43081a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430d7cd700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308102680 0x7f4308076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 -- 172.21.15.43:0/1873930474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43080758a0 msgr2=0x7f43081a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43080758a0 0x7f43081a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 -- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308102680 msgr2=0x7f4308076890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308102680 0x7f4308076890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 -- 172.21.15.43:0/1873930474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43081a24c0 con 0x7f4308074ee0 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f430cfcc700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4308074ee0 0x7f4308075360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42f800b780 tx=0x7f42f800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f42fe7fc700 1 -- 172.21.15.43:0/1873930474 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8004230 con 0x7f4308074ee0 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.520+0000 7f42fe7fc700 1 -- 172.21.15.43:0/1873930474 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f80043d0 con 0x7f4308074ee0 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.521+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43081a27b0 con 0x7f4308074ee0 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.521+0000 7f42fe7fc700 1 -- 172.21.15.43:0/1873930474 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8011c90 con 0x7f4308074ee0 2024-04-16T16:54:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.521+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43081a2dc0 con 0x7f4308074ee0 2024-04-16T16:54:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.521+0000 7f42fe7fc700 1 -- 172.21.15.43:0/1873930474 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f42f80036a0 con 0x7f4308074ee0 2024-04-16T16:54:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.522+0000 7f42fe7fc700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f4039ae0 0x7f42f403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.522+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4308064150 con 0x7f4308074ee0 2024-04-16T16:54:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.522+0000 7f42fe7fc700 1 -- 172.21.15.43:0/1873930474 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f42f804c320 con 0x7f4308074ee0 2024-04-16T16:54:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.522+0000 7f430d7cd700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f4039ae0 0x7f42f403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.523+0000 7f430d7cd700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f4039ae0 0x7f42f403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4304005e00 tx=0x7f43040052d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:39.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.527+0000 7f42fe7fc700 1 -- 172.21.15.43:0/1873930474 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42f8021e20 con 0x7f4308074ee0 2024-04-16T16:54:39.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.747+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi018=x;smithi022=y;smithi043=z", "target": ["mon-mgr", ""]}) -- 0x7f4308062f30 con 0x7f42f4039ae0 2024-04-16T16:54:39.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:39.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:39.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:39.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:39.749 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:39.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.760+0000 7f42fe7fc700 1 -- 172.21.15.43:0/1873930474 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4308062f30 con 0x7f42f4039ae0 2024-04-16T16:54:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:Scheduled mgr update... 2024-04-16T16:54:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.762+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f4039ae0 msgr2=0x7f42f403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.762+0000 7f430fa31700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f4039ae0 0x7f42f403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4304005e00 tx=0x7f43040052d0 comp rx=0 tx=0).stop 2024-04-16T16:54:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4308074ee0 msgr2=0x7f4308075360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4308074ee0 0x7f4308075360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42f800b780 tx=0x7f42f800ba90 comp rx=0 tx=0).stop 2024-04-16T16:54:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 shutdown_connections 2024-04-16T16:54:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f4039ae0 0x7f42f403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43080758a0 0x7f43081a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308102680 0x7f4308076890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 --2- 172.21.15.43:0/1873930474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4308074ee0 0x7f4308075360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 >> 172.21.15.43:0/1873930474 conn(0x7f43080fdc20 msgr2=0x7f43080fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 shutdown_connections 2024-04-16T16:54:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:54:39.763+0000 7f430fa31700 1 -- 172.21.15.43:0/1873930474 wait complete. 2024-04-16T16:54:39.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:39.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:39.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:39.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:39.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:40.412 DEBUG:teuthology.orchestra.run.smithi022:mgr.y> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.y.service 2024-04-16T16:54:40.415 DEBUG:teuthology.orchestra.run.smithi043:mgr.z> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.z.service 2024-04-16T16:54:40.455 INFO:tasks.cephadm:Deploying OSDs... 2024-04-16T16:54:40.455 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T16:54:40.455 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T16:54:40.518 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T16:54:40.518 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_1 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-16 16:53:37.489475516 +0000 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-16 16:48:27.475164868 +0000 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-16 16:48:27.475164868 +0000 2024-04-16T16:54:40.573 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-16 16:48:27.475164868 +0000 2024-04-16T16:54:40.574 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T16:54:40.661 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-04-16T16:54:40.661 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-04-16T16:54:40.661 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000300115 s, 1.7 MB/s 2024-04-16T16:54:40.664 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T16:54:40.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: Reconfiguring mon.b (monmap changed)... 2024-04-16T16:54:40.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: Reconfiguring daemon mon.b on smithi022 2024-04-16T16:54:40.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:40.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:40.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:40.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:40.721 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_2 2024-04-16T16:54:40.776 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T16:54:40.776 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:40.776 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-16T16:54:40.776 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:40.777 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:40.777 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-16 16:53:37.562474636 +0000 2024-04-16T16:54:40.777 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-16 16:48:28.103157460 +0000 2024-04-16T16:54:40.777 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-16 16:48:28.103157460 +0000 2024-04-16T16:54:40.777 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-16 16:48:28.103157460 +0000 2024-04-16T16:54:40.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T16:54:40.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: Reconfiguring mon.b (monmap changed)... 2024-04-16T16:54:40.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: Reconfiguring daemon mon.b on smithi022 2024-04-16T16:54:40.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:40.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-04-16T16:54:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-04-16T16:54:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000281914 s, 1.8 MB/s 2024-04-16T16:54:40.875 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: Reconfiguring mon.b (monmap changed)... 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: Reconfiguring daemon mon.b on smithi022 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:54:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:40.936 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_3 2024-04-16T16:54:40.988 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T16:54:40.989 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:40.989 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-16T16:54:40.989 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:40.989 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:40.989 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-16 16:53:37.618473960 +0000 2024-04-16T16:54:40.989 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-16 16:48:28.749149840 +0000 2024-04-16T16:54:40.989 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-16 16:48:28.749149840 +0000 2024-04-16T16:54:40.989 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-16 16:48:28.749149840 +0000 2024-04-16T16:54:40.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T16:54:41.082 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-04-16T16:54:41.082 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-04-16T16:54:41.082 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.00026169 s, 2.0 MB/s 2024-04-16T16:54:41.086 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T16:54:41.145 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_4 2024-04-16T16:54:41.199 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T16:54:41.199 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:41.199 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-16T16:54:41.199 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:41.200 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:41.200 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-16 16:53:37.683473176 +0000 2024-04-16T16:54:41.200 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-16 16:48:29.396142209 +0000 2024-04-16T16:54:41.200 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-16 16:48:29.396142209 +0000 2024-04-16T16:54:41.200 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-16 16:48:29.396142209 +0000 2024-04-16T16:54:41.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T16:54:41.298 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-04-16T16:54:41.298 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-04-16T16:54:41.298 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000286844 s, 1.8 MB/s 2024-04-16T16:54:41.302 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T16:54:41.364 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-16T16:54:41.364 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T16:54:41.384 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T16:54:41.384 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-16 16:54:26.887592790 +0000 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-16 16:47:10.200784519 +0000 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-16 16:47:10.200784519 +0000 2024-04-16T16:54:41.442 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-16 16:47:10.200784519 +0000 2024-04-16T16:54:41.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T16:54:41.515 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-16T16:54:41.515 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-16T16:54:41.515 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000118958 s, 4.3 MB/s 2024-04-16T16:54:41.516 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T16:54:41.574 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-04-16T16:54:41.631 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T16:54:41.631 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:41.631 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-16T16:54:41.631 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:41.631 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:41.632 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-16 16:54:26.887592790 +0000 2024-04-16T16:54:41.632 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-16 16:47:10.540780497 +0000 2024-04-16T16:54:41.632 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-16 16:47:10.540780497 +0000 2024-04-16T16:54:41.632 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-16 16:47:10.540780497 +0000 2024-04-16T16:54:41.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T16:54:41.702 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-16T16:54:41.702 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-16T16:54:41.703 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000115459 s, 4.4 MB/s 2024-04-16T16:54:41.706 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T16:54:41.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:41 smithi043 ceph-mon[36901]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi018=x;smithi022=y;smithi043=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:41.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:41 smithi043 ceph-mon[36901]: Saving service mgr spec with placement smithi018=x;smithi022=y;smithi043=z;count:3 2024-04-16T16:54:41.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:41 smithi043 ceph-mon[36901]: Reconfiguring mon.c (monmap changed)... 2024-04-16T16:54:41.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:41 smithi043 ceph-mon[36901]: Reconfiguring daemon mon.c on smithi043 2024-04-16T16:54:41.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:41.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:41.764 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-16 16:54:26.887592790 +0000 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-16 16:47:10.881776462 +0000 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-16 16:47:10.881776462 +0000 2024-04-16T16:54:41.820 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-16 16:47:10.881776462 +0000 2024-04-16T16:54:41.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T16:54:41.888 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:41 smithi022 ceph-mon[36023]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi018=x;smithi022=y;smithi043=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:41.888 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:41 smithi022 ceph-mon[36023]: Saving service mgr spec with placement smithi018=x;smithi022=y;smithi043=z;count:3 2024-04-16T16:54:41.888 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:41 smithi022 ceph-mon[36023]: Reconfiguring mon.c (monmap changed)... 2024-04-16T16:54:41.888 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:41 smithi022 ceph-mon[36023]: Reconfiguring daemon mon.c on smithi043 2024-04-16T16:54:41.888 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:41.888 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:41.890 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-16T16:54:41.890 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-16T16:54:41.890 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000114506 s, 4.5 MB/s 2024-04-16T16:54:41.891 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T16:54:41.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:41 smithi018 ceph-mon[31776]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi018=x;smithi022=y;smithi043=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:41.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:41 smithi018 ceph-mon[31776]: Saving service mgr spec with placement smithi018=x;smithi022=y;smithi043=z;count:3 2024-04-16T16:54:41.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:41 smithi018 ceph-mon[31776]: Reconfiguring mon.c (monmap changed)... 2024-04-16T16:54:41.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:41 smithi018 ceph-mon[31776]: Reconfiguring daemon mon.c on smithi043 2024-04-16T16:54:41.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:41.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:41.948 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-04-16T16:54:42.004 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T16:54:42.004 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:42.004 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-16T16:54:42.004 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:42.004 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:42.004 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-16 16:54:26.888592778 +0000 2024-04-16T16:54:42.004 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-16 16:47:11.217772487 +0000 2024-04-16T16:54:42.005 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-16 16:47:11.217772487 +0000 2024-04-16T16:54:42.005 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-16 16:47:11.217772487 +0000 2024-04-16T16:54:42.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T16:54:42.073 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-16T16:54:42.073 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-16T16:54:42.073 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000106582 s, 4.8 MB/s 2024-04-16T16:54:42.074 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T16:54:42.131 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-16T16:54:42.131 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T16:54:42.147 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T16:54:42.147 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-04-16T16:54:42.202 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T16:54:42.202 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:42.202 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-04-16T16:54:42.202 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:42.202 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:42.202 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-16 16:54:17.474901046 +0000 2024-04-16T16:54:42.202 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-16 16:47:12.747465841 +0000 2024-04-16T16:54:42.202 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-16 16:47:12.747465841 +0000 2024-04-16T16:54:42.203 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-16 16:47:12.747465841 +0000 2024-04-16T16:54:42.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T16:54:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-16T16:54:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-16T16:54:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 9.8311e-05 s, 5.2 MB/s 2024-04-16T16:54:42.273 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T16:54:42.328 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-04-16T16:54:42.383 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T16:54:42.384 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:42.384 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-16T16:54:42.384 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:42.384 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:42.384 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-16 16:54:17.474901046 +0000 2024-04-16T16:54:42.384 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-16 16:47:13.086462077 +0000 2024-04-16T16:54:42.384 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-16 16:47:13.086462077 +0000 2024-04-16T16:54:42.384 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-16 16:47:13.086462077 +0000 2024-04-16T16:54:42.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T16:54:42.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-16T16:54:42.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-16T16:54:42.453 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 9.456e-05 s, 5.4 MB/s 2024-04-16T16:54:42.454 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T16:54:42.509 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-04-16T16:54:42.565 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T16:54:42.565 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:42.565 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-04-16T16:54:42.565 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:42.565 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:42.565 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-16 16:54:17.474901046 +0000 2024-04-16T16:54:42.565 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-16 16:47:13.443458113 +0000 2024-04-16T16:54:42.565 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-16 16:47:13.443458113 +0000 2024-04-16T16:54:42.566 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-16 16:47:13.443458113 +0000 2024-04-16T16:54:42.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T16:54:42.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-16T16:54:42.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-16T16:54:42.634 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00010946 s, 4.7 MB/s 2024-04-16T16:54:42.636 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T16:54:42.691 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-04-16T16:54:42.746 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T16:54:42.746 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:54:42.746 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-16T16:54:42.747 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:54:42.747 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:54:42.747 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-16 16:54:17.475901035 +0000 2024-04-16T16:54:42.747 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-16 16:47:13.782454348 +0000 2024-04-16T16:54:42.747 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-16 16:47:13.782454348 +0000 2024-04-16T16:54:42.747 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-16 16:47:13.782454348 +0000 2024-04-16T16:54:42.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T16:54:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:54:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:42 smithi043 ceph-mon[36901]: Deploying daemon mgr.z on smithi043 2024-04-16T16:54:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-16T16:54:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-16T16:54:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 8.8842e-05 s, 5.8 MB/s 2024-04-16T16:54:42.817 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T16:54:42.872 INFO:tasks.cephadm:Deploying osd.0 on smithi018 with /dev/vg_nvme/lv_4... 2024-04-16T16:54:42.873 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T16:54:42.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:42.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:42.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:54:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:42 smithi022 ceph-mon[36023]: Deploying daemon mgr.z on smithi043 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:42.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:42 smithi018 ceph-mon[31776]: Deploying daemon mgr.z on smithi043 2024-04-16T16:54:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:54:43.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:43 smithi043 ceph-mon[36901]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:43.753 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:43 smithi043 systemd[1]: Starting Ceph mgr.z for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:54:43.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:43 smithi018 ceph-mon[31776]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:43.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:43 smithi022 ceph-mon[36023]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:44.133 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:43 smithi043 podman[38752]: 2024-04-16 16:54:43.778382683 +0000 UTC m=+0.018066688 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T16:54:44.134 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:43 smithi043 podman[38752]: 2024-04-16 16:54:43.904725923 +0000 UTC m=+0.144409934 container create 4749a67dbb9e422736ddd1b0a07e9ff26a70c2bb3071575a7c3b3a3b8446ece3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=) 2024-04-16T16:54:44.134 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:44 smithi043 podman[38752]: 2024-04-16 16:54:44.105951373 +0000 UTC m=+0.345635365 container init 4749a67dbb9e422736ddd1b0a07e9ff26a70c2bb3071575a7c3b3a3b8446ece3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T16:54:44.134 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:44 smithi043 podman[38752]: 2024-04-16 16:54:44.108553421 +0000 UTC m=+0.348237406 container start 4749a67dbb9e422736ddd1b0a07e9ff26a70c2bb3071575a7c3b3a3b8446ece3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-16T16:54:44.502 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:44 smithi043 ceph-mgr[38772]: -- 172.21.15.43:0/355040963 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cdd19e65a0 con 0x55cdd19bac00 2024-04-16T16:54:44.502 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:44 smithi043 bash[38752]: 4749a67dbb9e422736ddd1b0a07e9ff26a70c2bb3071575a7c3b3a3b8446ece3 2024-04-16T16:54:44.502 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:44 smithi043 systemd[1]: Started Ceph mgr.z for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:54:45.002 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:44.672+0000 7f2732e46200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:54:45.002 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:44.769+0000 7f2732e46200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:54:45.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:54:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:45 smithi018 ceph-mon[31776]: Deploying daemon mgr.y on smithi022 2024-04-16T16:54:45.532 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:45.444+0000 7f2732e46200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:54:45.532 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.532 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.533 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.533 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:45.533 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:54:45.533 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:45.533 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:45.533 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-mon[36901]: Deploying daemon mgr.y on smithi022 2024-04-16T16:54:45.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:45.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:45.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:54:45.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:45.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:45.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:45 smithi022 ceph-mon[36023]: Deploying daemon mgr.y on smithi022 2024-04-16T16:54:45.795 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:45.542+0000 7f2732e46200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:54:45.795 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:45.693+0000 7f2732e46200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:54:45.796 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:45.794+0000 7f2732e46200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:54:45.986 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T16:54:45.986 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T16:54:45.987 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-04-16T16:54:45.987 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-04-16T16:54:45.987 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.035198 s, 298 MB/s 2024-04-16T16:54:45.987 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-04-16T16:54:46.159 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:45.881+0000 7f2732e46200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:54:46.159 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:46.157+0000 7f2732e46200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:54:46.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:46 smithi043 ceph-mon[36901]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:46.460 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:46.271+0000 7f2732e46200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:54:46.460 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:46.360+0000 7f2732e46200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:54:46.460 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:46.459+0000 7f2732e46200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:54:46.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:46 smithi018 ceph-mon[31776]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:46.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:46 smithi022 ceph-mon[36023]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:46.675 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:46 smithi022 systemd[1]: Starting Ceph mgr.y for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:54:46.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_4 2024-04-16T16:54:46.752 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:46.643+0000 7f2732e46200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:54:46.972 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:46 smithi022 podman[37388]: 2024-04-16 16:54:46.703009578 +0000 UTC m=+0.111360595 container create d5136006e60ccf184225388893b2478f613e78c27060a403fbd7bc527847cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T16:54:46.972 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:46 smithi022 podman[37388]: 2024-04-16 16:54:46.609466211 +0000 UTC m=+0.017817222 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T16:54:46.972 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:46 smithi022 podman[37388]: 2024-04-16 16:54:46.90508619 +0000 UTC m=+0.313437194 container init d5136006e60ccf184225388893b2478f613e78c27060a403fbd7bc527847cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T16:54:46.972 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:46 smithi022 podman[37388]: 2024-04-16 16:54:46.907764388 +0000 UTC m=+0.316115386 container start d5136006e60ccf184225388893b2478f613e78c27060a403fbd7bc527847cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-16T16:54:46.972 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:46 smithi022 ceph-mgr[37407]: -- 172.21.15.22:0/495726985 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3ee87e5a0 con 0x55d3ee853000 2024-04-16T16:54:46.973 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:46 smithi022 bash[37388]: d5136006e60ccf184225388893b2478f613e78c27060a403fbd7bc527847cedc 2024-04-16T16:54:47.177 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:54:47.424 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:46 smithi022 systemd[1]: Started Ceph mgr.y for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:54:47.925 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:47.495+0000 7fa0eb429200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:54:47.925 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:47.594+0000 7fa0eb429200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:54:48.003 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:47 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:47.680+0000 7f2732e46200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:54:48.003 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:47 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:47.820+0000 7f2732e46200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:54:48.332 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:48 smithi018 ceph-mon[31776]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:48.332 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.332 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.332 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.332 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.332 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:48.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-mon[36901]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:48.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:48.347 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:48.180+0000 7f2732e46200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:54:48.347 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:48.275+0000 7f2732e46200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:54:48.395 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:48.297+0000 7fa0eb429200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:54:48.395 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:48.393+0000 7fa0eb429200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:54:48.395 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-mon[36023]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:48.395 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.395 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.395 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.395 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:48.395 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:48.650 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:48.545+0000 7fa0eb429200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:54:48.651 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:48.649+0000 7fa0eb429200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:54:48.677 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:48.481+0000 7f2732e46200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:54:48.924 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:48.739+0000 7fa0eb429200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:54:48.976 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:48.676+0000 7f2732e46200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:54:49.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 -- 172.21.15.18:0/1708684107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec10aa30 msgr2=0x7fa1ec10cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:49.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 --2- 172.21.15.18:0/1708684107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec10aa30 0x7fa1ec10cec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400b0a0 tx=0x7fa1e400b3b0 comp rx=0 tx=0).stop 2024-04-16T16:54:49.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 -- 172.21.15.18:0/1708684107 shutdown_connections 2024-04-16T16:54:49.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 --2- 172.21.15.18:0/1708684107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec10aa30 0x7fa1ec10cec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:49.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 --2- 172.21.15.18:0/1708684107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1ec076ac0 0x7fa1ec074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:49.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 --2- 172.21.15.18:0/1708684107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1ec0754f0 0x7fa1ec075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:49.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 -- 172.21.15.18:0/1708684107 >> 172.21.15.18:0/1708684107 conn(0x7fa1ec0fe0f0 msgr2=0x7fa1ec100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:49.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 -- 172.21.15.18:0/1708684107 shutdown_connections 2024-04-16T16:54:49.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.072+0000 7fa1f4600700 1 -- 172.21.15.18:0/1708684107 wait complete. 2024-04-16T16:54:49.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.073+0000 7fa1f4600700 1 Processor -- start 2024-04-16T16:54:49.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f4600700 1 -- start start 2024-04-16T16:54:49.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f4600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1ec0754f0 0x7fa1ec19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:49.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f4600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec076ac0 0x7fa1ec19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:49.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f4600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1ec10aa30 0x7fa1ec1a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f4600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec110940 con 0x7fa1ec0754f0 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f4600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec1107c0 con 0x7fa1ec076ac0 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f4600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec110ac0 con 0x7fa1ec10aa30 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f1200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec076ac0 0x7fa1ec19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1ec0754f0 0x7fa1ec19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f1200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec076ac0 0x7fa1ec19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60390/0 (socket says 172.21.15.18:60390) 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.074+0000 7fa1f1200700 1 -- 172.21.15.18:0/2002424701 learned_addr learned my addr 172.21.15.18:0/2002424701 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.075+0000 7fa1f2600700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1ec10aa30 0x7fa1ec1a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.075+0000 7fa1f1200700 1 -- 172.21.15.18:0/2002424701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1ec10aa30 msgr2=0x7fa1ec1a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.075+0000 7fa1f1200700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1ec10aa30 0x7fa1ec1a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.075+0000 7fa1f1200700 1 -- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1ec0754f0 msgr2=0x7fa1ec19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.075+0000 7fa1f1200700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1ec0754f0 0x7fa1ec19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.075+0000 7fa1f1200700 1 -- 172.21.15.18:0/2002424701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec1a2b90 con 0x7fa1ec076ac0 2024-04-16T16:54:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.075+0000 7fa1f1200700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec076ac0 0x7fa1ec19b470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800eb30 tx=0x7fa1d800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.076+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800cbd0 con 0x7fa1ec076ac0 2024-04-16T16:54:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.076+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8004500 con 0x7fa1ec076ac0 2024-04-16T16:54:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.076+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec1a2e80 con 0x7fa1ec076ac0 2024-04-16T16:54:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.076+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8010430 con 0x7fa1ec076ac0 2024-04-16T16:54:49.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.076+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1a3380 con 0x7fa1ec076ac0 2024-04-16T16:54:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.077+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa1d80105d0 con 0x7fa1ec076ac0 2024-04-16T16:54:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.077+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec0697e0 con 0x7fa1ec076ac0 2024-04-16T16:54:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.078+0000 7fa1ea000700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa1d4039890 0x7fa1d403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.078+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa1d804ceb0 con 0x7fa1ec076ac0 2024-04-16T16:54:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.078+0000 7fa1f1c00700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa1d4039890 0x7fa1d403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:49.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.079+0000 7fa1f1c00700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa1d4039890 0x7fa1d403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009c60 tx=0x7fa1e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:49.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.083+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1d801fde0 con 0x7fa1ec076ac0 2024-04-16T16:54:49.245 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:48.974+0000 7f2732e46200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:54:49.314 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:49.012+0000 7fa0eb429200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:54:49.315 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:49.126+0000 7fa0eb429200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:54:49.315 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:49.215+0000 7fa0eb429200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:54:49.315 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:49.313+0000 7fa0eb429200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:54:49.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:49.334+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa1ec062f30 con 0x7fa1d4039890 2024-04-16T16:54:49.502 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:49.244+0000 7f2732e46200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:54:49.502 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:49.489+0000 7f2732e46200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:54:49.603 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:49.495+0000 7fa0eb429200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T16:54:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: Reconfiguring daemon mgr.x on smithi018 2024-04-16T16:54:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:54:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:54:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: Reconfiguring daemon mgr.x on smithi018 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:54:49.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:54:49.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:50.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:50.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:50.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: Reconfiguring daemon mgr.x on smithi018 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:50.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:54:50.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:54:50.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:50.752 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:50.327+0000 7f2732e46200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:54:50.833 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:50.528+0000 7fa0eb429200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:54:50.833 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:50.664+0000 7fa0eb429200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:54:51.090 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:50 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:51.109 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:50 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:51.110 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:51.015+0000 7fa0eb429200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:54:51.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:50 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:51.244 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:50.919+0000 7f2732e46200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:54:51.244 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:51.019+0000 7f2732e46200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:54:51.244 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 16:54:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T16:54:51.243+0000 7f2732e46200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:54:51.424 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:51.108+0000 7fa0eb429200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:54:51.425 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:51.314+0000 7fa0eb429200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:54:51.802 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:51.505+0000 7fa0eb429200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:54:52.070 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: Standby manager daemon z started 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T16:54:52.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:52 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:51.801+0000 7fa0eb429200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T16:54:52.072 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:54:52.073 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: Standby manager daemon z started 2024-04-16T16:54:52.073 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T16:54:52.073 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:54:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:52.079+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fa1d8021950 con 0x7fa1ec076ac0 2024-04-16T16:54:52.425 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:52.070+0000 7fa0eb429200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:54:52.425 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:52.318+0000 7fa0eb429200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:54:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: Standby manager daemon z started 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T16:54:52.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:52 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.43:0/3043372347' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:54:53.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:53 smithi022 ceph-mon[36023]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-16T16:54:53.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:53 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T16:54:53.425 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:53.163+0000 7fa0eb429200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:54:53.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:53 smithi018 ceph-mon[31776]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-16T16:54:53.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T16:54:53.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:53 smithi043 ceph-mon[36901]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-16T16:54:53.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:53 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T16:54:54.080 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:53.755+0000 7fa0eb429200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:54:54.081 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:53.856+0000 7fa0eb429200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:54:54.363 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 16:54:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T16:54:54.079+0000 7fa0eb429200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:54:54.363 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:54 smithi022 ceph-mon[36023]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:54.364 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:54 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/502377313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]: dispatch 2024-04-16T16:54:54.364 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:54 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]: dispatch 2024-04-16T16:54:54.364 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:54 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]': finished 2024-04-16T16:54:54.364 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:54 smithi022 ceph-mon[36023]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T16:54:54.364 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:54:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:54 smithi018 ceph-mon[31776]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:54 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/502377313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]: dispatch 2024-04-16T16:54:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:54 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]: dispatch 2024-04-16T16:54:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:54 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]': finished 2024-04-16T16:54:54.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:54 smithi018 ceph-mon[31776]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T16:54:54.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:54:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:54 smithi043 ceph-mon[36901]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:54 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/502377313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]: dispatch 2024-04-16T16:54:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:54 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]: dispatch 2024-04-16T16:54:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:54 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a6d2ab-b5fc-4430-99dc-5074f1d7e51f"}]': finished 2024-04-16T16:54:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:54 smithi043 ceph-mon[36901]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T16:54:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:54:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:54:55.110+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mon.2 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fa1d8021430 con 0x7fa1ec076ac0 2024-04-16T16:54:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:55 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T16:54:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:55 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:54:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:55 smithi022 ceph-mon[36023]: Standby manager daemon y started 2024-04-16T16:54:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:55 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T16:54:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:55 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:54:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/906392534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:54:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:55 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T16:54:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:55 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:54:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:55 smithi018 ceph-mon[31776]: Standby manager daemon y started 2024-04-16T16:54:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:55 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T16:54:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:55 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:54:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/906392534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:54:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:55 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T16:54:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:55 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:54:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:55 smithi043 ceph-mon[36901]: Standby manager daemon y started 2024-04-16T16:54:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:55 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T16:54:55.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:55 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.22:0/1525758700' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:54:55.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/906392534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:54:56.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:56 smithi043 ceph-mon[36901]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:56 smithi043 ceph-mon[36901]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-16T16:54:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:56 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T16:54:56.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:56 smithi022 ceph-mon[36023]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:56.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:56 smithi022 ceph-mon[36023]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-16T16:54:56.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:56 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T16:54:56.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:56 smithi018 ceph-mon[31776]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:56.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:56 smithi018 ceph-mon[31776]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-16T16:54:56.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:56 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T16:54:58.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:54:58 smithi018 ceph-mon[31776]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:58.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:54:58 smithi022 ceph-mon[36023]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:54:58.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:54:58 smithi043 ceph-mon[36901]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:00 smithi022 ceph-mon[36023]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:00 smithi018 ceph-mon[31776]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:00.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:00 smithi043 ceph-mon[36901]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:02.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:02 smithi022 ceph-mon[36023]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:02 smithi018 ceph-mon[31776]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:02.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:02 smithi043 ceph-mon[36901]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:04.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:04 smithi018 ceph-mon[31776]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:04.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T16:55:04.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:04.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:04 smithi018 ceph-mon[31776]: Deploying daemon osd.0 on smithi018 2024-04-16T16:55:04.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:04 smithi022 ceph-mon[36023]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:04.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T16:55:04.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:04.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:04 smithi022 ceph-mon[36023]: Deploying daemon osd.0 on smithi018 2024-04-16T16:55:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:04 smithi043 ceph-mon[36901]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T16:55:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:04 smithi043 ceph-mon[36901]: Deploying daemon osd.0 on smithi018 2024-04-16T16:55:06.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:06 smithi018 ceph-mon[31776]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:06.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:06 smithi022 ceph-mon[36023]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:06 smithi043 ceph-mon[36901]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:08.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:08 smithi018 ceph-mon[31776]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:08 smithi022 ceph-mon[36023]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:08 smithi043 ceph-mon[36901]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:10.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:10 smithi022 ceph-mon[36023]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:10 smithi018 ceph-mon[31776]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:10.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:10 smithi043 ceph-mon[36901]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:12 smithi022 ceph-mon[36023]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:12.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:12 smithi018 ceph-mon[31776]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:12.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:12.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:12.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:12 smithi043 ceph-mon[36901]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:14.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:14 smithi018 ceph-mon[31776]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:14.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:14 smithi022 ceph-mon[36023]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:14.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:14 smithi043 ceph-mon[36901]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:16.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:16.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:16.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:15 smithi043 ceph-mon[36901]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:16.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:15 smithi043 ceph-mon[36901]: from='osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:55:16.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:15 smithi043 ceph-mon[36901]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:55:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:15 smithi022 ceph-mon[36023]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:15 smithi022 ceph-mon[36023]: from='osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:55:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:15 smithi022 ceph-mon[36023]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:55:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:15 smithi018 ceph-mon[31776]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:16.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:15 smithi018 ceph-mon[31776]: from='osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:55:16.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:15 smithi018 ceph-mon[31776]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:55:16.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.339+0000 7fa1ea000700 1 -- 172.21.15.18:0/2002424701 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa1ec062f30 con 0x7fa1d4039890 2024-04-16T16:55:16.342 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 0 on host 'smithi018' 2024-04-16T16:55:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.343+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa1d4039890 msgr2=0x7fa1d403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.343+0000 7fa1f4600700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa1d4039890 0x7fa1d403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009c60 tx=0x7fa1e0009260 comp rx=0 tx=0).stop 2024-04-16T16:55:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.343+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec076ac0 msgr2=0x7fa1ec19b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:16.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.343+0000 7fa1f4600700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec076ac0 0x7fa1ec19b470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800eb30 tx=0x7fa1d800ee40 comp rx=0 tx=0).stop 2024-04-16T16:55:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.343+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 shutdown_connections 2024-04-16T16:55:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.344+0000 7fa1f4600700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa1d4039890 0x7fa1d403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.344+0000 7fa1f4600700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1ec10aa30 0x7fa1ec1a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.344+0000 7fa1f4600700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1ec0754f0 0x7fa1ec19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.344+0000 7fa1f4600700 1 --2- 172.21.15.18:0/2002424701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec076ac0 0x7fa1ec19b470 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.344+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 >> 172.21.15.18:0/2002424701 conn(0x7fa1ec0fe0f0 msgr2=0x7fa1ec106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.344+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 shutdown_connections 2024-04-16T16:55:16.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:55:16.344+0000 7fa1f4600700 1 -- 172.21.15.18:0/2002424701 wait complete. 2024-04-16T16:55:17.029 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:16 smithi018 ceph-mon[31776]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T16:55:17.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:16 smithi018 ceph-mon[31776]: from='osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:55:17.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:16 smithi018 ceph-mon[31776]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T16:55:17.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:17.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:16 smithi018 ceph-mon[31776]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:55:17.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:17.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:17.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:17.098 DEBUG:teuthology.orchestra.run.smithi018:osd.0> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.0.service 2024-04-16T16:55:17.103 INFO:tasks.cephadm:Deploying osd.1 on smithi022 with /dev/vg_nvme/lv_4... 2024-04-16T16:55:17.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T16:55:17.137 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:16 smithi022 ceph-mon[36023]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T16:55:17.137 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:16 smithi022 ceph-mon[36023]: from='osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:55:17.137 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:16 smithi022 ceph-mon[36023]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T16:55:17.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:17.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:16 smithi022 ceph-mon[36023]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:55:17.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:17.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:17.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:16 smithi043 ceph-mon[36901]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T16:55:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:16 smithi043 ceph-mon[36901]: from='osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:55:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:16 smithi043 ceph-mon[36901]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T16:55:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:16 smithi043 ceph-mon[36901]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:55:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:17.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:17.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:17.360 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:55:17.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:17 smithi018 ceph-mon[31776]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:55:17.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:17 smithi018 ceph-mon[31776]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T16:55:17.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:17 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:17.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:17 smithi018 ceph-mon[31776]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:17.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:17 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:18.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:17 smithi022 ceph-mon[36023]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:55:18.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:17 smithi022 ceph-mon[36023]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T16:55:18.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:17 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:18.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:17 smithi022 ceph-mon[36023]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:18.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:17 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:18.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:17 smithi043 ceph-mon[36901]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:55:18.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:17 smithi043 ceph-mon[36901]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T16:55:18.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:17 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:18.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:17 smithi043 ceph-mon[36901]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:18.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:17 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:19.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:18 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:55:19.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:18 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:55:19.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:18 smithi018 ceph-mon[31776]: osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] boot 2024-04-16T16:55:19.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:18 smithi018 ceph-mon[31776]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T16:55:19.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:18 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:19.064 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T16:55:19.064 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T16:55:19.064 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-04-16T16:55:19.064 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-04-16T16:55:19.064 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00826511 s, 1.3 GB/s 2024-04-16T16:55:19.064 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-04-16T16:55:19.064 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:18 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:55:19.064 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:18 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:55:19.065 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:18 smithi022 ceph-mon[36023]: osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] boot 2024-04-16T16:55:19.065 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:18 smithi022 ceph-mon[36023]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T16:55:19.065 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:18 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:19.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:18 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:55:19.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:18 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:55:19.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:18 smithi043 ceph-mon[36901]: osd.0 [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] boot 2024-04-16T16:55:19.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:18 smithi043 ceph-mon[36901]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T16:55:19.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:18 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:55:19.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_4 2024-04-16T16:55:19.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:19 smithi022 ceph-mon[36023]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:19.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:19 smithi022 ceph-mon[36023]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T16:55:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:55:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:19 smithi018 ceph-mon[31776]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:19 smithi018 ceph-mon[31776]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T16:55:20.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:19 smithi043 ceph-mon[36901]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:20.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:19 smithi043 ceph-mon[36901]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T16:55:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 -- 172.21.15.22:0/2213529246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8105f00 msgr2=0x7fbfd81082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 --2- 172.21.15.22:0/2213529246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8105f00 0x7fbfd81082f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009990 tx=0x7fbfcc009ca0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 -- 172.21.15.22:0/2213529246 shutdown_connections 2024-04-16T16:55:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 --2- 172.21.15.22:0/2213529246 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd8108830 0x7fbfd810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 --2- 172.21.15.22:0/2213529246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd81035d0 0x7fbfd81059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 --2- 172.21.15.22:0/2213529246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8105f00 0x7fbfd81082f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 -- 172.21.15.22:0/2213529246 >> 172.21.15.22:0/2213529246 conn(0x7fbfd80fcfe0 msgr2=0x7fbfd80ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 -- 172.21.15.22:0/2213529246 shutdown_connections 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.307+0000 7fbfdf563700 1 -- 172.21.15.22:0/2213529246 wait complete. 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdf563700 1 Processor -- start 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdf563700 1 -- start start 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdf563700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd81035d0 0x7fbfd8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdf563700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd8105f00 0x7fbfd81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdf563700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8108830 0x7fbfd81a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdf563700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810e810 con 0x7fbfd8105f00 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdf563700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810e690 con 0x7fbfd8108830 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdf563700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810e990 con 0x7fbfd81035d0 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8108830 0x7fbfd81a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8108830 0x7fbfd81a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34182/0 (socket says 172.21.15.22:34182) 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 -- 172.21.15.22:0/1290097207 learned_addr learned my addr 172.21.15.22:0/1290097207 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfdd2ff700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd81035d0 0x7fbfd8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 -- 172.21.15.22:0/1290097207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd81035d0 msgr2=0x7fbfd8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd81035d0 0x7fbfd8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 -- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd8105f00 msgr2=0x7fbfd81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd8105f00 0x7fbfd81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 -- 172.21.15.22:0/1290097207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd81a0b00 con 0x7fbfd8108830 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.308+0000 7fbfddb00700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8108830 0x7fbfd81a0460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800b480 tx=0x7fbfc800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.309+0000 7fbfd67fc700 1 -- 172.21.15.22:0/1290097207 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8015020 con 0x7fbfd8108830 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.309+0000 7fbfd67fc700 1 -- 172.21.15.22:0/1290097207 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc8011d10 con 0x7fbfd8108830 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.309+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd81a0df0 con 0x7fbfd8108830 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.309+0000 7fbfd67fc700 1 -- 172.21.15.22:0/1290097207 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800f2e0 con 0x7fbfd8108830 2024-04-16T16:55:21.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.309+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd81a12d0 con 0x7fbfd8108830 2024-04-16T16:55:21.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.310+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd80697e0 con 0x7fbfd8108830 2024-04-16T16:55:21.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.312+0000 7fbfd67fc700 1 -- 172.21.15.22:0/1290097207 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fbfc8018020 con 0x7fbfd8108830 2024-04-16T16:55:21.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.312+0000 7fbfd67fc700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:21.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.312+0000 7fbfd67fc700 1 -- 172.21.15.22:0/1290097207 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fbfc80ce5a0 con 0x7fbfd8108830 2024-04-16T16:55:21.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.313+0000 7fbfdd2ff700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:21.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.314+0000 7fbfdd2ff700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0007dc0 tx=0x7fbfd00072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:21.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.316+0000 7fbfd67fc700 1 -- 172.21.15.22:0/1290097207 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfc801f020 con 0x7fbfd8108830 2024-04-16T16:55:21.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:21.536+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbfd8062f30 con 0x7fbfc00a3bb0 2024-04-16T16:55:21.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:21 smithi018 ceph-mon[31776]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:21.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:21 smithi018 ceph-mon[31776]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:21.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:21 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:55:21.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:21 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:55:21.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:21 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:22.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:21 smithi022 ceph-mon[36023]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:21 smithi022 ceph-mon[36023]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:21 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:55:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:21 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:55:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:21 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:21 smithi043 ceph-mon[36901]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:21 smithi043 ceph-mon[36901]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:21 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:55:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:21 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:55:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:21 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:23.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi018 2024-04-16T16:55:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:55:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi018 to 17094M 2024-04-16T16:55:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi018 2024-04-16T16:55:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:55:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi018 to 17094M 2024-04-16T16:55:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi018 2024-04-16T16:55:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:55:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi018 to 17094M 2024-04-16T16:55:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:23.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:24.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:23 smithi018 ceph-mon[31776]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:24.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:23 smithi022 ceph-mon[36023]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:23 smithi043 ceph-mon[36901]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:25.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:24 smithi022 ceph-mon[36023]: from='client.? 172.21.15.22:0/896413319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]: dispatch 2024-04-16T16:55:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:24 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]: dispatch 2024-04-16T16:55:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:24 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]': finished 2024-04-16T16:55:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:24 smithi022 ceph-mon[36023]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T16:55:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:24 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:24 smithi022 ceph-mon[36023]: from='client.? 172.21.15.22:0/4062826303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:55:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:24 smithi018 ceph-mon[31776]: from='client.? 172.21.15.22:0/896413319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]: dispatch 2024-04-16T16:55:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:24 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]: dispatch 2024-04-16T16:55:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:24 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]': finished 2024-04-16T16:55:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:24 smithi018 ceph-mon[31776]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T16:55:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:24 smithi018 ceph-mon[31776]: from='client.? 172.21.15.22:0/4062826303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:55:25.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:24 smithi043 ceph-mon[36901]: from='client.? 172.21.15.22:0/896413319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]: dispatch 2024-04-16T16:55:25.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:24 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]: dispatch 2024-04-16T16:55:25.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:24 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efef211f-28b9-4f7b-a3c7-59ccb4914bbe"}]': finished 2024-04-16T16:55:25.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:24 smithi043 ceph-mon[36901]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T16:55:25.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:24 smithi043 ceph-mon[36901]: from='client.? 172.21.15.22:0/4062826303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:55:26.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:26 smithi018 ceph-mon[31776]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:26 smithi043 ceph-mon[36901]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:26 smithi022 ceph-mon[36023]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:28.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:28 smithi018 ceph-mon[31776]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:28.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:28 smithi043 ceph-mon[36901]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:28.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:28 smithi022 ceph-mon[36023]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:30.268 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:30 smithi022 ceph-mon[36023]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:30.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:30 smithi018 ceph-mon[31776]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:30.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:30 smithi043 ceph-mon[36901]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:32.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:32 smithi022 ceph-mon[36023]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T16:55:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:32 smithi022 ceph-mon[36023]: Deploying daemon osd.1 on smithi022 2024-04-16T16:55:32.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:32 smithi043 ceph-mon[36901]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:32.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T16:55:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:32 smithi043 ceph-mon[36901]: Deploying daemon osd.1 on smithi022 2024-04-16T16:55:32.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:32 smithi018 ceph-mon[31776]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:32.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T16:55:32.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:32.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:32 smithi018 ceph-mon[31776]: Deploying daemon osd.1 on smithi022 2024-04-16T16:55:34.461 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:34 smithi022 ceph-mon[36023]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:34.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:34 smithi043 ceph-mon[36901]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:34 smithi018 ceph-mon[31776]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:36.464 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:36 smithi022 ceph-mon[36023]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:36.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:36 smithi043 ceph-mon[36901]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:36.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:36 smithi018 ceph-mon[31776]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:38.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:38.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:38.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:38.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:37 smithi043 ceph-mon[36901]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:38.168 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:38.168 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:38.168 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:38.168 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:37 smithi022 ceph-mon[36023]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:38.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:38.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:38.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:38.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:37 smithi018 ceph-mon[31776]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.180+0000 7fbfd67fc700 1 -- 172.21.15.22:0/1290097207 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbfd8062f30 con 0x7fbfc00a3bb0 2024-04-16T16:55:40.182 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 1 on host 'smithi022' 2024-04-16T16:55:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbfc00a3bb0 msgr2=0x7fbfc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0007dc0 tx=0x7fbfd00072f0 comp rx=0 tx=0).stop 2024-04-16T16:55:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8108830 msgr2=0x7fbfd81a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8108830 0x7fbfd81a0460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800b480 tx=0x7fbfc800b840 comp rx=0 tx=0).stop 2024-04-16T16:55:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 shutdown_connections 2024-04-16T16:55:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd8108830 0x7fbfd81a0460 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd81035d0 0x7fbfd8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 --2- 172.21.15.22:0/1290097207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd8105f00 0x7fbfd81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 >> 172.21.15.22:0/1290097207 conn(0x7fbfd80fcfe0 msgr2=0x7fbfd8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 shutdown_connections 2024-04-16T16:55:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:55:40.183+0000 7fbfdf563700 1 -- 172.21.15.22:0/1290097207 wait complete. 2024-04-16T16:55:40.537 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:40.537 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.537 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.538 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:40.538 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:40.538 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.538 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.538 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:40.538 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:40.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:40.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:40.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:40.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:40.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:40.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:40.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:40.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:40.896 DEBUG:teuthology.orchestra.run.smithi022:osd.1> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.1.service 2024-04-16T16:55:40.899 INFO:tasks.cephadm:Deploying osd.2 on smithi043 with /dev/vg_nvme/lv_4... 2024-04-16T16:55:40.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T16:55:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:55:41.523 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:41 smithi043 ceph-mon[36901]: from='osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:55:41.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:41 smithi043 ceph-mon[36901]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:55:41.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:41 smithi022 ceph-mon[36023]: from='osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:55:41.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:41 smithi022 ceph-mon[36023]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:55:41.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:41 smithi018 ceph-mon[31776]: from='osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:55:41.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:41 smithi018 ceph-mon[31776]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:55:42.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:42 smithi043 ceph-mon[36901]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:42.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:42 smithi043 ceph-mon[36901]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T16:55:42.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:42 smithi043 ceph-mon[36901]: from='osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:55:42.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:42 smithi043 ceph-mon[36901]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T16:55:42.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:42 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:42.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:42 smithi043 ceph-mon[36901]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:55:42.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:42 smithi022 ceph-mon[36023]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:42.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:42 smithi022 ceph-mon[36023]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T16:55:42.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:42 smithi022 ceph-mon[36023]: from='osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:55:42.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:42 smithi022 ceph-mon[36023]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T16:55:42.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:42 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:42.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:42 smithi022 ceph-mon[36023]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:55:42.675 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:55:42 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1[40615]: 2024-04-16T16:55:42.420+0000 7f0aec887700 -1 osd.1 0 waiting for initial osdmap 2024-04-16T16:55:42.675 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:55:42 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1[40615]: 2024-04-16T16:55:42.425+0000 7f0ae867a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:55:42.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:42 smithi018 ceph-mon[31776]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:42.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:42 smithi018 ceph-mon[31776]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T16:55:42.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:42 smithi018 ceph-mon[31776]: from='osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:55:42.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:42 smithi018 ceph-mon[31776]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T16:55:42.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:42 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:42.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:42 smithi018 ceph-mon[31776]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:55:43.124 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T16:55:43.124 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T16:55:43.124 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-16T16:55:43.124 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-16T16:55:43.124 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00831282 s, 1.3 GB/s 2024-04-16T16:55:43.124 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 2024-04-16T16:55:43.124 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-16T16:55:43.610 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:43 smithi022 ceph-mon[36023]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:55:43.610 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:43 smithi022 ceph-mon[36023]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T16:55:43.610 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:43 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:43.610 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:43 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:43.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:43 smithi043 ceph-mon[36901]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:55:43.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:43 smithi043 ceph-mon[36901]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T16:55:43.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:43 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:43.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:43 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:43.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:43 smithi018 ceph-mon[31776]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:55:43.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:43 smithi018 ceph-mon[31776]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T16:55:43.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:43 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:43.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:43 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:43.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-04-16T16:55:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:55:44.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:55:44.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:55:44.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:44.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] boot 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:44.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] boot 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:55:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:55:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:55:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:55:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: osd.1 [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] boot 2024-04-16T16:55:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T16:55:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:55:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:55:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:55:45.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:45 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi022 2024-04-16T16:55:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:45 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi022 to 17084M 2024-04-16T16:55:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:45 smithi043 ceph-mon[36901]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 -- 172.21.15.43:0/619889067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 msgr2=0x7f6a24101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 --2- 172.21.15.43:0/619889067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 0x7f6a24101150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009990 tx=0x7f6a18009ca0 comp rx=0 tx=0).stop 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 -- 172.21.15.43:0/619889067 shutdown_connections 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 --2- 172.21.15.43:0/619889067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a241046b0 0x7f6a2410af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 --2- 172.21.15.43:0/619889067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 0x7f6a24101150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 --2- 172.21.15.43:0/619889067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a24103d10 0x7f6a24104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 -- 172.21.15.43:0/619889067 >> 172.21.15.43:0/619889067 conn(0x7f6a240fc690 msgr2=0x7f6a240fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 -- 172.21.15.43:0/619889067 shutdown_connections 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.700+0000 7f6a29da1700 1 -- 172.21.15.43:0/619889067 wait complete. 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a29da1700 1 Processor -- start 2024-04-16T16:55:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a29da1700 1 -- start start 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a29da1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 0x7f6a24103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a29da1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24103d10 0x7f6a24101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a29da1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a241046b0 0x7f6a24102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a29da1700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e6c0 con 0x7f6a241046b0 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a29da1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e540 con 0x7f6a24103d10 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a29da1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e840 con 0x7f6a24100d30 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a28d9f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 0x7f6a24103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a28d9f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 0x7f6a24103670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37858/0 (socket says 172.21.15.43:37858) 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.701+0000 7f6a28d9f700 1 -- 172.21.15.43:0/896975854 learned_addr learned my addr 172.21.15.43:0/896975854 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a28d9f700 1 -- 172.21.15.43:0/896975854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24103d10 msgr2=0x7f6a24101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a23fff700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24103d10 0x7f6a24101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a28d9f700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24103d10 0x7f6a24101cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a28d9f700 1 -- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a241046b0 msgr2=0x7f6a24102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a28d9f700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a241046b0 0x7f6a24102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a28d9f700 1 -- 172.21.15.43:0/896975854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a24102ab0 con 0x7f6a24100d30 2024-04-16T16:55:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a28d9f700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 0x7f6a24103670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b8e0 tx=0x7f6a1800bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a21ffb700 1 -- 172.21.15.43:0/896975854 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18026070 con 0x7f6a24100d30 2024-04-16T16:55:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a21ffb700 1 -- 172.21.15.43:0/896975854 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1800be40 con 0x7f6a24100d30 2024-04-16T16:55:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a21ffb700 1 -- 172.21.15.43:0/896975854 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18021670 con 0x7f6a24100d30 2024-04-16T16:55:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241a2da0 con 0x7f6a24100d30 2024-04-16T16:55:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.702+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241a31a0 con 0x7f6a24100d30 2024-04-16T16:55:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.703+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a240697e0 con 0x7f6a24100d30 2024-04-16T16:55:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.704+0000 7f6a21ffb700 1 -- 172.21.15.43:0/896975854 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6a18029070 con 0x7f6a24100d30 2024-04-16T16:55:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.705+0000 7f6a21ffb700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6a140a3c30 0x7f6a140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.705+0000 7f6a21ffb700 1 -- 172.21.15.43:0/896975854 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f6a180cffa0 con 0x7f6a24100d30 2024-04-16T16:55:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.705+0000 7f6a23fff700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6a140a3c30 0x7f6a140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.706+0000 7f6a23fff700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6a140a3c30 0x7f6a140a60f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6a10006fd0 tx=0x7f6a10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.709+0000 7f6a21ffb700 1 -- 172.21.15.43:0/896975854 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a18030020 con 0x7f6a24100d30 2024-04-16T16:55:45.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:45 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi022 2024-04-16T16:55:45.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:45 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi022 to 17084M 2024-04-16T16:55:45.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:45 smithi022 ceph-mon[36023]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T16:55:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:55:45.925+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6a24062f30 con 0x7f6a140a3c30 2024-04-16T16:55:45.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:45 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi022 2024-04-16T16:55:45.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:45 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi022 to 17084M 2024-04-16T16:55:45.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:45 smithi018 ceph-mon[31776]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T16:55:46.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:46 smithi043 ceph-mon[36901]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:46.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:46 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:55:46.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:46 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:55:46.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:46 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:46.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:46 smithi022 ceph-mon[36023]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:46.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:46 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:55:46.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:46 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:55:46.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:46 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:46.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:46 smithi018 ceph-mon[31776]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:46.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:46 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:55:46.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:46 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:55:46.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:46 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:47.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:47 smithi043 ceph-mon[36901]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:47.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:47 smithi022 ceph-mon[36023]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:47.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:47 smithi018 ceph-mon[31776]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:48.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:48 smithi043 ceph-mon[36901]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:48 smithi022 ceph-mon[36023]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:48.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:48 smithi018 ceph-mon[31776]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:49 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/1043593602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]: dispatch 2024-04-16T16:55:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:49 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]: dispatch 2024-04-16T16:55:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:49 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]': finished 2024-04-16T16:55:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:49 smithi043 ceph-mon[36901]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T16:55:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:55:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:49 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/1326095128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:55:49.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:49 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/1043593602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]: dispatch 2024-04-16T16:55:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:49 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]: dispatch 2024-04-16T16:55:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:49 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]': finished 2024-04-16T16:55:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:49 smithi022 ceph-mon[36023]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T16:55:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:55:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:49 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/1326095128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:55:49.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:49 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/1043593602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]: dispatch 2024-04-16T16:55:49.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:49 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]: dispatch 2024-04-16T16:55:49.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:49 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f783df6-43e0-4a3e-b413-af66cf95e18f"}]': finished 2024-04-16T16:55:49.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:49 smithi018 ceph-mon[31776]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T16:55:49.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:55:49.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:49 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/1326095128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:55:50.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:50 smithi022 ceph-mon[36023]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:50 smithi018 ceph-mon[31776]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:51.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:50 smithi043 ceph-mon[36901]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:52 smithi043 ceph-mon[36901]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:52.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:52 smithi022 ceph-mon[36023]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:52.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:52 smithi018 ceph-mon[31776]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:53.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:53 smithi043 ceph-mon[36901]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:53.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:53 smithi022 ceph-mon[36023]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:53.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:53 smithi018 ceph-mon[31776]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:56.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:55 smithi022 ceph-mon[36023]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:56.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T16:55:56.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:55 smithi018 ceph-mon[31776]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T16:55:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:56.230 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:55 smithi043 ceph-mon[36901]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:56.230 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T16:55:56.231 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:57.093 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:56 smithi043 ceph-mon[36901]: Deploying daemon osd.2 on smithi043 2024-04-16T16:55:57.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:56 smithi022 ceph-mon[36023]: Deploying daemon osd.2 on smithi043 2024-04-16T16:55:57.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:56 smithi018 ceph-mon[31776]: Deploying daemon osd.2 on smithi043 2024-04-16T16:55:58.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:57 smithi043 ceph-mon[36901]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:58.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:57 smithi022 ceph-mon[36023]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:55:58.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:57 smithi018 ceph-mon[31776]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:00.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:55:59 smithi043 ceph-mon[36901]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:00.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:55:59 smithi022 ceph-mon[36023]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:00.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:55:59 smithi018 ceph-mon[31776]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:02.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:02 smithi022 ceph-mon[36023]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:02.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:02.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:02.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:02 smithi018 ceph-mon[31776]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:02.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:02 smithi043 ceph-mon[36901]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:02.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:02.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:02.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:04.410 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:04 smithi043 ceph-mon[36901]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:04.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:04.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:04.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:04 smithi022 ceph-mon[36023]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:04.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:04.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:04.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:04 smithi018 ceph-mon[31776]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:04.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.763+0000 7f6a21ffb700 1 -- 172.21.15.43:0/896975854 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6a24062f30 con 0x7f6a140a3c30 2024-04-16T16:56:04.765 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 2 on host 'smithi043' 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6a140a3c30 msgr2=0x7f6a140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6a140a3c30 0x7f6a140a60f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6a10006fd0 tx=0x7f6a10008040 comp rx=0 tx=0).stop 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 msgr2=0x7f6a24103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 0x7f6a24103670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b8e0 tx=0x7f6a1800bc60 comp rx=0 tx=0).stop 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 shutdown_connections 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6a140a3c30 0x7f6a140a60f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a241046b0 0x7f6a24102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a24100d30 0x7f6a24103670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 --2- 172.21.15.43:0/896975854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24103d10 0x7f6a24101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 >> 172.21.15.43:0/896975854 conn(0x7f6a240fc690 msgr2=0x7f6a24107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:04.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 shutdown_connections 2024-04-16T16:56:04.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:56:04.766+0000 7f6a29da1700 1 -- 172.21.15.43:0/896975854 wait complete. 2024-04-16T16:56:05.370 DEBUG:teuthology.orchestra.run.smithi043:osd.2> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.2.service 2024-04-16T16:56:05.372 INFO:tasks.cephadm:Deploying osd.3 on smithi018 with /dev/vg_nvme/lv_3... 2024-04-16T16:56:05.373 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T16:56:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:56:05.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:05.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:05.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:05.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:05 smithi018 ceph-mon[31776]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:05.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:05 smithi018 ceph-mon[31776]: from='osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:56:05.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:05 smithi018 ceph-mon[31776]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:56:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:05 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:05 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:05 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:05 smithi043 ceph-mon[36901]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:05 smithi043 ceph-mon[36901]: from='osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:56:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:05 smithi043 ceph-mon[36901]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:56:06.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:05 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:05 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:05 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:05 smithi022 ceph-mon[36023]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:05 smithi022 ceph-mon[36023]: from='osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:56:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:05 smithi022 ceph-mon[36023]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:56:06.844 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:06 smithi043 ceph-mon[36901]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T16:56:06.845 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:06 smithi043 ceph-mon[36901]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T16:56:06.845 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:06 smithi043 ceph-mon[36901]: from='osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:56:06.845 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:06 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:06.845 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:06 smithi043 ceph-mon[36901]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:56:07.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:06 smithi018 ceph-mon[31776]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T16:56:07.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:06 smithi018 ceph-mon[31776]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T16:56:07.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:06 smithi018 ceph-mon[31776]: from='osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:56:07.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:06 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:07.003 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:06 smithi018 ceph-mon[31776]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:56:07.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:06 smithi022 ceph-mon[36023]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T16:56:07.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:06 smithi022 ceph-mon[36023]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T16:56:07.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:06 smithi022 ceph-mon[36023]: from='osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:56:07.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:06 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:07.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:06 smithi022 ceph-mon[36023]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:56:07.503 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2[42083]: 2024-04-16T16:56:07.224+0000 7f3a998ee700 -1 osd.2 0 waiting for initial osdmap 2024-04-16T16:56:07.503 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2[42083]: 2024-04-16T16:56:07.230+0000 7f3a916d9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:56:07.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:07 smithi018 ceph-mon[31776]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:56:07.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:07 smithi018 ceph-mon[31776]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T16:56:07.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:07.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:07.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:07 smithi018 ceph-mon[31776]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:07.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:07 smithi018 ceph-mon[31776]: from='osd.2 ' entity='osd.2' 2024-04-16T16:56:07.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:07 smithi022 ceph-mon[36023]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:56:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:07 smithi022 ceph-mon[36023]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T16:56:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:07 smithi022 ceph-mon[36023]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:07 smithi022 ceph-mon[36023]: from='osd.2 ' entity='osd.2' 2024-04-16T16:56:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-mon[36901]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:56:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-mon[36901]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T16:56:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-mon[36901]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:56:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-mon[36901]: from='osd.2 ' entity='osd.2' 2024-04-16T16:56:08.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:08.774 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T16:56:08.774 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T16:56:08.774 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-04-16T16:56:08.774 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-04-16T16:56:08.774 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0389745 s, 269 MB/s 2024-04-16T16:56:08.774 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-04-16T16:56:09.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:56:09.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] boot 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi043 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi043 to 17084M 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:09.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.115 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:56:09.115 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:56:09.115 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] boot 2024-04-16T16:56:09.115 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T16:56:09.115 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi043 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi043 to 17084M 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:09.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: osd.2 [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] boot 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi043 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi043 to 17084M 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:09.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:09.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:09.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_3 2024-04-16T16:56:09.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:09 smithi018 ceph-mon[31776]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:09.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:09 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:56:10.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:09 smithi043 ceph-mon[36901]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:10.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:09 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:56:10.034 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:56:10.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:09 smithi022 ceph-mon[36023]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:10.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:09 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:56:11.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T16:56:11.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:10 smithi043 ceph-mon[36901]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T16:56:11.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:56:11.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T16:56:11.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:10 smithi018 ceph-mon[31776]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T16:56:11.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:56:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T16:56:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:10 smithi022 ceph-mon[36023]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T16:56:11.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:56:11.445 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48527]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:56:11.445 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48527]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.445 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48527]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.445 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48527]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.445 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48530]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:56:11.446 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48530]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.446 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48530]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.446 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48530]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.561 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44459]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:56:11.561 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44459]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.561 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44459]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.561 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44459]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.561 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44462]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:56:11.561 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.561 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44462]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.561 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44462]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48533]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:56:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48533]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48533]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48533]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48538]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:56:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48538]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48538]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 sudo[48538]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46007]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:56:11.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46007]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46007]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46007]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46010]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:56:11.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46010]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46010]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46010]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.753 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46001]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:56:11.753 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46001]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.753 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46001]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.753 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46001]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.753 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46004]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:56:11.753 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46004]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.753 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46004]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.753 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 16:56:11 smithi043 sudo[46004]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44465]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44465]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44465]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44465]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44468]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44468]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44468]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 sudo[44468]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:56:11.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:11.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:11.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:11.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:11.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:11.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:11.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:11 smithi022 ceph-mon[36023]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:11.988 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:11.989 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:11.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:11 smithi018 ceph-mon[31776]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:12.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T16:56:12.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:12.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:12.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:56:12.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:12.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:12.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:12.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:11 smithi043 ceph-mon[36901]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 -- 172.21.15.18:0/3902202711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8104c70 msgr2=0x7fdfe8105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 --2- 172.21.15.18:0/3902202711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8104c70 0x7fdfe8105090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80099c0 tx=0x7fdfd8009cd0 comp rx=0 tx=0).stop 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 -- 172.21.15.18:0/3902202711 shutdown_connections 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 --2- 172.21.15.18:0/3902202711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8104c70 0x7fdfe8105090 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 --2- 172.21.15.18:0/3902202711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe8106810 0x7fdfe810d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 --2- 172.21.15.18:0/3902202711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe8105e70 0x7fdfe81062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 -- 172.21.15.18:0/3902202711 >> 172.21.15.18:0/3902202711 conn(0x7fdfe8100210 msgr2=0x7fdfe8102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 -- 172.21.15.18:0/3902202711 shutdown_connections 2024-04-16T16:56:12.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.268+0000 7fdfed400700 1 -- 172.21.15.18:0/3902202711 wait complete. 2024-04-16T16:56:12.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.269+0000 7fdfed400700 1 Processor -- start 2024-04-16T16:56:12.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.270+0000 7fdfed400700 1 -- start start 2024-04-16T16:56:12.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.270+0000 7fdfed400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe8104c70 0x7fdfe819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:12.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.270+0000 7fdfed400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8105e70 0x7fdfe819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:12.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.270+0000 7fdfed400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe8106810 0x7fdfe81a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:12.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.270+0000 7fdfed400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8110880 con 0x7fdfe8106810 2024-04-16T16:56:12.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.270+0000 7fdfed400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8110700 con 0x7fdfe8105e70 2024-04-16T16:56:12.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.270+0000 7fdfed400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8110a00 con 0x7fdfe8104c70 2024-04-16T16:56:12.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe8104c70 0x7fdfe819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:12.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8105e70 0x7fdfe819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:12.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe8104c70 0x7fdfe819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43244/0 (socket says 172.21.15.18:43244) 2024-04-16T16:56:12.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7e00700 1 -- 172.21.15.18:0/3635831219 learned_addr learned my addr 172.21.15.18:0/3635831219 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:56:12.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfeca00700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe8106810 0x7fdfe81a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:12.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7400700 1 -- 172.21.15.18:0/3635831219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe8104c70 msgr2=0x7fdfe819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:12.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe8104c70 0x7fdfe819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:12.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7400700 1 -- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe8106810 msgr2=0x7fdfe81a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:12.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe8106810 0x7fdfe81a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:12.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7400700 1 -- 172.21.15.18:0/3635831219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81a2ac0 con 0x7fdfe8105e70 2024-04-16T16:56:12.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.271+0000 7fdfe7400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8105e70 0x7fdfe819b3a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400e9f0 tx=0x7fdfd400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:12.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.272+0000 7fdfe4c00700 1 -- 172.21.15.18:0/3635831219 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400c9c0 con 0x7fdfe8105e70 2024-04-16T16:56:12.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.272+0000 7fdfe4c00700 1 -- 172.21.15.18:0/3635831219 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd400cb60 con 0x7fdfe8105e70 2024-04-16T16:56:12.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.272+0000 7fdfe4c00700 1 -- 172.21.15.18:0/3635831219 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4018680 con 0x7fdfe8105e70 2024-04-16T16:56:12.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.272+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe81a2db0 con 0x7fdfe8105e70 2024-04-16T16:56:12.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.272+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe81a3210 con 0x7fdfe8105e70 2024-04-16T16:56:12.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.273+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe8069820 con 0x7fdfe8105e70 2024-04-16T16:56:12.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.275+0000 7fdfe4c00700 1 -- 172.21.15.18:0/3635831219 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fdfd4010430 con 0x7fdfe8105e70 2024-04-16T16:56:12.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.277+0000 7fdfe4c00700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdfcc0a3cb0 0x7fdfcc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:12.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.277+0000 7fdfe4c00700 1 -- 172.21.15.18:0/3635831219 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fdfd40cef80 con 0x7fdfe8105e70 2024-04-16T16:56:12.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.277+0000 7fdfe7e00700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdfcc0a3cb0 0x7fdfcc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:12.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.278+0000 7fdfe7e00700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdfcc0a3cb0 0x7fdfcc0a6170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80038c0 tx=0x7fdfd8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:12.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.281+0000 7fdfe4c00700 1 -- 172.21.15.18:0/3635831219 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfd409cd70 con 0x7fdfe8105e70 2024-04-16T16:56:12.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:12.523+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdfe8062f30 con 0x7fdfcc0a3cb0 2024-04-16T16:56:13.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:12 smithi022 ceph-mon[36023]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T16:56:13.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:12 smithi022 ceph-mon[36023]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:13.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:13.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:13.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:13.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:12 smithi018 ceph-mon[31776]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T16:56:13.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:12 smithi018 ceph-mon[31776]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:13.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:13.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:13.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:12 smithi043 ceph-mon[36901]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T16:56:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:12 smithi043 ceph-mon[36901]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:13.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:13.820+0000 7fdfe4c00700 1 -- 172.21.15.18:0/3635831219 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfd4098ce0 con 0x7fdfe8105e70 2024-04-16T16:56:14.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:13 smithi022 ceph-mon[36023]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:14.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:13 smithi018 ceph-mon[31776]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:14.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:13 smithi043 ceph-mon[36901]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:15.126 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:14 smithi018 ceph-mon[31776]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T16:56:15.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:14 smithi022 ceph-mon[36023]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T16:56:15.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:14 smithi043 ceph-mon[36901]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T16:56:16.177 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:15 smithi022 ceph-mon[36023]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:15 smithi018 ceph-mon[31776]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:16.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:15 smithi043 ceph-mon[36901]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:17.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:16 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2829952191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]: dispatch 2024-04-16T16:56:17.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:16 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]: dispatch 2024-04-16T16:56:17.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:16 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]': finished 2024-04-16T16:56:17.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:16 smithi022 ceph-mon[36023]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T16:56:17.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:17.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:16 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2829952191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]: dispatch 2024-04-16T16:56:17.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:16 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]: dispatch 2024-04-16T16:56:17.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:16 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]': finished 2024-04-16T16:56:17.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:16 smithi018 ceph-mon[31776]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T16:56:17.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:16 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2829952191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]: dispatch 2024-04-16T16:56:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:16 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]: dispatch 2024-04-16T16:56:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:16 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0da47d3-ae80-4e06-a98e-887862be0ce4"}]': finished 2024-04-16T16:56:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:16 smithi043 ceph-mon[36901]: osdmap e22: 4 total, 3 up, 4 in 2024-04-16T16:56:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:18.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:17 smithi022 ceph-mon[36023]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:18.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:17 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2743609584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:18.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:17 smithi018 ceph-mon[31776]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:18.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:17 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2743609584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:18.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:17 smithi043 ceph-mon[36901]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:18.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:17 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2743609584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:20.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:19 smithi022 ceph-mon[36023]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:19 smithi018 ceph-mon[31776]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:20.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:19 smithi043 ceph-mon[36901]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:21 smithi022 ceph-mon[36023]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:22.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:21 smithi018 ceph-mon[31776]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:21 smithi043 ceph-mon[36901]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:23 smithi018 ceph-mon[31776]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:24.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:23 smithi022 ceph-mon[36023]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:23 smithi043 ceph-mon[36901]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:25 smithi022 ceph-mon[36023]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T16:56:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:25 smithi022 ceph-mon[36023]: Deploying daemon osd.3 on smithi018 2024-04-16T16:56:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:25 smithi018 ceph-mon[31776]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T16:56:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:25 smithi018 ceph-mon[31776]: Deploying daemon osd.3 on smithi018 2024-04-16T16:56:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:25 smithi043 ceph-mon[36901]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T16:56:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:25 smithi043 ceph-mon[36901]: Deploying daemon osd.3 on smithi018 2024-04-16T16:56:28.124 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:27 smithi018 ceph-mon[31776]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:27 smithi022 ceph-mon[36023]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:28.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:27 smithi043 ceph-mon[36901]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:30.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:29 smithi022 ceph-mon[36023]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:30.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:29 smithi018 ceph-mon[31776]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:30.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:29 smithi043 ceph-mon[36901]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:31 smithi022 ceph-mon[36023]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:31 smithi018 ceph-mon[31776]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:32.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:31 smithi043 ceph-mon[36901]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:34.621 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:34 smithi018 ceph-mon[31776]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:34.622 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:34 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:34.622 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:34 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:34.622 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:34 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:34.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:34 smithi022 ceph-mon[36023]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:34.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:34.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:34.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:34 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:34.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:34 smithi043 ceph-mon[36901]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:34.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:34 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:34.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:34 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:34.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:34 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:36.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:36 smithi018 ceph-mon[31776]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:36.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:36 smithi022 ceph-mon[36023]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:36.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:36 smithi043 ceph-mon[36901]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:37.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:37.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:37.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:37.933 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:37.933 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:37.933 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:37 smithi018 ceph-mon[31776]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:37.933 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:37 smithi018 ceph-mon[31776]: from='osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:56:37.933 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:37 smithi018 ceph-mon[31776]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:56:38.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:38.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:38.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:38.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:38.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:38.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:37 smithi043 ceph-mon[36901]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:38.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:37 smithi043 ceph-mon[36901]: from='osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:56:38.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:37 smithi043 ceph-mon[36901]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:37 smithi022 ceph-mon[36023]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:37 smithi022 ceph-mon[36023]: from='osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:37 smithi022 ceph-mon[36023]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:56:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.370+0000 7fdfe4c00700 1 -- 172.21.15.18:0/3635831219 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdfe8062f30 con 0x7fdfcc0a3cb0 2024-04-16T16:56:38.373 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 3 on host 'smithi018' 2024-04-16T16:56:38.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.374+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdfcc0a3cb0 msgr2=0x7fdfcc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.374+0000 7fdfed400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdfcc0a3cb0 0x7fdfcc0a6170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80038c0 tx=0x7fdfd8005fb0 comp rx=0 tx=0).stop 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.374+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8105e70 msgr2=0x7fdfe819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.374+0000 7fdfed400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8105e70 0x7fdfe819b3a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400e9f0 tx=0x7fdfd400ed00 comp rx=0 tx=0).stop 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.375+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 shutdown_connections 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.375+0000 7fdfed400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdfcc0a3cb0 0x7fdfcc0a6170 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.375+0000 7fdfed400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe8106810 0x7fdfe81a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.375+0000 7fdfed400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe8104c70 0x7fdfe819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.375+0000 7fdfed400700 1 --2- 172.21.15.18:0/3635831219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe8105e70 0x7fdfe819b3a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.375+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 >> 172.21.15.18:0/3635831219 conn(0x7fdfe8100210 msgr2=0x7fdfe8109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.375+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 shutdown_connections 2024-04-16T16:56:38.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:56:38.375+0000 7fdfed400700 1 -- 172.21.15.18:0/3635831219 wait complete. 2024-04-16T16:56:39.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:38 smithi043 ceph-mon[36901]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T16:56:39.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:38 smithi043 ceph-mon[36901]: from='osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:56:39.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:38 smithi043 ceph-mon[36901]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T16:56:39.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:39.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:38 smithi043 ceph-mon[36901]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:56:39.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:39.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:39.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:39.082 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:38 smithi018 ceph-mon[31776]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T16:56:39.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:38 smithi018 ceph-mon[31776]: from='osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:56:39.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:38 smithi018 ceph-mon[31776]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T16:56:39.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:39.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:38 smithi018 ceph-mon[31776]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:56:39.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:39.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:39.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:39.137 DEBUG:teuthology.orchestra.run.smithi018:osd.3> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.3.service 2024-04-16T16:56:39.141 INFO:tasks.cephadm:Deploying osd.4 on smithi022 with /dev/vg_nvme/lv_3... 2024-04-16T16:56:39.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T16:56:39.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:38 smithi022 ceph-mon[36023]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T16:56:39.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:38 smithi022 ceph-mon[36023]: from='osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:56:39.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:38 smithi022 ceph-mon[36023]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T16:56:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:38 smithi022 ceph-mon[36023]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:56:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:39.402 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:56:39.696 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T16:56:39.373+0000 7fb31a800700 -1 osd.3 0 waiting for initial osdmap 2024-04-16T16:56:39.696 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T16:56:39.394+0000 7fb310600700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:56:39.800 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:56:39.800 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T16:56:39.801 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:39.801 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:39.801 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:39.801 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: from='osd.3 ' entity='osd.3' 2024-04-16T16:56:39.801 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] boot 2024-04-16T16:56:39.801 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T16:56:39.801 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:40.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:56:40.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T16:56:40.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:40.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:40.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:40.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: from='osd.3 ' entity='osd.3' 2024-04-16T16:56:40.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] boot 2024-04-16T16:56:40.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T16:56:40.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:40.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:56:40.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T16:56:40.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:40.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:40.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:56:40.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: from='osd.3 ' entity='osd.3' 2024-04-16T16:56:40.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: osd.3 [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] boot 2024-04-16T16:56:40.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: osdmap e25: 4 total, 4 up, 4 in 2024-04-16T16:56:40.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:56:40.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:40 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:56:40.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:40 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:56:40.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:40 smithi018 ceph-mon[31776]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T16:56:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:40 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:56:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:40 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:56:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:40 smithi043 ceph-mon[36901]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T16:56:41.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:40 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:56:41.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:40 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:56:41.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:40 smithi022 ceph-mon[36023]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T16:56:41.288 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T16:56:41.288 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T16:56:41.288 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-04-16T16:56:41.288 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-04-16T16:56:41.288 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00841429 s, 1.2 GB/s 2024-04-16T16:56:41.288 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-04-16T16:56:41.891 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:41 smithi018 ceph-mon[31776]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:41.891 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:41 smithi018 ceph-mon[31776]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T16:56:41.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_3 2024-04-16T16:56:42.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:41 smithi043 ceph-mon[36901]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:42.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:41 smithi043 ceph-mon[36901]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T16:56:42.040 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:41 smithi022 ceph-mon[36023]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:42.040 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:41 smithi022 ceph-mon[36023]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T16:56:42.261 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:56:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.886+0000 7f1db12fd700 1 -- 172.21.15.22:0/886342834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 msgr2=0x7f1dac10ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.886+0000 7f1db12fd700 1 --2- 172.21.15.22:0/886342834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 0x7f1dac10ad10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b0a0 tx=0x7f1da400b3b0 comp rx=0 tx=0).stop 2024-04-16T16:56:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.886+0000 7f1db12fd700 1 -- 172.21.15.22:0/886342834 shutdown_connections 2024-04-16T16:56:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.886+0000 7f1db12fd700 1 --2- 172.21.15.22:0/886342834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 0x7f1dac10ad10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.886+0000 7f1db12fd700 1 --2- 172.21.15.22:0/886342834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1dac1036c0 0x7f1dac105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.886+0000 7f1db12fd700 1 --2- 172.21.15.22:0/886342834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac105ff0 0x7f1dac1083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.886+0000 7f1db12fd700 1 -- 172.21.15.22:0/886342834 >> 172.21.15.22:0/886342834 conn(0x7f1dac0fd010 msgr2=0x7f1dac0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.887+0000 7f1db12fd700 1 -- 172.21.15.22:0/886342834 shutdown_connections 2024-04-16T16:56:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.887+0000 7f1db12fd700 1 -- 172.21.15.22:0/886342834 wait complete. 2024-04-16T16:56:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.887+0000 7f1db12fd700 1 Processor -- start 2024-04-16T16:56:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.887+0000 7f1db12fd700 1 -- start start 2024-04-16T16:56:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.887+0000 7f1db12fd700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1dac1036c0 0x7f1dac103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.887+0000 7f1db12fd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac105ff0 0x7f1dac101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1db12fd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 0x7f1dac101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1db12fd700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac10e810 con 0x7f1dac1036c0 2024-04-16T16:56:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1db12fd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac10e690 con 0x7f1dac108920 2024-04-16T16:56:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1db12fd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac10e990 con 0x7f1dac105ff0 2024-04-16T16:56:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 0x7f1dac101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 0x7f1dac101bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39714/0 (socket says 172.21.15.22:39714) 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1daa7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac105ff0 0x7f1dac101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 -- 172.21.15.22:0/3944840846 learned_addr learned my addr 172.21.15.22:0/3944840846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1daaffd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1dac1036c0 0x7f1dac103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 -- 172.21.15.22:0/3944840846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac105ff0 msgr2=0x7f1dac101670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac105ff0 0x7f1dac101670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 -- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1dac1036c0 msgr2=0x7f1dac103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1dac1036c0 0x7f1dac103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 -- 172.21.15.22:0/3944840846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac102460 con 0x7f1dac108920 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1daaffd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1dac1036c0 0x7f1dac103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1dab7fe700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 0x7f1dac101bb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1da4007f00 tx=0x7f1da400be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1d93fff700 1 -- 172.21.15.22:0/3944840846 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4021030 con 0x7f1dac108920 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1d93fff700 1 -- 172.21.15.22:0/3944840846 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da4007b00 con 0x7f1dac108920 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.888+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac1a2ec0 con 0x7f1dac108920 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.889+0000 7f1d93fff700 1 -- 172.21.15.22:0/3944840846 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da401d470 con 0x7f1dac108920 2024-04-16T16:56:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.889+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1a3340 con 0x7f1dac108920 2024-04-16T16:56:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.890+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac02dfe0 con 0x7f1dac108920 2024-04-16T16:56:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.891+0000 7f1d93fff700 1 -- 172.21.15.22:0/3944840846 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1da4024070 con 0x7f1dac108920 2024-04-16T16:56:43.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.891+0000 7f1d93fff700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1d940a3c40 0x7f1d940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:43.895 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:43 smithi022 ceph-mon[36023]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T16:56:43.895 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:43 smithi022 ceph-mon[36023]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.892+0000 7f1d93fff700 1 -- 172.21.15.22:0/3944840846 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f1da400f020 con 0x7f1dac108920 2024-04-16T16:56:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.892+0000 7f1daaffd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1d940a3c40 0x7f1d940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.893+0000 7f1daaffd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1d940a3c40 0x7f1d940a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c007900 tx=0x7f1d9c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:43.896+0000 7f1d93fff700 1 -- 172.21.15.22:0/3944840846 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1da4007cb0 con 0x7f1dac108920 2024-04-16T16:56:44.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:43 smithi043 ceph-mon[36901]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T16:56:44.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:43 smithi043 ceph-mon[36901]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:56:44.115+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1dac062f30 con 0x7f1d940a3c40 2024-04-16T16:56:44.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:43 smithi018 ceph-mon[31776]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T16:56:44.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:43 smithi018 ceph-mon[31776]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:44 smithi018 ceph-mon[31776]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T16:56:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:44 smithi018 ceph-mon[31776]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:44.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:45.066 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:44 smithi022 ceph-mon[36023]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T16:56:45.066 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:44 smithi022 ceph-mon[36023]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:45.066 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:45.066 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:45.066 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:45.066 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:45.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:44 smithi043 ceph-mon[36901]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T16:56:45.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:44 smithi043 ceph-mon[36901]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:45.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:45.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:45.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:45.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.056 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi018 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi018 to 8547M 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-16T16:56:46.057 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi018 2024-04-16T16:56:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:46.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:46.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi018 to 8547M 2024-04-16T16:56:46.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:46.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:46.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-16T16:56:46.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi018 2024-04-16T16:56:46.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:46.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:46.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi018 to 8547M 2024-04-16T16:56:46.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:46.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:46.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:46.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-16T16:56:46.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:56:47.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:46 smithi022 ceph-mon[36023]: from='client.? 172.21.15.22:0/4049453798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]: dispatch 2024-04-16T16:56:47.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:46 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]: dispatch 2024-04-16T16:56:47.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:46 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]': finished 2024-04-16T16:56:47.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:46 smithi022 ceph-mon[36023]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T16:56:47.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:46 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:56:47.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:46 smithi018 ceph-mon[31776]: from='client.? 172.21.15.22:0/4049453798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]: dispatch 2024-04-16T16:56:47.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:46 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]: dispatch 2024-04-16T16:56:47.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:46 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]': finished 2024-04-16T16:56:47.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:46 smithi018 ceph-mon[31776]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T16:56:47.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:46 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:56:47.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:46 smithi043 ceph-mon[36901]: from='client.? 172.21.15.22:0/4049453798' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]: dispatch 2024-04-16T16:56:47.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:46 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]: dispatch 2024-04-16T16:56:47.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:46 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2338ca28-bed3-415a-b762-28d2275a3940"}]': finished 2024-04-16T16:56:47.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:46 smithi043 ceph-mon[36901]: osdmap e30: 5 total, 4 up, 5 in 2024-04-16T16:56:47.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:46 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:56:48.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:47 smithi022 ceph-mon[36023]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:56:48.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:47 smithi022 ceph-mon[36023]: from='client.? 172.21.15.22:0/1578380342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:48.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:47 smithi018 ceph-mon[31776]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:56:48.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:47 smithi018 ceph-mon[31776]: from='client.? 172.21.15.22:0/1578380342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:48.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:47 smithi043 ceph-mon[36901]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:56:48.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:47 smithi043 ceph-mon[36901]: from='client.? 172.21.15.22:0/1578380342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:50.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:49 smithi022 ceph-mon[36023]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-16T16:56:50.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:49 smithi018 ceph-mon[31776]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-16T16:56:50.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:49 smithi043 ceph-mon[36901]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-16T16:56:52.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:51 smithi022 ceph-mon[36023]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T16:56:52.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:51 smithi018 ceph-mon[31776]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T16:56:52.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:51 smithi043 ceph-mon[36901]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T16:56:54.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:53 smithi022 ceph-mon[36023]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-16T16:56:54.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:53 smithi018 ceph-mon[31776]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-16T16:56:54.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:53 smithi043 ceph-mon[36901]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-16T16:56:55.045 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T16:56:55.045 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:55.045 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:54 smithi022 ceph-mon[36023]: Deploying daemon osd.4 on smithi022 2024-04-16T16:56:55.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T16:56:55.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:55.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:54 smithi018 ceph-mon[31776]: Deploying daemon osd.4 on smithi022 2024-04-16T16:56:55.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T16:56:55.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:55.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:54 smithi043 ceph-mon[36901]: Deploying daemon osd.4 on smithi022 2024-04-16T16:56:56.160 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:55 smithi022 ceph-mon[36023]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:56.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:55 smithi018 ceph-mon[31776]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:56.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:55 smithi043 ceph-mon[36901]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:57.990 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:57 smithi022 ceph-mon[36023]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:58.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:57 smithi018 ceph-mon[31776]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:56:58.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:57 smithi043 ceph-mon[36901]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:00.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:59 smithi022 ceph-mon[36023]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:00.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:00.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:00.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:56:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:00.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:59 smithi018 ceph-mon[31776]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:00.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:00.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:00.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:56:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:00.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:59 smithi043 ceph-mon[36901]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:00.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:00.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:00.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:56:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:02.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:01 smithi022 ceph-mon[36023]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:02.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:01 smithi018 ceph-mon[31776]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:02.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:01 smithi043 ceph-mon[36901]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:03.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.406+0000 7f1d93fff700 1 -- 172.21.15.22:0/3944840846 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1dac062f30 con 0x7f1d940a3c40 2024-04-16T16:57:03.408 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 4 on host 'smithi022' 2024-04-16T16:57:03.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1d940a3c40 msgr2=0x7f1d940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1d940a3c40 0x7f1d940a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c007900 tx=0x7f1d9c008040 comp rx=0 tx=0).stop 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 msgr2=0x7f1dac101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 0x7f1dac101bb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1da4007f00 tx=0x7f1da400be60 comp rx=0 tx=0).stop 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 shutdown_connections 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1d940a3c40 0x7f1d940a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1dac108920 0x7f1dac101bb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1dac1036c0 0x7f1dac103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 --2- 172.21.15.22:0/3944840846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac105ff0 0x7f1dac101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 >> 172.21.15.22:0/3944840846 conn(0x7f1dac0fd010 msgr2=0x7f1dac104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 shutdown_connections 2024-04-16T16:57:03.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:57:03.409+0000 7f1db12fd700 1 -- 172.21.15.22:0/3944840846 wait complete. 2024-04-16T16:57:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:03.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:57:03.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:03 smithi022 ceph-mon[36023]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:57:03.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:03.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:03.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:03.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:03.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:03.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:57:03.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:03 smithi018 ceph-mon[31776]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:57:04.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:04.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:04.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:04.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:04.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:04.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:04.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:04.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:04.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:04.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:57:04.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:03 smithi043 ceph-mon[36901]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:57:04.041 DEBUG:teuthology.orchestra.run.smithi022:osd.4> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.4.service 2024-04-16T16:57:04.084 INFO:tasks.cephadm:Deploying osd.5 on smithi043 with /dev/vg_nvme/lv_3... 2024-04-16T16:57:04.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T16:57:04.344 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:57:05.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:05 smithi043 ceph-mon[36901]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T16:57:05.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:05 smithi043 ceph-mon[36901]: from='osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:57:05.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:05 smithi043 ceph-mon[36901]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T16:57:05.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:05 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:05.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:05 smithi043 ceph-mon[36901]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:57:05.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:05 smithi022 ceph-mon[36023]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T16:57:05.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:05 smithi022 ceph-mon[36023]: from='osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:57:05.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:05 smithi022 ceph-mon[36023]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T16:57:05.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:05 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:05.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:05 smithi022 ceph-mon[36023]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:57:05.675 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 16:57:05 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4[47299]: 2024-04-16T16:57:05.372+0000 7ff6eb524700 -1 osd.4 0 waiting for initial osdmap 2024-04-16T16:57:05.675 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 16:57:05 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4[47299]: 2024-04-16T16:57:05.378+0000 7ff6e7317700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:57:05.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:05 smithi018 ceph-mon[31776]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T16:57:05.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:05 smithi018 ceph-mon[31776]: from='osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:57:05.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:05 smithi018 ceph-mon[31776]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T16:57:05.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:05.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:05 smithi018 ceph-mon[31776]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:57:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T16:57:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T16:57:06.298 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-16T16:57:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-16T16:57:06.298 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00832477 s, 1.3 GB/s 2024-04-16T16:57:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-16T16:57:06.607 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:06 smithi043 ceph-mon[36901]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:06.608 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:06 smithi043 ceph-mon[36901]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:57:06.608 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:06 smithi043 ceph-mon[36901]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T16:57:06.608 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:06 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:06.608 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:06 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:06 smithi022 ceph-mon[36023]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:06 smithi022 ceph-mon[36023]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:57:06.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:06 smithi022 ceph-mon[36023]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T16:57:06.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:06 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:06.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:06 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:06.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:06 smithi018 ceph-mon[31776]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:06.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:06 smithi018 ceph-mon[31776]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:57:06.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:06 smithi018 ceph-mon[31776]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T16:57:06.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:06 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:06.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:06 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:06.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-04-16T16:57:07.231 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:57:07.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:57:07.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:57:07.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] boot 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:07.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] boot 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:07.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:57:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:57:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: osd.4 [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] boot 2024-04-16T16:57:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: osdmap e33: 5 total, 5 up, 5 in 2024-04-16T16:57:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:08.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:08 smithi043 ceph-mon[36901]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:08.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:08 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi022 2024-04-16T16:57:08.616 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:08 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi022 to 8542M 2024-04-16T16:57:08.616 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:08 smithi043 ceph-mon[36901]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T16:57:08.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:08 smithi022 ceph-mon[36023]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:08.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:08 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi022 2024-04-16T16:57:08.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:08 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi022 to 8542M 2024-04-16T16:57:08.679 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:08 smithi022 ceph-mon[36023]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T16:57:08.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:08 smithi018 ceph-mon[31776]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:57:08.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:08 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi022 2024-04-16T16:57:08.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:08 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi022 to 8542M 2024-04-16T16:57:08.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:08 smithi018 ceph-mon[31776]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T16:57:08.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 -- 172.21.15.43:0/2974606743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4370102d80 msgr2=0x7f4370109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:08.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 --2- 172.21.15.43:0/2974606743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4370102d80 0x7f4370109630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4368009990 tx=0x7f4368009ca0 comp rx=0 tx=0).stop 2024-04-16T16:57:08.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 -- 172.21.15.43:0/2974606743 shutdown_connections 2024-04-16T16:57:08.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 --2- 172.21.15.43:0/2974606743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4370102d80 0x7f4370109630 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 --2- 172.21.15.43:0/2974606743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43701011e0 0x7f4370101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 --2- 172.21.15.43:0/2974606743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43701023e0 0x7f4370102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 -- 172.21.15.43:0/2974606743 >> 172.21.15.43:0/2974606743 conn(0x7f43700fc760 msgr2=0x7f43700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 -- 172.21.15.43:0/2974606743 shutdown_connections 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 -- 172.21.15.43:0/2974606743 wait complete. 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.848+0000 7f437759c700 1 Processor -- start 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f437759c700 1 -- start start 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f437759c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43701011e0 0x7f43700fe760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f437759c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43701023e0 0x7f437019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f437759c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4370102d80 0x7f43701a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f437759c700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f437010ce50 con 0x7f43701011e0 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f437759c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f437010ccd0 con 0x7f4370102d80 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f437759c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f437010cfd0 con 0x7f43701023e0 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43701023e0 0x7f437019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:08.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43701023e0 0x7f437019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55134/0 (socket says 172.21.15.43:55134) 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 -- 172.21.15.43:0/341012122 learned_addr learned my addr 172.21.15.43:0/341012122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4375b39700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4370102d80 0x7f43701a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 -- 172.21.15.43:0/341012122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4370102d80 msgr2=0x7f43701a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4375338700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43701011e0 0x7f43700fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4370102d80 0x7f43701a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 -- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43701011e0 msgr2=0x7f43700fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43701011e0 0x7f43700fe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 -- 172.21.15.43:0/341012122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43701a0a10 con 0x7f43701023e0 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.849+0000 7f4374b37700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43701023e0 0x7f437019db70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f436400c920 tx=0x7f436400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.850+0000 7f436e7fc700 1 -- 172.21.15.43:0/341012122 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364007b00 con 0x7f43701023e0 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.850+0000 7f436e7fc700 1 -- 172.21.15.43:0/341012122 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4364004d10 con 0x7f43701023e0 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.850+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43701a0d00 con 0x7f43701023e0 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.850+0000 7f436e7fc700 1 -- 172.21.15.43:0/341012122 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364005680 con 0x7f43701023e0 2024-04-16T16:57:08.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.850+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43701a1260 con 0x7f43701023e0 2024-04-16T16:57:08.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.851+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370108970 con 0x7f43701023e0 2024-04-16T16:57:08.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.852+0000 7f436e7fc700 1 -- 172.21.15.43:0/341012122 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4364004750 con 0x7f43701023e0 2024-04-16T16:57:08.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.852+0000 7f436e7fc700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43580a3c40 0x7f43580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:08.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.852+0000 7f436e7fc700 1 -- 172.21.15.43:0/341012122 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f43640cef10 con 0x7f43701023e0 2024-04-16T16:57:08.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.853+0000 7f4375338700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43580a3c40 0x7f43580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:08.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.854+0000 7f4375338700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43580a3c40 0x7f43580a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4370102240 tx=0x7f43600072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:57:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:08.857+0000 7f436e7fc700 1 -- 172.21.15.43:0/341012122 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4364014030 con 0x7f43701023e0 2024-04-16T16:57:09.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:09.077+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4370062f30 con 0x7f43580a3c40 2024-04-16T16:57:09.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:09 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:09.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:09 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:09.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:09 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:09.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:09 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:09.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:09 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:09.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:09 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:09.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:09 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:09.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:09 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:09.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:09 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:10.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:10 smithi022 ceph-mon[36023]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:10.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:10 smithi022 ceph-mon[36023]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:10.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:10 smithi043 ceph-mon[36901]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:10.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:10 smithi043 ceph-mon[36901]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:10.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:10 smithi018 ceph-mon[31776]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:10.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:10 smithi018 ceph-mon[31776]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:12.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:12 smithi043 ceph-mon[36901]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:12.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:12 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/4175539437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]: dispatch 2024-04-16T16:57:12.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:12 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]: dispatch 2024-04-16T16:57:12.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:12 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]': finished 2024-04-16T16:57:12.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:12 smithi043 ceph-mon[36901]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T16:57:12.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:12 smithi022 ceph-mon[36023]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:12 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/4175539437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]: dispatch 2024-04-16T16:57:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:12 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]: dispatch 2024-04-16T16:57:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:12 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]': finished 2024-04-16T16:57:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:12 smithi022 ceph-mon[36023]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T16:57:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:12 smithi018 ceph-mon[31776]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:12 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/4175539437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]: dispatch 2024-04-16T16:57:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:12 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]: dispatch 2024-04-16T16:57:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:12 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd247a85-21b7-4f15-bf4b-4f391dd6e543"}]': finished 2024-04-16T16:57:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:12 smithi018 ceph-mon[31776]: osdmap e35: 6 total, 5 up, 6 in 2024-04-16T16:57:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:13.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:13 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/3349590401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:13.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:13 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/3349590401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:13.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:13 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/3349590401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:14.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:14 smithi018 ceph-mon[31776]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:14.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:14 smithi043 ceph-mon[36901]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:14.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:14 smithi022 ceph-mon[36023]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:16.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:16 smithi018 ceph-mon[31776]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:16.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:16 smithi043 ceph-mon[36901]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:16.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:16 smithi022 ceph-mon[36023]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:18 smithi018 ceph-mon[31776]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:18.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:18 smithi043 ceph-mon[36901]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:18.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:18 smithi022 ceph-mon[36023]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:19.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T16:57:19.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:19.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T16:57:19.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T16:57:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:20.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:20 smithi043 ceph-mon[36901]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:20.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:20 smithi043 ceph-mon[36901]: Deploying daemon osd.5 on smithi043 2024-04-16T16:57:20.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:20 smithi022 ceph-mon[36023]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:20.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:20 smithi022 ceph-mon[36023]: Deploying daemon osd.5 on smithi043 2024-04-16T16:57:20.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:20 smithi018 ceph-mon[31776]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:20.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:20 smithi018 ceph-mon[31776]: Deploying daemon osd.5 on smithi043 2024-04-16T16:57:22.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:22 smithi043 ceph-mon[36901]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:22 smithi022 ceph-mon[36023]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:22.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:22 smithi018 ceph-mon[31776]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:24.697 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:24 smithi043 ceph-mon[36901]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:24.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:24 smithi022 ceph-mon[36023]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:24.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:24 smithi018 ceph-mon[31776]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:26 smithi022 ceph-mon[36023]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:26.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:26 smithi018 ceph-mon[31776]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:26.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:26.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:26.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:26 smithi043 ceph-mon[36901]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:28.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:28 smithi022 ceph-mon[36023]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:28.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:28 smithi018 ceph-mon[31776]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:28.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:28 smithi043 ceph-mon[36901]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:28.666 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 5 on host 'smithi043' 2024-04-16T16:57:28.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.664+0000 7f436e7fc700 1 -- 172.21.15.43:0/341012122 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4370062f30 con 0x7f43580a3c40 2024-04-16T16:57:28.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43580a3c40 msgr2=0x7f43580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:28.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43580a3c40 0x7f43580a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4370102240 tx=0x7f43600072f0 comp rx=0 tx=0).stop 2024-04-16T16:57:28.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43701023e0 msgr2=0x7f437019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:28.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43701023e0 0x7f437019db70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f436400c920 tx=0x7f436400cc30 comp rx=0 tx=0).stop 2024-04-16T16:57:28.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 shutdown_connections 2024-04-16T16:57:28.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43580a3c40 0x7f43580a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:28.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4370102d80 0x7f43701a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:28.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43701011e0 0x7f43700fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:28.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 --2- 172.21.15.43:0/341012122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43701023e0 0x7f437019db70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:28.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 >> 172.21.15.43:0/341012122 conn(0x7f43700fc760 msgr2=0x7f4370105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:57:28.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 shutdown_connections 2024-04-16T16:57:28.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:57:28.667+0000 7f437759c700 1 -- 172.21.15.43:0/341012122 wait complete. 2024-04-16T16:57:29.276 DEBUG:teuthology.orchestra.run.smithi043:osd.5> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.5.service 2024-04-16T16:57:29.279 INFO:tasks.cephadm:Deploying osd.6 on smithi018 with /dev/vg_nvme/lv_2... 2024-04-16T16:57:29.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T16:57:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:29 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:29 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:29 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:29 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:29 smithi018 ceph-mon[31776]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:29 smithi018 ceph-mon[31776]: from='osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:57:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:29 smithi018 ceph-mon[31776]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:57:29.757 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:57:29.811 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:29 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.811 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:29 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.811 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:29 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.811 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:29 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.811 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:29 smithi043 ceph-mon[36901]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:29.811 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:29 smithi043 ceph-mon[36901]: from='osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:57:29.811 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:29 smithi043 ceph-mon[36901]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:57:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:29 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:29 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:29 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:29 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:29.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:29 smithi022 ceph-mon[36023]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:29.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:29 smithi022 ceph-mon[36023]: from='osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:57:29.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:29 smithi022 ceph-mon[36023]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:57:31.060 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:30 smithi043 ceph-mon[36901]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T16:57:31.060 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:30 smithi043 ceph-mon[36901]: from='osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:57:31.060 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:30 smithi043 ceph-mon[36901]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T16:57:31.060 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:30 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:31.060 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:30 smithi043 ceph-mon[36901]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:57:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:30 smithi022 ceph-mon[36023]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T16:57:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:30 smithi022 ceph-mon[36023]: from='osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:57:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:30 smithi022 ceph-mon[36023]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T16:57:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:30 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:30 smithi022 ceph-mon[36023]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:57:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:30 smithi018 ceph-mon[31776]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T16:57:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:30 smithi018 ceph-mon[31776]: from='osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:57:31.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:30 smithi018 ceph-mon[31776]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T16:57:31.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:30 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:31.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:30 smithi018 ceph-mon[31776]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:57:31.503 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 16:57:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T16:57:31.236+0000 7f8efa477700 -1 osd.5 0 waiting for initial osdmap 2024-04-16T16:57:31.503 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 16:57:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T16:57:31.242+0000 7f8ef1a61700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:57:31.807 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:31 smithi018 ceph-mon[31776]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:57:31.807 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:31 smithi018 ceph-mon[31776]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T16:57:31.807 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:31 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:31.808 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:31 smithi018 ceph-mon[31776]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:31.808 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:31 smithi018 ceph-mon[31776]: from='osd.5 ' entity='osd.5' 2024-04-16T16:57:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:31 smithi043 ceph-mon[36901]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:57:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:31 smithi043 ceph-mon[36901]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T16:57:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:31 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:31 smithi043 ceph-mon[36901]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:32.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:31 smithi043 ceph-mon[36901]: from='osd.5 ' entity='osd.5' 2024-04-16T16:57:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:31 smithi022 ceph-mon[36023]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:57:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:31 smithi022 ceph-mon[36023]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T16:57:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:31 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:31 smithi022 ceph-mon[36023]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:57:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:31 smithi022 ceph-mon[36023]: from='osd.5 ' entity='osd.5' 2024-04-16T16:57:32.890 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T16:57:32.890 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T16:57:32.890 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-04-16T16:57:32.890 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-04-16T16:57:32.891 INFO:teuthology.orchestra.run.smithi018.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324774 s, 323 MB/s 2024-04-16T16:57:32.891 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-04-16T16:57:33.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:57:33.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:57:33.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] boot 2024-04-16T16:57:33.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T16:57:33.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:33.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi043 2024-04-16T16:57:33.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi043 to 8542M 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:32 smithi022 ceph-mon[36023]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] boot 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi043 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi043 to 8542M 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:33.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:33.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:33.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:33.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:33.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:32 smithi018 ceph-mon[31776]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: osd.5 [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] boot 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: osdmap e38: 6 total, 6 up, 6 in 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi043 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi043 to 8542M 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:33.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:33.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:33.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:33.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:33.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:33.254 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:32 smithi043 ceph-mon[36901]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T16:57:33.815 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_2 2024-04-16T16:57:34.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:33 smithi022 ceph-mon[36023]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:34.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:33 smithi022 ceph-mon[36023]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T16:57:34.184 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:33 smithi018 ceph-mon[31776]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:34.184 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:33 smithi018 ceph-mon[31776]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T16:57:34.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:33 smithi043 ceph-mon[36901]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:34.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:33 smithi043 ceph-mon[36901]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T16:57:34.261 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:57:36.050 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:35 smithi018 ceph-mon[31776]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T16:57:36.050 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:35 smithi018 ceph-mon[31776]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T16:57:36.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:35 smithi022 ceph-mon[36023]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T16:57:36.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:35 smithi022 ceph-mon[36023]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T16:57:36.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:35 smithi043 ceph-mon[36901]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T16:57:36.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:35 smithi043 ceph-mon[36901]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T16:57:36.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.299+0000 7f6c8b000700 1 -- 172.21.15.18:0/1999601500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c84105f10 msgr2=0x7f6c84108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:36.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.299+0000 7f6c8b000700 1 --2- 172.21.15.18:0/1999601500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c84105f10 0x7f6c84108300 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009990 tx=0x7f6c70009ca0 comp rx=0 tx=0).stop 2024-04-16T16:57:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.300+0000 7f6c8b000700 1 -- 172.21.15.18:0/1999601500 shutdown_connections 2024-04-16T16:57:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.300+0000 7f6c8b000700 1 --2- 172.21.15.18:0/1999601500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c84108840 0x7f6c8410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.300+0000 7f6c8b000700 1 --2- 172.21.15.18:0/1999601500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c841035e0 0x7f6c841059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.300+0000 7f6c8b000700 1 --2- 172.21.15.18:0/1999601500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c84105f10 0x7f6c84108300 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.300+0000 7f6c8b000700 1 -- 172.21.15.18:0/1999601500 >> 172.21.15.18:0/1999601500 conn(0x7f6c840fd010 msgr2=0x7f6c840ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:57:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.300+0000 7f6c8b000700 1 -- 172.21.15.18:0/1999601500 shutdown_connections 2024-04-16T16:57:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.300+0000 7f6c8b000700 1 -- 172.21.15.18:0/1999601500 wait complete. 2024-04-16T16:57:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.301+0000 7f6c8b000700 1 Processor -- start 2024-04-16T16:57:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.301+0000 7f6c8b000700 1 -- start start 2024-04-16T16:57:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.301+0000 7f6c8b000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c841035e0 0x7f6c8419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c8b000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c84105f10 0x7f6c8419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c8b000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c84108840 0x7f6c841a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c8b000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410e780 con 0x7f6c84108840 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c8b000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410e600 con 0x7f6c84105f10 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c8b000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410e900 con 0x7f6c841035e0 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c89000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c84108840 0x7f6c841a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c84105f10 0x7f6c8419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c84105f10 0x7f6c8419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52662/0 (socket says 172.21.15.18:52662) 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83400700 1 -- 172.21.15.18:0/864052444 learned_addr learned my addr 172.21.15.18:0/864052444 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:57:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83e00700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c841035e0 0x7f6c8419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83400700 1 -- 172.21.15.18:0/864052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c841035e0 msgr2=0x7f6c8419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83400700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c841035e0 0x7f6c8419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83400700 1 -- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c84108840 msgr2=0x7f6c841a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83400700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c84108840 0x7f6c841a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.302+0000 7f6c83400700 1 -- 172.21.15.18:0/864052444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c841a2be0 con 0x7f6c84105f10 2024-04-16T16:57:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.303+0000 7f6c83400700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c84105f10 0x7f6c8419b4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b7b0 tx=0x7f6c7000bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:57:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.303+0000 7f6c80c00700 1 -- 172.21.15.18:0/864052444 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70027070 con 0x7f6c84105f10 2024-04-16T16:57:36.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.303+0000 7f6c80c00700 1 -- 172.21.15.18:0/864052444 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c70022470 con 0x7f6c84105f10 2024-04-16T16:57:36.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.303+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c841a2e70 con 0x7f6c84105f10 2024-04-16T16:57:36.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.303+0000 7f6c80c00700 1 -- 172.21.15.18:0/864052444 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000f6a0 con 0x7f6c84105f10 2024-04-16T16:57:36.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.303+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841a3210 con 0x7f6c84105f10 2024-04-16T16:57:36.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.304+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8402dfe0 con 0x7f6c84105f10 2024-04-16T16:57:36.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.307+0000 7f6c80c00700 1 -- 172.21.15.18:0/864052444 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c7002a070 con 0x7f6c84105f10 2024-04-16T16:57:36.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.308+0000 7f6c80c00700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c680a3c40 0x7f6c680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.308+0000 7f6c80c00700 1 -- 172.21.15.18:0/864052444 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f6c700d05a0 con 0x7f6c84105f10 2024-04-16T16:57:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.308+0000 7f6c83e00700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c680a3c40 0x7f6c680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.309+0000 7f6c83e00700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c680a3c40 0x7f6c680a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c74007c60 tx=0x7f6c740073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:57:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.313+0000 7f6c80c00700 1 -- 172.21.15.18:0/864052444 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c7005eff0 con 0x7f6c84105f10 2024-04-16T16:57:36.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:57:36.554+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6c84062f30 con 0x7f6c680a3c40 2024-04-16T16:57:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:36 smithi018 ceph-mon[31776]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T16:57:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:36 smithi018 ceph-mon[31776]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:37.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:36 smithi022 ceph-mon[36023]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T16:57:37.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:36 smithi022 ceph-mon[36023]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:37.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:37.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:37.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:37.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:36 smithi043 ceph-mon[36901]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T16:57:37.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:36 smithi043 ceph-mon[36901]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:37.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:37.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:37.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:37 smithi022 ceph-mon[36023]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-16T16:57:38.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:37 smithi018 ceph-mon[31776]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-16T16:57:38.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:37 smithi043 ceph-mon[36901]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-16T16:57:40.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:39 smithi022 ceph-mon[36023]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:57:40.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:39 smithi018 ceph-mon[31776]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:57:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:39 smithi043 ceph-mon[36901]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:57:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:40 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/342899749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d5bb4a0-908c-41b1-9967-a35d4e77d851"}]: dispatch 2024-04-16T16:57:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:40 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/342899749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d5bb4a0-908c-41b1-9967-a35d4e77d851"}]': finished 2024-04-16T16:57:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:40 smithi043 ceph-mon[36901]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T16:57:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:40 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:57:41.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:40 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/342899749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d5bb4a0-908c-41b1-9967-a35d4e77d851"}]: dispatch 2024-04-16T16:57:41.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:40 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/342899749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d5bb4a0-908c-41b1-9967-a35d4e77d851"}]': finished 2024-04-16T16:57:41.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:40 smithi022 ceph-mon[36023]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T16:57:41.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:40 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:57:41.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:40 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/342899749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d5bb4a0-908c-41b1-9967-a35d4e77d851"}]: dispatch 2024-04-16T16:57:41.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:40 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/342899749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d5bb4a0-908c-41b1-9967-a35d4e77d851"}]': finished 2024-04-16T16:57:41.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:40 smithi018 ceph-mon[31776]: osdmap e43: 7 total, 6 up, 7 in 2024-04-16T16:57:41.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:40 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:57:42.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:41 smithi022 ceph-mon[36023]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:42.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:41 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/524144905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:42.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:41 smithi018 ceph-mon[31776]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:42.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:41 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/524144905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:42.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:41 smithi043 ceph-mon[36901]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:42.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:41 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/524144905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:43.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:43 smithi018 ceph-mon[31776]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:44.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:43 smithi022 ceph-mon[36023]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:44.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:43 smithi043 ceph-mon[36901]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:46.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:45 smithi022 ceph-mon[36023]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:45 smithi018 ceph-mon[31776]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:46.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:45 smithi043 ceph-mon[36901]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:48.141 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:47 smithi018 ceph-mon[31776]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:48.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:47 smithi022 ceph-mon[36023]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:48.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:47 smithi043 ceph-mon[36901]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:50.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:49 smithi022 ceph-mon[36023]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:50.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:49 smithi018 ceph-mon[31776]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:50.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:49 smithi043 ceph-mon[36901]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:51.126 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:50 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T16:57:51.126 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:50 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:51.126 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:50 smithi018 ceph-mon[31776]: Deploying daemon osd.6 on smithi018 2024-04-16T16:57:51.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:50 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T16:57:51.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:50 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:51.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:50 smithi022 ceph-mon[36023]: Deploying daemon osd.6 on smithi018 2024-04-16T16:57:51.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:50 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T16:57:51.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:50 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:51.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:50 smithi043 ceph-mon[36901]: Deploying daemon osd.6 on smithi018 2024-04-16T16:57:52.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:51 smithi022 ceph-mon[36023]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:52.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:51 smithi018 ceph-mon[31776]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:52.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:51 smithi043 ceph-mon[36901]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:53.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:53 smithi018 ceph-mon[31776]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:54.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:53 smithi022 ceph-mon[36023]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:54.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:53 smithi043 ceph-mon[36901]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:56.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:55 smithi022 ceph-mon[36023]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:55 smithi018 ceph-mon[31776]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:56.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:55 smithi043 ceph-mon[36901]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:57.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:57 smithi018 ceph-mon[31776]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:57 smithi022 ceph-mon[36023]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:58.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:57 smithi043 ceph-mon[36901]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:57:59.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:59.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:59.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:57:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:59.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:59.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:59.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:57:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:57:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:57:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:00.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:00 smithi022 ceph-mon[36023]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:00.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:00 smithi043 ceph-mon[36901]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:00.855 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:00 smithi018 ceph-mon[31776]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:02.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:02 smithi018 ceph-mon[31776]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:02.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:02.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:02.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:02 smithi018 ceph-mon[31776]: from='osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T16:58:02.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:02 smithi018 ceph-mon[31776]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T16:58:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:02 smithi043 ceph-mon[36901]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:02 smithi043 ceph-mon[36901]: from='osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T16:58:02.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:02 smithi043 ceph-mon[36901]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T16:58:02.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:02 smithi022 ceph-mon[36023]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:02.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:02.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:02.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:02 smithi022 ceph-mon[36023]: from='osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T16:58:02.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:02 smithi022 ceph-mon[36023]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T16:58:03.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.149+0000 7f6c80c00700 1 -- 172.21.15.18:0/864052444 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6c84062f30 con 0x7f6c680a3c40 2024-04-16T16:58:03.152 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 6 on host 'smithi018' 2024-04-16T16:58:03.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.152+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c680a3c40 msgr2=0x7f6c680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:03.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.152+0000 7f6c8b000700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c680a3c40 0x7f6c680a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c74007c60 tx=0x7f6c740073d0 comp rx=0 tx=0).stop 2024-04-16T16:58:03.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.152+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c84105f10 msgr2=0x7f6c8419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:03.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.152+0000 7f6c8b000700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c84105f10 0x7f6c8419b4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b7b0 tx=0x7f6c7000bbb0 comp rx=0 tx=0).stop 2024-04-16T16:58:03.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.153+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 shutdown_connections 2024-04-16T16:58:03.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.153+0000 7f6c8b000700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6c680a3c40 0x7f6c680a6100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:03.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.153+0000 7f6c8b000700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c84108840 0x7f6c841a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:03.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.153+0000 7f6c8b000700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c841035e0 0x7f6c8419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:03.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.153+0000 7f6c8b000700 1 --2- 172.21.15.18:0/864052444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c84105f10 0x7f6c8419b4c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:03.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.153+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 >> 172.21.15.18:0/864052444 conn(0x7f6c840fd010 msgr2=0x7f6c84102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:03.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.153+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 shutdown_connections 2024-04-16T16:58:03.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:58:03.153+0000 7f6c8b000700 1 -- 172.21.15.18:0/864052444 wait complete. 2024-04-16T16:58:03.950 DEBUG:teuthology.orchestra.run.smithi018:osd.6> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.6.service 2024-04-16T16:58:03.956 INFO:tasks.cephadm:Deploying osd.7 on smithi022 with /dev/vg_nvme/lv_2... 2024-04-16T16:58:03.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T16:58:04.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-mon[31776]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:04.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-mon[31776]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T16:58:04.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-mon[31776]: from='osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:58:04.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-mon[31776]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T16:58:04.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:04.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-mon[31776]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:58:04.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:04.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:04.222 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:58:04.344 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:04 smithi022 ceph-mon[36023]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:04.344 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:04 smithi022 ceph-mon[36023]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T16:58:04.344 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:04 smithi022 ceph-mon[36023]: from='osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:58:04.344 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:04 smithi022 ceph-mon[36023]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T16:58:04.344 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:04.344 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:04 smithi022 ceph-mon[36023]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:58:04.344 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:04.344 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:04 smithi043 ceph-mon[36901]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:04 smithi043 ceph-mon[36901]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T16:58:04.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:04 smithi043 ceph-mon[36901]: from='osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:58:04.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:04 smithi043 ceph-mon[36901]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T16:58:04.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:04.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:04 smithi043 ceph-mon[36901]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:58:04.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:04.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:04.947 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T16:58:04.649+0000 7f1f2dc00700 -1 osd.6 0 waiting for initial osdmap 2024-04-16T16:58:04.947 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 16:58:04 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T16:58:04.669+0000 7f1f23a00700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:58:05.383 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:05 smithi022 ceph-mon[36023]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:58:05.383 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:05 smithi022 ceph-mon[36023]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T16:58:05.383 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:05 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:05.383 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:05 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:05.383 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:05 smithi022 ceph-mon[36023]: from='osd.6 ' entity='osd.6' 2024-04-16T16:58:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:05 smithi018 ceph-mon[31776]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:58:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:05 smithi018 ceph-mon[31776]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T16:58:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:05.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:05.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:05 smithi018 ceph-mon[31776]: from='osd.6 ' entity='osd.6' 2024-04-16T16:58:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:05 smithi043 ceph-mon[36901]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:58:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:05 smithi043 ceph-mon[36901]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T16:58:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:05 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:05 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:05.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:05 smithi043 ceph-mon[36901]: from='osd.6 ' entity='osd.6' 2024-04-16T16:58:06.156 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:06 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:58:06.156 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:06 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:58:06.156 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:06 smithi018 ceph-mon[31776]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:06.156 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:06 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:06.157 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:06 smithi018 ceph-mon[31776]: osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] boot 2024-04-16T16:58:06.157 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:06 smithi018 ceph-mon[31776]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T16:58:06.157 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:06 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:06 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:58:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:06 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:58:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:06 smithi022 ceph-mon[36023]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:06 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:06 smithi022 ceph-mon[36023]: osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] boot 2024-04-16T16:58:06.177 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:06 smithi022 ceph-mon[36023]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T16:58:06.177 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:06 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T16:58:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T16:58:06.305 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-04-16T16:58:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-04-16T16:58:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00839992 s, 1.2 GB/s 2024-04-16T16:58:06.305 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 2024-04-16T16:58:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-04-16T16:58:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:06 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:58:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:06 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:58:06.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:06 smithi043 ceph-mon[36901]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:58:06.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:06 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:06.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:06 smithi043 ceph-mon[36901]: osd.6 [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] boot 2024-04-16T16:58:06.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:06 smithi043 ceph-mon[36901]: osdmap e46: 7 total, 7 up, 7 in 2024-04-16T16:58:06.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:06 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:58:06.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_2 2024-04-16T16:58:07.130 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:58:08.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:08 smithi022 ceph-mon[36023]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:08.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:08 smithi022 ceph-mon[36023]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T16:58:08.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:08 smithi018 ceph-mon[31776]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:08.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:08 smithi018 ceph-mon[31776]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T16:58:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:08 smithi043 ceph-mon[36901]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:08 smithi043 ceph-mon[36901]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T16:58:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- 172.21.15.22:0/1457743622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 msgr2=0x7eff7010afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 --2- 172.21.15.22:0/1457743622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 0x7eff7010afc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff6800b0d0 tx=0x7eff6800b3e0 comp rx=0 tx=0).stop 2024-04-16T16:58:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- 172.21.15.22:0/1457743622 shutdown_connections 2024-04-16T16:58:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 --2- 172.21.15.22:0/1457743622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 0x7eff7010afc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 --2- 172.21.15.22:0/1457743622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff70102b70 0x7eff70102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:08.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 --2- 172.21.15.22:0/1457743622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff70103d70 0x7eff701041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- 172.21.15.22:0/1457743622 >> 172.21.15.22:0/1457743622 conn(0x7eff700fe110 msgr2=0x7eff70100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- 172.21.15.22:0/1457743622 shutdown_connections 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- 172.21.15.22:0/1457743622 wait complete. 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 Processor -- start 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- start start 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff70102b70 0x7eff70076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff70103d70 0x7eff70074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 0x7eff70075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7eff7010e7d0 con 0x7eff70103d70 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7eff7010e650 con 0x7eff70104710 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff76540700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff7010e950 con 0x7eff70102b70 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 0x7eff70075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 0x7eff70075420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53414/0 (socket says 172.21.15.22:53414) 2024-04-16T16:58:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 -- 172.21.15.22:0/4106522950 learned_addr learned my addr 172.21.15.22:0/4106522950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff6ffff700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff70102b70 0x7eff70076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 -- 172.21.15.22:0/4106522950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff70102b70 msgr2=0x7eff70076890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff70102b70 0x7eff70076890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 -- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff70103d70 msgr2=0x7eff70074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff70103d70 0x7eff70074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 -- 172.21.15.22:0/4106522950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff70075cd0 con 0x7eff70104710 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.733+0000 7eff74add700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 0x7eff70075420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff68006070 tx=0x7eff68007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.734+0000 7eff6d7fa700 1 -- 172.21.15.22:0/4106522950 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68021020 con 0x7eff70104710 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.734+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff701a2e90 con 0x7eff70104710 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.734+0000 7eff6d7fa700 1 -- 172.21.15.22:0/4106522950 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6801d950 con 0x7eff70104710 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.734+0000 7eff6d7fa700 1 -- 172.21.15.22:0/4106522950 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68005390 con 0x7eff70104710 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.734+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701a3370 con 0x7eff70104710 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.735+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff700697e0 con 0x7eff70104710 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.736+0000 7eff6d7fa700 1 -- 172.21.15.22:0/4106522950 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff68024020 con 0x7eff70104710 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.736+0000 7eff6d7fa700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7eff580a3c40 0x7eff580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.737+0000 7eff6d7fa700 1 -- 172.21.15.22:0/4106522950 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7eff6800f020 con 0x7eff70104710 2024-04-16T16:58:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.737+0000 7eff6ffff700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7eff580a3c40 0x7eff580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.738+0000 7eff6ffff700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7eff580a3c40 0x7eff580a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff70103bd0 tx=0x7eff60009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:58:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.741+0000 7eff6d7fa700 1 -- 172.21.15.22:0/4106522950 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff6809e7e0 con 0x7eff70104710 2024-04-16T16:58:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:08.959+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7eff70062f30 con 0x7eff580a3c40 2024-04-16T16:58:09.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:09 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:09.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:09 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:09.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:09 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:09.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:09 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:09.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:09 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:09.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:09 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:09.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:09 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:09.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:09 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:09.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:09 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi018 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi018 to 5698M 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.651 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:10.652 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.652 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:10.652 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.652 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:10.652 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:10 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi018 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi018 to 5698M 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:10.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:10 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi018 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi018 to 5698M 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:10.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:10 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:12.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:12 smithi022 ceph-mon[36023]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:12.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:12 smithi022 ceph-mon[36023]: from='client.? 172.21.15.22:0/2767636806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]: dispatch 2024-04-16T16:58:12.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:12 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]: dispatch 2024-04-16T16:58:12.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:12 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]': finished 2024-04-16T16:58:12.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:12 smithi022 ceph-mon[36023]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T16:58:12.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:12 smithi018 ceph-mon[31776]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:12 smithi018 ceph-mon[31776]: from='client.? 172.21.15.22:0/2767636806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]: dispatch 2024-04-16T16:58:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:12 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]: dispatch 2024-04-16T16:58:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:12 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]': finished 2024-04-16T16:58:12.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:12 smithi018 ceph-mon[31776]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T16:58:12.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:12 smithi043 ceph-mon[36901]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:12 smithi043 ceph-mon[36901]: from='client.? 172.21.15.22:0/2767636806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]: dispatch 2024-04-16T16:58:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:12 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]: dispatch 2024-04-16T16:58:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:12 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3917a58-159e-4672-a8b1-83b0fe54efbb"}]': finished 2024-04-16T16:58:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:12 smithi043 ceph-mon[36901]: osdmap e48: 8 total, 7 up, 8 in 2024-04-16T16:58:12.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:13.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:13 smithi022 ceph-mon[36023]: from='client.? 172.21.15.22:0/720547285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:13.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:13 smithi018 ceph-mon[31776]: from='client.? 172.21.15.22:0/720547285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:13.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:13 smithi043 ceph-mon[36901]: from='client.? 172.21.15.22:0/720547285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:14.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:14 smithi022 ceph-mon[36023]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:14.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:14 smithi018 ceph-mon[31776]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:14.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:14 smithi043 ceph-mon[36901]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:16.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:16 smithi022 ceph-mon[36023]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:16.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:16 smithi018 ceph-mon[31776]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:16.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:16 smithi043 ceph-mon[36901]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:18.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:18 smithi022 ceph-mon[36023]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:18 smithi018 ceph-mon[31776]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:18.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:18 smithi043 ceph-mon[36901]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:19.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T16:58:19.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:19.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T16:58:19.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:19.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T16:58:19.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:20.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:20 smithi022 ceph-mon[36023]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:20.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:20 smithi022 ceph-mon[36023]: Deploying daemon osd.7 on smithi022 2024-04-16T16:58:20.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:20 smithi018 ceph-mon[31776]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:20.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:20 smithi018 ceph-mon[31776]: Deploying daemon osd.7 on smithi022 2024-04-16T16:58:20.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:20 smithi043 ceph-mon[36901]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:20.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:20 smithi043 ceph-mon[36901]: Deploying daemon osd.7 on smithi022 2024-04-16T16:58:22.612 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:22 smithi022 ceph-mon[36023]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:22.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:22 smithi018 ceph-mon[31776]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:22.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:22 smithi043 ceph-mon[36901]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:24.572 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:24 smithi022 ceph-mon[36023]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:24.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:24 smithi018 ceph-mon[31776]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:24.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:24 smithi043 ceph-mon[36901]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:25 smithi022 ceph-mon[36023]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:25 smithi018 ceph-mon[31776]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:25 smithi043 ceph-mon[36901]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:28.156 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:27 smithi022 ceph-mon[36023]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:28.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:27 smithi018 ceph-mon[31776]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:28.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:27 smithi043 ceph-mon[36901]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:28.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.847+0000 7eff6d7fa700 1 -- 172.21.15.22:0/4106522950 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff70062f30 con 0x7eff580a3c40 2024-04-16T16:58:28.849 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 7 on host 'smithi022' 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7eff580a3c40 msgr2=0x7eff580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7eff580a3c40 0x7eff580a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff70103bd0 tx=0x7eff60009190 comp rx=0 tx=0).stop 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 msgr2=0x7eff70075420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 0x7eff70075420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff68006070 tx=0x7eff68007d10 comp rx=0 tx=0).stop 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 shutdown_connections 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7eff580a3c40 0x7eff580a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff70104710 0x7eff70075420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff70102b70 0x7eff70076890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 --2- 172.21.15.22:0/4106522950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff70103d70 0x7eff70074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 >> 172.21.15.22:0/4106522950 conn(0x7eff700fe110 msgr2=0x7eff70107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:28.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 shutdown_connections 2024-04-16T16:58:28.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:58:28.849+0000 7eff76540700 1 -- 172.21.15.22:0/4106522950 wait complete. 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.162 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:29.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:29.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:29.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T16:58:29.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T16:58:29.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:29.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:29.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:29.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T16:58:29.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T16:58:29.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:29.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:29.495 DEBUG:teuthology.orchestra.run.smithi022:osd.7> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.7.service 2024-04-16T16:58:29.497 INFO:tasks.cephadm:Deploying osd.8 on smithi043 with /dev/vg_nvme/lv_2... 2024-04-16T16:58:29.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T16:58:29.744 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:58:30.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:30 smithi043 ceph-mon[36901]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:30.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:30 smithi043 ceph-mon[36901]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T16:58:30.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:30 smithi043 ceph-mon[36901]: from='osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:58:30.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:30 smithi043 ceph-mon[36901]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T16:58:30.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:30 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:30.123 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:30 smithi043 ceph-mon[36901]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:58:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:30 smithi022 ceph-mon[36023]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:30 smithi022 ceph-mon[36023]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T16:58:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:30 smithi022 ceph-mon[36023]: from='osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:58:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:30 smithi022 ceph-mon[36023]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T16:58:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:30 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:30 smithi022 ceph-mon[36023]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:58:30.175 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 16:58:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T16:58:30.007+0000 7f8cdcf5b700 -1 osd.7 0 waiting for initial osdmap 2024-04-16T16:58:30.175 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 16:58:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T16:58:30.016+0000 7f8cd8d4e700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:58:30.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:30 smithi018 ceph-mon[31776]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:30.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:30 smithi018 ceph-mon[31776]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T16:58:30.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:30 smithi018 ceph-mon[31776]: from='osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:58:30.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:30 smithi018 ceph-mon[31776]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T16:58:30.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:30 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:30.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:30 smithi018 ceph-mon[31776]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:58:31.167 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:31 smithi043 ceph-mon[36901]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:58:31.168 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:31 smithi043 ceph-mon[36901]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T16:58:31.168 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:31 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:31.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:31 smithi022 ceph-mon[36023]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:58:31.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:31 smithi022 ceph-mon[36023]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T16:58:31.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:31 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:31.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:31 smithi018 ceph-mon[31776]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:58:31.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:31 smithi018 ceph-mon[31776]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T16:58:31.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:31 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:32.039 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T16:58:32.039 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T16:58:32.039 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-16T16:58:32.039 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-16T16:58:32.039 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00848345 s, 1.2 GB/s 2024-04-16T16:58:32.039 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-16T16:58:32.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:32 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:58:32.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:32 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:58:32.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:32 smithi043 ceph-mon[36901]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:32.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:32 smithi043 ceph-mon[36901]: osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] boot 2024-04-16T16:58:32.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:32 smithi043 ceph-mon[36901]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T16:58:32.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:32.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:32 smithi043 ceph-mon[36901]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T16:58:32.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:32 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:58:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:32 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:58:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:32 smithi022 ceph-mon[36023]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:32 smithi022 ceph-mon[36023]: osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] boot 2024-04-16T16:58:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:32 smithi022 ceph-mon[36023]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T16:58:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:32 smithi022 ceph-mon[36023]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T16:58:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:32 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:58:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:32 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:58:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:32 smithi018 ceph-mon[31776]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-16T16:58:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:32 smithi018 ceph-mon[31776]: osd.7 [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] boot 2024-04-16T16:58:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:32 smithi018 ceph-mon[31776]: osdmap e51: 8 total, 8 up, 8 in 2024-04-16T16:58:32.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T16:58:32.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:32 smithi018 ceph-mon[31776]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T16:58:32.705 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-04-16T16:58:32.959 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:58:33.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi022 2024-04-16T16:58:33.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi022 to 5694M 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:33 smithi043 ceph-mon[36901]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi022 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi022 to 5694M 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:33.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:33.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:33 smithi022 ceph-mon[36023]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T16:58:33.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi022 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi022 to 5694M 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:33.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:33 smithi018 ceph-mon[31776]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T16:58:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 -- 172.21.15.43:0/1960145219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918102c50 msgr2=0x7fa9181030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 --2- 172.21.15.43:0/1960145219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918102c50 0x7fa9181030b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa908009990 tx=0x7fa908009ca0 comp rx=0 tx=0).stop 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 -- 172.21.15.43:0/1960145219 shutdown_connections 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 --2- 172.21.15.43:0/1960145219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9181035f0 0x7fa918109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 --2- 172.21.15.43:0/1960145219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa918101a50 0x7fa918101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 --2- 172.21.15.43:0/1960145219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918102c50 0x7fa9181030b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 -- 172.21.15.43:0/1960145219 >> 172.21.15.43:0/1960145219 conn(0x7fa9180fcff0 msgr2=0x7fa9180ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 -- 172.21.15.43:0/1960145219 shutdown_connections 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 -- 172.21.15.43:0/1960145219 wait complete. 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 Processor -- start 2024-04-16T16:58:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.546+0000 7fa91e76f700 1 -- start start 2024-04-16T16:58:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa91e76f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101a50 0x7fa91806bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa91e76f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa918102c50 0x7fa91819ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa91e76f700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9181035f0 0x7fa9181a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa91e76f700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810d700 con 0x7fa9181035f0 2024-04-16T16:58:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa91e76f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810d580 con 0x7fa918102c50 2024-04-16T16:58:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa91e76f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810d880 con 0x7fa918101a50 2024-04-16T16:58:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa917fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101a50 0x7fa91806bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa917fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101a50 0x7fa91806bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33278/0 (socket says 172.21.15.43:33278) 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa917fff700 1 -- 172.21.15.43:0/371674572 learned_addr learned my addr 172.21.15.43:0/371674572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa9177fe700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa918102c50 0x7fa91819ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa917fff700 1 -- 172.21.15.43:0/371674572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa918102c50 msgr2=0x7fa91819ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa917fff700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa918102c50 0x7fa91819ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa917fff700 1 -- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9181035f0 msgr2=0x7fa9181a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa917fff700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9181035f0 0x7fa9181a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa917fff700 1 -- 172.21.15.43:0/371674572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9181a2cd0 con 0x7fa918101a50 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.547+0000 7fa9177fe700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa918102c50 0x7fa91819ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.548+0000 7fa917fff700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101a50 0x7fa91806bd80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b500 tx=0x7fa90c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.548+0000 7fa9157fa700 1 -- 172.21.15.43:0/371674572 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c015020 con 0x7fa918101a50 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.548+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9181a2fc0 con 0x7fa918101a50 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.548+0000 7fa9157fa700 1 -- 172.21.15.43:0/371674572 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c011d10 con 0x7fa918101a50 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.548+0000 7fa9157fa700 1 -- 172.21.15.43:0/371674572 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00f2e0 con 0x7fa918101a50 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.548+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9181a34f0 con 0x7fa918101a50 2024-04-16T16:58:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.549+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9180697e0 con 0x7fa918101a50 2024-04-16T16:58:34.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.550+0000 7fa9157fa700 1 -- 172.21.15.43:0/371674572 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa90c018020 con 0x7fa918101a50 2024-04-16T16:58:34.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.550+0000 7fa9157fa700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9000a3c40 0x7fa9000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:34.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.551+0000 7fa9157fa700 1 -- 172.21.15.43:0/371674572 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7fa90c0cf4b0 con 0x7fa918101a50 2024-04-16T16:58:34.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.551+0000 7fa9177fe700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9000a3c40 0x7fa9000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:34.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.552+0000 7fa9177fe700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9000a3c40 0x7fa9000a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b5c0 tx=0x7fa908005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:58:34.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.554+0000 7fa9157fa700 1 -- 172.21.15.43:0/371674572 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa90c01f020 con 0x7fa918101a50 2024-04-16T16:58:34.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:34 smithi043 ceph-mon[36901]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:34.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:34 smithi043 ceph-mon[36901]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-16T16:58:34.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:34.774+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa918062f30 con 0x7fa9000a3c40 2024-04-16T16:58:34.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:34 smithi022 ceph-mon[36023]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:34.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:34 smithi022 ceph-mon[36023]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-16T16:58:34.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:34 smithi018 ceph-mon[31776]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:34.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:34 smithi018 ceph-mon[31776]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-16T16:58:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:35 smithi043 ceph-mon[36901]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T16:58:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:35 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:35 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:35.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:35 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:35 smithi022 ceph-mon[36023]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T16:58:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:35 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:35 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:35.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:35 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:35.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:35 smithi018 ceph-mon[31776]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T16:58:35.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:35 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:35.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:35 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:35.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:35 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:36.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:36 smithi043 ceph-mon[36901]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:36.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:36 smithi043 ceph-mon[36901]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:36.748 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:36 smithi043 ceph-mon[36901]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T16:58:36.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:36 smithi022 ceph-mon[36023]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:36 smithi022 ceph-mon[36023]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:36 smithi022 ceph-mon[36023]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T16:58:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:36 smithi018 ceph-mon[31776]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:36 smithi018 ceph-mon[31776]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:36 smithi018 ceph-mon[31776]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T16:58:37.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:37 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/1431037542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]: dispatch 2024-04-16T16:58:37.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:37 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]: dispatch 2024-04-16T16:58:37.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:37 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]': finished 2024-04-16T16:58:37.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:37 smithi043 ceph-mon[36901]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T16:58:37.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:37.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:37 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/1431037542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]: dispatch 2024-04-16T16:58:37.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:37 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]: dispatch 2024-04-16T16:58:37.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:37 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]': finished 2024-04-16T16:58:37.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:37 smithi022 ceph-mon[36023]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T16:58:37.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:37 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/1431037542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]: dispatch 2024-04-16T16:58:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:37 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]: dispatch 2024-04-16T16:58:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:37 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06e72a0a-097f-49cb-9626-d0de66ba5641"}]': finished 2024-04-16T16:58:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:37 smithi018 ceph-mon[31776]: osdmap e56: 9 total, 8 up, 9 in 2024-04-16T16:58:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:38.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:38 smithi022 ceph-mon[36023]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:38.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:38 smithi022 ceph-mon[36023]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-16T16:58:38.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:38 smithi022 ceph-mon[36023]: Cluster is now healthy 2024-04-16T16:58:38.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:38 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/3340497592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:38.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:38 smithi018 ceph-mon[31776]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:38.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:38 smithi018 ceph-mon[31776]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-16T16:58:38.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:38 smithi018 ceph-mon[31776]: Cluster is now healthy 2024-04-16T16:58:38.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:38 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/3340497592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:39.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:38 smithi043 ceph-mon[36901]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:39.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:38 smithi043 ceph-mon[36901]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-16T16:58:39.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:38 smithi043 ceph-mon[36901]: Cluster is now healthy 2024-04-16T16:58:39.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:38 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/3340497592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:40 smithi022 ceph-mon[36023]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:40.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:40 smithi018 ceph-mon[31776]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:40 smithi043 ceph-mon[36901]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:42.866 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:42 smithi043 ceph-mon[36901]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:42.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:42 smithi022 ceph-mon[36023]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:42 smithi018 ceph-mon[31776]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:43.820 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:43 smithi043 ceph-mon[36901]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:43.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:43 smithi022 ceph-mon[36023]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:43.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:43 smithi018 ceph-mon[31776]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:45.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T16:58:45.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:44 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:45.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T16:58:45.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:44 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:45.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T16:58:45.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:44 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:46.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:45 smithi022 ceph-mon[36023]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:46.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:45 smithi022 ceph-mon[36023]: Deploying daemon osd.8 on smithi043 2024-04-16T16:58:46.177 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:45 smithi043 ceph-mon[36901]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:46.177 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:45 smithi043 ceph-mon[36901]: Deploying daemon osd.8 on smithi043 2024-04-16T16:58:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:45 smithi018 ceph-mon[31776]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:45 smithi018 ceph-mon[31776]: Deploying daemon osd.8 on smithi043 2024-04-16T16:58:47.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:47 smithi043 ceph-mon[36901]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:48.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:47 smithi022 ceph-mon[36023]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:48.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:47 smithi018 ceph-mon[31776]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:50.178 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:49 smithi043 ceph-mon[36901]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:50.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:49 smithi018 ceph-mon[31776]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:50.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:49 smithi022 ceph-mon[36023]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:51 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:51 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:51 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:51.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:51 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:51.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:51 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:51.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:51 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:52.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:51 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:52.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:51 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:52.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:51 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:52.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:52 smithi022 ceph-mon[36023]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:52.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:52 smithi018 ceph-mon[31776]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:53.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:52 smithi043 ceph-mon[36901]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:54.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.561+0000 7fa9157fa700 1 -- 172.21.15.43:0/371674572 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa918062f30 con 0x7fa9000a3c40 2024-04-16T16:58:54.564 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 8 on host 'smithi043' 2024-04-16T16:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9000a3c40 msgr2=0x7fa9000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9000a3c40 0x7fa9000a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b5c0 tx=0x7fa908005fb0 comp rx=0 tx=0).stop 2024-04-16T16:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101a50 msgr2=0x7fa91806bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101a50 0x7fa91806bd80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b500 tx=0x7fa90c00b8c0 comp rx=0 tx=0).stop 2024-04-16T16:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 shutdown_connections 2024-04-16T16:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9000a3c40 0x7fa9000a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9181035f0 0x7fa9181a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101a50 0x7fa91806bd80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 --2- 172.21.15.43:0/371674572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa918102c50 0x7fa91819ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 >> 172.21.15.43:0/371674572 conn(0x7fa9180fcff0 msgr2=0x7fa918105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 shutdown_connections 2024-04-16T16:58:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T16:58:54.564+0000 7fa91e76f700 1 -- 172.21.15.43:0/371674572 wait complete. 2024-04-16T16:58:54.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:54.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:54.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:54.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T16:58:54.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T16:58:54.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:54 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:54 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:54.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:54.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:54.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:54.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T16:58:54.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T16:58:54.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:54.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:54 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:55.241 DEBUG:teuthology.orchestra.run.smithi043:osd.8> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.8.service 2024-04-16T16:58:55.243 INFO:tasks.cephadm:Deploying osd.9 on smithi018 with /dev/vg_nvme/lv_1... 2024-04-16T16:58:55.243 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T16:58:55.724 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:58:55.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:55 smithi022 ceph-mon[36023]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T16:58:55.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:55 smithi022 ceph-mon[36023]: from='osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:58:55.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:55 smithi022 ceph-mon[36023]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T16:58:55.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:55.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:55 smithi022 ceph-mon[36023]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:58:55.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:55 smithi022 ceph-mon[36023]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:55 smithi018 ceph-mon[31776]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T16:58:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:55 smithi018 ceph-mon[31776]: from='osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:58:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:55 smithi018 ceph-mon[31776]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T16:58:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:55 smithi018 ceph-mon[31776]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:58:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:55 smithi018 ceph-mon[31776]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:56.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:55 smithi043 ceph-mon[36901]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T16:58:56.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:55 smithi043 ceph-mon[36901]: from='osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:58:56.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:55 smithi043 ceph-mon[36901]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T16:58:56.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:56.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:55 smithi043 ceph-mon[36901]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T16:58:56.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:55 smithi043 ceph-mon[36901]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-16T16:58:56.405 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 16:58:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T16:58:56.111+0000 7f7121333700 -1 osd.8 0 waiting for initial osdmap 2024-04-16T16:58:56.405 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 16:58:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T16:58:56.123+0000 7f711911e700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:58:56.793 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:56 smithi018 ceph-mon[31776]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:58:56.793 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:56 smithi018 ceph-mon[31776]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T16:58:56.793 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:56 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:56.793 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:56 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:56.793 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:56 smithi018 ceph-mon[31776]: from='osd.8 ' entity='osd.8' 2024-04-16T16:58:56.793 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:56 smithi018 ceph-mon[31776]: osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] boot 2024-04-16T16:58:56.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:56 smithi043 ceph-mon[36901]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:58:56.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:56 smithi043 ceph-mon[36901]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T16:58:56.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:56 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:56.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:56 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:56.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:56 smithi043 ceph-mon[36901]: from='osd.8 ' entity='osd.8' 2024-04-16T16:58:56.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:56 smithi043 ceph-mon[36901]: osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] boot 2024-04-16T16:58:57.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:56 smithi022 ceph-mon[36023]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T16:58:57.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:56 smithi022 ceph-mon[36023]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T16:58:57.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:56 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:57.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:56 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:57.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:56 smithi022 ceph-mon[36023]: from='osd.8 ' entity='osd.8' 2024-04-16T16:58:57.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:56 smithi022 ceph-mon[36023]: osd.8 [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] boot 2024-04-16T16:58:57.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:57 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:58:57.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:57 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:58:57.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:57 smithi018 ceph-mon[31776]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T16:58:57.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:57 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:57.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:57 smithi018 ceph-mon[31776]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:58:57.782 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:57 smithi018 ceph-mon[31776]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T16:58:57.854 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:57 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:58:57.854 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:57 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:58:57.854 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:57 smithi043 ceph-mon[36901]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T16:58:57.854 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:57 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:57.855 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:57 smithi043 ceph-mon[36901]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:58:57.855 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:57 smithi043 ceph-mon[36901]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T16:58:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:57 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:58:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:57 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:58:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:57 smithi022 ceph-mon[36023]: osdmap e59: 9 total, 9 up, 9 in 2024-04-16T16:58:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:57 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T16:58:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:57 smithi022 ceph-mon[36023]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:58:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:57 smithi022 ceph-mon[36023]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T16:58:58.711 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T16:58:58.711 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T16:58:58.711 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-04-16T16:58:58.711 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-04-16T16:58:58.711 INFO:teuthology.orchestra.run.smithi018.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0364494 s, 288 MB/s 2024-04-16T16:58:58.711 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-04-16T16:58:59.451 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_1 2024-04-16T16:58:59.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi043 2024-04-16T16:58:59.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi043 to 5694M 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:58:59 smithi043 ceph-mon[36901]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T16:58:59.537 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi043 2024-04-16T16:58:59.537 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.537 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.537 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.537 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.537 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.537 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi043 to 5694M 2024-04-16T16:58:59.537 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:59.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:59.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:58:59 smithi018 ceph-mon[31776]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T16:58:59.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi043 2024-04-16T16:58:59.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:59.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi043 to 5694M 2024-04-16T16:58:59.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:59.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:59.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:58:59.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:58:59 smithi022 ceph-mon[36023]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T16:58:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:59:00.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:00 smithi043 ceph-mon[36901]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:00.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:00 smithi043 ceph-mon[36901]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T16:59:00.605 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:00 smithi018 ceph-mon[31776]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:00.605 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:00 smithi018 ceph-mon[31776]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T16:59:00.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:00 smithi022 ceph-mon[36023]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:00.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:00 smithi022 ceph-mon[36023]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T16:59:01.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:01 smithi018 ceph-mon[31776]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T16:59:01.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:01 smithi018 ceph-mon[31776]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:01.947 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:01 smithi022 ceph-mon[36023]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T16:59:01.947 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:01 smithi022 ceph-mon[36023]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:02.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:01 smithi043 ceph-mon[36901]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T16:59:02.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:01 smithi043 ceph-mon[36901]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.175+0000 7fd18de00700 1 -- 172.21.15.18:0/3805760911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd18810a280 msgr2=0x7fd18810a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.175+0000 7fd18de00700 1 --2- 172.21.15.18:0/3805760911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd18810a280 0x7fd18810a6e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd1740099c0 tx=0x7fd174009cd0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.176+0000 7fd18de00700 1 -- 172.21.15.18:0/3805760911 shutdown_connections 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.176+0000 7fd18de00700 1 --2- 172.21.15.18:0/3805760911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd18810ac20 0x7fd1881114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.176+0000 7fd18de00700 1 --2- 172.21.15.18:0/3805760911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd188109080 0x7fd1881094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.176+0000 7fd18de00700 1 --2- 172.21.15.18:0/3805760911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd18810a280 0x7fd18810a6e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.176+0000 7fd18de00700 1 -- 172.21.15.18:0/3805760911 >> 172.21.15.18:0/3805760911 conn(0x7fd188078050 msgr2=0x7fd18807a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.176+0000 7fd18de00700 1 -- 172.21.15.18:0/3805760911 shutdown_connections 2024-04-16T16:59:02.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.176+0000 7fd18de00700 1 -- 172.21.15.18:0/3805760911 wait complete. 2024-04-16T16:59:02.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18de00700 1 Processor -- start 2024-04-16T16:59:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18de00700 1 -- start start 2024-04-16T16:59:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd188109080 0x7fd18819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd18810a280 0x7fd18819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd18810ac20 0x7fd1881a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18de00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd188114c90 con 0x7fd18810ac20 2024-04-16T16:59:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18de00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd188114b10 con 0x7fd188109080 2024-04-16T16:59:02.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18de00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd188114e10 con 0x7fd18810a280 2024-04-16T16:59:02.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.178+0000 7fd18d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd18810ac20 0x7fd1881a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:02.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd18ca00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd188109080 0x7fd18819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:02.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd187e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd18810a280 0x7fd18819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:02.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd18d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd18810ac20 0x7fd1881a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51010/0 (socket says 172.21.15.18:51010) 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd18ca00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd188109080 0x7fd18819f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:43528/0 (socket says 172.21.15.18:43528) 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd18d400700 1 -- 172.21.15.18:0/790271538 learned_addr learned my addr 172.21.15.18:0/790271538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd187e00700 1 -- 172.21.15.18:0/790271538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd188109080 msgr2=0x7fd18819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd187e00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd188109080 0x7fd18819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd187e00700 1 -- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd18810ac20 msgr2=0x7fd1881a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd187e00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd18810ac20 0x7fd1881a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd187e00700 1 -- 172.21.15.18:0/790271538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1881a6ed0 con 0x7fd18810a280 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.179+0000 7fd187e00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd18810a280 0x7fd18819f7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd1740099c0 tx=0x7fd17400bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.180+0000 7fd185600700 1 -- 172.21.15.18:0/790271538 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174009e60 con 0x7fd18810a280 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.180+0000 7fd185600700 1 -- 172.21.15.18:0/790271538 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd174003e30 con 0x7fd18810a280 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.180+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1881a7160 con 0x7fd18810a280 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.180+0000 7fd185600700 1 -- 172.21.15.18:0/790271538 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740173f0 con 0x7fd18810a280 2024-04-16T16:59:02.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.180+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1881a7610 con 0x7fd18810a280 2024-04-16T16:59:02.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.181+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1880697e0 con 0x7fd18810a280 2024-04-16T16:59:02.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.183+0000 7fd185600700 1 -- 172.21.15.18:0/790271538 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd17400f580 con 0x7fd18810a280 2024-04-16T16:59:02.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.185+0000 7fd185600700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1700a3cc0 0x7fd1700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:02.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.185+0000 7fd185600700 1 -- 172.21.15.18:0/790271538 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fd1740d08c0 con 0x7fd18810a280 2024-04-16T16:59:02.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.185+0000 7fd18ca00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1700a3cc0 0x7fd1700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:02.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.186+0000 7fd18ca00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1700a3cc0 0x7fd1700a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd18810a0e0 tx=0x7fd17c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:59:02.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.187+0000 7fd185600700 1 -- 172.21.15.18:0/790271538 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd174013070 con 0x7fd18810a280 2024-04-16T16:59:02.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:02.430+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd188062f30 con 0x7fd1700a3cc0 2024-04-16T16:59:03.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:02 smithi043 ceph-mon[36901]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:03.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:03.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:02 smithi022 ceph-mon[36023]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:03.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:03.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:03.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:03.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:02 smithi018 ceph-mon[31776]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:03.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:03.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:03.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:04.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:03 smithi043 ceph-mon[36901]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-16T16:59:04.170 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:03 smithi018 ceph-mon[31776]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-16T16:59:04.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:03 smithi022 ceph-mon[36023]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-16T16:59:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:05 smithi022 ceph-mon[36023]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-16T16:59:06.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:05 smithi018 ceph-mon[31776]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-16T16:59:06.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:05 smithi043 ceph-mon[36901]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-16T16:59:07.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:06 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2602634173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cf9ee08-5ede-446d-9b67-cebda4d2be28"}]: dispatch 2024-04-16T16:59:07.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:06 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2602634173' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cf9ee08-5ede-446d-9b67-cebda4d2be28"}]': finished 2024-04-16T16:59:07.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:06 smithi022 ceph-mon[36023]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T16:59:07.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:06 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:07.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:06 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2602634173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cf9ee08-5ede-446d-9b67-cebda4d2be28"}]: dispatch 2024-04-16T16:59:07.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:06 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2602634173' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cf9ee08-5ede-446d-9b67-cebda4d2be28"}]': finished 2024-04-16T16:59:07.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:06 smithi018 ceph-mon[31776]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T16:59:07.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:06 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:07.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:06 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2602634173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cf9ee08-5ede-446d-9b67-cebda4d2be28"}]: dispatch 2024-04-16T16:59:07.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:06 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2602634173' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cf9ee08-5ede-446d-9b67-cebda4d2be28"}]': finished 2024-04-16T16:59:07.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:06 smithi043 ceph-mon[36901]: osdmap e64: 10 total, 9 up, 10 in 2024-04-16T16:59:07.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:06 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:08.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:07 smithi022 ceph-mon[36023]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-16T16:59:08.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:07 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2665513571' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:08.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:07 smithi018 ceph-mon[31776]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-16T16:59:08.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:07 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2665513571' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:08.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:07 smithi043 ceph-mon[36901]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-16T16:59:08.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:07 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2665513571' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:10.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:09 smithi022 ceph-mon[36023]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-16T16:59:10.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:09 smithi018 ceph-mon[31776]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-16T16:59:10.247 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:09 smithi043 ceph-mon[36901]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-16T16:59:12.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:11 smithi022 ceph-mon[36023]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T16:59:12.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:11 smithi018 ceph-mon[31776]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T16:59:12.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:11 smithi043 ceph-mon[36901]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T16:59:14.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:13 smithi018 ceph-mon[31776]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:14.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:13 smithi022 ceph-mon[36023]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:14.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:13 smithi043 ceph-mon[36901]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:16.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:15 smithi022 ceph-mon[36023]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T16:59:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:15 smithi018 ceph-mon[31776]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T16:59:16.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:16.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:15 smithi043 ceph-mon[36901]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:16.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T16:59:16.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:17.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:16 smithi018 ceph-mon[31776]: Deploying daemon osd.9 on smithi018 2024-04-16T16:59:17.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:16 smithi022 ceph-mon[36023]: Deploying daemon osd.9 on smithi018 2024-04-16T16:59:17.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:16 smithi043 ceph-mon[36901]: Deploying daemon osd.9 on smithi018 2024-04-16T16:59:18.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:17 smithi022 ceph-mon[36023]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:18.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:17 smithi018 ceph-mon[31776]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:18.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:17 smithi043 ceph-mon[36901]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:20.157 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:19 smithi018 ceph-mon[31776]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:20.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:19 smithi043 ceph-mon[36901]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:20.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:19 smithi022 ceph-mon[36023]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:22.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:21 smithi018 ceph-mon[31776]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:21 smithi043 ceph-mon[36901]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:22.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:21 smithi022 ceph-mon[36023]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:24.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:23 smithi018 ceph-mon[31776]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:23 smithi043 ceph-mon[36901]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:24.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:23 smithi022 ceph-mon[36023]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:25.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:25.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:25.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:25.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:25.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:25.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:25.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:26.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:26 smithi018 ceph-mon[31776]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:26 smithi043 ceph-mon[36901]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:26 smithi022 ceph-mon[36023]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:28.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:28 smithi018 ceph-mon[31776]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:28.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:28.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:28.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:28 smithi018 ceph-mon[31776]: from='osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T16:59:28.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:28 smithi018 ceph-mon[31776]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T16:59:28.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:28 smithi043 ceph-mon[36901]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:28.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:28.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:28.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:28 smithi043 ceph-mon[36901]: from='osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T16:59:28.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:28 smithi043 ceph-mon[36901]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T16:59:28.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:28 smithi022 ceph-mon[36023]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:28.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:28.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:28.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:28 smithi022 ceph-mon[36023]: from='osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T16:59:28.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:28 smithi022 ceph-mon[36023]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T16:59:28.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.987+0000 7fd185600700 1 -- 172.21.15.18:0/790271538 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd188062f30 con 0x7fd1700a3cc0 2024-04-16T16:59:28.989 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 9 on host 'smithi018' 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1700a3cc0 msgr2=0x7fd1700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1700a3cc0 0x7fd1700a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd18810a0e0 tx=0x7fd17c009260 comp rx=0 tx=0).stop 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd18810a280 msgr2=0x7fd18819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd18810a280 0x7fd18819f7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd1740099c0 tx=0x7fd17400bee0 comp rx=0 tx=0).stop 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 shutdown_connections 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd1700a3cc0 0x7fd1700a6180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd188109080 0x7fd18819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd18810ac20 0x7fd1881a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 --2- 172.21.15.18:0/790271538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd18810a280 0x7fd18819f7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:28.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 >> 172.21.15.18:0/790271538 conn(0x7fd188078050 msgr2=0x7fd18810d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:59:28.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.991+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 shutdown_connections 2024-04-16T16:59:28.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T16:59:28.992+0000 7fd18de00700 1 -- 172.21.15.18:0/790271538 wait complete. 2024-04-16T16:59:29.789 DEBUG:teuthology.orchestra.run.smithi018:osd.9> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.9.service 2024-04-16T16:59:29.796 INFO:tasks.cephadm:Deploying osd.10 on smithi022 with /dev/vg_nvme/lv_1... 2024-04-16T16:59:29.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T16:59:30.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:29 smithi018 ceph-mon[31776]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:30.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:29 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:30.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:29 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:30.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:29 smithi018 ceph-mon[31776]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T16:59:30.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:29 smithi018 ceph-mon[31776]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T16:59:30.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:29 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:30.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:29 smithi018 ceph-mon[31776]: from='osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:59:30.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:29 smithi018 ceph-mon[31776]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:59:30.056 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:59:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:29 smithi022 ceph-mon[36023]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:29 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:29 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:29 smithi022 ceph-mon[36023]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T16:59:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:29 smithi022 ceph-mon[36023]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T16:59:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:29 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:29 smithi022 ceph-mon[36023]: from='osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:59:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:29 smithi022 ceph-mon[36023]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:59:30.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:29 smithi043 ceph-mon[36901]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:30.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:29 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:30.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:29 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:30.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:29 smithi043 ceph-mon[36901]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T16:59:30.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:29 smithi043 ceph-mon[36901]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T16:59:30.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:29 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:30.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:29 smithi043 ceph-mon[36901]: from='osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:59:30.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:29 smithi043 ceph-mon[36901]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-16T16:59:30.446 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 16:59:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T16:59:30.018+0000 7f0d24800700 -1 osd.9 0 waiting for initial osdmap 2024-04-16T16:59:30.447 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 16:59:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T16:59:30.047+0000 7f0d1f600700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:59:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:31 smithi022 ceph-mon[36023]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:59:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:31 smithi022 ceph-mon[36023]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T16:59:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:31 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:31.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:31 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:31 smithi018 ceph-mon[31776]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:59:31.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:31 smithi018 ceph-mon[31776]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T16:59:31.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:31 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:31.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:31 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:31 smithi043 ceph-mon[36901]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-16T16:59:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:31 smithi043 ceph-mon[36901]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T16:59:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:31 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:31.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:31 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:32.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:32 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:59:32.109 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:32 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:59:32.109 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:32 smithi018 ceph-mon[31776]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:32.109 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:32 smithi018 ceph-mon[31776]: osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] boot 2024-04-16T16:59:32.109 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:32 smithi018 ceph-mon[31776]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T16:59:32.109 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:32.109 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:32 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:32 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:59:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:32 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:59:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:32 smithi022 ceph-mon[36023]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:32 smithi022 ceph-mon[36023]: osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] boot 2024-04-16T16:59:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:32 smithi022 ceph-mon[36023]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T16:59:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:32.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:32 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:32.186 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T16:59:32.186 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T16:59:32.186 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-04-16T16:59:32.186 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-04-16T16:59:32.186 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00828545 s, 1.3 GB/s 2024-04-16T16:59:32.186 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-04-16T16:59:32.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:32 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:59:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:32 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:59:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:32 smithi043 ceph-mon[36901]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-16T16:59:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:32 smithi043 ceph-mon[36901]: osd.9 [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] boot 2024-04-16T16:59:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:32 smithi043 ceph-mon[36901]: osdmap e67: 10 total, 10 up, 10 in 2024-04-16T16:59:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:32 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T16:59:32.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_1 2024-04-16T16:59:33.124 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:59:33.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:33 smithi022 ceph-mon[36023]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T16:59:33.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:33 smithi018 ceph-mon[31776]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T16:59:33.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:33 smithi043 ceph-mon[36901]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T16:59:34.326 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:34 smithi022 ceph-mon[36023]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:34.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:34 smithi018 ceph-mon[31776]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:34.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:34 smithi043 ceph-mon[36901]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 -- 172.21.15.22:0/4255600292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830102c50 msgr2=0x7f08301030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 --2- 172.21.15.22:0/4255600292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830102c50 0x7f08301030b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08180099c0 tx=0x7f0818009cd0 comp rx=0 tx=0).stop 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 -- 172.21.15.22:0/4255600292 shutdown_connections 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 --2- 172.21.15.22:0/4255600292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08301035f0 0x7f0830109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 --2- 172.21.15.22:0/4255600292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0830101a50 0x7f0830101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 --2- 172.21.15.22:0/4255600292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830102c50 0x7f08301030b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 -- 172.21.15.22:0/4255600292 >> 172.21.15.22:0/4255600292 conn(0x7f08300fcff0 msgr2=0x7f08300ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 -- 172.21.15.22:0/4255600292 shutdown_connections 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 -- 172.21.15.22:0/4255600292 wait complete. 2024-04-16T16:59:34.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.864+0000 7f0836318700 1 Processor -- start 2024-04-16T16:59:34.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f0836318700 1 -- start start 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f0836318700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830101a50 0x7f0830198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f0836318700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0830102c50 0x7f08301993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f0836318700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08301035f0 0x7f08301a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f0836318700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f083010d6b0 con 0x7f0830102c50 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f0836318700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f083010d530 con 0x7f0830101a50 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f0836318700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f083010d830 con 0x7f08301035f0 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f082ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830101a50 0x7f0830198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f082ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830101a50 0x7f0830198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32788/0 (socket says 172.21.15.22:32788) 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f082ffff700 1 -- 172.21.15.22:0/1605278813 learned_addr learned my addr 172.21.15.22:0/1605278813 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f08348b5700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08301035f0 0x7f08301a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:34.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f082ffff700 1 -- 172.21.15.22:0/1605278813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08301035f0 msgr2=0x7f08301a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f082ffff700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08301035f0 0x7f08301a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f082ffff700 1 -- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0830102c50 msgr2=0x7f08301993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f082ffff700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0830102c50 0x7f08301993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f082ffff700 1 -- 172.21.15.22:0/1605278813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08301a0a70 con 0x7f0830101a50 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.865+0000 7f08348b5700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08301035f0 0x7f08301a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.866+0000 7f082ffff700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830101a50 0x7f0830198e60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f082000d980 tx=0x7f082000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.866+0000 7f082d7fa700 1 -- 172.21.15.22:0/1605278813 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820017070 con 0x7f0830101a50 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.866+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08301a0d60 con 0x7f0830101a50 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.866+0000 7f082d7fa700 1 -- 172.21.15.22:0/1605278813 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0820009b40 con 0x7f0830101a50 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.866+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08301a1260 con 0x7f0830101a50 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.866+0000 7f082d7fa700 1 -- 172.21.15.22:0/1605278813 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082000f620 con 0x7f0830101a50 2024-04-16T16:59:34.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.867+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08300697e0 con 0x7f0830101a50 2024-04-16T16:59:34.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.868+0000 7f082d7fa700 1 -- 172.21.15.22:0/1605278813 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0820010460 con 0x7f0830101a50 2024-04-16T16:59:34.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.868+0000 7f082d7fa700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f081c0a3c40 0x7f081c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:34.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.868+0000 7f082d7fa700 1 -- 172.21.15.22:0/1605278813 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f08200cf890 con 0x7f0830101a50 2024-04-16T16:59:34.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.869+0000 7f082f7fe700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f081c0a3c40 0x7f081c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:34.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.870+0000 7f082f7fe700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f081c0a3c40 0x7f081c0a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0818009fd0 tx=0x7f0818000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:59:34.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:34.872+0000 7f082d7fa700 1 -- 172.21.15.22:0/1605278813 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f082009ca70 con 0x7f0830101a50 2024-04-16T16:59:35.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:35.090+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f08300ff1e0 con 0x7f081c0a3c40 2024-04-16T16:59:36.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:36.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:36.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:36.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:36.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi018 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi018 to 4273M 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:36.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.428 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:36.428 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:36 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:36.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi018 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi018 to 4273M 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:36.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:36.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:36.449 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:36 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:36.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi018 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi018 to 4273M 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:36.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:36.504 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.504 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:36.504 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:36 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:38.394 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:38 smithi018 ceph-mon[31776]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:38.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:38 smithi022 ceph-mon[36023]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:38.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:38 smithi043 ceph-mon[36901]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:39.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:39 smithi022 ceph-mon[36023]: from='client.? 172.21.15.22:0/2605778792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]: dispatch 2024-04-16T16:59:39.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:39 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]: dispatch 2024-04-16T16:59:39.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:39 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]': finished 2024-04-16T16:59:39.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:39 smithi022 ceph-mon[36023]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T16:59:39.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:39 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:39.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:39 smithi022 ceph-mon[36023]: from='client.? 172.21.15.22:0/2202377490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:39.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:39 smithi018 ceph-mon[31776]: from='client.? 172.21.15.22:0/2605778792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]: dispatch 2024-04-16T16:59:39.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:39 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]: dispatch 2024-04-16T16:59:39.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:39 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]': finished 2024-04-16T16:59:39.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:39 smithi018 ceph-mon[31776]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T16:59:39.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:39 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:39.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:39 smithi018 ceph-mon[31776]: from='client.? 172.21.15.22:0/2202377490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:39.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:39 smithi043 ceph-mon[36901]: from='client.? 172.21.15.22:0/2605778792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]: dispatch 2024-04-16T16:59:39.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:39 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]: dispatch 2024-04-16T16:59:39.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:39 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a70d7bc-095f-4be6-9096-7bf5d46cd26d"}]': finished 2024-04-16T16:59:39.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:39 smithi043 ceph-mon[36901]: osdmap e69: 11 total, 10 up, 11 in 2024-04-16T16:59:39.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:39 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:39.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:39 smithi043 ceph-mon[36901]: from='client.? 172.21.15.22:0/2202377490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:40.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:40 smithi022 ceph-mon[36023]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:40 smithi018 ceph-mon[31776]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:40.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:40 smithi043 ceph-mon[36901]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:42.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:42 smithi022 ceph-mon[36023]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:42.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:42 smithi018 ceph-mon[31776]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:42.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:42 smithi043 ceph-mon[36901]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:44.395 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:44 smithi022 ceph-mon[36023]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:44.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:44 smithi018 ceph-mon[31776]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:44.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:44 smithi043 ceph-mon[36901]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:46.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:46 smithi022 ceph-mon[36023]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:46.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:46 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T16:59:46.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:46 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:46.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:46 smithi018 ceph-mon[31776]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:46.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:46 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T16:59:46.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:46 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:46 smithi043 ceph-mon[36901]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:46.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:46 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T16:59:46.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:46 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:47.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:47 smithi022 ceph-mon[36023]: Deploying daemon osd.10 on smithi022 2024-04-16T16:59:47.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:47 smithi018 ceph-mon[31776]: Deploying daemon osd.10 on smithi022 2024-04-16T16:59:47.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:47 smithi043 ceph-mon[36901]: Deploying daemon osd.10 on smithi022 2024-04-16T16:59:48.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:48 smithi022 ceph-mon[36023]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:48.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:48 smithi018 ceph-mon[31776]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:48.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:48 smithi043 ceph-mon[36901]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:50.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:50 smithi022 ceph-mon[36023]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:50.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:50 smithi018 ceph-mon[31776]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:50.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:50 smithi043 ceph-mon[36901]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:52.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:52 smithi022 ceph-mon[36023]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:52.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:52.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:52.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:52 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:52.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:52 smithi018 ceph-mon[31776]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:52.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:52.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:52.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:52 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:52 smithi043 ceph-mon[36901]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:52 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:54.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:54 smithi022 ceph-mon[36023]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:54 smithi018 ceph-mon[31776]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:54 smithi043 ceph-mon[36901]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:55.877 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:55.877 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:55 smithi022 ceph-mon[36023]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:55.877 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:55.877 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:55.877 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:55.877 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:55.877 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:55 smithi022 ceph-mon[36023]: from='osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T16:59:55.877 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:55 smithi022 ceph-mon[36023]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T16:59:55.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.984+0000 7f082d7fa700 1 -- 172.21.15.22:0/1605278813 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f08300ff1e0 con 0x7f081c0a3c40 2024-04-16T16:59:55.986 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 10 on host 'smithi022' 2024-04-16T16:59:55.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f081c0a3c40 msgr2=0x7f081c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f081c0a3c40 0x7f081c0a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0818009fd0 tx=0x7f0818000bc0 comp rx=0 tx=0).stop 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830101a50 msgr2=0x7f0830198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830101a50 0x7f0830198e60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f082000d980 tx=0x7f082000dd40 comp rx=0 tx=0).stop 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 shutdown_connections 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f081c0a3c40 0x7f081c0a6100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0830101a50 0x7f0830198e60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08301035f0 0x7f08301a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 --2- 172.21.15.22:0/1605278813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0830102c50 0x7f08301993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 >> 172.21.15.22:0/1605278813 conn(0x7f08300fcff0 msgr2=0x7f08300fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 shutdown_connections 2024-04-16T16:59:55.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-16T16:59:55.987+0000 7f0836318700 1 -- 172.21.15.22:0/1605278813 wait complete. 2024-04-16T16:59:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:55 smithi018 ceph-mon[31776]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:56.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:56.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:56.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:55 smithi018 ceph-mon[31776]: from='osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T16:59:56.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:55 smithi018 ceph-mon[31776]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T16:59:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:55 smithi043 ceph-mon[36901]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:55 smithi043 ceph-mon[36901]: from='osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T16:59:56.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:55 smithi043 ceph-mon[36901]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T16:59:56.583 DEBUG:teuthology.orchestra.run.smithi022:osd.10> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.10.service 2024-04-16T16:59:56.585 INFO:tasks.cephadm:Deploying osd.11 on smithi043 with /dev/vg_nvme/lv_1... 2024-04-16T16:59:56.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T16:59:56.847 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:59:57.060 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:56 smithi022 ceph-mon[36023]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T16:59:57.060 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:56 smithi022 ceph-mon[36023]: from='osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:59:57.060 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:56 smithi022 ceph-mon[36023]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T16:59:57.060 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:56 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:57.060 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:56 smithi022 ceph-mon[36023]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:59:57.060 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:56 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:57.060 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:56 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:57.060 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:56 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:57.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:56 smithi018 ceph-mon[31776]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T16:59:57.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:56 smithi018 ceph-mon[31776]: from='osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:59:57.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:56 smithi018 ceph-mon[31776]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T16:59:57.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:56 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:57.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:56 smithi018 ceph-mon[31776]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:59:57.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:56 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:57.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:56 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:57.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:56 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:57.214 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:56 smithi043 ceph-mon[36901]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T16:59:57.214 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:56 smithi043 ceph-mon[36901]: from='osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:59:57.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:56 smithi043 ceph-mon[36901]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T16:59:57.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:56 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:57.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:56 smithi043 ceph-mon[36901]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-16T16:59:57.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:56 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:57.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:56 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T16:59:57.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:56 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:57.675 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T16:59:57.340+0000 7f3ad6707700 -1 osd.10 0 waiting for initial osdmap 2024-04-16T16:59:57.675 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T16:59:57.353+0000 7f3ace4f2700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:59:58.153 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:58.153 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:59:58.154 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T16:59:58.154 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:58.154 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:58.154 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: from='osd.10 ' entity='osd.10' 2024-04-16T16:59:58.154 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] boot 2024-04-16T16:59:58.154 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T16:59:58.154 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:57 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: from='osd.10 ' entity='osd.10' 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] boot 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T16:59:58.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:57 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: from='osd.10 ' entity='osd.10' 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: osd.10 [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] boot 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: osdmap e72: 11 total, 11 up, 11 in 2024-04-16T16:59:58.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:57 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T16:59:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T16:59:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T16:59:59.064 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-16T16:59:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-16T16:59:59.064 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00840653 s, 1.2 GB/s 2024-04-16T16:59:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-16T16:59:59.375 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:59 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T16:59:59.375 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:59 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T16:59:59.375 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 16:59:59 smithi022 ceph-mon[36023]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T16:59:59.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:59 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T16:59:59.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:59 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T16:59:59.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 16:59:59 smithi043 ceph-mon[36901]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T16:59:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:59 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T16:59:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:59 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T16:59:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 16:59:59 smithi018 ceph-mon[31776]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T16:59:59.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-04-16T17:00:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:00:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T17:00:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:00 smithi022 ceph-mon[36023]: overall HEALTH_OK 2024-04-16T17:00:00.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:00.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T17:00:00.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:00.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:00.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:00 smithi043 ceph-mon[36901]: overall HEALTH_OK 2024-04-16T17:00:00.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:00.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:00 smithi018 ceph-mon[31776]: overall HEALTH_OK 2024-04-16T17:00:01.418 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:01 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi022 2024-04-16T17:00:01.418 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:01 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi022 to 4271M 2024-04-16T17:00:01.418 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:01 smithi022 ceph-mon[36023]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T17:00:01.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:01 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi022 2024-04-16T17:00:01.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:01 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi022 to 4271M 2024-04-16T17:00:01.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:01 smithi018 ceph-mon[31776]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T17:00:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:01 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi022 2024-04-16T17:00:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:01 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi022 to 4271M 2024-04-16T17:00:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:01 smithi043 ceph-mon[36901]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T17:00:01.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 -- 172.21.15.43:0/2248861900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96641035f0 msgr2=0x7f9664109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 --2- 172.21.15.43:0/2248861900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96641035f0 0x7f9664109ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9654009990 tx=0x7f9654009ca0 comp rx=0 tx=0).stop 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 -- 172.21.15.43:0/2248861900 shutdown_connections 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 --2- 172.21.15.43:0/2248861900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664101a50 0x7f9664101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 --2- 172.21.15.43:0/2248861900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96641035f0 0x7f9664109ea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 --2- 172.21.15.43:0/2248861900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9664102c50 0x7f96641030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 -- 172.21.15.43:0/2248861900 >> 172.21.15.43:0/2248861900 conn(0x7f96640fcff0 msgr2=0x7f96640ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 -- 172.21.15.43:0/2248861900 shutdown_connections 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 -- 172.21.15.43:0/2248861900 wait complete. 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.730+0000 7f966af10700 1 Processor -- start 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f966af10700 1 -- start start 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f966af10700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664101a50 0x7f966419afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f966af10700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9664102c50 0x7f966419b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f966af10700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96641035f0 0x7f96641a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f966af10700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f966410d6b0 con 0x7f96641035f0 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f966af10700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f966410d530 con 0x7f9664101a50 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f966af10700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f966410d830 con 0x7f9664102c50 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9663fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9664102c50 0x7f966419b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9663fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9664102c50 0x7f966419b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44600/0 (socket says 172.21.15.43:44600) 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9668cac700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664101a50 0x7f966419afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9663fff700 1 -- 172.21.15.43:0/2847006333 learned_addr learned my addr 172.21.15.43:0/2847006333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9663fff700 1 -- 172.21.15.43:0/2847006333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664101a50 msgr2=0x7f966419afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9663fff700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664101a50 0x7f966419afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9663fff700 1 -- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96641035f0 msgr2=0x7f96641a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9663fff700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96641035f0 0x7f96641a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9663fff700 1 -- 172.21.15.43:0/2847006333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96641a2c20 con 0x7f9664102c50 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.731+0000 7f9668cac700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664101a50 0x7f966419afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.732+0000 7f9663fff700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9664102c50 0x7f966419b500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f965800b560 tx=0x7f965800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.732+0000 7f9661ffb700 1 -- 172.21.15.43:0/2847006333 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658015020 con 0x7f9664102c50 2024-04-16T17:00:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.732+0000 7f9661ffb700 1 -- 172.21.15.43:0/2847006333 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9658011d10 con 0x7f9664102c50 2024-04-16T17:00:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.732+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96641a2f10 con 0x7f9664102c50 2024-04-16T17:00:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.732+0000 7f9661ffb700 1 -- 172.21.15.43:0/2847006333 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658010e50 con 0x7f9664102c50 2024-04-16T17:00:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.732+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96641a3440 con 0x7f9664102c50 2024-04-16T17:00:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.733+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9664069810 con 0x7f9664102c50 2024-04-16T17:00:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.734+0000 7f9661ffb700 1 -- 172.21.15.43:0/2847006333 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9658018020 con 0x7f9664102c50 2024-04-16T17:00:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.734+0000 7f9661ffb700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f964c0a3de0 0x7f964c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.735+0000 7f9661ffb700 1 -- 172.21.15.43:0/2847006333 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f96580cf5b0 con 0x7f9664102c50 2024-04-16T17:00:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.735+0000 7f9668cac700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f964c0a3de0 0x7f964c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.736+0000 7f9668cac700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f964c0a3de0 0x7f964c0a62a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9664102ab0 tx=0x7f965c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.739+0000 7f9661ffb700 1 -- 172.21.15.43:0/2847006333 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9664069810 con 0x7f9664102c50 2024-04-16T17:00:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:01.957+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f96640ff190 con 0x7f964c0a3de0 2024-04-16T17:00:02.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:02 smithi043 ceph-mon[36901]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:02.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:02 smithi043 ceph-mon[36901]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T17:00:02.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:02.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:02.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:02.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:02 smithi022 ceph-mon[36023]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:02.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:02 smithi022 ceph-mon[36023]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T17:00:02.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:02.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:02.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:02 smithi018 ceph-mon[31776]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:02 smithi018 ceph-mon[31776]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T17:00:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:03.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:03 smithi022 ceph-mon[36023]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:03.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:03 smithi018 ceph-mon[31776]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:03.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:03 smithi043 ceph-mon[36901]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:04.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:04 smithi043 ceph-mon[36901]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-16T17:00:04.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:04 smithi022 ceph-mon[36023]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-16T17:00:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:04 smithi018 ceph-mon[31776]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-16T17:00:05.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:05 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/1492355387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]: dispatch 2024-04-16T17:00:05.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:05 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]: dispatch 2024-04-16T17:00:05.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:05 smithi022 ceph-mon[36023]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]': finished 2024-04-16T17:00:05.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:05 smithi022 ceph-mon[36023]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T17:00:05.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:05 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:05 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/1492355387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]: dispatch 2024-04-16T17:00:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:05 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]: dispatch 2024-04-16T17:00:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:05 smithi018 ceph-mon[31776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]': finished 2024-04-16T17:00:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:05 smithi018 ceph-mon[31776]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T17:00:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:05 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/1492355387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]: dispatch 2024-04-16T17:00:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:05 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]: dispatch 2024-04-16T17:00:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:05 smithi043 ceph-mon[36901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05"}]': finished 2024-04-16T17:00:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:05 smithi043 ceph-mon[36901]: osdmap e77: 12 total, 11 up, 12 in 2024-04-16T17:00:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:05 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:06 smithi018 ceph-mon[31776]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T17:00:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:06 smithi018 ceph-mon[31776]: from='client.? 172.21.15.43:0/920615186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:06 smithi043 ceph-mon[36901]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T17:00:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:06 smithi043 ceph-mon[36901]: from='client.? 172.21.15.43:0/920615186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:06.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:06 smithi022 ceph-mon[36023]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T17:00:06.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:06 smithi022 ceph-mon[36023]: from='client.? 172.21.15.43:0/920615186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:08 smithi018 ceph-mon[31776]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-16T17:00:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:08 smithi043 ceph-mon[36901]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-16T17:00:08.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:08 smithi022 ceph-mon[36023]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-16T17:00:10.422 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:10 smithi043 ceph-mon[36901]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T17:00:10.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:10 smithi018 ceph-mon[31776]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T17:00:10.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:10 smithi022 ceph-mon[36023]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T17:00:12.443 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:12 smithi043 ceph-mon[36901]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:12.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:12 smithi018 ceph-mon[31776]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:12.465 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:12 smithi022 ceph-mon[36023]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:13.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:13 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T17:00:13.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:13 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:13.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:13 smithi043 ceph-mon[36901]: Deploying daemon osd.11 on smithi043 2024-04-16T17:00:13.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:13 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T17:00:13.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:13 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:13.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:13 smithi022 ceph-mon[36023]: Deploying daemon osd.11 on smithi043 2024-04-16T17:00:13.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:13 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T17:00:13.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:13 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:13.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:13 smithi018 ceph-mon[31776]: Deploying daemon osd.11 on smithi043 2024-04-16T17:00:14.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:14 smithi043 ceph-mon[36901]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:14.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:14 smithi022 ceph-mon[36023]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:14.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:14 smithi018 ceph-mon[31776]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:16.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:16 smithi043 ceph-mon[36901]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:16.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:16 smithi022 ceph-mon[36023]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:16.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:16 smithi018 ceph-mon[31776]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:18.498 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:18 smithi043 ceph-mon[36901]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:18.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:18 smithi022 ceph-mon[36023]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:18 smithi018 ceph-mon[31776]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:19.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:19.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:19.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:20.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:20.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:20.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:20.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:20 smithi043 ceph-mon[36901]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:20.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:20 smithi022 ceph-mon[36023]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:20.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:20 smithi018 ceph-mon[31776]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:22.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:22 smithi043 ceph-mon[36901]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:22.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:22.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:22.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:22 smithi043 ceph-mon[36901]: from='osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:00:22.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:22 smithi043 ceph-mon[36901]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:00:22.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.748+0000 7f9661ffb700 1 -- 172.21.15.43:0/2847006333 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f96640ff190 con 0x7f964c0a3de0 2024-04-16T17:00:22.751 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 11 on host 'smithi043' 2024-04-16T17:00:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.750+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f964c0a3de0 msgr2=0x7f964c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.750+0000 7f966af10700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f964c0a3de0 0x7f964c0a62a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9664102ab0 tx=0x7f965c00b040 comp rx=0 tx=0).stop 2024-04-16T17:00:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9664102c50 msgr2=0x7f966419b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9664102c50 0x7f966419b500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f965800b560 tx=0x7f965800b870 comp rx=0 tx=0).stop 2024-04-16T17:00:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 shutdown_connections 2024-04-16T17:00:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f964c0a3de0 0x7f964c0a62a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664101a50 0x7f966419afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96641035f0 0x7f96641a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 --2- 172.21.15.43:0/2847006333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9664102c50 0x7f966419b500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 >> 172.21.15.43:0/2847006333 conn(0x7f96640fcff0 msgr2=0x7f96640fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 shutdown_connections 2024-04-16T17:00:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:22.751+0000 7f966af10700 1 -- 172.21.15.43:0/2847006333 wait complete. 2024-04-16T17:00:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:22 smithi022 ceph-mon[36023]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:22 smithi022 ceph-mon[36023]: from='osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:00:22.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:22 smithi022 ceph-mon[36023]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:00:22.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:22 smithi018 ceph-mon[31776]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:22.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:22.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:22.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:22.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:22 smithi018 ceph-mon[31776]: from='osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:00:22.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:22 smithi018 ceph-mon[31776]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:00:23.494 DEBUG:teuthology.orchestra.run.smithi043:osd.11> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.11.service 2024-04-16T17:00:23.496 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-16T17:00:23.497 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd stat -f json 2024-04-16T17:00:23.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:23.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:23.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:23.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:23.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T17:00:23.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: from='osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T17:00:23.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T17:00:23.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:23.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-mon[36901]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T17:00:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T17:00:23.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: from='osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T17:00:23.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T17:00:23.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:23.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:23 smithi018 ceph-mon[31776]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T17:00:23.978 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:00:24.003 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:00:23.885+0000 7fa4de0dc700 -1 osd.11 0 waiting for initial osdmap 2024-04-16T17:00:24.003 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:00:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:00:23.899+0000 7fa4d9ecf700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:00:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T17:00:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: from='osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T17:00:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T17:00:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:24.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:23 smithi022 ceph-mon[36023]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-16T17:00:25.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:24 smithi043 ceph-mon[36901]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T17:00:25.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:24 smithi043 ceph-mon[36901]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T17:00:25.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:25.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:25.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:24 smithi018 ceph-mon[31776]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T17:00:25.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:24 smithi018 ceph-mon[31776]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T17:00:25.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:25.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:25.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:24 smithi022 ceph-mon[36023]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-16T17:00:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:24 smithi022 ceph-mon[36023]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T17:00:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:24 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:24 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:26.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:25 smithi018 ceph-mon[31776]: purged_snaps scrub starts 2024-04-16T17:00:26.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:25 smithi018 ceph-mon[31776]: purged_snaps scrub ok 2024-04-16T17:00:26.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:25 smithi018 ceph-mon[31776]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:26.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:25 smithi018 ceph-mon[31776]: osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] boot 2024-04-16T17:00:26.175 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:25 smithi018 ceph-mon[31776]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T17:00:26.175 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:25 smithi022 ceph-mon[36023]: purged_snaps scrub starts 2024-04-16T17:00:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:25 smithi022 ceph-mon[36023]: purged_snaps scrub ok 2024-04-16T17:00:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:25 smithi022 ceph-mon[36023]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:25 smithi022 ceph-mon[36023]: osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] boot 2024-04-16T17:00:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:25 smithi022 ceph-mon[36023]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T17:00:26.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:26.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:25 smithi043 ceph-mon[36901]: purged_snaps scrub starts 2024-04-16T17:00:26.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:25 smithi043 ceph-mon[36901]: purged_snaps scrub ok 2024-04-16T17:00:26.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:25 smithi043 ceph-mon[36901]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:00:26.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:25 smithi043 ceph-mon[36901]: osd.11 [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] boot 2024-04-16T17:00:26.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:25 smithi043 ceph-mon[36901]: osdmap e80: 12 total, 12 up, 12 in 2024-04-16T17:00:26.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:00:26.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 -- 172.21.15.18:0/3976571659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c102b80 msgr2=0x7fdd1c102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 --2- 172.21.15.18:0/3976571659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c102b80 0x7fdd1c102fa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00b0d0 tx=0x7fdd0c00b3e0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 -- 172.21.15.18:0/3976571659 shutdown_connections 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 --2- 172.21.15.18:0/3976571659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c102b80 0x7fdd1c102fa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 --2- 172.21.15.18:0/3976571659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd1c104720 0x7fdd1c10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 --2- 172.21.15.18:0/3976571659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c103d80 0x7fdd1c1041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 -- 172.21.15.18:0/3976571659 >> 172.21.15.18:0/3976571659 conn(0x7fdd1c0fe140 msgr2=0x7fdd1c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 -- 172.21.15.18:0/3976571659 shutdown_connections 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.340+0000 7fdd21c00700 1 -- 172.21.15.18:0/3976571659 wait complete. 2024-04-16T17:00:26.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.341+0000 7fdd21c00700 1 Processor -- start 2024-04-16T17:00:26.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.341+0000 7fdd21c00700 1 -- start start 2024-04-16T17:00:26.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd21c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd1c102b80 0x7fdd1c19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd21c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c103d80 0x7fdd1c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd21c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c104720 0x7fdd1c1a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd21c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c10e740 con 0x7fdd1c102b80 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd21c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c10e5c0 con 0x7fdd1c103d80 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd21c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c10e8c0 con 0x7fdd1c104720 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd1aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd1c102b80 0x7fdd1c19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd1aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd1c102b80 0x7fdd1c19af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59176/0 (socket says 172.21.15.18:59176) 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd1aa00700 1 -- 172.21.15.18:0/684347771 learned_addr learned my addr 172.21.15.18:0/684347771 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd1b400700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c104720 0x7fdd1c1a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.342+0000 7fdd1a000700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c103d80 0x7fdd1c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd1aa00700 1 -- 172.21.15.18:0/684347771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c104720 msgr2=0x7fdd1c1a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd1aa00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c104720 0x7fdd1c1a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd1aa00700 1 -- 172.21.15.18:0/684347771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c103d80 msgr2=0x7fdd1c19b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:26.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd1aa00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c103d80 0x7fdd1c19b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd1aa00700 1 -- 172.21.15.18:0/684347771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c1a2bc0 con 0x7fdd1c102b80 2024-04-16T17:00:26.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd1b400700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c104720 0x7fdd1c1a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:00:26.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd1aa00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd1c102b80 0x7fdd1c19af60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00b770 tx=0x7fdd0c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:26.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd13400700 1 -- 172.21.15.18:0/684347771 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c003c00 con 0x7fdd1c102b80 2024-04-16T17:00:26.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd13400700 1 -- 172.21.15.18:0/684347771 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0c003da0 con 0x7fdd1c102b80 2024-04-16T17:00:26.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd13400700 1 -- 172.21.15.18:0/684347771 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c01b990 con 0x7fdd1c102b80 2024-04-16T17:00:26.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.343+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c1a2e50 con 0x7fdd1c102b80 2024-04-16T17:00:26.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.344+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c1a31f0 con 0x7fdd1c102b80 2024-04-16T17:00:26.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.345+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1c1a3480 con 0x7fdd1c102b80 2024-04-16T17:00:26.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.347+0000 7fdd13400700 1 -- 172.21.15.18:0/684347771 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd0c0056e0 con 0x7fdd1c102b80 2024-04-16T17:00:26.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.348+0000 7fdd13400700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdd000a3cc0 0x7fdd000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:26.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.348+0000 7fdd13400700 1 -- 172.21.15.18:0/684347771 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdd0c00f020 con 0x7fdd1c102b80 2024-04-16T17:00:26.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.349+0000 7fdd1a000700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdd000a3cc0 0x7fdd000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:26.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.350+0000 7fdd1a000700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdd000a3cc0 0x7fdd000a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdd08007c60 tx=0x7fdd080073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:26.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.352+0000 7fdd13400700 1 -- 172.21.15.18:0/684347771 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd0c02b020 con 0x7fdd1c102b80 2024-04-16T17:00:26.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.585+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdd1c1a3780 con 0x7fdd1c102b80 2024-04-16T17:00:26.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.586+0000 7fdd13400700 1 -- 172.21.15.18:0/684347771 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fdd0c01a3c0 con 0x7fdd1c102b80 2024-04-16T17:00:26.588 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdd000a3cc0 msgr2=0x7fdd000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdd000a3cc0 0x7fdd000a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdd08007c60 tx=0x7fdd080073d0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd1c102b80 msgr2=0x7fdd1c19af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd1c102b80 0x7fdd1c19af60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00b770 tx=0x7fdd0c007a20 comp rx=0 tx=0).stop 2024-04-16T17:00:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 shutdown_connections 2024-04-16T17:00:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdd000a3cc0 0x7fdd000a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c104720 0x7fdd1c1a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd1c102b80 0x7fdd1c19af60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 --2- 172.21.15.18:0/684347771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c103d80 0x7fdd1c19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 >> 172.21.15.18:0/684347771 conn(0x7fdd1c0fe140 msgr2=0x7fdd1c074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 shutdown_connections 2024-04-16T17:00:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:26.589+0000 7fdd21c00700 1 -- 172.21.15.18:0/684347771 wait complete. 2024-04-16T17:00:27.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T17:00:27.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/684347771' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T17:00:27.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:27.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:27.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/684347771' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:27.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T17:00:27.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/684347771' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:27.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:27.382 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713286824,"num_in_osds":12,"osd_in_since":1713286804,"num_remapped_pgs":0} 2024-04-16T17:00:27.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd dump --format=json 2024-04-16T17:00:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:00:27.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:27 smithi018 ceph-mon[31776]: Detected new or changed devices on smithi043 2024-04-16T17:00:27.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:27 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi043 to 4271M 2024-04-16T17:00:27.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:27 smithi018 ceph-mon[31776]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:28.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:27 smithi022 ceph-mon[36023]: Detected new or changed devices on smithi043 2024-04-16T17:00:28.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:27 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi043 to 4271M 2024-04-16T17:00:28.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:27 smithi022 ceph-mon[36023]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:28.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:27 smithi043 ceph-mon[36901]: Detected new or changed devices on smithi043 2024-04-16T17:00:28.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:27 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi043 to 4271M 2024-04-16T17:00:28.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:27 smithi043 ceph-mon[36901]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:29.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 -- 172.21.15.18:0/3330290698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef1410a050 msgr2=0x7fef14111c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:29.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 --2- 172.21.15.18:0/3330290698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef1410a050 0x7fef14111c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00b0d0 tx=0x7fef0c00b3e0 comp rx=0 tx=0).stop 2024-04-16T17:00:29.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 -- 172.21.15.18:0/3330290698 shutdown_connections 2024-04-16T17:00:29.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 --2- 172.21.15.18:0/3330290698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef1410a050 0x7fef14111c20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:29.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 --2- 172.21.15.18:0/3330290698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef14104f70 0x7fef14105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:29.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 --2- 172.21.15.18:0/3330290698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef14105920 0x7fef14109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:29.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 -- 172.21.15.18:0/3330290698 >> 172.21.15.18:0/3330290698 conn(0x7fef14100800 msgr2=0x7fef14102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:29.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 -- 172.21.15.18:0/3330290698 shutdown_connections 2024-04-16T17:00:29.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.971+0000 7fef1c400700 1 -- 172.21.15.18:0/3330290698 wait complete. 2024-04-16T17:00:29.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.972+0000 7fef1c400700 1 Processor -- start 2024-04-16T17:00:29.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef1c400700 1 -- start start 2024-04-16T17:00:29.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef1c400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef14104f70 0x7fef1419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef1c400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef14105920 0x7fef1419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef1c400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef1410a050 0x7fef1419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef1c400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fef14115090 con 0x7fef1410a050 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef1c400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fef14114f10 con 0x7fef14104f70 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef1c400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef14115210 con 0x7fef14105920 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef19000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef14105920 0x7fef1419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef19000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef14105920 0x7fef1419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39332/0 (socket says 172.21.15.18:39332) 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef1a400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef1410a050 0x7fef1419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.973+0000 7fef19000700 1 -- 172.21.15.18:0/3432959055 learned_addr learned my addr 172.21.15.18:0/3432959055 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef19a00700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef14104f70 0x7fef1419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef19000700 1 -- 172.21.15.18:0/3432959055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef14104f70 msgr2=0x7fef1419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef19000700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef14104f70 0x7fef1419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef19000700 1 -- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef1410a050 msgr2=0x7fef1419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:29.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef19000700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef1410a050 0x7fef1419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:29.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef19000700 1 -- 172.21.15.18:0/3432959055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef141a0470 con 0x7fef14105920 2024-04-16T17:00:29.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef19000700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef14105920 0x7fef1419b9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef0000e970 tx=0x7fef0000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:29.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef12000700 1 -- 172.21.15.18:0/3432959055 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00004a30 con 0x7fef14105920 2024-04-16T17:00:29.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef12000700 1 -- 172.21.15.18:0/3432959055 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0000ccc0 con 0x7fef14105920 2024-04-16T17:00:29.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.974+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef141a0760 con 0x7fef14105920 2024-04-16T17:00:29.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.975+0000 7fef12000700 1 -- 172.21.15.18:0/3432959055 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00005650 con 0x7fef14105920 2024-04-16T17:00:29.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.975+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef141a7fe0 con 0x7fef14105920 2024-04-16T17:00:29.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.976+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef140697e0 con 0x7fef14105920 2024-04-16T17:00:29.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.977+0000 7fef12000700 1 -- 172.21.15.18:0/3432959055 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef00004bd0 con 0x7fef14105920 2024-04-16T17:00:29.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.979+0000 7fef12000700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feefc0a3cc0 0x7feefc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:29.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.979+0000 7fef12000700 1 -- 172.21.15.18:0/3432959055 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fef000d00f0 con 0x7fef14105920 2024-04-16T17:00:29.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.979+0000 7fef19a00700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feefc0a3cc0 0x7feefc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:29.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.980+0000 7fef19a00700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feefc0a3cc0 0x7feefc0a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef08009c60 tx=0x7fef08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:29.983+0000 7fef12000700 1 -- 172.21.15.18:0/3432959055 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef0009ced0 con 0x7fef14105920 2024-04-16T17:00:30.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:29 smithi022 ceph-mon[36023]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:30.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:29 smithi018 ceph-mon[31776]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:30.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.220+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef1419c7c0 con 0x7fef14105920 2024-04-16T17:00:30.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.221+0000 7fef12000700 1 -- 172.21.15.18:0/3432959055 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18362 (secure 0 0 0) 0x7fef0009c7d0 con 0x7fef14105920 2024-04-16T17:00:30.223 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:30.223 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":81,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:00:25.899823+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feefc0a3cc0 msgr2=0x7feefc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feefc0a3cc0 0x7feefc0a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef08009c60 tx=0x7fef08009260 comp rx=0 tx=0).stop 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef14105920 msgr2=0x7fef1419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef14105920 0x7fef1419b9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef0000e970 tx=0x7fef0000ec80 comp rx=0 tx=0).stop 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 shutdown_connections 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feefc0a3cc0 0x7feefc0a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef14104f70 0x7fef1419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef1410a050 0x7fef1419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 --2- 172.21.15.18:0/3432959055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef14105920 0x7fef1419b9a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:30.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 >> 172.21.15.18:0/3432959055 conn(0x7fef14100800 msgr2=0x7fef14101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:30.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 shutdown_connections 2024-04-16T17:00:30.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:30.225+0000 7fef1c400700 1 -- 172.21.15.18:0/3432959055 wait complete. 2024-04-16T17:00:30.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:29 smithi043 ceph-mon[36901]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:30.956 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T17:00:30.956 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-16T17:00:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:30 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3432959055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:00:31.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:30 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3432959055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:00:31.361 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:30 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3432959055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:00:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:00:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:31 smithi018 ceph-mon[31776]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:32.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:31 smithi043 ceph-mon[36901]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:32.331 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:31 smithi022 ceph-mon[36023]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:33.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.483+0000 7fa059000700 1 -- 172.21.15.18:0/1436882280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0541023b0 msgr2=0x7fa05410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:33.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.483+0000 7fa059000700 1 --2- 172.21.15.18:0/1436882280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0541023b0 0x7fa05410e8a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00b0a0 tx=0x7fa04c00b3b0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.483+0000 7fa059000700 1 -- 172.21.15.18:0/1436882280 shutdown_connections 2024-04-16T17:00:33.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.483+0000 7fa059000700 1 --2- 172.21.15.18:0/1436882280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa054107a40 0x7fa054107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.483+0000 7fa059000700 1 --2- 172.21.15.18:0/1436882280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0541023b0 0x7fa05410e8a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.483+0000 7fa059000700 1 --2- 172.21.15.18:0/1436882280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0541019f0 0x7fa054101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.483+0000 7fa059000700 1 -- 172.21.15.18:0/1436882280 >> 172.21.15.18:0/1436882280 conn(0x7fa0540789a0 msgr2=0x7fa054078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:33.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.483+0000 7fa059000700 1 -- 172.21.15.18:0/1436882280 shutdown_connections 2024-04-16T17:00:33.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.484+0000 7fa059000700 1 -- 172.21.15.18:0/1436882280 wait complete. 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.485+0000 7fa059000700 1 Processor -- start 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.485+0000 7fa059000700 1 -- start start 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.485+0000 7fa059000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0541019f0 0x7fa0541035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.485+0000 7fa059000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0541023b0 0x7fa054103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.485+0000 7fa059000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa054107a40 0x7fa054104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.485+0000 7fa059000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa054111cf0 con 0x7fa054107a40 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.485+0000 7fa059000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa054111b70 con 0x7fa0541023b0 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.485+0000 7fa059000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa054111e70 con 0x7fa0541019f0 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0541023b0 0x7fa054103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0541019f0 0x7fa0541035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0541023b0 0x7fa054103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:39746/0 (socket says 172.21.15.18:39746) 2024-04-16T17:00:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 -- 172.21.15.18:0/1581760298 learned_addr learned my addr 172.21.15.18:0/1581760298 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa052800700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa054107a40 0x7fa054104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 -- 172.21.15.18:0/1581760298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0541019f0 msgr2=0x7fa0541035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0541019f0 0x7fa0541035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 -- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa054107a40 msgr2=0x7fa054104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa054107a40 0x7fa054104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 -- 172.21.15.18:0/1581760298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0541a7730 con 0x7fa0541023b0 2024-04-16T17:00:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.486+0000 7fa051400700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0541023b0 0x7fa054103ae0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa04000e9c0 tx=0x7fa04000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:33.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.487+0000 7fa04aa00700 1 -- 172.21.15.18:0/1581760298 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04000cbd0 con 0x7fa0541023b0 2024-04-16T17:00:33.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.487+0000 7fa04aa00700 1 -- 172.21.15.18:0/1581760298 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa04000ee60 con 0x7fa0541023b0 2024-04-16T17:00:33.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.487+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0541a7a20 con 0x7fa0541023b0 2024-04-16T17:00:33.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.487+0000 7fa04aa00700 1 -- 172.21.15.18:0/1581760298 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040005630 con 0x7fa0541023b0 2024-04-16T17:00:33.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.487+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a7f10 con 0x7fa0541023b0 2024-04-16T17:00:33.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.488+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0540697e0 con 0x7fa0541023b0 2024-04-16T17:00:33.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.490+0000 7fa04aa00700 1 -- 172.21.15.18:0/1581760298 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa040004750 con 0x7fa0541023b0 2024-04-16T17:00:33.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.491+0000 7fa04aa00700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa0380a3c40 0x7fa0380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:33.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.492+0000 7fa04aa00700 1 -- 172.21.15.18:0/1581760298 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa0400d0260 con 0x7fa0541023b0 2024-04-16T17:00:33.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.492+0000 7fa051e00700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa0380a3c40 0x7fa0380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:33.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.492+0000 7fa051e00700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa0380a3c40 0x7fa0380a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa044009be0 tx=0x7fa044009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:33.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.496+0000 7fa04aa00700 1 -- 172.21.15.18:0/1581760298 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa04009d040 con 0x7fa0541023b0 2024-04-16T17:00:33.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.727+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa05402dfe0 con 0x7fa0541023b0 2024-04-16T17:00:33.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.727+0000 7fa04aa00700 1 -- 172.21.15.18:0/1581760298 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fa04009c940 con 0x7fa0541023b0 2024-04-16T17:00:33.729 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:00:33.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa0380a3c40 msgr2=0x7fa0380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa0380a3c40 0x7fa0380a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa044009be0 tx=0x7fa044009260 comp rx=0 tx=0).stop 2024-04-16T17:00:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0541023b0 msgr2=0x7fa054103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0541023b0 0x7fa054103ae0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa04000e9c0 tx=0x7fa04000ecd0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 shutdown_connections 2024-04-16T17:00:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa0380a3c40 0x7fa0380a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa054107a40 0x7fa054104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0541019f0 0x7fa0541035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 --2- 172.21.15.18:0/1581760298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0541023b0 0x7fa054103ae0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:33.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 >> 172.21.15.18:0/1581760298 conn(0x7fa0540789a0 msgr2=0x7fa05406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:33.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 shutdown_connections 2024-04-16T17:00:33.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:33.731+0000 7fa059000700 1 -- 172.21.15.18:0/1581760298 wait complete. 2024-04-16T17:00:34.157 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:33 smithi018 ceph-mon[31776]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:34.157 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:33 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1581760298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:00:34.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:33 smithi043 ceph-mon[36901]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:34.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:33 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1581760298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:00:34.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:33 smithi022 ceph-mon[36023]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:34.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:33 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1581760298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:00:34.592 INFO:tasks.cephadm:Adding mds.a on smithi018 2024-04-16T17:00:34.592 INFO:tasks.cephadm:Adding mds.d on smithi018 2024-04-16T17:00:34.592 INFO:tasks.cephadm:Adding mds.g on smithi018 2024-04-16T17:00:34.592 INFO:tasks.cephadm:Adding mds.j on smithi018 2024-04-16T17:00:34.592 INFO:tasks.cephadm:Adding mds.b on smithi022 2024-04-16T17:00:34.593 INFO:tasks.cephadm:Adding mds.e on smithi022 2024-04-16T17:00:34.593 INFO:tasks.cephadm:Adding mds.h on smithi022 2024-04-16T17:00:34.593 INFO:tasks.cephadm:Adding mds.k on smithi022 2024-04-16T17:00:34.593 INFO:tasks.cephadm:Adding mds.c on smithi043 2024-04-16T17:00:34.593 INFO:tasks.cephadm:Adding mds.f on smithi043 2024-04-16T17:00:34.593 INFO:tasks.cephadm:Adding mds.i on smithi043 2024-04-16T17:00:34.593 INFO:tasks.cephadm:Adding mds.l on smithi043 2024-04-16T17:00:34.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph orch apply mds all '12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l' 2024-04-16T17:00:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:00:36.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:35 smithi043 ceph-mon[36901]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:36.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:35 smithi018 ceph-mon[31776]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:36.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:35 smithi022 ceph-mon[36023]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 -- 172.21.15.43:0/1574668895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40108830 msgr2=0x7f1e4010aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 --2- 172.21.15.43:0/1574668895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40108830 0x7f1e4010aca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800b0a0 tx=0x7f1e3800b3b0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 -- 172.21.15.43:0/1574668895 shutdown_connections 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 --2- 172.21.15.43:0/1574668895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e401035d0 0x7f1e401059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 --2- 172.21.15.43:0/1574668895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40108830 0x7f1e4010aca0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 --2- 172.21.15.43:0/1574668895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e40105f00 0x7f1e401082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 -- 172.21.15.43:0/1574668895 >> 172.21.15.43:0/1574668895 conn(0x7f1e400fcfe0 msgr2=0x7f1e400ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 -- 172.21.15.43:0/1574668895 shutdown_connections 2024-04-16T17:00:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.643+0000 7f1e47b54700 1 -- 172.21.15.43:0/1574668895 wait complete. 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e47b54700 1 Processor -- start 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e47b54700 1 -- start start 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e47b54700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e401035d0 0x7f1e40198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e47b54700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40105f00 0x7f1e401992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e47b54700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e40108830 0x7f1e401a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e47b54700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010e770 con 0x7f1e40108830 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e47b54700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010e5f0 con 0x7f1e401035d0 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e47b54700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010e8f0 con 0x7f1e40105f00 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.644+0000 7f1e450ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40105f00 0x7f1e401992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e450ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40105f00 0x7f1e401992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52042/0 (socket says 172.21.15.43:52042) 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e450ef700 1 -- 172.21.15.43:0/956303255 learned_addr learned my addr 172.21.15.43:0/956303255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e458f0700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e401035d0 0x7f1e40198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e450ef700 1 -- 172.21.15.43:0/956303255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e401035d0 msgr2=0x7f1e40198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e450ef700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e401035d0 0x7f1e40198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e450ef700 1 -- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e40108830 msgr2=0x7f1e401a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e450ef700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e40108830 0x7f1e401a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e450ef700 1 -- 172.21.15.43:0/956303255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e40101440 con 0x7f1e40105f00 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e450ef700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40105f00 0x7f1e401992e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000b530 tx=0x7f1e3000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e36ffd700 1 -- 172.21.15.43:0/956303255 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30015020 con 0x7f1e40105f00 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e40101730 con 0x7f1e40105f00 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e36ffd700 1 -- 172.21.15.43:0/956303255 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e30011d10 con 0x7f1e40105f00 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e36ffd700 1 -- 172.21.15.43:0/956303255 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30010e50 con 0x7f1e40105f00 2024-04-16T17:00:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.645+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e40101cc0 con 0x7f1e40105f00 2024-04-16T17:00:36.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.647+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e400697e0 con 0x7f1e40105f00 2024-04-16T17:00:36.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.648+0000 7f1e36ffd700 1 -- 172.21.15.43:0/956303255 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1e30018020 con 0x7f1e40105f00 2024-04-16T17:00:36.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.648+0000 7f1e36ffd700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1e2c0a3de0 0x7f1e2c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:36.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.649+0000 7f1e36ffd700 1 -- 172.21.15.43:0/956303255 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1e300cf800 con 0x7f1e40105f00 2024-04-16T17:00:36.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.649+0000 7f1e458f0700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1e2c0a3de0 0x7f1e2c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.650+0000 7f1e458f0700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1e2c0a3de0 0x7f1e2c0a62a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005950 tx=0x7f1e3c005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:36.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.653+0000 7f1e36ffd700 1 -- 172.21.15.43:0/956303255 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e3001f020 con 0x7f1e40105f00 2024-04-16T17:00:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.871+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l", "target": ["mon-mgr", ""]}) -- 0x7f1e400fe990 con 0x7f1e2c0a3de0 2024-04-16T17:00:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.888+0000 7f1e36ffd700 1 -- 172.21.15.43:0/956303255 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1e400fe990 con 0x7f1e2c0a3de0 2024-04-16T17:00:36.889 INFO:teuthology.orchestra.run.smithi043.stdout:Scheduled mds.all update... 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1e2c0a3de0 msgr2=0x7f1e2c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1e2c0a3de0 0x7f1e2c0a62a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005950 tx=0x7f1e3c005ed0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40105f00 msgr2=0x7f1e401992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40105f00 0x7f1e401992e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000b530 tx=0x7f1e3000b8f0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 shutdown_connections 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1e2c0a3de0 0x7f1e2c0a62a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e401035d0 0x7f1e40198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e40108830 0x7f1e401a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 --2- 172.21.15.43:0/956303255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e40105f00 0x7f1e401992e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 >> 172.21.15.43:0/956303255 conn(0x7f1e400fcfe0 msgr2=0x7f1e40103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 shutdown_connections 2024-04-16T17:00:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-16T17:00:36.890+0000 7f1e47b54700 1 -- 172.21.15.43:0/956303255 wait complete. 2024-04-16T17:00:37.586 DEBUG:teuthology.orchestra.run.smithi018:mds.a> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.a.service 2024-04-16T17:00:37.593 DEBUG:teuthology.orchestra.run.smithi018:mds.d> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.d.service 2024-04-16T17:00:37.638 DEBUG:teuthology.orchestra.run.smithi018:mds.g> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.g.service 2024-04-16T17:00:37.684 DEBUG:teuthology.orchestra.run.smithi018:mds.j> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.j.service 2024-04-16T17:00:37.730 DEBUG:teuthology.orchestra.run.smithi022:mds.b> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.b.service 2024-04-16T17:00:37.733 DEBUG:teuthology.orchestra.run.smithi022:mds.e> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.e.service 2024-04-16T17:00:37.776 DEBUG:teuthology.orchestra.run.smithi022:mds.h> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.h.service 2024-04-16T17:00:37.819 DEBUG:teuthology.orchestra.run.smithi022:mds.k> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.k.service 2024-04-16T17:00:37.862 DEBUG:teuthology.orchestra.run.smithi043:mds.c> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.c.service 2024-04-16T17:00:37.864 DEBUG:teuthology.orchestra.run.smithi043:mds.f> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.f.service 2024-04-16T17:00:37.906 DEBUG:teuthology.orchestra.run.smithi043:mds.i> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.i.service 2024-04-16T17:00:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:37 smithi018 ceph-mon[31776]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:37 smithi018 ceph-mon[31776]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:37 smithi018 ceph-mon[31776]: Saving service mds.all spec with placement smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l;count:12 2024-04-16T17:00:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:37 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:37.949 DEBUG:teuthology.orchestra.run.smithi043:mds.l> sudo journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.l.service 2024-04-16T17:00:37.993 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-16T17:00:37.993 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:00:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:37 smithi022 ceph-mon[36023]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:37 smithi022 ceph-mon[36023]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:37 smithi022 ceph-mon[36023]: Saving service mds.all spec with placement smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l;count:12 2024-04-16T17:00:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:38.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:37 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 -- 172.21.15.18:0/533896675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa74c105460 msgr2=0x7fa74c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 --2- 172.21.15.18:0/533896675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa74c105460 0x7fa74c1058e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa738009900 tx=0x7fa73802bc30 comp rx=0 tx=0).stop 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 -- 172.21.15.18:0/533896675 shutdown_connections 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 --2- 172.21.15.18:0/533896675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa74c105e20 0x7fa74c10c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 --2- 172.21.15.18:0/533896675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa74c105460 0x7fa74c1058e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 --2- 172.21.15.18:0/533896675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa74c104270 0x7fa74c104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 -- 172.21.15.18:0/533896675 >> 172.21.15.18:0/533896675 conn(0x7fa74c0ff820 msgr2=0x7fa74c101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 -- 172.21.15.18:0/533896675 shutdown_connections 2024-04-16T17:00:38.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.230+0000 7fa754200640 1 -- 172.21.15.18:0/533896675 wait complete. 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.231+0000 7fa754200640 1 Processor -- start 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.231+0000 7fa754200640 1 -- start start 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa754200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa74c104270 0x7fa74c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa754200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa74c105460 0x7fa74c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa754200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa74c105e20 0x7fa74c1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa754200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c10ff50 con 0x7fa74c105460 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa754200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c10fdd0 con 0x7fa74c105e20 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa754200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1100d0 con 0x7fa74c104270 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa751800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa74c104270 0x7fa74c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa751800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa74c104270 0x7fa74c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:60730/0 (socket says 172.21.15.18:60730) 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa751800640 1 -- 172.21.15.18:0/808799519 learned_addr learned my addr 172.21.15.18:0/808799519 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:38.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa750e00640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa74c105460 0x7fa74c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa752200640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa74c105e20 0x7fa74c1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa751800640 1 -- 172.21.15.18:0/808799519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa74c105e20 msgr2=0x7fa74c1a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa751800640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa74c105e20 0x7fa74c1a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa751800640 1 -- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa74c105460 msgr2=0x7fa74c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa751800640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa74c105460 0x7fa74c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa751800640 1 -- 172.21.15.18:0/808799519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c1a2180 con 0x7fa74c104270 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.232+0000 7fa752200640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa74c105e20 0x7fa74c1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.233+0000 7fa751800640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa74c104270 0x7fa74c19a560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa74000ebc0 tx=0x7fa74000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.233+0000 7fa74a000640 1 -- 172.21.15.18:0/808799519 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740003b70 con 0x7fa74c104270 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.233+0000 7fa74a000640 1 -- 172.21.15.18:0/808799519 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa740003d10 con 0x7fa74c104270 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.233+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c1a2470 con 0x7fa74c104270 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.233+0000 7fa74a000640 1 -- 172.21.15.18:0/808799519 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740019980 con 0x7fa74c104270 2024-04-16T17:00:38.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.233+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c1a2990 con 0x7fa74c104270 2024-04-16T17:00:38.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.235+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74c10b590 con 0x7fa74c104270 2024-04-16T17:00:38.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.236+0000 7fa74a000640 1 -- 172.21.15.18:0/808799519 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa740019b20 con 0x7fa74c104270 2024-04-16T17:00:38.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.238+0000 7fa74a000640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa7340a3bb0 0x7fa7340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:38.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.238+0000 7fa74a000640 1 -- 172.21.15.18:0/808799519 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa7400d08b0 con 0x7fa74c104270 2024-04-16T17:00:38.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.238+0000 7fa750e00640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa7340a3bb0 0x7fa7340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:38.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.239+0000 7fa750e00640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa7340a3bb0 0x7fa7340a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa73802c240 tx=0x7fa73802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:38.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.242+0000 7fa74a000640 1 -- 172.21.15.18:0/808799519 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa74009d690 con 0x7fa74c104270 2024-04-16T17:00:38.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:37 smithi043 ceph-mon[36901]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:38.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:37 smithi043 ceph-mon[36901]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:38.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:37 smithi043 ceph-mon[36901]: Saving service mds.all spec with placement smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l;count:12 2024-04-16T17:00:38.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:38.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:37 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:38.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.449+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa74c062c00 con 0x7fa74c104270 2024-04-16T17:00:38.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.451+0000 7fa74a000640 1 -- 172.21.15.18:0/808799519 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18362 (secure 0 0 0) 0x7fa74009cf90 con 0x7fa74c104270 2024-04-16T17:00:38.453 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:38.453 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":81,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:00:25.899823+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.453+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa7340a3bb0 msgr2=0x7fa7340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.453+0000 7fa754200640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa7340a3bb0 0x7fa7340a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa73802c240 tx=0x7fa73802df90 comp rx=0 tx=0).stop 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa74c104270 msgr2=0x7fa74c19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa74c104270 0x7fa74c19a560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa74000ebc0 tx=0x7fa74000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 shutdown_connections 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa74c105e20 0x7fa74c1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa7340a3bb0 0x7fa7340a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa74c105460 0x7fa74c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 --2- 172.21.15.18:0/808799519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa74c104270 0x7fa74c19a560 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 >> 172.21.15.18:0/808799519 conn(0x7fa74c0ff820 msgr2=0x7fa74c108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:38.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 shutdown_connections 2024-04-16T17:00:38.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.454+0000 7fa754200640 1 -- 172.21.15.18:0/808799519 wait complete. 2024-04-16T17:00:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T17:00:38.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:00:38.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.706+0000 7f5fbec00640 1 -- 172.21.15.18:0/1851809054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb8074a60 msgr2=0x7f5fb8074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.706+0000 7f5fbec00640 1 --2- 172.21.15.18:0/1851809054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb8074a60 0x7f5fb8074ee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009900 tx=0x7f5fa402bc30 comp rx=0 tx=0).stop 2024-04-16T17:00:38.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.706+0000 7f5fbec00640 1 -- 172.21.15.18:0/1851809054 shutdown_connections 2024-04-16T17:00:38.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.706+0000 7f5fbec00640 1 --2- 172.21.15.18:0/1851809054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fb810e420 0x7f5fb8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.706+0000 7f5fbec00640 1 --2- 172.21.15.18:0/1851809054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb8074a60 0x7f5fb8074ee0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.706+0000 7f5fbec00640 1 --2- 172.21.15.18:0/1851809054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fb8076040 0x7f5fb8074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.706+0000 7f5fbec00640 1 -- 172.21.15.18:0/1851809054 >> 172.21.15.18:0/1851809054 conn(0x7f5fb80fd3c0 msgr2=0x7f5fb80ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:38.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.706+0000 7f5fbec00640 1 -- 172.21.15.18:0/1851809054 shutdown_connections 2024-04-16T17:00:38.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.707+0000 7f5fbec00640 1 -- 172.21.15.18:0/1851809054 wait complete. 2024-04-16T17:00:38.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.708+0000 7f5fbec00640 1 Processor -- start 2024-04-16T17:00:38.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.708+0000 7f5fbec00640 1 -- start start 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.708+0000 7f5fbec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fb8074a60 0x7f5fb819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.708+0000 7f5fbec00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fb8076040 0x7f5fb819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb810e420 0x7f5fb81a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbec00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81140e0 con 0x7f5fb8076040 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8113f60 con 0x7f5fb8074a60 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8114260 con 0x7f5fb810e420 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbd800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fb8074a60 0x7f5fb819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbd800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fb8074a60 0x7f5fb819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53358/0 (socket says 172.21.15.18:53358) 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbd800640 1 -- 172.21.15.18:0/720043672 learned_addr learned my addr 172.21.15.18:0/720043672 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbce00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fb8076040 0x7f5fb819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbe200640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb810e420 0x7f5fb81a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbd800640 1 -- 172.21.15.18:0/720043672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb810e420 msgr2=0x7f5fb81a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbd800640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb810e420 0x7f5fb81a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbd800640 1 -- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fb8076040 msgr2=0x7f5fb819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbd800640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fb8076040 0x7f5fb819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.709+0000 7f5fbd800640 1 -- 172.21.15.18:0/720043672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb81a62f0 con 0x7f5fb8074a60 2024-04-16T17:00:38.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.710+0000 7f5fbd800640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fb8074a60 0x7f5fb819e6d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009d50 tx=0x7f5fa800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:38.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.710+0000 7f5fb6000640 1 -- 172.21.15.18:0/720043672 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8003b70 con 0x7f5fb8074a60 2024-04-16T17:00:38.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.710+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb81a65e0 con 0x7f5fb8074a60 2024-04-16T17:00:38.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.710+0000 7f5fb6000640 1 -- 172.21.15.18:0/720043672 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa8003d10 con 0x7f5fb8074a60 2024-04-16T17:00:38.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.710+0000 7f5fb6000640 1 -- 172.21.15.18:0/720043672 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8010a20 con 0x7f5fb8074a60 2024-04-16T17:00:38.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.710+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb81a6b50 con 0x7f5fb8074a60 2024-04-16T17:00:38.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.711+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb804fd90 con 0x7f5fb8074a60 2024-04-16T17:00:38.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.713+0000 7f5fb6000640 1 -- 172.21.15.18:0/720043672 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5fa8003ec0 con 0x7f5fb8074a60 2024-04-16T17:00:38.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.714+0000 7f5fb6000640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5f9c0a3ba0 0x7f5f9c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:38.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.715+0000 7f5fb6000640 1 -- 172.21.15.18:0/720043672 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5fa80d0670 con 0x7f5fb8074a60 2024-04-16T17:00:38.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.715+0000 7f5fbce00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5f9c0a3ba0 0x7f5f9c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:38.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.715+0000 7f5fbce00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5f9c0a3ba0 0x7f5f9c0a6060 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402c7e0 tx=0x7f5fa4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:38.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.719+0000 7f5fb6000640 1 -- 172.21.15.18:0/720043672 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fa809d450 con 0x7f5fb8074a60 2024-04-16T17:00:38.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.932+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fb8062c00 con 0x7f5fb8074a60 2024-04-16T17:00:38.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.933+0000 7f5fb6000640 1 -- 172.21.15.18:0/720043672 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f5fa809cd50 con 0x7f5fb8074a60 2024-04-16T17:00:38.935 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5f9c0a3ba0 msgr2=0x7f5f9c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5f9c0a3ba0 0x7f5f9c0a6060 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402c7e0 tx=0x7f5fa4000bc0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fb8074a60 msgr2=0x7f5fb819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fb8074a60 0x7f5fb819e6d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009d50 tx=0x7f5fa800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 shutdown_connections 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb810e420 0x7f5fb81a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5f9c0a3ba0 0x7f5f9c0a6060 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fb8076040 0x7f5fb819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 --2- 172.21.15.18:0/720043672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fb8074a60 0x7f5fb819e6d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 >> 172.21.15.18:0/720043672 conn(0x7f5fb80fd3c0 msgr2=0x7f5fb81017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:38.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 shutdown_connections 2024-04-16T17:00:38.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:38.936+0000 7f5fbec00640 1 -- 172.21.15.18:0/720043672 wait complete. 2024-04-16T17:00:38.959 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:39.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:39.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:39.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:39.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:39.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:39.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:38 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:39.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:38 smithi043 ceph-mon[36901]: Deploying daemon mds.l on smithi043 2024-04-16T17:00:39.091 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:38 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/808799519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:00:39.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 -- 172.21.15.18:0/2136423094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 msgr2=0x7f3ed41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:39.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 --2- 172.21.15.18:0/2136423094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 0x7f3ed41058e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009900 tx=0x7f3ebc02bc50 comp rx=0 tx=0).stop 2024-04-16T17:00:39.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 -- 172.21.15.18:0/2136423094 shutdown_connections 2024-04-16T17:00:39.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 --2- 172.21.15.18:0/2136423094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed4105e20 0x7f3ed410c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 --2- 172.21.15.18:0/2136423094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 0x7f3ed41058e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 --2- 172.21.15.18:0/2136423094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed4104270 0x7f3ed4104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 -- 172.21.15.18:0/2136423094 >> 172.21.15.18:0/2136423094 conn(0x7f3ed40ff820 msgr2=0x7f3ed4101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:39.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 -- 172.21.15.18:0/2136423094 shutdown_connections 2024-04-16T17:00:39.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.167+0000 7f3edac00640 1 -- 172.21.15.18:0/2136423094 wait complete. 2024-04-16T17:00:39.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.168+0000 7f3edac00640 1 Processor -- start 2024-04-16T17:00:39.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.168+0000 7f3edac00640 1 -- start start 2024-04-16T17:00:39.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.168+0000 7f3edac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed4104270 0x7f3ed419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:39.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.168+0000 7f3edac00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 0x7f3ed419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:39.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.168+0000 7f3edac00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed4105e20 0x7f3ed41a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:39.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.168+0000 7f3edac00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed410ff50 con 0x7f3ed4105e20 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.168+0000 7f3edac00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed410fdd0 con 0x7f3ed4105460 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.168+0000 7f3edac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41100d0 con 0x7f3ed4104270 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ecb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 0x7f3ed419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed4104270 0x7f3ed419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed8c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed4105e20 0x7f3ed41a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ecb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 0x7f3ed419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53372/0 (socket says 172.21.15.18:53372) 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ecb400640 1 -- 172.21.15.18:0/2738466416 learned_addr learned my addr 172.21.15.18:0/2738466416 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed4104270 0x7f3ed419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:60768/0 (socket says 172.21.15.18:60768) 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed8c00640 1 -- 172.21.15.18:0/2738466416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed4104270 msgr2=0x7f3ed419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed8c00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed4104270 0x7f3ed419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed8c00640 1 -- 172.21.15.18:0/2738466416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 msgr2=0x7f3ed419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:39.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed8c00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 0x7f3ed419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed8c00640 1 -- 172.21.15.18:0/2738466416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed41a21a0 con 0x7f3ed4105e20 2024-04-16T17:00:39.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.169+0000 7f3ed8c00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed4105e20 0x7f3ed41a1b30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00e970 tx=0x7f3ecc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:39.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.170+0000 7f3ed1600640 1 -- 172.21.15.18:0/2738466416 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00ccc0 con 0x7f3ed4105e20 2024-04-16T17:00:39.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.170+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed41a2490 con 0x7f3ed4105e20 2024-04-16T17:00:39.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.170+0000 7f3ed1600640 1 -- 172.21.15.18:0/2738466416 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ecc005020 con 0x7f3ed4105e20 2024-04-16T17:00:39.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.170+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed41a29e0 con 0x7f3ed4105e20 2024-04-16T17:00:39.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.170+0000 7f3ed1600640 1 -- 172.21.15.18:0/2738466416 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc01c070 con 0x7f3ed4105e20 2024-04-16T17:00:39.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.171+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed410b590 con 0x7f3ed4105e20 2024-04-16T17:00:39.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.173+0000 7f3ed1600640 1 -- 172.21.15.18:0/2738466416 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ecc0051d0 con 0x7f3ed4105e20 2024-04-16T17:00:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:38 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:38 smithi022 ceph-mon[36023]: Deploying daemon mds.l on smithi043 2024-04-16T17:00:39.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:38 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/808799519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:00:39.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.174+0000 7f3ed1600640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3eb80a3bb0 0x7f3eb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:39.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.174+0000 7f3ed1600640 1 -- 172.21.15.18:0/2738466416 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3ecc0cff20 con 0x7f3ed4105e20 2024-04-16T17:00:39.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.175+0000 7f3ed3e00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3eb80a3bb0 0x7f3eb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:39.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.175+0000 7f3ed3e00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3eb80a3bb0 0x7f3eb80a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ed41052c0 tx=0x7f3ec4009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:39.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.179+0000 7f3ed1600640 1 -- 172.21.15.18:0/2738466416 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ecc09cd00 con 0x7f3ed4105e20 2024-04-16T17:00:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:38 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:38 smithi018 ceph-mon[31776]: Deploying daemon mds.l on smithi043 2024-04-16T17:00:39.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:38 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/808799519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:00:39.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.431+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed402d620 con 0x7f3ed4105e20 2024-04-16T17:00:39.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.432+0000 7f3ed1600640 1 -- 172.21.15.18:0/2738466416 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f3ecc09c600 con 0x7f3ed4105e20 2024-04-16T17:00:39.434 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:39.434 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-16T17:00:39.434 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 1 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.434+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3eb80a3bb0 msgr2=0x7f3eb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.434+0000 7f3edac00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3eb80a3bb0 0x7f3eb80a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ed41052c0 tx=0x7f3ec4009720 comp rx=0 tx=0).stop 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.434+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed4105e20 msgr2=0x7f3ed41a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.434+0000 7f3edac00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed4105e20 0x7f3ed41a1b30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00e970 tx=0x7f3ecc00ee30 comp rx=0 tx=0).stop 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.435+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 shutdown_connections 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.435+0000 7f3edac00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3eb80a3bb0 0x7f3eb80a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.435+0000 7f3edac00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed4105e20 0x7f3ed41a1b30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.435+0000 7f3edac00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed4105460 0x7f3ed419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.435+0000 7f3edac00640 1 --2- 172.21.15.18:0/2738466416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed4104270 0x7f3ed419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.435+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 >> 172.21.15.18:0/2738466416 conn(0x7f3ed40ff820 msgr2=0x7f3ed4108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.435+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 shutdown_connections 2024-04-16T17:00:39.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:39.435+0000 7f3edac00640 1 -- 172.21.15.18:0/2738466416 wait complete. 2024-04-16T17:00:40.171 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:39 smithi043 ceph-mon[36901]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:40.171 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:39 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/720043672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:00:40.172 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:39 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2738466416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:40.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:39 smithi022 ceph-mon[36023]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:40.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:39 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/720043672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:00:40.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:39 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2738466416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:40.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:39 smithi018 ceph-mon[31776]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:40.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:39 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/720043672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:00:40.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:39 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2738466416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:40.502 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 systemd[1]: Starting Ceph mds.l for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:00:40.842 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 podman[66683]: 2024-04-16 17:00:40.48967732 +0000 UTC m=+0.018243695 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:00:40.842 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 podman[66683]: 2024-04-16 17:00:40.6137515 +0000 UTC m=+0.142317877 container create 83bd4e8f0a300ca3e7007ad18e6c93a98b9d395672734f5cd31d39ab78dc0533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:00:40.842 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 podman[66683]: 2024-04-16 17:00:40.841640156 +0000 UTC m=+0.370206529 container init 83bd4e8f0a300ca3e7007ad18e6c93a98b9d395672734f5cd31d39ab78dc0533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-16T17:00:41.252 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 podman[66683]: 2024-04-16 17:00:40.844546967 +0000 UTC m=+0.373113325 container start 83bd4e8f0a300ca3e7007ad18e6c93a98b9d395672734f5cd31d39ab78dc0533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-16T17:00:41.253 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 ceph-mds[66702]: -- 172.21.15.43:0/3853167145 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6ae5d3a40 con 0x55a6ae5a6c00 2024-04-16T17:00:41.253 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l[66698]: starting mds.l at 2024-04-16T17:00:41.253 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 bash[66683]: 83bd4e8f0a300ca3e7007ad18e6c93a98b9d395672734f5cd31d39ab78dc0533 2024-04-16T17:00:41.253 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:00:40 smithi043 systemd[1]: Started Ceph mds.l for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:00:41.457 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 -- 172.21.15.18:0/36377878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b88109740 msgr2=0x7f5b88109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 --2- 172.21.15.18:0/36377878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b88109740 0x7f5b88109bc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009900 tx=0x7f5b7002bc30 comp rx=0 tx=0).stop 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 -- 172.21.15.18:0/36377878 shutdown_connections 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 --2- 172.21.15.18:0/36377878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b8810a100 0x7f5b88110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 --2- 172.21.15.18:0/36377878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b88109740 0x7f5b88109bc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 --2- 172.21.15.18:0/36377878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b881085a0 0x7f5b881089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 -- 172.21.15.18:0/36377878 >> 172.21.15.18:0/36377878 conn(0x7f5b880774f0 msgr2=0x7f5b88079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 -- 172.21.15.18:0/36377878 shutdown_connections 2024-04-16T17:00:41.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.677+0000 7f5b87e00640 1 -- 172.21.15.18:0/36377878 wait complete. 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.678+0000 7f5b87e00640 1 Processor -- start 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.678+0000 7f5b87e00640 1 -- start start 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.678+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b881085a0 0x7f5b8819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.678+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b88109740 0x7f5b8819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.678+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b8810a100 0x7f5b881a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.678+0000 7f5b87e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5b881141e0 con 0x7f5b8810a100 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.678+0000 7f5b87e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5b88114060 con 0x7f5b881085a0 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.678+0000 7f5b87e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b88114360 con 0x7f5b88109740 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b86a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b881085a0 0x7f5b8819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b87400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b8810a100 0x7f5b881a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:41.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b86a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b881085a0 0x7f5b8819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53388/0 (socket says 172.21.15.18:53388) 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b86a00640 1 -- 172.21.15.18:0/791076077 learned_addr learned my addr 172.21.15.18:0/791076077 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b7fe00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b88109740 0x7f5b8819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b87400640 1 -- 172.21.15.18:0/791076077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b88109740 msgr2=0x7f5b8819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b87400640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b88109740 0x7f5b8819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b87400640 1 -- 172.21.15.18:0/791076077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b881085a0 msgr2=0x7f5b8819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b87400640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b881085a0 0x7f5b8819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.679+0000 7f5b87400640 1 -- 172.21.15.18:0/791076077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b881a6450 con 0x7f5b8810a100 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.680+0000 7f5b87400640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b8810a100 0x7f5b881a5de0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000ec50 tx=0x7f5b8000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.680+0000 7f5b7f400640 1 -- 172.21.15.18:0/791076077 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80003b70 con 0x7f5b8810a100 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.680+0000 7f5b7f400640 1 -- 172.21.15.18:0/791076077 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b80003d10 con 0x7f5b8810a100 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.680+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b881a6740 con 0x7f5b8810a100 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.680+0000 7f5b7f400640 1 -- 172.21.15.18:0/791076077 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80010890 con 0x7f5b8810a100 2024-04-16T17:00:41.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.680+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b881a6c10 con 0x7f5b8810a100 2024-04-16T17:00:41.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.681+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8810f870 con 0x7f5b8810a100 2024-04-16T17:00:41.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.683+0000 7f5b7f400640 1 -- 172.21.15.18:0/791076077 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b80003fc0 con 0x7f5b8810a100 2024-04-16T17:00:41.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.685+0000 7f5b7f400640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5b6c0a3ba0 0x7f5b6c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:41.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.685+0000 7f5b7f400640 1 -- 172.21.15.18:0/791076077 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5b800d0580 con 0x7f5b8810a100 2024-04-16T17:00:41.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.685+0000 7f5b86a00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5b6c0a3ba0 0x7f5b6c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:41.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.686+0000 7f5b86a00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5b6c0a3ba0 0x7f5b6c0a6060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b780058c0 tx=0x7f5b78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:41.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.690+0000 7f5b7f400640 1 -- 172.21.15.18:0/791076077 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b8009d360 con 0x7f5b8810a100 2024-04-16T17:00:41.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.944+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b8802d620 con 0x7f5b8810a100 2024-04-16T17:00:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.945+0000 7f5b7f400640 1 -- 172.21.15.18:0/791076077 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f5b800bbe20 con 0x7f5b8810a100 2024-04-16T17:00:41.947 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:41.947 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T17:00:41.947 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 2 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5b6c0a3ba0 msgr2=0x7f5b6c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5b6c0a3ba0 0x7f5b6c0a6060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b780058c0 tx=0x7f5b78005f90 comp rx=0 tx=0).stop 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b8810a100 msgr2=0x7f5b881a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b8810a100 0x7f5b881a5de0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000ec50 tx=0x7f5b8000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 shutdown_connections 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f5b6c0a3ba0 0x7f5b6c0a6060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b8810a100 0x7f5b881a5de0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b88109740 0x7f5b8819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 --2- 172.21.15.18:0/791076077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b881085a0 0x7f5b8819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 >> 172.21.15.18:0/791076077 conn(0x7f5b880774f0 msgr2=0x7f5b8810ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 shutdown_connections 2024-04-16T17:00:41.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:41.948+0000 7f5b87e00640 1 -- 172.21.15.18:0/791076077 wait complete. 2024-04-16T17:00:42.137 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.43:6832/3903003819,v1:172.21.15.43:6833/3903003819] up:boot 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: fsmap 1 up:standby 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:42.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:41 smithi022 ceph-mon[36023]: Deploying daemon mds.k on smithi022 2024-04-16T17:00:42.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.43:6832/3903003819,v1:172.21.15.43:6833/3903003819] up:boot 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: fsmap 1 up:standby 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:42.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:41 smithi018 ceph-mon[31776]: Deploying daemon mds.k on smithi022 2024-04-16T17:00:42.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:42.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.43:6832/3903003819,v1:172.21.15.43:6833/3903003819] up:boot 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: fsmap 1 up:standby 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:42.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:41 smithi043 ceph-mon[36901]: Deploying daemon mds.k on smithi022 2024-04-16T17:00:43.091 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:42 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/791076077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:43.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:42 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/791076077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:43.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:42 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/791076077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:43.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:44.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:43 smithi022 ceph-mon[36023]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:44.177 INFO:journalctl@ceph.mds.k.smithi022.stdout:Apr 16 17:00:44 smithi022 systemd[1]: Started Ceph mds.k for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:00:44.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:43 smithi018 ceph-mon[31776]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:44.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.211+0000 7ff43f400640 1 -- 172.21.15.18:0/3666342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff438103c70 msgr2=0x7ff43810a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:44.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.211+0000 7ff43f400640 1 --2- 172.21.15.18:0/3666342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff438103c70 0x7ff43810a500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff430009ca0 tx=0x7ff43002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.211+0000 7ff43f400640 1 -- 172.21.15.18:0/3666342 shutdown_connections 2024-04-16T17:00:44.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.211+0000 7ff43f400640 1 --2- 172.21.15.18:0/3666342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff438103c70 0x7ff43810a500 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.211+0000 7ff43f400640 1 --2- 172.21.15.18:0/3666342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4381032b0 0x7ff438103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.211+0000 7ff43f400640 1 --2- 172.21.15.18:0/3666342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4381020c0 0x7ff4381024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.211+0000 7ff43f400640 1 -- 172.21.15.18:0/3666342 >> 172.21.15.18:0/3666342 conn(0x7ff4380fd630 msgr2=0x7ff4380ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:44.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.213+0000 7ff43f400640 1 -- 172.21.15.18:0/3666342 shutdown_connections 2024-04-16T17:00:44.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.213+0000 7ff43f400640 1 -- 172.21.15.18:0/3666342 wait complete. 2024-04-16T17:00:44.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43f400640 1 Processor -- start 2024-04-16T17:00:44.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43f400640 1 -- start start 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4381020c0 0x7ff43819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4381032b0 0x7ff43819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff438103c70 0x7ff4381a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43f400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff43810dce0 con 0x7ff438103c70 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43f400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff43810db60 con 0x7ff4381032b0 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43f400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff43810de60 con 0x7ff4381020c0 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff438103c70 0x7ff4381a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff437e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4381032b0 0x7ff43819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff438103c70 0x7ff4381a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58208/0 (socket says 172.21.15.18:58208) 2024-04-16T17:00:44.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff437e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4381032b0 0x7ff43819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53404/0 (socket says 172.21.15.18:53404) 2024-04-16T17:00:44.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43d400640 1 -- 172.21.15.18:0/2183940500 learned_addr learned my addr 172.21.15.18:0/2183940500 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:44.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.214+0000 7ff43ca00640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4381020c0 0x7ff43819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.215+0000 7ff437e00640 1 -- 172.21.15.18:0/2183940500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4381020c0 msgr2=0x7ff43819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.215+0000 7ff437e00640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4381020c0 0x7ff43819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.215+0000 7ff437e00640 1 -- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff438103c70 msgr2=0x7ff4381a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.215+0000 7ff437e00640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff438103c70 0x7ff4381a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.215+0000 7ff437e00640 1 -- 172.21.15.18:0/2183940500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4381a2110 con 0x7ff4381032b0 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.215+0000 7ff437e00640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4381032b0 0x7ff43819aa30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff42400d720 tx=0x7ff42400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.215+0000 7ff435600640 1 -- 172.21.15.18:0/2183940500 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424017070 con 0x7ff4381032b0 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.216+0000 7ff435600640 1 -- 172.21.15.18:0/2183940500 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42400de30 con 0x7ff4381032b0 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.216+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4381a2400 con 0x7ff4381032b0 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.216+0000 7ff435600640 1 -- 172.21.15.18:0/2183940500 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4240047a0 con 0x7ff4381032b0 2024-04-16T17:00:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.216+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381a2950 con 0x7ff4381032b0 2024-04-16T17:00:44.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.217+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4381a2bb0 con 0x7ff4381032b0 2024-04-16T17:00:44.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.219+0000 7ff435600640 1 -- 172.21.15.18:0/2183940500 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff4240099d0 con 0x7ff4381032b0 2024-04-16T17:00:44.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.221+0000 7ff435600640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff4200a3c30 0x7ff4200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:44.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.221+0000 7ff435600640 1 -- 172.21.15.18:0/2183940500 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff4240cfb40 con 0x7ff4381032b0 2024-04-16T17:00:44.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.221+0000 7ff43ca00640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff4200a3c30 0x7ff4200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:44.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.222+0000 7ff43ca00640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff4200a3c30 0x7ff4200a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0095f0 tx=0x7ff42c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:44.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.225+0000 7ff435600640 1 -- 172.21.15.18:0/2183940500 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff424020020 con 0x7ff4381032b0 2024-04-16T17:00:44.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:43 smithi043 ceph-mon[36901]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:44.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.478+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4381a2d80 con 0x7ff4381032b0 2024-04-16T17:00:44.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.479+0000 7ff435600640 1 -- 172.21.15.18:0/2183940500 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7ff42409c920 con 0x7ff4381032b0 2024-04-16T17:00:44.481 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:44.481 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T17:00:44.481 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 2 2024-04-16T17:00:44.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff4200a3c30 msgr2=0x7ff4200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:44.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff4200a3c30 0x7ff4200a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0095f0 tx=0x7ff42c005f90 comp rx=0 tx=0).stop 2024-04-16T17:00:44.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4381032b0 msgr2=0x7ff43819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:44.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4381032b0 0x7ff43819aa30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff42400d720 tx=0x7ff42400dbe0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 shutdown_connections 2024-04-16T17:00:44.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff4200a3c30 0x7ff4200a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff438103c70 0x7ff4381a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4381032b0 0x7ff43819aa30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 --2- 172.21.15.18:0/2183940500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4381020c0 0x7ff43819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:44.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.481+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 >> 172.21.15.18:0/2183940500 conn(0x7ff4380fd630 msgr2=0x7ff438074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:44.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.482+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 shutdown_connections 2024-04-16T17:00:44.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:44.482+0000 7ff43f400640 1 -- 172.21.15.18:0/2183940500 wait complete. 2024-04-16T17:00:45.414 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.414 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.414 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.414 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:45.414 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:45.414 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:45.414 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: Deploying daemon mds.j on smithi018 2024-04-16T17:00:45.415 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2183940500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:45.415 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6832/4293504707,v1:172.21.15.22:6833/4293504707] up:boot 2024-04-16T17:00:45.415 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: fsmap 2 up:standby 2024-04-16T17:00:45.415 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:45 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: Deploying daemon mds.j on smithi018 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2183940500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6832/4293504707,v1:172.21.15.22:6833/4293504707] up:boot 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: fsmap 2 up:standby 2024-04-16T17:00:45.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:45 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:00:45.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: Deploying daemon mds.j on smithi018 2024-04-16T17:00:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2183940500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6832/4293504707,v1:172.21.15.22:6833/4293504707] up:boot 2024-04-16T17:00:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: fsmap 2 up:standby 2024-04-16T17:00:45.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:45 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:00:46.321 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:46 smithi018 ceph-mon[31776]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:46 smithi043 ceph-mon[36901]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:46.503 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:46.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:46 smithi022 ceph-mon[36023]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:46.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 -- 172.21.15.18:0/1429994559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a410b970 msgr2=0x7f59a410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:46.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 --2- 172.21.15.18:0/1429994559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a410b970 0x7f59a410bdd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5998009900 tx=0x7f599802bc30 comp rx=0 tx=0).stop 2024-04-16T17:00:46.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 -- 172.21.15.18:0/1429994559 shutdown_connections 2024-04-16T17:00:46.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 --2- 172.21.15.18:0/1429994559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59a4112740 0x7f59a4114c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:46.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 --2- 172.21.15.18:0/1429994559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a410b970 0x7f59a410bdd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:46.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 --2- 172.21.15.18:0/1429994559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59a4109000 0x7f59a410b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:46.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 -- 172.21.15.18:0/1429994559 >> 172.21.15.18:0/1429994559 conn(0x7f59a406f010 msgr2=0x7f59a4071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:46.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 -- 172.21.15.18:0/1429994559 shutdown_connections 2024-04-16T17:00:46.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.937+0000 7f59ac000640 1 -- 172.21.15.18:0/1429994559 wait complete. 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.938+0000 7f59ac000640 1 Processor -- start 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.938+0000 7f59ac000640 1 -- start start 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59ac000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a4109000 0x7f59a41a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59ac000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59a410b970 0x7f59a41a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59ac000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59a4112740 0x7f59a41aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59ac000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41187c0 con 0x7f59a4112740 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59ac000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4118640 con 0x7f59a4109000 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59ac000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4118940 con 0x7f59a410b970 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59a9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a4109000 0x7f59a41a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59a9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a4109000 0x7f59a41a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45864/0 (socket says 172.21.15.18:45864) 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59a9600640 1 -- 172.21.15.18:0/827191070 learned_addr learned my addr 172.21.15.18:0/827191070 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59aa000640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59a4112740 0x7f59a41aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.939+0000 7f59a8c00640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59a410b970 0x7f59a41a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.940+0000 7f59a9600640 1 -- 172.21.15.18:0/827191070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59a410b970 msgr2=0x7f59a41a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.940+0000 7f59a9600640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59a410b970 0x7f59a41a3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.940+0000 7f59a9600640 1 -- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59a4112740 msgr2=0x7f59a41aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.940+0000 7f59a9600640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59a4112740 0x7f59a41aa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.940+0000 7f59a9600640 1 -- 172.21.15.18:0/827191070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a41aaa60 con 0x7f59a4109000 2024-04-16T17:00:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.940+0000 7f59a9600640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a4109000 0x7f59a41a2e40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f599000c970 tx=0x7f599000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:46.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.941+0000 7f59a2000640 1 -- 172.21.15.18:0/827191070 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990007bb0 con 0x7f59a4109000 2024-04-16T17:00:46.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.941+0000 7f59a2000640 1 -- 172.21.15.18:0/827191070 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5990007d50 con 0x7f59a4109000 2024-04-16T17:00:46.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.941+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a41aad50 con 0x7f59a4109000 2024-04-16T17:00:46.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.941+0000 7f59a2000640 1 -- 172.21.15.18:0/827191070 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599000f450 con 0x7f59a4109000 2024-04-16T17:00:46.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.941+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41ab1c0 con 0x7f59a4109000 2024-04-16T17:00:46.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.942+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a4071b10 con 0x7f59a4109000 2024-04-16T17:00:46.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.944+0000 7f59a2000640 1 -- 172.21.15.18:0/827191070 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5990005020 con 0x7f59a4109000 2024-04-16T17:00:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.946+0000 7f59a2000640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f598c0a3bb0 0x7f598c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.947+0000 7f59a2000640 1 -- 172.21.15.18:0/827191070 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f59900cfd50 con 0x7f59a4109000 2024-04-16T17:00:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.947+0000 7f59a8c00640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f598c0a3bb0 0x7f598c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:46.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.947+0000 7f59a8c00640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f598c0a3bb0 0x7f598c0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f599802c240 tx=0x7f599802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:46.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:46.950+0000 7f59a2000640 1 -- 172.21.15.18:0/827191070 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5990014030 con 0x7f59a4109000 2024-04-16T17:00:47.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.199+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59a4062c00 con 0x7f59a4109000 2024-04-16T17:00:47.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.200+0000 7f59a2000640 1 -- 172.21.15.18:0/827191070 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f599009c4e0 con 0x7f59a4109000 2024-04-16T17:00:47.202 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:47.202 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T17:00:47.202 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 3 2024-04-16T17:00:47.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.202+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f598c0a3bb0 msgr2=0x7f598c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.202+0000 7f59ac000640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f598c0a3bb0 0x7f598c0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f599802c240 tx=0x7f599802df90 comp rx=0 tx=0).stop 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.202+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a4109000 msgr2=0x7f59a41a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.202+0000 7f59ac000640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a4109000 0x7f59a41a2e40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f599000c970 tx=0x7f599000ce30 comp rx=0 tx=0).stop 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.203+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 shutdown_connections 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.203+0000 7f59ac000640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f598c0a3bb0 0x7f598c0a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.203+0000 7f59ac000640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59a4112740 0x7f59a41aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.203+0000 7f59ac000640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59a410b970 0x7f59a41a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.203+0000 7f59ac000640 1 --2- 172.21.15.18:0/827191070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59a4109000 0x7f59a41a2e40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.203+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 >> 172.21.15.18:0/827191070 conn(0x7f59a406f010 msgr2=0x7f59a410a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.203+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 shutdown_connections 2024-04-16T17:00:47.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:47.203+0000 7f59ac000640 1 -- 172.21.15.18:0/827191070 wait complete. 2024-04-16T17:00:47.845 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:00:47 smithi018 podman[69730]: 2024-04-16 17:00:47.563895326 +0000 UTC m=+0.155753560 container create a65dd7e3a5af0e5a2f1f8c4cd4e2235920d3b9c2afeb1b3c6d0a3e42404315d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:00:47.845 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:00:47 smithi018 podman[69730]: 2024-04-16 17:00:47.843233291 +0000 UTC m=+0.435091569 container init a65dd7e3a5af0e5a2f1f8c4cd4e2235920d3b9c2afeb1b3c6d0a3e42404315d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-16T17:00:48.188 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:00:47 smithi018 podman[69730]: 2024-04-16 17:00:47.853978086 +0000 UTC m=+0.445836300 container start a65dd7e3a5af0e5a2f1f8c4cd4e2235920d3b9c2afeb1b3c6d0a3e42404315d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-16T17:00:48.188 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:00:47 smithi018 ceph-mds[69749]: -- 172.21.15.18:0/1643281183 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599afbd3a40 con 0x5599afba6800 2024-04-16T17:00:48.188 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:00:47 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j[69745]: starting mds.j at 2024-04-16T17:00:48.188 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:00:47 smithi018 bash[69730]: a65dd7e3a5af0e5a2f1f8c4cd4e2235920d3b9c2afeb1b3c6d0a3e42404315d9 2024-04-16T17:00:48.188 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:00:47 smithi018 systemd[1]: Started Ceph mds.j for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:00:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:48 smithi018 ceph-mon[31776]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:48 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/827191070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:48 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:48.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:48 smithi043 ceph-mon[36901]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:48.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:48 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/827191070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:48.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:48.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:48.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:48 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:48.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:48 smithi022 ceph-mon[36023]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:48.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:48 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/827191070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:48.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:48.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:48.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:48.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:48 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:49.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:49.401 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:49 smithi018 ceph-mon[31776]: Deploying daemon mds.d on smithi018 2024-04-16T17:00:49.401 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:49 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.18:6834/1011323908,v1:172.21.15.18:6835/1011323908] up:boot 2024-04-16T17:00:49.401 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:49 smithi018 ceph-mon[31776]: fsmap 3 up:standby 2024-04-16T17:00:49.401 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:49 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:00:49.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.464+0000 7f7906800640 1 -- 172.21.15.18:0/2027395092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001052f0 msgr2=0x7f79001076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:49.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.464+0000 7f7906800640 1 --2- 172.21.15.18:0/2027395092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001052f0 0x7f79001076e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009900 tx=0x7f78ec02bc30 comp rx=0 tx=0).stop 2024-04-16T17:00:49.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.465+0000 7f7906800640 1 -- 172.21.15.18:0/2027395092 shutdown_connections 2024-04-16T17:00:49.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.465+0000 7f7906800640 1 --2- 172.21.15.18:0/2027395092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7900107c20 0x7f790010a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.465+0000 7f7906800640 1 --2- 172.21.15.18:0/2027395092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001052f0 0x7f79001076e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.465+0000 7f7906800640 1 --2- 172.21.15.18:0/2027395092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79001029c0 0x7f7900104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.465+0000 7f7906800640 1 -- 172.21.15.18:0/2027395092 >> 172.21.15.18:0/2027395092 conn(0x7f79000fc3d0 msgr2=0x7f79000fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:49.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.465+0000 7f7906800640 1 -- 172.21.15.18:0/2027395092 shutdown_connections 2024-04-16T17:00:49.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.465+0000 7f7906800640 1 -- 172.21.15.18:0/2027395092 wait complete. 2024-04-16T17:00:49.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7906800640 1 Processor -- start 2024-04-16T17:00:49.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7906800640 1 -- start start 2024-04-16T17:00:49.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7906800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001029c0 0x7f790019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:49.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7906800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79001052f0 0x7f790019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:49.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7906800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7900107c20 0x7f79001a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:49.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7906800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f790010dba0 con 0x7f79001052f0 2024-04-16T17:00:49.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7906800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f790010da20 con 0x7f7900107c20 2024-04-16T17:00:49.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7906800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f790010dd20 con 0x7f79001029c0 2024-04-16T17:00:49.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.466+0000 7f7904a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79001052f0 0x7f790019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:49.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7900107c20 0x7f79001a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:49.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001029c0 0x7f790019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:49.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7904a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79001052f0 0x7f790019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34412/0 (socket says 172.21.15.18:34412) 2024-04-16T17:00:49.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7904a00640 1 -- 172.21.15.18:0/2997093655 learned_addr learned my addr 172.21.15.18:0/2997093655 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:49.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905400640 1 -- 172.21.15.18:0/2997093655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7900107c20 msgr2=0x7f79001a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:49.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905400640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7900107c20 0x7f79001a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905400640 1 -- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79001052f0 msgr2=0x7f790019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:49.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905400640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79001052f0 0x7f790019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905400640 1 -- 172.21.15.18:0/2997093655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79001a20c0 con 0x7f79001029c0 2024-04-16T17:00:49.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905e00640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7900107c20 0x7f79001a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:00:49.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.467+0000 7f7905400640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001029c0 0x7f790019a4a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78f400e9f0 tx=0x7f78f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:49.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.468+0000 7f78fe000640 1 -- 172.21.15.18:0/2997093655 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f400ccc0 con 0x7f79001029c0 2024-04-16T17:00:49.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.468+0000 7f78fe000640 1 -- 172.21.15.18:0/2997093655 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f4010910 con 0x7f79001029c0 2024-04-16T17:00:49.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.468+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79001a23b0 con 0x7f79001029c0 2024-04-16T17:00:49.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.468+0000 7f78fe000640 1 -- 172.21.15.18:0/2997093655 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4018960 con 0x7f79001029c0 2024-04-16T17:00:49.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.468+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001a28d0 con 0x7f79001029c0 2024-04-16T17:00:49.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.469+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7900109770 con 0x7f79001029c0 2024-04-16T17:00:49.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.472+0000 7f78fe000640 1 -- 172.21.15.18:0/2997093655 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f78f4010ac0 con 0x7f79001029c0 2024-04-16T17:00:49.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.473+0000 7f78fe000640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f78e80a3ba0 0x7f78e80a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:49.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.473+0000 7f78fe000640 1 -- 172.21.15.18:0/2997093655 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f78f40d0390 con 0x7f79001029c0 2024-04-16T17:00:49.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.473+0000 7f7904a00640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f78e80a3ba0 0x7f78e80a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:49.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.474+0000 7f7904a00640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f78e80a3ba0 0x7f78e80a6060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f78ec02c8e0 tx=0x7f78ec005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:49.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.475+0000 7f78fe000640 1 -- 172.21.15.18:0/2997093655 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78f409cb20 con 0x7f79001029c0 2024-04-16T17:00:49.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:49 smithi043 ceph-mon[36901]: Deploying daemon mds.d on smithi018 2024-04-16T17:00:49.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:49 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.18:6834/1011323908,v1:172.21.15.18:6835/1011323908] up:boot 2024-04-16T17:00:49.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:49 smithi043 ceph-mon[36901]: fsmap 3 up:standby 2024-04-16T17:00:49.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:49 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:00:49.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:49 smithi022 ceph-mon[36023]: Deploying daemon mds.d on smithi018 2024-04-16T17:00:49.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:49 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.18:6834/1011323908,v1:172.21.15.18:6835/1011323908] up:boot 2024-04-16T17:00:49.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:49 smithi022 ceph-mon[36023]: fsmap 3 up:standby 2024-04-16T17:00:49.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:49 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:00:49.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.744+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f790002d620 con 0x7f79001029c0 2024-04-16T17:00:49.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.745+0000 7f78fe000640 1 -- 172.21.15.18:0/2997093655 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f78f40a0e20 con 0x7f79001029c0 2024-04-16T17:00:49.747 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:49.747 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-16T17:00:49.747 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 4 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f78e80a3ba0 msgr2=0x7f78e80a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f78e80a3ba0 0x7f78e80a6060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f78ec02c8e0 tx=0x7f78ec005b50 comp rx=0 tx=0).stop 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001029c0 msgr2=0x7f790019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001029c0 0x7f790019a4a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78f400e9f0 tx=0x7f78f400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 shutdown_connections 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7900107c20 0x7f79001a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f78e80a3ba0 0x7f78e80a6060 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79001052f0 0x7f790019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 --2- 172.21.15.18:0/2997093655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79001029c0 0x7f790019a4a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 >> 172.21.15.18:0/2997093655 conn(0x7f79000fc3d0 msgr2=0x7f7900102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 shutdown_connections 2024-04-16T17:00:49.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:49.748+0000 7f7906800640 1 -- 172.21.15.18:0/2997093655 wait complete. 2024-04-16T17:00:50.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:50 smithi018 ceph-mon[31776]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:50.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:50 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2997093655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:50.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:50 smithi043 ceph-mon[36901]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:50.506 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:50 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2997093655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:50.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:50 smithi022 ceph-mon[36023]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:50.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:50 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2997093655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:51.697 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:51 smithi018 systemd[1]: Starting Ceph mds.d for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:00:51.769 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:51.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 -- 172.21.15.18:0/3754613868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 msgr2=0x7f103c1089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 --2- 172.21.15.18:0/3754613868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 0x7f103c1089c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f102c009900 tx=0x7f102c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 -- 172.21.15.18:0/3754613868 shutdown_connections 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 --2- 172.21.15.18:0/3754613868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f103c10a100 0x7f103c110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 --2- 172.21.15.18:0/3754613868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f103c109740 0x7f103c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 --2- 172.21.15.18:0/3754613868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 0x7f103c1089c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 -- 172.21.15.18:0/3754613868 >> 172.21.15.18:0/3754613868 conn(0x7f103c0774f0 msgr2=0x7f103c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 -- 172.21.15.18:0/3754613868 shutdown_connections 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.993+0000 7f1042a00640 1 -- 172.21.15.18:0/3754613868 wait complete. 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.994+0000 7f1042a00640 1 Processor -- start 2024-04-16T17:00:51.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.994+0000 7f1042a00640 1 -- start start 2024-04-16T17:00:51.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.994+0000 7f1042a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 0x7f103c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:51.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.994+0000 7f1042a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f103c109740 0x7f103c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:51.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.994+0000 7f1042a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f103c10a100 0x7f103c1a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:51.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1042a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1141e0 con 0x7f103c1085a0 2024-04-16T17:00:51.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1042a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f103c114060 con 0x7f103c109740 2024-04-16T17:00:51.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1042a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f103c114360 con 0x7f103c10a100 2024-04-16T17:00:51.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1041600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 0x7f103c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1040c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f103c109740 0x7f103c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1041600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 0x7f103c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34440/0 (socket says 172.21.15.18:34440) 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1041600640 1 -- 172.21.15.18:0/865945677 learned_addr learned my addr 172.21.15.18:0/865945677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1042000640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f103c10a100 0x7f103c1a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1040c00640 1 -- 172.21.15.18:0/865945677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f103c10a100 msgr2=0x7f103c1a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1040c00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f103c10a100 0x7f103c1a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1040c00640 1 -- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 msgr2=0x7f103c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1040c00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 0x7f103c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1040c00640 1 -- 172.21.15.18:0/865945677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c1a6430 con 0x7f103c109740 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.995+0000 7f1041600640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 0x7f103c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.996+0000 7f1040c00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f103c109740 0x7f103c19ed50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f102800e9f0 tx=0x7f102800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.996+0000 7f103a000640 1 -- 172.21.15.18:0/865945677 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102800ccc0 con 0x7f103c109740 2024-04-16T17:00:51.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.996+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c1a6720 con 0x7f103c109740 2024-04-16T17:00:52.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.996+0000 7f103a000640 1 -- 172.21.15.18:0/865945677 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1028010910 con 0x7f103c109740 2024-04-16T17:00:52.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.996+0000 7f103a000640 1 -- 172.21.15.18:0/865945677 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028018960 con 0x7f103c109740 2024-04-16T17:00:52.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.996+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1a6cc0 con 0x7f103c109740 2024-04-16T17:00:52.000 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:51 smithi018 podman[70069]: 2024-04-16 17:00:51.619892624 +0000 UTC m=+0.032969468 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:00:52.000 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:51 smithi018 podman[70069]: 2024-04-16 17:00:51.736941634 +0000 UTC m=+0.150018508 container create fe06c480a72fb72947438479df9aedeca788014f00a3c0ddc26a9a6a25f4b1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240409) 2024-04-16T17:00:52.000 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:51 smithi018 podman[70069]: 2024-04-16 17:00:51.998334371 +0000 UTC m=+0.411411213 container init fe06c480a72fb72947438479df9aedeca788014f00a3c0ddc26a9a6a25f4b1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2) 2024-04-16T17:00:52.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:51.999+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c10f8f0 con 0x7f103c109740 2024-04-16T17:00:52.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.001+0000 7f103a000640 1 -- 172.21.15.18:0/865945677 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1028010ac0 con 0x7f103c109740 2024-04-16T17:00:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.003+0000 7f103a000640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f10200a3c20 0x7f10200a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.003+0000 7f103a000640 1 -- 172.21.15.18:0/865945677 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f10280d0300 con 0x7f103c109740 2024-04-16T17:00:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.004+0000 7f1041600640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f10200a3c20 0x7f10200a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.004+0000 7f1041600640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f10200a3c20 0x7f10200a60e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f102c02c8e0 tx=0x7f102c005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.007+0000 7f103a000640 1 -- 172.21.15.18:0/865945677 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f102809ca90 con 0x7f103c109740 2024-04-16T17:00:52.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.281+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f103c02d620 con 0x7f103c109740 2024-04-16T17:00:52.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.284+0000 7f103a000640 1 -- 172.21.15.18:0/865945677 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7f10280a0d90 con 0x7f103c109740 2024-04-16T17:00:52.286 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:52.286 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-16T17:00:52.286 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 5 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f10200a3c20 msgr2=0x7f10200a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f10200a3c20 0x7f10200a60e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f102c02c8e0 tx=0x7f102c005b50 comp rx=0 tx=0).stop 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f103c109740 msgr2=0x7f103c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f103c109740 0x7f103c19ed50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f102800e9f0 tx=0x7f102800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 shutdown_connections 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f103c10a100 0x7f103c1a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f103c109740 0x7f103c19ed50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f10200a3c20 0x7f10200a60e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:52.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 --2- 172.21.15.18:0/865945677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1085a0 0x7f103c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:52.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 >> 172.21.15.18:0/865945677 conn(0x7f103c0774f0 msgr2=0x7f103c10cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:52.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 shutdown_connections 2024-04-16T17:00:52.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:52.287+0000 7f1042a00640 1 -- 172.21.15.18:0/865945677 wait complete. 2024-04-16T17:00:52.307 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:52 smithi018 ceph-mon[31776]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:52.307 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:52 smithi018 podman[70069]: 2024-04-16 17:00:52.006862001 +0000 UTC m=+0.419938838 container start fe06c480a72fb72947438479df9aedeca788014f00a3c0ddc26a9a6a25f4b1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T17:00:52.307 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:52 smithi018 ceph-mds[70127]: -- 172.21.15.18:0/1336185604 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564394435a40 con 0x564394409000 2024-04-16T17:00:52.307 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:52 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d[70102]: starting mds.d at 2024-04-16T17:00:52.307 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:52 smithi018 bash[70069]: fe06c480a72fb72947438479df9aedeca788014f00a3c0ddc26a9a6a25f4b1ac 2024-04-16T17:00:52.307 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:00:52 smithi018 systemd[1]: Started Ceph mds.d for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:00:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:52 smithi043 ceph-mon[36901]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:52.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:52 smithi022 ceph-mon[36023]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:53.292 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.18:6836/620951660,v1:172.21.15.18:6837/620951660] up:boot 2024-04-16T17:00:53.292 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: fsmap 4 up:standby 2024-04-16T17:00:53.292 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:00:53.292 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.292 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/865945677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:53.292 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.292 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.293 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:53.293 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:53.293 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:53.293 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:53 smithi022 ceph-mon[36023]: Deploying daemon mds.e on smithi022 2024-04-16T17:00:53.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.18:6836/620951660,v1:172.21.15.18:6837/620951660] up:boot 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: fsmap 4 up:standby 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/865945677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:53.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:53 smithi043 ceph-mon[36901]: Deploying daemon mds.e on smithi022 2024-04-16T17:00:53.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.18:6836/620951660,v1:172.21.15.18:6837/620951660] up:boot 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: fsmap 4 up:standby 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/865945677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:53.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:53 smithi018 ceph-mon[31776]: Deploying daemon mds.e on smithi022 2024-04-16T17:00:54.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:54.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:54 smithi022 ceph-mon[36023]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:54.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 -- 172.21.15.18:0/2679477484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8104160 msgr2=0x7fb4d8104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:54.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2679477484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8104160 0x7fb4d8104580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009860 tx=0x7fb4c802bb70 comp rx=0 tx=0).stop 2024-04-16T17:00:54.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 -- 172.21.15.18:0/2679477484 shutdown_connections 2024-04-16T17:00:54.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2679477484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4d8105d10 0x7fb4d810c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2679477484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d8105350 0x7fb4d81057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2679477484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8104160 0x7fb4d8104580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 -- 172.21.15.18:0/2679477484 >> 172.21.15.18:0/2679477484 conn(0x7fb4d80ff6d0 msgr2=0x7fb4d8101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:54.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 -- 172.21.15.18:0/2679477484 shutdown_connections 2024-04-16T17:00:54.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.542+0000 7fb4dd000640 1 -- 172.21.15.18:0/2679477484 wait complete. 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.543+0000 7fb4dd000640 1 Processor -- start 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.543+0000 7fb4dd000640 1 -- start start 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4dd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d8104160 0x7fb4d819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4dd000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4d8105350 0x7fb4d819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4dd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8105d10 0x7fb4d81a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4dd000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d810fdf0 con 0x7fb4d8105350 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4dd000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d810fc70 con 0x7fb4d8105d10 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4dd000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d810ff70 con 0x7fb4d8104160 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8105d10 0x7fb4d81a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8105d10 0x7fb4d81a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45952/0 (socket says 172.21.15.18:45952) 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d7e00640 1 -- 172.21.15.18:0/2219816224 learned_addr learned my addr 172.21.15.18:0/2219816224 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d6a00640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4d8105350 0x7fb4d819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d7e00640 1 -- 172.21.15.18:0/2219816224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d8104160 msgr2=0x7fb4d819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d7e00640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d8104160 0x7fb4d819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d7e00640 1 -- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4d8105350 msgr2=0x7fb4d819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d7e00640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4d8105350 0x7fb4d819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.544+0000 7fb4d7e00640 1 -- 172.21.15.18:0/2219816224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d81a2060 con 0x7fb4d8105d10 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.545+0000 7fb4d7e00640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8105d10 0x7fb4d81a19f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009fd0 tx=0x7fb4d000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.545+0000 7fb4cfe00640 1 -- 172.21.15.18:0/2219816224 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0004b20 con 0x7fb4d8105d10 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.545+0000 7fb4cfe00640 1 -- 172.21.15.18:0/2219816224 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d0003e50 con 0x7fb4d8105d10 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.545+0000 7fb4cfe00640 1 -- 172.21.15.18:0/2219816224 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000b630 con 0x7fb4d8105d10 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.545+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d81a2350 con 0x7fb4d8105d10 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.545+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81a2850 con 0x7fb4d8105d10 2024-04-16T17:00:54.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.546+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d810b500 con 0x7fb4d8105d10 2024-04-16T17:00:54.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.548+0000 7fb4cfe00640 1 -- 172.21.15.18:0/2219816224 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4d001d020 con 0x7fb4d8105d10 2024-04-16T17:00:54.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.549+0000 7fb4cfe00640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4bc0a3c20 0x7fb4bc0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:54.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.550+0000 7fb4cfe00640 1 -- 172.21.15.18:0/2219816224 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb4d00cffe0 con 0x7fb4d8105d10 2024-04-16T17:00:54.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.550+0000 7fb4d7400640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4bc0a3c20 0x7fb4bc0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:54.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.550+0000 7fb4d7400640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4bc0a3c20 0x7fb4bc0a60e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8037040 tx=0x7fb4c8009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:54.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.554+0000 7fb4cfe00640 1 -- 172.21.15.18:0/2219816224 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4d009cdc0 con 0x7fb4d8105d10 2024-04-16T17:00:54.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:54 smithi018 ceph-mon[31776]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:54.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:54 smithi043 ceph-mon[36901]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.800+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4d802d620 con 0x7fb4d8105d10 2024-04-16T17:00:54.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.801+0000 7fb4cfe00640 1 -- 172.21.15.18:0/2219816224 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fb4d009c6c0 con 0x7fb4d8105d10 2024-04-16T17:00:54.803 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:54.804 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-16T17:00:54.804 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 5 2024-04-16T17:00:54.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4bc0a3c20 msgr2=0x7fb4bc0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:54.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4bc0a3c20 0x7fb4bc0a60e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8037040 tx=0x7fb4c8009520 comp rx=0 tx=0).stop 2024-04-16T17:00:54.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8105d10 msgr2=0x7fb4d81a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:54.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8105d10 0x7fb4d81a19f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009fd0 tx=0x7fb4d000c800 comp rx=0 tx=0).stop 2024-04-16T17:00:54.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 shutdown_connections 2024-04-16T17:00:54.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4d8105d10 0x7fb4d81a19f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4bc0a3c20 0x7fb4bc0a60e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4d8105350 0x7fb4d819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 --2- 172.21.15.18:0/2219816224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4d8104160 0x7fb4d819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 >> 172.21.15.18:0/2219816224 conn(0x7fb4d80ff6d0 msgr2=0x7fb4d81086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:54.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.804+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 shutdown_connections 2024-04-16T17:00:54.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:54.805+0000 7fb4dd000640 1 -- 172.21.15.18:0/2219816224 wait complete. 2024-04-16T17:00:54.861 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:54 smithi022 systemd[1]: Starting Ceph mds.e for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:00:54.862 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:54 smithi022 podman[65711]: 2024-04-16 17:00:54.763737534 +0000 UTC m=+0.018403836 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:00:55.112 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:54 smithi022 podman[65711]: 2024-04-16 17:00:54.865007057 +0000 UTC m=+0.119673340 container create 596e7819f6cb225811acf23bd89e15d21acf167f37b0fe6ff34d7516a34de00f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409) 2024-04-16T17:00:55.112 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:55 smithi022 podman[65711]: 2024-04-16 17:00:55.084400422 +0000 UTC m=+0.339066717 container init 596e7819f6cb225811acf23bd89e15d21acf167f37b0fe6ff34d7516a34de00f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-16T17:00:55.112 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:55 smithi022 podman[65711]: 2024-04-16 17:00:55.087080859 +0000 UTC m=+0.341747152 container start 596e7819f6cb225811acf23bd89e15d21acf167f37b0fe6ff34d7516a34de00f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2) 2024-04-16T17:00:55.113 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-mds[65730]: -- 172.21.15.22:0/2386575760 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea77111a40 con 0x55ea770e4c00 2024-04-16T17:00:55.425 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e[65726]: starting mds.e at 2024-04-16T17:00:55.425 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:55 smithi022 bash[65711]: 596e7819f6cb225811acf23bd89e15d21acf167f37b0fe6ff34d7516a34de00f 2024-04-16T17:00:55.425 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:00:55 smithi022 systemd[1]: Started Ceph mds.e for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:00:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2219816224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:55.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:55.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:55 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:55.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2219816224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:55.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:55.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:55.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:55 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:55.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2219816224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:55.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:55.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:55.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:55.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:55 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:56.600 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:56 smithi018 ceph-mon[31776]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:56.600 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:56 smithi018 ceph-mon[31776]: Deploying daemon mds.f on smithi043 2024-04-16T17:00:56.600 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:56 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6834/3315767064,v1:172.21.15.22:6835/3315767064] up:boot 2024-04-16T17:00:56.600 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:56 smithi018 ceph-mon[31776]: fsmap 5 up:standby 2024-04-16T17:00:56.600 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:56 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:00:56.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:56 smithi022 ceph-mon[36023]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:56.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:56 smithi022 ceph-mon[36023]: Deploying daemon mds.f on smithi043 2024-04-16T17:00:56.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:56 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6834/3315767064,v1:172.21.15.22:6835/3315767064] up:boot 2024-04-16T17:00:56.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:56 smithi022 ceph-mon[36023]: fsmap 5 up:standby 2024-04-16T17:00:56.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:56 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:00:56.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:56 smithi043 ceph-mon[36901]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:56.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:56 smithi043 ceph-mon[36901]: Deploying daemon mds.f on smithi043 2024-04-16T17:00:56.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:56 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6834/3315767064,v1:172.21.15.22:6835/3315767064] up:boot 2024-04-16T17:00:56.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:56 smithi043 ceph-mon[36901]: fsmap 5 up:standby 2024-04-16T17:00:56.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:56 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:00:56.826 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:57.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 -- 172.21.15.18:0/1850667423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a01032b0 msgr2=0x7f85a0103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:57.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 --2- 172.21.15.18:0/1850667423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a01032b0 0x7f85a0103730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8588009860 tx=0x7f858802bb70 comp rx=0 tx=0).stop 2024-04-16T17:00:57.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 -- 172.21.15.18:0/1850667423 shutdown_connections 2024-04-16T17:00:57.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 --2- 172.21.15.18:0/1850667423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0103c70 0x7f85a010a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 --2- 172.21.15.18:0/1850667423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a01032b0 0x7f85a0103730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 --2- 172.21.15.18:0/1850667423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a01020c0 0x7f85a01024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 -- 172.21.15.18:0/1850667423 >> 172.21.15.18:0/1850667423 conn(0x7f85a00fd630 msgr2=0x7f85a00ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:57.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 -- 172.21.15.18:0/1850667423 shutdown_connections 2024-04-16T17:00:57.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.046+0000 7f85a7000640 1 -- 172.21.15.18:0/1850667423 wait complete. 2024-04-16T17:00:57.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.047+0000 7f85a7000640 1 Processor -- start 2024-04-16T17:00:57.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f85a7000640 1 -- start start 2024-04-16T17:00:57.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f85a7000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a01020c0 0x7f85a019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f85a7000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a01032b0 0x7f85a019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f85a7000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a0103c70 0x7f85a01a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f85a7000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010dce0 con 0x7f85a01032b0 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f85a7000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010db60 con 0x7f85a0103c70 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f85a7000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010de60 con 0x7f85a01020c0 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f8597e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a01032b0 0x7f85a019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f8597e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a01032b0 0x7f85a019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60598/0 (socket says 172.21.15.18:60598) 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f8597e00640 1 -- 172.21.15.18:0/257930620 learned_addr learned my addr 172.21.15.18:0/257930620 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f859fe00640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a01020c0 0x7f85a019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.048+0000 7f85a5000640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a0103c70 0x7f85a01a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f8597e00640 1 -- 172.21.15.18:0/257930620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a01020c0 msgr2=0x7f85a019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f8597e00640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a01020c0 0x7f85a019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f8597e00640 1 -- 172.21.15.18:0/257930620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a0103c70 msgr2=0x7f85a01a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:57.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f8597e00640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a0103c70 0x7f85a01a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f8597e00640 1 -- 172.21.15.18:0/257930620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a01a2140 con 0x7f85a01032b0 2024-04-16T17:00:57.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f8597e00640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a01032b0 0x7f85a019aa60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85880349e0 tx=0x7f8588032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:57.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f859d600640 1 -- 172.21.15.18:0/257930620 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588041030 con 0x7f85a01032b0 2024-04-16T17:00:57.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a01a23d0 con 0x7f85a01032b0 2024-04-16T17:00:57.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f859d600640 1 -- 172.21.15.18:0/257930620 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f858802c510 con 0x7f85a01032b0 2024-04-16T17:00:57.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.049+0000 7f859d600640 1 -- 172.21.15.18:0/257930620 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588039040 con 0x7f85a01032b0 2024-04-16T17:00:57.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.050+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a2910 con 0x7f85a01032b0 2024-04-16T17:00:57.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.051+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a01a2b70 con 0x7f85a01032b0 2024-04-16T17:00:57.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.053+0000 7f859d600640 1 -- 172.21.15.18:0/257930620 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8588044070 con 0x7f85a01032b0 2024-04-16T17:00:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.054+0000 7f859d600640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f85840a3c30 0x7f85840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.054+0000 7f859d600640 1 -- 172.21.15.18:0/257930620 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f85880f1b70 con 0x7f85a01032b0 2024-04-16T17:00:57.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.055+0000 7f859fe00640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f85840a3c30 0x7f85840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:57.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.055+0000 7f859fe00640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f85840a3c30 0x7f85840a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85a0103110 tx=0x7f8590009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:57.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.057+0000 7f859d600640 1 -- 172.21.15.18:0/257930620 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85880bea70 con 0x7f85a01032b0 2024-04-16T17:00:57.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.305+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85a01a2d40 con 0x7f85a01032b0 2024-04-16T17:00:57.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.306+0000 7f859d600640 1 -- 172.21.15.18:0/257930620 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f85880be370 con 0x7f85a01032b0 2024-04-16T17:00:57.308 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:57.308 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-16T17:00:57.308 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 6 2024-04-16T17:00:57.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.308+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f85840a3c30 msgr2=0x7f85840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:57.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.308+0000 7f85a7000640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f85840a3c30 0x7f85840a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85a0103110 tx=0x7f8590009260 comp rx=0 tx=0).stop 2024-04-16T17:00:57.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.308+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a01032b0 msgr2=0x7f85a019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:57.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.308+0000 7f85a7000640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a01032b0 0x7f85a019aa60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85880349e0 tx=0x7f8588032450 comp rx=0 tx=0).stop 2024-04-16T17:00:57.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.309+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 shutdown_connections 2024-04-16T17:00:57.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.309+0000 7f85a7000640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85a0103c70 0x7f85a01a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.309+0000 7f85a7000640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f85840a3c30 0x7f85840a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.309+0000 7f85a7000640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a01032b0 0x7f85a019aa60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.309+0000 7f85a7000640 1 --2- 172.21.15.18:0/257930620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a01020c0 0x7f85a019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:57.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.309+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 >> 172.21.15.18:0/257930620 conn(0x7f85a00fd630 msgr2=0x7f85a0074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:57.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.309+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 shutdown_connections 2024-04-16T17:00:57.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:57.309+0000 7f85a7000640 1 -- 172.21.15.18:0/257930620 wait complete. 2024-04-16T17:00:57.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:57 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/257930620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:57.587 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:57 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/257930620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:57.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:57 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/257930620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:00:58.481 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:58 smithi043 ceph-mon[36901]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:58.481 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:00:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f[67009]: starting mds.f at 2024-04-16T17:00:58.481 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:00:58 smithi043 bash[66994]: b8f38de83cbb10ebe0cf61b1904699445c2b1ab0cf2aff1e39a93cab633af6c7 2024-04-16T17:00:58.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:58 smithi022 ceph-mon[36023]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:58.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:58 smithi018 ceph-mon[31776]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:00:58.752 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:00:58 smithi043 systemd[1]: Started Ceph mds.f for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:00:59.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:00:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 -- 172.21.15.18:0/652641473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c10a100 msgr2=0x7f643c110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 --2- 172.21.15.18:0/652641473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c10a100 0x7f643c110990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6434009900 tx=0x7f643402bc40 comp rx=0 tx=0).stop 2024-04-16T17:00:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 -- 172.21.15.18:0/652641473 shutdown_connections 2024-04-16T17:00:59.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 --2- 172.21.15.18:0/652641473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c10a100 0x7f643c110990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 --2- 172.21.15.18:0/652641473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f643c109740 0x7f643c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 --2- 172.21.15.18:0/652641473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c1085a0 0x7f643c1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 -- 172.21.15.18:0/652641473 >> 172.21.15.18:0/652641473 conn(0x7f643c0774f0 msgr2=0x7f643c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:59.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 -- 172.21.15.18:0/652641473 shutdown_connections 2024-04-16T17:00:59.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.566+0000 7f6441200640 1 -- 172.21.15.18:0/652641473 wait complete. 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.567+0000 7f6441200640 1 Processor -- start 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.567+0000 7f6441200640 1 -- start start 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f6441200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f643c1085a0 0x7f643c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f6441200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c109740 0x7f643c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f6441200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c10a100 0x7f643c1a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f6441200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1141b0 con 0x7f643c1085a0 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f6441200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f643c114030 con 0x7f643c10a100 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f6441200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f643c114330 con 0x7f643c109740 2024-04-16T17:00:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f643b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f643c1085a0 0x7f643c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f643aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c109740 0x7f643c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f643b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f643c1085a0 0x7f643c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60614/0 (socket says 172.21.15.18:60614) 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f643b400640 1 -- 172.21.15.18:0/1419780662 learned_addr learned my addr 172.21.15.18:0/1419780662 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.568+0000 7f643be00640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c10a100 0x7f643c1a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f643aa00640 1 -- 172.21.15.18:0/1419780662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c10a100 msgr2=0x7f643c1a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f643aa00640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c10a100 0x7f643c1a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f643aa00640 1 -- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f643c1085a0 msgr2=0x7f643c19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f643aa00640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f643c1085a0 0x7f643c19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f643aa00640 1 -- 172.21.15.18:0/1419780662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c1a63b0 con 0x7f643c109740 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f643b400640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f643c1085a0 0x7f643c19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f643aa00640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c109740 0x7f643c19ecd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9f0 tx=0x7f642400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f6433e00640 1 -- 172.21.15.18:0/1419780662 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642400ccc0 con 0x7f643c109740 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c1a66a0 con 0x7f643c109740 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f6433e00640 1 -- 172.21.15.18:0/1419780662 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6424010910 con 0x7f643c109740 2024-04-16T17:00:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f6433e00640 1 -- 172.21.15.18:0/1419780662 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424018960 con 0x7f643c109740 2024-04-16T17:00:59.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.569+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a6bf0 con 0x7f643c109740 2024-04-16T17:00:59.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.570+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c10f810 con 0x7f643c109740 2024-04-16T17:00:59.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.572+0000 7f6433e00640 1 -- 172.21.15.18:0/1419780662 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6424010ac0 con 0x7f643c109740 2024-04-16T17:00:59.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.573+0000 7f6433e00640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f64200a3ba0 0x7f64200a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:59.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.574+0000 7f6433e00640 1 -- 172.21.15.18:0/1419780662 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f64240d0340 con 0x7f643c109740 2024-04-16T17:00:59.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.574+0000 7f643b400640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f64200a3ba0 0x7f64200a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:59.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.574+0000 7f643b400640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f64200a3ba0 0x7f64200a6060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f642c011750 tx=0x7f642c0096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.578+0000 7f6433e00640 1 -- 172.21.15.18:0/1419780662 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f642409cad0 con 0x7f643c109740 2024-04-16T17:00:59.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: Deploying daemon mds.c on smithi043 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.43:6834/1265446335,v1:172.21.15.43:6835/1265446335] up:boot 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: fsmap 6 up:standby 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:00:59.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:00:59 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:00:59.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.824+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f643c02d620 con 0x7f643c109740 2024-04-16T17:00:59.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.825+0000 7f6433e00640 1 -- 172.21.15.18:0/1419780662 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f64240a0dd0 con 0x7f643c109740 2024-04-16T17:00:59.826 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:00:59.826 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-16T17:00:59.826 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 7 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f64200a3ba0 msgr2=0x7f64200a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f64200a3ba0 0x7f64200a6060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f642c011750 tx=0x7f642c0096d0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c109740 msgr2=0x7f643c19ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c109740 0x7f643c19ecd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9f0 tx=0x7f642400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 shutdown_connections 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c10a100 0x7f643c1a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f643c109740 0x7f643c19ecd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f64200a3ba0 0x7f64200a6060 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 --2- 172.21.15.18:0/1419780662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f643c1085a0 0x7f643c19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:59.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 >> 172.21.15.18:0/1419780662 conn(0x7f643c0774f0 msgr2=0x7f643c10c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:59.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.828+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 shutdown_connections 2024-04-16T17:00:59.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:00:59.829+0000 7f6441200640 1 -- 172.21.15.18:0/1419780662 wait complete. 2024-04-16T17:00:59.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.842 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:59.842 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:59.842 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:59.842 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: Deploying daemon mds.c on smithi043 2024-04-16T17:00:59.842 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.43:6834/1265446335,v1:172.21.15.43:6835/1265446335] up:boot 2024-04-16T17:00:59.842 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: fsmap 6 up:standby 2024-04-16T17:00:59.842 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:00:59.842 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:00:59 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: Deploying daemon mds.c on smithi043 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.43:6834/1265446335,v1:172.21.15.43:6835/1265446335] up:boot 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: fsmap 6 up:standby 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:00:59.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:00:59 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:01:00.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:00 smithi043 ceph-mon[36901]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:00.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:00 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1419780662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:00.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:00 smithi022 ceph-mon[36023]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:00.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:00 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1419780662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:00.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:00 smithi018 ceph-mon[31776]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:00.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:00 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1419780662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:01.002 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:00 smithi043 systemd[1]: Starting Ceph mds.c for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:01:01.422 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:01 smithi043 podman[67313]: 2024-04-16 17:01:01.05358323 +0000 UTC m=+0.018349265 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:01:01.422 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:01 smithi043 podman[67313]: 2024-04-16 17:01:01.167406685 +0000 UTC m=+0.132172740 container create 66e397f7e4e148a97ea2f05510ea9b6939fb292040d4ee89ac47838725bfc931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:01:01.422 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:01 smithi043 podman[67313]: 2024-04-16 17:01:01.395297408 +0000 UTC m=+0.360063445 container init 66e397f7e4e148a97ea2f05510ea9b6939fb292040d4ee89ac47838725bfc931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=) 2024-04-16T17:01:01.422 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:01 smithi043 podman[67313]: 2024-04-16 17:01:01.398019184 +0000 UTC m=+0.362785221 container start 66e397f7e4e148a97ea2f05510ea9b6939fb292040d4ee89ac47838725bfc931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240409) 2024-04-16T17:01:01.681 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:01 smithi043 ceph-mds[67332]: -- 172.21.15.43:0/466455406 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600b2ec1a40 con 0x5600b2e94800 2024-04-16T17:01:01.681 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c[67328]: starting mds.c at 2024-04-16T17:01:01.681 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:01 smithi043 bash[67313]: 66e397f7e4e148a97ea2f05510ea9b6939fb292040d4ee89ac47838725bfc931 2024-04-16T17:01:01.681 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:01:01 smithi043 systemd[1]: Started Ceph mds.c for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:01:01.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:02.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 -- 172.21.15.18:0/2687157423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 msgr2=0x7f9290108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:02.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 --2- 172.21.15.18:0/2687157423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 0x7f9290108aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9280009860 tx=0x7f928002bb70 comp rx=0 tx=0).stop 2024-04-16T17:01:02.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 -- 172.21.15.18:0/2687157423 shutdown_connections 2024-04-16T17:01:02.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 --2- 172.21.15.18:0/2687157423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f929010a230 0x7f9290110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 --2- 172.21.15.18:0/2687157423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9290109870 0x7f9290109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 --2- 172.21.15.18:0/2687157423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 0x7f9290108aa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 -- 172.21.15.18:0/2687157423 >> 172.21.15.18:0/2687157423 conn(0x7f9290077650 msgr2=0x7f9290079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:02.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 -- 172.21.15.18:0/2687157423 shutdown_connections 2024-04-16T17:01:02.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.083+0000 7f9294a00640 1 -- 172.21.15.18:0/2687157423 wait complete. 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.084+0000 7f9294a00640 1 Processor -- start 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.084+0000 7f9294a00640 1 -- start start 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9294a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 0x7f929019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9294a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9290109870 0x7f929019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9294a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f929010a230 0x7f92901a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9294a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f92901142d0 con 0x7f9290109870 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9294a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9290114150 con 0x7f9290108680 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9294a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9290114450 con 0x7f929010a230 2024-04-16T17:01:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9287e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9290109870 0x7f929019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f928de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 0x7f929019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9287e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9290109870 0x7f929019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60622/0 (socket says 172.21.15.18:60622) 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f9287e00640 1 -- 172.21.15.18:0/226458458 learned_addr learned my addr 172.21.15.18:0/226458458 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f928de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 0x7f929019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45636/0 (socket says 172.21.15.18:45636) 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.085+0000 7f928e800640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f929010a230 0x7f92901a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f928de00640 1 -- 172.21.15.18:0/226458458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f929010a230 msgr2=0x7f92901a5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f928de00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f929010a230 0x7f92901a5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f928de00640 1 -- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9290109870 msgr2=0x7f929019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f928de00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9290109870 0x7f929019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f928de00640 1 -- 172.21.15.18:0/226458458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92901a65c0 con 0x7f9290108680 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f9287e00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9290109870 0x7f929019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f928de00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 0x7f929019e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9280009780 tx=0x7f9280032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f9286a00640 1 -- 172.21.15.18:0/226458458 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280041020 con 0x7f9290108680 2024-04-16T17:01:02.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f9286a00640 1 -- 172.21.15.18:0/226458458 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f928002c9f0 con 0x7f9290108680 2024-04-16T17:01:02.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92901a6850 con 0x7f9290108680 2024-04-16T17:01:02.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.086+0000 7f9286a00640 1 -- 172.21.15.18:0/226458458 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280039040 con 0x7f9290108680 2024-04-16T17:01:02.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.087+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92901a6d10 con 0x7f9290108680 2024-04-16T17:01:02.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.088+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929010fa20 con 0x7f9290108680 2024-04-16T17:01:02.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.089+0000 7f9286a00640 1 -- 172.21.15.18:0/226458458 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9280044020 con 0x7f9290108680 2024-04-16T17:01:02.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.091+0000 7f9286a00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f92740a3bb0 0x7f92740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:02.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.091+0000 7f9286a00640 1 -- 172.21.15.18:0/226458458 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f92800f17f0 con 0x7f9290108680 2024-04-16T17:01:02.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.091+0000 7f9287e00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f92740a3bb0 0x7f92740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:02.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.092+0000 7f9287e00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f92740a3bb0 0x7f92740a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f92780096c0 tx=0x7f9278009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:02.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.096+0000 7f9286a00640 1 -- 172.21.15.18:0/226458458 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f928004b020 con 0x7f9290108680 2024-04-16T17:01:02.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.347+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f929002d620 con 0x7f9290108680 2024-04-16T17:01:02.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.348+0000 7f9286a00640 1 -- 172.21.15.18:0/226458458 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7f92800be5d0 con 0x7f9290108680 2024-04-16T17:01:02.349 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:02.349 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-16T17:01:02.349 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 8 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f92740a3bb0 msgr2=0x7f92740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f92740a3bb0 0x7f92740a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f92780096c0 tx=0x7f9278009260 comp rx=0 tx=0).stop 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 msgr2=0x7f929019e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 0x7f929019e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9280009780 tx=0x7f9280032430 comp rx=0 tx=0).stop 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 shutdown_connections 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f929010a230 0x7f92901a5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f92740a3bb0 0x7f92740a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9290109870 0x7f929019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 --2- 172.21.15.18:0/226458458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9290108680 0x7f929019e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 >> 172.21.15.18:0/226458458 conn(0x7f9290077650 msgr2=0x7f929010cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 shutdown_connections 2024-04-16T17:01:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:02.351+0000 7f9294a00640 1 -- 172.21.15.18:0/226458458 wait complete. 2024-04-16T17:01:02.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:02.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:02.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:02.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:02.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:02.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:02.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:02.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: Deploying daemon mds.h on smithi022 2024-04-16T17:01:02.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.43:6836/1790976373,v1:172.21.15.43:6837/1790976373] up:boot 2024-04-16T17:01:02.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: fsmap 7 up:standby 2024-04-16T17:01:02.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:01:02.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:02 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/226458458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: Deploying daemon mds.h on smithi022 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.43:6836/1790976373,v1:172.21.15.43:6837/1790976373] up:boot 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: fsmap 7 up:standby 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:01:02.896 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:02 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/226458458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:03.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:03.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: Deploying daemon mds.h on smithi022 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.43:6836/1790976373,v1:172.21.15.43:6837/1790976373] up:boot 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: fsmap 7 up:standby 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:01:03.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:02 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/226458458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:03.925 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:03 smithi022 systemd[1]: Starting Ceph mds.h for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:01:04.311 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:04 smithi022 podman[66046]: 2024-04-16 17:01:03.962413537 +0000 UTC m=+0.018121732 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:01:04.311 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:04 smithi022 podman[66046]: 2024-04-16 17:01:04.062883662 +0000 UTC m=+0.118591857 container create f0ff3c200e564d60263389e33eb24fb279aaea899ef59d205ce763c354ed1f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-16T17:01:04.312 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:04 smithi022 podman[66046]: 2024-04-16 17:01:04.307239341 +0000 UTC m=+0.362947530 container init f0ff3c200e564d60263389e33eb24fb279aaea899ef59d205ce763c354ed1f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-16T17:01:04.312 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:04 smithi022 podman[66046]: 2024-04-16 17:01:04.310731203 +0000 UTC m=+0.366439392 container start f0ff3c200e564d60263389e33eb24fb279aaea899ef59d205ce763c354ed1f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-16T17:01:04.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:04.567 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-mds[66065]: -- 172.21.15.22:0/2929446116 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570cdf69a40 con 0x5570cdf3c800 2024-04-16T17:01:04.567 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: starting mds.h at 2024-04-16T17:01:04.567 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:04 smithi022 bash[66046]: f0ff3c200e564d60263389e33eb24fb279aaea899ef59d205ce763c354ed1f47 2024-04-16T17:01:04.567 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:01:04 smithi022 systemd[1]: Started Ceph mds.h for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.600+0000 7f9e45800640 1 -- 172.21.15.18:0/2075556439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e40077c00 msgr2=0x7f9e40078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.600+0000 7f9e45800640 1 --2- 172.21.15.18:0/2075556439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e40077c00 0x7f9e40078060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009900 tx=0x7f9e2802bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.600+0000 7f9e45800640 1 -- 172.21.15.18:0/2075556439 shutdown_connections 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.600+0000 7f9e45800640 1 --2- 172.21.15.18:0/2075556439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e400785a0 0x7f9e40078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.600+0000 7f9e45800640 1 --2- 172.21.15.18:0/2075556439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e40077c00 0x7f9e40078060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.600+0000 7f9e45800640 1 --2- 172.21.15.18:0/2075556439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e400769c0 0x7f9e40076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.600+0000 7f9e45800640 1 -- 172.21.15.18:0/2075556439 >> 172.21.15.18:0/2075556439 conn(0x7f9e400ff850 msgr2=0x7f9e40101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.600+0000 7f9e45800640 1 -- 172.21.15.18:0/2075556439 shutdown_connections 2024-04-16T17:01:04.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.601+0000 7f9e45800640 1 -- 172.21.15.18:0/2075556439 wait complete. 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e45800640 1 Processor -- start 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e45800640 1 -- start start 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e45800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400769c0 0x7f9e4019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e45800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e40077c00 0x7f9e4019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e45800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e400785a0 0x7f9e401a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e45800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40114340 con 0x7f9e40077c00 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e45800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e401141c0 con 0x7f9e400785a0 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e45800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e401144c0 con 0x7f9e400769c0 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e3ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400769c0 0x7f9e4019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e3ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400769c0 0x7f9e4019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42596/0 (socket says 172.21.15.18:42596) 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.602+0000 7f9e3ea00640 1 -- 172.21.15.18:0/624786534 learned_addr learned my addr 172.21.15.18:0/624786534 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e3f400640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e400785a0 0x7f9e401a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e3e000640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e40077c00 0x7f9e4019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e3ea00640 1 -- 172.21.15.18:0/624786534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e400785a0 msgr2=0x7f9e401a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e3ea00640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e400785a0 0x7f9e401a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e3ea00640 1 -- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e40077c00 msgr2=0x7f9e4019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e3ea00640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e40077c00 0x7f9e4019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e3ea00640 1 -- 172.21.15.18:0/624786534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e401a6540 con 0x7f9e400769c0 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e3ea00640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400769c0 0x7f9e4019e920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000d950 tx=0x7f9e3000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e37400640 1 -- 172.21.15.18:0/624786534 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30017030 con 0x7f9e400769c0 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e37400640 1 -- 172.21.15.18:0/624786534 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e30004020 con 0x7f9e400769c0 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e37400640 1 -- 172.21.15.18:0/624786534 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300047a0 con 0x7f9e400769c0 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e401a6830 con 0x7f9e400769c0 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.603+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e401a6da0 con 0x7f9e400769c0 2024-04-16T17:01:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.605+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e40110da0 con 0x7f9e400769c0 2024-04-16T17:01:04.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.606+0000 7f9e37400640 1 -- 172.21.15.18:0/624786534 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e300041d0 con 0x7f9e400769c0 2024-04-16T17:01:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.608+0000 7f9e37400640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e240a3c30 0x7f9e240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.608+0000 7f9e37400640 1 -- 172.21.15.18:0/624786534 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9e300cfb00 con 0x7f9e400769c0 2024-04-16T17:01:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.608+0000 7f9e3e000640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e240a3c30 0x7f9e240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.609+0000 7f9e3e000640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e240a3c30 0x7f9e240a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e2802c240 tx=0x7f9e2802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:04.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.610+0000 7f9e37400640 1 -- 172.21.15.18:0/624786534 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e3009c960 con 0x7f9e400769c0 2024-04-16T17:01:04.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.861+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e4002d620 con 0x7f9e400769c0 2024-04-16T17:01:04.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.862+0000 7f9e37400640 1 -- 172.21.15.18:0/624786534 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7159 (secure 0 0 0) 0x7f9e300cde20 con 0x7f9e400769c0 2024-04-16T17:01:04.863 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:04.863 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-16T17:01:04.863 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 9 2024-04-16T17:01:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.864+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e240a3c30 msgr2=0x7f9e240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.864+0000 7f9e45800640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e240a3c30 0x7f9e240a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e2802c240 tx=0x7f9e2802df90 comp rx=0 tx=0).stop 2024-04-16T17:01:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.864+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400769c0 msgr2=0x7f9e4019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.864+0000 7f9e45800640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400769c0 0x7f9e4019e920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000d950 tx=0x7f9e3000de10 comp rx=0 tx=0).stop 2024-04-16T17:01:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.865+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 shutdown_connections 2024-04-16T17:01:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.865+0000 7f9e45800640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e400785a0 0x7f9e401a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.865+0000 7f9e45800640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e240a3c30 0x7f9e240a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.865+0000 7f9e45800640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e40077c00 0x7f9e4019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.865+0000 7f9e45800640 1 --2- 172.21.15.18:0/624786534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400769c0 0x7f9e4019e920 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.865+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 >> 172.21.15.18:0/624786534 conn(0x7f9e400ff850 msgr2=0x7f9e40100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.865+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 shutdown_connections 2024-04-16T17:01:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:04.865+0000 7f9e45800640 1 -- 172.21.15.18:0/624786534 wait complete. 2024-04-16T17:01:04.882 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:04 smithi018 ceph-mon[31776]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:04.882 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:04.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:04.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:04.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:04.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:04.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:04 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:04.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-mon[36023]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:04.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:04.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:04.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:04.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:04.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:04.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:04 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:05.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:04 smithi043 ceph-mon[36901]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:05.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:05.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:05.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:05.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:05.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:05.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:04 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:05.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:05 smithi043 ceph-mon[36901]: Deploying daemon mds.i on smithi043 2024-04-16T17:01:05.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:05 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6836/94342022,v1:172.21.15.22:6837/94342022] up:boot 2024-04-16T17:01:05.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:05 smithi043 ceph-mon[36901]: fsmap 8 up:standby 2024-04-16T17:01:05.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:05 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:01:05.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:05 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/624786534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:05.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:05 smithi022 ceph-mon[36023]: Deploying daemon mds.i on smithi043 2024-04-16T17:01:05.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:05 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6836/94342022,v1:172.21.15.22:6837/94342022] up:boot 2024-04-16T17:01:05.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:05 smithi022 ceph-mon[36023]: fsmap 8 up:standby 2024-04-16T17:01:05.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:05 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:01:05.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:05 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/624786534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:05.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:05 smithi018 ceph-mon[31776]: Deploying daemon mds.i on smithi043 2024-04-16T17:01:05.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:05 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6836/94342022,v1:172.21.15.22:6837/94342022] up:boot 2024-04-16T17:01:05.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:05 smithi018 ceph-mon[31776]: fsmap 8 up:standby 2024-04-16T17:01:05.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:05 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:01:05.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:05 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/624786534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:06.887 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:06.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:06 smithi022 ceph-mon[36023]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:06.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:06 smithi018 ceph-mon[31776]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:07.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:06 smithi043 ceph-mon[36901]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:07.003 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:06 smithi043 systemd[1]: Starting Ceph mds.i for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:01:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 -- 172.21.15.18:0/1398083141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 msgr2=0x7f23941089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:07.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 --2- 172.21.15.18:0/1398083141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 0x7f23941089c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2388009900 tx=0x7f238802bc50 comp rx=0 tx=0).stop 2024-04-16T17:01:07.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 -- 172.21.15.18:0/1398083141 shutdown_connections 2024-04-16T17:01:07.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 --2- 172.21.15.18:0/1398083141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f239410a100 0x7f2394110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 --2- 172.21.15.18:0/1398083141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2394109740 0x7f2394109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 --2- 172.21.15.18:0/1398083141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 0x7f23941089c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 -- 172.21.15.18:0/1398083141 >> 172.21.15.18:0/1398083141 conn(0x7f23940774f0 msgr2=0x7f2394079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:07.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 -- 172.21.15.18:0/1398083141 shutdown_connections 2024-04-16T17:01:07.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.124+0000 7f2399e00640 1 -- 172.21.15.18:0/1398083141 wait complete. 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.125+0000 7f2399e00640 1 Processor -- start 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.125+0000 7f2399e00640 1 -- start start 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.125+0000 7f2399e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 0x7f239419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.125+0000 7f2399e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2394109740 0x7f239419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2399e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f239410a100 0x7f23941a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2399e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23941141b0 con 0x7f239410a100 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2399e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2394114030 con 0x7f23941085a0 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2399e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2394114330 con 0x7f2394109740 2024-04-16T17:01:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2398a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 0x7f239419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2398a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 0x7f239419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50988/0 (socket says 172.21.15.18:50988) 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2399400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f239410a100 0x7f23941a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2398a00640 1 -- 172.21.15.18:0/1502247338 learned_addr learned my addr 172.21.15.18:0/1502247338 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2393e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2394109740 0x7f239419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2398a00640 1 -- 172.21.15.18:0/1502247338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2394109740 msgr2=0x7f239419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2398a00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2394109740 0x7f239419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2398a00640 1 -- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f239410a100 msgr2=0x7f23941a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2398a00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f239410a100 0x7f23941a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2398a00640 1 -- 172.21.15.18:0/1502247338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23941a6520 con 0x7f23941085a0 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.126+0000 7f2399400640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f239410a100 0x7f23941a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.127+0000 7f2398a00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 0x7f239419e900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c00 tx=0x7f2388005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:07.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.127+0000 7f2391600640 1 -- 172.21.15.18:0/1502247338 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388042020 con 0x7f23941085a0 2024-04-16T17:01:07.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.127+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23941a67b0 con 0x7f23941085a0 2024-04-16T17:01:07.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.127+0000 7f2391600640 1 -- 172.21.15.18:0/1502247338 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23880044d0 con 0x7f23941085a0 2024-04-16T17:01:07.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.127+0000 7f2391600640 1 -- 172.21.15.18:0/1502247338 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238803e460 con 0x7f23941085a0 2024-04-16T17:01:07.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.127+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941a6d50 con 0x7f23941085a0 2024-04-16T17:01:07.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.128+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239410f8f0 con 0x7f23941085a0 2024-04-16T17:01:07.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.130+0000 7f2391600640 1 -- 172.21.15.18:0/1502247338 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2388045020 con 0x7f23941085a0 2024-04-16T17:01:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.131+0000 7f2391600640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f237c0a3ba0 0x7f237c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.132+0000 7f2391600640 1 -- 172.21.15.18:0/1502247338 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f23880f1f90 con 0x7f23941085a0 2024-04-16T17:01:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.132+0000 7f2393e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f237c0a3ba0 0x7f237c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:07.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.132+0000 7f2393e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f237c0a3ba0 0x7f237c0a6060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f239419fe10 tx=0x7f238000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:07.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.136+0000 7f2391600640 1 -- 172.21.15.18:0/1502247338 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f238804c020 con 0x7f23941085a0 2024-04-16T17:01:07.286 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:07 smithi043 podman[67635]: 2024-04-16 17:01:06.937387577 +0000 UTC m=+0.018327541 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:01:07.287 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:07 smithi043 podman[67635]: 2024-04-16 17:01:07.049734695 +0000 UTC m=+0.130674698 container create 859a77133dd022a6cd8f7958844b21da44d29bd577d49357c5b650052e53a6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS) 2024-04-16T17:01:07.287 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:07 smithi043 podman[67635]: 2024-04-16 17:01:07.285938755 +0000 UTC m=+0.366878719 container init 859a77133dd022a6cd8f7958844b21da44d29bd577d49357c5b650052e53a6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-16T17:01:07.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.383+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2394062c00 con 0x7f23941085a0 2024-04-16T17:01:07.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.384+0000 7f2391600640 1 -- 172.21.15.18:0/1502247338 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7159 (secure 0 0 0) 0x7f23880bed70 con 0x7f23941085a0 2024-04-16T17:01:07.385 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:07.385 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-16T17:01:07.386 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 9 2024-04-16T17:01:07.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.387+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f237c0a3ba0 msgr2=0x7f237c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:07.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.387+0000 7f2399e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f237c0a3ba0 0x7f237c0a6060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f239419fe10 tx=0x7f238000b410 comp rx=0 tx=0).stop 2024-04-16T17:01:07.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.387+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 msgr2=0x7f239419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.387+0000 7f2399e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 0x7f239419e900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c00 tx=0x7f2388005d50 comp rx=0 tx=0).stop 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.388+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 shutdown_connections 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.388+0000 7f2399e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f237c0a3ba0 0x7f237c0a6060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.388+0000 7f2399e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f239410a100 0x7f23941a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.388+0000 7f2399e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2394109740 0x7f239419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.388+0000 7f2399e00640 1 --2- 172.21.15.18:0/1502247338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23941085a0 0x7f239419e900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.388+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 >> 172.21.15.18:0/1502247338 conn(0x7f23940774f0 msgr2=0x7f239410cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.388+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 shutdown_connections 2024-04-16T17:01:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:07.388+0000 7f2399e00640 1 -- 172.21.15.18:0/1502247338 wait complete. 2024-04-16T17:01:07.598 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:07 smithi043 podman[67635]: 2024-04-16 17:01:07.288980952 +0000 UTC m=+0.369920925 container start 859a77133dd022a6cd8f7958844b21da44d29bd577d49357c5b650052e53a6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T17:01:07.598 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-mds[67654]: -- 172.21.15.43:0/1871205629 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f0a339a40 con 0x563f0a30c800 2024-04-16T17:01:07.599 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i[67650]: starting mds.i at 2024-04-16T17:01:07.599 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:07 smithi043 bash[67635]: 859a77133dd022a6cd8f7958844b21da44d29bd577d49357c5b650052e53a6bc 2024-04-16T17:01:07.599 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:01:07 smithi043 systemd[1]: Started Ceph mds.i for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:01:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:07 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1502247338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:07 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:07 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1502247338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:07.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:07 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1502247338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:08.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:08.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:08.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:08.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:08.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:07 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:08.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:08 smithi043 ceph-mon[36901]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:08.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:08 smithi043 ceph-mon[36901]: Deploying daemon mds.g on smithi018 2024-04-16T17:01:08.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:08 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] up:boot 2024-04-16T17:01:08.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:08 smithi043 ceph-mon[36901]: fsmap 9 up:standby 2024-04-16T17:01:08.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:08 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:01:08.905 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:08 smithi018 ceph-mon[31776]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:08.905 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:08 smithi018 ceph-mon[31776]: Deploying daemon mds.g on smithi018 2024-04-16T17:01:08.905 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:08 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] up:boot 2024-04-16T17:01:08.905 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:08 smithi018 ceph-mon[31776]: fsmap 9 up:standby 2024-04-16T17:01:08.905 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:08 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:01:08.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:08 smithi022 ceph-mon[36023]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:08.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:08 smithi022 ceph-mon[36023]: Deploying daemon mds.g on smithi018 2024-04-16T17:01:08.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:08 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] up:boot 2024-04-16T17:01:08.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:08 smithi022 ceph-mon[36023]: fsmap 9 up:standby 2024-04-16T17:01:08.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:08 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:01:09.410 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 -- 172.21.15.18:0/2199037960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 msgr2=0x7f15501046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 --2- 172.21.15.18:0/2199037960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 0x7f15501046f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1540009900 tx=0x7f154002bc50 comp rx=0 tx=0).stop 2024-04-16T17:01:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 -- 172.21.15.18:0/2199037960 shutdown_connections 2024-04-16T17:01:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 --2- 172.21.15.18:0/2199037960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1550105e80 0x7f155010c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 --2- 172.21.15.18:0/2199037960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15501054c0 0x7f1550105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 --2- 172.21.15.18:0/2199037960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 0x7f15501046f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 -- 172.21.15.18:0/2199037960 >> 172.21.15.18:0/2199037960 conn(0x7f15500ff880 msgr2=0x7f1550101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 -- 172.21.15.18:0/2199037960 shutdown_connections 2024-04-16T17:01:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.636+0000 7f1556e00640 1 -- 172.21.15.18:0/2199037960 wait complete. 2024-04-16T17:01:09.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.637+0000 7f1556e00640 1 Processor -- start 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f1556e00640 1 -- start start 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f1556e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 0x7f155019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f1556e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15501054c0 0x7f155019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f1556e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1550105e80 0x7f15501a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f1556e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f155010fec0 con 0x7f15501042d0 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f1556e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f155010fd40 con 0x7f15501054c0 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f1556e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1550110040 con 0x7f1550105e80 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f154fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 0x7f155019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f154fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 0x7f155019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41454/0 (socket says 172.21.15.18:41454) 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f154fe00640 1 -- 172.21.15.18:0/1661503693 learned_addr learned my addr 172.21.15.18:0/1661503693 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f154f400640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15501054c0 0x7f155019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.638+0000 7f1554e00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1550105e80 0x7f15501a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154f400640 1 -- 172.21.15.18:0/1661503693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1550105e80 msgr2=0x7f15501a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154f400640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1550105e80 0x7f15501a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154f400640 1 -- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 msgr2=0x7f155019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154f400640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 0x7f155019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154f400640 1 -- 172.21.15.18:0/1661503693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15501a2240 con 0x7f15501054c0 2024-04-16T17:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154fe00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 0x7f155019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154f400640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15501054c0 0x7f155019ab60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9f0 tx=0x7f153c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154cc00640 1 -- 172.21.15.18:0/1661503693 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00ccc0 con 0x7f15501054c0 2024-04-16T17:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15501a2530 con 0x7f15501054c0 2024-04-16T17:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154cc00640 1 -- 172.21.15.18:0/1661503693 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f153c0035f0 con 0x7f15501054c0 2024-04-16T17:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.639+0000 7f154cc00640 1 -- 172.21.15.18:0/1661503693 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00ce60 con 0x7f15501054c0 2024-04-16T17:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.640+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15501a2a80 con 0x7f15501054c0 2024-04-16T17:01:09.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.641+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155010b5f0 con 0x7f15501054c0 2024-04-16T17:01:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.643+0000 7f154cc00640 1 -- 172.21.15.18:0/1661503693 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f153c0037a0 con 0x7f15501054c0 2024-04-16T17:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.645+0000 7f154cc00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f15340a3bb0 0x7f15340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.645+0000 7f154cc00640 1 -- 172.21.15.18:0/1661503693 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f153c0d0240 con 0x7f15501054c0 2024-04-16T17:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.645+0000 7f154fe00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f15340a3bb0 0x7f15340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.646+0000 7f154fe00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f15340a3bb0 0x7f15340a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f154002c900 tx=0x7f1540005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.650+0000 7f154cc00640 1 -- 172.21.15.18:0/1661503693 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f153c09d020 con 0x7f15501054c0 2024-04-16T17:01:09.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.903+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f155002d620 con 0x7f15501054c0 2024-04-16T17:01:09.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.904+0000 7f154cc00640 1 -- 172.21.15.18:0/1661503693 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7986 (secure 0 0 0) 0x7f153c09c920 con 0x7f15501054c0 2024-04-16T17:01:09.906 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:09.906 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T17:01:09.907 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 10 2024-04-16T17:01:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.908+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f15340a3bb0 msgr2=0x7f15340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.908+0000 7f1556e00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f15340a3bb0 0x7f15340a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f154002c900 tx=0x7f1540005b50 comp rx=0 tx=0).stop 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.908+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15501054c0 msgr2=0x7f155019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.908+0000 7f1556e00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15501054c0 0x7f155019ab60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9f0 tx=0x7f153c00edf0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.909+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 shutdown_connections 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.909+0000 7f1556e00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1550105e80 0x7f15501a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.909+0000 7f1556e00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15501054c0 0x7f155019ab60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.909+0000 7f1556e00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f15340a3bb0 0x7f15340a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.909+0000 7f1556e00640 1 --2- 172.21.15.18:0/1661503693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15501042d0 0x7f155019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.909+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 >> 172.21.15.18:0/1661503693 conn(0x7f15500ff880 msgr2=0x7f15501087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.909+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 shutdown_connections 2024-04-16T17:01:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:09.909+0000 7f1556e00640 1 -- 172.21.15.18:0/1661503693 wait complete. 2024-04-16T17:01:10.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:10 smithi022 ceph-mon[36023]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:10.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:10 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1661503693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:10.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:10 smithi018 ceph-mon[31776]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:10.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:10 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1661503693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:11.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:10 smithi043 ceph-mon[36901]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:11.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:10 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1661503693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:11.376 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:01:11 smithi018 podman[70734]: 2024-04-16 17:01:11.084959111 +0000 UTC m=+0.165986042 container create fc88423e24a5c3a0951c35a216e01eb3d76346d5edd57526cb64d4e678b52ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g, ceph=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-16T17:01:11.377 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:01:11 smithi018 podman[70734]: 2024-04-16 17:01:11.372526813 +0000 UTC m=+0.453553771 container init fc88423e24a5c3a0951c35a216e01eb3d76346d5edd57526cb64d4e678b52ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g, ceph=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:01:11.696 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:01:11 smithi018 podman[70734]: 2024-04-16 17:01:11.381932613 +0000 UTC m=+0.462959514 container start fc88423e24a5c3a0951c35a216e01eb3d76346d5edd57526cb64d4e678b52ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:01:11.696 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:01:11 smithi018 ceph-mds[70759]: -- 172.21.15.18:0/1121050940 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561605e2fa40 con 0x561605e02c00 2024-04-16T17:01:11.696 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:01:11 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g[70753]: starting mds.g at 2024-04-16T17:01:11.696 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:01:11 smithi018 bash[70734]: fc88423e24a5c3a0951c35a216e01eb3d76346d5edd57526cb64d4e678b52ed7 2024-04-16T17:01:11.696 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:01:11 smithi018 systemd[1]: Started Ceph mds.g for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:01:11.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 -- 172.21.15.18:0/436189756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74074a00 msgr2=0x7fed74074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 --2- 172.21.15.18:0/436189756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74074a00 0x7fed74074e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fed60009900 tx=0x7fed6002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 -- 172.21.15.18:0/436189756 shutdown_connections 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 --2- 172.21.15.18:0/436189756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109de0 0x7fed7410c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 --2- 172.21.15.18:0/436189756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74074a00 0x7fed74074e60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 --2- 172.21.15.18:0/436189756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed74075fe0 0x7fed74074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 -- 172.21.15.18:0/436189756 >> 172.21.15.18:0/436189756 conn(0x7fed740fd4c0 msgr2=0x7fed740ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 -- 172.21.15.18:0/436189756 shutdown_connections 2024-04-16T17:01:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.136+0000 7fed7aa00640 1 -- 172.21.15.18:0/436189756 wait complete. 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.137+0000 7fed7aa00640 1 Processor -- start 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.137+0000 7fed7aa00640 1 -- start start 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.137+0000 7fed7aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed74074a00 0x7fed7419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.137+0000 7fed7aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74075fe0 0x7fed7419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.137+0000 7fed7aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109de0 0x7fed741a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.137+0000 7fed7aa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410fde0 con 0x7fed74075fe0 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.137+0000 7fed7aa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410fc60 con 0x7fed74074a00 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.137+0000 7fed7aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410ff60 con 0x7fed74109de0 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed78c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74075fe0 0x7fed7419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed78c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74075fe0 0x7fed7419a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41484/0 (socket says 172.21.15.18:41484) 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed78c00640 1 -- 172.21.15.18:0/783719017 learned_addr learned my addr 172.21.15.18:0/783719017 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed79600640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed74074a00 0x7fed7419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:12.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed7a000640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109de0 0x7fed741a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed79600640 1 -- 172.21.15.18:0/783719017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109de0 msgr2=0x7fed741a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed79600640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109de0 0x7fed741a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed79600640 1 -- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74075fe0 msgr2=0x7fed7419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed79600640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74075fe0 0x7fed7419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed79600640 1 -- 172.21.15.18:0/783719017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed741a2060 con 0x7fed74074a00 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed78c00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74075fe0 0x7fed7419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.138+0000 7fed79600640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed74074a00 0x7fed7419a440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed6800e9f0 tx=0x7fed6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.139+0000 7fed72000640 1 -- 172.21.15.18:0/783719017 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6800cd80 con 0x7fed74074a00 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.139+0000 7fed72000640 1 -- 172.21.15.18:0/783719017 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed680035f0 con 0x7fed74074a00 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.139+0000 7fed72000640 1 -- 172.21.15.18:0/783719017 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68010690 con 0x7fed74074a00 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.139+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed741a2350 con 0x7fed74074a00 2024-04-16T17:01:12.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.139+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed741a2820 con 0x7fed74074a00 2024-04-16T17:01:12.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.140+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed74108ba0 con 0x7fed74074a00 2024-04-16T17:01:12.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.142+0000 7fed72000640 1 -- 172.21.15.18:0/783719017 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fed68003b30 con 0x7fed74074a00 2024-04-16T17:01:12.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.143+0000 7fed72000640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fed5c0a3ba0 0x7fed5c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:12.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.143+0000 7fed72000640 1 -- 172.21.15.18:0/783719017 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fed680d02b0 con 0x7fed74074a00 2024-04-16T17:01:12.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.143+0000 7fed78c00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fed5c0a3ba0 0x7fed5c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:12.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.144+0000 7fed78c00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fed5c0a3ba0 0x7fed5c0a6060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed60006010 tx=0x7fed6002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:12.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.147+0000 7fed72000640 1 -- 172.21.15.18:0/783719017 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed6809d090 con 0x7fed74074a00 2024-04-16T17:01:12.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.398+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed7402d620 con 0x7fed74074a00 2024-04-16T17:01:12.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.399+0000 7fed72000640 1 -- 172.21.15.18:0/783719017 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8809 (secure 0 0 0) 0x7fed6809c990 con 0x7fed74074a00 2024-04-16T17:01:12.401 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:12.401 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-16T17:01:12.401 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 11 2024-04-16T17:01:12.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fed5c0a3ba0 msgr2=0x7fed5c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:12.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fed5c0a3ba0 0x7fed5c0a6060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed60006010 tx=0x7fed6002cf90 comp rx=0 tx=0).stop 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed74074a00 msgr2=0x7fed7419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed74074a00 0x7fed7419a440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed6800e9f0 tx=0x7fed6800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 shutdown_connections 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109de0 0x7fed741a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fed5c0a3ba0 0x7fed5c0a6060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74075fe0 0x7fed7419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 --2- 172.21.15.18:0/783719017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed74074a00 0x7fed7419a440 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 >> 172.21.15.18:0/783719017 conn(0x7fed740fd4c0 msgr2=0x7fed74105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 shutdown_connections 2024-04-16T17:01:12.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:12.402+0000 7fed7aa00640 1 -- 172.21.15.18:0/783719017 wait complete. 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] up:boot 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: fsmap 10 up:standby 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:12.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:12 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/783719017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:12.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] up:boot 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: fsmap 10 up:standby 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:12.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:12 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/783719017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] up:boot 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: fsmap 10 up:standby 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:13.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:12 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/783719017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:13.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:13 smithi022 ceph-mon[36023]: Deploying daemon mds.a on smithi018 2024-04-16T17:01:13.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:13 smithi018 ceph-mon[31776]: Deploying daemon mds.a on smithi018 2024-04-16T17:01:14.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:13 smithi043 ceph-mon[36901]: Deploying daemon mds.a on smithi018 2024-04-16T17:01:14.421 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.745+0000 7fe6e7800640 1 -- 172.21.15.18:0/2540700374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6e010e6a0 msgr2=0x7fe6e0110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.745+0000 7fe6e7800640 1 --2- 172.21.15.18:0/2540700374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6e010e6a0 0x7fe6e0110a90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009cd0 tx=0x7fe6d802c0e0 comp rx=0 tx=0).stop 2024-04-16T17:01:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.746+0000 7fe6e7800640 1 -- 172.21.15.18:0/2540700374 shutdown_connections 2024-04-16T17:01:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.746+0000 7fe6e7800640 1 --2- 172.21.15.18:0/2540700374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6e010e6a0 0x7fe6e0110a90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.746+0000 7fe6e7800640 1 --2- 172.21.15.18:0/2540700374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6e0074a60 0x7fe6e0074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.746+0000 7fe6e7800640 1 --2- 172.21.15.18:0/2540700374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e0076040 0x7fe6e0074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:14.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.746+0000 7fe6e7800640 1 -- 172.21.15.18:0/2540700374 >> 172.21.15.18:0/2540700374 conn(0x7fe6e00fd620 msgr2=0x7fe6e00ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.746+0000 7fe6e7800640 1 -- 172.21.15.18:0/2540700374 shutdown_connections 2024-04-16T17:01:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.746+0000 7fe6e7800640 1 -- 172.21.15.18:0/2540700374 wait complete. 2024-04-16T17:01:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.748+0000 7fe6e7800640 1 Processor -- start 2024-04-16T17:01:14.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.748+0000 7fe6e7800640 1 -- start start 2024-04-16T17:01:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.748+0000 7fe6e7800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6e0074a60 0x7fe6e00736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.748+0000 7fe6e7800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6e0076040 0x7fe6e0073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.748+0000 7fe6e7800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e010e6a0 0x7fe6e0071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.748+0000 7fe6e7800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e006c470 con 0x7fe6e0076040 2024-04-16T17:01:14.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.748+0000 7fe6e7800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e006c2f0 con 0x7fe6e0074a60 2024-04-16T17:01:14.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.748+0000 7fe6e7800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e006c5f0 con 0x7fe6e010e6a0 2024-04-16T17:01:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.749+0000 7fe6e4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6e0074a60 0x7fe6e00736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.749+0000 7fe6e4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6e0074a60 0x7fe6e00736d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51046/0 (socket says 172.21.15.18:51046) 2024-04-16T17:01:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.749+0000 7fe6e4e00640 1 -- 172.21.15.18:0/876332608 learned_addr learned my addr 172.21.15.18:0/876332608 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.749+0000 7fe6dfe00640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6e0076040 0x7fe6e0073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.750+0000 7fe6e5800640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e010e6a0 0x7fe6e0071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.750+0000 7fe6e4e00640 1 -- 172.21.15.18:0/876332608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e010e6a0 msgr2=0x7fe6e0071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:14.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.750+0000 7fe6e4e00640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e010e6a0 0x7fe6e0071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.750+0000 7fe6e4e00640 1 -- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6e0076040 msgr2=0x7fe6e0073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.750+0000 7fe6e4e00640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6e0076040 0x7fe6e0073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:14.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.750+0000 7fe6e4e00640 1 -- 172.21.15.18:0/876332608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e0072530 con 0x7fe6e0074a60 2024-04-16T17:01:14.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.754+0000 7fe6e4e00640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6e0074a60 0x7fe6e00736d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400e9f0 tx=0x7fe6d400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:14.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.755+0000 7fe6dd600640 1 -- 172.21.15.18:0/876332608 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400cd80 con 0x7fe6e0074a60 2024-04-16T17:01:14.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.755+0000 7fe6dd600640 1 -- 172.21.15.18:0/876332608 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d4005500 con 0x7fe6e0074a60 2024-04-16T17:01:14.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.755+0000 7fe6dd600640 1 -- 172.21.15.18:0/876332608 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d40186d0 con 0x7fe6e0074a60 2024-04-16T17:01:14.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.755+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e0072820 con 0x7fe6e0074a60 2024-04-16T17:01:14.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.755+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e01af5d0 con 0x7fe6e0074a60 2024-04-16T17:01:14.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.757+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e0104760 con 0x7fe6e0074a60 2024-04-16T17:01:14.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.761+0000 7fe6dd600640 1 -- 172.21.15.18:0/876332608 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe6d4005020 con 0x7fe6e0074a60 2024-04-16T17:01:14.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.763+0000 7fe6dd600640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe6c80a3d00 0x7fe6c80a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:14.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.763+0000 7fe6dd600640 1 -- 172.21.15.18:0/876332608 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe6d40d0440 con 0x7fe6e0074a60 2024-04-16T17:01:14.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.764+0000 7fe6dfe00640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe6c80a3d00 0x7fe6c80a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:14.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.765+0000 7fe6dfe00640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe6c80a3d00 0x7fe6c80a61c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0073440 tx=0x7fe6cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:14.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:14.766+0000 7fe6dd600640 1 -- 172.21.15.18:0/876332608 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6d409d220 con 0x7fe6e0074a60 2024-04-16T17:01:14.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:14 smithi022 ceph-mon[36023]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:14.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:14 smithi018 ceph-mon[31776]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:15.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:14 smithi043 ceph-mon[36901]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:15.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.017+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6e0062c00 con 0x7fe6e0074a60 2024-04-16T17:01:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.018+0000 7fe6dd600640 1 -- 172.21.15.18:0/876332608 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8809 (secure 0 0 0) 0x7fe6d409cb20 con 0x7fe6e0074a60 2024-04-16T17:01:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:15.020 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-16T17:01:15.021 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 11 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe6c80a3d00 msgr2=0x7fe6c80a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe6c80a3d00 0x7fe6c80a61c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0073440 tx=0x7fe6cc005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6e0074a60 msgr2=0x7fe6e00736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6e0074a60 0x7fe6e00736d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400e9f0 tx=0x7fe6d400edf0 comp rx=0 tx=0).stop 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 shutdown_connections 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e010e6a0 0x7fe6e0071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe6c80a3d00 0x7fe6c80a61c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6e0076040 0x7fe6e0073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 --2- 172.21.15.18:0/876332608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6e0074a60 0x7fe6e00736d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 >> 172.21.15.18:0/876332608 conn(0x7fe6e00fd620 msgr2=0x7fe6e0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 shutdown_connections 2024-04-16T17:01:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:15.021+0000 7fe6e7800640 1 -- 172.21.15.18:0/876332608 wait complete. 2024-04-16T17:01:15.618 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:01:15 smithi018 podman[71124]: 2024-04-16 17:01:15.391918384 +0000 UTC m=+0.159461126 container create af5e644397b54022807f5ed05e45fe8d256592f2204aa35f4cac6b381f71e85d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-16T17:01:15.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:15 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/876332608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:15.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:15 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/876332608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:15.947 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:01:15 smithi018 podman[71124]: 2024-04-16 17:01:15.704003435 +0000 UTC m=+0.471546181 container init af5e644397b54022807f5ed05e45fe8d256592f2204aa35f4cac6b381f71e85d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-16T17:01:15.947 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:01:15 smithi018 podman[71124]: 2024-04-16 17:01:15.715477704 +0000 UTC m=+0.483020422 container start af5e644397b54022807f5ed05e45fe8d256592f2204aa35f4cac6b381f71e85d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-16T17:01:15.947 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:01:15 smithi018 ceph-mds[71143]: -- 172.21.15.18:0/1350370996 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55590c8f5a40 con 0x55590c8c8c00 2024-04-16T17:01:15.947 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:01:15 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a[71139]: starting mds.a at 2024-04-16T17:01:15.947 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:01:15 smithi018 bash[71124]: af5e644397b54022807f5ed05e45fe8d256592f2204aa35f4cac6b381f71e85d 2024-04-16T17:01:15.947 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:01:15 smithi018 systemd[1]: Started Ceph mds.a for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:01:16.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:15 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/876332608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:16.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:16 smithi022 ceph-mon[36023]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:16.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:16.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:16.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:16.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:16.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:16.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:16.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:16 smithi018 ceph-mon[31776]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:16.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:16.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:16.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:16.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:16.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:16.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:17.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:16 smithi043 ceph-mon[36901]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:17.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:17.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:17.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:17.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:01:17.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:01:17.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:17.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:17.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.278+0000 7fa31ec00640 1 -- 172.21.15.18:0/684607355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 msgr2=0x7fa31810a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:17.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.278+0000 7fa31ec00640 1 --2- 172.21.15.18:0/684607355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 0x7fa31810a3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa310009900 tx=0x7fa31002bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:17.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.278+0000 7fa31ec00640 1 -- 172.21.15.18:0/684607355 shutdown_connections 2024-04-16T17:01:17.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.278+0000 7fa31ec00640 1 --2- 172.21.15.18:0/684607355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 0x7fa31810a3b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.278+0000 7fa31ec00640 1 --2- 172.21.15.18:0/684607355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3181030d0 0x7fa318103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.278+0000 7fa31ec00640 1 --2- 172.21.15.18:0/684607355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa318101f00 0x7fa318102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.278+0000 7fa31ec00640 1 -- 172.21.15.18:0/684607355 >> 172.21.15.18:0/684607355 conn(0x7fa3180fd4c0 msgr2=0x7fa3180ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:17.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.279+0000 7fa31ec00640 1 -- 172.21.15.18:0/684607355 shutdown_connections 2024-04-16T17:01:17.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.279+0000 7fa31ec00640 1 -- 172.21.15.18:0/684607355 wait complete. 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.279+0000 7fa31ec00640 1 Processor -- start 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31ec00640 1 -- start start 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa318101f00 0x7fa31819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31ec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3181030d0 0x7fa31819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31ec00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 0x7fa3181a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31ec00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa31810dc10 con 0x7fa318103a90 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31ec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa31810da90 con 0x7fa3181030d0 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31ec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa31810dd90 con 0x7fa318101f00 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31e200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 0x7fa3181a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa318101f00 0x7fa31819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31e200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 0x7fa3181a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58412/0 (socket says 172.21.15.18:58412) 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.280+0000 7fa31e200640 1 -- 172.21.15.18:0/3308528945 learned_addr learned my addr 172.21.15.18:0/3308528945 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:17.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa31ce00640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3181030d0 0x7fa31819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa31e200640 1 -- 172.21.15.18:0/3308528945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa318101f00 msgr2=0x7fa31819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa31e200640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa318101f00 0x7fa31819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa31e200640 1 -- 172.21.15.18:0/3308528945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3181030d0 msgr2=0x7fa31819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa31e200640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3181030d0 0x7fa31819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa31e200640 1 -- 172.21.15.18:0/3308528945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3181a1ff0 con 0x7fa318103a90 2024-04-16T17:01:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa31ce00640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3181030d0 0x7fa31819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:01:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa31e200640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 0x7fa3181a1980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa31002c8e0 tx=0x7fa31002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.281+0000 7fa316000640 1 -- 172.21.15.18:0/3308528945 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310043020 con 0x7fa318103a90 2024-04-16T17:01:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.282+0000 7fa316000640 1 -- 172.21.15.18:0/3308528945 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3100054e0 con 0x7fa318103a90 2024-04-16T17:01:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.282+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3181a2280 con 0x7fa318103a90 2024-04-16T17:01:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.282+0000 7fa316000640 1 -- 172.21.15.18:0/3308528945 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310003d00 con 0x7fa318103a90 2024-04-16T17:01:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.282+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3181a27d0 con 0x7fa318103a90 2024-04-16T17:01:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.283+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3181a2af0 con 0x7fa318103a90 2024-04-16T17:01:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.285+0000 7fa316000640 1 -- 172.21.15.18:0/3308528945 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa310046020 con 0x7fa318103a90 2024-04-16T17:01:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.286+0000 7fa316000640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa3000a3ba0 0x7fa3000a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:17.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.286+0000 7fa316000640 1 -- 172.21.15.18:0/3308528945 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa3100f24b0 con 0x7fa318103a90 2024-04-16T17:01:17.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.286+0000 7fa31d800640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa3000a3ba0 0x7fa3000a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:17.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.287+0000 7fa31d800640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa3000a3ba0 0x7fa3000a6060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa318101c20 tx=0x7fa30c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:17.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.289+0000 7fa316000640 1 -- 172.21.15.18:0/3308528945 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa31004d020 con 0x7fa318103a90 2024-04-16T17:01:17.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.540+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3181a2cc0 con 0x7fa318103a90 2024-04-16T17:01:17.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.542+0000 7fa316000640 1 -- 172.21.15.18:0/3308528945 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9635 (secure 0 0 0) 0x7fa3100bf320 con 0x7fa318103a90 2024-04-16T17:01:17.544 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:17.544 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-16T17:01:17.544 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 12 2024-04-16T17:01:17.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.544+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa3000a3ba0 msgr2=0x7fa3000a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.544+0000 7fa31ec00640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa3000a3ba0 0x7fa3000a6060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa318101c20 tx=0x7fa30c009500 comp rx=0 tx=0).stop 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.544+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 msgr2=0x7fa3181a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.544+0000 7fa31ec00640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 0x7fa3181a1980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa31002c8e0 tx=0x7fa31002c640 comp rx=0 tx=0).stop 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.545+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 shutdown_connections 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.545+0000 7fa31ec00640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa3000a3ba0 0x7fa3000a6060 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.545+0000 7fa31ec00640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa318103a90 0x7fa3181a1980 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.545+0000 7fa31ec00640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3181030d0 0x7fa31819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.545+0000 7fa31ec00640 1 --2- 172.21.15.18:0/3308528945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa318101f00 0x7fa31819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.545+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 >> 172.21.15.18:0/3308528945 conn(0x7fa3180fd4c0 msgr2=0x7fa318074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.545+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 shutdown_connections 2024-04-16T17:01:17.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:17.545+0000 7fa31ec00640 1 -- 172.21.15.18:0/3308528945 wait complete. 2024-04-16T17:01:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:17 smithi022 ceph-mon[36023]: Deploying daemon mds.b on smithi022 2024-04-16T17:01:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:17 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.18:6840/2108347898,v1:172.21.15.18:6841/2108347898] up:boot 2024-04-16T17:01:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:17 smithi022 ceph-mon[36023]: fsmap 11 up:standby 2024-04-16T17:01:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:17 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:01:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:17 smithi022 ceph-mon[36023]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:17 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3308528945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:17 smithi018 ceph-mon[31776]: Deploying daemon mds.b on smithi022 2024-04-16T17:01:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:17 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.18:6840/2108347898,v1:172.21.15.18:6841/2108347898] up:boot 2024-04-16T17:01:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:17 smithi018 ceph-mon[31776]: fsmap 11 up:standby 2024-04-16T17:01:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:17 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:01:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:17 smithi018 ceph-mon[31776]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:17 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3308528945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:17 smithi043 ceph-mon[36901]: Deploying daemon mds.b on smithi022 2024-04-16T17:01:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:17 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.18:6840/2108347898,v1:172.21.15.18:6841/2108347898] up:boot 2024-04-16T17:01:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:17 smithi043 ceph-mon[36901]: fsmap 11 up:standby 2024-04-16T17:01:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:17 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:01:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:17 smithi043 ceph-mon[36901]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:17 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3308528945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:18.420 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 systemd[1]: Starting Ceph mds.b for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:01:18.675 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 podman[66356]: 2024-04-16 17:01:18.322232151 +0000 UTC m=+0.018358715 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:01:18.675 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 podman[66356]: 2024-04-16 17:01:18.435290429 +0000 UTC m=+0.131417005 container create f45263a6338f69dd1d53fe9cdcbc24c0b2477ef91c721b0afffb37baf7868eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-16T17:01:19.175 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 podman[66356]: 2024-04-16 17:01:18.752722108 +0000 UTC m=+0.448848671 container init f45263a6338f69dd1d53fe9cdcbc24c0b2477ef91c721b0afffb37baf7868eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:01:19.175 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 podman[66356]: 2024-04-16 17:01:18.755500777 +0000 UTC m=+0.451627341 container start f45263a6338f69dd1d53fe9cdcbc24c0b2477ef91c721b0afffb37baf7868eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-16T17:01:19.175 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 ceph-mds[66375]: -- 172.21.15.22:0/2551548864 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c858749a40 con 0x55c85871c800 2024-04-16T17:01:19.175 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: starting mds.b at 2024-04-16T17:01:19.175 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 bash[66356]: f45263a6338f69dd1d53fe9cdcbc24c0b2477ef91c721b0afffb37baf7868eb8 2024-04-16T17:01:19.175 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:18 smithi022 systemd[1]: Started Ceph mds.b for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:01:19.566 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:19.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.781+0000 7f5918200640 1 -- 172.21.15.18:0/4025758132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59101020d0 msgr2=0x7f59101024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:19.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.781+0000 7f5918200640 1 --2- 172.21.15.18:0/4025758132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59101020d0 0x7f59101024f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5904009900 tx=0x7f590402bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.782+0000 7f5918200640 1 -- 172.21.15.18:0/4025758132 shutdown_connections 2024-04-16T17:01:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.782+0000 7f5918200640 1 --2- 172.21.15.18:0/4025758132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5910103c80 0x7f591010a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.782+0000 7f5918200640 1 --2- 172.21.15.18:0/4025758132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59101032c0 0x7f5910103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.782+0000 7f5918200640 1 --2- 172.21.15.18:0/4025758132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59101020d0 0x7f59101024f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.782+0000 7f5918200640 1 -- 172.21.15.18:0/4025758132 >> 172.21.15.18:0/4025758132 conn(0x7f59100fd620 msgr2=0x7f59100ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.782+0000 7f5918200640 1 -- 172.21.15.18:0/4025758132 shutdown_connections 2024-04-16T17:01:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.782+0000 7f5918200640 1 -- 172.21.15.18:0/4025758132 wait complete. 2024-04-16T17:01:19.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5918200640 1 Processor -- start 2024-04-16T17:01:19.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5918200640 1 -- start start 2024-04-16T17:01:19.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5918200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59101020d0 0x7f591019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:19.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5918200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59101032c0 0x7f591019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5918200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5910103c80 0x7f59101a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5918200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f591010dd40 con 0x7f59101032c0 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5918200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f591010dbc0 con 0x7f5910103c80 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5918200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f591010dec0 con 0x7f59101020d0 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5915800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59101020d0 0x7f591019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5915800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59101020d0 0x7f591019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40042/0 (socket says 172.21.15.18:40042) 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5915800640 1 -- 172.21.15.18:0/161179969 learned_addr learned my addr 172.21.15.18:0/161179969 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5916200640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5910103c80 0x7f59101a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.783+0000 7f5914e00640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59101032c0 0x7f591019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f5915800640 1 -- 172.21.15.18:0/161179969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5910103c80 msgr2=0x7f59101a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f5915800640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5910103c80 0x7f59101a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f5915800640 1 -- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59101032c0 msgr2=0x7f591019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f5915800640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59101032c0 0x7f591019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f5915800640 1 -- 172.21.15.18:0/161179969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59101a21b0 con 0x7f59101020d0 2024-04-16T17:01:19.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f5916200640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5910103c80 0x7f59101a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:19.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f5915800640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59101020d0 0x7f591019a590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5904000c00 tx=0x7f590402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:19.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f590e000640 1 -- 172.21.15.18:0/161179969 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904043020 con 0x7f59101020d0 2024-04-16T17:01:19.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59101a2440 con 0x7f59101020d0 2024-04-16T17:01:19.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f590e000640 1 -- 172.21.15.18:0/161179969 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59040054e0 con 0x7f59101020d0 2024-04-16T17:01:19.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.784+0000 7f590e000640 1 -- 172.21.15.18:0/161179969 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904003c40 con 0x7f59101020d0 2024-04-16T17:01:19.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.785+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59101a2990 con 0x7f59101020d0 2024-04-16T17:01:19.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.786+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59101a2b90 con 0x7f59101020d0 2024-04-16T17:01:19.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.788+0000 7f590e000640 1 -- 172.21.15.18:0/161179969 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5904046020 con 0x7f59101020d0 2024-04-16T17:01:19.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.789+0000 7f590e000640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f58f80a3bb0 0x7f58f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:19.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.789+0000 7f590e000640 1 -- 172.21.15.18:0/161179969 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f59040f1fd0 con 0x7f59101020d0 2024-04-16T17:01:19.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.789+0000 7f5914e00640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f58f80a3bb0 0x7f58f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:19.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.790+0000 7f5914e00640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f58f80a3bb0 0x7f58f80a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f591019baa0 tx=0x7f58fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:19.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:19.798+0000 7f590e000640 1 -- 172.21.15.18:0/161179969 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f590404d020 con 0x7f59101020d0 2024-04-16T17:01:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] up:boot 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: fsmap 12 up:standby 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:19.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:19 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:20.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.055+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59101a2d60 con 0x7f59101020d0 2024-04-16T17:01:20.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.056+0000 7f590e000640 1 -- 172.21.15.18:0/161179969 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10458 (secure 0 0 0) 0x7f59040bedb0 con 0x7f59101020d0 2024-04-16T17:01:20.058 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:20.058 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-16T17:01:20.058 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 13 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.059+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f58f80a3bb0 msgr2=0x7f58f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.059+0000 7f5918200640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f58f80a3bb0 0x7f58f80a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f591019baa0 tx=0x7f58fc005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.059+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59101020d0 msgr2=0x7f591019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.059+0000 7f5918200640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59101020d0 0x7f591019a590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5904000c00 tx=0x7f590402c650 comp rx=0 tx=0).stop 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.060+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 shutdown_connections 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.060+0000 7f5918200640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5910103c80 0x7f59101a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.060+0000 7f5918200640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f58f80a3bb0 0x7f58f80a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.060+0000 7f5918200640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59101032c0 0x7f591019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.060+0000 7f5918200640 1 --2- 172.21.15.18:0/161179969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59101020d0 0x7f591019a590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.060+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 >> 172.21.15.18:0/161179969 conn(0x7f59100fd620 msgr2=0x7f5910074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:20.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.060+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 shutdown_connections 2024-04-16T17:01:20.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.060+0000 7f5918200640 1 -- 172.21.15.18:0/161179969 wait complete. 2024-04-16T17:01:20.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:01:20.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] up:boot 2024-04-16T17:01:20.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: fsmap 12 up:standby 2024-04-16T17:01:20.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:01:20.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:20.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:19 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:20.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] up:boot 2024-04-16T17:01:20.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: fsmap 12 up:standby 2024-04-16T17:01:20.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:01:20.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:20.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:20.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:19 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:20.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 -- 172.21.15.18:0/1996245894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7684101f00 msgr2=0x7f7684102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 --2- 172.21.15.18:0/1996245894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7684101f00 0x7f7684102340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7678009900 tx=0x7f767802bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:20.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 -- 172.21.15.18:0/1996245894 shutdown_connections 2024-04-16T17:01:20.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 --2- 172.21.15.18:0/1996245894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7684103a90 0x7f768410a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 --2- 172.21.15.18:0/1996245894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76841030d0 0x7f7684103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 --2- 172.21.15.18:0/1996245894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7684101f00 0x7f7684102340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 -- 172.21.15.18:0/1996245894 >> 172.21.15.18:0/1996245894 conn(0x7f76840fd4c0 msgr2=0x7f76840ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:20.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 -- 172.21.15.18:0/1996245894 shutdown_connections 2024-04-16T17:01:20.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.289+0000 7f768a800640 1 -- 172.21.15.18:0/1996245894 wait complete. 2024-04-16T17:01:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.290+0000 7f768a800640 1 Processor -- start 2024-04-16T17:01:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.290+0000 7f768a800640 1 -- start start 2024-04-16T17:01:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.290+0000 7f768a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7684101f00 0x7f768419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.290+0000 7f768a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76841030d0 0x7f768419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.290+0000 7f768a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7684103a90 0x7f76841a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.290+0000 7f768a800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f768410dc20 con 0x7f7684101f00 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.290+0000 7f768a800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f768410daa0 con 0x7f7684103a90 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f768a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f768410dda0 con 0x7f76841030d0 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7688a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76841030d0 0x7f768419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7688a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76841030d0 0x7f768419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40058/0 (socket says 172.21.15.18:40058) 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7689400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7684101f00 0x7f768419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7688a00640 1 -- 172.21.15.18:0/3542416412 learned_addr learned my addr 172.21.15.18:0/3542416412 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7689e00640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7684103a90 0x7f76841a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7688a00640 1 -- 172.21.15.18:0/3542416412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7684103a90 msgr2=0x7f76841a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7688a00640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7684103a90 0x7f76841a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7688a00640 1 -- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7684101f00 msgr2=0x7f768419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7688a00640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7684101f00 0x7f768419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.291+0000 7f7688a00640 1 -- 172.21.15.18:0/3542416412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76841a20d0 con 0x7f76841030d0 2024-04-16T17:01:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.292+0000 7f7688a00640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76841030d0 0x7f768419a9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f767000e9f0 tx=0x7f767000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.292+0000 7f7682000640 1 -- 172.21.15.18:0/3542416412 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767000cd80 con 0x7f76841030d0 2024-04-16T17:01:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.292+0000 7f7682000640 1 -- 172.21.15.18:0/3542416412 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7670005500 con 0x7f76841030d0 2024-04-16T17:01:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.292+0000 7f7682000640 1 -- 172.21.15.18:0/3542416412 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670010690 con 0x7f76841030d0 2024-04-16T17:01:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.292+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76841a23c0 con 0x7f76841030d0 2024-04-16T17:01:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.292+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841a28c0 con 0x7f76841030d0 2024-04-16T17:01:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.294+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76841a2b80 con 0x7f76841030d0 2024-04-16T17:01:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.296+0000 7f7682000640 1 -- 172.21.15.18:0/3542416412 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f76700046c0 con 0x7f76841030d0 2024-04-16T17:01:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.297+0000 7f7682000640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f766c0a3c20 0x7f766c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.298+0000 7f7682000640 1 -- 172.21.15.18:0/3542416412 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f76700d0380 con 0x7f76841030d0 2024-04-16T17:01:20.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.298+0000 7f7689400640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f766c0a3c20 0x7f766c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:20.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.299+0000 7f7689400640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f766c0a3c20 0x7f766c0a60e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f767802c170 tx=0x7f767802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:20.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.302+0000 7f7682000640 1 -- 172.21.15.18:0/3542416412 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f767009d160 con 0x7f76841030d0 2024-04-16T17:01:20.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.531+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76841a2d50 con 0x7f76841030d0 2024-04-16T17:01:20.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.533+0000 7f7682000640 1 -- 172.21.15.18:0/3542416412 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18362 (secure 0 0 0) 0x7f767009ca60 con 0x7f76841030d0 2024-04-16T17:01:20.535 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:20.535 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":81,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:00:25.899823+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:01:20.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f766c0a3c20 msgr2=0x7f766c0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f766c0a3c20 0x7f766c0a60e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f767802c170 tx=0x7f767802df90 comp rx=0 tx=0).stop 2024-04-16T17:01:20.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76841030d0 msgr2=0x7f768419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76841030d0 0x7f768419a9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f767000e9f0 tx=0x7f767000edf0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 shutdown_connections 2024-04-16T17:01:20.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7684103a90 0x7f76841a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76841030d0 0x7f768419a9f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f766c0a3c20 0x7f766c0a60e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 --2- 172.21.15.18:0/3542416412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7684101f00 0x7f768419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 >> 172.21.15.18:0/3542416412 conn(0x7f76840fd4c0 msgr2=0x7f7684074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:20.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 shutdown_connections 2024-04-16T17:01:20.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.536+0000 7f768a800640 1 -- 172.21.15.18:0/3542416412 wait complete. 2024-04-16T17:01:20.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T17:01:20.558 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:01:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.775+0000 7f0759200640 1 -- 172.21.15.18:0/1599616649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541028c0 msgr2=0x7f0754104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.775+0000 7f0759200640 1 --2- 172.21.15.18:0/1599616649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541028c0 0x7f0754104cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0744009900 tx=0x7f074402bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.776+0000 7f0759200640 1 -- 172.21.15.18:0/1599616649 shutdown_connections 2024-04-16T17:01:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.776+0000 7f0759200640 1 --2- 172.21.15.18:0/1599616649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0754107b20 0x7f0754109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.776+0000 7f0759200640 1 --2- 172.21.15.18:0/1599616649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07541051f0 0x7f07541075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.776+0000 7f0759200640 1 --2- 172.21.15.18:0/1599616649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541028c0 0x7f0754104cb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.776+0000 7f0759200640 1 -- 172.21.15.18:0/1599616649 >> 172.21.15.18:0/1599616649 conn(0x7f07540fc2d0 msgr2=0x7f07540fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.776+0000 7f0759200640 1 -- 172.21.15.18:0/1599616649 shutdown_connections 2024-04-16T17:01:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.776+0000 7f0759200640 1 -- 172.21.15.18:0/1599616649 wait complete. 2024-04-16T17:01:20.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0759200640 1 Processor -- start 2024-04-16T17:01:20.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0759200640 1 -- start start 2024-04-16T17:01:20.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0759200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07541028c0 0x7f0754102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0759200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541051f0 0x7f0754100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0759200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0754107b20 0x7f0754100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0759200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f075410db10 con 0x7f0754107b20 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0759200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f075410d990 con 0x7f07541028c0 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0759200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f075410dc90 con 0x7f07541051f0 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0752a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541051f0 0x7f0754100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0753400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07541028c0 0x7f0754102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0752a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541051f0 0x7f0754100870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40070/0 (socket says 172.21.15.18:40070) 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0753e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0754107b20 0x7f0754100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0752a00640 1 -- 172.21.15.18:0/4032351235 learned_addr learned my addr 172.21.15.18:0/4032351235 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.777+0000 7f0753400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07541028c0 0x7f0754102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36494/0 (socket says 172.21.15.18:36494) 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f0752a00640 1 -- 172.21.15.18:0/4032351235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07541028c0 msgr2=0x7f0754102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f0752a00640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07541028c0 0x7f0754102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f0752a00640 1 -- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0754107b20 msgr2=0x7f0754100db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f0752a00640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0754107b20 0x7f0754100db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f0752a00640 1 -- 172.21.15.18:0/4032351235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0754101640 con 0x7f07541051f0 2024-04-16T17:01:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f0752a00640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541051f0 0x7f0754100870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f074000e9c0 tx=0x7f074000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f074be00640 1 -- 172.21.15.18:0/4032351235 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074000cd80 con 0x7f07541051f0 2024-04-16T17:01:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f074be00640 1 -- 172.21.15.18:0/4032351235 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0740005500 con 0x7f07541051f0 2024-04-16T17:01:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07541a2210 con 0x7f07541051f0 2024-04-16T17:01:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.778+0000 7f074be00640 1 -- 172.21.15.18:0/4032351235 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740010690 con 0x7f07541051f0 2024-04-16T17:01:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.779+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07541a27b0 con 0x7f07541051f0 2024-04-16T17:01:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.780+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07540fddc0 con 0x7f07541051f0 2024-04-16T17:01:20.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.782+0000 7f074be00640 1 -- 172.21.15.18:0/4032351235 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f07400046c0 con 0x7f07541051f0 2024-04-16T17:01:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.783+0000 7f074be00640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07380a3c20 0x7f07380a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.784+0000 7f074be00640 1 -- 172.21.15.18:0/4032351235 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f07400d0340 con 0x7f07541051f0 2024-04-16T17:01:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.784+0000 7f0753400640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07380a3c20 0x7f07380a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:20.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.785+0000 7f0753400640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07380a3c20 0x7f07380a60e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f074402c240 tx=0x7f074402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:20.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:20.788+0000 7f074be00640 1 -- 172.21.15.18:0/4032351235 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f074009d120 con 0x7f07541051f0 2024-04-16T17:01:21.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.003+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0754062c00 con 0x7f07541051f0 2024-04-16T17:01:21.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.004+0000 7f074be00640 1 -- 172.21.15.18:0/4032351235 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f074009ca20 con 0x7f07541051f0 2024-04-16T17:01:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:01:21.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07380a3c20 msgr2=0x7f07380a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07380a3c20 0x7f07380a60e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f074402c240 tx=0x7f074402df90 comp rx=0 tx=0).stop 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541051f0 msgr2=0x7f0754100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541051f0 0x7f0754100870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f074000e9c0 tx=0x7f074000ee80 comp rx=0 tx=0).stop 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 shutdown_connections 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07380a3c20 0x7f07380a60e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0754107b20 0x7f0754100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07541051f0 0x7f0754100870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 --2- 172.21.15.18:0/4032351235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07541028c0 0x7f0754102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 >> 172.21.15.18:0/4032351235 conn(0x7f07540fc2d0 msgr2=0x7f0754103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 shutdown_connections 2024-04-16T17:01:21.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.007+0000 7f0759200640 1 -- 172.21.15.18:0/4032351235 wait complete. 2024-04-16T17:01:21.027 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-16T17:01:21.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:20 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:21.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:20 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/161179969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:21.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:20 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3542416412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:21.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:20 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:21.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:20 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/161179969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:21.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:20 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3542416412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:21.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 -- 172.21.15.18:0/1707365635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa901085a0 msgr2=0x7faa901089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 --2- 172.21.15.18:0/1707365635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa901085a0 0x7faa901089c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa78009860 tx=0x7faa7802bb70 comp rx=0 tx=0).stop 2024-04-16T17:01:21.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 -- 172.21.15.18:0/1707365635 shutdown_connections 2024-04-16T17:01:21.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 --2- 172.21.15.18:0/1707365635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa9010a100 0x7faa90110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 --2- 172.21.15.18:0/1707365635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa90109740 0x7faa90109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 --2- 172.21.15.18:0/1707365635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa901085a0 0x7faa901089c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 -- 172.21.15.18:0/1707365635 >> 172.21.15.18:0/1707365635 conn(0x7faa900774f0 msgr2=0x7faa90079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:21.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 -- 172.21.15.18:0/1707365635 shutdown_connections 2024-04-16T17:01:21.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.247+0000 7faa94a00640 1 -- 172.21.15.18:0/1707365635 wait complete. 2024-04-16T17:01:21.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.248+0000 7faa94a00640 1 Processor -- start 2024-04-16T17:01:21.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa94a00640 1 -- start start 2024-04-16T17:01:21.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa94a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa901085a0 0x7faa9019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:21.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa94a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa90109740 0x7faa9019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:21.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa94a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa9010a100 0x7faa901a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa94a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faa901141b0 con 0x7faa901085a0 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa94a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa90114030 con 0x7faa90109740 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa94a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa90114330 con 0x7faa9010a100 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa8e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa901085a0 0x7faa9019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa8de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa90109740 0x7faa9019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa8e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa901085a0 0x7faa9019e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58506/0 (socket says 172.21.15.18:58506) 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa8e800640 1 -- 172.21.15.18:0/3064983067 learned_addr learned my addr 172.21.15.18:0/3064983067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.249+0000 7faa8f200640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa9010a100 0x7faa901a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.250+0000 7faa8e800640 1 -- 172.21.15.18:0/3064983067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa9010a100 msgr2=0x7faa901a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.250+0000 7faa8e800640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa9010a100 0x7faa901a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.250+0000 7faa8e800640 1 -- 172.21.15.18:0/3064983067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa90109740 msgr2=0x7faa9019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.250+0000 7faa8e800640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa90109740 0x7faa9019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.250+0000 7faa8e800640 1 -- 172.21.15.18:0/3064983067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa901a64d0 con 0x7faa901085a0 2024-04-16T17:01:21.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.250+0000 7faa8de00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa90109740 0x7faa9019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:21.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:20 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:21.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:20 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/161179969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:21.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:20 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3542416412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.250+0000 7faa8e800640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa901085a0 0x7faa9019e8b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa780349e0 tx=0x7faa780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.251+0000 7faa87400640 1 -- 172.21.15.18:0/3064983067 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78041020 con 0x7faa901085a0 2024-04-16T17:01:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.251+0000 7faa87400640 1 -- 172.21.15.18:0/3064983067 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa780323b0 con 0x7faa901085a0 2024-04-16T17:01:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.251+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa901a6760 con 0x7faa901085a0 2024-04-16T17:01:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.251+0000 7faa87400640 1 -- 172.21.15.18:0/3064983067 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78039040 con 0x7faa901085a0 2024-04-16T17:01:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.251+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901a6c50 con 0x7faa901085a0 2024-04-16T17:01:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.252+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9010f890 con 0x7faa901085a0 2024-04-16T17:01:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.254+0000 7faa87400640 1 -- 172.21.15.18:0/3064983067 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faa78044020 con 0x7faa901085a0 2024-04-16T17:01:21.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.256+0000 7faa87400640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faa740a3ba0 0x7faa740a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.256+0000 7faa87400640 1 -- 172.21.15.18:0/3064983067 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faa780f1ca0 con 0x7faa901085a0 2024-04-16T17:01:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.256+0000 7faa8de00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faa740a3ba0 0x7faa740a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.257+0000 7faa8de00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faa740a3ba0 0x7faa740a6060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa9019fdc0 tx=0x7faa800095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:21.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.259+0000 7faa87400640 1 -- 172.21.15.18:0/3064983067 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa7804b020 con 0x7faa901085a0 2024-04-16T17:01:21.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.483+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faa90062c00 con 0x7faa901085a0 2024-04-16T17:01:21.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.484+0000 7faa87400640 1 -- 172.21.15.18:0/3064983067 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7faa780bea80 con 0x7faa901085a0 2024-04-16T17:01:21.486 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:21.486 INFO:teuthology.orchestra.run.smithi018.stdout:[ 2024-04-16T17:01:21.486 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:01:21.486 INFO:teuthology.orchestra.run.smithi018.stdout: "poolnum": 1, 2024-04-16T17:01:21.486 INFO:teuthology.orchestra.run.smithi018.stdout: "poolname": ".mgr" 2024-04-16T17:01:21.486 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:01:21.486 INFO:teuthology.orchestra.run.smithi018.stdout:] 2024-04-16T17:01:21.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.486+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faa740a3ba0 msgr2=0x7faa740a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.486+0000 7faa94a00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faa740a3ba0 0x7faa740a6060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa9019fdc0 tx=0x7faa800095c0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.486+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa901085a0 msgr2=0x7faa9019e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.486+0000 7faa94a00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa901085a0 0x7faa9019e8b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa780349e0 tx=0x7faa780042f0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.487+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 shutdown_connections 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.487+0000 7faa94a00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa9010a100 0x7faa901a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.487+0000 7faa94a00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa90109740 0x7faa9019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.487+0000 7faa94a00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faa740a3ba0 0x7faa740a6060 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.487+0000 7faa94a00640 1 --2- 172.21.15.18:0/3064983067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa901085a0 0x7faa9019e8b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.487+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 >> 172.21.15.18:0/3064983067 conn(0x7faa900774f0 msgr2=0x7faa9010ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.487+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 shutdown_connections 2024-04-16T17:01:21.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.487+0000 7faa94a00640 1 -- 172.21.15.18:0/3064983067 wait complete. 2024-04-16T17:01:21.506 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-16T17:01:21.506 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.724+0000 7fc32b400640 1 -- 172.21.15.18:0/3078670622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324102ae0 msgr2=0x7fc324104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.724+0000 7fc32b400640 1 --2- 172.21.15.18:0/3078670622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324102ae0 0x7fc324104ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc318009900 tx=0x7fc31802bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.725+0000 7fc32b400640 1 -- 172.21.15.18:0/3078670622 shutdown_connections 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.725+0000 7fc32b400640 1 --2- 172.21.15.18:0/3078670622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc324107d40 0x7fc32410a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.725+0000 7fc32b400640 1 --2- 172.21.15.18:0/3078670622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc324105410 0x7fc324107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.725+0000 7fc32b400640 1 --2- 172.21.15.18:0/3078670622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324102ae0 0x7fc324104ed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.725+0000 7fc32b400640 1 -- 172.21.15.18:0/3078670622 >> 172.21.15.18:0/3078670622 conn(0x7fc3240fc4d0 msgr2=0x7fc3240fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.725+0000 7fc32b400640 1 -- 172.21.15.18:0/3078670622 shutdown_connections 2024-04-16T17:01:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.725+0000 7fc32b400640 1 -- 172.21.15.18:0/3078670622 wait complete. 2024-04-16T17:01:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.726+0000 7fc32b400640 1 Processor -- start 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.726+0000 7fc32b400640 1 -- start start 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.726+0000 7fc32b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc324102ae0 0x7fc3241961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.726+0000 7fc32b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324105410 0x7fc324196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.726+0000 7fc32b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc324107d40 0x7fc3241a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.726+0000 7fc32b400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410dd40 con 0x7fc324102ae0 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.726+0000 7fc32b400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410dbc0 con 0x7fc324105410 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.726+0000 7fc32b400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410dec0 con 0x7fc324107d40 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324105410 0x7fc324196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324105410 0x7fc324196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36528/0 (socket says 172.21.15.18:36528) 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 -- 172.21.15.18:0/2676348879 learned_addr learned my addr 172.21.15.18:0/2676348879 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc328a00640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc324102ae0 0x7fc3241961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc329400640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc324107d40 0x7fc3241a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 -- 172.21.15.18:0/2676348879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc324107d40 msgr2=0x7fc3241a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc324107d40 0x7fc3241a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 -- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc324102ae0 msgr2=0x7fc3241961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:21.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc324102ae0 0x7fc3241961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:21.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 -- 172.21.15.18:0/2676348879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3241a6630 con 0x7fc324105410 2024-04-16T17:01:21.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.727+0000 7fc323e00640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324105410 0x7fc324196720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc31000e9f0 tx=0x7fc31000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:21.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.728+0000 7fc321600640 1 -- 172.21.15.18:0/2676348879 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000ccc0 con 0x7fc324105410 2024-04-16T17:01:21.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.728+0000 7fc321600640 1 -- 172.21.15.18:0/2676348879 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc310005500 con 0x7fc324105410 2024-04-16T17:01:21.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.728+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3241a6920 con 0x7fc324105410 2024-04-16T17:01:21.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.728+0000 7fc321600640 1 -- 172.21.15.18:0/2676348879 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310010690 con 0x7fc324105410 2024-04-16T17:01:21.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.728+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3241a6e70 con 0x7fc324105410 2024-04-16T17:01:21.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.730+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc324109890 con 0x7fc324105410 2024-04-16T17:01:21.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.731+0000 7fc321600640 1 -- 172.21.15.18:0/2676348879 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc3100046c0 con 0x7fc324105410 2024-04-16T17:01:21.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.732+0000 7fc321600640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc30c0a3c30 0x7fc30c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:21.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.733+0000 7fc321600640 1 -- 172.21.15.18:0/2676348879 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc3100d0260 con 0x7fc324105410 2024-04-16T17:01:21.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.733+0000 7fc328a00640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc30c0a3c30 0x7fc30c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:21.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.733+0000 7fc328a00640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc30c0a3c30 0x7fc30c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc31802c240 tx=0x7fc31802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:21.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.737+0000 7fc321600640 1 -- 172.21.15.18:0/2676348879 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc31009d080 con 0x7fc324105410 2024-04-16T17:01:21.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:21.960+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc324062c00 con 0x7fc324105410 2024-04-16T17:01:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:21 smithi022 ceph-mon[36023]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:21 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4032351235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:21 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:21 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:21 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:21 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:21 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3064983067' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T17:01:22.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:21 smithi018 ceph-mon[31776]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:22.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:21 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4032351235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:22.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:21 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:21 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:21 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:21 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:21 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3064983067' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T17:01:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:21 smithi043 ceph-mon[36901]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:21 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4032351235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:21 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:21 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:21 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:21 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:22.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:21 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3064983067' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T17:01:22.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.895+0000 7fc321600640 1 -- 172.21.15.18:0/2676348879 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fc31009c980 con 0x7fc324105410 2024-04-16T17:01:22.900 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_metadata' created 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc30c0a3c30 msgr2=0x7fc30c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc30c0a3c30 0x7fc30c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc31802c240 tx=0x7fc31802df90 comp rx=0 tx=0).stop 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324105410 msgr2=0x7fc324196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324105410 0x7fc324196720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc31000e9f0 tx=0x7fc31000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 shutdown_connections 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc324107d40 0x7fc3241a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc324105410 0x7fc324196720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc30c0a3c30 0x7fc30c0a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 --2- 172.21.15.18:0/2676348879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc324102ae0 0x7fc3241961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:22.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.903+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 >> 172.21.15.18:0/2676348879 conn(0x7fc3240fc4d0 msgr2=0x7fc3241022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.904+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 shutdown_connections 2024-04-16T17:01:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:22.904+0000 7fc32b400640 1 -- 172.21.15.18:0/2676348879 wait complete. 2024-04-16T17:01:22.926 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 -- 172.21.15.18:0/1324472625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6c4107d20 msgr2=0x7fa6c410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 --2- 172.21.15.18:0/1324472625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6c4107d20 0x7fa6c410a170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009ca0 tx=0x7fa6bc02c100 comp rx=0 tx=0).stop 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 -- 172.21.15.18:0/1324472625 shutdown_connections 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 --2- 172.21.15.18:0/1324472625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6c4107d20 0x7fa6c410a170 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 --2- 172.21.15.18:0/1324472625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6c41053f0 0x7fa6c41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 --2- 172.21.15.18:0/1324472625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6c4102ac0 0x7fa6c4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 -- 172.21.15.18:0/1324472625 >> 172.21.15.18:0/1324472625 conn(0x7fa6c40fc4d0 msgr2=0x7fa6c40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 -- 172.21.15.18:0/1324472625 shutdown_connections 2024-04-16T17:01:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.129+0000 7fa6c9000640 1 -- 172.21.15.18:0/1324472625 wait complete. 2024-04-16T17:01:23.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.130+0000 7fa6c9000640 1 Processor -- start 2024-04-16T17:01:23.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.130+0000 7fa6c9000640 1 -- start start 2024-04-16T17:01:23.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.130+0000 7fa6c9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6c4102ac0 0x7fa6c4196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.130+0000 7fa6c9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6c41053f0 0x7fa6c4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c9000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6c4107d20 0x7fa6c4193870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c9000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c410dcd0 con 0x7fa6c4107d20 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c9000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c410db50 con 0x7fa6c4102ac0 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c9000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c410de50 con 0x7fa6c41053f0 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6c4102ac0 0x7fa6c4196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6c4102ac0 0x7fa6c4196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36546/0 (socket says 172.21.15.18:36546) 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1e00640 1 -- 172.21.15.18:0/515327099 learned_addr learned my addr 172.21.15.18:0/515327099 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1400640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6c41053f0 0x7fa6c4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c2800640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6c4107d20 0x7fa6c4193870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1e00640 1 -- 172.21.15.18:0/515327099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6c41053f0 msgr2=0x7fa6c4197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1e00640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6c41053f0 0x7fa6c4197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1e00640 1 -- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6c4107d20 msgr2=0x7fa6c4193870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1e00640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6c4107d20 0x7fa6c4193870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:23.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.131+0000 7fa6c1e00640 1 -- 172.21.15.18:0/515327099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c4194130 con 0x7fa6c4102ac0 2024-04-16T17:01:23.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.132+0000 7fa6c1e00640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6c4102ac0 0x7fa6c4196d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400ea60 tx=0x7fa6b400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:23.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.132+0000 7fa6baa00640 1 -- 172.21.15.18:0/515327099 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400cde0 con 0x7fa6c4102ac0 2024-04-16T17:01:23.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.132+0000 7fa6baa00640 1 -- 172.21.15.18:0/515327099 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b40035f0 con 0x7fa6c4102ac0 2024-04-16T17:01:23.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.132+0000 7fa6baa00640 1 -- 172.21.15.18:0/515327099 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4010670 con 0x7fa6c4102ac0 2024-04-16T17:01:23.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.132+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c41a6990 con 0x7fa6c4102ac0 2024-04-16T17:01:23.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.132+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c41a6da0 con 0x7fa6c4102ac0 2024-04-16T17:01:23.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.133+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c4109810 con 0x7fa6c4102ac0 2024-04-16T17:01:23.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.135+0000 7fa6baa00640 1 -- 172.21.15.18:0/515327099 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa6b40037a0 con 0x7fa6c4102ac0 2024-04-16T17:01:23.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.137+0000 7fa6baa00640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa6a80a3bb0 0x7fa6a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:23.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.137+0000 7fa6baa00640 1 -- 172.21.15.18:0/515327099 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fa6b40d0460 con 0x7fa6c4102ac0 2024-04-16T17:01:23.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.137+0000 7fa6c1400640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa6a80a3bb0 0x7fa6a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:23.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.138+0000 7fa6c1400640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa6a80a3bb0 0x7fa6a80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4194c80 tx=0x7fa6b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:23.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.140+0000 7fa6baa00640 1 -- 172.21.15.18:0/515327099 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6b409d0d0 con 0x7fa6c4102ac0 2024-04-16T17:01:23.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:22 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2676348879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:01:23.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:22 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:01:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:22 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2676348879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:01:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:22 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:01:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:22 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2676348879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:01:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:22 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:01:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:23.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:23.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:23.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:23.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.350+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa6c402d620 con 0x7fa6c4102ac0 2024-04-16T17:01:23.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.901+0000 7fa6baa00640 1 -- 172.21.15.18:0/515327099 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7fa6b409c9d0 con 0x7fa6c4102ac0 2024-04-16T17:01:23.906 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_data' created 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.910+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa6a80a3bb0 msgr2=0x7fa6a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.910+0000 7fa6c9000640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa6a80a3bb0 0x7fa6a80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4194c80 tx=0x7fa6b000b390 comp rx=0 tx=0).stop 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.910+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6c4102ac0 msgr2=0x7fa6c4196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.910+0000 7fa6c9000640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6c4102ac0 0x7fa6c4196d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400ea60 tx=0x7fa6b400ef20 comp rx=0 tx=0).stop 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.911+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 shutdown_connections 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.911+0000 7fa6c9000640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa6a80a3bb0 0x7fa6a80a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.911+0000 7fa6c9000640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6c4107d20 0x7fa6c4193870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.911+0000 7fa6c9000640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6c41053f0 0x7fa6c4197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.911+0000 7fa6c9000640 1 --2- 172.21.15.18:0/515327099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6c4102ac0 0x7fa6c4196d00 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400ea60 tx=0x7fa6b400ef20 comp rx=0 tx=0).stop 2024-04-16T17:01:23.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.911+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 >> 172.21.15.18:0/515327099 conn(0x7fa6c40fc4d0 msgr2=0x7fa6c4102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:23.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.913+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 shutdown_connections 2024-04-16T17:01:23.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:23.914+0000 7fa6c9000640 1 -- 172.21.15.18:0/515327099 wait complete. 2024-04-16T17:01:23.947 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-16T17:01:24.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 -- 172.21.15.18:0/1305751131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c0749e0 msgr2=0x7fb47c074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:24.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 --2- 172.21.15.18:0/1305751131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c0749e0 0x7fb47c074e40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb468009900 tx=0x7fb46802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 -- 172.21.15.18:0/1305751131 shutdown_connections 2024-04-16T17:01:24.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 --2- 172.21.15.18:0/1305751131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb47c109d40 0x7fb47c10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 --2- 172.21.15.18:0/1305751131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c0749e0 0x7fb47c074e40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 --2- 172.21.15.18:0/1305751131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb47c075fc0 0x7fb47c074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 -- 172.21.15.18:0/1305751131 >> 172.21.15.18:0/1305751131 conn(0x7fb47c0fd400 msgr2=0x7fb47c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:24.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 -- 172.21.15.18:0/1305751131 shutdown_connections 2024-04-16T17:01:24.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.137+0000 7fb482600640 1 -- 172.21.15.18:0/1305751131 wait complete. 2024-04-16T17:01:24.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb482600640 1 Processor -- start 2024-04-16T17:01:24.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb482600640 1 -- start start 2024-04-16T17:01:24.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb482600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb47c0749e0 0x7fb47c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb482600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c075fc0 0x7fb47c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb482600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb47c109d40 0x7fb47c1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb482600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10fd80 con 0x7fb47c109d40 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb482600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10fc00 con 0x7fb47c0749e0 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb482600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10ff00 con 0x7fb47c075fc0 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb481200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb47c0749e0 0x7fb47c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.139+0000 7fb47be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c075fc0 0x7fb47c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb481200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb47c0749e0 0x7fb47c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36568/0 (socket says 172.21.15.18:36568) 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb481200640 1 -- 172.21.15.18:0/3772155500 learned_addr learned my addr 172.21.15.18:0/3772155500 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb47be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c075fc0 0x7fb47c19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40146/0 (socket says 172.21.15.18:40146) 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb481c00640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb47c109d40 0x7fb47c1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb47be00640 1 -- 172.21.15.18:0/3772155500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb47c0749e0 msgr2=0x7fb47c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb47be00640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb47c0749e0 0x7fb47c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb47be00640 1 -- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb47c109d40 msgr2=0x7fb47c1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb47be00640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb47c109d40 0x7fb47c1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb47be00640 1 -- 172.21.15.18:0/3772155500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c1a1fc0 con 0x7fb47c075fc0 2024-04-16T17:01:24.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.140+0000 7fb47be00640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c075fc0 0x7fb47c19a8e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb468036040 tx=0x7fb468005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:24.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.141+0000 7fb479600640 1 -- 172.21.15.18:0/3772155500 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468043020 con 0x7fb47c075fc0 2024-04-16T17:01:24.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.141+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c1a2250 con 0x7fb47c075fc0 2024-04-16T17:01:24.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.141+0000 7fb479600640 1 -- 172.21.15.18:0/3772155500 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4680054e0 con 0x7fb47c075fc0 2024-04-16T17:01:24.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.141+0000 7fb479600640 1 -- 172.21.15.18:0/3772155500 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468003ca0 con 0x7fb47c075fc0 2024-04-16T17:01:24.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.141+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a2770 con 0x7fb47c075fc0 2024-04-16T17:01:24.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.142+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c108b00 con 0x7fb47c075fc0 2024-04-16T17:01:24.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.144+0000 7fb479600640 1 -- 172.21.15.18:0/3772155500 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb468046020 con 0x7fb47c075fc0 2024-04-16T17:01:24.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.145+0000 7fb479600640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4640a3bb0 0x7fb4640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:24.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.145+0000 7fb479600640 1 -- 172.21.15.18:0/3772155500 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fb4680f2380 con 0x7fb47c075fc0 2024-04-16T17:01:24.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.145+0000 7fb481200640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4640a3bb0 0x7fb4640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:24.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.146+0000 7fb481200640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4640a3bb0 0x7fb4640a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4700104e0 tx=0x7fb470009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:24.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.149+0000 7fb479600640 1 -- 172.21.15.18:0/3772155500 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb46804d020 con 0x7fb47c075fc0 2024-04-16T17:01:24.173 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:23 smithi018 ceph-mon[31776]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:24.173 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:23 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T17:01:24.173 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:23 smithi018 ceph-mon[31776]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T17:01:24.173 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:23 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/515327099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:01:24.173 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:23 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:01:24.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:23 smithi022 ceph-mon[36023]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:23 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T17:01:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:23 smithi022 ceph-mon[36023]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T17:01:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:23 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/515327099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:01:24.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:23 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:01:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:23 smithi043 ceph-mon[36901]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:23 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T17:01:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:23 smithi043 ceph-mon[36901]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T17:01:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:23 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/515327099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:01:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:23 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:01:24.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.395+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb47c062c00 con 0x7fb47c075fc0 2024-04-16T17:01:24.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.933+0000 7fb479600640 1 -- 172.21.15.18:0/3772155500 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb4680bee70 con 0x7fb47c075fc0 2024-04-16T17:01:24.935 INFO:teuthology.orchestra.run.smithi018.stderr:new fs with metadata pool 2 and data pool 3 2024-04-16T17:01:24.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.935+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4640a3bb0 msgr2=0x7fb4640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:24.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.935+0000 7fb482600640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4640a3bb0 0x7fb4640a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4700104e0 tx=0x7fb470009800 comp rx=0 tx=0).stop 2024-04-16T17:01:24.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c075fc0 msgr2=0x7fb47c19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:24.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c075fc0 0x7fb47c19a8e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb468036040 tx=0x7fb468005d50 comp rx=0 tx=0).stop 2024-04-16T17:01:24.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 shutdown_connections 2024-04-16T17:01:24.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb4640a3bb0 0x7fb4640a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb47c109d40 0x7fb47c1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb47c075fc0 0x7fb47c19a8e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 --2- 172.21.15.18:0/3772155500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb47c0749e0 0x7fb47c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:24.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 >> 172.21.15.18:0/3772155500 conn(0x7fb47c0fd400 msgr2=0x7fb47c105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:24.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.936+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 shutdown_connections 2024-04-16T17:01:24.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:24.937+0000 7fb482600640 1 -- 172.21.15.18:0/3772155500 wait complete. 2024-04-16T17:01:24.962 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-16T17:01:24.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1155784755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7ac101f60 msgr2=0x7fb7ac1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1155784755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7ac101f60 0x7fb7ac1023e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb798009900 tx=0x7fb79802bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1155784755 shutdown_connections 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1155784755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7ac102920 0x7fb7ac1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1155784755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7ac101f60 0x7fb7ac1023e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1155784755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7ac100d70 0x7fb7ac101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1155784755 >> 172.21.15.18:0/1155784755 conn(0x7fb7ac0fc2e0 msgr2=0x7fb7ac0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1155784755 shutdown_connections 2024-04-16T17:01:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.172+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1155784755 wait complete. 2024-04-16T17:01:25.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3772155500' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:24 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.173+0000 7fb7b0c00640 1 Processor -- start 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.173+0000 7fb7b0c00640 1 -- start start 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.173+0000 7fb7b0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7ac100d70 0x7fb7ac104e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7b0c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7ac101f60 0x7fb7ac103490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7b0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7ac102920 0x7fb7ac1039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7b0c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac10c9d0 con 0x7fb7ac101f60 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7b0c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac10c850 con 0x7fb7ac102920 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7b0c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac10cb50 con 0x7fb7ac100d70 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7aaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7ac101f60 0x7fb7ac103490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7ac100d70 0x7fb7ac104e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7aaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7ac101f60 0x7fb7ac103490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58576/0 (socket says 172.21.15.18:58576) 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7aaa00640 1 -- 172.21.15.18:0/1367471341 learned_addr learned my addr 172.21.15.18:0/1367471341 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7abe00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7ac102920 0x7fb7ac1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7ab400640 1 -- 172.21.15.18:0/1367471341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7ac102920 msgr2=0x7fb7ac1039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7ab400640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7ac102920 0x7fb7ac1039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7ab400640 1 -- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7ac101f60 msgr2=0x7fb7ac103490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7ab400640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7ac101f60 0x7fb7ac103490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.174+0000 7fb7ab400640 1 -- 172.21.15.18:0/1367471341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac104260 con 0x7fb7ac100d70 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.175+0000 7fb7ab400640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7ac100d70 0x7fb7ac104e50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00e9c0 tx=0x7fb79c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.175+0000 7fb7a3e00640 1 -- 172.21.15.18:0/1367471341 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00ccc0 con 0x7fb7ac100d70 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.175+0000 7fb7a3e00640 1 -- 172.21.15.18:0/1367471341 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79c0035f0 con 0x7fb7ac100d70 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.175+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ac1a22b0 con 0x7fb7ac100d70 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.175+0000 7fb7a3e00640 1 -- 172.21.15.18:0/1367471341 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00ce60 con 0x7fb7ac100d70 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.175+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ac1a27d0 con 0x7fb7ac100d70 2024-04-16T17:01:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.177+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ac04fd90 con 0x7fb7ac100d70 2024-04-16T17:01:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.178+0000 7fb7a3e00640 1 -- 172.21.15.18:0/1367471341 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb79c0037a0 con 0x7fb7ac100d70 2024-04-16T17:01:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.180+0000 7fb7a3e00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7900a3bb0 0x7fb7900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.180+0000 7fb7a3e00640 1 -- 172.21.15.18:0/1367471341 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fb79c0d06b0 con 0x7fb7ac100d70 2024-04-16T17:01:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.180+0000 7fb7aaa00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7900a3bb0 0x7fb7900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:25.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.181+0000 7fb7aaa00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7900a3bb0 0x7fb7900a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb798000c00 tx=0x7fb79802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:25.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.184+0000 7fb7a3e00640 1 -- 172.21.15.18:0/1367471341 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb79c09d140 con 0x7fb7ac100d70 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3772155500' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:24 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3772155500' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:24 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:25.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.418+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fb7ac1a30c0 con 0x7fb7ac100d70 2024-04-16T17:01:25.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.527+0000 7fb7a3e00640 1 -- 172.21.15.18:0/1367471341 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7fb79c09ca40 con 0x7fb7ac100d70 2024-04-16T17:01:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7900a3bb0 msgr2=0x7fb7900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7900a3bb0 0x7fb7900a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb798000c00 tx=0x7fb79802c880 comp rx=0 tx=0).stop 2024-04-16T17:01:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7ac100d70 msgr2=0x7fb7ac104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7ac100d70 0x7fb7ac104e50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00e9c0 tx=0x7fb79c00edc0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 shutdown_connections 2024-04-16T17:01:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7ac102920 0x7fb7ac1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7900a3bb0 0x7fb7900a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7ac101f60 0x7fb7ac103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 --2- 172.21.15.18:0/1367471341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7ac100d70 0x7fb7ac104e50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.530+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 >> 172.21.15.18:0/1367471341 conn(0x7fb7ac0fc2e0 msgr2=0x7fb7ac069300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.531+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 shutdown_connections 2024-04-16T17:01:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.531+0000 7fb7b0c00640 1 -- 172.21.15.18:0/1367471341 wait complete. 2024-04-16T17:01:25.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-16T17:01:25.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.759+0000 7f81ed800640 1 -- 172.21.15.18:0/4091118368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 msgr2=0x7f81e8110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:25.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.759+0000 7f81ed800640 1 --2- 172.21.15.18:0/4091118368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 0x7f81e8110990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009be0 tx=0x7f81e002bef0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.760+0000 7f81ed800640 1 -- 172.21.15.18:0/4091118368 shutdown_connections 2024-04-16T17:01:25.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.760+0000 7f81ed800640 1 --2- 172.21.15.18:0/4091118368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 0x7f81e8110990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.760+0000 7f81ed800640 1 --2- 172.21.15.18:0/4091118368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81e8109740 0x7f81e8109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.760+0000 7f81ed800640 1 --2- 172.21.15.18:0/4091118368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81085a0 0x7f81e81089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.760+0000 7f81ed800640 1 -- 172.21.15.18:0/4091118368 >> 172.21.15.18:0/4091118368 conn(0x7f81e80774f0 msgr2=0x7f81e8079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:25.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.760+0000 7f81ed800640 1 -- 172.21.15.18:0/4091118368 shutdown_connections 2024-04-16T17:01:25.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.760+0000 7f81ed800640 1 -- 172.21.15.18:0/4091118368 wait complete. 2024-04-16T17:01:25.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.761+0000 7f81ed800640 1 Processor -- start 2024-04-16T17:01:25.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.761+0000 7f81ed800640 1 -- start start 2024-04-16T17:01:25.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.761+0000 7f81ed800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81e81085a0 0x7f81e819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:25.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.761+0000 7f81ed800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e8109740 0x7f81e819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:25.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.761+0000 7f81ed800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 0x7f81e81a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:25.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.761+0000 7f81ed800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f81e81141e0 con 0x7f81e81085a0 2024-04-16T17:01:25.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.761+0000 7f81ed800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8114060 con 0x7f81e8109740 2024-04-16T17:01:25.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.761+0000 7f81ed800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8114360 con 0x7f81e810a100 2024-04-16T17:01:25.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81e7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81e81085a0 0x7f81e819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:25.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81e7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81e81085a0 0x7f81e819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58596/0 (socket says 172.21.15.18:58596) 2024-04-16T17:01:25.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81e7e00640 1 -- 172.21.15.18:0/1123981460 learned_addr learned my addr 172.21.15.18:0/1123981460 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:25.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81ece00640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 0x7f81e81a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:25.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81e7400640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e8109740 0x7f81e819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81ece00640 1 -- 172.21.15.18:0/1123981460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e8109740 msgr2=0x7f81e819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81ece00640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e8109740 0x7f81e819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81ece00640 1 -- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81e81085a0 msgr2=0x7f81e819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81ece00640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81e81085a0 0x7f81e819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81ece00640 1 -- 172.21.15.18:0/1123981460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e81a63f0 con 0x7f81e810a100 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.762+0000 7f81e7e00640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81e81085a0 0x7f81e819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.763+0000 7f81ece00640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 0x7f81e81a5d80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81e002c430 tx=0x7f81e002cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.763+0000 7f81e4c00640 1 -- 172.21.15.18:0/1123981460 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0043020 con 0x7f81e810a100 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.763+0000 7f81e4c00640 1 -- 172.21.15.18:0/1123981460 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81e0003e50 con 0x7f81e810a100 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.763+0000 7f81e4c00640 1 -- 172.21.15.18:0/1123981460 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0034430 con 0x7f81e810a100 2024-04-16T17:01:25.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.763+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e81a6680 con 0x7f81e810a100 2024-04-16T17:01:25.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.763+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e81a6b80 con 0x7f81e810a100 2024-04-16T17:01:25.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.764+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e810f7a0 con 0x7f81e810a100 2024-04-16T17:01:25.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.766+0000 7f81e4c00640 1 -- 172.21.15.18:0/1123981460 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81e0046020 con 0x7f81e810a100 2024-04-16T17:01:25.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.768+0000 7f81e4c00640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f81cc0a3ba0 0x7f81cc0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:25.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.768+0000 7f81e4c00640 1 -- 172.21.15.18:0/1123981460 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f81e00f3180 con 0x7f81e810a100 2024-04-16T17:01:25.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.768+0000 7f81e7e00640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f81cc0a3ba0 0x7f81cc0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:25.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.769+0000 7f81e7e00640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f81cc0a3ba0 0x7f81cc0a6060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81d80095f0 tx=0x7f81d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:25.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.770+0000 7f81e4c00640 1 -- 172.21.15.18:0/1123981460 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81e004d020 con 0x7f81e810a100 2024-04-16T17:01:25.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:25.984+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f81e802d620 con 0x7f81e810a100 2024-04-16T17:01:26.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: pgmap v314: 97 pgs: 9 creating+peering, 12 active+clean, 76 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:26.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T17:01:26.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T17:01:26.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T17:01:26.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1367471341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:01:26.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:01:26.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:26.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:26.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T17:01:26.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:25 smithi018 ceph-mon[31776]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T17:01:26.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: pgmap v314: 97 pgs: 9 creating+peering, 12 active+clean, 76 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1367471341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T17:01:26.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:25 smithi043 ceph-mon[36901]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: pgmap v314: 97 pgs: 9 creating+peering, 12 active+clean, 76 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1367471341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T17:01:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:25 smithi022 ceph-mon[36023]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T17:01:27.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:26 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] up:active 2024-04-16T17:01:27.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:26 smithi018 ceph-mon[31776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:27.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:26 smithi018 ceph-mon[31776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:27.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:26 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1123981460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:01:27.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:26 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:01:27.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:26 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] up:active 2024-04-16T17:01:27.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:26 smithi043 ceph-mon[36901]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:27.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:26 smithi043 ceph-mon[36901]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:27.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:26 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1123981460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:01:27.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:26 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:01:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:26 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] up:active 2024-04-16T17:01:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:26 smithi022 ceph-mon[36023]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:26 smithi022 ceph-mon[36023]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:26 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1123981460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:01:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:26 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:01:27.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.985+0000 7f81e4c00640 1 -- 172.21.15.18:0/1123981460 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f81e00bfad0 con 0x7f81e810a100 2024-04-16T17:01:27.988 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_data_ec' created 2024-04-16T17:01:27.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f81cc0a3ba0 msgr2=0x7f81cc0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:27.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f81cc0a3ba0 0x7f81cc0a6060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81d80095f0 tx=0x7f81d8005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:27.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 msgr2=0x7f81e81a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:27.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 0x7f81e81a5d80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81e002c430 tx=0x7f81e002cb90 comp rx=0 tx=0).stop 2024-04-16T17:01:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 shutdown_connections 2024-04-16T17:01:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81e810a100 0x7f81e81a5d80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e8109740 0x7f81e819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f81cc0a3ba0 0x7f81cc0a6060 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81d80095f0 tx=0x7f81d8005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.992+0000 7f81ed800640 1 --2- 172.21.15.18:0/1123981460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81e81085a0 0x7f81e819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.993+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 >> 172.21.15.18:0/1123981460 conn(0x7f81e80774f0 msgr2=0x7f81e810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.993+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 shutdown_connections 2024-04-16T17:01:27.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:27.993+0000 7f81ed800640 1 -- 172.21.15.18:0/1123981460 wait complete. 2024-04-16T17:01:28.022 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-16T17:01:28.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:27 smithi018 ceph-mon[31776]: pgmap v316: 97 pgs: 9 creating+peering, 49 active+clean, 39 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-04-16T17:01:28.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:27 smithi018 ceph-mon[31776]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T17:01:28.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 -- 172.21.15.18:0/3183635440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 msgr2=0x7f944c078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 --2- 172.21.15.18:0/3183635440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 0x7f944c078060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9438009900 tx=0x7f943802bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 -- 172.21.15.18:0/3183635440 shutdown_connections 2024-04-16T17:01:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 --2- 172.21.15.18:0/3183635440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944c0785a0 0x7f944c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 --2- 172.21.15.18:0/3183635440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 0x7f944c078060 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 --2- 172.21.15.18:0/3183635440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944c0769c0 0x7f944c076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 -- 172.21.15.18:0/3183635440 >> 172.21.15.18:0/3183635440 conn(0x7f944c0ff610 msgr2=0x7f944c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 -- 172.21.15.18:0/3183635440 shutdown_connections 2024-04-16T17:01:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.220+0000 7f9451800640 1 -- 172.21.15.18:0/3183635440 wait complete. 2024-04-16T17:01:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.221+0000 7f9451800640 1 Processor -- start 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.221+0000 7f9451800640 1 -- start start 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.221+0000 7f9451800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944c0769c0 0x7f944c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f9451800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 0x7f944c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f9451800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944c0785a0 0x7f944c1a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f9451800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f944c114150 con 0x7f944c0785a0 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f9451800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f944c113fd0 con 0x7f944c0769c0 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f9451800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1142d0 con 0x7f944c077c00 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 0x7f944c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944c0769c0 0x7f944c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 0x7f944c19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41524/0 (socket says 172.21.15.18:41524) 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944b400640 1 -- 172.21.15.18:0/2502262702 learned_addr learned my addr 172.21.15.18:0/2502262702 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f9450e00640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944c0785a0 0x7f944c1a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944be00640 1 -- 172.21.15.18:0/2502262702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 msgr2=0x7f944c19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944be00640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 0x7f944c19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944be00640 1 -- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944c0785a0 msgr2=0x7f944c1a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944be00640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944c0785a0 0x7f944c1a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944be00640 1 -- 172.21.15.18:0/2502262702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c1a6390 con 0x7f944c0769c0 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f944b400640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 0x7f944c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.222+0000 7f9450e00640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944c0785a0 0x7f944c1a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.223+0000 7f944be00640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944c0769c0 0x7f944c19e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f943c00e9f0 tx=0x7f943c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.223+0000 7f9448c00640 1 -- 172.21.15.18:0/2502262702 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c00ccc0 con 0x7f944c0769c0 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.223+0000 7f9448c00640 1 -- 172.21.15.18:0/2502262702 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943c010910 con 0x7f944c0769c0 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.223+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c1a6680 con 0x7f944c0769c0 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.223+0000 7f9448c00640 1 -- 172.21.15.18:0/2502262702 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c018960 con 0x7f944c0769c0 2024-04-16T17:01:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.223+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c1a6ba0 con 0x7f944c0769c0 2024-04-16T17:01:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.224+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c1109f0 con 0x7f944c0769c0 2024-04-16T17:01:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.226+0000 7f9448c00640 1 -- 172.21.15.18:0/2502262702 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f943c010ac0 con 0x7f944c0769c0 2024-04-16T17:01:28.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.227+0000 7f9448c00640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f94300a3bb0 0x7f94300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:28.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.228+0000 7f9448c00640 1 -- 172.21.15.18:0/2502262702 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f943c0d0eb0 con 0x7f944c0769c0 2024-04-16T17:01:28.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.228+0000 7f944b400640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f94300a3bb0 0x7f94300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.228+0000 7f944b400640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f94300a3bb0 0x7f94300a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9438006010 tx=0x7f943802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.230+0000 7f9448c00640 1 -- 172.21.15.18:0/2502262702 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f943c09cfc0 con 0x7f944c0769c0 2024-04-16T17:01:28.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:27 smithi043 ceph-mon[36901]: pgmap v316: 97 pgs: 9 creating+peering, 49 active+clean, 39 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-04-16T17:01:28.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:27 smithi043 ceph-mon[36901]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T17:01:28.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:27 smithi022 ceph-mon[36023]: pgmap v316: 97 pgs: 9 creating+peering, 49 active+clean, 39 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-04-16T17:01:28.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:27 smithi022 ceph-mon[36023]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T17:01:28.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.435+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f944c02d620 con 0x7f944c0769c0 2024-04-16T17:01:28.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.989+0000 7f9448c00640 1 -- 172.21.15.18:0/2502262702 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f943c0a12c0 con 0x7f944c0769c0 2024-04-16T17:01:28.991 INFO:teuthology.orchestra.run.smithi018.stderr:set pool 4 allow_ec_overwrites to true 2024-04-16T17:01:28.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.993+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f94300a3bb0 msgr2=0x7f94300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.993+0000 7f9451800640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f94300a3bb0 0x7f94300a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9438006010 tx=0x7f943802cf90 comp rx=0 tx=0).stop 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944c0769c0 msgr2=0x7f944c19e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944c0769c0 0x7f944c19e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f943c00e9f0 tx=0x7f943c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 shutdown_connections 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f94300a3bb0 0x7f94300a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f944c0785a0 0x7f944c1a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f944c077c00 0x7f944c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 --2- 172.21.15.18:0/2502262702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f944c0769c0 0x7f944c19e770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 >> 172.21.15.18:0/2502262702 conn(0x7f944c0ff610 msgr2=0x7f944c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 shutdown_connections 2024-04-16T17:01:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:28.994+0000 7f9451800640 1 -- 172.21.15.18:0/2502262702 wait complete. 2024-04-16T17:01:29.035 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-16T17:01:29.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.222+0000 7fdf41e00640 1 -- 172.21.15.18:0/4231009563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105450 msgr2=0x7fdf3c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:29.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.222+0000 7fdf41e00640 1 --2- 172.21.15.18:0/4231009563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105450 0x7fdf3c1058d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009900 tx=0x7fdf2402bc50 comp rx=0 tx=0).stop 2024-04-16T17:01:29.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.222+0000 7fdf41e00640 1 -- 172.21.15.18:0/4231009563 shutdown_connections 2024-04-16T17:01:29.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.222+0000 7fdf41e00640 1 --2- 172.21.15.18:0/4231009563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf3c105e10 0x7fdf3c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.222+0000 7fdf41e00640 1 --2- 172.21.15.18:0/4231009563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105450 0x7fdf3c1058d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.222+0000 7fdf41e00640 1 --2- 172.21.15.18:0/4231009563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf3c104260 0x7fdf3c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.222+0000 7fdf41e00640 1 -- 172.21.15.18:0/4231009563 >> 172.21.15.18:0/4231009563 conn(0x7fdf3c0ff7f0 msgr2=0x7fdf3c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:29.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.223+0000 7fdf41e00640 1 -- 172.21.15.18:0/4231009563 shutdown_connections 2024-04-16T17:01:29.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.223+0000 7fdf41e00640 1 -- 172.21.15.18:0/4231009563 wait complete. 2024-04-16T17:01:29.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.223+0000 7fdf41e00640 1 Processor -- start 2024-04-16T17:01:29.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf41e00640 1 -- start start 2024-04-16T17:01:29.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf41e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf3c104260 0x7fdf3c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:29.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf41e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf3c105450 0x7fdf3c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:29.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf41e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105e10 0x7fdf3c1a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:29.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf41e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c10ff40 con 0x7fdf3c104260 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf41e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c10fdc0 con 0x7fdf3c105450 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf41e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c1100c0 con 0x7fdf3c105e10 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf3aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf3c104260 0x7fdf3c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf3aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf3c104260 0x7fdf3c19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32848/0 (socket says 172.21.15.18:32848) 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf3b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105e10 0x7fdf3c1a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf3aa00640 1 -- 172.21.15.18:0/3756428209 learned_addr learned my addr 172.21.15.18:0/3756428209 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.224+0000 7fdf33e00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf3c105450 0x7fdf3c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf3b400640 1 -- 172.21.15.18:0/3756428209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf3c105450 msgr2=0x7fdf3c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf3b400640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf3c105450 0x7fdf3c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf3b400640 1 -- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf3c104260 msgr2=0x7fdf3c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf3b400640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf3c104260 0x7fdf3c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf3b400640 1 -- 172.21.15.18:0/3756428209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c1a2190 con 0x7fdf3c105e10 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf33e00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf3c105450 0x7fdf3c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:01:29.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf3b400640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105e10 0x7fdf3c1a1b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009fd0 tx=0x7fdf3400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf40a00640 1 -- 172.21.15.18:0/3756428209 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400ce20 con 0x7fdf3c105e10 2024-04-16T17:01:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf40a00640 1 -- 172.21.15.18:0/3756428209 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf34005020 con 0x7fdf3c105e10 2024-04-16T17:01:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3c1a2480 con 0x7fdf3c105e10 2024-04-16T17:01:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf40a00640 1 -- 172.21.15.18:0/3756428209 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34010720 con 0x7fdf3c105e10 2024-04-16T17:01:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.225+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3c1a2950 con 0x7fdf3c105e10 2024-04-16T17:01:29.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.227+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3c10b580 con 0x7fdf3c105e10 2024-04-16T17:01:29.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.228+0000 7fdf40a00640 1 -- 172.21.15.18:0/3756428209 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdf34005540 con 0x7fdf3c105e10 2024-04-16T17:01:29.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.230+0000 7fdf40a00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf200a3bb0 0x7fdf200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:29.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.230+0000 7fdf40a00640 1 -- 172.21.15.18:0/3756428209 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fdf340d0de0 con 0x7fdf3c105e10 2024-04-16T17:01:29.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.230+0000 7fdf3aa00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf200a3bb0 0x7fdf200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:29.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.231+0000 7fdf3aa00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf200a3bb0 0x7fdf200a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c005fd0 tx=0x7fdf2c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:29.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.234+0000 7fdf40a00640 1 -- 172.21.15.18:0/3756428209 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf3409d540 con 0x7fdf3c105e10 2024-04-16T17:01:29.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:29 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T17:01:29.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:29 smithi043 ceph-mon[36901]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T17:01:29.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:29 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2502262702' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:01:29.417 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:29 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:01:29.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:29 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T17:01:29.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:29 smithi022 ceph-mon[36023]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T17:01:29.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:29 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2502262702' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:01:29.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:29 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:01:29.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:29 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T17:01:29.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:29 smithi018 ceph-mon[31776]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T17:01:29.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:29 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2502262702' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:01:29.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:29 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:01:29.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:29.485+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fdf3c062c00 con 0x7fdf3c105e10 2024-04-16T17:01:30.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.155+0000 7fdf40a00640 1 -- 172.21.15.18:0/3756428209 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fdf3409ce40 con 0x7fdf3c105e10 2024-04-16T17:01:30.157 INFO:teuthology.orchestra.run.smithi018.stderr:added data pool 4 to fsmap 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf200a3bb0 msgr2=0x7fdf200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf200a3bb0 0x7fdf200a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c005fd0 tx=0x7fdf2c00b410 comp rx=0 tx=0).stop 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105e10 msgr2=0x7fdf3c1a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105e10 0x7fdf3c1a1b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009fd0 tx=0x7fdf3400ef10 comp rx=0 tx=0).stop 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 shutdown_connections 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c105e10 0x7fdf3c1a1b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf3c105450 0x7fdf3c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf200a3bb0 0x7fdf200a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 --2- 172.21.15.18:0/3756428209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf3c104260 0x7fdf3c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 >> 172.21.15.18:0/3756428209 conn(0x7fdf3c0ff7f0 msgr2=0x7fdf3c108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:30.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 shutdown_connections 2024-04-16T17:01:30.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.159+0000 7fdf41e00640 1 -- 172.21.15.18:0/3756428209 wait complete. 2024-04-16T17:01:30.175 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: 2024-04-16T17:01:29.857+0000 7f198be63700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-16T17:01:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:30 smithi022 ceph-mon[36023]: pgmap v319: 129 pgs: 12 creating+peering, 91 active+clean, 26 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-16T17:01:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:30 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T17:01:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:30 smithi022 ceph-mon[36023]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T17:01:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:30 smithi022 ceph-mon[36023]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:01:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:30 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3756428209' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:01:30.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:30 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:01:30.187 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:30 smithi018 ceph-mon[31776]: pgmap v319: 129 pgs: 12 creating+peering, 91 active+clean, 26 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-16T17:01:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:30 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T17:01:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:30 smithi018 ceph-mon[31776]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T17:01:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:30 smithi018 ceph-mon[31776]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:01:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:30 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3756428209' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:01:30.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:30 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:01:30.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.414+0000 7f0e9b800640 1 -- 172.21.15.18:0/3705903277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9410a230 msgr2=0x7f0e94110ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.414+0000 7f0e9b800640 1 --2- 172.21.15.18:0/3705903277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9410a230 0x7f0e94110ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009ca0 tx=0x7f0e8c02c100 comp rx=0 tx=0).stop 2024-04-16T17:01:30.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.414+0000 7f0e9b800640 1 -- 172.21.15.18:0/3705903277 shutdown_connections 2024-04-16T17:01:30.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.414+0000 7f0e9b800640 1 --2- 172.21.15.18:0/3705903277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9410a230 0x7f0e94110ac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.414+0000 7f0e9b800640 1 --2- 172.21.15.18:0/3705903277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e94109870 0x7f0e94109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.414+0000 7f0e9b800640 1 --2- 172.21.15.18:0/3705903277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e94108680 0x7f0e94108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.414+0000 7f0e9b800640 1 -- 172.21.15.18:0/3705903277 >> 172.21.15.18:0/3705903277 conn(0x7f0e94077650 msgr2=0x7f0e94079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.414+0000 7f0e9b800640 1 -- 172.21.15.18:0/3705903277 shutdown_connections 2024-04-16T17:01:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.415+0000 7f0e9b800640 1 -- 172.21.15.18:0/3705903277 wait complete. 2024-04-16T17:01:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.415+0000 7f0e9b800640 1 Processor -- start 2024-04-16T17:01:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e9b800640 1 -- start start 2024-04-16T17:01:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e9b800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e94108680 0x7f0e9419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e9b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e94109870 0x7f0e9419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e9b800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e9410a230 0x7f0e941a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e9b800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0e94114300 con 0x7f0e94108680 2024-04-16T17:01:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e9b800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0e94114180 con 0x7f0e9410a230 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e9b800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e94114480 con 0x7f0e94109870 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e98e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e94108680 0x7f0e9419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e98e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e94108680 0x7f0e9419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32862/0 (socket says 172.21.15.18:32862) 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e98e00640 1 -- 172.21.15.18:0/1232549538 learned_addr learned my addr 172.21.15.18:0/1232549538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e8be00640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e94109870 0x7f0e9419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.416+0000 7f0e99800640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e9410a230 0x7f0e941a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e98e00640 1 -- 172.21.15.18:0/1232549538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e94109870 msgr2=0x7f0e9419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e98e00640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e94109870 0x7f0e9419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e98e00640 1 -- 172.21.15.18:0/1232549538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e9410a230 msgr2=0x7f0e941a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e98e00640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e9410a230 0x7f0e941a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e98e00640 1 -- 172.21.15.18:0/1232549538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e941a65d0 con 0x7f0e94108680 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e98e00640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e94108680 0x7f0e9419e9b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e9c0 tx=0x7f0e8400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e92000640 1 -- 172.21.15.18:0/1232549538 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400cd80 con 0x7f0e94108680 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e941a68c0 con 0x7f0e94108680 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e92000640 1 -- 172.21.15.18:0/1232549538 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e840035f0 con 0x7f0e94108680 2024-04-16T17:01:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e941a6e10 con 0x7f0e94108680 2024-04-16T17:01:30.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.417+0000 7f0e92000640 1 -- 172.21.15.18:0/1232549538 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84010630 con 0x7f0e94108680 2024-04-16T17:01:30.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.419+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9410f940 con 0x7f0e94108680 2024-04-16T17:01:30.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.421+0000 7f0e92000640 1 -- 172.21.15.18:0/1232549538 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0e840037a0 con 0x7f0e94108680 2024-04-16T17:01:30.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.422+0000 7f0e92000640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0e780a3bb0 0x7f0e780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:30.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.422+0000 7f0e92000640 1 -- 172.21.15.18:0/1232549538 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f0e840d0bb0 con 0x7f0e94108680 2024-04-16T17:01:30.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.422+0000 7f0e8be00640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0e780a3bb0 0x7f0e780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:30.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.423+0000 7f0e8be00640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0e780a3bb0 0x7f0e780a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0095f0 tx=0x7f0e7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:30.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.427+0000 7f0e92000640 1 -- 172.21.15.18:0/1232549538 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e8409d350 con 0x7f0e94108680 2024-04-16T17:01:30.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:30 smithi043 ceph-mon[36901]: pgmap v319: 129 pgs: 12 creating+peering, 91 active+clean, 26 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-16T17:01:30.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:30 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T17:01:30.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:30 smithi043 ceph-mon[36901]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T17:01:30.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:30 smithi043 ceph-mon[36901]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:01:30.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:30 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3756428209' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:01:30.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:30 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:01:30.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.679+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e9402d620 con 0x7f0e94108680 2024-04-16T17:01:30.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.681+0000 7f0e92000640 1 -- 172.21.15.18:0/1232549538 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11776 (secure 0 0 0) 0x7f0e8409cc50 con 0x7f0e94108680 2024-04-16T17:01:30.683 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:30.683 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:01:29.488436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-16T17:01:30.683 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 17 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0e780a3bb0 msgr2=0x7f0e780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0e780a3bb0 0x7f0e780a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0095f0 tx=0x7f0e7c005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e94108680 msgr2=0x7f0e9419e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e94108680 0x7f0e9419e9b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e9c0 tx=0x7f0e8400ee80 comp rx=0 tx=0).stop 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 shutdown_connections 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e9410a230 0x7f0e941a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e94109870 0x7f0e9419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f0e780a3bb0 0x7f0e780a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 --2- 172.21.15.18:0/1232549538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e94108680 0x7f0e9419e9b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 >> 172.21.15.18:0/1232549538 conn(0x7f0e94077650 msgr2=0x7f0e9410cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:30.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 shutdown_connections 2024-04-16T17:01:30.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.684+0000 7f0e9b800640 1 -- 172.21.15.18:0/1232549538 wait complete. 2024-04-16T17:01:30.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 -- 172.21.15.18:0/4051508750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 msgr2=0x7f376c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 --2- 172.21.15.18:0/4051508750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 0x7f376c109cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3758009860 tx=0x7f375802bb70 comp rx=0 tx=0).stop 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 -- 172.21.15.18:0/4051508750 shutdown_connections 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 --2- 172.21.15.18:0/4051508750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376c10a230 0x7f376c110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 --2- 172.21.15.18:0/4051508750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 0x7f376c109cf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 --2- 172.21.15.18:0/4051508750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c108680 0x7f376c108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 -- 172.21.15.18:0/4051508750 >> 172.21.15.18:0/4051508750 conn(0x7f376c077650 msgr2=0x7f376c079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 -- 172.21.15.18:0/4051508750 shutdown_connections 2024-04-16T17:01:30.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.931+0000 7f3771200640 1 -- 172.21.15.18:0/4051508750 wait complete. 2024-04-16T17:01:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.932+0000 7f3771200640 1 Processor -- start 2024-04-16T17:01:30.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.932+0000 7f3771200640 1 -- start start 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.932+0000 7f3771200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376c108680 0x7f376c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.932+0000 7f3771200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 0x7f376c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.932+0000 7f3771200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c10a230 0x7f376c1a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.932+0000 7f3771200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1142e0 con 0x7f376c109870 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.932+0000 7f3771200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f376c114160 con 0x7f376c10a230 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.932+0000 7f3771200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f376c114460 con 0x7f376c108680 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 0x7f376c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 0x7f376c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32880/0 (socket says 172.21.15.18:32880) 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769400640 1 -- 172.21.15.18:0/2379600395 learned_addr learned my addr 172.21.15.18:0/2379600395 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376c108680 0x7f376c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f376a800640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c10a230 0x7f376c1a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769e00640 1 -- 172.21.15.18:0/2379600395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c10a230 msgr2=0x7f376c1a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769e00640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c10a230 0x7f376c1a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769e00640 1 -- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 msgr2=0x7f376c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769e00640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 0x7f376c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769e00640 1 -- 172.21.15.18:0/2379600395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c1a6580 con 0x7f376c108680 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769400640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 0x7f376c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f376a800640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c10a230 0x7f376c1a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.933+0000 7f3769e00640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376c108680 0x7f376c19e960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f376c1096d0 tx=0x7f375c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.934+0000 7f3762a00640 1 -- 172.21.15.18:0/2379600395 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c004440 con 0x7f376c108680 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.934+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c1a6870 con 0x7f376c108680 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.934+0000 7f3762a00640 1 -- 172.21.15.18:0/2379600395 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c00cda0 con 0x7f376c108680 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.934+0000 7f3762a00640 1 -- 172.21.15.18:0/2379600395 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0196e0 con 0x7f376c108680 2024-04-16T17:01:30.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.934+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c1a6dc0 con 0x7f376c108680 2024-04-16T17:01:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.935+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c10f9a0 con 0x7f376c108680 2024-04-16T17:01:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.937+0000 7f3762a00640 1 -- 172.21.15.18:0/2379600395 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f375c003780 con 0x7f376c108680 2024-04-16T17:01:30.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.938+0000 7f3762a00640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37500a3bb0 0x7f37500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:30.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.939+0000 7f3762a00640 1 -- 172.21.15.18:0/2379600395 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f375c0d0bc0 con 0x7f376c108680 2024-04-16T17:01:30.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.939+0000 7f3769400640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37500a3bb0 0x7f37500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:30.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.939+0000 7f3769400640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37500a3bb0 0x7f37500a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f376c19fe70 tx=0x7f375802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:30.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:30.941+0000 7f3762a00640 1 -- 172.21.15.18:0/2379600395 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f375c003e20 con 0x7f376c108680 2024-04-16T17:01:31.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.153+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f376c02d620 con 0x7f376c108680 2024-04-16T17:01:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.161+0000 7f3762a00640 1 -- 172.21.15.18:0/2379600395 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7f375c009510 con 0x7f376c108680 2024-04-16T17:01:31.164 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:31.164 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:01:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37500a3bb0 msgr2=0x7f37500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37500a3bb0 0x7f37500a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f376c19fe70 tx=0x7f375802e040 comp rx=0 tx=0).stop 2024-04-16T17:01:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376c108680 msgr2=0x7f376c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376c108680 0x7f376c19e960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f376c1096d0 tx=0x7f375c00ee50 comp rx=0 tx=0).stop 2024-04-16T17:01:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 shutdown_connections 2024-04-16T17:01:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f376c10a230 0x7f376c1a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f37500a3bb0 0x7f37500a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f376c109870 0x7f376c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 --2- 172.21.15.18:0/2379600395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376c108680 0x7f376c19e960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.164+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 >> 172.21.15.18:0/2379600395 conn(0x7f376c077650 msgr2=0x7f376c10cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.165+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 shutdown_connections 2024-04-16T17:01:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.165+0000 7f3771200640 1 -- 172.21.15.18:0/2379600395 wait complete. 2024-04-16T17:01:31.186 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:01:31.268 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:31 smithi018 ceph-mon[31776]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T17:01:31.268 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:31 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T17:01:31.268 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:31 smithi018 ceph-mon[31776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:31.268 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:31 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1232549538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 -- 172.21.15.18:0/308232744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0124100e30 msgr2=0x7f0124101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 --2- 172.21.15.18:0/308232744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0124100e30 0x7f0124101250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f010c009900 tx=0x7f010c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 -- 172.21.15.18:0/308232744 shutdown_connections 2024-04-16T17:01:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 --2- 172.21.15.18:0/308232744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124102940 0x7f0124109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 --2- 172.21.15.18:0/308232744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0124101f80 0x7f0124102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 --2- 172.21.15.18:0/308232744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0124100e30 0x7f0124101250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 -- 172.21.15.18:0/308232744 >> 172.21.15.18:0/308232744 conn(0x7f01240fc3a0 msgr2=0x7f01240fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:31.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 -- 172.21.15.18:0/308232744 shutdown_connections 2024-04-16T17:01:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.411+0000 7f012a000640 1 -- 172.21.15.18:0/308232744 wait complete. 2024-04-16T17:01:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f012a000640 1 Processor -- start 2024-04-16T17:01:31.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f012a000640 1 -- start start 2024-04-16T17:01:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f012a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0124100e30 0x7f012419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f012a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0124101f80 0x7f012419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f012a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124102940 0x7f01241a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f012a000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f012410cb00 con 0x7f0124102940 2024-04-16T17:01:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f012a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f012410c980 con 0x7f0124101f80 2024-04-16T17:01:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f012a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f012410cc80 con 0x7f0124100e30 2024-04-16T17:01:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.413+0000 7f0128c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0124100e30 0x7f012419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0129600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124102940 0x7f01241a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0128c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0124100e30 0x7f012419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41590/0 (socket says 172.21.15.18:41590) 2024-04-16T17:01:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0128c00640 1 -- 172.21.15.18:0/39320441 learned_addr learned my addr 172.21.15.18:0/39320441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:31.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f011be00640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0124101f80 0x7f012419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0128c00640 1 -- 172.21.15.18:0/39320441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0124101f80 msgr2=0x7f012419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0128c00640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0124101f80 0x7f012419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0128c00640 1 -- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124102940 msgr2=0x7f01241a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0128c00640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124102940 0x7f01241a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0128c00640 1 -- 172.21.15.18:0/39320441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01241a2030 con 0x7f0124100e30 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0129600640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124102940 0x7f01241a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.414+0000 7f0128c00640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0124100e30 0x7f012419a410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f010c000c00 tx=0x7f010c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.415+0000 7f0122000640 1 -- 172.21.15.18:0/39320441 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c043020 con 0x7f0124100e30 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.415+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01241a22c0 con 0x7f0124100e30 2024-04-16T17:01:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.415+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01241a2860 con 0x7f0124100e30 2024-04-16T17:01:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.415+0000 7f0122000640 1 -- 172.21.15.18:0/39320441 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c0054e0 con 0x7f0124100e30 2024-04-16T17:01:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.415+0000 7f0122000640 1 -- 172.21.15.18:0/39320441 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c003c60 con 0x7f0124100e30 2024-04-16T17:01:31.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.416+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01241a2a80 con 0x7f0124100e30 2024-04-16T17:01:31.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.418+0000 7f0122000640 1 -- 172.21.15.18:0/39320441 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f010c046020 con 0x7f0124100e30 2024-04-16T17:01:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.420+0000 7f0122000640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f01040a3c20 0x7f01040a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.420+0000 7f0122000640 1 -- 172.21.15.18:0/39320441 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f010c0f1fe0 con 0x7f0124100e30 2024-04-16T17:01:31.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.420+0000 7f011be00640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f01040a3c20 0x7f01040a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:31.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.421+0000 7f011be00640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f01040a3c20 0x7f01040a60e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f012419b920 tx=0x7f0114005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:31.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:31 smithi022 ceph-mon[36023]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T17:01:31.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:31 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T17:01:31.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:31 smithi022 ceph-mon[36023]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:31.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:31 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1232549538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:31.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.424+0000 7f0122000640 1 -- 172.21.15.18:0/39320441 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f010c04d030 con 0x7f0124100e30 2024-04-16T17:01:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:31 smithi043 ceph-mon[36901]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T17:01:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:31 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T17:01:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:31 smithi043 ceph-mon[36901]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:31.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:31 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1232549538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:31.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.632+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01241a2c50 con 0x7f0124100e30 2024-04-16T17:01:31.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.635+0000 7f0122000640 1 -- 172.21.15.18:0/39320441 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7f010c0be780 con 0x7f0124100e30 2024-04-16T17:01:31.637 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:31.637 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:01:31.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.637+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f01040a3c20 msgr2=0x7f01040a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.637+0000 7f012a000640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f01040a3c20 0x7f01040a60e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f012419b920 tx=0x7f0114005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:31.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.637+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0124100e30 msgr2=0x7f012419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.637+0000 7f012a000640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0124100e30 0x7f012419a410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f010c000c00 tx=0x7f010c02c650 comp rx=0 tx=0).stop 2024-04-16T17:01:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.637+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 shutdown_connections 2024-04-16T17:01:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.637+0000 7f012a000640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f01040a3c20 0x7f01040a60e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.638+0000 7f012a000640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0124102940 0x7f01241a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.638+0000 7f012a000640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0124101f80 0x7f012419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.638+0000 7f012a000640 1 --2- 172.21.15.18:0/39320441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0124100e30 0x7f012419a410 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.638+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 >> 172.21.15.18:0/39320441 conn(0x7f01240fc3a0 msgr2=0x7f01240fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.638+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 shutdown_connections 2024-04-16T17:01:31.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.638+0000 7f012a000640 1 -- 172.21.15.18:0/39320441 wait complete. 2024-04-16T17:01:31.673 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.893+0000 7f6547e00640 1 -- 172.21.15.18:0/1458089644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 msgr2=0x7f6548076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.893+0000 7f6547e00640 1 --2- 172.21.15.18:0/1458089644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 0x7f6548076de0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6538009900 tx=0x7f653802bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.893+0000 7f6547e00640 1 -- 172.21.15.18:0/1458089644 shutdown_connections 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.893+0000 7f6547e00640 1 --2- 172.21.15.18:0/1458089644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65480785a0 0x7f6548078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.893+0000 7f6547e00640 1 --2- 172.21.15.18:0/1458089644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6548077c00 0x7f6548078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.893+0000 7f6547e00640 1 --2- 172.21.15.18:0/1458089644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 0x7f6548076de0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.893+0000 7f6547e00640 1 -- 172.21.15.18:0/1458089644 >> 172.21.15.18:0/1458089644 conn(0x7f65480ff6d0 msgr2=0x7f6548101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.893+0000 7f6547e00640 1 -- 172.21.15.18:0/1458089644 shutdown_connections 2024-04-16T17:01:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.894+0000 7f6547e00640 1 -- 172.21.15.18:0/1458089644 wait complete. 2024-04-16T17:01:31.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.895+0000 7f6547e00640 1 Processor -- start 2024-04-16T17:01:31.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.895+0000 7f6547e00640 1 -- start start 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.895+0000 7f6547e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 0x7f654819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.895+0000 7f6547e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6548077c00 0x7f654819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.895+0000 7f6547e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65480785a0 0x7f65481a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.895+0000 7f6547e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f65481141f0 con 0x7f6548077c00 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.895+0000 7f6547e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6548114070 con 0x7f65480785a0 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.895+0000 7f6547e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6548114370 con 0x7f65480769c0 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 0x7f654819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6548077c00 0x7f654819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 0x7f654819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41602/0 (socket says 172.21.15.18:41602) 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546a00640 1 -- 172.21.15.18:0/1573355310 learned_addr learned my addr 172.21.15.18:0/1573355310 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6547400640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65480785a0 0x7f65481a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546a00640 1 -- 172.21.15.18:0/1573355310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65480785a0 msgr2=0x7f65481a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546a00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65480785a0 0x7f65481a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546a00640 1 -- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6548077c00 msgr2=0x7f654819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546a00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6548077c00 0x7f654819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546a00640 1 -- 172.21.15.18:0/1573355310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65481a6480 con 0x7f65480769c0 2024-04-16T17:01:31.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.896+0000 7f6546000640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6548077c00 0x7f654819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.897+0000 7f6546a00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 0x7f654819e860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6538000c00 tx=0x7f653802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.897+0000 7f654ca00640 1 -- 172.21.15.18:0/1573355310 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538043020 con 0x7f65480769c0 2024-04-16T17:01:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.897+0000 7f654ca00640 1 -- 172.21.15.18:0/1573355310 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65380054e0 con 0x7f65480769c0 2024-04-16T17:01:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.897+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65481a6710 con 0x7f65480769c0 2024-04-16T17:01:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.897+0000 7f654ca00640 1 -- 172.21.15.18:0/1573355310 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538003bd0 con 0x7f65480769c0 2024-04-16T17:01:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.897+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65481a6c60 con 0x7f65480769c0 2024-04-16T17:01:31.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.898+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6548110c60 con 0x7f65480769c0 2024-04-16T17:01:31.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.900+0000 7f654ca00640 1 -- 172.21.15.18:0/1573355310 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6538046020 con 0x7f65480769c0 2024-04-16T17:01:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.902+0000 7f654ca00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f652c0a3ba0 0x7f652c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.902+0000 7f654ca00640 1 -- 172.21.15.18:0/1573355310 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f65380f2ad0 con 0x7f65480769c0 2024-04-16T17:01:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.902+0000 7f6546000640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f652c0a3ba0 0x7f652c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.902+0000 7f6546000640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f652c0a3ba0 0x7f652c0a6060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f65300095f0 tx=0x7f6530005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:31.904+0000 7f654ca00640 1 -- 172.21.15.18:0/1573355310 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f653804d020 con 0x7f65480769c0 2024-04-16T17:01:32.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.156+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f654802d620 con 0x7f65480769c0 2024-04-16T17:01:32.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.157+0000 7f654ca00640 1 -- 172.21.15.18:0/1573355310 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11776 (secure 0 0 0) 0x7f65380bf210 con 0x7f65480769c0 2024-04-16T17:01:32.159 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:32.160 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:01:29.488436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-16T17:01:32.160 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 17 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.160+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f652c0a3ba0 msgr2=0x7f652c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.160+0000 7f6547e00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f652c0a3ba0 0x7f652c0a6060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f65300095f0 tx=0x7f6530005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.160+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 msgr2=0x7f654819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.160+0000 7f6547e00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 0x7f654819e860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6538000c00 tx=0x7f653802c650 comp rx=0 tx=0).stop 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.161+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 shutdown_connections 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.161+0000 7f6547e00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65480785a0 0x7f65481a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.161+0000 7f6547e00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f652c0a3ba0 0x7f652c0a6060 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.161+0000 7f6547e00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6548077c00 0x7f654819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.161+0000 7f6547e00640 1 --2- 172.21.15.18:0/1573355310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65480769c0 0x7f654819e860 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.161+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 >> 172.21.15.18:0/1573355310 conn(0x7f65480ff6d0 msgr2=0x7f6548100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.161+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 shutdown_connections 2024-04-16T17:01:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.161+0000 7f6547e00640 1 -- 172.21.15.18:0/1573355310 wait complete. 2024-04-16T17:01:32.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 -- 172.21.15.18:0/3600660955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f81030d0 msgr2=0x7fd2f8103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3600660955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f81030d0 0x7fd2f8103550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009900 tx=0x7fd2e002bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 -- 172.21.15.18:0/3600660955 shutdown_connections 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3600660955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f8103a90 0x7fd2f810a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3600660955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f81030d0 0x7fd2f8103550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3600660955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2f8101f00 0x7fd2f8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 -- 172.21.15.18:0/3600660955 >> 172.21.15.18:0/3600660955 conn(0x7fd2f80fd4c0 msgr2=0x7fd2f80ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 -- 172.21.15.18:0/3600660955 shutdown_connections 2024-04-16T17:01:32.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.400+0000 7fd2fca00640 1 -- 172.21.15.18:0/3600660955 wait complete. 2024-04-16T17:01:32.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.401+0000 7fd2fca00640 1 Processor -- start 2024-04-16T17:01:32.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.401+0000 7fd2fca00640 1 -- start start 2024-04-16T17:01:32.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2fca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2f8101f00 0x7fd2f819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2fca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f81030d0 0x7fd2f819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2fca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f8103a90 0x7fd2f81a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2fca00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f810dc20 con 0x7fd2f8103a90 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2fca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f810daa0 con 0x7fd2f8101f00 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2fca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f810dda0 con 0x7fd2f81030d0 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2f7200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f8103a90 0x7fd2f81a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2ede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f81030d0 0x7fd2f819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2ede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f81030d0 0x7fd2f819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41626/0 (socket says 172.21.15.18:41626) 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2f7200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f8103a90 0x7fd2f81a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32922/0 (socket says 172.21.15.18:32922) 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2ede00640 1 -- 172.21.15.18:0/3139186917 learned_addr learned my addr 172.21.15.18:0/3139186917 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2f6800640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2f8101f00 0x7fd2f819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2ede00640 1 -- 172.21.15.18:0/3139186917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2f8101f00 msgr2=0x7fd2f819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2ede00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2f8101f00 0x7fd2f819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2ede00640 1 -- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f8103a90 msgr2=0x7fd2f81a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2ede00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f8103a90 0x7fd2f81a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2ede00640 1 -- 172.21.15.18:0/3139186917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f81a2110 con 0x7fd2f81030d0 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.402+0000 7fd2f7200640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f8103a90 0x7fd2f81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:32.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.403+0000 7fd2ede00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f81030d0 0x7fd2f819aa30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005ab0 tx=0x7fd2e002c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:32.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.403+0000 7fd2efe00640 1 -- 172.21.15.18:0/3139186917 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0041070 con 0x7fd2f81030d0 2024-04-16T17:01:32.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.403+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f81a23a0 con 0x7fd2f81030d0 2024-04-16T17:01:32.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.403+0000 7fd2efe00640 1 -- 172.21.15.18:0/3139186917 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e0004450 con 0x7fd2f81030d0 2024-04-16T17:01:32.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.403+0000 7fd2efe00640 1 -- 172.21.15.18:0/3139186917 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0034440 con 0x7fd2f81030d0 2024-04-16T17:01:32.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.403+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f81a28e0 con 0x7fd2f81030d0 2024-04-16T17:01:32.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.404+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f81a2c00 con 0x7fd2f81030d0 2024-04-16T17:01:32.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.406+0000 7fd2efe00640 1 -- 172.21.15.18:0/3139186917 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd2e0044070 con 0x7fd2f81030d0 2024-04-16T17:01:32.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.407+0000 7fd2efe00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2dc0a3ba0 0x7fd2dc0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.408+0000 7fd2efe00640 1 -- 172.21.15.18:0/3139186917 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd2e00f21d0 con 0x7fd2f81030d0 2024-04-16T17:01:32.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.408+0000 7fd2f6800640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2dc0a3ba0 0x7fd2dc0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.408+0000 7fd2f6800640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2dc0a3ba0 0x7fd2dc0a6060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8101c20 tx=0x7fd2e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:32.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.412+0000 7fd2efe00640 1 -- 172.21.15.18:0/3139186917 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2e00be2c0 con 0x7fd2f81030d0 2024-04-16T17:01:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:32 smithi022 ceph-mon[36023]: pgmap v322: 129 pgs: 6 creating+peering, 103 active+clean, 20 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-16T17:01:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:32 smithi022 ceph-mon[36023]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:01:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:32 smithi022 ceph-mon[36023]: Cluster is now healthy 2024-04-16T17:01:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:32 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2379600395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:32 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/39320441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:32 smithi018 ceph-mon[31776]: pgmap v322: 129 pgs: 6 creating+peering, 103 active+clean, 20 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-16T17:01:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:32 smithi018 ceph-mon[31776]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:01:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:32 smithi018 ceph-mon[31776]: Cluster is now healthy 2024-04-16T17:01:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:32 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2379600395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:32 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/39320441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:32.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:32 smithi043 ceph-mon[36901]: pgmap v322: 129 pgs: 6 creating+peering, 103 active+clean, 20 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-16T17:01:32.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:32 smithi043 ceph-mon[36901]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:01:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:32 smithi043 ceph-mon[36901]: Cluster is now healthy 2024-04-16T17:01:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:32 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2379600395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:32 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/39320441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:32.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.616+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2f81a2dd0 con 0x7fd2f81030d0 2024-04-16T17:01:32.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.619+0000 7fd2efe00640 1 -- 172.21.15.18:0/3139186917 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7fd2e00c25c0 con 0x7fd2f81030d0 2024-04-16T17:01:32.621 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:32.621 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:01:32.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.621+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2dc0a3ba0 msgr2=0x7fd2dc0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.621+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2dc0a3ba0 0x7fd2dc0a6060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8101c20 tx=0x7fd2e8009500 comp rx=0 tx=0).stop 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.621+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f81030d0 msgr2=0x7fd2f819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.621+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f81030d0 0x7fd2f819aa30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005ab0 tx=0x7fd2e002c6d0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.622+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 shutdown_connections 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.622+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2dc0a3ba0 0x7fd2dc0a6060 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.622+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2f8103a90 0x7fd2f81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.622+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2f81030d0 0x7fd2f819aa30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.622+0000 7fd2fca00640 1 --2- 172.21.15.18:0/3139186917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2f8101f00 0x7fd2f819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.622+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 >> 172.21.15.18:0/3139186917 conn(0x7fd2f80fd4c0 msgr2=0x7fd2f8074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.622+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 shutdown_connections 2024-04-16T17:01:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.622+0000 7fd2fca00640 1 -- 172.21.15.18:0/3139186917 wait complete. 2024-04-16T17:01:32.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-16T17:01:24.398529+0000', 'modified': '2024-04-16T17:01:29.488436+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24596}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6839/836242731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6838', 'nonce': 836242731}, {'type': 'v1', 'addr': '172.21.15.22:6839', 'nonce': 836242731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24596, 'qdb_cluster': [24596]} 2024-04-16T17:01:32.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-16T17:01:32.642 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T17:01:32.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 -- 172.21.15.18:0/1562199196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb74c1028f0 msgr2=0x7fb74c104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 --2- 172.21.15.18:0/1562199196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb74c1028f0 0x7fb74c104ce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb740009ca0 tx=0x7fb74002c100 comp rx=0 tx=0).stop 2024-04-16T17:01:32.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 -- 172.21.15.18:0/1562199196 shutdown_connections 2024-04-16T17:01:32.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 --2- 172.21.15.18:0/1562199196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb74c107b50 0x7fb74c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 --2- 172.21.15.18:0/1562199196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c105220 0x7fb74c107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 --2- 172.21.15.18:0/1562199196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb74c1028f0 0x7fb74c104ce0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 -- 172.21.15.18:0/1562199196 >> 172.21.15.18:0/1562199196 conn(0x7fb74c0fc2e0 msgr2=0x7fb74c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 -- 172.21.15.18:0/1562199196 shutdown_connections 2024-04-16T17:01:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.865+0000 7fb752800640 1 -- 172.21.15.18:0/1562199196 wait complete. 2024-04-16T17:01:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.866+0000 7fb752800640 1 Processor -- start 2024-04-16T17:01:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.866+0000 7fb752800640 1 -- start start 2024-04-16T17:01:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb752800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb74c1028f0 0x7fb74c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb752800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb74c105220 0x7fb74c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb752800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c107b50 0x7fb74c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb752800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c10db20 con 0x7fb74c1028f0 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb752800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c10d9a0 con 0x7fb74c107b50 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb752800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c10dca0 con 0x7fb74c105220 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb74c1028f0 0x7fb74c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb74c1028f0 0x7fb74c19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32938/0 (socket says 172.21.15.18:32938) 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751400640 1 -- 172.21.15.18:0/2728052935 learned_addr learned my addr 172.21.15.18:0/2728052935 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb750a00640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb74c105220 0x7fb74c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751e00640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c107b50 0x7fb74c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751400640 1 -- 172.21.15.18:0/2728052935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb74c105220 msgr2=0x7fb74c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751400640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb74c105220 0x7fb74c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751400640 1 -- 172.21.15.18:0/2728052935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c107b50 msgr2=0x7fb74c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751400640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c107b50 0x7fb74c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751400640 1 -- 172.21.15.18:0/2728052935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c1a1fd0 con 0x7fb74c1028f0 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.867+0000 7fb751e00640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c107b50 0x7fb74c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:01:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.868+0000 7fb751400640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb74c1028f0 0x7fb74c19a3b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb740009dc0 tx=0x7fb740035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.868+0000 7fb74a000640 1 -- 172.21.15.18:0/2728052935 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740039040 con 0x7fb74c1028f0 2024-04-16T17:01:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.868+0000 7fb74a000640 1 -- 172.21.15.18:0/2728052935 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb740044070 con 0x7fb74c1028f0 2024-04-16T17:01:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.868+0000 7fb752800640 1 -- 172.21.15.18:0/2728052935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74c1a2260 con 0x7fb74c1028f0 2024-04-16T17:01:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.868+0000 7fb74a000640 1 -- 172.21.15.18:0/2728052935 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7400044a0 con 0x7fb74c1028f0 2024-04-16T17:01:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.868+0000 7fb752800640 1 -- 172.21.15.18:0/2728052935 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74c1a2750 con 0x7fb74c1028f0 2024-04-16T17:01:32.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.872+0000 7fb74a000640 1 -- 172.21.15.18:0/2728052935 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb740004640 con 0x7fb74c1028f0 2024-04-16T17:01:32.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.872+0000 7fb756c1d740 1 Processor -- start 2024-04-16T17:01:32.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.872+0000 7fb756c1d740 1 -- start start 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.872+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9cfd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.872+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d02990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.872+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d05490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb756c1d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556ba9c63980 con 0x556ba9d03000 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb756c1d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556ba9c0a220 con 0x556ba9cfde90 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb756c1d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556ba9c5f940 con 0x556ba9cfd4a0 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9cfd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9cfd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41654/0 (socket says 172.21.15.18:41654) 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751e00640 1 -- 172.21.15.18:0/3792264712 learned_addr learned my addr 172.21.15.18:0/3792264712 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb750a00640 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d05490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751400640 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d02990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751e00640 1 -- 172.21.15.18:0/3792264712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 msgr2=0x556ba9d02990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:01:32.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751e00640 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d02990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751e00640 1 -- 172.21.15.18:0/3792264712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 msgr2=0x556ba9d05490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:01:32.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751e00640 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d05490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb751e00640 1 -- 172.21.15.18:0/3792264712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba99762d0 con 0x556ba9cfd4a0 2024-04-16T17:01:32.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.873+0000 7fb750a00640 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d05490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:32.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.874+0000 7fb74a000640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7340a3bb0 0x7fb7340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.874+0000 7fb751e00640 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9cfd8c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb744009cc0 tx=0x7fb7440117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadfa1ff148db14c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.874+0000 7fb74a000640 1 -- 172.21.15.18:0/2728052935 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb7400f2d40 con 0x7fb74c1028f0 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.874+0000 7fb73e600640 1 -- 172.21.15.18:0/3792264712 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744012450 con 0x556ba9cfd4a0 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.874+0000 7fb751400640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7340a3bb0 0x7fb7340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.874+0000 7fb73e600640 1 -- 172.21.15.18:0/3792264712 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb744012ad0 con 0x556ba9cfd4a0 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.874+0000 7fb73e600640 1 -- 172.21.15.18:0/3792264712 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74400fc10 con 0x556ba9cfd4a0 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 -- 172.21.15.18:0/3792264712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 msgr2=0x556ba9cfd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb751400640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7340a3bb0 0x7fb7340a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb7400f05a0 tx=0x7fb7400f6780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9cfd8c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb744009cc0 tx=0x7fb7440117b0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 -- 172.21.15.18:0/3792264712 shutdown_connections 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d05490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d02990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 --2- 172.21.15.18:0/3792264712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9cfd8c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 -- 172.21.15.18:0/3792264712 >> 172.21.15.18:0/3792264712 conn(0x556ba9cf8a10 msgr2=0x556ba9cf8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 -- 172.21.15.18:0/3792264712 shutdown_connections 2024-04-16T17:01:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.875+0000 7fb756c1d740 1 -- 172.21.15.18:0/3792264712 wait complete. 2024-04-16T17:01:32.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.876+0000 7fb756c1d740 1 -- start start 2024-04-16T17:01:32.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.876+0000 7fb756c1d740 1 Processor -- start 2024-04-16T17:01:32.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9d07ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d07ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d0efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb756c1d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556ba9d05fb0 con 0x556ba9d03000 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb756c1d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556ba9c1d590 con 0x556ba9cfde90 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb756c1d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556ba9d06130 con 0x556ba9cfd4a0 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb756c1d740 10 client.0 _ll_register_callbacks cb 0x556ba9c25050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb751e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9d07ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb751e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d0efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb751e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d0efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32964/0 (socket says 172.21.15.18:32964) 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb751e00640 1 -- 172.21.15.18:0/4049633687 learned_addr learned my addr 172.21.15.18:0/4049633687 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.877+0000 7fb750a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d07ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb751e00640 1 -- 172.21.15.18:0/4049633687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 msgr2=0x556ba9d07ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb751e00640 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9d07ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb751e00640 1 -- 172.21.15.18:0/4049633687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 msgr2=0x556ba9d07ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb751e00640 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d07ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb751e00640 1 -- 172.21.15.18:0/4049633687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba9a406b0 con 0x556ba9d03000 2024-04-16T17:01:32.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb750a00640 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d07ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb751e00640 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d0efc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb744010010 tx=0x7fb744000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1ffea8f693eb94 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb732a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb732a00640 1 -- 172.21.15.18:0/4049633687 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74400fbc0 con 0x556ba9d03000 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb732a00640 1 -- 172.21.15.18:0/4049633687 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb744020040 con 0x556ba9d03000 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb756c1d740 1 -- 172.21.15.18:0/4049633687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb744012ad0 con 0x556ba9d03000 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb732a00640 1 -- 172.21.15.18:0/4049633687 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb744036050 con 0x556ba9d03000 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb756c1d740 10 client.14583 fetch_fsmap learned FSMap version 17 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.878+0000 7fb756c1d740 1 -- 172.21.15.18:0/4049633687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556ba9d12330 con 0x556ba9d03000 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.879+0000 7fb732a00640 1 -- 172.21.15.18:0/4049633687 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb744012ad0 con 0x556ba9d03000 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.879+0000 7fb732a00640 1 -- 172.21.15.18:0/4049633687 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fb744026040 con 0x556ba9d03000 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.879+0000 7fb756c1d740 10 client.14583 fetch_fsmap finished waiting for FSMap version 17 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.879+0000 7fb756c1d740 10 client.14583 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.879+0000 7fb756c1d740 20 client.14583 populate_metadata read hostname 'smithi018' 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.879+0000 7fb756c1d740 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556ba9d14640 0x556ba9d15ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.879+0000 7fb756c1d740 4 client.14583 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:01:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.879+0000 7fb756c1d740 1 -- 172.21.15.18:0/4049633687 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba9c713a0 con 0x556ba9d14640 2024-04-16T17:01:32.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.880+0000 7fb750a00640 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556ba9d14640 0x556ba9d15ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.880+0000 7fb750a00640 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556ba9d14640 0x556ba9d15ab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c39c96e1ad35c3 server_cookie=6d94d63556a5049d in_seq=0 out_seq=0 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.880+0000 7fb732a00640 10 client.14583 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb732a00640 1 -- 172.21.15.18:0/4049633687 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ba9c713a0 con 0x556ba9d14640 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb732a00640 10 client.14583 handle_command_reply: tid=0 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 client.14583 shutdown 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 -- 172.21.15.18:0/4049633687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 msgr2=0x556ba9d0efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d0efc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb744010010 tx=0x7fb744000bc0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 -- 172.21.15.18:0/4049633687 shutdown_connections 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556ba9d14640 0x556ba9d15ab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d0efc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d07ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 --2- 172.21.15.18:0/4049633687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9d07ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb732a00640 0 client.14583 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:01:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 -- 172.21.15.18:0/4049633687 >> 172.21.15.18:0/4049633687 conn(0x556ba9cf8a10 msgr2=0x556ba9cf7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.881+0000 7fb756c1d740 1 -- 172.21.15.18:0/4049633687 shutdown_connections 2024-04-16T17:01:32.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.882+0000 7fb756c1d740 1 -- 172.21.15.18:0/4049633687 wait complete. 2024-04-16T17:01:32.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.882+0000 7fb756c1d740 20 client.14583 trim_cache size 0 max 16384 2024-04-16T17:01:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.902+0000 7fb756c1d740 1 Processor -- start 2024-04-16T17:01:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.902+0000 7fb756c1d740 1 -- start start 2024-04-16T17:01:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.902+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfd4a0 0x556ba9cf8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.902+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9cfde90 0x556ba9d0ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.902+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9d03000 0x556ba9cff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.902+0000 7fb756c1d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556ba9d13df0 con 0x556ba9cfde90 2024-04-16T17:01:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.902+0000 7fb756c1d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556ba9d144c0 con 0x556ba9cfd4a0 2024-04-16T17:01:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.902+0000 7fb756c1d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556ba9d14b90 con 0x556ba9d03000 2024-04-16T17:01:32.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9d03000 0x556ba9cff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb750a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9cfde90 0x556ba9d0ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9d03000 0x556ba9cff460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41666/0 (socket says 172.21.15.18:41666) 2024-04-16T17:01:32.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 -- 172.21.15.18:0/1225524627 learned_addr learned my addr 172.21.15.18:0/1225524627 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:32.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 --2- 172.21.15.18:0/1225524627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfd4a0 0x556ba9cf8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 -- 172.21.15.18:0/1225524627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfd4a0 msgr2=0x556ba9cf8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:01:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 --2- 172.21.15.18:0/1225524627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfd4a0 0x556ba9cf8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 -- 172.21.15.18:0/1225524627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9cfde90 msgr2=0x556ba9d0ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:01:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 --2- 172.21.15.18:0/1225524627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9cfde90 0x556ba9d0ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 -- 172.21.15.18:0/1225524627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba9cf9780 con 0x556ba9d03000 2024-04-16T17:01:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb751e00640 1 --2- 172.21.15.18:0/1225524627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9d03000 0x556ba9cff460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb744009960 tx=0x7fb744005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25bcbbbe7cc8a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.903+0000 7fb73dc00640 1 -- 172.21.15.18:0/1225524627 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744009260 con 0x556ba9d03000 2024-04-16T17:01:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb73dc00640 1 -- 172.21.15.18:0/1225524627 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb744012aa0 con 0x556ba9d03000 2024-04-16T17:01:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb73dc00640 1 -- 172.21.15.18:0/1225524627 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744012570 con 0x556ba9d03000 2024-04-16T17:01:32.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb756c1d740 1 -- 172.21.15.18:0/1225524627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9d03000 msgr2=0x556ba9cff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb756c1d740 1 --2- 172.21.15.18:0/1225524627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9d03000 0x556ba9cff460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb744009960 tx=0x7fb744005b90 comp rx=0 tx=0).stop 2024-04-16T17:01:32.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb756c1d740 1 -- 172.21.15.18:0/1225524627 shutdown_connections 2024-04-16T17:01:32.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb756c1d740 1 --2- 172.21.15.18:0/1225524627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9d03000 0x556ba9cff460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb756c1d740 1 --2- 172.21.15.18:0/1225524627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9cfde90 0x556ba9d0ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb756c1d740 1 --2- 172.21.15.18:0/1225524627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfd4a0 0x556ba9cf8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb756c1d740 1 -- 172.21.15.18:0/1225524627 >> 172.21.15.18:0/1225524627 conn(0x556ba9cf8a10 msgr2=0x556ba9d06500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.904+0000 7fb756c1d740 1 -- 172.21.15.18:0/1225524627 shutdown_connections 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.905+0000 7fb756c1d740 1 -- 172.21.15.18:0/1225524627 wait complete. 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.905+0000 7fb756c1d740 1 -- start start 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.905+0000 7fb756c1d740 1 Processor -- start 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9d09ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d0a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb756c1d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d80df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb756c1d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556ba9d15930 con 0x556ba9d03000 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb756c1d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556ba9d15260 con 0x556ba9cfde90 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb756c1d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556ba9d16000 con 0x556ba9cfd4a0 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb756c1d740 10 client.0 _ll_register_callbacks cb 0x7fb738009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:01:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb750a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d0a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb750a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d0a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53612/0 (socket says 172.21.15.18:53612) 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb750a00640 1 -- 172.21.15.18:0/806309145 learned_addr learned my addr 172.21.15.18:0/806309145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb751e00640 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9d09ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb751e00640 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d80df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb750a00640 1 -- 172.21.15.18:0/806309145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 msgr2=0x556ba9d09ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb750a00640 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9d09ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb750a00640 1 -- 172.21.15.18:0/806309145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 msgr2=0x556ba9d80df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb750a00640 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d80df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.906+0000 7fb750a00640 1 -- 172.21.15.18:0/806309145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba9d81380 con 0x556ba9cfde90 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.907+0000 7fb750a00640 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d0a530 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb74c19b8c0 tx=0x7fb738012de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d14e6229a0de42 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.907+0000 7fb732a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.907+0000 7fb732a00640 1 -- 172.21.15.18:0/806309145 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738020a50 con 0x556ba9cfde90 2024-04-16T17:01:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.907+0000 7fb756c1d740 1 -- 172.21.15.18:0/806309145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb744012aa0 con 0x556ba9cfde90 2024-04-16T17:01:32.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.907+0000 7fb732a00640 1 -- 172.21.15.18:0/806309145 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb738020bf0 con 0x556ba9cfde90 2024-04-16T17:01:32.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.907+0000 7fb732a00640 1 -- 172.21.15.18:0/806309145 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb738020df0 con 0x556ba9cfde90 2024-04-16T17:01:32.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.907+0000 7fb756c1d740 10 client.24710 fetch_fsmap learned FSMap version 17 2024-04-16T17:01:32.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.907+0000 7fb756c1d740 1 -- 172.21.15.18:0/806309145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556ba9d827d0 con 0x556ba9cfde90 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.908+0000 7fb732a00640 1 -- 172.21.15.18:0/806309145 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb744012aa0 con 0x556ba9cfde90 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.908+0000 7fb732a00640 1 -- 172.21.15.18:0/806309145 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fb7380316d0 con 0x556ba9cfde90 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.908+0000 7fb756c1d740 10 client.24710 fetch_fsmap finished waiting for FSMap version 17 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.908+0000 7fb756c1d740 10 client.24710 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.908+0000 7fb756c1d740 20 client.24710 populate_metadata read hostname 'smithi018' 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.908+0000 7fb756c1d740 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556ba9d873d0 0x556ba9d860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.908+0000 7fb756c1d740 4 client.24710 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.908+0000 7fb756c1d740 1 -- 172.21.15.18:0/806309145 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "status"}) -- 0x556ba9c6b210 con 0x556ba9d873d0 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.909+0000 7fb751e00640 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556ba9d873d0 0x556ba9d860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.909+0000 7fb751e00640 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556ba9d873d0 0x556ba9d860e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d46b29e63d77f78 server_cookie=17ec14e5b8f016ad in_seq=0 out_seq=0 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.909+0000 7fb732a00640 10 client.24710 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.909+0000 7fb732a00640 1 -- 172.21.15.18:0/806309145 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556ba9c6b210 con 0x556ba9d873d0 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.909+0000 7fb732a00640 10 client.24710 handle_command_reply: tid=0 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.909+0000 7fb756c1d740 1 client.24710 shutdown 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 -- 172.21.15.18:0/806309145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 msgr2=0x556ba9d0a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d0a530 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb74c19b8c0 tx=0x7fb738012de0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 -- 172.21.15.18:0/806309145 shutdown_connections 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556ba9d873d0 0x556ba9d860e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ba9d03000 0x556ba9d80df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ba9cfde90 0x556ba9d0a530 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 --2- 172.21.15.18:0/806309145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ba9cfd4a0 0x556ba9d09ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb732a00640 0 client.24710 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:01:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 -- 172.21.15.18:0/806309145 >> 172.21.15.18:0/806309145 conn(0x556ba9cf8a10 msgr2=0x556ba9cf8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 -- 172.21.15.18:0/806309145 shutdown_connections 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 1 -- 172.21.15.18:0/806309145 wait complete. 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.910+0000 7fb756c1d740 20 client.24710 trim_cache size 0 max 16384 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 0, 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24596, 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-04-16T17:01:32.916 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-04-16T17:01:32.917 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 7.9776985150000002, 2024-04-16T17:01:32.917 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 17, 2024-04-16T17:01:32.917 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 89, 2024-04-16T17:01:32.917 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T17:01:32.917 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 14.128828800999999 2024-04-16T17:01:32.917 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:01:32.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 -- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7340a3bb0 msgr2=0x7fb7340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7340a3bb0 0x7fb7340a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb7400f05a0 tx=0x7fb7400f6780 comp rx=0 tx=0).stop 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 -- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb74c1028f0 msgr2=0x7fb74c19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb74c1028f0 0x7fb74c19a3b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb740009dc0 tx=0x7fb740035d90 comp rx=0 tx=0).stop 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 -- 172.21.15.18:0/2728052935 shutdown_connections 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c107b50 0x7fb74c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb74c105220 0x7fb74c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb7340a3bb0 0x7fb7340a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 --2- 172.21.15.18:0/2728052935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb74c1028f0 0x7fb74c19a3b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 -- 172.21.15.18:0/2728052935 >> 172.21.15.18:0/2728052935 conn(0x7fb74c0fc2e0 msgr2=0x7fb74c101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 -- 172.21.15.18:0/2728052935 shutdown_connections 2024-04-16T17:01:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:32.911+0000 7fb752800640 1 -- 172.21.15.18:0/2728052935 wait complete. 2024-04-16T17:01:32.932 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T17:01:32.932 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:32.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph fs ls 2024-04-16T17:01:33.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:33 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1573355310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:33.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:33 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3139186917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:33.400 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:01:33.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:33 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1573355310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:33.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:33 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3139186917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:33.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:33 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1573355310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:01:33.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:33 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3139186917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:34.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:34 smithi018 ceph-mon[31776]: pgmap v323: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-04-16T17:01:34.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:34 smithi022 ceph-mon[36023]: pgmap v323: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-04-16T17:01:34.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:34 smithi043 ceph-mon[36901]: pgmap v323: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-04-16T17:01:35.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.794+0000 7f3a76000700 1 -- 172.21.15.18:0/215614838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a7010f0e0 msgr2=0x7f3a7010f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:35.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.794+0000 7f3a76000700 1 --2- 172.21.15.18:0/215614838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a7010f0e0 0x7f3a7010f4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009990 tx=0x7f3a64009ca0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.794+0000 7f3a76000700 1 -- 172.21.15.18:0/215614838 shutdown_connections 2024-04-16T17:01:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.794+0000 7f3a76000700 1 --2- 172.21.15.18:0/215614838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a701099f0 0x7f3a70115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.794+0000 7f3a76000700 1 --2- 172.21.15.18:0/215614838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a7010f0e0 0x7f3a7010f4c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.794+0000 7f3a76000700 1 --2- 172.21.15.18:0/215614838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a70109030 0x7f3a701094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.794+0000 7f3a76000700 1 -- 172.21.15.18:0/215614838 >> 172.21.15.18:0/215614838 conn(0x7f3a70078880 msgr2=0x7f3a70107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.794+0000 7f3a76000700 1 -- 172.21.15.18:0/215614838 shutdown_connections 2024-04-16T17:01:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.795+0000 7f3a76000700 1 -- 172.21.15.18:0/215614838 wait complete. 2024-04-16T17:01:35.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.795+0000 7f3a76000700 1 Processor -- start 2024-04-16T17:01:35.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a76000700 1 -- start start 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a76000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a70109030 0x7f3a701a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a76000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a701099f0 0x7f3a701a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a76000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a7010f0e0 0x7f3a7019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a76000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70119370 con 0x7f3a701099f0 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a76000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701191f0 con 0x7f3a70109030 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a76000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701194f0 con 0x7f3a7010f0e0 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a74c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a70109030 0x7f3a701a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a6fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a701099f0 0x7f3a701a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a74c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a70109030 0x7f3a701a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53630/0 (socket says 172.21.15.18:53630) 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.796+0000 7f3a74c00700 1 -- 172.21.15.18:0/3491398780 learned_addr learned my addr 172.21.15.18:0/3491398780 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.797+0000 7f3a75600700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a7010f0e0 0x7f3a7019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.797+0000 7f3a74c00700 1 -- 172.21.15.18:0/3491398780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a7010f0e0 msgr2=0x7f3a7019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.797+0000 7f3a74c00700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a7010f0e0 0x7f3a7019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.797+0000 7f3a74c00700 1 -- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a701099f0 msgr2=0x7f3a701a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.797+0000 7f3a74c00700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a701099f0 0x7f3a701a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.797+0000 7f3a74c00700 1 -- 172.21.15.18:0/3491398780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7019ff10 con 0x7f3a70109030 2024-04-16T17:01:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.797+0000 7f3a74c00700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a70109030 0x7f3a701a5560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a64000c00 tx=0x7f3a6400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.797+0000 7f3a6d600700 1 -- 172.21.15.18:0/3491398780 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64027070 con 0x7f3a70109030 2024-04-16T17:01:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.798+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a701a01a0 con 0x7f3a70109030 2024-04-16T17:01:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.798+0000 7f3a6d600700 1 -- 172.21.15.18:0/3491398780 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a640056d0 con 0x7f3a70109030 2024-04-16T17:01:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.798+0000 7f3a6d600700 1 -- 172.21.15.18:0/3491398780 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64022e10 con 0x7f3a70109030 2024-04-16T17:01:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.798+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a701ac4e0 con 0x7f3a70109030 2024-04-16T17:01:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.799+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a701a0500 con 0x7f3a70109030 2024-04-16T17:01:35.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.801+0000 7f3a6d600700 1 -- 172.21.15.18:0/3491398780 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3a6402a070 con 0x7f3a70109030 2024-04-16T17:01:35.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.802+0000 7f3a6d600700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a580a3cc0 0x7f3a580a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:35.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.802+0000 7f3a6d600700 1 -- 172.21.15.18:0/3491398780 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3a640d1830 con 0x7f3a70109030 2024-04-16T17:01:35.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.802+0000 7f3a6fe00700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a580a3cc0 0x7f3a580a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:35.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.803+0000 7f3a6fe00700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a580a3cc0 0x7f3a580a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a701a0df0 tx=0x7f3a5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:35.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:35.805+0000 7f3a6d600700 1 -- 172.21.15.18:0/3491398780 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a64013070 con 0x7f3a70109030 2024-04-16T17:01:36.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.089+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3a701a0b60 con 0x7f3a70109030 2024-04-16T17:01:36.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.090+0000 7f3a6d600700 1 -- 172.21.15.18:0/3491398780 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f3a6409e000 con 0x7f3a70109030 2024-04-16T17:01:36.092 INFO:teuthology.orchestra.run.smithi018.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T17:01:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a580a3cc0 msgr2=0x7f3a580a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a580a3cc0 0x7f3a580a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a701a0df0 tx=0x7f3a5c009260 comp rx=0 tx=0).stop 2024-04-16T17:01:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a70109030 msgr2=0x7f3a701a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a70109030 0x7f3a701a5560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a64000c00 tx=0x7f3a6400bab0 comp rx=0 tx=0).stop 2024-04-16T17:01:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 shutdown_connections 2024-04-16T17:01:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a580a3cc0 0x7f3a580a6180 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a70109030 0x7f3a701a5560 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a7010f0e0 0x7f3a7019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 --2- 172.21.15.18:0/3491398780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a701099f0 0x7f3a701a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.093+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 >> 172.21.15.18:0/3491398780 conn(0x7f3a70078880 msgr2=0x7f3a70079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.094+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 shutdown_connections 2024-04-16T17:01:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:36.094+0000 7f3a76000700 1 -- 172.21.15.18:0/3491398780 wait complete. 2024-04-16T17:01:36.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:36 smithi022 ceph-mon[36023]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-04-16T17:01:36.437 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:36 smithi018 ceph-mon[31776]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-04-16T17:01:36.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:36 smithi043 ceph-mon[36901]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-04-16T17:01:36.844 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-16T17:01:36.844 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T17:01:36.845 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi018.front.sepia.ceph.com 2024-04-16T17:01:36.845 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-16T17:01:36.845 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:36.845 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-16T17:01:36.845 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T17:01:36.845 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T17:01:36.845 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T17:01:36.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:36.845 DEBUG:teuthology.orchestra.run.smithi018:> ip addr 2024-04-16T17:01:36.905 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f0 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f1 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T17:01:36.906 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:01:36.907 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f0 2024-04-16T17:01:36.907 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-16T17:01:36.907 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 42036sec preferred_lft 42036sec 2024-04-16T17:01:36.907 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link noprefixroute 2024-04-16T17:01:36.907 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:01:36.907 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:01:36.907 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:01:36.907 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f1 2024-04-16T17:01:36.908 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T17:01:36.908 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:01:36.908 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:01:36.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add name ceph-brx type bridge 2024-04-16T17:01:36.909 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr flush dev ceph-brx 2024-04-16T17:01:36.909 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set ceph-brx up 2024-04-16T17:01:36.909 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T17:01:36.909 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:01:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:01:37.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:37 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3491398780' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:01:37.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:37 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3491398780' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:01:37.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:01:37.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:37.469 DEBUG:teuthology.orchestra.run.smithi018:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T17:01:37.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:37 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3491398780' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:01:37.553 INFO:teuthology.orchestra.run.smithi018.stdout:1 2024-04-16T17:01:37.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:37.557 DEBUG:teuthology.orchestra.run.smithi018:> route 2024-04-16T17:01:37.624 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2024-04-16T17:01:37.624 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T17:01:37.624 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-16T17:01:37.624 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-16T17:01:37.625 INFO:teuthology.orchestra.run.smithi018.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T17:01:37.626 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:01:37.626 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:01:37.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-16T17:01:37.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-16T17:01:37.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-16T17:01:37.626 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:01:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:01:38.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:38 smithi022 ceph-mon[36023]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:38.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:38 smithi018 ceph-mon[31776]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:38.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:01:38.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:38.451 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-04-16T17:01:38.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:38 smithi043 ceph-mon[36901]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:38.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:38.510 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list-id 2024-04-16T17:01:38.564 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:01:38.564 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:01:38.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:01:38.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-16T17:01:38.565 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:01:38.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:01:38.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:01:38.783 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-16T17:01:38.783 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:01:38.783 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:01:38.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-16T17:01:38.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T17:01:38.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-16T17:01:38.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-16T17:01:38.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-16T17:01:38.783 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:01:38.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:01:39.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:01:39.147 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:01:39.147 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:01:39.147 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set brx.0 up 2024-04-16T17:01:39.147 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T17:01:39.148 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:01:39.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:01:39.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:01:39.398 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-16T17:01:39.398 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T17:01:39.398 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:39.446 INFO:teuthology.orchestra.run.smithi018.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T17:01:39.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T17:01:39.446 DEBUG:teuthology.orchestra.run.smithi018:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:39.494 DEBUG:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2024-04-16T17:01:39.582 DEBUG:teuthology.orchestra.run.smithi018:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T17:01:39.634 INFO:teuthology.orchestra.run.smithi018.stdout:/proc 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/sys 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/dev 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/security 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/shm 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/pts 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/run 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/cgroup 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/pstore 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/bpf 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/ 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/selinux 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/mqueue 2024-04-16T17:01:39.635 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/hugepages 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/debug 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/tracing 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/fuse/connections 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/config 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/1000 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/ceph 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/2b4a95aa721666f1f910010e8f3883dfcd22b7a292f14866f1a648cb8505a643/merged 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/81fb5cdcee1e670051ddb3deef164f6b7dc0e8401ddf8cb4d2ce8fe6de742d68/merged 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/0 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:01:39.636 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/d193e833ef489561fbfd165924d43e095b94a8569c0e229b8fe3a41772a7ea2e/merged 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/165c6d678a877b990584f69a0412e5ae96418f0feb90d45e02b2410671afbdad/merged 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/e4310a062d869bc37e55575913e9dc6292b82d4f4545105bfd2bad59bd1a33f8/merged 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/5f1ed2799ae6f620b9adcfa6f11f667d14d79f83f1703290f8b9bddb3d2956d2/merged 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/557418eac0b204e490ef4e78ff5b5153d8b8b7a2e470af93b372b429eab4d52f/merged 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/3f1948ebb37fe60457d7d9b398483c0f224b4c26e4ba99fc25eb9f29c27ae94f/merged 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/e7c9ad9e6eec43e253da855e5c985f0e301fdfb0ec144828853180d3135d8007/merged 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/615373a3debf182a8ab90f9d2b0456ff9b61b50e7b5d4a812be2ed636a1368c7/merged 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:01:39.637 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:01:39.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:39.638 DEBUG:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2024-04-16T17:01:39.686 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-16T17:01:39.686 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-16T17:01:39.691 DEBUG:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2024-04-16T17:01:39.768 DEBUG:teuthology.orchestra.run.smithi018:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/proc 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/sys 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/dev 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/security 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/shm 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/pts 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/run 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/cgroup 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/pstore 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/bpf 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/ 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/selinux 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/mqueue 2024-04-16T17:01:39.810 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/hugepages 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/debug 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/tracing 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/fuse/connections 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/config 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/1000 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/ceph 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/2b4a95aa721666f1f910010e8f3883dfcd22b7a292f14866f1a648cb8505a643/merged 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/81fb5cdcee1e670051ddb3deef164f6b7dc0e8401ddf8cb4d2ce8fe6de742d68/merged 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/0 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:01:39.811 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/d193e833ef489561fbfd165924d43e095b94a8569c0e229b8fe3a41772a7ea2e/merged 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/165c6d678a877b990584f69a0412e5ae96418f0feb90d45e02b2410671afbdad/merged 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/e4310a062d869bc37e55575913e9dc6292b82d4f4545105bfd2bad59bd1a33f8/merged 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/5f1ed2799ae6f620b9adcfa6f11f667d14d79f83f1703290f8b9bddb3d2956d2/merged 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/557418eac0b204e490ef4e78ff5b5153d8b8b7a2e470af93b372b429eab4d52f/merged 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/3f1948ebb37fe60457d7d9b398483c0f224b4c26e4ba99fc25eb9f29c27ae94f/merged 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/e7c9ad9e6eec43e253da855e5c985f0e301fdfb0ec144828853180d3135d8007/merged 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/615373a3debf182a8ab90f9d2b0456ff9b61b50e7b5d4a812be2ed636a1368c7/merged 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:01:39.812 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:01:39.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:39.813 DEBUG:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2024-04-16T17:01:39.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi018.stderr:2024-04-16T17:01:39.891+0000 7fdcb2c41480 -1 init, newargv = 0x563a3c869840 newargc=15 2024-04-16T17:01:39.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi018.stderr:ceph-fuse[73275]: starting ceph client 2024-04-16T17:01:39.904 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi018.stderr:ceph-fuse[73275]: starting fuse 2024-04-16T17:01:40.291 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:40 smithi018 ceph-mon[31776]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:40.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:40 smithi022 ceph-mon[36023]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:40.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:40 smithi043 ceph-mon[36901]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:41.004 DEBUG:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2024-04-16T17:01:41.102 DEBUG:teuthology.orchestra.run.smithi018:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T17:01:41.155 INFO:teuthology.orchestra.run.smithi018.stdout:/proc 2024-04-16T17:01:41.155 INFO:teuthology.orchestra.run.smithi018.stdout:/sys 2024-04-16T17:01:41.155 INFO:teuthology.orchestra.run.smithi018.stdout:/dev 2024-04-16T17:01:41.155 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/security 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/shm 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/pts 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/run 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/cgroup 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/pstore 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/bpf 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/ 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/selinux 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/mqueue 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/hugepages 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/debug 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/tracing 2024-04-16T17:01:41.156 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/fuse/connections 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/config 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/1000 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/ceph 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/2b4a95aa721666f1f910010e8f3883dfcd22b7a292f14866f1a648cb8505a643/merged 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/81fb5cdcee1e670051ddb3deef164f6b7dc0e8401ddf8cb4d2ce8fe6de742d68/merged 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/0 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/d193e833ef489561fbfd165924d43e095b94a8569c0e229b8fe3a41772a7ea2e/merged 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/165c6d678a877b990584f69a0412e5ae96418f0feb90d45e02b2410671afbdad/merged 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/e4310a062d869bc37e55575913e9dc6292b82d4f4545105bfd2bad59bd1a33f8/merged 2024-04-16T17:01:41.157 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/5f1ed2799ae6f620b9adcfa6f11f667d14d79f83f1703290f8b9bddb3d2956d2/merged 2024-04-16T17:01:41.158 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/557418eac0b204e490ef4e78ff5b5153d8b8b7a2e470af93b372b429eab4d52f/merged 2024-04-16T17:01:41.158 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/3f1948ebb37fe60457d7d9b398483c0f224b4c26e4ba99fc25eb9f29c27ae94f/merged 2024-04-16T17:01:41.158 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/e7c9ad9e6eec43e253da855e5c985f0e301fdfb0ec144828853180d3135d8007/merged 2024-04-16T17:01:41.158 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/615373a3debf182a8ab90f9d2b0456ff9b61b50e7b5d4a812be2ed636a1368c7/merged 2024-04-16T17:01:41.158 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns 2024-04-16T17:01:41.158 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:01:41.158 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:01:41.158 INFO:teuthology.orchestra.run.smithi018.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:41.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:41.159 DEBUG:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2024-04-16T17:01:41.208 INFO:teuthology.orchestra.run.smithi018.stdout:119 2024-04-16T17:01:41.208 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-16T17:01:41.208 DEBUG:teuthology.orchestra.run.smithi018:> sudo stdin-killer -- python3 -c ' 2024-04-16T17:01:41.208 DEBUG:teuthology.orchestra.run.smithi018:> import glob 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> import re 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> import os 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> import subprocess 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> def _find_admin_socket(client_name): 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> files = glob.glob(asok_path) 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> # Given a non-glob path, it better be there 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> if "*" not in asok_path: 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> assert(len(files) == 1) 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> return files[0] 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> for f in files: 2024-04-16T17:01:41.209 DEBUG:teuthology.orchestra.run.smithi018:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> contents = proc_f.read() 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> if mountpoint in contents: 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> return f 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> print(_find_admin_socket("client.admin")) 2024-04-16T17:01:41.210 DEBUG:teuthology.orchestra.run.smithi018:> ' 2024-04-16T17:01:41.361 INFO:teuthology.orchestra.run.smithi018.stdout:/var/run/ceph/ceph-client.admin.73275.asok 2024-04-16T17:01:41.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:01:41.379 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73275.asok 2024-04-16T17:01:41.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:41.379 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73275.asok status 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "metadata": { 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_sha1": "9b42bfbeda298874861ece590aa565cf64ac8c45", 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev)", 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "entity_id": "admin", 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": "smithi018", 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "pid": "73275", 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "root": "/" 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "dentry_count": 0, 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "dentry_pinned_count": 0, 2024-04-16T17:01:41.624 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24722, 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "inst": { 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "name": { 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "client", 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "num": 24722 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": { 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2062332025 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": { 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-04-16T17:01:41.625 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2062332025 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: "inst_str": "client.24722 172.21.15.18:0/2062332025", 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: "addr_str": "172.21.15.18:0/2062332025", 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: "inode_count": 1, 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_epoch": 17, 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_epoch": 89, 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_epoch_barrier": 0, 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: "blocklisted": false, 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs" 2024-04-16T17:01:41.626 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:01:41.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:41.640 DEBUG:teuthology.orchestra.run.smithi018:> stat --file-system '--printf=%T 2024-04-16T17:01:41.640 DEBUG:teuthology.orchestra.run.smithi018:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:41.696 INFO:teuthology.orchestra.run.smithi018.stdout:fuseblk 2024-04-16T17:01:41.696 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:41.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:41.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:41.790 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-16T17:01:41.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T17:01:41.941 INFO:teuthology.orchestra.run.smithi018.stdout:# file: . 2024-04-16T17:01:41.942 INFO:teuthology.orchestra.run.smithi018.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-16T17:01:41.942 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:41.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:01:41.952 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T17:01:41.952 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:01:42.007 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 -- 172.21.15.18:0/1418860810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c0769c0 msgr2=0x7f329c076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 --2- 172.21.15.18:0/1418860810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c0769c0 0x7f329c076de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f328c009900 tx=0x7f328c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 -- 172.21.15.18:0/1418860810 shutdown_connections 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 --2- 172.21.15.18:0/1418860810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f329c0785a0 0x7f329c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 --2- 172.21.15.18:0/1418860810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f329c077c00 0x7f329c078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 --2- 172.21.15.18:0/1418860810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c0769c0 0x7f329c076de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 -- 172.21.15.18:0/1418860810 >> 172.21.15.18:0/1418860810 conn(0x7f329c0ff6d0 msgr2=0x7f329c101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 -- 172.21.15.18:0/1418860810 shutdown_connections 2024-04-16T17:01:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.232+0000 7f329be00640 1 -- 172.21.15.18:0/1418860810 wait complete. 2024-04-16T17:01:42.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.233+0000 7f329be00640 1 Processor -- start 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.233+0000 7f329be00640 1 -- start start 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.233+0000 7f329be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c0769c0 0x7f329c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.233+0000 7f329be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f329c077c00 0x7f329c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.233+0000 7f329be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f329c0785a0 0x7f329c1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.233+0000 7f329be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f329c114230 con 0x7f329c077c00 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.233+0000 7f329be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1140b0 con 0x7f329c0785a0 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.233+0000 7f329be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1143b0 con 0x7f329c0769c0 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f3292000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f329c077c00 0x7f329c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f329aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c0769c0 0x7f329c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f3292000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f329c077c00 0x7f329c19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52252/0 (socket says 172.21.15.18:52252) 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f3292000640 1 -- 172.21.15.18:0/2053689920 learned_addr learned my addr 172.21.15.18:0/2053689920 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:42.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f329b400640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f329c0785a0 0x7f329c1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f329b400640 1 -- 172.21.15.18:0/2053689920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c0769c0 msgr2=0x7f329c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f329b400640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c0769c0 0x7f329c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f329b400640 1 -- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f329c077c00 msgr2=0x7f329c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f329b400640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f329c077c00 0x7f329c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f329b400640 1 -- 172.21.15.18:0/2053689920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c1a6420 con 0x7f329c0785a0 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.234+0000 7f3292000640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f329c077c00 0x7f329c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.235+0000 7f329b400640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f329c0785a0 0x7f329c1a5db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3294009fd0 tx=0x7f329400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.235+0000 7f3293e00640 1 -- 172.21.15.18:0/2053689920 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400ce20 con 0x7f329c0785a0 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.235+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c1a6710 con 0x7f329c0785a0 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.235+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c1a6c60 con 0x7f329c0785a0 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.235+0000 7f3293e00640 1 -- 172.21.15.18:0/2053689920 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294010430 con 0x7f329c0785a0 2024-04-16T17:01:42.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.235+0000 7f3293e00640 1 -- 172.21.15.18:0/2053689920 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400f620 con 0x7f329c0785a0 2024-04-16T17:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.236+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32780052c0 con 0x7f329c0785a0 2024-04-16T17:01:42.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.238+0000 7f3293e00640 1 -- 172.21.15.18:0/2053689920 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f32940105e0 con 0x7f329c0785a0 2024-04-16T17:01:42.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.239+0000 7f3293e00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f32800a3ba0 0x7f32800a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:42.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.240+0000 7f3293e00640 1 -- 172.21.15.18:0/2053689920 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f32940d0880 con 0x7f329c0785a0 2024-04-16T17:01:42.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.240+0000 7f329aa00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f32800a3ba0 0x7f32800a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:42.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.241+0000 7f329aa00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f32800a3ba0 0x7f32800a6060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f328c02c250 tx=0x7f328c02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.244+0000 7f3293e00640 1 -- 172.21.15.18:0/2053689920 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f329409d020 con 0x7f329c0785a0 2024-04-16T17:01:42.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:42 smithi022 ceph-mon[36023]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:42.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:42 smithi018 ceph-mon[31776]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:42.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.452+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32780055b0 con 0x7f329c0785a0 2024-04-16T17:01:42.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.454+0000 7f3293e00640 1 -- 172.21.15.18:0/2053689920 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7f329409c920 con 0x7f329c0785a0 2024-04-16T17:01:42.457 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:42.457 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:01:42.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f32800a3ba0 msgr2=0x7f32800a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f32800a3ba0 0x7f32800a6060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f328c02c250 tx=0x7f328c02cf90 comp rx=0 tx=0).stop 2024-04-16T17:01:42.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f329c0785a0 msgr2=0x7f329c1a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f329c0785a0 0x7f329c1a5db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3294009fd0 tx=0x7f329400ef10 comp rx=0 tx=0).stop 2024-04-16T17:01:42.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 shutdown_connections 2024-04-16T17:01:42.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f329c0785a0 0x7f329c1a5db0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f32800a3ba0 0x7f32800a6060 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f329c077c00 0x7f329c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 --2- 172.21.15.18:0/2053689920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c0769c0 0x7f329c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 >> 172.21.15.18:0/2053689920 conn(0x7f329c0ff6d0 msgr2=0x7f329c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:42.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 shutdown_connections 2024-04-16T17:01:42.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.457+0000 7f329be00640 1 -- 172.21.15.18:0/2053689920 wait complete. 2024-04-16T17:01:42.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:01:42.483 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:01:42.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:42 smithi043 ceph-mon[36901]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.708+0000 7f35d1800640 1 -- 172.21.15.18:0/3973050360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc105e20 msgr2=0x7f35cc10c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.708+0000 7f35d1800640 1 --2- 172.21.15.18:0/3973050360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc105e20 0x7f35cc10c6b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009ca0 tx=0x7f35c402c100 comp rx=0 tx=0).stop 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.709+0000 7f35d1800640 1 -- 172.21.15.18:0/3973050360 shutdown_connections 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.709+0000 7f35d1800640 1 --2- 172.21.15.18:0/3973050360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc105e20 0x7f35cc10c6b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.709+0000 7f35d1800640 1 --2- 172.21.15.18:0/3973050360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35cc105460 0x7f35cc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.709+0000 7f35d1800640 1 --2- 172.21.15.18:0/3973050360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc104270 0x7f35cc104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.709+0000 7f35d1800640 1 -- 172.21.15.18:0/3973050360 >> 172.21.15.18:0/3973050360 conn(0x7f35cc0ff820 msgr2=0x7f35cc101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.709+0000 7f35d1800640 1 -- 172.21.15.18:0/3973050360 shutdown_connections 2024-04-16T17:01:42.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.709+0000 7f35d1800640 1 -- 172.21.15.18:0/3973050360 wait complete. 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.710+0000 7f35d1800640 1 Processor -- start 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.710+0000 7f35d1800640 1 -- start start 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.710+0000 7f35d1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc104270 0x7f35cc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.710+0000 7f35d1800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35cc105460 0x7f35cc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.710+0000 7f35d1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc105e20 0x7f35cc1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.710+0000 7f35d1800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc10fed0 con 0x7f35cc105460 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35d1800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc10fd50 con 0x7f35cc104270 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35d1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc110050 con 0x7f35cc105e20 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35c2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35cc105460 0x7f35cc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc104270 0x7f35cc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc105e20 0x7f35cc1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:42.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc104270 0x7f35cc19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54872/0 (socket says 172.21.15.18:54872) 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc105e20 0x7f35cc1a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53216/0 (socket says 172.21.15.18:53216) 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35caa00640 1 -- 172.21.15.18:0/162468950 learned_addr learned my addr 172.21.15.18:0/162468950 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35cb400640 1 -- 172.21.15.18:0/162468950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc104270 msgr2=0x7f35cc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35cb400640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc104270 0x7f35cc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35cb400640 1 -- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35cc105460 msgr2=0x7f35cc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35cb400640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35cc105460 0x7f35cc19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.711+0000 7f35cb400640 1 -- 172.21.15.18:0/162468950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc1a2110 con 0x7f35cc105e20 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.712+0000 7f35cb400640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc105e20 0x7f35cc1a1aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35c4037040 tx=0x7f35c4035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.712+0000 7f35c3e00640 1 -- 172.21.15.18:0/162468950 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4039040 con 0x7f35cc105e20 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.712+0000 7f35c3e00640 1 -- 172.21.15.18:0/162468950 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c4044070 con 0x7f35cc105e20 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.712+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc1a23a0 con 0x7f35cc105e20 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.712+0000 7f35c3e00640 1 -- 172.21.15.18:0/162468950 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4004350 con 0x7f35cc105e20 2024-04-16T17:01:42.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.712+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc1a2860 con 0x7f35cc105e20 2024-04-16T17:01:42.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.713+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc10b530 con 0x7f35cc105e20 2024-04-16T17:01:42.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.716+0000 7f35c3e00640 1 -- 172.21.15.18:0/162468950 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f35c40044f0 con 0x7f35cc105e20 2024-04-16T17:01:42.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.717+0000 7f35c3e00640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f35b00a3bb0 0x7f35b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:42.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.717+0000 7f35c3e00640 1 -- 172.21.15.18:0/162468950 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f35c40f2ee0 con 0x7f35cc105e20 2024-04-16T17:01:42.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.718+0000 7f35caa00640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f35b00a3bb0 0x7f35b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:42.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.719+0000 7f35caa00640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f35b00a3bb0 0x7f35b00a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0095f0 tx=0x7f35bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:42.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.720+0000 7f35c3e00640 1 -- 172.21.15.18:0/162468950 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35c40bf620 con 0x7f35cc105e20 2024-04-16T17:01:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.925+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35cc02d7b0 con 0x7f35cc105e20 2024-04-16T17:01:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.926+0000 7f35c3e00640 1 -- 172.21.15.18:0/162468950 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f35c40098b0 con 0x7f35cc105e20 2024-04-16T17:01:42.927 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:01:42.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f35b00a3bb0 msgr2=0x7f35b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f35b00a3bb0 0x7f35b00a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0095f0 tx=0x7f35bc005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc105e20 msgr2=0x7f35cc1a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc105e20 0x7f35cc1a1aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35c4037040 tx=0x7f35c4035d10 comp rx=0 tx=0).stop 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 shutdown_connections 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc105e20 0x7f35cc1a1aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f35b00a3bb0 0x7f35b00a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35cc105460 0x7f35cc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 --2- 172.21.15.18:0/162468950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35cc104270 0x7f35cc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 >> 172.21.15.18:0/162468950 conn(0x7f35cc0ff820 msgr2=0x7f35cc108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 shutdown_connections 2024-04-16T17:01:42.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:42.929+0000 7f35d1800640 1 -- 172.21.15.18:0/162468950 wait complete. 2024-04-16T17:01:42.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:01:43.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.182+0000 7f226ca00640 1 -- 172.21.15.18:0/1729945294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268074a60 msgr2=0x7f2268074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.182+0000 7f226ca00640 1 --2- 172.21.15.18:0/1729945294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268074a60 0x7f2268074ee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2250009900 tx=0x7f225002bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:43.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.182+0000 7f226ca00640 1 -- 172.21.15.18:0/1729945294 shutdown_connections 2024-04-16T17:01:43.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.182+0000 7f226ca00640 1 --2- 172.21.15.18:0/1729945294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f226810e450 0x7f2268110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.182+0000 7f226ca00640 1 --2- 172.21.15.18:0/1729945294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268074a60 0x7f2268074ee0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.182+0000 7f226ca00640 1 --2- 172.21.15.18:0/1729945294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2268076040 0x7f2268074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.182+0000 7f226ca00640 1 -- 172.21.15.18:0/1729945294 >> 172.21.15.18:0/1729945294 conn(0x7f22680fd410 msgr2=0x7f22680ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:43.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.183+0000 7f226ca00640 1 -- 172.21.15.18:0/1729945294 shutdown_connections 2024-04-16T17:01:43.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.183+0000 7f226ca00640 1 -- 172.21.15.18:0/1729945294 wait complete. 2024-04-16T17:01:43.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.184+0000 7f226ca00640 1 Processor -- start 2024-04-16T17:01:43.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.184+0000 7f226ca00640 1 -- start start 2024-04-16T17:01:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.184+0000 7f226ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2268074a60 0x7f226819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.184+0000 7f226ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268076040 0x7f226819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.184+0000 7f226ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f226810e450 0x7f22681a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.184+0000 7f226ca00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2268114120 con 0x7f226810e450 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.184+0000 7f226ca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2268113fa0 con 0x7f2268074a60 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.184+0000 7f226ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22681142a0 con 0x7f2268076040 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2268074a60 0x7f226819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f225ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268076040 0x7f226819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2268074a60 0x7f226819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54886/0 (socket says 172.21.15.18:54886) 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f225ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268076040 0x7f226819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53232/0 (socket says 172.21.15.18:53232) 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 -- 172.21.15.18:0/4216399955 learned_addr learned my addr 172.21.15.18:0/4216399955 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267e00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f226810e450 0x7f22681a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 -- 172.21.15.18:0/4216399955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268076040 msgr2=0x7f226819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268076040 0x7f226819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 -- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f226810e450 msgr2=0x7f22681a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f226810e450 0x7f22681a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 -- 172.21.15.18:0/4216399955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22681a63b0 con 0x7f2268074a60 2024-04-16T17:01:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.185+0000 7f2267400640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2268074a60 0x7f226819e790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f225800e9f0 tx=0x7f225800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.186+0000 7f2264c00640 1 -- 172.21.15.18:0/4216399955 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225800cd80 con 0x7f2268074a60 2024-04-16T17:01:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.186+0000 7f2264c00640 1 -- 172.21.15.18:0/4216399955 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2258010910 con 0x7f2268074a60 2024-04-16T17:01:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.186+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22681a66a0 con 0x7f2268074a60 2024-04-16T17:01:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.186+0000 7f2264c00640 1 -- 172.21.15.18:0/4216399955 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258018990 con 0x7f2268074a60 2024-04-16T17:01:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.186+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22681a6bc0 con 0x7f2268074a60 2024-04-16T17:01:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.187+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2268104620 con 0x7f2268074a60 2024-04-16T17:01:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.189+0000 7f2264c00640 1 -- 172.21.15.18:0/4216399955 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2258010ac0 con 0x7f2268074a60 2024-04-16T17:01:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.190+0000 7f2264c00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f224c0a3bb0 0x7f224c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.191+0000 7f2264c00640 1 -- 172.21.15.18:0/4216399955 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f22580d09c0 con 0x7f2268074a60 2024-04-16T17:01:43.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.191+0000 7f225ea00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f224c0a3bb0 0x7f224c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:43.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.191+0000 7f225ea00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f224c0a3bb0 0x7f224c0a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f225002c250 tx=0x7f2250000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:43.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.197+0000 7f2264c00640 1 -- 172.21.15.18:0/4216399955 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f225809cab0 con 0x7f2268074a60 2024-04-16T17:01:43.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.404+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2268062c00 con 0x7f2268074a60 2024-04-16T17:01:43.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.405+0000 7f2264c00640 1 -- 172.21.15.18:0/4216399955 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f22580a0db0 con 0x7f2268074a60 2024-04-16T17:01:43.407 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:01:43.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f224c0a3bb0 msgr2=0x7f224c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f224c0a3bb0 0x7f224c0a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f225002c250 tx=0x7f2250000bc0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2268074a60 msgr2=0x7f226819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2268074a60 0x7f226819e790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f225800e9f0 tx=0x7f225800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 shutdown_connections 2024-04-16T17:01:43.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f224c0a3bb0 0x7f224c0a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f226810e450 0x7f22681a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2268076040 0x7f226819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 --2- 172.21.15.18:0/4216399955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2268074a60 0x7f226819e790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 >> 172.21.15.18:0/4216399955 conn(0x7f22680fd410 msgr2=0x7f22681017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:43.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.408+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 shutdown_connections 2024-04-16T17:01:43.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.409+0000 7f226ca00640 1 -- 172.21.15.18:0/4216399955 wait complete. 2024-04-16T17:01:43.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:43 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2053689920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:43.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:43 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/162468950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:43.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:43 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2053689920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:43.427 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:43 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/162468950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:43.432 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:01:43.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:43 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2053689920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:43.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:43 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/162468950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:43.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.664+0000 7f798de00640 1 -- 172.21.15.18:0/1761039992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79880769c0 msgr2=0x7f7988076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.664+0000 7f798de00640 1 --2- 172.21.15.18:0/1761039992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79880769c0 0x7f7988076de0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7970009900 tx=0x7f797002bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.664+0000 7f798de00640 1 -- 172.21.15.18:0/1761039992 shutdown_connections 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.665+0000 7f798de00640 1 --2- 172.21.15.18:0/1761039992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79880785a0 0x7f7988078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.665+0000 7f798de00640 1 --2- 172.21.15.18:0/1761039992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7988077c00 0x7f7988078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.665+0000 7f798de00640 1 --2- 172.21.15.18:0/1761039992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79880769c0 0x7f7988076de0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.665+0000 7f798de00640 1 -- 172.21.15.18:0/1761039992 >> 172.21.15.18:0/1761039992 conn(0x7f79880ff610 msgr2=0x7f7988101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.665+0000 7f798de00640 1 -- 172.21.15.18:0/1761039992 shutdown_connections 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.665+0000 7f798de00640 1 -- 172.21.15.18:0/1761039992 wait complete. 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.666+0000 7f798de00640 1 Processor -- start 2024-04-16T17:01:43.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.666+0000 7f798de00640 1 -- start start 2024-04-16T17:01:43.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.666+0000 7f798de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79880769c0 0x7f798819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:43.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.666+0000 7f798de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7988077c00 0x7f798819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:43.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f798de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79880785a0 0x7f79881a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:43.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f798de00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7988114110 con 0x7f79880769c0 2024-04-16T17:01:43.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f798de00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7988113f90 con 0x7f79880785a0 2024-04-16T17:01:43.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f798de00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7988114290 con 0x7f7988077c00 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f798ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79880769c0 0x7f798819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f797fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7988077c00 0x7f798819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f797fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7988077c00 0x7f798819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53246/0 (socket says 172.21.15.18:53246) 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f798ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79880769c0 0x7f798819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52318/0 (socket says 172.21.15.18:52318) 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f798d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79880785a0 0x7f79881a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f797fe00640 1 -- 172.21.15.18:0/4241332244 learned_addr learned my addr 172.21.15.18:0/4241332244 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f797fe00640 1 -- 172.21.15.18:0/4241332244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79880785a0 msgr2=0x7f79881a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f797fe00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79880785a0 0x7f79881a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f797fe00640 1 -- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79880769c0 msgr2=0x7f798819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f797fe00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79880769c0 0x7f798819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.667+0000 7f797fe00640 1 -- 172.21.15.18:0/4241332244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79881a63a0 con 0x7f7988077c00 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.668+0000 7f797fe00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7988077c00 0x7f798819ecc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f797800e9f0 tx=0x7f797800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.668+0000 7f7986000640 1 -- 172.21.15.18:0/4241332244 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797800cd80 con 0x7f7988077c00 2024-04-16T17:01:43.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.668+0000 7f7986000640 1 -- 172.21.15.18:0/4241332244 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7978010910 con 0x7f7988077c00 2024-04-16T17:01:43.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.668+0000 7f7986000640 1 -- 172.21.15.18:0/4241332244 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978018990 con 0x7f7988077c00 2024-04-16T17:01:43.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.668+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79881a6690 con 0x7f7988077c00 2024-04-16T17:01:43.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.668+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79881a6c30 con 0x7f7988077c00 2024-04-16T17:01:43.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.669+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79881109f0 con 0x7f7988077c00 2024-04-16T17:01:43.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.672+0000 7f7986000640 1 -- 172.21.15.18:0/4241332244 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7978010ac0 con 0x7f7988077c00 2024-04-16T17:01:43.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.673+0000 7f7986000640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f79680a3c30 0x7f79680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:43.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.673+0000 7f7986000640 1 -- 172.21.15.18:0/4241332244 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f79780d0910 con 0x7f7988077c00 2024-04-16T17:01:43.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.673+0000 7f798ca00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f79680a3c30 0x7f79680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:43.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.674+0000 7f798ca00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f79680a3c30 0x7f79680a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f797002c7c0 tx=0x7f7970005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:43.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.678+0000 7f7986000640 1 -- 172.21.15.18:0/4241332244 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f797809ca00 con 0x7f7988077c00 2024-04-16T17:01:43.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.887+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7988062c00 con 0x7f7988077c00 2024-04-16T17:01:43.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.888+0000 7f7986000640 1 -- 172.21.15.18:0/4241332244 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f79780a0d00 con 0x7f7988077c00 2024-04-16T17:01:43.890 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f79680a3c30 msgr2=0x7f79680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f79680a3c30 0x7f79680a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f797002c7c0 tx=0x7f7970005fd0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7988077c00 msgr2=0x7f798819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7988077c00 0x7f798819ecc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f797800e9f0 tx=0x7f797800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 shutdown_connections 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79880785a0 0x7f79881a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7988077c00 0x7f798819ecc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f79680a3c30 0x7f79680a60f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 --2- 172.21.15.18:0/4241332244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79880769c0 0x7f798819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:43.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 >> 172.21.15.18:0/4241332244 conn(0x7f79880ff610 msgr2=0x7f7988101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:43.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 shutdown_connections 2024-04-16T17:01:43.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:43.891+0000 7f798de00640 1 -- 172.21.15.18:0/4241332244 wait complete. 2024-04-16T17:01:43.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:01:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.146+0000 7fedd4e00640 1 -- 172.21.15.18:0/667534129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0104080 msgr2=0x7fedd01044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.146+0000 7fedd4e00640 1 --2- 172.21.15.18:0/667534129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0104080 0x7fedd01044a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009900 tx=0x7fedc002bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:44.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.146+0000 7fedd4e00640 1 -- 172.21.15.18:0/667534129 shutdown_connections 2024-04-16T17:01:44.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.146+0000 7fedd4e00640 1 --2- 172.21.15.18:0/667534129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0105c30 0x7fedd010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.146+0000 7fedd4e00640 1 --2- 172.21.15.18:0/667534129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0105270 0x7fedd01056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.146+0000 7fedd4e00640 1 --2- 172.21.15.18:0/667534129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0104080 0x7fedd01044a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.147+0000 7fedd4e00640 1 -- 172.21.15.18:0/667534129 >> 172.21.15.18:0/667534129 conn(0x7fedd00ff610 msgr2=0x7fedd0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:44.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.147+0000 7fedd4e00640 1 -- 172.21.15.18:0/667534129 shutdown_connections 2024-04-16T17:01:44.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.147+0000 7fedd4e00640 1 -- 172.21.15.18:0/667534129 wait complete. 2024-04-16T17:01:44.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedd4e00640 1 Processor -- start 2024-04-16T17:01:44.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedd4e00640 1 -- start start 2024-04-16T17:01:44.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedd4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0104080 0x7fedd019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:44.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedd4e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0105270 0x7fedd019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedd4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0105c30 0x7fedd01a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedd4e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010fd60 con 0x7fedd0105270 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedd4e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010fbe0 con 0x7fedd0104080 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedd4e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010fee0 con 0x7fedd0105c30 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedcea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0105270 0x7fedd019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedcfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0105c30 0x7fedd01a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedcea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0105270 0x7fedd019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52330/0 (socket says 172.21.15.18:52330) 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedcea00640 1 -- 172.21.15.18:0/3905795958 learned_addr learned my addr 172.21.15.18:0/3905795958 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.148+0000 7fedcf400640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0104080 0x7fedd019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.149+0000 7fedcfe00640 1 -- 172.21.15.18:0/3905795958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0104080 msgr2=0x7fedd019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.149+0000 7fedcfe00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0104080 0x7fedd019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.149+0000 7fedcfe00640 1 -- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0105270 msgr2=0x7fedd019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.149+0000 7fedcfe00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0105270 0x7fedd019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.149+0000 7fedcfe00640 1 -- 172.21.15.18:0/3905795958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd01a1fe0 con 0x7fedd0105c30 2024-04-16T17:01:44.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.149+0000 7fedcf400640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0104080 0x7fedd019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:44.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.149+0000 7fedcea00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0105270 0x7fedd019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:44.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.149+0000 7fedcfe00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0105c30 0x7fedd01a1970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009fd0 tx=0x7fedc800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:44.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.150+0000 7fedc7e00640 1 -- 172.21.15.18:0/3905795958 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc800ce20 con 0x7fedd0105c30 2024-04-16T17:01:44.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.150+0000 7fedc7e00640 1 -- 172.21.15.18:0/3905795958 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8005020 con 0x7fedd0105c30 2024-04-16T17:01:44.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.150+0000 7fedc7e00640 1 -- 172.21.15.18:0/3905795958 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8010720 con 0x7fedd0105c30 2024-04-16T17:01:44.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.150+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd01a22d0 con 0x7fedd0105c30 2024-04-16T17:01:44.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.150+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd01a27d0 con 0x7fedd0105c30 2024-04-16T17:01:44.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.151+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd010b420 con 0x7fedd0105c30 2024-04-16T17:01:44.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.153+0000 7fedc7e00640 1 -- 172.21.15.18:0/3905795958 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fedc8005540 con 0x7fedd0105c30 2024-04-16T17:01:44.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.154+0000 7fedc7e00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fedb40a3bb0 0x7fedb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:44.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.154+0000 7fedc7e00640 1 -- 172.21.15.18:0/3905795958 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fedc80d0d50 con 0x7fedd0105c30 2024-04-16T17:01:44.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.154+0000 7fedcf400640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fedb40a3bb0 0x7fedb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:44.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.155+0000 7fedcf400640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fedb40a3bb0 0x7fedb40a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fedc002c170 tx=0x7fedc002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:44.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.159+0000 7fedc7e00640 1 -- 172.21.15.18:0/3905795958 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedc809d490 con 0x7fedd0105c30 2024-04-16T17:01:44.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:44 smithi018 ceph-mon[31776]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:44.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:44 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4216399955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:01:44.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:44 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4241332244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:01:44.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.365+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fedd0062c00 con 0x7fedd0105c30 2024-04-16T17:01:44.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.366+0000 7fedc7e00640 1 -- 172.21.15.18:0/3905795958 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fedc809cd90 con 0x7fedd0105c30 2024-04-16T17:01:44.368 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fedb40a3bb0 msgr2=0x7fedb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fedb40a3bb0 0x7fedb40a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fedc002c170 tx=0x7fedc002df90 comp rx=0 tx=0).stop 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0105c30 msgr2=0x7fedd01a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0105c30 0x7fedd01a1970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009fd0 tx=0x7fedc800ef10 comp rx=0 tx=0).stop 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 shutdown_connections 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0105c30 0x7fedd01a1970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fedb40a3bb0 0x7fedb40a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fedd0105270 0x7fedd019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 --2- 172.21.15.18:0/3905795958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedd0104080 0x7fedd019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 >> 172.21.15.18:0/3905795958 conn(0x7fedd00ff610 msgr2=0x7fedd01085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:44.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 shutdown_connections 2024-04-16T17:01:44.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.369+0000 7fedd4e00640 1 -- 172.21.15.18:0/3905795958 wait complete. 2024-04-16T17:01:44.391 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T17:01:44.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:44 smithi022 ceph-mon[36023]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:44.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:44 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4216399955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:01:44.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:44 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4241332244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:01:44.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:44 smithi043 ceph-mon[36901]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:01:44.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:44 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4216399955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:01:44.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:44 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4241332244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:01:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.617+0000 7fb93ce00640 1 -- 172.21.15.18:0/4052877401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938102fc0 msgr2=0x7fb938103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.617+0000 7fb93ce00640 1 --2- 172.21.15.18:0/4052877401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938102fc0 0x7fb938103440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb924009900 tx=0x7fb92402bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.617+0000 7fb93ce00640 1 -- 172.21.15.18:0/4052877401 shutdown_connections 2024-04-16T17:01:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.617+0000 7fb93ce00640 1 --2- 172.21.15.18:0/4052877401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb938103980 0x7fb93810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.617+0000 7fb93ce00640 1 --2- 172.21.15.18:0/4052877401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938102fc0 0x7fb938103440 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.617+0000 7fb93ce00640 1 --2- 172.21.15.18:0/4052877401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb938101e70 0x7fb938102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.617+0000 7fb93ce00640 1 -- 172.21.15.18:0/4052877401 >> 172.21.15.18:0/4052877401 conn(0x7fb9380fd3e0 msgr2=0x7fb9380ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.618+0000 7fb93ce00640 1 -- 172.21.15.18:0/4052877401 shutdown_connections 2024-04-16T17:01:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.618+0000 7fb93ce00640 1 -- 172.21.15.18:0/4052877401 wait complete. 2024-04-16T17:01:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb93ce00640 1 Processor -- start 2024-04-16T17:01:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb93ce00640 1 -- start start 2024-04-16T17:01:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb93ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938101e70 0x7fb93819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb93ce00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb938102fc0 0x7fb93819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb93ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb938103980 0x7fb9381a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb93ce00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810db70 con 0x7fb938102fc0 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb93ce00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810d9f0 con 0x7fb938103980 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb93ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810dcf0 con 0x7fb938101e70 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.619+0000 7fb937400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938101e70 0x7fb93819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938101e70 0x7fb93819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53260/0 (socket says 172.21.15.18:53260) 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937400640 1 -- 172.21.15.18:0/188796762 learned_addr learned my addr 172.21.15.18:0/188796762 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb936a00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb938102fc0 0x7fb93819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937e00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb938103980 0x7fb9381a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937400640 1 -- 172.21.15.18:0/188796762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb938103980 msgr2=0x7fb9381a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937400640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb938103980 0x7fb9381a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937400640 1 -- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb938102fc0 msgr2=0x7fb93819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937400640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb938102fc0 0x7fb93819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937400640 1 -- 172.21.15.18:0/188796762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9381a1fc0 con 0x7fb938101e70 2024-04-16T17:01:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.620+0000 7fb937400640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938101e70 0x7fb93819a3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb92800ebc0 tx=0x7fb92800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.621+0000 7fb92fe00640 1 -- 172.21.15.18:0/188796762 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928010630 con 0x7fb938101e70 2024-04-16T17:01:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.621+0000 7fb92fe00640 1 -- 172.21.15.18:0/188796762 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb928010cb0 con 0x7fb938101e70 2024-04-16T17:01:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.621+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9381a22b0 con 0x7fb938101e70 2024-04-16T17:01:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.621+0000 7fb92fe00640 1 -- 172.21.15.18:0/188796762 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928018bd0 con 0x7fb938101e70 2024-04-16T17:01:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.621+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9381a27d0 con 0x7fb938101e70 2024-04-16T17:01:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.622+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9381a2a30 con 0x7fb938101e70 2024-04-16T17:01:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.624+0000 7fb92fe00640 1 -- 172.21.15.18:0/188796762 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9280107d0 con 0x7fb938101e70 2024-04-16T17:01:44.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.625+0000 7fb92fe00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:44.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.626+0000 7fb92fe00640 1 -- 172.21.15.18:0/188796762 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb9280d0c50 con 0x7fb938101e70 2024-04-16T17:01:44.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.626+0000 7fb936a00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:44.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.627+0000 7fb936a00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb92402c180 tx=0x7fb92402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:44.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.628+0000 7fb92fe00640 1 -- 172.21.15.18:0/188796762 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb92809cd40 con 0x7fb938101e70 2024-04-16T17:01:44.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.846+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9381a2c00 con 0x7fb938101e70 2024-04-16T17:01:44.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.847+0000 7fb92fe00640 1 -- 172.21.15.18:0/188796762 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+592 (secure 0 0 0) 0x7fb9280a1040 con 0x7fb938101e70 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2364490471 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1759186913 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2887519304 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2656869540 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1032645701 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1116733165 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1116733165 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1759186913 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2417150453 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:44.849 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3503805752 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:44.850 INFO:teuthology.orchestra.run.smithi018.stderr:listed 10 entries 2024-04-16T17:01:44.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.849+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb91c0a3bb0 msgr2=0x7fb91c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.849+0000 7fb93ce00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb92402c180 tx=0x7fb92402cf90 comp rx=0 tx=0).stop 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938101e70 msgr2=0x7fb93819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938101e70 0x7fb93819a3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb92800ebc0 tx=0x7fb92800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 shutdown_connections 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb938103980 0x7fb9381a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb938102fc0 0x7fb93819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 --2- 172.21.15.18:0/188796762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb938101e70 0x7fb93819a3a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 >> 172.21.15.18:0/188796762 conn(0x7fb9380fd3e0 msgr2=0x7fb938074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 shutdown_connections 2024-04-16T17:01:44.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:44.850+0000 7fb93ce00640 1 -- 172.21.15.18:0/188796762 wait complete. 2024-04-16T17:01:44.872 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T17:01:44.872 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:01:44.926 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 -- 172.21.15.18:0/2259032659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 msgr2=0x7fc7f010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 --2- 172.21.15.18:0/2259032659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 0x7fc7f010a360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009900 tx=0x7fc7e802bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 -- 172.21.15.18:0/2259032659 shutdown_connections 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 --2- 172.21.15.18:0/2259032659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 0x7fc7f010a360 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 --2- 172.21.15.18:0/2259032659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f0103080 0x7fc7f0103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 --2- 172.21.15.18:0/2259032659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f0101f30 0x7fc7f0102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 -- 172.21.15.18:0/2259032659 >> 172.21.15.18:0/2259032659 conn(0x7fc7f00fd4c0 msgr2=0x7fc7f00ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 -- 172.21.15.18:0/2259032659 shutdown_connections 2024-04-16T17:01:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.156+0000 7fc7f6400640 1 -- 172.21.15.18:0/2259032659 wait complete. 2024-04-16T17:01:45.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.157+0000 7fc7f6400640 1 Processor -- start 2024-04-16T17:01:45.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.157+0000 7fc7f6400640 1 -- start start 2024-04-16T17:01:45.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f0101f30 0x7fc7f019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:45.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f6400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f0103080 0x7fc7f019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:45.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f6400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 0x7fc7f01a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:45.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f6400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f010dc30 con 0x7fc7f0103080 2024-04-16T17:01:45.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f6400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f010dab0 con 0x7fc7f0103a40 2024-04-16T17:01:45.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f6400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f010ddb0 con 0x7fc7f0101f30 2024-04-16T17:01:45.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f0103080 0x7fc7f019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f0101f30 0x7fc7f019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 0x7fc7f01a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f0101f30 0x7fc7f019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53284/0 (socket says 172.21.15.18:53284) 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f5000640 1 -- 172.21.15.18:0/3953816089 learned_addr learned my addr 172.21.15.18:0/3953816089 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.158+0000 7fc7f5000640 1 -- 172.21.15.18:0/3953816089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 msgr2=0x7fc7f01a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7f5000640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 0x7fc7f01a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7f5000640 1 -- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f0103080 msgr2=0x7fc7f019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7f5000640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f0103080 0x7fc7f019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7f5000640 1 -- 172.21.15.18:0/3953816089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f01a2030 con 0x7fc7f0101f30 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7efe00640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f0103080 0x7fc7f019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7f5a00640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 0x7fc7f01a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:45.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7f5000640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f0101f30 0x7fc7f019a410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009d50 tx=0x7fc7e000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:45.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7ed600640 1 -- 172.21.15.18:0/3953816089 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0003b70 con 0x7fc7f0101f30 2024-04-16T17:01:45.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7ed600640 1 -- 172.21.15.18:0/3953816089 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e0003d10 con 0x7fc7f0101f30 2024-04-16T17:01:45.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7ed600640 1 -- 172.21.15.18:0/3953816089 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0010a20 con 0x7fc7f0101f30 2024-04-16T17:01:45.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f01a2320 con 0x7fc7f0101f30 2024-04-16T17:01:45.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.159+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f01a2820 con 0x7fc7f0101f30 2024-04-16T17:01:45.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.160+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f01a2ae0 con 0x7fc7f0101f30 2024-04-16T17:01:45.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.162+0000 7fc7ed600640 1 -- 172.21.15.18:0/3953816089 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc7e0003ec0 con 0x7fc7f0101f30 2024-04-16T17:01:45.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.163+0000 7fc7ed600640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc7d40a3c20 0x7fc7d40a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:45.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.164+0000 7fc7ed600640 1 -- 172.21.15.18:0/3953816089 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc7e00d0c90 con 0x7fc7f0101f30 2024-04-16T17:01:45.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.164+0000 7fc7efe00640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc7d40a3c20 0x7fc7d40a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:45.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.165+0000 7fc7efe00640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc7d40a3c20 0x7fc7d40a60e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7f019b920 tx=0x7fc7dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:45.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.168+0000 7fc7ed600640 1 -- 172.21.15.18:0/3953816089 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7e009d3d0 con 0x7fc7f0101f30 2024-04-16T17:01:45.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:45 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3905795958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:01:45.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:45 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/188796762' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:01:45.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.376+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7f01a2cb0 con 0x7fc7f0101f30 2024-04-16T17:01:45.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.378+0000 7fc7ed600640 1 -- 172.21.15.18:0/3953816089 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7fc7e009ccd0 con 0x7fc7f0101f30 2024-04-16T17:01:45.380 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:45.380 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc7d40a3c20 msgr2=0x7fc7d40a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc7d40a3c20 0x7fc7d40a60e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7f019b920 tx=0x7fc7dc005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f0101f30 msgr2=0x7fc7f019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f0101f30 0x7fc7f019a410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009d50 tx=0x7fc7e000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 shutdown_connections 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f0103a40 0x7fc7f01a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc7d40a3c20 0x7fc7d40a60e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f0103080 0x7fc7f019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 --2- 172.21.15.18:0/3953816089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f0101f30 0x7fc7f019a410 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 >> 172.21.15.18:0/3953816089 conn(0x7fc7f00fd4c0 msgr2=0x7fc7f0074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 shutdown_connections 2024-04-16T17:01:45.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.381+0000 7fc7f6400640 1 -- 172.21.15.18:0/3953816089 wait complete. 2024-04-16T17:01:45.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:01:45.404 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:01:45.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:45 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3905795958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:01:45.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:45 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/188796762' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:01:45.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:45 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3905795958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:01:45.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:45 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/188796762' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:01:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 -- 172.21.15.18:0/3187917248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698100890 msgr2=0x7f3698100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 --2- 172.21.15.18:0/3187917248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698100890 0x7f3698100cb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3688009ca0 tx=0x7f368802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 -- 172.21.15.18:0/3187917248 shutdown_connections 2024-04-16T17:01:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 --2- 172.21.15.18:0/3187917248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3698101b90 0x7f3698102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 --2- 172.21.15.18:0/3187917248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36981011f0 0x7f3698101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 --2- 172.21.15.18:0/3187917248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698100890 0x7f3698100cb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 -- 172.21.15.18:0/3187917248 >> 172.21.15.18:0/3187917248 conn(0x7f36980fc470 msgr2=0x7f36980fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:45.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 -- 172.21.15.18:0/3187917248 shutdown_connections 2024-04-16T17:01:45.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.621+0000 7f369e600640 1 -- 172.21.15.18:0/3187917248 wait complete. 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.622+0000 7f369e600640 1 Processor -- start 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.622+0000 7f369e600640 1 -- start start 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f369e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3698100890 0x7f369819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f369e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36981011f0 0x7f369819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f369e600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698101b90 0x7f36981a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f369e600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f369810fe90 con 0x7f3698100890 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f369e600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f369810fd10 con 0x7f3698101b90 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f369e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3698110010 con 0x7f36981011f0 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f3697400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3698100890 0x7f369819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f3697e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698101b90 0x7f36981a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f3697400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3698100890 0x7f369819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52388/0 (socket says 172.21.15.18:52388) 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f3697400640 1 -- 172.21.15.18:0/84455552 learned_addr learned my addr 172.21.15.18:0/84455552 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.623+0000 7f3696a00640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36981011f0 0x7f369819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f3697400640 1 -- 172.21.15.18:0/84455552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36981011f0 msgr2=0x7f369819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f3697400640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36981011f0 0x7f369819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f3697400640 1 -- 172.21.15.18:0/84455552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698101b90 msgr2=0x7f36981a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f3697400640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698101b90 0x7f36981a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f3697400640 1 -- 172.21.15.18:0/84455552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36981a20e0 con 0x7f3698100890 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f3697e00640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698101b90 0x7f36981a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f3696a00640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36981011f0 0x7f369819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f3697400640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3698100890 0x7f369819a4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f36880325f0 tx=0x7f3688032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f368fe00640 1 -- 172.21.15.18:0/84455552 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688041020 con 0x7f3698100890 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36981a2370 con 0x7f3698100890 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f368fe00640 1 -- 172.21.15.18:0/84455552 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3688034cd0 con 0x7f3698100890 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.624+0000 7f368fe00640 1 -- 172.21.15.18:0/84455552 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688039040 con 0x7f3698100890 2024-04-16T17:01:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.625+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36981a2880 con 0x7f3698100890 2024-04-16T17:01:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.626+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3698068eb0 con 0x7f3698100890 2024-04-16T17:01:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.628+0000 7f368fe00640 1 -- 172.21.15.18:0/84455552 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3688044020 con 0x7f3698100890 2024-04-16T17:01:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.630+0000 7f368fe00640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f367c0a3c30 0x7f367c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.630+0000 7f368fe00640 1 -- 172.21.15.18:0/84455552 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f36880f23a0 con 0x7f3698100890 2024-04-16T17:01:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.630+0000 7f3696a00640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f367c0a3c30 0x7f367c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.631+0000 7f3696a00640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f367c0a3c30 0x7f367c0a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f369819b9d0 tx=0x7f3684005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:45.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.635+0000 7f368fe00640 1 -- 172.21.15.18:0/84455552 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f368804b020 con 0x7f3698100890 2024-04-16T17:01:45.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.843+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3698062c00 con 0x7f3698100890 2024-04-16T17:01:45.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.844+0000 7f368fe00640 1 -- 172.21.15.18:0/84455552 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f36880beb70 con 0x7f3698100890 2024-04-16T17:01:45.846 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f367c0a3c30 msgr2=0x7f367c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f367c0a3c30 0x7f367c0a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f369819b9d0 tx=0x7f3684005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3698100890 msgr2=0x7f369819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3698100890 0x7f369819a4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f36880325f0 tx=0x7f3688032ec0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 shutdown_connections 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3698101b90 0x7f36981a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36981011f0 0x7f369819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f367c0a3c30 0x7f367c0a60f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 --2- 172.21.15.18:0/84455552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3698100890 0x7f369819a4c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 >> 172.21.15.18:0/84455552 conn(0x7f36980fc470 msgr2=0x7f36980fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 shutdown_connections 2024-04-16T17:01:45.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:45.847+0000 7f369e600640 1 -- 172.21.15.18:0/84455552 wait complete. 2024-04-16T17:01:45.869 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:01:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 -- 172.21.15.18:0/2219998122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3684100500 msgr2=0x7f3684100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 --2- 172.21.15.18:0/2219998122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3684100500 0x7f3684100920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3674009860 tx=0x7f367402bb70 comp rx=0 tx=0).stop 2024-04-16T17:01:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 -- 172.21.15.18:0/2219998122 shutdown_connections 2024-04-16T17:01:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 --2- 172.21.15.18:0/2219998122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36841020b0 0x7f3684108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 --2- 172.21.15.18:0/2219998122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36841016f0 0x7f3684101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 --2- 172.21.15.18:0/2219998122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3684100500 0x7f3684100920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 -- 172.21.15.18:0/2219998122 >> 172.21.15.18:0/2219998122 conn(0x7f36840fba70 msgr2=0x7f36840fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 -- 172.21.15.18:0/2219998122 shutdown_connections 2024-04-16T17:01:46.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.099+0000 7f3688a00640 1 -- 172.21.15.18:0/2219998122 wait complete. 2024-04-16T17:01:46.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.100+0000 7f3688a00640 1 Processor -- start 2024-04-16T17:01:46.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3688a00640 1 -- start start 2024-04-16T17:01:46.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3688a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3684100500 0x7f368419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3688a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36841016f0 0x7f368419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3688a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36841020b0 0x7f36841a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3688a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f368410b910 con 0x7f36841016f0 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3688a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f368410b790 con 0x7f3684100500 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3688a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f368410ba90 con 0x7f36841020b0 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3682a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36841016f0 0x7f368419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3683400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3684100500 0x7f368419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3683400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3684100500 0x7f368419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55004/0 (socket says 172.21.15.18:55004) 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.101+0000 7f3683400640 1 -- 172.21.15.18:0/1482033154 learned_addr learned my addr 172.21.15.18:0/1482033154 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.102+0000 7f3683e00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36841020b0 0x7f36841a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.102+0000 7f3683400640 1 -- 172.21.15.18:0/1482033154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36841020b0 msgr2=0x7f36841a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.102+0000 7f3683400640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36841020b0 0x7f36841a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.102+0000 7f3683400640 1 -- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36841016f0 msgr2=0x7f368419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.102+0000 7f3683400640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36841016f0 0x7f368419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.102+0000 7f3683400640 1 -- 172.21.15.18:0/1482033154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36841a2010 con 0x7f3684100500 2024-04-16T17:01:46.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.102+0000 7f3683400640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3684100500 0x7f368419a440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f36740349e0 tx=0x7f3674032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:46.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.103+0000 7f367be00640 1 -- 172.21.15.18:0/1482033154 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674041020 con 0x7f3684100500 2024-04-16T17:01:46.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.103+0000 7f367be00640 1 -- 172.21.15.18:0/1482033154 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f367402c9f0 con 0x7f3684100500 2024-04-16T17:01:46.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.103+0000 7f367be00640 1 -- 172.21.15.18:0/1482033154 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674039040 con 0x7f3684100500 2024-04-16T17:01:46.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.103+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36841a22a0 con 0x7f3684100500 2024-04-16T17:01:46.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.103+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36841a27e0 con 0x7f3684100500 2024-04-16T17:01:46.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.104+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3684107820 con 0x7f3684100500 2024-04-16T17:01:46.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.106+0000 7f367be00640 1 -- 172.21.15.18:0/1482033154 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3674044020 con 0x7f3684100500 2024-04-16T17:01:46.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.107+0000 7f367be00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f36680a3c30 0x7f36680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:46.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.107+0000 7f367be00640 1 -- 172.21.15.18:0/1482033154 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f36740f1e70 con 0x7f3684100500 2024-04-16T17:01:46.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.108+0000 7f3682a00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f36680a3c30 0x7f36680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:46.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.108+0000 7f3682a00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f36680a3c30 0x7f36680a60f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f367000a7d0 tx=0x7f3670010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:46.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.111+0000 7f367be00640 1 -- 172.21.15.18:0/1482033154 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f367404b020 con 0x7f3684100500 2024-04-16T17:01:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:46 smithi018 ceph-mon[31776]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:01:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:46 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3953816089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:46 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/84455552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:46.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.319+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3684062c00 con 0x7f3684100500 2024-04-16T17:01:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.320+0000 7f367be00640 1 -- 172.21.15.18:0/1482033154 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f36740be5b0 con 0x7f3684100500 2024-04-16T17:01:46.322 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f36680a3c30 msgr2=0x7f36680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f36680a3c30 0x7f36680a60f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f367000a7d0 tx=0x7f3670010040 comp rx=0 tx=0).stop 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3684100500 msgr2=0x7f368419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3684100500 0x7f368419a440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f36740349e0 tx=0x7f3674032430 comp rx=0 tx=0).stop 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 shutdown_connections 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36841020b0 0x7f36841a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f36680a3c30 0x7f36680a60f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36841016f0 0x7f368419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 --2- 172.21.15.18:0/1482033154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3684100500 0x7f368419a440 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 >> 172.21.15.18:0/1482033154 conn(0x7f36840fba70 msgr2=0x7f36841049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:46.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 shutdown_connections 2024-04-16T17:01:46.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.324+0000 7f3688a00640 1 -- 172.21.15.18:0/1482033154 wait complete. 2024-04-16T17:01:46.344 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:01:46.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:46 smithi022 ceph-mon[36023]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:01:46.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:46 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3953816089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:46.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:46 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/84455552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:46 smithi043 ceph-mon[36901]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:01:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:46 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3953816089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:46 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/84455552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:01:46.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.568+0000 7f045be00640 1 -- 172.21.15.18:0/2492905425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 msgr2=0x7f0454102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.568+0000 7f045be00640 1 --2- 172.21.15.18:0/2492905425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 0x7f0454102470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0448009900 tx=0x7f044802bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:46.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.568+0000 7f045be00640 1 -- 172.21.15.18:0/2492905425 shutdown_connections 2024-04-16T17:01:46.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.569+0000 7f045be00640 1 --2- 172.21.15.18:0/2492905425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0454103c00 0x7f045410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.569+0000 7f045be00640 1 --2- 172.21.15.18:0/2492905425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0454103240 0x7f04541036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.569+0000 7f045be00640 1 --2- 172.21.15.18:0/2492905425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 0x7f0454102470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.569+0000 7f045be00640 1 -- 172.21.15.18:0/2492905425 >> 172.21.15.18:0/2492905425 conn(0x7f04540fd5c0 msgr2=0x7f04540ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:46.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.569+0000 7f045be00640 1 -- 172.21.15.18:0/2492905425 shutdown_connections 2024-04-16T17:01:46.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.569+0000 7f045be00640 1 -- 172.21.15.18:0/2492905425 wait complete. 2024-04-16T17:01:46.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.570+0000 7f045be00640 1 Processor -- start 2024-04-16T17:01:46.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.570+0000 7f045be00640 1 -- start start 2024-04-16T17:01:46.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.570+0000 7f045be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 0x7f045419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.570+0000 7f045be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0454103240 0x7f045419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.570+0000 7f045be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0454103c00 0x7f04541a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f045be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f045410dce0 con 0x7f0454103240 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f045be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f045410db60 con 0x7f0454103c00 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f045be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f045410de60 con 0x7f0454102050 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 0x7f045419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0458a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0454103240 0x7f045419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 0x7f045419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53336/0 (socket says 172.21.15.18:53336) 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459400640 1 -- 172.21.15.18:0/650296756 learned_addr learned my addr 172.21.15.18:0/650296756 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459e00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0454103c00 0x7f04541a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459400640 1 -- 172.21.15.18:0/650296756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0454103c00 msgr2=0x7f04541a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459400640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0454103c00 0x7f04541a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459400640 1 -- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0454103240 msgr2=0x7f045419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459400640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0454103240 0x7f045419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.571+0000 7f0459400640 1 -- 172.21.15.18:0/650296756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04541a21a0 con 0x7f0454102050 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.572+0000 7f0459400640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 0x7f045419a580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0448000c00 tx=0x7f0448005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.572+0000 7f0452000640 1 -- 172.21.15.18:0/650296756 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448041070 con 0x7f0454102050 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.572+0000 7f0452000640 1 -- 172.21.15.18:0/650296756 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04480044d0 con 0x7f0454102050 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.572+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04541a2430 con 0x7f0454102050 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.572+0000 7f0452000640 1 -- 172.21.15.18:0/650296756 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448034440 con 0x7f0454102050 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.572+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04541a2970 con 0x7f0454102050 2024-04-16T17:01:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.573+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04541a2c90 con 0x7f0454102050 2024-04-16T17:01:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.575+0000 7f0452000640 1 -- 172.21.15.18:0/650296756 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0448044070 con 0x7f0454102050 2024-04-16T17:01:46.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.576+0000 7f0452000640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f043c0a3c30 0x7f043c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:46.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.577+0000 7f0452000640 1 -- 172.21.15.18:0/650296756 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f04480f2390 con 0x7f0454102050 2024-04-16T17:01:46.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.577+0000 7f0458a00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f043c0a3c30 0x7f043c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:46.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.577+0000 7f0458a00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f043c0a3c30 0x7f043c0a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f045419ba90 tx=0x7f0440005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:46.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.579+0000 7f0452000640 1 -- 172.21.15.18:0/650296756 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04480be480 con 0x7f0454102050 2024-04-16T17:01:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.788+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04541a2e60 con 0x7f0454102050 2024-04-16T17:01:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.789+0000 7f0452000640 1 -- 172.21.15.18:0/650296756 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f04480c2780 con 0x7f0454102050 2024-04-16T17:01:46.790 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:01:46.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f043c0a3c30 msgr2=0x7f043c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f043c0a3c30 0x7f043c0a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f045419ba90 tx=0x7f0440005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:46.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 msgr2=0x7f045419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 0x7f045419a580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0448000c00 tx=0x7f0448005d50 comp rx=0 tx=0).stop 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 shutdown_connections 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0454103c00 0x7f04541a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f043c0a3c30 0x7f043c0a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0454103240 0x7f045419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 --2- 172.21.15.18:0/650296756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0454102050 0x7f045419a580 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.791+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 >> 172.21.15.18:0/650296756 conn(0x7f04540fd5c0 msgr2=0x7f0454074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.792+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 shutdown_connections 2024-04-16T17:01:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:46.792+0000 7f045be00640 1 -- 172.21.15.18:0/650296756 wait complete. 2024-04-16T17:01:46.813 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:01:47.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 -- 172.21.15.18:0/143378118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b8075fc0 msgr2=0x7f83b8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 --2- 172.21.15.18:0/143378118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b8075fc0 0x7f83b8074410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009ca0 tx=0x7f83ac02c190 comp rx=0 tx=0).stop 2024-04-16T17:01:47.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 -- 172.21.15.18:0/143378118 shutdown_connections 2024-04-16T17:01:47.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 --2- 172.21.15.18:0/143378118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83b8109d40 0x7f83b810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 --2- 172.21.15.18:0/143378118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83b80749e0 0x7f83b8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 --2- 172.21.15.18:0/143378118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b8075fc0 0x7f83b8074410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 -- 172.21.15.18:0/143378118 >> 172.21.15.18:0/143378118 conn(0x7f83b80fd400 msgr2=0x7f83b80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:47.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 -- 172.21.15.18:0/143378118 shutdown_connections 2024-04-16T17:01:47.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.035+0000 7f83bee00640 1 -- 172.21.15.18:0/143378118 wait complete. 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.036+0000 7f83bee00640 1 Processor -- start 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.036+0000 7f83bee00640 1 -- start start 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b80749e0 0x7f83b819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bee00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83b8075fc0 0x7f83b819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bee00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83b8109d40 0x7f83b81a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bee00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f83b810fd30 con 0x7f83b8109d40 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bee00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f83b810fbb0 con 0x7f83b8075fc0 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f83b810feb0 con 0x7f83b80749e0 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83be400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83b8109d40 0x7f83b81a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:47.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b80749e0 0x7f83b819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b80749e0 0x7f83b819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50064/0 (socket says 172.21.15.18:50064) 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bda00640 1 -- 172.21.15.18:0/3153865324 learned_addr learned my addr 172.21.15.18:0/3153865324 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bd000640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83b8075fc0 0x7f83b819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bda00640 1 -- 172.21.15.18:0/3153865324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83b8075fc0 msgr2=0x7f83b819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.037+0000 7f83bda00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83b8075fc0 0x7f83b819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83bda00640 1 -- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83b8109d40 msgr2=0x7f83b81a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83bda00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83b8109d40 0x7f83b81a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83bda00640 1 -- 172.21.15.18:0/3153865324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b81a1f70 con 0x7f83b80749e0 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83bda00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b80749e0 0x7f83b819a350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0326d0 tx=0x7f83ac035f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83b6000640 1 -- 172.21.15.18:0/3153865324 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac039040 con 0x7f83b80749e0 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83b6000640 1 -- 172.21.15.18:0/3153865324 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ac044070 con 0x7f83b80749e0 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83b6000640 1 -- 172.21.15.18:0/3153865324 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0043a0 con 0x7f83b80749e0 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b81a2200 con 0x7f83b80749e0 2024-04-16T17:01:47.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.038+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b81a2670 con 0x7f83b80749e0 2024-04-16T17:01:47.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.039+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b8108b00 con 0x7f83b80749e0 2024-04-16T17:01:47.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.042+0000 7f83b6000640 1 -- 172.21.15.18:0/3153865324 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f83ac004540 con 0x7f83b80749e0 2024-04-16T17:01:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.043+0000 7f83b6000640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f83a00a3c30 0x7f83a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.043+0000 7f83b6000640 1 -- 172.21.15.18:0/3153865324 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f83ac0f3530 con 0x7f83b80749e0 2024-04-16T17:01:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.043+0000 7f83bd000640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f83a00a3c30 0x7f83a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.044+0000 7f83bd000640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f83a00a3c30 0x7f83a00a60f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f83b819b860 tx=0x7f83a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:47.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.045+0000 7f83b6000640 1 -- 172.21.15.18:0/3153865324 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83ac0bfc70 con 0x7f83b80749e0 2024-04-16T17:01:47.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.255+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f83b8062c00 con 0x7f83b80749e0 2024-04-16T17:01:47.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.256+0000 7f83b6000640 1 -- 172.21.15.18:0/3153865324 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f83ac0bf570 con 0x7f83b80749e0 2024-04-16T17:01:47.258 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.258+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f83a00a3c30 msgr2=0x7f83a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.258+0000 7f83bee00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f83a00a3c30 0x7f83a00a60f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f83b819b860 tx=0x7f83a4005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.258+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b80749e0 msgr2=0x7f83b819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.258+0000 7f83bee00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b80749e0 0x7f83b819a350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0326d0 tx=0x7f83ac035f60 comp rx=0 tx=0).stop 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.259+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 shutdown_connections 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.259+0000 7f83bee00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f83a00a3c30 0x7f83a00a60f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.259+0000 7f83bee00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83b8109d40 0x7f83b81a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.259+0000 7f83bee00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83b8075fc0 0x7f83b819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.259+0000 7f83bee00640 1 --2- 172.21.15.18:0/3153865324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83b80749e0 0x7f83b819a350 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.259+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 >> 172.21.15.18:0/3153865324 conn(0x7f83b80fd400 msgr2=0x7f83b8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.259+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 shutdown_connections 2024-04-16T17:01:47.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.259+0000 7f83bee00640 1 -- 172.21.15.18:0/3153865324 wait complete. 2024-04-16T17:01:47.280 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T17:01:47.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:47 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1482033154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:01:47.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:47 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/650296756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:01:47.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:47 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1482033154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:01:47.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:47 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/650296756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:01:47.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:47 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1482033154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:01:47.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:47 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/650296756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:01:47.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.505+0000 7fb326e00640 1 -- 172.21.15.18:0/1700133606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3201028f0 msgr2=0x7fb320104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.505+0000 7fb326e00640 1 --2- 172.21.15.18:0/1700133606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3201028f0 0x7fb320104ce0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb314009900 tx=0x7fb31402bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:47.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.506+0000 7fb326e00640 1 -- 172.21.15.18:0/1700133606 shutdown_connections 2024-04-16T17:01:47.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.506+0000 7fb326e00640 1 --2- 172.21.15.18:0/1700133606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb320107b50 0x7fb320109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.506+0000 7fb326e00640 1 --2- 172.21.15.18:0/1700133606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb320105220 0x7fb320107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.506+0000 7fb326e00640 1 --2- 172.21.15.18:0/1700133606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3201028f0 0x7fb320104ce0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.506+0000 7fb326e00640 1 -- 172.21.15.18:0/1700133606 >> 172.21.15.18:0/1700133606 conn(0x7fb3200fc2e0 msgr2=0x7fb3200fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:47.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.506+0000 7fb326e00640 1 -- 172.21.15.18:0/1700133606 shutdown_connections 2024-04-16T17:01:47.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.506+0000 7fb326e00640 1 -- 172.21.15.18:0/1700133606 wait complete. 2024-04-16T17:01:47.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.507+0000 7fb326e00640 1 Processor -- start 2024-04-16T17:01:47.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.507+0000 7fb326e00640 1 -- start start 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.507+0000 7fb326e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3201028f0 0x7fb320198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.507+0000 7fb326e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb320105220 0x7fb320198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.507+0000 7fb326e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb320107b50 0x7fb32019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb32010db10 con 0x7fb320107b50 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb32010d990 con 0x7fb320105220 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb32010dc90 con 0x7fb3201028f0 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb320107b50 0x7fb32019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb325a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3201028f0 0x7fb320198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb325000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb320105220 0x7fb320198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb320107b50 0x7fb32019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33300/0 (socket says 172.21.15.18:33300) 2024-04-16T17:01:47.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326400640 1 -- 172.21.15.18:0/1245928048 learned_addr learned my addr 172.21.15.18:0/1245928048 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:47.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326400640 1 -- 172.21.15.18:0/1245928048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3201028f0 msgr2=0x7fb320198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326400640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3201028f0 0x7fb320198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326400640 1 -- 172.21.15.18:0/1245928048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb320105220 msgr2=0x7fb320198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326400640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb320105220 0x7fb320198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.508+0000 7fb326400640 1 -- 172.21.15.18:0/1245928048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32019fda0 con 0x7fb320107b50 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.509+0000 7fb326400640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb320107b50 0x7fb32019f730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb31800eb10 tx=0x7fb31800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.509+0000 7fb31e000640 1 -- 172.21.15.18:0/1245928048 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318004b20 con 0x7fb320107b50 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.509+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3201a0090 con 0x7fb320107b50 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.509+0000 7fb31e000640 1 -- 172.21.15.18:0/1245928048 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb318004cc0 con 0x7fb320107b50 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.509+0000 7fb31e000640 1 -- 172.21.15.18:0/1245928048 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318010760 con 0x7fb320107b50 2024-04-16T17:01:47.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.509+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3201a05e0 con 0x7fb320107b50 2024-04-16T17:01:47.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.510+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3200fddd0 con 0x7fb320107b50 2024-04-16T17:01:47.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.512+0000 7fb31e000640 1 -- 172.21.15.18:0/1245928048 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb318005020 con 0x7fb320107b50 2024-04-16T17:01:47.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.514+0000 7fb31e000640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3080a3bb0 0x7fb3080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:47.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.514+0000 7fb31e000640 1 -- 172.21.15.18:0/1245928048 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb3180d0cc0 con 0x7fb320107b50 2024-04-16T17:01:47.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.514+0000 7fb325a00640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3080a3bb0 0x7fb3080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:47.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.515+0000 7fb325a00640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3080a3bb0 0x7fb3080a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb31402c7c0 tx=0x7fb314005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:47.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.518+0000 7fb31e000640 1 -- 172.21.15.18:0/1245928048 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3180d7080 con 0x7fb320107b50 2024-04-16T17:01:47.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.745+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb320062c00 con 0x7fb320107b50 2024-04-16T17:01:47.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.746+0000 7fb31e000640 1 -- 172.21.15.18:0/1245928048 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+592 (secure 0 0 0) 0x7fb31809d400 con 0x7fb320107b50 2024-04-16T17:01:47.748 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2364490471 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:47.748 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1759186913 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:47.748 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2887519304 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:47.748 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2656869540 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:47.748 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1032645701 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:47.749 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1116733165 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:47.749 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1116733165 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:47.749 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1759186913 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:47.749 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2417150453 2024-04-17T16:52:49.432499+0000 2024-04-16T17:01:47.749 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3503805752 2024-04-17T16:53:24.732531+0000 2024-04-16T17:01:47.749 INFO:teuthology.orchestra.run.smithi018.stderr:listed 10 entries 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3080a3bb0 msgr2=0x7fb3080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3080a3bb0 0x7fb3080a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb31402c7c0 tx=0x7fb314005fd0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb320107b50 msgr2=0x7fb32019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb320107b50 0x7fb32019f730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb31800eb10 tx=0x7fb31800c2d0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 shutdown_connections 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3080a3bb0 0x7fb3080a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb320107b50 0x7fb32019f730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb320105220 0x7fb320198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 --2- 172.21.15.18:0/1245928048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3201028f0 0x7fb320198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 >> 172.21.15.18:0/1245928048 conn(0x7fb3200fc2e0 msgr2=0x7fb320103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:47.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 shutdown_connections 2024-04-16T17:01:47.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:47.750+0000 7fb326e00640 1 -- 172.21.15.18:0/1245928048 wait complete. 2024-04-16T17:01:47.772 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi018.front.sepia.ceph.com... 2024-04-16T17:01:47.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:47.772 DEBUG:teuthology.orchestra.run.smithi018:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:01:47.870 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi018.stderr:ceph-fuse[73275]: fuse finished with error 0 and tester_r 0 2024-04-16T17:01:47.877 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T17:01:48.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:48 smithi022 ceph-mon[36023]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:01:48.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:48 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3153865324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:01:48.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:48 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1245928048' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:01:48.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:48 smithi018 ceph-mon[31776]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:01:48.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:48 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3153865324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:01:48.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:48 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1245928048' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:01:48.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:48 smithi043 ceph-mon[36901]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:01:48.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:48 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3153865324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:01:48.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:48 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1245928048' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:01:50.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:50 smithi022 ceph-mon[36023]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:50.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:50 smithi018 ceph-mon[31776]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:50.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:50 smithi043 ceph-mon[36901]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:52.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:52 smithi022 ceph-mon[36023]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:52.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:52 smithi018 ceph-mon[31776]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:52 smithi043 ceph-mon[36901]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:53.979 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:01:53.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:01:53.979 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-16T17:01:54.048 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:01:54.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 -- 172.21.15.18:0/2542565892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef04107c20 msgr2=0x7fef0410a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 --2- 172.21.15.18:0/2542565892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef04107c20 0x7fef0410a070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feefc009be0 tx=0x7feefc02bef0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 -- 172.21.15.18:0/2542565892 shutdown_connections 2024-04-16T17:01:54.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 --2- 172.21.15.18:0/2542565892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef04107c20 0x7fef0410a070 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 --2- 172.21.15.18:0/2542565892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef041052f0 0x7fef041076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 --2- 172.21.15.18:0/2542565892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef041029c0 0x7fef04104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 -- 172.21.15.18:0/2542565892 >> 172.21.15.18:0/2542565892 conn(0x7fef040fc3d0 msgr2=0x7fef040fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:54.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 -- 172.21.15.18:0/2542565892 shutdown_connections 2024-04-16T17:01:54.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.277+0000 7fef09a00640 1 -- 172.21.15.18:0/2542565892 wait complete. 2024-04-16T17:01:54.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.278+0000 7fef09a00640 1 Processor -- start 2024-04-16T17:01:54.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.278+0000 7fef09a00640 1 -- start start 2024-04-16T17:01:54.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef09a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef041029c0 0x7fef041981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:54.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef09a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef041052f0 0x7fef04198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:54.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef09a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef04107c20 0x7fef0419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:54.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef09a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410dc20 con 0x7fef041052f0 2024-04-16T17:01:54.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef09a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410daa0 con 0x7fef04107c20 2024-04-16T17:01:54.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef09a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410dda0 con 0x7fef041029c0 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef03e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef041029c0 0x7fef041981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7feefb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef041052f0 0x7fef04198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef03e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef041029c0 0x7fef041981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50098/0 (socket says 172.21.15.18:50098) 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef03e00640 1 -- 172.21.15.18:0/2918018384 learned_addr learned my addr 172.21.15.18:0/2918018384 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.279+0000 7fef09000640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef04107c20 0x7fef0419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef03e00640 1 -- 172.21.15.18:0/2918018384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef04107c20 msgr2=0x7fef0419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef03e00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef04107c20 0x7fef0419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef03e00640 1 -- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef041052f0 msgr2=0x7fef04198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef03e00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef041052f0 0x7fef04198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef03e00640 1 -- 172.21.15.18:0/2918018384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0419fe10 con 0x7fef041029c0 2024-04-16T17:01:54.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef03e00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef041029c0 0x7fef041981f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feef400e9c0 tx=0x7feef400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:54.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef01600640 1 -- 172.21.15.18:0/2918018384 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef400ccc0 con 0x7fef041029c0 2024-04-16T17:01:54.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef01600640 1 -- 172.21.15.18:0/2918018384 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef4005500 con 0x7fef041029c0 2024-04-16T17:01:54.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef01600640 1 -- 172.21.15.18:0/2918018384 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4010430 con 0x7fef041029c0 2024-04-16T17:01:54.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.280+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041a0100 con 0x7fef041029c0 2024-04-16T17:01:54.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.281+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a0600 con 0x7fef041029c0 2024-04-16T17:01:54.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.282+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef040fdec0 con 0x7fef041029c0 2024-04-16T17:01:54.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.284+0000 7fef01600640 1 -- 172.21.15.18:0/2918018384 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feef40046c0 con 0x7fef041029c0 2024-04-16T17:01:54.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.285+0000 7fef01600640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feee80a3c20 0x7feee80a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:54.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.285+0000 7fef01600640 1 -- 172.21.15.18:0/2918018384 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7feef40d08e0 con 0x7fef041029c0 2024-04-16T17:01:54.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.285+0000 7feefb400640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feee80a3c20 0x7feee80a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:54.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.286+0000 7feefb400640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feee80a3c20 0x7feee80a60e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef04199700 tx=0x7feeec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:54.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.290+0000 7fef01600640 1 -- 172.21.15.18:0/2918018384 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feef409d020 con 0x7fef041029c0 2024-04-16T17:01:54.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:54 smithi022 ceph-mon[36023]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:54 smithi018 ceph-mon[31776]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:54.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.499+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef04062c00 con 0x7fef041029c0 2024-04-16T17:01:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:54 smithi043 ceph-mon[36901]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:54.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.501+0000 7fef01600640 1 -- 172.21.15.18:0/2918018384 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7feef409c920 con 0x7fef041029c0 2024-04-16T17:01:54.503 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:54.503 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:01:54.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feee80a3c20 msgr2=0x7feee80a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feee80a3c20 0x7feee80a60e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef04199700 tx=0x7feeec005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:54.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef041029c0 msgr2=0x7fef041981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef041029c0 0x7fef041981f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feef400e9c0 tx=0x7feef400ee80 comp rx=0 tx=0).stop 2024-04-16T17:01:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 shutdown_connections 2024-04-16T17:01:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef04107c20 0x7fef0419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feee80a3c20 0x7feee80a60e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef041052f0 0x7fef04198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 --2- 172.21.15.18:0/2918018384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef041029c0 0x7fef041981f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.504+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 >> 172.21.15.18:0/2918018384 conn(0x7fef040fc3d0 msgr2=0x7fef041032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.505+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 shutdown_connections 2024-04-16T17:01:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.505+0000 7fef09a00640 1 -- 172.21.15.18:0/2918018384 wait complete. 2024-04-16T17:01:54.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:01:54.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.755+0000 7f4943e00640 1 -- 172.21.15.18:0/2993783831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944105350 msgr2=0x7f49441057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.755+0000 7f4943e00640 1 --2- 172.21.15.18:0/2993783831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944105350 0x7f49441057d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f492c009900 tx=0x7f492c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.756+0000 7f4943e00640 1 -- 172.21.15.18:0/2993783831 shutdown_connections 2024-04-16T17:01:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.756+0000 7f4943e00640 1 --2- 172.21.15.18:0/2993783831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4944105d10 0x7f494410c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.756+0000 7f4943e00640 1 --2- 172.21.15.18:0/2993783831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944105350 0x7f49441057d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.756+0000 7f4943e00640 1 --2- 172.21.15.18:0/2993783831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4944104160 0x7f4944104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.756+0000 7f4943e00640 1 -- 172.21.15.18:0/2993783831 >> 172.21.15.18:0/2993783831 conn(0x7f49440ff6d0 msgr2=0x7f4944101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.756+0000 7f4943e00640 1 -- 172.21.15.18:0/2993783831 shutdown_connections 2024-04-16T17:01:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.756+0000 7f4943e00640 1 -- 172.21.15.18:0/2993783831 wait complete. 2024-04-16T17:01:54.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.757+0000 7f4943e00640 1 Processor -- start 2024-04-16T17:01:54.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.757+0000 7f4943e00640 1 -- start start 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4943e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944104160 0x7f494419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4943e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4944105350 0x7f494419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4943e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4944105d10 0x7f49441a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4943e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f494410fe40 con 0x7f4944105350 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4943e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f494410fcc0 con 0x7f4944104160 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4943e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f494410ffc0 con 0x7f4944105d10 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4942a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944104160 0x7f494419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4942a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944104160 0x7f494419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34118/0 (socket says 172.21.15.18:34118) 2024-04-16T17:01:54.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4942a00640 1 -- 172.21.15.18:0/1694262808 learned_addr learned my addr 172.21.15.18:0/1694262808 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:54.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4942000640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4944105350 0x7f494419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:54.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.758+0000 7f4943400640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4944105d10 0x7f49441a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:54.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f4942a00640 1 -- 172.21.15.18:0/1694262808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4944105d10 msgr2=0x7f49441a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f4942a00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4944105d10 0x7f49441a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f4942a00640 1 -- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4944105350 msgr2=0x7f494419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f4942a00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4944105350 0x7f494419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f4942a00640 1 -- 172.21.15.18:0/1694262808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49441a2000 con 0x7f4944104160 2024-04-16T17:01:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f4942a00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944104160 0x7f494419a3e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f493400ebc0 tx=0x7f493400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f493b400640 1 -- 172.21.15.18:0/1694262808 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934003b70 con 0x7f4944104160 2024-04-16T17:01:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f493b400640 1 -- 172.21.15.18:0/1694262808 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4934003d10 con 0x7f4944104160 2024-04-16T17:01:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f493b400640 1 -- 172.21.15.18:0/1694262808 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934019980 con 0x7f4944104160 2024-04-16T17:01:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49441a22f0 con 0x7f4944104160 2024-04-16T17:01:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.759+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49441a27c0 con 0x7f4944104160 2024-04-16T17:01:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.761+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f494410b480 con 0x7f4944104160 2024-04-16T17:01:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.763+0000 7f493b400640 1 -- 172.21.15.18:0/1694262808 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4934004b60 con 0x7f4944104160 2024-04-16T17:01:54.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.764+0000 7f493b400640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49280a3c20 0x7f49280a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:54.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.764+0000 7f493b400640 1 -- 172.21.15.18:0/1694262808 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f49340d0fe0 con 0x7f4944104160 2024-04-16T17:01:54.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.765+0000 7f4942000640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49280a3c20 0x7f49280a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:54.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.765+0000 7f4942000640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49280a3c20 0x7f49280a60e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f492c02c240 tx=0x7f492c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:54.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.767+0000 7f493b400640 1 -- 172.21.15.18:0/1694262808 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f493409d780 con 0x7f4944104160 2024-04-16T17:01:54.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.976+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f494402d8d0 con 0x7f4944104160 2024-04-16T17:01:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.978+0000 7f493b400640 1 -- 172.21.15.18:0/1694262808 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7f493409d080 con 0x7f4944104160 2024-04-16T17:01:54.980 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:01:54.980 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49280a3c20 msgr2=0x7f49280a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49280a3c20 0x7f49280a60e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f492c02c240 tx=0x7f492c02df90 comp rx=0 tx=0).stop 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944104160 msgr2=0x7f494419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944104160 0x7f494419a3e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f493400ebc0 tx=0x7f493400c6a0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 shutdown_connections 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4944105d10 0x7f49441a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49280a3c20 0x7f49280a60e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4944105350 0x7f494419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 --2- 172.21.15.18:0/1694262808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4944104160 0x7f494419a3e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 >> 172.21.15.18:0/1694262808 conn(0x7f49440ff6d0 msgr2=0x7f4944108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 shutdown_connections 2024-04-16T17:01:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:54.981+0000 7f4943e00640 1 -- 172.21.15.18:0/1694262808 wait complete. 2024-04-16T17:01:55.003 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-16T17:01:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 -- 172.21.15.18:0/3204552801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1801019a0 msgr2=0x7fc180101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 --2- 172.21.15.18:0/3204552801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1801019a0 0x7fc180101e20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc178009ca0 tx=0x7fc17802c100 comp rx=0 tx=0).stop 2024-04-16T17:01:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 -- 172.21.15.18:0/3204552801 shutdown_connections 2024-04-16T17:01:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 --2- 172.21.15.18:0/3204552801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1801019a0 0x7fc180101e20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 --2- 172.21.15.18:0/3204552801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc180101000 0x7fc180101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 --2- 172.21.15.18:0/3204552801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1801006a0 0x7fc180100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 -- 172.21.15.18:0/3204552801 >> 172.21.15.18:0/3204552801 conn(0x7fc1800fc2c0 msgr2=0x7fc1800fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:55.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 -- 172.21.15.18:0/3204552801 shutdown_connections 2024-04-16T17:01:55.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.234+0000 7fc17f200640 1 -- 172.21.15.18:0/3204552801 wait complete. 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.235+0000 7fc17f200640 1 Processor -- start 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17f200640 1 -- start start 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1801006a0 0x7fc18019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc180101000 0x7fc18019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1801019a0 0x7fc1801a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc18010fcd0 con 0x7fc1801006a0 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc18010fb50 con 0x7fc180101000 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc18010fe50 con 0x7fc1801019a0 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1801006a0 0x7fc18019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1801006a0 0x7fc18019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33342/0 (socket says 172.21.15.18:33342) 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17de00640 1 -- 172.21.15.18:0/288939576 learned_addr learned my addr 172.21.15.18:0/288939576 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:55.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17e800640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1801019a0 0x7fc1801a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:55.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.236+0000 7fc17d400640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc180101000 0x7fc18019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:55.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.237+0000 7fc17e800640 1 -- 172.21.15.18:0/288939576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc180101000 msgr2=0x7fc18019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:55.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.237+0000 7fc17e800640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc180101000 0x7fc18019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:55.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.237+0000 7fc17e800640 1 -- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1801006a0 msgr2=0x7fc18019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:55.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.237+0000 7fc17e800640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1801006a0 0x7fc18019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:55.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.237+0000 7fc17e800640 1 -- 172.21.15.18:0/288939576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1801a1ed0 con 0x7fc1801019a0 2024-04-16T17:01:55.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.237+0000 7fc17de00640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1801006a0 0x7fc18019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:55.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.237+0000 7fc17e800640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1801019a0 0x7fc1801a1860 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc178037040 tx=0x7fc178035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:55.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.237+0000 7fc176a00640 1 -- 172.21.15.18:0/288939576 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178039040 con 0x7fc1801019a0 2024-04-16T17:01:55.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.238+0000 7fc176a00640 1 -- 172.21.15.18:0/288939576 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc178044070 con 0x7fc1801019a0 2024-04-16T17:01:55.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.238+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1801a2160 con 0x7fc1801019a0 2024-04-16T17:01:55.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.238+0000 7fc176a00640 1 -- 172.21.15.18:0/288939576 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178004350 con 0x7fc1801019a0 2024-04-16T17:01:55.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.238+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1801a2620 con 0x7fc1801019a0 2024-04-16T17:01:55.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.239+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18010b590 con 0x7fc1801019a0 2024-04-16T17:01:55.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.241+0000 7fc176a00640 1 -- 172.21.15.18:0/288939576 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc1780044f0 con 0x7fc1801019a0 2024-04-16T17:01:55.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.242+0000 7fc176a00640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc1640a3ba0 0x7fc1640a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:55.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.242+0000 7fc176a00640 1 -- 172.21.15.18:0/288939576 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc1780f2e80 con 0x7fc1801019a0 2024-04-16T17:01:55.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.242+0000 7fc17de00640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc1640a3ba0 0x7fc1640a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:55.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.243+0000 7fc17de00640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc1640a3ba0 0x7fc1640a6060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc170011550 tx=0x7fc170005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:55.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.244+0000 7fc176a00640 1 -- 172.21.15.18:0/288939576 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1780bf620 con 0x7fc1801019a0 2024-04-16T17:01:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2918018384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:55.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1694262808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2918018384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1694262808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:55.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:55.492+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc18002d620 con 0x7fc1801019a0 2024-04-16T17:01:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2918018384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1694262808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:01:56.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.119+0000 7fc176a00640 1 -- 172.21.15.18:0/288939576 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fc1780098b0 con 0x7fc1801019a0 2024-04-16T17:01:56.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.122+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc1640a3ba0 msgr2=0x7fc1640a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:56.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.122+0000 7fc17f200640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc1640a3ba0 0x7fc1640a6060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc170011550 tx=0x7fc170005f90 comp rx=0 tx=0).stop 2024-04-16T17:01:56.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.122+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1801019a0 msgr2=0x7fc1801a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:56.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.122+0000 7fc17f200640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1801019a0 0x7fc1801a1860 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc178037040 tx=0x7fc178035d10 comp rx=0 tx=0).stop 2024-04-16T17:01:56.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.123+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 shutdown_connections 2024-04-16T17:01:56.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.123+0000 7fc17f200640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1801019a0 0x7fc1801a1860 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.123+0000 7fc17f200640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc180101000 0x7fc18019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.123+0000 7fc17f200640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc1640a3ba0 0x7fc1640a6060 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.123+0000 7fc17f200640 1 --2- 172.21.15.18:0/288939576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1801006a0 0x7fc18019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.123+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 >> 172.21.15.18:0/288939576 conn(0x7fc1800fc2c0 msgr2=0x7fc1800fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:56.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.123+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 shutdown_connections 2024-04-16T17:01:56.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.123+0000 7fc17f200640 1 -- 172.21.15.18:0/288939576 wait complete. 2024-04-16T17:01:56.147 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-16T17:01:56.147 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-16T17:01:56.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.366+0000 7ffa0f200640 1 -- 172.21.15.18:0/828576245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08104270 msgr2=0x7ffa08104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:56.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.366+0000 7ffa0f200640 1 --2- 172.21.15.18:0/828576245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08104270 0x7ffa08104690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009900 tx=0x7ff9f802bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:56.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.367+0000 7ffa0f200640 1 -- 172.21.15.18:0/828576245 shutdown_connections 2024-04-16T17:01:56.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.367+0000 7ffa0f200640 1 --2- 172.21.15.18:0/828576245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa08105e20 0x7ffa0810c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.367+0000 7ffa0f200640 1 --2- 172.21.15.18:0/828576245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa08105460 0x7ffa081058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.367+0000 7ffa0f200640 1 --2- 172.21.15.18:0/828576245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08104270 0x7ffa08104690 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.367+0000 7ffa0f200640 1 -- 172.21.15.18:0/828576245 >> 172.21.15.18:0/828576245 conn(0x7ffa080ff820 msgr2=0x7ffa08101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.367+0000 7ffa0f200640 1 -- 172.21.15.18:0/828576245 shutdown_connections 2024-04-16T17:01:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.367+0000 7ffa0f200640 1 -- 172.21.15.18:0/828576245 wait complete. 2024-04-16T17:01:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.368+0000 7ffa0f200640 1 Processor -- start 2024-04-16T17:01:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.368+0000 7ffa0f200640 1 -- start start 2024-04-16T17:01:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.368+0000 7ffa0f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa08104270 0x7ffa0819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.368+0000 7ffa0f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa08105460 0x7ffa0819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.368+0000 7ffa0f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08105e20 0x7ffa081a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.368+0000 7ffa0f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810ff00 con 0x7ffa08105e20 2024-04-16T17:01:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.368+0000 7ffa0f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810fd80 con 0x7ffa08105460 2024-04-16T17:01:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.368+0000 7ffa0f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08110080 con 0x7ffa08104270 2024-04-16T17:01:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa08104270 0x7ffa0819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa08104270 0x7ffa0819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50144/0 (socket says 172.21.15.18:50144) 2024-04-16T17:01:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07e00640 1 -- 172.21.15.18:0/1798292646 learned_addr learned my addr 172.21.15.18:0/1798292646 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07400640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa08105460 0x7ffa0819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa0d200640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08105e20 0x7ffa081a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07e00640 1 -- 172.21.15.18:0/1798292646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa08105460 msgr2=0x7ffa0819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07e00640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa08105460 0x7ffa0819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07e00640 1 -- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08105e20 msgr2=0x7ffa081a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07e00640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08105e20 0x7ffa081a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa07e00640 1 -- 172.21.15.18:0/1798292646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa081a21a0 con 0x7ffa08104270 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.369+0000 7ffa0d200640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08105e20 0x7ffa081a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.370+0000 7ffa07e00640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa08104270 0x7ffa0819a5d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8000c00 tx=0x7ff9f802c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.370+0000 7ffa04c00640 1 -- 172.21.15.18:0/1798292646 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8041070 con 0x7ffa08104270 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.370+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa081a2430 con 0x7ffa08104270 2024-04-16T17:01:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.370+0000 7ffa04c00640 1 -- 172.21.15.18:0/1798292646 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f8004450 con 0x7ffa08104270 2024-04-16T17:01:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.370+0000 7ffa04c00640 1 -- 172.21.15.18:0/1798292646 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8034440 con 0x7ffa08104270 2024-04-16T17:01:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.370+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa081a2970 con 0x7ffa08104270 2024-04-16T17:01:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.371+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0810b590 con 0x7ffa08104270 2024-04-16T17:01:56.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.373+0000 7ffa04c00640 1 -- 172.21.15.18:0/1798292646 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff9f8044070 con 0x7ffa08104270 2024-04-16T17:01:56.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.374+0000 7ffa04c00640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff9ec0a3bb0 0x7ff9ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:56.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.374+0000 7ffa04c00640 1 -- 172.21.15.18:0/1798292646 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff9f80f2230 con 0x7ffa08104270 2024-04-16T17:01:56.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.375+0000 7ffa07400640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff9ec0a3bb0 0x7ff9ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:56.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.375+0000 7ffa07400640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff9ec0a3bb0 0x7ff9ec0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa0819ba90 tx=0x7ff9f4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:56.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.379+0000 7ffa04c00640 1 -- 172.21.15.18:0/1798292646 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9f80bea00 con 0x7ffa08104270 2024-04-16T17:01:56.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:56 smithi022 ceph-mon[36023]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:56.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:56 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/288939576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:01:56.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:56 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:01:56.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:56 smithi018 ceph-mon[31776]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:56.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:56 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/288939576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:01:56.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:56 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:01:56.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:56 smithi043 ceph-mon[36901]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:01:56.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:56 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/288939576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:01:56.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:56 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:01:56.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:56.638+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ffa0802d840 con 0x7ffa08104270 2024-04-16T17:01:57.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.131+0000 7ffa04c00640 1 -- 172.21.15.18:0/1798292646 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7ff9f80be300 con 0x7ffa08104270 2024-04-16T17:01:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff9ec0a3bb0 msgr2=0x7ff9ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff9ec0a3bb0 0x7ff9ec0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa0819ba90 tx=0x7ff9f4009500 comp rx=0 tx=0).stop 2024-04-16T17:01:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa08104270 msgr2=0x7ffa0819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa08104270 0x7ffa0819a5d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8000c00 tx=0x7ff9f802c640 comp rx=0 tx=0).stop 2024-04-16T17:01:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 shutdown_connections 2024-04-16T17:01:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff9ec0a3bb0 0x7ff9ec0a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa08105e20 0x7ffa081a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa08105460 0x7ffa0819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 --2- 172.21.15.18:0/1798292646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa08104270 0x7ffa0819a5d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 >> 172.21.15.18:0/1798292646 conn(0x7ffa080ff820 msgr2=0x7ffa08108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:57.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 shutdown_connections 2024-04-16T17:01:57.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.136+0000 7ffa0f200640 1 -- 172.21.15.18:0/1798292646 wait complete. 2024-04-16T17:01:57.169 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-16T17:01:57.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 -- 172.21.15.18:0/1549095499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed8102fc0 msgr2=0x7f3ed8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:57.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/1549095499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed8102fc0 0x7f3ed8103440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009860 tx=0x7f3ec002bb70 comp rx=0 tx=0).stop 2024-04-16T17:01:57.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 -- 172.21.15.18:0/1549095499 shutdown_connections 2024-04-16T17:01:57.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/1549095499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed8103980 0x7f3ed810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/1549095499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed8102fc0 0x7f3ed8103440 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/1549095499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed8101e70 0x7f3ed8102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 -- 172.21.15.18:0/1549095499 >> 172.21.15.18:0/1549095499 conn(0x7f3ed80fd3e0 msgr2=0x7f3ed80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:57.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 -- 172.21.15.18:0/1549095499 shutdown_connections 2024-04-16T17:01:57.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.373+0000 7f3ed7e00640 1 -- 172.21.15.18:0/1549095499 wait complete. 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.374+0000 7f3ed7e00640 1 Processor -- start 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7e00640 1 -- start start 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed8101e70 0x7f3ed819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed8102fc0 0x7f3ed819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed8103980 0x7f3ed81a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810daf0 con 0x7f3ed8103980 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810d970 con 0x7f3ed8101e70 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810dc70 con 0x7f3ed8102fc0 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed8103980 0x7f3ed81a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:57.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed8101e70 0x7f3ed819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:57.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed8103980 0x7f3ed81a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54182/0 (socket says 172.21.15.18:54182) 2024-04-16T17:01:57.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ed7400640 1 -- 172.21.15.18:0/261048392 learned_addr learned my addr 172.21.15.18:0/261048392 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:57.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.375+0000 7f3ece000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed8102fc0 0x7f3ed819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ed6a00640 1 -- 172.21.15.18:0/261048392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed8102fc0 msgr2=0x7f3ed819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ed6a00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed8102fc0 0x7f3ed819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ed6a00640 1 -- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed8103980 msgr2=0x7f3ed81a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ed6a00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed8103980 0x7f3ed81a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ed6a00640 1 -- 172.21.15.18:0/261048392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed81a2030 con 0x7f3ed8101e70 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ed6a00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed8101e70 0x7f3ed819a410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800e790 tx=0x7f3ec800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ecfe00640 1 -- 172.21.15.18:0/261048392 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800eea0 con 0x7f3ed8101e70 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed81a2320 con 0x7f3ed8101e70 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.376+0000 7f3ecfe00640 1 -- 172.21.15.18:0/261048392 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec800ccc0 con 0x7f3ed8101e70 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.377+0000 7f3ecfe00640 1 -- 172.21.15.18:0/261048392 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8018760 con 0x7f3ed8101e70 2024-04-16T17:01:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.377+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed81a2840 con 0x7f3ed8101e70 2024-04-16T17:01:57.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.378+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed81a2aa0 con 0x7f3ed8101e70 2024-04-16T17:01:57.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.380+0000 7f3ecfe00640 1 -- 172.21.15.18:0/261048392 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ec8010430 con 0x7f3ed8101e70 2024-04-16T17:01:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.381+0000 7f3ecfe00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3ebc0a3ba0 0x7f3ebc0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.381+0000 7f3ecfe00640 1 -- 172.21.15.18:0/261048392 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3ec80d07e0 con 0x7f3ed8101e70 2024-04-16T17:01:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.382+0000 7f3ece000640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3ebc0a3ba0 0x7f3ebc0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:57.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.382+0000 7f3ece000640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3ebc0a3ba0 0x7f3ebc0a6060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009830 tx=0x7f3ec0034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:57.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.386+0000 7f3ecfe00640 1 -- 172.21.15.18:0/261048392 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ec809c8d0 con 0x7f3ed8101e70 2024-04-16T17:01:57.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:57 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T17:01:57.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:57 smithi022 ceph-mon[36023]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:57.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:57 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1798292646' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:01:57.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:57 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:01:57.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:57 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T17:01:57.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:57 smithi018 ceph-mon[31776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:57.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:57 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1798292646' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:01:57.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:57 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:01:57.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:57 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T17:01:57.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:57 smithi043 ceph-mon[36901]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:57.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:57 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1798292646' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:01:57.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:57 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:01:57.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:57.635+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3ed81a2df0 con 0x7f3ed8101e70 2024-04-16T17:01:58.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.147+0000 7f3ecfe00640 1 -- 172.21.15.18:0/261048392 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f3ec80a0bd0 con 0x7f3ed8101e70 2024-04-16T17:01:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.150+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3ebc0a3ba0 msgr2=0x7f3ebc0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.150+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3ebc0a3ba0 0x7f3ebc0a6060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009830 tx=0x7f3ec0034ad0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.150+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed8101e70 msgr2=0x7f3ed819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.150+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed8101e70 0x7f3ed819a410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800e790 tx=0x7f3ec800ec50 comp rx=0 tx=0).stop 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.150+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 shutdown_connections 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.150+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3ebc0a3ba0 0x7f3ebc0a6060 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.150+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed8103980 0x7f3ed81a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.150+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed8102fc0 0x7f3ed819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.151+0000 7f3ed7e00640 1 --2- 172.21.15.18:0/261048392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ed8101e70 0x7f3ed819a410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.151+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 >> 172.21.15.18:0/261048392 conn(0x7f3ed80fd3e0 msgr2=0x7f3ed8074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.151+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 shutdown_connections 2024-04-16T17:01:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.151+0000 7f3ed7e00640 1 -- 172.21.15.18:0/261048392 wait complete. 2024-04-16T17:01:58.184 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-16T17:01:58.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 -- 172.21.15.18:0/3422379752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638105450 msgr2=0x7f66381058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:58.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 --2- 172.21.15.18:0/3422379752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638105450 0x7f66381058d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6624009900 tx=0x7f662402bc40 comp rx=0 tx=0).stop 2024-04-16T17:01:58.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 -- 172.21.15.18:0/3422379752 shutdown_connections 2024-04-16T17:01:58.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 --2- 172.21.15.18:0/3422379752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6638105e10 0x7f663810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 --2- 172.21.15.18:0/3422379752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638105450 0x7f66381058d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 --2- 172.21.15.18:0/3422379752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6638104260 0x7f6638104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 -- 172.21.15.18:0/3422379752 >> 172.21.15.18:0/3422379752 conn(0x7f66380ff7f0 msgr2=0x7f6638101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:58.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 -- 172.21.15.18:0/3422379752 shutdown_connections 2024-04-16T17:01:58.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.387+0000 7f663dc00640 1 -- 172.21.15.18:0/3422379752 wait complete. 2024-04-16T17:01:58.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.389+0000 7f663dc00640 1 Processor -- start 2024-04-16T17:01:58.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.389+0000 7f663dc00640 1 -- start start 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.389+0000 7f663dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638104260 0x7f663819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.389+0000 7f663dc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6638105450 0x7f663819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.389+0000 7f663dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6638105e10 0x7f66381a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f663dc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f663810fef0 con 0x7f6638105450 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f663dc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f663810fd70 con 0x7f6638104260 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f663dc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6638110070 con 0x7f6638105e10 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638104260 0x7f663819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638104260 0x7f663819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54140/0 (socket says 172.21.15.18:54140) 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636a00640 1 -- 172.21.15.18:0/1444198361 learned_addr learned my addr 172.21.15.18:0/1444198361 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:58.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636000640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6638105450 0x7f663819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:58.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6637400640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6638105e10 0x7f66381a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:58.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636a00640 1 -- 172.21.15.18:0/1444198361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6638105e10 msgr2=0x7f66381a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:58.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636a00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6638105e10 0x7f66381a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636a00640 1 -- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6638105450 msgr2=0x7f663819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:58.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636a00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6638105450 0x7f663819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:58.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.390+0000 7f6636a00640 1 -- 172.21.15.18:0/1444198361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66381a2190 con 0x7f6638104260 2024-04-16T17:01:58.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.391+0000 7f6636a00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638104260 0x7f663819a570 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f662800e9c0 tx=0x7f662800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:58.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.391+0000 7f662f400640 1 -- 172.21.15.18:0/1444198361 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662800cd80 con 0x7f6638104260 2024-04-16T17:01:58.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.391+0000 7f662f400640 1 -- 172.21.15.18:0/1444198361 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66280035f0 con 0x7f6638104260 2024-04-16T17:01:58.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.391+0000 7f662f400640 1 -- 172.21.15.18:0/1444198361 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628010690 con 0x7f6638104260 2024-04-16T17:01:58.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.391+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66381a2480 con 0x7f6638104260 2024-04-16T17:01:58.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.391+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66381a29a0 con 0x7f6638104260 2024-04-16T17:01:58.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.392+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663810b580 con 0x7f6638104260 2024-04-16T17:01:58.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.394+0000 7f662f400640 1 -- 172.21.15.18:0/1444198361 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f66280037a0 con 0x7f6638104260 2024-04-16T17:01:58.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.395+0000 7f662f400640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f661c0a3c30 0x7f661c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:58.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.396+0000 7f662f400640 1 -- 172.21.15.18:0/1444198361 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f66280d0a30 con 0x7f6638104260 2024-04-16T17:01:58.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.396+0000 7f6636000640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f661c0a3c30 0x7f661c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:58.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.397+0000 7f6636000640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f661c0a3c30 0x7f661c0a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f662402c7c0 tx=0x7f6624005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:58.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.400+0000 7f662f400640 1 -- 172.21.15.18:0/1444198361 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f662809d170 con 0x7f6638104260 2024-04-16T17:01:58.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-16T17:01:58.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T17:01:58.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T17:01:58.404 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T17:01:58.404 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:58.404 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-16T17:01:58.404 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-16T17:01:58.404 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/261048392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:01:58.404 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:58 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/261048392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:01:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:58 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:01:58.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-16T17:01:58.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T17:01:58.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T17:01:58.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T17:01:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:01:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-16T17:01:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-16T17:01:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/261048392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:01:58.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:58 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:01:58.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:58.655+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f663802d620 con 0x7f6638104260 2024-04-16T17:01:59.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.160+0000 7f662f400640 1 -- 172.21.15.18:0/1444198361 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f662809ca70 con 0x7f6638104260 2024-04-16T17:01:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f661c0a3c30 msgr2=0x7f661c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f661c0a3c30 0x7f661c0a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f662402c7c0 tx=0x7f6624005fd0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638104260 msgr2=0x7f663819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638104260 0x7f663819a570 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f662800e9c0 tx=0x7f662800ee80 comp rx=0 tx=0).stop 2024-04-16T17:01:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 shutdown_connections 2024-04-16T17:01:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6638105e10 0x7f66381a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f661c0a3c30 0x7f661c0a60f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6638105450 0x7f663819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 --2- 172.21.15.18:0/1444198361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6638104260 0x7f663819a570 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.163+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 >> 172.21.15.18:0/1444198361 conn(0x7f66380ff7f0 msgr2=0x7f6638108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.164+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 shutdown_connections 2024-04-16T17:01:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.164+0000 7f663dc00640 1 -- 172.21.15.18:0/1444198361 wait complete. 2024-04-16T17:01:59.188 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-16T17:01:59.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-16T17:01:59.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.413+0000 7fa2aa400640 1 -- 172.21.15.18:0/903755017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105e10 msgr2=0x7fa2a410c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.413+0000 7fa2aa400640 1 --2- 172.21.15.18:0/903755017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105e10 0x7fa2a410c6a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009900 tx=0x7fa29c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:59.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.413+0000 7fa2aa400640 1 -- 172.21.15.18:0/903755017 shutdown_connections 2024-04-16T17:01:59.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.413+0000 7fa2aa400640 1 --2- 172.21.15.18:0/903755017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105e10 0x7fa2a410c6a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.413+0000 7fa2aa400640 1 --2- 172.21.15.18:0/903755017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4105450 0x7fa2a41058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.413+0000 7fa2aa400640 1 --2- 172.21.15.18:0/903755017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4104260 0x7fa2a4104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.413+0000 7fa2aa400640 1 -- 172.21.15.18:0/903755017 >> 172.21.15.18:0/903755017 conn(0x7fa2a40ff7f0 msgr2=0x7fa2a4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:59.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.413+0000 7fa2aa400640 1 -- 172.21.15.18:0/903755017 shutdown_connections 2024-04-16T17:01:59.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.414+0000 7fa2aa400640 1 -- 172.21.15.18:0/903755017 wait complete. 2024-04-16T17:01:59.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.414+0000 7fa2aa400640 1 Processor -- start 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2aa400640 1 -- start start 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2aa400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4104260 0x7fa2a419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2aa400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105450 0x7fa2a419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2aa400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4105e10 0x7fa2a41a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2aa400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410fec0 con 0x7fa2a4104260 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2aa400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410fd40 con 0x7fa2a4105450 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2aa400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4110040 con 0x7fa2a4105e10 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2a3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4104260 0x7fa2a419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2a3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4104260 0x7fa2a419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54228/0 (socket says 172.21.15.18:54228) 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105450 0x7fa2a419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2a3400640 1 -- 172.21.15.18:0/1953292917 learned_addr learned my addr 172.21.15.18:0/1953292917 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.415+0000 7fa2a3e00640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4105e10 0x7fa2a41a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:59.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa2a2a00640 1 -- 172.21.15.18:0/1953292917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4105e10 msgr2=0x7fa2a41a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa2a2a00640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4105e10 0x7fa2a41a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa2a2a00640 1 -- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4104260 msgr2=0x7fa2a419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa2a2a00640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4104260 0x7fa2a419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa2a2a00640 1 -- 172.21.15.18:0/1953292917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a41a2120 con 0x7fa2a4105450 2024-04-16T17:01:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa2a2a00640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105450 0x7fa2a419aa40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00e9f0 tx=0x7fa28c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa29be00640 1 -- 172.21.15.18:0/1953292917 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c00ccc0 con 0x7fa2a4105450 2024-04-16T17:01:59.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa29be00640 1 -- 172.21.15.18:0/1953292917 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28c010910 con 0x7fa2a4105450 2024-04-16T17:01:59.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa29be00640 1 -- 172.21.15.18:0/1953292917 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c018960 con 0x7fa2a4105450 2024-04-16T17:01:59.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.416+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a41a2410 con 0x7fa2a4105450 2024-04-16T17:01:59.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.417+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a41a2910 con 0x7fa2a4105450 2024-04-16T17:01:59.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.418+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a410b520 con 0x7fa2a4105450 2024-04-16T17:01:59.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.420+0000 7fa29be00640 1 -- 172.21.15.18:0/1953292917 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa28c010ac0 con 0x7fa2a4105450 2024-04-16T17:01:59.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.421+0000 7fa29be00640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa2880a3c80 0x7fa2880a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:59.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.421+0000 7fa29be00640 1 -- 172.21.15.18:0/1953292917 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa28c0d09b0 con 0x7fa2a4105450 2024-04-16T17:01:59.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.421+0000 7fa2a3400640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa2880a3c80 0x7fa2880a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:59.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.422+0000 7fa2a3400640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa2880a3c80 0x7fa2880a6140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa294011440 tx=0x7fa294009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:59.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] up:active 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: Cluster is now healthy 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1444198361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:01:59.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:01:59.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.423+0000 7fa29be00640 1 -- 172.21.15.18:0/1953292917 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa28c09caa0 con 0x7fa2a4105450 2024-04-16T17:01:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T17:01:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] up:active 2024-04-16T17:01:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T17:01:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T17:01:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: Cluster is now healthy 2024-04-16T17:01:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-16T17:01:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-16T17:01:59.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T17:01:59.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1444198361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:01:59.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] up:active 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: Cluster is now healthy 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1444198361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:01:59.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:01:59 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:01:59.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.656+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa2a4062440 con 0x7fa2880a3c80 2024-04-16T17:01:59.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.684+0000 7fa29be00640 1 -- 172.21.15.18:0/1953292917 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa2a4062440 con 0x7fa2880a3c80 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa2880a3c80 msgr2=0x7fa2880a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa2880a3c80 0x7fa2880a6140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa294011440 tx=0x7fa294009500 comp rx=0 tx=0).stop 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105450 msgr2=0x7fa2a419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105450 0x7fa2a419aa40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00e9f0 tx=0x7fa28c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 shutdown_connections 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4105e10 0x7fa2a41a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a4105450 0x7fa2a419aa40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa2880a3c80 0x7fa2880a6140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 --2- 172.21.15.18:0/1953292917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4104260 0x7fa2a419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 >> 172.21.15.18:0/1953292917 conn(0x7fa2a40ff7f0 msgr2=0x7fa2a41086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 shutdown_connections 2024-04-16T17:01:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.687+0000 7fa2aa400640 1 -- 172.21.15.18:0/1953292917 wait complete. 2024-04-16T17:01:59.711 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-16T17:01:59.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 -- 172.21.15.18:0/1485260262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c074a60 msgr2=0x7ff66c074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 --2- 172.21.15.18:0/1485260262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c074a60 0x7ff66c074ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff658009900 tx=0x7ff65802bc30 comp rx=0 tx=0).stop 2024-04-16T17:01:59.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 -- 172.21.15.18:0/1485260262 shutdown_connections 2024-04-16T17:01:59.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 --2- 172.21.15.18:0/1485260262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff66c109d30 0x7ff66c10c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 --2- 172.21.15.18:0/1485260262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c074a60 0x7ff66c074ee0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 --2- 172.21.15.18:0/1485260262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff66c076040 0x7ff66c074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 -- 172.21.15.18:0/1485260262 >> 172.21.15.18:0/1485260262 conn(0x7ff66c0fd3e0 msgr2=0x7ff66c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:59.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 -- 172.21.15.18:0/1485260262 shutdown_connections 2024-04-16T17:01:59.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.921+0000 7ff672000640 1 -- 172.21.15.18:0/1485260262 wait complete. 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.922+0000 7ff672000640 1 Processor -- start 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.922+0000 7ff672000640 1 -- start start 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.922+0000 7ff672000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff66c074a60 0x7ff66c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff672000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff66c076040 0x7ff66c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff672000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c109d30 0x7ff66c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff672000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10fd40 con 0x7ff66c076040 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff672000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10fbc0 con 0x7ff66c109d30 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff672000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10fec0 con 0x7ff66c074a60 2024-04-16T17:01:59.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff66be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff66c076040 0x7ff66c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff66be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff66c076040 0x7ff66c19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54278/0 (socket says 172.21.15.18:54278) 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff66be00640 1 -- 172.21.15.18:0/1074452480 learned_addr learned my addr 172.21.15.18:0/1074452480 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff671600640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c109d30 0x7ff66c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff670c00640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff66c074a60 0x7ff66c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff671600640 1 -- 172.21.15.18:0/1074452480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff66c074a60 msgr2=0x7ff66c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff671600640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff66c074a60 0x7ff66c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff671600640 1 -- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff66c076040 msgr2=0x7ff66c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff671600640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff66c076040 0x7ff66c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff671600640 1 -- 172.21.15.18:0/1074452480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1a1f90 con 0x7ff66c109d30 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.923+0000 7ff66be00640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff66c076040 0x7ff66c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.924+0000 7ff671600640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c109d30 0x7ff66c1a1920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff66400d680 tx=0x7ff66400db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.924+0000 7ff669600640 1 -- 172.21.15.18:0/1074452480 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664017070 con 0x7ff66c109d30 2024-04-16T17:01:59.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.924+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c1a2280 con 0x7ff66c109d30 2024-04-16T17:01:59.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.924+0000 7ff669600640 1 -- 172.21.15.18:0/1074452480 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff664009820 con 0x7ff66c109d30 2024-04-16T17:01:59.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.924+0000 7ff669600640 1 -- 172.21.15.18:0/1074452480 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664004e00 con 0x7ff66c109d30 2024-04-16T17:01:59.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.924+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1a27a0 con 0x7ff66c109d30 2024-04-16T17:01:59.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.925+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c108a60 con 0x7ff66c109d30 2024-04-16T17:01:59.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.927+0000 7ff669600640 1 -- 172.21.15.18:0/1074452480 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff6640099d0 con 0x7ff66c109d30 2024-04-16T17:01:59.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.928+0000 7ff669600640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff6500a3bb0 0x7ff6500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:59.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.929+0000 7ff669600640 1 -- 172.21.15.18:0/1074452480 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff6640cfe90 con 0x7ff66c109d30 2024-04-16T17:01:59.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.929+0000 7ff670c00640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff6500a3bb0 0x7ff6500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:59.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.929+0000 7ff670c00640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff6500a3bb0 0x7ff6500a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff65c005fd0 tx=0x7ff65c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:59.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:01:59.933+0000 7ff669600640 1 -- 172.21.15.18:0/1074452480 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff66409bf80 con 0x7ff66c109d30 2024-04-16T17:01:59.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:01:59.667+0000 7f3816000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:01:59.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:01:59.667+0000 7f3816000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:01:59.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:01:59.667+0000 7f3816000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:01:59.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:01:59.667+0000 7f3816000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:01:59.947 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:01:59 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:01:59.667+0000 7f3816000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:02:00.161 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: 2024-04-16T17:01:59.859+0000 7f198be63700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-16T17:02:00.161 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:01:59 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: 2024-04-16T17:01:59.859+0000 7f198be63700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-16T17:02:00.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.173+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff66c0624d0 con 0x7ff6500a3bb0 2024-04-16T17:02:00.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.215+0000 7ff669600640 1 -- 172.21.15.18:0/1074452480 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff66c0624d0 con 0x7ff6500a3bb0 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.217+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff6500a3bb0 msgr2=0x7ff6500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.217+0000 7ff672000640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff6500a3bb0 0x7ff6500a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff65c005fd0 tx=0x7ff65c009500 comp rx=0 tx=0).stop 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c109d30 msgr2=0x7ff66c1a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c109d30 0x7ff66c1a1920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff66400d680 tx=0x7ff66400db40 comp rx=0 tx=0).stop 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 shutdown_connections 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff66c109d30 0x7ff66c1a1920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff6500a3bb0 0x7ff6500a6070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff66c076040 0x7ff66c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 --2- 172.21.15.18:0/1074452480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff66c074a60 0x7ff66c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 >> 172.21.15.18:0/1074452480 conn(0x7ff66c0fd3e0 msgr2=0x7ff66c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:00.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 shutdown_connections 2024-04-16T17:02:00.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.218+0000 7ff672000640 1 -- 172.21.15.18:0/1074452480 wait complete. 2024-04-16T17:02:00.239 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-16T17:02:00.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:00 smithi022 ceph-mon[36023]: pgmap v336: 129 pgs: 129 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-16T17:02:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:00 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T17:02:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:00 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] up:active 2024-04-16T17:02:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:00 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-16T17:02:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:00 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:02:00.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:00 smithi018 ceph-mon[31776]: pgmap v336: 129 pgs: 129 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-16T17:02:00.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:00 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T17:02:00.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:00 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] up:active 2024-04-16T17:02:00.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:00 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-16T17:02:00.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:00 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:02:00.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.470+0000 7f2969a00640 1 -- 172.21.15.18:0/249879333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2964105c50 msgr2=0x7f296410c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.470+0000 7f2969a00640 1 --2- 172.21.15.18:0/249879333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2964105c50 0x7f296410c4e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f295c009be0 tx=0x7f295c02bef0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.471+0000 7f2969a00640 1 -- 172.21.15.18:0/249879333 shutdown_connections 2024-04-16T17:02:00.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.471+0000 7f2969a00640 1 --2- 172.21.15.18:0/249879333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2964105c50 0x7f296410c4e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.471+0000 7f2969a00640 1 --2- 172.21.15.18:0/249879333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2964105290 0x7f2964105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.471+0000 7f2969a00640 1 --2- 172.21.15.18:0/249879333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29641040a0 0x7f29641044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.471+0000 7f2969a00640 1 -- 172.21.15.18:0/249879333 >> 172.21.15.18:0/249879333 conn(0x7f29640ff610 msgr2=0x7f2964101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:00.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.471+0000 7f2969a00640 1 -- 172.21.15.18:0/249879333 shutdown_connections 2024-04-16T17:02:00.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.471+0000 7f2969a00640 1 -- 172.21.15.18:0/249879333 wait complete. 2024-04-16T17:02:00.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.472+0000 7f2969a00640 1 Processor -- start 2024-04-16T17:02:00.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.472+0000 7f2969a00640 1 -- start start 2024-04-16T17:02:00.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2969a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29641040a0 0x7f296419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:00.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2969a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2964105290 0x7f296419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:00.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2969a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2964105c50 0x7f29641a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:00.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2969a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f296410fd30 con 0x7f2964105290 2024-04-16T17:02:00.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2969a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f296410fbb0 con 0x7f29641040a0 2024-04-16T17:02:00.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2969a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f296410feb0 con 0x7f2964105c50 2024-04-16T17:02:00.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2963400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2964105290 0x7f296419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:00.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2963e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29641040a0 0x7f296419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:00.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2963400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2964105290 0x7f296419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54294/0 (socket says 172.21.15.18:54294) 2024-04-16T17:02:00.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2963400640 1 -- 172.21.15.18:0/2455257543 learned_addr learned my addr 172.21.15.18:0/2455257543 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:00.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.473+0000 7f2969000640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2964105c50 0x7f29641a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:00.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2963400640 1 -- 172.21.15.18:0/2455257543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2964105c50 msgr2=0x7f29641a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2963400640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2964105c50 0x7f29641a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2963400640 1 -- 172.21.15.18:0/2455257543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29641040a0 msgr2=0x7f296419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2963400640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29641040a0 0x7f296419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2963400640 1 -- 172.21.15.18:0/2455257543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29641a1f70 con 0x7f2964105290 2024-04-16T17:02:00.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2963400640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2964105290 0x7f296419a890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e9f0 tx=0x7f294c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:00.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2960c00640 1 -- 172.21.15.18:0/2455257543 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c00cd80 con 0x7f2964105290 2024-04-16T17:02:00.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29641a2260 con 0x7f2964105290 2024-04-16T17:02:00.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2960c00640 1 -- 172.21.15.18:0/2455257543 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c005500 con 0x7f2964105290 2024-04-16T17:02:00.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2960c00640 1 -- 172.21.15.18:0/2455257543 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c018630 con 0x7f2964105290 2024-04-16T17:02:00.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.474+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29641a27b0 con 0x7f2964105290 2024-04-16T17:02:00.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.476+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296410b360 con 0x7f2964105290 2024-04-16T17:02:00.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.478+0000 7f2960c00640 1 -- 172.21.15.18:0/2455257543 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f294c0046c0 con 0x7f2964105290 2024-04-16T17:02:00.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.481+0000 7f2960c00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f29480a3bb0 0x7f29480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:00.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.481+0000 7f2960c00640 1 -- 172.21.15.18:0/2455257543 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f294c0d0af0 con 0x7f2964105290 2024-04-16T17:02:00.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.481+0000 7f2963e00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f29480a3bb0 0x7f29480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:00.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.482+0000 7f2963e00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f29480a3bb0 0x7f29480a6070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2954009530 tx=0x7f2954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:00.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.486+0000 7f2960c00640 1 -- 172.21.15.18:0/2455257543 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f294c09d230 con 0x7f2964105290 2024-04-16T17:02:00.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:00 smithi043 ceph-mon[36901]: pgmap v336: 129 pgs: 129 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-16T17:02:00.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:00 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T17:02:00.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:00 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] up:active 2024-04-16T17:02:00.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:00 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-16T17:02:00.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:00 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:02:00.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.725+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2964062440 con 0x7f29480a3bb0 2024-04-16T17:02:00.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.735+0000 7f2960c00640 1 -- 172.21.15.18:0/2455257543 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2964062440 con 0x7f29480a3bb0 2024-04-16T17:02:00.737 INFO:teuthology.orchestra.run.smithi018.stdout:/volumes/qa/sv_0/535f4225-a552-4802-9874-ff1e70c95632 2024-04-16T17:02:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f29480a3bb0 msgr2=0x7f29480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f29480a3bb0 0x7f29480a6070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2954009530 tx=0x7f2954009260 comp rx=0 tx=0).stop 2024-04-16T17:02:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2964105290 msgr2=0x7f296419a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2964105290 0x7f296419a890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e9f0 tx=0x7f294c00edf0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 shutdown_connections 2024-04-16T17:02:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2964105c50 0x7f29641a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f29480a3bb0 0x7f29480a6070 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2964105290 0x7f296419a890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 --2- 172.21.15.18:0/2455257543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29641040a0 0x7f296419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.738+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 >> 172.21.15.18:0/2455257543 conn(0x7f29640ff610 msgr2=0x7f2964108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.739+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 shutdown_connections 2024-04-16T17:02:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.739+0000 7f2969a00640 1 -- 172.21.15.18:0/2455257543 wait complete. 2024-04-16T17:02:00.762 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-16T17:02:00.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 -- 172.21.15.18:0/2858323695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105450 msgr2=0x7f31c41058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 --2- 172.21.15.18:0/2858323695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105450 0x7f31c41058d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009900 tx=0x7f31b002bc30 comp rx=0 tx=0).stop 2024-04-16T17:02:00.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 -- 172.21.15.18:0/2858323695 shutdown_connections 2024-04-16T17:02:00.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 --2- 172.21.15.18:0/2858323695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4105e10 0x7f31c410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 --2- 172.21.15.18:0/2858323695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105450 0x7f31c41058d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 --2- 172.21.15.18:0/2858323695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c4104260 0x7f31c4104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 -- 172.21.15.18:0/2858323695 >> 172.21.15.18:0/2858323695 conn(0x7f31c40ff7f0 msgr2=0x7f31c4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:00.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 -- 172.21.15.18:0/2858323695 shutdown_connections 2024-04-16T17:02:00.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.978+0000 7f31cb200640 1 -- 172.21.15.18:0/2858323695 wait complete. 2024-04-16T17:02:00.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.979+0000 7f31cb200640 1 Processor -- start 2024-04-16T17:02:00.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.979+0000 7f31cb200640 1 -- start start 2024-04-16T17:02:00.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.979+0000 7f31cb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c4104260 0x7f31c419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:00.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.979+0000 7f31cb200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4105450 0x7f31c419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:00.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.979+0000 7f31cb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105e10 0x7f31c41a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:00.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.979+0000 7f31cb200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f31c410fef0 con 0x7f31c4105450 2024-04-16T17:02:00.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.979+0000 7f31cb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f31c410fd70 con 0x7f31c4105e10 2024-04-16T17:02:00.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.979+0000 7f31cb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31c4110070 con 0x7f31c4104260 2024-04-16T17:02:00.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105e10 0x7f31c41a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:00.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c4104260 0x7f31c419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:00.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105e10 0x7f31c41a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54222/0 (socket says 172.21.15.18:54222) 2024-04-16T17:02:00.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c9200640 1 -- 172.21.15.18:0/352387160 learned_addr learned my addr 172.21.15.18:0/352387160 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:00.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c3400640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4105450 0x7f31c419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c9200640 1 -- 172.21.15.18:0/352387160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c4104260 msgr2=0x7f31c419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c9200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c4104260 0x7f31c419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c9200640 1 -- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4105450 msgr2=0x7f31c419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c9200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4105450 0x7f31c419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c9200640 1 -- 172.21.15.18:0/352387160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c41a2180 con 0x7f31c4105e10 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.980+0000 7f31c3e00640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c4104260 0x7f31c419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.981+0000 7f31c9200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105e10 0x7f31c41a1b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00eb10 tx=0x7f31bc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.981+0000 7f31c0c00640 1 -- 172.21.15.18:0/352387160 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc004b20 con 0x7f31c4105e10 2024-04-16T17:02:00.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.981+0000 7f31c0c00640 1 -- 172.21.15.18:0/352387160 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31bc004cc0 con 0x7f31c4105e10 2024-04-16T17:02:00.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.981+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c41a2470 con 0x7f31c4105e10 2024-04-16T17:02:00.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.981+0000 7f31c0c00640 1 -- 172.21.15.18:0/352387160 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc010760 con 0x7f31c4105e10 2024-04-16T17:02:00.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.981+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c41a2940 con 0x7f31c4105e10 2024-04-16T17:02:00.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.982+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c410b600 con 0x7f31c4105e10 2024-04-16T17:02:00.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.984+0000 7f31c0c00640 1 -- 172.21.15.18:0/352387160 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31bc010900 con 0x7f31c4105e10 2024-04-16T17:02:00.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.986+0000 7f31c0c00640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f31a80a3bb0 0x7f31a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:00.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.986+0000 7f31c0c00640 1 -- 172.21.15.18:0/352387160 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f31bc0d0be0 con 0x7f31c4105e10 2024-04-16T17:02:00.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.986+0000 7f31c3e00640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f31a80a3bb0 0x7f31a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:00.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.987+0000 7f31c3e00640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f31a80a3bb0 0x7f31a80a6070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f31c41052b0 tx=0x7f31b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:00.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:00.988+0000 7f31c0c00640 1 -- 172.21.15.18:0/352387160 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31bc09d320 con 0x7f31c4105e10 2024-04-16T17:02:01.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.194+0000 7f31c0c00640 1 -- 172.21.15.18:0/352387160 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f31bc099290 con 0x7f31c4105e10 2024-04-16T17:02:01.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.239+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f31c4062440 con 0x7f31a80a3bb0 2024-04-16T17:02:01.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.277+0000 7f31c0c00640 1 -- 172.21.15.18:0/352387160 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f31c4062440 con 0x7f31a80a3bb0 2024-04-16T17:02:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.280+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f31a80a3bb0 msgr2=0x7f31a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.280+0000 7f31cb200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f31a80a3bb0 0x7f31a80a6070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f31c41052b0 tx=0x7f31b4005f90 comp rx=0 tx=0).stop 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.280+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105e10 msgr2=0x7f31c41a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.280+0000 7f31cb200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105e10 0x7f31c41a1b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00eb10 tx=0x7f31bc00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.281+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 shutdown_connections 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.281+0000 7f31cb200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31c4105e10 0x7f31c41a1b10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.281+0000 7f31cb200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f31a80a3bb0 0x7f31a80a6070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.281+0000 7f31cb200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31c4105450 0x7f31c419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.281+0000 7f31cb200640 1 --2- 172.21.15.18:0/352387160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c4104260 0x7f31c419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.281+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 >> 172.21.15.18:0/352387160 conn(0x7f31c40ff7f0 msgr2=0x7f31c41087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.281+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 shutdown_connections 2024-04-16T17:02:01.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.281+0000 7f31cb200640 1 -- 172.21.15.18:0/352387160 wait complete. 2024-04-16T17:02:01.302 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-16T17:02:01.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:01 smithi018 ceph-mon[31776]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:01.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:01 smithi018 ceph-mon[31776]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:01.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:01 smithi043 ceph-mon[36901]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:01.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:01 smithi043 ceph-mon[36901]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:01.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 -- 172.21.15.18:0/463026831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39001028f0 msgr2=0x7f3900104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 --2- 172.21.15.18:0/463026831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39001028f0 0x7f3900104ce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009900 tx=0x7f38e802bc30 comp rx=0 tx=0).stop 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 -- 172.21.15.18:0/463026831 shutdown_connections 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 --2- 172.21.15.18:0/463026831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3900107b50 0x7f3900109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 --2- 172.21.15.18:0/463026831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3900105220 0x7f3900107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 --2- 172.21.15.18:0/463026831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39001028f0 0x7f3900104ce0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 -- 172.21.15.18:0/463026831 >> 172.21.15.18:0/463026831 conn(0x7f39000fc2e0 msgr2=0x7f39000fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 -- 172.21.15.18:0/463026831 shutdown_connections 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.527+0000 7f3905e00640 1 -- 172.21.15.18:0/463026831 wait complete. 2024-04-16T17:02:01.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.528+0000 7f3905e00640 1 Processor -- start 2024-04-16T17:02:01.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.528+0000 7f3905e00640 1 -- start start 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.528+0000 7f3905e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39001028f0 0x7f3900102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.528+0000 7f3905e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3900105220 0x7f39001008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3900107b50 0x7f3900100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f390010db60 con 0x7f3900107b50 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f390010d9e0 con 0x7f39001028f0 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f390010dce0 con 0x7f3900105220 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3900107b50 0x7f3900100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3904a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39001028f0 0x7f3900102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3900107b50 0x7f3900100de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54332/0 (socket says 172.21.15.18:54332) 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905400640 1 -- 172.21.15.18:0/2671954330 learned_addr learned my addr 172.21.15.18:0/2671954330 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f38ffe00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3900105220 0x7f39001008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:01.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905400640 1 -- 172.21.15.18:0/2671954330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3900105220 msgr2=0x7f39001008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905400640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3900105220 0x7f39001008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905400640 1 -- 172.21.15.18:0/2671954330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39001028f0 msgr2=0x7f3900102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905400640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39001028f0 0x7f3900102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f3905400640 1 -- 172.21.15.18:0/2671954330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3900101670 con 0x7f3900107b50 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.529+0000 7f38ffe00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3900105220 0x7f39001008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.530+0000 7f3905400640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3900107b50 0x7f3900100de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e970 tx=0x7f38f800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.530+0000 7f38fd600640 1 -- 172.21.15.18:0/2671954330 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f800ccc0 con 0x7f3900107b50 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.530+0000 7f38fd600640 1 -- 172.21.15.18:0/2671954330 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f800ce60 con 0x7f3900107b50 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.530+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39001a22d0 con 0x7f3900107b50 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.530+0000 7f38fd600640 1 -- 172.21.15.18:0/2671954330 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8010490 con 0x7f3900107b50 2024-04-16T17:02:01.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.530+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39001a2820 con 0x7f3900107b50 2024-04-16T17:02:01.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.531+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39000fddd0 con 0x7f3900107b50 2024-04-16T17:02:01.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.533+0000 7f38fd600640 1 -- 172.21.15.18:0/2671954330 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f38f8005020 con 0x7f3900107b50 2024-04-16T17:02:01.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.535+0000 7f38fd600640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38e40a3c40 0x7f38e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:01.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.535+0000 7f38fd600640 1 -- 172.21.15.18:0/2671954330 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f38f80d0ca0 con 0x7f3900107b50 2024-04-16T17:02:01.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.535+0000 7f3904a00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38e40a3c40 0x7f38e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:01.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.536+0000 7f3904a00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38e40a3c40 0x7f38e40a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38e802c240 tx=0x7f38e802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:01.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.539+0000 7f38fd600640 1 -- 172.21.15.18:0/2671954330 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38f809d280 con 0x7f3900107b50 2024-04-16T17:02:01.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:01 smithi022 ceph-mon[36023]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:01.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:01 smithi022 ceph-mon[36023]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:01.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.782+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f39000624d0 con 0x7f38e40a3c40 2024-04-16T17:02:01.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.799+0000 7f38fd600640 1 -- 172.21.15.18:0/2671954330 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f39000624d0 con 0x7f38e40a3c40 2024-04-16T17:02:01.801 INFO:teuthology.orchestra.run.smithi018.stdout:/volumes/qa/sv_1/6f0bb715-aa68-4ec4-ae07-0cc7013aa45c 2024-04-16T17:02:01.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38e40a3c40 msgr2=0x7f38e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38e40a3c40 0x7f38e40a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38e802c240 tx=0x7f38e802df90 comp rx=0 tx=0).stop 2024-04-16T17:02:01.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3900107b50 msgr2=0x7f3900100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3900107b50 0x7f3900100de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e970 tx=0x7f38f800ed70 comp rx=0 tx=0).stop 2024-04-16T17:02:01.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 shutdown_connections 2024-04-16T17:02:01.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38e40a3c40 0x7f38e40a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3900107b50 0x7f3900100de0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3900105220 0x7f39001008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 --2- 172.21.15.18:0/2671954330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39001028f0 0x7f3900102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.802+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 >> 172.21.15.18:0/2671954330 conn(0x7f39000fc2e0 msgr2=0x7f3900103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:01.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.803+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 shutdown_connections 2024-04-16T17:02:01.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:01.803+0000 7f3905e00640 1 -- 172.21.15.18:0/2671954330 wait complete. 2024-04-16T17:02:01.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:02.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.044+0000 7f44f7000640 1 -- 172.21.15.18:0/2516642602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0103080 msgr2=0x7f44f0103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.044+0000 7f44f7000640 1 --2- 172.21.15.18:0/2516642602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0103080 0x7f44f0103500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009900 tx=0x7f44e002bc30 comp rx=0 tx=0).stop 2024-04-16T17:02:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.045+0000 7f44f7000640 1 -- 172.21.15.18:0/2516642602 shutdown_connections 2024-04-16T17:02:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.045+0000 7f44f7000640 1 --2- 172.21.15.18:0/2516642602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f0103a40 0x7f44f010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.045+0000 7f44f7000640 1 --2- 172.21.15.18:0/2516642602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0103080 0x7f44f0103500 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.045+0000 7f44f7000640 1 --2- 172.21.15.18:0/2516642602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44f0101f30 0x7f44f0102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.045+0000 7f44f7000640 1 -- 172.21.15.18:0/2516642602 >> 172.21.15.18:0/2516642602 conn(0x7f44f00fd4c0 msgr2=0x7f44f00ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.045+0000 7f44f7000640 1 -- 172.21.15.18:0/2516642602 shutdown_connections 2024-04-16T17:02:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.045+0000 7f44f7000640 1 -- 172.21.15.18:0/2516642602 wait complete. 2024-04-16T17:02:02.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f7000640 1 Processor -- start 2024-04-16T17:02:02.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f7000640 1 -- start start 2024-04-16T17:02:02.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f7000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0101f30 0x7f44f019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f7000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44f0103080 0x7f44f019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f7000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f0103a40 0x7f44f01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f7000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f44f010dbe0 con 0x7f44f0101f30 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f7000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f44f010da60 con 0x7f44f0103080 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f7000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44f010dd60 con 0x7f44f0103a40 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.046+0000 7f44f5c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0101f30 0x7f44f019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0101f30 0x7f44f019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54352/0 (socket says 172.21.15.18:54352) 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5c00640 1 -- 172.21.15.18:0/1862170702 learned_addr learned my addr 172.21.15.18:0/1862170702 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f6600640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f0103a40 0x7f44f01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5200640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44f0103080 0x7f44f019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5c00640 1 -- 172.21.15.18:0/1862170702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f0103a40 msgr2=0x7f44f01a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5c00640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f0103a40 0x7f44f01a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5c00640 1 -- 172.21.15.18:0/1862170702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44f0103080 msgr2=0x7f44f019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5c00640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44f0103080 0x7f44f019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5c00640 1 -- 172.21.15.18:0/1862170702 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f01a20b0 con 0x7f44f0101f30 2024-04-16T17:02:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f6600640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f0103a40 0x7f44f01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44f5c00640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0101f30 0x7f44f019a490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44e400d950 tx=0x7f44e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.047+0000 7f44ee000640 1 -- 172.21.15.18:0/1862170702 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4017030 con 0x7f44f0101f30 2024-04-16T17:02:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.048+0000 7f44ee000640 1 -- 172.21.15.18:0/1862170702 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e4004020 con 0x7f44f0101f30 2024-04-16T17:02:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.048+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f01a23a0 con 0x7f44f0101f30 2024-04-16T17:02:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.048+0000 7f44ee000640 1 -- 172.21.15.18:0/1862170702 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40047a0 con 0x7f44f0101f30 2024-04-16T17:02:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.048+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f01a2910 con 0x7f44f0101f30 2024-04-16T17:02:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.049+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f01a2bd0 con 0x7f44f0101f30 2024-04-16T17:02:02.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.051+0000 7f44ee000640 1 -- 172.21.15.18:0/1862170702 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f44e40041d0 con 0x7f44f0101f30 2024-04-16T17:02:02.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.052+0000 7f44ee000640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f44d80a3cb0 0x7f44d80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.052+0000 7f44ee000640 1 -- 172.21.15.18:0/1862170702 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f44e40d01a0 con 0x7f44f0101f30 2024-04-16T17:02:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.052+0000 7f44f5200640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f44d80a3cb0 0x7f44d80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:02.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.053+0000 7f44f5200640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f44d80a3cb0 0x7f44d80a6170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f44e0000c00 tx=0x7f44e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:02.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.056+0000 7f44ee000640 1 -- 172.21.15.18:0/1862170702 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44e409c780 con 0x7f44f0101f30 2024-04-16T17:02:02.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.302+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44f01a2da0 con 0x7f44f0101f30 2024-04-16T17:02:02.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.304+0000 7f44ee000640 1 -- 172.21.15.18:0/1862170702 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11848 (secure 0 0 0) 0x7f44e409c080 con 0x7f44f0101f30 2024-04-16T17:02:02.306 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:02:02.306 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:01:59.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:02:02.306 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 24 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f44d80a3cb0 msgr2=0x7f44d80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f44d80a3cb0 0x7f44d80a6170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f44e0000c00 tx=0x7f44e0005fd0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0101f30 msgr2=0x7f44f019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0101f30 0x7f44f019a490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44e400d950 tx=0x7f44e400de10 comp rx=0 tx=0).stop 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 shutdown_connections 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f0103a40 0x7f44f01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44f0103080 0x7f44f019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f44d80a3cb0 0x7f44d80a6170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 --2- 172.21.15.18:0/1862170702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44f0101f30 0x7f44f019a490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 >> 172.21.15.18:0/1862170702 conn(0x7f44f00fd4c0 msgr2=0x7f44f0074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 shutdown_connections 2024-04-16T17:02:02.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.307+0000 7f44f7000640 1 -- 172.21.15.18:0/1862170702 wait complete. 2024-04-16T17:02:02.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:02:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:02 smithi018 ceph-mon[31776]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:02 smithi018 ceph-mon[31776]: pgmap v337: 129 pgs: 129 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-16T17:02:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:02 smithi018 ceph-mon[31776]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-16T17:02:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:02 smithi018 ceph-mon[31776]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:02.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:02 smithi043 ceph-mon[36901]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:02.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:02 smithi043 ceph-mon[36901]: pgmap v337: 129 pgs: 129 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-16T17:02:02.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:02 smithi043 ceph-mon[36901]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-16T17:02:02.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:02 smithi043 ceph-mon[36901]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:02.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.549+0000 7f7d31000640 1 -- 172.21.15.18:0/1866330107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c105e90 msgr2=0x7f7d2c10c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.549+0000 7f7d31000640 1 --2- 172.21.15.18:0/1866330107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c105e90 0x7f7d2c10c720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009be0 tx=0x7f7d2402bef0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.550+0000 7f7d31000640 1 -- 172.21.15.18:0/1866330107 shutdown_connections 2024-04-16T17:02:02.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.550+0000 7f7d31000640 1 --2- 172.21.15.18:0/1866330107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c105e90 0x7f7d2c10c720 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.550+0000 7f7d31000640 1 --2- 172.21.15.18:0/1866330107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c1054d0 0x7f7d2c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.550+0000 7f7d31000640 1 --2- 172.21.15.18:0/1866330107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d2c1042e0 0x7f7d2c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.550+0000 7f7d31000640 1 -- 172.21.15.18:0/1866330107 >> 172.21.15.18:0/1866330107 conn(0x7f7d2c0ff850 msgr2=0x7f7d2c101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.550+0000 7f7d31000640 1 -- 172.21.15.18:0/1866330107 shutdown_connections 2024-04-16T17:02:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.550+0000 7f7d31000640 1 -- 172.21.15.18:0/1866330107 wait complete. 2024-04-16T17:02:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.551+0000 7f7d31000640 1 Processor -- start 2024-04-16T17:02:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.551+0000 7f7d31000640 1 -- start start 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.551+0000 7f7d31000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c1042e0 0x7f7d2c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.551+0000 7f7d31000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d2c1054d0 0x7f7d2c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.551+0000 7f7d31000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c105e90 0x7f7d2c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.551+0000 7f7d31000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c10ff00 con 0x7f7d2c1054d0 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.551+0000 7f7d31000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c10fd80 con 0x7f7d2c1042e0 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.551+0000 7f7d31000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c110080 con 0x7f7d2c105e90 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d29e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c1042e0 0x7f7d2c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d29e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c1042e0 0x7f7d2c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54268/0 (socket says 172.21.15.18:54268) 2024-04-16T17:02:02.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d29e00640 1 -- 172.21.15.18:0/1144154775 learned_addr learned my addr 172.21.15.18:0/1144154775 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:02.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d23e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d2c1054d0 0x7f7d2c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:02.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d2a800640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c105e90 0x7f7d2c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:02.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d29e00640 1 -- 172.21.15.18:0/1144154775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c105e90 msgr2=0x7f7d2c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d29e00640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c105e90 0x7f7d2c1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d29e00640 1 -- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d2c1054d0 msgr2=0x7f7d2c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d29e00640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d2c1054d0 0x7f7d2c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d29e00640 1 -- 172.21.15.18:0/1144154775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c1a2160 con 0x7f7d2c1042e0 2024-04-16T17:02:02.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.552+0000 7f7d23e00640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d2c1054d0 0x7f7d2c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:02.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.553+0000 7f7d29e00640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c1042e0 0x7f7d2c19a540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e9f0 tx=0x7f7d1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:02.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.553+0000 7f7d22a00640 1 -- 172.21.15.18:0/1144154775 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00ccc0 con 0x7f7d2c1042e0 2024-04-16T17:02:02.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.553+0000 7f7d22a00640 1 -- 172.21.15.18:0/1144154775 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c0035f0 con 0x7f7d2c1042e0 2024-04-16T17:02:02.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.553+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c1a2450 con 0x7f7d2c1042e0 2024-04-16T17:02:02.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.553+0000 7f7d22a00640 1 -- 172.21.15.18:0/1144154775 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00ce60 con 0x7f7d2c1042e0 2024-04-16T17:02:02.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.553+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c1a29a0 con 0x7f7d2c1042e0 2024-04-16T17:02:02.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.554+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2c10b5a0 con 0x7f7d2c1042e0 2024-04-16T17:02:02.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.556+0000 7f7d22a00640 1 -- 172.21.15.18:0/1144154775 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d1c0037a0 con 0x7f7d2c1042e0 2024-04-16T17:02:02.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.558+0000 7f7d22a00640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7d100a3c40 0x7f7d100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:02.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.558+0000 7f7d22a00640 1 -- 172.21.15.18:0/1144154775 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7d1c0d0bc0 con 0x7f7d2c1042e0 2024-04-16T17:02:02.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.558+0000 7f7d23e00640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7d100a3c40 0x7f7d100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:02.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.559+0000 7f7d23e00640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7d100a3c40 0x7f7d100a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c19ba50 tx=0x7f7d14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:02.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.560+0000 7f7d22a00640 1 -- 172.21.15.18:0/1144154775 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d1c09d1a0 con 0x7f7d2c1042e0 2024-04-16T17:02:02.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:02 smithi022 ceph-mon[36023]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:02.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:02 smithi022 ceph-mon[36023]: pgmap v337: 129 pgs: 129 active+clean; 583 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-16T17:02:02.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:02 smithi022 ceph-mon[36023]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-16T17:02:02.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:02 smithi022 ceph-mon[36023]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.774+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d2c062c00 con 0x7f7d2c1042e0 2024-04-16T17:02:02.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.776+0000 7f7d22a00640 1 -- 172.21.15.18:0/1144154775 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7f7d1c0bd0e0 con 0x7f7d2c1042e0 2024-04-16T17:02:02.778 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:02:02.781 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:02:02.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7d100a3c40 msgr2=0x7f7d100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7d100a3c40 0x7f7d100a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c19ba50 tx=0x7f7d14005f90 comp rx=0 tx=0).stop 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c1042e0 msgr2=0x7f7d2c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c1042e0 0x7f7d2c19a540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e9f0 tx=0x7f7d1c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 shutdown_connections 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c105e90 0x7f7d2c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7d100a3c40 0x7f7d100a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d2c1054d0 0x7f7d2c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 --2- 172.21.15.18:0/1144154775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d2c1042e0 0x7f7d2c19a540 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.779+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 >> 172.21.15.18:0/1144154775 conn(0x7f7d2c0ff850 msgr2=0x7f7d2c108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.780+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 shutdown_connections 2024-04-16T17:02:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:02.780+0000 7f7d31000640 1 -- 172.21.15.18:0/1144154775 wait complete. 2024-04-16T17:02:02.803 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-16T17:02:02.803 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:02:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 -- 172.21.15.18:0/135073596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c104160 msgr2=0x7f7a3c104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 --2- 172.21.15.18:0/135073596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c104160 0x7f7a3c104580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009900 tx=0x7f7a2c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:02:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 -- 172.21.15.18:0/135073596 shutdown_connections 2024-04-16T17:02:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 --2- 172.21.15.18:0/135073596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3c105d10 0x7f7a3c10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 --2- 172.21.15.18:0/135073596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a3c105350 0x7f7a3c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 --2- 172.21.15.18:0/135073596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c104160 0x7f7a3c104580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 -- 172.21.15.18:0/135073596 >> 172.21.15.18:0/135073596 conn(0x7f7a3c0ff6d0 msgr2=0x7f7a3c101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 -- 172.21.15.18:0/135073596 shutdown_connections 2024-04-16T17:02:03.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.030+0000 7f7a40a00640 1 -- 172.21.15.18:0/135073596 wait complete. 2024-04-16T17:02:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.031+0000 7f7a40a00640 1 Processor -- start 2024-04-16T17:02:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.031+0000 7f7a40a00640 1 -- start start 2024-04-16T17:02:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.031+0000 7f7a40a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3c104160 0x7f7a3c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.031+0000 7f7a40a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a3c105350 0x7f7a3c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.031+0000 7f7a40a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c105d10 0x7f7a3c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.031+0000 7f7a40a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c10fe40 con 0x7f7a3c105d10 2024-04-16T17:02:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.031+0000 7f7a40a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c10fcc0 con 0x7f7a3c104160 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.031+0000 7f7a40a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c10ffc0 con 0x7f7a3c105350 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3c104160 0x7f7a3c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c105d10 0x7f7a3c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3c104160 0x7f7a3c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54280/0 (socket says 172.21.15.18:54280) 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3b400640 1 -- 172.21.15.18:0/1008390740 learned_addr learned my addr 172.21.15.18:0/1008390740 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3aa00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a3c105350 0x7f7a3c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3b400640 1 -- 172.21.15.18:0/1008390740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a3c105350 msgr2=0x7f7a3c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3b400640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a3c105350 0x7f7a3c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3b400640 1 -- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c105d10 msgr2=0x7f7a3c1a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3b400640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c105d10 0x7f7a3c1a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3b400640 1 -- 172.21.15.18:0/1008390740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c1a2070 con 0x7f7a3c104160 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.032+0000 7f7a3be00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c105d10 0x7f7a3c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.033+0000 7f7a3b400640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3c104160 0x7f7a3c19a450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c02efd0 tx=0x7f7a2c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.033+0000 7f7a33e00640 1 -- 172.21.15.18:0/1008390740 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c041070 con 0x7f7a3c104160 2024-04-16T17:02:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.033+0000 7f7a33e00640 1 -- 172.21.15.18:0/1008390740 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a2c0044d0 con 0x7f7a3c104160 2024-04-16T17:02:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.033+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3c1a2300 con 0x7f7a3c104160 2024-04-16T17:02:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.033+0000 7f7a33e00640 1 -- 172.21.15.18:0/1008390740 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c034440 con 0x7f7a3c104160 2024-04-16T17:02:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.033+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3c1a27c0 con 0x7f7a3c104160 2024-04-16T17:02:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.034+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3c10b480 con 0x7f7a3c104160 2024-04-16T17:02:03.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.036+0000 7f7a33e00640 1 -- 172.21.15.18:0/1008390740 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7a2c044070 con 0x7f7a3c104160 2024-04-16T17:02:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.037+0000 7f7a33e00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7a200a3c30 0x7f7a200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.038+0000 7f7a33e00640 1 -- 172.21.15.18:0/1008390740 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7a2c0f2260 con 0x7f7a3c104160 2024-04-16T17:02:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.038+0000 7f7a3aa00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7a200a3c30 0x7f7a200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.038+0000 7f7a3aa00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7a200a3c30 0x7f7a200a60f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a280095f0 tx=0x7f7a28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.042+0000 7f7a33e00640 1 -- 172.21.15.18:0/1008390740 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a2c0be300 con 0x7f7a3c104160 2024-04-16T17:02:03.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.247+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a3c062440 con 0x7f7a200a3c30 2024-04-16T17:02:03.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.272+0000 7f7a33e00640 1 -- 172.21.15.18:0/1008390740 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340623 (secure 0 0 0) 0x7f7a3c062440 con 0x7f7a200a3c30 2024-04-16T17:02:03.280 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:02:03.282 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.278+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7a200a3c30 msgr2=0x7f7a200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.278+0000 7f7a40a00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7a200a3c30 0x7f7a200a60f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a280095f0 tx=0x7f7a28005f90 comp rx=0 tx=0).stop 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.278+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3c104160 msgr2=0x7f7a3c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.278+0000 7f7a40a00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3c104160 0x7f7a3c19a450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c02efd0 tx=0x7f7a2c005d50 comp rx=0 tx=0).stop 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.279+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 shutdown_connections 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.279+0000 7f7a40a00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7a200a3c30 0x7f7a200a60f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.279+0000 7f7a40a00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a3c105d10 0x7f7a3c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.279+0000 7f7a40a00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a3c105350 0x7f7a3c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.279+0000 7f7a40a00640 1 --2- 172.21.15.18:0/1008390740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a3c104160 0x7f7a3c19a450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.279+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 >> 172.21.15.18:0/1008390740 conn(0x7f7a3c0ff6d0 msgr2=0x7f7a3c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:03.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.279+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 shutdown_connections 2024-04-16T17:02:03.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.279+0000 7f7a40a00640 1 -- 172.21.15.18:0/1008390740 wait complete. 2024-04-16T17:02:03.299 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-04-16T17:02:02.890752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598054,"num_objects":63,"num_object_clones":0,"num_object_copies":191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":162,"num_read_kb":137,"num_write":315,"num_write_kb":2986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373300,"kb_used_data":11668,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568524,"statfs":{"total":1151940427776,"available":1151558168576,"internally_reserved":0,"allocated":11948032,"data_stored":7543300,"data_compressed":283101,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3046,"num_objects":39,"num_object_clones":0,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":85,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006517"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512591+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:01:30.512591+0000","last_peered":"2024-04-16T17:01:30.512591+0000","last_clean":"2024-04-16T17:01:30.512591+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:01:30.512591+0000","last_undegraded":"2024-04-16T17:01:30.512591+0000","last_fullsized":"2024-04-16T17:01:30.512591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018183+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:01:30.018183+0000","last_peered":"2024-04-16T17:01:30.018183+0000","last_clean":"2024-04-16T17:01:30.018183+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:01:30.018183+0000","last_undegraded":"2024-04-16T17:01:30.018183+0000","last_fullsized":"2024-04-16T17:01:30.018183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.017615+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:01:30.017615+0000","last_peered":"2024-04-16T17:01:30.017615+0000","last_clean":"2024-04-16T17:01:30.017615+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:01:30.017615+0000","last_undegraded":"2024-04-16T17:01:30.017615+0000","last_fullsized":"2024-04-16T17:01:30.017615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845829+0000","last_change":"2024-04-16T17:01:24.537712+0000","last_active":"2024-04-16T17:01:30.845829+0000","last_peered":"2024-04-16T17:01:30.845829+0000","last_clean":"2024-04-16T17:01:30.845829+0000","last_became_active":"2024-04-16T17:01:24.537350+0000","last_became_peered":"2024-04-16T17:01:24.537350+0000","last_unstale":"2024-04-16T17:01:30.845829+0000","last_undegraded":"2024-04-16T17:01:30.845829+0000","last_fullsized":"2024-04-16T17:01:30.845829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:05.792403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511931+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:01:30.511931+0000","last_peered":"2024-04-16T17:01:30.511931+0000","last_clean":"2024-04-16T17:01:30.511931+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:01:30.511931+0000","last_undegraded":"2024-04-16T17:01:30.511931+0000","last_fullsized":"2024-04-16T17:01:30.511931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512250+0000","last_change":"2024-04-16T17:01:24.543268+0000","last_active":"2024-04-16T17:01:30.512250+0000","last_peered":"2024-04-16T17:01:30.512250+0000","last_clean":"2024-04-16T17:01:30.512250+0000","last_became_active":"2024-04-16T17:01:24.542926+0000","last_became_peered":"2024-04-16T17:01:24.542926+0000","last_unstale":"2024-04-16T17:01:30.512250+0000","last_undegraded":"2024-04-16T17:01:30.512250+0000","last_fullsized":"2024-04-16T17:01:30.512250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:03.105040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014291+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:01:30.014291+0000","last_peered":"2024-04-16T17:01:30.014291+0000","last_clean":"2024-04-16T17:01:30.014291+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:01:30.014291+0000","last_undegraded":"2024-04-16T17:01:30.014291+0000","last_fullsized":"2024-04-16T17:01:30.014291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035406+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:01:31.035406+0000","last_peered":"2024-04-16T17:01:31.035406+0000","last_clean":"2024-04-16T17:01:31.035406+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:01:31.035406+0000","last_undegraded":"2024-04-16T17:01:31.035406+0000","last_fullsized":"2024-04-16T17:01:31.035406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845490+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:01:30.845490+0000","last_peered":"2024-04-16T17:01:30.845490+0000","last_clean":"2024-04-16T17:01:30.845490+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:01:30.845490+0000","last_undegraded":"2024-04-16T17:01:30.845490+0000","last_fullsized":"2024-04-16T17:01:30.845490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512388+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:01:30.512388+0000","last_peered":"2024-04-16T17:01:30.512388+0000","last_clean":"2024-04-16T17:01:30.512388+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:01:30.512388+0000","last_undegraded":"2024-04-16T17:01:30.512388+0000","last_fullsized":"2024-04-16T17:01:30.512388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014984+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:01:30.014984+0000","last_peered":"2024-04-16T17:01:30.014984+0000","last_clean":"2024-04-16T17:01:30.014984+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:01:30.014984+0000","last_undegraded":"2024-04-16T17:01:30.014984+0000","last_fullsized":"2024-04-16T17:01:30.014984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845914+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:01:30.845914+0000","last_peered":"2024-04-16T17:01:30.845914+0000","last_clean":"2024-04-16T17:01:30.845914+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:01:30.845914+0000","last_undegraded":"2024-04-16T17:01:30.845914+0000","last_fullsized":"2024-04-16T17:01:30.845914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511578+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:01:30.511578+0000","last_peered":"2024-04-16T17:01:30.511578+0000","last_clean":"2024-04-16T17:01:30.511578+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:01:30.511578+0000","last_undegraded":"2024-04-16T17:01:30.511578+0000","last_fullsized":"2024-04-16T17:01:30.511578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.016618+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:01:30.016618+0000","last_peered":"2024-04-16T17:01:30.016618+0000","last_clean":"2024-04-16T17:01:30.016618+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:01:30.016618+0000","last_undegraded":"2024-04-16T17:01:30.016618+0000","last_fullsized":"2024-04-16T17:01:30.016618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.016502+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:01:30.016502+0000","last_peered":"2024-04-16T17:01:30.016502+0000","last_clean":"2024-04-16T17:01:30.016502+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:01:30.016502+0000","last_undegraded":"2024-04-16T17:01:30.016502+0000","last_fullsized":"2024-04-16T17:01:30.016502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.844658+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:01:30.844658+0000","last_peered":"2024-04-16T17:01:30.844658+0000","last_clean":"2024-04-16T17:01:30.844658+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:01:30.844658+0000","last_undegraded":"2024-04-16T17:01:30.844658+0000","last_fullsized":"2024-04-16T17:01:30.844658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512525+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:01:30.512525+0000","last_peered":"2024-04-16T17:01:30.512525+0000","last_clean":"2024-04-16T17:01:30.512525+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:01:30.512525+0000","last_undegraded":"2024-04-16T17:01:30.512525+0000","last_fullsized":"2024-04-16T17:01:30.512525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.387732+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:01:30.387732+0000","last_peered":"2024-04-16T17:01:30.387732+0000","last_clean":"2024-04-16T17:01:30.387732+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:01:30.387732+0000","last_undegraded":"2024-04-16T17:01:30.387732+0000","last_fullsized":"2024-04-16T17:01:30.387732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845571+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:01:30.845571+0000","last_peered":"2024-04-16T17:01:30.845571+0000","last_clean":"2024-04-16T17:01:30.845571+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:01:30.845571+0000","last_undegraded":"2024-04-16T17:01:30.845571+0000","last_fullsized":"2024-04-16T17:01:30.845571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845339+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:01:30.845339+0000","last_peered":"2024-04-16T17:01:30.845339+0000","last_clean":"2024-04-16T17:01:30.845339+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:01:30.845339+0000","last_undegraded":"2024-04-16T17:01:30.845339+0000","last_fullsized":"2024-04-16T17:01:30.845339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.172920+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:01:58.172920+0000","last_peered":"2024-04-16T17:01:58.172920+0000","last_clean":"2024-04-16T17:01:58.172920+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:01:58.172920+0000","last_undegraded":"2024-04-16T17:01:58.172920+0000","last_fullsized":"2024-04-16T17:01:58.172920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.036927+0000","last_change":"2024-04-16T17:01:24.562184+0000","last_active":"2024-04-16T17:01:31.036927+0000","last_peered":"2024-04-16T17:01:31.036927+0000","last_clean":"2024-04-16T17:01:31.036927+0000","last_became_active":"2024-04-16T17:01:24.558485+0000","last_became_peered":"2024-04-16T17:01:24.558485+0000","last_unstale":"2024-04-16T17:01:31.036927+0000","last_undegraded":"2024-04-16T17:01:31.036927+0000","last_fullsized":"2024-04-16T17:01:31.036927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:09.994752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845150+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:01:30.845150+0000","last_peered":"2024-04-16T17:01:30.845150+0000","last_clean":"2024-04-16T17:01:30.845150+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:01:30.845150+0000","last_undegraded":"2024-04-16T17:01:30.845150+0000","last_fullsized":"2024-04-16T17:01:30.845150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.037254+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:01:31.037254+0000","last_peered":"2024-04-16T17:01:31.037254+0000","last_clean":"2024-04-16T17:01:31.037254+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:01:31.037254+0000","last_undegraded":"2024-04-16T17:01:31.037254+0000","last_fullsized":"2024-04-16T17:01:31.037254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018417+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:01:30.018417+0000","last_peered":"2024-04-16T17:01:30.018417+0000","last_clean":"2024-04-16T17:01:30.018417+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:01:30.018417+0000","last_undegraded":"2024-04-16T17:01:30.018417+0000","last_fullsized":"2024-04-16T17:01:30.018417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845638+0000","last_change":"2024-04-16T17:01:29.035651+0000","last_active":"2024-04-16T17:01:30.845638+0000","last_peered":"2024-04-16T17:01:30.845638+0000","last_clean":"2024-04-16T17:01:30.845638+0000","last_became_active":"2024-04-16T17:01:29.034865+0000","last_became_peered":"2024-04-16T17:01:29.034865+0000","last_unstale":"2024-04-16T17:01:30.845638+0000","last_undegraded":"2024-04-16T17:01:30.845638+0000","last_fullsized":"2024-04-16T17:01:30.845638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:49:31.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.164582+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:01:58.164582+0000","last_peered":"2024-04-16T17:01:58.164582+0000","last_clean":"2024-04-16T17:01:58.164582+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:01:58.164582+0000","last_undegraded":"2024-04-16T17:01:58.164582+0000","last_fullsized":"2024-04-16T17:01:58.164582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845729+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:01:30.845729+0000","last_peered":"2024-04-16T17:01:30.845729+0000","last_clean":"2024-04-16T17:01:30.845729+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:01:30.845729+0000","last_undegraded":"2024-04-16T17:01:30.845729+0000","last_fullsized":"2024-04-16T17:01:30.845729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845226+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:01:30.845226+0000","last_peered":"2024-04-16T17:01:30.845226+0000","last_clean":"2024-04-16T17:01:30.845226+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:01:30.845226+0000","last_undegraded":"2024-04-16T17:01:30.845226+0000","last_fullsized":"2024-04-16T17:01:30.845226+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512267+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:01:30.512267+0000","last_peered":"2024-04-16T17:01:30.512267+0000","last_clean":"2024-04-16T17:01:30.512267+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:01:30.512267+0000","last_undegraded":"2024-04-16T17:01:30.512267+0000","last_fullsized":"2024-04-16T17:01:30.512267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223419+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:01:30.223419+0000","last_peered":"2024-04-16T17:01:30.223419+0000","last_clean":"2024-04-16T17:01:30.223419+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:01:30.223419+0000","last_undegraded":"2024-04-16T17:01:30.223419+0000","last_fullsized":"2024-04-16T17:01:30.223419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845558+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:01:30.845558+0000","last_peered":"2024-04-16T17:01:30.845558+0000","last_clean":"2024-04-16T17:01:30.845558+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:01:30.845558+0000","last_undegraded":"2024-04-16T17:01:30.845558+0000","last_fullsized":"2024-04-16T17:01:30.845558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:02:01.180669+0000","last_change":"2024-04-16T17:01:23.915773+0000","last_active":"2024-04-16T17:02:01.180669+0000","last_peered":"2024-04-16T17:02:01.180669+0000","last_clean":"2024-04-16T17:02:01.180669+0000","last_became_active":"2024-04-16T17:01:23.915398+0000","last_became_peered":"2024-04-16T17:01:23.915398+0000","last_unstale":"2024-04-16T17:02:01.180669+0000","last_undegraded":"2024-04-16T17:02:01.180669+0000","last_fullsized":"2024-04-16T17:02:01.180669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:32:50.294516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512590+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:01:30.512590+0000","last_peered":"2024-04-16T17:01:30.512590+0000","last_clean":"2024-04-16T17:01:30.512590+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:01:30.512590+0000","last_undegraded":"2024-04-16T17:01:30.512590+0000","last_fullsized":"2024-04-16T17:01:30.512590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845585+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:01:30.845585+0000","last_peered":"2024-04-16T17:01:30.845585+0000","last_clean":"2024-04-16T17:01:30.845585+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:01:30.845585+0000","last_undegraded":"2024-04-16T17:01:30.845585+0000","last_fullsized":"2024-04-16T17:01:30.845585+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512256+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:01:30.512256+0000","last_peered":"2024-04-16T17:01:30.512256+0000","last_clean":"2024-04-16T17:01:30.512256+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:01:30.512256+0000","last_undegraded":"2024-04-16T17:01:30.512256+0000","last_fullsized":"2024-04-16T17:01:30.512256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223374+0000","last_change":"2024-04-16T17:01:24.558878+0000","last_active":"2024-04-16T17:01:30.223374+0000","last_peered":"2024-04-16T17:01:30.223374+0000","last_clean":"2024-04-16T17:01:30.223374+0000","last_became_active":"2024-04-16T17:01:24.558504+0000","last_became_peered":"2024-04-16T17:01:24.558504+0000","last_unstale":"2024-04-16T17:01:30.223374+0000","last_undegraded":"2024-04-16T17:01:30.223374+0000","last_fullsized":"2024-04-16T17:01:30.223374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:34.526473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512389+0000","last_change":"2024-04-16T17:01:29.032665+0000","last_active":"2024-04-16T17:01:30.512389+0000","last_peered":"2024-04-16T17:01:30.512389+0000","last_clean":"2024-04-16T17:01:30.512389+0000","last_became_active":"2024-04-16T17:01:29.032293+0000","last_became_peered":"2024-04-16T17:01:29.032293+0000","last_unstale":"2024-04-16T17:01:30.512389+0000","last_undegraded":"2024-04-16T17:01:30.512389+0000","last_fullsized":"2024-04-16T17:01:30.512389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:46.193049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.172686+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:01:58.172686+0000","last_peered":"2024-04-16T17:01:58.172686+0000","last_clean":"2024-04-16T17:01:58.172686+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:01:58.172686+0000","last_undegraded":"2024-04-16T17:01:58.172686+0000","last_fullsized":"2024-04-16T17:01:58.172686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035300+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:01:31.035300+0000","last_peered":"2024-04-16T17:01:31.035300+0000","last_clean":"2024-04-16T17:01:31.035300+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:01:31.035300+0000","last_undegraded":"2024-04-16T17:01:31.035300+0000","last_fullsized":"2024-04-16T17:01:31.035300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018269+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:01:30.018269+0000","last_peered":"2024-04-16T17:01:30.018269+0000","last_clean":"2024-04-16T17:01:30.018269+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:01:30.018269+0000","last_undegraded":"2024-04-16T17:01:30.018269+0000","last_fullsized":"2024-04-16T17:01:30.018269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.222958+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:01:30.222958+0000","last_peered":"2024-04-16T17:01:30.222958+0000","last_clean":"2024-04-16T17:01:30.222958+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:01:30.222958+0000","last_undegraded":"2024-04-16T17:01:30.222958+0000","last_fullsized":"2024-04-16T17:01:30.222958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.386571+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:01:30.386571+0000","last_peered":"2024-04-16T17:01:30.386571+0000","last_clean":"2024-04-16T17:01:30.386571+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:01:30.386571+0000","last_undegraded":"2024-04-16T17:01:30.386571+0000","last_fullsized":"2024-04-16T17:01:30.386571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.036068+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:01:31.036068+0000","last_peered":"2024-04-16T17:01:31.036068+0000","last_clean":"2024-04-16T17:01:31.036068+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:01:31.036068+0000","last_undegraded":"2024-04-16T17:01:31.036068+0000","last_fullsized":"2024-04-16T17:01:31.036068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.158886+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:01:57.158886+0000","last_peered":"2024-04-16T17:01:57.158886+0000","last_clean":"2024-04-16T17:01:57.158886+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:01:57.158886+0000","last_undegraded":"2024-04-16T17:01:57.158886+0000","last_fullsized":"2024-04-16T17:01:57.158886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845789+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:01:30.845789+0000","last_peered":"2024-04-16T17:01:30.845789+0000","last_clean":"2024-04-16T17:01:30.845789+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:01:30.845789+0000","last_undegraded":"2024-04-16T17:01:30.845789+0000","last_fullsized":"2024-04-16T17:01:30.845789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.386690+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:01:30.386690+0000","last_peered":"2024-04-16T17:01:30.386690+0000","last_clean":"2024-04-16T17:01:30.386690+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:01:30.386690+0000","last_undegraded":"2024-04-16T17:01:30.386690+0000","last_fullsized":"2024-04-16T17:01:30.386690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.162007+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:01:57.162007+0000","last_peered":"2024-04-16T17:01:57.162007+0000","last_clean":"2024-04-16T17:01:57.162007+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:01:57.162007+0000","last_undegraded":"2024-04-16T17:01:57.162007+0000","last_fullsized":"2024-04-16T17:01:57.162007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018441+0000","last_change":"2024-04-16T17:01:24.558548+0000","last_active":"2024-04-16T17:01:30.018441+0000","last_peered":"2024-04-16T17:01:30.018441+0000","last_clean":"2024-04-16T17:01:30.018441+0000","last_became_active":"2024-04-16T17:01:24.557329+0000","last_became_peered":"2024-04-16T17:01:24.557329+0000","last_unstale":"2024-04-16T17:01:30.018441+0000","last_undegraded":"2024-04-16T17:01:30.018441+0000","last_fullsized":"2024-04-16T17:01:30.018441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:22.310143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845383+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:01:30.845383+0000","last_peered":"2024-04-16T17:01:30.845383+0000","last_clean":"2024-04-16T17:01:30.845383+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:01:30.845383+0000","last_undegraded":"2024-04-16T17:01:30.845383+0000","last_fullsized":"2024-04-16T17:01:30.845383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.163770+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:01:57.163770+0000","last_peered":"2024-04-16T17:01:57.163770+0000","last_clean":"2024-04-16T17:01:57.163770+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:01:57.163770+0000","last_undegraded":"2024-04-16T17:01:57.163770+0000","last_fullsized":"2024-04-16T17:01:57.163770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845838+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:01:30.845838+0000","last_peered":"2024-04-16T17:01:30.845838+0000","last_clean":"2024-04-16T17:01:30.845838+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:01:30.845838+0000","last_undegraded":"2024-04-16T17:01:30.845838+0000","last_fullsized":"2024-04-16T17:01:30.845838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.387377+0000","last_change":"2024-04-16T17:01:29.025629+0000","last_active":"2024-04-16T17:01:30.387377+0000","last_peered":"2024-04-16T17:01:30.387377+0000","last_clean":"2024-04-16T17:01:30.387377+0000","last_became_active":"2024-04-16T17:01:29.022689+0000","last_became_peered":"2024-04-16T17:01:29.022689+0000","last_unstale":"2024-04-16T17:01:30.387377+0000","last_undegraded":"2024-04-16T17:01:30.387377+0000","last_fullsized":"2024-04-16T17:01:30.387377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:40.917768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.156822+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:01:57.156822+0000","last_peered":"2024-04-16T17:01:57.156822+0000","last_clean":"2024-04-16T17:01:57.156822+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:01:57.156822+0000","last_undegraded":"2024-04-16T17:01:57.156822+0000","last_fullsized":"2024-04-16T17:01:57.156822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512442+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:01:30.512442+0000","last_peered":"2024-04-16T17:01:30.512442+0000","last_clean":"2024-04-16T17:01:30.512442+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:01:30.512442+0000","last_undegraded":"2024-04-16T17:01:30.512442+0000","last_fullsized":"2024-04-16T17:01:30.512442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223784+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:01:30.223784+0000","last_peered":"2024-04-16T17:01:30.223784+0000","last_clean":"2024-04-16T17:01:30.223784+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:01:30.223784+0000","last_undegraded":"2024-04-16T17:01:30.223784+0000","last_fullsized":"2024-04-16T17:01:30.223784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.167691+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:01:58.167691+0000","last_peered":"2024-04-16T17:01:58.167691+0000","last_clean":"2024-04-16T17:01:58.167691+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:01:58.167691+0000","last_undegraded":"2024-04-16T17:01:58.167691+0000","last_fullsized":"2024-04-16T17:01:58.167691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.510951+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:01:30.510951+0000","last_peered":"2024-04-16T17:01:30.510951+0000","last_clean":"2024-04-16T17:01:30.510951+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:01:30.510951+0000","last_undegraded":"2024-04-16T17:01:30.510951+0000","last_fullsized":"2024-04-16T17:01:30.510951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511892+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:01:30.511892+0000","last_peered":"2024-04-16T17:01:30.511892+0000","last_clean":"2024-04-16T17:01:30.511892+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:01:30.511892+0000","last_undegraded":"2024-04-16T17:01:30.511892+0000","last_fullsized":"2024-04-16T17:01:30.511892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.163365+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:01:57.163365+0000","last_peered":"2024-04-16T17:01:57.163365+0000","last_clean":"2024-04-16T17:01:57.163365+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:01:57.163365+0000","last_undegraded":"2024-04-16T17:01:57.163365+0000","last_fullsized":"2024-04-16T17:01:57.163365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014786+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:01:30.014786+0000","last_peered":"2024-04-16T17:01:30.014786+0000","last_clean":"2024-04-16T17:01:30.014786+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:01:30.014786+0000","last_undegraded":"2024-04-16T17:01:30.014786+0000","last_fullsized":"2024-04-16T17:01:30.014786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.387804+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:01:30.387804+0000","last_peered":"2024-04-16T17:01:30.387804+0000","last_clean":"2024-04-16T17:01:30.387804+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:01:30.387804+0000","last_undegraded":"2024-04-16T17:01:30.387804+0000","last_fullsized":"2024-04-16T17:01:30.387804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:02:01.182031+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:02:01.182031+0000","last_peered":"2024-04-16T17:02:01.182031+0000","last_clean":"2024-04-16T17:02:01.182031+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:02:01.182031+0000","last_undegraded":"2024-04-16T17:02:01.182031+0000","last_fullsized":"2024-04-16T17:02:01.182031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014621+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:01:30.014621+0000","last_peered":"2024-04-16T17:01:30.014621+0000","last_clean":"2024-04-16T17:01:30.014621+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:01:30.014621+0000","last_undegraded":"2024-04-16T17:01:30.014621+0000","last_fullsized":"2024-04-16T17:01:30.014621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512347+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:01:30.512347+0000","last_peered":"2024-04-16T17:01:30.512347+0000","last_clean":"2024-04-16T17:01:30.512347+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:01:30.512347+0000","last_undegraded":"2024-04-16T17:01:30.512347+0000","last_fullsized":"2024-04-16T17:01:30.512347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.172936+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:01:58.172936+0000","last_peered":"2024-04-16T17:01:58.172936+0000","last_clean":"2024-04-16T17:01:58.172936+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:01:58.172936+0000","last_undegraded":"2024-04-16T17:01:58.172936+0000","last_fullsized":"2024-04-16T17:01:58.172936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223012+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:01:30.223012+0000","last_peered":"2024-04-16T17:01:30.223012+0000","last_clean":"2024-04-16T17:01:30.223012+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:01:30.223012+0000","last_undegraded":"2024-04-16T17:01:30.223012+0000","last_fullsized":"2024-04-16T17:01:30.223012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223244+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:01:30.223244+0000","last_peered":"2024-04-16T17:01:30.223244+0000","last_clean":"2024-04-16T17:01:30.223244+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:01:30.223244+0000","last_undegraded":"2024-04-16T17:01:30.223244+0000","last_fullsized":"2024-04-16T17:01:30.223244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:59.173061+0000","last_change":"2024-04-16T17:01:23.910149+0000","last_active":"2024-04-16T17:01:59.173061+0000","last_peered":"2024-04-16T17:01:59.173061+0000","last_clean":"2024-04-16T17:01:59.173061+0000","last_became_active":"2024-04-16T17:01:23.909796+0000","last_became_peered":"2024-04-16T17:01:23.909796+0000","last_unstale":"2024-04-16T17:01:59.173061+0000","last_undegraded":"2024-04-16T17:01:59.173061+0000","last_fullsized":"2024-04-16T17:01:59.173061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:38.574698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512394+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:01:30.512394+0000","last_peered":"2024-04-16T17:01:30.512394+0000","last_clean":"2024-04-16T17:01:30.512394+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:01:30.512394+0000","last_undegraded":"2024-04-16T17:01:30.512394+0000","last_fullsized":"2024-04-16T17:01:30.512394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512723+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:01:30.512723+0000","last_peered":"2024-04-16T17:01:30.512723+0000","last_clean":"2024-04-16T17:01:30.512723+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:01:30.512723+0000","last_undegraded":"2024-04-16T17:01:30.512723+0000","last_fullsized":"2024-04-16T17:01:30.512723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.386901+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:01:30.386901+0000","last_peered":"2024-04-16T17:01:30.386901+0000","last_clean":"2024-04-16T17:01:30.386901+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:01:30.386901+0000","last_undegraded":"2024-04-16T17:01:30.386901+0000","last_fullsized":"2024-04-16T17:01:30.386901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512281+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:01:30.512281+0000","last_peered":"2024-04-16T17:01:30.512281+0000","last_clean":"2024-04-16T17:01:30.512281+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:01:30.512281+0000","last_undegraded":"2024-04-16T17:01:30.512281+0000","last_fullsized":"2024-04-16T17:01:30.512281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511442+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:01:30.511442+0000","last_peered":"2024-04-16T17:01:30.511442+0000","last_clean":"2024-04-16T17:01:30.511442+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:01:30.511442+0000","last_undegraded":"2024-04-16T17:01:30.511442+0000","last_fullsized":"2024-04-16T17:01:30.511442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014625+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:01:30.014625+0000","last_peered":"2024-04-16T17:01:30.014625+0000","last_clean":"2024-04-16T17:01:30.014625+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:01:30.014625+0000","last_undegraded":"2024-04-16T17:01:30.014625+0000","last_fullsized":"2024-04-16T17:01:30.014625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512037+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:01:30.512037+0000","last_peered":"2024-04-16T17:01:30.512037+0000","last_clean":"2024-04-16T17:01:30.512037+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:01:30.512037+0000","last_undegraded":"2024-04-16T17:01:30.512037+0000","last_fullsized":"2024-04-16T17:01:30.512037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512391+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:01:30.512391+0000","last_peered":"2024-04-16T17:01:30.512391+0000","last_clean":"2024-04-16T17:01:30.512391+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:01:30.512391+0000","last_undegraded":"2024-04-16T17:01:30.512391+0000","last_fullsized":"2024-04-16T17:01:30.512391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223033+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:01:30.223033+0000","last_peered":"2024-04-16T17:01:30.223033+0000","last_clean":"2024-04-16T17:01:30.223033+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:01:30.223033+0000","last_undegraded":"2024-04-16T17:01:30.223033+0000","last_fullsized":"2024-04-16T17:01:30.223033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:59.176751+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:01:59.176751+0000","last_peered":"2024-04-16T17:01:59.176751+0000","last_clean":"2024-04-16T17:01:59.176751+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:01:59.176751+0000","last_undegraded":"2024-04-16T17:01:59.176751+0000","last_fullsized":"2024-04-16T17:01:59.176751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.038230+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:01:31.038230+0000","last_peered":"2024-04-16T17:01:31.038230+0000","last_clean":"2024-04-16T17:01:31.038230+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:01:31.038230+0000","last_undegraded":"2024-04-16T17:01:31.038230+0000","last_fullsized":"2024-04-16T17:01:31.038230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035715+0000","last_change":"2024-04-16T17:01:24.549236+0000","last_active":"2024-04-16T17:01:31.035715+0000","last_peered":"2024-04-16T17:01:31.035715+0000","last_clean":"2024-04-16T17:01:31.035715+0000","last_became_active":"2024-04-16T17:01:24.548850+0000","last_became_peered":"2024-04-16T17:01:24.548850+0000","last_unstale":"2024-04-16T17:01:31.035715+0000","last_undegraded":"2024-04-16T17:01:31.035715+0000","last_fullsized":"2024-04-16T17:01:31.035715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:00.312829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:59.174058+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:01:59.174058+0000","last_peered":"2024-04-16T17:01:59.174058+0000","last_clean":"2024-04-16T17:01:59.174058+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:01:59.174058+0000","last_undegraded":"2024-04-16T17:01:59.174058+0000","last_fullsized":"2024-04-16T17:01:59.174058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014235+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:01:30.014235+0000","last_peered":"2024-04-16T17:01:30.014235+0000","last_clean":"2024-04-16T17:01:30.014235+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:01:30.014235+0000","last_undegraded":"2024-04-16T17:01:30.014235+0000","last_fullsized":"2024-04-16T17:01:30.014235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223640+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:01:30.223640+0000","last_peered":"2024-04-16T17:01:30.223640+0000","last_clean":"2024-04-16T17:01:30.223640+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:01:30.223640+0000","last_undegraded":"2024-04-16T17:01:30.223640+0000","last_fullsized":"2024-04-16T17:01:30.223640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.510360+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:01:30.510360+0000","last_peered":"2024-04-16T17:01:30.510360+0000","last_clean":"2024-04-16T17:01:30.510360+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:01:30.510360+0000","last_undegraded":"2024-04-16T17:01:30.510360+0000","last_fullsized":"2024-04-16T17:01:30.510360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.165151+0000","last_change":"2024-04-16T17:01:23.913714+0000","last_active":"2024-04-16T17:01:58.165151+0000","last_peered":"2024-04-16T17:01:58.165151+0000","last_clean":"2024-04-16T17:01:58.165151+0000","last_became_active":"2024-04-16T17:01:23.913246+0000","last_became_peered":"2024-04-16T17:01:23.913246+0000","last_unstale":"2024-04-16T17:01:58.165151+0000","last_undegraded":"2024-04-16T17:01:58.165151+0000","last_fullsized":"2024-04-16T17:01:58.165151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:14.850863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512027+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:01:30.512027+0000","last_peered":"2024-04-16T17:01:30.512027+0000","last_clean":"2024-04-16T17:01:30.512027+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:01:30.512027+0000","last_undegraded":"2024-04-16T17:01:30.512027+0000","last_fullsized":"2024-04-16T17:01:30.512027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.164407+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:01:57.164407+0000","last_peered":"2024-04-16T17:01:57.164407+0000","last_clean":"2024-04-16T17:01:57.164407+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:01:57.164407+0000","last_undegraded":"2024-04-16T17:01:57.164407+0000","last_fullsized":"2024-04-16T17:01:57.164407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512061+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:01:30.512061+0000","last_peered":"2024-04-16T17:01:30.512061+0000","last_clean":"2024-04-16T17:01:30.512061+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:01:30.512061+0000","last_undegraded":"2024-04-16T17:01:30.512061+0000","last_fullsized":"2024-04-16T17:01:30.512061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035272+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:01:31.035272+0000","last_peered":"2024-04-16T17:01:31.035272+0000","last_clean":"2024-04-16T17:01:31.035272+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:01:31.035272+0000","last_undegraded":"2024-04-16T17:01:31.035272+0000","last_fullsized":"2024-04-16T17:01:31.035272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:02:00.944881+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:02:00.944881+0000","last_peered":"2024-04-16T17:02:00.944881+0000","last_clean":"2024-04-16T17:02:00.944881+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:02:00.944881+0000","last_undegraded":"2024-04-16T17:02:00.944881+0000","last_fullsized":"2024-04-16T17:02:00.944881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.036780+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:01:31.036780+0000","last_peered":"2024-04-16T17:01:31.036780+0000","last_clean":"2024-04-16T17:01:31.036780+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:01:31.036780+0000","last_undegraded":"2024-04-16T17:01:31.036780+0000","last_fullsized":"2024-04-16T17:01:31.036780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845008+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:01:30.845008+0000","last_peered":"2024-04-16T17:01:30.845008+0000","last_clean":"2024-04-16T17:01:30.845008+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:01:30.845008+0000","last_undegraded":"2024-04-16T17:01:30.845008+0000","last_fullsized":"2024-04-16T17:01:30.845008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.157922+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:01:57.157922+0000","last_peered":"2024-04-16T17:01:57.157922+0000","last_clean":"2024-04-16T17:01:57.157922+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:01:57.157922+0000","last_undegraded":"2024-04-16T17:01:57.157922+0000","last_fullsized":"2024-04-16T17:01:57.157922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018477+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:01:30.018477+0000","last_peered":"2024-04-16T17:01:30.018477+0000","last_clean":"2024-04-16T17:01:30.018477+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:01:30.018477+0000","last_undegraded":"2024-04-16T17:01:30.018477+0000","last_fullsized":"2024-04-16T17:01:30.018477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845600+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:01:30.845600+0000","last_peered":"2024-04-16T17:01:30.845600+0000","last_clean":"2024-04-16T17:01:30.845600+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:01:30.845600+0000","last_undegraded":"2024-04-16T17:01:30.845600+0000","last_fullsized":"2024-04-16T17:01:30.845600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.163276+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:01:57.163276+0000","last_peered":"2024-04-16T17:01:57.163276+0000","last_clean":"2024-04-16T17:01:57.163276+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:01:57.163276+0000","last_undegraded":"2024-04-16T17:01:57.163276+0000","last_fullsized":"2024-04-16T17:01:57.163276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511662+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:01:30.511662+0000","last_peered":"2024-04-16T17:01:30.511662+0000","last_clean":"2024-04-16T17:01:30.511662+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:01:30.511662+0000","last_undegraded":"2024-04-16T17:01:30.511662+0000","last_fullsized":"2024-04-16T17:01:30.511662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.388105+0000","last_change":"2024-04-16T17:01:29.035782+0000","last_active":"2024-04-16T17:01:30.388105+0000","last_peered":"2024-04-16T17:01:30.388105+0000","last_clean":"2024-04-16T17:01:30.388105+0000","last_became_active":"2024-04-16T17:01:29.034812+0000","last_became_peered":"2024-04-16T17:01:29.034812+0000","last_unstale":"2024-04-16T17:01:30.388105+0000","last_undegraded":"2024-04-16T17:01:30.388105+0000","last_fullsized":"2024-04-16T17:01:30.388105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:27.655802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.166034+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:01:57.166034+0000","last_peered":"2024-04-16T17:01:57.166034+0000","last_clean":"2024-04-16T17:01:57.166034+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:01:57.166034+0000","last_undegraded":"2024-04-16T17:01:57.166034+0000","last_fullsized":"2024-04-16T17:01:57.166034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511763+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:01:30.511763+0000","last_peered":"2024-04-16T17:01:30.511763+0000","last_clean":"2024-04-16T17:01:30.511763+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:01:30.511763+0000","last_undegraded":"2024-04-16T17:01:30.511763+0000","last_fullsized":"2024-04-16T17:01:30.511763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:32.398949+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:01:32.398949+0000","last_peered":"2024-04-16T17:01:32.398949+0000","last_clean":"2024-04-16T17:01:32.398949+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:01:32.398949+0000","last_undegraded":"2024-04-16T17:01:32.398949+0000","last_fullsized":"2024-04-16T17:01:32.398949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845209+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:01:30.845209+0000","last_peered":"2024-04-16T17:01:30.845209+0000","last_clean":"2024-04-16T17:01:30.845209+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:01:30.845209+0000","last_undegraded":"2024-04-16T17:01:30.845209+0000","last_fullsized":"2024-04-16T17:01:30.845209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.036302+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:01:31.036302+0000","last_peered":"2024-04-16T17:01:31.036302+0000","last_clean":"2024-04-16T17:01:31.036302+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:01:31.036302+0000","last_undegraded":"2024-04-16T17:01:31.036302+0000","last_fullsized":"2024-04-16T17:01:31.036302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845642+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:01:30.845642+0000","last_peered":"2024-04-16T17:01:30.845642+0000","last_clean":"2024-04-16T17:01:30.845642+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:01:30.845642+0000","last_undegraded":"2024-04-16T17:01:30.845642+0000","last_fullsized":"2024-04-16T17:01:30.845642+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.161528+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:01:57.161528+0000","last_peered":"2024-04-16T17:01:57.161528+0000","last_clean":"2024-04-16T17:01:57.161528+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:01:57.161528+0000","last_undegraded":"2024-04-16T17:01:57.161528+0000","last_fullsized":"2024-04-16T17:01:57.161528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014777+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:01:30.014777+0000","last_peered":"2024-04-16T17:01:30.014777+0000","last_clean":"2024-04-16T17:01:30.014777+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:01:30.014777+0000","last_undegraded":"2024-04-16T17:01:30.014777+0000","last_fullsized":"2024-04-16T17:01:30.014777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.844835+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:01:30.844835+0000","last_peered":"2024-04-16T17:01:30.844835+0000","last_clean":"2024-04-16T17:01:30.844835+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:01:30.844835+0000","last_undegraded":"2024-04-16T17:01:30.844835+0000","last_fullsized":"2024-04-16T17:01:30.844835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.172288+0000","last_change":"2024-04-16T17:01:23.926672+0000","last_active":"2024-04-16T17:01:58.172288+0000","last_peered":"2024-04-16T17:01:58.172288+0000","last_clean":"2024-04-16T17:01:58.172288+0000","last_became_active":"2024-04-16T17:01:23.926330+0000","last_became_peered":"2024-04-16T17:01:23.926330+0000","last_unstale":"2024-04-16T17:01:58.172288+0000","last_undegraded":"2024-04-16T17:01:58.172288+0000","last_fullsized":"2024-04-16T17:01:58.172288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:30:23.951003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.015072+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:01:30.015072+0000","last_peered":"2024-04-16T17:01:30.015072+0000","last_clean":"2024-04-16T17:01:30.015072+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:01:30.015072+0000","last_undegraded":"2024-04-16T17:01:30.015072+0000","last_fullsized":"2024-04-16T17:01:30.015072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.017007+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:01:30.017007+0000","last_peered":"2024-04-16T17:01:30.017007+0000","last_clean":"2024-04-16T17:01:30.017007+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:01:30.017007+0000","last_undegraded":"2024-04-16T17:01:30.017007+0000","last_fullsized":"2024-04-16T17:01:30.017007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223551+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:01:30.223551+0000","last_peered":"2024-04-16T17:01:30.223551+0000","last_clean":"2024-04-16T17:01:30.223551+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:01:30.223551+0000","last_undegraded":"2024-04-16T17:01:30.223551+0000","last_fullsized":"2024-04-16T17:01:30.223551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014654+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:01:30.014654+0000","last_peered":"2024-04-16T17:01:30.014654+0000","last_clean":"2024-04-16T17:01:30.014654+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:01:30.014654+0000","last_undegraded":"2024-04-16T17:01:30.014654+0000","last_fullsized":"2024-04-16T17:01:30.014654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.222813+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:01:30.222813+0000","last_peered":"2024-04-16T17:01:30.222813+0000","last_clean":"2024-04-16T17:01:30.222813+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:01:30.222813+0000","last_undegraded":"2024-04-16T17:01:30.222813+0000","last_fullsized":"2024-04-16T17:01:30.222813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511894+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:01:30.511894+0000","last_peered":"2024-04-16T17:01:30.511894+0000","last_clean":"2024-04-16T17:01:30.511894+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:01:30.511894+0000","last_undegraded":"2024-04-16T17:01:30.511894+0000","last_fullsized":"2024-04-16T17:01:30.511894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.015062+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:01:30.015062+0000","last_peered":"2024-04-16T17:01:30.015062+0000","last_clean":"2024-04-16T17:01:30.015062+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:01:30.015062+0000","last_undegraded":"2024-04-16T17:01:30.015062+0000","last_fullsized":"2024-04-16T17:01:30.015062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511913+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:01:30.511913+0000","last_peered":"2024-04-16T17:01:30.511913+0000","last_clean":"2024-04-16T17:01:30.511913+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:01:30.511913+0000","last_undegraded":"2024-04-16T17:01:30.511913+0000","last_fullsized":"2024-04-16T17:01:30.511913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845591+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:01:30.845591+0000","last_peered":"2024-04-16T17:01:30.845591+0000","last_clean":"2024-04-16T17:01:30.845591+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:01:30.845591+0000","last_undegraded":"2024-04-16T17:01:30.845591+0000","last_fullsized":"2024-04-16T17:01:30.845591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845037+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:01:30.845037+0000","last_peered":"2024-04-16T17:01:30.845037+0000","last_clean":"2024-04-16T17:01:30.845037+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:01:30.845037+0000","last_undegraded":"2024-04-16T17:01:30.845037+0000","last_fullsized":"2024-04-16T17:01:30.845037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.017247+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:01:30.017247+0000","last_peered":"2024-04-16T17:01:30.017247+0000","last_clean":"2024-04-16T17:01:30.017247+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:01:30.017247+0000","last_undegraded":"2024-04-16T17:01:30.017247+0000","last_fullsized":"2024-04-16T17:01:30.017247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511783+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:01:30.511783+0000","last_peered":"2024-04-16T17:01:30.511783+0000","last_clean":"2024-04-16T17:01:30.511783+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:01:30.511783+0000","last_undegraded":"2024-04-16T17:01:30.511783+0000","last_fullsized":"2024-04-16T17:01:30.511783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512403+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:01:30.512403+0000","last_peered":"2024-04-16T17:01:30.512403+0000","last_clean":"2024-04-16T17:01:30.512403+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:01:30.512403+0000","last_undegraded":"2024-04-16T17:01:30.512403+0000","last_fullsized":"2024-04-16T17:01:30.512403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845692+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:01:30.845692+0000","last_peered":"2024-04-16T17:01:30.845692+0000","last_clean":"2024-04-16T17:01:30.845692+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:01:30.845692+0000","last_undegraded":"2024-04-16T17:01:30.845692+0000","last_fullsized":"2024-04-16T17:01:30.845692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.037072+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:01:31.037072+0000","last_peered":"2024-04-16T17:01:31.037072+0000","last_clean":"2024-04-16T17:01:31.037072+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:01:31.037072+0000","last_undegraded":"2024-04-16T17:01:31.037072+0000","last_fullsized":"2024-04-16T17:01:31.037072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.222984+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:01:30.222984+0000","last_peered":"2024-04-16T17:01:30.222984+0000","last_clean":"2024-04-16T17:01:30.222984+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:01:30.222984+0000","last_undegraded":"2024-04-16T17:01:30.222984+0000","last_fullsized":"2024-04-16T17:01:30.222984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035591+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:01:31.035591+0000","last_peered":"2024-04-16T17:01:31.035591+0000","last_clean":"2024-04-16T17:01:31.035591+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:01:31.035591+0000","last_undegraded":"2024-04-16T17:01:31.035591+0000","last_fullsized":"2024-04-16T17:01:31.035591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845257+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:01:30.845257+0000","last_peered":"2024-04-16T17:01:30.845257+0000","last_clean":"2024-04-16T17:01:30.845257+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:01:30.845257+0000","last_undegraded":"2024-04-16T17:01:30.845257+0000","last_fullsized":"2024-04-16T17:01:30.845257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511279+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:01:30.511279+0000","last_peered":"2024-04-16T17:01:30.511279+0000","last_clean":"2024-04-16T17:01:30.511279+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:01:30.511279+0000","last_undegraded":"2024-04-16T17:01:30.511279+0000","last_fullsized":"2024-04-16T17:01:30.511279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511971+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:01:30.511971+0000","last_peered":"2024-04-16T17:01:30.511971+0000","last_clean":"2024-04-16T17:01:30.511971+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:01:30.511971+0000","last_undegraded":"2024-04-16T17:01:30.511971+0000","last_fullsized":"2024-04-16T17:01:30.511971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245319,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":132,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":21734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383700,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":477797,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645338,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":1072381,"data_compressed":50228,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808865,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":486225,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070502,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":482049,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332139,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1073471,"data_compressed":50228,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":477305,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481057,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":478693,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":481713,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":477499,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":481955,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":1073155,"data_compressed":50228,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:02:03.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:02:03.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:03 smithi018 ceph-mon[31776]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:03.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:03 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1862170702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:03.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:03 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1144154775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:03.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:03 smithi043 ceph-mon[36901]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:03.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:03 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1862170702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:03.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:03 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1144154775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:03.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.567+0000 7f2182400640 1 -- 172.21.15.18:0/945093997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c109740 msgr2=0x7f217c109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.567+0000 7f2182400640 1 --2- 172.21.15.18:0/945093997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c109740 0x7f217c109bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f216c009900 tx=0x7f216c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:02:03.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.567+0000 7f2182400640 1 -- 172.21.15.18:0/945093997 shutdown_connections 2024-04-16T17:02:03.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.568+0000 7f2182400640 1 --2- 172.21.15.18:0/945093997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f217c10a100 0x7f217c110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.568+0000 7f2182400640 1 --2- 172.21.15.18:0/945093997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c109740 0x7f217c109bc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.568+0000 7f2182400640 1 --2- 172.21.15.18:0/945093997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c1085a0 0x7f217c1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.568+0000 7f2182400640 1 -- 172.21.15.18:0/945093997 >> 172.21.15.18:0/945093997 conn(0x7f217c0774f0 msgr2=0x7f217c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:03.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.568+0000 7f2182400640 1 -- 172.21.15.18:0/945093997 shutdown_connections 2024-04-16T17:02:03.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.568+0000 7f2182400640 1 -- 172.21.15.18:0/945093997 wait complete. 2024-04-16T17:02:03.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2182400640 1 Processor -- start 2024-04-16T17:02:03.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2182400640 1 -- start start 2024-04-16T17:02:03.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2182400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f217c1085a0 0x7f217c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2182400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109740 0x7f217c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2182400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c10a100 0x7f217c1a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2182400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f217c114230 con 0x7f217c109740 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2182400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1140b0 con 0x7f217c1085a0 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2182400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1143b0 con 0x7f217c10a100 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2173e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109740 0x7f217c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2181000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f217c1085a0 0x7f217c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2173e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109740 0x7f217c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54406/0 (socket says 172.21.15.18:54406) 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.569+0000 7f2173e00640 1 -- 172.21.15.18:0/3905283864 learned_addr learned my addr 172.21.15.18:0/3905283864 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f2181a00640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c10a100 0x7f217c1a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f2181000640 1 -- 172.21.15.18:0/3905283864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c10a100 msgr2=0x7f217c1a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f2181000640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c10a100 0x7f217c1a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f2181000640 1 -- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109740 msgr2=0x7f217c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f2181000640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109740 0x7f217c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f2181000640 1 -- 172.21.15.18:0/3905283864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c1a6440 con 0x7f217c1085a0 2024-04-16T17:02:03.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f2181a00640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c10a100 0x7f217c1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:02:03.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f2181000640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f217c1085a0 0x7f217c19e820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f216400e9c0 tx=0x7f216400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:03.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.570+0000 7f217a000640 1 -- 172.21.15.18:0/3905283864 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216400ccc0 con 0x7f217c1085a0 2024-04-16T17:02:03.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.571+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c1a6730 con 0x7f217c1085a0 2024-04-16T17:02:03.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.571+0000 7f217a000640 1 -- 172.21.15.18:0/3905283864 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21640035f0 con 0x7f217c1085a0 2024-04-16T17:02:03.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.571+0000 7f217a000640 1 -- 172.21.15.18:0/3905283864 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216400ce60 con 0x7f217c1085a0 2024-04-16T17:02:03.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.571+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c1a6ca0 con 0x7f217c1085a0 2024-04-16T17:02:03.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.572+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f217c10f810 con 0x7f217c1085a0 2024-04-16T17:02:03.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.574+0000 7f217a000640 1 -- 172.21.15.18:0/3905283864 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f21640037a0 con 0x7f217c1085a0 2024-04-16T17:02:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.575+0000 7f217a000640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f21600a3c30 0x7f21600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.575+0000 7f217a000640 1 -- 172.21.15.18:0/3905283864 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f21640d0e90 con 0x7f217c1085a0 2024-04-16T17:02:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.575+0000 7f2173e00640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f21600a3c30 0x7f21600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.576+0000 7f2173e00640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f21600a3c30 0x7f21600a60f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f216c02c8e0 tx=0x7f216c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.578+0000 7f217a000640 1 -- 172.21.15.18:0/3905283864 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f216409d470 con 0x7f217c1085a0 2024-04-16T17:02:03.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:03 smithi022 ceph-mon[36023]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:03.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:03 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1862170702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:03.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:03 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1144154775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:03.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.781+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f217c062440 con 0x7f21600a3c30 2024-04-16T17:02:03.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.796+0000 7f217a000640 1 -- 172.21.15.18:0/3905283864 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340623 (secure 0 0 0) 0x7f217c062440 con 0x7f21600a3c30 2024-04-16T17:02:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:02:03.805 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f21600a3c30 msgr2=0x7f21600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f21600a3c30 0x7f21600a60f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f216c02c8e0 tx=0x7f216c02df90 comp rx=0 tx=0).stop 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f217c1085a0 msgr2=0x7f217c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f217c1085a0 0x7f217c19e820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f216400e9c0 tx=0x7f216400ee80 comp rx=0 tx=0).stop 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 shutdown_connections 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217c10a100 0x7f217c1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f21600a3c30 0x7f21600a60f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109740 0x7f217c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 --2- 172.21.15.18:0/3905283864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f217c1085a0 0x7f217c19e820 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:03.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 >> 172.21.15.18:0/3905283864 conn(0x7f217c0774f0 msgr2=0x7f217c10c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:03.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 shutdown_connections 2024-04-16T17:02:03.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:03.805+0000 7f2182400640 1 -- 172.21.15.18:0/3905283864 wait complete. 2024-04-16T17:02:03.828 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-04-16T17:02:02.890752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598054,"num_objects":63,"num_object_clones":0,"num_object_copies":191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":162,"num_read_kb":137,"num_write":315,"num_write_kb":2986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373300,"kb_used_data":11668,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568524,"statfs":{"total":1151940427776,"available":1151558168576,"internally_reserved":0,"allocated":11948032,"data_stored":7543300,"data_compressed":283101,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3046,"num_objects":39,"num_object_clones":0,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":85,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006517"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512591+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:01:30.512591+0000","last_peered":"2024-04-16T17:01:30.512591+0000","last_clean":"2024-04-16T17:01:30.512591+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:01:30.512591+0000","last_undegraded":"2024-04-16T17:01:30.512591+0000","last_fullsized":"2024-04-16T17:01:30.512591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018183+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:01:30.018183+0000","last_peered":"2024-04-16T17:01:30.018183+0000","last_clean":"2024-04-16T17:01:30.018183+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:01:30.018183+0000","last_undegraded":"2024-04-16T17:01:30.018183+0000","last_fullsized":"2024-04-16T17:01:30.018183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.017615+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:01:30.017615+0000","last_peered":"2024-04-16T17:01:30.017615+0000","last_clean":"2024-04-16T17:01:30.017615+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:01:30.017615+0000","last_undegraded":"2024-04-16T17:01:30.017615+0000","last_fullsized":"2024-04-16T17:01:30.017615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845829+0000","last_change":"2024-04-16T17:01:24.537712+0000","last_active":"2024-04-16T17:01:30.845829+0000","last_peered":"2024-04-16T17:01:30.845829+0000","last_clean":"2024-04-16T17:01:30.845829+0000","last_became_active":"2024-04-16T17:01:24.537350+0000","last_became_peered":"2024-04-16T17:01:24.537350+0000","last_unstale":"2024-04-16T17:01:30.845829+0000","last_undegraded":"2024-04-16T17:01:30.845829+0000","last_fullsized":"2024-04-16T17:01:30.845829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:05.792403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511931+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:01:30.511931+0000","last_peered":"2024-04-16T17:01:30.511931+0000","last_clean":"2024-04-16T17:01:30.511931+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:01:30.511931+0000","last_undegraded":"2024-04-16T17:01:30.511931+0000","last_fullsized":"2024-04-16T17:01:30.511931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512250+0000","last_change":"2024-04-16T17:01:24.543268+0000","last_active":"2024-04-16T17:01:30.512250+0000","last_peered":"2024-04-16T17:01:30.512250+0000","last_clean":"2024-04-16T17:01:30.512250+0000","last_became_active":"2024-04-16T17:01:24.542926+0000","last_became_peered":"2024-04-16T17:01:24.542926+0000","last_unstale":"2024-04-16T17:01:30.512250+0000","last_undegraded":"2024-04-16T17:01:30.512250+0000","last_fullsized":"2024-04-16T17:01:30.512250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:03.105040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014291+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:01:30.014291+0000","last_peered":"2024-04-16T17:01:30.014291+0000","last_clean":"2024-04-16T17:01:30.014291+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:01:30.014291+0000","last_undegraded":"2024-04-16T17:01:30.014291+0000","last_fullsized":"2024-04-16T17:01:30.014291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035406+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:01:31.035406+0000","last_peered":"2024-04-16T17:01:31.035406+0000","last_clean":"2024-04-16T17:01:31.035406+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:01:31.035406+0000","last_undegraded":"2024-04-16T17:01:31.035406+0000","last_fullsized":"2024-04-16T17:01:31.035406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845490+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:01:30.845490+0000","last_peered":"2024-04-16T17:01:30.845490+0000","last_clean":"2024-04-16T17:01:30.845490+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:01:30.845490+0000","last_undegraded":"2024-04-16T17:01:30.845490+0000","last_fullsized":"2024-04-16T17:01:30.845490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512388+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:01:30.512388+0000","last_peered":"2024-04-16T17:01:30.512388+0000","last_clean":"2024-04-16T17:01:30.512388+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:01:30.512388+0000","last_undegraded":"2024-04-16T17:01:30.512388+0000","last_fullsized":"2024-04-16T17:01:30.512388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014984+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:01:30.014984+0000","last_peered":"2024-04-16T17:01:30.014984+0000","last_clean":"2024-04-16T17:01:30.014984+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:01:30.014984+0000","last_undegraded":"2024-04-16T17:01:30.014984+0000","last_fullsized":"2024-04-16T17:01:30.014984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845914+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:01:30.845914+0000","last_peered":"2024-04-16T17:01:30.845914+0000","last_clean":"2024-04-16T17:01:30.845914+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:01:30.845914+0000","last_undegraded":"2024-04-16T17:01:30.845914+0000","last_fullsized":"2024-04-16T17:01:30.845914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511578+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:01:30.511578+0000","last_peered":"2024-04-16T17:01:30.511578+0000","last_clean":"2024-04-16T17:01:30.511578+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:01:30.511578+0000","last_undegraded":"2024-04-16T17:01:30.511578+0000","last_fullsized":"2024-04-16T17:01:30.511578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.016618+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:01:30.016618+0000","last_peered":"2024-04-16T17:01:30.016618+0000","last_clean":"2024-04-16T17:01:30.016618+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:01:30.016618+0000","last_undegraded":"2024-04-16T17:01:30.016618+0000","last_fullsized":"2024-04-16T17:01:30.016618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.016502+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:01:30.016502+0000","last_peered":"2024-04-16T17:01:30.016502+0000","last_clean":"2024-04-16T17:01:30.016502+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:01:30.016502+0000","last_undegraded":"2024-04-16T17:01:30.016502+0000","last_fullsized":"2024-04-16T17:01:30.016502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.844658+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:01:30.844658+0000","last_peered":"2024-04-16T17:01:30.844658+0000","last_clean":"2024-04-16T17:01:30.844658+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:01:30.844658+0000","last_undegraded":"2024-04-16T17:01:30.844658+0000","last_fullsized":"2024-04-16T17:01:30.844658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512525+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:01:30.512525+0000","last_peered":"2024-04-16T17:01:30.512525+0000","last_clean":"2024-04-16T17:01:30.512525+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:01:30.512525+0000","last_undegraded":"2024-04-16T17:01:30.512525+0000","last_fullsized":"2024-04-16T17:01:30.512525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.387732+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:01:30.387732+0000","last_peered":"2024-04-16T17:01:30.387732+0000","last_clean":"2024-04-16T17:01:30.387732+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:01:30.387732+0000","last_undegraded":"2024-04-16T17:01:30.387732+0000","last_fullsized":"2024-04-16T17:01:30.387732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845571+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:01:30.845571+0000","last_peered":"2024-04-16T17:01:30.845571+0000","last_clean":"2024-04-16T17:01:30.845571+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:01:30.845571+0000","last_undegraded":"2024-04-16T17:01:30.845571+0000","last_fullsized":"2024-04-16T17:01:30.845571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845339+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:01:30.845339+0000","last_peered":"2024-04-16T17:01:30.845339+0000","last_clean":"2024-04-16T17:01:30.845339+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:01:30.845339+0000","last_undegraded":"2024-04-16T17:01:30.845339+0000","last_fullsized":"2024-04-16T17:01:30.845339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.172920+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:01:58.172920+0000","last_peered":"2024-04-16T17:01:58.172920+0000","last_clean":"2024-04-16T17:01:58.172920+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:01:58.172920+0000","last_undegraded":"2024-04-16T17:01:58.172920+0000","last_fullsized":"2024-04-16T17:01:58.172920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.036927+0000","last_change":"2024-04-16T17:01:24.562184+0000","last_active":"2024-04-16T17:01:31.036927+0000","last_peered":"2024-04-16T17:01:31.036927+0000","last_clean":"2024-04-16T17:01:31.036927+0000","last_became_active":"2024-04-16T17:01:24.558485+0000","last_became_peered":"2024-04-16T17:01:24.558485+0000","last_unstale":"2024-04-16T17:01:31.036927+0000","last_undegraded":"2024-04-16T17:01:31.036927+0000","last_fullsized":"2024-04-16T17:01:31.036927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:09.994752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845150+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:01:30.845150+0000","last_peered":"2024-04-16T17:01:30.845150+0000","last_clean":"2024-04-16T17:01:30.845150+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:01:30.845150+0000","last_undegraded":"2024-04-16T17:01:30.845150+0000","last_fullsized":"2024-04-16T17:01:30.845150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.037254+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:01:31.037254+0000","last_peered":"2024-04-16T17:01:31.037254+0000","last_clean":"2024-04-16T17:01:31.037254+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:01:31.037254+0000","last_undegraded":"2024-04-16T17:01:31.037254+0000","last_fullsized":"2024-04-16T17:01:31.037254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018417+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:01:30.018417+0000","last_peered":"2024-04-16T17:01:30.018417+0000","last_clean":"2024-04-16T17:01:30.018417+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:01:30.018417+0000","last_undegraded":"2024-04-16T17:01:30.018417+0000","last_fullsized":"2024-04-16T17:01:30.018417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845638+0000","last_change":"2024-04-16T17:01:29.035651+0000","last_active":"2024-04-16T17:01:30.845638+0000","last_peered":"2024-04-16T17:01:30.845638+0000","last_clean":"2024-04-16T17:01:30.845638+0000","last_became_active":"2024-04-16T17:01:29.034865+0000","last_became_peered":"2024-04-16T17:01:29.034865+0000","last_unstale":"2024-04-16T17:01:30.845638+0000","last_undegraded":"2024-04-16T17:01:30.845638+0000","last_fullsized":"2024-04-16T17:01:30.845638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:49:31.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.164582+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:01:58.164582+0000","last_peered":"2024-04-16T17:01:58.164582+0000","last_clean":"2024-04-16T17:01:58.164582+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:01:58.164582+0000","last_undegraded":"2024-04-16T17:01:58.164582+0000","last_fullsized":"2024-04-16T17:01:58.164582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845729+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:01:30.845729+0000","last_peered":"2024-04-16T17:01:30.845729+0000","last_clean":"2024-04-16T17:01:30.845729+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:01:30.845729+0000","last_undegraded":"2024-04-16T17:01:30.845729+0000","last_fullsized":"2024-04-16T17:01:30.845729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845226+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:01:30.845226+0000","last_peered":"2024-04-16T17:01:30.845226+0000","last_clean":"2024-04-16T17:01:30.845226+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:01:30.845226+0000","last_undegraded":"2024-04-16T17:01:30.845226+0000","last_fullsized":"2024-04-16T17:01:30.845226+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512267+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:01:30.512267+0000","last_peered":"2024-04-16T17:01:30.512267+0000","last_clean":"2024-04-16T17:01:30.512267+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:01:30.512267+0000","last_undegraded":"2024-04-16T17:01:30.512267+0000","last_fullsized":"2024-04-16T17:01:30.512267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223419+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:01:30.223419+0000","last_peered":"2024-04-16T17:01:30.223419+0000","last_clean":"2024-04-16T17:01:30.223419+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:01:30.223419+0000","last_undegraded":"2024-04-16T17:01:30.223419+0000","last_fullsized":"2024-04-16T17:01:30.223419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845558+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:01:30.845558+0000","last_peered":"2024-04-16T17:01:30.845558+0000","last_clean":"2024-04-16T17:01:30.845558+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:01:30.845558+0000","last_undegraded":"2024-04-16T17:01:30.845558+0000","last_fullsized":"2024-04-16T17:01:30.845558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:02:01.180669+0000","last_change":"2024-04-16T17:01:23.915773+0000","last_active":"2024-04-16T17:02:01.180669+0000","last_peered":"2024-04-16T17:02:01.180669+0000","last_clean":"2024-04-16T17:02:01.180669+0000","last_became_active":"2024-04-16T17:01:23.915398+0000","last_became_peered":"2024-04-16T17:01:23.915398+0000","last_unstale":"2024-04-16T17:02:01.180669+0000","last_undegraded":"2024-04-16T17:02:01.180669+0000","last_fullsized":"2024-04-16T17:02:01.180669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:32:50.294516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512590+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:01:30.512590+0000","last_peered":"2024-04-16T17:01:30.512590+0000","last_clean":"2024-04-16T17:01:30.512590+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:01:30.512590+0000","last_undegraded":"2024-04-16T17:01:30.512590+0000","last_fullsized":"2024-04-16T17:01:30.512590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845585+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:01:30.845585+0000","last_peered":"2024-04-16T17:01:30.845585+0000","last_clean":"2024-04-16T17:01:30.845585+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:01:30.845585+0000","last_undegraded":"2024-04-16T17:01:30.845585+0000","last_fullsized":"2024-04-16T17:01:30.845585+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512256+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:01:30.512256+0000","last_peered":"2024-04-16T17:01:30.512256+0000","last_clean":"2024-04-16T17:01:30.512256+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:01:30.512256+0000","last_undegraded":"2024-04-16T17:01:30.512256+0000","last_fullsized":"2024-04-16T17:01:30.512256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223374+0000","last_change":"2024-04-16T17:01:24.558878+0000","last_active":"2024-04-16T17:01:30.223374+0000","last_peered":"2024-04-16T17:01:30.223374+0000","last_clean":"2024-04-16T17:01:30.223374+0000","last_became_active":"2024-04-16T17:01:24.558504+0000","last_became_peered":"2024-04-16T17:01:24.558504+0000","last_unstale":"2024-04-16T17:01:30.223374+0000","last_undegraded":"2024-04-16T17:01:30.223374+0000","last_fullsized":"2024-04-16T17:01:30.223374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:34.526473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512389+0000","last_change":"2024-04-16T17:01:29.032665+0000","last_active":"2024-04-16T17:01:30.512389+0000","last_peered":"2024-04-16T17:01:30.512389+0000","last_clean":"2024-04-16T17:01:30.512389+0000","last_became_active":"2024-04-16T17:01:29.032293+0000","last_became_peered":"2024-04-16T17:01:29.032293+0000","last_unstale":"2024-04-16T17:01:30.512389+0000","last_undegraded":"2024-04-16T17:01:30.512389+0000","last_fullsized":"2024-04-16T17:01:30.512389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:46.193049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.172686+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:01:58.172686+0000","last_peered":"2024-04-16T17:01:58.172686+0000","last_clean":"2024-04-16T17:01:58.172686+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:01:58.172686+0000","last_undegraded":"2024-04-16T17:01:58.172686+0000","last_fullsized":"2024-04-16T17:01:58.172686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035300+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:01:31.035300+0000","last_peered":"2024-04-16T17:01:31.035300+0000","last_clean":"2024-04-16T17:01:31.035300+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:01:31.035300+0000","last_undegraded":"2024-04-16T17:01:31.035300+0000","last_fullsized":"2024-04-16T17:01:31.035300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018269+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:01:30.018269+0000","last_peered":"2024-04-16T17:01:30.018269+0000","last_clean":"2024-04-16T17:01:30.018269+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:01:30.018269+0000","last_undegraded":"2024-04-16T17:01:30.018269+0000","last_fullsized":"2024-04-16T17:01:30.018269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.222958+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:01:30.222958+0000","last_peered":"2024-04-16T17:01:30.222958+0000","last_clean":"2024-04-16T17:01:30.222958+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:01:30.222958+0000","last_undegraded":"2024-04-16T17:01:30.222958+0000","last_fullsized":"2024-04-16T17:01:30.222958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.386571+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:01:30.386571+0000","last_peered":"2024-04-16T17:01:30.386571+0000","last_clean":"2024-04-16T17:01:30.386571+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:01:30.386571+0000","last_undegraded":"2024-04-16T17:01:30.386571+0000","last_fullsized":"2024-04-16T17:01:30.386571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.036068+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:01:31.036068+0000","last_peered":"2024-04-16T17:01:31.036068+0000","last_clean":"2024-04-16T17:01:31.036068+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:01:31.036068+0000","last_undegraded":"2024-04-16T17:01:31.036068+0000","last_fullsized":"2024-04-16T17:01:31.036068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.158886+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:01:57.158886+0000","last_peered":"2024-04-16T17:01:57.158886+0000","last_clean":"2024-04-16T17:01:57.158886+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:01:57.158886+0000","last_undegraded":"2024-04-16T17:01:57.158886+0000","last_fullsized":"2024-04-16T17:01:57.158886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845789+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:01:30.845789+0000","last_peered":"2024-04-16T17:01:30.845789+0000","last_clean":"2024-04-16T17:01:30.845789+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:01:30.845789+0000","last_undegraded":"2024-04-16T17:01:30.845789+0000","last_fullsized":"2024-04-16T17:01:30.845789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.386690+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:01:30.386690+0000","last_peered":"2024-04-16T17:01:30.386690+0000","last_clean":"2024-04-16T17:01:30.386690+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:01:30.386690+0000","last_undegraded":"2024-04-16T17:01:30.386690+0000","last_fullsized":"2024-04-16T17:01:30.386690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.162007+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:01:57.162007+0000","last_peered":"2024-04-16T17:01:57.162007+0000","last_clean":"2024-04-16T17:01:57.162007+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:01:57.162007+0000","last_undegraded":"2024-04-16T17:01:57.162007+0000","last_fullsized":"2024-04-16T17:01:57.162007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018441+0000","last_change":"2024-04-16T17:01:24.558548+0000","last_active":"2024-04-16T17:01:30.018441+0000","last_peered":"2024-04-16T17:01:30.018441+0000","last_clean":"2024-04-16T17:01:30.018441+0000","last_became_active":"2024-04-16T17:01:24.557329+0000","last_became_peered":"2024-04-16T17:01:24.557329+0000","last_unstale":"2024-04-16T17:01:30.018441+0000","last_undegraded":"2024-04-16T17:01:30.018441+0000","last_fullsized":"2024-04-16T17:01:30.018441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:22.310143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845383+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:01:30.845383+0000","last_peered":"2024-04-16T17:01:30.845383+0000","last_clean":"2024-04-16T17:01:30.845383+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:01:30.845383+0000","last_undegraded":"2024-04-16T17:01:30.845383+0000","last_fullsized":"2024-04-16T17:01:30.845383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.163770+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:01:57.163770+0000","last_peered":"2024-04-16T17:01:57.163770+0000","last_clean":"2024-04-16T17:01:57.163770+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:01:57.163770+0000","last_undegraded":"2024-04-16T17:01:57.163770+0000","last_fullsized":"2024-04-16T17:01:57.163770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845838+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:01:30.845838+0000","last_peered":"2024-04-16T17:01:30.845838+0000","last_clean":"2024-04-16T17:01:30.845838+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:01:30.845838+0000","last_undegraded":"2024-04-16T17:01:30.845838+0000","last_fullsized":"2024-04-16T17:01:30.845838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.387377+0000","last_change":"2024-04-16T17:01:29.025629+0000","last_active":"2024-04-16T17:01:30.387377+0000","last_peered":"2024-04-16T17:01:30.387377+0000","last_clean":"2024-04-16T17:01:30.387377+0000","last_became_active":"2024-04-16T17:01:29.022689+0000","last_became_peered":"2024-04-16T17:01:29.022689+0000","last_unstale":"2024-04-16T17:01:30.387377+0000","last_undegraded":"2024-04-16T17:01:30.387377+0000","last_fullsized":"2024-04-16T17:01:30.387377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:40.917768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.156822+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:01:57.156822+0000","last_peered":"2024-04-16T17:01:57.156822+0000","last_clean":"2024-04-16T17:01:57.156822+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:01:57.156822+0000","last_undegraded":"2024-04-16T17:01:57.156822+0000","last_fullsized":"2024-04-16T17:01:57.156822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512442+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:01:30.512442+0000","last_peered":"2024-04-16T17:01:30.512442+0000","last_clean":"2024-04-16T17:01:30.512442+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:01:30.512442+0000","last_undegraded":"2024-04-16T17:01:30.512442+0000","last_fullsized":"2024-04-16T17:01:30.512442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223784+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:01:30.223784+0000","last_peered":"2024-04-16T17:01:30.223784+0000","last_clean":"2024-04-16T17:01:30.223784+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:01:30.223784+0000","last_undegraded":"2024-04-16T17:01:30.223784+0000","last_fullsized":"2024-04-16T17:01:30.223784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.167691+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:01:58.167691+0000","last_peered":"2024-04-16T17:01:58.167691+0000","last_clean":"2024-04-16T17:01:58.167691+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:01:58.167691+0000","last_undegraded":"2024-04-16T17:01:58.167691+0000","last_fullsized":"2024-04-16T17:01:58.167691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.510951+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:01:30.510951+0000","last_peered":"2024-04-16T17:01:30.510951+0000","last_clean":"2024-04-16T17:01:30.510951+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:01:30.510951+0000","last_undegraded":"2024-04-16T17:01:30.510951+0000","last_fullsized":"2024-04-16T17:01:30.510951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511892+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:01:30.511892+0000","last_peered":"2024-04-16T17:01:30.511892+0000","last_clean":"2024-04-16T17:01:30.511892+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:01:30.511892+0000","last_undegraded":"2024-04-16T17:01:30.511892+0000","last_fullsized":"2024-04-16T17:01:30.511892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.163365+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:01:57.163365+0000","last_peered":"2024-04-16T17:01:57.163365+0000","last_clean":"2024-04-16T17:01:57.163365+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:01:57.163365+0000","last_undegraded":"2024-04-16T17:01:57.163365+0000","last_fullsized":"2024-04-16T17:01:57.163365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014786+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:01:30.014786+0000","last_peered":"2024-04-16T17:01:30.014786+0000","last_clean":"2024-04-16T17:01:30.014786+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:01:30.014786+0000","last_undegraded":"2024-04-16T17:01:30.014786+0000","last_fullsized":"2024-04-16T17:01:30.014786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.387804+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:01:30.387804+0000","last_peered":"2024-04-16T17:01:30.387804+0000","last_clean":"2024-04-16T17:01:30.387804+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:01:30.387804+0000","last_undegraded":"2024-04-16T17:01:30.387804+0000","last_fullsized":"2024-04-16T17:01:30.387804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:02:01.182031+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:02:01.182031+0000","last_peered":"2024-04-16T17:02:01.182031+0000","last_clean":"2024-04-16T17:02:01.182031+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:02:01.182031+0000","last_undegraded":"2024-04-16T17:02:01.182031+0000","last_fullsized":"2024-04-16T17:02:01.182031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014621+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:01:30.014621+0000","last_peered":"2024-04-16T17:01:30.014621+0000","last_clean":"2024-04-16T17:01:30.014621+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:01:30.014621+0000","last_undegraded":"2024-04-16T17:01:30.014621+0000","last_fullsized":"2024-04-16T17:01:30.014621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512347+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:01:30.512347+0000","last_peered":"2024-04-16T17:01:30.512347+0000","last_clean":"2024-04-16T17:01:30.512347+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:01:30.512347+0000","last_undegraded":"2024-04-16T17:01:30.512347+0000","last_fullsized":"2024-04-16T17:01:30.512347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.172936+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:01:58.172936+0000","last_peered":"2024-04-16T17:01:58.172936+0000","last_clean":"2024-04-16T17:01:58.172936+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:01:58.172936+0000","last_undegraded":"2024-04-16T17:01:58.172936+0000","last_fullsized":"2024-04-16T17:01:58.172936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223012+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:01:30.223012+0000","last_peered":"2024-04-16T17:01:30.223012+0000","last_clean":"2024-04-16T17:01:30.223012+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:01:30.223012+0000","last_undegraded":"2024-04-16T17:01:30.223012+0000","last_fullsized":"2024-04-16T17:01:30.223012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223244+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:01:30.223244+0000","last_peered":"2024-04-16T17:01:30.223244+0000","last_clean":"2024-04-16T17:01:30.223244+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:01:30.223244+0000","last_undegraded":"2024-04-16T17:01:30.223244+0000","last_fullsized":"2024-04-16T17:01:30.223244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:59.173061+0000","last_change":"2024-04-16T17:01:23.910149+0000","last_active":"2024-04-16T17:01:59.173061+0000","last_peered":"2024-04-16T17:01:59.173061+0000","last_clean":"2024-04-16T17:01:59.173061+0000","last_became_active":"2024-04-16T17:01:23.909796+0000","last_became_peered":"2024-04-16T17:01:23.909796+0000","last_unstale":"2024-04-16T17:01:59.173061+0000","last_undegraded":"2024-04-16T17:01:59.173061+0000","last_fullsized":"2024-04-16T17:01:59.173061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:38.574698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512394+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:01:30.512394+0000","last_peered":"2024-04-16T17:01:30.512394+0000","last_clean":"2024-04-16T17:01:30.512394+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:01:30.512394+0000","last_undegraded":"2024-04-16T17:01:30.512394+0000","last_fullsized":"2024-04-16T17:01:30.512394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512723+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:01:30.512723+0000","last_peered":"2024-04-16T17:01:30.512723+0000","last_clean":"2024-04-16T17:01:30.512723+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:01:30.512723+0000","last_undegraded":"2024-04-16T17:01:30.512723+0000","last_fullsized":"2024-04-16T17:01:30.512723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.386901+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:01:30.386901+0000","last_peered":"2024-04-16T17:01:30.386901+0000","last_clean":"2024-04-16T17:01:30.386901+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:01:30.386901+0000","last_undegraded":"2024-04-16T17:01:30.386901+0000","last_fullsized":"2024-04-16T17:01:30.386901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512281+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:01:30.512281+0000","last_peered":"2024-04-16T17:01:30.512281+0000","last_clean":"2024-04-16T17:01:30.512281+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:01:30.512281+0000","last_undegraded":"2024-04-16T17:01:30.512281+0000","last_fullsized":"2024-04-16T17:01:30.512281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511442+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:01:30.511442+0000","last_peered":"2024-04-16T17:01:30.511442+0000","last_clean":"2024-04-16T17:01:30.511442+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:01:30.511442+0000","last_undegraded":"2024-04-16T17:01:30.511442+0000","last_fullsized":"2024-04-16T17:01:30.511442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014625+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:01:30.014625+0000","last_peered":"2024-04-16T17:01:30.014625+0000","last_clean":"2024-04-16T17:01:30.014625+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:01:30.014625+0000","last_undegraded":"2024-04-16T17:01:30.014625+0000","last_fullsized":"2024-04-16T17:01:30.014625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512037+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:01:30.512037+0000","last_peered":"2024-04-16T17:01:30.512037+0000","last_clean":"2024-04-16T17:01:30.512037+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:01:30.512037+0000","last_undegraded":"2024-04-16T17:01:30.512037+0000","last_fullsized":"2024-04-16T17:01:30.512037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512391+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:01:30.512391+0000","last_peered":"2024-04-16T17:01:30.512391+0000","last_clean":"2024-04-16T17:01:30.512391+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:01:30.512391+0000","last_undegraded":"2024-04-16T17:01:30.512391+0000","last_fullsized":"2024-04-16T17:01:30.512391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223033+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:01:30.223033+0000","last_peered":"2024-04-16T17:01:30.223033+0000","last_clean":"2024-04-16T17:01:30.223033+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:01:30.223033+0000","last_undegraded":"2024-04-16T17:01:30.223033+0000","last_fullsized":"2024-04-16T17:01:30.223033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:59.176751+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:01:59.176751+0000","last_peered":"2024-04-16T17:01:59.176751+0000","last_clean":"2024-04-16T17:01:59.176751+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:01:59.176751+0000","last_undegraded":"2024-04-16T17:01:59.176751+0000","last_fullsized":"2024-04-16T17:01:59.176751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.038230+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:01:31.038230+0000","last_peered":"2024-04-16T17:01:31.038230+0000","last_clean":"2024-04-16T17:01:31.038230+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:01:31.038230+0000","last_undegraded":"2024-04-16T17:01:31.038230+0000","last_fullsized":"2024-04-16T17:01:31.038230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035715+0000","last_change":"2024-04-16T17:01:24.549236+0000","last_active":"2024-04-16T17:01:31.035715+0000","last_peered":"2024-04-16T17:01:31.035715+0000","last_clean":"2024-04-16T17:01:31.035715+0000","last_became_active":"2024-04-16T17:01:24.548850+0000","last_became_peered":"2024-04-16T17:01:24.548850+0000","last_unstale":"2024-04-16T17:01:31.035715+0000","last_undegraded":"2024-04-16T17:01:31.035715+0000","last_fullsized":"2024-04-16T17:01:31.035715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:00.312829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:59.174058+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:01:59.174058+0000","last_peered":"2024-04-16T17:01:59.174058+0000","last_clean":"2024-04-16T17:01:59.174058+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:01:59.174058+0000","last_undegraded":"2024-04-16T17:01:59.174058+0000","last_fullsized":"2024-04-16T17:01:59.174058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014235+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:01:30.014235+0000","last_peered":"2024-04-16T17:01:30.014235+0000","last_clean":"2024-04-16T17:01:30.014235+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:01:30.014235+0000","last_undegraded":"2024-04-16T17:01:30.014235+0000","last_fullsized":"2024-04-16T17:01:30.014235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223640+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:01:30.223640+0000","last_peered":"2024-04-16T17:01:30.223640+0000","last_clean":"2024-04-16T17:01:30.223640+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:01:30.223640+0000","last_undegraded":"2024-04-16T17:01:30.223640+0000","last_fullsized":"2024-04-16T17:01:30.223640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.510360+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:01:30.510360+0000","last_peered":"2024-04-16T17:01:30.510360+0000","last_clean":"2024-04-16T17:01:30.510360+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:01:30.510360+0000","last_undegraded":"2024-04-16T17:01:30.510360+0000","last_fullsized":"2024-04-16T17:01:30.510360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.165151+0000","last_change":"2024-04-16T17:01:23.913714+0000","last_active":"2024-04-16T17:01:58.165151+0000","last_peered":"2024-04-16T17:01:58.165151+0000","last_clean":"2024-04-16T17:01:58.165151+0000","last_became_active":"2024-04-16T17:01:23.913246+0000","last_became_peered":"2024-04-16T17:01:23.913246+0000","last_unstale":"2024-04-16T17:01:58.165151+0000","last_undegraded":"2024-04-16T17:01:58.165151+0000","last_fullsized":"2024-04-16T17:01:58.165151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:14.850863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512027+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:01:30.512027+0000","last_peered":"2024-04-16T17:01:30.512027+0000","last_clean":"2024-04-16T17:01:30.512027+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:01:30.512027+0000","last_undegraded":"2024-04-16T17:01:30.512027+0000","last_fullsized":"2024-04-16T17:01:30.512027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.164407+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:01:57.164407+0000","last_peered":"2024-04-16T17:01:57.164407+0000","last_clean":"2024-04-16T17:01:57.164407+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:01:57.164407+0000","last_undegraded":"2024-04-16T17:01:57.164407+0000","last_fullsized":"2024-04-16T17:01:57.164407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512061+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:01:30.512061+0000","last_peered":"2024-04-16T17:01:30.512061+0000","last_clean":"2024-04-16T17:01:30.512061+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:01:30.512061+0000","last_undegraded":"2024-04-16T17:01:30.512061+0000","last_fullsized":"2024-04-16T17:01:30.512061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035272+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:01:31.035272+0000","last_peered":"2024-04-16T17:01:31.035272+0000","last_clean":"2024-04-16T17:01:31.035272+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:01:31.035272+0000","last_undegraded":"2024-04-16T17:01:31.035272+0000","last_fullsized":"2024-04-16T17:01:31.035272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:02:00.944881+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:02:00.944881+0000","last_peered":"2024-04-16T17:02:00.944881+0000","last_clean":"2024-04-16T17:02:00.944881+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:02:00.944881+0000","last_undegraded":"2024-04-16T17:02:00.944881+0000","last_fullsized":"2024-04-16T17:02:00.944881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.036780+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:01:31.036780+0000","last_peered":"2024-04-16T17:01:31.036780+0000","last_clean":"2024-04-16T17:01:31.036780+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:01:31.036780+0000","last_undegraded":"2024-04-16T17:01:31.036780+0000","last_fullsized":"2024-04-16T17:01:31.036780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845008+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:01:30.845008+0000","last_peered":"2024-04-16T17:01:30.845008+0000","last_clean":"2024-04-16T17:01:30.845008+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:01:30.845008+0000","last_undegraded":"2024-04-16T17:01:30.845008+0000","last_fullsized":"2024-04-16T17:01:30.845008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.157922+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:01:57.157922+0000","last_peered":"2024-04-16T17:01:57.157922+0000","last_clean":"2024-04-16T17:01:57.157922+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:01:57.157922+0000","last_undegraded":"2024-04-16T17:01:57.157922+0000","last_fullsized":"2024-04-16T17:01:57.157922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.018477+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:01:30.018477+0000","last_peered":"2024-04-16T17:01:30.018477+0000","last_clean":"2024-04-16T17:01:30.018477+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:01:30.018477+0000","last_undegraded":"2024-04-16T17:01:30.018477+0000","last_fullsized":"2024-04-16T17:01:30.018477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845600+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:01:30.845600+0000","last_peered":"2024-04-16T17:01:30.845600+0000","last_clean":"2024-04-16T17:01:30.845600+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:01:30.845600+0000","last_undegraded":"2024-04-16T17:01:30.845600+0000","last_fullsized":"2024-04-16T17:01:30.845600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.163276+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:01:57.163276+0000","last_peered":"2024-04-16T17:01:57.163276+0000","last_clean":"2024-04-16T17:01:57.163276+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:01:57.163276+0000","last_undegraded":"2024-04-16T17:01:57.163276+0000","last_fullsized":"2024-04-16T17:01:57.163276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511662+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:01:30.511662+0000","last_peered":"2024-04-16T17:01:30.511662+0000","last_clean":"2024-04-16T17:01:30.511662+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:01:30.511662+0000","last_undegraded":"2024-04-16T17:01:30.511662+0000","last_fullsized":"2024-04-16T17:01:30.511662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.388105+0000","last_change":"2024-04-16T17:01:29.035782+0000","last_active":"2024-04-16T17:01:30.388105+0000","last_peered":"2024-04-16T17:01:30.388105+0000","last_clean":"2024-04-16T17:01:30.388105+0000","last_became_active":"2024-04-16T17:01:29.034812+0000","last_became_peered":"2024-04-16T17:01:29.034812+0000","last_unstale":"2024-04-16T17:01:30.388105+0000","last_undegraded":"2024-04-16T17:01:30.388105+0000","last_fullsized":"2024-04-16T17:01:30.388105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:27.655802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.166034+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:01:57.166034+0000","last_peered":"2024-04-16T17:01:57.166034+0000","last_clean":"2024-04-16T17:01:57.166034+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:01:57.166034+0000","last_undegraded":"2024-04-16T17:01:57.166034+0000","last_fullsized":"2024-04-16T17:01:57.166034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511763+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:01:30.511763+0000","last_peered":"2024-04-16T17:01:30.511763+0000","last_clean":"2024-04-16T17:01:30.511763+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:01:30.511763+0000","last_undegraded":"2024-04-16T17:01:30.511763+0000","last_fullsized":"2024-04-16T17:01:30.511763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:32.398949+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:01:32.398949+0000","last_peered":"2024-04-16T17:01:32.398949+0000","last_clean":"2024-04-16T17:01:32.398949+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:01:32.398949+0000","last_undegraded":"2024-04-16T17:01:32.398949+0000","last_fullsized":"2024-04-16T17:01:32.398949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845209+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:01:30.845209+0000","last_peered":"2024-04-16T17:01:30.845209+0000","last_clean":"2024-04-16T17:01:30.845209+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:01:30.845209+0000","last_undegraded":"2024-04-16T17:01:30.845209+0000","last_fullsized":"2024-04-16T17:01:30.845209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.036302+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:01:31.036302+0000","last_peered":"2024-04-16T17:01:31.036302+0000","last_clean":"2024-04-16T17:01:31.036302+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:01:31.036302+0000","last_undegraded":"2024-04-16T17:01:31.036302+0000","last_fullsized":"2024-04-16T17:01:31.036302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845642+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:01:30.845642+0000","last_peered":"2024-04-16T17:01:30.845642+0000","last_clean":"2024-04-16T17:01:30.845642+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:01:30.845642+0000","last_undegraded":"2024-04-16T17:01:30.845642+0000","last_fullsized":"2024-04-16T17:01:30.845642+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:57.161528+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:01:57.161528+0000","last_peered":"2024-04-16T17:01:57.161528+0000","last_clean":"2024-04-16T17:01:57.161528+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:01:57.161528+0000","last_undegraded":"2024-04-16T17:01:57.161528+0000","last_fullsized":"2024-04-16T17:01:57.161528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014777+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:01:30.014777+0000","last_peered":"2024-04-16T17:01:30.014777+0000","last_clean":"2024-04-16T17:01:30.014777+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:01:30.014777+0000","last_undegraded":"2024-04-16T17:01:30.014777+0000","last_fullsized":"2024-04-16T17:01:30.014777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.844835+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:01:30.844835+0000","last_peered":"2024-04-16T17:01:30.844835+0000","last_clean":"2024-04-16T17:01:30.844835+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:01:30.844835+0000","last_undegraded":"2024-04-16T17:01:30.844835+0000","last_fullsized":"2024-04-16T17:01:30.844835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:58.172288+0000","last_change":"2024-04-16T17:01:23.926672+0000","last_active":"2024-04-16T17:01:58.172288+0000","last_peered":"2024-04-16T17:01:58.172288+0000","last_clean":"2024-04-16T17:01:58.172288+0000","last_became_active":"2024-04-16T17:01:23.926330+0000","last_became_peered":"2024-04-16T17:01:23.926330+0000","last_unstale":"2024-04-16T17:01:58.172288+0000","last_undegraded":"2024-04-16T17:01:58.172288+0000","last_fullsized":"2024-04-16T17:01:58.172288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:30:23.951003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.015072+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:01:30.015072+0000","last_peered":"2024-04-16T17:01:30.015072+0000","last_clean":"2024-04-16T17:01:30.015072+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:01:30.015072+0000","last_undegraded":"2024-04-16T17:01:30.015072+0000","last_fullsized":"2024-04-16T17:01:30.015072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.017007+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:01:30.017007+0000","last_peered":"2024-04-16T17:01:30.017007+0000","last_clean":"2024-04-16T17:01:30.017007+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:01:30.017007+0000","last_undegraded":"2024-04-16T17:01:30.017007+0000","last_fullsized":"2024-04-16T17:01:30.017007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.223551+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:01:30.223551+0000","last_peered":"2024-04-16T17:01:30.223551+0000","last_clean":"2024-04-16T17:01:30.223551+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:01:30.223551+0000","last_undegraded":"2024-04-16T17:01:30.223551+0000","last_fullsized":"2024-04-16T17:01:30.223551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.014654+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:01:30.014654+0000","last_peered":"2024-04-16T17:01:30.014654+0000","last_clean":"2024-04-16T17:01:30.014654+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:01:30.014654+0000","last_undegraded":"2024-04-16T17:01:30.014654+0000","last_fullsized":"2024-04-16T17:01:30.014654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.222813+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:01:30.222813+0000","last_peered":"2024-04-16T17:01:30.222813+0000","last_clean":"2024-04-16T17:01:30.222813+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:01:30.222813+0000","last_undegraded":"2024-04-16T17:01:30.222813+0000","last_fullsized":"2024-04-16T17:01:30.222813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511894+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:01:30.511894+0000","last_peered":"2024-04-16T17:01:30.511894+0000","last_clean":"2024-04-16T17:01:30.511894+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:01:30.511894+0000","last_undegraded":"2024-04-16T17:01:30.511894+0000","last_fullsized":"2024-04-16T17:01:30.511894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.015062+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:01:30.015062+0000","last_peered":"2024-04-16T17:01:30.015062+0000","last_clean":"2024-04-16T17:01:30.015062+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:01:30.015062+0000","last_undegraded":"2024-04-16T17:01:30.015062+0000","last_fullsized":"2024-04-16T17:01:30.015062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511913+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:01:30.511913+0000","last_peered":"2024-04-16T17:01:30.511913+0000","last_clean":"2024-04-16T17:01:30.511913+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:01:30.511913+0000","last_undegraded":"2024-04-16T17:01:30.511913+0000","last_fullsized":"2024-04-16T17:01:30.511913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845591+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:01:30.845591+0000","last_peered":"2024-04-16T17:01:30.845591+0000","last_clean":"2024-04-16T17:01:30.845591+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:01:30.845591+0000","last_undegraded":"2024-04-16T17:01:30.845591+0000","last_fullsized":"2024-04-16T17:01:30.845591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845037+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:01:30.845037+0000","last_peered":"2024-04-16T17:01:30.845037+0000","last_clean":"2024-04-16T17:01:30.845037+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:01:30.845037+0000","last_undegraded":"2024-04-16T17:01:30.845037+0000","last_fullsized":"2024-04-16T17:01:30.845037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.017247+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:01:30.017247+0000","last_peered":"2024-04-16T17:01:30.017247+0000","last_clean":"2024-04-16T17:01:30.017247+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:01:30.017247+0000","last_undegraded":"2024-04-16T17:01:30.017247+0000","last_fullsized":"2024-04-16T17:01:30.017247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511783+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:01:30.511783+0000","last_peered":"2024-04-16T17:01:30.511783+0000","last_clean":"2024-04-16T17:01:30.511783+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:01:30.511783+0000","last_undegraded":"2024-04-16T17:01:30.511783+0000","last_fullsized":"2024-04-16T17:01:30.511783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.512403+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:01:30.512403+0000","last_peered":"2024-04-16T17:01:30.512403+0000","last_clean":"2024-04-16T17:01:30.512403+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:01:30.512403+0000","last_undegraded":"2024-04-16T17:01:30.512403+0000","last_fullsized":"2024-04-16T17:01:30.512403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845692+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:01:30.845692+0000","last_peered":"2024-04-16T17:01:30.845692+0000","last_clean":"2024-04-16T17:01:30.845692+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:01:30.845692+0000","last_undegraded":"2024-04-16T17:01:30.845692+0000","last_fullsized":"2024-04-16T17:01:30.845692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.037072+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:01:31.037072+0000","last_peered":"2024-04-16T17:01:31.037072+0000","last_clean":"2024-04-16T17:01:31.037072+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:01:31.037072+0000","last_undegraded":"2024-04-16T17:01:31.037072+0000","last_fullsized":"2024-04-16T17:01:31.037072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.222984+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:01:30.222984+0000","last_peered":"2024-04-16T17:01:30.222984+0000","last_clean":"2024-04-16T17:01:30.222984+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:01:30.222984+0000","last_undegraded":"2024-04-16T17:01:30.222984+0000","last_fullsized":"2024-04-16T17:01:30.222984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:31.035591+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:01:31.035591+0000","last_peered":"2024-04-16T17:01:31.035591+0000","last_clean":"2024-04-16T17:01:31.035591+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:01:31.035591+0000","last_undegraded":"2024-04-16T17:01:31.035591+0000","last_fullsized":"2024-04-16T17:01:31.035591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.845257+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:01:30.845257+0000","last_peered":"2024-04-16T17:01:30.845257+0000","last_clean":"2024-04-16T17:01:30.845257+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:01:30.845257+0000","last_undegraded":"2024-04-16T17:01:30.845257+0000","last_fullsized":"2024-04-16T17:01:30.845257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511279+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:01:30.511279+0000","last_peered":"2024-04-16T17:01:30.511279+0000","last_clean":"2024-04-16T17:01:30.511279+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:01:30.511279+0000","last_undegraded":"2024-04-16T17:01:30.511279+0000","last_fullsized":"2024-04-16T17:01:30.511279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-16T17:01:30.511971+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:01:30.511971+0000","last_peered":"2024-04-16T17:01:30.511971+0000","last_clean":"2024-04-16T17:01:30.511971+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:01:30.511971+0000","last_undegraded":"2024-04-16T17:01:30.511971+0000","last_fullsized":"2024-04-16T17:01:30.511971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245319,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":132,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":21734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383700,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":477797,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645338,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":1072381,"data_compressed":50228,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808865,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":486225,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070502,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":482049,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332139,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1073471,"data_compressed":50228,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":477305,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481057,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":478693,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":481713,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":477499,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":481955,"data_compressed":14713,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":1073155,"data_compressed":50228,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:02:03.833 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-16T17:02:03.833 INFO:tasks.cephadm:Setting up client nodes... 2024-04-16T17:02:03.834 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-16T17:02:04.342 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:04 smithi018 ceph-mon[31776]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-04-16T17:02:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:04 smithi018 ceph-mon[31776]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:02:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:04 smithi043 ceph-mon[36901]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-04-16T17:02:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:04 smithi043 ceph-mon[36901]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:02:04.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:04 smithi022 ceph-mon[36023]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-04-16T17:02:04.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:04 smithi022 ceph-mon[36023]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:02:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:05 smithi043 ceph-mon[36901]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:02:05.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:05 smithi018 ceph-mon[31776]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:02:05.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:05 smithi022 ceph-mon[36023]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:02:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:06 smithi018 ceph-mon[31776]: pgmap v339: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-04-16T17:02:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:06 smithi043 ceph-mon[36901]: pgmap v339: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-04-16T17:02:06.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.660+0000 7fccbb200700 1 -- 172.21.15.18:0/2303717665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccbc10ab60 msgr2=0x7fccbc111410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:06.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.660+0000 7fccbb200700 1 --2- 172.21.15.18:0/2303717665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccbc10ab60 0x7fccbc111410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fccb400b0d0 tx=0x7fccb400b3e0 comp rx=0 tx=0).stop 2024-04-16T17:02:06.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.661+0000 7fccbb200700 1 -- 172.21.15.18:0/2303717665 shutdown_connections 2024-04-16T17:02:06.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.661+0000 7fccbb200700 1 --2- 172.21.15.18:0/2303717665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccbc10ab60 0x7fccbc111410 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:06.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.661+0000 7fccbb200700 1 --2- 172.21.15.18:0/2303717665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccbc108fc0 0x7fccbc1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:06.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.661+0000 7fccbb200700 1 --2- 172.21.15.18:0/2303717665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccbc10a1c0 0x7fccbc10a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:06.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.661+0000 7fccbb200700 1 -- 172.21.15.18:0/2303717665 >> 172.21.15.18:0/2303717665 conn(0x7fccbc077f90 msgr2=0x7fccbc07a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:06.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.661+0000 7fccbb200700 1 -- 172.21.15.18:0/2303717665 shutdown_connections 2024-04-16T17:02:06.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.661+0000 7fccbb200700 1 -- 172.21.15.18:0/2303717665 wait complete. 2024-04-16T17:02:06.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.662+0000 7fccbb200700 1 Processor -- start 2024-04-16T17:02:06.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.662+0000 7fccbb200700 1 -- start start 2024-04-16T17:02:06.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.662+0000 7fccbb200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccbc108fc0 0x7fccbc19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.662+0000 7fccbb200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccbc10a1c0 0x7fccbc19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.662+0000 7fccbb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccbc10ab60 0x7fccbc1a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.662+0000 7fccbb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc114c00 con 0x7fccbc10ab60 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccbb200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc114a80 con 0x7fccbc108fc0 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccbb200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc114d80 con 0x7fccbc10a1c0 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccba800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccbc10ab60 0x7fccbc1a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccbc108fc0 0x7fccbc19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccbc108fc0 0x7fccbc19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54326/0 (socket says 172.21.15.18:54326) 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9e00700 1 -- 172.21.15.18:0/2618807703 learned_addr learned my addr 172.21.15.18:0/2618807703 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9400700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccbc10a1c0 0x7fccbc19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9e00700 1 -- 172.21.15.18:0/2618807703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccbc10a1c0 msgr2=0x7fccbc19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9e00700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccbc10a1c0 0x7fccbc19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:06.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9e00700 1 -- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccbc10ab60 msgr2=0x7fccbc1a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:06.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9e00700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccbc10ab60 0x7fccbc1a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:06.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.663+0000 7fccb9e00700 1 -- 172.21.15.18:0/2618807703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc1a6ce0 con 0x7fccbc108fc0 2024-04-16T17:02:06.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.664+0000 7fccb9e00700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccbc108fc0 0x7fccbc19f080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e950 tx=0x7fccac00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:06.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.664+0000 7fccb2a00700 1 -- 172.21.15.18:0/2618807703 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac004230 con 0x7fccbc108fc0 2024-04-16T17:02:06.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.664+0000 7fccb2a00700 1 -- 172.21.15.18:0/2618807703 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac0043d0 con 0x7fccbc108fc0 2024-04-16T17:02:06.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.664+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccbc1a6fd0 con 0x7fccbc108fc0 2024-04-16T17:02:06.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.664+0000 7fccb2a00700 1 -- 172.21.15.18:0/2618807703 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac003dc0 con 0x7fccbc108fc0 2024-04-16T17:02:06.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.664+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccbc1a73d0 con 0x7fccbc108fc0 2024-04-16T17:02:06.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.666+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccbc02dfe0 con 0x7fccbc108fc0 2024-04-16T17:02:06.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.667+0000 7fccb2a00700 1 -- 172.21.15.18:0/2618807703 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fccac00c7e0 con 0x7fccbc108fc0 2024-04-16T17:02:06.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.669+0000 7fccb2a00700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcca00a3d80 0x7fcca00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:06.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.669+0000 7fccb2a00700 1 -- 172.21.15.18:0/2618807703 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fccac0d0a70 con 0x7fccbc108fc0 2024-04-16T17:02:06.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.669+0000 7fccb9400700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcca00a3d80 0x7fcca00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:06.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.670+0000 7fccb9400700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcca00a3d80 0x7fcca00a6240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009c60 tx=0x7fcca8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:06 smithi022 ceph-mon[36023]: pgmap v339: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-04-16T17:02:06.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:06.673+0000 7fccb2a00700 1 -- 172.21.15.18:0/2618807703 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fccac09d050 con 0x7fccbc108fc0 2024-04-16T17:02:07.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.003+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fccbc0637a0 con 0x7fccbc108fc0 2024-04-16T17:02:07.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.011+0000 7fccb2a00700 1 -- 172.21.15.18:0/2618807703 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fccac09c950 con 0x7fccbc108fc0 2024-04-16T17:02:07.013 INFO:teuthology.orchestra.run.smithi018.stdout:[client.0] 2024-04-16T17:02:07.013 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAPrx5mjudcABAAUr1dDptmMumjhaxK49W1hA== 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.014+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcca00a3d80 msgr2=0x7fcca00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.014+0000 7fccbb200700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcca00a3d80 0x7fcca00a6240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009c60 tx=0x7fcca8009260 comp rx=0 tx=0).stop 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccbc108fc0 msgr2=0x7fccbc19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccbc108fc0 0x7fccbc19f080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e950 tx=0x7fccac00ec60 comp rx=0 tx=0).stop 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 shutdown_connections 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcca00a3d80 0x7fcca00a6240 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccbc108fc0 0x7fccbc19f080 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccbc10ab60 0x7fccbc1a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 --2- 172.21.15.18:0/2618807703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccbc10a1c0 0x7fccbc19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 >> 172.21.15.18:0/2618807703 conn(0x7fccbc077f90 msgr2=0x7fccbc10d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:07.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 shutdown_connections 2024-04-16T17:02:07.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:07.015+0000 7fccbb200700 1 -- 172.21.15.18:0/2618807703 wait complete. 2024-04-16T17:02:07.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:07 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2618807703' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:02:07.425 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:07 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:02:07.425 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:07 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T17:02:07.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:07 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2618807703' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:02:07.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:07 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:02:07.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:07 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T17:02:07.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:07 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2618807703' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:02:07.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:07 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:02:07.677 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:07 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T17:02:07.863 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T17:02:07.863 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-16T17:02:07.863 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-16T17:02:08.025 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-16T17:02:08.025 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-16T17:02:08.026 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph mgr dump --format=json 2024-04-16T17:02:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:08 smithi018 ceph-mon[31776]: pgmap v340: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-04-16T17:02:08.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:08 smithi043 ceph-mon[36901]: pgmap v340: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-04-16T17:02:08.552 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:08.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:08 smithi022 ceph-mon[36023]: pgmap v340: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-04-16T17:02:10.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:10 smithi018 ceph-mon[31776]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-16T17:02:10.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:10 smithi043 ceph-mon[36901]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-16T17:02:10.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:10 smithi022 ceph-mon[36023]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.786+0000 7fd3a5800700 1 -- 172.21.15.18:0/716803023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3a0109d30 msgr2=0x7fd3a010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.786+0000 7fd3a5800700 1 --2- 172.21.15.18:0/716803023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3a0109d30 0x7fd3a010dd80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd388009990 tx=0x7fd388009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.787+0000 7fd3a5800700 1 -- 172.21.15.18:0/716803023 shutdown_connections 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.787+0000 7fd3a5800700 1 --2- 172.21.15.18:0/716803023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3a010e460 0x7fd3a0116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.787+0000 7fd3a5800700 1 --2- 172.21.15.18:0/716803023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0109380 0x7fd3a0109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.787+0000 7fd3a5800700 1 --2- 172.21.15.18:0/716803023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3a0109d30 0x7fd3a010dd80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.787+0000 7fd3a5800700 1 -- 172.21.15.18:0/716803023 >> 172.21.15.18:0/716803023 conn(0x7fd3a00762c0 msgr2=0x7fd3a0107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.787+0000 7fd3a5800700 1 -- 172.21.15.18:0/716803023 shutdown_connections 2024-04-16T17:02:10.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.787+0000 7fd3a5800700 1 -- 172.21.15.18:0/716803023 wait complete. 2024-04-16T17:02:10.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.788+0000 7fd3a5800700 1 Processor -- start 2024-04-16T17:02:10.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.788+0000 7fd3a5800700 1 -- start start 2024-04-16T17:02:10.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd3a5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3a0109380 0x7fd3a01a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd3a5800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3a0109d30 0x7fd3a01a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd3a5800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a010e460 0x7fd3a019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd3a5800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a0119490 con 0x7fd3a0109380 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd3a5800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a0119310 con 0x7fd3a0109d30 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd3a5800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a0119610 con 0x7fd3a010e460 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd39ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3a0109380 0x7fd3a01a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd39ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3a0109380 0x7fd3a01a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41878/0 (socket says 172.21.15.18:41878) 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd39ea00700 1 -- 172.21.15.18:0/738392739 learned_addr learned my addr 172.21.15.18:0/738392739 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd39f400700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a010e460 0x7fd3a019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:10.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.789+0000 7fd397e00700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3a0109d30 0x7fd3a01a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd39ea00700 1 -- 172.21.15.18:0/738392739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a010e460 msgr2=0x7fd3a019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd39ea00700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a010e460 0x7fd3a019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd39ea00700 1 -- 172.21.15.18:0/738392739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3a0109d30 msgr2=0x7fd3a01a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd39ea00700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3a0109d30 0x7fd3a01a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd39ea00700 1 -- 172.21.15.18:0/738392739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a01a0070 con 0x7fd3a0109380 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd39ea00700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3a0109380 0x7fd3a01a5670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd39000e9c0 tx=0x7fd39000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd397400700 1 -- 172.21.15.18:0/738392739 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39000cbd0 con 0x7fd3a0109380 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd397400700 1 -- 172.21.15.18:0/738392739 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39000ee60 con 0x7fd3a0109380 2024-04-16T17:02:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a01a0360 con 0x7fd3a0109380 2024-04-16T17:02:10.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.790+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a01ac5f0 con 0x7fd3a0109380 2024-04-16T17:02:10.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.791+0000 7fd397400700 1 -- 172.21.15.18:0/738392739 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390005630 con 0x7fd3a0109380 2024-04-16T17:02:10.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.792+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a010a860 con 0x7fd3a0109380 2024-04-16T17:02:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.794+0000 7fd397400700 1 -- 172.21.15.18:0/738392739 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd390004750 con 0x7fd3a0109380 2024-04-16T17:02:10.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.796+0000 7fd397400700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3840a3cd0 0x7fd3840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:10.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.797+0000 7fd397400700 1 -- 172.21.15.18:0/738392739 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd3900d0c80 con 0x7fd3a0109380 2024-04-16T17:02:10.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.797+0000 7fd397e00700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3840a3cd0 0x7fd3840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.797+0000 7fd397e00700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3840a3cd0 0x7fd3840a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3a01a0f90 tx=0x7fd38800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:10.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:10.800+0000 7fd397400700 1 -- 172.21.15.18:0/738392739 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd390053ff0 con 0x7fd3a0109380 2024-04-16T17:02:11.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.093+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd3a01a0c70 con 0x7fd3a0109380 2024-04-16T17:02:11.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.101+0000 7fd397400700 1 -- 172.21.15.18:0/738392739 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264998 (secure 0 0 0) 0x7fd39009d260 con 0x7fd3a0109380 2024-04-16T17:02:11.108 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3840a3cd0 msgr2=0x7fd3840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3840a3cd0 0x7fd3840a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3a01a0f90 tx=0x7fd38800b260 comp rx=0 tx=0).stop 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3a0109380 msgr2=0x7fd3a01a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3a0109380 0x7fd3a01a5670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd39000e9c0 tx=0x7fd39000ecd0 comp rx=0 tx=0).stop 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 shutdown_connections 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3840a3cd0 0x7fd3840a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a010e460 0x7fd3a019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3a0109380 0x7fd3a01a5670 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:11.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 --2- 172.21.15.18:0/738392739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3a0109d30 0x7fd3a01a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:11.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 >> 172.21.15.18:0/738392739 conn(0x7fd3a00762c0 msgr2=0x7fd3a0076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:11.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 shutdown_connections 2024-04-16T17:02:11.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:11.107+0000 7fd3a5800700 1 -- 172.21.15.18:0/738392739 wait complete. 2024-04-16T17:02:11.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:11 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/738392739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T17:02:11.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:11 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/738392739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T17:02:11.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:11 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/738392739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T17:02:11.963 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":2462414912},{"type":"v1","addr":"172.21.15.18:6801","nonce":2462414912}]},"active_addr":"172.21.15.18:6801/2462414912","active_change":"2024-04-16T16:53:24.732840+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.18:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":3045212308}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":2149549301}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":1033276779}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":3152065578}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":2221991422}]}]} 2024-04-16T17:02:11.967 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-16T17:02:11.968 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-16T17:02:11.968 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd dump --format=json 2024-04-16T17:02:12.418 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:12.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:12 smithi043 ceph-mon[36901]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-04-16T17:02:12.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:12 smithi018 ceph-mon[31776]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-04-16T17:02:12.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:12 smithi022 ceph-mon[36023]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-04-16T17:02:14.402 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:14 smithi018 ceph-mon[31776]: pgmap v343: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-16T17:02:14.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:14 smithi043 ceph-mon[36901]: pgmap v343: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-16T17:02:14.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:14 smithi022 ceph-mon[36023]: pgmap v343: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 -- 172.21.15.18:0/2930268170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c109d30 msgr2=0x7f7b2c10dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2930268170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c109d30 0x7f7b2c10dd80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7b180099c0 tx=0x7f7b18009cd0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 -- 172.21.15.18:0/2930268170 shutdown_connections 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2930268170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b2c109380 0x7f7b2c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2930268170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2c10e460 0x7f7b2c116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2930268170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c109d30 0x7f7b2c10dd80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 -- 172.21.15.18:0/2930268170 >> 172.21.15.18:0/2930268170 conn(0x7f7b2c0762c0 msgr2=0x7f7b2c107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 -- 172.21.15.18:0/2930268170 shutdown_connections 2024-04-16T17:02:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.698+0000 7f7b31e00700 1 -- 172.21.15.18:0/2930268170 wait complete. 2024-04-16T17:02:14.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.699+0000 7f7b31e00700 1 Processor -- start 2024-04-16T17:02:14.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.699+0000 7f7b31e00700 1 -- start start 2024-04-16T17:02:14.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b31e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b2c109380 0x7f7b2c1a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:14.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b31e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2c109d30 0x7f7b2c1a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:14.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b31e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c10e460 0x7f7b2c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:14.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b31e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c119490 con 0x7f7b2c109d30 2024-04-16T17:02:14.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b31e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c119310 con 0x7f7b2c109380 2024-04-16T17:02:14.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b31e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c119610 con 0x7f7b2c10e460 2024-04-16T17:02:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b2a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2c109d30 0x7f7b2c1a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b2b400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c10e460 0x7f7b2c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b2aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b2c109380 0x7f7b2c1a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b2b400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c10e460 0x7f7b2c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:33826/0 (socket says 172.21.15.18:33826) 2024-04-16T17:02:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b2b400700 1 -- 172.21.15.18:0/2956270158 learned_addr learned my addr 172.21.15.18:0/2956270158 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.700+0000 7f7b2a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2c109d30 0x7f7b2c1a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41904/0 (socket says 172.21.15.18:41904) 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.701+0000 7f7b2b400700 1 -- 172.21.15.18:0/2956270158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b2c109380 msgr2=0x7f7b2c1a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.701+0000 7f7b2b400700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b2c109380 0x7f7b2c1a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.701+0000 7f7b2b400700 1 -- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2c109d30 msgr2=0x7f7b2c1a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.701+0000 7f7b2b400700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2c109d30 0x7f7b2c1a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.701+0000 7f7b2b400700 1 -- 172.21.15.18:0/2956270158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c1a0030 con 0x7f7b2c10e460 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.701+0000 7f7b2a000700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2c109d30 0x7f7b2c1a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.701+0000 7f7b2b400700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c10e460 0x7f7b2c19f750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c370 tx=0x7f7b2400c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.702+0000 7f7b30a00700 1 -- 172.21.15.18:0/2956270158 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240092e0 con 0x7f7b2c10e460 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.702+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c1a02c0 con 0x7f7b2c10e460 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.702+0000 7f7b30a00700 1 -- 172.21.15.18:0/2956270158 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b24003a00 con 0x7f7b2c10e460 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.702+0000 7f7b30a00700 1 -- 172.21.15.18:0/2956270158 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240047c0 con 0x7f7b2c10e460 2024-04-16T17:02:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.702+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c1ac430 con 0x7f7b2c10e460 2024-04-16T17:02:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.703+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2c10a860 con 0x7f7b2c10e460 2024-04-16T17:02:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.705+0000 7f7b30a00700 1 -- 172.21.15.18:0/2956270158 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7b2401a020 con 0x7f7b2c10e460 2024-04-16T17:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.706+0000 7f7b30a00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7b100a3cd0 0x7f7b100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.706+0000 7f7b30a00700 1 -- 172.21.15.18:0/2956270158 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7b240d0090 con 0x7f7b2c10e460 2024-04-16T17:02:14.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.707+0000 7f7b2aa00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7b100a3cd0 0x7f7b100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.707+0000 7f7b2aa00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7b100a3cd0 0x7f7b100a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009c60 tx=0x7f7b1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:14.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.712+0000 7f7b30a00700 1 -- 172.21.15.18:0/2956270158 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b24010040 con 0x7f7b2c10e460 2024-04-16T17:02:14.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.968+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b2c02dfe0 con 0x7f7b2c10e460 2024-04-16T17:02:14.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.971+0000 7f7b30a00700 1 -- 172.21.15.18:0/2956270158 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7f7b2409c670 con 0x7f7b2c10e460 2024-04-16T17:02:14.973 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:02:14.973 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:02:14.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7b100a3cd0 msgr2=0x7f7b100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:14.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7b100a3cd0 0x7f7b100a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009c60 tx=0x7f7b1c009260 comp rx=0 tx=0).stop 2024-04-16T17:02:14.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c10e460 msgr2=0x7f7b2c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:14.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c10e460 0x7f7b2c19f750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c370 tx=0x7f7b2400c680 comp rx=0 tx=0).stop 2024-04-16T17:02:14.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 shutdown_connections 2024-04-16T17:02:14.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7b100a3cd0 0x7f7b100a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b2c109380 0x7f7b2c1a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b2c10e460 0x7f7b2c19f750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 --2- 172.21.15.18:0/2956270158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b2c109d30 0x7f7b2c1a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:14.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.974+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 >> 172.21.15.18:0/2956270158 conn(0x7f7b2c0762c0 msgr2=0x7f7b2c0768e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:14.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.975+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 shutdown_connections 2024-04-16T17:02:14.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:14.975+0000 7f7b31e00700 1 -- 172.21.15.18:0/2956270158 wait complete. 2024-04-16T17:02:15.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:15 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2956270158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:15.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:15 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2956270158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:15.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:15 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2956270158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:15.773 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-16T17:02:15.773 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd dump --format=json 2024-04-16T17:02:16.253 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:16.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:16 smithi018 ceph-mon[31776]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-16T17:02:16.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:16 smithi022 ceph-mon[36023]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-16T17:02:16.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:16 smithi043 ceph-mon[36901]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-16T17:02:18.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 -- 172.21.15.18:0/417976891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 msgr2=0x7f8b4810ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:18.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 --2- 172.21.15.18:0/417976891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 0x7f8b4810ed20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009990 tx=0x7f8b38009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:18 smithi018 ceph-mon[31776]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-16T17:02:18.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 -- 172.21.15.18:0/417976891 shutdown_connections 2024-04-16T17:02:18.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 --2- 172.21.15.18:0/417976891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4810f260 0x7f8b481116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 --2- 172.21.15.18:0/417976891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b48074f20 0x7f8b48075300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 --2- 172.21.15.18:0/417976891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 0x7f8b4810ed20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 -- 172.21.15.18:0/417976891 >> 172.21.15.18:0/417976891 conn(0x7f8b480fe570 msgr2=0x7f8b48100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:18.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 -- 172.21.15.18:0/417976891 shutdown_connections 2024-04-16T17:02:18.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.521+0000 7f8b4f200700 1 -- 172.21.15.18:0/417976891 wait complete. 2024-04-16T17:02:18.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.523+0000 7f8b4f200700 1 Processor -- start 2024-04-16T17:02:18.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.523+0000 7f8b4f200700 1 -- start start 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4f200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b48074f20 0x7f8b4819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 0x7f8b4819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4f200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4810f260 0x7f8b4819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48114f90 con 0x7f8b48075840 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4f200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48114e10 con 0x7f8b4810f260 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4f200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48115110 con 0x7f8b48074f20 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 0x7f8b4819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b48074f20 0x7f8b4819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:18.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 0x7f8b4819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52812/0 (socket says 172.21.15.18:52812) 2024-04-16T17:02:18.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4d400700 1 -- 172.21.15.18:0/3422413305 learned_addr learned my addr 172.21.15.18:0/3422413305 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:18.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.524+0000 7f8b4e800700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4810f260 0x7f8b4819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4de00700 1 -- 172.21.15.18:0/3422413305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4810f260 msgr2=0x7f8b4819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4de00700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4810f260 0x7f8b4819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4de00700 1 -- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 msgr2=0x7f8b4819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4de00700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 0x7f8b4819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4de00700 1 -- 172.21.15.18:0/3422413305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b481a0400 con 0x7f8b48074f20 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4e800700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4810f260 0x7f8b4819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4d400700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 0x7f8b4819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4de00700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b48074f20 0x7f8b4819b440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e9f0 tx=0x7f8b3c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b42a00700 1 -- 172.21.15.18:0/3422413305 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00cb70 con 0x7f8b48074f20 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b42a00700 1 -- 172.21.15.18:0/3422413305 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3c00ee90 con 0x7f8b48074f20 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b481a06f0 con 0x7f8b48074f20 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.525+0000 7f8b42a00700 1 -- 172.21.15.18:0/3422413305 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c003dc0 con 0x7f8b48074f20 2024-04-16T17:02:18.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.526+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b481a7f70 con 0x7f8b48074f20 2024-04-16T17:02:18.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.527+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b480697e0 con 0x7f8b48074f20 2024-04-16T17:02:18.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.529+0000 7f8b42a00700 1 -- 172.21.15.18:0/3422413305 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b3c004000 con 0x7f8b48074f20 2024-04-16T17:02:18.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.530+0000 7f8b42a00700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b300a3d50 0x7f8b300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:18.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.530+0000 7f8b42a00700 1 -- 172.21.15.18:0/3422413305 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f8b3c0d08c0 con 0x7f8b48074f20 2024-04-16T17:02:18.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.530+0000 7f8b4d400700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b300a3d50 0x7f8b300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.531+0000 7f8b4d400700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b300a3d50 0x7f8b300a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009750 tx=0x7f8b3800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:18.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.535+0000 7f8b42a00700 1 -- 172.21.15.18:0/3422413305 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b3c09cea0 con 0x7f8b48074f20 2024-04-16T17:02:18.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:18 smithi022 ceph-mon[36023]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-16T17:02:18.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:18 smithi043 ceph-mon[36901]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-16T17:02:18.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.767+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b4802dfe0 con 0x7f8b48074f20 2024-04-16T17:02:18.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.770+0000 7f8b42a00700 1 -- 172.21.15.18:0/3422413305 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24034 (secure 0 0 0) 0x7f8b3c09c7a0 con 0x7f8b48074f20 2024-04-16T17:02:18.771 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:02:18.771 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":89,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:01:29.487941+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:02:18.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b300a3d50 msgr2=0x7f8b300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:18.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b300a3d50 0x7f8b300a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009750 tx=0x7f8b3800b280 comp rx=0 tx=0).stop 2024-04-16T17:02:18.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b48074f20 msgr2=0x7f8b4819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:18.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b48074f20 0x7f8b4819b440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e9f0 tx=0x7f8b3c00ed00 comp rx=0 tx=0).stop 2024-04-16T17:02:18.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 shutdown_connections 2024-04-16T17:02:18.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b300a3d50 0x7f8b300a6210 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4810f260 0x7f8b4819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b48074f20 0x7f8b4819b440 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 --2- 172.21.15.18:0/3422413305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b48075840 0x7f8b4819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:18.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.772+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 >> 172.21.15.18:0/3422413305 conn(0x7f8b480fe570 msgr2=0x7f8b480ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:18.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.773+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 shutdown_connections 2024-04-16T17:02:18.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:18.773+0000 7f8b4f200700 1 -- 172.21.15.18:0/3422413305 wait complete. 2024-04-16T17:02:19.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:19 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3422413305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:19.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-16T17:02:19.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-16T17:02:19.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-16T17:02:19.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-16T17:02:19.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-16T17:02:19.627 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-16T17:02:19.627 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-16T17:02:19.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-16T17:02:19.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-16T17:02:19.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-16T17:02:19.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-16T17:02:19.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-16T17:02:19.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:19 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3422413305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:19.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:19 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3422413305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:20.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:20 smithi022 ceph-mon[36023]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:02:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:20 smithi018 ceph-mon[31776]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:02:20.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:20 smithi043 ceph-mon[36901]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:02:20.827 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.835 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.859 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.862 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.862 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.866 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.869 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.874 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.878 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.880 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:20.898 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:22 smithi018 ceph-mon[31776]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-16T17:02:22.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:22 smithi022 ceph-mon[36023]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-16T17:02:22.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:22 smithi043 ceph-mon[36901]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-16T17:02:24.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:24 smithi018 ceph-mon[31776]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:02:24.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:24 smithi022 ceph-mon[36023]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:02:24.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:24 smithi043 ceph-mon[36901]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:02:25.441 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-16T17:02:25.441 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-16T17:02:25.441 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-16T17:02:25.441 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-16T17:02:25.441 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-04-16T17:02:25.441 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-16T17:02:25.441 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-16T17:02:25.441 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-16T17:02:25.442 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-16T17:02:25.442 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:25 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T17:02:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-16T17:02:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-16T17:02:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-16T17:02:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-16T17:02:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-04-16T17:02:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-16T17:02:25.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-16T17:02:25.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-16T17:02:25.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-16T17:02:25.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:25 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T17:02:25.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-16T17:02:25.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-16T17:02:25.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-16T17:02:25.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-16T17:02:25.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-04-16T17:02:25.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-16T17:02:25.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-16T17:02:25.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-16T17:02:25.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-16T17:02:25.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:25 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T17:02:26.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T17:02:26.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T17:02:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T17:02:26.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:27.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:27 smithi018 ceph-mon[31776]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T17:02:27.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:27 smithi022 ceph-mon[36023]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T17:02:27.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:27 smithi043 ceph-mon[36901]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T17:02:28.718 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:28 smithi018 ceph-mon[31776]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s 2024-04-16T17:02:28.718 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:28.718 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:28.719 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:28.719 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:28 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:28.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:28 smithi022 ceph-mon[36023]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s 2024-04-16T17:02:28.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:28.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:28.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:28.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:28 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:29.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:28 smithi043 ceph-mon[36901]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s 2024-04-16T17:02:29.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:29.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:29.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:29.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:28 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:02:30.877 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:30 smithi018 ceph-mon[31776]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s; 77 B/s, 0 objects/s recovering 2024-04-16T17:02:30.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:30 smithi022 ceph-mon[36023]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s; 77 B/s, 0 objects/s recovering 2024-04-16T17:02:31.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:30 smithi043 ceph-mon[36901]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s; 77 B/s, 0 objects/s recovering 2024-04-16T17:02:32.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:32 smithi018 ceph-mon[31776]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s; 153 B/s, 0 objects/s recovering 2024-04-16T17:02:32.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:32 smithi022 ceph-mon[36023]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s; 153 B/s, 0 objects/s recovering 2024-04-16T17:02:33.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:32 smithi043 ceph-mon[36901]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s; 153 B/s, 0 objects/s recovering 2024-04-16T17:02:33.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 -- 172.21.15.18:0/4185020854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb581022e0 msgr2=0x7feb5810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:33.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 --2- 172.21.15.18:0/4185020854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb581022e0 0x7feb5810e7f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb50009f90 tx=0x7feb5000b330 comp rx=0 tx=0).stop 2024-04-16T17:02:33.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 -- 172.21.15.18:0/4185020854 shutdown_connections 2024-04-16T17:02:33.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 --2- 172.21.15.18:0/4185020854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58107920 0x7feb58107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:33.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 --2- 172.21.15.18:0/4185020854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb581022e0 0x7feb5810e7f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 --2- 172.21.15.18:0/4185020854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb58101920 0x7feb58101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 -- 172.21.15.18:0/4185020854 >> 172.21.15.18:0/4185020854 conn(0x7feb580fd410 msgr2=0x7feb580ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 -- 172.21.15.18:0/4185020854 shutdown_connections 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.091+0000 7feb5d000700 1 -- 172.21.15.18:0/4185020854 wait complete. 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.092+0000 7feb5d000700 1 Processor -- start 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.092+0000 7feb5d000700 1 -- start start 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb5d000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58101920 0x7feb58199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb5d000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb581022e0 0x7feb58199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb5d000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb58107920 0x7feb5819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb5d000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feb58111d00 con 0x7feb581022e0 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb5d000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb58111b80 con 0x7feb58101920 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb5d000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feb58111e80 con 0x7feb58107920 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb4f400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58101920 0x7feb58199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb57400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb581022e0 0x7feb58199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb4f400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58101920 0x7feb58199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45884/0 (socket says 172.21.15.18:45884) 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb4f400700 1 -- 172.21.15.18:0/3512766316 learned_addr learned my addr 172.21.15.18:0/3512766316 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb57e00700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb58107920 0x7feb5819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb4f400700 1 -- 172.21.15.18:0/3512766316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb58107920 msgr2=0x7feb5819da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb4f400700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb58107920 0x7feb5819da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb4f400700 1 -- 172.21.15.18:0/3512766316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb581022e0 msgr2=0x7feb58199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.093+0000 7feb4f400700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb581022e0 0x7feb58199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.094+0000 7feb4f400700 1 -- 172.21.15.18:0/3512766316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5819e150 con 0x7feb58101920 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.094+0000 7feb57400700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb581022e0 0x7feb58199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.094+0000 7feb4f400700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58101920 0x7feb58199140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb4800eb30 tx=0x7feb4800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.094+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4800cbd0 con 0x7feb58101920 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.094+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb48004500 con 0x7feb58101920 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.094+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5819e440 con 0x7feb58101920 2024-04-16T17:02:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.094+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48010430 con 0x7feb58101920 2024-04-16T17:02:33.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.094+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb581a5cc0 con 0x7feb58101920 2024-04-16T17:02:33.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.095+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb58063280 con 0x7feb58101920 2024-04-16T17:02:33.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.098+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feb48003f00 con 0x7feb58101920 2024-04-16T17:02:33.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.099+0000 7feb54c00700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feb3c0a3d50 0x7feb3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:33.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.099+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7feb480d0e30 con 0x7feb58101920 2024-04-16T17:02:33.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.099+0000 7feb57400700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feb3c0a3d50 0x7feb3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.100+0000 7feb54c00700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] conn(0x7feb3c0aeeb0 0x7feb3c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.100+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 --> [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb4801ee50 con 0x7feb3c0aeeb0 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.100+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7feb480d11e0 con 0x7feb58101920 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.100+0000 7feb57e00700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] conn(0x7feb3c0aeeb0 0x7feb3c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.100+0000 7feb57400700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feb3c0a3d50 0x7feb3c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb40009750 tx=0x7feb40009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.101+0000 7feb57e00700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] conn(0x7feb3c0aeeb0 0x7feb3c0b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.102+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 <== osd.8 v2:172.21.15.43:6816/546535747 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb4801ee50 con 0x7feb3c0aeeb0 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.120+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 --> [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb58063680 con 0x7feb3c0aeeb0 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.121+0000 7feb54c00700 1 -- 172.21.15.18:0/3512766316 <== osd.8 v2:172.21.15.43:6816/546535747 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feb58063680 con 0x7feb3c0aeeb0 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.121+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 >> [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] conn(0x7feb3c0aeeb0 msgr2=0x7feb3c0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.121+0000 7feb5d000700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] conn(0x7feb3c0aeeb0 0x7feb3c0b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.121+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feb3c0a3d50 msgr2=0x7feb3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.121+0000 7feb5d000700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feb3c0a3d50 0x7feb3c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb40009750 tx=0x7feb40009310 comp rx=0 tx=0).stop 2024-04-16T17:02:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.121+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58101920 msgr2=0x7feb58199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.122+0000 7feb5d000700 1 --2- 172.21.15.18:0/3512766316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb58101920 0x7feb58199140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb4800eb30 tx=0x7feb4800ee40 comp rx=0 tx=0).stop 2024-04-16T17:02:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.122+0000 7feb57e00700 1 -- 172.21.15.18:0/3512766316 reap_dead start 2024-04-16T17:02:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.122+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 shutdown_connections 2024-04-16T17:02:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.122+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 >> 172.21.15.18:0/3512766316 conn(0x7feb580fd410 msgr2=0x7feb5806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.122+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 shutdown_connections 2024-04-16T17:02:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.122+0000 7feb5d000700 1 -- 172.21.15.18:0/3512766316 wait complete. 2024-04-16T17:02:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.192+0000 7f8b9ec00700 1 -- 172.21.15.18:0/3547021798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b98075640 msgr2=0x7f8b9810efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.192+0000 7f8b9ec00700 1 --2- 172.21.15.18:0/3547021798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b98075640 0x7f8b9810efe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8b800098f0 tx=0x7f8b80009c00 comp rx=0 tx=0).stop 2024-04-16T17:02:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.192+0000 7f8b9ec00700 1 -- 172.21.15.18:0/3547021798 shutdown_connections 2024-04-16T17:02:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.192+0000 7f8b9ec00700 1 --2- 172.21.15.18:0/3547021798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b9810f520 0x7f8b98111910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.192+0000 7f8b9ec00700 1 --2- 172.21.15.18:0/3547021798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b98074d20 0x7f8b98075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.192+0000 7f8b9ec00700 1 --2- 172.21.15.18:0/3547021798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b98075640 0x7f8b9810efe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.192+0000 7f8b9ec00700 1 -- 172.21.15.18:0/3547021798 >> 172.21.15.18:0/3547021798 conn(0x7f8b98079bc0 msgr2=0x7f8b98079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.192+0000 7f8b9ec00700 1 -- 172.21.15.18:0/3547021798 shutdown_connections 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.193+0000 7f8b9ec00700 1 -- 172.21.15.18:0/3547021798 wait complete. 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.194+0000 7f8b9ec00700 1 Processor -- start 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.194+0000 7f8b9ec00700 1 -- start start 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.194+0000 7f8b9ec00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b98074d20 0x7f8b9819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.194+0000 7f8b9ec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b98075640 0x7f8b9819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.194+0000 7f8b9ec00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9810f520 0x7f8b9819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.194+0000 7f8b9ec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98115050 con 0x7f8b98075640 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.194+0000 7f8b9ec00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98114ed0 con 0x7f8b98074d20 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.194+0000 7f8b9ec00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981151d0 con 0x7f8b9810f520 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b97e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b98074d20 0x7f8b9819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b97e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b98074d20 0x7f8b9819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45912/0 (socket says 172.21.15.18:45912) 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b97e00700 1 -- 172.21.15.18:0/2219495141 learned_addr learned my addr 172.21.15.18:0/2219495141 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b9cc00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9810f520 0x7f8b9819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b8fe00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b98075640 0x7f8b9819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b9cc00700 1 -- 172.21.15.18:0/2219495141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b98074d20 msgr2=0x7f8b9819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b9cc00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b98074d20 0x7f8b9819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b9cc00700 1 -- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b98075640 msgr2=0x7f8b9819b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b9cc00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b98075640 0x7f8b9819b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b9cc00700 1 -- 172.21.15.18:0/2219495141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b981a0490 con 0x7f8b9810f520 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.195+0000 7f8b9cc00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9810f520 0x7f8b9819fd50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000ef90 tx=0x7f8b9000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.196+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b900092e0 con 0x7f8b9810f520 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.196+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b90004bb0 con 0x7f8b9810f520 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.196+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b981a0720 con 0x7f8b9810f520 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.196+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90004280 con 0x7f8b9810f520 2024-04-16T17:02:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.196+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981a8000 con 0x7f8b9810f520 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.197+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b98100750 con 0x7f8b9810f520 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.199+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b9001b030 con 0x7f8b9810f520 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.200+0000 7f8b95600700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b7c0a3cd0 0x7f8b7c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.201+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8b900d0e90 con 0x7f8b9810f520 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.201+0000 7f8b97e00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b7c0a3cd0 0x7f8b7c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.201+0000 7f8b95600700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] conn(0x7f8b7c0aee30 0x7f8b7c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.201+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 --> [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b90003dd0 con 0x7f8b7c0aee30 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.201+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8b900d1240 con 0x7f8b9810f520 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.202+0000 7f8b8fe00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] conn(0x7f8b7c0aee30 0x7f8b7c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.202+0000 7f8b97e00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b7c0a3cd0 0x7f8b7c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005950 tx=0x7f8b88009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.202+0000 7f8b8fe00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] conn(0x7f8b7c0aee30 0x7f8b7c0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.203+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 <== osd.9 v2:172.21.15.18:6826/3050280223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b90003dd0 con 0x7f8b7c0aee30 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.221+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 --> [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b98076410 con 0x7f8b7c0aee30 2024-04-16T17:02:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b95600700 1 -- 172.21.15.18:0/2219495141 <== osd.9 v2:172.21.15.18:6826/3050280223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8b98076410 con 0x7f8b7c0aee30 2024-04-16T17:02:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] conn(0x7f8b7c0aee30 msgr2=0x7f8b7c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] conn(0x7f8b7c0aee30 0x7f8b7c0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b7c0a3cd0 msgr2=0x7f8b7c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f8b7c0a3cd0 0x7f8b7c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005950 tx=0x7f8b88009310 comp rx=0 tx=0).stop 2024-04-16T17:02:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9810f520 msgr2=0x7f8b9819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 --2- 172.21.15.18:0/2219495141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9810f520 0x7f8b9819fd50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000ef90 tx=0x7f8b9000c3a0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9cc00700 1 -- 172.21.15.18:0/2219495141 reap_dead start 2024-04-16T17:02:34.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 shutdown_connections 2024-04-16T17:02:34.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 >> 172.21.15.18:0/2219495141 conn(0x7f8b98079bc0 msgr2=0x7f8b980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:34.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 shutdown_connections 2024-04-16T17:02:34.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:33.222+0000 7f8b9ec00700 1 -- 172.21.15.18:0/2219495141 wait complete. 2024-04-16T17:02:34.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:34 smithi018 ceph-mon[31776]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s; 153 B/s, 0 objects/s recovering 2024-04-16T17:02:34.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:34 smithi022 ceph-mon[36023]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s; 153 B/s, 0 objects/s recovering 2024-04-16T17:02:35.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:34 smithi043 ceph-mon[36901]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s; 153 B/s, 0 objects/s recovering 2024-04-16T17:02:36.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.684+0000 7f6f42c00700 1 -- 172.21.15.18:0/1133763592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c06a800 msgr2=0x7f6f3c06ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.684+0000 7f6f42c00700 1 --2- 172.21.15.18:0/1133763592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c06a800 0x7f6f3c06ac60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6f280098f0 tx=0x7f6f28009c00 comp rx=0 tx=0).stop 2024-04-16T17:02:36.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.685+0000 7f6f42c00700 1 -- 172.21.15.18:0/1133763592 shutdown_connections 2024-04-16T17:02:36.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.685+0000 7f6f42c00700 1 --2- 172.21.15.18:0/1133763592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f3c108b70 0x7f6f3c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.685+0000 7f6f42c00700 1 --2- 172.21.15.18:0/1133763592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c06b1a0 0x7f6f3c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.685+0000 7f6f42c00700 1 --2- 172.21.15.18:0/1133763592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c06a800 0x7f6f3c06ac60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.685+0000 7f6f42c00700 1 -- 172.21.15.18:0/1133763592 >> 172.21.15.18:0/1133763592 conn(0x7f6f3c0789a0 msgr2=0x7f6f3c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:36.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.685+0000 7f6f42c00700 1 -- 172.21.15.18:0/1133763592 shutdown_connections 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.685+0000 7f6f42c00700 1 -- 172.21.15.18:0/1133763592 wait complete. 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.686+0000 7f6f42c00700 1 Processor -- start 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.686+0000 7f6f42c00700 1 -- start start 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.686+0000 7f6f42c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c06b1a0 0x7f6f3c101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.686+0000 7f6f42c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c108b70 0x7f6f3c102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f42c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f3c102540 0x7f6f3c1a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f42c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c112e20 con 0x7f6f3c108b70 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f42c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c112ca0 con 0x7f6f3c102540 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f42c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c112fa0 con 0x7f6f3c06b1a0 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c108b70 0x7f6f3c102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c108b70 0x7f6f3c102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45628/0 (socket says 172.21.15.18:45628) 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3b400700 1 -- 172.21.15.18:0/3136755038 learned_addr learned my addr 172.21.15.18:0/3136755038 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3be00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c06b1a0 0x7f6f3c101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f40c00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f3c102540 0x7f6f3c1a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3b400700 1 -- 172.21.15.18:0/3136755038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c06b1a0 msgr2=0x7f6f3c101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3b400700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c06b1a0 0x7f6f3c101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3b400700 1 -- 172.21.15.18:0/3136755038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f3c102540 msgr2=0x7f6f3c1a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3b400700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f3c102540 0x7f6f3c1a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.687+0000 7f6f3b400700 1 -- 172.21.15.18:0/3136755038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c1a7eb0 con 0x7f6f3c108b70 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.688+0000 7f6f3b400700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c108b70 0x7f6f3c102000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6f28000c00 tx=0x7f6f2800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.688+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28005040 con 0x7f6f3c108b70 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.688+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f280056c0 con 0x7f6f3c108b70 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.688+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c1a8140 con 0x7f6f3c108b70 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.688+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28003a00 con 0x7f6f3c108b70 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.688+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c1a8540 con 0x7f6f3c108b70 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.689+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f3c063200 con 0x7f6f3c108b70 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.691+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6f2800b810 con 0x7f6f3c108b70 2024-04-16T17:02:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.693+0000 7f6f38c00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6f200a3cd0 0x7f6f200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.693+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6f280d1bf0 con 0x7f6f3c108b70 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.693+0000 7f6f3be00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6f200a3cd0 0x7f6f200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.693+0000 7f6f38c00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] conn(0x7f6f200aed00 0x7f6f200b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.693+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 --> [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f2801bec0 con 0x7f6f200aed00 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.693+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6f280d1fa0 con 0x7f6f3c108b70 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.694+0000 7f6f40c00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] conn(0x7f6f200aed00 0x7f6f200b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.694+0000 7f6f3be00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6f200a3cd0 0x7f6f200a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009be0 tx=0x7f6f2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.694+0000 7f6f40c00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] conn(0x7f6f200aed00 0x7f6f200b11a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.695+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 <== osd.5 v2:172.21.15.43:6808/3934456075 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f2801bec0 con 0x7f6f200aed00 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.714+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 --> [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f3c063680 con 0x7f6f200aed00 2024-04-16T17:02:36.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.715+0000 7f6f38c00700 1 -- 172.21.15.18:0/3136755038 <== osd.5 v2:172.21.15.43:6808/3934456075 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6f3c063680 con 0x7f6f200aed00 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.715+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 >> [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] conn(0x7f6f200aed00 msgr2=0x7f6f200b11a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.715+0000 7f6f42c00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] conn(0x7f6f200aed00 0x7f6f200b11a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.715+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6f200a3cd0 msgr2=0x7f6f200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.715+0000 7f6f42c00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f6f200a3cd0 0x7f6f200a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009be0 tx=0x7f6f2c009260 comp rx=0 tx=0).stop 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.715+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c108b70 msgr2=0x7f6f3c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.715+0000 7f6f42c00700 1 --2- 172.21.15.18:0/3136755038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f3c108b70 0x7f6f3c102000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6f28000c00 tx=0x7f6f2800fa00 comp rx=0 tx=0).stop 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.716+0000 7f6f40c00700 1 -- 172.21.15.18:0/3136755038 reap_dead start 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.716+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 shutdown_connections 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.716+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 >> 172.21.15.18:0/3136755038 conn(0x7f6f3c0789a0 msgr2=0x7f6f3c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.716+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 shutdown_connections 2024-04-16T17:02:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:35.716+0000 7f6f42c00700 1 -- 172.21.15.18:0/3136755038 wait complete. 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 -- 172.21.15.18:0/2150578722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 msgr2=0x7fa9f4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 --2- 172.21.15.18:0/2150578722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 0x7fa9f4109990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009990 tx=0x7fa9e0009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 -- 172.21.15.18:0/2150578722 shutdown_connections 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 --2- 172.21.15.18:0/2150578722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9f410a070 0x7fa9f4111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 --2- 172.21.15.18:0/2150578722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f4104f90 0x7fa9f4105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 --2- 172.21.15.18:0/2150578722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 0x7fa9f4109990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 -- 172.21.15.18:0/2150578722 >> 172.21.15.18:0/2150578722 conn(0x7fa9f4100820 msgr2=0x7fa9f4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 -- 172.21.15.18:0/2150578722 shutdown_connections 2024-04-16T17:02:36.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.689+0000 7fa9fba00700 1 -- 172.21.15.18:0/2150578722 wait complete. 2024-04-16T17:02:36.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.690+0000 7fa9fba00700 1 Processor -- start 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.690+0000 7fa9fba00700 1 -- start start 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.690+0000 7fa9fba00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f4104f90 0x7fa9f419b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.690+0000 7fa9fba00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 0x7fa9f419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.690+0000 7fa9fba00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9f410a070 0x7fa9f419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9fba00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4115030 con 0x7fa9f410a070 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9fba00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4114eb0 con 0x7fa9f4105940 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9fba00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41151b0 con 0x7fa9f4104f90 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f9000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f4104f90 0x7fa9f419b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9f410a070 0x7fa9f419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 0x7fa9f419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9f410a070 0x7fa9f419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45648/0 (socket says 172.21.15.18:45648) 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f9a00700 1 -- 172.21.15.18:0/3980050712 learned_addr learned my addr 172.21.15.18:0/3980050712 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:36.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 0x7fa9f419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45958/0 (socket says 172.21.15.18:45958) 2024-04-16T17:02:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f3e00700 1 -- 172.21.15.18:0/3980050712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f4104f90 msgr2=0x7fa9f419b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f3e00700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f4104f90 0x7fa9f419b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f3e00700 1 -- 172.21.15.18:0/3980050712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9f410a070 msgr2=0x7fa9f419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f3e00700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9f410a070 0x7fa9f419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.691+0000 7fa9f3e00700 1 -- 172.21.15.18:0/3980050712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f41a0510 con 0x7fa9f4105940 2024-04-16T17:02:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.692+0000 7fa9f3e00700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 0x7fa9f419ba40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00098e0 tx=0x7fa9e00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.692+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0027070 con 0x7fa9f4105940 2024-04-16T17:02:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.692+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e000be90 con 0x7fa9f4105940 2024-04-16T17:02:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.692+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00173f0 con 0x7fa9f4105940 2024-04-16T17:02:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.692+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f41a07a0 con 0x7fa9f4105940 2024-04-16T17:02:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.692+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41a8080 con 0x7fa9f4105940 2024-04-16T17:02:36.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.693+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9f4063280 con 0x7fa9f4105940 2024-04-16T17:02:36.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.695+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa9e002a070 con 0x7fa9f4105940 2024-04-16T17:02:36.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.697+0000 7fa9f1600700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9dc0a3d50 0x7fa9dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.697+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa9e00d1ac0 con 0x7fa9f4105940 2024-04-16T17:02:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.697+0000 7fa9f9000700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9dc0a3d50 0x7fa9dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.697+0000 7fa9f1600700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] conn(0x7fa9dc0aeeb0 0x7fa9dc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:36.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.697+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 --> [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9e000eea0 con 0x7fa9dc0aeeb0 2024-04-16T17:02:36.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.697+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa9e00d1e70 con 0x7fa9f4105940 2024-04-16T17:02:36.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.698+0000 7fa9f9a00700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] conn(0x7fa9dc0aeeb0 0x7fa9dc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:36.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.698+0000 7fa9f9000700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9dc0a3d50 0x7fa9dc0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009c60 tx=0x7fa9e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:36.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.698+0000 7fa9f9a00700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] conn(0x7fa9dc0aeeb0 0x7fa9dc0b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:36.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.699+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 <== osd.10 v2:172.21.15.22:6824/3445736292 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9e000eea0 con 0x7fa9dc0aeeb0 2024-04-16T17:02:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.718+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 --> [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9f4063680 con 0x7fa9dc0aeeb0 2024-04-16T17:02:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.719+0000 7fa9f1600700 1 -- 172.21.15.18:0/3980050712 <== osd.10 v2:172.21.15.22:6824/3445736292 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa9f4063680 con 0x7fa9dc0aeeb0 2024-04-16T17:02:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.719+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 >> [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] conn(0x7fa9dc0aeeb0 msgr2=0x7fa9dc0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.719+0000 7fa9fba00700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] conn(0x7fa9dc0aeeb0 0x7fa9dc0b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9dc0a3d50 msgr2=0x7fa9dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9fba00700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa9dc0a3d50 0x7fa9dc0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009c60 tx=0x7fa9e8009260 comp rx=0 tx=0).stop 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 msgr2=0x7fa9f419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9fba00700 1 --2- 172.21.15.18:0/3980050712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9f4105940 0x7fa9f419ba40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00098e0 tx=0x7fa9e00047b0 comp rx=0 tx=0).stop 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9f9a00700 1 -- 172.21.15.18:0/3980050712 reap_dead start 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 shutdown_connections 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 >> 172.21.15.18:0/3980050712 conn(0x7fa9f4100820 msgr2=0x7fa9f4101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 shutdown_connections 2024-04-16T17:02:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:36.720+0000 7fa9fba00700 1 -- 172.21.15.18:0/3980050712 wait complete. 2024-04-16T17:02:36.770 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:36 smithi018 ceph-mon[31776]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 213 B/s wr, 5 op/s; 127 B/s, 0 objects/s recovering 2024-04-16T17:02:36.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:36 smithi022 ceph-mon[36023]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 213 B/s wr, 5 op/s; 127 B/s, 0 objects/s recovering 2024-04-16T17:02:37.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:36 smithi043 ceph-mon[36901]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 213 B/s wr, 5 op/s; 127 B/s, 0 objects/s recovering 2024-04-16T17:02:37.240 INFO:teuthology.orchestra.run.smithi018.stdout:163208757311 2024-04-16T17:02:37.240 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-16T17:02:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 -- 172.21.15.18:0/2342519278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5298077940 msgr2=0x7f5298113490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 --2- 172.21.15.18:0/2342519278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5298077940 0x7f5298113490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f528c00b0a0 tx=0x7f528c00b3b0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 -- 172.21.15.18:0/2342519278 shutdown_connections 2024-04-16T17:02:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 --2- 172.21.15.18:0/2342519278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5298077020 0x7f5298077400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 --2- 172.21.15.18:0/2342519278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52981139d0 0x7f5298115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 --2- 172.21.15.18:0/2342519278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5298077940 0x7f5298113490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 -- 172.21.15.18:0/2342519278 >> 172.21.15.18:0/2342519278 conn(0x7f5298100820 msgr2=0x7f5298102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 -- 172.21.15.18:0/2342519278 shutdown_connections 2024-04-16T17:02:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.504+0000 7f529fe00700 1 -- 172.21.15.18:0/2342519278 wait complete. 2024-04-16T17:02:38.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.505+0000 7f529fe00700 1 Processor -- start 2024-04-16T17:02:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.505+0000 7f529fe00700 1 -- start start 2024-04-16T17:02:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5298077020 0x7f52981a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529fe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5298077940 0x7f52981a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52981139d0 0x7f529819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529fe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f52981194c0 con 0x7f5298077020 2024-04-16T17:02:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529fe00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5298119340 con 0x7f5298077940 2024-04-16T17:02:38.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529fe00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5298119640 con 0x7f52981139d0 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5298077020 0x7f52981a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5298077020 0x7f52981a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45256/0 (socket says 172.21.15.18:45256) 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529d400700 1 -- 172.21.15.18:0/2770374093 learned_addr learned my addr 172.21.15.18:0/2770374093 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529ca00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5298077940 0x7f52981a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.506+0000 7f529de00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52981139d0 0x7f529819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.507+0000 7f529de00700 1 -- 172.21.15.18:0/2770374093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5298077940 msgr2=0x7f52981a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.507+0000 7f529de00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5298077940 0x7f52981a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.507+0000 7f529de00700 1 -- 172.21.15.18:0/2770374093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5298077020 msgr2=0x7f52981a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.507+0000 7f529de00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5298077020 0x7f52981a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.507+0000 7f529de00700 1 -- 172.21.15.18:0/2770374093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529819ffe0 con 0x7f52981139d0 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.507+0000 7f529d400700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5298077020 0x7f52981a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:38.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.507+0000 7f529de00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52981139d0 0x7f529819f700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f529807b310 tx=0x7f529000da00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.508+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290014030 con 0x7f52981139d0 2024-04-16T17:02:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.508+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52981a02d0 con 0x7f52981139d0 2024-04-16T17:02:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.508+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5290004500 con 0x7f52981139d0 2024-04-16T17:02:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.508+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290005430 con 0x7f52981139d0 2024-04-16T17:02:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.508+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52981ac5b0 con 0x7f52981139d0 2024-04-16T17:02:38.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.509+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5298063240 con 0x7f52981139d0 2024-04-16T17:02:38.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.511+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5290004b30 con 0x7f52981139d0 2024-04-16T17:02:38.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.513+0000 7f5296000700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f52800a3cd0 0x7f52800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.513+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f52900d0830 con 0x7f52981139d0 2024-04-16T17:02:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.513+0000 7f529d400700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f52800a3cd0 0x7f52800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.513+0000 7f5296000700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] conn(0x7f52800aee30 0x7f52800b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.513+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 --> [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52800b19a0 con 0x7f52800aee30 2024-04-16T17:02:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.513+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f52900d0be0 con 0x7f52981139d0 2024-04-16T17:02:38.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.513+0000 7f529ca00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] conn(0x7f52800aee30 0x7f52800b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.514+0000 7f529d400700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f52800a3cd0 0x7f52800a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5288011590 tx=0x7f5288009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:38.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.514+0000 7f529ca00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] conn(0x7f52800aee30 0x7f52800b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:38.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.515+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 <== osd.2 v2:172.21.15.43:6800/3809455013 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52800b19a0 con 0x7f52800aee30 2024-04-16T17:02:38.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.533+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 --> [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52980636c0 con 0x7f52800aee30 2024-04-16T17:02:38.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.534+0000 7f5296000700 1 -- 172.21.15.18:0/2770374093 <== osd.2 v2:172.21.15.43:6800/3809455013 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f52980636c0 con 0x7f52800aee30 2024-04-16T17:02:38.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.534+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] conn(0x7f52800aee30 msgr2=0x7f52800b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.534+0000 7f529fe00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] conn(0x7f52800aee30 0x7f52800b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f52800a3cd0 msgr2=0x7f52800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529fe00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f52800a3cd0 0x7f52800a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5288011590 tx=0x7f5288009260 comp rx=0 tx=0).stop 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52981139d0 msgr2=0x7f529819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529fe00700 1 --2- 172.21.15.18:0/2770374093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52981139d0 0x7f529819f700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f529807b310 tx=0x7f529000da00 comp rx=0 tx=0).stop 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529de00700 1 -- 172.21.15.18:0/2770374093 reap_dead start 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 shutdown_connections 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 >> 172.21.15.18:0/2770374093 conn(0x7f5298100820 msgr2=0x7f5298102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 shutdown_connections 2024-04-16T17:02:38.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:38.535+0000 7f529fe00700 1 -- 172.21.15.18:0/2770374093 wait complete. 2024-04-16T17:02:38.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:38 smithi018 ceph-mon[31776]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 193 B/s wr, 6 op/s; 115 B/s, 0 objects/s recovering 2024-04-16T17:02:38.876 INFO:teuthology.orchestra.run.smithi018.stdout:309237645346 2024-04-16T17:02:38.876 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-16T17:02:38.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:38 smithi022 ceph-mon[36023]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 193 B/s wr, 6 op/s; 115 B/s, 0 objects/s recovering 2024-04-16T17:02:39.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:38 smithi043 ceph-mon[36901]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 193 B/s wr, 6 op/s; 115 B/s, 0 objects/s recovering 2024-04-16T17:02:40.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 -- 172.21.15.18:0/2744752079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eac0757b0 msgr2=0x7f9eac1134f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 --2- 172.21.15.18:0/2744752079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eac0757b0 0x7f9eac1134f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009990 tx=0x7f9e98009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 -- 172.21.15.18:0/2744752079 shutdown_connections 2024-04-16T17:02:40.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 --2- 172.21.15.18:0/2744752079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eac074e90 0x7f9eac075270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 --2- 172.21.15.18:0/2744752079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac113a30 0x7f9eac115e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 --2- 172.21.15.18:0/2744752079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eac0757b0 0x7f9eac1134f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 -- 172.21.15.18:0/2744752079 >> 172.21.15.18:0/2744752079 conn(0x7f9eac0fe570 msgr2=0x7f9eac100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 -- 172.21.15.18:0/2744752079 shutdown_connections 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.236+0000 7f9eab200700 1 -- 172.21.15.18:0/2744752079 wait complete. 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.237+0000 7f9eab200700 1 Processor -- start 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.237+0000 7f9eab200700 1 -- start start 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9eab200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074e90 0x7f9eac1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9eab200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eac0757b0 0x7f9eac1a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9eab200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eac113a30 0x7f9eac19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9eab200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1193b0 con 0x7f9eac113a30 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9eab200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac119230 con 0x7f9eac0757b0 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9eab200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac119530 con 0x7f9eac074e90 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074e90 0x7f9eac1a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074e90 0x7f9eac1a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:46308/0 (socket says 172.21.15.18:46308) 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9e00700 1 -- 172.21.15.18:0/3573847576 learned_addr learned my addr 172.21.15.18:0/3573847576 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9eaa800700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eac113a30 0x7f9eac19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9400700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eac0757b0 0x7f9eac1a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9e00700 1 -- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eac0757b0 msgr2=0x7f9eac1a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9e00700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eac0757b0 0x7f9eac1a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9e00700 1 -- 172.21.15.18:0/3573847576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eac113a30 msgr2=0x7f9eac19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9e00700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eac113a30 0x7f9eac19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.238+0000 7f9ea9e00700 1 -- 172.21.15.18:0/3573847576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac19ff70 con 0x7f9eac074e90 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.239+0000 7f9ea9400700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eac0757b0 0x7f9eac1a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.239+0000 7f9ea9e00700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074e90 0x7f9eac1a55c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e9a0 tx=0x7f9e9c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:40.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.239+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00ca10 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.239+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac1a0260 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.239+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c00cbb0 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.239+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c018680 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.239+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1ac540 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.240+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9eac063240 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.242+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9e9c010430 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.244+0000 7f9ea2a00700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e900a3cd0 0x7f9e900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.244+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9e9c0d13c0 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.244+0000 7f9ea9400700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e900a3cd0 0x7f9e900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.244+0000 7f9ea2a00700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] conn(0x7f9e900aee30 0x7f9e900b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.244+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 --> [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e9c027e50 con 0x7f9e900aee30 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.244+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f9e9c0d1770 con 0x7f9eac074e90 2024-04-16T17:02:40.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.245+0000 7f9eaa800700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] conn(0x7f9e900aee30 0x7f9e900b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.245+0000 7f9ea9400700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e900a3cd0 0x7f9e900a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9eac1a0e50 tx=0x7f9e9800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.245+0000 7f9eaa800700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] conn(0x7f9e900aee30 0x7f9e900b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.246+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 <== osd.4 v2:172.21.15.22:6808/301052168 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e9c027e50 con 0x7f9e900aee30 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.265+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 --> [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9eac0636c0 con 0x7f9e900aee30 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.265+0000 7f9ea2a00700 1 -- 172.21.15.18:0/3573847576 <== osd.4 v2:172.21.15.22:6808/301052168 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9eac0636c0 con 0x7f9e900aee30 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] conn(0x7f9e900aee30 msgr2=0x7f9e900b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] conn(0x7f9e900aee30 0x7f9e900b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e900a3cd0 msgr2=0x7f9e900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9e900a3cd0 0x7f9e900a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9eac1a0e50 tx=0x7f9e9800b260 comp rx=0 tx=0).stop 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074e90 msgr2=0x7f9eac1a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 --2- 172.21.15.18:0/3573847576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074e90 0x7f9eac1a55c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e9a0 tx=0x7f9e9c00ecb0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eaa800700 1 -- 172.21.15.18:0/3573847576 reap_dead start 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 shutdown_connections 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 >> 172.21.15.18:0/3573847576 conn(0x7f9eac0fe570 msgr2=0x7f9eac0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 shutdown_connections 2024-04-16T17:02:40.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:40.266+0000 7f9eab200700 1 -- 172.21.15.18:0/3573847576 wait complete. 2024-04-16T17:02:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 -- 172.21.15.18:0/3162755987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43141139a0 msgr2=0x7f4314115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 --2- 172.21.15.18:0/3162755987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43141139a0 0x7f4314115d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f430c00b0a0 tx=0x7f430c00b3b0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 -- 172.21.15.18:0/3162755987 shutdown_connections 2024-04-16T17:02:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 --2- 172.21.15.18:0/3162755987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4314077020 0x7f4314077400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 --2- 172.21.15.18:0/3162755987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43141139a0 0x7f4314115d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 --2- 172.21.15.18:0/3162755987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4314077940 0x7f4314113460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 -- 172.21.15.18:0/3162755987 >> 172.21.15.18:0/3162755987 conn(0x7f43141007d0 msgr2=0x7f4314102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 -- 172.21.15.18:0/3162755987 shutdown_connections 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.109+0000 7f431b000700 1 -- 172.21.15.18:0/3162755987 wait complete. 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.110+0000 7f431b000700 1 Processor -- start 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.110+0000 7f431b000700 1 -- start start 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f431b000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4314077020 0x7f43141a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f431b000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4314077940 0x7f43141a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f431b000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43141139a0 0x7f431419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f431b000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4314119470 con 0x7f43141139a0 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f431b000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43141192f0 con 0x7f4314077020 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f431b000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43141195f0 con 0x7f4314077940 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f4319000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43141139a0 0x7f431419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f4319000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43141139a0 0x7f431419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45286/0 (socket says 172.21.15.18:45286) 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f4319000700 1 -- 172.21.15.18:0/531127442 learned_addr learned my addr 172.21.15.18:0/531127442 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f4313400700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4314077940 0x7f43141a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.111+0000 7f4313e00700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4314077020 0x7f43141a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4313400700 1 -- 172.21.15.18:0/531127442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4314077020 msgr2=0x7f43141a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4313400700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4314077020 0x7f43141a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4313400700 1 -- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43141139a0 msgr2=0x7f431419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4313400700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43141139a0 0x7f431419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4313400700 1 -- 172.21.15.18:0/531127442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431419ff90 con 0x7f4314077940 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4319000700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43141139a0 0x7f431419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4313e00700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4314077020 0x7f43141a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4313400700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4314077940 0x7f43141a5b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f430000e9f0 tx=0x7f430000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430000c9c0 con 0x7f4314077940 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430000cb60 con 0x7f4314077940 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43141a0280 con 0x7f4314077940 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300018680 con 0x7f4314077940 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.112+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141ac560 con 0x7f4314077940 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.113+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4314063240 con 0x7f4314077940 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.115+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4300010430 con 0x7f4314077940 2024-04-16T17:02:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.117+0000 7f4310c00700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f80a3cd0 0x7f42f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.117+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f43000d0e40 con 0x7f4314077940 2024-04-16T17:02:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.117+0000 7f4310c00700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] conn(0x7f42f80aee30 0x7f42f80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.117+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 --> [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4300019ec0 con 0x7f42f80aee30 2024-04-16T17:02:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.117+0000 7f4313e00700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f80a3cd0 0x7f42f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.117+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f43000d11f0 con 0x7f4314077940 2024-04-16T17:02:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.118+0000 7f4319000700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] conn(0x7f42f80aee30 0x7f42f80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.118+0000 7f4313e00700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f80a3cd0 0x7f42f80a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4304011490 tx=0x7f4304009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.118+0000 7f4319000700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] conn(0x7f42f80aee30 0x7f42f80b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.119+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 <== osd.3 v2:172.21.15.18:6810/2424709764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4300019ec0 con 0x7f42f80aee30 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.159+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 --> [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f43140636c0 con 0x7f42f80aee30 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.159+0000 7f4310c00700 1 -- 172.21.15.18:0/531127442 <== osd.3 v2:172.21.15.18:6810/2424709764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f43140636c0 con 0x7f42f80aee30 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.160+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] conn(0x7f42f80aee30 msgr2=0x7f42f80b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.160+0000 7f431b000700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] conn(0x7f42f80aee30 0x7f42f80b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.160+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f80a3cd0 msgr2=0x7f42f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.160+0000 7f431b000700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f42f80a3cd0 0x7f42f80a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4304011490 tx=0x7f4304009260 comp rx=0 tx=0).stop 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.160+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4314077940 msgr2=0x7f43141a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.160+0000 7f431b000700 1 --2- 172.21.15.18:0/531127442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4314077940 0x7f43141a5b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f430000e9f0 tx=0x7f430000ed00 comp rx=0 tx=0).stop 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.161+0000 7f4319000700 1 -- 172.21.15.18:0/531127442 reap_dead start 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.161+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 shutdown_connections 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.161+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 >> 172.21.15.18:0/531127442 conn(0x7f43141007d0 msgr2=0x7f4314102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.161+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 shutdown_connections 2024-04-16T17:02:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:39.161+0000 7f431b000700 1 -- 172.21.15.18:0/531127442 wait complete. 2024-04-16T17:02:40.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:40 smithi018 ceph-mon[31776]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s; 102 B/s, 0 objects/s recovering 2024-04-16T17:02:40.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:40 smithi022 ceph-mon[36023]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s; 102 B/s, 0 objects/s recovering 2024-04-16T17:02:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:40 smithi043 ceph-mon[36901]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s; 102 B/s, 0 objects/s recovering 2024-04-16T17:02:41.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 -- 172.21.15.18:0/1653736952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba78103500 msgr2=0x7fba7810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:41.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 --2- 172.21.15.18:0/1653736952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba78103500 0x7fba7810f960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba70009f90 tx=0x7fba7000b330 comp rx=0 tx=0).stop 2024-04-16T17:02:41.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 -- 172.21.15.18:0/1653736952 shutdown_connections 2024-04-16T17:02:41.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 --2- 172.21.15.18:0/1653736952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba78108b90 0x7fba78108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 --2- 172.21.15.18:0/1653736952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba78103500 0x7fba7810f960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 --2- 172.21.15.18:0/1653736952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba78102b40 0x7fba78102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 -- 172.21.15.18:0/1653736952 >> 172.21.15.18:0/1653736952 conn(0x7fba78079c50 msgr2=0x7fba7807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 -- 172.21.15.18:0/1653736952 shutdown_connections 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.658+0000 7fba7d000700 1 -- 172.21.15.18:0/1653736952 wait complete. 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.659+0000 7fba7d000700 1 Processor -- start 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.659+0000 7fba7d000700 1 -- start start 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba7d000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba78102b40 0x7fba7819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba7d000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba78103500 0x7fba7819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba7d000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba78108b90 0x7fba7819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba7d000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba78112df0 con 0x7fba78103500 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba7d000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba78112c70 con 0x7fba78108b90 2024-04-16T17:02:41.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba7d000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba78112f70 con 0x7fba78102b40 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba75400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba78103500 0x7fba7819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba75400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba78103500 0x7fba7819b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45334/0 (socket says 172.21.15.18:45334) 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba75400700 1 -- 172.21.15.18:0/1275799498 learned_addr learned my addr 172.21.15.18:0/1275799498 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba75e00700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba78102b40 0x7fba7819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.660+0000 7fba76800700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba78108b90 0x7fba7819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.661+0000 7fba75400700 1 -- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba78102b40 msgr2=0x7fba7819b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.661+0000 7fba75400700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba78102b40 0x7fba7819b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.661+0000 7fba75400700 1 -- 172.21.15.18:0/1275799498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba78108b90 msgr2=0x7fba7819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.661+0000 7fba75400700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba78108b90 0x7fba7819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.661+0000 7fba75400700 1 -- 172.21.15.18:0/1275799498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba781a04b0 con 0x7fba78103500 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.661+0000 7fba75e00700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba78102b40 0x7fba7819b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.661+0000 7fba76800700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba78108b90 0x7fba7819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:41.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.661+0000 7fba75400700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba78103500 0x7fba7819b9e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fba6400e9a0 tx=0x7fba6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.662+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6400c9c0 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.662+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6400cb60 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.662+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64018660 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.662+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba781a07a0 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.662+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba781a8020 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.663+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba7810f930 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.665+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fba6400cd10 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.666+0000 7fba6ea00700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fba5c0a3cd0 0x7fba5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.667+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fba640d0fa0 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.667+0000 7fba75e00700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fba5c0a3cd0 0x7fba5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.667+0000 7fba6ea00700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] conn(0x7fba5c0aee30 0x7fba5c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.667+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 --> [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba64019ec0 con 0x7fba5c0aee30 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.667+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fba640d1350 con 0x7fba78103500 2024-04-16T17:02:41.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.668+0000 7fba76800700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] conn(0x7fba5c0aee30 0x7fba5c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:41.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.668+0000 7fba75e00700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fba5c0a3cd0 0x7fba5c0a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba68005fd0 tx=0x7fba68009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:41.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.668+0000 7fba76800700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] conn(0x7fba5c0aee30 0x7fba5c0b12d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:41.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.669+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 <== osd.11 v2:172.21.15.43:6824/504916242 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba64019ec0 con 0x7fba5c0aee30 2024-04-16T17:02:41.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.688+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 --> [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba781041f0 con 0x7fba5c0aee30 2024-04-16T17:02:41.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.689+0000 7fba6ea00700 1 -- 172.21.15.18:0/1275799498 <== osd.11 v2:172.21.15.43:6824/504916242 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fba781041f0 con 0x7fba5c0aee30 2024-04-16T17:02:41.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.689+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] conn(0x7fba5c0aee30 msgr2=0x7fba5c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.689+0000 7fba7d000700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] conn(0x7fba5c0aee30 0x7fba5c0b12d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.689+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fba5c0a3cd0 msgr2=0x7fba5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.689+0000 7fba7d000700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fba5c0a3cd0 0x7fba5c0a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba68005fd0 tx=0x7fba68009310 comp rx=0 tx=0).stop 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.689+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba78103500 msgr2=0x7fba7819b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.689+0000 7fba7d000700 1 --2- 172.21.15.18:0/1275799498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba78103500 0x7fba7819b9e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fba6400e9a0 tx=0x7fba6400ecb0 comp rx=0 tx=0).stop 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.689+0000 7fba76800700 1 -- 172.21.15.18:0/1275799498 reap_dead start 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.690+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 shutdown_connections 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.690+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 >> 172.21.15.18:0/1275799498 conn(0x7fba78079c50 msgr2=0x7fba7810df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.690+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 shutdown_connections 2024-04-16T17:02:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.690+0000 7fba7d000700 1 -- 172.21.15.18:0/1275799498 wait complete. 2024-04-16T17:02:42.466 INFO:teuthology.orchestra.run.smithi018.stdout:287762808872 2024-04-16T17:02:42.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 -- 172.21.15.18:0/4104419846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a4108b70 msgr2=0x7f67a4108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 --2- 172.21.15.18:0/4104419846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a4108b70 0x7f67a4108f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6798009990 tx=0x7f6798009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 -- 172.21.15.18:0/4104419846 shutdown_connections 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 --2- 172.21.15.18:0/4104419846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a4108b70 0x7f67a4108f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 --2- 172.21.15.18:0/4104419846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a406b1a0 0x7f67a410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 --2- 172.21.15.18:0/4104419846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67a406a800 0x7f67a406ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 -- 172.21.15.18:0/4104419846 >> 172.21.15.18:0/4104419846 conn(0x7f67a40789a0 msgr2=0x7f67a4078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 -- 172.21.15.18:0/4104419846 shutdown_connections 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.553+0000 7f67ac800700 1 -- 172.21.15.18:0/4104419846 wait complete. 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.554+0000 7f67ac800700 1 Processor -- start 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67ac800700 1 -- start start 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67ac800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67a406a800 0x7f67a4101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67ac800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a406b1a0 0x7f67a4102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67ac800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a4108b70 0x7f67a4102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67ac800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4112dd0 con 0x7f67a406a800 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67ac800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4112c50 con 0x7f67a4108b70 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67ac800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4112f50 con 0x7f67a406b1a0 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67a9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67a406a800 0x7f67a4101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.555+0000 7f67a9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67a406a800 0x7f67a4101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45328/0 (socket says 172.21.15.18:45328) 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67aa800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a4108b70 0x7f67a4102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67a9e00700 1 -- 172.21.15.18:0/1822913224 learned_addr learned my addr 172.21.15.18:0/1822913224 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67a9400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a406b1a0 0x7f67a4102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67aa800700 1 -- 172.21.15.18:0/1822913224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a406b1a0 msgr2=0x7f67a4102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67aa800700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a406b1a0 0x7f67a4102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67aa800700 1 -- 172.21.15.18:0/1822913224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67a406a800 msgr2=0x7f67a4101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67aa800700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67a406a800 0x7f67a4101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67aa800700 1 -- 172.21.15.18:0/1822913224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a41a77b0 con 0x7f67a4108b70 2024-04-16T17:02:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67a9400700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a406b1a0 0x7f67a4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:42.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.556+0000 7f67aa800700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a4108b70 0x7f67a4102540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67a000efc0 tx=0x7f67a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:42.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.557+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a00092e0 con 0x7f67a4108b70 2024-04-16T17:02:42.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.557+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a41a7a40 con 0x7f67a4108b70 2024-04-16T17:02:42.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.557+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a0004bb0 con 0x7f67a4108b70 2024-04-16T17:02:42.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.557+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0004280 con 0x7f67a4108b70 2024-04-16T17:02:42.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.557+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a41a7f80 con 0x7f67a4108b70 2024-04-16T17:02:42.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.558+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6788001030 con 0x7f67a4108b70 2024-04-16T17:02:42.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.560+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f67a001b030 con 0x7f67a4108b70 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.561+0000 7f679ea00700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f678c0a3cd0 0x7f678c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.562+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f67a00d0bf0 con 0x7f67a4108b70 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.562+0000 7f67a9e00700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f678c0a3cd0 0x7f678c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.562+0000 7f679ea00700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] conn(0x7f678c0aee30 0x7f678c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.562+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 --> [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67a00048a0 con 0x7f678c0aee30 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.562+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f67a00d0fa0 con 0x7f67a4108b70 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.562+0000 7f67a9400700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] conn(0x7f678c0aee30 0x7f678c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.563+0000 7f67a9e00700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f678c0a3cd0 0x7f678c0a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f679800b2e0 tx=0x7f6798009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.563+0000 7f67a9400700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] conn(0x7f678c0aee30 0x7f678c0b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.564+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 <== osd.7 v2:172.21.15.22:6816/1748007509 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67a00048a0 con 0x7f678c0aee30 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.583+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 --> [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6788002e60 con 0x7f678c0aee30 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.583+0000 7f679ea00700 1 -- 172.21.15.18:0/1822913224 <== osd.7 v2:172.21.15.22:6816/1748007509 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6788002e60 con 0x7f678c0aee30 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 >> [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] conn(0x7f678c0aee30 msgr2=0x7f678c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67ac800700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] conn(0x7f678c0aee30 0x7f678c0b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f678c0a3cd0 msgr2=0x7f678c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67ac800700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f678c0a3cd0 0x7f678c0a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f679800b2e0 tx=0x7f6798009590 comp rx=0 tx=0).stop 2024-04-16T17:02:42.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a4108b70 msgr2=0x7f67a4102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67ac800700 1 --2- 172.21.15.18:0/1822913224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67a4108b70 0x7f67a4102540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67a000efc0 tx=0x7f67a000c3a0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67aa800700 1 -- 172.21.15.18:0/1822913224 reap_dead start 2024-04-16T17:02:42.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 shutdown_connections 2024-04-16T17:02:42.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.584+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 >> 172.21.15.18:0/1822913224 conn(0x7f67a40789a0 msgr2=0x7f67a40ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:42.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.585+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 shutdown_connections 2024-04-16T17:02:42.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:41.585+0000 7f67ac800700 1 -- 172.21.15.18:0/1822913224 wait complete. 2024-04-16T17:02:42.685 INFO:teuthology.orchestra.run.smithi018.stdout:107374182473 2024-04-16T17:02:42.686 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-16T17:02:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:42 smithi022 ceph-mon[36023]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s; 50 B/s, 0 objects/s recovering 2024-04-16T17:02:42.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:42 smithi018 ceph-mon[31776]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s; 50 B/s, 0 objects/s recovering 2024-04-16T17:02:43.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:42 smithi043 ceph-mon[36901]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s; 50 B/s, 0 objects/s recovering 2024-04-16T17:02:43.537 INFO:teuthology.orchestra.run.smithi018.stdout:253403070509 2024-04-16T17:02:43.537 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-16T17:02:44.902 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:44 smithi018 ceph-mon[31776]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-16T17:02:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:44 smithi022 ceph-mon[36023]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-16T17:02:44.985 INFO:teuthology.orchestra.run.smithi018.stdout:77309411407 2024-04-16T17:02:44.986 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-16T17:02:45.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:44 smithi043 ceph-mon[36901]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-16T17:02:45.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.463+0000 7fb3ee200700 1 -- 172.21.15.18:0/908706326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e8104f60 msgr2=0x7fb3e8105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.463+0000 7fb3ee200700 1 --2- 172.21.15.18:0/908706326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e8104f60 0x7fb3e8105340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009990 tx=0x7fb3d8009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.463+0000 7fb3ee200700 1 -- 172.21.15.18:0/908706326 shutdown_connections 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.463+0000 7fb3ee200700 1 --2- 172.21.15.18:0/908706326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e8104f60 0x7fb3e8105340 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.463+0000 7fb3ee200700 1 --2- 172.21.15.18:0/908706326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3e810a040 0x7fb3e8111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.463+0000 7fb3ee200700 1 --2- 172.21.15.18:0/908706326 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3e8105910 0x7fb3e8109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.463+0000 7fb3ee200700 1 -- 172.21.15.18:0/908706326 >> 172.21.15.18:0/908706326 conn(0x7fb3e81007f0 msgr2=0x7fb3e8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.463+0000 7fb3ee200700 1 -- 172.21.15.18:0/908706326 shutdown_connections 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.465+0000 7fb3ee200700 1 -- 172.21.15.18:0/908706326 wait complete. 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.466+0000 7fb3ee200700 1 Processor -- start 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.466+0000 7fb3ee200700 1 -- start start 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.466+0000 7fb3ee200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3e8104f60 0x7fb3e819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.466+0000 7fb3ee200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e8105910 0x7fb3e819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.466+0000 7fb3ee200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3e810a040 0x7fb3e819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:45.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.466+0000 7fb3ee200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e8115030 con 0x7fb3e810a040 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.466+0000 7fb3ee200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e8114eb0 con 0x7fb3e8105910 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.466+0000 7fb3ee200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e81151b0 con 0x7fb3e8104f60 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e8105910 0x7fb3e819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3e8104f60 0x7fb3e819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e8105910 0x7fb3e819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51778/0 (socket says 172.21.15.18:51778) 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e6a00700 1 -- 172.21.15.18:0/1940928085 learned_addr learned my addr 172.21.15.18:0/1940928085 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e7e00700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3e810a040 0x7fb3e819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e7400700 1 -- 172.21.15.18:0/1940928085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e8105910 msgr2=0x7fb3e819ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e7400700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e8105910 0x7fb3e819ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e7400700 1 -- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3e810a040 msgr2=0x7fb3e819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e7400700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3e810a040 0x7fb3e819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e7400700 1 -- 172.21.15.18:0/1940928085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e81a04d0 con 0x7fb3e8104f60 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.467+0000 7fb3e7e00700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3e810a040 0x7fb3e819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.468+0000 7fb3e7400700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3e8104f60 0x7fb3e819b4c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80059d0 tx=0x7fb3d800b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.468+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8027070 con 0x7fb3e8104f60 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.468+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d8022470 con 0x7fb3e8104f60 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.468+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8017610 con 0x7fb3e8104f60 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.468+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e81a0760 con 0x7fb3e8104f60 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.468+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e81a0bc0 con 0x7fb3e8104f60 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.469+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3e8063280 con 0x7fb3e8104f60 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.473+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb3d802a070 con 0x7fb3e8104f60 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.474+0000 7fb3dfe00700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3cc0a3cd0 0x7fb3cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.475+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb3d80d1e20 con 0x7fb3e8104f60 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.475+0000 7fb3e6a00700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3cc0a3cd0 0x7fb3cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:45.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.475+0000 7fb3dfe00700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] conn(0x7fb3cc0aee30 0x7fb3cc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.475+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 --> [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3d800ce90 con 0x7fb3cc0aee30 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.475+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fb3d80d2270 con 0x7fb3e8104f60 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.476+0000 7fb3e7e00700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] conn(0x7fb3cc0aee30 0x7fb3cc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.476+0000 7fb3e6a00700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3cc0a3cd0 0x7fb3cc0a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8101c80 tx=0x7fb3d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.476+0000 7fb3e7e00700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] conn(0x7fb3cc0aee30 0x7fb3cc0b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.477+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 <== osd.6 v2:172.21.15.18:6818/3519009309 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3d800ce90 con 0x7fb3cc0aee30 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.498+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 --> [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb3e8063700 con 0x7fb3cc0aee30 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.498+0000 7fb3dfe00700 1 -- 172.21.15.18:0/1940928085 <== osd.6 v2:172.21.15.18:6818/3519009309 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb3e8063700 con 0x7fb3cc0aee30 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] conn(0x7fb3cc0aee30 msgr2=0x7fb3cc0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] conn(0x7fb3cc0aee30 0x7fb3cc0b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3cc0a3cd0 msgr2=0x7fb3cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb3cc0a3cd0 0x7fb3cc0a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8101c80 tx=0x7fb3d4009260 comp rx=0 tx=0).stop 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3e8104f60 msgr2=0x7fb3e819b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:45.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 --2- 172.21.15.18:0/1940928085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3e8104f60 0x7fb3e819b4c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80059d0 tx=0x7fb3d800b6c0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3e7e00700 1 -- 172.21.15.18:0/1940928085 reap_dead start 2024-04-16T17:02:45.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 shutdown_connections 2024-04-16T17:02:45.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 >> 172.21.15.18:0/1940928085 conn(0x7fb3e81007f0 msgr2=0x7fb3e8102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:45.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 shutdown_connections 2024-04-16T17:02:45.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.499+0000 7fb3ee200700 1 -- 172.21.15.18:0/1940928085 wait complete. 2024-04-16T17:02:45.953 INFO:teuthology.orchestra.run.smithi018.stdout:219043332148 2024-04-16T17:02:45.954 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-16T17:02:46.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.542+0000 7fc3e1400700 1 -- 172.21.15.18:0/3938794197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3dc075640 msgr2=0x7fc3dc10efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:46.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.542+0000 7fc3e1400700 1 --2- 172.21.15.18:0/3938794197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3dc075640 0x7fc3dc10efb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009990 tx=0x7fc3c8009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:46.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.543+0000 7fc3e1400700 1 -- 172.21.15.18:0/3938794197 shutdown_connections 2024-04-16T17:02:46.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.543+0000 7fc3e1400700 1 --2- 172.21.15.18:0/3938794197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3dc074d20 0x7fc3dc075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:46.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.543+0000 7fc3e1400700 1 --2- 172.21.15.18:0/3938794197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3dc10f4f0 0x7fc3dc1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:46.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.543+0000 7fc3e1400700 1 --2- 172.21.15.18:0/3938794197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3dc075640 0x7fc3dc10efb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:46.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.543+0000 7fc3e1400700 1 -- 172.21.15.18:0/3938794197 >> 172.21.15.18:0/3938794197 conn(0x7fc3dc079c50 msgr2=0x7fc3dc07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:46.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.544+0000 7fc3e1400700 1 -- 172.21.15.18:0/3938794197 shutdown_connections 2024-04-16T17:02:46.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.544+0000 7fc3e1400700 1 -- 172.21.15.18:0/3938794197 wait complete. 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.545+0000 7fc3e1400700 1 Processor -- start 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.545+0000 7fc3e1400700 1 -- start start 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3e1400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3dc074d20 0x7fc3dc1a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3e1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3dc075640 0x7fc3dc1a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3e1400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3dc10f4f0 0x7fc3dc10e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3e1400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1046c0 con 0x7fc3dc075640 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3e1400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc104540 con 0x7fc3dc074d20 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3da800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3dc074d20 0x7fc3dc1a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3da800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3dc074d20 0x7fc3dc1a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51802/0 (socket says 172.21.15.18:51802) 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3da800700 1 -- 172.21.15.18:0/37143336 learned_addr learned my addr 172.21.15.18:0/37143336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3d9e00700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3dc075640 0x7fc3dc1a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3db200700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3dc10f4f0 0x7fc3dc10e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.546+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc104840 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3db200700 1 -- 172.21.15.18:0/37143336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3dc074d20 msgr2=0x7fc3dc1a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3db200700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3dc074d20 0x7fc3dc1a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3db200700 1 -- 172.21.15.18:0/37143336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3dc075640 msgr2=0x7fc3dc1a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3db200700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3dc075640 0x7fc3dc1a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3db200700 1 -- 172.21.15.18:0/37143336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc10eac0 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3db200700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3dc10f4f0 0x7fc3dc10e1c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400ef90 tx=0x7fc3d400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40092e0 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc10ed50 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d4004bb0 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.547+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4004280 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.548+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc1ac570 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.548+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3dc100720 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.551+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc3d401b030 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.552+0000 7fc3d3400700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc3c00a3d50 0x7fc3c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.552+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc3d40d0e10 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.552+0000 7fc3da800700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc3c00a3d50 0x7fc3c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.552+0000 7fc3d3400700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] conn(0x7fc3c00aeeb0 0x7fc3c00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.553+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 --> [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3c00b1a20 con 0x7fc3c00aeeb0 2024-04-16T17:02:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.553+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fc3d40d11c0 con 0x7fc3dc10f4f0 2024-04-16T17:02:46.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.553+0000 7fc3d9e00700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] conn(0x7fc3c00aeeb0 0x7fc3c00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:46.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.553+0000 7fc3da800700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc3c00a3d50 0x7fc3c00a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009d20 tx=0x7fc3cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:46.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.553+0000 7fc3d9e00700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] conn(0x7fc3c00aeeb0 0x7fc3c00b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:46.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.554+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 <== osd.1 v2:172.21.15.22:6800/1527176746 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3c00b1a20 con 0x7fc3c00aeeb0 2024-04-16T17:02:46.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.572+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 --> [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc3dc076410 con 0x7fc3c00aeeb0 2024-04-16T17:02:46.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.573+0000 7fc3d3400700 1 -- 172.21.15.18:0/37143336 <== osd.1 v2:172.21.15.22:6800/1527176746 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc3dc076410 con 0x7fc3c00aeeb0 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.573+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 >> [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] conn(0x7fc3c00aeeb0 msgr2=0x7fc3c00b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] conn(0x7fc3c00aeeb0 0x7fc3c00b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc3c00a3d50 msgr2=0x7fc3c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc3c00a3d50 0x7fc3c00a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009d20 tx=0x7fc3cc009260 comp rx=0 tx=0).stop 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3dc10f4f0 msgr2=0x7fc3dc10e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 --2- 172.21.15.18:0/37143336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3dc10f4f0 0x7fc3dc10e1c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400ef90 tx=0x7fc3d400c3a0 comp rx=0 tx=0).stop 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3db200700 1 -- 172.21.15.18:0/37143336 reap_dead start 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 shutdown_connections 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 >> 172.21.15.18:0/37143336 conn(0x7fc3dc079c50 msgr2=0x7fc3dc0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 shutdown_connections 2024-04-16T17:02:46.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:46.574+0000 7fc3e1400700 1 -- 172.21.15.18:0/37143336 wait complete. 2024-04-16T17:02:46.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:46 smithi022 ceph-mon[36023]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:46.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:46 smithi018 ceph-mon[31776]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:47.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:46 smithi043 ceph-mon[36901]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:47.399 INFO:teuthology.orchestra.run.smithi018.stdout:197568495673 2024-04-16T17:02:47.399 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-16T17:02:47.585 INFO:teuthology.orchestra.run.smithi018.stdout:343597383709 2024-04-16T17:02:47.585 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-16T17:02:48.230 INFO:teuthology.orchestra.run.smithi018.stdout:55834574935 2024-04-16T17:02:48.230 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 -- 172.21.15.18:0/4062629003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c108b90 msgr2=0x7fd57c108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 --2- 172.21.15.18:0/4062629003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c108b90 0x7fd57c108f70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098f0 tx=0x7fd570009c00 comp rx=0 tx=0).stop 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 -- 172.21.15.18:0/4062629003 shutdown_connections 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 --2- 172.21.15.18:0/4062629003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd57c103500 0x7fd57c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 --2- 172.21.15.18:0/4062629003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c108b90 0x7fd57c108f70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 --2- 172.21.15.18:0/4062629003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd57c102b40 0x7fd57c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 -- 172.21.15.18:0/4062629003 >> 172.21.15.18:0/4062629003 conn(0x7fd57c079c50 msgr2=0x7fd57c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 -- 172.21.15.18:0/4062629003 shutdown_connections 2024-04-16T17:02:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.598+0000 7fd584000700 1 -- 172.21.15.18:0/4062629003 wait complete. 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.599+0000 7fd584000700 1 Processor -- start 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd584000700 1 -- start start 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd584000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd57c102b40 0x7fd57c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd584000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c103500 0x7fd57c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd584000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd57c108b90 0x7fd57c19dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd584000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c112e50 con 0x7fd57c108b90 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd584000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c112cd0 con 0x7fd57c102b40 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd584000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c112fd0 con 0x7fd57c103500 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd582000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd57c108b90 0x7fd57c19dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd580c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c103500 0x7fd57c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd581600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd57c102b40 0x7fd57c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd580c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c103500 0x7fd57c1998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:46358/0 (socket says 172.21.15.18:46358) 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd580c00700 1 -- 172.21.15.18:0/3754403409 learned_addr learned my addr 172.21.15.18:0/3754403409 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:48.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd581600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd57c102b40 0x7fd57c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51786/0 (socket says 172.21.15.18:51786) 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.600+0000 7fd582000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd57c108b90 0x7fd57c19dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45390/0 (socket says 172.21.15.18:45390) 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd580c00700 1 -- 172.21.15.18:0/3754403409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd57c102b40 msgr2=0x7fd57c199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd580c00700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd57c102b40 0x7fd57c199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd580c00700 1 -- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd57c108b90 msgr2=0x7fd57c19dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd580c00700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd57c108b90 0x7fd57c19dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd580c00700 1 -- 172.21.15.18:0/3754403409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c19e370 con 0x7fd57c103500 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd582000700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd57c108b90 0x7fd57c19dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd580c00700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c103500 0x7fd57c1998a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd56800e9c0 tx=0x7fd56800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568004a30 con 0x7fd57c103500 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56800cca0 con 0x7fd57c103500 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.601+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568005610 con 0x7fd57c103500 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.602+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c19e660 con 0x7fd57c103500 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.602+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c1a5ee0 con 0x7fd57c103500 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.603+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd57c063240 con 0x7fd57c103500 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.605+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd568004cf0 con 0x7fd57c103500 2024-04-16T17:02:48.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.607+0000 7fd57a000700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5640a3d50 0x7fd5640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.607+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd5680d0dc0 con 0x7fd57c103500 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.607+0000 7fd581600700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5640a3d50 0x7fd5640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.607+0000 7fd57a000700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] conn(0x7fd5640aeeb0 0x7fd5640b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.607+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 --> [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd56800ce50 con 0x7fd5640aeeb0 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.607+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd5680d1170 con 0x7fd57c103500 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.608+0000 7fd582000700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] conn(0x7fd5640aeeb0 0x7fd5640b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.608+0000 7fd581600700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5640a3d50 0x7fd5640a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd570009810 tx=0x7fd570019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.608+0000 7fd582000700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] conn(0x7fd5640aeeb0 0x7fd5640b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.609+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 <== osd.0 v2:172.21.15.18:6802/2892109649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd56800ce50 con 0x7fd5640aeeb0 2024-04-16T17:02:48.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.628+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 --> [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd57c063710 con 0x7fd5640aeeb0 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.628+0000 7fd57a000700 1 -- 172.21.15.18:0/3754403409 <== osd.0 v2:172.21.15.18:6802/2892109649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd57c063710 con 0x7fd5640aeeb0 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] conn(0x7fd5640aeeb0 msgr2=0x7fd5640b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] conn(0x7fd5640aeeb0 0x7fd5640b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5640a3d50 msgr2=0x7fd5640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5640a3d50 0x7fd5640a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd570009810 tx=0x7fd570019040 comp rx=0 tx=0).stop 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c103500 msgr2=0x7fd57c1998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 --2- 172.21.15.18:0/3754403409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd57c103500 0x7fd57c1998a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd56800e9c0 tx=0x7fd56800ecd0 comp rx=0 tx=0).stop 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd582000700 1 -- 172.21.15.18:0/3754403409 reap_dead start 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 shutdown_connections 2024-04-16T17:02:48.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 >> 172.21.15.18:0/3754403409 conn(0x7fd57c079c50 msgr2=0x7fd57c101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:48.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 shutdown_connections 2024-04-16T17:02:48.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:45.629+0000 7fd584000700 1 -- 172.21.15.18:0/3754403409 wait complete. 2024-04-16T17:02:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:48 smithi022 ceph-mon[36023]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:02:48.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:48 smithi018 ceph-mon[31776]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:02:48.957 INFO:teuthology.orchestra.run.smithi018.stdout:141733920837 2024-04-16T17:02:48.957 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-16T17:02:49.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:48 smithi043 ceph-mon[36901]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:02:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:49.569 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:49.579 INFO:teuthology.orchestra.run.smithi018.stdout:34359738460 2024-04-16T17:02:49.579 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-16T17:02:49.608 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:49.616 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:49.618 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:49.753 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:49.759 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:49.772 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:50.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:50 smithi018 ceph-mon[31776]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-16T17:02:50.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:50 smithi022 ceph-mon[36023]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-16T17:02:51.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:50 smithi043 ceph-mon[36901]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-16T17:02:52.886 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:52 smithi018 ceph-mon[31776]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:52.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:52 smithi022 ceph-mon[36023]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:53.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:52 smithi043 ceph-mon[36901]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:53.977 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:53 smithi018 ceph-mon[31776]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:02:54.000 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:54.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:53 smithi043 ceph-mon[36901]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:02:54.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:53 smithi022 ceph-mon[36023]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:02:56.240 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:55 smithi018 ceph-mon[31776]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:56.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:55 smithi043 ceph-mon[36901]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:56.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:55 smithi022 ceph-mon[36023]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:02:58.184 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:57 smithi018 ceph-mon[31776]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:02:58.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:57 smithi043 ceph-mon[36901]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:02:58.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:57 smithi022 ceph-mon[36023]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:02:59.304 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:59.305 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.902+0000 7f76c6800700 1 -- 172.21.15.18:0/3704958924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 msgr2=0x7f76c0075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.902+0000 7f76c6800700 1 --2- 172.21.15.18:0/3704958924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 0x7f76c0075100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f76b00098f0 tx=0x7f76b0009c00 comp rx=0 tx=0).stop 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.902+0000 7f76c6800700 1 -- 172.21.15.18:0/3704958924 shutdown_connections 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.902+0000 7f76c6800700 1 --2- 172.21.15.18:0/3704958924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76c010f550 0x7f76c0111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.902+0000 7f76c6800700 1 --2- 172.21.15.18:0/3704958924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 0x7f76c0075100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.902+0000 7f76c6800700 1 --2- 172.21.15.18:0/3704958924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c0075640 0x7f76c010f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.902+0000 7f76c6800700 1 -- 172.21.15.18:0/3704958924 >> 172.21.15.18:0/3704958924 conn(0x7f76c0079c50 msgr2=0x7f76c007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.903+0000 7f76c6800700 1 -- 172.21.15.18:0/3704958924 shutdown_connections 2024-04-16T17:02:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.903+0000 7f76c6800700 1 -- 172.21.15.18:0/3704958924 wait complete. 2024-04-16T17:02:59.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.904+0000 7f76c6800700 1 Processor -- start 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.904+0000 7f76c6800700 1 -- start start 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.904+0000 7f76c6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 0x7f76c019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.904+0000 7f76c6800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c0075640 0x7f76c019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76c6800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76c010f550 0x7f76c019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76c6800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0115090 con 0x7f76c0074d20 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76c6800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0114f10 con 0x7f76c010f550 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76c6800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0115210 con 0x7f76c0075640 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 0x7f76c019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 0x7f76c019b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45426/0 (socket says 172.21.15.18:45426) 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bf400700 1 -- 172.21.15.18:0/3099980804 learned_addr learned my addr 172.21.15.18:0/3099980804 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bea00700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c0075640 0x7f76c019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bfe00700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76c010f550 0x7f76c019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bf400700 1 -- 172.21.15.18:0/3099980804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c0075640 msgr2=0x7f76c019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bf400700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c0075640 0x7f76c019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bf400700 1 -- 172.21.15.18:0/3099980804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76c010f550 msgr2=0x7f76c019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bf400700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76c010f550 0x7f76c019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bf400700 1 -- 172.21.15.18:0/3099980804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c01a04c0 con 0x7f76c0074d20 2024-04-16T17:02:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bfe00700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76c010f550 0x7f76c019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:02:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.905+0000 7f76bea00700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c0075640 0x7f76c019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.906+0000 7f76bf400700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 0x7f76c019b500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76b0005950 tx=0x7f76b0005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.906+0000 7f76b7e00700 1 -- 172.21.15.18:0/3099980804 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b00052e0 con 0x7f76c0074d20 2024-04-16T17:02:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.906+0000 7f76b7e00700 1 -- 172.21.15.18:0/3099980804 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b000ba40 con 0x7f76c0074d20 2024-04-16T17:02:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.906+0000 7f76b7e00700 1 -- 172.21.15.18:0/3099980804 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0003b80 con 0x7f76c0074d20 2024-04-16T17:02:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.906+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c01a0750 con 0x7f76c0074d20 2024-04-16T17:02:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.906+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c01a8030 con 0x7f76c0074d20 2024-04-16T17:02:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.908+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c00763e0 con 0x7f76c0074d20 2024-04-16T17:02:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.910+0000 7f76b7e00700 1 -- 172.21.15.18:0/3099980804 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f76b0005480 con 0x7f76c0074d20 2024-04-16T17:02:59.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.911+0000 7f76b7e00700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f76a40a3cd0 0x7f76a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:59.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.911+0000 7f76b7e00700 1 -- 172.21.15.18:0/3099980804 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f76b00d1a30 con 0x7f76c0074d20 2024-04-16T17:02:59.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.912+0000 7f76bea00700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f76a40a3cd0 0x7f76a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.912+0000 7f76bea00700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f76a40a3cd0 0x7f76a40a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f76c0111030 tx=0x7f76ac009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:59.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:02:59.919+0000 7f76b7e00700 1 -- 172.21.15.18:0/3099980804 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76b0013030 con 0x7f76c0074d20 2024-04-16T17:03:00.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.161+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f76c01a8200 con 0x7f76c0074d20 2024-04-16T17:03:00.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.161+0000 7f76b7e00700 1 -- 172.21.15.18:0/3099980804 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f76b009df80 con 0x7f76c0074d20 2024-04-16T17:03:00.163 INFO:teuthology.orchestra.run.smithi018.stdout:287762808877 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f76a40a3cd0 msgr2=0x7f76a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f76a40a3cd0 0x7f76a40a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f76c0111030 tx=0x7f76ac009310 comp rx=0 tx=0).stop 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 msgr2=0x7f76c019b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 0x7f76c019b500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76b0005950 tx=0x7f76b0005040 comp rx=0 tx=0).stop 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 shutdown_connections 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f76a40a3cd0 0x7f76a40a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76c010f550 0x7f76c019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76c0074d20 0x7f76c019b500 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 --2- 172.21.15.18:0/3099980804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c0075640 0x7f76c019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 >> 172.21.15.18:0/3099980804 conn(0x7f76c0079c50 msgr2=0x7f76c00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:00.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 shutdown_connections 2024-04-16T17:03:00.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.164+0000 7f76c6800700 1 -- 172.21.15.18:0/3099980804 wait complete. 2024-04-16T17:03:00.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:02:59 smithi043 ceph-mon[36901]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:00.331 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:02:59 smithi018 ceph-mon[31776]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:00.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:02:59 smithi022 ceph-mon[36023]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.719+0000 7faf09000700 1 -- 172.21.15.18:0/3619201357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0410a020 msgr2=0x7faf04111bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.719+0000 7faf09000700 1 --2- 172.21.15.18:0/3619201357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0410a020 0x7faf04111bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faefc009fc0 tx=0x7faefc00b330 comp rx=0 tx=0).stop 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.719+0000 7faf09000700 1 -- 172.21.15.18:0/3619201357 shutdown_connections 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.719+0000 7faf09000700 1 --2- 172.21.15.18:0/3619201357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0410a020 0x7faf04111bf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.719+0000 7faf09000700 1 --2- 172.21.15.18:0/3619201357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf04104f40 0x7faf04105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.719+0000 7faf09000700 1 --2- 172.21.15.18:0/3619201357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf041058f0 0x7faf04109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.719+0000 7faf09000700 1 -- 172.21.15.18:0/3619201357 >> 172.21.15.18:0/3619201357 conn(0x7faf041007d0 msgr2=0x7faf04102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.720+0000 7faf09000700 1 -- 172.21.15.18:0/3619201357 shutdown_connections 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.720+0000 7faf09000700 1 -- 172.21.15.18:0/3619201357 wait complete. 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.721+0000 7faf09000700 1 Processor -- start 2024-04-16T17:03:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.721+0000 7faf09000700 1 -- start start 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.721+0000 7faf09000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf04104f40 0x7faf0419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.721+0000 7faf09000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf041058f0 0x7faf0419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.721+0000 7faf09000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0410a020 0x7faf0419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.721+0000 7faf09000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faf04115060 con 0x7faf041058f0 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.721+0000 7faf09000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf04114ee0 con 0x7faf04104f40 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.721+0000 7faf09000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf041151e0 con 0x7faf0410a020 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf041058f0 0x7faf0419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf041058f0 0x7faf0419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45446/0 (socket says 172.21.15.18:45446) 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01400700 1 -- 172.21.15.18:0/810508890 learned_addr learned my addr 172.21.15.18:0/810508890 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf02800700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0410a020 0x7faf0419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01e00700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf04104f40 0x7faf0419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01e00700 1 -- 172.21.15.18:0/810508890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0410a020 msgr2=0x7faf0419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01e00700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0410a020 0x7faf0419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01e00700 1 -- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf041058f0 msgr2=0x7faf0419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01e00700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf041058f0 0x7faf0419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01e00700 1 -- 172.21.15.18:0/810508890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf041a04a0 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01400700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf041058f0 0x7faf0419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.722+0000 7faf01e00700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf04104f40 0x7faf0419b490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faef400e9c0 tx=0x7faef400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.723+0000 7faefaa00700 1 -- 172.21.15.18:0/810508890 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef400c9c0 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.723+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf041a0790 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.723+0000 7faefaa00700 1 -- 172.21.15.18:0/810508890 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef400cb60 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.723+0000 7faefaa00700 1 -- 172.21.15.18:0/810508890 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4018680 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.723+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf041a8010 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.724+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf040697e0 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.726+0000 7faefaa00700 1 -- 172.21.15.18:0/810508890 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faef400cd10 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.728+0000 7faefaa00700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faee80a3cd0 0x7faee80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.728+0000 7faefaa00700 1 -- 172.21.15.18:0/810508890 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faef40d0dc0 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.729+0000 7faf01400700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faee80a3cd0 0x7faee80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.730+0000 7faefaa00700 1 -- 172.21.15.18:0/810508890 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faef409d280 con 0x7faf04104f40 2024-04-16T17:03:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.730+0000 7faf01400700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faee80a3cd0 0x7faee80a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faef0009680 tx=0x7faef0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:00.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.970+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7faf041a81e0 con 0x7faf04104f40 2024-04-16T17:03:00.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.971+0000 7faefaa00700 1 -- 172.21.15.18:0/810508890 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7faef409cb80 con 0x7faf04104f40 2024-04-16T17:03:00.973 INFO:teuthology.orchestra.run.smithi018.stdout:309237645351 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.973+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faee80a3cd0 msgr2=0x7faee80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.973+0000 7faf09000700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faee80a3cd0 0x7faee80a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faef0009680 tx=0x7faef0009260 comp rx=0 tx=0).stop 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf04104f40 msgr2=0x7faf0419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf04104f40 0x7faf0419b490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faef400e9c0 tx=0x7faef400ecd0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 shutdown_connections 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7faee80a3cd0 0x7faee80a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf04104f40 0x7faf0419b490 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0410a020 0x7faf0419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 --2- 172.21.15.18:0/810508890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf041058f0 0x7faf0419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 >> 172.21.15.18:0/810508890 conn(0x7faf041007d0 msgr2=0x7faf04101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 shutdown_connections 2024-04-16T17:03:00.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:00.974+0000 7faf09000700 1 -- 172.21.15.18:0/810508890 wait complete. 2024-04-16T17:03:01.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:00 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3099980804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T17:03:01.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:00 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/810508890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T17:03:01.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:00 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3099980804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T17:03:01.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:00 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/810508890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T17:03:01.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:00 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3099980804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T17:03:01.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:00 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/810508890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T17:03:02.173 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:01 smithi018 ceph-mon[31776]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:02.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:01 smithi043 ceph-mon[36901]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:02.326 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:01 smithi022 ceph-mon[36023]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:04.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 -- 172.21.15.18:0/3830898496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2474108b90 msgr2=0x7f2474108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 --2- 172.21.15.18:0/3830898496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2474108b90 0x7f2474108f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24640098f0 tx=0x7f2464009c00 comp rx=0 tx=0).stop 2024-04-16T17:03:04.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 -- 172.21.15.18:0/3830898496 shutdown_connections 2024-04-16T17:03:04.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 --2- 172.21.15.18:0/3830898496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2474103500 0x7f247410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 --2- 172.21.15.18:0/3830898496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2474108b90 0x7f2474108f70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 --2- 172.21.15.18:0/3830898496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2474102b40 0x7f2474102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 -- 172.21.15.18:0/3830898496 >> 172.21.15.18:0/3830898496 conn(0x7f2474079c50 msgr2=0x7f247407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:04.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 -- 172.21.15.18:0/3830898496 shutdown_connections 2024-04-16T17:03:04.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.115+0000 7f2478a00700 1 -- 172.21.15.18:0/3830898496 wait complete. 2024-04-16T17:03:04.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.117+0000 7f2478a00700 1 Processor -- start 2024-04-16T17:03:04.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.117+0000 7f2478a00700 1 -- start start 2024-04-16T17:03:04.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2478a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2474102b40 0x7f2474199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2478a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2474103500 0x7f2474199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2478a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2474108b90 0x7f247419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2478a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2474112e60 con 0x7f2474102b40 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2478a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2474112ce0 con 0x7f2474108b90 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2478a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2474112fe0 con 0x7f2474103500 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2471e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2474102b40 0x7f2474199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2472800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2474108b90 0x7f247419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2472800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2474108b90 0x7f247419dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49722/0 (socket says 172.21.15.18:49722) 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2472800700 1 -- 172.21.15.18:0/117874663 learned_addr learned my addr 172.21.15.18:0/117874663 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2469400700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2474103500 0x7f2474199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.118+0000 7f2471e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2474102b40 0x7f2474199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45480/0 (socket says 172.21.15.18:45480) 2024-04-16T17:03:04.121 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:03 smithi018 ceph-mon[31776]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.119+0000 7f2472800700 1 -- 172.21.15.18:0/117874663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2474103500 msgr2=0x7f2474199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.119+0000 7f2472800700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2474103500 0x7f2474199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.119+0000 7f2472800700 1 -- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2474102b40 msgr2=0x7f2474199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.119+0000 7f2472800700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2474102b40 0x7f2474199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.119+0000 7f2472800700 1 -- 172.21.15.18:0/117874663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247419e360 con 0x7f2474108b90 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.119+0000 7f2472800700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2474108b90 0x7f247419dc20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f246c00c110 tx=0x7f246c00c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.119+0000 7f246b400700 1 -- 172.21.15.18:0/117874663 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0092e0 con 0x7f2474108b90 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.120+0000 7f246b400700 1 -- 172.21.15.18:0/117874663 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f246c004bb0 con 0x7f2474108b90 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.120+0000 7f246b400700 1 -- 172.21.15.18:0/117874663 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c007d10 con 0x7f2474108b90 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.120+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247419e5f0 con 0x7f2474108b90 2024-04-16T17:03:04.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.120+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741a5ed0 con 0x7f2474108b90 2024-04-16T17:03:04.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.121+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24740697e0 con 0x7f2474108b90 2024-04-16T17:03:04.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.123+0000 7f246b400700 1 -- 172.21.15.18:0/117874663 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f246c01a020 con 0x7f2474108b90 2024-04-16T17:03:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.124+0000 7f246b400700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f24580a3d50 0x7f24580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.125+0000 7f246b400700 1 -- 172.21.15.18:0/117874663 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f246c0d0d60 con 0x7f2474108b90 2024-04-16T17:03:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.125+0000 7f2471e00700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f24580a3d50 0x7f24580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.126+0000 7f2471e00700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f24580a3d50 0x7f24580a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2464009810 tx=0x7f2464019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:04.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.127+0000 7f246b400700 1 -- 172.21.15.18:0/117874663 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f246c010040 con 0x7f2474108b90 2024-04-16T17:03:04.247 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808872 got 287762808877 for osd.9 2024-04-16T17:03:04.248 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:04.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:03 smithi043 ceph-mon[36901]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:04.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 -- 172.21.15.18:0/1762256074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e1810e460 msgr2=0x7f9e18116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/1762256074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e1810e460 0x7f9e18116030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000b0a0 tx=0x7f9e1000b3b0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 -- 172.21.15.18:0/1762256074 shutdown_connections 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/1762256074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e1810e460 0x7f9e18116030 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/1762256074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e18109380 0x7f9e18109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/1762256074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18109d30 0x7f9e1810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 -- 172.21.15.18:0/1762256074 >> 172.21.15.18:0/1762256074 conn(0x7f9e180762c0 msgr2=0x7f9e18107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 -- 172.21.15.18:0/1762256074 shutdown_connections 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.047+0000 7f9e1ea00700 1 -- 172.21.15.18:0/1762256074 wait complete. 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.048+0000 7f9e1ea00700 1 Processor -- start 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e1ea00700 1 -- start start 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e1ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18109380 0x7f9e181a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e1ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e18109d30 0x7f9e181a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e1ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e1810e460 0x7f9e1819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e1ea00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e181194c0 con 0x7f9e1810e460 2024-04-16T17:03:04.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e1ea00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18119340 con 0x7f9e18109d30 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e1ea00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18119640 con 0x7f9e18109380 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e0fe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e18109d30 0x7f9e181a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e0fe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e18109d30 0x7f9e181a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49712/0 (socket says 172.21.15.18:49712) 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.049+0000 7f9e0fe00700 1 -- 172.21.15.18:0/2161334503 learned_addr learned my addr 172.21.15.18:0/2161334503 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e17e00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18109380 0x7f9e181a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e1ca00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e1810e460 0x7f9e1819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e0fe00700 1 -- 172.21.15.18:0/2161334503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18109380 msgr2=0x7f9e181a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e0fe00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18109380 0x7f9e181a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e0fe00700 1 -- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e1810e460 msgr2=0x7f9e1819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e0fe00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e1810e460 0x7f9e1819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e0fe00700 1 -- 172.21.15.18:0/2161334503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1819ffa0 con 0x7f9e18109d30 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e1ca00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e1810e460 0x7f9e1819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:04.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e17e00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18109380 0x7f9e181a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e0fe00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e18109d30 0x7f9e181a5ae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000e9f0 tx=0x7f9e0000edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.050+0000 7f9e15600700 1 -- 172.21.15.18:0/2161334503 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000cbd0 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.051+0000 7f9e15600700 1 -- 172.21.15.18:0/2161334503 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e0000cd70 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.051+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e181a0290 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.051+0000 7f9e15600700 1 -- 172.21.15.18:0/2161334503 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00010430 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.051+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e181ac520 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.052+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1810a860 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.054+0000 7f9e15600700 1 -- 172.21.15.18:0/2161334503 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9e00004750 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.055+0000 7f9e15600700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9dfc0a3d50 0x7f9dfc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.055+0000 7f9e15600700 1 -- 172.21.15.18:0/2161334503 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9e000d0bf0 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.055+0000 7f9e17e00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9dfc0a3d50 0x7f9dfc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.056+0000 7f9e17e00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9dfc0a3d50 0x7f9dfc0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009cb0 tx=0x7f9e08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.059+0000 7f9e15600700 1 -- 172.21.15.18:0/2161334503 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e0009d0b0 con 0x7f9e18109d30 2024-04-16T17:03:04.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.304+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9e181ac6f0 con 0x7f9e18109d30 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.304+0000 7f9e15600700 1 -- 172.21.15.18:0/2161334503 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9e0009c9b0 con 0x7f9e18109d30 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stdout:219043332152 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.307+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9dfc0a3d50 msgr2=0x7f9dfc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.307+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9dfc0a3d50 0x7f9dfc0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009cb0 tx=0x7f9e08009260 comp rx=0 tx=0).stop 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.307+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e18109d30 msgr2=0x7f9e181a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.307+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e18109d30 0x7f9e181a5ae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000e9f0 tx=0x7f9e0000edb0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.307+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 shutdown_connections 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.308+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9dfc0a3d50 0x7f9dfc0a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.308+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e1810e460 0x7f9e1819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.308+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18109380 0x7f9e181a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.308+0000 7f9e1ea00700 1 --2- 172.21.15.18:0/2161334503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e18109d30 0x7f9e181a5ae0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.308+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 >> 172.21.15.18:0/2161334503 conn(0x7f9e180762c0 msgr2=0x7f9e180767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:04.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.308+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 shutdown_connections 2024-04-16T17:03:04.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.308+0000 7f9e1ea00700 1 -- 172.21.15.18:0/2161334503 wait complete. 2024-04-16T17:03:04.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.366+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f247419a6b0 con 0x7f2474108b90 2024-04-16T17:03:04.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.367+0000 7f246b400700 1 -- 172.21.15.18:0/117874663 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f246c09d220 con 0x7f2474108b90 2024-04-16T17:03:04.369 INFO:teuthology.orchestra.run.smithi018.stdout:253403070515 2024-04-16T17:03:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f24580a3d50 msgr2=0x7f24580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f24580a3d50 0x7f24580a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2464009810 tx=0x7f2464019040 comp rx=0 tx=0).stop 2024-04-16T17:03:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2474108b90 msgr2=0x7f247419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2474108b90 0x7f247419dc20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f246c00c110 tx=0x7f246c00c420 comp rx=0 tx=0).stop 2024-04-16T17:03:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 shutdown_connections 2024-04-16T17:03:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f24580a3d50 0x7f24580a6210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2474108b90 0x7f247419dc20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2474102b40 0x7f2474199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 --2- 172.21.15.18:0/117874663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2474103500 0x7f2474199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.370+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 >> 172.21.15.18:0/117874663 conn(0x7f2474079c50 msgr2=0x7f2474101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.372+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 shutdown_connections 2024-04-16T17:03:04.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.372+0000 7f2478a00700 1 -- 172.21.15.18:0/117874663 wait complete. 2024-04-16T17:03:04.426 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:03 smithi022 ceph-mon[36023]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:05.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:05 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2161334503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T17:03:05.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:05 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/117874663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T17:03:05.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:05 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2161334503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T17:03:05.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:05 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/117874663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T17:03:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:05 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2161334503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T17:03:05.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:05 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/117874663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T17:03:05.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 -- 172.21.15.18:0/2207628890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 msgr2=0x7f7944102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:05.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 --2- 172.21.15.18:0/2207628890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 0x7f7944102fc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7934009990 tx=0x7f7934009ca0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 -- 172.21.15.18:0/2207628890 shutdown_connections 2024-04-16T17:03:05.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 --2- 172.21.15.18:0/2207628890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944103500 0x7f794410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 --2- 172.21.15.18:0/2207628890 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7944108b90 0x7f7944108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 --2- 172.21.15.18:0/2207628890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 0x7f7944102fc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 -- 172.21.15.18:0/2207628890 >> 172.21.15.18:0/2207628890 conn(0x7f7944079c50 msgr2=0x7f794407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 -- 172.21.15.18:0/2207628890 shutdown_connections 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.973+0000 7f794c800700 1 -- 172.21.15.18:0/2207628890 wait complete. 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.974+0000 7f794c800700 1 Processor -- start 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.974+0000 7f794c800700 1 -- start start 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.974+0000 7f794c800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 0x7f7944199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.974+0000 7f794c800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7944103500 0x7f79441998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.974+0000 7f794c800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944108b90 0x7f794419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.974+0000 7f794c800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7944112e50 con 0x7f7944103500 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.974+0000 7f794c800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7944112cd0 con 0x7f7944108b90 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.974+0000 7f794c800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7944112fd0 con 0x7f7944102b40 2024-04-16T17:03:05.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7944103500 0x7f79441998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 0x7f7944199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7944103500 0x7f79441998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45490/0 (socket says 172.21.15.18:45490) 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 0x7f7944199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59300/0 (socket says 172.21.15.18:59300) 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949400700 1 -- 172.21.15.18:0/867272058 learned_addr learned my addr 172.21.15.18:0/867272058 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f794a800700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944108b90 0x7f794419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949e00700 1 -- 172.21.15.18:0/867272058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944108b90 msgr2=0x7f794419dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949e00700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944108b90 0x7f794419dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949e00700 1 -- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7944103500 msgr2=0x7f79441998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949e00700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7944103500 0x7f79441998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949e00700 1 -- 172.21.15.18:0/867272058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794419e390 con 0x7f7944102b40 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.975+0000 7f7949400700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7944103500 0x7f79441998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.976+0000 7f7949e00700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 0x7f7944199380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f793800e9f0 tx=0x7f793800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.976+0000 7f793ea00700 1 -- 172.21.15.18:0/867272058 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793800c9c0 con 0x7f7944102b40 2024-04-16T17:03:05.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.976+0000 7f793ea00700 1 -- 172.21.15.18:0/867272058 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793800cb60 con 0x7f7944102b40 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.976+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794419e680 con 0x7f7944102b40 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.976+0000 7f793ea00700 1 -- 172.21.15.18:0/867272058 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938018680 con 0x7f7944102b40 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.976+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79441a5f00 con 0x7f7944102b40 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.977+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79440697e0 con 0x7f7944102b40 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.979+0000 7f793ea00700 1 -- 172.21.15.18:0/867272058 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7938010430 con 0x7f7944102b40 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.981+0000 7f793ea00700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f792c0a3cd0 0x7f792c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.981+0000 7f793ea00700 1 -- 172.21.15.18:0/867272058 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f79380d09c0 con 0x7f7944102b40 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.981+0000 7f7949400700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f792c0a3cd0 0x7f792c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.982+0000 7f7949400700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f792c0a3cd0 0x7f792c0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7934009750 tx=0x7f793400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:04.985+0000 7f793ea00700 1 -- 172.21.15.18:0/867272058 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f793809ce80 con 0x7f7944102b40 2024-04-16T17:03:05.676 INFO:teuthology.orchestra.run.smithi018.stdout:77309411413 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.228+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f794402dfe0 con 0x7f7944102b40 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.229+0000 7f793ea00700 1 -- 172.21.15.18:0/867272058 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f793809c780 con 0x7f7944102b40 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f792c0a3cd0 msgr2=0x7f792c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f792c0a3cd0 0x7f792c0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7934009750 tx=0x7f793400b280 comp rx=0 tx=0).stop 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 msgr2=0x7f7944199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 0x7f7944199380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f793800e9f0 tx=0x7f793800ed00 comp rx=0 tx=0).stop 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 shutdown_connections 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f792c0a3cd0 0x7f792c0a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944108b90 0x7f794419dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7944102b40 0x7f7944199380 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 --2- 172.21.15.18:0/867272058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7944103500 0x7f79441998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.232+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 >> 172.21.15.18:0/867272058 conn(0x7f7944079c50 msgr2=0x7f79441018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:05.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.233+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 shutdown_connections 2024-04-16T17:03:05.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:05.233+0000 7f794c800700 1 -- 172.21.15.18:0/867272058 wait complete. 2024-04-16T17:03:06.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:06 smithi022 ceph-mon[36023]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:06.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:06 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/867272058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T17:03:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:06 smithi018 ceph-mon[31776]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:06 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/867272058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T17:03:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:06 smithi043 ceph-mon[36901]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:06 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/867272058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T17:03:06.586 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070509 got 253403070515 for osd.8 2024-04-16T17:03:06.586 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:07.199 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411407 got 77309411413 for osd.2 2024-04-16T17:03:07.199 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:08.267 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:08 smithi018 ceph-mon[31776]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.157+0000 7f7cb1000700 1 -- 172.21.15.18:0/1720020340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cac109d70 msgr2=0x7f7cac10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.157+0000 7f7cb1000700 1 --2- 172.21.15.18:0/1720020340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cac109d70 0x7f7cac10dd70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c980099c0 tx=0x7f7c98009cd0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.158+0000 7f7cb1000700 1 -- 172.21.15.18:0/1720020340 shutdown_connections 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.158+0000 7f7cb1000700 1 --2- 172.21.15.18:0/1720020340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cac1093c0 0x7f7cac1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.158+0000 7f7cb1000700 1 --2- 172.21.15.18:0/1720020340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac10e450 0x7f7cac116020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.158+0000 7f7cb1000700 1 --2- 172.21.15.18:0/1720020340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cac109d70 0x7f7cac10dd70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.158+0000 7f7cb1000700 1 -- 172.21.15.18:0/1720020340 >> 172.21.15.18:0/1720020340 conn(0x7f7cac0762c0 msgr2=0x7f7cac0786e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.158+0000 7f7cb1000700 1 -- 172.21.15.18:0/1720020340 shutdown_connections 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.158+0000 7f7cb1000700 1 -- 172.21.15.18:0/1720020340 wait complete. 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.159+0000 7f7cb1000700 1 Processor -- start 2024-04-16T17:03:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.159+0000 7f7cb1000700 1 -- start start 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.159+0000 7f7cb1000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cac1093c0 0x7f7cac1a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.159+0000 7f7cb1000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac109d70 0x7f7cac1a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.159+0000 7f7cb1000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cac10e450 0x7f7cac19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.159+0000 7f7cb1000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac119480 con 0x7f7cac1093c0 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.159+0000 7f7cb1000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac119300 con 0x7f7cac10e450 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.159+0000 7f7cb1000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac119600 con 0x7f7cac109d70 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac109d70 0x7f7cac1a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cac1093c0 0x7f7cac1a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac109d70 0x7f7cac1a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42596/0 (socket says 172.21.15.18:42596) 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 -- 172.21.15.18:0/3552106894 learned_addr learned my addr 172.21.15.18:0/3552106894 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7caa800700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cac10e450 0x7f7cac19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 -- 172.21.15.18:0/3552106894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cac10e450 msgr2=0x7f7cac19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cac10e450 0x7f7cac19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 -- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cac1093c0 msgr2=0x7f7cac1a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cac1093c0 0x7f7cac1a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 -- 172.21.15.18:0/3552106894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac1a0060 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.160+0000 7f7ca9400700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac109d70 0x7f7cac1a5ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005f50 tx=0x7f7c98003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.161+0000 7f7ca2a00700 1 -- 172.21.15.18:0/3552106894 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98003680 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.161+0000 7f7ca2a00700 1 -- 172.21.15.18:0/3552106894 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c980055a0 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.161+0000 7f7ca2a00700 1 -- 172.21.15.18:0/3552106894 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800fce0 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.161+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac1a02f0 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.161+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac1ac410 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.162+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cac10a850 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.164+0000 7f7ca2a00700 1 -- 172.21.15.18:0/3552106894 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7c9800f460 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.165+0000 7f7ca2a00700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7c900a3e70 0x7f7c900a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.166+0000 7f7ca2a00700 1 -- 172.21.15.18:0/3552106894 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7c980d1860 con 0x7f7cac109d70 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.166+0000 7f7ca9e00700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7c900a3e70 0x7f7c900a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.166+0000 7f7ca9e00700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7c900a3e70 0x7f7c900a6330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009c60 tx=0x7f7c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:08.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.168+0000 7f7ca2a00700 1 -- 172.21.15.18:0/3552106894 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c98013070 con 0x7f7cac109d70 2024-04-16T17:03:08.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.420+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f7cac02dfe0 con 0x7f7cac109d70 2024-04-16T17:03:08.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.421+0000 7f7ca2a00700 1 -- 172.21.15.18:0/3552106894 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7c9809ddb0 con 0x7f7cac109d70 2024-04-16T17:03:08.423 INFO:teuthology.orchestra.run.smithi018.stdout:343597383714 2024-04-16T17:03:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:08 smithi022 ceph-mon[36023]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:08.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7c900a3e70 msgr2=0x7f7c900a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:08.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7c900a3e70 0x7f7c900a6330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009c60 tx=0x7f7c9c009260 comp rx=0 tx=0).stop 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac109d70 msgr2=0x7f7cac1a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac109d70 0x7f7cac1a5ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005f50 tx=0x7f7c98003a90 comp rx=0 tx=0).stop 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 shutdown_connections 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7c900a3e70 0x7f7c900a6330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cac10e450 0x7f7cac19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cac1093c0 0x7f7cac1a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 --2- 172.21.15.18:0/3552106894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac109d70 0x7f7cac1a5ba0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.424+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 >> 172.21.15.18:0/3552106894 conn(0x7f7cac0762c0 msgr2=0x7f7cac0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.425+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 shutdown_connections 2024-04-16T17:03:08.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.425+0000 7f7cb1000700 1 -- 172.21.15.18:0/3552106894 wait complete. 2024-04-16T17:03:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:08 smithi043 ceph-mon[36901]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:09.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:09 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3552106894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T17:03:09.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:09 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/590049904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T17:03:09.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:09 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3552106894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T17:03:09.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:09 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/590049904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T17:03:09.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:09 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3552106894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T17:03:09.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:09 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/590049904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 -- 172.21.15.18:0/898863330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4481023b0 msgr2=0x7fc44810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 --2- 172.21.15.18:0/898863330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4481023b0 0x7fc44810e8a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc440009f90 tx=0x7fc44000b330 comp rx=0 tx=0).stop 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 -- 172.21.15.18:0/898863330 shutdown_connections 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 --2- 172.21.15.18:0/898863330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4481023b0 0x7fc44810e8a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 --2- 172.21.15.18:0/898863330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc448107a40 0x7fc448107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 --2- 172.21.15.18:0/898863330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4481019f0 0x7fc448101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 -- 172.21.15.18:0/898863330 >> 172.21.15.18:0/898863330 conn(0x7fc4480789a0 msgr2=0x7fc448078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 -- 172.21.15.18:0/898863330 shutdown_connections 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.710+0000 7fc44e000700 1 -- 172.21.15.18:0/898863330 wait complete. 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.711+0000 7fc44e000700 1 Processor -- start 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc44e000700 1 -- start start 2024-04-16T17:03:09.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc44e000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4481019f0 0x7fc44819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc44e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4481023b0 0x7fc44819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc44e000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc448107a40 0x7fc44819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc44e000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc448111ce0 con 0x7fc4481023b0 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc44e000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc448111b60 con 0x7fc448107a40 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc44e000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc448111e60 con 0x7fc4481019f0 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc447400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4481019f0 0x7fc44819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc43fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4481023b0 0x7fc44819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc447400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4481019f0 0x7fc44819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42616/0 (socket says 172.21.15.18:42616) 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc447400700 1 -- 172.21.15.18:0/590049904 learned_addr learned my addr 172.21.15.18:0/590049904 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.712+0000 7fc447e00700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc448107a40 0x7fc44819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:09.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.713+0000 7fc447400700 1 -- 172.21.15.18:0/590049904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc448107a40 msgr2=0x7fc44819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stdout:55834574939 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.713+0000 7fc447400700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc448107a40 0x7fc44819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.713+0000 7fc447400700 1 -- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4481023b0 msgr2=0x7fc44819b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.713+0000 7fc447400700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4481023b0 0x7fc44819b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.713+0000 7fc447400700 1 -- 172.21.15.18:0/590049904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4481a0430 con 0x7fc4481019f0 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.713+0000 7fc43fe00700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4481023b0 0x7fc44819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.713+0000 7fc447e00700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc448107a40 0x7fc44819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.713+0000 7fc447400700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4481019f0 0x7fc44819b420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc43800e9a0 tx=0x7fc43800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:09.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.714+0000 7fc444c00700 1 -- 172.21.15.18:0/590049904 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43800ca10 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.714+0000 7fc444c00700 1 -- 172.21.15.18:0/590049904 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43800cbb0 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.714+0000 7fc444c00700 1 -- 172.21.15.18:0/590049904 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438018680 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.714+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4481a0720 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.714+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4481a7fa0 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.715+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4480697e0 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.717+0000 7fc444c00700 1 -- 172.21.15.18:0/590049904 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc438010430 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.719+0000 7fc444c00700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc42c0a3d50 0x7fc42c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.719+0000 7fc444c00700 1 -- 172.21.15.18:0/590049904 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc4380d0e90 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.719+0000 7fc43fe00700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc42c0a3d50 0x7fc42c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.720+0000 7fc43fe00700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc42c0a3d50 0x7fc42c0a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc430009750 tx=0x7fc430009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.721+0000 7fc444c00700 1 -- 172.21.15.18:0/590049904 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc43809d350 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.957+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc4481a8210 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.958+0000 7fc444c00700 1 -- 172.21.15.18:0/590049904 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc43809cc50 con 0x7fc4481019f0 2024-04-16T17:03:09.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.961+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc42c0a3d50 msgr2=0x7fc42c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.961+0000 7fc44e000700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc42c0a3d50 0x7fc42c0a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc430009750 tx=0x7fc430009310 comp rx=0 tx=0).stop 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.961+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4481019f0 msgr2=0x7fc44819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.961+0000 7fc44e000700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4481019f0 0x7fc44819b420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc43800e9a0 tx=0x7fc43800ecb0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.962+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 shutdown_connections 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.962+0000 7fc44e000700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc42c0a3d50 0x7fc42c0a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.962+0000 7fc44e000700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc448107a40 0x7fc44819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.962+0000 7fc44e000700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4481019f0 0x7fc44819b420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.962+0000 7fc44e000700 1 --2- 172.21.15.18:0/590049904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4481023b0 0x7fc44819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.962+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 >> 172.21.15.18:0/590049904 conn(0x7fc4480789a0 msgr2=0x7fc44806a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.962+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 shutdown_connections 2024-04-16T17:03:09.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:08.962+0000 7fc44e000700 1 -- 172.21.15.18:0/590049904 wait complete. 2024-04-16T17:03:10.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:10 smithi022 ceph-mon[36023]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:10 smithi018 ceph-mon[31776]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:10.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:10 smithi043 ceph-mon[36901]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.756+0000 7fd495c00700 1 -- 172.21.15.18:0/1707312320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49006b090 msgr2=0x7fd49006b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.756+0000 7fd495c00700 1 --2- 172.21.15.18:0/1707312320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49006b090 0x7fd49006b470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd480009990 tx=0x7fd480009ca0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.757+0000 7fd495c00700 1 -- 172.21.15.18:0/1707312320 shutdown_connections 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.757+0000 7fd495c00700 1 --2- 172.21.15.18:0/1707312320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49006b090 0x7fd49006b470 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.757+0000 7fd495c00700 1 --2- 172.21.15.18:0/1707312320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd49010f590 0x7fd490111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.757+0000 7fd495c00700 1 --2- 172.21.15.18:0/1707312320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd49006b9b0 0x7fd49010f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.757+0000 7fd495c00700 1 -- 172.21.15.18:0/1707312320 >> 172.21.15.18:0/1707312320 conn(0x7fd490078960 msgr2=0x7fd490078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.757+0000 7fd495c00700 1 -- 172.21.15.18:0/1707312320 shutdown_connections 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.757+0000 7fd495c00700 1 -- 172.21.15.18:0/1707312320 wait complete. 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.758+0000 7fd495c00700 1 Processor -- start 2024-04-16T17:03:10.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.758+0000 7fd495c00700 1 -- start start 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd495c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd49006b090 0x7fd49019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd495c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd49006b9b0 0x7fd49019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd495c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49010f590 0x7fd49019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd495c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd490115060 con 0x7fd49006b9b0 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd495c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd490114ee0 con 0x7fd49010f590 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd495c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901151e0 con 0x7fd49006b090 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd49006b090 0x7fd49019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd49006b090 0x7fd49019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42646/0 (socket says 172.21.15.18:42646) 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48ea00700 1 -- 172.21.15.18:0/134593889 learned_addr learned my addr 172.21.15.18:0/134593889 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48e000700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd49006b9b0 0x7fd49019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48f400700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49010f590 0x7fd49019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48ea00700 1 -- 172.21.15.18:0/134593889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49010f590 msgr2=0x7fd49019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48ea00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49010f590 0x7fd49019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48ea00700 1 -- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd49006b9b0 msgr2=0x7fd49019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:10.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48ea00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd49006b9b0 0x7fd49019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48ea00700 1 -- 172.21.15.18:0/134593889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4901a04d0 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48f400700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49010f590 0x7fd49019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.759+0000 7fd48e000700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd49006b9b0 0x7fd49019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.760+0000 7fd48ea00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd49006b090 0x7fd49019b510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd48000b820 tx=0x7fd48000bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.760+0000 7fd487400700 1 -- 172.21.15.18:0/134593889 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480003890 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.760+0000 7fd487400700 1 -- 172.21.15.18:0/134593889 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd480022470 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.760+0000 7fd487400700 1 -- 172.21.15.18:0/134593889 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48000f6a0 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.760+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4901a0760 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.760+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4901a0b90 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.762+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4900697e0 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.764+0000 7fd487400700 1 -- 172.21.15.18:0/134593889 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd480009d10 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.765+0000 7fd487400700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd4740a3d50 0x7fd4740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.765+0000 7fd487400700 1 -- 172.21.15.18:0/134593889 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd4800d1850 con 0x7fd49006b090 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.766+0000 7fd48e000700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd4740a3d50 0x7fd4740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:10.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.766+0000 7fd48e000700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd4740a3d50 0x7fd4740a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd490102ed0 tx=0x7fd47c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:10.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:10.769+0000 7fd487400700 1 -- 172.21.15.18:0/134593889 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd480013070 con 0x7fd49006b090 2024-04-16T17:03:11.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.014+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd49002e100 con 0x7fd49006b090 2024-04-16T17:03:11.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.015+0000 7fd487400700 1 -- 172.21.15.18:0/134593889 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd48009dda0 con 0x7fd49006b090 2024-04-16T17:03:11.016 INFO:teuthology.orchestra.run.smithi018.stdout:163208757318 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd4740a3d50 msgr2=0x7fd4740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd4740a3d50 0x7fd4740a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd490102ed0 tx=0x7fd47c009260 comp rx=0 tx=0).stop 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd49006b090 msgr2=0x7fd49019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd49006b090 0x7fd49019b510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd48000b820 tx=0x7fd48000bb60 comp rx=0 tx=0).stop 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 shutdown_connections 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd4740a3d50 0x7fd4740a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd49010f590 0x7fd49019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd49006b090 0x7fd49019b510 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 --2- 172.21.15.18:0/134593889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd49006b9b0 0x7fd49019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:11.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 >> 172.21.15.18:0/134593889 conn(0x7fd490078960 msgr2=0x7fd490110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:11.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 shutdown_connections 2024-04-16T17:03:11.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:11.018+0000 7fd495c00700 1 -- 172.21.15.18:0/134593889 wait complete. 2024-04-16T17:03:11.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:11 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/134593889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T17:03:11.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:11 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/134593889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T17:03:11.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:11 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/134593889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T17:03:11.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332148 got 219043332152 for osd.7 2024-04-16T17:03:11.672 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:12.288 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:12 smithi018 ceph-mon[31776]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:12.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:12 smithi022 ceph-mon[36023]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:12.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:12 smithi043 ceph-mon[36901]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:12.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383709 got 343597383714 for osd.11 2024-04-16T17:03:12.532 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:13.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.056+0000 7fe39fc00700 1 -- 172.21.15.18:0/304863490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe398107a30 msgr2=0x7fe398107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.057+0000 7fe39fc00700 1 --2- 172.21.15.18:0/304863490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe398107a30 0x7fe398107e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe380009990 tx=0x7fe380009ca0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.057+0000 7fe39fc00700 1 -- 172.21.15.18:0/304863490 shutdown_connections 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.057+0000 7fe39fc00700 1 --2- 172.21.15.18:0/304863490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3981023a0 0x7fe39810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.057+0000 7fe39fc00700 1 --2- 172.21.15.18:0/304863490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe398107a30 0x7fe398107e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.057+0000 7fe39fc00700 1 --2- 172.21.15.18:0/304863490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3981019e0 0x7fe398101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.057+0000 7fe39fc00700 1 -- 172.21.15.18:0/304863490 >> 172.21.15.18:0/304863490 conn(0x7fe398078960 msgr2=0x7fe398078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.057+0000 7fe39fc00700 1 -- 172.21.15.18:0/304863490 shutdown_connections 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.057+0000 7fe39fc00700 1 -- 172.21.15.18:0/304863490 wait complete. 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.058+0000 7fe39fc00700 1 Processor -- start 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.058+0000 7fe39fc00700 1 -- start start 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.058+0000 7fe39fc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3981019e0 0x7fe39819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.058+0000 7fe39fc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3981023a0 0x7fe39819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39fc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe398107a30 0x7fe39819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39fc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe398111ce0 con 0x7fe3981023a0 2024-04-16T17:03:13.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39fc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe398111b60 con 0x7fe398107a30 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39fc00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe398111e60 con 0x7fe3981019e0 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39d200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3981019e0 0x7fe39819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe38fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3981023a0 0x7fe39819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39d200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3981019e0 0x7fe39819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42662/0 (socket says 172.21.15.18:42662) 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39dc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe398107a30 0x7fe39819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39d200700 1 -- 172.21.15.18:0/18205907 learned_addr learned my addr 172.21.15.18:0/18205907 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39d200700 1 -- 172.21.15.18:0/18205907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe398107a30 msgr2=0x7fe39819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39d200700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe398107a30 0x7fe39819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39d200700 1 -- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3981023a0 msgr2=0x7fe39819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39d200700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3981023a0 0x7fe39819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.059+0000 7fe39d200700 1 -- 172.21.15.18:0/18205907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3981a04b0 con 0x7fe3981019e0 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.060+0000 7fe39d200700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3981019e0 0x7fe39819b4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe380000c00 tx=0x7fe38000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.060+0000 7fe396000700 1 -- 172.21.15.18:0/18205907 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380027070 con 0x7fe3981019e0 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.060+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3981a0740 con 0x7fe3981019e0 2024-04-16T17:03:13.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.060+0000 7fe396000700 1 -- 172.21.15.18:0/18205907 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe380003d70 con 0x7fe3981019e0 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.060+0000 7fe396000700 1 -- 172.21.15.18:0/18205907 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38000f460 con 0x7fe3981019e0 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.060+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981a8020 con 0x7fe3981019e0 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.062+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3980697e0 con 0x7fe3981019e0 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.063+0000 7fe396000700 1 -- 172.21.15.18:0/18205907 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe38002a070 con 0x7fe3981019e0 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.065+0000 7fe396000700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3780a3cd0 0x7fe3780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.065+0000 7fe396000700 1 -- 172.21.15.18:0/18205907 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe3800d16e0 con 0x7fe3981019e0 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.065+0000 7fe38fe00700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3780a3cd0 0x7fe3780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.066+0000 7fe38fe00700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3780a3cd0 0x7fe3780a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe39810ecd0 tx=0x7fe388009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:13.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.069+0000 7fe396000700 1 -- 172.21.15.18:0/18205907 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe380013070 con 0x7fe3981019e0 2024-04-16T17:03:13.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.314+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe39802dfe0 con 0x7fe3981019e0 2024-04-16T17:03:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.315+0000 7fe396000700 1 -- 172.21.15.18:0/18205907 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe38009dc30 con 0x7fe3981019e0 2024-04-16T17:03:13.317 INFO:teuthology.orchestra.run.smithi018.stdout:197568495678 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.317+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3780a3cd0 msgr2=0x7fe3780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3780a3cd0 0x7fe3780a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe39810ecd0 tx=0x7fe388009260 comp rx=0 tx=0).stop 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3981019e0 msgr2=0x7fe39819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3981019e0 0x7fe39819b4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe380000c00 tx=0x7fe38000bab0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 shutdown_connections 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3780a3cd0 0x7fe3780a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe398107a30 0x7fe39819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3981019e0 0x7fe39819b4a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 --2- 172.21.15.18:0/18205907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3981023a0 0x7fe39819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 >> 172.21.15.18:0/18205907 conn(0x7fe398078960 msgr2=0x7fe398100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:13.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 shutdown_connections 2024-04-16T17:03:13.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:13.318+0000 7fe39fc00700 1 -- 172.21.15.18:0/18205907 wait complete. 2024-04-16T17:03:13.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757311 got 163208757318 for osd.5 2024-04-16T17:03:13.461 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:13.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:13 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:13.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:13 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:13.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:13 smithi022 ceph-mon[36023]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:13.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:13 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/18205907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T17:03:13.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:13 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:13.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:13 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:13.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:13 smithi018 ceph-mon[31776]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:13.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:13 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/18205907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T17:03:14.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:13 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:14.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:13 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:14.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:13 smithi043 ceph-mon[36901]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:14.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:13 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/18205907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T17:03:14.076 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645346 got 309237645351 for osd.10 2024-04-16T17:03:14.076 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:14.338 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:14.380 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574935 got 55834574939 for osd.1 2024-04-16T17:03:14.380 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:14.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:14.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:14.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:14 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:15.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:14 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:15.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:14 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:15.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:14 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:15.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:14 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:15.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:14 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:15.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:14 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:03:15.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.148+0000 7fe798c00700 1 -- 172.21.15.18:0/3952093122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe794109290 msgr2=0x7fe794109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.148+0000 7fe798c00700 1 --2- 172.21.15.18:0/3952093122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe794109290 0x7fe794109670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe7840099c0 tx=0x7fe784009cd0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.149+0000 7fe798c00700 1 -- 172.21.15.18:0/3952093122 shutdown_connections 2024-04-16T17:03:15.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.149+0000 7fe798c00700 1 --2- 172.21.15.18:0/3952093122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe79410e360 0x7fe794115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.149+0000 7fe798c00700 1 --2- 172.21.15.18:0/3952093122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe794109290 0x7fe794109670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.149+0000 7fe798c00700 1 --2- 172.21.15.18:0/3952093122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe794109c40 0x7fe79410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.149+0000 7fe798c00700 1 -- 172.21.15.18:0/3952093122 >> 172.21.15.18:0/3952093122 conn(0x7fe7940788f0 msgr2=0x7fe794107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.149+0000 7fe798c00700 1 -- 172.21.15.18:0/3952093122 shutdown_connections 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.149+0000 7fe798c00700 1 -- 172.21.15.18:0/3952093122 wait complete. 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.150+0000 7fe798c00700 1 Processor -- start 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.150+0000 7fe798c00700 1 -- start start 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe798c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe794109290 0x7fe7941a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe798c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe794109c40 0x7fe7941a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe798c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe79410e360 0x7fe79419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe798c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe794119390 con 0x7fe79410e360 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe798c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe794119210 con 0x7fe794109c40 2024-04-16T17:03:15.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe798c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe794119510 con 0x7fe794109290 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe793400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe794109290 0x7fe7941a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe793e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe79410e360 0x7fe79419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe793400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe794109290 0x7fe7941a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42688/0 (socket says 172.21.15.18:42688) 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe793400700 1 -- 172.21.15.18:0/3230561667 learned_addr learned my addr 172.21.15.18:0/3230561667 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.151+0000 7fe78aa00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe794109c40 0x7fe7941a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe793400700 1 -- 172.21.15.18:0/3230561667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe794109c40 msgr2=0x7fe7941a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe793400700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe794109c40 0x7fe7941a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe793400700 1 -- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe79410e360 msgr2=0x7fe79419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe793400700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe79410e360 0x7fe79419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe793400700 1 -- 172.21.15.18:0/3230561667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79419fdd0 con 0x7fe794109290 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe793400700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe794109290 0x7fe7941a5460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe784006010 tx=0x7fe784003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe790c00700 1 -- 172.21.15.18:0/3230561667 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784003e50 con 0x7fe794109290 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe790c00700 1 -- 172.21.15.18:0/3230561667 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7840055a0 con 0x7fe794109290 2024-04-16T17:03:15.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7941a0060 con 0x7fe794109290 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.152+0000 7fe790c00700 1 -- 172.21.15.18:0/3230561667 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78400fe30 con 0x7fe794109290 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.153+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7941ac330 con 0x7fe794109290 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.154+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79410a760 con 0x7fe794109290 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.156+0000 7fe790c00700 1 -- 172.21.15.18:0/3230561667 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe78400f4f0 con 0x7fe794109290 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.157+0000 7fe790c00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe7780a3e70 0x7fe7780a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.157+0000 7fe790c00700 1 -- 172.21.15.18:0/3230561667 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7840d1870 con 0x7fe794109290 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.158+0000 7fe78aa00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe7780a3e70 0x7fe7780a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.158+0000 7fe78aa00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe7780a3e70 0x7fe7780a6330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe794079710 tx=0x7fe77c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.162+0000 7fe790c00700 1 -- 172.21.15.18:0/3230561667 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe784013070 con 0x7fe794109290 2024-04-16T17:03:15.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.401+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe7940697e0 con 0x7fe794109290 2024-04-16T17:03:15.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.402+0000 7fe790c00700 1 -- 172.21.15.18:0/3230561667 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe78409ddc0 con 0x7fe794109290 2024-04-16T17:03:15.404 INFO:teuthology.orchestra.run.smithi018.stdout:107374182480 2024-04-16T17:03:15.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe7780a3e70 msgr2=0x7fe7780a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe7780a3e70 0x7fe7780a6330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe794079710 tx=0x7fe77c009260 comp rx=0 tx=0).stop 2024-04-16T17:03:15.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe794109290 msgr2=0x7fe7941a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe794109290 0x7fe7941a5460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe784006010 tx=0x7fe784003a90 comp rx=0 tx=0).stop 2024-04-16T17:03:15.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 shutdown_connections 2024-04-16T17:03:15.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe7780a3e70 0x7fe7780a6330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe79410e360 0x7fe79419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe794109290 0x7fe7941a5460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 --2- 172.21.15.18:0/3230561667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe794109c40 0x7fe7941a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.405+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 >> 172.21.15.18:0/3230561667 conn(0x7fe7940788f0 msgr2=0x7fe79407a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:15.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.406+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 shutdown_connections 2024-04-16T17:03:15.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:15.406+0000 7fe798c00700 1 -- 172.21.15.18:0/3230561667 wait complete. 2024-04-16T17:03:15.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495673 got 197568495678 for osd.6 2024-04-16T17:03:15.808 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:15.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:15 smithi018 ceph-mon[31776]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:15.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:15 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3230561667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T17:03:16.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:15 smithi043 ceph-mon[36901]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:16.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:15 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3230561667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T17:03:16.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:15 smithi022 ceph-mon[36023]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:16.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:15 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3230561667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T17:03:16.637 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182473 got 107374182480 for osd.3 2024-04-16T17:03:16.637 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.446+0000 7f9911000700 1 -- 172.21.15.18:0/1281726808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f990c10e460 msgr2=0x7f990c116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.446+0000 7f9911000700 1 --2- 172.21.15.18:0/1281726808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f990c10e460 0x7f990c116030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f990400b0d0 tx=0x7f990400b3e0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.446+0000 7f9911000700 1 -- 172.21.15.18:0/1281726808 shutdown_connections 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.446+0000 7f9911000700 1 --2- 172.21.15.18:0/1281726808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f990c10e460 0x7f990c116030 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.446+0000 7f9911000700 1 --2- 172.21.15.18:0/1281726808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f990c109380 0x7f990c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.446+0000 7f9911000700 1 --2- 172.21.15.18:0/1281726808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c109d30 0x7f990c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.446+0000 7f9911000700 1 -- 172.21.15.18:0/1281726808 >> 172.21.15.18:0/1281726808 conn(0x7f990c0762c0 msgr2=0x7f990c107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.446+0000 7f9911000700 1 -- 172.21.15.18:0/1281726808 shutdown_connections 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.447+0000 7f9911000700 1 -- 172.21.15.18:0/1281726808 wait complete. 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9911000700 1 Processor -- start 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9911000700 1 -- start start 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9911000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f990c109380 0x7f990c1a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9911000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c109d30 0x7f990c1a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9911000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f990c10e460 0x7f990c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9911000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1194c0 con 0x7f990c109380 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9911000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f990c119340 con 0x7f990c109d30 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9911000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f990c119640 con 0x7f990c10e460 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9909e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f990c109380 0x7f990c1a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9909e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f990c109380 0x7f990c1a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48390/0 (socket says 172.21.15.18:48390) 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.448+0000 7f9909e00700 1 -- 172.21.15.18:0/410640880 learned_addr learned my addr 172.21.15.18:0/410640880 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.449+0000 7f9901400700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c109d30 0x7f990c1a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.449+0000 7f990a800700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f990c10e460 0x7f990c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.449+0000 7f9909e00700 1 -- 172.21.15.18:0/410640880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f990c10e460 msgr2=0x7f990c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.449+0000 7f9909e00700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f990c10e460 0x7f990c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.449+0000 7f9909e00700 1 -- 172.21.15.18:0/410640880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c109d30 msgr2=0x7f990c1a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.449+0000 7f9909e00700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c109d30 0x7f990c1a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.449+0000 7f9909e00700 1 -- 172.21.15.18:0/410640880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c19ff60 con 0x7f990c109380 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.449+0000 7f9909e00700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f990c109380 0x7f990c1a55b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00e9a0 tx=0x7f98fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.450+0000 7f9903400700 1 -- 172.21.15.18:0/410640880 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00ca10 con 0x7f990c109380 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.450+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c1a0250 con 0x7f990c109380 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.450+0000 7f9903400700 1 -- 172.21.15.18:0/410640880 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98fc00cbb0 con 0x7f990c109380 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.450+0000 7f9903400700 1 -- 172.21.15.18:0/410640880 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc018680 con 0x7f990c109380 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.450+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c1ac530 con 0x7f990c109380 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.451+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990c10a860 con 0x7f990c109380 2024-04-16T17:03:16.729 INFO:teuthology.orchestra.run.smithi018.stdout:141733920844 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.453+0000 7f9903400700 1 -- 172.21.15.18:0/410640880 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f98fc010430 con 0x7f990c109380 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.454+0000 7f9903400700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f98f00a3d50 0x7f98f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.455+0000 7f9903400700 1 -- 172.21.15.18:0/410640880 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f98fc0d0ad0 con 0x7f990c109380 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.455+0000 7f9901400700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f98f00a3d50 0x7f98f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.456+0000 7f9901400700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f98f00a3d50 0x7f98f00a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f990c1a0e40 tx=0x7f98f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.459+0000 7f9903400700 1 -- 172.21.15.18:0/410640880 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98fc09cf90 con 0x7f990c109380 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.704+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f990c0697e0 con 0x7f990c109380 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.705+0000 7f9903400700 1 -- 172.21.15.18:0/410640880 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f98fc085e20 con 0x7f990c109380 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.707+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f98f00a3d50 msgr2=0x7f98f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.707+0000 7f9911000700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f98f00a3d50 0x7f98f00a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f990c1a0e40 tx=0x7f98f4009260 comp rx=0 tx=0).stop 2024-04-16T17:03:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.707+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f990c109380 msgr2=0x7f990c1a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.707+0000 7f9911000700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f990c109380 0x7f990c1a55b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00e9a0 tx=0x7f98fc00ecb0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.708+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 shutdown_connections 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.708+0000 7f9911000700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f98f00a3d50 0x7f98f00a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.708+0000 7f9911000700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f990c10e460 0x7f990c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.708+0000 7f9911000700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f990c109380 0x7f990c1a55b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.708+0000 7f9911000700 1 --2- 172.21.15.18:0/410640880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c109d30 0x7f990c1a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.708+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 >> 172.21.15.18:0/410640880 conn(0x7f990c0762c0 msgr2=0x7f990c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.708+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 shutdown_connections 2024-04-16T17:03:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:16.708+0000 7f9911000700 1 -- 172.21.15.18:0/410640880 wait complete. 2024-04-16T17:03:16.974 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:16 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/410640880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T17:03:17.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:16 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/410640880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T17:03:17.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:16 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/410640880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T17:03:17.742 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920837 got 141733920844 for osd.4 2024-04-16T17:03:17.742 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:17 smithi018 ceph-mon[31776]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:17 smithi043 ceph-mon[36901]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:18.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:17 smithi022 ceph-mon[36023]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:19.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.646+0000 7f89cd200700 1 -- 172.21.15.18:0/2077907463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8104e60 msgr2=0x7f89c8105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:19.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.646+0000 7f89cd200700 1 --2- 172.21.15.18:0/2077907463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8104e60 0x7f89c8105240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009990 tx=0x7f89b8009ca0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.646+0000 7f89cd200700 1 -- 172.21.15.18:0/2077907463 shutdown_connections 2024-04-16T17:03:19.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.646+0000 7f89cd200700 1 --2- 172.21.15.18:0/2077907463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8104e60 0x7f89c8105240 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.646+0000 7f89cd200700 1 --2- 172.21.15.18:0/2077907463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89c8109f40 0x7f89c8111b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.646+0000 7f89cd200700 1 --2- 172.21.15.18:0/2077907463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89c8105810 0x7f89c8109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.646+0000 7f89cd200700 1 -- 172.21.15.18:0/2077907463 >> 172.21.15.18:0/2077907463 conn(0x7f89c81006d0 msgr2=0x7f89c8102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:19.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.646+0000 7f89cd200700 1 -- 172.21.15.18:0/2077907463 shutdown_connections 2024-04-16T17:03:19.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.647+0000 7f89cd200700 1 -- 172.21.15.18:0/2077907463 wait complete. 2024-04-16T17:03:19.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89cd200700 1 Processor -- start 2024-04-16T17:03:19.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89cd200700 1 -- start start 2024-04-16T17:03:19.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89cd200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89c8104e60 0x7f89c819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:19.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89cd200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8105810 0x7f89c819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:19.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89cd200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89c8109f40 0x7f89c819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:19.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89cd200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8114f20 con 0x7f89c8104e60 2024-04-16T17:03:19.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89cd200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89c8114da0 con 0x7f89c8105810 2024-04-16T17:03:19.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89cd200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f89c81150a0 con 0x7f89c8109f40 2024-04-16T17:03:19.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.648+0000 7f89c7e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89c8109f40 0x7f89c819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:19.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89c8104e60 0x7f89c819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:19.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8105810 0x7f89c819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:19.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89c8109f40 0x7f89c819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48180/0 (socket says 172.21.15.18:48180) 2024-04-16T17:03:19.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7e00700 1 -- 172.21.15.18:0/3827549935 learned_addr learned my addr 172.21.15.18:0/3827549935 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:19.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7e00700 1 -- 172.21.15.18:0/3827549935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8105810 msgr2=0x7f89c819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:19.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7e00700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8105810 0x7f89c819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7e00700 1 -- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89c8104e60 msgr2=0x7f89c819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:19.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7e00700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89c8104e60 0x7f89c819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7e00700 1 -- 172.21.15.18:0/3827549935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c81a03b0 con 0x7f89c8109f40 2024-04-16T17:03:19.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c6a00700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8105810 0x7f89c819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:19.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.649+0000 7f89c7e00700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89c8109f40 0x7f89c819fc70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c370 tx=0x7f89c000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:19.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.650+0000 7f89bfe00700 1 -- 172.21.15.18:0/3827549935 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c00092e0 con 0x7f89c8109f40 2024-04-16T17:03:19.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.650+0000 7f89bfe00700 1 -- 172.21.15.18:0/3827549935 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c0003a00 con 0x7f89c8109f40 2024-04-16T17:03:19.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.650+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c81a0640 con 0x7f89c8109f40 2024-04-16T17:03:19.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.650+0000 7f89bfe00700 1 -- 172.21.15.18:0/3827549935 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c00047c0 con 0x7f89c8109f40 2024-04-16T17:03:19.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.650+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c81a7f20 con 0x7f89c8109f40 2024-04-16T17:03:19.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.651+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c80697e0 con 0x7f89c8109f40 2024-04-16T17:03:19.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.653+0000 7f89bfe00700 1 -- 172.21.15.18:0/3827549935 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f89c001a020 con 0x7f89c8109f40 2024-04-16T17:03:19.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.655+0000 7f89bfe00700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f89ac0a3d50 0x7f89ac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:19.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.655+0000 7f89bfe00700 1 -- 172.21.15.18:0/3827549935 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f89c00d0140 con 0x7f89c8109f40 2024-04-16T17:03:19.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.655+0000 7f89c7400700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f89ac0a3d50 0x7f89ac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:19.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.656+0000 7f89c7400700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f89ac0a3d50 0x7f89ac0a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f89b80098e0 tx=0x7f89b8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:19.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.659+0000 7f89bfe00700 1 -- 172.21.15.18:0/3827549935 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89c0010040 con 0x7f89c8109f40 2024-04-16T17:03:19.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.894+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f89c802dfe0 con 0x7f89c8109f40 2024-04-16T17:03:19.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.895+0000 7f89bfe00700 1 -- 172.21.15.18:0/3827549935 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f89c009c600 con 0x7f89c8109f40 2024-04-16T17:03:19.897 INFO:teuthology.orchestra.run.smithi018.stdout:34359738467 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f89ac0a3d50 msgr2=0x7f89ac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f89ac0a3d50 0x7f89ac0a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f89b80098e0 tx=0x7f89b8005960 comp rx=0 tx=0).stop 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89c8109f40 msgr2=0x7f89c819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89c8109f40 0x7f89c819fc70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c370 tx=0x7f89c000c680 comp rx=0 tx=0).stop 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 shutdown_connections 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f89ac0a3d50 0x7f89ac0a6210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89c8109f40 0x7f89c819fc70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89c8104e60 0x7f89c819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 --2- 172.21.15.18:0/3827549935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8105810 0x7f89c819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:19.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.898+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 >> 172.21.15.18:0/3827549935 conn(0x7f89c81006d0 msgr2=0x7f89c81090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:19.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.899+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 shutdown_connections 2024-04-16T17:03:19.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:19.899+0000 7f89cd200700 1 -- 172.21.15.18:0/3827549935 wait complete. 2024-04-16T17:03:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:19 smithi018 ceph-mon[31776]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:20.197 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:19 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3827549935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T17:03:20.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:19 smithi043 ceph-mon[36901]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:20.253 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:19 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3827549935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T17:03:20.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:19 smithi022 ceph-mon[36023]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:03:20.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:19 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3827549935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T17:03:20.709 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738460 got 34359738467 for osd.0 2024-04-16T17:03:20.709 DEBUG:teuthology.parallel:result is None 2024-04-16T17:03:20.709 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-16T17:03:20.709 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:03:21.186 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:22.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:21 smithi043 ceph-mon[36901]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:22.262 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:21 smithi018 ceph-mon[31776]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:22.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:21 smithi022 ceph-mon[36023]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:23.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 -- 172.21.15.18:0/3543178253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f801010f0 msgr2=0x7f7f801014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 --2- 172.21.15.18:0/3543178253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f801010f0 0x7f7f801014d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009990 tx=0x7f7f70009ca0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 -- 172.21.15.18:0/3543178253 shutdown_connections 2024-04-16T17:03:23.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 --2- 172.21.15.18:0/3543178253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8010d070 0x7f7f8010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 --2- 172.21.15.18:0/3543178253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f801010f0 0x7f7f801014d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 --2- 172.21.15.18:0/3543178253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f80101a10 0x7f7f8010caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 -- 172.21.15.18:0/3543178253 >> 172.21.15.18:0/3543178253 conn(0x7f7f800767b0 msgr2=0x7f7f80076bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:23.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 -- 172.21.15.18:0/3543178253 shutdown_connections 2024-04-16T17:03:23.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.470+0000 7f7f85c00700 1 -- 172.21.15.18:0/3543178253 wait complete. 2024-04-16T17:03:23.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.471+0000 7f7f85c00700 1 Processor -- start 2024-04-16T17:03:23.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.471+0000 7f7f85c00700 1 -- start start 2024-04-16T17:03:23.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f85c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f801010f0 0x7f7f8019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:23.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f85c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80101a10 0x7f7f8019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:23.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f85c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8010d070 0x7f7f8019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:23.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f85c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80112dc0 con 0x7f7f80101a10 2024-04-16T17:03:23.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f85c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80112c40 con 0x7f7f801010f0 2024-04-16T17:03:23.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f85c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80112f40 con 0x7f7f8010d070 2024-04-16T17:03:23.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f7f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80101a10 0x7f7f8019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f7f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80101a10 0x7f7f8019b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57994/0 (socket says 172.21.15.18:57994) 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f7f400700 1 -- 172.21.15.18:0/2498087628 learned_addr learned my addr 172.21.15.18:0/2498087628 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f85200700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8010d070 0x7f7f8019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.472+0000 7f7f7fe00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f801010f0 0x7f7f8019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.473+0000 7f7f7f400700 1 -- 172.21.15.18:0/2498087628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8010d070 msgr2=0x7f7f8019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.473+0000 7f7f7f400700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8010d070 0x7f7f8019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.473+0000 7f7f7f400700 1 -- 172.21.15.18:0/2498087628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f801010f0 msgr2=0x7f7f8019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.473+0000 7f7f7f400700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f801010f0 0x7f7f8019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.473+0000 7f7f7f400700 1 -- 172.21.15.18:0/2498087628 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f801a0430 con 0x7f7f80101a10 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.473+0000 7f7f7f400700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80101a10 0x7f7f8019b960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e9c0 tx=0x7f7f6c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.473+0000 7f7f7cc00700 1 -- 172.21.15.18:0/2498087628 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c004a30 con 0x7f7f80101a10 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.474+0000 7f7f7cc00700 1 -- 172.21.15.18:0/2498087628 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f6c00cca0 con 0x7f7f80101a10 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.474+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f801a0720 con 0x7f7f80101a10 2024-04-16T17:03:23.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.474+0000 7f7f7cc00700 1 -- 172.21.15.18:0/2498087628 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c005610 con 0x7f7f80101a10 2024-04-16T17:03:23.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.474+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801a7fa0 con 0x7f7f80101a10 2024-04-16T17:03:23.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.475+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f800697e0 con 0x7f7f80101a10 2024-04-16T17:03:23.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.477+0000 7f7f7cc00700 1 -- 172.21.15.18:0/2498087628 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7f6c00c7c0 con 0x7f7f80101a10 2024-04-16T17:03:23.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.478+0000 7f7f7cc00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7f640a3cd0 0x7f7f640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:23.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.479+0000 7f7f7cc00700 1 -- 172.21.15.18:0/2498087628 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7f6c0d0a30 con 0x7f7f80101a10 2024-04-16T17:03:23.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.479+0000 7f7f7fe00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7f640a3cd0 0x7f7f640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:23.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.480+0000 7f7f7fe00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7f640a3cd0 0x7f7f640a6190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000b2e0 tx=0x7f7f70005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:23.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.485+0000 7f7f7cc00700 1 -- 172.21.15.18:0/2498087628 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f6c09cef0 con 0x7f7f80101a10 2024-04-16T17:03:23.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.715+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f800fe580 con 0x7f7f640a3cd0 2024-04-16T17:03:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.728+0000 7f7f7cc00700 1 -- 172.21.15.18:0/2498087628 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340651 (secure 0 0 0) 0x7f7f800fe580 con 0x7f7f640a3cd0 2024-04-16T17:03:23.737 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.735+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7f640a3cd0 msgr2=0x7f7f640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.735+0000 7f7f85c00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7f640a3cd0 0x7f7f640a6190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000b2e0 tx=0x7f7f70005fd0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80101a10 msgr2=0x7f7f8019b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80101a10 0x7f7f8019b960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e9c0 tx=0x7f7f6c00ecd0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 shutdown_connections 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f7f640a3cd0 0x7f7f640a6190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f801010f0 0x7f7f8019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8010d070 0x7f7f8019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 --2- 172.21.15.18:0/2498087628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f80101a10 0x7f7f8019b960 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 >> 172.21.15.18:0/2498087628 conn(0x7f7f800767b0 msgr2=0x7f7f800fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 shutdown_connections 2024-04-16T17:03:23.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:23.736+0000 7f7f85c00700 1 -- 172.21.15.18:0/2498087628 wait complete. 2024-04-16T17:03:23.741 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:03:24.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:23 smithi043 ceph-mon[36901]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:24.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:23 smithi022 ceph-mon[36023]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:24.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:23 smithi018 ceph-mon[31776]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:24.547 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-04-16T17:03:22.933075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658418,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":657,"num_read_kb":453,"num_write":347,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373852,"kb_used_data":12152,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567972,"statfs":{"total":1151940427776,"available":1151557603328,"internally_reserved":0,"allocated":12443648,"data_stored":7961060,"data_compressed":360267,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005617"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.305491+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:02:26.305491+0000","last_peered":"2024-04-16T17:02:26.305491+0000","last_clean":"2024-04-16T17:02:26.305491+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:02:26.305491+0000","last_undegraded":"2024-04-16T17:02:26.305491+0000","last_fullsized":"2024-04-16T17:02:26.305491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314053+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:02:26.314053+0000","last_peered":"2024-04-16T17:02:26.314053+0000","last_clean":"2024-04-16T17:02:26.314053+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:02:26.314053+0000","last_undegraded":"2024-04-16T17:02:26.314053+0000","last_fullsized":"2024-04-16T17:02:26.314053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314410+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:02:26.314410+0000","last_peered":"2024-04-16T17:02:26.314410+0000","last_clean":"2024-04-16T17:02:26.314410+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:02:26.314410+0000","last_undegraded":"2024-04-16T17:02:26.314410+0000","last_fullsized":"2024-04-16T17:02:26.314410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.319837+0000","last_change":"2024-04-16T17:02:26.318719+0000","last_active":"2024-04-16T17:02:26.319837+0000","last_peered":"2024-04-16T17:02:26.319837+0000","last_clean":"2024-04-16T17:02:26.319837+0000","last_became_active":"2024-04-16T17:02:26.317743+0000","last_became_peered":"2024-04-16T17:02:26.317743+0000","last_unstale":"2024-04-16T17:02:26.319837+0000","last_undegraded":"2024-04-16T17:02:26.319837+0000","last_fullsized":"2024-04-16T17:02:26.319837+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:26.827859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302786+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:02:26.302786+0000","last_peered":"2024-04-16T17:02:26.302786+0000","last_clean":"2024-04-16T17:02:26.302786+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:02:26.302786+0000","last_undegraded":"2024-04-16T17:02:26.302786+0000","last_fullsized":"2024-04-16T17:02:26.302786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315910+0000","last_change":"2024-04-16T17:02:26.315910+0000","last_active":"2024-04-16T17:02:26.315910+0000","last_peered":"2024-04-16T17:02:26.315910+0000","last_clean":"2024-04-16T17:02:26.315910+0000","last_became_active":"2024-04-16T17:02:26.315493+0000","last_became_peered":"2024-04-16T17:02:26.315493+0000","last_unstale":"2024-04-16T17:02:26.315910+0000","last_undegraded":"2024-04-16T17:02:26.315910+0000","last_fullsized":"2024-04-16T17:02:26.315910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:27:48.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306354+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:02:26.306354+0000","last_peered":"2024-04-16T17:02:26.306354+0000","last_clean":"2024-04-16T17:02:26.306354+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:02:26.306354+0000","last_undegraded":"2024-04-16T17:02:26.306354+0000","last_fullsized":"2024-04-16T17:02:26.306354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.316447+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:02:26.316447+0000","last_peered":"2024-04-16T17:02:26.316447+0000","last_clean":"2024-04-16T17:02:26.316447+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:02:26.316447+0000","last_undegraded":"2024-04-16T17:02:26.316447+0000","last_fullsized":"2024-04-16T17:02:26.316447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308040+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:02:26.308040+0000","last_peered":"2024-04-16T17:02:26.308040+0000","last_clean":"2024-04-16T17:02:26.308040+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:02:26.308040+0000","last_undegraded":"2024-04-16T17:02:26.308040+0000","last_fullsized":"2024-04-16T17:02:26.308040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304968+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:02:26.304968+0000","last_peered":"2024-04-16T17:02:26.304968+0000","last_clean":"2024-04-16T17:02:26.304968+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:02:26.304968+0000","last_undegraded":"2024-04-16T17:02:26.304968+0000","last_fullsized":"2024-04-16T17:02:26.304968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306704+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:02:26.306704+0000","last_peered":"2024-04-16T17:02:26.306704+0000","last_clean":"2024-04-16T17:02:26.306704+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:02:26.306704+0000","last_undegraded":"2024-04-16T17:02:26.306704+0000","last_fullsized":"2024-04-16T17:02:26.306704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302850+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:02:26.302850+0000","last_peered":"2024-04-16T17:02:26.302850+0000","last_clean":"2024-04-16T17:02:26.302850+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:02:26.302850+0000","last_undegraded":"2024-04-16T17:02:26.302850+0000","last_fullsized":"2024-04-16T17:02:26.302850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750063+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:02:26.750063+0000","last_peered":"2024-04-16T17:02:26.750063+0000","last_clean":"2024-04-16T17:02:26.750063+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:02:26.750063+0000","last_undegraded":"2024-04-16T17:02:26.750063+0000","last_fullsized":"2024-04-16T17:02:26.750063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307157+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:02:26.307157+0000","last_peered":"2024-04-16T17:02:26.307157+0000","last_clean":"2024-04-16T17:02:26.307157+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:02:26.307157+0000","last_undegraded":"2024-04-16T17:02:26.307157+0000","last_fullsized":"2024-04-16T17:02:26.307157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.309599+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:02:26.309599+0000","last_peered":"2024-04-16T17:02:26.309599+0000","last_clean":"2024-04-16T17:02:26.309599+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:02:26.309599+0000","last_undegraded":"2024-04-16T17:02:26.309599+0000","last_fullsized":"2024-04-16T17:02:26.309599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307533+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:02:26.307533+0000","last_peered":"2024-04-16T17:02:26.307533+0000","last_clean":"2024-04-16T17:02:26.307533+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:02:26.307533+0000","last_undegraded":"2024-04-16T17:02:26.307533+0000","last_fullsized":"2024-04-16T17:02:26.307533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.305392+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:02:26.305392+0000","last_peered":"2024-04-16T17:02:26.305392+0000","last_clean":"2024-04-16T17:02:26.305392+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:02:26.305392+0000","last_undegraded":"2024-04-16T17:02:26.305392+0000","last_fullsized":"2024-04-16T17:02:26.305392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314904+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:02:26.314904+0000","last_peered":"2024-04-16T17:02:26.314904+0000","last_clean":"2024-04-16T17:02:26.314904+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:02:26.314904+0000","last_undegraded":"2024-04-16T17:02:26.314904+0000","last_fullsized":"2024-04-16T17:02:26.314904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310771+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:02:26.310771+0000","last_peered":"2024-04-16T17:02:26.310771+0000","last_clean":"2024-04-16T17:02:26.310771+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:02:26.310771+0000","last_undegraded":"2024-04-16T17:02:26.310771+0000","last_fullsized":"2024-04-16T17:02:26.310771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310212+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:02:26.310212+0000","last_peered":"2024-04-16T17:02:26.310212+0000","last_clean":"2024-04-16T17:02:26.310212+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:02:26.310212+0000","last_undegraded":"2024-04-16T17:02:26.310212+0000","last_fullsized":"2024-04-16T17:02:26.310212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308041+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:02:26.308041+0000","last_peered":"2024-04-16T17:02:26.308041+0000","last_clean":"2024-04-16T17:02:26.308041+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:02:26.308041+0000","last_undegraded":"2024-04-16T17:02:26.308041+0000","last_fullsized":"2024-04-16T17:02:26.308041+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.319248+0000","last_change":"2024-04-16T17:02:26.318546+0000","last_active":"2024-04-16T17:02:26.319248+0000","last_peered":"2024-04-16T17:02:26.319248+0000","last_clean":"2024-04-16T17:02:26.319248+0000","last_became_active":"2024-04-16T17:02:26.317794+0000","last_became_peered":"2024-04-16T17:02:26.317794+0000","last_unstale":"2024-04-16T17:02:26.319248+0000","last_undegraded":"2024-04-16T17:02:26.319248+0000","last_fullsized":"2024-04-16T17:02:26.319248+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:48.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302417+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:02:26.302417+0000","last_peered":"2024-04-16T17:02:26.302417+0000","last_clean":"2024-04-16T17:02:26.302417+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:02:26.302417+0000","last_undegraded":"2024-04-16T17:02:26.302417+0000","last_fullsized":"2024-04-16T17:02:26.302417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315817+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:02:26.315817+0000","last_peered":"2024-04-16T17:02:26.315817+0000","last_clean":"2024-04-16T17:02:26.315817+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:02:26.315817+0000","last_undegraded":"2024-04-16T17:02:26.315817+0000","last_fullsized":"2024-04-16T17:02:26.315817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310760+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:02:26.310760+0000","last_peered":"2024-04-16T17:02:26.310760+0000","last_clean":"2024-04-16T17:02:26.310760+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:02:26.310760+0000","last_undegraded":"2024-04-16T17:02:26.310760+0000","last_fullsized":"2024-04-16T17:02:26.310760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302548+0000","last_change":"2024-04-16T17:01:29.035651+0000","last_active":"2024-04-16T17:02:26.302548+0000","last_peered":"2024-04-16T17:02:26.302548+0000","last_clean":"2024-04-16T17:02:26.302548+0000","last_became_active":"2024-04-16T17:01:29.034865+0000","last_became_peered":"2024-04-16T17:01:29.034865+0000","last_unstale":"2024-04-16T17:02:26.302548+0000","last_undegraded":"2024-04-16T17:02:26.302548+0000","last_fullsized":"2024-04-16T17:02:26.302548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:49:31.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.303327+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:02:26.303327+0000","last_peered":"2024-04-16T17:02:26.303327+0000","last_clean":"2024-04-16T17:02:26.303327+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:02:26.303327+0000","last_undegraded":"2024-04-16T17:02:26.303327+0000","last_fullsized":"2024-04-16T17:02:26.303327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308314+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:02:26.308314+0000","last_peered":"2024-04-16T17:02:26.308314+0000","last_clean":"2024-04-16T17:02:26.308314+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:02:26.308314+0000","last_undegraded":"2024-04-16T17:02:26.308314+0000","last_fullsized":"2024-04-16T17:02:26.308314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302429+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:02:26.302429+0000","last_peered":"2024-04-16T17:02:26.302429+0000","last_clean":"2024-04-16T17:02:26.302429+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:02:26.302429+0000","last_undegraded":"2024-04-16T17:02:26.302429+0000","last_fullsized":"2024-04-16T17:02:26.302429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.303137+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:02:26.303137+0000","last_peered":"2024-04-16T17:02:26.303137+0000","last_clean":"2024-04-16T17:02:26.303137+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:02:26.303137+0000","last_undegraded":"2024-04-16T17:02:26.303137+0000","last_fullsized":"2024-04-16T17:02:26.303137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307359+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:02:26.307359+0000","last_peered":"2024-04-16T17:02:26.307359+0000","last_clean":"2024-04-16T17:02:26.307359+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:02:26.307359+0000","last_undegraded":"2024-04-16T17:02:26.307359+0000","last_fullsized":"2024-04-16T17:02:26.307359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310669+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:02:26.310669+0000","last_peered":"2024-04-16T17:02:26.310669+0000","last_clean":"2024-04-16T17:02:26.310669+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:02:26.310669+0000","last_undegraded":"2024-04-16T17:02:26.310669+0000","last_fullsized":"2024-04-16T17:02:26.310669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:21.388843+0000","last_change":"2024-04-16T17:02:26.315434+0000","last_active":"2024-04-16T17:03:21.388843+0000","last_peered":"2024-04-16T17:03:21.388843+0000","last_clean":"2024-04-16T17:03:21.388843+0000","last_became_active":"2024-04-16T17:02:26.310169+0000","last_became_peered":"2024-04-16T17:02:26.310169+0000","last_unstale":"2024-04-16T17:03:21.388843+0000","last_undegraded":"2024-04-16T17:03:21.388843+0000","last_fullsized":"2024-04-16T17:03:21.388843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:51.033415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302671+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:02:26.302671+0000","last_peered":"2024-04-16T17:02:26.302671+0000","last_clean":"2024-04-16T17:02:26.302671+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:02:26.302671+0000","last_undegraded":"2024-04-16T17:02:26.302671+0000","last_fullsized":"2024-04-16T17:02:26.302671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310945+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:02:26.310945+0000","last_peered":"2024-04-16T17:02:26.310945+0000","last_clean":"2024-04-16T17:02:26.310945+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:02:26.310945+0000","last_undegraded":"2024-04-16T17:02:26.310945+0000","last_fullsized":"2024-04-16T17:02:26.310945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304606+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:02:26.304606+0000","last_peered":"2024-04-16T17:02:26.304606+0000","last_clean":"2024-04-16T17:02:26.304606+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:02:26.304606+0000","last_undegraded":"2024-04-16T17:02:26.304606+0000","last_fullsized":"2024-04-16T17:02:26.304606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.326169+0000","last_change":"2024-04-16T17:02:26.321763+0000","last_active":"2024-04-16T17:02:26.326169+0000","last_peered":"2024-04-16T17:02:26.326169+0000","last_clean":"2024-04-16T17:02:26.326169+0000","last_became_active":"2024-04-16T17:02:26.321438+0000","last_became_peered":"2024-04-16T17:02:26.321438+0000","last_unstale":"2024-04-16T17:02:26.326169+0000","last_undegraded":"2024-04-16T17:02:26.326169+0000","last_fullsized":"2024-04-16T17:02:26.326169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:47.231035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.305279+0000","last_change":"2024-04-16T17:01:29.032665+0000","last_active":"2024-04-16T17:02:26.305279+0000","last_peered":"2024-04-16T17:02:26.305279+0000","last_clean":"2024-04-16T17:02:26.305279+0000","last_became_active":"2024-04-16T17:01:29.032293+0000","last_became_peered":"2024-04-16T17:01:29.032293+0000","last_unstale":"2024-04-16T17:02:26.305279+0000","last_undegraded":"2024-04-16T17:02:26.305279+0000","last_fullsized":"2024-04-16T17:02:26.305279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:46.193049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.749687+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:02:26.749687+0000","last_peered":"2024-04-16T17:02:26.749687+0000","last_clean":"2024-04-16T17:02:26.749687+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:02:26.749687+0000","last_undegraded":"2024-04-16T17:02:26.749687+0000","last_fullsized":"2024-04-16T17:02:26.749687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.316346+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:02:26.316346+0000","last_peered":"2024-04-16T17:02:26.316346+0000","last_clean":"2024-04-16T17:02:26.316346+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:02:26.316346+0000","last_undegraded":"2024-04-16T17:02:26.316346+0000","last_fullsized":"2024-04-16T17:02:26.316346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.313879+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:02:26.313879+0000","last_peered":"2024-04-16T17:02:26.313879+0000","last_clean":"2024-04-16T17:02:26.313879+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:02:26.313879+0000","last_undegraded":"2024-04-16T17:02:26.313879+0000","last_fullsized":"2024-04-16T17:02:26.313879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307178+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:02:26.307178+0000","last_peered":"2024-04-16T17:02:26.307178+0000","last_clean":"2024-04-16T17:02:26.307178+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:02:26.307178+0000","last_undegraded":"2024-04-16T17:02:26.307178+0000","last_fullsized":"2024-04-16T17:02:26.307178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308952+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:02:26.308952+0000","last_peered":"2024-04-16T17:02:26.308952+0000","last_clean":"2024-04-16T17:02:26.308952+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:02:26.308952+0000","last_undegraded":"2024-04-16T17:02:26.308952+0000","last_fullsized":"2024-04-16T17:02:26.308952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314314+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:02:26.314314+0000","last_peered":"2024-04-16T17:02:26.314314+0000","last_clean":"2024-04-16T17:02:26.314314+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:02:26.314314+0000","last_undegraded":"2024-04-16T17:02:26.314314+0000","last_fullsized":"2024-04-16T17:02:26.314314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:21.388226+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:03:21.388226+0000","last_peered":"2024-04-16T17:03:21.388226+0000","last_clean":"2024-04-16T17:03:21.388226+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:03:21.388226+0000","last_undegraded":"2024-04-16T17:03:21.388226+0000","last_fullsized":"2024-04-16T17:03:21.388226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307922+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:02:26.307922+0000","last_peered":"2024-04-16T17:02:26.307922+0000","last_clean":"2024-04-16T17:02:26.307922+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:02:26.307922+0000","last_undegraded":"2024-04-16T17:02:26.307922+0000","last_fullsized":"2024-04-16T17:02:26.307922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306897+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:02:26.306897+0000","last_peered":"2024-04-16T17:02:26.306897+0000","last_clean":"2024-04-16T17:02:26.306897+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:02:26.306897+0000","last_undegraded":"2024-04-16T17:02:26.306897+0000","last_fullsized":"2024-04-16T17:02:26.306897+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306687+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:02:26.306687+0000","last_peered":"2024-04-16T17:02:26.306687+0000","last_clean":"2024-04-16T17:02:26.306687+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:02:26.306687+0000","last_undegraded":"2024-04-16T17:02:26.306687+0000","last_fullsized":"2024-04-16T17:02:26.306687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.752275+0000","last_change":"2024-04-16T17:02:26.752275+0000","last_active":"2024-04-16T17:02:26.752275+0000","last_peered":"2024-04-16T17:02:26.752275+0000","last_clean":"2024-04-16T17:02:26.752275+0000","last_became_active":"2024-04-16T17:02:26.751729+0000","last_became_peered":"2024-04-16T17:02:26.751729+0000","last_unstale":"2024-04-16T17:02:26.752275+0000","last_undegraded":"2024-04-16T17:02:26.752275+0000","last_fullsized":"2024-04-16T17:02:26.752275+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:41:35.777691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302647+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:02:26.302647+0000","last_peered":"2024-04-16T17:02:26.302647+0000","last_clean":"2024-04-16T17:02:26.302647+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:02:26.302647+0000","last_undegraded":"2024-04-16T17:02:26.302647+0000","last_fullsized":"2024-04-16T17:02:26.302647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307316+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:02:26.307316+0000","last_peered":"2024-04-16T17:02:26.307316+0000","last_clean":"2024-04-16T17:02:26.307316+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:02:26.307316+0000","last_undegraded":"2024-04-16T17:02:26.307316+0000","last_fullsized":"2024-04-16T17:02:26.307316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308414+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:02:26.308414+0000","last_peered":"2024-04-16T17:02:26.308414+0000","last_clean":"2024-04-16T17:02:26.308414+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:02:26.308414+0000","last_undegraded":"2024-04-16T17:02:26.308414+0000","last_fullsized":"2024-04-16T17:02:26.308414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306799+0000","last_change":"2024-04-16T17:01:29.025629+0000","last_active":"2024-04-16T17:02:26.306799+0000","last_peered":"2024-04-16T17:02:26.306799+0000","last_clean":"2024-04-16T17:02:26.306799+0000","last_became_active":"2024-04-16T17:01:29.022689+0000","last_became_peered":"2024-04-16T17:01:29.022689+0000","last_unstale":"2024-04-16T17:02:26.306799+0000","last_undegraded":"2024-04-16T17:02:26.306799+0000","last_fullsized":"2024-04-16T17:02:26.306799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:40.917768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'33","reported_seq":237,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:22.389518+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:03:22.389518+0000","last_peered":"2024-04-16T17:03:22.389518+0000","last_clean":"2024-04-16T17:03:22.389518+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:03:22.389518+0000","last_undegraded":"2024-04-16T17:03:22.389518+0000","last_fullsized":"2024-04-16T17:03:22.389518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64206,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":177,"num_read_kb":152,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.303077+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:02:26.303077+0000","last_peered":"2024-04-16T17:02:26.303077+0000","last_clean":"2024-04-16T17:02:26.303077+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:02:26.303077+0000","last_undegraded":"2024-04-16T17:02:26.303077+0000","last_fullsized":"2024-04-16T17:02:26.303077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307684+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:02:26.307684+0000","last_peered":"2024-04-16T17:02:26.307684+0000","last_clean":"2024-04-16T17:02:26.307684+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:02:26.307684+0000","last_undegraded":"2024-04-16T17:02:26.307684+0000","last_fullsized":"2024-04-16T17:02:26.307684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310384+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:02:26.310384+0000","last_peered":"2024-04-16T17:02:26.310384+0000","last_clean":"2024-04-16T17:02:26.310384+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:02:26.310384+0000","last_undegraded":"2024-04-16T17:02:26.310384+0000","last_fullsized":"2024-04-16T17:02:26.310384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.749916+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:02:26.749916+0000","last_peered":"2024-04-16T17:02:26.749916+0000","last_clean":"2024-04-16T17:02:26.749916+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:02:26.749916+0000","last_undegraded":"2024-04-16T17:02:26.749916+0000","last_fullsized":"2024-04-16T17:02:26.749916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304833+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:02:26.304833+0000","last_peered":"2024-04-16T17:02:26.304833+0000","last_clean":"2024-04-16T17:02:26.304833+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:02:26.304833+0000","last_undegraded":"2024-04-16T17:02:26.304833+0000","last_fullsized":"2024-04-16T17:02:26.304833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307125+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:02:26.307125+0000","last_peered":"2024-04-16T17:02:26.307125+0000","last_clean":"2024-04-16T17:02:26.307125+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:02:26.307125+0000","last_undegraded":"2024-04-16T17:02:26.307125+0000","last_fullsized":"2024-04-16T17:02:26.307125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306806+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:02:26.306806+0000","last_peered":"2024-04-16T17:02:26.306806+0000","last_clean":"2024-04-16T17:02:26.306806+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:02:26.306806+0000","last_undegraded":"2024-04-16T17:02:26.306806+0000","last_fullsized":"2024-04-16T17:02:26.306806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307217+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:02:26.307217+0000","last_peered":"2024-04-16T17:02:26.307217+0000","last_clean":"2024-04-16T17:02:26.307217+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:02:26.307217+0000","last_undegraded":"2024-04-16T17:02:26.307217+0000","last_fullsized":"2024-04-16T17:02:26.307217+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":116,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:21.358272+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:03:21.358272+0000","last_peered":"2024-04-16T17:03:21.358272+0000","last_clean":"2024-04-16T17:03:21.358272+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:03:21.358272+0000","last_undegraded":"2024-04-16T17:03:21.358272+0000","last_fullsized":"2024-04-16T17:03:21.358272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306848+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:02:26.306848+0000","last_peered":"2024-04-16T17:02:26.306848+0000","last_clean":"2024-04-16T17:02:26.306848+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:02:26.306848+0000","last_undegraded":"2024-04-16T17:02:26.306848+0000","last_fullsized":"2024-04-16T17:02:26.306848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750497+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:02:26.750497+0000","last_peered":"2024-04-16T17:02:26.750497+0000","last_clean":"2024-04-16T17:02:26.750497+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:02:26.750497+0000","last_undegraded":"2024-04-16T17:02:26.750497+0000","last_fullsized":"2024-04-16T17:02:26.750497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302635+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:02:26.302635+0000","last_peered":"2024-04-16T17:02:26.302635+0000","last_clean":"2024-04-16T17:02:26.302635+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:02:26.302635+0000","last_undegraded":"2024-04-16T17:02:26.302635+0000","last_fullsized":"2024-04-16T17:02:26.302635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306827+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:02:26.306827+0000","last_peered":"2024-04-16T17:02:26.306827+0000","last_clean":"2024-04-16T17:02:26.306827+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:02:26.306827+0000","last_undegraded":"2024-04-16T17:02:26.306827+0000","last_fullsized":"2024-04-16T17:02:26.306827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306772+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:02:26.306772+0000","last_peered":"2024-04-16T17:02:26.306772+0000","last_clean":"2024-04-16T17:02:26.306772+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:02:26.306772+0000","last_undegraded":"2024-04-16T17:02:26.306772+0000","last_fullsized":"2024-04-16T17:02:26.306772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.320827+0000","last_change":"2024-04-16T17:02:26.320405+0000","last_active":"2024-04-16T17:02:26.320827+0000","last_peered":"2024-04-16T17:02:26.320827+0000","last_clean":"2024-04-16T17:02:26.320827+0000","last_became_active":"2024-04-16T17:02:26.307921+0000","last_became_peered":"2024-04-16T17:02:26.307921+0000","last_unstale":"2024-04-16T17:02:26.320827+0000","last_undegraded":"2024-04-16T17:02:26.320827+0000","last_fullsized":"2024-04-16T17:02:26.320827+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:19.156260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304843+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:02:26.304843+0000","last_peered":"2024-04-16T17:02:26.304843+0000","last_clean":"2024-04-16T17:02:26.304843+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:02:26.304843+0000","last_undegraded":"2024-04-16T17:02:26.304843+0000","last_fullsized":"2024-04-16T17:02:26.304843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302792+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:02:26.302792+0000","last_peered":"2024-04-16T17:02:26.302792+0000","last_clean":"2024-04-16T17:02:26.302792+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:02:26.302792+0000","last_undegraded":"2024-04-16T17:02:26.302792+0000","last_fullsized":"2024-04-16T17:02:26.302792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307972+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:02:26.307972+0000","last_peered":"2024-04-16T17:02:26.307972+0000","last_clean":"2024-04-16T17:02:26.307972+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:02:26.307972+0000","last_undegraded":"2024-04-16T17:02:26.307972+0000","last_fullsized":"2024-04-16T17:02:26.307972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304810+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:02:26.304810+0000","last_peered":"2024-04-16T17:02:26.304810+0000","last_clean":"2024-04-16T17:02:26.304810+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:02:26.304810+0000","last_undegraded":"2024-04-16T17:02:26.304810+0000","last_fullsized":"2024-04-16T17:02:26.304810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750101+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:02:26.750101+0000","last_peered":"2024-04-16T17:02:26.750101+0000","last_clean":"2024-04-16T17:02:26.750101+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:02:26.750101+0000","last_undegraded":"2024-04-16T17:02:26.750101+0000","last_fullsized":"2024-04-16T17:02:26.750101+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306451+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:02:26.306451+0000","last_peered":"2024-04-16T17:02:26.306451+0000","last_clean":"2024-04-16T17:02:26.306451+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:02:26.306451+0000","last_undegraded":"2024-04-16T17:02:26.306451+0000","last_fullsized":"2024-04-16T17:02:26.306451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304550+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:02:26.304550+0000","last_peered":"2024-04-16T17:02:26.304550+0000","last_clean":"2024-04-16T17:02:26.304550+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:02:26.304550+0000","last_undegraded":"2024-04-16T17:02:26.304550+0000","last_fullsized":"2024-04-16T17:02:26.304550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750345+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:02:26.750345+0000","last_peered":"2024-04-16T17:02:26.750345+0000","last_clean":"2024-04-16T17:02:26.750345+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:02:26.750345+0000","last_undegraded":"2024-04-16T17:02:26.750345+0000","last_fullsized":"2024-04-16T17:02:26.750345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307566+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:02:26.307566+0000","last_peered":"2024-04-16T17:02:26.307566+0000","last_clean":"2024-04-16T17:02:26.307566+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:02:26.307566+0000","last_undegraded":"2024-04-16T17:02:26.307566+0000","last_fullsized":"2024-04-16T17:02:26.307566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:19.352865+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:03:19.352865+0000","last_peered":"2024-04-16T17:03:19.352865+0000","last_clean":"2024-04-16T17:03:19.352865+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:03:19.352865+0000","last_undegraded":"2024-04-16T17:03:19.352865+0000","last_fullsized":"2024-04-16T17:03:19.352865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306573+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:02:26.306573+0000","last_peered":"2024-04-16T17:02:26.306573+0000","last_clean":"2024-04-16T17:02:26.306573+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:02:26.306573+0000","last_undegraded":"2024-04-16T17:02:26.306573+0000","last_fullsized":"2024-04-16T17:02:26.306573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.322313+0000","last_change":"2024-04-16T17:02:26.322313+0000","last_active":"2024-04-16T17:02:26.322313+0000","last_peered":"2024-04-16T17:02:26.322313+0000","last_clean":"2024-04-16T17:02:26.322313+0000","last_became_active":"2024-04-16T17:02:26.320328+0000","last_became_peered":"2024-04-16T17:02:26.320328+0000","last_unstale":"2024-04-16T17:02:26.322313+0000","last_undegraded":"2024-04-16T17:02:26.322313+0000","last_fullsized":"2024-04-16T17:02:26.322313+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:40.943183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302574+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:02:26.302574+0000","last_peered":"2024-04-16T17:02:26.302574+0000","last_clean":"2024-04-16T17:02:26.302574+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:02:26.302574+0000","last_undegraded":"2024-04-16T17:02:26.302574+0000","last_fullsized":"2024-04-16T17:02:26.302574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306928+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:02:26.306928+0000","last_peered":"2024-04-16T17:02:26.306928+0000","last_clean":"2024-04-16T17:02:26.306928+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:02:26.306928+0000","last_undegraded":"2024-04-16T17:02:26.306928+0000","last_fullsized":"2024-04-16T17:02:26.306928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307316+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:02:26.307316+0000","last_peered":"2024-04-16T17:02:26.307316+0000","last_clean":"2024-04-16T17:02:26.307316+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:02:26.307316+0000","last_undegraded":"2024-04-16T17:02:26.307316+0000","last_fullsized":"2024-04-16T17:02:26.307316+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.749520+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:02:26.749520+0000","last_peered":"2024-04-16T17:02:26.749520+0000","last_clean":"2024-04-16T17:02:26.749520+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:02:26.749520+0000","last_undegraded":"2024-04-16T17:02:26.749520+0000","last_fullsized":"2024-04-16T17:02:26.749520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.313304+0000","last_change":"2024-04-16T17:02:26.312852+0000","last_active":"2024-04-16T17:02:26.313304+0000","last_peered":"2024-04-16T17:02:26.313304+0000","last_clean":"2024-04-16T17:02:26.313304+0000","last_became_active":"2024-04-16T17:02:26.308279+0000","last_became_peered":"2024-04-16T17:02:26.308279+0000","last_unstale":"2024-04-16T17:02:26.313304+0000","last_undegraded":"2024-04-16T17:02:26.313304+0000","last_fullsized":"2024-04-16T17:02:26.313304+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:25.511349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304722+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:02:26.304722+0000","last_peered":"2024-04-16T17:02:26.304722+0000","last_clean":"2024-04-16T17:02:26.304722+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:02:26.304722+0000","last_undegraded":"2024-04-16T17:02:26.304722+0000","last_fullsized":"2024-04-16T17:02:26.304722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.303164+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:02:26.303164+0000","last_peered":"2024-04-16T17:02:26.303164+0000","last_clean":"2024-04-16T17:02:26.303164+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:02:26.303164+0000","last_undegraded":"2024-04-16T17:02:26.303164+0000","last_fullsized":"2024-04-16T17:02:26.303164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750470+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:02:26.750470+0000","last_peered":"2024-04-16T17:02:26.750470+0000","last_clean":"2024-04-16T17:02:26.750470+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:02:26.750470+0000","last_undegraded":"2024-04-16T17:02:26.750470+0000","last_fullsized":"2024-04-16T17:02:26.750470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.316232+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:02:26.316232+0000","last_peered":"2024-04-16T17:02:26.316232+0000","last_clean":"2024-04-16T17:02:26.316232+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:02:26.316232+0000","last_undegraded":"2024-04-16T17:02:26.316232+0000","last_fullsized":"2024-04-16T17:02:26.316232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315049+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:02:26.315049+0000","last_peered":"2024-04-16T17:02:26.315049+0000","last_clean":"2024-04-16T17:02:26.315049+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:02:26.315049+0000","last_undegraded":"2024-04-16T17:02:26.315049+0000","last_fullsized":"2024-04-16T17:02:26.315049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315403+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:02:26.315403+0000","last_peered":"2024-04-16T17:02:26.315403+0000","last_clean":"2024-04-16T17:02:26.315403+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:02:26.315403+0000","last_undegraded":"2024-04-16T17:02:26.315403+0000","last_fullsized":"2024-04-16T17:02:26.315403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310015+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:02:26.310015+0000","last_peered":"2024-04-16T17:02:26.310015+0000","last_clean":"2024-04-16T17:02:26.310015+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:02:26.310015+0000","last_undegraded":"2024-04-16T17:02:26.310015+0000","last_fullsized":"2024-04-16T17:02:26.310015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.303086+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:02:26.303086+0000","last_peered":"2024-04-16T17:02:26.303086+0000","last_clean":"2024-04-16T17:02:26.303086+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:02:26.303086+0000","last_undegraded":"2024-04-16T17:02:26.303086+0000","last_fullsized":"2024-04-16T17:02:26.303086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314197+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:02:26.314197+0000","last_peered":"2024-04-16T17:02:26.314197+0000","last_clean":"2024-04-16T17:02:26.314197+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:02:26.314197+0000","last_undegraded":"2024-04-16T17:02:26.314197+0000","last_fullsized":"2024-04-16T17:02:26.314197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302933+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:02:26.302933+0000","last_peered":"2024-04-16T17:02:26.302933+0000","last_clean":"2024-04-16T17:02:26.302933+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:02:26.302933+0000","last_undegraded":"2024-04-16T17:02:26.302933+0000","last_fullsized":"2024-04-16T17:02:26.302933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307585+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:02:26.307585+0000","last_peered":"2024-04-16T17:02:26.307585+0000","last_clean":"2024-04-16T17:02:26.307585+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:02:26.307585+0000","last_undegraded":"2024-04-16T17:02:26.307585+0000","last_fullsized":"2024-04-16T17:02:26.307585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302653+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:02:26.302653+0000","last_peered":"2024-04-16T17:02:26.302653+0000","last_clean":"2024-04-16T17:02:26.302653+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:02:26.302653+0000","last_undegraded":"2024-04-16T17:02:26.302653+0000","last_fullsized":"2024-04-16T17:02:26.302653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307747+0000","last_change":"2024-04-16T17:01:29.035782+0000","last_active":"2024-04-16T17:02:26.307747+0000","last_peered":"2024-04-16T17:02:26.307747+0000","last_clean":"2024-04-16T17:02:26.307747+0000","last_became_active":"2024-04-16T17:01:29.034812+0000","last_became_peered":"2024-04-16T17:01:29.034812+0000","last_unstale":"2024-04-16T17:02:26.307747+0000","last_undegraded":"2024-04-16T17:02:26.307747+0000","last_fullsized":"2024-04-16T17:02:26.307747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:27.655802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.313820+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:02:26.313820+0000","last_peered":"2024-04-16T17:02:26.313820+0000","last_clean":"2024-04-16T17:02:26.313820+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:02:26.313820+0000","last_undegraded":"2024-04-16T17:02:26.313820+0000","last_fullsized":"2024-04-16T17:02:26.313820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.305309+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:02:26.305309+0000","last_peered":"2024-04-16T17:02:26.305309+0000","last_clean":"2024-04-16T17:02:26.305309+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:02:26.305309+0000","last_undegraded":"2024-04-16T17:02:26.305309+0000","last_fullsized":"2024-04-16T17:02:26.305309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302709+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:02:26.302709+0000","last_peered":"2024-04-16T17:02:26.302709+0000","last_clean":"2024-04-16T17:02:26.302709+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:02:26.302709+0000","last_undegraded":"2024-04-16T17:02:26.302709+0000","last_fullsized":"2024-04-16T17:02:26.302709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310257+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:02:26.310257+0000","last_peered":"2024-04-16T17:02:26.310257+0000","last_clean":"2024-04-16T17:02:26.310257+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:02:26.310257+0000","last_undegraded":"2024-04-16T17:02:26.310257+0000","last_fullsized":"2024-04-16T17:02:26.310257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315784+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:02:26.315784+0000","last_peered":"2024-04-16T17:02:26.315784+0000","last_clean":"2024-04-16T17:02:26.315784+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:02:26.315784+0000","last_undegraded":"2024-04-16T17:02:26.315784+0000","last_fullsized":"2024-04-16T17:02:26.315784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302865+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:02:26.302865+0000","last_peered":"2024-04-16T17:02:26.302865+0000","last_clean":"2024-04-16T17:02:26.302865+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:02:26.302865+0000","last_undegraded":"2024-04-16T17:02:26.302865+0000","last_fullsized":"2024-04-16T17:02:26.302865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302896+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:02:26.302896+0000","last_peered":"2024-04-16T17:02:26.302896+0000","last_clean":"2024-04-16T17:02:26.302896+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:02:26.302896+0000","last_undegraded":"2024-04-16T17:02:26.302896+0000","last_fullsized":"2024-04-16T17:02:26.302896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306788+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:02:26.306788+0000","last_peered":"2024-04-16T17:02:26.306788+0000","last_clean":"2024-04-16T17:02:26.306788+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:02:26.306788+0000","last_undegraded":"2024-04-16T17:02:26.306788+0000","last_fullsized":"2024-04-16T17:02:26.306788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307765+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:02:26.307765+0000","last_peered":"2024-04-16T17:02:26.307765+0000","last_clean":"2024-04-16T17:02:26.307765+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:02:26.307765+0000","last_undegraded":"2024-04-16T17:02:26.307765+0000","last_fullsized":"2024-04-16T17:02:26.307765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.328363+0000","last_change":"2024-04-16T17:02:26.328059+0000","last_active":"2024-04-16T17:02:26.328363+0000","last_peered":"2024-04-16T17:02:26.328363+0000","last_clean":"2024-04-16T17:02:26.328363+0000","last_became_active":"2024-04-16T17:02:26.323315+0000","last_became_peered":"2024-04-16T17:02:26.323315+0000","last_unstale":"2024-04-16T17:02:26.328363+0000","last_undegraded":"2024-04-16T17:02:26.328363+0000","last_fullsized":"2024-04-16T17:02:26.328363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:01.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306926+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:02:26.306926+0000","last_peered":"2024-04-16T17:02:26.306926+0000","last_clean":"2024-04-16T17:02:26.306926+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:02:26.306926+0000","last_undegraded":"2024-04-16T17:02:26.306926+0000","last_fullsized":"2024-04-16T17:02:26.306926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.309414+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:02:26.309414+0000","last_peered":"2024-04-16T17:02:26.309414+0000","last_clean":"2024-04-16T17:02:26.309414+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:02:26.309414+0000","last_undegraded":"2024-04-16T17:02:26.309414+0000","last_fullsized":"2024-04-16T17:02:26.309414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307466+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:02:26.307466+0000","last_peered":"2024-04-16T17:02:26.307466+0000","last_clean":"2024-04-16T17:02:26.307466+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:02:26.307466+0000","last_undegraded":"2024-04-16T17:02:26.307466+0000","last_fullsized":"2024-04-16T17:02:26.307466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306686+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:02:26.306686+0000","last_peered":"2024-04-16T17:02:26.306686+0000","last_clean":"2024-04-16T17:02:26.306686+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:02:26.306686+0000","last_undegraded":"2024-04-16T17:02:26.306686+0000","last_fullsized":"2024-04-16T17:02:26.306686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306856+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:02:26.306856+0000","last_peered":"2024-04-16T17:02:26.306856+0000","last_clean":"2024-04-16T17:02:26.306856+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:02:26.306856+0000","last_undegraded":"2024-04-16T17:02:26.306856+0000","last_fullsized":"2024-04-16T17:02:26.306856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750355+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:02:26.750355+0000","last_peered":"2024-04-16T17:02:26.750355+0000","last_clean":"2024-04-16T17:02:26.750355+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:02:26.750355+0000","last_undegraded":"2024-04-16T17:02:26.750355+0000","last_fullsized":"2024-04-16T17:02:26.750355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306774+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:02:26.306774+0000","last_peered":"2024-04-16T17:02:26.306774+0000","last_clean":"2024-04-16T17:02:26.306774+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:02:26.306774+0000","last_undegraded":"2024-04-16T17:02:26.306774+0000","last_fullsized":"2024-04-16T17:02:26.306774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.305422+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:02:26.305422+0000","last_peered":"2024-04-16T17:02:26.305422+0000","last_clean":"2024-04-16T17:02:26.305422+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:02:26.305422+0000","last_undegraded":"2024-04-16T17:02:26.305422+0000","last_fullsized":"2024-04-16T17:02:26.305422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310440+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:02:26.310440+0000","last_peered":"2024-04-16T17:02:26.310440+0000","last_clean":"2024-04-16T17:02:26.310440+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:02:26.310440+0000","last_undegraded":"2024-04-16T17:02:26.310440+0000","last_fullsized":"2024-04-16T17:02:26.310440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.309661+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:02:26.309661+0000","last_peered":"2024-04-16T17:02:26.309661+0000","last_clean":"2024-04-16T17:02:26.309661+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:02:26.309661+0000","last_undegraded":"2024-04-16T17:02:26.309661+0000","last_fullsized":"2024-04-16T17:02:26.309661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310690+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:02:26.310690+0000","last_peered":"2024-04-16T17:02:26.310690+0000","last_clean":"2024-04-16T17:02:26.310690+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:02:26.310690+0000","last_undegraded":"2024-04-16T17:02:26.310690+0000","last_fullsized":"2024-04-16T17:02:26.310690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750406+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:02:26.750406+0000","last_peered":"2024-04-16T17:02:26.750406+0000","last_clean":"2024-04-16T17:02:26.750406+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:02:26.750406+0000","last_undegraded":"2024-04-16T17:02:26.750406+0000","last_fullsized":"2024-04-16T17:02:26.750406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.305315+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:02:26.305315+0000","last_peered":"2024-04-16T17:02:26.305315+0000","last_clean":"2024-04-16T17:02:26.305315+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:02:26.305315+0000","last_undegraded":"2024-04-16T17:02:26.305315+0000","last_fullsized":"2024-04-16T17:02:26.305315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.311058+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:02:26.311058+0000","last_peered":"2024-04-16T17:02:26.311058+0000","last_clean":"2024-04-16T17:02:26.311058+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:02:26.311058+0000","last_undegraded":"2024-04-16T17:02:26.311058+0000","last_fullsized":"2024-04-16T17:02:26.311058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315585+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:02:26.315585+0000","last_peered":"2024-04-16T17:02:26.315585+0000","last_clean":"2024-04-16T17:02:26.315585+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:02:26.315585+0000","last_undegraded":"2024-04-16T17:02:26.315585+0000","last_fullsized":"2024-04-16T17:02:26.315585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307452+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:02:26.307452+0000","last_peered":"2024-04-16T17:02:26.307452+0000","last_clean":"2024-04-16T17:02:26.307452+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:02:26.307452+0000","last_undegraded":"2024-04-16T17:02:26.307452+0000","last_fullsized":"2024-04-16T17:02:26.307452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315749+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:02:26.315749+0000","last_peered":"2024-04-16T17:02:26.315749+0000","last_clean":"2024-04-16T17:02:26.315749+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:02:26.315749+0000","last_undegraded":"2024-04-16T17:02:26.315749+0000","last_fullsized":"2024-04-16T17:02:26.315749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308340+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:02:26.308340+0000","last_peered":"2024-04-16T17:02:26.308340+0000","last_clean":"2024-04-16T17:02:26.308340+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:02:26.308340+0000","last_undegraded":"2024-04-16T17:02:26.308340+0000","last_fullsized":"2024-04-16T17:02:26.308340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.749927+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:02:26.749927+0000","last_peered":"2024-04-16T17:02:26.749927+0000","last_clean":"2024-04-16T17:02:26.749927+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:02:26.749927+0000","last_undegraded":"2024-04-16T17:02:26.749927+0000","last_fullsized":"2024-04-16T17:02:26.749927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304837+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:02:26.304837+0000","last_peered":"2024-04-16T17:02:26.304837+0000","last_clean":"2024-04-16T17:02:26.304837+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:02:26.304837+0000","last_undegraded":"2024-04-16T17:02:26.304837+0000","last_fullsized":"2024-04-16T17:02:26.304837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245319,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67816,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":521,"num_read_kb":334,"num_write":164,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":203448,"data_compressed":5478,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":500845,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":1152297,"data_compressed":58028,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":505155,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070519,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":500957,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1096497,"data_compressed":56202,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":496235,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":560933,"data_compressed":22513,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":497623,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":560497,"data_compressed":22513,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":496429,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":501507,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":1092085,"data_compressed":56202,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64788,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64352,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65784,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:03:24.553 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:03:25.086 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:25.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:25 smithi018 ceph-mon[31776]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:25.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:25 smithi022 ceph-mon[36023]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:25.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:25 smithi043 ceph-mon[36901]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:26 smithi022 ceph-mon[36023]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-16T17:03:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-16T17:03:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-04-16T17:03:26.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:26 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-16T17:03:26.431 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:26 smithi018 ceph-mon[31776]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:26.431 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-16T17:03:26.431 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-16T17:03:26.431 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-04-16T17:03:26.431 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:26 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-16T17:03:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:26 smithi043 ceph-mon[36901]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-16T17:03:26.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-16T17:03:26.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-04-16T17:03:26.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:26 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-16T17:03:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 -- 172.21.15.18:0/201018219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb930108b60 msgr2=0x7fb930108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 --2- 172.21.15.18:0/201018219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb930108b60 0x7fb930108f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9200098f0 tx=0x7fb920009c00 comp rx=0 tx=0).stop 2024-04-16T17:03:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 -- 172.21.15.18:0/201018219 shutdown_connections 2024-04-16T17:03:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 --2- 172.21.15.18:0/201018219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb93006ba30 0x7fb93010f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 --2- 172.21.15.18:0/201018219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb930108b60 0x7fb930108f40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 --2- 172.21.15.18:0/201018219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb93006b090 0x7fb93006b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 -- 172.21.15.18:0/201018219 >> 172.21.15.18:0/201018219 conn(0x7fb930078960 msgr2=0x7fb930078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:27.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 -- 172.21.15.18:0/201018219 shutdown_connections 2024-04-16T17:03:27.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.292+0000 7fb936000700 1 -- 172.21.15.18:0/201018219 wait complete. 2024-04-16T17:03:27.296 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:27 smithi018 ceph-mon[31776]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-16T17:03:27.296 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:27 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-16T17:03:27.296 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:27 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-16T17:03:27.296 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:27 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-04-16T17:03:27.296 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:27 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-16T17:03:27.296 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:27 smithi018 ceph-mon[31776]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.293+0000 7fb936000700 1 Processor -- start 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.293+0000 7fb936000700 1 -- start start 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb936000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb93006b090 0x7fb93019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb936000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb93006ba30 0x7fb93019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb936000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb930108b60 0x7fb93019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb936000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb930112df0 con 0x7fb930108b60 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb936000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb930112c70 con 0x7fb93006b090 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb936000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb930112f70 con 0x7fb93006ba30 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb927e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb93006ba30 0x7fb93019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb927e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb93006ba30 0x7fb93019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52882/0 (socket says 172.21.15.18:52882) 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb927e00700 1 -- 172.21.15.18:0/1338647057 learned_addr learned my addr 172.21.15.18:0/1338647057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:27.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.294+0000 7fb92f400700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb93006b090 0x7fb93019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.295+0000 7fb92fe00700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb930108b60 0x7fb93019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.295+0000 7fb927e00700 1 -- 172.21.15.18:0/1338647057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb93006b090 msgr2=0x7fb93019b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.295+0000 7fb927e00700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb93006b090 0x7fb93019b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.295+0000 7fb927e00700 1 -- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb930108b60 msgr2=0x7fb93019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.295+0000 7fb927e00700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb930108b60 0x7fb93019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.295+0000 7fb927e00700 1 -- 172.21.15.18:0/1338647057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9301a0590 con 0x7fb93006ba30 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.295+0000 7fb927e00700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb93006ba30 0x7fb93019bac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb91800e9c0 tx=0x7fb91800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.296+0000 7fb92cc00700 1 -- 172.21.15.18:0/1338647057 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91800cbd0 con 0x7fb93006ba30 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.296+0000 7fb92cc00700 1 -- 172.21.15.18:0/1338647057 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91800cd70 con 0x7fb93006ba30 2024-04-16T17:03:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.296+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9301a0880 con 0x7fb93006ba30 2024-04-16T17:03:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.296+0000 7fb92cc00700 1 -- 172.21.15.18:0/1338647057 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918005660 con 0x7fb93006ba30 2024-04-16T17:03:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.296+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9301a8100 con 0x7fb93006ba30 2024-04-16T17:03:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.297+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c005330 con 0x7fb93006ba30 2024-04-16T17:03:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.299+0000 7fb92cc00700 1 -- 172.21.15.18:0/1338647057 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb918004750 con 0x7fb93006ba30 2024-04-16T17:03:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.301+0000 7fb92cc00700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9140a3cc0 0x7fb9140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.301+0000 7fb92cc00700 1 -- 172.21.15.18:0/1338647057 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fb9180d07f0 con 0x7fb93006ba30 2024-04-16T17:03:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.301+0000 7fb92f400700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9140a3cc0 0x7fb9140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:27.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.302+0000 7fb92f400700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9140a3cc0 0x7fb9140a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb920009810 tx=0x7fb920019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:27.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.303+0000 7fb92cc00700 1 -- 172.21.15.18:0/1338647057 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb91809cda0 con 0x7fb93006ba30 2024-04-16T17:03:27.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:27 smithi022 ceph-mon[36023]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-16T17:03:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-16T17:03:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-16T17:03:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-04-16T17:03:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:27 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-16T17:03:27.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:27 smithi022 ceph-mon[36023]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T17:03:27.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:27 smithi043 ceph-mon[36901]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-16T17:03:27.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:27 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-16T17:03:27.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:27 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-16T17:03:27.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:27 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-04-16T17:03:27.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:27 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-16T17:03:27.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:27 smithi043 ceph-mon[36901]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T17:03:27.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.539+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb90c000bc0 con 0x7fb9140a3cc0 2024-04-16T17:03:27.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.557+0000 7fb92cc00700 1 -- 172.21.15.18:0/1338647057 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340697 (secure 0 0 0) 0x7fb92007a2f0 con 0x7fb9140a3cc0 2024-04-16T17:03:27.567 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9140a3cc0 msgr2=0x7fb9140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9140a3cc0 0x7fb9140a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb920009810 tx=0x7fb920019040 comp rx=0 tx=0).stop 2024-04-16T17:03:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb93006ba30 msgr2=0x7fb93019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb93006ba30 0x7fb93019bac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb91800e9c0 tx=0x7fb91800ecd0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 shutdown_connections 2024-04-16T17:03:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9140a3cc0 0x7fb9140a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb93006b090 0x7fb93019b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb930108b60 0x7fb93019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 --2- 172.21.15.18:0/1338647057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb93006ba30 0x7fb93019bac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 >> 172.21.15.18:0/1338647057 conn(0x7fb930078960 msgr2=0x7fb93010df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 shutdown_connections 2024-04-16T17:03:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:27.564+0000 7fb936000700 1 -- 172.21.15.18:0/1338647057 wait complete. 2024-04-16T17:03:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:03:28.307 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:28 smithi018 ceph-mon[31776]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T17:03:28.307 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:28 smithi018 ceph-mon[31776]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-16T17:03:28.307 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:28 smithi018 ceph-mon[31776]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T17:03:28.307 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:28 smithi018 ceph-mon[31776]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:28.376 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-16T17:03:27.046466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658418,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":662,"num_read_kb":458,"num_write":347,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":459,"ondisk_log_size":459,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378008,"kb_used_data":12164,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563816,"statfs":{"total":1151940427776,"available":1151553347584,"internally_reserved":0,"allocated":12455936,"data_stored":7970584,"data_compressed":363259,"data_compressed_allocated":495616,"data_compressed_original":2490368,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,0,11,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":20,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.115428"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.913599+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:03:24.913599+0000","last_peered":"2024-04-16T17:03:24.913599+0000","last_clean":"2024-04-16T17:03:24.913599+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:03:24.913599+0000","last_undegraded":"2024-04-16T17:03:24.913599+0000","last_fullsized":"2024-04-16T17:03:24.913599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314053+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:02:26.314053+0000","last_peered":"2024-04-16T17:02:26.314053+0000","last_clean":"2024-04-16T17:02:26.314053+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:02:26.314053+0000","last_undegraded":"2024-04-16T17:02:26.314053+0000","last_fullsized":"2024-04-16T17:02:26.314053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314410+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:02:26.314410+0000","last_peered":"2024-04-16T17:02:26.314410+0000","last_clean":"2024-04-16T17:02:26.314410+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:02:26.314410+0000","last_undegraded":"2024-04-16T17:02:26.314410+0000","last_fullsized":"2024-04-16T17:02:26.314410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.319837+0000","last_change":"2024-04-16T17:02:26.318719+0000","last_active":"2024-04-16T17:02:26.319837+0000","last_peered":"2024-04-16T17:02:26.319837+0000","last_clean":"2024-04-16T17:02:26.319837+0000","last_became_active":"2024-04-16T17:02:26.317743+0000","last_became_peered":"2024-04-16T17:02:26.317743+0000","last_unstale":"2024-04-16T17:02:26.319837+0000","last_undegraded":"2024-04-16T17:02:26.319837+0000","last_fullsized":"2024-04-16T17:02:26.319837+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:26.827859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046914+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:03:26.046914+0000","last_peered":"2024-04-16T17:03:26.046914+0000","last_clean":"2024-04-16T17:03:26.046914+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:03:26.046914+0000","last_undegraded":"2024-04-16T17:03:26.046914+0000","last_fullsized":"2024-04-16T17:03:26.046914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315910+0000","last_change":"2024-04-16T17:02:26.315910+0000","last_active":"2024-04-16T17:02:26.315910+0000","last_peered":"2024-04-16T17:02:26.315910+0000","last_clean":"2024-04-16T17:02:26.315910+0000","last_became_active":"2024-04-16T17:02:26.315493+0000","last_became_peered":"2024-04-16T17:02:26.315493+0000","last_unstale":"2024-04-16T17:02:26.315910+0000","last_undegraded":"2024-04-16T17:02:26.315910+0000","last_fullsized":"2024-04-16T17:02:26.315910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:27:48.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.894912+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:03:24.894912+0000","last_peered":"2024-04-16T17:03:24.894912+0000","last_clean":"2024-04-16T17:03:24.894912+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:03:24.894912+0000","last_undegraded":"2024-04-16T17:03:24.894912+0000","last_fullsized":"2024-04-16T17:03:24.894912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.316447+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:02:26.316447+0000","last_peered":"2024-04-16T17:02:26.316447+0000","last_clean":"2024-04-16T17:02:26.316447+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:02:26.316447+0000","last_undegraded":"2024-04-16T17:02:26.316447+0000","last_fullsized":"2024-04-16T17:02:26.316447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308040+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:02:26.308040+0000","last_peered":"2024-04-16T17:02:26.308040+0000","last_clean":"2024-04-16T17:02:26.308040+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:02:26.308040+0000","last_undegraded":"2024-04-16T17:02:26.308040+0000","last_fullsized":"2024-04-16T17:02:26.308040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304968+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:02:26.304968+0000","last_peered":"2024-04-16T17:02:26.304968+0000","last_clean":"2024-04-16T17:02:26.304968+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:02:26.304968+0000","last_undegraded":"2024-04-16T17:02:26.304968+0000","last_fullsized":"2024-04-16T17:02:26.304968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.881296+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:03:24.881296+0000","last_peered":"2024-04-16T17:03:24.881296+0000","last_clean":"2024-04-16T17:03:24.881296+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:03:24.881296+0000","last_undegraded":"2024-04-16T17:03:24.881296+0000","last_fullsized":"2024-04-16T17:03:24.881296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302850+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:02:26.302850+0000","last_peered":"2024-04-16T17:02:26.302850+0000","last_clean":"2024-04-16T17:02:26.302850+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:02:26.302850+0000","last_undegraded":"2024-04-16T17:02:26.302850+0000","last_fullsized":"2024-04-16T17:02:26.302850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.909528+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:03:24.909528+0000","last_peered":"2024-04-16T17:03:24.909528+0000","last_clean":"2024-04-16T17:03:24.909528+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:03:24.909528+0000","last_undegraded":"2024-04-16T17:03:24.909528+0000","last_fullsized":"2024-04-16T17:03:24.909528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307157+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:02:26.307157+0000","last_peered":"2024-04-16T17:02:26.307157+0000","last_clean":"2024-04-16T17:02:26.307157+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:02:26.307157+0000","last_undegraded":"2024-04-16T17:02:26.307157+0000","last_fullsized":"2024-04-16T17:02:26.307157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.309599+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:02:26.309599+0000","last_peered":"2024-04-16T17:02:26.309599+0000","last_clean":"2024-04-16T17:02:26.309599+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:02:26.309599+0000","last_undegraded":"2024-04-16T17:02:26.309599+0000","last_fullsized":"2024-04-16T17:02:26.309599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307533+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:02:26.307533+0000","last_peered":"2024-04-16T17:02:26.307533+0000","last_clean":"2024-04-16T17:02:26.307533+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:02:26.307533+0000","last_undegraded":"2024-04-16T17:02:26.307533+0000","last_fullsized":"2024-04-16T17:02:26.307533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.913790+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:03:24.913790+0000","last_peered":"2024-04-16T17:03:24.913790+0000","last_clean":"2024-04-16T17:03:24.913790+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:03:24.913790+0000","last_undegraded":"2024-04-16T17:03:24.913790+0000","last_fullsized":"2024-04-16T17:03:24.913790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314904+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:02:26.314904+0000","last_peered":"2024-04-16T17:02:26.314904+0000","last_clean":"2024-04-16T17:02:26.314904+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:02:26.314904+0000","last_undegraded":"2024-04-16T17:02:26.314904+0000","last_fullsized":"2024-04-16T17:02:26.314904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310771+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:02:26.310771+0000","last_peered":"2024-04-16T17:02:26.310771+0000","last_clean":"2024-04-16T17:02:26.310771+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:02:26.310771+0000","last_undegraded":"2024-04-16T17:02:26.310771+0000","last_fullsized":"2024-04-16T17:02:26.310771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310212+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:02:26.310212+0000","last_peered":"2024-04-16T17:02:26.310212+0000","last_clean":"2024-04-16T17:02:26.310212+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:02:26.310212+0000","last_undegraded":"2024-04-16T17:02:26.310212+0000","last_fullsized":"2024-04-16T17:02:26.310212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308041+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:02:26.308041+0000","last_peered":"2024-04-16T17:02:26.308041+0000","last_clean":"2024-04-16T17:02:26.308041+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:02:26.308041+0000","last_undegraded":"2024-04-16T17:02:26.308041+0000","last_fullsized":"2024-04-16T17:02:26.308041+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.319248+0000","last_change":"2024-04-16T17:02:26.318546+0000","last_active":"2024-04-16T17:02:26.319248+0000","last_peered":"2024-04-16T17:02:26.319248+0000","last_clean":"2024-04-16T17:02:26.319248+0000","last_became_active":"2024-04-16T17:02:26.317794+0000","last_became_peered":"2024-04-16T17:02:26.317794+0000","last_unstale":"2024-04-16T17:02:26.319248+0000","last_undegraded":"2024-04-16T17:02:26.319248+0000","last_fullsized":"2024-04-16T17:02:26.319248+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:48.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302417+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:02:26.302417+0000","last_peered":"2024-04-16T17:02:26.302417+0000","last_clean":"2024-04-16T17:02:26.302417+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:02:26.302417+0000","last_undegraded":"2024-04-16T17:02:26.302417+0000","last_fullsized":"2024-04-16T17:02:26.302417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315876+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:02:26.315876+0000","last_peered":"2024-04-16T17:02:26.315876+0000","last_clean":"2024-04-16T17:02:26.315876+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:02:26.315876+0000","last_undegraded":"2024-04-16T17:02:26.315876+0000","last_fullsized":"2024-04-16T17:02:26.315876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310760+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:02:26.310760+0000","last_peered":"2024-04-16T17:02:26.310760+0000","last_clean":"2024-04-16T17:02:26.310760+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:02:26.310760+0000","last_undegraded":"2024-04-16T17:02:26.310760+0000","last_fullsized":"2024-04-16T17:02:26.310760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302548+0000","last_change":"2024-04-16T17:01:29.035651+0000","last_active":"2024-04-16T17:02:26.302548+0000","last_peered":"2024-04-16T17:02:26.302548+0000","last_clean":"2024-04-16T17:02:26.302548+0000","last_became_active":"2024-04-16T17:01:29.034865+0000","last_became_peered":"2024-04-16T17:01:29.034865+0000","last_unstale":"2024-04-16T17:02:26.302548+0000","last_undegraded":"2024-04-16T17:02:26.302548+0000","last_fullsized":"2024-04-16T17:02:26.302548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:49:31.323484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047329+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:03:26.047329+0000","last_peered":"2024-04-16T17:03:26.047329+0000","last_clean":"2024-04-16T17:03:26.047329+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:03:26.047329+0000","last_undegraded":"2024-04-16T17:03:26.047329+0000","last_fullsized":"2024-04-16T17:03:26.047329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308314+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:02:26.308314+0000","last_peered":"2024-04-16T17:02:26.308314+0000","last_clean":"2024-04-16T17:02:26.308314+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:02:26.308314+0000","last_undegraded":"2024-04-16T17:02:26.308314+0000","last_fullsized":"2024-04-16T17:02:26.308314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302429+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:02:26.302429+0000","last_peered":"2024-04-16T17:02:26.302429+0000","last_clean":"2024-04-16T17:02:26.302429+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:02:26.302429+0000","last_undegraded":"2024-04-16T17:02:26.302429+0000","last_fullsized":"2024-04-16T17:02:26.302429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047181+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:03:26.047181+0000","last_peered":"2024-04-16T17:03:26.047181+0000","last_clean":"2024-04-16T17:03:26.047181+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:03:26.047181+0000","last_undegraded":"2024-04-16T17:03:26.047181+0000","last_fullsized":"2024-04-16T17:03:26.047181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307359+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:02:26.307359+0000","last_peered":"2024-04-16T17:02:26.307359+0000","last_clean":"2024-04-16T17:02:26.307359+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:02:26.307359+0000","last_undegraded":"2024-04-16T17:02:26.307359+0000","last_fullsized":"2024-04-16T17:02:26.307359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310669+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:02:26.310669+0000","last_peered":"2024-04-16T17:02:26.310669+0000","last_clean":"2024-04-16T17:02:26.310669+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:02:26.310669+0000","last_undegraded":"2024-04-16T17:02:26.310669+0000","last_fullsized":"2024-04-16T17:02:26.310669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:21.388843+0000","last_change":"2024-04-16T17:02:26.315434+0000","last_active":"2024-04-16T17:03:21.388843+0000","last_peered":"2024-04-16T17:03:21.388843+0000","last_clean":"2024-04-16T17:03:21.388843+0000","last_became_active":"2024-04-16T17:02:26.310169+0000","last_became_peered":"2024-04-16T17:02:26.310169+0000","last_unstale":"2024-04-16T17:03:21.388843+0000","last_undegraded":"2024-04-16T17:03:21.388843+0000","last_fullsized":"2024-04-16T17:03:21.388843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:51.033415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046779+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:03:26.046779+0000","last_peered":"2024-04-16T17:03:26.046779+0000","last_clean":"2024-04-16T17:03:26.046779+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:03:26.046779+0000","last_undegraded":"2024-04-16T17:03:26.046779+0000","last_fullsized":"2024-04-16T17:03:26.046779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310945+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:02:26.310945+0000","last_peered":"2024-04-16T17:02:26.310945+0000","last_clean":"2024-04-16T17:02:26.310945+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:02:26.310945+0000","last_undegraded":"2024-04-16T17:02:26.310945+0000","last_fullsized":"2024-04-16T17:02:26.310945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304606+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:02:26.304606+0000","last_peered":"2024-04-16T17:02:26.304606+0000","last_clean":"2024-04-16T17:02:26.304606+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:02:26.304606+0000","last_undegraded":"2024-04-16T17:02:26.304606+0000","last_fullsized":"2024-04-16T17:02:26.304606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.326169+0000","last_change":"2024-04-16T17:02:26.321763+0000","last_active":"2024-04-16T17:02:26.326169+0000","last_peered":"2024-04-16T17:02:26.326169+0000","last_clean":"2024-04-16T17:02:26.326169+0000","last_became_active":"2024-04-16T17:02:26.321438+0000","last_became_peered":"2024-04-16T17:02:26.321438+0000","last_unstale":"2024-04-16T17:02:26.326169+0000","last_undegraded":"2024-04-16T17:02:26.326169+0000","last_fullsized":"2024-04-16T17:02:26.326169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:47.231035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.305279+0000","last_change":"2024-04-16T17:01:29.032665+0000","last_active":"2024-04-16T17:02:26.305279+0000","last_peered":"2024-04-16T17:02:26.305279+0000","last_clean":"2024-04-16T17:02:26.305279+0000","last_became_active":"2024-04-16T17:01:29.032293+0000","last_became_peered":"2024-04-16T17:01:29.032293+0000","last_unstale":"2024-04-16T17:02:26.305279+0000","last_undegraded":"2024-04-16T17:02:26.305279+0000","last_fullsized":"2024-04-16T17:02:26.305279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:46.193049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.749687+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:02:26.749687+0000","last_peered":"2024-04-16T17:02:26.749687+0000","last_clean":"2024-04-16T17:02:26.749687+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:02:26.749687+0000","last_undegraded":"2024-04-16T17:02:26.749687+0000","last_fullsized":"2024-04-16T17:02:26.749687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.316346+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:02:26.316346+0000","last_peered":"2024-04-16T17:02:26.316346+0000","last_clean":"2024-04-16T17:02:26.316346+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:02:26.316346+0000","last_undegraded":"2024-04-16T17:02:26.316346+0000","last_fullsized":"2024-04-16T17:02:26.316346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.313879+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:02:26.313879+0000","last_peered":"2024-04-16T17:02:26.313879+0000","last_clean":"2024-04-16T17:02:26.313879+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:02:26.313879+0000","last_undegraded":"2024-04-16T17:02:26.313879+0000","last_fullsized":"2024-04-16T17:02:26.313879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307178+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:02:26.307178+0000","last_peered":"2024-04-16T17:02:26.307178+0000","last_clean":"2024-04-16T17:02:26.307178+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:02:26.307178+0000","last_undegraded":"2024-04-16T17:02:26.307178+0000","last_fullsized":"2024-04-16T17:02:26.307178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308952+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:02:26.308952+0000","last_peered":"2024-04-16T17:02:26.308952+0000","last_clean":"2024-04-16T17:02:26.308952+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:02:26.308952+0000","last_undegraded":"2024-04-16T17:02:26.308952+0000","last_fullsized":"2024-04-16T17:02:26.308952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314314+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:02:26.314314+0000","last_peered":"2024-04-16T17:02:26.314314+0000","last_clean":"2024-04-16T17:02:26.314314+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:02:26.314314+0000","last_undegraded":"2024-04-16T17:02:26.314314+0000","last_fullsized":"2024-04-16T17:02:26.314314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:21.388226+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:03:21.388226+0000","last_peered":"2024-04-16T17:03:21.388226+0000","last_clean":"2024-04-16T17:03:21.388226+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:03:21.388226+0000","last_undegraded":"2024-04-16T17:03:21.388226+0000","last_fullsized":"2024-04-16T17:03:21.388226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307922+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:02:26.307922+0000","last_peered":"2024-04-16T17:02:26.307922+0000","last_clean":"2024-04-16T17:02:26.307922+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:02:26.307922+0000","last_undegraded":"2024-04-16T17:02:26.307922+0000","last_fullsized":"2024-04-16T17:02:26.307922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306897+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:02:26.306897+0000","last_peered":"2024-04-16T17:02:26.306897+0000","last_clean":"2024-04-16T17:02:26.306897+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:02:26.306897+0000","last_undegraded":"2024-04-16T17:02:26.306897+0000","last_fullsized":"2024-04-16T17:02:26.306897+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306687+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:02:26.306687+0000","last_peered":"2024-04-16T17:02:26.306687+0000","last_clean":"2024-04-16T17:02:26.306687+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:02:26.306687+0000","last_undegraded":"2024-04-16T17:02:26.306687+0000","last_fullsized":"2024-04-16T17:02:26.306687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.752275+0000","last_change":"2024-04-16T17:02:26.752275+0000","last_active":"2024-04-16T17:02:26.752275+0000","last_peered":"2024-04-16T17:02:26.752275+0000","last_clean":"2024-04-16T17:02:26.752275+0000","last_became_active":"2024-04-16T17:02:26.751729+0000","last_became_peered":"2024-04-16T17:02:26.751729+0000","last_unstale":"2024-04-16T17:02:26.752275+0000","last_undegraded":"2024-04-16T17:02:26.752275+0000","last_fullsized":"2024-04-16T17:02:26.752275+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:41:35.777691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302647+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:02:26.302647+0000","last_peered":"2024-04-16T17:02:26.302647+0000","last_clean":"2024-04-16T17:02:26.302647+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:02:26.302647+0000","last_undegraded":"2024-04-16T17:02:26.302647+0000","last_fullsized":"2024-04-16T17:02:26.302647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307316+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:02:26.307316+0000","last_peered":"2024-04-16T17:02:26.307316+0000","last_clean":"2024-04-16T17:02:26.307316+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:02:26.307316+0000","last_undegraded":"2024-04-16T17:02:26.307316+0000","last_fullsized":"2024-04-16T17:02:26.307316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308414+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:02:26.308414+0000","last_peered":"2024-04-16T17:02:26.308414+0000","last_clean":"2024-04-16T17:02:26.308414+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:02:26.308414+0000","last_undegraded":"2024-04-16T17:02:26.308414+0000","last_fullsized":"2024-04-16T17:02:26.308414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306799+0000","last_change":"2024-04-16T17:01:29.025629+0000","last_active":"2024-04-16T17:02:26.306799+0000","last_peered":"2024-04-16T17:02:26.306799+0000","last_clean":"2024-04-16T17:02:26.306799+0000","last_became_active":"2024-04-16T17:01:29.022689+0000","last_became_peered":"2024-04-16T17:01:29.022689+0000","last_unstale":"2024-04-16T17:02:26.306799+0000","last_undegraded":"2024-04-16T17:02:26.306799+0000","last_fullsized":"2024-04-16T17:02:26.306799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:40.917768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'33","reported_seq":237,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:22.389518+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:03:22.389518+0000","last_peered":"2024-04-16T17:03:22.389518+0000","last_clean":"2024-04-16T17:03:22.389518+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:03:22.389518+0000","last_undegraded":"2024-04-16T17:03:22.389518+0000","last_fullsized":"2024-04-16T17:03:22.389518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64206,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":177,"num_read_kb":152,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047288+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:03:26.047288+0000","last_peered":"2024-04-16T17:03:26.047288+0000","last_clean":"2024-04-16T17:03:26.047288+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:03:26.047288+0000","last_undegraded":"2024-04-16T17:03:26.047288+0000","last_fullsized":"2024-04-16T17:03:26.047288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307684+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:02:26.307684+0000","last_peered":"2024-04-16T17:02:26.307684+0000","last_clean":"2024-04-16T17:02:26.307684+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:02:26.307684+0000","last_undegraded":"2024-04-16T17:02:26.307684+0000","last_fullsized":"2024-04-16T17:02:26.307684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310384+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:02:26.310384+0000","last_peered":"2024-04-16T17:02:26.310384+0000","last_clean":"2024-04-16T17:02:26.310384+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:02:26.310384+0000","last_undegraded":"2024-04-16T17:02:26.310384+0000","last_fullsized":"2024-04-16T17:02:26.310384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.911618+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:03:24.911618+0000","last_peered":"2024-04-16T17:03:24.911618+0000","last_clean":"2024-04-16T17:03:24.911618+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:03:24.911618+0000","last_undegraded":"2024-04-16T17:03:24.911618+0000","last_fullsized":"2024-04-16T17:03:24.911618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304833+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:02:26.304833+0000","last_peered":"2024-04-16T17:02:26.304833+0000","last_clean":"2024-04-16T17:02:26.304833+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:02:26.304833+0000","last_undegraded":"2024-04-16T17:02:26.304833+0000","last_fullsized":"2024-04-16T17:02:26.304833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307125+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:02:26.307125+0000","last_peered":"2024-04-16T17:02:26.307125+0000","last_clean":"2024-04-16T17:02:26.307125+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:02:26.307125+0000","last_undegraded":"2024-04-16T17:02:26.307125+0000","last_fullsized":"2024-04-16T17:02:26.307125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.909273+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:03:24.909273+0000","last_peered":"2024-04-16T17:03:24.909273+0000","last_clean":"2024-04-16T17:03:24.909273+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:03:24.909273+0000","last_undegraded":"2024-04-16T17:03:24.909273+0000","last_fullsized":"2024-04-16T17:03:24.909273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307217+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:02:26.307217+0000","last_peered":"2024-04-16T17:02:26.307217+0000","last_clean":"2024-04-16T17:02:26.307217+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:02:26.307217+0000","last_undegraded":"2024-04-16T17:02:26.307217+0000","last_fullsized":"2024-04-16T17:02:26.307217+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":116,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:21.358272+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:03:21.358272+0000","last_peered":"2024-04-16T17:03:21.358272+0000","last_clean":"2024-04-16T17:03:21.358272+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:03:21.358272+0000","last_undegraded":"2024-04-16T17:03:21.358272+0000","last_fullsized":"2024-04-16T17:03:21.358272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.889205+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:03:24.889205+0000","last_peered":"2024-04-16T17:03:24.889205+0000","last_clean":"2024-04-16T17:03:24.889205+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:03:24.889205+0000","last_undegraded":"2024-04-16T17:03:24.889205+0000","last_fullsized":"2024-04-16T17:03:24.889205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750497+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:02:26.750497+0000","last_peered":"2024-04-16T17:02:26.750497+0000","last_clean":"2024-04-16T17:02:26.750497+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:02:26.750497+0000","last_undegraded":"2024-04-16T17:02:26.750497+0000","last_fullsized":"2024-04-16T17:02:26.750497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046789+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:03:26.046789+0000","last_peered":"2024-04-16T17:03:26.046789+0000","last_clean":"2024-04-16T17:03:26.046789+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:03:26.046789+0000","last_undegraded":"2024-04-16T17:03:26.046789+0000","last_fullsized":"2024-04-16T17:03:26.046789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306827+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:02:26.306827+0000","last_peered":"2024-04-16T17:02:26.306827+0000","last_clean":"2024-04-16T17:02:26.306827+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:02:26.306827+0000","last_undegraded":"2024-04-16T17:02:26.306827+0000","last_fullsized":"2024-04-16T17:02:26.306827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306772+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:02:26.306772+0000","last_peered":"2024-04-16T17:02:26.306772+0000","last_clean":"2024-04-16T17:02:26.306772+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:02:26.306772+0000","last_undegraded":"2024-04-16T17:02:26.306772+0000","last_fullsized":"2024-04-16T17:02:26.306772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.320827+0000","last_change":"2024-04-16T17:02:26.320405+0000","last_active":"2024-04-16T17:02:26.320827+0000","last_peered":"2024-04-16T17:02:26.320827+0000","last_clean":"2024-04-16T17:02:26.320827+0000","last_became_active":"2024-04-16T17:02:26.307921+0000","last_became_peered":"2024-04-16T17:02:26.307921+0000","last_unstale":"2024-04-16T17:02:26.320827+0000","last_undegraded":"2024-04-16T17:02:26.320827+0000","last_fullsized":"2024-04-16T17:02:26.320827+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:19.156260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.891262+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:03:24.891262+0000","last_peered":"2024-04-16T17:03:24.891262+0000","last_clean":"2024-04-16T17:03:24.891262+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:03:24.891262+0000","last_undegraded":"2024-04-16T17:03:24.891262+0000","last_fullsized":"2024-04-16T17:03:24.891262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047049+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:03:26.047049+0000","last_peered":"2024-04-16T17:03:26.047049+0000","last_clean":"2024-04-16T17:03:26.047049+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:03:26.047049+0000","last_undegraded":"2024-04-16T17:03:26.047049+0000","last_fullsized":"2024-04-16T17:03:26.047049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307972+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:02:26.307972+0000","last_peered":"2024-04-16T17:02:26.307972+0000","last_clean":"2024-04-16T17:02:26.307972+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:02:26.307972+0000","last_undegraded":"2024-04-16T17:02:26.307972+0000","last_fullsized":"2024-04-16T17:02:26.307972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304810+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:02:26.304810+0000","last_peered":"2024-04-16T17:02:26.304810+0000","last_clean":"2024-04-16T17:02:26.304810+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:02:26.304810+0000","last_undegraded":"2024-04-16T17:02:26.304810+0000","last_fullsized":"2024-04-16T17:02:26.304810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750101+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:02:26.750101+0000","last_peered":"2024-04-16T17:02:26.750101+0000","last_clean":"2024-04-16T17:02:26.750101+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:02:26.750101+0000","last_undegraded":"2024-04-16T17:02:26.750101+0000","last_fullsized":"2024-04-16T17:02:26.750101+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306451+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:02:26.306451+0000","last_peered":"2024-04-16T17:02:26.306451+0000","last_clean":"2024-04-16T17:02:26.306451+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:02:26.306451+0000","last_undegraded":"2024-04-16T17:02:26.306451+0000","last_fullsized":"2024-04-16T17:02:26.306451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304550+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:02:26.304550+0000","last_peered":"2024-04-16T17:02:26.304550+0000","last_clean":"2024-04-16T17:02:26.304550+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:02:26.304550+0000","last_undegraded":"2024-04-16T17:02:26.304550+0000","last_fullsized":"2024-04-16T17:02:26.304550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750345+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:02:26.750345+0000","last_peered":"2024-04-16T17:02:26.750345+0000","last_clean":"2024-04-16T17:02:26.750345+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:02:26.750345+0000","last_undegraded":"2024-04-16T17:02:26.750345+0000","last_fullsized":"2024-04-16T17:02:26.750345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307566+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:02:26.307566+0000","last_peered":"2024-04-16T17:02:26.307566+0000","last_clean":"2024-04-16T17:02:26.307566+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:02:26.307566+0000","last_undegraded":"2024-04-16T17:02:26.307566+0000","last_fullsized":"2024-04-16T17:02:26.307566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.364287+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:03:24.364287+0000","last_peered":"2024-04-16T17:03:24.364287+0000","last_clean":"2024-04-16T17:03:24.364287+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:03:24.364287+0000","last_undegraded":"2024-04-16T17:03:24.364287+0000","last_fullsized":"2024-04-16T17:03:24.364287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306573+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:02:26.306573+0000","last_peered":"2024-04-16T17:02:26.306573+0000","last_clean":"2024-04-16T17:02:26.306573+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:02:26.306573+0000","last_undegraded":"2024-04-16T17:02:26.306573+0000","last_fullsized":"2024-04-16T17:02:26.306573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.322313+0000","last_change":"2024-04-16T17:02:26.322313+0000","last_active":"2024-04-16T17:02:26.322313+0000","last_peered":"2024-04-16T17:02:26.322313+0000","last_clean":"2024-04-16T17:02:26.322313+0000","last_became_active":"2024-04-16T17:02:26.320328+0000","last_became_peered":"2024-04-16T17:02:26.320328+0000","last_unstale":"2024-04-16T17:02:26.322313+0000","last_undegraded":"2024-04-16T17:02:26.322313+0000","last_fullsized":"2024-04-16T17:02:26.322313+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:40.943183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302574+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:02:26.302574+0000","last_peered":"2024-04-16T17:02:26.302574+0000","last_clean":"2024-04-16T17:02:26.302574+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:02:26.302574+0000","last_undegraded":"2024-04-16T17:02:26.302574+0000","last_fullsized":"2024-04-16T17:02:26.302574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306928+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:02:26.306928+0000","last_peered":"2024-04-16T17:02:26.306928+0000","last_clean":"2024-04-16T17:02:26.306928+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:02:26.306928+0000","last_undegraded":"2024-04-16T17:02:26.306928+0000","last_fullsized":"2024-04-16T17:02:26.306928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307316+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:02:26.307316+0000","last_peered":"2024-04-16T17:02:26.307316+0000","last_clean":"2024-04-16T17:02:26.307316+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:02:26.307316+0000","last_undegraded":"2024-04-16T17:02:26.307316+0000","last_fullsized":"2024-04-16T17:02:26.307316+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.895629+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:03:24.895629+0000","last_peered":"2024-04-16T17:03:24.895629+0000","last_clean":"2024-04-16T17:03:24.895629+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:03:24.895629+0000","last_undegraded":"2024-04-16T17:03:24.895629+0000","last_fullsized":"2024-04-16T17:03:24.895629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.313304+0000","last_change":"2024-04-16T17:02:26.312852+0000","last_active":"2024-04-16T17:02:26.313304+0000","last_peered":"2024-04-16T17:02:26.313304+0000","last_clean":"2024-04-16T17:02:26.313304+0000","last_became_active":"2024-04-16T17:02:26.308279+0000","last_became_peered":"2024-04-16T17:02:26.308279+0000","last_unstale":"2024-04-16T17:02:26.313304+0000","last_undegraded":"2024-04-16T17:02:26.313304+0000","last_fullsized":"2024-04-16T17:02:26.313304+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:25.511349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.304722+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:02:26.304722+0000","last_peered":"2024-04-16T17:02:26.304722+0000","last_clean":"2024-04-16T17:02:26.304722+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:02:26.304722+0000","last_undegraded":"2024-04-16T17:02:26.304722+0000","last_fullsized":"2024-04-16T17:02:26.304722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047138+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:03:26.047138+0000","last_peered":"2024-04-16T17:03:26.047138+0000","last_clean":"2024-04-16T17:03:26.047138+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:03:26.047138+0000","last_undegraded":"2024-04-16T17:03:26.047138+0000","last_fullsized":"2024-04-16T17:03:26.047138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.905706+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:03:24.905706+0000","last_peered":"2024-04-16T17:03:24.905706+0000","last_clean":"2024-04-16T17:03:24.905706+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:03:24.905706+0000","last_undegraded":"2024-04-16T17:03:24.905706+0000","last_fullsized":"2024-04-16T17:03:24.905706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.316232+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:02:26.316232+0000","last_peered":"2024-04-16T17:02:26.316232+0000","last_clean":"2024-04-16T17:02:26.316232+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:02:26.316232+0000","last_undegraded":"2024-04-16T17:02:26.316232+0000","last_fullsized":"2024-04-16T17:02:26.316232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315049+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:02:26.315049+0000","last_peered":"2024-04-16T17:02:26.315049+0000","last_clean":"2024-04-16T17:02:26.315049+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:02:26.315049+0000","last_undegraded":"2024-04-16T17:02:26.315049+0000","last_fullsized":"2024-04-16T17:02:26.315049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315403+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:02:26.315403+0000","last_peered":"2024-04-16T17:02:26.315403+0000","last_clean":"2024-04-16T17:02:26.315403+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:02:26.315403+0000","last_undegraded":"2024-04-16T17:02:26.315403+0000","last_fullsized":"2024-04-16T17:02:26.315403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310015+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:02:26.310015+0000","last_peered":"2024-04-16T17:02:26.310015+0000","last_clean":"2024-04-16T17:02:26.310015+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:02:26.310015+0000","last_undegraded":"2024-04-16T17:02:26.310015+0000","last_fullsized":"2024-04-16T17:02:26.310015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.303086+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:02:26.303086+0000","last_peered":"2024-04-16T17:02:26.303086+0000","last_clean":"2024-04-16T17:02:26.303086+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:02:26.303086+0000","last_undegraded":"2024-04-16T17:02:26.303086+0000","last_fullsized":"2024-04-16T17:02:26.303086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.314197+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:02:26.314197+0000","last_peered":"2024-04-16T17:02:26.314197+0000","last_clean":"2024-04-16T17:02:26.314197+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:02:26.314197+0000","last_undegraded":"2024-04-16T17:02:26.314197+0000","last_fullsized":"2024-04-16T17:02:26.314197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302933+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:02:26.302933+0000","last_peered":"2024-04-16T17:02:26.302933+0000","last_clean":"2024-04-16T17:02:26.302933+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:02:26.302933+0000","last_undegraded":"2024-04-16T17:02:26.302933+0000","last_fullsized":"2024-04-16T17:02:26.302933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307585+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:02:26.307585+0000","last_peered":"2024-04-16T17:02:26.307585+0000","last_clean":"2024-04-16T17:02:26.307585+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:02:26.307585+0000","last_undegraded":"2024-04-16T17:02:26.307585+0000","last_fullsized":"2024-04-16T17:02:26.307585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046828+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:03:26.046828+0000","last_peered":"2024-04-16T17:03:26.046828+0000","last_clean":"2024-04-16T17:03:26.046828+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:03:26.046828+0000","last_undegraded":"2024-04-16T17:03:26.046828+0000","last_fullsized":"2024-04-16T17:03:26.046828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2024-04-16T17:01:28.999759+0000","last_change":"2024-04-16T17:03:26.709595+0000","last_active":"2024-04-16T17:01:27.966977+0000","last_peered":"2024-04-16T17:01:27.966977+0000","last_clean":"2024-04-16T17:01:27.966977+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T17:01:28.999759+0000","last_undegraded":"2024-04-16T17:01:28.999759+0000","last_fullsized":"2024-04-16T17:01:28.999759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.313820+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:02:26.313820+0000","last_peered":"2024-04-16T17:02:26.313820+0000","last_clean":"2024-04-16T17:02:26.313820+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:02:26.313820+0000","last_undegraded":"2024-04-16T17:02:26.313820+0000","last_fullsized":"2024-04-16T17:02:26.313820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.912963+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:03:24.912963+0000","last_peered":"2024-04-16T17:03:24.912963+0000","last_clean":"2024-04-16T17:03:24.912963+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:03:24.912963+0000","last_undegraded":"2024-04-16T17:03:24.912963+0000","last_fullsized":"2024-04-16T17:03:24.912963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302709+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:02:26.302709+0000","last_peered":"2024-04-16T17:02:26.302709+0000","last_clean":"2024-04-16T17:02:26.302709+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:02:26.302709+0000","last_undegraded":"2024-04-16T17:02:26.302709+0000","last_fullsized":"2024-04-16T17:02:26.302709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310257+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:02:26.310257+0000","last_peered":"2024-04-16T17:02:26.310257+0000","last_clean":"2024-04-16T17:02:26.310257+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:02:26.310257+0000","last_undegraded":"2024-04-16T17:02:26.310257+0000","last_fullsized":"2024-04-16T17:02:26.310257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315784+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:02:26.315784+0000","last_peered":"2024-04-16T17:02:26.315784+0000","last_clean":"2024-04-16T17:02:26.315784+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:02:26.315784+0000","last_undegraded":"2024-04-16T17:02:26.315784+0000","last_fullsized":"2024-04-16T17:02:26.315784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.302865+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:02:26.302865+0000","last_peered":"2024-04-16T17:02:26.302865+0000","last_clean":"2024-04-16T17:02:26.302865+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:02:26.302865+0000","last_undegraded":"2024-04-16T17:02:26.302865+0000","last_fullsized":"2024-04-16T17:02:26.302865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046799+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:03:26.046799+0000","last_peered":"2024-04-16T17:03:26.046799+0000","last_clean":"2024-04-16T17:03:26.046799+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:03:26.046799+0000","last_undegraded":"2024-04-16T17:03:26.046799+0000","last_fullsized":"2024-04-16T17:03:26.046799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.894770+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:03:24.894770+0000","last_peered":"2024-04-16T17:03:24.894770+0000","last_clean":"2024-04-16T17:03:24.894770+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:03:24.894770+0000","last_undegraded":"2024-04-16T17:03:24.894770+0000","last_fullsized":"2024-04-16T17:03:24.894770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307765+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:02:26.307765+0000","last_peered":"2024-04-16T17:02:26.307765+0000","last_clean":"2024-04-16T17:02:26.307765+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:02:26.307765+0000","last_undegraded":"2024-04-16T17:02:26.307765+0000","last_fullsized":"2024-04-16T17:02:26.307765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.328363+0000","last_change":"2024-04-16T17:02:26.328059+0000","last_active":"2024-04-16T17:02:26.328363+0000","last_peered":"2024-04-16T17:02:26.328363+0000","last_clean":"2024-04-16T17:02:26.328363+0000","last_became_active":"2024-04-16T17:02:26.323315+0000","last_became_peered":"2024-04-16T17:02:26.323315+0000","last_unstale":"2024-04-16T17:02:26.328363+0000","last_undegraded":"2024-04-16T17:02:26.328363+0000","last_fullsized":"2024-04-16T17:02:26.328363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:01.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.893270+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:03:24.893270+0000","last_peered":"2024-04-16T17:03:24.893270+0000","last_clean":"2024-04-16T17:03:24.893270+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:03:24.893270+0000","last_undegraded":"2024-04-16T17:03:24.893270+0000","last_fullsized":"2024-04-16T17:03:24.893270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.309414+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:02:26.309414+0000","last_peered":"2024-04-16T17:02:26.309414+0000","last_clean":"2024-04-16T17:02:26.309414+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:02:26.309414+0000","last_undegraded":"2024-04-16T17:02:26.309414+0000","last_fullsized":"2024-04-16T17:02:26.309414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307466+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:02:26.307466+0000","last_peered":"2024-04-16T17:02:26.307466+0000","last_clean":"2024-04-16T17:02:26.307466+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:02:26.307466+0000","last_undegraded":"2024-04-16T17:02:26.307466+0000","last_fullsized":"2024-04-16T17:02:26.307466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.902565+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:03:24.902565+0000","last_peered":"2024-04-16T17:03:24.902565+0000","last_clean":"2024-04-16T17:03:24.902565+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:03:24.902565+0000","last_undegraded":"2024-04-16T17:03:24.902565+0000","last_fullsized":"2024-04-16T17:03:24.902565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306856+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:02:26.306856+0000","last_peered":"2024-04-16T17:02:26.306856+0000","last_clean":"2024-04-16T17:02:26.306856+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:02:26.306856+0000","last_undegraded":"2024-04-16T17:02:26.306856+0000","last_fullsized":"2024-04-16T17:02:26.306856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750355+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:02:26.750355+0000","last_peered":"2024-04-16T17:02:26.750355+0000","last_clean":"2024-04-16T17:02:26.750355+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:02:26.750355+0000","last_undegraded":"2024-04-16T17:02:26.750355+0000","last_fullsized":"2024-04-16T17:02:26.750355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.897406+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:03:24.897406+0000","last_peered":"2024-04-16T17:03:24.897406+0000","last_clean":"2024-04-16T17:03:24.897406+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:03:24.897406+0000","last_undegraded":"2024-04-16T17:03:24.897406+0000","last_fullsized":"2024-04-16T17:03:24.897406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.889560+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:03:24.889560+0000","last_peered":"2024-04-16T17:03:24.889560+0000","last_clean":"2024-04-16T17:03:24.889560+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:03:24.889560+0000","last_undegraded":"2024-04-16T17:03:24.889560+0000","last_fullsized":"2024-04-16T17:03:24.889560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310440+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:02:26.310440+0000","last_peered":"2024-04-16T17:02:26.310440+0000","last_clean":"2024-04-16T17:02:26.310440+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:02:26.310440+0000","last_undegraded":"2024-04-16T17:02:26.310440+0000","last_fullsized":"2024-04-16T17:02:26.310440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.309661+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:02:26.309661+0000","last_peered":"2024-04-16T17:02:26.309661+0000","last_clean":"2024-04-16T17:02:26.309661+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:02:26.309661+0000","last_undegraded":"2024-04-16T17:02:26.309661+0000","last_fullsized":"2024-04-16T17:02:26.309661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.310690+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:02:26.310690+0000","last_peered":"2024-04-16T17:02:26.310690+0000","last_clean":"2024-04-16T17:02:26.310690+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:02:26.310690+0000","last_undegraded":"2024-04-16T17:02:26.310690+0000","last_fullsized":"2024-04-16T17:02:26.310690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.913782+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:03:24.913782+0000","last_peered":"2024-04-16T17:03:24.913782+0000","last_clean":"2024-04-16T17:03:24.913782+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:03:24.913782+0000","last_undegraded":"2024-04-16T17:03:24.913782+0000","last_fullsized":"2024-04-16T17:03:24.913782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.889008+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:03:24.889008+0000","last_peered":"2024-04-16T17:03:24.889008+0000","last_clean":"2024-04-16T17:03:24.889008+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:03:24.889008+0000","last_undegraded":"2024-04-16T17:03:24.889008+0000","last_fullsized":"2024-04-16T17:03:24.889008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.311058+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:02:26.311058+0000","last_peered":"2024-04-16T17:02:26.311058+0000","last_clean":"2024-04-16T17:02:26.311058+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:02:26.311058+0000","last_undegraded":"2024-04-16T17:02:26.311058+0000","last_fullsized":"2024-04-16T17:02:26.311058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315585+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:02:26.315585+0000","last_peered":"2024-04-16T17:02:26.315585+0000","last_clean":"2024-04-16T17:02:26.315585+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:02:26.315585+0000","last_undegraded":"2024-04-16T17:02:26.315585+0000","last_fullsized":"2024-04-16T17:02:26.315585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.307452+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:02:26.307452+0000","last_peered":"2024-04-16T17:02:26.307452+0000","last_clean":"2024-04-16T17:02:26.307452+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:02:26.307452+0000","last_undegraded":"2024-04-16T17:02:26.307452+0000","last_fullsized":"2024-04-16T17:02:26.307452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.315749+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:02:26.315749+0000","last_peered":"2024-04-16T17:02:26.315749+0000","last_clean":"2024-04-16T17:02:26.315749+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:02:26.315749+0000","last_undegraded":"2024-04-16T17:02:26.315749+0000","last_fullsized":"2024-04-16T17:02:26.315749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.308340+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:02:26.308340+0000","last_peered":"2024-04-16T17:02:26.308340+0000","last_clean":"2024-04-16T17:02:26.308340+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:02:26.308340+0000","last_undegraded":"2024-04-16T17:02:26.308340+0000","last_fullsized":"2024-04-16T17:02:26.308340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.915075+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:03:24.915075+0000","last_peered":"2024-04-16T17:03:24.915075+0000","last_clean":"2024-04-16T17:03:24.915075+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:03:24.915075+0000","last_undegraded":"2024-04-16T17:03:24.915075+0000","last_fullsized":"2024-04-16T17:03:24.915075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.890744+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:03:24.890744+0000","last_peered":"2024-04-16T17:03:24.890744+0000","last_clean":"2024-04-16T17:03:24.890744+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:03:24.890744+0000","last_undegraded":"2024-04-16T17:03:24.890744+0000","last_fullsized":"2024-04-16T17:03:24.890744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245319,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67816,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":526,"num_read_kb":339,"num_write":164,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":203448,"data_compressed":5478,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":500845,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":1152297,"data_compressed":58028,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":505155,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":510481,"data_compressed":23679,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1096497,"data_compressed":56202,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,0,11,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":496235,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":560933,"data_compressed":22513,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":497623,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":560497,"data_compressed":22513,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":496429,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":501507,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":1092085,"data_compressed":56202,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64788,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64352,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65784,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:03:28.381 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:03:28.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:28 smithi043 ceph-mon[36901]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T17:03:28.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:28 smithi043 ceph-mon[36901]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-16T17:03:28.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:28 smithi043 ceph-mon[36901]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T17:03:28.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:28 smithi043 ceph-mon[36901]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:28.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:28 smithi022 ceph-mon[36023]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-16T17:03:28.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:28 smithi022 ceph-mon[36023]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-16T17:03:28.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:28 smithi022 ceph-mon[36023]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T17:03:28.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:28 smithi022 ceph-mon[36023]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:28.833 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:30.156 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:30 smithi018 ceph-mon[31776]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-16T17:03:30.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:30 smithi022 ceph-mon[36023]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-16T17:03:30.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:30 smithi043 ceph-mon[36901]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-16T17:03:30.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.960+0000 7fb9c2800700 1 -- 172.21.15.18:0/1898925831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9bc06a800 msgr2=0x7fb9bc06ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.960+0000 7fb9c2800700 1 --2- 172.21.15.18:0/1898925831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9bc06a800 0x7fb9bc06ac60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009990 tx=0x7fb9a8009ca0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.960+0000 7fb9c2800700 1 -- 172.21.15.18:0/1898925831 shutdown_connections 2024-04-16T17:03:30.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.960+0000 7fb9c2800700 1 --2- 172.21.15.18:0/1898925831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9bc108b70 0x7fb9bc108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.960+0000 7fb9c2800700 1 --2- 172.21.15.18:0/1898925831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9bc06b1a0 0x7fb9bc10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.960+0000 7fb9c2800700 1 --2- 172.21.15.18:0/1898925831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9bc06a800 0x7fb9bc06ac60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.960+0000 7fb9c2800700 1 -- 172.21.15.18:0/1898925831 >> 172.21.15.18:0/1898925831 conn(0x7fb9bc0789a0 msgr2=0x7fb9bc078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:30.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.961+0000 7fb9c2800700 1 -- 172.21.15.18:0/1898925831 shutdown_connections 2024-04-16T17:03:30.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.961+0000 7fb9c2800700 1 -- 172.21.15.18:0/1898925831 wait complete. 2024-04-16T17:03:30.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.962+0000 7fb9c2800700 1 Processor -- start 2024-04-16T17:03:30.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.962+0000 7fb9c2800700 1 -- start start 2024-04-16T17:03:30.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.962+0000 7fb9c2800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9bc06a800 0x7fb9bc1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:30.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.962+0000 7fb9c2800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9bc06b1a0 0x7fb9bc199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:30.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.962+0000 7fb9c2800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9bc108b70 0x7fb9bc19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:30.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.962+0000 7fb9c2800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc112e20 con 0x7fb9bc06a800 2024-04-16T17:03:30.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.962+0000 7fb9c2800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc112ca0 con 0x7fb9bc108b70 2024-04-16T17:03:30.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.962+0000 7fb9c2800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc112fa0 con 0x7fb9bc06b1a0 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9bc108b70 0x7fb9bc19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9bc06a800 0x7fb9bc1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9baa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9bc06b1a0 0x7fb9bc199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9bc108b70 0x7fb9bc19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:41460/0 (socket says 172.21.15.18:41460) 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bbe00700 1 -- 172.21.15.18:0/3964288257 learned_addr learned my addr 172.21.15.18:0/3964288257 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bbe00700 1 -- 172.21.15.18:0/3964288257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9bc06b1a0 msgr2=0x7fb9bc199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bbe00700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9bc06b1a0 0x7fb9bc199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bbe00700 1 -- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9bc06a800 msgr2=0x7fb9bc1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bbe00700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9bc06a800 0x7fb9bc1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.963+0000 7fb9bbe00700 1 -- 172.21.15.18:0/3964288257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc19e2e0 con 0x7fb9bc108b70 2024-04-16T17:03:30.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.964+0000 7fb9bbe00700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9bc108b70 0x7fb9bc19dba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4007ee0 tx=0x7fb9b400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:30.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.964+0000 7fb9b3e00700 1 -- 172.21.15.18:0/3964288257 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400dd20 con 0x7fb9bc108b70 2024-04-16T17:03:30.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.964+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc19e5d0 con 0x7fb9bc108b70 2024-04-16T17:03:30.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.964+0000 7fb9b3e00700 1 -- 172.21.15.18:0/3964288257 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b4004020 con 0x7fb9bc108b70 2024-04-16T17:03:30.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.964+0000 7fb9b3e00700 1 -- 172.21.15.18:0/3964288257 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4015660 con 0x7fb9bc108b70 2024-04-16T17:03:30.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.964+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1a5e50 con 0x7fb9bc108b70 2024-04-16T17:03:30.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.965+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9bc0697e0 con 0x7fb9bc108b70 2024-04-16T17:03:30.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.967+0000 7fb9b3e00700 1 -- 172.21.15.18:0/3964288257 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb9b40041d0 con 0x7fb9bc108b70 2024-04-16T17:03:30.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.968+0000 7fb9b3e00700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9a00a3c40 0x7fb9a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:30.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.969+0000 7fb9b3e00700 1 -- 172.21.15.18:0/3964288257 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fb9b40d0350 con 0x7fb9bc108b70 2024-04-16T17:03:30.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.969+0000 7fb9bb400700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9a00a3c40 0x7fb9a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:30.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.970+0000 7fb9bb400700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9a00a3c40 0x7fb9a00a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009be0 tx=0x7fb9ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:30.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:30.971+0000 7fb9b3e00700 1 -- 172.21.15.18:0/3964288257 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9b4010070 con 0x7fb9bc108b70 2024-04-16T17:03:31.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.202+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9bc062f30 con 0x7fb9a00a3c40 2024-04-16T17:03:31.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.214+0000 7fb9b3e00700 1 -- 172.21.15.18:0/3964288257 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340870 (secure 0 0 0) 0x7fb9bc062f30 con 0x7fb9a00a3c40 2024-04-16T17:03:31.222 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9a00a3c40 msgr2=0x7fb9a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9a00a3c40 0x7fb9a00a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009be0 tx=0x7fb9ac009260 comp rx=0 tx=0).stop 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9bc108b70 msgr2=0x7fb9bc19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9bc108b70 0x7fb9bc19dba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4007ee0 tx=0x7fb9b400d3b0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 shutdown_connections 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9a00a3c40 0x7fb9a00a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9bc108b70 0x7fb9bc19dba0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9bc06a800 0x7fb9bc1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 --2- 172.21.15.18:0/3964288257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9bc06b1a0 0x7fb9bc199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 >> 172.21.15.18:0/3964288257 conn(0x7fb9bc0789a0 msgr2=0x7fb9bc0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:31.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.220+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 shutdown_connections 2024-04-16T17:03:31.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:31.221+0000 7fb9c2800700 1 -- 172.21.15.18:0/3964288257 wait complete. 2024-04-16T17:03:31.226 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:03:32.044 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-16T17:03:30.936806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658667,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":688,"num_read_kb":469,"num_write":352,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378200,"kb_used_data":12344,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563624,"statfs":{"total":1151940427776,"available":1151553150976,"internally_reserved":0,"allocated":12640256,"data_stored":8113687,"data_compressed":408181,"data_compressed_allocated":557056,"data_compressed_original":2613248,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,0,11,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003731"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.054050+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:03:27.054050+0000","last_peered":"2024-04-16T17:03:27.054050+0000","last_clean":"2024-04-16T17:03:27.054050+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:03:27.054050+0000","last_undegraded":"2024-04-16T17:03:27.054050+0000","last_fullsized":"2024-04-16T17:03:27.054050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449882+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:03:27.449882+0000","last_peered":"2024-04-16T17:03:27.449882+0000","last_clean":"2024-04-16T17:03:27.449882+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:03:27.449882+0000","last_undegraded":"2024-04-16T17:03:27.449882+0000","last_fullsized":"2024-04-16T17:03:27.449882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449608+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:03:27.449608+0000","last_peered":"2024-04-16T17:03:27.449608+0000","last_clean":"2024-04-16T17:03:27.449608+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:03:27.449608+0000","last_undegraded":"2024-04-16T17:03:27.449608+0000","last_fullsized":"2024-04-16T17:03:27.449608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.050926+0000","last_change":"2024-04-16T17:02:26.318719+0000","last_active":"2024-04-16T17:03:26.050926+0000","last_peered":"2024-04-16T17:03:26.050926+0000","last_clean":"2024-04-16T17:03:26.050926+0000","last_became_active":"2024-04-16T17:02:26.317743+0000","last_became_peered":"2024-04-16T17:02:26.317743+0000","last_unstale":"2024-04-16T17:03:26.050926+0000","last_undegraded":"2024-04-16T17:03:26.050926+0000","last_fullsized":"2024-04-16T17:03:26.050926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:26.827859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046914+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:03:26.046914+0000","last_peered":"2024-04-16T17:03:26.046914+0000","last_clean":"2024-04-16T17:03:26.046914+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:03:26.046914+0000","last_undegraded":"2024-04-16T17:03:26.046914+0000","last_fullsized":"2024-04-16T17:03:26.046914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051435+0000","last_change":"2024-04-16T17:02:26.315910+0000","last_active":"2024-04-16T17:03:27.051435+0000","last_peered":"2024-04-16T17:03:27.051435+0000","last_clean":"2024-04-16T17:03:27.051435+0000","last_became_active":"2024-04-16T17:02:26.315493+0000","last_became_peered":"2024-04-16T17:02:26.315493+0000","last_unstale":"2024-04-16T17:03:27.051435+0000","last_undegraded":"2024-04-16T17:03:27.051435+0000","last_fullsized":"2024-04-16T17:03:27.051435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:27:48.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.894912+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:03:24.894912+0000","last_peered":"2024-04-16T17:03:24.894912+0000","last_clean":"2024-04-16T17:03:24.894912+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:03:24.894912+0000","last_undegraded":"2024-04-16T17:03:24.894912+0000","last_fullsized":"2024-04-16T17:03:24.894912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050577+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:03:27.050577+0000","last_peered":"2024-04-16T17:03:27.050577+0000","last_clean":"2024-04-16T17:03:27.050577+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:03:27.050577+0000","last_undegraded":"2024-04-16T17:03:27.050577+0000","last_fullsized":"2024-04-16T17:03:27.050577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267199+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:03:27.267199+0000","last_peered":"2024-04-16T17:03:27.267199+0000","last_clean":"2024-04-16T17:03:27.267199+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:03:27.267199+0000","last_undegraded":"2024-04-16T17:03:27.267199+0000","last_fullsized":"2024-04-16T17:03:27.267199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052004+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:03:27.052004+0000","last_peered":"2024-04-16T17:03:27.052004+0000","last_clean":"2024-04-16T17:03:27.052004+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:03:27.052004+0000","last_undegraded":"2024-04-16T17:03:27.052004+0000","last_fullsized":"2024-04-16T17:03:27.052004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.881296+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:03:24.881296+0000","last_peered":"2024-04-16T17:03:24.881296+0000","last_clean":"2024-04-16T17:03:24.881296+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:03:24.881296+0000","last_undegraded":"2024-04-16T17:03:24.881296+0000","last_fullsized":"2024-04-16T17:03:24.881296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051237+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:03:27.051237+0000","last_peered":"2024-04-16T17:03:27.051237+0000","last_clean":"2024-04-16T17:03:27.051237+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:03:27.051237+0000","last_undegraded":"2024-04-16T17:03:27.051237+0000","last_fullsized":"2024-04-16T17:03:27.051237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.909528+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:03:24.909528+0000","last_peered":"2024-04-16T17:03:24.909528+0000","last_clean":"2024-04-16T17:03:24.909528+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:03:24.909528+0000","last_undegraded":"2024-04-16T17:03:24.909528+0000","last_fullsized":"2024-04-16T17:03:24.909528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448788+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:03:27.448788+0000","last_peered":"2024-04-16T17:03:27.448788+0000","last_clean":"2024-04-16T17:03:27.448788+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:03:27.448788+0000","last_undegraded":"2024-04-16T17:03:27.448788+0000","last_fullsized":"2024-04-16T17:03:27.448788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448708+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:03:27.448708+0000","last_peered":"2024-04-16T17:03:27.448708+0000","last_clean":"2024-04-16T17:03:27.448708+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:03:27.448708+0000","last_undegraded":"2024-04-16T17:03:27.448708+0000","last_fullsized":"2024-04-16T17:03:27.448708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266225+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:03:27.266225+0000","last_peered":"2024-04-16T17:03:27.266225+0000","last_clean":"2024-04-16T17:03:27.266225+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:03:27.266225+0000","last_undegraded":"2024-04-16T17:03:27.266225+0000","last_fullsized":"2024-04-16T17:03:27.266225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053979+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:03:27.053979+0000","last_peered":"2024-04-16T17:03:27.053979+0000","last_clean":"2024-04-16T17:03:27.053979+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:03:27.053979+0000","last_undegraded":"2024-04-16T17:03:27.053979+0000","last_fullsized":"2024-04-16T17:03:27.053979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.050556+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:03:26.050556+0000","last_peered":"2024-04-16T17:03:26.050556+0000","last_clean":"2024-04-16T17:03:26.050556+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:03:26.050556+0000","last_undegraded":"2024-04-16T17:03:26.050556+0000","last_fullsized":"2024-04-16T17:03:26.050556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267524+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:03:27.267524+0000","last_peered":"2024-04-16T17:03:27.267524+0000","last_clean":"2024-04-16T17:03:27.267524+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:03:27.267524+0000","last_undegraded":"2024-04-16T17:03:27.267524+0000","last_fullsized":"2024-04-16T17:03:27.267524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267227+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:03:27.267227+0000","last_peered":"2024-04-16T17:03:27.267227+0000","last_clean":"2024-04-16T17:03:27.267227+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:03:27.267227+0000","last_undegraded":"2024-04-16T17:03:27.267227+0000","last_fullsized":"2024-04-16T17:03:27.267227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.053866+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:03:26.053866+0000","last_peered":"2024-04-16T17:03:26.053866+0000","last_clean":"2024-04-16T17:03:26.053866+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:03:26.053866+0000","last_undegraded":"2024-04-16T17:03:26.053866+0000","last_fullsized":"2024-04-16T17:03:26.053866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.050223+0000","last_change":"2024-04-16T17:02:26.318546+0000","last_active":"2024-04-16T17:03:26.050223+0000","last_peered":"2024-04-16T17:03:26.050223+0000","last_clean":"2024-04-16T17:03:26.050223+0000","last_became_active":"2024-04-16T17:02:26.317794+0000","last_became_peered":"2024-04-16T17:02:26.317794+0000","last_unstale":"2024-04-16T17:03:26.050223+0000","last_undegraded":"2024-04-16T17:03:26.050223+0000","last_fullsized":"2024-04-16T17:03:26.050223+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:48.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050304+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:03:27.050304+0000","last_peered":"2024-04-16T17:03:27.050304+0000","last_clean":"2024-04-16T17:03:27.050304+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:03:27.050304+0000","last_undegraded":"2024-04-16T17:03:27.050304+0000","last_fullsized":"2024-04-16T17:03:27.050304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052748+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:03:27.052748+0000","last_peered":"2024-04-16T17:03:27.052748+0000","last_clean":"2024-04-16T17:03:27.052748+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:03:27.052748+0000","last_undegraded":"2024-04-16T17:03:27.052748+0000","last_fullsized":"2024-04-16T17:03:27.052748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449900+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:03:27.449900+0000","last_peered":"2024-04-16T17:03:27.449900+0000","last_clean":"2024-04-16T17:03:27.449900+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:03:27.449900+0000","last_undegraded":"2024-04-16T17:03:27.449900+0000","last_fullsized":"2024-04-16T17:03:27.449900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.271585+0000","last_change":"2024-04-16T17:03:27.271585+0000","last_active":"2024-04-16T17:03:27.271585+0000","last_peered":"2024-04-16T17:03:27.271585+0000","last_clean":"2024-04-16T17:03:27.271585+0000","last_became_active":"2024-04-16T17:03:27.271251+0000","last_became_peered":"2024-04-16T17:03:27.271251+0000","last_unstale":"2024-04-16T17:03:27.271585+0000","last_undegraded":"2024-04-16T17:03:27.271585+0000","last_fullsized":"2024-04-16T17:03:27.271585+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:51.217466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047329+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:03:26.047329+0000","last_peered":"2024-04-16T17:03:26.047329+0000","last_clean":"2024-04-16T17:03:26.047329+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:03:26.047329+0000","last_undegraded":"2024-04-16T17:03:26.047329+0000","last_fullsized":"2024-04-16T17:03:26.047329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266687+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:03:27.266687+0000","last_peered":"2024-04-16T17:03:27.266687+0000","last_clean":"2024-04-16T17:03:27.266687+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:03:27.266687+0000","last_undegraded":"2024-04-16T17:03:27.266687+0000","last_fullsized":"2024-04-16T17:03:27.266687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050850+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:03:27.050850+0000","last_peered":"2024-04-16T17:03:27.050850+0000","last_clean":"2024-04-16T17:03:27.050850+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:03:27.050850+0000","last_undegraded":"2024-04-16T17:03:27.050850+0000","last_fullsized":"2024-04-16T17:03:27.050850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047181+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:03:26.047181+0000","last_peered":"2024-04-16T17:03:26.047181+0000","last_clean":"2024-04-16T17:03:26.047181+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:03:26.047181+0000","last_undegraded":"2024-04-16T17:03:26.047181+0000","last_fullsized":"2024-04-16T17:03:26.047181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048934+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:03:27.048934+0000","last_peered":"2024-04-16T17:03:27.048934+0000","last_clean":"2024-04-16T17:03:27.048934+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:03:27.048934+0000","last_undegraded":"2024-04-16T17:03:27.048934+0000","last_fullsized":"2024-04-16T17:03:27.048934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267582+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:03:27.267582+0000","last_peered":"2024-04-16T17:03:27.267582+0000","last_clean":"2024-04-16T17:03:27.267582+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:03:27.267582+0000","last_undegraded":"2024-04-16T17:03:27.267582+0000","last_fullsized":"2024-04-16T17:03:27.267582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":139,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048938+0000","last_change":"2024-04-16T17:02:26.315434+0000","last_active":"2024-04-16T17:03:27.048938+0000","last_peered":"2024-04-16T17:03:27.048938+0000","last_clean":"2024-04-16T17:03:27.048938+0000","last_became_active":"2024-04-16T17:02:26.310169+0000","last_became_peered":"2024-04-16T17:02:26.310169+0000","last_unstale":"2024-04-16T17:03:27.048938+0000","last_undegraded":"2024-04-16T17:03:27.048938+0000","last_fullsized":"2024-04-16T17:03:27.048938+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:51.033415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046779+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:03:26.046779+0000","last_peered":"2024-04-16T17:03:26.046779+0000","last_clean":"2024-04-16T17:03:26.046779+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:03:26.046779+0000","last_undegraded":"2024-04-16T17:03:26.046779+0000","last_fullsized":"2024-04-16T17:03:26.046779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267268+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:03:27.267268+0000","last_peered":"2024-04-16T17:03:27.267268+0000","last_clean":"2024-04-16T17:03:27.267268+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:03:27.267268+0000","last_undegraded":"2024-04-16T17:03:27.267268+0000","last_fullsized":"2024-04-16T17:03:27.267268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051285+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:03:27.051285+0000","last_peered":"2024-04-16T17:03:27.051285+0000","last_clean":"2024-04-16T17:03:27.051285+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:03:27.051285+0000","last_undegraded":"2024-04-16T17:03:27.051285+0000","last_fullsized":"2024-04-16T17:03:27.051285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.053337+0000","last_change":"2024-04-16T17:02:26.321763+0000","last_active":"2024-04-16T17:03:26.053337+0000","last_peered":"2024-04-16T17:03:26.053337+0000","last_clean":"2024-04-16T17:03:26.053337+0000","last_became_active":"2024-04-16T17:02:26.321438+0000","last_became_peered":"2024-04-16T17:02:26.321438+0000","last_unstale":"2024-04-16T17:03:26.053337+0000","last_undegraded":"2024-04-16T17:03:26.053337+0000","last_fullsized":"2024-04-16T17:03:26.053337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:47.231035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.451220+0000","last_change":"2024-04-16T17:03:27.451220+0000","last_active":"2024-04-16T17:03:27.451220+0000","last_peered":"2024-04-16T17:03:27.451220+0000","last_clean":"2024-04-16T17:03:27.451220+0000","last_became_active":"2024-04-16T17:03:27.450680+0000","last_became_peered":"2024-04-16T17:03:27.450680+0000","last_unstale":"2024-04-16T17:03:27.451220+0000","last_undegraded":"2024-04-16T17:03:27.451220+0000","last_fullsized":"2024-04-16T17:03:27.451220+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:23.270332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.749687+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:02:26.749687+0000","last_peered":"2024-04-16T17:02:26.749687+0000","last_clean":"2024-04-16T17:02:26.749687+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:02:26.749687+0000","last_undegraded":"2024-04-16T17:02:26.749687+0000","last_fullsized":"2024-04-16T17:02:26.749687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050234+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:03:27.050234+0000","last_peered":"2024-04-16T17:03:27.050234+0000","last_clean":"2024-04-16T17:03:27.050234+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:03:27.050234+0000","last_undegraded":"2024-04-16T17:03:27.050234+0000","last_fullsized":"2024-04-16T17:03:27.050234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449714+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:03:27.449714+0000","last_peered":"2024-04-16T17:03:27.449714+0000","last_clean":"2024-04-16T17:03:27.449714+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:03:27.449714+0000","last_undegraded":"2024-04-16T17:03:27.449714+0000","last_fullsized":"2024-04-16T17:03:27.449714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048433+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:03:27.048433+0000","last_peered":"2024-04-16T17:03:27.048433+0000","last_clean":"2024-04-16T17:03:27.048433+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:03:27.048433+0000","last_undegraded":"2024-04-16T17:03:27.048433+0000","last_fullsized":"2024-04-16T17:03:27.048433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.050276+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:03:26.050276+0000","last_peered":"2024-04-16T17:03:26.050276+0000","last_clean":"2024-04-16T17:03:26.050276+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:03:26.050276+0000","last_undegraded":"2024-04-16T17:03:26.050276+0000","last_fullsized":"2024-04-16T17:03:26.050276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051478+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:03:27.051478+0000","last_peered":"2024-04-16T17:03:27.051478+0000","last_clean":"2024-04-16T17:03:27.051478+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:03:27.051478+0000","last_undegraded":"2024-04-16T17:03:27.051478+0000","last_fullsized":"2024-04-16T17:03:27.051478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":123,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051826+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:03:27.051826+0000","last_peered":"2024-04-16T17:03:27.051826+0000","last_clean":"2024-04-16T17:03:27.051826+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:03:27.051826+0000","last_undegraded":"2024-04-16T17:03:27.051826+0000","last_fullsized":"2024-04-16T17:03:27.051826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267330+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:03:27.267330+0000","last_peered":"2024-04-16T17:03:27.267330+0000","last_clean":"2024-04-16T17:03:27.267330+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:03:27.267330+0000","last_undegraded":"2024-04-16T17:03:27.267330+0000","last_fullsized":"2024-04-16T17:03:27.267330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.050799+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:03:26.050799+0000","last_peered":"2024-04-16T17:03:26.050799+0000","last_clean":"2024-04-16T17:03:26.050799+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:03:26.050799+0000","last_undegraded":"2024-04-16T17:03:26.050799+0000","last_fullsized":"2024-04-16T17:03:26.050799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306687+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:02:26.306687+0000","last_peered":"2024-04-16T17:02:26.306687+0000","last_clean":"2024-04-16T17:02:26.306687+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:02:26.306687+0000","last_undegraded":"2024-04-16T17:02:26.306687+0000","last_fullsized":"2024-04-16T17:02:26.306687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449950+0000","last_change":"2024-04-16T17:02:26.752275+0000","last_active":"2024-04-16T17:03:27.449950+0000","last_peered":"2024-04-16T17:03:27.449950+0000","last_clean":"2024-04-16T17:03:27.449950+0000","last_became_active":"2024-04-16T17:02:26.751729+0000","last_became_peered":"2024-04-16T17:02:26.751729+0000","last_unstale":"2024-04-16T17:03:27.449950+0000","last_undegraded":"2024-04-16T17:03:27.449950+0000","last_fullsized":"2024-04-16T17:03:27.449950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:41:35.777691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051500+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:03:27.051500+0000","last_peered":"2024-04-16T17:03:27.051500+0000","last_clean":"2024-04-16T17:03:27.051500+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:03:27.051500+0000","last_undegraded":"2024-04-16T17:03:27.051500+0000","last_fullsized":"2024-04-16T17:03:27.051500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048902+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:03:27.048902+0000","last_peered":"2024-04-16T17:03:27.048902+0000","last_clean":"2024-04-16T17:03:27.048902+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:03:27.048902+0000","last_undegraded":"2024-04-16T17:03:27.048902+0000","last_fullsized":"2024-04-16T17:03:27.048902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266867+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:03:27.266867+0000","last_peered":"2024-04-16T17:03:27.266867+0000","last_clean":"2024-04-16T17:03:27.266867+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:03:27.266867+0000","last_undegraded":"2024-04-16T17:03:27.266867+0000","last_fullsized":"2024-04-16T17:03:27.266867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.055021+0000","last_change":"2024-04-16T17:03:27.054627+0000","last_active":"2024-04-16T17:03:27.055021+0000","last_peered":"2024-04-16T17:03:27.055021+0000","last_clean":"2024-04-16T17:03:27.055021+0000","last_became_active":"2024-04-16T17:03:27.054326+0000","last_became_peered":"2024-04-16T17:03:27.054326+0000","last_unstale":"2024-04-16T17:03:27.055021+0000","last_undegraded":"2024-04-16T17:03:27.055021+0000","last_fullsized":"2024-04-16T17:03:27.055021+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:43.535958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":255,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.397478+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:03:27.397478+0000","last_peered":"2024-04-16T17:03:27.397478+0000","last_clean":"2024-04-16T17:03:27.397478+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:03:27.397478+0000","last_undegraded":"2024-04-16T17:03:27.397478+0000","last_fullsized":"2024-04-16T17:03:27.397478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64455,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":188,"num_read_kb":158,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047288+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:03:26.047288+0000","last_peered":"2024-04-16T17:03:26.047288+0000","last_clean":"2024-04-16T17:03:26.047288+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:03:26.047288+0000","last_undegraded":"2024-04-16T17:03:26.047288+0000","last_fullsized":"2024-04-16T17:03:26.047288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.049199+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:03:27.049199+0000","last_peered":"2024-04-16T17:03:27.049199+0000","last_clean":"2024-04-16T17:03:27.049199+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:03:27.049199+0000","last_undegraded":"2024-04-16T17:03:27.049199+0000","last_fullsized":"2024-04-16T17:03:27.049199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267266+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:03:27.267266+0000","last_peered":"2024-04-16T17:03:27.267266+0000","last_clean":"2024-04-16T17:03:27.267266+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:03:27.267266+0000","last_undegraded":"2024-04-16T17:03:27.267266+0000","last_fullsized":"2024-04-16T17:03:27.267266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.911618+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:03:24.911618+0000","last_peered":"2024-04-16T17:03:24.911618+0000","last_clean":"2024-04-16T17:03:24.911618+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:03:24.911618+0000","last_undegraded":"2024-04-16T17:03:24.911618+0000","last_fullsized":"2024-04-16T17:03:24.911618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053205+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:03:27.053205+0000","last_peered":"2024-04-16T17:03:27.053205+0000","last_clean":"2024-04-16T17:03:27.053205+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:03:27.053205+0000","last_undegraded":"2024-04-16T17:03:27.053205+0000","last_fullsized":"2024-04-16T17:03:27.053205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048636+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:03:27.048636+0000","last_peered":"2024-04-16T17:03:27.048636+0000","last_clean":"2024-04-16T17:03:27.048636+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:03:27.048636+0000","last_undegraded":"2024-04-16T17:03:27.048636+0000","last_fullsized":"2024-04-16T17:03:27.048636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.909273+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:03:24.909273+0000","last_peered":"2024-04-16T17:03:24.909273+0000","last_clean":"2024-04-16T17:03:24.909273+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:03:24.909273+0000","last_undegraded":"2024-04-16T17:03:24.909273+0000","last_fullsized":"2024-04-16T17:03:24.909273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.051892+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:03:26.051892+0000","last_peered":"2024-04-16T17:03:26.051892+0000","last_clean":"2024-04-16T17:03:26.051892+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:03:26.051892+0000","last_undegraded":"2024-04-16T17:03:26.051892+0000","last_fullsized":"2024-04-16T17:03:26.051892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050636+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:03:27.050636+0000","last_peered":"2024-04-16T17:03:27.050636+0000","last_clean":"2024-04-16T17:03:27.050636+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:03:27.050636+0000","last_undegraded":"2024-04-16T17:03:27.050636+0000","last_fullsized":"2024-04-16T17:03:27.050636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.889205+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:03:24.889205+0000","last_peered":"2024-04-16T17:03:24.889205+0000","last_clean":"2024-04-16T17:03:24.889205+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:03:24.889205+0000","last_undegraded":"2024-04-16T17:03:24.889205+0000","last_fullsized":"2024-04-16T17:03:24.889205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750497+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:02:26.750497+0000","last_peered":"2024-04-16T17:02:26.750497+0000","last_clean":"2024-04-16T17:02:26.750497+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:02:26.750497+0000","last_undegraded":"2024-04-16T17:02:26.750497+0000","last_fullsized":"2024-04-16T17:02:26.750497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046789+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:03:26.046789+0000","last_peered":"2024-04-16T17:03:26.046789+0000","last_clean":"2024-04-16T17:03:26.046789+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:03:26.046789+0000","last_undegraded":"2024-04-16T17:03:26.046789+0000","last_fullsized":"2024-04-16T17:03:26.046789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048626+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:03:27.048626+0000","last_peered":"2024-04-16T17:03:27.048626+0000","last_clean":"2024-04-16T17:03:27.048626+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:03:27.048626+0000","last_undegraded":"2024-04-16T17:03:27.048626+0000","last_fullsized":"2024-04-16T17:03:27.048626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048451+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:03:27.048451+0000","last_peered":"2024-04-16T17:03:27.048451+0000","last_clean":"2024-04-16T17:03:27.048451+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:03:27.048451+0000","last_undegraded":"2024-04-16T17:03:27.048451+0000","last_fullsized":"2024-04-16T17:03:27.048451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053451+0000","last_change":"2024-04-16T17:02:26.320405+0000","last_active":"2024-04-16T17:03:27.053451+0000","last_peered":"2024-04-16T17:03:27.053451+0000","last_clean":"2024-04-16T17:03:27.053451+0000","last_became_active":"2024-04-16T17:02:26.307921+0000","last_became_peered":"2024-04-16T17:02:26.307921+0000","last_unstale":"2024-04-16T17:03:27.053451+0000","last_undegraded":"2024-04-16T17:03:27.053451+0000","last_fullsized":"2024-04-16T17:03:27.053451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:19.156260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053407+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:03:27.053407+0000","last_peered":"2024-04-16T17:03:27.053407+0000","last_clean":"2024-04-16T17:03:27.053407+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:03:27.053407+0000","last_undegraded":"2024-04-16T17:03:27.053407+0000","last_fullsized":"2024-04-16T17:03:27.053407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047049+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:03:26.047049+0000","last_peered":"2024-04-16T17:03:26.047049+0000","last_clean":"2024-04-16T17:03:26.047049+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:03:26.047049+0000","last_undegraded":"2024-04-16T17:03:26.047049+0000","last_fullsized":"2024-04-16T17:03:26.047049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.052491+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:03:26.052491+0000","last_peered":"2024-04-16T17:03:26.052491+0000","last_clean":"2024-04-16T17:03:26.052491+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:03:26.052491+0000","last_undegraded":"2024-04-16T17:03:26.052491+0000","last_fullsized":"2024-04-16T17:03:26.052491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051550+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:03:27.051550+0000","last_peered":"2024-04-16T17:03:27.051550+0000","last_clean":"2024-04-16T17:03:27.051550+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:03:27.051550+0000","last_undegraded":"2024-04-16T17:03:27.051550+0000","last_fullsized":"2024-04-16T17:03:27.051550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750101+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:02:26.750101+0000","last_peered":"2024-04-16T17:02:26.750101+0000","last_clean":"2024-04-16T17:02:26.750101+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:02:26.750101+0000","last_undegraded":"2024-04-16T17:02:26.750101+0000","last_fullsized":"2024-04-16T17:02:26.750101+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306451+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:02:26.306451+0000","last_peered":"2024-04-16T17:02:26.306451+0000","last_clean":"2024-04-16T17:02:26.306451+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:02:26.306451+0000","last_undegraded":"2024-04-16T17:02:26.306451+0000","last_fullsized":"2024-04-16T17:02:26.306451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050933+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:03:27.050933+0000","last_peered":"2024-04-16T17:03:27.050933+0000","last_clean":"2024-04-16T17:03:27.050933+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:03:27.050933+0000","last_undegraded":"2024-04-16T17:03:27.050933+0000","last_fullsized":"2024-04-16T17:03:27.050933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750345+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:02:26.750345+0000","last_peered":"2024-04-16T17:02:26.750345+0000","last_clean":"2024-04-16T17:02:26.750345+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:02:26.750345+0000","last_undegraded":"2024-04-16T17:02:26.750345+0000","last_fullsized":"2024-04-16T17:02:26.750345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048660+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:03:27.048660+0000","last_peered":"2024-04-16T17:03:27.048660+0000","last_clean":"2024-04-16T17:03:27.048660+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:03:27.048660+0000","last_undegraded":"2024-04-16T17:03:27.048660+0000","last_fullsized":"2024-04-16T17:03:27.048660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.364287+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:03:24.364287+0000","last_peered":"2024-04-16T17:03:24.364287+0000","last_clean":"2024-04-16T17:03:24.364287+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:03:24.364287+0000","last_undegraded":"2024-04-16T17:03:24.364287+0000","last_fullsized":"2024-04-16T17:03:24.364287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306573+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:02:26.306573+0000","last_peered":"2024-04-16T17:02:26.306573+0000","last_clean":"2024-04-16T17:02:26.306573+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:02:26.306573+0000","last_undegraded":"2024-04-16T17:02:26.306573+0000","last_fullsized":"2024-04-16T17:02:26.306573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051850+0000","last_change":"2024-04-16T17:02:26.322313+0000","last_active":"2024-04-16T17:03:27.051850+0000","last_peered":"2024-04-16T17:03:27.051850+0000","last_clean":"2024-04-16T17:03:27.051850+0000","last_became_active":"2024-04-16T17:02:26.320328+0000","last_became_peered":"2024-04-16T17:02:26.320328+0000","last_unstale":"2024-04-16T17:03:27.051850+0000","last_undegraded":"2024-04-16T17:03:27.051850+0000","last_fullsized":"2024-04-16T17:03:27.051850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:40.943183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051362+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:03:27.051362+0000","last_peered":"2024-04-16T17:03:27.051362+0000","last_clean":"2024-04-16T17:03:27.051362+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:03:27.051362+0000","last_undegraded":"2024-04-16T17:03:27.051362+0000","last_fullsized":"2024-04-16T17:03:27.051362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.306928+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:02:26.306928+0000","last_peered":"2024-04-16T17:02:26.306928+0000","last_clean":"2024-04-16T17:02:26.306928+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:02:26.306928+0000","last_undegraded":"2024-04-16T17:02:26.306928+0000","last_fullsized":"2024-04-16T17:02:26.306928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048901+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:03:27.048901+0000","last_peered":"2024-04-16T17:03:27.048901+0000","last_clean":"2024-04-16T17:03:27.048901+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:03:27.048901+0000","last_undegraded":"2024-04-16T17:03:27.048901+0000","last_fullsized":"2024-04-16T17:03:27.048901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.895629+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:03:24.895629+0000","last_peered":"2024-04-16T17:03:24.895629+0000","last_clean":"2024-04-16T17:03:24.895629+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:03:24.895629+0000","last_undegraded":"2024-04-16T17:03:24.895629+0000","last_fullsized":"2024-04-16T17:03:24.895629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052945+0000","last_change":"2024-04-16T17:02:26.312852+0000","last_active":"2024-04-16T17:03:27.052945+0000","last_peered":"2024-04-16T17:03:27.052945+0000","last_clean":"2024-04-16T17:03:27.052945+0000","last_became_active":"2024-04-16T17:02:26.308279+0000","last_became_peered":"2024-04-16T17:02:26.308279+0000","last_unstale":"2024-04-16T17:03:27.052945+0000","last_undegraded":"2024-04-16T17:03:27.052945+0000","last_fullsized":"2024-04-16T17:03:27.052945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:25.511349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053201+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:03:27.053201+0000","last_peered":"2024-04-16T17:03:27.053201+0000","last_clean":"2024-04-16T17:03:27.053201+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:03:27.053201+0000","last_undegraded":"2024-04-16T17:03:27.053201+0000","last_fullsized":"2024-04-16T17:03:27.053201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.047138+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:03:26.047138+0000","last_peered":"2024-04-16T17:03:26.047138+0000","last_clean":"2024-04-16T17:03:26.047138+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:03:26.047138+0000","last_undegraded":"2024-04-16T17:03:26.047138+0000","last_fullsized":"2024-04-16T17:03:26.047138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.905706+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:03:24.905706+0000","last_peered":"2024-04-16T17:03:24.905706+0000","last_clean":"2024-04-16T17:03:24.905706+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:03:24.905706+0000","last_undegraded":"2024-04-16T17:03:24.905706+0000","last_fullsized":"2024-04-16T17:03:24.905706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050407+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:03:27.050407+0000","last_peered":"2024-04-16T17:03:27.050407+0000","last_clean":"2024-04-16T17:03:27.050407+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:03:27.050407+0000","last_undegraded":"2024-04-16T17:03:27.050407+0000","last_fullsized":"2024-04-16T17:03:27.050407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.050867+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:03:26.050867+0000","last_peered":"2024-04-16T17:03:26.050867+0000","last_clean":"2024-04-16T17:03:26.050867+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:03:26.050867+0000","last_undegraded":"2024-04-16T17:03:26.050867+0000","last_fullsized":"2024-04-16T17:03:26.050867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053562+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:03:27.053562+0000","last_peered":"2024-04-16T17:03:27.053562+0000","last_clean":"2024-04-16T17:03:27.053562+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:03:27.053562+0000","last_undegraded":"2024-04-16T17:03:27.053562+0000","last_fullsized":"2024-04-16T17:03:27.053562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266852+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:03:27.266852+0000","last_peered":"2024-04-16T17:03:27.266852+0000","last_clean":"2024-04-16T17:03:27.266852+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:03:27.266852+0000","last_undegraded":"2024-04-16T17:03:27.266852+0000","last_fullsized":"2024-04-16T17:03:27.266852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050953+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:03:27.050953+0000","last_peered":"2024-04-16T17:03:27.050953+0000","last_clean":"2024-04-16T17:03:27.050953+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:03:27.050953+0000","last_undegraded":"2024-04-16T17:03:27.050953+0000","last_fullsized":"2024-04-16T17:03:27.050953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.450226+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:03:27.450226+0000","last_peered":"2024-04-16T17:03:27.450226+0000","last_clean":"2024-04-16T17:03:27.450226+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:03:27.450226+0000","last_undegraded":"2024-04-16T17:03:27.450226+0000","last_fullsized":"2024-04-16T17:03:27.450226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050691+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:03:27.050691+0000","last_peered":"2024-04-16T17:03:27.050691+0000","last_clean":"2024-04-16T17:03:27.050691+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:03:27.050691+0000","last_undegraded":"2024-04-16T17:03:27.050691+0000","last_fullsized":"2024-04-16T17:03:27.050691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266239+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:03:27.266239+0000","last_peered":"2024-04-16T17:03:27.266239+0000","last_clean":"2024-04-16T17:03:27.266239+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:03:27.266239+0000","last_undegraded":"2024-04-16T17:03:27.266239+0000","last_fullsized":"2024-04-16T17:03:27.266239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046828+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:03:26.046828+0000","last_peered":"2024-04-16T17:03:26.046828+0000","last_clean":"2024-04-16T17:03:26.046828+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:03:26.046828+0000","last_undegraded":"2024-04-16T17:03:26.046828+0000","last_fullsized":"2024-04-16T17:03:26.046828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2024-04-16T17:01:28.999759+0000","last_change":"2024-04-16T17:03:26.709595+0000","last_active":"2024-04-16T17:01:27.966977+0000","last_peered":"2024-04-16T17:01:27.966977+0000","last_clean":"2024-04-16T17:01:27.966977+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T17:01:28.999759+0000","last_undegraded":"2024-04-16T17:01:28.999759+0000","last_fullsized":"2024-04-16T17:01:28.999759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050765+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:03:27.050765+0000","last_peered":"2024-04-16T17:03:27.050765+0000","last_clean":"2024-04-16T17:03:27.050765+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:03:27.050765+0000","last_undegraded":"2024-04-16T17:03:27.050765+0000","last_fullsized":"2024-04-16T17:03:27.050765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053099+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:03:27.053099+0000","last_peered":"2024-04-16T17:03:27.053099+0000","last_clean":"2024-04-16T17:03:27.053099+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:03:27.053099+0000","last_undegraded":"2024-04-16T17:03:27.053099+0000","last_fullsized":"2024-04-16T17:03:27.053099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050922+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:03:27.050922+0000","last_peered":"2024-04-16T17:03:27.050922+0000","last_clean":"2024-04-16T17:03:27.050922+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:03:27.050922+0000","last_undegraded":"2024-04-16T17:03:27.050922+0000","last_fullsized":"2024-04-16T17:03:27.050922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267000+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:03:27.267000+0000","last_peered":"2024-04-16T17:03:27.267000+0000","last_clean":"2024-04-16T17:03:27.267000+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:03:27.267000+0000","last_undegraded":"2024-04-16T17:03:27.267000+0000","last_fullsized":"2024-04-16T17:03:27.267000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052332+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:03:27.052332+0000","last_peered":"2024-04-16T17:03:27.052332+0000","last_clean":"2024-04-16T17:03:27.052332+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:03:27.052332+0000","last_undegraded":"2024-04-16T17:03:27.052332+0000","last_fullsized":"2024-04-16T17:03:27.052332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050542+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:03:27.050542+0000","last_peered":"2024-04-16T17:03:27.050542+0000","last_clean":"2024-04-16T17:03:27.050542+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:03:27.050542+0000","last_undegraded":"2024-04-16T17:03:27.050542+0000","last_fullsized":"2024-04-16T17:03:27.050542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:03:26.046799+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:03:26.046799+0000","last_peered":"2024-04-16T17:03:26.046799+0000","last_clean":"2024-04-16T17:03:26.046799+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:03:26.046799+0000","last_undegraded":"2024-04-16T17:03:26.046799+0000","last_fullsized":"2024-04-16T17:03:26.046799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.894770+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:03:24.894770+0000","last_peered":"2024-04-16T17:03:24.894770+0000","last_clean":"2024-04-16T17:03:24.894770+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:03:24.894770+0000","last_undegraded":"2024-04-16T17:03:24.894770+0000","last_fullsized":"2024-04-16T17:03:24.894770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266288+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:03:27.266288+0000","last_peered":"2024-04-16T17:03:27.266288+0000","last_clean":"2024-04-16T17:03:27.266288+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:03:27.266288+0000","last_undegraded":"2024-04-16T17:03:27.266288+0000","last_fullsized":"2024-04-16T17:03:27.266288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266452+0000","last_change":"2024-04-16T17:02:26.328059+0000","last_active":"2024-04-16T17:03:27.266452+0000","last_peered":"2024-04-16T17:03:27.266452+0000","last_clean":"2024-04-16T17:03:27.266452+0000","last_became_active":"2024-04-16T17:02:26.323315+0000","last_became_peered":"2024-04-16T17:02:26.323315+0000","last_unstale":"2024-04-16T17:03:27.266452+0000","last_undegraded":"2024-04-16T17:03:27.266452+0000","last_fullsized":"2024-04-16T17:03:27.266452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:01.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.893270+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:03:24.893270+0000","last_peered":"2024-04-16T17:03:24.893270+0000","last_clean":"2024-04-16T17:03:24.893270+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:03:24.893270+0000","last_undegraded":"2024-04-16T17:03:24.893270+0000","last_fullsized":"2024-04-16T17:03:24.893270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448803+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:03:27.448803+0000","last_peered":"2024-04-16T17:03:27.448803+0000","last_clean":"2024-04-16T17:03:27.448803+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:03:27.448803+0000","last_undegraded":"2024-04-16T17:03:27.448803+0000","last_fullsized":"2024-04-16T17:03:27.448803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.049078+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:03:27.049078+0000","last_peered":"2024-04-16T17:03:27.049078+0000","last_clean":"2024-04-16T17:03:27.049078+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:03:27.049078+0000","last_undegraded":"2024-04-16T17:03:27.049078+0000","last_fullsized":"2024-04-16T17:03:27.049078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.902565+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:03:24.902565+0000","last_peered":"2024-04-16T17:03:24.902565+0000","last_clean":"2024-04-16T17:03:24.902565+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:03:24.902565+0000","last_undegraded":"2024-04-16T17:03:24.902565+0000","last_fullsized":"2024-04-16T17:03:24.902565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048220+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:03:27.048220+0000","last_peered":"2024-04-16T17:03:27.048220+0000","last_clean":"2024-04-16T17:03:27.048220+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:03:27.048220+0000","last_undegraded":"2024-04-16T17:03:27.048220+0000","last_fullsized":"2024-04-16T17:03:27.048220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:02:26.750355+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:02:26.750355+0000","last_peered":"2024-04-16T17:02:26.750355+0000","last_clean":"2024-04-16T17:02:26.750355+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:02:26.750355+0000","last_undegraded":"2024-04-16T17:02:26.750355+0000","last_fullsized":"2024-04-16T17:02:26.750355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.897406+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:03:24.897406+0000","last_peered":"2024-04-16T17:03:24.897406+0000","last_clean":"2024-04-16T17:03:24.897406+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:03:24.897406+0000","last_undegraded":"2024-04-16T17:03:24.897406+0000","last_fullsized":"2024-04-16T17:03:24.897406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053402+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:03:27.053402+0000","last_peered":"2024-04-16T17:03:27.053402+0000","last_clean":"2024-04-16T17:03:27.053402+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:03:27.053402+0000","last_undegraded":"2024-04-16T17:03:27.053402+0000","last_fullsized":"2024-04-16T17:03:27.053402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267414+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:03:27.267414+0000","last_peered":"2024-04-16T17:03:27.267414+0000","last_clean":"2024-04-16T17:03:27.267414+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:03:27.267414+0000","last_undegraded":"2024-04-16T17:03:27.267414+0000","last_fullsized":"2024-04-16T17:03:27.267414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266941+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:03:27.266941+0000","last_peered":"2024-04-16T17:03:27.266941+0000","last_clean":"2024-04-16T17:03:27.266941+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:03:27.266941+0000","last_undegraded":"2024-04-16T17:03:27.266941+0000","last_fullsized":"2024-04-16T17:03:27.266941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449088+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:03:27.449088+0000","last_peered":"2024-04-16T17:03:27.449088+0000","last_clean":"2024-04-16T17:03:27.449088+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:03:27.449088+0000","last_undegraded":"2024-04-16T17:03:27.449088+0000","last_fullsized":"2024-04-16T17:03:27.449088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.913782+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:03:24.913782+0000","last_peered":"2024-04-16T17:03:24.913782+0000","last_clean":"2024-04-16T17:03:24.913782+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:03:24.913782+0000","last_undegraded":"2024-04-16T17:03:24.913782+0000","last_fullsized":"2024-04-16T17:03:24.913782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053501+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:03:27.053501+0000","last_peered":"2024-04-16T17:03:27.053501+0000","last_clean":"2024-04-16T17:03:27.053501+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:03:27.053501+0000","last_undegraded":"2024-04-16T17:03:27.053501+0000","last_fullsized":"2024-04-16T17:03:27.053501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267623+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:03:27.267623+0000","last_peered":"2024-04-16T17:03:27.267623+0000","last_clean":"2024-04-16T17:03:27.267623+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:03:27.267623+0000","last_undegraded":"2024-04-16T17:03:27.267623+0000","last_fullsized":"2024-04-16T17:03:27.267623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053734+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:03:27.053734+0000","last_peered":"2024-04-16T17:03:27.053734+0000","last_clean":"2024-04-16T17:03:27.053734+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:03:27.053734+0000","last_undegraded":"2024-04-16T17:03:27.053734+0000","last_fullsized":"2024-04-16T17:03:27.053734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048462+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:03:27.048462+0000","last_peered":"2024-04-16T17:03:27.048462+0000","last_clean":"2024-04-16T17:03:27.048462+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:03:27.048462+0000","last_undegraded":"2024-04-16T17:03:27.048462+0000","last_fullsized":"2024-04-16T17:03:27.048462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050912+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:03:27.050912+0000","last_peered":"2024-04-16T17:03:27.050912+0000","last_clean":"2024-04-16T17:03:27.050912+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:03:27.050912+0000","last_undegraded":"2024-04-16T17:03:27.050912+0000","last_fullsized":"2024-04-16T17:03:27.050912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266610+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:03:27.266610+0000","last_peered":"2024-04-16T17:03:27.266610+0000","last_clean":"2024-04-16T17:03:27.266610+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:03:27.266610+0000","last_undegraded":"2024-04-16T17:03:27.266610+0000","last_fullsized":"2024-04-16T17:03:27.266610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-16T17:03:24.915075+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:03:24.915075+0000","last_peered":"2024-04-16T17:03:24.915075+0000","last_clean":"2024-04-16T17:03:24.915075+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:03:24.915075+0000","last_undegraded":"2024-04-16T17:03:24.915075+0000","last_fullsized":"2024-04-16T17:03:24.915075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053482+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:03:27.053482+0000","last_peered":"2024-04-16T17:03:27.053482+0000","last_clean":"2024-04-16T17:03:27.053482+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:03:27.053482+0000","last_undegraded":"2024-04-16T17:03:27.053482+0000","last_fullsized":"2024-04-16T17:03:27.053482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245319,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68065,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":552,"num_read_kb":350,"num_write":169,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":204195,"data_compressed":5478,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":519809,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1163264,"data_stored":1171510,"data_compressed":64019,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":514679,"data_compressed":23679,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":510481,"data_compressed":23679,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1096497,"data_compressed":56202,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,0,11,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":496235,"data_compressed":20687,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":580146,"data_compressed":28504,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":516587,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":579710,"data_compressed":28504,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":515393,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":511031,"data_compressed":23679,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":1101609,"data_compressed":59194,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65037,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64601,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66033,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:03:32.050 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.f is not active+clean 2024-04-16T17:03:32.050 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 92, 'state': 'peering', 'last_fresh': '2024-04-16T17:01:28.999759+0000', 'last_change': '2024-04-16T17:03:26.709595+0000', 'last_active': '2024-04-16T17:01:27.966977+0000', 'last_peered': '2024-04-16T17:01:27.966977+0000', 'last_clean': '2024-04-16T17:01:27.966977+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-16T17:01:28.999759+0000', 'last_undegraded': '2024-04-16T17:01:28.999759+0000', 'last_fullsized': '2024-04-16T17:01:28.999759+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 88, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-16T17:01:27.966977+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-16T17:01:27.966977+0000', 'last_clean_scrub_stamp': '2024-04-16T17:01:27.966977+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 3, 2, 5], 'acting': [8, 3, 2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2024-04-16T17:03:32.050 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph status --format=json 2024-04-16T17:03:32.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:32 smithi022 ceph-mon[36023]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-16T17:03:32.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:32 smithi022 ceph-mon[36023]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:32 smithi018 ceph-mon[31776]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-16T17:03:32.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:32 smithi018 ceph-mon[31776]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:32.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:32 smithi043 ceph-mon[36901]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-16T17:03:32.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:32 smithi043 ceph-mon[36901]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:32.533 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:33.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:33 smithi022 ceph-mon[36023]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-16T17:03:33.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:33 smithi022 ceph-mon[36023]: Cluster is now healthy 2024-04-16T17:03:33.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:33 smithi018 ceph-mon[31776]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-16T17:03:33.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:33 smithi018 ceph-mon[31776]: Cluster is now healthy 2024-04-16T17:03:33.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:33 smithi043 ceph-mon[36901]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-16T17:03:33.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:33 smithi043 ceph-mon[36901]: Cluster is now healthy 2024-04-16T17:03:34.318 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:34 smithi018 ceph-mon[31776]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-16T17:03:34.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:34 smithi022 ceph-mon[36023]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-16T17:03:34.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:34 smithi043 ceph-mon[36901]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-16T17:03:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.854+0000 7f49d5400700 1 -- 172.21.15.18:0/86865515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49d0109d30 msgr2=0x7f49d010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.854+0000 7f49d5400700 1 --2- 172.21.15.18:0/86865515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49d0109d30 0x7f49d010dd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0099c0 tx=0x7f49bc009cd0 comp rx=0 tx=0).stop 2024-04-16T17:03:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.855+0000 7f49d5400700 1 -- 172.21.15.18:0/86865515 shutdown_connections 2024-04-16T17:03:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.855+0000 7f49d5400700 1 --2- 172.21.15.18:0/86865515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49d0109380 0x7f49d0109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.855+0000 7f49d5400700 1 --2- 172.21.15.18:0/86865515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49d010e460 0x7f49d0116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.855+0000 7f49d5400700 1 --2- 172.21.15.18:0/86865515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49d0109d30 0x7f49d010dd80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.855+0000 7f49d5400700 1 -- 172.21.15.18:0/86865515 >> 172.21.15.18:0/86865515 conn(0x7f49d00762c0 msgr2=0x7f49d0107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.855+0000 7f49d5400700 1 -- 172.21.15.18:0/86865515 shutdown_connections 2024-04-16T17:03:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.855+0000 7f49d5400700 1 -- 172.21.15.18:0/86865515 wait complete. 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.856+0000 7f49d5400700 1 Processor -- start 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.856+0000 7f49d5400700 1 -- start start 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.856+0000 7f49d5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49d0109380 0x7f49d01a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.856+0000 7f49d5400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49d0109d30 0x7f49d01a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49d5400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49d010e460 0x7f49d019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49d5400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0119490 con 0x7f49d0109380 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49d5400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0119310 con 0x7f49d0109d30 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49d5400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0119610 con 0x7f49d010e460 2024-04-16T17:03:34.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49ce800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49d0109380 0x7f49d01a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49ce800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49d0109380 0x7f49d01a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60954/0 (socket says 172.21.15.18:60954) 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49ce800700 1 -- 172.21.15.18:0/4017379225 learned_addr learned my addr 172.21.15.18:0/4017379225 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49cde00700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49d0109d30 0x7f49d01a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49cf200700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49d010e460 0x7f49d019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49cde00700 1 -- 172.21.15.18:0/4017379225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49d010e460 msgr2=0x7f49d019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49cde00700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49d010e460 0x7f49d019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49cde00700 1 -- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49d0109380 msgr2=0x7f49d01a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49cde00700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49d0109380 0x7f49d01a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.857+0000 7f49cde00700 1 -- 172.21.15.18:0/4017379225 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d019fff0 con 0x7f49d0109d30 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.858+0000 7f49cde00700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49d0109d30 0x7f49d01a5b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0098e0 tx=0x7f49bc00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.858+0000 7f49c7400700 1 -- 172.21.15.18:0/4017379225 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc027070 con 0x7f49d0109d30 2024-04-16T17:03:34.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.858+0000 7f49c7400700 1 -- 172.21.15.18:0/4017379225 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49bc003680 con 0x7f49d0109d30 2024-04-16T17:03:34.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.858+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d01a0280 con 0x7f49d0109d30 2024-04-16T17:03:34.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.858+0000 7f49c7400700 1 -- 172.21.15.18:0/4017379225 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0173f0 con 0x7f49d0109d30 2024-04-16T17:03:34.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.858+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01ac4a0 con 0x7f49d0109d30 2024-04-16T17:03:34.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.860+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d010a860 con 0x7f49d0109d30 2024-04-16T17:03:34.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.862+0000 7f49c7400700 1 -- 172.21.15.18:0/4017379225 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f49bc02a070 con 0x7f49d0109d30 2024-04-16T17:03:34.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.863+0000 7f49c7400700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49b40a3c40 0x7f49b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:34.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.863+0000 7f49c7400700 1 -- 172.21.15.18:0/4017379225 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f49bc0d1640 con 0x7f49d0109d30 2024-04-16T17:03:34.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.863+0000 7f49ce800700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49b40a3c40 0x7f49b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:34.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.864+0000 7f49ce800700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49b40a3c40 0x7f49b40a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009d60 tx=0x7f49c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:34.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:34.865+0000 7f49c7400700 1 -- 172.21.15.18:0/4017379225 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49bc031080 con 0x7f49d0109d30 2024-04-16T17:03:35.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.189+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f49d01a0cf0 con 0x7f49d0109d30 2024-04-16T17:03:35.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.190+0000 7f49c7400700 1 -- 172.21.15.18:0/4017379225 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1495 (secure 0 0 0) 0x7f49bc013070 con 0x7f49d0109d30 2024-04-16T17:03:35.192 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:35.192 INFO:teuthology.orchestra.run.smithi018.stdout:{"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":540,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":12,"num_up_osds":12,"osd_up_since":1713286824,"num_in_osds":12,"osd_in_since":1713286804,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":658667,"bytes_used":387391488,"bytes_avail":1151553036288,"bytes_total":1151940427776,"read_bytes_sec":3565,"write_bytes_sec":345,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":24,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24511},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24523},{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24536},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24547},{"filesystem_id":1,"rank":1,"name":"g","status":"up:active","gid":24566},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24596}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.18:8443/"}},"servicemap":{"epoch":29,"modified":"2024-04-16T17:03:06.925609+0000","services":{}},"progress_events":{}} 2024-04-16T17:03:35.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49b40a3c40 msgr2=0x7f49b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:35.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49b40a3c40 0x7f49b40a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009d60 tx=0x7f49c0009260 comp rx=0 tx=0).stop 2024-04-16T17:03:35.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49d0109d30 msgr2=0x7f49d01a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:35.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49d0109d30 0x7f49d01a5b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0098e0 tx=0x7f49bc00fa00 comp rx=0 tx=0).stop 2024-04-16T17:03:35.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 shutdown_connections 2024-04-16T17:03:35.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49b40a3c40 0x7f49b40a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49d010e460 0x7f49d019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49d0109380 0x7f49d01a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 --2- 172.21.15.18:0/4017379225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49d0109d30 0x7f49d01a5b80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 >> 172.21.15.18:0/4017379225 conn(0x7f49d00762c0 msgr2=0x7f49d0076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:35.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.193+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 shutdown_connections 2024-04-16T17:03:35.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:35.194+0000 7f49d5400700 1 -- 172.21.15.18:0/4017379225 wait complete. 2024-04-16T17:03:36.021 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-04-16T17:03:36.022 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:03:36.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:36 smithi022 ceph-mon[36023]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 345 B/s wr, 7 op/s 2024-04-16T17:03:36.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:36 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4017379225' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-16T17:03:36.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:36 smithi018 ceph-mon[31776]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 345 B/s wr, 7 op/s 2024-04-16T17:03:36.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:36 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4017379225' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-16T17:03:36.492 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:36.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:36 smithi043 ceph-mon[36901]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 345 B/s wr, 7 op/s 2024-04-16T17:03:36.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:36 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4017379225' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-16T17:03:38.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:38 smithi018 ceph-mon[31776]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-16T17:03:38.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:38 smithi022 ceph-mon[36023]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-16T17:03:38.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:38 smithi043 ceph-mon[36901]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 -- 172.21.15.18:0/3324082358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 msgr2=0x7f91bc105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 --2- 172.21.15.18:0/3324082358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 0x7f91bc105340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91b00099c0 tx=0x7f91b0009cd0 comp rx=0 tx=0).stop 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 -- 172.21.15.18:0/3324082358 shutdown_connections 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 --2- 172.21.15.18:0/3324082358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91bc10a040 0x7f91bc111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 --2- 172.21.15.18:0/3324082358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 0x7f91bc105340 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 --2- 172.21.15.18:0/3324082358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91bc105910 0x7f91bc109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 -- 172.21.15.18:0/3324082358 >> 172.21.15.18:0/3324082358 conn(0x7f91bc1007d0 msgr2=0x7f91bc102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 -- 172.21.15.18:0/3324082358 shutdown_connections 2024-04-16T17:03:38.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.741+0000 7f91c4800700 1 -- 172.21.15.18:0/3324082358 wait complete. 2024-04-16T17:03:38.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.742+0000 7f91c4800700 1 Processor -- start 2024-04-16T17:03:38.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.742+0000 7f91c4800700 1 -- start start 2024-04-16T17:03:38.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c4800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 0x7f91bc19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:38.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c4800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91bc105910 0x7f91bc19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:38.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c4800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91bc10a040 0x7f91bc19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:38.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c4800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1150a0 con 0x7f91bc105910 2024-04-16T17:03:38.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c4800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc114f20 con 0x7f91bc10a040 2024-04-16T17:03:38.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c4800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc115220 con 0x7f91bc104f60 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c2800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91bc10a040 0x7f91bc19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c1e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 0x7f91bc19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91bc105910 0x7f91bc19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c2800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91bc10a040 0x7f91bc19fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50492/0 (socket says 172.21.15.18:50492) 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.743+0000 7f91c2800700 1 -- 172.21.15.18:0/3833979778 learned_addr learned my addr 172.21.15.18:0/3833979778 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.744+0000 7f91c1e00700 1 -- 172.21.15.18:0/3833979778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91bc10a040 msgr2=0x7f91bc19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.744+0000 7f91c1e00700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91bc10a040 0x7f91bc19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.744+0000 7f91c1e00700 1 -- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91bc105910 msgr2=0x7f91bc19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.744+0000 7f91c1e00700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91bc105910 0x7f91bc19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.744+0000 7f91c1e00700 1 -- 172.21.15.18:0/3833979778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc1a0520 con 0x7f91bc104f60 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.744+0000 7f91c2800700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91bc10a040 0x7f91bc19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:38.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.744+0000 7f91c1e00700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 0x7f91bc19b510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91b00098e0 tx=0x7f91b0003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:38.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.745+0000 7f91b6a00700 1 -- 172.21.15.18:0/3833979778 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0003680 con 0x7f91bc104f60 2024-04-16T17:03:38.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.745+0000 7f91b6a00700 1 -- 172.21.15.18:0/3833979778 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b00055a0 con 0x7f91bc104f60 2024-04-16T17:03:38.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.745+0000 7f91b6a00700 1 -- 172.21.15.18:0/3833979778 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b000fc90 con 0x7f91bc104f60 2024-04-16T17:03:38.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.745+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc1a07b0 con 0x7f91bc104f60 2024-04-16T17:03:38.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.745+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc1a8090 con 0x7f91bc104f60 2024-04-16T17:03:38.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.746+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc0697e0 con 0x7f91bc104f60 2024-04-16T17:03:38.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.749+0000 7f91b6a00700 1 -- 172.21.15.18:0/3833979778 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f91b000f460 con 0x7f91bc104f60 2024-04-16T17:03:38.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.751+0000 7f91b6a00700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f91a40a3e60 0x7f91a40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:38.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.751+0000 7f91b6a00700 1 -- 172.21.15.18:0/3833979778 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f91b00d1520 con 0x7f91bc104f60 2024-04-16T17:03:38.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.751+0000 7f91c1400700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f91a40a3e60 0x7f91a40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:38.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.752+0000 7f91c1400700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f91a40a3e60 0x7f91a40a6320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f91bc101c40 tx=0x7f91ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:38.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.756+0000 7f91b6a00700 1 -- 172.21.15.18:0/3833979778 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91b0013070 con 0x7f91bc104f60 2024-04-16T17:03:38.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:38.991+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91bc19c870 con 0x7f91a40a3e60 2024-04-16T17:03:39.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.020+0000 7f91b6a00700 1 -- 172.21.15.18:0/3833979778 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340773 (secure 0 0 0) 0x7f91bc19c870 con 0x7f91a40a3e60 2024-04-16T17:03:39.026 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:39.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f91a40a3e60 msgr2=0x7f91a40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:39.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f91a40a3e60 0x7f91a40a6320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f91bc101c40 tx=0x7f91ac009260 comp rx=0 tx=0).stop 2024-04-16T17:03:39.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 msgr2=0x7f91bc19b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:39.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 0x7f91bc19b510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91b00098e0 tx=0x7f91b0003a90 comp rx=0 tx=0).stop 2024-04-16T17:03:39.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 shutdown_connections 2024-04-16T17:03:39.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f91a40a3e60 0x7f91a40a6320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:39.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91bc10a040 0x7f91bc19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:39.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91bc104f60 0x7f91bc19b510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:39.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 --2- 172.21.15.18:0/3833979778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91bc105910 0x7f91bc19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:39.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.030+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 >> 172.21.15.18:0/3833979778 conn(0x7f91bc1007d0 msgr2=0x7f91bc1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:39.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.031+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 shutdown_connections 2024-04-16T17:03:39.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:39.031+0000 7f91c4800700 1 -- 172.21.15.18:0/3833979778 wait complete. 2024-04-16T17:03:39.034 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:03:39.884 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-04-16T17:03:38.940220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658667,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":748,"num_read_kb":499,"num_write":352,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378312,"kb_used_data":12440,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563512,"statfs":{"total":1151940427776,"available":1151553036288,"internally_reserved":0,"allocated":12738560,"data_stored":8189375,"data_compressed":432159,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":41,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.893754"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.054050+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:03:27.054050+0000","last_peered":"2024-04-16T17:03:27.054050+0000","last_clean":"2024-04-16T17:03:27.054050+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:03:27.054050+0000","last_undegraded":"2024-04-16T17:03:27.054050+0000","last_fullsized":"2024-04-16T17:03:27.054050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.450032+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:03:27.450032+0000","last_peered":"2024-04-16T17:03:27.450032+0000","last_clean":"2024-04-16T17:03:27.450032+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:03:27.450032+0000","last_undegraded":"2024-04-16T17:03:27.450032+0000","last_fullsized":"2024-04-16T17:03:27.450032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449738+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:03:27.449738+0000","last_peered":"2024-04-16T17:03:27.449738+0000","last_clean":"2024-04-16T17:03:27.449738+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:03:27.449738+0000","last_undegraded":"2024-04-16T17:03:27.449738+0000","last_fullsized":"2024-04-16T17:03:27.449738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.270996+0000","last_change":"2024-04-16T17:02:26.318719+0000","last_active":"2024-04-16T17:03:27.270996+0000","last_peered":"2024-04-16T17:03:27.270996+0000","last_clean":"2024-04-16T17:03:27.270996+0000","last_became_active":"2024-04-16T17:02:26.317743+0000","last_became_peered":"2024-04-16T17:02:26.317743+0000","last_unstale":"2024-04-16T17:03:27.270996+0000","last_undegraded":"2024-04-16T17:03:27.270996+0000","last_fullsized":"2024-04-16T17:03:27.270996+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:26.827859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050853+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:03:27.050853+0000","last_peered":"2024-04-16T17:03:27.050853+0000","last_clean":"2024-04-16T17:03:27.050853+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:03:27.050853+0000","last_undegraded":"2024-04-16T17:03:27.050853+0000","last_fullsized":"2024-04-16T17:03:27.050853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051435+0000","last_change":"2024-04-16T17:02:26.315910+0000","last_active":"2024-04-16T17:03:27.051435+0000","last_peered":"2024-04-16T17:03:27.051435+0000","last_clean":"2024-04-16T17:03:27.051435+0000","last_became_active":"2024-04-16T17:02:26.315493+0000","last_became_peered":"2024-04-16T17:02:26.315493+0000","last_unstale":"2024-04-16T17:03:27.051435+0000","last_undegraded":"2024-04-16T17:03:27.051435+0000","last_fullsized":"2024-04-16T17:03:27.051435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:27:48.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267060+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:03:27.267060+0000","last_peered":"2024-04-16T17:03:27.267060+0000","last_clean":"2024-04-16T17:03:27.267060+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:03:27.267060+0000","last_undegraded":"2024-04-16T17:03:27.267060+0000","last_fullsized":"2024-04-16T17:03:27.267060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050577+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:03:27.050577+0000","last_peered":"2024-04-16T17:03:27.050577+0000","last_clean":"2024-04-16T17:03:27.050577+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:03:27.050577+0000","last_undegraded":"2024-04-16T17:03:27.050577+0000","last_fullsized":"2024-04-16T17:03:27.050577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267199+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:03:27.267199+0000","last_peered":"2024-04-16T17:03:27.267199+0000","last_clean":"2024-04-16T17:03:27.267199+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:03:27.267199+0000","last_undegraded":"2024-04-16T17:03:27.267199+0000","last_fullsized":"2024-04-16T17:03:27.267199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052004+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:03:27.052004+0000","last_peered":"2024-04-16T17:03:27.052004+0000","last_clean":"2024-04-16T17:03:27.052004+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:03:27.052004+0000","last_undegraded":"2024-04-16T17:03:27.052004+0000","last_fullsized":"2024-04-16T17:03:27.052004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267455+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:03:27.267455+0000","last_peered":"2024-04-16T17:03:27.267455+0000","last_clean":"2024-04-16T17:03:27.267455+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:03:27.267455+0000","last_undegraded":"2024-04-16T17:03:27.267455+0000","last_fullsized":"2024-04-16T17:03:27.267455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051237+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:03:27.051237+0000","last_peered":"2024-04-16T17:03:27.051237+0000","last_clean":"2024-04-16T17:03:27.051237+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:03:27.051237+0000","last_undegraded":"2024-04-16T17:03:27.051237+0000","last_fullsized":"2024-04-16T17:03:27.051237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268482+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:03:27.268482+0000","last_peered":"2024-04-16T17:03:27.268482+0000","last_clean":"2024-04-16T17:03:27.268482+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:03:27.268482+0000","last_undegraded":"2024-04-16T17:03:27.268482+0000","last_fullsized":"2024-04-16T17:03:27.268482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448788+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:03:27.448788+0000","last_peered":"2024-04-16T17:03:27.448788+0000","last_clean":"2024-04-16T17:03:27.448788+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:03:27.448788+0000","last_undegraded":"2024-04-16T17:03:27.448788+0000","last_fullsized":"2024-04-16T17:03:27.448788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448708+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:03:27.448708+0000","last_peered":"2024-04-16T17:03:27.448708+0000","last_clean":"2024-04-16T17:03:27.448708+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:03:27.448708+0000","last_undegraded":"2024-04-16T17:03:27.448708+0000","last_fullsized":"2024-04-16T17:03:27.448708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266225+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:03:27.266225+0000","last_peered":"2024-04-16T17:03:27.266225+0000","last_clean":"2024-04-16T17:03:27.266225+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:03:27.266225+0000","last_undegraded":"2024-04-16T17:03:27.266225+0000","last_fullsized":"2024-04-16T17:03:27.266225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053979+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:03:27.053979+0000","last_peered":"2024-04-16T17:03:27.053979+0000","last_clean":"2024-04-16T17:03:27.053979+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:03:27.053979+0000","last_undegraded":"2024-04-16T17:03:27.053979+0000","last_fullsized":"2024-04-16T17:03:27.053979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273453+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:03:27.273453+0000","last_peered":"2024-04-16T17:03:27.273453+0000","last_clean":"2024-04-16T17:03:27.273453+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:03:27.273453+0000","last_undegraded":"2024-04-16T17:03:27.273453+0000","last_fullsized":"2024-04-16T17:03:27.273453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267524+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:03:27.267524+0000","last_peered":"2024-04-16T17:03:27.267524+0000","last_clean":"2024-04-16T17:03:27.267524+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:03:27.267524+0000","last_undegraded":"2024-04-16T17:03:27.267524+0000","last_fullsized":"2024-04-16T17:03:27.267524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267227+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:03:27.267227+0000","last_peered":"2024-04-16T17:03:27.267227+0000","last_clean":"2024-04-16T17:03:27.267227+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:03:27.267227+0000","last_undegraded":"2024-04-16T17:03:27.267227+0000","last_fullsized":"2024-04-16T17:03:27.267227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.274411+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:03:27.274411+0000","last_peered":"2024-04-16T17:03:27.274411+0000","last_clean":"2024-04-16T17:03:27.274411+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:03:27.274411+0000","last_undegraded":"2024-04-16T17:03:27.274411+0000","last_fullsized":"2024-04-16T17:03:27.274411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273261+0000","last_change":"2024-04-16T17:02:26.318546+0000","last_active":"2024-04-16T17:03:27.273261+0000","last_peered":"2024-04-16T17:03:27.273261+0000","last_clean":"2024-04-16T17:03:27.273261+0000","last_became_active":"2024-04-16T17:02:26.317794+0000","last_became_peered":"2024-04-16T17:02:26.317794+0000","last_unstale":"2024-04-16T17:03:27.273261+0000","last_undegraded":"2024-04-16T17:03:27.273261+0000","last_fullsized":"2024-04-16T17:03:27.273261+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:48.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050304+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:03:27.050304+0000","last_peered":"2024-04-16T17:03:27.050304+0000","last_clean":"2024-04-16T17:03:27.050304+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:03:27.050304+0000","last_undegraded":"2024-04-16T17:03:27.050304+0000","last_fullsized":"2024-04-16T17:03:27.050304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052748+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:03:27.052748+0000","last_peered":"2024-04-16T17:03:27.052748+0000","last_clean":"2024-04-16T17:03:27.052748+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:03:27.052748+0000","last_undegraded":"2024-04-16T17:03:27.052748+0000","last_fullsized":"2024-04-16T17:03:27.052748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449960+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:03:27.449960+0000","last_peered":"2024-04-16T17:03:27.449960+0000","last_clean":"2024-04-16T17:03:27.449960+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:03:27.449960+0000","last_undegraded":"2024-04-16T17:03:27.449960+0000","last_fullsized":"2024-04-16T17:03:27.449960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.271585+0000","last_change":"2024-04-16T17:03:27.271585+0000","last_active":"2024-04-16T17:03:27.271585+0000","last_peered":"2024-04-16T17:03:27.271585+0000","last_clean":"2024-04-16T17:03:27.271585+0000","last_became_active":"2024-04-16T17:03:27.271251+0000","last_became_peered":"2024-04-16T17:03:27.271251+0000","last_unstale":"2024-04-16T17:03:27.271585+0000","last_undegraded":"2024-04-16T17:03:27.271585+0000","last_fullsized":"2024-04-16T17:03:27.271585+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:51.217466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051925+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:03:27.051925+0000","last_peered":"2024-04-16T17:03:27.051925+0000","last_clean":"2024-04-16T17:03:27.051925+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:03:27.051925+0000","last_undegraded":"2024-04-16T17:03:27.051925+0000","last_fullsized":"2024-04-16T17:03:27.051925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266687+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:03:27.266687+0000","last_peered":"2024-04-16T17:03:27.266687+0000","last_clean":"2024-04-16T17:03:27.266687+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:03:27.266687+0000","last_undegraded":"2024-04-16T17:03:27.266687+0000","last_fullsized":"2024-04-16T17:03:27.266687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050850+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:03:27.050850+0000","last_peered":"2024-04-16T17:03:27.050850+0000","last_clean":"2024-04-16T17:03:27.050850+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:03:27.050850+0000","last_undegraded":"2024-04-16T17:03:27.050850+0000","last_fullsized":"2024-04-16T17:03:27.050850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051204+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:03:27.051204+0000","last_peered":"2024-04-16T17:03:27.051204+0000","last_clean":"2024-04-16T17:03:27.051204+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:03:27.051204+0000","last_undegraded":"2024-04-16T17:03:27.051204+0000","last_fullsized":"2024-04-16T17:03:27.051204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048934+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:03:27.048934+0000","last_peered":"2024-04-16T17:03:27.048934+0000","last_clean":"2024-04-16T17:03:27.048934+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:03:27.048934+0000","last_undegraded":"2024-04-16T17:03:27.048934+0000","last_fullsized":"2024-04-16T17:03:27.048934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267582+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:03:27.267582+0000","last_peered":"2024-04-16T17:03:27.267582+0000","last_clean":"2024-04-16T17:03:27.267582+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:03:27.267582+0000","last_undegraded":"2024-04-16T17:03:27.267582+0000","last_fullsized":"2024-04-16T17:03:27.267582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:36.410754+0000","last_change":"2024-04-16T17:02:26.315434+0000","last_active":"2024-04-16T17:03:36.410754+0000","last_peered":"2024-04-16T17:03:36.410754+0000","last_clean":"2024-04-16T17:03:36.410754+0000","last_became_active":"2024-04-16T17:02:26.310169+0000","last_became_peered":"2024-04-16T17:02:26.310169+0000","last_unstale":"2024-04-16T17:03:36.410754+0000","last_undegraded":"2024-04-16T17:03:36.410754+0000","last_fullsized":"2024-04-16T17:03:36.410754+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:51.033415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050694+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:03:27.050694+0000","last_peered":"2024-04-16T17:03:27.050694+0000","last_clean":"2024-04-16T17:03:27.050694+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:03:27.050694+0000","last_undegraded":"2024-04-16T17:03:27.050694+0000","last_fullsized":"2024-04-16T17:03:27.050694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267268+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:03:27.267268+0000","last_peered":"2024-04-16T17:03:27.267268+0000","last_clean":"2024-04-16T17:03:27.267268+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:03:27.267268+0000","last_undegraded":"2024-04-16T17:03:27.267268+0000","last_fullsized":"2024-04-16T17:03:27.267268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051285+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:03:27.051285+0000","last_peered":"2024-04-16T17:03:27.051285+0000","last_clean":"2024-04-16T17:03:27.051285+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:03:27.051285+0000","last_undegraded":"2024-04-16T17:03:27.051285+0000","last_fullsized":"2024-04-16T17:03:27.051285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.270459+0000","last_change":"2024-04-16T17:02:26.321763+0000","last_active":"2024-04-16T17:03:27.270459+0000","last_peered":"2024-04-16T17:03:27.270459+0000","last_clean":"2024-04-16T17:03:27.270459+0000","last_became_active":"2024-04-16T17:02:26.321438+0000","last_became_peered":"2024-04-16T17:02:26.321438+0000","last_unstale":"2024-04-16T17:03:27.270459+0000","last_undegraded":"2024-04-16T17:03:27.270459+0000","last_fullsized":"2024-04-16T17:03:27.270459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:47.231035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.451719+0000","last_change":"2024-04-16T17:03:27.451220+0000","last_active":"2024-04-16T17:03:27.451719+0000","last_peered":"2024-04-16T17:03:27.451719+0000","last_clean":"2024-04-16T17:03:27.451719+0000","last_became_active":"2024-04-16T17:03:27.450680+0000","last_became_peered":"2024-04-16T17:03:27.450680+0000","last_unstale":"2024-04-16T17:03:27.451719+0000","last_undegraded":"2024-04-16T17:03:27.451719+0000","last_fullsized":"2024-04-16T17:03:27.451719+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:23.270332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267430+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:03:27.267430+0000","last_peered":"2024-04-16T17:03:27.267430+0000","last_clean":"2024-04-16T17:03:27.267430+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:03:27.267430+0000","last_undegraded":"2024-04-16T17:03:27.267430+0000","last_fullsized":"2024-04-16T17:03:27.267430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050234+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:03:27.050234+0000","last_peered":"2024-04-16T17:03:27.050234+0000","last_clean":"2024-04-16T17:03:27.050234+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:03:27.050234+0000","last_undegraded":"2024-04-16T17:03:27.050234+0000","last_fullsized":"2024-04-16T17:03:27.050234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449714+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:03:27.449714+0000","last_peered":"2024-04-16T17:03:27.449714+0000","last_clean":"2024-04-16T17:03:27.449714+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:03:27.449714+0000","last_undegraded":"2024-04-16T17:03:27.449714+0000","last_fullsized":"2024-04-16T17:03:27.449714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048433+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:03:27.048433+0000","last_peered":"2024-04-16T17:03:27.048433+0000","last_clean":"2024-04-16T17:03:27.048433+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:03:27.048433+0000","last_undegraded":"2024-04-16T17:03:27.048433+0000","last_fullsized":"2024-04-16T17:03:27.048433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273462+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:03:27.273462+0000","last_peered":"2024-04-16T17:03:27.273462+0000","last_clean":"2024-04-16T17:03:27.273462+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:03:27.273462+0000","last_undegraded":"2024-04-16T17:03:27.273462+0000","last_fullsized":"2024-04-16T17:03:27.273462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051478+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:03:27.051478+0000","last_peered":"2024-04-16T17:03:27.051478+0000","last_clean":"2024-04-16T17:03:27.051478+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:03:27.051478+0000","last_undegraded":"2024-04-16T17:03:27.051478+0000","last_fullsized":"2024-04-16T17:03:27.051478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":133,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:36.410163+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:03:36.410163+0000","last_peered":"2024-04-16T17:03:36.410163+0000","last_clean":"2024-04-16T17:03:36.410163+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:03:36.410163+0000","last_undegraded":"2024-04-16T17:03:36.410163+0000","last_fullsized":"2024-04-16T17:03:36.410163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267330+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:03:27.267330+0000","last_peered":"2024-04-16T17:03:27.267330+0000","last_clean":"2024-04-16T17:03:27.267330+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:03:27.267330+0000","last_undegraded":"2024-04-16T17:03:27.267330+0000","last_fullsized":"2024-04-16T17:03:27.267330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.269250+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:03:27.269250+0000","last_peered":"2024-04-16T17:03:27.269250+0000","last_clean":"2024-04-16T17:03:27.269250+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:03:27.269250+0000","last_undegraded":"2024-04-16T17:03:27.269250+0000","last_fullsized":"2024-04-16T17:03:27.269250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267661+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:03:27.267661+0000","last_peered":"2024-04-16T17:03:27.267661+0000","last_clean":"2024-04-16T17:03:27.267661+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:03:27.267661+0000","last_undegraded":"2024-04-16T17:03:27.267661+0000","last_fullsized":"2024-04-16T17:03:27.267661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449950+0000","last_change":"2024-04-16T17:02:26.752275+0000","last_active":"2024-04-16T17:03:27.449950+0000","last_peered":"2024-04-16T17:03:27.449950+0000","last_clean":"2024-04-16T17:03:27.449950+0000","last_became_active":"2024-04-16T17:02:26.751729+0000","last_became_peered":"2024-04-16T17:02:26.751729+0000","last_unstale":"2024-04-16T17:03:27.449950+0000","last_undegraded":"2024-04-16T17:03:27.449950+0000","last_fullsized":"2024-04-16T17:03:27.449950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:41:35.777691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051500+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:03:27.051500+0000","last_peered":"2024-04-16T17:03:27.051500+0000","last_clean":"2024-04-16T17:03:27.051500+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:03:27.051500+0000","last_undegraded":"2024-04-16T17:03:27.051500+0000","last_fullsized":"2024-04-16T17:03:27.051500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048902+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:03:27.048902+0000","last_peered":"2024-04-16T17:03:27.048902+0000","last_clean":"2024-04-16T17:03:27.048902+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:03:27.048902+0000","last_undegraded":"2024-04-16T17:03:27.048902+0000","last_fullsized":"2024-04-16T17:03:27.048902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266867+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:03:27.266867+0000","last_peered":"2024-04-16T17:03:27.266867+0000","last_clean":"2024-04-16T17:03:27.266867+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:03:27.266867+0000","last_undegraded":"2024-04-16T17:03:27.266867+0000","last_fullsized":"2024-04-16T17:03:27.266867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.055021+0000","last_change":"2024-04-16T17:03:27.054627+0000","last_active":"2024-04-16T17:03:27.055021+0000","last_peered":"2024-04-16T17:03:27.055021+0000","last_clean":"2024-04-16T17:03:27.055021+0000","last_became_active":"2024-04-16T17:03:27.054326+0000","last_became_peered":"2024-04-16T17:03:27.054326+0000","last_unstale":"2024-04-16T17:03:27.055021+0000","last_undegraded":"2024-04-16T17:03:27.055021+0000","last_fullsized":"2024-04-16T17:03:27.055021+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:43.535958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":275,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:37.411277+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:03:37.411277+0000","last_peered":"2024-04-16T17:03:37.411277+0000","last_clean":"2024-04-16T17:03:37.411277+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:03:37.411277+0000","last_undegraded":"2024-04-16T17:03:37.411277+0000","last_fullsized":"2024-04-16T17:03:37.411277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64455,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":208,"num_read_kb":168,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051676+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:03:27.051676+0000","last_peered":"2024-04-16T17:03:27.051676+0000","last_clean":"2024-04-16T17:03:27.051676+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:03:27.051676+0000","last_undegraded":"2024-04-16T17:03:27.051676+0000","last_fullsized":"2024-04-16T17:03:27.051676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.049199+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:03:27.049199+0000","last_peered":"2024-04-16T17:03:27.049199+0000","last_clean":"2024-04-16T17:03:27.049199+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:03:27.049199+0000","last_undegraded":"2024-04-16T17:03:27.049199+0000","last_fullsized":"2024-04-16T17:03:27.049199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267266+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:03:27.267266+0000","last_peered":"2024-04-16T17:03:27.267266+0000","last_clean":"2024-04-16T17:03:27.267266+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:03:27.267266+0000","last_undegraded":"2024-04-16T17:03:27.267266+0000","last_fullsized":"2024-04-16T17:03:27.267266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267852+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:03:27.267852+0000","last_peered":"2024-04-16T17:03:27.267852+0000","last_clean":"2024-04-16T17:03:27.267852+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:03:27.267852+0000","last_undegraded":"2024-04-16T17:03:27.267852+0000","last_fullsized":"2024-04-16T17:03:27.267852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053205+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:03:27.053205+0000","last_peered":"2024-04-16T17:03:27.053205+0000","last_clean":"2024-04-16T17:03:27.053205+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:03:27.053205+0000","last_undegraded":"2024-04-16T17:03:27.053205+0000","last_fullsized":"2024-04-16T17:03:27.053205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048636+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:03:27.048636+0000","last_peered":"2024-04-16T17:03:27.048636+0000","last_clean":"2024-04-16T17:03:27.048636+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:03:27.048636+0000","last_undegraded":"2024-04-16T17:03:27.048636+0000","last_fullsized":"2024-04-16T17:03:27.048636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267640+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:03:27.267640+0000","last_peered":"2024-04-16T17:03:27.267640+0000","last_clean":"2024-04-16T17:03:27.267640+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:03:27.267640+0000","last_undegraded":"2024-04-16T17:03:27.267640+0000","last_fullsized":"2024-04-16T17:03:27.267640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273890+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:03:27.273890+0000","last_peered":"2024-04-16T17:03:27.273890+0000","last_clean":"2024-04-16T17:03:27.273890+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:03:27.273890+0000","last_undegraded":"2024-04-16T17:03:27.273890+0000","last_fullsized":"2024-04-16T17:03:27.273890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:36.393351+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:03:36.393351+0000","last_peered":"2024-04-16T17:03:36.393351+0000","last_clean":"2024-04-16T17:03:36.393351+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:03:36.393351+0000","last_undegraded":"2024-04-16T17:03:36.393351+0000","last_fullsized":"2024-04-16T17:03:36.393351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267027+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:03:27.267027+0000","last_peered":"2024-04-16T17:03:27.267027+0000","last_clean":"2024-04-16T17:03:27.267027+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:03:27.267027+0000","last_undegraded":"2024-04-16T17:03:27.267027+0000","last_fullsized":"2024-04-16T17:03:27.267027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267963+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:03:27.267963+0000","last_peered":"2024-04-16T17:03:27.267963+0000","last_clean":"2024-04-16T17:03:27.267963+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:03:27.267963+0000","last_undegraded":"2024-04-16T17:03:27.267963+0000","last_fullsized":"2024-04-16T17:03:27.267963+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050660+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:03:27.050660+0000","last_peered":"2024-04-16T17:03:27.050660+0000","last_clean":"2024-04-16T17:03:27.050660+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:03:27.050660+0000","last_undegraded":"2024-04-16T17:03:27.050660+0000","last_fullsized":"2024-04-16T17:03:27.050660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048626+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:03:27.048626+0000","last_peered":"2024-04-16T17:03:27.048626+0000","last_clean":"2024-04-16T17:03:27.048626+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:03:27.048626+0000","last_undegraded":"2024-04-16T17:03:27.048626+0000","last_fullsized":"2024-04-16T17:03:27.048626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048451+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:03:27.048451+0000","last_peered":"2024-04-16T17:03:27.048451+0000","last_clean":"2024-04-16T17:03:27.048451+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:03:27.048451+0000","last_undegraded":"2024-04-16T17:03:27.048451+0000","last_fullsized":"2024-04-16T17:03:27.048451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053451+0000","last_change":"2024-04-16T17:02:26.320405+0000","last_active":"2024-04-16T17:03:27.053451+0000","last_peered":"2024-04-16T17:03:27.053451+0000","last_clean":"2024-04-16T17:03:27.053451+0000","last_became_active":"2024-04-16T17:02:26.307921+0000","last_became_peered":"2024-04-16T17:02:26.307921+0000","last_unstale":"2024-04-16T17:03:27.053451+0000","last_undegraded":"2024-04-16T17:03:27.053451+0000","last_fullsized":"2024-04-16T17:03:27.053451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:19.156260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053407+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:03:27.053407+0000","last_peered":"2024-04-16T17:03:27.053407+0000","last_clean":"2024-04-16T17:03:27.053407+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:03:27.053407+0000","last_undegraded":"2024-04-16T17:03:27.053407+0000","last_fullsized":"2024-04-16T17:03:27.053407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050931+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:03:27.050931+0000","last_peered":"2024-04-16T17:03:27.050931+0000","last_clean":"2024-04-16T17:03:27.050931+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:03:27.050931+0000","last_undegraded":"2024-04-16T17:03:27.050931+0000","last_fullsized":"2024-04-16T17:03:27.050931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.270987+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:03:27.270987+0000","last_peered":"2024-04-16T17:03:27.270987+0000","last_clean":"2024-04-16T17:03:27.270987+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:03:27.270987+0000","last_undegraded":"2024-04-16T17:03:27.270987+0000","last_fullsized":"2024-04-16T17:03:27.270987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051645+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:03:27.051645+0000","last_peered":"2024-04-16T17:03:27.051645+0000","last_clean":"2024-04-16T17:03:27.051645+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:03:27.051645+0000","last_undegraded":"2024-04-16T17:03:27.051645+0000","last_fullsized":"2024-04-16T17:03:27.051645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268420+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:03:27.268420+0000","last_peered":"2024-04-16T17:03:27.268420+0000","last_clean":"2024-04-16T17:03:27.268420+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:03:27.268420+0000","last_undegraded":"2024-04-16T17:03:27.268420+0000","last_fullsized":"2024-04-16T17:03:27.268420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267443+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:03:27.267443+0000","last_peered":"2024-04-16T17:03:27.267443+0000","last_clean":"2024-04-16T17:03:27.267443+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:03:27.267443+0000","last_undegraded":"2024-04-16T17:03:27.267443+0000","last_fullsized":"2024-04-16T17:03:27.267443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051006+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:03:27.051006+0000","last_peered":"2024-04-16T17:03:27.051006+0000","last_clean":"2024-04-16T17:03:27.051006+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:03:27.051006+0000","last_undegraded":"2024-04-16T17:03:27.051006+0000","last_fullsized":"2024-04-16T17:03:27.051006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268877+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:03:27.268877+0000","last_peered":"2024-04-16T17:03:27.268877+0000","last_clean":"2024-04-16T17:03:27.268877+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:03:27.268877+0000","last_undegraded":"2024-04-16T17:03:27.268877+0000","last_fullsized":"2024-04-16T17:03:27.268877+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048660+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:03:27.048660+0000","last_peered":"2024-04-16T17:03:27.048660+0000","last_clean":"2024-04-16T17:03:27.048660+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:03:27.048660+0000","last_undegraded":"2024-04-16T17:03:27.048660+0000","last_fullsized":"2024-04-16T17:03:27.048660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":133,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:34.387706+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:03:34.387706+0000","last_peered":"2024-04-16T17:03:34.387706+0000","last_clean":"2024-04-16T17:03:34.387706+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:03:34.387706+0000","last_undegraded":"2024-04-16T17:03:34.387706+0000","last_fullsized":"2024-04-16T17:03:34.387706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267230+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:03:27.267230+0000","last_peered":"2024-04-16T17:03:27.267230+0000","last_clean":"2024-04-16T17:03:27.267230+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:03:27.267230+0000","last_undegraded":"2024-04-16T17:03:27.267230+0000","last_fullsized":"2024-04-16T17:03:27.267230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051850+0000","last_change":"2024-04-16T17:02:26.322313+0000","last_active":"2024-04-16T17:03:27.051850+0000","last_peered":"2024-04-16T17:03:27.051850+0000","last_clean":"2024-04-16T17:03:27.051850+0000","last_became_active":"2024-04-16T17:02:26.320328+0000","last_became_peered":"2024-04-16T17:02:26.320328+0000","last_unstale":"2024-04-16T17:03:27.051850+0000","last_undegraded":"2024-04-16T17:03:27.051850+0000","last_fullsized":"2024-04-16T17:03:27.051850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:40.943183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051419+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:03:27.051419+0000","last_peered":"2024-04-16T17:03:27.051419+0000","last_clean":"2024-04-16T17:03:27.051419+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:03:27.051419+0000","last_undegraded":"2024-04-16T17:03:27.051419+0000","last_fullsized":"2024-04-16T17:03:27.051419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266776+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:03:27.266776+0000","last_peered":"2024-04-16T17:03:27.266776+0000","last_clean":"2024-04-16T17:03:27.266776+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:03:27.266776+0000","last_undegraded":"2024-04-16T17:03:27.266776+0000","last_fullsized":"2024-04-16T17:03:27.266776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048901+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:03:27.048901+0000","last_peered":"2024-04-16T17:03:27.048901+0000","last_clean":"2024-04-16T17:03:27.048901+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:03:27.048901+0000","last_undegraded":"2024-04-16T17:03:27.048901+0000","last_fullsized":"2024-04-16T17:03:27.048901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267262+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:03:27.267262+0000","last_peered":"2024-04-16T17:03:27.267262+0000","last_clean":"2024-04-16T17:03:27.267262+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:03:27.267262+0000","last_undegraded":"2024-04-16T17:03:27.267262+0000","last_fullsized":"2024-04-16T17:03:27.267262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052945+0000","last_change":"2024-04-16T17:02:26.312852+0000","last_active":"2024-04-16T17:03:27.052945+0000","last_peered":"2024-04-16T17:03:27.052945+0000","last_clean":"2024-04-16T17:03:27.052945+0000","last_became_active":"2024-04-16T17:02:26.308279+0000","last_became_peered":"2024-04-16T17:02:26.308279+0000","last_unstale":"2024-04-16T17:03:27.052945+0000","last_undegraded":"2024-04-16T17:03:27.052945+0000","last_fullsized":"2024-04-16T17:03:27.052945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:25.511349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053414+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:03:27.053414+0000","last_peered":"2024-04-16T17:03:27.053414+0000","last_clean":"2024-04-16T17:03:27.053414+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:03:27.053414+0000","last_undegraded":"2024-04-16T17:03:27.053414+0000","last_fullsized":"2024-04-16T17:03:27.053414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051892+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:03:27.051892+0000","last_peered":"2024-04-16T17:03:27.051892+0000","last_clean":"2024-04-16T17:03:27.051892+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:03:27.051892+0000","last_undegraded":"2024-04-16T17:03:27.051892+0000","last_fullsized":"2024-04-16T17:03:27.051892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268730+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:03:27.268730+0000","last_peered":"2024-04-16T17:03:27.268730+0000","last_clean":"2024-04-16T17:03:27.268730+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:03:27.268730+0000","last_undegraded":"2024-04-16T17:03:27.268730+0000","last_fullsized":"2024-04-16T17:03:27.268730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050407+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:03:27.050407+0000","last_peered":"2024-04-16T17:03:27.050407+0000","last_clean":"2024-04-16T17:03:27.050407+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:03:27.050407+0000","last_undegraded":"2024-04-16T17:03:27.050407+0000","last_fullsized":"2024-04-16T17:03:27.050407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273741+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:03:27.273741+0000","last_peered":"2024-04-16T17:03:27.273741+0000","last_clean":"2024-04-16T17:03:27.273741+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:03:27.273741+0000","last_undegraded":"2024-04-16T17:03:27.273741+0000","last_fullsized":"2024-04-16T17:03:27.273741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053724+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:03:27.053724+0000","last_peered":"2024-04-16T17:03:27.053724+0000","last_clean":"2024-04-16T17:03:27.053724+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:03:27.053724+0000","last_undegraded":"2024-04-16T17:03:27.053724+0000","last_fullsized":"2024-04-16T17:03:27.053724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266977+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:03:27.266977+0000","last_peered":"2024-04-16T17:03:27.266977+0000","last_clean":"2024-04-16T17:03:27.266977+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:03:27.266977+0000","last_undegraded":"2024-04-16T17:03:27.266977+0000","last_fullsized":"2024-04-16T17:03:27.266977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051034+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:03:27.051034+0000","last_peered":"2024-04-16T17:03:27.051034+0000","last_clean":"2024-04-16T17:03:27.051034+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:03:27.051034+0000","last_undegraded":"2024-04-16T17:03:27.051034+0000","last_fullsized":"2024-04-16T17:03:27.051034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.450365+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:03:27.450365+0000","last_peered":"2024-04-16T17:03:27.450365+0000","last_clean":"2024-04-16T17:03:27.450365+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:03:27.450365+0000","last_undegraded":"2024-04-16T17:03:27.450365+0000","last_fullsized":"2024-04-16T17:03:27.450365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050691+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:03:27.050691+0000","last_peered":"2024-04-16T17:03:27.050691+0000","last_clean":"2024-04-16T17:03:27.050691+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:03:27.050691+0000","last_undegraded":"2024-04-16T17:03:27.050691+0000","last_fullsized":"2024-04-16T17:03:27.050691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266305+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:03:27.266305+0000","last_peered":"2024-04-16T17:03:27.266305+0000","last_clean":"2024-04-16T17:03:27.266305+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:03:27.266305+0000","last_undegraded":"2024-04-16T17:03:27.266305+0000","last_fullsized":"2024-04-16T17:03:27.266305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050728+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:03:27.050728+0000","last_peered":"2024-04-16T17:03:27.050728+0000","last_clean":"2024-04-16T17:03:27.050728+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:03:27.050728+0000","last_undegraded":"2024-04-16T17:03:27.050728+0000","last_fullsized":"2024-04-16T17:03:27.050728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268760+0000","last_change":"2024-04-16T17:03:27.268760+0000","last_active":"2024-04-16T17:03:27.268760+0000","last_peered":"2024-04-16T17:03:27.268760+0000","last_clean":"2024-04-16T17:03:27.268760+0000","last_became_active":"2024-04-16T17:03:27.268209+0000","last_became_peered":"2024-04-16T17:03:27.268209+0000","last_unstale":"2024-04-16T17:03:27.268760+0000","last_undegraded":"2024-04-16T17:03:27.268760+0000","last_fullsized":"2024-04-16T17:03:27.268760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:24:57.457026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050876+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:03:27.050876+0000","last_peered":"2024-04-16T17:03:27.050876+0000","last_clean":"2024-04-16T17:03:27.050876+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:03:27.050876+0000","last_undegraded":"2024-04-16T17:03:27.050876+0000","last_fullsized":"2024-04-16T17:03:27.050876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053161+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:03:27.053161+0000","last_peered":"2024-04-16T17:03:27.053161+0000","last_clean":"2024-04-16T17:03:27.053161+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:03:27.053161+0000","last_undegraded":"2024-04-16T17:03:27.053161+0000","last_fullsized":"2024-04-16T17:03:27.053161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050922+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:03:27.050922+0000","last_peered":"2024-04-16T17:03:27.050922+0000","last_clean":"2024-04-16T17:03:27.050922+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:03:27.050922+0000","last_undegraded":"2024-04-16T17:03:27.050922+0000","last_fullsized":"2024-04-16T17:03:27.050922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267061+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:03:27.267061+0000","last_peered":"2024-04-16T17:03:27.267061+0000","last_clean":"2024-04-16T17:03:27.267061+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:03:27.267061+0000","last_undegraded":"2024-04-16T17:03:27.267061+0000","last_fullsized":"2024-04-16T17:03:27.267061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052332+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:03:27.052332+0000","last_peered":"2024-04-16T17:03:27.052332+0000","last_clean":"2024-04-16T17:03:27.052332+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:03:27.052332+0000","last_undegraded":"2024-04-16T17:03:27.052332+0000","last_fullsized":"2024-04-16T17:03:27.052332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050542+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:03:27.050542+0000","last_peered":"2024-04-16T17:03:27.050542+0000","last_clean":"2024-04-16T17:03:27.050542+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:03:27.050542+0000","last_undegraded":"2024-04-16T17:03:27.050542+0000","last_fullsized":"2024-04-16T17:03:27.050542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050871+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:03:27.050871+0000","last_peered":"2024-04-16T17:03:27.050871+0000","last_clean":"2024-04-16T17:03:27.050871+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:03:27.050871+0000","last_undegraded":"2024-04-16T17:03:27.050871+0000","last_fullsized":"2024-04-16T17:03:27.050871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267291+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:03:27.267291+0000","last_peered":"2024-04-16T17:03:27.267291+0000","last_clean":"2024-04-16T17:03:27.267291+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:03:27.267291+0000","last_undegraded":"2024-04-16T17:03:27.267291+0000","last_fullsized":"2024-04-16T17:03:27.267291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266288+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:03:27.266288+0000","last_peered":"2024-04-16T17:03:27.266288+0000","last_clean":"2024-04-16T17:03:27.266288+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:03:27.266288+0000","last_undegraded":"2024-04-16T17:03:27.266288+0000","last_fullsized":"2024-04-16T17:03:27.266288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266452+0000","last_change":"2024-04-16T17:02:26.328059+0000","last_active":"2024-04-16T17:03:27.266452+0000","last_peered":"2024-04-16T17:03:27.266452+0000","last_clean":"2024-04-16T17:03:27.266452+0000","last_became_active":"2024-04-16T17:02:26.323315+0000","last_became_peered":"2024-04-16T17:02:26.323315+0000","last_unstale":"2024-04-16T17:03:27.266452+0000","last_undegraded":"2024-04-16T17:03:27.266452+0000","last_fullsized":"2024-04-16T17:03:27.266452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:01.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267585+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:03:27.267585+0000","last_peered":"2024-04-16T17:03:27.267585+0000","last_clean":"2024-04-16T17:03:27.267585+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:03:27.267585+0000","last_undegraded":"2024-04-16T17:03:27.267585+0000","last_fullsized":"2024-04-16T17:03:27.267585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448803+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:03:27.448803+0000","last_peered":"2024-04-16T17:03:27.448803+0000","last_clean":"2024-04-16T17:03:27.448803+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:03:27.448803+0000","last_undegraded":"2024-04-16T17:03:27.448803+0000","last_fullsized":"2024-04-16T17:03:27.448803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.049078+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:03:27.049078+0000","last_peered":"2024-04-16T17:03:27.049078+0000","last_clean":"2024-04-16T17:03:27.049078+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:03:27.049078+0000","last_undegraded":"2024-04-16T17:03:27.049078+0000","last_fullsized":"2024-04-16T17:03:27.049078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267529+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:03:27.267529+0000","last_peered":"2024-04-16T17:03:27.267529+0000","last_clean":"2024-04-16T17:03:27.267529+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:03:27.267529+0000","last_undegraded":"2024-04-16T17:03:27.267529+0000","last_fullsized":"2024-04-16T17:03:27.267529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048220+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:03:27.048220+0000","last_peered":"2024-04-16T17:03:27.048220+0000","last_clean":"2024-04-16T17:03:27.048220+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:03:27.048220+0000","last_undegraded":"2024-04-16T17:03:27.048220+0000","last_fullsized":"2024-04-16T17:03:27.048220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267805+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:03:27.267805+0000","last_peered":"2024-04-16T17:03:27.267805+0000","last_clean":"2024-04-16T17:03:27.267805+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:03:27.267805+0000","last_undegraded":"2024-04-16T17:03:27.267805+0000","last_fullsized":"2024-04-16T17:03:27.267805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267798+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:03:27.267798+0000","last_peered":"2024-04-16T17:03:27.267798+0000","last_clean":"2024-04-16T17:03:27.267798+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:03:27.267798+0000","last_undegraded":"2024-04-16T17:03:27.267798+0000","last_fullsized":"2024-04-16T17:03:27.267798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053402+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:03:27.053402+0000","last_peered":"2024-04-16T17:03:27.053402+0000","last_clean":"2024-04-16T17:03:27.053402+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:03:27.053402+0000","last_undegraded":"2024-04-16T17:03:27.053402+0000","last_fullsized":"2024-04-16T17:03:27.053402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267486+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:03:27.267486+0000","last_peered":"2024-04-16T17:03:27.267486+0000","last_clean":"2024-04-16T17:03:27.267486+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:03:27.267486+0000","last_undegraded":"2024-04-16T17:03:27.267486+0000","last_fullsized":"2024-04-16T17:03:27.267486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267014+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:03:27.267014+0000","last_peered":"2024-04-16T17:03:27.267014+0000","last_clean":"2024-04-16T17:03:27.267014+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:03:27.267014+0000","last_undegraded":"2024-04-16T17:03:27.267014+0000","last_fullsized":"2024-04-16T17:03:27.267014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449161+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:03:27.449161+0000","last_peered":"2024-04-16T17:03:27.449161+0000","last_clean":"2024-04-16T17:03:27.449161+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:03:27.449161+0000","last_undegraded":"2024-04-16T17:03:27.449161+0000","last_fullsized":"2024-04-16T17:03:27.449161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.269015+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:03:27.269015+0000","last_peered":"2024-04-16T17:03:27.269015+0000","last_clean":"2024-04-16T17:03:27.269015+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:03:27.269015+0000","last_undegraded":"2024-04-16T17:03:27.269015+0000","last_fullsized":"2024-04-16T17:03:27.269015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053571+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:03:27.053571+0000","last_peered":"2024-04-16T17:03:27.053571+0000","last_clean":"2024-04-16T17:03:27.053571+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:03:27.053571+0000","last_undegraded":"2024-04-16T17:03:27.053571+0000","last_fullsized":"2024-04-16T17:03:27.053571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267683+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:03:27.267683+0000","last_peered":"2024-04-16T17:03:27.267683+0000","last_clean":"2024-04-16T17:03:27.267683+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:03:27.267683+0000","last_undegraded":"2024-04-16T17:03:27.267683+0000","last_fullsized":"2024-04-16T17:03:27.267683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053879+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:03:27.053879+0000","last_peered":"2024-04-16T17:03:27.053879+0000","last_clean":"2024-04-16T17:03:27.053879+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:03:27.053879+0000","last_undegraded":"2024-04-16T17:03:27.053879+0000","last_fullsized":"2024-04-16T17:03:27.053879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048526+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:03:27.048526+0000","last_peered":"2024-04-16T17:03:27.048526+0000","last_clean":"2024-04-16T17:03:27.048526+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:03:27.048526+0000","last_undegraded":"2024-04-16T17:03:27.048526+0000","last_fullsized":"2024-04-16T17:03:27.048526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050912+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:03:27.050912+0000","last_peered":"2024-04-16T17:03:27.050912+0000","last_clean":"2024-04-16T17:03:27.050912+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:03:27.050912+0000","last_undegraded":"2024-04-16T17:03:27.050912+0000","last_fullsized":"2024-04-16T17:03:27.050912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266610+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:03:27.266610+0000","last_peered":"2024-04-16T17:03:27.266610+0000","last_clean":"2024-04-16T17:03:27.266610+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:03:27.266610+0000","last_undegraded":"2024-04-16T17:03:27.266610+0000","last_fullsized":"2024-04-16T17:03:27.266610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268263+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:03:27.268263+0000","last_peered":"2024-04-16T17:03:27.268263+0000","last_clean":"2024-04-16T17:03:27.268263+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:03:27.268263+0000","last_undegraded":"2024-04-16T17:03:27.268263+0000","last_fullsized":"2024-04-16T17:03:27.268263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053482+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:03:27.053482+0000","last_peered":"2024-04-16T17:03:27.053482+0000","last_clean":"2024-04-16T17:03:27.053482+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:03:27.053482+0000","last_undegraded":"2024-04-16T17:03:27.053482+0000","last_fullsized":"2024-04-16T17:03:27.053482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245319,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68065,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":612,"num_read_kb":380,"num_write":169,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":204195,"data_compressed":5478,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":519809,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1163264,"data_stored":1171510,"data_compressed":64019,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":524119,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":519921,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1115461,"data_compressed":62193,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":515199,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":580146,"data_compressed":28504,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":516587,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":579710,"data_compressed":28504,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":515393,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":520471,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":1111049,"data_compressed":62193,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65037,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64601,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66033,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:03:40.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:40 smithi022 ceph-mon[36023]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 258 B/s wr, 7 op/s 2024-04-16T17:03:40.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:40 smithi022 ceph-mon[36023]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:40 smithi018 ceph-mon[31776]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 258 B/s wr, 7 op/s 2024-04-16T17:03:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:40 smithi018 ceph-mon[31776]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:40.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:40 smithi043 ceph-mon[36901]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 258 B/s wr, 7 op/s 2024-04-16T17:03:40.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:40 smithi043 ceph-mon[36901]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:42.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:42 smithi018 ceph-mon[31776]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:42.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:42 smithi022 ceph-mon[36023]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:42.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:42 smithi043 ceph-mon[36901]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:42.890 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:03:43.403 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:44.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:44 smithi018 ceph-mon[31776]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:44.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:44 smithi022 ceph-mon[36023]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:44.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:44 smithi043 ceph-mon[36901]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.647+0000 7facd3e00700 1 -- 172.21.15.18:0/211637372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 msgr2=0x7facd4107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.647+0000 7facd3e00700 1 --2- 172.21.15.18:0/211637372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 0x7facd4107290 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faccc00b0a0 tx=0x7faccc00b3b0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.647+0000 7facd3e00700 1 -- 172.21.15.18:0/211637372 shutdown_connections 2024-04-16T17:03:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.647+0000 7facd3e00700 1 --2- 172.21.15.18:0/211637372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 0x7facd4107290 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.647+0000 7facd3e00700 1 --2- 172.21.15.18:0/211637372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41019d0 0x7facd4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.648+0000 7facd3e00700 1 --2- 172.21.15.18:0/211637372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facd41022f0 0x7facd41067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.648+0000 7facd3e00700 1 -- 172.21.15.18:0/211637372 >> 172.21.15.18:0/211637372 conn(0x7facd40fd3f0 msgr2=0x7facd40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.648+0000 7facd3e00700 1 -- 172.21.15.18:0/211637372 shutdown_connections 2024-04-16T17:03:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.648+0000 7facd3e00700 1 -- 172.21.15.18:0/211637372 wait complete. 2024-04-16T17:03:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.649+0000 7facd3e00700 1 Processor -- start 2024-04-16T17:03:45.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.649+0000 7facd3e00700 1 -- start start 2024-04-16T17:03:45.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.649+0000 7facd3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41019d0 0x7facd41a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.649+0000 7facd3e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facd41022f0 0x7facd41a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.649+0000 7facd3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 0x7facd419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.649+0000 7facd3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7facd41193d0 con 0x7facd41019d0 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.649+0000 7facd3e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7facd4119250 con 0x7facd4106de0 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.649+0000 7facd3e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7facd4119550 con 0x7facd41022f0 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41019d0 0x7facd41a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41019d0 0x7facd41a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42224/0 (socket says 172.21.15.18:42224) 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2a00700 1 -- 172.21.15.18:0/1842803406 learned_addr learned my addr 172.21.15.18:0/1842803406 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2000700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facd41022f0 0x7facd41a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd3400700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 0x7facd419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2a00700 1 -- 172.21.15.18:0/1842803406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facd41022f0 msgr2=0x7facd41a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2a00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facd41022f0 0x7facd41a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2a00700 1 -- 172.21.15.18:0/1842803406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 msgr2=0x7facd419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2a00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 0x7facd419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2a00700 1 -- 172.21.15.18:0/1842803406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd419fdb0 con 0x7facd41019d0 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd3400700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 0x7facd419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.650+0000 7facd2000700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facd41022f0 0x7facd41a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.651+0000 7facd2a00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41019d0 0x7facd41a5440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7facc400e9a0 tx=0x7facc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.651+0000 7faccb400700 1 -- 172.21.15.18:0/1842803406 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc400ca10 con 0x7facd41019d0 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.651+0000 7faccb400700 1 -- 172.21.15.18:0/1842803406 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc400cbb0 con 0x7facd41019d0 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.651+0000 7faccb400700 1 -- 172.21.15.18:0/1842803406 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4018680 con 0x7facd41019d0 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.651+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd41a00a0 con 0x7facd41019d0 2024-04-16T17:03:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.651+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd41ac310 con 0x7facd41019d0 2024-04-16T17:03:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.652+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd40697e0 con 0x7facd41019d0 2024-04-16T17:03:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.654+0000 7faccb400700 1 -- 172.21.15.18:0/1842803406 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7facc4010430 con 0x7facd41019d0 2024-04-16T17:03:45.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.656+0000 7faccb400700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7facb80a3cc0 0x7facb80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.656+0000 7faccb400700 1 -- 172.21.15.18:0/1842803406 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7facc40d09b0 con 0x7facd41019d0 2024-04-16T17:03:45.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.656+0000 7facd2000700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7facb80a3cc0 0x7facb80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.657+0000 7facd2000700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7facb80a3cc0 0x7facb80a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7facd41074c0 tx=0x7facc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:45.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.661+0000 7faccb400700 1 -- 172.21.15.18:0/1842803406 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facc409cf60 con 0x7facd41019d0 2024-04-16T17:03:45.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.886+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facd4105eb0 con 0x7facb80a3cc0 2024-04-16T17:03:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.902+0000 7faccb400700 1 -- 172.21.15.18:0/1842803406 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340776 (secure 0 0 0) 0x7facd4105eb0 con 0x7facb80a3cc0 2024-04-16T17:03:45.910 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.909+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7facb80a3cc0 msgr2=0x7facb80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.909+0000 7facd3e00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7facb80a3cc0 0x7facb80a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7facd41074c0 tx=0x7facc0009260 comp rx=0 tx=0).stop 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.909+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41019d0 msgr2=0x7facd41a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.909+0000 7facd3e00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41019d0 0x7facd41a5440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7facc400e9a0 tx=0x7facc400ecb0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.910+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 shutdown_connections 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.910+0000 7facd3e00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7facb80a3cc0 0x7facb80a6180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.910+0000 7facd3e00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facd4106de0 0x7facd419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.910+0000 7facd3e00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facd41019d0 0x7facd41a5440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.910+0000 7facd3e00700 1 --2- 172.21.15.18:0/1842803406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facd41022f0 0x7facd41a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.910+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 >> 172.21.15.18:0/1842803406 conn(0x7facd40fd3f0 msgr2=0x7facd4104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:45.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.910+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 shutdown_connections 2024-04-16T17:03:45.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:45.910+0000 7facd3e00700 1 -- 172.21.15.18:0/1842803406 wait complete. 2024-04-16T17:03:45.914 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:03:46.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:46 smithi018 ceph-mon[31776]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:46.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:46 smithi022 ceph-mon[36023]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:46 smithi043 ceph-mon[36901]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:46.777 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-04-16T17:03:44.943488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658667,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":778,"num_read_kb":514,"num_write":352,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378312,"kb_used_data":12440,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563512,"statfs":{"total":1151940427776,"available":1151553036288,"internally_reserved":0,"allocated":12738560,"data_stored":8189375,"data_compressed":432159,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005929"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.054050+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:03:27.054050+0000","last_peered":"2024-04-16T17:03:27.054050+0000","last_clean":"2024-04-16T17:03:27.054050+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:03:27.054050+0000","last_undegraded":"2024-04-16T17:03:27.054050+0000","last_fullsized":"2024-04-16T17:03:27.054050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.450032+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:03:27.450032+0000","last_peered":"2024-04-16T17:03:27.450032+0000","last_clean":"2024-04-16T17:03:27.450032+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:03:27.450032+0000","last_undegraded":"2024-04-16T17:03:27.450032+0000","last_fullsized":"2024-04-16T17:03:27.450032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449738+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:03:27.449738+0000","last_peered":"2024-04-16T17:03:27.449738+0000","last_clean":"2024-04-16T17:03:27.449738+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:03:27.449738+0000","last_undegraded":"2024-04-16T17:03:27.449738+0000","last_fullsized":"2024-04-16T17:03:27.449738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.270996+0000","last_change":"2024-04-16T17:02:26.318719+0000","last_active":"2024-04-16T17:03:27.270996+0000","last_peered":"2024-04-16T17:03:27.270996+0000","last_clean":"2024-04-16T17:03:27.270996+0000","last_became_active":"2024-04-16T17:02:26.317743+0000","last_became_peered":"2024-04-16T17:02:26.317743+0000","last_unstale":"2024-04-16T17:03:27.270996+0000","last_undegraded":"2024-04-16T17:03:27.270996+0000","last_fullsized":"2024-04-16T17:03:27.270996+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:26.827859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050853+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:03:27.050853+0000","last_peered":"2024-04-16T17:03:27.050853+0000","last_clean":"2024-04-16T17:03:27.050853+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:03:27.050853+0000","last_undegraded":"2024-04-16T17:03:27.050853+0000","last_fullsized":"2024-04-16T17:03:27.050853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051435+0000","last_change":"2024-04-16T17:02:26.315910+0000","last_active":"2024-04-16T17:03:27.051435+0000","last_peered":"2024-04-16T17:03:27.051435+0000","last_clean":"2024-04-16T17:03:27.051435+0000","last_became_active":"2024-04-16T17:02:26.315493+0000","last_became_peered":"2024-04-16T17:02:26.315493+0000","last_unstale":"2024-04-16T17:03:27.051435+0000","last_undegraded":"2024-04-16T17:03:27.051435+0000","last_fullsized":"2024-04-16T17:03:27.051435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:27:48.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267060+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:03:27.267060+0000","last_peered":"2024-04-16T17:03:27.267060+0000","last_clean":"2024-04-16T17:03:27.267060+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:03:27.267060+0000","last_undegraded":"2024-04-16T17:03:27.267060+0000","last_fullsized":"2024-04-16T17:03:27.267060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050577+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:03:27.050577+0000","last_peered":"2024-04-16T17:03:27.050577+0000","last_clean":"2024-04-16T17:03:27.050577+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:03:27.050577+0000","last_undegraded":"2024-04-16T17:03:27.050577+0000","last_fullsized":"2024-04-16T17:03:27.050577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267199+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:03:27.267199+0000","last_peered":"2024-04-16T17:03:27.267199+0000","last_clean":"2024-04-16T17:03:27.267199+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:03:27.267199+0000","last_undegraded":"2024-04-16T17:03:27.267199+0000","last_fullsized":"2024-04-16T17:03:27.267199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052004+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:03:27.052004+0000","last_peered":"2024-04-16T17:03:27.052004+0000","last_clean":"2024-04-16T17:03:27.052004+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:03:27.052004+0000","last_undegraded":"2024-04-16T17:03:27.052004+0000","last_fullsized":"2024-04-16T17:03:27.052004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267455+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:03:27.267455+0000","last_peered":"2024-04-16T17:03:27.267455+0000","last_clean":"2024-04-16T17:03:27.267455+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:03:27.267455+0000","last_undegraded":"2024-04-16T17:03:27.267455+0000","last_fullsized":"2024-04-16T17:03:27.267455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051237+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:03:27.051237+0000","last_peered":"2024-04-16T17:03:27.051237+0000","last_clean":"2024-04-16T17:03:27.051237+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:03:27.051237+0000","last_undegraded":"2024-04-16T17:03:27.051237+0000","last_fullsized":"2024-04-16T17:03:27.051237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268482+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:03:27.268482+0000","last_peered":"2024-04-16T17:03:27.268482+0000","last_clean":"2024-04-16T17:03:27.268482+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:03:27.268482+0000","last_undegraded":"2024-04-16T17:03:27.268482+0000","last_fullsized":"2024-04-16T17:03:27.268482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448788+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:03:27.448788+0000","last_peered":"2024-04-16T17:03:27.448788+0000","last_clean":"2024-04-16T17:03:27.448788+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:03:27.448788+0000","last_undegraded":"2024-04-16T17:03:27.448788+0000","last_fullsized":"2024-04-16T17:03:27.448788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448708+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:03:27.448708+0000","last_peered":"2024-04-16T17:03:27.448708+0000","last_clean":"2024-04-16T17:03:27.448708+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:03:27.448708+0000","last_undegraded":"2024-04-16T17:03:27.448708+0000","last_fullsized":"2024-04-16T17:03:27.448708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266297+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:03:27.266297+0000","last_peered":"2024-04-16T17:03:27.266297+0000","last_clean":"2024-04-16T17:03:27.266297+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:03:27.266297+0000","last_undegraded":"2024-04-16T17:03:27.266297+0000","last_fullsized":"2024-04-16T17:03:27.266297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.054021+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:03:27.054021+0000","last_peered":"2024-04-16T17:03:27.054021+0000","last_clean":"2024-04-16T17:03:27.054021+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:03:27.054021+0000","last_undegraded":"2024-04-16T17:03:27.054021+0000","last_fullsized":"2024-04-16T17:03:27.054021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273453+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:03:27.273453+0000","last_peered":"2024-04-16T17:03:27.273453+0000","last_clean":"2024-04-16T17:03:27.273453+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:03:27.273453+0000","last_undegraded":"2024-04-16T17:03:27.273453+0000","last_fullsized":"2024-04-16T17:03:27.273453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267590+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:03:27.267590+0000","last_peered":"2024-04-16T17:03:27.267590+0000","last_clean":"2024-04-16T17:03:27.267590+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:03:27.267590+0000","last_undegraded":"2024-04-16T17:03:27.267590+0000","last_fullsized":"2024-04-16T17:03:27.267590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267227+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:03:27.267227+0000","last_peered":"2024-04-16T17:03:27.267227+0000","last_clean":"2024-04-16T17:03:27.267227+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:03:27.267227+0000","last_undegraded":"2024-04-16T17:03:27.267227+0000","last_fullsized":"2024-04-16T17:03:27.267227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.274411+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:03:27.274411+0000","last_peered":"2024-04-16T17:03:27.274411+0000","last_clean":"2024-04-16T17:03:27.274411+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:03:27.274411+0000","last_undegraded":"2024-04-16T17:03:27.274411+0000","last_fullsized":"2024-04-16T17:03:27.274411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273261+0000","last_change":"2024-04-16T17:02:26.318546+0000","last_active":"2024-04-16T17:03:27.273261+0000","last_peered":"2024-04-16T17:03:27.273261+0000","last_clean":"2024-04-16T17:03:27.273261+0000","last_became_active":"2024-04-16T17:02:26.317794+0000","last_became_peered":"2024-04-16T17:02:26.317794+0000","last_unstale":"2024-04-16T17:03:27.273261+0000","last_undegraded":"2024-04-16T17:03:27.273261+0000","last_fullsized":"2024-04-16T17:03:27.273261+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:48.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050304+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:03:27.050304+0000","last_peered":"2024-04-16T17:03:27.050304+0000","last_clean":"2024-04-16T17:03:27.050304+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:03:27.050304+0000","last_undegraded":"2024-04-16T17:03:27.050304+0000","last_fullsized":"2024-04-16T17:03:27.050304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052748+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:03:27.052748+0000","last_peered":"2024-04-16T17:03:27.052748+0000","last_clean":"2024-04-16T17:03:27.052748+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:03:27.052748+0000","last_undegraded":"2024-04-16T17:03:27.052748+0000","last_fullsized":"2024-04-16T17:03:27.052748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449960+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:03:27.449960+0000","last_peered":"2024-04-16T17:03:27.449960+0000","last_clean":"2024-04-16T17:03:27.449960+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:03:27.449960+0000","last_undegraded":"2024-04-16T17:03:27.449960+0000","last_fullsized":"2024-04-16T17:03:27.449960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.271585+0000","last_change":"2024-04-16T17:03:27.271585+0000","last_active":"2024-04-16T17:03:27.271585+0000","last_peered":"2024-04-16T17:03:27.271585+0000","last_clean":"2024-04-16T17:03:27.271585+0000","last_became_active":"2024-04-16T17:03:27.271251+0000","last_became_peered":"2024-04-16T17:03:27.271251+0000","last_unstale":"2024-04-16T17:03:27.271585+0000","last_undegraded":"2024-04-16T17:03:27.271585+0000","last_fullsized":"2024-04-16T17:03:27.271585+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:51.217466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051925+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:03:27.051925+0000","last_peered":"2024-04-16T17:03:27.051925+0000","last_clean":"2024-04-16T17:03:27.051925+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:03:27.051925+0000","last_undegraded":"2024-04-16T17:03:27.051925+0000","last_fullsized":"2024-04-16T17:03:27.051925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266687+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:03:27.266687+0000","last_peered":"2024-04-16T17:03:27.266687+0000","last_clean":"2024-04-16T17:03:27.266687+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:03:27.266687+0000","last_undegraded":"2024-04-16T17:03:27.266687+0000","last_fullsized":"2024-04-16T17:03:27.266687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050850+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:03:27.050850+0000","last_peered":"2024-04-16T17:03:27.050850+0000","last_clean":"2024-04-16T17:03:27.050850+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:03:27.050850+0000","last_undegraded":"2024-04-16T17:03:27.050850+0000","last_fullsized":"2024-04-16T17:03:27.050850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051204+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:03:27.051204+0000","last_peered":"2024-04-16T17:03:27.051204+0000","last_clean":"2024-04-16T17:03:27.051204+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:03:27.051204+0000","last_undegraded":"2024-04-16T17:03:27.051204+0000","last_fullsized":"2024-04-16T17:03:27.051204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048934+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:03:27.048934+0000","last_peered":"2024-04-16T17:03:27.048934+0000","last_clean":"2024-04-16T17:03:27.048934+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:03:27.048934+0000","last_undegraded":"2024-04-16T17:03:27.048934+0000","last_fullsized":"2024-04-16T17:03:27.048934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267582+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:03:27.267582+0000","last_peered":"2024-04-16T17:03:27.267582+0000","last_clean":"2024-04-16T17:03:27.267582+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:03:27.267582+0000","last_undegraded":"2024-04-16T17:03:27.267582+0000","last_fullsized":"2024-04-16T17:03:27.267582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":154,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:41.418118+0000","last_change":"2024-04-16T17:02:26.315434+0000","last_active":"2024-04-16T17:03:41.418118+0000","last_peered":"2024-04-16T17:03:41.418118+0000","last_clean":"2024-04-16T17:03:41.418118+0000","last_became_active":"2024-04-16T17:02:26.310169+0000","last_became_peered":"2024-04-16T17:02:26.310169+0000","last_unstale":"2024-04-16T17:03:41.418118+0000","last_undegraded":"2024-04-16T17:03:41.418118+0000","last_fullsized":"2024-04-16T17:03:41.418118+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:51.033415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050694+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:03:27.050694+0000","last_peered":"2024-04-16T17:03:27.050694+0000","last_clean":"2024-04-16T17:03:27.050694+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:03:27.050694+0000","last_undegraded":"2024-04-16T17:03:27.050694+0000","last_fullsized":"2024-04-16T17:03:27.050694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267268+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:03:27.267268+0000","last_peered":"2024-04-16T17:03:27.267268+0000","last_clean":"2024-04-16T17:03:27.267268+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:03:27.267268+0000","last_undegraded":"2024-04-16T17:03:27.267268+0000","last_fullsized":"2024-04-16T17:03:27.267268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051285+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:03:27.051285+0000","last_peered":"2024-04-16T17:03:27.051285+0000","last_clean":"2024-04-16T17:03:27.051285+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:03:27.051285+0000","last_undegraded":"2024-04-16T17:03:27.051285+0000","last_fullsized":"2024-04-16T17:03:27.051285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.270459+0000","last_change":"2024-04-16T17:02:26.321763+0000","last_active":"2024-04-16T17:03:27.270459+0000","last_peered":"2024-04-16T17:03:27.270459+0000","last_clean":"2024-04-16T17:03:27.270459+0000","last_became_active":"2024-04-16T17:02:26.321438+0000","last_became_peered":"2024-04-16T17:02:26.321438+0000","last_unstale":"2024-04-16T17:03:27.270459+0000","last_undegraded":"2024-04-16T17:03:27.270459+0000","last_fullsized":"2024-04-16T17:03:27.270459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:47.231035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.451719+0000","last_change":"2024-04-16T17:03:27.451220+0000","last_active":"2024-04-16T17:03:27.451719+0000","last_peered":"2024-04-16T17:03:27.451719+0000","last_clean":"2024-04-16T17:03:27.451719+0000","last_became_active":"2024-04-16T17:03:27.450680+0000","last_became_peered":"2024-04-16T17:03:27.450680+0000","last_unstale":"2024-04-16T17:03:27.451719+0000","last_undegraded":"2024-04-16T17:03:27.451719+0000","last_fullsized":"2024-04-16T17:03:27.451719+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:23.270332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267430+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:03:27.267430+0000","last_peered":"2024-04-16T17:03:27.267430+0000","last_clean":"2024-04-16T17:03:27.267430+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:03:27.267430+0000","last_undegraded":"2024-04-16T17:03:27.267430+0000","last_fullsized":"2024-04-16T17:03:27.267430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050234+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:03:27.050234+0000","last_peered":"2024-04-16T17:03:27.050234+0000","last_clean":"2024-04-16T17:03:27.050234+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:03:27.050234+0000","last_undegraded":"2024-04-16T17:03:27.050234+0000","last_fullsized":"2024-04-16T17:03:27.050234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449714+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:03:27.449714+0000","last_peered":"2024-04-16T17:03:27.449714+0000","last_clean":"2024-04-16T17:03:27.449714+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:03:27.449714+0000","last_undegraded":"2024-04-16T17:03:27.449714+0000","last_fullsized":"2024-04-16T17:03:27.449714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048433+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:03:27.048433+0000","last_peered":"2024-04-16T17:03:27.048433+0000","last_clean":"2024-04-16T17:03:27.048433+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:03:27.048433+0000","last_undegraded":"2024-04-16T17:03:27.048433+0000","last_fullsized":"2024-04-16T17:03:27.048433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273462+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:03:27.273462+0000","last_peered":"2024-04-16T17:03:27.273462+0000","last_clean":"2024-04-16T17:03:27.273462+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:03:27.273462+0000","last_undegraded":"2024-04-16T17:03:27.273462+0000","last_fullsized":"2024-04-16T17:03:27.273462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051478+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:03:27.051478+0000","last_peered":"2024-04-16T17:03:27.051478+0000","last_clean":"2024-04-16T17:03:27.051478+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:03:27.051478+0000","last_undegraded":"2024-04-16T17:03:27.051478+0000","last_fullsized":"2024-04-16T17:03:27.051478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":138,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:41.417389+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:03:41.417389+0000","last_peered":"2024-04-16T17:03:41.417389+0000","last_clean":"2024-04-16T17:03:41.417389+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:03:41.417389+0000","last_undegraded":"2024-04-16T17:03:41.417389+0000","last_fullsized":"2024-04-16T17:03:41.417389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267330+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:03:27.267330+0000","last_peered":"2024-04-16T17:03:27.267330+0000","last_clean":"2024-04-16T17:03:27.267330+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:03:27.267330+0000","last_undegraded":"2024-04-16T17:03:27.267330+0000","last_fullsized":"2024-04-16T17:03:27.267330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.269250+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:03:27.269250+0000","last_peered":"2024-04-16T17:03:27.269250+0000","last_clean":"2024-04-16T17:03:27.269250+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:03:27.269250+0000","last_undegraded":"2024-04-16T17:03:27.269250+0000","last_fullsized":"2024-04-16T17:03:27.269250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267661+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:03:27.267661+0000","last_peered":"2024-04-16T17:03:27.267661+0000","last_clean":"2024-04-16T17:03:27.267661+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:03:27.267661+0000","last_undegraded":"2024-04-16T17:03:27.267661+0000","last_fullsized":"2024-04-16T17:03:27.267661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449950+0000","last_change":"2024-04-16T17:02:26.752275+0000","last_active":"2024-04-16T17:03:27.449950+0000","last_peered":"2024-04-16T17:03:27.449950+0000","last_clean":"2024-04-16T17:03:27.449950+0000","last_became_active":"2024-04-16T17:02:26.751729+0000","last_became_peered":"2024-04-16T17:02:26.751729+0000","last_unstale":"2024-04-16T17:03:27.449950+0000","last_undegraded":"2024-04-16T17:03:27.449950+0000","last_fullsized":"2024-04-16T17:03:27.449950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:41:35.777691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051500+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:03:27.051500+0000","last_peered":"2024-04-16T17:03:27.051500+0000","last_clean":"2024-04-16T17:03:27.051500+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:03:27.051500+0000","last_undegraded":"2024-04-16T17:03:27.051500+0000","last_fullsized":"2024-04-16T17:03:27.051500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048902+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:03:27.048902+0000","last_peered":"2024-04-16T17:03:27.048902+0000","last_clean":"2024-04-16T17:03:27.048902+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:03:27.048902+0000","last_undegraded":"2024-04-16T17:03:27.048902+0000","last_fullsized":"2024-04-16T17:03:27.048902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266926+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:03:27.266926+0000","last_peered":"2024-04-16T17:03:27.266926+0000","last_clean":"2024-04-16T17:03:27.266926+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:03:27.266926+0000","last_undegraded":"2024-04-16T17:03:27.266926+0000","last_fullsized":"2024-04-16T17:03:27.266926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.055021+0000","last_change":"2024-04-16T17:03:27.054627+0000","last_active":"2024-04-16T17:03:27.055021+0000","last_peered":"2024-04-16T17:03:27.055021+0000","last_clean":"2024-04-16T17:03:27.055021+0000","last_became_active":"2024-04-16T17:03:27.054326+0000","last_became_peered":"2024-04-16T17:03:27.054326+0000","last_unstale":"2024-04-16T17:03:27.055021+0000","last_undegraded":"2024-04-16T17:03:27.055021+0000","last_fullsized":"2024-04-16T17:03:27.055021+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:43.535958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":285,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:42.418794+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:03:42.418794+0000","last_peered":"2024-04-16T17:03:42.418794+0000","last_clean":"2024-04-16T17:03:42.418794+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:03:42.418794+0000","last_undegraded":"2024-04-16T17:03:42.418794+0000","last_fullsized":"2024-04-16T17:03:42.418794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64455,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":218,"num_read_kb":173,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051676+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:03:27.051676+0000","last_peered":"2024-04-16T17:03:27.051676+0000","last_clean":"2024-04-16T17:03:27.051676+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:03:27.051676+0000","last_undegraded":"2024-04-16T17:03:27.051676+0000","last_fullsized":"2024-04-16T17:03:27.051676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.049199+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:03:27.049199+0000","last_peered":"2024-04-16T17:03:27.049199+0000","last_clean":"2024-04-16T17:03:27.049199+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:03:27.049199+0000","last_undegraded":"2024-04-16T17:03:27.049199+0000","last_fullsized":"2024-04-16T17:03:27.049199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267328+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:03:27.267328+0000","last_peered":"2024-04-16T17:03:27.267328+0000","last_clean":"2024-04-16T17:03:27.267328+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:03:27.267328+0000","last_undegraded":"2024-04-16T17:03:27.267328+0000","last_fullsized":"2024-04-16T17:03:27.267328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267852+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:03:27.267852+0000","last_peered":"2024-04-16T17:03:27.267852+0000","last_clean":"2024-04-16T17:03:27.267852+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:03:27.267852+0000","last_undegraded":"2024-04-16T17:03:27.267852+0000","last_fullsized":"2024-04-16T17:03:27.267852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053205+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:03:27.053205+0000","last_peered":"2024-04-16T17:03:27.053205+0000","last_clean":"2024-04-16T17:03:27.053205+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:03:27.053205+0000","last_undegraded":"2024-04-16T17:03:27.053205+0000","last_fullsized":"2024-04-16T17:03:27.053205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048636+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:03:27.048636+0000","last_peered":"2024-04-16T17:03:27.048636+0000","last_clean":"2024-04-16T17:03:27.048636+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:03:27.048636+0000","last_undegraded":"2024-04-16T17:03:27.048636+0000","last_fullsized":"2024-04-16T17:03:27.048636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267640+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:03:27.267640+0000","last_peered":"2024-04-16T17:03:27.267640+0000","last_clean":"2024-04-16T17:03:27.267640+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:03:27.267640+0000","last_undegraded":"2024-04-16T17:03:27.267640+0000","last_fullsized":"2024-04-16T17:03:27.267640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273890+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:03:27.273890+0000","last_peered":"2024-04-16T17:03:27.273890+0000","last_clean":"2024-04-16T17:03:27.273890+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:03:27.273890+0000","last_undegraded":"2024-04-16T17:03:27.273890+0000","last_fullsized":"2024-04-16T17:03:27.273890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":140,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:41.405584+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:03:41.405584+0000","last_peered":"2024-04-16T17:03:41.405584+0000","last_clean":"2024-04-16T17:03:41.405584+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:03:41.405584+0000","last_undegraded":"2024-04-16T17:03:41.405584+0000","last_fullsized":"2024-04-16T17:03:41.405584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267027+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:03:27.267027+0000","last_peered":"2024-04-16T17:03:27.267027+0000","last_clean":"2024-04-16T17:03:27.267027+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:03:27.267027+0000","last_undegraded":"2024-04-16T17:03:27.267027+0000","last_fullsized":"2024-04-16T17:03:27.267027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267963+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:03:27.267963+0000","last_peered":"2024-04-16T17:03:27.267963+0000","last_clean":"2024-04-16T17:03:27.267963+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:03:27.267963+0000","last_undegraded":"2024-04-16T17:03:27.267963+0000","last_fullsized":"2024-04-16T17:03:27.267963+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050660+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:03:27.050660+0000","last_peered":"2024-04-16T17:03:27.050660+0000","last_clean":"2024-04-16T17:03:27.050660+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:03:27.050660+0000","last_undegraded":"2024-04-16T17:03:27.050660+0000","last_fullsized":"2024-04-16T17:03:27.050660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048626+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:03:27.048626+0000","last_peered":"2024-04-16T17:03:27.048626+0000","last_clean":"2024-04-16T17:03:27.048626+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:03:27.048626+0000","last_undegraded":"2024-04-16T17:03:27.048626+0000","last_fullsized":"2024-04-16T17:03:27.048626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048523+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:03:27.048523+0000","last_peered":"2024-04-16T17:03:27.048523+0000","last_clean":"2024-04-16T17:03:27.048523+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:03:27.048523+0000","last_undegraded":"2024-04-16T17:03:27.048523+0000","last_fullsized":"2024-04-16T17:03:27.048523+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053451+0000","last_change":"2024-04-16T17:02:26.320405+0000","last_active":"2024-04-16T17:03:27.053451+0000","last_peered":"2024-04-16T17:03:27.053451+0000","last_clean":"2024-04-16T17:03:27.053451+0000","last_became_active":"2024-04-16T17:02:26.307921+0000","last_became_peered":"2024-04-16T17:02:26.307921+0000","last_unstale":"2024-04-16T17:03:27.053451+0000","last_undegraded":"2024-04-16T17:03:27.053451+0000","last_fullsized":"2024-04-16T17:03:27.053451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:19.156260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053466+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:03:27.053466+0000","last_peered":"2024-04-16T17:03:27.053466+0000","last_clean":"2024-04-16T17:03:27.053466+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:03:27.053466+0000","last_undegraded":"2024-04-16T17:03:27.053466+0000","last_fullsized":"2024-04-16T17:03:27.053466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050931+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:03:27.050931+0000","last_peered":"2024-04-16T17:03:27.050931+0000","last_clean":"2024-04-16T17:03:27.050931+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:03:27.050931+0000","last_undegraded":"2024-04-16T17:03:27.050931+0000","last_fullsized":"2024-04-16T17:03:27.050931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.270987+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:03:27.270987+0000","last_peered":"2024-04-16T17:03:27.270987+0000","last_clean":"2024-04-16T17:03:27.270987+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:03:27.270987+0000","last_undegraded":"2024-04-16T17:03:27.270987+0000","last_fullsized":"2024-04-16T17:03:27.270987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051645+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:03:27.051645+0000","last_peered":"2024-04-16T17:03:27.051645+0000","last_clean":"2024-04-16T17:03:27.051645+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:03:27.051645+0000","last_undegraded":"2024-04-16T17:03:27.051645+0000","last_fullsized":"2024-04-16T17:03:27.051645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268420+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:03:27.268420+0000","last_peered":"2024-04-16T17:03:27.268420+0000","last_clean":"2024-04-16T17:03:27.268420+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:03:27.268420+0000","last_undegraded":"2024-04-16T17:03:27.268420+0000","last_fullsized":"2024-04-16T17:03:27.268420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267443+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:03:27.267443+0000","last_peered":"2024-04-16T17:03:27.267443+0000","last_clean":"2024-04-16T17:03:27.267443+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:03:27.267443+0000","last_undegraded":"2024-04-16T17:03:27.267443+0000","last_fullsized":"2024-04-16T17:03:27.267443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051006+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:03:27.051006+0000","last_peered":"2024-04-16T17:03:27.051006+0000","last_clean":"2024-04-16T17:03:27.051006+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:03:27.051006+0000","last_undegraded":"2024-04-16T17:03:27.051006+0000","last_fullsized":"2024-04-16T17:03:27.051006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268936+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:03:27.268936+0000","last_peered":"2024-04-16T17:03:27.268936+0000","last_clean":"2024-04-16T17:03:27.268936+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:03:27.268936+0000","last_undegraded":"2024-04-16T17:03:27.268936+0000","last_fullsized":"2024-04-16T17:03:27.268936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048660+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:03:27.048660+0000","last_peered":"2024-04-16T17:03:27.048660+0000","last_clean":"2024-04-16T17:03:27.048660+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:03:27.048660+0000","last_undegraded":"2024-04-16T17:03:27.048660+0000","last_fullsized":"2024-04-16T17:03:27.048660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":138,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:39.399787+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:03:39.399787+0000","last_peered":"2024-04-16T17:03:39.399787+0000","last_clean":"2024-04-16T17:03:39.399787+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:03:39.399787+0000","last_undegraded":"2024-04-16T17:03:39.399787+0000","last_fullsized":"2024-04-16T17:03:39.399787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267230+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:03:27.267230+0000","last_peered":"2024-04-16T17:03:27.267230+0000","last_clean":"2024-04-16T17:03:27.267230+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:03:27.267230+0000","last_undegraded":"2024-04-16T17:03:27.267230+0000","last_fullsized":"2024-04-16T17:03:27.267230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051850+0000","last_change":"2024-04-16T17:02:26.322313+0000","last_active":"2024-04-16T17:03:27.051850+0000","last_peered":"2024-04-16T17:03:27.051850+0000","last_clean":"2024-04-16T17:03:27.051850+0000","last_became_active":"2024-04-16T17:02:26.320328+0000","last_became_peered":"2024-04-16T17:02:26.320328+0000","last_unstale":"2024-04-16T17:03:27.051850+0000","last_undegraded":"2024-04-16T17:03:27.051850+0000","last_fullsized":"2024-04-16T17:03:27.051850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:40.943183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051419+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:03:27.051419+0000","last_peered":"2024-04-16T17:03:27.051419+0000","last_clean":"2024-04-16T17:03:27.051419+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:03:27.051419+0000","last_undegraded":"2024-04-16T17:03:27.051419+0000","last_fullsized":"2024-04-16T17:03:27.051419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266776+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:03:27.266776+0000","last_peered":"2024-04-16T17:03:27.266776+0000","last_clean":"2024-04-16T17:03:27.266776+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:03:27.266776+0000","last_undegraded":"2024-04-16T17:03:27.266776+0000","last_fullsized":"2024-04-16T17:03:27.266776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048973+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:03:27.048973+0000","last_peered":"2024-04-16T17:03:27.048973+0000","last_clean":"2024-04-16T17:03:27.048973+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:03:27.048973+0000","last_undegraded":"2024-04-16T17:03:27.048973+0000","last_fullsized":"2024-04-16T17:03:27.048973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267262+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:03:27.267262+0000","last_peered":"2024-04-16T17:03:27.267262+0000","last_clean":"2024-04-16T17:03:27.267262+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:03:27.267262+0000","last_undegraded":"2024-04-16T17:03:27.267262+0000","last_fullsized":"2024-04-16T17:03:27.267262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052945+0000","last_change":"2024-04-16T17:02:26.312852+0000","last_active":"2024-04-16T17:03:27.052945+0000","last_peered":"2024-04-16T17:03:27.052945+0000","last_clean":"2024-04-16T17:03:27.052945+0000","last_became_active":"2024-04-16T17:02:26.308279+0000","last_became_peered":"2024-04-16T17:02:26.308279+0000","last_unstale":"2024-04-16T17:03:27.052945+0000","last_undegraded":"2024-04-16T17:03:27.052945+0000","last_fullsized":"2024-04-16T17:03:27.052945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:25.511349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053414+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:03:27.053414+0000","last_peered":"2024-04-16T17:03:27.053414+0000","last_clean":"2024-04-16T17:03:27.053414+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:03:27.053414+0000","last_undegraded":"2024-04-16T17:03:27.053414+0000","last_fullsized":"2024-04-16T17:03:27.053414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051892+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:03:27.051892+0000","last_peered":"2024-04-16T17:03:27.051892+0000","last_clean":"2024-04-16T17:03:27.051892+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:03:27.051892+0000","last_undegraded":"2024-04-16T17:03:27.051892+0000","last_fullsized":"2024-04-16T17:03:27.051892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268730+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:03:27.268730+0000","last_peered":"2024-04-16T17:03:27.268730+0000","last_clean":"2024-04-16T17:03:27.268730+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:03:27.268730+0000","last_undegraded":"2024-04-16T17:03:27.268730+0000","last_fullsized":"2024-04-16T17:03:27.268730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050407+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:03:27.050407+0000","last_peered":"2024-04-16T17:03:27.050407+0000","last_clean":"2024-04-16T17:03:27.050407+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:03:27.050407+0000","last_undegraded":"2024-04-16T17:03:27.050407+0000","last_fullsized":"2024-04-16T17:03:27.050407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.273741+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:03:27.273741+0000","last_peered":"2024-04-16T17:03:27.273741+0000","last_clean":"2024-04-16T17:03:27.273741+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:03:27.273741+0000","last_undegraded":"2024-04-16T17:03:27.273741+0000","last_fullsized":"2024-04-16T17:03:27.273741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053724+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:03:27.053724+0000","last_peered":"2024-04-16T17:03:27.053724+0000","last_clean":"2024-04-16T17:03:27.053724+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:03:27.053724+0000","last_undegraded":"2024-04-16T17:03:27.053724+0000","last_fullsized":"2024-04-16T17:03:27.053724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266977+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:03:27.266977+0000","last_peered":"2024-04-16T17:03:27.266977+0000","last_clean":"2024-04-16T17:03:27.266977+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:03:27.266977+0000","last_undegraded":"2024-04-16T17:03:27.266977+0000","last_fullsized":"2024-04-16T17:03:27.266977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051034+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:03:27.051034+0000","last_peered":"2024-04-16T17:03:27.051034+0000","last_clean":"2024-04-16T17:03:27.051034+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:03:27.051034+0000","last_undegraded":"2024-04-16T17:03:27.051034+0000","last_fullsized":"2024-04-16T17:03:27.051034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.450365+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:03:27.450365+0000","last_peered":"2024-04-16T17:03:27.450365+0000","last_clean":"2024-04-16T17:03:27.450365+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:03:27.450365+0000","last_undegraded":"2024-04-16T17:03:27.450365+0000","last_fullsized":"2024-04-16T17:03:27.450365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050691+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:03:27.050691+0000","last_peered":"2024-04-16T17:03:27.050691+0000","last_clean":"2024-04-16T17:03:27.050691+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:03:27.050691+0000","last_undegraded":"2024-04-16T17:03:27.050691+0000","last_fullsized":"2024-04-16T17:03:27.050691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266305+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:03:27.266305+0000","last_peered":"2024-04-16T17:03:27.266305+0000","last_clean":"2024-04-16T17:03:27.266305+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:03:27.266305+0000","last_undegraded":"2024-04-16T17:03:27.266305+0000","last_fullsized":"2024-04-16T17:03:27.266305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050728+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:03:27.050728+0000","last_peered":"2024-04-16T17:03:27.050728+0000","last_clean":"2024-04-16T17:03:27.050728+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:03:27.050728+0000","last_undegraded":"2024-04-16T17:03:27.050728+0000","last_fullsized":"2024-04-16T17:03:27.050728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268760+0000","last_change":"2024-04-16T17:03:27.268760+0000","last_active":"2024-04-16T17:03:27.268760+0000","last_peered":"2024-04-16T17:03:27.268760+0000","last_clean":"2024-04-16T17:03:27.268760+0000","last_became_active":"2024-04-16T17:03:27.268209+0000","last_became_peered":"2024-04-16T17:03:27.268209+0000","last_unstale":"2024-04-16T17:03:27.268760+0000","last_undegraded":"2024-04-16T17:03:27.268760+0000","last_fullsized":"2024-04-16T17:03:27.268760+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:24:57.457026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050876+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:03:27.050876+0000","last_peered":"2024-04-16T17:03:27.050876+0000","last_clean":"2024-04-16T17:03:27.050876+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:03:27.050876+0000","last_undegraded":"2024-04-16T17:03:27.050876+0000","last_fullsized":"2024-04-16T17:03:27.050876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053161+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:03:27.053161+0000","last_peered":"2024-04-16T17:03:27.053161+0000","last_clean":"2024-04-16T17:03:27.053161+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:03:27.053161+0000","last_undegraded":"2024-04-16T17:03:27.053161+0000","last_fullsized":"2024-04-16T17:03:27.053161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050922+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:03:27.050922+0000","last_peered":"2024-04-16T17:03:27.050922+0000","last_clean":"2024-04-16T17:03:27.050922+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:03:27.050922+0000","last_undegraded":"2024-04-16T17:03:27.050922+0000","last_fullsized":"2024-04-16T17:03:27.050922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267061+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:03:27.267061+0000","last_peered":"2024-04-16T17:03:27.267061+0000","last_clean":"2024-04-16T17:03:27.267061+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:03:27.267061+0000","last_undegraded":"2024-04-16T17:03:27.267061+0000","last_fullsized":"2024-04-16T17:03:27.267061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.052501+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:03:27.052501+0000","last_peered":"2024-04-16T17:03:27.052501+0000","last_clean":"2024-04-16T17:03:27.052501+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:03:27.052501+0000","last_undegraded":"2024-04-16T17:03:27.052501+0000","last_fullsized":"2024-04-16T17:03:27.052501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050542+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:03:27.050542+0000","last_peered":"2024-04-16T17:03:27.050542+0000","last_clean":"2024-04-16T17:03:27.050542+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:03:27.050542+0000","last_undegraded":"2024-04-16T17:03:27.050542+0000","last_fullsized":"2024-04-16T17:03:27.050542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.050937+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:03:27.050937+0000","last_peered":"2024-04-16T17:03:27.050937+0000","last_clean":"2024-04-16T17:03:27.050937+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:03:27.050937+0000","last_undegraded":"2024-04-16T17:03:27.050937+0000","last_fullsized":"2024-04-16T17:03:27.050937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267291+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:03:27.267291+0000","last_peered":"2024-04-16T17:03:27.267291+0000","last_clean":"2024-04-16T17:03:27.267291+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:03:27.267291+0000","last_undegraded":"2024-04-16T17:03:27.267291+0000","last_fullsized":"2024-04-16T17:03:27.267291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266352+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:03:27.266352+0000","last_peered":"2024-04-16T17:03:27.266352+0000","last_clean":"2024-04-16T17:03:27.266352+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:03:27.266352+0000","last_undegraded":"2024-04-16T17:03:27.266352+0000","last_fullsized":"2024-04-16T17:03:27.266352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266452+0000","last_change":"2024-04-16T17:02:26.328059+0000","last_active":"2024-04-16T17:03:27.266452+0000","last_peered":"2024-04-16T17:03:27.266452+0000","last_clean":"2024-04-16T17:03:27.266452+0000","last_became_active":"2024-04-16T17:02:26.323315+0000","last_became_peered":"2024-04-16T17:02:26.323315+0000","last_unstale":"2024-04-16T17:03:27.266452+0000","last_undegraded":"2024-04-16T17:03:27.266452+0000","last_fullsized":"2024-04-16T17:03:27.266452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:01.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267651+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:03:27.267651+0000","last_peered":"2024-04-16T17:03:27.267651+0000","last_clean":"2024-04-16T17:03:27.267651+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:03:27.267651+0000","last_undegraded":"2024-04-16T17:03:27.267651+0000","last_fullsized":"2024-04-16T17:03:27.267651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.448885+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:03:27.448885+0000","last_peered":"2024-04-16T17:03:27.448885+0000","last_clean":"2024-04-16T17:03:27.448885+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:03:27.448885+0000","last_undegraded":"2024-04-16T17:03:27.448885+0000","last_fullsized":"2024-04-16T17:03:27.448885+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.049131+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:03:27.049131+0000","last_peered":"2024-04-16T17:03:27.049131+0000","last_clean":"2024-04-16T17:03:27.049131+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:03:27.049131+0000","last_undegraded":"2024-04-16T17:03:27.049131+0000","last_fullsized":"2024-04-16T17:03:27.049131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267591+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:03:27.267591+0000","last_peered":"2024-04-16T17:03:27.267591+0000","last_clean":"2024-04-16T17:03:27.267591+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:03:27.267591+0000","last_undegraded":"2024-04-16T17:03:27.267591+0000","last_fullsized":"2024-04-16T17:03:27.267591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048275+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:03:27.048275+0000","last_peered":"2024-04-16T17:03:27.048275+0000","last_clean":"2024-04-16T17:03:27.048275+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:03:27.048275+0000","last_undegraded":"2024-04-16T17:03:27.048275+0000","last_fullsized":"2024-04-16T17:03:27.048275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267877+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:03:27.267877+0000","last_peered":"2024-04-16T17:03:27.267877+0000","last_clean":"2024-04-16T17:03:27.267877+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:03:27.267877+0000","last_undegraded":"2024-04-16T17:03:27.267877+0000","last_fullsized":"2024-04-16T17:03:27.267877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267840+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:03:27.267840+0000","last_peered":"2024-04-16T17:03:27.267840+0000","last_clean":"2024-04-16T17:03:27.267840+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:03:27.267840+0000","last_undegraded":"2024-04-16T17:03:27.267840+0000","last_fullsized":"2024-04-16T17:03:27.267840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053402+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:03:27.053402+0000","last_peered":"2024-04-16T17:03:27.053402+0000","last_clean":"2024-04-16T17:03:27.053402+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:03:27.053402+0000","last_undegraded":"2024-04-16T17:03:27.053402+0000","last_fullsized":"2024-04-16T17:03:27.053402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267486+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:03:27.267486+0000","last_peered":"2024-04-16T17:03:27.267486+0000","last_clean":"2024-04-16T17:03:27.267486+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:03:27.267486+0000","last_undegraded":"2024-04-16T17:03:27.267486+0000","last_fullsized":"2024-04-16T17:03:27.267486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267014+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:03:27.267014+0000","last_peered":"2024-04-16T17:03:27.267014+0000","last_clean":"2024-04-16T17:03:27.267014+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:03:27.267014+0000","last_undegraded":"2024-04-16T17:03:27.267014+0000","last_fullsized":"2024-04-16T17:03:27.267014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.449161+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:03:27.449161+0000","last_peered":"2024-04-16T17:03:27.449161+0000","last_clean":"2024-04-16T17:03:27.449161+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:03:27.449161+0000","last_undegraded":"2024-04-16T17:03:27.449161+0000","last_fullsized":"2024-04-16T17:03:27.449161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.269015+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:03:27.269015+0000","last_peered":"2024-04-16T17:03:27.269015+0000","last_clean":"2024-04-16T17:03:27.269015+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:03:27.269015+0000","last_undegraded":"2024-04-16T17:03:27.269015+0000","last_fullsized":"2024-04-16T17:03:27.269015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053571+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:03:27.053571+0000","last_peered":"2024-04-16T17:03:27.053571+0000","last_clean":"2024-04-16T17:03:27.053571+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:03:27.053571+0000","last_undegraded":"2024-04-16T17:03:27.053571+0000","last_fullsized":"2024-04-16T17:03:27.053571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.267683+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:03:27.267683+0000","last_peered":"2024-04-16T17:03:27.267683+0000","last_clean":"2024-04-16T17:03:27.267683+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:03:27.267683+0000","last_undegraded":"2024-04-16T17:03:27.267683+0000","last_fullsized":"2024-04-16T17:03:27.267683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053879+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:03:27.053879+0000","last_peered":"2024-04-16T17:03:27.053879+0000","last_clean":"2024-04-16T17:03:27.053879+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:03:27.053879+0000","last_undegraded":"2024-04-16T17:03:27.053879+0000","last_fullsized":"2024-04-16T17:03:27.053879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.048526+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:03:27.048526+0000","last_peered":"2024-04-16T17:03:27.048526+0000","last_clean":"2024-04-16T17:03:27.048526+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:03:27.048526+0000","last_undegraded":"2024-04-16T17:03:27.048526+0000","last_fullsized":"2024-04-16T17:03:27.048526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.051083+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:03:27.051083+0000","last_peered":"2024-04-16T17:03:27.051083+0000","last_clean":"2024-04-16T17:03:27.051083+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:03:27.051083+0000","last_undegraded":"2024-04-16T17:03:27.051083+0000","last_fullsized":"2024-04-16T17:03:27.051083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.266679+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:03:27.266679+0000","last_peered":"2024-04-16T17:03:27.266679+0000","last_clean":"2024-04-16T17:03:27.266679+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:03:27.266679+0000","last_undegraded":"2024-04-16T17:03:27.266679+0000","last_fullsized":"2024-04-16T17:03:27.266679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.268320+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:03:27.268320+0000","last_peered":"2024-04-16T17:03:27.268320+0000","last_clean":"2024-04-16T17:03:27.268320+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:03:27.268320+0000","last_undegraded":"2024-04-16T17:03:27.268320+0000","last_fullsized":"2024-04-16T17:03:27.268320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:03:27.053547+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:03:27.053547+0000","last_peered":"2024-04-16T17:03:27.053547+0000","last_clean":"2024-04-16T17:03:27.053547+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:03:27.053547+0000","last_undegraded":"2024-04-16T17:03:27.053547+0000","last_fullsized":"2024-04-16T17:03:27.053547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245319,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68065,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":642,"num_read_kb":395,"num_write":169,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":204195,"data_compressed":5478,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":519809,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1163264,"data_stored":1171510,"data_compressed":64019,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":524119,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":519921,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1115461,"data_compressed":62193,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":515199,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":580146,"data_compressed":28504,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":516587,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":579710,"data_compressed":28504,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":515393,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":520471,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":1111049,"data_compressed":62193,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33872,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65037,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64601,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66033,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:03:46.781 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-16T17:03:46.781 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-16T17:03:46.782 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-16T17:03:46.782 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph health --format=json 2024-04-16T17:03:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:47.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:47 smithi022 ceph-mon[36023]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:47.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:47 smithi018 ceph-mon[31776]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:47.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:47 smithi043 ceph-mon[36901]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:48.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:48 smithi022 ceph-mon[36023]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:48.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:48 smithi018 ceph-mon[31776]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:48.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:48 smithi043 ceph-mon[36901]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 -- 172.21.15.18:0/559334632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce2006b230 msgr2=0x7fce2006b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 --2- 172.21.15.18:0/559334632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce2006b230 0x7fce2006b690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009990 tx=0x7fce0c009ca0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 -- 172.21.15.18:0/559334632 shutdown_connections 2024-04-16T17:03:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 --2- 172.21.15.18:0/559334632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce2006bbd0 0x7fce2010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 --2- 172.21.15.18:0/559334632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce20107b60 0x7fce20107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 --2- 172.21.15.18:0/559334632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce2006b230 0x7fce2006b690 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 -- 172.21.15.18:0/559334632 >> 172.21.15.18:0/559334632 conn(0x7fce200787f0 msgr2=0x7fce20078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:49.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 -- 172.21.15.18:0/559334632 shutdown_connections 2024-04-16T17:03:49.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.626+0000 7fce28000700 1 -- 172.21.15.18:0/559334632 wait complete. 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.628+0000 7fce28000700 1 Processor -- start 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.628+0000 7fce28000700 1 -- start start 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce28000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce2006b230 0x7fce2019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce28000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2006bbd0 0x7fce2019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce28000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce20107b60 0x7fce2019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce28000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fce20112e60 con 0x7fce20107b60 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce28000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fce20112ce0 con 0x7fce2006b230 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce28000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce20112fe0 con 0x7fce2006bbd0 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce26000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce20107b60 0x7fce2019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce25600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce2006b230 0x7fce2019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce25600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce2006b230 0x7fce2019b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42768/0 (socket says 172.21.15.18:42768) 2024-04-16T17:03:49.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce25600700 1 -- 172.21.15.18:0/1037210725 learned_addr learned my addr 172.21.15.18:0/1037210725 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:49.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.629+0000 7fce24c00700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2006bbd0 0x7fce2019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:49.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce25600700 1 -- 172.21.15.18:0/1037210725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2006bbd0 msgr2=0x7fce2019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce25600700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2006bbd0 0x7fce2019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce25600700 1 -- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce20107b60 msgr2=0x7fce2019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce25600700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce20107b60 0x7fce2019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce25600700 1 -- 172.21.15.18:0/1037210725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce201a0510 con 0x7fce2006b230 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce25600700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce2006b230 0x7fce2019b500 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce1400e970 tx=0x7fce1400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce1e000700 1 -- 172.21.15.18:0/1037210725 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14004220 con 0x7fce2006b230 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce1e000700 1 -- 172.21.15.18:0/1037210725 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce140043c0 con 0x7fce2006b230 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce201a0800 con 0x7fce2006b230 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.630+0000 7fce1e000700 1 -- 172.21.15.18:0/1037210725 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14003dc0 con 0x7fce2006b230 2024-04-16T17:03:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.631+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce201a8080 con 0x7fce2006b230 2024-04-16T17:03:49.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.632+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce200697e0 con 0x7fce2006b230 2024-04-16T17:03:49.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.634+0000 7fce1e000700 1 -- 172.21.15.18:0/1037210725 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fce1400c7e0 con 0x7fce2006b230 2024-04-16T17:03:49.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.635+0000 7fce1e000700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fce080a3c40 0x7fce080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:49.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.635+0000 7fce1e000700 1 -- 172.21.15.18:0/1037210725 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fce140d09d0 con 0x7fce2006b230 2024-04-16T17:03:49.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.636+0000 7fce24c00700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fce080a3c40 0x7fce080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:49.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.636+0000 7fce24c00700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fce080a3c40 0x7fce080a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fce20078d80 tx=0x7fce0c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:49.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.640+0000 7fce1e000700 1 -- 172.21.15.18:0/1037210725 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce1409cf80 con 0x7fce2006b230 2024-04-16T17:03:49.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.926+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fce2002dfe0 con 0x7fce2006b230 2024-04-16T17:03:49.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.927+0000 7fce1e000700 1 -- 172.21.15.18:0/1037210725 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fce1409c880 con 0x7fce2006b230 2024-04-16T17:03:49.928 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:49.928 INFO:teuthology.orchestra.run.smithi018.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fce080a3c40 msgr2=0x7fce080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fce080a3c40 0x7fce080a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fce20078d80 tx=0x7fce0c005960 comp rx=0 tx=0).stop 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce2006b230 msgr2=0x7fce2019b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce2006b230 0x7fce2019b500 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce1400e970 tx=0x7fce1400ec80 comp rx=0 tx=0).stop 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 shutdown_connections 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fce080a3c40 0x7fce080a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce2006b230 0x7fce2019b500 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce20107b60 0x7fce2019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 --2- 172.21.15.18:0/1037210725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2006bbd0 0x7fce2019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 >> 172.21.15.18:0/1037210725 conn(0x7fce200787f0 msgr2=0x7fce2010df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 shutdown_connections 2024-04-16T17:03:49.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:49.930+0000 7fce28000700 1 -- 172.21.15.18:0/1037210725 wait complete. 2024-04-16T17:03:50.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:50 smithi018 ceph-mon[31776]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:03:50.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:50 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1037210725' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T17:03:50.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:50 smithi022 ceph-mon[36023]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:03:50.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:50 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1037210725' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T17:03:50.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:50 smithi043 ceph-mon[36901]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:03:50.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:50 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1037210725' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T17:03:50.758 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-16T17:03:50.758 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 -- 172.21.15.18:0/1996176524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105d10 msgr2=0x7f9b4810c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 --2- 172.21.15.18:0/1996176524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105d10 0x7f9b4810c5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ca0 tx=0x7f9b4002c100 comp rx=0 tx=0).stop 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 -- 172.21.15.18:0/1996176524 shutdown_connections 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 --2- 172.21.15.18:0/1996176524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105d10 0x7f9b4810c5a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 --2- 172.21.15.18:0/1996176524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b48105350 0x7f9b481057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 --2- 172.21.15.18:0/1996176524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b48104160 0x7f9b48104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 -- 172.21.15.18:0/1996176524 >> 172.21.15.18:0/1996176524 conn(0x7f9b480ff6d0 msgr2=0x7f9b48101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 -- 172.21.15.18:0/1996176524 shutdown_connections 2024-04-16T17:03:50.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.995+0000 7f9b4e800640 1 -- 172.21.15.18:0/1996176524 wait complete. 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.996+0000 7f9b4e800640 1 Processor -- start 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.996+0000 7f9b4e800640 1 -- start start 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b48104160 0x7f9b4819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105350 0x7f9b4819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b48105d10 0x7f9b481a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4e800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810fdf0 con 0x7f9b48105350 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4e800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810fc70 con 0x7f9b48105d10 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810ff70 con 0x7f9b48104160 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105350 0x7f9b4819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b48104160 0x7f9b4819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b48104160 0x7f9b4819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42166/0 (socket says 172.21.15.18:42166) 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4d400640 1 -- 172.21.15.18:0/3178716038 learned_addr learned my addr 172.21.15.18:0/3178716038 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:50.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4de00640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b48105d10 0x7f9b481a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4d400640 1 -- 172.21.15.18:0/3178716038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b48105d10 msgr2=0x7f9b481a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4d400640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b48105d10 0x7f9b481a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4d400640 1 -- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105350 msgr2=0x7f9b4819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4d400640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105350 0x7f9b4819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4d400640 1 -- 172.21.15.18:0/3178716038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b481a1fc0 con 0x7f9b48104160 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.997+0000 7f9b4ca00640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105350 0x7f9b4819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.998+0000 7f9b4d400640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b48104160 0x7f9b4819a3a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e9c0 tx=0x7f9b3c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.998+0000 7f9b46000640 1 -- 172.21.15.18:0/3178716038 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00cde0 con 0x7f9b48104160 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.998+0000 7f9b46000640 1 -- 172.21.15.18:0/3178716038 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3c010910 con 0x7f9b48104160 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.998+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b481a22b0 con 0x7f9b48104160 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.998+0000 7f9b46000640 1 -- 172.21.15.18:0/3178716038 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c018a10 con 0x7f9b48104160 2024-04-16T17:03:51.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.998+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481a2800 con 0x7f9b48104160 2024-04-16T17:03:51.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:50.999+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4810b420 con 0x7f9b48104160 2024-04-16T17:03:51.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.001+0000 7f9b46000640 1 -- 172.21.15.18:0/3178716038 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b3c01f020 con 0x7f9b48104160 2024-04-16T17:03:51.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.003+0000 7f9b46000640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9b300a3ba0 0x7f9b300a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.003+0000 7f9b46000640 1 -- 172.21.15.18:0/3178716038 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9b3c0d0fe0 con 0x7f9b48104160 2024-04-16T17:03:51.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.003+0000 7f9b4ca00640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9b300a3ba0 0x7f9b300a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.004+0000 7f9b4ca00640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9b300a3ba0 0x7f9b300a6060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9b4819b8b0 tx=0x7f9b38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:51.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.005+0000 7f9b46000640 1 -- 172.21.15.18:0/3178716038 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b3c09d5d0 con 0x7f9b48104160 2024-04-16T17:03:51.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.212+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b48062c00 con 0x7f9b48104160 2024-04-16T17:03:51.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.214+0000 7f9b46000640 1 -- 172.21.15.18:0/3178716038 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24748 (secure 0 0 0) 0x7f9b3c09ced0 con 0x7f9b48104160 2024-04-16T17:03:51.215 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:51.215 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":93,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:03:27.035238+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9b300a3ba0 msgr2=0x7f9b300a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9b300a3ba0 0x7f9b300a6060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9b4819b8b0 tx=0x7f9b38005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b48104160 msgr2=0x7f9b4819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b48104160 0x7f9b4819a3a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e9c0 tx=0x7f9b3c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 shutdown_connections 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b48105d10 0x7f9b481a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9b300a3ba0 0x7f9b300a6060 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b48105350 0x7f9b4819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 --2- 172.21.15.18:0/3178716038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b48104160 0x7f9b4819a3a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 >> 172.21.15.18:0/3178716038 conn(0x7f9b480ff6d0 msgr2=0x7f9b481085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 shutdown_connections 2024-04-16T17:03:51.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.217+0000 7f9b4e800640 1 -- 172.21.15.18:0/3178716038 wait complete. 2024-04-16T17:03:51.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:03:51.239 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:03:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 -- 172.21.15.18:0/2999017693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c104060 msgr2=0x7f480c104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 --2- 172.21.15.18:0/2999017693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c104060 0x7f480c104480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4800009900 tx=0x7f480002bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 -- 172.21.15.18:0/2999017693 shutdown_connections 2024-04-16T17:03:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 --2- 172.21.15.18:0/2999017693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f480c105c10 0x7f480c10c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 --2- 172.21.15.18:0/2999017693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f480c105250 0x7f480c1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 --2- 172.21.15.18:0/2999017693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c104060 0x7f480c104480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 -- 172.21.15.18:0/2999017693 >> 172.21.15.18:0/2999017693 conn(0x7f480c0ff610 msgr2=0x7f480c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 -- 172.21.15.18:0/2999017693 shutdown_connections 2024-04-16T17:03:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.473+0000 7f4811e00640 1 -- 172.21.15.18:0/2999017693 wait complete. 2024-04-16T17:03:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.475+0000 7f4811e00640 1 Processor -- start 2024-04-16T17:03:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.475+0000 7f4811e00640 1 -- start start 2024-04-16T17:03:51.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.475+0000 7f4811e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f480c104060 0x7f480c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.475+0000 7f4811e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f480c105250 0x7f480c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.475+0000 7f4811e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c105c10 0x7f480c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.475+0000 7f4811e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f480c10fd00 con 0x7f480c104060 2024-04-16T17:03:51.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.475+0000 7f4811e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f480c10fb80 con 0x7f480c105250 2024-04-16T17:03:51.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.475+0000 7f4811e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f480c10fe80 con 0x7f480c105c10 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4811400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c105c10 0x7f480c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4810a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f480c104060 0x7f480c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f480be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f480c105250 0x7f480c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4811400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c105c10 0x7f480c1a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42182/0 (socket says 172.21.15.18:42182) 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4811400640 1 -- 172.21.15.18:0/4077283313 learned_addr learned my addr 172.21.15.18:0/4077283313 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4811400640 1 -- 172.21.15.18:0/4077283313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f480c105250 msgr2=0x7f480c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4811400640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f480c105250 0x7f480c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4811400640 1 -- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f480c104060 msgr2=0x7f480c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4811400640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f480c104060 0x7f480c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f4811400640 1 -- 172.21.15.18:0/4077283313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1a1f90 con 0x7f480c105c10 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.476+0000 7f480be00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f480c105250 0x7f480c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.477+0000 7f4811400640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c105c10 0x7f480c1a1920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f480400eaa0 tx=0x7f480400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:51.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.477+0000 7f4809600640 1 -- 172.21.15.18:0/4077283313 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480400cdb0 con 0x7f480c105c10 2024-04-16T17:03:51.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.477+0000 7f4809600640 1 -- 172.21.15.18:0/4077283313 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4804005020 con 0x7f480c105c10 2024-04-16T17:03:51.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.477+0000 7f4809600640 1 -- 172.21.15.18:0/4077283313 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040116b0 con 0x7f480c105c10 2024-04-16T17:03:51.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.477+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c1a2280 con 0x7f480c105c10 2024-04-16T17:03:51.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.477+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c1a27d0 con 0x7f480c105c10 2024-04-16T17:03:51.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.478+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480c10b400 con 0x7f480c105c10 2024-04-16T17:03:51.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.480+0000 7f4809600640 1 -- 172.21.15.18:0/4077283313 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4804005560 con 0x7f480c105c10 2024-04-16T17:03:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.482+0000 7f4809600640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f47f40a3c20 0x7f47f40a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.482+0000 7f4809600640 1 -- 172.21.15.18:0/4077283313 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f48040d0970 con 0x7f480c105c10 2024-04-16T17:03:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.483+0000 7f4810a00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f47f40a3c20 0x7f47f40a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.483+0000 7f4810a00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f47f40a3c20 0x7f47f40a60e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f480002c180 tx=0x7f480002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:51.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.486+0000 7f4809600640 1 -- 172.21.15.18:0/4077283313 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4804016030 con 0x7f480c105c10 2024-04-16T17:03:51.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.693+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f480c062c00 con 0x7f480c105c10 2024-04-16T17:03:51.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.694+0000 7f4809600640 1 -- 172.21.15.18:0/4077283313 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f480409c8d0 con 0x7f480c105c10 2024-04-16T17:03:51.695 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f47f40a3c20 msgr2=0x7f47f40a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f47f40a3c20 0x7f47f40a60e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f480002c180 tx=0x7f480002cf90 comp rx=0 tx=0).stop 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c105c10 msgr2=0x7f480c1a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c105c10 0x7f480c1a1920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f480400eaa0 tx=0x7f480400ef60 comp rx=0 tx=0).stop 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 shutdown_connections 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480c105c10 0x7f480c1a1920 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f480c105250 0x7f480c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f47f40a3c20 0x7f47f40a60e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 --2- 172.21.15.18:0/4077283313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f480c104060 0x7f480c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 >> 172.21.15.18:0/4077283313 conn(0x7f480c0ff610 msgr2=0x7f480c1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:51.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 shutdown_connections 2024-04-16T17:03:51.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.697+0000 7f4811e00640 1 -- 172.21.15.18:0/4077283313 wait complete. 2024-04-16T17:03:51.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:03:51.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.949+0000 7f2239400640 1 -- 172.21.15.18:0/317927983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 msgr2=0x7f22341044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.949+0000 7f2239400640 1 --2- 172.21.15.18:0/317927983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 0x7f22341044a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2224009ca0 tx=0x7f222402c100 comp rx=0 tx=0).stop 2024-04-16T17:03:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.950+0000 7f2239400640 1 -- 172.21.15.18:0/317927983 shutdown_connections 2024-04-16T17:03:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.950+0000 7f2239400640 1 --2- 172.21.15.18:0/317927983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2234105c30 0x7f223410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.950+0000 7f2239400640 1 --2- 172.21.15.18:0/317927983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2234105270 0x7f22341056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.950+0000 7f2239400640 1 --2- 172.21.15.18:0/317927983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 0x7f22341044a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.950+0000 7f2239400640 1 -- 172.21.15.18:0/317927983 >> 172.21.15.18:0/317927983 conn(0x7f22340ff610 msgr2=0x7f2234101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.950+0000 7f2239400640 1 -- 172.21.15.18:0/317927983 shutdown_connections 2024-04-16T17:03:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.950+0000 7f2239400640 1 -- 172.21.15.18:0/317927983 wait complete. 2024-04-16T17:03:51.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2239400640 1 Processor -- start 2024-04-16T17:03:51.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2239400640 1 -- start start 2024-04-16T17:03:51.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2239400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 0x7f223419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2239400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2234105270 0x7f223419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2239400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2234105c30 0x7f22341a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2239400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f223410fd10 con 0x7f2234105c30 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2239400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f223410fb90 con 0x7f2234105270 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2239400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f223410fe90 con 0x7f2234104080 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2238a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2234105c30 0x7f22341a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2238a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2234105c30 0x7f22341a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37538/0 (socket says 172.21.15.18:37538) 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.951+0000 7f2238a00640 1 -- 172.21.15.18:0/2591288757 learned_addr learned my addr 172.21.15.18:0/2591288757 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2233e00640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 0x7f223419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2233400640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2234105270 0x7f223419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2238a00640 1 -- 172.21.15.18:0/2591288757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 msgr2=0x7f223419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2238a00640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 0x7f223419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2238a00640 1 -- 172.21.15.18:0/2591288757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2234105270 msgr2=0x7f223419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2238a00640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2234105270 0x7f223419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2238a00640 1 -- 172.21.15.18:0/2591288757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22341a2010 con 0x7f2234105c30 2024-04-16T17:03:51.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2233e00640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 0x7f223419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:51.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.952+0000 7f2238a00640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2234105c30 0x7f22341a19a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f222c00e9b0 tx=0x7f222c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:51.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.953+0000 7f2230c00640 1 -- 172.21.15.18:0/2591288757 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c00cd80 con 0x7f2234105c30 2024-04-16T17:03:51.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.953+0000 7f2230c00640 1 -- 172.21.15.18:0/2591288757 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c0035f0 con 0x7f2234105c30 2024-04-16T17:03:51.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.953+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22341a2300 con 0x7f2234105c30 2024-04-16T17:03:51.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.953+0000 7f2230c00640 1 -- 172.21.15.18:0/2591288757 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c010670 con 0x7f2234105c30 2024-04-16T17:03:51.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.953+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22341a2800 con 0x7f2234105c30 2024-04-16T17:03:51.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.954+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f223410b3f0 con 0x7f2234105c30 2024-04-16T17:03:51.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.956+0000 7f2230c00640 1 -- 172.21.15.18:0/2591288757 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f222c0037a0 con 0x7f2234105c30 2024-04-16T17:03:51.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.958+0000 7f2230c00640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f22180a3bb0 0x7f22180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.959+0000 7f2230c00640 1 -- 172.21.15.18:0/2591288757 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f222c0d0b00 con 0x7f2234105c30 2024-04-16T17:03:51.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.959+0000 7f2233e00640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f22180a3bb0 0x7f22180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.960+0000 7f2233e00640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f22180a3bb0 0x7f22180a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f222402c640 tx=0x7f2224009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:51.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:51.964+0000 7f2230c00640 1 -- 172.21.15.18:0/2591288757 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f222c09d0b0 con 0x7f2234105c30 2024-04-16T17:03:52.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.171+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2234062c00 con 0x7f2234105c30 2024-04-16T17:03:52.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.171+0000 7f2230c00640 1 -- 172.21.15.18:0/2591288757 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f222c09c9b0 con 0x7f2234105c30 2024-04-16T17:03:52.172 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:03:52.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.174+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f22180a3bb0 msgr2=0x7f22180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.174+0000 7f2239400640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f22180a3bb0 0x7f22180a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f222402c640 tx=0x7f2224009360 comp rx=0 tx=0).stop 2024-04-16T17:03:52.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.174+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2234105c30 msgr2=0x7f22341a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.174+0000 7f2239400640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2234105c30 0x7f22341a19a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f222c00e9b0 tx=0x7f222c00ee70 comp rx=0 tx=0).stop 2024-04-16T17:03:52.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.175+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 shutdown_connections 2024-04-16T17:03:52.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.175+0000 7f2239400640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f22180a3bb0 0x7f22180a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.175+0000 7f2239400640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2234105c30 0x7f22341a19a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.175+0000 7f2239400640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2234105270 0x7f223419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.175+0000 7f2239400640 1 --2- 172.21.15.18:0/2591288757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2234104080 0x7f223419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.175+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 >> 172.21.15.18:0/2591288757 conn(0x7f22340ff610 msgr2=0x7f22341085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:52.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.175+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 shutdown_connections 2024-04-16T17:03:52.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.175+0000 7f2239400640 1 -- 172.21.15.18:0/2591288757 wait complete. 2024-04-16T17:03:52.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:03:52.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:52 smithi022 ceph-mon[36023]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:52.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:52 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3178716038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:52.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:52 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4077283313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:52.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 -- 172.21.15.18:0/359943695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1480769c0 msgr2=0x7fa148076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 --2- 172.21.15.18:0/359943695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1480769c0 0x7fa148076de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa138009900 tx=0x7fa13802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:52.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 -- 172.21.15.18:0/359943695 shutdown_connections 2024-04-16T17:03:52.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 --2- 172.21.15.18:0/359943695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480785a0 0x7fa148078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 --2- 172.21.15.18:0/359943695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa148077c00 0x7fa148078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 --2- 172.21.15.18:0/359943695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1480769c0 0x7fa148076de0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 -- 172.21.15.18:0/359943695 >> 172.21.15.18:0/359943695 conn(0x7fa1480ff610 msgr2=0x7fa148101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:52.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 -- 172.21.15.18:0/359943695 shutdown_connections 2024-04-16T17:03:52.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.426+0000 7fa147e00640 1 -- 172.21.15.18:0/359943695 wait complete. 2024-04-16T17:03:52.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.427+0000 7fa147e00640 1 Processor -- start 2024-04-16T17:03:52.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa147e00640 1 -- start start 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa147e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480769c0 0x7fa14819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa147e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa148077c00 0x7fa14819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa147e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1480785a0 0x7fa1481a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa147e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa148114150 con 0x7fa1480785a0 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa147e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa148113fd0 con 0x7fa148077c00 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa147e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481142d0 con 0x7fa1480769c0 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa146a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480769c0 0x7fa14819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa146a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480769c0 0x7fa14819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42234/0 (socket says 172.21.15.18:42234) 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.428+0000 7fa146a00640 1 -- 172.21.15.18:0/1181839931 learned_addr learned my addr 172.21.15.18:0/1181839931 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa146000640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa148077c00 0x7fa14819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa147400640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1480785a0 0x7fa1481a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa146a00640 1 -- 172.21.15.18:0/1181839931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa148077c00 msgr2=0x7fa14819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa146a00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa148077c00 0x7fa14819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa146a00640 1 -- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1480785a0 msgr2=0x7fa1481a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa146a00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1480785a0 0x7fa1481a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa146a00640 1 -- 172.21.15.18:0/1181839931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1481a6380 con 0x7fa1480769c0 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa147400640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1480785a0 0x7fa1481a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.429+0000 7fa146a00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480769c0 0x7fa14819e760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa138000c00 tx=0x7fa13802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.430+0000 7fa13f400640 1 -- 172.21.15.18:0/1181839931 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138043020 con 0x7fa1480769c0 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.430+0000 7fa13f400640 1 -- 172.21.15.18:0/1181839931 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1380054e0 con 0x7fa1480769c0 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.430+0000 7fa13f400640 1 -- 172.21.15.18:0/1181839931 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138003c40 con 0x7fa1480769c0 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.430+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1481a6610 con 0x7fa1480769c0 2024-04-16T17:03:52.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.430+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a6b10 con 0x7fa1480769c0 2024-04-16T17:03:52.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.432+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1481109f0 con 0x7fa1480769c0 2024-04-16T17:03:52.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.433+0000 7fa13f400640 1 -- 172.21.15.18:0/1181839931 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa138046020 con 0x7fa1480769c0 2024-04-16T17:03:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.435+0000 7fa13f400640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa12c0a3bb0 0x7fa12c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.435+0000 7fa13f400640 1 -- 172.21.15.18:0/1181839931 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fa1380f28b0 con 0x7fa1480769c0 2024-04-16T17:03:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.435+0000 7fa146000640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa12c0a3bb0 0x7fa12c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.436+0000 7fa146000640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa12c0a3bb0 0x7fa12c0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa14819fc70 tx=0x7fa134005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:52.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.439+0000 7fa13f400640 1 -- 172.21.15.18:0/1181839931 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa13804d020 con 0x7fa1480769c0 2024-04-16T17:03:52.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:52 smithi018 ceph-mon[31776]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:52.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:52 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3178716038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:52.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:52 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4077283313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:52 smithi043 ceph-mon[36901]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:52 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3178716038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:52.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:52 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4077283313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:52.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.649+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa148062c30 con 0x7fa1480769c0 2024-04-16T17:03:52.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.651+0000 7fa13f400640 1 -- 172.21.15.18:0/1181839931 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fa1380bee60 con 0x7fa1480769c0 2024-04-16T17:03:52.651 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:03:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.653+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa12c0a3bb0 msgr2=0x7fa12c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.653+0000 7fa147e00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa12c0a3bb0 0x7fa12c0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa14819fc70 tx=0x7fa134005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.653+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480769c0 msgr2=0x7fa14819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.653+0000 7fa147e00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480769c0 0x7fa14819e760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa138000c00 tx=0x7fa13802c650 comp rx=0 tx=0).stop 2024-04-16T17:03:52.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.654+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 shutdown_connections 2024-04-16T17:03:52.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.654+0000 7fa147e00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fa12c0a3bb0 0x7fa12c0a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.654+0000 7fa147e00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1480785a0 0x7fa1481a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.654+0000 7fa147e00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa148077c00 0x7fa14819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.654+0000 7fa147e00640 1 --2- 172.21.15.18:0/1181839931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480769c0 0x7fa14819e760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.654+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 >> 172.21.15.18:0/1181839931 conn(0x7fa1480ff610 msgr2=0x7fa148101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:52.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.654+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 shutdown_connections 2024-04-16T17:03:52.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.654+0000 7fa147e00640 1 -- 172.21.15.18:0/1181839931 wait complete. 2024-04-16T17:03:52.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:03:52.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.900+0000 7fe380e00640 1 -- 172.21.15.18:0/2359151868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 msgr2=0x7fe37c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.900+0000 7fe380e00640 1 --2- 172.21.15.18:0/2359151868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 0x7fe37c103440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe368009900 tx=0x7fe36802bc50 comp rx=0 tx=0).stop 2024-04-16T17:03:52.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.900+0000 7fe380e00640 1 -- 172.21.15.18:0/2359151868 shutdown_connections 2024-04-16T17:03:52.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.900+0000 7fe380e00640 1 --2- 172.21.15.18:0/2359151868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37c103980 0x7fe37c10a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.900+0000 7fe380e00640 1 --2- 172.21.15.18:0/2359151868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 0x7fe37c103440 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.900+0000 7fe380e00640 1 --2- 172.21.15.18:0/2359151868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe37c101e70 0x7fe37c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.900+0000 7fe380e00640 1 -- 172.21.15.18:0/2359151868 >> 172.21.15.18:0/2359151868 conn(0x7fe37c0fd3e0 msgr2=0x7fe37c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:52.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.900+0000 7fe380e00640 1 -- 172.21.15.18:0/2359151868 shutdown_connections 2024-04-16T17:03:52.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.901+0000 7fe380e00640 1 -- 172.21.15.18:0/2359151868 wait complete. 2024-04-16T17:03:52.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe380e00640 1 Processor -- start 2024-04-16T17:03:52.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe380e00640 1 -- start start 2024-04-16T17:03:52.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe380e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe37c101e70 0x7fe37c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe380e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 0x7fe37c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe380e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37c103980 0x7fe37c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe380e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10db40 con 0x7fe37c102fc0 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe380e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10d9c0 con 0x7fe37c103980 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe380e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10dcc0 con 0x7fe37c101e70 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe37b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe37c101e70 0x7fe37c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.902+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 0x7fe37c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37c103980 0x7fe37c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe37c101e70 0x7fe37c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42264/0 (socket says 172.21.15.18:42264) 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37b400640 1 -- 172.21.15.18:0/2228910320 learned_addr learned my addr 172.21.15.18:0/2228910320 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37aa00640 1 -- 172.21.15.18:0/2228910320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe37c101e70 msgr2=0x7fe37c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37aa00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe37c101e70 0x7fe37c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37aa00640 1 -- 172.21.15.18:0/2228910320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37c103980 msgr2=0x7fe37c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37aa00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37c103980 0x7fe37c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37aa00640 1 -- 172.21.15.18:0/2228910320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c19fda0 con 0x7fe37c102fc0 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37b400640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe37c101e70 0x7fe37c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.903+0000 7fe37aa00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 0x7fe37c1986c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe368009230 tx=0x7fe36802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.904+0000 7fe373e00640 1 -- 172.21.15.18:0/2228910320 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368041070 con 0x7fe37c102fc0 2024-04-16T17:03:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.904+0000 7fe373e00640 1 -- 172.21.15.18:0/2228910320 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3680054e0 con 0x7fe37c102fc0 2024-04-16T17:03:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.904+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1a0030 con 0x7fe37c102fc0 2024-04-16T17:03:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.904+0000 7fe373e00640 1 -- 172.21.15.18:0/2228910320 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368003e60 con 0x7fe37c102fc0 2024-04-16T17:03:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.904+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a05a0 con 0x7fe37c102fc0 2024-04-16T17:03:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.905+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c109180 con 0x7fe37c102fc0 2024-04-16T17:03:52.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.907+0000 7fe373e00640 1 -- 172.21.15.18:0/2228910320 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe368044070 con 0x7fe37c102fc0 2024-04-16T17:03:52.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.908+0000 7fe373e00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3600a3c30 0x7fe3600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.909+0000 7fe373e00640 1 -- 172.21.15.18:0/2228910320 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fe3680f2130 con 0x7fe37c102fc0 2024-04-16T17:03:52.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.909+0000 7fe37b400640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3600a3c30 0x7fe3600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.910+0000 7fe37b400640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3600a3c30 0x7fe3600a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe36c011550 tx=0x7fe36c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:52.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:52.913+0000 7fe373e00640 1 -- 172.21.15.18:0/2228910320 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3680be090 con 0x7fe37c102fc0 2024-04-16T17:03:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.130+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe37c062c00 con 0x7fe37c102fc0 2024-04-16T17:03:53.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.131+0000 7fe373e00640 1 -- 172.21.15.18:0/2228910320 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fe3680c2390 con 0x7fe37c102fc0 2024-04-16T17:03:53.131 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:03:53.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3600a3c30 msgr2=0x7fe3600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3600a3c30 0x7fe3600a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe36c011550 tx=0x7fe36c005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 msgr2=0x7fe37c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 0x7fe37c1986c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe368009230 tx=0x7fe36802c670 comp rx=0 tx=0).stop 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 shutdown_connections 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37c103980 0x7fe37c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fe3600a3c30 0x7fe3600a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe37c102fc0 0x7fe37c1986c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 --2- 172.21.15.18:0/2228910320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe37c101e70 0x7fe37c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 >> 172.21.15.18:0/2228910320 conn(0x7fe37c0fd3e0 msgr2=0x7fe37c106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 shutdown_connections 2024-04-16T17:03:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.134+0000 7fe380e00640 1 -- 172.21.15.18:0/2228910320 wait complete. 2024-04-16T17:03:53.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 -- 172.21.15.18:0/371393409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cdc105cf0 msgr2=0x7f4cdc10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/371393409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cdc105cf0 0x7f4cdc10c580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009be0 tx=0x7f4cd402bef0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 -- 172.21.15.18:0/371393409 shutdown_connections 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/371393409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cdc105cf0 0x7f4cdc10c580 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/371393409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cdc105330 0x7f4cdc1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/371393409 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cdc104140 0x7f4cdc104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 -- 172.21.15.18:0/371393409 >> 172.21.15.18:0/371393409 conn(0x7f4cdc0ff6d0 msgr2=0x7f4cdc101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 -- 172.21.15.18:0/371393409 shutdown_connections 2024-04-16T17:03:53.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.375+0000 7f4cdbe00640 1 -- 172.21.15.18:0/371393409 wait complete. 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdbe00640 1 Processor -- start 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdbe00640 1 -- start start 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cdc104140 0x7f4cdc19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cdc105330 0x7f4cdc19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cdc105cf0 0x7f4cdc1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdbe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc10fdf0 con 0x7f4cdc104140 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc10fc70 con 0x7f4cdc105330 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc10ff70 con 0x7f4cdc105cf0 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cdc104140 0x7f4cdc19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cdc104140 0x7f4cdc19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37582/0 (socket says 172.21.15.18:37582) 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.377+0000 7f4cdaa00640 1 -- 172.21.15.18:0/3630385838 learned_addr learned my addr 172.21.15.18:0/3630385838 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:53.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cdb400640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cdc105cf0 0x7f4cdc1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cd3e00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cdc105330 0x7f4cdc19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cdaa00640 1 -- 172.21.15.18:0/3630385838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cdc105cf0 msgr2=0x7f4cdc1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cdaa00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cdc105cf0 0x7f4cdc1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cdaa00640 1 -- 172.21.15.18:0/3630385838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cdc105330 msgr2=0x7f4cdc19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cdaa00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cdc105330 0x7f4cdc19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cdaa00640 1 -- 172.21.15.18:0/3630385838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc1a2000 con 0x7f4cdc104140 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cdaa00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cdc104140 0x7f4cdc19a3e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e9a0 tx=0x7f4ccc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.378+0000 7f4cd3400640 1 -- 172.21.15.18:0/3630385838 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00cd80 con 0x7f4cdc104140 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.379+0000 7f4cd3400640 1 -- 172.21.15.18:0/3630385838 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ccc010910 con 0x7f4cdc104140 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.379+0000 7f4cd3400640 1 -- 172.21.15.18:0/3630385838 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc018a30 con 0x7f4cdc104140 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.379+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc1a22f0 con 0x7f4cdc104140 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.379+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc1a27f0 con 0x7f4cdc104140 2024-04-16T17:03:53.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.380+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cdc10b400 con 0x7f4cdc104140 2024-04-16T17:03:53.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.382+0000 7f4cd3400640 1 -- 172.21.15.18:0/3630385838 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ccc010ac0 con 0x7f4cdc104140 2024-04-16T17:03:53.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.384+0000 7f4cd3400640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4cc00a3ba0 0x7f4cc00a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.385+0000 7f4cd3400640 1 -- 172.21.15.18:0/3630385838 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4ccc0d1220 con 0x7f4cdc104140 2024-04-16T17:03:53.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.385+0000 7f4cd3e00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4cc00a3ba0 0x7f4cc00a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.386+0000 7f4cd3e00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4cc00a3ba0 0x7f4cc00a6060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc19b8f0 tx=0x7f4cc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:53.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.388+0000 7f4cd3400640 1 -- 172.21.15.18:0/3630385838 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ccc026070 con 0x7f4cdc104140 2024-04-16T17:03:53.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:53 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2591288757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:53 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1181839931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:53 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2228910320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:53 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2591288757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:53 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1181839931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:53 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2228910320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:53 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2591288757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:53 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1181839931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:53 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2228910320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:53.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.637+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cdc02d620 con 0x7f4cdc104140 2024-04-16T17:03:53.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.639+0000 7f4cd3400640 1 -- 172.21.15.18:0/3630385838 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11848 (secure 0 0 0) 0x7f4ccc09d7d0 con 0x7f4cdc104140 2024-04-16T17:03:53.640 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:53.640 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:01:59.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:03:53.640 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 24 2024-04-16T17:03:53.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.642+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4cc00a3ba0 msgr2=0x7f4cc00a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.642+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4cc00a3ba0 0x7f4cc00a6060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc19b8f0 tx=0x7f4cc4005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:53.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.642+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cdc104140 msgr2=0x7f4cdc19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.642+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cdc104140 0x7f4cdc19a3e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e9a0 tx=0x7f4ccc00ee60 comp rx=0 tx=0).stop 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.643+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 shutdown_connections 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.643+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cdc105cf0 0x7f4cdc1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.643+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4cdc105330 0x7f4cdc19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.643+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4cc00a3ba0 0x7f4cc00a6060 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.643+0000 7f4cdbe00640 1 --2- 172.21.15.18:0/3630385838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cdc104140 0x7f4cdc19a3e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.643+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 >> 172.21.15.18:0/3630385838 conn(0x7f4cdc0ff6d0 msgr2=0x7f4cdc1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.643+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 shutdown_connections 2024-04-16T17:03:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.643+0000 7f4cdbe00640 1 -- 172.21.15.18:0/3630385838 wait complete. 2024-04-16T17:03:53.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:53.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.875+0000 7f7725c00640 1 -- 172.21.15.18:0/4011069124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7720108680 msgr2=0x7f7720108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.875+0000 7f7725c00640 1 --2- 172.21.15.18:0/4011069124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7720108680 0x7f7720108aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7710009900 tx=0x7f771002bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:53.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.876+0000 7f7725c00640 1 -- 172.21.15.18:0/4011069124 shutdown_connections 2024-04-16T17:03:53.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.876+0000 7f7725c00640 1 --2- 172.21.15.18:0/4011069124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f772010a230 0x7f7720110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.876+0000 7f7725c00640 1 --2- 172.21.15.18:0/4011069124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720109870 0x7f7720109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.876+0000 7f7725c00640 1 --2- 172.21.15.18:0/4011069124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7720108680 0x7f7720108aa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.876+0000 7f7725c00640 1 -- 172.21.15.18:0/4011069124 >> 172.21.15.18:0/4011069124 conn(0x7f7720077650 msgr2=0x7f7720079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:53.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.876+0000 7f7725c00640 1 -- 172.21.15.18:0/4011069124 shutdown_connections 2024-04-16T17:03:53.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.876+0000 7f7725c00640 1 -- 172.21.15.18:0/4011069124 wait complete. 2024-04-16T17:03:53.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.877+0000 7f7725c00640 1 Processor -- start 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.877+0000 7f7725c00640 1 -- start start 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f7725c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7720108680 0x7f772019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f7725c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720109870 0x7f772019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f7725c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f772010a230 0x7f77201a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f7725c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7720114340 con 0x7f7720108680 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f7725c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77201141c0 con 0x7f772010a230 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f7725c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77201144c0 con 0x7f7720109870 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f771ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7720108680 0x7f772019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f771ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7720108680 0x7f772019e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37600/0 (socket says 172.21.15.18:37600) 2024-04-16T17:03:53.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f771ea00640 1 -- 172.21.15.18:0/1592846834 learned_addr learned my addr 172.21.15.18:0/1592846834 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f771e000640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720109870 0x7f772019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.878+0000 7f771f400640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f772010a230 0x7f77201a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.879+0000 7f771ea00640 1 -- 172.21.15.18:0/1592846834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720109870 msgr2=0x7f772019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.879+0000 7f771ea00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720109870 0x7f772019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.879+0000 7f771ea00640 1 -- 172.21.15.18:0/1592846834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f772010a230 msgr2=0x7f77201a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.879+0000 7f771ea00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f772010a230 0x7f77201a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.879+0000 7f771ea00640 1 -- 172.21.15.18:0/1592846834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77201a65e0 con 0x7f7720108680 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.879+0000 7f771ea00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7720108680 0x7f772019e9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7710000c00 tx=0x7f771002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.880+0000 7f7717400640 1 -- 172.21.15.18:0/1592846834 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710043020 con 0x7f7720108680 2024-04-16T17:03:53.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.880+0000 7f7717400640 1 -- 172.21.15.18:0/1592846834 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77100054e0 con 0x7f7720108680 2024-04-16T17:03:53.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.880+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77201a6870 con 0x7f7720108680 2024-04-16T17:03:53.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.880+0000 7f7717400640 1 -- 172.21.15.18:0/1592846834 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710003c40 con 0x7f7720108680 2024-04-16T17:03:53.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.880+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77201a6dc0 con 0x7f7720108680 2024-04-16T17:03:53.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.881+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772010fa20 con 0x7f7720108680 2024-04-16T17:03:53.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.884+0000 7f7717400640 1 -- 172.21.15.18:0/1592846834 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7710046020 con 0x7f7720108680 2024-04-16T17:03:53.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.886+0000 7f7717400640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77000a3bb0 0x7f77000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.886+0000 7f7717400640 1 -- 172.21.15.18:0/1592846834 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f77100f27d0 con 0x7f7720108680 2024-04-16T17:03:53.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.886+0000 7f771e000640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77000a3bb0 0x7f77000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.887+0000 7f771e000640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77000a3bb0 0x7f77000a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f772019fed0 tx=0x7f7708009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:53.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:53.891+0000 7f7717400640 1 -- 172.21.15.18:0/1592846834 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f771004d020 con 0x7f7720108680 2024-04-16T17:03:54.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.098+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7720062c00 con 0x7f7720108680 2024-04-16T17:03:54.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.103+0000 7f7717400640 1 -- 172.21.15.18:0/1592846834 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24748 (secure 0 0 0) 0x7f77100bed80 con 0x7f7720108680 2024-04-16T17:03:54.105 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:54.106 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":93,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:03:27.035238+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:54.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.106+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77000a3bb0 msgr2=0x7f77000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.106+0000 7f7725c00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77000a3bb0 0x7f77000a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f772019fed0 tx=0x7f7708009500 comp rx=0 tx=0).stop 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.106+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7720108680 msgr2=0x7f772019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.106+0000 7f7725c00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7720108680 0x7f772019e9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7710000c00 tx=0x7f771002c650 comp rx=0 tx=0).stop 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.107+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 shutdown_connections 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.107+0000 7f7725c00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f772010a230 0x7f77201a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.107+0000 7f7725c00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720109870 0x7f772019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.107+0000 7f7725c00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f77000a3bb0 0x7f77000a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.107+0000 7f7725c00640 1 --2- 172.21.15.18:0/1592846834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7720108680 0x7f772019e9c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.107+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 >> 172.21.15.18:0/1592846834 conn(0x7f7720077650 msgr2=0x7f772010cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.107+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 shutdown_connections 2024-04-16T17:03:54.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.107+0000 7f7725c00640 1 -- 172.21.15.18:0/1592846834 wait complete. 2024-04-16T17:03:54.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:03:54.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:03:54.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 -- 172.21.15.18:0/3220535314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98109650 msgr2=0x7fdf98109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 --2- 172.21.15.18:0/3220535314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98109650 0x7fdf98109ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009900 tx=0x7fdf8402bc50 comp rx=0 tx=0).stop 2024-04-16T17:03:54.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 -- 172.21.15.18:0/3220535314 shutdown_connections 2024-04-16T17:03:54.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 --2- 172.21.15.18:0/3220535314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9810a010 0x7fdf981108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 --2- 172.21.15.18:0/3220535314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98109650 0x7fdf98109ad0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 --2- 172.21.15.18:0/3220535314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf98108460 0x7fdf98108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 -- 172.21.15.18:0/3220535314 >> 172.21.15.18:0/3220535314 conn(0x7fdf98077440 msgr2=0x7fdf98079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:54.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 -- 172.21.15.18:0/3220535314 shutdown_connections 2024-04-16T17:03:54.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.351+0000 7fdf97e00640 1 -- 172.21.15.18:0/3220535314 wait complete. 2024-04-16T17:03:54.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.352+0000 7fdf97e00640 1 Processor -- start 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97e00640 1 -- start start 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf98108460 0x7fdf9819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98109650 0x7fdf9819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf9810a010 0x7fdf981a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdf98114140 con 0x7fdf9810a010 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf98113fc0 con 0x7fdf98108460 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981142c0 con 0x7fdf98109650 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf9810a010 0x7fdf981a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf96000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98109650 0x7fdf9819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf98108460 0x7fdf9819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf98108460 0x7fdf9819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42912/0 (socket says 172.21.15.18:42912) 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf96a00640 1 -- 172.21.15.18:0/2665827524 learned_addr learned my addr 172.21.15.18:0/2665827524 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:54.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.353+0000 7fdf97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf9810a010 0x7fdf981a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37626/0 (socket says 172.21.15.18:37626) 2024-04-16T17:03:54.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.354+0000 7fdf96a00640 1 -- 172.21.15.18:0/2665827524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98109650 msgr2=0x7fdf9819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.354+0000 7fdf96a00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98109650 0x7fdf9819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.354+0000 7fdf96a00640 1 -- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf9810a010 msgr2=0x7fdf981a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.354+0000 7fdf96a00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf9810a010 0x7fdf981a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.354+0000 7fdf96a00640 1 -- 172.21.15.18:0/2665827524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf981a6370 con 0x7fdf98108460 2024-04-16T17:03:54.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.354+0000 7fdf96a00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf98108460 0x7fdf9819e750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800e9c0 tx=0x7fdf8800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:54.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.355+0000 7fdf8f400640 1 -- 172.21.15.18:0/2665827524 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800cd80 con 0x7fdf98108460 2024-04-16T17:03:54.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.355+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf981a6660 con 0x7fdf98108460 2024-04-16T17:03:54.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.355+0000 7fdf8f400640 1 -- 172.21.15.18:0/2665827524 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf880035f0 con 0x7fdf98108460 2024-04-16T17:03:54.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.355+0000 7fdf8f400640 1 -- 172.21.15.18:0/2665827524 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88010690 con 0x7fdf98108460 2024-04-16T17:03:54.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.355+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf981a6b80 con 0x7fdf98108460 2024-04-16T17:03:54.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.356+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9810f720 con 0x7fdf98108460 2024-04-16T17:03:54.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.358+0000 7fdf8f400640 1 -- 172.21.15.18:0/2665827524 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf880037a0 con 0x7fdf98108460 2024-04-16T17:03:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.359+0000 7fdf8f400640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf7c0a3ba0 0x7fdf7c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.359+0000 7fdf8f400640 1 -- 172.21.15.18:0/2665827524 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fdf880d0c10 con 0x7fdf98108460 2024-04-16T17:03:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.360+0000 7fdf96000640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf7c0a3ba0 0x7fdf7c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.360+0000 7fdf96000640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf7c0a3ba0 0x7fdf7c0a6060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdf8402c260 tx=0x7fdf84000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:54.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.364+0000 7fdf8f400640 1 -- 172.21.15.18:0/2665827524 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf8809d1c0 con 0x7fdf98108460 2024-04-16T17:03:54.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:54 smithi022 ceph-mon[36023]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:54.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:54 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3630385838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:54.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:54 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1592846834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:54 smithi018 ceph-mon[31776]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:54 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3630385838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:54.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:54 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1592846834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:54 smithi043 ceph-mon[36901]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:03:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:54 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3630385838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:54 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1592846834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:54.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.570+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf98062c00 con 0x7fdf98108460 2024-04-16T17:03:54.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.571+0000 7fdf8f400640 1 -- 172.21.15.18:0/2665827524 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fdf8809cac0 con 0x7fdf98108460 2024-04-16T17:03:54.573 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf7c0a3ba0 msgr2=0x7fdf7c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf7c0a3ba0 0x7fdf7c0a6060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdf8402c260 tx=0x7fdf84000bc0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf98108460 msgr2=0x7fdf9819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf98108460 0x7fdf9819e750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800e9c0 tx=0x7fdf8800ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 shutdown_connections 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fdf7c0a3ba0 0x7fdf7c0a6060 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf9810a010 0x7fdf981a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98109650 0x7fdf9819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 --2- 172.21.15.18:0/2665827524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf98108460 0x7fdf9819e750 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 >> 172.21.15.18:0/2665827524 conn(0x7fdf98077440 msgr2=0x7fdf9810c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 shutdown_connections 2024-04-16T17:03:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.574+0000 7fdf97e00640 1 -- 172.21.15.18:0/2665827524 wait complete. 2024-04-16T17:03:54.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:03:54.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.808+0000 7f57dd600640 1 -- 172.21.15.18:0/2504343503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d81028e0 msgr2=0x7f57d8104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.808+0000 7f57dd600640 1 --2- 172.21.15.18:0/2504343503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d81028e0 0x7f57d8104cd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009900 tx=0x7f57c802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:54.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.809+0000 7f57dd600640 1 -- 172.21.15.18:0/2504343503 shutdown_connections 2024-04-16T17:03:54.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.809+0000 7f57dd600640 1 --2- 172.21.15.18:0/2504343503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57d8107b40 0x7f57d8109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.809+0000 7f57dd600640 1 --2- 172.21.15.18:0/2504343503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d8105210 0x7f57d8107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.809+0000 7f57dd600640 1 --2- 172.21.15.18:0/2504343503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d81028e0 0x7f57d8104cd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.809+0000 7f57dd600640 1 -- 172.21.15.18:0/2504343503 >> 172.21.15.18:0/2504343503 conn(0x7f57d80fc330 msgr2=0x7f57d80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:54.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.809+0000 7f57dd600640 1 -- 172.21.15.18:0/2504343503 shutdown_connections 2024-04-16T17:03:54.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.809+0000 7f57dd600640 1 -- 172.21.15.18:0/2504343503 wait complete. 2024-04-16T17:03:54.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dd600640 1 Processor -- start 2024-04-16T17:03:54.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dd600640 1 -- start start 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d81028e0 0x7f57d81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dd600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d8105210 0x7f57d81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dd600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57d8107b40 0x7f57d819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dd600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810db00 con 0x7f57d8105210 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dd600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810d980 con 0x7f57d81028e0 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dd600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810dc80 con 0x7f57d8107b40 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57cfe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d8105210 0x7f57d81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57cfe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d8105210 0x7f57d81986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37638/0 (socket says 172.21.15.18:37638) 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57cfe00640 1 -- 172.21.15.18:0/4113075302 learned_addr learned my addr 172.21.15.18:0/4113075302 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57d7e00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d81028e0 0x7f57d81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.810+0000 7f57dcc00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57d8107b40 0x7f57d819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.811+0000 7f57cfe00640 1 -- 172.21.15.18:0/4113075302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57d8107b40 msgr2=0x7f57d819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.811+0000 7f57cfe00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57d8107b40 0x7f57d819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.811+0000 7f57cfe00640 1 -- 172.21.15.18:0/4113075302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d81028e0 msgr2=0x7f57d81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.811+0000 7f57cfe00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d81028e0 0x7f57d81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.811+0000 7f57cfe00640 1 -- 172.21.15.18:0/4113075302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d819fdc0 con 0x7f57d8105210 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.811+0000 7f57d7e00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d81028e0 0x7f57d81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.811+0000 7f57dcc00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57d8107b40 0x7f57d819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.811+0000 7f57cfe00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d8105210 0x7f57d81986e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57c000e9a0 tx=0x7f57c000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.812+0000 7f57d5600640 1 -- 172.21.15.18:0/4113075302 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c000cd80 con 0x7f57d8105210 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.812+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d81a00b0 con 0x7f57d8105210 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.812+0000 7f57d5600640 1 -- 172.21.15.18:0/4113075302 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c00035f0 con 0x7f57d8105210 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.812+0000 7f57d5600640 1 -- 172.21.15.18:0/4113075302 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0010670 con 0x7f57d8105210 2024-04-16T17:03:54.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.812+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81a0600 con 0x7f57d8105210 2024-04-16T17:03:54.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.813+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d80fde20 con 0x7f57d8105210 2024-04-16T17:03:54.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.815+0000 7f57d5600640 1 -- 172.21.15.18:0/4113075302 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f57c00037a0 con 0x7f57d8105210 2024-04-16T17:03:54.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.817+0000 7f57d5600640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f57bc0a3bb0 0x7f57bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.818+0000 7f57d5600640 1 -- 172.21.15.18:0/4113075302 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f57c00d0ab0 con 0x7f57d8105210 2024-04-16T17:03:54.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.818+0000 7f57d7e00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f57bc0a3bb0 0x7f57bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.819+0000 7f57d7e00640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f57bc0a3bb0 0x7f57bc0a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f57c802c7b0 tx=0x7f57c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:54.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:54.822+0000 7f57d5600640 1 -- 172.21.15.18:0/4113075302 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57c009d060 con 0x7f57d8105210 2024-04-16T17:03:55.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.029+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f57d8062c00 con 0x7f57d8105210 2024-04-16T17:03:55.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.030+0000 7f57d5600640 1 -- 172.21.15.18:0/4113075302 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f57c009c960 con 0x7f57d8105210 2024-04-16T17:03:55.032 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:03:55.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f57bc0a3bb0 msgr2=0x7f57bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f57bc0a3bb0 0x7f57bc0a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f57c802c7b0 tx=0x7f57c8005fd0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d8105210 msgr2=0x7f57d81986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d8105210 0x7f57d81986e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57c000e9a0 tx=0x7f57c000ee60 comp rx=0 tx=0).stop 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 shutdown_connections 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57d8107b40 0x7f57d819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f57bc0a3bb0 0x7f57bc0a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d8105210 0x7f57d81986e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 --2- 172.21.15.18:0/4113075302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57d81028e0 0x7f57d81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.033+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 >> 172.21.15.18:0/4113075302 conn(0x7f57d80fc330 msgr2=0x7f57d8103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.034+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 shutdown_connections 2024-04-16T17:03:55.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.034+0000 7f57dd600640 1 -- 172.21.15.18:0/4113075302 wait complete. 2024-04-16T17:03:55.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:03:55.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 -- 172.21.15.18:0/1216646907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274105e10 msgr2=0x7f627410c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 --2- 172.21.15.18:0/1216646907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274105e10 0x7f627410c6a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f626c009ca0 tx=0x7f626c02c100 comp rx=0 tx=0).stop 2024-04-16T17:03:55.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 -- 172.21.15.18:0/1216646907 shutdown_connections 2024-04-16T17:03:55.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 --2- 172.21.15.18:0/1216646907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274105e10 0x7f627410c6a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 --2- 172.21.15.18:0/1216646907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6274105450 0x7f62741058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 --2- 172.21.15.18:0/1216646907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6274104260 0x7f6274104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 -- 172.21.15.18:0/1216646907 >> 172.21.15.18:0/1216646907 conn(0x7f62740ff7f0 msgr2=0x7f6274101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 -- 172.21.15.18:0/1216646907 shutdown_connections 2024-04-16T17:03:55.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.274+0000 7f627b600640 1 -- 172.21.15.18:0/1216646907 wait complete. 2024-04-16T17:03:55.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.275+0000 7f627b600640 1 Processor -- start 2024-04-16T17:03:55.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.275+0000 7f627b600640 1 -- start start 2024-04-16T17:03:55.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.275+0000 7f627b600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274104260 0x7f627419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.275+0000 7f627b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6274105450 0x7f627419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.275+0000 7f627b600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6274105e10 0x7f62741a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.275+0000 7f627b600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f627410fec0 con 0x7f6274105e10 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.275+0000 7f627b600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f627410fd40 con 0x7f6274104260 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.275+0000 7f627b600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6274110040 con 0x7f6274105450 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6278c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274104260 0x7f627419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6273e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6274105450 0x7f627419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6278c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274104260 0x7f627419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42948/0 (socket says 172.21.15.18:42948) 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6279600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6274105e10 0x7f62741a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6278c00640 1 -- 172.21.15.18:0/774979023 learned_addr learned my addr 172.21.15.18:0/774979023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6273e00640 1 -- 172.21.15.18:0/774979023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274104260 msgr2=0x7f627419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6273e00640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274104260 0x7f627419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6273e00640 1 -- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6274105e10 msgr2=0x7f62741a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6273e00640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6274105e10 0x7f62741a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6273e00640 1 -- 172.21.15.18:0/774979023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62741a2150 con 0x7f6274105450 2024-04-16T17:03:55.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.276+0000 7f6273e00640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6274105450 0x7f627419aa70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f626400e9f0 tx=0x7f626400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:55.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.277+0000 7f6271600640 1 -- 172.21.15.18:0/774979023 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626400ccc0 con 0x7f6274105450 2024-04-16T17:03:55.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.277+0000 7f6271600640 1 -- 172.21.15.18:0/774979023 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264010910 con 0x7f6274105450 2024-04-16T17:03:55.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.277+0000 7f6271600640 1 -- 172.21.15.18:0/774979023 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264018960 con 0x7f6274105450 2024-04-16T17:03:55.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.277+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62741a2440 con 0x7f6274105450 2024-04-16T17:03:55.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.277+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62741a2940 con 0x7f6274105450 2024-04-16T17:03:55.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.278+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f627410b580 con 0x7f6274105450 2024-04-16T17:03:55.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.280+0000 7f6271600640 1 -- 172.21.15.18:0/774979023 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6264010ac0 con 0x7f6274105450 2024-04-16T17:03:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.281+0000 7f6271600640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f625c0a3c30 0x7f625c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.282+0000 7f6271600640 1 -- 172.21.15.18:0/774979023 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f62640d0b40 con 0x7f6274105450 2024-04-16T17:03:55.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.282+0000 7f6278c00640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f625c0a3c30 0x7f625c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.282+0000 7f6278c00640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f625c0a3c30 0x7f625c0a60f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62680058c0 tx=0x7f6268005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:55.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.284+0000 7f6271600640 1 -- 172.21.15.18:0/774979023 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f626409caa0 con 0x7f6274105450 2024-04-16T17:03:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2665827524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4113075302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:55.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.483+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f627402d7b0 con 0x7f6274105450 2024-04-16T17:03:55.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.484+0000 7f6271600640 1 -- 172.21.15.18:0/774979023 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f62640a0da0 con 0x7f6274105450 2024-04-16T17:03:55.485 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:03:55.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.486+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f625c0a3c30 msgr2=0x7f625c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.486+0000 7f627b600640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f625c0a3c30 0x7f625c0a60f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62680058c0 tx=0x7f6268005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.486+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6274105450 msgr2=0x7f627419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.486+0000 7f627b600640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6274105450 0x7f627419aa70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f626400e9f0 tx=0x7f626400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.487+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 shutdown_connections 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.487+0000 7f627b600640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f625c0a3c30 0x7f625c0a60f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.487+0000 7f627b600640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6274105e10 0x7f62741a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.487+0000 7f627b600640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6274105450 0x7f627419aa70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.487+0000 7f627b600640 1 --2- 172.21.15.18:0/774979023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274104260 0x7f627419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.487+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 >> 172.21.15.18:0/774979023 conn(0x7f62740ff7f0 msgr2=0x7f6274108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.487+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 shutdown_connections 2024-04-16T17:03:55.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.487+0000 7f627b600640 1 -- 172.21.15.18:0/774979023 wait complete. 2024-04-16T17:03:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2665827524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4113075302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:55.508 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:03:55.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2665827524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:55.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4113075302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:55.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.730+0000 7f4f37800640 1 -- 172.21.15.18:0/4283942328 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30077c00 msgr2=0x7f4f30078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.730+0000 7f4f37800640 1 --2- 172.21.15.18:0/4283942328 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30077c00 0x7f4f30078060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009ca0 tx=0x7f4f1802c100 comp rx=0 tx=0).stop 2024-04-16T17:03:55.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.730+0000 7f4f37800640 1 -- 172.21.15.18:0/4283942328 shutdown_connections 2024-04-16T17:03:55.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.730+0000 7f4f37800640 1 --2- 172.21.15.18:0/4283942328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f300785a0 0x7f4f30078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.730+0000 7f4f37800640 1 --2- 172.21.15.18:0/4283942328 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30077c00 0x7f4f30078060 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.730+0000 7f4f37800640 1 --2- 172.21.15.18:0/4283942328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f300769c0 0x7f4f30076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.730+0000 7f4f37800640 1 -- 172.21.15.18:0/4283942328 >> 172.21.15.18:0/4283942328 conn(0x7f4f300ff7f0 msgr2=0x7f4f30101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.731+0000 7f4f37800640 1 -- 172.21.15.18:0/4283942328 shutdown_connections 2024-04-16T17:03:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.731+0000 7f4f37800640 1 -- 172.21.15.18:0/4283942328 wait complete. 2024-04-16T17:03:55.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.731+0000 7f4f37800640 1 Processor -- start 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f37800640 1 -- start start 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f37800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f300769c0 0x7f4f3019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f37800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30077c00 0x7f4f3019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f37800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f300785a0 0x7f4f301a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f37800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301142f0 con 0x7f4f30077c00 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f37800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30114170 con 0x7f4f300785a0 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f37800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30114470 con 0x7f4f300769c0 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f34e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f300769c0 0x7f4f3019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f34e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f300769c0 0x7f4f3019e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42382/0 (socket says 172.21.15.18:42382) 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f34e00640 1 -- 172.21.15.18:0/3419322779 learned_addr learned my addr 172.21.15.18:0/3419322779 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f2fe00640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30077c00 0x7f4f3019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.732+0000 7f4f35800640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f300785a0 0x7f4f301a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f34e00640 1 -- 172.21.15.18:0/3419322779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f300785a0 msgr2=0x7f4f301a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f34e00640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f300785a0 0x7f4f301a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f34e00640 1 -- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30077c00 msgr2=0x7f4f3019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f34e00640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30077c00 0x7f4f3019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f34e00640 1 -- 172.21.15.18:0/3419322779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f301a65a0 con 0x7f4f300769c0 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f35800640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f300785a0 0x7f4f301a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f34e00640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f300769c0 0x7f4f3019e980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e9f0 tx=0x7f4f2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f2d600640 1 -- 172.21.15.18:0/3419322779 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800cd80 con 0x7f4f300769c0 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f301a6890 con 0x7f4f300769c0 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.733+0000 7f4f2d600640 1 -- 172.21.15.18:0/3419322779 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f28010910 con 0x7f4f300769c0 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.734+0000 7f4f2d600640 1 -- 172.21.15.18:0/3419322779 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f280188f0 con 0x7f4f300769c0 2024-04-16T17:03:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.734+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f301a6e00 con 0x7f4f300769c0 2024-04-16T17:03:55.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.735+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f30110d50 con 0x7f4f300769c0 2024-04-16T17:03:55.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.737+0000 7f4f2d600640 1 -- 172.21.15.18:0/3419322779 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4f28010ac0 con 0x7f4f300769c0 2024-04-16T17:03:55.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.738+0000 7f4f2d600640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4f140a3c30 0x7f4f140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.738+0000 7f4f2d600640 1 -- 172.21.15.18:0/3419322779 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4f280d0be0 con 0x7f4f300769c0 2024-04-16T17:03:55.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.738+0000 7f4f2fe00640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4f140a3c30 0x7f4f140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.739+0000 7f4f2fe00640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4f140a3c30 0x7f4f140a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f18006010 tx=0x7f4f18032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:55.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.742+0000 7f4f2d600640 1 -- 172.21.15.18:0/3419322779 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f2809d190 con 0x7f4f300769c0 2024-04-16T17:03:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.952+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4f30062c30 con 0x7f4f300769c0 2024-04-16T17:03:55.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.953+0000 7f4f2d600640 1 -- 172.21.15.18:0/3419322779 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f4f2809ca90 con 0x7f4f300769c0 2024-04-16T17:03:55.954 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4f140a3c30 msgr2=0x7f4f140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4f140a3c30 0x7f4f140a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f18006010 tx=0x7f4f18032590 comp rx=0 tx=0).stop 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f300769c0 msgr2=0x7f4f3019e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f300769c0 0x7f4f3019e980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e9f0 tx=0x7f4f2800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 shutdown_connections 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f300785a0 0x7f4f301a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4f140a3c30 0x7f4f140a60f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f30077c00 0x7f4f3019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 --2- 172.21.15.18:0/3419322779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f300769c0 0x7f4f3019e980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 >> 172.21.15.18:0/3419322779 conn(0x7f4f300ff7f0 msgr2=0x7f4f30100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 shutdown_connections 2024-04-16T17:03:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:55.956+0000 7f4f37800640 1 -- 172.21.15.18:0/3419322779 wait complete. 2024-04-16T17:03:55.977 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:56.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 -- 172.21.15.18:0/4067272239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 msgr2=0x7f43ec104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 --2- 172.21.15.18:0/4067272239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 0x7f43ec104690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009ca0 tx=0x7f43e402c100 comp rx=0 tx=0).stop 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 -- 172.21.15.18:0/4067272239 shutdown_connections 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 --2- 172.21.15.18:0/4067272239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43ec105e20 0x7f43ec10c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 --2- 172.21.15.18:0/4067272239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43ec105460 0x7f43ec1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 --2- 172.21.15.18:0/4067272239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 0x7f43ec104690 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 -- 172.21.15.18:0/4067272239 >> 172.21.15.18:0/4067272239 conn(0x7f43ec0ff820 msgr2=0x7f43ec101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 -- 172.21.15.18:0/4067272239 shutdown_connections 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.203+0000 7f43f1c00640 1 -- 172.21.15.18:0/4067272239 wait complete. 2024-04-16T17:03:56.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.204+0000 7f43f1c00640 1 Processor -- start 2024-04-16T17:03:56.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.204+0000 7f43f1c00640 1 -- start start 2024-04-16T17:03:56.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.204+0000 7f43f1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 0x7f43ec19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43f1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43ec105460 0x7f43ec19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43f1c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43ec105e20 0x7f43ec1a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43f1c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec10ff00 con 0x7f43ec105e20 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43f1c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec10fd80 con 0x7f43ec105460 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43f1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec110080 con 0x7f43ec104270 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 0x7f43ec19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 0x7f43ec19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42402/0 (socket says 172.21.15.18:42402) 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eaa00640 1 -- 172.21.15.18:0/1201874904 learned_addr learned my addr 172.21.15.18:0/1201874904 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eb400640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43ec105e20 0x7f43ec1a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43ea000640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43ec105460 0x7f43ec19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eaa00640 1 -- 172.21.15.18:0/1201874904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43ec105460 msgr2=0x7f43ec19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eaa00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43ec105460 0x7f43ec19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eaa00640 1 -- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43ec105e20 msgr2=0x7f43ec1a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eaa00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43ec105e20 0x7f43ec1a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eaa00640 1 -- 172.21.15.18:0/1201874904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec1a21e0 con 0x7f43ec104270 2024-04-16T17:03:56.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.205+0000 7f43eb400640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43ec105e20 0x7f43ec1a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:56.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.206+0000 7f43eaa00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 0x7f43ec19a5c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000f80 tx=0x7f43e4035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:56.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.206+0000 7f43e3400640 1 -- 172.21.15.18:0/1201874904 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4039040 con 0x7f43ec104270 2024-04-16T17:03:56.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.206+0000 7f43e3400640 1 -- 172.21.15.18:0/1201874904 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e4044070 con 0x7f43ec104270 2024-04-16T17:03:56.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.206+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec1a2470 con 0x7f43ec104270 2024-04-16T17:03:56.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.206+0000 7f43e3400640 1 -- 172.21.15.18:0/1201874904 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e40044a0 con 0x7f43ec104270 2024-04-16T17:03:56.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.206+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec1a2980 con 0x7f43ec104270 2024-04-16T17:03:56.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.207+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ec10b610 con 0x7f43ec104270 2024-04-16T17:03:56.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.209+0000 7f43e3400640 1 -- 172.21.15.18:0/1201874904 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f43e4004640 con 0x7f43ec104270 2024-04-16T17:03:56.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.211+0000 7f43e3400640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43d00a3c30 0x7f43d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.211+0000 7f43e3400640 1 -- 172.21.15.18:0/1201874904 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f43e40f2bf0 con 0x7f43ec104270 2024-04-16T17:03:56.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.211+0000 7f43ea000640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43d00a3c30 0x7f43d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.212+0000 7f43ea000640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43d00a3c30 0x7f43d00a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f43ec19bad0 tx=0x7f43d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:56.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.216+0000 7f43e3400640 1 -- 172.21.15.18:0/1201874904 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43e404e020 con 0x7f43ec104270 2024-04-16T17:03:56.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:56 smithi018 ceph-mon[31776]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:56.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:56 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/774979023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:56.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:56 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3419322779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:56.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.464+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43ec02d620 con 0x7f43ec104270 2024-04-16T17:03:56.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.466+0000 7f43e3400640 1 -- 172.21.15.18:0/1201874904 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11848 (secure 0 0 0) 0x7f43e40bf1a0 con 0x7f43ec104270 2024-04-16T17:03:56.467 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:56.468 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:01:59.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:03:56.468 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 24 2024-04-16T17:03:56.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.468+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43d00a3c30 msgr2=0x7f43d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.468+0000 7f43f1c00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43d00a3c30 0x7f43d00a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f43ec19bad0 tx=0x7f43d8005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:56.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.468+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 msgr2=0x7f43ec19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.468+0000 7f43f1c00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 0x7f43ec19a5c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000f80 tx=0x7f43e4035d10 comp rx=0 tx=0).stop 2024-04-16T17:03:56.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.469+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 shutdown_connections 2024-04-16T17:03:56.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.469+0000 7f43f1c00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f43d00a3c30 0x7f43d00a60f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.469+0000 7f43f1c00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43ec105e20 0x7f43ec1a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.469+0000 7f43f1c00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43ec105460 0x7f43ec19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.469+0000 7f43f1c00640 1 --2- 172.21.15.18:0/1201874904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec104270 0x7f43ec19a5c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.469+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 >> 172.21.15.18:0/1201874904 conn(0x7f43ec0ff820 msgr2=0x7f43ec1087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.469+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 shutdown_connections 2024-04-16T17:03:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.469+0000 7f43f1c00640 1 -- 172.21.15.18:0/1201874904 wait complete. 2024-04-16T17:03:56.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:56.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:56 smithi043 ceph-mon[36901]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:56.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:56 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/774979023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:56.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:56 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3419322779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:56.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:56 smithi022 ceph-mon[36023]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:03:56.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:56 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/774979023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:56.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:56 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3419322779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 -- 172.21.15.18:0/3005091016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 msgr2=0x7f2984104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 --2- 172.21.15.18:0/3005091016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 0x7f2984104db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2978009900 tx=0x7f297802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 -- 172.21.15.18:0/3005091016 shutdown_connections 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 --2- 172.21.15.18:0/3005091016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2984107c20 0x7f298410a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 --2- 172.21.15.18:0/3005091016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29841052f0 0x7f29841076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 --2- 172.21.15.18:0/3005091016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 0x7f2984104db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 -- 172.21.15.18:0/3005091016 >> 172.21.15.18:0/3005091016 conn(0x7f29840fc3d0 msgr2=0x7f29840fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 -- 172.21.15.18:0/3005091016 shutdown_connections 2024-04-16T17:03:56.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.693+0000 7f2989e00640 1 -- 172.21.15.18:0/3005091016 wait complete. 2024-04-16T17:03:56.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.694+0000 7f2989e00640 1 Processor -- start 2024-04-16T17:03:56.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.694+0000 7f2989e00640 1 -- start start 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2989e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 0x7f298419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2989e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29841052f0 0x7f298419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2989e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2984107c20 0x7f29841a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2989e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f298410dc20 con 0x7f29841029c0 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2989e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f298410daa0 con 0x7f2984107c20 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2989e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f298410dda0 con 0x7f29841052f0 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2988a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 0x7f298419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2988a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 0x7f298419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37728/0 (socket says 172.21.15.18:37728) 2024-04-16T17:03:56.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2988a00640 1 -- 172.21.15.18:0/2894074516 learned_addr learned my addr 172.21.15.18:0/2894074516 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2983e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29841052f0 0x7f298419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.695+0000 7f2989400640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2984107c20 0x7f29841a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2988a00640 1 -- 172.21.15.18:0/2894074516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29841052f0 msgr2=0x7f298419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2988a00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29841052f0 0x7f298419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2988a00640 1 -- 172.21.15.18:0/2894074516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2984107c20 msgr2=0x7f29841a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2988a00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2984107c20 0x7f29841a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2988a00640 1 -- 172.21.15.18:0/2894074516 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29841a2090 con 0x7f29841029c0 2024-04-16T17:03:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2988a00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 0x7f298419a470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2978000c00 tx=0x7f297802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:56.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2981600640 1 -- 172.21.15.18:0/2894074516 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978043020 con 0x7f29841029c0 2024-04-16T17:03:56.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2981600640 1 -- 172.21.15.18:0/2894074516 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29780054e0 con 0x7f29841029c0 2024-04-16T17:03:56.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29841a2320 con 0x7f29841029c0 2024-04-16T17:03:56.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2981600640 1 -- 172.21.15.18:0/2894074516 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978003c40 con 0x7f29841029c0 2024-04-16T17:03:56.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.696+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841a28c0 con 0x7f29841029c0 2024-04-16T17:03:56.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.697+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2984109770 con 0x7f29841029c0 2024-04-16T17:03:56.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.699+0000 7f2981600640 1 -- 172.21.15.18:0/2894074516 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2978046020 con 0x7f29841029c0 2024-04-16T17:03:56.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.700+0000 7f2981600640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f296c0a3c20 0x7f296c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.701+0000 7f2981600640 1 -- 172.21.15.18:0/2894074516 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f29780f27d0 con 0x7f29841029c0 2024-04-16T17:03:56.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.701+0000 7f2983e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f296c0a3c20 0x7f296c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.701+0000 7f2983e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f296c0a3c20 0x7f296c0a60e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f298419b980 tx=0x7f2970005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:56.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.705+0000 7f2981600640 1 -- 172.21.15.18:0/2894074516 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f297804d020 con 0x7f29841029c0 2024-04-16T17:03:56.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.905+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2984062c00 con 0x7f29841029c0 2024-04-16T17:03:56.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.909+0000 7f2981600640 1 -- 172.21.15.18:0/2894074516 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24748 (secure 0 0 0) 0x7f29780bed80 con 0x7f29841029c0 2024-04-16T17:03:56.911 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:56.911 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":93,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:03:27.035238+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:56.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f296c0a3c20 msgr2=0x7f296c0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f296c0a3c20 0x7f296c0a60e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f298419b980 tx=0x7f2970005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:56.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 msgr2=0x7f298419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 0x7f298419a470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2978000c00 tx=0x7f297802c650 comp rx=0 tx=0).stop 2024-04-16T17:03:56.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 shutdown_connections 2024-04-16T17:03:56.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2984107c20 0x7f29841a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29841052f0 0x7f298419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f296c0a3c20 0x7f296c0a60e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 --2- 172.21.15.18:0/2894074516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29841029c0 0x7f298419a470 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.911+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 >> 172.21.15.18:0/2894074516 conn(0x7f29840fc3d0 msgr2=0x7f2984102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:56.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.912+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 shutdown_connections 2024-04-16T17:03:56.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:56.912+0000 7f2989e00640 1 -- 172.21.15.18:0/2894074516 wait complete. 2024-04-16T17:03:56.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:57.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 -- 172.21.15.18:0/3864500185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990103a40 msgr2=0x7fb99010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 --2- 172.21.15.18:0/3864500185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990103a40 0x7fb99010a360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb988009ca0 tx=0x7fb98802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 -- 172.21.15.18:0/3864500185 shutdown_connections 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 --2- 172.21.15.18:0/3864500185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990103a40 0x7fb99010a360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 --2- 172.21.15.18:0/3864500185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb990103080 0x7fb990103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 --2- 172.21.15.18:0/3864500185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb990101f30 0x7fb990102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 -- 172.21.15.18:0/3864500185 >> 172.21.15.18:0/3864500185 conn(0x7fb9900fd4c0 msgr2=0x7fb9900ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 -- 172.21.15.18:0/3864500185 shutdown_connections 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.114+0000 7fb996c00640 1 -- 172.21.15.18:0/3864500185 wait complete. 2024-04-16T17:03:57.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.115+0000 7fb996c00640 1 Processor -- start 2024-04-16T17:03:57.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb996c00640 1 -- start start 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb996c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990101f30 0x7fb990075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb996c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb990103080 0x7fb9900743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb996c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb990103a40 0x7fb990074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb996c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010dbc0 con 0x7fb990101f30 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb996c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010da40 con 0x7fb990103080 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb996c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010dd40 con 0x7fb990103a40 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb995800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990101f30 0x7fb990075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb994e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb990103080 0x7fb9900743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb994e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb990103080 0x7fb9900743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59098/0 (socket says 172.21.15.18:59098) 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb994e00640 1 -- 172.21.15.18:0/783504979 learned_addr learned my addr 172.21.15.18:0/783504979 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb995800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990101f30 0x7fb990075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43830/0 (socket says 172.21.15.18:43830) 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.116+0000 7fb996200640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb990103a40 0x7fb990074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb995800640 1 -- 172.21.15.18:0/783504979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb990103a40 msgr2=0x7fb990074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb995800640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb990103a40 0x7fb990074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb995800640 1 -- 172.21.15.18:0/783504979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb990103080 msgr2=0x7fb9900743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb995800640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb990103080 0x7fb9900743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb995800640 1 -- 172.21.15.18:0/783504979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9900751c0 con 0x7fb990101f30 2024-04-16T17:03:57.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb994e00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb990103080 0x7fb9900743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:57.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb995800640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990101f30 0x7fb990075db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb98400e9f0 tx=0x7fb98400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb98e000640 1 -- 172.21.15.18:0/783504979 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98400ccc0 con 0x7fb990101f30 2024-04-16T17:03:57.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb98e000640 1 -- 172.21.15.18:0/783504979 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb984010910 con 0x7fb990101f30 2024-04-16T17:03:57.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9901a22f0 con 0x7fb990101f30 2024-04-16T17:03:57.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb98e000640 1 -- 172.21.15.18:0/783504979 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984018960 con 0x7fb990101f30 2024-04-16T17:03:57.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.117+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9901a2890 con 0x7fb990101f30 2024-04-16T17:03:57.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.119+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9901091e0 con 0x7fb990101f30 2024-04-16T17:03:57.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.120+0000 7fb98e000640 1 -- 172.21.15.18:0/783504979 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb984010ac0 con 0x7fb990101f30 2024-04-16T17:03:57.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.122+0000 7fb98e000640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9780a3c20 0x7fb9780a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.122+0000 7fb98e000640 1 -- 172.21.15.18:0/783504979 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fb9840d0b30 con 0x7fb990101f30 2024-04-16T17:03:57.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.122+0000 7fb994e00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9780a3c20 0x7fb9780a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.123+0000 7fb994e00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9780a3c20 0x7fb9780a60e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb990075b20 tx=0x7fb97c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.127+0000 7fb98e000640 1 -- 172.21.15.18:0/783504979 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb98409ca90 con 0x7fb990101f30 2024-04-16T17:03:57.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.377+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb99002d620 con 0x7fb990101f30 2024-04-16T17:03:57.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.379+0000 7fb98e000640 1 -- 172.21.15.18:0/783504979 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11848 (secure 0 0 0) 0x7fb9840a0d90 con 0x7fb990101f30 2024-04-16T17:03:57.380 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:03:57.380 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:01:59.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:03:57.381 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 24 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9780a3c20 msgr2=0x7fb9780a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9780a3c20 0x7fb9780a60e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb990075b20 tx=0x7fb97c005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990101f30 msgr2=0x7fb990075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990101f30 0x7fb990075db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb98400e9f0 tx=0x7fb98400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 shutdown_connections 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb990103a40 0x7fb990074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb990103080 0x7fb9900743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fb9780a3c20 0x7fb9780a60e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 --2- 172.21.15.18:0/783504979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb990101f30 0x7fb990075db0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.381+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 >> 172.21.15.18:0/783504979 conn(0x7fb9900fd4c0 msgr2=0x7fb9901063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.382+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 shutdown_connections 2024-04-16T17:03:57.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.382+0000 7fb996c00640 1 -- 172.21.15.18:0/783504979 wait complete. 2024-04-16T17:03:57.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:57 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1201874904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:57 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2894074516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-16T17:01:24.398529+0000', 'modified': '2024-04-16T17:01:59.165885+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24596, 'mds_1': 24566, 'mds_2': 24547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/1265446335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 1265446335}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 1265446335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6837/1790976373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6836', 'nonce': 1790976373}, {'type': 'v1', 'addr': '172.21.15.43:6837', 'nonce': 1790976373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.22:6837/94342022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6836', 'nonce': 94342022}, {'type': 'v1', 'addr': '172.21.15.22:6837', 'nonce': 94342022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.43:6839/3993514244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6838', 'nonce': 3993514244}, {'type': 'v1', 'addr': '172.21.15.43:6839', 'nonce': 3993514244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.18:6839/294785837', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6838', 'nonce': 294785837}, {'type': 'v1', 'addr': '172.21.15.18:6839', 'nonce': 294785837}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6839/836242731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6838', 'nonce': 836242731}, {'type': 'v1', 'addr': '172.21.15.22:6839', 'nonce': 836242731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24566, 24547, 24596]} 2024-04-16T17:03:57.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-16T17:03:57.403 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-16T17:03:57.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:57 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1201874904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:57 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2894074516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 -- 172.21.15.18:0/754089480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec104080 msgr2=0x7f38ec1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 --2- 172.21.15.18:0/754089480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec104080 0x7f38ec1044a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009900 tx=0x7f38dc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 -- 172.21.15.18:0/754089480 shutdown_connections 2024-04-16T17:03:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 --2- 172.21.15.18:0/754089480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38ec105c30 0x7f38ec10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 --2- 172.21.15.18:0/754089480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec105270 0x7f38ec1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 --2- 172.21.15.18:0/754089480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec104080 0x7f38ec1044a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 -- 172.21.15.18:0/754089480 >> 172.21.15.18:0/754089480 conn(0x7f38ec0ff610 msgr2=0x7f38ec101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 -- 172.21.15.18:0/754089480 shutdown_connections 2024-04-16T17:03:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.604+0000 7f38f1800640 1 -- 172.21.15.18:0/754089480 wait complete. 2024-04-16T17:03:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.605+0000 7f38f1800640 1 Processor -- start 2024-04-16T17:03:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.605+0000 7f38f1800640 1 -- start start 2024-04-16T17:03:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.605+0000 7f38f1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec104080 0x7f38ec19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.605+0000 7f38f1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38ec105270 0x7f38ec19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.605+0000 7f38f1800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105c30 0x7f38ec1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.605+0000 7f38f1800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec10fd60 con 0x7f38ec105c30 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.605+0000 7f38f1800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec10fbe0 con 0x7f38ec105270 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.605+0000 7f38f1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec10fee0 con 0x7f38ec104080 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38ec105270 0x7f38ec19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec104080 0x7f38ec19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38ec105270 0x7f38ec19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59120/0 (socket says 172.21.15.18:59120) 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec104080 0x7f38ec19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44858/0 (socket says 172.21.15.18:44858) 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 -- 172.21.15.18:0/485919932 learned_addr learned my addr 172.21.15.18:0/485919932 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38f0e00640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105c30 0x7f38ec1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 -- 172.21.15.18:0/485919932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec104080 msgr2=0x7f38ec19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec104080 0x7f38ec19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 -- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105c30 msgr2=0x7f38ec1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105c30 0x7f38ec1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 -- 172.21.15.18:0/485919932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec1a1f90 con 0x7f38ec105270 2024-04-16T17:03:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.606+0000 7f38eb400640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38ec105270 0x7f38ec19a8b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38d800e9f0 tx=0x7f38d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.607+0000 7f38e8c00640 1 -- 172.21.15.18:0/485919932 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d800cd80 con 0x7f38ec105270 2024-04-16T17:03:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.607+0000 7f38f1800640 1 -- 172.21.15.18:0/485919932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec1a2280 con 0x7f38ec105270 2024-04-16T17:03:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.607+0000 7f38e8c00640 1 -- 172.21.15.18:0/485919932 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d8010910 con 0x7f38ec105270 2024-04-16T17:03:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.607+0000 7f38e8c00640 1 -- 172.21.15.18:0/485919932 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d80188f0 con 0x7f38ec105270 2024-04-16T17:03:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.607+0000 7f38f1800640 1 -- 172.21.15.18:0/485919932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec1a27d0 con 0x7f38ec105270 2024-04-16T17:03:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f59ff740 1 Processor -- start 2024-04-16T17:03:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f59ff740 1 -- start start 2024-04-16T17:03:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb0464a0 0x55b2cb0468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb04b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb04c000 0x55b2cb04e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f59ff740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b2cafac980 con 0x55b2cb04c000 2024-04-16T17:03:57.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f59ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b2caf53220 con 0x55b2cb0464a0 2024-04-16T17:03:57.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f59ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b2cafa8940 con 0x55b2cb046e90 2024-04-16T17:03:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38e8c00640 1 -- 172.21.15.18:0/485919932 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f38d8010ac0 con 0x7f38ec105270 2024-04-16T17:03:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb04b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.610+0000 7f38f0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb0464a0 0x55b2cb0468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb04b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44866/0 (socket says 172.21.15.18:44866) 2024-04-16T17:03:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38f0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb0464a0 0x55b2cb0468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59124/0 (socket says 172.21.15.18:59124) 2024-04-16T17:03:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38ebe00640 1 -- 172.21.15.18:0/1066577270 learned_addr learned my addr 172.21.15.18:0/1066577270 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38eb400640 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb04c000 0x55b2cb04e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38f0e00640 1 -- 172.21.15.18:0/1066577270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 msgr2=0x55b2cb04b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38f0e00640 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb04b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38f0e00640 1 -- 172.21.15.18:0/1066577270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb04c000 msgr2=0x55b2cb04e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38f0e00640 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb04c000 0x55b2cb04e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38f0e00640 1 -- 172.21.15.18:0/1066577270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2cacbf2d0 con 0x55b2cb0464a0 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38eb400640 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb04c000 0x55b2cb04e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.611+0000 7f38f0e00640 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb0464a0 0x55b2cb0468c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009ed0 tx=0x7f38e40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f078d9455c4ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.612+0000 7f38e1600640 1 -- 172.21.15.18:0/1066577270 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4003b70 con 0x55b2cb0464a0 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.612+0000 7f38e1600640 1 -- 172.21.15.18:0/1066577270 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e4015070 con 0x55b2cb0464a0 2024-04-16T17:03:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.612+0000 7f38e1600640 1 -- 172.21.15.18:0/1066577270 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4011870 con 0x55b2cb0464a0 2024-04-16T17:03:57.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.612+0000 7f38e8c00640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38d00a3bb0 0x7f38d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.612+0000 7f38e8c00640 1 -- 172.21.15.18:0/485919932 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f38d80d0c70 con 0x7f38ec105270 2024-04-16T17:03:57.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 -- 172.21.15.18:0/1066577270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb0464a0 msgr2=0x55b2cb0468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb0464a0 0x55b2cb0468c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009ed0 tx=0x7f38e40107a0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38ebe00640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38d00a3bb0 0x7f38d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 -- 172.21.15.18:0/1066577270 shutdown_connections 2024-04-16T17:03:57.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb04c000 0x55b2cb04e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb04b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1066577270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb0464a0 0x55b2cb0468c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 -- 172.21.15.18:0/1066577270 >> 172.21.15.18:0/1066577270 conn(0x55b2cb041a10 msgr2=0x55b2cb041e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 -- 172.21.15.18:0/1066577270 shutdown_connections 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38f59ff740 1 -- 172.21.15.18:0/1066577270 wait complete. 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.613+0000 7f38ebe00640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38d00a3bb0 0x7f38d00a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b2cb047900 tx=0x7f38dc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 1 -- start start 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 1 Processor -- start 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb0509f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb050f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb057f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb04efb0 con 0x55b2cb0464a0 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b2caf66590 con 0x55b2cb046e90 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb04f130 con 0x55b2cb04c000 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38f59ff740 10 client.0 _ll_register_callbacks cb 0x55b2caf6e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:03:57.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.614+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb0509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb0509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43862/0 (socket says 172.21.15.18:43862) 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38ebe00640 1 -- 172.21.15.18:0/1500101567 learned_addr learned my addr 172.21.15.18:0/1500101567 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38eb400640 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb050f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38f0e00640 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb057f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38ebe00640 1 -- 172.21.15.18:0/1500101567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 msgr2=0x55b2cb057f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38ebe00640 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb057f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38ebe00640 1 -- 172.21.15.18:0/1500101567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 msgr2=0x55b2cb050f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38ebe00640 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb050f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38ebe00640 1 -- 172.21.15.18:0/1500101567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2cad896b0 con 0x55b2cb0464a0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38eb400640 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb050f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38ebe00640 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb0509f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38dc02c170 tx=0x7f38dc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b623c77cf5b29a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.615+0000 7f38d6000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.616+0000 7f38d6000640 1 -- 172.21.15.18:0/1500101567 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc045070 con 0x55b2cb0464a0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.616+0000 7f38d6000640 1 -- 172.21.15.18:0/1500101567 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f38dc0035f0 con 0x55b2cb0464a0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.616+0000 7f38f59ff740 1 -- 172.21.15.18:0/1500101567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38e4015070 con 0x55b2cb0464a0 2024-04-16T17:03:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.616+0000 7f38d6000640 1 -- 172.21.15.18:0/1500101567 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f38dc0037f0 con 0x55b2cb0464a0 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.616+0000 7f38f59ff740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.616+0000 7f38f59ff740 1 -- 172.21.15.18:0/1500101567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b2cb05b2c0 con 0x55b2cb0464a0 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.616+0000 7f38d6000640 1 -- 172.21.15.18:0/1500101567 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e4015070 con 0x55b2cb0464a0 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.617+0000 7f38d6000640 1 -- 172.21.15.18:0/1500101567 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f38dc034b80 con 0x55b2cb0464a0 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.617+0000 7f38f59ff740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.617+0000 7f38f59ff740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.617+0000 7f38f59ff740 20 client.15066 populate_metadata read hostname 'smithi018' 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.617+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b2cb05d990 0x55b2cb05ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.617+0000 7f38f59ff740 4 client.15066 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:03:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.617+0000 7f38f59ff740 1 -- 172.21.15.18:0/1500101567 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2cafba3a0 con 0x55b2cb05d990 2024-04-16T17:03:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.617+0000 7f38eb400640 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b2cb05d990 0x55b2cb05ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.618+0000 7f38eb400640 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b2cb05d990 0x55b2cb05ee00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5d6aa7c494e79bc server_cookie=629efae35c585c05 in_seq=0 out_seq=0 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.618+0000 7f38d6000640 10 client.15066 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38d6000640 1 -- 172.21.15.18:0/1500101567 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2cafba3a0 con 0x55b2cb05d990 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38d6000640 10 client.15066 handle_command_reply: tid=0 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 client.15066 shutdown 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 -- 172.21.15.18:0/1500101567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 msgr2=0x55b2cb0509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb0509f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38dc02c170 tx=0x7f38dc009260 comp rx=0 tx=0).stop 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 -- 172.21.15.18:0/1500101567 shutdown_connections 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b2cb05d990 0x55b2cb05ee00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb057f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb050f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1500101567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb0509f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 -- 172.21.15.18:0/1500101567 >> 172.21.15.18:0/1500101567 conn(0x55b2cb041a10 msgr2=0x55b2cb040c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38d6000640 0 client.15066 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 -- 172.21.15.18:0/1500101567 shutdown_connections 2024-04-16T17:03:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 1 -- 172.21.15.18:0/1500101567 wait complete. 2024-04-16T17:03:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.619+0000 7f38f59ff740 20 client.15066 trim_cache size 0 max 16384 2024-04-16T17:03:57.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.649+0000 7f38f59ff740 1 Processor -- start 2024-04-16T17:03:57.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.649+0000 7f38f59ff740 1 -- start start 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.649+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb04ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.649+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb057f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.649+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb0483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.649+0000 7f38f59ff740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb05d140 con 0x55b2cb0464a0 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.649+0000 7f38f59ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb05d810 con 0x55b2cb046e90 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.649+0000 7f38f59ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb05dee0 con 0x55b2cb04c000 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb04ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb0483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb0483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44874/0 (socket says 172.21.15.18:44874) 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 -- 172.21.15.18:0/3417705756 learned_addr learned my addr 172.21.15.18:0/3417705756 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb04ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43872/0 (socket says 172.21.15.18:43872) 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38eb400640 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb057f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 -- 172.21.15.18:0/3417705756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 msgr2=0x55b2cb057f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb057f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 -- 172.21.15.18:0/3417705756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 msgr2=0x55b2cb04ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb04ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 -- 172.21.15.18:0/3417705756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2cb041360 con 0x55b2cb04c000 2024-04-16T17:03:57.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38ebe00640 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb04ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:57.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.650+0000 7f38f0e00640 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb0483d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b2cb0476f0 tx=0x7f38e4003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe93ba4caf0d1887 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.651+0000 7f38e0c00640 1 -- 172.21.15.18:0/3417705756 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4003ce0 con 0x55b2cb04c000 2024-04-16T17:03:57.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.651+0000 7f38e0c00640 1 -- 172.21.15.18:0/3417705756 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e4015040 con 0x55b2cb04c000 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.651+0000 7f38e0c00640 1 -- 172.21.15.18:0/3417705756 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4029900 con 0x55b2cb04c000 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 -- 172.21.15.18:0/3417705756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 msgr2=0x55b2cb0483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb0483d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b2cb0476f0 tx=0x7f38e4003970 comp rx=0 tx=0).stop 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 -- 172.21.15.18:0/3417705756 shutdown_connections 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb04c000 0x55b2cb0483d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb046e90 0x55b2cb057f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 --2- 172.21.15.18:0/3417705756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb04ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 -- 172.21.15.18:0/3417705756 >> 172.21.15.18:0/3417705756 conn(0x55b2cb041a10 msgr2=0x55b2cb041df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 -- 172.21.15.18:0/3417705756 shutdown_connections 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 -- 172.21.15.18:0/3417705756 wait complete. 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 -- start start 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.652+0000 7f38f59ff740 1 Processor -- start 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb053750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb053c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f59ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb04c000 0x55b2cb056a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f59ff740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb05ec80 con 0x55b2cb0464a0 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f59ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb05e5b0 con 0x55b2cb04c000 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f59ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b2cb05f350 con 0x55b2cb046e90 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f59ff740 10 client.0 _ll_register_callbacks cb 0x7f38dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb053750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43878/0 (socket says 172.21.15.18:43878) 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38ebe00640 1 -- 172.21.15.18:0/1853140411 learned_addr learned my addr 172.21.15.18:0/1853140411 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38eb400640 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb053c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f0e00640 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb04c000 0x55b2cb056a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38ebe00640 1 -- 172.21.15.18:0/1853140411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 msgr2=0x55b2cb053c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38ebe00640 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb053c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38ebe00640 1 -- 172.21.15.18:0/1853140411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb04c000 msgr2=0x55b2cb056a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38ebe00640 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb04c000 0x55b2cb056a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38ebe00640 1 -- 172.21.15.18:0/1853140411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2cb057050 con 0x55b2cb0464a0 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.653+0000 7f38f0e00640 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb04c000 0x55b2cb056a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.654+0000 7f38ebe00640 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb053750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009230 tx=0x7f38dc02c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f819c0ef6530245b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.654+0000 7f38d6000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:03:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.654+0000 7f38d6000640 1 -- 172.21.15.18:0/1853140411 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc067070 con 0x55b2cb0464a0 2024-04-16T17:03:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.654+0000 7f38d6000640 1 -- 172.21.15.18:0/1853140411 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f38dc0057f0 con 0x55b2cb0464a0 2024-04-16T17:03:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.654+0000 7f38f59ff740 1 -- 172.21.15.18:0/1853140411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38e4015040 con 0x55b2cb0464a0 2024-04-16T17:03:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.654+0000 7f38d6000640 1 -- 172.21.15.18:0/1853140411 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f38dc073050 con 0x55b2cb0464a0 2024-04-16T17:03:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.654+0000 7f38f59ff740 10 client.15072 fetch_fsmap learned FSMap version 24 2024-04-16T17:03:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.654+0000 7f38f59ff740 1 -- 172.21.15.18:0/1853140411 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b2cb0c3940 con 0x55b2cb0464a0 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38d6000640 1 -- 172.21.15.18:0/1853140411 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e4015040 con 0x55b2cb0464a0 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38d6000640 1 -- 172.21.15.18:0/1853140411 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f38dc034560 con 0x55b2cb0464a0 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38f59ff740 10 client.15072 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38f59ff740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38f59ff740 20 client.15072 populate_metadata read hostname 'smithi018' 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b2cb0c84d0 0x55b2cb0c4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38f59ff740 4 client.15072 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38f59ff740 1 -- 172.21.15.18:0/1853140411 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "status"}) -- 0x55b2cafb4210 con 0x55b2cb0c84d0 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.655+0000 7f38eb400640 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b2cb0c84d0 0x55b2cb0c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38eb400640 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b2cb0c84d0 0x55b2cb0c4910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3577e10dccd1f231 server_cookie=ea8dba8dd3721821 in_seq=0 out_seq=0 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38d6000640 10 client.15072 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38d6000640 1 -- 172.21.15.18:0/1853140411 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b2cafb4210 con 0x55b2cb0c84d0 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38d6000640 10 client.15072 handle_command_reply: tid=0 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 client.15072 shutdown 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 -- 172.21.15.18:0/1853140411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 msgr2=0x55b2cb053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb053750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009230 tx=0x7f38dc02c6a0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 -- 172.21.15.18:0/1853140411 shutdown_connections 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b2cb0c84d0 0x55b2cb0c4910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2cb04c000 0x55b2cb056a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2cb046e90 0x55b2cb053c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 --2- 172.21.15.18:0/1853140411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b2cb0464a0 0x55b2cb053750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38f59ff740 1 -- 172.21.15.18:0/1853140411 >> 172.21.15.18:0/1853140411 conn(0x55b2cb041a10 msgr2=0x55b2cb0483d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.656+0000 7f38d6000640 0 client.15072 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f59ff740 1 -- 172.21.15.18:0/1853140411 shutdown_connections 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f59ff740 1 -- 172.21.15.18:0/1853140411 wait complete. 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f59ff740 20 client.15072 trim_cache size 0 max 16384 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:03:57.661 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 2, 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24547, 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 119.501406155, 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 24, 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 93, 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 170.34271911499999 2024-04-16T17:03:57.662 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 -- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38d00a3bb0 msgr2=0x7f38d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38d00a3bb0 0x7f38d00a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b2cb047900 tx=0x7f38dc02df90 comp rx=0 tx=0).stop 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 -- 172.21.15.18:0/485919932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38ec105270 msgr2=0x7f38ec19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38ec105270 0x7f38ec19a8b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38d800e9f0 tx=0x7f38d800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 -- 172.21.15.18:0/485919932 shutdown_connections 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f38d00a3bb0 0x7f38d00a6070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105c30 0x7f38ec1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38ec105270 0x7f38ec19a8b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 --2- 172.21.15.18:0/485919932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ec104080 0x7f38ec19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.657+0000 7f38f1800640 1 -- 172.21.15.18:0/485919932 >> 172.21.15.18:0/485919932 conn(0x7f38ec0ff610 msgr2=0x7f38ec108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.658+0000 7f38f1800640 1 -- 172.21.15.18:0/485919932 shutdown_connections 2024-04-16T17:03:57.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.658+0000 7f38f1800640 1 -- 172.21.15.18:0/485919932 wait complete. 2024-04-16T17:03:57.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:57 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1201874904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:57 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2894074516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.675 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 -- 172.21.15.18:0/3877614581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 msgr2=0x7fd234107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 --2- 172.21.15.18:0/3877614581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 0x7fd234107610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd220009860 tx=0x7fd22002bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 -- 172.21.15.18:0/3877614581 shutdown_connections 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 --2- 172.21.15.18:0/3877614581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd234107b50 0x7fd234109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 --2- 172.21.15.18:0/3877614581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 0x7fd234107610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 --2- 172.21.15.18:0/3877614581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2341028f0 0x7fd234104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 -- 172.21.15.18:0/3877614581 >> 172.21.15.18:0/3877614581 conn(0x7fd2340fc2e0 msgr2=0x7fd2340fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 -- 172.21.15.18:0/3877614581 shutdown_connections 2024-04-16T17:03:57.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.882+0000 7fd233e00640 1 -- 172.21.15.18:0/3877614581 wait complete. 2024-04-16T17:03:57.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.883+0000 7fd233e00640 1 Processor -- start 2024-04-16T17:03:57.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.883+0000 7fd233e00640 1 -- start start 2024-04-16T17:03:57.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd233e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2341028f0 0x7fd234195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd233e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 0x7fd2341964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd233e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd234107b50 0x7fd2341a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd233e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd23410dae0 con 0x7fd2341028f0 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd233e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd23410d960 con 0x7fd234105220 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd233e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd23410dc60 con 0x7fd234107b50 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2341028f0 0x7fd234195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2341028f0 0x7fd234195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43890/0 (socket says 172.21.15.18:43890) 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232a00640 1 -- 172.21.15.18:0/673135465 learned_addr learned my addr 172.21.15.18:0/673135465 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 0x7fd2341964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd233400640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd234107b50 0x7fd2341a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232a00640 1 -- 172.21.15.18:0/673135465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd234107b50 msgr2=0x7fd2341a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232a00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd234107b50 0x7fd2341a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232a00640 1 -- 172.21.15.18:0/673135465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 msgr2=0x7fd2341964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232a00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 0x7fd2341964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232a00640 1 -- 172.21.15.18:0/673135465 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2341a63d0 con 0x7fd2341028f0 2024-04-16T17:03:57.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.884+0000 7fd232000640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 0x7fd2341964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:57.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.885+0000 7fd232a00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2341028f0 0x7fd234195f90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd22400ea20 tx=0x7fd22400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.885+0000 7fd238a00640 1 -- 172.21.15.18:0/673135465 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22400cd80 con 0x7fd2341028f0 2024-04-16T17:03:57.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.885+0000 7fd233e00640 1 -- 172.21.15.18:0/673135465 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2341a66c0 con 0x7fd2341028f0 2024-04-16T17:03:57.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.885+0000 7fd238a00640 1 -- 172.21.15.18:0/673135465 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd224010910 con 0x7fd2341028f0 2024-04-16T17:03:57.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.885+0000 7fd238a00640 1 -- 172.21.15.18:0/673135465 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224018920 con 0x7fd2341028f0 2024-04-16T17:03:57.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.885+0000 7fd233e00640 1 -- 172.21.15.18:0/673135465 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2341a6c30 con 0x7fd2341028f0 2024-04-16T17:03:57.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.889+0000 7fd238a00640 1 -- 172.21.15.18:0/673135465 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd224010ac0 con 0x7fd2341028f0 2024-04-16T17:03:57.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd23cbb0740 1 Processor -- start 2024-04-16T17:03:57.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd23cbb0740 1 -- start start 2024-04-16T17:03:57.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 0x5583a68d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d1e90 0x5583a68d6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd238a00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2180a3bb0 0x7fd2180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5583a6837980 con 0x5583a68d1e90 2024-04-16T17:03:57.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5583a67de220 con 0x5583a68d7000 2024-04-16T17:03:57.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.890+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5583a6833940 con 0x5583a68d14a0 2024-04-16T17:03:57.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd233400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 0x5583a68d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd238a00640 1 -- 172.21.15.18:0/673135465 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd2240d0ba0 con 0x7fd2341028f0 2024-04-16T17:03:57.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d1e90 0x5583a68d6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68d9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59176/0 (socket says 172.21.15.18:59176) 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232000640 1 -- 172.21.15.18:0/3938567766 learned_addr learned my addr 172.21.15.18:0/3938567766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd233400640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2180a3bb0 0x7fd2180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232a00640 1 -- 172.21.15.18:0/3938567766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 msgr2=0x5583a68d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232a00640 1 --2- 172.21.15.18:0/3938567766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 0x5583a68d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232a00640 1 -- 172.21.15.18:0/3938567766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 msgr2=0x5583a68d9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232a00640 1 --2- 172.21.15.18:0/3938567766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68d9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232a00640 1 -- 172.21.15.18:0/3938567766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a654a2d0 con 0x5583a68d1e90 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.891+0000 7fd232a00640 1 --2- 172.21.15.18:0/3938567766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d1e90 0x5583a68d6990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd22409c800 tx=0x7fd22409cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc88e1d28b23d385 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.892+0000 7fd233400640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2180a3bb0 0x7fd2180a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5583a68d26f0 tx=0x7fd22c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.892+0000 7fd21fe00640 1 -- 172.21.15.18:0/3938567766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240bd0b0 con 0x5583a68d1e90 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.892+0000 7fd21fe00640 1 -- 172.21.15.18:0/3938567766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2240ce380 con 0x5583a68d1e90 2024-04-16T17:03:57.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.892+0000 7fd21fe00640 1 -- 172.21.15.18:0/3938567766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240973b0 con 0x5583a68d1e90 2024-04-16T17:03:57.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3938567766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d1e90 msgr2=0x5583a68d6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3938567766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d1e90 0x5583a68d6990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd22409c800 tx=0x7fd22409cc40 comp rx=0 tx=0).stop 2024-04-16T17:03:57.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3938567766 shutdown_connections 2024-04-16T17:03:57.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3938567766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68d9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3938567766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d1e90 0x5583a68d6990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3938567766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 0x5583a68d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3938567766 >> 172.21.15.18:0/3938567766 conn(0x5583a68cca10 msgr2=0x5583a68cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3938567766 shutdown_connections 2024-04-16T17:03:57.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3938567766 wait complete. 2024-04-16T17:03:57.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.893+0000 7fd23cbb0740 1 -- start start 2024-04-16T17:03:57.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.894+0000 7fd23cbb0740 1 Processor -- start 2024-04-16T17:03:57.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.894+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 0x5583a68db920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.894+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68dbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.894+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d7000 0x5583a68e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.894+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5583a68d9f50 con 0x5583a68d7000 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5583a67f1590 con 0x5583a68d1e90 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5583a68da0d0 con 0x5583a68d14a0 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd23cbb0740 10 client.0 _ll_register_callbacks cb 0x5583a67f9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68dbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68dbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59178/0 (socket says 172.21.15.18:59178) 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232000640 1 -- 172.21.15.18:0/3281705389 learned_addr learned my addr 172.21.15.18:0/3281705389 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd233400640 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d7000 0x5583a68e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232a00640 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 0x5583a68db920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232000640 1 -- 172.21.15.18:0/3281705389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 msgr2=0x5583a68db920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232000640 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 0x5583a68db920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232000640 1 -- 172.21.15.18:0/3281705389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d7000 msgr2=0x5583a68e2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232000640 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d7000 0x5583a68e2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.895+0000 7fd232000640 1 -- 172.21.15.18:0/3281705389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a66146b0 con 0x5583a68d1e90 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.896+0000 7fd232000640 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68dbe60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2200349e0 tx=0x7fd220005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5766fdc9f7a51aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.896+0000 7fd21cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.896+0000 7fd21cc00640 1 -- 172.21.15.18:0/3281705389 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220045070 con 0x5583a68d1e90 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.896+0000 7fd21cc00640 1 -- 172.21.15.18:0/3281705389 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd220004240 con 0x5583a68d1e90 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.896+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3281705389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2240ce380 con 0x5583a68d1e90 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.896+0000 7fd21cc00640 1 -- 172.21.15.18:0/3281705389 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd220051050 con 0x5583a68d1e90 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.896+0000 7fd23cbb0740 10 client.25214 fetch_fsmap learned FSMap version 24 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.896+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3281705389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5583a68e62d0 con 0x5583a68d1e90 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd21cc00640 1 -- 172.21.15.18:0/3281705389 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2240ce380 con 0x5583a68d1e90 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd21cc00640 1 -- 172.21.15.18:0/3281705389 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd220032970 con 0x5583a68d1e90 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd23cbb0740 10 client.25214 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd23cbb0740 10 client.25214 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd23cbb0740 20 client.25214 populate_metadata read hostname 'smithi018' 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5583a68e8900 0x5583a68e9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd23cbb0740 4 client.25214 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:03:57.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3281705389 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583a68453a0 con 0x5583a68e8900 2024-04-16T17:03:57.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.897+0000 7fd232a00640 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5583a68e8900 0x5583a68e9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.898+0000 7fd232a00640 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5583a68e8900 0x5583a68e9d70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=216884a82418f621 server_cookie=164b1440dd11764f in_seq=0 out_seq=0 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.898+0000 7fd21cc00640 10 client.25214 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd21cc00640 1 -- 172.21.15.18:0/3281705389 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583a68453a0 con 0x5583a68e8900 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd21cc00640 10 client.25214 handle_command_reply: tid=0 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 client.25214 shutdown 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3281705389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 msgr2=0x5583a68dbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68dbe60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2200349e0 tx=0x7fd220005a90 comp rx=0 tx=0).stop 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3281705389 shutdown_connections 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5583a68e8900 0x5583a68e9d70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d7000 0x5583a68e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68dbe60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/3281705389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d14a0 0x5583a68db920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd21cc00640 0 client.25214 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:03:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.899+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3281705389 >> 172.21.15.18:0/3281705389 conn(0x5583a68cca10 msgr2=0x5583a68cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.900+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3281705389 shutdown_connections 2024-04-16T17:03:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.900+0000 7fd23cbb0740 1 -- 172.21.15.18:0/3281705389 wait complete. 2024-04-16T17:03:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.900+0000 7fd23cbb0740 20 client.25214 trim_cache size 0 max 16384 2024-04-16T17:03:57.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd23cbb0740 1 Processor -- start 2024-04-16T17:03:57.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd23cbb0740 1 -- start start 2024-04-16T17:03:57.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d7000 0x5583a68e2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5583a68e80b0 con 0x5583a68d14a0 2024-04-16T17:03:57.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5583a68e8780 con 0x5583a68d1e90 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5583a68e8e50 con 0x5583a68d7000 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd232a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd232a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68d3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43920/0 (socket says 172.21.15.18:43920) 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.920+0000 7fd232a00640 1 -- 172.21.15.18:0/2376511899 learned_addr learned my addr 172.21.15.18:0/2376511899 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd232000640 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd233400640 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d7000 0x5583a68e2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd232a00640 1 -- 172.21.15.18:0/2376511899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d7000 msgr2=0x5583a68e2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd232a00640 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d7000 0x5583a68e2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd232a00640 1 -- 172.21.15.18:0/2376511899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 msgr2=0x5583a68d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd232a00640 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd232a00640 1 -- 172.21.15.18:0/2376511899 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a68d2270 con 0x5583a68d14a0 2024-04-16T17:03:57.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd232a00640 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68d3330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd22409adb0 tx=0x7fd22400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e6350c09b13828 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd21f400640 1 -- 172.21.15.18:0/2376511899 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224091ec0 con 0x5583a68d14a0 2024-04-16T17:03:57.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd21f400640 1 -- 172.21.15.18:0/2376511899 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd224099170 con 0x5583a68d14a0 2024-04-16T17:03:57.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.921+0000 7fd21f400640 1 -- 172.21.15.18:0/2376511899 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22409bf50 con 0x5583a68d14a0 2024-04-16T17:03:57.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 -- 172.21.15.18:0/2376511899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 msgr2=0x5583a68d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68d3330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd22409adb0 tx=0x7fd22400cf20 comp rx=0 tx=0).stop 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 -- 172.21.15.18:0/2376511899 shutdown_connections 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d7000 0x5583a68e2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d1e90 0x5583a68d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/2376511899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68d3330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 -- 172.21.15.18:0/2376511899 >> 172.21.15.18:0/2376511899 conn(0x5583a68cca10 msgr2=0x5583a68ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 -- 172.21.15.18:0/2376511899 shutdown_connections 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.922+0000 7fd23cbb0740 1 -- 172.21.15.18:0/2376511899 wait complete. 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 1 -- start start 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 1 Processor -- start 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d1e90 0x5583a68de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5583a68e9bf0 con 0x5583a68d14a0 2024-04-16T17:03:57.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5583a68e9520 con 0x5583a68d7000 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5583a68ea2c0 con 0x5583a68d1e90 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.923+0000 7fd23cbb0740 10 client.0 _ll_register_callbacks cb 0x7fd224005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68de1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43924/0 (socket says 172.21.15.18:43924) 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 -- 172.21.15.18:0/1214940498 learned_addr learned my addr 172.21.15.18:0/1214940498 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd233400640 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232000640 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d1e90 0x5583a68de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 -- 172.21.15.18:0/1214940498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d1e90 msgr2=0x5583a68de6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d1e90 0x5583a68de6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 -- 172.21.15.18:0/1214940498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 msgr2=0x5583a68e1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68e1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 -- 172.21.15.18:0/1214940498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a68e1cf0 con 0x5583a68d14a0 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd233400640 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd232a00640 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68de1a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2240bd010 tx=0x7fd22400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc01f97c5b40a81f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.924+0000 7fd21cc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:03:57.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd21cc00640 1 -- 172.21.15.18:0/1214940498 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224098da0 con 0x5583a68d14a0 2024-04-16T17:03:57.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd21cc00640 1 -- 172.21.15.18:0/1214940498 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd22409ae00 con 0x5583a68d14a0 2024-04-16T17:03:57.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd23cbb0740 1 -- 172.21.15.18:0/1214940498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd224099170 con 0x5583a68d14a0 2024-04-16T17:03:57.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd21cc00640 1 -- 172.21.15.18:0/1214940498 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd224091ec0 con 0x5583a68d14a0 2024-04-16T17:03:57.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd23cbb0740 10 client.15090 fetch_fsmap learned FSMap version 24 2024-04-16T17:03:57.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd23cbb0740 1 -- 172.21.15.18:0/1214940498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5583a68e2ff0 con 0x5583a68d14a0 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd21cc00640 1 -- 172.21.15.18:0/1214940498 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd224099170 con 0x5583a68d14a0 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd21cc00640 1 -- 172.21.15.18:0/1214940498 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd2240b9070 con 0x5583a68d14a0 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.925+0000 7fd23cbb0740 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.926+0000 7fd23cbb0740 10 client.15090 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.926+0000 7fd23cbb0740 20 client.15090 populate_metadata read hostname 'smithi018' 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.926+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5583a6952990 0x5583a69516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.926+0000 7fd23cbb0740 4 client.15090 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.926+0000 7fd23cbb0740 1 -- 172.21.15.18:0/1214940498 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "status"}) -- 0x5583a683f210 con 0x5583a6952990 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.926+0000 7fd232000640 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5583a6952990 0x5583a69516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.926+0000 7fd232000640 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5583a6952990 0x5583a69516a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88e4ac2996355875 server_cookie=32eafa26f5eefbcd in_seq=0 out_seq=0 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.926+0000 7fd21cc00640 10 client.15090 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd21cc00640 1 -- 172.21.15.18:0/1214940498 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5583a683f210 con 0x5583a6952990 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd21cc00640 10 client.15090 handle_command_reply: tid=0 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 client.15090 shutdown 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 -- 172.21.15.18:0/1214940498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 msgr2=0x5583a68de1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68de1a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2240bd010 tx=0x7fd22400ef50 comp rx=0 tx=0).stop 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 -- 172.21.15.18:0/1214940498 shutdown_connections 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5583a6952990 0x5583a69516a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583a68d7000 0x5583a68e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583a68d1e90 0x5583a68de6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 --2- 172.21.15.18:0/1214940498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583a68d14a0 0x5583a68de1a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd21cc00640 0 client.15090 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 -- 172.21.15.18:0/1214940498 >> 172.21.15.18:0/1214940498 conn(0x5583a68cca10 msgr2=0x5583a68e76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 -- 172.21.15.18:0/1214940498 shutdown_connections 2024-04-16T17:03:57.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.927+0000 7fd23cbb0740 1 -- 172.21.15.18:0/1214940498 wait complete. 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd23cbb0740 20 client.15090 trim_cache size 0 max 16384 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 1, 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24566, 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 120.783644693, 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 24, 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 93, 2024-04-16T17:03:57.934 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T17:03:57.934 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 166.49489468499999 2024-04-16T17:03:57.934 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:03:57.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 -- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2180a3bb0 msgr2=0x7fd2180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2180a3bb0 0x7fd2180a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5583a68d26f0 tx=0x7fd22c0095c0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 -- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2341028f0 msgr2=0x7fd234195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2341028f0 0x7fd234195f90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd22400ea20 tx=0x7fd22400eee0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 -- 172.21.15.18:0/673135465 shutdown_connections 2024-04-16T17:03:57.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd234107b50 0x7fd2341a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd234105220 0x7fd2341964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd2180a3bb0 0x7fd2180a6070 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 --2- 172.21.15.18:0/673135465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2341028f0 0x7fd234195f90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.928+0000 7fd233e00640 1 -- 172.21.15.18:0/673135465 >> 172.21.15.18:0/673135465 conn(0x7fd2340fc2e0 msgr2=0x7fd234102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.929+0000 7fd233e00640 1 -- 172.21.15.18:0/673135465 shutdown_connections 2024-04-16T17:03:57.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:57.929+0000 7fd233e00640 1 -- 172.21.15.18:0/673135465 wait complete. 2024-04-16T17:03:57.947 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T17:03:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 -- 172.21.15.18:0/2003147276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 msgr2=0x7f3aac108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 --2- 172.21.15.18:0/2003147276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 0x7f3aac108a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009860 tx=0x7f3aa002bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 -- 172.21.15.18:0/2003147276 shutdown_connections 2024-04-16T17:03:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 --2- 172.21.15.18:0/2003147276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3aac10a210 0x7f3aac110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 --2- 172.21.15.18:0/2003147276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aac109850 0x7f3aac109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 --2- 172.21.15.18:0/2003147276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 0x7f3aac108a80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 -- 172.21.15.18:0/2003147276 >> 172.21.15.18:0/2003147276 conn(0x7f3aac077610 msgr2=0x7f3aac079a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:58.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 -- 172.21.15.18:0/2003147276 shutdown_connections 2024-04-16T17:03:58.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.150+0000 7f3ab4400640 1 -- 172.21.15.18:0/2003147276 wait complete. 2024-04-16T17:03:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.151+0000 7f3ab4400640 1 Processor -- start 2024-04-16T17:03:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.151+0000 7f3ab4400640 1 -- start start 2024-04-16T17:03:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab4400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 0x7f3aac19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab4400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aac109850 0x7f3aac19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab4400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3aac10a210 0x7f3aac1a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab4400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1142d0 con 0x7f3aac109850 2024-04-16T17:03:58.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab4400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac114150 con 0x7f3aac10a210 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab4400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac114450 con 0x7f3aac108660 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aac109850 0x7f3aac19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aac109850 0x7f3aac19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43942/0 (socket says 172.21.15.18:43942) 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1000640 1 -- 172.21.15.18:0/660584531 learned_addr learned my addr 172.21.15.18:0/660584531 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 0x7f3aac19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab2400640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3aac10a210 0x7f3aac1a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1000640 1 -- 172.21.15.18:0/660584531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 msgr2=0x7f3aac19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1000640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 0x7f3aac19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1000640 1 -- 172.21.15.18:0/660584531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3aac10a210 msgr2=0x7f3aac1a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1000640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3aac10a210 0x7f3aac1a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1000640 1 -- 172.21.15.18:0/660584531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac1a65d0 con 0x7f3aac109850 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.152+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 0x7f3aac19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.153+0000 7f3ab1000640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aac109850 0x7f3aac19eef0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800e9c0 tx=0x7f3a9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.153+0000 7f3aaa000640 1 -- 172.21.15.18:0/660584531 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800ccc0 con 0x7f3aac109850 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.153+0000 7f3aaa000640 1 -- 172.21.15.18:0/660584531 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a98005500 con 0x7f3aac109850 2024-04-16T17:03:58.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.153+0000 7f3ab4400640 1 -- 172.21.15.18:0/660584531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aac1a68c0 con 0x7f3aac109850 2024-04-16T17:03:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.153+0000 7f3aaa000640 1 -- 172.21.15.18:0/660584531 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98010430 con 0x7f3aac109850 2024-04-16T17:03:58.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.153+0000 7f3ab4400640 1 -- 172.21.15.18:0/660584531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aac1a6e60 con 0x7f3aac109850 2024-04-16T17:03:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab7177740 1 Processor -- start 2024-04-16T17:03:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3aaa000640 1 -- 172.21.15.18:0/660584531 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3a980046c0 con 0x7f3aac109850 2024-04-16T17:03:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab7177740 1 -- start start 2024-04-16T17:03:58.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab7177740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fc679c4980 con 0x55fc67a63e30 2024-04-16T17:03:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab7177740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fc6796b220 con 0x55fc67a5e2d0 2024-04-16T17:03:58.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab7177740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc679c0940 con 0x55fc67a5ecc0 2024-04-16T17:03:58.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab1000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab2400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab2400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59232/0 (socket says 172.21.15.18:59232) 2024-04-16T17:03:58.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab2400640 1 -- 172.21.15.18:0/3037905704 learned_addr learned my addr 172.21.15.18:0/3037905704 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab1000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43952/0 (socket says 172.21.15.18:43952) 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.157+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/3037905704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3ab2400640 1 -- 172.21.15.18:0/3037905704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 msgr2=0x55fc67a637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3ab2400640 1 --2- 172.21.15.18:0/3037905704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3ab2400640 1 -- 172.21.15.18:0/3037905704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 msgr2=0x55fc67a662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3ab2400640 1 --2- 172.21.15.18:0/3037905704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3ab2400640 1 -- 172.21.15.18:0/3037905704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc676d72d0 con 0x55fc67a5e2d0 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3aaa000640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a940a3c30 0x7f3a940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3ab2400640 1 --2- 172.21.15.18:0/3037905704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5e6f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4010370 tx=0x7f3aa4010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e44278c6ff087acc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3aaa000640 1 -- 172.21.15.18:0/660584531 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f3a980d0a10 con 0x7f3aac109850 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.158+0000 7f3a9ea00640 1 -- 172.21.15.18:0/3037905704 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4016070 con 0x55fc67a5e2d0 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3a9ea00640 1 -- 172.21.15.18:0/3037905704 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa4004d20 con 0x55fc67a5e2d0 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3a9ea00640 1 -- 172.21.15.18:0/3037905704 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4005480 con 0x55fc67a5e2d0 2024-04-16T17:03:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a940a3c30 0x7f3a940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a940a3c30 0x7f3a940a60f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00349e0 tx=0x7f3aa002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab7177740 1 -- 172.21.15.18:0/3037905704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 msgr2=0x55fc67a5e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab7177740 1 --2- 172.21.15.18:0/3037905704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5e6f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4010370 tx=0x7f3aa4010830 comp rx=0 tx=0).stop 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab7177740 1 -- 172.21.15.18:0/3037905704 shutdown_connections 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab7177740 1 --2- 172.21.15.18:0/3037905704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab7177740 1 --2- 172.21.15.18:0/3037905704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab7177740 1 --2- 172.21.15.18:0/3037905704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5e6f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.159+0000 7f3ab7177740 1 -- 172.21.15.18:0/3037905704 >> 172.21.15.18:0/3037905704 conn(0x55fc67a59840 msgr2=0x55fc67a59c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.160+0000 7f3ab7177740 1 -- 172.21.15.18:0/3037905704 shutdown_connections 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.160+0000 7f3ab7177740 1 -- 172.21.15.18:0/3037905704 wait complete. 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.160+0000 7f3ab7177740 1 -- start start 2024-04-16T17:03:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.160+0000 7f3ab7177740 1 Processor -- start 2024-04-16T17:03:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5e2d0 0x55fc67a68900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5ecc0 0x55fc67a68e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a6fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab7177740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fc67a66de0 con 0x55fc67a63e30 2024-04-16T17:03:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab7177740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fc6797e590 con 0x55fc67a5ecc0 2024-04-16T17:03:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab7177740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc67a66f60 con 0x55fc67a5e2d0 2024-04-16T17:03:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab7177740 10 client.0 _ll_register_callbacks cb 0x55fc67986050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:03:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5e2d0 0x55fc67a68900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab2400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a6fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5e2d0 0x55fc67a68900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44978/0 (socket says 172.21.15.18:44978) 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1a00640 1 -- 172.21.15.18:0/1241914410 learned_addr learned my addr 172.21.15.18:0/1241914410 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1000640 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5ecc0 0x55fc67a68e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1a00640 1 -- 172.21.15.18:0/1241914410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5ecc0 msgr2=0x55fc67a68e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5ecc0 0x55fc67a68e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1a00640 1 -- 172.21.15.18:0/1241914410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 msgr2=0x55fc67a6fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a6fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.161+0000 7f3ab1a00640 1 -- 172.21.15.18:0/1241914410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc677a16b0 con 0x55fc67a5e2d0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.162+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5e2d0 0x55fc67a68900 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009830 tx=0x7f3aa002bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9aa0413ab3e356b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.162+0000 7f3a93400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.162+0000 7f3a93400640 1 -- 172.21.15.18:0/1241914410 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0039040 con 0x55fc67a5e2d0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.162+0000 7f3a93400640 1 -- 172.21.15.18:0/1241914410 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f3aa002c920 con 0x55fc67a5e2d0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.162+0000 7f3ab7177740 1 -- 172.21.15.18:0/1241914410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa4004d20 con 0x55fc67a5e2d0 2024-04-16T17:03:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.162+0000 7f3a93400640 1 -- 172.21.15.18:0/1241914410 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3aa005b050 con 0x55fc67a5e2d0 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.162+0000 7f3ab7177740 10 client.25219 fetch_fsmap learned FSMap version 24 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.162+0000 7f3ab7177740 1 -- 172.21.15.18:0/1241914410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc67a73180 con 0x55fc67a5e2d0 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3a93400640 1 -- 172.21.15.18:0/1241914410 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa4004d20 con 0x55fc67a5e2d0 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3a93400640 1 -- 172.21.15.18:0/1241914410 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3aa00035f0 con 0x55fc67a5e2d0 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3ab7177740 10 client.25219 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3ab7177740 10 client.25219 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3ab7177740 20 client.25219 populate_metadata read hostname 'smithi018' 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fc67a758c0 0x55fc67a76d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3ab7177740 4 client.25219 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3ab7177740 1 -- 172.21.15.18:0/1241914410 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc679d23a0 con 0x55fc67a758c0 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.163+0000 7f3ab1000640 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fc67a758c0 0x55fc67a76d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.164+0000 7f3ab1000640 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fc67a758c0 0x55fc67a76d30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7931663d4fd74fe server_cookie=688e0f6eeafb4c1 in_seq=0 out_seq=0 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.164+0000 7f3a93400640 10 client.25219 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3a93400640 1 -- 172.21.15.18:0/1241914410 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc679d23a0 con 0x55fc67a758c0 2024-04-16T17:03:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3a93400640 10 client.25219 handle_command_reply: tid=0 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 client.25219 shutdown 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 -- 172.21.15.18:0/1241914410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5e2d0 msgr2=0x55fc67a68900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5e2d0 0x55fc67a68900 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009830 tx=0x7f3aa002bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 -- 172.21.15.18:0/1241914410 shutdown_connections 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fc67a758c0 0x55fc67a76d30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a6fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5ecc0 0x55fc67a68e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3a93400640 0 client.25219 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1241914410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5e2d0 0x55fc67a68900 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 -- 172.21.15.18:0/1241914410 >> 172.21.15.18:0/1241914410 conn(0x55fc67a59840 msgr2=0x55fc67a58a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 -- 172.21.15.18:0/1241914410 shutdown_connections 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 1 -- 172.21.15.18:0/1241914410 wait complete. 2024-04-16T17:03:58.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.165+0000 7f3ab7177740 20 client.25219 trim_cache size 0 max 16384 2024-04-16T17:03:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.185+0000 7f3ab7177740 1 Processor -- start 2024-04-16T17:03:58.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.185+0000 7f3ab7177740 1 -- start start 2024-04-16T17:03:58.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a74640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab7177740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fc67a75070 con 0x55fc67a63e30 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab7177740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fc67a75740 con 0x55fc67a5e2d0 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab7177740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc67a75e10 con 0x55fc67a5ecc0 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab1a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab1a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59252/0 (socket says 172.21.15.18:59252) 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab1a00640 1 -- 172.21.15.18:0/2471966868 learned_addr learned my addr 172.21.15.18:0/2471966868 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab1000640 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.186+0000 7f3ab2400640 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a74640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.187+0000 7f3ab1a00640 1 -- 172.21.15.18:0/2471966868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 msgr2=0x55fc67a6fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.187+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.187+0000 7f3ab1a00640 1 -- 172.21.15.18:0/2471966868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 msgr2=0x55fc67a74640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.187+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a74640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.187+0000 7f3ab1a00640 1 -- 172.21.15.18:0/2471966868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc67a5f0a0 con 0x55fc67a5e2d0 2024-04-16T17:03:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.187+0000 7f3ab1000640 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.187+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5f720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009780 tx=0x7f3aa0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b586c4efefbe84c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3a9e000640 1 -- 172.21.15.18:0/2471966868 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0064070 con 0x55fc67a5e2d0 2024-04-16T17:03:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3a9e000640 1 -- 172.21.15.18:0/2471966868 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa0052070 con 0x55fc67a5e2d0 2024-04-16T17:03:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3a9e000640 1 -- 172.21.15.18:0/2471966868 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0039040 con 0x55fc67a5e2d0 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3ab7177740 1 -- 172.21.15.18:0/2471966868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 msgr2=0x55fc67a5f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3ab7177740 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5f720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009780 tx=0x7f3aa0000bc0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3ab7177740 1 -- 172.21.15.18:0/2471966868 shutdown_connections 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3ab7177740 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a63e30 0x55fc67a74640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3ab7177740 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3ab7177740 1 --2- 172.21.15.18:0/2471966868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a5e2d0 0x55fc67a5f720 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.190+0000 7f3ab7177740 1 -- 172.21.15.18:0/2471966868 >> 172.21.15.18:0/2471966868 conn(0x55fc67a59840 msgr2=0x55fc67a59c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.191+0000 7f3ab7177740 1 -- 172.21.15.18:0/2471966868 shutdown_connections 2024-04-16T17:03:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.191+0000 7f3ab7177740 1 -- 172.21.15.18:0/2471966868 wait complete. 2024-04-16T17:03:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.191+0000 7f3ab7177740 1 -- start start 2024-04-16T17:03:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.191+0000 7f3ab7177740 1 Processor -- start 2024-04-16T17:03:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a5e2d0 0x55fc67a6b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab7177740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a63e30 0x55fc67a6e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab7177740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fc67a76bb0 con 0x55fc67a5e2d0 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab7177740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fc67a764e0 con 0x55fc67a63e30 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab7177740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc67a77280 con 0x55fc67a5ecc0 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab7177740 10 client.0 _ll_register_callbacks cb 0x7f3aa00096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a5e2d0 0x55fc67a6b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a5e2d0 0x55fc67a6b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43966/0 (socket says 172.21.15.18:43966) 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1a00640 1 -- 172.21.15.18:0/1375400286 learned_addr learned my addr 172.21.15.18:0/1375400286 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1000640 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab2400640 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a63e30 0x55fc67a6e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1a00640 1 -- 172.21.15.18:0/1375400286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 msgr2=0x55fc67a6b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1a00640 1 -- 172.21.15.18:0/1375400286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a63e30 msgr2=0x55fc67a6e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a63e30 0x55fc67a6e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1a00640 1 -- 172.21.15.18:0/1375400286 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc67a6eb20 con 0x55fc67a5e2d0 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.192+0000 7f3ab1000640 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:58.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.193+0000 7f3ab1a00640 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a5e2d0 0x55fc67a6b020 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0003ae0 tx=0x7f3aa00056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca78da455ea408c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.193+0000 7f3a93400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.193+0000 7f3a93400640 1 -- 172.21.15.18:0/1375400286 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0064070 con 0x55fc67a5e2d0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.193+0000 7f3ab7177740 1 -- 172.21.15.18:0/1375400286 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa0052070 con 0x55fc67a5e2d0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.193+0000 7f3a93400640 1 -- 172.21.15.18:0/1375400286 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f3aa002c920 con 0x55fc67a5e2d0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.193+0000 7f3a93400640 1 -- 172.21.15.18:0/1375400286 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3aa0043040 con 0x55fc67a5e2d0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.193+0000 7f3ab7177740 10 client.15108 fetch_fsmap learned FSMap version 24 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.193+0000 7f3ab7177740 1 -- 172.21.15.18:0/1375400286 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc67a6fe20 con 0x55fc67a5e2d0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.194+0000 7f3a93400640 1 -- 172.21.15.18:0/1375400286 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa0052070 con 0x55fc67a5e2d0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.194+0000 7f3a93400640 1 -- 172.21.15.18:0/1375400286 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3aa004d460 con 0x55fc67a5e2d0 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.194+0000 7f3ab7177740 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.194+0000 7f3ab7177740 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:03:58.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.194+0000 7f3ab7177740 20 client.15108 populate_metadata read hostname 'smithi018' 2024-04-16T17:03:58.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:58 smithi018 ceph-mon[31776]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:58.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:03:58 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/783504979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.194+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fc67adfd50 0x55fc67adea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.194+0000 7f3ab7177740 4 client.15108 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:03:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.194+0000 7f3ab7177740 1 -- 172.21.15.18:0/1375400286 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "status"}) -- 0x55fc679cc210 con 0x55fc67adfd50 2024-04-16T17:03:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.195+0000 7f3ab1000640 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fc67adfd50 0x55fc67adea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.195+0000 7f3ab1000640 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fc67adfd50 0x55fc67adea60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5226f5a32ef6c4fb server_cookie=e7098502a7d47958 in_seq=0 out_seq=0 2024-04-16T17:03:58.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.195+0000 7f3a93400640 10 client.15108 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.195+0000 7f3a93400640 1 -- 172.21.15.18:0/1375400286 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fc679cc210 con 0x55fc67adfd50 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.195+0000 7f3a93400640 10 client.15108 handle_command_reply: tid=0 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.195+0000 7f3ab7177740 1 client.15108 shutdown 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 -- 172.21.15.18:0/1375400286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a5e2d0 msgr2=0x55fc67a6b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a5e2d0 0x55fc67a6b020 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0003ae0 tx=0x7f3aa00056f0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 -- 172.21.15.18:0/1375400286 shutdown_connections 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fc67adfd50 0x55fc67adea60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc67a63e30 0x55fc67a6e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67a5ecc0 0x55fc67a6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 --2- 172.21.15.18:0/1375400286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc67a5e2d0 0x55fc67a6b020 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3a93400640 0 client.15108 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 -- 172.21.15.18:0/1375400286 >> 172.21.15.18:0/1375400286 conn(0x55fc67a59840 msgr2=0x55fc67a54910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 -- 172.21.15.18:0/1375400286 shutdown_connections 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 1 -- 172.21.15.18:0/1375400286 wait complete. 2024-04-16T17:03:58.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.196+0000 7f3ab7177740 20 client.15108 trim_cache size 0 max 16384 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 0, 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24596, 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 153.263782096, 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 24, 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 93, 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 159.41491516299999 2024-04-16T17:03:58.201 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 -- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a940a3c30 msgr2=0x7f3a940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a940a3c30 0x7f3a940a60f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00349e0 tx=0x7f3aa002e040 comp rx=0 tx=0).stop 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 -- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aac109850 msgr2=0x7f3aac19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aac109850 0x7f3aac19eef0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800e9c0 tx=0x7f3a9800ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 -- 172.21.15.18:0/660584531 shutdown_connections 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3aac10a210 0x7f3aac1a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3a940a3c30 0x7f3a940a60f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aac109850 0x7f3aac19eef0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 --2- 172.21.15.18:0/660584531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aac108660 0x7f3aac19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:58.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 -- 172.21.15.18:0/660584531 >> 172.21.15.18:0/660584531 conn(0x7f3aac077610 msgr2=0x7f3aac10cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:58.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 -- 172.21.15.18:0/660584531 shutdown_connections 2024-04-16T17:03:58.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:03:58.197+0000 7f3ab4400640 1 -- 172.21.15.18:0/660584531 wait complete. 2024-04-16T17:03:58.215 INFO:tasks.cephadm:Setup complete, yielding 2024-04-16T17:03:58.216 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-16T17:03:58.226 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:03:58.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch status' 2024-04-16T17:03:58.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:58 smithi043 ceph-mon[36901]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:58.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:03:58 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/783504979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:58.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:58 smithi022 ceph-mon[36023]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:03:58.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:03:58 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/783504979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:58.679 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:00.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:00 smithi018 ceph-mon[31776]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:04:00.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:00 smithi043 ceph-mon[36901]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:04:00.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:00 smithi022 ceph-mon[36023]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:04:01.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.003+0000 7fbeee200700 1 -- 172.21.15.18:0/3575309125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 msgr2=0x7fbee81095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.003+0000 7fbeee200700 1 --2- 172.21.15.18:0/3575309125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 0x7fbee81095b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbed80099c0 tx=0x7fbed8009cd0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.004+0000 7fbeee200700 1 -- 172.21.15.18:0/3575309125 shutdown_connections 2024-04-16T17:04:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.004+0000 7fbeee200700 1 --2- 172.21.15.18:0/3575309125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee810ad30 0x7fbee81115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.004+0000 7fbeee200700 1 --2- 172.21.15.18:0/3575309125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 0x7fbee81095b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.004+0000 7fbeee200700 1 --2- 172.21.15.18:0/3575309125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbee810a390 0x7fbee810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.004+0000 7fbeee200700 1 -- 172.21.15.18:0/3575309125 >> 172.21.15.18:0/3575309125 conn(0x7fbee8078160 msgr2=0x7fbee807a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.004+0000 7fbeee200700 1 -- 172.21.15.18:0/3575309125 shutdown_connections 2024-04-16T17:04:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.004+0000 7fbeee200700 1 -- 172.21.15.18:0/3575309125 wait complete. 2024-04-16T17:04:01.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.005+0000 7fbeee200700 1 Processor -- start 2024-04-16T17:04:01.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.005+0000 7fbeee200700 1 -- start start 2024-04-16T17:04:01.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.005+0000 7fbeee200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 0x7fbee819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:01.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.005+0000 7fbeee200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbee810a390 0x7fbee819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.005+0000 7fbeee200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee810ad30 0x7fbee81a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.005+0000 7fbeee200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8114dd0 con 0x7fbee8109190 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.005+0000 7fbeee200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8114c50 con 0x7fbee810a390 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.005+0000 7fbeee200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8114f50 con 0x7fbee810ad30 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 0x7fbee819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbee810a390 0x7fbee819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 0x7fbee819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43986/0 (socket says 172.21.15.18:43986) 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbee810a390 0x7fbee819f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59286/0 (socket says 172.21.15.18:59286) 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee7400700 1 -- 172.21.15.18:0/18012012 learned_addr learned my addr 172.21.15.18:0/18012012 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee7e00700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee810ad30 0x7fbee81a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee6a00700 1 -- 172.21.15.18:0/18012012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee810ad30 msgr2=0x7fbee81a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee6a00700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee810ad30 0x7fbee81a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee6a00700 1 -- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 msgr2=0x7fbee819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:01.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee6a00700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 0x7fbee819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee6a00700 1 -- 172.21.15.18:0/18012012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee81a7050 con 0x7fbee810a390 2024-04-16T17:04:01.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.006+0000 7fbee6a00700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbee810a390 0x7fbee819f930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbed400e9f0 tx=0x7fbed400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:01.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.007+0000 7fbedfe00700 1 -- 172.21.15.18:0/18012012 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed400c9c0 con 0x7fbee810a390 2024-04-16T17:04:01.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.007+0000 7fbedfe00700 1 -- 172.21.15.18:0/18012012 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed400cb60 con 0x7fbee810a390 2024-04-16T17:04:01.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.007+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee81a7340 con 0x7fbee810a390 2024-04-16T17:04:01.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.007+0000 7fbedfe00700 1 -- 172.21.15.18:0/18012012 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4018680 con 0x7fbee810a390 2024-04-16T17:04:01.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.007+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee81a7880 con 0x7fbee810a390 2024-04-16T17:04:01.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.008+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee80697e0 con 0x7fbee810a390 2024-04-16T17:04:01.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.010+0000 7fbedfe00700 1 -- 172.21.15.18:0/18012012 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbed4010430 con 0x7fbee810a390 2024-04-16T17:04:01.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.012+0000 7fbedfe00700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbecc0a3c40 0x7fbecc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:01.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.012+0000 7fbedfe00700 1 -- 172.21.15.18:0/18012012 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fbed40d0d90 con 0x7fbee810a390 2024-04-16T17:04:01.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.012+0000 7fbee7400700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbecc0a3c40 0x7fbecc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:01.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.013+0000 7fbee7400700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbecc0a3c40 0x7fbecc0a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbed80038c0 tx=0x7fbed8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:01.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.014+0000 7fbedfe00700 1 -- 172.21.15.18:0/18012012 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbed409d340 con 0x7fbee810a390 2024-04-16T17:04:01.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.270+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fbee8062ff0 con 0x7fbecc0a3c40 2024-04-16T17:04:01.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.272+0000 7fbedfe00700 1 -- 172.21.15.18:0/18012012 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fbee8062ff0 con 0x7fbecc0a3c40 2024-04-16T17:04:01.274 INFO:teuthology.orchestra.run.smithi018.stdout:Backend: cephadm 2024-04-16T17:04:01.274 INFO:teuthology.orchestra.run.smithi018.stdout:Available: Yes 2024-04-16T17:04:01.274 INFO:teuthology.orchestra.run.smithi018.stdout:Paused: No 2024-04-16T17:04:01.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.274+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbecc0a3c40 msgr2=0x7fbecc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:01.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.274+0000 7fbeee200700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbecc0a3c40 0x7fbecc0a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbed80038c0 tx=0x7fbed8005fb0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.274+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbee810a390 msgr2=0x7fbee819f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.274+0000 7fbeee200700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbee810a390 0x7fbee819f930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbed400e9f0 tx=0x7fbed400ed00 comp rx=0 tx=0).stop 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.275+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 shutdown_connections 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.275+0000 7fbeee200700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fbecc0a3c40 0x7fbecc0a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.275+0000 7fbeee200700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee810ad30 0x7fbee81a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.275+0000 7fbeee200700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbee8109190 0x7fbee819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.275+0000 7fbeee200700 1 --2- 172.21.15.18:0/18012012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbee810a390 0x7fbee819f930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.275+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 >> 172.21.15.18:0/18012012 conn(0x7fbee8078160 msgr2=0x7fbee810d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.275+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 shutdown_connections 2024-04-16T17:04:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:01.275+0000 7fbeee200700 1 -- 172.21.15.18:0/18012012 wait complete. 2024-04-16T17:04:02.252 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-16T17:04:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:02 smithi018 ceph-mon[31776]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:02.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:02 smithi018 ceph-mon[31776]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:02.477 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:02 smithi022 ceph-mon[36023]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:02.477 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:02 smithi022 ceph-mon[36023]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:02.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:02 smithi043 ceph-mon[36901]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:02.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:02 smithi043 ceph-mon[36901]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:02.732 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:04.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:04 smithi043 ceph-mon[36901]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:04.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:04 smithi018 ceph-mon[31776]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:04.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:04 smithi022 ceph-mon[36023]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.104+0000 7f6921c00700 1 -- 172.21.15.18:0/1695811728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 msgr2=0x7f691c10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.104+0000 7f6921c00700 1 --2- 172.21.15.18:0/1695811728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 0x7f691c10acb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f691400b0a0 tx=0x7f691400b3b0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.105+0000 7f6921c00700 1 -- 172.21.15.18:0/1695811728 shutdown_connections 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.105+0000 7f6921c00700 1 --2- 172.21.15.18:0/1695811728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 0x7f691c10acb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.105+0000 7f6921c00700 1 --2- 172.21.15.18:0/1695811728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f691c103640 0x7f691c105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.105+0000 7f6921c00700 1 --2- 172.21.15.18:0/1695811728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c105f70 0x7f691c108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.105+0000 7f6921c00700 1 -- 172.21.15.18:0/1695811728 >> 172.21.15.18:0/1695811728 conn(0x7f691c0fcff0 msgr2=0x7f691c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.105+0000 7f6921c00700 1 -- 172.21.15.18:0/1695811728 shutdown_connections 2024-04-16T17:04:05.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.105+0000 7f6921c00700 1 -- 172.21.15.18:0/1695811728 wait complete. 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.106+0000 7f6921c00700 1 Processor -- start 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.106+0000 7f6921c00700 1 -- start start 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.106+0000 7f6921c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f691c103640 0x7f691c102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.106+0000 7f6921c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c105f70 0x7f691c1015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.106+0000 7f6921c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 0x7f691c101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.106+0000 7f6921c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10e770 con 0x7f691c1088a0 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.106+0000 7f6921c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10e5f0 con 0x7f691c105f70 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.106+0000 7f6921c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10e8f0 con 0x7f691c103640 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f691c103640 0x7f691c102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f691c103640 0x7f691c102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:45028/0 (socket says 172.21.15.18:45028) 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691aa00700 1 -- 172.21.15.18:0/1682502378 learned_addr learned my addr 172.21.15.18:0/1682502378 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691a000700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c105f70 0x7f691c1015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691b400700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 0x7f691c101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:05.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691aa00700 1 -- 172.21.15.18:0/1682502378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c105f70 msgr2=0x7f691c1015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691aa00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c105f70 0x7f691c1015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691aa00700 1 -- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 msgr2=0x7f691c101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691aa00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 0x7f691c101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691aa00700 1 -- 172.21.15.18:0/1682502378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c102370 con 0x7f691c103640 2024-04-16T17:04:05.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691b400700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 0x7f691c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:05.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.107+0000 7f691a000700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c105f70 0x7f691c1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:05.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.108+0000 7f691aa00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f691c103640 0x7f691c102f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f690c00e9c0 tx=0x7f690c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:05.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.108+0000 7f6913400700 1 -- 172.21.15.18:0/1682502378 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c00cbd0 con 0x7f691c103640 2024-04-16T17:04:05.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.108+0000 7f6913400700 1 -- 172.21.15.18:0/1682502378 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f690c00cd70 con 0x7f691c103640 2024-04-16T17:04:05.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.108+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1a2ee0 con 0x7f691c103640 2024-04-16T17:04:05.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.108+0000 7f6913400700 1 -- 172.21.15.18:0/1682502378 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c010430 con 0x7f691c103640 2024-04-16T17:04:05.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.108+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1a32d0 con 0x7f691c103640 2024-04-16T17:04:05.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.109+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c0697e0 con 0x7f691c103640 2024-04-16T17:04:05.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.112+0000 7f6913400700 1 -- 172.21.15.18:0/1682502378 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f690c004750 con 0x7f691c103640 2024-04-16T17:04:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.113+0000 7f6913400700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f69000a3c40 0x7f69000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:05.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.113+0000 7f6913400700 1 -- 172.21.15.18:0/1682502378 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f690c0d0ad0 con 0x7f691c103640 2024-04-16T17:04:05.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.113+0000 7f691a000700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f69000a3c40 0x7f69000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:05.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.114+0000 7f691a000700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f69000a3c40 0x7f69000a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6908009680 tx=0x7f6908009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.117+0000 7f6913400700 1 -- 172.21.15.18:0/1682502378 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f690c09d080 con 0x7f691c103640 2024-04-16T17:04:05.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.362+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f691c062f30 con 0x7f69000a3c40 2024-04-16T17:04:05.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.403+0000 7f6913400700 1 -- 172.21.15.18:0/1682502378 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f691c062f30 con 0x7f69000a3c40 2024-04-16T17:04:05.405 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.a smithi018 running (2m) 52s ago 2m 12.4M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe af5e644397b5 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.b smithi022 running (2m) 97s ago 2m 26.0M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe f45263a6338f 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.c smithi043 running (3m) 97s ago 3m 17.2M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 66e397f7e4e1 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.d smithi018 running (3m) 52s ago 3m 12.6M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe fe06c480a72f 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.e smithi022 running (3m) 97s ago 3m 13.2M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 596e7819f6cb 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.f smithi043 running (3m) 97s ago 3m 18.1M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe b8f38de83cbb 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.g smithi018 running (2m) 52s ago 2m 16.4M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe fc88423e24a5 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.h smithi022 running (3m) 97s ago 3m 17.0M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe f0ff3c200e56 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.i smithi043 running (2m) 97s ago 2m 17.0M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 859a77133dd0 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.j smithi018 running (3m) 52s ago 3m 12.7M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe a65dd7e3a5af 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.k smithi022 running (3m) 97s ago 3m 16.4M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe a99abe97caad 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mds.l smithi043 running (3m) 97s ago 3m 16.4M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 83bd4e8f0a30 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.x smithi018 *:9283,8765 running (11m) 52s ago 11m 518M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 29fb07aec5fd 2024-04-16T17:04:05.406 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.y smithi022 *:8443,8765 running (9m) 97s ago 9m 454M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe d5136006e60c 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.z smithi043 *:8443,8765 running (9m) 97s ago 9m 454M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 4749a67dbb9e 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:mon.a smithi018 running (11m) 52s ago 11m 48.8M 2048M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 387238bc74ca 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:mon.b smithi022 running (9m) 97s ago 9m 41.8M 2048M 19.0.0-3101-g9b42bfbe e5ce10bdffbe a4591baa9f1d 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:mon.c smithi043 running (9m) 97s ago 9m 40.7M 2048M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 986fbbd02f66 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (8m) 52s ago 8m 44.9M 4273M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 52b5ee851b17 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi022 running (8m) 97s ago 8m 46.9M 4271M 19.0.0-3101-g9b42bfbe e5ce10bdffbe e2558225937c 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi043 running (8m) 97s ago 8m 57.9M 4271M 19.0.0-3101-g9b42bfbe e5ce10bdffbe a1f0e6eb9fb0 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi018 running (7m) 52s ago 7m 53.8M 4273M 19.0.0-3101-g9b42bfbe e5ce10bdffbe fe37aec92614 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi022 running (7m) 97s ago 7m 43.3M 4271M 19.0.0-3101-g9b42bfbe e5ce10bdffbe d55c4e4f4715 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi043 running (6m) 97s ago 6m 56.5M 4271M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 9a4fb67392f5 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6 smithi018 running (6m) 52s ago 6m 52.1M 4273M 19.0.0-3101-g9b42bfbe e5ce10bdffbe c80c80357d27 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7 smithi022 running (5m) 97s ago 5m 43.9M 4271M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 341a3492349b 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.8 smithi043 running (5m) 97s ago 5m 56.4M 4271M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 56b999447a87 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.9 smithi018 running (4m) 52s ago 4m 42.1M 4273M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 332f0d9e2081 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.10 smithi022 running (4m) 97s ago 4m 54.3M 4271M 19.0.0-3101-g9b42bfbe e5ce10bdffbe b5d529b53511 2024-04-16T17:04:05.407 INFO:teuthology.orchestra.run.smithi018.stdout:osd.11 smithi043 running (3m) 97s ago 3m 42.2M 4271M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 3831aec41608 2024-04-16T17:04:05.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f69000a3c40 msgr2=0x7f69000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f69000a3c40 0x7f69000a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6908009680 tx=0x7f6908009260 comp rx=0 tx=0).stop 2024-04-16T17:04:05.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f691c103640 msgr2=0x7f691c102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f691c103640 0x7f691c102f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f690c00e9c0 tx=0x7f690c00ecd0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 shutdown_connections 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f69000a3c40 0x7f69000a6100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f691c1088a0 0x7f691c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f691c103640 0x7f691c102f30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 --2- 172.21.15.18:0/1682502378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c105f70 0x7f691c1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.407+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 >> 172.21.15.18:0/1682502378 conn(0x7f691c0fcff0 msgr2=0x7f691c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.408+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 shutdown_connections 2024-04-16T17:04:05.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:05.408+0000 7f6921c00700 1 -- 172.21.15.18:0/1682502378 wait complete. 2024-04-16T17:04:06.263 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-16T17:04:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:06 smithi018 ceph-mon[31776]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:06.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:06 smithi018 ceph-mon[31776]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:06 smithi043 ceph-mon[36901]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:06.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:06 smithi043 ceph-mon[36901]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:06 smithi022 ceph-mon[36023]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:06 smithi022 ceph-mon[36023]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:06.751 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:08 smithi018 ceph-mon[31776]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:08 smithi043 ceph-mon[36901]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:08.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:08 smithi022 ceph-mon[36023]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:09.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.067+0000 7f9b12800700 1 -- 172.21.15.18:0/2009465013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c105f70 msgr2=0x7f9b0c108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.067+0000 7f9b12800700 1 --2- 172.21.15.18:0/2009465013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c105f70 0x7f9b0c108360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9af80098f0 tx=0x7f9af8009c00 comp rx=0 tx=0).stop 2024-04-16T17:04:09.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.067+0000 7f9b12800700 1 -- 172.21.15.18:0/2009465013 shutdown_connections 2024-04-16T17:04:09.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.067+0000 7f9b12800700 1 --2- 172.21.15.18:0/2009465013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b0c103640 0x7f9b0c105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.067+0000 7f9b12800700 1 --2- 172.21.15.18:0/2009465013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b0c1088a0 0x7f9b0c10acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.067+0000 7f9b12800700 1 --2- 172.21.15.18:0/2009465013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c105f70 0x7f9b0c108360 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.067+0000 7f9b12800700 1 -- 172.21.15.18:0/2009465013 >> 172.21.15.18:0/2009465013 conn(0x7f9b0c0fcff0 msgr2=0x7f9b0c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:09.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.067+0000 7f9b12800700 1 -- 172.21.15.18:0/2009465013 shutdown_connections 2024-04-16T17:04:09.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.068+0000 7f9b12800700 1 -- 172.21.15.18:0/2009465013 wait complete. 2024-04-16T17:04:09.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b12800700 1 Processor -- start 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b12800700 1 -- start start 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b12800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c103640 0x7f9b0c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b12800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b0c105f70 0x7f9b0c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b12800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b0c1088a0 0x7f9b0c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b12800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10e7a0 con 0x7f9b0c105f70 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b12800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10e620 con 0x7f9b0c1088a0 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b12800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10e920 con 0x7f9b0c103640 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b0b400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c103640 0x7f9b0c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b0b400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c103640 0x7f9b0c198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:33012/0 (socket says 172.21.15.18:33012) 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b0b400700 1 -- 172.21.15.18:0/3407807951 learned_addr learned my addr 172.21.15.18:0/3407807951 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.069+0000 7f9b0aa00700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b0c105f70 0x7f9b0c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.070+0000 7f9b0be00700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b0c1088a0 0x7f9b0c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:09.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.070+0000 7f9b0b400700 1 -- 172.21.15.18:0/3407807951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b0c1088a0 msgr2=0x7f9b0c1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.070+0000 7f9b0b400700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b0c1088a0 0x7f9b0c1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.070+0000 7f9b0b400700 1 -- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b0c105f70 msgr2=0x7f9b0c1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.070+0000 7f9b0b400700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b0c105f70 0x7f9b0c1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.070+0000 7f9b0b400700 1 -- 172.21.15.18:0/3407807951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c1a0ae0 con 0x7f9b0c103640 2024-04-16T17:04:09.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.070+0000 7f9b0b400700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c103640 0x7f9b0c198e80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00e9f0 tx=0x7f9afc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:09.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.071+0000 7f9b03e00700 1 -- 172.21.15.18:0/3407807951 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00cbd0 con 0x7f9b0c103640 2024-04-16T17:04:09.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.071+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c1a0dd0 con 0x7f9b0c103640 2024-04-16T17:04:09.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.071+0000 7f9b03e00700 1 -- 172.21.15.18:0/3407807951 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc00ee90 con 0x7f9b0c103640 2024-04-16T17:04:09.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.071+0000 7f9b03e00700 1 -- 172.21.15.18:0/3407807951 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0056c0 con 0x7f9b0c103640 2024-04-16T17:04:09.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.071+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c1a11d0 con 0x7f9b0c103640 2024-04-16T17:04:09.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.072+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b0c0697e0 con 0x7f9b0c103640 2024-04-16T17:04:09.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.074+0000 7f9b03e00700 1 -- 172.21.15.18:0/3407807951 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9afc004750 con 0x7f9b0c103640 2024-04-16T17:04:09.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.076+0000 7f9b03e00700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9af00a3c40 0x7f9af00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:09.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.076+0000 7f9b03e00700 1 -- 172.21.15.18:0/3407807951 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f9afc0d09f0 con 0x7f9b0c103640 2024-04-16T17:04:09.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.076+0000 7f9b0aa00700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9af00a3c40 0x7f9af00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:09.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.077+0000 7f9b0aa00700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9af00a3c40 0x7f9af00a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009810 tx=0x7f9af8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.078+0000 7f9b03e00700 1 -- 172.21.15.18:0/3407807951 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9afc09cfa0 con 0x7f9b0c103640 2024-04-16T17:04:09.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.318+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9b0c062f30 con 0x7f9af00a3c40 2024-04-16T17:04:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.336+0000 7f9b03e00700 1 -- 172.21.15.18:0/3407807951 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9b0c062f30 con 0x7f9af00a3c40 2024-04-16T17:04:09.338 INFO:teuthology.orchestra.run.smithi018.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-16T17:04:09.338 INFO:teuthology.orchestra.run.smithi018.stdout:mds.all 12/12 101s ago 3m smithi018=a;smithi018=d;smithi018=g;smithi018=j;smithi022=b;smithi022=e;smithi022=h;smithi022=k;smithi043=c;smithi043=f;smithi043=i;smithi043=l;count:12 2024-04-16T17:04:09.338 INFO:teuthology.orchestra.run.smithi018.stdout:mgr 3/3 101s ago 9m smithi018=x;smithi022=y;smithi043=z;count:3 2024-04-16T17:04:09.338 INFO:teuthology.orchestra.run.smithi018.stdout:mon 3/3 101s ago 9m smithi018:172.21.15.18=a;smithi022:172.21.15.22=b;smithi043:172.21.15.43=c;count:3 2024-04-16T17:04:09.338 INFO:teuthology.orchestra.run.smithi018.stdout:osd 12 101s ago - 2024-04-16T17:04:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.339+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9af00a3c40 msgr2=0x7f9af00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.339+0000 7f9b12800700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9af00a3c40 0x7f9af00a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009810 tx=0x7f9af8019040 comp rx=0 tx=0).stop 2024-04-16T17:04:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.339+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c103640 msgr2=0x7f9b0c198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.339+0000 7f9b12800700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c103640 0x7f9b0c198e80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00e9f0 tx=0x7f9afc00ed00 comp rx=0 tx=0).stop 2024-04-16T17:04:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.340+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 shutdown_connections 2024-04-16T17:04:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.340+0000 7f9b12800700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f9af00a3c40 0x7f9af00a6100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.340+0000 7f9b12800700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b0c1088a0 0x7f9b0c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.340+0000 7f9b12800700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b0c103640 0x7f9b0c198e80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.340+0000 7f9b12800700 1 --2- 172.21.15.18:0/3407807951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b0c105f70 0x7f9b0c1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.340+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 >> 172.21.15.18:0/3407807951 conn(0x7f9b0c0fcff0 msgr2=0x7f9b0c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.340+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 shutdown_connections 2024-04-16T17:04:09.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:09.340+0000 7f9b12800700 1 -- 172.21.15.18:0/3407807951 wait complete. 2024-04-16T17:04:10.127 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-16T17:04:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:10 smithi018 ceph-mon[31776]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:10 smithi018 ceph-mon[31776]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:10.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:10 smithi043 ceph-mon[36901]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:10.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:10 smithi043 ceph-mon[36901]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:10.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:10 smithi022 ceph-mon[36023]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:10.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:10 smithi022 ceph-mon[36023]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:12.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:12 smithi043 ceph-mon[36901]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:12.548 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:12 smithi018 ceph-mon[31776]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:12.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:12 smithi022 ceph-mon[36023]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:12.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.880+0000 7fd5de800700 1 -- 172.21.15.18:0/1206448053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d8103d40 msgr2=0x7fd5d81061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:12.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.880+0000 7fd5de800700 1 --2- 172.21.15.18:0/1206448053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d8103d40 0x7fd5d81061d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009990 tx=0x7fd5c4009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:12.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.881+0000 7fd5de800700 1 -- 172.21.15.18:0/1206448053 shutdown_connections 2024-04-16T17:04:12.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.881+0000 7fd5de800700 1 --2- 172.21.15.18:0/1206448053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5d8106710 0x7fd5d8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:12.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.881+0000 7fd5de800700 1 --2- 172.21.15.18:0/1206448053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d81013d0 0x7fd5d8103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:12.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.881+0000 7fd5de800700 1 --2- 172.21.15.18:0/1206448053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d8103d40 0x7fd5d81061d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:12.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.881+0000 7fd5de800700 1 -- 172.21.15.18:0/1206448053 >> 172.21.15.18:0/1206448053 conn(0x7fd5d80fcff0 msgr2=0x7fd5d80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:12.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.881+0000 7fd5de800700 1 -- 172.21.15.18:0/1206448053 shutdown_connections 2024-04-16T17:04:12.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.881+0000 7fd5de800700 1 -- 172.21.15.18:0/1206448053 wait complete. 2024-04-16T17:04:12.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.883+0000 7fd5de800700 1 Processor -- start 2024-04-16T17:04:12.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.883+0000 7fd5de800700 1 -- start start 2024-04-16T17:04:12.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5de800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d81013d0 0x7fd5d819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:12.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5de800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5d8103d40 0x7fd5d819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:12.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5de800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d8106710 0x7fd5d81a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:12.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5de800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8114d00 con 0x7fd5d81013d0 2024-04-16T17:04:12.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5de800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8114b80 con 0x7fd5d8103d40 2024-04-16T17:04:12.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5de800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8114e80 con 0x7fd5d8106710 2024-04-16T17:04:12.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5d7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d81013d0 0x7fd5d819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:12.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5d7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d81013d0 0x7fd5d819f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34370/0 (socket says 172.21.15.18:34370) 2024-04-16T17:04:12.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5d7400700 1 -- 172.21.15.18:0/472792184 learned_addr learned my addr 172.21.15.18:0/472792184 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:12.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5d6a00700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5d8103d40 0x7fd5d819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:12.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.884+0000 7fd5d7e00700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d8106710 0x7fd5d81a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:12.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5d6a00700 1 -- 172.21.15.18:0/472792184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d8106710 msgr2=0x7fd5d81a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:12.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5d6a00700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d8106710 0x7fd5d81a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:12.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5d6a00700 1 -- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d81013d0 msgr2=0x7fd5d819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5d6a00700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d81013d0 0x7fd5d819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5d6a00700 1 -- 172.21.15.18:0/472792184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d81a6fe0 con 0x7fd5d8103d40 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5d7400700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d81013d0 0x7fd5d819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5d6a00700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5d8103d40 0x7fd5d819f8c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40098e0 tx=0x7fd5c4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5cfe00700 1 -- 172.21.15.18:0/472792184 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4027070 con 0x7fd5d8103d40 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5cfe00700 1 -- 172.21.15.18:0/472792184 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c4003d70 con 0x7fd5d8103d40 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5cfe00700 1 -- 172.21.15.18:0/472792184 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400f460 con 0x7fd5d8103d40 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.885+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d81a7270 con 0x7fd5d8103d40 2024-04-16T17:04:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.886+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d81a7730 con 0x7fd5d8103d40 2024-04-16T17:04:12.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.887+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d80697e0 con 0x7fd5d8103d40 2024-04-16T17:04:12.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.889+0000 7fd5cfe00700 1 -- 172.21.15.18:0/472792184 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd5c402a070 con 0x7fd5d8103d40 2024-04-16T17:04:12.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.890+0000 7fd5cfe00700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5bc0a3c40 0x7fd5bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:12.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.890+0000 7fd5cfe00700 1 -- 172.21.15.18:0/472792184 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd5c40d1e90 con 0x7fd5d8103d40 2024-04-16T17:04:12.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.891+0000 7fd5d7400700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5bc0a3c40 0x7fd5bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:12.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.891+0000 7fd5d7400700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5bc0a3c40 0x7fd5bc0a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8011590 tx=0x7fd5c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:12.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:12.893+0000 7fd5cfe00700 1 -- 172.21.15.18:0/472792184 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5c4031020 con 0x7fd5d8103d40 2024-04-16T17:04:13.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.129+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fd5d8062f30 con 0x7fd5bc0a3c40 2024-04-16T17:04:13.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.133+0000 7fd5cfe00700 1 -- 172.21.15.18:0/472792184 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7fd5d8062f30 con 0x7fd5bc0a3c40 2024-04-16T17:04:13.135 INFO:teuthology.orchestra.run.smithi018.stdout:HOST ADDR LABELS STATUS 2024-04-16T17:04:13.135 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018 172.21.15.18 2024-04-16T17:04:13.135 INFO:teuthology.orchestra.run.smithi018.stdout:smithi022 172.21.15.22 2024-04-16T17:04:13.135 INFO:teuthology.orchestra.run.smithi018.stdout:smithi043 172.21.15.43 2024-04-16T17:04:13.136 INFO:teuthology.orchestra.run.smithi018.stdout:3 hosts in cluster 2024-04-16T17:04:13.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.136+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5bc0a3c40 msgr2=0x7fd5bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:13.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.136+0000 7fd5de800700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5bc0a3c40 0x7fd5bc0a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8011590 tx=0x7fd5c8009260 comp rx=0 tx=0).stop 2024-04-16T17:04:13.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.136+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5d8103d40 msgr2=0x7fd5d819f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.136+0000 7fd5de800700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5d8103d40 0x7fd5d819f8c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40098e0 tx=0x7fd5c4005f10 comp rx=0 tx=0).stop 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.137+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 shutdown_connections 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.137+0000 7fd5de800700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd5bc0a3c40 0x7fd5bc0a6100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.137+0000 7fd5de800700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d8106710 0x7fd5d81a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.137+0000 7fd5de800700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5d81013d0 0x7fd5d819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.137+0000 7fd5de800700 1 --2- 172.21.15.18:0/472792184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5d8103d40 0x7fd5d819f8c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.137+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 >> 172.21.15.18:0/472792184 conn(0x7fd5d80fcff0 msgr2=0x7fd5d81021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.137+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 shutdown_connections 2024-04-16T17:04:13.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:13.137+0000 7fd5de800700 1 -- 172.21.15.18:0/472792184 wait complete. 2024-04-16T17:04:13.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-16T17:04:14.421 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:14.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:14 smithi043 ceph-mon[36901]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:14.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:14 smithi043 ceph-mon[36901]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:14.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:14 smithi018 ceph-mon[31776]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:14.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:14 smithi018 ceph-mon[31776]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:14.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:14 smithi022 ceph-mon[36023]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:14.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:14 smithi022 ceph-mon[36023]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:15.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:15.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.482 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.483 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:15 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:15.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:15 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:15.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:15.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:15 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:16.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:16 smithi043 ceph-mon[36901]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:16.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:16.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:16 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:16.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:16 smithi018 ceph-mon[31776]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:16.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:16.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:16 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:16.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.723+0000 7ff011600700 1 -- 172.21.15.18:0/2099555312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c105f10 msgr2=0x7ff00c108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:16.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.723+0000 7ff011600700 1 --2- 172.21.15.18:0/2099555312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c105f10 0x7ff00c108300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feff4009990 tx=0x7feff4009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:16.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.723+0000 7ff011600700 1 -- 172.21.15.18:0/2099555312 shutdown_connections 2024-04-16T17:04:16.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.723+0000 7ff011600700 1 --2- 172.21.15.18:0/2099555312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c108840 0x7ff00c10acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:16.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.723+0000 7ff011600700 1 --2- 172.21.15.18:0/2099555312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c1035e0 0x7ff00c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.723+0000 7ff011600700 1 --2- 172.21.15.18:0/2099555312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c105f10 0x7ff00c108300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.723+0000 7ff011600700 1 -- 172.21.15.18:0/2099555312 >> 172.21.15.18:0/2099555312 conn(0x7ff00c0fd010 msgr2=0x7ff00c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.723+0000 7ff011600700 1 -- 172.21.15.18:0/2099555312 shutdown_connections 2024-04-16T17:04:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.724+0000 7ff011600700 1 -- 172.21.15.18:0/2099555312 wait complete. 2024-04-16T17:04:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff011600700 1 Processor -- start 2024-04-16T17:04:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff011600700 1 -- start start 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff011600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c1035e0 0x7ff00c19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff011600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c105f10 0x7ff00c19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff011600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c108840 0x7ff00c1a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff011600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10e7a0 con 0x7ff00c105f10 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff011600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10e620 con 0x7ff00c108840 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff011600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10e920 con 0x7ff00c1035e0 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff00aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c1035e0 0x7ff00c19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.725+0000 7ff00b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c108840 0x7ff00c1a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c1035e0 0x7ff00c19b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:33050/0 (socket says 172.21.15.18:33050) 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00aa00700 1 -- 172.21.15.18:0/1553140807 learned_addr learned my addr 172.21.15.18:0/1553140807 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff003e00700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c105f10 0x7ff00c19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00aa00700 1 -- 172.21.15.18:0/1553140807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c108840 msgr2=0x7ff00c1a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00aa00700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c108840 0x7ff00c1a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00aa00700 1 -- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c105f10 msgr2=0x7ff00c19b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00aa00700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c105f10 0x7ff00c19b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00aa00700 1 -- 172.21.15.18:0/1553140807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c1a2c80 con 0x7ff00c1035e0 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff003e00700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c105f10 0x7ff00c19b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00b400700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c108840 0x7ff00c1a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.726+0000 7ff00aa00700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c1035e0 0x7ff00c19b020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feffc00e9f0 tx=0x7feffc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.727+0000 7ff003400700 1 -- 172.21.15.18:0/1553140807 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc00cbd0 con 0x7ff00c1035e0 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.727+0000 7ff003400700 1 -- 172.21.15.18:0/1553140807 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feffc00cd70 con 0x7ff00c1035e0 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.727+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c1a2f70 con 0x7ff00c1035e0 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.727+0000 7ff003400700 1 -- 172.21.15.18:0/1553140807 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc010430 con 0x7ff00c1035e0 2024-04-16T17:04:16.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.727+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c1a33c0 con 0x7ff00c1035e0 2024-04-16T17:04:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.728+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c0697e0 con 0x7ff00c1035e0 2024-04-16T17:04:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.730+0000 7ff003400700 1 -- 172.21.15.18:0/1553140807 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feffc004750 con 0x7ff00c1035e0 2024-04-16T17:04:16.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.731+0000 7ff003400700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feff00a3cc0 0x7feff00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:16.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.732+0000 7ff003400700 1 -- 172.21.15.18:0/1553140807 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7feffc0d0aa0 con 0x7ff00c1035e0 2024-04-16T17:04:16.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.732+0000 7ff003e00700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feff00a3cc0 0x7feff00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:16.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.733+0000 7ff003e00700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feff00a3cc0 0x7feff00a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feff400b2e0 tx=0x7feff401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:16.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:16.734+0000 7ff003400700 1 -- 172.21.15.18:0/1553140807 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feffc09d050 con 0x7ff00c1035e0 2024-04-16T17:04:16.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:16 smithi022 ceph-mon[36023]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:16.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:16.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:16 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:17.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.017+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff00c062f30 con 0x7feff00a3cc0 2024-04-16T17:04:17.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.020+0000 7ff003400700 1 -- 172.21.15.18:0/1553140807 <== mgr.14152 v2:172.21.15.18:6800/2462414912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff00c062f30 con 0x7feff00a3cc0 2024-04-16T17:04:17.022 INFO:teuthology.orchestra.run.smithi018.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-16T17:04:17.022 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120091480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-04-16T17:04:17.022 INFO:teuthology.orchestra.run.smithi018.stdout:smithi022 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402EC480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-04-16T17:04:17.022 INFO:teuthology.orchestra.run.smithi018.stdout:smithi043 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GH480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.022+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feff00a3cc0 msgr2=0x7feff00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.022+0000 7ff011600700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feff00a3cc0 0x7feff00a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feff400b2e0 tx=0x7feff401a040 comp rx=0 tx=0).stop 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c1035e0 msgr2=0x7ff00c19b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c1035e0 0x7ff00c19b020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feffc00e9f0 tx=0x7feffc00ed00 comp rx=0 tx=0).stop 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 shutdown_connections 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7feff00a3cc0 0x7feff00a6180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff00c108840 0x7ff00c1a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c1035e0 0x7ff00c19b020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 --2- 172.21.15.18:0/1553140807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff00c105f10 0x7ff00c19b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:17.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 >> 172.21.15.18:0/1553140807 conn(0x7ff00c0fd010 msgr2=0x7ff00c102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:17.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 shutdown_connections 2024-04-16T17:04:17.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:17.023+0000 7ff011600700 1 -- 172.21.15.18:0/1553140807 wait complete. 2024-04-16T17:04:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:17 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:17 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:17 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:17.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:17 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:17 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:17 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:17 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:17 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:17 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:17 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:17 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:17 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:18.126 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-16T17:04:18.136 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:04:18.136 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-16T17:04:18.615 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:18.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:18 smithi022 ceph-mon[36023]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:18.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:18 smithi022 ceph-mon[36023]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:18.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:18 smithi018 ceph-mon[31776]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:18.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:18 smithi018 ceph-mon[31776]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:19.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:18 smithi043 ceph-mon[36901]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:19.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:18 smithi043 ceph-mon[36901]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:20.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.788+0000 7f1fcc000700 1 -- 172.21.15.18:0/4112984142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc41035e0 msgr2=0x7f1fc41059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:20.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.788+0000 7f1fcc000700 1 --2- 172.21.15.18:0/4112984142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc41035e0 0x7f1fc41059d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80098f0 tx=0x7f1fb8009c00 comp rx=0 tx=0).stop 2024-04-16T17:04:20.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.788+0000 7f1fcc000700 1 -- 172.21.15.18:0/4112984142 shutdown_connections 2024-04-16T17:04:20.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.788+0000 7f1fcc000700 1 --2- 172.21.15.18:0/4112984142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc4108840 0x7f1fc410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:20.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.788+0000 7f1fcc000700 1 --2- 172.21.15.18:0/4112984142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc41035e0 0x7f1fc41059d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:20.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.788+0000 7f1fcc000700 1 --2- 172.21.15.18:0/4112984142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc4105f10 0x7f1fc4108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:20.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.788+0000 7f1fcc000700 1 -- 172.21.15.18:0/4112984142 >> 172.21.15.18:0/4112984142 conn(0x7f1fc40fcff0 msgr2=0x7f1fc40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:20.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:20 smithi018 ceph-mon[31776]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.789+0000 7f1fcc000700 1 -- 172.21.15.18:0/4112984142 shutdown_connections 2024-04-16T17:04:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.789+0000 7f1fcc000700 1 -- 172.21.15.18:0/4112984142 wait complete. 2024-04-16T17:04:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.791+0000 7f1fcc000700 1 Processor -- start 2024-04-16T17:04:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.791+0000 7f1fcc000700 1 -- start start 2024-04-16T17:04:20.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fcc000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41035e0 0x7f1fc4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:20.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fcc000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc4105f10 0x7f1fc4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:20.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fcc000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc4108840 0x7f1fc41a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fcc000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc410e770 con 0x7f1fc41035e0 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fcc000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc410e5f0 con 0x7f1fc4105f10 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fcc000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc410e8f0 con 0x7f1fc4108840 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fc8c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc4105f10 0x7f1fc4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fc9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41035e0 0x7f1fc4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fc8c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc4105f10 0x7f1fc4199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34816/0 (socket says 172.21.15.18:34816) 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fc8c00700 1 -- 172.21.15.18:0/172444186 learned_addr learned my addr 172.21.15.18:0/172444186 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fc9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41035e0 0x7f1fc4198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33280/0 (socket says 172.21.15.18:33280) 2024-04-16T17:04:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.792+0000 7f1fca000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc4108840 0x7f1fc41a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.793+0000 7f1fc9600700 1 -- 172.21.15.18:0/172444186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc4108840 msgr2=0x7f1fc41a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.793+0000 7f1fc9600700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc4108840 0x7f1fc41a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.793+0000 7f1fc9600700 1 -- 172.21.15.18:0/172444186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc4105f10 msgr2=0x7f1fc4199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.793+0000 7f1fc9600700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc4105f10 0x7f1fc4199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.793+0000 7f1fc9600700 1 -- 172.21.15.18:0/172444186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc41a0a80 con 0x7f1fc41035e0 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.793+0000 7f1fca000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc4108840 0x7f1fc41a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.793+0000 7f1fc9600700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41035e0 0x7f1fc4198e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009230 tx=0x7f1fb8010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.794+0000 7f1fc2000700 1 -- 172.21.15.18:0/172444186 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8005350 con 0x7f1fc41035e0 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.794+0000 7f1fc2000700 1 -- 172.21.15.18:0/172444186 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb80054f0 con 0x7f1fc41035e0 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.794+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc41a0d10 con 0x7f1fc41035e0 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.794+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc41a11c0 con 0x7f1fc41035e0 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.794+0000 7f1fc2000700 1 -- 172.21.15.18:0/172444186 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8003cc0 con 0x7f1fc41035e0 2024-04-16T17:04:20.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.795+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa4005360 con 0x7f1fc41035e0 2024-04-16T17:04:20.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.797+0000 7f1fc2000700 1 -- 172.21.15.18:0/172444186 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1fb800b560 con 0x7f1fc41035e0 2024-04-16T17:04:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.799+0000 7f1fc2000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1fac0a3cc0 0x7f1fac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.800+0000 7f1fc2000700 1 -- 172.21.15.18:0/172444186 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f1fb80d1550 con 0x7f1fc41035e0 2024-04-16T17:04:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.800+0000 7f1fc8c00700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1fac0a3cc0 0x7f1fac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:20.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.801+0000 7f1fc8c00700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1fac0a3cc0 0x7f1fac0a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009680 tx=0x7f1fb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:20.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:20.802+0000 7f1fc2000700 1 -- 172.21.15.18:0/172444186 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fb805cff0 con 0x7f1fc41035e0 2024-04-16T17:04:20.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:20 smithi022 ceph-mon[36023]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:21.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:20 smithi043 ceph-mon[36901]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:21.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.103+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1fa4005920 con 0x7f1fc41035e0 2024-04-16T17:04:21.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.104+0000 7f1fc2000700 1 -- 172.21.15.18:0/172444186 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3036 (secure 0 0 0) 0x7f1fb8013030 con 0x7f1fc41035e0 2024-04-16T17:04:21.106 INFO:teuthology.orchestra.run.smithi018.stdout:e24 2024-04-16T17:04:21.106 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T17:04:21.106 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T17:04:21.106 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-16T17:04:21.106 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:21.106 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-16T17:04:21.106 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-16T17:04:21.106 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-16T17:01:24.398529+0000 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-16T17:01:59.165885+0000 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 300 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 0 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T17:04:21.107 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 3 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1,2 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=24596,1=24566,2=24547} 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3,4] 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 0 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 24596 members: 24566,24547,24596 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.b{0:24596} state up:active seq 3 addr [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.h{0:24536} state up:standby-replay seq 1 addr [v2:172.21.15.22:6836/94342022,v1:172.21.15.22:6837/94342022] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.g{1:24566} state up:active seq 13 addr [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.c{1:24523} state up:standby-replay seq 1 addr [v2:172.21.15.43:6836/1790976373,v1:172.21.15.43:6837/1790976373] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.i{2:24547} state up:active seq 14 addr [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.108 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.f{2:24511} state up:standby-replay seq 1 addr [v2:172.21.15.43:6834/1265446335,v1:172.21.15.43:6835/1265446335] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.d{-1:14406} state up:standby seq 1 addr [v2:172.21.15.18:6836/620951660,v1:172.21.15.18:6837/620951660] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.a{-1:14469} state up:standby seq 1 addr [v2:172.21.15.18:6840/2108347898,v1:172.21.15.18:6841/2108347898] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.43:6832/3903003819,v1:172.21.15.43:6833/3903003819] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.k{-1:24461} state up:standby seq 1 addr [v2:172.21.15.22:6832/4293504707,v1:172.21.15.22:6833/4293504707] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.j{-1:24479} state up:standby seq 1 addr [v2:172.21.15.18:6834/1011323908,v1:172.21.15.18:6835/1011323908] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.109 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.e{-1:24509} state up:standby seq 1 addr [v2:172.21.15.22:6834/3315767064,v1:172.21.15.22:6835/3315767064] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:04:21.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1fac0a3cc0 msgr2=0x7f1fac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:21.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1fac0a3cc0 0x7f1fac0a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009680 tx=0x7f1fb0009260 comp rx=0 tx=0).stop 2024-04-16T17:04:21.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41035e0 msgr2=0x7f1fc4198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:21.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41035e0 0x7f1fc4198e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009230 tx=0x7f1fb8010f80 comp rx=0 tx=0).stop 2024-04-16T17:04:21.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 shutdown_connections 2024-04-16T17:04:21.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1fac0a3cc0 0x7f1fac0a6180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:21.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fc4108840 0x7f1fc41a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:21.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fc41035e0 0x7f1fc4198e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:21.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 --2- 172.21.15.18:0/172444186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc4105f10 0x7f1fc4199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:21.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 >> 172.21.15.18:0/172444186 conn(0x7f1fc40fcff0 msgr2=0x7f1fc4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:21.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 shutdown_connections 2024-04-16T17:04:21.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:21.108+0000 7f1fcc000700 1 -- 172.21.15.18:0/172444186 wait complete. 2024-04-16T17:04:21.112 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 24 2024-04-16T17:04:21.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-16T17:04:22.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:22 smithi018 ceph-mon[31776]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:22.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:22.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:22 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:22.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:22 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/172444186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T17:04:22.401 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:22.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:22 smithi022 ceph-mon[36023]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:22.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:22.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:22 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:22.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:22 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/172444186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T17:04:22.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:22 smithi043 ceph-mon[36901]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:22.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:22.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:22 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:22.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:22 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/172444186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T17:04:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:23.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:23 smithi018 ceph-mon[31776]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:23.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:23 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:23.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:23 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:23.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:23 smithi022 ceph-mon[36023]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:23.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:23.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:23.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:23 smithi043 ceph-mon[36901]: from='mgr.14152 172.21.15.18:0/3374077423' entity='mgr.x' 2024-04-16T17:04:24.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:24 smithi022 ceph-mon[36023]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:24.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:24 smithi018 ceph-mon[31776]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:24.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:24 smithi043 ceph-mon[36901]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 -- 172.21.15.18:0/833183749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c84105e90 msgr2=0x7f1c841062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 --2- 172.21.15.18:0/833183749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c84105e90 0x7f1c841062f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1c700099c0 tx=0x7f1c70009cd0 comp rx=0 tx=0).stop 2024-04-16T17:04:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 -- 172.21.15.18:0/833183749 shutdown_connections 2024-04-16T17:04:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 --2- 172.21.15.18:0/833183749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c84104c90 0x7f1c841050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 --2- 172.21.15.18:0/833183749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c84106830 0x7f1c8410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 --2- 172.21.15.18:0/833183749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c84105e90 0x7f1c841062f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:24.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 -- 172.21.15.18:0/833183749 >> 172.21.15.18:0/833183749 conn(0x7f1c84100210 msgr2=0x7f1c84102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:24.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 -- 172.21.15.18:0/833183749 shutdown_connections 2024-04-16T17:04:24.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.905+0000 7f1c89c00700 1 -- 172.21.15.18:0/833183749 wait complete. 2024-04-16T17:04:24.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.907+0000 7f1c89c00700 1 Processor -- start 2024-04-16T17:04:24.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.907+0000 7f1c89c00700 1 -- start start 2024-04-16T17:04:24.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.907+0000 7f1c89c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c84104c90 0x7f1c8419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.907+0000 7f1c89c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c84105e90 0x7f1c8419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.907+0000 7f1c89c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c84106830 0x7f1c841a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.907+0000 7f1c89c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841108a0 con 0x7f1c84106830 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.907+0000 7f1c89c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84110720 con 0x7f1c84104c90 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.907+0000 7f1c89c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84110a20 con 0x7f1c84105e90 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c84104c90 0x7f1c8419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c84104c90 0x7f1c8419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34834/0 (socket says 172.21.15.18:34834) 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c89200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c84106830 0x7f1c841a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83e00700 1 -- 172.21.15.18:0/1287631309 learned_addr learned my addr 172.21.15.18:0/1287631309 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:24.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83400700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c84105e90 0x7f1c8419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:24.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83e00700 1 -- 172.21.15.18:0/1287631309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c84105e90 msgr2=0x7f1c8419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:24.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83e00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c84105e90 0x7f1c8419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:24.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83e00700 1 -- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c84106830 msgr2=0x7f1c841a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:24.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83e00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c84106830 0x7f1c841a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:24.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.908+0000 7f1c83e00700 1 -- 172.21.15.18:0/1287631309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c841a2ab0 con 0x7f1c84104c90 2024-04-16T17:04:24.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.909+0000 7f1c83e00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c84104c90 0x7f1c8419ae50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400efc0 tx=0x7f1c7400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:24.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.909+0000 7f1c80c00700 1 -- 172.21.15.18:0/1287631309 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c740092e0 con 0x7f1c84104c90 2024-04-16T17:04:24.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.909+0000 7f1c80c00700 1 -- 172.21.15.18:0/1287631309 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7400c9c0 con 0x7f1c84104c90 2024-04-16T17:04:24.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.909+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c841a2d40 con 0x7f1c84104c90 2024-04-16T17:04:24.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.909+0000 7f1c80c00700 1 -- 172.21.15.18:0/1287631309 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74004610 con 0x7f1c84104c90 2024-04-16T17:04:24.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.909+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c841a3210 con 0x7f1c84104c90 2024-04-16T17:04:24.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.910+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c840697e0 con 0x7f1c84104c90 2024-04-16T17:04:24.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.912+0000 7f1c80c00700 1 -- 172.21.15.18:0/1287631309 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c7401b030 con 0x7f1c84104c90 2024-04-16T17:04:24.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.914+0000 7f1c80c00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1c680a3cc0 0x7f1c680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:24.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.914+0000 7f1c80c00700 1 -- 172.21.15.18:0/1287631309 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f1c740d0ff0 con 0x7f1c84104c90 2024-04-16T17:04:24.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.914+0000 7f1c83400700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1c680a3cc0 0x7f1c680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:24.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.915+0000 7f1c83400700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1c680a3cc0 0x7f1c680a6180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000c00 tx=0x7f1c70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:24.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:24.916+0000 7f1c80c00700 1 -- 172.21.15.18:0/1287631309 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c74022020 con 0x7f1c84104c90 2024-04-16T17:04:25.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.156+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f1c84063700 con 0x7f1c84104c90 2024-04-16T17:04:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.158+0000 7f1c80c00700 1 -- 172.21.15.18:0/1287631309 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5675 (secure 0 0 0) 0x7f1c74010040 con 0x7f1c84104c90 2024-04-16T17:04:25.160 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 93 2024-04-16T17:04:25.160 INFO:teuthology.orchestra.run.smithi018.stdout:fsid 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T17:04:25.161 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-16T16:52:16.225349+0000 2024-04-16T17:04:25.161 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-16T17:03:27.035238+0000 2024-04-16T17:04:25.161 INFO:teuthology.orchestra.run.smithi018.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-16T17:04:25.161 INFO:teuthology.orchestra.run.smithi018.stdout:crush_version 27 2024-04-16T17:04:25.161 INFO:teuthology.orchestra.run.smithi018.stdout:full_ratio 0.95 2024-04-16T17:04:25.161 INFO:teuthology.orchestra.run.smithi018.stdout:backfillfull_ratio 0.9 2024-04-16T17:04:25.161 INFO:teuthology.orchestra.run.smithi018.stdout:nearfull_ratio 0.85 2024-04-16T17:04:25.162 INFO:teuthology.orchestra.run.smithi018.stdout:require_min_compat_client luminous 2024-04-16T17:04:25.162 INFO:teuthology.orchestra.run.smithi018.stdout:min_compat_client luminous 2024-04-16T17:04:25.162 INFO:teuthology.orchestra.run.smithi018.stdout:require_osd_release squid 2024-04-16T17:04:25.162 INFO:teuthology.orchestra.run.smithi018.stdout:stretch_mode_enabled false 2024-04-16T17:04:25.163 INFO:teuthology.orchestra.run.smithi018.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-16T17:04:25.163 INFO:teuthology.orchestra.run.smithi018.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-16T17:04:25.163 INFO:teuthology.orchestra.run.smithi018.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-16T17:04:25.163 INFO:teuthology.orchestra.run.smithi018.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-16T17:04:25.163 INFO:teuthology.orchestra.run.smithi018.stdout:max_osd 12 2024-04-16T17:04:25.163 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6802/2892109649,v1:172.21.15.18:6803/2892109649] [v2:172.21.15.18:6804/2892109649,v1:172.21.15.18:6805/2892109649] exists,up 50a6d2ab-b5fc-4430-99dc-5074f1d7e51f 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6800/1527176746,v1:172.21.15.22:6801/1527176746] [v2:172.21.15.22:6802/1527176746,v1:172.21.15.22:6803/1527176746] exists,up efef211f-28b9-4f7b-a3c7-59ccb4914bbe 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6800/3809455013,v1:172.21.15.43:6801/3809455013] [v2:172.21.15.43:6802/3809455013,v1:172.21.15.43:6803/3809455013] exists,up 0f783df6-43e0-4a3e-b413-af66cf95e18f 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 up in weight 1 up_from 25 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6810/2424709764,v1:172.21.15.18:6811/2424709764] [v2:172.21.15.18:6812/2424709764,v1:172.21.15.18:6813/2424709764] exists,up c0da47d3-ae80-4e06-a98e-887862be0ce4 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 up in weight 1 up_from 33 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6808/301052168,v1:172.21.15.22:6809/301052168] [v2:172.21.15.22:6810/301052168,v1:172.21.15.22:6811/301052168] exists,up 2338ca28-bed3-415a-b762-28d2275a3940 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6808/3934456075,v1:172.21.15.43:6809/3934456075] [v2:172.21.15.43:6810/3934456075,v1:172.21.15.43:6811/3934456075] exists,up dd247a85-21b7-4f15-bf4b-4f391dd6e543 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6818/3519009309,v1:172.21.15.18:6819/3519009309] [v2:172.21.15.18:6820/3519009309,v1:172.21.15.18:6821/3519009309] exists,up 1d5bb4a0-908c-41b1-9967-a35d4e77d851 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6816/1748007509,v1:172.21.15.22:6817/1748007509] [v2:172.21.15.22:6818/1748007509,v1:172.21.15.22:6819/1748007509] exists,up d3917a58-159e-4672-a8b1-83b0fe54efbb 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6816/546535747,v1:172.21.15.43:6817/546535747] [v2:172.21.15.43:6818/546535747,v1:172.21.15.43:6819/546535747] exists,up 06e72a0a-097f-49cb-9626-d0de66ba5641 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6826/3050280223,v1:172.21.15.18:6827/3050280223] [v2:172.21.15.18:6828/3050280223,v1:172.21.15.18:6829/3050280223] exists,up 7cf9ee08-5ede-446d-9b67-cebda4d2be28 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.10 up in weight 1 up_from 72 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6824/3445736292,v1:172.21.15.22:6825/3445736292] [v2:172.21.15.22:6826/3445736292,v1:172.21.15.22:6827/3445736292] exists,up 1a70d7bc-095f-4be6-9096-7bf5d46cd26d 2024-04-16T17:04:25.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6824/504916242,v1:172.21.15.43:6825/504916242] [v2:172.21.15.43:6826/504916242,v1:172.21.15.43:6827/504916242] exists,up 35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.0 [7,11] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.c [2,1] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.f [8,11] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.1b [2,4] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.2 [5,7] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.17 [4,6] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.1b [5,9] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.1e [3,9] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.3a [4,8] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 3.3c [4,9] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 4.f [9,8] 2024-04-16T17:04:25.165 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 4.12 [9,10] 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 4.1b [5,8] 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 4.1f [9,8] 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2364490471 expires 2024-04-17T16:53:24.732531+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6801/1759186913 expires 2024-04-17T16:53:24.732531+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2887519304 expires 2024-04-17T16:53:24.732531+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2656869540 expires 2024-04-17T16:52:49.432499+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/1032645701 expires 2024-04-17T16:52:49.432499+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6801/1116733165 expires 2024-04-17T16:52:49.432499+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6800/1116733165 expires 2024-04-17T16:52:49.432499+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6800/1759186913 expires 2024-04-17T16:53:24.732531+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2417150453 expires 2024-04-17T16:52:49.432499+0000 2024-04-16T17:04:25.166 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/3503805752 expires 2024-04-17T16:53:24.732531+0000 2024-04-16T17:04:25.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1c680a3cc0 msgr2=0x7f1c680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:25.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1c680a3cc0 0x7f1c680a6180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000c00 tx=0x7f1c70005fb0 comp rx=0 tx=0).stop 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c84104c90 msgr2=0x7f1c8419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c84104c90 0x7f1c8419ae50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400efc0 tx=0x7f1c7400c3a0 comp rx=0 tx=0).stop 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 shutdown_connections 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f1c680a3cc0 0x7f1c680a6180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c84104c90 0x7f1c8419ae50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c84106830 0x7f1c841a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1287631309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c84105e90 0x7f1c8419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 >> 172.21.15.18:0/1287631309 conn(0x7f1c84100210 msgr2=0x7f1c84109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 shutdown_connections 2024-04-16T17:04:25.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:25.161+0000 7f1c89c00700 1 -- 172.21.15.18:0/1287631309 wait complete. 2024-04-16T17:04:25.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:25 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1287631309' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T17:04:25.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:25 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1287631309' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T17:04:25.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:25 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1287631309' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T17:04:26.035 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-16T17:04:26.048 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:04:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.284+0000 7f9533200640 1 -- 172.21.15.18:0/241351110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105350 msgr2=0x7f95341057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.284+0000 7f9533200640 1 --2- 172.21.15.18:0/241351110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105350 0x7f95341057d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9520009900 tx=0x7f952002bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.285+0000 7f9533200640 1 -- 172.21.15.18:0/241351110 shutdown_connections 2024-04-16T17:04:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.285+0000 7f9533200640 1 --2- 172.21.15.18:0/241351110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9534105d10 0x7f953410c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.285+0000 7f9533200640 1 --2- 172.21.15.18:0/241351110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105350 0x7f95341057d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.285+0000 7f9533200640 1 --2- 172.21.15.18:0/241351110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9534104160 0x7f9534104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.285+0000 7f9533200640 1 -- 172.21.15.18:0/241351110 >> 172.21.15.18:0/241351110 conn(0x7f95340ff6d0 msgr2=0x7f9534101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:26.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.285+0000 7f9533200640 1 -- 172.21.15.18:0/241351110 shutdown_connections 2024-04-16T17:04:26.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.285+0000 7f9533200640 1 -- 172.21.15.18:0/241351110 wait complete. 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.286+0000 7f9533200640 1 Processor -- start 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.286+0000 7f9533200640 1 -- start start 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9533200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9534104160 0x7f953419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9533200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9534105350 0x7f953419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9533200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105d10 0x7f95341a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9533200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f953410fdf0 con 0x7f9534105d10 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9533200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f953410fc70 con 0x7f9534105350 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9533200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f953410ff70 con 0x7f9534104160 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9532800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105d10 0x7f95341a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9531400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9534105350 0x7f953419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9531e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9534104160 0x7f953419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9531400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9534105350 0x7f953419a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34848/0 (socket says 172.21.15.18:34848) 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9532800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105d10 0x7f95341a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33306/0 (socket says 172.21.15.18:33306) 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9531400640 1 -- 172.21.15.18:0/1551488791 learned_addr learned my addr 172.21.15.18:0/1551488791 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9531400640 1 -- 172.21.15.18:0/1551488791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9534104160 msgr2=0x7f953419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.287+0000 7f9531400640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9534104160 0x7f953419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f9531400640 1 -- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105d10 msgr2=0x7f95341a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f9531400640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105d10 0x7f95341a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f9531400640 1 -- 172.21.15.18:0/1551488791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95341a2020 con 0x7f9534105350 2024-04-16T17:04:26.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f9531400640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9534105350 0x7f953419a940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9520009230 tx=0x7f952002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:26.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f952aa00640 1 -- 172.21.15.18:0/1551488791 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520041070 con 0x7f9534105350 2024-04-16T17:04:26.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f952aa00640 1 -- 172.21.15.18:0/1551488791 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520004450 con 0x7f9534105350 2024-04-16T17:04:26.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95341a22b0 con 0x7f9534105350 2024-04-16T17:04:26.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f952aa00640 1 -- 172.21.15.18:0/1551488791 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520003e20 con 0x7f9534105350 2024-04-16T17:04:26.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.288+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341a27a0 con 0x7f9534105350 2024-04-16T17:04:26.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.290+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953410b4d0 con 0x7f9534105350 2024-04-16T17:04:26.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.292+0000 7f952aa00640 1 -- 172.21.15.18:0/1551488791 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9520044070 con 0x7f9534105350 2024-04-16T17:04:26.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.293+0000 7f952aa00640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95180a3ba0 0x7f95180a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.293+0000 7f952aa00640 1 -- 172.21.15.18:0/1551488791 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f95200f2380 con 0x7f9534105350 2024-04-16T17:04:26.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.293+0000 7f9531e00640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95180a3ba0 0x7f95180a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.294+0000 7f9531e00640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95180a3ba0 0x7f95180a6060 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f95341051b0 tx=0x7f9524005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.297+0000 7f952aa00640 1 -- 172.21.15.18:0/1551488791 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95200be2e0 con 0x7f9534105350 2024-04-16T17:04:26.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.511+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9534062c00 con 0x7f9534105350 2024-04-16T17:04:26.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.513+0000 7f952aa00640 1 -- 172.21.15.18:0/1551488791 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24748 (secure 0 0 0) 0x7f95200c25e0 con 0x7f9534105350 2024-04-16T17:04:26.516 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:26.516 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":93,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:03:27.035238+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95180a3ba0 msgr2=0x7f95180a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95180a3ba0 0x7f95180a6060 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f95341051b0 tx=0x7f9524005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9534105350 msgr2=0x7f953419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9534105350 0x7f953419a940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9520009230 tx=0x7f952002c650 comp rx=0 tx=0).stop 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 shutdown_connections 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95180a3ba0 0x7f95180a6060 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9534105d10 0x7f95341a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9534105350 0x7f953419a940 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 --2- 172.21.15.18:0/1551488791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9534104160 0x7f953419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 >> 172.21.15.18:0/1551488791 conn(0x7f95340ff6d0 msgr2=0x7f95341086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:26.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 shutdown_connections 2024-04-16T17:04:26.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.516+0000 7f9533200640 1 -- 172.21.15.18:0/1551488791 wait complete. 2024-04-16T17:04:26.540 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:04:26.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:04:26.629 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:26 smithi018 ceph-mon[31776]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:26.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:26 smithi022 ceph-mon[36023]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:26.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:26 smithi043 ceph-mon[36901]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 -- 172.21.15.18:0/3103684638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630108460 msgr2=0x7f8630108880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 --2- 172.21.15.18:0/3103684638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630108460 0x7f8630108880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8624009900 tx=0x7f862402bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 -- 172.21.15.18:0/3103684638 shutdown_connections 2024-04-16T17:04:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 --2- 172.21.15.18:0/3103684638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f863010a010 0x7f86301108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 --2- 172.21.15.18:0/3103684638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630109650 0x7f8630109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 --2- 172.21.15.18:0/3103684638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630108460 0x7f8630108880 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 -- 172.21.15.18:0/3103684638 >> 172.21.15.18:0/3103684638 conn(0x7f8630077440 msgr2=0x7f8630079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 -- 172.21.15.18:0/3103684638 shutdown_connections 2024-04-16T17:04:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.775+0000 7f8637200640 1 -- 172.21.15.18:0/3103684638 wait complete. 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.776+0000 7f8637200640 1 Processor -- start 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.776+0000 7f8637200640 1 -- start start 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8637200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630108460 0x7f863019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8637200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630109650 0x7f863019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8637200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f863010a010 0x7f86301a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8637200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f86301140c0 con 0x7f8630109650 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8637200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8630113f40 con 0x7f863010a010 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8637200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8630114240 con 0x7f8630108460 2024-04-16T17:04:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8635400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630109650 0x7f863019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8635400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630109650 0x7f863019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33326/0 (socket says 172.21.15.18:33326) 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8635400640 1 -- 172.21.15.18:0/2321367362 learned_addr learned my addr 172.21.15.18:0/2321367362 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8636800640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f863010a010 0x7f86301a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.777+0000 7f8635e00640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630108460 0x7f863019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8635400640 1 -- 172.21.15.18:0/2321367362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630108460 msgr2=0x7f863019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8635400640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630108460 0x7f863019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8635400640 1 -- 172.21.15.18:0/2321367362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f863010a010 msgr2=0x7f86301a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8635400640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f863010a010 0x7f86301a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8635400640 1 -- 172.21.15.18:0/2321367362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86301a6380 con 0x7f8630109650 2024-04-16T17:04:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8635e00640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630108460 0x7f863019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8635400640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630109650 0x7f863019ecf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f862000e9f0 tx=0x7f862000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f862aa00640 1 -- 172.21.15.18:0/2321367362 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862000cd80 con 0x7f8630109650 2024-04-16T17:04:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f862aa00640 1 -- 172.21.15.18:0/2321367362 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8620010910 con 0x7f8630109650 2024-04-16T17:04:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86301a6670 con 0x7f8630109650 2024-04-16T17:04:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f862aa00640 1 -- 172.21.15.18:0/2321367362 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620018990 con 0x7f8630109650 2024-04-16T17:04:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.778+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301a6bc0 con 0x7f8630109650 2024-04-16T17:04:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.780+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863010f780 con 0x7f8630109650 2024-04-16T17:04:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.783+0000 7f862aa00640 1 -- 172.21.15.18:0/2321367362 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8620010ac0 con 0x7f8630109650 2024-04-16T17:04:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.784+0000 7f862aa00640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f86180a3ba0 0x7f86180a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.784+0000 7f862aa00640 1 -- 172.21.15.18:0/2321367362 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f86200d0b60 con 0x7f8630109650 2024-04-16T17:04:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.785+0000 7f8635e00640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f86180a3ba0 0x7f86180a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.785+0000 7f8635e00640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f86180a3ba0 0x7f86180a6060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8624000c00 tx=0x7f8624005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:26.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.789+0000 7f862aa00640 1 -- 172.21.15.18:0/2321367362 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f862009cac0 con 0x7f8630109650 2024-04-16T17:04:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.993+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f863002d620 con 0x7f8630109650 2024-04-16T17:04:26.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.994+0000 7f862aa00640 1 -- 172.21.15.18:0/2321367362 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f86200a0dc0 con 0x7f8630109650 2024-04-16T17:04:26.996 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:04:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f86180a3ba0 msgr2=0x7f86180a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f86180a3ba0 0x7f86180a6060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8624000c00 tx=0x7f8624005fd0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630109650 msgr2=0x7f863019ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630109650 0x7f863019ecf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f862000e9f0 tx=0x7f862000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 shutdown_connections 2024-04-16T17:04:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f863010a010 0x7f86301a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f86180a3ba0 0x7f86180a6060 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8630109650 0x7f863019ecf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 --2- 172.21.15.18:0/2321367362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630108460 0x7f863019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.997+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 >> 172.21.15.18:0/2321367362 conn(0x7f8630077440 msgr2=0x7f863010c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:27.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.998+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 shutdown_connections 2024-04-16T17:04:27.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:26.998+0000 7f8637200640 1 -- 172.21.15.18:0/2321367362 wait complete. 2024-04-16T17:04:27.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:04:27.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.244+0000 7ff849a00640 1 -- 172.21.15.18:0/4183286564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844107bf0 msgr2=0x7ff844109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.244+0000 7ff849a00640 1 --2- 172.21.15.18:0/4183286564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844107bf0 0x7ff844109fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009be0 tx=0x7ff83c02bef0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.245+0000 7ff849a00640 1 -- 172.21.15.18:0/4183286564 shutdown_connections 2024-04-16T17:04:27.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.245+0000 7ff849a00640 1 --2- 172.21.15.18:0/4183286564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844107bf0 0x7ff844109fe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.245+0000 7ff849a00640 1 --2- 172.21.15.18:0/4183286564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8441052c0 0x7ff8441076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.245+0000 7ff849a00640 1 --2- 172.21.15.18:0/4183286564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff844102990 0x7ff844104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.245+0000 7ff849a00640 1 -- 172.21.15.18:0/4183286564 >> 172.21.15.18:0/4183286564 conn(0x7ff8440fc2e0 msgr2=0x7ff8440fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:27.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.245+0000 7ff849a00640 1 -- 172.21.15.18:0/4183286564 shutdown_connections 2024-04-16T17:04:27.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.245+0000 7ff849a00640 1 -- 172.21.15.18:0/4183286564 wait complete. 2024-04-16T17:04:27.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.246+0000 7ff849a00640 1 Processor -- start 2024-04-16T17:04:27.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.246+0000 7ff849a00640 1 -- start start 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff849a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844102990 0x7ff84419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff849a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8441052c0 0x7ff84419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff849a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff844107bf0 0x7ff8441a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff849a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff84410db10 con 0x7ff8441052c0 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff849a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff84410d990 con 0x7ff844107bf0 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff849a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff84410dc90 con 0x7ff844102990 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff843e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844102990 0x7ff84419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff843e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844102990 0x7ff84419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39684/0 (socket says 172.21.15.18:39684) 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff843400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8441052c0 0x7ff84419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff843e00640 1 -- 172.21.15.18:0/3776473528 learned_addr learned my addr 172.21.15.18:0/3776473528 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff849000640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff844107bf0 0x7ff8441a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.247+0000 7ff843e00640 1 -- 172.21.15.18:0/3776473528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff844107bf0 msgr2=0x7ff8441a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff843e00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff844107bf0 0x7ff8441a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff843e00640 1 -- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8441052c0 msgr2=0x7ff84419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff843e00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8441052c0 0x7ff84419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff843e00640 1 -- 172.21.15.18:0/3776473528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8441a1f60 con 0x7ff844102990 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff843e00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844102990 0x7ff84419a340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff83400e9c0 tx=0x7ff83400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff840c00640 1 -- 172.21.15.18:0/3776473528 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83400ccc0 con 0x7ff844102990 2024-04-16T17:04:27.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff840c00640 1 -- 172.21.15.18:0/3776473528 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff834005500 con 0x7ff844102990 2024-04-16T17:04:27.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8441a2250 con 0x7ff844102990 2024-04-16T17:04:27.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff840c00640 1 -- 172.21.15.18:0/3776473528 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834010430 con 0x7ff844102990 2024-04-16T17:04:27.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.248+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8441a27f0 con 0x7ff844102990 2024-04-16T17:04:27.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.250+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8441096e0 con 0x7ff844102990 2024-04-16T17:04:27.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.252+0000 7ff840c00640 1 -- 172.21.15.18:0/3776473528 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff8340046c0 con 0x7ff844102990 2024-04-16T17:04:27.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.253+0000 7ff840c00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff8280a3c30 0x7ff8280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:27.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.253+0000 7ff840c00640 1 -- 172.21.15.18:0/3776473528 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ff8340d0a70 con 0x7ff844102990 2024-04-16T17:04:27.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.253+0000 7ff843400640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff8280a3c30 0x7ff8280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:27.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.254+0000 7ff843400640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff8280a3c30 0x7ff8280a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff84419b850 tx=0x7ff830005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:27.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.257+0000 7ff840c00640 1 -- 172.21.15.18:0/3776473528 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff83409d020 con 0x7ff844102990 2024-04-16T17:04:27.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.464+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff844062c00 con 0x7ff844102990 2024-04-16T17:04:27.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.465+0000 7ff840c00640 1 -- 172.21.15.18:0/3776473528 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7ff83409c920 con 0x7ff844102990 2024-04-16T17:04:27.467 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.467+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff8280a3c30 msgr2=0x7ff8280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.467+0000 7ff849a00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff8280a3c30 0x7ff8280a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff84419b850 tx=0x7ff830005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.467+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844102990 msgr2=0x7ff84419a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.467+0000 7ff849a00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844102990 0x7ff84419a340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff83400e9c0 tx=0x7ff83400ee80 comp rx=0 tx=0).stop 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.468+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 shutdown_connections 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.468+0000 7ff849a00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff844107bf0 0x7ff8441a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.468+0000 7ff849a00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ff8280a3c30 0x7ff8280a60f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.468+0000 7ff849a00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8441052c0 0x7ff84419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.468+0000 7ff849a00640 1 --2- 172.21.15.18:0/3776473528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff844102990 0x7ff84419a340 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.468+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 >> 172.21.15.18:0/3776473528 conn(0x7ff8440fc2e0 msgr2=0x7ff844102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:27.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.468+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 shutdown_connections 2024-04-16T17:04:27.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.468+0000 7ff849a00640 1 -- 172.21.15.18:0/3776473528 wait complete. 2024-04-16T17:04:27.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:04:27.673 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:27 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1551488791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:27.673 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:27 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2321367362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:27.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:27 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1551488791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:27.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:27 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2321367362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:27.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 -- 172.21.15.18:0/3848797983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66c0101e90 msgr2=0x7f66c01022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 --2- 172.21.15.18:0/3848797983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66c0101e90 0x7f66c01022b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009900 tx=0x7f66b402bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:27.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 -- 172.21.15.18:0/3848797983 shutdown_connections 2024-04-16T17:04:27.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 --2- 172.21.15.18:0/3848797983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c01039a0 0x7f66c010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 --2- 172.21.15.18:0/3848797983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c0102fe0 0x7f66c0103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 --2- 172.21.15.18:0/3848797983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66c0101e90 0x7f66c01022b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 -- 172.21.15.18:0/3848797983 >> 172.21.15.18:0/3848797983 conn(0x7f66c00fd400 msgr2=0x7f66c00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:27.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 -- 172.21.15.18:0/3848797983 shutdown_connections 2024-04-16T17:04:27.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.683+0000 7f66c6600640 1 -- 172.21.15.18:0/3848797983 wait complete. 2024-04-16T17:04:27.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.684+0000 7f66c6600640 1 Processor -- start 2024-04-16T17:04:27.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.684+0000 7f66c6600640 1 -- start start 2024-04-16T17:04:27.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c0101e90 0x7f66c019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:27.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c6600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c0102fe0 0x7f66c019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:27.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c6600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66c01039a0 0x7f66c01a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:27.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c6600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010db60 con 0x7f66c01039a0 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c6600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010d9e0 con 0x7f66c0102fe0 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c6600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010dce0 con 0x7f66c0101e90 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66c01039a0 0x7f66c01a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c0101e90 0x7f66c019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c0101e90 0x7f66c019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39698/0 (socket says 172.21.15.18:39698) 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5200640 1 -- 172.21.15.18:0/1592344211 learned_addr learned my addr 172.21.15.18:0/1592344211 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66bfe00640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c0102fe0 0x7f66c019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5200640 1 -- 172.21.15.18:0/1592344211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c0102fe0 msgr2=0x7f66c019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5200640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c0102fe0 0x7f66c019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5200640 1 -- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66c01039a0 msgr2=0x7f66c01a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5200640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66c01039a0 0x7f66c01a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.685+0000 7f66c5200640 1 -- 172.21.15.18:0/1592344211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c01a20a0 con 0x7f66c0101e90 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.686+0000 7f66c5200640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c0101e90 0x7f66c019a480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66b402c8e0 tx=0x7f66b4005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.686+0000 7f66bd600640 1 -- 172.21.15.18:0/1592344211 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4043020 con 0x7f66c0101e90 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.686+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c01a2330 con 0x7f66c0101e90 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.686+0000 7f66bd600640 1 -- 172.21.15.18:0/1592344211 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b40054e0 con 0x7f66c0101e90 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.686+0000 7f66bd600640 1 -- 172.21.15.18:0/1592344211 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4003d40 con 0x7f66c0101e90 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.686+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c01a28d0 con 0x7f66c0101e90 2024-04-16T17:04:27.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.687+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c01a2bf0 con 0x7f66c0101e90 2024-04-16T17:04:27.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.690+0000 7f66bd600640 1 -- 172.21.15.18:0/1592344211 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f66b4046020 con 0x7f66c0101e90 2024-04-16T17:04:27.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.691+0000 7f66bd600640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f66a80a3bb0 0x7f66a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:27.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.691+0000 7f66bd600640 1 -- 172.21.15.18:0/1592344211 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f66b40f2980 con 0x7f66c0101e90 2024-04-16T17:04:27.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.691+0000 7f66bfe00640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f66a80a3bb0 0x7f66a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:27.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.692+0000 7f66bfe00640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f66a80a3bb0 0x7f66a80a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0095f0 tx=0x7f66ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:27.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.695+0000 7f66bd600640 1 -- 172.21.15.18:0/1592344211 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66b404d020 con 0x7f66c0101e90 2024-04-16T17:04:27.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:27 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1551488791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:27.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:27 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2321367362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:27.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.893+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66c01a2dc0 con 0x7f66c0101e90 2024-04-16T17:04:27.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.894+0000 7f66bd600640 1 -- 172.21.15.18:0/1592344211 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f66b40bef30 con 0x7f66c0101e90 2024-04-16T17:04:27.895 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:04:27.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f66a80a3bb0 msgr2=0x7f66a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f66a80a3bb0 0x7f66a80a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0095f0 tx=0x7f66ac005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:27.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c0101e90 msgr2=0x7f66c019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:27.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c0101e90 0x7f66c019a480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66b402c8e0 tx=0x7f66b4005d50 comp rx=0 tx=0).stop 2024-04-16T17:04:27.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 shutdown_connections 2024-04-16T17:04:27.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f66a80a3bb0 0x7f66a80a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66c01039a0 0x7f66c01a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66c0102fe0 0x7f66c019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 --2- 172.21.15.18:0/1592344211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c0101e90 0x7f66c019a480 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:27.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 >> 172.21.15.18:0/1592344211 conn(0x7f66c00fd400 msgr2=0x7f66c0074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:27.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 shutdown_connections 2024-04-16T17:04:27.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:27.896+0000 7f66c6600640 1 -- 172.21.15.18:0/1592344211 wait complete. 2024-04-16T17:04:27.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:04:28.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.116+0000 7f491b200640 1 -- 172.21.15.18:0/2948039024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c107b50 msgr2=0x7f491c109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.116+0000 7f491b200640 1 --2- 172.21.15.18:0/2948039024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c107b50 0x7f491c109fa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4914009ca0 tx=0x7f491402c100 comp rx=0 tx=0).stop 2024-04-16T17:04:28.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.117+0000 7f491b200640 1 -- 172.21.15.18:0/2948039024 shutdown_connections 2024-04-16T17:04:28.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.117+0000 7f491b200640 1 --2- 172.21.15.18:0/2948039024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c107b50 0x7f491c109fa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.117+0000 7f491b200640 1 --2- 172.21.15.18:0/2948039024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f491c105220 0x7f491c107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.117+0000 7f491b200640 1 --2- 172.21.15.18:0/2948039024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f491c1028f0 0x7f491c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.117+0000 7f491b200640 1 -- 172.21.15.18:0/2948039024 >> 172.21.15.18:0/2948039024 conn(0x7f491c0fc2e0 msgr2=0x7f491c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:28.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.117+0000 7f491b200640 1 -- 172.21.15.18:0/2948039024 shutdown_connections 2024-04-16T17:04:28.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.117+0000 7f491b200640 1 -- 172.21.15.18:0/2948039024 wait complete. 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.118+0000 7f491b200640 1 Processor -- start 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.118+0000 7f491b200640 1 -- start start 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f491b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c1028f0 0x7f491c102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f491b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f491c105220 0x7f491c1008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f491b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f491c107b50 0x7f491c100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f491b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f491c10db10 con 0x7f491c1028f0 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f491b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f491c10d990 con 0x7f491c107b50 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f491b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f491c10dc90 con 0x7f491c105220 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f4919e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c1028f0 0x7f491c102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:28.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f4919e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c1028f0 0x7f491c102260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40748/0 (socket says 172.21.15.18:40748) 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f4913e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f491c105220 0x7f491c1008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f4919e00640 1 -- 172.21.15.18:0/865745604 learned_addr learned my addr 172.21.15.18:0/865745604 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f491a800640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f491c107b50 0x7f491c100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f4913e00640 1 -- 172.21.15.18:0/865745604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f491c107b50 msgr2=0x7f491c100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f4913e00640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f491c107b50 0x7f491c100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f4913e00640 1 -- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c1028f0 msgr2=0x7f491c102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.119+0000 7f4913e00640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c1028f0 0x7f491c102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.120+0000 7f4913e00640 1 -- 172.21.15.18:0/865745604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c101670 con 0x7f491c105220 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.120+0000 7f4913e00640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f491c105220 0x7f491c1008a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f490400e9c0 tx=0x7f490400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.120+0000 7f4912a00640 1 -- 172.21.15.18:0/865745604 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490400ccc0 con 0x7f491c105220 2024-04-16T17:04:28.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.120+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c1a2220 con 0x7f491c105220 2024-04-16T17:04:28.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.120+0000 7f4912a00640 1 -- 172.21.15.18:0/865745604 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4904005020 con 0x7f491c105220 2024-04-16T17:04:28.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.120+0000 7f4912a00640 1 -- 172.21.15.18:0/865745604 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904004a00 con 0x7f491c105220 2024-04-16T17:04:28.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.120+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c1a2770 con 0x7f491c105220 2024-04-16T17:04:28.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.121+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491c0fddd0 con 0x7f491c105220 2024-04-16T17:04:28.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.123+0000 7f4912a00640 1 -- 172.21.15.18:0/865745604 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f49040051d0 con 0x7f491c105220 2024-04-16T17:04:28.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.125+0000 7f4912a00640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49000a3bb0 0x7f49000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.125+0000 7f4912a00640 1 -- 172.21.15.18:0/865745604 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f49040d07c0 con 0x7f491c105220 2024-04-16T17:04:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.125+0000 7f4919e00640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49000a3bb0 0x7f49000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.126+0000 7f4919e00640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49000a3bb0 0x7f49000a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f490c0095f0 tx=0x7f490c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:28.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.129+0000 7f4912a00640 1 -- 172.21.15.18:0/865745604 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f490409cd70 con 0x7f491c105220 2024-04-16T17:04:28.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.336+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f491c062c00 con 0x7f491c105220 2024-04-16T17:04:28.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.337+0000 7f4912a00640 1 -- 172.21.15.18:0/865745604 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f490409c670 con 0x7f491c105220 2024-04-16T17:04:28.338 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49000a3bb0 msgr2=0x7f49000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49000a3bb0 0x7f49000a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f490c0095f0 tx=0x7f490c005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f491c105220 msgr2=0x7f491c1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f491c105220 0x7f491c1008a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f490400e9c0 tx=0x7f490400ee80 comp rx=0 tx=0).stop 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 shutdown_connections 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f491c107b50 0x7f491c100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f491c105220 0x7f491c1008a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f49000a3bb0 0x7f49000a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 --2- 172.21.15.18:0/865745604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f491c1028f0 0x7f491c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 >> 172.21.15.18:0/865745604 conn(0x7f491c0fc2e0 msgr2=0x7f491c103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 shutdown_connections 2024-04-16T17:04:28.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.340+0000 7f491b200640 1 -- 172.21.15.18:0/865745604 wait complete. 2024-04-16T17:04:28.360 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:04:28.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.571+0000 7f0789a00640 1 -- 172.21.15.18:0/2079203169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784101f90 msgr2=0x7f0784102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.571+0000 7f0789a00640 1 --2- 172.21.15.18:0/2079203169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784101f90 0x7f0784102410 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0770009900 tx=0x7f077002bc40 comp rx=0 tx=0).stop 2024-04-16T17:04:28.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.571+0000 7f0789a00640 1 -- 172.21.15.18:0/2079203169 shutdown_connections 2024-04-16T17:04:28.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.571+0000 7f0789a00640 1 --2- 172.21.15.18:0/2079203169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784102950 0x7f0784109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.571+0000 7f0789a00640 1 --2- 172.21.15.18:0/2079203169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784101f90 0x7f0784102410 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.571+0000 7f0789a00640 1 --2- 172.21.15.18:0/2079203169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0784100e40 0x7f0784101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.571+0000 7f0789a00640 1 -- 172.21.15.18:0/2079203169 >> 172.21.15.18:0/2079203169 conn(0x7f07840fc3d0 msgr2=0x7f07840fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:28.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.572+0000 7f0789a00640 1 -- 172.21.15.18:0/2079203169 shutdown_connections 2024-04-16T17:04:28.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.572+0000 7f0789a00640 1 -- 172.21.15.18:0/2079203169 wait complete. 2024-04-16T17:04:28.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789a00640 1 Processor -- start 2024-04-16T17:04:28.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789a00640 1 -- start start 2024-04-16T17:04:28.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784100e40 0x7f078419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:28.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0784101f90 0x7f078419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:28.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784102950 0x7f07841a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f078410cb30 con 0x7f0784100e40 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f078410c9b0 con 0x7f0784102950 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f078410ccb0 con 0x7f0784101f90 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0783e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784100e40 0x7f078419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0783400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0784101f90 0x7f078419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0783e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784100e40 0x7f078419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40776/0 (socket says 172.21.15.18:40776) 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0783e00640 1 -- 172.21.15.18:0/3436097453 learned_addr learned my addr 172.21.15.18:0/3436097453 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.573+0000 7f0789000640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784102950 0x7f07841a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.574+0000 7f0783400640 1 -- 172.21.15.18:0/3436097453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784102950 msgr2=0x7f07841a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.574+0000 7f0783400640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784102950 0x7f07841a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.574+0000 7f0783400640 1 -- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784100e40 msgr2=0x7f078419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.574+0000 7f0783400640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784100e40 0x7f078419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.574+0000 7f0783400640 1 -- 172.21.15.18:0/3436097453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07841a20e0 con 0x7f0784101f90 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.574+0000 7f0789000640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784102950 0x7f07841a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:28.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.574+0000 7f0783400640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0784101f90 0x7f078419aa00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0770009230 tx=0x7f077002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:28.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.575+0000 7f0780c00640 1 -- 172.21.15.18:0/3436097453 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770041070 con 0x7f0784101f90 2024-04-16T17:04:28.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.575+0000 7f0780c00640 1 -- 172.21.15.18:0/3436097453 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07700054e0 con 0x7f0784101f90 2024-04-16T17:04:28.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.575+0000 7f0780c00640 1 -- 172.21.15.18:0/3436097453 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770003e40 con 0x7f0784101f90 2024-04-16T17:04:28.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.575+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07841a2370 con 0x7f0784101f90 2024-04-16T17:04:28.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.575+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07841a27e0 con 0x7f0784101f90 2024-04-16T17:04:28.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.576+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07841a2b00 con 0x7f0784101f90 2024-04-16T17:04:28.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.578+0000 7f0780c00640 1 -- 172.21.15.18:0/3436097453 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0770044070 con 0x7f0784101f90 2024-04-16T17:04:28.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.580+0000 7f0780c00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07680a3c20 0x7f07680a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:28.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.580+0000 7f0780c00640 1 -- 172.21.15.18:0/3436097453 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f07700f2160 con 0x7f0784101f90 2024-04-16T17:04:28.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.580+0000 7f0783e00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07680a3c20 0x7f07680a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:28.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.581+0000 7f0783e00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07680a3c20 0x7f07680a60e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0774009cd0 tx=0x7f0774005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:28.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.584+0000 7f0780c00640 1 -- 172.21.15.18:0/3436097453 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07700be0c0 con 0x7f0784101f90 2024-04-16T17:04:28.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:28 smithi018 ceph-mon[31776]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:28.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:28 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3776473528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:28 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1592344211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:28 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/865745604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:28 smithi043 ceph-mon[36901]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:28.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:28 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3776473528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:28 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1592344211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:28 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/865745604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:28 smithi022 ceph-mon[36023]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-16T17:04:28.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:28 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3776473528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:28 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1592344211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:28 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/865745604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:28.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.823+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07841a3120 con 0x7f0784101f90 2024-04-16T17:04:28.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.825+0000 7f0780c00640 1 -- 172.21.15.18:0/3436097453 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11848 (secure 0 0 0) 0x7f07700c23c0 con 0x7f0784101f90 2024-04-16T17:04:28.827 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:28.827 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:01:59.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:04:28.827 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 24 2024-04-16T17:04:28.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.827+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07680a3c20 msgr2=0x7f07680a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.827+0000 7f0789a00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07680a3c20 0x7f07680a60e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0774009cd0 tx=0x7f0774005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:28.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.827+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0784101f90 msgr2=0x7f078419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:28.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.827+0000 7f0789a00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0784101f90 0x7f078419aa00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0770009230 tx=0x7f077002c660 comp rx=0 tx=0).stop 2024-04-16T17:04:28.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.828+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 shutdown_connections 2024-04-16T17:04:28.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.828+0000 7f0789a00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784102950 0x7f07841a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.828+0000 7f0789a00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0784101f90 0x7f078419aa00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.828+0000 7f0789a00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07680a3c20 0x7f07680a60e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.828+0000 7f0789a00640 1 --2- 172.21.15.18:0/3436097453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0784100e40 0x7f078419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:28.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.828+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 >> 172.21.15.18:0/3436097453 conn(0x7f07840fc3d0 msgr2=0x7f07840fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:28.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.828+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 shutdown_connections 2024-04-16T17:04:28.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:28.828+0000 7f0789a00640 1 -- 172.21.15.18:0/3436097453 wait complete. 2024-04-16T17:04:28.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:04:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 -- 172.21.15.18:0/2376746562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c109740 msgr2=0x7fd40c109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 --2- 172.21.15.18:0/2376746562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c109740 0x7fd40c109bc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009900 tx=0x7fd3f802bc50 comp rx=0 tx=0).stop 2024-04-16T17:04:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 -- 172.21.15.18:0/2376746562 shutdown_connections 2024-04-16T17:04:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 --2- 172.21.15.18:0/2376746562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c10a100 0x7fd40c110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 --2- 172.21.15.18:0/2376746562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c109740 0x7fd40c109bc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 --2- 172.21.15.18:0/2376746562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd40c1085a0 0x7fd40c1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 -- 172.21.15.18:0/2376746562 >> 172.21.15.18:0/2376746562 conn(0x7fd40c0774f0 msgr2=0x7fd40c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 -- 172.21.15.18:0/2376746562 shutdown_connections 2024-04-16T17:04:29.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.069+0000 7fd410c00640 1 -- 172.21.15.18:0/2376746562 wait complete. 2024-04-16T17:04:29.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.070+0000 7fd410c00640 1 Processor -- start 2024-04-16T17:04:29.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.071+0000 7fd410c00640 1 -- start start 2024-04-16T17:04:29.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.071+0000 7fd410c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c1085a0 0x7fd40c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.071+0000 7fd410c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd40c109740 0x7fd40c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.071+0000 7fd410c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c10a100 0x7fd40c1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.071+0000 7fd410c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1141e0 con 0x7fd40c109740 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.071+0000 7fd410c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c114060 con 0x7fd40c1085a0 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.071+0000 7fd410c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c114360 con 0x7fd40c10a100 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.071+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd40c109740 0x7fd40c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd40c109740 0x7fd40c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40796/0 (socket says 172.21.15.18:40796) 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40aa00640 1 -- 172.21.15.18:0/1050402006 learned_addr learned my addr 172.21.15.18:0/1050402006 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40b400640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c1085a0 0x7fd40c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40be00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c10a100 0x7fd40c1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40be00640 1 -- 172.21.15.18:0/1050402006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c1085a0 msgr2=0x7fd40c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40be00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c1085a0 0x7fd40c19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40be00640 1 -- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd40c109740 msgr2=0x7fd40c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40be00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd40c109740 0x7fd40c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40be00640 1 -- 172.21.15.18:0/1050402006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c1a6530 con 0x7fd40c10a100 2024-04-16T17:04:29.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40aa00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd40c109740 0x7fd40c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:29.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.072+0000 7fd40be00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c10a100 0x7fd40c1a5ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd40400ec50 tx=0x7fd40400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:29.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.073+0000 7fd403e00640 1 -- 172.21.15.18:0/1050402006 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404003b70 con 0x7fd40c10a100 2024-04-16T17:04:29.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.073+0000 7fd403e00640 1 -- 172.21.15.18:0/1050402006 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd404003d10 con 0x7fd40c10a100 2024-04-16T17:04:29.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.073+0000 7fd403e00640 1 -- 172.21.15.18:0/1050402006 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404010890 con 0x7fd40c10a100 2024-04-16T17:04:29.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.073+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c1a6820 con 0x7fd40c10a100 2024-04-16T17:04:29.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.073+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c1a6cf0 con 0x7fd40c10a100 2024-04-16T17:04:29.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.074+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c10f870 con 0x7fd40c10a100 2024-04-16T17:04:29.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.076+0000 7fd403e00640 1 -- 172.21.15.18:0/1050402006 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd404004b40 con 0x7fd40c10a100 2024-04-16T17:04:29.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.077+0000 7fd403e00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3f00a3c20 0x7fd3f00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:29.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.077+0000 7fd403e00640 1 -- 172.21.15.18:0/1050402006 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd4040d0e20 con 0x7fd40c10a100 2024-04-16T17:04:29.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.077+0000 7fd40b400640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3f00a3c20 0x7fd3f00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:29.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.078+0000 7fd40b400640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3f00a3c20 0x7fd3f00a60e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd40c1095a0 tx=0x7fd3fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:29.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.080+0000 7fd403e00640 1 -- 172.21.15.18:0/1050402006 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd40409d460 con 0x7fd40c10a100 2024-04-16T17:04:29.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.287+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd40c062c00 con 0x7fd40c10a100 2024-04-16T17:04:29.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.289+0000 7fd403e00640 1 -- 172.21.15.18:0/1050402006 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24748 (secure 0 0 0) 0x7fd40409cd60 con 0x7fd40c10a100 2024-04-16T17:04:29.291 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:29.292 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":93,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:03:27.035238+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3f00a3c20 msgr2=0x7fd3f00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3f00a3c20 0x7fd3f00a60e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd40c1095a0 tx=0x7fd3fc005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c10a100 msgr2=0x7fd40c1a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c10a100 0x7fd40c1a5ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd40400ec50 tx=0x7fd40400c6a0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 shutdown_connections 2024-04-16T17:04:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd40c10a100 0x7fd40c1a5ec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fd3f00a3c20 0x7fd3f00a60e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd40c109740 0x7fd40c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 --2- 172.21.15.18:0/1050402006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c1085a0 0x7fd40c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 >> 172.21.15.18:0/1050402006 conn(0x7fd40c0774f0 msgr2=0x7fd40c10ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 shutdown_connections 2024-04-16T17:04:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.292+0000 7fd410c00640 1 -- 172.21.15.18:0/1050402006 wait complete. 2024-04-16T17:04:29.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:04:29.315 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:04:29.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 -- 172.21.15.18:0/2238198020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8105e00 msgr2=0x7f4fc810c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 --2- 172.21.15.18:0/2238198020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8105e00 0x7f4fc810c690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009900 tx=0x7f4fb802bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:29.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 -- 172.21.15.18:0/2238198020 shutdown_connections 2024-04-16T17:04:29.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 --2- 172.21.15.18:0/2238198020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8105e00 0x7f4fc810c690 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 --2- 172.21.15.18:0/2238198020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8105440 0x7f4fc81058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 --2- 172.21.15.18:0/2238198020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc8104250 0x7f4fc8104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 -- 172.21.15.18:0/2238198020 >> 172.21.15.18:0/2238198020 conn(0x7f4fc80ff7c0 msgr2=0x7f4fc8101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:29.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 -- 172.21.15.18:0/2238198020 shutdown_connections 2024-04-16T17:04:29.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.544+0000 7f4fccc00640 1 -- 172.21.15.18:0/2238198020 wait complete. 2024-04-16T17:04:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.545+0000 7f4fccc00640 1 Processor -- start 2024-04-16T17:04:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.545+0000 7f4fccc00640 1 -- start start 2024-04-16T17:04:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.545+0000 7f4fccc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8104250 0x7f4fc819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.545+0000 7f4fccc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc8105440 0x7f4fc819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fccc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8105e00 0x7f4fc81a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fccc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc810ff30 con 0x7f4fc8105e00 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fccc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc810fdb0 con 0x7f4fc8104250 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fccc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc81100b0 con 0x7f4fc8105440 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8104250 0x7f4fc819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8104250 0x7f4fc819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45174/0 (socket says 172.21.15.18:45174) 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5e00640 1 -- 172.21.15.18:0/3231433520 learned_addr learned my addr 172.21.15.18:0/3231433520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5400640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc8105440 0x7f4fc819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc6800640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8105e00 0x7f4fc81a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5e00640 1 -- 172.21.15.18:0/3231433520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc8105440 msgr2=0x7f4fc819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5e00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc8105440 0x7f4fc819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5e00640 1 -- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8105e00 msgr2=0x7f4fc81a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5e00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8105e00 0x7f4fc81a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc5e00640 1 -- 172.21.15.18:0/3231433520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc81a2120 con 0x7f4fc8104250 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.546+0000 7f4fc6800640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8105e00 0x7f4fc81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.547+0000 7f4fc5e00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8104250 0x7f4fc819a500 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000eb70 tx=0x7f4fc000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.547+0000 7f4fbea00640 1 -- 172.21.15.18:0/3231433520 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0003b70 con 0x7f4fc8104250 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.547+0000 7f4fbea00640 1 -- 172.21.15.18:0/3231433520 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc000ee70 con 0x7f4fc8104250 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.547+0000 7f4fbea00640 1 -- 172.21.15.18:0/3231433520 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0010830 con 0x7f4fc8104250 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.547+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc81a2410 con 0x7f4fc8104250 2024-04-16T17:04:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.547+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc81a2960 con 0x7f4fc8104250 2024-04-16T17:04:29.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.548+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc810b4a0 con 0x7f4fc8104250 2024-04-16T17:04:29.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.550+0000 7f4fbea00640 1 -- 172.21.15.18:0/3231433520 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4fc0004020 con 0x7f4fc8104250 2024-04-16T17:04:29.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.552+0000 7f4fbea00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4fac0a3c30 0x7f4fac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:29.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.552+0000 7f4fbea00640 1 -- 172.21.15.18:0/3231433520 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f4fc00d0ee0 con 0x7f4fc8104250 2024-04-16T17:04:29.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.552+0000 7f4fc5400640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4fac0a3c30 0x7f4fac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:29.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.553+0000 7f4fc5400640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4fac0a3c30 0x7f4fac0a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4fc819ba10 tx=0x7f4fb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:29.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.556+0000 7f4fbea00640 1 -- 172.21.15.18:0/3231433520 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fc009d490 con 0x7f4fc8104250 2024-04-16T17:04:29.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:29 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3436097453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:29.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:29 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1050402006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:29 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3436097453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:29 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1050402006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:29.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:29 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3436097453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:29.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:29 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1050402006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:29.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.766+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4fc8062c30 con 0x7f4fc8104250 2024-04-16T17:04:29.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.767+0000 7f4fbea00640 1 -- 172.21.15.18:0/3231433520 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f4fc009cd90 con 0x7f4fc8104250 2024-04-16T17:04:29.768 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4fac0a3c30 msgr2=0x7f4fac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4fac0a3c30 0x7f4fac0a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4fc819ba10 tx=0x7f4fb0005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8104250 msgr2=0x7f4fc819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8104250 0x7f4fc819a500 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000eb70 tx=0x7f4fc000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 shutdown_connections 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f4fac0a3c30 0x7f4fac0a60f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8105e00 0x7f4fc81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc8105440 0x7f4fc819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 --2- 172.21.15.18:0/3231433520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fc8104250 0x7f4fc819a500 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:29.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 >> 172.21.15.18:0/3231433520 conn(0x7f4fc80ff7c0 msgr2=0x7f4fc8108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:29.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.770+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 shutdown_connections 2024-04-16T17:04:29.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:29.771+0000 7f4fccc00640 1 -- 172.21.15.18:0/3231433520 wait complete. 2024-04-16T17:04:29.790 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:04:30.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.016+0000 7f0750a00640 1 -- 172.21.15.18:0/3378386078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 msgr2=0x7f074c078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.016+0000 7f0750a00640 1 --2- 172.21.15.18:0/3378386078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 0x7f074c078060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0738009900 tx=0x7f073802bc40 comp rx=0 tx=0).stop 2024-04-16T17:04:30.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.017+0000 7f0750a00640 1 -- 172.21.15.18:0/3378386078 shutdown_connections 2024-04-16T17:04:30.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.017+0000 7f0750a00640 1 --2- 172.21.15.18:0/3378386078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f074c0785a0 0x7f074c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.017+0000 7f0750a00640 1 --2- 172.21.15.18:0/3378386078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 0x7f074c078060 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.017+0000 7f0750a00640 1 --2- 172.21.15.18:0/3378386078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f074c0769c0 0x7f074c076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.017+0000 7f0750a00640 1 -- 172.21.15.18:0/3378386078 >> 172.21.15.18:0/3378386078 conn(0x7f074c0ff610 msgr2=0x7f074c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:30.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.017+0000 7f0750a00640 1 -- 172.21.15.18:0/3378386078 shutdown_connections 2024-04-16T17:04:30.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.017+0000 7f0750a00640 1 -- 172.21.15.18:0/3378386078 wait complete. 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.018+0000 7f0750a00640 1 Processor -- start 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.018+0000 7f0750a00640 1 -- start start 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f0750a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f074c0769c0 0x7f074c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f0750a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 0x7f074c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f0750a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f074c0785a0 0x7f074c1a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f0750a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f074c114110 con 0x7f074c0769c0 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f0750a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f074c113f90 con 0x7f074c077c00 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f0750a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f074c114290 con 0x7f074c0785a0 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f074aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 0x7f074c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f074b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f074c0769c0 0x7f074c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f074aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 0x7f074c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45184/0 (socket says 172.21.15.18:45184) 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f074aa00640 1 -- 172.21.15.18:0/1406056573 learned_addr learned my addr 172.21.15.18:0/1406056573 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:30.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.019+0000 7f074be00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f074c0785a0 0x7f074c1a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f074aa00640 1 -- 172.21.15.18:0/1406056573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f074c0785a0 msgr2=0x7f074c1a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f074aa00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f074c0785a0 0x7f074c1a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f074aa00640 1 -- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f074c0769c0 msgr2=0x7f074c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f074aa00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f074c0769c0 0x7f074c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f074aa00640 1 -- 172.21.15.18:0/1406056573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c1a63e0 con 0x7f074c077c00 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f074be00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f074c0785a0 0x7f074c1a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f074aa00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 0x7f074c19ed00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0738009230 tx=0x7f073802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f0743e00640 1 -- 172.21.15.18:0/1406056573 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738041070 con 0x7f074c077c00 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f0743e00640 1 -- 172.21.15.18:0/1406056573 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0738004450 con 0x7f074c077c00 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f0743e00640 1 -- 172.21.15.18:0/1406056573 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738003e00 con 0x7f074c077c00 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.020+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074c1a6670 con 0x7f074c077c00 2024-04-16T17:04:30.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.021+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074c1a6bb0 con 0x7f074c077c00 2024-04-16T17:04:30.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.022+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074c1109f0 con 0x7f074c077c00 2024-04-16T17:04:30.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.024+0000 7f0743e00640 1 -- 172.21.15.18:0/1406056573 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0738044070 con 0x7f074c077c00 2024-04-16T17:04:30.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.025+0000 7f0743e00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07300a3c30 0x7f07300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.025+0000 7f0743e00640 1 -- 172.21.15.18:0/1406056573 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f07380f23d0 con 0x7f074c077c00 2024-04-16T17:04:30.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.025+0000 7f074b400640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07300a3c30 0x7f07300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.026+0000 7f074b400640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07300a3c30 0x7f07300a60f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f073c009bd0 tx=0x7f073c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.029+0000 7f0743e00640 1 -- 172.21.15.18:0/1406056573 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07380be980 con 0x7f074c077c00 2024-04-16T17:04:30.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.233+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f074c062c00 con 0x7f074c077c00 2024-04-16T17:04:30.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.234+0000 7f0743e00640 1 -- 172.21.15.18:0/1406056573 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f07380be280 con 0x7f074c077c00 2024-04-16T17:04:30.236 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07300a3c30 msgr2=0x7f07300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07300a3c30 0x7f07300a60f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f073c009bd0 tx=0x7f073c005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 msgr2=0x7f074c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 0x7f074c19ed00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0738009230 tx=0x7f073802c660 comp rx=0 tx=0).stop 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 shutdown_connections 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f074c0785a0 0x7f074c1a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f074c077c00 0x7f074c19ed00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f07300a3c30 0x7f07300a60f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 --2- 172.21.15.18:0/1406056573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f074c0769c0 0x7f074c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.237+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 >> 172.21.15.18:0/1406056573 conn(0x7f074c0ff610 msgr2=0x7f074c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:30.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.238+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 shutdown_connections 2024-04-16T17:04:30.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.238+0000 7f0750a00640 1 -- 172.21.15.18:0/1406056573 wait complete. 2024-04-16T17:04:30.258 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:04:30.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.494+0000 7fadb7a00640 1 -- 172.21.15.18:0/910999146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0076040 msgr2=0x7fadb0074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.494+0000 7fadb7a00640 1 --2- 172.21.15.18:0/910999146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0076040 0x7fadb0074490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fada0009890 tx=0x7fada002bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:30.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.495+0000 7fadb7a00640 1 -- 172.21.15.18:0/910999146 shutdown_connections 2024-04-16T17:04:30.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.495+0000 7fadb7a00640 1 --2- 172.21.15.18:0/910999146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadb010e620 0x7fadb0110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.495+0000 7fadb7a00640 1 --2- 172.21.15.18:0/910999146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb0074a60 0x7fadb0074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.495+0000 7fadb7a00640 1 --2- 172.21.15.18:0/910999146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0076040 0x7fadb0074490 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.495+0000 7fadb7a00640 1 -- 172.21.15.18:0/910999146 >> 172.21.15.18:0/910999146 conn(0x7fadb00fd5e0 msgr2=0x7fadb00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:30.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.495+0000 7fadb7a00640 1 -- 172.21.15.18:0/910999146 shutdown_connections 2024-04-16T17:04:30.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.495+0000 7fadb7a00640 1 -- 172.21.15.18:0/910999146 wait complete. 2024-04-16T17:04:30.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.496+0000 7fadb7a00640 1 Processor -- start 2024-04-16T17:04:30.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.496+0000 7fadb7a00640 1 -- start start 2024-04-16T17:04:30.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb7a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0074a60 0x7fadb019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb7a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadb0076040 0x7fadb019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb7a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb010e620 0x7fadb01a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb7a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01142d0 con 0x7fadb010e620 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb7a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0114150 con 0x7fadb0074a60 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb7a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0114450 con 0x7fadb0076040 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb5a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb010e620 0x7fadb01a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadafe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadb0076040 0x7fadb019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb5a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb010e620 0x7fadb01a5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40852/0 (socket says 172.21.15.18:40852) 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb5a00640 1 -- 172.21.15.18:0/2061737927 learned_addr learned my addr 172.21.15.18:0/2061737927 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadafe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadb0076040 0x7fadb019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39816/0 (socket says 172.21.15.18:39816) 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.497+0000 7fadb5000640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0074a60 0x7fadb019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.498+0000 7fadb5a00640 1 -- 172.21.15.18:0/2061737927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadb0076040 msgr2=0x7fadb019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.498+0000 7fadb5a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadb0076040 0x7fadb019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.498+0000 7fadb5a00640 1 -- 172.21.15.18:0/2061737927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0074a60 msgr2=0x7fadb019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.498+0000 7fadb5a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0074a60 0x7fadb019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.498+0000 7fadb5a00640 1 -- 172.21.15.18:0/2061737927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb01a65f0 con 0x7fadb010e620 2024-04-16T17:04:30.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.498+0000 7fadb5000640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0074a60 0x7fadb019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:30.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.498+0000 7fadb5a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb010e620 0x7fadb01a5f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fada800eb10 tx=0x7fada800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.498+0000 7fadad600640 1 -- 172.21.15.18:0/2061737927 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada800ce20 con 0x7fadb010e620 2024-04-16T17:04:30.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.499+0000 7fadad600640 1 -- 172.21.15.18:0/2061737927 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada8005020 con 0x7fadb010e620 2024-04-16T17:04:30.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.499+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb01a68e0 con 0x7fadb010e620 2024-04-16T17:04:30.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.499+0000 7fadad600640 1 -- 172.21.15.18:0/2061737927 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8010690 con 0x7fadb010e620 2024-04-16T17:04:30.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.499+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb01a6e30 con 0x7fadb010e620 2024-04-16T17:04:30.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.500+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb01047f0 con 0x7fadb010e620 2024-04-16T17:04:30.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.502+0000 7fadad600640 1 -- 172.21.15.18:0/2061737927 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fada8005540 con 0x7fadb010e620 2024-04-16T17:04:30.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.503+0000 7fadad600640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fad940a3bb0 0x7fad940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.503+0000 7fadad600640 1 -- 172.21.15.18:0/2061737927 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fada80d0ee0 con 0x7fadb010e620 2024-04-16T17:04:30.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.504+0000 7fadb5000640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fad940a3bb0 0x7fad940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.504+0000 7fadb5000640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fad940a3bb0 0x7fad940a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fada002c180 tx=0x7fada002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.508+0000 7fadad600640 1 -- 172.21.15.18:0/2061737927 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fada809d490 con 0x7fadb010e620 2024-04-16T17:04:30.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:30 smithi022 ceph-mon[36023]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:30.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:30 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3231433520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:30.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:30 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1406056573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:30.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:30 smithi018 ceph-mon[31776]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:30.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:30 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3231433520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:30.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:30 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1406056573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:30.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.713+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fadb0062c30 con 0x7fadb010e620 2024-04-16T17:04:30.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.714+0000 7fadad600640 1 -- 172.21.15.18:0/2061737927 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fada809cd90 con 0x7fadb010e620 2024-04-16T17:04:30.716 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fad940a3bb0 msgr2=0x7fad940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fad940a3bb0 0x7fad940a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fada002c180 tx=0x7fada002e040 comp rx=0 tx=0).stop 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb010e620 msgr2=0x7fadb01a5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb010e620 0x7fadb01a5f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fada800eb10 tx=0x7fada800ef10 comp rx=0 tx=0).stop 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 shutdown_connections 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fad940a3bb0 0x7fad940a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb010e620 0x7fadb01a5f80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadb0076040 0x7fadb019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 --2- 172.21.15.18:0/2061737927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadb0074a60 0x7fadb019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.717+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 >> 172.21.15.18:0/2061737927 conn(0x7fadb00fd5e0 msgr2=0x7fadb01019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:30.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.718+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 shutdown_connections 2024-04-16T17:04:30.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.718+0000 7fadb7a00640 1 -- 172.21.15.18:0/2061737927 wait complete. 2024-04-16T17:04:30.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:04:30.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:30 smithi043 ceph-mon[36901]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-16T17:04:30.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:30 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3231433520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:30.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:30 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1406056573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:30.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 -- 172.21.15.18:0/620162172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc108480 msgr2=0x7fc2bc1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/620162172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc108480 0x7fc2bc1088a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009860 tx=0x7fc2ac02bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 -- 172.21.15.18:0/620162172 shutdown_connections 2024-04-16T17:04:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/620162172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2bc10a030 0x7fc2bc1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/620162172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2bc109670 0x7fc2bc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/620162172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc108480 0x7fc2bc1088a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 -- 172.21.15.18:0/620162172 >> 172.21.15.18:0/620162172 conn(0x7fc2bc077440 msgr2=0x7fc2bc079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 -- 172.21.15.18:0/620162172 shutdown_connections 2024-04-16T17:04:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.971+0000 7fc2bbe00640 1 -- 172.21.15.18:0/620162172 wait complete. 2024-04-16T17:04:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2bbe00640 1 Processor -- start 2024-04-16T17:04:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2bbe00640 1 -- start start 2024-04-16T17:04:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2bc108480 0x7fc2bc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc109670 0x7fc2bc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2bc10a030 0x7fc2bc1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2bbe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1140e0 con 0x7fc2bc10a030 2024-04-16T17:04:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2bbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc113f60 con 0x7fc2bc109670 2024-04-16T17:04:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2bbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc114260 con 0x7fc2bc108480 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.972+0000 7fc2baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2bc108480 0x7fc2bc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2bc108480 0x7fc2bc19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39826/0 (socket says 172.21.15.18:39826) 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2baa00640 1 -- 172.21.15.18:0/3216918392 learned_addr learned my addr 172.21.15.18:0/3216918392 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2ba000640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc109670 0x7fc2bc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2bb400640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2bc10a030 0x7fc2bc1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2baa00640 1 -- 172.21.15.18:0/3216918392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc109670 msgr2=0x7fc2bc19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2baa00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc109670 0x7fc2bc19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2baa00640 1 -- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2bc10a030 msgr2=0x7fc2bc1a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2baa00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2bc10a030 0x7fc2bc1a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2baa00640 1 -- 172.21.15.18:0/3216918392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc1a6420 con 0x7fc2bc108480 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2bb400640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2bc10a030 0x7fc2bc1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.973+0000 7fc2baa00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2bc108480 0x7fc2bc19e800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0349e0 tx=0x7fc2ac0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.974+0000 7fc2b3400640 1 -- 172.21.15.18:0/3216918392 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac041020 con 0x7fc2bc108480 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.974+0000 7fc2b3400640 1 -- 172.21.15.18:0/3216918392 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac0323b0 con 0x7fc2bc108480 2024-04-16T17:04:30.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.974+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc1a66b0 con 0x7fc2bc108480 2024-04-16T17:04:30.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.974+0000 7fc2b3400640 1 -- 172.21.15.18:0/3216918392 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac039040 con 0x7fc2bc108480 2024-04-16T17:04:30.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.974+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc1a6ba0 con 0x7fc2bc108480 2024-04-16T17:04:30.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.975+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2bc10f840 con 0x7fc2bc108480 2024-04-16T17:04:30.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.977+0000 7fc2b3400640 1 -- 172.21.15.18:0/3216918392 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc2ac044020 con 0x7fc2bc108480 2024-04-16T17:04:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.978+0000 7fc2b3400640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.978+0000 7fc2b3400640 1 -- 172.21.15.18:0/3216918392 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fc2ac0f25a0 con 0x7fc2bc108480 2024-04-16T17:04:30.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.978+0000 7fc2ba000640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.979+0000 7fc2ba000640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc19fd10 tx=0x7fc2a800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:30.981+0000 7fc2b3400640 1 -- 172.21.15.18:0/3216918392 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2ac04b020 con 0x7fc2bc108480 2024-04-16T17:04:31.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.191+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc2bc062c00 con 0x7fc2bc108480 2024-04-16T17:04:31.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.192+0000 7fc2b3400640 1 -- 172.21.15.18:0/3216918392 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fc2ac0beb50 con 0x7fc2bc108480 2024-04-16T17:04:31.193 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:31.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc2a00a3bb0 msgr2=0x7fc2a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc19fd10 tx=0x7fc2a800b390 comp rx=0 tx=0).stop 2024-04-16T17:04:31.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2bc108480 msgr2=0x7fc2bc19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2bc108480 0x7fc2bc19e800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0349e0 tx=0x7fc2ac0042f0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 shutdown_connections 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2bc10a030 0x7fc2bc1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc109670 0x7fc2bc19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 --2- 172.21.15.18:0/3216918392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2bc108480 0x7fc2bc19e800 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 >> 172.21.15.18:0/3216918392 conn(0x7fc2bc077440 msgr2=0x7fc2bc10ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 shutdown_connections 2024-04-16T17:04:31.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.195+0000 7fc2bbe00640 1 -- 172.21.15.18:0/3216918392 wait complete. 2024-04-16T17:04:31.216 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:04:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.432+0000 7fc8a4800640 1 -- 172.21.15.18:0/26416116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc89c109870 msgr2=0x7fc89c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.432+0000 7fc8a4800640 1 --2- 172.21.15.18:0/26416116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc89c109870 0x7fc89c109cf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009900 tx=0x7fc88c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.432+0000 7fc8a4800640 1 -- 172.21.15.18:0/26416116 shutdown_connections 2024-04-16T17:04:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.432+0000 7fc8a4800640 1 --2- 172.21.15.18:0/26416116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc89c10a230 0x7fc89c110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.432+0000 7fc8a4800640 1 --2- 172.21.15.18:0/26416116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc89c109870 0x7fc89c109cf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.432+0000 7fc8a4800640 1 --2- 172.21.15.18:0/26416116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc89c108680 0x7fc89c108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.432+0000 7fc8a4800640 1 -- 172.21.15.18:0/26416116 >> 172.21.15.18:0/26416116 conn(0x7fc89c077650 msgr2=0x7fc89c079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.433+0000 7fc8a4800640 1 -- 172.21.15.18:0/26416116 shutdown_connections 2024-04-16T17:04:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.433+0000 7fc8a4800640 1 -- 172.21.15.18:0/26416116 wait complete. 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.433+0000 7fc8a4800640 1 Processor -- start 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a4800640 1 -- start start 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a4800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc89c108680 0x7fc89c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a4800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc89c109870 0x7fc89c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a4800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc89c10a230 0x7fc89c1a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a4800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c114330 con 0x7fc89c109870 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a4800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1141b0 con 0x7fc89c108680 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a4800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1144b0 con 0x7fc89c10a230 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc89c109870 0x7fc89c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc89c108680 0x7fc89c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc89c108680 0x7fc89c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45234/0 (socket says 172.21.15.18:45234) 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.434+0000 7fc8a1e00640 1 -- 172.21.15.18:0/3873790501 learned_addr learned my addr 172.21.15.18:0/3873790501 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc8a2800640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc89c10a230 0x7fc89c1a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc8a1e00640 1 -- 172.21.15.18:0/3873790501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc89c10a230 msgr2=0x7fc89c1a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc8a1e00640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc89c10a230 0x7fc89c1a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc8a1e00640 1 -- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc89c109870 msgr2=0x7fc89c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc8a1e00640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc89c109870 0x7fc89c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc8a1e00640 1 -- 172.21.15.18:0/3873790501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c1a6560 con 0x7fc89c108680 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc8a1e00640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc89c108680 0x7fc89c19e940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9c0 tx=0x7fc89000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc896a00640 1 -- 172.21.15.18:0/3873790501 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89000cd80 con 0x7fc89c108680 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.435+0000 7fc896a00640 1 -- 172.21.15.18:0/3873790501 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8900035f0 con 0x7fc89c108680 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.436+0000 7fc896a00640 1 -- 172.21.15.18:0/3873790501 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890010690 con 0x7fc89c108680 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.436+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c1a6850 con 0x7fc89c108680 2024-04-16T17:04:31.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.436+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c1a6d70 con 0x7fc89c108680 2024-04-16T17:04:31.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.437+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c10f9a0 con 0x7fc89c108680 2024-04-16T17:04:31.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.439+0000 7fc896a00640 1 -- 172.21.15.18:0/3873790501 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc8900037a0 con 0x7fc89c108680 2024-04-16T17:04:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.440+0000 7fc896a00640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc8840a3bb0 0x7fc8840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:31.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.440+0000 7fc896a00640 1 -- 172.21.15.18:0/3873790501 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fc8900d0c30 con 0x7fc89c108680 2024-04-16T17:04:31.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.440+0000 7fc8a1400640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc8840a3bb0 0x7fc8840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:31.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.441+0000 7fc8a1400640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc8840a3bb0 0x7fc8840a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc88c02c240 tx=0x7fc88c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:31.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.445+0000 7fc896a00640 1 -- 172.21.15.18:0/3873790501 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc89009d1e0 con 0x7fc89c108680 2024-04-16T17:04:31.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:31 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2061737927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:31.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:31 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3216918392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:31.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:31 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2061737927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:31.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:31 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3216918392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:31.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.695+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc89c02d620 con 0x7fc89c108680 2024-04-16T17:04:31.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.696+0000 7fc896a00640 1 -- 172.21.15.18:0/3873790501 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11848 (secure 0 0 0) 0x7fc89009cae0 con 0x7fc89c108680 2024-04-16T17:04:31.698 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:31.698 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:01:59.165885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:04:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 24 2024-04-16T17:04:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc8840a3bb0 msgr2=0x7fc8840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc8840a3bb0 0x7fc8840a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc88c02c240 tx=0x7fc88c02df90 comp rx=0 tx=0).stop 2024-04-16T17:04:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc89c108680 msgr2=0x7fc89c19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc89c108680 0x7fc89c19e940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9c0 tx=0x7fc89000ee80 comp rx=0 tx=0).stop 2024-04-16T17:04:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 shutdown_connections 2024-04-16T17:04:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc89c10a230 0x7fc89c1a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fc8840a3bb0 0x7fc8840a6070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc89c109870 0x7fc89c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 --2- 172.21.15.18:0/3873790501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc89c108680 0x7fc89c19e940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 >> 172.21.15.18:0/3873790501 conn(0x7fc89c077650 msgr2=0x7fc89c10cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 shutdown_connections 2024-04-16T17:04:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.699+0000 7fc8a4800640 1 -- 172.21.15.18:0/3873790501 wait complete. 2024-04-16T17:04:31.721 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:04:31.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:31 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2061737927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:31.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:31 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3216918392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 -- 172.21.15.18:0/783031896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c104160 msgr2=0x7f198c104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 --2- 172.21.15.18:0/783031896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c104160 0x7f198c104580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f197c009860 tx=0x7f197c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 -- 172.21.15.18:0/783031896 shutdown_connections 2024-04-16T17:04:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 --2- 172.21.15.18:0/783031896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f198c105d10 0x7f198c10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 --2- 172.21.15.18:0/783031896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f198c105350 0x7f198c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 --2- 172.21.15.18:0/783031896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c104160 0x7f198c104580 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 -- 172.21.15.18:0/783031896 >> 172.21.15.18:0/783031896 conn(0x7f198c0ff6d0 msgr2=0x7f198c101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 -- 172.21.15.18:0/783031896 shutdown_connections 2024-04-16T17:04:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.946+0000 7f198be00640 1 -- 172.21.15.18:0/783031896 wait complete. 2024-04-16T17:04:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198be00640 1 Processor -- start 2024-04-16T17:04:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198be00640 1 -- start start 2024-04-16T17:04:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f198c104160 0x7f198c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f198c105350 0x7f198c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c105d10 0x7f198c1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10fe10 con 0x7f198c104160 2024-04-16T17:04:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10fc90 con 0x7f198c105350 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10ff90 con 0x7f198c105d10 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.947+0000 7f198aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f198c104160 0x7f198c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c105d10 0x7f198c1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f198c105350 0x7f198c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c105d10 0x7f198c1a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39876/0 (socket says 172.21.15.18:39876) 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 -- 172.21.15.18:0/4234834597 learned_addr learned my addr 172.21.15.18:0/4234834597 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f198c104160 0x7f198c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40908/0 (socket says 172.21.15.18:40908) 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 -- 172.21.15.18:0/4234834597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f198c105350 msgr2=0x7f198c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f198c105350 0x7f198c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 -- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f198c104160 msgr2=0x7f198c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f198c104160 0x7f198c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 -- 172.21.15.18:0/4234834597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c1a2060 con 0x7f198c105d10 2024-04-16T17:04:31.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f198b400640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c105d10 0x7f198c1a19f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f198400eb10 tx=0x7f198400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:31.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.948+0000 7f1983400640 1 -- 172.21.15.18:0/4234834597 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198400ce20 con 0x7f198c105d10 2024-04-16T17:04:31.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.949+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c1a2350 con 0x7f198c105d10 2024-04-16T17:04:31.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.949+0000 7f1983400640 1 -- 172.21.15.18:0/4234834597 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1984005020 con 0x7f198c105d10 2024-04-16T17:04:31.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.949+0000 7f1983400640 1 -- 172.21.15.18:0/4234834597 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984010690 con 0x7f198c105d10 2024-04-16T17:04:31.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.949+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c1a28f0 con 0x7f198c105d10 2024-04-16T17:04:31.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.950+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c10b480 con 0x7f198c105d10 2024-04-16T17:04:31.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.953+0000 7f1983400640 1 -- 172.21.15.18:0/4234834597 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1984005540 con 0x7f198c105d10 2024-04-16T17:04:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.954+0000 7f1983400640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19700a3c20 0x7f19700a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.954+0000 7f1983400640 1 -- 172.21.15.18:0/4234834597 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f19840d11a0 con 0x7f198c105d10 2024-04-16T17:04:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.954+0000 7f198aa00640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19700a3c20 0x7f19700a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:31.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.955+0000 7f198aa00640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19700a3c20 0x7f19700a60e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f197c0349e0 tx=0x7f197c034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:31.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:31.955+0000 7f1983400640 1 -- 172.21.15.18:0/4234834597 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f198409d750 con 0x7f198c105d10 2024-04-16T17:04:32.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.163+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f198c062c00 con 0x7f198c105d10 2024-04-16T17:04:32.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.166+0000 7f1983400640 1 -- 172.21.15.18:0/4234834597 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24748 (secure 0 0 0) 0x7f198409d050 con 0x7f198c105d10 2024-04-16T17:04:32.168 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:32.168 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":93,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:03:27.035238+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.168+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19700a3c20 msgr2=0x7f19700a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.168+0000 7f198be00640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19700a3c20 0x7f19700a60e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f197c0349e0 tx=0x7f197c034ab0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.168+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c105d10 msgr2=0x7f198c1a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.168+0000 7f198be00640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c105d10 0x7f198c1a19f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f198400eb10 tx=0x7f198400ef10 comp rx=0 tx=0).stop 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.169+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 shutdown_connections 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.169+0000 7f198be00640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198c105d10 0x7f198c1a19f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.169+0000 7f198be00640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f198c105350 0x7f198c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.169+0000 7f198be00640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f19700a3c20 0x7f19700a60e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.169+0000 7f198be00640 1 --2- 172.21.15.18:0/4234834597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f198c104160 0x7f198c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.169+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 >> 172.21.15.18:0/4234834597 conn(0x7f198c0ff6d0 msgr2=0x7f198c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.169+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 shutdown_connections 2024-04-16T17:04:32.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.169+0000 7f198be00640 1 -- 172.21.15.18:0/4234834597 wait complete. 2024-04-16T17:04:32.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-16T17:01:24.398529+0000', 'modified': '2024-04-16T17:01:59.165885+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24596, 'mds_1': 24566, 'mds_2': 24547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/1265446335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 1265446335}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 1265446335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6837/1790976373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6836', 'nonce': 1790976373}, {'type': 'v1', 'addr': '172.21.15.43:6837', 'nonce': 1790976373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.22:6837/94342022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6836', 'nonce': 94342022}, {'type': 'v1', 'addr': '172.21.15.22:6837', 'nonce': 94342022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.43:6839/3993514244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6838', 'nonce': 3993514244}, {'type': 'v1', 'addr': '172.21.15.43:6839', 'nonce': 3993514244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.18:6839/294785837', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6838', 'nonce': 294785837}, {'type': 'v1', 'addr': '172.21.15.18:6839', 'nonce': 294785837}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6839/836242731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6838', 'nonce': 836242731}, {'type': 'v1', 'addr': '172.21.15.22:6839', 'nonce': 836242731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24566, 24547, 24596]} 2024-04-16T17:04:32.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-16T17:04:32.189 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-16T17:04:32.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 -- 172.21.15.18:0/1923935425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f781039a0 msgr2=0x7f3f7810a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 --2- 172.21.15.18:0/1923935425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f781039a0 0x7f3f7810a2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009ca0 tx=0x7f3f7002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 -- 172.21.15.18:0/1923935425 shutdown_connections 2024-04-16T17:04:32.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 --2- 172.21.15.18:0/1923935425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f781039a0 0x7f3f7810a2c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 --2- 172.21.15.18:0/1923935425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f78102fe0 0x7f3f78103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 --2- 172.21.15.18:0/1923935425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f78101e90 0x7f3f781022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 -- 172.21.15.18:0/1923935425 >> 172.21.15.18:0/1923935425 conn(0x7f3f780fd400 msgr2=0x7f3f780ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 -- 172.21.15.18:0/1923935425 shutdown_connections 2024-04-16T17:04:32.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.412+0000 7f3f77e00640 1 -- 172.21.15.18:0/1923935425 wait complete. 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.413+0000 7f3f77e00640 1 Processor -- start 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.413+0000 7f3f77e00640 1 -- start start 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f78101e90 0x7f3f78075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f78102fe0 0x7f3f780743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f781039a0 0x7f3f78074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7810db20 con 0x7f3f781039a0 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7810d9a0 con 0x7f3f78101e90 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7810dca0 con 0x7f3f78102fe0 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f781039a0 0x7f3f78074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f781039a0 0x7f3f78074930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40916/0 (socket says 172.21.15.18:40916) 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f77400640 1 -- 172.21.15.18:0/2482343471 learned_addr learned my addr 172.21.15.18:0/2482343471 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f76a00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f78101e90 0x7f3f78075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f76000640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f78102fe0 0x7f3f780743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.414+0000 7f3f76a00640 1 -- 172.21.15.18:0/2482343471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f78102fe0 msgr2=0x7f3f780743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f76a00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f78102fe0 0x7f3f780743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f76a00640 1 -- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f781039a0 msgr2=0x7f3f78074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f76a00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f781039a0 0x7f3f78074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f76a00640 1 -- 172.21.15.18:0/2482343471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f780751c0 con 0x7f3f78101e90 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f76000640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f78102fe0 0x7f3f780743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:32.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f76a00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f78101e90 0x7f3f78075db0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800e9f0 tx=0x7f3f6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f7ca00640 1 -- 172.21.15.18:0/2482343471 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6800ccc0 con 0x7f3f78101e90 2024-04-16T17:04:32.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f7ca00640 1 -- 172.21.15.18:0/2482343471 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f68010910 con 0x7f3f78101e90 2024-04-16T17:04:32.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f7ca00640 1 -- 172.21.15.18:0/2482343471 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68018960 con 0x7f3f78101e90 2024-04-16T17:04:32.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f77e00640 1 -- 172.21.15.18:0/2482343471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f781a2250 con 0x7f3f78101e90 2024-04-16T17:04:32.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.415+0000 7f3f77e00640 1 -- 172.21.15.18:0/2482343471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f781a2750 con 0x7f3f78101e90 2024-04-16T17:04:32.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.419+0000 7f3f7ca00640 1 -- 172.21.15.18:0/2482343471 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3f68010ac0 con 0x7f3f78101e90 2024-04-16T17:04:32.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.420+0000 7f3f80a71740 1 Processor -- start 2024-04-16T17:04:32.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.420+0000 7f3f80a71740 1 -- start start 2024-04-16T17:04:32.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.420+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab0e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.420+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.420+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab16490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.420+0000 7f3f80a71740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b1daa74980 con 0x55b1dab0ee90 2024-04-16T17:04:32.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.420+0000 7f3f80a71740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b1daa1b220 con 0x55b1dab14000 2024-04-16T17:04:32.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.420+0000 7f3f80a71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b1daa70940 con 0x55b1dab0e4a0 2024-04-16T17:04:32.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab0e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f76a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab0e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39908/0 (socket says 172.21.15.18:39908) 2024-04-16T17:04:32.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 -- 172.21.15.18:0/1364450587 learned_addr learned my addr 172.21.15.18:0/1364450587 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f76000640 1 --2- 172.21.15.18:0/1364450587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab16490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 -- 172.21.15.18:0/1364450587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 msgr2=0x55b1dab16490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 --2- 172.21.15.18:0/1364450587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab16490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 -- 172.21.15.18:0/1364450587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 msgr2=0x55b1dab13990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 --2- 172.21.15.18:0/1364450587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab13990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 -- 172.21.15.18:0/1364450587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1da7872d0 con 0x55b1dab0e4a0 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f7ca00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3f5c0a3c30 0x7f3f5c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.421+0000 7f3f77400640 1 --2- 172.21.15.18:0/1364450587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab0e8c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3f700323d0 tx=0x7f3f70005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71e51c4129b379c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.422+0000 7f3f7ca00640 1 -- 172.21.15.18:0/2482343471 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f3f680d0be0 con 0x7f3f78101e90 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.422+0000 7f3f76a00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3f5c0a3c30 0x7f3f5c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.422+0000 7f3f67e00640 1 -- 172.21.15.18:0/1364450587 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70043070 con 0x55b1dab0e4a0 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.422+0000 7f3f67e00640 1 -- 172.21.15.18:0/1364450587 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70032c90 con 0x55b1dab0e4a0 2024-04-16T17:04:32.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.422+0000 7f3f67e00640 1 -- 172.21.15.18:0/1364450587 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70039040 con 0x55b1dab0e4a0 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 -- 172.21.15.18:0/1364450587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 msgr2=0x55b1dab0e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1364450587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab0e8c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3f700323d0 tx=0x7f3f70005ef0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f76a00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3f5c0a3c30 0x7f3f5c0a60f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3f68099460 tx=0x7f3f680d4620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 -- 172.21.15.18:0/1364450587 shutdown_connections 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1364450587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab16490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1364450587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab13990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1364450587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab0e8c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 -- 172.21.15.18:0/1364450587 >> 172.21.15.18:0/1364450587 conn(0x55b1dab09a10 msgr2=0x55b1dab09e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 -- 172.21.15.18:0/1364450587 shutdown_connections 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 -- 172.21.15.18:0/1364450587 wait complete. 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 -- start start 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.423+0000 7f3f80a71740 1 Processor -- start 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab18f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab1ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f80a71740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b1dab16f50 con 0x55b1dab0ee90 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f80a71740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b1daa2e590 con 0x55b1dab14000 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f80a71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b1dab170d0 con 0x55b1dab0e4a0 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f80a71740 10 client.0 _ll_register_callbacks cb 0x55b1daa36050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f76000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab18f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f76000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab18f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40922/0 (socket says 172.21.15.18:40922) 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f76000640 1 -- 172.21.15.18:0/1928712131 learned_addr learned my addr 172.21.15.18:0/1928712131 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f77400640 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.424+0000 7f3f77400640 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab1ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f76000640 1 -- 172.21.15.18:0/1928712131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 msgr2=0x55b1dab189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f76000640 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f76000640 1 -- 172.21.15.18:0/1928712131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 msgr2=0x55b1dab1ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f76000640 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab1ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f76000640 1 -- 172.21.15.18:0/1928712131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1da8516b0 con 0x55b1dab0ee90 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f77400640 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab1ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f76000640 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab18f00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000bf60 tx=0x7f3f6000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cecead5d8a92c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f64c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f64c00640 1 -- 172.21.15.18:0/1928712131 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6001f060 con 0x55b1dab0ee90 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f64c00640 1 -- 172.21.15.18:0/1928712131 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f3f60010960 con 0x55b1dab0ee90 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.425+0000 7f3f80a71740 1 -- 172.21.15.18:0/1928712131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f70032c90 con 0x55b1dab0ee90 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.426+0000 7f3f64c00640 1 -- 172.21.15.18:0/1928712131 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f60026050 con 0x55b1dab0ee90 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.426+0000 7f3f80a71740 10 client.15240 fetch_fsmap learned FSMap version 24 2024-04-16T17:04:32.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.426+0000 7f3f80a71740 1 -- 172.21.15.18:0/1928712131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b1dab23380 con 0x55b1dab0ee90 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.426+0000 7f3f64c00640 1 -- 172.21.15.18:0/1928712131 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70032c90 con 0x55b1dab0ee90 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.426+0000 7f3f64c00640 1 -- 172.21.15.18:0/1928712131 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3f60019030 con 0x55b1dab0ee90 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.426+0000 7f3f80a71740 10 client.15240 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.426+0000 7f3f80a71740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.427+0000 7f3f80a71740 20 client.15240 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.427+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b1dab25a00 0x55b1dab26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.427+0000 7f3f80a71740 4 client.15240 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.427+0000 7f3f80a71740 1 -- 172.21.15.18:0/1928712131 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1daa823a0 con 0x55b1dab25a00 2024-04-16T17:04:32.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.427+0000 7f3f77400640 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b1dab25a00 0x55b1dab26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.427+0000 7f3f77400640 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b1dab25a00 0x55b1dab26e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d90a532f059294f server_cookie=fbcdc9df97cdcf2d in_seq=0 out_seq=0 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.427+0000 7f3f64c00640 10 client.15240 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.428+0000 7f3f64c00640 1 -- 172.21.15.18:0/1928712131 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1daa823a0 con 0x55b1dab25a00 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.428+0000 7f3f64c00640 10 client.15240 handle_command_reply: tid=0 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.428+0000 7f3f80a71740 1 client.15240 shutdown 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 -- 172.21.15.18:0/1928712131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 msgr2=0x55b1dab18f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab18f00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000bf60 tx=0x7f3f6000bf90 comp rx=0 tx=0).stop 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 -- 172.21.15.18:0/1928712131 shutdown_connections 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b1dab25a00 0x55b1dab26e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab1ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab18f00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1928712131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 -- 172.21.15.18:0/1928712131 >> 172.21.15.18:0/1928712131 conn(0x55b1dab09a10 msgr2=0x55b1dab08c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f64c00640 0 client.15240 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:04:32.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 -- 172.21.15.18:0/1928712131 shutdown_connections 2024-04-16T17:04:32.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 1 -- 172.21.15.18:0/1928712131 wait complete. 2024-04-16T17:04:32.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.429+0000 7f3f80a71740 20 client.15240 trim_cache size 0 max 16384 2024-04-16T17:04:32.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:32 smithi018 ceph-mon[31776]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:32.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:32 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3873790501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:32.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:32 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4234834597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f80a71740 1 Processor -- start 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f80a71740 1 -- start start 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab09df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab1ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab10420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f80a71740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b1dab251b0 con 0x55b1dab0ee90 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f80a71740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b1dab25880 con 0x55b1dab14000 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f80a71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b1dab25f50 con 0x55b1dab0e4a0 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f76000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab1ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab09df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab09df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39920/0 (socket says 172.21.15.18:39920) 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f77400640 1 -- 172.21.15.18:0/1882241238 learned_addr learned my addr 172.21.15.18:0/1882241238 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.450+0000 7f3f77400640 1 --2- 172.21.15.18:0/1882241238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab10420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f77400640 1 -- 172.21.15.18:0/1882241238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 msgr2=0x55b1dab10420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f77400640 1 --2- 172.21.15.18:0/1882241238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab10420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f77400640 1 -- 172.21.15.18:0/1882241238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 msgr2=0x55b1dab1ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f77400640 1 --2- 172.21.15.18:0/1882241238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab1ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f77400640 1 -- 172.21.15.18:0/1882241238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1dab0f720 con 0x55b1dab0e4a0 2024-04-16T17:04:32.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f77400640 1 --2- 172.21.15.18:0/1882241238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab09df0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f70005f60 tx=0x7f3f70005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89dc4952133480a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f67400640 1 -- 172.21.15.18:0/1882241238 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70039040 con 0x55b1dab0e4a0 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f67400640 1 -- 172.21.15.18:0/1882241238 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70003a70 con 0x55b1dab0e4a0 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.451+0000 7f3f67400640 1 -- 172.21.15.18:0/1882241238 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70043040 con 0x55b1dab0e4a0 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 -- 172.21.15.18:0/1882241238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 msgr2=0x55b1dab09df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1882241238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab09df0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f70005f60 tx=0x7f3f70005ef0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 -- 172.21.15.18:0/1882241238 shutdown_connections 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1882241238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab14000 0x55b1dab10420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1882241238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab0ee90 0x55b1dab1ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 --2- 172.21.15.18:0/1882241238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0e4a0 0x55b1dab09df0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 -- 172.21.15.18:0/1882241238 >> 172.21.15.18:0/1882241238 conn(0x55b1dab09a10 msgr2=0x55b1dab17470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 -- 172.21.15.18:0/1882241238 shutdown_connections 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 -- 172.21.15.18:0/1882241238 wait complete. 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.452+0000 7f3f80a71740 1 -- start start 2024-04-16T17:04:32.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f80a71740 1 Processor -- start 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab0e4a0 0x55b1dab122b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0ee90 0x55b1dab127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f80a71740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab14000 0x55b1dab921b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f80a71740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b1dab26cf0 con 0x55b1dab14000 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f80a71740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b1dab26620 con 0x55b1dab0e4a0 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f80a71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b1dab273c0 con 0x55b1dab0ee90 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f80a71740 10 client.0 _ll_register_callbacks cb 0x7f3f6000b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.453+0000 7f3f77400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab0e4a0 0x55b1dab122b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f76000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0ee90 0x55b1dab127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f77400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab0e4a0 0x55b1dab122b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45324/0 (socket says 172.21.15.18:45324) 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f77400640 1 -- 172.21.15.18:0/4251489508 learned_addr learned my addr 172.21.15.18:0/4251489508 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f77400640 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab14000 0x55b1dab921b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f76000640 1 -- 172.21.15.18:0/4251489508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab0e4a0 msgr2=0x55b1dab122b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f76000640 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab0e4a0 0x55b1dab122b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f76000640 1 -- 172.21.15.18:0/4251489508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab14000 msgr2=0x55b1dab921b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f76000640 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab14000 0x55b1dab921b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f76000640 1 -- 172.21.15.18:0/4251489508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1dab926f0 con 0x55b1dab0ee90 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f76000640 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0ee90 0x55b1dab127f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f60000c00 tx=0x7f3f60012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec7d3bd271d3c710 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.454+0000 7f3f64c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f64c00640 1 -- 172.21.15.18:0/4251489508 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6002e020 con 0x55b1dab0ee90 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f80a71740 1 -- 172.21.15.18:0/4251489508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f70003a70 con 0x55b1dab0ee90 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f64c00640 1 -- 172.21.15.18:0/4251489508 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f3f600054d0 con 0x55b1dab0ee90 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f64c00640 1 -- 172.21.15.18:0/4251489508 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f60039050 con 0x55b1dab0ee90 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f80a71740 10 client.25363 fetch_fsmap learned FSMap version 24 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f80a71740 1 -- 172.21.15.18:0/4251489508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b1dab93920 con 0x55b1dab0ee90 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f64c00640 1 -- 172.21.15.18:0/4251489508 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70003a70 con 0x55b1dab0ee90 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f64c00640 1 -- 172.21.15.18:0/4251489508 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3f60019040 con 0x55b1dab0ee90 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f80a71740 10 client.25363 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f80a71740 10 client.25363 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f80a71740 20 client.25363 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.455+0000 7f3f80a71740 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b1dab98540 0x55b1dab97250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.456+0000 7f3f80a71740 4 client.25363 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.456+0000 7f3f80a71740 1 -- 172.21.15.18:0/4251489508 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "status"}) -- 0x55b1daa7c210 con 0x55b1dab98540 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.456+0000 7f3f77400640 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b1dab98540 0x55b1dab97250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.456+0000 7f3f77400640 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b1dab98540 0x55b1dab97250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=860df908e67bab01 server_cookie=d668812e74538f5c in_seq=0 out_seq=0 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.456+0000 7f3f64c00640 10 client.25363 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f64c00640 1 -- 172.21.15.18:0/4251489508 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b1daa7c210 con 0x55b1dab98540 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f64c00640 10 client.25363 handle_command_reply: tid=0 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 client.25363 shutdown 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 -- 172.21.15.18:0/4251489508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0ee90 msgr2=0x55b1dab127f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0ee90 0x55b1dab127f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f60000c00 tx=0x7f3f60012eb0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 -- 172.21.15.18:0/4251489508 shutdown_connections 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b1dab98540 0x55b1dab97250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1dab14000 0x55b1dab921b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1dab0ee90 0x55b1dab127f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f64c00640 0 client.25363 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 --2- 172.21.15.18:0/4251489508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1dab0e4a0 0x55b1dab122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 -- 172.21.15.18:0/4251489508 >> 172.21.15.18:0/4251489508 conn(0x55b1dab09a10 msgr2=0x55b1dab10420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 -- 172.21.15.18:0/4251489508 shutdown_connections 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 1 -- 172.21.15.18:0/4251489508 wait complete. 2024-04-16T17:04:32.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.457+0000 7f3f80a71740 20 client.25363 trim_cache size 0 max 16384 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 2, 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24547, 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 154.30196530800001, 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 24, 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 93, 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T17:04:32.462 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 205.143278823 2024-04-16T17:04:32.463 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:04:32.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 -- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3f5c0a3c30 msgr2=0x7f3f5c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3f5c0a3c30 0x7f3f5c0a60f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3f68099460 tx=0x7f3f680d4620 comp rx=0 tx=0).stop 2024-04-16T17:04:32.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 -- 172.21.15.18:0/2482343471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f78101e90 msgr2=0x7f3f78075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f78101e90 0x7f3f78075db0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800e9f0 tx=0x7f3f6800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 -- 172.21.15.18:0/2482343471 shutdown_connections 2024-04-16T17:04:32.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f3f5c0a3c30 0x7f3f5c0a60f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f781039a0 0x7f3f78074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f78102fe0 0x7f3f780743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 --2- 172.21.15.18:0/2482343471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f78101e90 0x7f3f78075db0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 -- 172.21.15.18:0/2482343471 >> 172.21.15.18:0/2482343471 conn(0x7f3f780fd400 msgr2=0x7f3f78106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 -- 172.21.15.18:0/2482343471 shutdown_connections 2024-04-16T17:04:32.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.458+0000 7f3f77e00640 1 -- 172.21.15.18:0/2482343471 wait complete. 2024-04-16T17:04:32.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-16T17:04:32.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:32 smithi022 ceph-mon[36023]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:32.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:32 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3873790501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:32.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:32 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4234834597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:32.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.694+0000 7fac63200640 1 -- 172.21.15.18:0/1771792857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac64105210 msgr2=0x7fac64107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.694+0000 7fac63200640 1 --2- 172.21.15.18:0/1771792857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac64105210 0x7fac64107600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fac50009860 tx=0x7fac5002bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:32.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.695+0000 7fac63200640 1 -- 172.21.15.18:0/1771792857 shutdown_connections 2024-04-16T17:04:32.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.695+0000 7fac63200640 1 --2- 172.21.15.18:0/1771792857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac64107b40 0x7fac64109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.695+0000 7fac63200640 1 --2- 172.21.15.18:0/1771792857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac64105210 0x7fac64107600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.695+0000 7fac63200640 1 --2- 172.21.15.18:0/1771792857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac641028e0 0x7fac64104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.695+0000 7fac63200640 1 -- 172.21.15.18:0/1771792857 >> 172.21.15.18:0/1771792857 conn(0x7fac640fc2f0 msgr2=0x7fac640fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.695+0000 7fac63200640 1 -- 172.21.15.18:0/1771792857 shutdown_connections 2024-04-16T17:04:32.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.695+0000 7fac63200640 1 -- 172.21.15.18:0/1771792857 wait complete. 2024-04-16T17:04:32.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.696+0000 7fac63200640 1 Processor -- start 2024-04-16T17:04:32.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.696+0000 7fac63200640 1 -- start start 2024-04-16T17:04:32.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac63200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac641028e0 0x7fac64102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac63200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac64105210 0x7fac64100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac63200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac64107b40 0x7fac64100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac63200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fac6410dad0 con 0x7fac641028e0 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac63200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fac6410d950 con 0x7fac64107b40 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac63200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac6410dc50 con 0x7fac64105210 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac641028e0 0x7fac64102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac641028e0 0x7fac64102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40958/0 (socket says 172.21.15.18:40958) 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61e00640 1 -- 172.21.15.18:0/2191056554 learned_addr learned my addr 172.21.15.18:0/2191056554 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61400640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac64105210 0x7fac64100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac62800640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac64107b40 0x7fac64100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61e00640 1 -- 172.21.15.18:0/2191056554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac64105210 msgr2=0x7fac64100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61e00640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac64105210 0x7fac64100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61e00640 1 -- 172.21.15.18:0/2191056554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac64107b40 msgr2=0x7fac64100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61e00640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac64107b40 0x7fac64100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.697+0000 7fac61e00640 1 -- 172.21.15.18:0/2191056554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac64101660 con 0x7fac641028e0 2024-04-16T17:04:32.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.698+0000 7fac61e00640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac641028e0 0x7fac64102250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac5400e9f0 tx=0x7fac5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.698+0000 7fac5aa00640 1 -- 172.21.15.18:0/2191056554 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5400cd80 con 0x7fac641028e0 2024-04-16T17:04:32.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.698+0000 7fac63200640 1 -- 172.21.15.18:0/2191056554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac641a22c0 con 0x7fac641028e0 2024-04-16T17:04:32.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.698+0000 7fac5aa00640 1 -- 172.21.15.18:0/2191056554 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac54010910 con 0x7fac641028e0 2024-04-16T17:04:32.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.698+0000 7fac5aa00640 1 -- 172.21.15.18:0/2191056554 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54018950 con 0x7fac641028e0 2024-04-16T17:04:32.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.698+0000 7fac63200640 1 -- 172.21.15.18:0/2191056554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac641a27e0 con 0x7fac641028e0 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.702+0000 7fac5aa00640 1 -- 172.21.15.18:0/2191056554 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fac54010ac0 con 0x7fac641028e0 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.702+0000 7fac6bba3740 1 Processor -- start 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.702+0000 7fac6bba3740 1 -- start start 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6c4a0 0x55c657c6c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 0x55c657c71990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c72000 0x55c657c74490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac6bba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c657bd2980 con 0x55c657c6ce90 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac6bba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c657b79220 con 0x55c657c72000 2024-04-16T17:04:32.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac6bba3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c657bce940 con 0x55c657c6c4a0 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 0x55c657c71990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c72000 0x55c657c74490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c72000 0x55c657c74490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45356/0 (socket says 172.21.15.18:45356) 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61400640 1 -- 172.21.15.18:0/437512483 learned_addr learned my addr 172.21.15.18:0/437512483 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac62800640 1 --2- 172.21.15.18:0/437512483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6c4a0 0x55c657c6c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61400640 1 -- 172.21.15.18:0/437512483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6c4a0 msgr2=0x55c657c6c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61400640 1 --2- 172.21.15.18:0/437512483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6c4a0 0x55c657c6c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61400640 1 -- 172.21.15.18:0/437512483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 msgr2=0x55c657c71990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61400640 1 --2- 172.21.15.18:0/437512483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 0x55c657c71990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.703+0000 7fac61400640 1 -- 172.21.15.18:0/437512483 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6578e52d0 con 0x55c657c72000 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.704+0000 7fac5aa00640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fac480a3bb0 0x7fac480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.704+0000 7fac61400640 1 --2- 172.21.15.18:0/437512483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c72000 0x55c657c74490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac500057c0 tx=0x7fac50009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5078731ee5f1575 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.704+0000 7fac5aa00640 1 -- 172.21.15.18:0/2191056554 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fac540d0bd0 con 0x7fac641028e0 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.704+0000 7fac4f400640 1 -- 172.21.15.18:0/437512483 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50043020 con 0x55c657c72000 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.704+0000 7fac62800640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fac480a3bb0 0x7fac480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.704+0000 7fac4f400640 1 -- 172.21.15.18:0/437512483 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac50034db0 con 0x55c657c72000 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.704+0000 7fac4f400640 1 -- 172.21.15.18:0/437512483 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50039040 con 0x55c657c72000 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 -- 172.21.15.18:0/437512483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c72000 msgr2=0x55c657c74490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac62800640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fac480a3bb0 0x7fac480a6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55c657c6d6f0 tx=0x7fac5c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 --2- 172.21.15.18:0/437512483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c72000 0x55c657c74490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac500057c0 tx=0x7fac50009490 comp rx=0 tx=0).stop 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 -- 172.21.15.18:0/437512483 shutdown_connections 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 --2- 172.21.15.18:0/437512483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c72000 0x55c657c74490 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 --2- 172.21.15.18:0/437512483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 0x55c657c71990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 --2- 172.21.15.18:0/437512483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6c4a0 0x55c657c6c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 -- 172.21.15.18:0/437512483 >> 172.21.15.18:0/437512483 conn(0x55c657c67a10 msgr2=0x55c657c67e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 -- 172.21.15.18:0/437512483 shutdown_connections 2024-04-16T17:04:32.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.705+0000 7fac6bba3740 1 -- 172.21.15.18:0/437512483 wait complete. 2024-04-16T17:04:32.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.706+0000 7fac6bba3740 1 -- start start 2024-04-16T17:04:32.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.706+0000 7fac6bba3740 1 Processor -- start 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.706+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.706+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 0x55c657c76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.706+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c72000 0x55c657c7df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.706+0000 7fac6bba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c657c74f50 con 0x55c657c6ce90 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.706+0000 7fac6bba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c657b8c590 con 0x55c657c6c4a0 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.706+0000 7fac6bba3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c657c750d0 con 0x55c657c72000 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c769b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45372/0 (socket says 172.21.15.18:45372) 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac6bba3740 10 client.0 _ll_register_callbacks cb 0x55c657b94050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 -- 172.21.15.18:0/317681691 learned_addr learned my addr 172.21.15.18:0/317681691 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61400640 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 0x55c657c76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac62800640 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c72000 0x55c657c7df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 -- 172.21.15.18:0/317681691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c72000 msgr2=0x55c657c7df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c72000 0x55c657c7df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 -- 172.21.15.18:0/317681691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 msgr2=0x55c657c76ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 0x55c657c76ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 -- 172.21.15.18:0/317681691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6579af6b0 con 0x55c657c6c4a0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac61e00640 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c769b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c6a0 tx=0x7fac5409c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4f552a478376b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.707+0000 7fac43e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac43e00640 1 -- 172.21.15.18:0/317681691 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5409cb30 con 0x55c657c6c4a0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac43e00640 1 -- 172.21.15.18:0/317681691 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fac54018b50 con 0x55c657c6c4a0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac6bba3740 1 -- 172.21.15.18:0/317681691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac50034db0 con 0x55c657c6c4a0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac43e00640 1 -- 172.21.15.18:0/317681691 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fac540a0e30 con 0x55c657c6c4a0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac6bba3740 10 client.25388 fetch_fsmap learned FSMap version 24 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac6bba3740 1 -- 172.21.15.18:0/317681691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c657c81310 con 0x55c657c6c4a0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac43e00640 1 -- 172.21.15.18:0/317681691 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac50034db0 con 0x55c657c6c4a0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac43e00640 1 -- 172.21.15.18:0/317681691 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fac5409c110 con 0x55c657c6c4a0 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.708+0000 7fac6bba3740 10 client.25388 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.709+0000 7fac6bba3740 10 client.25388 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-16T17:04:32.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.709+0000 7fac6bba3740 20 client.25388 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.709+0000 7fac6bba3740 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c657c838f0 0x55c657c84d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.709+0000 7fac6bba3740 4 client.25388 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.709+0000 7fac6bba3740 1 -- 172.21.15.18:0/317681691 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c657be03a0 con 0x55c657c838f0 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.709+0000 7fac61400640 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c657c838f0 0x55c657c84d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.710+0000 7fac61400640 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c657c838f0 0x55c657c84d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a00a56800d43325a server_cookie=42523837be89f12b in_seq=0 out_seq=0 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.710+0000 7fac43e00640 10 client.25388 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.710+0000 7fac43e00640 1 -- 172.21.15.18:0/317681691 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c657be03a0 con 0x55c657c838f0 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.710+0000 7fac43e00640 10 client.25388 handle_command_reply: tid=0 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.710+0000 7fac6bba3740 1 client.25388 shutdown 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 -- 172.21.15.18:0/317681691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 msgr2=0x55c657c769b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c769b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c6a0 tx=0x7fac5409c900 comp rx=0 tx=0).stop 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 -- 172.21.15.18:0/317681691 shutdown_connections 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c657c838f0 0x55c657c84d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c72000 0x55c657c7df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c6ce90 0x55c657c76ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 --2- 172.21.15.18:0/317681691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c769b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 -- 172.21.15.18:0/317681691 >> 172.21.15.18:0/317681691 conn(0x55c657c67a10 msgr2=0x55c657c66c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac43e00640 0 client.25388 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:04:32.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 -- 172.21.15.18:0/317681691 shutdown_connections 2024-04-16T17:04:32.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 1 -- 172.21.15.18:0/317681691 wait complete. 2024-04-16T17:04:32.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.711+0000 7fac6bba3740 20 client.25388 trim_cache size 0 max 16384 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.731+0000 7fac6bba3740 1 Processor -- start 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.731+0000 7fac6bba3740 1 -- start start 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c62a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 0x55c657c7def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 0x55c657c6e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac6bba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c657c830a0 con 0x55c657c72000 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac6bba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c657c83770 con 0x55c657c6c4a0 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac6bba3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c657c83e40 con 0x55c657c6ce90 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac62800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 0x55c657c6e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c62a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c62a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45384/0 (socket says 172.21.15.18:45384) 2024-04-16T17:04:32.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac61e00640 1 -- 172.21.15.18:0/2088014122 learned_addr learned my addr 172.21.15.18:0/2088014122 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.732+0000 7fac61400640 1 --2- 172.21.15.18:0/2088014122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 0x55c657c7def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac61e00640 1 -- 172.21.15.18:0/2088014122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 msgr2=0x55c657c7def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac61e00640 1 --2- 172.21.15.18:0/2088014122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 0x55c657c7def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac61e00640 1 -- 172.21.15.18:0/2088014122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 msgr2=0x55c657c6e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac61e00640 1 --2- 172.21.15.18:0/2088014122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 0x55c657c6e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac61e00640 1 -- 172.21.15.18:0/2088014122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c657c68780 con 0x55c657c6c4a0 2024-04-16T17:04:32.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac61e00640 1 --2- 172.21.15.18:0/2088014122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c62a70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac5409c830 tx=0x7fac54005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2621a1903e1bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac4ea00640 1 -- 172.21.15.18:0/2088014122 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540bd0e0 con 0x55c657c6c4a0 2024-04-16T17:04:32.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac4ea00640 1 -- 172.21.15.18:0/2088014122 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac5409cb30 con 0x55c657c6c4a0 2024-04-16T17:04:32.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.733+0000 7fac4ea00640 1 -- 172.21.15.18:0/2088014122 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5409bf80 con 0x55c657c6c4a0 2024-04-16T17:04:32.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 -- 172.21.15.18:0/2088014122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 msgr2=0x55c657c62a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2088014122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c62a70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac5409c830 tx=0x7fac54005e70 comp rx=0 tx=0).stop 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 -- 172.21.15.18:0/2088014122 shutdown_connections 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2088014122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 0x55c657c6e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2088014122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 0x55c657c7def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2088014122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c62a70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 -- 172.21.15.18:0/2088014122 >> 172.21.15.18:0/2088014122 conn(0x55c657c67a10 msgr2=0x55c657c82770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 -- 172.21.15.18:0/2088014122 shutdown_connections 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.734+0000 7fac6bba3740 1 -- 172.21.15.18:0/2088014122 wait complete. 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 1 -- start start 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 1 Processor -- start 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c79510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 0x55c657c79a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 0x55c657c7ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c657c84be0 con 0x55c657c72000 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c657c84510 con 0x55c657c6c4a0 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c657c852b0 con 0x55c657c6ce90 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.735+0000 7fac6bba3740 10 client.0 _ll_register_callbacks cb 0x7fac540d4300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c79510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac62800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 0x55c657c7ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 0x55c657c79a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c79510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45398/0 (socket says 172.21.15.18:45398) 2024-04-16T17:04:32.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 -- 172.21.15.18:0/2436907439 learned_addr learned my addr 172.21.15.18:0/2436907439 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 -- 172.21.15.18:0/2436907439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 msgr2=0x55c657c79a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 0x55c657c79a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 -- 172.21.15.18:0/2436907439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 msgr2=0x55c657c7ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 0x55c657c7ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 -- 172.21.15.18:0/2436907439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c657c7d060 con 0x55c657c6c4a0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac61e00640 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c79510 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac540bd260 tx=0x7fac540ce320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a9b260af551f34e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.736+0000 7fac43e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac43e00640 1 -- 172.21.15.18:0/2436907439 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54098bf0 con 0x55c657c6c4a0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac43e00640 1 -- 172.21.15.18:0/2436907439 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fac5409bb30 con 0x55c657c6c4a0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac6bba3740 1 -- 172.21.15.18:0/2436907439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac5409cb30 con 0x55c657c6c4a0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac43e00640 1 -- 172.21.15.18:0/2436907439 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fac540ac050 con 0x55c657c6c4a0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac6bba3740 10 client.25394 fetch_fsmap learned FSMap version 24 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac6bba3740 1 -- 172.21.15.18:0/2436907439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c657ce98a0 con 0x55c657c6c4a0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac43e00640 1 -- 172.21.15.18:0/2436907439 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac5409cb30 con 0x55c657c6c4a0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac43e00640 1 -- 172.21.15.18:0/2436907439 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fac540a2180 con 0x55c657c6c4a0 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac6bba3740 10 client.25394 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:04:32.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac6bba3740 10 client.25394 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.737+0000 7fac6bba3740 20 client.25394 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.738+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c657cedf50 0x55c657cea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.738+0000 7fac6bba3740 4 client.25394 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.738+0000 7fac6bba3740 1 -- 172.21.15.18:0/2436907439 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "status"}) -- 0x55c657bda210 con 0x55c657cedf50 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.738+0000 7fac61400640 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c657cedf50 0x55c657cea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.738+0000 7fac43e00640 10 client.25394 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.738+0000 7fac61400640 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c657cedf50 0x55c657cea390 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c99e16222d203f8 server_cookie=e8af9ffd21a676f2 in_seq=0 out_seq=0 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac43e00640 1 -- 172.21.15.18:0/2436907439 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c657bda210 con 0x55c657cedf50 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac43e00640 10 client.25394 handle_command_reply: tid=0 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 client.25394 shutdown 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 -- 172.21.15.18:0/2436907439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 msgr2=0x55c657c79510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c79510 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac540bd260 tx=0x7fac540ce320 comp rx=0 tx=0).stop 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 -- 172.21.15.18:0/2436907439 shutdown_connections 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c657cedf50 0x55c657cea390 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c657c72000 0x55c657c7ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c657c6ce90 0x55c657c79a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac43e00640 0 client.25394 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 --2- 172.21.15.18:0/2436907439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c657c6c4a0 0x55c657c79510 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 -- 172.21.15.18:0/2436907439 >> 172.21.15.18:0/2436907439 conn(0x55c657c67a10 msgr2=0x55c657c62a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 -- 172.21.15.18:0/2436907439 shutdown_connections 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 1 -- 172.21.15.18:0/2436907439 wait complete. 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.739+0000 7fac6bba3740 20 client.25394 trim_cache size 0 max 16384 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:04:32.745 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 1, 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24566, 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 155.59546461299999, 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 24, 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 93, 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 201.306710192 2024-04-16T17:04:32.746 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:04:32.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 -- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fac480a3bb0 msgr2=0x7fac480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fac480a3bb0 0x7fac480a6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55c657c6d6f0 tx=0x7fac5c010040 comp rx=0 tx=0).stop 2024-04-16T17:04:32.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 -- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac641028e0 msgr2=0x7fac64102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac641028e0 0x7fac64102250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac5400e9f0 tx=0x7fac5400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 -- 172.21.15.18:0/2191056554 shutdown_connections 2024-04-16T17:04:32.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac64107b40 0x7fac64100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac64105210 0x7fac64100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fac480a3bb0 0x7fac480a6070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 --2- 172.21.15.18:0/2191056554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac641028e0 0x7fac64102250 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 -- 172.21.15.18:0/2191056554 >> 172.21.15.18:0/2191056554 conn(0x7fac640fc2f0 msgr2=0x7fac64103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 -- 172.21.15.18:0/2191056554 shutdown_connections 2024-04-16T17:04:32.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.740+0000 7fac63200640 1 -- 172.21.15.18:0/2191056554 wait complete. 2024-04-16T17:04:32.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:32 smithi043 ceph-mon[36901]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:32.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:32 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3873790501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:32.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:32 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4234834597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:32.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T17:04:32.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.986+0000 7f6277000640 1 -- 172.21.15.18:0/2239859026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 msgr2=0x7f62701088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.986+0000 7f6277000640 1 --2- 172.21.15.18:0/2239859026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 0x7f62701088a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6264009900 tx=0x7f626402bc40 comp rx=0 tx=0).stop 2024-04-16T17:04:32.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.987+0000 7f6277000640 1 -- 172.21.15.18:0/2239859026 shutdown_connections 2024-04-16T17:04:32.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.987+0000 7f6277000640 1 --2- 172.21.15.18:0/2239859026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f627010a030 0x7f62701108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.987+0000 7f6277000640 1 --2- 172.21.15.18:0/2239859026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6270109670 0x7f6270109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.987+0000 7f6277000640 1 --2- 172.21.15.18:0/2239859026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 0x7f62701088a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.987+0000 7f6277000640 1 -- 172.21.15.18:0/2239859026 >> 172.21.15.18:0/2239859026 conn(0x7f6270077440 msgr2=0x7f6270079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.987+0000 7f6277000640 1 -- 172.21.15.18:0/2239859026 shutdown_connections 2024-04-16T17:04:32.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.987+0000 7f6277000640 1 -- 172.21.15.18:0/2239859026 wait complete. 2024-04-16T17:04:32.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.988+0000 7f6277000640 1 Processor -- start 2024-04-16T17:04:32.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.988+0000 7f6277000640 1 -- start start 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6277000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 0x7f627019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6277000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6270109670 0x7f627019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6277000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f627010a030 0x7f62701a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6277000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6270114160 con 0x7f627010a030 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6277000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6270113fe0 con 0x7f6270108480 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6277000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62701142e0 con 0x7f6270109670 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6275c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 0x7f627019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6275c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 0x7f627019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45414/0 (socket says 172.21.15.18:45414) 2024-04-16T17:04:32.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6275c00640 1 -- 172.21.15.18:0/1504365477 learned_addr learned my addr 172.21.15.18:0/1504365477 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6276600640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f627010a030 0x7f62701a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.989+0000 7f6275200640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6270109670 0x7f627019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f6275c00640 1 -- 172.21.15.18:0/1504365477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6270109670 msgr2=0x7f627019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f6275c00640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6270109670 0x7f627019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f6275c00640 1 -- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f627010a030 msgr2=0x7f62701a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f6275c00640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f627010a030 0x7f62701a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f6275c00640 1 -- 172.21.15.18:0/1504365477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62701a6430 con 0x7f6270108480 2024-04-16T17:04:32.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f6275c00640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 0x7f627019e810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f626402efd0 tx=0x7f626402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f626e000640 1 -- 172.21.15.18:0/1504365477 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264041070 con 0x7f6270108480 2024-04-16T17:04:32.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f6277000640 1 -- 172.21.15.18:0/1504365477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62701a66c0 con 0x7f6270108480 2024-04-16T17:04:32.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f626e000640 1 -- 172.21.15.18:0/1504365477 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264004450 con 0x7f6270108480 2024-04-16T17:04:32.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f626e000640 1 -- 172.21.15.18:0/1504365477 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264003d90 con 0x7f6270108480 2024-04-16T17:04:32.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.990+0000 7f6277000640 1 -- 172.21.15.18:0/1504365477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62701a6bd0 con 0x7f6270108480 2024-04-16T17:04:32.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f627b3f0740 1 Processor -- start 2024-04-16T17:04:32.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f627b3f0740 1 -- start start 2024-04-16T17:04:32.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f626e000640 1 -- 172.21.15.18:0/1504365477 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6264044070 con 0x7f6270108480 2024-04-16T17:04:32.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb42e90 0x560fecb47990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb4a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f627b3f0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560fecaa8980 con 0x560fecb42e90 2024-04-16T17:04:32.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f627b3f0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560feca4f220 con 0x560fecb424a0 2024-04-16T17:04:32.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f627b3f0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560fecaa4940 con 0x560fecb48000 2024-04-16T17:04:32.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f6275c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb42e90 0x560fecb47990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f6275200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb4a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f6275200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb4a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39998/0 (socket says 172.21.15.18:39998) 2024-04-16T17:04:32.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.994+0000 7f6275200640 1 -- 172.21.15.18:0/2303640866 learned_addr learned my addr 172.21.15.18:0/2303640866 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:32.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.995+0000 7f6276600640 1 --2- 172.21.15.18:0/2303640866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.995+0000 7f6275200640 1 -- 172.21.15.18:0/2303640866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 msgr2=0x560fecb428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.995+0000 7f6275200640 1 --2- 172.21.15.18:0/2303640866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.995+0000 7f6275200640 1 -- 172.21.15.18:0/2303640866 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb42e90 msgr2=0x560fecb47990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.995+0000 7f6275200640 1 --2- 172.21.15.18:0/2303640866 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb42e90 0x560fecb47990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.995+0000 7f6275200640 1 -- 172.21.15.18:0/2303640866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fec7bb2d0 con 0x560fecb48000 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.995+0000 7f6275200640 1 --2- 172.21.15.18:0/2303640866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb4a490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f625c009ca0 tx=0x7f625c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927d09b0541c8053 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f626e000640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f62580a3bb0 0x7f62580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f6262a00640 1 -- 172.21.15.18:0/2303640866 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c01e070 con 0x560fecb48000 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f6262a00640 1 -- 172.21.15.18:0/2303640866 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c019b10 con 0x560fecb48000 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f6262a00640 1 -- 172.21.15.18:0/2303640866 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c00ea80 con 0x560fecb48000 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f626e000640 1 -- 172.21.15.18:0/1504365477 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f62640f2310 con 0x7f6270108480 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f6276600640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f62580a3bb0 0x7f62580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f627b3f0740 1 -- 172.21.15.18:0/2303640866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 msgr2=0x560fecb4a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f627b3f0740 1 --2- 172.21.15.18:0/2303640866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb4a490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f625c009ca0 tx=0x7f625c011810 comp rx=0 tx=0).stop 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f627b3f0740 1 -- 172.21.15.18:0/2303640866 shutdown_connections 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f627b3f0740 1 --2- 172.21.15.18:0/2303640866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb4a490 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f627b3f0740 1 --2- 172.21.15.18:0/2303640866 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb42e90 0x560fecb47990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f627b3f0740 1 --2- 172.21.15.18:0/2303640866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.996+0000 7f627b3f0740 1 -- 172.21.15.18:0/2303640866 >> 172.21.15.18:0/2303640866 conn(0x560fecb3da10 msgr2=0x560fecb3de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.997+0000 7f627b3f0740 1 -- 172.21.15.18:0/2303640866 shutdown_connections 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.997+0000 7f627b3f0740 1 -- 172.21.15.18:0/2303640866 wait complete. 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.997+0000 7f6276600640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f62580a3bb0 0x7f62580a6070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x560fecb436f0 tx=0x7f6268009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.997+0000 7f627b3f0740 1 -- start start 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.997+0000 7f627b3f0740 1 Processor -- start 2024-04-16T17:04:33.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb424a0 0x560fecb4c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb42e90 0x560fecb4cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb53f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f627b3f0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560fecb4afb0 con 0x560fecb424a0 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f627b3f0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560feca62590 con 0x560fecb42e90 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f627b3f0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560fecb4b130 con 0x560fecb48000 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f627b3f0740 10 client.0 _ll_register_callbacks cb 0x560feca6a050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f6275c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb424a0 0x560fecb4c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f6275200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb42e90 0x560fecb4cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f6275c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb424a0 0x560fecb4c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41044/0 (socket says 172.21.15.18:41044) 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f6275c00640 1 -- 172.21.15.18:0/1524403691 learned_addr learned my addr 172.21.15.18:0/1524403691 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.998+0000 7f6276600640 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb53f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6275200640 1 -- 172.21.15.18:0/1524403691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 msgr2=0x560fecb53f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6275200640 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb53f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6275200640 1 -- 172.21.15.18:0/1524403691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb424a0 msgr2=0x560fecb4c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6275200640 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb424a0 0x560fecb4c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6275200640 1 -- 172.21.15.18:0/1524403691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fec8856b0 con 0x560fecb42e90 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6275c00640 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb424a0 0x560fecb4c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6275200640 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb42e90 0x560fecb4cee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f625c011d10 tx=0x7f625c005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86dd163c1e4b7739 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6257400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6257400640 1 -- 172.21.15.18:0/1524403691 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c027030 con 0x560fecb42e90 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f6257400640 1 -- 172.21.15.18:0/1524403691 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f625c005260 con 0x560fecb42e90 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:32.999+0000 7f627b3f0740 1 -- 172.21.15.18:0/1524403691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f625c019b10 con 0x560fecb42e90 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.000+0000 7f6257400640 1 -- 172.21.15.18:0/1524403691 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f625c031050 con 0x560fecb42e90 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.000+0000 7f627b3f0740 10 client.25406 fetch_fsmap learned FSMap version 24 2024-04-16T17:04:33.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.000+0000 7f627b3f0740 1 -- 172.21.15.18:0/1524403691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560fecb57290 con 0x560fecb42e90 2024-04-16T17:04:33.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.000+0000 7f6257400640 1 -- 172.21.15.18:0/1524403691 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c019b10 con 0x560fecb42e90 2024-04-16T17:04:33.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.000+0000 7f6257400640 1 -- 172.21.15.18:0/1524403691 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f625c004830 con 0x560fecb42e90 2024-04-16T17:04:33.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.001+0000 7f627b3f0740 10 client.25406 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.001+0000 7f627b3f0740 10 client.25406 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.001+0000 7f627b3f0740 20 client.25406 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.001+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560fecb59610 0x560fecb5aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.001+0000 7f627b3f0740 4 client.25406 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.001+0000 7f627b3f0740 1 -- 172.21.15.18:0/1524403691 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fecab63a0 con 0x560fecb59610 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.001+0000 7f6275c00640 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560fecb59610 0x560fecb5aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.002+0000 7f6275c00640 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560fecb59610 0x560fecb5aa80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9f71a0293bf87c1 server_cookie=b28d1196ccca1efd in_seq=0 out_seq=0 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.002+0000 7f6257400640 10 client.25406 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f6257400640 1 -- 172.21.15.18:0/1524403691 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fecab63a0 con 0x560fecb59610 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f6257400640 10 client.25406 handle_command_reply: tid=0 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 client.25406 shutdown 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 -- 172.21.15.18:0/1524403691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb42e90 msgr2=0x560fecb4cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb42e90 0x560fecb4cee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f625c011d10 tx=0x7f625c005db0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 -- 172.21.15.18:0/1524403691 shutdown_connections 2024-04-16T17:04:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560fecb59610 0x560fecb5aa80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb48000 0x560fecb53f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb42e90 0x560fecb4cee0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1524403691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb424a0 0x560fecb4c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f6257400640 0 client.25406 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:33.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 -- 172.21.15.18:0/1524403691 >> 172.21.15.18:0/1524403691 conn(0x560fecb3da10 msgr2=0x560fecb3cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:33.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 -- 172.21.15.18:0/1524403691 shutdown_connections 2024-04-16T17:04:33.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.003+0000 7f627b3f0740 1 -- 172.21.15.18:0/1524403691 wait complete. 2024-04-16T17:04:33.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.004+0000 7f627b3f0740 20 client.25406 trim_cache size 0 max 16384 2024-04-16T17:04:33.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.024+0000 7f627b3f0740 1 Processor -- start 2024-04-16T17:04:33.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.024+0000 7f627b3f0740 1 -- start start 2024-04-16T17:04:33.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb4a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb53ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 0x560fecb43720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f627b3f0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560fecb58dc0 con 0x560fecb48000 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f627b3f0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560fecb59490 con 0x560fecb424a0 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f627b3f0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560fecb59b60 con 0x560fecb42e90 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb4a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb4a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45448/0 (socket says 172.21.15.18:45448) 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275c00640 1 -- 172.21.15.18:0/3778010097 learned_addr learned my addr 172.21.15.18:0/3778010097 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6276600640 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 0x560fecb43720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275200640 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb53ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275c00640 1 -- 172.21.15.18:0/3778010097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 msgr2=0x560fecb53ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:33.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275c00640 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb53ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275c00640 1 -- 172.21.15.18:0/3778010097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 msgr2=0x560fecb43720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:33.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275c00640 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 0x560fecb43720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275c00640 1 -- 172.21.15.18:0/3778010097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fecb4dad0 con 0x560fecb424a0 2024-04-16T17:04:33.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.025+0000 7f6275200640 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb53ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:04:33.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.026+0000 7f6275c00640 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb4a3f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f62640bab90 tx=0x7f62640f5d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd48b3c1eb041205 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.026+0000 7f6262000640 1 -- 172.21.15.18:0/3778010097 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626404b040 con 0x560fecb424a0 2024-04-16T17:04:33.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.026+0000 7f6262000640 1 -- 172.21.15.18:0/3778010097 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62640be8c0 con 0x560fecb424a0 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.026+0000 7f6262000640 1 -- 172.21.15.18:0/3778010097 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62640be1c0 con 0x560fecb424a0 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 -- 172.21.15.18:0/3778010097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 msgr2=0x560fecb4a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb4a3f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f62640bab90 tx=0x7f62640f5d50 comp rx=0 tx=0).stop 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 -- 172.21.15.18:0/3778010097 shutdown_connections 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 0x560fecb43720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb53ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 --2- 172.21.15.18:0/3778010097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb4a3f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 -- 172.21.15.18:0/3778010097 >> 172.21.15.18:0/3778010097 conn(0x560fecb3da10 msgr2=0x560fecb3ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 -- 172.21.15.18:0/3778010097 shutdown_connections 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.027+0000 7f627b3f0740 1 -- 172.21.15.18:0/3778010097 wait complete. 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 1 -- start start 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 1 Processor -- start 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb46490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 0x560fecb52520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560fecb5a900 con 0x560fecb48000 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560fecb5a230 con 0x560fecb424a0 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560fecb5afd0 con 0x560fecb42e90 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.028+0000 7f627b3f0740 10 client.0 _ll_register_callbacks cb 0x7f625c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb46490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb46490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45458/0 (socket says 172.21.15.18:45458) 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 -- 172.21.15.18:0/1934377294 learned_addr learned my addr 172.21.15.18:0/1934377294 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275200640 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6276600640 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 0x560fecb52520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 -- 172.21.15.18:0/1934377294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 msgr2=0x560fecb469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 -- 172.21.15.18:0/1934377294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 msgr2=0x560fecb52520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:33.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 0x560fecb52520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 -- 172.21.15.18:0/1934377294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fecb52b10 con 0x560fecb424a0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6275c00640 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb46490 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f626402c180 tx=0x7f62640bdc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a06ce8361a54873 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.029+0000 7f6257400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.030+0000 7f6257400640 1 -- 172.21.15.18:0/1934377294 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62640a7020 con 0x560fecb424a0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.030+0000 7f627b3f0740 1 -- 172.21.15.18:0/1934377294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62640be8c0 con 0x560fecb424a0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.030+0000 7f6257400640 1 -- 172.21.15.18:0/1934377294 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f62640035f0 con 0x560fecb424a0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.030+0000 7f6257400640 1 -- 172.21.15.18:0/1934377294 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f626404b040 con 0x560fecb424a0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.030+0000 7f627b3f0740 10 client.25412 fetch_fsmap learned FSMap version 24 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.030+0000 7f627b3f0740 1 -- 172.21.15.18:0/1934377294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560fecb53d80 con 0x560fecb424a0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.030+0000 7f6257400640 1 -- 172.21.15.18:0/1934377294 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62640be8c0 con 0x560fecb424a0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.031+0000 7f6257400640 1 -- 172.21.15.18:0/1934377294 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f62640c2160 con 0x560fecb424a0 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.031+0000 7f627b3f0740 10 client.25412 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.031+0000 7f627b3f0740 10 client.25412 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.031+0000 7f627b3f0740 20 client.25412 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.031+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560fecbc38a0 0x560fecbc25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.031+0000 7f627b3f0740 4 client.25412 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:04:33.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.031+0000 7f627b3f0740 1 -- 172.21.15.18:0/1934377294 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "status"}) -- 0x560fecab0210 con 0x560fecbc38a0 2024-04-16T17:04:33.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.031+0000 7f6275200640 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560fecbc38a0 0x560fecbc25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.032+0000 7f6275200640 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560fecbc38a0 0x560fecbc25b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b00a52e42f442b server_cookie=1e1f14629c6aee2b in_seq=0 out_seq=0 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.032+0000 7f6257400640 10 client.25412 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.032+0000 7f6257400640 1 -- 172.21.15.18:0/1934377294 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560fecab0210 con 0x560fecbc38a0 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.032+0000 7f6257400640 10 client.25412 handle_command_reply: tid=0 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.032+0000 7f627b3f0740 1 client.25412 shutdown 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 -- 172.21.15.18:0/1934377294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 msgr2=0x560fecb46490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb46490 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f626402c180 tx=0x7f62640bdc20 comp rx=0 tx=0).stop 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 -- 172.21.15.18:0/1934377294 shutdown_connections 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560fecbc38a0 0x560fecbc25b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560fecb48000 0x560fecb52520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fecb42e90 0x560fecb469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 --2- 172.21.15.18:0/1934377294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fecb424a0 0x560fecb46490 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f6257400640 0 client.25412 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:33.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 -- 172.21.15.18:0/1934377294 >> 172.21.15.18:0/1934377294 conn(0x560fecb3da10 msgr2=0x560fecb43720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 -- 172.21.15.18:0/1934377294 shutdown_connections 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 1 -- 172.21.15.18:0/1934377294 wait complete. 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.033+0000 7f627b3f0740 20 client.25412 trim_cache size 0 max 16384 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_fsid": "872930f8-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout: "whoami": 0, 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 24596, 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout: "want_state": "up:active", 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout: "state": "up:active", 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs", 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout: "rank_uptime": 188.10028744600001, 2024-04-16T17:04:33.038 INFO:teuthology.orchestra.run.smithi018.stdout: "mdsmap_epoch": 24, 2024-04-16T17:04:33.039 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch": 93, 2024-04-16T17:04:33.039 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_epoch_barrier": 89, 2024-04-16T17:04:33.039 INFO:teuthology.orchestra.run.smithi018.stdout: "uptime": 194.251417199 2024-04-16T17:04:33.039 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:04:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 -- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f62580a3bb0 msgr2=0x7f62580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f62580a3bb0 0x7f62580a6070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x560fecb436f0 tx=0x7f6268009500 comp rx=0 tx=0).stop 2024-04-16T17:04:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 -- 172.21.15.18:0/1504365477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 msgr2=0x7f627019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 0x7f627019e810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f626402efd0 tx=0x7f626402c660 comp rx=0 tx=0).stop 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 -- 172.21.15.18:0/1504365477 shutdown_connections 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f62580a3bb0 0x7f62580a6070 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f627010a030 0x7f62701a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6270109670 0x7f627019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 --2- 172.21.15.18:0/1504365477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6270108480 0x7f627019e810 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 -- 172.21.15.18:0/1504365477 >> 172.21.15.18:0/1504365477 conn(0x7f6270077440 msgr2=0x7f627010c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 -- 172.21.15.18:0/1504365477 shutdown_connections 2024-04-16T17:04:33.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.034+0000 7f6277000640 1 -- 172.21.15.18:0/1504365477 wait complete. 2024-04-16T17:04:33.057 INFO:teuthology.run_tasks:Running task sequential... 2024-04-16T17:04:33.066 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-16T17:04:33.067 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-16T17:04:33.067 INFO:teuthology.task.print:Enabling mgr modules 2024-04-16T17:04:33.067 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-16T17:04:33.068 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T17:04:33.068 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:04:33.068 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-16T17:04:33.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.278+0000 7f3979c00640 1 -- 172.21.15.18:0/2451392925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3974107bb0 msgr2=0x7f3974109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.278+0000 7f3979c00640 1 --2- 172.21.15.18:0/2451392925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3974107bb0 0x7f3974109fa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f396c009be0 tx=0x7f396c02bef0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.279+0000 7f3979c00640 1 -- 172.21.15.18:0/2451392925 shutdown_connections 2024-04-16T17:04:33.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.279+0000 7f3979c00640 1 --2- 172.21.15.18:0/2451392925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3974107bb0 0x7f3974109fa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.279+0000 7f3979c00640 1 --2- 172.21.15.18:0/2451392925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3974100490 0x7f3974107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.279+0000 7f3979c00640 1 --2- 172.21.15.18:0/2451392925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39740ffaa0 0x7f39740ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.279+0000 7f3979c00640 1 -- 172.21.15.18:0/2451392925 >> 172.21.15.18:0/2451392925 conn(0x7f39740fb640 msgr2=0x7f39740fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:33.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.279+0000 7f3979c00640 1 -- 172.21.15.18:0/2451392925 shutdown_connections 2024-04-16T17:04:33.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.279+0000 7f3979c00640 1 -- 172.21.15.18:0/2451392925 wait complete. 2024-04-16T17:04:33.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.280+0000 7f3979c00640 1 Processor -- start 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.280+0000 7f3979c00640 1 -- start start 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.280+0000 7f3979c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39740ffaa0 0x7f397419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.280+0000 7f3979c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3974100490 0x7f397419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.280+0000 7f3979c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3974107bb0 0x7f39741a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.280+0000 7f3979c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f397410dae0 con 0x7f39740ffaa0 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.280+0000 7f3979c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f397410d960 con 0x7f3974107bb0 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.280+0000 7f3979c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f397410dc60 con 0x7f3974100490 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f3973e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39740ffaa0 0x7f397419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f396be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3974100490 0x7f397419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f3973e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39740ffaa0 0x7f397419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41084/0 (socket says 172.21.15.18:41084) 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f3973e00640 1 -- 172.21.15.18:0/784253958 learned_addr learned my addr 172.21.15.18:0/784253958 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f3979200640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3974107bb0 0x7f39741a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f396be00640 1 -- 172.21.15.18:0/784253958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3974107bb0 msgr2=0x7f39741a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f396be00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3974107bb0 0x7f39741a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f396be00640 1 -- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39740ffaa0 msgr2=0x7f397419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f396be00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39740ffaa0 0x7f397419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f396be00640 1 -- 172.21.15.18:0/784253958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39741a1f70 con 0x7f3974100490 2024-04-16T17:04:33.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.281+0000 7f396be00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3974100490 0x7f397419a8e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f395c005fd0 tx=0x7f395c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.282+0000 7f3971600640 1 -- 172.21.15.18:0/784253958 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c00cd00 con 0x7f3974100490 2024-04-16T17:04:33.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.282+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39741a2260 con 0x7f3974100490 2024-04-16T17:04:33.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.282+0000 7f3971600640 1 -- 172.21.15.18:0/784253958 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f395c0035f0 con 0x7f3974100490 2024-04-16T17:04:33.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.282+0000 7f3971600640 1 -- 172.21.15.18:0/784253958 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c010690 con 0x7f3974100490 2024-04-16T17:04:33.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.282+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39741a2800 con 0x7f3974100490 2024-04-16T17:04:33.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.283+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39741a2ac0 con 0x7f3974100490 2024-04-16T17:04:33.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.285+0000 7f3971600640 1 -- 172.21.15.18:0/784253958 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f395c0037a0 con 0x7f3974100490 2024-04-16T17:04:33.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.287+0000 7f3971600640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f39580a3b90 0x7f39580a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.287+0000 7f3971600640 1 -- 172.21.15.18:0/784253958 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f395c0d0b30 con 0x7f3974100490 2024-04-16T17:04:33.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.287+0000 7f3973e00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f39580a3b90 0x7f39580a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.288+0000 7f3973e00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f39580a3b90 0x7f39580a6050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3964009c90 tx=0x7f3964005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.289+0000 7f3971600640 1 -- 172.21.15.18:0/784253958 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f395c009510 con 0x7f3974100490 2024-04-16T17:04:33.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:33.550+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f39741a2c90 con 0x7f3974100490 2024-04-16T17:04:34.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.441+0000 7f3971600640 1 -- 172.21.15.18:0/784253958 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f395c0990e0 con 0x7f3974100490 2024-04-16T17:04:34.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.444+0000 7f3971600640 1 -- 172.21.15.18:0/784253958 <== mon.1 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f395c09d170 con 0x7f3974100490 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f39580a3b90 msgr2=0x7f39580a6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f39580a3b90 0x7f39580a6050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3964009c90 tx=0x7f3964005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3974100490 msgr2=0x7f397419a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3974100490 0x7f397419a8e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f395c005fd0 tx=0x7f395c00ee50 comp rx=0 tx=0).stop 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 shutdown_connections 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3974107bb0 0x7f39741a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3974100490 0x7f397419a8e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f39580a3b90 0x7f39580a6050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 --2- 172.21.15.18:0/784253958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39740ffaa0 0x7f397419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 >> 172.21.15.18:0/784253958 conn(0x7f39740fb640 msgr2=0x7f39740fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:34.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 shutdown_connections 2024-04-16T17:04:34.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.448+0000 7f3979c00640 1 -- 172.21.15.18:0/784253958 wait complete. 2024-04-16T17:04:34.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-16T17:04:34.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 -- 172.21.15.18:0/977720273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 msgr2=0x7f00b410a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:34.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 --2- 172.21.15.18:0/977720273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 0x7f00b410a2d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009be0 tx=0x7f00ac02bef0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 -- 172.21.15.18:0/977720273 shutdown_connections 2024-04-16T17:04:34.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 --2- 172.21.15.18:0/977720273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 0x7f00b410a2d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 --2- 172.21.15.18:0/977720273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b4102ff0 0x7f00b4103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 --2- 172.21.15.18:0/977720273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00b4101ea0 0x7f00b41022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 -- 172.21.15.18:0/977720273 >> 172.21.15.18:0/977720273 conn(0x7f00b40fd410 msgr2=0x7f00b40ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:34.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 -- 172.21.15.18:0/977720273 shutdown_connections 2024-04-16T17:04:34.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.661+0000 7f00b3200640 1 -- 172.21.15.18:0/977720273 wait complete. 2024-04-16T17:04:34.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.662+0000 7f00b3200640 1 Processor -- start 2024-04-16T17:04:34.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.662+0000 7f00b3200640 1 -- start start 2024-04-16T17:04:34.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b3200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b4101ea0 0x7f00b4075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00b4102ff0 0x7f00b40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 0x7f00b4074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b3200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f00b410db80 con 0x7f00b4101ea0 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00b410da00 con 0x7f00b4102ff0 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00b410dd00 con 0x7f00b41039b0 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00b4102ff0 0x7f00b40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b4101ea0 0x7f00b4075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 0x7f00b4074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00b4102ff0 0x7f00b40743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45500/0 (socket says 172.21.15.18:45500) 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.663+0000 7f00b1400640 1 -- 172.21.15.18:0/2457248980 learned_addr learned my addr 172.21.15.18:0/2457248980 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b1400640 1 -- 172.21.15.18:0/2457248980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 msgr2=0x7f00b4074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b1400640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 0x7f00b4074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b1400640 1 -- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b4101ea0 msgr2=0x7f00b4075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b1400640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b4101ea0 0x7f00b4075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b1400640 1 -- 172.21.15.18:0/2457248980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b4074e70 con 0x7f00b4102ff0 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b2800640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 0x7f00b4074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b1400640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00b4102ff0 0x7f00b40743f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e9c0 tx=0x7f00a000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00aaa00640 1 -- 172.21.15.18:0/2457248980 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a000ccc0 con 0x7f00b4102ff0 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b4075160 con 0x7f00b4102ff0 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00aaa00640 1 -- 172.21.15.18:0/2457248980 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a00035f0 con 0x7f00b4102ff0 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00aaa00640 1 -- 172.21.15.18:0/2457248980 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a000ce60 con 0x7f00b4102ff0 2024-04-16T17:04:34.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.664+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b41a6be0 con 0x7f00b4102ff0 2024-04-16T17:04:34.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.665+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b40688a0 con 0x7f00b4102ff0 2024-04-16T17:04:34.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.668+0000 7f00aaa00640 1 -- 172.21.15.18:0/2457248980 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f00a00037a0 con 0x7f00b4102ff0 2024-04-16T17:04:34.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.670+0000 7f00aaa00640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00980a3c60 0x7f00980a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:34.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.670+0000 7f00b1e00640 1 -- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00980a3c60 msgr2=0x7f00980a6120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:34.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.670+0000 7f00aaa00640 1 -- 172.21.15.18:0/2457248980 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f00a00d0d60 con 0x7f00b4102ff0 2024-04-16T17:04:34.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.670+0000 7f00b1e00640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00980a3c60 0x7f00980a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:04:34.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.671+0000 7f00aaa00640 1 -- 172.21.15.18:0/2457248980 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00a0043e20 con 0x7f00b4102ff0 2024-04-16T17:04:34.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:34 smithi022 ceph-mon[36023]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:34.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:34 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/784253958' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:04:34.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:34 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:04:34.675 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:34 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: ignoring --setuser ceph since I am not root 2024-04-16T17:04:34.675 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:34 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: ignoring --setgroup ceph since I am not root 2024-04-16T17:04:34.675 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:34 smithi022 ceph-mgr[37407]: -- 172.21.15.22:0/3957766495 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0fdbfc5a0 con 0x55e0fdbd0800 2024-04-16T17:04:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:34 smithi018 ceph-mon[31776]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:34 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/784253958' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:04:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:34 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:04:34.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: ignoring --setuser ceph since I am not root 2024-04-16T17:04:34.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: ignoring --setgroup ceph since I am not root 2024-04-16T17:04:34.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:34 smithi043 ceph-mon[36901]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:34.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:34 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/784253958' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:04:34.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:34 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:04:34.753 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: ignoring --setuser ceph since I am not root 2024-04-16T17:04:34.753 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: ignoring --setgroup ceph since I am not root 2024-04-16T17:04:34.753 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:34 smithi043 ceph-mgr[38772]: -- 172.21.15.43:0/474340405 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f4b9825a0 con 0x560f4b956c00 2024-04-16T17:04:34.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.871+0000 7f00b1e00640 1 -- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00980a3c60 msgr2=0x7f00980a6120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:34.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.871+0000 7f00b1e00640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00980a3c60 0x7f00980a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:04:34.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.881+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f00b4109150 con 0x7f00b4102ff0 2024-04-16T17:04:34.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.887+0000 7f00aaa00640 1 -- 172.21.15.18:0/2457248980 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f00a009d310 con 0x7f00b4102ff0 2024-04-16T17:04:34.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00980a3c60 msgr2=0x7f00980a6120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:04:34.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00980a3c60 0x7f00980a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00b4102ff0 msgr2=0x7f00b40743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00b4102ff0 0x7f00b40743f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e9c0 tx=0x7f00a000edc0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 shutdown_connections 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b41039b0 0x7f00b4074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00b4102ff0 0x7f00b40743f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f00980a3c60 0x7f00980a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 --2- 172.21.15.18:0/2457248980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b4101ea0 0x7f00b4075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 >> 172.21.15.18:0/2457248980 conn(0x7f00b40fd410 msgr2=0x7f00b4106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 shutdown_connections 2024-04-16T17:04:34.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:34.890+0000 7f00b3200640 1 -- 172.21.15.18:0/2457248980 wait complete. 2024-04-16T17:04:34.912 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-16T17:04:35.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 -- 172.21.15.18:0/2762431017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec102070 msgr2=0x7f18ec102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2762431017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec102070 0x7f18ec102490 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009900 tx=0x7f18dc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 -- 172.21.15.18:0/2762431017 shutdown_connections 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2762431017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18ec103c20 0x7f18ec10a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2762431017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ec103260 0x7f18ec1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2762431017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec102070 0x7f18ec102490 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 -- 172.21.15.18:0/2762431017 >> 172.21.15.18:0/2762431017 conn(0x7f18ec0fd620 msgr2=0x7f18ec0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 -- 172.21.15.18:0/2762431017 shutdown_connections 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.104+0000 7f18f2e00640 1 -- 172.21.15.18:0/2762431017 wait complete. 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.105+0000 7f18f2e00640 1 Processor -- start 2024-04-16T17:04:35.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.105+0000 7f18f2e00640 1 -- start start 2024-04-16T17:04:35.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18f2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18ec102070 0x7f18ec075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18f2e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec103260 0x7f18ec0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18f2e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ec103c20 0x7f18ec074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18f2e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec10dd10 con 0x7f18ec103c20 2024-04-16T17:04:35.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18f2e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec10db90 con 0x7f18ec103260 2024-04-16T17:04:35.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18f2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec10de90 con 0x7f18ec102070 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec103260 0x7f18ec0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18ec102070 0x7f18ec075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec103260 0x7f18ec0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45508/0 (socket says 172.21.15.18:45508) 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18ec102070 0x7f18ec075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40102/0 (socket says 172.21.15.18:40102) 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18e3e00640 1 -- 172.21.15.18:0/2904688660 learned_addr learned my addr 172.21.15.18:0/2904688660 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18f0e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ec103c20 0x7f18ec074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18ebe00640 1 -- 172.21.15.18:0/2904688660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec103260 msgr2=0x7f18ec0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18ebe00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec103260 0x7f18ec0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18ebe00640 1 -- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ec103c20 msgr2=0x7f18ec074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.106+0000 7f18ebe00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ec103c20 0x7f18ec074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.107+0000 7f18ebe00640 1 -- 172.21.15.18:0/2904688660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec0751c0 con 0x7f18ec102070 2024-04-16T17:04:35.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.107+0000 7f18ebe00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18ec102070 0x7f18ec075db0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009230 tx=0x7f18dc02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:35.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.107+0000 7f18e9600640 1 -- 172.21.15.18:0/2904688660 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc043020 con 0x7f18ec102070 2024-04-16T17:04:35.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.107+0000 7f18e9600640 1 -- 172.21.15.18:0/2904688660 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18dc0054e0 con 0x7f18ec102070 2024-04-16T17:04:35.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.107+0000 7f18e9600640 1 -- 172.21.15.18:0/2904688660 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc003b60 con 0x7f18ec102070 2024-04-16T17:04:35.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.107+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec1a2470 con 0x7f18ec102070 2024-04-16T17:04:35.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.107+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1a2940 con 0x7f18ec102070 2024-04-16T17:04:35.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.108+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec109330 con 0x7f18ec102070 2024-04-16T17:04:35.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.110+0000 7f18e9600640 1 -- 172.21.15.18:0/2904688660 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f18dc046020 con 0x7f18ec102070 2024-04-16T17:04:35.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.111+0000 7f18e9600640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f18d00a3bf0 0x7f18d00a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.112+0000 7f18e3e00640 1 -- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f18d00a3bf0 msgr2=0x7f18d00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:35.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.112+0000 7f18e9600640 1 -- 172.21.15.18:0/2904688660 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f18dc0f2800 con 0x7f18ec102070 2024-04-16T17:04:35.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.112+0000 7f18e3e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f18d00a3bf0 0x7f18d00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:04:35.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.116+0000 7f18e9600640 1 -- 172.21.15.18:0/2904688660 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18dc04d020 con 0x7f18ec102070 2024-04-16T17:04:35.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.313+0000 7f18e3e00640 1 -- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f18d00a3bf0 msgr2=0x7f18d00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:35.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.313+0000 7f18e3e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f18d00a3bf0 0x7f18d00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:04:35.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.316+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f18ec02d620 con 0x7f18ec102070 2024-04-16T17:04:35.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.322+0000 7f18e9600640 1 -- 172.21.15.18:0/2904688660 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f18dc0bedb0 con 0x7f18ec102070 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f18d00a3bf0 msgr2=0x7f18d00a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f18d00a3bf0 0x7f18d00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18ec102070 msgr2=0x7f18ec075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18ec102070 0x7f18ec075db0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009230 tx=0x7f18dc02c5d0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 shutdown_connections 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f18d00a3bf0 0x7f18d00a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18ec103c20 0x7f18ec074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18ec103260 0x7f18ec0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 --2- 172.21.15.18:0/2904688660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18ec102070 0x7f18ec075db0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 >> 172.21.15.18:0/2904688660 conn(0x7f18ec0fd620 msgr2=0x7f18ec106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 shutdown_connections 2024-04-16T17:04:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:35.325+0000 7f18f2e00640 1 -- 172.21.15.18:0/2904688660 wait complete. 2024-04-16T17:04:35.345 INFO:teuthology.run_tasks:Running task kclient... 2024-04-16T17:04:35.361 INFO:tasks.kclient:Mounting kernel clients... 2024-04-16T17:04:35.361 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-16T17:04:35.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:35.362 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-04-16T17:04:35.410 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-16T17:04:35.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:35.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:04:35.424 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:35.022+0000 7f7f9cbd6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T17:04:35.424 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:35.120+0000 7f7f9cbd6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T17:04:35.428 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:35.021+0000 7f459579a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T17:04:35.428 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:35.118+0000 7f459579a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T17:04:35.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:35.497 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-04-16T17:04:35.564 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:35 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T17:04:35.564 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:35 smithi018 ceph-mon[31776]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-16T17:04:35.564 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:35 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' 2024-04-16T17:04:35.564 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:35 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' 2024-04-16T17:04:35.564 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:35 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:35.298+0000 7f3572ad7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T17:04:35.564 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:35 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:35.426+0000 7f3572ad7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T17:04:35.610 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-16T17:04:35.610 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T17:04:35.610 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T17:04:35.610 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph fs ls 2024-04-16T17:04:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:35 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T17:04:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:35 smithi043 ceph-mon[36901]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-16T17:04:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:35 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' 2024-04-16T17:04:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:35 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' 2024-04-16T17:04:35.795 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:35 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T17:04:35.795 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:35 smithi022 ceph-mon[36023]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-16T17:04:35.795 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:35 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' 2024-04-16T17:04:35.795 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:35 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' 2024-04-16T17:04:36.047 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:36.129 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:35.786+0000 7f459579a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T17:04:36.130 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:35.881+0000 7f459579a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T17:04:36.130 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:36.030+0000 7f459579a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T17:04:36.130 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:36.128+0000 7f459579a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T17:04:36.139 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:35.794+0000 7f7f9cbd6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T17:04:36.139 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:35.890+0000 7f7f9cbd6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T17:04:36.139 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:36.038+0000 7f7f9cbd6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T17:04:36.140 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:36.138+0000 7f7f9cbd6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T17:04:36.424 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:36.225+0000 7f7f9cbd6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T17:04:36.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:36.299+0000 7f3572ad7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T17:04:36.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:36.434+0000 7f3572ad7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T17:04:36.490 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:36.215+0000 7f459579a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T17:04:36.490 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:36.489+0000 7f459579a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T17:04:36.752 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:36.605+0000 7f459579a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T17:04:36.753 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:36.694+0000 7f459579a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T17:04:36.807 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:36.501+0000 7f7f9cbd6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T17:04:36.807 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:36.617+0000 7f7f9cbd6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T17:04:36.807 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:36.707+0000 7f7f9cbd6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T17:04:36.895 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:36.641+0000 7f3572ad7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T17:04:36.895 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:36.775+0000 7f3572ad7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T17:04:36.895 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:36.893+0000 7f3572ad7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T17:04:37.174 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:36.806+0000 7f7f9cbd6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T17:04:37.175 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:36.988+0000 7f7f9cbd6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T17:04:37.252 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:36.792+0000 7f459579a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T17:04:37.252 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:36.974+0000 7f459579a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T17:04:37.499 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:37.244+0000 7f3572ad7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T17:04:37.499 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:37.390+0000 7f3572ad7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T17:04:37.798 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:37.511+0000 7f3572ad7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T17:04:37.798 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:37.642+0000 7f3572ad7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T17:04:38.127 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:37.873+0000 7f3572ad7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T17:04:38.252 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:37.984+0000 7f459579a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T17:04:38.252 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:38.118+0000 7f459579a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T17:04:38.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 -- 172.21.15.18:0/1063679582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9554103be0 msgr2=0x7f9554103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 --2- 172.21.15.18:0/1063679582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9554103be0 0x7f9554103fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9544009990 tx=0x7f9544009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 -- 172.21.15.18:0/1063679582 shutdown_connections 2024-04-16T17:04:38.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 --2- 172.21.15.18:0/1063679582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f955410d010 0x7f955410f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 --2- 172.21.15.18:0/1063679582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9554103be0 0x7f9554103fc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 --2- 172.21.15.18:0/1063679582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9554104500 0x7f955410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 -- 172.21.15.18:0/1063679582 >> 172.21.15.18:0/1063679582 conn(0x7f95540fd410 msgr2=0x7f95540ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:38.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 -- 172.21.15.18:0/1063679582 shutdown_connections 2024-04-16T17:04:38.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.287+0000 7f9558c00700 1 -- 172.21.15.18:0/1063679582 wait complete. 2024-04-16T17:04:38.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.288+0000 7f9558c00700 1 Processor -- start 2024-04-16T17:04:38.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.288+0000 7f9558c00700 1 -- start start 2024-04-16T17:04:38.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9558c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9554103be0 0x7f9554101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9558c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9554104500 0x7f9554101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9558c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f955410d010 0x7f95541022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9558c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9554112d70 con 0x7f9554104500 2024-04-16T17:04:38.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9558c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9554112bf0 con 0x7f9554103be0 2024-04-16T17:04:38.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9558c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9554112ef0 con 0x7f955410d010 2024-04-16T17:04:38.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9553400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9554103be0 0x7f9554101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:38.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9553400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9554103be0 0x7f9554101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56694/0 (socket says 172.21.15.18:56694) 2024-04-16T17:04:38.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9553400700 1 -- 172.21.15.18:0/3727508346 learned_addr learned my addr 172.21.15.18:0/3727508346 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9552a00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9554104500 0x7f9554101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.289+0000 7f9553e00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f955410d010 0x7f95541022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f9553400700 1 -- 172.21.15.18:0/3727508346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f955410d010 msgr2=0x7f95541022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f9553400700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f955410d010 0x7f95541022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f9553400700 1 -- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9554104500 msgr2=0x7f9554101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f9553400700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9554104500 0x7f9554101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f9553400700 1 -- 172.21.15.18:0/3727508346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95541a7700 con 0x7f9554103be0 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f9553400700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9554103be0 0x7f9554101840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f954400bae0 tx=0x7f954400bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f954be00700 1 -- 172.21.15.18:0/3727508346 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544026070 con 0x7f9554103be0 2024-04-16T17:04:38.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f954be00700 1 -- 172.21.15.18:0/3727508346 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9544003d70 con 0x7f9554103be0 2024-04-16T17:04:38.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f954be00700 1 -- 172.21.15.18:0/3727508346 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544021410 con 0x7f9554103be0 2024-04-16T17:04:38.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.290+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95541a7990 con 0x7f9554103be0 2024-04-16T17:04:38.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.291+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95541a7e70 con 0x7f9554103be0 2024-04-16T17:04:38.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.292+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95540697e0 con 0x7f9554103be0 2024-04-16T17:04:38.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.293+0000 7f954be00700 1 -- 172.21.15.18:0/3727508346 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f9544029070 con 0x7f9554103be0 2024-04-16T17:04:38.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.295+0000 7f954be00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95380a3c90 0x7f95380a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.295+0000 7f954be00700 1 -- 172.21.15.18:0/3727508346 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f95440d15e0 con 0x7f9554103be0 2024-04-16T17:04:38.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.295+0000 7f9552a00700 1 -- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95380a3c90 msgr2=0x7f95380a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:38.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.295+0000 7f9552a00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95380a3c90 0x7f95380a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:04:38.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.299+0000 7f954be00700 1 -- 172.21.15.18:0/3727508346 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f954405dff0 con 0x7f9554103be0 2024-04-16T17:04:38.424 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:38.010+0000 7f7f9cbd6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T17:04:38.425 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:38.146+0000 7f7f9cbd6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T17:04:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.496+0000 7f9552a00700 1 -- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95380a3c90 msgr2=0x7f95380a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:38.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.496+0000 7f9552a00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95380a3c90 0x7f95380a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:04:38.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.619+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f955402dfe0 con 0x7f9554103be0 2024-04-16T17:04:38.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.620+0000 7f954be00700 1 -- 172.21.15.18:0/3727508346 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f9544013070 con 0x7f9554103be0 2024-04-16T17:04:38.622 INFO:teuthology.orchestra.run.smithi018.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T17:04:38.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95380a3c90 msgr2=0x7f95380a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:04:38.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95380a3c90 0x7f95380a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9554103be0 msgr2=0x7f9554101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9554103be0 0x7f9554101840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f954400bae0 tx=0x7f954400bb10 comp rx=0 tx=0).stop 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 shutdown_connections 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7f95380a3c90 0x7f95380a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9554103be0 0x7f9554101840 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f955410d010 0x7f95541022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 --2- 172.21.15.18:0/3727508346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9554104500 0x7f9554101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.623+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 >> 172.21.15.18:0/3727508346 conn(0x7f95540fd410 msgr2=0x7f95540fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.624+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 shutdown_connections 2024-04-16T17:04:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:38.624+0000 7f9558c00700 1 -- 172.21.15.18:0/3727508346 wait complete. 2024-04-16T17:04:38.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:38 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3727508346' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:04:38.753 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:38.464+0000 7f459579a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T17:04:38.753 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:38.557+0000 7f459579a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T17:04:38.791 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:38 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3727508346' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:04:38.791 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:38.492+0000 7f7f9cbd6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T17:04:38.791 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:38.585+0000 7f7f9cbd6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T17:04:38.791 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:38.789+0000 7f7f9cbd6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:04:38.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:38 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3727508346' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:04:39.174 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:38.982+0000 7f7f9cbd6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T17:04:39.249 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:38.761+0000 7f459579a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:04:39.250 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:38.952+0000 7f459579a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T17:04:39.502 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-16T17:04:39.502 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T17:04:39.502 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi018.front.sepia.ceph.com 2024-04-16T17:04:39.502 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-16T17:04:39.502 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T17:04:39.503 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-16T17:04:39.503 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T17:04:39.503 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T17:04:39.503 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-16T17:04:39.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:39.503 DEBUG:teuthology.orchestra.run.smithi018:> ip addr 2024-04-16T17:04:39.504 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:39.248+0000 7f459579a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:04:39.547 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:39.278+0000 7f7f9cbd6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:04:39.555 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T17:04:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T17:04:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T17:04:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:04:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2024-04-16T17:04:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:04:39.556 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:04:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:04:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f0 2024-04-16T17:04:39.556 INFO:teuthology.orchestra.run.smithi018.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:04:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:04:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f1 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f0 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 41853sec preferred_lft 41853sec 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link noprefixroute 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:04:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f1 2024-04-16T17:04:39.559 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T17:04:39.559 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:04:39.559 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:04:39.559 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add name ceph-brx type bridge 2024-04-16T17:04:39.559 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr flush dev ceph-brx 2024-04-16T17:04:39.559 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set ceph-brx up 2024-04-16T17:04:39.559 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T17:04:39.559 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:04:39.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:04:39.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:39.331+0000 7f3572ad7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T17:04:39.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:39.519+0000 7f3572ad7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T17:04:39.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:39.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:39.857 DEBUG:teuthology.orchestra.run.smithi018:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T17:04:39.924 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:39.546+0000 7f7f9cbd6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T17:04:39.924 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:39.794+0000 7f7f9cbd6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T17:04:39.933 INFO:teuthology.orchestra.run.smithi018.stdout:1 2024-04-16T17:04:39.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:39.936 DEBUG:teuthology.orchestra.run.smithi018:> route 2024-04-16T17:04:39.985 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2024-04-16T17:04:39.985 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T17:04:39.985 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-16T17:04:39.985 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-16T17:04:39.985 INFO:teuthology.orchestra.run.smithi018.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T17:04:39.985 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:04:39.986 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:04:39.986 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-16T17:04:39.986 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-16T17:04:39.986 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-16T17:04:39.986 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:04:40.002 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:39.517+0000 7f459579a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T17:04:40.002 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:39.762+0000 7f459579a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T17:04:40.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:04:40.252 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:39.969+0000 7f3572ad7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T17:04:40.252 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:40.097+0000 7f3572ad7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T17:04:40.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:40.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:40.273 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-04-16T17:04:40.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:40.316 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list-id 2024-04-16T17:04:40.404 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:04:40.404 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:04:40.404 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T17:04:40.404 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-16T17:04:40.404 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:04:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:04:40.570 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:40.378+0000 7f3572ad7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:04:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:40.638 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-16T17:04:40.638 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:04:40.638 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:04:40.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-16T17:04:40.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T17:04:40.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-16T17:04:40.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-16T17:04:40.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-16T17:04:40.638 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:04:40.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:04:40.843 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:40.642+0000 7f3572ad7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T17:04:40.924 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:40 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:40.636+0000 7f7f9cbd6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T17:04:40.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:40.939 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:04:40.939 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-16T17:04:40.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set brx.0 up 2024-04-16T17:04:40.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T17:04:40.939 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-16T17:04:41.002 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:40.600+0000 7f459579a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T17:04:41.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:04:41.120 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:41.055+0000 7f3572ad7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:04:41.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:41.172 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T17:04:41.173 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-16T17:04:41.221 INFO:teuthology.orchestra.run.smithi018.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-16T17:04:41.221 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T17:04:41.221 DEBUG:teuthology.orchestra.run.smithi018:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-16T17:04:41.314 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-16T17:04:41.314 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-16T17:04:41.314 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/6f0bb715-aa68-4ec4-ae07-0cc7013aa45c 2024-04-16T17:04:41.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:41.314 DEBUG:teuthology.orchestra.run.smithi018:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/6f0bb715-aa68-4ec4-ae07-0cc7013aa45c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-16T17:04:41.450 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-16T17:04:41.450 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-16T17:04:41.450 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-04-16T17:04:41.502 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:41.198+0000 7f459579a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T17:04:41.502 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:41.299+0000 7f459579a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T17:04:41.565 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:41.236+0000 7f7f9cbd6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T17:04:41.565 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:41.338+0000 7f7f9cbd6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T17:04:41.565 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y[37403]: 2024-04-16T17:04:41.563+0000 7f7f9cbd6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T17:04:41.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:41.395+0000 7f3572ad7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T17:04:41.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: Standby manager daemon z restarted 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: Standby manager daemon z started 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: Standby manager daemon y restarted 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: Standby manager daemon y started 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T17:04:41.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-mon[31776]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: Standby manager daemon z restarted 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: Standby manager daemon z started 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: Standby manager daemon y restarted 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: Standby manager daemon y started 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T17:04:41.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:41 smithi022 ceph-mon[36023]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:04:42.002 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z[38768]: 2024-04-16T17:04:41.525+0000 7f459579a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T17:04:42.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: Standby manager daemon z restarted 2024-04-16T17:04:42.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: Standby manager daemon z started 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.43:0/3757451390' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: Standby manager daemon y restarted 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: Standby manager daemon y started 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T17:04:42.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:41 smithi043 ceph-mon[36901]: from='mgr.? 172.21.15.22:0/725731600' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:04:42.073 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:41.761+0000 7f3572ad7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T17:04:42.267 INFO:teuthology.orchestra.run.smithi018.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-16T17:04:42.267 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-16T17:04:42.267 INFO:teuthology.orchestra.run.smithi018.stdout:invalid new device string format 2024-04-16T17:04:42.267 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: resolved to: "172.21.15.18:3300,172.21.15.22:3300,172.21.15.43:3300" 2024-04-16T17:04:42.267 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.18:3300,172.21.15.22:3300,172.21.15.43:3300:/volumes/qa/sv_1/6f0bb715-aa68-4ec4-ae07-0cc7013aa45c 2024-04-16T17:04:42.267 INFO:teuthology.orchestra.run.smithi018.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=872930f8-fc11-11ee-bc90-c7b262605968" will pass to kernel 2024-04-16T17:04:42.271 INFO:teuthology.orchestra.run.smithi018.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-16T17:04:42.271 INFO:teuthology.orchestra.run.smithi018.stdout: You just mounted a file system that supports labels which does not 2024-04-16T17:04:42.271 INFO:teuthology.orchestra.run.smithi018.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-16T17:04:42.271 INFO:teuthology.orchestra.run.smithi018.stdout: applications will generate AVC messages and not be allowed access to 2024-04-16T17:04:42.272 INFO:teuthology.orchestra.run.smithi018.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-16T17:04:42.275 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-16T17:04:42.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:42.275 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-16T17:04:42.357 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T17:04:42.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T17:04:42.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:42.464 INFO:teuthology.orchestra.run.smithi018.stdout:client15321 2024-04-16T17:04:42.464 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T17:04:42.464 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:04:42.506 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T17:04:42.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T17:04:42.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:42.659 INFO:teuthology.orchestra.run.smithi018.stdout:client15321 2024-04-16T17:04:42.659 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T17:04:42.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T17:04:42.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:42.772 INFO:teuthology.orchestra.run.smithi018.stdout:client15321 2024-04-16T17:04:42.772 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/872930f8-fc11-11ee-bc90-c7b262605968.client15321/status') 2024-04-16T17:04:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:04:42.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:42 smithi018 ceph-mon[31776]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-16T17:04:42.902 INFO:teuthology.orchestra.run.smithi018.stdout:instance: client.15321 (3)192.168.0.1:0/2082690895 2024-04-16T17:04:42.902 INFO:teuthology.orchestra.run.smithi018.stdout:blocklisted: false 2024-04-16T17:04:42.902 INFO:teuthology.orchestra.run.smithi018.stderr:0+1 records in 2024-04-16T17:04:42.903 INFO:teuthology.orchestra.run.smithi018.stderr:0+1 records out 2024-04-16T17:04:42.903 INFO:teuthology.orchestra.run.smithi018.stderr:70 bytes copied, 6.1112e-05 s, 1.1 MB/s 2024-04-16T17:04:42.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:04:42.915 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T17:04:42.924 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T17:04:42.925 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:04:42.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-16T17:04:42.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:42 smithi022 ceph-mon[36023]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-16T17:04:43.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:42 smithi043 ceph-mon[36901]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-16T17:04:43.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 -- 172.21.15.18:0/1099188442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00103c20 msgr2=0x7fcd0010a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 --2- 172.21.15.18:0/1099188442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00103c20 0x7fcd0010a4b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ca0 tx=0x7fccf802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 -- 172.21.15.18:0/1099188442 shutdown_connections 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 --2- 172.21.15.18:0/1099188442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00103c20 0x7fcd0010a4b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 --2- 172.21.15.18:0/1099188442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd00103260 0x7fcd001036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 --2- 172.21.15.18:0/1099188442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd00102070 0x7fcd00102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 -- 172.21.15.18:0/1099188442 >> 172.21.15.18:0/1099188442 conn(0x7fcd000fd5e0 msgr2=0x7fcd000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 -- 172.21.15.18:0/1099188442 shutdown_connections 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.130+0000 7fcd06400640 1 -- 172.21.15.18:0/1099188442 wait complete. 2024-04-16T17:04:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.131+0000 7fcd06400640 1 Processor -- start 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fcd06400640 1 -- start start 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fcd06400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd00102070 0x7fcd00075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fcd06400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd00103260 0x7fcd000743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fcd06400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00103c20 0x7fcd00074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fcd06400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010dcc0 con 0x7fcd00103260 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fcd06400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010db40 con 0x7fcd00102070 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fcd06400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010de40 con 0x7fcd00103c20 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fccfea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd00103260 0x7fcd000743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fccfea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd00103260 0x7fcd000743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42364/0 (socket says 172.21.15.18:42364) 2024-04-16T17:04:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.132+0000 7fccfea00640 1 -- 172.21.15.18:0/2095708274 learned_addr learned my addr 172.21.15.18:0/2095708274 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccffe00640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00103c20 0x7fcd00074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccff400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd00102070 0x7fcd00075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccfea00640 1 -- 172.21.15.18:0/2095708274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00103c20 msgr2=0x7fcd00074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccfea00640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00103c20 0x7fcd00074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccfea00640 1 -- 172.21.15.18:0/2095708274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd00102070 msgr2=0x7fcd00075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccfea00640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd00102070 0x7fcd00075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccfea00640 1 -- 172.21.15.18:0/2095708274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd000751c0 con 0x7fcd00103260 2024-04-16T17:04:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccfea00640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd00103260 0x7fcd000743f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fccec00ea60 tx=0x7fccec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccf7e00640 1 -- 172.21.15.18:0/2095708274 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec00cde0 con 0x7fcd00103260 2024-04-16T17:04:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd001a23f0 con 0x7fcd00103260 2024-04-16T17:04:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.133+0000 7fccf7e00640 1 -- 172.21.15.18:0/2095708274 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccec0035f0 con 0x7fcd00103260 2024-04-16T17:04:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.134+0000 7fccf7e00640 1 -- 172.21.15.18:0/2095708274 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec010670 con 0x7fcd00103260 2024-04-16T17:04:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.134+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001a2940 con 0x7fcd00103260 2024-04-16T17:04:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.135+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd00109390 con 0x7fcd00103260 2024-04-16T17:04:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.138+0000 7fccf7e00640 1 -- 172.21.15.18:0/2095708274 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fccec0037a0 con 0x7fcd00103260 2024-04-16T17:04:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.139+0000 7fccf7e00640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcce40a3bf0 0x7fcce40a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.139+0000 7fccff400640 1 -- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcce40a3bf0 msgr2=0x7fcce40a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.139+0000 7fccf7e00640 1 -- 172.21.15.18:0/2095708274 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fccec0d0c50 con 0x7fcd00103260 2024-04-16T17:04:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.139+0000 7fccff400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcce40a3bf0 0x7fcce40a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:04:43.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.141+0000 7fccf7e00640 1 -- 172.21.15.18:0/2095708274 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fccec09d290 con 0x7fcd00103260 2024-04-16T17:04:43.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:42.883+0000 7f3572ad7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T17:04:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.340+0000 7fccff400640 1 -- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcce40a3bf0 msgr2=0x7fcce40a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.340+0000 7fccff400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcce40a3bf0 0x7fcce40a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:04:43.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.393+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fcd0002d620 con 0x7fcd00103260 2024-04-16T17:04:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.618+0000 7fccf7e00640 1 -- 172.21.15.18:0/2095708274 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fccec09cb90 con 0x7fcd00103260 2024-04-16T17:04:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcce40a3bf0 msgr2=0x7fcce40a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:04:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcce40a3bf0 0x7fcce40a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd00103260 msgr2=0x7fcd000743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd00103260 0x7fcd000743f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fccec00ea60 tx=0x7fccec00ef20 comp rx=0 tx=0).stop 2024-04-16T17:04:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 shutdown_connections 2024-04-16T17:04:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00103c20 0x7fcd00074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7fcce40a3bf0 0x7fcce40a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd00103260 0x7fcd000743f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.621+0000 7fcd06400640 1 --2- 172.21.15.18:0/2095708274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd00102070 0x7fcd00075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.622+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 >> 172.21.15.18:0/2095708274 conn(0x7fcd000fd5e0 msgr2=0x7fcd00106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.622+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 shutdown_connections 2024-04-16T17:04:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.622+0000 7fcd06400640 1 -- 172.21.15.18:0/2095708274 wait complete. 2024-04-16T17:04:43.642 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-16T17:04:43.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2959875381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbc0101ea0 msgr2=0x7ffbc01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:43.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 --2- 172.21.15.18:0/2959875381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbc0101ea0 0x7ffbc01022c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009860 tx=0x7ffbb402bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:43.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2959875381 shutdown_connections 2024-04-16T17:04:43.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 --2- 172.21.15.18:0/2959875381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbc01039b0 0x7ffbc010a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 --2- 172.21.15.18:0/2959875381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbc0102ff0 0x7ffbc0103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 --2- 172.21.15.18:0/2959875381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbc0101ea0 0x7ffbc01022c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2959875381 >> 172.21.15.18:0/2959875381 conn(0x7ffbc00fd410 msgr2=0x7ffbc00ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:43.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2959875381 shutdown_connections 2024-04-16T17:04:43.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.846+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2959875381 wait complete. 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.847+0000 7ffbc6a00640 1 Processor -- start 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.847+0000 7ffbc6a00640 1 -- start start 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.847+0000 7ffbc6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbc0101ea0 0x7ffbc019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.847+0000 7ffbc6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbc0102ff0 0x7ffbc019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.847+0000 7ffbc6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbc01039b0 0x7ffbc01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.847+0000 7ffbc6a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc010db40 con 0x7ffbc0102ff0 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.847+0000 7ffbc6a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc010d9c0 con 0x7ffbc0101ea0 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.847+0000 7ffbc6a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc010dcc0 con 0x7ffbc01039b0 2024-04-16T17:04:43.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbc0102ff0 0x7ffbc019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbc0101ea0 0x7ffbc019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbc01039b0 0x7ffbc01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbc0102ff0 0x7ffbc019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42378/0 (socket says 172.21.15.18:42378) 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc4c00640 1 -- 172.21.15.18:0/2924010537 learned_addr learned my addr 172.21.15.18:0/2924010537 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc5600640 1 -- 172.21.15.18:0/2924010537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbc01039b0 msgr2=0x7ffbc01a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc5600640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbc01039b0 0x7ffbc01a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc5600640 1 -- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbc0102ff0 msgr2=0x7ffbc019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc5600640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbc0102ff0 0x7ffbc019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc5600640 1 -- 172.21.15.18:0/2924010537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc01a2020 con 0x7ffbc0101ea0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.848+0000 7ffbc5600640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbc0101ea0 0x7ffbc019a4a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffbb402c0b0 tx=0x7ffbb4032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.849+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4041020 con 0x7ffbc0101ea0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.849+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb402c9f0 con 0x7ffbc0101ea0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.849+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc01a22b0 con 0x7ffbc0101ea0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.849+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4039040 con 0x7ffbc0101ea0 2024-04-16T17:04:43.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.849+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc01a27a0 con 0x7ffbc0101ea0 2024-04-16T17:04:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.850+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc01a2a00 con 0x7ffbc0101ea0 2024-04-16T17:04:43.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.852+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7ffbb4044020 con 0x7ffbc0101ea0 2024-04-16T17:04:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.853+0000 7ffbbe000640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ffba80a3be0 0x7ffba80a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.853+0000 7ffbc4c00640 1 -- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ffba80a3be0 msgr2=0x7ffba80a60a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.854+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7ffbb40f20a0 con 0x7ffbc0101ea0 2024-04-16T17:04:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.854+0000 7ffbc4c00640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ffba80a3be0 0x7ffba80a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:04:43.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:43.858+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbb404b020 con 0x7ffbc0101ea0 2024-04-16T17:04:43.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:43 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2095708274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-16T17:04:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:43 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2095708274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-16T17:04:43.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:43.636+0000 7f3572ad7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T17:04:43.946 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:43.777+0000 7f3572ad7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T17:04:44.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:43 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2095708274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-16T17:04:44.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:44.055+0000 7ffbc4c00640 1 -- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ffba80a3be0 msgr2=0x7ffba80a60a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/2462414912 2024-04-16T17:04:44.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:44.055+0000 7ffbc4c00640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ffba80a3be0 0x7ffba80a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:04:44.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:44.101+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 --> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7ffbc00759f0 con 0x7ffba80a3be0 2024-04-16T17:04:44.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:44.113+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7ffbb4032640 con 0x7ffbc0101ea0 2024-04-16T17:04:44.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:44.114+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ffba80a3be0 msgr2=0x7ffba80a60a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:04:44.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:44.114+0000 7ffbbe000640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/2462414912,v1:172.21.15.18:6801/2462414912] conn(0x7ffba80a3be0 0x7ffba80a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:44.409 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.086+0000 7f3572ad7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T17:04:44.409 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.406+0000 7f34c8e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.409 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.406+0000 7f34c8e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.409 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.406+0000 7f34c8e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.409 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.406+0000 7f34c8e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.409 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.406+0000 7f34c8e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.673 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.408+0000 7f34c5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.673 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.408+0000 7f34c5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.673 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.408+0000 7f34c5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.673 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.408+0000 7f34c5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.673 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:44.408+0000 7f34c5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2095708274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: Active manager daemon x restarted 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: Activating manager daemon x 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: mgrmap e21: x(active, starting, since 0.0126825s), standbys: z, y 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:04:44.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T17:04:44.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: Manager daemon x is now available 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:04:44.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:44 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2095708274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: Active manager daemon x restarted 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: Activating manager daemon x 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: mgrmap e21: x(active, starting, since 0.0126825s), standbys: z, y 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:04:44.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T17:04:44.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: Manager daemon x is now available 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:04:44.927 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:04:44.928 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:04:44.928 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:44 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:04:45.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2095708274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-16T17:04:45.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:04:45.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: Active manager daemon x restarted 2024-04-16T17:04:45.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: Activating manager daemon x 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: mgrmap e21: x(active, starting, since 0.0126825s), standbys: z, y 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:04:45.003 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T17:04:45.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T17:04:45.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T17:04:45.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: Manager daemon x is now available 2024-04-16T17:04:45.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:45.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:04:45.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:04:45.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:04:45.005 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:44 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:04:45.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.124+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mon.2 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7ffbb40ef7f0 con 0x7ffbc0101ea0 2024-04-16T17:04:45.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.125+0000 7ffbbe000640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffba8052110 0x7ffba80524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.125+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7ffbb4036ec0 con 0x7ffba8052110 2024-04-16T17:04:45.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.131+0000 7ffbc4c00640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffba8052110 0x7ffba80524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.141+0000 7ffbc4c00640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffba8052110 0x7ffba80524f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffbc019b910 tx=0x7ffbac005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:45.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.168+0000 7ffbbe000640 1 -- 172.21.15.18:0/2924010537 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7ffbb4036ec0 con 0x7ffba8052110 2024-04-16T17:04:45.169 INFO:teuthology.orchestra.run.smithi018.stdout:{} 2024-04-16T17:04:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.170+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffba8052110 msgr2=0x7ffba80524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.170+0000 7ffbc6a00640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffba8052110 0x7ffba80524f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffbc019b910 tx=0x7ffbac005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.170+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbc0101ea0 msgr2=0x7ffbc019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.171+0000 7ffbc6a00640 1 --2- 172.21.15.18:0/2924010537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbc0101ea0 0x7ffbc019a4a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffbb402c0b0 tx=0x7ffbb4032430 comp rx=0 tx=0).stop 2024-04-16T17:04:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.171+0000 7ffbc6000640 1 -- 172.21.15.18:0/2924010537 reap_dead start 2024-04-16T17:04:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.171+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 shutdown_connections 2024-04-16T17:04:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.171+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 >> 172.21.15.18:0/2924010537 conn(0x7ffbc00fd410 msgr2=0x7ffbc0074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.171+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 shutdown_connections 2024-04-16T17:04:45.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.171+0000 7ffbc6a00640 1 -- 172.21.15.18:0/2924010537 wait complete. 2024-04-16T17:04:45.189 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-16T17:04:45.203 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-16T17:04:45.213 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:04:45.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.413+0000 7f896c400640 1 -- 172.21.15.18:0/30893106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8964105e80 msgr2=0x7f896410c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.413+0000 7f896c400640 1 --2- 172.21.15.18:0/30893106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8964105e80 0x7f896410c710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f895c009ca0 tx=0x7f895c02c0f0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.413+0000 7f896c400640 1 -- 172.21.15.18:0/30893106 shutdown_connections 2024-04-16T17:04:45.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.413+0000 7f896c400640 1 --2- 172.21.15.18:0/30893106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8964105e80 0x7f896410c710 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.413+0000 7f896c400640 1 --2- 172.21.15.18:0/30893106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89641054c0 0x7f8964105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.413+0000 7f896c400640 1 --2- 172.21.15.18:0/30893106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89641042d0 0x7f89641046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.414+0000 7f896c400640 1 -- 172.21.15.18:0/30893106 >> 172.21.15.18:0/30893106 conn(0x7f89640ff880 msgr2=0x7f8964101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:45.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.414+0000 7f896c400640 1 -- 172.21.15.18:0/30893106 shutdown_connections 2024-04-16T17:04:45.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.414+0000 7f896c400640 1 -- 172.21.15.18:0/30893106 wait complete. 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896c400640 1 Processor -- start 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896c400640 1 -- start start 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896c400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89641042d0 0x7f8964079db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896c400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89641054c0 0x7f896407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896c400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8964105e80 0x7f8964076920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896c400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f896410fef0 con 0x7f89641054c0 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896c400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f896410fd70 con 0x7f8964105e80 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896c400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8964110070 con 0x7f89641042d0 2024-04-16T17:04:45.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f8969a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89641042d0 0x7f8964079db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f8969a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89641042d0 0x7f8964079db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:33140/0 (socket says 172.21.15.18:33140) 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f8969a00640 1 -- 172.21.15.18:0/1624962429 learned_addr learned my addr 172.21.15.18:0/1624962429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.415+0000 7f896a400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8964105e80 0x7f8964076920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.416+0000 7f8969000640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89641054c0 0x7f896407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.416+0000 7f896a400640 1 -- 172.21.15.18:0/1624962429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89641042d0 msgr2=0x7f8964079db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.416+0000 7f896a400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89641042d0 0x7f8964079db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.416+0000 7f896a400640 1 -- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89641054c0 msgr2=0x7f896407a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.416+0000 7f896a400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89641054c0 0x7f896407a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.416+0000 7f896a400640 1 -- 172.21.15.18:0/1624962429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89640771e0 con 0x7f8964105e80 2024-04-16T17:04:45.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.416+0000 7f8969a00640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89641042d0 0x7f8964079db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:45.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.416+0000 7f896a400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8964105e80 0x7f8964076920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f895c037040 tx=0x7f895c032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:45.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.417+0000 7f8962000640 1 -- 172.21.15.18:0/1624962429 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c041020 con 0x7f8964105e80 2024-04-16T17:04:45.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.417+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89641a6910 con 0x7f8964105e80 2024-04-16T17:04:45.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.417+0000 7f8962000640 1 -- 172.21.15.18:0/1624962429 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c034cd0 con 0x7f8964105e80 2024-04-16T17:04:45.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.417+0000 7f8962000640 1 -- 172.21.15.18:0/1624962429 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c039040 con 0x7f8964105e80 2024-04-16T17:04:45.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.417+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89641a6cc0 con 0x7f8964105e80 2024-04-16T17:04:45.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.418+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896410b590 con 0x7f8964105e80 2024-04-16T17:04:45.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.420+0000 7f8962000640 1 -- 172.21.15.18:0/1624962429 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f895c044020 con 0x7f8964105e80 2024-04-16T17:04:45.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.422+0000 7f8962000640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f894c0a39b0 0x7f894c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.422+0000 7f8962000640 1 -- 172.21.15.18:0/1624962429 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f895c0f2730 con 0x7f8964105e80 2024-04-16T17:04:45.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.422+0000 7f8969a00640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f894c0a39b0 0x7f894c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.423+0000 7f8969a00640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f894c0a39b0 0x7f894c0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f89580058c0 tx=0x7f8958005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:45.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.424+0000 7f8962000640 1 -- 172.21.15.18:0/1624962429 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f895c04b020 con 0x7f8964105e80 2024-04-16T17:04:45.445 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:45.156+0000 7f34c1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:45.445 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:45.156+0000 7f34c1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:45.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:45.156+0000 7f34c1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:45.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:45.156+0000 7f34c1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:45.446 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:04:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:04:45.156+0000 7f34c1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:04:45.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.642+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89640773b0 con 0x7f8964105e80 2024-04-16T17:04:45.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.644+0000 7f8962000640 1 -- 172.21.15.18:0/1624962429 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25126 (secure 0 0 0) 0x7f895c0bef80 con 0x7f8964105e80 2024-04-16T17:04:45.645 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:45.646 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":94,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:04:44.091776+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3045212308":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/3152065578":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/1033276779":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6801/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/2149549301":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.646+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f894c0a39b0 msgr2=0x7f894c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.646+0000 7f896c400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f894c0a39b0 0x7f894c0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f89580058c0 tx=0x7f8958005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8964105e80 msgr2=0x7f8964076920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8964105e80 0x7f8964076920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f895c037040 tx=0x7f895c032ea0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 shutdown_connections 2024-04-16T17:04:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f894c0a39b0 0x7f894c0a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8964105e80 0x7f8964076920 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89641054c0 0x7f896407a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 --2- 172.21.15.18:0/1624962429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89641042d0 0x7f8964079db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 >> 172.21.15.18:0/1624962429 conn(0x7f89640ff880 msgr2=0x7f8964108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 shutdown_connections 2024-04-16T17:04:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.647+0000 7f896c400640 1 -- 172.21.15.18:0/1624962429 wait complete. 2024-04-16T17:04:45.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:04:45.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 -- 172.21.15.18:0/1091333228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0104040 msgr2=0x7f8eb0104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1091333228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0104040 0x7f8eb0104460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009860 tx=0x7f8ea002bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 -- 172.21.15.18:0/1091333228 shutdown_connections 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1091333228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8eb0105bf0 0x7f8eb010c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1091333228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8eb0105230 0x7f8eb01056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1091333228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0104040 0x7f8eb0104460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 -- 172.21.15.18:0/1091333228 >> 172.21.15.18:0/1091333228 conn(0x7f8eb00ff5d0 msgr2=0x7f8eb01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 -- 172.21.15.18:0/1091333228 shutdown_connections 2024-04-16T17:04:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.855+0000 7f8eb6400640 1 -- 172.21.15.18:0/1091333228 wait complete. 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.856+0000 7f8eb6400640 1 Processor -- start 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.856+0000 7f8eb6400640 1 -- start start 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb6400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8eb0104040 0x7f8eb019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb6400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8eb0105230 0x7f8eb019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0105bf0 0x7f8eb01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb6400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb010fcd0 con 0x7f8eb0104040 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb6400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb010fb50 con 0x7f8eb0105230 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb6400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb010fe50 con 0x7f8eb0105bf0 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8eb0104040 0x7f8eb019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8eb0104040 0x7f8eb019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42556/0 (socket says 172.21.15.18:42556) 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5000640 1 -- 172.21.15.18:0/1308849063 learned_addr learned my addr 172.21.15.18:0/1308849063 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:45.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5a00640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0105bf0 0x7f8eb01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eafe00640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8eb0105230 0x7f8eb019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5000640 1 -- 172.21.15.18:0/1308849063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0105bf0 msgr2=0x7f8eb01a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5000640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0105bf0 0x7f8eb01a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5000640 1 -- 172.21.15.18:0/1308849063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8eb0105230 msgr2=0x7f8eb019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5000640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8eb0105230 0x7f8eb019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5000640 1 -- 172.21.15.18:0/1308849063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb01a1f70 con 0x7f8eb0104040 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eafe00640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8eb0105230 0x7f8eb019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.857+0000 7f8eb5a00640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0105bf0 0x7f8eb01a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.858+0000 7f8eb5000640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8eb0104040 0x7f8eb019a3f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ea002c0b0 tx=0x7f8ea0032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.858+0000 7f8ead600640 1 -- 172.21.15.18:0/1308849063 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0041020 con 0x7f8eb0104040 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.858+0000 7f8ead600640 1 -- 172.21.15.18:0/1308849063 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea002c9f0 con 0x7f8eb0104040 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.858+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb01a2200 con 0x7f8eb0104040 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.858+0000 7f8ead600640 1 -- 172.21.15.18:0/1308849063 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0039040 con 0x7f8eb0104040 2024-04-16T17:04:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.858+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb01a26f0 con 0x7f8eb0104040 2024-04-16T17:04:45.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.859+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb010b480 con 0x7f8eb0104040 2024-04-16T17:04:45.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.861+0000 7f8ead600640 1 -- 172.21.15.18:0/1308849063 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f8ea0044020 con 0x7f8eb0104040 2024-04-16T17:04:45.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.862+0000 7f8ead600640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8e980a3990 0x7f8e980a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.863+0000 7f8ead600640 1 -- 172.21.15.18:0/1308849063 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8ea00f1f30 con 0x7f8eb0104040 2024-04-16T17:04:45.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.863+0000 7f8eafe00640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8e980a3990 0x7f8e980a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.864+0000 7f8eafe00640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8e980a3990 0x7f8e980a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8eb019b860 tx=0x7f8ea4009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:45.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:45.867+0000 7f8ead600640 1 -- 172.21.15.18:0/1308849063 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ea004b020 con 0x7f8eb0104040 2024-04-16T17:04:46.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.078+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8eb002d840 con 0x7f8eb0104040 2024-04-16T17:04:46.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.079+0000 7f8ead600640 1 -- 172.21.15.18:0/1308849063 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f8ea00de020 con 0x7f8eb0104040 2024-04-16T17:04:46.081 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:04:46.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8e980a3990 msgr2=0x7f8e980a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8e980a3990 0x7f8e980a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8eb019b860 tx=0x7f8ea4009260 comp rx=0 tx=0).stop 2024-04-16T17:04:46.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8eb0104040 msgr2=0x7f8eb019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8eb0104040 0x7f8eb019a3f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ea002c0b0 tx=0x7f8ea0032430 comp rx=0 tx=0).stop 2024-04-16T17:04:46.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 shutdown_connections 2024-04-16T17:04:46.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8e980a3990 0x7f8e980a5e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eb0105bf0 0x7f8eb01a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8eb0105230 0x7f8eb019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 --2- 172.21.15.18:0/1308849063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8eb0104040 0x7f8eb019a3f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 >> 172.21.15.18:0/1308849063 conn(0x7f8eb00ff5d0 msgr2=0x7f8eb0108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:46.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 shutdown_connections 2024-04-16T17:04:46.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.083+0000 7f8eb6400640 1 -- 172.21.15.18:0/1308849063 wait complete. 2024-04-16T17:04:46.110 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:04:46.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 -- 172.21.15.18:0/3930490307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc105220 msgr2=0x7f41fc107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 --2- 172.21.15.18:0/3930490307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc105220 0x7f41fc107610 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009900 tx=0x7f41e802bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:46.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 -- 172.21.15.18:0/3930490307 shutdown_connections 2024-04-16T17:04:46.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 --2- 172.21.15.18:0/3930490307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc107b50 0x7f41fc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 --2- 172.21.15.18:0/3930490307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc105220 0x7f41fc107610 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 --2- 172.21.15.18:0/3930490307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc1028f0 0x7f41fc104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 -- 172.21.15.18:0/3930490307 >> 172.21.15.18:0/3930490307 conn(0x7f41fc0fc2e0 msgr2=0x7f41fc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:46.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 -- 172.21.15.18:0/3930490307 shutdown_connections 2024-04-16T17:04:46.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.314+0000 7f4202a00640 1 -- 172.21.15.18:0/3930490307 wait complete. 2024-04-16T17:04:46.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.315+0000 7f4202a00640 1 Processor -- start 2024-04-16T17:04:46.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.315+0000 7f4202a00640 1 -- start start 2024-04-16T17:04:46.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.315+0000 7f4202a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc1028f0 0x7f41fc1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:46.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.315+0000 7f4202a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc105220 0x7f41fc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4202a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc107b50 0x7f41fc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4202a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc10db40 con 0x7f41fc107b50 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4202a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc10d9c0 con 0x7f41fc105220 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4202a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc10dcc0 con 0x7f41fc1028f0 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4201600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc1028f0 0x7f41fc1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4200c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc105220 0x7f41fc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4201600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc1028f0 0x7f41fc1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:33182/0 (socket says 172.21.15.18:33182) 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4201600640 1 -- 172.21.15.18:0/486867946 learned_addr learned my addr 172.21.15.18:0/486867946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4202000640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc107b50 0x7f41fc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4201600640 1 -- 172.21.15.18:0/486867946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc105220 msgr2=0x7f41fc1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4201600640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc105220 0x7f41fc1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4201600640 1 -- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc107b50 msgr2=0x7f41fc19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4201600640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc107b50 0x7f41fc19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.316+0000 7f4201600640 1 -- 172.21.15.18:0/486867946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc19fdd0 con 0x7f41fc1028f0 2024-04-16T17:04:46.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.317+0000 7f4201600640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc1028f0 0x7f41fc1981b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f41f000e9f0 tx=0x7f41f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:46.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.317+0000 7f41fa000640 1 -- 172.21.15.18:0/486867946 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f000cd80 con 0x7f41fc1028f0 2024-04-16T17:04:46.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.317+0000 7f41fa000640 1 -- 172.21.15.18:0/486867946 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0010910 con 0x7f41fc1028f0 2024-04-16T17:04:46.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.317+0000 7f41fa000640 1 -- 172.21.15.18:0/486867946 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0018990 con 0x7f41fc1028f0 2024-04-16T17:04:46.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.317+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc1a00c0 con 0x7f41fc1028f0 2024-04-16T17:04:46.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.317+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc1a0630 con 0x7f41fc1028f0 2024-04-16T17:04:46.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.318+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41fc0fddd0 con 0x7f41fc1028f0 2024-04-16T17:04:46.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.320+0000 7f41fa000640 1 -- 172.21.15.18:0/486867946 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f41f0010ac0 con 0x7f41fc1028f0 2024-04-16T17:04:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.321+0000 7f41fa000640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e40a3e30 0x7f41e40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.322+0000 7f41fa000640 1 -- 172.21.15.18:0/486867946 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f41f00d0ff0 con 0x7f41fc1028f0 2024-04-16T17:04:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.322+0000 7f4200c00640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e40a3e30 0x7f41e40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:46.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.323+0000 7f4200c00640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e40a3e30 0x7f41e40a62f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41e802c170 tx=0x7f41e802df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:46.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.327+0000 7f41fa000640 1 -- 172.21.15.18:0/486867946 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41f009cb00 con 0x7f41fc1028f0 2024-04-16T17:04:46.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: [16/Apr/2024:17:04:44] ENGINE Bus STARTING 2024-04-16T17:04:46.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: [16/Apr/2024:17:04:44] ENGINE Serving on http://172.21.15.18:8765 2024-04-16T17:04:46.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: [16/Apr/2024:17:04:44] ENGINE Serving on https://172.21.15.18:7150 2024-04-16T17:04:46.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: [16/Apr/2024:17:04:44] ENGINE Bus STARTED 2024-04-16T17:04:46.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: mgrmap e22: x(active, since 1.03374s), standbys: z, y 2024-04-16T17:04:46.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: from='client.25463 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:46.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:46.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1624962429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:46.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:46 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1308849063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:46.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: [16/Apr/2024:17:04:44] ENGINE Bus STARTING 2024-04-16T17:04:46.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: [16/Apr/2024:17:04:44] ENGINE Serving on http://172.21.15.18:8765 2024-04-16T17:04:46.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: [16/Apr/2024:17:04:44] ENGINE Serving on https://172.21.15.18:7150 2024-04-16T17:04:46.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: [16/Apr/2024:17:04:44] ENGINE Bus STARTED 2024-04-16T17:04:46.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: mgrmap e22: x(active, since 1.03374s), standbys: z, y 2024-04-16T17:04:46.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: from='client.25463 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:46.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:46.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1624962429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:46.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:46 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1308849063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: [16/Apr/2024:17:04:44] ENGINE Bus STARTING 2024-04-16T17:04:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: [16/Apr/2024:17:04:44] ENGINE Serving on http://172.21.15.18:8765 2024-04-16T17:04:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: [16/Apr/2024:17:04:44] ENGINE Serving on https://172.21.15.18:7150 2024-04-16T17:04:46.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: [16/Apr/2024:17:04:44] ENGINE Bus STARTED 2024-04-16T17:04:46.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: mgrmap e22: x(active, since 1.03374s), standbys: z, y 2024-04-16T17:04:46.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: from='client.25463 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:46.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:46.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1624962429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:46.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:46 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1308849063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.560+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41fc062c00 con 0x7f41fc1028f0 2024-04-16T17:04:46.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.561+0000 7f41fa000640 1 -- 172.21.15.18:0/486867946 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f41f00a0e00 con 0x7f41fc1028f0 2024-04-16T17:04:46.562 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e40a3e30 msgr2=0x7f41e40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e40a3e30 0x7f41e40a62f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41e802c170 tx=0x7f41e802df90 comp rx=0 tx=0).stop 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc1028f0 msgr2=0x7f41fc1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc1028f0 0x7f41fc1981b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f41f000e9f0 tx=0x7f41f000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 shutdown_connections 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e40a3e30 0x7f41e40a62f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc107b50 0x7f41fc19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc105220 0x7f41fc1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 --2- 172.21.15.18:0/486867946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc1028f0 0x7f41fc1981b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.563+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 >> 172.21.15.18:0/486867946 conn(0x7f41fc0fc2e0 msgr2=0x7f41fc103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.564+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 shutdown_connections 2024-04-16T17:04:46.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.564+0000 7f4202a00640 1 -- 172.21.15.18:0/486867946 wait complete. 2024-04-16T17:04:46.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 -- 172.21.15.18:0/3936651305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb9c105e00 msgr2=0x7ffb9c10c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 --2- 172.21.15.18:0/3936651305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb9c105e00 0x7ffb9c10c690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009860 tx=0x7ffb8c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 -- 172.21.15.18:0/3936651305 shutdown_connections 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 --2- 172.21.15.18:0/3936651305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb9c105e00 0x7ffb9c10c690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 --2- 172.21.15.18:0/3936651305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9c105440 0x7ffb9c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 --2- 172.21.15.18:0/3936651305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb9c104250 0x7ffb9c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 -- 172.21.15.18:0/3936651305 >> 172.21.15.18:0/3936651305 conn(0x7ffb9c0ff7c0 msgr2=0x7ffb9c101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 -- 172.21.15.18:0/3936651305 shutdown_connections 2024-04-16T17:04:46.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.785+0000 7ffba1600640 1 -- 172.21.15.18:0/3936651305 wait complete. 2024-04-16T17:04:46.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.786+0000 7ffba1600640 1 Processor -- start 2024-04-16T17:04:46.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.786+0000 7ffba1600640 1 -- start start 2024-04-16T17:04:46.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffba1600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb9c104250 0x7ffb9c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffba1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9c105440 0x7ffb9c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffba1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb9c105e00 0x7ffb9c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffba1600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c10fee0 con 0x7ffb9c104250 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffba1600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c10fd60 con 0x7ffb9c105e00 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffba1600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c110060 con 0x7ffb9c105440 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb9c104250 0x7ffb9c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb9c104250 0x7ffb9c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42592/0 (socket says 172.21.15.18:42592) 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9aa00640 1 -- 172.21.15.18:0/751497278 learned_addr learned my addr 172.21.15.18:0/751497278 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9a000640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9c105440 0x7ffb9c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9b400640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb9c105e00 0x7ffb9c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9aa00640 1 -- 172.21.15.18:0/751497278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9c105440 msgr2=0x7ffb9c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9aa00640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9c105440 0x7ffb9c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9aa00640 1 -- 172.21.15.18:0/751497278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb9c105e00 msgr2=0x7ffb9c1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:46.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9aa00640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb9c105e00 0x7ffb9c1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:46.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.787+0000 7ffb9aa00640 1 -- 172.21.15.18:0/751497278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c1a2150 con 0x7ffb9c104250 2024-04-16T17:04:46.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.788+0000 7ffb9aa00640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb9c104250 0x7ffb9c19a530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400c3f0 tx=0x7ffb94009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:46.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.788+0000 7ffb93400640 1 -- 172.21.15.18:0/751497278 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9401b070 con 0x7ffb9c104250 2024-04-16T17:04:46.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.788+0000 7ffb93400640 1 -- 172.21.15.18:0/751497278 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9400bb10 con 0x7ffb9c104250 2024-04-16T17:04:46.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.788+0000 7ffb93400640 1 -- 172.21.15.18:0/751497278 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94016d80 con 0x7ffb9c104250 2024-04-16T17:04:46.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.788+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9c1a2440 con 0x7ffb9c104250 2024-04-16T17:04:46.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.788+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9c1a2940 con 0x7ffb9c104250 2024-04-16T17:04:46.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.790+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb9c10b570 con 0x7ffb9c104250 2024-04-16T17:04:46.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.792+0000 7ffb93400640 1 -- 172.21.15.18:0/751497278 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7ffb9401e070 con 0x7ffb9c104250 2024-04-16T17:04:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.794+0000 7ffb93400640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffb800a3db0 0x7ffb800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.794+0000 7ffb93400640 1 -- 172.21.15.18:0/751497278 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ffb940d0c40 con 0x7ffb9c104250 2024-04-16T17:04:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.794+0000 7ffb9a000640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffb800a3db0 0x7ffb800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:46.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.795+0000 7ffb9a000640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffb800a3db0 0x7ffb800a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c19ba40 tx=0x7ffb84005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:46.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:46.798+0000 7ffb93400640 1 -- 172.21.15.18:0/751497278 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb9409c710 con 0x7ffb9c104250 2024-04-16T17:04:47.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.016+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb9c02d8d0 con 0x7ffb9c104250 2024-04-16T17:04:47.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.017+0000 7ffb93400640 1 -- 172.21.15.18:0/751497278 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7ffb940a0a10 con 0x7ffb9c104250 2024-04-16T17:04:47.019 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.019+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffb800a3db0 msgr2=0x7ffb800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.019+0000 7ffba1600640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffb800a3db0 0x7ffb800a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c19ba40 tx=0x7ffb84005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.019+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb9c104250 msgr2=0x7ffb9c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.019+0000 7ffba1600640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb9c104250 0x7ffb9c19a530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400c3f0 tx=0x7ffb94009f90 comp rx=0 tx=0).stop 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.019+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 shutdown_connections 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.019+0000 7ffba1600640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffb800a3db0 0x7ffb800a6270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.019+0000 7ffba1600640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb9c105e00 0x7ffb9c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.019+0000 7ffba1600640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9c105440 0x7ffb9c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.020+0000 7ffba1600640 1 --2- 172.21.15.18:0/751497278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb9c104250 0x7ffb9c19a530 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.020+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 >> 172.21.15.18:0/751497278 conn(0x7ffb9c0ff7c0 msgr2=0x7ffb9c108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:47.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.020+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 shutdown_connections 2024-04-16T17:04:47.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.020+0000 7ffba1600640 1 -- 172.21.15.18:0/751497278 wait complete. 2024-04-16T17:04:47.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:04:47.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.269+0000 7f0979400640 1 -- 172.21.15.18:0/1386884544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09740749e0 msgr2=0x7f0974074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.269+0000 7f0979400640 1 --2- 172.21.15.18:0/1386884544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09740749e0 0x7f0974074e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0960009860 tx=0x7f096002bc20 comp rx=0 tx=0).stop 2024-04-16T17:04:47.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.269+0000 7f0979400640 1 -- 172.21.15.18:0/1386884544 shutdown_connections 2024-04-16T17:04:47.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.269+0000 7f0979400640 1 --2- 172.21.15.18:0/1386884544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0974109d40 0x7f097410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.269+0000 7f0979400640 1 --2- 172.21.15.18:0/1386884544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09740749e0 0x7f0974074e40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.269+0000 7f0979400640 1 --2- 172.21.15.18:0/1386884544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0974075fc0 0x7f0974074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.269+0000 7f0979400640 1 -- 172.21.15.18:0/1386884544 >> 172.21.15.18:0/1386884544 conn(0x7f09740fd400 msgr2=0x7f09740ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:47.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.270+0000 7f0979400640 1 -- 172.21.15.18:0/1386884544 shutdown_connections 2024-04-16T17:04:47.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.270+0000 7f0979400640 1 -- 172.21.15.18:0/1386884544 wait complete. 2024-04-16T17:04:47.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.271+0000 7f0979400640 1 Processor -- start 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.271+0000 7f0979400640 1 -- start start 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0979400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09740749e0 0x7f097419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0979400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0974075fc0 0x7f097419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0979400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0974109d40 0x7f09741a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0979400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f097410fd30 con 0x7f09740749e0 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0979400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f097410fbb0 con 0x7f0974109d40 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0979400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f097410feb0 con 0x7f0974075fc0 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0974075fc0 0x7f097419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09740749e0 0x7f097419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0974075fc0 0x7f097419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50982/0 (socket says 172.21.15.18:50982) 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973400640 1 -- 172.21.15.18:0/2256965760 learned_addr learned my addr 172.21.15.18:0/2256965760 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0978a00640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0974109d40 0x7f09741a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973400640 1 -- 172.21.15.18:0/2256965760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0974109d40 msgr2=0x7f09741a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0974109d40 0x7f09741a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973400640 1 -- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09740749e0 msgr2=0x7f097419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09740749e0 0x7f097419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.272+0000 7f0973400640 1 -- 172.21.15.18:0/2256965760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09741a1f80 con 0x7f0974075fc0 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.273+0000 7f0973400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0974075fc0 0x7f097419a8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0960034a80 tx=0x7f09600042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.273+0000 7f0970c00640 1 -- 172.21.15.18:0/2256965760 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960041030 con 0x7f0974075fc0 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.273+0000 7f0970c00640 1 -- 172.21.15.18:0/2256965760 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096002ce50 con 0x7f0974075fc0 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.273+0000 7f0970c00640 1 -- 172.21.15.18:0/2256965760 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960039040 con 0x7f0974075fc0 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.273+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09741a2210 con 0x7f0974075fc0 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.273+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09741a26b0 con 0x7f0974075fc0 2024-04-16T17:04:47.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.275+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0974108b00 con 0x7f0974075fc0 2024-04-16T17:04:47.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.276+0000 7f0970c00640 1 -- 172.21.15.18:0/2256965760 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f0960044070 con 0x7f0974075fc0 2024-04-16T17:04:47.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.278+0000 7f0970c00640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f09580a3e30 0x7f09580a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:47.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.278+0000 7f0970c00640 1 -- 172.21.15.18:0/2256965760 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f09600f1ed0 con 0x7f0974075fc0 2024-04-16T17:04:47.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.278+0000 7f0973e00640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f09580a3e30 0x7f09580a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:47.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.279+0000 7f0973e00640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f09580a3e30 0x7f09580a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0964005fd0 tx=0x7f0964009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:47.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.282+0000 7f0970c00640 1 -- 172.21.15.18:0/2256965760 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09600be100 con 0x7f0974075fc0 2024-04-16T17:04:47.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:47 smithi022 ceph-mon[36023]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:47.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:47 smithi022 ceph-mon[36023]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T17:04:47.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:47 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:47 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:47 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/486867946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:47.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:47 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:47 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:47 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/751497278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:47.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:47 smithi018 ceph-mon[31776]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:47.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:47 smithi018 ceph-mon[31776]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T17:04:47.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:47 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:47 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:47 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/486867946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:47.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:47 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:47 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:47 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/751497278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:47.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.500+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f097402d820 con 0x7f0974075fc0 2024-04-16T17:04:47.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:47 smithi043 ceph-mon[36901]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:47.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:47 smithi043 ceph-mon[36901]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T17:04:47.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:47 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:47 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:47 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/486867946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:47.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:47 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:47 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:47.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:47 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/751497278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:47.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.501+0000 7f0970c00640 1 -- 172.21.15.18:0/2256965760 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f09600bda00 con 0x7f0974075fc0 2024-04-16T17:04:47.503 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.503+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f09580a3e30 msgr2=0x7f09580a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.503+0000 7f0979400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f09580a3e30 0x7f09580a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0964005fd0 tx=0x7f0964009260 comp rx=0 tx=0).stop 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.503+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0974075fc0 msgr2=0x7f097419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.503+0000 7f0979400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0974075fc0 0x7f097419a8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0960034a80 tx=0x7f09600042f0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.504+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 shutdown_connections 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.504+0000 7f0979400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f09580a3e30 0x7f09580a62f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.504+0000 7f0979400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0974109d40 0x7f09741a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.504+0000 7f0979400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0974075fc0 0x7f097419a8a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.504+0000 7f0979400640 1 --2- 172.21.15.18:0/2256965760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f09740749e0 0x7f097419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.504+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 >> 172.21.15.18:0/2256965760 conn(0x7f09740fd400 msgr2=0x7f0974105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:47.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.504+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 shutdown_connections 2024-04-16T17:04:47.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.504+0000 7f0979400640 1 -- 172.21.15.18:0/2256965760 wait complete. 2024-04-16T17:04:47.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:04:47.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.723+0000 7f15d9200640 1 -- 172.21.15.18:0/4138321607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d4105e80 msgr2=0x7f15d410c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.723+0000 7f15d9200640 1 --2- 172.21.15.18:0/4138321607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d4105e80 0x7f15d410c710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009ca0 tx=0x7f15cc02c0f0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.723+0000 7f15d9200640 1 -- 172.21.15.18:0/4138321607 shutdown_connections 2024-04-16T17:04:47.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.723+0000 7f15d9200640 1 --2- 172.21.15.18:0/4138321607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d4105e80 0x7f15d410c710 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.723+0000 7f15d9200640 1 --2- 172.21.15.18:0/4138321607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d41054c0 0x7f15d4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.723+0000 7f15d9200640 1 --2- 172.21.15.18:0/4138321607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d41042d0 0x7f15d41046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.723+0000 7f15d9200640 1 -- 172.21.15.18:0/4138321607 >> 172.21.15.18:0/4138321607 conn(0x7f15d40ff880 msgr2=0x7f15d4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:47.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.724+0000 7f15d9200640 1 -- 172.21.15.18:0/4138321607 shutdown_connections 2024-04-16T17:04:47.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.724+0000 7f15d9200640 1 -- 172.21.15.18:0/4138321607 wait complete. 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.725+0000 7f15d9200640 1 Processor -- start 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.725+0000 7f15d9200640 1 -- start start 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.725+0000 7f15d9200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d41042d0 0x7f15d419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.725+0000 7f15d9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d41054c0 0x7f15d419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.725+0000 7f15d9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d4105e80 0x7f15d41a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.725+0000 7f15d9200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f15d410fef0 con 0x7f15d41042d0 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.725+0000 7f15d9200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f15d410fd70 con 0x7f15d4105e80 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d9200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4110070 con 0x7f15d41054c0 2024-04-16T17:04:47.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d4105e80 0x7f15d41a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d41054c0 0x7f15d419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d4105e80 0x7f15d41a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54348/0 (socket says 172.21.15.18:54348) 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 -- 172.21.15.18:0/4159862031 learned_addr learned my addr 172.21.15.18:0/4159862031 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d1e00640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d41042d0 0x7f15d419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 -- 172.21.15.18:0/4159862031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d41054c0 msgr2=0x7f15d419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d41054c0 0x7f15d419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 -- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d41042d0 msgr2=0x7f15d419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d41042d0 0x7f15d419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 -- 172.21.15.18:0/4159862031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d41a20e0 con 0x7f15d4105e80 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d1e00640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d41042d0 0x7f15d419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:47.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.726+0000 7f15d2800640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d4105e80 0x7f15d41a1a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f15cc02c630 tx=0x7f15cc032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:47.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.727+0000 7f15caa00640 1 -- 172.21.15.18:0/4159862031 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc041020 con 0x7f15d4105e80 2024-04-16T17:04:47.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.727+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41a2370 con 0x7f15d4105e80 2024-04-16T17:04:47.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.727+0000 7f15caa00640 1 -- 172.21.15.18:0/4159862031 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc034cd0 con 0x7f15d4105e80 2024-04-16T17:04:47.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.727+0000 7f15caa00640 1 -- 172.21.15.18:0/4159862031 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc039040 con 0x7f15d4105e80 2024-04-16T17:04:47.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.727+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41a28b0 con 0x7f15d4105e80 2024-04-16T17:04:47.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.728+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d410b590 con 0x7f15d4105e80 2024-04-16T17:04:47.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.730+0000 7f15caa00640 1 -- 172.21.15.18:0/4159862031 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f15cc044020 con 0x7f15d4105e80 2024-04-16T17:04:47.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.731+0000 7f15caa00640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f15b80a3e30 0x7f15b80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:47.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.732+0000 7f15caa00640 1 -- 172.21.15.18:0/4159862031 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f15cc0f2b30 con 0x7f15d4105e80 2024-04-16T17:04:47.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.732+0000 7f15d1e00640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f15b80a3e30 0x7f15b80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:47.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.733+0000 7f15d1e00640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f15b80a3e30 0x7f15b80a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f15d4105320 tx=0x7f15c4005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:47.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.736+0000 7f15caa00640 1 -- 172.21.15.18:0/4159862031 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15cc04b020 con 0x7f15d4105e80 2024-04-16T17:04:47.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.949+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15d4062c00 con 0x7f15d4105e80 2024-04-16T17:04:47.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.951+0000 7f15caa00640 1 -- 172.21.15.18:0/4159862031 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25126 (secure 0 0 0) 0x7f15cc0bedf0 con 0x7f15d4105e80 2024-04-16T17:04:47.954 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:47.954 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":94,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:04:44.091776+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3045212308":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/3152065578":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/1033276779":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6801/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/2149549301":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:47.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f15b80a3e30 msgr2=0x7f15b80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f15b80a3e30 0x7f15b80a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f15d4105320 tx=0x7f15c4005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d4105e80 msgr2=0x7f15d41a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d4105e80 0x7f15d41a1a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f15cc02c630 tx=0x7f15cc032ea0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 shutdown_connections 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f15b80a3e30 0x7f15b80a62f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d4105e80 0x7f15d41a1a70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d41054c0 0x7f15d419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 --2- 172.21.15.18:0/4159862031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15d41042d0 0x7f15d419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.954+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 >> 172.21.15.18:0/4159862031 conn(0x7f15d40ff880 msgr2=0x7f15d4108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.955+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 shutdown_connections 2024-04-16T17:04:47.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:47.955+0000 7f15d9200640 1 -- 172.21.15.18:0/4159862031 wait complete. 2024-04-16T17:04:47.976 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:04:47.976 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 -- 172.21.15.18:0/686419430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c04077c00 msgr2=0x7f1c04078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/686419430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c04077c00 0x7f1c04078060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009900 tx=0x7f1bf002bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 -- 172.21.15.18:0/686419430 shutdown_connections 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/686419430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c040785a0 0x7f1c04078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/686419430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c04077c00 0x7f1c04078060 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/686419430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c040769c0 0x7f1c04076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 -- 172.21.15.18:0/686419430 >> 172.21.15.18:0/686419430 conn(0x7f1c040ff780 msgr2=0x7f1c04101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 -- 172.21.15.18:0/686419430 shutdown_connections 2024-04-16T17:04:48.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.191+0000 7f1c0aa00640 1 -- 172.21.15.18:0/686419430 wait complete. 2024-04-16T17:04:48.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.192+0000 7f1c0aa00640 1 Processor -- start 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c0aa00640 1 -- start start 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c0aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c040769c0 0x7f1c0419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c0aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c04077c00 0x7f1c0419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c040785a0 0x7f1c041a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c0aa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041142e0 con 0x7f1c040769c0 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c0aa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04114160 con 0x7f1c040785a0 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c0aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04114460 con 0x7f1c04077c00 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c040769c0 0x7f1c0419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c040769c0 0x7f1c0419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55456/0 (socket says 172.21.15.18:55456) 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c03e00640 1 -- 172.21.15.18:0/3505188813 learned_addr learned my addr 172.21.15.18:0/3505188813 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c08a00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c040785a0 0x7f1c041a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.193+0000 7f1c03400640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c04077c00 0x7f1c0419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c03e00640 1 -- 172.21.15.18:0/3505188813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c04077c00 msgr2=0x7f1c0419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c03e00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c04077c00 0x7f1c0419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c03e00640 1 -- 172.21.15.18:0/3505188813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c040785a0 msgr2=0x7f1c041a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c03e00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c040785a0 0x7f1c041a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c03e00640 1 -- 172.21.15.18:0/3505188813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c041a6560 con 0x7f1c040769c0 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c03e00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c040769c0 0x7f1c0419e940 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400e9f0 tx=0x7f1bf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c00c00640 1 -- 172.21.15.18:0/3505188813 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf400cd80 con 0x7f1c040769c0 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c041a6850 con 0x7f1c040769c0 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c00c00640 1 -- 172.21.15.18:0/3505188813 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf4005500 con 0x7f1c040769c0 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.194+0000 7f1c00c00640 1 -- 172.21.15.18:0/3505188813 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4010690 con 0x7f1c040769c0 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.195+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c041a6d70 con 0x7f1c040769c0 2024-04-16T17:04:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.196+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c04110d70 con 0x7f1c040769c0 2024-04-16T17:04:48.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.198+0000 7f1c00c00640 1 -- 172.21.15.18:0/3505188813 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f1bf40046c0 con 0x7f1c040769c0 2024-04-16T17:04:48.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.199+0000 7f1c00c00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1be80a3db0 0x7f1be80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:48.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.200+0000 7f1c00c00640 1 -- 172.21.15.18:0/3505188813 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1bf40d11c0 con 0x7f1c040769c0 2024-04-16T17:04:48.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.200+0000 7f1c03400640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1be80a3db0 0x7f1be80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:48.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.201+0000 7f1c03400640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1be80a3db0 0x7f1be80a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1bf002c7b0 tx=0x7f1bf0005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:48.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.204+0000 7f1c00c00640 1 -- 172.21.15.18:0/3505188813 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bf409d2e0 con 0x7f1c040769c0 2024-04-16T17:04:48.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.412+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c0402d620 con 0x7f1c040769c0 2024-04-16T17:04:48.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.413+0000 7f1c00c00640 1 -- 172.21.15.18:0/3505188813 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f1c0402d620 con 0x7f1c040769c0 2024-04-16T17:04:48.414 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1be80a3db0 msgr2=0x7f1be80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1be80a3db0 0x7f1be80a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1bf002c7b0 tx=0x7f1bf0005fd0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c040769c0 msgr2=0x7f1c0419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c040769c0 0x7f1c0419e940 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400e9f0 tx=0x7f1bf400edf0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 shutdown_connections 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1be80a3db0 0x7f1be80a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c040785a0 0x7f1c041a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c04077c00 0x7f1c0419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 --2- 172.21.15.18:0/3505188813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c040769c0 0x7f1c0419e940 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.415+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 >> 172.21.15.18:0/3505188813 conn(0x7f1c040ff780 msgr2=0x7f1c04100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:48.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.416+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 shutdown_connections 2024-04-16T17:04:48.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.416+0000 7f1c0aa00640 1 -- 172.21.15.18:0/3505188813 wait complete. 2024-04-16T17:04:48.438 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:04:48.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.655+0000 7f552be00640 1 -- 172.21.15.18:0/2130375115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c102fe0 msgr2=0x7f552c103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.655+0000 7f552be00640 1 --2- 172.21.15.18:0/2130375115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c102fe0 0x7f552c103460 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5518009860 tx=0x7f551802bb90 comp rx=0 tx=0).stop 2024-04-16T17:04:48.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.655+0000 7f552be00640 1 -- 172.21.15.18:0/2130375115 shutdown_connections 2024-04-16T17:04:48.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.655+0000 7f552be00640 1 --2- 172.21.15.18:0/2130375115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f552c1039a0 0x7f552c10a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.655+0000 7f552be00640 1 --2- 172.21.15.18:0/2130375115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c102fe0 0x7f552c103460 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.655+0000 7f552be00640 1 --2- 172.21.15.18:0/2130375115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552c101e90 0x7f552c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.655+0000 7f552be00640 1 -- 172.21.15.18:0/2130375115 >> 172.21.15.18:0/2130375115 conn(0x7f552c0fd400 msgr2=0x7f552c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:48.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.656+0000 7f552be00640 1 -- 172.21.15.18:0/2130375115 shutdown_connections 2024-04-16T17:04:48.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.656+0000 7f552be00640 1 -- 172.21.15.18:0/2130375115 wait complete. 2024-04-16T17:04:48.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.657+0000 7f552be00640 1 Processor -- start 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.657+0000 7f552be00640 1 -- start start 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c101e90 0x7f552c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f552c102fe0 0x7f552c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552c1039a0 0x7f552c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f552c10db20 con 0x7f552c102fe0 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f552c10d9a0 con 0x7f552c1039a0 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f552c10dca0 con 0x7f552c101e90 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f552c102fe0 0x7f552c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f552c102fe0 0x7f552c19a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55470/0 (socket says 172.21.15.18:55470) 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552a000640 1 -- 172.21.15.18:0/1882524347 learned_addr learned my addr 172.21.15.18:0/1882524347 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552aa00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c101e90 0x7f552c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552b400640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552c1039a0 0x7f552c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:48.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552a000640 1 -- 172.21.15.18:0/1882524347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c101e90 msgr2=0x7f552c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552a000640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c101e90 0x7f552c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552a000640 1 -- 172.21.15.18:0/1882524347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552c1039a0 msgr2=0x7f552c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552a000640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552c1039a0 0x7f552c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552a000640 1 -- 172.21.15.18:0/1882524347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c1a1fd0 con 0x7f552c102fe0 2024-04-16T17:04:48.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.658+0000 7f552aa00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c101e90 0x7f552c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:48.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.659+0000 7f552a000640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f552c102fe0 0x7f552c19a940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f55180349e0 tx=0x7f5518032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:48.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.659+0000 7f5523400640 1 -- 172.21.15.18:0/1882524347 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518041020 con 0x7f552c102fe0 2024-04-16T17:04:48.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.659+0000 7f5523400640 1 -- 172.21.15.18:0/1882524347 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551802ca10 con 0x7f552c102fe0 2024-04-16T17:04:48.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.659+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c1a2260 con 0x7f552c102fe0 2024-04-16T17:04:48.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.659+0000 7f5523400640 1 -- 172.21.15.18:0/1882524347 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518039040 con 0x7f552c102fe0 2024-04-16T17:04:48.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.659+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a2750 con 0x7f552c102fe0 2024-04-16T17:04:48.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.661+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c1a29b0 con 0x7f552c102fe0 2024-04-16T17:04:48.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.663+0000 7f5523400640 1 -- 172.21.15.18:0/1882524347 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f5518044020 con 0x7f552c102fe0 2024-04-16T17:04:48.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.664+0000 7f5523400640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55100a3db0 0x7f55100a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:48.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.664+0000 7f5523400640 1 -- 172.21.15.18:0/1882524347 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f55180f2510 con 0x7f552c102fe0 2024-04-16T17:04:48.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.665+0000 7f552aa00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55100a3db0 0x7f55100a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:48.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.665+0000 7f552aa00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55100a3db0 0x7f55100a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f551c009c20 tx=0x7f551c005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:48.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.669+0000 7f5523400640 1 -- 172.21.15.18:0/1882524347 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f551804b020 con 0x7f552c102fe0 2024-04-16T17:04:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2256965760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4159862031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:48.698 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:48 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3505188813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:48.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.742+0000 7f5523400640 1 -- 172.21.15.18:0/1882524347 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f55180042c0 con 0x7f552c102fe0 2024-04-16T17:04:48.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2256965760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4159862031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:48.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:48 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3505188813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:48.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.882+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f552c1a2b80 con 0x7f552c102fe0 2024-04-16T17:04:48.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.883+0000 7f5523400640 1 -- 172.21.15.18:0/1882524347 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f55180be7a0 con 0x7f552c102fe0 2024-04-16T17:04:48.885 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:48.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55100a3db0 msgr2=0x7f55100a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55100a3db0 0x7f55100a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f551c009c20 tx=0x7f551c005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f552c102fe0 msgr2=0x7f552c19a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f552c102fe0 0x7f552c19a940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f55180349e0 tx=0x7f5518032430 comp rx=0 tx=0).stop 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 shutdown_connections 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55100a3db0 0x7f55100a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f552c1039a0 0x7f552c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f552c102fe0 0x7f552c19a940 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 --2- 172.21.15.18:0/1882524347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c101e90 0x7f552c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 >> 172.21.15.18:0/1882524347 conn(0x7f552c0fd400 msgr2=0x7f552c074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 shutdown_connections 2024-04-16T17:04:48.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:48.886+0000 7f552be00640 1 -- 172.21.15.18:0/1882524347 wait complete. 2024-04-16T17:04:48.907 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2256965760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4159862031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:48.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:48 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3505188813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:49.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 -- 172.21.15.18:0/4179683328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c48102fe0 msgr2=0x7f8c48103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 --2- 172.21.15.18:0/4179683328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c48102fe0 0x7f8c48103460 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009900 tx=0x7f8c3402bc40 comp rx=0 tx=0).stop 2024-04-16T17:04:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 -- 172.21.15.18:0/4179683328 shutdown_connections 2024-04-16T17:04:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 --2- 172.21.15.18:0/4179683328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c481039a0 0x7f8c4810a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 --2- 172.21.15.18:0/4179683328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c48102fe0 0x7f8c48103460 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 --2- 172.21.15.18:0/4179683328 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c48101e90 0x7f8c481022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 -- 172.21.15.18:0/4179683328 >> 172.21.15.18:0/4179683328 conn(0x7f8c480fd400 msgr2=0x7f8c480ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 -- 172.21.15.18:0/4179683328 shutdown_connections 2024-04-16T17:04:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.123+0000 7f8c4ee00640 1 -- 172.21.15.18:0/4179683328 wait complete. 2024-04-16T17:04:49.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.124+0000 7f8c4ee00640 1 Processor -- start 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.124+0000 7f8c4ee00640 1 -- start start 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4ee00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c48101e90 0x7f8c4819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c48102fe0 0x7f8c4819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4ee00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c481039a0 0x7f8c481a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4ee00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4810db30 con 0x7f8c48101e90 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4ee00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4810d9b0 con 0x7f8c481039a0 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4ee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4810dcb0 con 0x7f8c48102fe0 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c48101e90 0x7f8c4819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c48101e90 0x7f8c4819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55488/0 (socket says 172.21.15.18:55488) 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4da00640 1 -- 172.21.15.18:0/891184681 learned_addr learned my addr 172.21.15.18:0/891184681 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4d000640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c48102fe0 0x7f8c4819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4e400640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c481039a0 0x7f8c481a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4da00640 1 -- 172.21.15.18:0/891184681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c48102fe0 msgr2=0x7f8c4819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4da00640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c48102fe0 0x7f8c4819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4da00640 1 -- 172.21.15.18:0/891184681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c481039a0 msgr2=0x7f8c481a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4da00640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c481039a0 0x7f8c481a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4da00640 1 -- 172.21.15.18:0/891184681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c481a2020 con 0x7f8c48101e90 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.125+0000 7f8c4e400640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c481039a0 0x7f8c481a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.126+0000 7f8c4da00640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c48101e90 0x7f8c4819a450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00ea60 tx=0x7f8c3c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.126+0000 7f8c46000640 1 -- 172.21.15.18:0/891184681 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00cde0 con 0x7f8c48101e90 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.126+0000 7f8c46000640 1 -- 172.21.15.18:0/891184681 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c0035f0 con 0x7f8c48101e90 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.126+0000 7f8c4ee00640 1 -- 172.21.15.18:0/891184681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c481a2310 con 0x7f8c48101e90 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.126+0000 7f8c46000640 1 -- 172.21.15.18:0/891184681 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c010670 con 0x7f8c48101e90 2024-04-16T17:04:49.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.126+0000 7f8c4ee00640 1 -- 172.21.15.18:0/891184681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c481a2880 con 0x7f8c48101e90 2024-04-16T17:04:49.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.128+0000 7f8c3b400640 1 -- 172.21.15.18:0/891184681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c48062c00 con 0x7f8c48101e90 2024-04-16T17:04:49.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.130+0000 7f8c46000640 1 -- 172.21.15.18:0/891184681 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8c3c0037a0 con 0x7f8c48101e90 2024-04-16T17:04:49.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.131+0000 7f8c46000640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c300a3ec0 0x7f8c300a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.131+0000 7f8c46000640 1 -- 172.21.15.18:0/891184681 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8c3c0d1210 con 0x7f8c48101e90 2024-04-16T17:04:49.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.132+0000 7f8c4d000640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c300a3ec0 0x7f8c300a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.133+0000 7f8c4d000640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c300a3ec0 0x7f8c300a6380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c34000c00 tx=0x7f8c34005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.137+0000 7f8c46000640 1 -- 172.21.15.18:0/891184681 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c3c09d250 con 0x7f8c48101e90 2024-04-16T17:04:49.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.348+0000 7f8c3b400640 1 -- 172.21.15.18:0/891184681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8c48062dd0 con 0x7f8c48101e90 2024-04-16T17:04:49.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.348+0000 7f8c46000640 1 -- 172.21.15.18:0/891184681 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f8c3c09cb50 con 0x7f8c48101e90 2024-04-16T17:04:49.350 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:04:49.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 -- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c300a3ec0 msgr2=0x7f8c300a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c300a3ec0 0x7f8c300a6380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c34000c00 tx=0x7f8c34005fd0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 -- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c48101e90 msgr2=0x7f8c4819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c48101e90 0x7f8c4819a450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00ea60 tx=0x7f8c3c00ef20 comp rx=0 tx=0).stop 2024-04-16T17:04:49.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 -- 172.21.15.18:0/891184681 shutdown_connections 2024-04-16T17:04:49.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c300a3ec0 0x7f8c300a6380 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c481039a0 0x7f8c481a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c48102fe0 0x7f8c4819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 --2- 172.21.15.18:0/891184681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c48101e90 0x7f8c4819a450 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.351+0000 7f8c3b400640 1 -- 172.21.15.18:0/891184681 >> 172.21.15.18:0/891184681 conn(0x7f8c480fd400 msgr2=0x7f8c48074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:49.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.352+0000 7f8c3b400640 1 -- 172.21.15.18:0/891184681 shutdown_connections 2024-04-16T17:04:49.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.352+0000 7f8c3b400640 1 -- 172.21.15.18:0/891184681 wait complete. 2024-04-16T17:04:49.372 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:04:49.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 -- 172.21.15.18:0/4277489359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18105440 msgr2=0x7f8e181058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/4277489359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18105440 0x7f8e181058c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009860 tx=0x7f8e0402bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:49.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 -- 172.21.15.18:0/4277489359 shutdown_connections 2024-04-16T17:04:49.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/4277489359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e18105e00 0x7f8e1810c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/4277489359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18105440 0x7f8e181058c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/4277489359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e18104250 0x7f8e18104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 -- 172.21.15.18:0/4277489359 >> 172.21.15.18:0/4277489359 conn(0x7f8e180ff7c0 msgr2=0x7f8e18101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:49.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 -- 172.21.15.18:0/4277489359 shutdown_connections 2024-04-16T17:04:49.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.618+0000 7f8e1cc00640 1 -- 172.21.15.18:0/4277489359 wait complete. 2024-04-16T17:04:49.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.619+0000 7f8e1cc00640 1 Processor -- start 2024-04-16T17:04:49.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e1cc00640 1 -- start start 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e1cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18104250 0x7f8e1819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e1cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e18105440 0x7f8e1819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e1cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e18105e00 0x7f8e181a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e1cc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1810fee0 con 0x7f8e18105e00 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e1cc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1810fd60 con 0x7f8e18104250 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e1cc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18110060 con 0x7f8e18105440 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e15e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18104250 0x7f8e1819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e15400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e18105440 0x7f8e1819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e15e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18104250 0x7f8e1819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54414/0 (socket says 172.21.15.18:54414) 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e15e00640 1 -- 172.21.15.18:0/2584144377 learned_addr learned my addr 172.21.15.18:0/2584144377 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:49.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.620+0000 7f8e16800640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e18105e00 0x7f8e181a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e15e00640 1 -- 172.21.15.18:0/2584144377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e18105440 msgr2=0x7f8e1819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e15e00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e18105440 0x7f8e1819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e15e00640 1 -- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e18105e00 msgr2=0x7f8e181a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e15e00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e18105e00 0x7f8e181a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e15e00640 1 -- 172.21.15.18:0/2584144377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e181a2150 con 0x7f8e18104250 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e16800640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e18105e00 0x7f8e181a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e15400640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e18105440 0x7f8e1819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e15e00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18104250 0x7f8e1819a530 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800e9a0 tx=0x7f8e0800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e0ea00640 1 -- 172.21.15.18:0/2584144377 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0800cd80 con 0x7f8e18104250 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e0ea00640 1 -- 172.21.15.18:0/2584144377 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e08010910 con 0x7f8e18104250 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e0ea00640 1 -- 172.21.15.18:0/2584144377 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08018a30 con 0x7f8e18104250 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e181a2440 con 0x7f8e18104250 2024-04-16T17:04:49.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.621+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e181a29b0 con 0x7f8e18104250 2024-04-16T17:04:49.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.623+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1810b570 con 0x7f8e18104250 2024-04-16T17:04:49.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.625+0000 7f8e0ea00640 1 -- 172.21.15.18:0/2584144377 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8e08010ac0 con 0x7f8e18104250 2024-04-16T17:04:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.626+0000 7f8e0ea00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8dfc0a3ec0 0x7f8dfc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.626+0000 7f8e0ea00640 1 -- 172.21.15.18:0/2584144377 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8e080d1220 con 0x7f8e18104250 2024-04-16T17:04:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.627+0000 7f8e15400640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8dfc0a3ec0 0x7f8dfc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.627+0000 7f8e15400640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8dfc0a3ec0 0x7f8dfc0a6380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009830 tx=0x7f8e0402e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:49.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.630+0000 7f8e0ea00640 1 -- 172.21.15.18:0/2584144377 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e08053ff0 con 0x7f8e18104250 2024-04-16T17:04:49.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:04:49.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:04:49.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:04:49.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:04:49.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:49.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T17:04:49.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1882524347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/891184681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:49.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:49 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:04:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:04:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:04:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:04:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1882524347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/891184681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:49.754 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:49 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:49.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.853+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8e18062c00 con 0x7f8e18104250 2024-04-16T17:04:49.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.853+0000 7f8e0ea00640 1 -- 172.21.15.18:0/2584144377 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f8e0809d2c0 con 0x7f8e18104250 2024-04-16T17:04:49.855 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:49.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8dfc0a3ec0 msgr2=0x7f8dfc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8dfc0a3ec0 0x7f8dfc0a6380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009830 tx=0x7f8e0402e040 comp rx=0 tx=0).stop 2024-04-16T17:04:49.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18104250 msgr2=0x7f8e1819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18104250 0x7f8e1819a530 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800e9a0 tx=0x7f8e0800ee60 comp rx=0 tx=0).stop 2024-04-16T17:04:49.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 shutdown_connections 2024-04-16T17:04:49.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8dfc0a3ec0 0x7f8dfc0a6380 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e18105e00 0x7f8e181a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e18105440 0x7f8e1819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 --2- 172.21.15.18:0/2584144377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e18104250 0x7f8e1819a530 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 >> 172.21.15.18:0/2584144377 conn(0x7f8e180ff7c0 msgr2=0x7f8e18108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:49.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 shutdown_connections 2024-04-16T17:04:49.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:49.856+0000 7f8e1cc00640 1 -- 172.21.15.18:0/2584144377 wait complete. 2024-04-16T17:04:49.876 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-16T17:04:49.877 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:04:49.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1882524347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/891184681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:49.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:49.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:49 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 -- 172.21.15.18:0/921938967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc710107c20 msgr2=0x7fc71010a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 --2- 172.21.15.18:0/921938967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc710107c20 0x7fc71010a070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc708009ca0 tx=0x7fc70802c100 comp rx=0 tx=0).stop 2024-04-16T17:04:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 -- 172.21.15.18:0/921938967 shutdown_connections 2024-04-16T17:04:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 --2- 172.21.15.18:0/921938967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc710107c20 0x7fc71010a070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 --2- 172.21.15.18:0/921938967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7101052f0 0x7fc7101076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 --2- 172.21.15.18:0/921938967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7101029c0 0x7fc710104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 -- 172.21.15.18:0/921938967 >> 172.21.15.18:0/921938967 conn(0x7fc7100fc3d0 msgr2=0x7fc7100fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 -- 172.21.15.18:0/921938967 shutdown_connections 2024-04-16T17:04:50.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.061+0000 7fc70fe00640 1 -- 172.21.15.18:0/921938967 wait complete. 2024-04-16T17:04:50.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70fe00640 1 Processor -- start 2024-04-16T17:04:50.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70fe00640 1 -- start start 2024-04-16T17:04:50.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7101029c0 0x7fc710102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:50.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7101052f0 0x7fc710100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:50.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710107c20 0x7fc710100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70fe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010dbc0 con 0x7fc7101029c0 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70fe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010da40 con 0x7fc710107c20 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010dd40 con 0x7fc7101052f0 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7101052f0 0x7fc710100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7101029c0 0x7fc710102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7101052f0 0x7fc710100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51082/0 (socket says 172.21.15.18:51082) 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.062+0000 7fc70e000640 1 -- 172.21.15.18:0/3635647859 learned_addr learned my addr 172.21.15.18:0/3635647859 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc70f400640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710107c20 0x7fc710100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc70e000640 1 -- 172.21.15.18:0/3635647859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710107c20 msgr2=0x7fc710100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc70e000640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710107c20 0x7fc710100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc70e000640 1 -- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7101029c0 msgr2=0x7fc710102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc70e000640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7101029c0 0x7fc710102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc70e000640 1 -- 172.21.15.18:0/3635647859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7101013f0 con 0x7fc7101052f0 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc70e000640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7101052f0 0x7fc710100970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e9c0 tx=0x7fc6fc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc714a00640 1 -- 172.21.15.18:0/3635647859 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00cd80 con 0x7fc7101052f0 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc714a00640 1 -- 172.21.15.18:0/3635647859 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc005500 con 0x7fc7101052f0 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7101016e0 con 0x7fc7101052f0 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.063+0000 7fc714a00640 1 -- 172.21.15.18:0/3635647859 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc010690 con 0x7fc7101052f0 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.064+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a6cf0 con 0x7fc7101052f0 2024-04-16T17:04:50.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.065+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71004fe00 con 0x7fc7101052f0 2024-04-16T17:04:50.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.067+0000 7fc714a00640 1 -- 172.21.15.18:0/3635647859 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc6fc0046c0 con 0x7fc7101052f0 2024-04-16T17:04:50.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.069+0000 7fc714a00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6f40a3eb0 0x7fc6f40a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:50.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.069+0000 7fc714a00640 1 -- 172.21.15.18:0/3635647859 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc6fc0d11c0 con 0x7fc7101052f0 2024-04-16T17:04:50.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.069+0000 7fc70ea00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6f40a3eb0 0x7fc6f40a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:50.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.070+0000 7fc70ea00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6f40a3eb0 0x7fc6f40a6370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc700009bd0 tx=0x7fc700005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:50.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.070+0000 7fc714a00640 1 -- 172.21.15.18:0/3635647859 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6fc09d200 con 0x7fc7101052f0 2024-04-16T17:04:50.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.339+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7100fdec0 con 0x7fc7101052f0 2024-04-16T17:04:50.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.340+0000 7fc714a00640 1 -- 172.21.15.18:0/3635647859 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7fc6fc09cb00 con 0x7fc7101052f0 2024-04-16T17:04:50.341 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:50.341 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:04:50.342 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:04:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.342+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6f40a3eb0 msgr2=0x7fc6f40a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.342+0000 7fc70fe00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6f40a3eb0 0x7fc6f40a6370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc700009bd0 tx=0x7fc700005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.342+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7101052f0 msgr2=0x7fc710100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.342+0000 7fc70fe00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7101052f0 0x7fc710100970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e9c0 tx=0x7fc6fc00ee80 comp rx=0 tx=0).stop 2024-04-16T17:04:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.343+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 shutdown_connections 2024-04-16T17:04:50.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.343+0000 7fc70fe00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6f40a3eb0 0x7fc6f40a6370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.343+0000 7fc70fe00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710107c20 0x7fc710100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.343+0000 7fc70fe00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7101052f0 0x7fc710100970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.343+0000 7fc70fe00640 1 --2- 172.21.15.18:0/3635647859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7101029c0 0x7fc710102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.343+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 >> 172.21.15.18:0/3635647859 conn(0x7fc7100fc3d0 msgr2=0x7fc710103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.343+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 shutdown_connections 2024-04-16T17:04:50.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.343+0000 7fc70fe00640 1 -- 172.21.15.18:0/3635647859 wait complete. 2024-04-16T17:04:50.361 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-16T17:04:50.361 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-16T17:04:50.361 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:04:50.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 -- 172.21.15.18:0/1682639571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6124077c00 msgr2=0x7f6124078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 --2- 172.21.15.18:0/1682639571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6124077c00 0x7f6124078060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6110009900 tx=0x7f611002bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:50.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 -- 172.21.15.18:0/1682639571 shutdown_connections 2024-04-16T17:04:50.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 --2- 172.21.15.18:0/1682639571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61240785a0 0x7f6124078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 --2- 172.21.15.18:0/1682639571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6124077c00 0x7f6124078060 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 --2- 172.21.15.18:0/1682639571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61240769c0 0x7f6124076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 -- 172.21.15.18:0/1682639571 >> 172.21.15.18:0/1682639571 conn(0x7f61240ff880 msgr2=0x7f6124101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:50.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 -- 172.21.15.18:0/1682639571 shutdown_connections 2024-04-16T17:04:50.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.554+0000 7f612b600640 1 -- 172.21.15.18:0/1682639571 wait complete. 2024-04-16T17:04:50.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.555+0000 7f612b600640 1 Processor -- start 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f612b600640 1 -- start start 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f612b600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61240769c0 0x7f612419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f612b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6124077c00 0x7f612419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f612b600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61240785a0 0x7f61241a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f612b600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f61241142f0 con 0x7f61240769c0 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f612b600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6124114170 con 0x7f61240785a0 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f612b600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6124114470 con 0x7f6124077c00 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f6128c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61240769c0 0x7f612419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f6123e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6124077c00 0x7f612419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f6128c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61240769c0 0x7f612419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55540/0 (socket says 172.21.15.18:55540) 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f6128c00640 1 -- 172.21.15.18:0/3446910927 learned_addr learned my addr 172.21.15.18:0/3446910927 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.556+0000 7f6129600640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61240785a0 0x7f61241a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6128c00640 1 -- 172.21.15.18:0/3446910927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6124077c00 msgr2=0x7f612419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6128c00640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6124077c00 0x7f612419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6128c00640 1 -- 172.21.15.18:0/3446910927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61240785a0 msgr2=0x7f61241a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6128c00640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61240785a0 0x7f61241a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6128c00640 1 -- 172.21.15.18:0/3446910927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61241a6560 con 0x7f61240769c0 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6128c00640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61240769c0 0x7f612419e940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f611800e9f0 tx=0x7f611800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6121600640 1 -- 172.21.15.18:0/3446910927 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800cd80 con 0x7f61240769c0 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6121600640 1 -- 172.21.15.18:0/3446910927 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6118010910 con 0x7f61240769c0 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61241a6850 con 0x7f61240769c0 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.557+0000 7f6121600640 1 -- 172.21.15.18:0/3446910927 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118018990 con 0x7f61240769c0 2024-04-16T17:04:50.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.558+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a6d70 con 0x7f61240769c0 2024-04-16T17:04:50.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.559+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6124110dd0 con 0x7f61240769c0 2024-04-16T17:04:50.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.561+0000 7f6121600640 1 -- 172.21.15.18:0/3446910927 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6118010ac0 con 0x7f61240769c0 2024-04-16T17:04:50.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.562+0000 7f6121600640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f610c0a3e40 0x7f610c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:50.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.563+0000 7f6121600640 1 -- 172.21.15.18:0/3446910927 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f61180d11f0 con 0x7f61240769c0 2024-04-16T17:04:50.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.563+0000 7f6123e00640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f610c0a3e40 0x7f610c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:50.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.564+0000 7f6123e00640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f610c0a3e40 0x7f610c0a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f611002c7b0 tx=0x7f6110005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:50.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.569+0000 7f6121600640 1 -- 172.21.15.18:0/3446910927 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f611809cbe0 con 0x7f61240769c0 2024-04-16T17:04:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:50 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:04:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:50 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:04:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:50 smithi018 ceph-mon[31776]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-16T17:04:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:50 smithi018 ceph-mon[31776]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T17:04:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:50 smithi018 ceph-mon[31776]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T17:04:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:50 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2584144377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:50.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:50 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3635647859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:50.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:50 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:04:50.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:50 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:04:50.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:50 smithi043 ceph-mon[36901]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-16T17:04:50.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:50 smithi043 ceph-mon[36901]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T17:04:50.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:50 smithi043 ceph-mon[36901]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T17:04:50.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:50 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2584144377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:50.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:50 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3635647859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:50.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.772+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6124062440 con 0x7f610c0a3e40 2024-04-16T17:04:50.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.784+0000 7f6121600640 1 -- 172.21.15.18:0/3446910927 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339716 (secure 0 0 0) 0x7f6124062440 con 0x7f610c0a3e40 2024-04-16T17:04:50.787 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:50.793 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:04:50.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f610c0a3e40 msgr2=0x7f610c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f610c0a3e40 0x7f610c0a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f611002c7b0 tx=0x7f6110005fd0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61240769c0 msgr2=0x7f612419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61240769c0 0x7f612419e940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f611800e9f0 tx=0x7f611800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 shutdown_connections 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f610c0a3e40 0x7f610c0a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61240785a0 0x7f61241a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6124077c00 0x7f612419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 --2- 172.21.15.18:0/3446910927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61240769c0 0x7f612419e940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.789+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 >> 172.21.15.18:0/3446910927 conn(0x7f61240ff880 msgr2=0x7f6124100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.790+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 shutdown_connections 2024-04-16T17:04:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:50.790+0000 7f612b600640 1 -- 172.21.15.18:0/3446910927 wait complete. 2024-04-16T17:04:50.809 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-16T17:04:50.125915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191431,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1249,"num_read_kb":792,"num_write":475,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378580,"kb_used_data":12740,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563244,"statfs":{"total":1151940427776,"available":1151552761856,"internally_reserved":0,"allocated":13045760,"data_stored":9908102,"data_compressed":543993,"data_compressed_allocated":737280,"data_compressed_original":4325376,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.970620"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104573+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:04:44.104573+0000","last_peered":"2024-04-16T17:04:44.104573+0000","last_clean":"2024-04-16T17:04:44.104573+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:04:44.104573+0000","last_undegraded":"2024-04-16T17:04:44.104573+0000","last_fullsized":"2024-04-16T17:04:44.104573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419294+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:04:44.419294+0000","last_peered":"2024-04-16T17:04:44.419294+0000","last_clean":"2024-04-16T17:04:44.419294+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:04:44.419294+0000","last_undegraded":"2024-04-16T17:04:44.419294+0000","last_fullsized":"2024-04-16T17:04:44.419294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418965+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:04:44.418965+0000","last_peered":"2024-04-16T17:04:44.418965+0000","last_clean":"2024-04-16T17:04:44.418965+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:04:44.418965+0000","last_undegraded":"2024-04-16T17:04:44.418965+0000","last_fullsized":"2024-04-16T17:04:44.418965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237775+0000","last_change":"2024-04-16T17:02:26.318719+0000","last_active":"2024-04-16T17:04:44.237775+0000","last_peered":"2024-04-16T17:04:44.237775+0000","last_clean":"2024-04-16T17:04:44.237775+0000","last_became_active":"2024-04-16T17:02:26.317743+0000","last_became_peered":"2024-04-16T17:02:26.317743+0000","last_unstale":"2024-04-16T17:04:44.237775+0000","last_undegraded":"2024-04-16T17:04:44.237775+0000","last_fullsized":"2024-04-16T17:04:44.237775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:26.827859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240559+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:04:44.240559+0000","last_peered":"2024-04-16T17:04:44.240559+0000","last_clean":"2024-04-16T17:04:44.240559+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:04:44.240559+0000","last_undegraded":"2024-04-16T17:04:44.240559+0000","last_fullsized":"2024-04-16T17:04:44.240559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104192+0000","last_change":"2024-04-16T17:02:26.315910+0000","last_active":"2024-04-16T17:04:44.104192+0000","last_peered":"2024-04-16T17:04:44.104192+0000","last_clean":"2024-04-16T17:04:44.104192+0000","last_became_active":"2024-04-16T17:02:26.315493+0000","last_became_peered":"2024-04-16T17:02:26.315493+0000","last_unstale":"2024-04-16T17:04:44.104192+0000","last_undegraded":"2024-04-16T17:04:44.104192+0000","last_fullsized":"2024-04-16T17:04:44.104192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:27:48.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114020+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:04:44.114020+0000","last_peered":"2024-04-16T17:04:44.114020+0000","last_clean":"2024-04-16T17:04:44.114020+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:04:44.114020+0000","last_undegraded":"2024-04-16T17:04:44.114020+0000","last_fullsized":"2024-04-16T17:04:44.114020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240629+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:04:44.240629+0000","last_peered":"2024-04-16T17:04:44.240629+0000","last_clean":"2024-04-16T17:04:44.240629+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:04:44.240629+0000","last_undegraded":"2024-04-16T17:04:44.240629+0000","last_fullsized":"2024-04-16T17:04:44.240629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207849+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:04:44.207849+0000","last_peered":"2024-04-16T17:04:44.207849+0000","last_clean":"2024-04-16T17:04:44.207849+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:04:44.207849+0000","last_undegraded":"2024-04-16T17:04:44.207849+0000","last_fullsized":"2024-04-16T17:04:44.207849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104511+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:04:44.104511+0000","last_peered":"2024-04-16T17:04:44.104511+0000","last_clean":"2024-04-16T17:04:44.104511+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:04:44.104511+0000","last_undegraded":"2024-04-16T17:04:44.104511+0000","last_fullsized":"2024-04-16T17:04:44.104511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114664+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:04:44.114664+0000","last_peered":"2024-04-16T17:04:44.114664+0000","last_clean":"2024-04-16T17:04:44.114664+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:04:44.114664+0000","last_undegraded":"2024-04-16T17:04:44.114664+0000","last_fullsized":"2024-04-16T17:04:44.114664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414524+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:04:44.414524+0000","last_peered":"2024-04-16T17:04:44.414524+0000","last_clean":"2024-04-16T17:04:44.414524+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:04:44.414524+0000","last_undegraded":"2024-04-16T17:04:44.414524+0000","last_fullsized":"2024-04-16T17:04:44.414524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415190+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:04:44.415190+0000","last_peered":"2024-04-16T17:04:44.415190+0000","last_clean":"2024-04-16T17:04:44.415190+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:04:44.415190+0000","last_undegraded":"2024-04-16T17:04:44.415190+0000","last_fullsized":"2024-04-16T17:04:44.415190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418945+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:04:44.418945+0000","last_peered":"2024-04-16T17:04:44.418945+0000","last_clean":"2024-04-16T17:04:44.418945+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:04:44.418945+0000","last_undegraded":"2024-04-16T17:04:44.418945+0000","last_fullsized":"2024-04-16T17:04:44.418945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419182+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:04:44.419182+0000","last_peered":"2024-04-16T17:04:44.419182+0000","last_clean":"2024-04-16T17:04:44.419182+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:04:44.419182+0000","last_undegraded":"2024-04-16T17:04:44.419182+0000","last_fullsized":"2024-04-16T17:04:44.419182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207293+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:04:44.207293+0000","last_peered":"2024-04-16T17:04:44.207293+0000","last_clean":"2024-04-16T17:04:44.207293+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:04:44.207293+0000","last_undegraded":"2024-04-16T17:04:44.207293+0000","last_fullsized":"2024-04-16T17:04:44.207293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104522+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:04:44.104522+0000","last_peered":"2024-04-16T17:04:44.104522+0000","last_clean":"2024-04-16T17:04:44.104522+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:04:44.104522+0000","last_undegraded":"2024-04-16T17:04:44.104522+0000","last_fullsized":"2024-04-16T17:04:44.104522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.238023+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:04:44.238023+0000","last_peered":"2024-04-16T17:04:44.238023+0000","last_clean":"2024-04-16T17:04:44.238023+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:04:44.238023+0000","last_undegraded":"2024-04-16T17:04:44.238023+0000","last_fullsized":"2024-04-16T17:04:44.238023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275555+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:04:44.275555+0000","last_peered":"2024-04-16T17:04:44.275555+0000","last_clean":"2024-04-16T17:04:44.275555+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:04:44.275555+0000","last_undegraded":"2024-04-16T17:04:44.275555+0000","last_fullsized":"2024-04-16T17:04:44.275555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274972+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:04:44.274972+0000","last_peered":"2024-04-16T17:04:44.274972+0000","last_clean":"2024-04-16T17:04:44.274972+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:04:44.274972+0000","last_undegraded":"2024-04-16T17:04:44.274972+0000","last_fullsized":"2024-04-16T17:04:44.274972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237649+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:04:44.237649+0000","last_peered":"2024-04-16T17:04:44.237649+0000","last_clean":"2024-04-16T17:04:44.237649+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:04:44.237649+0000","last_undegraded":"2024-04-16T17:04:44.237649+0000","last_fullsized":"2024-04-16T17:04:44.237649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.238023+0000","last_change":"2024-04-16T17:02:26.318546+0000","last_active":"2024-04-16T17:04:44.238023+0000","last_peered":"2024-04-16T17:04:44.238023+0000","last_clean":"2024-04-16T17:04:44.238023+0000","last_became_active":"2024-04-16T17:02:26.317794+0000","last_became_peered":"2024-04-16T17:02:26.317794+0000","last_unstale":"2024-04-16T17:04:44.238023+0000","last_undegraded":"2024-04-16T17:04:44.238023+0000","last_fullsized":"2024-04-16T17:04:44.238023+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:48.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414178+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:04:44.414178+0000","last_peered":"2024-04-16T17:04:44.414178+0000","last_clean":"2024-04-16T17:04:44.414178+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:04:44.414178+0000","last_undegraded":"2024-04-16T17:04:44.414178+0000","last_fullsized":"2024-04-16T17:04:44.414178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241729+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:04:44.241729+0000","last_peered":"2024-04-16T17:04:44.241729+0000","last_clean":"2024-04-16T17:04:44.241729+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:04:44.241729+0000","last_undegraded":"2024-04-16T17:04:44.241729+0000","last_fullsized":"2024-04-16T17:04:44.241729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419515+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:04:44.419515+0000","last_peered":"2024-04-16T17:04:44.419515+0000","last_clean":"2024-04-16T17:04:44.419515+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:04:44.419515+0000","last_undegraded":"2024-04-16T17:04:44.419515+0000","last_fullsized":"2024-04-16T17:04:44.419515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414312+0000","last_change":"2024-04-16T17:03:27.271585+0000","last_active":"2024-04-16T17:04:44.414312+0000","last_peered":"2024-04-16T17:04:44.414312+0000","last_clean":"2024-04-16T17:04:44.414312+0000","last_became_active":"2024-04-16T17:03:27.271251+0000","last_became_peered":"2024-04-16T17:03:27.271251+0000","last_unstale":"2024-04-16T17:04:44.414312+0000","last_undegraded":"2024-04-16T17:04:44.414312+0000","last_fullsized":"2024-04-16T17:04:44.414312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:51.217466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241064+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:04:44.241064+0000","last_peered":"2024-04-16T17:04:44.241064+0000","last_clean":"2024-04-16T17:04:44.241064+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:04:44.241064+0000","last_undegraded":"2024-04-16T17:04:44.241064+0000","last_fullsized":"2024-04-16T17:04:44.241064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207578+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:04:44.207578+0000","last_peered":"2024-04-16T17:04:44.207578+0000","last_clean":"2024-04-16T17:04:44.207578+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:04:44.207578+0000","last_undegraded":"2024-04-16T17:04:44.207578+0000","last_fullsized":"2024-04-16T17:04:44.207578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414211+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:04:44.414211+0000","last_peered":"2024-04-16T17:04:44.414211+0000","last_clean":"2024-04-16T17:04:44.414211+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:04:44.414211+0000","last_undegraded":"2024-04-16T17:04:44.414211+0000","last_fullsized":"2024-04-16T17:04:44.414211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240928+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:04:44.240928+0000","last_peered":"2024-04-16T17:04:44.240928+0000","last_clean":"2024-04-16T17:04:44.240928+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:04:44.240928+0000","last_undegraded":"2024-04-16T17:04:44.240928+0000","last_fullsized":"2024-04-16T17:04:44.240928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248909+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:04:44.248909+0000","last_peered":"2024-04-16T17:04:44.248909+0000","last_clean":"2024-04-16T17:04:44.248909+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:04:44.248909+0000","last_undegraded":"2024-04-16T17:04:44.248909+0000","last_fullsized":"2024-04-16T17:04:44.248909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275231+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:04:44.275231+0000","last_peered":"2024-04-16T17:04:44.275231+0000","last_clean":"2024-04-16T17:04:44.275231+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:04:44.275231+0000","last_undegraded":"2024-04-16T17:04:44.275231+0000","last_fullsized":"2024-04-16T17:04:44.275231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":223,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.506138+0000","last_change":"2024-04-16T17:02:26.315434+0000","last_active":"2024-04-16T17:04:44.506138+0000","last_peered":"2024-04-16T17:04:44.506138+0000","last_clean":"2024-04-16T17:04:44.506138+0000","last_became_active":"2024-04-16T17:02:26.310169+0000","last_became_peered":"2024-04-16T17:02:26.310169+0000","last_unstale":"2024-04-16T17:04:44.506138+0000","last_undegraded":"2024-04-16T17:04:44.506138+0000","last_fullsized":"2024-04-16T17:04:44.506138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:51.033415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":164,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240363+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:04:44.240363+0000","last_peered":"2024-04-16T17:04:44.240363+0000","last_clean":"2024-04-16T17:04:44.240363+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:04:44.240363+0000","last_undegraded":"2024-04-16T17:04:44.240363+0000","last_fullsized":"2024-04-16T17:04:44.240363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275338+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:04:44.275338+0000","last_peered":"2024-04-16T17:04:44.275338+0000","last_clean":"2024-04-16T17:04:44.275338+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:04:44.275338+0000","last_undegraded":"2024-04-16T17:04:44.275338+0000","last_fullsized":"2024-04-16T17:04:44.275338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104007+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:04:44.104007+0000","last_peered":"2024-04-16T17:04:44.104007+0000","last_clean":"2024-04-16T17:04:44.104007+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:04:44.104007+0000","last_undegraded":"2024-04-16T17:04:44.104007+0000","last_fullsized":"2024-04-16T17:04:44.104007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237470+0000","last_change":"2024-04-16T17:02:26.321763+0000","last_active":"2024-04-16T17:04:44.237470+0000","last_peered":"2024-04-16T17:04:44.237470+0000","last_clean":"2024-04-16T17:04:44.237470+0000","last_became_active":"2024-04-16T17:02:26.321438+0000","last_became_peered":"2024-04-16T17:02:26.321438+0000","last_unstale":"2024-04-16T17:04:44.237470+0000","last_undegraded":"2024-04-16T17:04:44.237470+0000","last_fullsized":"2024-04-16T17:04:44.237470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:47.231035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104252+0000","last_change":"2024-04-16T17:03:27.451220+0000","last_active":"2024-04-16T17:04:44.104252+0000","last_peered":"2024-04-16T17:04:44.104252+0000","last_clean":"2024-04-16T17:04:44.104252+0000","last_became_active":"2024-04-16T17:03:27.450680+0000","last_became_peered":"2024-04-16T17:03:27.450680+0000","last_unstale":"2024-04-16T17:04:44.104252+0000","last_undegraded":"2024-04-16T17:04:44.104252+0000","last_fullsized":"2024-04-16T17:04:44.104252+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:23.270332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414603+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:04:44.414603+0000","last_peered":"2024-04-16T17:04:44.414603+0000","last_clean":"2024-04-16T17:04:44.414603+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:04:44.414603+0000","last_undegraded":"2024-04-16T17:04:44.414603+0000","last_fullsized":"2024-04-16T17:04:44.414603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241007+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:04:44.241007+0000","last_peered":"2024-04-16T17:04:44.241007+0000","last_clean":"2024-04-16T17:04:44.241007+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:04:44.241007+0000","last_undegraded":"2024-04-16T17:04:44.241007+0000","last_fullsized":"2024-04-16T17:04:44.241007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419271+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:04:44.419271+0000","last_peered":"2024-04-16T17:04:44.419271+0000","last_clean":"2024-04-16T17:04:44.419271+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:04:44.419271+0000","last_undegraded":"2024-04-16T17:04:44.419271+0000","last_fullsized":"2024-04-16T17:04:44.419271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248395+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:04:44.248395+0000","last_peered":"2024-04-16T17:04:44.248395+0000","last_clean":"2024-04-16T17:04:44.248395+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:04:44.248395+0000","last_undegraded":"2024-04-16T17:04:44.248395+0000","last_fullsized":"2024-04-16T17:04:44.248395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237507+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:04:44.237507+0000","last_peered":"2024-04-16T17:04:44.237507+0000","last_clean":"2024-04-16T17:04:44.237507+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:04:44.237507+0000","last_undegraded":"2024-04-16T17:04:44.237507+0000","last_fullsized":"2024-04-16T17:04:44.237507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240883+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:04:44.240883+0000","last_peered":"2024-04-16T17:04:44.240883+0000","last_clean":"2024-04-16T17:04:44.240883+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:04:44.240883+0000","last_undegraded":"2024-04-16T17:04:44.240883+0000","last_fullsized":"2024-04-16T17:04:44.240883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":204,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104418+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:04:44.104418+0000","last_peered":"2024-04-16T17:04:44.104418+0000","last_clean":"2024-04-16T17:04:44.104418+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:04:44.104418+0000","last_undegraded":"2024-04-16T17:04:44.104418+0000","last_fullsized":"2024-04-16T17:04:44.104418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207667+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:04:44.207667+0000","last_peered":"2024-04-16T17:04:44.207667+0000","last_clean":"2024-04-16T17:04:44.207667+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:04:44.207667+0000","last_undegraded":"2024-04-16T17:04:44.207667+0000","last_fullsized":"2024-04-16T17:04:44.207667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237402+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:04:44.237402+0000","last_peered":"2024-04-16T17:04:44.237402+0000","last_clean":"2024-04-16T17:04:44.237402+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:04:44.237402+0000","last_undegraded":"2024-04-16T17:04:44.237402+0000","last_fullsized":"2024-04-16T17:04:44.237402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114484+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:04:44.114484+0000","last_peered":"2024-04-16T17:04:44.114484+0000","last_clean":"2024-04-16T17:04:44.114484+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:04:44.114484+0000","last_undegraded":"2024-04-16T17:04:44.114484+0000","last_fullsized":"2024-04-16T17:04:44.114484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419897+0000","last_change":"2024-04-16T17:02:26.752275+0000","last_active":"2024-04-16T17:04:44.419897+0000","last_peered":"2024-04-16T17:04:44.419897+0000","last_clean":"2024-04-16T17:04:44.419897+0000","last_became_active":"2024-04-16T17:02:26.751729+0000","last_became_peered":"2024-04-16T17:02:26.751729+0000","last_unstale":"2024-04-16T17:04:44.419897+0000","last_undegraded":"2024-04-16T17:04:44.419897+0000","last_fullsized":"2024-04-16T17:04:44.419897+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:41:35.777691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414493+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:04:44.414493+0000","last_peered":"2024-04-16T17:04:44.414493+0000","last_clean":"2024-04-16T17:04:44.414493+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:04:44.414493+0000","last_undegraded":"2024-04-16T17:04:44.414493+0000","last_fullsized":"2024-04-16T17:04:44.414493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248851+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:04:44.248851+0000","last_peered":"2024-04-16T17:04:44.248851+0000","last_clean":"2024-04-16T17:04:44.248851+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:04:44.248851+0000","last_undegraded":"2024-04-16T17:04:44.248851+0000","last_fullsized":"2024-04-16T17:04:44.248851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207706+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:04:44.207706+0000","last_peered":"2024-04-16T17:04:44.207706+0000","last_clean":"2024-04-16T17:04:44.207706+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:04:44.207706+0000","last_undegraded":"2024-04-16T17:04:44.207706+0000","last_fullsized":"2024-04-16T17:04:44.207706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103750+0000","last_change":"2024-04-16T17:03:27.054627+0000","last_active":"2024-04-16T17:04:44.103750+0000","last_peered":"2024-04-16T17:04:44.103750+0000","last_clean":"2024-04-16T17:04:44.103750+0000","last_became_active":"2024-04-16T17:03:27.054326+0000","last_became_peered":"2024-04-16T17:03:27.054326+0000","last_unstale":"2024-04-16T17:04:44.103750+0000","last_undegraded":"2024-04-16T17:04:44.103750+0000","last_fullsized":"2024-04-16T17:04:44.103750+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:43.535958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":419,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.505895+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:04:44.505895+0000","last_peered":"2024-04-16T17:04:44.505895+0000","last_clean":"2024-04-16T17:04:44.505895+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:04:44.505895+0000","last_undegraded":"2024-04-16T17:04:44.505895+0000","last_fullsized":"2024-04-16T17:04:44.505895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70920,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":344,"num_read_kb":243,"num_write":42,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240855+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:04:44.240855+0000","last_peered":"2024-04-16T17:04:44.240855+0000","last_clean":"2024-04-16T17:04:44.240855+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:04:44.240855+0000","last_undegraded":"2024-04-16T17:04:44.240855+0000","last_fullsized":"2024-04-16T17:04:44.240855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.249232+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:04:44.249232+0000","last_peered":"2024-04-16T17:04:44.249232+0000","last_clean":"2024-04-16T17:04:44.249232+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:04:44.249232+0000","last_undegraded":"2024-04-16T17:04:44.249232+0000","last_fullsized":"2024-04-16T17:04:44.249232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275182+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:04:44.275182+0000","last_peered":"2024-04-16T17:04:44.275182+0000","last_clean":"2024-04-16T17:04:44.275182+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:04:44.275182+0000","last_undegraded":"2024-04-16T17:04:44.275182+0000","last_fullsized":"2024-04-16T17:04:44.275182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415497+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:04:44.415497+0000","last_peered":"2024-04-16T17:04:44.415497+0000","last_clean":"2024-04-16T17:04:44.415497+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:04:44.415497+0000","last_undegraded":"2024-04-16T17:04:44.415497+0000","last_fullsized":"2024-04-16T17:04:44.415497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104076+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:04:44.104076+0000","last_peered":"2024-04-16T17:04:44.104076+0000","last_clean":"2024-04-16T17:04:44.104076+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:04:44.104076+0000","last_undegraded":"2024-04-16T17:04:44.104076+0000","last_fullsized":"2024-04-16T17:04:44.104076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248492+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:04:44.248492+0000","last_peered":"2024-04-16T17:04:44.248492+0000","last_clean":"2024-04-16T17:04:44.248492+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:04:44.248492+0000","last_undegraded":"2024-04-16T17:04:44.248492+0000","last_fullsized":"2024-04-16T17:04:44.248492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114507+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:04:44.114507+0000","last_peered":"2024-04-16T17:04:44.114507+0000","last_clean":"2024-04-16T17:04:44.114507+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:04:44.114507+0000","last_undegraded":"2024-04-16T17:04:44.114507+0000","last_fullsized":"2024-04-16T17:04:44.114507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237521+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:04:44.237521+0000","last_peered":"2024-04-16T17:04:44.237521+0000","last_clean":"2024-04-16T17:04:44.237521+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:04:44.237521+0000","last_undegraded":"2024-04-16T17:04:44.237521+0000","last_fullsized":"2024-04-16T17:04:44.237521+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":208,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.536313+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:04:44.536313+0000","last_peered":"2024-04-16T17:04:44.536313+0000","last_clean":"2024-04-16T17:04:44.536313+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:04:44.536313+0000","last_undegraded":"2024-04-16T17:04:44.536313+0000","last_fullsized":"2024-04-16T17:04:44.536313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":170,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.113893+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:04:44.113893+0000","last_peered":"2024-04-16T17:04:44.113893+0000","last_clean":"2024-04-16T17:04:44.113893+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:04:44.113893+0000","last_undegraded":"2024-04-16T17:04:44.113893+0000","last_fullsized":"2024-04-16T17:04:44.113893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415000+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:04:44.415000+0000","last_peered":"2024-04-16T17:04:44.415000+0000","last_clean":"2024-04-16T17:04:44.415000+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:04:44.415000+0000","last_undegraded":"2024-04-16T17:04:44.415000+0000","last_fullsized":"2024-04-16T17:04:44.415000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240346+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:04:44.240346+0000","last_peered":"2024-04-16T17:04:44.240346+0000","last_clean":"2024-04-16T17:04:44.240346+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:04:44.240346+0000","last_undegraded":"2024-04-16T17:04:44.240346+0000","last_fullsized":"2024-04-16T17:04:44.240346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248651+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:04:44.248651+0000","last_peered":"2024-04-16T17:04:44.248651+0000","last_clean":"2024-04-16T17:04:44.248651+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:04:44.248651+0000","last_undegraded":"2024-04-16T17:04:44.248651+0000","last_fullsized":"2024-04-16T17:04:44.248651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248417+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:04:44.248417+0000","last_peered":"2024-04-16T17:04:44.248417+0000","last_clean":"2024-04-16T17:04:44.248417+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:04:44.248417+0000","last_undegraded":"2024-04-16T17:04:44.248417+0000","last_fullsized":"2024-04-16T17:04:44.248417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104487+0000","last_change":"2024-04-16T17:02:26.320405+0000","last_active":"2024-04-16T17:04:44.104487+0000","last_peered":"2024-04-16T17:04:44.104487+0000","last_clean":"2024-04-16T17:04:44.104487+0000","last_became_active":"2024-04-16T17:02:26.307921+0000","last_became_peered":"2024-04-16T17:02:26.307921+0000","last_unstale":"2024-04-16T17:04:44.104487+0000","last_undegraded":"2024-04-16T17:04:44.104487+0000","last_fullsized":"2024-04-16T17:04:44.104487+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:19.156260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104216+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:04:44.104216+0000","last_peered":"2024-04-16T17:04:44.104216+0000","last_clean":"2024-04-16T17:04:44.104216+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:04:44.104216+0000","last_undegraded":"2024-04-16T17:04:44.104216+0000","last_fullsized":"2024-04-16T17:04:44.104216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240562+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:04:44.240562+0000","last_peered":"2024-04-16T17:04:44.240562+0000","last_clean":"2024-04-16T17:04:44.240562+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:04:44.240562+0000","last_undegraded":"2024-04-16T17:04:44.240562+0000","last_fullsized":"2024-04-16T17:04:44.240562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237720+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:04:44.237720+0000","last_peered":"2024-04-16T17:04:44.237720+0000","last_clean":"2024-04-16T17:04:44.237720+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:04:44.237720+0000","last_undegraded":"2024-04-16T17:04:44.237720+0000","last_fullsized":"2024-04-16T17:04:44.237720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104042+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:04:44.104042+0000","last_peered":"2024-04-16T17:04:44.104042+0000","last_clean":"2024-04-16T17:04:44.104042+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:04:44.104042+0000","last_undegraded":"2024-04-16T17:04:44.104042+0000","last_fullsized":"2024-04-16T17:04:44.104042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414819+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:04:44.414819+0000","last_peered":"2024-04-16T17:04:44.414819+0000","last_clean":"2024-04-16T17:04:44.414819+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:04:44.414819+0000","last_undegraded":"2024-04-16T17:04:44.414819+0000","last_fullsized":"2024-04-16T17:04:44.414819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114221+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:04:44.114221+0000","last_peered":"2024-04-16T17:04:44.114221+0000","last_clean":"2024-04-16T17:04:44.114221+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:04:44.114221+0000","last_undegraded":"2024-04-16T17:04:44.114221+0000","last_fullsized":"2024-04-16T17:04:44.114221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104271+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:04:44.104271+0000","last_peered":"2024-04-16T17:04:44.104271+0000","last_clean":"2024-04-16T17:04:44.104271+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:04:44.104271+0000","last_undegraded":"2024-04-16T17:04:44.104271+0000","last_fullsized":"2024-04-16T17:04:44.104271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415017+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:04:44.415017+0000","last_peered":"2024-04-16T17:04:44.415017+0000","last_clean":"2024-04-16T17:04:44.415017+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:04:44.415017+0000","last_undegraded":"2024-04-16T17:04:44.415017+0000","last_fullsized":"2024-04-16T17:04:44.415017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248603+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:04:44.248603+0000","last_peered":"2024-04-16T17:04:44.248603+0000","last_clean":"2024-04-16T17:04:44.248603+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:04:44.248603+0000","last_undegraded":"2024-04-16T17:04:44.248603+0000","last_fullsized":"2024-04-16T17:04:44.248603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":205,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.417263+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:04:44.417263+0000","last_peered":"2024-04-16T17:04:44.417263+0000","last_clean":"2024-04-16T17:04:44.417263+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:04:44.417263+0000","last_undegraded":"2024-04-16T17:04:44.417263+0000","last_fullsized":"2024-04-16T17:04:44.417263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114408+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:04:44.114408+0000","last_peered":"2024-04-16T17:04:44.114408+0000","last_clean":"2024-04-16T17:04:44.114408+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:04:44.114408+0000","last_undegraded":"2024-04-16T17:04:44.114408+0000","last_fullsized":"2024-04-16T17:04:44.114408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240965+0000","last_change":"2024-04-16T17:02:26.322313+0000","last_active":"2024-04-16T17:04:44.240965+0000","last_peered":"2024-04-16T17:04:44.240965+0000","last_clean":"2024-04-16T17:04:44.240965+0000","last_became_active":"2024-04-16T17:02:26.320328+0000","last_became_peered":"2024-04-16T17:02:26.320328+0000","last_unstale":"2024-04-16T17:04:44.240965+0000","last_undegraded":"2024-04-16T17:04:44.240965+0000","last_fullsized":"2024-04-16T17:04:44.240965+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:40.943183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414366+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:04:44.414366+0000","last_peered":"2024-04-16T17:04:44.414366+0000","last_clean":"2024-04-16T17:04:44.414366+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:04:44.414366+0000","last_undegraded":"2024-04-16T17:04:44.414366+0000","last_fullsized":"2024-04-16T17:04:44.414366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.113827+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:04:44.113827+0000","last_peered":"2024-04-16T17:04:44.113827+0000","last_clean":"2024-04-16T17:04:44.113827+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:04:44.113827+0000","last_undegraded":"2024-04-16T17:04:44.113827+0000","last_fullsized":"2024-04-16T17:04:44.113827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248980+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:04:44.248980+0000","last_peered":"2024-04-16T17:04:44.248980+0000","last_clean":"2024-04-16T17:04:44.248980+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:04:44.248980+0000","last_undegraded":"2024-04-16T17:04:44.248980+0000","last_fullsized":"2024-04-16T17:04:44.248980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414448+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:04:44.414448+0000","last_peered":"2024-04-16T17:04:44.414448+0000","last_clean":"2024-04-16T17:04:44.414448+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:04:44.414448+0000","last_undegraded":"2024-04-16T17:04:44.414448+0000","last_fullsized":"2024-04-16T17:04:44.414448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103918+0000","last_change":"2024-04-16T17:02:26.312852+0000","last_active":"2024-04-16T17:04:44.103918+0000","last_peered":"2024-04-16T17:04:44.103918+0000","last_clean":"2024-04-16T17:04:44.103918+0000","last_became_active":"2024-04-16T17:02:26.308279+0000","last_became_peered":"2024-04-16T17:02:26.308279+0000","last_unstale":"2024-04-16T17:04:44.103918+0000","last_undegraded":"2024-04-16T17:04:44.103918+0000","last_fullsized":"2024-04-16T17:04:44.103918+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:25.511349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103933+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:04:44.103933+0000","last_peered":"2024-04-16T17:04:44.103933+0000","last_clean":"2024-04-16T17:04:44.103933+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:04:44.103933+0000","last_undegraded":"2024-04-16T17:04:44.103933+0000","last_fullsized":"2024-04-16T17:04:44.103933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240849+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:04:44.240849+0000","last_peered":"2024-04-16T17:04:44.240849+0000","last_clean":"2024-04-16T17:04:44.240849+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:04:44.240849+0000","last_undegraded":"2024-04-16T17:04:44.240849+0000","last_fullsized":"2024-04-16T17:04:44.240849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415023+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:04:44.415023+0000","last_peered":"2024-04-16T17:04:44.415023+0000","last_clean":"2024-04-16T17:04:44.415023+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:04:44.415023+0000","last_undegraded":"2024-04-16T17:04:44.415023+0000","last_fullsized":"2024-04-16T17:04:44.415023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240499+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:04:44.240499+0000","last_peered":"2024-04-16T17:04:44.240499+0000","last_clean":"2024-04-16T17:04:44.240499+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:04:44.240499+0000","last_undegraded":"2024-04-16T17:04:44.240499+0000","last_fullsized":"2024-04-16T17:04:44.240499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'47","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.374558+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:04:44.374558+0000","last_peered":"2024-04-16T17:04:44.374558+0000","last_clean":"2024-04-16T17:04:44.374558+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:04:44.374558+0000","last_undegraded":"2024-04-16T17:04:44.374558+0000","last_fullsized":"2024-04-16T17:04:44.374558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241537+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:04:44.241537+0000","last_peered":"2024-04-16T17:04:44.241537+0000","last_clean":"2024-04-16T17:04:44.241537+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:04:44.241537+0000","last_undegraded":"2024-04-16T17:04:44.241537+0000","last_fullsized":"2024-04-16T17:04:44.241537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274800+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:04:44.274800+0000","last_peered":"2024-04-16T17:04:44.274800+0000","last_clean":"2024-04-16T17:04:44.274800+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:04:44.274800+0000","last_undegraded":"2024-04-16T17:04:44.274800+0000","last_fullsized":"2024-04-16T17:04:44.274800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414738+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:04:44.414738+0000","last_peered":"2024-04-16T17:04:44.414738+0000","last_clean":"2024-04-16T17:04:44.414738+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:04:44.414738+0000","last_undegraded":"2024-04-16T17:04:44.414738+0000","last_fullsized":"2024-04-16T17:04:44.414738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419383+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:04:44.419383+0000","last_peered":"2024-04-16T17:04:44.419383+0000","last_clean":"2024-04-16T17:04:44.419383+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:04:44.419383+0000","last_undegraded":"2024-04-16T17:04:44.419383+0000","last_fullsized":"2024-04-16T17:04:44.419383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414608+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:04:44.414608+0000","last_peered":"2024-04-16T17:04:44.414608+0000","last_clean":"2024-04-16T17:04:44.414608+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:04:44.414608+0000","last_undegraded":"2024-04-16T17:04:44.414608+0000","last_fullsized":"2024-04-16T17:04:44.414608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207210+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:04:44.207210+0000","last_peered":"2024-04-16T17:04:44.207210+0000","last_clean":"2024-04-16T17:04:44.207210+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:04:44.207210+0000","last_undegraded":"2024-04-16T17:04:44.207210+0000","last_fullsized":"2024-04-16T17:04:44.207210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240332+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:04:44.240332+0000","last_peered":"2024-04-16T17:04:44.240332+0000","last_clean":"2024-04-16T17:04:44.240332+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:04:44.240332+0000","last_undegraded":"2024-04-16T17:04:44.240332+0000","last_fullsized":"2024-04-16T17:04:44.240332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241095+0000","last_change":"2024-04-16T17:03:27.268760+0000","last_active":"2024-04-16T17:04:44.241095+0000","last_peered":"2024-04-16T17:04:44.241095+0000","last_clean":"2024-04-16T17:04:44.241095+0000","last_became_active":"2024-04-16T17:03:27.268209+0000","last_became_peered":"2024-04-16T17:03:27.268209+0000","last_unstale":"2024-04-16T17:04:44.241095+0000","last_undegraded":"2024-04-16T17:04:44.241095+0000","last_fullsized":"2024-04-16T17:04:44.241095+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:24:57.457026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240565+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:04:44.240565+0000","last_peered":"2024-04-16T17:04:44.240565+0000","last_clean":"2024-04-16T17:04:44.240565+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:04:44.240565+0000","last_undegraded":"2024-04-16T17:04:44.240565+0000","last_fullsized":"2024-04-16T17:04:44.240565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104268+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:04:44.104268+0000","last_peered":"2024-04-16T17:04:44.104268+0000","last_clean":"2024-04-16T17:04:44.104268+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:04:44.104268+0000","last_undegraded":"2024-04-16T17:04:44.104268+0000","last_fullsized":"2024-04-16T17:04:44.104268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414352+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:04:44.414352+0000","last_peered":"2024-04-16T17:04:44.414352+0000","last_clean":"2024-04-16T17:04:44.414352+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:04:44.414352+0000","last_undegraded":"2024-04-16T17:04:44.414352+0000","last_fullsized":"2024-04-16T17:04:44.414352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274993+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:04:44.274993+0000","last_peered":"2024-04-16T17:04:44.274993+0000","last_clean":"2024-04-16T17:04:44.274993+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:04:44.274993+0000","last_undegraded":"2024-04-16T17:04:44.274993+0000","last_fullsized":"2024-04-16T17:04:44.274993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241448+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:04:44.241448+0000","last_peered":"2024-04-16T17:04:44.241448+0000","last_clean":"2024-04-16T17:04:44.241448+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:04:44.241448+0000","last_undegraded":"2024-04-16T17:04:44.241448+0000","last_fullsized":"2024-04-16T17:04:44.241448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414414+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:04:44.414414+0000","last_peered":"2024-04-16T17:04:44.414414+0000","last_clean":"2024-04-16T17:04:44.414414+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:04:44.414414+0000","last_undegraded":"2024-04-16T17:04:44.414414+0000","last_fullsized":"2024-04-16T17:04:44.414414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240470+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:04:44.240470+0000","last_peered":"2024-04-16T17:04:44.240470+0000","last_clean":"2024-04-16T17:04:44.240470+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:04:44.240470+0000","last_undegraded":"2024-04-16T17:04:44.240470+0000","last_fullsized":"2024-04-16T17:04:44.240470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114478+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:04:44.114478+0000","last_peered":"2024-04-16T17:04:44.114478+0000","last_clean":"2024-04-16T17:04:44.114478+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:04:44.114478+0000","last_undegraded":"2024-04-16T17:04:44.114478+0000","last_fullsized":"2024-04-16T17:04:44.114478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207182+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:04:44.207182+0000","last_peered":"2024-04-16T17:04:44.207182+0000","last_clean":"2024-04-16T17:04:44.207182+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:04:44.207182+0000","last_undegraded":"2024-04-16T17:04:44.207182+0000","last_fullsized":"2024-04-16T17:04:44.207182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207367+0000","last_change":"2024-04-16T17:02:26.328059+0000","last_active":"2024-04-16T17:04:44.207367+0000","last_peered":"2024-04-16T17:04:44.207367+0000","last_clean":"2024-04-16T17:04:44.207367+0000","last_became_active":"2024-04-16T17:02:26.323315+0000","last_became_peered":"2024-04-16T17:02:26.323315+0000","last_unstale":"2024-04-16T17:04:44.207367+0000","last_undegraded":"2024-04-16T17:04:44.207367+0000","last_fullsized":"2024-04-16T17:04:44.207367+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:01.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114711+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:04:44.114711+0000","last_peered":"2024-04-16T17:04:44.114711+0000","last_clean":"2024-04-16T17:04:44.114711+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:04:44.114711+0000","last_undegraded":"2024-04-16T17:04:44.114711+0000","last_fullsized":"2024-04-16T17:04:44.114711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418863+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:04:44.418863+0000","last_peered":"2024-04-16T17:04:44.418863+0000","last_clean":"2024-04-16T17:04:44.418863+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:04:44.418863+0000","last_undegraded":"2024-04-16T17:04:44.418863+0000","last_fullsized":"2024-04-16T17:04:44.418863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'39","reported_seq":81,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.368875+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:04:44.368875+0000","last_peered":"2024-04-16T17:04:44.368875+0000","last_clean":"2024-04-16T17:04:44.368875+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:04:44.368875+0000","last_undegraded":"2024-04-16T17:04:44.368875+0000","last_fullsized":"2024-04-16T17:04:44.368875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114226+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:04:44.114226+0000","last_peered":"2024-04-16T17:04:44.114226+0000","last_clean":"2024-04-16T17:04:44.114226+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:04:44.114226+0000","last_undegraded":"2024-04-16T17:04:44.114226+0000","last_fullsized":"2024-04-16T17:04:44.114226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248179+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:04:44.248179+0000","last_peered":"2024-04-16T17:04:44.248179+0000","last_clean":"2024-04-16T17:04:44.248179+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:04:44.248179+0000","last_undegraded":"2024-04-16T17:04:44.248179+0000","last_fullsized":"2024-04-16T17:04:44.248179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414927+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:04:44.414927+0000","last_peered":"2024-04-16T17:04:44.414927+0000","last_clean":"2024-04-16T17:04:44.414927+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:04:44.414927+0000","last_undegraded":"2024-04-16T17:04:44.414927+0000","last_fullsized":"2024-04-16T17:04:44.414927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114728+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:04:44.114728+0000","last_peered":"2024-04-16T17:04:44.114728+0000","last_clean":"2024-04-16T17:04:44.114728+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:04:44.114728+0000","last_undegraded":"2024-04-16T17:04:44.114728+0000","last_fullsized":"2024-04-16T17:04:44.114728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104236+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:04:44.104236+0000","last_peered":"2024-04-16T17:04:44.104236+0000","last_clean":"2024-04-16T17:04:44.104236+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:04:44.104236+0000","last_undegraded":"2024-04-16T17:04:44.104236+0000","last_fullsized":"2024-04-16T17:04:44.104236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275188+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:04:44.275188+0000","last_peered":"2024-04-16T17:04:44.275188+0000","last_clean":"2024-04-16T17:04:44.275188+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:04:44.275188+0000","last_undegraded":"2024-04-16T17:04:44.275188+0000","last_fullsized":"2024-04-16T17:04:44.275188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274905+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:04:44.274905+0000","last_peered":"2024-04-16T17:04:44.274905+0000","last_clean":"2024-04-16T17:04:44.274905+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:04:44.274905+0000","last_undegraded":"2024-04-16T17:04:44.274905+0000","last_fullsized":"2024-04-16T17:04:44.274905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419653+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:04:44.419653+0000","last_peered":"2024-04-16T17:04:44.419653+0000","last_clean":"2024-04-16T17:04:44.419653+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:04:44.419653+0000","last_undegraded":"2024-04-16T17:04:44.419653+0000","last_fullsized":"2024-04-16T17:04:44.419653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415072+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:04:44.415072+0000","last_peered":"2024-04-16T17:04:44.415072+0000","last_clean":"2024-04-16T17:04:44.415072+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:04:44.415072+0000","last_undegraded":"2024-04-16T17:04:44.415072+0000","last_fullsized":"2024-04-16T17:04:44.415072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104917+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:04:44.104917+0000","last_peered":"2024-04-16T17:04:44.104917+0000","last_clean":"2024-04-16T17:04:44.104917+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:04:44.104917+0000","last_undegraded":"2024-04-16T17:04:44.104917+0000","last_fullsized":"2024-04-16T17:04:44.104917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275489+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:04:44.275489+0000","last_peered":"2024-04-16T17:04:44.275489+0000","last_clean":"2024-04-16T17:04:44.275489+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:04:44.275489+0000","last_undegraded":"2024-04-16T17:04:44.275489+0000","last_fullsized":"2024-04-16T17:04:44.275489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241877+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:04:44.241877+0000","last_peered":"2024-04-16T17:04:44.241877+0000","last_clean":"2024-04-16T17:04:44.241877+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:04:44.241877+0000","last_undegraded":"2024-04-16T17:04:44.241877+0000","last_fullsized":"2024-04-16T17:04:44.241877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248434+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:04:44.248434+0000","last_peered":"2024-04-16T17:04:44.248434+0000","last_clean":"2024-04-16T17:04:44.248434+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:04:44.248434+0000","last_undegraded":"2024-04-16T17:04:44.248434+0000","last_fullsized":"2024-04-16T17:04:44.248434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240652+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:04:44.240652+0000","last_peered":"2024-04-16T17:04:44.240652+0000","last_clean":"2024-04-16T17:04:44.240652+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:04:44.240652+0000","last_undegraded":"2024-04-16T17:04:44.240652+0000","last_fullsized":"2024-04-16T17:04:44.240652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207400+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:04:44.207400+0000","last_peered":"2024-04-16T17:04:44.207400+0000","last_clean":"2024-04-16T17:04:44.207400+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:04:44.207400+0000","last_undegraded":"2024-04-16T17:04:44.207400+0000","last_fullsized":"2024-04-16T17:04:44.207400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415063+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:04:44.415063+0000","last_peered":"2024-04-16T17:04:44.415063+0000","last_clean":"2024-04-16T17:04:44.415063+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:04:44.415063+0000","last_undegraded":"2024-04-16T17:04:44.415063+0000","last_fullsized":"2024-04-16T17:04:44.415063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104214+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:04:44.104214+0000","last_peered":"2024-04-16T17:04:44.104214+0000","last_clean":"2024-04-16T17:04:44.104214+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:04:44.104214+0000","last_undegraded":"2024-04-16T17:04:44.104214+0000","last_fullsized":"2024-04-16T17:04:44.104214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106545,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600829,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1113,"num_read_kb":673,"num_write":292,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1805398,"data_compressed":81384,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383732,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":529733,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36512,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708640,"statfs":{"total":95995035648,"available":95957647360,"internally_reserved":0,"allocated":1204224,"data_stored":1320281,"data_compressed":73169,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808897,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":927259,"data_compressed":48818,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070534,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":923061,"data_compressed":48818,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332171,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1125695,"data_compressed":65187,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495696,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":525123,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":599446,"data_compressed":31498,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":526511,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":596099,"data_compressed":31498,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182497,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1050133,"data_compressed":54974,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":531878,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757336,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":1252883,"data_compressed":71343,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":74413,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71066,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133777,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":204880,"data_compressed":7982,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:04:50.815 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:04:50.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:50 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:04:50.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:50 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:04:50.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:50 smithi022 ceph-mon[36023]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-16T17:04:50.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:50 smithi022 ceph-mon[36023]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-16T17:04:50.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:50 smithi022 ceph-mon[36023]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-16T17:04:50.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:50 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2584144377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:50.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:50 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3635647859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 -- 172.21.15.18:0/1866730667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63501030d0 msgr2=0x7f6350103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 --2- 172.21.15.18:0/1866730667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63501030d0 0x7f6350103550 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f633c009900 tx=0x7f633c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 -- 172.21.15.18:0/1866730667 shutdown_connections 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 --2- 172.21.15.18:0/1866730667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103a90 0x7f635010a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 --2- 172.21.15.18:0/1866730667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63501030d0 0x7f6350103550 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 --2- 172.21.15.18:0/1866730667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6350101f00 0x7f6350102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 -- 172.21.15.18:0/1866730667 >> 172.21.15.18:0/1866730667 conn(0x7f63500fd4c0 msgr2=0x7f63500ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 -- 172.21.15.18:0/1866730667 shutdown_connections 2024-04-16T17:04:51.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.031+0000 7f6354a00640 1 -- 172.21.15.18:0/1866730667 wait complete. 2024-04-16T17:04:51.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f6354a00640 1 Processor -- start 2024-04-16T17:04:51.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f6354a00640 1 -- start start 2024-04-16T17:04:51.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f6354a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6350101f00 0x7f635019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:51.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f6354a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63501030d0 0x7f635019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:51.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f6354a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6350103a90 0x7f63501a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:51.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f6354a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f635010dc20 con 0x7f6350101f00 2024-04-16T17:04:51.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f6354a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f635010daa0 con 0x7f63501030d0 2024-04-16T17:04:51.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f6354a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f635010dda0 con 0x7f6350103a90 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f634e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6350101f00 0x7f635019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.033+0000 7f634f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6350103a90 0x7f63501a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63501030d0 0x7f635019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6350101f00 0x7f635019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55566/0 (socket says 172.21.15.18:55566) 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634e800640 1 -- 172.21.15.18:0/3390175833 learned_addr learned my addr 172.21.15.18:0/3390175833 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634f200640 1 -- 172.21.15.18:0/3390175833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63501030d0 msgr2=0x7f635019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634f200640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63501030d0 0x7f635019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634f200640 1 -- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6350101f00 msgr2=0x7f635019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634f200640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6350101f00 0x7f635019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634f200640 1 -- 172.21.15.18:0/3390175833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63501a2070 con 0x7f6350103a90 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634de00640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63501030d0 0x7f635019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f634f200640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6350103a90 0x7f63501a1a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6348009fd0 tx=0x7f634800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f6347400640 1 -- 172.21.15.18:0/3390175833 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634800ce20 con 0x7f6350103a90 2024-04-16T17:04:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.034+0000 7f6347400640 1 -- 172.21.15.18:0/3390175833 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6348005020 con 0x7f6350103a90 2024-04-16T17:04:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.035+0000 7f6347400640 1 -- 172.21.15.18:0/3390175833 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348010720 con 0x7f6350103a90 2024-04-16T17:04:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.035+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63501a2360 con 0x7f6350103a90 2024-04-16T17:04:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.035+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63501a2830 con 0x7f6350103a90 2024-04-16T17:04:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.036+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63501a2af0 con 0x7f6350103a90 2024-04-16T17:04:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.038+0000 7f6347400640 1 -- 172.21.15.18:0/3390175833 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6348005540 con 0x7f6350103a90 2024-04-16T17:04:51.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.039+0000 7f6347400640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f63340a3eb0 0x7f63340a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:51.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.040+0000 7f6347400640 1 -- 172.21.15.18:0/3390175833 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f63480d1450 con 0x7f6350103a90 2024-04-16T17:04:51.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.040+0000 7f634e800640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f63340a3eb0 0x7f63340a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:51.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.040+0000 7f634e800640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f63340a3eb0 0x7f63340a6370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63400058c0 tx=0x7f6340005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:51.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.041+0000 7f6347400640 1 -- 172.21.15.18:0/3390175833 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f634809d5a0 con 0x7f6350103a90 2024-04-16T17:04:51.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.238+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63501a2cc0 con 0x7f63340a3eb0 2024-04-16T17:04:51.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.253+0000 7f6347400640 1 -- 172.21.15.18:0/3390175833 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339716 (secure 0 0 0) 0x7f63501a2cc0 con 0x7f63340a3eb0 2024-04-16T17:04:51.258 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:51.260 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:04:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f63340a3eb0 msgr2=0x7f63340a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f63340a3eb0 0x7f63340a6370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63400058c0 tx=0x7f6340005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6350103a90 msgr2=0x7f63501a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6350103a90 0x7f63501a1a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6348009fd0 tx=0x7f634800ef10 comp rx=0 tx=0).stop 2024-04-16T17:04:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 shutdown_connections 2024-04-16T17:04:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f63340a3eb0 0x7f63340a6370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6350103a90 0x7f63501a1a00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63501030d0 0x7f635019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 --2- 172.21.15.18:0/3390175833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6350101f00 0x7f635019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 >> 172.21.15.18:0/3390175833 conn(0x7f63500fd4c0 msgr2=0x7f6350074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 shutdown_connections 2024-04-16T17:04:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.257+0000 7f6354a00640 1 -- 172.21.15.18:0/3390175833 wait complete. 2024-04-16T17:04:51.277 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-16T17:04:50.125915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191431,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1249,"num_read_kb":792,"num_write":475,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378580,"kb_used_data":12740,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563244,"statfs":{"total":1151940427776,"available":1151552761856,"internally_reserved":0,"allocated":13045760,"data_stored":9908102,"data_compressed":543993,"data_compressed_allocated":737280,"data_compressed_original":4325376,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.970620"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104573+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:04:44.104573+0000","last_peered":"2024-04-16T17:04:44.104573+0000","last_clean":"2024-04-16T17:04:44.104573+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:04:44.104573+0000","last_undegraded":"2024-04-16T17:04:44.104573+0000","last_fullsized":"2024-04-16T17:04:44.104573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419294+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:04:44.419294+0000","last_peered":"2024-04-16T17:04:44.419294+0000","last_clean":"2024-04-16T17:04:44.419294+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:04:44.419294+0000","last_undegraded":"2024-04-16T17:04:44.419294+0000","last_fullsized":"2024-04-16T17:04:44.419294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418965+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:04:44.418965+0000","last_peered":"2024-04-16T17:04:44.418965+0000","last_clean":"2024-04-16T17:04:44.418965+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:04:44.418965+0000","last_undegraded":"2024-04-16T17:04:44.418965+0000","last_fullsized":"2024-04-16T17:04:44.418965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237775+0000","last_change":"2024-04-16T17:02:26.318719+0000","last_active":"2024-04-16T17:04:44.237775+0000","last_peered":"2024-04-16T17:04:44.237775+0000","last_clean":"2024-04-16T17:04:44.237775+0000","last_became_active":"2024-04-16T17:02:26.317743+0000","last_became_peered":"2024-04-16T17:02:26.317743+0000","last_unstale":"2024-04-16T17:04:44.237775+0000","last_undegraded":"2024-04-16T17:04:44.237775+0000","last_fullsized":"2024-04-16T17:04:44.237775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:26.827859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240559+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:04:44.240559+0000","last_peered":"2024-04-16T17:04:44.240559+0000","last_clean":"2024-04-16T17:04:44.240559+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:04:44.240559+0000","last_undegraded":"2024-04-16T17:04:44.240559+0000","last_fullsized":"2024-04-16T17:04:44.240559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104192+0000","last_change":"2024-04-16T17:02:26.315910+0000","last_active":"2024-04-16T17:04:44.104192+0000","last_peered":"2024-04-16T17:04:44.104192+0000","last_clean":"2024-04-16T17:04:44.104192+0000","last_became_active":"2024-04-16T17:02:26.315493+0000","last_became_peered":"2024-04-16T17:02:26.315493+0000","last_unstale":"2024-04-16T17:04:44.104192+0000","last_undegraded":"2024-04-16T17:04:44.104192+0000","last_fullsized":"2024-04-16T17:04:44.104192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:27:48.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114020+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:04:44.114020+0000","last_peered":"2024-04-16T17:04:44.114020+0000","last_clean":"2024-04-16T17:04:44.114020+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:04:44.114020+0000","last_undegraded":"2024-04-16T17:04:44.114020+0000","last_fullsized":"2024-04-16T17:04:44.114020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240629+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:04:44.240629+0000","last_peered":"2024-04-16T17:04:44.240629+0000","last_clean":"2024-04-16T17:04:44.240629+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:04:44.240629+0000","last_undegraded":"2024-04-16T17:04:44.240629+0000","last_fullsized":"2024-04-16T17:04:44.240629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207849+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:04:44.207849+0000","last_peered":"2024-04-16T17:04:44.207849+0000","last_clean":"2024-04-16T17:04:44.207849+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:04:44.207849+0000","last_undegraded":"2024-04-16T17:04:44.207849+0000","last_fullsized":"2024-04-16T17:04:44.207849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104511+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:04:44.104511+0000","last_peered":"2024-04-16T17:04:44.104511+0000","last_clean":"2024-04-16T17:04:44.104511+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:04:44.104511+0000","last_undegraded":"2024-04-16T17:04:44.104511+0000","last_fullsized":"2024-04-16T17:04:44.104511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114664+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:04:44.114664+0000","last_peered":"2024-04-16T17:04:44.114664+0000","last_clean":"2024-04-16T17:04:44.114664+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:04:44.114664+0000","last_undegraded":"2024-04-16T17:04:44.114664+0000","last_fullsized":"2024-04-16T17:04:44.114664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414524+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:04:44.414524+0000","last_peered":"2024-04-16T17:04:44.414524+0000","last_clean":"2024-04-16T17:04:44.414524+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:04:44.414524+0000","last_undegraded":"2024-04-16T17:04:44.414524+0000","last_fullsized":"2024-04-16T17:04:44.414524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415190+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:04:44.415190+0000","last_peered":"2024-04-16T17:04:44.415190+0000","last_clean":"2024-04-16T17:04:44.415190+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:04:44.415190+0000","last_undegraded":"2024-04-16T17:04:44.415190+0000","last_fullsized":"2024-04-16T17:04:44.415190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418945+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:04:44.418945+0000","last_peered":"2024-04-16T17:04:44.418945+0000","last_clean":"2024-04-16T17:04:44.418945+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:04:44.418945+0000","last_undegraded":"2024-04-16T17:04:44.418945+0000","last_fullsized":"2024-04-16T17:04:44.418945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419182+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:04:44.419182+0000","last_peered":"2024-04-16T17:04:44.419182+0000","last_clean":"2024-04-16T17:04:44.419182+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:04:44.419182+0000","last_undegraded":"2024-04-16T17:04:44.419182+0000","last_fullsized":"2024-04-16T17:04:44.419182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207293+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:04:44.207293+0000","last_peered":"2024-04-16T17:04:44.207293+0000","last_clean":"2024-04-16T17:04:44.207293+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:04:44.207293+0000","last_undegraded":"2024-04-16T17:04:44.207293+0000","last_fullsized":"2024-04-16T17:04:44.207293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104522+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:04:44.104522+0000","last_peered":"2024-04-16T17:04:44.104522+0000","last_clean":"2024-04-16T17:04:44.104522+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:04:44.104522+0000","last_undegraded":"2024-04-16T17:04:44.104522+0000","last_fullsized":"2024-04-16T17:04:44.104522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.238023+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:04:44.238023+0000","last_peered":"2024-04-16T17:04:44.238023+0000","last_clean":"2024-04-16T17:04:44.238023+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:04:44.238023+0000","last_undegraded":"2024-04-16T17:04:44.238023+0000","last_fullsized":"2024-04-16T17:04:44.238023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275555+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:04:44.275555+0000","last_peered":"2024-04-16T17:04:44.275555+0000","last_clean":"2024-04-16T17:04:44.275555+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:04:44.275555+0000","last_undegraded":"2024-04-16T17:04:44.275555+0000","last_fullsized":"2024-04-16T17:04:44.275555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274972+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:04:44.274972+0000","last_peered":"2024-04-16T17:04:44.274972+0000","last_clean":"2024-04-16T17:04:44.274972+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:04:44.274972+0000","last_undegraded":"2024-04-16T17:04:44.274972+0000","last_fullsized":"2024-04-16T17:04:44.274972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237649+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:04:44.237649+0000","last_peered":"2024-04-16T17:04:44.237649+0000","last_clean":"2024-04-16T17:04:44.237649+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:04:44.237649+0000","last_undegraded":"2024-04-16T17:04:44.237649+0000","last_fullsized":"2024-04-16T17:04:44.237649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.238023+0000","last_change":"2024-04-16T17:02:26.318546+0000","last_active":"2024-04-16T17:04:44.238023+0000","last_peered":"2024-04-16T17:04:44.238023+0000","last_clean":"2024-04-16T17:04:44.238023+0000","last_became_active":"2024-04-16T17:02:26.317794+0000","last_became_peered":"2024-04-16T17:02:26.317794+0000","last_unstale":"2024-04-16T17:04:44.238023+0000","last_undegraded":"2024-04-16T17:04:44.238023+0000","last_fullsized":"2024-04-16T17:04:44.238023+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:48.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414178+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:04:44.414178+0000","last_peered":"2024-04-16T17:04:44.414178+0000","last_clean":"2024-04-16T17:04:44.414178+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:04:44.414178+0000","last_undegraded":"2024-04-16T17:04:44.414178+0000","last_fullsized":"2024-04-16T17:04:44.414178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241729+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:04:44.241729+0000","last_peered":"2024-04-16T17:04:44.241729+0000","last_clean":"2024-04-16T17:04:44.241729+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:04:44.241729+0000","last_undegraded":"2024-04-16T17:04:44.241729+0000","last_fullsized":"2024-04-16T17:04:44.241729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419515+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:04:44.419515+0000","last_peered":"2024-04-16T17:04:44.419515+0000","last_clean":"2024-04-16T17:04:44.419515+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:04:44.419515+0000","last_undegraded":"2024-04-16T17:04:44.419515+0000","last_fullsized":"2024-04-16T17:04:44.419515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414312+0000","last_change":"2024-04-16T17:03:27.271585+0000","last_active":"2024-04-16T17:04:44.414312+0000","last_peered":"2024-04-16T17:04:44.414312+0000","last_clean":"2024-04-16T17:04:44.414312+0000","last_became_active":"2024-04-16T17:03:27.271251+0000","last_became_peered":"2024-04-16T17:03:27.271251+0000","last_unstale":"2024-04-16T17:04:44.414312+0000","last_undegraded":"2024-04-16T17:04:44.414312+0000","last_fullsized":"2024-04-16T17:04:44.414312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:51.217466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241064+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:04:44.241064+0000","last_peered":"2024-04-16T17:04:44.241064+0000","last_clean":"2024-04-16T17:04:44.241064+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:04:44.241064+0000","last_undegraded":"2024-04-16T17:04:44.241064+0000","last_fullsized":"2024-04-16T17:04:44.241064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207578+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:04:44.207578+0000","last_peered":"2024-04-16T17:04:44.207578+0000","last_clean":"2024-04-16T17:04:44.207578+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:04:44.207578+0000","last_undegraded":"2024-04-16T17:04:44.207578+0000","last_fullsized":"2024-04-16T17:04:44.207578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414211+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:04:44.414211+0000","last_peered":"2024-04-16T17:04:44.414211+0000","last_clean":"2024-04-16T17:04:44.414211+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:04:44.414211+0000","last_undegraded":"2024-04-16T17:04:44.414211+0000","last_fullsized":"2024-04-16T17:04:44.414211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240928+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:04:44.240928+0000","last_peered":"2024-04-16T17:04:44.240928+0000","last_clean":"2024-04-16T17:04:44.240928+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:04:44.240928+0000","last_undegraded":"2024-04-16T17:04:44.240928+0000","last_fullsized":"2024-04-16T17:04:44.240928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248909+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:04:44.248909+0000","last_peered":"2024-04-16T17:04:44.248909+0000","last_clean":"2024-04-16T17:04:44.248909+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:04:44.248909+0000","last_undegraded":"2024-04-16T17:04:44.248909+0000","last_fullsized":"2024-04-16T17:04:44.248909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275231+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:04:44.275231+0000","last_peered":"2024-04-16T17:04:44.275231+0000","last_clean":"2024-04-16T17:04:44.275231+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:04:44.275231+0000","last_undegraded":"2024-04-16T17:04:44.275231+0000","last_fullsized":"2024-04-16T17:04:44.275231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":223,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.506138+0000","last_change":"2024-04-16T17:02:26.315434+0000","last_active":"2024-04-16T17:04:44.506138+0000","last_peered":"2024-04-16T17:04:44.506138+0000","last_clean":"2024-04-16T17:04:44.506138+0000","last_became_active":"2024-04-16T17:02:26.310169+0000","last_became_peered":"2024-04-16T17:02:26.310169+0000","last_unstale":"2024-04-16T17:04:44.506138+0000","last_undegraded":"2024-04-16T17:04:44.506138+0000","last_fullsized":"2024-04-16T17:04:44.506138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:51.033415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":164,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240363+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:04:44.240363+0000","last_peered":"2024-04-16T17:04:44.240363+0000","last_clean":"2024-04-16T17:04:44.240363+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:04:44.240363+0000","last_undegraded":"2024-04-16T17:04:44.240363+0000","last_fullsized":"2024-04-16T17:04:44.240363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275338+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:04:44.275338+0000","last_peered":"2024-04-16T17:04:44.275338+0000","last_clean":"2024-04-16T17:04:44.275338+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:04:44.275338+0000","last_undegraded":"2024-04-16T17:04:44.275338+0000","last_fullsized":"2024-04-16T17:04:44.275338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104007+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:04:44.104007+0000","last_peered":"2024-04-16T17:04:44.104007+0000","last_clean":"2024-04-16T17:04:44.104007+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:04:44.104007+0000","last_undegraded":"2024-04-16T17:04:44.104007+0000","last_fullsized":"2024-04-16T17:04:44.104007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237470+0000","last_change":"2024-04-16T17:02:26.321763+0000","last_active":"2024-04-16T17:04:44.237470+0000","last_peered":"2024-04-16T17:04:44.237470+0000","last_clean":"2024-04-16T17:04:44.237470+0000","last_became_active":"2024-04-16T17:02:26.321438+0000","last_became_peered":"2024-04-16T17:02:26.321438+0000","last_unstale":"2024-04-16T17:04:44.237470+0000","last_undegraded":"2024-04-16T17:04:44.237470+0000","last_fullsized":"2024-04-16T17:04:44.237470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:47.231035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104252+0000","last_change":"2024-04-16T17:03:27.451220+0000","last_active":"2024-04-16T17:04:44.104252+0000","last_peered":"2024-04-16T17:04:44.104252+0000","last_clean":"2024-04-16T17:04:44.104252+0000","last_became_active":"2024-04-16T17:03:27.450680+0000","last_became_peered":"2024-04-16T17:03:27.450680+0000","last_unstale":"2024-04-16T17:04:44.104252+0000","last_undegraded":"2024-04-16T17:04:44.104252+0000","last_fullsized":"2024-04-16T17:04:44.104252+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:23.270332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414603+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:04:44.414603+0000","last_peered":"2024-04-16T17:04:44.414603+0000","last_clean":"2024-04-16T17:04:44.414603+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:04:44.414603+0000","last_undegraded":"2024-04-16T17:04:44.414603+0000","last_fullsized":"2024-04-16T17:04:44.414603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241007+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:04:44.241007+0000","last_peered":"2024-04-16T17:04:44.241007+0000","last_clean":"2024-04-16T17:04:44.241007+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:04:44.241007+0000","last_undegraded":"2024-04-16T17:04:44.241007+0000","last_fullsized":"2024-04-16T17:04:44.241007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419271+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:04:44.419271+0000","last_peered":"2024-04-16T17:04:44.419271+0000","last_clean":"2024-04-16T17:04:44.419271+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:04:44.419271+0000","last_undegraded":"2024-04-16T17:04:44.419271+0000","last_fullsized":"2024-04-16T17:04:44.419271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248395+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:04:44.248395+0000","last_peered":"2024-04-16T17:04:44.248395+0000","last_clean":"2024-04-16T17:04:44.248395+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:04:44.248395+0000","last_undegraded":"2024-04-16T17:04:44.248395+0000","last_fullsized":"2024-04-16T17:04:44.248395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237507+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:04:44.237507+0000","last_peered":"2024-04-16T17:04:44.237507+0000","last_clean":"2024-04-16T17:04:44.237507+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:04:44.237507+0000","last_undegraded":"2024-04-16T17:04:44.237507+0000","last_fullsized":"2024-04-16T17:04:44.237507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240883+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:04:44.240883+0000","last_peered":"2024-04-16T17:04:44.240883+0000","last_clean":"2024-04-16T17:04:44.240883+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:04:44.240883+0000","last_undegraded":"2024-04-16T17:04:44.240883+0000","last_fullsized":"2024-04-16T17:04:44.240883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":204,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104418+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:04:44.104418+0000","last_peered":"2024-04-16T17:04:44.104418+0000","last_clean":"2024-04-16T17:04:44.104418+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:04:44.104418+0000","last_undegraded":"2024-04-16T17:04:44.104418+0000","last_fullsized":"2024-04-16T17:04:44.104418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207667+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:04:44.207667+0000","last_peered":"2024-04-16T17:04:44.207667+0000","last_clean":"2024-04-16T17:04:44.207667+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:04:44.207667+0000","last_undegraded":"2024-04-16T17:04:44.207667+0000","last_fullsized":"2024-04-16T17:04:44.207667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237402+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:04:44.237402+0000","last_peered":"2024-04-16T17:04:44.237402+0000","last_clean":"2024-04-16T17:04:44.237402+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:04:44.237402+0000","last_undegraded":"2024-04-16T17:04:44.237402+0000","last_fullsized":"2024-04-16T17:04:44.237402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114484+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:04:44.114484+0000","last_peered":"2024-04-16T17:04:44.114484+0000","last_clean":"2024-04-16T17:04:44.114484+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:04:44.114484+0000","last_undegraded":"2024-04-16T17:04:44.114484+0000","last_fullsized":"2024-04-16T17:04:44.114484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419897+0000","last_change":"2024-04-16T17:02:26.752275+0000","last_active":"2024-04-16T17:04:44.419897+0000","last_peered":"2024-04-16T17:04:44.419897+0000","last_clean":"2024-04-16T17:04:44.419897+0000","last_became_active":"2024-04-16T17:02:26.751729+0000","last_became_peered":"2024-04-16T17:02:26.751729+0000","last_unstale":"2024-04-16T17:04:44.419897+0000","last_undegraded":"2024-04-16T17:04:44.419897+0000","last_fullsized":"2024-04-16T17:04:44.419897+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:41:35.777691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414493+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:04:44.414493+0000","last_peered":"2024-04-16T17:04:44.414493+0000","last_clean":"2024-04-16T17:04:44.414493+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:04:44.414493+0000","last_undegraded":"2024-04-16T17:04:44.414493+0000","last_fullsized":"2024-04-16T17:04:44.414493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248851+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:04:44.248851+0000","last_peered":"2024-04-16T17:04:44.248851+0000","last_clean":"2024-04-16T17:04:44.248851+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:04:44.248851+0000","last_undegraded":"2024-04-16T17:04:44.248851+0000","last_fullsized":"2024-04-16T17:04:44.248851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207706+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:04:44.207706+0000","last_peered":"2024-04-16T17:04:44.207706+0000","last_clean":"2024-04-16T17:04:44.207706+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:04:44.207706+0000","last_undegraded":"2024-04-16T17:04:44.207706+0000","last_fullsized":"2024-04-16T17:04:44.207706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103750+0000","last_change":"2024-04-16T17:03:27.054627+0000","last_active":"2024-04-16T17:04:44.103750+0000","last_peered":"2024-04-16T17:04:44.103750+0000","last_clean":"2024-04-16T17:04:44.103750+0000","last_became_active":"2024-04-16T17:03:27.054326+0000","last_became_peered":"2024-04-16T17:03:27.054326+0000","last_unstale":"2024-04-16T17:04:44.103750+0000","last_undegraded":"2024-04-16T17:04:44.103750+0000","last_fullsized":"2024-04-16T17:04:44.103750+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:43.535958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":419,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.505895+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:04:44.505895+0000","last_peered":"2024-04-16T17:04:44.505895+0000","last_clean":"2024-04-16T17:04:44.505895+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:04:44.505895+0000","last_undegraded":"2024-04-16T17:04:44.505895+0000","last_fullsized":"2024-04-16T17:04:44.505895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70920,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":344,"num_read_kb":243,"num_write":42,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240855+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:04:44.240855+0000","last_peered":"2024-04-16T17:04:44.240855+0000","last_clean":"2024-04-16T17:04:44.240855+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:04:44.240855+0000","last_undegraded":"2024-04-16T17:04:44.240855+0000","last_fullsized":"2024-04-16T17:04:44.240855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.249232+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:04:44.249232+0000","last_peered":"2024-04-16T17:04:44.249232+0000","last_clean":"2024-04-16T17:04:44.249232+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:04:44.249232+0000","last_undegraded":"2024-04-16T17:04:44.249232+0000","last_fullsized":"2024-04-16T17:04:44.249232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275182+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:04:44.275182+0000","last_peered":"2024-04-16T17:04:44.275182+0000","last_clean":"2024-04-16T17:04:44.275182+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:04:44.275182+0000","last_undegraded":"2024-04-16T17:04:44.275182+0000","last_fullsized":"2024-04-16T17:04:44.275182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415497+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:04:44.415497+0000","last_peered":"2024-04-16T17:04:44.415497+0000","last_clean":"2024-04-16T17:04:44.415497+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:04:44.415497+0000","last_undegraded":"2024-04-16T17:04:44.415497+0000","last_fullsized":"2024-04-16T17:04:44.415497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104076+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:04:44.104076+0000","last_peered":"2024-04-16T17:04:44.104076+0000","last_clean":"2024-04-16T17:04:44.104076+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:04:44.104076+0000","last_undegraded":"2024-04-16T17:04:44.104076+0000","last_fullsized":"2024-04-16T17:04:44.104076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248492+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:04:44.248492+0000","last_peered":"2024-04-16T17:04:44.248492+0000","last_clean":"2024-04-16T17:04:44.248492+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:04:44.248492+0000","last_undegraded":"2024-04-16T17:04:44.248492+0000","last_fullsized":"2024-04-16T17:04:44.248492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114507+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:04:44.114507+0000","last_peered":"2024-04-16T17:04:44.114507+0000","last_clean":"2024-04-16T17:04:44.114507+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:04:44.114507+0000","last_undegraded":"2024-04-16T17:04:44.114507+0000","last_fullsized":"2024-04-16T17:04:44.114507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237521+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:04:44.237521+0000","last_peered":"2024-04-16T17:04:44.237521+0000","last_clean":"2024-04-16T17:04:44.237521+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:04:44.237521+0000","last_undegraded":"2024-04-16T17:04:44.237521+0000","last_fullsized":"2024-04-16T17:04:44.237521+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":208,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.536313+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:04:44.536313+0000","last_peered":"2024-04-16T17:04:44.536313+0000","last_clean":"2024-04-16T17:04:44.536313+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:04:44.536313+0000","last_undegraded":"2024-04-16T17:04:44.536313+0000","last_fullsized":"2024-04-16T17:04:44.536313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":170,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.113893+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:04:44.113893+0000","last_peered":"2024-04-16T17:04:44.113893+0000","last_clean":"2024-04-16T17:04:44.113893+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:04:44.113893+0000","last_undegraded":"2024-04-16T17:04:44.113893+0000","last_fullsized":"2024-04-16T17:04:44.113893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415000+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:04:44.415000+0000","last_peered":"2024-04-16T17:04:44.415000+0000","last_clean":"2024-04-16T17:04:44.415000+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:04:44.415000+0000","last_undegraded":"2024-04-16T17:04:44.415000+0000","last_fullsized":"2024-04-16T17:04:44.415000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240346+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:04:44.240346+0000","last_peered":"2024-04-16T17:04:44.240346+0000","last_clean":"2024-04-16T17:04:44.240346+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:04:44.240346+0000","last_undegraded":"2024-04-16T17:04:44.240346+0000","last_fullsized":"2024-04-16T17:04:44.240346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248651+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:04:44.248651+0000","last_peered":"2024-04-16T17:04:44.248651+0000","last_clean":"2024-04-16T17:04:44.248651+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:04:44.248651+0000","last_undegraded":"2024-04-16T17:04:44.248651+0000","last_fullsized":"2024-04-16T17:04:44.248651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248417+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:04:44.248417+0000","last_peered":"2024-04-16T17:04:44.248417+0000","last_clean":"2024-04-16T17:04:44.248417+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:04:44.248417+0000","last_undegraded":"2024-04-16T17:04:44.248417+0000","last_fullsized":"2024-04-16T17:04:44.248417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104487+0000","last_change":"2024-04-16T17:02:26.320405+0000","last_active":"2024-04-16T17:04:44.104487+0000","last_peered":"2024-04-16T17:04:44.104487+0000","last_clean":"2024-04-16T17:04:44.104487+0000","last_became_active":"2024-04-16T17:02:26.307921+0000","last_became_peered":"2024-04-16T17:02:26.307921+0000","last_unstale":"2024-04-16T17:04:44.104487+0000","last_undegraded":"2024-04-16T17:04:44.104487+0000","last_fullsized":"2024-04-16T17:04:44.104487+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:19.156260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104216+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:04:44.104216+0000","last_peered":"2024-04-16T17:04:44.104216+0000","last_clean":"2024-04-16T17:04:44.104216+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:04:44.104216+0000","last_undegraded":"2024-04-16T17:04:44.104216+0000","last_fullsized":"2024-04-16T17:04:44.104216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240562+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:04:44.240562+0000","last_peered":"2024-04-16T17:04:44.240562+0000","last_clean":"2024-04-16T17:04:44.240562+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:04:44.240562+0000","last_undegraded":"2024-04-16T17:04:44.240562+0000","last_fullsized":"2024-04-16T17:04:44.240562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237720+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:04:44.237720+0000","last_peered":"2024-04-16T17:04:44.237720+0000","last_clean":"2024-04-16T17:04:44.237720+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:04:44.237720+0000","last_undegraded":"2024-04-16T17:04:44.237720+0000","last_fullsized":"2024-04-16T17:04:44.237720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104042+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:04:44.104042+0000","last_peered":"2024-04-16T17:04:44.104042+0000","last_clean":"2024-04-16T17:04:44.104042+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:04:44.104042+0000","last_undegraded":"2024-04-16T17:04:44.104042+0000","last_fullsized":"2024-04-16T17:04:44.104042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414819+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:04:44.414819+0000","last_peered":"2024-04-16T17:04:44.414819+0000","last_clean":"2024-04-16T17:04:44.414819+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:04:44.414819+0000","last_undegraded":"2024-04-16T17:04:44.414819+0000","last_fullsized":"2024-04-16T17:04:44.414819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114221+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:04:44.114221+0000","last_peered":"2024-04-16T17:04:44.114221+0000","last_clean":"2024-04-16T17:04:44.114221+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:04:44.114221+0000","last_undegraded":"2024-04-16T17:04:44.114221+0000","last_fullsized":"2024-04-16T17:04:44.114221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104271+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:04:44.104271+0000","last_peered":"2024-04-16T17:04:44.104271+0000","last_clean":"2024-04-16T17:04:44.104271+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:04:44.104271+0000","last_undegraded":"2024-04-16T17:04:44.104271+0000","last_fullsized":"2024-04-16T17:04:44.104271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415017+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:04:44.415017+0000","last_peered":"2024-04-16T17:04:44.415017+0000","last_clean":"2024-04-16T17:04:44.415017+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:04:44.415017+0000","last_undegraded":"2024-04-16T17:04:44.415017+0000","last_fullsized":"2024-04-16T17:04:44.415017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248603+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:04:44.248603+0000","last_peered":"2024-04-16T17:04:44.248603+0000","last_clean":"2024-04-16T17:04:44.248603+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:04:44.248603+0000","last_undegraded":"2024-04-16T17:04:44.248603+0000","last_fullsized":"2024-04-16T17:04:44.248603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":205,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.417263+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:04:44.417263+0000","last_peered":"2024-04-16T17:04:44.417263+0000","last_clean":"2024-04-16T17:04:44.417263+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:04:44.417263+0000","last_undegraded":"2024-04-16T17:04:44.417263+0000","last_fullsized":"2024-04-16T17:04:44.417263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114408+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:04:44.114408+0000","last_peered":"2024-04-16T17:04:44.114408+0000","last_clean":"2024-04-16T17:04:44.114408+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:04:44.114408+0000","last_undegraded":"2024-04-16T17:04:44.114408+0000","last_fullsized":"2024-04-16T17:04:44.114408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240965+0000","last_change":"2024-04-16T17:02:26.322313+0000","last_active":"2024-04-16T17:04:44.240965+0000","last_peered":"2024-04-16T17:04:44.240965+0000","last_clean":"2024-04-16T17:04:44.240965+0000","last_became_active":"2024-04-16T17:02:26.320328+0000","last_became_peered":"2024-04-16T17:02:26.320328+0000","last_unstale":"2024-04-16T17:04:44.240965+0000","last_undegraded":"2024-04-16T17:04:44.240965+0000","last_fullsized":"2024-04-16T17:04:44.240965+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:40.943183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414366+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:04:44.414366+0000","last_peered":"2024-04-16T17:04:44.414366+0000","last_clean":"2024-04-16T17:04:44.414366+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:04:44.414366+0000","last_undegraded":"2024-04-16T17:04:44.414366+0000","last_fullsized":"2024-04-16T17:04:44.414366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.113827+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:04:44.113827+0000","last_peered":"2024-04-16T17:04:44.113827+0000","last_clean":"2024-04-16T17:04:44.113827+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:04:44.113827+0000","last_undegraded":"2024-04-16T17:04:44.113827+0000","last_fullsized":"2024-04-16T17:04:44.113827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248980+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:04:44.248980+0000","last_peered":"2024-04-16T17:04:44.248980+0000","last_clean":"2024-04-16T17:04:44.248980+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:04:44.248980+0000","last_undegraded":"2024-04-16T17:04:44.248980+0000","last_fullsized":"2024-04-16T17:04:44.248980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414448+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:04:44.414448+0000","last_peered":"2024-04-16T17:04:44.414448+0000","last_clean":"2024-04-16T17:04:44.414448+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:04:44.414448+0000","last_undegraded":"2024-04-16T17:04:44.414448+0000","last_fullsized":"2024-04-16T17:04:44.414448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103918+0000","last_change":"2024-04-16T17:02:26.312852+0000","last_active":"2024-04-16T17:04:44.103918+0000","last_peered":"2024-04-16T17:04:44.103918+0000","last_clean":"2024-04-16T17:04:44.103918+0000","last_became_active":"2024-04-16T17:02:26.308279+0000","last_became_peered":"2024-04-16T17:02:26.308279+0000","last_unstale":"2024-04-16T17:04:44.103918+0000","last_undegraded":"2024-04-16T17:04:44.103918+0000","last_fullsized":"2024-04-16T17:04:44.103918+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:25.511349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103933+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:04:44.103933+0000","last_peered":"2024-04-16T17:04:44.103933+0000","last_clean":"2024-04-16T17:04:44.103933+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:04:44.103933+0000","last_undegraded":"2024-04-16T17:04:44.103933+0000","last_fullsized":"2024-04-16T17:04:44.103933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240849+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:04:44.240849+0000","last_peered":"2024-04-16T17:04:44.240849+0000","last_clean":"2024-04-16T17:04:44.240849+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:04:44.240849+0000","last_undegraded":"2024-04-16T17:04:44.240849+0000","last_fullsized":"2024-04-16T17:04:44.240849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415023+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:04:44.415023+0000","last_peered":"2024-04-16T17:04:44.415023+0000","last_clean":"2024-04-16T17:04:44.415023+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:04:44.415023+0000","last_undegraded":"2024-04-16T17:04:44.415023+0000","last_fullsized":"2024-04-16T17:04:44.415023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240499+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:04:44.240499+0000","last_peered":"2024-04-16T17:04:44.240499+0000","last_clean":"2024-04-16T17:04:44.240499+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:04:44.240499+0000","last_undegraded":"2024-04-16T17:04:44.240499+0000","last_fullsized":"2024-04-16T17:04:44.240499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'47","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.374558+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:04:44.374558+0000","last_peered":"2024-04-16T17:04:44.374558+0000","last_clean":"2024-04-16T17:04:44.374558+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:04:44.374558+0000","last_undegraded":"2024-04-16T17:04:44.374558+0000","last_fullsized":"2024-04-16T17:04:44.374558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241537+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:04:44.241537+0000","last_peered":"2024-04-16T17:04:44.241537+0000","last_clean":"2024-04-16T17:04:44.241537+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:04:44.241537+0000","last_undegraded":"2024-04-16T17:04:44.241537+0000","last_fullsized":"2024-04-16T17:04:44.241537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274800+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:04:44.274800+0000","last_peered":"2024-04-16T17:04:44.274800+0000","last_clean":"2024-04-16T17:04:44.274800+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:04:44.274800+0000","last_undegraded":"2024-04-16T17:04:44.274800+0000","last_fullsized":"2024-04-16T17:04:44.274800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414738+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:04:44.414738+0000","last_peered":"2024-04-16T17:04:44.414738+0000","last_clean":"2024-04-16T17:04:44.414738+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:04:44.414738+0000","last_undegraded":"2024-04-16T17:04:44.414738+0000","last_fullsized":"2024-04-16T17:04:44.414738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419383+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:04:44.419383+0000","last_peered":"2024-04-16T17:04:44.419383+0000","last_clean":"2024-04-16T17:04:44.419383+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:04:44.419383+0000","last_undegraded":"2024-04-16T17:04:44.419383+0000","last_fullsized":"2024-04-16T17:04:44.419383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414608+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:04:44.414608+0000","last_peered":"2024-04-16T17:04:44.414608+0000","last_clean":"2024-04-16T17:04:44.414608+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:04:44.414608+0000","last_undegraded":"2024-04-16T17:04:44.414608+0000","last_fullsized":"2024-04-16T17:04:44.414608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207210+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:04:44.207210+0000","last_peered":"2024-04-16T17:04:44.207210+0000","last_clean":"2024-04-16T17:04:44.207210+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:04:44.207210+0000","last_undegraded":"2024-04-16T17:04:44.207210+0000","last_fullsized":"2024-04-16T17:04:44.207210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240332+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:04:44.240332+0000","last_peered":"2024-04-16T17:04:44.240332+0000","last_clean":"2024-04-16T17:04:44.240332+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:04:44.240332+0000","last_undegraded":"2024-04-16T17:04:44.240332+0000","last_fullsized":"2024-04-16T17:04:44.240332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241095+0000","last_change":"2024-04-16T17:03:27.268760+0000","last_active":"2024-04-16T17:04:44.241095+0000","last_peered":"2024-04-16T17:04:44.241095+0000","last_clean":"2024-04-16T17:04:44.241095+0000","last_became_active":"2024-04-16T17:03:27.268209+0000","last_became_peered":"2024-04-16T17:03:27.268209+0000","last_unstale":"2024-04-16T17:04:44.241095+0000","last_undegraded":"2024-04-16T17:04:44.241095+0000","last_fullsized":"2024-04-16T17:04:44.241095+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:24:57.457026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240565+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:04:44.240565+0000","last_peered":"2024-04-16T17:04:44.240565+0000","last_clean":"2024-04-16T17:04:44.240565+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:04:44.240565+0000","last_undegraded":"2024-04-16T17:04:44.240565+0000","last_fullsized":"2024-04-16T17:04:44.240565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104268+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:04:44.104268+0000","last_peered":"2024-04-16T17:04:44.104268+0000","last_clean":"2024-04-16T17:04:44.104268+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:04:44.104268+0000","last_undegraded":"2024-04-16T17:04:44.104268+0000","last_fullsized":"2024-04-16T17:04:44.104268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414352+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:04:44.414352+0000","last_peered":"2024-04-16T17:04:44.414352+0000","last_clean":"2024-04-16T17:04:44.414352+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:04:44.414352+0000","last_undegraded":"2024-04-16T17:04:44.414352+0000","last_fullsized":"2024-04-16T17:04:44.414352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274993+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:04:44.274993+0000","last_peered":"2024-04-16T17:04:44.274993+0000","last_clean":"2024-04-16T17:04:44.274993+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:04:44.274993+0000","last_undegraded":"2024-04-16T17:04:44.274993+0000","last_fullsized":"2024-04-16T17:04:44.274993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241448+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:04:44.241448+0000","last_peered":"2024-04-16T17:04:44.241448+0000","last_clean":"2024-04-16T17:04:44.241448+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:04:44.241448+0000","last_undegraded":"2024-04-16T17:04:44.241448+0000","last_fullsized":"2024-04-16T17:04:44.241448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414414+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:04:44.414414+0000","last_peered":"2024-04-16T17:04:44.414414+0000","last_clean":"2024-04-16T17:04:44.414414+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:04:44.414414+0000","last_undegraded":"2024-04-16T17:04:44.414414+0000","last_fullsized":"2024-04-16T17:04:44.414414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240470+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:04:44.240470+0000","last_peered":"2024-04-16T17:04:44.240470+0000","last_clean":"2024-04-16T17:04:44.240470+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:04:44.240470+0000","last_undegraded":"2024-04-16T17:04:44.240470+0000","last_fullsized":"2024-04-16T17:04:44.240470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114478+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:04:44.114478+0000","last_peered":"2024-04-16T17:04:44.114478+0000","last_clean":"2024-04-16T17:04:44.114478+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:04:44.114478+0000","last_undegraded":"2024-04-16T17:04:44.114478+0000","last_fullsized":"2024-04-16T17:04:44.114478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207182+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:04:44.207182+0000","last_peered":"2024-04-16T17:04:44.207182+0000","last_clean":"2024-04-16T17:04:44.207182+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:04:44.207182+0000","last_undegraded":"2024-04-16T17:04:44.207182+0000","last_fullsized":"2024-04-16T17:04:44.207182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207367+0000","last_change":"2024-04-16T17:02:26.328059+0000","last_active":"2024-04-16T17:04:44.207367+0000","last_peered":"2024-04-16T17:04:44.207367+0000","last_clean":"2024-04-16T17:04:44.207367+0000","last_became_active":"2024-04-16T17:02:26.323315+0000","last_became_peered":"2024-04-16T17:02:26.323315+0000","last_unstale":"2024-04-16T17:04:44.207367+0000","last_undegraded":"2024-04-16T17:04:44.207367+0000","last_fullsized":"2024-04-16T17:04:44.207367+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:01.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114711+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:04:44.114711+0000","last_peered":"2024-04-16T17:04:44.114711+0000","last_clean":"2024-04-16T17:04:44.114711+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:04:44.114711+0000","last_undegraded":"2024-04-16T17:04:44.114711+0000","last_fullsized":"2024-04-16T17:04:44.114711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418863+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:04:44.418863+0000","last_peered":"2024-04-16T17:04:44.418863+0000","last_clean":"2024-04-16T17:04:44.418863+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:04:44.418863+0000","last_undegraded":"2024-04-16T17:04:44.418863+0000","last_fullsized":"2024-04-16T17:04:44.418863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'39","reported_seq":81,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.368875+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:04:44.368875+0000","last_peered":"2024-04-16T17:04:44.368875+0000","last_clean":"2024-04-16T17:04:44.368875+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:04:44.368875+0000","last_undegraded":"2024-04-16T17:04:44.368875+0000","last_fullsized":"2024-04-16T17:04:44.368875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114226+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:04:44.114226+0000","last_peered":"2024-04-16T17:04:44.114226+0000","last_clean":"2024-04-16T17:04:44.114226+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:04:44.114226+0000","last_undegraded":"2024-04-16T17:04:44.114226+0000","last_fullsized":"2024-04-16T17:04:44.114226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248179+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:04:44.248179+0000","last_peered":"2024-04-16T17:04:44.248179+0000","last_clean":"2024-04-16T17:04:44.248179+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:04:44.248179+0000","last_undegraded":"2024-04-16T17:04:44.248179+0000","last_fullsized":"2024-04-16T17:04:44.248179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414927+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:04:44.414927+0000","last_peered":"2024-04-16T17:04:44.414927+0000","last_clean":"2024-04-16T17:04:44.414927+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:04:44.414927+0000","last_undegraded":"2024-04-16T17:04:44.414927+0000","last_fullsized":"2024-04-16T17:04:44.414927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114728+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:04:44.114728+0000","last_peered":"2024-04-16T17:04:44.114728+0000","last_clean":"2024-04-16T17:04:44.114728+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:04:44.114728+0000","last_undegraded":"2024-04-16T17:04:44.114728+0000","last_fullsized":"2024-04-16T17:04:44.114728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104236+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:04:44.104236+0000","last_peered":"2024-04-16T17:04:44.104236+0000","last_clean":"2024-04-16T17:04:44.104236+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:04:44.104236+0000","last_undegraded":"2024-04-16T17:04:44.104236+0000","last_fullsized":"2024-04-16T17:04:44.104236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275188+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:04:44.275188+0000","last_peered":"2024-04-16T17:04:44.275188+0000","last_clean":"2024-04-16T17:04:44.275188+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:04:44.275188+0000","last_undegraded":"2024-04-16T17:04:44.275188+0000","last_fullsized":"2024-04-16T17:04:44.275188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274905+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:04:44.274905+0000","last_peered":"2024-04-16T17:04:44.274905+0000","last_clean":"2024-04-16T17:04:44.274905+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:04:44.274905+0000","last_undegraded":"2024-04-16T17:04:44.274905+0000","last_fullsized":"2024-04-16T17:04:44.274905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419653+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:04:44.419653+0000","last_peered":"2024-04-16T17:04:44.419653+0000","last_clean":"2024-04-16T17:04:44.419653+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:04:44.419653+0000","last_undegraded":"2024-04-16T17:04:44.419653+0000","last_fullsized":"2024-04-16T17:04:44.419653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415072+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:04:44.415072+0000","last_peered":"2024-04-16T17:04:44.415072+0000","last_clean":"2024-04-16T17:04:44.415072+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:04:44.415072+0000","last_undegraded":"2024-04-16T17:04:44.415072+0000","last_fullsized":"2024-04-16T17:04:44.415072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104917+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:04:44.104917+0000","last_peered":"2024-04-16T17:04:44.104917+0000","last_clean":"2024-04-16T17:04:44.104917+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:04:44.104917+0000","last_undegraded":"2024-04-16T17:04:44.104917+0000","last_fullsized":"2024-04-16T17:04:44.104917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275489+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:04:44.275489+0000","last_peered":"2024-04-16T17:04:44.275489+0000","last_clean":"2024-04-16T17:04:44.275489+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:04:44.275489+0000","last_undegraded":"2024-04-16T17:04:44.275489+0000","last_fullsized":"2024-04-16T17:04:44.275489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241877+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:04:44.241877+0000","last_peered":"2024-04-16T17:04:44.241877+0000","last_clean":"2024-04-16T17:04:44.241877+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:04:44.241877+0000","last_undegraded":"2024-04-16T17:04:44.241877+0000","last_fullsized":"2024-04-16T17:04:44.241877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248434+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:04:44.248434+0000","last_peered":"2024-04-16T17:04:44.248434+0000","last_clean":"2024-04-16T17:04:44.248434+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:04:44.248434+0000","last_undegraded":"2024-04-16T17:04:44.248434+0000","last_fullsized":"2024-04-16T17:04:44.248434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240652+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:04:44.240652+0000","last_peered":"2024-04-16T17:04:44.240652+0000","last_clean":"2024-04-16T17:04:44.240652+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:04:44.240652+0000","last_undegraded":"2024-04-16T17:04:44.240652+0000","last_fullsized":"2024-04-16T17:04:44.240652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207400+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:04:44.207400+0000","last_peered":"2024-04-16T17:04:44.207400+0000","last_clean":"2024-04-16T17:04:44.207400+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:04:44.207400+0000","last_undegraded":"2024-04-16T17:04:44.207400+0000","last_fullsized":"2024-04-16T17:04:44.207400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415063+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:04:44.415063+0000","last_peered":"2024-04-16T17:04:44.415063+0000","last_clean":"2024-04-16T17:04:44.415063+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:04:44.415063+0000","last_undegraded":"2024-04-16T17:04:44.415063+0000","last_fullsized":"2024-04-16T17:04:44.415063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104214+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:04:44.104214+0000","last_peered":"2024-04-16T17:04:44.104214+0000","last_clean":"2024-04-16T17:04:44.104214+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:04:44.104214+0000","last_undegraded":"2024-04-16T17:04:44.104214+0000","last_fullsized":"2024-04-16T17:04:44.104214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106545,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600829,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1113,"num_read_kb":673,"num_write":292,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1805398,"data_compressed":81384,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383732,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":529733,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36512,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708640,"statfs":{"total":95995035648,"available":95957647360,"internally_reserved":0,"allocated":1204224,"data_stored":1320281,"data_compressed":73169,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808897,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":927259,"data_compressed":48818,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070534,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":923061,"data_compressed":48818,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332171,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1125695,"data_compressed":65187,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495696,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":525123,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":599446,"data_compressed":31498,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":526511,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":596099,"data_compressed":31498,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182497,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1050133,"data_compressed":54974,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":531878,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757336,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":1252883,"data_compressed":71343,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":74413,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71066,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133777,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":204880,"data_compressed":7982,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:04:51.281 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-16T17:04:51.282 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:04:51.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.518+0000 7fa205a00640 1 -- 172.21.15.18:0/2089597846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa200074cc0 msgr2=0x7fa200112990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.518+0000 7fa205a00640 1 --2- 172.21.15.18:0/2089597846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa200074cc0 0x7fa200112990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009860 tx=0x7fa1ec02bc10 comp rx=0 tx=0).stop 2024-04-16T17:04:51.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.519+0000 7fa205a00640 1 -- 172.21.15.18:0/2089597846 shutdown_connections 2024-04-16T17:04:51.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.519+0000 7fa205a00640 1 --2- 172.21.15.18:0/2089597846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa200112ed0 0x7fa2001152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.519+0000 7fa205a00640 1 --2- 172.21.15.18:0/2089597846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa200074cc0 0x7fa200112990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.519+0000 7fa205a00640 1 --2- 172.21.15.18:0/2089597846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2000743a0 0x7fa200074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.519+0000 7fa205a00640 1 -- 172.21.15.18:0/2089597846 >> 172.21.15.18:0/2089597846 conn(0x7fa2000fd9f0 msgr2=0x7fa2000ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:51.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.519+0000 7fa205a00640 1 -- 172.21.15.18:0/2089597846 shutdown_connections 2024-04-16T17:04:51.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.519+0000 7fa205a00640 1 -- 172.21.15.18:0/2089597846 wait complete. 2024-04-16T17:04:51.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.520+0000 7fa205a00640 1 Processor -- start 2024-04-16T17:04:51.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.520+0000 7fa205a00640 1 -- start start 2024-04-16T17:04:51.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.520+0000 7fa205a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2000743a0 0x7fa20019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:51.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.520+0000 7fa205a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200074cc0 0x7fa20019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:51.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.520+0000 7fa205a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa200112ed0 0x7fa2001a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:51.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.520+0000 7fa205a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001188a0 con 0x7fa2000743a0 2024-04-16T17:04:51.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.520+0000 7fa205a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa200118720 con 0x7fa200074cc0 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.520+0000 7fa205a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa200118a20 con 0x7fa200112ed0 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ffe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2000743a0 0x7fa20019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200074cc0 0x7fa20019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200074cc0 0x7fa20019f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54480/0 (socket says 172.21.15.18:54480) 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 -- 172.21.15.18:0/897780152 learned_addr learned my addr 172.21.15.18:0/897780152 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa205000640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa200112ed0 0x7fa2001a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 -- 172.21.15.18:0/897780152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa200112ed0 msgr2=0x7fa2001a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa200112ed0 0x7fa2001a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 -- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2000743a0 msgr2=0x7fa20019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2000743a0 0x7fa20019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 -- 172.21.15.18:0/897780152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2001a3df0 con 0x7fa200074cc0 2024-04-16T17:04:51.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.521+0000 7fa1ff400640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200074cc0 0x7fa20019f340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec037040 tx=0x7fa1ec0344c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:51.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.522+0000 7fa1fcc00640 1 -- 172.21.15.18:0/897780152 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec041020 con 0x7fa200074cc0 2024-04-16T17:04:51.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.522+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2001a4080 con 0x7fa200074cc0 2024-04-16T17:04:51.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.522+0000 7fa1fcc00640 1 -- 172.21.15.18:0/897780152 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ec02ca90 con 0x7fa200074cc0 2024-04-16T17:04:51.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.522+0000 7fa1fcc00640 1 -- 172.21.15.18:0/897780152 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec039040 con 0x7fa200074cc0 2024-04-16T17:04:51.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.522+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2001ab970 con 0x7fa200074cc0 2024-04-16T17:04:51.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.523+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20004fe00 con 0x7fa200074cc0 2024-04-16T17:04:51.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.525+0000 7fa1fcc00640 1 -- 172.21.15.18:0/897780152 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa1ec044020 con 0x7fa200074cc0 2024-04-16T17:04:51.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.526+0000 7fa1fcc00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1e40a3e30 0x7fa1e40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:51.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.527+0000 7fa1fcc00640 1 -- 172.21.15.18:0/897780152 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa1ec0f24b0 con 0x7fa200074cc0 2024-04-16T17:04:51.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.527+0000 7fa1ffe00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1e40a3e30 0x7fa1e40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:51.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.527+0000 7fa1ffe00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1e40a3e30 0x7fa1e40a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009530 tx=0x7fa1f0009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:51.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.531+0000 7fa1fcc00640 1 -- 172.21.15.18:0/897780152 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1ec04b020 con 0x7fa200074cc0 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:51 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:51 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.757+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa200062440 con 0x7fa1e40a3e30 2024-04-16T17:04:51.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.771+0000 7fa1fcc00640 1 -- 172.21.15.18:0/897780152 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339716 (secure 0 0 0) 0x7fa200062440 con 0x7fa1e40a3e30 2024-04-16T17:04:51.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:51.779 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1e40a3e30 msgr2=0x7fa1e40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1e40a3e30 0x7fa1e40a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009530 tx=0x7fa1f0009260 comp rx=0 tx=0).stop 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200074cc0 msgr2=0x7fa20019f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200074cc0 0x7fa20019f340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec037040 tx=0x7fa1ec0344c0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 shutdown_connections 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1e40a3e30 0x7fa1e40a62f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa200112ed0 0x7fa2001a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200074cc0 0x7fa20019f340 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 --2- 172.21.15.18:0/897780152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2000743a0 0x7fa20019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.777+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 >> 172.21.15.18:0/897780152 conn(0x7fa2000fd9f0 msgr2=0x7fa2000fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.778+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 shutdown_connections 2024-04-16T17:04:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:51.778+0000 7fa205a00640 1 -- 172.21.15.18:0/897780152 wait complete. 2024-04-16T17:04:51.798 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-16T17:04:50.125915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191431,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1249,"num_read_kb":792,"num_write":475,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378580,"kb_used_data":12740,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563244,"statfs":{"total":1151940427776,"available":1151552761856,"internally_reserved":0,"allocated":13045760,"data_stored":9908102,"data_compressed":543993,"data_compressed_allocated":737280,"data_compressed_original":4325376,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.970620"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104573+0000","last_change":"2024-04-16T17:01:24.564253+0000","last_active":"2024-04-16T17:04:44.104573+0000","last_peered":"2024-04-16T17:04:44.104573+0000","last_clean":"2024-04-16T17:04:44.104573+0000","last_became_active":"2024-04-16T17:01:24.563959+0000","last_became_peered":"2024-04-16T17:01:24.563959+0000","last_unstale":"2024-04-16T17:04:44.104573+0000","last_undegraded":"2024-04-16T17:04:44.104573+0000","last_fullsized":"2024-04-16T17:04:44.104573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:32:15.022630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419294+0000","last_change":"2024-04-16T17:01:24.560030+0000","last_active":"2024-04-16T17:04:44.419294+0000","last_peered":"2024-04-16T17:04:44.419294+0000","last_clean":"2024-04-16T17:04:44.419294+0000","last_became_active":"2024-04-16T17:01:24.544429+0000","last_became_peered":"2024-04-16T17:01:24.544429+0000","last_unstale":"2024-04-16T17:04:44.419294+0000","last_undegraded":"2024-04-16T17:04:44.419294+0000","last_fullsized":"2024-04-16T17:04:44.419294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:04.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418965+0000","last_change":"2024-04-16T17:01:24.564496+0000","last_active":"2024-04-16T17:04:44.418965+0000","last_peered":"2024-04-16T17:04:44.418965+0000","last_clean":"2024-04-16T17:04:44.418965+0000","last_became_active":"2024-04-16T17:01:24.564059+0000","last_became_peered":"2024-04-16T17:01:24.564059+0000","last_unstale":"2024-04-16T17:04:44.418965+0000","last_undegraded":"2024-04-16T17:04:44.418965+0000","last_fullsized":"2024-04-16T17:04:44.418965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:54.427134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237775+0000","last_change":"2024-04-16T17:02:26.318719+0000","last_active":"2024-04-16T17:04:44.237775+0000","last_peered":"2024-04-16T17:04:44.237775+0000","last_clean":"2024-04-16T17:04:44.237775+0000","last_became_active":"2024-04-16T17:02:26.317743+0000","last_became_peered":"2024-04-16T17:02:26.317743+0000","last_unstale":"2024-04-16T17:04:44.237775+0000","last_undegraded":"2024-04-16T17:04:44.237775+0000","last_fullsized":"2024-04-16T17:04:44.237775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:26.827859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240559+0000","last_change":"2024-04-16T17:01:24.558740+0000","last_active":"2024-04-16T17:04:44.240559+0000","last_peered":"2024-04-16T17:04:44.240559+0000","last_clean":"2024-04-16T17:04:44.240559+0000","last_became_active":"2024-04-16T17:01:24.558382+0000","last_became_peered":"2024-04-16T17:01:24.558382+0000","last_unstale":"2024-04-16T17:04:44.240559+0000","last_undegraded":"2024-04-16T17:04:44.240559+0000","last_fullsized":"2024-04-16T17:04:44.240559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:58.062787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104192+0000","last_change":"2024-04-16T17:02:26.315910+0000","last_active":"2024-04-16T17:04:44.104192+0000","last_peered":"2024-04-16T17:04:44.104192+0000","last_clean":"2024-04-16T17:04:44.104192+0000","last_became_active":"2024-04-16T17:02:26.315493+0000","last_became_peered":"2024-04-16T17:02:26.315493+0000","last_unstale":"2024-04-16T17:04:44.104192+0000","last_undegraded":"2024-04-16T17:04:44.104192+0000","last_fullsized":"2024-04-16T17:04:44.104192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:27:48.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114020+0000","last_change":"2024-04-16T17:01:24.537088+0000","last_active":"2024-04-16T17:04:44.114020+0000","last_peered":"2024-04-16T17:04:44.114020+0000","last_clean":"2024-04-16T17:04:44.114020+0000","last_became_active":"2024-04-16T17:01:24.536749+0000","last_became_peered":"2024-04-16T17:01:24.536749+0000","last_unstale":"2024-04-16T17:04:44.114020+0000","last_undegraded":"2024-04-16T17:04:44.114020+0000","last_fullsized":"2024-04-16T17:04:44.114020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:54.299045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240629+0000","last_change":"2024-04-16T17:01:24.554683+0000","last_active":"2024-04-16T17:04:44.240629+0000","last_peered":"2024-04-16T17:04:44.240629+0000","last_clean":"2024-04-16T17:04:44.240629+0000","last_became_active":"2024-04-16T17:01:24.553920+0000","last_became_peered":"2024-04-16T17:01:24.553920+0000","last_unstale":"2024-04-16T17:04:44.240629+0000","last_undegraded":"2024-04-16T17:04:44.240629+0000","last_fullsized":"2024-04-16T17:04:44.240629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:11.553526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207849+0000","last_change":"2024-04-16T17:01:24.535094+0000","last_active":"2024-04-16T17:04:44.207849+0000","last_peered":"2024-04-16T17:04:44.207849+0000","last_clean":"2024-04-16T17:04:44.207849+0000","last_became_active":"2024-04-16T17:01:24.534749+0000","last_became_peered":"2024-04-16T17:01:24.534749+0000","last_unstale":"2024-04-16T17:04:44.207849+0000","last_undegraded":"2024-04-16T17:04:44.207849+0000","last_fullsized":"2024-04-16T17:04:44.207849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:01.260425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104511+0000","last_change":"2024-04-16T17:01:24.540335+0000","last_active":"2024-04-16T17:04:44.104511+0000","last_peered":"2024-04-16T17:04:44.104511+0000","last_clean":"2024-04-16T17:04:44.104511+0000","last_became_active":"2024-04-16T17:01:24.540016+0000","last_became_peered":"2024-04-16T17:01:24.540016+0000","last_unstale":"2024-04-16T17:04:44.104511+0000","last_undegraded":"2024-04-16T17:04:44.104511+0000","last_fullsized":"2024-04-16T17:04:44.104511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.368718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114664+0000","last_change":"2024-04-16T17:01:24.534036+0000","last_active":"2024-04-16T17:04:44.114664+0000","last_peered":"2024-04-16T17:04:44.114664+0000","last_clean":"2024-04-16T17:04:44.114664+0000","last_became_active":"2024-04-16T17:01:24.533679+0000","last_became_peered":"2024-04-16T17:01:24.533679+0000","last_unstale":"2024-04-16T17:04:44.114664+0000","last_undegraded":"2024-04-16T17:04:44.114664+0000","last_fullsized":"2024-04-16T17:04:44.114664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:24.817872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414524+0000","last_change":"2024-04-16T17:01:24.538483+0000","last_active":"2024-04-16T17:04:44.414524+0000","last_peered":"2024-04-16T17:04:44.414524+0000","last_clean":"2024-04-16T17:04:44.414524+0000","last_became_active":"2024-04-16T17:01:24.538187+0000","last_became_peered":"2024-04-16T17:01:24.538187+0000","last_unstale":"2024-04-16T17:04:44.414524+0000","last_undegraded":"2024-04-16T17:04:44.414524+0000","last_fullsized":"2024-04-16T17:04:44.414524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:16.920427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415190+0000","last_change":"2024-04-16T17:01:24.534536+0000","last_active":"2024-04-16T17:04:44.415190+0000","last_peered":"2024-04-16T17:04:44.415190+0000","last_clean":"2024-04-16T17:04:44.415190+0000","last_became_active":"2024-04-16T17:01:24.534170+0000","last_became_peered":"2024-04-16T17:01:24.534170+0000","last_unstale":"2024-04-16T17:04:44.415190+0000","last_undegraded":"2024-04-16T17:04:44.415190+0000","last_fullsized":"2024-04-16T17:04:44.415190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:19.099205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418945+0000","last_change":"2024-04-16T17:01:24.556126+0000","last_active":"2024-04-16T17:04:44.418945+0000","last_peered":"2024-04-16T17:04:44.418945+0000","last_clean":"2024-04-16T17:04:44.418945+0000","last_became_active":"2024-04-16T17:01:24.553117+0000","last_became_peered":"2024-04-16T17:01:24.553117+0000","last_unstale":"2024-04-16T17:04:44.418945+0000","last_undegraded":"2024-04-16T17:04:44.418945+0000","last_fullsized":"2024-04-16T17:04:44.418945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:40.610519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419182+0000","last_change":"2024-04-16T17:01:24.556660+0000","last_active":"2024-04-16T17:04:44.419182+0000","last_peered":"2024-04-16T17:04:44.419182+0000","last_clean":"2024-04-16T17:04:44.419182+0000","last_became_active":"2024-04-16T17:01:24.556211+0000","last_became_peered":"2024-04-16T17:01:24.556211+0000","last_unstale":"2024-04-16T17:04:44.419182+0000","last_undegraded":"2024-04-16T17:04:44.419182+0000","last_fullsized":"2024-04-16T17:04:44.419182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:49.627065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207293+0000","last_change":"2024-04-16T17:01:24.565080+0000","last_active":"2024-04-16T17:04:44.207293+0000","last_peered":"2024-04-16T17:04:44.207293+0000","last_clean":"2024-04-16T17:04:44.207293+0000","last_became_active":"2024-04-16T17:01:24.564510+0000","last_became_peered":"2024-04-16T17:01:24.564510+0000","last_unstale":"2024-04-16T17:04:44.207293+0000","last_undegraded":"2024-04-16T17:04:44.207293+0000","last_fullsized":"2024-04-16T17:04:44.207293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:35.208573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104522+0000","last_change":"2024-04-16T17:01:24.569470+0000","last_active":"2024-04-16T17:04:44.104522+0000","last_peered":"2024-04-16T17:04:44.104522+0000","last_clean":"2024-04-16T17:04:44.104522+0000","last_became_active":"2024-04-16T17:01:24.569217+0000","last_became_peered":"2024-04-16T17:01:24.569217+0000","last_unstale":"2024-04-16T17:04:44.104522+0000","last_undegraded":"2024-04-16T17:04:44.104522+0000","last_fullsized":"2024-04-16T17:04:44.104522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:24.384082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.238023+0000","last_change":"2024-04-16T17:01:24.543868+0000","last_active":"2024-04-16T17:04:44.238023+0000","last_peered":"2024-04-16T17:04:44.238023+0000","last_clean":"2024-04-16T17:04:44.238023+0000","last_became_active":"2024-04-16T17:01:24.543418+0000","last_became_peered":"2024-04-16T17:01:24.543418+0000","last_unstale":"2024-04-16T17:04:44.238023+0000","last_undegraded":"2024-04-16T17:04:44.238023+0000","last_fullsized":"2024-04-16T17:04:44.238023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:15.080906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275555+0000","last_change":"2024-04-16T17:01:24.559123+0000","last_active":"2024-04-16T17:04:44.275555+0000","last_peered":"2024-04-16T17:04:44.275555+0000","last_clean":"2024-04-16T17:04:44.275555+0000","last_became_active":"2024-04-16T17:01:24.558775+0000","last_became_peered":"2024-04-16T17:01:24.558775+0000","last_unstale":"2024-04-16T17:04:44.275555+0000","last_undegraded":"2024-04-16T17:04:44.275555+0000","last_fullsized":"2024-04-16T17:04:44.275555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:22:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274972+0000","last_change":"2024-04-16T17:01:29.027138+0000","last_active":"2024-04-16T17:04:44.274972+0000","last_peered":"2024-04-16T17:04:44.274972+0000","last_clean":"2024-04-16T17:04:44.274972+0000","last_became_active":"2024-04-16T17:01:29.026752+0000","last_became_peered":"2024-04-16T17:01:29.026752+0000","last_unstale":"2024-04-16T17:04:44.274972+0000","last_undegraded":"2024-04-16T17:04:44.274972+0000","last_fullsized":"2024-04-16T17:04:44.274972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:23.091270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237649+0000","last_change":"2024-04-16T17:01:23.919264+0000","last_active":"2024-04-16T17:04:44.237649+0000","last_peered":"2024-04-16T17:04:44.237649+0000","last_clean":"2024-04-16T17:04:44.237649+0000","last_became_active":"2024-04-16T17:01:23.918382+0000","last_became_peered":"2024-04-16T17:01:23.918382+0000","last_unstale":"2024-04-16T17:04:44.237649+0000","last_undegraded":"2024-04-16T17:04:44.237649+0000","last_fullsized":"2024-04-16T17:04:44.237649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:56:11.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.238023+0000","last_change":"2024-04-16T17:02:26.318546+0000","last_active":"2024-04-16T17:04:44.238023+0000","last_peered":"2024-04-16T17:04:44.238023+0000","last_clean":"2024-04-16T17:04:44.238023+0000","last_became_active":"2024-04-16T17:02:26.317794+0000","last_became_peered":"2024-04-16T17:02:26.317794+0000","last_unstale":"2024-04-16T17:04:44.238023+0000","last_undegraded":"2024-04-16T17:04:44.238023+0000","last_fullsized":"2024-04-16T17:04:44.238023+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:48.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414178+0000","last_change":"2024-04-16T17:01:29.030131+0000","last_active":"2024-04-16T17:04:44.414178+0000","last_peered":"2024-04-16T17:04:44.414178+0000","last_clean":"2024-04-16T17:04:44.414178+0000","last_became_active":"2024-04-16T17:01:29.029634+0000","last_became_peered":"2024-04-16T17:01:29.029634+0000","last_unstale":"2024-04-16T17:04:44.414178+0000","last_undegraded":"2024-04-16T17:04:44.414178+0000","last_fullsized":"2024-04-16T17:04:44.414178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:34.822560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241729+0000","last_change":"2024-04-16T17:01:23.939470+0000","last_active":"2024-04-16T17:04:44.241729+0000","last_peered":"2024-04-16T17:04:44.241729+0000","last_clean":"2024-04-16T17:04:44.241729+0000","last_became_active":"2024-04-16T17:01:23.939089+0000","last_became_peered":"2024-04-16T17:01:23.939089+0000","last_unstale":"2024-04-16T17:04:44.241729+0000","last_undegraded":"2024-04-16T17:04:44.241729+0000","last_fullsized":"2024-04-16T17:04:44.241729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:42.296202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419515+0000","last_change":"2024-04-16T17:01:24.558709+0000","last_active":"2024-04-16T17:04:44.419515+0000","last_peered":"2024-04-16T17:04:44.419515+0000","last_clean":"2024-04-16T17:04:44.419515+0000","last_became_active":"2024-04-16T17:01:24.557348+0000","last_became_peered":"2024-04-16T17:01:24.557348+0000","last_unstale":"2024-04-16T17:04:44.419515+0000","last_undegraded":"2024-04-16T17:04:44.419515+0000","last_fullsized":"2024-04-16T17:04:44.419515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:12.428837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414312+0000","last_change":"2024-04-16T17:03:27.271585+0000","last_active":"2024-04-16T17:04:44.414312+0000","last_peered":"2024-04-16T17:04:44.414312+0000","last_clean":"2024-04-16T17:04:44.414312+0000","last_became_active":"2024-04-16T17:03:27.271251+0000","last_became_peered":"2024-04-16T17:03:27.271251+0000","last_unstale":"2024-04-16T17:04:44.414312+0000","last_undegraded":"2024-04-16T17:04:44.414312+0000","last_fullsized":"2024-04-16T17:04:44.414312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:51.217466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241064+0000","last_change":"2024-04-16T17:01:23.914752+0000","last_active":"2024-04-16T17:04:44.241064+0000","last_peered":"2024-04-16T17:04:44.241064+0000","last_clean":"2024-04-16T17:04:44.241064+0000","last_became_active":"2024-04-16T17:01:23.914235+0000","last_became_peered":"2024-04-16T17:01:23.914235+0000","last_unstale":"2024-04-16T17:04:44.241064+0000","last_undegraded":"2024-04-16T17:04:44.241064+0000","last_fullsized":"2024-04-16T17:04:44.241064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:45:27.900021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207578+0000","last_change":"2024-04-16T17:01:24.534327+0000","last_active":"2024-04-16T17:04:44.207578+0000","last_peered":"2024-04-16T17:04:44.207578+0000","last_clean":"2024-04-16T17:04:44.207578+0000","last_became_active":"2024-04-16T17:01:24.533922+0000","last_became_peered":"2024-04-16T17:01:24.533922+0000","last_unstale":"2024-04-16T17:04:44.207578+0000","last_undegraded":"2024-04-16T17:04:44.207578+0000","last_fullsized":"2024-04-16T17:04:44.207578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:43:21.376035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414211+0000","last_change":"2024-04-16T17:01:29.029630+0000","last_active":"2024-04-16T17:04:44.414211+0000","last_peered":"2024-04-16T17:04:44.414211+0000","last_clean":"2024-04-16T17:04:44.414211+0000","last_became_active":"2024-04-16T17:01:29.029176+0000","last_became_peered":"2024-04-16T17:01:29.029176+0000","last_unstale":"2024-04-16T17:04:44.414211+0000","last_undegraded":"2024-04-16T17:04:44.414211+0000","last_fullsized":"2024-04-16T17:04:44.414211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:30.710704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240928+0000","last_change":"2024-04-16T17:01:23.909231+0000","last_active":"2024-04-16T17:04:44.240928+0000","last_peered":"2024-04-16T17:04:44.240928+0000","last_clean":"2024-04-16T17:04:44.240928+0000","last_became_active":"2024-04-16T17:01:23.908905+0000","last_became_peered":"2024-04-16T17:01:23.908905+0000","last_unstale":"2024-04-16T17:04:44.240928+0000","last_undegraded":"2024-04-16T17:04:44.240928+0000","last_fullsized":"2024-04-16T17:04:44.240928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:51.542023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248909+0000","last_change":"2024-04-16T17:01:24.535087+0000","last_active":"2024-04-16T17:04:44.248909+0000","last_peered":"2024-04-16T17:04:44.248909+0000","last_clean":"2024-04-16T17:04:44.248909+0000","last_became_active":"2024-04-16T17:01:24.534758+0000","last_became_peered":"2024-04-16T17:01:24.534758+0000","last_unstale":"2024-04-16T17:04:44.248909+0000","last_undegraded":"2024-04-16T17:04:44.248909+0000","last_fullsized":"2024-04-16T17:04:44.248909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:31:13.371890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275231+0000","last_change":"2024-04-16T17:01:29.008593+0000","last_active":"2024-04-16T17:04:44.275231+0000","last_peered":"2024-04-16T17:04:44.275231+0000","last_clean":"2024-04-16T17:04:44.275231+0000","last_became_active":"2024-04-16T17:01:29.008174+0000","last_became_peered":"2024-04-16T17:01:29.008174+0000","last_unstale":"2024-04-16T17:04:44.275231+0000","last_undegraded":"2024-04-16T17:04:44.275231+0000","last_fullsized":"2024-04-16T17:04:44.275231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:54.652788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":223,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.506138+0000","last_change":"2024-04-16T17:02:26.315434+0000","last_active":"2024-04-16T17:04:44.506138+0000","last_peered":"2024-04-16T17:04:44.506138+0000","last_clean":"2024-04-16T17:04:44.506138+0000","last_became_active":"2024-04-16T17:02:26.310169+0000","last_became_peered":"2024-04-16T17:02:26.310169+0000","last_unstale":"2024-04-16T17:04:44.506138+0000","last_undegraded":"2024-04-16T17:04:44.506138+0000","last_fullsized":"2024-04-16T17:04:44.506138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:51.033415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":164,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240363+0000","last_change":"2024-04-16T17:01:24.543167+0000","last_active":"2024-04-16T17:04:44.240363+0000","last_peered":"2024-04-16T17:04:44.240363+0000","last_clean":"2024-04-16T17:04:44.240363+0000","last_became_active":"2024-04-16T17:01:24.542809+0000","last_became_peered":"2024-04-16T17:01:24.542809+0000","last_unstale":"2024-04-16T17:04:44.240363+0000","last_undegraded":"2024-04-16T17:04:44.240363+0000","last_fullsized":"2024-04-16T17:04:44.240363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:24.969567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275338+0000","last_change":"2024-04-16T17:01:29.024938+0000","last_active":"2024-04-16T17:04:44.275338+0000","last_peered":"2024-04-16T17:04:44.275338+0000","last_clean":"2024-04-16T17:04:44.275338+0000","last_became_active":"2024-04-16T17:01:29.024481+0000","last_became_peered":"2024-04-16T17:01:29.024481+0000","last_unstale":"2024-04-16T17:04:44.275338+0000","last_undegraded":"2024-04-16T17:04:44.275338+0000","last_fullsized":"2024-04-16T17:04:44.275338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:48:39.146994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104007+0000","last_change":"2024-04-16T17:01:23.935818+0000","last_active":"2024-04-16T17:04:44.104007+0000","last_peered":"2024-04-16T17:04:44.104007+0000","last_clean":"2024-04-16T17:04:44.104007+0000","last_became_active":"2024-04-16T17:01:23.935564+0000","last_became_peered":"2024-04-16T17:01:23.935564+0000","last_unstale":"2024-04-16T17:04:44.104007+0000","last_undegraded":"2024-04-16T17:04:44.104007+0000","last_fullsized":"2024-04-16T17:04:44.104007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:09.864085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237470+0000","last_change":"2024-04-16T17:02:26.321763+0000","last_active":"2024-04-16T17:04:44.237470+0000","last_peered":"2024-04-16T17:04:44.237470+0000","last_clean":"2024-04-16T17:04:44.237470+0000","last_became_active":"2024-04-16T17:02:26.321438+0000","last_became_peered":"2024-04-16T17:02:26.321438+0000","last_unstale":"2024-04-16T17:04:44.237470+0000","last_undegraded":"2024-04-16T17:04:44.237470+0000","last_fullsized":"2024-04-16T17:04:44.237470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:47.231035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104252+0000","last_change":"2024-04-16T17:03:27.451220+0000","last_active":"2024-04-16T17:04:44.104252+0000","last_peered":"2024-04-16T17:04:44.104252+0000","last_clean":"2024-04-16T17:04:44.104252+0000","last_became_active":"2024-04-16T17:03:27.450680+0000","last_became_peered":"2024-04-16T17:03:27.450680+0000","last_unstale":"2024-04-16T17:04:44.104252+0000","last_undegraded":"2024-04-16T17:04:44.104252+0000","last_fullsized":"2024-04-16T17:04:44.104252+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:23.270332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414603+0000","last_change":"2024-04-16T17:01:23.921895+0000","last_active":"2024-04-16T17:04:44.414603+0000","last_peered":"2024-04-16T17:04:44.414603+0000","last_clean":"2024-04-16T17:04:44.414603+0000","last_became_active":"2024-04-16T17:01:23.921511+0000","last_became_peered":"2024-04-16T17:01:23.921511+0000","last_unstale":"2024-04-16T17:04:44.414603+0000","last_undegraded":"2024-04-16T17:04:44.414603+0000","last_fullsized":"2024-04-16T17:04:44.414603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:56.298529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241007+0000","last_change":"2024-04-16T17:01:24.554947+0000","last_active":"2024-04-16T17:04:44.241007+0000","last_peered":"2024-04-16T17:04:44.241007+0000","last_clean":"2024-04-16T17:04:44.241007+0000","last_became_active":"2024-04-16T17:01:24.554295+0000","last_became_peered":"2024-04-16T17:01:24.554295+0000","last_unstale":"2024-04-16T17:04:44.241007+0000","last_undegraded":"2024-04-16T17:04:44.241007+0000","last_fullsized":"2024-04-16T17:04:44.241007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:18.689594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419271+0000","last_change":"2024-04-16T17:01:29.016381+0000","last_active":"2024-04-16T17:04:44.419271+0000","last_peered":"2024-04-16T17:04:44.419271+0000","last_clean":"2024-04-16T17:04:44.419271+0000","last_became_active":"2024-04-16T17:01:29.015826+0000","last_became_peered":"2024-04-16T17:01:29.015826+0000","last_unstale":"2024-04-16T17:04:44.419271+0000","last_undegraded":"2024-04-16T17:04:44.419271+0000","last_fullsized":"2024-04-16T17:04:44.419271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:07.806113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248395+0000","last_change":"2024-04-16T17:01:23.912394+0000","last_active":"2024-04-16T17:04:44.248395+0000","last_peered":"2024-04-16T17:04:44.248395+0000","last_clean":"2024-04-16T17:04:44.248395+0000","last_became_active":"2024-04-16T17:01:23.912058+0000","last_became_peered":"2024-04-16T17:01:23.912058+0000","last_unstale":"2024-04-16T17:04:44.248395+0000","last_undegraded":"2024-04-16T17:04:44.248395+0000","last_fullsized":"2024-04-16T17:04:44.248395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:21.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237507+0000","last_change":"2024-04-16T17:01:24.568488+0000","last_active":"2024-04-16T17:04:44.237507+0000","last_peered":"2024-04-16T17:04:44.237507+0000","last_clean":"2024-04-16T17:04:44.237507+0000","last_became_active":"2024-04-16T17:01:24.568109+0000","last_became_peered":"2024-04-16T17:01:24.568109+0000","last_unstale":"2024-04-16T17:04:44.237507+0000","last_undegraded":"2024-04-16T17:04:44.237507+0000","last_fullsized":"2024-04-16T17:04:44.237507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.259199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240883+0000","last_change":"2024-04-16T17:01:29.024551+0000","last_active":"2024-04-16T17:04:44.240883+0000","last_peered":"2024-04-16T17:04:44.240883+0000","last_clean":"2024-04-16T17:04:44.240883+0000","last_became_active":"2024-04-16T17:01:29.024034+0000","last_became_peered":"2024-04-16T17:01:29.024034+0000","last_unstale":"2024-04-16T17:04:44.240883+0000","last_undegraded":"2024-04-16T17:04:44.240883+0000","last_fullsized":"2024-04-16T17:04:44.240883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:35:11.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":204,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104418+0000","last_change":"2024-04-16T17:01:23.920537+0000","last_active":"2024-04-16T17:04:44.104418+0000","last_peered":"2024-04-16T17:04:44.104418+0000","last_clean":"2024-04-16T17:04:44.104418+0000","last_became_active":"2024-04-16T17:01:23.920265+0000","last_became_peered":"2024-04-16T17:01:23.920265+0000","last_unstale":"2024-04-16T17:04:44.104418+0000","last_undegraded":"2024-04-16T17:04:44.104418+0000","last_fullsized":"2024-04-16T17:04:44.104418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:49.911747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207667+0000","last_change":"2024-04-16T17:01:24.545518+0000","last_active":"2024-04-16T17:04:44.207667+0000","last_peered":"2024-04-16T17:04:44.207667+0000","last_clean":"2024-04-16T17:04:44.207667+0000","last_became_active":"2024-04-16T17:01:24.545199+0000","last_became_peered":"2024-04-16T17:01:24.545199+0000","last_unstale":"2024-04-16T17:04:44.207667+0000","last_undegraded":"2024-04-16T17:04:44.207667+0000","last_fullsized":"2024-04-16T17:04:44.207667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:39.627860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237402+0000","last_change":"2024-04-16T17:01:29.024960+0000","last_active":"2024-04-16T17:04:44.237402+0000","last_peered":"2024-04-16T17:04:44.237402+0000","last_clean":"2024-04-16T17:04:44.237402+0000","last_became_active":"2024-04-16T17:01:29.023349+0000","last_became_peered":"2024-04-16T17:01:29.023349+0000","last_unstale":"2024-04-16T17:04:44.237402+0000","last_undegraded":"2024-04-16T17:04:44.237402+0000","last_fullsized":"2024-04-16T17:04:44.237402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:09:45.373837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114484+0000","last_change":"2024-04-16T17:01:23.914254+0000","last_active":"2024-04-16T17:04:44.114484+0000","last_peered":"2024-04-16T17:04:44.114484+0000","last_clean":"2024-04-16T17:04:44.114484+0000","last_became_active":"2024-04-16T17:01:23.913751+0000","last_became_peered":"2024-04-16T17:01:23.913751+0000","last_unstale":"2024-04-16T17:04:44.114484+0000","last_undegraded":"2024-04-16T17:04:44.114484+0000","last_fullsized":"2024-04-16T17:04:44.114484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:29.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419897+0000","last_change":"2024-04-16T17:02:26.752275+0000","last_active":"2024-04-16T17:04:44.419897+0000","last_peered":"2024-04-16T17:04:44.419897+0000","last_clean":"2024-04-16T17:04:44.419897+0000","last_became_active":"2024-04-16T17:02:26.751729+0000","last_became_peered":"2024-04-16T17:02:26.751729+0000","last_unstale":"2024-04-16T17:04:44.419897+0000","last_undegraded":"2024-04-16T17:04:44.419897+0000","last_fullsized":"2024-04-16T17:04:44.419897+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:41:35.777691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414493+0000","last_change":"2024-04-16T17:01:29.035248+0000","last_active":"2024-04-16T17:04:44.414493+0000","last_peered":"2024-04-16T17:04:44.414493+0000","last_clean":"2024-04-16T17:04:44.414493+0000","last_became_active":"2024-04-16T17:01:29.034715+0000","last_became_peered":"2024-04-16T17:01:29.034715+0000","last_unstale":"2024-04-16T17:04:44.414493+0000","last_undegraded":"2024-04-16T17:04:44.414493+0000","last_fullsized":"2024-04-16T17:04:44.414493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:01.811837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248851+0000","last_change":"2024-04-16T17:01:23.932247+0000","last_active":"2024-04-16T17:04:44.248851+0000","last_peered":"2024-04-16T17:04:44.248851+0000","last_clean":"2024-04-16T17:04:44.248851+0000","last_became_active":"2024-04-16T17:01:23.932010+0000","last_became_peered":"2024-04-16T17:01:23.932010+0000","last_unstale":"2024-04-16T17:04:44.248851+0000","last_undegraded":"2024-04-16T17:04:44.248851+0000","last_fullsized":"2024-04-16T17:04:44.248851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:03.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207706+0000","last_change":"2024-04-16T17:01:24.538686+0000","last_active":"2024-04-16T17:04:44.207706+0000","last_peered":"2024-04-16T17:04:44.207706+0000","last_clean":"2024-04-16T17:04:44.207706+0000","last_became_active":"2024-04-16T17:01:24.538321+0000","last_became_peered":"2024-04-16T17:01:24.538321+0000","last_unstale":"2024-04-16T17:04:44.207706+0000","last_undegraded":"2024-04-16T17:04:44.207706+0000","last_fullsized":"2024-04-16T17:04:44.207706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:18:36.977929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103750+0000","last_change":"2024-04-16T17:03:27.054627+0000","last_active":"2024-04-16T17:04:44.103750+0000","last_peered":"2024-04-16T17:04:44.103750+0000","last_clean":"2024-04-16T17:04:44.103750+0000","last_became_active":"2024-04-16T17:03:27.054326+0000","last_became_peered":"2024-04-16T17:03:27.054326+0000","last_unstale":"2024-04-16T17:04:44.103750+0000","last_undegraded":"2024-04-16T17:04:44.103750+0000","last_fullsized":"2024-04-16T17:04:44.103750+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:43.535958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":419,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.505895+0000","last_change":"2024-04-16T17:01:23.914164+0000","last_active":"2024-04-16T17:04:44.505895+0000","last_peered":"2024-04-16T17:04:44.505895+0000","last_clean":"2024-04-16T17:04:44.505895+0000","last_became_active":"2024-04-16T17:01:23.913760+0000","last_became_peered":"2024-04-16T17:01:23.913760+0000","last_unstale":"2024-04-16T17:04:44.505895+0000","last_undegraded":"2024-04-16T17:04:44.505895+0000","last_fullsized":"2024-04-16T17:04:44.505895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:56.697627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70920,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":344,"num_read_kb":243,"num_write":42,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240855+0000","last_change":"2024-04-16T17:01:24.532763+0000","last_active":"2024-04-16T17:04:44.240855+0000","last_peered":"2024-04-16T17:04:44.240855+0000","last_clean":"2024-04-16T17:04:44.240855+0000","last_became_active":"2024-04-16T17:01:24.531945+0000","last_became_peered":"2024-04-16T17:01:24.531945+0000","last_unstale":"2024-04-16T17:04:44.240855+0000","last_undegraded":"2024-04-16T17:04:44.240855+0000","last_fullsized":"2024-04-16T17:04:44.240855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:05.764870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.249232+0000","last_change":"2024-04-16T17:01:29.027881+0000","last_active":"2024-04-16T17:04:44.249232+0000","last_peered":"2024-04-16T17:04:44.249232+0000","last_clean":"2024-04-16T17:04:44.249232+0000","last_became_active":"2024-04-16T17:01:29.027471+0000","last_became_peered":"2024-04-16T17:01:29.027471+0000","last_unstale":"2024-04-16T17:04:44.249232+0000","last_undegraded":"2024-04-16T17:04:44.249232+0000","last_fullsized":"2024-04-16T17:04:44.249232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:03:09.186852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275182+0000","last_change":"2024-04-16T17:01:23.928302+0000","last_active":"2024-04-16T17:04:44.275182+0000","last_peered":"2024-04-16T17:04:44.275182+0000","last_clean":"2024-04-16T17:04:44.275182+0000","last_became_active":"2024-04-16T17:01:23.927971+0000","last_became_peered":"2024-04-16T17:01:23.927971+0000","last_unstale":"2024-04-16T17:04:44.275182+0000","last_undegraded":"2024-04-16T17:04:44.275182+0000","last_fullsized":"2024-04-16T17:04:44.275182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:03:00.301589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415497+0000","last_change":"2024-04-16T17:01:24.550235+0000","last_active":"2024-04-16T17:04:44.415497+0000","last_peered":"2024-04-16T17:04:44.415497+0000","last_clean":"2024-04-16T17:04:44.415497+0000","last_became_active":"2024-04-16T17:01:24.549663+0000","last_became_peered":"2024-04-16T17:01:24.549663+0000","last_unstale":"2024-04-16T17:04:44.415497+0000","last_undegraded":"2024-04-16T17:04:44.415497+0000","last_fullsized":"2024-04-16T17:04:44.415497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:14:12.003682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104076+0000","last_change":"2024-04-16T17:01:29.034393+0000","last_active":"2024-04-16T17:04:44.104076+0000","last_peered":"2024-04-16T17:04:44.104076+0000","last_clean":"2024-04-16T17:04:44.104076+0000","last_became_active":"2024-04-16T17:01:29.034020+0000","last_became_peered":"2024-04-16T17:01:29.034020+0000","last_unstale":"2024-04-16T17:04:44.104076+0000","last_undegraded":"2024-04-16T17:04:44.104076+0000","last_fullsized":"2024-04-16T17:04:44.104076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:33.825813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248492+0000","last_change":"2024-04-16T17:01:23.940552+0000","last_active":"2024-04-16T17:04:44.248492+0000","last_peered":"2024-04-16T17:04:44.248492+0000","last_clean":"2024-04-16T17:04:44.248492+0000","last_became_active":"2024-04-16T17:01:23.940239+0000","last_became_peered":"2024-04-16T17:01:23.940239+0000","last_unstale":"2024-04-16T17:04:44.248492+0000","last_undegraded":"2024-04-16T17:04:44.248492+0000","last_fullsized":"2024-04-16T17:04:44.248492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:21.571129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114507+0000","last_change":"2024-04-16T17:01:24.538057+0000","last_active":"2024-04-16T17:04:44.114507+0000","last_peered":"2024-04-16T17:04:44.114507+0000","last_clean":"2024-04-16T17:04:44.114507+0000","last_became_active":"2024-04-16T17:01:24.537718+0000","last_became_peered":"2024-04-16T17:01:24.537718+0000","last_unstale":"2024-04-16T17:04:44.114507+0000","last_undegraded":"2024-04-16T17:04:44.114507+0000","last_fullsized":"2024-04-16T17:04:44.114507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:33.917479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237521+0000","last_change":"2024-04-16T17:01:29.030495+0000","last_active":"2024-04-16T17:04:44.237521+0000","last_peered":"2024-04-16T17:04:44.237521+0000","last_clean":"2024-04-16T17:04:44.237521+0000","last_became_active":"2024-04-16T17:01:29.030063+0000","last_became_peered":"2024-04-16T17:01:29.030063+0000","last_unstale":"2024-04-16T17:04:44.237521+0000","last_undegraded":"2024-04-16T17:04:44.237521+0000","last_fullsized":"2024-04-16T17:04:44.237521+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.491434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":208,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.536313+0000","last_change":"2024-04-16T17:01:23.911917+0000","last_active":"2024-04-16T17:04:44.536313+0000","last_peered":"2024-04-16T17:04:44.536313+0000","last_clean":"2024-04-16T17:04:44.536313+0000","last_became_active":"2024-04-16T17:01:23.911550+0000","last_became_peered":"2024-04-16T17:01:23.911550+0000","last_unstale":"2024-04-16T17:04:44.536313+0000","last_undegraded":"2024-04-16T17:04:44.536313+0000","last_fullsized":"2024-04-16T17:04:44.536313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:37:31.493137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":170,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.113893+0000","last_change":"2024-04-16T17:01:24.535640+0000","last_active":"2024-04-16T17:04:44.113893+0000","last_peered":"2024-04-16T17:04:44.113893+0000","last_clean":"2024-04-16T17:04:44.113893+0000","last_became_active":"2024-04-16T17:01:24.535328+0000","last_became_peered":"2024-04-16T17:01:24.535328+0000","last_unstale":"2024-04-16T17:04:44.113893+0000","last_undegraded":"2024-04-16T17:04:44.113893+0000","last_fullsized":"2024-04-16T17:04:44.113893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:05:41.267910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415000+0000","last_change":"2024-04-16T17:01:29.030269+0000","last_active":"2024-04-16T17:04:44.415000+0000","last_peered":"2024-04-16T17:04:44.415000+0000","last_clean":"2024-04-16T17:04:44.415000+0000","last_became_active":"2024-04-16T17:01:29.029100+0000","last_became_peered":"2024-04-16T17:01:29.029100+0000","last_unstale":"2024-04-16T17:04:44.415000+0000","last_undegraded":"2024-04-16T17:04:44.415000+0000","last_fullsized":"2024-04-16T17:04:44.415000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:45.785749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240346+0000","last_change":"2024-04-16T17:01:23.919397+0000","last_active":"2024-04-16T17:04:44.240346+0000","last_peered":"2024-04-16T17:04:44.240346+0000","last_clean":"2024-04-16T17:04:44.240346+0000","last_became_active":"2024-04-16T17:01:23.919019+0000","last_became_peered":"2024-04-16T17:01:23.919019+0000","last_unstale":"2024-04-16T17:04:44.240346+0000","last_undegraded":"2024-04-16T17:04:44.240346+0000","last_fullsized":"2024-04-16T17:04:44.240346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:52.511915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248651+0000","last_change":"2024-04-16T17:01:24.536098+0000","last_active":"2024-04-16T17:04:44.248651+0000","last_peered":"2024-04-16T17:04:44.248651+0000","last_clean":"2024-04-16T17:04:44.248651+0000","last_became_active":"2024-04-16T17:01:24.535778+0000","last_became_peered":"2024-04-16T17:01:24.535778+0000","last_unstale":"2024-04-16T17:04:44.248651+0000","last_undegraded":"2024-04-16T17:04:44.248651+0000","last_fullsized":"2024-04-16T17:04:44.248651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:13.525847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248417+0000","last_change":"2024-04-16T17:01:29.027426+0000","last_active":"2024-04-16T17:04:44.248417+0000","last_peered":"2024-04-16T17:04:44.248417+0000","last_clean":"2024-04-16T17:04:44.248417+0000","last_became_active":"2024-04-16T17:01:29.026884+0000","last_became_peered":"2024-04-16T17:01:29.026884+0000","last_unstale":"2024-04-16T17:04:44.248417+0000","last_undegraded":"2024-04-16T17:04:44.248417+0000","last_fullsized":"2024-04-16T17:04:44.248417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:34.535694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104487+0000","last_change":"2024-04-16T17:02:26.320405+0000","last_active":"2024-04-16T17:04:44.104487+0000","last_peered":"2024-04-16T17:04:44.104487+0000","last_clean":"2024-04-16T17:04:44.104487+0000","last_became_active":"2024-04-16T17:02:26.307921+0000","last_became_peered":"2024-04-16T17:02:26.307921+0000","last_unstale":"2024-04-16T17:04:44.104487+0000","last_undegraded":"2024-04-16T17:04:44.104487+0000","last_fullsized":"2024-04-16T17:04:44.104487+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:19.156260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104216+0000","last_change":"2024-04-16T17:01:24.536137+0000","last_active":"2024-04-16T17:04:44.104216+0000","last_peered":"2024-04-16T17:04:44.104216+0000","last_clean":"2024-04-16T17:04:44.104216+0000","last_became_active":"2024-04-16T17:01:24.535631+0000","last_became_peered":"2024-04-16T17:01:24.535631+0000","last_unstale":"2024-04-16T17:04:44.104216+0000","last_undegraded":"2024-04-16T17:04:44.104216+0000","last_fullsized":"2024-04-16T17:04:44.104216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:41.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240562+0000","last_change":"2024-04-16T17:01:29.015535+0000","last_active":"2024-04-16T17:04:44.240562+0000","last_peered":"2024-04-16T17:04:44.240562+0000","last_clean":"2024-04-16T17:04:44.240562+0000","last_became_active":"2024-04-16T17:01:29.015094+0000","last_became_peered":"2024-04-16T17:01:29.015094+0000","last_unstale":"2024-04-16T17:04:44.240562+0000","last_undegraded":"2024-04-16T17:04:44.240562+0000","last_fullsized":"2024-04-16T17:04:44.240562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:44:40.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.237720+0000","last_change":"2024-04-16T17:01:23.933793+0000","last_active":"2024-04-16T17:04:44.237720+0000","last_peered":"2024-04-16T17:04:44.237720+0000","last_clean":"2024-04-16T17:04:44.237720+0000","last_became_active":"2024-04-16T17:01:23.933246+0000","last_became_peered":"2024-04-16T17:01:23.933246+0000","last_unstale":"2024-04-16T17:04:44.237720+0000","last_undegraded":"2024-04-16T17:04:44.237720+0000","last_fullsized":"2024-04-16T17:04:44.237720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:27.293351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104042+0000","last_change":"2024-04-16T17:01:24.559320+0000","last_active":"2024-04-16T17:04:44.104042+0000","last_peered":"2024-04-16T17:04:44.104042+0000","last_clean":"2024-04-16T17:04:44.104042+0000","last_became_active":"2024-04-16T17:01:24.558933+0000","last_became_peered":"2024-04-16T17:01:24.558933+0000","last_unstale":"2024-04-16T17:04:44.104042+0000","last_undegraded":"2024-04-16T17:04:44.104042+0000","last_fullsized":"2024-04-16T17:04:44.104042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:40.625876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414819+0000","last_change":"2024-04-16T17:01:29.020625+0000","last_active":"2024-04-16T17:04:44.414819+0000","last_peered":"2024-04-16T17:04:44.414819+0000","last_clean":"2024-04-16T17:04:44.414819+0000","last_became_active":"2024-04-16T17:01:29.020040+0000","last_became_peered":"2024-04-16T17:01:29.020040+0000","last_unstale":"2024-04-16T17:04:44.414819+0000","last_undegraded":"2024-04-16T17:04:44.414819+0000","last_fullsized":"2024-04-16T17:04:44.414819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:16.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114221+0000","last_change":"2024-04-16T17:01:23.933271+0000","last_active":"2024-04-16T17:04:44.114221+0000","last_peered":"2024-04-16T17:04:44.114221+0000","last_clean":"2024-04-16T17:04:44.114221+0000","last_became_active":"2024-04-16T17:01:23.932862+0000","last_became_peered":"2024-04-16T17:01:23.932862+0000","last_unstale":"2024-04-16T17:04:44.114221+0000","last_undegraded":"2024-04-16T17:04:44.114221+0000","last_fullsized":"2024-04-16T17:04:44.114221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:45.893937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104271+0000","last_change":"2024-04-16T17:01:24.560706+0000","last_active":"2024-04-16T17:04:44.104271+0000","last_peered":"2024-04-16T17:04:44.104271+0000","last_clean":"2024-04-16T17:04:44.104271+0000","last_became_active":"2024-04-16T17:01:24.560327+0000","last_became_peered":"2024-04-16T17:01:24.560327+0000","last_unstale":"2024-04-16T17:04:44.104271+0000","last_undegraded":"2024-04-16T17:04:44.104271+0000","last_fullsized":"2024-04-16T17:04:44.104271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:57.617867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415017+0000","last_change":"2024-04-16T17:01:29.024468+0000","last_active":"2024-04-16T17:04:44.415017+0000","last_peered":"2024-04-16T17:04:44.415017+0000","last_clean":"2024-04-16T17:04:44.415017+0000","last_became_active":"2024-04-16T17:01:29.021324+0000","last_became_peered":"2024-04-16T17:01:29.021324+0000","last_unstale":"2024-04-16T17:04:44.415017+0000","last_undegraded":"2024-04-16T17:04:44.415017+0000","last_fullsized":"2024-04-16T17:04:44.415017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:59.549008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248603+0000","last_change":"2024-04-16T17:01:24.533592+0000","last_active":"2024-04-16T17:04:44.248603+0000","last_peered":"2024-04-16T17:04:44.248603+0000","last_clean":"2024-04-16T17:04:44.248603+0000","last_became_active":"2024-04-16T17:01:24.533358+0000","last_became_peered":"2024-04-16T17:01:24.533358+0000","last_unstale":"2024-04-16T17:04:44.248603+0000","last_undegraded":"2024-04-16T17:04:44.248603+0000","last_fullsized":"2024-04-16T17:04:44.248603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:39.906126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":205,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.417263+0000","last_change":"2024-04-16T17:01:23.938983+0000","last_active":"2024-04-16T17:04:44.417263+0000","last_peered":"2024-04-16T17:04:44.417263+0000","last_clean":"2024-04-16T17:04:44.417263+0000","last_became_active":"2024-04-16T17:01:23.938549+0000","last_became_peered":"2024-04-16T17:01:23.938549+0000","last_unstale":"2024-04-16T17:04:44.417263+0000","last_undegraded":"2024-04-16T17:04:44.417263+0000","last_fullsized":"2024-04-16T17:04:44.417263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:52.146649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114408+0000","last_change":"2024-04-16T17:01:29.014428+0000","last_active":"2024-04-16T17:04:44.114408+0000","last_peered":"2024-04-16T17:04:44.114408+0000","last_clean":"2024-04-16T17:04:44.114408+0000","last_became_active":"2024-04-16T17:01:29.013992+0000","last_became_peered":"2024-04-16T17:01:29.013992+0000","last_unstale":"2024-04-16T17:04:44.114408+0000","last_undegraded":"2024-04-16T17:04:44.114408+0000","last_fullsized":"2024-04-16T17:04:44.114408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:49:32.693800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240965+0000","last_change":"2024-04-16T17:02:26.322313+0000","last_active":"2024-04-16T17:04:44.240965+0000","last_peered":"2024-04-16T17:04:44.240965+0000","last_clean":"2024-04-16T17:04:44.240965+0000","last_became_active":"2024-04-16T17:02:26.320328+0000","last_became_peered":"2024-04-16T17:02:26.320328+0000","last_unstale":"2024-04-16T17:04:44.240965+0000","last_undegraded":"2024-04-16T17:04:44.240965+0000","last_fullsized":"2024-04-16T17:04:44.240965+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:40.943183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414366+0000","last_change":"2024-04-16T17:01:23.919033+0000","last_active":"2024-04-16T17:04:44.414366+0000","last_peered":"2024-04-16T17:04:44.414366+0000","last_clean":"2024-04-16T17:04:44.414366+0000","last_became_active":"2024-04-16T17:01:23.918617+0000","last_became_peered":"2024-04-16T17:01:23.918617+0000","last_unstale":"2024-04-16T17:04:44.414366+0000","last_undegraded":"2024-04-16T17:04:44.414366+0000","last_fullsized":"2024-04-16T17:04:44.414366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:44.031010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.113827+0000","last_change":"2024-04-16T17:00:01.154321+0000","last_active":"2024-04-16T17:04:44.113827+0000","last_peered":"2024-04-16T17:04:44.113827+0000","last_clean":"2024-04-16T17:04:44.113827+0000","last_became_active":"2024-04-16T17:00:01.153764+0000","last_became_peered":"2024-04-16T17:00:01.153764+0000","last_unstale":"2024-04-16T17:04:44.113827+0000","last_undegraded":"2024-04-16T17:04:44.113827+0000","last_fullsized":"2024-04-16T17:04:44.113827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:56:09.738923+0000","last_clean_scrub_stamp":"2024-04-16T16:56:09.738923+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:31.325294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248980+0000","last_change":"2024-04-16T17:01:29.023341+0000","last_active":"2024-04-16T17:04:44.248980+0000","last_peered":"2024-04-16T17:04:44.248980+0000","last_clean":"2024-04-16T17:04:44.248980+0000","last_became_active":"2024-04-16T17:01:29.023037+0000","last_became_peered":"2024-04-16T17:01:29.023037+0000","last_unstale":"2024-04-16T17:04:44.248980+0000","last_undegraded":"2024-04-16T17:04:44.248980+0000","last_fullsized":"2024-04-16T17:04:44.248980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:01.483395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414448+0000","last_change":"2024-04-16T17:01:24.547175+0000","last_active":"2024-04-16T17:04:44.414448+0000","last_peered":"2024-04-16T17:04:44.414448+0000","last_clean":"2024-04-16T17:04:44.414448+0000","last_became_active":"2024-04-16T17:01:24.546806+0000","last_became_peered":"2024-04-16T17:01:24.546806+0000","last_unstale":"2024-04-16T17:04:44.414448+0000","last_undegraded":"2024-04-16T17:04:44.414448+0000","last_fullsized":"2024-04-16T17:04:44.414448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:52:44.598231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103918+0000","last_change":"2024-04-16T17:02:26.312852+0000","last_active":"2024-04-16T17:04:44.103918+0000","last_peered":"2024-04-16T17:04:44.103918+0000","last_clean":"2024-04-16T17:04:44.103918+0000","last_became_active":"2024-04-16T17:02:26.308279+0000","last_became_peered":"2024-04-16T17:02:26.308279+0000","last_unstale":"2024-04-16T17:04:44.103918+0000","last_undegraded":"2024-04-16T17:04:44.103918+0000","last_fullsized":"2024-04-16T17:04:44.103918+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:25.511349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.103933+0000","last_change":"2024-04-16T17:01:29.010310+0000","last_active":"2024-04-16T17:04:44.103933+0000","last_peered":"2024-04-16T17:04:44.103933+0000","last_clean":"2024-04-16T17:04:44.103933+0000","last_became_active":"2024-04-16T17:01:29.009873+0000","last_became_peered":"2024-04-16T17:01:29.009873+0000","last_unstale":"2024-04-16T17:04:44.103933+0000","last_undegraded":"2024-04-16T17:04:44.103933+0000","last_fullsized":"2024-04-16T17:04:44.103933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:10.399140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240849+0000","last_change":"2024-04-16T17:01:23.936502+0000","last_active":"2024-04-16T17:04:44.240849+0000","last_peered":"2024-04-16T17:04:44.240849+0000","last_clean":"2024-04-16T17:04:44.240849+0000","last_became_active":"2024-04-16T17:01:23.936158+0000","last_became_peered":"2024-04-16T17:01:23.936158+0000","last_unstale":"2024-04-16T17:04:44.240849+0000","last_undegraded":"2024-04-16T17:04:44.240849+0000","last_fullsized":"2024-04-16T17:04:44.240849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:28:10.767039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415023+0000","last_change":"2024-04-16T17:01:24.546386+0000","last_active":"2024-04-16T17:04:44.415023+0000","last_peered":"2024-04-16T17:04:44.415023+0000","last_clean":"2024-04-16T17:04:44.415023+0000","last_became_active":"2024-04-16T17:01:24.546023+0000","last_became_peered":"2024-04-16T17:01:24.546023+0000","last_unstale":"2024-04-16T17:04:44.415023+0000","last_undegraded":"2024-04-16T17:04:44.415023+0000","last_fullsized":"2024-04-16T17:04:44.415023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:07.659099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240499+0000","last_change":"2024-04-16T17:01:29.027317+0000","last_active":"2024-04-16T17:04:44.240499+0000","last_peered":"2024-04-16T17:04:44.240499+0000","last_clean":"2024-04-16T17:04:44.240499+0000","last_became_active":"2024-04-16T17:01:29.026793+0000","last_became_peered":"2024-04-16T17:01:29.026793+0000","last_unstale":"2024-04-16T17:04:44.240499+0000","last_undegraded":"2024-04-16T17:04:44.240499+0000","last_fullsized":"2024-04-16T17:04:44.240499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:54.280370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'47","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.374558+0000","last_change":"2024-04-16T17:01:23.935319+0000","last_active":"2024-04-16T17:04:44.374558+0000","last_peered":"2024-04-16T17:04:44.374558+0000","last_clean":"2024-04-16T17:04:44.374558+0000","last_became_active":"2024-04-16T17:01:23.934895+0000","last_became_peered":"2024-04-16T17:01:23.934895+0000","last_unstale":"2024-04-16T17:04:44.374558+0000","last_undegraded":"2024-04-16T17:04:44.374558+0000","last_fullsized":"2024-04-16T17:04:44.374558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:11:11.622614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241537+0000","last_change":"2024-04-16T17:01:24.560583+0000","last_active":"2024-04-16T17:04:44.241537+0000","last_peered":"2024-04-16T17:04:44.241537+0000","last_clean":"2024-04-16T17:04:44.241537+0000","last_became_active":"2024-04-16T17:01:24.558442+0000","last_became_peered":"2024-04-16T17:01:24.558442+0000","last_unstale":"2024-04-16T17:04:44.241537+0000","last_undegraded":"2024-04-16T17:04:44.241537+0000","last_fullsized":"2024-04-16T17:04:44.241537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:41.424291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274800+0000","last_change":"2024-04-16T17:01:29.008738+0000","last_active":"2024-04-16T17:04:44.274800+0000","last_peered":"2024-04-16T17:04:44.274800+0000","last_clean":"2024-04-16T17:04:44.274800+0000","last_became_active":"2024-04-16T17:01:29.008307+0000","last_became_peered":"2024-04-16T17:01:29.008307+0000","last_unstale":"2024-04-16T17:04:44.274800+0000","last_undegraded":"2024-04-16T17:04:44.274800+0000","last_fullsized":"2024-04-16T17:04:44.274800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:11.071760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414738+0000","last_change":"2024-04-16T17:01:23.921463+0000","last_active":"2024-04-16T17:04:44.414738+0000","last_peered":"2024-04-16T17:04:44.414738+0000","last_clean":"2024-04-16T17:04:44.414738+0000","last_became_active":"2024-04-16T17:01:23.921083+0000","last_became_peered":"2024-04-16T17:01:23.921083+0000","last_unstale":"2024-04-16T17:04:44.414738+0000","last_undegraded":"2024-04-16T17:04:44.414738+0000","last_fullsized":"2024-04-16T17:04:44.414738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:39:53.023297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419383+0000","last_change":"2024-04-16T17:01:24.567204+0000","last_active":"2024-04-16T17:04:44.419383+0000","last_peered":"2024-04-16T17:04:44.419383+0000","last_clean":"2024-04-16T17:04:44.419383+0000","last_became_active":"2024-04-16T17:01:24.566828+0000","last_became_peered":"2024-04-16T17:01:24.566828+0000","last_unstale":"2024-04-16T17:04:44.419383+0000","last_undegraded":"2024-04-16T17:04:44.419383+0000","last_fullsized":"2024-04-16T17:04:44.419383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:54:49.823805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414608+0000","last_change":"2024-04-16T17:01:29.031391+0000","last_active":"2024-04-16T17:04:44.414608+0000","last_peered":"2024-04-16T17:04:44.414608+0000","last_clean":"2024-04-16T17:04:44.414608+0000","last_became_active":"2024-04-16T17:01:29.030957+0000","last_became_peered":"2024-04-16T17:01:29.030957+0000","last_unstale":"2024-04-16T17:04:44.414608+0000","last_undegraded":"2024-04-16T17:04:44.414608+0000","last_fullsized":"2024-04-16T17:04:44.414608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:44.526267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207210+0000","last_change":"2024-04-16T17:01:23.919429+0000","last_active":"2024-04-16T17:04:44.207210+0000","last_peered":"2024-04-16T17:04:44.207210+0000","last_clean":"2024-04-16T17:04:44.207210+0000","last_became_active":"2024-04-16T17:01:23.919063+0000","last_became_peered":"2024-04-16T17:01:23.919063+0000","last_unstale":"2024-04-16T17:04:44.207210+0000","last_undegraded":"2024-04-16T17:04:44.207210+0000","last_fullsized":"2024-04-16T17:04:44.207210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:07:47.445117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240332+0000","last_change":"2024-04-16T17:01:24.529836+0000","last_active":"2024-04-16T17:04:44.240332+0000","last_peered":"2024-04-16T17:04:44.240332+0000","last_clean":"2024-04-16T17:04:44.240332+0000","last_became_active":"2024-04-16T17:01:24.529502+0000","last_became_peered":"2024-04-16T17:01:24.529502+0000","last_unstale":"2024-04-16T17:04:44.240332+0000","last_undegraded":"2024-04-16T17:04:44.240332+0000","last_fullsized":"2024-04-16T17:04:44.240332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:47:13.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241095+0000","last_change":"2024-04-16T17:03:27.268760+0000","last_active":"2024-04-16T17:04:44.241095+0000","last_peered":"2024-04-16T17:04:44.241095+0000","last_clean":"2024-04-16T17:04:44.241095+0000","last_became_active":"2024-04-16T17:03:27.268209+0000","last_became_peered":"2024-04-16T17:03:27.268209+0000","last_unstale":"2024-04-16T17:04:44.241095+0000","last_undegraded":"2024-04-16T17:04:44.241095+0000","last_fullsized":"2024-04-16T17:04:44.241095+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:24:57.457026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240565+0000","last_change":"2024-04-16T17:01:23.930672+0000","last_active":"2024-04-16T17:04:44.240565+0000","last_peered":"2024-04-16T17:04:44.240565+0000","last_clean":"2024-04-16T17:04:44.240565+0000","last_became_active":"2024-04-16T17:01:23.927399+0000","last_became_peered":"2024-04-16T17:01:23.927399+0000","last_unstale":"2024-04-16T17:04:44.240565+0000","last_undegraded":"2024-04-16T17:04:44.240565+0000","last_fullsized":"2024-04-16T17:04:44.240565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:07.993159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104268+0000","last_change":"2024-04-16T17:01:24.571502+0000","last_active":"2024-04-16T17:04:44.104268+0000","last_peered":"2024-04-16T17:04:44.104268+0000","last_clean":"2024-04-16T17:04:44.104268+0000","last_became_active":"2024-04-16T17:01:24.571082+0000","last_became_peered":"2024-04-16T17:01:24.571082+0000","last_unstale":"2024-04-16T17:04:44.104268+0000","last_undegraded":"2024-04-16T17:04:44.104268+0000","last_fullsized":"2024-04-16T17:04:44.104268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:43.577517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414352+0000","last_change":"2024-04-16T17:01:29.027800+0000","last_active":"2024-04-16T17:04:44.414352+0000","last_peered":"2024-04-16T17:04:44.414352+0000","last_clean":"2024-04-16T17:04:44.414352+0000","last_became_active":"2024-04-16T17:01:29.027178+0000","last_became_peered":"2024-04-16T17:01:29.027178+0000","last_unstale":"2024-04-16T17:04:44.414352+0000","last_undegraded":"2024-04-16T17:04:44.414352+0000","last_fullsized":"2024-04-16T17:04:44.414352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:01:58.373491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274993+0000","last_change":"2024-04-16T17:01:23.933713+0000","last_active":"2024-04-16T17:04:44.274993+0000","last_peered":"2024-04-16T17:04:44.274993+0000","last_clean":"2024-04-16T17:04:44.274993+0000","last_became_active":"2024-04-16T17:01:23.933349+0000","last_became_peered":"2024-04-16T17:01:23.933349+0000","last_unstale":"2024-04-16T17:04:44.274993+0000","last_undegraded":"2024-04-16T17:04:44.274993+0000","last_fullsized":"2024-04-16T17:04:44.274993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:03.071215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241448+0000","last_change":"2024-04-16T17:01:24.570504+0000","last_active":"2024-04-16T17:04:44.241448+0000","last_peered":"2024-04-16T17:04:44.241448+0000","last_clean":"2024-04-16T17:04:44.241448+0000","last_became_active":"2024-04-16T17:01:24.569435+0000","last_became_peered":"2024-04-16T17:01:24.569435+0000","last_unstale":"2024-04-16T17:04:44.241448+0000","last_undegraded":"2024-04-16T17:04:44.241448+0000","last_fullsized":"2024-04-16T17:04:44.241448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:28.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414414+0000","last_change":"2024-04-16T17:01:29.030184+0000","last_active":"2024-04-16T17:04:44.414414+0000","last_peered":"2024-04-16T17:04:44.414414+0000","last_clean":"2024-04-16T17:04:44.414414+0000","last_became_active":"2024-04-16T17:01:29.029562+0000","last_became_peered":"2024-04-16T17:01:29.029562+0000","last_unstale":"2024-04-16T17:04:44.414414+0000","last_undegraded":"2024-04-16T17:04:44.414414+0000","last_fullsized":"2024-04-16T17:04:44.414414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:45.822893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240470+0000","last_change":"2024-04-16T17:01:23.915513+0000","last_active":"2024-04-16T17:04:44.240470+0000","last_peered":"2024-04-16T17:04:44.240470+0000","last_clean":"2024-04-16T17:04:44.240470+0000","last_became_active":"2024-04-16T17:01:23.915189+0000","last_became_peered":"2024-04-16T17:01:23.915189+0000","last_unstale":"2024-04-16T17:04:44.240470+0000","last_undegraded":"2024-04-16T17:04:44.240470+0000","last_fullsized":"2024-04-16T17:04:44.240470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:28.604617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114478+0000","last_change":"2024-04-16T17:01:24.539078+0000","last_active":"2024-04-16T17:04:44.114478+0000","last_peered":"2024-04-16T17:04:44.114478+0000","last_clean":"2024-04-16T17:04:44.114478+0000","last_became_active":"2024-04-16T17:01:24.538777+0000","last_became_peered":"2024-04-16T17:01:24.538777+0000","last_unstale":"2024-04-16T17:04:44.114478+0000","last_undegraded":"2024-04-16T17:04:44.114478+0000","last_fullsized":"2024-04-16T17:04:44.114478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:49:32.975934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207182+0000","last_change":"2024-04-16T17:01:29.020938+0000","last_active":"2024-04-16T17:04:44.207182+0000","last_peered":"2024-04-16T17:04:44.207182+0000","last_clean":"2024-04-16T17:04:44.207182+0000","last_became_active":"2024-04-16T17:01:29.020423+0000","last_became_peered":"2024-04-16T17:01:29.020423+0000","last_unstale":"2024-04-16T17:04:44.207182+0000","last_undegraded":"2024-04-16T17:04:44.207182+0000","last_fullsized":"2024-04-16T17:04:44.207182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:58:53.587324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207367+0000","last_change":"2024-04-16T17:02:26.328059+0000","last_active":"2024-04-16T17:04:44.207367+0000","last_peered":"2024-04-16T17:04:44.207367+0000","last_clean":"2024-04-16T17:04:44.207367+0000","last_became_active":"2024-04-16T17:02:26.323315+0000","last_became_peered":"2024-04-16T17:02:26.323315+0000","last_unstale":"2024-04-16T17:04:44.207367+0000","last_undegraded":"2024-04-16T17:04:44.207367+0000","last_fullsized":"2024-04-16T17:04:44.207367+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:01.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114711+0000","last_change":"2024-04-16T17:01:24.532078+0000","last_active":"2024-04-16T17:04:44.114711+0000","last_peered":"2024-04-16T17:04:44.114711+0000","last_clean":"2024-04-16T17:04:44.114711+0000","last_became_active":"2024-04-16T17:01:24.531724+0000","last_became_peered":"2024-04-16T17:01:24.531724+0000","last_unstale":"2024-04-16T17:04:44.114711+0000","last_undegraded":"2024-04-16T17:04:44.114711+0000","last_fullsized":"2024-04-16T17:04:44.114711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:37.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.418863+0000","last_change":"2024-04-16T17:01:29.017027+0000","last_active":"2024-04-16T17:04:44.418863+0000","last_peered":"2024-04-16T17:04:44.418863+0000","last_clean":"2024-04-16T17:04:44.418863+0000","last_became_active":"2024-04-16T17:01:29.016315+0000","last_became_peered":"2024-04-16T17:01:29.016315+0000","last_unstale":"2024-04-16T17:04:44.418863+0000","last_undegraded":"2024-04-16T17:04:44.418863+0000","last_fullsized":"2024-04-16T17:04:44.418863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:12.926558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'39","reported_seq":81,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.368875+0000","last_change":"2024-04-16T17:01:23.933667+0000","last_active":"2024-04-16T17:04:44.368875+0000","last_peered":"2024-04-16T17:04:44.368875+0000","last_clean":"2024-04-16T17:04:44.368875+0000","last_became_active":"2024-04-16T17:01:23.933304+0000","last_became_peered":"2024-04-16T17:01:23.933304+0000","last_unstale":"2024-04-16T17:04:44.368875+0000","last_undegraded":"2024-04-16T17:04:44.368875+0000","last_fullsized":"2024-04-16T17:04:44.368875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:30.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114226+0000","last_change":"2024-04-16T17:01:24.549482+0000","last_active":"2024-04-16T17:04:44.114226+0000","last_peered":"2024-04-16T17:04:44.114226+0000","last_clean":"2024-04-16T17:04:44.114226+0000","last_became_active":"2024-04-16T17:01:24.549099+0000","last_became_peered":"2024-04-16T17:01:24.549099+0000","last_unstale":"2024-04-16T17:04:44.114226+0000","last_undegraded":"2024-04-16T17:04:44.114226+0000","last_fullsized":"2024-04-16T17:04:44.114226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:34.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248179+0000","last_change":"2024-04-16T17:01:29.009847+0000","last_active":"2024-04-16T17:04:44.248179+0000","last_peered":"2024-04-16T17:04:44.248179+0000","last_clean":"2024-04-16T17:04:44.248179+0000","last_became_active":"2024-04-16T17:01:29.009439+0000","last_became_peered":"2024-04-16T17:01:29.009439+0000","last_unstale":"2024-04-16T17:04:44.248179+0000","last_undegraded":"2024-04-16T17:04:44.248179+0000","last_fullsized":"2024-04-16T17:04:44.248179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:27.966977+0000","last_clean_scrub_stamp":"2024-04-16T17:01:27.966977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:10.632505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.414927+0000","last_change":"2024-04-16T17:01:23.938326+0000","last_active":"2024-04-16T17:04:44.414927+0000","last_peered":"2024-04-16T17:04:44.414927+0000","last_clean":"2024-04-16T17:04:44.414927+0000","last_became_active":"2024-04-16T17:01:23.937850+0000","last_became_peered":"2024-04-16T17:01:23.937850+0000","last_unstale":"2024-04-16T17:04:44.414927+0000","last_undegraded":"2024-04-16T17:04:44.414927+0000","last_fullsized":"2024-04-16T17:04:44.414927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:22.883910+0000","last_clean_scrub_stamp":"2024-04-16T17:01:22.883910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:58:54.754384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.114728+0000","last_change":"2024-04-16T17:01:24.536259+0000","last_active":"2024-04-16T17:04:44.114728+0000","last_peered":"2024-04-16T17:04:44.114728+0000","last_clean":"2024-04-16T17:04:44.114728+0000","last_became_active":"2024-04-16T17:01:24.535813+0000","last_became_peered":"2024-04-16T17:01:24.535813+0000","last_unstale":"2024-04-16T17:04:44.114728+0000","last_undegraded":"2024-04-16T17:04:44.114728+0000","last_fullsized":"2024-04-16T17:04:44.114728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:23.810841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104236+0000","last_change":"2024-04-16T17:01:24.535343+0000","last_active":"2024-04-16T17:04:44.104236+0000","last_peered":"2024-04-16T17:04:44.104236+0000","last_clean":"2024-04-16T17:04:44.104236+0000","last_became_active":"2024-04-16T17:01:24.535019+0000","last_became_peered":"2024-04-16T17:01:24.535019+0000","last_unstale":"2024-04-16T17:04:44.104236+0000","last_undegraded":"2024-04-16T17:04:44.104236+0000","last_fullsized":"2024-04-16T17:04:44.104236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:04:21.705120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275188+0000","last_change":"2024-04-16T17:01:24.538169+0000","last_active":"2024-04-16T17:04:44.275188+0000","last_peered":"2024-04-16T17:04:44.275188+0000","last_clean":"2024-04-16T17:04:44.275188+0000","last_became_active":"2024-04-16T17:01:24.537816+0000","last_became_peered":"2024-04-16T17:01:24.537816+0000","last_unstale":"2024-04-16T17:04:44.275188+0000","last_undegraded":"2024-04-16T17:04:44.275188+0000","last_fullsized":"2024-04-16T17:04:44.275188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:17:59.033137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.274905+0000","last_change":"2024-04-16T17:01:24.560467+0000","last_active":"2024-04-16T17:04:44.274905+0000","last_peered":"2024-04-16T17:04:44.274905+0000","last_clean":"2024-04-16T17:04:44.274905+0000","last_became_active":"2024-04-16T17:01:24.560075+0000","last_became_peered":"2024-04-16T17:01:24.560075+0000","last_unstale":"2024-04-16T17:04:44.274905+0000","last_undegraded":"2024-04-16T17:04:44.274905+0000","last_fullsized":"2024-04-16T17:04:44.274905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:19.576870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.419653+0000","last_change":"2024-04-16T17:01:24.564735+0000","last_active":"2024-04-16T17:04:44.419653+0000","last_peered":"2024-04-16T17:04:44.419653+0000","last_clean":"2024-04-16T17:04:44.419653+0000","last_became_active":"2024-04-16T17:01:24.564364+0000","last_became_peered":"2024-04-16T17:01:24.564364+0000","last_unstale":"2024-04-16T17:04:44.419653+0000","last_undegraded":"2024-04-16T17:04:44.419653+0000","last_fullsized":"2024-04-16T17:04:44.419653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:48.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415072+0000","last_change":"2024-04-16T17:01:24.570932+0000","last_active":"2024-04-16T17:04:44.415072+0000","last_peered":"2024-04-16T17:04:44.415072+0000","last_clean":"2024-04-16T17:04:44.415072+0000","last_became_active":"2024-04-16T17:01:24.569760+0000","last_became_peered":"2024-04-16T17:01:24.569760+0000","last_unstale":"2024-04-16T17:04:44.415072+0000","last_undegraded":"2024-04-16T17:04:44.415072+0000","last_fullsized":"2024-04-16T17:04:44.415072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:28.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104917+0000","last_change":"2024-04-16T17:01:24.535286+0000","last_active":"2024-04-16T17:04:44.104917+0000","last_peered":"2024-04-16T17:04:44.104917+0000","last_clean":"2024-04-16T17:04:44.104917+0000","last_became_active":"2024-04-16T17:01:24.534949+0000","last_became_peered":"2024-04-16T17:01:24.534949+0000","last_unstale":"2024-04-16T17:04:44.104917+0000","last_undegraded":"2024-04-16T17:04:44.104917+0000","last_fullsized":"2024-04-16T17:04:44.104917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:34.439757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.275489+0000","last_change":"2024-04-16T17:01:24.534477+0000","last_active":"2024-04-16T17:04:44.275489+0000","last_peered":"2024-04-16T17:04:44.275489+0000","last_clean":"2024-04-16T17:04:44.275489+0000","last_became_active":"2024-04-16T17:01:24.534139+0000","last_became_peered":"2024-04-16T17:01:24.534139+0000","last_unstale":"2024-04-16T17:04:44.275489+0000","last_undegraded":"2024-04-16T17:04:44.275489+0000","last_fullsized":"2024-04-16T17:04:44.275489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:11.268802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.241877+0000","last_change":"2024-04-16T17:01:24.570658+0000","last_active":"2024-04-16T17:04:44.241877+0000","last_peered":"2024-04-16T17:04:44.241877+0000","last_clean":"2024-04-16T17:04:44.241877+0000","last_became_active":"2024-04-16T17:01:24.568550+0000","last_became_peered":"2024-04-16T17:01:24.568550+0000","last_unstale":"2024-04-16T17:04:44.241877+0000","last_undegraded":"2024-04-16T17:04:44.241877+0000","last_fullsized":"2024-04-16T17:04:44.241877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:27:49.724550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.248434+0000","last_change":"2024-04-16T17:01:24.558753+0000","last_active":"2024-04-16T17:04:44.248434+0000","last_peered":"2024-04-16T17:04:44.248434+0000","last_clean":"2024-04-16T17:04:44.248434+0000","last_became_active":"2024-04-16T17:01:24.558401+0000","last_became_peered":"2024-04-16T17:01:24.558401+0000","last_unstale":"2024-04-16T17:04:44.248434+0000","last_undegraded":"2024-04-16T17:04:44.248434+0000","last_fullsized":"2024-04-16T17:04:44.248434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:44.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.240652+0000","last_change":"2024-04-16T17:01:24.557147+0000","last_active":"2024-04-16T17:04:44.240652+0000","last_peered":"2024-04-16T17:04:44.240652+0000","last_clean":"2024-04-16T17:04:44.240652+0000","last_became_active":"2024-04-16T17:01:24.556772+0000","last_became_peered":"2024-04-16T17:01:24.556772+0000","last_unstale":"2024-04-16T17:04:44.240652+0000","last_undegraded":"2024-04-16T17:04:44.240652+0000","last_fullsized":"2024-04-16T17:04:44.240652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:52.748303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.207400+0000","last_change":"2024-04-16T17:01:24.549908+0000","last_active":"2024-04-16T17:04:44.207400+0000","last_peered":"2024-04-16T17:04:44.207400+0000","last_clean":"2024-04-16T17:04:44.207400+0000","last_became_active":"2024-04-16T17:01:24.549550+0000","last_became_peered":"2024-04-16T17:01:24.549550+0000","last_unstale":"2024-04-16T17:04:44.207400+0000","last_undegraded":"2024-04-16T17:04:44.207400+0000","last_fullsized":"2024-04-16T17:04:44.207400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:12:22.799668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.415063+0000","last_change":"2024-04-16T17:01:24.573174+0000","last_active":"2024-04-16T17:04:44.415063+0000","last_peered":"2024-04-16T17:04:44.415063+0000","last_clean":"2024-04-16T17:04:44.415063+0000","last_became_active":"2024-04-16T17:01:24.572673+0000","last_became_peered":"2024-04-16T17:01:24.572673+0000","last_unstale":"2024-04-16T17:04:44.415063+0000","last_undegraded":"2024-04-16T17:04:44.415063+0000","last_fullsized":"2024-04-16T17:04:44.415063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:39.665063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:04:44.104214+0000","last_change":"2024-04-16T17:01:24.538877+0000","last_active":"2024-04-16T17:04:44.104214+0000","last_peered":"2024-04-16T17:04:44.104214+0000","last_clean":"2024-04-16T17:04:44.104214+0000","last_became_active":"2024-04-16T17:01:24.538544+0000","last_became_peered":"2024-04-16T17:01:24.538544+0000","last_unstale":"2024-04-16T17:04:44.104214+0000","last_undegraded":"2024-04-16T17:04:44.104214+0000","last_fullsized":"2024-04-16T17:04:44.104214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:01:23.892425+0000","last_clean_scrub_stamp":"2024-04-16T17:01:23.892425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:12:57.380924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106545,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600829,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1113,"num_read_kb":673,"num_write":292,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1805398,"data_compressed":81384,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383732,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":529733,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36512,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708640,"statfs":{"total":95995035648,"available":95957647360,"internally_reserved":0,"allocated":1204224,"data_stored":1320281,"data_compressed":73169,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808897,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":927259,"data_compressed":48818,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070534,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":923061,"data_compressed":48818,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332171,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1125695,"data_compressed":65187,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495696,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":525123,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":599446,"data_compressed":31498,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":526511,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":596099,"data_compressed":31498,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182497,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1050133,"data_compressed":54974,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":531878,"data_compressed":29672,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757336,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":1252883,"data_compressed":71343,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35515,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":74413,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71066,"data_compressed":1826,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133777,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":204880,"data_compressed":7982,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:04:51.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:04:51.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: Updating smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: Updating smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:51.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:51 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 -- 172.21.15.18:0/3467272145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 msgr2=0x7fe11c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 --2- 172.21.15.18:0/3467272145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 0x7fe11c104810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009900 tx=0x7fe10c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:04:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 -- 172.21.15.18:0/3467272145 shutdown_connections 2024-04-16T17:04:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 --2- 172.21.15.18:0/3467272145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11c109500 0x7fe11c1110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 --2- 172.21.15.18:0/3467272145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe11c104de0 0x7fe11c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 --2- 172.21.15.18:0/3467272145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 0x7fe11c104810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 -- 172.21.15.18:0/3467272145 >> 172.21.15.18:0/3467272145 conn(0x7fe11c0ffcb0 msgr2=0x7fe11c1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 -- 172.21.15.18:0/3467272145 shutdown_connections 2024-04-16T17:04:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.047+0000 7fe120c00640 1 -- 172.21.15.18:0/3467272145 wait complete. 2024-04-16T17:04:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.048+0000 7fe120c00640 1 Processor -- start 2024-04-16T17:04:52.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.048+0000 7fe120c00640 1 -- start start 2024-04-16T17:04:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe120c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 0x7fe11c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe120c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11c109500 0x7fe11c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe120c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe11c19a990 0x7fe11c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe120c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c114610 con 0x7fe11c19a990 2024-04-16T17:04:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe120c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c114490 con 0x7fe11c104430 2024-04-16T17:04:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe120c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c114790 con 0x7fe11c109500 2024-04-16T17:04:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe119e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 0x7fe11c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe119e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 0x7fe11c1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54496/0 (socket says 172.21.15.18:54496) 2024-04-16T17:04:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe119e00640 1 -- 172.21.15.18:0/1561128354 learned_addr learned my addr 172.21.15.18:0/1561128354 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe11a800640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe11c19a990 0x7fe11c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.049+0000 7fe119400640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11c109500 0x7fe11c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe119e00640 1 -- 172.21.15.18:0/1561128354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11c109500 msgr2=0x7fe11c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe119e00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11c109500 0x7fe11c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe119e00640 1 -- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe11c19a990 msgr2=0x7fe11c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe119e00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe11c19a990 0x7fe11c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe119e00640 1 -- 172.21.15.18:0/1561128354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c19b4b0 con 0x7fe11c104430 2024-04-16T17:04:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe119e00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 0x7fe11c1a0790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005ab0 tx=0x7fe10c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe112a00640 1 -- 172.21.15.18:0/1561128354 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c041070 con 0x7fe11c104430 2024-04-16T17:04:52.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe112a00640 1 -- 172.21.15.18:0/1561128354 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c02cca0 con 0x7fe11c104430 2024-04-16T17:04:52.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe112a00640 1 -- 172.21.15.18:0/1561128354 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c003d90 con 0x7fe11c104430 2024-04-16T17:04:52.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.050+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c19b740 con 0x7fe11c104430 2024-04-16T17:04:52.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.051+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c1a7940 con 0x7fe11c104430 2024-04-16T17:04:52.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.052+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe11c069300 con 0x7fe11c104430 2024-04-16T17:04:52.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.054+0000 7fe112a00640 1 -- 172.21.15.18:0/1561128354 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe10c044070 con 0x7fe11c104430 2024-04-16T17:04:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.055+0000 7fe112a00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1000a3ec0 0x7fe1000a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.055+0000 7fe112a00640 1 -- 172.21.15.18:0/1561128354 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe10c0f2650 con 0x7fe11c104430 2024-04-16T17:04:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.055+0000 7fe119400640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1000a3ec0 0x7fe1000a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.056+0000 7fe119400640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1000a3ec0 0x7fe1000a6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe11c1000d0 tx=0x7fe108005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:52.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.059+0000 7fe112a00640 1 -- 172.21.15.18:0/1561128354 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe10c0be150 con 0x7fe11c104430 2024-04-16T17:04:52.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.274+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe11c19be70 con 0x7fe11c104430 2024-04-16T17:04:52.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.276+0000 7fe112a00640 1 -- 172.21.15.18:0/1561128354 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25126 (secure 0 0 0) 0x7fe10c0c2450 con 0x7fe11c104430 2024-04-16T17:04:52.278 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:52.278 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":94,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:04:44.091776+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3045212308":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/3152065578":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/1033276779":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6801/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/2149549301":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:52.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1000a3ec0 msgr2=0x7fe1000a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1000a3ec0 0x7fe1000a6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe11c1000d0 tx=0x7fe108005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:52.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 msgr2=0x7fe11c1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 0x7fe11c1a0790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005ab0 tx=0x7fe10c0042f0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 shutdown_connections 2024-04-16T17:04:52.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1000a3ec0 0x7fe1000a6380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe11c19a990 0x7fe11c19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11c109500 0x7fe11c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 --2- 172.21.15.18:0/1561128354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe11c104430 0x7fe11c1a0790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 >> 172.21.15.18:0/1561128354 conn(0x7fe11c0ffcb0 msgr2=0x7fe11c102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:52.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 shutdown_connections 2024-04-16T17:04:52.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.279+0000 7fe120c00640 1 -- 172.21.15.18:0/1561128354 wait complete. 2024-04-16T17:04:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:04:52.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:04:52.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.499+0000 7f3460c00640 1 -- 172.21.15.18:0/2077057017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c10c3b0 msgr2=0x7f345c10e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.499+0000 7f3460c00640 1 --2- 172.21.15.18:0/2077057017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c10c3b0 0x7f345c10e850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3454009ca0 tx=0x7f345402c180 comp rx=0 tx=0).stop 2024-04-16T17:04:52.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.500+0000 7f3460c00640 1 -- 172.21.15.18:0/2077057017 shutdown_connections 2024-04-16T17:04:52.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.500+0000 7f3460c00640 1 --2- 172.21.15.18:0/2077057017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c10c3b0 0x7f345c10e850 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.500+0000 7f3460c00640 1 --2- 172.21.15.18:0/2077057017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f345c103900 0x7f345c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.500+0000 7f3460c00640 1 --2- 172.21.15.18:0/2077057017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f345c102fe0 0x7f345c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.500+0000 7f3460c00640 1 -- 172.21.15.18:0/2077057017 >> 172.21.15.18:0/2077057017 conn(0x7f345c077ab0 msgr2=0x7f345c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.500+0000 7f3460c00640 1 -- 172.21.15.18:0/2077057017 shutdown_connections 2024-04-16T17:04:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.500+0000 7f3460c00640 1 -- 172.21.15.18:0/2077057017 wait complete. 2024-04-16T17:04:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.501+0000 7f3460c00640 1 Processor -- start 2024-04-16T17:04:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.501+0000 7f3460c00640 1 -- start start 2024-04-16T17:04:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f3460c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c102fe0 0x7f345c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f3460c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f345c103900 0x7f345c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f3460c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f345c10c3b0 0x7f345c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f3460c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f345c112190 con 0x7f345c102fe0 2024-04-16T17:04:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f3460c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f345c112010 con 0x7f345c103900 2024-04-16T17:04:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f3460c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f345c112310 con 0x7f345c10c3b0 2024-04-16T17:04:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f345b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c102fe0 0x7f345c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f345b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c102fe0 0x7f345c1a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55622/0 (socket says 172.21.15.18:55622) 2024-04-16T17:04:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f345b400640 1 -- 172.21.15.18:0/2081480951 learned_addr learned my addr 172.21.15.18:0/2081480951 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f345be00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f345c10c3b0 0x7f345c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f3452a00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f345c103900 0x7f345c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f345b400640 1 -- 172.21.15.18:0/2081480951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f345c10c3b0 msgr2=0x7f345c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f345b400640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f345c10c3b0 0x7f345c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.502+0000 7f345b400640 1 -- 172.21.15.18:0/2081480951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f345c103900 msgr2=0x7f345c1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f345b400640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f345c103900 0x7f345c1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f345b400640 1 -- 172.21.15.18:0/2081480951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c19af50 con 0x7f345c102fe0 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f345be00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f345c10c3b0 0x7f345c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f345b400640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c102fe0 0x7f345c1a05d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e9f0 tx=0x7f344c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f3458c00640 1 -- 172.21.15.18:0/2081480951 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c00cd80 con 0x7f345c102fe0 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c19b240 con 0x7f345c102fe0 2024-04-16T17:04:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f3458c00640 1 -- 172.21.15.18:0/2081480951 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c005500 con 0x7f345c102fe0 2024-04-16T17:04:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f3458c00640 1 -- 172.21.15.18:0/2081480951 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c018630 con 0x7f345c102fe0 2024-04-16T17:04:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.503+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c1a7550 con 0x7f345c102fe0 2024-04-16T17:04:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.504+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345c069ba0 con 0x7f345c102fe0 2024-04-16T17:04:52.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.506+0000 7f3458c00640 1 -- 172.21.15.18:0/2081480951 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f344c0046c0 con 0x7f345c102fe0 2024-04-16T17:04:52.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.508+0000 7f3458c00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f343c0a3e30 0x7f343c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.508+0000 7f3458c00640 1 -- 172.21.15.18:0/2081480951 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f344c0d13c0 con 0x7f345c102fe0 2024-04-16T17:04:52.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.508+0000 7f3452a00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f343c0a3e30 0x7f343c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.509+0000 7f3452a00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f343c0a3e30 0x7f343c0a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f345c19bee0 tx=0x7f3444005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.512+0000 7f3458c00640 1 -- 172.21.15.18:0/2081480951 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f344c09d400 con 0x7f345c102fe0 2024-04-16T17:04:52.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:52.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:52.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='client.25570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:52 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1561128354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.727+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f345c1a7720 con 0x7f345c102fe0 2024-04-16T17:04:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.728+0000 7f3458c00640 1 -- 172.21.15.18:0/2081480951 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f344c09cd00 con 0x7f345c102fe0 2024-04-16T17:04:52.730 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f343c0a3e30 msgr2=0x7f343c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f343c0a3e30 0x7f343c0a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f345c19bee0 tx=0x7f3444005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c102fe0 msgr2=0x7f345c1a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c102fe0 0x7f345c1a05d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e9f0 tx=0x7f344c00edf0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 shutdown_connections 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f343c0a3e30 0x7f343c0a62f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f345c10c3b0 0x7f345c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f345c103900 0x7f345c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 --2- 172.21.15.18:0/2081480951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f345c102fe0 0x7f345c1a05d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 >> 172.21.15.18:0/2081480951 conn(0x7f345c077ab0 msgr2=0x7f345c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.731+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 shutdown_connections 2024-04-16T17:04:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.732+0000 7f3460c00640 1 -- 172.21.15.18:0/2081480951 wait complete. 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='client.25570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:52.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:52.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:52 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1561128354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:52.754 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:04:52.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:04:52.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='client.25570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: Updating smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:04:52.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:52 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1561128354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:52.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 -- 172.21.15.18:0/2483986645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 msgr2=0x7fc54c103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 --2- 172.21.15.18:0/2483986645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 0x7fc54c103650 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc540009900 tx=0x7fc54002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 -- 172.21.15.18:0/2483986645 shutdown_connections 2024-04-16T17:04:52.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 --2- 172.21.15.18:0/2483986645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc54c10c5b0 0x7fc54c10ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 --2- 172.21.15.18:0/2483986645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc54c103b90 0x7fc54c10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 --2- 172.21.15.18:0/2483986645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 0x7fc54c103650 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 -- 172.21.15.18:0/2483986645 >> 172.21.15.18:0/2483986645 conn(0x7fc54c0780b0 msgr2=0x7fc54c0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:52.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 -- 172.21.15.18:0/2483986645 shutdown_connections 2024-04-16T17:04:52.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.970+0000 7fc554400640 1 -- 172.21.15.18:0/2483986645 wait complete. 2024-04-16T17:04:52.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.971+0000 7fc554400640 1 Processor -- start 2024-04-16T17:04:52.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.971+0000 7fc554400640 1 -- start start 2024-04-16T17:04:52.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.971+0000 7fc554400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 0x7fc54c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.971+0000 7fc554400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc54c103b90 0x7fc54c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.971+0000 7fc554400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc54c10c5b0 0x7fc54c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.971+0000 7fc554400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c1123c0 con 0x7fc54c103270 2024-04-16T17:04:52.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.971+0000 7fc554400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c112240 con 0x7fc54c10c5b0 2024-04-16T17:04:52.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.971+0000 7fc554400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c112540 con 0x7fc54c103b90 2024-04-16T17:04:52.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 0x7fc54c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc552400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc54c10c5b0 0x7fc54c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc54c103b90 0x7fc54c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 0x7fc54c1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55632/0 (socket says 172.21.15.18:55632) 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551a00640 1 -- 172.21.15.18:0/2391580167 learned_addr learned my addr 172.21.15.18:0/2391580167 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551a00640 1 -- 172.21.15.18:0/2391580167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc54c103b90 msgr2=0x7fc54c1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551a00640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc54c103b90 0x7fc54c1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551a00640 1 -- 172.21.15.18:0/2391580167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc54c10c5b0 msgr2=0x7fc54c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551a00640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc54c10c5b0 0x7fc54c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.972+0000 7fc551a00640 1 -- 172.21.15.18:0/2391580167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c19b080 con 0x7fc54c103270 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.973+0000 7fc551a00640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 0x7fc54c1a0700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc540000c00 tx=0x7fc5400043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.973+0000 7fc54a000640 1 -- 172.21.15.18:0/2391580167 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540042020 con 0x7fc54c103270 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.973+0000 7fc54a000640 1 -- 172.21.15.18:0/2391580167 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5400054e0 con 0x7fc54c103270 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.973+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c19b310 con 0x7fc54c103270 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.973+0000 7fc54a000640 1 -- 172.21.15.18:0/2391580167 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540003af0 con 0x7fc54c103270 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.973+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c1a75c0 con 0x7fc54c103270 2024-04-16T17:04:52.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.974+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c068eb0 con 0x7fc54c103270 2024-04-16T17:04:52.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.976+0000 7fc54a000640 1 -- 172.21.15.18:0/2391580167 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc540045020 con 0x7fc54c103270 2024-04-16T17:04:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.978+0000 7fc54a000640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5340a3e40 0x7fc5340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.978+0000 7fc54a000640 1 -- 172.21.15.18:0/2391580167 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc5400f29e0 con 0x7fc54c103270 2024-04-16T17:04:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.978+0000 7fc551000640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5340a3e40 0x7fc5340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:52.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.979+0000 7fc551000640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5340a3e40 0x7fc5340a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc54c10bf30 tx=0x7fc538005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:52.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:52.984+0000 7fc54a000640 1 -- 172.21.15.18:0/2391580167 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc54004c020 con 0x7fc54c103270 2024-04-16T17:04:53.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.195+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc54c1a7790 con 0x7fc54c103270 2024-04-16T17:04:53.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.196+0000 7fc54a000640 1 -- 172.21.15.18:0/2391580167 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fc5400bec00 con 0x7fc54c103270 2024-04-16T17:04:53.198 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5340a3e40 msgr2=0x7fc5340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5340a3e40 0x7fc5340a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc54c10bf30 tx=0x7fc538005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 msgr2=0x7fc54c1a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 0x7fc54c1a0700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc540000c00 tx=0x7fc5400043c0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 shutdown_connections 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5340a3e40 0x7fc5340a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc54c10c5b0 0x7fc54c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc54c103b90 0x7fc54c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 --2- 172.21.15.18:0/2391580167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc54c103270 0x7fc54c1a0700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 >> 172.21.15.18:0/2391580167 conn(0x7fc54c0780b0 msgr2=0x7fc54c1084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.199+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 shutdown_connections 2024-04-16T17:04:53.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.200+0000 7fc554400640 1 -- 172.21.15.18:0/2391580167 wait complete. 2024-04-16T17:04:53.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:04:53.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 -- 172.21.15.18:0/417066216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 msgr2=0x7fb470106270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 --2- 172.21.15.18:0/417066216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 0x7fb470106270 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb458009cd0 tx=0x7fb45802c0e0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 -- 172.21.15.18:0/417066216 shutdown_connections 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 --2- 172.21.15.18:0/417066216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4701067b0 0x7fb4701111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 --2- 172.21.15.18:0/417066216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 0x7fb470106270 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 --2- 172.21.15.18:0/417066216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb470104530 0x7fb470104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 -- 172.21.15.18:0/417066216 >> 172.21.15.18:0/417066216 conn(0x7fb4700ffc90 msgr2=0x7fb4701020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 -- 172.21.15.18:0/417066216 shutdown_connections 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.450+0000 7fb474a00640 1 -- 172.21.15.18:0/417066216 wait complete. 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.451+0000 7fb474a00640 1 Processor -- start 2024-04-16T17:04:53.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.451+0000 7fb474a00640 1 -- start start 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb474a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb470104530 0x7fb4701a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb474a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 0x7fb4701a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb474a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701067b0 0x7fb47019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb474a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb4701146a0 con 0x7fb470105e10 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb474a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb470114520 con 0x7fb470104530 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb474a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb470114820 con 0x7fb4701067b0 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb465400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 0x7fb4701a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb465400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 0x7fb4701a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55648/0 (socket says 172.21.15.18:55648) 2024-04-16T17:04:53.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb465400640 1 -- 172.21.15.18:0/1539521206 learned_addr learned my addr 172.21.15.18:0/1539521206 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb46de00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb470104530 0x7fb4701a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.452+0000 7fb46e800640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701067b0 0x7fb47019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb465400640 1 -- 172.21.15.18:0/1539521206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701067b0 msgr2=0x7fb47019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb465400640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701067b0 0x7fb47019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb465400640 1 -- 172.21.15.18:0/1539521206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb470104530 msgr2=0x7fb4701a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb465400640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb470104530 0x7fb4701a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb465400640 1 -- 172.21.15.18:0/1539521206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47019b160 con 0x7fb470105e10 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb46e800640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701067b0 0x7fb47019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:53.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb465400640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 0x7fb4701a0d70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb458000f80 tx=0x7fb458032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:53.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb467400640 1 -- 172.21.15.18:0/1539521206 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458039040 con 0x7fb470105e10 2024-04-16T17:04:53.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb467400640 1 -- 172.21.15.18:0/1539521206 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb458044070 con 0x7fb470105e10 2024-04-16T17:04:53.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47019b3f0 con 0x7fb470105e10 2024-04-16T17:04:53.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb467400640 1 -- 172.21.15.18:0/1539521206 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4580042a0 con 0x7fb470105e10 2024-04-16T17:04:53.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.453+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4701a76f0 con 0x7fb470105e10 2024-04-16T17:04:53.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.455+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb470069300 con 0x7fb470105e10 2024-04-16T17:04:53.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.457+0000 7fb467400640 1 -- 172.21.15.18:0/1539521206 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fb458003970 con 0x7fb470105e10 2024-04-16T17:04:53.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.458+0000 7fb467400640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4540a3ec0 0x7fb4540a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:53.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.458+0000 7fb467400640 1 -- 172.21.15.18:0/1539521206 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb4580f3250 con 0x7fb470105e10 2024-04-16T17:04:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.458+0000 7fb46de00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4540a3ec0 0x7fb4540a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.459+0000 7fb46de00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4540a3ec0 0x7fb4540a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb47019bb20 tx=0x7fb460005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.463+0000 7fb467400640 1 -- 172.21.15.18:0/1539521206 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb458035de0 con 0x7fb470105e10 2024-04-16T17:04:53.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.671+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb47010c800 con 0x7fb470105e10 2024-04-16T17:04:53.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.672+0000 7fb467400640 1 -- 172.21.15.18:0/1539521206 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fb4580bf430 con 0x7fb470105e10 2024-04-16T17:04:53.673 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:04:53.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4540a3ec0 msgr2=0x7fb4540a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4540a3ec0 0x7fb4540a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb47019bb20 tx=0x7fb460005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 msgr2=0x7fb4701a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 0x7fb4701a0d70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb458000f80 tx=0x7fb458032460 comp rx=0 tx=0).stop 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 shutdown_connections 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4540a3ec0 0x7fb4540a6380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4701067b0 0x7fb47019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb470105e10 0x7fb4701a0d70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 --2- 172.21.15.18:0/1539521206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb470104530 0x7fb4701a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 >> 172.21.15.18:0/1539521206 conn(0x7fb4700ffc90 msgr2=0x7fb4701004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 shutdown_connections 2024-04-16T17:04:53.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.675+0000 7fb474a00640 1 -- 172.21.15.18:0/1539521206 wait complete. 2024-04-16T17:04:53.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:04:53.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:53 smithi043 ceph-mon[36901]: from='client.25589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:53.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:53 smithi043 ceph-mon[36901]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-16T17:04:53.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:53 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2081480951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:53.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:53 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2391580167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:53.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:53 smithi018 ceph-mon[31776]: from='client.25589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:53.784 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:53 smithi018 ceph-mon[31776]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-16T17:04:53.784 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:53 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2081480951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:53.784 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:53 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2391580167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:53.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:53 smithi022 ceph-mon[36023]: from='client.25589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:04:53.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:53 smithi022 ceph-mon[36023]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-16T17:04:53.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:53 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2081480951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:04:53.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:53 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2391580167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:04:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 -- 172.21.15.18:0/801249290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 msgr2=0x7f47bc112750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 --2- 172.21.15.18:0/801249290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 0x7f47bc112750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009ca0 tx=0x7f47ac02c100 comp rx=0 tx=0).stop 2024-04-16T17:04:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 -- 172.21.15.18:0/801249290 shutdown_connections 2024-04-16T17:04:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 --2- 172.21.15.18:0/801249290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47bc112c90 0x7f47bc115080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 --2- 172.21.15.18:0/801249290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 0x7f47bc112750 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 --2- 172.21.15.18:0/801249290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc076530 0x7f47bc076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 -- 172.21.15.18:0/801249290 >> 172.21.15.18:0/801249290 conn(0x7f47bc0ffaa0 msgr2=0x7f47bc101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 -- 172.21.15.18:0/801249290 shutdown_connections 2024-04-16T17:04:53.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.930+0000 7f47bbe00640 1 -- 172.21.15.18:0/801249290 wait complete. 2024-04-16T17:04:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.931+0000 7f47bbe00640 1 Processor -- start 2024-04-16T17:04:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47bbe00640 1 -- start start 2024-04-16T17:04:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47bc076530 0x7f47bc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47bbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 0x7f47bc19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc112c90 0x7f47bc1a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47bbe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc118800 con 0x7f47bc076e50 2024-04-16T17:04:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47bbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc118680 con 0x7f47bc076530 2024-04-16T17:04:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47bbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc118980 con 0x7f47bc112c90 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47ba000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 0x7f47bc19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47ba000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 0x7f47bc19f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55658/0 (socket says 172.21.15.18:55658) 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47ba000640 1 -- 172.21.15.18:0/2109669394 learned_addr learned my addr 172.21.15.18:0/2109669394 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47bb400640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc112c90 0x7f47bc1a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.932+0000 7f47baa00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47bc076530 0x7f47bc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.933+0000 7f47ba000640 1 -- 172.21.15.18:0/2109669394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc112c90 msgr2=0x7f47bc1a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.933+0000 7f47ba000640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc112c90 0x7f47bc1a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.933+0000 7f47ba000640 1 -- 172.21.15.18:0/2109669394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47bc076530 msgr2=0x7f47bc19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.933+0000 7f47ba000640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47bc076530 0x7f47bc19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.933+0000 7f47ba000640 1 -- 172.21.15.18:0/2109669394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc1a3d80 con 0x7f47bc076e50 2024-04-16T17:04:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.933+0000 7f47ba000640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 0x7f47bc19f320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47ac005e30 tx=0x7f47ac0346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.933+0000 7f47b3400640 1 -- 172.21.15.18:0/2109669394 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac039050 con 0x7f47bc076e50 2024-04-16T17:04:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.934+0000 7f47b3400640 1 -- 172.21.15.18:0/2109669394 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47ac045070 con 0x7f47bc076e50 2024-04-16T17:04:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.934+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc1a4010 con 0x7f47bc076e50 2024-04-16T17:04:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.934+0000 7f47b3400640 1 -- 172.21.15.18:0/2109669394 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac004500 con 0x7f47bc076e50 2024-04-16T17:04:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.934+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc1ab900 con 0x7f47bc076e50 2024-04-16T17:04:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.935+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47bc04fd90 con 0x7f47bc076e50 2024-04-16T17:04:53.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.937+0000 7f47b3400640 1 -- 172.21.15.18:0/2109669394 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f47ac0046a0 con 0x7f47bc076e50 2024-04-16T17:04:53.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.938+0000 7f47b3400640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47a00a3e40 0x7f47a00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:53.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.939+0000 7f47b3400640 1 -- 172.21.15.18:0/2109669394 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f47ac0f3360 con 0x7f47bc076e50 2024-04-16T17:04:53.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.939+0000 7f47baa00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47a00a3e40 0x7f47a00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:53.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.939+0000 7f47baa00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47a00a3e40 0x7f47a00a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009bd0 tx=0x7f47a4005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:53.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:53.943+0000 7f47b3400640 1 -- 172.21.15.18:0/2109669394 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47ac0bf4b0 con 0x7f47bc076e50 2024-04-16T17:04:54.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.148+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f47bc10de20 con 0x7f47bc076e50 2024-04-16T17:04:54.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.149+0000 7f47b3400640 1 -- 172.21.15.18:0/2109669394 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f47ac0bedb0 con 0x7f47bc076e50 2024-04-16T17:04:54.151 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:04:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.151+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47a00a3e40 msgr2=0x7f47a00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.151+0000 7f47bbe00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47a00a3e40 0x7f47a00a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009bd0 tx=0x7f47a4005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.151+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 msgr2=0x7f47bc19f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.151+0000 7f47bbe00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 0x7f47bc19f320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47ac005e30 tx=0x7f47ac0346f0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.152+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 shutdown_connections 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.152+0000 7f47bbe00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47a00a3e40 0x7f47a00a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.152+0000 7f47bbe00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc112c90 0x7f47bc1a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.152+0000 7f47bbe00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47bc076e50 0x7f47bc19f320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.152+0000 7f47bbe00640 1 --2- 172.21.15.18:0/2109669394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47bc076530 0x7f47bc19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.152+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 >> 172.21.15.18:0/2109669394 conn(0x7f47bc0ffaa0 msgr2=0x7f47bc1143c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.152+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 shutdown_connections 2024-04-16T17:04:54.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.152+0000 7f47bbe00640 1 -- 172.21.15.18:0/2109669394 wait complete. 2024-04-16T17:04:54.176 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:04:54.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 -- 172.21.15.18:0/1291026942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390c074430 msgr2=0x7f390c074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 --2- 172.21.15.18:0/1291026942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390c074430 0x7f390c074810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009900 tx=0x7f38fc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 -- 172.21.15.18:0/1291026942 shutdown_connections 2024-04-16T17:04:54.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 --2- 172.21.15.18:0/1291026942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f390c10e800 0x7f390c110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 --2- 172.21.15.18:0/1291026942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f390c074d50 0x7f390c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 --2- 172.21.15.18:0/1291026942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390c074430 0x7f390c074810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 -- 172.21.15.18:0/1291026942 >> 172.21.15.18:0/1291026942 conn(0x7f390c0fdad0 msgr2=0x7f390c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:54.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 -- 172.21.15.18:0/1291026942 shutdown_connections 2024-04-16T17:04:54.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.400+0000 7f3912000640 1 -- 172.21.15.18:0/1291026942 wait complete. 2024-04-16T17:04:54.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.401+0000 7f3912000640 1 Processor -- start 2024-04-16T17:04:54.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.401+0000 7f3912000640 1 -- start start 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.401+0000 7f3912000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f390c074430 0x7f390c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.401+0000 7f3912000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f390c074d50 0x7f390c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.401+0000 7f3912000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390c10e800 0x7f390c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.401+0000 7f3912000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f390c1145e0 con 0x7f390c074430 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.401+0000 7f3912000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f390c114460 con 0x7f390c074d50 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.401+0000 7f3912000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f390c114760 con 0x7f390c10e800 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f390c074d50 0x7f390c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f390c074430 0x7f390c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:54.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f390c074d50 0x7f390c1a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54578/0 (socket says 172.21.15.18:54578) 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 -- 172.21.15.18:0/1967793510 learned_addr learned my addr 172.21.15.18:0/1967793510 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390be00640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390c10e800 0x7f390c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 -- 172.21.15.18:0/1967793510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390c10e800 msgr2=0x7f390c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390c10e800 0x7f390c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 -- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f390c074430 msgr2=0x7f390c1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f390c074430 0x7f390c1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 -- 172.21.15.18:0/1967793510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c19b0e0 con 0x7f390c074d50 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.402+0000 7f390aa00640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f390c074d50 0x7f390c1a0ca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e9f0 tx=0x7f38f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.403+0000 7f3903e00640 1 -- 172.21.15.18:0/1967793510 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f800cd80 con 0x7f390c074d50 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.403+0000 7f3903e00640 1 -- 172.21.15.18:0/1967793510 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8010910 con 0x7f390c074d50 2024-04-16T17:04:54.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.403+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c19b3d0 con 0x7f390c074d50 2024-04-16T17:04:54.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.403+0000 7f3903e00640 1 -- 172.21.15.18:0/1967793510 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80188f0 con 0x7f390c074d50 2024-04-16T17:04:54.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.403+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c1a7620 con 0x7f390c074d50 2024-04-16T17:04:54.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.404+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c069300 con 0x7f390c074d50 2024-04-16T17:04:54.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.407+0000 7f3903e00640 1 -- 172.21.15.18:0/1967793510 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f38f8010ac0 con 0x7f390c074d50 2024-04-16T17:04:54.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.408+0000 7f3903e00640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f38f00a3e40 0x7f38f00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:54.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.409+0000 7f3903e00640 1 -- 172.21.15.18:0/1967793510 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f38f80d1690 con 0x7f390c074d50 2024-04-16T17:04:54.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.409+0000 7f390b400640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f38f00a3e40 0x7f38f00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:54.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.409+0000 7f390b400640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f38f00a3e40 0x7f38f00a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38fc005ab0 tx=0x7f38fc005a20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:54.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.410+0000 7f3903e00640 1 -- 172.21.15.18:0/1967793510 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38f809d6d0 con 0x7f390c074d50 2024-04-16T17:04:54.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.667+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f390c1099e0 con 0x7f390c074d50 2024-04-16T17:04:54.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.668+0000 7f3903e00640 1 -- 172.21.15.18:0/1967793510 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7f38f809cfd0 con 0x7f390c074d50 2024-04-16T17:04:54.670 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:54.670 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:04:54.670 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:04:54.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f38f00a3e40 msgr2=0x7f38f00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f38f00a3e40 0x7f38f00a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38fc005ab0 tx=0x7f38fc005a20 comp rx=0 tx=0).stop 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f390c074d50 msgr2=0x7f390c1a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f390c074d50 0x7f390c1a0ca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e9f0 tx=0x7f38f800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 shutdown_connections 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f38f00a3e40 0x7f38f00a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390c10e800 0x7f390c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f390c074d50 0x7f390c1a0ca0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 --2- 172.21.15.18:0/1967793510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f390c074430 0x7f390c1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 >> 172.21.15.18:0/1967793510 conn(0x7f390c0fdad0 msgr2=0x7f390c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 shutdown_connections 2024-04-16T17:04:54.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.671+0000 7f3912000640 1 -- 172.21.15.18:0/1967793510 wait complete. 2024-04-16T17:04:54.692 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:04:54.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:54 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1539521206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:54.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:54 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2109669394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:54.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:54 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1539521206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:54.771 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:54 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2109669394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 -- 172.21.15.18:0/3787456101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3470102ac0 msgr2=0x7f3470106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 --2- 172.21.15.18:0/3787456101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3470102ac0 0x7f3470106b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:04:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 -- 172.21.15.18:0/3787456101 shutdown_connections 2024-04-16T17:04:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 --2- 172.21.15.18:0/3787456101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34701071e0 0x7f347010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 --2- 172.21.15.18:0/3787456101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3470102ac0 0x7f3470106b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 --2- 172.21.15.18:0/3787456101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3470102110 0x7f34701024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 -- 172.21.15.18:0/3787456101 >> 172.21.15.18:0/3787456101 conn(0x7f34700fda10 msgr2=0x7f34700ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:54.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 -- 172.21.15.18:0/3787456101 shutdown_connections 2024-04-16T17:04:54.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.912+0000 7f3475200640 1 -- 172.21.15.18:0/3787456101 wait complete. 2024-04-16T17:04:54.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.913+0000 7f3475200640 1 Processor -- start 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.913+0000 7f3475200640 1 -- start start 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.913+0000 7f3475200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3470102110 0x7f34701a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f3475200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34701071e0 0x7f34701a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f3475200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f347019a800 0x7f347019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f3475200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f34701122c0 con 0x7f34701071e0 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f3475200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3470112140 con 0x7f347019a800 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f3475200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3470112440 con 0x7f3470102110 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34701071e0 0x7f34701a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3470102110 0x7f34701a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34701071e0 0x7f34701a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55700/0 (socket says 172.21.15.18:55700) 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3470102110 0x7f34701a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51258/0 (socket says 172.21.15.18:51258) 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346ea00640 1 -- 172.21.15.18:0/2136938999 learned_addr learned my addr 172.21.15.18:0/2136938999 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346fe00640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f347019a800 0x7f347019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346f400640 1 -- 172.21.15.18:0/2136938999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f347019a800 msgr2=0x7f347019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346f400640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f347019a800 0x7f347019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346f400640 1 -- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34701071e0 msgr2=0x7f34701a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346f400640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34701071e0 0x7f34701a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.914+0000 7f346f400640 1 -- 172.21.15.18:0/2136938999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347019b390 con 0x7f3470102110 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.915+0000 7f346f400640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3470102110 0x7f34701a0690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f346000e9f0 tx=0x7f346000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.915+0000 7f3467e00640 1 -- 172.21.15.18:0/2136938999 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346000ccc0 con 0x7f3470102110 2024-04-16T17:04:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.915+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347019b680 con 0x7f3470102110 2024-04-16T17:04:54.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.915+0000 7f3467e00640 1 -- 172.21.15.18:0/2136938999 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460010910 con 0x7f3470102110 2024-04-16T17:04:54.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.915+0000 7f3467e00640 1 -- 172.21.15.18:0/2136938999 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460018960 con 0x7f3470102110 2024-04-16T17:04:54.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.915+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34701a7860 con 0x7f3470102110 2024-04-16T17:04:54.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.916+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34701035e0 con 0x7f3470102110 2024-04-16T17:04:54.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.918+0000 7f3467e00640 1 -- 172.21.15.18:0/2136938999 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3460010ac0 con 0x7f3470102110 2024-04-16T17:04:54.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.919+0000 7f3467e00640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34540a3e30 0x7f34540a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:54.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.920+0000 7f3467e00640 1 -- 172.21.15.18:0/2136938999 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f34600d1210 con 0x7f3470102110 2024-04-16T17:04:54.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.920+0000 7f346ea00640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34540a3e30 0x7f34540a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.920+0000 7f346ea00640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34540a3e30 0x7f34540a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34701012e0 tx=0x7f345c02cf90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:54.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:54 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1539521206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:04:54.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:54 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2109669394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:04:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:54.924+0000 7f3467e00640 1 -- 172.21.15.18:0/2136938999 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f347019b680 con 0x7f3470102110 2024-04-16T17:04:55.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.123+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3470102ac0 con 0x7f3470102110 2024-04-16T17:04:55.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.126+0000 7f3467e00640 1 -- 172.21.15.18:0/2136938999 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25126 (secure 0 0 0) 0x7f346009cc00 con 0x7f3470102110 2024-04-16T17:04:55.127 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:04:55.128 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":94,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:04:44.091776+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3045212308":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/3152065578":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/1033276779":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6801/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/2149549301":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34540a3e30 msgr2=0x7f34540a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34540a3e30 0x7f34540a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34701012e0 tx=0x7f345c02cf90 comp rx=0 tx=0).stop 2024-04-16T17:04:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3470102110 msgr2=0x7f34701a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3470102110 0x7f34701a0690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f346000e9f0 tx=0x7f346000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 shutdown_connections 2024-04-16T17:04:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34540a3e30 0x7f34540a62f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f347019a800 0x7f347019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34701071e0 0x7f34701a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 --2- 172.21.15.18:0/2136938999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3470102110 0x7f34701a0690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 >> 172.21.15.18:0/2136938999 conn(0x7f34700fda10 msgr2=0x7f34700ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 shutdown_connections 2024-04-16T17:04:55.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.128+0000 7f3475200640 1 -- 172.21.15.18:0/2136938999 wait complete. 2024-04-16T17:04:55.150 DEBUG:tasks.fwd_scrub:Yielding 2024-04-16T17:04:55.151 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T17:04:55.162 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T17:04:55.162 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:04:55.162 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-16T17:04:55.162 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-16T17:04:55.163 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:04:55.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:04:55.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.363+0000 7f4a22800640 1 -- 172.21.15.18:0/2583572918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c107160 msgr2=0x7f4a1c10ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.363+0000 7f4a22800640 1 --2- 172.21.15.18:0/2583572918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c107160 0x7f4a1c10ed30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009ca0 tx=0x7f4a1402c0f0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f4a22800640 1 -- 172.21.15.18:0/2583572918 shutdown_connections 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f4a22800640 1 --2- 172.21.15.18:0/2583572918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c107160 0x7f4a1c10ed30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f4a22800640 1 --2- 172.21.15.18:0/2583572918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a1c102a40 0x7f4a1c106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f4a22800640 1 --2- 172.21.15.18:0/2583572918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a1c102090 0x7f4a1c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f4a22800640 1 -- 172.21.15.18:0/2583572918 >> 172.21.15.18:0/2583572918 conn(0x7f4a1c0fd910 msgr2=0x7f4a1c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f4a22800640 1 -- 172.21.15.18:0/2583572918 shutdown_connections 2024-04-16T17:04:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f4a22800640 1 -- 172.21.15.18:0/2583572918 wait complete. 2024-04-16T17:04:55.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f8056000640 1 -- 172.21.15.18:0/1333251438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8050104c30 msgr2=0x7f8050108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.364+0000 7f8056000640 1 --2- 172.21.15.18:0/1333251438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8050104c30 0x7f8050108c70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f803c009860 tx=0x7f803c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f8056000640 1 -- 172.21.15.18:0/1333251438 shutdown_connections 2024-04-16T17:04:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f8056000640 1 --2- 172.21.15.18:0/1333251438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8050109350 0x7f8050110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f8056000640 1 --2- 172.21.15.18:0/1333251438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8050104c30 0x7f8050108c70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f8056000640 1 --2- 172.21.15.18:0/1333251438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8050104280 0x7f8050104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f8056000640 1 -- 172.21.15.18:0/1333251438 >> 172.21.15.18:0/1333251438 conn(0x7f80500ffae0 msgr2=0x7f8050101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f8056000640 1 -- 172.21.15.18:0/1333251438 shutdown_connections 2024-04-16T17:04:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f8056000640 1 -- 172.21.15.18:0/1333251438 wait complete. 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f4a22800640 1 Processor -- start 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f4a22800640 1 -- start start 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f4a22800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c102090 0x7f4a1c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f4a22800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a1c107160 0x7f4a1c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f4a22800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a1c19a720 0x7f4a1c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f4a22800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c112220 con 0x7f4a1c102090 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f4a22800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1120a0 con 0x7f4a1c107160 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.365+0000 7f4a22800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1123a0 con 0x7f4a1c19a720 2024-04-16T17:04:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c102090 0x7f4a1c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c102090 0x7f4a1c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55726/0 (socket says 172.21.15.18:55726) 2024-04-16T17:04:55.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21400640 1 -- 172.21.15.18:0/1240075377 learned_addr learned my addr 172.21.15.18:0/1240075377 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21e00640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a1c19a720 0x7f4a1c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a20a00640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a1c107160 0x7f4a1c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f8056000640 1 Processor -- start 2024-04-16T17:04:55.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21e00640 1 -- 172.21.15.18:0/1240075377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a1c107160 msgr2=0x7f4a1c1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21e00640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a1c107160 0x7f4a1c1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21e00640 1 -- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c102090 msgr2=0x7f4a1c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21e00640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c102090 0x7f4a1c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21e00640 1 -- 172.21.15.18:0/1240075377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c19b320 con 0x7f4a1c19a720 2024-04-16T17:04:55.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f8056000640 1 -- start start 2024-04-16T17:04:55.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.366+0000 7f4a21e00640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a1c19a720 0x7f4a1c19aba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a1402c700 tx=0x7f4a14032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8056000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8050104280 0x7f80501a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8056000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8050109350 0x7f80501a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8056000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f805019a7c0 0x7f805019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8056000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80501143e0 con 0x7f805019a7c0 2024-04-16T17:04:55.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f4a1a000640 1 -- 172.21.15.18:0/1240075377 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14041020 con 0x7f4a1c19a720 2024-04-16T17:04:55.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8056000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8050114260 con 0x7f8050109350 2024-04-16T17:04:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8056000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8050114560 con 0x7f8050104280 2024-04-16T17:04:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c19b5b0 con 0x7f4a1c19a720 2024-04-16T17:04:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c1a76f0 con 0x7f4a1c19a720 2024-04-16T17:04:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f4a1a000640 1 -- 172.21.15.18:0/1240075377 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a14034cd0 con 0x7f4a1c19a720 2024-04-16T17:04:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f4a1a000640 1 -- 172.21.15.18:0/1240075377 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14039040 con 0x7f4a1c19a720 2024-04-16T17:04:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8054c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8050104280 0x7f80501a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8054c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8050104280 0x7f80501a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51294/0 (socket says 172.21.15.18:51294) 2024-04-16T17:04:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8054c00640 1 -- 172.21.15.18:0/3444900659 learned_addr learned my addr 172.21.15.18:0/3444900659 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f804fe00640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8050109350 0x7f80501a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.367+0000 7f8055600640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f805019a7c0 0x7f805019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f8054c00640 1 -- 172.21.15.18:0/3444900659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8050109350 msgr2=0x7f80501a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f8054c00640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8050109350 0x7f80501a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f8054c00640 1 -- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f805019a7c0 msgr2=0x7f805019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f8054c00640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f805019a7c0 0x7f805019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f8054c00640 1 -- 172.21.15.18:0/3444900659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805019b350 con 0x7f8050104280 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f804fe00640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8050109350 0x7f80501a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f8054c00640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8050104280 0x7f80501a0650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f804400e9c0 tx=0x7f804400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f804d600640 1 -- 172.21.15.18:0/3444900659 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804400cde0 con 0x7f8050104280 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f8056000640 1 -- 172.21.15.18:0/3444900659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805019b640 con 0x7f8050104280 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f804d600640 1 -- 172.21.15.18:0/3444900659 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80440035f0 con 0x7f8050104280 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f804d600640 1 -- 172.21.15.18:0/3444900659 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044010650 con 0x7f8050104280 2024-04-16T17:04:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f8056000640 1 -- 172.21.15.18:0/3444900659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80501a77f0 con 0x7f8050104280 2024-04-16T17:04:55.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.368+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c103560 con 0x7f4a1c19a720 2024-04-16T17:04:55.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.371+0000 7f4a1a000640 1 -- 172.21.15.18:0/1240075377 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4a14044020 con 0x7f4a1c19a720 2024-04-16T17:04:55.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.372+0000 7f804d600640 1 -- 172.21.15.18:0/3444900659 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f80440037a0 con 0x7f8050104280 2024-04-16T17:04:55.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.372+0000 7f805a385740 1 Processor -- start 2024-04-16T17:04:55.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f805a385740 1 -- start start 2024-04-16T17:04:55.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f4a1a000640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4a040a3e20 0x7f4a040a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f805a385740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28ee90 0x56408b28f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f805a385740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28f7b0 0x56408b297e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f4a1a000640 1 -- 172.21.15.18:0/1240075377 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f4a140f2ae0 con 0x7f4a1c19a720 2024-04-16T17:04:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f805a385740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 0x56408b29a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f805a385740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56408b1f3cb0 con 0x56408b28f7b0 2024-04-16T17:04:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f805a385740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56408b19a370 con 0x56408b298390 2024-04-16T17:04:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f805a385740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56408b1efc70 con 0x56408b28ee90 2024-04-16T17:04:55.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f4a21400640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4a040a3e20 0x7f4a040a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f8055600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28ee90 0x56408b28f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f8055600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28ee90 0x56408b28f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51302/0 (socket says 172.21.15.18:51302) 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f8055600640 1 -- 172.21.15.18:0/3858950018 learned_addr learned my addr 172.21.15.18:0/3858950018 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.373+0000 7f804d600640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80380a3e40 0x7f80380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f804fe00640 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 0x56408b29a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f804d600640 1 -- 172.21.15.18:0/3444900659 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f80440d11c0 con 0x7f8050104280 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f8055600640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80380a3e40 0x7f80380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f8055600640 1 -- 172.21.15.18:0/3858950018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 msgr2=0x56408b29a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f8055600640 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 0x56408b29a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f8054c00640 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28f7b0 0x56408b297e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f8055600640 1 -- 172.21.15.18:0/3858950018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28f7b0 msgr2=0x56408b297e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f8055600640 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28f7b0 0x56408b297e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f4a21400640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4a040a3e20 0x7f4a040a62e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009cd0 tx=0x7f4a10005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f8055600640 1 -- 172.21.15.18:0/3858950018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408b1b9910 con 0x56408b28ee90 2024-04-16T17:04:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.374+0000 7f8054c00640 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28f7b0 0x56408b297e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:04:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.375+0000 7f8055600640 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28ee90 0x56408b28f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8048005c40 tx=0x7f8048013f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df314c0bbe2d94e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.375+0000 7f8055600640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80380a3e40 0x7f80380a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f804801d000 tx=0x7f804801f000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.376+0000 7f8041c00640 1 -- 172.21.15.18:0/3858950018 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804800fe00 con 0x56408b28ee90 2024-04-16T17:04:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.376+0000 7f8041c00640 1 -- 172.21.15.18:0/3858950018 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8048022020 con 0x56408b28ee90 2024-04-16T17:04:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.376+0000 7f8041c00640 1 -- 172.21.15.18:0/3858950018 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048010690 con 0x56408b28ee90 2024-04-16T17:04:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.376+0000 7f4a1a000640 1 -- 172.21.15.18:0/1240075377 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a1404b020 con 0x7f4a1c19a720 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 -- 172.21.15.18:0/3858950018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28ee90 msgr2=0x56408b28f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28ee90 0x56408b28f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8048005c40 tx=0x7f8048013f50 comp rx=0 tx=0).stop 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 -- 172.21.15.18:0/3858950018 shutdown_connections 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 0x56408b29a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28f7b0 0x56408b297e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 --2- 172.21.15.18:0/3858950018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28ee90 0x56408b28f270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 -- 172.21.15.18:0/3858950018 >> 172.21.15.18:0/3858950018 conn(0x56408b2895c0 msgr2=0x56408b2899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 -- 172.21.15.18:0/3858950018 shutdown_connections 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.377+0000 7f805a385740 1 -- 172.21.15.18:0/3858950018 wait complete. 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.378+0000 7f805a385740 1 -- start start 2024-04-16T17:04:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.378+0000 7f805a385740 1 Processor -- start 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f805a385740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b2954a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f805a385740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f805a385740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 0x56408b295f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f805a385740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56408b29b120 con 0x56408b28ee90 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f805a385740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56408b1ad730 con 0x56408b298390 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f805a385740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56408b29b2a0 con 0x56408b28f7b0 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f805a385740 10 client.0 _ll_register_callbacks cb 0x56408b17c1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f8054c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b2954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f8054c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b2954a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55750/0 (socket says 172.21.15.18:55750) 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f8054c00640 1 -- 172.21.15.18:0/2998278063 learned_addr learned my addr 172.21.15.18:0/2998278063 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f804fe00640 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.379+0000 7f8055600640 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 0x56408b295f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8054c00640 1 -- 172.21.15.18:0/2998278063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 msgr2=0x56408b2959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8054c00640 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8054c00640 1 -- 172.21.15.18:0/2998278063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 msgr2=0x56408b295f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8054c00640 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 0x56408b295f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8054c00640 1 -- 172.21.15.18:0/2998278063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408afce7e0 con 0x56408b28ee90 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f804fe00640 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8054c00640 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b2954a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f804400c6a0 tx=0x7f804409d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d75f26a5f26ff58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8036000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8036000640 1 -- 172.21.15.18:0/2998278063 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80440a5770 con 0x56408b28ee90 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8036000640 1 -- 172.21.15.18:0/2998278063 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f80440d3110 con 0x56408b28ee90 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f805a385740 1 -- 172.21.15.18:0/2998278063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8048022020 con 0x56408b28ee90 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f8036000640 1 -- 172.21.15.18:0/2998278063 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f80440d3330 con 0x56408b28ee90 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f805a385740 10 client.15522 fetch_fsmap learned FSMap version 25 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.380+0000 7f805a385740 1 -- 172.21.15.18:0/2998278063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56408b02ef70 con 0x56408b28ee90 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.381+0000 7f8036000640 1 -- 172.21.15.18:0/2998278063 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8048022020 con 0x56408b28ee90 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.381+0000 7f8036000640 1 -- 172.21.15.18:0/2998278063 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f804409c0d0 con 0x56408b28ee90 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.381+0000 7f805a385740 10 client.15522 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.381+0000 7f805a385740 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:04:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.381+0000 7f805a385740 20 client.15522 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.381+0000 7f805a385740 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56408b2a8650 0x56408b2aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.381+0000 7f805a385740 4 client.15522 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.381+0000 7f805a385740 1 -- 172.21.15.18:0/2998278063 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56408b2016d0 con 0x56408b2a8650 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.382+0000 7f804fe00640 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56408b2a8650 0x56408b2aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.382+0000 7f804fe00640 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56408b2a8650 0x56408b2aaa40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d32102d0ec7837 server_cookie=9c9b4939bb5e44f3 in_seq=0 out_seq=0 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.382+0000 7f8036000640 10 client.15522 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f8036000640 1 -- 172.21.15.18:0/2998278063 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56408b2016d0 con 0x56408b2a8650 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f8036000640 10 client.15522 handle_command_reply: tid=0 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 client.15522 shutdown 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 -- 172.21.15.18:0/2998278063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 msgr2=0x56408b2954a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b2954a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f804400c6a0 tx=0x7f804409d340 comp rx=0 tx=0).stop 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 -- 172.21.15.18:0/2998278063 shutdown_connections 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56408b2a8650 0x56408b2aaa40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b298390 0x56408b295f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 --2- 172.21.15.18:0/2998278063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b2954a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f805a385740 1 -- 172.21.15.18:0/2998278063 >> 172.21.15.18:0/2998278063 conn(0x56408b2895c0 msgr2=0x56408b28b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.383+0000 7f8036000640 0 client.15522 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:55.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.384+0000 7f805a385740 1 -- 172.21.15.18:0/2998278063 shutdown_connections 2024-04-16T17:04:55.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.384+0000 7f805a385740 1 -- 172.21.15.18:0/2998278063 wait complete. 2024-04-16T17:04:55.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.384+0000 7f805a385740 20 client.15522 trim_cache size 0 max 16384 2024-04-16T17:04:55.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.402+0000 7f805a385740 1 Processor -- start 2024-04-16T17:04:55.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.402+0000 7f805a385740 1 -- start start 2024-04-16T17:04:55.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f805a385740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b297bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f805a385740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28f7b0 0x56408b2a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f805a385740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b298390 0x56408b2b0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f805a385740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80501b3f20 con 0x56408b28ee90 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f805a385740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80501b17f0 con 0x56408b28f7b0 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f805a385740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80501af640 con 0x56408b298390 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f8054c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b297bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f8054c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b297bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55756/0 (socket says 172.21.15.18:55756) 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f8054c00640 1 -- 172.21.15.18:0/1345422778 learned_addr learned my addr 172.21.15.18:0/1345422778 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f804fe00640 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28f7b0 0x56408b2a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.403+0000 7f8055600640 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b298390 0x56408b2b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f804fe00640 1 -- 172.21.15.18:0/1345422778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b298390 msgr2=0x56408b2b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f804fe00640 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b298390 0x56408b2b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f804fe00640 1 -- 172.21.15.18:0/1345422778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 msgr2=0x56408b297bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f804fe00640 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b297bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f804fe00640 1 -- 172.21.15.18:0/1345422778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408b29aaf0 con 0x56408b28f7b0 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f8054c00640 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b297bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f804fe00640 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28f7b0 0x56408b2a3fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f803c009780 tx=0x7f803c02bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68bc4a3aa0a4061 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f8041200640 1 -- 172.21.15.18:0/1345422778 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c039040 con 0x56408b28f7b0 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f8041200640 1 -- 172.21.15.18:0/1345422778 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803c047d50 con 0x56408b28f7b0 2024-04-16T17:04:55.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.404+0000 7f8041200640 1 -- 172.21.15.18:0/1345422778 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c047860 con 0x56408b28f7b0 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 -- 172.21.15.18:0/1345422778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28f7b0 msgr2=0x56408b2a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28f7b0 0x56408b2a3fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f803c009780 tx=0x7f803c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 -- 172.21.15.18:0/1345422778 shutdown_connections 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b298390 0x56408b2b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28f7b0 0x56408b2a3fe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 --2- 172.21.15.18:0/1345422778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b28ee90 0x56408b297bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 -- 172.21.15.18:0/1345422778 >> 172.21.15.18:0/1345422778 conn(0x56408b2895c0 msgr2=0x56408b288560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 -- 172.21.15.18:0/1345422778 shutdown_connections 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.405+0000 7f805a385740 1 -- 172.21.15.18:0/1345422778 wait complete. 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 1 -- start start 2024-04-16T17:04:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 1 Processor -- start 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28ee90 0x56408b2a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b298390 0x56408b29d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80501afcb0 con 0x56408b298390 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80501b1120 con 0x56408b28ee90 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80501b0380 con 0x56408b28f7b0 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f805a385740 10 client.0 _ll_register_callbacks cb 0x7f804409c460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f8055600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b298390 0x56408b29d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f8054c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28ee90 0x56408b2a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.406+0000 7f8055600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b298390 0x56408b29d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55760/0 (socket says 172.21.15.18:55760) 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8055600640 1 -- 172.21.15.18:0/2465345627 learned_addr learned my addr 172.21.15.18:0/2465345627 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f804fe00640 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8055600640 1 -- 172.21.15.18:0/2465345627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 msgr2=0x56408b2a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8055600640 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8055600640 1 -- 172.21.15.18:0/2465345627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28ee90 msgr2=0x56408b2a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8055600640 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28ee90 0x56408b2a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8055600640 1 -- 172.21.15.18:0/2465345627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408aec6170 con 0x56408b298390 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8054c00640 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28ee90 0x56408b2a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8055600640 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b298390 0x56408b29d670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f804800a010 tx=0x7f8048013a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75c49ccfa74a9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.407+0000 7f8036000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f8036000640 1 -- 172.21.15.18:0/2465345627 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048036060 con 0x56408b298390 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 1 -- 172.21.15.18:0/2465345627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803c047d50 con 0x56408b298390 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f8036000640 1 -- 172.21.15.18:0/2465345627 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8048012a20 con 0x56408b298390 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f8036000640 1 -- 172.21.15.18:0/2465345627 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f804803d050 con 0x56408b298390 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 10 client.15528 fetch_fsmap learned FSMap version 25 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 1 -- 172.21.15.18:0/2465345627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56408b317d80 con 0x56408b298390 2024-04-16T17:04:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f8036000640 1 -- 172.21.15.18:0/2465345627 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803c047d50 con 0x56408b298390 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f8036000640 1 -- 172.21.15.18:0/2465345627 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8048030070 con 0x56408b298390 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 20 client.15528 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56408b318330 0x56408b31a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 4 client.15528 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.408+0000 7f805a385740 1 -- 172.21.15.18:0/2465345627 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56408b1fb540 con 0x56408b318330 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.409+0000 7f8054c00640 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56408b318330 0x56408b31a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.409+0000 7f8054c00640 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56408b318330 0x56408b31a750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a7b6eb75183a9 server_cookie=3622178c17e82d4b in_seq=0 out_seq=0 2024-04-16T17:04:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.409+0000 7f8036000640 10 client.15528 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:55.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.410+0000 7f8036000640 1 -- 172.21.15.18:0/2465345627 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56408b1fb540 con 0x56408b318330 2024-04-16T17:04:55.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.410+0000 7f8036000640 10 client.15528 handle_command_reply: tid=0 2024-04-16T17:04:55.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.410+0000 7f805a385740 1 client.15528 shutdown 2024-04-16T17:04:55.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 -- 172.21.15.18:0/2465345627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b298390 msgr2=0x56408b29d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b298390 0x56408b29d670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f804800a010 tx=0x7f8048013a30 comp rx=0 tx=0).stop 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 -- 172.21.15.18:0/2465345627 shutdown_connections 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56408b318330 0x56408b31a750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56408b298390 0x56408b29d670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56408b28f7b0 0x56408b2a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 --2- 172.21.15.18:0/2465345627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408b28ee90 0x56408b2a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 -- 172.21.15.18:0/2465345627 >> 172.21.15.18:0/2465345627 conn(0x56408b2895c0 msgr2=0x56408b28a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f8036000640 0 client.15528 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 -- 172.21.15.18:0/2465345627 shutdown_connections 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 1 -- 172.21.15.18:0/2465345627 wait complete. 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.411+0000 7f805a385740 20 client.15528 trim_cache size 0 max 16384 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:04:55.415 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:04:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "893ae7de-09ee-4870-8693-9bae5f5ab2bb", 2024-04-16T17:04:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:04:55.416 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:04:55.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 -- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80380a3e40 msgr2=0x7f80380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80380a3e40 0x7f80380a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f804801d000 tx=0x7f804801f000 comp rx=0 tx=0).stop 2024-04-16T17:04:55.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 -- 172.21.15.18:0/3444900659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8050104280 msgr2=0x7f80501a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8050104280 0x7f80501a0650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f804400e9c0 tx=0x7f804400ee80 comp rx=0 tx=0).stop 2024-04-16T17:04:55.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 -- 172.21.15.18:0/3444900659 shutdown_connections 2024-04-16T17:04:55.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80380a3e40 0x7f80380a6300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f805019a7c0 0x7f805019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8050109350 0x7f80501a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 --2- 172.21.15.18:0/3444900659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8050104280 0x7f80501a0650 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 -- 172.21.15.18:0/3444900659 >> 172.21.15.18:0/3444900659 conn(0x7f80500ffae0 msgr2=0x7f8050100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 -- 172.21.15.18:0/3444900659 shutdown_connections 2024-04-16T17:04:55.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.412+0000 7f8056000640 1 -- 172.21.15.18:0/3444900659 wait complete. 2024-04-16T17:04:55.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:04:55.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:04:55.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.611+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4a1c102a40 con 0x7f4a040a3e20 2024-04-16T17:04:55.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 -- 172.21.15.18:0/590741336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f008c104df0 msgr2=0x7f008c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 --2- 172.21.15.18:0/590741336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f008c104df0 0x7f008c108e30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0078009900 tx=0x7f007802bc50 comp rx=0 tx=0).stop 2024-04-16T17:04:55.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 -- 172.21.15.18:0/590741336 shutdown_connections 2024-04-16T17:04:55.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 --2- 172.21.15.18:0/590741336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c109510 0x7f008c1110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 --2- 172.21.15.18:0/590741336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f008c104df0 0x7f008c108e30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 --2- 172.21.15.18:0/590741336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f008c104440 0x7f008c104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 -- 172.21.15.18:0/590741336 >> 172.21.15.18:0/590741336 conn(0x7f008c0ffc80 msgr2=0x7f008c1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 -- 172.21.15.18:0/590741336 shutdown_connections 2024-04-16T17:04:55.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.626+0000 7f0093a00640 1 -- 172.21.15.18:0/590741336 wait complete. 2024-04-16T17:04:55.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.627+0000 7f0093a00640 1 Processor -- start 2024-04-16T17:04:55.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.627+0000 7f0093a00640 1 -- start start 2024-04-16T17:04:55.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.627+0000 7f0093a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f008c104440 0x7f008c1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0093a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f008c109510 0x7f008c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0093a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c19aa00 0x7f008c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0093a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f008c114580 con 0x7f008c19aa00 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0093a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f008c114400 con 0x7f008c104440 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0093a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f008c114700 con 0x7f008c109510 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f008c104440 0x7f008c1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f008c104440 0x7f008c1a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54670/0 (socket says 172.21.15.18:54670) 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091000640 1 -- 172.21.15.18:0/2744960106 learned_addr learned my addr 172.21.15.18:0/2744960106 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f008be00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f008c109510 0x7f008c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c19aa00 0x7f008c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091000640 1 -- 172.21.15.18:0/2744960106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f008c109510 msgr2=0x7f008c1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091000640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f008c109510 0x7f008c1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091000640 1 -- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c19aa00 msgr2=0x7f008c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091000640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c19aa00 0x7f008c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091000640 1 -- 172.21.15.18:0/2744960106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c19b520 con 0x7f008c104440 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.628+0000 7f0091a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c19aa00 0x7f008c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.629+0000 7f0091000640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f008c104440 0x7f008c1a0850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f008000e9f0 tx=0x7f008000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.629+0000 7f0089600640 1 -- 172.21.15.18:0/2744960106 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008000cd80 con 0x7f008c104440 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.629+0000 7f0093a00640 1 -- 172.21.15.18:0/2744960106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008c19b810 con 0x7f008c104440 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.629+0000 7f0089600640 1 -- 172.21.15.18:0/2744960106 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0080010910 con 0x7f008c104440 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.629+0000 7f0089600640 1 -- 172.21.15.18:0/2744960106 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080018980 con 0x7f008c104440 2024-04-16T17:04:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.629+0000 7f0093a00640 1 -- 172.21.15.18:0/2744960106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008c1a7ab0 con 0x7f008c104440 2024-04-16T17:04:55.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.633+0000 7f0096617740 1 Processor -- start 2024-04-16T17:04:55.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.633+0000 7f0096617740 1 -- start start 2024-04-16T17:04:55.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.633+0000 7f0089600640 1 -- 172.21.15.18:0/2744960106 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0080010ac0 con 0x7f008c104440 2024-04-16T17:04:55.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.633+0000 7f0096617740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a4fc0 0x5635566a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0096617740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0096617740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566b6ec0 0x5635566b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0096617740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56355660b120 con 0x5635566a4fc0 2024-04-16T17:04:55.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0096617740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5635565b1340 con 0x5635566a5970 2024-04-16T17:04:55.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0096617740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5635566070e0 con 0x5635566b6ec0 2024-04-16T17:04:55.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0091000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0091000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54672/0 (socket says 172.21.15.18:54672) 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0091000640 1 -- 172.21.15.18:0/3732106712 learned_addr learned my addr 172.21.15.18:0/3732106712 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f0091a00640 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a4fc0 0x5635566a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.634+0000 7f008be00640 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566b6ec0 0x5635566b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.635+0000 7f0091000640 1 -- 172.21.15.18:0/3732106712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566b6ec0 msgr2=0x5635566b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.635+0000 7f0091000640 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566b6ec0 0x5635566b92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.635+0000 7f0091000640 1 -- 172.21.15.18:0/3732106712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a4fc0 msgr2=0x5635566a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.635+0000 7f0091000640 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a4fc0 0x5635566a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.635+0000 7f0091000640 1 -- 172.21.15.18:0/3732106712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635563e77d0 con 0x5635566a5970 2024-04-16T17:04:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.635+0000 7f008be00640 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566b6ec0 0x5635566b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:04:55.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.636+0000 7f0091000640 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566b6980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00800ce560 tx=0x7f00800d9930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f844b270eeef13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.636+0000 7f0089600640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00740a3e40 0x7f00740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.636+0000 7f0089600640 1 -- 172.21.15.18:0/2744960106 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f00800d1230 con 0x7f008c104440 2024-04-16T17:04:55.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.636+0000 7f007e000640 1 -- 172.21.15.18:0/3732106712 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080098e70 con 0x5635566a5970 2024-04-16T17:04:55.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.637+0000 7f007e000640 1 -- 172.21.15.18:0/3732106712 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f008009d2d0 con 0x5635566a5970 2024-04-16T17:04:55.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.637+0000 7f007e000640 1 -- 172.21.15.18:0/3732106712 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008009c530 con 0x5635566a5970 2024-04-16T17:04:55.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.637+0000 7f0091a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00740a3e40 0x7f00740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 -- 172.21.15.18:0/3732106712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 msgr2=0x5635566b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0091a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00740a3e40 0x7f00740a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00840099a0 tx=0x7f0084009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566b6980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00800ce560 tx=0x7f00800d9930 comp rx=0 tx=0).stop 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 -- 172.21.15.18:0/3732106712 shutdown_connections 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566b6ec0 0x5635566b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566b6980 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 --2- 172.21.15.18:0/3732106712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a4fc0 0x5635566a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 -- 172.21.15.18:0/3732106712 >> 172.21.15.18:0/3732106712 conn(0x56355669f8e0 msgr2=0x56355669fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 -- 172.21.15.18:0/3732106712 shutdown_connections 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.638+0000 7f0096617740 1 -- 172.21.15.18:0/3732106712 wait complete. 2024-04-16T17:04:55.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.639+0000 7f0096617740 1 -- start start 2024-04-16T17:04:55.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.639+0000 7f0096617740 1 Processor -- start 2024-04-16T17:04:55.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.639+0000 7f0096617740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a4fc0 0x5635566b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0096617740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a5970 0x5635566b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0096617740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x5635566b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a4fc0 0x5635566b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0096617740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5635566b9c80 con 0x5635566b6ec0 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0096617740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5635565c4710 con 0x5635566a4fc0 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0096617740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5635566b9e00 con 0x5635566a5970 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a4fc0 0x5635566b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54682/0 (socket says 172.21.15.18:54682) 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091000640 1 -- 172.21.15.18:0/1178077986 learned_addr learned my addr 172.21.15.18:0/1178077986 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0096617740 10 client.0 _ll_register_callbacks cb 0x5635565bfaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f008be00640 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a5970 0x5635566b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091a00640 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x5635566b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091000640 1 -- 172.21.15.18:0/1178077986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a5970 msgr2=0x5635566b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091000640 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a5970 0x5635566b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091000640 1 -- 172.21.15.18:0/1178077986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 msgr2=0x5635566b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091000640 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x5635566b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091000640 1 -- 172.21.15.18:0/1178077986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355631d3f0 con 0x5635566a4fc0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.640+0000 7f0091a00640 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x5635566b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0091000640 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a4fc0 0x5635566b3f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00800ce560 tx=0x7f0080009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1960b30fd1dcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0072a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0072a00640 1 -- 172.21.15.18:0/1178077986 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008009c710 con 0x5635566a4fc0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0072a00640 1 -- 172.21.15.18:0/1178077986 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f008009ce30 con 0x5635566a4fc0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0096617740 1 -- 172.21.15.18:0/1178077986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f008009d2d0 con 0x5635566a4fc0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0072a00640 1 -- 172.21.15.18:0/1178077986 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f008009c9a0 con 0x5635566a4fc0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0096617740 10 client.25667 fetch_fsmap learned FSMap version 25 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0096617740 1 -- 172.21.15.18:0/1178077986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635562dd0f0 con 0x5635566a4fc0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.641+0000 7f0072a00640 1 -- 172.21.15.18:0/1178077986 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f008009d2d0 con 0x5635566a4fc0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.642+0000 7f0072a00640 1 -- 172.21.15.18:0/1178077986 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f00800a53b0 con 0x5635566a4fc0 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.642+0000 7f0096617740 10 client.25667 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:04:55.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.642+0000 7f0096617740 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:04:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.642+0000 7f0096617740 20 client.25667 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.642+0000 7f0096617740 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5635566c71a0 0x5635566c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.642+0000 7f0096617740 4 client.25667 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:04:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.642+0000 7f0096617740 1 -- 172.21.15.18:0/1178077986 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563556618b40 con 0x5635566c71a0 2024-04-16T17:04:55.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.642+0000 7f008be00640 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5635566c71a0 0x5635566c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.643+0000 7f008be00640 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5635566c71a0 0x5635566c9590 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29409f7b7e4af9d0 server_cookie=eef9e3be73ca17d5 in_seq=0 out_seq=0 2024-04-16T17:04:55.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.643+0000 7f0072a00640 10 client.25667 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:55.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.643+0000 7f0072a00640 1 -- 172.21.15.18:0/1178077986 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563556618b40 con 0x5635566c71a0 2024-04-16T17:04:55.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.643+0000 7f0072a00640 10 client.25667 handle_command_reply: tid=0 2024-04-16T17:04:55.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 client.25667 shutdown 2024-04-16T17:04:55.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 -- 172.21.15.18:0/1178077986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a4fc0 msgr2=0x5635566b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a4fc0 0x5635566b3f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00800ce560 tx=0x7f0080009590 comp rx=0 tx=0).stop 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 -- 172.21.15.18:0/1178077986 shutdown_connections 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5635566c71a0 0x5635566c9590 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x5635566b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a5970 0x5635566b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 --2- 172.21.15.18:0/1178077986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a4fc0 0x5635566b3f90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 -- 172.21.15.18:0/1178077986 >> 172.21.15.18:0/1178077986 conn(0x56355669f8e0 msgr2=0x5635566a1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0072a00640 0 client.25667 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 -- 172.21.15.18:0/1178077986 shutdown_connections 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.644+0000 7f0096617740 1 -- 172.21.15.18:0/1178077986 wait complete. 2024-04-16T17:04:55.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.645+0000 7f0096617740 20 client.25667 trim_cache size 0 max 16384 2024-04-16T17:04:55.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.668+0000 7f0096617740 1 Processor -- start 2024-04-16T17:04:55.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.668+0000 7f0096617740 1 -- start start 2024-04-16T17:04:55.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.668+0000 7f0096617740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.668+0000 7f0096617740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a5970 0x56355669e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.668+0000 7f0096617740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566b6ec0 0x5635566cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.668+0000 7f0096617740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1afd10 con 0x5635566a5970 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.668+0000 7f0096617740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1ad5e0 con 0x5635566b6ec0 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.668+0000 7f0096617740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1ab450 con 0x5635566a4fc0 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f008be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a5970 0x56355669e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51394/0 (socket says 172.21.15.18:51394) 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091000640 1 -- 172.21.15.18:0/1393231373 learned_addr learned my addr 172.21.15.18:0/1393231373 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091a00640 1 --2- 172.21.15.18:0/1393231373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566b6ec0 0x5635566cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091000640 1 -- 172.21.15.18:0/1393231373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566b6ec0 msgr2=0x5635566cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091000640 1 --2- 172.21.15.18:0/1393231373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566b6ec0 0x5635566cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091000640 1 -- 172.21.15.18:0/1393231373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a5970 msgr2=0x56355669e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091000640 1 --2- 172.21.15.18:0/1393231373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a5970 0x56355669e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.669+0000 7f0091000640 1 -- 172.21.15.18:0/1393231373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635566b9e10 con 0x5635566a4fc0 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.670+0000 7f0091000640 1 --2- 172.21.15.18:0/1393231373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566b66c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f008009ae80 tx=0x7f008009b0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=977b5ac7e9214ba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.670+0000 7f007d600640 1 -- 172.21.15.18:0/1393231373 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008009d2d0 con 0x5635566a4fc0 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.670+0000 7f007d600640 1 -- 172.21.15.18:0/1393231373 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00800a5840 con 0x5635566a4fc0 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.670+0000 7f007d600640 1 -- 172.21.15.18:0/1393231373 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800d4af0 con 0x5635566a4fc0 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 -- 172.21.15.18:0/1393231373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 msgr2=0x5635566b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 --2- 172.21.15.18:0/1393231373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566b66c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f008009ae80 tx=0x7f008009b0e0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 -- 172.21.15.18:0/1393231373 shutdown_connections 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 --2- 172.21.15.18:0/1393231373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566b6ec0 0x5635566cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 --2- 172.21.15.18:0/1393231373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566a5970 0x56355669e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 --2- 172.21.15.18:0/1393231373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566b66c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 -- 172.21.15.18:0/1393231373 >> 172.21.15.18:0/1393231373 conn(0x56355669f8e0 msgr2=0x56355669e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 -- 172.21.15.18:0/1393231373 shutdown_connections 2024-04-16T17:04:55.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 -- 172.21.15.18:0/1393231373 wait complete. 2024-04-16T17:04:55.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.671+0000 7f0096617740 1 -- start start 2024-04-16T17:04:55.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0096617740 1 Processor -- start 2024-04-16T17:04:55.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0096617740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0096617740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0096617740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x56355672c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0096617740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1abaa0 con 0x5635566b6ec0 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0096617740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1acf10 con 0x5635566a5970 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0096617740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1ac170 con 0x5635566a4fc0 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0096617740 10 client.0 _ll_register_callbacks cb 0x7f00800a0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0091000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0091000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51408/0 (socket says 172.21.15.18:51408) 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.672+0000 7f0091000640 1 -- 172.21.15.18:0/733368064 learned_addr learned my addr 172.21.15.18:0/733368064 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0091a00640 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x56355672c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f008be00640 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0091000640 1 -- 172.21.15.18:0/733368064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 msgr2=0x5635566ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0091000640 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0091000640 1 -- 172.21.15.18:0/733368064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 msgr2=0x56355672c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0091000640 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x56355672c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0091000640 1 -- 172.21.15.18:0/733368064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635566cb0d0 con 0x5635566a4fc0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0091000640 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566ca300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f008000e9c0 tx=0x7f0080009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85c5a1a6cd15bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0072a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.673+0000 7f0072a00640 1 -- 172.21.15.18:0/733368064 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008009d2d0 con 0x5635566a4fc0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.674+0000 7f0096617740 1 -- 172.21.15.18:0/733368064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00800a5840 con 0x5635566a4fc0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.674+0000 7f0072a00640 1 -- 172.21.15.18:0/733368064 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f008009ce30 con 0x5635566a4fc0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.674+0000 7f0072a00640 1 -- 172.21.15.18:0/733368064 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00800b60d0 con 0x5635566a4fc0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.674+0000 7f0096617740 10 client.25660 fetch_fsmap learned FSMap version 25 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.674+0000 7f0096617740 1 -- 172.21.15.18:0/733368064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635566cb0d0 con 0x5635566a4fc0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.674+0000 7f0072a00640 1 -- 172.21.15.18:0/733368064 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00800a5840 con 0x5635566a4fc0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.674+0000 7f0072a00640 1 -- 172.21.15.18:0/733368064 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f00800ce730 con 0x5635566a4fc0 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f0096617740 10 client.25660 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f0096617740 10 client.25660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f0096617740 20 client.25660 populate_metadata read hostname 'smithi018' 2024-04-16T17:04:55.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f0096617740 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56355672dd50 0x563556730140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f0096617740 4 client.25660 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:04:55.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f0096617740 1 -- 172.21.15.18:0/733368064 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635566129b0 con 0x56355672dd50 2024-04-16T17:04:55.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f008be00640 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56355672dd50 0x563556730140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f008be00640 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56355672dd50 0x563556730140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b64e423cbea2539 server_cookie=bc8817eb23973b27 in_seq=0 out_seq=0 2024-04-16T17:04:55.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.675+0000 7f0072a00640 10 client.25660 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0072a00640 1 -- 172.21.15.18:0/733368064 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5635566129b0 con 0x56355672dd50 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0072a00640 10 client.25660 handle_command_reply: tid=0 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 client.25660 shutdown 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 -- 172.21.15.18:0/733368064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 msgr2=0x5635566ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566ca300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f008000e9c0 tx=0x7f0080009590 comp rx=0 tx=0).stop 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 -- 172.21.15.18:0/733368064 shutdown_connections 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56355672dd50 0x563556730140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5635566b6ec0 0x56355672c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635566a5970 0x5635566ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 --2- 172.21.15.18:0/733368064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635566a4fc0 0x5635566ca300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0072a00640 0 client.25660 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 -- 172.21.15.18:0/733368064 >> 172.21.15.18:0/733368064 conn(0x56355669f8e0 msgr2=0x5635566a1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 -- 172.21.15.18:0/733368064 shutdown_connections 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 1 -- 172.21.15.18:0/733368064 wait complete. 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.676+0000 7f0096617740 20 client.25660 trim_cache size 0 max 16384 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stdout: "893ae7de-09ee-4870-8693-9bae5f5ab2bb": { 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "893ae7de-09ee-4870-8693-9bae5f5ab2bb", 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:04:55.681 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:04:55.682 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:04:55.682 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:04:55.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 -- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00740a3e40 msgr2=0x7f00740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00740a3e40 0x7f00740a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00840099a0 tx=0x7f0084009500 comp rx=0 tx=0).stop 2024-04-16T17:04:55.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 -- 172.21.15.18:0/2744960106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f008c104440 msgr2=0x7f008c1a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f008c104440 0x7f008c1a0850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f008000e9f0 tx=0x7f008000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 -- 172.21.15.18:0/2744960106 shutdown_connections 2024-04-16T17:04:55.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00740a3e40 0x7f00740a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f008c19aa00 0x7f008c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f008c109510 0x7f008c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 --2- 172.21.15.18:0/2744960106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f008c104440 0x7f008c1a0850 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 -- 172.21.15.18:0/2744960106 >> 172.21.15.18:0/2744960106 conn(0x7f008c0ffc80 msgr2=0x7f008c101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 -- 172.21.15.18:0/2744960106 shutdown_connections 2024-04-16T17:04:55.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.677+0000 7f0093a00640 1 -- 172.21.15.18:0/2744960106 wait complete. 2024-04-16T17:04:55.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.692+0000 7f4a1a000640 1 -- 172.21.15.18:0/1240075377 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4a1c102a40 con 0x7f4a040a3e20 2024-04-16T17:04:55.693 INFO:teuthology.orchestra.run.smithi018.stdout:Schedule set for path / 2024-04-16T17:04:55.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:55 smithi018 ceph-mon[31776]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-16T17:04:55.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1967793510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:55.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2136938999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:55.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4a040a3e20 msgr2=0x7f4a040a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4a040a3e20 0x7f4a040a62e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009cd0 tx=0x7f4a10005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:55.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a1c19a720 msgr2=0x7f4a1c19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a1c19a720 0x7f4a1c19aba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a1402c700 tx=0x7f4a14032f20 comp rx=0 tx=0).stop 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 shutdown_connections 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4a040a3e20 0x7f4a040a62e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a1c19a720 0x7f4a1c19aba0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a1c107160 0x7f4a1c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 --2- 172.21.15.18:0/1240075377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a1c102090 0x7f4a1c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 >> 172.21.15.18:0/1240075377 conn(0x7f4a1c0fd910 msgr2=0x7f4a1c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 shutdown_connections 2024-04-16T17:04:55.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.695+0000 7f4a22800640 1 -- 172.21.15.18:0/1240075377 wait complete. 2024-04-16T17:04:55.698 INFO:tasks.cephfs.filesystem:scrub status for tag:893ae7de-09ee-4870-8693-9bae5f5ab2bb - {'path': '/', 'tag': '893ae7de-09ee-4870-8693-9bae5f5ab2bb', 'options': 'recursive,force'} 2024-04-16T17:04:55.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-16T17:04:55.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:55 smithi043 ceph-mon[36901]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-16T17:04:55.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1967793510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:55.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2136938999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:55.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:55 smithi022 ceph-mon[36023]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-16T17:04:55.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1967793510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:04:55.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2136938999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 -- 172.21.15.18:0/1098522563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634107140 msgr2=0x7fc63410ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 --2- 172.21.15.18:0/1098522563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634107140 0x7fc63410ed10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009ca0 tx=0x7fc62c02c100 comp rx=0 tx=0).stop 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 -- 172.21.15.18:0/1098522563 shutdown_connections 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 --2- 172.21.15.18:0/1098522563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634107140 0x7fc63410ed10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 --2- 172.21.15.18:0/1098522563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc634102a20 0x7fc634106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 --2- 172.21.15.18:0/1098522563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc634102070 0x7fc634102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 -- 172.21.15.18:0/1098522563 >> 172.21.15.18:0/1098522563 conn(0x7fc6340fd930 msgr2=0x7fc6340ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 -- 172.21.15.18:0/1098522563 shutdown_connections 2024-04-16T17:04:55.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.965+0000 7fc639e00640 1 -- 172.21.15.18:0/1098522563 wait complete. 2024-04-16T17:04:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.967+0000 7fc639e00640 1 Processor -- start 2024-04-16T17:04:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.967+0000 7fc639e00640 1 -- start start 2024-04-16T17:04:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.967+0000 7fc639e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102070 0x7fc634074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.967+0000 7fc639e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc634107140 0x7fc634074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.967+0000 7fc639e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc634074d80 0x7fc6341a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.967+0000 7fc639e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc634112200 con 0x7fc634102070 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.967+0000 7fc639e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc634112080 con 0x7fc634107140 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.967+0000 7fc639e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc634112380 con 0x7fc634074d80 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc638a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102070 0x7fc634074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc639400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc634074d80 0x7fc6341a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc639400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc634074d80 0x7fc6341a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51418/0 (socket says 172.21.15.18:51418) 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc639400640 1 -- 172.21.15.18:0/3003306057 learned_addr learned my addr 172.21.15.18:0/3003306057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc633e00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc634107140 0x7fc634074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc639400640 1 -- 172.21.15.18:0/3003306057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc634107140 msgr2=0x7fc634074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc639400640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc634107140 0x7fc634074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc639400640 1 -- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102070 msgr2=0x7fc634074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc639400640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102070 0x7fc634074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.968+0000 7fc639400640 1 -- 172.21.15.18:0/3003306057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6341a6ff0 con 0x7fc634074d80 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.969+0000 7fc639400640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc634074d80 0x7fc6341a68f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02c710 tx=0x7fc62c035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.969+0000 7fc631600640 1 -- 172.21.15.18:0/3003306057 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c039040 con 0x7fc634074d80 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.969+0000 7fc631600640 1 -- 172.21.15.18:0/3003306057 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c044070 con 0x7fc634074d80 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.969+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6341a7280 con 0x7fc634074d80 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.969+0000 7fc631600640 1 -- 172.21.15.18:0/3003306057 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c004330 con 0x7fc634074d80 2024-04-16T17:04:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.969+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341a7770 con 0x7fc634074d80 2024-04-16T17:04:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.970+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6340688a0 con 0x7fc634074d80 2024-04-16T17:04:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.972+0000 7fc631600640 1 -- 172.21.15.18:0/3003306057 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc62c0044d0 con 0x7fc634074d80 2024-04-16T17:04:55.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.974+0000 7fc631600640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc61c0a3e50 0x7fc61c0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:55.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.974+0000 7fc631600640 1 -- 172.21.15.18:0/3003306057 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc62c0f39b0 con 0x7fc634074d80 2024-04-16T17:04:55.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.974+0000 7fc638a00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc61c0a3e50 0x7fc61c0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:55.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.975+0000 7fc638a00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc61c0a3e50 0x7fc61c0a6310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc628009bd0 tx=0x7fc628005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:55.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:55.976+0000 7fc631600640 1 -- 172.21.15.18:0/3003306057 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc62c0bfb00 con 0x7fc634074d80 2024-04-16T17:04:56.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.212+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fc63410a440 con 0x7fc61c0a3e50 2024-04-16T17:04:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.351+0000 7fc631600640 1 -- 172.21.15.18:0/3003306057 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fc628004030 con 0x7fc61c0a3e50 2024-04-16T17:04:56.352 INFO:teuthology.orchestra.run.smithi018.stdout:Retention added to path / 2024-04-16T17:04:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.354+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc61c0a3e50 msgr2=0x7fc61c0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.354+0000 7fc639e00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc61c0a3e50 0x7fc61c0a6310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc628009bd0 tx=0x7fc628005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.354+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc634074d80 msgr2=0x7fc6341a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.354+0000 7fc639e00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc634074d80 0x7fc6341a68f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02c710 tx=0x7fc62c035d90 comp rx=0 tx=0).stop 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.355+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 shutdown_connections 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.355+0000 7fc639e00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc61c0a3e50 0x7fc61c0a6310 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.355+0000 7fc639e00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc634074d80 0x7fc6341a68f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.355+0000 7fc639e00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc634107140 0x7fc634074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.355+0000 7fc639e00640 1 --2- 172.21.15.18:0/3003306057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102070 0x7fc634074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.355+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 >> 172.21.15.18:0/3003306057 conn(0x7fc6340fd930 msgr2=0x7fc6340ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.355+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 shutdown_connections 2024-04-16T17:04:56.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.355+0000 7fc639e00640 1 -- 172.21.15.18:0/3003306057 wait complete. 2024-04-16T17:04:56.378 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-16T17:04:56.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 -- 172.21.15.18:0/485999545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5881093a0 msgr2=0x7ff588110f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:56.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 --2- 172.21.15.18:0/485999545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5881093a0 0x7ff588110f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff580009ca0 tx=0x7ff58002c100 comp rx=0 tx=0).stop 2024-04-16T17:04:56.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 -- 172.21.15.18:0/485999545 shutdown_connections 2024-04-16T17:04:56.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 --2- 172.21.15.18:0/485999545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5881093a0 0x7ff588110f70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 --2- 172.21.15.18:0/485999545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff588104c80 0x7ff588108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 --2- 172.21.15.18:0/485999545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5881042d0 0x7ff5881046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 -- 172.21.15.18:0/485999545 >> 172.21.15.18:0/485999545 conn(0x7ff5880ffb10 msgr2=0x7ff588101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:56.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 -- 172.21.15.18:0/485999545 shutdown_connections 2024-04-16T17:04:56.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.587+0000 7ff58ce00640 1 -- 172.21.15.18:0/485999545 wait complete. 2024-04-16T17:04:56.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.588+0000 7ff58ce00640 1 Processor -- start 2024-04-16T17:04:56.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.588+0000 7ff58ce00640 1 -- start start 2024-04-16T17:04:56.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.588+0000 7ff58ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5881042d0 0x7ff5881a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:56.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.588+0000 7ff58ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5881093a0 0x7ff5881a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.588+0000 7ff58ce00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff58819a6e0 0x7ff58819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.588+0000 7ff58ce00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff588114440 con 0x7ff58819a6e0 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.588+0000 7ff58ce00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881142c0 con 0x7ff5881093a0 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.588+0000 7ff58ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881145c0 con 0x7ff5881042d0 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff58819a6e0 0x7ff58819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5881042d0 0x7ff5881a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff58819a6e0 0x7ff58819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55838/0 (socket says 172.21.15.18:55838) 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff586a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5881093a0 0x7ff5881a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587e00640 1 -- 172.21.15.18:0/2642073464 learned_addr learned my addr 172.21.15.18:0/2642073464 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587e00640 1 -- 172.21.15.18:0/2642073464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5881042d0 msgr2=0x7ff5881a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587e00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5881042d0 0x7ff5881a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587e00640 1 -- 172.21.15.18:0/2642073464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5881093a0 msgr2=0x7ff5881a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:56.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587e00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5881093a0 0x7ff5881a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.589+0000 7ff587e00640 1 -- 172.21.15.18:0/2642073464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58819b2e0 con 0x7ff58819a6e0 2024-04-16T17:04:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.590+0000 7ff587e00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff58819a6e0 0x7ff58819ab60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff580037040 tx=0x7ff580032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.590+0000 7ff57fe00640 1 -- 172.21.15.18:0/2642073464 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580039040 con 0x7ff58819a6e0 2024-04-16T17:04:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.590+0000 7ff57fe00640 1 -- 172.21.15.18:0/2642073464 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff580044070 con 0x7ff58819a6e0 2024-04-16T17:04:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.590+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58819b570 con 0x7ff58819a6e0 2024-04-16T17:04:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.590+0000 7ff57fe00640 1 -- 172.21.15.18:0/2642073464 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5800042a0 con 0x7ff58819a6e0 2024-04-16T17:04:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.590+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5881a7700 con 0x7ff58819a6e0 2024-04-16T17:04:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.591+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5881057a0 con 0x7ff58819a6e0 2024-04-16T17:04:56.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.593+0000 7ff57fe00640 1 -- 172.21.15.18:0/2642073464 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff580003970 con 0x7ff58819a6e0 2024-04-16T17:04:56.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.594+0000 7ff57fe00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff56c0a3ea0 0x7ff56c0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:56.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.595+0000 7ff57fe00640 1 -- 172.21.15.18:0/2642073464 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff5800f3130 con 0x7ff58819a6e0 2024-04-16T17:04:56.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.595+0000 7ff587400640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff56c0a3ea0 0x7ff56c0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:56.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.596+0000 7ff587400640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff56c0a3ea0 0x7ff56c0a6360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff578009bd0 tx=0x7ff578005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:56.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.599+0000 7ff57fe00640 1 -- 172.21.15.18:0/2642073464 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5800bf310 con 0x7ff58819a6e0 2024-04-16T17:04:56.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:56 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:04:56.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:56 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:04:56.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:56 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:04:56.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:56 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:04:56.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:56 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:04:56.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:56 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:04:56.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:56 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:04:56.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:56 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:04:56.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.838+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff588104c80 con 0x7ff56c0a3ea0 2024-04-16T17:04:56.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.852+0000 7ff57fe00640 1 -- 172.21.15.18:0/2642073464 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7ff588104c80 con 0x7ff56c0a3ea0 2024-04-16T17:04:56.854 INFO:teuthology.orchestra.run.smithi018.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-16T00:00:00", "created": "2024-04-16T17:04:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-16T17:04:56.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff56c0a3ea0 msgr2=0x7ff56c0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:56.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff56c0a3ea0 0x7ff56c0a6360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff578009bd0 tx=0x7ff578005f90 comp rx=0 tx=0).stop 2024-04-16T17:04:56.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff58819a6e0 msgr2=0x7ff58819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:56.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff58819a6e0 0x7ff58819ab60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff580037040 tx=0x7ff580032f30 comp rx=0 tx=0).stop 2024-04-16T17:04:56.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 shutdown_connections 2024-04-16T17:04:56.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff56c0a3ea0 0x7ff56c0a6360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff58819a6e0 0x7ff58819ab60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5881093a0 0x7ff5881a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 --2- 172.21.15.18:0/2642073464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5881042d0 0x7ff5881a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:56.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 >> 172.21.15.18:0/2642073464 conn(0x7ff5880ffb10 msgr2=0x7ff588100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:56.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 shutdown_connections 2024-04-16T17:04:56.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:56.855+0000 7ff58ce00640 1 -- 172.21.15.18:0/2642073464 wait complete. 2024-04-16T17:04:56.873 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-16T17:04:56.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:56 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:04:56.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:56 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:04:56.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:56 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:04:56.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:56 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 -- 172.21.15.18:0/3615508481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc074cc0 msgr2=0x7f52fc112ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 --2- 172.21.15.18:0/3615508481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc074cc0 0x7f52fc112ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009900 tx=0x7f52ec02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 -- 172.21.15.18:0/3615508481 shutdown_connections 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 --2- 172.21.15.18:0/3615508481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52fc113000 0x7f52fc1153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 --2- 172.21.15.18:0/3615508481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc074cc0 0x7f52fc112ac0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 --2- 172.21.15.18:0/3615508481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52fc0743a0 0x7f52fc074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 -- 172.21.15.18:0/3615508481 >> 172.21.15.18:0/3615508481 conn(0x7f52fc0fdac0 msgr2=0x7f52fc0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 -- 172.21.15.18:0/3615508481 shutdown_connections 2024-04-16T17:04:57.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.086+0000 7f5304800640 1 -- 172.21.15.18:0/3615508481 wait complete. 2024-04-16T17:04:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.087+0000 7f5304800640 1 Processor -- start 2024-04-16T17:04:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5304800640 1 -- start start 2024-04-16T17:04:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5304800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52fc0743a0 0x7f52fc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5304800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52fc074cc0 0x7f52fc19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5304800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc113000 0x7f52fc1a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5304800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc118a00 con 0x7f52fc074cc0 2024-04-16T17:04:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5304800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc118880 con 0x7f52fc0743a0 2024-04-16T17:04:57.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5304800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc118b80 con 0x7f52fc113000 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5301400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52fc074cc0 0x7f52fc19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5301400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52fc074cc0 0x7f52fc19f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57964/0 (socket says 172.21.15.18:57964) 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5301400640 1 -- 172.21.15.18:0/3292667207 learned_addr learned my addr 172.21.15.18:0/3292667207 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.088+0000 7f5301e00640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52fc0743a0 0x7f52fc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5302800640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc113000 0x7f52fc1a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5301e00640 1 -- 172.21.15.18:0/3292667207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc113000 msgr2=0x7f52fc1a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5301e00640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc113000 0x7f52fc1a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5301e00640 1 -- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52fc074cc0 msgr2=0x7f52fc19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5301e00640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52fc074cc0 0x7f52fc19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5301e00640 1 -- 172.21.15.18:0/3292667207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc1a3f90 con 0x7f52fc0743a0 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5301400640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52fc074cc0 0x7f52fc19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5302800640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc113000 0x7f52fc1a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:57.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.089+0000 7f5301e00640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52fc0743a0 0x7f52fc19efa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f52f000d6d0 tx=0x7f52f000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.090+0000 7f52f6a00640 1 -- 172.21.15.18:0/3292667207 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0017070 con 0x7f52fc0743a0 2024-04-16T17:04:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.090+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc1a4280 con 0x7f52fc0743a0 2024-04-16T17:04:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.090+0000 7f52f6a00640 1 -- 172.21.15.18:0/3292667207 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f0009820 con 0x7f52fc0743a0 2024-04-16T17:04:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.090+0000 7f52f6a00640 1 -- 172.21.15.18:0/3292667207 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0004cd0 con 0x7f52fc0743a0 2024-04-16T17:04:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.090+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc1abb10 con 0x7f52fc0743a0 2024-04-16T17:04:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.091+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52fc0505e0 con 0x7f52fc0743a0 2024-04-16T17:04:57.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.093+0000 7f52f6a00640 1 -- 172.21.15.18:0/3292667207 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f52f00099d0 con 0x7f52fc0743a0 2024-04-16T17:04:57.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.094+0000 7f52f6a00640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f52e40a3e30 0x7f52e40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:57.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.095+0000 7f52f6a00640 1 -- 172.21.15.18:0/3292667207 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f52f00d09a0 con 0x7f52fc0743a0 2024-04-16T17:04:57.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.095+0000 7f5301400640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f52e40a3e30 0x7f52e40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:57.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.095+0000 7f5301400640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f52e40a3e30 0x7f52e40a62f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0096c0 tx=0x7f52ec02df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:57.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.099+0000 7f52f6a00640 1 -- 172.21.15.18:0/3292667207 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52f009c390 con 0x7f52fc0743a0 2024-04-16T17:04:57.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.332+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f52fc1054c0 con 0x7f52e40a3e30 2024-04-16T17:04:57.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.345+0000 7f52f6a00640 1 -- 172.21.15.18:0/3292667207 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f52ec004af0 con 0x7f52e40a3e30 2024-04-16T17:04:57.346 INFO:teuthology.orchestra.run.smithi018.stdout:/ 1m 6m3h 2024-04-16T17:04:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.348+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f52e40a3e30 msgr2=0x7f52e40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.348+0000 7f5304800640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f52e40a3e30 0x7f52e40a62f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0096c0 tx=0x7f52ec02df90 comp rx=0 tx=0).stop 2024-04-16T17:04:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.348+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52fc0743a0 msgr2=0x7f52fc19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.348+0000 7f5304800640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52fc0743a0 0x7f52fc19efa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f52f000d6d0 tx=0x7f52f000db90 comp rx=0 tx=0).stop 2024-04-16T17:04:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.348+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 shutdown_connections 2024-04-16T17:04:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.348+0000 7f5304800640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f52e40a3e30 0x7f52e40a62f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.348+0000 7f5304800640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc113000 0x7f52fc1a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.349+0000 7f5304800640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52fc074cc0 0x7f52fc19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.349+0000 7f5304800640 1 --2- 172.21.15.18:0/3292667207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52fc0743a0 0x7f52fc19efa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.349+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 >> 172.21.15.18:0/3292667207 conn(0x7f52fc0fdac0 msgr2=0x7f52fc0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.349+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 shutdown_connections 2024-04-16T17:04:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:04:57.349+0000 7f5304800640 1 -- 172.21.15.18:0/3292667207 wait complete. 2024-04-16T17:04:57.371 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-16T17:04:57.456 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-16T17:04:57.466 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-16T17:04:57.475 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:04:57.475 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-16T17:04:57.481 INFO:teuthology.run_tasks:Running task workunit... 2024-04-16T17:04:57.494 INFO:tasks.workunit:Pulling workunits from ref b137e4a675398e295567f9652210865a4d573fac 2024-04-16T17:04:57.494 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-16T17:04:57.494 INFO:tasks.workunit:timeout=3h 2024-04-16T17:04:57.494 INFO:tasks.workunit:cleanup=True 2024-04-16T17:04:57.495 DEBUG:teuthology.orchestra.run.smithi018:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-16T17:04:57.540 INFO:teuthology.orchestra.run.smithi018.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-16T17:04:57.540 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-16T17:04:57.540 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-04-16T17:04:57.541 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:04:57.541 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-16T17:04:57.541 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-16 17:02:01.246196139 +0000 2024-04-16T17:04:57.541 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-16 17:02:01.246196139 +0000 2024-04-16T17:04:57.541 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-16 17:04:42.350412662 +0000 2024-04-16T17:04:57.541 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-16 17:02:01.246196139 +0000 2024-04-16T17:04:57.541 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-16T17:04:57.541 DEBUG:teuthology.orchestra.run.smithi018:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-16T17:04:57.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:57 smithi018 ceph-mon[31776]: from='client.25627 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:57.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:57 smithi018 ceph-mon[31776]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 653 B/s wr, 17 op/s 2024-04-16T17:04:57.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:57 smithi018 ceph-mon[31776]: from='client.25666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:57.673 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/mchangir/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b137e4a675398e295567f9652210865a4d573fac 2024-04-16T17:04:57.721 INFO:tasks.workunit.client.0.smithi018.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-16T17:04:57.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:57 smithi043 ceph-mon[36901]: from='client.25627 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:57.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:57 smithi043 ceph-mon[36901]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 653 B/s wr, 17 op/s 2024-04-16T17:04:57.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:57 smithi043 ceph-mon[36901]: from='client.25666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:57 smithi022 ceph-mon[36023]: from='client.25627 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:57 smithi022 ceph-mon[36023]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 653 B/s wr, 17 op/s 2024-04-16T17:04:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:57 smithi022 ceph-mon[36023]: from='client.25666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:58.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:58 smithi018 ceph-mon[31776]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:58.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:58 smithi018 ceph-mon[31776]: from='client.25691 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:58.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:58 smithi043 ceph-mon[36901]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:58.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:58 smithi043 ceph-mon[36901]: from='client.25691 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:58.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:58 smithi022 ceph-mon[36023]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:58.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:58 smithi022 ceph-mon[36023]: from='client.25691 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:04:59.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:04:59 smithi043 ceph-mon[36901]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-04-16T17:04:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:04:59 smithi022 ceph-mon[36023]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-04-16T17:04:59.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:04:59 smithi018 ceph-mon[31776]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-04-16T17:05:00.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:05:00 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:05:00.371+0000 7f3493c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:05:00.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:05:00 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:05:00.371+0000 7f3493c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:05:00.696 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:05:00 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:05:00.371+0000 7f3493c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:05:00.697 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:05:00 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:05:00.371+0000 7f3493c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:05:00.697 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:05:00 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:05:00.371+0000 7f3493c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:05:01.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:01 smithi043 ceph-mon[36901]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-04-16T17:05:01.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:01 smithi022 ceph-mon[36023]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-04-16T17:05:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:01 smithi018 ceph-mon[31776]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-04-16T17:05:03.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:03 smithi043 ceph-mon[36901]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-16T17:05:03.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:03 smithi043 ceph-mon[36901]: mgrmap e25: x(active, since 18s), standbys: z, y 2024-04-16T17:05:03.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:03 smithi022 ceph-mon[36023]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-16T17:05:03.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:03 smithi022 ceph-mon[36023]: mgrmap e25: x(active, since 18s), standbys: z, y 2024-04-16T17:05:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:03 smithi018 ceph-mon[31776]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-16T17:05:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:03 smithi018 ceph-mon[31776]: mgrmap e25: x(active, since 18s), standbys: z, y 2024-04-16T17:05:05.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:05 smithi022 ceph-mon[36023]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-16T17:05:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:05 smithi018 ceph-mon[31776]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-16T17:05:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:05 smithi043 ceph-mon[36901]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-16T17:05:07.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:07 smithi018 ceph-mon[31776]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-04-16T17:05:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:07 smithi022 ceph-mon[36023]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-04-16T17:05:08.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:07 smithi043 ceph-mon[36901]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-04-16T17:05:09.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:09 smithi043 ceph-mon[36901]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-04-16T17:05:09.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:09 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:05:09.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:09 smithi022 ceph-mon[36023]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-04-16T17:05:09.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:09 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:05:09.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:09 smithi018 ceph-mon[31776]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-04-16T17:05:09.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:09 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:05:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 -- 172.21.15.18:0/2611694314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c00102080 msgr2=0x7f2c00102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2611694314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c00102080 0x7f2c00102460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009ca0 tx=0x7f2bf002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 -- 172.21.15.18:0/2611694314 shutdown_connections 2024-04-16T17:05:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2611694314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c00107150 0x7f2c0010ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2611694314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c00102a30 0x7f2c00106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2611694314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c00102080 0x7f2c00102460 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 -- 172.21.15.18:0/2611694314 >> 172.21.15.18:0/2611694314 conn(0x7f2c000fd940 msgr2=0x7f2c000ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 -- 172.21.15.18:0/2611694314 shutdown_connections 2024-04-16T17:05:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.717+0000 7f2bffe00640 1 -- 172.21.15.18:0/2611694314 wait complete. 2024-04-16T17:05:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bffe00640 1 Processor -- start 2024-04-16T17:05:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bffe00640 1 -- start start 2024-04-16T17:05:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c00102080 0x7f2c001a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bffe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c00107150 0x7f2c001a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0019a7e0 0x7f2c0019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bffe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001121e0 con 0x7f2c00107150 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bffe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00112060 con 0x7f2c00102080 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bffe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00112360 con 0x7f2c0019a7e0 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c00107150 0x7f2c001a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0019a7e0 0x7f2c0019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.719+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c00102080 0x7f2c001a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0019a7e0 0x7f2c0019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40782/0 (socket says 172.21.15.18:40782) 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c00107150 0x7f2c001a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38338/0 (socket says 172.21.15.18:38338) 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bff400640 1 -- 172.21.15.18:0/2204245442 learned_addr learned my addr 172.21.15.18:0/2204245442 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c00102080 0x7f2c001a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50178/0 (socket says 172.21.15.18:50178) 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bff400640 1 -- 172.21.15.18:0/2204245442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c00102080 msgr2=0x7f2c001a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bff400640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c00102080 0x7f2c001a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bff400640 1 -- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c00107150 msgr2=0x7f2c001a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bff400640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c00107150 0x7f2c001a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bff400640 1 -- 172.21.15.18:0/2204245442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0019b370 con 0x7f2c0019a7e0 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bfe000640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c00107150 0x7f2c001a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.720+0000 7f2bff400640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0019a7e0 0x7f2c0019ac90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800d950 tx=0x7f2bf800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.721+0000 7f2bf7400640 1 -- 172.21.15.18:0/2204245442 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8017030 con 0x7f2c0019a7e0 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.721+0000 7f2bf7400640 1 -- 172.21.15.18:0/2204245442 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8004020 con 0x7f2c0019a7e0 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.721+0000 7f2bffe00640 1 -- 172.21.15.18:0/2204245442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0019b660 con 0x7f2c0019a7e0 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.721+0000 7f2bf7400640 1 -- 172.21.15.18:0/2204245442 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8003c00 con 0x7f2c0019a7e0 2024-04-16T17:05:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.721+0000 7f2bffe00640 1 -- 172.21.15.18:0/2204245442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001a77f0 con 0x7f2c0019a7e0 2024-04-16T17:05:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.724+0000 7f2c0824f740 1 Processor -- start 2024-04-16T17:05:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.726+0000 7f2bf7400640 1 -- 172.21.15.18:0/2204245442 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f2bf801a070 con 0x7f2c0019a7e0 2024-04-16T17:05:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.726+0000 7f2c0824f740 1 -- start start 2024-04-16T17:05:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.726+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a71a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.726+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a71af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.726+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a724000 0x55778a7263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.726+0000 7f2c0824f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55778a67f780 con 0x55778a724000 2024-04-16T17:05:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.726+0000 7f2c0824f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55778a625f40 con 0x55778a71aa80 2024-04-16T17:05:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.726+0000 7f2c0824f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55778a67b740 con 0x55778a71b7f0 2024-04-16T17:05:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a71a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a71a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40798/0 (socket says 172.21.15.18:40798) 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfe000640 1 -- 172.21.15.18:0/1272209296 learned_addr learned my addr 172.21.15.18:0/1272209296 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfea00640 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a71af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfe000640 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a724000 0x55778a7263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfea00640 1 -- 172.21.15.18:0/1272209296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 msgr2=0x55778a71a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfea00640 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a71a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfea00640 1 -- 172.21.15.18:0/1272209296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a724000 msgr2=0x55778a7263f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfea00640 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a724000 0x55778a7263f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.727+0000 7f2bfea00640 1 -- 172.21.15.18:0/1272209296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778a392210 con 0x55778a71aa80 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.728+0000 7f2bf7400640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2be40a3f30 0x7f2be40a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.728+0000 7f2bfea00640 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a71af00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2bf002c630 tx=0x7f2bf0005fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c7643ff3f3fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.728+0000 7f2bf7400640 1 -- 172.21.15.18:0/2204245442 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2bf80d0570 con 0x7f2c0019a7e0 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.728+0000 7f2bfe000640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2be40a3f30 0x7f2be40a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.728+0000 7f2bef400640 1 -- 172.21.15.18:0/1272209296 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0043070 con 0x55778a71aa80 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.728+0000 7f2bef400640 1 -- 172.21.15.18:0/1272209296 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0032c90 con 0x55778a71aa80 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.728+0000 7f2bef400640 1 -- 172.21.15.18:0/1272209296 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0039040 con 0x55778a71aa80 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 -- 172.21.15.18:0/1272209296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 msgr2=0x55778a71af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2bfe000640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2be40a3f30 0x7f2be40a63f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55778a716900 tx=0x7f2be8005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a71af00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2bf002c630 tx=0x7f2bf0005fc0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 -- 172.21.15.18:0/1272209296 shutdown_connections 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a724000 0x55778a7263f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a71af00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 --2- 172.21.15.18:0/1272209296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a71a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 -- 172.21.15.18:0/1272209296 >> 172.21.15.18:0/1272209296 conn(0x55778a6b55d0 msgr2=0x55778a715770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 -- 172.21.15.18:0/1272209296 shutdown_connections 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 -- 172.21.15.18:0/1272209296 wait complete. 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.729+0000 7f2c0824f740 1 -- start start 2024-04-16T17:05:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2c0824f740 1 Processor -- start 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 0x55778a720ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a7213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a721920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2c0824f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55778a726cd0 con 0x55778a71aa80 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2c0824f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55778a639390 con 0x55778a724000 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2c0824f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55778a726e50 con 0x55778a71b7f0 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2c0824f740 10 client.0 _ll_register_callbacks cb 0x55778a622f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a7213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.730+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 0x55778a720ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bff400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a721920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a7213e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40802/0 (socket says 172.21.15.18:40802) 2024-04-16T17:05:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bfe000640 1 -- 172.21.15.18:0/484407440 learned_addr learned my addr 172.21.15.18:0/484407440 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bff400640 1 -- 172.21.15.18:0/484407440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 msgr2=0x55778a7213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bff400640 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a7213e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bff400640 1 -- 172.21.15.18:0/484407440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 msgr2=0x55778a720ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bff400640 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 0x55778a720ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bff400640 1 -- 172.21.15.18:0/484407440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778a45e620 con 0x55778a724000 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2bff400640 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a721920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80cd710 tx=0x7f2bf8005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61f8351fd4f02657 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2be3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2be3e00640 1 -- 172.21.15.18:0/484407440 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80a4bd0 con 0x55778a724000 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2be3e00640 1 -- 172.21.15.18:0/484407440 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2bf80985d0 con 0x55778a724000 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.731+0000 7f2c0824f740 1 -- 172.21.15.18:0/484407440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf0032c90 con 0x55778a724000 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2be3e00640 1 -- 172.21.15.18:0/484407440 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bf80041d0 con 0x55778a724000 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2c0824f740 10 client.25706 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2c0824f740 1 -- 172.21.15.18:0/484407440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55778a732e80 con 0x55778a724000 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2be3e00640 1 -- 172.21.15.18:0/484407440 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0032c90 con 0x55778a724000 2024-04-16T17:05:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2be3e00640 1 -- 172.21.15.18:0/484407440 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2bf80b5340 con 0x55778a724000 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2c0824f740 10 client.25706 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2c0824f740 10 client.25706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2c0824f740 20 client.25706 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2c0824f740 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55778a734340 0x55778a736790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2c0824f740 4 client.25706 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.732+0000 7f2c0824f740 1 -- 172.21.15.18:0/484407440 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55778a68d1a0 con 0x55778a734340 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.733+0000 7f2bfea00640 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55778a734340 0x55778a736790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.733+0000 7f2bfea00640 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55778a734340 0x55778a736790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991214bcffd91e30 server_cookie=9150a666c6792f60 in_seq=0 out_seq=0 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.733+0000 7f2be3e00640 10 client.25706 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2be3e00640 1 -- 172.21.15.18:0/484407440 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55778a68d1a0 con 0x55778a734340 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2be3e00640 10 client.25706 handle_command_reply: tid=0 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 client.25706 shutdown 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 -- 172.21.15.18:0/484407440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 msgr2=0x55778a721920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a721920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80cd710 tx=0x7f2bf8005810 comp rx=0 tx=0).stop 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 -- 172.21.15.18:0/484407440 shutdown_connections 2024-04-16T17:05:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55778a734340 0x55778a736790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a721920 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a7213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 --2- 172.21.15.18:0/484407440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 0x55778a720ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 -- 172.21.15.18:0/484407440 >> 172.21.15.18:0/484407440 conn(0x55778a6b55d0 msgr2=0x55778a7174c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2be3e00640 0 client.25706 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.734+0000 7f2c0824f740 1 -- 172.21.15.18:0/484407440 shutdown_connections 2024-04-16T17:05:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.735+0000 7f2c0824f740 1 -- 172.21.15.18:0/484407440 wait complete. 2024-04-16T17:05:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.735+0000 7f2c0824f740 20 client.25706 trim_cache size 0 max 16384 2024-04-16T17:05:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2c0824f740 1 Processor -- start 2024-04-16T17:05:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2c0824f740 1 -- start start 2024-04-16T17:05:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 0x55778a7235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a72fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a720000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2c0824f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001b3de0 con 0x55778a71aa80 2024-04-16T17:05:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2c0824f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001b16b0 con 0x55778a724000 2024-04-16T17:05:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2c0824f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001af500 con 0x55778a71b7f0 2024-04-16T17:05:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.751+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 0x55778a7235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a72fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bff400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a720000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a72fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40814/0 (socket says 172.21.15.18:40814) 2024-04-16T17:05:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 -- 172.21.15.18:0/421758745 learned_addr learned my addr 172.21.15.18:0/421758745 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 -- 172.21.15.18:0/421758745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 msgr2=0x55778a720000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 --2- 172.21.15.18:0/421758745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a720000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 -- 172.21.15.18:0/421758745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 msgr2=0x55778a7235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 --2- 172.21.15.18:0/421758745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 0x55778a7235d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 -- 172.21.15.18:0/421758745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778a719d50 con 0x55778a71b7f0 2024-04-16T17:05:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.752+0000 7f2bfe000640 1 --2- 172.21.15.18:0/421758745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a72fbd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2be8011960 tx=0x7f2be80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3e79b100572f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.753+0000 7f2beea00640 1 -- 172.21.15.18:0/421758745 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8020410 con 0x55778a71b7f0 2024-04-16T17:05:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.753+0000 7f2beea00640 1 -- 172.21.15.18:0/421758745 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be8030070 con 0x55778a71b7f0 2024-04-16T17:05:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.753+0000 7f2beea00640 1 -- 172.21.15.18:0/421758745 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be802ba30 con 0x55778a71b7f0 2024-04-16T17:05:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.753+0000 7f2c0824f740 1 -- 172.21.15.18:0/421758745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 msgr2=0x55778a72fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.753+0000 7f2c0824f740 1 --2- 172.21.15.18:0/421758745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a72fbd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2be8011960 tx=0x7f2be80046c0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 -- 172.21.15.18:0/421758745 shutdown_connections 2024-04-16T17:05:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 --2- 172.21.15.18:0/421758745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a724000 0x55778a720000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 --2- 172.21.15.18:0/421758745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a71b7f0 0x55778a72fbd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 --2- 172.21.15.18:0/421758745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71aa80 0x55778a7235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 -- 172.21.15.18:0/421758745 >> 172.21.15.18:0/421758745 conn(0x55778a6b55d0 msgr2=0x55778a720e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 -- 172.21.15.18:0/421758745 shutdown_connections 2024-04-16T17:05:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 -- 172.21.15.18:0/421758745 wait complete. 2024-04-16T17:05:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 -- start start 2024-04-16T17:05:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.754+0000 7f2c0824f740 1 Processor -- start 2024-04-16T17:05:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a724d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71b7f0 0x55778a725280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2c0824f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a724000 0x55778a7257e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2c0824f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001afb70 con 0x55778a71b7f0 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2c0824f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001b0fe0 con 0x55778a71aa80 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2c0824f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001b0240 con 0x55778a724000 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2c0824f740 10 client.0 _ll_register_callbacks cb 0x7f2bf8005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71b7f0 0x55778a725280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a724d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71b7f0 0x55778a725280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38364/0 (socket says 172.21.15.18:38364) 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfe000640 1 -- 172.21.15.18:0/3969075142 learned_addr learned my addr 172.21.15.18:0/3969075142 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a724d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50196/0 (socket says 172.21.15.18:50196) 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bff400640 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a724000 0x55778a7257e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfe000640 1 -- 172.21.15.18:0/3969075142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a724000 msgr2=0x55778a7257e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfe000640 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a724000 0x55778a7257e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfe000640 1 -- 172.21.15.18:0/3969075142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 msgr2=0x55778a724d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.755+0000 7f2bfe000640 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a724d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.756+0000 7f2bfe000640 1 -- 172.21.15.18:0/3969075142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778a72a830 con 0x55778a71b7f0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.756+0000 7f2bff400640 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a724000 0x55778a7257e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.756+0000 7f2bfea00640 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a724d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.756+0000 7f2bfe000640 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71b7f0 0x55778a725280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009df0 tx=0x7f2be8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578bfbbdf04a5f52 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.756+0000 7f2be3e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.756+0000 7f2be3e00640 1 -- 172.21.15.18:0/3969075142 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8038070 con 0x55778a71b7f0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.756+0000 7f2c0824f740 1 -- 172.21.15.18:0/3969075142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be8030070 con 0x55778a71b7f0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.756+0000 7f2be3e00640 1 -- 172.21.15.18:0/3969075142 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2be802b770 con 0x55778a71b7f0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2be3e00640 1 -- 172.21.15.18:0/3969075142 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2be8044050 con 0x55778a71b7f0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2c0824f740 10 client.15585 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2c0824f740 1 -- 172.21.15.18:0/3969075142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55778a79aee0 con 0x55778a71b7f0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2be3e00640 1 -- 172.21.15.18:0/3969075142 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be8030070 con 0x55778a71b7f0 2024-04-16T17:05:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2be3e00640 1 -- 172.21.15.18:0/3969075142 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2be8004950 con 0x55778a71b7f0 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2c0824f740 10 client.15585 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2c0824f740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2c0824f740 20 client.15585 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2c0824f740 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55778a79b5e0 0x55778a79da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2c0824f740 4 client.15585 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.757+0000 7f2c0824f740 1 -- 172.21.15.18:0/3969075142 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55778a687010 con 0x55778a79b5e0 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.758+0000 7f2bfea00640 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55778a79b5e0 0x55778a79da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.758+0000 7f2bfea00640 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55778a79b5e0 0x55778a79da00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73f3ca6b453745f server_cookie=f11b209893893f08 in_seq=0 out_seq=0 2024-04-16T17:05:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.758+0000 7f2be3e00640 10 client.15585 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.772+0000 7f2be3e00640 1 -- 172.21.15.18:0/3969075142 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55778a687010 con 0x55778a79b5e0 2024-04-16T17:05:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.772+0000 7f2be3e00640 10 client.15585 handle_command_reply: tid=0 2024-04-16T17:05:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.772+0000 7f2c0824f740 1 client.15585 shutdown 2024-04-16T17:05:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 -- 172.21.15.18:0/3969075142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71b7f0 msgr2=0x55778a725280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71b7f0 0x55778a725280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009df0 tx=0x7f2be8011af0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 -- 172.21.15.18:0/3969075142 shutdown_connections 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55778a79b5e0 0x55778a79da00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778a724000 0x55778a7257e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55778a71b7f0 0x55778a725280 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2be3e00640 0 client.15585 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 --2- 172.21.15.18:0/3969075142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55778a71aa80 0x55778a724d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 -- 172.21.15.18:0/3969075142 >> 172.21.15.18:0/3969075142 conn(0x55778a6b55d0 msgr2=0x55778a720dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 -- 172.21.15.18:0/3969075142 shutdown_connections 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 1 -- 172.21.15.18:0/3969075142 wait complete. 2024-04-16T17:05:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.773+0000 7f2c0824f740 20 client.15585 trim_cache size 0 max 16384 2024-04-16T17:05:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:05:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:05:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:05:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:05:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 -- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2be40a3f30 msgr2=0x7f2be40a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2be40a3f30 0x7f2be40a63f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55778a716900 tx=0x7f2be8005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 -- 172.21.15.18:0/2204245442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0019a7e0 msgr2=0x7f2c0019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0019a7e0 0x7f2c0019ac90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800d950 tx=0x7f2bf800de10 comp rx=0 tx=0).stop 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 -- 172.21.15.18:0/2204245442 shutdown_connections 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2be40a3f30 0x7f2be40a63f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0019a7e0 0x7f2c0019ac90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c00107150 0x7f2c001a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 --2- 172.21.15.18:0/2204245442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c00102080 0x7f2c001a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 -- 172.21.15.18:0/2204245442 >> 172.21.15.18:0/2204245442 conn(0x7f2c000fd940 msgr2=0x7f2c000ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 -- 172.21.15.18:0/2204245442 shutdown_connections 2024-04-16T17:05:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:10.774+0000 7f2bffe00640 1 -- 172.21.15.18:0/2204245442 wait complete. 2024-04-16T17:05:11.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:11 smithi022 ceph-mon[36023]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-04-16T17:05:11.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:11 smithi018 ceph-mon[31776]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-04-16T17:05:12.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:11 smithi043 ceph-mon[36901]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-04-16T17:05:13.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:13 smithi022 ceph-mon[36023]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-16T17:05:13.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:13 smithi018 ceph-mon[31776]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-16T17:05:14.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:13 smithi043 ceph-mon[36901]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-16T17:05:15.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:15 smithi022 ceph-mon[36023]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-16T17:05:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:15 smithi018 ceph-mon[31776]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-16T17:05:16.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:15 smithi043 ceph-mon[36901]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-16T17:05:17.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:17 smithi022 ceph-mon[36023]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-04-16T17:05:17.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:17 smithi018 ceph-mon[31776]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-04-16T17:05:18.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:17 smithi043 ceph-mon[36901]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-04-16T17:05:19.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:19 smithi022 ceph-mon[36023]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:05:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:19 smithi018 ceph-mon[31776]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:05:20.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:19 smithi043 ceph-mon[36901]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:05:21.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:21 smithi022 ceph-mon[36023]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:05:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:21 smithi018 ceph-mon[31776]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:05:22.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:21 smithi043 ceph-mon[36901]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:05:23.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:23 smithi022 ceph-mon[36023]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-16T17:05:23.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:23 smithi018 ceph-mon[31776]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-16T17:05:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 -- 172.21.15.18:0/787129818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb394102250 msgr2=0x7fb394102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 --2- 172.21.15.18:0/787129818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb394102250 0x7fb394102630 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb384009900 tx=0x7fb38402bc40 comp rx=0 tx=0).stop 2024-04-16T17:05:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 -- 172.21.15.18:0/787129818 shutdown_connections 2024-04-16T17:05:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 --2- 172.21.15.18:0/787129818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb394107320 0x7fb39410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 --2- 172.21.15.18:0/787129818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb394102c00 0x7fb394106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 --2- 172.21.15.18:0/787129818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb394102250 0x7fb394102630 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 -- 172.21.15.18:0/787129818 >> 172.21.15.18:0/787129818 conn(0x7fb3940fda90 msgr2=0x7fb3940ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 -- 172.21.15.18:0/787129818 shutdown_connections 2024-04-16T17:05:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.957+0000 7fb39b400640 1 -- 172.21.15.18:0/787129818 wait complete. 2024-04-16T17:05:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.958+0000 7fb39b400640 1 Processor -- start 2024-04-16T17:05:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.958+0000 7fb39b400640 1 -- start start 2024-04-16T17:05:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb39b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb394102250 0x7fb3941a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb39b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb394107320 0x7fb3941a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb39b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb39419a990 0x7fb39419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb39b400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3941123b0 con 0x7fb394102250 2024-04-16T17:05:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb39b400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb394112230 con 0x7fb39419a990 2024-04-16T17:05:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb39b400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb394112530 con 0x7fb394107320 2024-04-16T17:05:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb398a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb394102250 0x7fb3941a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb398a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb394102250 0x7fb3941a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41498/0 (socket says 172.21.15.18:41498) 2024-04-16T17:05:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb399400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb39419a990 0x7fb39419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.959+0000 7fb398a00640 1 -- 172.21.15.18:0/3017489481 learned_addr learned my addr 172.21.15.18:0/3017489481 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb399400640 1 -- 172.21.15.18:0/3017489481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb394107320 msgr2=0x7fb3941a0d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb399400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb394107320 0x7fb3941a0d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb399400640 1 -- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb394102250 msgr2=0x7fb3941a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb399400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb394102250 0x7fb3941a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb399400640 1 -- 172.21.15.18:0/3017489481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39419b4b0 con 0x7fb39419a990 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb399400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb39419a990 0x7fb39419ae40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00e940 tx=0x7fb38c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb391600640 1 -- 172.21.15.18:0/3017489481 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c004d20 con 0x7fb39419a990 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb39b400640 1 -- 172.21.15.18:0/3017489481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39419b7a0 con 0x7fb39419a990 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb391600640 1 -- 172.21.15.18:0/3017489481 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb38c005020 con 0x7fb39419a990 2024-04-16T17:05:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.960+0000 7fb391600640 1 -- 172.21.15.18:0/3017489481 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0116d0 con 0x7fb39419a990 2024-04-16T17:05:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.961+0000 7fb39b400640 1 -- 172.21.15.18:0/3017489481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3941a79f0 con 0x7fb39419a990 2024-04-16T17:05:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.964+0000 7fb391600640 1 -- 172.21.15.18:0/3017489481 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fb38c0054b0 con 0x7fb39419a990 2024-04-16T17:05:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.965+0000 7fb39e19d740 1 Processor -- start 2024-04-16T17:05:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.965+0000 7fb39e19d740 1 -- start start 2024-04-16T17:05:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.965+0000 7fb391600640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb37c0a3ec0 0x7fb37c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.965+0000 7fb391600640 1 -- 172.21.15.18:0/3017489481 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb38c0d0f50 con 0x7fb39419a990 2024-04-16T17:05:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.965+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d34610 0x5569a6d33360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d33d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d3ce20 0x5569a6d3f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb39e19d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5569a6c98780 con 0x5569a6d338a0 2024-04-16T17:05:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb39e19d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5569a6c3ef40 con 0x5569a6d3ce20 2024-04-16T17:05:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb39e19d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569a6c94740 con 0x5569a6d34610 2024-04-16T17:05:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb398a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d34610 0x5569a6d33360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb398a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d34610 0x5569a6d33360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57870/0 (socket says 172.21.15.18:57870) 2024-04-16T17:05:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb393e00640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb37c0a3ec0 0x7fb37c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb398a00640 1 -- 172.21.15.18:0/1714378872 learned_addr learned my addr 172.21.15.18:0/1714378872 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.966+0000 7fb393e00640 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d33d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.967+0000 7fb399400640 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d3ce20 0x5569a6d3f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.967+0000 7fb398a00640 1 -- 172.21.15.18:0/1714378872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d3ce20 msgr2=0x5569a6d3f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.967+0000 7fb398a00640 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d3ce20 0x5569a6d3f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.967+0000 7fb398a00640 1 -- 172.21.15.18:0/1714378872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 msgr2=0x5569a6d33d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.967+0000 7fb398a00640 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d33d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.967+0000 7fb398a00640 1 -- 172.21.15.18:0/1714378872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a69ab210 con 0x5569a6d34610 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.967+0000 7fb393e00640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb37c0a3ec0 0x7fb37c0a6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb388005d00 tx=0x7fb3880099b0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.967+0000 7fb398a00640 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d34610 0x5569a6d33360 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb384036040 tx=0x7fb3840095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f13a0d57ef9fad03 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.968+0000 7fb381c00640 1 -- 172.21.15.18:0/1714378872 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384042070 con 0x5569a6d34610 2024-04-16T17:05:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.968+0000 7fb381c00640 1 -- 172.21.15.18:0/1714378872 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb38402ca90 con 0x5569a6d34610 2024-04-16T17:05:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.968+0000 7fb381c00640 1 -- 172.21.15.18:0/1714378872 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384004ee0 con 0x5569a6d34610 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 -- 172.21.15.18:0/1714378872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d34610 msgr2=0x5569a6d33360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d34610 0x5569a6d33360 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb384036040 tx=0x7fb3840095e0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 -- 172.21.15.18:0/1714378872 shutdown_connections 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d3ce20 0x5569a6d3f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d33d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1714378872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d34610 0x5569a6d33360 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 -- 172.21.15.18:0/1714378872 >> 172.21.15.18:0/1714378872 conn(0x5569a6cce5d0 msgr2=0x5569a6d2e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 -- 172.21.15.18:0/1714378872 shutdown_connections 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.969+0000 7fb39e19d740 1 -- 172.21.15.18:0/1714378872 wait complete. 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.970+0000 7fb39e19d740 1 -- start start 2024-04-16T17:05:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.970+0000 7fb39e19d740 1 Processor -- start 2024-04-16T17:05:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.970+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d3c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d34610 0x5569a6d39c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb39e19d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5569a6d3faf0 con 0x5569a6d338a0 2024-04-16T17:05:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb39e19d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5569a6c52390 con 0x5569a6d34610 2024-04-16T17:05:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb39e19d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569a6d3fc70 con 0x5569a6d3ce20 2024-04-16T17:05:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb39e19d740 10 client.0 _ll_register_callbacks cb 0x5569a6c3bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb398a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d3c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb399400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb399400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57880/0 (socket says 172.21.15.18:57880) 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb398a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d3c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41514/0 (socket says 172.21.15.18:41514) 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb399400640 1 -- 172.21.15.18:0/876786862 learned_addr learned my addr 172.21.15.18:0/876786862 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb393e00640 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d34610 0x5569a6d39c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb399400640 1 -- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d34610 msgr2=0x5569a6d39c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb399400640 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d34610 0x5569a6d39c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.971+0000 7fb399400640 1 -- 172.21.15.18:0/876786862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 msgr2=0x5569a6d3c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb399400640 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d3c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb399400640 1 -- 172.21.15.18:0/876786862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a6a77620 con 0x5569a6d3ce20 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb393e00640 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d34610 0x5569a6d39c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb398a00640 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d3c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb399400640 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3a3b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb38c09c600 tx=0x7fb38c09d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c75f717b78ac5d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb376000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb376000640 1 -- 172.21.15.18:0/876786862 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c01f040 con 0x5569a6d3ce20 2024-04-16T17:05:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb376000640 1 -- 172.21.15.18:0/876786862 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb38c09bd50 con 0x5569a6d3ce20 2024-04-16T17:05:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.972+0000 7fb39e19d740 1 -- 172.21.15.18:0/876786862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb38402ca90 con 0x5569a6d3ce20 2024-04-16T17:05:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.973+0000 7fb376000640 1 -- 172.21.15.18:0/876786862 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb38c0872c0 con 0x5569a6d3ce20 2024-04-16T17:05:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.973+0000 7fb39e19d740 10 client.25711 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.973+0000 7fb39e19d740 1 -- 172.21.15.18:0/876786862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569a6d4bb50 con 0x5569a6d3ce20 2024-04-16T17:05:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.973+0000 7fb376000640 1 -- 172.21.15.18:0/876786862 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb38402ca90 con 0x5569a6d3ce20 2024-04-16T17:05:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.974+0000 7fb376000640 1 -- 172.21.15.18:0/876786862 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb38c00f040 con 0x5569a6d3ce20 2024-04-16T17:05:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.974+0000 7fb39e19d740 10 client.25711 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.974+0000 7fb39e19d740 10 client.25711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.974+0000 7fb39e19d740 20 client.25711 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.974+0000 7fb39e19d740 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5569a6d4d010 0x5569a6d4f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.974+0000 7fb39e19d740 4 client.25711 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.974+0000 7fb39e19d740 1 -- 172.21.15.18:0/876786862 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569a6ca61a0 con 0x5569a6d4d010 2024-04-16T17:05:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.975+0000 7fb398a00640 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5569a6d4d010 0x5569a6d4f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.975+0000 7fb398a00640 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5569a6d4d010 0x5569a6d4f440 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3beadfc6828f3fe server_cookie=a10920beb867a4f0 in_seq=0 out_seq=0 2024-04-16T17:05:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.975+0000 7fb376000640 10 client.25711 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.976+0000 7fb376000640 1 -- 172.21.15.18:0/876786862 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569a6ca61a0 con 0x5569a6d4d010 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.976+0000 7fb376000640 10 client.25711 handle_command_reply: tid=0 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.976+0000 7fb39e19d740 1 client.25711 shutdown 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.976+0000 7fb39e19d740 1 -- 172.21.15.18:0/876786862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 msgr2=0x5569a6d3a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.976+0000 7fb39e19d740 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3a3b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb38c09c600 tx=0x7fb38c09d000 comp rx=0 tx=0).stop 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 1 -- 172.21.15.18:0/876786862 shutdown_connections 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5569a6d4d010 0x5569a6d4f440 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3a3b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d34610 0x5569a6d39c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 1 --2- 172.21.15.18:0/876786862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d338a0 0x5569a6d3c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 1 -- 172.21.15.18:0/876786862 >> 172.21.15.18:0/876786862 conn(0x5569a6cce5d0 msgr2=0x5569a6d302e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb376000640 0 client.25711 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 1 -- 172.21.15.18:0/876786862 shutdown_connections 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 1 -- 172.21.15.18:0/876786862 wait complete. 2024-04-16T17:05:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.977+0000 7fb39e19d740 20 client.25711 trim_cache size 0 max 16384 2024-04-16T17:05:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.993+0000 7fb39e19d740 1 Processor -- start 2024-04-16T17:05:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.993+0000 7fb39e19d740 1 -- start start 2024-04-16T17:05:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.993+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d489f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.993+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d3c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.993+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.993+0000 7fb39e19d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3941b4020 con 0x5569a6d34610 2024-04-16T17:05:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.993+0000 7fb39e19d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3941b18f0 con 0x5569a6d338a0 2024-04-16T17:05:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.993+0000 7fb39e19d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3941af740 con 0x5569a6d3ce20 2024-04-16T17:05:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb393e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d3c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb399400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb393e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d3c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41516/0 (socket says 172.21.15.18:41516) 2024-04-16T17:05:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb393e00640 1 -- 172.21.15.18:0/3344564563 learned_addr learned my addr 172.21.15.18:0/3344564563 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb398a00640 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb399400640 1 -- 172.21.15.18:0/3344564563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 msgr2=0x5569a6d489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb399400640 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb399400640 1 -- 172.21.15.18:0/3344564563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 msgr2=0x5569a6d3c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb399400640 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d3c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb399400640 1 -- 172.21.15.18:0/3344564563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a6d381c0 con 0x5569a6d3ce20 2024-04-16T17:05:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.994+0000 7fb393e00640 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d3c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.995+0000 7fb399400640 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3c8e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0ce1a0 tx=0x7fb38c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5fb828cae856027 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.995+0000 7fb381200640 1 -- 172.21.15.18:0/3344564563 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c01f070 con 0x5569a6d3ce20 2024-04-16T17:05:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.995+0000 7fb381200640 1 -- 172.21.15.18:0/3344564563 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb38c0a0cb0 con 0x5569a6d3ce20 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.995+0000 7fb381200640 1 -- 172.21.15.18:0/3344564563 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0bbac0 con 0x5569a6d3ce20 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 -- 172.21.15.18:0/3344564563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 msgr2=0x5569a6d3c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3c8e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0ce1a0 tx=0x7fb38c004a40 comp rx=0 tx=0).stop 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 -- 172.21.15.18:0/3344564563 shutdown_connections 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d3c8e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d3c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 --2- 172.21.15.18:0/3344564563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 -- 172.21.15.18:0/3344564563 >> 172.21.15.18:0/3344564563 conn(0x5569a6cce5d0 msgr2=0x5569a6d38540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 -- 172.21.15.18:0/3344564563 shutdown_connections 2024-04-16T17:05:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 -- 172.21.15.18:0/3344564563 wait complete. 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.996+0000 7fb39e19d740 1 -- start start 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.997+0000 7fb39e19d740 1 Processor -- start 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.997+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d45410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.997+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d36740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.997+0000 7fb39e19d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d36e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.997+0000 7fb39e19d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3941afdb0 con 0x5569a6d34610 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.997+0000 7fb39e19d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3941b1220 con 0x5569a6d338a0 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.997+0000 7fb39e19d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3941b0480 con 0x5569a6d3ce20 2024-04-16T17:05:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.997+0000 7fb39e19d740 10 client.0 _ll_register_callbacks cb 0x7fb38c00ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb399400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d36e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d45410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d45410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:41960/0 (socket says 172.21.15.18:41960) 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 -- 172.21.15.18:0/1967195877 learned_addr learned my addr 172.21.15.18:0/1967195877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb399400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d36e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57892/0 (socket says 172.21.15.18:57892) 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb393e00640 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d36740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 -- 172.21.15.18:0/1967195877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 msgr2=0x5569a6d36e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d36e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 -- 172.21.15.18:0/1967195877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 msgr2=0x5569a6d36740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d36740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 -- 172.21.15.18:0/1967195877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a6db25c0 con 0x5569a6d338a0 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb393e00640 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d36740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb398a00640 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d45410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb38402c200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3614bf5b3febddb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.998+0000 7fb376000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.999+0000 7fb376000640 1 -- 172.21.15.18:0/1967195877 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384055060 con 0x5569a6d338a0 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.999+0000 7fb39e19d740 1 -- 172.21.15.18:0/1967195877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb38c0a0cb0 con 0x5569a6d338a0 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.999+0000 7fb376000640 1 -- 172.21.15.18:0/1967195877 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb384004040 con 0x5569a6d338a0 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.999+0000 7fb376000640 1 -- 172.21.15.18:0/1967195877 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb38405c050 con 0x5569a6d338a0 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.999+0000 7fb39e19d740 10 client.25730 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:23.999+0000 7fb39e19d740 1 -- 172.21.15.18:0/1967195877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569a6db3740 con 0x5569a6d338a0 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb376000640 1 -- 172.21.15.18:0/1967195877 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb38c0a0cb0 con 0x5569a6d338a0 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb376000640 1 -- 172.21.15.18:0/1967195877 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb38404e070 con 0x5569a6d338a0 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb39e19d740 10 client.25730 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb39e19d740 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb39e19d740 20 client.25730 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5569a6db3a80 0x5569a6db5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb39e19d740 4 client.25730 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:05:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb39e19d740 1 -- 172.21.15.18:0/1967195877 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569a6ca0010 con 0x5569a6db3a80 2024-04-16T17:05:24.004 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:23 smithi043 ceph-mon[36901]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-16T17:05:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.000+0000 7fb393e00640 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5569a6db3a80 0x5569a6db5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.001+0000 7fb393e00640 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5569a6db3a80 0x5569a6db5e70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5db0f7c4a68c09 server_cookie=3776dff1dcf64e25 in_seq=0 out_seq=0 2024-04-16T17:05:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.001+0000 7fb376000640 10 client.25730 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb376000640 1 -- 172.21.15.18:0/1967195877 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569a6ca0010 con 0x5569a6db3a80 2024-04-16T17:05:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb376000640 10 client.25730 handle_command_reply: tid=0 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 client.25730 shutdown 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 -- 172.21.15.18:0/1967195877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 msgr2=0x5569a6d45410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d45410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb38402c200 comp rx=0 tx=0).stop 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 -- 172.21.15.18:0/1967195877 shutdown_connections 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5569a6db3a80 0x5569a6db5e70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569a6d3ce20 0x5569a6d36e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5569a6d34610 0x5569a6d36740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 --2- 172.21.15.18:0/1967195877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569a6d338a0 0x5569a6d45410 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb39e19d740 1 -- 172.21.15.18:0/1967195877 >> 172.21.15.18:0/1967195877 conn(0x5569a6cce5d0 msgr2=0x5569a6d2f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.005+0000 7fb376000640 0 client.25730 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.006+0000 7fb39e19d740 1 -- 172.21.15.18:0/1967195877 shutdown_connections 2024-04-16T17:05:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.006+0000 7fb39e19d740 1 -- 172.21.15.18:0/1967195877 wait complete. 2024-04-16T17:05:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.006+0000 7fb39e19d740 20 client.25730 trim_cache size 0 max 16384 2024-04-16T17:05:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:05:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:05:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:05:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:05:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 -- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb37c0a3ec0 msgr2=0x7fb37c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb37c0a3ec0 0x7fb37c0a6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb388005d00 tx=0x7fb3880099b0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 -- 172.21.15.18:0/3017489481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb39419a990 msgr2=0x7fb39419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb39419a990 0x7fb39419ae40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00e940 tx=0x7fb38c00ee00 comp rx=0 tx=0).stop 2024-04-16T17:05:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 -- 172.21.15.18:0/3017489481 shutdown_connections 2024-04-16T17:05:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb37c0a3ec0 0x7fb37c0a6380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb39419a990 0x7fb39419ae40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb394107320 0x7fb3941a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 --2- 172.21.15.18:0/3017489481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb394102250 0x7fb3941a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 -- 172.21.15.18:0/3017489481 >> 172.21.15.18:0/3017489481 conn(0x7fb3940fda90 msgr2=0x7fb3940ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 -- 172.21.15.18:0/3017489481 shutdown_connections 2024-04-16T17:05:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:24.007+0000 7fb39b400640 1 -- 172.21.15.18:0/3017489481 wait complete. 2024-04-16T17:05:25.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:05:25.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:05:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 -- 172.21.15.18:0/2458148253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d4074cd0 msgr2=0x7fc5d410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 --2- 172.21.15.18:0/2458148253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d4074cd0 0x7fc5d410e180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009900 tx=0x7fc5c002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 -- 172.21.15.18:0/2458148253 shutdown_connections 2024-04-16T17:05:25.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 --2- 172.21.15.18:0/2458148253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d410e6c0 0x7fc5d4110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 --2- 172.21.15.18:0/2458148253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d4074cd0 0x7fc5d410e180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 --2- 172.21.15.18:0/2458148253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d40743b0 0x7fc5d4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 -- 172.21.15.18:0/2458148253 >> 172.21.15.18:0/2458148253 conn(0x7fc5d40fd950 msgr2=0x7fc5d40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:25.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 -- 172.21.15.18:0/2458148253 shutdown_connections 2024-04-16T17:05:25.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.892+0000 7fc5d3200640 1 -- 172.21.15.18:0/2458148253 wait complete. 2024-04-16T17:05:25.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.894+0000 7fc5d3200640 1 Processor -- start 2024-04-16T17:05:25.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.895+0000 7fc5d3200640 1 -- start start 2024-04-16T17:05:25.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.895+0000 7fc5d3200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d40743b0 0x7fc5d41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.895+0000 7fc5d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d4074cd0 0x7fc5d41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.895+0000 7fc5d3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d410e6c0 0x7fc5d419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.895+0000 7fc5d3200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4114420 con 0x7fc5d40743b0 2024-04-16T17:05:25.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.895+0000 7fc5d3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41142a0 con 0x7fc5d410e6c0 2024-04-16T17:05:25.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.895+0000 7fc5d3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41145a0 con 0x7fc5d4074cd0 2024-04-16T17:05:25.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d40743b0 0x7fc5d41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d4074cd0 0x7fc5d41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d4074cd0 0x7fc5d41a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57914/0 (socket says 172.21.15.18:57914) 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1400640 1 -- 172.21.15.18:0/3005541909 learned_addr learned my addr 172.21.15.18:0/3005541909 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d2800640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d410e6c0 0x7fc5d419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1400640 1 -- 172.21.15.18:0/3005541909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d410e6c0 msgr2=0x7fc5d419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1400640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d410e6c0 0x7fc5d419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1400640 1 -- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d40743b0 msgr2=0x7fc5d41a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1400640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d40743b0 0x7fc5d41a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1400640 1 -- 172.21.15.18:0/3005541909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d419b000 con 0x7fc5d4074cd0 2024-04-16T17:05:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.896+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d40743b0 0x7fc5d41a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.897+0000 7fc5d1400640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d4074cd0 0x7fc5d41a0bc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0000c00 tx=0x7fc5c0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.897+0000 7fc5caa00640 1 -- 172.21.15.18:0/3005541909 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0042020 con 0x7fc5d4074cd0 2024-04-16T17:05:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.897+0000 7fc5caa00640 1 -- 172.21.15.18:0/3005541909 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c00054e0 con 0x7fc5d4074cd0 2024-04-16T17:05:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.897+0000 7fc5d3200640 1 -- 172.21.15.18:0/3005541909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d419b290 con 0x7fc5d4074cd0 2024-04-16T17:05:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.897+0000 7fc5caa00640 1 -- 172.21.15.18:0/3005541909 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0003b20 con 0x7fc5d4074cd0 2024-04-16T17:05:25.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.897+0000 7fc5d3200640 1 -- 172.21.15.18:0/3005541909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d41a7600 con 0x7fc5d4074cd0 2024-04-16T17:05:25.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.901+0000 7fc5db619740 1 Processor -- start 2024-04-16T17:05:25.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.901+0000 7fc5db619740 1 -- start start 2024-04-16T17:05:25.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5caa00640 1 -- 172.21.15.18:0/3005541909 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fc5c0045020 con 0x7fc5d4074cd0 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5db619740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1d390 0x55cd47e1d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5db619740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1dcb0 0x55cd47e26350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5db619740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 0x55cd47e28c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5db619740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cd47d83120 con 0x55cd47e1d390 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5db619740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cd47d29340 con 0x55cd47e1dcb0 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5db619740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cd47d7f0e0 con 0x55cd47e26890 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1dcb0 0x55cd47e26350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5d2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1d390 0x55cd47e1d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1dcb0 0x55cd47e26350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:41978/0 (socket says 172.21.15.18:41978) 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5d1e00640 1 -- 172.21.15.18:0/1710196622 learned_addr learned my addr 172.21.15.18:0/1710196622 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.902+0000 7fc5d1400640 1 --2- 172.21.15.18:0/1710196622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 0x55cd47e28c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.903+0000 7fc5d2800640 1 -- 172.21.15.18:0/1710196622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 msgr2=0x55cd47e28c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:25.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.903+0000 7fc5d2800640 1 --2- 172.21.15.18:0/1710196622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 0x55cd47e28c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.903+0000 7fc5d2800640 1 -- 172.21.15.18:0/1710196622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1dcb0 msgr2=0x55cd47e26350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.903+0000 7fc5d2800640 1 --2- 172.21.15.18:0/1710196622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1dcb0 0x55cd47e26350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.903+0000 7fc5d2800640 1 -- 172.21.15.18:0/1710196622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd47b5f7d0 con 0x55cd47e1d390 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.903+0000 7fc5d2800640 1 --2- 172.21.15.18:0/1710196622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1d390 0x55cd47e1d770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009e50 tx=0x7fc5cc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79bd92ed333a4b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.904+0000 7fc5bf000640 1 -- 172.21.15.18:0/1710196622 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc003b70 con 0x55cd47e1d390 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.904+0000 7fc5bf000640 1 -- 172.21.15.18:0/1710196622 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5cc015070 con 0x55cd47e1d390 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.904+0000 7fc5bf000640 1 -- 172.21.15.18:0/1710196622 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc011800 con 0x55cd47e1d390 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.904+0000 7fc5caa00640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5b80a3ed0 0x7fc5b80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.904+0000 7fc5caa00640 1 -- 172.21.15.18:0/3005541909 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc5c00f2aa0 con 0x7fc5d4074cd0 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.904+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5b80a3ed0 0x7fc5b80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.904+0000 7fc5db619740 1 -- 172.21.15.18:0/1710196622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1d390 msgr2=0x55cd47e1d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.904+0000 7fc5db619740 1 --2- 172.21.15.18:0/1710196622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1d390 0x55cd47e1d770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009e50 tx=0x7fc5cc0107a0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.905+0000 7fc5db619740 1 -- 172.21.15.18:0/1710196622 shutdown_connections 2024-04-16T17:05:25.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.905+0000 7fc5db619740 1 --2- 172.21.15.18:0/1710196622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 0x55cd47e28c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.905+0000 7fc5db619740 1 --2- 172.21.15.18:0/1710196622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1dcb0 0x55cd47e26350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.905+0000 7fc5db619740 1 --2- 172.21.15.18:0/1710196622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1d390 0x55cd47e1d770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.905+0000 7fc5db619740 1 -- 172.21.15.18:0/1710196622 >> 172.21.15.18:0/1710196622 conn(0x55cd47e17ac0 msgr2=0x55cd47e17ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:25.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.905+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5b80a3ed0 0x7fc5b80a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55cd47e1ce60 tx=0x7fc5c4005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:25.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.907+0000 7fc5db619740 1 -- 172.21.15.18:0/1710196622 shutdown_connections 2024-04-16T17:05:25.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.908+0000 7fc5db619740 1 -- 172.21.15.18:0/1710196622 wait complete. 2024-04-16T17:05:25.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.908+0000 7fc5db619740 1 -- start start 2024-04-16T17:05:25.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.908+0000 7fc5db619740 1 Processor -- start 2024-04-16T17:05:25.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5db619740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e23910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5db619740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e23e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5db619740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 0x55cd47e24390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5db619740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cd47e29680 con 0x55cd47e1dcb0 2024-04-16T17:05:25.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5db619740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cd47d3c710 con 0x55cd47e1d390 2024-04-16T17:05:25.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5db619740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cd47e29800 con 0x55cd47e26890 2024-04-16T17:05:25.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5db619740 10 client.0 _ll_register_callbacks cb 0x55cd47d37af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:25.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e23910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5d1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e23e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e23910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:41988/0 (socket says 172.21.15.18:41988) 2024-04-16T17:05:25.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5d1e00640 1 -- 172.21.15.18:0/4024651112 learned_addr learned my addr 172.21.15.18:0/4024651112 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:25.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.909+0000 7fc5d2800640 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 0x55cd47e24390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.910+0000 7fc5d1e00640 1 -- 172.21.15.18:0/4024651112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 msgr2=0x55cd47e24390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:25.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.910+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 0x55cd47e24390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.910+0000 7fc5d1e00640 1 -- 172.21.15.18:0/4024651112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 msgr2=0x55cd47e23e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:25.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.910+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e23e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.910+0000 7fc5d1e00640 1 -- 172.21.15.18:0/4024651112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd47a953f0 con 0x55cd47e1d390 2024-04-16T17:05:25.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.910+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e23910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40209e0 tx=0x7fc5c4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3cc3c6d5bf7f3b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:25.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.910+0000 7fc5b3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:25.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.911+0000 7fc5b3400640 1 -- 172.21.15.18:0/4024651112 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4009b30 con 0x55cd47e1d390 2024-04-16T17:05:25.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.911+0000 7fc5b3400640 1 -- 172.21.15.18:0/4024651112 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc5c4011b50 con 0x55cd47e1d390 2024-04-16T17:05:25.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.911+0000 7fc5db619740 1 -- 172.21.15.18:0/4024651112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5cc015070 con 0x55cd47e1d390 2024-04-16T17:05:25.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.911+0000 7fc5b3400640 1 -- 172.21.15.18:0/4024651112 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5c4011d50 con 0x55cd47e1d390 2024-04-16T17:05:25.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.911+0000 7fc5db619740 10 client.25742 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:25.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.911+0000 7fc5db619740 1 -- 172.21.15.18:0/4024651112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd47a550f0 con 0x55cd47e1d390 2024-04-16T17:05:25.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.912+0000 7fc5b3400640 1 -- 172.21.15.18:0/4024651112 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5cc015070 con 0x55cd47e1d390 2024-04-16T17:05:25.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.912+0000 7fc5b3400640 1 -- 172.21.15.18:0/4024651112 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc5c4027070 con 0x55cd47e1d390 2024-04-16T17:05:25.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.912+0000 7fc5db619740 10 client.25742 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:25.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.912+0000 7fc5db619740 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:25.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.912+0000 7fc5db619740 20 client.25742 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:25.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.912+0000 7fc5db619740 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cd47e36d10 0x55cd47e39180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.912+0000 7fc5db619740 4 client.25742 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:25.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.912+0000 7fc5db619740 1 -- 172.21.15.18:0/4024651112 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd47d90b40 con 0x55cd47e36d10 2024-04-16T17:05:25.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.913+0000 7fc5d1400640 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cd47e36d10 0x55cd47e39180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.913+0000 7fc5d1400640 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cd47e36d10 0x55cd47e39180 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ef57717ef2142c server_cookie=42d8d7c9491b8644 in_seq=0 out_seq=0 2024-04-16T17:05:25.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.913+0000 7fc5b3400640 10 client.25742 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:25.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5b3400640 1 -- 172.21.15.18:0/4024651112 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd47d90b40 con 0x55cd47e36d10 2024-04-16T17:05:25.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5b3400640 10 client.25742 handle_command_reply: tid=0 2024-04-16T17:05:25.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 client.25742 shutdown 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 -- 172.21.15.18:0/4024651112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 msgr2=0x55cd47e23910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e23910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40209e0 tx=0x7fc5c4004730 comp rx=0 tx=0).stop 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 -- 172.21.15.18:0/4024651112 shutdown_connections 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cd47e36d10 0x55cd47e39180 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e26890 0x55cd47e24390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e23e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 --2- 172.21.15.18:0/4024651112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e23910 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5b3400640 0 client.25742 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.914+0000 7fc5db619740 1 -- 172.21.15.18:0/4024651112 >> 172.21.15.18:0/4024651112 conn(0x55cd47e17ac0 msgr2=0x55cd47e19810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.915+0000 7fc5db619740 1 -- 172.21.15.18:0/4024651112 shutdown_connections 2024-04-16T17:05:25.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.915+0000 7fc5db619740 1 -- 172.21.15.18:0/4024651112 wait complete. 2024-04-16T17:05:25.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.915+0000 7fc5db619740 20 client.25742 trim_cache size 0 max 16384 2024-04-16T17:05:25.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:25 smithi022 ceph-mon[36023]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T17:05:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.934+0000 7fc5db619740 1 Processor -- start 2024-04-16T17:05:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.934+0000 7fc5db619740 1 -- start start 2024-04-16T17:05:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5db619740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e17ac0 0x55cd47e26040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5db619740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e22850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5db619740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e1dcb0 0x55cd47e32540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5db619740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41b3e90 con 0x55cd47e17ac0 2024-04-16T17:05:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5db619740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41b1760 con 0x55cd47e1d390 2024-04-16T17:05:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5db619740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41af5b0 con 0x55cd47e1dcb0 2024-04-16T17:05:25.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e17ac0 0x55cd47e26040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e22850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e17ac0 0x55cd47e26040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41562/0 (socket says 172.21.15.18:41562) 2024-04-16T17:05:25.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5d1e00640 1 -- 172.21.15.18:0/248407186 learned_addr learned my addr 172.21.15.18:0/248407186 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:25.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e22850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:41992/0 (socket says 172.21.15.18:41992) 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.935+0000 7fc5d2800640 1 --2- 172.21.15.18:0/248407186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e1dcb0 0x55cd47e32540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5d1400640 1 -- 172.21.15.18:0/248407186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e1dcb0 msgr2=0x55cd47e32540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5d1400640 1 --2- 172.21.15.18:0/248407186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e1dcb0 0x55cd47e32540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5d1400640 1 -- 172.21.15.18:0/248407186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e17ac0 msgr2=0x55cd47e26040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5d1400640 1 --2- 172.21.15.18:0/248407186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e17ac0 0x55cd47e26040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5d1400640 1 -- 172.21.15.18:0/248407186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd47e297a0 con 0x55cd47e1d390 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5d1400640 1 --2- 172.21.15.18:0/248407186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e22850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00efd10 tx=0x7fc5c00f4ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d8211c4f6b34ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5be600640 1 -- 172.21.15.18:0/248407186 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00ba800 con 0x55cd47e1d390 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5be600640 1 -- 172.21.15.18:0/248407186 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c00fb040 con 0x55cd47e1d390 2024-04-16T17:05:25.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.936+0000 7fc5be600640 1 -- 172.21.15.18:0/248407186 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00bdec0 con 0x55cd47e1d390 2024-04-16T17:05:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 -- 172.21.15.18:0/248407186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 msgr2=0x55cd47e22850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 --2- 172.21.15.18:0/248407186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e22850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00efd10 tx=0x7fc5c00f4ef0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 -- 172.21.15.18:0/248407186 shutdown_connections 2024-04-16T17:05:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 --2- 172.21.15.18:0/248407186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e1dcb0 0x55cd47e32540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 --2- 172.21.15.18:0/248407186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e22850 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 --2- 172.21.15.18:0/248407186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e17ac0 0x55cd47e26040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 -- 172.21.15.18:0/248407186 >> 172.21.15.18:0/248407186 conn(0x55cd47e12f50 msgr2=0x55cd47e19780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 -- 172.21.15.18:0/248407186 shutdown_connections 2024-04-16T17:05:25.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.937+0000 7fc5db619740 1 -- 172.21.15.18:0/248407186 wait complete. 2024-04-16T17:05:25.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.938+0000 7fc5db619740 1 -- start start 2024-04-16T17:05:25.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.938+0000 7fc5db619740 1 Processor -- start 2024-04-16T17:05:25.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.938+0000 7fc5db619740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e17ac0 0x55cd47e2ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.938+0000 7fc5db619740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e2ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.938+0000 7fc5db619740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e2ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.938+0000 7fc5db619740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41afc20 con 0x55cd47e1dcb0 2024-04-16T17:05:25.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.938+0000 7fc5db619740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41b1090 con 0x55cd47e1d390 2024-04-16T17:05:25.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5db619740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41b02f0 con 0x55cd47e17ac0 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5db619740 10 client.0 _ll_register_callbacks cb 0x7fc5c40208a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e17ac0 0x55cd47e2ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e17ac0 0x55cd47e2ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57940/0 (socket says 172.21.15.18:57940) 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1e00640 1 -- 172.21.15.18:0/2651319908 learned_addr learned my addr 172.21.15.18:0/2651319908 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1400640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e2ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d2800640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e2ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1e00640 1 -- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 msgr2=0x55cd47e2ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e2ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1e00640 1 -- 172.21.15.18:0/2651319908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 msgr2=0x55cd47e2ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e2ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1e00640 1 -- 172.21.15.18:0/2651319908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd47e2c060 con 0x55cd47e17ac0 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d1400640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e2ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.939+0000 7fc5d2800640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e2ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.940+0000 7fc5d1e00640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e17ac0 0x55cd47e2ea50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5c401d940 tx=0x7fc5c4011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e213fb5e136b7d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.940+0000 7fc5b3400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.940+0000 7fc5b3400640 1 -- 172.21.15.18:0/2651319908 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4029640 con 0x55cd47e17ac0 2024-04-16T17:05:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.940+0000 7fc5db619740 1 -- 172.21.15.18:0/2651319908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c00fb040 con 0x55cd47e17ac0 2024-04-16T17:05:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.940+0000 7fc5b3400640 1 -- 172.21.15.18:0/2651319908 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc5c40297e0 con 0x55cd47e17ac0 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.940+0000 7fc5b3400640 1 -- 172.21.15.18:0/2651319908 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5c4029a00 con 0x55cd47e17ac0 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.941+0000 7fc5db619740 10 client.25735 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.941+0000 7fc5db619740 1 -- 172.21.15.18:0/2651319908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd47ea63f0 con 0x55cd47e17ac0 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.941+0000 7fc5b3400640 1 -- 172.21.15.18:0/2651319908 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c00fb040 con 0x55cd47e17ac0 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.941+0000 7fc5b3400640 1 -- 172.21.15.18:0/2651319908 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc5c4027040 con 0x55cd47e17ac0 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5db619740 10 client.25735 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5db619740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5db619740 20 client.25735 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5db619740 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cd47e26890 0x55cd47ea8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5db619740 4 client.25735 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5db619740 1 -- 172.21.15.18:0/2651319908 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd47d8a9b0 con 0x55cd47e26890 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5d1400640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cd47e26890 0x55cd47ea8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5d1400640 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cd47e26890 0x55cd47ea8bd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e873e5507fdc3b58 server_cookie=ef6add2d6415ff5f in_seq=0 out_seq=0 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.942+0000 7fc5b3400640 10 client.25735 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5b3400640 1 -- 172.21.15.18:0/2651319908 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cd47d8a9b0 con 0x55cd47e26890 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5b3400640 10 client.25735 handle_command_reply: tid=0 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 client.25735 shutdown 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 -- 172.21.15.18:0/2651319908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e17ac0 msgr2=0x55cd47e2ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e17ac0 0x55cd47e2ea50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5c401d940 tx=0x7fc5c4011b50 comp rx=0 tx=0).stop 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 -- 172.21.15.18:0/2651319908 shutdown_connections 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cd47e26890 0x55cd47ea8bd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cd47e1dcb0 0x55cd47e2ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd47e1d390 0x55cd47e2ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 --2- 172.21.15.18:0/2651319908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd47e17ac0 0x55cd47e2ea50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 -- 172.21.15.18:0/2651319908 >> 172.21.15.18:0/2651319908 conn(0x55cd47e12f50 msgr2=0x55cd47e16290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5b3400640 0 client.25735 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.943+0000 7fc5db619740 1 -- 172.21.15.18:0/2651319908 shutdown_connections 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.944+0000 7fc5db619740 1 -- 172.21.15.18:0/2651319908 wait complete. 2024-04-16T17:05:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.944+0000 7fc5db619740 20 client.25735 trim_cache size 0 max 16384 2024-04-16T17:05:25.947 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:05:25.947 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:05:25.947 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:05:25.947 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:05:25.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:25 smithi018 ceph-mon[31776]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.944+0000 7fc5d3200640 1 -- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5b80a3ed0 msgr2=0x7fc5b80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.944+0000 7fc5d3200640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5b80a3ed0 0x7fc5b80a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55cd47e1ce60 tx=0x7fc5c4005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.944+0000 7fc5d3200640 1 -- 172.21.15.18:0/3005541909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d4074cd0 msgr2=0x7fc5d41a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.944+0000 7fc5d3200640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d4074cd0 0x7fc5d41a0bc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0000c00 tx=0x7fc5c0004360 comp rx=0 tx=0).stop 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.945+0000 7fc5d3200640 1 -- 172.21.15.18:0/3005541909 shutdown_connections 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.945+0000 7fc5d3200640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5b80a3ed0 0x7fc5b80a6390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.945+0000 7fc5d3200640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5d410e6c0 0x7fc5d419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.945+0000 7fc5d3200640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d4074cd0 0x7fc5d41a0bc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.945+0000 7fc5d3200640 1 --2- 172.21.15.18:0/3005541909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5d40743b0 0x7fc5d41a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.945+0000 7fc5d3200640 1 -- 172.21.15.18:0/3005541909 >> 172.21.15.18:0/3005541909 conn(0x7fc5d40fd950 msgr2=0x7fc5d40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.945+0000 7fc5d3200640 1 -- 172.21.15.18:0/3005541909 shutdown_connections 2024-04-16T17:05:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:25.945+0000 7fc5d3200640 1 -- 172.21.15.18:0/3005541909 wait complete. 2024-04-16T17:05:25.969 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:05:25.969 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:05:26.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:25 smithi043 ceph-mon[36901]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-16T17:05:26.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 -- 172.21.15.18:0/3143036133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104de0 msgr2=0x7f4528108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 --2- 172.21.15.18:0/3143036133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104de0 0x7f4528108e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4514009860 tx=0x7f451402bb70 comp rx=0 tx=0).stop 2024-04-16T17:05:26.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 -- 172.21.15.18:0/3143036133 shutdown_connections 2024-04-16T17:05:26.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 --2- 172.21.15.18:0/3143036133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4528109500 0x7f45281110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 --2- 172.21.15.18:0/3143036133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104de0 0x7f4528108e20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 --2- 172.21.15.18:0/3143036133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4528104430 0x7f4528104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 -- 172.21.15.18:0/3143036133 >> 172.21.15.18:0/3143036133 conn(0x7f45280ffcb0 msgr2=0x7f45281020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 -- 172.21.15.18:0/3143036133 shutdown_connections 2024-04-16T17:05:26.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.151+0000 7f452dc00640 1 -- 172.21.15.18:0/3143036133 wait complete. 2024-04-16T17:05:26.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.153+0000 7f452dc00640 1 Processor -- start 2024-04-16T17:05:26.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.153+0000 7f452dc00640 1 -- start start 2024-04-16T17:05:26.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.153+0000 7f452dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104430 0x7f45281a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.153+0000 7f452dc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4528109500 0x7f45281a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.153+0000 7f452dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f452819a8f0 0x7f452819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.153+0000 7f452dc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4528114570 con 0x7f4528109500 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.153+0000 7f452dc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45281143f0 con 0x7f452819a8f0 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.153+0000 7f452dc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45281146f0 con 0x7f4528104430 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104430 0x7f45281a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104430 0x7f45281a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57956/0 (socket says 172.21.15.18:57956) 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526a00640 1 -- 172.21.15.18:0/1362444722 learned_addr learned my addr 172.21.15.18:0/1362444722 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526000640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4528109500 0x7f45281a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4527400640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f452819a8f0 0x7f452819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526a00640 1 -- 172.21.15.18:0/1362444722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f452819a8f0 msgr2=0x7f452819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526a00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f452819a8f0 0x7f452819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526a00640 1 -- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4528109500 msgr2=0x7f45281a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526a00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4528109500 0x7f45281a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526a00640 1 -- 172.21.15.18:0/1362444722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452819b410 con 0x7f4528104430 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4527400640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f452819a8f0 0x7f452819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.154+0000 7f4526000640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4528109500 0x7f45281a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.155+0000 7f4526a00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104430 0x7f45281a0740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f451800e9c0 tx=0x7f451800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.155+0000 7f451f400640 1 -- 172.21.15.18:0/1362444722 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451800ccc0 con 0x7f4528104430 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.155+0000 7f451f400640 1 -- 172.21.15.18:0/1362444722 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4518005500 con 0x7f4528104430 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.155+0000 7f451f400640 1 -- 172.21.15.18:0/1362444722 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518010430 con 0x7f4528104430 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.155+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452819b700 con 0x7f4528104430 2024-04-16T17:05:26.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.155+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45281a7900 con 0x7f4528104430 2024-04-16T17:05:26.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.156+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4528069300 con 0x7f4528104430 2024-04-16T17:05:26.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.158+0000 7f451f400640 1 -- 172.21.15.18:0/1362444722 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f45180046c0 con 0x7f4528104430 2024-04-16T17:05:26.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.161+0000 7f451f400640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f450c0a3f50 0x7f450c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.161+0000 7f451f400640 1 -- 172.21.15.18:0/1362444722 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f45180d1250 con 0x7f4528104430 2024-04-16T17:05:26.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.161+0000 7f4526000640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f450c0a3f50 0x7f450c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.162+0000 7f4526000640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f450c0a3f50 0x7f450c0a6410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4514009830 tx=0x7f4514035b00 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:26.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.165+0000 7f451f400640 1 -- 172.21.15.18:0/1362444722 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f451809d300 con 0x7f4528104430 2024-04-16T17:05:26.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.545+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f452819be20 con 0x7f4528104430 2024-04-16T17:05:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.546+0000 7f451f400640 1 -- 172.21.15.18:0/1362444722 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7f451809cc00 con 0x7f4528104430 2024-04-16T17:05:26.548 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:05:26.548 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:05:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:05:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.549+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f450c0a3f50 msgr2=0x7f450c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.549+0000 7f452dc00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f450c0a3f50 0x7f450c0a6410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4514009830 tx=0x7f4514035b00 comp rx=0 tx=0).stop 2024-04-16T17:05:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.549+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104430 msgr2=0x7f45281a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.549+0000 7f452dc00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104430 0x7f45281a0740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f451800e9c0 tx=0x7f451800ee80 comp rx=0 tx=0).stop 2024-04-16T17:05:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.550+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 shutdown_connections 2024-04-16T17:05:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.550+0000 7f452dc00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f450c0a3f50 0x7f450c0a6410 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.550+0000 7f452dc00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f452819a8f0 0x7f452819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.550+0000 7f452dc00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4528109500 0x7f45281a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.550+0000 7f452dc00640 1 --2- 172.21.15.18:0/1362444722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4528104430 0x7f45281a0740 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.550+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 >> 172.21.15.18:0/1362444722 conn(0x7f45280ffcb0 msgr2=0x7f4528102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.551+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 shutdown_connections 2024-04-16T17:05:26.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.551+0000 7f452dc00640 1 -- 172.21.15.18:0/1362444722 wait complete. 2024-04-16T17:05:26.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:05:26.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.765+0000 7fe717e00640 1 -- 172.21.15.18:0/1040288428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe718074cd0 msgr2=0x7fe71810e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.765+0000 7fe717e00640 1 --2- 172.21.15.18:0/1040288428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe718074cd0 0x7fe71810e180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe704009900 tx=0x7fe70402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.766+0000 7fe717e00640 1 -- 172.21.15.18:0/1040288428 shutdown_connections 2024-04-16T17:05:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.766+0000 7fe717e00640 1 --2- 172.21.15.18:0/1040288428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe71810e6c0 0x7fe718110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.766+0000 7fe717e00640 1 --2- 172.21.15.18:0/1040288428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe718074cd0 0x7fe71810e180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.766+0000 7fe717e00640 1 --2- 172.21.15.18:0/1040288428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7180743b0 0x7fe718074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.766+0000 7fe717e00640 1 -- 172.21.15.18:0/1040288428 >> 172.21.15.18:0/1040288428 conn(0x7fe7180fd950 msgr2=0x7fe7180ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.766+0000 7fe717e00640 1 -- 172.21.15.18:0/1040288428 shutdown_connections 2024-04-16T17:05:26.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.766+0000 7fe717e00640 1 -- 172.21.15.18:0/1040288428 wait complete. 2024-04-16T17:05:26.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.767+0000 7fe717e00640 1 Processor -- start 2024-04-16T17:05:26.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.767+0000 7fe717e00640 1 -- start start 2024-04-16T17:05:26.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe717e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7180743b0 0x7fe7181a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe717e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe718074cd0 0x7fe7181a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe717e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe71810e6c0 0x7fe71819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe717e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe718114400 con 0x7fe71810e6c0 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe717e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe718114280 con 0x7fe7180743b0 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe717e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe718114580 con 0x7fe718074cd0 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7180743b0 0x7fe7181a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7180743b0 0x7fe7181a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42044/0 (socket says 172.21.15.18:42044) 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716a00640 1 -- 172.21.15.18:0/507300309 learned_addr learned my addr 172.21.15.18:0/507300309 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716000640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe718074cd0 0x7fe7181a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe717400640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe71810e6c0 0x7fe71819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716a00640 1 -- 172.21.15.18:0/507300309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe718074cd0 msgr2=0x7fe7181a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716a00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe718074cd0 0x7fe7181a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716a00640 1 -- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe71810e6c0 msgr2=0x7fe71819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716a00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe71810e6c0 0x7fe71819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.768+0000 7fe716a00640 1 -- 172.21.15.18:0/507300309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71819afe0 con 0x7fe7180743b0 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.769+0000 7fe717400640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe71810e6c0 0x7fe71819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.769+0000 7fe716a00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7180743b0 0x7fe7181a0660 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e9f0 tx=0x7fe70800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.769+0000 7fe70f400640 1 -- 172.21.15.18:0/507300309 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70800cd80 con 0x7fe7180743b0 2024-04-16T17:05:26.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.769+0000 7fe70f400640 1 -- 172.21.15.18:0/507300309 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe708010910 con 0x7fe7180743b0 2024-04-16T17:05:26.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.769+0000 7fe70f400640 1 -- 172.21.15.18:0/507300309 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708018980 con 0x7fe7180743b0 2024-04-16T17:05:26.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.769+0000 7fe717e00640 1 -- 172.21.15.18:0/507300309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71819b2d0 con 0x7fe7180743b0 2024-04-16T17:05:26.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.769+0000 7fe717e00640 1 -- 172.21.15.18:0/507300309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7181a75e0 con 0x7fe7180743b0 2024-04-16T17:05:26.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.773+0000 7fe70f400640 1 -- 172.21.15.18:0/507300309 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe708010ac0 con 0x7fe7180743b0 2024-04-16T17:05:26.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.774+0000 7fe72041d740 1 Processor -- start 2024-04-16T17:05:26.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.774+0000 7fe72041d740 1 -- start start 2024-04-16T17:05:26.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.774+0000 7fe72041d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a6f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.774+0000 7fe72041d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a77ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.774+0000 7fe72041d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 0x55cb61a7a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.774+0000 7fe72041d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cb619d4020 con 0x55cb61a78410 2024-04-16T17:05:26.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.774+0000 7fe72041d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cb6197a460 con 0x55cb61a6ef10 2024-04-16T17:05:26.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.774+0000 7fe72041d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb619cffe0 con 0x55cb61a6f830 2024-04-16T17:05:26.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe717400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a6f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe717400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a6f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42056/0 (socket says 172.21.15.18:42056) 2024-04-16T17:05:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe717400640 1 -- 172.21.15.18:0/580028438 learned_addr learned my addr 172.21.15.18:0/580028438 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe70f400640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6fc0a3f50 0x7fe6fc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe70f400640 1 -- 172.21.15.18:0/507300309 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe7080d1250 con 0x7fe7180743b0 2024-04-16T17:05:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe716a00640 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a77ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe716a00640 1 -- 172.21.15.18:0/580028438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 msgr2=0x55cb61a6f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe716a00640 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a6f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.775+0000 7fe716a00640 1 -- 172.21.15.18:0/580028438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 msgr2=0x55cb61a7a800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:05:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.776+0000 7fe716a00640 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 0x55cb61a7a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.776+0000 7fe716a00640 1 -- 172.21.15.18:0/580028438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb61982040 con 0x55cb61a6f830 2024-04-16T17:05:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.776+0000 7fe716a00640 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a77ed0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7080ce3b0 tx=0x7fe708009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae8103c4763b647d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.776+0000 7fe717400640 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a6f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.776+0000 7fe703400640 1 -- 172.21.15.18:0/580028438 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708099270 con 0x55cb61a6f830 2024-04-16T17:05:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.776+0000 7fe703400640 1 -- 172.21.15.18:0/580028438 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7080bd0e0 con 0x55cb61a6f830 2024-04-16T17:05:26.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.776+0000 7fe703400640 1 -- 172.21.15.18:0/580028438 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708010430 con 0x55cb61a6f830 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.777+0000 7fe72041d740 1 -- 172.21.15.18:0/580028438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 msgr2=0x55cb61a77ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.777+0000 7fe72041d740 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a77ed0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7080ce3b0 tx=0x7fe708009590 comp rx=0 tx=0).stop 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.777+0000 7fe72041d740 1 -- 172.21.15.18:0/580028438 shutdown_connections 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.777+0000 7fe72041d740 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 0x55cb61a7a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.777+0000 7fe72041d740 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a77ed0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.777+0000 7fe72041d740 1 --2- 172.21.15.18:0/580028438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a6f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.777+0000 7fe72041d740 1 -- 172.21.15.18:0/580028438 >> 172.21.15.18:0/580028438 conn(0x55cb61a69640 msgr2=0x55cb61a69a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.778+0000 7fe717400640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6fc0a3f50 0x7fe6fc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.778+0000 7fe72041d740 1 -- 172.21.15.18:0/580028438 shutdown_connections 2024-04-16T17:05:26.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.778+0000 7fe72041d740 1 -- 172.21.15.18:0/580028438 wait complete. 2024-04-16T17:05:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.778+0000 7fe72041d740 1 -- start start 2024-04-16T17:05:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.779+0000 7fe72041d740 1 Processor -- start 2024-04-16T17:05:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.779+0000 7fe717400640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6fc0a3f50 0x7fe6fc0a6410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe710005fd0 tx=0x7fe710004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.779+0000 7fe72041d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a75460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe72041d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe72041d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 0x55cb61a75ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe716000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe716a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a75460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe717400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 0x55cb61a75ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe716000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a759a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57972/0 (socket says 172.21.15.18:57972) 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe716000640 1 -- 172.21.15.18:0/2670696407 learned_addr learned my addr 172.21.15.18:0/2670696407 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cb61a7b140 con 0x55cb61a78410 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cb6198d830 con 0x55cb61a6ef10 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb61a7b2c0 con 0x55cb61a6f830 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.780+0000 7fe72041d740 10 client.0 _ll_register_callbacks cb 0x55cb61995150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.781+0000 7fe716a00640 1 -- 172.21.15.18:0/2670696407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 msgr2=0x55cb61a759a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.781+0000 7fe716a00640 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a759a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.781+0000 7fe716a00640 1 -- 172.21.15.18:0/2670696407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 msgr2=0x55cb61a75ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.781+0000 7fe716a00640 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 0x55cb61a75ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.781+0000 7fe716a00640 1 -- 172.21.15.18:0/2670696407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb61991920 con 0x55cb61a6ef10 2024-04-16T17:05:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.781+0000 7fe716a00640 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a75460 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55cb61a6e9e0 tx=0x7fe70809cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcec42c1c278052e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:26.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.781+0000 7fe6f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:26.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.782+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2670696407 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70809c050 con 0x55cb61a6ef10 2024-04-16T17:05:26.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.782+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2670696407 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe708018d30 con 0x55cb61a6ef10 2024-04-16T17:05:26.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.782+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7080bd0e0 con 0x55cb61a6ef10 2024-04-16T17:05:26.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.782+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2670696407 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe7080b60d0 con 0x55cb61a6ef10 2024-04-16T17:05:26.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.782+0000 7fe72041d740 10 client.25766 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:26.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.782+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb6195f7f0 con 0x55cb61a6ef10 2024-04-16T17:05:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.783+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2670696407 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7080bd0e0 con 0x55cb61a6ef10 2024-04-16T17:05:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.784+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2670696407 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe708098950 con 0x55cb61a6ef10 2024-04-16T17:05:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.784+0000 7fe72041d740 10 client.25766 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.784+0000 7fe72041d740 10 client.25766 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:05:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.784+0000 7fe72041d740 20 client.25766 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.784+0000 7fe72041d740 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55cb61a88580 0x55cb61a8a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.784+0000 7fe72041d740 4 client.25766 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:26.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.784+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb619e1a40 con 0x55cb61a88580 2024-04-16T17:05:26.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.784+0000 7fe716000640 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55cb61a88580 0x55cb61a8a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.785+0000 7fe716000640 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55cb61a88580 0x55cb61a8a970 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99b11031087a2a5b server_cookie=8648f0f0a4630c4e in_seq=0 out_seq=0 2024-04-16T17:05:26.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.785+0000 7fe6f7e00640 10 client.25766 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:05:26.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.786+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2670696407 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb619e1a40 con 0x55cb61a88580 2024-04-16T17:05:26.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.786+0000 7fe6f7e00640 10 client.25766 handle_command_reply: tid=0 2024-04-16T17:05:26.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.786+0000 7fe72041d740 1 client.25766 shutdown 2024-04-16T17:05:26.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.786+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 msgr2=0x55cb61a75460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.786+0000 7fe72041d740 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a75460 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55cb61a6e9e0 tx=0x7fe70809cd80 comp rx=0 tx=0).stop 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 shutdown_connections 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55cb61a88580 0x55cb61a8a970 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a78410 0x55cb61a75ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 1 --2- 172.21.15.18:0/2670696407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a6ef10 0x55cb61a75460 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe6f7e00640 0 client.25766 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 >> 172.21.15.18:0/2670696407 conn(0x55cb61a69640 msgr2=0x55cb61a6b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 shutdown_connections 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 1 -- 172.21.15.18:0/2670696407 wait complete. 2024-04-16T17:05:26.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.787+0000 7fe72041d740 20 client.25766 trim_cache size 0 max 16384 2024-04-16T17:05:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.804+0000 7fe72041d740 1 Processor -- start 2024-04-16T17:05:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.804+0000 7fe72041d740 1 -- start start 2024-04-16T17:05:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe72041d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6ef10 0x55cb61a77b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe72041d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6f830 0x55cb61a746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe72041d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe72041d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe7181afb50 con 0x55cb61a6f830 2024-04-16T17:05:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe72041d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe7181ad420 con 0x55cb61a78410 2024-04-16T17:05:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe72041d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7181ab270 con 0x55cb61a6ef10 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe716000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6f830 0x55cb61a746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe716a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6ef10 0x55cb61a77b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe716000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6f830 0x55cb61a746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41626/0 (socket says 172.21.15.18:41626) 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe716000640 1 -- 172.21.15.18:0/1112134720 learned_addr learned my addr 172.21.15.18:0/1112134720 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.805+0000 7fe717400640 1 --2- 172.21.15.18:0/1112134720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe716a00640 1 -- 172.21.15.18:0/1112134720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 msgr2=0x55cb61a8d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe716a00640 1 --2- 172.21.15.18:0/1112134720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe716a00640 1 -- 172.21.15.18:0/1112134720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6f830 msgr2=0x55cb61a746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe716a00640 1 --2- 172.21.15.18:0/1112134720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6f830 0x55cb61a746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe716a00640 1 -- 172.21.15.18:0/1112134720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb616e63f0 con 0x55cb61a6ef10 2024-04-16T17:05:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe716a00640 1 --2- 172.21.15.18:0/1112134720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6ef10 0x55cb61a77b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe7080a1170 tx=0x7fe708000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7b24bc336836a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:26.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe702a00640 1 -- 172.21.15.18:0/1112134720 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708098ea0 con 0x55cb61a6ef10 2024-04-16T17:05:26.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe702a00640 1 -- 172.21.15.18:0/1112134720 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7080bd0b0 con 0x55cb61a6ef10 2024-04-16T17:05:26.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.806+0000 7fe702a00640 1 -- 172.21.15.18:0/1112134720 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70809bac0 con 0x55cb61a6ef10 2024-04-16T17:05:26.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 -- 172.21.15.18:0/1112134720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6ef10 msgr2=0x55cb61a77b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 --2- 172.21.15.18:0/1112134720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6ef10 0x55cb61a77b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe7080a1170 tx=0x7fe708000bc0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 -- 172.21.15.18:0/1112134720 shutdown_connections 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 --2- 172.21.15.18:0/1112134720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 --2- 172.21.15.18:0/1112134720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6f830 0x55cb61a746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 --2- 172.21.15.18:0/1112134720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6ef10 0x55cb61a77b90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 -- 172.21.15.18:0/1112134720 >> 172.21.15.18:0/1112134720 conn(0x55cb61a69640 msgr2=0x55cb61a83da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 -- 172.21.15.18:0/1112134720 shutdown_connections 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.807+0000 7fe72041d740 1 -- 172.21.15.18:0/1112134720 wait complete. 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.808+0000 7fe72041d740 1 -- start start 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.808+0000 7fe72041d740 1 Processor -- start 2024-04-16T17:05:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe72041d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6ef10 0x55cb61a8b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe72041d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a8b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe72041d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe72041d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe7181ab8e0 con 0x55cb61a6ef10 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe72041d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe7181acd50 con 0x55cb61a78410 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe72041d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7181abfb0 con 0x55cb61a6f830 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe72041d740 10 client.0 _ll_register_callbacks cb 0x7fe708009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe716a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6ef10 0x55cb61a8b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe716a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6ef10 0x55cb61a8b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41638/0 (socket says 172.21.15.18:41638) 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe716a00640 1 -- 172.21.15.18:0/2449518227 learned_addr learned my addr 172.21.15.18:0/2449518227 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.809+0000 7fe717400640 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe716a00640 1 -- 172.21.15.18:0/2449518227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 msgr2=0x55cb61a8b970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:05:26.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe716a00640 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a8b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe716a00640 1 -- 172.21.15.18:0/2449518227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 msgr2=0x55cb61a8beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:26.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe716a00640 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe716a00640 1 -- 172.21.15.18:0/2449518227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb617b07d0 con 0x55cb61a6ef10 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe717400640 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe716a00640 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6ef10 0x55cb61a8b430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe7080097b0 tx=0x7fe7080108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933d0f7fce975d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe6f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2449518227 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708098ea0 con 0x55cb61a6ef10 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2449518227 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe70809af00 con 0x55cb61a6ef10 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.810+0000 7fe72041d740 1 -- 172.21.15.18:0/2449518227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7080bd0b0 con 0x55cb61a6ef10 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2449518227 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe7080a5e20 con 0x55cb61a6ef10 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe72041d740 10 client.15636 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe72041d740 1 -- 172.21.15.18:0/2449518227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb617b07d0 con 0x55cb61a6ef10 2024-04-16T17:05:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2449518227 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7080bd0b0 con 0x55cb61a6ef10 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2449518227 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe7080ce7d0 con 0x55cb61a6ef10 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe72041d740 10 client.15636 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe72041d740 10 client.15636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe72041d740 20 client.15636 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe72041d740 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55cb61aeec30 0x55cb61af1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe72041d740 4 client.15636 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.811+0000 7fe72041d740 1 -- 172.21.15.18:0/2449518227 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb619db8b0 con 0x55cb61aeec30 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.812+0000 7fe716000640 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55cb61aeec30 0x55cb61af1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.812+0000 7fe6f7e00640 10 client.15636 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.812+0000 7fe716000640 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55cb61aeec30 0x55cb61af1020 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d74c3edc40386e92 server_cookie=f99b9522444d95a5 in_seq=0 out_seq=0 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe6f7e00640 1 -- 172.21.15.18:0/2449518227 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb619db8b0 con 0x55cb61aeec30 2024-04-16T17:05:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe6f7e00640 10 client.15636 handle_command_reply: tid=0 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 client.15636 shutdown 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 -- 172.21.15.18:0/2449518227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6ef10 msgr2=0x55cb61a8b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6ef10 0x55cb61a8b430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe7080097b0 tx=0x7fe7080108a0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 -- 172.21.15.18:0/2449518227 shutdown_connections 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55cb61aeec30 0x55cb61af1020 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb61a78410 0x55cb61a8beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb61a6f830 0x55cb61a8b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe6f7e00640 0 client.15636 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 --2- 172.21.15.18:0/2449518227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb61a6ef10 0x55cb61a8b430 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.813+0000 7fe72041d740 1 -- 172.21.15.18:0/2449518227 >> 172.21.15.18:0/2449518227 conn(0x55cb61a69640 msgr2=0x55cb61a75400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.814+0000 7fe72041d740 1 -- 172.21.15.18:0/2449518227 shutdown_connections 2024-04-16T17:05:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.814+0000 7fe72041d740 1 -- 172.21.15.18:0/2449518227 wait complete. 2024-04-16T17:05:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.814+0000 7fe72041d740 20 client.15636 trim_cache size 0 max 16384 2024-04-16T17:05:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 -- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6fc0a3f50 msgr2=0x7fe6fc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6fc0a3f50 0x7fe6fc0a6410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe710005fd0 tx=0x7fe710004a40 comp rx=0 tx=0).stop 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 -- 172.21.15.18:0/507300309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7180743b0 msgr2=0x7fe7181a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7180743b0 0x7fe7181a0660 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e9f0 tx=0x7fe70800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 -- 172.21.15.18:0/507300309 shutdown_connections 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6fc0a3f50 0x7fe6fc0a6410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe71810e6c0 0x7fe71819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe718074cd0 0x7fe7181a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 --2- 172.21.15.18:0/507300309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7180743b0 0x7fe7181a0660 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 -- 172.21.15.18:0/507300309 >> 172.21.15.18:0/507300309 conn(0x7fe7180fd950 msgr2=0x7fe7180fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 -- 172.21.15.18:0/507300309 shutdown_connections 2024-04-16T17:05:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:26.815+0000 7fe717e00640 1 -- 172.21.15.18:0/507300309 wait complete. 2024-04-16T17:05:26.839 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:26 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1362444722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:26.843 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:05:26.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:26 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1362444722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:27.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:26 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1362444722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:27.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 -- 172.21.15.18:0/2574844708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4541016f0 msgr2=0x7fc454105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 --2- 172.21.15.18:0/2574844708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4541016f0 0x7fc454105bb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc440009900 tx=0x7fc44002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 -- 172.21.15.18:0/2574844708 shutdown_connections 2024-04-16T17:05:27.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 --2- 172.21.15.18:0/2574844708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4541061d0 0x7fc454106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 --2- 172.21.15.18:0/2574844708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4541016f0 0x7fc454105bb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 --2- 172.21.15.18:0/2574844708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc454100dd0 0x7fc4541011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 -- 172.21.15.18:0/2574844708 >> 172.21.15.18:0/2574844708 conn(0x7fc454077ab0 msgr2=0x7fc4540fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 -- 172.21.15.18:0/2574844708 shutdown_connections 2024-04-16T17:05:27.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.029+0000 7fc453200640 1 -- 172.21.15.18:0/2574844708 wait complete. 2024-04-16T17:05:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.030+0000 7fc453200640 1 Processor -- start 2024-04-16T17:05:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.030+0000 7fc453200640 1 -- start start 2024-04-16T17:05:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.030+0000 7fc453200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc454100dd0 0x7fc45419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.030+0000 7fc453200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4541016f0 0x7fc45419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.030+0000 7fc453200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4541061d0 0x7fc4541a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.031+0000 7fc453200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541187f0 con 0x7fc4541016f0 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.031+0000 7fc453200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc454118670 con 0x7fc454100dd0 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.031+0000 7fc453200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc454118970 con 0x7fc4541061d0 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.031+0000 7fc451e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc454100dd0 0x7fc45419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.031+0000 7fc451e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc454100dd0 0x7fc45419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:43708/0 (socket says 172.21.15.18:43708) 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.031+0000 7fc451e00640 1 -- 172.21.15.18:0/1935481906 learned_addr learned my addr 172.21.15.18:0/1935481906 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.031+0000 7fc452800640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4541061d0 0x7fc4541a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.031+0000 7fc451400640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4541016f0 0x7fc45419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.032+0000 7fc451400640 1 -- 172.21.15.18:0/1935481906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4541061d0 msgr2=0x7fc4541a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.032+0000 7fc451400640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4541061d0 0x7fc4541a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.032+0000 7fc451400640 1 -- 172.21.15.18:0/1935481906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc454100dd0 msgr2=0x7fc45419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.032+0000 7fc451400640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc454100dd0 0x7fc45419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.032+0000 7fc451400640 1 -- 172.21.15.18:0/1935481906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4541a3e00 con 0x7fc4541016f0 2024-04-16T17:05:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.032+0000 7fc451e00640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc454100dd0 0x7fc45419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:27.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.032+0000 7fc451400640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4541016f0 0x7fc45419f350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc44002c200 tx=0x7fc4400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.033+0000 7fc44aa00640 1 -- 172.21.15.18:0/1935481906 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440042020 con 0x7fc4541016f0 2024-04-16T17:05:27.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.033+0000 7fc44aa00640 1 -- 172.21.15.18:0/1935481906 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4400054e0 con 0x7fc4541016f0 2024-04-16T17:05:27.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.033+0000 7fc44aa00640 1 -- 172.21.15.18:0/1935481906 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400039b0 con 0x7fc4541016f0 2024-04-16T17:05:27.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.033+0000 7fc453200640 1 -- 172.21.15.18:0/1935481906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4541a4090 con 0x7fc4541016f0 2024-04-16T17:05:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.033+0000 7fc453200640 1 -- 172.21.15.18:0/1935481906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4541ab980 con 0x7fc4541016f0 2024-04-16T17:05:27.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.038+0000 7fc45b79b740 1 Processor -- start 2024-04-16T17:05:27.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.038+0000 7fc45b79b740 1 -- start start 2024-04-16T17:05:27.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.038+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.038+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e369ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.038+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e36c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.038+0000 7fc45b79b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56114e2c6020 con 0x56114e36a410 2024-04-16T17:05:27.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.038+0000 7fc45b79b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56114e26c460 con 0x56114e361830 2024-04-16T17:05:27.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.038+0000 7fc45b79b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56114e2c1fe0 con 0x56114e360f10 2024-04-16T17:05:27.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.039+0000 7fc44aa00640 1 -- 172.21.15.18:0/1935481906 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fc440045020 con 0x7fc4541016f0 2024-04-16T17:05:27.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.039+0000 7fc452800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.039+0000 7fc452800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55942/0 (socket says 172.21.15.18:55942) 2024-04-16T17:05:27.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.039+0000 7fc452800640 1 -- 172.21.15.18:0/2786668743 learned_addr learned my addr 172.21.15.18:0/2786668743 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.039+0000 7fc451e00640 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e369ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.039+0000 7fc451400640 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e36c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.040+0000 7fc451e00640 1 -- 172.21.15.18:0/2786668743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 msgr2=0x56114e3612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.040+0000 7fc451e00640 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.040+0000 7fc451e00640 1 -- 172.21.15.18:0/2786668743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 msgr2=0x56114e36c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.040+0000 7fc451e00640 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e36c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.040+0000 7fc451e00640 1 -- 172.21.15.18:0/2786668743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56114e274040 con 0x56114e361830 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.040+0000 7fc452800640 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.040+0000 7fc451400640 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e36c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.040+0000 7fc451e00640 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e369ed0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc444011510 tx=0x7fc4440119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34323ac14a96b946 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.041+0000 7fc43f000640 1 -- 172.21.15.18:0/2786668743 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44401e070 con 0x56114e361830 2024-04-16T17:05:27.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.041+0000 7fc43f000640 1 -- 172.21.15.18:0/2786668743 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44400fb60 con 0x56114e361830 2024-04-16T17:05:27.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.041+0000 7fc43f000640 1 -- 172.21.15.18:0/2786668743 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444019c50 con 0x56114e361830 2024-04-16T17:05:27.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.041+0000 7fc44aa00640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4380a3f50 0x7fc4380a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc44aa00640 1 -- 172.21.15.18:0/1935481906 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc4400f2840 con 0x7fc4541016f0 2024-04-16T17:05:27.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc45b79b740 1 -- 172.21.15.18:0/2786668743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 msgr2=0x56114e369ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc45b79b740 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e369ed0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc444011510 tx=0x7fc4440119d0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc452800640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4380a3f50 0x7fc4380a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc45b79b740 1 -- 172.21.15.18:0/2786668743 shutdown_connections 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc45b79b740 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e36c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc45b79b740 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e369ed0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc45b79b740 1 --2- 172.21.15.18:0/2786668743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.042+0000 7fc45b79b740 1 -- 172.21.15.18:0/2786668743 >> 172.21.15.18:0/2786668743 conn(0x56114e35b640 msgr2=0x56114e35ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.043+0000 7fc45b79b740 1 -- 172.21.15.18:0/2786668743 shutdown_connections 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.043+0000 7fc452800640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4380a3f50 0x7fc4380a6410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009d60 tx=0x7fc44c009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.043+0000 7fc45b79b740 1 -- 172.21.15.18:0/2786668743 wait complete. 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.043+0000 7fc45b79b740 1 -- start start 2024-04-16T17:05:27.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.044+0000 7fc45b79b740 1 Processor -- start 2024-04-16T17:05:27.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.044+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.044+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e3678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.044+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e367e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.044+0000 7fc45b79b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56114e36d200 con 0x56114e36a410 2024-04-16T17:05:27.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.044+0000 7fc45b79b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56114e27f830 con 0x56114e361830 2024-04-16T17:05:27.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.044+0000 7fc45b79b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56114e36d380 con 0x56114e360f10 2024-04-16T17:05:27.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.044+0000 7fc45b79b740 10 client.0 _ll_register_callbacks cb 0x56114e287150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:27.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e3678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e3678f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:43724/0 (socket says 172.21.15.18:43724) 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 -- 172.21.15.18:0/3283103663 learned_addr learned my addr 172.21.15.18:0/3283103663 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc452800640 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e367e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 -- 172.21.15.18:0/3283103663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 msgr2=0x56114e3673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 -- 172.21.15.18:0/3283103663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 msgr2=0x56114e367e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e367e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 -- 172.21.15.18:0/3283103663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56114e283920 con 0x56114e361830 2024-04-16T17:05:27.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc451400640 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e3678f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc440006010 tx=0x7fc44002ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfecdf5270c33b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc433400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.045+0000 7fc433400640 1 -- 172.21.15.18:0/3283103663 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400be0d0 con 0x56114e361830 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 1 -- 172.21.15.18:0/3283103663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44400fb60 con 0x56114e361830 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc433400640 1 -- 172.21.15.18:0/3283103663 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc4400c6ce0 con 0x56114e361830 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc433400640 1 -- 172.21.15.18:0/3283103663 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4400bdc60 con 0x56114e361830 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 10 client.25781 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 1 -- 172.21.15.18:0/3283103663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56114e2517f0 con 0x56114e361830 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc433400640 1 -- 172.21.15.18:0/3283103663 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44400fb60 con 0x56114e361830 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc433400640 1 -- 172.21.15.18:0/3283103663 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4400f2500 con 0x56114e361830 2024-04-16T17:05:27.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 10 client.25781 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:27.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 10 client.25781 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:05:27.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 20 client.25781 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:27.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56114e37a710 0x56114e37cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 4 client.25781 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:27.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.046+0000 7fc45b79b740 1 -- 172.21.15.18:0/3283103663 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56114e2d3a40 con 0x56114e37a710 2024-04-16T17:05:27.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.047+0000 7fc451e00640 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56114e37a710 0x56114e37cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.049+0000 7fc451e00640 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56114e37a710 0x56114e37cb30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff445158bf63e6b4 server_cookie=7ce92713ea665027 in_seq=0 out_seq=0 2024-04-16T17:05:27.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.049+0000 7fc433400640 10 client.25781 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:05:27.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc433400640 1 -- 172.21.15.18:0/3283103663 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56114e2d3a40 con 0x56114e37a710 2024-04-16T17:05:27.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc433400640 10 client.25781 handle_command_reply: tid=0 2024-04-16T17:05:27.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 client.25781 shutdown 2024-04-16T17:05:27.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 -- 172.21.15.18:0/3283103663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 msgr2=0x56114e3678f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e3678f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc440006010 tx=0x7fc44002ce80 comp rx=0 tx=0).stop 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 -- 172.21.15.18:0/3283103663 shutdown_connections 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56114e37a710 0x56114e37cb30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e367e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e3678f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 --2- 172.21.15.18:0/3283103663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e3673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc45b79b740 1 -- 172.21.15.18:0/3283103663 >> 172.21.15.18:0/3283103663 conn(0x56114e35b640 msgr2=0x56114e35d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.050+0000 7fc433400640 0 client.25781 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.051+0000 7fc45b79b740 1 -- 172.21.15.18:0/3283103663 shutdown_connections 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.051+0000 7fc45b79b740 1 -- 172.21.15.18:0/3283103663 wait complete. 2024-04-16T17:05:27.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.051+0000 7fc45b79b740 20 client.25781 trim_cache size 0 max 16384 2024-04-16T17:05:27.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.065+0000 7fc45b79b740 1 Processor -- start 2024-04-16T17:05:27.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc45b79b740 1 -- start start 2024-04-16T17:05:27.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e360f10 0x56114e3760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e367c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e36a410 0x56114e382440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc45b79b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b8290 con 0x56114e360f10 2024-04-16T17:05:27.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc45b79b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b5b60 con 0x56114e361830 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc45b79b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b39b0 con 0x56114e36a410 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc451400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e367c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc451400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e367c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:43738/0 (socket says 172.21.15.18:43738) 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc451400640 1 -- 172.21.15.18:0/1699886930 learned_addr learned my addr 172.21.15.18:0/1699886930 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc452800640 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e36a410 0x56114e382440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.066+0000 7fc451e00640 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e360f10 0x56114e3760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc451400640 1 -- 172.21.15.18:0/1699886930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e36a410 msgr2=0x56114e382440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc451400640 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e36a410 0x56114e382440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc451400640 1 -- 172.21.15.18:0/1699886930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e360f10 msgr2=0x56114e3760f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc451400640 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e360f10 0x56114e3760f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc451400640 1 -- 172.21.15.18:0/1699886930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56114e3621e0 con 0x56114e361830 2024-04-16T17:05:27.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc451e00640 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e360f10 0x56114e3760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:27.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc452800640 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e36a410 0x56114e382440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:27.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc451400640 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e367c50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc4400be000 tx=0x7fc4400045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72901a30db12b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc43e600640 1 -- 172.21.15.18:0/1699886930 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400bde40 con 0x56114e361830 2024-04-16T17:05:27.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc43e600640 1 -- 172.21.15.18:0/1699886930 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44009f070 con 0x56114e361830 2024-04-16T17:05:27.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.067+0000 7fc43e600640 1 -- 172.21.15.18:0/1699886930 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400bda80 con 0x56114e361830 2024-04-16T17:05:27.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.068+0000 7fc45b79b740 1 -- 172.21.15.18:0/1699886930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 msgr2=0x56114e367c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.068+0000 7fc45b79b740 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e367c50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc4400be000 tx=0x7fc4400045e0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.068+0000 7fc45b79b740 1 -- 172.21.15.18:0/1699886930 shutdown_connections 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.068+0000 7fc45b79b740 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e36a410 0x56114e382440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.068+0000 7fc45b79b740 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e367c50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.068+0000 7fc45b79b740 1 --2- 172.21.15.18:0/1699886930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e360f10 0x56114e3760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.068+0000 7fc45b79b740 1 -- 172.21.15.18:0/1699886930 >> 172.21.15.18:0/1699886930 conn(0x56114e35b640 msgr2=0x56114e35a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.069+0000 7fc45b79b740 1 -- 172.21.15.18:0/1699886930 shutdown_connections 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.069+0000 7fc45b79b740 1 -- 172.21.15.18:0/1699886930 wait complete. 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.069+0000 7fc45b79b740 1 -- start start 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.069+0000 7fc45b79b740 1 Processor -- start 2024-04-16T17:05:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e36e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e36ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc45b79b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e3dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc45b79b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b4020 con 0x56114e36a410 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc45b79b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b5490 con 0x56114e361830 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc45b79b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541b46f0 con 0x56114e360f10 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc45b79b740 10 client.0 _ll_register_callbacks cb 0x7fc440009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc451400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e36ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc451400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e36ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:43748/0 (socket says 172.21.15.18:43748) 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc451400640 1 -- 172.21.15.18:0/4036802880 learned_addr learned my addr 172.21.15.18:0/4036802880 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc451e00640 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e36e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.070+0000 7fc452800640 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e3dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc451400640 1 -- 172.21.15.18:0/4036802880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 msgr2=0x56114e36e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc451400640 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e36e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc451400640 1 -- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 msgr2=0x56114e3dfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc451400640 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e3dfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc451400640 1 -- 172.21.15.18:0/4036802880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56114dfd83f0 con 0x56114e361830 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc452800640 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e3dfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc451400640 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e36ec80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc4400bc5b0 tx=0x7fc4400bda30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4dc3cb77abf40d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc433400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc433400640 1 -- 172.21.15.18:0/4036802880 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400bde40 con 0x56114e361830 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc45b79b740 1 -- 172.21.15.18:0/4036802880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44009f070 con 0x56114e361830 2024-04-16T17:05:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.071+0000 7fc433400640 1 -- 172.21.15.18:0/4036802880 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc4400f23e0 con 0x56114e361830 2024-04-16T17:05:27.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.072+0000 7fc433400640 1 -- 172.21.15.18:0/4036802880 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc440003e60 con 0x56114e361830 2024-04-16T17:05:27.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.072+0000 7fc45b79b740 10 client.25787 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:27.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.072+0000 7fc45b79b740 1 -- 172.21.15.18:0/4036802880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56114e0a27d0 con 0x56114e361830 2024-04-16T17:05:27.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.072+0000 7fc433400640 1 -- 172.21.15.18:0/4036802880 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44009f070 con 0x56114e361830 2024-04-16T17:05:27.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.072+0000 7fc433400640 1 -- 172.21.15.18:0/4036802880 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4400ba050 con 0x56114e361830 2024-04-16T17:05:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.073+0000 7fc45b79b740 10 client.25787 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.073+0000 7fc45b79b740 10 client.25787 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:05:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.073+0000 7fc45b79b740 20 client.25787 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.073+0000 7fc45b79b740 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56114e3e1350 0x56114e3e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.073+0000 7fc45b79b740 4 client.25787 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:05:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.073+0000 7fc45b79b740 1 -- 172.21.15.18:0/4036802880 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56114e2cd8b0 con 0x56114e3e1350 2024-04-16T17:05:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.073+0000 7fc451e00640 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56114e3e1350 0x56114e3e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc451e00640 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56114e3e1350 0x56114e3e3740 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0c5a81d7e2d17aa server_cookie=475cdb64a51d2859 in_seq=0 out_seq=0 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc433400640 10 client.25787 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc433400640 1 -- 172.21.15.18:0/4036802880 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc44401f040 con 0x56114e3e1350 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc433400640 10 client.25787 handle_command_reply: tid=0 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 client.25787 shutdown 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 -- 172.21.15.18:0/4036802880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 msgr2=0x56114e36ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e36ec80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc4400bc5b0 tx=0x7fc4400bda30 comp rx=0 tx=0).stop 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 -- 172.21.15.18:0/4036802880 shutdown_connections 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56114e3e1350 0x56114e3e3740 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56114e36a410 0x56114e3dfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56114e361830 0x56114e36ec80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 --2- 172.21.15.18:0/4036802880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56114e360f10 0x56114e36e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc433400640 0 client.25787 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:05:27.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.074+0000 7fc45b79b740 1 -- 172.21.15.18:0/4036802880 >> 172.21.15.18:0/4036802880 conn(0x56114e35b640 msgr2=0x56114e35c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.075+0000 7fc45b79b740 1 -- 172.21.15.18:0/4036802880 shutdown_connections 2024-04-16T17:05:27.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.075+0000 7fc45b79b740 1 -- 172.21.15.18:0/4036802880 wait complete. 2024-04-16T17:05:27.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.075+0000 7fc45b79b740 20 client.25787 trim_cache size 0 max 16384 2024-04-16T17:05:27.081 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:05:27.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.076+0000 7fc453200640 1 -- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4380a3f50 msgr2=0x7fc4380a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.076+0000 7fc453200640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4380a3f50 0x7fc4380a6410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009d60 tx=0x7fc44c009500 comp rx=0 tx=0).stop 2024-04-16T17:05:27.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.076+0000 7fc453200640 1 -- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4541016f0 msgr2=0x7fc45419f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.076+0000 7fc453200640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4541016f0 0x7fc45419f350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc44002c200 tx=0x7fc4400042f0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.076+0000 7fc453200640 1 -- 172.21.15.18:0/1935481906 shutdown_connections 2024-04-16T17:05:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.077+0000 7fc453200640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4380a3f50 0x7fc4380a6410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.077+0000 7fc453200640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4541061d0 0x7fc4541a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.077+0000 7fc453200640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4541016f0 0x7fc45419f350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.077+0000 7fc453200640 1 --2- 172.21.15.18:0/1935481906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc454100dd0 0x7fc45419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.077+0000 7fc453200640 1 -- 172.21.15.18:0/1935481906 >> 172.21.15.18:0/1935481906 conn(0x7fc454077ab0 msgr2=0x7fc454103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.077+0000 7fc453200640 1 -- 172.21.15.18:0/1935481906 shutdown_connections 2024-04-16T17:05:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.077+0000 7fc453200640 1 -- 172.21.15.18:0/1935481906 wait complete. 2024-04-16T17:05:27.096 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:05:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.322+0000 7f2bd7200640 1 -- 172.21.15.18:0/3152499314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cf0 msgr2=0x7f2bd810e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.322+0000 7f2bd7200640 1 --2- 172.21.15.18:0/3152499314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cf0 0x7f2bd810e160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009900 tx=0x7f2bc402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.323+0000 7f2bd7200640 1 -- 172.21.15.18:0/3152499314 shutdown_connections 2024-04-16T17:05:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.323+0000 7f2bd7200640 1 --2- 172.21.15.18:0/3152499314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd810e6a0 0x7f2bd8110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.323+0000 7f2bd7200640 1 --2- 172.21.15.18:0/3152499314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cf0 0x7f2bd810e160 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.323+0000 7f2bd7200640 1 --2- 172.21.15.18:0/3152499314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd80743d0 0x7f2bd80747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.323+0000 7f2bd7200640 1 -- 172.21.15.18:0/3152499314 >> 172.21.15.18:0/3152499314 conn(0x7f2bd80fd950 msgr2=0x7f2bd80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.323+0000 7f2bd7200640 1 -- 172.21.15.18:0/3152499314 shutdown_connections 2024-04-16T17:05:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.323+0000 7f2bd7200640 1 -- 172.21.15.18:0/3152499314 wait complete. 2024-04-16T17:05:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.324+0000 7f2bd7200640 1 Processor -- start 2024-04-16T17:05:27.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd7200640 1 -- start start 2024-04-16T17:05:27.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd80743d0 0x7f2bd81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cf0 0x7f2bd81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd810e6a0 0x7f2bd819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd7200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8114470 con 0x7f2bd80743d0 2024-04-16T17:05:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81142f0 con 0x7f2bd810e6a0 2024-04-16T17:05:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81145f0 con 0x7f2bd8074cf0 2024-04-16T17:05:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd80743d0 0x7f2bd81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd80743d0 0x7f2bd81a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56394/0 (socket says 172.21.15.18:56394) 2024-04-16T17:05:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd5e00640 1 -- 172.21.15.18:0/1781871653 learned_addr learned my addr 172.21.15.18:0/1781871653 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.325+0000 7f2bd5400640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cf0 0x7f2bd81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.326+0000 7f2bd6800640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd810e6a0 0x7f2bd819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.326+0000 7f2bd5e00640 1 -- 172.21.15.18:0/1781871653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cf0 msgr2=0x7f2bd81a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.326+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cf0 0x7f2bd81a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.326+0000 7f2bd5e00640 1 -- 172.21.15.18:0/1781871653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd810e6a0 msgr2=0x7f2bd819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.326+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd810e6a0 0x7f2bd819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.326+0000 7f2bd5e00640 1 -- 172.21.15.18:0/1781871653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd819aff0 con 0x7f2bd80743d0 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.326+0000 7f2bd6800640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd810e6a0 0x7f2bd819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.327+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd80743d0 0x7f2bd81a0670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c920 tx=0x7f2bc800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.327+0000 7f2bcea00640 1 -- 172.21.15.18:0/1781871653 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8007c70 con 0x7f2bd80743d0 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.327+0000 7f2bd7200640 1 -- 172.21.15.18:0/1781871653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd819b2e0 con 0x7f2bd80743d0 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.327+0000 7f2bcea00640 1 -- 172.21.15.18:0/1781871653 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc8007e10 con 0x7f2bd80743d0 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.327+0000 7f2bcea00640 1 -- 172.21.15.18:0/1781871653 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80188e0 con 0x7f2bd80743d0 2024-04-16T17:05:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.327+0000 7f2bd7200640 1 -- 172.21.15.18:0/1781871653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81a75f0 con 0x7f2bd80743d0 2024-04-16T17:05:27.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.331+0000 7f2bcea00640 1 -- 172.21.15.18:0/1781871653 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f2bc8018a80 con 0x7f2bd80743d0 2024-04-16T17:05:27.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.332+0000 7f2bdfe00740 1 Processor -- start 2024-04-16T17:05:27.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.332+0000 7f2bdfe00740 1 -- start start 2024-04-16T17:05:27.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019674f10 0x5590196752f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 0x55901967ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x559019680800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5590195da020 con 0x559019675830 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559019580460 con 0x559019674f10 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5590195d5fe0 con 0x55901967e410 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bcea00640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3f50 0x7f2bbc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x559019680800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x559019680800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55974/0 (socket says 172.21.15.18:55974) 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bd5400640 1 -- 172.21.15.18:0/3987371554 learned_addr learned my addr 172.21.15.18:0/3987371554 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bcea00640 1 -- 172.21.15.18:0/1781871653 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2bc80d0eb0 con 0x7f2bd80743d0 2024-04-16T17:05:27.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.333+0000 7f2bd6800640 1 --2- 172.21.15.18:0/3987371554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019674f10 0x5590196752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.334+0000 7f2bd5400640 1 -- 172.21.15.18:0/3987371554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019674f10 msgr2=0x5590196752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.334+0000 7f2bd5400640 1 --2- 172.21.15.18:0/3987371554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019674f10 0x5590196752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.334+0000 7f2bd5400640 1 -- 172.21.15.18:0/3987371554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 msgr2=0x55901967ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.334+0000 7f2bd5400640 1 --2- 172.21.15.18:0/3987371554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 0x55901967ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.334+0000 7f2bd5400640 1 -- 172.21.15.18:0/3987371554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559019588040 con 0x55901967e410 2024-04-16T17:05:27.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.334+0000 7f2bd6800640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3f50 0x7f2bbc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.334+0000 7f2bd5400640 1 --2- 172.21.15.18:0/3987371554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x559019680800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009230 tx=0x7f2bc402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a992ef6834fe2731 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.335+0000 7f2bc3400640 1 -- 172.21.15.18:0/3987371554 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4043070 con 0x55901967e410 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.335+0000 7f2bc3400640 1 -- 172.21.15.18:0/3987371554 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc402c910 con 0x55901967e410 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.335+0000 7f2bc3400640 1 -- 172.21.15.18:0/3987371554 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004c60 con 0x55901967e410 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3987371554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 msgr2=0x559019680800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3987371554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x559019680800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009230 tx=0x7f2bc402c660 comp rx=0 tx=0).stop 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bd6800640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3f50 0x7f2bbc0a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0005c40 tx=0x7f2bd0009f00 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3987371554 shutdown_connections 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3987371554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x559019680800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3987371554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 0x55901967ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3987371554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019674f10 0x5590196752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3987371554 >> 172.21.15.18:0/3987371554 conn(0x55901966f640 msgr2=0x55901966fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3987371554 shutdown_connections 2024-04-16T17:05:27.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.336+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3987371554 wait complete. 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.337+0000 7f2bdfe00740 1 -- start start 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.337+0000 7f2bdfe00740 1 Processor -- start 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.337+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901967b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.337+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 0x55901967b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.337+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x55901967be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.337+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5590196811a0 con 0x559019674f10 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.337+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559019593830 con 0x559019675830 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.337+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559019681320 con 0x55901967e410 2024-04-16T17:05:27.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.338+0000 7f2bdfe00740 10 client.0 _ll_register_callbacks cb 0x55901959b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.338+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901967b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.338+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901967b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56422/0 (socket says 172.21.15.18:56422) 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.338+0000 7f2bd5e00640 1 -- 172.21.15.18:0/3611627640 learned_addr learned my addr 172.21.15.18:0/3611627640 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.338+0000 7f2bd5400640 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 0x55901967b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.338+0000 7f2bd5e00640 1 -- 172.21.15.18:0/3611627640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 msgr2=0x55901967be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x55901967be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bd5e00640 1 -- 172.21.15.18:0/3611627640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 msgr2=0x55901967b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 0x55901967b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bd5e00640 1 -- 172.21.15.18:0/3611627640 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559019597920 con 0x559019674f10 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901967b3f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80ce100 tx=0x7f2bc809d020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60b8b56e7a64203 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bb7e00640 1 -- 172.21.15.18:0/3611627640 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80a54d0 con 0x559019674f10 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bb7e00640 1 -- 172.21.15.18:0/3611627640 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2bc80870b0 con 0x559019674f10 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.339+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3611627640 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc402c910 con 0x559019674f10 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.340+0000 7f2bb7e00640 1 -- 172.21.15.18:0/3611627640 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bc80a5040 con 0x559019674f10 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.340+0000 7f2bdfe00740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.340+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3611627640 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590195657f0 con 0x559019674f10 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.340+0000 7f2bb7e00640 1 -- 172.21.15.18:0/3611627640 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc402c910 con 0x559019674f10 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.340+0000 7f2bb7e00640 1 -- 172.21.15.18:0/3611627640 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2bc809bcb0 con 0x559019674f10 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.340+0000 7f2bdfe00740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.340+0000 7f2bdfe00740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:27.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.340+0000 7f2bdfe00740 20 client.15654 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:27.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.341+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559019697010 0x559019699400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.341+0000 7f2bdfe00740 4 client.15654 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:27.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.341+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3611627640 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590195e7a40 con 0x559019697010 2024-04-16T17:05:27.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.341+0000 7f2bd5400640 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559019697010 0x559019699400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.342+0000 7f2bd5400640 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559019697010 0x559019699400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c563d78f26360c1 server_cookie=729078350b479c14 in_seq=0 out_seq=0 2024-04-16T17:05:27.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.342+0000 7f2bb7e00640 10 client.15654 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:27.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.342+0000 7f2bb7e00640 1 -- 172.21.15.18:0/3611627640 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590195e7a40 con 0x559019697010 2024-04-16T17:05:27.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.342+0000 7f2bb7e00640 10 client.15654 handle_command_reply: tid=0 2024-04-16T17:05:27.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 client.15654 shutdown 2024-04-16T17:05:27.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3611627640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 msgr2=0x55901967b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901967b3f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80ce100 tx=0x7f2bc809d020 comp rx=0 tx=0).stop 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3611627640 shutdown_connections 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559019697010 0x559019699400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x55901967be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 0x55901967b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/3611627640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901967b3f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3611627640 >> 172.21.15.18:0/3611627640 conn(0x55901966f640 msgr2=0x559019671390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bb7e00640 0 client.15654 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.343+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3611627640 shutdown_connections 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.344+0000 7f2bdfe00740 1 -- 172.21.15.18:0/3611627640 wait complete. 2024-04-16T17:05:27.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.344+0000 7f2bdfe00740 20 client.15654 trim_cache size 0 max 16384 2024-04-16T17:05:27.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.358+0000 7f2bdfe00740 1 Processor -- start 2024-04-16T17:05:27.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.358+0000 7f2bdfe00740 1 -- start start 2024-04-16T17:05:27.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559019674f10 0x55901967db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 0x55901967a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55901967e410 0x55901967bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81afae0 con 0x559019675830 2024-04-16T17:05:27.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81ad3b0 con 0x55901967e410 2024-04-16T17:05:27.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81ab200 con 0x559019674f10 2024-04-16T17:05:27.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559019674f10 0x55901967db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559019674f10 0x55901967db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55984/0 (socket says 172.21.15.18:55984) 2024-04-16T17:05:27.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bd5e00640 1 -- 172.21.15.18:0/97177494 learned_addr learned my addr 172.21.15.18:0/97177494 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.359+0000 7f2bd6800640 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55901967e410 0x55901967bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.360+0000 7f2bd5400640 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 0x55901967a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.360+0000 7f2bd6800640 1 -- 172.21.15.18:0/97177494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559019674f10 msgr2=0x55901967db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.360+0000 7f2bd6800640 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559019674f10 0x55901967db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.360+0000 7f2bd6800640 1 -- 172.21.15.18:0/97177494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 msgr2=0x55901967a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.360+0000 7f2bd6800640 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 0x55901967a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.360+0000 7f2bd6800640 1 -- 172.21.15.18:0/97177494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590192ec3f0 con 0x55901967e410 2024-04-16T17:05:27.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.360+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559019674f10 0x55901967db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:27.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.361+0000 7f2bd6800640 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55901967e410 0x55901967bc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00099e0 tx=0x7f2bd001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3769f4f73e56a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.361+0000 7f2bc2a00640 1 -- 172.21.15.18:0/97177494 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0010800 con 0x55901967e410 2024-04-16T17:05:27.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.361+0000 7f2bc2a00640 1 -- 172.21.15.18:0/97177494 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd001d420 con 0x55901967e410 2024-04-16T17:05:27.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.361+0000 7f2bc2a00640 1 -- 172.21.15.18:0/97177494 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00253b0 con 0x55901967e410 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 -- 172.21.15.18:0/97177494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55901967e410 msgr2=0x55901967bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55901967e410 0x55901967bc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00099e0 tx=0x7f2bd001b000 comp rx=0 tx=0).stop 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 -- 172.21.15.18:0/97177494 shutdown_connections 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55901967e410 0x55901967bc90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019675830 0x55901967a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/97177494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559019674f10 0x55901967db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 -- 172.21.15.18:0/97177494 >> 172.21.15.18:0/97177494 conn(0x55901966f640 msgr2=0x55901966de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 -- 172.21.15.18:0/97177494 shutdown_connections 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.362+0000 7f2bdfe00740 1 -- 172.21.15.18:0/97177494 wait complete. 2024-04-16T17:05:27.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.363+0000 7f2bdfe00740 1 -- start start 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.363+0000 7f2bdfe00740 1 Processor -- start 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901968a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 0x55901968a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bdfe00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x5590196fb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81ab870 con 0x559019674f10 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81acce0 con 0x559019675830 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bdfe00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81abf40 con 0x55901967e410 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bdfe00740 10 client.0 _ll_register_callbacks cb 0x7f2bc800a5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901968a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901968a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56446/0 (socket says 172.21.15.18:56446) 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bd6800640 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x5590196fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.364+0000 7f2bd5e00640 1 -- 172.21.15.18:0/2475544568 learned_addr learned my addr 172.21.15.18:0/2475544568 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.365+0000 7f2bd5400640 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 0x55901968a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.365+0000 7f2bd5e00640 1 -- 172.21.15.18:0/2475544568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 msgr2=0x5590196fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.365+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x5590196fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.365+0000 7f2bd5e00640 1 -- 172.21.15.18:0/2475544568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 msgr2=0x55901968a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.365+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 0x55901968a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.365+0000 7f2bd5e00640 1 -- 172.21.15.18:0/2475544568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590193b67d0 con 0x559019674f10 2024-04-16T17:05:27.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.365+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901968a020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2bc809c7f0 tx=0x7f2bc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24dcf035b706c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.365+0000 7f2bb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.366+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2475544568 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc809c1c0 con 0x559019674f10 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.366+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2475544568 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2bc80a51e0 con 0x559019674f10 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.366+0000 7f2bdfe00740 1 -- 172.21.15.18:0/2475544568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd001d420 con 0x559019674f10 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.366+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2475544568 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bc80a54d0 con 0x559019674f10 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.366+0000 7f2bdfe00740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.366+0000 7f2bdfe00740 1 -- 172.21.15.18:0/2475544568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590196fcb10 con 0x559019674f10 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2475544568 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd001d420 con 0x559019674f10 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2475544568 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2bc809b720 con 0x559019674f10 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bdfe00740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bdfe00740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:27.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bdfe00740 20 client.15660 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:27.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5590196fd050 0x5590196ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:27.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bdfe00740 4 client.15660 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:05:27.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bdfe00740 1 -- 172.21.15.18:0/2475544568 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590195e18b0 con 0x5590196fd050 2024-04-16T17:05:27.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.367+0000 7f2bd5400640 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5590196fd050 0x5590196ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.368+0000 7f2bd5400640 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5590196fd050 0x5590196ff440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1e61ad98c5a956 server_cookie=83787071a25ce7b5 in_seq=0 out_seq=0 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.368+0000 7f2bb7e00640 10 client.15660 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.368+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2475544568 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590195e18b0 con 0x5590196fd050 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.368+0000 7f2bb7e00640 10 client.15660 handle_command_reply: tid=0 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.368+0000 7f2bdfe00740 1 client.15660 shutdown 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.368+0000 7f2bdfe00740 1 -- 172.21.15.18:0/2475544568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 msgr2=0x55901968a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.368+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901968a020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2bc809c7f0 tx=0x7f2bc8000bc0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.369+0000 7f2bdfe00740 1 -- 172.21.15.18:0/2475544568 shutdown_connections 2024-04-16T17:05:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.369+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5590196fd050 0x5590196ff440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.369+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55901967e410 0x5590196fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.369+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559019675830 0x55901968a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.369+0000 7f2bdfe00740 1 --2- 172.21.15.18:0/2475544568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559019674f10 0x55901968a020 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2bc809c7f0 tx=0x7f2bc8000bc0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.369+0000 7f2bdfe00740 1 -- 172.21.15.18:0/2475544568 >> 172.21.15.18:0/2475544568 conn(0x55901966f640 msgr2=0x55901967bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.369+0000 7f2bb7e00640 0 client.15660 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:27.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.371+0000 7f2bdfe00740 1 -- 172.21.15.18:0/2475544568 shutdown_connections 2024-04-16T17:05:27.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.371+0000 7f2bdfe00740 1 -- 172.21.15.18:0/2475544568 wait complete. 2024-04-16T17:05:27.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.371+0000 7f2bdfe00740 20 client.15660 trim_cache size 0 max 16384 2024-04-16T17:05:27.373 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 -- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3f50 msgr2=0x7f2bbc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3f50 0x7f2bbc0a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0005c40 tx=0x7f2bd0009f00 comp rx=0 tx=0).stop 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 -- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd80743d0 msgr2=0x7f2bd81a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd80743d0 0x7f2bd81a0670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c920 tx=0x7f2bc800cde0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 -- 172.21.15.18:0/1781871653 shutdown_connections 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3f50 0x7f2bbc0a6410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd810e6a0 0x7f2bd819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cf0 0x7f2bd81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 --2- 172.21.15.18:0/1781871653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd80743d0 0x7f2bd81a0670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 -- 172.21.15.18:0/1781871653 >> 172.21.15.18:0/1781871653 conn(0x7f2bd80fd950 msgr2=0x7f2bd80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 -- 172.21.15.18:0/1781871653 shutdown_connections 2024-04-16T17:05:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:27.372+0000 7f2bd7200640 1 -- 172.21.15.18:0/1781871653 wait complete. 2024-04-16T17:05:27.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:27 smithi018 ceph-mon[31776]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-16T17:05:27.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:27 smithi022 ceph-mon[36023]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-16T17:05:28.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:27 smithi043 ceph-mon[36901]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-16T17:05:28.392 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:05:28.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:05:28.392 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.679+0000 7fa145e00640 1 -- 172.21.15.18:0/4167795393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa14010c400 msgr2=0x7fa14010e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.679+0000 7fa145e00640 1 --2- 172.21.15.18:0/4167795393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa14010c400 0x7fa14010e870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa138009ca0 tx=0x7fa13802c190 comp rx=0 tx=0).stop 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.679+0000 7fa145e00640 1 -- 172.21.15.18:0/4167795393 shutdown_connections 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.679+0000 7fa145e00640 1 --2- 172.21.15.18:0/4167795393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa14010c400 0x7fa14010e870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.679+0000 7fa145e00640 1 --2- 172.21.15.18:0/4167795393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa140103900 0x7fa14010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.679+0000 7fa145e00640 1 --2- 172.21.15.18:0/4167795393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa140102fe0 0x7fa1401033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.679+0000 7fa145e00640 1 -- 172.21.15.18:0/4167795393 >> 172.21.15.18:0/4167795393 conn(0x7fa140077ab0 msgr2=0x7fa1400fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.680+0000 7fa145e00640 1 -- 172.21.15.18:0/4167795393 shutdown_connections 2024-04-16T17:05:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.680+0000 7fa145e00640 1 -- 172.21.15.18:0/4167795393 wait complete. 2024-04-16T17:05:28.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.681+0000 7fa145e00640 1 Processor -- start 2024-04-16T17:05:28.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.682+0000 7fa145e00640 1 -- start start 2024-04-16T17:05:28.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa145e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa140102fe0 0x7fa14019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa145e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140103900 0x7fa14019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa145e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa14010c400 0x7fa14010b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa145e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa140112190 con 0x7fa14010c400 2024-04-16T17:05:28.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa145e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa140112010 con 0x7fa140102fe0 2024-04-16T17:05:28.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa145e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa140112310 con 0x7fa140103900 2024-04-16T17:05:28.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140103900 0x7fa14019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140103900 0x7fa14019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55994/0 (socket says 172.21.15.18:55994) 2024-04-16T17:05:28.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa13fe00640 1 -- 172.21.15.18:0/412887292 learned_addr learned my addr 172.21.15.18:0/412887292 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:28.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.683+0000 7fa144a00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa140102fe0 0x7fa14019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.684+0000 7fa145400640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa14010c400 0x7fa14010b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.684+0000 7fa144a00640 1 -- 172.21.15.18:0/412887292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140103900 msgr2=0x7fa14019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.684+0000 7fa144a00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140103900 0x7fa14019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.684+0000 7fa144a00640 1 -- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa14010c400 msgr2=0x7fa14010b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.684+0000 7fa144a00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa14010c400 0x7fa14010b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.684+0000 7fa144a00640 1 -- 172.21.15.18:0/412887292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14010ba20 con 0x7fa140102fe0 2024-04-16T17:05:28.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.684+0000 7fa13fe00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140103900 0x7fa14019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:28.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.685+0000 7fa144a00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa140102fe0 0x7fa14019f3b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa13400e9f0 tx=0x7fa13400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:28.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.686+0000 7fa13d600640 1 -- 172.21.15.18:0/412887292 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13400ccc0 con 0x7fa140102fe0 2024-04-16T17:05:28.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.686+0000 7fa13d600640 1 -- 172.21.15.18:0/412887292 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa134010910 con 0x7fa140102fe0 2024-04-16T17:05:28.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.686+0000 7fa145e00640 1 -- 172.21.15.18:0/412887292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14010bd10 con 0x7fa140102fe0 2024-04-16T17:05:28.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.686+0000 7fa13d600640 1 -- 172.21.15.18:0/412887292 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134018960 con 0x7fa140102fe0 2024-04-16T17:05:28.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.686+0000 7fa145e00640 1 -- 172.21.15.18:0/412887292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14019afb0 con 0x7fa140102fe0 2024-04-16T17:05:28.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.691+0000 7fa14a1a2740 1 Processor -- start 2024-04-16T17:05:28.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.692+0000 7fa13d600640 1 -- 172.21.15.18:0/412887292 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa134010ac0 con 0x7fa140102fe0 2024-04-16T17:05:28.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.692+0000 7fa14a1a2740 1 -- start start 2024-04-16T17:05:28.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.692+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d7189270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.692+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d71897b0 0x5577d7191e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.692+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d7194780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.692+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5577d70edcb0 con 0x5577d7192390 2024-04-16T17:05:28.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.692+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577d7094370 con 0x5577d71897b0 2024-04-16T17:05:28.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.692+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5577d70e9c70 con 0x5577d7188e90 2024-04-16T17:05:28.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.693+0000 7fa145400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d7189270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.693+0000 7fa145400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d7189270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56008/0 (socket says 172.21.15.18:56008) 2024-04-16T17:05:28.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.693+0000 7fa145400640 1 -- 172.21.15.18:0/3319516299 learned_addr learned my addr 172.21.15.18:0/3319516299 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.693+0000 7fa13fe00640 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d7194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.693+0000 7fa145400640 1 -- 172.21.15.18:0/3319516299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d71897b0 msgr2=0x5577d7191e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.693+0000 7fa145400640 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d71897b0 0x5577d7191e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.693+0000 7fa145400640 1 -- 172.21.15.18:0/3319516299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 msgr2=0x5577d7194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.694+0000 7fa145400640 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d7194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.694+0000 7fa145400640 1 -- 172.21.15.18:0/3319516299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d70b3910 con 0x5577d7188e90 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.694+0000 7fa13fe00640 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d7194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.694+0000 7fa145400640 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d7189270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa13802c7a0 tx=0x7fa138005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba4c87f303e5316 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.695+0000 7fa131c00640 1 -- 172.21.15.18:0/3319516299 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138043070 con 0x5577d7188e90 2024-04-16T17:05:28.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.695+0000 7fa131c00640 1 -- 172.21.15.18:0/3319516299 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa138003e50 con 0x5577d7188e90 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.695+0000 7fa131c00640 1 -- 172.21.15.18:0/3319516299 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138039040 con 0x5577d7188e90 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.695+0000 7fa13d600640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1280a3fd0 0x7fa1280a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.695+0000 7fa13d600640 1 -- 172.21.15.18:0/412887292 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa1340d1160 con 0x7fa140102fe0 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.695+0000 7fa144a00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1280a3fd0 0x7fa1280a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3319516299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 msgr2=0x5577d7189270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d7189270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa13802c7a0 tx=0x7fa138005f80 comp rx=0 tx=0).stop 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3319516299 shutdown_connections 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d7194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d71897b0 0x5577d7191e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3319516299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d7189270 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3319516299 >> 172.21.15.18:0/3319516299 conn(0x5577d71835c0 msgr2=0x5577d71839a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3319516299 shutdown_connections 2024-04-16T17:05:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.696+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3319516299 wait complete. 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.697+0000 7fa14a1a2740 1 -- start start 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.697+0000 7fa14a1a2740 1 Processor -- start 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.697+0000 7fa144a00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1280a3fd0 0x7fa1280a6490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1340ce320 tx=0x7fa134098e60 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.698+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7188e90 0x5577d718f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.698+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.698+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d718fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.698+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5577d7195120 con 0x5577d7192390 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.698+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577d70a7730 con 0x5577d7188e90 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.698+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5577d71952a0 con 0x5577d71897b0 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.698+0000 7fa14a1a2740 10 client.0 _ll_register_callbacks cb 0x5577d70761e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.698+0000 7fa145400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7188e90 0x5577d718f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56020/0 (socket says 172.21.15.18:56020) 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa13fe00640 1 -- 172.21.15.18:0/3468482384 learned_addr learned my addr 172.21.15.18:0/3468482384 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa145400640 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d718fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa13fe00640 1 -- 172.21.15.18:0/3468482384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7188e90 msgr2=0x5577d718f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa13fe00640 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7188e90 0x5577d718f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa13fe00640 1 -- 172.21.15.18:0/3468482384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 msgr2=0x5577d718fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa13fe00640 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d718fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa13fe00640 1 -- 172.21.15.18:0/3468482384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d6ec87e0 con 0x5577d71897b0 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.699+0000 7fa145400640 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d718fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.700+0000 7fa13fe00640 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718f960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0116f0 tx=0x7fa12c011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=780bd6c4c254a754 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.700+0000 7fa126000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:28.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.700+0000 7fa126000640 1 -- 172.21.15.18:0/3468482384 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c018070 con 0x5577d71897b0 2024-04-16T17:05:28.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.700+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3468482384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa138003e50 con 0x5577d71897b0 2024-04-16T17:05:28.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.700+0000 7fa126000640 1 -- 172.21.15.18:0/3468482384 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa12c00fd50 con 0x5577d71897b0 2024-04-16T17:05:28.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.700+0000 7fa126000640 1 -- 172.21.15.18:0/3468482384 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa12c021660 con 0x5577d71897b0 2024-04-16T17:05:28.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa14a1a2740 10 client.25798 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:28.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3468482384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577d6f28f70 con 0x5577d71897b0 2024-04-16T17:05:28.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa126000640 1 -- 172.21.15.18:0/3468482384 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa138003e50 con 0x5577d71897b0 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa126000640 1 -- 172.21.15.18:0/3468482384 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa12c012450 con 0x5577d71897b0 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa14a1a2740 10 client.25798 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa14a1a2740 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa14a1a2740 20 client.25798 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5577d71a2770 0x5577d71a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa14a1a2740 4 client.25798 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.701+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3468482384 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577d70fb6d0 con 0x5577d71a2770 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.702+0000 7fa145400640 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5577d71a2770 0x5577d71a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.703+0000 7fa145400640 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5577d71a2770 0x5577d71a4b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a276dd43c9ba50 server_cookie=c39feb38e19b40fb in_seq=0 out_seq=0 2024-04-16T17:05:28.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.703+0000 7fa126000640 10 client.25798 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.703+0000 7fa126000640 1 -- 172.21.15.18:0/3468482384 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577d70fb6d0 con 0x5577d71a2770 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.703+0000 7fa126000640 10 client.25798 handle_command_reply: tid=0 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 client.25798 shutdown 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3468482384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 msgr2=0x5577d718f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718f960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0116f0 tx=0x7fa12c011bb0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3468482384 shutdown_connections 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5577d71a2770 0x5577d71a4b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7192390 0x5577d718fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718f960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa126000640 0 client.25798 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:28.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3468482384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7188e90 0x5577d718f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.704+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3468482384 >> 172.21.15.18:0/3468482384 conn(0x5577d71835c0 msgr2=0x5577d7185310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:28.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.705+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3468482384 shutdown_connections 2024-04-16T17:05:28.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.705+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3468482384 wait complete. 2024-04-16T17:05:28.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.705+0000 7fa14a1a2740 20 client.25798 trim_cache size 0 max 16384 2024-04-16T17:05:28.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.742+0000 7fa14a1a2740 1 Processor -- start 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa14a1a2740 1 -- start start 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7188e90 0x5577d7191b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d719dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1401c0b50 con 0x5577d7188e90 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1401be420 con 0x5577d7192390 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1401bc270 con 0x5577d71897b0 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa145400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7188e90 0x5577d7191b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56028/0 (socket says 172.21.15.18:56028) 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa13fe00640 1 -- 172.21.15.18:0/3747243289 learned_addr learned my addr 172.21.15.18:0/3747243289 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:28.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.743+0000 7fa145400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7188e90 0x5577d7191b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56498/0 (socket says 172.21.15.18:56498) 2024-04-16T17:05:28.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.744+0000 7fa13fe00640 1 -- 172.21.15.18:0/3747243289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 msgr2=0x5577d719dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:28.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.744+0000 7fa13fe00640 1 --2- 172.21.15.18:0/3747243289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d719dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.744+0000 7fa13fe00640 1 -- 172.21.15.18:0/3747243289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7188e90 msgr2=0x5577d7191b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:28.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.744+0000 7fa13fe00640 1 --2- 172.21.15.18:0/3747243289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7188e90 0x5577d7191b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.744+0000 7fa13fe00640 1 -- 172.21.15.18:0/3747243289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d71895e0 con 0x5577d71897b0 2024-04-16T17:05:28.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.744+0000 7fa13fe00640 1 --2- 172.21.15.18:0/3747243289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718e380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006010 tx=0x7fa12c0113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131bb39532e77d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:28.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.745+0000 7fa131200640 1 -- 172.21.15.18:0/3747243289 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c018030 con 0x5577d71897b0 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.745+0000 7fa131200640 1 -- 172.21.15.18:0/3747243289 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12c02f070 con 0x5577d71897b0 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.745+0000 7fa131200640 1 -- 172.21.15.18:0/3747243289 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0050e0 con 0x5577d71897b0 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3747243289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 msgr2=0x5577d718e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3747243289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718e380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006010 tx=0x7fa12c0113e0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3747243289 shutdown_connections 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3747243289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d719dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3747243289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d71897b0 0x5577d718e380 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/3747243289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d7188e90 0x5577d7191b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3747243289 >> 172.21.15.18:0/3747243289 conn(0x5577d71835c0 msgr2=0x5577d718fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3747243289 shutdown_connections 2024-04-16T17:05:28.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 -- 172.21.15.18:0/3747243289 wait complete. 2024-04-16T17:05:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.746+0000 7fa14a1a2740 1 -- start start 2024-04-16T17:05:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.747+0000 7fa14a1a2740 1 Processor -- start 2024-04-16T17:05:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.747+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d718c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.747+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d71897b0 0x5577d718c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.747+0000 7fa14a1a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d718caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.747+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1401bc8e0 con 0x5577d71897b0 2024-04-16T17:05:28.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.747+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1401bdd50 con 0x5577d7192390 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.747+0000 7fa14a1a2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1401bcfb0 con 0x5577d7188e90 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.747+0000 7fa14a1a2740 10 client.0 _ll_register_callbacks cb 0x7fa12c00bf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d718caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d718caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:43826/0 (socket says 172.21.15.18:43826) 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 -- 172.21.15.18:0/2993465897 learned_addr learned my addr 172.21.15.18:0/2993465897 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d718c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa13fe00640 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d71897b0 0x5577d718c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 -- 172.21.15.18:0/2993465897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 msgr2=0x5577d718c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d718c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 -- 172.21.15.18:0/2993465897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d71897b0 msgr2=0x5577d718c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d71897b0 0x5577d718c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 -- 172.21.15.18:0/2993465897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d6dc0170 con 0x5577d7192390 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa145400640 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d718caa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa138005e30 tx=0x7fa138005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e6c22d50116679c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:28.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.748+0000 7fa126000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.749+0000 7fa126000640 1 -- 172.21.15.18:0/2993465897 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138043040 con 0x5577d7192390 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.749+0000 7fa126000640 1 -- 172.21.15.18:0/2993465897 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa138052bd0 con 0x5577d7192390 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.749+0000 7fa14a1a2740 1 -- 172.21.15.18:0/2993465897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa12c02f070 con 0x5577d7192390 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.749+0000 7fa126000640 1 -- 172.21.15.18:0/2993465897 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa138039040 con 0x5577d7192390 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.749+0000 7fa14a1a2740 10 client.25811 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.749+0000 7fa14a1a2740 1 -- 172.21.15.18:0/2993465897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577d7208e80 con 0x5577d7192390 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.749+0000 7fa126000640 1 -- 172.21.15.18:0/2993465897 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12c02f070 con 0x5577d7192390 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.749+0000 7fa126000640 1 -- 172.21.15.18:0/2993465897 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa138052270 con 0x5577d7192390 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.750+0000 7fa14a1a2740 10 client.25811 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.750+0000 7fa14a1a2740 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.750+0000 7fa14a1a2740 20 client.25811 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.750+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5577d71a2770 0x5577d720b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:28.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.750+0000 7fa14a1a2740 4 client.25811 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:05:28.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.750+0000 7fa14a1a2740 1 -- 172.21.15.18:0/2993465897 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5577d70f5540 con 0x5577d71a2770 2024-04-16T17:05:28.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.750+0000 7fa13fe00640 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5577d71a2770 0x5577d720b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:28.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.751+0000 7fa13fe00640 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5577d71a2770 0x5577d720b270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613cb3cf2b9092d2 server_cookie=ed5cdc5aa19dc745 in_seq=0 out_seq=0 2024-04-16T17:05:28.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.751+0000 7fa126000640 10 client.25811 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:28.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.752+0000 7fa126000640 1 -- 172.21.15.18:0/2993465897 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5577d70f5540 con 0x5577d71a2770 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.752+0000 7fa126000640 10 client.25811 handle_command_reply: tid=0 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 client.25811 shutdown 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 -- 172.21.15.18:0/2993465897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 msgr2=0x5577d718caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d718caa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa138005e30 tx=0x7fa138005f80 comp rx=0 tx=0).stop 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 -- 172.21.15.18:0/2993465897 shutdown_connections 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5577d71a2770 0x5577d720b270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577d7192390 0x5577d718caa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577d71897b0 0x5577d718c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa126000640 0 client.25811 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 --2- 172.21.15.18:0/2993465897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577d7188e90 0x5577d718c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 -- 172.21.15.18:0/2993465897 >> 172.21.15.18:0/2993465897 conn(0x5577d71835c0 msgr2=0x5577d718f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 -- 172.21.15.18:0/2993465897 shutdown_connections 2024-04-16T17:05:28.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 1 -- 172.21.15.18:0/2993465897 wait complete. 2024-04-16T17:05:28.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.753+0000 7fa14a1a2740 20 client.25811 trim_cache size 0 max 16384 2024-04-16T17:05:28.757 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:05:28.757 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:05:28.757 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "53a7fe24-8c21-41e6-97a2-107f54769532", 2024-04-16T17:05:28.757 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:05:28.757 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:05:28.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.754+0000 7fa145e00640 1 -- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1280a3fd0 msgr2=0x7fa1280a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.754+0000 7fa145e00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1280a3fd0 0x7fa1280a6490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1340ce320 tx=0x7fa134098e60 comp rx=0 tx=0).stop 2024-04-16T17:05:28.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.754+0000 7fa145e00640 1 -- 172.21.15.18:0/412887292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa140102fe0 msgr2=0x7fa14019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:28.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.754+0000 7fa145e00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa140102fe0 0x7fa14019f3b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa13400e9f0 tx=0x7fa13400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.754+0000 7fa145e00640 1 -- 172.21.15.18:0/412887292 shutdown_connections 2024-04-16T17:05:28.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.754+0000 7fa145e00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1280a3fd0 0x7fa1280a6490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.754+0000 7fa145e00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa14010c400 0x7fa14010b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.755+0000 7fa145e00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140103900 0x7fa14019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.755+0000 7fa145e00640 1 --2- 172.21.15.18:0/412887292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa140102fe0 0x7fa14019f3b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:28.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.755+0000 7fa145e00640 1 -- 172.21.15.18:0/412887292 >> 172.21.15.18:0/412887292 conn(0x7fa140077ab0 msgr2=0x7fa140100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:28.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.756+0000 7fa145e00640 1 -- 172.21.15.18:0/412887292 shutdown_connections 2024-04-16T17:05:28.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:28.756+0000 7fa145e00640 1 -- 172.21.15.18:0/412887292 wait complete. 2024-04-16T17:05:28.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:05:28.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:05:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.012+0000 7ff625000640 1 -- 172.21.15.18:0/407569948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff620112e10 msgr2=0x7ff620115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.012+0000 7ff625000640 1 --2- 172.21.15.18:0/407569948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff620112e10 0x7ff620115200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff618009ca0 tx=0x7ff61802c190 comp rx=0 tx=0).stop 2024-04-16T17:05:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.012+0000 7ff625000640 1 -- 172.21.15.18:0/407569948 shutdown_connections 2024-04-16T17:05:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.012+0000 7ff625000640 1 --2- 172.21.15.18:0/407569948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff620112e10 0x7ff620115200 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.012+0000 7ff625000640 1 --2- 172.21.15.18:0/407569948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff620074cc0 0x7ff6201128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.012+0000 7ff625000640 1 --2- 172.21.15.18:0/407569948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6200743a0 0x7ff620074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.012+0000 7ff625000640 1 -- 172.21.15.18:0/407569948 >> 172.21.15.18:0/407569948 conn(0x7ff6200fd950 msgr2=0x7ff6200ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.013+0000 7ff625000640 1 -- 172.21.15.18:0/407569948 shutdown_connections 2024-04-16T17:05:29.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.013+0000 7ff625000640 1 -- 172.21.15.18:0/407569948 wait complete. 2024-04-16T17:05:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.014+0000 7ff625000640 1 Processor -- start 2024-04-16T17:05:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.014+0000 7ff625000640 1 -- start start 2024-04-16T17:05:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff625000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6200743a0 0x7ff62019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff625000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff620074cc0 0x7ff62019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff625000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff620112e10 0x7ff6201a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff625000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201187d0 con 0x7ff6200743a0 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff625000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff620118650 con 0x7ff620074cc0 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff625000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff620118950 con 0x7ff620112e10 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff61f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6200743a0 0x7ff62019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff61f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6200743a0 0x7ff62019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56526/0 (socket says 172.21.15.18:56526) 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff61f400640 1 -- 172.21.15.18:0/2356919846 learned_addr learned my addr 172.21.15.18:0/2356919846 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff61ea00640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff620074cc0 0x7ff62019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.015+0000 7ff61fe00640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff620112e10 0x7ff6201a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff61ea00640 1 -- 172.21.15.18:0/2356919846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff620112e10 msgr2=0x7ff6201a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff61ea00640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff620112e10 0x7ff6201a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff61ea00640 1 -- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6200743a0 msgr2=0x7ff62019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff61ea00640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6200743a0 0x7ff62019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff61ea00640 1 -- 172.21.15.18:0/2356919846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6201a3d40 con 0x7ff620074cc0 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff61f400640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6200743a0 0x7ff62019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff61ea00640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff620074cc0 0x7ff62019f290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00e9f0 tx=0x7ff60c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff617e00640 1 -- 172.21.15.18:0/2356919846 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00ccc0 con 0x7ff620074cc0 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff625000640 1 -- 172.21.15.18:0/2356919846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6201a4030 con 0x7ff620074cc0 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff617e00640 1 -- 172.21.15.18:0/2356919846 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff60c005020 con 0x7ff620074cc0 2024-04-16T17:05:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.016+0000 7ff617e00640 1 -- 172.21.15.18:0/2356919846 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0049e0 con 0x7ff620074cc0 2024-04-16T17:05:29.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.017+0000 7ff625000640 1 -- 172.21.15.18:0/2356919846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6201ab8c0 con 0x7ff620074cc0 2024-04-16T17:05:29.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.020+0000 7ff629395740 1 Processor -- start 2024-04-16T17:05:29.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff629395740 1 -- start start 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff629395740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642390 0x55e02d642770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff629395740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 0x55e02d64b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff629395740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d64b890 0x55e02d64dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff629395740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e02d5a8120 con 0x55e02d642390 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff629395740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e02d54e340 con 0x55e02d642cb0 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff629395740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e02d5a40e0 con 0x55e02d64b890 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff61fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642390 0x55e02d642770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.021+0000 7ff61fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642390 0x55e02d642770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56532/0 (socket says 172.21.15.18:56532) 2024-04-16T17:05:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.022+0000 7ff61fe00640 1 -- 172.21.15.18:0/4069677548 learned_addr learned my addr 172.21.15.18:0/4069677548 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.022+0000 7ff61f400640 1 --2- 172.21.15.18:0/4069677548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 0x55e02d64b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.022+0000 7ff617e00640 1 -- 172.21.15.18:0/2356919846 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff60c005560 con 0x7ff620074cc0 2024-04-16T17:05:29.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.022+0000 7ff61fe00640 1 -- 172.21.15.18:0/4069677548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d64b890 msgr2=0x55e02d64dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.022+0000 7ff61fe00640 1 --2- 172.21.15.18:0/4069677548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d64b890 0x55e02d64dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.022+0000 7ff61fe00640 1 -- 172.21.15.18:0/4069677548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 msgr2=0x55e02d64b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.022+0000 7ff61fe00640 1 --2- 172.21.15.18:0/4069677548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 0x55e02d64b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.022+0000 7ff61fe00640 1 -- 172.21.15.18:0/4069677548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02d3847d0 con 0x55e02d642390 2024-04-16T17:05:29.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.023+0000 7ff61fe00640 1 --2- 172.21.15.18:0/4069677548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642390 0x55e02d642770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff618004af0 tx=0x7ff618005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48faf4db841638b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.023+0000 7ff614c00640 1 -- 172.21.15.18:0/4069677548 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618043070 con 0x55e02d642390 2024-04-16T17:05:29.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.023+0000 7ff614c00640 1 -- 172.21.15.18:0/4069677548 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff618003e50 con 0x55e02d642390 2024-04-16T17:05:29.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.023+0000 7ff614c00640 1 -- 172.21.15.18:0/4069677548 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618039040 con 0x55e02d642390 2024-04-16T17:05:29.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff629395740 1 -- 172.21.15.18:0/4069677548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642390 msgr2=0x55e02d642770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff629395740 1 --2- 172.21.15.18:0/4069677548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642390 0x55e02d642770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff618004af0 tx=0x7ff618005f80 comp rx=0 tx=0).stop 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff617e00640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6040a3ed0 0x7ff6040a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff629395740 1 -- 172.21.15.18:0/4069677548 shutdown_connections 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff629395740 1 --2- 172.21.15.18:0/4069677548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d64b890 0x55e02d64dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff629395740 1 --2- 172.21.15.18:0/4069677548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 0x55e02d64b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff629395740 1 --2- 172.21.15.18:0/4069677548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642390 0x55e02d642770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff629395740 1 -- 172.21.15.18:0/4069677548 >> 172.21.15.18:0/4069677548 conn(0x55e02d63cac0 msgr2=0x55e02d63cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff617e00640 1 -- 172.21.15.18:0/2356919846 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff60c0d0d70 con 0x7ff620074cc0 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.024+0000 7ff61fe00640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6040a3ed0 0x7ff6040a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.025+0000 7ff629395740 1 -- 172.21.15.18:0/4069677548 shutdown_connections 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.025+0000 7ff629395740 1 -- 172.21.15.18:0/4069677548 wait complete. 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.025+0000 7ff629395740 1 -- start start 2024-04-16T17:05:29.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.025+0000 7ff629395740 1 Processor -- start 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.026+0000 7ff61fe00640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6040a3ed0 0x7ff6040a6390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6180047b0 tx=0x7ff61802c190 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.026+0000 7ff629395740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d6488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.026+0000 7ff629395740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 0x55e02d648de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.026+0000 7ff629395740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d64b890 0x55e02d649320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.026+0000 7ff629395740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e02d64e620 con 0x55e02d64b890 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.026+0000 7ff629395740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e02d561710 con 0x55e02d642cb0 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.026+0000 7ff629395740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e02d64e7a0 con 0x55e02d642390 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.026+0000 7ff629395740 10 client.0 _ll_register_callbacks cb 0x55e02d55caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d6488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 0x55e02d648de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d6488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56066/0 (socket says 172.21.15.18:56066) 2024-04-16T17:05:29.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61f400640 1 -- 172.21.15.18:0/4002934249 learned_addr learned my addr 172.21.15.18:0/4002934249 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61fe00640 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d64b890 0x55e02d649320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61f400640 1 -- 172.21.15.18:0/4002934249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 msgr2=0x55e02d648de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61f400640 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 0x55e02d648de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61f400640 1 -- 172.21.15.18:0/4002934249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d64b890 msgr2=0x55e02d649320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61f400640 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d64b890 0x55e02d649320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.027+0000 7ff61f400640 1 -- 172.21.15.18:0/4002934249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02d2ba3f0 con 0x55e02d642390 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.028+0000 7ff61f400640 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d6488a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff610012d50 tx=0x7ff61000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec1705a0868c8633 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.028+0000 7ff609600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.028+0000 7ff609600640 1 -- 172.21.15.18:0/4002934249 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61000fe20 con 0x55e02d642390 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.028+0000 7ff609600640 1 -- 172.21.15.18:0/4002934249 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff610010430 con 0x55e02d642390 2024-04-16T17:05:29.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.029+0000 7ff629395740 1 -- 172.21.15.18:0/4002934249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff618003e50 con 0x55e02d642390 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.029+0000 7ff609600640 1 -- 172.21.15.18:0/4002934249 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff610010630 con 0x55e02d642390 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.029+0000 7ff629395740 10 client.25813 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.029+0000 7ff629395740 1 -- 172.21.15.18:0/4002934249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e02d27a0f0 con 0x55e02d642390 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff609600640 1 -- 172.21.15.18:0/4002934249 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff618003e50 con 0x55e02d642390 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff609600640 1 -- 172.21.15.18:0/4002934249 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff610026860 con 0x55e02d642390 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff629395740 10 client.25813 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff629395740 10 client.25813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff629395740 20 client.25813 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff629395740 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e02d65bb90 0x55e02d65df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff629395740 4 client.25813 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff629395740 1 -- 172.21.15.18:0/4002934249 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e02d5b5b40 con 0x55e02d65bb90 2024-04-16T17:05:29.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.030+0000 7ff61ea00640 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e02d65bb90 0x55e02d65df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.031+0000 7ff61ea00640 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e02d65bb90 0x55e02d65df80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737e0f2f643770f1 server_cookie=53c18df0c196b72d in_seq=0 out_seq=0 2024-04-16T17:05:29.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.031+0000 7ff609600640 10 client.25813 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:29.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.032+0000 7ff609600640 1 -- 172.21.15.18:0/4002934249 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e02d5b5b40 con 0x55e02d65bb90 2024-04-16T17:05:29.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.032+0000 7ff609600640 10 client.25813 handle_command_reply: tid=0 2024-04-16T17:05:29.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.032+0000 7ff629395740 1 client.25813 shutdown 2024-04-16T17:05:29.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.032+0000 7ff629395740 1 -- 172.21.15.18:0/4002934249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 msgr2=0x55e02d6488a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.032+0000 7ff629395740 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d6488a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff610012d50 tx=0x7ff61000f790 comp rx=0 tx=0).stop 2024-04-16T17:05:29.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 1 -- 172.21.15.18:0/4002934249 shutdown_connections 2024-04-16T17:05:29.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e02d65bb90 0x55e02d65df80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d64b890 0x55e02d649320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d642cb0 0x55e02d648de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 1 --2- 172.21.15.18:0/4002934249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d6488a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff609600640 0 client.25813 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 1 -- 172.21.15.18:0/4002934249 >> 172.21.15.18:0/4002934249 conn(0x55e02d63cac0 msgr2=0x55e02d63e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 1 -- 172.21.15.18:0/4002934249 shutdown_connections 2024-04-16T17:05:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 1 -- 172.21.15.18:0/4002934249 wait complete. 2024-04-16T17:05:29.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.033+0000 7ff629395740 20 client.25813 trim_cache size 0 max 16384 2024-04-16T17:05:29.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff629395740 1 Processor -- start 2024-04-16T17:05:29.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff629395740 1 -- start start 2024-04-16T17:05:29.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff629395740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d647820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff629395740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d6574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff629395740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d663830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff629395740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201b8320 con 0x55e02d642cb0 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff629395740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201b5bf0 con 0x55e02d64b890 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff629395740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201b3a40 con 0x55e02d642390 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff61f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d647820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.055+0000 7ff61f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d647820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56078/0 (socket says 172.21.15.18:56078) 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61f400640 1 -- 172.21.15.18:0/3547516529 learned_addr learned my addr 172.21.15.18:0/3547516529 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61ea00640 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d6574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61fe00640 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d663830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61f400640 1 -- 172.21.15.18:0/3547516529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 msgr2=0x55e02d663830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61f400640 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d663830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61f400640 1 -- 172.21.15.18:0/3547516529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 msgr2=0x55e02d6574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61f400640 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d6574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61f400640 1 -- 172.21.15.18:0/3547516529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02d65a7e0 con 0x55e02d642390 2024-04-16T17:05:29.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61ea00640 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d6574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:29.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61fe00640 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d663830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:29.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.056+0000 7ff61f400640 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d647820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff610000c00 tx=0x7ff61000fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6173885e4a8b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.057+0000 7ff60be00640 1 -- 172.21.15.18:0/3547516529 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61002b030 con 0x55e02d642390 2024-04-16T17:05:29.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.057+0000 7ff60be00640 1 -- 172.21.15.18:0/3547516529 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff610005bf0 con 0x55e02d642390 2024-04-16T17:05:29.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.057+0000 7ff60be00640 1 -- 172.21.15.18:0/3547516529 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610026700 con 0x55e02d642390 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.057+0000 7ff629395740 1 -- 172.21.15.18:0/3547516529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 msgr2=0x55e02d647820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.057+0000 7ff629395740 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d647820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff610000c00 tx=0x7ff61000fd10 comp rx=0 tx=0).stop 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 -- 172.21.15.18:0/3547516529 shutdown_connections 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d663830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d6574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 --2- 172.21.15.18:0/3547516529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d647820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 -- 172.21.15.18:0/3547516529 >> 172.21.15.18:0/3547516529 conn(0x55e02d63cac0 msgr2=0x55e02d63e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 -- 172.21.15.18:0/3547516529 shutdown_connections 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 -- 172.21.15.18:0/3547516529 wait complete. 2024-04-16T17:05:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 -- start start 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.058+0000 7ff629395740 1 Processor -- start 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff629395740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d645ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff629395740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d646220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff629395740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d6c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff629395740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201b40b0 con 0x55e02d642cb0 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff629395740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201b5520 con 0x55e02d64b890 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff629395740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201b4780 con 0x55e02d642390 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff629395740 10 client.0 _ll_register_callbacks cb 0x7ff610009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff61f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d645ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff61f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d645ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56082/0 (socket says 172.21.15.18:56082) 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff61f400640 1 -- 172.21.15.18:0/3335409559 learned_addr learned my addr 172.21.15.18:0/3335409559 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.059+0000 7ff61ea00640 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d646220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff61fe00640 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d6c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff61f400640 1 -- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 msgr2=0x55e02d6c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff61f400640 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d6c10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff61f400640 1 -- 172.21.15.18:0/3335409559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 msgr2=0x55e02d646220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff61f400640 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d646220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff61f400640 1 -- 172.21.15.18:0/3335409559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02d6c15e0 con 0x55e02d642390 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff61fe00640 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d6c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff61f400640 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d645ce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff610012d50 tx=0x7ff6100094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20597fb526392dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.060+0000 7ff609600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.061+0000 7ff609600640 1 -- 172.21.15.18:0/3335409559 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610034060 con 0x55e02d642390 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.061+0000 7ff629395740 1 -- 172.21.15.18:0/3335409559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e02d5af9b0 con 0x55e02d642390 2024-04-16T17:05:29.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.061+0000 7ff609600640 1 -- 172.21.15.18:0/3335409559 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff6100050e0 con 0x55e02d642390 2024-04-16T17:05:29.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.061+0000 7ff609600640 1 -- 172.21.15.18:0/3335409559 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6100052e0 con 0x55e02d642390 2024-04-16T17:05:29.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.061+0000 7ff629395740 10 client.25819 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:29.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.061+0000 7ff629395740 1 -- 172.21.15.18:0/3335409559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e02d6c28f0 con 0x55e02d642390 2024-04-16T17:05:29.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff609600640 1 -- 172.21.15.18:0/3335409559 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e02d5af9b0 con 0x55e02d642390 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff609600640 1 -- 172.21.15.18:0/3335409559 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff610003950 con 0x55e02d642390 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff629395740 10 client.25819 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff629395740 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff629395740 20 client.25819 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff629395740 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e02d6c2e80 0x55e02d6c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff629395740 4 client.25819 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff629395740 1 -- 172.21.15.18:0/3335409559 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e02d52b420 con 0x55e02d6c2e80 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.062+0000 7ff61ea00640 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e02d6c2e80 0x55e02d6c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.063+0000 7ff609600640 10 client.25819 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.063+0000 7ff61ea00640 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e02d6c2e80 0x55e02d6c52a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc12bef09523a4d6 server_cookie=626ef141b987eea5 in_seq=0 out_seq=0 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.063+0000 7ff609600640 1 -- 172.21.15.18:0/3335409559 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e02d52b420 con 0x55e02d6c2e80 2024-04-16T17:05:29.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.063+0000 7ff609600640 10 client.25819 handle_command_reply: tid=0 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.063+0000 7ff629395740 1 client.25819 shutdown 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 -- 172.21.15.18:0/3335409559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 msgr2=0x55e02d645ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d645ce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff610012d50 tx=0x7ff6100094d0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 -- 172.21.15.18:0/3335409559 shutdown_connections 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e02d6c2e80 0x55e02d6c52a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e02d64b890 0x55e02d6c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e02d642cb0 0x55e02d646220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 --2- 172.21.15.18:0/3335409559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e02d642390 0x55e02d645ce0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff609600640 0 client.25819 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 -- 172.21.15.18:0/3335409559 >> 172.21.15.18:0/3335409559 conn(0x55e02d63cac0 msgr2=0x55e02d63b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 -- 172.21.15.18:0/3335409559 shutdown_connections 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 1 -- 172.21.15.18:0/3335409559 wait complete. 2024-04-16T17:05:29.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.064+0000 7ff629395740 20 client.25819 trim_cache size 0 max 16384 2024-04-16T17:05:29.070 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:05:29.070 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:05:29.070 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:05:29.070 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:05:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 -- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6040a3ed0 msgr2=0x7ff6040a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6040a3ed0 0x7ff6040a6390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6180047b0 tx=0x7ff61802c190 comp rx=0 tx=0).stop 2024-04-16T17:05:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 -- 172.21.15.18:0/2356919846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff620074cc0 msgr2=0x7ff62019f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff620074cc0 0x7ff62019f290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00e9f0 tx=0x7ff60c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 -- 172.21.15.18:0/2356919846 shutdown_connections 2024-04-16T17:05:29.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6040a3ed0 0x7ff6040a6390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff620112e10 0x7ff6201a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff620074cc0 0x7ff62019f290 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 --2- 172.21.15.18:0/2356919846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6200743a0 0x7ff62019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 -- 172.21.15.18:0/2356919846 >> 172.21.15.18:0/2356919846 conn(0x7ff6200fd950 msgr2=0x7ff6200fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 -- 172.21.15.18:0/2356919846 shutdown_connections 2024-04-16T17:05:29.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.065+0000 7ff625000640 1 -- 172.21.15.18:0/2356919846 wait complete. 2024-04-16T17:05:29.083 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:05:29.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:05:29.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 -- 172.21.15.18:0/3488581038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c10e7a0 msgr2=0x7f3f8c110b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 --2- 172.21.15.18:0/3488581038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c10e7a0 0x7f3f8c110b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009ca0 tx=0x7f3f8402c180 comp rx=0 tx=0).stop 2024-04-16T17:05:29.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 -- 172.21.15.18:0/3488581038 shutdown_connections 2024-04-16T17:05:29.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 --2- 172.21.15.18:0/3488581038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c10e7a0 0x7f3f8c110b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 --2- 172.21.15.18:0/3488581038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f8c074cd0 0x7f3f8c10e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 --2- 172.21.15.18:0/3488581038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f8c0743b0 0x7f3f8c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 -- 172.21.15.18:0/3488581038 >> 172.21.15.18:0/3488581038 conn(0x7f3f8c0fd9f0 msgr2=0x7f3f8c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 -- 172.21.15.18:0/3488581038 shutdown_connections 2024-04-16T17:05:29.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.287+0000 7f3f8b200640 1 -- 172.21.15.18:0/3488581038 wait complete. 2024-04-16T17:05:29.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.288+0000 7f3f8b200640 1 Processor -- start 2024-04-16T17:05:29.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.288+0000 7f3f8b200640 1 -- start start 2024-04-16T17:05:29.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.288+0000 7f3f8b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c0743b0 0x7f3f8c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.288+0000 7f3f8b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f8c074cd0 0x7f3f8c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f8b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f8c10e7a0 0x7f3f8c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f8b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c114480 con 0x7f3f8c0743b0 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f8b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c114300 con 0x7f3f8c074cd0 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f8b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c114600 con 0x7f3f8c10e7a0 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f89e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c0743b0 0x7f3f8c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f89e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c0743b0 0x7f3f8c1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56566/0 (socket says 172.21.15.18:56566) 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f89e00640 1 -- 172.21.15.18:0/1525277783 learned_addr learned my addr 172.21.15.18:0/1525277783 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f83e00640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f8c074cd0 0x7f3f8c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f8a800640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f8c10e7a0 0x7f3f8c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f89e00640 1 -- 172.21.15.18:0/1525277783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f8c10e7a0 msgr2=0x7f3f8c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f89e00640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f8c10e7a0 0x7f3f8c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f89e00640 1 -- 172.21.15.18:0/1525277783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f8c074cd0 msgr2=0x7f3f8c1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f89e00640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f8c074cd0 0x7f3f8c1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f89e00640 1 -- 172.21.15.18:0/1525277783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c19af90 con 0x7f3f8c0743b0 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f8a800640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f8c10e7a0 0x7f3f8c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.289+0000 7f3f83e00640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f8c074cd0 0x7f3f8c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.290+0000 7f3f89e00640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c0743b0 0x7f3f8c1a0650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00e9f0 tx=0x7f3f7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.290+0000 7f3f82a00640 1 -- 172.21.15.18:0/1525277783 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00cd80 con 0x7f3f8c0743b0 2024-04-16T17:05:29.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.290+0000 7f3f82a00640 1 -- 172.21.15.18:0/1525277783 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f7c010910 con 0x7f3f8c0743b0 2024-04-16T17:05:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.290+0000 7f3f82a00640 1 -- 172.21.15.18:0/1525277783 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c018990 con 0x7f3f8c0743b0 2024-04-16T17:05:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.290+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c19b280 con 0x7f3f8c0743b0 2024-04-16T17:05:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.290+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c1a74a0 con 0x7f3f8c0743b0 2024-04-16T17:05:29.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.292+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c075a60 con 0x7f3f8c0743b0 2024-04-16T17:05:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.294+0000 7f3f82a00640 1 -- 172.21.15.18:0/1525277783 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f3f7c010ac0 con 0x7f3f8c0743b0 2024-04-16T17:05:29.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.295+0000 7f3f82a00640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3f700a3f40 0x7f3f700a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.295+0000 7f3f82a00640 1 -- 172.21.15.18:0/1525277783 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f3f7c0d1130 con 0x7f3f8c0743b0 2024-04-16T17:05:29.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.295+0000 7f3f83e00640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3f700a3f40 0x7f3f700a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.296+0000 7f3f83e00640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3f700a3f40 0x7f3f700a6400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f740095f0 tx=0x7f3f74005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.300+0000 7f3f82a00640 1 -- 172.21.15.18:0/1525277783 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f7c09cb90 con 0x7f3f8c0743b0 2024-04-16T17:05:29.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.550+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f8c109920 con 0x7f3f8c0743b0 2024-04-16T17:05:29.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.552+0000 7f3f82a00640 1 -- 172.21.15.18:0/1525277783 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7f3f7c0a0e90 con 0x7f3f8c0743b0 2024-04-16T17:05:29.553 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:05:29.554 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:05:29.554 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:05:29.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3f700a3f40 msgr2=0x7f3f700a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3f700a3f40 0x7f3f700a6400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f740095f0 tx=0x7f3f74005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:29.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c0743b0 msgr2=0x7f3f8c1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c0743b0 0x7f3f8c1a0650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00e9f0 tx=0x7f3f7c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 shutdown_connections 2024-04-16T17:05:29.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3f700a3f40 0x7f3f700a6400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f8c10e7a0 0x7f3f8c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f8c074cd0 0x7f3f8c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 --2- 172.21.15.18:0/1525277783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f8c0743b0 0x7f3f8c1a0650 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.554+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 >> 172.21.15.18:0/1525277783 conn(0x7f3f8c0fd9f0 msgr2=0x7f3f8c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.555+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 shutdown_connections 2024-04-16T17:05:29.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.555+0000 7f3f8b200640 1 -- 172.21.15.18:0/1525277783 wait complete. 2024-04-16T17:05:29.573 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:05:29.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 -- 172.21.15.18:0/4191833955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c103940 msgr2=0x7f257c10be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 --2- 172.21.15.18:0/4191833955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c103940 0x7f257c10be20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2568009900 tx=0x7f256802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 -- 172.21.15.18:0/4191833955 shutdown_connections 2024-04-16T17:05:29.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 --2- 172.21.15.18:0/4191833955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f257c10c3f0 0x7f257c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 --2- 172.21.15.18:0/4191833955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c103940 0x7f257c10be20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 --2- 172.21.15.18:0/4191833955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f257c103020 0x7f257c103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 -- 172.21.15.18:0/4191833955 >> 172.21.15.18:0/4191833955 conn(0x7f257c077ab0 msgr2=0x7f257c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 -- 172.21.15.18:0/4191833955 shutdown_connections 2024-04-16T17:05:29.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.774+0000 7f2580e00640 1 -- 172.21.15.18:0/4191833955 wait complete. 2024-04-16T17:05:29.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.775+0000 7f2580e00640 1 Processor -- start 2024-04-16T17:05:29.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.775+0000 7f2580e00640 1 -- start start 2024-04-16T17:05:29.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.775+0000 7f2580e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f257c103020 0x7f257c103e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.775+0000 7f2580e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f257c103940 0x7f257c1043d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.775+0000 7f2580e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c10c3f0 0x7f257c104910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f2580e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1121f0 con 0x7f257c103020 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f2580e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f257c112070 con 0x7f257c103940 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f2580e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f257c112370 con 0x7f257c10c3f0 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f257c103020 0x7f257c103e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f257c103940 0x7f257c1043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f257c103020 0x7f257c103e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56584/0 (socket says 172.21.15.18:56584) 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257b400640 1 -- 172.21.15.18:0/2945491738 learned_addr learned my addr 172.21.15.18:0/2945491738 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f257c103940 0x7f257c1043d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:43900/0 (socket says 172.21.15.18:43900) 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257be00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c10c3f0 0x7f257c104910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257aa00640 1 -- 172.21.15.18:0/2945491738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c10c3f0 msgr2=0x7f257c104910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257aa00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c10c3f0 0x7f257c104910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257aa00640 1 -- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f257c103020 msgr2=0x7f257c103e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.776+0000 7f257aa00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f257c103020 0x7f257c103e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.777+0000 7f257aa00640 1 -- 172.21.15.18:0/2945491738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c1ab0a0 con 0x7f257c103940 2024-04-16T17:05:29.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.777+0000 7f257be00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c10c3f0 0x7f257c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:29.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.777+0000 7f257aa00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f257c103940 0x7f257c1043d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f256802c200 tx=0x7f2568005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.777+0000 7f2573e00640 1 -- 172.21.15.18:0/2945491738 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568041020 con 0x7f257c103940 2024-04-16T17:05:29.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.778+0000 7f2573e00640 1 -- 172.21.15.18:0/2945491738 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f256802cdf0 con 0x7f257c103940 2024-04-16T17:05:29.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.778+0000 7f2573e00640 1 -- 172.21.15.18:0/2945491738 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568003990 con 0x7f257c103940 2024-04-16T17:05:29.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.778+0000 7f2580e00640 1 -- 172.21.15.18:0/2945491738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257c1ab330 con 0x7f257c103940 2024-04-16T17:05:29.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.778+0000 7f2580e00640 1 -- 172.21.15.18:0/2945491738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257c1ab7d0 con 0x7f257c103940 2024-04-16T17:05:29.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.782+0000 7f25851c6740 1 Processor -- start 2024-04-16T17:05:29.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.783+0000 7f25851c6740 1 -- start start 2024-04-16T17:05:29.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.783+0000 7f2573e00640 1 -- 172.21.15.18:0/2945491738 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f2568044020 con 0x7f257c103940 2024-04-16T17:05:29.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.783+0000 7f25851c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4bd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.783+0000 7f25851c6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 0x55f36b4c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.783+0000 7f25851c6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 0x55f36b4c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.783+0000 7f25851c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f36b422020 con 0x55f36b4bcf10 2024-04-16T17:05:29.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.784+0000 7f25851c6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f36b3c8460 con 0x55f36b4c6410 2024-04-16T17:05:29.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.784+0000 7f25851c6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f36b41dfe0 con 0x55f36b4bd830 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.784+0000 7f257aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 0x55f36b4c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.784+0000 7f257be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4bd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.784+0000 7f257be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4bd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56600/0 (socket says 172.21.15.18:56600) 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.784+0000 7f257be00640 1 -- 172.21.15.18:0/204921148 learned_addr learned my addr 172.21.15.18:0/204921148 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.784+0000 7f257b400640 1 --2- 172.21.15.18:0/204921148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 0x55f36b4c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.785+0000 7f257be00640 1 -- 172.21.15.18:0/204921148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 msgr2=0x55f36b4c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.785+0000 7f257be00640 1 --2- 172.21.15.18:0/204921148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 0x55f36b4c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.785+0000 7f257be00640 1 -- 172.21.15.18:0/204921148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 msgr2=0x55f36b4c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.785+0000 7f257be00640 1 --2- 172.21.15.18:0/204921148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 0x55f36b4c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.785+0000 7f257be00640 1 -- 172.21.15.18:0/204921148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36b3d0040 con 0x55f36b4bcf10 2024-04-16T17:05:29.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.785+0000 7f257be00640 1 --2- 172.21.15.18:0/204921148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4bd2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2574011790 tx=0x7f2574011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed36316cccf8d5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.785+0000 7f2573e00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f25600a3f50 0x7f25600a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.786+0000 7f2573e00640 1 -- 172.21.15.18:0/2945491738 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f25680f2370 con 0x7f257c103940 2024-04-16T17:05:29.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.786+0000 7f2570c00640 1 -- 172.21.15.18:0/204921148 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574004d20 con 0x55f36b4bcf10 2024-04-16T17:05:29.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.786+0000 7f257b400640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f25600a3f50 0x7f25600a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.786+0000 7f2570c00640 1 -- 172.21.15.18:0/204921148 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2574003970 con 0x55f36b4bcf10 2024-04-16T17:05:29.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.786+0000 7f2570c00640 1 -- 172.21.15.18:0/204921148 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574004560 con 0x55f36b4bcf10 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 -- 172.21.15.18:0/204921148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 msgr2=0x55f36b4bd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f257b400640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f25600a3f50 0x7f25600a6410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55f36b4bc9e0 tx=0x7f256c005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 --2- 172.21.15.18:0/204921148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4bd2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2574011790 tx=0x7f2574011c50 comp rx=0 tx=0).stop 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 -- 172.21.15.18:0/204921148 shutdown_connections 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 --2- 172.21.15.18:0/204921148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 0x55f36b4c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 --2- 172.21.15.18:0/204921148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 0x55f36b4c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 --2- 172.21.15.18:0/204921148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4bd2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 -- 172.21.15.18:0/204921148 >> 172.21.15.18:0/204921148 conn(0x55f36b4b7640 msgr2=0x55f36b4b7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 -- 172.21.15.18:0/204921148 shutdown_connections 2024-04-16T17:05:29.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.787+0000 7f25851c6740 1 -- 172.21.15.18:0/204921148 wait complete. 2024-04-16T17:05:29.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.788+0000 7f25851c6740 1 -- start start 2024-04-16T17:05:29.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.788+0000 7f25851c6740 1 Processor -- start 2024-04-16T17:05:29.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.789+0000 7f25851c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.789+0000 7f25851c6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.789+0000 7f25851c6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.789+0000 7f25851c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f36b4c9200 con 0x55f36b4bcf10 2024-04-16T17:05:29.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.789+0000 7f25851c6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f36b3db830 con 0x55f36b4bd830 2024-04-16T17:05:29.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.789+0000 7f25851c6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f36b4c9380 con 0x55f36b4c6410 2024-04-16T17:05:29.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.789+0000 7f25851c6740 10 client.0 _ll_register_callbacks cb 0x55f36b3e3150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.789+0000 7f257b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56616/0 (socket says 172.21.15.18:56616) 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257b400640 1 -- 172.21.15.18:0/3179653580 learned_addr learned my addr 172.21.15.18:0/3179653580 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257aa00640 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257be00640 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257aa00640 1 -- 172.21.15.18:0/3179653580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 msgr2=0x55f36b4c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257aa00640 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257aa00640 1 -- 172.21.15.18:0/3179653580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 msgr2=0x55f36b4c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257aa00640 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257aa00640 1 -- 172.21.15.18:0/3179653580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36b3df920 con 0x55f36b4bd830 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.790+0000 7f257aa00640 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4c3a80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f25680ef5e0 tx=0x7f2568004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d27b7a8e876bc115 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.791+0000 7f2565600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.791+0000 7f2565600640 1 -- 172.21.15.18:0/3179653580 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680efe60 con 0x55f36b4bd830 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.791+0000 7f25851c6740 1 -- 172.21.15.18:0/3179653580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2574003970 con 0x55f36b4bd830 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.791+0000 7f2565600640 1 -- 172.21.15.18:0/3179653580 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2568005250 con 0x55f36b4bd830 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.791+0000 7f2565600640 1 -- 172.21.15.18:0/3179653580 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25680bdc00 con 0x55f36b4bd830 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.791+0000 7f25851c6740 10 client.25847 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.791+0000 7f25851c6740 1 -- 172.21.15.18:0/3179653580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f36b3ad7f0 con 0x55f36b4bd830 2024-04-16T17:05:29.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.791+0000 7f2565600640 1 -- 172.21.15.18:0/3179653580 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2574003970 con 0x55f36b4bd830 2024-04-16T17:05:29.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.792+0000 7f2565600640 1 -- 172.21.15.18:0/3179653580 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f25680c6610 con 0x55f36b4bd830 2024-04-16T17:05:29.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.793+0000 7f25851c6740 10 client.25847 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:29.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.793+0000 7f25851c6740 10 client.25847 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:05:29.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.793+0000 7f25851c6740 20 client.25847 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:29.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.793+0000 7f25851c6740 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f36b4d6730 0x55f36b4d8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.793+0000 7f25851c6740 4 client.25847 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:29.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.793+0000 7f25851c6740 1 -- 172.21.15.18:0/3179653580 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f36b42fa40 con 0x55f36b4d6730 2024-04-16T17:05:29.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.793+0000 7f257b400640 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f36b4d6730 0x55f36b4d8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.794+0000 7f257b400640 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f36b4d6730 0x55f36b4d8bb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af140711db6b01fb server_cookie=394b2c4587631aa8 in_seq=0 out_seq=0 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.794+0000 7f2565600640 10 client.25847 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f2565600640 1 -- 172.21.15.18:0/3179653580 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f36b42fa40 con 0x55f36b4d6730 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f2565600640 10 client.25847 handle_command_reply: tid=0 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 client.25847 shutdown 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 -- 172.21.15.18:0/3179653580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 msgr2=0x55f36b4c3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4c3a80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f25680ef5e0 tx=0x7f2568004ec0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 -- 172.21.15.18:0/3179653580 shutdown_connections 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f36b4d6730 0x55f36b4d8bb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f2565600640 0 client.25847 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:05:29.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4c3a80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 --2- 172.21.15.18:0/3179653580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 -- 172.21.15.18:0/3179653580 >> 172.21.15.18:0/3179653580 conn(0x55f36b4b7640 msgr2=0x55f36b4b9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 -- 172.21.15.18:0/3179653580 shutdown_connections 2024-04-16T17:05:29.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.795+0000 7f25851c6740 1 -- 172.21.15.18:0/3179653580 wait complete. 2024-04-16T17:05:29.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.796+0000 7f25851c6740 20 client.25847 trim_cache size 0 max 16384 2024-04-16T17:05:29.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.810+0000 7f25851c6740 1 Processor -- start 2024-04-16T17:05:29.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.810+0000 7f25851c6740 1 -- start start 2024-04-16T17:05:29.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.810+0000 7f25851c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.810+0000 7f25851c6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f25851c6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f25851c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1b8340 con 0x55f36b4bcf10 2024-04-16T17:05:29.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f25851c6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1b5c10 con 0x55f36b4bd830 2024-04-16T17:05:29.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f25851c6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1b3a60 con 0x55f36b4c6410 2024-04-16T17:05:29.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56618/0 (socket says 172.21.15.18:56618) 2024-04-16T17:05:29.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257b400640 1 -- 172.21.15.18:0/3784015594 learned_addr learned my addr 172.21.15.18:0/3784015594 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257aa00640 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257be00640 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257aa00640 1 -- 172.21.15.18:0/3784015594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 msgr2=0x55f36b4c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257aa00640 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257aa00640 1 -- 172.21.15.18:0/3784015594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 msgr2=0x55f36b4c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.811+0000 7f257aa00640 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.812+0000 7f257aa00640 1 -- 172.21.15.18:0/3784015594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36b1343f0 con 0x55f36b4bd830 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.812+0000 7f257b400640 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.812+0000 7f257be00640 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.812+0000 7f257aa00640 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4d20c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f25680ba290 tx=0x7f2568000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7803afd89437ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.812+0000 7f2567e00640 1 -- 172.21.15.18:0/3784015594 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680efe60 con 0x55f36b4bd830 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.812+0000 7f2567e00640 1 -- 172.21.15.18:0/3784015594 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25680f5b10 con 0x55f36b4bd830 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.812+0000 7f2567e00640 1 -- 172.21.15.18:0/3784015594 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680bd790 con 0x55f36b4bd830 2024-04-16T17:05:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 -- 172.21.15.18:0/3784015594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 msgr2=0x55f36b4d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4d20c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f25680ba290 tx=0x7f2568000bc0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 -- 172.21.15.18:0/3784015594 shutdown_connections 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4c6410 0x55f36b4c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4bd830 0x55f36b4d20c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 --2- 172.21.15.18:0/3784015594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 -- 172.21.15.18:0/3784015594 >> 172.21.15.18:0/3784015594 conn(0x55f36b4b7640 msgr2=0x55f36b4c3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 -- 172.21.15.18:0/3784015594 shutdown_connections 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.813+0000 7f25851c6740 1 -- 172.21.15.18:0/3784015594 wait complete. 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.814+0000 7f25851c6740 1 -- start start 2024-04-16T17:05:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.814+0000 7f25851c6740 1 Processor -- start 2024-04-16T17:05:29.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f25851c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f25851c6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 0x55f36b4c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f25851c6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 0x55f36b53bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f25851c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1b40d0 con 0x55f36b4bcf10 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f25851c6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1b5540 con 0x55f36b4c6410 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f25851c6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1b47a0 con 0x55f36b4bd830 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f25851c6740 10 client.0 _ll_register_callbacks cb 0x7f2568005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f257b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f257b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56628/0 (socket says 172.21.15.18:56628) 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f257b400640 1 -- 172.21.15.18:0/3878705058 learned_addr learned my addr 172.21.15.18:0/3878705058 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f257be00640 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 0x55f36b53bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.815+0000 7f257aa00640 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 0x55f36b4c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.816+0000 7f257b400640 1 -- 172.21.15.18:0/3878705058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 msgr2=0x55f36b4c0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.816+0000 7f257b400640 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 0x55f36b4c0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.816+0000 7f257b400640 1 -- 172.21.15.18:0/3878705058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 msgr2=0x55f36b53bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.816+0000 7f257b400640 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 0x55f36b53bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.816+0000 7f257b400640 1 -- 172.21.15.18:0/3878705058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36b1fe7d0 con 0x55f36b4bcf10 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.816+0000 7f257b400640 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c05e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55f36b4d9470 tx=0x7f256c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ed4fd17a71f350 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.816+0000 7f2565600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.817+0000 7f2565600640 1 -- 172.21.15.18:0/3878705058 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c01db70 con 0x55f36b4bcf10 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.817+0000 7f2565600640 1 -- 172.21.15.18:0/3878705058 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f256c01dd10 con 0x55f36b4bcf10 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.817+0000 7f25851c6740 1 -- 172.21.15.18:0/3878705058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25680f5b10 con 0x55f36b4bcf10 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.817+0000 7f2565600640 1 -- 172.21.15.18:0/3878705058 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f256c03b050 con 0x55f36b4bcf10 2024-04-16T17:05:29.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.817+0000 7f25851c6740 10 client.15717 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.817+0000 7f25851c6740 1 -- 172.21.15.18:0/3878705058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f36b53d250 con 0x55f36b4bcf10 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f2565600640 1 -- 172.21.15.18:0/3878705058 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25680f5b10 con 0x55f36b4bcf10 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f2565600640 1 -- 172.21.15.18:0/3878705058 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f256c0047a0 con 0x55f36b4bcf10 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f25851c6740 10 client.15717 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f25851c6740 10 client.15717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f25851c6740 20 client.15717 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f25851c6740 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f36b53da00 0x55f36b53fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f25851c6740 4 client.15717 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f25851c6740 1 -- 172.21.15.18:0/3878705058 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f36b4298b0 con 0x55f36b53da00 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.818+0000 7f257aa00640 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f36b53da00 0x55f36b53fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.819+0000 7f257aa00640 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f36b53da00 0x55f36b53fe20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee5af91873c75136 server_cookie=d427fdc84a705f91 in_seq=0 out_seq=0 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.819+0000 7f2565600640 10 client.15717 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:05:29.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.819+0000 7f2565600640 1 -- 172.21.15.18:0/3878705058 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f36b4298b0 con 0x55f36b53da00 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.819+0000 7f2565600640 10 client.15717 handle_command_reply: tid=0 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 client.15717 shutdown 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 -- 172.21.15.18:0/3878705058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 msgr2=0x55f36b4c05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c05e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55f36b4d9470 tx=0x7f256c0046c0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 -- 172.21.15.18:0/3878705058 shutdown_connections 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f36b53da00 0x55f36b53fe20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f36b4c6410 0x55f36b53bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f36b4bd830 0x55f36b4c0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 --2- 172.21.15.18:0/3878705058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f36b4bcf10 0x55f36b4c05e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f2565600640 0 client.15717 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 -- 172.21.15.18:0/3878705058 >> 172.21.15.18:0/3878705058 conn(0x55f36b4b7640 msgr2=0x55f36b4c3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 -- 172.21.15.18:0/3878705058 shutdown_connections 2024-04-16T17:05:29.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 1 -- 172.21.15.18:0/3878705058 wait complete. 2024-04-16T17:05:29.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.820+0000 7f25851c6740 20 client.15717 trim_cache size 0 max 16384 2024-04-16T17:05:29.826 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 -- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f25600a3f50 msgr2=0x7f25600a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f25600a3f50 0x7f25600a6410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55f36b4bc9e0 tx=0x7f256c005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 -- 172.21.15.18:0/2945491738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f257c103940 msgr2=0x7f257c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f257c103940 0x7f257c1043d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f256802c200 tx=0x7f2568005e80 comp rx=0 tx=0).stop 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 -- 172.21.15.18:0/2945491738 shutdown_connections 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f25600a3f50 0x7f25600a6410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c10c3f0 0x7f257c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f257c103940 0x7f257c1043d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 --2- 172.21.15.18:0/2945491738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f257c103020 0x7f257c103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.821+0000 7f2580e00640 1 -- 172.21.15.18:0/2945491738 >> 172.21.15.18:0/2945491738 conn(0x7f257c077ab0 msgr2=0x7f257c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.822+0000 7f2580e00640 1 -- 172.21.15.18:0/2945491738 shutdown_connections 2024-04-16T17:05:29.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:29.822+0000 7f2580e00640 1 -- 172.21.15.18:0/2945491738 wait complete. 2024-04-16T17:05:29.843 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:29 smithi018 ceph-mon[31776]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:05:29.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:29 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:05:29.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:29 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:29.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:29 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:05:29.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:29 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:29.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:29 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:05:29.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:29 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1525277783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:29.847 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:05:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:29 smithi022 ceph-mon[36023]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:05:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:29 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:05:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:29 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:29 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:05:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:29 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:29.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:29 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:05:29.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:29 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1525277783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:30.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:29 smithi043 ceph-mon[36901]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:05:30.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:29 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:05:30.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:29 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:30.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:29 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:05:30.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:29 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:30.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:29 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:05:30.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:29 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1525277783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:30.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.048+0000 7fdd95400640 1 -- 172.21.15.18:0/1373614585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 msgr2=0x7fdd9010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.048+0000 7fdd95400640 1 --2- 172.21.15.18:0/1373614585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 0x7fdd9010e180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009860 tx=0x7fdd7c02bc00 comp rx=0 tx=0).stop 2024-04-16T17:05:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.049+0000 7fdd95400640 1 -- 172.21.15.18:0/1373614585 shutdown_connections 2024-04-16T17:05:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.049+0000 7fdd95400640 1 --2- 172.21.15.18:0/1373614585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9010e6c0 0x7fdd90110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.049+0000 7fdd95400640 1 --2- 172.21.15.18:0/1373614585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 0x7fdd9010e180 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.049+0000 7fdd95400640 1 --2- 172.21.15.18:0/1373614585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd900743b0 0x7fdd90074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.049+0000 7fdd95400640 1 -- 172.21.15.18:0/1373614585 >> 172.21.15.18:0/1373614585 conn(0x7fdd900fd950 msgr2=0x7fdd900ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.049+0000 7fdd95400640 1 -- 172.21.15.18:0/1373614585 shutdown_connections 2024-04-16T17:05:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.049+0000 7fdd95400640 1 -- 172.21.15.18:0/1373614585 wait complete. 2024-04-16T17:05:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.050+0000 7fdd95400640 1 Processor -- start 2024-04-16T17:05:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.051+0000 7fdd95400640 1 -- start start 2024-04-16T17:05:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.051+0000 7fdd95400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd900743b0 0x7fdd901a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.051+0000 7fdd95400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 0x7fdd901a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.051+0000 7fdd95400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9010e6c0 0x7fdd9019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.051+0000 7fdd95400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901143d0 con 0x7fdd9010e6c0 2024-04-16T17:05:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.051+0000 7fdd95400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90114250 con 0x7fdd900743b0 2024-04-16T17:05:30.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.051+0000 7fdd95400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90114550 con 0x7fdd90074cd0 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd900743b0 0x7fdd901a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd900743b0 0x7fdd901a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:43938/0 (socket says 172.21.15.18:43938) 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 -- 172.21.15.18:0/560003556 learned_addr learned my addr 172.21.15.18:0/560003556 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8f400640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 0x7fdd901a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd94a00640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9010e6c0 0x7fdd9019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 -- 172.21.15.18:0/560003556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 msgr2=0x7fdd901a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 0x7fdd901a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 -- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9010e6c0 msgr2=0x7fdd9019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9010e6c0 0x7fdd9019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 -- 172.21.15.18:0/560003556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9019b050 con 0x7fdd900743b0 2024-04-16T17:05:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8f400640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 0x7fdd901a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:30.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.052+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd900743b0 0x7fdd901a0680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000e9c0 tx=0x7fdd8000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.053+0000 7fdd8cc00640 1 -- 172.21.15.18:0/560003556 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8000ccc0 con 0x7fdd900743b0 2024-04-16T17:05:30.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.053+0000 7fdd8cc00640 1 -- 172.21.15.18:0/560003556 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd80005500 con 0x7fdd900743b0 2024-04-16T17:05:30.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.053+0000 7fdd8cc00640 1 -- 172.21.15.18:0/560003556 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80010430 con 0x7fdd900743b0 2024-04-16T17:05:30.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.054+0000 7fdd95400640 1 -- 172.21.15.18:0/560003556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9019b340 con 0x7fdd900743b0 2024-04-16T17:05:30.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.054+0000 7fdd95400640 1 -- 172.21.15.18:0/560003556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd901a7600 con 0x7fdd900743b0 2024-04-16T17:05:30.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.059+0000 7fdd99649740 1 Processor -- start 2024-04-16T17:05:30.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.059+0000 7fdd8cc00640 1 -- 172.21.15.18:0/560003556 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fdd800046c0 con 0x7fdd900743b0 2024-04-16T17:05:30.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.059+0000 7fdd99649740 1 -- start start 2024-04-16T17:05:30.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.059+0000 7fdd99649740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e481f10 0x56357e4822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.059+0000 7fdd99649740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e48aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd99649740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e48b410 0x56357e48d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd99649740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56357e3e7020 con 0x56357e482830 2024-04-16T17:05:30.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd99649740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56357e38d460 con 0x56357e481f10 2024-04-16T17:05:30.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd99649740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56357e3e2fe0 con 0x56357e48b410 2024-04-16T17:05:30.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e48b410 0x56357e48d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e48b410 0x56357e48d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56170/0 (socket says 172.21.15.18:56170) 2024-04-16T17:05:30.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd8f400640 1 -- 172.21.15.18:0/767397182 learned_addr learned my addr 172.21.15.18:0/767397182 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e48aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.060+0000 7fdd94a00640 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e481f10 0x56357e4822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd8f400640 1 -- 172.21.15.18:0/767397182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e481f10 msgr2=0x56357e4822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd8f400640 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e481f10 0x56357e4822f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd8f400640 1 -- 172.21.15.18:0/767397182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 msgr2=0x56357e48aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd8f400640 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e48aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd8f400640 1 -- 172.21.15.18:0/767397182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357e395040 con 0x56357e48b410 2024-04-16T17:05:30.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e48aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:30.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd94a00640 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e481f10 0x56357e4822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:30.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd8cc00640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd740a3ed0 0x7fdd740a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.061+0000 7fdd8f400640 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e48b410 0x56357e48d800 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009860 tx=0x7fdd7c0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7499bdec60a32f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.062+0000 7fdd8cc00640 1 -- 172.21.15.18:0/560003556 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fdd800d1300 con 0x7fdd900743b0 2024-04-16T17:05:30.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.062+0000 7fdd85200640 1 -- 172.21.15.18:0/767397182 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c044020 con 0x56357e48b410 2024-04-16T17:05:30.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.062+0000 7fdd85200640 1 -- 172.21.15.18:0/767397182 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd7c034920 con 0x56357e48b410 2024-04-16T17:05:30.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.062+0000 7fdd94a00640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd740a3ed0 0x7fdd740a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.062+0000 7fdd85200640 1 -- 172.21.15.18:0/767397182 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c039040 con 0x56357e48b410 2024-04-16T17:05:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.063+0000 7fdd99649740 1 -- 172.21.15.18:0/767397182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e48b410 msgr2=0x56357e48d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.063+0000 7fdd99649740 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e48b410 0x56357e48d800 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009860 tx=0x7fdd7c0056d0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.063+0000 7fdd94a00640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd740a3ed0 0x7fdd740a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005fd0 tx=0x7fdd88009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.063+0000 7fdd99649740 1 -- 172.21.15.18:0/767397182 shutdown_connections 2024-04-16T17:05:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.063+0000 7fdd99649740 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e48b410 0x56357e48d800 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.063+0000 7fdd99649740 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e48aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.063+0000 7fdd99649740 1 --2- 172.21.15.18:0/767397182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e481f10 0x56357e4822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.063+0000 7fdd99649740 1 -- 172.21.15.18:0/767397182 >> 172.21.15.18:0/767397182 conn(0x56357e47c640 msgr2=0x56357e47ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.064+0000 7fdd99649740 1 -- 172.21.15.18:0/767397182 shutdown_connections 2024-04-16T17:05:30.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.064+0000 7fdd99649740 1 -- 172.21.15.18:0/767397182 wait complete. 2024-04-16T17:05:30.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.064+0000 7fdd99649740 1 -- start start 2024-04-16T17:05:30.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.065+0000 7fdd99649740 1 Processor -- start 2024-04-16T17:05:30.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.065+0000 7fdd99649740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e4883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.065+0000 7fdd99649740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e482830 0x56357e4888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.065+0000 7fdd99649740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e48b410 0x56357e488e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.065+0000 7fdd99649740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56357e48e1a0 con 0x56357e48b410 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.065+0000 7fdd99649740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56357e3a0830 con 0x56357e482830 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.065+0000 7fdd99649740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56357e48e320 con 0x56357e481f10 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.065+0000 7fdd99649740 10 client.0 _ll_register_callbacks cb 0x56357e3a8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.066+0000 7fdd94a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e48b410 0x56357e488e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.066+0000 7fdd94a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e48b410 0x56357e488e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56664/0 (socket says 172.21.15.18:56664) 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.066+0000 7fdd94a00640 1 -- 172.21.15.18:0/956617882 learned_addr learned my addr 172.21.15.18:0/956617882 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.066+0000 7fdd8f400640 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e482830 0x56357e4888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.066+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e4883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.066+0000 7fdd94a00640 1 -- 172.21.15.18:0/956617882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 msgr2=0x56357e4883a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd94a00640 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e4883a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd94a00640 1 -- 172.21.15.18:0/956617882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e482830 msgr2=0x56357e4888e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd94a00640 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e482830 0x56357e4888e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd94a00640 1 -- 172.21.15.18:0/956617882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357e3a4920 con 0x56357e48b410 2024-04-16T17:05:30.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd8f400640 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e482830 0x56357e4888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:30.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd94a00640 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e48b410 0x56357e488e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdd88010c20 tx=0x7fdd88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28601b9559e5e4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd79600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:05:30.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd79600640 1 -- 172.21.15.18:0/956617882 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8801fc80 con 0x56357e48b410 2024-04-16T17:05:30.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.067+0000 7fdd79600640 1 -- 172.21.15.18:0/956617882 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fdd8801c3f0 con 0x56357e48b410 2024-04-16T17:05:30.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.068+0000 7fdd99649740 1 -- 172.21.15.18:0/956617882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd7c034920 con 0x56357e48b410 2024-04-16T17:05:30.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.068+0000 7fdd79600640 1 -- 172.21.15.18:0/956617882 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd8801fe20 con 0x56357e48b410 2024-04-16T17:05:30.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.068+0000 7fdd99649740 10 client.15729 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.068+0000 7fdd99649740 1 -- 172.21.15.18:0/956617882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56357e3727f0 con 0x56357e48b410 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.069+0000 7fdd79600640 1 -- 172.21.15.18:0/956617882 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd7c034920 con 0x56357e48b410 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.069+0000 7fdd79600640 1 -- 172.21.15.18:0/956617882 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdd88026070 con 0x56357e48b410 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.069+0000 7fdd99649740 10 client.15729 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.069+0000 7fdd99649740 10 client.15729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.069+0000 7fdd99649740 20 client.15729 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.069+0000 7fdd99649740 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56357e4a3ed0 0x56357e4a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.069+0000 7fdd99649740 4 client.15729 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.069+0000 7fdd99649740 1 -- 172.21.15.18:0/956617882 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357e3f4a40 con 0x56357e4a3ed0 2024-04-16T17:05:30.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.070+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56357e4a3ed0 0x56357e4a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.070+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56357e4a3ed0 0x56357e4a6320 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17102951efd14103 server_cookie=5d50638c1c2f126e in_seq=0 out_seq=0 2024-04-16T17:05:30.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.070+0000 7fdd79600640 10 client.15729 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:05:30.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.071+0000 7fdd79600640 1 -- 172.21.15.18:0/956617882 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56357e3f4a40 con 0x56357e4a3ed0 2024-04-16T17:05:30.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.071+0000 7fdd79600640 10 client.15729 handle_command_reply: tid=0 2024-04-16T17:05:30.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 client.15729 shutdown 2024-04-16T17:05:30.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 -- 172.21.15.18:0/956617882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e48b410 msgr2=0x56357e488e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e48b410 0x56357e488e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdd88010c20 tx=0x7fdd88004ab0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 -- 172.21.15.18:0/956617882 shutdown_connections 2024-04-16T17:05:30.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56357e4a3ed0 0x56357e4a6320 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e48b410 0x56357e488e20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e482830 0x56357e4888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 --2- 172.21.15.18:0/956617882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e4883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd99649740 1 -- 172.21.15.18:0/956617882 >> 172.21.15.18:0/956617882 conn(0x56357e47c640 msgr2=0x56357e47e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.072+0000 7fdd79600640 0 client.15729 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:05:30.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.073+0000 7fdd99649740 1 -- 172.21.15.18:0/956617882 shutdown_connections 2024-04-16T17:05:30.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.073+0000 7fdd99649740 1 -- 172.21.15.18:0/956617882 wait complete. 2024-04-16T17:05:30.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.073+0000 7fdd99649740 20 client.15729 trim_cache size 0 max 16384 2024-04-16T17:05:30.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.101+0000 7fdd99649740 1 Processor -- start 2024-04-16T17:05:30.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.101+0000 7fdd99649740 1 -- start start 2024-04-16T17:05:30.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd99649740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e481f10 0x56357e48aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd99649740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e482830 0x56357e49f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd99649740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e47e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd99649740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901afa40 con 0x56357e481f10 2024-04-16T17:05:30.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd99649740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901ad310 con 0x56357e48b410 2024-04-16T17:05:30.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd99649740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901ab160 con 0x56357e482830 2024-04-16T17:05:30.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e482830 0x56357e49f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e482830 0x56357e49f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56184/0 (socket says 172.21.15.18:56184) 2024-04-16T17:05:30.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd8f400640 1 -- 172.21.15.18:0/2412206299 learned_addr learned my addr 172.21.15.18:0/2412206299 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e481f10 0x56357e48aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.102+0000 7fdd94a00640 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e47e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd8f400640 1 -- 172.21.15.18:0/2412206299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 msgr2=0x56357e47e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd8f400640 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e47e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd8f400640 1 -- 172.21.15.18:0/2412206299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e481f10 msgr2=0x56357e48aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd8f400640 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e481f10 0x56357e48aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd8f400640 1 -- 172.21.15.18:0/2412206299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357e0f93f0 con 0x56357e482830 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd8f400640 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e482830 0x56357e49f920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009830 tx=0x7fdd7c02c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf82097f0676210d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd7be00640 1 -- 172.21.15.18:0/2412206299 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c044040 con 0x56357e482830 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd7be00640 1 -- 172.21.15.18:0/2412206299 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd7c034440 con 0x56357e482830 2024-04-16T17:05:30.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.103+0000 7fdd7be00640 1 -- 172.21.15.18:0/2412206299 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c039040 con 0x56357e482830 2024-04-16T17:05:30.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.104+0000 7fdd99649740 1 -- 172.21.15.18:0/2412206299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e482830 msgr2=0x56357e49f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.104+0000 7fdd99649740 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e482830 0x56357e49f920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009830 tx=0x7fdd7c02c8c0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.104+0000 7fdd99649740 1 -- 172.21.15.18:0/2412206299 shutdown_connections 2024-04-16T17:05:30.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.104+0000 7fdd99649740 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e47e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.104+0000 7fdd99649740 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e482830 0x56357e49f920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.104+0000 7fdd99649740 1 --2- 172.21.15.18:0/2412206299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e481f10 0x56357e48aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.104+0000 7fdd99649740 1 -- 172.21.15.18:0/2412206299 >> 172.21.15.18:0/2412206299 conn(0x56357e47c640 msgr2=0x56357e47da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.105+0000 7fdd99649740 1 -- 172.21.15.18:0/2412206299 shutdown_connections 2024-04-16T17:05:30.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.105+0000 7fdd99649740 1 -- 172.21.15.18:0/2412206299 wait complete. 2024-04-16T17:05:30.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.105+0000 7fdd99649740 1 -- start start 2024-04-16T17:05:30.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.105+0000 7fdd99649740 1 Processor -- start 2024-04-16T17:05:30.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd99649740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e49c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd99649740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e484cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd99649740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e485210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd99649740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901ab7d0 con 0x56357e482830 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd99649740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901acc40 con 0x56357e48b410 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd99649740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901abea0 con 0x56357e481f10 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd99649740 10 client.0 _ll_register_callbacks cb 0x7fdd88009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd8fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e49c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd8fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e49c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56190/0 (socket says 172.21.15.18:56190) 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.106+0000 7fdd8fe00640 1 -- 172.21.15.18:0/2626698331 learned_addr learned my addr 172.21.15.18:0/2626698331 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd8f400640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e484cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd94a00640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e485210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd8fe00640 1 -- 172.21.15.18:0/2626698331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 msgr2=0x56357e485210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e485210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd8fe00640 1 -- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 msgr2=0x56357e484cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e484cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd8fe00640 1 -- 172.21.15.18:0/2626698331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357e1c37d0 con 0x56357e481f10 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd94a00640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e485210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd8f400640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e484cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd8fe00640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e49c120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdd80099680 tx=0x7fdd80009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0cb23072da80b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd79600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd79600640 1 -- 172.21.15.18:0/2626698331 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800d1120 con 0x56357e481f10 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.107+0000 7fdd79600640 1 -- 172.21.15.18:0/2626698331 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fdd80099320 con 0x56357e481f10 2024-04-16T17:05:30.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.108+0000 7fdd99649740 1 -- 172.21.15.18:0/2626698331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd7c034440 con 0x56357e481f10 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.108+0000 7fdd79600640 1 -- 172.21.15.18:0/2626698331 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd8009ccb0 con 0x56357e481f10 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.108+0000 7fdd99649740 10 client.25861 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.108+0000 7fdd99649740 1 -- 172.21.15.18:0/2626698331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56357e50a1c0 con 0x56357e481f10 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.109+0000 7fdd79600640 1 -- 172.21.15.18:0/2626698331 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd7c034440 con 0x56357e481f10 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.109+0000 7fdd79600640 1 -- 172.21.15.18:0/2626698331 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdd800a5490 con 0x56357e481f10 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.109+0000 7fdd99649740 10 client.25861 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.109+0000 7fdd99649740 10 client.25861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.109+0000 7fdd99649740 20 client.25861 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.109+0000 7fdd99649740 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56357e50a8c0 0x56357e50cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.109+0000 7fdd99649740 4 client.25861 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:05:30.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.109+0000 7fdd99649740 1 -- 172.21.15.18:0/2626698331 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56357e3ee8b0 con 0x56357e50a8c0 2024-04-16T17:05:30.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.110+0000 7fdd8f400640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56357e50a8c0 0x56357e50cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.110+0000 7fdd8f400640 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56357e50a8c0 0x56357e50cce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adc60c2313782a8e server_cookie=f3fadf4766c9ed91 in_seq=0 out_seq=0 2024-04-16T17:05:30.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.110+0000 7fdd79600640 10 client.25861 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:05:30.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd79600640 1 -- 172.21.15.18:0/2626698331 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56357e3ee8b0 con 0x56357e50a8c0 2024-04-16T17:05:30.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd79600640 10 client.25861 handle_command_reply: tid=0 2024-04-16T17:05:30.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 client.25861 shutdown 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 -- 172.21.15.18:0/2626698331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 msgr2=0x56357e49c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e49c120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdd80099680 tx=0x7fdd80009f90 comp rx=0 tx=0).stop 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 -- 172.21.15.18:0/2626698331 shutdown_connections 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56357e50a8c0 0x56357e50cce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357e48b410 0x56357e485210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56357e482830 0x56357e484cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 --2- 172.21.15.18:0/2626698331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e481f10 0x56357e49c120 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.111+0000 7fdd99649740 1 -- 172.21.15.18:0/2626698331 >> 172.21.15.18:0/2626698331 conn(0x56357e47c640 msgr2=0x56357e488360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.112+0000 7fdd79600640 0 client.25861 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.112+0000 7fdd99649740 1 -- 172.21.15.18:0/2626698331 shutdown_connections 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.112+0000 7fdd99649740 1 -- 172.21.15.18:0/2626698331 wait complete. 2024-04-16T17:05:30.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.113+0000 7fdd99649740 20 client.25861 trim_cache size 0 max 16384 2024-04-16T17:05:30.115 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:05:30.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 -- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd740a3ed0 msgr2=0x7fdd740a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd740a3ed0 0x7fdd740a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005fd0 tx=0x7fdd88009500 comp rx=0 tx=0).stop 2024-04-16T17:05:30.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 -- 172.21.15.18:0/560003556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd900743b0 msgr2=0x7fdd901a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd900743b0 0x7fdd901a0680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000e9c0 tx=0x7fdd8000ee80 comp rx=0 tx=0).stop 2024-04-16T17:05:30.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 -- 172.21.15.18:0/560003556 shutdown_connections 2024-04-16T17:05:30.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd740a3ed0 0x7fdd740a6390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9010e6c0 0x7fdd9019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90074cd0 0x7fdd901a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 --2- 172.21.15.18:0/560003556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd900743b0 0x7fdd901a0680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.114+0000 7fdd95400640 1 -- 172.21.15.18:0/560003556 >> 172.21.15.18:0/560003556 conn(0x7fdd900fd950 msgr2=0x7fdd900fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.117+0000 7fdd95400640 1 -- 172.21.15.18:0/560003556 shutdown_connections 2024-04-16T17:05:30.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.117+0000 7fdd95400640 1 -- 172.21.15.18:0/560003556 wait complete. 2024-04-16T17:05:30.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:05:30.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 -- 172.21.15.18:0/2914255230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa244102a20 msgr2=0x7fa244106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 --2- 172.21.15.18:0/2914255230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa244102a20 0x7fa244106a60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa230009900 tx=0x7fa23002bc30 comp rx=0 tx=0).stop 2024-04-16T17:05:30.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 -- 172.21.15.18:0/2914255230 shutdown_connections 2024-04-16T17:05:30.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 --2- 172.21.15.18:0/2914255230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa244107140 0x7fa24410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 --2- 172.21.15.18:0/2914255230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa244102a20 0x7fa244106a60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 --2- 172.21.15.18:0/2914255230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa244102070 0x7fa244102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 -- 172.21.15.18:0/2914255230 >> 172.21.15.18:0/2914255230 conn(0x7fa2440fd950 msgr2=0x7fa2440ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 -- 172.21.15.18:0/2914255230 shutdown_connections 2024-04-16T17:05:30.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.388+0000 7fa248a00640 1 -- 172.21.15.18:0/2914255230 wait complete. 2024-04-16T17:05:30.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.389+0000 7fa248a00640 1 Processor -- start 2024-04-16T17:05:30.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.389+0000 7fa248a00640 1 -- start start 2024-04-16T17:05:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa248a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa244102070 0x7fa2441a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa248a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa244107140 0x7fa2441a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa248a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24419a7f0 0x7fa24419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa248a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441121b0 con 0x7fa244102070 2024-04-16T17:05:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa248a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa244112030 con 0x7fa244107140 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa248a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa244112330 con 0x7fa24419a7f0 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa242800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa244102070 0x7fa2441a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa242800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa244102070 0x7fa2441a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56690/0 (socket says 172.21.15.18:56690) 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa242800640 1 -- 172.21.15.18:0/3770755157 learned_addr learned my addr 172.21.15.18:0/3770755157 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa243200640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24419a7f0 0x7fa24419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.390+0000 7fa241e00640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa244107140 0x7fa2441a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.391+0000 7fa242800640 1 -- 172.21.15.18:0/3770755157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24419a7f0 msgr2=0x7fa24419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.391+0000 7fa242800640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24419a7f0 0x7fa24419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.391+0000 7fa242800640 1 -- 172.21.15.18:0/3770755157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa244107140 msgr2=0x7fa2441a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.391+0000 7fa242800640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa244107140 0x7fa2441a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.391+0000 7fa242800640 1 -- 172.21.15.18:0/3770755157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24419b380 con 0x7fa244102070 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.391+0000 7fa243200640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24419a7f0 0x7fa24419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.391+0000 7fa242800640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa244102070 0x7fa2441a0680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa23400ea60 tx=0x7fa23400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.391+0000 7fa23b400640 1 -- 172.21.15.18:0/3770755157 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23400cde0 con 0x7fa244102070 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.392+0000 7fa248a00640 1 -- 172.21.15.18:0/3770755157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24419b670 con 0x7fa244102070 2024-04-16T17:05:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.392+0000 7fa23b400640 1 -- 172.21.15.18:0/3770755157 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2340035f0 con 0x7fa244102070 2024-04-16T17:05:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.392+0000 7fa23b400640 1 -- 172.21.15.18:0/3770755157 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234010670 con 0x7fa244102070 2024-04-16T17:05:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.392+0000 7fa248a00640 1 -- 172.21.15.18:0/3770755157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2441a7870 con 0x7fa244102070 2024-04-16T17:05:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.397+0000 7fa23b400640 1 -- 172.21.15.18:0/3770755157 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa2340037a0 con 0x7fa244102070 2024-04-16T17:05:30.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.397+0000 7fa24c1c4740 1 Processor -- start 2024-04-16T17:05:30.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.397+0000 7fa24c1c4740 1 -- start start 2024-04-16T17:05:30.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 0x55814ac5ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac60800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55814abba020 con 0x55814ac55830 2024-04-16T17:05:30.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55814ab60460 con 0x55814ac54f10 2024-04-16T17:05:30.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55814abb5fe0 con 0x55814ac5e410 2024-04-16T17:05:30.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa241e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac60800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa243200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa241e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac60800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56198/0 (socket says 172.21.15.18:56198) 2024-04-16T17:05:30.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.398+0000 7fa241e00640 1 -- 172.21.15.18:0/904428618 learned_addr learned my addr 172.21.15.18:0/904428618 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa243200640 1 -- 172.21.15.18:0/904428618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 msgr2=0x55814ac60800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa243200640 1 --2- 172.21.15.18:0/904428618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac60800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa243200640 1 -- 172.21.15.18:0/904428618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 msgr2=0x55814ac5ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa243200640 1 --2- 172.21.15.18:0/904428618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 0x55814ac5ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa243200640 1 -- 172.21.15.18:0/904428618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814ab68040 con 0x55814ac54f10 2024-04-16T17:05:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa23b400640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2280a3f50 0x7fa2280a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa243200640 1 --2- 172.21.15.18:0/904428618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac552f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005fd0 tx=0x7fa23c011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9b9e937d22a54a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa23b400640 1 -- 172.21.15.18:0/3770755157 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa2340d10d0 con 0x7fa244102070 2024-04-16T17:05:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa22f400640 1 -- 172.21.15.18:0/904428618 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c004d20 con 0x55814ac54f10 2024-04-16T17:05:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa22f400640 1 -- 172.21.15.18:0/904428618 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23c00f9a0 con 0x55814ac54f10 2024-04-16T17:05:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa22f400640 1 -- 172.21.15.18:0/904428618 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0045b0 con 0x55814ac54f10 2024-04-16T17:05:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.399+0000 7fa242800640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2280a3f50 0x7fa2280a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 -- 172.21.15.18:0/904428618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 msgr2=0x55814ac552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/904428618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac552f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005fd0 tx=0x7fa23c011bf0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa242800640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2280a3f50 0x7fa2280a6410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2340ce290 tx=0x7fa234009580 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 -- 172.21.15.18:0/904428618 shutdown_connections 2024-04-16T17:05:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/904428618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac60800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/904428618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 0x55814ac5ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/904428618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac552f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 -- 172.21.15.18:0/904428618 >> 172.21.15.18:0/904428618 conn(0x55814ac4f640 msgr2=0x55814ac4fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 -- 172.21.15.18:0/904428618 shutdown_connections 2024-04-16T17:05:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.401+0000 7fa24c1c4740 1 -- 172.21.15.18:0/904428618 wait complete. 2024-04-16T17:05:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.402+0000 7fa24c1c4740 1 -- start start 2024-04-16T17:05:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.402+0000 7fa24c1c4740 1 Processor -- start 2024-04-16T17:05:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.402+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac5b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 0x55814ac5ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac5bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55814ac61200 con 0x55814ac55830 2024-04-16T17:05:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55814ab73830 con 0x55814ac54f10 2024-04-16T17:05:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55814ac61380 con 0x55814ac5e410 2024-04-16T17:05:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa24c1c4740 10 client.0 _ll_register_callbacks cb 0x55814ab7b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa243200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac5b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa243200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac5b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44006/0 (socket says 172.21.15.18:44006) 2024-04-16T17:05:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa243200640 1 -- 172.21.15.18:0/1611139752 learned_addr learned my addr 172.21.15.18:0/1611139752 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa243200640 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac5bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa243200640 1 -- 172.21.15.18:0/1611139752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 msgr2=0x55814ac5b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa243200640 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac5b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa243200640 1 -- 172.21.15.18:0/1611139752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 msgr2=0x55814ac5ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.403+0000 7fa241e00640 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 0x55814ac5ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.404+0000 7fa243200640 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 0x55814ac5ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.404+0000 7fa243200640 1 -- 172.21.15.18:0/1611139752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814ab77920 con 0x55814ac5e410 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.404+0000 7fa241e00640 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 0x55814ac5ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.404+0000 7fa243200640 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac5bfd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa23c003f50 tx=0x7fa23c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=101ca7865202928f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.404+0000 7fa223e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.404+0000 7fa223e00640 1 -- 172.21.15.18:0/1611139752 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c004770 con 0x55814ac5e410 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.404+0000 7fa223e00640 1 -- 172.21.15.18:0/1611139752 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa23c004910 con 0x55814ac5e410 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.405+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1611139752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa23c00f9a0 con 0x55814ac5e410 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.405+0000 7fa223e00640 1 -- 172.21.15.18:0/1611139752 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa23c004b50 con 0x55814ac5e410 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.405+0000 7fa24c1c4740 10 client.25873 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.405+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1611139752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55814ab457f0 con 0x55814ac5e410 2024-04-16T17:05:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa223e00640 1 -- 172.21.15.18:0/1611139752 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23c00f9a0 con 0x55814ac5e410 2024-04-16T17:05:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa223e00640 1 -- 172.21.15.18:0/1611139752 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa23c018070 con 0x55814ac5e410 2024-04-16T17:05:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa24c1c4740 10 client.25873 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa24c1c4740 10 client.25873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa24c1c4740 20 client.25873 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55814ac77000 0x55814ac793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa24c1c4740 4 client.25873 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1611139752 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55814abc7a40 con 0x55814ac77000 2024-04-16T17:05:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.406+0000 7fa241e00640 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55814ac77000 0x55814ac793f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.407+0000 7fa241e00640 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55814ac77000 0x55814ac793f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6592bb4f724897c server_cookie=3c20a71e96f5ecf0 in_seq=0 out_seq=0 2024-04-16T17:05:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.407+0000 7fa223e00640 10 client.25873 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:30.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.408+0000 7fa223e00640 1 -- 172.21.15.18:0/1611139752 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55814abc7a40 con 0x55814ac77000 2024-04-16T17:05:30.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.408+0000 7fa223e00640 10 client.25873 handle_command_reply: tid=0 2024-04-16T17:05:30.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.408+0000 7fa24c1c4740 1 client.25873 shutdown 2024-04-16T17:05:30.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.408+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1611139752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 msgr2=0x55814ac5bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.408+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac5bfd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa23c003f50 tx=0x7fa23c000f40 comp rx=0 tx=0).stop 2024-04-16T17:05:30.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1611139752 shutdown_connections 2024-04-16T17:05:30.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55814ac77000 0x55814ac793f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac5e410 0x55814ac5bfd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac55830 0x55814ac5ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1611139752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac5b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1611139752 >> 172.21.15.18:0/1611139752 conn(0x55814ac4f640 msgr2=0x55814ac51390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa223e00640 0 client.25873 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:30.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1611139752 shutdown_connections 2024-04-16T17:05:30.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1611139752 wait complete. 2024-04-16T17:05:30.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.409+0000 7fa24c1c4740 20 client.25873 trim_cache size 0 max 16384 2024-04-16T17:05:30.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.437+0000 7fa24c1c4740 1 Processor -- start 2024-04-16T17:05:30.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.437+0000 7fa24c1c4740 1 -- start start 2024-04-16T17:05:30.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.437+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac6a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.437+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac55830 0x55814ac5a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.437+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac5bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.437+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441afb50 con 0x55814ac5e410 2024-04-16T17:05:30.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.437+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441ad420 con 0x55814ac54f10 2024-04-16T17:05:30.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.437+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441ab2b0 con 0x55814ac55830 2024-04-16T17:05:30.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa241e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac55830 0x55814ac5a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa241e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac55830 0x55814ac5a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56220/0 (socket says 172.21.15.18:56220) 2024-04-16T17:05:30.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa241e00640 1 -- 172.21.15.18:0/1927426345 learned_addr learned my addr 172.21.15.18:0/1927426345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa243200640 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac6a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa243200640 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac5bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa241e00640 1 -- 172.21.15.18:0/1927426345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 msgr2=0x55814ac6a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa241e00640 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac6a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa241e00640 1 -- 172.21.15.18:0/1927426345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 msgr2=0x55814ac5bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa241e00640 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac5bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa241e00640 1 -- 172.21.15.18:0/1927426345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814a8cc3f0 con 0x55814ac55830 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.438+0000 7fa243200640 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac5bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.439+0000 7fa241e00640 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac55830 0x55814ac5a6b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa230036040 tx=0x7fa230005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46878d614088cffb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.439+0000 7fa22ea00640 1 -- 172.21.15.18:0/1927426345 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230047bc0 con 0x55814ac55830 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.439+0000 7fa22ea00640 1 -- 172.21.15.18:0/1927426345 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa230047d60 con 0x55814ac55830 2024-04-16T17:05:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.439+0000 7fa22ea00640 1 -- 172.21.15.18:0/1927426345 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2300477d0 con 0x55814ac55830 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1927426345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac55830 msgr2=0x55814ac5a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac55830 0x55814ac5a6b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa230036040 tx=0x7fa230005fd0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1927426345 shutdown_connections 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac5bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac55830 0x55814ac5a6b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/1927426345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac54f10 0x55814ac6a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1927426345 >> 172.21.15.18:0/1927426345 conn(0x55814ac4f640 msgr2=0x55814ac4de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1927426345 shutdown_connections 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.440+0000 7fa24c1c4740 1 -- 172.21.15.18:0/1927426345 wait complete. 2024-04-16T17:05:30.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.441+0000 7fa24c1c4740 1 -- start start 2024-04-16T17:05:30.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.441+0000 7fa24c1c4740 1 Processor -- start 2024-04-16T17:05:30.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac54f10 0x55814ac61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac55830 0x55814ac61ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa24c1c4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac62010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441ab900 con 0x55814ac5e410 2024-04-16T17:05:30.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441acd50 con 0x55814ac55830 2024-04-16T17:05:30.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa24c1c4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441abfb0 con 0x55814ac54f10 2024-04-16T17:05:30.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa24c1c4740 10 client.0 _ll_register_callbacks cb 0x7fa23c011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa243200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac54f10 0x55814ac61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa243200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac54f10 0x55814ac61590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56234/0 (socket says 172.21.15.18:56234) 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa243200640 1 -- 172.21.15.18:0/2382121428 learned_addr learned my addr 172.21.15.18:0/2382121428 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa241e00640 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac55830 0x55814ac61ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.442+0000 7fa243200640 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac62010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa243200640 1 -- 172.21.15.18:0/2382121428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac55830 msgr2=0x55814ac61ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa243200640 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac55830 0x55814ac61ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa243200640 1 -- 172.21.15.18:0/2382121428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 msgr2=0x55814ac62010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa243200640 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac62010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa243200640 1 -- 172.21.15.18:0/2382121428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814a9967d0 con 0x55814ac54f10 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa243200640 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac54f10 0x55814ac61590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005ba0 tx=0x7fa23c0040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1bcf8fa140aa9c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:30.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa223e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa223e00640 1 -- 172.21.15.18:0/2382121428 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c004980 con 0x55814ac54f10 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa24c1c4740 1 -- 172.21.15.18:0/2382121428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa230047d60 con 0x55814ac54f10 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.443+0000 7fa223e00640 1 -- 172.21.15.18:0/2382121428 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa23c027530 con 0x55814ac54f10 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.444+0000 7fa223e00640 1 -- 172.21.15.18:0/2382121428 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa23c027730 con 0x55814ac54f10 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.444+0000 7fa24c1c4740 10 client.25879 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.444+0000 7fa24c1c4740 1 -- 172.21.15.18:0/2382121428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55814acdc4a0 con 0x55814ac54f10 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.444+0000 7fa223e00640 1 -- 172.21.15.18:0/2382121428 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa230047d60 con 0x55814ac54f10 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.445+0000 7fa223e00640 1 -- 172.21.15.18:0/2382121428 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa23c018040 con 0x55814ac54f10 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.445+0000 7fa24c1c4740 10 client.25879 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.445+0000 7fa24c1c4740 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.445+0000 7fa24c1c4740 20 client.25879 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.445+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55814acdcb10 0x55814acdef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.445+0000 7fa24c1c4740 4 client.25879 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:05:30.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.445+0000 7fa24c1c4740 1 -- 172.21.15.18:0/2382121428 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55814abc18b0 con 0x55814acdcb10 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.445+0000 7fa241e00640 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55814acdcb10 0x55814acdef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.446+0000 7fa241e00640 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55814acdcb10 0x55814acdef00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e70a3c5706579f server_cookie=fd3571b58aa694d4 in_seq=0 out_seq=0 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.446+0000 7fa223e00640 10 client.25879 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.446+0000 7fa223e00640 1 -- 172.21.15.18:0/2382121428 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55814abc18b0 con 0x55814acdcb10 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.446+0000 7fa223e00640 10 client.25879 handle_command_reply: tid=0 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.446+0000 7fa24c1c4740 1 client.25879 shutdown 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 -- 172.21.15.18:0/2382121428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac54f10 msgr2=0x55814ac61590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac54f10 0x55814ac61590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005ba0 tx=0x7fa23c0040f0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 -- 172.21.15.18:0/2382121428 shutdown_connections 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55814acdcb10 0x55814acdef00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55814ac5e410 0x55814ac62010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55814ac55830 0x55814ac61ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa223e00640 0 client.25879 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:30.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 --2- 172.21.15.18:0/2382121428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55814ac54f10 0x55814ac61590 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 -- 172.21.15.18:0/2382121428 >> 172.21.15.18:0/2382121428 conn(0x55814ac4f640 msgr2=0x55814ac50940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 -- 172.21.15.18:0/2382121428 shutdown_connections 2024-04-16T17:05:30.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 1 -- 172.21.15.18:0/2382121428 wait complete. 2024-04-16T17:05:30.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.447+0000 7fa24c1c4740 20 client.25879 trim_cache size 0 max 16384 2024-04-16T17:05:30.450 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 -- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2280a3f50 msgr2=0x7fa2280a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2280a3f50 0x7fa2280a6410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2340ce290 tx=0x7fa234009580 comp rx=0 tx=0).stop 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 -- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa244102070 msgr2=0x7fa2441a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa244102070 0x7fa2441a0680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa23400ea60 tx=0x7fa23400ef20 comp rx=0 tx=0).stop 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 -- 172.21.15.18:0/3770755157 shutdown_connections 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2280a3f50 0x7fa2280a6410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24419a7f0 0x7fa24419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa244107140 0x7fa2441a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 --2- 172.21.15.18:0/3770755157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa244102070 0x7fa2441a0680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.448+0000 7fa248a00640 1 -- 172.21.15.18:0/3770755157 >> 172.21.15.18:0/3770755157 conn(0x7fa2440fd950 msgr2=0x7fa2440ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:30.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.449+0000 7fa248a00640 1 -- 172.21.15.18:0/3770755157 shutdown_connections 2024-04-16T17:05:30.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:30.449+0000 7fa248a00640 1 -- 172.21.15.18:0/3770755157 wait complete. 2024-04-16T17:05:31.474 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:05:31.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:05:31.475 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:05:31.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.680+0000 7f40bbe00640 1 -- 172.21.15.18:0/4027529570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 msgr2=0x7f40bc10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.680+0000 7f40bbe00640 1 --2- 172.21.15.18:0/4027529570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 0x7f40bc10bde0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009900 tx=0x7f40a802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.680+0000 7f40bbe00640 1 -- 172.21.15.18:0/4027529570 shutdown_connections 2024-04-16T17:05:31.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.680+0000 7f40bbe00640 1 --2- 172.21.15.18:0/4027529570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40bc10c400 0x7f40bc10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.680+0000 7f40bbe00640 1 --2- 172.21.15.18:0/4027529570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 0x7f40bc10bde0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.680+0000 7f40bbe00640 1 --2- 172.21.15.18:0/4027529570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc102fe0 0x7f40bc1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.680+0000 7f40bbe00640 1 -- 172.21.15.18:0/4027529570 >> 172.21.15.18:0/4027529570 conn(0x7f40bc077ab0 msgr2=0x7f40bc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.681+0000 7f40bbe00640 1 -- 172.21.15.18:0/4027529570 shutdown_connections 2024-04-16T17:05:31.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.681+0000 7f40bbe00640 1 -- 172.21.15.18:0/4027529570 wait complete. 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.681+0000 7f40bbe00640 1 Processor -- start 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40bbe00640 1 -- start start 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40bbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40bc102fe0 0x7f40bc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 0x7f40bc19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc10c400 0x7f40bc1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40bbe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc110030 con 0x7f40bc102fe0 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40bbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc10feb0 con 0x7f40bc103900 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40bbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1101b0 con 0x7f40bc10c400 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40ba000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 0x7f40bc19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40baa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40bc102fe0 0x7f40bc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40ba000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 0x7f40bc19f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44046/0 (socket says 172.21.15.18:44046) 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40ba000640 1 -- 172.21.15.18:0/2936581547 learned_addr learned my addr 172.21.15.18:0/2936581547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.682+0000 7f40bb400640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc10c400 0x7f40bc1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.683+0000 7f40ba000640 1 -- 172.21.15.18:0/2936581547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc10c400 msgr2=0x7f40bc1a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.683+0000 7f40ba000640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc10c400 0x7f40bc1a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.683+0000 7f40ba000640 1 -- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40bc102fe0 msgr2=0x7f40bc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.683+0000 7f40ba000640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40bc102fe0 0x7f40bc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.683+0000 7f40ba000640 1 -- 172.21.15.18:0/2936581547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc1a3d40 con 0x7f40bc103900 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.683+0000 7f40baa00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40bc102fe0 0x7f40bc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.683+0000 7f40bb400640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc10c400 0x7f40bc1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.683+0000 7f40ba000640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 0x7f40bc19f2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f40a802c2d0 tx=0x7f40a80043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.684+0000 7f40b3400640 1 -- 172.21.15.18:0/2936581547 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8041020 con 0x7f40bc103900 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.684+0000 7f40b3400640 1 -- 172.21.15.18:0/2936581547 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a8005000 con 0x7f40bc103900 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.684+0000 7f40b3400640 1 -- 172.21.15.18:0/2936581547 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003b90 con 0x7f40bc103900 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.684+0000 7f40bbe00640 1 -- 172.21.15.18:0/2936581547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc1a3fd0 con 0x7f40bc103900 2024-04-16T17:05:31.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.684+0000 7f40bbe00640 1 -- 172.21.15.18:0/2936581547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc1a4470 con 0x7f40bc103900 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.687+0000 7f40c43d0740 1 Processor -- start 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.687+0000 7f40c43d0740 1 -- start start 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.687+0000 7f40b3400640 1 -- 172.21.15.18:0/2936581547 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f40a8044020 con 0x7f40bc103900 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d4e90 0x55b7c90d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 0x55b7c90dde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90de390 0x55b7c90e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40c43d0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9039cb0 con 0x55b7c90d4e90 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40c43d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b7c8fe0370 con 0x55b7c90de390 2024-04-16T17:05:31.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40c43d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9035c70 con 0x55b7c90d57b0 2024-04-16T17:05:31.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40bb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d4e90 0x55b7c90d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40bb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d4e90 0x55b7c90d5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56744/0 (socket says 172.21.15.18:56744) 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40bb400640 1 -- 172.21.15.18:0/1388560698 learned_addr learned my addr 172.21.15.18:0/1388560698 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40baa00640 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 0x55b7c90dde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.688+0000 7f40ba000640 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90de390 0x55b7c90e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40bb400640 1 -- 172.21.15.18:0/1388560698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 msgr2=0x55b7c90dde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40bb400640 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 0x55b7c90dde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40bb400640 1 -- 172.21.15.18:0/1388560698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90de390 msgr2=0x55b7c90e0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40bb400640 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90de390 0x55b7c90e0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40bb400640 1 -- 172.21.15.18:0/1388560698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c8fff910 con 0x55b7c90d4e90 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40ba000640 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90de390 0x55b7c90e0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40b3400640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40a00a3fc0 0x7f40a00a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40bb400640 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d4e90 0x55b7c90d5270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009fd0 tx=0x7f40b4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4dbf1eab49a0e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.689+0000 7f40b3400640 1 -- 172.21.15.18:0/2936581547 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f40a80f2720 con 0x7f40bc103900 2024-04-16T17:05:31.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.690+0000 7f40baa00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40a00a3fc0 0x7f40a00a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.690+0000 7f40a7400640 1 -- 172.21.15.18:0/1388560698 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4011700 con 0x55b7c90d4e90 2024-04-16T17:05:31.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.690+0000 7f40a7400640 1 -- 172.21.15.18:0/1388560698 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b4011d80 con 0x55b7c90d4e90 2024-04-16T17:05:31.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.690+0000 7f40a7400640 1 -- 172.21.15.18:0/1388560698 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b400e3c0 con 0x55b7c90d4e90 2024-04-16T17:05:31.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 -- 172.21.15.18:0/1388560698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d4e90 msgr2=0x55b7c90d5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d4e90 0x55b7c90d5270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009fd0 tx=0x7f40b4010860 comp rx=0 tx=0).stop 2024-04-16T17:05:31.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40baa00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40a00a3fc0 0x7f40a00a6480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b7c90d4960 tx=0x7f40ac00b390 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 -- 172.21.15.18:0/1388560698 shutdown_connections 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90de390 0x55b7c90e0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 0x55b7c90dde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 --2- 172.21.15.18:0/1388560698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d4e90 0x55b7c90d5270 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 -- 172.21.15.18:0/1388560698 >> 172.21.15.18:0/1388560698 conn(0x55b7c90cf5c0 msgr2=0x55b7c90cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 -- 172.21.15.18:0/1388560698 shutdown_connections 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.691+0000 7f40c43d0740 1 -- 172.21.15.18:0/1388560698 wait complete. 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 1 -- start start 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 1 Processor -- start 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 0x55b7c90db9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90dbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b7c90e1180 con 0x55b7c90de390 2024-04-16T17:05:31.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b7c8ff3730 con 0x55b7c90d4e90 2024-04-16T17:05:31.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b7c90e1300 con 0x55b7c90d57b0 2024-04-16T17:05:31.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.692+0000 7f40c43d0740 10 client.0 _ll_register_callbacks cb 0x55b7c8fc21e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:31.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90db490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44078/0 (socket says 172.21.15.18:44078) 2024-04-16T17:05:31.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 -- 172.21.15.18:0/141773215 learned_addr learned my addr 172.21.15.18:0/141773215 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40bb400640 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90dbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40ba000640 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 0x55b7c90db9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 -- 172.21.15.18:0/141773215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 msgr2=0x55b7c90db9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 0x55b7c90db9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 -- 172.21.15.18:0/141773215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 msgr2=0x55b7c90dbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90dbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 -- 172.21.15.18:0/141773215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c8e147e0 con 0x55b7c90d4e90 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40bb400640 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90dbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f40baa00640 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90db490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40ac010850 tx=0x7f40ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cead78bc02f07ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.693+0000 7f409be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.694+0000 7f409be00640 1 -- 172.21.15.18:0/141773215 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac02b060 con 0x55b7c90d4e90 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.694+0000 7f409be00640 1 -- 172.21.15.18:0/141773215 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f40ac01c840 con 0x55b7c90d4e90 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.694+0000 7f40c43d0740 1 -- 172.21.15.18:0/141773215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b4011d80 con 0x55b7c90d4e90 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.694+0000 7f409be00640 1 -- 172.21.15.18:0/141773215 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40ac032050 con 0x55b7c90d4e90 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.694+0000 7f40c43d0740 10 client.25898 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.694+0000 7f40c43d0740 1 -- 172.21.15.18:0/141773215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7c8e74f70 con 0x55b7c90d4e90 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.695+0000 7f409be00640 1 -- 172.21.15.18:0/141773215 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b4011d80 con 0x55b7c90d4e90 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.695+0000 7f409be00640 1 -- 172.21.15.18:0/141773215 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40ac025030 con 0x55b7c90d4e90 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.695+0000 7f40c43d0740 10 client.25898 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.695+0000 7f40c43d0740 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.695+0000 7f40c43d0740 20 client.25898 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.695+0000 7f40c43d0740 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b7c90ee850 0x55b7c90f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.695+0000 7f40c43d0740 4 client.25898 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.695+0000 7f40c43d0740 1 -- 172.21.15.18:0/141773215 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c90476d0 con 0x55b7c90ee850 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.696+0000 7f40ba000640 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b7c90ee850 0x55b7c90f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.696+0000 7f40ba000640 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b7c90ee850 0x55b7c90f0cb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b3acc9655e85b9 server_cookie=b6b77920b407af27 in_seq=0 out_seq=0 2024-04-16T17:05:31.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.696+0000 7f409be00640 10 client.25898 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.697+0000 7f409be00640 1 -- 172.21.15.18:0/141773215 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7c90476d0 con 0x55b7c90ee850 2024-04-16T17:05:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.697+0000 7f409be00640 10 client.25898 handle_command_reply: tid=0 2024-04-16T17:05:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.697+0000 7f40c43d0740 1 client.25898 shutdown 2024-04-16T17:05:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.697+0000 7f40c43d0740 1 -- 172.21.15.18:0/141773215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 msgr2=0x55b7c90db490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.697+0000 7f40c43d0740 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90db490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40ac010850 tx=0x7f40ac0046c0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 1 -- 172.21.15.18:0/141773215 shutdown_connections 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b7c90ee850 0x55b7c90f0cb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90dbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d57b0 0x55b7c90db9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 1 --2- 172.21.15.18:0/141773215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90db490 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 1 -- 172.21.15.18:0/141773215 >> 172.21.15.18:0/141773215 conn(0x55b7c90cf5c0 msgr2=0x55b7c90d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f409be00640 0 client.25898 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 1 -- 172.21.15.18:0/141773215 shutdown_connections 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 1 -- 172.21.15.18:0/141773215 wait complete. 2024-04-16T17:05:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.698+0000 7f40c43d0740 20 client.25898 trim_cache size 0 max 16384 2024-04-16T17:05:31.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.716+0000 7f40c43d0740 1 Processor -- start 2024-04-16T17:05:31.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40c43d0740 1 -- start start 2024-04-16T17:05:31.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d4e90 0x55b7c90ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d57b0 0x55b7c90ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40c43d0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1c0ad0 con 0x55b7c90de390 2024-04-16T17:05:31.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40c43d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1be3a0 con 0x55b7c90d57b0 2024-04-16T17:05:31.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40c43d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc19e3b0 con 0x55b7c90d4e90 2024-04-16T17:05:31.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d4e90 0x55b7c90ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40bb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d4e90 0x55b7c90ddbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56268/0 (socket says 172.21.15.18:56268) 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.717+0000 7f40baa00640 1 -- 172.21.15.18:0/617380244 learned_addr learned my addr 172.21.15.18:0/617380244 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40ba000640 1 --2- 172.21.15.18:0/617380244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d57b0 0x55b7c90ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40baa00640 1 -- 172.21.15.18:0/617380244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d57b0 msgr2=0x55b7c90ea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40baa00640 1 --2- 172.21.15.18:0/617380244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d57b0 0x55b7c90ea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40baa00640 1 -- 172.21.15.18:0/617380244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 msgr2=0x55b7c90ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40baa00640 1 --2- 172.21.15.18:0/617380244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40baa00640 1 -- 172.21.15.18:0/617380244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c8d0c170 con 0x55b7c90d4e90 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40baa00640 1 --2- 172.21.15.18:0/617380244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d4e90 0x55b7c90ddbc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f40ac017040 tx=0x7f40ac01df00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=104ff81ca314a01f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40a6a00640 1 -- 172.21.15.18:0/617380244 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac010ea0 con 0x55b7c90d4e90 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40a6a00640 1 -- 172.21.15.18:0/617380244 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ac00be50 con 0x55b7c90d4e90 2024-04-16T17:05:31.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.718+0000 7f40a6a00640 1 -- 172.21.15.18:0/617380244 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac026d70 con 0x55b7c90d4e90 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 -- 172.21.15.18:0/617380244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d4e90 msgr2=0x55b7c90ddbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 --2- 172.21.15.18:0/617380244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d4e90 0x55b7c90ddbc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f40ac017040 tx=0x7f40ac01df00 comp rx=0 tx=0).stop 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 -- 172.21.15.18:0/617380244 shutdown_connections 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 --2- 172.21.15.18:0/617380244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90de390 0x55b7c90ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 --2- 172.21.15.18:0/617380244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d57b0 0x55b7c90ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 --2- 172.21.15.18:0/617380244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90d4e90 0x55b7c90ddbc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 -- 172.21.15.18:0/617380244 >> 172.21.15.18:0/617380244 conn(0x55b7c90cf5c0 msgr2=0x55b7c90cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 -- 172.21.15.18:0/617380244 shutdown_connections 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.719+0000 7f40c43d0740 1 -- 172.21.15.18:0/617380244 wait complete. 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.720+0000 7f40c43d0740 1 -- start start 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.720+0000 7f40c43d0740 1 Processor -- start 2024-04-16T17:05:31.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.720+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90d88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.720+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d57b0 0x55b7c90d8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.720+0000 7f40c43d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90de390 0x55b7c9153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.720+0000 7f40c43d0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1bc900 con 0x55b7c90d57b0 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.720+0000 7f40c43d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1bdcd0 con 0x55b7c90d4e90 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.720+0000 7f40c43d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1bcf30 con 0x55b7c90de390 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40c43d0740 10 client.0 _ll_register_callbacks cb 0x7f40ac010710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40baa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90d88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40baa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90d88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44092/0 (socket says 172.21.15.18:44092) 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40baa00640 1 -- 172.21.15.18:0/776528028 learned_addr learned my addr 172.21.15.18:0/776528028 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40bb400640 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90de390 0x55b7c9153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40ba000640 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d57b0 0x55b7c90d8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40baa00640 1 -- 172.21.15.18:0/776528028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90de390 msgr2=0x55b7c9153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40baa00640 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90de390 0x55b7c9153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40baa00640 1 -- 172.21.15.18:0/776528028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d57b0 msgr2=0x55b7c90d8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40baa00640 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d57b0 0x55b7c90d8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40baa00640 1 -- 172.21.15.18:0/776528028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c9154330 con 0x55b7c90d4e90 2024-04-16T17:05:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.721+0000 7f40ba000640 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d57b0 0x55b7c90d8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.722+0000 7f40baa00640 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90d88b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40ac004770 tx=0x7f40ac026710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa151cc997d9830d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.722+0000 7f409be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:31.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.722+0000 7f409be00640 1 -- 172.21.15.18:0/776528028 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac01d410 con 0x55b7c90d4e90 2024-04-16T17:05:31.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.722+0000 7f40c43d0740 1 -- 172.21.15.18:0/776528028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40ac00be50 con 0x55b7c90d4e90 2024-04-16T17:05:31.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.722+0000 7f409be00640 1 -- 172.21.15.18:0/776528028 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f40ac01d5b0 con 0x55b7c90d4e90 2024-04-16T17:05:31.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.722+0000 7f409be00640 1 -- 172.21.15.18:0/776528028 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40ac04e050 con 0x55b7c90d4e90 2024-04-16T17:05:31.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.722+0000 7f40c43d0740 10 client.25904 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:31.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.722+0000 7f40c43d0740 1 -- 172.21.15.18:0/776528028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7c9155620 con 0x55b7c90d4e90 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f409be00640 1 -- 172.21.15.18:0/776528028 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ac00be50 con 0x55b7c90d4e90 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f409be00640 1 -- 172.21.15.18:0/776528028 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40ac025040 con 0x55b7c90d4e90 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f40c43d0740 10 client.25904 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f40c43d0740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f40c43d0740 20 client.25904 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f40c43d0740 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b7c9155bd0 0x55b7c9157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f40c43d0740 4 client.25904 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f40c43d0740 1 -- 172.21.15.18:0/776528028 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b7c9041540 con 0x55b7c9155bd0 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.723+0000 7f40ba000640 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b7c9155bd0 0x55b7c9157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.724+0000 7f40ba000640 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b7c9155bd0 0x55b7c9157fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c027bd85e7d82 server_cookie=2a6d584437c6fe79 in_seq=0 out_seq=0 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.724+0000 7f409be00640 10 client.25904 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f409be00640 1 -- 172.21.15.18:0/776528028 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b7c9041540 con 0x55b7c9155bd0 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f409be00640 10 client.25904 handle_command_reply: tid=0 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 client.25904 shutdown 2024-04-16T17:05:31.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 -- 172.21.15.18:0/776528028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 msgr2=0x55b7c90d88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90d88b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40ac004770 tx=0x7f40ac026710 comp rx=0 tx=0).stop 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 -- 172.21.15.18:0/776528028 shutdown_connections 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b7c9155bd0 0x55b7c9157fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7c90de390 0x55b7c9153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b7c90d57b0 0x55b7c90d8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 --2- 172.21.15.18:0/776528028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7c90d4e90 0x55b7c90d88b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f40c43d0740 1 -- 172.21.15.18:0/776528028 >> 172.21.15.18:0/776528028 conn(0x55b7c90cf5c0 msgr2=0x55b7c90ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.725+0000 7f409be00640 0 client.25904 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.726+0000 7f40c43d0740 1 -- 172.21.15.18:0/776528028 shutdown_connections 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.726+0000 7f40c43d0740 1 -- 172.21.15.18:0/776528028 wait complete. 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.726+0000 7f40c43d0740 20 client.25904 trim_cache size 0 max 16384 2024-04-16T17:05:31.729 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:05:31.730 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:05:31.730 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "4279e0e9-c11f-4f6b-9098-67f0e911d3b1", 2024-04-16T17:05:31.730 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:05:31.730 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:05:31.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.727+0000 7f40bbe00640 1 -- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40a00a3fc0 msgr2=0x7f40a00a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.727+0000 7f40bbe00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40a00a3fc0 0x7f40a00a6480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b7c90d4960 tx=0x7f40ac00b390 comp rx=0 tx=0).stop 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.727+0000 7f40bbe00640 1 -- 172.21.15.18:0/2936581547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 msgr2=0x7f40bc19f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.727+0000 7f40bbe00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 0x7f40bc19f2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f40a802c2d0 tx=0x7f40a80043c0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.728+0000 7f40bbe00640 1 -- 172.21.15.18:0/2936581547 shutdown_connections 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.728+0000 7f40bbe00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40a00a3fc0 0x7f40a00a6480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.728+0000 7f40bbe00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc10c400 0x7f40bc1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.728+0000 7f40bbe00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40bc103900 0x7f40bc19f2c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.728+0000 7f40bbe00640 1 --2- 172.21.15.18:0/2936581547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40bc102fe0 0x7f40bc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.728+0000 7f40bbe00640 1 -- 172.21.15.18:0/2936581547 >> 172.21.15.18:0/2936581547 conn(0x7f40bc077ab0 msgr2=0x7f40bc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.728+0000 7f40bbe00640 1 -- 172.21.15.18:0/2936581547 shutdown_connections 2024-04-16T17:05:31.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.728+0000 7f40bbe00640 1 -- 172.21.15.18:0/2936581547 wait complete. 2024-04-16T17:05:31.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:05:31.754 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:05:31.816 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:31 smithi018 ceph-mon[31776]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:05:31.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:31 smithi022 ceph-mon[36023]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:05:31.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 -- 172.21.15.18:0/3182683027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c4074cd0 msgr2=0x7f17c410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 --2- 172.21.15.18:0/3182683027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c4074cd0 0x7f17c410e180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009900 tx=0x7f17ac02bce0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 -- 172.21.15.18:0/3182683027 shutdown_connections 2024-04-16T17:05:31.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 --2- 172.21.15.18:0/3182683027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c410e6c0 0x7f17c4110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 --2- 172.21.15.18:0/3182683027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c4074cd0 0x7f17c410e180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 --2- 172.21.15.18:0/3182683027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c40743b0 0x7f17c4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 -- 172.21.15.18:0/3182683027 >> 172.21.15.18:0/3182683027 conn(0x7f17c40fd950 msgr2=0x7f17c40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 -- 172.21.15.18:0/3182683027 shutdown_connections 2024-04-16T17:05:31.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.943+0000 7f17c3e00640 1 -- 172.21.15.18:0/3182683027 wait complete. 2024-04-16T17:05:31.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.945+0000 7f17c3e00640 1 Processor -- start 2024-04-16T17:05:31.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.945+0000 7f17c3e00640 1 -- start start 2024-04-16T17:05:31.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.945+0000 7f17c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c40743b0 0x7f17c41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.945+0000 7f17c3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c4074cd0 0x7f17c41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.945+0000 7f17c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c410e6c0 0x7f17c419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.945+0000 7f17c3e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4114400 con 0x7f17c410e6c0 2024-04-16T17:05:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.945+0000 7f17c3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4114280 con 0x7f17c4074cd0 2024-04-16T17:05:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.945+0000 7f17c3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4114580 con 0x7f17c40743b0 2024-04-16T17:05:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.946+0000 7f17c2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c40743b0 0x7f17c41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.946+0000 7f17c2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c40743b0 0x7f17c41a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56292/0 (socket says 172.21.15.18:56292) 2024-04-16T17:05:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.946+0000 7f17c2a00640 1 -- 172.21.15.18:0/706768190 learned_addr learned my addr 172.21.15.18:0/706768190 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.946+0000 7f17c2a00640 1 -- 172.21.15.18:0/706768190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c4074cd0 msgr2=0x7f17c41a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.946+0000 7f17c2a00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c4074cd0 0x7f17c41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.946+0000 7f17c2a00640 1 -- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c410e6c0 msgr2=0x7f17c419a740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:05:31.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.947+0000 7f17c3400640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c410e6c0 0x7f17c419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.947+0000 7f17c2a00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c410e6c0 0x7f17c419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.947+0000 7f17c2a00640 1 -- 172.21.15.18:0/706768190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c419b000 con 0x7f17c40743b0 2024-04-16T17:05:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.947+0000 7f17c3400640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c410e6c0 0x7f17c419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.947+0000 7f17c2a00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c40743b0 0x7f17c41a0680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17b400d950 tx=0x7f17b400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.947+0000 7f17bb400640 1 -- 172.21.15.18:0/706768190 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4017070 con 0x7f17c40743b0 2024-04-16T17:05:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.948+0000 7f17bb400640 1 -- 172.21.15.18:0/706768190 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b4009c30 con 0x7f17c40743b0 2024-04-16T17:05:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.948+0000 7f17c3e00640 1 -- 172.21.15.18:0/706768190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c419b2f0 con 0x7f17c40743b0 2024-04-16T17:05:31.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.948+0000 7f17bb400640 1 -- 172.21.15.18:0/706768190 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b400f6b0 con 0x7f17c40743b0 2024-04-16T17:05:31.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.948+0000 7f17c3e00640 1 -- 172.21.15.18:0/706768190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41a7600 con 0x7f17c40743b0 2024-04-16T17:05:31.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.952+0000 7f17cc852740 1 Processor -- start 2024-04-16T17:05:31.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.953+0000 7f17cc852740 1 -- start start 2024-04-16T17:05:31.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.953+0000 7f17bb400640 1 -- 172.21.15.18:0/706768190 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f17b40109d0 con 0x7f17c40743b0 2024-04-16T17:05:31.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.953+0000 7f17cc852740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e191519580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17cc852740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17cc852740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 0x55e19152d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17cc852740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e19147f120 con 0x55e19152b0a0 2024-04-16T17:05:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17cc852740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e191425340 con 0x55e191519b50 2024-04-16T17:05:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17cc852740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e19147b0e0 con 0x55e1915191a0 2024-04-16T17:05:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17c3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e191519580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17c3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e191519580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56308/0 (socket says 172.21.15.18:56308) 2024-04-16T17:05:31.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17c2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17c3400640 1 -- 172.21.15.18:0/2637016213 learned_addr learned my addr 172.21.15.18:0/2637016213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.954+0000 7f17c2a00640 1 -- 172.21.15.18:0/2637016213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 msgr2=0x55e191519580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.955+0000 7f17c2a00640 1 --2- 172.21.15.18:0/2637016213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e191519580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.955+0000 7f17c2a00640 1 -- 172.21.15.18:0/2637016213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 msgr2=0x55e19152d490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:05:31.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.955+0000 7f17c2a00640 1 --2- 172.21.15.18:0/2637016213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 0x55e19152d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.955+0000 7f17c2a00640 1 -- 172.21.15.18:0/2637016213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e19125b7d0 con 0x55e191519b50 2024-04-16T17:05:31.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.955+0000 7f17c3400640 1 --2- 172.21.15.18:0/2637016213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e191519580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:31.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.955+0000 7f17c2a00640 1 --2- 172.21.15.18:0/2637016213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152ab60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17b40ce070 tx=0x7f17b40d9220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a49142cde8b433 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.955+0000 7f17bb400640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f17a80a3f50 0x7f17a80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.956+0000 7f17b3e00640 1 -- 172.21.15.18:0/2637016213 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b40e4070 con 0x55e191519b50 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.956+0000 7f17b3e00640 1 -- 172.21.15.18:0/2637016213 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b409cf10 con 0x55e191519b50 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.956+0000 7f17b3e00640 1 -- 172.21.15.18:0/2637016213 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b407c810 con 0x55e191519b50 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.956+0000 7f17bb400640 1 -- 172.21.15.18:0/706768190 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f17b40d0f40 con 0x7f17c40743b0 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.956+0000 7f17c3400640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f17a80a3f50 0x7f17a80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 -- 172.21.15.18:0/2637016213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 msgr2=0x55e19152ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 --2- 172.21.15.18:0/2637016213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152ab60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17b40ce070 tx=0x7f17b40d9220 comp rx=0 tx=0).stop 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 -- 172.21.15.18:0/2637016213 shutdown_connections 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 --2- 172.21.15.18:0/2637016213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 0x55e19152d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 --2- 172.21.15.18:0/2637016213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152ab60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 --2- 172.21.15.18:0/2637016213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e191519580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 -- 172.21.15.18:0/2637016213 >> 172.21.15.18:0/2637016213 conn(0x55e191513ac0 msgr2=0x55e191513ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17c3400640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f17a80a3f50 0x7f17a80a6410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009fd0 tx=0x7f17bc009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 -- 172.21.15.18:0/2637016213 shutdown_connections 2024-04-16T17:05:31.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.957+0000 7f17cc852740 1 -- 172.21.15.18:0/2637016213 wait complete. 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.958+0000 7f17cc852740 1 -- start start 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.958+0000 7f17cc852740 1 Processor -- start 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17cc852740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1915191a0 0x55e191528160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17cc852740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e191519b50 0x55e1915286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17cc852740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e191528be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17cc852740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e19152de60 con 0x55e1915191a0 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17cc852740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e191438710 con 0x55e19152b0a0 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17cc852740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e19152dfe0 con 0x55e191519b50 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17c2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1915191a0 0x55e191528160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17cc852740 10 client.0 _ll_register_callbacks cb 0x55e191433af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:31.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17c2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1915191a0 0x55e191528160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56804/0 (socket says 172.21.15.18:56804) 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.959+0000 7f17c2a00640 1 -- 172.21.15.18:0/4226578775 learned_addr learned my addr 172.21.15.18:0/4226578775 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17c3400640 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e191528be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17c2000640 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e191519b50 0x55e1915286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17c3400640 1 -- 172.21.15.18:0/4226578775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e191519b50 msgr2=0x55e1915286a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17c3400640 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e191519b50 0x55e1915286a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17c3400640 1 -- 172.21.15.18:0/4226578775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1915191a0 msgr2=0x55e191528160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17c3400640 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1915191a0 0x55e191528160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17c3400640 1 -- 172.21.15.18:0/4226578775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1911913f0 con 0x55e19152b0a0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17c3400640 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e191528be0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f17bc003bf0 tx=0x7f17bc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d76dd1ad789d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.960+0000 7f17b0c00640 1 -- 172.21.15.18:0/4226578775 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc004450 con 0x55e19152b0a0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 1 -- 172.21.15.18:0/4226578775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17b409cf10 con 0x55e19152b0a0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17b0c00640 1 -- 172.21.15.18:0/4226578775 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f17bc0045f0 con 0x55e19152b0a0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17b0c00640 1 -- 172.21.15.18:0/4226578775 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f17bc030050 con 0x55e19152b0a0 2024-04-16T17:05:31.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 10 client.25913 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:31.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 1 -- 172.21.15.18:0/4226578775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1911510f0 con 0x55e19152b0a0 2024-04-16T17:05:31.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17b0c00640 1 -- 172.21.15.18:0/4226578775 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b409cf10 con 0x55e19152b0a0 2024-04-16T17:05:31.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17b0c00640 1 -- 172.21.15.18:0/4226578775 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f17bc01d3b0 con 0x55e19152b0a0 2024-04-16T17:05:31.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 10 client.25913 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:31.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 10 client.25913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:31.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 20 client.25913 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:31.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e19153b4c0 0x55e19153d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 4 client.25913 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.961+0000 7f17cc852740 1 -- 172.21.15.18:0/4226578775 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e19148cb40 con 0x55e19153b4c0 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.962+0000 7f17c2a00640 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e19153b4c0 0x55e19153d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.962+0000 7f17c2a00640 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e19153b4c0 0x55e19153d910 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4deda36921514de2 server_cookie=4760124dcb2442b6 in_seq=0 out_seq=0 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.962+0000 7f17b0c00640 10 client.25913 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.963+0000 7f17b0c00640 1 -- 172.21.15.18:0/4226578775 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e19148cb40 con 0x55e19153b4c0 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.963+0000 7f17b0c00640 10 client.25913 handle_command_reply: tid=0 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.963+0000 7f17cc852740 1 client.25913 shutdown 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 -- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 msgr2=0x55e191528be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e191528be0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f17bc003bf0 tx=0x7f17bc004a40 comp rx=0 tx=0).stop 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 -- 172.21.15.18:0/4226578775 shutdown_connections 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e19153b4c0 0x55e19153d910 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e191528be0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e191519b50 0x55e1915286a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 --2- 172.21.15.18:0/4226578775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1915191a0 0x55e191528160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 -- 172.21.15.18:0/4226578775 >> 172.21.15.18:0/4226578775 conn(0x55e191513ac0 msgr2=0x55e191515810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17b0c00640 0 client.25913 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:31.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 -- 172.21.15.18:0/4226578775 shutdown_connections 2024-04-16T17:05:31.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 1 -- 172.21.15.18:0/4226578775 wait complete. 2024-04-16T17:05:31.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.964+0000 7f17cc852740 20 client.25913 trim_cache size 0 max 16384 2024-04-16T17:05:31.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.992+0000 7f17cc852740 1 Processor -- start 2024-04-16T17:05:31.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.992+0000 7f17cc852740 1 -- start start 2024-04-16T17:05:31.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.992+0000 7f17cc852740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e19152a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.992+0000 7f17cc852740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e191519b50 0x55e191536d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.992+0000 7f17cc852740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e1915272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.992+0000 7f17cc852740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41afbf0 con 0x55e191519b50 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.992+0000 7f17cc852740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41ad4c0 con 0x55e19152b0a0 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.992+0000 7f17cc852740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41ab310 con 0x55e1915191a0 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e191519b50 0x55e191536d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e1915272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e191519b50 0x55e191536d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56814/0 (socket says 172.21.15.18:56814) 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c2000640 1 -- 172.21.15.18:0/1717445512 learned_addr learned my addr 172.21.15.18:0/1717445512 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c2a00640 1 --2- 172.21.15.18:0/1717445512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e19152a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c3400640 1 -- 172.21.15.18:0/1717445512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 msgr2=0x55e19152a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c3400640 1 --2- 172.21.15.18:0/1717445512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e19152a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c3400640 1 -- 172.21.15.18:0/1717445512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e191519b50 msgr2=0x55e191536d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:31.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c3400640 1 --2- 172.21.15.18:0/1717445512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e191519b50 0x55e191536d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.993+0000 7f17c3400640 1 -- 172.21.15.18:0/1717445512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1915198f0 con 0x55e19152b0a0 2024-04-16T17:05:31.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.994+0000 7f17c3400640 1 --2- 172.21.15.18:0/1717445512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e1915272c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f17bc01d780 tx=0x7f17bc012570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c985181d5bdc69f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.994+0000 7f17b3400640 1 -- 172.21.15.18:0/1717445512 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc010bb0 con 0x55e19152b0a0 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.994+0000 7f17b3400640 1 -- 172.21.15.18:0/1717445512 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17bc009c20 con 0x55e19152b0a0 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.994+0000 7f17b3400640 1 -- 172.21.15.18:0/1717445512 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc005030 con 0x55e19152b0a0 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 -- 172.21.15.18:0/1717445512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 msgr2=0x55e1915272c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 --2- 172.21.15.18:0/1717445512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e1915272c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f17bc01d780 tx=0x7f17bc012570 comp rx=0 tx=0).stop 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 -- 172.21.15.18:0/1717445512 shutdown_connections 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 --2- 172.21.15.18:0/1717445512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e19152b0a0 0x55e1915272c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 --2- 172.21.15.18:0/1717445512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e191519b50 0x55e191536d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 --2- 172.21.15.18:0/1717445512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e19152a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 -- 172.21.15.18:0/1717445512 >> 172.21.15.18:0/1717445512 conn(0x55e191513ac0 msgr2=0x55e191512290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 -- 172.21.15.18:0/1717445512 shutdown_connections 2024-04-16T17:05:31.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.995+0000 7f17cc852740 1 -- 172.21.15.18:0/1717445512 wait complete. 2024-04-16T17:05:31.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 1 -- start start 2024-04-16T17:05:31.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 1 Processor -- start 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e1915335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 0x55e19152ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41ab980 con 0x55e19152b0a0 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41acdf0 con 0x55e191519b50 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41ac050 con 0x55e1915191a0 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.996+0000 7f17cc852740 10 client.0 _ll_register_callbacks cb 0x7f17bc009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e1915335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e1915335e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56344/0 (socket says 172.21.15.18:56344) 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 -- 172.21.15.18:0/1865517530 learned_addr learned my addr 172.21.15.18:0/1865517530 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c3400640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 0x55e19152ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:32.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2000640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 -- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 msgr2=0x55e19152e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 -- 172.21.15.18:0/1865517530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 msgr2=0x55e19152ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 0x55e19152ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 -- 172.21.15.18:0/1865517530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e19152f670 con 0x55e1915191a0 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2000640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c3400640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 0x55e19152ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.997+0000 7f17c2a00640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e1915335e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f17b40ce190 tx=0x7f17b400bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb321b70c08b18d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.998+0000 7f17b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.998+0000 7f17b0c00640 1 -- 172.21.15.18:0/1865517530 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b407d500 con 0x55e1915191a0 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.998+0000 7f17b0c00640 1 -- 172.21.15.18:0/1865517530 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f17b409ab10 con 0x55e1915191a0 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.998+0000 7f17cc852740 1 -- 172.21.15.18:0/1865517530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1914869b0 con 0x55e1915191a0 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.998+0000 7f17b0c00640 1 -- 172.21.15.18:0/1865517530 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f17b407b3d0 con 0x55e1915191a0 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.998+0000 7f17cc852740 10 client.25915 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.998+0000 7f17cc852740 1 -- 172.21.15.18:0/1865517530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1915a19f0 con 0x55e1915191a0 2024-04-16T17:05:32.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.999+0000 7f17b0c00640 1 -- 172.21.15.18:0/1865517530 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e1914869b0 con 0x55e1915191a0 2024-04-16T17:05:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:31 smithi043 ceph-mon[36901]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.999+0000 7f17b0c00640 1 -- 172.21.15.18:0/1865517530 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f17b4098e80 con 0x55e1915191a0 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.999+0000 7f17cc852740 10 client.25915 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.999+0000 7f17cc852740 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.999+0000 7f17cc852740 20 client.25915 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.999+0000 7f17cc852740 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e1915a1c20 0x55e1915a4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.999+0000 7f17cc852740 4 client.25915 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:31.999+0000 7f17cc852740 1 -- 172.21.15.18:0/1865517530 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e191402420 con 0x55e1915a1c20 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.000+0000 7f17c2000640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e1915a1c20 0x55e1915a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.000+0000 7f17c2000640 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e1915a1c20 0x55e1915a4010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d15698e5d6cb59 server_cookie=297fe8da7321cc87 in_seq=0 out_seq=0 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.000+0000 7f17b0c00640 10 client.25915 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17b0c00640 1 -- 172.21.15.18:0/1865517530 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e191402420 con 0x55e1915a1c20 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17b0c00640 10 client.25915 handle_command_reply: tid=0 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 client.25915 shutdown 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 -- 172.21.15.18:0/1865517530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 msgr2=0x55e1915335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e1915335e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f17b40ce190 tx=0x7f17b400bf60 comp rx=0 tx=0).stop 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 -- 172.21.15.18:0/1865517530 shutdown_connections 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e1915a1c20 0x55e1915a4010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e19152b0a0 0x55e19152ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e191519b50 0x55e19152e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 --2- 172.21.15.18:0/1865517530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1915191a0 0x55e1915335e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 -- 172.21.15.18:0/1865517530 >> 172.21.15.18:0/1865517530 conn(0x55e191513ac0 msgr2=0x55e191514ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17b0c00640 0 client.25915 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 -- 172.21.15.18:0/1865517530 shutdown_connections 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 1 -- 172.21.15.18:0/1865517530 wait complete. 2024-04-16T17:05:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.001+0000 7f17cc852740 20 client.25915 trim_cache size 0 max 16384 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout: "4279e0e9-c11f-4f6b-9098-67f0e911d3b1": { 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "4279e0e9-c11f-4f6b-9098-67f0e911d3b1", 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:05:32.006 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 -- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f17a80a3f50 msgr2=0x7f17a80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f17a80a3f50 0x7f17a80a6410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009fd0 tx=0x7f17bc009500 comp rx=0 tx=0).stop 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 -- 172.21.15.18:0/706768190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c40743b0 msgr2=0x7f17c41a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c40743b0 0x7f17c41a0680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17b400d950 tx=0x7f17b400de10 comp rx=0 tx=0).stop 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 -- 172.21.15.18:0/706768190 shutdown_connections 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f17a80a3f50 0x7f17a80a6410 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c410e6c0 0x7f17c419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c4074cd0 0x7f17c41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 --2- 172.21.15.18:0/706768190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c40743b0 0x7f17c41a0680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 -- 172.21.15.18:0/706768190 >> 172.21.15.18:0/706768190 conn(0x7f17c40fd950 msgr2=0x7f17c40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:32.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.002+0000 7f17c3e00640 1 -- 172.21.15.18:0/706768190 shutdown_connections 2024-04-16T17:05:32.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:05:32.003+0000 7f17c3e00640 1 -- 172.21.15.18:0/706768190 wait complete. 2024-04-16T17:05:32.021 INFO:tasks.cephfs.filesystem:scrub status for tag:4279e0e9-c11f-4f6b-9098-67f0e911d3b1 - {'path': '/', 'tag': '4279e0e9-c11f-4f6b-9098-67f0e911d3b1', 'options': 'recursive,force'} 2024-04-16T17:05:32.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:32 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:05:32.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:32 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:32.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:32 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:05:32.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:32 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:32 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:05:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:32 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:32 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:05:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:32 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:33.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:32 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:05:33.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:32 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:33.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:32 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:05:33.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:32 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:05:33.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:33 smithi022 ceph-mon[36023]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T17:05:33.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:33 smithi018 ceph-mon[31776]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T17:05:34.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:33 smithi043 ceph-mon[36901]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-16T17:05:34.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:34 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:05:34.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:34 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:05:35.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:34 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:05:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:35 smithi022 ceph-mon[36023]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T17:05:35.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:35 smithi018 ceph-mon[31776]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T17:05:36.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:35 smithi043 ceph-mon[36901]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-16T17:05:36.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:36 smithi022 ceph-mon[36023]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-16T17:05:36.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:36 smithi018 ceph-mon[31776]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-16T17:05:37.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:36 smithi043 ceph-mon[36901]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-16T17:05:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 -- 172.21.15.18:0/4266298170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 msgr2=0x7f9eb01033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 --2- 172.21.15.18:0/4266298170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 0x7f9eb01033c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009900 tx=0x7f9ea002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 -- 172.21.15.18:0/4266298170 shutdown_connections 2024-04-16T17:05:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 --2- 172.21.15.18:0/4266298170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb010c400 0x7f9eb010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 --2- 172.21.15.18:0/4266298170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0103900 0x7f9eb010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 --2- 172.21.15.18:0/4266298170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 0x7f9eb01033c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 -- 172.21.15.18:0/4266298170 >> 172.21.15.18:0/4266298170 conn(0x7f9eb00fc7e0 msgr2=0x7f9eb00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 -- 172.21.15.18:0/4266298170 shutdown_connections 2024-04-16T17:05:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.154+0000 7f9eb5800640 1 -- 172.21.15.18:0/4266298170 wait complete. 2024-04-16T17:05:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.156+0000 7f9eb5800640 1 Processor -- start 2024-04-16T17:05:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.156+0000 7f9eb5800640 1 -- start start 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.156+0000 7f9eb5800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 0x7f9eb01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.156+0000 7f9eb5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0103900 0x7f9eb01a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eb5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb010c400 0x7f9eb019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eb5800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0112210 con 0x7f9eb0102fe0 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eb5800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0112090 con 0x7f9eb0103900 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eb5800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0112390 con 0x7f9eb010c400 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 0x7f9eb01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 0x7f9eb01a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47360/0 (socket says 172.21.15.18:47360) 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eafe00640 1 -- 172.21.15.18:0/1319169650 learned_addr learned my addr 172.21.15.18:0/1319169650 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eb4e00640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb010c400 0x7f9eb019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eaf400640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0103900 0x7f9eb01a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eafe00640 1 -- 172.21.15.18:0/1319169650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb010c400 msgr2=0x7f9eb019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eafe00640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb010c400 0x7f9eb019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eafe00640 1 -- 172.21.15.18:0/1319169650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0103900 msgr2=0x7f9eb01a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eafe00640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0103900 0x7f9eb01a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.157+0000 7f9eafe00640 1 -- 172.21.15.18:0/1319169650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb019afc0 con 0x7f9eb0102fe0 2024-04-16T17:05:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.158+0000 7f9eafe00640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 0x7f9eb01a0640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000c00 tx=0x7f9ea0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.158+0000 7f9eacc00640 1 -- 172.21.15.18:0/1319169650 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0042020 con 0x7f9eb0102fe0 2024-04-16T17:05:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.158+0000 7f9eb5800640 1 -- 172.21.15.18:0/1319169650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb019b250 con 0x7f9eb0102fe0 2024-04-16T17:05:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.158+0000 7f9eacc00640 1 -- 172.21.15.18:0/1319169650 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00054e0 con 0x7f9eb0102fe0 2024-04-16T17:05:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.158+0000 7f9eacc00640 1 -- 172.21.15.18:0/1319169650 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0003af0 con 0x7f9eb0102fe0 2024-04-16T17:05:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.158+0000 7f9eb5800640 1 -- 172.21.15.18:0/1319169650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01a75c0 con 0x7f9eb0102fe0 2024-04-16T17:05:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.162+0000 7f9eb9a40740 1 Processor -- start 2024-04-16T17:05:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.163+0000 7f9eb9a40740 1 -- start start 2024-04-16T17:05:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.163+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca317f0 0x55848ca30540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.163+0000 7f9eacc00640 1 -- 172.21.15.18:0/1319169650 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9ea0045020 con 0x7f9eb0102fe0 2024-04-16T17:05:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.163+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca30f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca3a000 0x55848ca3c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55848c995780 con 0x55848ca317f0 2024-04-16T17:05:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55848c93bf40 con 0x55848ca30a80 2024-04-16T17:05:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55848c991740 con 0x55848ca3a000 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca30f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca3a000 0x55848ca3c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca30f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53548/0 (socket says 172.21.15.18:53548) 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eafe00640 1 -- 172.21.15.18:0/922855625 learned_addr learned my addr 172.21.15.18:0/922855625 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca3a000 0x55848ca3c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51900/0 (socket says 172.21.15.18:51900) 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eafe00640 1 -- 172.21.15.18:0/922855625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca3a000 msgr2=0x55848ca3c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eafe00640 1 --2- 172.21.15.18:0/922855625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca3a000 0x55848ca3c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eafe00640 1 -- 172.21.15.18:0/922855625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca317f0 msgr2=0x55848ca30540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.164+0000 7f9eafe00640 1 --2- 172.21.15.18:0/922855625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca317f0 0x55848ca30540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.165+0000 7f9eafe00640 1 -- 172.21.15.18:0/922855625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55848c6a8210 con 0x55848ca30a80 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.165+0000 7f9eafe00640 1 --2- 172.21.15.18:0/922855625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca30f00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00efae0 tx=0x7f9ea0102320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=278097d438de0b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.165+0000 7f9ea5600640 1 -- 172.21.15.18:0/922855625 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00f8070 con 0x55848ca30a80 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.165+0000 7f9ea5600640 1 -- 172.21.15.18:0/922855625 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea01005b0 con 0x55848ca30a80 2024-04-16T17:05:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.165+0000 7f9ea5600640 1 -- 172.21.15.18:0/922855625 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00f4110 con 0x55848ca30a80 2024-04-16T17:05:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.166+0000 7f9eacc00640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e940a3eb0 0x7f9e940a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.166+0000 7f9eacc00640 1 -- 172.21.15.18:0/1319169650 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9ea00f28d0 con 0x7f9eb0102fe0 2024-04-16T17:05:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.166+0000 7f9eb9a40740 1 -- 172.21.15.18:0/922855625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 msgr2=0x55848ca30f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.166+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/922855625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca30f00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00efae0 tx=0x7f9ea0102320 comp rx=0 tx=0).stop 2024-04-16T17:05:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.166+0000 7f9eb4e00640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e940a3eb0 0x7f9e940a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.167+0000 7f9eb9a40740 1 -- 172.21.15.18:0/922855625 shutdown_connections 2024-04-16T17:05:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.167+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/922855625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca3a000 0x55848ca3c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.167+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/922855625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca30f00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.167+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/922855625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca317f0 0x55848ca30540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.167+0000 7f9eb9a40740 1 -- 172.21.15.18:0/922855625 >> 172.21.15.18:0/922855625 conn(0x55848c9cb5d0 msgr2=0x55848ca2b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.167+0000 7f9eb9a40740 1 -- 172.21.15.18:0/922855625 shutdown_connections 2024-04-16T17:05:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.167+0000 7f9eb9a40740 1 -- 172.21.15.18:0/922855625 wait complete. 2024-04-16T17:05:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.167+0000 7f9eb4e00640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e940a3eb0 0x7f9e940a6370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8005fd0 tx=0x7f9ea8009a90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.168+0000 7f9eb9a40740 1 -- start start 2024-04-16T17:05:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.168+0000 7f9eb9a40740 1 Processor -- start 2024-04-16T17:05:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca39a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca317f0 0x55848ca36df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca37510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55848ca3caf0 con 0x55848ca3a000 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55848c94f390 con 0x55848ca317f0 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55848ca3cc70 con 0x55848ca30a80 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eb9a40740 10 client.0 _ll_register_callbacks cb 0x55848c938f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca39a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca39a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51904/0 (socket says 172.21.15.18:51904) 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eafe00640 1 -- 172.21.15.18:0/3575404347 learned_addr learned my addr 172.21.15.18:0/3575404347 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eaf400640 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca317f0 0x55848ca36df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.169+0000 7f9eb4e00640 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca37510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.170+0000 7f9eaf400640 1 -- 172.21.15.18:0/3575404347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 msgr2=0x55848ca39a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.170+0000 7f9eaf400640 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca39a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.170+0000 7f9eaf400640 1 -- 172.21.15.18:0/3575404347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 msgr2=0x55848ca37510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.170+0000 7f9eaf400640 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca37510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.170+0000 7f9eaf400640 1 -- 172.21.15.18:0/3575404347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55848c774620 con 0x55848ca317f0 2024-04-16T17:05:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.170+0000 7f9eb4e00640 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca37510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.170+0000 7f9eaf400640 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca317f0 0x55848ca36df0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55848ca31520 tx=0x7f9e9c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73de4159b9b1b11d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.170+0000 7f9e9a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.171+0000 7f9e9a000640 1 -- 172.21.15.18:0/3575404347 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00f910 con 0x55848ca317f0 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.171+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3575404347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea01005b0 con 0x55848ca317f0 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.171+0000 7f9e9a000640 1 -- 172.21.15.18:0/3575404347 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9e9c00fab0 con 0x55848ca317f0 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.171+0000 7f9e9a000640 1 -- 172.21.15.18:0/3575404347 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e9c039050 con 0x55848ca317f0 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.171+0000 7f9eb9a40740 10 client.25931 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.171+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3575404347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55848ca48bd0 con 0x55848ca317f0 2024-04-16T17:05:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.171+0000 7f9e9a000640 1 -- 172.21.15.18:0/3575404347 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea01005b0 con 0x55848ca317f0 2024-04-16T17:05:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.172+0000 7f9e9a000640 1 -- 172.21.15.18:0/3575404347 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e9c034030 con 0x55848ca317f0 2024-04-16T17:05:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.172+0000 7f9eb9a40740 10 client.25931 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.172+0000 7f9eb9a40740 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.172+0000 7f9eb9a40740 20 client.25931 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.172+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55848ca4a100 0x55848ca4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.172+0000 7f9eb9a40740 4 client.25931 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.172+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3575404347 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55848c9a31a0 con 0x55848ca4a100 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.172+0000 7f9eafe00640 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55848ca4a100 0x55848ca4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.173+0000 7f9eafe00640 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55848ca4a100 0x55848ca4c4f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691ff09abb5c503c server_cookie=c223d79775b1ee46 in_seq=0 out_seq=0 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.173+0000 7f9e9a000640 10 client.25931 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9e9a000640 1 -- 172.21.15.18:0/3575404347 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55848c9a31a0 con 0x55848ca4a100 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9e9a000640 10 client.25931 handle_command_reply: tid=0 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 client.25931 shutdown 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca317f0 msgr2=0x55848ca36df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca317f0 0x55848ca36df0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55848ca31520 tx=0x7f9e9c009f90 comp rx=0 tx=0).stop 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3575404347 shutdown_connections 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55848ca4a100 0x55848ca4c4f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca37510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca317f0 0x55848ca36df0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3575404347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca39a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9e9a000640 0 client.25931 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.174+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3575404347 >> 172.21.15.18:0/3575404347 conn(0x55848c9cb5d0 msgr2=0x55848ca2d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.178+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3575404347 shutdown_connections 2024-04-16T17:05:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.178+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3575404347 wait complete. 2024-04-16T17:05:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.178+0000 7f9eb9a40740 20 client.25931 trim_cache size 0 max 16384 2024-04-16T17:05:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.194+0000 7f9eb9a40740 1 Processor -- start 2024-04-16T17:05:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.194+0000 7f9eb9a40740 1 -- start start 2024-04-16T17:05:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.194+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.194+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca317f0 0x55848ca35e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.194+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca3a000 0x55848ca51db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.194+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01afad0 con 0x55848ca317f0 2024-04-16T17:05:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.194+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01ad3a0 con 0x55848ca3a000 2024-04-16T17:05:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.194+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01ab250 con 0x55848ca30a80 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51910/0 (socket says 172.21.15.18:51910) 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 -- 172.21.15.18:0/3227178791 learned_addr learned my addr 172.21.15.18:0/3227178791 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eb4e00640 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca3a000 0x55848ca51db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 -- 172.21.15.18:0/3227178791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca3a000 msgr2=0x55848ca51db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca3a000 0x55848ca51db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 -- 172.21.15.18:0/3227178791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca317f0 msgr2=0x55848ca35e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca317f0 0x55848ca35e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 -- 172.21.15.18:0/3227178791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55848ca4c860 con 0x55848ca30a80 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eb4e00640 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca3a000 0x55848ca51db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.195+0000 7f9eafe00640 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca459b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ea002cb70 tx=0x7f9ea00fe720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef4d1c5da1eaf4b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.196+0000 7f9ea4c00640 1 -- 172.21.15.18:0/3227178791 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00f8070 con 0x55848ca30a80 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.196+0000 7f9ea4c00640 1 -- 172.21.15.18:0/3227178791 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00fe790 con 0x55848ca30a80 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.196+0000 7f9ea4c00640 1 -- 172.21.15.18:0/3227178791 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00bea90 con 0x55848ca30a80 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.196+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3227178791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 msgr2=0x55848ca459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.196+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca459b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ea002cb70 tx=0x7f9ea00fe720 comp rx=0 tx=0).stop 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3227178791 shutdown_connections 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca3a000 0x55848ca51db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca317f0 0x55848ca35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/3227178791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca30a80 0x55848ca459b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3227178791 >> 172.21.15.18:0/3227178791 conn(0x55848c9cb5d0 msgr2=0x55848ca35720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3227178791 shutdown_connections 2024-04-16T17:05:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 -- 172.21.15.18:0/3227178791 wait complete. 2024-04-16T17:05:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 -- start start 2024-04-16T17:05:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.197+0000 7f9eb9a40740 1 Processor -- start 2024-04-16T17:05:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca33c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca317f0 0x55848ca34180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eb9a40740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca346c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01ab8a0 con 0x55848ca3a000 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01accd0 con 0x55848ca30a80 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eb9a40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01abf30 con 0x55848ca317f0 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eb9a40740 10 client.0 _ll_register_callbacks cb 0x7f9e9c009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eb4e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca317f0 0x55848ca34180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca317f0 0x55848ca34180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51926/0 (socket says 172.21.15.18:51926) 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eaf400640 1 -- 172.21.15.18:0/2383041600 learned_addr learned my addr 172.21.15.18:0/2383041600 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.198+0000 7f9eafe00640 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca33c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9eaf400640 1 -- 172.21.15.18:0/2383041600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 msgr2=0x55848ca33c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9eaf400640 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca33c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9eaf400640 1 -- 172.21.15.18:0/2383041600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 msgr2=0x55848ca346c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9eaf400640 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca346c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9eaf400640 1 -- 172.21.15.18:0/2383041600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55848caaf720 con 0x55848ca317f0 2024-04-16T17:05:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9eaf400640 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca317f0 0x55848ca34180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c02b480 tx=0x7f9e9c02b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5a80662be3018 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9e9a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9e9a000640 1 -- 172.21.15.18:0/2383041600 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00fb20 con 0x55848ca317f0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9e9a000640 1 -- 172.21.15.18:0/2383041600 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9e9c00fcc0 con 0x55848ca317f0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.199+0000 7f9eb9a40740 1 -- 172.21.15.18:0/2383041600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea00fe790 con 0x55848ca317f0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9e9a000640 1 -- 172.21.15.18:0/2383041600 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e9c04a050 con 0x55848ca317f0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9eb9a40740 10 client.25933 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9eb9a40740 1 -- 172.21.15.18:0/2383041600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55848cab0940 con 0x55848ca317f0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9e9a000640 1 -- 172.21.15.18:0/2383041600 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00fe790 con 0x55848ca317f0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9e9a000640 1 -- 172.21.15.18:0/2383041600 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e9c045030 con 0x55848ca317f0 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9eb9a40740 10 client.25933 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9eb9a40740 10 client.25933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9eb9a40740 20 client.25933 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55848cab10b0 0x55848cab34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9eb9a40740 4 client.25933 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:05:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.200+0000 7f9eb9a40740 1 -- 172.21.15.18:0/2383041600 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55848c99d010 con 0x55848cab10b0 2024-04-16T17:05:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.201+0000 7f9eafe00640 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55848cab10b0 0x55848cab34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.201+0000 7f9eafe00640 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55848cab10b0 0x55848cab34d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329203bd561fc473 server_cookie=d19e73ca2d113de5 in_seq=0 out_seq=0 2024-04-16T17:05:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.201+0000 7f9e9a000640 10 client.25933 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.209+0000 7f9e9a000640 1 -- 172.21.15.18:0/2383041600 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55848c99d010 con 0x55848cab10b0 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.209+0000 7f9e9a000640 10 client.25933 handle_command_reply: tid=0 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.209+0000 7f9eb9a40740 1 client.25933 shutdown 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 -- 172.21.15.18:0/2383041600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca317f0 msgr2=0x55848ca34180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca317f0 0x55848ca34180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c02b480 tx=0x7f9e9c02b7b0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 -- 172.21.15.18:0/2383041600 shutdown_connections 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55848cab10b0 0x55848cab34d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55848ca3a000 0x55848ca346c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55848ca317f0 0x55848ca34180 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 --2- 172.21.15.18:0/2383041600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55848ca30a80 0x55848ca33c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9e9a000640 0 client.25933 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 -- 172.21.15.18:0/2383041600 >> 172.21.15.18:0/2383041600 conn(0x55848c9cb5d0 msgr2=0x55848ca29f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 -- 172.21.15.18:0/2383041600 shutdown_connections 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 1 -- 172.21.15.18:0/2383041600 wait complete. 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.210+0000 7f9eb9a40740 20 client.25933 trim_cache size 0 max 16384 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:05:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:05:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:05:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 -- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e940a3eb0 msgr2=0x7f9e940a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e940a3eb0 0x7f9e940a6370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8005fd0 tx=0x7f9ea8009a90 comp rx=0 tx=0).stop 2024-04-16T17:05:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 -- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 msgr2=0x7f9eb01a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 0x7f9eb01a0640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000c00 tx=0x7f9ea0004360 comp rx=0 tx=0).stop 2024-04-16T17:05:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 -- 172.21.15.18:0/1319169650 shutdown_connections 2024-04-16T17:05:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e940a3eb0 0x7f9e940a6370 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb010c400 0x7f9eb019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0103900 0x7f9eb01a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 --2- 172.21.15.18:0/1319169650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9eb0102fe0 0x7f9eb01a0640 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 -- 172.21.15.18:0/1319169650 >> 172.21.15.18:0/1319169650 conn(0x7f9eb00fc7e0 msgr2=0x7f9eb0100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 -- 172.21.15.18:0/1319169650 shutdown_connections 2024-04-16T17:05:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:37.211+0000 7f9eb5800640 1 -- 172.21.15.18:0/1319169650 wait complete. 2024-04-16T17:05:39.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:39 smithi043 ceph-mon[36901]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:05:39.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:39 smithi022 ceph-mon[36023]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:05:39.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:39 smithi018 ceph-mon[31776]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:05:41.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:41 smithi043 ceph-mon[36901]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:05:41.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:41 smithi022 ceph-mon[36023]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:05:41.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:41 smithi018 ceph-mon[31776]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-16T17:05:43.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:43 smithi043 ceph-mon[36901]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-16T17:05:43.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:43 smithi022 ceph-mon[36023]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-16T17:05:43.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:43 smithi018 ceph-mon[31776]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-16T17:05:45.428 INFO:tasks.workunit.client.0.smithi018.stderr:Updating files: 62% (7790/12482) Updating files: 63% (7864/12482) Updating files: 64% (7989/12482) Updating files: 65% (8114/12482) Updating files: 66% (8239/12482) Updating files: 67% (8363/12482) Updating files: 68% (8488/12482) Updating files: 69% (8613/12482) Updating files: 70% (8738/12482) Updating files: 71% (8863/12482) Updating files: 72% (8988/12482) Updating files: 73% (9112/12482) Updating files: 74% (9237/12482) Updating files: 75% (9362/12482) Updating files: 76% (9487/12482) Updating files: 77% (9612/12482) Updating files: 78% (9736/12482) Updating files: 79% (9861/12482) Updating files: 80% (9986/12482) Updating files: 81% (10111/12482) Updating files: 82% (10236/12482) Updating files: 83% (10361/12482) Updating files: 84% (10485/12482) Updating files: 85% (10610/12482) Updating files: 86% (10735/12482) Updating files: 87% (10860/12482) Updating files: 88% (10985/12482) Updating files: 89% (11109/12482) Updating files: 90% (11234/12482) Updating files: 91% (11359/12482) Updating files: 92% (11484/12482) Updating files: 93% (11609/12482) Updating files: 94% (11734/12482) Updating files: 95% (11858/12482) Updating files: 96% (11983/12482) Updating files: 97% (12108/12482) Updating files: 98% (12233/12482) Updating files: 99% (12358/12482) Updating files: 100% (12482/12482) Updating files: 100% (12482/12482), done. 2024-04-16T17:05:45.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:45 smithi043 ceph-mon[36901]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-16T17:05:45.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:45 smithi022 ceph-mon[36023]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-16T17:05:45.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:45 smithi018 ceph-mon[31776]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr:Note: switching to 'b137e4a675398e295567f9652210865a4d573fac'. 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr:state without impacting any branches by switching back to a branch. 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr: git switch -c 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr:Or undo this operation with: 2024-04-16T17:05:45.807 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-16T17:05:45.808 INFO:tasks.workunit.client.0.smithi018.stderr: git switch - 2024-04-16T17:05:45.808 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-16T17:05:45.808 INFO:tasks.workunit.client.0.smithi018.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-16T17:05:45.808 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-16T17:05:45.808 INFO:tasks.workunit.client.0.smithi018.stderr:HEAD is now at b137e4a6753 qa: enable dynamic debug for dbench workunit 2024-04-16T17:05:45.810 DEBUG:teuthology.orchestra.run.smithi018:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-16T17:05:45.873 INFO:tasks.workunit.client.0.smithi018.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-16T17:05:45.881 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T17:05:45.881 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-16T17:05:46.824 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-16T17:05:46.950 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-16T17:05:47.041 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T17:05:47.047 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T17:05:47.047 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-16T17:05:47.134 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T17:05:47.145 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T17:05:47.145 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-16T17:05:47.193 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-16T17:05:47.194 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-16T17:05:47.194 DEBUG:teuthology.orchestra.run.smithi018:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b137e4a675398e295567f9652210865a4d573fac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-16T17:05:47.313 INFO:tasks.workunit.client.0.smithi018.stderr:+ sudo -s eval 'echo -n '\''module ceph +mp'\'' > /sys/kernel/debug/dynamic_debug/control' 2024-04-16T17:05:47.381 INFO:tasks.workunit.client.0.smithi018.stderr:+ dbench 1 2024-04-16T17:05:47.393 INFO:tasks.workunit.client.0.smithi018.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-16T17:05:47.393 INFO:tasks.workunit.client.0.smithi018.stdout: 2024-04-16T17:05:47.393 INFO:tasks.workunit.client.0.smithi018.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-16T17:05:47.397 INFO:tasks.workunit.client.0.smithi018.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-16T17:05:47.497 INFO:tasks.workunit.client.0.smithi018.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-16T17:05:47.497 INFO:tasks.workunit.client.0.smithi018.stdout:releasing clients 2024-04-16T17:05:47.498 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:47 smithi018 ceph-mon[31776]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-16T17:05:47.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:47 smithi043 ceph-mon[36901]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-16T17:05:47.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:47 smithi022 ceph-mon[36023]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-16T17:05:48.498 INFO:tasks.workunit.client.0.smithi018.stdout: 1 297 10.96 MB/sec warmup 1 sec latency 59.200 ms 2024-04-16T17:05:49.498 INFO:tasks.workunit.client.0.smithi018.stdout: 1 603 10.48 MB/sec warmup 2 sec latency 85.023 ms 2024-04-16T17:05:49.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:49 smithi043 ceph-mon[36901]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:05:49.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:49 smithi022 ceph-mon[36023]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:05:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 -- 172.21.15.18:0/3791622717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ab0104470 msgr2=0x7f6ab0104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3791622717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ab0104470 0x7f6ab0104850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009900 tx=0x7f6aa002bc30 comp rx=0 tx=0).stop 2024-04-16T17:05:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 -- 172.21.15.18:0/3791622717 shutdown_connections 2024-04-16T17:05:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3791622717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab0109540 0x7f6ab0111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3791622717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ab0104e20 0x7f6ab0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3791622717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ab0104470 0x7f6ab0104850 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 -- 172.21.15.18:0/3791622717 >> 172.21.15.18:0/3791622717 conn(0x7f6ab00ffcd0 msgr2=0x7f6ab01020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 -- 172.21.15.18:0/3791622717 shutdown_connections 2024-04-16T17:05:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.379+0000 7f6ab7000640 1 -- 172.21.15.18:0/3791622717 wait complete. 2024-04-16T17:05:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.380+0000 7f6ab7000640 1 Processor -- start 2024-04-16T17:05:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.380+0000 7f6ab7000640 1 -- start start 2024-04-16T17:05:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6ab7000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ab0104470 0x7f6ab01a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6ab7000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab0109540 0x7f6ab01a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6ab7000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ab019a9a0 0x7f6ab019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6ab7000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0114620 con 0x7f6ab0104470 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6ab7000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01144a0 con 0x7f6ab019a9a0 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6ab7000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01147a0 con 0x7f6ab0109540 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6aafe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ab0104470 0x7f6ab01a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6aafe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ab0104470 0x7f6ab01a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35990/0 (socket says 172.21.15.18:35990) 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6aafe00640 1 -- 172.21.15.18:0/3074241038 learned_addr learned my addr 172.21.15.18:0/3074241038 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6ab5000640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ab019a9a0 0x7f6ab019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.381+0000 7f6aaf400640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab0109540 0x7f6ab01a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aafe00640 1 -- 172.21.15.18:0/3074241038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab0109540 msgr2=0x7f6ab01a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aafe00640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab0109540 0x7f6ab01a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aafe00640 1 -- 172.21.15.18:0/3074241038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ab019a9a0 msgr2=0x7f6ab019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aafe00640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ab019a9a0 0x7f6ab019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aafe00640 1 -- 172.21.15.18:0/3074241038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab019b4c0 con 0x7f6ab0104470 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aafe00640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ab0104470 0x7f6ab01a07a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6aa002c240 tx=0x7f6aa00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aacc00640 1 -- 172.21.15.18:0/3074241038 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0043020 con 0x7f6ab0104470 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aacc00640 1 -- 172.21.15.18:0/3074241038 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa00054e0 con 0x7f6ab0104470 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6ab7000640 1 -- 172.21.15.18:0/3074241038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab019b750 con 0x7f6ab0104470 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.382+0000 7f6aacc00640 1 -- 172.21.15.18:0/3074241038 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0003b20 con 0x7f6ab0104470 2024-04-16T17:05:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.383+0000 7f6ab7000640 1 -- 172.21.15.18:0/3074241038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab01a7950 con 0x7f6ab0104470 2024-04-16T17:05:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.387+0000 7f6aacc00640 1 -- 172.21.15.18:0/3074241038 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f6aa0046020 con 0x7f6ab0104470 2024-04-16T17:05:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.387+0000 7f6ab9d93740 1 Processor -- start 2024-04-16T17:05:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.387+0000 7f6ab9d93740 1 -- start start 2024-04-16T17:05:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c8610 0x55dfea7c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dfea72c780 con 0x55dfea7d0e20 2024-04-16T17:05:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dfea6d2f40 con 0x55dfea7c8610 2024-04-16T17:05:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfea728740 con 0x55dfea7c78a0 2024-04-16T17:05:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c8610 0x55dfea7c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c8610 0x55dfea7c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51638/0 (socket says 172.21.15.18:51638) 2024-04-16T17:05:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6ab5000640 1 -- 172.21.15.18:0/1970390510 learned_addr learned my addr 172.21.15.18:0/1970390510 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.388+0000 7f6aafe00640 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6ab5000640 1 -- 172.21.15.18:0/1970390510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 msgr2=0x55dfea7c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6ab5000640 1 -- 172.21.15.18:0/1970390510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 msgr2=0x55dfea7d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6ab5000640 1 -- 172.21.15.18:0/1970390510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfea43f210 con 0x55dfea7c8610 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6aacc00640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6a940a3ec0 0x7f6a940a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6aacc00640 1 -- 172.21.15.18:0/3074241038 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f6aa00f2d00 con 0x7f6ab0104470 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c8610 0x55dfea7c7360 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009f50 tx=0x7f6aa80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d9ef58e079c5dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6aafe00640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6a940a3ec0 0x7f6a940a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6aa5600640 1 -- 172.21.15.18:0/1970390510 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8004d20 con 0x55dfea7c8610 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6aa5600640 1 -- 172.21.15.18:0/1970390510 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa8004050 con 0x55dfea7c8610 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.389+0000 7f6aa5600640 1 -- 172.21.15.18:0/1970390510 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8005280 con 0x55dfea7c8610 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1970390510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c8610 msgr2=0x55dfea7c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c8610 0x55dfea7c7360 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009f50 tx=0x7f6aa80118d0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6aafe00640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6a940a3ec0 0x7f6a940a6380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00effd0 tx=0x7f6aa00f67b0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1970390510 shutdown_connections 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1970390510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c8610 0x55dfea7c7360 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1970390510 >> 172.21.15.18:0/1970390510 conn(0x55dfea7625d0 msgr2=0x55dfea7c2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.390+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1970390510 shutdown_connections 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.391+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1970390510 wait complete. 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.391+0000 7f6ab9d93740 1 -- start start 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.391+0000 7f6ab9d93740 1 Processor -- start 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 0x55dfea7ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dfea7d3b50 con 0x55dfea7c8610 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dfea6e6390 con 0x55dfea7d0e20 2024-04-16T17:05:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfea7d3cd0 con 0x55dfea7c78a0 2024-04-16T17:05:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab9d93740 10 client.0 _ll_register_callbacks cb 0x55dfea6cff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40138/0 (socket says 172.21.15.18:40138) 2024-04-16T17:05:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 -- 172.21.15.18:0/1293928950 learned_addr learned my addr 172.21.15.18:0/1293928950 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 0x55dfea7ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 -- 172.21.15.18:0/1293928950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 msgr2=0x55dfea7ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 -- 172.21.15.18:0/1293928950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 msgr2=0x55dfea7ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 0x55dfea7ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.392+0000 7f6ab5000640 1 -- 172.21.15.18:0/1293928950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfea50b620 con 0x55dfea7c78a0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.393+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7cdcb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009250 tx=0x7f6aa80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d505d8a791b4608 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.393+0000 7f6a9a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.393+0000 7f6a9a000640 1 -- 172.21.15.18:0/1293928950 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa802ab40 con 0x55dfea7c78a0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.393+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1293928950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa8004050 con 0x55dfea7c78a0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.393+0000 7f6a9a000640 1 -- 172.21.15.18:0/1293928950 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f6aa802ace0 con 0x55dfea7c78a0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.393+0000 7f6a9a000640 1 -- 172.21.15.18:0/1293928950 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6aa8004200 con 0x55dfea7c78a0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.393+0000 7f6ab9d93740 10 client.25945 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.393+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1293928950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfea7dfc00 con 0x55dfea7c78a0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.394+0000 7f6a9a000640 1 -- 172.21.15.18:0/1293928950 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa8004050 con 0x55dfea7c78a0 2024-04-16T17:05:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.394+0000 7f6a9a000640 1 -- 172.21.15.18:0/1293928950 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6aa8003a30 con 0x55dfea7c78a0 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6ab9d93740 10 client.25945 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6ab9d93740 10 client.25945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6ab9d93740 20 client.25945 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfea7e1070 0x55dfea7e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6ab9d93740 4 client.25945 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1293928950 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfea73a1a0 con 0x55dfea7e1070 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfea7e1070 0x55dfea7e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfea7e1070 0x55dfea7e3460 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed082903b1934b9 server_cookie=6c7be0ea3fe87929 in_seq=0 out_seq=0 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.395+0000 7f6a9a000640 10 client.25945 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.396+0000 7f6a9a000640 1 -- 172.21.15.18:0/1293928950 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfea73a1a0 con 0x55dfea7e1070 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.396+0000 7f6a9a000640 10 client.25945 handle_command_reply: tid=0 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.396+0000 7f6ab9d93740 1 client.25945 shutdown 2024-04-16T17:05:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1293928950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 msgr2=0x55dfea7cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7cdcb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009250 tx=0x7f6aa80047b0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1293928950 shutdown_connections 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfea7e1070 0x55dfea7e3460 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 0x55dfea7ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6a9a000640 0 client.25945 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1293928950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7cdcb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1293928950 >> 172.21.15.18:0/1293928950 conn(0x55dfea7625d0 msgr2=0x55dfea7c42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1293928950 shutdown_connections 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1293928950 wait complete. 2024-04-16T17:05:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.397+0000 7f6ab9d93740 20 client.25945 trim_cache size 0 max 16384 2024-04-16T17:05:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.414+0000 7f6ab9d93740 1 Processor -- start 2024-04-16T17:05:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab9d93740 1 -- start start 2024-04-16T17:05:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 0x55dfea7e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01afcf0 con 0x55dfea7c8610 2024-04-16T17:05:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ad5c0 con 0x55dfea7d0e20 2024-04-16T17:05:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ab430 con 0x55dfea7c78a0 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40144/0 (socket says 172.21.15.18:40144) 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab5000640 1 -- 172.21.15.18:0/3497389953 learned_addr learned my addr 172.21.15.18:0/3497389953 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6ab5000640 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.415+0000 7f6aaf400640 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 0x55dfea7e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6ab5000640 1 -- 172.21.15.18:0/3497389953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 msgr2=0x55dfea7d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6ab5000640 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6ab5000640 1 -- 172.21.15.18:0/3497389953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 msgr2=0x55dfea7e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6ab5000640 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 0x55dfea7e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6ab5000640 1 -- 172.21.15.18:0/3497389953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfea7c6b70 con 0x55dfea7d0e20 2024-04-16T17:05:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6ab5000640 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7e9090 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8005d70 tx=0x7f6aa8004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5efe07c273682dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6aa4c00640 1 -- 172.21.15.18:0/3497389953 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8003970 con 0x55dfea7d0e20 2024-04-16T17:05:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6aa4c00640 1 -- 172.21.15.18:0/3497389953 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa8003b10 con 0x55dfea7d0e20 2024-04-16T17:05:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.416+0000 7f6aa4c00640 1 -- 172.21.15.18:0/3497389953 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa802c9d0 con 0x55dfea7d0e20 2024-04-16T17:05:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 -- 172.21.15.18:0/3497389953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 msgr2=0x55dfea7e9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7e9090 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8005d70 tx=0x7f6aa8004820 comp rx=0 tx=0).stop 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 -- 172.21.15.18:0/3497389953 shutdown_connections 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7d0e20 0x55dfea7e9090 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7c8610 0x55dfea7e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/3497389953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c78a0 0x55dfea7d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 -- 172.21.15.18:0/3497389953 >> 172.21.15.18:0/3497389953 conn(0x55dfea7625d0 msgr2=0x55dfea7c37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 -- 172.21.15.18:0/3497389953 shutdown_connections 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.417+0000 7f6ab9d93740 1 -- 172.21.15.18:0/3497389953 wait complete. 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.418+0000 7f6ab9d93740 1 -- start start 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.418+0000 7f6ab9d93740 1 Processor -- start 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.418+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c78a0 0x55dfea7d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.418+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c8610 0x55dfea7d9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab9d93740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01aba80 con 0x55dfea7d0e20 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01acef0 con 0x55dfea7c78a0 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab9d93740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ac150 con 0x55dfea7c8610 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab9d93740 10 client.0 _ll_register_callbacks cb 0x7f6aa00f6680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c78a0 0x55dfea7d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c78a0 0x55dfea7d8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51650/0 (socket says 172.21.15.18:51650) 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab5000640 1 -- 172.21.15.18:0/1599027459 learned_addr learned my addr 172.21.15.18:0/1599027459 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c8610 0x55dfea7d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6aaf400640 1 -- 172.21.15.18:0/1599027459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c78a0 msgr2=0x55dfea7d8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c78a0 0x55dfea7d8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6aaf400640 1 -- 172.21.15.18:0/1599027459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 msgr2=0x55dfea7d6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.419+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6aaf400640 1 -- 172.21.15.18:0/1599027459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfea7d6d10 con 0x55dfea7c8610 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6aaf400640 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c8610 0x55dfea7d9470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55dfea7c3720 tx=0x7f6a9c014900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7adef5cd9c7359b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6a9a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6a9a000640 1 -- 172.21.15.18:0/1599027459 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c01e640 con 0x55dfea7c8610 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6a9a000640 1 -- 172.21.15.18:0/1599027459 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f6a9c01e7e0 con 0x55dfea7c8610 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1599027459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa8003b10 con 0x55dfea7c8610 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6a9a000640 1 -- 172.21.15.18:0/1599027459 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c012450 con 0x55dfea7c8610 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6ab9d93740 10 client.25951 fetch_fsmap learned FSMap version 25 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.420+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1599027459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfea847050 con 0x55dfea7c8610 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6a9a000640 1 -- 172.21.15.18:0/1599027459 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa8003b10 con 0x55dfea7c8610 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6a9a000640 1 -- 172.21.15.18:0/1599027459 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6a9c026660 con 0x55dfea7c8610 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6ab9d93740 10 client.25951 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6ab9d93740 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:05:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6ab9d93740 20 client.25951 populate_metadata read hostname 'smithi018' 2024-04-16T17:05:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfea847820 0x55dfea849c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6ab9d93740 4 client.25951 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:05:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1599027459 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfea734010 con 0x55dfea847820 2024-04-16T17:05:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.421+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfea847820 0x55dfea849c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.422+0000 7f6ab5000640 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfea847820 0x55dfea849c40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdca0133db05589c server_cookie=7b28a2c0e509955d in_seq=0 out_seq=0 2024-04-16T17:05:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.422+0000 7f6a9a000640 10 client.25951 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:50.498 INFO:tasks.workunit.client.0.smithi018.stdout: 1 1564 8.70 MB/sec warmup 3 sec latency 73.412 ms 2024-04-16T17:05:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6a9a000640 1 -- 172.21.15.18:0/1599027459 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfea734010 con 0x55dfea847820 2024-04-16T17:05:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6a9a000640 10 client.25951 handle_command_reply: tid=0 2024-04-16T17:05:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 client.25951 shutdown 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1599027459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c8610 msgr2=0x55dfea7d9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c8610 0x55dfea7d9470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55dfea7c3720 tx=0x7f6a9c014900 comp rx=0 tx=0).stop 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1599027459 shutdown_connections 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfea847820 0x55dfea849c40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfea7d0e20 0x55dfea7d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfea7c8610 0x55dfea7d9470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 --2- 172.21.15.18:0/1599027459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfea7c78a0 0x55dfea7d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1599027459 >> 172.21.15.18:0/1599027459 conn(0x55dfea7625d0 msgr2=0x55dfea7c4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.500+0000 7f6a9a000640 0 client.25951 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.501+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1599027459 shutdown_connections 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.501+0000 7f6ab9d93740 1 -- 172.21.15.18:0/1599027459 wait complete. 2024-04-16T17:05:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.501+0000 7f6ab9d93740 20 client.25951 trim_cache size 0 max 16384 2024-04-16T17:05:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:05:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:05:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:05:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:05:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 -- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6a940a3ec0 msgr2=0x7f6a940a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6a940a3ec0 0x7f6a940a6380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00effd0 tx=0x7f6aa00f67b0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 -- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ab0104470 msgr2=0x7f6ab01a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ab0104470 0x7f6ab01a07a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6aa002c240 tx=0x7f6aa00042f0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 -- 172.21.15.18:0/3074241038 shutdown_connections 2024-04-16T17:05:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6a940a3ec0 0x7f6a940a6380 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ab019a9a0 0x7f6ab019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab0109540 0x7f6ab01a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 --2- 172.21.15.18:0/3074241038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ab0104470 0x7f6ab01a07a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 -- 172.21.15.18:0/3074241038 >> 172.21.15.18:0/3074241038 conn(0x7f6ab00ffcd0 msgr2=0x7f6ab0102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 -- 172.21.15.18:0/3074241038 shutdown_connections 2024-04-16T17:05:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:05:50.502+0000 7f6ab7000640 1 -- 172.21.15.18:0/3074241038 wait complete. 2024-04-16T17:05:51.498 INFO:tasks.workunit.client.0.smithi018.stdout: 1 3167 8.53 MB/sec warmup 4 sec latency 10.542 ms 2024-04-16T17:05:51.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:51 smithi043 ceph-mon[36901]: pgmap v36: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 72 KiB/s wr, 18 op/s 2024-04-16T17:05:51.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:51 smithi022 ceph-mon[36023]: pgmap v36: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 72 KiB/s wr, 18 op/s 2024-04-16T17:05:52.498 INFO:tasks.workunit.client.0.smithi018.stdout: 1 3985 8.01 MB/sec warmup 5 sec latency 10.369 ms 2024-04-16T17:05:53.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 4627 7.00 MB/sec warmup 6 sec latency 62.027 ms 2024-04-16T17:05:53.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:53 smithi043 ceph-mon[36901]: pgmap v37: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 79 KiB/s wr, 18 op/s 2024-04-16T17:05:53.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:53 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:05:53.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:53 smithi022 ceph-mon[36023]: pgmap v37: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 79 KiB/s wr, 18 op/s 2024-04-16T17:05:53.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:53 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:05:54.498 INFO:tasks.workunit.client.0.smithi018.stdout: 1 6037 6.83 MB/sec warmup 7 sec latency 14.841 ms 2024-04-16T17:05:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:54 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:05:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:54 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:05:54.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:54 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:05:54.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:54 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:05:54.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:54 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:05:54.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:54 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:05:55.498 INFO:tasks.workunit.client.0.smithi018.stdout: 1 7336 7.36 MB/sec warmup 8 sec latency 19.111 ms 2024-04-16T17:05:55.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:55 smithi043 ceph-mon[36901]: pgmap v38: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-04-16T17:05:55.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:55 smithi022 ceph-mon[36023]: pgmap v38: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-04-16T17:05:56.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 8009 6.66 MB/sec warmup 9 sec latency 26.999 ms 2024-04-16T17:05:57.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 9189 6.49 MB/sec warmup 10 sec latency 11.055 ms 2024-04-16T17:05:57.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:57 smithi043 ceph-mon[36901]: pgmap v39: 129 pgs: 129 active+clean; 16 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-04-16T17:05:57.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:57 smithi022 ceph-mon[36023]: pgmap v39: 129 pgs: 129 active+clean; 16 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-04-16T17:05:58.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:49 smithi018 ceph-mon[31776]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:05:58.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:51 smithi018 ceph-mon[31776]: pgmap v36: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 72 KiB/s wr, 18 op/s 2024-04-16T17:05:58.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:53 smithi018 ceph-mon[31776]: pgmap v37: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 79 KiB/s wr, 18 op/s 2024-04-16T17:05:58.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:53 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:05:58.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:54 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:05:58.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:54 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:05:58.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:54 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:05:58.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:55 smithi018 ceph-mon[31776]: pgmap v38: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-04-16T17:05:58.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:57 smithi018 ceph-mon[31776]: pgmap v39: 129 pgs: 129 active+clean; 16 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-04-16T17:05:58.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 9487 6.02 MB/sec warmup 11 sec latency 474.533 ms 2024-04-16T17:05:59.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 10792 6.42 MB/sec warmup 12 sec latency 77.060 ms 2024-04-16T17:05:59.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:05:59 smithi043 ceph-mon[36901]: pgmap v40: 129 pgs: 129 active+clean; 16 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-16T17:05:59.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:05:59 smithi022 ceph-mon[36023]: pgmap v40: 129 pgs: 129 active+clean; 16 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-16T17:06:00.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 11314 6.03 MB/sec warmup 13 sec latency 17.636 ms 2024-04-16T17:06:01.163 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:05:59 smithi018 ceph-mon[31776]: pgmap v40: 129 pgs: 129 active+clean; 16 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-16T17:06:01.417 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:01 smithi018 ceph-mon[31776]: pgmap v41: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-04-16T17:06:01.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 11683 5.73 MB/sec warmup 14 sec latency 53.154 ms 2024-04-16T17:06:01.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:01 smithi043 ceph-mon[36901]: pgmap v41: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-04-16T17:06:01.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:01 smithi022 ceph-mon[36023]: pgmap v41: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-04-16T17:06:02.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:06:02.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:06:02.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 -- 172.21.15.18:0/3981368103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 msgr2=0x7fe56810eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 --2- 172.21.15.18:0/3981368103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 0x7fe56810eed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe560009ca0 tx=0x7fe56002c100 comp rx=0 tx=0).stop 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 -- 172.21.15.18:0/3981368103 shutdown_connections 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 --2- 172.21.15.18:0/3981368103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 0x7fe56810eed0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 --2- 172.21.15.18:0/3981368103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe568102be0 0x7fe568106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 --2- 172.21.15.18:0/3981368103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe568102230 0x7fe568102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 -- 172.21.15.18:0/3981368103 >> 172.21.15.18:0/3981368103 conn(0x7fe5680fdab0 msgr2=0x7fe5680ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 -- 172.21.15.18:0/3981368103 shutdown_connections 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.241+0000 7fe570000640 1 -- 172.21.15.18:0/3981368103 wait complete. 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.242+0000 7fe570000640 1 Processor -- start 2024-04-16T17:06:02.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe570000640 1 -- start start 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe570000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe568102230 0x7fe56819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe570000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 0x7fe56819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe570000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe568198720 0x7fe568198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe570000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681123c0 con 0x7fe568198720 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe570000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe568112240 con 0x7fe568107300 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe570000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe568112540 con 0x7fe568102230 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe56cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 0x7fe56819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe56cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 0x7fe56819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48466/0 (socket says 172.21.15.18:48466) 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe56cc00640 1 -- 172.21.15.18:0/2479609360 learned_addr learned my addr 172.21.15.18:0/2479609360 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.243+0000 7fe56d600640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe568102230 0x7fe56819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.244+0000 7fe56e000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe568198720 0x7fe568198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.244+0000 7fe56cc00640 1 -- 172.21.15.18:0/2479609360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe568102230 msgr2=0x7fe56819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.244+0000 7fe56cc00640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe568102230 0x7fe56819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.244+0000 7fe56cc00640 1 -- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe568198720 msgr2=0x7fe568198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.244+0000 7fe56cc00640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe568198720 0x7fe568198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.244+0000 7fe56cc00640 1 -- 172.21.15.18:0/2479609360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe568199320 con 0x7fe568107300 2024-04-16T17:06:02.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.244+0000 7fe56e000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe568198720 0x7fe568198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:02.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.244+0000 7fe56cc00640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 0x7fe56819eb50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe55400e9c0 tx=0x7fe55400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:02.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.245+0000 7fe566000640 1 -- 172.21.15.18:0/2479609360 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55400cd80 con 0x7fe568107300 2024-04-16T17:06:02.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.245+0000 7fe566000640 1 -- 172.21.15.18:0/2479609360 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe554005500 con 0x7fe568107300 2024-04-16T17:06:02.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.245+0000 7fe566000640 1 -- 172.21.15.18:0/2479609360 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554010690 con 0x7fe568107300 2024-04-16T17:06:02.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.245+0000 7fe570000640 1 -- 172.21.15.18:0/2479609360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe568074230 con 0x7fe568107300 2024-04-16T17:06:02.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.245+0000 7fe570000640 1 -- 172.21.15.18:0/2479609360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe568074730 con 0x7fe568107300 2024-04-16T17:06:02.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe572e35740 1 Processor -- start 2024-04-16T17:06:02.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe566000640 1 -- 172.21.15.18:0/2479609360 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe5540046c0 con 0x7fe568107300 2024-04-16T17:06:02.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe572e35740 1 -- start start 2024-04-16T17:06:02.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe572e35740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c0198590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe572e35740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe572e35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01a16b0 0x5643c01a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe572e35740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5643c00fe120 con 0x5643c0198ad0 2024-04-16T17:06:02.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe572e35740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5643c00a4340 con 0x5643c01981b0 2024-04-16T17:06:02.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.250+0000 7fe572e35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643c00fa0e0 con 0x5643c01a16b0 2024-04-16T17:06:02.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.251+0000 7fe56e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.251+0000 7fe56cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01a16b0 0x5643c01a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.251+0000 7fe56e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c0198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48472/0 (socket says 172.21.15.18:48472) 2024-04-16T17:06:02.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.251+0000 7fe56e000640 1 -- 172.21.15.18:0/3985234729 learned_addr learned my addr 172.21.15.18:0/3985234729 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:02.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe56e000640 1 -- 172.21.15.18:0/3985234729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01a16b0 msgr2=0x5643c01a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:02.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe56e000640 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01a16b0 0x5643c01a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe56e000640 1 -- 172.21.15.18:0/3985234729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 msgr2=0x5643c01a1170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:06:02.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe56d600640 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe566000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5500a4020 0x7fe5500a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe56e000640 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe56e000640 1 -- 172.21.15.18:0/3985234729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643bfeda7d0 con 0x5643c01981b0 2024-04-16T17:06:02.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe566000640 1 -- 172.21.15.18:0/2479609360 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe5540d1170 con 0x7fe568107300 2024-04-16T17:06:02.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.252+0000 7fe56cc00640 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01a16b0 0x5643c01a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:02.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.253+0000 7fe56d600640 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:02.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.253+0000 7fe56e000640 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c0198590 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe56002c710 tx=0x7fe5600323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbabef4d3b458fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:02.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.253+0000 7fe55a600640 1 -- 172.21.15.18:0/3985234729 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56004a070 con 0x5643c01981b0 2024-04-16T17:06:02.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.253+0000 7fe56cc00640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5500a4020 0x7fe5500a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.253+0000 7fe55a600640 1 -- 172.21.15.18:0/3985234729 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5600098c0 con 0x5643c01981b0 2024-04-16T17:06:02.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.253+0000 7fe55a600640 1 -- 172.21.15.18:0/3985234729 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560004040 con 0x5643c01981b0 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.254+0000 7fe56cc00640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5500a4020 0x7fe5500a64e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe5540ce330 tx=0x7fe5540d4c40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.254+0000 7fe572e35740 1 -- 172.21.15.18:0/3985234729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 msgr2=0x5643c0198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.254+0000 7fe572e35740 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c0198590 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe56002c710 tx=0x7fe5600323d0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.255+0000 7fe572e35740 1 -- 172.21.15.18:0/3985234729 shutdown_connections 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.255+0000 7fe572e35740 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01a16b0 0x5643c01a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.255+0000 7fe572e35740 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.255+0000 7fe572e35740 1 --2- 172.21.15.18:0/3985234729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c0198590 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.255+0000 7fe572e35740 1 -- 172.21.15.18:0/3985234729 >> 172.21.15.18:0/3985234729 conn(0x5643c01928e0 msgr2=0x5643c0192cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:02.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.255+0000 7fe572e35740 1 -- 172.21.15.18:0/3985234729 shutdown_connections 2024-04-16T17:06:02.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.255+0000 7fe572e35740 1 -- 172.21.15.18:0/3985234729 wait complete. 2024-04-16T17:06:02.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.256+0000 7fe572e35740 1 -- start start 2024-04-16T17:06:02.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.256+0000 7fe572e35740 1 Processor -- start 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe572e35740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c019e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe572e35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c0198ad0 0x5643c019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe572e35740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01a16b0 0x5643c019f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe572e35740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5643c01b5450 con 0x5643c01a16b0 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe572e35740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5643c00b7710 con 0x5643c01981b0 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe572e35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643c01b55d0 con 0x5643c0198ad0 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe572e35740 10 client.0 _ll_register_callbacks cb 0x5643c00b2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe56e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe56e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48482/0 (socket says 172.21.15.18:48482) 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe56e000640 1 -- 172.21.15.18:0/2351067452 learned_addr learned my addr 172.21.15.18:0/2351067452 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:02.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.257+0000 7fe56e000640 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01a16b0 0x5643c019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.258+0000 7fe56e000640 1 -- 172.21.15.18:0/2351067452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c0198ad0 msgr2=0x5643c019ecc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:02.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.258+0000 7fe56e000640 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c0198ad0 0x5643c019ecc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.258+0000 7fe56e000640 1 -- 172.21.15.18:0/2351067452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01a16b0 msgr2=0x5643c019f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:02.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.258+0000 7fe56e000640 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01a16b0 0x5643c019f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.258+0000 7fe56e000640 1 -- 172.21.15.18:0/2351067452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643bfe103f0 con 0x5643c01981b0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.258+0000 7fe56e000640 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c019e780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe560009620 tx=0x7fe560005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd47f415a8b6f1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.258+0000 7fe54ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.259+0000 7fe54ea00640 1 -- 172.21.15.18:0/2351067452 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56004a070 con 0x5643c01981b0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.259+0000 7fe54ea00640 1 -- 172.21.15.18:0/2351067452 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe5600041e0 con 0x5643c01981b0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.259+0000 7fe572e35740 1 -- 172.21.15.18:0/2351067452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5600098c0 con 0x5643c01981b0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.259+0000 7fe54ea00640 1 -- 172.21.15.18:0/2351067452 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe560039070 con 0x5643c01981b0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.259+0000 7fe572e35740 10 client.25964 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.259+0000 7fe572e35740 1 -- 172.21.15.18:0/2351067452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643bfdd00f0 con 0x5643c01981b0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.259+0000 7fe54ea00640 1 -- 172.21.15.18:0/2351067452 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5600098c0 con 0x5643c01981b0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.260+0000 7fe54ea00640 1 -- 172.21.15.18:0/2351067452 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe560034820 con 0x5643c01981b0 2024-04-16T17:06:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.260+0000 7fe572e35740 10 client.25964 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.260+0000 7fe572e35740 10 client.25964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.260+0000 7fe572e35740 20 client.25964 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.260+0000 7fe572e35740 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5643c01cb080 0x5643c01cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.260+0000 7fe572e35740 4 client.25964 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.260+0000 7fe572e35740 1 -- 172.21.15.18:0/2351067452 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c010bb40 con 0x5643c01cb080 2024-04-16T17:06:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.260+0000 7fe56d600640 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5643c01cb080 0x5643c01cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.261+0000 7fe54ea00640 10 client.25964 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.262+0000 7fe56d600640 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5643c01cb080 0x5643c01cd470 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417f3f6b0c3d5fb6 server_cookie=bd2e8696fe54fbe4 in_seq=0 out_seq=0 2024-04-16T17:06:02.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe54ea00640 1 -- 172.21.15.18:0/2351067452 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643c010bb40 con 0x5643c01cb080 2024-04-16T17:06:02.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe54ea00640 10 client.25964 handle_command_reply: tid=0 2024-04-16T17:06:02.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 client.25964 shutdown 2024-04-16T17:06:02.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 -- 172.21.15.18:0/2351067452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 msgr2=0x5643c019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c019e780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe560009620 tx=0x7fe560005ef0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 -- 172.21.15.18:0/2351067452 shutdown_connections 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5643c01cb080 0x5643c01cd470 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe54ea00640 0 client.25964 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01a16b0 0x5643c019f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c0198ad0 0x5643c019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 --2- 172.21.15.18:0/2351067452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c019e780 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 -- 172.21.15.18:0/2351067452 >> 172.21.15.18:0/2351067452 conn(0x5643c01928e0 msgr2=0x5643c0194630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 -- 172.21.15.18:0/2351067452 shutdown_connections 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.263+0000 7fe572e35740 1 -- 172.21.15.18:0/2351067452 wait complete. 2024-04-16T17:06:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.264+0000 7fe572e35740 20 client.25964 trim_cache size 0 max 16384 2024-04-16T17:06:02.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.285+0000 7fe572e35740 1 Processor -- start 2024-04-16T17:06:02.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.285+0000 7fe572e35740 1 -- start start 2024-04-16T17:06:02.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.285+0000 7fe572e35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.285+0000 7fe572e35740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c01c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.285+0000 7fe572e35740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.285+0000 7fe572e35740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681a7530 con 0x5643c0198ad0 2024-04-16T17:06:02.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.285+0000 7fe572e35740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe568073c80 con 0x5643c01981b0 2024-04-16T17:06:02.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.285+0000 7fe572e35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe568071ad0 con 0x5643c01928e0 2024-04-16T17:06:02.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c01c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c01c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48490/0 (socket says 172.21.15.18:48490) 2024-04-16T17:06:02.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56d600640 1 -- 172.21.15.18:0/652619959 learned_addr learned my addr 172.21.15.18:0/652619959 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56e000640 1 --2- 172.21.15.18:0/652619959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56e000640 1 -- 172.21.15.18:0/652619959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 msgr2=0x5643c01c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56e000640 1 --2- 172.21.15.18:0/652619959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c01c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56e000640 1 -- 172.21.15.18:0/652619959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 msgr2=0x5643c01d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56e000640 1 --2- 172.21.15.18:0/652619959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01d0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56e000640 1 -- 172.21.15.18:0/652619959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c0192410 con 0x5643c01928e0 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.286+0000 7fe56e000640 1 --2- 172.21.15.18:0/652619959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01a0eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe560009bf0 tx=0x7fe560032af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a701534ff6e00e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.287+0000 7fe559c00640 1 -- 172.21.15.18:0/652619959 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56004a040 con 0x5643c01928e0 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.287+0000 7fe559c00640 1 -- 172.21.15.18:0/652619959 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56004b040 con 0x5643c01928e0 2024-04-16T17:06:02.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.287+0000 7fe559c00640 1 -- 172.21.15.18:0/652619959 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560058020 con 0x5643c01928e0 2024-04-16T17:06:02.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 -- 172.21.15.18:0/652619959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 msgr2=0x5643c01a0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 --2- 172.21.15.18:0/652619959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01a0eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe560009bf0 tx=0x7fe560032af0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 -- 172.21.15.18:0/652619959 shutdown_connections 2024-04-16T17:06:02.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 --2- 172.21.15.18:0/652619959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c0198ad0 0x5643c01d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 --2- 172.21.15.18:0/652619959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c01981b0 0x5643c01c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 --2- 172.21.15.18:0/652619959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01a0eb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 -- 172.21.15.18:0/652619959 >> 172.21.15.18:0/652619959 conn(0x5643c019a150 msgr2=0x5643c01c60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:02.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 -- 172.21.15.18:0/652619959 shutdown_connections 2024-04-16T17:06:02.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.288+0000 7fe572e35740 1 -- 172.21.15.18:0/652619959 wait complete. 2024-04-16T17:06:02.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.289+0000 7fe572e35740 1 -- start start 2024-04-16T17:06:02.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.289+0000 7fe572e35740 1 Processor -- start 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe572e35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe572e35740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01981b0 0x5643c01ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe572e35740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c0198ad0 0x5643c01ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe572e35740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe568072140 con 0x5643c01981b0 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe572e35740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680735b0 con 0x5643c0198ad0 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe572e35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe568072810 con 0x5643c01928e0 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe572e35740 10 client.0 _ll_register_callbacks cb 0x7fe560048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c0198ad0 0x5643c01ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01981b0 0x5643c01ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c0198ad0 0x5643c01ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48494/0 (socket says 172.21.15.18:48494) 2024-04-16T17:06:02.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 -- 172.21.15.18:0/1364723836 learned_addr learned my addr 172.21.15.18:0/1364723836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:02.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 -- 172.21.15.18:0/1364723836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 msgr2=0x5643c01cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:02.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 -- 172.21.15.18:0/1364723836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01981b0 msgr2=0x5643c01ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:02.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01981b0 0x5643c01ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.290+0000 7fe56e000640 1 -- 172.21.15.18:0/1364723836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c0230290 con 0x5643c0198ad0 2024-04-16T17:06:02.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.291+0000 7fe56e000640 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c0198ad0 0x5643c01ce970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe560034f10 tx=0x7fe560003b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0d5dd050def3b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:02.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.291+0000 7fe54ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.291+0000 7fe54ea00640 1 -- 172.21.15.18:0/1364723836 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560058040 con 0x5643c0198ad0 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.291+0000 7fe54ea00640 1 -- 172.21.15.18:0/1364723836 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe560002030 con 0x5643c0198ad0 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.291+0000 7fe572e35740 1 -- 172.21.15.18:0/1364723836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56004b040 con 0x5643c0198ad0 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.291+0000 7fe54ea00640 1 -- 172.21.15.18:0/1364723836 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe56004a040 con 0x5643c0198ad0 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.291+0000 7fe572e35740 10 client.25970 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.291+0000 7fe572e35740 1 -- 172.21.15.18:0/1364723836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643c0231560 con 0x5643c0198ad0 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe54ea00640 1 -- 172.21.15.18:0/1364723836 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56004b040 con 0x5643c0198ad0 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe54ea00640 1 -- 172.21.15.18:0/1364723836 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe560004100 con 0x5643c0198ad0 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe572e35740 10 client.25970 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe572e35740 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe572e35740 20 client.25970 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe572e35740 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5643c01a16b0 0x5643c0233a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe572e35740 4 client.25970 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:06:02.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe572e35740 1 -- 172.21.15.18:0/1364723836 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643c01059b0 con 0x5643c01a16b0 2024-04-16T17:06:02.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.292+0000 7fe56d600640 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5643c01a16b0 0x5643c0233a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.293+0000 7fe56d600640 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5643c01a16b0 0x5643c0233a70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27125a132f666dde server_cookie=c69f9a4d961b9f21 in_seq=0 out_seq=0 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.293+0000 7fe54ea00640 10 client.25970 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.293+0000 7fe54ea00640 1 -- 172.21.15.18:0/1364723836 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5643c01059b0 con 0x5643c01a16b0 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.293+0000 7fe54ea00640 10 client.25970 handle_command_reply: tid=0 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.293+0000 7fe572e35740 1 client.25970 shutdown 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.293+0000 7fe572e35740 1 -- 172.21.15.18:0/1364723836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c0198ad0 msgr2=0x5643c01ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.293+0000 7fe572e35740 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c0198ad0 0x5643c01ce970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe560034f10 tx=0x7fe560003b30 comp rx=0 tx=0).stop 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 1 -- 172.21.15.18:0/1364723836 shutdown_connections 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5643c01a16b0 0x5643c0233a70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643c0198ad0 0x5643c01ce970 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5643c01981b0 0x5643c01ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe54ea00640 0 client.25970 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 1 --2- 172.21.15.18:0/1364723836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c01928e0 0x5643c01cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 1 -- 172.21.15.18:0/1364723836 >> 172.21.15.18:0/1364723836 conn(0x5643c019a150 msgr2=0x5643c019efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:02.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 1 -- 172.21.15.18:0/1364723836 shutdown_connections 2024-04-16T17:06:02.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 1 -- 172.21.15.18:0/1364723836 wait complete. 2024-04-16T17:06:02.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.294+0000 7fe572e35740 20 client.25970 trim_cache size 0 max 16384 2024-04-16T17:06:02.297 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:06:02.297 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:06:02.297 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:06:02.297 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 -- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5500a4020 msgr2=0x7fe5500a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5500a4020 0x7fe5500a64e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe5540ce330 tx=0x7fe5540d4c40 comp rx=0 tx=0).stop 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 -- 172.21.15.18:0/2479609360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 msgr2=0x7fe56819eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 0x7fe56819eb50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe55400e9c0 tx=0x7fe55400ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 -- 172.21.15.18:0/2479609360 shutdown_connections 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5500a4020 0x7fe5500a64e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe568198720 0x7fe568198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe568107300 0x7fe56819eb50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 --2- 172.21.15.18:0/2479609360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe568102230 0x7fe56819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 -- 172.21.15.18:0/2479609360 >> 172.21.15.18:0/2479609360 conn(0x7fe5680fdab0 msgr2=0x7fe5680ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 -- 172.21.15.18:0/2479609360 shutdown_connections 2024-04-16T17:06:02.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.295+0000 7fe570000640 1 -- 172.21.15.18:0/2479609360 wait complete. 2024-04-16T17:06:02.321 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:06:02.322 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:06:02.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 13012 5.68 MB/sec warmup 15 sec latency 45.051 ms 2024-04-16T17:06:02.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 -- 172.21.15.18:0/3004710234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcacc0743a0 msgr2=0x7fcacc074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 --2- 172.21.15.18:0/3004710234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcacc0743a0 0x7fcacc074780 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009900 tx=0x7fcabc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 -- 172.21.15.18:0/3004710234 shutdown_connections 2024-04-16T17:06:02.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 --2- 172.21.15.18:0/3004710234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcacc112ed0 0x7fcacc1152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 --2- 172.21.15.18:0/3004710234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcacc074cc0 0x7fcacc112990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 --2- 172.21.15.18:0/3004710234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcacc0743a0 0x7fcacc074780 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 -- 172.21.15.18:0/3004710234 >> 172.21.15.18:0/3004710234 conn(0x7fcacc0fd9f0 msgr2=0x7fcacc0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:02.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 -- 172.21.15.18:0/3004710234 shutdown_connections 2024-04-16T17:06:02.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.534+0000 7fcad1600640 1 -- 172.21.15.18:0/3004710234 wait complete. 2024-04-16T17:06:02.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.535+0000 7fcad1600640 1 Processor -- start 2024-04-16T17:06:02.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.535+0000 7fcad1600640 1 -- start start 2024-04-16T17:06:02.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcad1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcacc0743a0 0x7fcacc19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcad1600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcacc074cc0 0x7fcacc19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcad1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcacc112ed0 0x7fcacc1a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcad1600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc1188a0 con 0x7fcacc074cc0 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcad1600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc118720 con 0x7fcacc0743a0 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcad1600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc118a20 con 0x7fcacc112ed0 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcacbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcacc0743a0 0x7fcacc19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcacbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcacc0743a0 0x7fcacc19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48526/0 (socket says 172.21.15.18:48526) 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcacbe00640 1 -- 172.21.15.18:0/279610650 learned_addr learned my addr 172.21.15.18:0/279610650 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcad0c00640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcacc112ed0 0x7fcacc1a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.536+0000 7fcacb400640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcacc074cc0 0x7fcacc19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcacbe00640 1 -- 172.21.15.18:0/279610650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcacc112ed0 msgr2=0x7fcacc1a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcacbe00640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcacc112ed0 0x7fcacc1a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcacbe00640 1 -- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcacc074cc0 msgr2=0x7fcacc19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcacbe00640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcacc074cc0 0x7fcacc19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcacbe00640 1 -- 172.21.15.18:0/279610650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc1a3de0 con 0x7fcacc0743a0 2024-04-16T17:06:02.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcacbe00640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcacc0743a0 0x7fcacc19ee40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcabc005940 tx=0x7fcabc02cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:02.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcac8c00640 1 -- 172.21.15.18:0/279610650 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc041070 con 0x7fcacc0743a0 2024-04-16T17:06:02.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcac8c00640 1 -- 172.21.15.18:0/279610650 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc004450 con 0x7fcacc0743a0 2024-04-16T17:06:02.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc1a4070 con 0x7fcacc0743a0 2024-04-16T17:06:02.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.537+0000 7fcac8c00640 1 -- 172.21.15.18:0/279610650 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc003d80 con 0x7fcacc0743a0 2024-04-16T17:06:02.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.538+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc1ab960 con 0x7fcacc0743a0 2024-04-16T17:06:02.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.539+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcacc04fe00 con 0x7fcacc0743a0 2024-04-16T17:06:02.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.542+0000 7fcac8c00640 1 -- 172.21.15.18:0/279610650 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fcabc044070 con 0x7fcacc0743a0 2024-04-16T17:06:02.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.543+0000 7fcac8c00640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcab00a3f40 0x7fcab00a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:02.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.544+0000 7fcac8c00640 1 -- 172.21.15.18:0/279610650 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fcabc0f27c0 con 0x7fcacc0743a0 2024-04-16T17:06:02.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.544+0000 7fcacb400640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcab00a3f40 0x7fcab00a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:02.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.545+0000 7fcacb400640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcab00a3f40 0x7fcab00a6400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcacc105cc0 tx=0x7fcab8005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:02.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.548+0000 7fcac8c00640 1 -- 172.21.15.18:0/279610650 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcabc0be330 con 0x7fcacc0743a0 2024-04-16T17:06:02.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.827+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcacc105360 con 0x7fcacc0743a0 2024-04-16T17:06:02.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.828+0000 7fcac8c00640 1 -- 172.21.15.18:0/279610650 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7fcabc0c2630 con 0x7fcacc0743a0 2024-04-16T17:06:02.829 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:06:02.829 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:06:02.829 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:06:02.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcab00a3f40 msgr2=0x7fcab00a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcab00a3f40 0x7fcab00a6400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcacc105cc0 tx=0x7fcab8005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:02.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcacc0743a0 msgr2=0x7fcacc19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:02.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcacc0743a0 0x7fcacc19ee40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcabc005940 tx=0x7fcabc02cbd0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 shutdown_connections 2024-04-16T17:06:02.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcab00a3f40 0x7fcab00a6400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcacc112ed0 0x7fcacc1a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcacc074cc0 0x7fcacc19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 --2- 172.21.15.18:0/279610650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcacc0743a0 0x7fcacc19ee40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:02.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 >> 172.21.15.18:0/279610650 conn(0x7fcacc0fd9f0 msgr2=0x7fcacc0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:02.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 shutdown_connections 2024-04-16T17:06:02.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:02.830+0000 7fcad1600640 1 -- 172.21.15.18:0/279610650 wait complete. 2024-04-16T17:06:02.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:06:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- 172.21.15.18:0/3782356704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798102230 msgr2=0x7ff798102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 --2- 172.21.15.18:0/3782356704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798102230 0x7ff798102610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009860 tx=0x7ff78c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- 172.21.15.18:0/3782356704 shutdown_connections 2024-04-16T17:06:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 --2- 172.21.15.18:0/3782356704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798107300 0x7ff79810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 --2- 172.21.15.18:0/3782356704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff798102be0 0x7ff798106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 --2- 172.21.15.18:0/3782356704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798102230 0x7ff798102610 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- 172.21.15.18:0/3782356704 >> 172.21.15.18:0/3782356704 conn(0x7ff7980fdab0 msgr2=0x7ff7980ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- 172.21.15.18:0/3782356704 shutdown_connections 2024-04-16T17:06:03.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- 172.21.15.18:0/3782356704 wait complete. 2024-04-16T17:06:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 Processor -- start 2024-04-16T17:06:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- start start 2024-04-16T17:06:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798102230 0x7ff7981a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798107300 0x7ff7981a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79819a9a0 0x7ff79819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981123c0 con 0x7ff79819a9a0 2024-04-16T17:06:03.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff798112240 con 0x7ff798102230 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff7a0000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff798112540 con 0x7ff798107300 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79e000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79819a9a0 0x7ff79819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798102230 0x7ff7981a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798102230 0x7ff7981a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48554/0 (socket says 172.21.15.18:48554) 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79d600640 1 -- 172.21.15.18:0/1440543585 learned_addr learned my addr 172.21.15.18:0/1440543585 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798107300 0x7ff7981a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79e000640 1 -- 172.21.15.18:0/1440543585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798107300 msgr2=0x7ff7981a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79e000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798107300 0x7ff7981a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79e000640 1 -- 172.21.15.18:0/1440543585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798102230 msgr2=0x7ff7981a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79e000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798102230 0x7ff7981a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79e000640 1 -- 172.21.15.18:0/1440543585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79819b4c0 con 0x7ff79819a9a0 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79d600640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798102230 0x7ff7981a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:03.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff79e000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79819a9a0 0x7ff79819ae50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff790009fd0 tx=0x7ff79000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.028+0000 7ff796000640 1 -- 172.21.15.18:0/1440543585 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79000ce20 con 0x7ff79819a9a0 2024-04-16T17:06:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a0000640 1 -- 172.21.15.18:0/1440543585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79819b7b0 con 0x7ff79819a9a0 2024-04-16T17:06:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff796000640 1 -- 172.21.15.18:0/1440543585 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff790005020 con 0x7ff79819a9a0 2024-04-16T17:06:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff796000640 1 -- 172.21.15.18:0/1440543585 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900106c0 con 0x7ff79819a9a0 2024-04-16T17:06:03.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a0000640 1 -- 172.21.15.18:0/1440543585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7981a7a00 con 0x7ff79819a9a0 2024-04-16T17:06:03.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a2d70740 1 Processor -- start 2024-04-16T17:06:03.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff796000640 1 -- 172.21.15.18:0/1440543585 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff790005540 con 0x7ff79819a9a0 2024-04-16T17:06:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a2d70740 1 -- start start 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e032130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e03ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e03b250 0x556a2e03d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556a2df97020 con 0x556a2e03b250 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556a2df3d460 con 0x556a2e032670 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a2df92fe0 con 0x556a2e031d50 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.034+0000 7ff79cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e032130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e03ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e032130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48904/0 (socket says 172.21.15.18:48904) 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79cc00640 1 -- 172.21.15.18:0/3272687219 learned_addr learned my addr 172.21.15.18:0/3272687219 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79cc00640 1 --2- 172.21.15.18:0/3272687219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e03b250 0x556a2e03d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79d600640 1 -- 172.21.15.18:0/3272687219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 msgr2=0x556a2e032130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79d600640 1 --2- 172.21.15.18:0/3272687219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e032130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79d600640 1 -- 172.21.15.18:0/3272687219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e03b250 msgr2=0x556a2e03d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79d600640 1 --2- 172.21.15.18:0/3272687219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e03b250 0x556a2e03d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79d600640 1 -- 172.21.15.18:0/3272687219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a2df45040 con 0x556a2e032670 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79d600640 1 --2- 172.21.15.18:0/3272687219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e03ad10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff79819bae0 tx=0x7ff78c0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb86c5696c35a7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff78a600640 1 -- 172.21.15.18:0/3272687219 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c039040 con 0x556a2e032670 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff796000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7800a3ed0 0x7ff7800a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff78a600640 1 -- 172.21.15.18:0/3272687219 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78c004030 con 0x556a2e032670 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff78a600640 1 -- 172.21.15.18:0/3272687219 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c02c8c0 con 0x556a2e032670 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff796000640 1 -- 172.21.15.18:0/1440543585 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff7900d1440 con 0x7ff79819a9a0 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7800a3ed0 0x7ff7800a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 -- 172.21.15.18:0/3272687219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 msgr2=0x556a2e03ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/3272687219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e03ad10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff79819bae0 tx=0x7ff78c0056d0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 -- 172.21.15.18:0/3272687219 shutdown_connections 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/3272687219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e03b250 0x556a2e03d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/3272687219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e03ad10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/3272687219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e032130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 -- 172.21.15.18:0/3272687219 >> 172.21.15.18:0/3272687219 conn(0x556a2e02c480 msgr2=0x556a2e02c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 -- 172.21.15.18:0/3272687219 shutdown_connections 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7800a3ed0 0x7ff7800a6390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x556a2e02d610 tx=0x7ff784009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.039+0000 7ff7a2d70740 1 -- 172.21.15.18:0/3272687219 wait complete. 2024-04-16T17:06:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 1 -- start start 2024-04-16T17:06:03.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 1 Processor -- start 2024-04-16T17:06:03.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e031d50 0x556a2e0381d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e038710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e03b250 0x556a2e038c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556a2e03dfe0 con 0x556a2e031d50 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556a2df50830 con 0x556a2e032670 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a2e03e160 con 0x556a2e03b250 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.042+0000 7ff7a2d70740 10 client.0 _ll_register_callbacks cb 0x556a2df58150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e031d50 0x556a2e0381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e038710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e031d50 0x556a2e0381d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52260/0 (socket says 172.21.15.18:52260) 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79d600640 1 -- 172.21.15.18:0/1145985281 learned_addr learned my addr 172.21.15.18:0/1145985281 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79e000640 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e03b250 0x556a2e038c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79cc00640 1 -- 172.21.15.18:0/1145985281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e03b250 msgr2=0x556a2e038c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e03b250 0x556a2e038c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79cc00640 1 -- 172.21.15.18:0/1145985281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e031d50 msgr2=0x556a2e0381d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e031d50 0x556a2e0381d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79cc00640 1 -- 172.21.15.18:0/1145985281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a2df54920 con 0x556a2e032670 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.043+0000 7ff79d600640 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e031d50 0x556a2e0381d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.044+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e038710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff784009820 tx=0x7ff784004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a040c89795f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.044+0000 7ff77ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.044+0000 7ff77ea00640 1 -- 172.21.15.18:0/1145985281 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78400fe70 con 0x556a2e032670 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.044+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1145985281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78c004030 con 0x556a2e032670 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.044+0000 7ff77ea00640 1 -- 172.21.15.18:0/1145985281 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff78401a050 con 0x556a2e032670 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.044+0000 7ff77ea00640 1 -- 172.21.15.18:0/1145985281 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff784036050 con 0x556a2e032670 2024-04-16T17:06:03.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.044+0000 7ff7a2d70740 10 client.25988 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:03.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.044+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1145985281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a2df227f0 con 0x556a2e032670 2024-04-16T17:06:03.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.045+0000 7ff77ea00640 1 -- 172.21.15.18:0/1145985281 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78c004030 con 0x556a2e032670 2024-04-16T17:06:03.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.045+0000 7ff77ea00640 1 -- 172.21.15.18:0/1145985281 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff7840035f0 con 0x556a2e032670 2024-04-16T17:06:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.045+0000 7ff7a2d70740 10 client.25988 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.045+0000 7ff7a2d70740 10 client.25988 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:06:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.045+0000 7ff7a2d70740 20 client.25988 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.045+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556a2e04b4b0 0x556a2e04d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.045+0000 7ff7a2d70740 4 client.25988 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.045+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1145985281 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a2dfa4a40 con 0x556a2e04b4b0 2024-04-16T17:06:03.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.049+0000 7ff79d600640 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556a2e04b4b0 0x556a2e04d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.050+0000 7ff79d600640 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556a2e04b4b0 0x556a2e04d920 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d07806912e117f3d server_cookie=8f9a55557c49ca0f in_seq=0 out_seq=0 2024-04-16T17:06:03.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.050+0000 7ff77ea00640 10 client.25988 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:06:03.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff77ea00640 1 -- 172.21.15.18:0/1145985281 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a2dfa4a40 con 0x556a2e04b4b0 2024-04-16T17:06:03.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff77ea00640 10 client.25988 handle_command_reply: tid=0 2024-04-16T17:06:03.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 client.25988 shutdown 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1145985281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 msgr2=0x556a2e038710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e038710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff784009820 tx=0x7ff784004730 comp rx=0 tx=0).stop 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1145985281 shutdown_connections 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556a2e04b4b0 0x556a2e04d920 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e03b250 0x556a2e038c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e032670 0x556a2e038710 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1145985281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e031d50 0x556a2e0381d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1145985281 >> 172.21.15.18:0/1145985281 conn(0x556a2e02c480 msgr2=0x556a2e02e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff77ea00640 0 client.25988 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:06:03.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.051+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1145985281 shutdown_connections 2024-04-16T17:06:03.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.052+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1145985281 wait complete. 2024-04-16T17:06:03.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.052+0000 7ff7a2d70740 20 client.25988 trim_cache size 0 max 16384 2024-04-16T17:06:03.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff7a2d70740 1 Processor -- start 2024-04-16T17:06:03.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff7a2d70740 1 -- start start 2024-04-16T17:06:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e03a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e046ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 0x556a2e0531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b3fb0 con 0x556a2e032670 2024-04-16T17:06:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b1880 con 0x556a2e03b250 2024-04-16T17:06:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981af6d0 con 0x556a2e031d50 2024-04-16T17:06:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff79cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e046ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff79cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e046ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52264/0 (socket says 172.21.15.18:52264) 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff79cc00640 1 -- 172.21.15.18:0/1308982066 learned_addr learned my addr 172.21.15.18:0/1308982066 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff79d600640 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e03a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff79cc00640 1 -- 172.21.15.18:0/1308982066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 msgr2=0x556a2e03a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e03a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.068+0000 7ff79cc00640 1 -- 172.21.15.18:0/1308982066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 msgr2=0x556a2e0531f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 0x556a2e0531f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff79cc00640 1 -- 172.21.15.18:0/1308982066 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a2e0324a0 con 0x556a2e032670 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff79d600640 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e03a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e046ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff784006010 tx=0x7ff784005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3726b074bee7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff789c00640 1 -- 172.21.15.18:0/1308982066 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78403f070 con 0x556a2e032670 2024-04-16T17:06:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff789c00640 1 -- 172.21.15.18:0/1308982066 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78400fe70 con 0x556a2e032670 2024-04-16T17:06:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff789c00640 1 -- 172.21.15.18:0/1308982066 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784003c80 con 0x556a2e032670 2024-04-16T17:06:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1308982066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 msgr2=0x556a2e046ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e046ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff784006010 tx=0x7ff784005f30 comp rx=0 tx=0).stop 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1308982066 shutdown_connections 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 0x556a2e0531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e046ea0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1308982066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e03a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.070+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1308982066 >> 172.21.15.18:0/1308982066 conn(0x556a2e02c480 msgr2=0x556a2e033530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.071+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1308982066 shutdown_connections 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.071+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1308982066 wait complete. 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.071+0000 7ff7a2d70740 1 -- start start 2024-04-16T17:06:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.071+0000 7ff7a2d70740 1 Processor -- start 2024-04-16T17:06:03.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e043840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e0356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff7a2d70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 0x556a2e035cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981afd40 con 0x556a2e032670 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b11b0 con 0x556a2e03b250 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff7a2d70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b0410 con 0x556a2e031d50 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff7a2d70740 10 client.0 _ll_register_callbacks cb 0x7ff784011e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e0356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e0356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52276/0 (socket says 172.21.15.18:52276) 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79cc00640 1 -- 172.21.15.18:0/1990722388 learned_addr learned my addr 172.21.15.18:0/1990722388 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79e000640 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 0x556a2e035cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79cc00640 1 -- 172.21.15.18:0/1990722388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 msgr2=0x556a2e043840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e043840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79cc00640 1 -- 172.21.15.18:0/1990722388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 msgr2=0x556a2e035cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 0x556a2e035cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.072+0000 7ff79cc00640 1 -- 172.21.15.18:0/1990722388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a2dca93f0 con 0x556a2e032670 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff79cc00640 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e0356e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff784009530 tx=0x7ff784022a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb6cf9ff717a8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff77ea00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:06:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff77ea00640 1 -- 172.21.15.18:0/1990722388 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78403f040 con 0x556a2e032670 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1990722388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78400fe70 con 0x556a2e032670 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff77ea00640 1 -- 172.21.15.18:0/1990722388 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff784003c00 con 0x556a2e032670 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff77ea00640 1 -- 172.21.15.18:0/1990722388 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff78404b050 con 0x556a2e032670 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 10 client.15840 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1990722388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a2dd737d0 con 0x556a2e032670 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff77ea00640 1 -- 172.21.15.18:0/1990722388 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78400fe70 con 0x556a2e032670 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff77ea00640 1 -- 172.21.15.18:0/1990722388 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff7840454d0 con 0x556a2e032670 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 10 client.15840 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 20 client.15840 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556a2e0b1080 0x556a2e0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 4 client.15840 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:06:03.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.074+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1990722388 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a2df9e8b0 con 0x556a2e0b1080 2024-04-16T17:06:03.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.089+0000 7ff79d600640 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556a2e0b1080 0x556a2e0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.096+0000 7ff79d600640 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556a2e0b1080 0x556a2e0b3470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5057a3c8126080b server_cookie=ba4af4ffb50c9e48 in_seq=0 out_seq=0 2024-04-16T17:06:03.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.096+0000 7ff77ea00640 10 client.15840 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:06:03.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.096+0000 7ff77ea00640 1 -- 172.21.15.18:0/1990722388 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a2df9e8b0 con 0x556a2e0b1080 2024-04-16T17:06:03.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.096+0000 7ff77ea00640 10 client.15840 handle_command_reply: tid=0 2024-04-16T17:06:03.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.096+0000 7ff7a2d70740 1 client.15840 shutdown 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.096+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1990722388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 msgr2=0x556a2e0356e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.096+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e0356e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff784009530 tx=0x7ff784022a90 comp rx=0 tx=0).stop 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1990722388 shutdown_connections 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556a2e0b1080 0x556a2e0b3470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a2e03b250 0x556a2e035cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556a2e032670 0x556a2e0356e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 1 --2- 172.21.15.18:0/1990722388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a2e031d50 0x556a2e043840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1990722388 >> 172.21.15.18:0/1990722388 conn(0x556a2e02c480 msgr2=0x556a2e0369e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff77ea00640 0 client.15840 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1990722388 shutdown_connections 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 1 -- 172.21.15.18:0/1990722388 wait complete. 2024-04-16T17:06:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.097+0000 7ff7a2d70740 20 client.15840 trim_cache size 0 max 16384 2024-04-16T17:06:03.100 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 -- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7800a3ed0 msgr2=0x7ff7800a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7800a3ed0 0x7ff7800a6390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x556a2e02d610 tx=0x7ff784009260 comp rx=0 tx=0).stop 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 -- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79819a9a0 msgr2=0x7ff79819ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79819a9a0 0x7ff79819ae50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff790009fd0 tx=0x7ff79000ef10 comp rx=0 tx=0).stop 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 -- 172.21.15.18:0/1440543585 shutdown_connections 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7800a3ed0 0x7ff7800a6390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79819a9a0 0x7ff79819ae50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798107300 0x7ff7981a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 --2- 172.21.15.18:0/1440543585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798102230 0x7ff7981a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 -- 172.21.15.18:0/1440543585 >> 172.21.15.18:0/1440543585 conn(0x7ff7980fdab0 msgr2=0x7ff7980ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 -- 172.21.15.18:0/1440543585 shutdown_connections 2024-04-16T17:06:03.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.098+0000 7ff7a0000640 1 -- 172.21.15.18:0/1440543585 wait complete. 2024-04-16T17:06:03.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:06:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 -- 172.21.15.18:0/3718088829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0102a20 msgr2=0x7f5db0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 --2- 172.21.15.18:0/3718088829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0102a20 0x7f5db0106a60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009860 tx=0x7f5d9c02bb90 comp rx=0 tx=0).stop 2024-04-16T17:06:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 -- 172.21.15.18:0/3718088829 shutdown_connections 2024-04-16T17:06:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 --2- 172.21.15.18:0/3718088829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db0107140 0x7f5db010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 --2- 172.21.15.18:0/3718088829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0102a20 0x7f5db0106a60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 --2- 172.21.15.18:0/3718088829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0102070 0x7f5db0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 -- 172.21.15.18:0/3718088829 >> 172.21.15.18:0/3718088829 conn(0x7f5db00fd950 msgr2=0x7f5db00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 -- 172.21.15.18:0/3718088829 shutdown_connections 2024-04-16T17:06:03.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.300+0000 7f5daf200640 1 -- 172.21.15.18:0/3718088829 wait complete. 2024-04-16T17:06:03.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.301+0000 7f5daf200640 1 Processor -- start 2024-04-16T17:06:03.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.302+0000 7f5daf200640 1 -- start start 2024-04-16T17:06:03.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.302+0000 7f5daf200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db0102070 0x7f5db0074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.302+0000 7f5daf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0107140 0x7f5db0074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.302+0000 7f5daf200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0074d80 0x7f5db01a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.302+0000 7f5daf200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5db0112250 con 0x7f5db0074d80 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.302+0000 7f5daf200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01120d0 con 0x7f5db0102070 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.302+0000 7f5daf200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01123d0 con 0x7f5db0107140 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dae800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0074d80 0x7f5db01a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dae800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0074d80 0x7f5db01a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52282/0 (socket says 172.21.15.18:52282) 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dae800640 1 -- 172.21.15.18:0/3310252044 learned_addr learned my addr 172.21.15.18:0/3310252044 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dad400640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0107140 0x7f5db0074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dade00640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db0102070 0x7f5db0074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dad400640 1 -- 172.21.15.18:0/3310252044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db0102070 msgr2=0x7f5db0074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dad400640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db0102070 0x7f5db0074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dad400640 1 -- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0074d80 msgr2=0x7f5db01a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dad400640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0074d80 0x7f5db01a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dad400640 1 -- 172.21.15.18:0/3310252044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db01a7120 con 0x7f5db0107140 2024-04-16T17:06:03.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dade00640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db0102070 0x7f5db0074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:03.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dae800640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0074d80 0x7f5db01a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:03.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.303+0000 7f5dad400640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0107140 0x7f5db0074840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c02c0d0 tx=0x7f5d9c0323b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.304+0000 7f5da6a00640 1 -- 172.21.15.18:0/3310252044 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c041020 con 0x7f5db0107140 2024-04-16T17:06:03.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.304+0000 7f5da6a00640 1 -- 172.21.15.18:0/3310252044 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c02ca10 con 0x7f5db0107140 2024-04-16T17:06:03.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.304+0000 7f5daf200640 1 -- 172.21.15.18:0/3310252044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db01a73b0 con 0x7f5db0107140 2024-04-16T17:06:03.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.304+0000 7f5da6a00640 1 -- 172.21.15.18:0/3310252044 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c039040 con 0x7f5db0107140 2024-04-16T17:06:03.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.304+0000 7f5daf200640 1 -- 172.21.15.18:0/3310252044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db01a78c0 con 0x7f5db0107140 2024-04-16T17:06:03.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.307+0000 7f5da6a00640 1 -- 172.21.15.18:0/3310252044 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5d9c044020 con 0x7f5db0107140 2024-04-16T17:06:03.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.308+0000 7f5da6a00640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d940a3f80 0x7f5d940a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.309+0000 7f5da6a00640 1 -- 172.21.15.18:0/3310252044 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f5d9c0f2540 con 0x7f5db0107140 2024-04-16T17:06:03.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.309+0000 7f5dade00640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d940a3f80 0x7f5d940a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.309+0000 7f5db765d740 1 Processor -- start 2024-04-16T17:06:03.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.309+0000 7f5db765d740 1 -- start start 2024-04-16T17:06:03.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5db765d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bdb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5db765d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05be3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dade00640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d940a3f80 0x7f5d940a6440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5da0005fd0 tx=0x7f5da0009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5db765d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5db765d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556e05b40020 con 0x556e05bdb830 2024-04-16T17:06:03.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5db765d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556e05ae6460 con 0x556e05be4410 2024-04-16T17:06:03.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5db765d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556e05b3bfe0 con 0x556e05bdaf10 2024-04-16T17:06:03.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05be3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05be3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52288/0 (socket says 172.21.15.18:52288) 2024-04-16T17:06:03.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dad400640 1 -- 172.21.15.18:0/3752656597 learned_addr learned my addr 172.21.15.18:0/3752656597 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dad400640 1 -- 172.21.15.18:0/3752656597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 msgr2=0x556e05bdb2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dad400640 1 --2- 172.21.15.18:0/3752656597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bdb2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dad400640 1 -- 172.21.15.18:0/3752656597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 msgr2=0x556e05be6800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dad400640 1 --2- 172.21.15.18:0/3752656597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be6800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.310+0000 7f5dad400640 1 -- 172.21.15.18:0/3752656597 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e05aee040 con 0x556e05bdb830 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.311+0000 7f5dad400640 1 --2- 172.21.15.18:0/3752656597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05be3ed0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0bdd40 tx=0x7f5d9c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfcb106afa32701 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.311+0000 7f5d9b000640 1 -- 172.21.15.18:0/3752656597 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0be040 con 0x556e05bdb830 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.311+0000 7f5d9b000640 1 -- 172.21.15.18:0/3752656597 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c0c23f0 con 0x556e05bdb830 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.311+0000 7f5d9b000640 1 -- 172.21.15.18:0/3752656597 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0bc340 con 0x556e05bdb830 2024-04-16T17:06:03.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 -- 172.21.15.18:0/3752656597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 msgr2=0x556e05be3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 --2- 172.21.15.18:0/3752656597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05be3ed0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0bdd40 tx=0x7f5d9c005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 -- 172.21.15.18:0/3752656597 shutdown_connections 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 --2- 172.21.15.18:0/3752656597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 --2- 172.21.15.18:0/3752656597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05be3ed0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 --2- 172.21.15.18:0/3752656597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bdb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 -- 172.21.15.18:0/3752656597 >> 172.21.15.18:0/3752656597 conn(0x556e05bd5640 msgr2=0x556e05bd5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 -- 172.21.15.18:0/3752656597 shutdown_connections 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.312+0000 7f5db765d740 1 -- 172.21.15.18:0/3752656597 wait complete. 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.313+0000 7f5db765d740 1 -- start start 2024-04-16T17:06:03.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.313+0000 7f5db765d740 1 Processor -- start 2024-04-16T17:06:03.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.313+0000 7f5db765d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdaf10 0x556e05be1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.313+0000 7f5db765d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdb830 0x556e05be1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.313+0000 7f5db765d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.313+0000 7f5db765d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556e05b40020 con 0x556e05bdaf10 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.313+0000 7f5db765d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556e05af9830 con 0x556e05be4410 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.313+0000 7f5db765d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556e05be6fc0 con 0x556e05bdb830 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5db765d740 10 client.0 _ll_register_callbacks cb 0x556e05b01150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dade00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdaf10 0x556e05be1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dae800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dae800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48606/0 (socket says 172.21.15.18:48606) 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dae800640 1 -- 172.21.15.18:0/3858353806 learned_addr learned my addr 172.21.15.18:0/3858353806 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dad400640 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdb830 0x556e05be1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dae800640 1 -- 172.21.15.18:0/3858353806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdb830 msgr2=0x556e05be1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dae800640 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdb830 0x556e05be1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dae800640 1 -- 172.21.15.18:0/3858353806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdaf10 msgr2=0x556e05be1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dae800640 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdaf10 0x556e05be1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.314+0000 7f5dae800640 1 -- 172.21.15.18:0/3858353806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e05afd920 con 0x556e05be4410 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.315+0000 7f5dae800640 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be1ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5da80099e0 tx=0x7f5da800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb59f71c71a3cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.315+0000 7f5d8f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.315+0000 7f5d8f400640 1 -- 172.21.15.18:0/3858353806 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8017020 con 0x556e05be4410 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.315+0000 7f5d8f400640 1 -- 172.21.15.18:0/3858353806 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f5da8013400 con 0x556e05be4410 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.315+0000 7f5db765d740 1 -- 172.21.15.18:0/3858353806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9c0c23f0 con 0x556e05be4410 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.315+0000 7f5d8f400640 1 -- 172.21.15.18:0/3858353806 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5da8022050 con 0x556e05be4410 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.315+0000 7f5db765d740 10 client.25997 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.315+0000 7f5db765d740 1 -- 172.21.15.18:0/3858353806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556e05acb7f0 con 0x556e05be4410 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5d8f400640 1 -- 172.21.15.18:0/3858353806 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c0c23f0 con 0x556e05be4410 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5d8f400640 1 -- 172.21.15.18:0/3858353806 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5da801b730 con 0x556e05be4410 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5db765d740 10 client.25997 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5db765d740 10 client.25997 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5db765d740 20 client.25997 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:03.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5db765d740 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556e05bf4600 0x556e05bf6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5db765d740 4 client.25997 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:03.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5db765d740 1 -- 172.21.15.18:0/3858353806 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e05b4da40 con 0x556e05bf4600 2024-04-16T17:06:03.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.316+0000 7f5dade00640 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556e05bf4600 0x556e05bf6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.317+0000 7f5dade00640 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556e05bf4600 0x556e05bf6a50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc5b3040dc10a84f server_cookie=961445dd419ef004 in_seq=0 out_seq=0 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.317+0000 7f5d8f400640 10 client.25997 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5d8f400640 1 -- 172.21.15.18:0/3858353806 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e05b4da40 con 0x556e05bf4600 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5d8f400640 10 client.25997 handle_command_reply: tid=0 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 client.25997 shutdown 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 -- 172.21.15.18:0/3858353806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 msgr2=0x556e05be1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be1ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5da80099e0 tx=0x7f5da800c490 comp rx=0 tx=0).stop 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 -- 172.21.15.18:0/3858353806 shutdown_connections 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556e05bf4600 0x556e05bf6a50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05be1ff0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdb830 0x556e05be1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5d8f400640 0 client.25997 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:06:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 --2- 172.21.15.18:0/3858353806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdaf10 0x556e05be1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 -- 172.21.15.18:0/3858353806 >> 172.21.15.18:0/3858353806 conn(0x556e05bd5640 msgr2=0x556e05bd7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 -- 172.21.15.18:0/3858353806 shutdown_connections 2024-04-16T17:06:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 1 -- 172.21.15.18:0/3858353806 wait complete. 2024-04-16T17:06:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.318+0000 7f5db765d740 20 client.25997 trim_cache size 0 max 16384 2024-04-16T17:06:03.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.333+0000 7f5db765d740 1 Processor -- start 2024-04-16T17:06:03.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.333+0000 7f5db765d740 1 -- start start 2024-04-16T17:06:03.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.333+0000 7f5db765d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05bdaf10 0x556e05be3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.333+0000 7f5db765d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05befd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.333+0000 7f5db765d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05be4410 0x556e05bd4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.333+0000 7f5db765d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01b3db0 con 0x556e05bdb830 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.333+0000 7f5db765d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01b1680 con 0x556e05bdaf10 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.333+0000 7f5db765d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01af4d0 con 0x556e05be4410 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05befd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05bdaf10 0x556e05be3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05befd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52314/0 (socket says 172.21.15.18:52314) 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dad400640 1 -- 172.21.15.18:0/2987113021 learned_addr learned my addr 172.21.15.18:0/2987113021 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dae800640 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05be4410 0x556e05bd4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dade00640 1 -- 172.21.15.18:0/2987113021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05be4410 msgr2=0x556e05bd4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dade00640 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05be4410 0x556e05bd4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dade00640 1 -- 172.21.15.18:0/2987113021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 msgr2=0x556e05befd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dade00640 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05befd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dade00640 1 -- 172.21.15.18:0/2987113021 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e058523f0 con 0x556e05bdaf10 2024-04-16T17:06:03.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dae800640 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05be4410 0x556e05bd4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:03.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.334+0000 7f5dad400640 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05befd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:03.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.335+0000 7f5dade00640 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05bdaf10 0x556e05be3ca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x556e05bf7260 tx=0x7f5da0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883d122521f284fc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.335+0000 7f5d9a600640 1 -- 172.21.15.18:0/2987113021 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da002e070 con 0x556e05bdaf10 2024-04-16T17:06:03.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.335+0000 7f5d9a600640 1 -- 172.21.15.18:0/2987113021 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da0028d90 con 0x556e05bdaf10 2024-04-16T17:06:03.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.335+0000 7f5d9a600640 1 -- 172.21.15.18:0/2987113021 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da001fbe0 con 0x556e05bdaf10 2024-04-16T17:06:03.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 -- 172.21.15.18:0/2987113021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05bdaf10 msgr2=0x556e05be3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05bdaf10 0x556e05be3ca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x556e05bf7260 tx=0x7f5da0014000 comp rx=0 tx=0).stop 2024-04-16T17:06:03.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 -- 172.21.15.18:0/2987113021 shutdown_connections 2024-04-16T17:06:03.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05be4410 0x556e05bd4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05befd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 --2- 172.21.15.18:0/2987113021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05bdaf10 0x556e05be3ca0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 -- 172.21.15.18:0/2987113021 >> 172.21.15.18:0/2987113021 conn(0x556e05bd5640 msgr2=0x556e05be1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 -- 172.21.15.18:0/2987113021 shutdown_connections 2024-04-16T17:06:03.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.336+0000 7f5db765d740 1 -- 172.21.15.18:0/2987113021 wait complete. 2024-04-16T17:06:03.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.337+0000 7f5db765d740 1 -- start start 2024-04-16T17:06:03.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.337+0000 7f5db765d740 1 Processor -- start 2024-04-16T17:06:03.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5db765d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bec730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5db765d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05bde060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5db765d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05bde7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5db765d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01afb40 con 0x556e05bdb830 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5db765d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01b0fb0 con 0x556e05be4410 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5db765d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01b0210 con 0x556e05bdaf10 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5db765d740 10 client.0 _ll_register_callbacks cb 0x7f5da8009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5dad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05bde060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5dade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5dade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bec730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48986/0 (socket says 172.21.15.18:48986) 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5dade00640 1 -- 172.21.15.18:0/3528879932 learned_addr learned my addr 172.21.15.18:0/3528879932 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5dad400640 1 -- 172.21.15.18:0/3528879932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 msgr2=0x556e05bec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.338+0000 7f5dad400640 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bec730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5dad400640 1 -- 172.21.15.18:0/3528879932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 msgr2=0x556e05bde7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5dad400640 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05bde7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5dad400640 1 -- 172.21.15.18:0/3528879932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0591c7d0 con 0x556e05bdb830 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5dade00640 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5dad400640 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05bde060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ef930 tx=0x7f5d9c0c23f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b586d5a878538f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5d8f400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5d8f400640 1 -- 172.21.15.18:0/3528879932 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c09ce60 con 0x556e05bdb830 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5d8f400640 1 -- 172.21.15.18:0/3528879932 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f5d9c0bc340 con 0x556e05bdb830 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5db765d740 1 -- 172.21.15.18:0/3528879932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5da0028d90 con 0x556e05bdb830 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5d8f400640 1 -- 172.21.15.18:0/3528879932 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d9c032dd0 con 0x556e05bdb830 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5db765d740 10 client.15858 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.339+0000 7f5db765d740 1 -- 172.21.15.18:0/3528879932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556e05c5aaa0 con 0x556e05bdb830 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.340+0000 7f5d8f400640 1 -- 172.21.15.18:0/3528879932 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da0028d90 con 0x556e05bdb830 2024-04-16T17:06:03.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.340+0000 7f5d8f400640 1 -- 172.21.15.18:0/3528879932 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d9c03c030 con 0x556e05bdb830 2024-04-16T17:06:03.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.340+0000 7f5db765d740 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:03.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.340+0000 7f5db765d740 10 client.15858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:06:03.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.340+0000 7f5db765d740 20 client.15858 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:03.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.340+0000 7f5db765d740 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556e05c5b110 0x556e05c5d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.340+0000 7f5db765d740 4 client.15858 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:06:03.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.340+0000 7f5db765d740 1 -- 172.21.15.18:0/3528879932 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e05b478b0 con 0x556e05c5b110 2024-04-16T17:06:03.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.341+0000 7f5dade00640 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556e05c5b110 0x556e05c5d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.341+0000 7f5dade00640 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556e05c5b110 0x556e05c5d530 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9f45451a7c8cf94 server_cookie=db5ff7c1858913f2 in_seq=0 out_seq=0 2024-04-16T17:06:03.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.341+0000 7f5d8f400640 10 client.15858 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:06:03.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5d8f400640 1 -- 172.21.15.18:0/3528879932 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e05b478b0 con 0x556e05c5b110 2024-04-16T17:06:03.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5d8f400640 10 client.15858 handle_command_reply: tid=0 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 client.15858 shutdown 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 -- 172.21.15.18:0/3528879932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 msgr2=0x556e05bde060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05bde060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ef930 tx=0x7f5d9c0c23f0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 -- 172.21.15.18:0/3528879932 shutdown_connections 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556e05c5b110 0x556e05c5d530 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e05be4410 0x556e05bde7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556e05bdb830 0x556e05bde060 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5d8f400640 0 client.15858 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 --2- 172.21.15.18:0/3528879932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e05bdaf10 0x556e05bec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 -- 172.21.15.18:0/3528879932 >> 172.21.15.18:0/3528879932 conn(0x556e05bd5640 msgr2=0x556e05bdfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 -- 172.21.15.18:0/3528879932 shutdown_connections 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.342+0000 7f5db765d740 1 -- 172.21.15.18:0/3528879932 wait complete. 2024-04-16T17:06:03.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5db765d740 20 client.15858 trim_cache size 0 max 16384 2024-04-16T17:06:03.347 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:06:03.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 -- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d940a3f80 msgr2=0x7f5d940a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d940a3f80 0x7f5d940a6440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5da0005fd0 tx=0x7f5da0009260 comp rx=0 tx=0).stop 2024-04-16T17:06:03.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 -- 172.21.15.18:0/3310252044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0107140 msgr2=0x7f5db0074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0107140 0x7f5db0074840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c02c0d0 tx=0x7f5d9c0323b0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 -- 172.21.15.18:0/3310252044 shutdown_connections 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d940a3f80 0x7f5d940a6440 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0074d80 0x7f5db01a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0107140 0x7f5db0074840 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 --2- 172.21.15.18:0/3310252044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db0102070 0x7f5db0074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 -- 172.21.15.18:0/3310252044 >> 172.21.15.18:0/3310252044 conn(0x7f5db00fd950 msgr2=0x7f5db00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.343+0000 7f5daf200640 1 -- 172.21.15.18:0/3310252044 shutdown_connections 2024-04-16T17:06:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.344+0000 7f5daf200640 1 -- 172.21.15.18:0/3310252044 wait complete. 2024-04-16T17:06:03.362 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:06:03.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 13424 5.58 MB/sec warmup 16 sec latency 47.597 ms 2024-04-16T17:06:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 -- 172.21.15.18:0/869694845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf00743b0 msgr2=0x7f9bf0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/869694845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf00743b0 0x7f9bf0074790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009900 tx=0x7f9be002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 -- 172.21.15.18:0/869694845 shutdown_connections 2024-04-16T17:06:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/869694845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bf010e6c0 0x7f9bf0110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/869694845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf0074cd0 0x7f9bf010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/869694845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf00743b0 0x7f9bf0074790 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 -- 172.21.15.18:0/869694845 >> 172.21.15.18:0/869694845 conn(0x7f9bf00fd950 msgr2=0x7f9bf00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 -- 172.21.15.18:0/869694845 shutdown_connections 2024-04-16T17:06:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.577+0000 7f9bf4e00640 1 -- 172.21.15.18:0/869694845 wait complete. 2024-04-16T17:06:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.578+0000 7f9bf4e00640 1 Processor -- start 2024-04-16T17:06:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.578+0000 7f9bf4e00640 1 -- start start 2024-04-16T17:06:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9bf4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00743b0 0x7f9bf01a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9bf4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf0074cd0 0x7f9bf01a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9bf4e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bf010e6c0 0x7f9bf019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9bf4e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf0114420 con 0x7f9bf010e6c0 2024-04-16T17:06:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9bf4e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01142a0 con 0x7f9bf0074cd0 2024-04-16T17:06:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9bf4e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01145a0 con 0x7f9bf00743b0 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9beea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf0074cd0 0x7f9bf01a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9bef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00743b0 0x7f9bf01a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9befe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bf010e6c0 0x7f9bf019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9beea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf0074cd0 0x7f9bf01a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48642/0 (socket says 172.21.15.18:48642) 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9bef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00743b0 0x7f9bf01a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49006/0 (socket says 172.21.15.18:49006) 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.579+0000 7f9beea00640 1 -- 172.21.15.18:0/2697080428 learned_addr learned my addr 172.21.15.18:0/2697080428 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9beea00640 1 -- 172.21.15.18:0/2697080428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00743b0 msgr2=0x7f9bf01a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9beea00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00743b0 0x7f9bf01a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9beea00640 1 -- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bf010e6c0 msgr2=0x7f9bf019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9beea00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bf010e6c0 0x7f9bf019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9beea00640 1 -- 172.21.15.18:0/2697080428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf019b000 con 0x7f9bf0074cd0 2024-04-16T17:06:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9beea00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf0074cd0 0x7f9bf01a0bc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00e9f0 tx=0x7f9bdc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9be7e00640 1 -- 172.21.15.18:0/2697080428 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00cd80 con 0x7f9bf0074cd0 2024-04-16T17:06:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9be7e00640 1 -- 172.21.15.18:0/2697080428 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bdc005500 con 0x7f9bf0074cd0 2024-04-16T17:06:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9bf4e00640 1 -- 172.21.15.18:0/2697080428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf019b2f0 con 0x7f9bf0074cd0 2024-04-16T17:06:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.580+0000 7f9be7e00640 1 -- 172.21.15.18:0/2697080428 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc010690 con 0x7f9bf0074cd0 2024-04-16T17:06:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.581+0000 7f9bf4e00640 1 -- 172.21.15.18:0/2697080428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf01a7600 con 0x7f9bf0074cd0 2024-04-16T17:06:03.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.584+0000 7f9bf9199740 1 Processor -- start 2024-04-16T17:06:03.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.584+0000 7f9bf9199740 1 -- start start 2024-04-16T17:06:03.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.584+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d49f10 0x556f31d4a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.584+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d52ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.584+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d53410 0x556f31d55800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.584+0000 7f9bf9199740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556f31caf020 con 0x556f31d53410 2024-04-16T17:06:03.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.584+0000 7f9bf9199740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556f31c55460 con 0x556f31d49f10 2024-04-16T17:06:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.584+0000 7f9bf9199740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556f31caafe0 con 0x556f31d4a830 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d52ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9befe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d49f10 0x556f31d4a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d52ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49010/0 (socket says 172.21.15.18:49010) 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 -- 172.21.15.18:0/1935245022 learned_addr learned my addr 172.21.15.18:0/1935245022 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 -- 172.21.15.18:0/1935245022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d49f10 msgr2=0x556f31d4a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 --2- 172.21.15.18:0/1935245022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d49f10 0x556f31d4a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 -- 172.21.15.18:0/1935245022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d53410 msgr2=0x556f31d55800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 --2- 172.21.15.18:0/1935245022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d53410 0x556f31d55800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 -- 172.21.15.18:0/1935245022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f31c5d040 con 0x556f31d4a830 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9be7e00640 1 -- 172.21.15.18:0/2697080428 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9bdc0046c0 con 0x7f9bf0074cd0 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.585+0000 7f9bef400640 1 --2- 172.21.15.18:0/1935245022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d52ed0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000c00 tx=0x7f9be0009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=943c3f9a2df8941c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.586+0000 7f9be4c00640 1 -- 172.21.15.18:0/1935245022 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0043070 con 0x556f31d4a830 2024-04-16T17:06:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.586+0000 7f9be4c00640 1 -- 172.21.15.18:0/1935245022 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be002c6f0 con 0x556f31d4a830 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9be4c00640 1 -- 172.21.15.18:0/1935245022 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0004a00 con 0x556f31d4a830 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 -- 172.21.15.18:0/1935245022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 msgr2=0x556f31d52ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1935245022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d52ed0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000c00 tx=0x7f9be0009560 comp rx=0 tx=0).stop 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 -- 172.21.15.18:0/1935245022 shutdown_connections 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1935245022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d53410 0x556f31d55800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1935245022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d52ed0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1935245022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d49f10 0x556f31d4a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 -- 172.21.15.18:0/1935245022 >> 172.21.15.18:0/1935245022 conn(0x556f31d44640 msgr2=0x556f31d44a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 -- 172.21.15.18:0/1935245022 shutdown_connections 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 -- 172.21.15.18:0/1935245022 wait complete. 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9be7e00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9bd40a3de0 0x7f9bd40a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9be7e00640 1 -- 172.21.15.18:0/2697080428 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9bdc0d1280 con 0x7f9bf0074cd0 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bf9199740 1 -- start start 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.587+0000 7f9bef400640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9bd40a3de0 0x7f9bd40a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bf9199740 1 Processor -- start 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d50400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d50940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 0x556f31d50e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bf9199740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556f31d56240 con 0x556f31d49f10 2024-04-16T17:06:03.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bf9199740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556f31c68830 con 0x556f31d4a830 2024-04-16T17:06:03.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bf9199740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556f31d563c0 con 0x556f31d53410 2024-04-16T17:06:03.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bf9199740 10 client.0 _ll_register_callbacks cb 0x556f31c70150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:03.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9beea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d50940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.588+0000 7f9bef400640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9bd40a3de0 0x7f9bd40a62a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9be0005ab0 tx=0x7f9be002bcd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9beea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d50940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48662/0 (socket says 172.21.15.18:48662) 2024-04-16T17:06:03.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9beea00640 1 -- 172.21.15.18:0/1709761321 learned_addr learned my addr 172.21.15.18:0/1709761321 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9bef400640 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d50400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9befe00640 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 0x556f31d50e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9beea00640 1 -- 172.21.15.18:0/1709761321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 msgr2=0x556f31d50e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9beea00640 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 0x556f31d50e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9beea00640 1 -- 172.21.15.18:0/1709761321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 msgr2=0x556f31d50400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9beea00640 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d50400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9beea00640 1 -- 172.21.15.18:0/1709761321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f31c6c920 con 0x556f31d4a830 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9beea00640 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d50940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00e9c0 tx=0x7f9bdc09c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d4e70f1fca214b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9bd9600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9bd9600640 1 -- 172.21.15.18:0/1709761321 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0b6060 con 0x556f31d4a830 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9bd9600640 1 -- 172.21.15.18:0/1709761321 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9bdc0a5410 con 0x556f31d4a830 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.589+0000 7f9bf9199740 1 -- 172.21.15.18:0/1709761321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be002c6f0 con 0x556f31d4a830 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bd9600640 1 -- 172.21.15.18:0/1709761321 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9bdc087040 con 0x556f31d4a830 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bf9199740 10 client.26012 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:03.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bf9199740 1 -- 172.21.15.18:0/1709761321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556f31c3a7f0 con 0x556f31d4a830 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bd9600640 1 -- 172.21.15.18:0/1709761321 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be002c6f0 con 0x556f31d4a830 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bd9600640 1 -- 172.21.15.18:0/1709761321 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9bdc09c110 con 0x556f31d4a830 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bf9199740 10 client.26012 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bf9199740 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bf9199740 20 client.26012 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556f31d6c030 0x556f31d6e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bf9199740 4 client.26012 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.590+0000 7f9bf9199740 1 -- 172.21.15.18:0/1709761321 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f31cbca40 con 0x556f31d6c030 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.591+0000 7f9bef400640 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556f31d6c030 0x556f31d6e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.591+0000 7f9bd9600640 10 client.26012 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.591+0000 7f9bef400640 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556f31d6c030 0x556f31d6e420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b682486b6cc01ca server_cookie=f007760c33940bd0 in_seq=0 out_seq=0 2024-04-16T17:06:03.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.592+0000 7f9bd9600640 1 -- 172.21.15.18:0/1709761321 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f31cbca40 con 0x556f31d6c030 2024-04-16T17:06:03.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.592+0000 7f9bd9600640 10 client.26012 handle_command_reply: tid=0 2024-04-16T17:06:03.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.592+0000 7f9bf9199740 1 client.26012 shutdown 2024-04-16T17:06:03.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 -- 172.21.15.18:0/1709761321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 msgr2=0x556f31d50940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d50940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00e9c0 tx=0x7f9bdc09c650 comp rx=0 tx=0).stop 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 -- 172.21.15.18:0/1709761321 shutdown_connections 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556f31d6c030 0x556f31d6e420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 0x556f31d50e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d50940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 --2- 172.21.15.18:0/1709761321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d50400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 -- 172.21.15.18:0/1709761321 >> 172.21.15.18:0/1709761321 conn(0x556f31d44640 msgr2=0x556f31d46390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bd9600640 0 client.26012 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 -- 172.21.15.18:0/1709761321 shutdown_connections 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 1 -- 172.21.15.18:0/1709761321 wait complete. 2024-04-16T17:06:03.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.593+0000 7f9bf9199740 20 client.26012 trim_cache size 0 max 16384 2024-04-16T17:06:03.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bf9199740 1 Processor -- start 2024-04-16T17:06:03.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bf9199740 1 -- start start 2024-04-16T17:06:03.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d52b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 0x556f31d43580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bf9199740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01af9d0 con 0x556f31d49f10 2024-04-16T17:06:03.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bf9199740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01ad2a0 con 0x556f31d4a830 2024-04-16T17:06:03.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bf9199740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01ab0f0 con 0x556f31d53410 2024-04-16T17:06:03.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9beea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9bef400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d52b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9beea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d679e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48666/0 (socket says 172.21.15.18:48666) 2024-04-16T17:06:03.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.609+0000 7f9beea00640 1 -- 172.21.15.18:0/916433587 learned_addr learned my addr 172.21.15.18:0/916433587 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9befe00640 1 --2- 172.21.15.18:0/916433587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 0x556f31d43580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9beea00640 1 -- 172.21.15.18:0/916433587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 msgr2=0x556f31d43580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9beea00640 1 --2- 172.21.15.18:0/916433587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 0x556f31d43580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9beea00640 1 -- 172.21.15.18:0/916433587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 msgr2=0x556f31d52b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9beea00640 1 --2- 172.21.15.18:0/916433587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d52b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9beea00640 1 -- 172.21.15.18:0/916433587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f319c13f0 con 0x556f31d4a830 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9beea00640 1 --2- 172.21.15.18:0/916433587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d679e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc09cbc0 tx=0x7f9bdc09c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93037693c4b06520 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9bdbe00640 1 -- 172.21.15.18:0/916433587 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0a0fe0 con 0x556f31d4a830 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.610+0000 7f9bdbe00640 1 -- 172.21.15.18:0/916433587 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bdc0a58a0 con 0x556f31d4a830 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.611+0000 7f9bdbe00640 1 -- 172.21.15.18:0/916433587 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0a5410 con 0x556f31d4a830 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.611+0000 7f9bf9199740 1 -- 172.21.15.18:0/916433587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 msgr2=0x556f31d679e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.611+0000 7f9bf9199740 1 --2- 172.21.15.18:0/916433587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d679e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc09cbc0 tx=0x7f9bdc09c6c0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.611+0000 7f9bf9199740 1 -- 172.21.15.18:0/916433587 shutdown_connections 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.611+0000 7f9bf9199740 1 --2- 172.21.15.18:0/916433587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d53410 0x556f31d43580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.611+0000 7f9bf9199740 1 --2- 172.21.15.18:0/916433587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d4a830 0x556f31d679e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.611+0000 7f9bf9199740 1 --2- 172.21.15.18:0/916433587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d52b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.611+0000 7f9bf9199740 1 -- 172.21.15.18:0/916433587 >> 172.21.15.18:0/916433587 conn(0x556f31d44640 msgr2=0x556f31d42e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.612+0000 7f9bf9199740 1 -- 172.21.15.18:0/916433587 shutdown_connections 2024-04-16T17:06:03.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.612+0000 7f9bf9199740 1 -- 172.21.15.18:0/916433587 wait complete. 2024-04-16T17:06:03.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.612+0000 7f9bf9199740 1 -- start start 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.612+0000 7f9bf9199740 1 Processor -- start 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bf9199740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d53410 0x556f31dd1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bf9199740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01ab760 con 0x556f31d49f10 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bf9199740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01acbd0 con 0x556f31d53410 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bf9199740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf01abe30 con 0x556f31d4a830 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bf9199740 10 client.0 _ll_register_callbacks cb 0x7f9bdc009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bef400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bef400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d4d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52368/0 (socket says 172.21.15.18:52368) 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9bef400640 1 -- 172.21.15.18:0/2190174967 learned_addr learned my addr 172.21.15.18:0/2190174967 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9beea00640 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.613+0000 7f9befe00640 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d53410 0x556f31dd1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bef400640 1 -- 172.21.15.18:0/2190174967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 msgr2=0x556f31d4dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bef400640 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d4dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bef400640 1 -- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d53410 msgr2=0x556f31dd1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bef400640 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d53410 0x556f31dd1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bef400640 1 -- 172.21.15.18:0/2190174967 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f31a8b7d0 con 0x556f31d49f10 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9befe00640 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d53410 0x556f31dd1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bef400640 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d4d780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9be0006010 tx=0x7f9be0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f32fa50fad62cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bd9600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bd9600640 1 -- 172.21.15.18:0/2190174967 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0043050 con 0x556f31d49f10 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bd9600640 1 -- 172.21.15.18:0/2190174967 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9be0004ee0 con 0x556f31d49f10 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.614+0000 7f9bf9199740 1 -- 172.21.15.18:0/2190174967 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bdc0a58a0 con 0x556f31d49f10 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bd9600640 1 -- 172.21.15.18:0/2190174967 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9be006b050 con 0x556f31d49f10 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bf9199740 10 client.15873 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bf9199740 1 -- 172.21.15.18:0/2190174967 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556f31dd2ae0 con 0x556f31d49f10 2024-04-16T17:06:03.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bd9600640 1 -- 172.21.15.18:0/2190174967 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bdc0a58a0 con 0x556f31d49f10 2024-04-16T17:06:03.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bd9600640 1 -- 172.21.15.18:0/2190174967 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9be005b460 con 0x556f31d49f10 2024-04-16T17:06:03.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bf9199740 10 client.15873 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:03.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bf9199740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:03.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bf9199740 20 client.15873 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:03.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bf9199740 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556f31dd2e20 0x556f31dd5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bf9199740 4 client.15873 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:06:03.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.615+0000 7f9bf9199740 1 -- 172.21.15.18:0/2190174967 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f31cb68b0 con 0x556f31dd2e20 2024-04-16T17:06:03.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.616+0000 7f9beea00640 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556f31dd2e20 0x556f31dd5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.616+0000 7f9beea00640 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556f31dd2e20 0x556f31dd5240 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876f4aba196b1e3b server_cookie=65875e2c9e53aec0 in_seq=0 out_seq=0 2024-04-16T17:06:03.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.616+0000 7f9bd9600640 10 client.15873 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:03.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.616+0000 7f9bd9600640 1 -- 172.21.15.18:0/2190174967 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f31cb68b0 con 0x556f31dd2e20 2024-04-16T17:06:03.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.616+0000 7f9bd9600640 10 client.15873 handle_command_reply: tid=0 2024-04-16T17:06:03.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.616+0000 7f9bf9199740 1 client.15873 shutdown 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 -- 172.21.15.18:0/2190174967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 msgr2=0x556f31d4d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d4d780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9be0006010 tx=0x7f9be0009560 comp rx=0 tx=0).stop 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 -- 172.21.15.18:0/2190174967 shutdown_connections 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556f31dd2e20 0x556f31dd5240 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f31d53410 0x556f31dd1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f31d4a830 0x556f31d4dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 --2- 172.21.15.18:0/2190174967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556f31d49f10 0x556f31d4d780 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bd9600640 0 client.15873 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 -- 172.21.15.18:0/2190174967 >> 172.21.15.18:0/2190174967 conn(0x556f31d44640 msgr2=0x556f31d43700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 -- 172.21.15.18:0/2190174967 shutdown_connections 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 1 -- 172.21.15.18:0/2190174967 wait complete. 2024-04-16T17:06:03.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.617+0000 7f9bf9199740 20 client.15873 trim_cache size 0 max 16384 2024-04-16T17:06:03.622 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:06:03.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 -- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9bd40a3de0 msgr2=0x7f9bd40a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9bd40a3de0 0x7f9bd40a62a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9be0005ab0 tx=0x7f9be002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 -- 172.21.15.18:0/2697080428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf0074cd0 msgr2=0x7f9bf01a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf0074cd0 0x7f9bf01a0bc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00e9f0 tx=0x7f9bdc00edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 -- 172.21.15.18:0/2697080428 shutdown_connections 2024-04-16T17:06:03.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9bd40a3de0 0x7f9bd40a62a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bf010e6c0 0x7f9bf019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf0074cd0 0x7f9bf01a0bc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 --2- 172.21.15.18:0/2697080428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00743b0 0x7f9bf01a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 -- 172.21.15.18:0/2697080428 >> 172.21.15.18:0/2697080428 conn(0x7f9bf00fd950 msgr2=0x7f9bf00fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 -- 172.21.15.18:0/2697080428 shutdown_connections 2024-04-16T17:06:03.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:03.618+0000 7f9bf4e00640 1 -- 172.21.15.18:0/2697080428 wait complete. 2024-04-16T17:06:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.637+0000 7f22b9e00640 1 -- 172.21.15.18:0/3663652968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104e20 msgr2=0x7f22b4108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.637+0000 7f22b9e00640 1 --2- 172.21.15.18:0/3663652968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104e20 0x7f22b4108e60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f229c009900 tx=0x7f229c02bc50 comp rx=0 tx=0).stop 2024-04-16T17:06:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.637+0000 7f22b9e00640 1 -- 172.21.15.18:0/3663652968 shutdown_connections 2024-04-16T17:06:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.637+0000 7f22b9e00640 1 --2- 172.21.15.18:0/3663652968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b4109540 0x7f22b4111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.637+0000 7f22b9e00640 1 --2- 172.21.15.18:0/3663652968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104e20 0x7f22b4108e60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.637+0000 7f22b9e00640 1 --2- 172.21.15.18:0/3663652968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22b4104470 0x7f22b4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.637+0000 7f22b9e00640 1 -- 172.21.15.18:0/3663652968 >> 172.21.15.18:0/3663652968 conn(0x7f22b40ffcd0 msgr2=0x7f22b41020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.637+0000 7f22b9e00640 1 -- 172.21.15.18:0/3663652968 shutdown_connections 2024-04-16T17:06:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.638+0000 7f22b9e00640 1 -- 172.21.15.18:0/3663652968 wait complete. 2024-04-16T17:06:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.639+0000 7f22b9e00640 1 Processor -- start 2024-04-16T17:06:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.639+0000 7f22b9e00640 1 -- start start 2024-04-16T17:06:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104470 0x7f22b41a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b4109540 0x7f22b41a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22b419aa00 0x7f22b419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b9e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41146c0 con 0x7f22b4104470 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b9e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4114540 con 0x7f22b419aa00 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b9e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4114840 con 0x7f22b4109540 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104470 0x7f22b41a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104470 0x7f22b41a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52380/0 (socket says 172.21.15.18:52380) 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b2a00640 1 -- 172.21.15.18:0/588428713 learned_addr learned my addr 172.21.15.18:0/588428713 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b3400640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22b419aa00 0x7f22b419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b2a00640 1 -- 172.21.15.18:0/588428713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b4109540 msgr2=0x7f22b41a0d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b2a00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b4109540 0x7f22b41a0d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b2a00640 1 -- 172.21.15.18:0/588428713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22b419aa00 msgr2=0x7f22b419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.640+0000 7f22b2a00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22b419aa00 0x7f22b419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.641+0000 7f22b2a00640 1 -- 172.21.15.18:0/588428713 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b419b520 con 0x7f22b4104470 2024-04-16T17:06:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.641+0000 7f22b2a00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104470 0x7f22b41a0850 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b450 tx=0x7f22a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.641+0000 7f22b8a00640 1 -- 172.21.15.18:0/588428713 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4015020 con 0x7f22b4104470 2024-04-16T17:06:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.641+0000 7f22b8a00640 1 -- 172.21.15.18:0/588428713 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4011d70 con 0x7f22b4104470 2024-04-16T17:06:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.641+0000 7f22b9e00640 1 -- 172.21.15.18:0/588428713 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b419b810 con 0x7f22b4104470 2024-04-16T17:06:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.641+0000 7f22b9e00640 1 -- 172.21.15.18:0/588428713 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b41a7a60 con 0x7f22b4104470 2024-04-16T17:06:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.642+0000 7f22b8a00640 1 -- 172.21.15.18:0/588428713 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a400f340 con 0x7f22b4104470 2024-04-16T17:06:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22bca6f740 1 Processor -- start 2024-04-16T17:06:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22bca6f740 1 -- start start 2024-04-16T17:06:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c7f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c7f8a0 0x563d16c7fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c8b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22bca6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563d16be4780 con 0x563d16c7f8a0 2024-04-16T17:06:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22bca6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563d16b8af40 con 0x563d16c80610 2024-04-16T17:06:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22bca6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d16be0740 con 0x563d16c88e20 2024-04-16T17:06:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22b3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c7f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22b3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c7f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48696/0 (socket says 172.21.15.18:48696) 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.645+0000 7f22b3400640 1 -- 172.21.15.18:0/2217315248 learned_addr learned my addr 172.21.15.18:0/2217315248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22abe00640 1 --2- 172.21.15.18:0/2217315248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c8b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22b3400640 1 -- 172.21.15.18:0/2217315248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 msgr2=0x563d16c8b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22b3400640 1 --2- 172.21.15.18:0/2217315248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c8b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22b3400640 1 -- 172.21.15.18:0/2217315248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c7f8a0 msgr2=0x563d16c7fd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22b3400640 1 --2- 172.21.15.18:0/2217315248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c7f8a0 0x563d16c7fd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22b3400640 1 -- 172.21.15.18:0/2217315248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d168f7210 con 0x563d16c80610 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22b3400640 1 --2- 172.21.15.18:0/2217315248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c7f360 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f22ac010370 tx=0x7f22ac010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db5dea3578028578 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22a3e00640 1 -- 172.21.15.18:0/2217315248 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac016070 con 0x563d16c80610 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22a3e00640 1 -- 172.21.15.18:0/2217315248 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ac004d20 con 0x563d16c80610 2024-04-16T17:06:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.646+0000 7f22a3e00640 1 -- 172.21.15.18:0/2217315248 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac005470 con 0x563d16c80610 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 -- 172.21.15.18:0/2217315248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 msgr2=0x563d16c7f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2217315248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c7f360 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f22ac010370 tx=0x7f22ac010830 comp rx=0 tx=0).stop 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22b8a00640 1 -- 172.21.15.18:0/588428713 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f22a4018020 con 0x7f22b4104470 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 -- 172.21.15.18:0/2217315248 shutdown_connections 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2217315248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c8b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2217315248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c7f8a0 0x563d16c7fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2217315248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c7f360 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 -- 172.21.15.18:0/2217315248 >> 172.21.15.18:0/2217315248 conn(0x563d16c1a5d0 msgr2=0x563d16c7a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 -- 172.21.15.18:0/2217315248 shutdown_connections 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.647+0000 7f22bca6f740 1 -- 172.21.15.18:0/2217315248 wait complete. 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.648+0000 7f22bca6f740 1 -- start start 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.648+0000 7f22bca6f740 1 Processor -- start 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c85ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c86220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c86760 0x563d16c95000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22bca6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563d16c8bb50 con 0x563d16c86760 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22bca6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563d16b9e390 con 0x563d16c80610 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22bca6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d16c8bcd0 con 0x563d16c88e20 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22bca6f740 10 client.0 _ll_register_callbacks cb 0x563d16b87f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22b3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c85ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22b3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c85ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48710/0 (socket says 172.21.15.18:48710) 2024-04-16T17:06:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22b3400640 1 -- 172.21.15.18:0/2279213441 learned_addr learned my addr 172.21.15.18:0/2279213441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22b8a00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f22980a3dd0 0x7f22980a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22abe00640 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c86760 0x563d16c95000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.649+0000 7f22b8a00640 1 -- 172.21.15.18:0/588428713 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f22a40d0820 con 0x7f22b4104470 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22b2a00640 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c86220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22b3400640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f22980a3dd0 0x7f22980a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22abe00640 1 -- 172.21.15.18:0/2279213441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 msgr2=0x563d16c86220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22abe00640 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c86220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22abe00640 1 -- 172.21.15.18:0/2279213441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 msgr2=0x563d16c85ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22abe00640 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c85ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22abe00640 1 -- 172.21.15.18:0/2279213441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d169c3620 con 0x563d16c86760 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22b2a00640 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c86220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22abe00640 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c86760 0x563d16c95000 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f22b4103400 tx=0x7f229c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3f1ebf6afc38b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.650+0000 7f22a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.651+0000 7f22b3400640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f22980a3dd0 0x7f22980a6290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009db0 tx=0x7f22ac009600 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.651+0000 7f22a0c00640 1 -- 172.21.15.18:0/2279213441 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c042070 con 0x563d16c86760 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.651+0000 7f22a0c00640 1 -- 172.21.15.18:0/2279213441 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f229c02ccb0 con 0x563d16c86760 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.651+0000 7f22bca6f740 1 -- 172.21.15.18:0/2279213441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22ac004d20 con 0x563d16c86760 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.651+0000 7f22a0c00640 1 -- 172.21.15.18:0/2279213441 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f229c04d050 con 0x563d16c86760 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.651+0000 7f22bca6f740 10 client.15882 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.651+0000 7f22bca6f740 1 -- 172.21.15.18:0/2279213441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d16c98130 con 0x563d16c86760 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.651+0000 7f22a0c00640 1 -- 172.21.15.18:0/2279213441 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ac004d20 con 0x563d16c86760 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.652+0000 7f22a0c00640 1 -- 172.21.15.18:0/2279213441 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f229c0035f0 con 0x563d16c86760 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.652+0000 7f22bca6f740 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.652+0000 7f22bca6f740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.652+0000 7f22bca6f740 20 client.15882 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.652+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d16c995f0 0x563d16c9b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.652+0000 7f22bca6f740 4 client.15882 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.652+0000 7f22bca6f740 1 -- 172.21.15.18:0/2279213441 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d16bf21a0 con 0x563d16c995f0 2024-04-16T17:06:03.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.652+0000 7f22b2a00640 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d16c995f0 0x563d16c9b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.653+0000 7f22a0c00640 10 client.15882 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.653+0000 7f22b2a00640 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d16c995f0 0x563d16c9b9e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721b9e131de127c8 server_cookie=cc27c6415df63556 in_seq=0 out_seq=0 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22a0c00640 1 -- 172.21.15.18:0/2279213441 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d16bf21a0 con 0x563d16c995f0 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22a0c00640 10 client.15882 handle_command_reply: tid=0 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 client.15882 shutdown 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 -- 172.21.15.18:0/2279213441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c86760 msgr2=0x563d16c95000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c86760 0x563d16c95000 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f22b4103400 tx=0x7f229c004610 comp rx=0 tx=0).stop 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 -- 172.21.15.18:0/2279213441 shutdown_connections 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d16c995f0 0x563d16c9b9e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c86760 0x563d16c95000 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c88e20 0x563d16c86220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2279213441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 -- 172.21.15.18:0/2279213441 >> 172.21.15.18:0/2279213441 conn(0x563d16c1a5d0 msgr2=0x563d16c7c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22a0c00640 0 client.15882 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 -- 172.21.15.18:0/2279213441 shutdown_connections 2024-04-16T17:06:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 1 -- 172.21.15.18:0/2279213441 wait complete. 2024-04-16T17:06:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.654+0000 7f22bca6f740 20 client.15882 trim_cache size 0 max 16384 2024-04-16T17:06:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.670+0000 7f22bca6f740 1 Processor -- start 2024-04-16T17:06:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22bca6f740 1 -- start start 2024-04-16T17:06:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c80610 0x563d16ca1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c86760 0x563d16ca17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c94a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22bca6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41b4100 con 0x563d16c88e20 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22bca6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41b19d0 con 0x563d16c86760 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22bca6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41af820 con 0x563d16c80610 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c86760 0x563d16ca17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22b3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c94a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c86760 0x563d16ca17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48718/0 (socket says 172.21.15.18:48718) 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22abe00640 1 -- 172.21.15.18:0/2418918741 learned_addr learned my addr 172.21.15.18:0/2418918741 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.671+0000 7f22b2a00640 1 --2- 172.21.15.18:0/2418918741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c80610 0x563d16ca1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22abe00640 1 -- 172.21.15.18:0/2418918741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c80610 msgr2=0x563d16ca1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22abe00640 1 --2- 172.21.15.18:0/2418918741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c80610 0x563d16ca1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22abe00640 1 -- 172.21.15.18:0/2418918741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 msgr2=0x563d16c94a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22abe00640 1 --2- 172.21.15.18:0/2418918741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c94a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22abe00640 1 -- 172.21.15.18:0/2418918741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d16c7eb70 con 0x563d16c86760 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22abe00640 1 --2- 172.21.15.18:0/2418918741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c86760 0x563d16ca17c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f229c02c190 tx=0x7f229c009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552bcd9708df109c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22a3400640 1 -- 172.21.15.18:0/2418918741 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c042070 con 0x563d16c86760 2024-04-16T17:06:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22a3400640 1 -- 172.21.15.18:0/2418918741 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c003be0 con 0x563d16c86760 2024-04-16T17:06:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.672+0000 7f22a3400640 1 -- 172.21.15.18:0/2418918741 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c034920 con 0x563d16c86760 2024-04-16T17:06:03.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:03 smithi022 ceph-mon[36023]: pgmap v42: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.4 MiB/s wr, 107 op/s 2024-04-16T17:06:03.675 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:03 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/279610650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 -- 172.21.15.18:0/2418918741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c86760 msgr2=0x563d16ca17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2418918741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c86760 0x563d16ca17c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f229c02c190 tx=0x7f229c009290 comp rx=0 tx=0).stop 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 -- 172.21.15.18:0/2418918741 shutdown_connections 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2418918741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c94a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2418918741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c86760 0x563d16ca17c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 --2- 172.21.15.18:0/2418918741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c80610 0x563d16ca1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 -- 172.21.15.18:0/2418918741 >> 172.21.15.18:0/2418918741 conn(0x563d16c1a5d0 msgr2=0x563d16c7a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 -- 172.21.15.18:0/2418918741 shutdown_connections 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 -- 172.21.15.18:0/2418918741 wait complete. 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.673+0000 7f22bca6f740 1 -- start start 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22bca6f740 1 Processor -- start 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c86760 0x563d16c827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22bca6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c82df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22bca6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41afe90 con 0x563d16c88e20 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22bca6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41b1300 con 0x563d16c80610 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22bca6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41b0560 con 0x563d16c86760 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22bca6f740 10 client.0 _ll_register_callbacks cb 0x7f229c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22b3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c82df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22b2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22b2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c913f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48722/0 (socket says 172.21.15.18:48722) 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22b3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c82df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52394/0 (socket says 172.21.15.18:52394) 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.674+0000 7f22b2a00640 1 -- 172.21.15.18:0/216586416 learned_addr learned my addr 172.21.15.18:0/216586416 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.675+0000 7f22b3400640 1 -- 172.21.15.18:0/216586416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c86760 msgr2=0x563d16c827e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.675+0000 7f22b3400640 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c86760 0x563d16c827e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.675+0000 7f22b3400640 1 -- 172.21.15.18:0/216586416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 msgr2=0x563d16c913f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.675+0000 7f22b3400640 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c913f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.675+0000 7f22b3400640 1 -- 172.21.15.18:0/216586416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d16c83620 con 0x563d16c88e20 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.675+0000 7f22b3400640 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c82df0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009bf0 tx=0x7f22ac010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e9e379630348d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.675+0000 7f22a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:06:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.676+0000 7f22a0c00640 1 -- 172.21.15.18:0/216586416 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac031070 con 0x563d16c88e20 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.676+0000 7f22a0c00640 1 -- 172.21.15.18:0/216586416 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f22ac012510 con 0x563d16c88e20 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.676+0000 7f22bca6f740 1 -- 172.21.15.18:0/216586416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c003be0 con 0x563d16c88e20 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.676+0000 7f22a0c00640 1 -- 172.21.15.18:0/216586416 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22ac024070 con 0x563d16c88e20 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.676+0000 7f22bca6f740 10 client.15888 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.676+0000 7f22bca6f740 1 -- 172.21.15.18:0/216586416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d16cffc00 con 0x563d16c88e20 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.676+0000 7f22a0c00640 1 -- 172.21.15.18:0/216586416 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c003be0 con 0x563d16c88e20 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22a0c00640 1 -- 172.21.15.18:0/216586416 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f22ac02a040 con 0x563d16c88e20 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22bca6f740 10 client.15888 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22bca6f740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22bca6f740 20 client.15888 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22bca6f740 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d16d000c0 0x563d16d024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22bca6f740 4 client.15888 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22bca6f740 1 -- 172.21.15.18:0/216586416 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d16bec010 con 0x563d16d000c0 2024-04-16T17:06:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22b2a00640 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d16d000c0 0x563d16d024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22b2a00640 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d16d000c0 0x563d16d024e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb4c6a6cbe99582 server_cookie=b5320287f7c31a5d in_seq=0 out_seq=0 2024-04-16T17:06:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.677+0000 7f22a0c00640 10 client.15888 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:03.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:03 smithi043 ceph-mon[36901]: pgmap v42: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.4 MiB/s wr, 107 op/s 2024-04-16T17:06:03.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:03 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/279610650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.809+0000 7f22a0c00640 1 -- 172.21.15.18:0/216586416 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d16bec010 con 0x563d16d000c0 2024-04-16T17:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.809+0000 7f22a0c00640 10 client.15888 handle_command_reply: tid=0 2024-04-16T17:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.809+0000 7f22bca6f740 1 client.15888 shutdown 2024-04-16T17:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 -- 172.21.15.18:0/216586416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 msgr2=0x563d16c82df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c82df0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009bf0 tx=0x7f22ac010830 comp rx=0 tx=0).stop 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 -- 172.21.15.18:0/216586416 shutdown_connections 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d16d000c0 0x563d16d024e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d16c88e20 0x563d16c82df0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d16c86760 0x563d16c827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 --2- 172.21.15.18:0/216586416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d16c80610 0x563d16c913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 -- 172.21.15.18:0/216586416 >> 172.21.15.18:0/216586416 conn(0x563d16c1a5d0 msgr2=0x563d16c84530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22a0c00640 0 client.15888 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 -- 172.21.15.18:0/216586416 shutdown_connections 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 1 -- 172.21.15.18:0/216586416 wait complete. 2024-04-16T17:06:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.810+0000 7f22bca6f740 20 client.15888 trim_cache size 0 max 16384 2024-04-16T17:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 -- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f22980a3dd0 msgr2=0x7f22980a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f22980a3dd0 0x7f22980a6290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009db0 tx=0x7f22ac009600 comp rx=0 tx=0).stop 2024-04-16T17:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 -- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104470 msgr2=0x7f22b41a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104470 0x7f22b41a0850 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b450 tx=0x7f22a400b910 comp rx=0 tx=0).stop 2024-04-16T17:06:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 -- 172.21.15.18:0/588428713 shutdown_connections 2024-04-16T17:06:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f22980a3dd0 0x7f22980a6290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22b419aa00 0x7f22b419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b4109540 0x7f22b41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 --2- 172.21.15.18:0/588428713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22b4104470 0x7f22b41a0850 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 -- 172.21.15.18:0/588428713 >> 172.21.15.18:0/588428713 conn(0x7f22b40ffcd0 msgr2=0x7f22b4102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 -- 172.21.15.18:0/588428713 shutdown_connections 2024-04-16T17:06:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:03.811+0000 7f22b9e00640 1 -- 172.21.15.18:0/588428713 wait complete. 2024-04-16T17:06:04.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 13839 5.39 MB/sec warmup 17 sec latency 90.070 ms 2024-04-16T17:06:04.513 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:03 smithi018 ceph-mon[31776]: pgmap v42: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.4 MiB/s wr, 107 op/s 2024-04-16T17:06:04.514 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:03 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/279610650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:04.639 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:06:04.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:04.639 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:06:04.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 -- 172.21.15.18:0/890792687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5c4109800 msgr2=0x7fa5c4115330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 --2- 172.21.15.18:0/890792687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5c4109800 0x7fa5c4115330 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009890 tx=0x7fa5bc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:04.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 -- 172.21.15.18:0/890792687 shutdown_connections 2024-04-16T17:06:04.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 --2- 172.21.15.18:0/890792687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5c4109800 0x7fa5c4115330 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 --2- 172.21.15.18:0/890792687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c4108e40 0x7fa5c41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 --2- 172.21.15.18:0/890792687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c410e5b0 0x7fa5c410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 -- 172.21.15.18:0/890792687 >> 172.21.15.18:0/890792687 conn(0x7fa5c4077a50 msgr2=0x7fa5c4079e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:04.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 -- 172.21.15.18:0/890792687 shutdown_connections 2024-04-16T17:06:04.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.853+0000 7fa5c9200640 1 -- 172.21.15.18:0/890792687 wait complete. 2024-04-16T17:06:04.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.854+0000 7fa5c9200640 1 Processor -- start 2024-04-16T17:06:04.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.854+0000 7fa5c9200640 1 -- start start 2024-04-16T17:06:04.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c4108e40 0x7fa5c41a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5c4109800 0x7fa5c41a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c9200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c410e5b0 0x7fa5c419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c9200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4118800 con 0x7fa5c410e5b0 2024-04-16T17:06:04.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c9200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4118680 con 0x7fa5c4109800 2024-04-16T17:06:04.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c9200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4118980 con 0x7fa5c4108e40 2024-04-16T17:06:04.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c410e5b0 0x7fa5c419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c410e5b0 0x7fa5c419eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52418/0 (socket says 172.21.15.18:52418) 2024-04-16T17:06:04.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c3e00640 1 -- 172.21.15.18:0/4252827508 learned_addr learned my addr 172.21.15.18:0/4252827508 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:04.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5c3400640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c4108e40 0x7fa5c41a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.855+0000 7fa5baa00640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5c4109800 0x7fa5c41a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.856+0000 7fa5c3e00640 1 -- 172.21.15.18:0/4252827508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c4108e40 msgr2=0x7fa5c41a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.856+0000 7fa5c3e00640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c4108e40 0x7fa5c41a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.856+0000 7fa5c3e00640 1 -- 172.21.15.18:0/4252827508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5c4109800 msgr2=0x7fa5c41a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.856+0000 7fa5c3e00640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5c4109800 0x7fa5c41a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.856+0000 7fa5c3e00640 1 -- 172.21.15.18:0/4252827508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c419f220 con 0x7fa5c410e5b0 2024-04-16T17:06:04.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.856+0000 7fa5c3400640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c4108e40 0x7fa5c41a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:04.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.856+0000 7fa5c3e00640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c410e5b0 0x7fa5c419eaa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009860 tx=0x7fa5bc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:04.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.857+0000 7fa5c0c00640 1 -- 172.21.15.18:0/4252827508 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc041020 con 0x7fa5c410e5b0 2024-04-16T17:06:04.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.857+0000 7fa5c0c00640 1 -- 172.21.15.18:0/4252827508 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5bc02c9f0 con 0x7fa5c410e5b0 2024-04-16T17:06:04.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.857+0000 7fa5c9200640 1 -- 172.21.15.18:0/4252827508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419f4b0 con 0x7fa5c410e5b0 2024-04-16T17:06:04.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.857+0000 7fa5c0c00640 1 -- 172.21.15.18:0/4252827508 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc039040 con 0x7fa5c410e5b0 2024-04-16T17:06:04.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.857+0000 7fa5c9200640 1 -- 172.21.15.18:0/4252827508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41ab760 con 0x7fa5c410e5b0 2024-04-16T17:06:04.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.860+0000 7fa5cd5a7740 1 Processor -- start 2024-04-16T17:06:04.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.862+0000 7fa5c0c00640 1 -- 172.21.15.18:0/4252827508 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa5bc044020 con 0x7fa5c410e5b0 2024-04-16T17:06:04.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.863+0000 7fa5cd5a7740 1 -- start start 2024-04-16T17:06:04.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.863+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.863+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.863+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.863+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558768336cb0 con 0x5587683d1e90 2024-04-16T17:06:04.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.863+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587682dd370 con 0x5587683db390 2024-04-16T17:06:04.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.863+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558768332c70 con 0x5587683d27b0 2024-04-16T17:06:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5c3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5baa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5c3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49124/0 (socket says 172.21.15.18:49124) 2024-04-16T17:06:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5c3400640 1 -- 172.21.15.18:0/2905850770 learned_addr learned my addr 172.21.15.18:0/2905850770 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5baa00640 1 --2- 172.21.15.18:0/2905850770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5c0c00640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa5a80a3ed0 0x7fa5a80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5baa00640 1 -- 172.21.15.18:0/2905850770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 msgr2=0x5587683dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5baa00640 1 --2- 172.21.15.18:0/2905850770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5baa00640 1 -- 172.21.15.18:0/2905850770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 msgr2=0x5587683d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:04.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5baa00640 1 --2- 172.21.15.18:0/2905850770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.864+0000 7fa5baa00640 1 -- 172.21.15.18:0/2905850770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587682fc910 con 0x5587683db390 2024-04-16T17:06:04.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.865+0000 7fa5c0c00640 1 -- 172.21.15.18:0/4252827508 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa5bc0f24a0 con 0x7fa5c410e5b0 2024-04-16T17:06:04.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.865+0000 7fa5baa00640 1 --2- 172.21.15.18:0/2905850770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683dd780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5c410db20 tx=0x7fa5ac005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df9196141ffb216 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:04.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.865+0000 7fa5c3e00640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa5a80a3ed0 0x7fa5a80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.865+0000 7fa5b8c00640 1 -- 172.21.15.18:0/2905850770 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac005020 con 0x5587683db390 2024-04-16T17:06:04.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.865+0000 7fa5b8c00640 1 -- 172.21.15.18:0/2905850770 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5ac0047a0 con 0x5587683db390 2024-04-16T17:06:04.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.865+0000 7fa5b8c00640 1 -- 172.21.15.18:0/2905850770 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00fb10 con 0x5587683db390 2024-04-16T17:06:04.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.866+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2905850770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 msgr2=0x5587683dd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.866+0000 7fa5c3e00640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa5a80a3ed0 0x7fa5a80a6390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0ef770 tx=0x7fa5bc02bb70 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:04.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.866+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/2905850770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683dd780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5c410db20 tx=0x7fa5ac005850 comp rx=0 tx=0).stop 2024-04-16T17:06:04.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.866+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2905850770 shutdown_connections 2024-04-16T17:06:04.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.866+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/2905850770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683dd780 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.866+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/2905850770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.866+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/2905850770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.866+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2905850770 >> 172.21.15.18:0/2905850770 conn(0x5587683cc5c0 msgr2=0x5587683cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:04.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.867+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2905850770 shutdown_connections 2024-04-16T17:06:04.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.867+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2905850770 wait complete. 2024-04-16T17:06:04.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.867+0000 7fa5cd5a7740 1 -- start start 2024-04-16T17:06:04.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.868+0000 7fa5cd5a7740 1 Processor -- start 2024-04-16T17:06:04.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.868+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.868+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683d89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.868+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683d8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.868+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5587683de180 con 0x5587683d1e90 2024-04-16T17:06:04.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.868+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587682f0730 con 0x5587683db390 2024-04-16T17:06:04.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.868+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587683de300 con 0x5587683d27b0 2024-04-16T17:06:04.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.868+0000 7fa5cd5a7740 10 client.0 _ll_register_callbacks cb 0x5587682bf1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:04.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683d89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683d89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49138/0 (socket says 172.21.15.18:49138) 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 -- 172.21.15.18:0/157660043 learned_addr learned my addr 172.21.15.18:0/157660043 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5baa00640 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 -- 172.21.15.18:0/157660043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 msgr2=0x5587683d8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683d8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 -- 172.21.15.18:0/157660043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 msgr2=0x5587683d8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683d8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 -- 172.21.15.18:0/157660043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587681117e0 con 0x5587683d27b0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5c3400640 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683d89b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400fe20 tx=0x7fa5b4012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d3b3ddb840060c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.869+0000 7fa5b1600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.870+0000 7fa5b1600640 1 -- 172.21.15.18:0/157660043 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4003970 con 0x5587683d27b0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.870+0000 7fa5b1600640 1 -- 172.21.15.18:0/157660043 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa5b4003b10 con 0x5587683d27b0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.870+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/157660043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5ac0047a0 con 0x5587683d27b0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.870+0000 7fa5b1600640 1 -- 172.21.15.18:0/157660043 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa5b4003d10 con 0x5587683d27b0 2024-04-16T17:06:04.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.870+0000 7fa5cd5a7740 10 client.26041 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:04.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.870+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/157660043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558768171f70 con 0x5587683d27b0 2024-04-16T17:06:04.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.870+0000 7fa5b1600640 1 -- 172.21.15.18:0/157660043 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004530 con 0x5587683d27b0 2024-04-16T17:06:04.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.871+0000 7fa5b1600640 1 -- 172.21.15.18:0/157660043 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5b4010850 con 0x5587683d27b0 2024-04-16T17:06:04.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.871+0000 7fa5cd5a7740 10 client.26041 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.871+0000 7fa5cd5a7740 10 client.26041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.871+0000 7fa5cd5a7740 20 client.26041 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.871+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587683eb860 0x5587683edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.871+0000 7fa5cd5a7740 4 client.26041 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.871+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/157660043 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa5ac0047a0 con 0x5587683eb860 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.871+0000 7fa5baa00640 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587683eb860 0x5587683edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.872+0000 7fa5baa00640 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587683eb860 0x5587683edc70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee47918280299dc6 server_cookie=8c17bcbfa474e919 in_seq=0 out_seq=0 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5b1600640 10 client.26041 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5b1600640 1 -- 172.21.15.18:0/157660043 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fa5ac0047a0 con 0x5587683eb860 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5b1600640 10 client.26041 handle_command_reply: tid=0 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 client.26041 shutdown 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/157660043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 msgr2=0x5587683d89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683d89b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400fe20 tx=0x7fa5b4012820 comp rx=0 tx=0).stop 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/157660043 shutdown_connections 2024-04-16T17:06:04.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587683eb860 0x5587683edc70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683d8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d27b0 0x5587683d89b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/157660043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683d8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/157660043 >> 172.21.15.18:0/157660043 conn(0x5587683cc5c0 msgr2=0x5587683ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5b1600640 0 client.26041 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/157660043 shutdown_connections 2024-04-16T17:06:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/157660043 wait complete. 2024-04-16T17:06:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.873+0000 7fa5cd5a7740 20 client.26041 trim_cache size 0 max 16384 2024-04-16T17:06:04.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.893+0000 7fa5cd5a7740 1 Processor -- start 2024-04-16T17:06:04.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.893+0000 7fa5cd5a7740 1 -- start start 2024-04-16T17:06:04.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d1e90 0x5587683daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d27b0 0x5587683e7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41b81d0 con 0x5587683d27b0 2024-04-16T17:06:04.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41b5aa0 con 0x5587683db390 2024-04-16T17:06:04.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41b38f0 con 0x5587683d1e90 2024-04-16T17:06:04.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d1e90 0x5587683daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d1e90 0x5587683daba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49140/0 (socket says 172.21.15.18:49140) 2024-04-16T17:06:04.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5baa00640 1 -- 172.21.15.18:0/2264877693 learned_addr learned my addr 172.21.15.18:0/2264877693 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:04.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5c3400640 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d27b0 0x5587683e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.894+0000 7fa5baa00640 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5c3400640 1 -- 172.21.15.18:0/2264877693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d1e90 msgr2=0x5587683daba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5c3400640 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d1e90 0x5587683daba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5c3400640 1 -- 172.21.15.18:0/2264877693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 msgr2=0x5587683cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5c3400640 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5c3400640 1 -- 172.21.15.18:0/2264877693 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587683d7420 con 0x5587683d27b0 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5c3400640 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d27b0 0x5587683e7070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400fdf0 tx=0x7fa5b400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f27b57f535d2427 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5b3e00640 1 -- 172.21.15.18:0/2264877693 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40039f0 con 0x5587683d27b0 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5b3e00640 1 -- 172.21.15.18:0/2264877693 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4010910 con 0x5587683d27b0 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.895+0000 7fa5b3e00640 1 -- 172.21.15.18:0/2264877693 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b401ccb0 con 0x5587683d27b0 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2264877693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d27b0 msgr2=0x5587683e7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d27b0 0x5587683e7070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400fdf0 tx=0x7fa5b400f3c0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2264877693 shutdown_connections 2024-04-16T17:06:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683db390 0x5587683cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d27b0 0x5587683e7070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/2264877693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683d1e90 0x5587683daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2264877693 >> 172.21.15.18:0/2264877693 conn(0x5587683cc5c0 msgr2=0x5587683cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2264877693 shutdown_connections 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.896+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/2264877693 wait complete. 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.897+0000 7fa5cd5a7740 1 -- start start 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.897+0000 7fa5cd5a7740 1 Processor -- start 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.897+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.897+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683d27b0 0x5587683df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.897+0000 7fa5cd5a7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683db390 0x558768450df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.897+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41b3f60 con 0x5587683d1e90 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.897+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41b53d0 con 0x5587683d27b0 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.898+0000 7fa5cd5a7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41b4630 con 0x5587683db390 2024-04-16T17:06:04.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.898+0000 7fa5cd5a7740 10 client.0 _ll_register_callbacks cb 0x7fa5b400fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.898+0000 7fa5baa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.898+0000 7fa5baa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52448/0 (socket says 172.21.15.18:52448) 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.898+0000 7fa5baa00640 1 -- 172.21.15.18:0/897597958 learned_addr learned my addr 172.21.15.18:0/897597958 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.898+0000 7fa5baa00640 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683db390 0x558768450df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.898+0000 7fa5c3400640 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683d27b0 0x5587683df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.899+0000 7fa5baa00640 1 -- 172.21.15.18:0/897597958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683db390 msgr2=0x558768450df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.899+0000 7fa5baa00640 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683db390 0x558768450df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.899+0000 7fa5baa00640 1 -- 172.21.15.18:0/897597958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683d27b0 msgr2=0x5587683df890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.899+0000 7fa5baa00640 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683d27b0 0x5587683df890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.899+0000 7fa5baa00640 1 -- 172.21.15.18:0/897597958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558768009170 con 0x5587683d1e90 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.899+0000 7fa5baa00640 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683df350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0115e0 tx=0x7fa5b400f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad835ed26181df85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.899+0000 7fa5b1600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.900+0000 7fa5b1600640 1 -- 172.21.15.18:0/897597958 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac011030 con 0x5587683d1e90 2024-04-16T17:06:04.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.900+0000 7fa5b1600640 1 -- 172.21.15.18:0/897597958 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa5ac0111d0 con 0x5587683d1e90 2024-04-16T17:06:04.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.900+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/897597958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b4010910 con 0x5587683d1e90 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.900+0000 7fa5b1600640 1 -- 172.21.15.18:0/897597958 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa5ac00f790 con 0x5587683d1e90 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.900+0000 7fa5cd5a7740 10 client.15906 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.900+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/897597958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587684632b0 con 0x5587683d1e90 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.901+0000 7fa5b1600640 1 -- 172.21.15.18:0/897597958 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4010910 con 0x5587683d1e90 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.901+0000 7fa5b1600640 1 -- 172.21.15.18:0/897597958 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5ac01d030 con 0x5587683d1e90 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.901+0000 7fa5cd5a7740 10 client.15906 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.901+0000 7fa5cd5a7740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.901+0000 7fa5cd5a7740 20 client.15906 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.901+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558768463890 0x558768465c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.901+0000 7fa5cd5a7740 4 client.15906 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.901+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/897597958 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587683446d0 con 0x558768463890 2024-04-16T17:06:04.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.902+0000 7fa5c3400640 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558768463890 0x558768465c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.902+0000 7fa5c3400640 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558768463890 0x558768465c80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8291eeec6ce4a7 server_cookie=6c31d9f848f4a0e9 in_seq=0 out_seq=0 2024-04-16T17:06:04.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.902+0000 7fa5b1600640 10 client.15906 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:04.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.904+0000 7fa5b1600640 1 -- 172.21.15.18:0/897597958 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587683446d0 con 0x558768463890 2024-04-16T17:06:04.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.904+0000 7fa5b1600640 10 client.15906 handle_command_reply: tid=0 2024-04-16T17:06:04.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.904+0000 7fa5cd5a7740 1 client.15906 shutdown 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.904+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/897597958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 msgr2=0x5587683df350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.904+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683df350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0115e0 tx=0x7fa5b400f9a0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/897597958 shutdown_connections 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558768463890 0x558768465c80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587683db390 0x558768450df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587683d27b0 0x5587683df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 1 --2- 172.21.15.18:0/897597958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587683d1e90 0x5587683df350 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/897597958 >> 172.21.15.18:0/897597958 conn(0x5587683cc5c0 msgr2=0x5587683d8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5b1600640 0 client.15906 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/897597958 shutdown_connections 2024-04-16T17:06:04.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 1 -- 172.21.15.18:0/897597958 wait complete. 2024-04-16T17:06:04.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.905+0000 7fa5cd5a7740 20 client.15906 trim_cache size 0 max 16384 2024-04-16T17:06:04.907 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:06:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:06:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "6a4c33c8-966b-4eaa-a4f3-621751eab67a", 2024-04-16T17:06:04.907 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:06:04.907 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:06:04.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 -- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa5a80a3ed0 msgr2=0x7fa5a80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa5a80a3ed0 0x7fa5a80a6390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0ef770 tx=0x7fa5bc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:04.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 -- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c410e5b0 msgr2=0x7fa5c419eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c410e5b0 0x7fa5c419eaa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009860 tx=0x7fa5bc004360 comp rx=0 tx=0).stop 2024-04-16T17:06:04.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 -- 172.21.15.18:0/4252827508 shutdown_connections 2024-04-16T17:06:04.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa5a80a3ed0 0x7fa5a80a6390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c410e5b0 0x7fa5c419eaa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5c4109800 0x7fa5c41a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 --2- 172.21.15.18:0/4252827508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c4108e40 0x7fa5c41a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 -- 172.21.15.18:0/4252827508 >> 172.21.15.18:0/4252827508 conn(0x7fa5c4077a50 msgr2=0x7fa5c407a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:04.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 -- 172.21.15.18:0/4252827508 shutdown_connections 2024-04-16T17:06:04.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:04.906+0000 7fa5c9200640 1 -- 172.21.15.18:0/4252827508 wait complete. 2024-04-16T17:06:04.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:06:04.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.158+0000 7f28e4a00640 1 -- 172.21.15.18:0/3421200761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0101980 msgr2=0x7f28e0105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.158+0000 7f28e4a00640 1 --2- 172.21.15.18:0/3421200761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0101980 0x7f28e0105e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009900 tx=0x7f28cc02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.159+0000 7f28e4a00640 1 -- 172.21.15.18:0/3421200761 shutdown_connections 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.159+0000 7f28e4a00640 1 --2- 172.21.15.18:0/3421200761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e0106380 0x7f28e0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.159+0000 7f28e4a00640 1 --2- 172.21.15.18:0/3421200761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0101980 0x7f28e0105e40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.159+0000 7f28e4a00640 1 --2- 172.21.15.18:0/3421200761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28e0101060 0x7f28e0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.159+0000 7f28e4a00640 1 -- 172.21.15.18:0/3421200761 >> 172.21.15.18:0/3421200761 conn(0x7f28e00780b0 msgr2=0x7f28e00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.159+0000 7f28e4a00640 1 -- 172.21.15.18:0/3421200761 shutdown_connections 2024-04-16T17:06:05.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.159+0000 7f28e4a00640 1 -- 172.21.15.18:0/3421200761 wait complete. 2024-04-16T17:06:05.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.160+0000 7f28e4a00640 1 Processor -- start 2024-04-16T17:06:05.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.160+0000 7f28e4a00640 1 -- start start 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28e4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e0101060 0x7f28e019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28e4a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28e0101980 0x7f28e019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28e4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0106380 0x7f28e01a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28e4a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0118ac0 con 0x7f28e0101980 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28e4a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0118940 con 0x7f28e0106380 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28e4a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0118c40 con 0x7f28e0101060 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28dde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e0101060 0x7f28e019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28dde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e0101060 0x7f28e019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49162/0 (socket says 172.21.15.18:49162) 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28dde00640 1 -- 172.21.15.18:0/1594954648 learned_addr learned my addr 172.21.15.18:0/1594954648 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.161+0000 7f28de800640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0106380 0x7f28e01a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.162+0000 7f28dde00640 1 -- 172.21.15.18:0/1594954648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0106380 msgr2=0x7f28e01a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.162+0000 7f28dde00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0106380 0x7f28e01a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.162+0000 7f28dde00640 1 -- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28e0101980 msgr2=0x7f28e019f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.162+0000 7f28dde00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28e0101980 0x7f28e019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.162+0000 7f28dde00640 1 -- 172.21.15.18:0/1594954648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e01a3f60 con 0x7f28e0101060 2024-04-16T17:06:05.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.162+0000 7f28de800640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0106380 0x7f28e01a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:05.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.162+0000 7f28dde00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e0101060 0x7f28e019efc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28d000b4a0 tx=0x7f28d000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:05.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.162+0000 7f28d6a00640 1 -- 172.21.15.18:0/1594954648 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0014070 con 0x7f28e0101060 2024-04-16T17:06:05.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.163+0000 7f28e4a00640 1 -- 172.21.15.18:0/1594954648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e01a4250 con 0x7f28e0101060 2024-04-16T17:06:05.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.163+0000 7f28d6a00640 1 -- 172.21.15.18:0/1594954648 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d0010af0 con 0x7f28e0101060 2024-04-16T17:06:05.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.163+0000 7f28d6a00640 1 -- 172.21.15.18:0/1594954648 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0017d10 con 0x7f28e0101060 2024-04-16T17:06:05.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.163+0000 7f28e4a00640 1 -- 172.21.15.18:0/1594954648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e01abae0 con 0x7f28e0101060 2024-04-16T17:06:05.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.167+0000 7f28e77bf740 1 Processor -- start 2024-04-16T17:06:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.167+0000 7f28e77bf740 1 -- start start 2024-04-16T17:06:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.168+0000 7f28d6a00640 1 -- 172.21.15.18:0/1594954648 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f28d0017070 con 0x7f28e0101060 2024-04-16T17:06:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.168+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30baab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.168+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baabad0 0x55e30bab4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.168+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30bab46b0 0x55e30bab6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.168+0000 7f28e77bf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e30ba11120 con 0x55e30bab46b0 2024-04-16T17:06:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.168+0000 7f28e77bf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e30b9b7340 con 0x55e30baabad0 2024-04-16T17:06:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.168+0000 7f28e77bf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e30ba0d0e0 con 0x55e30baab1b0 2024-04-16T17:06:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.169+0000 7f28de800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30baab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.169+0000 7f28de800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30baab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49178/0 (socket says 172.21.15.18:49178) 2024-04-16T17:06:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.169+0000 7f28de800640 1 -- 172.21.15.18:0/2288926370 learned_addr learned my addr 172.21.15.18:0/2288926370 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:05.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.169+0000 7f28de800640 1 -- 172.21.15.18:0/2288926370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baabad0 msgr2=0x55e30bab4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:05.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.169+0000 7f28de800640 1 --2- 172.21.15.18:0/2288926370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baabad0 0x55e30bab4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.169+0000 7f28de800640 1 -- 172.21.15.18:0/2288926370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30bab46b0 msgr2=0x55e30bab6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:05.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.169+0000 7f28de800640 1 --2- 172.21.15.18:0/2288926370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30bab46b0 0x55e30bab6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.169+0000 7f28de800640 1 -- 172.21.15.18:0/2288926370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e30b7ed7d0 con 0x55e30baab1b0 2024-04-16T17:06:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.170+0000 7f28de800640 1 --2- 172.21.15.18:0/2288926370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30baab590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28d8007f20 tx=0x7f28d800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d6e650dffabcd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.170+0000 7f28cb000640 1 -- 172.21.15.18:0/2288926370 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d800e850 con 0x55e30baab1b0 2024-04-16T17:06:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.170+0000 7f28cb000640 1 -- 172.21.15.18:0/2288926370 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d800d3d0 con 0x55e30baab1b0 2024-04-16T17:06:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.170+0000 7f28cb000640 1 -- 172.21.15.18:0/2288926370 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d800a620 con 0x55e30baab1b0 2024-04-16T17:06:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.170+0000 7f28d6a00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28c40a4000 0x7f28c40a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28dde00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28c40a4000 0x7f28c40a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28e77bf740 1 -- 172.21.15.18:0/2288926370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 msgr2=0x55e30baab590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28e77bf740 1 --2- 172.21.15.18:0/2288926370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30baab590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28d8007f20 tx=0x7f28d800c7f0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28d6a00640 1 -- 172.21.15.18:0/1594954648 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f28d00d01e0 con 0x7f28e0101060 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28e77bf740 1 -- 172.21.15.18:0/2288926370 shutdown_connections 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28e77bf740 1 --2- 172.21.15.18:0/2288926370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30bab46b0 0x55e30bab6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28e77bf740 1 --2- 172.21.15.18:0/2288926370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baabad0 0x55e30bab4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28e77bf740 1 --2- 172.21.15.18:0/2288926370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30baab590 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.171+0000 7f28e77bf740 1 -- 172.21.15.18:0/2288926370 >> 172.21.15.18:0/2288926370 conn(0x55e30baa58e0 msgr2=0x55e30baa5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.172+0000 7f28e77bf740 1 -- 172.21.15.18:0/2288926370 shutdown_connections 2024-04-16T17:06:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.172+0000 7f28e77bf740 1 -- 172.21.15.18:0/2288926370 wait complete. 2024-04-16T17:06:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.172+0000 7f28dde00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28c40a4000 0x7f28c40a64c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f28d00cd4b0 tx=0x7f28d00661c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.172+0000 7f28e77bf740 1 -- start start 2024-04-16T17:06:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.173+0000 7f28e77bf740 1 Processor -- start 2024-04-16T17:06:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.174+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bab1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.174+0000 7f28de800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bab1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.174+0000 7f28de800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bab1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48786/0 (socket says 172.21.15.18:48786) 2024-04-16T17:06:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.174+0000 7f28de800640 1 -- 172.21.15.18:0/4284918327 learned_addr learned my addr 172.21.15.18:0/4284918327 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.174+0000 7f28e77bf740 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baabad0 0x55e30bab1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.175+0000 7f28e77bf740 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30bab46b0 0x55e30bab21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.175+0000 7f28dd400640 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baabad0 0x55e30bab1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.175+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e30bab74a0 con 0x55e30bab46b0 2024-04-16T17:06:05.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.175+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e30b9ca710 con 0x55e30baab1b0 2024-04-16T17:06:05.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.175+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e30bab7620 con 0x55e30baabad0 2024-04-16T17:06:05.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.176+0000 7f28e77bf740 10 client.0 _ll_register_callbacks cb 0x55e30b9c5af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:05.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.176+0000 7f28dd400640 1 -- 172.21.15.18:0/4284918327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 msgr2=0x55e30bab1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:05.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.176+0000 7f28dd400640 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bab1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.176+0000 7f28dd400640 1 -- 172.21.15.18:0/4284918327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30bab46b0 msgr2=0x55e30bab21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:05.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.176+0000 7f28dd400640 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30bab46b0 0x55e30bab21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.176+0000 7f28dd400640 1 -- 172.21.15.18:0/4284918327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e30b7233f0 con 0x55e30baabad0 2024-04-16T17:06:05.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.177+0000 7f28bf400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:05.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.177+0000 7f28dd400640 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baabad0 0x55e30bab1ca0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009850 tx=0x7f28cc0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a54fd64ba3bf4b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:05.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.177+0000 7f28bf400640 1 -- 172.21.15.18:0/4284918327 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc041020 con 0x55e30baabad0 2024-04-16T17:06:05.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.177+0000 7f28bf400640 1 -- 172.21.15.18:0/4284918327 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f28cc02cbe0 con 0x55e30baabad0 2024-04-16T17:06:05.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.178+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d800d3d0 con 0x55e30baabad0 2024-04-16T17:06:05.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.178+0000 7f28bf400640 1 -- 172.21.15.18:0/4284918327 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28cc02cde0 con 0x55e30baabad0 2024-04-16T17:06:05.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.178+0000 7f28e77bf740 10 client.26056 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:05.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.178+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e30b6e30f0 con 0x55e30baabad0 2024-04-16T17:06:05.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.179+0000 7f28bf400640 1 -- 172.21.15.18:0/4284918327 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d800d3d0 con 0x55e30baabad0 2024-04-16T17:06:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.179+0000 7f28bf400640 1 -- 172.21.15.18:0/4284918327 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f28cc004c60 con 0x55e30baabad0 2024-04-16T17:06:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.179+0000 7f28e77bf740 10 client.26056 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.179+0000 7f28e77bf740 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.180+0000 7f28e77bf740 20 client.26056 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.180+0000 7f28e77bf740 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e30bac4a20 0x55e30bac6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.180+0000 7f28e77bf740 4 client.26056 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.180+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e30ba1eb40 con 0x55e30bac4a20 2024-04-16T17:06:05.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.183+0000 7f28de800640 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e30bac4a20 0x55e30bac6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.187+0000 7f28bf400640 10 client.26056 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:05.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.187+0000 7f28de800640 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e30bac4a20 0x55e30bac6e10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98db71affa9e5536 server_cookie=48883830ac226084 in_seq=0 out_seq=0 2024-04-16T17:06:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.188+0000 7f28bf400640 1 -- 172.21.15.18:0/4284918327 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e30ba1eb40 con 0x55e30bac4a20 2024-04-16T17:06:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.188+0000 7f28bf400640 10 client.26056 handle_command_reply: tid=0 2024-04-16T17:06:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.188+0000 7f28e77bf740 1 client.26056 shutdown 2024-04-16T17:06:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.188+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baabad0 msgr2=0x55e30bab1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.189+0000 7f28e77bf740 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baabad0 0x55e30bab1ca0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009850 tx=0x7f28cc0092d0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.189+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 shutdown_connections 2024-04-16T17:06:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.189+0000 7f28e77bf740 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e30bac4a20 0x55e30bac6e10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.189+0000 7f28bf400640 0 client.26056 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.189+0000 7f28e77bf740 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30bab46b0 0x55e30bab21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.189+0000 7f28e77bf740 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baabad0 0x55e30bab1ca0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.189+0000 7f28e77bf740 1 --2- 172.21.15.18:0/4284918327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bab1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.190+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 >> 172.21.15.18:0/4284918327 conn(0x55e30baa58e0 msgr2=0x55e30baa7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:05.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.190+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 shutdown_connections 2024-04-16T17:06:05.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.190+0000 7f28e77bf740 1 -- 172.21.15.18:0/4284918327 wait complete. 2024-04-16T17:06:05.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.190+0000 7f28e77bf740 20 client.26056 trim_cache size 0 max 16384 2024-04-16T17:06:05.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.214+0000 7f28e77bf740 1 Processor -- start 2024-04-16T17:06:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28e77bf740 1 -- start start 2024-04-16T17:06:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30bab3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30bac0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30bab46b0 0x55e30bab06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28e77bf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b85c0 con 0x55e30baabad0 2024-04-16T17:06:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28e77bf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b5e90 con 0x55e30bab46b0 2024-04-16T17:06:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28e77bf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b3ce0 con 0x55e30baab1b0 2024-04-16T17:06:05.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28dd400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30bac0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28dd400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30bac0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52476/0 (socket says 172.21.15.18:52476) 2024-04-16T17:06:05.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.215+0000 7f28dd400640 1 -- 172.21.15.18:0/2163364655 learned_addr learned my addr 172.21.15.18:0/2163364655 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:05.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.216+0000 7f28de800640 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30bab46b0 0x55e30bab06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.216+0000 7f28de800640 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30bab3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.216+0000 7f28dd400640 1 -- 172.21.15.18:0/2163364655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 msgr2=0x55e30bab3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:05.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.216+0000 7f28dd400640 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30bab3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.216+0000 7f28dd400640 1 -- 172.21.15.18:0/2163364655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30bab46b0 msgr2=0x55e30bab06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:05.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.216+0000 7f28dd400640 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30bab46b0 0x55e30bab06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.216+0000 7f28dd400640 1 -- 172.21.15.18:0/2163364655 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e30ba9fd20 con 0x55e30baabad0 2024-04-16T17:06:05.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.217+0000 7f28dd400640 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30bac0360 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28cc003640 tx=0x7f28cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eee72483568e64f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:05.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.217+0000 7f28ca600640 1 -- 172.21.15.18:0/2163364655 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc041070 con 0x55e30baabad0 2024-04-16T17:06:05.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.217+0000 7f28ca600640 1 -- 172.21.15.18:0/2163364655 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc053070 con 0x55e30baabad0 2024-04-16T17:06:05.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.217+0000 7f28ca600640 1 -- 172.21.15.18:0/2163364655 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc003cd0 con 0x55e30baabad0 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.218+0000 7f28e77bf740 1 -- 172.21.15.18:0/2163364655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 msgr2=0x55e30bac0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.218+0000 7f28e77bf740 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30bac0360 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28cc003640 tx=0x7f28cc005fd0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.218+0000 7f28e77bf740 1 -- 172.21.15.18:0/2163364655 shutdown_connections 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.218+0000 7f28e77bf740 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30bab46b0 0x55e30bab06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.218+0000 7f28e77bf740 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30bac0360 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.219+0000 7f28e77bf740 1 --2- 172.21.15.18:0/2163364655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30baab1b0 0x55e30bab3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.219+0000 7f28e77bf740 1 -- 172.21.15.18:0/2163364655 >> 172.21.15.18:0/2163364655 conn(0x55e30baa58e0 msgr2=0x55e30baa75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.219+0000 7f28e77bf740 1 -- 172.21.15.18:0/2163364655 shutdown_connections 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.219+0000 7f28e77bf740 1 -- 172.21.15.18:0/2163364655 wait complete. 2024-04-16T17:06:05.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.219+0000 7f28e77bf740 1 -- start start 2024-04-16T17:06:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.219+0000 7f28e77bf740 1 Processor -- start 2024-04-16T17:06:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bb26880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30babc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28e77bf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30bab46b0 0x55e30babcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28e77bf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b4350 con 0x55e30baabad0 2024-04-16T17:06:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28e77bf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b57c0 con 0x55e30baab1b0 2024-04-16T17:06:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28e77bf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b4a20 con 0x55e30bab46b0 2024-04-16T17:06:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28e77bf740 10 client.0 _ll_register_callbacks cb 0x7f28cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28de800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bb26880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28de800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bb26880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48810/0 (socket says 172.21.15.18:48810) 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28de800640 1 -- 172.21.15.18:0/979280512 learned_addr learned my addr 172.21.15.18:0/979280512 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.220+0000 7f28de800640 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30bab46b0 0x55e30babcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.221+0000 7f28dd400640 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30babc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.221+0000 7f28de800640 1 -- 172.21.15.18:0/979280512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30bab46b0 msgr2=0x55e30babcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.221+0000 7f28de800640 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30bab46b0 0x55e30babcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.221+0000 7f28de800640 1 -- 172.21.15.18:0/979280512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 msgr2=0x55e30babc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.221+0000 7f28de800640 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30babc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.221+0000 7f28de800640 1 -- 172.21.15.18:0/979280512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e30baae940 con 0x55e30baab1b0 2024-04-16T17:06:05.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.221+0000 7f28de800640 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bb26880 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f28d8007ec0 tx=0x7f28cc0044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=663b2ce75516e0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:05.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.221+0000 7f28bf400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.222+0000 7f28bf400640 1 -- 172.21.15.18:0/979280512 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d80288e0 con 0x55e30baab1b0 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.222+0000 7f28e77bf740 1 -- 172.21.15.18:0/979280512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28cc053070 con 0x55e30baab1b0 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.222+0000 7f28bf400640 1 -- 172.21.15.18:0/979280512 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f28d800e650 con 0x55e30baab1b0 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.222+0000 7f28bf400640 1 -- 172.21.15.18:0/979280512 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28d800e850 con 0x55e30baab1b0 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.222+0000 7f28e77bf740 10 client.26060 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.222+0000 7f28e77bf740 1 -- 172.21.15.18:0/979280512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e30bb4d570 con 0x55e30baab1b0 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28bf400640 1 -- 172.21.15.18:0/979280512 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc053070 con 0x55e30baab1b0 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28bf400640 1 -- 172.21.15.18:0/979280512 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f28d8017480 con 0x55e30baab1b0 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28e77bf740 10 client.26060 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28e77bf740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28e77bf740 20 client.26060 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28e77bf740 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e30bac4a20 0x55e30bb4f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28e77bf740 4 client.26060 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:06:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28e77bf740 1 -- 172.21.15.18:0/979280512 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e30ba189b0 con 0x55e30bac4a20 2024-04-16T17:06:05.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.223+0000 7f28dd400640 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e30bac4a20 0x55e30bb4f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.224+0000 7f28dd400640 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e30bac4a20 0x55e30bb4f980 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd1c68223383973d server_cookie=54214fa45d638ff2 in_seq=0 out_seq=0 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.224+0000 7f28bf400640 10 client.26060 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.224+0000 7f28bf400640 1 -- 172.21.15.18:0/979280512 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e30ba189b0 con 0x55e30bac4a20 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.224+0000 7f28bf400640 10 client.26060 handle_command_reply: tid=0 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.224+0000 7f28e77bf740 1 client.26060 shutdown 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.224+0000 7f28e77bf740 1 -- 172.21.15.18:0/979280512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 msgr2=0x55e30bb26880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.224+0000 7f28e77bf740 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bb26880 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f28d8007ec0 tx=0x7f28cc0044b0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.225+0000 7f28e77bf740 1 -- 172.21.15.18:0/979280512 shutdown_connections 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.225+0000 7f28e77bf740 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e30bac4a20 0x55e30bb4f980 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.225+0000 7f28e77bf740 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e30bab46b0 0x55e30babcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.225+0000 7f28bf400640 0 client.26060 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.225+0000 7f28e77bf740 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e30baabad0 0x55e30babc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.225+0000 7f28e77bf740 1 --2- 172.21.15.18:0/979280512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e30baab1b0 0x55e30bb26880 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.225+0000 7f28e77bf740 1 -- 172.21.15.18:0/979280512 >> 172.21.15.18:0/979280512 conn(0x55e30baa58e0 msgr2=0x55e30bab1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.225+0000 7f28e77bf740 1 -- 172.21.15.18:0/979280512 shutdown_connections 2024-04-16T17:06:05.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.226+0000 7f28e77bf740 1 -- 172.21.15.18:0/979280512 wait complete. 2024-04-16T17:06:05.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.226+0000 7f28e77bf740 20 client.26060 trim_cache size 0 max 16384 2024-04-16T17:06:05.228 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "6a4c33c8-966b-4eaa-a4f3-621751eab67a": { 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "6a4c33c8-966b-4eaa-a4f3-621751eab67a", 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:06:05.229 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.227+0000 7f28e4a00640 1 -- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28c40a4000 msgr2=0x7f28c40a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.227+0000 7f28e4a00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28c40a4000 0x7f28c40a64c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f28d00cd4b0 tx=0x7f28d00661c0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.227+0000 7f28e4a00640 1 -- 172.21.15.18:0/1594954648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e0101060 msgr2=0x7f28e019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.227+0000 7f28e4a00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e0101060 0x7f28e019efc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28d000b4a0 tx=0x7f28d000b960 comp rx=0 tx=0).stop 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.228+0000 7f28e4a00640 1 -- 172.21.15.18:0/1594954648 shutdown_connections 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.228+0000 7f28e4a00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28c40a4000 0x7f28c40a64c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.228+0000 7f28e4a00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28e0106380 0x7f28e01a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.228+0000 7f28e4a00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28e0101980 0x7f28e019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.228+0000 7f28e4a00640 1 --2- 172.21.15.18:0/1594954648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e0101060 0x7f28e019efc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.228+0000 7f28e4a00640 1 -- 172.21.15.18:0/1594954648 >> 172.21.15.18:0/1594954648 conn(0x7f28e00780b0 msgr2=0x7f28e0103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.228+0000 7f28e4a00640 1 -- 172.21.15.18:0/1594954648 shutdown_connections 2024-04-16T17:06:05.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:05.228+0000 7f28e4a00640 1 -- 172.21.15.18:0/1594954648 wait complete. 2024-04-16T17:06:05.249 INFO:tasks.cephfs.filesystem:scrub status for tag:6a4c33c8-966b-4eaa-a4f3-621751eab67a - {'path': '/', 'tag': '6a4c33c8-966b-4eaa-a4f3-621751eab67a', 'options': 'recursive,force'} 2024-04-16T17:06:05.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 14703 5.41 MB/sec warmup 18 sec latency 30.458 ms 2024-04-16T17:06:05.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:05 smithi022 ceph-mon[36023]: pgmap v43: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.4 MiB/s wr, 106 op/s 2024-04-16T17:06:05.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:05 smithi043 ceph-mon[36901]: pgmap v43: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.4 MiB/s wr, 106 op/s 2024-04-16T17:06:06.499 INFO:tasks.workunit.client.0.smithi018.stdout: 1 15664 5.40 MB/sec warmup 19 sec latency 16.146 ms 2024-04-16T17:06:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:06 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:06:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:06 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:06 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:06:06.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:06 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:06.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:05 smithi018 ceph-mon[31776]: pgmap v43: 129 pgs: 129 active+clean; 26 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 2.4 MiB/s wr, 106 op/s 2024-04-16T17:06:06.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:06 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:06:06.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:06 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:06.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:06 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:06:06.704 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:06 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:06.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:06 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:06:06.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:06 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:06.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:06 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:06:06.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:06 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:07.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 16693 5.26 MB/sec warmup 20 sec latency 30.056 ms 2024-04-16T17:06:07.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:07 smithi022 ceph-mon[36023]: pgmap v44: 129 pgs: 129 active+clean; 24 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-04-16T17:06:07.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:07 smithi043 ceph-mon[36901]: pgmap v44: 129 pgs: 129 active+clean; 24 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-04-16T17:06:08.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 17110 5.24 MB/sec warmup 21 sec latency 82.688 ms 2024-04-16T17:06:09.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 17557 5.09 MB/sec warmup 22 sec latency 134.299 ms 2024-04-16T17:06:09.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:09 smithi022 ceph-mon[36023]: pgmap v45: 129 pgs: 129 active+clean; 24 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-04-16T17:06:09.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:09 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:06:09.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:09 smithi043 ceph-mon[36901]: pgmap v45: 129 pgs: 129 active+clean; 24 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-04-16T17:06:09.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:09 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:06:10.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 18329 5.07 MB/sec warmup 23 sec latency 11.614 ms 2024-04-16T17:06:11.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 19378 5.07 MB/sec warmup 24 sec latency 23.098 ms 2024-04-16T17:06:11.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:11 smithi022 ceph-mon[36023]: pgmap v46: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-04-16T17:06:11.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:11 smithi043 ceph-mon[36901]: pgmap v46: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-04-16T17:06:12.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 20698 5.16 MB/sec warmup 25 sec latency 12.198 ms 2024-04-16T17:06:13.326 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:07 smithi018 ceph-mon[31776]: pgmap v44: 129 pgs: 129 active+clean; 24 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-04-16T17:06:13.326 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:09 smithi018 ceph-mon[31776]: pgmap v45: 129 pgs: 129 active+clean; 24 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-04-16T17:06:13.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:09 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:06:13.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:11 smithi018 ceph-mon[31776]: pgmap v46: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-04-16T17:06:13.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 21176 5.04 MB/sec warmup 26 sec latency 137.299 ms 2024-04-16T17:06:13.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:13 smithi022 ceph-mon[36023]: pgmap v47: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 593 KiB/s wr, 165 op/s 2024-04-16T17:06:13.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:13 smithi043 ceph-mon[36901]: pgmap v47: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 593 KiB/s wr, 165 op/s 2024-04-16T17:06:14.387 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:13 smithi018 ceph-mon[31776]: pgmap v47: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 593 KiB/s wr, 165 op/s 2024-04-16T17:06:14.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 21590 5.02 MB/sec warmup 27 sec latency 59.566 ms 2024-04-16T17:06:15.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 21986 4.85 MB/sec warmup 28 sec latency 55.519 ms 2024-04-16T17:06:15.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:15 smithi022 ceph-mon[36023]: pgmap v48: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 504 KiB/s wr, 162 op/s 2024-04-16T17:06:15.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:15 smithi043 ceph-mon[36901]: pgmap v48: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 504 KiB/s wr, 162 op/s 2024-04-16T17:06:16.500 INFO:tasks.workunit.client.0.smithi018.stdout: 1 23013 4.86 MB/sec warmup 29 sec latency 18.340 ms 2024-04-16T17:06:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.945+0000 7f7fadc00640 1 -- 172.21.15.18:0/347406320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fa8102080 msgr2=0x7f7fa8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.945+0000 7f7fadc00640 1 --2- 172.21.15.18:0/347406320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fa8102080 0x7f7fa8102460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009860 tx=0x7f7f9802bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.945+0000 7f7fadc00640 1 -- 172.21.15.18:0/347406320 shutdown_connections 2024-04-16T17:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.945+0000 7f7fadc00640 1 --2- 172.21.15.18:0/347406320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa8107150 0x7f7fa810ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.945+0000 7f7fadc00640 1 --2- 172.21.15.18:0/347406320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8102a30 0x7f7fa8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.945+0000 7f7fadc00640 1 --2- 172.21.15.18:0/347406320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fa8102080 0x7f7fa8102460 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.945+0000 7f7fadc00640 1 -- 172.21.15.18:0/347406320 >> 172.21.15.18:0/347406320 conn(0x7f7fa80fd940 msgr2=0x7f7fa80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.946+0000 7f7fadc00640 1 -- 172.21.15.18:0/347406320 shutdown_connections 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.946+0000 7f7fadc00640 1 -- 172.21.15.18:0/347406320 wait complete. 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.946+0000 7f7fadc00640 1 Processor -- start 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fadc00640 1 -- start start 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fadc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fa8102080 0x7f7fa8074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fadc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8107150 0x7f7fa8074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fadc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa8074d80 0x7f7fa81a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fadc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8112210 con 0x7f7fa8102080 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fadc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8112090 con 0x7f7fa8107150 2024-04-16T17:06:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fadc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8112390 con 0x7f7fa8074d80 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fa8102080 0x7f7fa8074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fad200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa8074d80 0x7f7fa81a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fad200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa8074d80 0x7f7fa81a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:46220/0 (socket says 172.21.15.18:46220) 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.947+0000 7f7fad200640 1 -- 172.21.15.18:0/1718282411 learned_addr learned my addr 172.21.15.18:0/1718282411 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.948+0000 7f7fa7400640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8107150 0x7f7fa8074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.948+0000 7f7fad200640 1 -- 172.21.15.18:0/1718282411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8107150 msgr2=0x7f7fa8074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.948+0000 7f7fad200640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8107150 0x7f7fa8074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.948+0000 7f7fad200640 1 -- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fa8102080 msgr2=0x7f7fa8074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.948+0000 7f7fad200640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fa8102080 0x7f7fa8074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.948+0000 7f7fad200640 1 -- 172.21.15.18:0/1718282411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa81a6fe0 con 0x7f7fa8074d80 2024-04-16T17:06:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.948+0000 7f7fad200640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa8074d80 0x7f7fa81a68e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000eb10 tx=0x7f7fa000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.949+0000 7f7fa4c00640 1 -- 172.21.15.18:0/1718282411 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0004b20 con 0x7f7fa8074d80 2024-04-16T17:06:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.949+0000 7f7fa4c00640 1 -- 172.21.15.18:0/1718282411 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa0004cc0 con 0x7f7fa8074d80 2024-04-16T17:06:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.949+0000 7f7fadc00640 1 -- 172.21.15.18:0/1718282411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa81a72d0 con 0x7f7fa8074d80 2024-04-16T17:06:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.949+0000 7f7fa4c00640 1 -- 172.21.15.18:0/1718282411 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0010760 con 0x7f7fa8074d80 2024-04-16T17:06:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.949+0000 7f7fadc00640 1 -- 172.21.15.18:0/1718282411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa81a7820 con 0x7f7fa8074d80 2024-04-16T17:06:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.953+0000 7f7fb1e74740 1 Processor -- start 2024-04-16T17:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.954+0000 7f7fa4c00640 1 -- 172.21.15.18:0/1718282411 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7fa0005020 con 0x7f7fa8074d80 2024-04-16T17:06:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.954+0000 7f7fb1e74740 1 -- start start 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.954+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 0x5592dc4bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5592dc418780 con 0x5592dc4bd000 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5592dc3bef40 con 0x5592dc4b3a80 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5592dc414740 con 0x5592dc4b47f0 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33824/0 (socket says 172.21.15.18:33824) 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7e00640 1 -- 172.21.15.18:0/2016304287 learned_addr learned my addr 172.21.15.18:0/2016304287 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7400640 1 --2- 172.21.15.18:0/2016304287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 0x5592dc4bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7e00640 1 -- 172.21.15.18:0/2016304287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 msgr2=0x5592dc4b3540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/2016304287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4b3540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7e00640 1 -- 172.21.15.18:0/2016304287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 msgr2=0x5592dc4bf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/2016304287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 0x5592dc4bf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.955+0000 7f7fa7e00640 1 -- 172.21.15.18:0/2016304287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592dc12b210 con 0x5592dc4b3a80 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.956+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/2016304287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4b3f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009780 tx=0x7f7f98009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce971bb30addff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.956+0000 7f7fa4c00640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f8c0a3d30 0x7f7f8c0a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.956+0000 7f7f9d600640 1 -- 172.21.15.18:0/2016304287 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98043020 con 0x5592dc4b3a80 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.956+0000 7f7f9d600640 1 -- 172.21.15.18:0/2016304287 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f98034cb0 con 0x5592dc4b3a80 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.956+0000 7f7f9d600640 1 -- 172.21.15.18:0/2016304287 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98039040 con 0x5592dc4b3a80 2024-04-16T17:06:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.956+0000 7f7fa4c00640 1 -- 172.21.15.18:0/1718282411 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f7fa00d1020 con 0x7f7fa8074d80 2024-04-16T17:06:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fa7400640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f8c0a3d30 0x7f7f8c0a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2016304287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 msgr2=0x5592dc4b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2016304287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4b3f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009780 tx=0x7f7f98009510 comp rx=0 tx=0).stop 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2016304287 shutdown_connections 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2016304287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 0x5592dc4bf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2016304287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4b3f00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2016304287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2016304287 >> 172.21.15.18:0/2016304287 conn(0x5592dc44e5d0 msgr2=0x5592dc4ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2016304287 shutdown_connections 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.957+0000 7f7fa7400640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f8c0a3d30 0x7f7f8c0a61f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5592dc4af900 tx=0x7f7f94009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.958+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2016304287 wait complete. 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.958+0000 7f7fb1e74740 1 -- start start 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.958+0000 7f7fb1e74740 1 Processor -- start 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b3a80 0x5592dc4bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b47f0 0x5592dc4b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4bd000 0x5592dc4ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5592dc4bfcd0 con 0x5592dc4b47f0 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5592dc3d2390 con 0x5592dc4bd000 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5592dc4bfe50 con 0x5592dc4b3a80 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fb1e74740 10 client.0 _ll_register_callbacks cb 0x5592dc3bbf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b47f0 0x5592dc4b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b3a80 0x5592dc4bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b47f0 0x5592dc4b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32856/0 (socket says 172.21.15.18:32856) 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fa7400640 1 -- 172.21.15.18:0/2432292575 learned_addr learned my addr 172.21.15.18:0/2432292575 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fad200640 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4bd000 0x5592dc4ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fa7e00640 1 -- 172.21.15.18:0/2432292575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4bd000 msgr2=0x5592dc4ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4bd000 0x5592dc4ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.959+0000 7f7fa7e00640 1 -- 172.21.15.18:0/2432292575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b47f0 msgr2=0x5592dc4b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.960+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b47f0 0x5592dc4b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.960+0000 7f7fa7e00640 1 -- 172.21.15.18:0/2432292575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592dc1f7620 con 0x5592dc4b3a80 2024-04-16T17:06:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.960+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b3a80 0x5592dc4bca20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009830 tx=0x7f7f9802bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1073475f3ed8d2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.960+0000 7f7f92000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.960+0000 7f7f92000640 1 -- 172.21.15.18:0/2432292575 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98043030 con 0x5592dc4b3a80 2024-04-16T17:06:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.960+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2432292575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f98034cb0 con 0x5592dc4b3a80 2024-04-16T17:06:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.960+0000 7f7f92000640 1 -- 172.21.15.18:0/2432292575 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f7f9802c510 con 0x5592dc4b3a80 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7f92000640 1 -- 172.21.15.18:0/2432292575 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f9805a050 con 0x5592dc4b3a80 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7fb1e74740 10 client.26071 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2432292575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592dc4bfcd0 con 0x5592dc4b3a80 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7f92000640 1 -- 172.21.15.18:0/2432292575 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f98034cb0 con 0x5592dc4b3a80 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7f92000640 1 -- 172.21.15.18:0/2432292575 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7f98032a80 con 0x5592dc4b3a80 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7fb1e74740 10 client.26071 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7fb1e74740 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7fb1e74740 20 client.26071 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5592dc4ccf20 0x5592dc4cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7fb1e74740 4 client.26071 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.961+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2432292575 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592dc4261a0 con 0x5592dc4ccf20 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.962+0000 7f7fa7400640 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5592dc4ccf20 0x5592dc4cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.962+0000 7f7fa7400640 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5592dc4ccf20 0x5592dc4cf310 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82549118f2e81c2 server_cookie=999983f374e827fc in_seq=0 out_seq=0 2024-04-16T17:06:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.962+0000 7f7f92000640 10 client.26071 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7f92000640 1 -- 172.21.15.18:0/2432292575 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592dc4261a0 con 0x5592dc4ccf20 2024-04-16T17:06:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7f92000640 10 client.26071 handle_command_reply: tid=0 2024-04-16T17:06:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 client.26071 shutdown 2024-04-16T17:06:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2432292575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b3a80 msgr2=0x5592dc4bca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b3a80 0x5592dc4bca20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009830 tx=0x7f7f9802bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2432292575 shutdown_connections 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5592dc4ccf20 0x5592dc4cf310 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4bd000 0x5592dc4ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b47f0 0x5592dc4b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/2432292575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b3a80 0x5592dc4bca20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7f92000640 0 client.26071 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2432292575 >> 172.21.15.18:0/2432292575 conn(0x5592dc44e5d0 msgr2=0x5592dc4b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2432292575 shutdown_connections 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.964+0000 7f7fb1e74740 1 -- 172.21.15.18:0/2432292575 wait complete. 2024-04-16T17:06:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.965+0000 7f7fb1e74740 20 client.26071 trim_cache size 0 max 16384 2024-04-16T17:06:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.983+0000 7f7fb1e74740 1 Processor -- start 2024-04-16T17:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fb1e74740 1 -- start start 2024-04-16T17:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 0x5592dc4b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81b3ec0 con 0x5592dc4bd000 2024-04-16T17:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81b1790 con 0x5592dc4b3a80 2024-04-16T17:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81af5e0 con 0x5592dc4b47f0 2024-04-16T17:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:46248/0 (socket says 172.21.15.18:46248) 2024-04-16T17:06:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.984+0000 7f7fa7400640 1 -- 172.21.15.18:0/3304217242 learned_addr learned my addr 172.21.15.18:0/3304217242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7fad200640 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 0x5592dc4b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7fa7400640 1 -- 172.21.15.18:0/3304217242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 msgr2=0x5592dc4ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7fa7400640 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7fa7400640 1 -- 172.21.15.18:0/3304217242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 msgr2=0x5592dc4b9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7fa7400640 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 0x5592dc4b9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7fa7400640 1 -- 172.21.15.18:0/3304217242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592dc4b2d50 con 0x5592dc4b47f0 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7fa7400640 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4c8c30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5592dc4ae140 tx=0x7f7f94016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7938fdbe34539928 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.985+0000 7f7f9cc00640 1 -- 172.21.15.18:0/3304217242 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f940295b0 con 0x5592dc4b47f0 2024-04-16T17:06:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.986+0000 7f7f9cc00640 1 -- 172.21.15.18:0/3304217242 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9402e070 con 0x5592dc4b47f0 2024-04-16T17:06:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.986+0000 7f7f9cc00640 1 -- 172.21.15.18:0/3304217242 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f940288e0 con 0x5592dc4b47f0 2024-04-16T17:06:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3304217242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 msgr2=0x5592dc4c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4c8c30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5592dc4ae140 tx=0x7f7f94016000 comp rx=0 tx=0).stop 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3304217242 shutdown_connections 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4bd000 0x5592dc4b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4b47f0 0x5592dc4c8c30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3304217242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b3a80 0x5592dc4ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3304217242 >> 172.21.15.18:0/3304217242 conn(0x5592dc44e5d0 msgr2=0x5592dc4b9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3304217242 shutdown_connections 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3304217242 wait complete. 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.987+0000 7f7fb1e74740 1 -- start start 2024-04-16T17:06:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.988+0000 7f7fb1e74740 1 Processor -- start 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.988+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b3a80 0x5592dc4c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.988+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b47f0 0x5592dc4b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.988+0000 7f7fb1e74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4bd000 0x5592dc4b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.988+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81afc50 con 0x5592dc4b3a80 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.988+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81b10c0 con 0x5592dc4b47f0 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.988+0000 7f7fb1e74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81b0320 con 0x5592dc4bd000 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.988+0000 7f7fb1e74740 10 client.0 _ll_register_callbacks cb 0x7f7fa00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b47f0 0x5592dc4b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b47f0 0x5592dc4b73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33846/0 (socket says 172.21.15.18:33846) 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 -- 172.21.15.18:0/3171786349 learned_addr learned my addr 172.21.15.18:0/3171786349 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fad200640 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4bd000 0x5592dc4b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b3a80 0x5592dc4c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 -- 172.21.15.18:0/3171786349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4bd000 msgr2=0x5592dc4b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4bd000 0x5592dc4b7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 -- 172.21.15.18:0/3171786349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b3a80 msgr2=0x5592dc4c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b3a80 0x5592dc4c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 -- 172.21.15.18:0/3171786349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592dc4bdd40 con 0x5592dc4b47f0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b3a80 0x5592dc4c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7fa7400640 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b47f0 0x5592dc4b73f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7f940284a0 tx=0x7f7f940116b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ad8e23b3e9f7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.989+0000 7f7f92000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.990+0000 7f7f92000640 1 -- 172.21.15.18:0/3171786349 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9401e8d0 con 0x5592dc4b47f0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.990+0000 7f7f92000640 1 -- 172.21.15.18:0/3171786349 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f7f9401ea70 con 0x5592dc4b47f0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.990+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3171786349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9402e070 con 0x5592dc4b47f0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.990+0000 7f7f92000640 1 -- 172.21.15.18:0/3171786349 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f9401ec70 con 0x5592dc4b47f0 2024-04-16T17:06:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.990+0000 7f7fb1e74740 10 client.26075 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.990+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3171786349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592dc5337e0 con 0x5592dc4b47f0 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.990+0000 7f7f92000640 1 -- 172.21.15.18:0/3171786349 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9402e070 con 0x5592dc4b47f0 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.991+0000 7f7f92000640 1 -- 172.21.15.18:0/3171786349 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7f9401a040 con 0x5592dc4b47f0 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.991+0000 7f7fb1e74740 10 client.26075 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.991+0000 7f7fb1e74740 10 client.26075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.991+0000 7f7fb1e74740 20 client.26075 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.991+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5592dc533ee0 0x5592dc536300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.991+0000 7f7fb1e74740 4 client.26075 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.991+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3171786349 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592dc420010 con 0x5592dc533ee0 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.991+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5592dc533ee0 0x5592dc536300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.992+0000 7f7fa7e00640 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5592dc533ee0 0x5592dc536300 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37ba10ed5273299d server_cookie=ed66cb7cb9db4d23 in_seq=0 out_seq=0 2024-04-16T17:06:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:16.992+0000 7f7f92000640 10 client.26075 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7f92000640 1 -- 172.21.15.18:0/3171786349 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592dc420010 con 0x5592dc533ee0 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7f92000640 10 client.26075 handle_command_reply: tid=0 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 client.26075 shutdown 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3171786349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b47f0 msgr2=0x5592dc4b73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b47f0 0x5592dc4b73f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7f940284a0 tx=0x7f7f940116b0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3171786349 shutdown_connections 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5592dc533ee0 0x5592dc536300 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592dc4bd000 0x5592dc4b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592dc4b47f0 0x5592dc4b73f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 --2- 172.21.15.18:0/3171786349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592dc4b3a80 0x5592dc4c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7f92000640 0 client.26075 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3171786349 >> 172.21.15.18:0/3171786349 conn(0x5592dc44e5d0 msgr2=0x5592dc4b8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3171786349 shutdown_connections 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.106+0000 7f7fb1e74740 1 -- 172.21.15.18:0/3171786349 wait complete. 2024-04-16T17:06:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.107+0000 7f7fb1e74740 20 client.26075 trim_cache size 0 max 16384 2024-04-16T17:06:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:06:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:06:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:06:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:06:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 -- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f8c0a3d30 msgr2=0x7f7f8c0a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f8c0a3d30 0x7f7f8c0a61f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5592dc4af900 tx=0x7f7f94009260 comp rx=0 tx=0).stop 2024-04-16T17:06:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 -- 172.21.15.18:0/1718282411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa8074d80 msgr2=0x7f7fa81a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa8074d80 0x7f7fa81a68e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000eb10 tx=0x7f7fa000c2d0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 -- 172.21.15.18:0/1718282411 shutdown_connections 2024-04-16T17:06:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f8c0a3d30 0x7f7f8c0a61f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa8074d80 0x7f7fa81a68e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8107150 0x7f7fa8074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 --2- 172.21.15.18:0/1718282411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fa8102080 0x7f7fa8074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 -- 172.21.15.18:0/1718282411 >> 172.21.15.18:0/1718282411 conn(0x7f7fa80fd940 msgr2=0x7f7fa80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 -- 172.21.15.18:0/1718282411 shutdown_connections 2024-04-16T17:06:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:17.109+0000 7f7fadc00640 1 -- 172.21.15.18:0/1718282411 wait complete. 2024-04-16T17:06:17.186 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:15 smithi018 ceph-mon[31776]: pgmap v48: 129 pgs: 129 active+clean; 26 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 504 KiB/s wr, 162 op/s 2024-04-16T17:06:17.501 INFO:tasks.workunit.client.0.smithi018.stdout: 1 24305 4.94 MB/sec warmup 30 sec latency 75.885 ms 2024-04-16T17:06:17.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:17 smithi022 ceph-mon[36023]: pgmap v49: 129 pgs: 129 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 708 KiB/s wr, 188 op/s 2024-04-16T17:06:17.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:17 smithi022 ceph-mon[36023]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T17:06:17.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:17 smithi043 ceph-mon[36901]: pgmap v49: 129 pgs: 129 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 708 KiB/s wr, 188 op/s 2024-04-16T17:06:17.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:17 smithi043 ceph-mon[36901]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T17:06:18.501 INFO:tasks.workunit.client.0.smithi018.stdout: 1 24689 4.85 MB/sec warmup 31 sec latency 97.983 ms 2024-04-16T17:06:19.501 INFO:tasks.workunit.client.0.smithi018.stdout: 1 25100 4.83 MB/sec warmup 32 sec latency 69.723 ms 2024-04-16T17:06:19.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:19 smithi022 ceph-mon[36023]: pgmap v50: 129 pgs: 129 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 400 KiB/s wr, 54 op/s 2024-04-16T17:06:19.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:19 smithi043 ceph-mon[36901]: pgmap v50: 129 pgs: 129 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 400 KiB/s wr, 54 op/s 2024-04-16T17:06:20.501 INFO:tasks.workunit.client.0.smithi018.stdout: 1 25787 4.71 MB/sec warmup 33 sec latency 28.256 ms 2024-04-16T17:06:21.501 INFO:tasks.workunit.client.0.smithi018.stdout: 1 26910 4.72 MB/sec warmup 34 sec latency 23.383 ms 2024-04-16T17:06:21.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:21 smithi022 ceph-mon[36023]: pgmap v51: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 601 KiB/s wr, 122 op/s 2024-04-16T17:06:21.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:21 smithi043 ceph-mon[36901]: pgmap v51: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 601 KiB/s wr, 122 op/s 2024-04-16T17:06:22.501 INFO:tasks.workunit.client.0.smithi018.stdout: 1 28083 4.81 MB/sec warmup 35 sec latency 10.868 ms 2024-04-16T17:06:23.506 INFO:tasks.workunit.client.0.smithi018.stdout: 1 28514 4.79 MB/sec warmup 36 sec latency 67.317 ms 2024-04-16T17:06:23.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:23 smithi022 ceph-mon[36023]: pgmap v52: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 425 KiB/s wr, 100 op/s 2024-04-16T17:06:23.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:23 smithi043 ceph-mon[36901]: pgmap v52: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 425 KiB/s wr, 100 op/s 2024-04-16T17:06:24.505 INFO:tasks.workunit.client.0.smithi018.stdout: 1 28730 4.69 MB/sec warmup 37 sec latency 108.789 ms 2024-04-16T17:06:25.506 INFO:tasks.workunit.client.0.smithi018.stdout: 1 29475 4.63 MB/sec warmup 38 sec latency 47.721 ms 2024-04-16T17:06:25.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:25 smithi022 ceph-mon[36023]: pgmap v53: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 417 KiB/s wr, 98 op/s 2024-04-16T17:06:25.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:25 smithi043 ceph-mon[36901]: pgmap v53: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 417 KiB/s wr, 98 op/s 2024-04-16T17:06:26.506 INFO:tasks.workunit.client.0.smithi018.stdout: 1 30743 4.64 MB/sec warmup 39 sec latency 13.853 ms 2024-04-16T17:06:27.506 INFO:tasks.workunit.client.0.smithi018.stdout: 1 32052 4.79 MB/sec warmup 40 sec latency 11.421 ms 2024-04-16T17:06:27.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:27 smithi022 ceph-mon[36023]: pgmap v54: 129 pgs: 129 active+clean; 28 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 650 KiB/s wr, 130 op/s 2024-04-16T17:06:27.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:27 smithi043 ceph-mon[36901]: pgmap v54: 129 pgs: 129 active+clean; 28 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 650 KiB/s wr, 130 op/s 2024-04-16T17:06:28.513 INFO:tasks.workunit.client.0.smithi018.stdout: 1 32344 4.71 MB/sec warmup 41 sec latency 59.792 ms 2024-04-16T17:06:29.514 INFO:tasks.workunit.client.0.smithi018.stdout: 1 32488 4.60 MB/sec warmup 42 sec latency 248.049 ms 2024-04-16T17:06:29.597 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:17 smithi018 ceph-mon[31776]: pgmap v49: 129 pgs: 129 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 708 KiB/s wr, 188 op/s 2024-04-16T17:06:29.597 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:17 smithi018 ceph-mon[31776]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T17:06:29.597 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:19 smithi018 ceph-mon[31776]: pgmap v50: 129 pgs: 129 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 400 KiB/s wr, 54 op/s 2024-04-16T17:06:29.597 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:21 smithi018 ceph-mon[31776]: pgmap v51: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 601 KiB/s wr, 122 op/s 2024-04-16T17:06:29.597 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:23 smithi018 ceph-mon[31776]: pgmap v52: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 425 KiB/s wr, 100 op/s 2024-04-16T17:06:29.597 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:25 smithi018 ceph-mon[31776]: pgmap v53: 129 pgs: 129 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 417 KiB/s wr, 98 op/s 2024-04-16T17:06:29.597 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:27 smithi018 ceph-mon[31776]: pgmap v54: 129 pgs: 129 active+clean; 28 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 650 KiB/s wr, 130 op/s 2024-04-16T17:06:29.597 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:29 smithi018 ceph-mon[31776]: pgmap v55: 129 pgs: 129 active+clean; 28 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 446 KiB/s wr, 104 op/s 2024-04-16T17:06:29.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:29 smithi022 ceph-mon[36023]: pgmap v55: 129 pgs: 129 active+clean; 28 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 446 KiB/s wr, 104 op/s 2024-04-16T17:06:29.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:29 smithi043 ceph-mon[36901]: pgmap v55: 129 pgs: 129 active+clean; 28 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 446 KiB/s wr, 104 op/s 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.295+0000 7fbef6c00640 1 -- 172.21.15.18:0/2502466095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101950 msgr2=0x7fbef0105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.295+0000 7fbef6c00640 1 --2- 172.21.15.18:0/2502466095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101950 0x7fbef0105e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009860 tx=0x7fbedc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.295+0000 7fbef6c00640 1 -- 172.21.15.18:0/2502466095 shutdown_connections 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.295+0000 7fbef6c00640 1 --2- 172.21.15.18:0/2502466095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0106430 0x7fbef01068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.295+0000 7fbef6c00640 1 --2- 172.21.15.18:0/2502466095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101950 0x7fbef0105e10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.295+0000 7fbef6c00640 1 --2- 172.21.15.18:0/2502466095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbef0101030 0x7fbef0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.295+0000 7fbef6c00640 1 -- 172.21.15.18:0/2502466095 >> 172.21.15.18:0/2502466095 conn(0x7fbef00fca20 msgr2=0x7fbef00fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.296+0000 7fbef6c00640 1 -- 172.21.15.18:0/2502466095 shutdown_connections 2024-04-16T17:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.296+0000 7fbef6c00640 1 -- 172.21.15.18:0/2502466095 wait complete. 2024-04-16T17:06:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.297+0000 7fbef6c00640 1 Processor -- start 2024-04-16T17:06:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.297+0000 7fbef6c00640 1 -- start start 2024-04-16T17:06:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbef6c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101030 0x7fbef019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbef6c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0101950 0x7fbef019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbef6c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbef0106430 0x7fbef01a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbef6c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0118a60 con 0x7fbef0106430 2024-04-16T17:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbef6c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01188e0 con 0x7fbef0101030 2024-04-16T17:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbef6c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0118be0 con 0x7fbef0101950 2024-04-16T17:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101030 0x7fbef019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101030 0x7fbef019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46478/0 (socket says 172.21.15.18:46478) 2024-04-16T17:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbeefe00640 1 -- 172.21.15.18:0/3137451486 learned_addr learned my addr 172.21.15.18:0/3137451486 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.298+0000 7fbeef400640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0101950 0x7fbef019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.299+0000 7fbef4c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbef0106430 0x7fbef01a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.299+0000 7fbeefe00640 1 -- 172.21.15.18:0/3137451486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0101950 msgr2=0x7fbef019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.299+0000 7fbeefe00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0101950 0x7fbef019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.299+0000 7fbeefe00640 1 -- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbef0106430 msgr2=0x7fbef01a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.299+0000 7fbeefe00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbef0106430 0x7fbef01a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.299+0000 7fbeefe00640 1 -- 172.21.15.18:0/3137451486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef01a3f30 con 0x7fbef0101030 2024-04-16T17:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.299+0000 7fbeef400640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0101950 0x7fbef019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.299+0000 7fbeefe00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101030 0x7fbef019ef40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbee000e9f0 tx=0x7fbee000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.300+0000 7fbeecc00640 1 -- 172.21.15.18:0/3137451486 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee000cd80 con 0x7fbef0101030 2024-04-16T17:06:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.300+0000 7fbeecc00640 1 -- 172.21.15.18:0/3137451486 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee0010910 con 0x7fbef0101030 2024-04-16T17:06:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.300+0000 7fbeecc00640 1 -- 172.21.15.18:0/3137451486 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0018990 con 0x7fbef0101030 2024-04-16T17:06:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.300+0000 7fbef6c00640 1 -- 172.21.15.18:0/3137451486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef01a4220 con 0x7fbef0101030 2024-04-16T17:06:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.300+0000 7fbef6c00640 1 -- 172.21.15.18:0/3137451486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef01abab0 con 0x7fbef0101030 2024-04-16T17:06:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.304+0000 7fbef9a38740 1 Processor -- start 2024-04-16T17:06:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.305+0000 7fbef9a38740 1 -- start start 2024-04-16T17:06:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.305+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d9610 0x55c4e79d8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.306+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79d8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.306+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79e1e20 0x55c4e79e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.306+0000 7fbef9a38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c4e793d780 con 0x55c4e79d9610 2024-04-16T17:06:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.306+0000 7fbef9a38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c4e78e3f40 con 0x55c4e79e1e20 2024-04-16T17:06:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.306+0000 7fbef9a38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7939740 con 0x55c4e79d88a0 2024-04-16T17:06:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.306+0000 7fbeef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79e1e20 0x55c4e79e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.306+0000 7fbeef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79e1e20 0x55c4e79e4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46490/0 (socket says 172.21.15.18:46490) 2024-04-16T17:06:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.306+0000 7fbeef400640 1 -- 172.21.15.18:0/926937260 learned_addr learned my addr 172.21.15.18:0/926937260 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.307+0000 7fbeef400640 1 -- 172.21.15.18:0/926937260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 msgr2=0x55c4e79d8d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:06:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.307+0000 7fbeef400640 1 --2- 172.21.15.18:0/926937260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79d8d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.307+0000 7fbeef400640 1 -- 172.21.15.18:0/926937260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d9610 msgr2=0x55c4e79d8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.307+0000 7fbeef400640 1 --2- 172.21.15.18:0/926937260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d9610 0x55c4e79d8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.307+0000 7fbeef400640 1 -- 172.21.15.18:0/926937260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e7650210 con 0x55c4e79e1e20 2024-04-16T17:06:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.307+0000 7fbeef400640 1 --2- 172.21.15.18:0/926937260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79e1e20 0x55c4e79e4210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009860 tx=0x7fbedc0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cefb32125f3f55a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.307+0000 7fbeecc00640 1 -- 172.21.15.18:0/3137451486 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbee0010ac0 con 0x7fbef0101030 2024-04-16T17:06:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.309+0000 7fbee5200640 1 -- 172.21.15.18:0/926937260 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc044070 con 0x55c4e79e1e20 2024-04-16T17:06:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.309+0000 7fbeecc00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbed40a3e10 0x7fbed40a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.309+0000 7fbee5200640 1 -- 172.21.15.18:0/926937260 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc034920 con 0x55c4e79e1e20 2024-04-16T17:06:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.309+0000 7fbee5200640 1 -- 172.21.15.18:0/926937260 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc039040 con 0x55c4e79e1e20 2024-04-16T17:06:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.309+0000 7fbeecc00640 1 -- 172.21.15.18:0/3137451486 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fbee00d0f40 con 0x7fbef0101030 2024-04-16T17:06:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.310+0000 7fbef4c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbed40a3e10 0x7fbed40a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.310+0000 7fbef9a38740 1 -- 172.21.15.18:0/926937260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79e1e20 msgr2=0x55c4e79e4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef4c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbed40a3e10 0x7fbed40a62d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005c40 tx=0x7fbee8005bd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef9a38740 1 --2- 172.21.15.18:0/926937260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79e1e20 0x55c4e79e4210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009860 tx=0x7fbedc0056d0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef9a38740 1 -- 172.21.15.18:0/926937260 shutdown_connections 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef9a38740 1 --2- 172.21.15.18:0/926937260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79e1e20 0x55c4e79e4210 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef9a38740 1 --2- 172.21.15.18:0/926937260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79d8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef9a38740 1 --2- 172.21.15.18:0/926937260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d9610 0x55c4e79d8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef9a38740 1 -- 172.21.15.18:0/926937260 >> 172.21.15.18:0/926937260 conn(0x55c4e79735d0 msgr2=0x55c4e79d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef9a38740 1 -- 172.21.15.18:0/926937260 shutdown_connections 2024-04-16T17:06:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.311+0000 7fbef9a38740 1 -- 172.21.15.18:0/926937260 wait complete. 2024-04-16T17:06:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.312+0000 7fbef9a38740 1 -- start start 2024-04-16T17:06:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.312+0000 7fbef9a38740 1 Processor -- start 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.313+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d88a0 0x55c4e79decd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.313+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d9610 0x55c4e79df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.313+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79df750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.313+0000 7fbef9a38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c4e79e4af0 con 0x55c4e79e1e20 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.313+0000 7fbef9a38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c4e78f7390 con 0x55c4e79d88a0 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.313+0000 7fbef9a38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4e79e4c70 con 0x55c4e79d9610 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.313+0000 7fbef9a38740 10 client.0 _ll_register_callbacks cb 0x55c4e78e0f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d88a0 0x55c4e79decd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d88a0 0x55c4e79decd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46496/0 (socket says 172.21.15.18:46496) 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeefe00640 1 -- 172.21.15.18:0/3142304057 learned_addr learned my addr 172.21.15.18:0/3142304057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbef4c00640 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79df750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeef400640 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d9610 0x55c4e79df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeefe00640 1 -- 172.21.15.18:0/3142304057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d9610 msgr2=0x55c4e79df210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeefe00640 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d9610 0x55c4e79df210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeefe00640 1 -- 172.21.15.18:0/3142304057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 msgr2=0x55c4e79df750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeefe00640 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79df750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.314+0000 7fbeefe00640 1 -- 172.21.15.18:0/3142304057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e771c620 con 0x55c4e79d88a0 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.315+0000 7fbeefe00640 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d88a0 0x55c4e79decd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ce2e0 tx=0x7fbee009c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83419f7e85c33d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.315+0000 7fbed9600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.315+0000 7fbed9600640 1 -- 172.21.15.18:0/3142304057 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00a0e80 con 0x55c4e79d88a0 2024-04-16T17:06:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.315+0000 7fbed9600640 1 -- 172.21.15.18:0/3142304057 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fbee00ce500 con 0x55c4e79d88a0 2024-04-16T17:06:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.315+0000 7fbef9a38740 1 -- 172.21.15.18:0/3142304057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbedc034920 con 0x55c4e79d88a0 2024-04-16T17:06:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.316+0000 7fbed9600640 1 -- 172.21.15.18:0/3142304057 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbee00a5740 con 0x55c4e79d88a0 2024-04-16T17:06:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.316+0000 7fbef9a38740 10 client.26087 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.316+0000 7fbef9a38740 1 -- 172.21.15.18:0/3142304057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c4e79f0c90 con 0x55c4e79d88a0 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.316+0000 7fbed9600640 1 -- 172.21.15.18:0/3142304057 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc034920 con 0x55c4e79d88a0 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.316+0000 7fbed9600640 1 -- 172.21.15.18:0/3142304057 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbee0098820 con 0x55c4e79d88a0 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.316+0000 7fbef9a38740 10 client.26087 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.317+0000 7fbef9a38740 10 client.26087 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.317+0000 7fbef9a38740 20 client.26087 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.317+0000 7fbef9a38740 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4e79f2150 0x55c4e79f4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.317+0000 7fbef9a38740 4 client.26087 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.317+0000 7fbef9a38740 1 -- 172.21.15.18:0/3142304057 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4e794b1a0 con 0x55c4e79f2150 2024-04-16T17:06:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.317+0000 7fbeef400640 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4e79f2150 0x55c4e79f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.318+0000 7fbeef400640 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4e79f2150 0x55c4e79f4540 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32872a0845953716 server_cookie=ba2d9b5aa7b14578 in_seq=0 out_seq=0 2024-04-16T17:06:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.318+0000 7fbed9600640 10 client.26087 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.318+0000 7fbed9600640 1 -- 172.21.15.18:0/3142304057 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4e794b1a0 con 0x55c4e79f2150 2024-04-16T17:06:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.318+0000 7fbed9600640 10 client.26087 handle_command_reply: tid=0 2024-04-16T17:06:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.319+0000 7fbef9a38740 1 client.26087 shutdown 2024-04-16T17:06:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.319+0000 7fbef9a38740 1 -- 172.21.15.18:0/3142304057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d88a0 msgr2=0x55c4e79decd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.319+0000 7fbef9a38740 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d88a0 0x55c4e79decd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ce2e0 tx=0x7fbee009c4b0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.319+0000 7fbef9a38740 1 -- 172.21.15.18:0/3142304057 shutdown_connections 2024-04-16T17:06:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.319+0000 7fbef9a38740 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4e79f2150 0x55c4e79f4540 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.320+0000 7fbef9a38740 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79df750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.320+0000 7fbef9a38740 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d9610 0x55c4e79df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.320+0000 7fbef9a38740 1 --2- 172.21.15.18:0/3142304057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d88a0 0x55c4e79decd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.320+0000 7fbef9a38740 1 -- 172.21.15.18:0/3142304057 >> 172.21.15.18:0/3142304057 conn(0x55c4e79735d0 msgr2=0x55c4e79d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.320+0000 7fbed9600640 0 client.26087 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.320+0000 7fbef9a38740 1 -- 172.21.15.18:0/3142304057 shutdown_connections 2024-04-16T17:06:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.320+0000 7fbef9a38740 1 -- 172.21.15.18:0/3142304057 wait complete. 2024-04-16T17:06:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.320+0000 7fbef9a38740 20 client.26087 trim_cache size 0 max 16384 2024-04-16T17:06:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.352+0000 7fbef9a38740 1 Processor -- start 2024-04-16T17:06:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbef9a38740 1 -- start start 2024-04-16T17:06:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d88a0 0x55c4e79eda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79dde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79e1e20 0x55c4e79df570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbef9a38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b8400 con 0x55c4e79d88a0 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbef9a38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b5cd0 con 0x55c4e79d9610 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbef9a38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b3b20 con 0x55c4e79e1e20 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbeef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d88a0 0x55c4e79eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbeef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79dde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46506/0 (socket says 172.21.15.18:46506) 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbeef400640 1 -- 172.21.15.18:0/4108210125 learned_addr learned my addr 172.21.15.18:0/4108210125 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.353+0000 7fbef4c00640 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79e1e20 0x55c4e79df570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbeef400640 1 -- 172.21.15.18:0/4108210125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79e1e20 msgr2=0x55c4e79df570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbeef400640 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79e1e20 0x55c4e79df570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbeef400640 1 -- 172.21.15.18:0/4108210125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d88a0 msgr2=0x55c4e79eda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbeef400640 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d88a0 0x55c4e79eda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbeef400640 1 -- 172.21.15.18:0/4108210125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e79da8d0 con 0x55c4e79d9610 2024-04-16T17:06:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbeefe00640 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d88a0 0x55c4e79eda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbeef400640 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79dde30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009780 tx=0x7fbedc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d062c4e7c753094e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbedbe00640 1 -- 172.21.15.18:0/4108210125 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc039040 con 0x55c4e79d9610 2024-04-16T17:06:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbedbe00640 1 -- 172.21.15.18:0/4108210125 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc034920 con 0x55c4e79d9610 2024-04-16T17:06:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.354+0000 7fbedbe00640 1 -- 172.21.15.18:0/4108210125 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc044040 con 0x55c4e79d9610 2024-04-16T17:06:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.355+0000 7fbef9a38740 1 -- 172.21.15.18:0/4108210125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 msgr2=0x55c4e79dde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.355+0000 7fbef9a38740 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79dde30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009780 tx=0x7fbedc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.356+0000 7fbef9a38740 1 -- 172.21.15.18:0/4108210125 shutdown_connections 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.356+0000 7fbef9a38740 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79e1e20 0x55c4e79df570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.356+0000 7fbef9a38740 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79dde30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.356+0000 7fbef9a38740 1 --2- 172.21.15.18:0/4108210125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79d88a0 0x55c4e79eda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.356+0000 7fbef9a38740 1 -- 172.21.15.18:0/4108210125 >> 172.21.15.18:0/4108210125 conn(0x55c4e79735d0 msgr2=0x55c4e79d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.356+0000 7fbef9a38740 1 -- 172.21.15.18:0/4108210125 shutdown_connections 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.356+0000 7fbef9a38740 1 -- 172.21.15.18:0/4108210125 wait complete. 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.356+0000 7fbef9a38740 1 -- start start 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.357+0000 7fbef9a38740 1 Processor -- start 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.357+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.357+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79ea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.357+0000 7fbef9a38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79db330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.357+0000 7fbef9a38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b4190 con 0x55c4e79e1e20 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.357+0000 7fbef9a38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b5600 con 0x55c4e79d9610 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.357+0000 7fbef9a38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b4860 con 0x55c4e79d88a0 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.357+0000 7fbef9a38740 10 client.0 _ll_register_callbacks cb 0x7fbee000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79e9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53606/0 (socket says 172.21.15.18:53606) 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeefe00640 1 -- 172.21.15.18:0/203056789 learned_addr learned my addr 172.21.15.18:0/203056789 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeef400640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeefe00640 1 -- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 msgr2=0x55c4e79ea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeefe00640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79ea470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeefe00640 1 -- 172.21.15.18:0/203056789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 msgr2=0x55c4e79db330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:06:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbef4c00640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79db330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeefe00640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79db330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeefe00640 1 -- 172.21.15.18:0/203056789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e79dbb60 con 0x55c4e79d88a0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbeef400640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.358+0000 7fbef4c00640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79db330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbeefe00640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79e9f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ce330 tx=0x7fbee0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b1227d61da3ab85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbed9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbed9600640 1 -- 172.21.15.18:0/203056789 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00a0e80 con 0x55c4e79d88a0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbed9600640 1 -- 172.21.15.18:0/203056789 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fbee009bc90 con 0x55c4e79d88a0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbef9a38740 1 -- 172.21.15.18:0/203056789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbedc034920 con 0x55c4e79d88a0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbed9600640 1 -- 172.21.15.18:0/203056789 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbee009c8a0 con 0x55c4e79d88a0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbef9a38740 10 client.26092 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbef9a38740 1 -- 172.21.15.18:0/203056789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c4e7a58230 con 0x55c4e79d88a0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbed9600640 1 -- 172.21.15.18:0/203056789 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc034920 con 0x55c4e79d88a0 2024-04-16T17:06:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.359+0000 7fbed9600640 1 -- 172.21.15.18:0/203056789 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbee00a2010 con 0x55c4e79d88a0 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.360+0000 7fbef9a38740 10 client.26092 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.360+0000 7fbef9a38740 10 client.26092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.360+0000 7fbef9a38740 20 client.26092 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.360+0000 7fbef9a38740 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4e7a58a00 0x55c4e7a5ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.360+0000 7fbef9a38740 4 client.26092 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.360+0000 7fbef9a38740 1 -- 172.21.15.18:0/203056789 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4e7945010 con 0x55c4e7a58a00 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.360+0000 7fbeef400640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4e7a58a00 0x55c4e7a5ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.361+0000 7fbed9600640 10 client.26092 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.361+0000 7fbeef400640 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4e7a58a00 0x55c4e7a5ae40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e843313186ca64 server_cookie=32a2e4aafe1a51ab in_seq=0 out_seq=0 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.481+0000 7fbed9600640 1 -- 172.21.15.18:0/203056789 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4e7945010 con 0x55c4e7a58a00 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.481+0000 7fbed9600640 10 client.26092 handle_command_reply: tid=0 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.481+0000 7fbef9a38740 1 client.26092 shutdown 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 -- 172.21.15.18:0/203056789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 msgr2=0x55c4e79e9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79e9f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ce330 tx=0x7fbee0000bc0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 -- 172.21.15.18:0/203056789 shutdown_connections 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4e7a58a00 0x55c4e7a5ae40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4e79e1e20 0x55c4e79db330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4e79d9610 0x55c4e79ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 --2- 172.21.15.18:0/203056789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4e79d88a0 0x55c4e79e9f30 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ce330 tx=0x7fbee0000bc0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 -- 172.21.15.18:0/203056789 >> 172.21.15.18:0/203056789 conn(0x55c4e79735d0 msgr2=0x55c4e79d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbed9600640 0 client.26092 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 -- 172.21.15.18:0/203056789 shutdown_connections 2024-04-16T17:06:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 1 -- 172.21.15.18:0/203056789 wait complete. 2024-04-16T17:06:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.482+0000 7fbef9a38740 20 client.26092 trim_cache size 0 max 16384 2024-04-16T17:06:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:06:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:06:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:06:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:06:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 -- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbed40a3e10 msgr2=0x7fbed40a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbed40a3e10 0x7fbed40a62d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005c40 tx=0x7fbee8005bd0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 -- 172.21.15.18:0/3137451486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101030 msgr2=0x7fbef019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101030 0x7fbef019ef40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbee000e9f0 tx=0x7fbee000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 -- 172.21.15.18:0/3137451486 shutdown_connections 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbed40a3e10 0x7fbed40a62d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbef0106430 0x7fbef01a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0101950 0x7fbef019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 --2- 172.21.15.18:0/3137451486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbef0101030 0x7fbef019ef40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 -- 172.21.15.18:0/3137451486 >> 172.21.15.18:0/3137451486 conn(0x7fbef00fca20 msgr2=0x7fbef00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 -- 172.21.15.18:0/3137451486 shutdown_connections 2024-04-16T17:06:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:30.483+0000 7fbef6c00640 1 -- 172.21.15.18:0/3137451486 wait complete. 2024-04-16T17:06:30.514 INFO:tasks.workunit.client.0.smithi018.stdout: 1 33128 4.54 MB/sec warmup 43 sec latency 388.922 ms 2024-04-16T17:06:31.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 34443 4.56 MB/sec warmup 44 sec latency 11.220 ms 2024-04-16T17:06:31.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:31 smithi022 ceph-mon[36023]: pgmap v56: 129 pgs: 129 active+clean; 34 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-04-16T17:06:31.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:31 smithi043 ceph-mon[36901]: pgmap v56: 129 pgs: 129 active+clean; 34 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-04-16T17:06:32.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 35736 4.71 MB/sec warmup 45 sec latency 11.588 ms 2024-04-16T17:06:33.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 35950 4.61 MB/sec warmup 46 sec latency 47.458 ms 2024-04-16T17:06:33.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:33 smithi022 ceph-mon[36023]: pgmap v57: 129 pgs: 129 active+clean; 36 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1005 KiB/s wr, 71 op/s 2024-04-16T17:06:33.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:33 smithi043 ceph-mon[36901]: pgmap v57: 129 pgs: 129 active+clean; 36 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1005 KiB/s wr, 71 op/s 2024-04-16T17:06:34.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 36264 4.52 MB/sec warmup 47 sec latency 61.730 ms 2024-04-16T17:06:35.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:06:35.251 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:06:35.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 -- 172.21.15.18:0/1159754728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0104280 msgr2=0x7f71d0104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 --2- 172.21.15.18:0/1159754728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0104280 0x7f71d0104660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009900 tx=0x7f71c402bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 -- 172.21.15.18:0/1159754728 shutdown_connections 2024-04-16T17:06:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 --2- 172.21.15.18:0/1159754728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71d0109350 0x7f71d0110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 --2- 172.21.15.18:0/1159754728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71d0104c30 0x7f71d0108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 --2- 172.21.15.18:0/1159754728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0104280 0x7f71d0104660 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 -- 172.21.15.18:0/1159754728 >> 172.21.15.18:0/1159754728 conn(0x7f71d00ffae0 msgr2=0x7f71d0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 -- 172.21.15.18:0/1159754728 shutdown_connections 2024-04-16T17:06:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.481+0000 7f71d6000640 1 -- 172.21.15.18:0/1159754728 wait complete. 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.482+0000 7f71d6000640 1 Processor -- start 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.482+0000 7f71d6000640 1 -- start start 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d6000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71d0104280 0x7f71d01a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0109350 0x7f71d01a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71d019a790 0x7f71d019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d6000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0114430 con 0x7f71d0104280 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d6000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01142b0 con 0x7f71d019a790 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d6000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01145b0 con 0x7f71d0109350 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71d0104280 0x7f71d01a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71d0104280 0x7f71d01a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48966/0 (socket says 172.21.15.18:48966) 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d4c00640 1 -- 172.21.15.18:0/3496996459 learned_addr learned my addr 172.21.15.18:0/3496996459 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71cfe00640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0109350 0x7f71d01a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.483+0000 7f71d5600640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71d019a790 0x7f71d019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cfe00640 1 -- 172.21.15.18:0/3496996459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71d019a790 msgr2=0x7f71d019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cfe00640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71d019a790 0x7f71d019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cfe00640 1 -- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71d0104280 msgr2=0x7f71d01a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cfe00640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71d0104280 0x7f71d01a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cfe00640 1 -- 172.21.15.18:0/3496996459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d019b390 con 0x7f71d0109350 2024-04-16T17:06:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cfe00640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0109350 0x7f71d01a0bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00e9c0 tx=0x7f71bc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cd600640 1 -- 172.21.15.18:0/3496996459 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00cd80 con 0x7f71d0109350 2024-04-16T17:06:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cd600640 1 -- 172.21.15.18:0/3496996459 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71bc0035f0 con 0x7f71d0109350 2024-04-16T17:06:35.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71d6000640 1 -- 172.21.15.18:0/3496996459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d01a7420 con 0x7f71d0109350 2024-04-16T17:06:35.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71cd600640 1 -- 172.21.15.18:0/3496996459 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc010690 con 0x7f71d0109350 2024-04-16T17:06:35.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.484+0000 7f71d6000640 1 -- 172.21.15.18:0/3496996459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d01a7820 con 0x7f71d0109350 2024-04-16T17:06:35.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.489+0000 7f71da39c740 1 Processor -- start 2024-04-16T17:06:35.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.489+0000 7f71da39c740 1 -- start start 2024-04-16T17:06:35.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.489+0000 7f71da39c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed26770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.489+0000 7f71da39c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26cb0 0x56468ed2f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.489+0000 7f71da39c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed2f890 0x56468ed31c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.489+0000 7f71da39c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56468ec8c120 con 0x56468ed26cb0 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.489+0000 7f71da39c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56468ec32340 con 0x56468ed26390 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.489+0000 7f71da39c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56468ec880e0 con 0x56468ed2f890 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71cd600640 1 -- 172.21.15.18:0/3496996459 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71bc0037a0 con 0x7f71d0109350 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed26770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed26770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46540/0 (socket says 172.21.15.18:46540) 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d5600640 1 -- 172.21.15.18:0/3930076311 learned_addr learned my addr 172.21.15.18:0/3930076311 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d4c00640 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26cb0 0x56468ed2f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71cfe00640 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed2f890 0x56468ed31c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d5600640 1 -- 172.21.15.18:0/3930076311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed2f890 msgr2=0x56468ed31c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d5600640 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed2f890 0x56468ed31c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d5600640 1 -- 172.21.15.18:0/3930076311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26cb0 msgr2=0x56468ed2f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d5600640 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26cb0 0x56468ed2f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d5600640 1 -- 172.21.15.18:0/3930076311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468ea687d0 con 0x56468ed26390 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.490+0000 7f71d4c00640 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26cb0 0x56468ed2f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.491+0000 7f71d5600640 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed26770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009f90 tx=0x7f71c8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b085e3838a39a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.491+0000 7f71c1c00640 1 -- 172.21.15.18:0/3930076311 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8011700 con 0x56468ed26390 2024-04-16T17:06:35.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.491+0000 7f71c1c00640 1 -- 172.21.15.18:0/3930076311 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c8011d80 con 0x56468ed26390 2024-04-16T17:06:35.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.491+0000 7f71c1c00640 1 -- 172.21.15.18:0/3930076311 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8018380 con 0x56468ed26390 2024-04-16T17:06:35.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.491+0000 7f71cd600640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71b80a3d20 0x7f71b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71cd600640 1 -- 172.21.15.18:0/3496996459 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f71bc0d0dd0 con 0x7f71d0109350 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 -- 172.21.15.18:0/3930076311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 msgr2=0x56468ed26770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed26770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009f90 tx=0x7f71c8010860 comp rx=0 tx=0).stop 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71d4c00640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71b80a3d20 0x7f71b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 -- 172.21.15.18:0/3930076311 shutdown_connections 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed2f890 0x56468ed31c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26cb0 0x56468ed2f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 --2- 172.21.15.18:0/3930076311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed26770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 -- 172.21.15.18:0/3930076311 >> 172.21.15.18:0/3930076311 conn(0x56468ed20ac0 msgr2=0x56468ed20ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 -- 172.21.15.18:0/3930076311 shutdown_connections 2024-04-16T17:06:35.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.492+0000 7f71da39c740 1 -- 172.21.15.18:0/3930076311 wait complete. 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.493+0000 7f71d4c00640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71b80a3d20 0x7f71b80a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x56468ed25e60 tx=0x7f71c402df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.493+0000 7f71da39c740 1 -- start start 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.493+0000 7f71da39c740 1 Processor -- start 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71da39c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26390 0x56468ed2c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71da39c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26cb0 0x56468ed2ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71da39c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed2d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71da39c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56468ed32680 con 0x56468ed2f890 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71da39c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56468ec45710 con 0x56468ed26cb0 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71da39c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56468ed32800 con 0x56468ed26390 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71da39c740 10 client.0 _ll_register_callbacks cb 0x56468ec40af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71d4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26390 0x56468ed2c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71d4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26390 0x56468ed2c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53630/0 (socket says 172.21.15.18:53630) 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71d4c00640 1 -- 172.21.15.18:0/4155349410 learned_addr learned my addr 172.21.15.18:0/4155349410 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71cfe00640 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26cb0 0x56468ed2ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.494+0000 7f71d5600640 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed2d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.495+0000 7f71d4c00640 1 -- 172.21.15.18:0/4155349410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26cb0 msgr2=0x56468ed2ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:35.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.495+0000 7f71d4c00640 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26cb0 0x56468ed2ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.495+0000 7f71d4c00640 1 -- 172.21.15.18:0/4155349410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 msgr2=0x56468ed2d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:35.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.495+0000 7f71d4c00640 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed2d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.495+0000 7f71d4c00640 1 -- 172.21.15.18:0/4155349410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468e99e3f0 con 0x56468ed26390 2024-04-16T17:06:35.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.495+0000 7f71d5600640 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed2d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:35.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.495+0000 7f71d4c00640 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26390 0x56468ed2c940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f71c402cb30 tx=0x7f71c4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333a53b9a7b4d7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.495+0000 7f71b6000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:35.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.496+0000 7f71b6000640 1 -- 172.21.15.18:0/4155349410 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4045070 con 0x56468ed26390 2024-04-16T17:06:35.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.496+0000 7f71b6000640 1 -- 172.21.15.18:0/4155349410 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f71c40046c0 con 0x56468ed26390 2024-04-16T17:06:35.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.496+0000 7f71da39c740 1 -- 172.21.15.18:0/4155349410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c8011d80 con 0x56468ed26390 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.496+0000 7f71b6000640 1 -- 172.21.15.18:0/4155349410 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71c40048c0 con 0x56468ed26390 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.496+0000 7f71da39c740 10 client.26104 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.496+0000 7f71da39c740 1 -- 172.21.15.18:0/4155349410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56468e95e0f0 con 0x56468ed26390 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.496+0000 7f71b6000640 1 -- 172.21.15.18:0/4155349410 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c8011d80 con 0x56468ed26390 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.496+0000 7f71b6000640 1 -- 172.21.15.18:0/4155349410 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f71c40037f0 con 0x56468ed26390 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71da39c740 10 client.26104 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71da39c740 10 client.26104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71da39c740 20 client.26104 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71da39c740 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56468ed3fcf0 0x56468ed42160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71da39c740 4 client.26104 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71da39c740 1 -- 172.21.15.18:0/4155349410 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468ec99b40 con 0x56468ed3fcf0 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71cfe00640 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56468ed3fcf0 0x56468ed42160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71cfe00640 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56468ed3fcf0 0x56468ed42160 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece322043413a647 server_cookie=3aee3a8f70d51bba in_seq=0 out_seq=0 2024-04-16T17:06:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.497+0000 7f71b6000640 10 client.26104 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:35.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.499+0000 7f71b6000640 1 -- 172.21.15.18:0/4155349410 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56468ec99b40 con 0x56468ed3fcf0 2024-04-16T17:06:35.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.499+0000 7f71b6000640 10 client.26104 handle_command_reply: tid=0 2024-04-16T17:06:35.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.499+0000 7f71da39c740 1 client.26104 shutdown 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 -- 172.21.15.18:0/4155349410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26390 msgr2=0x56468ed2c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26390 0x56468ed2c940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f71c402cb30 tx=0x7f71c4009260 comp rx=0 tx=0).stop 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 -- 172.21.15.18:0/4155349410 shutdown_connections 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56468ed3fcf0 0x56468ed42160 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed2d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26cb0 0x56468ed2ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 --2- 172.21.15.18:0/4155349410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26390 0x56468ed2c940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71b6000640 0 client.26104 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 -- 172.21.15.18:0/4155349410 >> 172.21.15.18:0/4155349410 conn(0x56468ed20ac0 msgr2=0x56468ed22810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 -- 172.21.15.18:0/4155349410 shutdown_connections 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 1 -- 172.21.15.18:0/4155349410 wait complete. 2024-04-16T17:06:35.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.500+0000 7f71da39c740 20 client.26104 trim_cache size 0 max 16384 2024-04-16T17:06:35.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 37511 4.54 MB/sec warmup 48 sec latency 25.183 ms 2024-04-16T17:06:35.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.536+0000 7f71da39c740 1 Processor -- start 2024-04-16T17:06:35.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.536+0000 7f71da39c740 1 -- start start 2024-04-16T17:06:35.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.536+0000 7f71da39c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed2f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.536+0000 7f71da39c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed2baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.536+0000 7f71da39c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed3b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.536+0000 7f71da39c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01b3ea0 con 0x56468ed2f890 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.536+0000 7f71da39c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01b1770 con 0x56468ed26390 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.536+0000 7f71da39c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01af5c0 con 0x56468ed26cb0 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed2baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed2baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53640/0 (socket says 172.21.15.18:53640) 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 -- 172.21.15.18:0/3717613415 learned_addr learned my addr 172.21.15.18:0/3717613415 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71d4c00640 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed2f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71d5600640 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed3b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 -- 172.21.15.18:0/3717613415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 msgr2=0x56468ed2f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed2f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 -- 172.21.15.18:0/3717613415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 msgr2=0x56468ed3b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed3b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 -- 172.21.15.18:0/3717613415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468ed3ba80 con 0x56468ed26cb0 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71d4c00640 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed2f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.537+0000 7f71cfe00640 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed2baa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00c6a0 tx=0x7f71bc0d32e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6860d627b9ca219 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.538+0000 7f71c1200640 1 -- 172.21.15.18:0/3717613415 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc09c9c0 con 0x56468ed26cb0 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.538+0000 7f71c1200640 1 -- 172.21.15.18:0/3717613415 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71bc0a0d70 con 0x56468ed26cb0 2024-04-16T17:06:35.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.538+0000 7f71c1200640 1 -- 172.21.15.18:0/3717613415 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc099030 con 0x56468ed26cb0 2024-04-16T17:06:35.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.538+0000 7f71da39c740 1 -- 172.21.15.18:0/3717613415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 msgr2=0x56468ed2baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.538+0000 7f71da39c740 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed2baa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00c6a0 tx=0x7f71bc0d32e0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 -- 172.21.15.18:0/3717613415 shutdown_connections 2024-04-16T17:06:35.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed2f890 0x56468ed3b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed2baa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 --2- 172.21.15.18:0/3717613415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed26390 0x56468ed2f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 -- 172.21.15.18:0/3717613415 >> 172.21.15.18:0/3717613415 conn(0x56468ed20ac0 msgr2=0x56468ed1f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 -- 172.21.15.18:0/3717613415 shutdown_connections 2024-04-16T17:06:35.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 -- 172.21.15.18:0/3717613415 wait complete. 2024-04-16T17:06:35.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 -- start start 2024-04-16T17:06:35.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.539+0000 7f71da39c740 1 Processor -- start 2024-04-16T17:06:35.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71da39c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26390 0x56468ed37e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71da39c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71da39c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed2f890 0x56468ed33700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71da39c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01afc30 con 0x56468ed26390 2024-04-16T17:06:35.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71da39c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01b10a0 con 0x56468ed2f890 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71da39c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01b0300 con 0x56468ed26cb0 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71da39c740 10 client.0 _ll_register_callbacks cb 0x7f71c402c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71d4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26390 0x56468ed37e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed331c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53648/0 (socket says 172.21.15.18:53648) 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71cfe00640 1 -- 172.21.15.18:0/2084856169 learned_addr learned my addr 172.21.15.18:0/2084856169 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.540+0000 7f71d5600640 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed2f890 0x56468ed33700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71cfe00640 1 -- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed2f890 msgr2=0x56468ed33700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71cfe00640 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed2f890 0x56468ed33700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71cfe00640 1 -- 172.21.15.18:0/2084856169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26390 msgr2=0x56468ed37e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71cfe00640 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26390 0x56468ed37e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71cfe00640 1 -- 172.21.15.18:0/2084856169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468ed33f30 con 0x56468ed26cb0 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71d5600640 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed2f890 0x56468ed33700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71cfe00640 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed331c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0ce220 tx=0x7f71bc0a5630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e09906173a7fdec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71b6000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71b6000640 1 -- 172.21.15.18:0/2084856169 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc09c9c0 con 0x56468ed26cb0 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71da39c740 1 -- 172.21.15.18:0/2084856169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc0a0d70 con 0x56468ed26cb0 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.541+0000 7f71b6000640 1 -- 172.21.15.18:0/2084856169 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f71bc003b30 con 0x56468ed26cb0 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71b6000640 1 -- 172.21.15.18:0/2084856169 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71bc0ce3f0 con 0x56468ed26cb0 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71da39c740 10 client.26110 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71da39c740 1 -- 172.21.15.18:0/2084856169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56468eda6310 con 0x56468ed26cb0 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71b6000640 1 -- 172.21.15.18:0/2084856169 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71bc0a0d70 con 0x56468ed26cb0 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71b6000640 1 -- 172.21.15.18:0/2084856169 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f71bc0870b0 con 0x56468ed26cb0 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71da39c740 10 client.26110 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71da39c740 10 client.26110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71da39c740 20 client.26110 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71da39c740 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56468eda6980 0x56468eda8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71da39c740 4 client.26110 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:06:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.542+0000 7f71da39c740 1 -- 172.21.15.18:0/2084856169 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56468ec939b0 con 0x56468eda6980 2024-04-16T17:06:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.543+0000 7f71d4c00640 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56468eda6980 0x56468eda8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.544+0000 7f71d4c00640 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56468eda6980 0x56468eda8d70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9874bb4b4d982e40 server_cookie=6fe4fba984fa3684 in_seq=0 out_seq=0 2024-04-16T17:06:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.544+0000 7f71b6000640 10 client.26110 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.544+0000 7f71b6000640 1 -- 172.21.15.18:0/2084856169 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56468ec939b0 con 0x56468eda6980 2024-04-16T17:06:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.544+0000 7f71b6000640 10 client.26110 handle_command_reply: tid=0 2024-04-16T17:06:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.544+0000 7f71da39c740 1 client.26110 shutdown 2024-04-16T17:06:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.544+0000 7f71da39c740 1 -- 172.21.15.18:0/2084856169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 msgr2=0x56468ed331c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.544+0000 7f71da39c740 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed331c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0ce220 tx=0x7f71bc0a5630 comp rx=0 tx=0).stop 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 1 -- 172.21.15.18:0/2084856169 shutdown_connections 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56468eda6980 0x56468eda8d70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468ed2f890 0x56468ed33700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56468ed26cb0 0x56468ed331c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 1 --2- 172.21.15.18:0/2084856169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56468ed26390 0x56468ed37e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71b6000640 0 client.26110 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 1 -- 172.21.15.18:0/2084856169 >> 172.21.15.18:0/2084856169 conn(0x56468ed20ac0 msgr2=0x56468ed1f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 1 -- 172.21.15.18:0/2084856169 shutdown_connections 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 1 -- 172.21.15.18:0/2084856169 wait complete. 2024-04-16T17:06:35.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.548+0000 7f71da39c740 20 client.26110 trim_cache size 0 max 16384 2024-04-16T17:06:35.549 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:06:35.549 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:06:35.549 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:06:35.549 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:06:35.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 -- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71b80a3d20 msgr2=0x7f71b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71b80a3d20 0x7f71b80a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x56468ed25e60 tx=0x7f71c402df90 comp rx=0 tx=0).stop 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 -- 172.21.15.18:0/3496996459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0109350 msgr2=0x7f71d01a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0109350 0x7f71d01a0bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00e9c0 tx=0x7f71bc00ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 -- 172.21.15.18:0/3496996459 shutdown_connections 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71b80a3d20 0x7f71b80a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71d019a790 0x7f71d019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d0109350 0x7f71d01a0bc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 --2- 172.21.15.18:0/3496996459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71d0104280 0x7f71d01a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 -- 172.21.15.18:0/3496996459 >> 172.21.15.18:0/3496996459 conn(0x7f71d00ffae0 msgr2=0x7f71d0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 -- 172.21.15.18:0/3496996459 shutdown_connections 2024-04-16T17:06:35.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.549+0000 7f71d6000640 1 -- 172.21.15.18:0/3496996459 wait complete. 2024-04-16T17:06:35.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:06:35.572 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:06:35.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:35 smithi022 ceph-mon[36023]: pgmap v58: 129 pgs: 129 active+clean; 36 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 992 KiB/s wr, 67 op/s 2024-04-16T17:06:35.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:35 smithi043 ceph-mon[36901]: pgmap v58: 129 pgs: 129 active+clean; 36 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 992 KiB/s wr, 67 op/s 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 -- 172.21.15.18:0/4083566952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8bc103090 msgr2=0x7fb8bc103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/4083566952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8bc103090 0x7fb8bc103470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009900 tx=0x7fb8ac02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 -- 172.21.15.18:0/4083566952 shutdown_connections 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/4083566952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc10c4b0 0x7fb8bc10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/4083566952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8bc1039b0 0x7fb8bc10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/4083566952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8bc103090 0x7fb8bc103470 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 -- 172.21.15.18:0/4083566952 >> 172.21.15.18:0/4083566952 conn(0x7fb8bc0fc890 msgr2=0x7fb8bc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 -- 172.21.15.18:0/4083566952 shutdown_connections 2024-04-16T17:06:35.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.769+0000 7fb8c0a00640 1 -- 172.21.15.18:0/4083566952 wait complete. 2024-04-16T17:06:35.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.770+0000 7fb8c0a00640 1 Processor -- start 2024-04-16T17:06:35.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.770+0000 7fb8c0a00640 1 -- start start 2024-04-16T17:06:35.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.770+0000 7fb8c0a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8bc103090 0x7fb8bc1091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.770+0000 7fb8c0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc1039b0 0x7fb8bc109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8c0a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8bc10c4b0 0x7fb8bc109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8c0a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc102960 con 0x7fb8bc10c4b0 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8c0a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc1027e0 con 0x7fb8bc103090 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8c0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc102ae0 con 0x7fb8bc1039b0 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8bb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8bc103090 0x7fb8bc1091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8bb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8bc103090 0x7fb8bc1091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46596/0 (socket says 172.21.15.18:46596) 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8bb400640 1 -- 172.21.15.18:0/963207620 learned_addr learned my addr 172.21.15.18:0/963207620 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8b3e00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc1039b0 0x7fb8bc109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8bbe00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8bc10c4b0 0x7fb8bc109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8b3e00640 1 -- 172.21.15.18:0/963207620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8bc103090 msgr2=0x7fb8bc1091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8b3e00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8bc103090 0x7fb8bc1091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8b3e00640 1 -- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8bc10c4b0 msgr2=0x7fb8bc109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8b3e00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8bc10c4b0 0x7fb8bc109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8b3e00640 1 -- 172.21.15.18:0/963207620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc1ab0f0 con 0x7fb8bc1039b0 2024-04-16T17:06:35.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.771+0000 7fb8bb400640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8bc103090 0x7fb8bc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:35.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.772+0000 7fb8b3e00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc1039b0 0x7fb8bc109700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400e9f0 tx=0x7fb8a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.772+0000 7fb8b8c00640 1 -- 172.21.15.18:0/963207620 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a400ccc0 con 0x7fb8bc1039b0 2024-04-16T17:06:35.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.772+0000 7fb8b8c00640 1 -- 172.21.15.18:0/963207620 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a40035f0 con 0x7fb8bc1039b0 2024-04-16T17:06:35.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.772+0000 7fb8b8c00640 1 -- 172.21.15.18:0/963207620 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a400ce60 con 0x7fb8bc1039b0 2024-04-16T17:06:35.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.772+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8bc1ab3e0 con 0x7fb8bc1039b0 2024-04-16T17:06:35.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.772+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8bc1ab980 con 0x7fb8bc1039b0 2024-04-16T17:06:35.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.774+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8bc069be0 con 0x7fb8bc1039b0 2024-04-16T17:06:35.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.776+0000 7fb8b8c00640 1 -- 172.21.15.18:0/963207620 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb8a4003b30 con 0x7fb8bc1039b0 2024-04-16T17:06:35.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.778+0000 7fb8b8c00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8a00a3d90 0x7fb8a00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.778+0000 7fb8bb400640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8a00a3d90 0x7fb8a00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.778+0000 7fb8b8c00640 1 -- 172.21.15.18:0/963207620 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb8a40d0df0 con 0x7fb8bc1039b0 2024-04-16T17:06:35.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.780+0000 7fb8bb400640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8a00a3d90 0x7fb8a00a6250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0096c0 tx=0x7fb8ac005a20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:35.781+0000 7fb8b8c00640 1 -- 172.21.15.18:0/963207620 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8a409d080 con 0x7fb8bc1039b0 2024-04-16T17:06:36.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.136+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8bc113fb0 con 0x7fb8bc1039b0 2024-04-16T17:06:36.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.137+0000 7fb8b8c00640 1 -- 172.21.15.18:0/963207620 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7fb8a409c980 con 0x7fb8bc1039b0 2024-04-16T17:06:36.138 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:06:36.138 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:06:36.138 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:06:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.141+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8a00a3d90 msgr2=0x7fb8a00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.141+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8a00a3d90 0x7fb8a00a6250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0096c0 tx=0x7fb8ac005a20 comp rx=0 tx=0).stop 2024-04-16T17:06:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.141+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc1039b0 msgr2=0x7fb8bc109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.141+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc1039b0 0x7fb8bc109700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400e9f0 tx=0x7fb8a400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.142+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 shutdown_connections 2024-04-16T17:06:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.142+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8a00a3d90 0x7fb8a00a6250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.142+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8bc10c4b0 0x7fb8bc109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.142+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc1039b0 0x7fb8bc109700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.142+0000 7fb8c0a00640 1 --2- 172.21.15.18:0/963207620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8bc103090 0x7fb8bc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.142+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 >> 172.21.15.18:0/963207620 conn(0x7fb8bc0fc890 msgr2=0x7fb8bc0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.142+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 shutdown_connections 2024-04-16T17:06:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.142+0000 7fb8c0a00640 1 -- 172.21.15.18:0/963207620 wait complete. 2024-04-16T17:06:36.167 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:06:36.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.393+0000 7faba6000640 1 -- 172.21.15.18:0/2305210225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faba00743a0 msgr2=0x7faba0074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.393+0000 7faba6000640 1 --2- 172.21.15.18:0/2305210225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faba00743a0 0x7faba0074780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab9402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.394+0000 7faba6000640 1 -- 172.21.15.18:0/2305210225 shutdown_connections 2024-04-16T17:06:36.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.394+0000 7faba6000640 1 --2- 172.21.15.18:0/2305210225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faba0112e10 0x7faba0115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.394+0000 7faba6000640 1 --2- 172.21.15.18:0/2305210225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faba0074cc0 0x7faba01128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.394+0000 7faba6000640 1 --2- 172.21.15.18:0/2305210225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faba00743a0 0x7faba0074780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.394+0000 7faba6000640 1 -- 172.21.15.18:0/2305210225 >> 172.21.15.18:0/2305210225 conn(0x7faba00fd950 msgr2=0x7faba00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.394+0000 7faba6000640 1 -- 172.21.15.18:0/2305210225 shutdown_connections 2024-04-16T17:06:36.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.394+0000 7faba6000640 1 -- 172.21.15.18:0/2305210225 wait complete. 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.395+0000 7faba6000640 1 Processor -- start 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.395+0000 7faba6000640 1 -- start start 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.395+0000 7faba6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faba00743a0 0x7faba019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.395+0000 7faba6000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faba0074cc0 0x7faba019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faba0112e10 0x7faba01a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba6000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faba0118850 con 0x7faba0074cc0 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba6000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faba01186d0 con 0x7faba00743a0 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba6000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faba01189d0 con 0x7faba0112e10 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faba00743a0 0x7faba019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faba00743a0 0x7faba019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46618/0 (socket says 172.21.15.18:46618) 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba4c00640 1 -- 172.21.15.18:0/1023816051 learned_addr learned my addr 172.21.15.18:0/1023816051 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba5600640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faba0112e10 0x7faba01a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7fab9fe00640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faba0074cc0 0x7faba019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba4c00640 1 -- 172.21.15.18:0/1023816051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faba0112e10 msgr2=0x7faba01a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba4c00640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faba0112e10 0x7faba01a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba4c00640 1 -- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faba0074cc0 msgr2=0x7faba019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba4c00640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faba0074cc0 0x7faba019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7faba4c00640 1 -- 172.21.15.18:0/1023816051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba01a3d60 con 0x7faba00743a0 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.396+0000 7fab9fe00640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faba0074cc0 0x7faba019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.397+0000 7faba4c00640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faba00743a0 0x7faba019ed70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab94000c00 tx=0x7fab940043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.397+0000 7fab9d600640 1 -- 172.21.15.18:0/1023816051 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94042020 con 0x7faba00743a0 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.397+0000 7fab9d600640 1 -- 172.21.15.18:0/1023816051 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab94005000 con 0x7faba00743a0 2024-04-16T17:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.397+0000 7faba6000640 1 -- 172.21.15.18:0/1023816051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba01a3ff0 con 0x7faba00743a0 2024-04-16T17:06:36.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.397+0000 7fab9d600640 1 -- 172.21.15.18:0/1023816051 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94003b00 con 0x7faba00743a0 2024-04-16T17:06:36.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.397+0000 7faba6000640 1 -- 172.21.15.18:0/1023816051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba01ab8e0 con 0x7faba00743a0 2024-04-16T17:06:36.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.400+0000 7fab9d600640 1 -- 172.21.15.18:0/1023816051 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab94045020 con 0x7faba00743a0 2024-04-16T17:06:36.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.401+0000 7fabaa3e4740 1 Processor -- start 2024-04-16T17:06:36.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.401+0000 7fabaa3e4740 1 -- start start 2024-04-16T17:06:36.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.401+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 0x55bf982832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.401+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98283830 0x55bf9828bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.401+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf9828e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.401+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bf981e8020 con 0x55bf98282f10 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.401+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bf9818e460 con 0x55bf98283830 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.401+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf981e3fe0 con 0x55bf9828c410 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9d600640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab880a3d20 0x7fab880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf9828e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf9828e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53698/0 (socket says 172.21.15.18:53698) 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 -- 172.21.15.18:0/4123605248 learned_addr learned my addr 172.21.15.18:0/4123605248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9d600640 1 -- 172.21.15.18:0/1023816051 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fab940f28f0 con 0x7faba00743a0 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7faba5600640 1 --2- 172.21.15.18:0/4123605248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 0x55bf982832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 -- 172.21.15.18:0/4123605248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98283830 msgr2=0x55bf9828bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 --2- 172.21.15.18:0/4123605248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98283830 0x55bf9828bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 -- 172.21.15.18:0/4123605248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 msgr2=0x55bf982832f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 --2- 172.21.15.18:0/4123605248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 0x55bf982832f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 -- 172.21.15.18:0/4123605248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf98196040 con 0x55bf9828c410 2024-04-16T17:06:36.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7fab9fe00640 1 --2- 172.21.15.18:0/4123605248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf9828e800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fab8c005fd0 tx=0x7fab8c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be1635b4500a454 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.402+0000 7faba5600640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab880a3d20 0x7fab880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.403+0000 7fab91c00640 1 -- 172.21.15.18:0/4123605248 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c016070 con 0x55bf9828c410 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.403+0000 7fab91c00640 1 -- 172.21.15.18:0/4123605248 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c005500 con 0x55bf9828c410 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.403+0000 7fab91c00640 1 -- 172.21.15.18:0/4123605248 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0116d0 con 0x55bf9828c410 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7faba5600640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab880a3d20 0x7fab880a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faba006b470 tx=0x7fab98014040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 -- 172.21.15.18:0/4123605248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 msgr2=0x55bf9828e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/4123605248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf9828e800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fab8c005fd0 tx=0x7fab8c00e6c0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 -- 172.21.15.18:0/4123605248 shutdown_connections 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/4123605248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf9828e800 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/4123605248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98283830 0x55bf9828bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/4123605248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 0x55bf982832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 -- 172.21.15.18:0/4123605248 >> 172.21.15.18:0/4123605248 conn(0x55bf9827d640 msgr2=0x55bf9827da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 -- 172.21.15.18:0/4123605248 shutdown_connections 2024-04-16T17:06:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.404+0000 7fabaa3e4740 1 -- 172.21.15.18:0/4123605248 wait complete. 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.405+0000 7fabaa3e4740 1 -- start start 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.405+0000 7fabaa3e4740 1 Processor -- start 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf982894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98283830 0x55bf982899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf98289f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bf9828f200 con 0x55bf98283830 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bf981a1830 con 0x55bf98282f10 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf9828f380 con 0x55bf9828c410 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7fabaa3e4740 10 client.0 _ll_register_callbacks cb 0x55bf981a9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7faba4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf982894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7faba4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf982894a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46638/0 (socket says 172.21.15.18:46638) 2024-04-16T17:06:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.406+0000 7faba4c00640 1 -- 172.21.15.18:0/1521516554 learned_addr learned my addr 172.21.15.18:0/1521516554 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7faba5600640 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf98289f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7faba4c00640 1 -- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 msgr2=0x55bf98289f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7faba4c00640 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf98289f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7faba4c00640 1 -- 172.21.15.18:0/1521516554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98283830 msgr2=0x55bf982899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7faba4c00640 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98283830 0x55bf982899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7faba4c00640 1 -- 172.21.15.18:0/1521516554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf981a5920 con 0x55bf98282f10 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7faba5600640 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf98289f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7faba4c00640 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf982894a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab940ba9f0 tx=0x7fab940be420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98f8776b5d22afe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7fab86000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7fab86000640 1 -- 172.21.15.18:0/1521516554 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940be590 con 0x55bf98282f10 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7fab86000640 1 -- 172.21.15.18:0/1521516554 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fab94003ca0 con 0x55bf98282f10 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.407+0000 7fabaa3e4740 1 -- 172.21.15.18:0/1521516554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8c005500 con 0x55bf98282f10 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fab86000640 1 -- 172.21.15.18:0/1521516554 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab940c2940 con 0x55bf98282f10 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fabaa3e4740 10 client.26123 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fabaa3e4740 1 -- 172.21.15.18:0/1521516554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf981737f0 con 0x55bf98282f10 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fab86000640 1 -- 172.21.15.18:0/1521516554 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c005500 con 0x55bf98282f10 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fab86000640 1 -- 172.21.15.18:0/1521516554 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab940efeb0 con 0x55bf98282f10 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fabaa3e4740 10 client.26123 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fabaa3e4740 10 client.26123 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fabaa3e4740 20 client.26123 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55bf9829c8a0 0x55bf9829ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fabaa3e4740 4 client.26123 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.408+0000 7fabaa3e4740 1 -- 172.21.15.18:0/1521516554 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf981f5a40 con 0x55bf9829c8a0 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.409+0000 7fab9fe00640 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55bf9829c8a0 0x55bf9829ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.409+0000 7fab9fe00640 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55bf9829c8a0 0x55bf9829ece0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=230ffcc150f67a0d server_cookie=38c7eae136fc073a in_seq=0 out_seq=0 2024-04-16T17:06:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.409+0000 7fab86000640 10 client.26123 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:06:36.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fab86000640 1 -- 172.21.15.18:0/1521516554 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf981f5a40 con 0x55bf9829c8a0 2024-04-16T17:06:36.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fab86000640 10 client.26123 handle_command_reply: tid=0 2024-04-16T17:06:36.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 client.26123 shutdown 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 -- 172.21.15.18:0/1521516554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 msgr2=0x55bf982894a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf982894a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab940ba9f0 tx=0x7fab940be420 comp rx=0 tx=0).stop 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 -- 172.21.15.18:0/1521516554 shutdown_connections 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55bf9829c8a0 0x55bf9829ece0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf9828c410 0x55bf98289f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98283830 0x55bf982899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/1521516554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf982894a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fabaa3e4740 1 -- 172.21.15.18:0/1521516554 >> 172.21.15.18:0/1521516554 conn(0x55bf9827d640 msgr2=0x55bf9827f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.410+0000 7fab86000640 0 client.26123 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.411+0000 7fabaa3e4740 1 -- 172.21.15.18:0/1521516554 shutdown_connections 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.411+0000 7fabaa3e4740 1 -- 172.21.15.18:0/1521516554 wait complete. 2024-04-16T17:06:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.411+0000 7fabaa3e4740 20 client.26123 trim_cache size 0 max 16384 2024-04-16T17:06:36.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7fabaa3e4740 1 Processor -- start 2024-04-16T17:06:36.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7fabaa3e4740 1 -- start start 2024-04-16T17:06:36.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 0x55bf98298150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf98289d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf9828c410 0x55bf982a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b8160 con 0x55bf98282f10 2024-04-16T17:06:36.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b5a30 con 0x55bf9828c410 2024-04-16T17:06:36.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b3880 con 0x55bf98283830 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.428+0000 7faba4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 0x55bf98298150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf98289d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7faba5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf9828c410 0x55bf982a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf98289d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53704/0 (socket says 172.21.15.18:53704) 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 -- 172.21.15.18:0/3074707313 learned_addr learned my addr 172.21.15.18:0/3074707313 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 -- 172.21.15.18:0/3074707313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf9828c410 msgr2=0x55bf982a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 --2- 172.21.15.18:0/3074707313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf9828c410 0x55bf982a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 -- 172.21.15.18:0/3074707313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 msgr2=0x55bf98298150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 --2- 172.21.15.18:0/3074707313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 0x55bf98298150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 -- 172.21.15.18:0/3074707313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf97efa3f0 con 0x55bf98283830 2024-04-16T17:06:36.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7faba5600640 1 --2- 172.21.15.18:0/3074707313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf9828c410 0x55bf982a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:36.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.429+0000 7fab9fe00640 1 --2- 172.21.15.18:0/3074707313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf98289d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab8c004c40 tx=0x7fab8c004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f717159dbcbfe943 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.430+0000 7fab91200640 1 -- 172.21.15.18:0/3074707313 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c016040 con 0x55bf98283830 2024-04-16T17:06:36.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.430+0000 7fab91200640 1 -- 172.21.15.18:0/3074707313 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c009d00 con 0x55bf98283830 2024-04-16T17:06:36.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.430+0000 7fab91200640 1 -- 172.21.15.18:0/3074707313 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c005000 con 0x55bf98283830 2024-04-16T17:06:36.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 -- 172.21.15.18:0/3074707313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 msgr2=0x55bf98289d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/3074707313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf98289d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab8c004c40 tx=0x7fab8c004c70 comp rx=0 tx=0).stop 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 -- 172.21.15.18:0/3074707313 shutdown_connections 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/3074707313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf9828c410 0x55bf982a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/3074707313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf98289d40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/3074707313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf98282f10 0x55bf98298150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 -- 172.21.15.18:0/3074707313 >> 172.21.15.18:0/3074707313 conn(0x55bf9827d640 msgr2=0x55bf9827c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 -- 172.21.15.18:0/3074707313 shutdown_connections 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 -- 172.21.15.18:0/3074707313 wait complete. 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.431+0000 7fabaa3e4740 1 -- start start 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fabaa3e4740 1 Processor -- start 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf98294850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf9828fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fabaa3e4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf9828c410 0x55bf98290100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b3ef0 con 0x55bf9828c410 2024-04-16T17:06:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b5360 con 0x55bf98282f10 2024-04-16T17:06:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fabaa3e4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b45c0 con 0x55bf98283830 2024-04-16T17:06:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fabaa3e4740 10 client.0 _ll_register_callbacks cb 0x7fab940ba830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf9828fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf9828fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53712/0 (socket says 172.21.15.18:53712) 2024-04-16T17:06:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7fab9fe00640 1 -- 172.21.15.18:0/295417065 learned_addr learned my addr 172.21.15.18:0/295417065 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7faba4c00640 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf98294850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.432+0000 7faba5600640 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf9828c410 0x55bf98290100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.433+0000 7faba4c00640 1 -- 172.21.15.18:0/295417065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 msgr2=0x55bf9828fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.433+0000 7faba4c00640 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf9828fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.433+0000 7faba4c00640 1 -- 172.21.15.18:0/295417065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf9828c410 msgr2=0x55bf98290100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.433+0000 7faba4c00640 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf9828c410 0x55bf98290100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.433+0000 7faba4c00640 1 -- 172.21.15.18:0/295417065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf97fc47d0 con 0x55bf98282f10 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.433+0000 7faba5600640 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf9828c410 0x55bf98290100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.434+0000 7faba4c00640 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf98294850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fab94005ab0 tx=0x7fab94005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b134c23738bf340 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.434+0000 7fab86000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.434+0000 7fab86000640 1 -- 172.21.15.18:0/295417065 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940be590 con 0x55bf98282f10 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.434+0000 7fab86000640 1 -- 172.21.15.18:0/295417065 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fab940c6d70 con 0x55bf98282f10 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.434+0000 7fabaa3e4740 1 -- 172.21.15.18:0/295417065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8c009d00 con 0x55bf98282f10 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.434+0000 7fab86000640 1 -- 172.21.15.18:0/295417065 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab940c7200 con 0x55bf98282f10 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.434+0000 7fabaa3e4740 10 client.26129 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.434+0000 7fabaa3e4740 1 -- 172.21.15.18:0/295417065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf97fc47d0 con 0x55bf98282f10 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fab86000640 1 -- 172.21.15.18:0/295417065 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c009d00 con 0x55bf98282f10 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fab86000640 1 -- 172.21.15.18:0/295417065 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab940bd9e0 con 0x55bf98282f10 2024-04-16T17:06:36.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fabaa3e4740 10 client.26129 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fabaa3e4740 10 client.26129 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:06:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fabaa3e4740 20 client.26129 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55bf98302e80 0x55bf98305270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fabaa3e4740 4 client.26129 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:06:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fabaa3e4740 1 -- 172.21.15.18:0/295417065 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf981ef8b0 con 0x55bf98302e80 2024-04-16T17:06:36.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.435+0000 7fab9fe00640 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55bf98302e80 0x55bf98305270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fab9fe00640 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55bf98302e80 0x55bf98305270 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c656adffb02d9b6 server_cookie=361732a601a050c4 in_seq=0 out_seq=0 2024-04-16T17:06:36.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fab86000640 10 client.26129 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:06:36.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fab86000640 1 -- 172.21.15.18:0/295417065 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf981ef8b0 con 0x55bf98302e80 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fab86000640 10 client.26129 handle_command_reply: tid=0 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 client.26129 shutdown 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 -- 172.21.15.18:0/295417065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 msgr2=0x55bf98294850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf98294850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fab94005ab0 tx=0x7fab94005e60 comp rx=0 tx=0).stop 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 -- 172.21.15.18:0/295417065 shutdown_connections 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55bf98302e80 0x55bf98305270 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bf9828c410 0x55bf98290100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf98283830 0x55bf9828fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 --2- 172.21.15.18:0/295417065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf98282f10 0x55bf98294850 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fabaa3e4740 1 -- 172.21.15.18:0/295417065 >> 172.21.15.18:0/295417065 conn(0x55bf9827d640 msgr2=0x55bf9827e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.436+0000 7fab86000640 0 client.26129 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.437+0000 7fabaa3e4740 1 -- 172.21.15.18:0/295417065 shutdown_connections 2024-04-16T17:06:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.437+0000 7fabaa3e4740 1 -- 172.21.15.18:0/295417065 wait complete. 2024-04-16T17:06:36.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.437+0000 7fabaa3e4740 20 client.26129 trim_cache size 0 max 16384 2024-04-16T17:06:36.440 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:06:36.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 -- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab880a3d20 msgr2=0x7fab880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab880a3d20 0x7fab880a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faba006b470 tx=0x7fab98014040 comp rx=0 tx=0).stop 2024-04-16T17:06:36.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 -- 172.21.15.18:0/1023816051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faba00743a0 msgr2=0x7faba019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faba00743a0 0x7faba019ed70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab94000c00 tx=0x7fab940043c0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 -- 172.21.15.18:0/1023816051 shutdown_connections 2024-04-16T17:06:36.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab880a3d20 0x7fab880a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faba0112e10 0x7faba01a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faba0074cc0 0x7faba019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 --2- 172.21.15.18:0/1023816051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faba00743a0 0x7faba019ed70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 -- 172.21.15.18:0/1023816051 >> 172.21.15.18:0/1023816051 conn(0x7faba00fd950 msgr2=0x7faba00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 -- 172.21.15.18:0/1023816051 shutdown_connections 2024-04-16T17:06:36.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.438+0000 7faba6000640 1 -- 172.21.15.18:0/1023816051 wait complete. 2024-04-16T17:06:36.455 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:06:36.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 39170 4.69 MB/sec warmup 49 sec latency 10.410 ms 2024-04-16T17:06:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 -- 172.21.15.18:0/2677095784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc24103900 msgr2=0x7fcc2410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2677095784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc24103900 0x7fcc2410bde0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009900 tx=0x7fcc0c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 -- 172.21.15.18:0/2677095784 shutdown_connections 2024-04-16T17:06:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2677095784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc2410c400 0x7fcc2410e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2677095784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc24103900 0x7fcc2410bde0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2677095784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc24102fe0 0x7fcc241033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 -- 172.21.15.18:0/2677095784 >> 172.21.15.18:0/2677095784 conn(0x7fcc24077ab0 msgr2=0x7fcc240fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 -- 172.21.15.18:0/2677095784 shutdown_connections 2024-04-16T17:06:36.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.668+0000 7fcc28c00640 1 -- 172.21.15.18:0/2677095784 wait complete. 2024-04-16T17:06:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.669+0000 7fcc28c00640 1 Processor -- start 2024-04-16T17:06:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.669+0000 7fcc28c00640 1 -- start start 2024-04-16T17:06:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.669+0000 7fcc28c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc24102fe0 0x7fcc24100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.669+0000 7fcc28c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc24103900 0x7fcc24101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.669+0000 7fcc28c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc2410c400 0x7fcc241016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.669+0000 7fcc28c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241121e0 con 0x7fcc24102fe0 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.669+0000 7fcc28c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24112060 con 0x7fcc2410c400 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.669+0000 7fcc28c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24112360 con 0x7fcc24103900 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc24102fe0 0x7fcc24100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc2410c400 0x7fcc241016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc24102fe0 0x7fcc24100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49076/0 (socket says 172.21.15.18:49076) 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23400640 1 -- 172.21.15.18:0/2324018077 learned_addr learned my addr 172.21.15.18:0/2324018077 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc2410c400 0x7fcc241016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46674/0 (socket says 172.21.15.18:46674) 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc1be00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc24103900 0x7fcc24101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23400640 1 -- 172.21.15.18:0/2324018077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc24103900 msgr2=0x7fcc24101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23400640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc24103900 0x7fcc24101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23400640 1 -- 172.21.15.18:0/2324018077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc2410c400 msgr2=0x7fcc241016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23400640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc2410c400 0x7fcc241016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23400640 1 -- 172.21.15.18:0/2324018077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc241a6ca0 con 0x7fcc24102fe0 2024-04-16T17:06:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.670+0000 7fcc23e00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc2410c400 0x7fcc241016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:36.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.671+0000 7fcc23400640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc24102fe0 0x7fcc24100c30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400ea60 tx=0x7fcc1400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.671+0000 7fcc20c00640 1 -- 172.21.15.18:0/2324018077 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400cde0 con 0x7fcc24102fe0 2024-04-16T17:06:36.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.671+0000 7fcc28c00640 1 -- 172.21.15.18:0/2324018077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc241a6f90 con 0x7fcc24102fe0 2024-04-16T17:06:36.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.671+0000 7fcc20c00640 1 -- 172.21.15.18:0/2324018077 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc140035f0 con 0x7fcc24102fe0 2024-04-16T17:06:36.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.671+0000 7fcc20c00640 1 -- 172.21.15.18:0/2324018077 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14010670 con 0x7fcc24102fe0 2024-04-16T17:06:36.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.671+0000 7fcc28c00640 1 -- 172.21.15.18:0/2324018077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc241a7450 con 0x7fcc24102fe0 2024-04-16T17:06:36.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:36 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/963207620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.676+0000 7fcc20c00640 1 -- 172.21.15.18:0/2324018077 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc140037a0 con 0x7fcc24102fe0 2024-04-16T17:06:36.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.676+0000 7fcc2ce41740 1 Processor -- start 2024-04-16T17:06:36.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.676+0000 7fcc2ce41740 1 -- start start 2024-04-16T17:06:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 0x5581c2f2a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f2a830 0x5581c2f32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f35800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5581c2e8f020 con 0x5581c2f29f10 2024-04-16T17:06:36.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581c2e35460 con 0x5581c2f2a830 2024-04-16T17:06:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5581c2e8afe0 con 0x5581c2f33410 2024-04-16T17:06:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f35800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f35800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53726/0 (socket says 172.21.15.18:53726) 2024-04-16T17:06:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc1be00640 1 -- 172.21.15.18:0/3553722542 learned_addr learned my addr 172.21.15.18:0/3553722542 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc23400640 1 --2- 172.21.15.18:0/3553722542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f2a830 0x5581c2f32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc20c00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcc080a3da0 0x7fcc080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc1be00640 1 -- 172.21.15.18:0/3553722542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f2a830 msgr2=0x5581c2f32ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc1be00640 1 --2- 172.21.15.18:0/3553722542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f2a830 0x5581c2f32ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc1be00640 1 -- 172.21.15.18:0/3553722542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 msgr2=0x5581c2f2a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.677+0000 7fcc1be00640 1 --2- 172.21.15.18:0/3553722542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 0x5581c2f2a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.678+0000 7fcc1be00640 1 -- 172.21.15.18:0/3553722542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c2e3d040 con 0x5581c2f33410 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.678+0000 7fcc20c00640 1 -- 172.21.15.18:0/2324018077 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fcc140d0e70 con 0x7fcc24102fe0 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.678+0000 7fcc23e00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcc080a3da0 0x7fcc080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.678+0000 7fcc1be00640 1 --2- 172.21.15.18:0/3553722542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f35800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c000c00 tx=0x7fcc0c0095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b864e8baf06c38b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.679+0000 7fcc13e00640 1 -- 172.21.15.18:0/3553722542 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c043070 con 0x5581c2f33410 2024-04-16T17:06:36.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.679+0000 7fcc13e00640 1 -- 172.21.15.18:0/3553722542 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0c02c920 con 0x5581c2f33410 2024-04-16T17:06:36.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.679+0000 7fcc13e00640 1 -- 172.21.15.18:0/3553722542 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c004b60 con 0x5581c2f33410 2024-04-16T17:06:36.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.680+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3553722542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 msgr2=0x5581c2f35800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.680+0000 7fcc23e00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcc080a3da0 0x7fcc080a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fd0 tx=0x7fcc1c009920 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.680+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/3553722542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f35800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c000c00 tx=0x7fcc0c0095e0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3553722542 shutdown_connections 2024-04-16T17:06:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/3553722542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f35800 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/3553722542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f2a830 0x5581c2f32ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/3553722542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 0x5581c2f2a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3553722542 >> 172.21.15.18:0/3553722542 conn(0x5581c2f24640 msgr2=0x5581c2f24a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3553722542 shutdown_connections 2024-04-16T17:06:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3553722542 wait complete. 2024-04-16T17:06:36.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 -- start start 2024-04-16T17:06:36.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.681+0000 7fcc2ce41740 1 Processor -- start 2024-04-16T17:06:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 0x5581c2f30430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f2a830 0x5581c2f30970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f30eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5581c2f361a0 con 0x5581c2f29f10 2024-04-16T17:06:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581c2e48830 con 0x5581c2f33410 2024-04-16T17:06:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5581c2f36320 con 0x5581c2f2a830 2024-04-16T17:06:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc2ce41740 10 client.0 _ll_register_callbacks cb 0x5581c2e50150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:36.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f30eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f30eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46698/0 (socket says 172.21.15.18:46698) 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc23e00640 1 -- 172.21.15.18:0/2684305918 learned_addr learned my addr 172.21.15.18:0/2684305918 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.682+0000 7fcc1be00640 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f2a830 0x5581c2f30970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.683+0000 7fcc23400640 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 0x5581c2f30430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.683+0000 7fcc23e00640 1 -- 172.21.15.18:0/2684305918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f2a830 msgr2=0x5581c2f30970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.683+0000 7fcc23e00640 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f2a830 0x5581c2f30970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.683+0000 7fcc23e00640 1 -- 172.21.15.18:0/2684305918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 msgr2=0x5581c2f30430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.683+0000 7fcc23e00640 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 0x5581c2f30430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.683+0000 7fcc23e00640 1 -- 172.21.15.18:0/2684305918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c2e4c920 con 0x5581c2f33410 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.683+0000 7fcc23e00640 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f30eb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c004a80 tx=0x7fcc1c0096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78e5b99480984e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.683+0000 7fcc10c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.684+0000 7fcc10c00640 1 -- 172.21.15.18:0/2684305918 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c01e420 con 0x5581c2f33410 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.684+0000 7fcc2ce41740 1 -- 172.21.15.18:0/2684305918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0c02c920 con 0x5581c2f33410 2024-04-16T17:06:36.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.684+0000 7fcc10c00640 1 -- 172.21.15.18:0/2684305918 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fcc1c01e5c0 con 0x5581c2f33410 2024-04-16T17:06:36.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.684+0000 7fcc10c00640 1 -- 172.21.15.18:0/2684305918 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc1c01e7c0 con 0x5581c2f33410 2024-04-16T17:06:36.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.684+0000 7fcc2ce41740 10 client.26141 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:36.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.684+0000 7fcc2ce41740 1 -- 172.21.15.18:0/2684305918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5581c2e1a7f0 con 0x5581c2f33410 2024-04-16T17:06:36.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.685+0000 7fcc10c00640 1 -- 172.21.15.18:0/2684305918 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0c02c920 con 0x5581c2f33410 2024-04-16T17:06:36.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.685+0000 7fcc10c00640 1 -- 172.21.15.18:0/2684305918 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcc1c030980 con 0x5581c2f33410 2024-04-16T17:06:36.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.685+0000 7fcc2ce41740 10 client.26141 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:36.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.685+0000 7fcc2ce41740 10 client.26141 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:06:36.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.685+0000 7fcc2ce41740 20 client.26141 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:36.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.685+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5581c2f43740 0x5581c2f45b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.685+0000 7fcc2ce41740 4 client.26141 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.685+0000 7fcc2ce41740 1 -- 172.21.15.18:0/2684305918 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581c2e9ca40 con 0x5581c2f43740 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.686+0000 7fcc23400640 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5581c2f43740 0x5581c2f45b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.686+0000 7fcc23400640 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5581c2f43740 0x5581c2f45b30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1b5ab23d51aae7a server_cookie=3556b514c9bad115 in_seq=0 out_seq=0 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.686+0000 7fcc10c00640 10 client.26141 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.687+0000 7fcc10c00640 1 -- 172.21.15.18:0/2684305918 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581c2e9ca40 con 0x5581c2f43740 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.687+0000 7fcc10c00640 10 client.26141 handle_command_reply: tid=0 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.687+0000 7fcc2ce41740 1 client.26141 shutdown 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.687+0000 7fcc2ce41740 1 -- 172.21.15.18:0/2684305918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 msgr2=0x5581c2f30eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.687+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f30eb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c004a80 tx=0x7fcc1c0096a0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 1 -- 172.21.15.18:0/2684305918 shutdown_connections 2024-04-16T17:06:36.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5581c2f43740 0x5581c2f45b30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc10c00640 0 client.26141 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:06:36.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f30eb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f2a830 0x5581c2f30970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/2684305918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f29f10 0x5581c2f30430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 1 -- 172.21.15.18:0/2684305918 >> 172.21.15.18:0/2684305918 conn(0x5581c2f24640 msgr2=0x5581c2f26390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 1 -- 172.21.15.18:0/2684305918 shutdown_connections 2024-04-16T17:06:36.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 1 -- 172.21.15.18:0/2684305918 wait complete. 2024-04-16T17:06:36.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.688+0000 7fcc2ce41740 20 client.26141 trim_cache size 0 max 16384 2024-04-16T17:06:36.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.709+0000 7fcc2ce41740 1 Processor -- start 2024-04-16T17:06:36.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.709+0000 7fcc2ce41740 1 -- start start 2024-04-16T17:06:36.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.709+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f29f10 0x5581c2f32b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.709+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.709+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f23580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.709+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241ab6c0 con 0x5581c2f2a830 2024-04-16T17:06:36.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.709+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241a8f90 con 0x5581c2f29f10 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.709+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc240738d0 con 0x5581c2f33410 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f29f10 0x5581c2f32b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f29f10 0x5581c2f32b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46712/0 (socket says 172.21.15.18:46712) 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 -- 172.21.15.18:0/3712496502 learned_addr learned my addr 172.21.15.18:0/3712496502 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23e00640 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f23580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 -- 172.21.15.18:0/3712496502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 msgr2=0x5581c2f23580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc1be00640 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f23580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 -- 172.21.15.18:0/3712496502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 msgr2=0x5581c2f26300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f26300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 -- 172.21.15.18:0/3712496502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c2ba13f0 con 0x5581c2f29f10 2024-04-16T17:06:36.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc1be00640 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:36.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.710+0000 7fcc23400640 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f29f10 0x5581c2f32b60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc1409c700 tx=0x7fcc140d3330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b94be7ff67cebb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc13400640 1 -- 172.21.15.18:0/3712496502 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc140b5070 con 0x5581c2f29f10 2024-04-16T17:06:36.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc13400640 1 -- 172.21.15.18:0/3712496502 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc140b7070 con 0x5581c2f29f10 2024-04-16T17:06:36.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc13400640 1 -- 172.21.15.18:0/3712496502 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14099070 con 0x5581c2f29f10 2024-04-16T17:06:36.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3712496502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f29f10 msgr2=0x5581c2f32b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f29f10 0x5581c2f32b60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc1409c700 tx=0x7fcc140d3330 comp rx=0 tx=0).stop 2024-04-16T17:06:36.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3712496502 shutdown_connections 2024-04-16T17:06:36.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f33410 0x5581c2f23580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/3712496502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f29f10 0x5581c2f32b60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.711+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3712496502 >> 172.21.15.18:0/3712496502 conn(0x5581c2f24640 msgr2=0x5581c2f22e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.712+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3712496502 shutdown_connections 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.712+0000 7fcc2ce41740 1 -- 172.21.15.18:0/3712496502 wait complete. 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.713+0000 7fcc2ce41740 1 -- start start 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.713+0000 7fcc2ce41740 1 Processor -- start 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.713+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f29f10 0x5581c2f3b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.713+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f36cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.713+0000 7fcc2ce41740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.713+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24073fa0 con 0x5581c2f2a830 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.713+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241a88c0 con 0x5581c2f33410 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.713+0000 7fcc2ce41740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241a7b20 con 0x5581c2f29f10 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc2ce41740 10 client.0 _ll_register_callbacks cb 0x7fcc1c0103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f29f10 0x5581c2f3b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc1be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f36cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f29f10 0x5581c2f3b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53746/0 (socket says 172.21.15.18:53746) 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23400640 1 -- 172.21.15.18:0/542038974 learned_addr learned my addr 172.21.15.18:0/542038974 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23e00640 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23400640 1 -- 172.21.15.18:0/542038974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 msgr2=0x5581c2f37230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23400640 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f37230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23400640 1 -- 172.21.15.18:0/542038974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 msgr2=0x5581c2f36cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23400640 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f36cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.714+0000 7fcc23400640 1 -- 172.21.15.18:0/542038974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c2c6b7d0 con 0x5581c2f29f10 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.715+0000 7fcc23400640 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f29f10 0x5581c2f3b930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc140bd0b0 tx=0x7fcc1409ada0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae15d4effe491d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.715+0000 7fcc10c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.715+0000 7fcc10c00640 1 -- 172.21.15.18:0/542038974 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc140b5070 con 0x5581c2f29f10 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.715+0000 7fcc10c00640 1 -- 172.21.15.18:0/542038974 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fcc140ce430 con 0x5581c2f29f10 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.715+0000 7fcc2ce41740 1 -- 172.21.15.18:0/542038974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc140b7070 con 0x5581c2f29f10 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.715+0000 7fcc10c00640 1 -- 172.21.15.18:0/542038974 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc140ce630 con 0x5581c2f29f10 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.715+0000 7fcc2ce41740 10 client.26149 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.715+0000 7fcc2ce41740 1 -- 172.21.15.18:0/542038974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5581c2fa99b0 con 0x5581c2f29f10 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.716+0000 7fcc10c00640 1 -- 172.21.15.18:0/542038974 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc140b7070 con 0x5581c2f29f10 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.716+0000 7fcc10c00640 1 -- 172.21.15.18:0/542038974 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcc1409be50 con 0x5581c2f29f10 2024-04-16T17:06:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.716+0000 7fcc2ce41740 10 client.26149 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.716+0000 7fcc2ce41740 10 client.26149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:06:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.716+0000 7fcc2ce41740 20 client.26149 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.716+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5581c2faa080 0x5581c2fac470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.716+0000 7fcc2ce41740 4 client.26149 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:06:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.716+0000 7fcc2ce41740 1 -- 172.21.15.18:0/542038974 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581c2e968b0 con 0x5581c2faa080 2024-04-16T17:06:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.717+0000 7fcc1be00640 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5581c2faa080 0x5581c2fac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.717+0000 7fcc1be00640 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5581c2faa080 0x5581c2fac470 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97ccce082ea44f36 server_cookie=9331d9700eaf8ce7 in_seq=0 out_seq=0 2024-04-16T17:06:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.717+0000 7fcc10c00640 10 client.26149 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.718+0000 7fcc10c00640 1 -- 172.21.15.18:0/542038974 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581c2e968b0 con 0x5581c2faa080 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.718+0000 7fcc10c00640 10 client.26149 handle_command_reply: tid=0 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.718+0000 7fcc2ce41740 1 client.26149 shutdown 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.718+0000 7fcc2ce41740 1 -- 172.21.15.18:0/542038974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f29f10 msgr2=0x5581c2f3b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.718+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f29f10 0x5581c2f3b930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc140bd0b0 tx=0x7fcc1409ada0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc2ce41740 1 -- 172.21.15.18:0/542038974 shutdown_connections 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5581c2faa080 0x5581c2fac470 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c2f33410 0x5581c2f37230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5581c2f2a830 0x5581c2f36cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc10c00640 0 client.26149 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc2ce41740 1 --2- 172.21.15.18:0/542038974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581c2f29f10 0x5581c2f3b930 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc2ce41740 1 -- 172.21.15.18:0/542038974 >> 172.21.15.18:0/542038974 conn(0x5581c2f24640 msgr2=0x5581c2f23700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc2ce41740 1 -- 172.21.15.18:0/542038974 shutdown_connections 2024-04-16T17:06:36.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.719+0000 7fcc2ce41740 1 -- 172.21.15.18:0/542038974 wait complete. 2024-04-16T17:06:36.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.720+0000 7fcc2ce41740 20 client.26149 trim_cache size 0 max 16384 2024-04-16T17:06:36.725 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:06:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.720+0000 7fcc28c00640 1 -- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcc080a3da0 msgr2=0x7fcc080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.720+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcc080a3da0 0x7fcc080a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fd0 tx=0x7fcc1c009920 comp rx=0 tx=0).stop 2024-04-16T17:06:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.720+0000 7fcc28c00640 1 -- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc24102fe0 msgr2=0x7fcc24100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.720+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc24102fe0 0x7fcc24100c30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400ea60 tx=0x7fcc1400ef20 comp rx=0 tx=0).stop 2024-04-16T17:06:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.721+0000 7fcc28c00640 1 -- 172.21.15.18:0/2324018077 shutdown_connections 2024-04-16T17:06:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.721+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcc080a3da0 0x7fcc080a6260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.721+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc2410c400 0x7fcc241016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.721+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc24103900 0x7fcc24101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.721+0000 7fcc28c00640 1 --2- 172.21.15.18:0/2324018077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc24102fe0 0x7fcc24100c30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.721+0000 7fcc28c00640 1 -- 172.21.15.18:0/2324018077 >> 172.21.15.18:0/2324018077 conn(0x7fcc24077ab0 msgr2=0x7fcc240fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.721+0000 7fcc28c00640 1 -- 172.21.15.18:0/2324018077 shutdown_connections 2024-04-16T17:06:36.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.721+0000 7fcc28c00640 1 -- 172.21.15.18:0/2324018077 wait complete. 2024-04-16T17:06:36.740 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:06:36.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:36 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/963207620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:36.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 -- 172.21.15.18:0/3683867723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 msgr2=0x7f413c1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 --2- 172.21.15.18:0/3683867723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 0x7f413c1033c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f412c009900 tx=0x7f412c02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 -- 172.21.15.18:0/3683867723 shutdown_connections 2024-04-16T17:06:36.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 --2- 172.21.15.18:0/3683867723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413c10c400 0x7f413c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 --2- 172.21.15.18:0/3683867723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f413c103900 0x7f413c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 --2- 172.21.15.18:0/3683867723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 0x7f413c1033c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 -- 172.21.15.18:0/3683867723 >> 172.21.15.18:0/3683867723 conn(0x7f413c077ab0 msgr2=0x7f413c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 -- 172.21.15.18:0/3683867723 shutdown_connections 2024-04-16T17:06:36.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.948+0000 7f4140c00640 1 -- 172.21.15.18:0/3683867723 wait complete. 2024-04-16T17:06:36.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.949+0000 7f4140c00640 1 Processor -- start 2024-04-16T17:06:36.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.949+0000 7f4140c00640 1 -- start start 2024-04-16T17:06:36.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.949+0000 7f4140c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 0x7f413c100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.949+0000 7f4140c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f413c103900 0x7f413c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.949+0000 7f4140c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413c10c400 0x7f413c1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.949+0000 7f4140c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f413c112210 con 0x7f413c103900 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.949+0000 7f4140c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f413c112090 con 0x7f413c10c400 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.949+0000 7f4140c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f413c112390 con 0x7f413c102fe0 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 0x7f413c100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f413c103900 0x7f413c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413c10c400 0x7f413c1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 0x7f413c100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49246/0 (socket says 172.21.15.18:49246) 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 -- 172.21.15.18:0/1586726564 learned_addr learned my addr 172.21.15.18:0/1586726564 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 -- 172.21.15.18:0/1586726564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413c10c400 msgr2=0x7f413c1016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413c10c400 0x7f413c1016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 -- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f413c103900 msgr2=0x7f413c101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f413c103900 0x7f413c101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 -- 172.21.15.18:0/1586726564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c1a6c90 con 0x7f413c102fe0 2024-04-16T17:06:36.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413aa00640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f413c103900 0x7f413c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:36.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.950+0000 7f413b400640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 0x7f413c100c30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f412c036040 tx=0x7f412c02cd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.951+0000 7f4133e00640 1 -- 172.21.15.18:0/1586726564 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c043020 con 0x7f413c102fe0 2024-04-16T17:06:36.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.951+0000 7f4133e00640 1 -- 172.21.15.18:0/1586726564 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c0054e0 con 0x7f413c102fe0 2024-04-16T17:06:36.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.951+0000 7f4140c00640 1 -- 172.21.15.18:0/1586726564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c1984e0 con 0x7f413c102fe0 2024-04-16T17:06:36.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.951+0000 7f4133e00640 1 -- 172.21.15.18:0/1586726564 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c003da0 con 0x7f413c102fe0 2024-04-16T17:06:36.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.951+0000 7f4140c00640 1 -- 172.21.15.18:0/1586726564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c198a80 con 0x7f413c102fe0 2024-04-16T17:06:36.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.955+0000 7f4144f8f740 1 Processor -- start 2024-04-16T17:06:36.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f4144f8f740 1 -- start start 2024-04-16T17:06:36.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f4133e00640 1 -- 172.21.15.18:0/1586726564 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f412c046020 con 0x7f413c102fe0 2024-04-16T17:06:36.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f4144f8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dfff43b020 con 0x55dfff4df410 2024-04-16T17:06:36.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f4144f8f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dfff3e1460 con 0x55dfff4d6830 2024-04-16T17:06:36.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f4144f8f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfff436fe0 con 0x55dfff4d5f10 2024-04-16T17:06:36.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f413be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f413b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f413be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4d62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49252/0 (socket says 172.21.15.18:49252) 2024-04-16T17:06:36.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.956+0000 7f413be00640 1 -- 172.21.15.18:0/4010093586 learned_addr learned my addr 172.21.15.18:0/4010093586 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f413aa00640 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f413be00640 1 -- 172.21.15.18:0/4010093586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 msgr2=0x55dfff4deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f413be00640 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4deed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f413be00640 1 -- 172.21.15.18:0/4010093586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 msgr2=0x55dfff4e1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f413be00640 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f413be00640 1 -- 172.21.15.18:0/4010093586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfff3e9040 con 0x55dfff4d5f10 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f413b400640 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f413be00640 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4d62f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4134010370 tx=0x7f4134010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3db98731c80540b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f4133e00640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41200a3da0 0x7f41200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f4127e00640 1 -- 172.21.15.18:0/4010093586 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134016070 con 0x55dfff4d5f10 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f4127e00640 1 -- 172.21.15.18:0/4010093586 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4134004d20 con 0x55dfff4d5f10 2024-04-16T17:06:36.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.957+0000 7f4127e00640 1 -- 172.21.15.18:0/4010093586 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134005480 con 0x55dfff4d5f10 2024-04-16T17:06:36.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.958+0000 7f4133e00640 1 -- 172.21.15.18:0/1586726564 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f412c0f2c60 con 0x7f413c102fe0 2024-04-16T17:06:36.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.958+0000 7f4144f8f740 1 -- 172.21.15.18:0/4010093586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 msgr2=0x55dfff4d62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.958+0000 7f4144f8f740 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4d62f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4134010370 tx=0x7f4134010830 comp rx=0 tx=0).stop 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f4144f8f740 1 -- 172.21.15.18:0/4010093586 shutdown_connections 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f4144f8f740 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f4144f8f740 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f4144f8f740 1 --2- 172.21.15.18:0/4010093586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4d62f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f413b400640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41200a3da0 0x7f41200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f4144f8f740 1 -- 172.21.15.18:0/4010093586 >> 172.21.15.18:0/4010093586 conn(0x55dfff4d0640 msgr2=0x55dfff4d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f4144f8f740 1 -- 172.21.15.18:0/4010093586 shutdown_connections 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f4144f8f740 1 -- 172.21.15.18:0/4010093586 wait complete. 2024-04-16T17:06:36.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f4144f8f740 1 -- start start 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.959+0000 7f413b400640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41200a3da0 0x7f41200a6260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55dfff4d59e0 tx=0x7f412c0f6710 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f4144f8f740 1 Processor -- start 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4d5f10 0x55dfff4dc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4df410 0x55dfff4dcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f4144f8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dfff4e2200 con 0x55dfff4d5f10 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f4144f8f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dfff3f4830 con 0x55dfff4d6830 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f4144f8f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfff4e2380 con 0x55dfff4df410 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f4144f8f740 10 client.0 _ll_register_callbacks cb 0x55dfff3fc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.960+0000 7f413be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4d5f10 0x55dfff4dc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4dcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59950/0 (socket says 172.21.15.18:59950) 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413aa00640 1 -- 172.21.15.18:0/3405845662 learned_addr learned my addr 172.21.15.18:0/3405845662 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413be00640 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4df410 0x55dfff4dcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413aa00640 1 -- 172.21.15.18:0/3405845662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4df410 msgr2=0x55dfff4dcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413aa00640 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4df410 0x55dfff4dcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413aa00640 1 -- 172.21.15.18:0/3405845662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4d5f10 msgr2=0x55dfff4dc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413aa00640 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4d5f10 0x55dfff4dc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413aa00640 1 -- 172.21.15.18:0/3405845662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfff3f8920 con 0x55dfff4d6830 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.961+0000 7f413be00640 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4d5f10 0x55dfff4dc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.962+0000 7f413aa00640 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4dcab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4128017cb0 tx=0x7f41280154a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e21136762826fe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.962+0000 7f4124c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.962+0000 7f4124c00640 1 -- 172.21.15.18:0/3405845662 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128015db0 con 0x55dfff4d6830 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.962+0000 7f4144f8f740 1 -- 172.21.15.18:0/3405845662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4134004d20 con 0x55dfff4d6830 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.962+0000 7f4124c00640 1 -- 172.21.15.18:0/3405845662 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f412801e420 con 0x55dfff4d6830 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.962+0000 7f4124c00640 1 -- 172.21.15.18:0/3405845662 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f412802c050 con 0x55dfff4d6830 2024-04-16T17:06:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.962+0000 7f4144f8f740 10 client.26159 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.962+0000 7f4144f8f740 1 -- 172.21.15.18:0/3405845662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfff3c67f0 con 0x55dfff4d6830 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.963+0000 7f4124c00640 1 -- 172.21.15.18:0/3405845662 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4134004d20 con 0x55dfff4d6830 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.963+0000 7f4124c00640 1 -- 172.21.15.18:0/3405845662 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f412801c570 con 0x55dfff4d6830 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.963+0000 7f4144f8f740 10 client.26159 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.963+0000 7f4144f8f740 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.963+0000 7f4144f8f740 20 client.26159 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.963+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfff4ef8c0 0x55dfff4f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.963+0000 7f4144f8f740 4 client.26159 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.963+0000 7f4144f8f740 1 -- 172.21.15.18:0/3405845662 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfff448a40 con 0x55dfff4ef8c0 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.964+0000 7f413be00640 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfff4ef8c0 0x55dfff4f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.964+0000 7f4124c00640 10 client.26159 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.964+0000 7f413be00640 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfff4ef8c0 0x55dfff4f1cd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d71921be099526b server_cookie=eeb131ffabd1f954 in_seq=0 out_seq=0 2024-04-16T17:06:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4124c00640 1 -- 172.21.15.18:0/3405845662 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfff448a40 con 0x55dfff4ef8c0 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4124c00640 10 client.26159 handle_command_reply: tid=0 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 client.26159 shutdown 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 -- 172.21.15.18:0/3405845662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 msgr2=0x55dfff4dcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4dcab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4128017cb0 tx=0x7f41280154a0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 -- 172.21.15.18:0/3405845662 shutdown_connections 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfff4ef8c0 0x55dfff4f1cd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4df410 0x55dfff4dcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4dcab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3405845662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4d5f10 0x55dfff4dc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.965+0000 7f4144f8f740 1 -- 172.21.15.18:0/3405845662 >> 172.21.15.18:0/3405845662 conn(0x55dfff4d0640 msgr2=0x55dfff4d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.966+0000 7f4124c00640 0 client.26159 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.966+0000 7f4144f8f740 1 -- 172.21.15.18:0/3405845662 shutdown_connections 2024-04-16T17:06:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.966+0000 7f4144f8f740 1 -- 172.21.15.18:0/3405845662 wait complete. 2024-04-16T17:06:36.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.966+0000 7f4144f8f740 20 client.26159 trim_cache size 0 max 16384 2024-04-16T17:06:36.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.982+0000 7f4144f8f740 1 Processor -- start 2024-04-16T17:06:36.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.982+0000 7f4144f8f740 1 -- start start 2024-04-16T17:06:36.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.982+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d5f10 0x55dfff4eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.982+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d6830 0x55dfff4f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.982+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.982+0000 7f4144f8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1afac0 con 0x55dfff4df410 2024-04-16T17:06:36.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.982+0000 7f4144f8f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1ad390 con 0x55dfff4d5f10 2024-04-16T17:06:36.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.982+0000 7f4144f8f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1ab1e0 con 0x55dfff4d6830 2024-04-16T17:06:36.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d6830 0x55dfff4f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d6830 0x55dfff4f75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49276/0 (socket says 172.21.15.18:49276) 2024-04-16T17:06:36.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413aa00640 1 -- 172.21.15.18:0/3634360700 learned_addr learned my addr 172.21.15.18:0/3634360700 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413be00640 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d5f10 0x55dfff4eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413be00640 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413aa00640 1 -- 172.21.15.18:0/3634360700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d5f10 msgr2=0x55dfff4eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413aa00640 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d5f10 0x55dfff4eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413aa00640 1 -- 172.21.15.18:0/3634360700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 msgr2=0x55dfff4f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413aa00640 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.983+0000 7f413aa00640 1 -- 172.21.15.18:0/3634360700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfff14d3f0 con 0x55dfff4d6830 2024-04-16T17:06:36.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.984+0000 7f413aa00640 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d6830 0x55dfff4f75a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4128001eb0 tx=0x7f412800f9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea75022c25f2e759 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.984+0000 7f4127400640 1 -- 172.21.15.18:0/3634360700 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128034070 con 0x55dfff4d6830 2024-04-16T17:06:36.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.984+0000 7f4127400640 1 -- 172.21.15.18:0/3634360700 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4128023070 con 0x55dfff4d6830 2024-04-16T17:06:36.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.984+0000 7f4127400640 1 -- 172.21.15.18:0/3634360700 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128028680 con 0x55dfff4d6830 2024-04-16T17:06:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 -- 172.21.15.18:0/3634360700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d6830 msgr2=0x55dfff4f75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d6830 0x55dfff4f75a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4128001eb0 tx=0x7f412800f9b0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 -- 172.21.15.18:0/3634360700 shutdown_connections 2024-04-16T17:06:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d6830 0x55dfff4f75a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3634360700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d5f10 0x55dfff4eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 -- 172.21.15.18:0/3634360700 >> 172.21.15.18:0/3634360700 conn(0x55dfff4d0640 msgr2=0x55dfff4d7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 -- 172.21.15.18:0/3634360700 shutdown_connections 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 -- 172.21.15.18:0/3634360700 wait complete. 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.985+0000 7f4144f8f740 1 -- start start 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.986+0000 7f4144f8f740 1 Processor -- start 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.986+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4e7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.986+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.986+0000 7f4144f8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.986+0000 7f4144f8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1ab850 con 0x55dfff4df410 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.986+0000 7f4144f8f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1accc0 con 0x55dfff4d6830 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.986+0000 7f4144f8f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1abf20 con 0x55dfff4d5f10 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.986+0000 7f4144f8f740 10 client.0 _ll_register_callbacks cb 0x7f412800fa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4e7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59964/0 (socket says 172.21.15.18:59964) 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 -- 172.21.15.18:0/3022001784 learned_addr learned my addr 172.21.15.18:0/3022001784 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413be00640 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 -- 172.21.15.18:0/3022001784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 msgr2=0x55dfff4e7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4e7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 -- 172.21.15.18:0/3022001784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 msgr2=0x55dfff4e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 -- 172.21.15.18:0/3022001784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfff2177d0 con 0x55dfff4d6830 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413be00640 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f413aa00640 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4e7a90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f412800fa40 tx=0x7f4128015de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c7045f8ac4e0ab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.987+0000 7f4124c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.988+0000 7f4124c00640 1 -- 172.21.15.18:0/3022001784 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128040070 con 0x55dfff4d6830 2024-04-16T17:06:36.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.988+0000 7f4124c00640 1 -- 172.21.15.18:0/3022001784 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f412801c5d0 con 0x55dfff4d6830 2024-04-16T17:06:36.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.988+0000 7f4144f8f740 1 -- 172.21.15.18:0/3022001784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4128023070 con 0x55dfff4d6830 2024-04-16T17:06:36.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.988+0000 7f4124c00640 1 -- 172.21.15.18:0/3022001784 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4128049050 con 0x55dfff4d6830 2024-04-16T17:06:36.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.988+0000 7f4144f8f740 10 client.26165 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:36.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.988+0000 7f4144f8f740 1 -- 172.21.15.18:0/3022001784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfff555810 con 0x55dfff4d6830 2024-04-16T17:06:36.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.988+0000 7f4124c00640 1 -- 172.21.15.18:0/3022001784 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4128023070 con 0x55dfff4d6830 2024-04-16T17:06:36.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.988+0000 7f4124c00640 1 -- 172.21.15.18:0/3022001784 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4128043020 con 0x55dfff4d6830 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.989+0000 7f4144f8f740 10 client.26165 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.989+0000 7f4144f8f740 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.989+0000 7f4144f8f740 20 client.26165 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.989+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfff555e90 0x55dfff5582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.989+0000 7f4144f8f740 4 client.26165 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.989+0000 7f4144f8f740 1 -- 172.21.15.18:0/3022001784 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfff4428b0 con 0x55dfff555e90 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.989+0000 7f413be00640 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfff555e90 0x55dfff5582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f413be00640 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfff555e90 0x55dfff5582b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd48faefcf84ccd9 server_cookie=b7bd2b64e9e603db in_seq=0 out_seq=0 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4124c00640 10 client.26165 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4124c00640 1 -- 172.21.15.18:0/3022001784 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfff4428b0 con 0x55dfff555e90 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4124c00640 10 client.26165 handle_command_reply: tid=0 2024-04-16T17:06:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 client.26165 shutdown 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 -- 172.21.15.18:0/3022001784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 msgr2=0x55dfff4e7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4e7a90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f412800fa40 tx=0x7f4128015de0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 -- 172.21.15.18:0/3022001784 shutdown_connections 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfff555e90 0x55dfff5582b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfff4df410 0x55dfff4e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfff4d6830 0x55dfff4e7a90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 --2- 172.21.15.18:0/3022001784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfff4d5f10 0x55dfff4e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 -- 172.21.15.18:0/3022001784 >> 172.21.15.18:0/3022001784 conn(0x55dfff4d0640 msgr2=0x55dfff4cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4124c00640 0 client.26165 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.990+0000 7f4144f8f740 1 -- 172.21.15.18:0/3022001784 shutdown_connections 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.991+0000 7f4144f8f740 1 -- 172.21.15.18:0/3022001784 wait complete. 2024-04-16T17:06:36.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.991+0000 7f4144f8f740 20 client.26165 trim_cache size 0 max 16384 2024-04-16T17:06:36.995 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:06:36.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.991+0000 7f4140c00640 1 -- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41200a3da0 msgr2=0x7f41200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.991+0000 7f4140c00640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41200a3da0 0x7f41200a6260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55dfff4d59e0 tx=0x7f412c0f6710 comp rx=0 tx=0).stop 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.991+0000 7f4140c00640 1 -- 172.21.15.18:0/1586726564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 msgr2=0x7f413c100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.991+0000 7f4140c00640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 0x7f413c100c30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f412c036040 tx=0x7f412c02cd00 comp rx=0 tx=0).stop 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.992+0000 7f4140c00640 1 -- 172.21.15.18:0/1586726564 shutdown_connections 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.992+0000 7f4140c00640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41200a3da0 0x7f41200a6260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.992+0000 7f4140c00640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f413c10c400 0x7f413c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.992+0000 7f4140c00640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f413c103900 0x7f413c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.992+0000 7f4140c00640 1 --2- 172.21.15.18:0/1586726564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f413c102fe0 0x7f413c100c30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.992+0000 7f4140c00640 1 -- 172.21.15.18:0/1586726564 >> 172.21.15.18:0/1586726564 conn(0x7f413c077ab0 msgr2=0x7f413c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.992+0000 7f4140c00640 1 -- 172.21.15.18:0/1586726564 shutdown_connections 2024-04-16T17:06:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:36.992+0000 7f4140c00640 1 -- 172.21.15.18:0/1586726564 wait complete. 2024-04-16T17:06:37.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 39846 4.63 MB/sec warmup 50 sec latency 20.141 ms 2024-04-16T17:06:37.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:37 smithi022 ceph-mon[36023]: pgmap v59: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-16T17:06:37.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:31 smithi018 ceph-mon[31776]: pgmap v56: 129 pgs: 129 active+clean; 34 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-04-16T17:06:37.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:33 smithi018 ceph-mon[31776]: pgmap v57: 129 pgs: 129 active+clean; 36 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1005 KiB/s wr, 71 op/s 2024-04-16T17:06:37.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:35 smithi018 ceph-mon[31776]: pgmap v58: 129 pgs: 129 active+clean; 36 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 992 KiB/s wr, 67 op/s 2024-04-16T17:06:37.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:36 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/963207620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:37.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:37 smithi018 ceph-mon[31776]: pgmap v59: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-16T17:06:37.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:37 smithi043 ceph-mon[36901]: pgmap v59: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-16T17:06:38.016 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:06:38.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:38.016 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:06:38.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.250+0000 7fc4d1000640 1 -- 172.21.15.18:0/3874507747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4cc109350 msgr2=0x7fc4cc110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.250+0000 7fc4d1000640 1 --2- 172.21.15.18:0/3874507747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4cc109350 0x7fc4cc110f20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009be0 tx=0x7fc4c402bef0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.250+0000 7fc4d1000640 1 -- 172.21.15.18:0/3874507747 shutdown_connections 2024-04-16T17:06:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.250+0000 7fc4d1000640 1 --2- 172.21.15.18:0/3874507747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4cc109350 0x7fc4cc110f20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.250+0000 7fc4d1000640 1 --2- 172.21.15.18:0/3874507747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4cc104c30 0x7fc4cc108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.250+0000 7fc4d1000640 1 --2- 172.21.15.18:0/3874507747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc104280 0x7fc4cc104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.250+0000 7fc4d1000640 1 -- 172.21.15.18:0/3874507747 >> 172.21.15.18:0/3874507747 conn(0x7fc4cc0ffae0 msgr2=0x7fc4cc101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.251+0000 7fc4d1000640 1 -- 172.21.15.18:0/3874507747 shutdown_connections 2024-04-16T17:06:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.251+0000 7fc4d1000640 1 -- 172.21.15.18:0/3874507747 wait complete. 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.251+0000 7fc4d1000640 1 Processor -- start 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4d1000640 1 -- start start 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4d1000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4cc104280 0x7fc4cc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4d1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4cc109350 0x7fc4cc1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4d1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc19a700 0x7fc4cc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4d1000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc114460 con 0x7fc4cc104280 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4d1000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1142e0 con 0x7fc4cc19a700 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4d1000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1145e0 con 0x7fc4cc109350 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4cb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4cc104280 0x7fc4cc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4cb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4cc104280 0x7fc4cc1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54010/0 (socket says 172.21.15.18:54010) 2024-04-16T17:06:38.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.252+0000 7fc4cb400640 1 -- 172.21.15.18:0/2164398603 learned_addr learned my addr 172.21.15.18:0/2164398603 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.253+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc19a700 0x7fc4cc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.253+0000 7fc4cb400640 1 -- 172.21.15.18:0/2164398603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4cc109350 msgr2=0x7fc4cc1a0b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:06:38.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.253+0000 7fc4cb400640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4cc109350 0x7fc4cc1a0b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.253+0000 7fc4cb400640 1 -- 172.21.15.18:0/2164398603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc19a700 msgr2=0x7fc4cc19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.253+0000 7fc4cb400640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc19a700 0x7fc4cc19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.253+0000 7fc4cb400640 1 -- 172.21.15.18:0/2164398603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc19b300 con 0x7fc4cc104280 2024-04-16T17:06:38.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.253+0000 7fc4cb400640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4cc104280 0x7fc4cc1a0640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00e9f0 tx=0x7fc4bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.253+0000 7fc4c3e00640 1 -- 172.21.15.18:0/2164398603 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00cd80 con 0x7fc4cc104280 2024-04-16T17:06:38.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.254+0000 7fc4c3e00640 1 -- 172.21.15.18:0/2164398603 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4bc005500 con 0x7fc4cc104280 2024-04-16T17:06:38.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.254+0000 7fc4c3e00640 1 -- 172.21.15.18:0/2164398603 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc010690 con 0x7fc4cc104280 2024-04-16T17:06:38.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.254+0000 7fc4d1000640 1 -- 172.21.15.18:0/2164398603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc1a73e0 con 0x7fc4cc104280 2024-04-16T17:06:38.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.254+0000 7fc4d1000640 1 -- 172.21.15.18:0/2164398603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc1a7790 con 0x7fc4cc104280 2024-04-16T17:06:38.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.258+0000 7fc4d5388740 1 Processor -- start 2024-04-16T17:06:38.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.258+0000 7fc4d5388740 1 -- start start 2024-04-16T17:06:38.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.258+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b5e90 0x561a246b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.258+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246bee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.259+0000 7fc4c3e00640 1 -- 172.21.15.18:0/2164398603 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc4bc0046c0 con 0x7fc4cc104280 2024-04-16T17:06:38.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.259+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b5e90 0x561a246b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.259+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b5e90 0x561a246b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59986/0 (socket says 172.21.15.18:59986) 2024-04-16T17:06:38.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.259+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 0x561a246c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.259+0000 7fc4d5388740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561a2461acb0 con 0x561a246bf390 2024-04-16T17:06:38.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.259+0000 7fc4d5388740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561a245c1370 con 0x561a246b5e90 2024-04-16T17:06:38.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.259+0000 7fc4d5388740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a24616c70 con 0x561a246b67b0 2024-04-16T17:06:38.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.259+0000 7fc4cbe00640 1 -- 172.21.15.18:0/2945640741 learned_addr learned my addr 172.21.15.18:0/2945640741 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.260+0000 7fc4cb400640 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246bee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.260+0000 7fc4cbe00640 1 -- 172.21.15.18:0/2945640741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 msgr2=0x561a246bee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.260+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246bee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.260+0000 7fc4cbe00640 1 -- 172.21.15.18:0/2945640741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 msgr2=0x561a246c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.260+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 0x561a246c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.260+0000 7fc4cbe00640 1 -- 172.21.15.18:0/2945640741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a245e0910 con 0x561a246b5e90 2024-04-16T17:06:38.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.260+0000 7fc4cb400640 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246bee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:38.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.260+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b5e90 0x561a246b6270 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009be0 tx=0x7fc4c4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e22d80c4aca20a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.261+0000 7fc4c3e00640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4b00a3e20 0x7fc4b00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.261+0000 7fc4c0c00640 1 -- 172.21.15.18:0/2945640741 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4043030 con 0x561a246b5e90 2024-04-16T17:06:38.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.261+0000 7fc4c3e00640 1 -- 172.21.15.18:0/2164398603 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc4bc0d1050 con 0x7fc4cc104280 2024-04-16T17:06:38.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.261+0000 7fc4c0c00640 1 -- 172.21.15.18:0/2945640741 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4033c00 con 0x561a246b5e90 2024-04-16T17:06:38.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.261+0000 7fc4c0c00640 1 -- 172.21.15.18:0/2945640741 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c402c890 con 0x561a246b5e90 2024-04-16T17:06:38.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.261+0000 7fc4cb400640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4b00a3e20 0x7fc4b00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 -- 172.21.15.18:0/2945640741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b5e90 msgr2=0x561a246b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4cb400640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4b00a3e20 0x7fc4b00a62e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x561a246b5960 tx=0x7fc4bc005740 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b5e90 0x561a246b6270 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009be0 tx=0x7fc4c4009840 comp rx=0 tx=0).stop 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 -- 172.21.15.18:0/2945640741 shutdown_connections 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 0x561a246c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246bee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 --2- 172.21.15.18:0/2945640741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b5e90 0x561a246b6270 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 -- 172.21.15.18:0/2945640741 >> 172.21.15.18:0/2945640741 conn(0x561a246b05c0 msgr2=0x561a246b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 -- 172.21.15.18:0/2945640741 shutdown_connections 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 -- 172.21.15.18:0/2945640741 wait complete. 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.263+0000 7fc4d5388740 1 -- start start 2024-04-16T17:06:38.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4d5388740 1 Processor -- start 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246bc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b67b0 0x561a246bc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 0x561a246bcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4d5388740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561a246c2120 con 0x561a246b67b0 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4d5388740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561a245d4730 con 0x561a246bf390 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4d5388740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a246c22a0 con 0x561a246b5e90 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4d5388740 10 client.0 _ll_register_callbacks cb 0x561a245a31e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246bc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246bc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49340/0 (socket says 172.21.15.18:49340) 2024-04-16T17:06:38.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4cbe00640 1 -- 172.21.15.18:0/1948423631 learned_addr learned my addr 172.21.15.18:0/1948423631 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.264+0000 7fc4caa00640 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b67b0 0x561a246bc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 0x561a246bcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4cbe00640 1 -- 172.21.15.18:0/1948423631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 msgr2=0x561a246bcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 0x561a246bcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4cbe00640 1 -- 172.21.15.18:0/1948423631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b67b0 msgr2=0x561a246bc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b67b0 0x561a246bc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4cbe00640 1 -- 172.21.15.18:0/1948423631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a243f57e0 con 0x561a246b5e90 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246bc420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4003cb0 tx=0x7fc4c4003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf96eb171befcd02 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4b5600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.265+0000 7fc4b5600640 1 -- 172.21.15.18:0/1948423631 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4043040 con 0x561a246b5e90 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4b5600640 1 -- 172.21.15.18:0/1948423631 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc4c40334a0 con 0x561a246b5e90 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4d5388740 1 -- 172.21.15.18:0/1948423631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c4033c00 con 0x561a246b5e90 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4b5600640 1 -- 172.21.15.18:0/1948423631 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4c40336a0 con 0x561a246b5e90 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4d5388740 10 client.26173 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:38.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4d5388740 1 -- 172.21.15.18:0/1948423631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561a24455f70 con 0x561a246b5e90 2024-04-16T17:06:38.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4b5600640 1 -- 172.21.15.18:0/1948423631 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4033c00 con 0x561a246b5e90 2024-04-16T17:06:38.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4b5600640 1 -- 172.21.15.18:0/1948423631 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4c4004640 con 0x561a246b5e90 2024-04-16T17:06:38.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4d5388740 10 client.26173 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:38.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4d5388740 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:38.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.266+0000 7fc4d5388740 20 client.26173 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:38.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.267+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561a246cf540 0x561a246d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.267+0000 7fc4d5388740 4 client.26173 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:38.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.267+0000 7fc4d5388740 1 -- 172.21.15.18:0/1948423631 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a246286d0 con 0x561a246cf540 2024-04-16T17:06:38.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.282+0000 7fc4caa00640 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561a246cf540 0x561a246d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.282+0000 7fc4caa00640 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561a246cf540 0x561a246d1930 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f15033c7cee6f7 server_cookie=b68e121f785c72f in_seq=0 out_seq=0 2024-04-16T17:06:38.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.282+0000 7fc4b5600640 10 client.26173 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:38.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.283+0000 7fc4b5600640 1 -- 172.21.15.18:0/1948423631 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a246286d0 con 0x561a246cf540 2024-04-16T17:06:38.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.283+0000 7fc4b5600640 10 client.26173 handle_command_reply: tid=0 2024-04-16T17:06:38.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 client.26173 shutdown 2024-04-16T17:06:38.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 -- 172.21.15.18:0/1948423631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 msgr2=0x561a246bc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246bc420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4003cb0 tx=0x7fc4c4003db0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 -- 172.21.15.18:0/1948423631 shutdown_connections 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561a246cf540 0x561a246d1930 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 0x561a246bcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b67b0 0x561a246bc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4b5600640 0 client.26173 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1948423631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246bc420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 -- 172.21.15.18:0/1948423631 >> 172.21.15.18:0/1948423631 conn(0x561a246b05c0 msgr2=0x561a246b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 -- 172.21.15.18:0/1948423631 shutdown_connections 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 1 -- 172.21.15.18:0/1948423631 wait complete. 2024-04-16T17:06:38.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.284+0000 7fc4d5388740 20 client.26173 trim_cache size 0 max 16384 2024-04-16T17:06:38.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.321+0000 7fc4d5388740 1 Processor -- start 2024-04-16T17:06:38.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.321+0000 7fc4d5388740 1 -- start start 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b5e90 0x561a246beb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 0x561a246d4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4d5388740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b3e10 con 0x561a246b5e90 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4d5388740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b16e0 con 0x561a246bf390 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4d5388740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1af530 con 0x561a246b67b0 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b5e90 0x561a246beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b5e90 0x561a246beb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54024/0 (socket says 172.21.15.18:54024) 2024-04-16T17:06:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4cbe00640 1 -- 172.21.15.18:0/3432014068 learned_addr learned my addr 172.21.15.18:0/3432014068 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.322+0000 7fc4caa00640 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 0x561a246d4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4caa00640 1 -- 172.21.15.18:0/3432014068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 msgr2=0x561a246d4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4caa00640 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 0x561a246d4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4caa00640 1 -- 172.21.15.18:0/3432014068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b5e90 msgr2=0x561a246beb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4caa00640 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b5e90 0x561a246beb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4caa00640 1 -- 172.21.15.18:0/3432014068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a242ed170 con 0x561a246b67b0 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b5e90 0x561a246beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:38.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4caa00640 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246cac30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561a246bbc80 tx=0x7fc4b8005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6df147bcbf75eb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.323+0000 7fc4b7e00640 1 -- 172.21.15.18:0/3432014068 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b801e020 con 0x561a246b67b0 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4b7e00640 1 -- 172.21.15.18:0/3432014068 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b801a880 con 0x561a246b67b0 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4b7e00640 1 -- 172.21.15.18:0/3432014068 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8011bd0 con 0x561a246b67b0 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4d5388740 1 -- 172.21.15.18:0/3432014068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 msgr2=0x561a246cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4d5388740 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246cac30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x561a246bbc80 tx=0x7fc4b8005760 comp rx=0 tx=0).stop 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4d5388740 1 -- 172.21.15.18:0/3432014068 shutdown_connections 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4d5388740 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246bf390 0x561a246d4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4d5388740 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b67b0 0x561a246cac30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4d5388740 1 --2- 172.21.15.18:0/3432014068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246b5e90 0x561a246beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.324+0000 7fc4d5388740 1 -- 172.21.15.18:0/3432014068 >> 172.21.15.18:0/3432014068 conn(0x561a246b05c0 msgr2=0x561a246ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.325+0000 7fc4d5388740 1 -- 172.21.15.18:0/3432014068 shutdown_connections 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.325+0000 7fc4d5388740 1 -- 172.21.15.18:0/3432014068 wait complete. 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.325+0000 7fc4d5388740 1 -- start start 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.325+0000 7fc4d5388740 1 Processor -- start 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b67b0 0x561a246d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4d5388740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 0x561a246d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4d5388740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1afba0 con 0x561a246bf390 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4d5388740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b1010 con 0x561a246b67b0 2024-04-16T17:06:38.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4d5388740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b0270 con 0x561a246b5e90 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4d5388740 10 client.0 _ll_register_callbacks cb 0x7fc4c402ce00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49356/0 (socket says 172.21.15.18:49356) 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4cbe00640 1 -- 172.21.15.18:0/1206376459 learned_addr learned my addr 172.21.15.18:0/1206376459 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 0x561a246d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4caa00640 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b67b0 0x561a246d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4cbe00640 1 -- 172.21.15.18:0/1206376459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b67b0 msgr2=0x561a246d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b67b0 0x561a246d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4cbe00640 1 -- 172.21.15.18:0/1206376459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 msgr2=0x561a246d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.326+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 0x561a246d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4cbe00640 1 -- 172.21.15.18:0/1206376459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a247347a0 con 0x561a246b5e90 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4cbe00640 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246d2410 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4002230 tx=0x7fc4c402ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e8a84a935ee7d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4b5600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4b5600640 1 -- 172.21.15.18:0/1206376459 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4043030 con 0x561a246b5e90 2024-04-16T17:06:38.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4d5388740 1 -- 172.21.15.18:0/1206376459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b801a880 con 0x561a246b5e90 2024-04-16T17:06:38.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4b5600640 1 -- 172.21.15.18:0/1206376459 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc4c4033840 con 0x561a246b5e90 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4b5600640 1 -- 172.21.15.18:0/1206376459 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4c4009800 con 0x561a246b5e90 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4d5388740 10 client.26179 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.327+0000 7fc4d5388740 1 -- 172.21.15.18:0/1206376459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561a24735ac0 con 0x561a246b5e90 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.328+0000 7fc4b5600640 1 -- 172.21.15.18:0/1206376459 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b801a880 con 0x561a246b5e90 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.328+0000 7fc4b5600640 1 -- 172.21.15.18:0/1206376459 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4c403fb00 con 0x561a246b5e90 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.328+0000 7fc4d5388740 10 client.26179 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.328+0000 7fc4d5388740 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.328+0000 7fc4d5388740 20 client.26179 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.328+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561a247361b0 0x561a247385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.328+0000 7fc4d5388740 4 client.26179 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.328+0000 7fc4d5388740 1 -- 172.21.15.18:0/1206376459 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561a24622540 con 0x561a247361b0 2024-04-16T17:06:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.329+0000 7fc4caa00640 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561a247361b0 0x561a247385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.329+0000 7fc4caa00640 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561a247361b0 0x561a247385d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a55de1d6d7da678 server_cookie=8b728c6decfb3c61 in_seq=0 out_seq=0 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.329+0000 7fc4b5600640 10 client.26179 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4b5600640 1 -- 172.21.15.18:0/1206376459 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a24622540 con 0x561a247361b0 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4b5600640 10 client.26179 handle_command_reply: tid=0 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 client.26179 shutdown 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 -- 172.21.15.18:0/1206376459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 msgr2=0x561a246d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246d2410 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4002230 tx=0x7fc4c402ca40 comp rx=0 tx=0).stop 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 -- 172.21.15.18:0/1206376459 shutdown_connections 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561a247361b0 0x561a247385d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561a246bf390 0x561a246d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a246b67b0 0x561a246d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 --2- 172.21.15.18:0/1206376459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a246b5e90 0x561a246d2410 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 -- 172.21.15.18:0/1206376459 >> 172.21.15.18:0/1206376459 conn(0x561a246b05c0 msgr2=0x561a246bcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4b5600640 0 client.26179 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.331+0000 7fc4d5388740 1 -- 172.21.15.18:0/1206376459 shutdown_connections 2024-04-16T17:06:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.332+0000 7fc4d5388740 1 -- 172.21.15.18:0/1206376459 wait complete. 2024-04-16T17:06:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.332+0000 7fc4d5388740 20 client.26179 trim_cache size 0 max 16384 2024-04-16T17:06:38.335 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:06:38.335 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:06:38.335 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "a40d8296-bfe8-4e5d-94bd-ec3f945f94ce", 2024-04-16T17:06:38.335 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:06:38.336 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:06:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.332+0000 7fc4d1000640 1 -- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4b00a3e20 msgr2=0x7fc4b00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.332+0000 7fc4d1000640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4b00a3e20 0x7fc4b00a62e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x561a246b5960 tx=0x7fc4bc005740 comp rx=0 tx=0).stop 2024-04-16T17:06:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.332+0000 7fc4d1000640 1 -- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4cc104280 msgr2=0x7fc4cc1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.332+0000 7fc4d1000640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4cc104280 0x7fc4cc1a0640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00e9f0 tx=0x7fc4bc00edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.333+0000 7fc4d1000640 1 -- 172.21.15.18:0/2164398603 shutdown_connections 2024-04-16T17:06:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.333+0000 7fc4d1000640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc4b00a3e20 0x7fc4b00a62e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.333+0000 7fc4d1000640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc19a700 0x7fc4cc19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.333+0000 7fc4d1000640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4cc109350 0x7fc4cc1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.333+0000 7fc4d1000640 1 --2- 172.21.15.18:0/2164398603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4cc104280 0x7fc4cc1a0640 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.333+0000 7fc4d1000640 1 -- 172.21.15.18:0/2164398603 >> 172.21.15.18:0/2164398603 conn(0x7fc4cc0ffae0 msgr2=0x7fc4cc1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.333+0000 7fc4d1000640 1 -- 172.21.15.18:0/2164398603 shutdown_connections 2024-04-16T17:06:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.334+0000 7fc4d1000640 1 -- 172.21.15.18:0/2164398603 wait complete. 2024-04-16T17:06:38.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:06:38.353 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:06:38.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 40159 4.57 MB/sec warmup 51 sec latency 112.022 ms 2024-04-16T17:06:38.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 -- 172.21.15.18:0/3449576878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5680743a0 msgr2=0x7fc568074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3449576878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5680743a0 0x7fc568074780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc558009ca0 tx=0x7fc55802c180 comp rx=0 tx=0).stop 2024-04-16T17:06:38.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 -- 172.21.15.18:0/3449576878 shutdown_connections 2024-04-16T17:06:38.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3449576878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc568112e10 0x7fc568115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3449576878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc568074cc0 0x7fc5681128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3449576878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5680743a0 0x7fc568074780 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 -- 172.21.15.18:0/3449576878 >> 172.21.15.18:0/3449576878 conn(0x7fc5680fd950 msgr2=0x7fc5680ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 -- 172.21.15.18:0/3449576878 shutdown_connections 2024-04-16T17:06:38.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.537+0000 7fc56cc00640 1 -- 172.21.15.18:0/3449576878 wait complete. 2024-04-16T17:06:38.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.538+0000 7fc56cc00640 1 Processor -- start 2024-04-16T17:06:38.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.538+0000 7fc56cc00640 1 -- start start 2024-04-16T17:06:38.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.538+0000 7fc56cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5680743a0 0x7fc5681a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.538+0000 7fc56cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc568074cc0 0x7fc5681a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.538+0000 7fc56cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc568112e10 0x7fc56819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc56cc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681187e0 con 0x7fc5680743a0 2024-04-16T17:06:38.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc56cc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc568118660 con 0x7fc568074cc0 2024-04-16T17:06:38.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc56cc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc568118960 con 0x7fc568112e10 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc55ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc568074cc0 0x7fc5681a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc567400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5680743a0 0x7fc5681a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc55ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc568074cc0 0x7fc5681a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60046/0 (socket says 172.21.15.18:60046) 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc55ea00640 1 -- 172.21.15.18:0/3028426846 learned_addr learned my addr 172.21.15.18:0/3028426846 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc567e00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc568112e10 0x7fc56819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc55ea00640 1 -- 172.21.15.18:0/3028426846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc568112e10 msgr2=0x7fc56819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc568112e10 0x7fc56819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc55ea00640 1 -- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5680743a0 msgr2=0x7fc5681a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5680743a0 0x7fc5681a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.539+0000 7fc55ea00640 1 -- 172.21.15.18:0/3028426846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56819f3e0 con 0x7fc568074cc0 2024-04-16T17:06:38.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.540+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc568074cc0 0x7fc5681a4fa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc55000e9f0 tx=0x7fc55000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.540+0000 7fc564c00640 1 -- 172.21.15.18:0/3028426846 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55000cd80 con 0x7fc568074cc0 2024-04-16T17:06:38.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.540+0000 7fc564c00640 1 -- 172.21.15.18:0/3028426846 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc550010910 con 0x7fc568074cc0 2024-04-16T17:06:38.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.540+0000 7fc564c00640 1 -- 172.21.15.18:0/3028426846 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc550018980 con 0x7fc568074cc0 2024-04-16T17:06:38.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.540+0000 7fc56cc00640 1 -- 172.21.15.18:0/3028426846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56819f6d0 con 0x7fc568074cc0 2024-04-16T17:06:38.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.540+0000 7fc56cc00640 1 -- 172.21.15.18:0/3028426846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5681ab9e0 con 0x7fc568074cc0 2024-04-16T17:06:38.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.544+0000 7fc564c00640 1 -- 172.21.15.18:0/3028426846 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc550018b20 con 0x7fc568074cc0 2024-04-16T17:06:38.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.544+0000 7fc570e2c740 1 Processor -- start 2024-04-16T17:06:38.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.544+0000 7fc570e2c740 1 -- start start 2024-04-16T17:06:38.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.544+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.544+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 0x557e6fcfbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc570e2c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557e6fc56120 con 0x557e6fcf0390 2024-04-16T17:06:38.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc570e2c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e6fbfc340 con 0x557e6fcf0cb0 2024-04-16T17:06:38.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc570e2c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557e6fc520e0 con 0x557e6fcf9890 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54060/0 (socket says 172.21.15.18:54060) 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567e00640 1 -- 172.21.15.18:0/134477511 learned_addr learned my addr 172.21.15.18:0/134477511 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc55ea00640 1 --2- 172.21.15.18:0/134477511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 0x557e6fcfbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567e00640 1 -- 172.21.15.18:0/134477511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 msgr2=0x557e6fcfbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567e00640 1 --2- 172.21.15.18:0/134477511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 0x557e6fcfbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567e00640 1 -- 172.21.15.18:0/134477511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0cb0 msgr2=0x557e6fcf9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567e00640 1 --2- 172.21.15.18:0/134477511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.545+0000 7fc567e00640 1 -- 172.21.15.18:0/134477511 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6fa327d0 con 0x557e6fcf0390 2024-04-16T17:06:38.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.546+0000 7fc567e00640 1 --2- 172.21.15.18:0/134477511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf0770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc560009ba0 tx=0x7fc560011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae7e47830f2740e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.546+0000 7fc557e00640 1 -- 172.21.15.18:0/134477511 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560004230 con 0x557e6fcf0390 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.546+0000 7fc557e00640 1 -- 172.21.15.18:0/134477511 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc560009e10 con 0x557e6fcf0390 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.546+0000 7fc557e00640 1 -- 172.21.15.18:0/134477511 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560004d70 con 0x557e6fcf0390 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.547+0000 7fc570e2c740 1 -- 172.21.15.18:0/134477511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 msgr2=0x557e6fcf0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.547+0000 7fc570e2c740 1 --2- 172.21.15.18:0/134477511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf0770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc560009ba0 tx=0x7fc560011810 comp rx=0 tx=0).stop 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.547+0000 7fc564c00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc54c0a3da0 0x7fc54c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.547+0000 7fc570e2c740 1 -- 172.21.15.18:0/134477511 shutdown_connections 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.547+0000 7fc570e2c740 1 --2- 172.21.15.18:0/134477511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 0x557e6fcfbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.547+0000 7fc570e2c740 1 --2- 172.21.15.18:0/134477511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.547+0000 7fc570e2c740 1 --2- 172.21.15.18:0/134477511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf0770 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.547+0000 7fc570e2c740 1 -- 172.21.15.18:0/134477511 >> 172.21.15.18:0/134477511 conn(0x557e6fceaac0 msgr2=0x557e6fceaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.548+0000 7fc564c00640 1 -- 172.21.15.18:0/3028426846 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc5500d0ec0 con 0x7fc568074cc0 2024-04-16T17:06:38.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.548+0000 7fc567e00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc54c0a3da0 0x7fc54c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.548+0000 7fc570e2c740 1 -- 172.21.15.18:0/134477511 shutdown_connections 2024-04-16T17:06:38.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.548+0000 7fc570e2c740 1 -- 172.21.15.18:0/134477511 wait complete. 2024-04-16T17:06:38.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc570e2c740 1 -- start start 2024-04-16T17:06:38.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc567e00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc54c0a3da0 0x7fc54c0a6260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc560011d10 tx=0x7fc560005850 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc570e2c740 1 Processor -- start 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fcf7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc570e2c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557e6fcfc680 con 0x557e6fcf0390 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc570e2c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e6fc0f710 con 0x557e6fcf9890 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.549+0000 7fc570e2c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557e6fcfc800 con 0x557e6fcf0cb0 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc570e2c740 10 client.0 _ll_register_callbacks cb 0x557e6fc0aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc55ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc567e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fcf7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc567e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fcf7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60064/0 (socket says 172.21.15.18:60064) 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc55ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49392/0 (socket says 172.21.15.18:49392) 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc567e00640 1 -- 172.21.15.18:0/3207286577 learned_addr learned my addr 172.21.15.18:0/3207286577 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc567400640 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc55ea00640 1 -- 172.21.15.18:0/3207286577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 msgr2=0x557e6fcf7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fcf7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.550+0000 7fc55ea00640 1 -- 172.21.15.18:0/3207286577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 msgr2=0x557e6fcf68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc55ea00640 1 -- 172.21.15.18:0/3207286577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6f9683f0 con 0x557e6fcf0cb0 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc567e00640 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fcf7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc567400640 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf6e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc5500ce200 tx=0x7fc550018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e33dee9b61040508 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc554c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc554c00640 1 -- 172.21.15.18:0/3207286577 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5500bd0b0 con 0x557e6fcf0cb0 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc554c00640 1 -- 172.21.15.18:0/3207286577 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc5500ce480 con 0x557e6fcf0cb0 2024-04-16T17:06:38.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc570e2c740 1 -- 172.21.15.18:0/3207286577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc560009e10 con 0x557e6fcf0cb0 2024-04-16T17:06:38.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc554c00640 1 -- 172.21.15.18:0/3207286577 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5500ce680 con 0x557e6fcf0cb0 2024-04-16T17:06:38.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc570e2c740 10 client.26191 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:38.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.551+0000 7fc570e2c740 1 -- 172.21.15.18:0/3207286577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e6f9280f0 con 0x557e6fcf0cb0 2024-04-16T17:06:38.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.552+0000 7fc554c00640 1 -- 172.21.15.18:0/3207286577 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc560009e10 con 0x557e6fcf0cb0 2024-04-16T17:06:38.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.552+0000 7fc554c00640 1 -- 172.21.15.18:0/3207286577 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc55009bf50 con 0x557e6fcf0cb0 2024-04-16T17:06:38.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.552+0000 7fc570e2c740 10 client.26191 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:38.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.552+0000 7fc570e2c740 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:38.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.553+0000 7fc570e2c740 20 client.26191 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.553+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e6fd09bb0 0x557e6fd0c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.553+0000 7fc570e2c740 4 client.26191 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.553+0000 7fc570e2c740 1 -- 172.21.15.18:0/3207286577 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e6fc63b40 con 0x557e6fd09bb0 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.553+0000 7fc567400640 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e6fd09bb0 0x557e6fd0c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.554+0000 7fc567400640 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e6fd09bb0 0x557e6fd0c000 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3226da3c63745f server_cookie=8f5d19de152bc84 in_seq=0 out_seq=0 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.554+0000 7fc554c00640 10 client.26191 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.554+0000 7fc554c00640 1 -- 172.21.15.18:0/3207286577 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e6fc63b40 con 0x557e6fd09bb0 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.554+0000 7fc554c00640 10 client.26191 handle_command_reply: tid=0 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.554+0000 7fc570e2c740 1 client.26191 shutdown 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 -- 172.21.15.18:0/3207286577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf0cb0 msgr2=0x557e6fcf6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf6e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc5500ce200 tx=0x7fc550018420 comp rx=0 tx=0).stop 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 -- 172.21.15.18:0/3207286577 shutdown_connections 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e6fd09bb0 0x557e6fd0c000 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fcf7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf0cb0 0x557e6fcf6e10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3207286577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc554c00640 0 client.26191 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:38.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 -- 172.21.15.18:0/3207286577 >> 172.21.15.18:0/3207286577 conn(0x557e6fceaac0 msgr2=0x557e6fcec810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 -- 172.21.15.18:0/3207286577 shutdown_connections 2024-04-16T17:06:38.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 1 -- 172.21.15.18:0/3207286577 wait complete. 2024-04-16T17:06:38.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.555+0000 7fc570e2c740 20 client.26191 trim_cache size 0 max 16384 2024-04-16T17:06:38.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.585+0000 7fc570e2c740 1 Processor -- start 2024-04-16T17:06:38.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.585+0000 7fc570e2c740 1 -- start start 2024-04-16T17:06:38.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.585+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.585+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fd05570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcec600 0x557e6fd118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc570e2c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681afb10 con 0x557e6fcf0390 2024-04-16T17:06:38.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc570e2c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681ad3e0 con 0x557e6fcf9890 2024-04-16T17:06:38.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc570e2c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc56810df20 con 0x557e6fcec600 2024-04-16T17:06:38.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc567400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc567400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54080/0 (socket says 172.21.15.18:54080) 2024-04-16T17:06:38.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc567400640 1 -- 172.21.15.18:0/766615054 learned_addr learned my addr 172.21.15.18:0/766615054 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc55ea00640 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fd05570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc567e00640 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcec600 0x557e6fd118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc55ea00640 1 -- 172.21.15.18:0/766615054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcec600 msgr2=0x557e6fd118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc55ea00640 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcec600 0x557e6fd118d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc55ea00640 1 -- 172.21.15.18:0/766615054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 msgr2=0x557e6fcf9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc55ea00640 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc55ea00640 1 -- 172.21.15.18:0/766615054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6fcf4e20 con 0x557e6fcf9890 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc567e00640 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcec600 0x557e6fd118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.586+0000 7fc567400640 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.590+0000 7fc55ea00640 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fd05570 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x557e6fcf7ea0 tx=0x7fc550000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1bd1e02673ffb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.590+0000 7fc557400640 1 -- 172.21.15.18:0/766615054 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55009cb00 con 0x557e6fcf9890 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.590+0000 7fc557400640 1 -- 172.21.15.18:0/766615054 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5500a0e00 con 0x557e6fcf9890 2024-04-16T17:06:38.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.590+0000 7fc557400640 1 -- 172.21.15.18:0/766615054 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc550098850 con 0x557e6fcf9890 2024-04-16T17:06:38.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.591+0000 7fc570e2c740 1 -- 172.21.15.18:0/766615054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 msgr2=0x557e6fd05570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.591+0000 7fc570e2c740 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fd05570 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x557e6fcf7ea0 tx=0x7fc550000bc0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.591+0000 7fc570e2c740 1 -- 172.21.15.18:0/766615054 shutdown_connections 2024-04-16T17:06:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.591+0000 7fc570e2c740 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcec600 0x557e6fd118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.591+0000 7fc570e2c740 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf9890 0x557e6fd05570 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.591+0000 7fc570e2c740 1 --2- 172.21.15.18:0/766615054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcf0390 0x557e6fcf9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.591+0000 7fc570e2c740 1 -- 172.21.15.18:0/766615054 >> 172.21.15.18:0/766615054 conn(0x557e6fceaac0 msgr2=0x557e6fcebef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.592+0000 7fc570e2c740 1 -- 172.21.15.18:0/766615054 shutdown_connections 2024-04-16T17:06:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.592+0000 7fc570e2c740 1 -- 172.21.15.18:0/766615054 wait complete. 2024-04-16T17:06:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.592+0000 7fc570e2c740 1 -- start start 2024-04-16T17:06:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.592+0000 7fc570e2c740 1 Processor -- start 2024-04-16T17:06:38.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.593+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcec600 0x557e6fd68480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.593+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0390 0x557e6fd689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.593+0000 7fc570e2c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 0x557e6fcfdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.593+0000 7fc570e2c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc56810e590 con 0x557e6fcec600 2024-04-16T17:06:38.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.593+0000 7fc570e2c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681acd10 con 0x557e6fcf0390 2024-04-16T17:06:38.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.593+0000 7fc570e2c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681abf70 con 0x557e6fcf9890 2024-04-16T17:06:38.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.593+0000 7fc570e2c740 10 client.0 _ll_register_callbacks cb 0x7fc5500d4660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:38.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc567400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcec600 0x557e6fd68480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc55ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0390 0x557e6fd689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc55ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0390 0x557e6fd689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60082/0 (socket says 172.21.15.18:60082) 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc55ea00640 1 -- 172.21.15.18:0/3192561649 learned_addr learned my addr 172.21.15.18:0/3192561649 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc567400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcec600 0x557e6fd68480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54094/0 (socket says 172.21.15.18:54094) 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc567e00640 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 0x557e6fcfdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc55ea00640 1 -- 172.21.15.18:0/3192561649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 msgr2=0x557e6fcfdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 0x557e6fcfdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc55ea00640 1 -- 172.21.15.18:0/3192561649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcec600 msgr2=0x557e6fd68480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcec600 0x557e6fd68480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.594+0000 7fc55ea00640 1 -- 172.21.15.18:0/3192561649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6fcfe560 con 0x557e6fcf0390 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc55ea00640 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0390 0x557e6fd689c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc550013040 tx=0x7fc55009b010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2444576ff4803ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc554c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc554c00640 1 -- 172.21.15.18:0/3192561649 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55009c640 con 0x557e6fcf0390 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc554c00640 1 -- 172.21.15.18:0/3192561649 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc550010630 con 0x557e6fcf0390 2024-04-16T17:06:38.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc570e2c740 1 -- 172.21.15.18:0/3192561649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5500a0e00 con 0x557e6fcf0390 2024-04-16T17:06:38.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc554c00640 1 -- 172.21.15.18:0/3192561649 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc55009cb00 con 0x557e6fcf0390 2024-04-16T17:06:38.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc570e2c740 10 client.26201 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:38.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc570e2c740 1 -- 172.21.15.18:0/3192561649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e6fd70d20 con 0x557e6fcf0390 2024-04-16T17:06:38.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc554c00640 1 -- 172.21.15.18:0/3192561649 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5500a0e00 con 0x557e6fcf0390 2024-04-16T17:06:38.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.595+0000 7fc554c00640 1 -- 172.21.15.18:0/3192561649 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc5500a1f90 con 0x557e6fcf0390 2024-04-16T17:06:38.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc570e2c740 10 client.26201 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:38.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc570e2c740 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:38.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc570e2c740 20 client.26201 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e6fd71110 0x557e6fd73530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc570e2c740 4 client.26201 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc570e2c740 1 -- 172.21.15.18:0/3192561649 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e6fc5d9b0 con 0x557e6fd71110 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc567400640 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e6fd71110 0x557e6fd73530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc567400640 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e6fd71110 0x557e6fd73530 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f15f7cc3a663a6 server_cookie=1047e61bff892a72 in_seq=0 out_seq=0 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.596+0000 7fc554c00640 10 client.26201 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc554c00640 1 -- 172.21.15.18:0/3192561649 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557e6fc5d9b0 con 0x557e6fd71110 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc554c00640 10 client.26201 handle_command_reply: tid=0 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 client.26201 shutdown 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 -- 172.21.15.18:0/3192561649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0390 msgr2=0x557e6fd689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0390 0x557e6fd689c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc550013040 tx=0x7fc55009b010 comp rx=0 tx=0).stop 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 -- 172.21.15.18:0/3192561649 shutdown_connections 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e6fd71110 0x557e6fd73530 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e6fcf9890 0x557e6fcfdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e6fcf0390 0x557e6fd689c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 --2- 172.21.15.18:0/3192561649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e6fcec600 0x557e6fd68480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc554c00640 0 client.26201 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 -- 172.21.15.18:0/3192561649 >> 172.21.15.18:0/3192561649 conn(0x557e6fceaac0 msgr2=0x557e6fcf6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 -- 172.21.15.18:0/3192561649 shutdown_connections 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 1 -- 172.21.15.18:0/3192561649 wait complete. 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.597+0000 7fc570e2c740 20 client.26201 trim_cache size 0 max 16384 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:06:38.601 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:06:38.602 INFO:teuthology.orchestra.run.smithi018.stdout: "a40d8296-bfe8-4e5d-94bd-ec3f945f94ce": { 2024-04-16T17:06:38.602 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:06:38.602 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "a40d8296-bfe8-4e5d-94bd-ec3f945f94ce", 2024-04-16T17:06:38.602 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:06:38.602 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:06:38.602 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:06:38.602 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.598+0000 7fc56cc00640 1 -- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc54c0a3da0 msgr2=0x7fc54c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.598+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc54c0a3da0 0x7fc54c0a6260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc560011d10 tx=0x7fc560005850 comp rx=0 tx=0).stop 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.598+0000 7fc56cc00640 1 -- 172.21.15.18:0/3028426846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc568074cc0 msgr2=0x7fc5681a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.598+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc568074cc0 0x7fc5681a4fa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc55000e9f0 tx=0x7fc55000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.600+0000 7fc56cc00640 1 -- 172.21.15.18:0/3028426846 shutdown_connections 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.600+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc54c0a3da0 0x7fc54c0a6260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.600+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc568112e10 0x7fc56819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.600+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc568074cc0 0x7fc5681a4fa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.600+0000 7fc56cc00640 1 --2- 172.21.15.18:0/3028426846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5680743a0 0x7fc5681a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.600+0000 7fc56cc00640 1 -- 172.21.15.18:0/3028426846 >> 172.21.15.18:0/3028426846 conn(0x7fc5680fd950 msgr2=0x7fc5680fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.600+0000 7fc56cc00640 1 -- 172.21.15.18:0/3028426846 shutdown_connections 2024-04-16T17:06:38.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:06:38.600+0000 7fc56cc00640 1 -- 172.21.15.18:0/3028426846 wait complete. 2024-04-16T17:06:38.620 INFO:tasks.cephfs.filesystem:scrub status for tag:a40d8296-bfe8-4e5d-94bd-ec3f945f94ce - {'path': '/', 'tag': 'a40d8296-bfe8-4e5d-94bd-ec3f945f94ce', 'options': 'recursive,force'} 2024-04-16T17:06:39.515 INFO:tasks.workunit.client.0.smithi018.stdout: 1 40512 4.54 MB/sec warmup 52 sec latency 96.569 ms 2024-04-16T17:06:39.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:39 smithi022 ceph-mon[36023]: pgmap v60: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-04-16T17:06:39.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:39 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:06:39.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:39 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:39.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:39 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:06:39.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:39 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:39.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:39 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:06:39.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:39 smithi043 ceph-mon[36901]: pgmap v60: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-04-16T17:06:39.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:39 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:06:39.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:39 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:39.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:39 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:06:39.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:39 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:39.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:39 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:06:40.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 42029 4.60 MB/sec warmup 53 sec latency 16.534 ms 2024-04-16T17:06:41.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 42986 4.64 MB/sec warmup 54 sec latency 15.672 ms 2024-04-16T17:06:41.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:41 smithi022 ceph-mon[36023]: pgmap v61: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-04-16T17:06:41.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:41 smithi043 ceph-mon[36901]: pgmap v61: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-04-16T17:06:42.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 43656 4.59 MB/sec warmup 55 sec latency 26.232 ms 2024-04-16T17:06:43.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 44023 4.56 MB/sec warmup 56 sec latency 55.042 ms 2024-04-16T17:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 -- 172.21.15.18:0/3703016057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c107160 msgr2=0x7fd50c10ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 --2- 172.21.15.18:0/3703016057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c107160 0x7fd50c10ed30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd504009ca0 tx=0x7fd50402c100 comp rx=0 tx=0).stop 2024-04-16T17:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 -- 172.21.15.18:0/3703016057 shutdown_connections 2024-04-16T17:06:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 --2- 172.21.15.18:0/3703016057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c107160 0x7fd50c10ed30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 --2- 172.21.15.18:0/3703016057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd50c102a40 0x7fd50c106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 --2- 172.21.15.18:0/3703016057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd50c102090 0x7fd50c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 -- 172.21.15.18:0/3703016057 >> 172.21.15.18:0/3703016057 conn(0x7fd50c0fd910 msgr2=0x7fd50c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 -- 172.21.15.18:0/3703016057 shutdown_connections 2024-04-16T17:06:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.622+0000 7fd512000640 1 -- 172.21.15.18:0/3703016057 wait complete. 2024-04-16T17:06:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.623+0000 7fd512000640 1 Processor -- start 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.623+0000 7fd512000640 1 -- start start 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.623+0000 7fd512000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c102090 0x7fd50c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.623+0000 7fd512000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd50c107160 0x7fd50c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.623+0000 7fd512000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd50c19a7a0 0x7fd50c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.623+0000 7fd512000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c112220 con 0x7fd50c107160 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.623+0000 7fd512000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1120a0 con 0x7fd50c102090 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.623+0000 7fd512000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1123a0 con 0x7fd50c19a7a0 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c102090 0x7fd50c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c102090 0x7fd50c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60100/0 (socket says 172.21.15.18:60100) 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 -- 172.21.15.18:0/391898679 learned_addr learned my addr 172.21.15.18:0/391898679 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd511600640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd50c19a7a0 0x7fd50c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd50be00640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd50c107160 0x7fd50c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 -- 172.21.15.18:0/391898679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd50c19a7a0 msgr2=0x7fd50c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd50c19a7a0 0x7fd50c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 -- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd50c107160 msgr2=0x7fd50c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd50c107160 0x7fd50c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 -- 172.21.15.18:0/391898679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50c19b330 con 0x7fd50c102090 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.624+0000 7fd510c00640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c102090 0x7fd50c1a0630 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd50000d770 tx=0x7fd50000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.625+0000 7fd509600640 1 -- 172.21.15.18:0/391898679 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500017070 con 0x7fd50c102090 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.625+0000 7fd509600640 1 -- 172.21.15.18:0/391898679 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd500005500 con 0x7fd50c102090 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.625+0000 7fd512000640 1 -- 172.21.15.18:0/391898679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50c19b620 con 0x7fd50c102090 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.625+0000 7fd509600640 1 -- 172.21.15.18:0/391898679 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500004e00 con 0x7fd50c102090 2024-04-16T17:06:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.625+0000 7fd512000640 1 -- 172.21.15.18:0/391898679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50c1a77b0 con 0x7fd50c102090 2024-04-16T17:06:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.628+0000 7fd509600640 1 -- 172.21.15.18:0/391898679 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd500004fa0 con 0x7fd50c102090 2024-04-16T17:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.629+0000 7fd5163e6740 1 Processor -- start 2024-04-16T17:06:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.629+0000 7fd5163e6740 1 -- start start 2024-04-16T17:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.629+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.629+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b206f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.629+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 0x556b207033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.629+0000 7fd5163e6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556b2065c780 con 0x556b206f7a80 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.629+0000 7fd5163e6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556b20602f40 con 0x556b20701000 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.629+0000 7fd5163e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556b20658740 con 0x556b206f87f0 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd509600640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4f40a3d00 0x7fd4f40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd510c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b206f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd511600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd510c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b206f7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54132/0 (socket says 172.21.15.18:54132) 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd50be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 0x556b207033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd510c00640 1 -- 172.21.15.18:0/390898198 learned_addr learned my addr 172.21.15.18:0/390898198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd509600640 1 -- 172.21.15.18:0/391898679 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd5000d05e0 con 0x7fd50c102090 2024-04-16T17:06:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd510c00640 1 -- 172.21.15.18:0/390898198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 msgr2=0x556b206f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd510c00640 1 --2- 172.21.15.18:0/390898198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd510c00640 1 -- 172.21.15.18:0/390898198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 msgr2=0x556b207033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd510c00640 1 --2- 172.21.15.18:0/390898198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 0x556b207033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.630+0000 7fd510c00640 1 -- 172.21.15.18:0/390898198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b2036f210 con 0x556b206f7a80 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.631+0000 7fd511600640 1 --2- 172.21.15.18:0/390898198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.631+0000 7fd511600640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4f40a3d00 0x7fd4f40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.631+0000 7fd510c00640 1 --2- 172.21.15.18:0/390898198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b206f7f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd50009c1b0 tx=0x7fd5000048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aad35da58a2cadd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.631+0000 7fd4fdc00640 1 -- 172.21.15.18:0/390898198 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500020040 con 0x556b206f7a80 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.631+0000 7fd4fdc00640 1 -- 172.21.15.18:0/390898198 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5000201e0 con 0x556b206f7a80 2024-04-16T17:06:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.631+0000 7fd4fdc00640 1 -- 172.21.15.18:0/390898198 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000cdba0 con 0x556b206f7a80 2024-04-16T17:06:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 -- 172.21.15.18:0/390898198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 msgr2=0x556b206f7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 --2- 172.21.15.18:0/390898198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b206f7f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd50009c1b0 tx=0x7fd5000048a0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd511600640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4f40a3d00 0x7fd4f40a61c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd504005e30 tx=0x7fd50402e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 -- 172.21.15.18:0/390898198 shutdown_connections 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 --2- 172.21.15.18:0/390898198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 0x556b207033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 --2- 172.21.15.18:0/390898198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b206f7f00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 --2- 172.21.15.18:0/390898198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 -- 172.21.15.18:0/390898198 >> 172.21.15.18:0/390898198 conn(0x556b206925d0 msgr2=0x556b206f2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 -- 172.21.15.18:0/390898198 shutdown_connections 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.632+0000 7fd5163e6740 1 -- 172.21.15.18:0/390898198 wait complete. 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 1 -- start start 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 1 Processor -- start 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f7a80 0x556b206fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f87f0 0x556b206fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 0x556b206fe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556b20703cd0 con 0x556b206f87f0 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556b20616390 con 0x556b20701000 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556b20703e50 con 0x556b206f7a80 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.633+0000 7fd5163e6740 10 client.0 _ll_register_callbacks cb 0x556b205fff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd50be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f87f0 0x556b206fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f7a80 0x556b206fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f7a80 0x556b206fdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49428/0 (socket says 172.21.15.18:49428) 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 -- 172.21.15.18:0/96207262 learned_addr learned my addr 172.21.15.18:0/96207262 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd50be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f87f0 0x556b206fe400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54140/0 (socket says 172.21.15.18:54140) 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 -- 172.21.15.18:0/96207262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 msgr2=0x556b206fe940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:06:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 0x556b206fe940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 -- 172.21.15.18:0/96207262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f87f0 msgr2=0x556b206fe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f87f0 0x556b206fe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 -- 172.21.15.18:0/96207262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b2043b620 con 0x556b206f7a80 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd510c00640 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f7a80 0x556b206fdec0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd5000d08d0 tx=0x7fd5000a0590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb576665e0ac98d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd4f2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.634+0000 7fd4f2000640 1 -- 172.21.15.18:0/96207262 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000d4070 con 0x556b206f7a80 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 1 -- 172.21.15.18:0/96207262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b2066a1a0 con 0x556b206f7a80 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd4f2000640 1 -- 172.21.15.18:0/96207262 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd5000cdba0 con 0x556b206f7a80 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd4f2000640 1 -- 172.21.15.18:0/96207262 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd5000b5020 con 0x556b206f7a80 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 10 client.26209 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 1 -- 172.21.15.18:0/96207262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b20703cd0 con 0x556b206f7a80 2024-04-16T17:06:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd4f2000640 1 -- 172.21.15.18:0/96207262 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556b2066a1a0 con 0x556b206f7a80 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd4f2000640 1 -- 172.21.15.18:0/96207262 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd50009b670 con 0x556b206f7a80 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 10 client.26209 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 20 client.26209 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556b20710fc0 0x556b207133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 4 client.26209 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.635+0000 7fd5163e6740 1 -- 172.21.15.18:0/96207262 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b20664010 con 0x556b20710fc0 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.636+0000 7fd50be00640 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556b20710fc0 0x556b207133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.636+0000 7fd50be00640 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556b20710fc0 0x556b207133b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8442ce42eaddf9ee server_cookie=1639fd84997e6e40 in_seq=0 out_seq=0 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.636+0000 7fd4f2000640 10 client.26209 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd4f2000640 1 -- 172.21.15.18:0/96207262 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b20664010 con 0x556b20710fc0 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd4f2000640 10 client.26209 handle_command_reply: tid=0 2024-04-16T17:06:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 client.26209 shutdown 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 -- 172.21.15.18:0/96207262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f7a80 msgr2=0x556b206fdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f7a80 0x556b206fdec0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd5000d08d0 tx=0x7fd5000a0590 comp rx=0 tx=0).stop 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 -- 172.21.15.18:0/96207262 shutdown_connections 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556b20710fc0 0x556b207133b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b20701000 0x556b206fe940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f87f0 0x556b206fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 --2- 172.21.15.18:0/96207262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f7a80 0x556b206fdec0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd4f2000640 0 client.26209 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 -- 172.21.15.18:0/96207262 >> 172.21.15.18:0/96207262 conn(0x556b206925d0 msgr2=0x556b206f44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.637+0000 7fd5163e6740 1 -- 172.21.15.18:0/96207262 shutdown_connections 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.638+0000 7fd5163e6740 1 -- 172.21.15.18:0/96207262 wait complete. 2024-04-16T17:06:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.638+0000 7fd5163e6740 20 client.26209 trim_cache size 0 max 16384 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.653+0000 7fd5163e6740 1 Processor -- start 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.653+0000 7fd5163e6740 1 -- start start 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b2070cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f87f0 0x556b206fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b20701000 0x556b20718aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd5163e6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1b3ef0 con 0x556b206f7a80 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd5163e6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1b17c0 con 0x556b206f87f0 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd5163e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1af610 con 0x556b20701000 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd510c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b2070cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd511600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b20701000 0x556b20718aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd510c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b2070cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54152/0 (socket says 172.21.15.18:54152) 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd510c00640 1 -- 172.21.15.18:0/860204814 learned_addr learned my addr 172.21.15.18:0/860204814 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd50be00640 1 --2- 172.21.15.18:0/860204814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f87f0 0x556b206fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd510c00640 1 -- 172.21.15.18:0/860204814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b20701000 msgr2=0x556b20718aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd510c00640 1 --2- 172.21.15.18:0/860204814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b20701000 0x556b20718aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd510c00640 1 -- 172.21.15.18:0/860204814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f87f0 msgr2=0x556b206fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd510c00640 1 --2- 172.21.15.18:0/860204814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f87f0 0x556b206fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.654+0000 7fd510c00640 1 -- 172.21.15.18:0/860204814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b206f6d50 con 0x556b206f7a80 2024-04-16T17:06:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.655+0000 7fd510c00640 1 --2- 172.21.15.18:0/860204814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b2070cbd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd50009bf20 tx=0x7fd5000048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5148f61dd6af6f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.655+0000 7fd4fd200640 1 -- 172.21.15.18:0/860204814 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000d4040 con 0x556b206f7a80 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.655+0000 7fd4fd200640 1 -- 172.21.15.18:0/860204814 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd50007f070 con 0x556b206f7a80 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.655+0000 7fd4fd200640 1 -- 172.21.15.18:0/860204814 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500020040 con 0x556b206f7a80 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 -- 172.21.15.18:0/860204814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 msgr2=0x556b2070cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 --2- 172.21.15.18:0/860204814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b2070cbd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd50009bf20 tx=0x7fd5000048a0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 -- 172.21.15.18:0/860204814 shutdown_connections 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 --2- 172.21.15.18:0/860204814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b20701000 0x556b20718aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 --2- 172.21.15.18:0/860204814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f87f0 0x556b206fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 --2- 172.21.15.18:0/860204814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b206f7a80 0x556b2070cbd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 -- 172.21.15.18:0/860204814 >> 172.21.15.18:0/860204814 conn(0x556b206925d0 msgr2=0x556b206f1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 -- 172.21.15.18:0/860204814 shutdown_connections 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 -- 172.21.15.18:0/860204814 wait complete. 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.656+0000 7fd5163e6740 1 -- start start 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd5163e6740 1 Processor -- start 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f7a80 0x556b206fad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd5163e6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b20701000 0x556b206fb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd5163e6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1afc80 con 0x556b20701000 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd5163e6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1b10f0 con 0x556b206f7a80 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd5163e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1b0350 con 0x556b206f87f0 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd5163e6740 10 client.0 _ll_register_callbacks cb 0x7fd4f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd510c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f7a80 0x556b206fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd510c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f7a80 0x556b206fad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60128/0 (socket says 172.21.15.18:60128) 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd510c00640 1 -- 172.21.15.18:0/1220895418 learned_addr learned my addr 172.21.15.18:0/1220895418 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.657+0000 7fd50be00640 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd510c00640 1 -- 172.21.15.18:0/1220895418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 msgr2=0x556b206fb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd510c00640 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206fb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd510c00640 1 -- 172.21.15.18:0/1220895418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b20701000 msgr2=0x556b206fb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd510c00640 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b20701000 0x556b206fb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd510c00640 1 -- 172.21.15.18:0/1220895418 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b207764c0 con 0x556b206f7a80 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd510c00640 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f7a80 0x556b206fad60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd500009610 tx=0x7fd5000a0400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af72655a41028b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd4f2000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd4f2000640 1 -- 172.21.15.18:0/1220895418 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000d4050 con 0x556b206f7a80 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd4f2000640 1 -- 172.21.15.18:0/1220895418 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd5000d0160 con 0x556b206f7a80 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd5163e6740 1 -- 172.21.15.18:0/1220895418 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd50007f070 con 0x556b206f7a80 2024-04-16T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd4f2000640 1 -- 172.21.15.18:0/1220895418 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd500020040 con 0x556b206f7a80 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd5163e6740 10 client.26216 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.658+0000 7fd5163e6740 1 -- 172.21.15.18:0/1220895418 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b207777b0 con 0x556b206f7a80 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd4f2000640 1 -- 172.21.15.18:0/1220895418 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd50007f070 con 0x556b206f7a80 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd4f2000640 1 -- 172.21.15.18:0/1220895418 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd5000a16b0 con 0x556b206f7a80 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd5163e6740 10 client.26216 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd5163e6740 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd5163e6740 20 client.26216 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd5163e6740 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556b20777c60 0x556b2077a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd5163e6740 4 client.26216 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd5163e6740 1 -- 172.21.15.18:0/1220895418 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b205e0530 con 0x556b20777c60 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.659+0000 7fd50be00640 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556b20777c60 0x556b2077a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.660+0000 7fd50be00640 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556b20777c60 0x556b2077a080 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=605f38cb9598386c server_cookie=785ba4c92fd28273 in_seq=0 out_seq=0 2024-04-16T17:06:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.660+0000 7fd4f2000640 10 client.26216 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:43.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:43 smithi022 ceph-mon[36023]: pgmap v62: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-16T17:06:43.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:43 smithi043 ceph-mon[36901]: pgmap v62: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-16T17:06:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.773+0000 7fd4f2000640 1 -- 172.21.15.18:0/1220895418 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b205e0530 con 0x556b20777c60 2024-04-16T17:06:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.773+0000 7fd4f2000640 10 client.26216 handle_command_reply: tid=0 2024-04-16T17:06:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.773+0000 7fd5163e6740 1 client.26216 shutdown 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd5163e6740 1 -- 172.21.15.18:0/1220895418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f7a80 msgr2=0x556b206fad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd5163e6740 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f7a80 0x556b206fad60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd500009610 tx=0x7fd5000a0400 comp rx=0 tx=0).stop 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd5163e6740 1 -- 172.21.15.18:0/1220895418 shutdown_connections 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd5163e6740 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556b20777c60 0x556b2077a080 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd5163e6740 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556b20701000 0x556b206fb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd5163e6740 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b206f87f0 0x556b206fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd5163e6740 1 --2- 172.21.15.18:0/1220895418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b206f7a80 0x556b206fad60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd5163e6740 1 -- 172.21.15.18:0/1220895418 >> 172.21.15.18:0/1220895418 conn(0x556b206925d0 msgr2=0x556b206f3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.774+0000 7fd4f2000640 0 client.26216 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.775+0000 7fd5163e6740 1 -- 172.21.15.18:0/1220895418 shutdown_connections 2024-04-16T17:06:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.775+0000 7fd5163e6740 1 -- 172.21.15.18:0/1220895418 wait complete. 2024-04-16T17:06:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.775+0000 7fd5163e6740 20 client.26216 trim_cache size 0 max 16384 2024-04-16T17:06:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:06:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:06:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:06:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:06:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 -- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4f40a3d00 msgr2=0x7fd4f40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4f40a3d00 0x7fd4f40a61c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd504005e30 tx=0x7fd50402e040 comp rx=0 tx=0).stop 2024-04-16T17:06:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 -- 172.21.15.18:0/391898679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c102090 msgr2=0x7fd50c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c102090 0x7fd50c1a0630 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd50000d770 tx=0x7fd50000dc30 comp rx=0 tx=0).stop 2024-04-16T17:06:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 -- 172.21.15.18:0/391898679 shutdown_connections 2024-04-16T17:06:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4f40a3d00 0x7fd4f40a61c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd50c19a7a0 0x7fd50c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd50c107160 0x7fd50c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 --2- 172.21.15.18:0/391898679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd50c102090 0x7fd50c1a0630 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 -- 172.21.15.18:0/391898679 >> 172.21.15.18:0/391898679 conn(0x7fd50c0fd910 msgr2=0x7fd50c10d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 -- 172.21.15.18:0/391898679 shutdown_connections 2024-04-16T17:06:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:43.776+0000 7fd512000640 1 -- 172.21.15.18:0/391898679 wait complete. 2024-04-16T17:06:44.483 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:39 smithi018 ceph-mon[31776]: pgmap v60: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-04-16T17:06:44.484 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:39 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:06:44.484 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:39 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:44.484 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:39 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:06:44.484 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:39 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:44.484 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:39 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:06:44.484 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:41 smithi018 ceph-mon[31776]: pgmap v61: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-04-16T17:06:44.484 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:43 smithi018 ceph-mon[31776]: pgmap v62: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-16T17:06:44.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 44297 4.48 MB/sec warmup 57 sec latency 84.529 ms 2024-04-16T17:06:45.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 45772 4.54 MB/sec warmup 58 sec latency 37.095 ms 2024-04-16T17:06:45.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:45 smithi022 ceph-mon[36023]: pgmap v63: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 952 KiB/s wr, 141 op/s 2024-04-16T17:06:45.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:45 smithi043 ceph-mon[36901]: pgmap v63: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 952 KiB/s wr, 141 op/s 2024-04-16T17:06:46.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 46613 4.57 MB/sec warmup 59 sec latency 10.375 ms 2024-04-16T17:06:47.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 47570 4.58 MB/sec warmup 60 sec latency 16.220 ms 2024-04-16T17:06:47.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:47 smithi022 ceph-mon[36023]: pgmap v64: 129 pgs: 129 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-04-16T17:06:47.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:47 smithi043 ceph-mon[36901]: pgmap v64: 129 pgs: 129 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-04-16T17:06:48.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 47766 4.50 MB/sec warmup 61 sec latency 75.613 ms 2024-04-16T17:06:49.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 48347 4.44 MB/sec warmup 62 sec latency 138.004 ms 2024-04-16T17:06:49.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:49 smithi022 ceph-mon[36023]: pgmap v65: 129 pgs: 129 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 968 KiB/s wr, 138 op/s 2024-04-16T17:06:49.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:49 smithi043 ceph-mon[36901]: pgmap v65: 129 pgs: 129 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 968 KiB/s wr, 138 op/s 2024-04-16T17:06:50.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 49777 4.56 MB/sec warmup 63 sec latency 34.794 ms 2024-04-16T17:06:51.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 50288 4.51 MB/sec warmup 64 sec latency 12.681 ms 2024-04-16T17:06:51.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:51 smithi022 ceph-mon[36023]: pgmap v66: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-16T17:06:51.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:51 smithi043 ceph-mon[36901]: pgmap v66: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-16T17:06:52.293 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:45 smithi018 ceph-mon[31776]: pgmap v63: 129 pgs: 129 active+clean; 33 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 952 KiB/s wr, 141 op/s 2024-04-16T17:06:52.293 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:47 smithi018 ceph-mon[31776]: pgmap v64: 129 pgs: 129 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-04-16T17:06:52.293 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:49 smithi018 ceph-mon[31776]: pgmap v65: 129 pgs: 129 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 968 KiB/s wr, 138 op/s 2024-04-16T17:06:52.293 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:51 smithi018 ceph-mon[31776]: pgmap v66: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-16T17:06:52.516 INFO:tasks.workunit.client.0.smithi018.stdout: 1 51201 4.52 MB/sec warmup 65 sec latency 20.734 ms 2024-04-16T17:06:53.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 51536 4.45 MB/sec warmup 66 sec latency 67.605 ms 2024-04-16T17:06:53.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:53 smithi022 ceph-mon[36023]: pgmap v67: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 948 KiB/s wr, 129 op/s 2024-04-16T17:06:53.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:53 smithi043 ceph-mon[36901]: pgmap v67: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 948 KiB/s wr, 129 op/s 2024-04-16T17:06:54.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 52032 4.41 MB/sec warmup 67 sec latency 49.993 ms 2024-04-16T17:06:54.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:54 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:54.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:54 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:55.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 53329 4.50 MB/sec warmup 68 sec latency 21.645 ms 2024-04-16T17:06:55.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:55 smithi022 ceph-mon[36023]: pgmap v68: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 948 KiB/s wr, 129 op/s 2024-04-16T17:06:55.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:55 smithi043 ceph-mon[36901]: pgmap v68: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 948 KiB/s wr, 129 op/s 2024-04-16T17:06:56.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 53913 4.46 MB/sec warmup 69 sec latency 18.873 ms 2024-04-16T17:06:56.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:56 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:56.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:56 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:56.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:56 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:06:56.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:56 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:56.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:56 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:56.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:56 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:06:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 -- 172.21.15.18:0/1818815998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8c0104c60 msgr2=0x7fc8c0108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1818815998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8c0104c60 0x7fc8c0108ca0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009900 tx=0x7fc8ac02bc50 comp rx=0 tx=0).stop 2024-04-16T17:06:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 -- 172.21.15.18:0/1818815998 shutdown_connections 2024-04-16T17:06:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1818815998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c0109380 0x7fc8c0110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1818815998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8c0104c60 0x7fc8c0108ca0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1818815998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c01042b0 0x7fc8c0104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 -- 172.21.15.18:0/1818815998 >> 172.21.15.18:0/1818815998 conn(0x7fc8c00ffb10 msgr2=0x7fc8c0101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 -- 172.21.15.18:0/1818815998 shutdown_connections 2024-04-16T17:06:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.925+0000 7fc8c5600640 1 -- 172.21.15.18:0/1818815998 wait complete. 2024-04-16T17:06:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.926+0000 7fc8c5600640 1 Processor -- start 2024-04-16T17:06:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.926+0000 7fc8c5600640 1 -- start start 2024-04-16T17:06:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8c5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c01042b0 0x7fc8c01a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8c5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c0109380 0x7fc8c01a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8c5600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8c019ab10 0x7fc8c019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8c5600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01143f0 con 0x7fc8c019ab10 2024-04-16T17:06:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8c5600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0114270 con 0x7fc8c0109380 2024-04-16T17:06:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8c5600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0114570 con 0x7fc8c01042b0 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c0109380 0x7fc8c01a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c01042b0 0x7fc8c01a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c0109380 0x7fc8c01a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35562/0 (socket says 172.21.15.18:35562) 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bf400640 1 -- 172.21.15.18:0/1441090953 learned_addr learned my addr 172.21.15.18:0/1441090953 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8c019ab10 0x7fc8c019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bfe00640 1 -- 172.21.15.18:0/1441090953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c0109380 msgr2=0x7fc8c01a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bfe00640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c0109380 0x7fc8c01a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bfe00640 1 -- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8c019ab10 msgr2=0x7fc8c019af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bfe00640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8c019ab10 0x7fc8c019af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bfe00640 1 -- 172.21.15.18:0/1441090953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c019b710 con 0x7fc8c01042b0 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.927+0000 7fc8bf400640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c0109380 0x7fc8c01a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.928+0000 7fc8bfe00640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c01042b0 0x7fc8c01a0a50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e9f0 tx=0x7fc8b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.928+0000 7fc8bcc00640 1 -- 172.21.15.18:0/1441090953 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000ccc0 con 0x7fc8c01042b0 2024-04-16T17:06:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.928+0000 7fc8c5600640 1 -- 172.21.15.18:0/1441090953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c01a77f0 con 0x7fc8c01042b0 2024-04-16T17:06:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.928+0000 7fc8bcc00640 1 -- 172.21.15.18:0/1441090953 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b00035f0 con 0x7fc8c01042b0 2024-04-16T17:06:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.928+0000 7fc8bcc00640 1 -- 172.21.15.18:0/1441090953 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000ce60 con 0x7fc8c01042b0 2024-04-16T17:06:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.928+0000 7fc8c5600640 1 -- 172.21.15.18:0/1441090953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c01a7c50 con 0x7fc8c01042b0 2024-04-16T17:06:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.931+0000 7fc8c982b740 1 Processor -- start 2024-04-16T17:06:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.931+0000 7fc8bcc00640 1 -- 172.21.15.18:0/1441090953 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc8b00037a0 con 0x7fc8c01042b0 2024-04-16T17:06:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.931+0000 7fc8c982b740 1 -- start start 2024-04-16T17:06:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d38540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d38a80 0x5574f0d38f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d42000 0x5574f0d443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c982b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5574f0c9d780 con 0x5574f0d38a80 2024-04-16T17:06:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c982b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5574f0c43f40 con 0x5574f0d42000 2024-04-16T17:06:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c982b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5574f0c99740 con 0x5574f0d397f0 2024-04-16T17:06:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d38540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8bf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d42000 0x5574f0d443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d38540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36712/0 (socket says 172.21.15.18:36712) 2024-04-16T17:06:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.932+0000 7fc8c4c00640 1 -- 172.21.15.18:0/565426827 learned_addr learned my addr 172.21.15.18:0/565426827 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.933+0000 7fc8c4c00640 1 -- 172.21.15.18:0/565426827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d42000 msgr2=0x5574f0d443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.933+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d42000 0x5574f0d443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.933+0000 7fc8c4c00640 1 -- 172.21.15.18:0/565426827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d38a80 msgr2=0x5574f0d38f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:06:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.933+0000 7fc8bfe00640 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d38a80 0x5574f0d38f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d38a80 0x5574f0d38f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8c4c00640 1 -- 172.21.15.18:0/565426827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f09b0210 con 0x5574f0d397f0 2024-04-16T17:06:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8bf400640 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d42000 0x5574f0d443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8bcc00640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc8a40a3d00 0x7fc8a40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8bfe00640 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d38a80 0x5574f0d38f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8bcc00640 1 -- 172.21.15.18:0/1441090953 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc8b00d0e60 con 0x7fc8c01042b0 2024-04-16T17:06:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d38540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009f60 tx=0x7fc8b8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2c584c2c00bf487 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8bfe00640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc8a40a3d00 0x7fc8a40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8b5200640 1 -- 172.21.15.18:0/565426827 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b801e070 con 0x5574f0d397f0 2024-04-16T17:06:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8b5200640 1 -- 172.21.15.18:0/565426827 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b8004030 con 0x5574f0d397f0 2024-04-16T17:06:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.934+0000 7fc8b5200640 1 -- 172.21.15.18:0/565426827 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800f720 con 0x5574f0d397f0 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.935+0000 7fc8c982b740 1 -- 172.21.15.18:0/565426827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 msgr2=0x5574f0d38540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.935+0000 7fc8bfe00640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc8a40a3d00 0x7fc8a40a61c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5574f0d39f50 tx=0x7fc8b0004580 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.935+0000 7fc8c982b740 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d38540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009f60 tx=0x7fc8b8009f90 comp rx=0 tx=0).stop 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.936+0000 7fc8c982b740 1 -- 172.21.15.18:0/565426827 shutdown_connections 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.936+0000 7fc8c982b740 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d42000 0x5574f0d443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.936+0000 7fc8c982b740 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d38a80 0x5574f0d38f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.936+0000 7fc8c982b740 1 --2- 172.21.15.18:0/565426827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d38540 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.936+0000 7fc8c982b740 1 -- 172.21.15.18:0/565426827 >> 172.21.15.18:0/565426827 conn(0x5574f0cd35d0 msgr2=0x5574f0d33770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.936+0000 7fc8c982b740 1 -- 172.21.15.18:0/565426827 shutdown_connections 2024-04-16T17:06:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.936+0000 7fc8c982b740 1 -- 172.21.15.18:0/565426827 wait complete. 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.936+0000 7fc8c982b740 1 -- start start 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8c982b740 1 Processor -- start 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d38a80 0x5574f0d3ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d397f0 0x5574f0d3f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 0x5574f0d3f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8c982b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5574f0d44d30 con 0x5574f0d42000 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8c982b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5574f0c57390 con 0x5574f0d397f0 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8c982b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5574f0d44eb0 con 0x5574f0d38a80 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8c982b740 10 client.0 _ll_register_callbacks cb 0x5574f0c40f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8bf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d397f0 0x5574f0d3f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8bf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d397f0 0x5574f0d3f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35582/0 (socket says 172.21.15.18:35582) 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.937+0000 7fc8bf400640 1 -- 172.21.15.18:0/2227896154 learned_addr learned my addr 172.21.15.18:0/2227896154 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d38a80 0x5574f0d3ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8bf400640 1 -- 172.21.15.18:0/2227896154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d38a80 msgr2=0x5574f0d3ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8bf400640 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d38a80 0x5574f0d3ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8bf400640 1 -- 172.21.15.18:0/2227896154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 msgr2=0x5574f0d3f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8bf400640 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 0x5574f0d3f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8bf400640 1 -- 172.21.15.18:0/2227896154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f0a7c620 con 0x5574f0d397f0 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8bf400640 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d397f0 0x5574f0d3f3b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac000c00 tx=0x7fc8ac02c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e54baf8002b2f856 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d38a80 0x5574f0d3ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8a9600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.938+0000 7fc8a9600640 1 -- 172.21.15.18:0/2227896154 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac004040 con 0x5574f0d397f0 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.939+0000 7fc8c982b740 1 -- 172.21.15.18:0/2227896154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b8004030 con 0x5574f0d397f0 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.939+0000 7fc8a9600640 1 -- 172.21.15.18:0/2227896154 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc8ac0041e0 con 0x5574f0d397f0 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.939+0000 7fc8a9600640 1 -- 172.21.15.18:0/2227896154 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8ac051050 con 0x5574f0d397f0 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.939+0000 7fc8c982b740 10 client.26228 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.939+0000 7fc8c982b740 1 -- 172.21.15.18:0/2227896154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5574f0d50e20 con 0x5574f0d397f0 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.939+0000 7fc8a9600640 1 -- 172.21.15.18:0/2227896154 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b8004030 con 0x5574f0d397f0 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.940+0000 7fc8a9600640 1 -- 172.21.15.18:0/2227896154 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8ac003950 con 0x5574f0d397f0 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.940+0000 7fc8c982b740 10 client.26228 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.940+0000 7fc8c982b740 10 client.26228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.940+0000 7fc8c982b740 20 client.26228 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.940+0000 7fc8c982b740 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574f0d52290 0x5574f0d54680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.940+0000 7fc8c982b740 4 client.26228 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.940+0000 7fc8c982b740 1 -- 172.21.15.18:0/2227896154 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574f0cab1a0 con 0x5574f0d52290 2024-04-16T17:06:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.940+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574f0d52290 0x5574f0d54680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.941+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574f0d52290 0x5574f0d54680 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f07b831ab96fba server_cookie=812d320e77aff217 in_seq=0 out_seq=0 2024-04-16T17:06:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.941+0000 7fc8a9600640 10 client.26228 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8a9600640 1 -- 172.21.15.18:0/2227896154 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574f0cab1a0 con 0x5574f0d52290 2024-04-16T17:06:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8a9600640 10 client.26228 handle_command_reply: tid=0 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 client.26228 shutdown 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 -- 172.21.15.18:0/2227896154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d397f0 msgr2=0x5574f0d3f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d397f0 0x5574f0d3f3b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac000c00 tx=0x7fc8ac02c800 comp rx=0 tx=0).stop 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 -- 172.21.15.18:0/2227896154 shutdown_connections 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574f0d52290 0x5574f0d54680 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 0x5574f0d3f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d397f0 0x5574f0d3f3b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 --2- 172.21.15.18:0/2227896154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d38a80 0x5574f0d3ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 -- 172.21.15.18:0/2227896154 >> 172.21.15.18:0/2227896154 conn(0x5574f0cd35d0 msgr2=0x5574f0d354c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8a9600640 0 client.26228 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 -- 172.21.15.18:0/2227896154 shutdown_connections 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 1 -- 172.21.15.18:0/2227896154 wait complete. 2024-04-16T17:06:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.942+0000 7fc8c982b740 20 client.26228 trim_cache size 0 max 16384 2024-04-16T17:06:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.960+0000 7fc8c982b740 1 Processor -- start 2024-04-16T17:06:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.960+0000 7fc8c982b740 1 -- start start 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d4dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d397f0 0x5574f0d3f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d42000 0x5574f0d59f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c982b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b4000 con 0x5574f0d397f0 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c982b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b18d0 con 0x5574f0d38a80 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c982b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01af720 con 0x5574f0d42000 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d4dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d4dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35584/0 (socket says 172.21.15.18:35584) 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 -- 172.21.15.18:0/305462900 learned_addr learned my addr 172.21.15.18:0/305462900 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8bf400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d397f0 0x5574f0d3f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/305462900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d42000 0x5574f0d59f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 -- 172.21.15.18:0/305462900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d42000 msgr2=0x5574f0d59f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/305462900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d42000 0x5574f0d59f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 -- 172.21.15.18:0/305462900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d397f0 msgr2=0x5574f0d3f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/305462900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d397f0 0x5574f0d3f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.961+0000 7fc8c4c00640 1 -- 172.21.15.18:0/305462900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f0d37d50 con 0x5574f0d38a80 2024-04-16T17:06:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.962+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/305462900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d4dc30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8011d60 tx=0x7fc8b8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ed4f46282d878d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.962+0000 7fc8abe00640 1 -- 172.21.15.18:0/305462900 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b802ae60 con 0x5574f0d38a80 2024-04-16T17:06:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.962+0000 7fc8abe00640 1 -- 172.21.15.18:0/305462900 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b8003bd0 con 0x5574f0d38a80 2024-04-16T17:06:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.962+0000 7fc8abe00640 1 -- 172.21.15.18:0/305462900 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b801e040 con 0x5574f0d38a80 2024-04-16T17:06:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 -- 172.21.15.18:0/305462900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 msgr2=0x5574f0d4dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 --2- 172.21.15.18:0/305462900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d4dc30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8011d60 tx=0x7fc8b8000f40 comp rx=0 tx=0).stop 2024-04-16T17:06:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 -- 172.21.15.18:0/305462900 shutdown_connections 2024-04-16T17:06:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 --2- 172.21.15.18:0/305462900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d42000 0x5574f0d59f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 --2- 172.21.15.18:0/305462900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d397f0 0x5574f0d3f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 --2- 172.21.15.18:0/305462900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d4dc30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 -- 172.21.15.18:0/305462900 >> 172.21.15.18:0/305462900 conn(0x5574f0cd35d0 msgr2=0x5574f0d31f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 -- 172.21.15.18:0/305462900 shutdown_connections 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 -- 172.21.15.18:0/305462900 wait complete. 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 -- start start 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.963+0000 7fc8c982b740 1 Processor -- start 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d45ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d46000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c982b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 0x5574f0d46540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c982b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01afd90 con 0x5574f0d42000 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c982b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b1200 con 0x5574f0d38a80 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c982b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b0460 con 0x5574f0d397f0 2024-04-16T17:06:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c982b740 10 client.0 _ll_register_callbacks cb 0x5574f0d41f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d45ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d45ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35586/0 (socket says 172.21.15.18:35586) 2024-04-16T17:06:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c4c00640 1 -- 172.21.15.18:0/361434991 learned_addr learned my addr 172.21.15.18:0/361434991 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:06:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8bf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d46000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.964+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 0x5574f0d46540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.965+0000 7fc8bf400640 1 -- 172.21.15.18:0/361434991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 msgr2=0x5574f0d45ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.965+0000 7fc8bf400640 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d45ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.965+0000 7fc8bf400640 1 -- 172.21.15.18:0/361434991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 msgr2=0x5574f0d46540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.965+0000 7fc8bf400640 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 0x5574f0d46540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.965+0000 7fc8bf400640 1 -- 172.21.15.18:0/361434991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f0db7790 con 0x5574f0d397f0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.965+0000 7fc8bf400640 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d46000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac005c40 tx=0x7fc8ac0095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8836eb317a66332 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.965+0000 7fc8a9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.965+0000 7fc8a9600640 1 -- 172.21.15.18:0/361434991 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac005db0 con 0x5574f0d397f0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 1 -- 172.21.15.18:0/361434991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b8003bd0 con 0x5574f0d397f0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8a9600640 1 -- 172.21.15.18:0/361434991 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc8ac04e040 con 0x5574f0d397f0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8a9600640 1 -- 172.21.15.18:0/361434991 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8ac062050 con 0x5574f0d397f0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 10 client.26230 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 1 -- 172.21.15.18:0/361434991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5574f0db89e0 con 0x5574f0d397f0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8a9600640 1 -- 172.21.15.18:0/361434991 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b8003bd0 con 0x5574f0d397f0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8a9600640 1 -- 172.21.15.18:0/361434991 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8ac004f20 con 0x5574f0d397f0 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 10 client.26230 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 20 client.26230 populate_metadata read hostname 'smithi018' 2024-04-16T17:06:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574f0db91b0 0x5574f0dbb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 4 client.26230 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:06:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.966+0000 7fc8c982b740 1 -- 172.21.15.18:0/361434991 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574f0ca5010 con 0x5574f0db91b0 2024-04-16T17:06:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.967+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574f0db91b0 0x5574f0dbb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.967+0000 7fc8c4c00640 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574f0db91b0 0x5574f0dbb5d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a5558fb295e00e server_cookie=9910f56bec7f9dc5 in_seq=0 out_seq=0 2024-04-16T17:06:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:56.967+0000 7fc8a9600640 10 client.26230 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.102+0000 7fc8a9600640 1 -- 172.21.15.18:0/361434991 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574f0ca5010 con 0x5574f0db91b0 2024-04-16T17:06:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.102+0000 7fc8a9600640 10 client.26230 handle_command_reply: tid=0 2024-04-16T17:06:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.102+0000 7fc8c982b740 1 client.26230 shutdown 2024-04-16T17:06:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 -- 172.21.15.18:0/361434991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 msgr2=0x5574f0d46000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d46000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac005c40 tx=0x7fc8ac0095d0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 -- 172.21.15.18:0/361434991 shutdown_connections 2024-04-16T17:06:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574f0db91b0 0x5574f0dbb5d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574f0d42000 0x5574f0d46540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574f0d397f0 0x5574f0d46000 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8a9600640 0 client.26230 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 --2- 172.21.15.18:0/361434991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574f0d38a80 0x5574f0d45ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 -- 172.21.15.18:0/361434991 >> 172.21.15.18:0/361434991 conn(0x5574f0cd35d0 msgr2=0x5574f0d32810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 -- 172.21.15.18:0/361434991 shutdown_connections 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 1 -- 172.21.15.18:0/361434991 wait complete. 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.104+0000 7fc8c982b740 20 client.26230 trim_cache size 0 max 16384 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:06:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 -- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc8a40a3d00 msgr2=0x7fc8a40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc8a40a3d00 0x7fc8a40a61c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5574f0d39f50 tx=0x7fc8b0004580 comp rx=0 tx=0).stop 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 -- 172.21.15.18:0/1441090953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c01042b0 msgr2=0x7fc8c01a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c01042b0 0x7fc8c01a0a50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e9f0 tx=0x7fc8b000edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 -- 172.21.15.18:0/1441090953 shutdown_connections 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc8a40a3d00 0x7fc8a40a61c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8c019ab10 0x7fc8c019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c0109380 0x7fc8c01a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 --2- 172.21.15.18:0/1441090953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c01042b0 0x7fc8c01a0a50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.105+0000 7fc8c5600640 1 -- 172.21.15.18:0/1441090953 >> 172.21.15.18:0/1441090953 conn(0x7fc8c00ffb10 msgr2=0x7fc8c0100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.106+0000 7fc8c5600640 1 -- 172.21.15.18:0/1441090953 shutdown_connections 2024-04-16T17:06:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:06:57.106+0000 7fc8c5600640 1 -- 172.21.15.18:0/1441090953 wait complete. 2024-04-16T17:06:57.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 54897 4.47 MB/sec warmup 70 sec latency 103.978 ms 2024-04-16T17:06:57.728 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:53 smithi018 ceph-mon[31776]: pgmap v67: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 948 KiB/s wr, 129 op/s 2024-04-16T17:06:57.728 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:54 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:57.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:55 smithi018 ceph-mon[31776]: pgmap v68: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 948 KiB/s wr, 129 op/s 2024-04-16T17:06:57.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:56 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:57.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:56 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:57.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:56 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:06:57.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:57 smithi043 ceph-mon[36901]: pgmap v69: 129 pgs: 129 active+clean; 34 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-16T17:06:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:57 smithi022 ceph-mon[36023]: pgmap v69: 129 pgs: 129 active+clean; 34 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-16T17:06:58.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 55239 4.41 MB/sec warmup 71 sec latency 67.493 ms 2024-04-16T17:06:59.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 55666 4.37 MB/sec warmup 72 sec latency 66.980 ms 2024-04-16T17:06:59.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:06:59 smithi043 ceph-mon[36901]: pgmap v70: 129 pgs: 129 active+clean; 34 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 919 KiB/s wr, 76 op/s 2024-04-16T17:06:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:06:59 smithi022 ceph-mon[36023]: pgmap v70: 129 pgs: 129 active+clean; 34 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 919 KiB/s wr, 76 op/s 2024-04-16T17:07:00.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 56889 4.46 MB/sec warmup 73 sec latency 22.416 ms 2024-04-16T17:07:00.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:57 smithi018 ceph-mon[31776]: pgmap v69: 129 pgs: 129 active+clean; 34 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-16T17:07:00.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:06:59 smithi018 ceph-mon[31776]: pgmap v70: 129 pgs: 129 active+clean; 34 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 919 KiB/s wr, 76 op/s 2024-04-16T17:07:01.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 57052 4.41 MB/sec warmup 74 sec latency 60.044 ms 2024-04-16T17:07:01.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:01 smithi043 ceph-mon[36901]: pgmap v71: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-04-16T17:07:01.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:01 smithi022 ceph-mon[36023]: pgmap v71: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-04-16T17:07:02.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 57603 4.37 MB/sec warmup 75 sec latency 18.046 ms 2024-04-16T17:07:03.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 58001 4.33 MB/sec warmup 76 sec latency 70.161 ms 2024-04-16T17:07:03.725 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:03 smithi022 ceph-mon[36023]: pgmap v72: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 928 KiB/s wr, 135 op/s 2024-04-16T17:07:03.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:03 smithi043 ceph-mon[36901]: pgmap v72: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 928 KiB/s wr, 135 op/s 2024-04-16T17:07:04.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 58286 4.31 MB/sec warmup 77 sec latency 104.477 ms 2024-04-16T17:07:05.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 59355 4.29 MB/sec warmup 78 sec latency 39.109 ms 2024-04-16T17:07:05.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:05 smithi043 ceph-mon[36901]: pgmap v73: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 928 KiB/s wr, 135 op/s 2024-04-16T17:07:05.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:05 smithi022 ceph-mon[36023]: pgmap v73: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 928 KiB/s wr, 135 op/s 2024-04-16T17:07:06.517 INFO:tasks.workunit.client.0.smithi018.stdout: 1 60547 4.38 MB/sec warmup 79 sec latency 17.941 ms 2024-04-16T17:07:07.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 61136 4.33 MB/sec warmup 80 sec latency 24.354 ms 2024-04-16T17:07:07.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:07 smithi043 ceph-mon[36901]: pgmap v74: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-04-16T17:07:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:07 smithi022 ceph-mon[36023]: pgmap v74: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-04-16T17:07:08.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:01 smithi018 ceph-mon[31776]: pgmap v71: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-04-16T17:07:08.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:03 smithi018 ceph-mon[31776]: pgmap v72: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 928 KiB/s wr, 135 op/s 2024-04-16T17:07:08.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:05 smithi018 ceph-mon[31776]: pgmap v73: 129 pgs: 129 active+clean; 31 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 928 KiB/s wr, 135 op/s 2024-04-16T17:07:08.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 61498 4.30 MB/sec warmup 81 sec latency 84.749 ms 2024-04-16T17:07:08.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:07:08.621 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:07:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 -- 172.21.15.18:0/42478049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde4810c400 msgr2=0x7fde4810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 --2- 172.21.15.18:0/42478049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde4810c400 0x7fde4810e870 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fde40009ca0 tx=0x7fde4002c1b0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 -- 172.21.15.18:0/42478049 shutdown_connections 2024-04-16T17:07:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 --2- 172.21.15.18:0/42478049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde4810c400 0x7fde4810e870 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 --2- 172.21.15.18:0/42478049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde48103900 0x7fde4810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 --2- 172.21.15.18:0/42478049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde48102fe0 0x7fde481033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 -- 172.21.15.18:0/42478049 >> 172.21.15.18:0/42478049 conn(0x7fde48077ab0 msgr2=0x7fde480fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:08.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 -- 172.21.15.18:0/42478049 shutdown_connections 2024-04-16T17:07:08.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.818+0000 7fde4ee00640 1 -- 172.21.15.18:0/42478049 wait complete. 2024-04-16T17:07:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.819+0000 7fde4ee00640 1 Processor -- start 2024-04-16T17:07:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.819+0000 7fde4ee00640 1 -- start start 2024-04-16T17:07:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.819+0000 7fde4ee00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde48102fe0 0x7fde481a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.819+0000 7fde4ee00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde48103900 0x7fde481a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde4810c400 0x7fde4819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4ee00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fde48112230 con 0x7fde48103900 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4ee00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fde481120b0 con 0x7fde48102fe0 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4ee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde481123b0 con 0x7fde4810c400 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde48103900 0x7fde481a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde48103900 0x7fde481a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33868/0 (socket says 172.21.15.18:33868) 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4d000640 1 -- 172.21.15.18:0/2411269369 learned_addr learned my addr 172.21.15.18:0/2411269369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4e400640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde4810c400 0x7fde4819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4d000640 1 -- 172.21.15.18:0/2411269369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde4810c400 msgr2=0x7fde4819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4d000640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde4810c400 0x7fde4819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4d000640 1 -- 172.21.15.18:0/2411269369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde48102fe0 msgr2=0x7fde481a0630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:07:08.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4d000640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde48102fe0 0x7fde481a0630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4d000640 1 -- 172.21.15.18:0/2411269369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4819af70 con 0x7fde48103900 2024-04-16T17:07:08.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.820+0000 7fde4e400640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde4810c400 0x7fde4819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.821+0000 7fde4d000640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde48103900 0x7fde481a0b70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fde3400e9f0 tx=0x7fde3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.821+0000 7fde46000640 1 -- 172.21.15.18:0/2411269369 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3400cd80 con 0x7fde48103900 2024-04-16T17:07:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.821+0000 7fde4ee00640 1 -- 172.21.15.18:0/2411269369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4819b260 con 0x7fde48103900 2024-04-16T17:07:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.821+0000 7fde46000640 1 -- 172.21.15.18:0/2411269369 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde34010910 con 0x7fde48103900 2024-04-16T17:07:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.821+0000 7fde4ee00640 1 -- 172.21.15.18:0/2411269369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde481a7480 con 0x7fde48103900 2024-04-16T17:07:08.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.822+0000 7fde46000640 1 -- 172.21.15.18:0/2411269369 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34018990 con 0x7fde48103900 2024-04-16T17:07:08.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.826+0000 7fde52fea740 1 Processor -- start 2024-04-16T17:07:08.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.826+0000 7fde52fea740 1 -- start start 2024-04-16T17:07:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde52fea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690390 0x56304c690770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde46000640 1 -- 172.21.15.18:0/2411269369 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde34018b30 con 0x7fde48103900 2024-04-16T17:07:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde52fea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690cb0 0x56304c699350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde52fea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c699890 0x56304c69bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde52fea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56304c5f6120 con 0x56304c699890 2024-04-16T17:07:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde52fea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56304c59c340 con 0x56304c690cb0 2024-04-16T17:07:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde52fea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56304c5f20e0 con 0x56304c690390 2024-04-16T17:07:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde4d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c699890 0x56304c69bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde4d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c699890 0x56304c69bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33876/0 (socket says 172.21.15.18:33876) 2024-04-16T17:07:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.827+0000 7fde4d000640 1 -- 172.21.15.18:0/2950558344 learned_addr learned my addr 172.21.15.18:0/2950558344 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:08.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.828+0000 7fde4da00640 1 --2- 172.21.15.18:0/2950558344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690cb0 0x56304c699350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.828+0000 7fde4d000640 1 -- 172.21.15.18:0/2950558344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690390 msgr2=0x56304c690770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.828+0000 7fde4d000640 1 --2- 172.21.15.18:0/2950558344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690390 0x56304c690770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.828+0000 7fde4d000640 1 -- 172.21.15.18:0/2950558344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690cb0 msgr2=0x56304c699350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.828+0000 7fde4d000640 1 --2- 172.21.15.18:0/2950558344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690cb0 0x56304c699350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.828+0000 7fde4d000640 1 -- 172.21.15.18:0/2950558344 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304c3d27d0 con 0x56304c699890 2024-04-16T17:07:08.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.828+0000 7fde4d000640 1 --2- 172.21.15.18:0/2950558344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c699890 0x56304c69bc80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fde340ce0e0 tx=0x7fde340d9570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7c61082d9c7750 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.829+0000 7fde46000640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fde300a3da0 0x7fde300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.829+0000 7fde3aa00640 1 -- 172.21.15.18:0/2950558344 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde340e4070 con 0x56304c699890 2024-04-16T17:07:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.829+0000 7fde3aa00640 1 -- 172.21.15.18:0/2950558344 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde3409d0c0 con 0x56304c699890 2024-04-16T17:07:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.829+0000 7fde3aa00640 1 -- 172.21.15.18:0/2950558344 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3409c9c0 con 0x56304c699890 2024-04-16T17:07:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde46000640 1 -- 172.21.15.18:0/2411269369 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fde340d0dd0 con 0x7fde48103900 2024-04-16T17:07:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde52fea740 1 -- 172.21.15.18:0/2950558344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c699890 msgr2=0x56304c69bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:08.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde52fea740 1 --2- 172.21.15.18:0/2950558344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c699890 0x56304c69bc80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fde340ce0e0 tx=0x7fde340d9570 comp rx=0 tx=0).stop 2024-04-16T17:07:08.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde4e400640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fde300a3da0 0x7fde300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde52fea740 1 -- 172.21.15.18:0/2950558344 shutdown_connections 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde52fea740 1 --2- 172.21.15.18:0/2950558344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c699890 0x56304c69bc80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde52fea740 1 --2- 172.21.15.18:0/2950558344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690cb0 0x56304c699350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde52fea740 1 --2- 172.21.15.18:0/2950558344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690390 0x56304c690770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.830+0000 7fde52fea740 1 -- 172.21.15.18:0/2950558344 >> 172.21.15.18:0/2950558344 conn(0x56304c68aac0 msgr2=0x56304c68aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.831+0000 7fde52fea740 1 -- 172.21.15.18:0/2950558344 shutdown_connections 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.831+0000 7fde52fea740 1 -- 172.21.15.18:0/2950558344 wait complete. 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.831+0000 7fde4e400640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fde300a3da0 0x7fde300a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fde40004af0 tx=0x7fde40005c50 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.831+0000 7fde52fea740 1 -- start start 2024-04-16T17:07:08.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde52fea740 1 Processor -- start 2024-04-16T17:07:08.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde52fea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c696950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde52fea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c696e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde52fea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c6973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde52fea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56304c69c680 con 0x56304c690390 2024-04-16T17:07:08.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde52fea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56304c5af710 con 0x56304c699890 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde52fea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56304c69c800 con 0x56304c690cb0 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde52fea740 10 client.0 _ll_register_callbacks cb 0x56304c5aaaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde4d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c696e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde4d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c696e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55026/0 (socket says 172.21.15.18:55026) 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.832+0000 7fde4d000640 1 -- 172.21.15.18:0/1483418119 learned_addr learned my addr 172.21.15.18:0/1483418119 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4e400640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c6973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4da00640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c696950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4d000640 1 -- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 msgr2=0x56304c6973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4d000640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c6973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4d000640 1 -- 172.21.15.18:0/1483418119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 msgr2=0x56304c696950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4d000640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c696950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4d000640 1 -- 172.21.15.18:0/1483418119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304c3083f0 con 0x56304c690cb0 2024-04-16T17:07:08.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4e400640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c6973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4da00640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c696950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde4d000640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c696e90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fde34018500 tx=0x7fde34097420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a59862acd8ae13d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.833+0000 7fde2f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.834+0000 7fde2f400640 1 -- 172.21.15.18:0/1483418119 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3409c790 con 0x56304c690cb0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.834+0000 7fde52fea740 1 -- 172.21.15.18:0/1483418119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde3409d0c0 con 0x56304c690cb0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.834+0000 7fde2f400640 1 -- 172.21.15.18:0/1483418119 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fde340ce3f0 con 0x56304c690cb0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.834+0000 7fde2f400640 1 -- 172.21.15.18:0/1483418119 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde3409c320 con 0x56304c690cb0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.834+0000 7fde52fea740 10 client.26239 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.834+0000 7fde52fea740 1 -- 172.21.15.18:0/1483418119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56304c2c80f0 con 0x56304c690cb0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.834+0000 7fde2f400640 1 -- 172.21.15.18:0/1483418119 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde3409d0c0 con 0x56304c690cb0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.835+0000 7fde2f400640 1 -- 172.21.15.18:0/1483418119 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fde34099030 con 0x56304c690cb0 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.835+0000 7fde52fea740 10 client.26239 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.835+0000 7fde52fea740 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:08.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.835+0000 7fde52fea740 20 client.26239 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.835+0000 7fde52fea740 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56304c6a9bc0 0x56304c6abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.835+0000 7fde52fea740 4 client.26239 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.835+0000 7fde52fea740 1 -- 172.21.15.18:0/1483418119 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56304c603b40 con 0x56304c6a9bc0 2024-04-16T17:07:08.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.835+0000 7fde4da00640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56304c6a9bc0 0x56304c6abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.836+0000 7fde2f400640 10 client.26239 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.836+0000 7fde4da00640 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56304c6a9bc0 0x56304c6abfb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f48722cbd5e23fee server_cookie=daa3d282881711fb in_seq=0 out_seq=0 2024-04-16T17:07:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.837+0000 7fde2f400640 1 -- 172.21.15.18:0/1483418119 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56304c603b40 con 0x56304c6a9bc0 2024-04-16T17:07:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.837+0000 7fde2f400640 10 client.26239 handle_command_reply: tid=0 2024-04-16T17:07:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.837+0000 7fde52fea740 1 client.26239 shutdown 2024-04-16T17:07:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.837+0000 7fde52fea740 1 -- 172.21.15.18:0/1483418119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 msgr2=0x56304c696e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.837+0000 7fde52fea740 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c696e90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fde34018500 tx=0x7fde34097420 comp rx=0 tx=0).stop 2024-04-16T17:07:08.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 1 -- 172.21.15.18:0/1483418119 shutdown_connections 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56304c6a9bc0 0x56304c6abfb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c6973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c696e90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde2f400640 0 client.26239 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 1 --2- 172.21.15.18:0/1483418119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c696950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 1 -- 172.21.15.18:0/1483418119 >> 172.21.15.18:0/1483418119 conn(0x56304c68aac0 msgr2=0x56304c68c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 1 -- 172.21.15.18:0/1483418119 shutdown_connections 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 1 -- 172.21.15.18:0/1483418119 wait complete. 2024-04-16T17:07:08.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.838+0000 7fde52fea740 20 client.26239 trim_cache size 0 max 16384 2024-04-16T17:07:08.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.859+0000 7fde52fea740 1 Processor -- start 2024-04-16T17:07:08.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.859+0000 7fde52fea740 1 -- start start 2024-04-16T17:07:08.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde52fea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690390 0x56304c699080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde52fea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690cb0 0x56304c693bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde52fea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c699890 0x56304c6940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde52fea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fde28012820 con 0x56304c690cb0 2024-04-16T17:07:08.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde52fea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fde280100f0 con 0x56304c690390 2024-04-16T17:07:08.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde52fea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde2800df40 con 0x56304c699890 2024-04-16T17:07:08.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde4d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690cb0 0x56304c693bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde4e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c699890 0x56304c6940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde4e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c699890 0x56304c6940f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55036/0 (socket says 172.21.15.18:55036) 2024-04-16T17:07:08.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde4e400640 1 -- 172.21.15.18:0/3103729339 learned_addr learned my addr 172.21.15.18:0/3103729339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:08.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.860+0000 7fde4da00640 1 --2- 172.21.15.18:0/3103729339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690390 0x56304c699080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde4e400640 1 -- 172.21.15.18:0/3103729339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690390 msgr2=0x56304c699080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde4e400640 1 --2- 172.21.15.18:0/3103729339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690390 0x56304c699080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde4e400640 1 -- 172.21.15.18:0/3103729339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690cb0 msgr2=0x56304c693bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde4e400640 1 --2- 172.21.15.18:0/3103729339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690cb0 0x56304c693bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde4e400640 1 -- 172.21.15.18:0/3103729339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304c69c6d0 con 0x56304c699890 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde4e400640 1 --2- 172.21.15.18:0/3103729339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c699890 0x56304c6940f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fde40009ca0 tx=0x7fde4003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2ca2f45a4dad96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde3a000640 1 -- 172.21.15.18:0/3103729339 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40039040 con 0x56304c699890 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde3a000640 1 -- 172.21.15.18:0/3103729339 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde40051070 con 0x56304c699890 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.861+0000 7fde3a000640 1 -- 172.21.15.18:0/3103729339 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde400328a0 con 0x56304c699890 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 -- 172.21.15.18:0/3103729339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c699890 msgr2=0x56304c6940f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 --2- 172.21.15.18:0/3103729339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c699890 0x56304c6940f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fde40009ca0 tx=0x7fde4003e000 comp rx=0 tx=0).stop 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 -- 172.21.15.18:0/3103729339 shutdown_connections 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 --2- 172.21.15.18:0/3103729339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c699890 0x56304c6940f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 --2- 172.21.15.18:0/3103729339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690cb0 0x56304c693bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 --2- 172.21.15.18:0/3103729339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c690390 0x56304c699080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 -- 172.21.15.18:0/3103729339 >> 172.21.15.18:0/3103729339 conn(0x56304c68aac0 msgr2=0x56304c691090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:08.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 -- 172.21.15.18:0/3103729339 shutdown_connections 2024-04-16T17:07:08.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.862+0000 7fde52fea740 1 -- 172.21.15.18:0/3103729339 wait complete. 2024-04-16T17:07:08.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.863+0000 7fde52fea740 1 -- start start 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.863+0000 7fde52fea740 1 Processor -- start 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.863+0000 7fde52fea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c6ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.863+0000 7fde52fea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c6ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde52fea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c70eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde52fea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fde2800e5b0 con 0x56304c690390 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde52fea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fde2800fa20 con 0x56304c699890 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde52fea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde2800ec80 con 0x56304c690cb0 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde52fea740 10 client.0 _ll_register_callbacks cb 0x7fde340e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c6ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c70eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c70eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57772/0 (socket says 172.21.15.18:57772) 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4e400640 1 -- 172.21.15.18:0/4106064439 learned_addr learned my addr 172.21.15.18:0/4106064439 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:08.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4d000640 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c6ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4da00640 1 -- 172.21.15.18:0/4106064439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 msgr2=0x56304c6ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4da00640 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c6ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4da00640 1 -- 172.21.15.18:0/4106064439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 msgr2=0x56304c70eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4da00640 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c70eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4da00640 1 -- 172.21.15.18:0/4106064439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304c70f2e0 con 0x56304c690390 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.864+0000 7fde4e400640 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c70eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.865+0000 7fde4da00640 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c6ace20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x56304c59c490 tx=0x7fde3c00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ba718f7201a741 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.865+0000 7fde2f400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.865+0000 7fde2f400640 1 -- 172.21.15.18:0/4106064439 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c027060 con 0x56304c690390 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.865+0000 7fde52fea740 1 -- 172.21.15.18:0/4106064439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde40051070 con 0x56304c690390 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.865+0000 7fde2f400640 1 -- 172.21.15.18:0/4106064439 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fde3c019800 con 0x56304c690390 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.865+0000 7fde2f400640 1 -- 172.21.15.18:0/4106064439 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde3c02e050 con 0x56304c690390 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.865+0000 7fde52fea740 10 client.16071 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:08.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.865+0000 7fde52fea740 1 -- 172.21.15.18:0/4106064439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56304c7104f0 con 0x56304c690390 2024-04-16T17:07:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde2f400640 1 -- 172.21.15.18:0/4106064439 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde40051070 con 0x56304c690390 2024-04-16T17:07:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde2f400640 1 -- 172.21.15.18:0/4106064439 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fde3c014540 con 0x56304c690390 2024-04-16T17:07:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde52fea740 10 client.16071 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde52fea740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde52fea740 20 client.16071 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde52fea740 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56304c710bc0 0x56304c712fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde52fea740 4 client.16071 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:07:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde52fea740 1 -- 172.21.15.18:0/4106064439 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56304c5fd9b0 con 0x56304c710bc0 2024-04-16T17:07:08.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.866+0000 7fde4d000640 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56304c710bc0 0x56304c712fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:08.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde4d000640 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56304c710bc0 0x56304c712fe0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b58f3ae4c6eef3 server_cookie=1a619a3792215741 in_seq=0 out_seq=0 2024-04-16T17:07:08.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde2f400640 10 client.16071 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde2f400640 1 -- 172.21.15.18:0/4106064439 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56304c5fd9b0 con 0x56304c710bc0 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde2f400640 10 client.16071 handle_command_reply: tid=0 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 client.16071 shutdown 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 -- 172.21.15.18:0/4106064439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 msgr2=0x56304c6ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c6ace20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x56304c59c490 tx=0x7fde3c00bbe0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 -- 172.21.15.18:0/4106064439 shutdown_connections 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56304c710bc0 0x56304c712fe0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56304c699890 0x56304c70eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56304c690cb0 0x56304c6ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 --2- 172.21.15.18:0/4106064439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56304c690390 0x56304c6ace20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 -- 172.21.15.18:0/4106064439 >> 172.21.15.18:0/4106064439 conn(0x56304c68aac0 msgr2=0x56304c6968f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde2f400640 0 client.16071 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 -- 172.21.15.18:0/4106064439 shutdown_connections 2024-04-16T17:07:08.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.867+0000 7fde52fea740 1 -- 172.21.15.18:0/4106064439 wait complete. 2024-04-16T17:07:08.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde52fea740 20 client.16071 trim_cache size 0 max 16384 2024-04-16T17:07:08.873 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:07:08.873 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:07:08.873 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:07:08.873 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 -- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fde300a3da0 msgr2=0x7fde300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fde300a3da0 0x7fde300a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fde40004af0 tx=0x7fde40005c50 comp rx=0 tx=0).stop 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 -- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde48103900 msgr2=0x7fde481a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde48103900 0x7fde481a0b70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fde3400e9f0 tx=0x7fde3400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 -- 172.21.15.18:0/2411269369 shutdown_connections 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fde300a3da0 0x7fde300a6260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde4810c400 0x7fde4819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde48103900 0x7fde481a0b70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 --2- 172.21.15.18:0/2411269369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde48102fe0 0x7fde481a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.868+0000 7fde4ee00640 1 -- 172.21.15.18:0/2411269369 >> 172.21.15.18:0/2411269369 conn(0x7fde48077ab0 msgr2=0x7fde480fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:08.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.869+0000 7fde4ee00640 1 -- 172.21.15.18:0/2411269369 shutdown_connections 2024-04-16T17:07:08.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:08.869+0000 7fde4ee00640 1 -- 172.21.15.18:0/2411269369 wait complete. 2024-04-16T17:07:08.886 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:07:08.886 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.077+0000 7fa287e00640 1 -- 172.21.15.18:0/136463425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288107220 msgr2=0x7fa28810edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.077+0000 7fa287e00640 1 --2- 172.21.15.18:0/136463425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288107220 0x7fa28810edf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa280009be0 tx=0x7fa28002bef0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.077+0000 7fa287e00640 1 -- 172.21.15.18:0/136463425 shutdown_connections 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.077+0000 7fa287e00640 1 --2- 172.21.15.18:0/136463425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288107220 0x7fa28810edf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.077+0000 7fa287e00640 1 --2- 172.21.15.18:0/136463425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa288102b00 0x7fa288106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.077+0000 7fa287e00640 1 --2- 172.21.15.18:0/136463425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa288102150 0x7fa288102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.077+0000 7fa287e00640 1 -- 172.21.15.18:0/136463425 >> 172.21.15.18:0/136463425 conn(0x7fa2880fd9f0 msgr2=0x7fa2880ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.078+0000 7fa287e00640 1 -- 172.21.15.18:0/136463425 shutdown_connections 2024-04-16T17:07:09.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.078+0000 7fa287e00640 1 -- 172.21.15.18:0/136463425 wait complete. 2024-04-16T17:07:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.079+0000 7fa287e00640 1 Processor -- start 2024-04-16T17:07:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.079+0000 7fa287e00640 1 -- start start 2024-04-16T17:07:09.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288102150 0x7fa28819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa288107220 0x7fa28819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa288198600 0x7fa288198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881122e0 con 0x7fa288198600 2024-04-16T17:07:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa288112160 con 0x7fa288107220 2024-04-16T17:07:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa288112460 con 0x7fa288102150 2024-04-16T17:07:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa288198600 0x7fa288198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa286a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288102150 0x7fa28819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa288198600 0x7fa288198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33928/0 (socket says 172.21.15.18:33928) 2024-04-16T17:07:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa287400640 1 -- 172.21.15.18:0/842014523 learned_addr learned my addr 172.21.15.18:0/842014523 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa286a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288102150 0x7fa28819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55056/0 (socket says 172.21.15.18:55056) 2024-04-16T17:07:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.080+0000 7fa286000640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa288107220 0x7fa28819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.081+0000 7fa287400640 1 -- 172.21.15.18:0/842014523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288102150 msgr2=0x7fa28819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.081+0000 7fa287400640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288102150 0x7fa28819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.081+0000 7fa287400640 1 -- 172.21.15.18:0/842014523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa288107220 msgr2=0x7fa28819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.081+0000 7fa287400640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa288107220 0x7fa28819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.081+0000 7fa287400640 1 -- 172.21.15.18:0/842014523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa288199120 con 0x7fa288198600 2024-04-16T17:07:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.081+0000 7fa287400640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa288198600 0x7fa288198ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa2800096f0 tx=0x7fa2800336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.082+0000 7fa27f400640 1 -- 172.21.15.18:0/842014523 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280043020 con 0x7fa288198600 2024-04-16T17:07:09.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.082+0000 7fa27f400640 1 -- 172.21.15.18:0/842014523 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2800338a0 con 0x7fa288198600 2024-04-16T17:07:09.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.082+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2881993b0 con 0x7fa288198600 2024-04-16T17:07:09.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.082+0000 7fa27f400640 1 -- 172.21.15.18:0/842014523 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28003e6b0 con 0x7fa288198600 2024-04-16T17:07:09.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.082+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2881a55b0 con 0x7fa288198600 2024-04-16T17:07:09.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.083+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28804fe00 con 0x7fa288198600 2024-04-16T17:07:09.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.087+0000 7fa27f400640 1 -- 172.21.15.18:0/842014523 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa280046020 con 0x7fa288198600 2024-04-16T17:07:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.089+0000 7fa27f400640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa26c0a3d10 0x7fa26c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.089+0000 7fa27f400640 1 -- 172.21.15.18:0/842014523 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa2800f2fe0 con 0x7fa288198600 2024-04-16T17:07:09.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.089+0000 7fa286a00640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa26c0a3d10 0x7fa26c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.090+0000 7fa286a00640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa26c0a3d10 0x7fa26c0a61d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa278009bd0 tx=0x7fa278005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.095+0000 7fa27f400640 1 -- 172.21.15.18:0/842014523 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa28004d020 con 0x7fa288198600 2024-04-16T17:07:09.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.349+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa28810a4e0 con 0x7fa288198600 2024-04-16T17:07:09.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.350+0000 7fa27f400640 1 -- 172.21.15.18:0/842014523 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7fa2800bf3e0 con 0x7fa288198600 2024-04-16T17:07:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:07:09.355 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:07:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:07:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa26c0a3d10 msgr2=0x7fa26c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa26c0a3d10 0x7fa26c0a61d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa278009bd0 tx=0x7fa278005f90 comp rx=0 tx=0).stop 2024-04-16T17:07:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa288198600 msgr2=0x7fa288198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa288198600 0x7fa288198ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa2800096f0 tx=0x7fa2800336b0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 shutdown_connections 2024-04-16T17:07:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa26c0a3d10 0x7fa26c0a61d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa288198600 0x7fa288198ab0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa288107220 0x7fa28819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 --2- 172.21.15.18:0/842014523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa288102150 0x7fa28819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.352+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 >> 172.21.15.18:0/842014523 conn(0x7fa2880fd9f0 msgr2=0x7fa2880ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.353+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 shutdown_connections 2024-04-16T17:07:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.353+0000 7fa287e00640 1 -- 172.21.15.18:0/842014523 wait complete. 2024-04-16T17:07:09.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:07:09.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 61822 4.28 MB/sec warmup 82 sec latency 63.249 ms 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.588+0000 7f1a5a000640 1 -- 172.21.15.18:0/1671149642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54104450 msgr2=0x7f1a54104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.588+0000 7f1a5a000640 1 --2- 172.21.15.18:0/1671149642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54104450 0x7f1a54104830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4402bc40 comp rx=0 tx=0).stop 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.589+0000 7f1a5a000640 1 -- 172.21.15.18:0/1671149642 shutdown_connections 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.589+0000 7f1a5a000640 1 --2- 172.21.15.18:0/1671149642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a54109520 0x7f1a541110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.589+0000 7f1a5a000640 1 --2- 172.21.15.18:0/1671149642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a54104e00 0x7f1a54108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.589+0000 7f1a5a000640 1 --2- 172.21.15.18:0/1671149642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54104450 0x7f1a54104830 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.589+0000 7f1a5a000640 1 -- 172.21.15.18:0/1671149642 >> 172.21.15.18:0/1671149642 conn(0x7f1a540ffc90 msgr2=0x7f1a541020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.589+0000 7f1a5a000640 1 -- 172.21.15.18:0/1671149642 shutdown_connections 2024-04-16T17:07:09.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.589+0000 7f1a5a000640 1 -- 172.21.15.18:0/1671149642 wait complete. 2024-04-16T17:07:09.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.590+0000 7f1a5a000640 1 Processor -- start 2024-04-16T17:07:09.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.590+0000 7f1a5a000640 1 -- start start 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.590+0000 7f1a5a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a54104450 0x7f1a541a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.590+0000 7f1a5a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54109520 0x7f1a541a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a5a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5419a910 0x7f1a5419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a5a000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541145b0 con 0x7f1a54104450 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a5a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54114430 con 0x7f1a54109520 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a5a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54114730 con 0x7f1a5419a910 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a53400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a54104450 0x7f1a541a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a53400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a54104450 0x7f1a541a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33934/0 (socket says 172.21.15.18:33934) 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a53400640 1 -- 172.21.15.18:0/265152410 learned_addr learned my addr 172.21.15.18:0/265152410 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a52a00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54109520 0x7f1a541a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a53e00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5419a910 0x7f1a5419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a52a00640 1 -- 172.21.15.18:0/265152410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5419a910 msgr2=0x7f1a5419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a52a00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5419a910 0x7f1a5419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a52a00640 1 -- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a54104450 msgr2=0x7f1a541a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a52a00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a54104450 0x7f1a541a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a52a00640 1 -- 172.21.15.18:0/265152410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5419b430 con 0x7f1a54109520 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a53e00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5419a910 0x7f1a5419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.591+0000 7f1a53400640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a54104450 0x7f1a541a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.592+0000 7f1a52a00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54109520 0x7f1a541a0ca0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e9c0 tx=0x7f1a4000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.592+0000 7f1a4be00640 1 -- 172.21.15.18:0/265152410 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000ccc0 con 0x7f1a54109520 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.592+0000 7f1a4be00640 1 -- 172.21.15.18:0/265152410 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a40010910 con 0x7f1a54109520 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.592+0000 7f1a4be00640 1 -- 172.21.15.18:0/265152410 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a400188c0 con 0x7f1a54109520 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.592+0000 7f1a5a000640 1 -- 172.21.15.18:0/265152410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5419b720 con 0x7f1a54109520 2024-04-16T17:07:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.592+0000 7f1a5a000640 1 -- 172.21.15.18:0/265152410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a541a7920 con 0x7f1a54109520 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a5cdc2740 1 Processor -- start 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a5cdc2740 1 -- start start 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca799130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca7a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 0x559aca7a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559aca6fe020 con 0x559aca798d50 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559aca6a4460 con 0x559aca799670 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559aca6f9fe0 con 0x559aca7a2250 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.596+0000 7f1a4be00640 1 -- 172.21.15.18:0/265152410 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a40010ac0 con 0x7f1a54109520 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a52a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 0x559aca7a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca799130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca799130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33944/0 (socket says 172.21.15.18:33944) 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53e00640 1 -- 172.21.15.18:0/2992474771 learned_addr learned my addr 172.21.15.18:0/2992474771 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53400640 1 --2- 172.21.15.18:0/2992474771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca7a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53400640 1 -- 172.21.15.18:0/2992474771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 msgr2=0x559aca7a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53400640 1 --2- 172.21.15.18:0/2992474771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 0x559aca7a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53400640 1 -- 172.21.15.18:0/2992474771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 msgr2=0x559aca799130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53400640 1 --2- 172.21.15.18:0/2992474771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca799130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53400640 1 -- 172.21.15.18:0/2992474771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aca6ac040 con 0x559aca799670 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.597+0000 7f1a53400640 1 --2- 172.21.15.18:0/2992474771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca7a1d10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a5419ba50 tx=0x7f1a44009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfd01bd9906bbf06 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.598+0000 7f1a3fe00640 1 -- 172.21.15.18:0/2992474771 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44042070 con 0x559aca799670 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.598+0000 7f1a3fe00640 1 -- 172.21.15.18:0/2992474771 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a44004a70 con 0x559aca799670 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.598+0000 7f1a3fe00640 1 -- 172.21.15.18:0/2992474771 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44005190 con 0x559aca799670 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.598+0000 7f1a4be00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a380a3e20 0x7f1a380a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.598+0000 7f1a4be00640 1 -- 172.21.15.18:0/265152410 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1a400d0fb0 con 0x7f1a54109520 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.598+0000 7f1a53e00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a380a3e20 0x7f1a380a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.598+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2992474771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 msgr2=0x559aca7a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.598+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/2992474771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca7a1d10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a5419ba50 tx=0x7f1a44009560 comp rx=0 tx=0).stop 2024-04-16T17:07:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.599+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2992474771 shutdown_connections 2024-04-16T17:07:09.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.599+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/2992474771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 0x559aca7a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.599+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/2992474771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca7a1d10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.599+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/2992474771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca799130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.599+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2992474771 >> 172.21.15.18:0/2992474771 conn(0x559aca793480 msgr2=0x559aca793860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.599+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2992474771 shutdown_connections 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.599+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2992474771 wait complete. 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.599+0000 7f1a53e00640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a380a3e20 0x7f1a380a62e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fd0 tx=0x7f1a4c009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.600+0000 7f1a5cdc2740 1 -- start start 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.600+0000 7f1a5cdc2740 1 Processor -- start 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.600+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca79f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.600+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca79f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.600+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 0x559aca79fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.600+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559aca7a4fe0 con 0x559aca798d50 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559aca6b7830 con 0x559aca799670 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559aca7a5160 con 0x559aca7a2250 2024-04-16T17:07:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a5cdc2740 10 client.0 _ll_register_callbacks cb 0x559aca6bf150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a53400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca79f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a52a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca79f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a52a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca79f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57826/0 (socket says 172.21.15.18:57826) 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a52a00640 1 -- 172.21.15.18:0/3822038063 learned_addr learned my addr 172.21.15.18:0/3822038063 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a53e00640 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 0x559aca79fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a52a00640 1 -- 172.21.15.18:0/3822038063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 msgr2=0x559aca79fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a52a00640 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 0x559aca79fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a52a00640 1 -- 172.21.15.18:0/3822038063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 msgr2=0x559aca79f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a52a00640 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca79f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a52a00640 1 -- 172.21.15.18:0/3822038063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aca6bb920 con 0x559aca799670 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.601+0000 7f1a53400640 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca79f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.602+0000 7f1a52a00640 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca79f8f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1a40098fa0 tx=0x7f1a400ce350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7c0d72cabdcb1a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.602+0000 7f1a3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.602+0000 7f1a3cc00640 1 -- 172.21.15.18:0/3822038063 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4009c680 con 0x559aca799670 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.602+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/3822038063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a44004a70 con 0x559aca799670 2024-04-16T17:07:09.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.602+0000 7f1a3cc00640 1 -- 172.21.15.18:0/3822038063 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1a400d4b60 con 0x559aca799670 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.602+0000 7f1a3cc00640 1 -- 172.21.15.18:0/3822038063 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a4009c4a0 con 0x559aca799670 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.602+0000 7f1a5cdc2740 10 client.26264 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.602+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/3822038063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559aca6897f0 con 0x559aca799670 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a3cc00640 1 -- 172.21.15.18:0/3822038063 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a44004a70 con 0x559aca799670 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a3cc00640 1 -- 172.21.15.18:0/3822038063 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1a4009bf90 con 0x559aca799670 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a5cdc2740 10 client.26264 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a5cdc2740 10 client.26264 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a5cdc2740 20 client.26264 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559aca7b2640 0x559aca7b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a5cdc2740 4 client.26264 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/3822038063 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aca70ba40 con 0x559aca7b2640 2024-04-16T17:07:09.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.603+0000 7f1a53400640 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559aca7b2640 0x559aca7b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.604+0000 7f1a53400640 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559aca7b2640 0x559aca7b4ab0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd08e5d9b234d4b4 server_cookie=2b79ce8b65e7c6b8 in_seq=0 out_seq=0 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.604+0000 7f1a3cc00640 10 client.26264 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.605+0000 7f1a3cc00640 1 -- 172.21.15.18:0/3822038063 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559aca70ba40 con 0x559aca7b2640 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.605+0000 7f1a3cc00640 10 client.26264 handle_command_reply: tid=0 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.605+0000 7f1a5cdc2740 1 client.26264 shutdown 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/3822038063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 msgr2=0x559aca79f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca79f8f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1a40098fa0 tx=0x7f1a400ce350 comp rx=0 tx=0).stop 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/3822038063 shutdown_connections 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559aca7b2640 0x559aca7b4ab0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca7a2250 0x559aca79fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca799670 0x559aca79f8f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a3cc00640 0 client.26264 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/3822038063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca798d50 0x559aca79f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/3822038063 >> 172.21.15.18:0/3822038063 conn(0x559aca793480 msgr2=0x559aca7951d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/3822038063 shutdown_connections 2024-04-16T17:07:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/3822038063 wait complete. 2024-04-16T17:07:09.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.606+0000 7f1a5cdc2740 20 client.26264 trim_cache size 0 max 16384 2024-04-16T17:07:09.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.635+0000 7f1a5cdc2740 1 Processor -- start 2024-04-16T17:07:09.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.635+0000 7f1a5cdc2740 1 -- start start 2024-04-16T17:07:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca798d50 0x559aca7adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca799670 0x559aca792360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca7a2250 0x559aca7ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b4060 con 0x559aca799670 2024-04-16T17:07:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b1930 con 0x559aca7a2250 2024-04-16T17:07:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541af780 con 0x559aca798d50 2024-04-16T17:07:09.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca798d50 0x559aca7adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca798d50 0x559aca7adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55102/0 (socket says 172.21.15.18:55102) 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53400640 1 -- 172.21.15.18:0/2077758801 learned_addr learned my addr 172.21.15.18:0/2077758801 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53e00640 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca7a2250 0x559aca7ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a52a00640 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca799670 0x559aca792360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53400640 1 -- 172.21.15.18:0/2077758801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca7a2250 msgr2=0x559aca7ba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53400640 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca7a2250 0x559aca7ba380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53400640 1 -- 172.21.15.18:0/2077758801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca799670 msgr2=0x559aca792360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53400640 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca799670 0x559aca792360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.636+0000 7f1a53400640 1 -- 172.21.15.18:0/2077758801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aca4103f0 con 0x559aca798d50 2024-04-16T17:07:09.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.637+0000 7f1a53400640 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca798d50 0x559aca7adea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1a44004e00 tx=0x7f1a44048040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=537523d59133f86b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.637+0000 7f1a3f400640 1 -- 172.21.15.18:0/2077758801 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44042040 con 0x559aca798d50 2024-04-16T17:07:09.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.637+0000 7f1a3f400640 1 -- 172.21.15.18:0/2077758801 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a44004a70 con 0x559aca798d50 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.637+0000 7f1a3f400640 1 -- 172.21.15.18:0/2077758801 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44053460 con 0x559aca798d50 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.638+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2077758801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca798d50 msgr2=0x559aca7adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.638+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca798d50 0x559aca7adea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1a44004e00 tx=0x7f1a44048040 comp rx=0 tx=0).stop 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.638+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2077758801 shutdown_connections 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.638+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca7a2250 0x559aca7ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.638+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca799670 0x559aca792360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.638+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/2077758801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca798d50 0x559aca7adea0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.638+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2077758801 >> 172.21.15.18:0/2077758801 conn(0x559aca793480 msgr2=0x559aca791c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.639+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2077758801 shutdown_connections 2024-04-16T17:07:09.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.639+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/2077758801 wait complete. 2024-04-16T17:07:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.639+0000 7f1a5cdc2740 1 -- start start 2024-04-16T17:07:09.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.640+0000 7f1a5cdc2740 1 Processor -- start 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.640+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca798d50 0x559aca79c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.640+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca799670 0x559aca79cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.640+0000 7f1a5cdc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca7a2250 0x559aca817bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.640+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541afdf0 con 0x559aca7a2250 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.640+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b1260 con 0x559aca798d50 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.640+0000 7f1a5cdc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a541b04c0 con 0x559aca799670 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.640+0000 7f1a5cdc2740 10 client.0 _ll_register_callbacks cb 0x7f1a400d46e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca798d50 0x559aca79c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca798d50 0x559aca79c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57844/0 (socket says 172.21.15.18:57844) 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 -- 172.21.15.18:0/1636297248 learned_addr learned my addr 172.21.15.18:0/1636297248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a52a00640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca799670 0x559aca79cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53e00640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca7a2250 0x559aca817bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 -- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca799670 msgr2=0x559aca79cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca799670 0x559aca79cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 -- 172.21.15.18:0/1636297248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca7a2250 msgr2=0x559aca817bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca7a2250 0x559aca817bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 -- 172.21.15.18:0/1636297248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aca4da7d0 con 0x559aca798d50 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a52a00640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca799670 0x559aca79cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53e00640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca7a2250 0x559aca817bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a53400640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca798d50 0x559aca79c6b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1a44036040 tx=0x7f1a4402c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5c9b5194425b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.641+0000 7f1a3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.642+0000 7f1a3cc00640 1 -- 172.21.15.18:0/1636297248 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44042040 con 0x559aca798d50 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.642+0000 7f1a3cc00640 1 -- 172.21.15.18:0/1636297248 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1a44053400 con 0x559aca798d50 2024-04-16T17:07:09.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.642+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/1636297248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a44004a70 con 0x559aca798d50 2024-04-16T17:07:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.642+0000 7f1a3cc00640 1 -- 172.21.15.18:0/1636297248 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a44053600 con 0x559aca798d50 2024-04-16T17:07:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.642+0000 7f1a5cdc2740 10 client.26270 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.642+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/1636297248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559aca819160 con 0x559aca798d50 2024-04-16T17:07:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a3cc00640 1 -- 172.21.15.18:0/1636297248 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a44004a70 con 0x559aca798d50 2024-04-16T17:07:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a3cc00640 1 -- 172.21.15.18:0/1636297248 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1a44065020 con 0x559aca798d50 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a5cdc2740 10 client.26270 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a5cdc2740 10 client.26270 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a5cdc2740 20 client.26270 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559aca819500 0x559aca81b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a5cdc2740 4 client.26270 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/1636297248 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559aca7058b0 con 0x559aca819500 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.643+0000 7f1a52a00640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559aca819500 0x559aca81b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.644+0000 7f1a3cc00640 10 client.26270 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.644+0000 7f1a52a00640 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559aca819500 0x559aca81b8f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b42a73484d463274 server_cookie=188f6b887d7c49cd in_seq=0 out_seq=0 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.644+0000 7f1a3cc00640 1 -- 172.21.15.18:0/1636297248 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559aca7058b0 con 0x559aca819500 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.644+0000 7f1a3cc00640 10 client.26270 handle_command_reply: tid=0 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.644+0000 7f1a5cdc2740 1 client.26270 shutdown 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.644+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/1636297248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca798d50 msgr2=0x559aca79c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.644+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca798d50 0x559aca79c6b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1a44036040 tx=0x7f1a4402c3c0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/1636297248 shutdown_connections 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559aca819500 0x559aca81b8f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559aca7a2250 0x559aca817bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559aca799670 0x559aca79cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 1 --2- 172.21.15.18:0/1636297248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559aca798d50 0x559aca79c6b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a3cc00640 0 client.26270 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/1636297248 >> 172.21.15.18:0/1636297248 conn(0x559aca793480 msgr2=0x559aca791c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/1636297248 shutdown_connections 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 1 -- 172.21.15.18:0/1636297248 wait complete. 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.645+0000 7f1a5cdc2740 20 client.26270 trim_cache size 0 max 16384 2024-04-16T17:07:09.648 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 -- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a380a3e20 msgr2=0x7f1a380a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a380a3e20 0x7f1a380a62e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fd0 tx=0x7f1a4c009500 comp rx=0 tx=0).stop 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 -- 172.21.15.18:0/265152410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54109520 msgr2=0x7f1a541a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54109520 0x7f1a541a0ca0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e9c0 tx=0x7f1a4000ee80 comp rx=0 tx=0).stop 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 -- 172.21.15.18:0/265152410 shutdown_connections 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a380a3e20 0x7f1a380a62e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5419a910 0x7f1a5419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a54109520 0x7f1a541a0ca0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 --2- 172.21.15.18:0/265152410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a54104450 0x7f1a541a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.646+0000 7f1a5a000640 1 -- 172.21.15.18:0/265152410 >> 172.21.15.18:0/265152410 conn(0x7f1a540ffc90 msgr2=0x7f1a54101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.647+0000 7f1a5a000640 1 -- 172.21.15.18:0/265152410 shutdown_connections 2024-04-16T17:07:09.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.647+0000 7f1a5a000640 1 -- 172.21.15.18:0/265152410 wait complete. 2024-04-16T17:07:09.672 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:07:09.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:09 smithi043 ceph-mon[36901]: pgmap v75: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-16T17:07:09.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:09 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/842014523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:09.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 -- 172.21.15.18:0/1372983616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9480743b0 msgr2=0x7fd948074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 --2- 172.21.15.18:0/1372983616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9480743b0 0x7fd948074790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009900 tx=0x7fd93c02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 -- 172.21.15.18:0/1372983616 shutdown_connections 2024-04-16T17:07:09.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 --2- 172.21.15.18:0/1372983616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd94810e6c0 0x7fd948110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 --2- 172.21.15.18:0/1372983616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd948074cd0 0x7fd94810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 --2- 172.21.15.18:0/1372983616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9480743b0 0x7fd948074790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 -- 172.21.15.18:0/1372983616 >> 172.21.15.18:0/1372983616 conn(0x7fd9480fd950 msgr2=0x7fd9480ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 -- 172.21.15.18:0/1372983616 shutdown_connections 2024-04-16T17:07:09.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.876+0000 7fd94f200640 1 -- 172.21.15.18:0/1372983616 wait complete. 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.877+0000 7fd94f200640 1 Processor -- start 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94f200640 1 -- start start 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9480743b0 0x7fd9481a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd948074cd0 0x7fd9481a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd94810e6c0 0x7fd94819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481143c0 con 0x7fd94810e6c0 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd948114240 con 0x7fd948074cd0 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd948114540 con 0x7fd9480743b0 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9480743b0 0x7fd9481a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9480743b0 0x7fd9481a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55134/0 (socket says 172.21.15.18:55134) 2024-04-16T17:07:09.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd948074cd0 0x7fd9481a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.878+0000 7fd94de00640 1 -- 172.21.15.18:0/2725942325 learned_addr learned my addr 172.21.15.18:0/2725942325 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.879+0000 7fd94e800640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd94810e6c0 0x7fd94819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.879+0000 7fd94de00640 1 -- 172.21.15.18:0/2725942325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd948074cd0 msgr2=0x7fd9481a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.879+0000 7fd94de00640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd948074cd0 0x7fd9481a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.879+0000 7fd94de00640 1 -- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd94810e6c0 msgr2=0x7fd94819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.879+0000 7fd94de00640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd94810e6c0 0x7fd94819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.879+0000 7fd94de00640 1 -- 172.21.15.18:0/2725942325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94819b040 con 0x7fd9480743b0 2024-04-16T17:07:09.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.879+0000 7fd94de00640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9480743b0 0x7fd9481a0670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd93c036040 tx=0x7fd93c02cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.880+0000 7fd942a00640 1 -- 172.21.15.18:0/2725942325 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c043020 con 0x7fd9480743b0 2024-04-16T17:07:09.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.880+0000 7fd942a00640 1 -- 172.21.15.18:0/2725942325 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93c0054e0 con 0x7fd9480743b0 2024-04-16T17:07:09.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.880+0000 7fd942a00640 1 -- 172.21.15.18:0/2725942325 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c003c20 con 0x7fd9480743b0 2024-04-16T17:07:09.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.880+0000 7fd94f200640 1 -- 172.21.15.18:0/2725942325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94819b2d0 con 0x7fd9480743b0 2024-04-16T17:07:09.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.880+0000 7fd94f200640 1 -- 172.21.15.18:0/2725942325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9481a75f0 con 0x7fd9480743b0 2024-04-16T17:07:09.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.883+0000 7fd95341e740 1 Processor -- start 2024-04-16T17:07:09.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd95341e740 1 -- start start 2024-04-16T17:07:09.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd95341e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd95341e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd95341e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 0x56087cbfc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd95341e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56087cb56020 con 0x56087cbf0f10 2024-04-16T17:07:09.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd95341e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56087cafc460 con 0x56087cbf1830 2024-04-16T17:07:09.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd95341e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56087cb51fe0 con 0x56087cbfa410 2024-04-16T17:07:09.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd94e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd94e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33994/0 (socket says 172.21.15.18:33994) 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd94e800640 1 -- 172.21.15.18:0/4211667654 learned_addr learned my addr 172.21.15.18:0/4211667654 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.884+0000 7fd94d400640 1 --2- 172.21.15.18:0/4211667654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 0x56087cbfc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.885+0000 7fd94e800640 1 -- 172.21.15.18:0/4211667654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 msgr2=0x56087cbfc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.885+0000 7fd94e800640 1 --2- 172.21.15.18:0/4211667654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 0x56087cbfc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.885+0000 7fd94e800640 1 -- 172.21.15.18:0/4211667654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 msgr2=0x56087cbf9ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.885+0000 7fd94e800640 1 --2- 172.21.15.18:0/4211667654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf9ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.885+0000 7fd94e800640 1 -- 172.21.15.18:0/4211667654 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087cb04040 con 0x56087cbf0f10 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.885+0000 7fd942a00640 1 -- 172.21.15.18:0/2725942325 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd93c046020 con 0x7fd9480743b0 2024-04-16T17:07:09.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.885+0000 7fd94e800640 1 --2- 172.21.15.18:0/4211667654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf12f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd944009f40 tx=0x7fd944010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b21f01475aaa361 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.886+0000 7fd937000640 1 -- 172.21.15.18:0/4211667654 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944003b70 con 0x56087cbf0f10 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.886+0000 7fd937000640 1 -- 172.21.15.18:0/4211667654 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd944003d10 con 0x56087cbf0f10 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.886+0000 7fd937000640 1 -- 172.21.15.18:0/4211667654 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944011a40 con 0x56087cbf0f10 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.886+0000 7fd95341e740 1 -- 172.21.15.18:0/4211667654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 msgr2=0x56087cbf12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.886+0000 7fd95341e740 1 --2- 172.21.15.18:0/4211667654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf12f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd944009f40 tx=0x7fd944010860 comp rx=0 tx=0).stop 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.886+0000 7fd95341e740 1 -- 172.21.15.18:0/4211667654 shutdown_connections 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.886+0000 7fd95341e740 1 --2- 172.21.15.18:0/4211667654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 0x56087cbfc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd95341e740 1 --2- 172.21.15.18:0/4211667654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd95341e740 1 --2- 172.21.15.18:0/4211667654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf12f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd95341e740 1 -- 172.21.15.18:0/4211667654 >> 172.21.15.18:0/4211667654 conn(0x56087cbeb640 msgr2=0x56087cbeba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd95341e740 1 -- 172.21.15.18:0/4211667654 shutdown_connections 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd95341e740 1 -- 172.21.15.18:0/4211667654 wait complete. 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd942a00640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9300a3cb0 0x7fd9300a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd942a00640 1 -- 172.21.15.18:0/2725942325 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd93c0f2cf0 con 0x7fd9480743b0 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd95341e740 1 -- start start 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.887+0000 7fd94d400640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9300a3cb0 0x7fd9300a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd95341e740 1 Processor -- start 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd95341e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd95341e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd95341e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cbf7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd95341e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56087cbfd200 con 0x56087cbfa410 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd95341e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56087cb0f830 con 0x56087cbf1830 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd95341e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56087cbfd380 con 0x56087cbf0f10 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd95341e740 10 client.0 _ll_register_callbacks cb 0x56087cb17150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd94e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cbf7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd94de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd94e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cbf7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34000/0 (socket says 172.21.15.18:34000) 2024-04-16T17:07:09.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.888+0000 7fd94e800640 1 -- 172.21.15.18:0/1410141780 learned_addr learned my addr 172.21.15.18:0/1410141780 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd94de00640 1 -- 172.21.15.18:0/1410141780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 msgr2=0x56087cbf79f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd94de00640 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf79f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd94de00640 1 -- 172.21.15.18:0/1410141780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 msgr2=0x56087cbf7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd94de00640 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cbf7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd94de00640 1 -- 172.21.15.18:0/1410141780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087cb13920 con 0x56087cbf0f10 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd94d400640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9300a3cb0 0x7fd9300a6170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x56087cbec7d0 tx=0x7fd93800b4d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd94de00640 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf74b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0f0140 tx=0x7fd93c0be4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ab96e45dd195ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd92b400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd92b400640 1 -- 172.21.15.18:0/1410141780 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c005690 con 0x56087cbf0f10 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd92b400640 1 -- 172.21.15.18:0/1410141780 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd93c0bb000 con 0x56087cbf0f10 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.889+0000 7fd95341e740 1 -- 172.21.15.18:0/1410141780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd944003d10 con 0x56087cbf0f10 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd92b400640 1 -- 172.21.15.18:0/1410141780 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd93c0bf090 con 0x56087cbf0f10 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd95341e740 10 client.26281 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd95341e740 1 -- 172.21.15.18:0/1410141780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56087cae17f0 con 0x56087cbf0f10 2024-04-16T17:07:09.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd92b400640 1 -- 172.21.15.18:0/1410141780 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd944003d10 con 0x56087cbf0f10 2024-04-16T17:07:09.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd92b400640 1 -- 172.21.15.18:0/1410141780 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd93c0bebf0 con 0x56087cbf0f10 2024-04-16T17:07:09.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd95341e740 10 client.26281 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd95341e740 10 client.26281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd95341e740 20 client.26281 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd95341e740 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56087cc0a820 0x56087cc0cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd95341e740 4 client.26281 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.890+0000 7fd95341e740 1 -- 172.21.15.18:0/1410141780 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56087cb63a40 con 0x56087cc0a820 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.891+0000 7fd94d400640 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56087cc0a820 0x56087cc0cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.891+0000 7fd94d400640 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56087cc0a820 0x56087cc0cc50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=773afd3aa6a4269a server_cookie=f424080102869fda in_seq=0 out_seq=0 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.891+0000 7fd92b400640 10 client.26281 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.892+0000 7fd92b400640 1 -- 172.21.15.18:0/1410141780 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56087cb63a40 con 0x56087cc0a820 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.892+0000 7fd92b400640 10 client.26281 handle_command_reply: tid=0 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.892+0000 7fd95341e740 1 client.26281 shutdown 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 -- 172.21.15.18:0/1410141780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 msgr2=0x56087cbf74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf74b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0f0140 tx=0x7fd93c0be4d0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 -- 172.21.15.18:0/1410141780 shutdown_connections 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56087cc0a820 0x56087cc0cc50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cbf7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 --2- 172.21.15.18:0/1410141780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf74b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd92b400640 0 client.26281 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 -- 172.21.15.18:0/1410141780 >> 172.21.15.18:0/1410141780 conn(0x56087cbeb640 msgr2=0x56087cbed390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 -- 172.21.15.18:0/1410141780 shutdown_connections 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 1 -- 172.21.15.18:0/1410141780 wait complete. 2024-04-16T17:07:09.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.893+0000 7fd95341e740 20 client.26281 trim_cache size 0 max 16384 2024-04-16T17:07:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.908+0000 7fd95341e740 1 Processor -- start 2024-04-16T17:07:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.908+0000 7fd95341e740 1 -- start start 2024-04-16T17:07:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.908+0000 7fd95341e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.908+0000 7fd95341e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cc06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.908+0000 7fd95341e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cc12560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.908+0000 7fd95341e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481b3e60 con 0x56087cbfa410 2024-04-16T17:07:09.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.908+0000 7fd95341e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481b1730 con 0x56087cbf1830 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.908+0000 7fd95341e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481af580 con 0x56087cbf0f10 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cc06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cc06120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57872/0 (socket says 172.21.15.18:57872) 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 -- 172.21.15.18:0/3711582764 learned_addr learned my addr 172.21.15.18:0/3711582764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94e800640 1 --2- 172.21.15.18:0/3711582764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cc12560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 -- 172.21.15.18:0/3711582764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 msgr2=0x56087cbf9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 --2- 172.21.15.18:0/3711582764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 -- 172.21.15.18:0/3711582764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 msgr2=0x56087cc12560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 --2- 172.21.15.18:0/3711582764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cc12560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 -- 172.21.15.18:0/3711582764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087c8683f0 con 0x56087cbf1830 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.909+0000 7fd94d400640 1 --2- 172.21.15.18:0/3711582764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cc06120 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x56087cbf6d10 tx=0x7fd938015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0ac080e7a0016e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.910+0000 7fd936600640 1 -- 172.21.15.18:0/3711582764 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93802e070 con 0x56087cbf1830 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.910+0000 7fd936600640 1 -- 172.21.15.18:0/3711582764 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd938027710 con 0x56087cbf1830 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.910+0000 7fd936600640 1 -- 172.21.15.18:0/3711582764 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93801e6e0 con 0x56087cbf1830 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 -- 172.21.15.18:0/3711582764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 msgr2=0x56087cc06120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 --2- 172.21.15.18:0/3711582764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cc06120 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x56087cbf6d10 tx=0x7fd938015000 comp rx=0 tx=0).stop 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 -- 172.21.15.18:0/3711582764 shutdown_connections 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 --2- 172.21.15.18:0/3711582764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbfa410 0x56087cc12560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 --2- 172.21.15.18:0/3711582764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cc06120 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 --2- 172.21.15.18:0/3711582764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbf0f10 0x56087cbf9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 -- 172.21.15.18:0/3711582764 >> 172.21.15.18:0/3711582764 conn(0x56087cbeb640 msgr2=0x56087cbed300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 -- 172.21.15.18:0/3711582764 shutdown_connections 2024-04-16T17:07:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 -- 172.21.15.18:0/3711582764 wait complete. 2024-04-16T17:07:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 -- start start 2024-04-16T17:07:09.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.911+0000 7fd95341e740 1 Processor -- start 2024-04-16T17:07:09.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd95341e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd95341e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd95341e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 0x56087cbf4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd95341e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481afbf0 con 0x56087cbf0f10 2024-04-16T17:07:09.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd95341e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481b1060 con 0x56087cbf1830 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd95341e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481b02c0 con 0x56087cbfa410 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd94d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd95341e740 10 client.0 _ll_register_callbacks cb 0x7fd93c02cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd94d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57886/0 (socket says 172.21.15.18:57886) 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd94d400640 1 -- 172.21.15.18:0/2627082649 learned_addr learned my addr 172.21.15.18:0/2627082649 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd94de00640 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.912+0000 7fd94d400640 1 -- 172.21.15.18:0/2627082649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 msgr2=0x56087cbf4da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd94d400640 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 0x56087cbf4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd94d400640 1 -- 172.21.15.18:0/2627082649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 msgr2=0x56087cbf4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd94d400640 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd94d400640 1 -- 172.21.15.18:0/2627082649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087c9327d0 con 0x56087cbf1830 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd94d400640 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf4860 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd93801de90 tx=0x7fd938027940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fabfb0f2931f04f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:09.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd92b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd92b400640 1 -- 172.21.15.18:0/2627082649 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938037070 con 0x56087cbf1830 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd92b400640 1 -- 172.21.15.18:0/2627082649 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd938005510 con 0x56087cbf1830 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.913+0000 7fd95341e740 1 -- 172.21.15.18:0/2627082649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd938027710 con 0x56087cbf1830 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd92b400640 1 -- 172.21.15.18:0/2627082649 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd938043050 con 0x56087cbf1830 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd95341e740 10 client.26282 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd95341e740 1 -- 172.21.15.18:0/2627082649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56087cc820b0 con 0x56087cbf1830 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd92b400640 1 -- 172.21.15.18:0/2627082649 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd938027710 con 0x56087cbf1830 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd92b400640 1 -- 172.21.15.18:0/2627082649 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd938027070 con 0x56087cbf1830 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd95341e740 10 client.26282 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd95341e740 10 client.26282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd95341e740 20 client.26282 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd95341e740 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56087cc827a0 0x56087cc84bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd95341e740 4 client.26282 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd95341e740 1 -- 172.21.15.18:0/2627082649 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56087cb5d8b0 con 0x56087cc827a0 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.914+0000 7fd94de00640 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56087cc827a0 0x56087cc84bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd94de00640 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56087cc827a0 0x56087cc84bc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcc2e861cd442124 server_cookie=415930dd87cac228 in_seq=0 out_seq=0 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd92b400640 10 client.26282 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd92b400640 1 -- 172.21.15.18:0/2627082649 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56087cb5d8b0 con 0x56087cc827a0 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd92b400640 10 client.26282 handle_command_reply: tid=0 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 client.26282 shutdown 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 -- 172.21.15.18:0/2627082649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 msgr2=0x56087cbf4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf4860 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd93801de90 tx=0x7fd938027940 comp rx=0 tx=0).stop 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 -- 172.21.15.18:0/2627082649 shutdown_connections 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56087cc827a0 0x56087cc84bc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087cbfa410 0x56087cbf4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd92b400640 0 client.26282 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:07:09.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087cbf1830 0x56087cbf4860 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 --2- 172.21.15.18:0/2627082649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087cbf0f10 0x56087cbf4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.915+0000 7fd95341e740 1 -- 172.21.15.18:0/2627082649 >> 172.21.15.18:0/2627082649 conn(0x56087cbeb640 msgr2=0x56087cbf7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.916+0000 7fd95341e740 1 -- 172.21.15.18:0/2627082649 shutdown_connections 2024-04-16T17:07:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.916+0000 7fd95341e740 1 -- 172.21.15.18:0/2627082649 wait complete. 2024-04-16T17:07:09.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.916+0000 7fd95341e740 20 client.26282 trim_cache size 0 max 16384 2024-04-16T17:07:09.921 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:07:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 -- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9300a3cb0 msgr2=0x7fd9300a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9300a3cb0 0x7fd9300a6170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x56087cbec7d0 tx=0x7fd93800b4d0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 -- 172.21.15.18:0/2725942325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9480743b0 msgr2=0x7fd9481a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9480743b0 0x7fd9481a0670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd93c036040 tx=0x7fd93c02cbc0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 -- 172.21.15.18:0/2725942325 shutdown_connections 2024-04-16T17:07:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9300a3cb0 0x7fd9300a6170 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd94810e6c0 0x7fd94819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd948074cd0 0x7fd9481a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 --2- 172.21.15.18:0/2725942325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9480743b0 0x7fd9481a0670 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 -- 172.21.15.18:0/2725942325 >> 172.21.15.18:0/2725942325 conn(0x7fd9480fd950 msgr2=0x7fd9480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 -- 172.21.15.18:0/2725942325 shutdown_connections 2024-04-16T17:07:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:09.917+0000 7fd94f200640 1 -- 172.21.15.18:0/2725942325 wait complete. 2024-04-16T17:07:09.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:09 smithi022 ceph-mon[36023]: pgmap v75: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-16T17:07:09.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:09 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/842014523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:09.941 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:07:10.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 -- 172.21.15.18:0/2452119966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4144074cd0 msgr2=0x7f414410e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 --2- 172.21.15.18:0/2452119966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4144074cd0 0x7f414410e340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4130009900 tx=0x7f413002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 -- 172.21.15.18:0/2452119966 shutdown_connections 2024-04-16T17:07:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 --2- 172.21.15.18:0/2452119966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414410e880 0x7f4144110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 --2- 172.21.15.18:0/2452119966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4144074cd0 0x7f414410e340 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 --2- 172.21.15.18:0/2452119966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41440743b0 0x7f4144074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 -- 172.21.15.18:0/2452119966 >> 172.21.15.18:0/2452119966 conn(0x7f41440fdab0 msgr2=0x7f41440ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 -- 172.21.15.18:0/2452119966 shutdown_connections 2024-04-16T17:07:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.137+0000 7f4149600640 1 -- 172.21.15.18:0/2452119966 wait complete. 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.138+0000 7f4149600640 1 Processor -- start 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.139+0000 7f4149600640 1 -- start start 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.139+0000 7f4149600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41440743b0 0x7f41441a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.139+0000 7f4149600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4144074cd0 0x7f41441a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.139+0000 7f4149600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414410e880 0x7f414419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.139+0000 7f4149600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4144114590 con 0x7f4144074cd0 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.139+0000 7f4149600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4144114410 con 0x7f41440743b0 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.139+0000 7f4149600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4144114710 con 0x7f414410e880 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4144074cd0 0x7f41441a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41440743b0 0x7f41441a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4144074cd0 0x7f41441a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34018/0 (socket says 172.21.15.18:34018) 2024-04-16T17:07:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41440743b0 0x7f41441a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57916/0 (socket says 172.21.15.18:57916) 2024-04-16T17:07:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142000640 1 -- 172.21.15.18:0/3037080268 learned_addr learned my addr 172.21.15.18:0/3037080268 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4143400640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414410e880 0x7f414419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142000640 1 -- 172.21.15.18:0/3037080268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414410e880 msgr2=0x7f414419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142000640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414410e880 0x7f414419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142000640 1 -- 172.21.15.18:0/3037080268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41440743b0 msgr2=0x7f41441a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142000640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41440743b0 0x7f41441a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142000640 1 -- 172.21.15.18:0/3037080268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414419b110 con 0x7f4144074cd0 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4143400640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414410e880 0x7f414419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.140+0000 7f4142a00640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41440743b0 0x7f41441a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.141+0000 7f4142000640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4144074cd0 0x7f41441a0cd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4130036040 tx=0x7f413002cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.141+0000 7f413b400640 1 -- 172.21.15.18:0/3037080268 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130043020 con 0x7f4144074cd0 2024-04-16T17:07:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.141+0000 7f413b400640 1 -- 172.21.15.18:0/3037080268 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41300054e0 con 0x7f4144074cd0 2024-04-16T17:07:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.141+0000 7f413b400640 1 -- 172.21.15.18:0/3037080268 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130003c20 con 0x7f4144074cd0 2024-04-16T17:07:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.141+0000 7f4149600640 1 -- 172.21.15.18:0/3037080268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414419b3a0 con 0x7f4144074cd0 2024-04-16T17:07:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.141+0000 7f4149600640 1 -- 172.21.15.18:0/3037080268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a7650 con 0x7f4144074cd0 2024-04-16T17:07:10.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.145+0000 7f414c3ed740 1 Processor -- start 2024-04-16T17:07:10.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.145+0000 7f414c3ed740 1 -- start start 2024-04-16T17:07:10.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eda5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56edab30 0x561d56ed3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eec4d0 0x561d56eee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f414c3ed740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561d56e37020 con 0x561d56eec4d0 2024-04-16T17:07:10.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f414c3ed740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d56ddd460 con 0x561d56edab30 2024-04-16T17:07:10.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f414c3ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561d56e32fe0 con 0x561d56eda210 2024-04-16T17:07:10.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f4142a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56edab30 0x561d56ed3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f4142a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56edab30 0x561d56ed3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57932/0 (socket says 172.21.15.18:57932) 2024-04-16T17:07:10.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f4143400640 1 --2- 172.21.15.18:0/1095988986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eda5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.146+0000 7f4142a00640 1 -- 172.21.15.18:0/1095988986 learned_addr learned my addr 172.21.15.18:0/1095988986 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.147+0000 7f4142a00640 1 -- 172.21.15.18:0/1095988986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 msgr2=0x561d56eda5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.147+0000 7f4142a00640 1 --2- 172.21.15.18:0/1095988986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eda5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.147+0000 7f4142a00640 1 -- 172.21.15.18:0/1095988986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eec4d0 msgr2=0x561d56eee8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.147+0000 7f4142a00640 1 --2- 172.21.15.18:0/1095988986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eec4d0 0x561d56eee8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.147+0000 7f4142a00640 1 -- 172.21.15.18:0/1095988986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d56de5040 con 0x561d56edab30 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.147+0000 7f4142a00640 1 --2- 172.21.15.18:0/1095988986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56edab30 0x561d56ed3db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41340114f0 tx=0x7f41340119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6673fff1798241a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.148+0000 7f412f400640 1 -- 172.21.15.18:0/1095988986 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134005020 con 0x561d56edab30 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.148+0000 7f412f400640 1 -- 172.21.15.18:0/1095988986 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41340047a0 con 0x561d56edab30 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.148+0000 7f412f400640 1 -- 172.21.15.18:0/1095988986 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413400f9c0 con 0x561d56edab30 2024-04-16T17:07:10.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.148+0000 7f413b400640 1 -- 172.21.15.18:0/3037080268 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4130046020 con 0x7f4144074cd0 2024-04-16T17:07:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.149+0000 7f414c3ed740 1 -- 172.21.15.18:0/1095988986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56edab30 msgr2=0x561d56ed3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.149+0000 7f414c3ed740 1 --2- 172.21.15.18:0/1095988986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56edab30 0x561d56ed3db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41340114f0 tx=0x7f41340119b0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.151+0000 7f413b400640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41280a3da0 0x7f41280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.152+0000 7f413b400640 1 -- 172.21.15.18:0/3037080268 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f41300f2b10 con 0x7f4144074cd0 2024-04-16T17:07:10.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.155+0000 7f414c3ed740 1 -- 172.21.15.18:0/1095988986 shutdown_connections 2024-04-16T17:07:10.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.155+0000 7f414c3ed740 1 --2- 172.21.15.18:0/1095988986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eec4d0 0x561d56eee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.155+0000 7f414c3ed740 1 --2- 172.21.15.18:0/1095988986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56edab30 0x561d56ed3db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.155+0000 7f414c3ed740 1 --2- 172.21.15.18:0/1095988986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eda5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.155+0000 7f414c3ed740 1 -- 172.21.15.18:0/1095988986 >> 172.21.15.18:0/1095988986 conn(0x561d56ed4c90 msgr2=0x561d56ed5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.156+0000 7f414c3ed740 1 -- 172.21.15.18:0/1095988986 shutdown_connections 2024-04-16T17:07:10.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.156+0000 7f414c3ed740 1 -- 172.21.15.18:0/1095988986 wait complete. 2024-04-16T17:07:10.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.156+0000 7f414c3ed740 1 -- start start 2024-04-16T17:07:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.157+0000 7f414c3ed740 1 Processor -- start 2024-04-16T17:07:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.157+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56ed13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.157+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56ed1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.157+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56ed1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.157+0000 7f414c3ed740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561d56eef100 con 0x561d56edab30 2024-04-16T17:07:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.157+0000 7f414c3ed740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d56df0830 con 0x561d56eec4d0 2024-04-16T17:07:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.157+0000 7f414c3ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561d56eef280 con 0x561d56eda210 2024-04-16T17:07:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.158+0000 7f414c3ed740 10 client.0 _ll_register_callbacks cb 0x561d56df8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.158+0000 7f4143400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56ed1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.158+0000 7f4143400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56ed1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57948/0 (socket says 172.21.15.18:57948) 2024-04-16T17:07:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.158+0000 7f4143400640 1 -- 172.21.15.18:0/2861918256 learned_addr learned my addr 172.21.15.18:0/2861918256 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.158+0000 7f4142a00640 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56ed13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.159+0000 7f4142000640 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56ed1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.159+0000 7f4143400640 1 -- 172.21.15.18:0/2861918256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 msgr2=0x561d56ed13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.159+0000 7f4143400640 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56ed13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.159+0000 7f4143400640 1 -- 172.21.15.18:0/2861918256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 msgr2=0x561d56ed1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.159+0000 7f4143400640 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56ed1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.159+0000 7f4143400640 1 -- 172.21.15.18:0/2861918256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d56df4920 con 0x561d56eec4d0 2024-04-16T17:07:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.159+0000 7f4143400640 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56ed1e50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f413c010ef0 tx=0x7f413c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6037d88663f2d3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.160+0000 7f4123e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:10.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.160+0000 7f4123e00640 1 -- 172.21.15.18:0/2861918256 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c021060 con 0x561d56eec4d0 2024-04-16T17:07:10.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.160+0000 7f414c3ed740 1 -- 172.21.15.18:0/2861918256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41340047a0 con 0x561d56eec4d0 2024-04-16T17:07:10.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.161+0000 7f4123e00640 1 -- 172.21.15.18:0/2861918256 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f413c00e7c0 con 0x561d56eec4d0 2024-04-16T17:07:10.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.161+0000 7f4123e00640 1 -- 172.21.15.18:0/2861918256 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f413c028050 con 0x561d56eec4d0 2024-04-16T17:07:10.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.161+0000 7f4143400640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41280a3da0 0x7f41280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.162+0000 7f414c3ed740 10 client.26294 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.162+0000 7f414c3ed740 1 -- 172.21.15.18:0/2861918256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d56dc27f0 con 0x561d56eec4d0 2024-04-16T17:07:10.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.163+0000 7f4123e00640 1 -- 172.21.15.18:0/2861918256 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41340047a0 con 0x561d56eec4d0 2024-04-16T17:07:10.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.163+0000 7f4123e00640 1 -- 172.21.15.18:0/2861918256 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f413c02d3d0 con 0x561d56eec4d0 2024-04-16T17:07:10.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.163+0000 7f414c3ed740 10 client.26294 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:10.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.163+0000 7f414c3ed740 10 client.26294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:10.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.163+0000 7f414c3ed740 20 client.26294 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:10.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.163+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d56efc6f0 0x561d56efeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.163+0000 7f414c3ed740 4 client.26294 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.163+0000 7f414c3ed740 1 -- 172.21.15.18:0/2861918256 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d56e44a40 con 0x561d56efc6f0 2024-04-16T17:07:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.164+0000 7f4142a00640 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d56efc6f0 0x561d56efeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.165+0000 7f4142a00640 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d56efc6f0 0x561d56efeb30 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de8cb9665798f65 server_cookie=82506848509d4e06 in_seq=0 out_seq=0 2024-04-16T17:07:10.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.165+0000 7f4143400640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41280a3da0 0x7f41280a6260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f413c025000 tx=0x7f413c008000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.165+0000 7f4123e00640 10 client.26294 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.166+0000 7f4123e00640 1 -- 172.21.15.18:0/2861918256 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d56e44a40 con 0x561d56efc6f0 2024-04-16T17:07:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.167+0000 7f4123e00640 10 client.26294 handle_command_reply: tid=0 2024-04-16T17:07:10.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.167+0000 7f414c3ed740 1 client.26294 shutdown 2024-04-16T17:07:10.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.168+0000 7f414c3ed740 1 -- 172.21.15.18:0/2861918256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 msgr2=0x561d56ed1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.168+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56ed1e50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f413c010ef0 tx=0x7f413c00f790 comp rx=0 tx=0).stop 2024-04-16T17:07:10.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.170+0000 7f414c3ed740 1 -- 172.21.15.18:0/2861918256 shutdown_connections 2024-04-16T17:07:10.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.170+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d56efc6f0 0x561d56efeb30 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.170+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56ed1e50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.170+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56ed1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.170+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2861918256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56ed13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.170+0000 7f414c3ed740 1 -- 172.21.15.18:0/2861918256 >> 172.21.15.18:0/2861918256 conn(0x561d56ed4c90 msgr2=0x561d56ed69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.171+0000 7f4123e00640 0 client.26294 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:10.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.171+0000 7f414c3ed740 1 -- 172.21.15.18:0/2861918256 shutdown_connections 2024-04-16T17:07:10.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.172+0000 7f414c3ed740 1 -- 172.21.15.18:0/2861918256 wait complete. 2024-04-16T17:07:10.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.172+0000 7f414c3ed740 20 client.26294 trim_cache size 0 max 16384 2024-04-16T17:07:10.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.207+0000 7f414c3ed740 1 Processor -- start 2024-04-16T17:07:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.207+0000 7f414c3ed740 1 -- start start 2024-04-16T17:07:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.208+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eda210 0x561d56ef7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.208+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56edab30 0x561d56ed1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.208+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f04400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.208+0000 7f414c3ed740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f41441afd20 con 0x561d56eda210 2024-04-16T17:07:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.208+0000 7f414c3ed740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ad5f0 con 0x561d56eec4d0 2024-04-16T17:07:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.208+0000 7f414c3ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ab440 con 0x561d56edab30 2024-04-16T17:07:10.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.209+0000 7f4142000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56edab30 0x561d56ed1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.209+0000 7f4142a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eda210 0x561d56ef7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.209+0000 7f4143400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f04400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.209+0000 7f4142a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eda210 0x561d56ef7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34042/0 (socket says 172.21.15.18:34042) 2024-04-16T17:07:10.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.209+0000 7f4142a00640 1 -- 172.21.15.18:0/3026914588 learned_addr learned my addr 172.21.15.18:0/3026914588 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.210+0000 7f4143400640 1 -- 172.21.15.18:0/3026914588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56edab30 msgr2=0x561d56ed1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.210+0000 7f4143400640 1 --2- 172.21.15.18:0/3026914588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56edab30 0x561d56ed1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.210+0000 7f4143400640 1 -- 172.21.15.18:0/3026914588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eda210 msgr2=0x561d56ef7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.210+0000 7f4143400640 1 --2- 172.21.15.18:0/3026914588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eda210 0x561d56ef7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.210+0000 7f4143400640 1 -- 172.21.15.18:0/3026914588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d56b493f0 con 0x561d56eec4d0 2024-04-16T17:07:10.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.210+0000 7f4143400640 1 --2- 172.21.15.18:0/3026914588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f04400 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f413c02da10 tx=0x7f413c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d049c60ef5b0537a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.211+0000 7f412ea00640 1 -- 172.21.15.18:0/3026914588 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c03b070 con 0x561d56eec4d0 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.211+0000 7f412ea00640 1 -- 172.21.15.18:0/3026914588 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413c02d3d0 con 0x561d56eec4d0 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.211+0000 7f412ea00640 1 -- 172.21.15.18:0/3026914588 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c033070 con 0x561d56eec4d0 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.213+0000 7f414c3ed740 1 -- 172.21.15.18:0/3026914588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 msgr2=0x561d56f04400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.213+0000 7f414c3ed740 1 --2- 172.21.15.18:0/3026914588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f04400 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f413c02da10 tx=0x7f413c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.214+0000 7f414c3ed740 1 -- 172.21.15.18:0/3026914588 shutdown_connections 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.214+0000 7f414c3ed740 1 --2- 172.21.15.18:0/3026914588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f04400 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.214+0000 7f414c3ed740 1 --2- 172.21.15.18:0/3026914588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56edab30 0x561d56ed1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.214+0000 7f414c3ed740 1 --2- 172.21.15.18:0/3026914588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56eda210 0x561d56ef7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.214+0000 7f414c3ed740 1 -- 172.21.15.18:0/3026914588 >> 172.21.15.18:0/3026914588 conn(0x561d56ed4c90 msgr2=0x561d56ecc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.217+0000 7f414c3ed740 1 -- 172.21.15.18:0/3026914588 shutdown_connections 2024-04-16T17:07:10.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.217+0000 7f414c3ed740 1 -- 172.21.15.18:0/3026914588 wait complete. 2024-04-16T17:07:10.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.218+0000 7f414c3ed740 1 -- start start 2024-04-16T17:07:10.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.218+0000 7f414c3ed740 1 Processor -- start 2024-04-16T17:07:10.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.220+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.220+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56eefbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.220+0000 7f4142a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.220+0000 7f414c3ed740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f60c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.220+0000 7f4142a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eef680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55226/0 (socket says 172.21.15.18:55226) 2024-04-16T17:07:10.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.221+0000 7f4142a00640 1 -- 172.21.15.18:0/2648596817 learned_addr learned my addr 172.21.15.18:0/2648596817 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.221+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f41441abab0 con 0x561d56edab30 2024-04-16T17:07:10.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.221+0000 7f4143400640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f60c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.221+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41441acf20 con 0x561d56eec4d0 2024-04-16T17:07:10.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.221+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ac180 con 0x561d56eda210 2024-04-16T17:07:10.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.221+0000 7f414c3ed740 10 client.0 _ll_register_callbacks cb 0x7f413c010510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:10.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.222+0000 7f4142000640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56eefbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.223+0000 7f4143400640 1 -- 172.21.15.18:0/2648596817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 msgr2=0x561d56eef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.223+0000 7f4143400640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.223+0000 7f4143400640 1 -- 172.21.15.18:0/2648596817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 msgr2=0x561d56eefbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.223+0000 7f4143400640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56eefbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.223+0000 7f4143400640 1 -- 172.21.15.18:0/2648596817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d56c137d0 con 0x561d56eec4d0 2024-04-16T17:07:10.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.223+0000 7f4142000640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56eefbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:10.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.223+0000 7f4142a00640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:10.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.224+0000 7f4143400640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f60c60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f413c02d820 tx=0x7f413c010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3673812bd87d95df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.224+0000 7f4123e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:10.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.224+0000 7f4123e00640 1 -- 172.21.15.18:0/2648596817 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c03b070 con 0x561d56eec4d0 2024-04-16T17:07:10.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.224+0000 7f4123e00640 1 -- 172.21.15.18:0/2648596817 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f413c02dc40 con 0x561d56eec4d0 2024-04-16T17:07:10.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.224+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f413c02d3d0 con 0x561d56eec4d0 2024-04-16T17:07:10.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.225+0000 7f4123e00640 1 -- 172.21.15.18:0/2648596817 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f413c033040 con 0x561d56eec4d0 2024-04-16T17:07:10.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.225+0000 7f414c3ed740 10 client.26300 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:10.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.225+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d56f62270 con 0x561d56eec4d0 2024-04-16T17:07:10.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.226+0000 7f4123e00640 1 -- 172.21.15.18:0/2648596817 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413c02d3d0 con 0x561d56eec4d0 2024-04-16T17:07:10.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.227+0000 7f4123e00640 1 -- 172.21.15.18:0/2648596817 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f413c00e5c0 con 0x561d56eec4d0 2024-04-16T17:07:10.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.227+0000 7f414c3ed740 10 client.26300 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:10.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.227+0000 7f414c3ed740 10 client.26300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:10.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.227+0000 7f414c3ed740 20 client.26300 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:10.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.227+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d56f62820 0x561d56f64c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.227+0000 7f414c3ed740 4 client.26300 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:10.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.227+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d56e3e8b0 con 0x561d56f62820 2024-04-16T17:07:10.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.229+0000 7f4142a00640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d56f62820 0x561d56f64c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.230+0000 7f4142a00640 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d56f62820 0x561d56f64c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd87345598ea44d server_cookie=bcaaef2181ca12e6 in_seq=0 out_seq=0 2024-04-16T17:07:10.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.230+0000 7f4123e00640 10 client.26300 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:10.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.232+0000 7f4123e00640 1 -- 172.21.15.18:0/2648596817 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d56e3e8b0 con 0x561d56f62820 2024-04-16T17:07:10.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.232+0000 7f4123e00640 10 client.26300 handle_command_reply: tid=0 2024-04-16T17:07:10.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.232+0000 7f414c3ed740 1 client.26300 shutdown 2024-04-16T17:07:10.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.232+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 msgr2=0x561d56f60c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.232+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f60c60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f413c02d820 tx=0x7f413c010430 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.233+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 shutdown_connections 2024-04-16T17:07:10.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.233+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d56f62820 0x561d56f64c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.233+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d56eec4d0 0x561d56f60c60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.233+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d56edab30 0x561d56eefbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.233+0000 7f414c3ed740 1 --2- 172.21.15.18:0/2648596817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d56eda210 0x561d56eef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.233+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 >> 172.21.15.18:0/2648596817 conn(0x561d56ed4c90 msgr2=0x561d56ecac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.233+0000 7f4123e00640 0 client.26300 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:10.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.235+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 shutdown_connections 2024-04-16T17:07:10.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.236+0000 7f414c3ed740 1 -- 172.21.15.18:0/2648596817 wait complete. 2024-04-16T17:07:10.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.236+0000 7f414c3ed740 20 client.26300 trim_cache size 0 max 16384 2024-04-16T17:07:10.238 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 -- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41280a3da0 msgr2=0x7f41280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41280a3da0 0x7f41280a6260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f413c025000 tx=0x7f413c008000 comp rx=0 tx=0).stop 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 -- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4144074cd0 msgr2=0x7f41441a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4144074cd0 0x7f41441a0cd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4130036040 tx=0x7f413002cbc0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 -- 172.21.15.18:0/3037080268 shutdown_connections 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41280a3da0 0x7f41280a6260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414410e880 0x7f414419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4144074cd0 0x7f41441a0cd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 --2- 172.21.15.18:0/3037080268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41440743b0 0x7f41441a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 -- 172.21.15.18:0/3037080268 >> 172.21.15.18:0/3037080268 conn(0x7f41440fdab0 msgr2=0x7f41440ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 -- 172.21.15.18:0/3037080268 shutdown_connections 2024-04-16T17:07:10.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:10.238+0000 7f4149600640 1 -- 172.21.15.18:0/3037080268 wait complete. 2024-04-16T17:07:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.348+0000 7fb3f4c00640 1 -- 172.21.15.18:0/3540409003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0109540 msgr2=0x7fb3f0111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.348+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/3540409003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0109540 0x7fb3f0111110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009ca0 tx=0x7fb3e802c100 comp rx=0 tx=0).stop 2024-04-16T17:07:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.348+0000 7fb3f4c00640 1 -- 172.21.15.18:0/3540409003 shutdown_connections 2024-04-16T17:07:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.348+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/3540409003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0109540 0x7fb3f0111110 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.348+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/3540409003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3f0104e20 0x7fb3f0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.348+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/3540409003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3f0104470 0x7fb3f0104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.348+0000 7fb3f4c00640 1 -- 172.21.15.18:0/3540409003 >> 172.21.15.18:0/3540409003 conn(0x7fb3f00ffcf0 msgr2=0x7fb3f0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.349+0000 7fb3f4c00640 1 -- 172.21.15.18:0/3540409003 shutdown_connections 2024-04-16T17:07:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.349+0000 7fb3f4c00640 1 -- 172.21.15.18:0/3540409003 wait complete. 2024-04-16T17:07:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.350+0000 7fb3f4c00640 1 Processor -- start 2024-04-16T17:07:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.350+0000 7fb3f4c00640 1 -- start start 2024-04-16T17:07:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3f4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0104470 0x7fb3f0076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3f4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3f0109540 0x7fb3f0076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3f4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3f0076f90 0x7fb3f01ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3f4c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0114650 con 0x7fb3f0109540 2024-04-16T17:07:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3f4c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01144d0 con 0x7fb3f0104470 2024-04-16T17:07:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3f4c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01147d0 con 0x7fb3f0076f90 2024-04-16T17:07:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0104470 0x7fb3f0076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0104470 0x7fb3f0076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57994/0 (socket says 172.21.15.18:57994) 2024-04-16T17:07:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3ede00640 1 -- 172.21.15.18:0/2820089562 learned_addr learned my addr 172.21.15.18:0/2820089562 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3f0109540 0x7fb3f0076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.351+0000 7fb3ee800640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3f0076f90 0x7fb3f01ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3ede00640 1 -- 172.21.15.18:0/2820089562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3f0076f90 msgr2=0x7fb3f01ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3ede00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3f0076f90 0x7fb3f01ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3ede00640 1 -- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3f0109540 msgr2=0x7fb3f0076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3ede00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3f0109540 0x7fb3f0076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3ede00640 1 -- 172.21.15.18:0/2820089562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f01ab680 con 0x7fb3f0104470 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3ede00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0104470 0x7fb3f0076510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000b4a0 tx=0x7fb3e000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3e6a00640 1 -- 172.21.15.18:0/2820089562 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0014070 con 0x7fb3f0104470 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3f4c00640 1 -- 172.21.15.18:0/2820089562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f01ab910 con 0x7fb3f0104470 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3e6a00640 1 -- 172.21.15.18:0/2820089562 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e0010af0 con 0x7fb3f0104470 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.352+0000 7fb3e6a00640 1 -- 172.21.15.18:0/2820089562 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0017cc0 con 0x7fb3f0104470 2024-04-16T17:07:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.353+0000 7fb3f4c00640 1 -- 172.21.15.18:0/2820089562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f01abe80 con 0x7fb3f0104470 2024-04-16T17:07:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.356+0000 7fb3e6a00640 1 -- 172.21.15.18:0/2820089562 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb3e0017070 con 0x7fb3f0104470 2024-04-16T17:07:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.357+0000 7fb3e6a00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb3d40a3d90 0x7fb3d40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.357+0000 7fb3e6a00640 1 -- 172.21.15.18:0/2820089562 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb3e00cfd90 con 0x7fb3f0104470 2024-04-16T17:07:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.358+0000 7fb3f7a19740 1 Processor -- start 2024-04-16T17:07:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.358+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb3d40a3d90 0x7fb3d40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.358+0000 7fb3f7a19740 1 -- start start 2024-04-16T17:07:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.358+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f25360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.358+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f258a0 0x55cfe7f25d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.358+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f31210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7e8a780 con 0x55cfe7f258a0 2024-04-16T17:07:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7e30f40 con 0x55cfe7f2ee20 2024-04-16T17:07:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7e86740 con 0x55cfe7f26610 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ee800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f31210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ee800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f31210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58008/0 (socket says 172.21.15.18:58008) 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ee800640 1 -- 172.21.15.18:0/918242538 learned_addr learned my addr 172.21.15.18:0/918242538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ede00640 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f25360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb3d40a3d90 0x7fb3d40a6250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0077c80 tx=0x7fb3d800b390 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ee800640 1 -- 172.21.15.18:0/918242538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 msgr2=0x55cfe7f25360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f258a0 0x55cfe7f25d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ee800640 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f25360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ee800640 1 -- 172.21.15.18:0/918242538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f258a0 msgr2=0x55cfe7f25d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ee800640 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f258a0 0x55cfe7f25d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ee800640 1 -- 172.21.15.18:0/918242538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe7b9d210 con 0x55cfe7f2ee20 2024-04-16T17:07:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3ede00640 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f25360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.359+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f258a0 0x55cfe7f25d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.360+0000 7fb3ee800640 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f31210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009ca0 tx=0x7fb3e8009970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d0c1a32d6f0a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.360+0000 7fb3df000640 1 -- 172.21.15.18:0/918242538 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8043070 con 0x55cfe7f2ee20 2024-04-16T17:07:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.360+0000 7fb3df000640 1 -- 172.21.15.18:0/918242538 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e8003970 con 0x55cfe7f2ee20 2024-04-16T17:07:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.360+0000 7fb3df000640 1 -- 172.21.15.18:0/918242538 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8039040 con 0x55cfe7f2ee20 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.361+0000 7fb3f7a19740 1 -- 172.21.15.18:0/918242538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f2ee20 msgr2=0x55cfe7f31210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.361+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f31210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009ca0 tx=0x7fb3e8009970 comp rx=0 tx=0).stop 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.361+0000 7fb3f7a19740 1 -- 172.21.15.18:0/918242538 shutdown_connections 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.361+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f31210 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.361+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f258a0 0x55cfe7f25d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.361+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/918242538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f25360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.361+0000 7fb3f7a19740 1 -- 172.21.15.18:0/918242538 >> 172.21.15.18:0/918242538 conn(0x55cfe7ec05d0 msgr2=0x55cfe7f20590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.361+0000 7fb3f7a19740 1 -- 172.21.15.18:0/918242538 shutdown_connections 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.362+0000 7fb3f7a19740 1 -- 172.21.15.18:0/918242538 wait complete. 2024-04-16T17:07:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.362+0000 7fb3f7a19740 1 -- start start 2024-04-16T17:07:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.362+0000 7fb3f7a19740 1 Processor -- start 2024-04-16T17:07:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f258a0 0x55cfe7f2e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f2c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7f31af0 con 0x55cfe7f2ee20 2024-04-16T17:07:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7e44390 con 0x55cfe7f26610 2024-04-16T17:07:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7f31c70 con 0x55cfe7f258a0 2024-04-16T17:07:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3f7a19740 10 client.0 _ll_register_callbacks cb 0x55cfe7e2df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3ede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f258a0 0x55cfe7f2e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3ede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f258a0 0x55cfe7f2e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55262/0 (socket says 172.21.15.18:55262) 2024-04-16T17:07:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.363+0000 7fb3ede00640 1 -- 172.21.15.18:0/4116568193 learned_addr learned my addr 172.21.15.18:0/4116568193 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3ede00640 1 -- 172.21.15.18:0/4116568193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f26610 msgr2=0x55cfe7f2bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3ede00640 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3ede00640 1 -- 172.21.15.18:0/4116568193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 msgr2=0x55cfe7f2c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3ede00640 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f2c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3ede00640 1 -- 172.21.15.18:0/4116568193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe7c69620 con 0x55cfe7f258a0 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3ede00640 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f258a0 0x55cfe7f2e7d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000b470 tx=0x7fb3e0017420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=936739a0ee566e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3d3400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3d3400640 1 -- 172.21.15.18:0/4116568193 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e001e050 con 0x55cfe7f258a0 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.364+0000 7fb3d3400640 1 -- 172.21.15.18:0/4116568193 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb3e0098150 con 0x55cfe7f258a0 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.365+0000 7fb3f7a19740 1 -- 172.21.15.18:0/4116568193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e8003970 con 0x55cfe7f258a0 2024-04-16T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.365+0000 7fb3d3400640 1 -- 172.21.15.18:0/4116568193 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3e009fde0 con 0x55cfe7f258a0 2024-04-16T17:07:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.365+0000 7fb3f7a19740 10 client.26311 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.365+0000 7fb3f7a19740 1 -- 172.21.15.18:0/4116568193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cfe7f3dae0 con 0x55cfe7f258a0 2024-04-16T17:07:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.365+0000 7fb3d3400640 1 -- 172.21.15.18:0/4116568193 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e8003970 con 0x55cfe7f258a0 2024-04-16T17:07:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.366+0000 7fb3d3400640 1 -- 172.21.15.18:0/4116568193 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb3e0097830 con 0x55cfe7f258a0 2024-04-16T17:07:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.366+0000 7fb3f7a19740 10 client.26311 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.366+0000 7fb3f7a19740 10 client.26311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.366+0000 7fb3f7a19740 20 client.26311 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.366+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfe7f3ef50 0x55cfe7f413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.366+0000 7fb3f7a19740 4 client.26311 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.366+0000 7fb3f7a19740 1 -- 172.21.15.18:0/4116568193 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfe7e981a0 con 0x55cfe7f3ef50 2024-04-16T17:07:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.367+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfe7f3ef50 0x55cfe7f413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.368+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfe7f3ef50 0x55cfe7f413a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba081577e95775f server_cookie=d7ef574764dc3f27 in_seq=0 out_seq=0 2024-04-16T17:07:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.368+0000 7fb3d3400640 10 client.26311 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.368+0000 7fb3d3400640 1 -- 172.21.15.18:0/4116568193 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfe7e981a0 con 0x55cfe7f3ef50 2024-04-16T17:07:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.368+0000 7fb3d3400640 10 client.26311 handle_command_reply: tid=0 2024-04-16T17:07:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.368+0000 7fb3f7a19740 1 client.26311 shutdown 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 -- 172.21.15.18:0/4116568193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f258a0 msgr2=0x55cfe7f2e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f258a0 0x55cfe7f2e7d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000b470 tx=0x7fb3e0017420 comp rx=0 tx=0).stop 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 -- 172.21.15.18:0/4116568193 shutdown_connections 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfe7f3ef50 0x55cfe7f413a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f2c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/4116568193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f258a0 0x55cfe7f2e7d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3d3400640 0 client.26311 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 -- 172.21.15.18:0/4116568193 >> 172.21.15.18:0/4116568193 conn(0x55cfe7ec05d0 msgr2=0x55cfe7f222e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 -- 172.21.15.18:0/4116568193 shutdown_connections 2024-04-16T17:07:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 1 -- 172.21.15.18:0/4116568193 wait complete. 2024-04-16T17:07:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.369+0000 7fb3f7a19740 20 client.26311 trim_cache size 0 max 16384 2024-04-16T17:07:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.386+0000 7fb3f7a19740 1 Processor -- start 2024-04-16T17:07:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.386+0000 7fb3f7a19740 1 -- start start 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.386+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.386+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.386+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f27100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.386+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b8400 con 0x55cfe7f26610 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.386+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b5cd0 con 0x55cfe7f258a0 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.386+0000 7fb3f7a19740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b3b20 con 0x55cfe7f2ee20 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f3aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f3aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58030/0 (socket says 172.21.15.18:58030) 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 -- 172.21.15.18:0/1690433500 learned_addr learned my addr 172.21.15.18:0/1690433500 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ee800640 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f27100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 -- 172.21.15.18:0/1690433500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f2ee20 msgr2=0x55cfe7f27100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f27100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 -- 172.21.15.18:0/1690433500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f26610 msgr2=0x55cfe7f2bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 -- 172.21.15.18:0/1690433500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe7f41a20 con 0x55cfe7f258a0 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.387+0000 7fb3ede00640 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f3aa20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00a4170 tx=0x7fb3e0010eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7dc96e2a6fce278 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.388+0000 7fb3de600640 1 -- 172.21.15.18:0/1690433500 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e001e070 con 0x55cfe7f258a0 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.388+0000 7fb3de600640 1 -- 172.21.15.18:0/1690433500 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e00a46a0 con 0x55cfe7f258a0 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.388+0000 7fb3de600640 1 -- 172.21.15.18:0/1690433500 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00982b0 con 0x55cfe7f258a0 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.388+0000 7fb3f7a19740 1 -- 172.21.15.18:0/1690433500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 msgr2=0x55cfe7f3aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.388+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f3aa20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00a4170 tx=0x7fb3e0010eb0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 -- 172.21.15.18:0/1690433500 shutdown_connections 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f27100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f26610 0x55cfe7f2bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/1690433500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f3aa20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 -- 172.21.15.18:0/1690433500 >> 172.21.15.18:0/1690433500 conn(0x55cfe7ec05d0 msgr2=0x55cfe7f2b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 -- 172.21.15.18:0/1690433500 shutdown_connections 2024-04-16T17:07:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 -- 172.21.15.18:0/1690433500 wait complete. 2024-04-16T17:07:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 -- start start 2024-04-16T17:07:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.389+0000 7fb3f7a19740 1 Processor -- start 2024-04-16T17:07:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f36f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f37470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3f7a19740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f33f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f37470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f37470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55286/0 (socket says 172.21.15.18:55286) 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3e7e00640 1 -- 172.21.15.18:0/2421374674 learned_addr learned my addr 172.21.15.18:0/2421374674 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b4190 con 0x55cfe7f2ee20 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b5600 con 0x55cfe7f258a0 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b4860 con 0x55cfe7f26610 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3f7a19740 10 client.0 _ll_register_callbacks cb 0x7fb3e0017f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3ede00640 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f36f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3ee800640 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f33f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3e7e00640 1 -- 172.21.15.18:0/2421374674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 msgr2=0x55cfe7f36f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f36f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3e7e00640 1 -- 172.21.15.18:0/2421374674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 msgr2=0x55cfe7f33f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f33f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.390+0000 7fb3e7e00640 1 -- 172.21.15.18:0/2421374674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe7f24b70 con 0x55cfe7f26610 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.391+0000 7fb3e7e00640 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f37470 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8010e90 tx=0x7fb3d8013070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e25b86fa449115d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.391+0000 7fb3d3400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.391+0000 7fb3d3400640 1 -- 172.21.15.18:0/2421374674 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8026b50 con 0x55cfe7f26610 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.391+0000 7fb3d3400640 1 -- 172.21.15.18:0/2421374674 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb3d8026cf0 con 0x55cfe7f26610 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.391+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e00a46a0 con 0x55cfe7f26610 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.391+0000 7fb3d3400640 1 -- 172.21.15.18:0/2421374674 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3d8034050 con 0x55cfe7f26610 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.392+0000 7fb3f7a19740 10 client.26317 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.392+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cfe7fa5ce0 con 0x55cfe7f26610 2024-04-16T17:07:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.392+0000 7fb3d3400640 1 -- 172.21.15.18:0/2421374674 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e00a46a0 con 0x55cfe7f26610 2024-04-16T17:07:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.392+0000 7fb3d3400640 1 -- 172.21.15.18:0/2421374674 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb3d801b740 con 0x55cfe7f26610 2024-04-16T17:07:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.392+0000 7fb3f7a19740 10 client.26317 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.392+0000 7fb3f7a19740 10 client.26317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.392+0000 7fb3f7a19740 20 client.26317 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.392+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfe7fa6430 0x55cfe7fa8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.393+0000 7fb3f7a19740 4 client.26317 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:07:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.393+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfe7e92010 con 0x55cfe7fa6430 2024-04-16T17:07:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.393+0000 7fb3ede00640 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfe7fa6430 0x55cfe7fa8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.393+0000 7fb3ede00640 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfe7fa6430 0x55cfe7fa8850 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e030aa2bb6bfe2 server_cookie=93614314df7385ca in_seq=0 out_seq=0 2024-04-16T17:07:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.393+0000 7fb3d3400640 10 client.26317 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:10.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 63228 4.31 MB/sec warmup 83 sec latency 34.079 ms 2024-04-16T17:07:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.524+0000 7fb3d3400640 1 -- 172.21.15.18:0/2421374674 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cfe7e92010 con 0x55cfe7fa6430 2024-04-16T17:07:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.524+0000 7fb3d3400640 10 client.26317 handle_command_reply: tid=0 2024-04-16T17:07:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.525+0000 7fb3f7a19740 1 client.26317 shutdown 2024-04-16T17:07:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 msgr2=0x55cfe7f37470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f37470 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8010e90 tx=0x7fb3d8013070 comp rx=0 tx=0).stop 2024-04-16T17:07:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 shutdown_connections 2024-04-16T17:07:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfe7fa6430 0x55cfe7fa8850 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfe7f2ee20 0x55cfe7f33f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3d3400640 0 client.26317 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfe7f26610 0x55cfe7f37470 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 --2- 172.21.15.18:0/2421374674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfe7f258a0 0x55cfe7f36f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 >> 172.21.15.18:0/2421374674 conn(0x55cfe7ec05d0 msgr2=0x55cfe7f1ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.526+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 shutdown_connections 2024-04-16T17:07:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.527+0000 7fb3f7a19740 1 -- 172.21.15.18:0/2421374674 wait complete. 2024-04-16T17:07:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.529+0000 7fb3f7a19740 20 client.26317 trim_cache size 0 max 16384 2024-04-16T17:07:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:07:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:07:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:07:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:07:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 -- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb3d40a3d90 msgr2=0x7fb3d40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb3d40a3d90 0x7fb3d40a6250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0077c80 tx=0x7fb3d800b390 comp rx=0 tx=0).stop 2024-04-16T17:07:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 -- 172.21.15.18:0/2820089562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0104470 msgr2=0x7fb3f0076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0104470 0x7fb3f0076510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000b4a0 tx=0x7fb3e000b960 comp rx=0 tx=0).stop 2024-04-16T17:07:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 -- 172.21.15.18:0/2820089562 shutdown_connections 2024-04-16T17:07:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb3d40a3d90 0x7fb3d40a6250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3f0076f90 0x7fb3f01ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3f0109540 0x7fb3f0076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 --2- 172.21.15.18:0/2820089562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3f0104470 0x7fb3f0076510 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.530+0000 7fb3f4c00640 1 -- 172.21.15.18:0/2820089562 >> 172.21.15.18:0/2820089562 conn(0x7fb3f00ffcf0 msgr2=0x7fb3f0102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.531+0000 7fb3f4c00640 1 -- 172.21.15.18:0/2820089562 shutdown_connections 2024-04-16T17:07:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:10.531+0000 7fb3f4c00640 1 -- 172.21.15.18:0/2820089562 wait complete. 2024-04-16T17:07:11.269 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:07:11.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:07:11.269 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:07:11.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.454+0000 7fcbaea00640 1 -- 172.21.15.18:0/523434385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8102fe0 msgr2=0x7fcba81033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.454+0000 7fcbaea00640 1 --2- 172.21.15.18:0/523434385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8102fe0 0x7fcba81033c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009ca0 tx=0x7fcb9c02c190 comp rx=0 tx=0).stop 2024-04-16T17:07:11.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.454+0000 7fcbaea00640 1 -- 172.21.15.18:0/523434385 shutdown_connections 2024-04-16T17:07:11.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.454+0000 7fcbaea00640 1 --2- 172.21.15.18:0/523434385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba810c400 0x7fcba810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.454+0000 7fcbaea00640 1 --2- 172.21.15.18:0/523434385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba8103900 0x7fcba810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.454+0000 7fcbaea00640 1 --2- 172.21.15.18:0/523434385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8102fe0 0x7fcba81033c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.454+0000 7fcbaea00640 1 -- 172.21.15.18:0/523434385 >> 172.21.15.18:0/523434385 conn(0x7fcba8077ab0 msgr2=0x7fcba80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.454+0000 7fcbaea00640 1 -- 172.21.15.18:0/523434385 shutdown_connections 2024-04-16T17:07:11.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.455+0000 7fcbaea00640 1 -- 172.21.15.18:0/523434385 wait complete. 2024-04-16T17:07:11.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.455+0000 7fcbaea00640 1 Processor -- start 2024-04-16T17:07:11.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbaea00640 1 -- start start 2024-04-16T17:07:11.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba8102fe0 0x7fcba8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8103900 0x7fcba8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbaea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba810c400 0x7fcba819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbaea00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8112190 con 0x7fcba8102fe0 2024-04-16T17:07:11.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbaea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8112010 con 0x7fcba810c400 2024-04-16T17:07:11.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbaea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8112310 con 0x7fcba8103900 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbad600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba8102fe0 0x7fcba8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbacc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8103900 0x7fcba8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbacc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8103900 0x7fcba8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55312/0 (socket says 172.21.15.18:55312) 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbad600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba8102fe0 0x7fcba8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34130/0 (socket says 172.21.15.18:34130) 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbacc00640 1 -- 172.21.15.18:0/795167564 learned_addr learned my addr 172.21.15.18:0/795167564 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.456+0000 7fcbae000640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba810c400 0x7fcba819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcbacc00640 1 -- 172.21.15.18:0/795167564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba810c400 msgr2=0x7fcba819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcbacc00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba810c400 0x7fcba819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcbacc00640 1 -- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba8102fe0 msgr2=0x7fcba8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcbacc00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba8102fe0 0x7fcba8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcbacc00640 1 -- 172.21.15.18:0/795167564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba819d700 con 0x7fcba8103900 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcbacc00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8103900 0x7fcba8198c50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400e9f0 tx=0x7fcb9400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcba6000640 1 -- 172.21.15.18:0/795167564 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400cd80 con 0x7fcba8103900 2024-04-16T17:07:11.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcbaea00640 1 -- 172.21.15.18:0/795167564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba819d9f0 con 0x7fcba8103900 2024-04-16T17:07:11.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcba6000640 1 -- 172.21.15.18:0/795167564 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb94010910 con 0x7fcba8103900 2024-04-16T17:07:11.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.457+0000 7fcba6000640 1 -- 172.21.15.18:0/795167564 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940188f0 con 0x7fcba8103900 2024-04-16T17:07:11.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.458+0000 7fcbaea00640 1 -- 172.21.15.18:0/795167564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81a5280 con 0x7fcba8103900 2024-04-16T17:07:11.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.461+0000 7fcba6000640 1 -- 172.21.15.18:0/795167564 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcb94010ac0 con 0x7fcba8103900 2024-04-16T17:07:11.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.461+0000 7fcbb2d98740 1 Processor -- start 2024-04-16T17:07:11.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.461+0000 7fcbb2d98740 1 -- start start 2024-04-16T17:07:11.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fae90 0x55f0e54fb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fb7b0 0x55f0e5503e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 0x55f0e5506780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f0e545fcb0 con 0x55f0e54fae90 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f0e5406370 con 0x55f0e54fb7b0 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f0e545bc70 con 0x55f0e5504390 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbae000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fae90 0x55f0e54fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbae000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fae90 0x55f0e54fb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34142/0 (socket says 172.21.15.18:34142) 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbae000640 1 -- 172.21.15.18:0/2485483715 learned_addr learned my addr 172.21.15.18:0/2485483715 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.462+0000 7fcbad600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fb7b0 0x55f0e5503e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcbae000640 1 -- 172.21.15.18:0/2485483715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 msgr2=0x55f0e5506780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcbae000640 1 --2- 172.21.15.18:0/2485483715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 0x55f0e5506780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcbae000640 1 -- 172.21.15.18:0/2485483715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fb7b0 msgr2=0x55f0e5503e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcbae000640 1 --2- 172.21.15.18:0/2485483715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fb7b0 0x55f0e5503e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcbae000640 1 -- 172.21.15.18:0/2485483715 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e5425910 con 0x55f0e54fae90 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcba6000640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb900a3d20 0x7fcb900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcba6000640 1 -- 172.21.15.18:0/795167564 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fcb940d0fc0 con 0x7fcba8103900 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcbae000640 1 --2- 172.21.15.18:0/2485483715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fae90 0x55f0e54fb270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009c20 tx=0x7fcba0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ac1263d129e892 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcb9a600640 1 -- 172.21.15.18:0/2485483715 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba001e070 con 0x55f0e54fae90 2024-04-16T17:07:11.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcb9a600640 1 -- 172.21.15.18:0/2485483715 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba0019b10 con 0x55f0e54fae90 2024-04-16T17:07:11.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.463+0000 7fcb9a600640 1 -- 172.21.15.18:0/2485483715 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba000ea80 con 0x55f0e54fae90 2024-04-16T17:07:11.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.464+0000 7fcbad600640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb900a3d20 0x7fcb900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.464+0000 7fcbb2d98740 1 -- 172.21.15.18:0/2485483715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fae90 msgr2=0x55f0e54fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.464+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/2485483715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fae90 0x55f0e54fb270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009c20 tx=0x7fcba0011810 comp rx=0 tx=0).stop 2024-04-16T17:07:11.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.464+0000 7fcbb2d98740 1 -- 172.21.15.18:0/2485483715 shutdown_connections 2024-04-16T17:07:11.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.464+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/2485483715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 0x55f0e5506780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.464+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/2485483715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fb7b0 0x55f0e5503e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.464+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/2485483715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fae90 0x55f0e54fb270 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.464+0000 7fcbb2d98740 1 -- 172.21.15.18:0/2485483715 >> 172.21.15.18:0/2485483715 conn(0x55f0e54f55c0 msgr2=0x55f0e54f59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.465+0000 7fcbad600640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb900a3d20 0x7fcb900a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c004af0 tx=0x7fcb9c005c50 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.465+0000 7fcbb2d98740 1 -- 172.21.15.18:0/2485483715 shutdown_connections 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.465+0000 7fcbb2d98740 1 -- 172.21.15.18:0/2485483715 wait complete. 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.465+0000 7fcbb2d98740 1 -- start start 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.465+0000 7fcbb2d98740 1 Processor -- start 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fae90 0x55f0e55014b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 0x55f0e55019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e5504390 0x55f0e5501f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f0e5518190 con 0x55f0e54fb7b0 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f0e5419730 con 0x55f0e5504390 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f0e5518310 con 0x55f0e54fae90 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbb2d98740 10 client.0 _ll_register_callbacks cb 0x55f0e53e81e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:11.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbad600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fae90 0x55f0e55014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbad600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fae90 0x55f0e55014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55336/0 (socket says 172.21.15.18:55336) 2024-04-16T17:07:11.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbad600640 1 -- 172.21.15.18:0/3756917862 learned_addr learned my addr 172.21.15.18:0/3756917862 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbacc00640 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 0x55f0e55019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.466+0000 7fcbae000640 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e5504390 0x55f0e5501f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.467+0000 7fcbad600640 1 -- 172.21.15.18:0/3756917862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e5504390 msgr2=0x55f0e5501f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.467+0000 7fcbad600640 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e5504390 0x55f0e5501f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.467+0000 7fcbad600640 1 -- 172.21.15.18:0/3756917862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 msgr2=0x55f0e55019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.467+0000 7fcbad600640 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 0x55f0e55019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.467+0000 7fcbad600640 1 -- 172.21.15.18:0/3756917862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e523a7e0 con 0x55f0e54fae90 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.467+0000 7fcbad600640 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fae90 0x55f0e55014b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009ca0 tx=0x7fcb9c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7302a6bb9a18e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.467+0000 7fcb8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.468+0000 7fcb8ea00640 1 -- 172.21.15.18:0/3756917862 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c051020 con 0x55f0e54fae90 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.468+0000 7fcbb2d98740 1 -- 172.21.15.18:0/3756917862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba0019b10 con 0x55f0e54fae90 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.468+0000 7fcb8ea00640 1 -- 172.21.15.18:0/3756917862 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fcb9c004790 con 0x55f0e54fae90 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.468+0000 7fcb8ea00640 1 -- 172.21.15.18:0/3756917862 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcb9c05c050 con 0x55f0e54fae90 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.468+0000 7fcbb2d98740 10 client.26326 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.468+0000 7fcbb2d98740 1 -- 172.21.15.18:0/3756917862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f0e529af70 con 0x55f0e54fae90 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.468+0000 7fcb8ea00640 1 -- 172.21.15.18:0/3756917862 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba0019b10 con 0x55f0e54fae90 2024-04-16T17:07:11.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.469+0000 7fcb8ea00640 1 -- 172.21.15.18:0/3756917862 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcb9c04d6d0 con 0x55f0e54fae90 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.469+0000 7fcbb2d98740 10 client.26326 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.469+0000 7fcbb2d98740 10 client.26326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.469+0000 7fcbb2d98740 20 client.26326 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.469+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f0e5529bc0 0x55f0e552bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.469+0000 7fcbb2d98740 4 client.26326 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.469+0000 7fcbb2d98740 1 -- 172.21.15.18:0/3756917862 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0e546d6d0 con 0x55f0e5529bc0 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.469+0000 7fcbacc00640 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f0e5529bc0 0x55f0e552bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.470+0000 7fcbacc00640 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f0e5529bc0 0x55f0e552bfb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93126eedf21a3d33 server_cookie=76b1a912b8e15a7d in_seq=0 out_seq=0 2024-04-16T17:07:11.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.470+0000 7fcb8ea00640 10 client.26326 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.470+0000 7fcb8ea00640 1 -- 172.21.15.18:0/3756917862 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0e546d6d0 con 0x55f0e5529bc0 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.470+0000 7fcb8ea00640 10 client.26326 handle_command_reply: tid=0 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.470+0000 7fcbb2d98740 1 client.26326 shutdown 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.470+0000 7fcbb2d98740 1 -- 172.21.15.18:0/3756917862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fae90 msgr2=0x55f0e55014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.470+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fae90 0x55f0e55014b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009ca0 tx=0x7fcb9c0095c0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 1 -- 172.21.15.18:0/3756917862 shutdown_connections 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f0e5529bc0 0x55f0e552bfb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e5504390 0x55f0e5501f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 0x55f0e55019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcb8ea00640 0 client.26326 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:11.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/3756917862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fae90 0x55f0e55014b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 1 -- 172.21.15.18:0/3756917862 >> 172.21.15.18:0/3756917862 conn(0x55f0e54f55c0 msgr2=0x55f0e54f7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 1 -- 172.21.15.18:0/3756917862 shutdown_connections 2024-04-16T17:07:11.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 1 -- 172.21.15.18:0/3756917862 wait complete. 2024-04-16T17:07:11.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.471+0000 7fcbb2d98740 20 client.26326 trim_cache size 0 max 16384 2024-04-16T17:07:11.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.491+0000 7fcbb2d98740 1 Processor -- start 2024-04-16T17:07:11.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.491+0000 7fcbb2d98740 1 -- start start 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e54f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fb7b0 0x55f0e5501d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e5504390 0x55f0e5531880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81a7240 con 0x55f0e5504390 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcba80738e0 con 0x55f0e54fae90 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8071730 con 0x55f0e54fb7b0 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbad600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e54f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbae000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e5504390 0x55f0e5531880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbad600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e54f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58082/0 (socket says 172.21.15.18:58082) 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbad600640 1 -- 172.21.15.18:0/839889991 learned_addr learned my addr 172.21.15.18:0/839889991 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbacc00640 1 --2- 172.21.15.18:0/839889991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fb7b0 0x55f0e5501d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbad600640 1 -- 172.21.15.18:0/839889991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fb7b0 msgr2=0x55f0e5501d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbad600640 1 --2- 172.21.15.18:0/839889991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fb7b0 0x55f0e5501d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbad600640 1 -- 172.21.15.18:0/839889991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e5504390 msgr2=0x55f0e5531880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbad600640 1 --2- 172.21.15.18:0/839889991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e5504390 0x55f0e5531880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.492+0000 7fcbad600640 1 -- 172.21.15.18:0/839889991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e552c320 con 0x55f0e54fae90 2024-04-16T17:07:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.493+0000 7fcbad600640 1 --2- 172.21.15.18:0/839889991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e54f7100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c04daf0 tx=0x7fcb9c0325d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1dc082b81cc694c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.493+0000 7fcb99c00640 1 -- 172.21.15.18:0/839889991 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c051040 con 0x55f0e54fae90 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.493+0000 7fcb99c00640 1 -- 172.21.15.18:0/839889991 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c065070 con 0x55f0e54fae90 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.493+0000 7fcb99c00640 1 -- 172.21.15.18:0/839889991 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c039040 con 0x55f0e54fae90 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 -- 172.21.15.18:0/839889991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 msgr2=0x55f0e54f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/839889991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e54f7100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c04daf0 tx=0x7fcb9c0325d0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 -- 172.21.15.18:0/839889991 shutdown_connections 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/839889991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e5504390 0x55f0e5531880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/839889991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e54fb7b0 0x55f0e5501d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/839889991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e54f7100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 -- 172.21.15.18:0/839889991 >> 172.21.15.18:0/839889991 conn(0x55f0e54f55c0 msgr2=0x55f0e54f69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 -- 172.21.15.18:0/839889991 shutdown_connections 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.494+0000 7fcbb2d98740 1 -- 172.21.15.18:0/839889991 wait complete. 2024-04-16T17:07:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 1 -- start start 2024-04-16T17:07:11.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 1 Processor -- start 2024-04-16T17:07:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e5588430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 0x55f0e5588970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 0x55f0e551dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8071da0 con 0x55f0e54fb7b0 2024-04-16T17:07:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8073210 con 0x55f0e54fae90 2024-04-16T17:07:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8072470 con 0x55f0e5504390 2024-04-16T17:07:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.495+0000 7fcbb2d98740 10 client.0 _ll_register_callbacks cb 0x7fcb9c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e5588430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e5588430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58086/0 (socket says 172.21.15.18:58086) 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 -- 172.21.15.18:0/4227753856 learned_addr learned my addr 172.21.15.18:0/4227753856 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbacc00640 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 0x55f0e5588970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbae000640 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 0x55f0e551dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 -- 172.21.15.18:0/4227753856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 msgr2=0x55f0e551dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 0x55f0e551dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 -- 172.21.15.18:0/4227753856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 msgr2=0x55f0e5588970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 0x55f0e5588970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 -- 172.21.15.18:0/4227753856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e5132170 con 0x55f0e54fae90 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcbad600640 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e5588430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c02f020 tx=0x7fcb9c04efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4fb35c19d22f963 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.496+0000 7fcb8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.497+0000 7fcb8ea00640 1 -- 172.21.15.18:0/4227753856 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c039070 con 0x55f0e54fae90 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.497+0000 7fcb8ea00640 1 -- 172.21.15.18:0/4227753856 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fcb9c004410 con 0x55f0e54fae90 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.497+0000 7fcbb2d98740 1 -- 172.21.15.18:0/4227753856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9c065070 con 0x55f0e54fae90 2024-04-16T17:07:11.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.497+0000 7fcb8ea00640 1 -- 172.21.15.18:0/4227753856 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcb9c004610 con 0x55f0e54fae90 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.497+0000 7fcbb2d98740 10 client.26336 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.497+0000 7fcbb2d98740 1 -- 172.21.15.18:0/4227753856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f0e5590b20 con 0x55f0e54fae90 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.497+0000 7fcb8ea00640 1 -- 172.21.15.18:0/4227753856 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c065070 con 0x55f0e54fae90 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcb8ea00640 1 -- 172.21.15.18:0/4227753856 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcb9c034660 con 0x55f0e54fae90 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcbb2d98740 10 client.26336 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcbb2d98740 10 client.26336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcbb2d98740 20 client.26336 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f0e5591170 0x55f0e5593590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcbb2d98740 4 client.26336 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcbb2d98740 1 -- 172.21.15.18:0/4227753856 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f0e5467540 con 0x55f0e5591170 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcbacc00640 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f0e5591170 0x55f0e5593590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcbacc00640 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f0e5591170 0x55f0e5593590 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cbd3c2be47fb0ea server_cookie=d87037657ccc6257 in_seq=0 out_seq=0 2024-04-16T17:07:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.498+0000 7fcb8ea00640 10 client.26336 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:11.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.499+0000 7fcb8ea00640 1 -- 172.21.15.18:0/4227753856 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f0e5467540 con 0x55f0e5591170 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcb8ea00640 10 client.26336 handle_command_reply: tid=0 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 client.26336 shutdown 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 -- 172.21.15.18:0/4227753856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 msgr2=0x55f0e5588430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e5588430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c02f020 tx=0x7fcb9c04efa0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 -- 172.21.15.18:0/4227753856 shutdown_connections 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f0e5591170 0x55f0e5593590 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0e5504390 0x55f0e551dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f0e54fb7b0 0x55f0e5588970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 --2- 172.21.15.18:0/4227753856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f0e54fae90 0x55f0e5588430 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 -- 172.21.15.18:0/4227753856 >> 172.21.15.18:0/4227753856 conn(0x55f0e54f55c0 msgr2=0x55f0e54f7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcb8ea00640 0 client.26336 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 -- 172.21.15.18:0/4227753856 shutdown_connections 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 1 -- 172.21.15.18:0/4227753856 wait complete. 2024-04-16T17:07:11.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.500+0000 7fcbb2d98740 20 client.26336 trim_cache size 0 max 16384 2024-04-16T17:07:11.504 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:07:11.504 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:07:11.504 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "a869ec44-1d2a-4de7-8ed6-6ad3fcc90e30", 2024-04-16T17:07:11.504 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:07:11.504 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 -- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb900a3d20 msgr2=0x7fcb900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb900a3d20 0x7fcb900a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c004af0 tx=0x7fcb9c005c50 comp rx=0 tx=0).stop 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 -- 172.21.15.18:0/795167564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8103900 msgr2=0x7fcba8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8103900 0x7fcba8198c50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400e9f0 tx=0x7fcb9400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 -- 172.21.15.18:0/795167564 shutdown_connections 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb900a3d20 0x7fcb900a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba810c400 0x7fcba819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba8103900 0x7fcba8198c50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 --2- 172.21.15.18:0/795167564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba8102fe0 0x7fcba8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 -- 172.21.15.18:0/795167564 >> 172.21.15.18:0/795167564 conn(0x7fcba8077ab0 msgr2=0x7fcba80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 -- 172.21.15.18:0/795167564 shutdown_connections 2024-04-16T17:07:11.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.501+0000 7fcbaea00640 1 -- 172.21.15.18:0/795167564 wait complete. 2024-04-16T17:07:11.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 64312 4.35 MB/sec warmup 84 sec latency 19.065 ms 2024-04-16T17:07:11.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:07:11.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:07:11.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 -- 172.21.15.18:0/2863911493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 msgr2=0x7fec701110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 --2- 172.21.15.18:0/2863911493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 0x7fec701110f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fec68009ca0 tx=0x7fec6802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 -- 172.21.15.18:0/2863911493 shutdown_connections 2024-04-16T17:07:11.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 --2- 172.21.15.18:0/2863911493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 0x7fec701110f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 --2- 172.21.15.18:0/2863911493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec70104e00 0x7fec70108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 --2- 172.21.15.18:0/2863911493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec70104450 0x7fec70104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 -- 172.21.15.18:0/2863911493 >> 172.21.15.18:0/2863911493 conn(0x7fec700ffc90 msgr2=0x7fec701020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 -- 172.21.15.18:0/2863911493 shutdown_connections 2024-04-16T17:07:11.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.720+0000 7fec78200640 1 -- 172.21.15.18:0/2863911493 wait complete. 2024-04-16T17:07:11.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.721+0000 7fec78200640 1 Processor -- start 2024-04-16T17:07:11.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.722+0000 7fec78200640 1 -- start start 2024-04-16T17:07:11.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.722+0000 7fec78200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec70104450 0x7fec701a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.722+0000 7fec78200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 0x7fec701a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.722+0000 7fec78200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7019a8f0 0x7fec7019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.722+0000 7fec78200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fec701145d0 con 0x7fec70104450 2024-04-16T17:07:11.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.722+0000 7fec78200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fec70114450 con 0x7fec70109520 2024-04-16T17:07:11.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.722+0000 7fec78200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fec70114750 con 0x7fec7019a8f0 2024-04-16T17:07:11.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec75800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec70104450 0x7fec701a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec75800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec70104450 0x7fec701a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34174/0 (socket says 172.21.15.18:34174) 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec75800640 1 -- 172.21.15.18:0/251441116 learned_addr learned my addr 172.21.15.18:0/251441116 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec74e00640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 0x7fec701a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec76200640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7019a8f0 0x7fec7019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec74e00640 1 -- 172.21.15.18:0/251441116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7019a8f0 msgr2=0x7fec7019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec74e00640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7019a8f0 0x7fec7019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec74e00640 1 -- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec70104450 msgr2=0x7fec701a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec74e00640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec70104450 0x7fec701a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec74e00640 1 -- 172.21.15.18:0/251441116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7019b410 con 0x7fec70109520 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec76200640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7019a8f0 0x7fec7019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:11.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.723+0000 7fec74e00640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 0x7fec701a0c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00e9c0 tx=0x7fec5c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.724+0000 7fec6e000640 1 -- 172.21.15.18:0/251441116 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00ccc0 con 0x7fec70109520 2024-04-16T17:07:11.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.724+0000 7fec6e000640 1 -- 172.21.15.18:0/251441116 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5c005020 con 0x7fec70109520 2024-04-16T17:07:11.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.724+0000 7fec78200640 1 -- 172.21.15.18:0/251441116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7019b700 con 0x7fec70109520 2024-04-16T17:07:11.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.724+0000 7fec6e000640 1 -- 172.21.15.18:0/251441116 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004a00 con 0x7fec70109520 2024-04-16T17:07:11.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.724+0000 7fec78200640 1 -- 172.21.15.18:0/251441116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec701a7900 con 0x7fec70109520 2024-04-16T17:07:11.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec7af6a740 1 Processor -- start 2024-04-16T17:07:11.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec7af6a740 1 -- start start 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec6e000640 1 -- 172.21.15.18:0/251441116 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fec5c0051d0 con 0x7fec70109520 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff4b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 0x55846ff56aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec7af6a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55846feb1120 con 0x55846ff546b0 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec7af6a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55846fe57340 con 0x55846ff4bad0 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.729+0000 7fec7af6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55846fead0e0 con 0x55846ff4b1b0 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec75800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec76200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff4b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec76200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff4b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55378/0 (socket says 172.21.15.18:55378) 2024-04-16T17:07:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec76200640 1 -- 172.21.15.18:0/2625198470 learned_addr learned my addr 172.21.15.18:0/2625198470 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec75800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff54170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58114/0 (socket says 172.21.15.18:58114) 2024-04-16T17:07:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec74e00640 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 0x55846ff56aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec76200640 1 -- 172.21.15.18:0/2625198470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 msgr2=0x55846ff54170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec76200640 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff54170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec76200640 1 -- 172.21.15.18:0/2625198470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 msgr2=0x55846ff56aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec76200640 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 0x55846ff56aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec76200640 1 -- 172.21.15.18:0/2625198470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846fc8d7d0 con 0x55846ff4b1b0 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.730+0000 7fec75800640 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff54170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.731+0000 7fec76200640 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff4b590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fec68009ca0 tx=0x7fec68005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4755df27900507b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.731+0000 7fec62a00640 1 -- 172.21.15.18:0/2625198470 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68043020 con 0x55846ff4b1b0 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.731+0000 7fec62a00640 1 -- 172.21.15.18:0/2625198470 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6802cca0 con 0x55846ff4b1b0 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.731+0000 7fec62a00640 1 -- 172.21.15.18:0/2625198470 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68039040 con 0x55846ff4b1b0 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.731+0000 7fec6e000640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fec580a3d20 0x7fec580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec6e000640 1 -- 172.21.15.18:0/251441116 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fec5c0d0b40 con 0x7fec70109520 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec75800640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fec580a3d20 0x7fec580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec7af6a740 1 -- 172.21.15.18:0/2625198470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 msgr2=0x55846ff4b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec7af6a740 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff4b590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fec68009ca0 tx=0x7fec68005fa0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec7af6a740 1 -- 172.21.15.18:0/2625198470 shutdown_connections 2024-04-16T17:07:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec7af6a740 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 0x55846ff56aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec7af6a740 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff54170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec7af6a740 1 --2- 172.21.15.18:0/2625198470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff4b590 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec7af6a740 1 -- 172.21.15.18:0/2625198470 >> 172.21.15.18:0/2625198470 conn(0x55846ff458e0 msgr2=0x55846ff45cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.732+0000 7fec7af6a740 1 -- 172.21.15.18:0/2625198470 shutdown_connections 2024-04-16T17:07:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.733+0000 7fec7af6a740 1 -- 172.21.15.18:0/2625198470 wait complete. 2024-04-16T17:07:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.733+0000 7fec75800640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fec580a3d20 0x7fec580a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55846ff4ac80 tx=0x7fec64005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.733+0000 7fec7af6a740 1 -- start start 2024-04-16T17:07:11.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.733+0000 7fec7af6a740 1 Processor -- start 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff51790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff51cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 0x55846ff52210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec7af6a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55846ff57440 con 0x55846ff546b0 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec7af6a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55846fe6a710 con 0x55846ff4bad0 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec7af6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55846ff575c0 con 0x55846ff4b1b0 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec7af6a740 10 client.0 _ll_register_callbacks cb 0x55846fe65af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec74e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff51cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec74e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff51cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58116/0 (socket says 172.21.15.18:58116) 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec74e00640 1 -- 172.21.15.18:0/3185668091 learned_addr learned my addr 172.21.15.18:0/3185668091 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec75800640 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff51790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.734+0000 7fec76200640 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 0x55846ff52210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec74e00640 1 -- 172.21.15.18:0/3185668091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 msgr2=0x55846ff51790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec74e00640 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff51790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec74e00640 1 -- 172.21.15.18:0/3185668091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 msgr2=0x55846ff52210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec74e00640 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 0x55846ff52210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec74e00640 1 -- 172.21.15.18:0/3185668091 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846fbc33f0 con 0x55846ff4bad0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec74e00640 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff51cd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec5c004610 tx=0x7fec5c09c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb5469daf89b32f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec57400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec57400640 1 -- 172.21.15.18:0/3185668091 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c01c070 con 0x55846ff4bad0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec57400640 1 -- 172.21.15.18:0/3185668091 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fec5c09aa30 con 0x55846ff4bad0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec7af6a740 1 -- 172.21.15.18:0/3185668091 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec6802cca0 con 0x55846ff4bad0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec57400640 1 -- 172.21.15.18:0/3185668091 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec5c0ba050 con 0x55846ff4bad0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec7af6a740 10 client.26348 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.735+0000 7fec7af6a740 1 -- 172.21.15.18:0/3185668091 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55846fb830f0 con 0x55846ff4bad0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec57400640 1 -- 172.21.15.18:0/3185668091 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6802cca0 con 0x55846ff4bad0 2024-04-16T17:07:11.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec57400640 1 -- 172.21.15.18:0/3185668091 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec5c0a4f10 con 0x55846ff4bad0 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec7af6a740 10 client.26348 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec7af6a740 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec7af6a740 20 client.26348 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec7af6a740 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55846ff64af0 0x55846ff66ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec7af6a740 4 client.26348 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec7af6a740 1 -- 172.21.15.18:0/3185668091 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846febeb40 con 0x55846ff64af0 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.736+0000 7fec75800640 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55846ff64af0 0x55846ff66ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.737+0000 7fec75800640 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55846ff64af0 0x55846ff66ee0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2931aee954de77e7 server_cookie=d06c58c3b0c0d7b in_seq=0 out_seq=0 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.737+0000 7fec57400640 10 client.26348 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec57400640 1 -- 172.21.15.18:0/3185668091 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55846febeb40 con 0x55846ff64af0 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec57400640 10 client.26348 handle_command_reply: tid=0 2024-04-16T17:07:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 client.26348 shutdown 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 -- 172.21.15.18:0/3185668091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 msgr2=0x55846ff51cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff51cd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec5c004610 tx=0x7fec5c09c500 comp rx=0 tx=0).stop 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 -- 172.21.15.18:0/3185668091 shutdown_connections 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55846ff64af0 0x55846ff66ee0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff546b0 0x55846ff52210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff4bad0 0x55846ff51cd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 --2- 172.21.15.18:0/3185668091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff51790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec57400640 0 client.26348 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.738+0000 7fec7af6a740 1 -- 172.21.15.18:0/3185668091 >> 172.21.15.18:0/3185668091 conn(0x55846ff458e0 msgr2=0x55846ff47630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.739+0000 7fec7af6a740 1 -- 172.21.15.18:0/3185668091 shutdown_connections 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.739+0000 7fec7af6a740 1 -- 172.21.15.18:0/3185668091 wait complete. 2024-04-16T17:07:11.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.739+0000 7fec7af6a740 20 client.26348 trim_cache size 0 max 16384 2024-04-16T17:07:11.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:11 smithi043 ceph-mon[36901]: pgmap v76: 129 pgs: 129 active+clean; 31 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-04-16T17:07:11.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.758+0000 7fec7af6a740 1 Processor -- start 2024-04-16T17:07:11.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.758+0000 7fec7af6a740 1 -- start start 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4b1b0 0x55846ff53ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4bad0 0x55846ff506d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff60300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec7af6a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fec701afc10 con 0x55846ff4b1b0 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec7af6a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fec701ad4e0 con 0x55846ff546b0 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec7af6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fec701ab330 con 0x55846ff4bad0 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec76200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff60300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec74e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4bad0 0x55846ff506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec75800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4b1b0 0x55846ff53ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec74e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4bad0 0x55846ff506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55392/0 (socket says 172.21.15.18:55392) 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec76200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff60300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58124/0 (socket says 172.21.15.18:58124) 2024-04-16T17:07:11.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec74e00640 1 -- 172.21.15.18:0/610035985 learned_addr learned my addr 172.21.15.18:0/610035985 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec74e00640 1 -- 172.21.15.18:0/610035985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 msgr2=0x55846ff60300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec74e00640 1 --2- 172.21.15.18:0/610035985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff60300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec74e00640 1 -- 172.21.15.18:0/610035985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4b1b0 msgr2=0x55846ff53ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec74e00640 1 --2- 172.21.15.18:0/610035985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4b1b0 0x55846ff53ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.759+0000 7fec74e00640 1 -- 172.21.15.18:0/610035985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846ff45410 con 0x55846ff4bad0 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.760+0000 7fec74e00640 1 --2- 172.21.15.18:0/610035985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4bad0 0x55846ff506d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fec5c09c430 tx=0x7fec5c09c630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8479eeff308de624 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.760+0000 7fec62000640 1 -- 172.21.15.18:0/610035985 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c01c070 con 0x55846ff4bad0 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.760+0000 7fec62000640 1 -- 172.21.15.18:0/610035985 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5c098da0 con 0x55846ff4bad0 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.760+0000 7fec62000640 1 -- 172.21.15.18:0/610035985 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0a53a0 con 0x55846ff4bad0 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 -- 172.21.15.18:0/610035985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4bad0 msgr2=0x55846ff506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 --2- 172.21.15.18:0/610035985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4bad0 0x55846ff506d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fec5c09c430 tx=0x7fec5c09c630 comp rx=0 tx=0).stop 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 -- 172.21.15.18:0/610035985 shutdown_connections 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 --2- 172.21.15.18:0/610035985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff60300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 --2- 172.21.15.18:0/610035985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4bad0 0x55846ff506d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 --2- 172.21.15.18:0/610035985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4b1b0 0x55846ff53ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 -- 172.21.15.18:0/610035985 >> 172.21.15.18:0/610035985 conn(0x55846ff458e0 msgr2=0x55846ff448d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 -- 172.21.15.18:0/610035985 shutdown_connections 2024-04-16T17:07:11.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 -- 172.21.15.18:0/610035985 wait complete. 2024-04-16T17:07:11.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 -- start start 2024-04-16T17:07:11.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.761+0000 7fec7af6a740 1 Processor -- start 2024-04-16T17:07:11.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff58460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4bad0 0x55846ff589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec7af6a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec7af6a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fec701ab9a0 con 0x55846ff4bad0 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec7af6a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fec701ace10 con 0x55846ff546b0 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec7af6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fec701ac070 con 0x55846ff4b1b0 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec7af6a740 10 client.0 _ll_register_callbacks cb 0x7fec5c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec74e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4bad0 0x55846ff589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec75800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff58460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec74e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4bad0 0x55846ff589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34226/0 (socket says 172.21.15.18:34226) 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec74e00640 1 -- 172.21.15.18:0/4118217076 learned_addr learned my addr 172.21.15.18:0/4118217076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.762+0000 7fec76200640 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec75800640 1 -- 172.21.15.18:0/4118217076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 msgr2=0x55846ff58ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec75800640 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff58ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec75800640 1 -- 172.21.15.18:0/4118217076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4bad0 msgr2=0x55846ff589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec75800640 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4bad0 0x55846ff589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec75800640 1 -- 172.21.15.18:0/4118217076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846ffca230 con 0x55846ff4b1b0 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec74e00640 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4bad0 0x55846ff589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:11.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec75800640 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff58460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fec5c09a9e0 tx=0x7fec640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673762482d4ef2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:11.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec57400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:11.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec57400640 1 -- 172.21.15.18:0/4118217076 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6401de00 con 0x55846ff4b1b0 2024-04-16T17:07:11.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec7af6a740 1 -- 172.21.15.18:0/4118217076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5c098da0 con 0x55846ff4b1b0 2024-04-16T17:07:11.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.763+0000 7fec57400640 1 -- 172.21.15.18:0/4118217076 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fec64005020 con 0x55846ff4b1b0 2024-04-16T17:07:11.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec57400640 1 -- 172.21.15.18:0/4118217076 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec6403b050 con 0x55846ff4b1b0 2024-04-16T17:07:11.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec7af6a740 10 client.26350 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:11.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec7af6a740 1 -- 172.21.15.18:0/4118217076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55846ffcb580 con 0x55846ff4b1b0 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec57400640 1 -- 172.21.15.18:0/4118217076 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5c098da0 con 0x55846ff4b1b0 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec57400640 1 -- 172.21.15.18:0/4118217076 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec6402b050 con 0x55846ff4b1b0 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec7af6a740 10 client.26350 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec7af6a740 10 client.26350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec7af6a740 20 client.26350 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec7af6a740 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55846ffcbc50 0x55846ffce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec7af6a740 4 client.26350 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec7af6a740 1 -- 172.21.15.18:0/4118217076 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55846feb89b0 con 0x55846ffcbc50 2024-04-16T17:07:11.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.764+0000 7fec74e00640 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55846ffcbc50 0x55846ffce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:11.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.765+0000 7fec74e00640 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55846ffcbc50 0x55846ffce040 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91db9f5e0c0b8c54 server_cookie=a7414ba542980572 in_seq=0 out_seq=0 2024-04-16T17:07:11.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.765+0000 7fec57400640 10 client.26350 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.766+0000 7fec57400640 1 -- 172.21.15.18:0/4118217076 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55846feb89b0 con 0x55846ffcbc50 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.766+0000 7fec57400640 10 client.26350 handle_command_reply: tid=0 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 client.26350 shutdown 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 -- 172.21.15.18:0/4118217076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 msgr2=0x55846ff58460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff58460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fec5c09a9e0 tx=0x7fec640046c0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 -- 172.21.15.18:0/4118217076 shutdown_connections 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55846ffcbc50 0x55846ffce040 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55846ff546b0 0x55846ff58ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55846ff4bad0 0x55846ff589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 --2- 172.21.15.18:0/4118217076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55846ff4b1b0 0x55846ff58460 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec57400640 0 client.26350 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 -- 172.21.15.18:0/4118217076 >> 172.21.15.18:0/4118217076 conn(0x55846ff458e0 msgr2=0x55846ff51750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 -- 172.21.15.18:0/4118217076 shutdown_connections 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 1 -- 172.21.15.18:0/4118217076 wait complete. 2024-04-16T17:07:11.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.767+0000 7fec7af6a740 20 client.26350 trim_cache size 0 max 16384 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout: "a869ec44-1d2a-4de7-8ed6-6ad3fcc90e30": { 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "a869ec44-1d2a-4de7-8ed6-6ad3fcc90e30", 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:07:11.770 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.768+0000 7fec78200640 1 -- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fec580a3d20 msgr2=0x7fec580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.768+0000 7fec78200640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fec580a3d20 0x7fec580a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55846ff4ac80 tx=0x7fec64005f90 comp rx=0 tx=0).stop 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.768+0000 7fec78200640 1 -- 172.21.15.18:0/251441116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 msgr2=0x7fec701a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.768+0000 7fec78200640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 0x7fec701a0c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00e9c0 tx=0x7fec5c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.769+0000 7fec78200640 1 -- 172.21.15.18:0/251441116 shutdown_connections 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.769+0000 7fec78200640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fec580a3d20 0x7fec580a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.769+0000 7fec78200640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7019a8f0 0x7fec7019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.769+0000 7fec78200640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec70109520 0x7fec701a0c80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.769+0000 7fec78200640 1 --2- 172.21.15.18:0/251441116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec70104450 0x7fec701a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:11.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.769+0000 7fec78200640 1 -- 172.21.15.18:0/251441116 >> 172.21.15.18:0/251441116 conn(0x7fec700ffc90 msgr2=0x7fec70101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:11.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.769+0000 7fec78200640 1 -- 172.21.15.18:0/251441116 shutdown_connections 2024-04-16T17:07:11.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:11.769+0000 7fec78200640 1 -- 172.21.15.18:0/251441116 wait complete. 2024-04-16T17:07:11.790 INFO:tasks.cephfs.filesystem:scrub status for tag:a869ec44-1d2a-4de7-8ed6-6ad3fcc90e30 - {'path': '/', 'tag': 'a869ec44-1d2a-4de7-8ed6-6ad3fcc90e30', 'options': 'recursive,force'} 2024-04-16T17:07:11.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:11 smithi022 ceph-mon[36023]: pgmap v76: 129 pgs: 129 active+clean; 31 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-04-16T17:07:12.218 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:07 smithi018 ceph-mon[31776]: pgmap v74: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-04-16T17:07:12.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:09 smithi018 ceph-mon[31776]: pgmap v75: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-16T17:07:12.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:09 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/842014523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:12.219 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:11 smithi018 ceph-mon[31776]: pgmap v76: 129 pgs: 129 active+clean; 31 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-04-16T17:07:12.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 65272 4.35 MB/sec warmup 85 sec latency 30.867 ms 2024-04-16T17:07:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:12 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:07:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:12 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:12 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:07:12.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:12 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:12.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:12 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:07:12.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:12 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:12.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:12 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:07:12.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:12 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:13.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 65491 4.31 MB/sec warmup 86 sec latency 104.312 ms 2024-04-16T17:07:13.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:13 smithi043 ceph-mon[36901]: pgmap v77: 129 pgs: 129 active+clean; 32 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 851 KiB/s wr, 116 op/s 2024-04-16T17:07:13.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:13 smithi022 ceph-mon[36023]: pgmap v77: 129 pgs: 129 active+clean; 32 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 851 KiB/s wr, 116 op/s 2024-04-16T17:07:14.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:12 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:07:14.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:12 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:14.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:12 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:07:14.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:12 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:14.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:13 smithi018 ceph-mon[31776]: pgmap v77: 129 pgs: 129 active+clean; 32 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 851 KiB/s wr, 116 op/s 2024-04-16T17:07:14.518 INFO:tasks.workunit.client.0.smithi018.stdout: 1 66619 4.32 MB/sec warmup 87 sec latency 67.551 ms 2024-04-16T17:07:14.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:14 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:07:15.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:14 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:07:15.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 67725 4.36 MB/sec warmup 88 sec latency 30.922 ms 2024-04-16T17:07:15.926 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:15 smithi022 ceph-mon[36023]: pgmap v78: 129 pgs: 129 active+clean; 32 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 851 KiB/s wr, 116 op/s 2024-04-16T17:07:16.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:15 smithi043 ceph-mon[36901]: pgmap v78: 129 pgs: 129 active+clean; 32 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 851 KiB/s wr, 116 op/s 2024-04-16T17:07:16.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 68528 4.34 MB/sec warmup 89 sec latency 15.542 ms 2024-04-16T17:07:17.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 69828 4.35 MB/sec warmup 90 sec latency 12.664 ms 2024-04-16T17:07:17.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:17 smithi022 ceph-mon[36023]: pgmap v79: 129 pgs: 129 active+clean; 33 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-16T17:07:18.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:17 smithi043 ceph-mon[36901]: pgmap v79: 129 pgs: 129 active+clean; 33 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-16T17:07:18.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 70348 4.36 MB/sec warmup 91 sec latency 59.546 ms 2024-04-16T17:07:19.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 70752 4.33 MB/sec warmup 92 sec latency 89.310 ms 2024-04-16T17:07:19.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:19 smithi022 ceph-mon[36023]: pgmap v80: 129 pgs: 129 active+clean; 33 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1009 KiB/s wr, 144 op/s 2024-04-16T17:07:20.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:19 smithi043 ceph-mon[36901]: pgmap v80: 129 pgs: 129 active+clean; 33 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1009 KiB/s wr, 144 op/s 2024-04-16T17:07:20.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 71396 4.34 MB/sec warmup 93 sec latency 45.319 ms 2024-04-16T17:07:21.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 72469 4.34 MB/sec warmup 94 sec latency 29.231 ms 2024-04-16T17:07:21.897 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:14 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:07:21.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:15 smithi018 ceph-mon[31776]: pgmap v78: 129 pgs: 129 active+clean; 32 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 851 KiB/s wr, 116 op/s 2024-04-16T17:07:21.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:17 smithi018 ceph-mon[31776]: pgmap v79: 129 pgs: 129 active+clean; 33 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-16T17:07:21.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:19 smithi018 ceph-mon[31776]: pgmap v80: 129 pgs: 129 active+clean; 33 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1009 KiB/s wr, 144 op/s 2024-04-16T17:07:21.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:21 smithi022 ceph-mon[36023]: pgmap v81: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-16T17:07:22.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:21 smithi043 ceph-mon[36901]: pgmap v81: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-16T17:07:22.519 INFO:tasks.workunit.client.0.smithi018.stdout: 1 73866 4.36 MB/sec warmup 95 sec latency 20.122 ms 2024-04-16T17:07:23.525 INFO:tasks.workunit.client.0.smithi018.stdout: 1 74300 4.35 MB/sec warmup 96 sec latency 82.022 ms 2024-04-16T17:07:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 -- 172.21.15.18:0/2374191745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 msgr2=0x7fc0a4074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2374191745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 0x7fc0a4074810 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc098009900 tx=0x7fc09802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 -- 172.21.15.18:0/2374191745 shutdown_connections 2024-04-16T17:07:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2374191745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a410e660 0x7fc0a4110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2374191745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a4074d50 0x7fc0a410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2374191745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 0x7fc0a4074810 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 -- 172.21.15.18:0/2374191745 >> 172.21.15.18:0/2374191745 conn(0x7fc0a40fd910 msgr2=0x7fc0a40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 -- 172.21.15.18:0/2374191745 shutdown_connections 2024-04-16T17:07:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.677+0000 7fc0ab200640 1 -- 172.21.15.18:0/2374191745 wait complete. 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.678+0000 7fc0ab200640 1 Processor -- start 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.678+0000 7fc0ab200640 1 -- start start 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.678+0000 7fc0ab200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 0x7fc0a41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.678+0000 7fc0ab200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a4074d50 0x7fc0a41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.678+0000 7fc0ab200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a410e660 0x7fc0a419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.678+0000 7fc0ab200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4114440 con 0x7fc0a4074430 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.678+0000 7fc0ab200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41142c0 con 0x7fc0a410e660 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.678+0000 7fc0ab200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41145c0 con 0x7fc0a4074d50 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 0x7fc0a41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a4074d50 0x7fc0a41a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 0x7fc0a41a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33004/0 (socket says 172.21.15.18:33004) 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 -- 172.21.15.18:0/2505463176 learned_addr learned my addr 172.21.15.18:0/2505463176 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a4074d50 0x7fc0a41a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36772/0 (socket says 172.21.15.18:36772) 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0aa800640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a410e660 0x7fc0a419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 -- 172.21.15.18:0/2505463176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a4074d50 msgr2=0x7fc0a41a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a4074d50 0x7fc0a41a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 -- 172.21.15.18:0/2505463176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a410e660 msgr2=0x7fc0a419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a410e660 0x7fc0a419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 -- 172.21.15.18:0/2505463176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a419afe0 con 0x7fc0a4074430 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.679+0000 7fc0a9e00640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 0x7fc0a41a0660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc0980043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.680+0000 7fc09ea00640 1 -- 172.21.15.18:0/2505463176 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098042020 con 0x7fc0a4074430 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.680+0000 7fc09ea00640 1 -- 172.21.15.18:0/2505463176 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0980054e0 con 0x7fc0a4074430 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.680+0000 7fc0ab200640 1 -- 172.21.15.18:0/2505463176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a419b270 con 0x7fc0a4074430 2024-04-16T17:07:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.680+0000 7fc09ea00640 1 -- 172.21.15.18:0/2505463176 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098003b20 con 0x7fc0a4074430 2024-04-16T17:07:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.680+0000 7fc0ab200640 1 -- 172.21.15.18:0/2505463176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a41a75e0 con 0x7fc0a4074430 2024-04-16T17:07:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.684+0000 7fc0af451740 1 Processor -- start 2024-04-16T17:07:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0af451740 1 -- start start 2024-04-16T17:07:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc09ea00640 1 -- 172.21.15.18:0/2505463176 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc098045020 con 0x7fc0a4074430 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0af451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b47f0 0x562c772b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0af451740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0af451740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772bd000 0x562c772bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0af451740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562c77218780 con 0x562c772b3a80 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0af451740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562c771bef40 con 0x562c772bd000 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0af451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c77214740 con 0x562c772b47f0 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0a9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0a9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33018/0 (socket says 172.21.15.18:33018) 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0a9e00640 1 -- 172.21.15.18:0/328573798 learned_addr learned my addr 172.21.15.18:0/328573798 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.685+0000 7fc0aa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b47f0 0x562c772b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc0a9400640 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772bd000 0x562c772bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc0aa800640 1 -- 172.21.15.18:0/328573798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772bd000 msgr2=0x562c772bf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc0aa800640 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772bd000 0x562c772bf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc0aa800640 1 -- 172.21.15.18:0/328573798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 msgr2=0x562c772b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc0aa800640 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc0aa800640 1 -- 172.21.15.18:0/328573798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c76f2b210 con 0x562c772b47f0 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc0a9400640 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772bd000 0x562c772bf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc0aa800640 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b47f0 0x562c772b3540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009eb0 tx=0x7fc0a0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd121a1d6ab59c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.686+0000 7fc09ea00640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc08c0a3d00 0x7fc08c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.687+0000 7fc093000640 1 -- 172.21.15.18:0/328573798 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0016070 con 0x562c772b47f0 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.687+0000 7fc093000640 1 -- 172.21.15.18:0/328573798 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a0004d20 con 0x562c772b47f0 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.687+0000 7fc093000640 1 -- 172.21.15.18:0/328573798 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0012810 con 0x562c772b47f0 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.687+0000 7fc09ea00640 1 -- 172.21.15.18:0/2505463176 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc0980f27e0 con 0x7fc0a4074430 2024-04-16T17:07:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.687+0000 7fc0a9e00640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc08c0a3d00 0x7fc08c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 -- 172.21.15.18:0/328573798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b47f0 msgr2=0x562c772b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b47f0 0x562c772b3540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009eb0 tx=0x7fc0a0010740 comp rx=0 tx=0).stop 2024-04-16T17:07:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0a9e00640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc08c0a3d00 0x7fc08c0a61c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0980baeb0 tx=0x7fc0980f8550 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 -- 172.21.15.18:0/328573798 shutdown_connections 2024-04-16T17:07:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772bd000 0x562c772bf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 --2- 172.21.15.18:0/328573798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b47f0 0x562c772b3540 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 -- 172.21.15.18:0/328573798 >> 172.21.15.18:0/328573798 conn(0x562c7724e5d0 msgr2=0x562c772ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 -- 172.21.15.18:0/328573798 shutdown_connections 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 -- 172.21.15.18:0/328573798 wait complete. 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 -- start start 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.688+0000 7fc0af451740 1 Processor -- start 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0af451740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0af451740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0af451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 0x562c772ba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0af451740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562c772bfd30 con 0x562c772b3a80 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0af451740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562c771d2390 con 0x562c772b47f0 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0af451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c772bfeb0 con 0x562c772bd000 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0af451740 10 client.0 _ll_register_callbacks cb 0x562c771bbf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0aa800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0aa800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772bcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33034/0 (socket says 172.21.15.18:33034) 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0aa800640 1 -- 172.21.15.18:0/1604235661 learned_addr learned my addr 172.21.15.18:0/1604235661 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.689+0000 7fc0a9400640 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0aa800640 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 0x562c772ba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0a9400640 1 -- 172.21.15.18:0/1604235661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 msgr2=0x562c772ba590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0a9400640 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 0x562c772ba590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0a9400640 1 -- 172.21.15.18:0/1604235661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 msgr2=0x562c772bcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0a9400640 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772bcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0a9400640 1 -- 172.21.15.18:0/1604235661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c76ff7620 con 0x562c772b47f0 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0aa800640 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0a9400640 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772b9e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x562c772b4520 tx=0x7fc094012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8b782cd2d17da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc087400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc087400640 1 -- 172.21.15.18:0/1604235661 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094004050 con 0x562c772b47f0 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc0af451740 1 -- 172.21.15.18:0/1604235661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a0004d20 con 0x562c772b47f0 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.690+0000 7fc087400640 1 -- 172.21.15.18:0/1604235661 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc0940041f0 con 0x562c772b47f0 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc087400640 1 -- 172.21.15.18:0/1604235661 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc094025050 con 0x562c772b47f0 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc0af451740 10 client.26366 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc0af451740 1 -- 172.21.15.18:0/1604235661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562c772cbeb0 con 0x562c772b47f0 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc087400640 1 -- 172.21.15.18:0/1604235661 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a0004d20 con 0x562c772b47f0 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc087400640 1 -- 172.21.15.18:0/1604235661 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc094003a10 con 0x562c772b47f0 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc0af451740 10 client.26366 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc0af451740 10 client.26366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc0af451740 20 client.26366 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc0af451740 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562c772cd370 0x562c772cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc0af451740 4 client.26366 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.691+0000 7fc0af451740 1 -- 172.21.15.18:0/1604235661 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c772261a0 con 0x562c772cd370 2024-04-16T17:07:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.704+0000 7fc0aa800640 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562c772cd370 0x562c772cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.727+0000 7fc0aa800640 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562c772cd370 0x562c772cf760 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925f19c7f9b12d7f server_cookie=995ea83145705bac in_seq=0 out_seq=0 2024-04-16T17:07:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.727+0000 7fc087400640 10 client.26366 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.727+0000 7fc087400640 1 -- 172.21.15.18:0/1604235661 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c772261a0 con 0x562c772cd370 2024-04-16T17:07:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.727+0000 7fc087400640 10 client.26366 handle_command_reply: tid=0 2024-04-16T17:07:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 client.26366 shutdown 2024-04-16T17:07:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 -- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 msgr2=0x562c772b9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772b9e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x562c772b4520 tx=0x7fc094012b60 comp rx=0 tx=0).stop 2024-04-16T17:07:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 -- 172.21.15.18:0/1604235661 shutdown_connections 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562c772cd370 0x562c772cf760 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 0x562c772ba590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772b9e50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 --2- 172.21.15.18:0/1604235661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b3a80 0x562c772bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc087400640 0 client.26366 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 -- 172.21.15.18:0/1604235661 >> 172.21.15.18:0/1604235661 conn(0x562c7724e5d0 msgr2=0x562c772b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 -- 172.21.15.18:0/1604235661 shutdown_connections 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 1 -- 172.21.15.18:0/1604235661 wait complete. 2024-04-16T17:07:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.728+0000 7fc0af451740 20 client.26366 trim_cache size 0 max 16384 2024-04-16T17:07:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0af451740 1 Processor -- start 2024-04-16T17:07:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0af451740 1 -- start start 2024-04-16T17:07:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0af451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b3a80 0x562c772c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0af451740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0af451740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772bd000 0x562c772bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0af451740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41b3e10 con 0x562c772bd000 2024-04-16T17:07:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0af451740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41b16e0 con 0x562c772b47f0 2024-04-16T17:07:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0af451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41af530 con 0x562c772b3a80 2024-04-16T17:07:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0a9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.752+0000 7fc0a9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53276/0 (socket says 172.21.15.18:53276) 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0a9400640 1 -- 172.21.15.18:0/3587915849 learned_addr learned my addr 172.21.15.18:0/3587915849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0aa800640 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b3a80 0x562c772c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0aa800640 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772bd000 0x562c772bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0a9400640 1 -- 172.21.15.18:0/3587915849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b3a80 msgr2=0x562c772c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0a9400640 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b3a80 0x562c772c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0a9400640 1 -- 172.21.15.18:0/3587915849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772bd000 msgr2=0x562c772bcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0a9400640 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772bd000 0x562c772bcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0a9400640 1 -- 172.21.15.18:0/3587915849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c772a8c20 con 0x562c772b47f0 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.753+0000 7fc0a9400640 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772bc580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc094012420 tx=0x7fc094010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc9fc5dac8721792 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.754+0000 7fc092600640 1 -- 172.21.15.18:0/3587915849 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09400f8f0 con 0x562c772b47f0 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.754+0000 7fc092600640 1 -- 172.21.15.18:0/3587915849 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc09402e070 con 0x562c772b47f0 2024-04-16T17:07:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.754+0000 7fc092600640 1 -- 172.21.15.18:0/3587915849 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094003bd0 con 0x562c772b47f0 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.754+0000 7fc0af451740 1 -- 172.21.15.18:0/3587915849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 msgr2=0x562c772bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772bc580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc094012420 tx=0x7fc094010710 comp rx=0 tx=0).stop 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 -- 172.21.15.18:0/3587915849 shutdown_connections 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772bd000 0x562c772bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b47f0 0x562c772bc580 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 --2- 172.21.15.18:0/3587915849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772b3a80 0x562c772c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 -- 172.21.15.18:0/3587915849 >> 172.21.15.18:0/3587915849 conn(0x562c7724e5d0 msgr2=0x562c772ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 -- 172.21.15.18:0/3587915849 shutdown_connections 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 -- 172.21.15.18:0/3587915849 wait complete. 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 -- start start 2024-04-16T17:07:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.755+0000 7fc0af451740 1 Processor -- start 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0af451740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b3a80 0x562c772b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0af451740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b47f0 0x562c772b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0af451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 0x562c772b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0af451740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41afba0 con 0x562c772b47f0 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0af451740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41b1010 con 0x562c772b3a80 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0af451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41b0270 con 0x562c772bd000 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0af451740 10 client.0 _ll_register_callbacks cb 0x7fc094009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0a9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b47f0 0x562c772b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0aa800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b3a80 0x562c772b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0a9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b47f0 0x562c772b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33060/0 (socket says 172.21.15.18:33060) 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0a9400640 1 -- 172.21.15.18:0/3775773083 learned_addr learned my addr 172.21.15.18:0/3775773083 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.756+0000 7fc0aa800640 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 0x562c772b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0aa800640 1 -- 172.21.15.18:0/3775773083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 msgr2=0x562c772b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0aa800640 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 0x562c772b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0aa800640 1 -- 172.21.15.18:0/3775773083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b47f0 msgr2=0x562c772b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0aa800640 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b47f0 0x562c772b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0aa800640 1 -- 172.21.15.18:0/3775773083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c77332b60 con 0x562c772b3a80 2024-04-16T17:07:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0aa800640 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b3a80 0x562c772b6dc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00098b0 tx=0x7fc0a0005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d09753713b74581d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc087400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc087400640 1 -- 172.21.15.18:0/3775773083 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0016040 con 0x562c772b3a80 2024-04-16T17:07:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc087400640 1 -- 172.21.15.18:0/3775773083 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc0a0022040 con 0x562c772b3a80 2024-04-16T17:07:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0af451740 1 -- 172.21.15.18:0/3775773083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc09402e070 con 0x562c772b3a80 2024-04-16T17:07:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc087400640 1 -- 172.21.15.18:0/3775773083 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0a002c1d0 con 0x562c772b3a80 2024-04-16T17:07:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0af451740 10 client.26372 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.757+0000 7fc0af451740 1 -- 172.21.15.18:0/3775773083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562c77333e70 con 0x562c772b3a80 2024-04-16T17:07:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc087400640 1 -- 172.21.15.18:0/3775773083 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc09402e070 con 0x562c772b3a80 2024-04-16T17:07:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc087400640 1 -- 172.21.15.18:0/3775773083 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc0a000f040 con 0x562c772b3a80 2024-04-16T17:07:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc0af451740 10 client.26372 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc0af451740 10 client.26372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc0af451740 20 client.26372 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc0af451740 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562c773341b0 0x562c773365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc0af451740 4 client.26372 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:07:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc0af451740 1 -- 172.21.15.18:0/3775773083 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c77220010 con 0x562c773341b0 2024-04-16T17:07:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc0a9400640 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562c773341b0 0x562c773365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc0a9400640 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562c773341b0 0x562c773365a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e73ef5160fe7e3 server_cookie=6c064381ee5cc0a1 in_seq=0 out_seq=0 2024-04-16T17:07:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.758+0000 7fc087400640 10 client.26372 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.905+0000 7fc087400640 1 -- 172.21.15.18:0/3775773083 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c77220010 con 0x562c773341b0 2024-04-16T17:07:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.905+0000 7fc087400640 10 client.26372 handle_command_reply: tid=0 2024-04-16T17:07:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 client.26372 shutdown 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 -- 172.21.15.18:0/3775773083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b3a80 msgr2=0x562c772b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b3a80 0x562c772b6dc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00098b0 tx=0x7fc0a0005c40 comp rx=0 tx=0).stop 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 -- 172.21.15.18:0/3775773083 shutdown_connections 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562c773341b0 0x562c773365a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c772bd000 0x562c772b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562c772b47f0 0x562c772b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 --2- 172.21.15.18:0/3775773083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c772b3a80 0x562c772b6dc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 -- 172.21.15.18:0/3775773083 >> 172.21.15.18:0/3775773083 conn(0x562c7724e5d0 msgr2=0x562c772afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc087400640 0 client.26372 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 -- 172.21.15.18:0/3775773083 shutdown_connections 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 1 -- 172.21.15.18:0/3775773083 wait complete. 2024-04-16T17:07:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.906+0000 7fc0af451740 20 client.26372 trim_cache size 0 max 16384 2024-04-16T17:07:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:07:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:07:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:07:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 -- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc08c0a3d00 msgr2=0x7fc08c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc08c0a3d00 0x7fc08c0a61c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0980baeb0 tx=0x7fc0980f8550 comp rx=0 tx=0).stop 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 -- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 msgr2=0x7fc0a41a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 0x7fc0a41a0660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc0980043c0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 -- 172.21.15.18:0/2505463176 shutdown_connections 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc08c0a3d00 0x7fc08c0a61c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a410e660 0x7fc0a419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a4074d50 0x7fc0a41a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 --2- 172.21.15.18:0/2505463176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0a4074430 0x7fc0a41a0660 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.907+0000 7fc0ab200640 1 -- 172.21.15.18:0/2505463176 >> 172.21.15.18:0/2505463176 conn(0x7fc0a40fd910 msgr2=0x7fc0a40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.908+0000 7fc0ab200640 1 -- 172.21.15.18:0/2505463176 shutdown_connections 2024-04-16T17:07:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:23.908+0000 7fc0ab200640 1 -- 172.21.15.18:0/2505463176 wait complete. 2024-04-16T17:07:23.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:23 smithi022 ceph-mon[36023]: pgmap v82: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-16T17:07:24.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:23 smithi043 ceph-mon[36901]: pgmap v82: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-16T17:07:24.525 INFO:tasks.workunit.client.0.smithi018.stdout: 1 74686 4.34 MB/sec warmup 97 sec latency 75.052 ms 2024-04-16T17:07:25.526 INFO:tasks.workunit.client.0.smithi018.stdout: 1 75110 4.31 MB/sec warmup 98 sec latency 67.320 ms 2024-04-16T17:07:25.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:21 smithi018 ceph-mon[31776]: pgmap v81: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-16T17:07:25.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:23 smithi018 ceph-mon[31776]: pgmap v82: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-16T17:07:25.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:25 smithi018 ceph-mon[31776]: pgmap v83: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-04-16T17:07:25.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:25 smithi022 ceph-mon[36023]: pgmap v83: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-04-16T17:07:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:25 smithi043 ceph-mon[36901]: pgmap v83: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-04-16T17:07:26.526 INFO:tasks.workunit.client.0.smithi018.stdout: 1 76062 4.32 MB/sec warmup 99 sec latency 79.237 ms 2024-04-16T17:07:27.526 INFO:tasks.workunit.client.0.smithi018.stdout: 1 77322 4.33 MB/sec warmup 100 sec latency 12.461 ms 2024-04-16T17:07:27.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:27 smithi022 ceph-mon[36023]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-04-16T17:07:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:27 smithi043 ceph-mon[36901]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-04-16T17:07:28.533 INFO:tasks.workunit.client.0.smithi018.stdout: 1 77860 4.33 MB/sec warmup 101 sec latency 59.764 ms 2024-04-16T17:07:29.314 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:27 smithi018 ceph-mon[31776]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-04-16T17:07:29.533 INFO:tasks.workunit.client.0.smithi018.stdout: 1 78236 4.32 MB/sec warmup 102 sec latency 60.072 ms 2024-04-16T17:07:29.804 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:29 smithi043 ceph-mon[36901]: pgmap v85: 129 pgs: 129 active+clean; 32 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:07:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:29 smithi022 ceph-mon[36023]: pgmap v85: 129 pgs: 129 active+clean; 32 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:07:30.533 INFO:tasks.workunit.client.0.smithi018.stdout: 1 78803 4.29 MB/sec warmup 103 sec latency 31.452 ms 2024-04-16T17:07:31.533 INFO:tasks.workunit.client.0.smithi018.stdout: 1 79930 4.30 MB/sec warmup 104 sec latency 26.204 ms 2024-04-16T17:07:31.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:31 smithi022 ceph-mon[36023]: pgmap v86: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:07:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:31 smithi043 ceph-mon[36901]: pgmap v86: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:07:32.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 81260 4.33 MB/sec warmup 105 sec latency 20.082 ms 2024-04-16T17:07:32.705 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:29 smithi018 ceph-mon[31776]: pgmap v85: 129 pgs: 129 active+clean; 32 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:07:32.706 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:31 smithi018 ceph-mon[31776]: pgmap v86: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:07:33.533 INFO:tasks.workunit.client.0.smithi018.stdout: 1 81758 4.33 MB/sec warmup 106 sec latency 67.262 ms 2024-04-16T17:07:33.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:33 smithi022 ceph-mon[36023]: pgmap v87: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-04-16T17:07:34.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:33 smithi043 ceph-mon[36901]: pgmap v87: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-04-16T17:07:34.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 81988 4.31 MB/sec warmup 107 sec latency 91.222 ms 2024-04-16T17:07:35.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:33 smithi018 ceph-mon[31776]: pgmap v87: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-04-16T17:07:35.533 INFO:tasks.workunit.client.0.smithi018.stdout: 1 82595 4.27 MB/sec warmup 108 sec latency 40.272 ms 2024-04-16T17:07:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:35 smithi022 ceph-mon[36023]: pgmap v88: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:07:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:35 smithi043 ceph-mon[36901]: pgmap v88: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:07:36.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 83823 4.28 MB/sec warmup 109 sec latency 17.572 ms 2024-04-16T17:07:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 -- 172.21.15.18:0/4092974727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 msgr2=0x7f8064102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 --2- 172.21.15.18:0/4092974727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 0x7f8064102460 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8054009900 tx=0x7f805402bc40 comp rx=0 tx=0).stop 2024-04-16T17:07:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 -- 172.21.15.18:0/4092974727 shutdown_connections 2024-04-16T17:07:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 --2- 172.21.15.18:0/4092974727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8064107150 0x7f806410ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 --2- 172.21.15.18:0/4092974727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8064102a30 0x7f8064106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 --2- 172.21.15.18:0/4092974727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 0x7f8064102460 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 -- 172.21.15.18:0/4092974727 >> 172.21.15.18:0/4092974727 conn(0x7f80640fd940 msgr2=0x7f80640ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 -- 172.21.15.18:0/4092974727 shutdown_connections 2024-04-16T17:07:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.063+0000 7f8068e00640 1 -- 172.21.15.18:0/4092974727 wait complete. 2024-04-16T17:07:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.065+0000 7f8068e00640 1 Processor -- start 2024-04-16T17:07:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.065+0000 7f8068e00640 1 -- start start 2024-04-16T17:07:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.065+0000 7f8068e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 0x7f806410dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.065+0000 7f8068e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8064107150 0x7f806410e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.065+0000 7f8068e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80641a3630 0x7f80641a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.065+0000 7f8068e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8064112230 con 0x7f8064107150 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.065+0000 7f8068e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80641120b0 con 0x7f80641a3630 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.065+0000 7f8068e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80641123b0 con 0x7f8064102080 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 0x7f806410dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 0x7f806410dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48138/0 (socket says 172.21.15.18:48138) 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063400640 1 -- 172.21.15.18:0/518140588 learned_addr learned my addr 172.21.15.18:0/518140588 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f805be00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8064107150 0x7f806410e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80641a3630 0x7f80641a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063400640 1 -- 172.21.15.18:0/518140588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80641a3630 msgr2=0x7f80641a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063400640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80641a3630 0x7f80641a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063400640 1 -- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8064107150 msgr2=0x7f806410e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063400640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8064107150 0x7f806410e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063400640 1 -- 172.21.15.18:0/518140588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80641a4120 con 0x7f8064102080 2024-04-16T17:07:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.066+0000 7f8063e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80641a3630 0x7f80641a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.067+0000 7f8063400640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 0x7f806410dfc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8054005ab0 tx=0x7f80540043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.067+0000 7f8060c00640 1 -- 172.21.15.18:0/518140588 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054041070 con 0x7f8064102080 2024-04-16T17:07:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.067+0000 7f8068e00640 1 -- 172.21.15.18:0/518140588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80641a43b0 con 0x7f8064102080 2024-04-16T17:07:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.067+0000 7f8060c00640 1 -- 172.21.15.18:0/518140588 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805402cca0 con 0x7f8064102080 2024-04-16T17:07:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.067+0000 7f8060c00640 1 -- 172.21.15.18:0/518140588 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054034440 con 0x7f8064102080 2024-04-16T17:07:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.067+0000 7f8068e00640 1 -- 172.21.15.18:0/518140588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80641abbd0 con 0x7f8064102080 2024-04-16T17:07:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.071+0000 7f806d20a740 1 Processor -- start 2024-04-16T17:07:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.071+0000 7f8060c00640 1 -- 172.21.15.18:0/518140588 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8054044070 con 0x7f8064102080 2024-04-16T17:07:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.071+0000 7f806d20a740 1 -- start start 2024-04-16T17:07:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.071+0000 7f806d20a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73b99540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.071+0000 7f806d20a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73b99a80 0x55fb73b99f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f806d20a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73ba3000 0x55fb73ba53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f806d20a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fb73afe780 con 0x55fb73b9a7f0 2024-04-16T17:07:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f806d20a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb73aa4f40 con 0x55fb73b99a80 2024-04-16T17:07:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f806d20a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb73afa740 con 0x55fb73ba3000 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f8063400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73b99a80 0x55fb73b99f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f8063e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73b99540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f8063400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73b99a80 0x55fb73b99f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33932/0 (socket says 172.21.15.18:33932) 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f8063400640 1 -- 172.21.15.18:0/1798203960 learned_addr learned my addr 172.21.15.18:0/1798203960 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f805be00640 1 --2- 172.21.15.18:0/1798203960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73ba3000 0x55fb73ba53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f8063400640 1 -- 172.21.15.18:0/1798203960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73ba3000 msgr2=0x55fb73ba53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.072+0000 7f8063400640 1 --2- 172.21.15.18:0/1798203960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73ba3000 0x55fb73ba53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8063400640 1 -- 172.21.15.18:0/1798203960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 msgr2=0x55fb73b99540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8063400640 1 --2- 172.21.15.18:0/1798203960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73b99540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8063400640 1 -- 172.21.15.18:0/1798203960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb73811210 con 0x55fb73b99a80 2024-04-16T17:07:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8063400640 1 --2- 172.21.15.18:0/1798203960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73b99a80 0x55fb73b99f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f80540efa80 tx=0x7f80540f8b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0b380a8eebd74e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8060c00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80480a3d00 0x7f80480a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8058c00640 1 -- 172.21.15.18:0/1798203960 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054101070 con 0x55fb73b99a80 2024-04-16T17:07:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8058c00640 1 -- 172.21.15.18:0/1798203960 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8054004c80 con 0x55fb73b99a80 2024-04-16T17:07:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8058c00640 1 -- 172.21.15.18:0/1798203960 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80540bdcf0 con 0x55fb73b99a80 2024-04-16T17:07:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.073+0000 7f8060c00640 1 -- 172.21.15.18:0/518140588 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f80540f26f0 con 0x7f8064102080 2024-04-16T17:07:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.074+0000 7f8063e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80480a3d00 0x7f80480a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.074+0000 7f806d20a740 1 -- 172.21.15.18:0/1798203960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73b99a80 msgr2=0x55fb73b99f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.074+0000 7f806d20a740 1 --2- 172.21.15.18:0/1798203960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73b99a80 0x55fb73b99f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f80540efa80 tx=0x7f80540f8b20 comp rx=0 tx=0).stop 2024-04-16T17:07:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.074+0000 7f806d20a740 1 -- 172.21.15.18:0/1798203960 shutdown_connections 2024-04-16T17:07:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.074+0000 7f806d20a740 1 --2- 172.21.15.18:0/1798203960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73ba3000 0x55fb73ba53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.074+0000 7f806d20a740 1 --2- 172.21.15.18:0/1798203960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73b99a80 0x55fb73b99f00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.074+0000 7f806d20a740 1 --2- 172.21.15.18:0/1798203960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73b99540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.074+0000 7f806d20a740 1 -- 172.21.15.18:0/1798203960 >> 172.21.15.18:0/1798203960 conn(0x55fb73b345d0 msgr2=0x55fb73b94770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.075+0000 7f806d20a740 1 -- 172.21.15.18:0/1798203960 shutdown_connections 2024-04-16T17:07:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.075+0000 7f806d20a740 1 -- 172.21.15.18:0/1798203960 wait complete. 2024-04-16T17:07:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.075+0000 7f8063e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80480a3d00 0x7f80480a61c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f805c00af00 tx=0x7f805c00a6b0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.075+0000 7f806d20a740 1 -- start start 2024-04-16T17:07:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f806d20a740 1 Processor -- start 2024-04-16T17:07:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f806d20a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 0x55fb73ba2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f806d20a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f806d20a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f806d20a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fb73ba5af0 con 0x55fb73b99a80 2024-04-16T17:07:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f806d20a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb73ab8390 con 0x55fb73ba3000 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f806d20a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb73ba5c70 con 0x55fb73b9a7f0 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f806d20a740 10 client.0 _ll_register_callbacks cb 0x55fb73aa1f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f805be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f8063e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f805be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48152/0 (socket says 172.21.15.18:48152) 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.076+0000 7f805be00640 1 -- 172.21.15.18:0/550885613 learned_addr learned my addr 172.21.15.18:0/550885613 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8063400640 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 0x55fb73ba2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8063e00640 1 -- 172.21.15.18:0/550885613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 msgr2=0x55fb73b9fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8063e00640 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8063e00640 1 -- 172.21.15.18:0/550885613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 msgr2=0x55fb73ba2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8063e00640 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 0x55fb73ba2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8063e00640 1 -- 172.21.15.18:0/550885613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb738dd620 con 0x55fb73ba3000 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8063e00640 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba03b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f805c00ed60 tx=0x7f805c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a53c1793838df58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8051600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8051600640 1 -- 172.21.15.18:0/550885613 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c01cd30 con 0x55fb73ba3000 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.077+0000 7f8051600640 1 -- 172.21.15.18:0/550885613 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f805c028480 con 0x55fb73ba3000 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.078+0000 7f806d20a740 1 -- 172.21.15.18:0/550885613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8054004c80 con 0x55fb73ba3000 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.078+0000 7f8051600640 1 -- 172.21.15.18:0/550885613 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f805c031050 con 0x55fb73ba3000 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.078+0000 7f806d20a740 10 client.26384 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.078+0000 7f806d20a740 1 -- 172.21.15.18:0/550885613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb73bb1b40 con 0x55fb73ba3000 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.078+0000 7f8051600640 1 -- 172.21.15.18:0/550885613 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8054004c80 con 0x55fb73ba3000 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.079+0000 7f8051600640 1 -- 172.21.15.18:0/550885613 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f805c01d810 con 0x55fb73ba3000 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.079+0000 7f806d20a740 10 client.26384 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.079+0000 7f806d20a740 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.079+0000 7f806d20a740 20 client.26384 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.079+0000 7f806d20a740 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb73bb3050 0x55fb73bb5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.079+0000 7f806d20a740 4 client.26384 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.079+0000 7f806d20a740 1 -- 172.21.15.18:0/550885613 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb73b0c1a0 con 0x55fb73bb3050 2024-04-16T17:07:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.079+0000 7f8063400640 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb73bb3050 0x55fb73bb5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.080+0000 7f8063400640 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb73bb3050 0x55fb73bb5460 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853ad8159f97caeb server_cookie=8c0376c43ec8fbd1 in_seq=0 out_seq=0 2024-04-16T17:07:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.080+0000 7f8051600640 10 client.26384 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f8051600640 1 -- 172.21.15.18:0/550885613 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb73b0c1a0 con 0x55fb73bb3050 2024-04-16T17:07:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f8051600640 10 client.26384 handle_command_reply: tid=0 2024-04-16T17:07:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 client.26384 shutdown 2024-04-16T17:07:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 -- 172.21.15.18:0/550885613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 msgr2=0x55fb73ba03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba03b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f805c00ed60 tx=0x7f805c004a40 comp rx=0 tx=0).stop 2024-04-16T17:07:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 -- 172.21.15.18:0/550885613 shutdown_connections 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb73bb3050 0x55fb73bb5460 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba03b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 --2- 172.21.15.18:0/550885613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 0x55fb73ba2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 -- 172.21.15.18:0/550885613 >> 172.21.15.18:0/550885613 conn(0x55fb73b345d0 msgr2=0x55fb73b964c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f8051600640 0 client.26384 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 -- 172.21.15.18:0/550885613 shutdown_connections 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 1 -- 172.21.15.18:0/550885613 wait complete. 2024-04-16T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.081+0000 7f806d20a740 20 client.26384 trim_cache size 0 max 16384 2024-04-16T17:07:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f806d20a740 1 Processor -- start 2024-04-16T17:07:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f806d20a740 1 -- start start 2024-04-16T17:07:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f806d20a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b99a80 0x55fb73bae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f806d20a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73ba24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f806d20a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f806d20a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80641b82e0 con 0x55fb73b9a7f0 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f806d20a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80641b5bb0 con 0x55fb73ba3000 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f806d20a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80641b3a00 con 0x55fb73b99a80 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f8063400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b99a80 0x55fb73bae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f805be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73ba24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f8063e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f8063400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b99a80 0x55fb73bae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48160/0 (socket says 172.21.15.18:48160) 2024-04-16T17:07:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.099+0000 7f8063400640 1 -- 172.21.15.18:0/1523377778 learned_addr learned my addr 172.21.15.18:0/1523377778 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.100+0000 7f8063e00640 1 -- 172.21.15.18:0/1523377778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b99a80 msgr2=0x55fb73bae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.100+0000 7f8063e00640 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b99a80 0x55fb73bae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.100+0000 7f8063e00640 1 -- 172.21.15.18:0/1523377778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 msgr2=0x55fb73ba24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.100+0000 7f8063e00640 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73ba24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.100+0000 7f8063e00640 1 -- 172.21.15.18:0/1523377778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb73b98d50 con 0x55fb73ba3000 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.100+0000 7f8063400640 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b99a80 0x55fb73bae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.100+0000 7f805be00640 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73ba24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.100+0000 7f8063e00640 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba2a10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f805c011ca0 tx=0x7f805c00a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fde7e6791023198 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.101+0000 7f8053e00640 1 -- 172.21.15.18:0/1523377778 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c01c770 con 0x55fb73ba3000 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.101+0000 7f8053e00640 1 -- 172.21.15.18:0/1523377778 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805c004ab0 con 0x55fb73ba3000 2024-04-16T17:07:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.101+0000 7f8053e00640 1 -- 172.21.15.18:0/1523377778 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c005040 con 0x55fb73ba3000 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.101+0000 7f806d20a740 1 -- 172.21.15.18:0/1523377778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 msgr2=0x55fb73ba2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.101+0000 7f806d20a740 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba2a10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f805c011ca0 tx=0x7f805c00a8d0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.102+0000 7f806d20a740 1 -- 172.21.15.18:0/1523377778 shutdown_connections 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.102+0000 7f806d20a740 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73ba2a10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.102+0000 7f806d20a740 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b9a7f0 0x55fb73ba24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.102+0000 7f806d20a740 1 --2- 172.21.15.18:0/1523377778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b99a80 0x55fb73bae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.102+0000 7f806d20a740 1 -- 172.21.15.18:0/1523377778 >> 172.21.15.18:0/1523377778 conn(0x55fb73b345d0 msgr2=0x55fb73b942a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.102+0000 7f806d20a740 1 -- 172.21.15.18:0/1523377778 shutdown_connections 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.102+0000 7f806d20a740 1 -- 172.21.15.18:0/1523377778 wait complete. 2024-04-16T17:07:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.102+0000 7f806d20a740 1 -- start start 2024-04-16T17:07:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.103+0000 7f806d20a740 1 Processor -- start 2024-04-16T17:07:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.103+0000 7f806d20a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 0x55fb73bab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.103+0000 7f806d20a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.103+0000 7f806d20a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73b9d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.103+0000 7f806d20a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80641b4070 con 0x55fb73b99a80 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.103+0000 7f806d20a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80641b54e0 con 0x55fb73ba3000 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.103+0000 7f806d20a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80641b4740 con 0x55fb73b9a7f0 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.103+0000 7f806d20a740 10 client.0 _ll_register_callbacks cb 0x7f805c00ea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 0x55fb73bab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73b9d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73b9d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33966/0 (socket says 172.21.15.18:33966) 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 -- 172.21.15.18:0/2559734573 learned_addr learned my addr 172.21.15.18:0/2559734573 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f805be00640 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 -- 172.21.15.18:0/2559734573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 msgr2=0x55fb73b9d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 -- 172.21.15.18:0/2559734573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 msgr2=0x55fb73bab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 0x55fb73bab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 -- 172.21.15.18:0/2559734573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb73b9ddb0 con 0x55fb73ba3000 2024-04-16T17:07:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8063e00640 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73b9d6c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f805c016040 tx=0x7f805c004e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79008a4ec0f02c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.104+0000 7f8051600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.105+0000 7f8051600640 1 -- 172.21.15.18:0/2559734573 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c044060 con 0x55fb73ba3000 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.105+0000 7f8051600640 1 -- 172.21.15.18:0/2559734573 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f805c01d4c0 con 0x55fb73ba3000 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.105+0000 7f806d20a740 1 -- 172.21.15.18:0/2559734573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f805c004ab0 con 0x55fb73ba3000 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.105+0000 7f8051600640 1 -- 172.21.15.18:0/2559734573 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f805c04b050 con 0x55fb73ba3000 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.105+0000 7f806d20a740 10 client.26390 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.105+0000 7f806d20a740 1 -- 172.21.15.18:0/2559734573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb73c19640 con 0x55fb73ba3000 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.105+0000 7f8051600640 1 -- 172.21.15.18:0/2559734573 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805c004ab0 con 0x55fb73ba3000 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.105+0000 7f8051600640 1 -- 172.21.15.18:0/2559734573 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f805c028bc0 con 0x55fb73ba3000 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f806d20a740 10 client.26390 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f806d20a740 10 client.26390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f806d20a740 20 client.26390 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f806d20a740 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb73c19ca0 0x55fb73c1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f806d20a740 4 client.26390 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:07:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f806d20a740 1 -- 172.21.15.18:0/2559734573 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb73ba5800 con 0x55fb73c19ca0 2024-04-16T17:07:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f8063400640 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb73c19ca0 0x55fb73c1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f8063400640 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb73c19ca0 0x55fb73c1c090 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4ecda162026a0e server_cookie=e525bc308c38f4b4 in_seq=0 out_seq=0 2024-04-16T17:07:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.106+0000 7f8051600640 10 client.26390 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.221+0000 7f8051600640 1 -- 172.21.15.18:0/2559734573 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb73ba5800 con 0x55fb73c19ca0 2024-04-16T17:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f8051600640 10 client.26390 handle_command_reply: tid=0 2024-04-16T17:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 client.26390 shutdown 2024-04-16T17:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 -- 172.21.15.18:0/2559734573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 msgr2=0x55fb73b9d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73b9d6c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f805c016040 tx=0x7f805c004e90 comp rx=0 tx=0).stop 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 -- 172.21.15.18:0/2559734573 shutdown_connections 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb73c19ca0 0x55fb73c1c090 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb73ba3000 0x55fb73b9d6c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb73b9a7f0 0x55fb73b9d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 --2- 172.21.15.18:0/2559734573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb73b99a80 0x55fb73bab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 -- 172.21.15.18:0/2559734573 >> 172.21.15.18:0/2559734573 conn(0x55fb73b345d0 msgr2=0x55fb73b9e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f8051600640 0 client.26390 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 -- 172.21.15.18:0/2559734573 shutdown_connections 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 1 -- 172.21.15.18:0/2559734573 wait complete. 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.222+0000 7f806d20a740 20 client.26390 trim_cache size 0 max 16384 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:07:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:07:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 -- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80480a3d00 msgr2=0x7f80480a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80480a3d00 0x7f80480a61c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f805c00af00 tx=0x7f805c00a6b0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 -- 172.21.15.18:0/518140588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 msgr2=0x7f806410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 0x7f806410dfc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8054005ab0 tx=0x7f80540043c0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 -- 172.21.15.18:0/518140588 shutdown_connections 2024-04-16T17:07:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80480a3d00 0x7f80480a61c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80641a3630 0x7f80641a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8064107150 0x7f806410e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 --2- 172.21.15.18:0/518140588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8064102080 0x7f806410dfc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.223+0000 7f8068e00640 1 -- 172.21.15.18:0/518140588 >> 172.21.15.18:0/518140588 conn(0x7f80640fd940 msgr2=0x7f80640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.224+0000 7f8068e00640 1 -- 172.21.15.18:0/518140588 shutdown_connections 2024-04-16T17:07:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:37.224+0000 7f8068e00640 1 -- 172.21.15.18:0/518140588 wait complete. 2024-04-16T17:07:37.533 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85267 4.35 MB/sec warmup 110 sec latency 38.441 ms 2024-04-16T17:07:37.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:37 smithi022 ceph-mon[36023]: pgmap v89: 129 pgs: 129 active+clean; 36 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-16T17:07:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:37 smithi043 ceph-mon[36901]: pgmap v89: 129 pgs: 129 active+clean; 36 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-16T17:07:38.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85549 4.32 MB/sec warmup 111 sec latency 41.538 ms 2024-04-16T17:07:39.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 85738 4.28 MB/sec warmup 112 sec latency 76.489 ms 2024-04-16T17:07:39.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:39 smithi022 ceph-mon[36023]: pgmap v90: 129 pgs: 129 active+clean; 36 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 749 KiB/s wr, 73 op/s 2024-04-16T17:07:40.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:39 smithi043 ceph-mon[36901]: pgmap v90: 129 pgs: 129 active+clean; 36 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 749 KiB/s wr, 73 op/s 2024-04-16T17:07:40.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:35 smithi018 ceph-mon[31776]: pgmap v88: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:07:40.301 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:37 smithi018 ceph-mon[31776]: pgmap v89: 129 pgs: 129 active+clean; 36 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-16T17:07:40.301 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:39 smithi018 ceph-mon[31776]: pgmap v90: 129 pgs: 129 active+clean; 36 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 749 KiB/s wr, 73 op/s 2024-04-16T17:07:40.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 86559 4.29 MB/sec warmup 113 sec latency 89.566 ms 2024-04-16T17:07:41.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 87944 4.31 MB/sec warmup 114 sec latency 20.171 ms 2024-04-16T17:07:41.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:07:41.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:07:41.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:41 smithi022 ceph-mon[36023]: pgmap v91: 129 pgs: 129 active+clean; 34 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-04-16T17:07:42.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:41 smithi043 ceph-mon[36901]: pgmap v91: 129 pgs: 129 active+clean; 34 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 -- 172.21.15.18:0/1639514600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c103900 msgr2=0x7f9a2c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 --2- 172.21.15.18:0/1639514600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c103900 0x7f9a2c10bde0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009900 tx=0x7f9a1802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 -- 172.21.15.18:0/1639514600 shutdown_connections 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 --2- 172.21.15.18:0/1639514600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a2c10c400 0x7f9a2c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 --2- 172.21.15.18:0/1639514600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c103900 0x7f9a2c10bde0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 --2- 172.21.15.18:0/1639514600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c102fe0 0x7f9a2c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 -- 172.21.15.18:0/1639514600 >> 172.21.15.18:0/1639514600 conn(0x7f9a2c077ab0 msgr2=0x7f9a2c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 -- 172.21.15.18:0/1639514600 shutdown_connections 2024-04-16T17:07:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.009+0000 7f9a2b200640 1 -- 172.21.15.18:0/1639514600 wait complete. 2024-04-16T17:07:42.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.010+0000 7f9a2b200640 1 Processor -- start 2024-04-16T17:07:42.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.010+0000 7f9a2b200640 1 -- start start 2024-04-16T17:07:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c102fe0 0x7f9a2c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a2c103900 0x7f9a2c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c10c400 0x7f9a2c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c112190 con 0x7f9a2c102fe0 2024-04-16T17:07:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c112010 con 0x7f9a2c103900 2024-04-16T17:07:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c112310 con 0x7f9a2c10c400 2024-04-16T17:07:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c10c400 0x7f9a2c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c10c400 0x7f9a2c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48188/0 (socket says 172.21.15.18:48188) 2024-04-16T17:07:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2a800640 1 -- 172.21.15.18:0/568659007 learned_addr learned my addr 172.21.15.18:0/568659007 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a29400640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a2c103900 0x7f9a2c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a29e00640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c102fe0 0x7f9a2c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2a800640 1 -- 172.21.15.18:0/568659007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a2c103900 msgr2=0x7f9a2c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2a800640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a2c103900 0x7f9a2c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2a800640 1 -- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c102fe0 msgr2=0x7f9a2c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2a800640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c102fe0 0x7f9a2c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.011+0000 7f9a2a800640 1 -- 172.21.15.18:0/568659007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c19b030 con 0x7f9a2c10c400 2024-04-16T17:07:42.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.012+0000 7f9a2a800640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c10c400 0x7f9a2c19a770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400eb10 tx=0x7f9a2400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.012+0000 7f9a22a00640 1 -- 172.21.15.18:0/568659007 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24004b20 con 0x7f9a2c10c400 2024-04-16T17:07:42.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.012+0000 7f9a22a00640 1 -- 172.21.15.18:0/568659007 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a24004cc0 con 0x7f9a2c10c400 2024-04-16T17:07:42.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.012+0000 7f9a22a00640 1 -- 172.21.15.18:0/568659007 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24010770 con 0x7f9a2c10c400 2024-04-16T17:07:42.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.012+0000 7f9a2b200640 1 -- 172.21.15.18:0/568659007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c19b320 con 0x7f9a2c10c400 2024-04-16T17:07:42.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.013+0000 7f9a2b200640 1 -- 172.21.15.18:0/568659007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a7630 con 0x7f9a2c10c400 2024-04-16T17:07:42.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.017+0000 7f9a33f8a740 1 Processor -- start 2024-04-16T17:07:42.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.018+0000 7f9a22a00640 1 -- 172.21.15.18:0/568659007 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a24005020 con 0x7f9a2c10c400 2024-04-16T17:07:42.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.019+0000 7f9a33f8a740 1 -- start start 2024-04-16T17:07:42.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.019+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde07770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.019+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde10350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.019+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde12c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.019+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5570fdd6d120 con 0x5570fde07390 2024-04-16T17:07:42.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.019+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5570fdd13340 con 0x5570fde10890 2024-04-16T17:07:42.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.019+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570fdd690e0 con 0x5570fde07cb0 2024-04-16T17:07:42.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde07770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde07770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56472/0 (socket says 172.21.15.18:56472) 2024-04-16T17:07:42.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29400640 1 -- 172.21.15.18:0/1843166206 learned_addr learned my addr 172.21.15.18:0/1843166206 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29e00640 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde10350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29400640 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde12c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29e00640 1 -- 172.21.15.18:0/1843166206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 msgr2=0x5570fde12c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29e00640 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde12c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29e00640 1 -- 172.21.15.18:0/1843166206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 msgr2=0x5570fde07770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29e00640 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde07770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29e00640 1 -- 172.21.15.18:0/1843166206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570fdb497d0 con 0x5570fde07cb0 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.020+0000 7f9a29400640 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde07770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.021+0000 7f9a22a00640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9a100a3da0 0x7f9a100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.021+0000 7f9a22a00640 1 -- 172.21.15.18:0/568659007 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9a240d1190 con 0x7f9a2c10c400 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.021+0000 7f9a29400640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9a100a3da0 0x7f9a100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.021+0000 7f9a29e00640 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde10350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009b70 tx=0x7f9a1c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2021ccf895a90d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.022+0000 7f9a17400640 1 -- 172.21.15.18:0/1843166206 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c009e60 con 0x5570fde07cb0 2024-04-16T17:07:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.022+0000 7f9a17400640 1 -- 172.21.15.18:0/1843166206 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c012930 con 0x5570fde07cb0 2024-04-16T17:07:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.022+0000 7f9a17400640 1 -- 172.21.15.18:0/1843166206 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00fae0 con 0x5570fde07cb0 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 -- 172.21.15.18:0/1843166206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 msgr2=0x5570fde10350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a29400640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9a100a3da0 0x7f9a100a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005c40 tx=0x7f9a18045040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde10350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009b70 tx=0x7f9a1c0117b0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 -- 172.21.15.18:0/1843166206 shutdown_connections 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde12c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde10350 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/1843166206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde07770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 -- 172.21.15.18:0/1843166206 >> 172.21.15.18:0/1843166206 conn(0x5570fde01ac0 msgr2=0x5570fde01ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 -- 172.21.15.18:0/1843166206 shutdown_connections 2024-04-16T17:07:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 -- 172.21.15.18:0/1843166206 wait complete. 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.023+0000 7f9a33f8a740 1 -- start start 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.024+0000 7f9a33f8a740 1 Processor -- start 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.024+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde0d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.024+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde0dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.024+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde0e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.024+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5570fde13680 con 0x5570fde07390 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.024+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5570fdd26710 con 0x5570fde10890 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.024+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570fde13800 con 0x5570fde07cb0 2024-04-16T17:07:42.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.024+0000 7f9a33f8a740 10 client.0 _ll_register_callbacks cb 0x5570fdd21af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a29400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde0dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a29400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde0dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48200/0 (socket says 172.21.15.18:48200) 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a29400640 1 -- 172.21.15.18:0/928607024 learned_addr learned my addr 172.21.15.18:0/928607024 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a2a800640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde0e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a29e00640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde0d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a2a800640 1 -- 172.21.15.18:0/928607024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 msgr2=0x5570fde0dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a2a800640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde0dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a2a800640 1 -- 172.21.15.18:0/928607024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 msgr2=0x5570fde0d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a2a800640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde0d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a2a800640 1 -- 172.21.15.18:0/928607024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570fda7f3f0 con 0x5570fde10890 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a29e00640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde0d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a29400640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde0dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a2a800640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde0e2e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9a240ce4d0 tx=0x7f9a2409c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce15c4bfb5fc41bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.025+0000 7f9a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:42.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a0be00640 1 -- 172.21.15.18:0/928607024 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a240a5700 con 0x5570fde10890 2024-04-16T17:07:42.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a0be00640 1 -- 172.21.15.18:0/928607024 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9a240bd070 con 0x5570fde10890 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a33f8a740 1 -- 172.21.15.18:0/928607024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c012930 con 0x5570fde10890 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a0be00640 1 -- 172.21.15.18:0/928607024 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a240a10d0 con 0x5570fde10890 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a33f8a740 10 client.26402 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a33f8a740 1 -- 172.21.15.18:0/928607024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570fda3f0f0 con 0x5570fde10890 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a0be00640 1 -- 172.21.15.18:0/928607024 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c012930 con 0x5570fde10890 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a0be00640 1 -- 172.21.15.18:0/928607024 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a2407bb40 con 0x5570fde10890 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a33f8a740 10 client.26402 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a33f8a740 10 client.26402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.026+0000 7f9a33f8a740 20 client.26402 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.027+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5570fde20c90 0x5570fde23100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.027+0000 7f9a33f8a740 4 client.26402 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.027+0000 7f9a33f8a740 1 -- 172.21.15.18:0/928607024 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570fdd7ab40 con 0x5570fde20c90 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.027+0000 7f9a29e00640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5570fde20c90 0x5570fde23100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.027+0000 7f9a29e00640 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5570fde20c90 0x5570fde23100 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a10056eab695d19f server_cookie=4f2ff04f6bd57d4e in_seq=0 out_seq=0 2024-04-16T17:07:42.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.027+0000 7f9a0be00640 10 client.26402 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:42.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.028+0000 7f9a0be00640 1 -- 172.21.15.18:0/928607024 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570fdd7ab40 con 0x5570fde20c90 2024-04-16T17:07:42.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.028+0000 7f9a0be00640 10 client.26402 handle_command_reply: tid=0 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.028+0000 7f9a33f8a740 1 client.26402 shutdown 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.028+0000 7f9a33f8a740 1 -- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 msgr2=0x5570fde0e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.028+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde0e2e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9a240ce4d0 tx=0x7f9a2409c740 comp rx=0 tx=0).stop 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 1 -- 172.21.15.18:0/928607024 shutdown_connections 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5570fde20c90 0x5570fde23100 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde10890 0x5570fde0e2e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07cb0 0x5570fde0dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/928607024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde07390 0x5570fde0d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a0be00640 0 client.26402 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 1 -- 172.21.15.18:0/928607024 >> 172.21.15.18:0/928607024 conn(0x5570fde01ac0 msgr2=0x5570fde03810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 1 -- 172.21.15.18:0/928607024 shutdown_connections 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 1 -- 172.21.15.18:0/928607024 wait complete. 2024-04-16T17:07:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.029+0000 7f9a33f8a740 20 client.26402 trim_cache size 0 max 16384 2024-04-16T17:07:42.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.056+0000 7f9a33f8a740 1 Processor -- start 2024-04-16T17:07:42.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.056+0000 7f9a33f8a740 1 -- start start 2024-04-16T17:07:42.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde1c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde0d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b3f40 con 0x5570fde10890 2024-04-16T17:07:42.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b1810 con 0x5570fde07cb0 2024-04-16T17:07:42.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1af660 con 0x5570fde07390 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a2a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a29e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde1c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a2a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56478/0 (socket says 172.21.15.18:56478) 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a2a800640 1 -- 172.21.15.18:0/2781581865 learned_addr learned my addr 172.21.15.18:0/2781581865 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.057+0000 7f9a29400640 1 --2- 172.21.15.18:0/2781581865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde0d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a29e00640 1 -- 172.21.15.18:0/2781581865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 msgr2=0x5570fde0d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a29e00640 1 --2- 172.21.15.18:0/2781581865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde0d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a29e00640 1 -- 172.21.15.18:0/2781581865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 msgr2=0x5570fde009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a29e00640 1 --2- 172.21.15.18:0/2781581865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a29e00640 1 -- 172.21.15.18:0/2781581865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570fde018f0 con 0x5570fde07390 2024-04-16T17:07:42.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a29e00640 1 --2- 172.21.15.18:0/2781581865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde1c570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c01c040 tx=0x7f9a1c009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5522b589deef75d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a16a00640 1 -- 172.21.15.18:0/2781581865 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c005040 con 0x5570fde07390 2024-04-16T17:07:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a16a00640 1 -- 172.21.15.18:0/2781581865 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c02f030 con 0x5570fde07390 2024-04-16T17:07:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.058+0000 7f9a16a00640 1 -- 172.21.15.18:0/2781581865 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c004c60 con 0x5570fde07390 2024-04-16T17:07:42.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 -- 172.21.15.18:0/2781581865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 msgr2=0x5570fde1c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/2781581865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde1c570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c01c040 tx=0x7f9a1c009a70 comp rx=0 tx=0).stop 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 -- 172.21.15.18:0/2781581865 shutdown_connections 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/2781581865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/2781581865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde0d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/2781581865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde1c570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 -- 172.21.15.18:0/2781581865 >> 172.21.15.18:0/2781581865 conn(0x5570fde01ac0 msgr2=0x5570fde00290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 -- 172.21.15.18:0/2781581865 shutdown_connections 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.059+0000 7f9a33f8a740 1 -- 172.21.15.18:0/2781581865 wait complete. 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 1 -- start start 2024-04-16T17:07:42.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 1 Processor -- start 2024-04-16T17:07:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde18a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde18f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde13f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1afcd0 con 0x5570fde10890 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b1140 con 0x5570fde07cb0 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b03a0 con 0x5570fde07390 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.060+0000 7f9a33f8a740 10 client.0 _ll_register_callbacks cb 0x7f9a2400cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde18a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde18a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48220/0 (socket says 172.21.15.18:48220) 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 -- 172.21.15.18:0/844171778 learned_addr learned my addr 172.21.15.18:0/844171778 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a2a800640 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde13f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29400640 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde18f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 -- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 msgr2=0x5570fde18f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde18f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 -- 172.21.15.18:0/844171778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 msgr2=0x5570fde13f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde13f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 -- 172.21.15.18:0/844171778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570fde14730 con 0x5570fde07390 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29400640 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde18f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a29e00640 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde18a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009340 tx=0x7f9a1c004680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a3343404e78121 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.061+0000 7f9a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a0be00640 1 -- 172.21.15.18:0/844171778 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c009450 con 0x5570fde07390 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a33f8a740 1 -- 172.21.15.18:0/844171778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c02f030 con 0x5570fde07390 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a0be00640 1 -- 172.21.15.18:0/844171778 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9a1c004860 con 0x5570fde07390 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a0be00640 1 -- 172.21.15.18:0/844171778 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a1c042050 con 0x5570fde07390 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a33f8a740 10 client.26404 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a33f8a740 1 -- 172.21.15.18:0/844171778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570fde86e20 con 0x5570fde07390 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a0be00640 1 -- 172.21.15.18:0/844171778 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c02f030 con 0x5570fde07390 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a0be00640 1 -- 172.21.15.18:0/844171778 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a1c012450 con 0x5570fde07390 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.062+0000 7f9a33f8a740 10 client.26404 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.063+0000 7f9a33f8a740 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.063+0000 7f9a33f8a740 20 client.26404 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.063+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5570fde87340 0x5570fde89730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.063+0000 7f9a33f8a740 4 client.26404 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:07:42.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.063+0000 7f9a33f8a740 1 -- 172.21.15.18:0/844171778 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5570fdd749b0 con 0x5570fde87340 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.063+0000 7f9a29400640 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5570fde87340 0x5570fde89730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a29400640 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5570fde87340 0x5570fde89730 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ada15ae1c234f9c server_cookie=eb318cc3d75ec00b in_seq=0 out_seq=0 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a0be00640 10 client.26404 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a0be00640 1 -- 172.21.15.18:0/844171778 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5570fdd749b0 con 0x5570fde87340 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a0be00640 10 client.26404 handle_command_reply: tid=0 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 client.26404 shutdown 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 -- 172.21.15.18:0/844171778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 msgr2=0x5570fde18a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde18a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009340 tx=0x7f9a1c004680 comp rx=0 tx=0).stop 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 -- 172.21.15.18:0/844171778 shutdown_connections 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5570fde87340 0x5570fde89730 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5570fde10890 0x5570fde13f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5570fde07cb0 0x5570fde18f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a0be00640 0 client.26404 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:42.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 --2- 172.21.15.18:0/844171778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570fde07390 0x5570fde18a20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.064+0000 7f9a33f8a740 1 -- 172.21.15.18:0/844171778 >> 172.21.15.18:0/844171778 conn(0x5570fde01ac0 msgr2=0x5570fde00b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.065+0000 7f9a33f8a740 1 -- 172.21.15.18:0/844171778 shutdown_connections 2024-04-16T17:07:42.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.065+0000 7f9a33f8a740 1 -- 172.21.15.18:0/844171778 wait complete. 2024-04-16T17:07:42.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.065+0000 7f9a33f8a740 20 client.26404 trim_cache size 0 max 16384 2024-04-16T17:07:42.068 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:07:42.068 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:07:42.068 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:07:42.068 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 -- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9a100a3da0 msgr2=0x7f9a100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9a100a3da0 0x7f9a100a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005c40 tx=0x7f9a18045040 comp rx=0 tx=0).stop 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 -- 172.21.15.18:0/568659007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c10c400 msgr2=0x7f9a2c19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c10c400 0x7f9a2c19a770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400eb10 tx=0x7f9a2400c2d0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 -- 172.21.15.18:0/568659007 shutdown_connections 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9a100a3da0 0x7f9a100a6260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a2c10c400 0x7f9a2c19a770 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a2c103900 0x7f9a2c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 --2- 172.21.15.18:0/568659007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a2c102fe0 0x7f9a2c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 -- 172.21.15.18:0/568659007 >> 172.21.15.18:0/568659007 conn(0x7f9a2c077ab0 msgr2=0x7f9a2c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 -- 172.21.15.18:0/568659007 shutdown_connections 2024-04-16T17:07:42.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.066+0000 7f9a2b200640 1 -- 172.21.15.18:0/568659007 wait complete. 2024-04-16T17:07:42.090 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:07:42.091 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:07:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.302+0000 7fa0a8a00640 1 -- 172.21.15.18:0/4014274442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a4102130 msgr2=0x7fa0a4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.302+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/4014274442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a4102130 0x7fa0a4102510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa094009900 tx=0x7fa09402bc50 comp rx=0 tx=0).stop 2024-04-16T17:07:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.302+0000 7fa0a8a00640 1 -- 172.21.15.18:0/4014274442 shutdown_connections 2024-04-16T17:07:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.302+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/4014274442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0a4107200 0x7fa0a410edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.302+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/4014274442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a4102ae0 0x7fa0a4106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.302+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/4014274442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a4102130 0x7fa0a4102510 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.302+0000 7fa0a8a00640 1 -- 172.21.15.18:0/4014274442 >> 172.21.15.18:0/4014274442 conn(0x7fa0a40fd9f0 msgr2=0x7fa0a40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.303+0000 7fa0a8a00640 1 -- 172.21.15.18:0/4014274442 shutdown_connections 2024-04-16T17:07:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.303+0000 7fa0a8a00640 1 -- 172.21.15.18:0/4014274442 wait complete. 2024-04-16T17:07:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.304+0000 7fa0a8a00640 1 Processor -- start 2024-04-16T17:07:42.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.304+0000 7fa0a8a00640 1 -- start start 2024-04-16T17:07:42.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.304+0000 7fa0a8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0a4102130 0x7fa0a4074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.304+0000 7fa0a8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a4107200 0x7fa0a4074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a4074d80 0x7fa0a41a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a8a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41122e0 con 0x7fa0a4107200 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a8a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4112160 con 0x7fa0a4102130 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4112460 con 0x7fa0a4074d80 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0a4102130 0x7fa0a4074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0a4102130 0x7fa0a4074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34036/0 (socket says 172.21.15.18:34036) 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a2800640 1 -- 172.21.15.18:0/3897765893 learned_addr learned my addr 172.21.15.18:0/3897765893 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa09be00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a4107200 0x7fa0a4074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a3200640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a4074d80 0x7fa0a41a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a2800640 1 -- 172.21.15.18:0/3897765893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a4074d80 msgr2=0x7fa0a41a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a2800640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a4074d80 0x7fa0a41a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a2800640 1 -- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a4107200 msgr2=0x7fa0a4074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a2800640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a4107200 0x7fa0a4074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.305+0000 7fa0a2800640 1 -- 172.21.15.18:0/3897765893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a41a7070 con 0x7fa0a4102130 2024-04-16T17:07:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.306+0000 7fa0a2800640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0a4102130 0x7fa0a4074300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa094005ab0 tx=0x7fa094004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.306+0000 7fa09b400640 1 -- 172.21.15.18:0/3897765893 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094041070 con 0x7fa0a4102130 2024-04-16T17:07:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.306+0000 7fa09b400640 1 -- 172.21.15.18:0/3897765893 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09402ccb0 con 0x7fa0a4102130 2024-04-16T17:07:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.306+0000 7fa09b400640 1 -- 172.21.15.18:0/3897765893 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094003df0 con 0x7fa0a4102130 2024-04-16T17:07:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.306+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a41a7300 con 0x7fa0a4102130 2024-04-16T17:07:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.306+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a41a78a0 con 0x7fa0a4102130 2024-04-16T17:07:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.307+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a404fe00 con 0x7fa0a4102130 2024-04-16T17:07:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.311+0000 7fa09b400640 1 -- 172.21.15.18:0/3897765893 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa094044070 con 0x7fa0a4102130 2024-04-16T17:07:42.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.312+0000 7fa09b400640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0880a3dc0 0x7fa0880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.312+0000 7fa09b400640 1 -- 172.21.15.18:0/3897765893 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa0940f2410 con 0x7fa0a4102130 2024-04-16T17:07:42.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.313+0000 7fa09be00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0880a3dc0 0x7fa0880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.313+0000 7fa09be00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0880a3dc0 0x7fa0880a6280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4075d90 tx=0x7fa08c005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.317+0000 7fa09b400640 1 -- 172.21.15.18:0/3897765893 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0940be160 con 0x7fa0a4102130 2024-04-16T17:07:42.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 89113 4.34 MB/sec warmup 115 sec latency 9.109 ms 2024-04-16T17:07:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.588+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0a410a430 con 0x7fa0a4102130 2024-04-16T17:07:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.589+0000 7fa09b400640 1 -- 172.21.15.18:0/3897765893 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7fa0940c2460 con 0x7fa0a4102130 2024-04-16T17:07:42.590 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:07:42.590 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:07:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0880a3dc0 msgr2=0x7fa0880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0880a3dc0 0x7fa0880a6280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4075d90 tx=0x7fa08c005f90 comp rx=0 tx=0).stop 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0a4102130 msgr2=0x7fa0a4074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0a4102130 0x7fa0a4074300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa094005ab0 tx=0x7fa094004340 comp rx=0 tx=0).stop 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 shutdown_connections 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0880a3dc0 0x7fa0880a6280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a4074d80 0x7fa0a41a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0a4107200 0x7fa0a4074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 --2- 172.21.15.18:0/3897765893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0a4102130 0x7fa0a4074300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 >> 172.21.15.18:0/3897765893 conn(0x7fa0a40fd9f0 msgr2=0x7fa0a40ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 shutdown_connections 2024-04-16T17:07:42.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.592+0000 7fa0a8a00640 1 -- 172.21.15.18:0/3897765893 wait complete. 2024-04-16T17:07:42.614 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:07:42.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 -- 172.21.15.18:0/3932089478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c074cd0 msgr2=0x7f279c10e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 --2- 172.21.15.18:0/3932089478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c074cd0 0x7f279c10e180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2788009900 tx=0x7f278802bcf0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 -- 172.21.15.18:0/3932089478 shutdown_connections 2024-04-16T17:07:42.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 --2- 172.21.15.18:0/3932089478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f279c10e6c0 0x7f279c110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 --2- 172.21.15.18:0/3932089478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c074cd0 0x7f279c10e180 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 --2- 172.21.15.18:0/3932089478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f279c0743b0 0x7f279c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 -- 172.21.15.18:0/3932089478 >> 172.21.15.18:0/3932089478 conn(0x7f279c0fd950 msgr2=0x7f279c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 -- 172.21.15.18:0/3932089478 shutdown_connections 2024-04-16T17:07:42.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.796+0000 7f27a3000640 1 -- 172.21.15.18:0/3932089478 wait complete. 2024-04-16T17:07:42.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.797+0000 7f27a3000640 1 Processor -- start 2024-04-16T17:07:42.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a3000640 1 -- start start 2024-04-16T17:07:42.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f279c0743b0 0x7f279c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a3000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f279c074cd0 0x7f279c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a3000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c10e6c0 0x7f279c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a3000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1144a0 con 0x7f279c10e6c0 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a3000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f279c114320 con 0x7f279c074cd0 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a3000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f279c114620 con 0x7f279c0743b0 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a2600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c10e6c0 0x7f279c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a2600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c10e6c0 0x7f279c19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56504/0 (socket says 172.21.15.18:56504) 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a2600640 1 -- 172.21.15.18:0/4035223662 learned_addr learned my addr 172.21.15.18:0/4035223662 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.798+0000 7f27a1200640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f279c074cd0 0x7f279c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.799+0000 7f27a2600640 1 -- 172.21.15.18:0/4035223662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f279c0743b0 msgr2=0x7f279c1a06b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:07:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.799+0000 7f27a2600640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f279c0743b0 0x7f279c1a06b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.799+0000 7f27a2600640 1 -- 172.21.15.18:0/4035223662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f279c074cd0 msgr2=0x7f279c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.799+0000 7f27a2600640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f279c074cd0 0x7f279c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.799+0000 7f27a2600640 1 -- 172.21.15.18:0/4035223662 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c19aff0 con 0x7f279c10e6c0 2024-04-16T17:07:42.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.799+0000 7f27a2600640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c10e6c0 0x7f279c19a820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f279400efd0 tx=0x7f279400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.799+0000 7f279a000640 1 -- 172.21.15.18:0/4035223662 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794003b70 con 0x7f279c10e6c0 2024-04-16T17:07:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.800+0000 7f27a3000640 1 -- 172.21.15.18:0/4035223662 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c19b2e0 con 0x7f279c10e6c0 2024-04-16T17:07:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.800+0000 7f279a000640 1 -- 172.21.15.18:0/4035223662 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2794003d10 con 0x7f279c10e6c0 2024-04-16T17:07:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.800+0000 7f279a000640 1 -- 172.21.15.18:0/4035223662 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27940109f0 con 0x7f279c10e6c0 2024-04-16T17:07:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.800+0000 7f27a3000640 1 -- 172.21.15.18:0/4035223662 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c1a7500 con 0x7f279c10e6c0 2024-04-16T17:07:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.803+0000 7f279a000640 1 -- 172.21.15.18:0/4035223662 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2794004020 con 0x7f279c10e6c0 2024-04-16T17:07:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.803+0000 7f27a7421740 1 Processor -- start 2024-04-16T17:07:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.804+0000 7f27a7421740 1 -- start start 2024-04-16T17:07:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.804+0000 7f27a7421740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b6f10 0x55e3919b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.804+0000 7f27a7421740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.804+0000 7f27a7421740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919c2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.804+0000 7f27a7421740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e39191c020 con 0x55e3919b7830 2024-04-16T17:07:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.804+0000 7f27a7421740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e3918c2460 con 0x55e3919b6f10 2024-04-16T17:07:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.804+0000 7f27a7421740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e391917fe0 con 0x55e3919c0410 2024-04-16T17:07:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.804+0000 7f27a1c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.805+0000 7f27a1c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919bfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56514/0 (socket says 172.21.15.18:56514) 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.805+0000 7f27a1c00640 1 -- 172.21.15.18:0/3457536283 learned_addr learned my addr 172.21.15.18:0/3457536283 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.805+0000 7f279a000640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27840a3da0 0x7f27840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.805+0000 7f279a000640 1 -- 172.21.15.18:0/4035223662 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f27940d10a0 con 0x7f279c10e6c0 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.805+0000 7f27a2600640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27840a3da0 0x7f27840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.805+0000 7f27a1c00640 1 -- 172.21.15.18:0/3457536283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 msgr2=0x55e3919c2800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.805+0000 7f27a1c00640 1 --2- 172.21.15.18:0/3457536283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919c2800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.806+0000 7f27a1c00640 1 -- 172.21.15.18:0/3457536283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b6f10 msgr2=0x55e3919b72f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.806+0000 7f27a1c00640 1 --2- 172.21.15.18:0/3457536283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b6f10 0x55e3919b72f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.806+0000 7f27a1c00640 1 -- 172.21.15.18:0/3457536283 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3918ca040 con 0x55e3919b7830 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.806+0000 7f27a1c00640 1 --2- 172.21.15.18:0/3457536283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919bfed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2790009770 tx=0x7f2790009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7e75dee8e0f661 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.806+0000 7f27a2600640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27840a3da0 0x7f27840a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f279400cba0 tx=0x7f2794004920 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.806+0000 7f278ea00640 1 -- 172.21.15.18:0/3457536283 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790015020 con 0x55e3919b7830 2024-04-16T17:07:42.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.806+0000 7f278ea00640 1 -- 172.21.15.18:0/3457536283 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2790011420 con 0x55e3919b7830 2024-04-16T17:07:42.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.806+0000 7f278ea00640 1 -- 172.21.15.18:0/3457536283 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27900105e0 con 0x55e3919b7830 2024-04-16T17:07:42.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.807+0000 7f27a7421740 1 -- 172.21.15.18:0/3457536283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 msgr2=0x55e3919bfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.807+0000 7f27a7421740 1 --2- 172.21.15.18:0/3457536283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919bfed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2790009770 tx=0x7f2790009c30 comp rx=0 tx=0).stop 2024-04-16T17:07:42.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.807+0000 7f27a7421740 1 -- 172.21.15.18:0/3457536283 shutdown_connections 2024-04-16T17:07:42.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.807+0000 7f27a7421740 1 --2- 172.21.15.18:0/3457536283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.807+0000 7f27a7421740 1 --2- 172.21.15.18:0/3457536283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919bfed0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.807+0000 7f27a7421740 1 --2- 172.21.15.18:0/3457536283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b6f10 0x55e3919b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.807+0000 7f27a7421740 1 -- 172.21.15.18:0/3457536283 >> 172.21.15.18:0/3457536283 conn(0x55e3919b1640 msgr2=0x55e3919b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.808+0000 7f27a7421740 1 -- 172.21.15.18:0/3457536283 shutdown_connections 2024-04-16T17:07:42.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.808+0000 7f27a7421740 1 -- 172.21.15.18:0/3457536283 wait complete. 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.808+0000 7f27a7421740 1 -- start start 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.808+0000 7f27a7421740 1 Processor -- start 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a7421740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a7421740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919bd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a7421740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a7421740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e3919c3200 con 0x55e3919b6f10 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a7421740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e3918d5830 con 0x55e3919b7830 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a7421740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e3919c3380 con 0x55e3919c0410 2024-04-16T17:07:42.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a7421740 10 client.0 _ll_register_callbacks cb 0x55e3918dd150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a1200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a1200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56516/0 (socket says 172.21.15.18:56516) 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a1200640 1 -- 172.21.15.18:0/2183731987 learned_addr learned my addr 172.21.15.18:0/2183731987 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.809+0000 7f27a1c00640 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919bd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.810+0000 7f27a1200640 1 -- 172.21.15.18:0/2183731987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 msgr2=0x55e3919bdeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.810+0000 7f27a1200640 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919bdeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.810+0000 7f27a1200640 1 -- 172.21.15.18:0/2183731987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 msgr2=0x55e3919bd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.810+0000 7f27a1200640 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919bd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.810+0000 7f27a1200640 1 -- 172.21.15.18:0/2183731987 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3918d9920 con 0x55e3919b6f10 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.810+0000 7f27a1c00640 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919bd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.810+0000 7f27a1200640 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bd430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2788009230 tx=0x7f278802c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ff6502d684151c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.810+0000 7f2783400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:07:42.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.811+0000 7f2783400640 1 -- 172.21.15.18:0/2183731987 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788045d50 con 0x55e3919b6f10 2024-04-16T17:07:42.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.811+0000 7f27a7421740 1 -- 172.21.15.18:0/2183731987 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2790011420 con 0x55e3919b6f10 2024-04-16T17:07:42.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.811+0000 7f2783400640 1 -- 172.21.15.18:0/2183731987 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f278802c940 con 0x55e3919b6f10 2024-04-16T17:07:42.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.811+0000 7f2783400640 1 -- 172.21.15.18:0/2183731987 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f278802cb40 con 0x55e3919b6f10 2024-04-16T17:07:42.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.811+0000 7f27a7421740 10 client.16233 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:42.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.811+0000 7f27a7421740 1 -- 172.21.15.18:0/2183731987 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3918a77f0 con 0x55e3919b6f10 2024-04-16T17:07:42.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.811+0000 7f2783400640 1 -- 172.21.15.18:0/2183731987 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2790011420 con 0x55e3919b6f10 2024-04-16T17:07:42.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.811+0000 7f2783400640 1 -- 172.21.15.18:0/2183731987 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2788004fb0 con 0x55e3919b6f10 2024-04-16T17:07:42.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.812+0000 7f27a7421740 10 client.16233 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:42.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.812+0000 7f27a7421740 10 client.16233 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:07:42.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.812+0000 7f27a7421740 20 client.16233 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:42.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.812+0000 7f27a7421740 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55e3919d06b0 0x55e3919d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.812+0000 7f27a7421740 4 client.16233 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:42.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.812+0000 7f27a7421740 1 -- 172.21.15.18:0/2183731987 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e391929a40 con 0x55e3919d06b0 2024-04-16T17:07:42.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.823+0000 7f27a1c00640 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55e3919d06b0 0x55e3919d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.846+0000 7f27a1c00640 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55e3919d06b0 0x55e3919d2ae0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=738165f41a951341 server_cookie=7f5001fa3e258cbf in_seq=0 out_seq=0 2024-04-16T17:07:42.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.846+0000 7f2783400640 10 client.16233 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:07:42.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f2783400640 1 -- 172.21.15.18:0/2183731987 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e391929a40 con 0x55e3919d06b0 2024-04-16T17:07:42.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f2783400640 10 client.16233 handle_command_reply: tid=0 2024-04-16T17:07:42.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 client.16233 shutdown 2024-04-16T17:07:42.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 -- 172.21.15.18:0/2183731987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 msgr2=0x55e3919bd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bd430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2788009230 tx=0x7f278802c8a0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 -- 172.21.15.18:0/2183731987 shutdown_connections 2024-04-16T17:07:42.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55e3919d06b0 0x55e3919d2ae0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919bdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919bd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 --2- 172.21.15.18:0/2183731987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bd430 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f2783400640 0 client.16233 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:07:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 -- 172.21.15.18:0/2183731987 >> 172.21.15.18:0/2183731987 conn(0x55e3919b1640 msgr2=0x55e3919b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 -- 172.21.15.18:0/2183731987 shutdown_connections 2024-04-16T17:07:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 1 -- 172.21.15.18:0/2183731987 wait complete. 2024-04-16T17:07:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.847+0000 7f27a7421740 20 client.16233 trim_cache size 0 max 16384 2024-04-16T17:07:42.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.862+0000 7f27a7421740 1 Processor -- start 2024-04-16T17:07:42.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.862+0000 7f27a7421740 1 -- start start 2024-04-16T17:07:42.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.862+0000 7f27a7421740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.862+0000 7f27a7421740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.862+0000 7f27a7421740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.862+0000 7f27a7421740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1afa80 con 0x55e3919b6f10 2024-04-16T17:07:42.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.862+0000 7f27a7421740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1ad350 con 0x55e3919b7830 2024-04-16T17:07:42.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.862+0000 7f27a7421740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1ab1a0 con 0x55e3919c0410 2024-04-16T17:07:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56518/0 (socket says 172.21.15.18:56518) 2024-04-16T17:07:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 -- 172.21.15.18:0/2294976887 learned_addr learned my addr 172.21.15.18:0/2294976887 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 -- 172.21.15.18:0/2294976887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 msgr2=0x55e3919b3180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 --2- 172.21.15.18:0/2294976887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919b3180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 -- 172.21.15.18:0/2294976887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 msgr2=0x55e3919cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 --2- 172.21.15.18:0/2294976887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 -- 172.21.15.18:0/2294976887 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39162e3f0 con 0x55e3919b6f10 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1c00640 1 --2- 172.21.15.18:0/2294976887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.863+0000 7f27a1200640 1 --2- 172.21.15.18:0/2294976887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bfb60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2788009850 tx=0x7f2788005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a9ff18deaf4c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.864+0000 7f278e000640 1 -- 172.21.15.18:0/2294976887 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788005640 con 0x55e3919b6f10 2024-04-16T17:07:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.864+0000 7f278e000640 1 -- 172.21.15.18:0/2294976887 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2788009520 con 0x55e3919b6f10 2024-04-16T17:07:42.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.864+0000 7f278e000640 1 -- 172.21.15.18:0/2294976887 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788003900 con 0x55e3919b6f10 2024-04-16T17:07:42.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.864+0000 7f27a7421740 1 -- 172.21.15.18:0/2294976887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 msgr2=0x55e3919bfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.864+0000 7f27a7421740 1 --2- 172.21.15.18:0/2294976887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bfb60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2788009850 tx=0x7f2788005b50 comp rx=0 tx=0).stop 2024-04-16T17:07:42.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.864+0000 7f27a7421740 1 -- 172.21.15.18:0/2294976887 shutdown_connections 2024-04-16T17:07:42.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.865+0000 7f27a7421740 1 --2- 172.21.15.18:0/2294976887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919c0410 0x55e3919b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.865+0000 7f27a7421740 1 --2- 172.21.15.18:0/2294976887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919b7830 0x55e3919cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.865+0000 7f27a7421740 1 --2- 172.21.15.18:0/2294976887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b6f10 0x55e3919bfb60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.865+0000 7f27a7421740 1 -- 172.21.15.18:0/2294976887 >> 172.21.15.18:0/2294976887 conn(0x55e3919b1640 msgr2=0x55e3919b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.865+0000 7f27a7421740 1 -- 172.21.15.18:0/2294976887 shutdown_connections 2024-04-16T17:07:42.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.865+0000 7f27a7421740 1 -- 172.21.15.18:0/2294976887 wait complete. 2024-04-16T17:07:42.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.865+0000 7f27a7421740 1 -- start start 2024-04-16T17:07:42.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.865+0000 7f27a7421740 1 Processor -- start 2024-04-16T17:07:42.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a7421740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919b6f10 0x55e3919c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a7421740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a7421740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919c0410 0x55e3919ba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a7421740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1ab810 con 0x55e3919b7830 2024-04-16T17:07:42.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a7421740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1acc80 con 0x55e3919c0410 2024-04-16T17:07:42.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a7421740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1abee0 con 0x55e3919b6f10 2024-04-16T17:07:42.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a7421740 10 client.0 _ll_register_callbacks cb 0x7f2788009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:42.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919c0410 0x55e3919ba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a1c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919c0410 0x55e3919ba6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34086/0 (socket says 172.21.15.18:34086) 2024-04-16T17:07:42.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a1200640 1 -- 172.21.15.18:0/1809366883 learned_addr learned my addr 172.21.15.18:0/1809366883 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:42.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a1c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919b9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56524/0 (socket says 172.21.15.18:56524) 2024-04-16T17:07:42.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.866+0000 7f27a1200640 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919b6f10 0x55e3919c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f27a1c00640 1 -- 172.21.15.18:0/1809366883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919b6f10 msgr2=0x55e3919c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f27a1c00640 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919b6f10 0x55e3919c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f27a1c00640 1 -- 172.21.15.18:0/1809366883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919c0410 msgr2=0x55e3919ba6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:42.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f27a1c00640 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919c0410 0x55e3919ba6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f27a1c00640 1 -- 172.21.15.18:0/1809366883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3916f87d0 con 0x55e3919b7830 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f27a1200640 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919c0410 0x55e3919ba6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f27a1c00640 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919b9f70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2790011ee0 tx=0x7f27900046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4759d352971b9d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f2783400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f2783400640 1 -- 172.21.15.18:0/1809366883 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790015040 con 0x55e3919b7830 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f2783400640 1 -- 172.21.15.18:0/1809366883 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f279000be00 con 0x55e3919b7830 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.867+0000 7f27a7421740 1 -- 172.21.15.18:0/1809366883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2788009520 con 0x55e3919b7830 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f2783400640 1 -- 172.21.15.18:0/1809366883 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2790035050 con 0x55e3919b7830 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f27a7421740 10 client.16239 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f27a7421740 1 -- 172.21.15.18:0/1809366883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e391a36bf0 con 0x55e3919b7830 2024-04-16T17:07:42.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f2783400640 1 -- 172.21.15.18:0/1809366883 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2788009520 con 0x55e3919b7830 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f2783400640 1 -- 172.21.15.18:0/1809366883 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f27900263a0 con 0x55e3919b7830 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f27a7421740 10 client.16239 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f27a7421740 10 client.16239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f27a7421740 20 client.16239 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f27a7421740 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55e391a37230 0x55e391a39650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f27a7421740 4 client.16239 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.868+0000 7f27a7421740 1 -- 172.21.15.18:0/1809366883 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3919238b0 con 0x55e391a37230 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.869+0000 7f27a1200640 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55e391a37230 0x55e391a39650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.869+0000 7f27a1200640 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55e391a37230 0x55e391a39650 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b28b2d085449224a server_cookie=18355d40ce259246 in_seq=0 out_seq=0 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.869+0000 7f2783400640 10 client.16239 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.869+0000 7f2783400640 1 -- 172.21.15.18:0/1809366883 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3919238b0 con 0x55e391a37230 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.869+0000 7f2783400640 10 client.16239 handle_command_reply: tid=0 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.869+0000 7f27a7421740 1 client.16239 shutdown 2024-04-16T17:07:42.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 -- 172.21.15.18:0/1809366883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 msgr2=0x55e3919b9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919b9f70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2790011ee0 tx=0x7f27900046a0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 -- 172.21.15.18:0/1809366883 shutdown_connections 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55e391a37230 0x55e391a39650 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e3919c0410 0x55e3919ba6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e3919b7830 0x55e3919b9f70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 --2- 172.21.15.18:0/1809366883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3919b6f10 0x55e3919c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f2783400640 0 client.16239 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 -- 172.21.15.18:0/1809366883 >> 172.21.15.18:0/1809366883 conn(0x55e3919b1640 msgr2=0x55e3919b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 -- 172.21.15.18:0/1809366883 shutdown_connections 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 1 -- 172.21.15.18:0/1809366883 wait complete. 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.870+0000 7f27a7421740 20 client.16239 trim_cache size 0 max 16384 2024-04-16T17:07:42.873 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 -- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27840a3da0 msgr2=0x7f27840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27840a3da0 0x7f27840a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f279400cba0 tx=0x7f2794004920 comp rx=0 tx=0).stop 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 -- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c10e6c0 msgr2=0x7f279c19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c10e6c0 0x7f279c19a820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f279400efd0 tx=0x7f279400c6a0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 -- 172.21.15.18:0/4035223662 shutdown_connections 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27840a3da0 0x7f27840a6260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279c10e6c0 0x7f279c19a820 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f279c074cd0 0x7f279c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 --2- 172.21.15.18:0/4035223662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f279c0743b0 0x7f279c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:42.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 -- 172.21.15.18:0/4035223662 >> 172.21.15.18:0/4035223662 conn(0x7f279c0fd950 msgr2=0x7f279c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 -- 172.21.15.18:0/4035223662 shutdown_connections 2024-04-16T17:07:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:42.871+0000 7f27a3000640 1 -- 172.21.15.18:0/4035223662 wait complete. 2024-04-16T17:07:42.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:07:43.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 -- 172.21.15.18:0/3905259892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc102040 msgr2=0x7f54fc102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 --2- 172.21.15.18:0/3905259892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc102040 0x7f54fc102420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009860 tx=0x7f54ec02bb70 comp rx=0 tx=0).stop 2024-04-16T17:07:43.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 -- 172.21.15.18:0/3905259892 shutdown_connections 2024-04-16T17:07:43.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 --2- 172.21.15.18:0/3905259892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc107110 0x7f54fc10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 --2- 172.21.15.18:0/3905259892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54fc1029f0 0x7f54fc106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 --2- 172.21.15.18:0/3905259892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc102040 0x7f54fc102420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 -- 172.21.15.18:0/3905259892 >> 172.21.15.18:0/3905259892 conn(0x7f54fc0fd8e0 msgr2=0x7f54fc0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 -- 172.21.15.18:0/3905259892 shutdown_connections 2024-04-16T17:07:43.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.078+0000 7f54fb200640 1 -- 172.21.15.18:0/3905259892 wait complete. 2024-04-16T17:07:43.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.079+0000 7f54fb200640 1 Processor -- start 2024-04-16T17:07:43.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54fb200640 1 -- start start 2024-04-16T17:07:43.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54fb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc102040 0x7f54fc074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54fb200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54fc107110 0x7f54fc074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54fb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc074d80 0x7f54fc1a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54fb200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1121d0 con 0x7f54fc107110 2024-04-16T17:07:43.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54fb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc112050 con 0x7f54fc074d80 2024-04-16T17:07:43.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54fb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc112350 con 0x7f54fc102040 2024-04-16T17:07:43.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc102040 0x7f54fc074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc102040 0x7f54fc074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48322/0 (socket says 172.21.15.18:48322) 2024-04-16T17:07:43.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54f9e00640 1 -- 172.21.15.18:0/3566462272 learned_addr learned my addr 172.21.15.18:0/3566462272 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.080+0000 7f54fa800640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc074d80 0x7f54fc1a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.081+0000 7f54f9e00640 1 -- 172.21.15.18:0/3566462272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc074d80 msgr2=0x7f54fc1a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.081+0000 7f54f9e00640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc074d80 0x7f54fc1a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.081+0000 7f54f9e00640 1 -- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54fc107110 msgr2=0x7f54fc074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.081+0000 7f54f9e00640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54fc107110 0x7f54fc074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.081+0000 7f54f9e00640 1 -- 172.21.15.18:0/3566462272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc1a7020 con 0x7f54fc102040 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.081+0000 7f54fa800640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc074d80 0x7f54fc1a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.081+0000 7f54f9e00640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc102040 0x7f54fc074300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f54ec037040 tx=0x7f54ec0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.082+0000 7f54f2a00640 1 -- 172.21.15.18:0/3566462272 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec041020 con 0x7f54fc102040 2024-04-16T17:07:43.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.082+0000 7f54fb200640 1 -- 172.21.15.18:0/3566462272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc1a72b0 con 0x7f54fc102040 2024-04-16T17:07:43.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.082+0000 7f54f2a00640 1 -- 172.21.15.18:0/3566462272 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec02c9f0 con 0x7f54fc102040 2024-04-16T17:07:43.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.082+0000 7f54f2a00640 1 -- 172.21.15.18:0/3566462272 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec039040 con 0x7f54fc102040 2024-04-16T17:07:43.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.082+0000 7f54fb200640 1 -- 172.21.15.18:0/3566462272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc1a77f0 con 0x7f54fc102040 2024-04-16T17:07:43.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.085+0000 7f54f2a00640 1 -- 172.21.15.18:0/3566462272 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f54ec044020 con 0x7f54fc102040 2024-04-16T17:07:43.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.086+0000 7f550377e740 1 Processor -- start 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.086+0000 7f550377e740 1 -- start start 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.086+0000 7f550377e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef3e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.086+0000 7f550377e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef46ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.086+0000 7f550377e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef49800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.086+0000 7f550377e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557b4eea3020 con 0x557b4ef47410 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.086+0000 7f550377e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557b4ee49460 con 0x557b4ef3e830 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.086+0000 7f550377e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557b4ee9efe0 con 0x557b4ef3df10 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef46ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54fa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef3e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54fa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef3e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48328/0 (socket says 172.21.15.18:48328) 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef46ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34128/0 (socket says 172.21.15.18:34128) 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54fa800640 1 -- 172.21.15.18:0/1705921242 learned_addr learned my addr 172.21.15.18:0/1705921242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54f9400640 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef49800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54f2a00640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54e00a3d20 0x7f54e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54f2a00640 1 -- 172.21.15.18:0/3566462272 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f54ec0f23b0 con 0x7f54fc102040 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54f9e00640 1 -- 172.21.15.18:0/1705921242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 msgr2=0x557b4ef3e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54f9e00640 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef3e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.087+0000 7f54f9e00640 1 -- 172.21.15.18:0/1705921242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 msgr2=0x557b4ef49800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.088+0000 7f54f9e00640 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef49800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.088+0000 7f54f9e00640 1 -- 172.21.15.18:0/1705921242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4ee51040 con 0x557b4ef3e830 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.088+0000 7f54f9400640 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef49800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.088+0000 7f54fa800640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54e00a3d20 0x7f54e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.088+0000 7f54f9e00640 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef46ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0ef800 tx=0x7f54ec0327b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a15192015f56220 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.088+0000 7f54e7000640 1 -- 172.21.15.18:0/1705921242 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0bdb90 con 0x557b4ef3e830 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.088+0000 7f54e7000640 1 -- 172.21.15.18:0/1705921242 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec034640 con 0x557b4ef3e830 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.088+0000 7f54e7000640 1 -- 172.21.15.18:0/1705921242 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0bd9b0 con 0x557b4ef3e830 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.089+0000 7f550377e740 1 -- 172.21.15.18:0/1705921242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 msgr2=0x557b4ef46ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.089+0000 7f54fa800640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54e00a3d20 0x7f54e00a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54f4003e70 tx=0x7f54f4004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.089+0000 7f550377e740 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef46ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0ef800 tx=0x7f54ec0327b0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.090+0000 7f550377e740 1 -- 172.21.15.18:0/1705921242 shutdown_connections 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.090+0000 7f550377e740 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef49800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.090+0000 7f550377e740 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef46ed0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.090+0000 7f550377e740 1 --2- 172.21.15.18:0/1705921242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef3e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.090+0000 7f550377e740 1 -- 172.21.15.18:0/1705921242 >> 172.21.15.18:0/1705921242 conn(0x557b4ef38640 msgr2=0x557b4ef38a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.090+0000 7f550377e740 1 -- 172.21.15.18:0/1705921242 shutdown_connections 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.090+0000 7f550377e740 1 -- 172.21.15.18:0/1705921242 wait complete. 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.091+0000 7f550377e740 1 -- start start 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.091+0000 7f550377e740 1 Processor -- start 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.091+0000 7f550377e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3df10 0x557b4ef445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f550377e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 0x557b4ef44af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f550377e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef45030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f550377e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557b4ee49460 con 0x557b4ef47410 2024-04-16T17:07:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f550377e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557b4ee5c830 con 0x557b4ef3df10 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f550377e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557b4ef49fc0 con 0x557b4ef3e830 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f550377e740 10 client.0 _ll_register_callbacks cb 0x557b4ee64150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3df10 0x557b4ef445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3df10 0x557b4ef445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34132/0 (socket says 172.21.15.18:34132) 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54f9e00640 1 -- 172.21.15.18:0/4177190559 learned_addr learned my addr 172.21.15.18:0/4177190559 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54f9400640 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 0x557b4ef44af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54f9e00640 1 -- 172.21.15.18:0/4177190559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 msgr2=0x557b4ef44af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54f9e00640 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 0x557b4ef44af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54f9e00640 1 -- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 msgr2=0x557b4ef45030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54fa800640 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef45030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.092+0000 7f54f9e00640 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef45030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f54f9e00640 1 -- 172.21.15.18:0/4177190559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4ee60920 con 0x557b4ef3df10 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f54fa800640 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef45030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f54f9e00640 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3df10 0x557b4ef445b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000c00 tx=0x7f54ec0c6dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=175f758a9abe3329 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f54db400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f54db400640 1 -- 172.21.15.18:0/4177190559 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0be750 con 0x557b4ef3df10 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f550377e740 1 -- 172.21.15.18:0/4177190559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b4eeb0a40 con 0x557b4ef3df10 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f54db400640 1 -- 172.21.15.18:0/4177190559 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f54ec0c2080 con 0x557b4ef3df10 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f54db400640 1 -- 172.21.15.18:0/4177190559 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54ec0f2170 con 0x557b4ef3df10 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f550377e740 10 client.26441 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.093+0000 7f550377e740 1 -- 172.21.15.18:0/4177190559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b4ee2e7f0 con 0x557b4ef3df10 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f54db400640 1 -- 172.21.15.18:0/4177190559 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557b4eeb0a40 con 0x557b4ef3df10 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f54db400640 1 -- 172.21.15.18:0/4177190559 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54ec03c030 con 0x557b4ef3df10 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f550377e740 10 client.26441 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f550377e740 10 client.26441 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f550377e740 20 client.26441 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f550377e740 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557b4ef5fbf0 0x557b4ef62010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f550377e740 4 client.26441 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f550377e740 1 -- 172.21.15.18:0/4177190559 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4eeaa8b0 con 0x557b4ef5fbf0 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.094+0000 7f54f9400640 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557b4ef5fbf0 0x557b4ef62010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.095+0000 7f54f9400640 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557b4ef5fbf0 0x557b4ef62010 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9f07733cc84dd0 server_cookie=4295c1dbe17ea6d8 in_seq=0 out_seq=0 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.095+0000 7f54db400640 10 client.26441 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.096+0000 7f54db400640 1 -- 172.21.15.18:0/4177190559 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b4eeaa8b0 con 0x557b4ef5fbf0 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.096+0000 7f54db400640 10 client.26441 handle_command_reply: tid=0 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.096+0000 7f550377e740 1 client.26441 shutdown 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 -- 172.21.15.18:0/4177190559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3df10 msgr2=0x557b4ef445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3df10 0x557b4ef445b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000c00 tx=0x7f54ec0c6dd0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 -- 172.21.15.18:0/4177190559 shutdown_connections 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557b4ef5fbf0 0x557b4ef62010 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef45030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 0x557b4ef44af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f54db400640 0 client.26441 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:07:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 --2- 172.21.15.18:0/4177190559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3df10 0x557b4ef445b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 -- 172.21.15.18:0/4177190559 >> 172.21.15.18:0/4177190559 conn(0x557b4ef38640 msgr2=0x557b4ef3a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 -- 172.21.15.18:0/4177190559 shutdown_connections 2024-04-16T17:07:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 1 -- 172.21.15.18:0/4177190559 wait complete. 2024-04-16T17:07:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.097+0000 7f550377e740 20 client.26441 trim_cache size 0 max 16384 2024-04-16T17:07:43.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.120+0000 7f550377e740 1 Processor -- start 2024-04-16T17:07:43.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.120+0000 7f550377e740 1 -- start start 2024-04-16T17:07:43.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f550377e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef3df10 0x557b4ef52d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f550377e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 0x557b4ef44e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f550377e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef47410 0x557b4ef67840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f550377e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1afae0 con 0x557b4ef3df10 2024-04-16T17:07:43.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f550377e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1ad3b0 con 0x557b4ef47410 2024-04-16T17:07:43.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f550377e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1ab220 con 0x557b4ef3e830 2024-04-16T17:07:43.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef3df10 0x557b4ef52d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef3df10 0x557b4ef52d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56572/0 (socket says 172.21.15.18:56572) 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f54f9e00640 1 -- 172.21.15.18:0/1866520041 learned_addr learned my addr 172.21.15.18:0/1866520041 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f54fa800640 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef47410 0x557b4ef67840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.121+0000 7f54f9400640 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 0x557b4ef44e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54f9e00640 1 -- 172.21.15.18:0/1866520041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 msgr2=0x557b4ef44e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54f9e00640 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 0x557b4ef44e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54f9e00640 1 -- 172.21.15.18:0/1866520041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef47410 msgr2=0x557b4ef67840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54f9e00640 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef47410 0x557b4ef67840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54f9e00640 1 -- 172.21.15.18:0/1866520041 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4ebb53f0 con 0x557b4ef3df10 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54fa800640 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef47410 0x557b4ef67840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54f9e00640 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef3df10 0x557b4ef52d00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0bdd50 tx=0x7f54ec0c6e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470c54b8d8ec364e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54e6600640 1 -- 172.21.15.18:0/1866520041 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0ba2f0 con 0x557b4ef3df10 2024-04-16T17:07:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54e6600640 1 -- 172.21.15.18:0/1866520041 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec0be750 con 0x557b4ef3df10 2024-04-16T17:07:43.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.122+0000 7f54e6600640 1 -- 172.21.15.18:0/1866520041 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0bd9b0 con 0x557b4ef3df10 2024-04-16T17:07:43.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.123+0000 7f550377e740 1 -- 172.21.15.18:0/1866520041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef3df10 msgr2=0x557b4ef52d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.123+0000 7f550377e740 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef3df10 0x557b4ef52d00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0bdd50 tx=0x7f54ec0c6e40 comp rx=0 tx=0).stop 2024-04-16T17:07:43.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.123+0000 7f550377e740 1 -- 172.21.15.18:0/1866520041 shutdown_connections 2024-04-16T17:07:43.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.123+0000 7f550377e740 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef47410 0x557b4ef67840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.124+0000 7f550377e740 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3e830 0x557b4ef44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.124+0000 7f550377e740 1 --2- 172.21.15.18:0/1866520041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef3df10 0x557b4ef52d00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.124+0000 7f550377e740 1 -- 172.21.15.18:0/1866520041 >> 172.21.15.18:0/1866520041 conn(0x557b4ef38640 msgr2=0x557b4ef37700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.124+0000 7f550377e740 1 -- 172.21.15.18:0/1866520041 shutdown_connections 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.124+0000 7f550377e740 1 -- 172.21.15.18:0/1866520041 wait complete. 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.124+0000 7f550377e740 1 -- start start 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.124+0000 7f550377e740 1 Processor -- start 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f550377e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef4f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f550377e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef4a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f550377e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef4ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f550377e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1ab870 con 0x557b4ef47410 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f550377e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1acce0 con 0x557b4ef3e830 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f550377e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1abf40 con 0x557b4ef3df10 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f550377e740 10 client.0 _ll_register_callbacks cb 0x7f54e8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f54fa800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef4ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f54fa800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef4ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56578/0 (socket says 172.21.15.18:56578) 2024-04-16T17:07:43.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f54fa800640 1 -- 172.21.15.18:0/3546855049 learned_addr learned my addr 172.21.15.18:0/3546855049 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f54f9400640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef4a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.125+0000 7f54f9e00640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef4f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54fa800640 1 -- 172.21.15.18:0/3546855049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 msgr2=0x557b4ef4f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54fa800640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef4f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54fa800640 1 -- 172.21.15.18:0/3546855049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 msgr2=0x557b4ef4a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54fa800640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef4a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54fa800640 1 -- 172.21.15.18:0/3546855049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4ec7f7d0 con 0x557b4ef47410 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54f9e00640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef4f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54f9400640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef4a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54fa800640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef4ad30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54f40048a0 tx=0x7f54f4010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604fc8b02ac3bde9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54db400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54db400640 1 -- 172.21.15.18:0/3546855049 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f402b020 con 0x557b4ef47410 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f550377e740 1 -- 172.21.15.18:0/3546855049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ec0be750 con 0x557b4ef47410 2024-04-16T17:07:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.126+0000 7f54db400640 1 -- 172.21.15.18:0/3546855049 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f54f4027400 con 0x557b4ef47410 2024-04-16T17:07:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f54db400640 1 -- 172.21.15.18:0/3546855049 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54f4036050 con 0x557b4ef47410 2024-04-16T17:07:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f550377e740 10 client.16251 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f550377e740 1 -- 172.21.15.18:0/3546855049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b4efc5d20 con 0x557b4ef47410 2024-04-16T17:07:43.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f54db400640 1 -- 172.21.15.18:0/3546855049 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec0be750 con 0x557b4ef47410 2024-04-16T17:07:43.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f54db400640 1 -- 172.21.15.18:0/3546855049 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54f4026040 con 0x557b4ef47410 2024-04-16T17:07:43.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f550377e740 10 client.16251 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:43.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f550377e740 10 client.16251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f550377e740 20 client.16251 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f550377e740 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557b4efc6470 0x557b4efc8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f550377e740 4 client.16251 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f550377e740 1 -- 172.21.15.18:0/3546855049 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b4ee26570 con 0x557b4efc6470 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.127+0000 7f54f9e00640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557b4efc6470 0x557b4efc8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.128+0000 7f54f9e00640 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557b4efc6470 0x557b4efc8890 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33b6833f57ec2733 server_cookie=66d8842a063963e1 in_seq=0 out_seq=0 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.128+0000 7f54db400640 10 client.16251 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.128+0000 7f54db400640 1 -- 172.21.15.18:0/3546855049 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b4ee26570 con 0x557b4efc6470 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.128+0000 7f54db400640 10 client.16251 handle_command_reply: tid=0 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.128+0000 7f550377e740 1 client.16251 shutdown 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.128+0000 7f550377e740 1 -- 172.21.15.18:0/3546855049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 msgr2=0x557b4ef4ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.128+0000 7f550377e740 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef4ad30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54f40048a0 tx=0x7f54f4010040 comp rx=0 tx=0).stop 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 1 -- 172.21.15.18:0/3546855049 shutdown_connections 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557b4efc6470 0x557b4efc8890 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557b4ef47410 0x557b4ef4ad30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b4ef3e830 0x557b4ef4a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 1 --2- 172.21.15.18:0/3546855049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b4ef3df10 0x557b4ef4f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 1 -- 172.21.15.18:0/3546855049 >> 172.21.15.18:0/3546855049 conn(0x557b4ef38640 msgr2=0x557b4ef39800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f54db400640 0 client.16251 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:07:43.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 1 -- 172.21.15.18:0/3546855049 shutdown_connections 2024-04-16T17:07:43.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 1 -- 172.21.15.18:0/3546855049 wait complete. 2024-04-16T17:07:43.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.129+0000 7f550377e740 20 client.16251 trim_cache size 0 max 16384 2024-04-16T17:07:43.133 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 -- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54e00a3d20 msgr2=0x7f54e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54e00a3d20 0x7f54e00a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54f4003e70 tx=0x7f54f4004a40 comp rx=0 tx=0).stop 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 -- 172.21.15.18:0/3566462272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc102040 msgr2=0x7f54fc074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc102040 0x7f54fc074300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f54ec037040 tx=0x7f54ec0045b0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 -- 172.21.15.18:0/3566462272 shutdown_connections 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54e00a3d20 0x7f54e00a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54fc074d80 0x7f54fc1a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54fc107110 0x7f54fc074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 --2- 172.21.15.18:0/3566462272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc102040 0x7f54fc074300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 -- 172.21.15.18:0/3566462272 >> 172.21.15.18:0/3566462272 conn(0x7f54fc0fd8e0 msgr2=0x7f54fc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 -- 172.21.15.18:0/3566462272 shutdown_connections 2024-04-16T17:07:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.130+0000 7f54fb200640 1 -- 172.21.15.18:0/3566462272 wait complete. 2024-04-16T17:07:43.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:07:43.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.332+0000 7f9072e00640 1 -- 172.21.15.18:0/3884676728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 msgr2=0x7f906c10ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.332+0000 7f9072e00640 1 --2- 172.21.15.18:0/3884676728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 0x7f906c10ed10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9064009be0 tx=0x7f906402bef0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.332+0000 7f9072e00640 1 -- 172.21.15.18:0/3884676728 shutdown_connections 2024-04-16T17:07:43.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.332+0000 7f9072e00640 1 --2- 172.21.15.18:0/3884676728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 0x7f906c10ed10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.332+0000 7f9072e00640 1 --2- 172.21.15.18:0/3884676728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f906c102a20 0x7f906c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.332+0000 7f9072e00640 1 --2- 172.21.15.18:0/3884676728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c102070 0x7f906c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.333+0000 7f9072e00640 1 -- 172.21.15.18:0/3884676728 >> 172.21.15.18:0/3884676728 conn(0x7f906c0fd950 msgr2=0x7f906c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.333+0000 7f9072e00640 1 -- 172.21.15.18:0/3884676728 shutdown_connections 2024-04-16T17:07:43.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.333+0000 7f9072e00640 1 -- 172.21.15.18:0/3884676728 wait complete. 2024-04-16T17:07:43.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.334+0000 7f9072e00640 1 Processor -- start 2024-04-16T17:07:43.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.334+0000 7f9072e00640 1 -- start start 2024-04-16T17:07:43.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9072e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f906c102070 0x7f906c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9072e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 0x7f906c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9072e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c19a6e0 0x7f906c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9072e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f906c112230 con 0x7f906c107140 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9072e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1120b0 con 0x7f906c102070 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9072e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1123b0 con 0x7f906c19a6e0 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f906c102070 0x7f906c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f906c102070 0x7f906c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34150/0 (socket says 172.21.15.18:34150) 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071a00640 1 -- 172.21.15.18:0/322886037 learned_addr learned my addr 172.21.15.18:0/322886037 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071000640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 0x7f906c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9072400640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c19a6e0 0x7f906c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071a00640 1 -- 172.21.15.18:0/322886037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c19a6e0 msgr2=0x7f906c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071a00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c19a6e0 0x7f906c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071a00640 1 -- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 msgr2=0x7f906c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071a00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 0x7f906c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071a00640 1 -- 172.21.15.18:0/322886037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c19b2e0 con 0x7f906c102070 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9071000640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 0x7f906c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.335+0000 7f9072400640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c19a6e0 0x7f906c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.336+0000 7f9071a00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f906c102070 0x7f906c1a0620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f906000d950 tx=0x7f906000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.336+0000 7f906a000640 1 -- 172.21.15.18:0/322886037 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060017030 con 0x7f906c102070 2024-04-16T17:07:43.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.336+0000 7f9072e00640 1 -- 172.21.15.18:0/322886037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906c1a73c0 con 0x7f906c102070 2024-04-16T17:07:43.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.336+0000 7f906a000640 1 -- 172.21.15.18:0/322886037 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9060004020 con 0x7f906c102070 2024-04-16T17:07:43.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.336+0000 7f906a000640 1 -- 172.21.15.18:0/322886037 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90600047a0 con 0x7f906c102070 2024-04-16T17:07:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.336+0000 7f9072e00640 1 -- 172.21.15.18:0/322886037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906c1a77c0 con 0x7f906c102070 2024-04-16T17:07:43.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.339+0000 7f906a000640 1 -- 172.21.15.18:0/322886037 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f90600041d0 con 0x7f906c102070 2024-04-16T17:07:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.340+0000 7f90771b5740 1 Processor -- start 2024-04-16T17:07:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.340+0000 7f90771b5740 1 -- start start 2024-04-16T17:07:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f90771b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd73f10 0x5628cfd742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f90771b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f90771b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd7d410 0x5628cfd7f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f90771b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5628cfcd9020 con 0x5628cfd73f10 2024-04-16T17:07:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f90771b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5628cfc7f460 con 0x5628cfd7d410 2024-04-16T17:07:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f90771b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5628cfcd4fe0 con 0x5628cfd74830 2024-04-16T17:07:43.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f9072400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd73f10 0x5628cfd742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f9072400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd73f10 0x5628cfd742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56620/0 (socket says 172.21.15.18:56620) 2024-04-16T17:07:43.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f9072400640 1 -- 172.21.15.18:0/3660977093 learned_addr learned my addr 172.21.15.18:0/3660977093 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.341+0000 7f9071a00640 1 --2- 172.21.15.18:0/3660977093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f9072400640 1 -- 172.21.15.18:0/3660977093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 msgr2=0x5628cfd7ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f9072400640 1 --2- 172.21.15.18:0/3660977093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f9072400640 1 -- 172.21.15.18:0/3660977093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd7d410 msgr2=0x5628cfd7f800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f9072400640 1 --2- 172.21.15.18:0/3660977093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd7d410 0x5628cfd7f800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f9072400640 1 -- 172.21.15.18:0/3660977093 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628cfc87040 con 0x5628cfd73f10 2024-04-16T17:07:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f906a000640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f90540a3d20 0x7f90540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f9072400640 1 --2- 172.21.15.18:0/3660977093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd73f10 0x5628cfd742f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9064006010 tx=0x7f9064033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5473932b0db8e8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f906a000640 1 -- 172.21.15.18:0/322886037 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f90600d0710 con 0x7f906c102070 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f905ea00640 1 -- 172.21.15.18:0/3660977093 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064042070 con 0x5628cfd73f10 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f905ea00640 1 -- 172.21.15.18:0/3660977093 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064003970 con 0x5628cfd73f10 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.342+0000 7f905ea00640 1 -- 172.21.15.18:0/3660977093 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906402cdf0 con 0x5628cfd73f10 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 -- 172.21.15.18:0/3660977093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd73f10 msgr2=0x5628cfd742f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 --2- 172.21.15.18:0/3660977093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd73f10 0x5628cfd742f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9064006010 tx=0x7f9064033bf0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 -- 172.21.15.18:0/3660977093 shutdown_connections 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 --2- 172.21.15.18:0/3660977093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd7d410 0x5628cfd7f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 --2- 172.21.15.18:0/3660977093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 --2- 172.21.15.18:0/3660977093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd73f10 0x5628cfd742f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 -- 172.21.15.18:0/3660977093 >> 172.21.15.18:0/3660977093 conn(0x5628cfd6e640 msgr2=0x5628cfd6ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 -- 172.21.15.18:0/3660977093 shutdown_connections 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.343+0000 7f90771b5740 1 -- 172.21.15.18:0/3660977093 wait complete. 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.344+0000 7f9071a00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f90540a3d20 0x7f90540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.344+0000 7f90771b5740 1 -- start start 2024-04-16T17:07:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.344+0000 7f90771b5740 1 Processor -- start 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.344+0000 7f9071a00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f90540a3d20 0x7f90540a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5628cfd739e0 tx=0x7f90600d6530 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f90771b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd73f10 0x5628cfd7a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f90771b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f90771b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f90771b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5628cfd801a0 con 0x5628cfd7d410 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f90771b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5628cfc92830 con 0x5628cfd73f10 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f90771b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5628cfd80320 con 0x5628cfd74830 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f90771b5740 10 client.0 _ll_register_callbacks cb 0x5628cfc9a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f9072400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f9071000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f9071000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48374/0 (socket says 172.21.15.18:48374) 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f9072400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56636/0 (socket says 172.21.15.18:56636) 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.345+0000 7f9071000640 1 -- 172.21.15.18:0/749145177 learned_addr learned my addr 172.21.15.18:0/749145177 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9071000640 1 -- 172.21.15.18:0/749145177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd73f10 msgr2=0x5628cfd7a4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9071000640 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd73f10 0x5628cfd7a4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9071000640 1 -- 172.21.15.18:0/749145177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 msgr2=0x5628cfd7af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9071000640 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9071000640 1 -- 172.21.15.18:0/749145177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628cfc96920 con 0x5628cfd74830 2024-04-16T17:07:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9071000640 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7aa10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9058005fd0 tx=0x7f905800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=894398de44c62283 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9053400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9053400640 1 -- 172.21.15.18:0/749145177 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058015070 con 0x5628cfd74830 2024-04-16T17:07:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f90771b5740 1 -- 172.21.15.18:0/749145177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9064003970 con 0x5628cfd74830 2024-04-16T17:07:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.346+0000 7f9053400640 1 -- 172.21.15.18:0/749145177 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9058011650 con 0x5628cfd74830 2024-04-16T17:07:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.347+0000 7f9053400640 1 -- 172.21.15.18:0/749145177 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9058023050 con 0x5628cfd74830 2024-04-16T17:07:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.347+0000 7f90771b5740 10 client.26440 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.347+0000 7f90771b5740 1 -- 172.21.15.18:0/749145177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5628cfc647f0 con 0x5628cfd74830 2024-04-16T17:07:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.347+0000 7f9053400640 1 -- 172.21.15.18:0/749145177 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064003970 con 0x5628cfd74830 2024-04-16T17:07:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.347+0000 7f9053400640 1 -- 172.21.15.18:0/749145177 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f905801ac20 con 0x5628cfd74830 2024-04-16T17:07:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.348+0000 7f90771b5740 10 client.26440 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.348+0000 7f90771b5740 10 client.26440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.348+0000 7f90771b5740 20 client.26440 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.348+0000 7f90771b5740 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628cfd8d7f0 0x5628cfd8fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.348+0000 7f90771b5740 4 client.26440 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.348+0000 7f90771b5740 1 -- 172.21.15.18:0/749145177 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628cfce6a40 con 0x5628cfd8d7f0 2024-04-16T17:07:43.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.352+0000 7f9072400640 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628cfd8d7f0 0x5628cfd8fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.353+0000 7f9072400640 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628cfd8d7f0 0x5628cfd8fc70 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572b1b1ac15606d1 server_cookie=d58e0d5d6becf34 in_seq=0 out_seq=0 2024-04-16T17:07:43.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.353+0000 7f9053400640 10 client.26440 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.353+0000 7f9053400640 1 -- 172.21.15.18:0/749145177 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628cfce6a40 con 0x5628cfd8d7f0 2024-04-16T17:07:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.353+0000 7f9053400640 10 client.26440 handle_command_reply: tid=0 2024-04-16T17:07:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.353+0000 7f90771b5740 1 client.26440 shutdown 2024-04-16T17:07:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 -- 172.21.15.18:0/749145177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 msgr2=0x5628cfd7aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7aa10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9058005fd0 tx=0x7f905800f730 comp rx=0 tx=0).stop 2024-04-16T17:07:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 -- 172.21.15.18:0/749145177 shutdown_connections 2024-04-16T17:07:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628cfd8d7f0 0x5628cfd8fc70 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd7aa10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 --2- 172.21.15.18:0/749145177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd73f10 0x5628cfd7a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f9053400640 0 client.26440 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 -- 172.21.15.18:0/749145177 >> 172.21.15.18:0/749145177 conn(0x5628cfd6e640 msgr2=0x5628cfd70390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 -- 172.21.15.18:0/749145177 shutdown_connections 2024-04-16T17:07:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 1 -- 172.21.15.18:0/749145177 wait complete. 2024-04-16T17:07:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.354+0000 7f90771b5740 20 client.26440 trim_cache size 0 max 16384 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.379+0000 7f90771b5740 1 Processor -- start 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.379+0000 7f90771b5740 1 -- start start 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.379+0000 7f90771b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd73f10 0x5628cfd7cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.379+0000 7f90771b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd74830 0x5628cfd89090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.379+0000 7f90771b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.379+0000 7f90771b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1b3ef0 con 0x5628cfd7d410 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.379+0000 7f90771b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1b17c0 con 0x5628cfd74830 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.379+0000 7f90771b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1af610 con 0x5628cfd73f10 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56642/0 (socket says 172.21.15.18:56642) 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 -- 172.21.15.18:0/2704681966 learned_addr learned my addr 172.21.15.18:0/2704681966 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 --2- 172.21.15.18:0/2704681966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd73f10 0x5628cfd7cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 -- 172.21.15.18:0/2704681966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd73f10 msgr2=0x5628cfd7cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 --2- 172.21.15.18:0/2704681966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd73f10 0x5628cfd7cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 -- 172.21.15.18:0/2704681966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd74830 msgr2=0x5628cfd89090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 --2- 172.21.15.18:0/2704681966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd74830 0x5628cfd89090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 -- 172.21.15.18:0/2704681966 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628cfd79480 con 0x5628cfd7d410 2024-04-16T17:07:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.380+0000 7f9072400640 1 --2- 172.21.15.18:0/2704681966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7ad20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f90640096f0 tx=0x7f9064009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9d34e9b7866f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.381+0000 7f905e000640 1 -- 172.21.15.18:0/2704681966 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064042050 con 0x5628cfd7d410 2024-04-16T17:07:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.381+0000 7f905e000640 1 -- 172.21.15.18:0/2704681966 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064050d50 con 0x5628cfd7d410 2024-04-16T17:07:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.381+0000 7f905e000640 1 -- 172.21.15.18:0/2704681966 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064050430 con 0x5628cfd7d410 2024-04-16T17:07:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 -- 172.21.15.18:0/2704681966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 msgr2=0x5628cfd7ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 --2- 172.21.15.18:0/2704681966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7ad20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f90640096f0 tx=0x7f9064009500 comp rx=0 tx=0).stop 2024-04-16T17:07:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 -- 172.21.15.18:0/2704681966 shutdown_connections 2024-04-16T17:07:43.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 --2- 172.21.15.18:0/2704681966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd7ad20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 --2- 172.21.15.18:0/2704681966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd74830 0x5628cfd89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 --2- 172.21.15.18:0/2704681966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd73f10 0x5628cfd7cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 -- 172.21.15.18:0/2704681966 >> 172.21.15.18:0/2704681966 conn(0x5628cfd6e640 msgr2=0x5628cfd6ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 -- 172.21.15.18:0/2704681966 shutdown_connections 2024-04-16T17:07:43.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 -- 172.21.15.18:0/2704681966 wait complete. 2024-04-16T17:07:43.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 -- start start 2024-04-16T17:07:43.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.382+0000 7f90771b5740 1 Processor -- start 2024-04-16T17:07:43.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f90771b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd73f10 0x5628cfd85790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f90771b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd82570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f90771b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd82ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f90771b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1afc80 con 0x5628cfd7d410 2024-04-16T17:07:43.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f90771b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1b10f0 con 0x5628cfd73f10 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f90771b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1b0350 con 0x5628cfd74830 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f90771b5740 10 client.0 _ll_register_callbacks cb 0x7f905800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f9072400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd82ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f9071000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd82570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f9072400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd82ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56650/0 (socket says 172.21.15.18:56650) 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.383+0000 7f9072400640 1 -- 172.21.15.18:0/2860672882 learned_addr learned my addr 172.21.15.18:0/2860672882 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9072400640 1 -- 172.21.15.18:0/2860672882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 msgr2=0x5628cfd82570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9072400640 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd82570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9072400640 1 -- 172.21.15.18:0/2860672882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd73f10 msgr2=0x5628cfd85790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9072400640 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd73f10 0x5628cfd85790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9072400640 1 -- 172.21.15.18:0/2860672882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628cf9eb3f0 con 0x5628cfd7d410 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9071000640 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd82570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9072400640 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd82ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90640096c0 tx=0x7f906402bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb1a997eb273fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9053400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9053400640 1 -- 172.21.15.18:0/2860672882 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064042040 con 0x5628cfd7d410 2024-04-16T17:07:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f9053400640 1 -- 172.21.15.18:0/2860672882 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9064033680 con 0x5628cfd7d410 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.384+0000 7f90771b5740 1 -- 172.21.15.18:0/2860672882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9064050d50 con 0x5628cfd7d410 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f9053400640 1 -- 172.21.15.18:0/2860672882 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9064003b80 con 0x5628cfd7d410 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f90771b5740 10 client.16269 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f90771b5740 1 -- 172.21.15.18:0/2860672882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5628cfab57d0 con 0x5628cfd7d410 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f9053400640 1 -- 172.21.15.18:0/2860672882 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064050d50 con 0x5628cfd7d410 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f9053400640 1 -- 172.21.15.18:0/2860672882 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9064047450 con 0x5628cfd7d410 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f90771b5740 10 client.16269 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f90771b5740 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f90771b5740 20 client.16269 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f90771b5740 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628cfdf4960 0x5628cfdf6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f90771b5740 4 client.16269 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.385+0000 7f90771b5740 1 -- 172.21.15.18:0/2860672882 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628cfce08b0 con 0x5628cfdf4960 2024-04-16T17:07:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.386+0000 7f9071000640 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628cfdf4960 0x5628cfdf6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.386+0000 7f9071000640 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628cfdf4960 0x5628cfdf6d50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b15825965eb9a41b server_cookie=7ad20387788521e5 in_seq=0 out_seq=0 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.386+0000 7f9053400640 10 client.16269 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.387+0000 7f9053400640 1 -- 172.21.15.18:0/2860672882 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628cfce08b0 con 0x5628cfdf4960 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.387+0000 7f9053400640 10 client.16269 handle_command_reply: tid=0 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.387+0000 7f90771b5740 1 client.16269 shutdown 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.387+0000 7f90771b5740 1 -- 172.21.15.18:0/2860672882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 msgr2=0x5628cfd82ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.387+0000 7f90771b5740 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd82ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90640096c0 tx=0x7f906402bef0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 1 -- 172.21.15.18:0/2860672882 shutdown_connections 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628cfdf4960 0x5628cfdf6d50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5628cfd7d410 0x5628cfd82ab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5628cfd74830 0x5628cfd82570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 1 --2- 172.21.15.18:0/2860672882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628cfd73f10 0x5628cfd85790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 1 -- 172.21.15.18:0/2860672882 >> 172.21.15.18:0/2860672882 conn(0x5628cfd6e640 msgr2=0x5628cfd6d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f9053400640 0 client.16269 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:43.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 1 -- 172.21.15.18:0/2860672882 shutdown_connections 2024-04-16T17:07:43.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 1 -- 172.21.15.18:0/2860672882 wait complete. 2024-04-16T17:07:43.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.388+0000 7f90771b5740 20 client.16269 trim_cache size 0 max 16384 2024-04-16T17:07:43.391 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 -- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f90540a3d20 msgr2=0x7f90540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f90540a3d20 0x7f90540a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5628cfd739e0 tx=0x7f90600d6530 comp rx=0 tx=0).stop 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 -- 172.21.15.18:0/322886037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f906c102070 msgr2=0x7f906c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f906c102070 0x7f906c1a0620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f906000d950 tx=0x7f906000de10 comp rx=0 tx=0).stop 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 -- 172.21.15.18:0/322886037 shutdown_connections 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f90540a3d20 0x7f90540a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c19a6e0 0x7f906c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f906c107140 0x7f906c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 --2- 172.21.15.18:0/322886037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f906c102070 0x7f906c1a0620 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 -- 172.21.15.18:0/322886037 >> 172.21.15.18:0/322886037 conn(0x7f906c0fd950 msgr2=0x7f906c10d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 -- 172.21.15.18:0/322886037 shutdown_connections 2024-04-16T17:07:43.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:43.389+0000 7f9072e00640 1 -- 172.21.15.18:0/322886037 wait complete. 2024-04-16T17:07:43.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 89286 4.30 MB/sec warmup 116 sec latency 122.779 ms 2024-04-16T17:07:43.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:43 smithi022 ceph-mon[36023]: pgmap v92: 129 pgs: 129 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 804 KiB/s wr, 130 op/s 2024-04-16T17:07:43.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:43 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3897765893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:44.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:43 smithi043 ceph-mon[36901]: pgmap v92: 129 pgs: 129 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 804 KiB/s wr, 130 op/s 2024-04-16T17:07:44.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:43 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3897765893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:44.407 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:07:44.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:07:44.408 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:07:44.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 89636 4.27 MB/sec warmup 117 sec latency 111.974 ms 2024-04-16T17:07:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.600+0000 7faeefe00640 1 -- 172.21.15.18:0/2798265741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104df0 msgr2=0x7faee8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.600+0000 7faeefe00640 1 --2- 172.21.15.18:0/2798265741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104df0 0x7faee8108e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faed4009900 tx=0x7faed402bc60 comp rx=0 tx=0).stop 2024-04-16T17:07:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.600+0000 7faeefe00640 1 -- 172.21.15.18:0/2798265741 shutdown_connections 2024-04-16T17:07:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.600+0000 7faeefe00640 1 --2- 172.21.15.18:0/2798265741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faee8109510 0x7faee81110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.600+0000 7faeefe00640 1 --2- 172.21.15.18:0/2798265741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104df0 0x7faee8108e30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.600+0000 7faeefe00640 1 --2- 172.21.15.18:0/2798265741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee8104440 0x7faee8104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.600+0000 7faeefe00640 1 -- 172.21.15.18:0/2798265741 >> 172.21.15.18:0/2798265741 conn(0x7faee80ffc80 msgr2=0x7faee81020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.601+0000 7faeefe00640 1 -- 172.21.15.18:0/2798265741 shutdown_connections 2024-04-16T17:07:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.601+0000 7faeefe00640 1 -- 172.21.15.18:0/2798265741 wait complete. 2024-04-16T17:07:44.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.602+0000 7faeefe00640 1 Processor -- start 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.602+0000 7faeefe00640 1 -- start start 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeefe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104440 0x7faee8076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeefe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faee8109510 0x7faee8076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee8076f90 0x7faee81ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeefe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faee8114620 con 0x7faee8109510 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeefe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faee81144a0 con 0x7faee8104440 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeefe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faee81147a0 con 0x7faee8076f90 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeed400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104440 0x7faee8076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeed400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104440 0x7faee8076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34206/0 (socket says 172.21.15.18:34206) 2024-04-16T17:07:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeed400640 1 -- 172.21.15.18:0/1034117682 learned_addr learned my addr 172.21.15.18:0/1034117682 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeeca00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faee8109510 0x7faee8076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.603+0000 7faeede00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee8076f90 0x7faee81ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeed400640 1 -- 172.21.15.18:0/1034117682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee8076f90 msgr2=0x7faee81ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeed400640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee8076f90 0x7faee81ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeed400640 1 -- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faee8109510 msgr2=0x7faee8076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeed400640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faee8109510 0x7faee8076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeed400640 1 -- 172.21.15.18:0/1034117682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee81ab670 con 0x7faee8104440 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeede00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee8076f90 0x7faee81ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeed400640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104440 0x7faee8076510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faedc00b450 tx=0x7faedc00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faee6000640 1 -- 172.21.15.18:0/1034117682 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc015020 con 0x7faee8104440 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faee6000640 1 -- 172.21.15.18:0/1034117682 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faedc011d70 con 0x7faee8104440 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeefe00640 1 -- 172.21.15.18:0/1034117682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee81ab960 con 0x7faee8104440 2024-04-16T17:07:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faee6000640 1 -- 172.21.15.18:0/1034117682 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc00f340 con 0x7faee8104440 2024-04-16T17:07:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faeefe00640 1 -- 172.21.15.18:0/1034117682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee81abed0 con 0x7faee8104440 2024-04-16T17:07:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faee6000640 1 -- 172.21.15.18:0/1034117682 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faedc018020 con 0x7faee8104440 2024-04-16T17:07:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faef2bb7740 1 Processor -- start 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.604+0000 7faef2bb7740 1 -- start start 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fb2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faef2bb7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55be07f16cb0 con 0x55be07fbb1a0 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faef2bb7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55be07ebd370 con 0x55be07fb25c0 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faef2bb7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be07f12c70 con 0x55be07fb1ca0 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fb2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fb2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48422/0 (socket says 172.21.15.18:48422) 2024-04-16T17:07:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeed400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 -- 172.21.15.18:0/2331669096 learned_addr learned my addr 172.21.15.18:0/2331669096 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeeca00640 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faee6000640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faed00a3da0 0x7faed00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 -- 172.21.15.18:0/2331669096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 msgr2=0x55be07fbac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 -- 172.21.15.18:0/2331669096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 msgr2=0x55be07fbd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 -- 172.21.15.18:0/2331669096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be07edc910 con 0x55be07fb1ca0 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeed400640 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faee6000640 1 -- 172.21.15.18:0/1034117682 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7faedc0d0600 con 0x7faee8104440 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faed00a3da0 0x7faed00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeede00640 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fb2080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faee0009fd0 tx=0x7faee00108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2852ea5915e3f563 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeda600640 1 -- 172.21.15.18:0/2331669096 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0011ac0 con 0x55be07fb1ca0 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeda600640 1 -- 172.21.15.18:0/2331669096 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee0011ce0 con 0x55be07fb1ca0 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.609+0000 7faeda600640 1 -- 172.21.15.18:0/2331669096 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee001a8c0 con 0x55be07fb1ca0 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- 172.21.15.18:0/2331669096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 msgr2=0x55be07fb2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fb2080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faee0009fd0 tx=0x7faee00108e0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeede00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faed00a3da0 0x7faed00a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faee001d040 tx=0x7faee001ab40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- 172.21.15.18:0/2331669096 shutdown_connections 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 --2- 172.21.15.18:0/2331669096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fb2080 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- 172.21.15.18:0/2331669096 >> 172.21.15.18:0/2331669096 conn(0x55be07fac3d0 msgr2=0x55be07fac7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- 172.21.15.18:0/2331669096 shutdown_connections 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- 172.21.15.18:0/2331669096 wait complete. 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- start start 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 Processor -- start 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 0x55be07fb8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fb25c0 0x55be07fb86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fbb1a0 0x55be07fb8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55be07fbdf90 con 0x55be07fb25c0 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55be07ed0730 con 0x55be07fb1ca0 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be07fbe110 con 0x55be07fbb1a0 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 10 client.0 _ll_register_callbacks cb 0x55be07e9f1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeeca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fb25c0 0x55be07fb86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeeca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fb25c0 0x55be07fb86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56696/0 (socket says 172.21.15.18:56696) 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeeca00640 1 -- 172.21.15.18:0/4250361143 learned_addr learned my addr 172.21.15.18:0/4250361143 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeed400640 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 0x55be07fb8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeed400640 1 -- 172.21.15.18:0/4250361143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fbb1a0 msgr2=0x55be07fb8bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeed400640 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fbb1a0 0x55be07fb8bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeed400640 1 -- 172.21.15.18:0/4250361143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fb25c0 msgr2=0x55be07fb86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeed400640 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fb25c0 0x55be07fb86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeed400640 1 -- 172.21.15.18:0/4250361143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be07cf17e0 con 0x55be07fb1ca0 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faeed400640 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 0x55be07fb8170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faedc0cda50 tx=0x7faedc009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28054648447dd529 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faecea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faecea00640 1 -- 172.21.15.18:0/4250361143 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc09bde0 con 0x55be07fb1ca0 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faecea00640 1 -- 172.21.15.18:0/4250361143 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7faedc0d0190 con 0x55be07fb1ca0 2024-04-16T17:07:44.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- 172.21.15.18:0/4250361143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faee0011ce0 con 0x55be07fb1ca0 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faecea00640 1 -- 172.21.15.18:0/4250361143 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faedc09bc00 con 0x55be07fb1ca0 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 10 client.26465 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- 172.21.15.18:0/4250361143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be07d51f70 con 0x55be07fb1ca0 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faecea00640 1 -- 172.21.15.18:0/4250361143 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee0011ce0 con 0x55be07fb1ca0 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faecea00640 1 -- 172.21.15.18:0/4250361143 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faedc0b5020 con 0x55be07fb1ca0 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 10 client.26465 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 20 client.26465 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55be07fcb520 0x55be07fcd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 4 client.26465 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.612+0000 7faef2bb7740 1 -- 172.21.15.18:0/4250361143 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be07f246d0 con 0x55be07fcb520 2024-04-16T17:07:44.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faeeca00640 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55be07fcb520 0x55be07fcd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faeeca00640 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55be07fcb520 0x55be07fcd970 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7119e9c0f4509c7 server_cookie=9ad978b90ba5fadb in_seq=0 out_seq=0 2024-04-16T17:07:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faecea00640 10 client.26465 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faecea00640 1 -- 172.21.15.18:0/4250361143 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be07f246d0 con 0x55be07fcb520 2024-04-16T17:07:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faecea00640 10 client.26465 handle_command_reply: tid=0 2024-04-16T17:07:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 client.26465 shutdown 2024-04-16T17:07:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 -- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 msgr2=0x55be07fb8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 0x55be07fb8170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faedc0cda50 tx=0x7faedc009370 comp rx=0 tx=0).stop 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 -- 172.21.15.18:0/4250361143 shutdown_connections 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55be07fcb520 0x55be07fcd970 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fbb1a0 0x55be07fb8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fb25c0 0x55be07fb86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faecea00640 0 client.26465 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 --2- 172.21.15.18:0/4250361143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 0x55be07fb8170 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 -- 172.21.15.18:0/4250361143 >> 172.21.15.18:0/4250361143 conn(0x55be07fac3d0 msgr2=0x55be07fae120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 -- 172.21.15.18:0/4250361143 shutdown_connections 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 1 -- 172.21.15.18:0/4250361143 wait complete. 2024-04-16T17:07:44.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.617+0000 7faef2bb7740 20 client.26465 trim_cache size 0 max 16384 2024-04-16T17:07:44.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faef2bb7740 1 Processor -- start 2024-04-16T17:07:44.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faef2bb7740 1 -- start start 2024-04-16T17:07:44.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 0x55be07fc6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb25c0 0x55be07fb8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fd3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faef2bb7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b8480 con 0x55be07fbb1a0 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faef2bb7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b5d50 con 0x55be07fb1ca0 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faef2bb7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b3ba0 con 0x55be07fb25c0 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faeede00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fd3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faeede00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fd3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56704/0 (socket says 172.21.15.18:56704) 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.658+0000 7faeede00640 1 -- 172.21.15.18:0/3987709896 learned_addr learned my addr 172.21.15.18:0/3987709896 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.659+0000 7faeede00640 1 -- 172.21.15.18:0/3987709896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb25c0 msgr2=0x55be07fb8a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.659+0000 7faeede00640 1 --2- 172.21.15.18:0/3987709896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb25c0 0x55be07fb8a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.659+0000 7faeede00640 1 -- 172.21.15.18:0/3987709896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 msgr2=0x55be07fc6e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.659+0000 7faeede00640 1 --2- 172.21.15.18:0/3987709896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 0x55be07fc6e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.659+0000 7faeede00640 1 -- 172.21.15.18:0/3987709896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be07be9170 con 0x55be07fbb1a0 2024-04-16T17:07:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.659+0000 7faeede00640 1 --2- 172.21.15.18:0/3987709896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fd3240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faee00097c0 tx=0x7faee0027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b48065f7e167cce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.659+0000 7faed9c00640 1 -- 172.21.15.18:0/3987709896 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0036070 con 0x55be07fbb1a0 2024-04-16T17:07:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.660+0000 7faed9c00640 1 -- 172.21.15.18:0/3987709896 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee001ac70 con 0x55be07fbb1a0 2024-04-16T17:07:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.660+0000 7faed9c00640 1 -- 172.21.15.18:0/3987709896 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee002d040 con 0x55be07fbb1a0 2024-04-16T17:07:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.660+0000 7faef2bb7740 1 -- 172.21.15.18:0/3987709896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 msgr2=0x55be07fd3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.660+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3987709896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fd3240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faee00097c0 tx=0x7faee0027000 comp rx=0 tx=0).stop 2024-04-16T17:07:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.660+0000 7faef2bb7740 1 -- 172.21.15.18:0/3987709896 shutdown_connections 2024-04-16T17:07:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.660+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3987709896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fd3240 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.660+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3987709896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb25c0 0x55be07fb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.660+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3987709896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb1ca0 0x55be07fc6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.661+0000 7faef2bb7740 1 -- 172.21.15.18:0/3987709896 >> 172.21.15.18:0/3987709896 conn(0x55be07fac3d0 msgr2=0x55be07fae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.661+0000 7faef2bb7740 1 -- 172.21.15.18:0/3987709896 shutdown_connections 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.661+0000 7faef2bb7740 1 -- 172.21.15.18:0/3987709896 wait complete. 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.661+0000 7faef2bb7740 1 -- start start 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.661+0000 7faef2bb7740 1 Processor -- start 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fbf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faef2bb7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faef2bb7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b4210 con 0x55be07fbb1a0 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faef2bb7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b5680 con 0x55be07fb25c0 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faef2bb7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faee81b48e0 con 0x55be07fb1ca0 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faef2bb7740 10 client.0 _ll_register_callbacks cb 0x7faedc0d3da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faeeca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faeeca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34250/0 (socket says 172.21.15.18:34250) 2024-04-16T17:07:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faeeca00640 1 -- 172.21.15.18:0/3693969474 learned_addr learned my addr 172.21.15.18:0/3693969474 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faeed400640 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fbf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.662+0000 7faeede00640 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faeed400640 1 -- 172.21.15.18:0/3693969474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 msgr2=0x55be07fbf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faeed400640 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faeed400640 1 -- 172.21.15.18:0/3693969474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 msgr2=0x55be07fbfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faeed400640 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faeed400640 1 -- 172.21.15.18:0/3693969474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be08030dd0 con 0x55be07fb1ca0 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faeede00640 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faeed400640 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fbf030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faedc09a550 tx=0x7faedc0cdee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252c75357ed4eb65 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faecea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:44.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faecea00640 1 -- 172.21.15.18:0/3693969474 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc09bde0 con 0x55be07fb1ca0 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faecea00640 1 -- 172.21.15.18:0/3693969474 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7faedc09b6f0 con 0x55be07fb1ca0 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faef2bb7740 1 -- 172.21.15.18:0/3693969474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faee001ac70 con 0x55be07fb1ca0 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.663+0000 7faecea00640 1 -- 172.21.15.18:0/3693969474 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faedc09c9a0 con 0x55be07fb1ca0 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faef2bb7740 10 client.26455 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faef2bb7740 1 -- 172.21.15.18:0/3693969474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be080320b0 con 0x55be07fb1ca0 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faecea00640 1 -- 172.21.15.18:0/3693969474 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee001ac70 con 0x55be07fb1ca0 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faecea00640 1 -- 172.21.15.18:0/3693969474 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faedc0b5070 con 0x55be07fb1ca0 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faef2bb7740 10 client.26455 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faef2bb7740 10 client.26455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faef2bb7740 20 client.26455 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55be08032820 0x55be08034c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faef2bb7740 4 client.26455 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:07:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faef2bb7740 1 -- 172.21.15.18:0/3693969474 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55be07f1e540 con 0x55be08032820 2024-04-16T17:07:44.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.664+0000 7faeeca00640 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55be08032820 0x55be08034c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.665+0000 7faeeca00640 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55be08032820 0x55be08034c10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f005816db2d63a7d server_cookie=5336cc59986a3a53 in_seq=0 out_seq=0 2024-04-16T17:07:44.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.665+0000 7faecea00640 10 client.26455 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:44.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faecea00640 1 -- 172.21.15.18:0/3693969474 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55be07f1e540 con 0x55be08032820 2024-04-16T17:07:44.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faecea00640 10 client.26455 handle_command_reply: tid=0 2024-04-16T17:07:44.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 client.26455 shutdown 2024-04-16T17:07:44.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 -- 172.21.15.18:0/3693969474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 msgr2=0x55be07fbf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fbf030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faedc09a550 tx=0x7faedc0cdee0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 -- 172.21.15.18:0/3693969474 shutdown_connections 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55be08032820 0x55be08034c10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55be07fbb1a0 0x55be07fbfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be07fb25c0 0x55be07fbf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 --2- 172.21.15.18:0/3693969474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be07fb1ca0 0x55be07fbf030 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faecea00640 0 client.26455 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.666+0000 7faef2bb7740 1 -- 172.21.15.18:0/3693969474 >> 172.21.15.18:0/3693969474 conn(0x55be07fac3d0 msgr2=0x55be07faaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faef2bb7740 1 -- 172.21.15.18:0/3693969474 shutdown_connections 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faef2bb7740 1 -- 172.21.15.18:0/3693969474 wait complete. 2024-04-16T17:07:44.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faef2bb7740 20 client.26455 trim_cache size 0 max 16384 2024-04-16T17:07:44.673 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:07:44.673 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:07:44.673 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "a5c4f732-1bb6-4046-a825-eb02980c42bd", 2024-04-16T17:07:44.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:07:44.673 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:07:44.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 -- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faed00a3da0 msgr2=0x7faed00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faed00a3da0 0x7faed00a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faee001d040 tx=0x7faee001ab40 comp rx=0 tx=0).stop 2024-04-16T17:07:44.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 -- 172.21.15.18:0/1034117682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104440 msgr2=0x7faee8076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104440 0x7faee8076510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faedc00b450 tx=0x7faedc00b910 comp rx=0 tx=0).stop 2024-04-16T17:07:44.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 -- 172.21.15.18:0/1034117682 shutdown_connections 2024-04-16T17:07:44.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faed00a3da0 0x7faed00a6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee8076f90 0x7faee81ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faee8109510 0x7faee8076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 --2- 172.21.15.18:0/1034117682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8104440 0x7faee8076510 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.667+0000 7faeefe00640 1 -- 172.21.15.18:0/1034117682 >> 172.21.15.18:0/1034117682 conn(0x7faee80ffc80 msgr2=0x7faee8101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.668+0000 7faeefe00640 1 -- 172.21.15.18:0/1034117682 shutdown_connections 2024-04-16T17:07:44.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.668+0000 7faeefe00640 1 -- 172.21.15.18:0/1034117682 wait complete. 2024-04-16T17:07:44.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:07:44.691 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.891+0000 7fdb42800640 1 -- 172.21.15.18:0/349228996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb3c102a20 msgr2=0x7fdb3c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.891+0000 7fdb42800640 1 --2- 172.21.15.18:0/349228996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb3c102a20 0x7fdb3c106a60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009900 tx=0x7fdb2802bc40 comp rx=0 tx=0).stop 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.892+0000 7fdb42800640 1 -- 172.21.15.18:0/349228996 shutdown_connections 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.892+0000 7fdb42800640 1 --2- 172.21.15.18:0/349228996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb3c107140 0x7fdb3c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.892+0000 7fdb42800640 1 --2- 172.21.15.18:0/349228996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb3c102a20 0x7fdb3c106a60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.892+0000 7fdb42800640 1 --2- 172.21.15.18:0/349228996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3c102070 0x7fdb3c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.892+0000 7fdb42800640 1 -- 172.21.15.18:0/349228996 >> 172.21.15.18:0/349228996 conn(0x7fdb3c0fd950 msgr2=0x7fdb3c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.892+0000 7fdb42800640 1 -- 172.21.15.18:0/349228996 shutdown_connections 2024-04-16T17:07:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.892+0000 7fdb42800640 1 -- 172.21.15.18:0/349228996 wait complete. 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.893+0000 7fdb42800640 1 Processor -- start 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.893+0000 7fdb42800640 1 -- start start 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.893+0000 7fdb42800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3c102070 0x7fdb3c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.893+0000 7fdb42800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb3c107140 0x7fdb3c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.893+0000 7fdb42800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb3c19a750 0x7fdb3c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.893+0000 7fdb42800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1121b0 con 0x7fdb3c107140 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.893+0000 7fdb42800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c112030 con 0x7fdb3c19a750 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.893+0000 7fdb42800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c112330 con 0x7fdb3c102070 2024-04-16T17:07:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3c102070 0x7fdb3c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3c102070 0x7fdb3c1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48450/0 (socket says 172.21.15.18:48450) 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 -- 172.21.15.18:0/2667131907 learned_addr learned my addr 172.21.15.18:0/2667131907 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb40a00640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb3c107140 0x7fdb3c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41e00640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb3c19a750 0x7fdb3c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 -- 172.21.15.18:0/2667131907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb3c19a750 msgr2=0x7fdb3c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb3c19a750 0x7fdb3c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 -- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb3c107140 msgr2=0x7fdb3c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb3c107140 0x7fdb3c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 -- 172.21.15.18:0/2667131907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c19b350 con 0x7fdb3c102070 2024-04-16T17:07:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.894+0000 7fdb41400640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3c102070 0x7fdb3c1a0690 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000ea60 tx=0x7fdb3000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.895+0000 7fdb3a000640 1 -- 172.21.15.18:0/2667131907 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000cde0 con 0x7fdb3c102070 2024-04-16T17:07:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.895+0000 7fdb42800640 1 -- 172.21.15.18:0/2667131907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c1a7430 con 0x7fdb3c102070 2024-04-16T17:07:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.895+0000 7fdb3a000640 1 -- 172.21.15.18:0/2667131907 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb300035f0 con 0x7fdb3c102070 2024-04-16T17:07:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.895+0000 7fdb3a000640 1 -- 172.21.15.18:0/2667131907 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30010670 con 0x7fdb3c102070 2024-04-16T17:07:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.895+0000 7fdb42800640 1 -- 172.21.15.18:0/2667131907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c1a7830 con 0x7fdb3c102070 2024-04-16T17:07:44.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.899+0000 7fdb3a000640 1 -- 172.21.15.18:0/2667131907 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb300037a0 con 0x7fdb3c102070 2024-04-16T17:07:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.900+0000 7fdb46a06740 1 Processor -- start 2024-04-16T17:07:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.900+0000 7fdb46a06740 1 -- start start 2024-04-16T17:07:44.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.900+0000 7fdb3a000640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdb240a3da0 0x7fdb240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.900+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a91a0 0x55fced8a9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.900+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a9b50 0x55fced8bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.900+0000 7fdb3a000640 1 -- 172.21.15.18:0/2667131907 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fdb300d0fa0 con 0x7fdb3c102070 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8bd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb46a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fced80f120 con 0x55fced8a91a0 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb46a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fced7b5340 con 0x55fced8bb0a0 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb46a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fced80b0e0 con 0x55fced8a9b50 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb40a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a9b50 0x55fced8bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb40a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a9b50 0x55fced8bab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48464/0 (socket says 172.21.15.18:48464) 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb40a00640 1 -- 172.21.15.18:0/1302416806 learned_addr learned my addr 172.21.15.18:0/1302416806 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb41e00640 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8bd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb41400640 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a91a0 0x55fced8a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.901+0000 7fdb41e00640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdb240a3da0 0x7fdb240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb40a00640 1 -- 172.21.15.18:0/1302416806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 msgr2=0x55fced8bd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb40a00640 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8bd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb40a00640 1 -- 172.21.15.18:0/1302416806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a91a0 msgr2=0x55fced8a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb40a00640 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a91a0 0x55fced8a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb40a00640 1 -- 172.21.15.18:0/1302416806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fced5eb7d0 con 0x55fced8a9b50 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb41e00640 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8bd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb41400640 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a91a0 0x55fced8a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb40a00640 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a9b50 0x55fced8bab60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009900 tx=0x7fdb28005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345695296f5c137b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.902+0000 7fdb41e00640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdb240a3da0 0x7fdb240a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55fced8a4ee0 tx=0x7fdb34009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.903+0000 7fdb2e600640 1 -- 172.21.15.18:0/1302416806 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2802ca90 con 0x55fced8a9b50 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.903+0000 7fdb2e600640 1 -- 172.21.15.18:0/1302416806 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2802cc30 con 0x55fced8a9b50 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.903+0000 7fdb2e600640 1 -- 172.21.15.18:0/1302416806 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28005650 con 0x55fced8a9b50 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.903+0000 7fdb46a06740 1 -- 172.21.15.18:0/1302416806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a9b50 msgr2=0x55fced8bab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.903+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a9b50 0x55fced8bab60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009900 tx=0x7fdb28005d50 comp rx=0 tx=0).stop 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.904+0000 7fdb46a06740 1 -- 172.21.15.18:0/1302416806 shutdown_connections 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.904+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8bd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.904+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a9b50 0x55fced8bab60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.904+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1302416806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a91a0 0x55fced8a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.904+0000 7fdb46a06740 1 -- 172.21.15.18:0/1302416806 >> 172.21.15.18:0/1302416806 conn(0x55fced8a3ac0 msgr2=0x55fced8a3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.904+0000 7fdb46a06740 1 -- 172.21.15.18:0/1302416806 shutdown_connections 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.904+0000 7fdb46a06740 1 -- 172.21.15.18:0/1302416806 wait complete. 2024-04-16T17:07:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.904+0000 7fdb46a06740 1 -- start start 2024-04-16T17:07:44.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb46a06740 1 Processor -- start 2024-04-16T17:07:44.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a91a0 0x55fced8b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8bb0a0 0x55fced8b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb46a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fced8bde00 con 0x55fced8a9b50 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb46a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fced7c8710 con 0x55fced8a91a0 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb46a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fced8bdf80 con 0x55fced8bb0a0 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb46a06740 10 client.0 _ll_register_callbacks cb 0x55fced7c3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb41400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a91a0 0x55fced8b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb41400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a91a0 0x55fced8b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34286/0 (socket says 172.21.15.18:34286) 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb41400640 1 -- 172.21.15.18:0/1709878352 learned_addr learned my addr 172.21.15.18:0/1709878352 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.905+0000 7fdb40a00640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb41400640 1 -- 172.21.15.18:0/1709878352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8bb0a0 msgr2=0x55fced8b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb41e00640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8bb0a0 0x55fced8b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb41400640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8bb0a0 0x55fced8b8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb41400640 1 -- 172.21.15.18:0/1709878352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 msgr2=0x55fced8b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb41400640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb41400640 1 -- 172.21.15.18:0/1709878352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fced5213f0 con 0x55fced8a91a0 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb41e00640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8bb0a0 0x55fced8b8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb40a00640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb41400640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a91a0 0x55fced8b8200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb300ce340 tx=0x7fdb3009cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8575d915e2caa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb22a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb22a00640 1 -- 172.21.15.18:0/1709878352 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb300b6060 con 0x55fced8a91a0 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb22a00640 1 -- 172.21.15.18:0/1709878352 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fdb30003b30 con 0x55fced8a91a0 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.906+0000 7fdb46a06740 1 -- 172.21.15.18:0/1709878352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2802cc30 con 0x55fced8a91a0 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.907+0000 7fdb22a00640 1 -- 172.21.15.18:0/1709878352 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb3009de60 con 0x55fced8a91a0 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.907+0000 7fdb46a06740 10 client.26480 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.907+0000 7fdb46a06740 1 -- 172.21.15.18:0/1709878352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fced4e10f0 con 0x55fced8a91a0 2024-04-16T17:07:44.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.907+0000 7fdb22a00640 1 -- 172.21.15.18:0/1709878352 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2802cc30 con 0x55fced8a91a0 2024-04-16T17:07:44.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.908+0000 7fdb22a00640 1 -- 172.21.15.18:0/1709878352 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdb3009bf80 con 0x55fced8a91a0 2024-04-16T17:07:44.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.908+0000 7fdb46a06740 10 client.26480 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:44.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.908+0000 7fdb46a06740 10 client.26480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.908+0000 7fdb46a06740 20 client.26480 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.908+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fced8cb4d0 0x55fced8cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.908+0000 7fdb46a06740 4 client.26480 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.908+0000 7fdb46a06740 1 -- 172.21.15.18:0/1709878352 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fced81cb40 con 0x55fced8cb4d0 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.908+0000 7fdb40a00640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fced8cb4d0 0x55fced8cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.909+0000 7fdb22a00640 10 client.26480 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.909+0000 7fdb40a00640 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fced8cb4d0 0x55fced8cd900 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a895ca0564e7f server_cookie=7f09b587b0b4dbde in_seq=0 out_seq=0 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.910+0000 7fdb22a00640 1 -- 172.21.15.18:0/1709878352 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fced81cb40 con 0x55fced8cb4d0 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.910+0000 7fdb22a00640 10 client.26480 handle_command_reply: tid=0 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.910+0000 7fdb46a06740 1 client.26480 shutdown 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.910+0000 7fdb46a06740 1 -- 172.21.15.18:0/1709878352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a91a0 msgr2=0x55fced8b8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.910+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a91a0 0x55fced8b8200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb300ce340 tx=0x7fdb3009cb30 comp rx=0 tx=0).stop 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.910+0000 7fdb46a06740 1 -- 172.21.15.18:0/1709878352 shutdown_connections 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.910+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fced8cb4d0 0x55fced8cd900 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.911+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8bb0a0 0x55fced8b8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.911+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.911+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1709878352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a91a0 0x55fced8b8200 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.911+0000 7fdb46a06740 1 -- 172.21.15.18:0/1709878352 >> 172.21.15.18:0/1709878352 conn(0x55fced8a3ac0 msgr2=0x55fced8a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.911+0000 7fdb22a00640 0 client.26480 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:44.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.911+0000 7fdb46a06740 1 -- 172.21.15.18:0/1709878352 shutdown_connections 2024-04-16T17:07:44.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.911+0000 7fdb46a06740 1 -- 172.21.15.18:0/1709878352 wait complete. 2024-04-16T17:07:44.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.911+0000 7fdb46a06740 20 client.26480 trim_cache size 0 max 16384 2024-04-16T17:07:44.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.931+0000 7fdb46a06740 1 Processor -- start 2024-04-16T17:07:44.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb46a06740 1 -- start start 2024-04-16T17:07:44.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a9b50 0x55fced8c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8bb0a0 0x55fced8b7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb46a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1af960 con 0x55fced8bb0a0 2024-04-16T17:07:44.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb46a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1ad230 con 0x55fced8a9b50 2024-04-16T17:07:44.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb46a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1ab0c0 con 0x55fced8a91a0 2024-04-16T17:07:44.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb40a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a9b50 0x55fced8c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb40a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a9b50 0x55fced8c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34296/0 (socket says 172.21.15.18:34296) 2024-04-16T17:07:44.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb41400640 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb40a00640 1 -- 172.21.15.18:0/1692292671 learned_addr learned my addr 172.21.15.18:0/1692292671 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.932+0000 7fdb41e00640 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8bb0a0 0x55fced8b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.933+0000 7fdb41400640 1 -- 172.21.15.18:0/1692292671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a9b50 msgr2=0x55fced8c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.933+0000 7fdb41400640 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a9b50 0x55fced8c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.933+0000 7fdb41400640 1 -- 172.21.15.18:0/1692292671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8bb0a0 msgr2=0x55fced8b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.933+0000 7fdb41400640 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8bb0a0 0x55fced8b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.933+0000 7fdb41400640 1 -- 172.21.15.18:0/1692292671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fced8bd800 con 0x55fced8a91a0 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.933+0000 7fdb40a00640 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a9b50 0x55fced8c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.933+0000 7fdb41400640 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8ba930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000c760 tx=0x7fdb30099210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b2f069a4c3027 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.933+0000 7fdb2dc00640 1 -- 172.21.15.18:0/1692292671 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb300b6060 con 0x55fced8a91a0 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.934+0000 7fdb2dc00640 1 -- 172.21.15.18:0/1692292671 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb300bd0d0 con 0x55fced8a91a0 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.934+0000 7fdb2dc00640 1 -- 172.21.15.18:0/1692292671 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30098880 con 0x55fced8a91a0 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.934+0000 7fdb46a06740 1 -- 172.21.15.18:0/1692292671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 msgr2=0x55fced8ba930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.934+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8ba930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000c760 tx=0x7fdb30099210 comp rx=0 tx=0).stop 2024-04-16T17:07:44.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.935+0000 7fdb46a06740 1 -- 172.21.15.18:0/1692292671 shutdown_connections 2024-04-16T17:07:44.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.935+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8bb0a0 0x55fced8b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.935+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8a9b50 0x55fced8c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.935+0000 7fdb46a06740 1 --2- 172.21.15.18:0/1692292671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8ba930 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.935+0000 7fdb46a06740 1 -- 172.21.15.18:0/1692292671 >> 172.21.15.18:0/1692292671 conn(0x55fced8a3ac0 msgr2=0x55fced8a4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.935+0000 7fdb46a06740 1 -- 172.21.15.18:0/1692292671 shutdown_connections 2024-04-16T17:07:44.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.935+0000 7fdb46a06740 1 -- 172.21.15.18:0/1692292671 wait complete. 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.935+0000 7fdb46a06740 1 -- start start 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb46a06740 1 Processor -- start 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb46a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb46a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1ab710 con 0x55fced8a9b50 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb46a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1acb60 con 0x55fced8bb0a0 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb46a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1abdc0 con 0x55fced8a91a0 2024-04-16T17:07:44.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb46a06740 10 client.0 _ll_register_callbacks cb 0x7fdb30009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb41400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb41e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.936+0000 7fdb41e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34306/0 (socket says 172.21.15.18:34306) 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb41e00640 1 -- 172.21.15.18:0/2450755058 learned_addr learned my addr 172.21.15.18:0/2450755058 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb40a00640 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb41400640 1 -- 172.21.15.18:0/2450755058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 msgr2=0x55fced8b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb41400640 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb41400640 1 -- 172.21.15.18:0/2450755058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 msgr2=0x55fced8b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb41400640 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb41400640 1 -- 172.21.15.18:0/2450755058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fced8b5510 con 0x55fced8a91a0 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb40a00640 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb41400640 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8c3400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdb300a5270 tx=0x7fdb3009b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59d4f751f253203 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:44.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb22a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.937+0000 7fdb22a00640 1 -- 172.21.15.18:0/2450755058 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb300b6060 con 0x55fced8a91a0 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb22a00640 1 -- 172.21.15.18:0/2450755058 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fdb300ce560 con 0x55fced8a91a0 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 1 -- 172.21.15.18:0/2450755058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb300bd0d0 con 0x55fced8a91a0 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb22a00640 1 -- 172.21.15.18:0/2450755058 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb30004000 con 0x55fced8a91a0 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 10 client.26470 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 1 -- 172.21.15.18:0/2450755058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fced931760 con 0x55fced8a91a0 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb22a00640 1 -- 172.21.15.18:0/2450755058 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb300bd0d0 con 0x55fced8a91a0 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb22a00640 1 -- 172.21.15.18:0/2450755058 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdb300a2070 con 0x55fced8a91a0 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 10 client.26470 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 10 client.26470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 20 client.26470 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fced931b40 0x55fced933f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 4 client.26470 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:07:44.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.938+0000 7fdb46a06740 1 -- 172.21.15.18:0/2450755058 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fced89f180 con 0x55fced931b40 2024-04-16T17:07:44.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.939+0000 7fdb40a00640 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fced931b40 0x55fced933f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.939+0000 7fdb40a00640 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fced931b40 0x55fced933f30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c7aee90e39dea40 server_cookie=b3303fc57c9a9a1 in_seq=0 out_seq=0 2024-04-16T17:07:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.939+0000 7fdb22a00640 10 client.26470 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb22a00640 1 -- 172.21.15.18:0/2450755058 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fced89f180 con 0x55fced931b40 2024-04-16T17:07:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb22a00640 10 client.26470 handle_command_reply: tid=0 2024-04-16T17:07:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 client.26470 shutdown 2024-04-16T17:07:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 -- 172.21.15.18:0/2450755058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 msgr2=0x55fced8c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8c3400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdb300a5270 tx=0x7fdb3009b040 comp rx=0 tx=0).stop 2024-04-16T17:07:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 -- 172.21.15.18:0/2450755058 shutdown_connections 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fced931b40 0x55fced933f30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fced8bb0a0 0x55fced8b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fced8a9b50 0x55fced8b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb22a00640 0 client.26470 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 --2- 172.21.15.18:0/2450755058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fced8a91a0 0x55fced8c3400 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 -- 172.21.15.18:0/2450755058 >> 172.21.15.18:0/2450755058 conn(0x55fced8a3ac0 msgr2=0x55fced8a4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 -- 172.21.15.18:0/2450755058 shutdown_connections 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 1 -- 172.21.15.18:0/2450755058 wait complete. 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.940+0000 7fdb46a06740 20 client.26470 trim_cache size 0 max 16384 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:07:44.945 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:07:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:07:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "a5c4f732-1bb6-4046-a825-eb02980c42bd": { 2024-04-16T17:07:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:07:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "a5c4f732-1bb6-4046-a825-eb02980c42bd", 2024-04-16T17:07:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:07:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:07:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:07:44.946 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 -- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdb240a3da0 msgr2=0x7fdb240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdb240a3da0 0x7fdb240a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55fced8a4ee0 tx=0x7fdb34009500 comp rx=0 tx=0).stop 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 -- 172.21.15.18:0/2667131907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3c102070 msgr2=0x7fdb3c1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3c102070 0x7fdb3c1a0690 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000ea60 tx=0x7fdb3000ef20 comp rx=0 tx=0).stop 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 -- 172.21.15.18:0/2667131907 shutdown_connections 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdb240a3da0 0x7fdb240a6260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb3c19a750 0x7fdb3c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb3c107140 0x7fdb3c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 --2- 172.21.15.18:0/2667131907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3c102070 0x7fdb3c1a0690 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 -- 172.21.15.18:0/2667131907 >> 172.21.15.18:0/2667131907 conn(0x7fdb3c0fd950 msgr2=0x7fdb3c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.941+0000 7fdb42800640 1 -- 172.21.15.18:0/2667131907 shutdown_connections 2024-04-16T17:07:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:07:44.942+0000 7fdb42800640 1 -- 172.21.15.18:0/2667131907 wait complete. 2024-04-16T17:07:44.959 INFO:tasks.cephfs.filesystem:scrub status for tag:a5c4f732-1bb6-4046-a825-eb02980c42bd - {'path': '/', 'tag': 'a5c4f732-1bb6-4046-a825-eb02980c42bd', 'options': 'recursive,force'} 2024-04-16T17:07:45.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 90867 4.28 MB/sec warmup 118 sec latency 37.401 ms 2024-04-16T17:07:45.709 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:41 smithi018 ceph-mon[31776]: pgmap v91: 129 pgs: 129 active+clean; 34 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-04-16T17:07:45.709 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:43 smithi018 ceph-mon[31776]: pgmap v92: 129 pgs: 129 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 804 KiB/s wr, 130 op/s 2024-04-16T17:07:45.709 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:43 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3897765893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:45.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:45 smithi022 ceph-mon[36023]: pgmap v93: 129 pgs: 129 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 794 KiB/s wr, 126 op/s 2024-04-16T17:07:45.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:45 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:07:45.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:45 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:45.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:45 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:07:45.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:45 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:45 smithi043 ceph-mon[36901]: pgmap v93: 129 pgs: 129 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 794 KiB/s wr, 126 op/s 2024-04-16T17:07:46.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:45 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:07:46.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:45 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:46.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:45 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:07:46.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:45 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:46.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:45 smithi018 ceph-mon[31776]: pgmap v93: 129 pgs: 129 active+clean; 34 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 794 KiB/s wr, 126 op/s 2024-04-16T17:07:46.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:45 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:07:46.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:45 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:46.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:45 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:07:46.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:45 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:46.534 INFO:tasks.workunit.client.0.smithi018.stdout: 1 92203 4.34 MB/sec warmup 119 sec latency 32.179 ms 2024-04-16T17:07:47.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:47 smithi022 ceph-mon[36023]: pgmap v94: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-16T17:07:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:47 smithi043 ceph-mon[36901]: pgmap v94: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-16T17:07:48.047 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:47 smithi018 ceph-mon[31776]: pgmap v94: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-16T17:07:48.535 INFO:tasks.workunit.client.0.smithi018.stdout: 1 93207 0.83 MB/sec execute 1 sec latency 82.331 ms 2024-04-16T17:07:49.538 INFO:tasks.workunit.client.0.smithi018.stdout: 1 93487 1.05 MB/sec execute 2 sec latency 89.222 ms 2024-04-16T17:07:49.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:49 smithi022 ceph-mon[36023]: pgmap v95: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-16T17:07:49.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:49 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:07:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:49 smithi043 ceph-mon[36901]: pgmap v95: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-16T17:07:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:49 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:07:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.384+0000 7ff3a8a00640 1 -- 172.21.15.18:0/3816446691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4102080 msgr2=0x7ff3a4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.384+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/3816446691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4102080 0x7ff3a4102460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff394009900 tx=0x7ff39402bc30 comp rx=0 tx=0).stop 2024-04-16T17:07:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.384+0000 7ff3a8a00640 1 -- 172.21.15.18:0/3816446691 shutdown_connections 2024-04-16T17:07:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.384+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/3816446691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3a4107150 0x7ff3a410ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.384+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/3816446691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a4102a30 0x7ff3a4106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.384+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/3816446691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4102080 0x7ff3a4102460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.384+0000 7ff3a8a00640 1 -- 172.21.15.18:0/3816446691 >> 172.21.15.18:0/3816446691 conn(0x7ff3a40fd940 msgr2=0x7ff3a40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.384+0000 7ff3a8a00640 1 -- 172.21.15.18:0/3816446691 shutdown_connections 2024-04-16T17:07:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.386+0000 7ff3a8a00640 1 -- 172.21.15.18:0/3816446691 wait complete. 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.387+0000 7ff3a8a00640 1 Processor -- start 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.387+0000 7ff3a8a00640 1 -- start start 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a4102080 0x7ff3a4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4107150 0x7ff3a4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3a419cfc0 0x7ff3a419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a8a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a4112200 con 0x7ff3a419cfc0 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a8a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a4112080 con 0x7ff3a4107150 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a4112380 con 0x7ff3a4102080 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4107150 0x7ff3a4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4107150 0x7ff3a4198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45046/0 (socket says 172.21.15.18:45046) 2024-04-16T17:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a1e00640 1 -- 172.21.15.18:0/1225788250 learned_addr learned my addr 172.21.15.18:0/1225788250 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a2800640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a4102080 0x7ff3a4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.388+0000 7ff3a3200640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3a419cfc0 0x7ff3a419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff3a1e00640 1 -- 172.21.15.18:0/1225788250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a4102080 msgr2=0x7ff3a4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a4102080 0x7ff3a4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff3a1e00640 1 -- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3a419cfc0 msgr2=0x7ff3a419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3a419cfc0 0x7ff3a419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff3a1e00640 1 -- 172.21.15.18:0/1225788250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a419dab0 con 0x7ff3a4107150 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff3a2800640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a4102080 0x7ff3a4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4107150 0x7ff3a4198c40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e9f0 tx=0x7ff39000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff39b400640 1 -- 172.21.15.18:0/1225788250 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39000ccc0 con 0x7ff3a4107150 2024-04-16T17:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff39b400640 1 -- 172.21.15.18:0/1225788250 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff390005500 con 0x7ff3a4107150 2024-04-16T17:07:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff3a8a00640 1 -- 172.21.15.18:0/1225788250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a419dda0 con 0x7ff3a4107150 2024-04-16T17:07:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.389+0000 7ff39b400640 1 -- 172.21.15.18:0/1225788250 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390010690 con 0x7ff3a4107150 2024-04-16T17:07:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.390+0000 7ff3a8a00640 1 -- 172.21.15.18:0/1225788250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a41a5590 con 0x7ff3a4107150 2024-04-16T17:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.393+0000 7ff3abff2740 1 Processor -- start 2024-04-16T17:07:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.393+0000 7ff3abff2740 1 -- start start 2024-04-16T17:07:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.393+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f52103b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.393+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f52103bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.393+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f52104b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.393+0000 7ff3abff2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f520fa0780 con 0x55f521049410 2024-04-16T17:07:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.393+0000 7ff3abff2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f520f46f40 con 0x55f52103c7f0 2024-04-16T17:07:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.393+0000 7ff3abff2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f520f9c740 con 0x55f52103ba80 2024-04-16T17:07:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f52103b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f52103b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45050/0 (socket says 172.21.15.18:45050) 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 -- 172.21.15.18:0/4171569103 learned_addr learned my addr 172.21.15.18:0/4171569103 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a2800640 1 --2- 172.21.15.18:0/4171569103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f52103bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 -- 172.21.15.18:0/4171569103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 msgr2=0x55f52103bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 --2- 172.21.15.18:0/4171569103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f52103bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 -- 172.21.15.18:0/4171569103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 msgr2=0x55f52104b800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 --2- 172.21.15.18:0/4171569103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f52104b800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff39b400640 1 -- 172.21.15.18:0/1225788250 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff390004f20 con 0x7ff3a4107150 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 -- 172.21.15.18:0/4171569103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f520cb3210 con 0x55f52103c7f0 2024-04-16T17:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.394+0000 7ff3a3200640 1 --2- 172.21.15.18:0/4171569103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f52103b540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009f50 tx=0x7ff39c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a74652e856c367d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.395+0000 7ff38f400640 1 -- 172.21.15.18:0/4171569103 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c004d20 con 0x55f52103c7f0 2024-04-16T17:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.395+0000 7ff38f400640 1 -- 172.21.15.18:0/4171569103 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c004050 con 0x55f52103c7f0 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.395+0000 7ff38f400640 1 -- 172.21.15.18:0/4171569103 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c005280 con 0x55f52103c7f0 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.395+0000 7ff3abff2740 1 -- 172.21.15.18:0/4171569103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 msgr2=0x55f52103b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.395+0000 7ff3abff2740 1 --2- 172.21.15.18:0/4171569103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f52103b540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009f50 tx=0x7ff39c0118d0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff3abff2740 1 -- 172.21.15.18:0/4171569103 shutdown_connections 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff3abff2740 1 --2- 172.21.15.18:0/4171569103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f52104b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff3abff2740 1 --2- 172.21.15.18:0/4171569103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f52103bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff3abff2740 1 --2- 172.21.15.18:0/4171569103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f52103b540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff3abff2740 1 -- 172.21.15.18:0/4171569103 >> 172.21.15.18:0/4171569103 conn(0x55f520fd65d0 msgr2=0x55f521036770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff3abff2740 1 -- 172.21.15.18:0/4171569103 shutdown_connections 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff3abff2740 1 -- 172.21.15.18:0/4171569103 wait complete. 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff3abff2740 1 -- start start 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.396+0000 7ff39b400640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3880a3c90 0x7ff3880a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.397+0000 7ff39b400640 1 -- 172.21.15.18:0/1225788250 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff3900d0d90 con 0x7ff3a4107150 2024-04-16T17:07:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.397+0000 7ff3abff2740 1 Processor -- start 2024-04-16T17:07:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.397+0000 7ff3a3200640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3880a3c90 0x7ff3880a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.397+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103ba80 0x55f521046280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103c7f0 0x55f5210467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521046d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3abff2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f52104c140 con 0x55f521049410 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3abff2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f520f5a390 con 0x55f52103ba80 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3abff2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f52104c2c0 con 0x55f52103c7f0 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3abff2740 10 client.0 _ll_register_callbacks cb 0x55f520f43f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3a2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103ba80 0x55f521046280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3a2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103ba80 0x55f521046280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45060/0 (socket says 172.21.15.18:45060) 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3a2800640 1 -- 172.21.15.18:0/3907351250 learned_addr learned my addr 172.21.15.18:0/3907351250 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3a3200640 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521046d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103c7f0 0x55f5210467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3a2800640 1 -- 172.21.15.18:0/3907351250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103c7f0 msgr2=0x55f5210467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.398+0000 7ff3a2800640 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103c7f0 0x55f5210467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff3a2800640 1 -- 172.21.15.18:0/3907351250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 msgr2=0x55f521046d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff3a2800640 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521046d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff3a2800640 1 -- 172.21.15.18:0/3907351250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f520d7f620 con 0x55f52103ba80 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff3a3200640 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521046d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff3a3200640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3880a3c90 0x7ff3880a6150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009d60 tx=0x7ff39c004700 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff3a2800640 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103ba80 0x55f521046280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff39402c170 tx=0x7ff39402c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e924f799500f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff383e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff383e00640 1 -- 172.21.15.18:0/3907351250 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39402cb70 con 0x55f52103ba80 2024-04-16T17:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff383e00640 1 -- 172.21.15.18:0/3907351250 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff394004040 con 0x55f52103ba80 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.399+0000 7ff3abff2740 1 -- 172.21.15.18:0/3907351250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff39c004050 con 0x55f52103ba80 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff383e00640 1 -- 172.21.15.18:0/3907351250 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff394004240 con 0x55f52103ba80 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff3abff2740 10 client.26498 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff3abff2740 1 -- 172.21.15.18:0/3907351250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5210582a0 con 0x55f52103ba80 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff383e00640 1 -- 172.21.15.18:0/3907351250 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c004050 con 0x55f52103ba80 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff383e00640 1 -- 172.21.15.18:0/3907351250 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff394003770 con 0x55f52103ba80 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff3abff2740 10 client.26498 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff3abff2740 10 client.26498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff3abff2740 20 client.26498 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff3abff2740 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f5210597d0 0x55f52105bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff3abff2740 4 client.26498 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.400+0000 7ff3abff2740 1 -- 172.21.15.18:0/3907351250 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f520fae1a0 con 0x55f5210597d0 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.401+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f5210597d0 0x55f52105bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.401+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f5210597d0 0x55f52105bbc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8235452465c9b982 server_cookie=73db281aa2e2703 in_seq=0 out_seq=0 2024-04-16T17:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.401+0000 7ff383e00640 10 client.26498 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff383e00640 1 -- 172.21.15.18:0/3907351250 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f520fae1a0 con 0x55f5210597d0 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff383e00640 10 client.26498 handle_command_reply: tid=0 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 client.26498 shutdown 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 -- 172.21.15.18:0/3907351250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103ba80 msgr2=0x55f521046280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103ba80 0x55f521046280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff39402c170 tx=0x7ff39402c880 comp rx=0 tx=0).stop 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 -- 172.21.15.18:0/3907351250 shutdown_connections 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f5210597d0 0x55f52105bbc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521046d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103c7f0 0x55f5210467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 --2- 172.21.15.18:0/3907351250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103ba80 0x55f521046280 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.403+0000 7ff3abff2740 1 -- 172.21.15.18:0/3907351250 >> 172.21.15.18:0/3907351250 conn(0x55f520fd65d0 msgr2=0x55f5210384c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.404+0000 7ff383e00640 0 client.26498 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.404+0000 7ff3abff2740 1 -- 172.21.15.18:0/3907351250 shutdown_connections 2024-04-16T17:07:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.404+0000 7ff3abff2740 1 -- 172.21.15.18:0/3907351250 wait complete. 2024-04-16T17:07:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.404+0000 7ff3abff2740 20 client.26498 trim_cache size 0 max 16384 2024-04-16T17:07:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.435+0000 7ff3abff2740 1 Processor -- start 2024-04-16T17:07:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3abff2740 1 -- start start 2024-04-16T17:07:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f52103ba80 0x55f5210489b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521055070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f521049410 0x55f5210452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3abff2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff384012820 con 0x55f52103ba80 2024-04-16T17:07:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3abff2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff3840100f0 con 0x55f52103c7f0 2024-04-16T17:07:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3abff2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff38400df40 con 0x55f521049410 2024-04-16T17:07:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521055070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3a2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f52103ba80 0x55f5210489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521055070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45064/0 (socket says 172.21.15.18:45064) 2024-04-16T17:07:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.436+0000 7ff3a1e00640 1 -- 172.21.15.18:0/1153650497 learned_addr learned my addr 172.21.15.18:0/1153650497 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.437+0000 7ff3a3200640 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f521049410 0x55f5210452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.437+0000 7ff3a2800640 1 -- 172.21.15.18:0/1153650497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f521049410 msgr2=0x55f5210452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.437+0000 7ff3a2800640 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f521049410 0x55f5210452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.437+0000 7ff3a2800640 1 -- 172.21.15.18:0/1153650497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 msgr2=0x55f521055070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.437+0000 7ff3a2800640 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521055070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.437+0000 7ff3a2800640 1 -- 172.21.15.18:0/1153650497 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f521041ec0 con 0x55f52103ba80 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.437+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521055070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.437+0000 7ff3a2800640 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f52103ba80 0x55f5210489b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff39402c610 tx=0x7ff394009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d53d22afecfa80 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.438+0000 7ff38ea00640 1 -- 172.21.15.18:0/1153650497 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394000e60 con 0x55f52103ba80 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.438+0000 7ff38ea00640 1 -- 172.21.15.18:0/1153650497 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff394009780 con 0x55f52103ba80 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.438+0000 7ff38ea00640 1 -- 172.21.15.18:0/1153650497 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3940346e0 con 0x55f52103ba80 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 -- 172.21.15.18:0/1153650497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f52103ba80 msgr2=0x55f5210489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f52103ba80 0x55f5210489b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff39402c610 tx=0x7ff394009660 comp rx=0 tx=0).stop 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 -- 172.21.15.18:0/1153650497 shutdown_connections 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f521049410 0x55f5210452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521055070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 --2- 172.21.15.18:0/1153650497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f52103ba80 0x55f5210489b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 -- 172.21.15.18:0/1153650497 >> 172.21.15.18:0/1153650497 conn(0x55f520fd65d0 msgr2=0x55f521044b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 -- 172.21.15.18:0/1153650497 shutdown_connections 2024-04-16T17:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 -- 172.21.15.18:0/1153650497 wait complete. 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.439+0000 7ff3abff2740 1 -- start start 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.440+0000 7ff3abff2740 1 Processor -- start 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.440+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f521051770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.440+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521042900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.440+0000 7ff3abff2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521042e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.440+0000 7ff3abff2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff38400e5b0 con 0x55f521049410 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.440+0000 7ff3abff2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff38400fa20 con 0x55f52103c7f0 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.440+0000 7ff3abff2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff38400ec80 con 0x55f52103ba80 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.440+0000 7ff3abff2740 10 client.0 _ll_register_callbacks cb 0x55f521049360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f521051770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521042900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f521051770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35892/0 (socket says 172.21.15.18:35892) 2024-04-16T17:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a2800640 1 -- 172.21.15.18:0/2522190922 learned_addr learned my addr 172.21.15.18:0/2522190922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a3200640 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521042e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a2800640 1 -- 172.21.15.18:0/2522190922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 msgr2=0x55f521042900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a2800640 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521042900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a2800640 1 -- 172.21.15.18:0/2522190922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 msgr2=0x55f521042e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a2800640 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521042e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a2800640 1 -- 172.21.15.18:0/2522190922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f521043380 con 0x55f52103ba80 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.441+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521042900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.442+0000 7ff3a2800640 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f521051770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff394003c10 tx=0x7ff3940344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651d81a153de8b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.442+0000 7ff383e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.442+0000 7ff383e00640 1 -- 172.21.15.18:0/2522190922 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394004010 con 0x55f52103ba80 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.442+0000 7ff3abff2740 1 -- 172.21.15.18:0/2522190922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff394009780 con 0x55f52103ba80 2024-04-16T17:07:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.442+0000 7ff383e00640 1 -- 172.21.15.18:0/2522190922 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff39402caa0 con 0x55f52103ba80 2024-04-16T17:07:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.442+0000 7ff383e00640 1 -- 172.21.15.18:0/2522190922 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff394066050 con 0x55f52103ba80 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.442+0000 7ff3abff2740 10 client.26488 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.442+0000 7ff3abff2740 1 -- 172.21.15.18:0/2522190922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5210bfa00 con 0x55f52103ba80 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff383e00640 1 -- 172.21.15.18:0/2522190922 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff394009780 con 0x55f52103ba80 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff383e00640 1 -- 172.21.15.18:0/2522190922 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff394034690 con 0x55f52103ba80 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff3abff2740 10 client.26488 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff3abff2740 10 client.26488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff3abff2740 20 client.26488 populate_metadata read hostname 'smithi018' 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff3abff2740 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f5210bfe80 0x55f5210c2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff3abff2740 4 client.26488 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff3abff2740 1 -- 172.21.15.18:0/2522190922 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f520fa8010 con 0x55f5210bfe80 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.443+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f5210bfe80 0x55f5210c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.444+0000 7ff3a1e00640 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f5210bfe80 0x55f5210c2270 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8ae36abe88ee97 server_cookie=b7a2520a24e9e017 in_seq=0 out_seq=0 2024-04-16T17:07:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.444+0000 7ff383e00640 10 client.26488 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:50.540 INFO:tasks.workunit.client.0.smithi018.stdout: 1 94431 1.92 MB/sec execute 3 sec latency 62.594 ms 2024-04-16T17:07:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff383e00640 1 -- 172.21.15.18:0/2522190922 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f520fa8010 con 0x55f5210bfe80 2024-04-16T17:07:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff383e00640 10 client.26488 handle_command_reply: tid=0 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 client.26488 shutdown 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 -- 172.21.15.18:0/2522190922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 msgr2=0x55f521051770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f521051770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff394003c10 tx=0x7ff3940344c0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 -- 172.21.15.18:0/2522190922 shutdown_connections 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f5210bfe80 0x55f5210c2270 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f521049410 0x55f521042e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f52103c7f0 0x55f521042900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 --2- 172.21.15.18:0/2522190922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f52103ba80 0x55f521051770 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 -- 172.21.15.18:0/2522190922 >> 172.21.15.18:0/2522190922 conn(0x55f520fd65d0 msgr2=0x55f521045420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff383e00640 0 client.26488 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.551+0000 7ff3abff2740 1 -- 172.21.15.18:0/2522190922 shutdown_connections 2024-04-16T17:07:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.552+0000 7ff3abff2740 1 -- 172.21.15.18:0/2522190922 wait complete. 2024-04-16T17:07:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.553+0000 7ff3abff2740 20 client.26488 trim_cache size 0 max 16384 2024-04-16T17:07:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:07:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:07:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:07:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:07:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 -- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3880a3c90 msgr2=0x7ff3880a6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3880a3c90 0x7ff3880a6150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009d60 tx=0x7ff39c004700 comp rx=0 tx=0).stop 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 -- 172.21.15.18:0/1225788250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4107150 msgr2=0x7ff3a4198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4107150 0x7ff3a4198c40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e9f0 tx=0x7ff39000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 -- 172.21.15.18:0/1225788250 shutdown_connections 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3880a3c90 0x7ff3880a6150 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3a419cfc0 0x7ff3a419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3a4107150 0x7ff3a4198c40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 --2- 172.21.15.18:0/1225788250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a4102080 0x7ff3a4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 -- 172.21.15.18:0/1225788250 >> 172.21.15.18:0/1225788250 conn(0x7ff3a40fd940 msgr2=0x7ff3a40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 -- 172.21.15.18:0/1225788250 shutdown_connections 2024-04-16T17:07:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:07:50.554+0000 7ff3a8a00640 1 -- 172.21.15.18:0/1225788250 wait complete. 2024-04-16T17:07:50.801 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:49 smithi018 ceph-mon[31776]: pgmap v95: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-16T17:07:50.801 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:49 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:07:51.538 INFO:tasks.workunit.client.0.smithi018.stdout: 1 95797 4.35 MB/sec execute 4 sec latency 33.148 ms 2024-04-16T17:07:51.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:51 smithi022 ceph-mon[36023]: pgmap v96: 129 pgs: 129 active+clean; 37 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-04-16T17:07:52.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:51 smithi043 ceph-mon[36901]: pgmap v96: 129 pgs: 129 active+clean; 37 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-04-16T17:07:52.538 INFO:tasks.workunit.client.0.smithi018.stdout: 1 96384 3.83 MB/sec execute 5 sec latency 12.095 ms 2024-04-16T17:07:53.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:51 smithi018 ceph-mon[31776]: pgmap v96: 129 pgs: 129 active+clean; 37 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-04-16T17:07:53.538 INFO:tasks.workunit.client.0.smithi018.stdout: 1 96907 3.52 MB/sec execute 6 sec latency 54.607 ms 2024-04-16T17:07:53.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:53 smithi022 ceph-mon[36023]: pgmap v97: 129 pgs: 129 active+clean; 37 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-04-16T17:07:54.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:53 smithi043 ceph-mon[36901]: pgmap v97: 129 pgs: 129 active+clean; 37 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-04-16T17:07:54.540 INFO:tasks.workunit.client.0.smithi018.stdout: 1 97295 3.46 MB/sec execute 7 sec latency 68.628 ms 2024-04-16T17:07:55.066 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:53 smithi018 ceph-mon[31776]: pgmap v97: 129 pgs: 129 active+clean; 37 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-04-16T17:07:55.540 INFO:tasks.workunit.client.0.smithi018.stdout: 1 98246 3.28 MB/sec execute 8 sec latency 61.583 ms 2024-04-16T17:07:55.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:55 smithi022 ceph-mon[36023]: pgmap v98: 129 pgs: 129 active+clean; 37 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-04-16T17:07:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:55 smithi043 ceph-mon[36901]: pgmap v98: 129 pgs: 129 active+clean; 37 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-04-16T17:07:56.540 INFO:tasks.workunit.client.0.smithi018.stdout: 1 99478 4.10 MB/sec execute 9 sec latency 59.609 ms 2024-04-16T17:07:56.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:56 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:07:57.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:56 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:07:57.540 INFO:tasks.workunit.client.0.smithi018.stdout: 1 100152 3.87 MB/sec execute 10 sec latency 25.925 ms 2024-04-16T17:07:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:57 smithi022 ceph-mon[36023]: pgmap v99: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:07:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:57 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:07:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:57 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:07:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:57 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:07:58.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:57 smithi043 ceph-mon[36901]: pgmap v99: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:07:58.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:57 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:07:58.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:57 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:07:58.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:57 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:07:58.540 INFO:tasks.workunit.client.0.smithi018.stdout: 1 100525 3.66 MB/sec execute 11 sec latency 106.744 ms 2024-04-16T17:07:59.540 INFO:tasks.workunit.client.0.smithi018.stdout: 1 100880 3.61 MB/sec execute 12 sec latency 67.297 ms 2024-04-16T17:07:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:07:59 smithi022 ceph-mon[36023]: pgmap v100: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 982 KiB/s wr, 126 op/s 2024-04-16T17:08:00.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:07:59 smithi043 ceph-mon[36901]: pgmap v100: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 982 KiB/s wr, 126 op/s 2024-04-16T17:08:00.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 101968 3.56 MB/sec execute 13 sec latency 45.180 ms 2024-04-16T17:08:01.244 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:55 smithi018 ceph-mon[31776]: pgmap v98: 129 pgs: 129 active+clean; 37 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-04-16T17:08:01.244 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:56 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:08:01.244 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:57 smithi018 ceph-mon[31776]: pgmap v99: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:08:01.244 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:57 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:08:01.245 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:57 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:08:01.245 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:57 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:08:01.245 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:07:59 smithi018 ceph-mon[31776]: pgmap v100: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 982 KiB/s wr, 126 op/s 2024-04-16T17:08:01.540 INFO:tasks.workunit.client.0.smithi018.stdout: 1 102618 3.75 MB/sec execute 14 sec latency 93.012 ms 2024-04-16T17:08:01.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:01 smithi018 ceph-mon[31776]: pgmap v101: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-16T17:08:01.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:01 smithi022 ceph-mon[36023]: pgmap v101: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-16T17:08:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:01 smithi043 ceph-mon[36901]: pgmap v101: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-16T17:08:02.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 103329 3.82 MB/sec execute 15 sec latency 42.337 ms 2024-04-16T17:08:03.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 103736 3.62 MB/sec execute 16 sec latency 65.243 ms 2024-04-16T17:08:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 -- 172.21.15.18:0/457410344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4104e20 msgr2=0x7f20d4108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 --2- 172.21.15.18:0/457410344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4104e20 0x7f20d4108e60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009900 tx=0x7f20c002bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 -- 172.21.15.18:0/457410344 shutdown_connections 2024-04-16T17:08:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 --2- 172.21.15.18:0/457410344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d4109540 0x7f20d4111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 --2- 172.21.15.18:0/457410344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4104e20 0x7f20d4108e60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 --2- 172.21.15.18:0/457410344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104470 0x7f20d4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 -- 172.21.15.18:0/457410344 >> 172.21.15.18:0/457410344 conn(0x7f20d40ffcd0 msgr2=0x7f20d41020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 -- 172.21.15.18:0/457410344 shutdown_connections 2024-04-16T17:08:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.693+0000 7f20d9000640 1 -- 172.21.15.18:0/457410344 wait complete. 2024-04-16T17:08:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.694+0000 7f20d9000640 1 Processor -- start 2024-04-16T17:08:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.695+0000 7f20d9000640 1 -- start start 2024-04-16T17:08:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.695+0000 7f20d9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104470 0x7f20d419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.695+0000 7f20d9000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4109540 0x7f20d419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.695+0000 7f20d9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419e4d0 0x7f20d419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.695+0000 7f20d9000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4114680 con 0x7f20d4109540 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.695+0000 7f20d9000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4114500 con 0x7f20d419e4d0 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.695+0000 7f20d9000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4114800 con 0x7f20d4104470 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4109540 0x7f20d419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4109540 0x7f20d419b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53660/0 (socket says 172.21.15.18:53660) 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1400640 1 -- 172.21.15.18:0/3431368918 learned_addr learned my addr 172.21.15.18:0/3431368918 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1e00640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104470 0x7f20d419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d2800640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419e4d0 0x7f20d419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1400640 1 -- 172.21.15.18:0/3431368918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104470 msgr2=0x7f20d419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1400640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104470 0x7f20d419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1400640 1 -- 172.21.15.18:0/3431368918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419e4d0 msgr2=0x7f20d419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1400640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419e4d0 0x7f20d419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d1400640 1 -- 172.21.15.18:0/3431368918 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d419efc0 con 0x7f20d4109540 2024-04-16T17:08:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.696+0000 7f20d2800640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419e4d0 0x7f20d419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.697+0000 7f20d1400640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4109540 0x7f20d419b200 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f20c002c240 tx=0x7f20c00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.697+0000 7f20caa00640 1 -- 172.21.15.18:0/3431368918 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0043020 con 0x7f20d4109540 2024-04-16T17:08:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.697+0000 7f20caa00640 1 -- 172.21.15.18:0/3431368918 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c00044d0 con 0x7f20d4109540 2024-04-16T17:08:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.697+0000 7f20d9000640 1 -- 172.21.15.18:0/3431368918 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d419f250 con 0x7f20d4109540 2024-04-16T17:08:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.697+0000 7f20caa00640 1 -- 172.21.15.18:0/3431368918 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0003b20 con 0x7f20d4109540 2024-04-16T17:08:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.697+0000 7f20d9000640 1 -- 172.21.15.18:0/3431368918 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41a7b50 con 0x7f20d4109540 2024-04-16T17:08:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.700+0000 7f20dbd6e740 1 Processor -- start 2024-04-16T17:08:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.700+0000 7f20dbd6e740 1 -- start start 2024-04-16T17:08:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.700+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 0x55d161007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.700+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161007d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.700+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d161010e20 0x55d161013210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.700+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d160f6c780 con 0x55d161010e20 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.700+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d160f12f40 con 0x55d1610078a0 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.700+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d160f68740 con 0x55d161008610 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d161010e20 0x55d161013210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161007d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d161010e20 0x55d161013210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53670/0 (socket says 172.21.15.18:53670) 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1400640 1 -- 172.21.15.18:0/3198351737 learned_addr learned my addr 172.21.15.18:0/3198351737 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1e00640 1 -- 172.21.15.18:0/3198351737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 msgr2=0x55d161007360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1e00640 1 --2- 172.21.15.18:0/3198351737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 0x55d161007360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1e00640 1 -- 172.21.15.18:0/3198351737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d161010e20 msgr2=0x55d161013210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1e00640 1 --2- 172.21.15.18:0/3198351737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d161010e20 0x55d161013210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.701+0000 7f20d1e00640 1 -- 172.21.15.18:0/3198351737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d160c7f210 con 0x55d1610078a0 2024-04-16T17:08:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.702+0000 7f20d1e00640 1 --2- 172.21.15.18:0/3198351737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161007d20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20c40099a0 tx=0x7f20c4009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbdbb431fabb1e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.702+0000 7f20bf400640 1 -- 172.21.15.18:0/3198351737 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4017070 con 0x55d1610078a0 2024-04-16T17:08:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.702+0000 7f20bf400640 1 -- 172.21.15.18:0/3198351737 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c4013470 con 0x55d1610078a0 2024-04-16T17:08:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.702+0000 7f20bf400640 1 -- 172.21.15.18:0/3198351737 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40125f0 con 0x55d1610078a0 2024-04-16T17:08:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.703+0000 7f20dbd6e740 1 -- 172.21.15.18:0/3198351737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 msgr2=0x55d161007d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.703+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/3198351737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161007d20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20c40099a0 tx=0x7f20c4009e60 comp rx=0 tx=0).stop 2024-04-16T17:08:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.703+0000 7f20dbd6e740 1 -- 172.21.15.18:0/3198351737 shutdown_connections 2024-04-16T17:08:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.703+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/3198351737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d161010e20 0x55d161013210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.703+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/3198351737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161007d20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.703+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/3198351737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 0x55d161007360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.703+0000 7f20dbd6e740 1 -- 172.21.15.18:0/3198351737 >> 172.21.15.18:0/3198351737 conn(0x55d160fa25d0 msgr2=0x55d161002590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.704+0000 7f20caa00640 1 -- 172.21.15.18:0/3431368918 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f20c0046020 con 0x7f20d4109540 2024-04-16T17:08:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.704+0000 7f20dbd6e740 1 -- 172.21.15.18:0/3198351737 shutdown_connections 2024-04-16T17:08:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.704+0000 7f20dbd6e740 1 -- 172.21.15.18:0/3198351737 wait complete. 2024-04-16T17:08:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.704+0000 7f20dbd6e740 1 -- start start 2024-04-16T17:08:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20dbd6e740 1 Processor -- start 2024-04-16T17:08:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1610078a0 0x55d16100dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d16100e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 0x55d16101d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d161013b50 con 0x55d16100e690 2024-04-16T17:08:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d160f26390 con 0x55d161008610 2024-04-16T17:08:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d161013cd0 con 0x55d1610078a0 2024-04-16T17:08:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20caa00640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20b80a3cf0 0x7f20b80a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20dbd6e740 10 client.0 _ll_register_callbacks cb 0x55d160f0ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.705+0000 7f20caa00640 1 -- 172.21.15.18:0/3431368918 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f20c00f9a40 con 0x7f20d4109540 2024-04-16T17:08:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d16100e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d16100e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36060/0 (socket says 172.21.15.18:36060) 2024-04-16T17:08:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1e00640 1 -- 172.21.15.18:0/847932519 learned_addr learned my addr 172.21.15.18:0/847932519 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1400640 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 0x55d16101d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d2800640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20b80a3cf0 0x7f20b80a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1e00640 1 -- 172.21.15.18:0/847932519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1610078a0 msgr2=0x55d16100dc10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1e00640 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1610078a0 0x55d16100dc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1e00640 1 -- 172.21.15.18:0/847932519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 msgr2=0x55d16101d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1e00640 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 0x55d16101d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1e00640 1 -- 172.21.15.18:0/847932519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d160d4b620 con 0x55d161008610 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.706+0000 7f20d1400640 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 0x55d16101d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.707+0000 7f20d1e00640 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d16100e150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f20c4013e70 tx=0x7f20c4006ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a20af8a474fb21 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.707+0000 7f20b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.707+0000 7f20b3e00640 1 -- 172.21.15.18:0/847932519 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4017050 con 0x55d161008610 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.707+0000 7f20b3e00640 1 -- 172.21.15.18:0/847932519 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f20c401c450 con 0x55d161008610 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.707+0000 7f20dbd6e740 1 -- 172.21.15.18:0/847932519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c4013470 con 0x55d161008610 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.707+0000 7f20b3e00640 1 -- 172.21.15.18:0/847932519 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20c402f050 con 0x55d161008610 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.707+0000 7f20dbd6e740 10 client.26516 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.707+0000 7f20dbd6e740 1 -- 172.21.15.18:0/847932519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d161020060 con 0x55d161008610 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20d2800640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20b80a3cf0 0x7f20b80a61b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f20cc005fd0 tx=0x7f20cc009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20b3e00640 1 -- 172.21.15.18:0/847932519 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c4013470 con 0x55d161008610 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20b3e00640 1 -- 172.21.15.18:0/847932519 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f20c4026810 con 0x55d161008610 2024-04-16T17:08:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20dbd6e740 10 client.26516 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20dbd6e740 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20dbd6e740 20 client.26516 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d161021460 0x55d161023850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20dbd6e740 4 client.26516 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.708+0000 7f20dbd6e740 1 -- 172.21.15.18:0/847932519 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d160f7a1a0 con 0x55d161021460 2024-04-16T17:08:03.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.709+0000 7f20d1400640 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d161021460 0x55d161023850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.709+0000 7f20d1400640 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d161021460 0x55d161023850 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec72af5708e3f8d2 server_cookie=b9b217d7cfb1381b in_seq=0 out_seq=0 2024-04-16T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.709+0000 7f20b3e00640 10 client.26516 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20b3e00640 1 -- 172.21.15.18:0/847932519 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d160f7a1a0 con 0x55d161021460 2024-04-16T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20b3e00640 10 client.26516 handle_command_reply: tid=0 2024-04-16T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 client.26516 shutdown 2024-04-16T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 -- 172.21.15.18:0/847932519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 msgr2=0x55d16100e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d16100e150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f20c4013e70 tx=0x7f20c4006ec0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 -- 172.21.15.18:0/847932519 shutdown_connections 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d161021460 0x55d161023850 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 0x55d16101d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d16100e150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/847932519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1610078a0 0x55d16100dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 -- 172.21.15.18:0/847932519 >> 172.21.15.18:0/847932519 conn(0x55d160fa25d0 msgr2=0x55d1610042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20b3e00640 0 client.26516 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 -- 172.21.15.18:0/847932519 shutdown_connections 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.710+0000 7f20dbd6e740 1 -- 172.21.15.18:0/847932519 wait complete. 2024-04-16T17:08:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.711+0000 7f20dbd6e740 20 client.26516 trim_cache size 0 max 16384 2024-04-16T17:08:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.733+0000 7f20dbd6e740 1 Processor -- start 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.733+0000 7f20dbd6e740 1 -- start start 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.733+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1610078a0 0x55d16101cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.733+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d161012210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.733+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d16100e690 0x55d161012750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41b4120 con 0x55d1610078a0 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41b19f0 con 0x55d161008610 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41af840 con 0x55d16100e690 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d161012210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1610078a0 0x55d16101cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d161012210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36074/0 (socket says 172.21.15.18:36074) 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1400640 1 -- 172.21.15.18:0/431279938 learned_addr learned my addr 172.21.15.18:0/431279938 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d2800640 1 --2- 172.21.15.18:0/431279938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d16100e690 0x55d161012750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1400640 1 -- 172.21.15.18:0/431279938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d16100e690 msgr2=0x55d161012750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1400640 1 --2- 172.21.15.18:0/431279938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d16100e690 0x55d161012750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1400640 1 -- 172.21.15.18:0/431279938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1610078a0 msgr2=0x55d16101cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1400640 1 --2- 172.21.15.18:0/431279938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1610078a0 0x55d16101cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.734+0000 7f20d1400640 1 -- 172.21.15.18:0/431279938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d161006b70 con 0x55d161008610 2024-04-16T17:08:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.735+0000 7f20d1400640 1 --2- 172.21.15.18:0/431279938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d161012210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f20c0044000 tx=0x7f20c004f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0de851ebe74beb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.735+0000 7f20bea00640 1 -- 172.21.15.18:0/431279938 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00c50a0 con 0x55d161008610 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.735+0000 7f20bea00640 1 -- 172.21.15.18:0/431279938 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c00c5e40 con 0x55d161008610 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.735+0000 7f20bea00640 1 -- 172.21.15.18:0/431279938 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00b00b0 con 0x55d161008610 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.735+0000 7f20dbd6e740 1 -- 172.21.15.18:0/431279938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 msgr2=0x55d161012210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.735+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/431279938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d161012210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f20c0044000 tx=0x7f20c004f000 comp rx=0 tx=0).stop 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 -- 172.21.15.18:0/431279938 shutdown_connections 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/431279938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d16100e690 0x55d161012750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/431279938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d161008610 0x55d161012210 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/431279938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1610078a0 0x55d16101cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 -- 172.21.15.18:0/431279938 >> 172.21.15.18:0/431279938 conn(0x55d160fa25d0 msgr2=0x55d1610020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 -- 172.21.15.18:0/431279938 shutdown_connections 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 -- 172.21.15.18:0/431279938 wait complete. 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 -- start start 2024-04-16T17:08:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.736+0000 7f20dbd6e740 1 Processor -- start 2024-04-16T17:08:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161021460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 0x55d1610219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20dbd6e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 0x55d161021f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41afeb0 con 0x55d16100e690 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41b1320 con 0x55d1610078a0 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20dbd6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41b0580 con 0x55d161008610 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20dbd6e740 10 client.0 _ll_register_callbacks cb 0x7f20c0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161021460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161021460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36086/0 (socket says 172.21.15.18:36086) 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1e00640 1 -- 172.21.15.18:0/1085752157 learned_addr learned my addr 172.21.15.18:0/1085752157 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1400640 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 0x55d1610219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1e00640 1 -- 172.21.15.18:0/1085752157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 msgr2=0x55d1610219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1e00640 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 0x55d1610219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1e00640 1 -- 172.21.15.18:0/1085752157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 msgr2=0x55d161021f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1e00640 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 0x55d161021f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.737+0000 7f20d1e00640 1 -- 172.21.15.18:0/1085752157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d161085910 con 0x55d1610078a0 2024-04-16T17:08:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.738+0000 7f20d1e00640 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161021460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55d16100d470 tx=0x7f20c40093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2019f3370ef61720 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.738+0000 7f20b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.738+0000 7f20b3e00640 1 -- 172.21.15.18:0/1085752157 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4017070 con 0x55d1610078a0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.738+0000 7f20b3e00640 1 -- 172.21.15.18:0/1085752157 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f20c40265f0 con 0x55d1610078a0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.738+0000 7f20dbd6e740 1 -- 172.21.15.18:0/1085752157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c00c5e40 con 0x55d1610078a0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.738+0000 7f20b3e00640 1 -- 172.21.15.18:0/1085752157 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20c4013470 con 0x55d1610078a0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.738+0000 7f20dbd6e740 10 client.26522 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.738+0000 7f20dbd6e740 1 -- 172.21.15.18:0/1085752157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d161086bc0 con 0x55d1610078a0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.739+0000 7f20b3e00640 1 -- 172.21.15.18:0/1085752157 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c00c5e40 con 0x55d1610078a0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.739+0000 7f20b3e00640 1 -- 172.21.15.18:0/1085752157 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f20c401cc00 con 0x55d1610078a0 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.739+0000 7f20dbd6e740 10 client.26522 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.739+0000 7f20dbd6e740 10 client.26522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.739+0000 7f20dbd6e740 20 client.26522 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.739+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d161087210 0x55d161089660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.739+0000 7f20dbd6e740 4 client.26522 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.739+0000 7f20dbd6e740 1 -- 172.21.15.18:0/1085752157 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d160f74010 con 0x55d161087210 2024-04-16T17:08:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.740+0000 7f20d1400640 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d161087210 0x55d161089660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.740+0000 7f20b3e00640 10 client.26522 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.740+0000 7f20d1400640 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d161087210 0x55d161089660 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d57afa05a3a52d5 server_cookie=bf891ec608654a8a in_seq=0 out_seq=0 2024-04-16T17:08:03.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:03 smithi018 ceph-mon[31776]: pgmap v102: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 968 KiB/s wr, 126 op/s 2024-04-16T17:08:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.871+0000 7f20b3e00640 1 -- 172.21.15.18:0/1085752157 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d160f74010 con 0x55d161087210 2024-04-16T17:08:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.871+0000 7f20b3e00640 10 client.26522 handle_command_reply: tid=0 2024-04-16T17:08:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.871+0000 7f20dbd6e740 1 client.26522 shutdown 2024-04-16T17:08:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.871+0000 7f20dbd6e740 1 -- 172.21.15.18:0/1085752157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 msgr2=0x55d161021460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.871+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161021460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55d16100d470 tx=0x7f20c40093a0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.871+0000 7f20dbd6e740 1 -- 172.21.15.18:0/1085752157 shutdown_connections 2024-04-16T17:08:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.871+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d161087210 0x55d161089660 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.872+0000 7f20b3e00640 0 client.26522 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.871+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d16100e690 0x55d161021f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.872+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d161008610 0x55d1610219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.872+0000 7f20dbd6e740 1 --2- 172.21.15.18:0/1085752157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1610078a0 0x55d161021460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.872+0000 7f20dbd6e740 1 -- 172.21.15.18:0/1085752157 >> 172.21.15.18:0/1085752157 conn(0x55d160fa25d0 msgr2=0x55d161000e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.872+0000 7f20dbd6e740 1 -- 172.21.15.18:0/1085752157 shutdown_connections 2024-04-16T17:08:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.872+0000 7f20dbd6e740 1 -- 172.21.15.18:0/1085752157 wait complete. 2024-04-16T17:08:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.872+0000 7f20dbd6e740 20 client.26522 trim_cache size 0 max 16384 2024-04-16T17:08:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:08:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:08:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:08:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:08:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 -- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20b80a3cf0 msgr2=0x7f20b80a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20b80a3cf0 0x7f20b80a61b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f20cc005fd0 tx=0x7f20cc009500 comp rx=0 tx=0).stop 2024-04-16T17:08:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 -- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4109540 msgr2=0x7f20d419b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4109540 0x7f20d419b200 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f20c002c240 tx=0x7f20c00042e0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 -- 172.21.15.18:0/3431368918 shutdown_connections 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20b80a3cf0 0x7f20b80a61b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419e4d0 0x7f20d419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20d4109540 0x7f20d419b200 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 --2- 172.21.15.18:0/3431368918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d4104470 0x7f20d419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.875+0000 7f20d9000640 1 -- 172.21.15.18:0/3431368918 >> 172.21.15.18:0/3431368918 conn(0x7f20d40ffcd0 msgr2=0x7f20d4102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.876+0000 7f20d9000640 1 -- 172.21.15.18:0/3431368918 shutdown_connections 2024-04-16T17:08:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:03.876+0000 7f20d9000640 1 -- 172.21.15.18:0/3431368918 wait complete. 2024-04-16T17:08:03.925 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:03 smithi022 ceph-mon[36023]: pgmap v102: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 968 KiB/s wr, 126 op/s 2024-04-16T17:08:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:03 smithi043 ceph-mon[36901]: pgmap v102: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 968 KiB/s wr, 126 op/s 2024-04-16T17:08:04.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 104085 3.50 MB/sec execute 17 sec latency 79.272 ms 2024-04-16T17:08:05.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 104878 3.49 MB/sec execute 18 sec latency 82.107 ms 2024-04-16T17:08:05.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:05 smithi022 ceph-mon[36023]: pgmap v103: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 955 KiB/s wr, 125 op/s 2024-04-16T17:08:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:05 smithi043 ceph-mon[36901]: pgmap v103: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 955 KiB/s wr, 125 op/s 2024-04-16T17:08:06.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 106104 3.72 MB/sec execute 19 sec latency 20.210 ms 2024-04-16T17:08:07.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 106936 3.82 MB/sec execute 20 sec latency 21.349 ms 2024-04-16T17:08:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:07 smithi022 ceph-mon[36023]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-04-16T17:08:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:07 smithi043 ceph-mon[36901]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-04-16T17:08:08.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 107358 3.68 MB/sec execute 21 sec latency 76.762 ms 2024-04-16T17:08:09.230 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:05 smithi018 ceph-mon[31776]: pgmap v103: 129 pgs: 129 active+clean; 36 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 955 KiB/s wr, 125 op/s 2024-04-16T17:08:09.230 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:07 smithi018 ceph-mon[31776]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-04-16T17:08:09.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 107698 3.57 MB/sec execute 22 sec latency 63.706 ms 2024-04-16T17:08:09.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:09 smithi022 ceph-mon[36023]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-16T17:08:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:09 smithi043 ceph-mon[36901]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-16T17:08:10.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 108892 3.64 MB/sec execute 23 sec latency 24.880 ms 2024-04-16T17:08:11.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 110143 3.93 MB/sec execute 24 sec latency 11.998 ms 2024-04-16T17:08:11.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:11 smithi022 ceph-mon[36023]: pgmap v106: 129 pgs: 129 active+clean; 36 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 217 op/s 2024-04-16T17:08:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:11 smithi043 ceph-mon[36901]: pgmap v106: 129 pgs: 129 active+clean; 36 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 217 op/s 2024-04-16T17:08:12.541 INFO:tasks.workunit.client.0.smithi018.stdout: 1 110791 3.84 MB/sec execute 25 sec latency 22.782 ms 2024-04-16T17:08:13.543 INFO:tasks.workunit.client.0.smithi018.stdout: 1 111187 3.76 MB/sec execute 26 sec latency 99.571 ms 2024-04-16T17:08:13.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:13 smithi022 ceph-mon[36023]: pgmap v107: 129 pgs: 129 active+clean; 36 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-16T17:08:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:13 smithi043 ceph-mon[36901]: pgmap v107: 129 pgs: 129 active+clean; 36 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-16T17:08:14.543 INFO:tasks.workunit.client.0.smithi018.stdout: 1 112389 3.80 MB/sec execute 27 sec latency 51.942 ms 2024-04-16T17:08:14.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:08:14.961 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:08:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 -- 172.21.15.18:0/2959311623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec8102070 msgr2=0x7faec8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 --2- 172.21.15.18:0/2959311623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec8102070 0x7faec8102450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faebc009900 tx=0x7faebc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 -- 172.21.15.18:0/2959311623 shutdown_connections 2024-04-16T17:08:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 --2- 172.21.15.18:0/2959311623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faec8107140 0x7faec810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 --2- 172.21.15.18:0/2959311623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faec8102a20 0x7faec8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 --2- 172.21.15.18:0/2959311623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec8102070 0x7faec8102450 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 -- 172.21.15.18:0/2959311623 >> 172.21.15.18:0/2959311623 conn(0x7faec80fd950 msgr2=0x7faec80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 -- 172.21.15.18:0/2959311623 shutdown_connections 2024-04-16T17:08:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.183+0000 7faecee00640 1 -- 172.21.15.18:0/2959311623 wait complete. 2024-04-16T17:08:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.184+0000 7faecee00640 1 Processor -- start 2024-04-16T17:08:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecee00640 1 -- start start 2024-04-16T17:08:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecee00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faec8102070 0x7faec819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecee00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faec8107140 0x7faec819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81a35f0 0x7faec81a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecee00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faec8118860 con 0x7faec8102070 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecee00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faec81186e0 con 0x7faec8107140 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faec81189e0 con 0x7faec81a35f0 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faec8107140 0x7faec819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecda00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faec8102070 0x7faec819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faec8107140 0x7faec819f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35262/0 (socket says 172.21.15.18:35262) 2024-04-16T17:08:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faecd000640 1 -- 172.21.15.18:0/1753137962 learned_addr learned my addr 172.21.15.18:0/1753137962 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.185+0000 7faece400640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81a35f0 0x7faec81a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faecd000640 1 -- 172.21.15.18:0/1753137962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81a35f0 msgr2=0x7faec81a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faecd000640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81a35f0 0x7faec81a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faecd000640 1 -- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faec8102070 msgr2=0x7faec819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faecd000640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faec8102070 0x7faec819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faecd000640 1 -- 172.21.15.18:0/1753137962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec81a40e0 con 0x7faec8107140 2024-04-16T17:08:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faecd000640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faec8107140 0x7faec819f2c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faeb400e9f0 tx=0x7faeb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faec6000640 1 -- 172.21.15.18:0/1753137962 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb400ccc0 con 0x7faec8107140 2024-04-16T17:08:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faec6000640 1 -- 172.21.15.18:0/1753137962 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeb4005500 con 0x7faec8107140 2024-04-16T17:08:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faecee00640 1 -- 172.21.15.18:0/1753137962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec81a43d0 con 0x7faec8107140 2024-04-16T17:08:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faec6000640 1 -- 172.21.15.18:0/1753137962 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4010690 con 0x7faec8107140 2024-04-16T17:08:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.186+0000 7faecee00640 1 -- 172.21.15.18:0/1753137962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec81abb70 con 0x7faec8107140 2024-04-16T17:08:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.190+0000 7faed3039740 1 Processor -- start 2024-04-16T17:08:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.190+0000 7faed3039740 1 -- start start 2024-04-16T17:08:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.190+0000 7faec6000640 1 -- 172.21.15.18:0/1753137962 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faeb40046c0 con 0x7faec8107140 2024-04-16T17:08:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.190+0000 7faed3039740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 0x5644ccba5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.190+0000 7faed3039740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5cb0 0x5644ccbae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.190+0000 7faed3039740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccbae890 0x5644ccbb0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faed3039740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5644ccb0b120 con 0x5644ccba5390 2024-04-16T17:08:15.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faed3039740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5644ccab1340 con 0x5644ccbae890 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faed3039740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5644ccb070e0 con 0x5644ccba5cb0 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5cb0 0x5644ccbae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccbae890 0x5644ccbb0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5cb0 0x5644ccbae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43906/0 (socket says 172.21.15.18:43906) 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecda00640 1 -- 172.21.15.18:0/552619713 learned_addr learned my addr 172.21.15.18:0/552619713 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faece400640 1 --2- 172.21.15.18:0/552619713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 0x5644ccba5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecda00640 1 -- 172.21.15.18:0/552619713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccbae890 msgr2=0x5644ccbb0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecda00640 1 --2- 172.21.15.18:0/552619713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccbae890 0x5644ccbb0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecda00640 1 -- 172.21.15.18:0/552619713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 msgr2=0x5644ccba5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecda00640 1 --2- 172.21.15.18:0/552619713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 0x5644ccba5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.191+0000 7faecda00640 1 -- 172.21.15.18:0/552619713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644cc8e77d0 con 0x5644ccba5cb0 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.192+0000 7faecda00640 1 --2- 172.21.15.18:0/552619713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5cb0 0x5644ccbae350 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faebc009900 tx=0x7faebc009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2a862e1cf8063e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.192+0000 7faec6000640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faeb00a3d00 0x7faeb00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.192+0000 7faebaa00640 1 -- 172.21.15.18:0/552619713 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc045070 con 0x5644ccba5cb0 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.192+0000 7faebaa00640 1 -- 172.21.15.18:0/552619713 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc02c880 con 0x5644ccba5cb0 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.192+0000 7faebaa00640 1 -- 172.21.15.18:0/552619713 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc004bf0 con 0x5644ccba5cb0 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.192+0000 7faec6000640 1 -- 172.21.15.18:0/1753137962 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7faeb40d0f60 con 0x7faec8107140 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.193+0000 7faece400640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faeb00a3d00 0x7faeb00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.193+0000 7faed3039740 1 -- 172.21.15.18:0/552619713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5cb0 msgr2=0x5644ccbae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.193+0000 7faed3039740 1 --2- 172.21.15.18:0/552619713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5cb0 0x5644ccbae350 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faebc009900 tx=0x7faebc009610 comp rx=0 tx=0).stop 2024-04-16T17:08:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.193+0000 7faed3039740 1 -- 172.21.15.18:0/552619713 shutdown_connections 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.193+0000 7faed3039740 1 --2- 172.21.15.18:0/552619713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccbae890 0x5644ccbb0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.193+0000 7faed3039740 1 --2- 172.21.15.18:0/552619713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5cb0 0x5644ccbae350 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.193+0000 7faed3039740 1 --2- 172.21.15.18:0/552619713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 0x5644ccba5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.193+0000 7faed3039740 1 -- 172.21.15.18:0/552619713 >> 172.21.15.18:0/552619713 conn(0x5644ccb9fac0 msgr2=0x5644ccb9fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.194+0000 7faed3039740 1 -- 172.21.15.18:0/552619713 shutdown_connections 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.194+0000 7faece400640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faeb00a3d00 0x7faeb00a61c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faec0005fd0 tx=0x7faec00096a0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.194+0000 7faed3039740 1 -- 172.21.15.18:0/552619713 wait complete. 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.194+0000 7faed3039740 1 -- start start 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.194+0000 7faed3039740 1 Processor -- start 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faed3039740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faed3039740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbabee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faed3039740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 0x5644ccbac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faed3039740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5644ccbb1620 con 0x5644ccbae890 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faed3039740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5644ccac4710 con 0x5644ccba5cb0 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faed3039740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5644ccbb17a0 con 0x5644ccba5390 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faed3039740 10 client.0 _ll_register_callbacks cb 0x5644ccabfaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faecda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faecd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbabee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faecda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbab9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43914/0 (socket says 172.21.15.18:43914) 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faecda00640 1 -- 172.21.15.18:0/3252718719 learned_addr learned my addr 172.21.15.18:0/3252718719 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.195+0000 7faece400640 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 0x5644ccbac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faecd000640 1 -- 172.21.15.18:0/3252718719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 msgr2=0x5644ccbab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faecd000640 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faecd000640 1 -- 172.21.15.18:0/3252718719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 msgr2=0x5644ccbac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faecd000640 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 0x5644ccbac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faecd000640 1 -- 172.21.15.18:0/3252718719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644cc81d3f0 con 0x5644ccba5cb0 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faecda00640 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faecd000640 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbabee0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faeb40ce2a0 tx=0x7faeb4004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78aae71455a80d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faeaf400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faeaf400640 1 -- 172.21.15.18:0/3252718719 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb409c450 con 0x5644ccba5cb0 2024-04-16T17:08:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faeaf400640 1 -- 172.21.15.18:0/3252718719 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7faeb40a52d0 con 0x5644ccba5cb0 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faed3039740 1 -- 172.21.15.18:0/3252718719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faebc02c880 con 0x5644ccba5cb0 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.196+0000 7faeaf400640 1 -- 172.21.15.18:0/3252718719 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faeb40056b0 con 0x5644ccba5cb0 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faed3039740 10 client.26534 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faed3039740 1 -- 172.21.15.18:0/3252718719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644cc7dd0f0 con 0x5644ccba5cb0 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faeaf400640 1 -- 172.21.15.18:0/3252718719 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc02c880 con 0x5644ccba5cb0 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faeaf400640 1 -- 172.21.15.18:0/3252718719 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faeb40d4b80 con 0x5644ccba5cb0 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faed3039740 10 client.26534 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faed3039740 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faed3039740 20 client.26534 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faed3039740 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5644ccbbebd0 0x5644ccbc0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faed3039740 4 client.26534 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faed3039740 1 -- 172.21.15.18:0/3252718719 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644ccb18b40 con 0x5644ccbbebd0 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.197+0000 7faecda00640 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5644ccbbebd0 0x5644ccbc0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.198+0000 7faecda00640 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5644ccbbebd0 0x5644ccbc0ff0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c333efdc0343d2 server_cookie=bf665768b252d0df in_seq=0 out_seq=0 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.198+0000 7faeaf400640 10 client.26534 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faeaf400640 1 -- 172.21.15.18:0/3252718719 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644ccb18b40 con 0x5644ccbbebd0 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faeaf400640 10 client.26534 handle_command_reply: tid=0 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 client.26534 shutdown 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 -- 172.21.15.18:0/3252718719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 msgr2=0x5644ccbabee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbabee0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faeb40ce2a0 tx=0x7faeb4004b10 comp rx=0 tx=0).stop 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 -- 172.21.15.18:0/3252718719 shutdown_connections 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5644ccbbebd0 0x5644ccbc0ff0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 0x5644ccbac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbabee0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 --2- 172.21.15.18:0/3252718719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faeaf400640 0 client.26534 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:15.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 -- 172.21.15.18:0/3252718719 >> 172.21.15.18:0/3252718719 conn(0x5644ccb9fac0 msgr2=0x5644ccba1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 -- 172.21.15.18:0/3252718719 shutdown_connections 2024-04-16T17:08:15.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.199+0000 7faed3039740 1 -- 172.21.15.18:0/3252718719 wait complete. 2024-04-16T17:08:15.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.200+0000 7faed3039740 20 client.26534 trim_cache size 0 max 16384 2024-04-16T17:08:15.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.221+0000 7faed3039740 1 Processor -- start 2024-04-16T17:08:15.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.221+0000 7faed3039740 1 -- start start 2024-04-16T17:08:15.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faed3039740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 0x5644ccbaa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faed3039740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faed3039740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccbae890 0x5644ccbac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faed3039740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faec81b81b0 con 0x5644ccba5390 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faed3039740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faec81b5a80 con 0x5644ccba5cb0 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faed3039740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faec81b38d0 con 0x5644ccbae890 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faece400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccbae890 0x5644ccbac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faecda00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 0x5644ccbaa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faece400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccbae890 0x5644ccbac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43922/0 (socket says 172.21.15.18:43922) 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faece400640 1 -- 172.21.15.18:0/1315434419 learned_addr learned my addr 172.21.15.18:0/1315434419 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faecd000640 1 --2- 172.21.15.18:0/1315434419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faece400640 1 -- 172.21.15.18:0/1315434419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 msgr2=0x5644ccbba4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faece400640 1 --2- 172.21.15.18:0/1315434419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbba4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faece400640 1 -- 172.21.15.18:0/1315434419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 msgr2=0x5644ccbaa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.222+0000 7faece400640 1 --2- 172.21.15.18:0/1315434419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 0x5644ccbaa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.223+0000 7faece400640 1 -- 172.21.15.18:0/1315434419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ccb9f5f0 con 0x5644ccbae890 2024-04-16T17:08:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.223+0000 7faece400640 1 --2- 172.21.15.18:0/1315434419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccbae890 0x5644ccbac240 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faec0010600 tx=0x7faec0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e227234d66682e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.223+0000 7faeba000640 1 -- 172.21.15.18:0/1315434419 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0028440 con 0x5644ccbae890 2024-04-16T17:08:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.223+0000 7faeba000640 1 -- 172.21.15.18:0/1315434419 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec001e8f0 con 0x5644ccbae890 2024-04-16T17:08:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.223+0000 7faeba000640 1 -- 172.21.15.18:0/1315434419 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0027850 con 0x5644ccbae890 2024-04-16T17:08:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 -- 172.21.15.18:0/1315434419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccbae890 msgr2=0x5644ccbac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 --2- 172.21.15.18:0/1315434419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccbae890 0x5644ccbac240 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faec0010600 tx=0x7faec0015000 comp rx=0 tx=0).stop 2024-04-16T17:08:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 -- 172.21.15.18:0/1315434419 shutdown_connections 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 --2- 172.21.15.18:0/1315434419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccbae890 0x5644ccbac240 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 --2- 172.21.15.18:0/1315434419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbba4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 --2- 172.21.15.18:0/1315434419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccba5390 0x5644ccbaa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 -- 172.21.15.18:0/1315434419 >> 172.21.15.18:0/1315434419 conn(0x5644ccb9fac0 msgr2=0x5644ccb9eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 -- 172.21.15.18:0/1315434419 shutdown_connections 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.224+0000 7faed3039740 1 -- 172.21.15.18:0/1315434419 wait complete. 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.225+0000 7faed3039740 1 -- start start 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.225+0000 7faed3039740 1 Processor -- start 2024-04-16T17:08:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.225+0000 7faed3039740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbb6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.225+0000 7faed3039740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbb21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.225+0000 7faed3039740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 0x5644ccbb2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.225+0000 7faed3039740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faec81b3f40 con 0x5644ccbae890 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.225+0000 7faed3039740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faec81b53b0 con 0x5644ccba5cb0 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.225+0000 7faed3039740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faec81b4610 con 0x5644ccba5390 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faed3039740 10 client.0 _ll_register_callbacks cb 0x7faeb400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbb6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbb6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43930/0 (socket says 172.21.15.18:43930) 2024-04-16T17:08:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbb21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecda00640 1 -- 172.21.15.18:0/3089967230 learned_addr learned my addr 172.21.15.18:0/3089967230 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faece400640 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 0x5644ccbb2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecda00640 1 -- 172.21.15.18:0/3089967230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 msgr2=0x5644ccbb21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecda00640 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbb21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecda00640 1 -- 172.21.15.18:0/3089967230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 msgr2=0x5644ccbb2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecda00640 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 0x5644ccbb2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecda00640 1 -- 172.21.15.18:0/3089967230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ccbb2f70 con 0x5644ccba5390 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.226+0000 7faecd000640 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbb21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.227+0000 7faecda00640 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbb6e50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faebc006010 tx=0x7faebc0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f762101cb247d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.227+0000 7faeaf400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.227+0000 7faeaf400640 1 -- 172.21.15.18:0/3089967230 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc045040 con 0x5644ccba5390 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.227+0000 7faed3039740 1 -- 172.21.15.18:0/3089967230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec001e8f0 con 0x5644ccba5390 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.227+0000 7faeaf400640 1 -- 172.21.15.18:0/3089967230 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7faebc04a860 con 0x5644ccba5390 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.227+0000 7faeaf400640 1 -- 172.21.15.18:0/3089967230 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faebc060050 con 0x5644ccba5390 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.227+0000 7faed3039740 10 client.26515 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.227+0000 7faed3039740 1 -- 172.21.15.18:0/3089967230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644ccc25240 con 0x5644ccba5390 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faeaf400640 1 -- 172.21.15.18:0/3089967230 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec001e8f0 con 0x5644ccba5390 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faeaf400640 1 -- 172.21.15.18:0/3089967230 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faebc034880 con 0x5644ccba5390 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faed3039740 10 client.26515 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faed3039740 10 client.26515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faed3039740 20 client.26515 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faed3039740 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5644ccc25640 0x5644ccc27a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faed3039740 4 client.26515 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faed3039740 1 -- 172.21.15.18:0/3089967230 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644ccb129b0 con 0x5644ccc25640 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.228+0000 7faecd000640 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5644ccc25640 0x5644ccc27a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.229+0000 7faecd000640 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5644ccc25640 0x5644ccc27a30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fef9754b5b45b1c server_cookie=b007bfb9595c19e0 in_seq=0 out_seq=0 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.229+0000 7faeaf400640 10 client.26515 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.229+0000 7faeaf400640 1 -- 172.21.15.18:0/3089967230 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5644ccb129b0 con 0x5644ccc25640 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.229+0000 7faeaf400640 10 client.26515 handle_command_reply: tid=0 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.229+0000 7faed3039740 1 client.26515 shutdown 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.229+0000 7faed3039740 1 -- 172.21.15.18:0/3089967230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 msgr2=0x5644ccbb6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.229+0000 7faed3039740 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbb6e50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faebc006010 tx=0x7faebc0047d0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 1 -- 172.21.15.18:0/3089967230 shutdown_connections 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5644ccc25640 0x5644ccc27a30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5644ccbae890 0x5644ccbb2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644ccba5cb0 0x5644ccbb21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 1 --2- 172.21.15.18:0/3089967230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5644ccba5390 0x5644ccbb6e50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faeaf400640 0 client.26515 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:15.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 1 -- 172.21.15.18:0/3089967230 >> 172.21.15.18:0/3089967230 conn(0x5644ccb9fac0 msgr2=0x5644ccba0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 1 -- 172.21.15.18:0/3089967230 shutdown_connections 2024-04-16T17:08:15.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 1 -- 172.21.15.18:0/3089967230 wait complete. 2024-04-16T17:08:15.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.230+0000 7faed3039740 20 client.26515 trim_cache size 0 max 16384 2024-04-16T17:08:15.239 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:08:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:08:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:08:15.239 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:08:15.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 -- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faeb00a3d00 msgr2=0x7faeb00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faeb00a3d00 0x7faeb00a61c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faec0005fd0 tx=0x7faec00096a0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 -- 172.21.15.18:0/1753137962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faec8107140 msgr2=0x7faec819f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faec8107140 0x7faec819f2c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faeb400e9f0 tx=0x7faeb400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 -- 172.21.15.18:0/1753137962 shutdown_connections 2024-04-16T17:08:15.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faeb00a3d00 0x7faeb00a61c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81a35f0 0x7faec81a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faec8107140 0x7faec819f2c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 --2- 172.21.15.18:0/1753137962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faec8102070 0x7faec819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 -- 172.21.15.18:0/1753137962 >> 172.21.15.18:0/1753137962 conn(0x7faec80fd950 msgr2=0x7faec80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 -- 172.21.15.18:0/1753137962 shutdown_connections 2024-04-16T17:08:15.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.231+0000 7faecee00640 1 -- 172.21.15.18:0/1753137962 wait complete. 2024-04-16T17:08:15.251 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:08:15.252 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:08:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 -- 172.21.15.18:0/3221829714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da4074430 msgr2=0x7f2da4074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 --2- 172.21.15.18:0/3221829714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da4074430 0x7f2da4074810 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009900 tx=0x7f2d9402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 -- 172.21.15.18:0/3221829714 shutdown_connections 2024-04-16T17:08:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 --2- 172.21.15.18:0/3221829714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2da410e700 0x7f2da4110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 --2- 172.21.15.18:0/3221829714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4074d50 0x7f2da410e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 --2- 172.21.15.18:0/3221829714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da4074430 0x7f2da4074810 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 -- 172.21.15.18:0/3221829714 >> 172.21.15.18:0/3221829714 conn(0x7f2da40fd9f0 msgr2=0x7f2da40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 -- 172.21.15.18:0/3221829714 shutdown_connections 2024-04-16T17:08:15.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.451+0000 7f2da3e00640 1 -- 172.21.15.18:0/3221829714 wait complete. 2024-04-16T17:08:15.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.452+0000 7f2da3e00640 1 Processor -- start 2024-04-16T17:08:15.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.452+0000 7f2da3e00640 1 -- start start 2024-04-16T17:08:15.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2da4074430 0x7f2da41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4074d50 0x7f2da41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da410e700 0x7f2da419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da3e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41144e0 con 0x7f2da4074430 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4114360 con 0x7f2da4074d50 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4114660 con 0x7f2da410e700 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2da4074430 0x7f2da41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2da4074430 0x7f2da41a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37644/0 (socket says 172.21.15.18:37644) 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da2a00640 1 -- 172.21.15.18:0/2179302067 learned_addr learned my addr 172.21.15.18:0/2179302067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.453+0000 7f2da3400640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da410e700 0x7f2da419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da2000640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4074d50 0x7f2da41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da2a00640 1 -- 172.21.15.18:0/2179302067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da410e700 msgr2=0x7f2da419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da2a00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da410e700 0x7f2da419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da2a00640 1 -- 172.21.15.18:0/2179302067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4074d50 msgr2=0x7f2da41a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da2a00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4074d50 0x7f2da41a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da2a00640 1 -- 172.21.15.18:0/2179302067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da419afb0 con 0x7f2da4074430 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da3400640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da410e700 0x7f2da419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da2000640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4074d50 0x7f2da41a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2da2a00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2da4074430 0x7f2da41a0670 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d94004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.454+0000 7f2d9b400640 1 -- 172.21.15.18:0/2179302067 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94042020 con 0x7f2da4074430 2024-04-16T17:08:15.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.455+0000 7f2d9b400640 1 -- 172.21.15.18:0/2179302067 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d940054e0 con 0x7f2da4074430 2024-04-16T17:08:15.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.455+0000 7f2d9b400640 1 -- 172.21.15.18:0/2179302067 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94003a50 con 0x7f2da4074430 2024-04-16T17:08:15.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.455+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da419b240 con 0x7f2da4074430 2024-04-16T17:08:15.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.455+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41a74c0 con 0x7f2da4074430 2024-04-16T17:08:15.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.456+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da4075ae0 con 0x7f2da4074430 2024-04-16T17:08:15.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.458+0000 7f2d9b400640 1 -- 172.21.15.18:0/2179302067 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d94045020 con 0x7f2da4074430 2024-04-16T17:08:15.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.460+0000 7f2d9b400640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2d880a3d90 0x7f2d880a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.460+0000 7f2d9b400640 1 -- 172.21.15.18:0/2179302067 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2d940f2c00 con 0x7f2da4074430 2024-04-16T17:08:15.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.460+0000 7f2da2000640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2d880a3d90 0x7f2d880a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.461+0000 7f2da2000640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2d880a3d90 0x7f2d880a6250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2da410fa10 tx=0x7f2d90005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.467+0000 7f2d9b400640 1 -- 172.21.15.18:0/2179302067 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d9404c020 con 0x7f2da4074430 2024-04-16T17:08:15.543 INFO:tasks.workunit.client.0.smithi018.stdout: 1 113678 4.05 MB/sec execute 28 sec latency 12.805 ms 2024-04-16T17:08:15.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.736+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2da4109880 con 0x7f2da4074430 2024-04-16T17:08:15.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.737+0000 7f2d9b400640 1 -- 172.21.15.18:0/2179302067 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7f2d940bf0c0 con 0x7f2da4074430 2024-04-16T17:08:15.739 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:08:15.739 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:08:15.739 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.739+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2d880a3d90 msgr2=0x7f2d880a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.739+0000 7f2da3e00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2d880a3d90 0x7f2d880a6250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2da410fa10 tx=0x7f2d90005f90 comp rx=0 tx=0).stop 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.739+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2da4074430 msgr2=0x7f2da41a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.739+0000 7f2da3e00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2da4074430 0x7f2da41a0670 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d94004360 comp rx=0 tx=0).stop 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.739+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 shutdown_connections 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.740+0000 7f2da3e00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2d880a3d90 0x7f2d880a6250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.740+0000 7f2da3e00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da410e700 0x7f2da419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.740+0000 7f2da3e00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4074d50 0x7f2da41a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.740+0000 7f2da3e00640 1 --2- 172.21.15.18:0/2179302067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2da4074430 0x7f2da41a0670 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.740+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 >> 172.21.15.18:0/2179302067 conn(0x7f2da40fd9f0 msgr2=0x7f2da40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.740+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 shutdown_connections 2024-04-16T17:08:15.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.740+0000 7f2da3e00640 1 -- 172.21.15.18:0/2179302067 wait complete. 2024-04-16T17:08:15.760 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:08:15.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:15 smithi022 ceph-mon[36023]: pgmap v108: 129 pgs: 129 active+clean; 36 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-04-16T17:08:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 -- 172.21.15.18:0/4154799168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc074cc0 msgr2=0x7f50cc1128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4154799168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc074cc0 0x7f50cc1128d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009900 tx=0x7f50b802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 -- 172.21.15.18:0/4154799168 shutdown_connections 2024-04-16T17:08:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4154799168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50cc112e10 0x7f50cc115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4154799168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc074cc0 0x7f50cc1128d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4154799168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50cc0743a0 0x7f50cc074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 -- 172.21.15.18:0/4154799168 >> 172.21.15.18:0/4154799168 conn(0x7f50cc0fd950 msgr2=0x7f50cc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 -- 172.21.15.18:0/4154799168 shutdown_connections 2024-04-16T17:08:15.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.966+0000 7f50d0c00640 1 -- 172.21.15.18:0/4154799168 wait complete. 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.967+0000 7f50d0c00640 1 Processor -- start 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.967+0000 7f50d0c00640 1 -- start start 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50d0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50cc0743a0 0x7f50cc19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50d0c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50cc074cc0 0x7f50cc19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50d0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc112e10 0x7f50cc1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50d0c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc118830 con 0x7f50cc074cc0 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50d0c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc1186b0 con 0x7f50cc112e10 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50d0c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc1189b0 con 0x7f50cc0743a0 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc112e10 0x7f50cc1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50caa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50cc074cc0 0x7f50cc19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc112e10 0x7f50cc1a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35316/0 (socket says 172.21.15.18:35316) 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cbe00640 1 -- 172.21.15.18:0/4133975000 learned_addr learned my addr 172.21.15.18:0/4133975000 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cb400640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50cc0743a0 0x7f50cc19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cbe00640 1 -- 172.21.15.18:0/4133975000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50cc0743a0 msgr2=0x7f50cc19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cbe00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50cc0743a0 0x7f50cc19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cbe00640 1 -- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50cc074cc0 msgr2=0x7f50cc19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cbe00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50cc074cc0 0x7f50cc19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cbe00640 1 -- 172.21.15.18:0/4133975000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc1a3d70 con 0x7f50cc112e10 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.968+0000 7f50cb400640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50cc0743a0 0x7f50cc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.969+0000 7f50cbe00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc112e10 0x7f50cc1a3640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f50c400eb40 tx=0x7f50c400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.969+0000 7f50c3e00640 1 -- 172.21.15.18:0/4133975000 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4004b20 con 0x7f50cc112e10 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.969+0000 7f50c3e00640 1 -- 172.21.15.18:0/4133975000 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c4004cc0 con 0x7f50cc112e10 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.969+0000 7f50c3e00640 1 -- 172.21.15.18:0/4133975000 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4010850 con 0x7f50cc112e10 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.969+0000 7f50d0c00640 1 -- 172.21.15.18:0/4133975000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc1a4060 con 0x7f50cc112e10 2024-04-16T17:08:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.970+0000 7f50d0c00640 1 -- 172.21.15.18:0/4133975000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc1ab8f0 con 0x7f50cc112e10 2024-04-16T17:08:15.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.973+0000 7f50d4fea740 1 Processor -- start 2024-04-16T17:08:15.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.975+0000 7f50c3e00640 1 -- 172.21.15.18:0/4133975000 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f50c4005020 con 0x7f50cc112e10 2024-04-16T17:08:15.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.975+0000 7f50d4fea740 1 -- start start 2024-04-16T17:08:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.975+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84811b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.975+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811b830 0x55f848123ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.975+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848126800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.975+0000 7f50d4fea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f848080020 con 0x55f84811af10 2024-04-16T17:08:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.975+0000 7f50d4fea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f848026460 con 0x55f848124410 2024-04-16T17:08:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.975+0000 7f50d4fea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f84807bfe0 con 0x55f84811b830 2024-04-16T17:08:15.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811b830 0x55f848123ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50caa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84811b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811b830 0x55f848123ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43968/0 (socket says 172.21.15.18:43968) 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50cb400640 1 -- 172.21.15.18:0/3044475048 learned_addr learned my addr 172.21.15.18:0/3044475048 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50caa00640 1 --2- 172.21.15.18:0/3044475048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848126800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50c3e00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50b00a3da0 0x7f50b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50cb400640 1 -- 172.21.15.18:0/3044475048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 msgr2=0x55f848126800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50cb400640 1 --2- 172.21.15.18:0/3044475048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848126800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50cb400640 1 -- 172.21.15.18:0/3044475048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 msgr2=0x55f84811b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.976+0000 7f50cb400640 1 --2- 172.21.15.18:0/3044475048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84811b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.977+0000 7f50cb400640 1 -- 172.21.15.18:0/3044475048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f84802e040 con 0x55f84811b830 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.977+0000 7f50c3e00640 1 -- 172.21.15.18:0/4133975000 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f50c40d1340 con 0x7f50cc112e10 2024-04-16T17:08:15.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.977+0000 7f50cbe00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50b00a3da0 0x7f50b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.977+0000 7f50cb400640 1 --2- 172.21.15.18:0/3044475048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811b830 0x55f848123ed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00bd40 tx=0x7f50bc00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07632225dd59c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.978+0000 7f50b7e00640 1 -- 172.21.15.18:0/3044475048 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc015070 con 0x55f84811b830 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.978+0000 7f50b7e00640 1 -- 172.21.15.18:0/3044475048 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc00be30 con 0x55f84811b830 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.978+0000 7f50b7e00640 1 -- 172.21.15.18:0/3044475048 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc011a30 con 0x55f84811b830 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 -- 172.21.15.18:0/3044475048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811b830 msgr2=0x55f848123ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50cbe00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50b00a3da0 0x7f50b00a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f50c409cec0 tx=0x7f50c40d1640 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 --2- 172.21.15.18:0/3044475048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811b830 0x55f848123ed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00bd40 tx=0x7f50bc00f730 comp rx=0 tx=0).stop 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 -- 172.21.15.18:0/3044475048 shutdown_connections 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 --2- 172.21.15.18:0/3044475048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848126800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 --2- 172.21.15.18:0/3044475048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811b830 0x55f848123ed0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 --2- 172.21.15.18:0/3044475048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84811b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 -- 172.21.15.18:0/3044475048 >> 172.21.15.18:0/3044475048 conn(0x55f848115640 msgr2=0x55f848115a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 -- 172.21.15.18:0/3044475048 shutdown_connections 2024-04-16T17:08:15.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 -- 172.21.15.18:0/3044475048 wait complete. 2024-04-16T17:08:15.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.979+0000 7f50d4fea740 1 -- start start 2024-04-16T17:08:15.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.980+0000 7f50d4fea740 1 Processor -- start 2024-04-16T17:08:15.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.980+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f8481214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.980+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f848121a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.980+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848121f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.980+0000 7f50d4fea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f848127200 con 0x55f84811b830 2024-04-16T17:08:15.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.980+0000 7f50d4fea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f848039830 con 0x55f848124410 2024-04-16T17:08:15.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.980+0000 7f50d4fea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f848127380 con 0x55f84811af10 2024-04-16T17:08:15.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.980+0000 7f50d4fea740 10 client.0 _ll_register_callbacks cb 0x55f848041150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:15.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f8481214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f8481214d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43974/0 (socket says 172.21.15.18:43974) 2024-04-16T17:08:15.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50caa00640 1 -- 172.21.15.18:0/626986378 learned_addr learned my addr 172.21.15.18:0/626986378 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50cb400640 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f848121a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50caa00640 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848121f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50cb400640 1 -- 172.21.15.18:0/626986378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 msgr2=0x55f8481214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50cb400640 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f8481214d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50cb400640 1 -- 172.21.15.18:0/626986378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 msgr2=0x55f848121f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50cb400640 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848121f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50cb400640 1 -- 172.21.15.18:0/626986378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f84803d920 con 0x55f84811b830 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.981+0000 7f50caa00640 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f8481214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.982+0000 7f50cb400640 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f848121a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0115b0 tx=0x7f50bc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331fb205ff433f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.982+0000 7f50b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.982+0000 7f50b4c00640 1 -- 172.21.15.18:0/626986378 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc028060 con 0x55f84811b830 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.982+0000 7f50b4c00640 1 -- 172.21.15.18:0/626986378 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f50bc005450 con 0x55f84811b830 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.982+0000 7f50d4fea740 1 -- 172.21.15.18:0/626986378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50bc00be30 con 0x55f84811b830 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.982+0000 7f50b4c00640 1 -- 172.21.15.18:0/626986378 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50bc02f050 con 0x55f84811b830 2024-04-16T17:08:15.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.982+0000 7f50d4fea740 10 client.16368 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:15.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.982+0000 7f50d4fea740 1 -- 172.21.15.18:0/626986378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f84800b7f0 con 0x55f84811b830 2024-04-16T17:08:15.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.983+0000 7f50b4c00640 1 -- 172.21.15.18:0/626986378 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc00be30 con 0x55f84811b830 2024-04-16T17:08:15.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.983+0000 7f50b4c00640 1 -- 172.21.15.18:0/626986378 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50bc022030 con 0x55f84811b830 2024-04-16T17:08:15.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.983+0000 7f50d4fea740 10 client.16368 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:15.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.983+0000 7f50d4fea740 10 client.16368 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:08:15.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.983+0000 7f50d4fea740 20 client.16368 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:15.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.983+0000 7f50d4fea740 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f848145720 0x55f848147b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:15.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.983+0000 7f50d4fea740 4 client.16368 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:15.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.983+0000 7f50d4fea740 1 -- 172.21.15.18:0/626986378 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f84808da40 con 0x55f848145720 2024-04-16T17:08:15.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.984+0000 7f50caa00640 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f848145720 0x55f848147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.984+0000 7f50caa00640 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f848145720 0x55f848147b10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2d4653a0135491a server_cookie=ec01516b650f4fcb in_seq=0 out_seq=0 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.984+0000 7f50b4c00640 10 client.16368 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50b4c00640 1 -- 172.21.15.18:0/626986378 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f84808da40 con 0x55f848145720 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50b4c00640 10 client.16368 handle_command_reply: tid=0 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 client.16368 shutdown 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 -- 172.21.15.18:0/626986378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 msgr2=0x55f848121a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f848121a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0115b0 tx=0x7f50bc00bab0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 -- 172.21.15.18:0/626986378 shutdown_connections 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f848145720 0x55f848147b10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848121f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f848121a10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 --2- 172.21.15.18:0/626986378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f8481214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50d4fea740 1 -- 172.21.15.18:0/626986378 >> 172.21.15.18:0/626986378 conn(0x55f848115640 msgr2=0x55f848117390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:15.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.985+0000 7f50b4c00640 0 client.16368 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:08:15.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.986+0000 7f50d4fea740 1 -- 172.21.15.18:0/626986378 shutdown_connections 2024-04-16T17:08:15.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.986+0000 7f50d4fea740 1 -- 172.21.15.18:0/626986378 wait complete. 2024-04-16T17:08:15.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:15.986+0000 7f50d4fea740 20 client.16368 trim_cache size 0 max 16384 2024-04-16T17:08:16.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:15 smithi043 ceph-mon[36901]: pgmap v108: 129 pgs: 129 active+clean; 36 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50d4fea740 1 Processor -- start 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50d4fea740 1 -- start start 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f848123c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f8481300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848120630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50d4fea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc1afb10 con 0x55f84811b830 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50d4fea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc1ad3e0 con 0x55f848124410 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50d4fea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc10e010 con 0x55f84811af10 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f848123c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50cb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f8481300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f848123c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43980/0 (socket says 172.21.15.18:43980) 2024-04-16T17:08:16.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50caa00640 1 -- 172.21.15.18:0/1394524881 learned_addr learned my addr 172.21.15.18:0/1394524881 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.001+0000 7f50caa00640 1 --2- 172.21.15.18:0/1394524881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848120630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50cb400640 1 -- 172.21.15.18:0/1394524881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 msgr2=0x55f848123c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50cb400640 1 --2- 172.21.15.18:0/1394524881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f848123c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50cb400640 1 -- 172.21.15.18:0/1394524881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 msgr2=0x55f848120630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50cb400640 1 --2- 172.21.15.18:0/1394524881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848120630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50cb400640 1 -- 172.21.15.18:0/1394524881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f847d923f0 con 0x55f84811b830 2024-04-16T17:08:16.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50cb400640 1 --2- 172.21.15.18:0/1394524881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f8481300c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0049e0 tx=0x7f50bc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b8b740952ebc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50b7400640 1 -- 172.21.15.18:0/1394524881 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc028050 con 0x55f84811b830 2024-04-16T17:08:16.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50b7400640 1 -- 172.21.15.18:0/1394524881 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc00bbc0 con 0x55f84811b830 2024-04-16T17:08:16.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.002+0000 7f50b7400640 1 -- 172.21.15.18:0/1394524881 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc015040 con 0x55f84811b830 2024-04-16T17:08:16.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.003+0000 7f50d4fea740 1 -- 172.21.15.18:0/1394524881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 msgr2=0x55f8481300c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.003+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1394524881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f8481300c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0049e0 tx=0x7f50bc00f730 comp rx=0 tx=0).stop 2024-04-16T17:08:16.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.003+0000 7f50d4fea740 1 -- 172.21.15.18:0/1394524881 shutdown_connections 2024-04-16T17:08:16.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.003+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1394524881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f848124410 0x55f848120630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.003+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1394524881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811b830 0x55f8481300c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.003+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1394524881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f84811af10 0x55f848123c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.003+0000 7f50d4fea740 1 -- 172.21.15.18:0/1394524881 >> 172.21.15.18:0/1394524881 conn(0x55f848115640 msgr2=0x55f848113e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.004+0000 7f50d4fea740 1 -- 172.21.15.18:0/1394524881 shutdown_connections 2024-04-16T17:08:16.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.004+0000 7f50d4fea740 1 -- 172.21.15.18:0/1394524881 wait complete. 2024-04-16T17:08:16.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.004+0000 7f50d4fea740 1 -- start start 2024-04-16T17:08:16.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.004+0000 7f50d4fea740 1 Processor -- start 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84812caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f84811b830 0x55f84811eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50d4fea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f848124410 0x55f84811f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50d4fea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc10e680 con 0x55f84811af10 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50d4fea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc1acd10 con 0x55f84811b830 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50d4fea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc1abf70 con 0x55f848124410 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50d4fea740 10 client.0 _ll_register_callbacks cb 0x7f50b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f84811b830 0x55f84811eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f84811b830 0x55f84811eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35356/0 (socket says 172.21.15.18:35356) 2024-04-16T17:08:16.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50cb400640 1 -- 172.21.15.18:0/1562848660 learned_addr learned my addr 172.21.15.18:0/1562848660 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50caa00640 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84812caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50caa00640 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f848124410 0x55f84811f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50cb400640 1 -- 172.21.15.18:0/1562848660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f848124410 msgr2=0x55f84811f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50cb400640 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f848124410 0x55f84811f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50cb400640 1 -- 172.21.15.18:0/1562848660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 msgr2=0x55f84812caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50cb400640 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84812caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50cb400640 1 -- 172.21.15.18:0/1562848660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f847e5c7d0 con 0x55f84811b830 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.005+0000 7f50caa00640 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84812caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.006+0000 7f50cb400640 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f84811b830 0x55f84811eab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0058c0 tx=0x7f50bc00b3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b31b6e7acf9afbd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.006+0000 7f50b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.006+0000 7f50b4c00640 1 -- 172.21.15.18:0/1562848660 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc040060 con 0x55f84811b830 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.006+0000 7f50d4fea740 1 -- 172.21.15.18:0/1562848660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50bc00bbc0 con 0x55f84811b830 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.006+0000 7f50b4c00640 1 -- 172.21.15.18:0/1562848660 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f50bc005250 con 0x55f84811b830 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50b4c00640 1 -- 172.21.15.18:0/1562848660 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50bc015040 con 0x55f84811b830 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50d4fea740 10 client.26564 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50d4fea740 1 -- 172.21.15.18:0/1562848660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8481aabf0 con 0x55f84811b830 2024-04-16T17:08:16.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50b4c00640 1 -- 172.21.15.18:0/1562848660 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc00bbc0 con 0x55f84811b830 2024-04-16T17:08:16.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50b4c00640 1 -- 172.21.15.18:0/1562848660 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50bc022040 con 0x55f84811b830 2024-04-16T17:08:16.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50d4fea740 10 client.26564 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:16.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50d4fea740 10 client.26564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:08:16.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50d4fea740 20 client.26564 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:16.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f8481ab250 0x55f8481ad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50d4fea740 4 client.26564 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:16.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.007+0000 7f50d4fea740 1 -- 172.21.15.18:0/1562848660 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8480878b0 con 0x55f8481ab250 2024-04-16T17:08:16.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.008+0000 7f50caa00640 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f8481ab250 0x55f8481ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.008+0000 7f50caa00640 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f8481ab250 0x55f8481ad640 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=175b38eeee811bb2 server_cookie=a240807ce8678d1b in_seq=0 out_seq=0 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.008+0000 7f50b4c00640 10 client.26564 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50b4c00640 1 -- 172.21.15.18:0/1562848660 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8480878b0 con 0x55f8481ab250 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50b4c00640 10 client.26564 handle_command_reply: tid=0 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 client.26564 shutdown 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 -- 172.21.15.18:0/1562848660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f84811b830 msgr2=0x55f84811eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f84811b830 0x55f84811eab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0058c0 tx=0x7f50bc00b3a0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 -- 172.21.15.18:0/1562848660 shutdown_connections 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f8481ab250 0x55f8481ad640 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f848124410 0x55f84811f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f84811b830 0x55f84811eab0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 --2- 172.21.15.18:0/1562848660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f84811af10 0x55f84812caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 -- 172.21.15.18:0/1562848660 >> 172.21.15.18:0/1562848660 conn(0x55f848115640 msgr2=0x55f848121d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50b4c00640 0 client.26564 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:08:16.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 -- 172.21.15.18:0/1562848660 shutdown_connections 2024-04-16T17:08:16.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 1 -- 172.21.15.18:0/1562848660 wait complete. 2024-04-16T17:08:16.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.009+0000 7f50d4fea740 20 client.26564 trim_cache size 0 max 16384 2024-04-16T17:08:16.013 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 -- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50b00a3da0 msgr2=0x7f50b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50b00a3da0 0x7f50b00a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f50c409cec0 tx=0x7f50c40d1640 comp rx=0 tx=0).stop 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 -- 172.21.15.18:0/4133975000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc112e10 msgr2=0x7f50cc1a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc112e10 0x7f50cc1a3640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f50c400eb40 tx=0x7f50c400c2d0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 -- 172.21.15.18:0/4133975000 shutdown_connections 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50b00a3da0 0x7f50b00a6260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50cc112e10 0x7f50cc1a3640 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50cc074cc0 0x7f50cc19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 --2- 172.21.15.18:0/4133975000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50cc0743a0 0x7f50cc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 -- 172.21.15.18:0/4133975000 >> 172.21.15.18:0/4133975000 conn(0x7f50cc0fd950 msgr2=0x7f50cc0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.010+0000 7f50d0c00640 1 -- 172.21.15.18:0/4133975000 shutdown_connections 2024-04-16T17:08:16.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.011+0000 7f50d0c00640 1 -- 172.21.15.18:0/4133975000 wait complete. 2024-04-16T17:08:16.029 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:08:16.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 -- 172.21.15.18:0/1183035998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc112e10 msgr2=0x7f39fc115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 --2- 172.21.15.18:0/1183035998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc112e10 0x7f39fc115200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009be0 tx=0x7f39f402bf80 comp rx=0 tx=0).stop 2024-04-16T17:08:16.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 -- 172.21.15.18:0/1183035998 shutdown_connections 2024-04-16T17:08:16.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 --2- 172.21.15.18:0/1183035998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc112e10 0x7f39fc115200 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 --2- 172.21.15.18:0/1183035998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39fc074cc0 0x7f39fc1128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 --2- 172.21.15.18:0/1183035998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39fc0743a0 0x7f39fc074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 -- 172.21.15.18:0/1183035998 >> 172.21.15.18:0/1183035998 conn(0x7f39fc0fd950 msgr2=0x7f39fc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 -- 172.21.15.18:0/1183035998 shutdown_connections 2024-04-16T17:08:16.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.234+0000 7f3a02400640 1 -- 172.21.15.18:0/1183035998 wait complete. 2024-04-16T17:08:16.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.235+0000 7f3a02400640 1 Processor -- start 2024-04-16T17:08:16.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a02400640 1 -- start start 2024-04-16T17:08:16.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a02400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39fc0743a0 0x7f39fc1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a02400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc074cc0 0x7f39fc1a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a02400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39fc112e10 0x7f39fc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a02400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc118800 con 0x7f39fc112e10 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a02400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc118680 con 0x7f39fc074cc0 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a02400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc118980 con 0x7f39fc0743a0 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f39fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc074cc0 0x7f39fc1a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f39fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc074cc0 0x7f39fc1a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35368/0 (socket says 172.21.15.18:35368) 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a01000640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39fc0743a0 0x7f39fc1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f39fbe00640 1 -- 172.21.15.18:0/1892060795 learned_addr learned my addr 172.21.15.18:0/1892060795 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.236+0000 7f3a01a00640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39fc112e10 0x7f39fc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.237+0000 7f3a01000640 1 -- 172.21.15.18:0/1892060795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc074cc0 msgr2=0x7f39fc1a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.237+0000 7f3a01000640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc074cc0 0x7f39fc1a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.237+0000 7f3a01000640 1 -- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39fc112e10 msgr2=0x7f39fc19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.237+0000 7f3a01000640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39fc112e10 0x7f39fc19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.237+0000 7f3a01000640 1 -- 172.21.15.18:0/1892060795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc19f3e0 con 0x7f39fc0743a0 2024-04-16T17:08:16.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.237+0000 7f39fbe00640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc074cc0 0x7f39fc1a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:16.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.237+0000 7f3a01a00640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39fc112e10 0x7f39fc19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:16.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.237+0000 7f3a01000640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39fc0743a0 0x7f39fc1a4a60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00e790 tx=0x7f39ec00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.238+0000 7f39f9600640 1 -- 172.21.15.18:0/1892060795 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00eea0 con 0x7f39fc0743a0 2024-04-16T17:08:16.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.238+0000 7f3a02400640 1 -- 172.21.15.18:0/1892060795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc19f6d0 con 0x7f39fc0743a0 2024-04-16T17:08:16.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.238+0000 7f39f9600640 1 -- 172.21.15.18:0/1892060795 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ec00ccc0 con 0x7f39fc0743a0 2024-04-16T17:08:16.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.238+0000 7f39f9600640 1 -- 172.21.15.18:0/1892060795 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec018760 con 0x7f39fc0743a0 2024-04-16T17:08:16.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.238+0000 7f3a02400640 1 -- 172.21.15.18:0/1892060795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc1ab9e0 con 0x7f39fc0743a0 2024-04-16T17:08:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.241+0000 7f3a06684740 1 Processor -- start 2024-04-16T17:08:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.241+0000 7f39f9600640 1 -- 172.21.15.18:0/1892060795 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f39ec010430 con 0x7f39fc0743a0 2024-04-16T17:08:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.241+0000 7f3a06684740 1 -- start start 2024-04-16T17:08:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a06684740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccd72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a06684740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccdfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a06684740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 0x55956cce2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a06684740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55956cc3c020 con 0x55956ccd7830 2024-04-16T17:08:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a06684740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55956cbe2460 con 0x55956ccd6f10 2024-04-16T17:08:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a06684740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55956cc37fe0 con 0x55956cce0410 2024-04-16T17:08:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccd72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccd72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35384/0 (socket says 172.21.15.18:35384) 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01a00640 1 -- 172.21.15.18:0/2404582674 learned_addr learned my addr 172.21.15.18:0/2404582674 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01000640 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccdfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f39fbe00640 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 0x55956cce2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01a00640 1 -- 172.21.15.18:0/2404582674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 msgr2=0x55956cce2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01a00640 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 0x55956cce2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01a00640 1 -- 172.21.15.18:0/2404582674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 msgr2=0x55956ccdfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01a00640 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccdfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.242+0000 7f3a01a00640 1 -- 172.21.15.18:0/2404582674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956cbea040 con 0x55956ccd6f10 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.243+0000 7f3a01a00640 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccd72f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f39fc06b470 tx=0x7f39f40098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450f59a1d14eda9e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.243+0000 7f39f9600640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39e00a3da0 0x7f39e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.243+0000 7f39f9600640 1 -- 172.21.15.18:0/1892060795 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f39ec0d0d20 con 0x7f39fc0743a0 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.243+0000 7f39f2000640 1 -- 172.21.15.18:0/2404582674 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4043030 con 0x55956ccd6f10 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.243+0000 7f39f2000640 1 -- 172.21.15.18:0/2404582674 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f402ce00 con 0x55956ccd6f10 2024-04-16T17:08:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.243+0000 7f39f2000640 1 -- 172.21.15.18:0/2404582674 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f40047d0 con 0x55956ccd6f10 2024-04-16T17:08:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.243+0000 7f3a01000640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39e00a3da0 0x7f39e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 -- 172.21.15.18:0/2404582674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 msgr2=0x55956ccd72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a01000640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39e00a3da0 0x7f39e00a6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55956ccd69e0 tx=0x7f39ec010e30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccd72f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f39fc06b470 tx=0x7f39f40098f0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 -- 172.21.15.18:0/2404582674 shutdown_connections 2024-04-16T17:08:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 0x55956cce2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccdfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 --2- 172.21.15.18:0/2404582674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccd72f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 -- 172.21.15.18:0/2404582674 >> 172.21.15.18:0/2404582674 conn(0x55956ccd1640 msgr2=0x55956ccd1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 -- 172.21.15.18:0/2404582674 shutdown_connections 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.244+0000 7f3a06684740 1 -- 172.21.15.18:0/2404582674 wait complete. 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 1 -- start start 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 1 Processor -- start 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccdd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccdd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 0x55956ccddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55956cce31a0 con 0x55956ccd7830 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55956cbf5830 con 0x55956cce0410 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55956cce3320 con 0x55956ccd6f10 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.245+0000 7f3a06684740 10 client.0 _ll_register_callbacks cb 0x55956cbfd150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccdd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccdd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44022/0 (socket says 172.21.15.18:44022) 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f39fbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccdd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 -- 172.21.15.18:0/1189469797 learned_addr learned my addr 172.21.15.18:0/1189469797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 0x55956ccddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 -- 172.21.15.18:0/1189469797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 msgr2=0x55956ccddf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 0x55956ccddf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 -- 172.21.15.18:0/1189469797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 msgr2=0x55956ccdd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccdd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 -- 172.21.15.18:0/1189469797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956cbf9920 con 0x55956ccd6f10 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f3a01a00640 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccdd4b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009560 tx=0x7f39f40045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96957f2407489269 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.246+0000 7f39eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f39eaa00640 1 -- 172.21.15.18:0/1189469797 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4043040 con 0x55956ccd6f10 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f39eaa00640 1 -- 172.21.15.18:0/1189469797 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f39f4033520 con 0x55956ccd6f10 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 1 -- 172.21.15.18:0/1189469797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39f402ce00 con 0x55956ccd6f10 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f39eaa00640 1 -- 172.21.15.18:0/1189469797 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f39f405a050 con 0x55956ccd6f10 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 10 client.26551 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 1 -- 172.21.15.18:0/1189469797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55956cbc77f0 con 0x55956ccd6f10 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f39eaa00640 1 -- 172.21.15.18:0/1189469797 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f402ce00 con 0x55956ccd6f10 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f39eaa00640 1 -- 172.21.15.18:0/1189469797 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f39f40347a0 con 0x55956ccd6f10 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 10 client.26551 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 10 client.26551 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 20 client.26551 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55956ccf0620 0x55956ccf2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 4 client.26551 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.247+0000 7f3a06684740 1 -- 172.21.15.18:0/1189469797 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55956cc49a40 con 0x55956ccf0620 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.248+0000 7f39fbe00640 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55956ccf0620 0x55956ccf2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.248+0000 7f39fbe00640 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55956ccf0620 0x55956ccf2a10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=921c36ad2c24b66c server_cookie=dfdb6741fb461530 in_seq=0 out_seq=0 2024-04-16T17:08:16.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.248+0000 7f39eaa00640 10 client.26551 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:08:16.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f39eaa00640 1 -- 172.21.15.18:0/1189469797 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55956cc49a40 con 0x55956ccf0620 2024-04-16T17:08:16.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f39eaa00640 10 client.26551 handle_command_reply: tid=0 2024-04-16T17:08:16.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 client.26551 shutdown 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 -- 172.21.15.18:0/1189469797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 msgr2=0x55956ccdd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccdd4b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009560 tx=0x7f39f40045b0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 -- 172.21.15.18:0/1189469797 shutdown_connections 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55956ccf0620 0x55956ccf2a10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 0x55956ccddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccdd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 --2- 172.21.15.18:0/1189469797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccdd4b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f3a06684740 1 -- 172.21.15.18:0/1189469797 >> 172.21.15.18:0/1189469797 conn(0x55956ccd1640 msgr2=0x55956ccd3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.249+0000 7f39eaa00640 0 client.26551 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.250+0000 7f3a06684740 1 -- 172.21.15.18:0/1189469797 shutdown_connections 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.250+0000 7f3a06684740 1 -- 172.21.15.18:0/1189469797 wait complete. 2024-04-16T17:08:16.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.250+0000 7f3a06684740 20 client.26551 trim_cache size 0 max 16384 2024-04-16T17:08:16.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.265+0000 7f3a06684740 1 Processor -- start 2024-04-16T17:08:16.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.265+0000 7f3a06684740 1 -- start start 2024-04-16T17:08:16.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a06684740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccdfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a06684740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a06684740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 0x55956ccf5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a06684740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1b81d0 con 0x55956ccd7830 2024-04-16T17:08:16.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a06684740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1b5aa0 con 0x55956ccd6f10 2024-04-16T17:08:16.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a06684740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1b38f0 con 0x55956cce0410 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a01a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccdfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f39fbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a01a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccdfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35400/0 (socket says 172.21.15.18:35400) 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a01a00640 1 -- 172.21.15.18:0/3634047257 learned_addr learned my addr 172.21.15.18:0/3634047257 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.266+0000 7f3a01a00640 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 0x55956ccf5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f3a01a00640 1 -- 172.21.15.18:0/3634047257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 msgr2=0x55956ccf5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f3a01a00640 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 0x55956ccf5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f3a01a00640 1 -- 172.21.15.18:0/3634047257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 msgr2=0x55956ccddd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f3a01a00640 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccddd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f3a01a00640 1 -- 172.21.15.18:0/3634047257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956c94e3f0 con 0x55956ccd6f10 2024-04-16T17:08:16.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f39fbe00640 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccddd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:16.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f3a01a00640 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccdfbe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f39f40049d0 tx=0x7f39f40098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d35bf150a5135c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f39f1600640 1 -- 172.21.15.18:0/3634047257 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4043040 con 0x55956ccd6f10 2024-04-16T17:08:16.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f39f1600640 1 -- 172.21.15.18:0/3634047257 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f404c070 con 0x55956ccd6f10 2024-04-16T17:08:16.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.267+0000 7f39f1600640 1 -- 172.21.15.18:0/3634047257 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4033780 con 0x55956ccd6f10 2024-04-16T17:08:16.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 -- 172.21.15.18:0/3634047257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 msgr2=0x55956ccdfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccdfbe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f39f40049d0 tx=0x7f39f40098f0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 -- 172.21.15.18:0/3634047257 shutdown_connections 2024-04-16T17:08:16.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956cce0410 0x55956ccf5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccddd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 --2- 172.21.15.18:0/3634047257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956ccd6f10 0x55956ccdfbe0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 -- 172.21.15.18:0/3634047257 >> 172.21.15.18:0/3634047257 conn(0x55956ccd1640 msgr2=0x55956ccebe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 -- 172.21.15.18:0/3634047257 shutdown_connections 2024-04-16T17:08:16.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.268+0000 7f3a06684740 1 -- 172.21.15.18:0/3634047257 wait complete. 2024-04-16T17:08:16.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.269+0000 7f3a06684740 1 -- start start 2024-04-16T17:08:16.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.269+0000 7f3a06684740 1 Processor -- start 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.269+0000 7f3a06684740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccf5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.269+0000 7f3a06684740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccf5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.269+0000 7f3a06684740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 0x55956ccf2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.270+0000 7f3a06684740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1b3f60 con 0x55956ccd7830 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.270+0000 7f3a06684740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1b53d0 con 0x55956cce0410 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.270+0000 7f3a06684740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1b4630 con 0x55956ccd6f10 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.270+0000 7f3a06684740 10 client.0 _ll_register_callbacks cb 0x7f39f4009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.270+0000 7f3a01a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccf5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.270+0000 7f3a01a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccf5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44040/0 (socket says 172.21.15.18:44040) 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.270+0000 7f3a01a00640 1 -- 172.21.15.18:0/3167625212 learned_addr learned my addr 172.21.15.18:0/3167625212 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.270+0000 7f3a01a00640 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 0x55956ccf2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f3a01a00640 1 -- 172.21.15.18:0/3167625212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 msgr2=0x55956ccf2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f3a01a00640 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 0x55956ccf2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f3a01a00640 1 -- 172.21.15.18:0/3167625212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 msgr2=0x55956ccf5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f3a01a00640 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccf5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f3a01a00640 1 -- 172.21.15.18:0/3167625212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956ca187d0 con 0x55956ccd6f10 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f3a01a00640 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccf5a00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f39f4002230 tx=0x7f39f4004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c990e8a96d2d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f39eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:16.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f39eaa00640 1 -- 172.21.15.18:0/3167625212 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4043070 con 0x55956ccd6f10 2024-04-16T17:08:16.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f3a06684740 1 -- 172.21.15.18:0/3167625212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39f404c070 con 0x55956ccd6f10 2024-04-16T17:08:16.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.271+0000 7f39eaa00640 1 -- 172.21.15.18:0/3167625212 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f39f4034870 con 0x55956ccd6f10 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.272+0000 7f39eaa00640 1 -- 172.21.15.18:0/3167625212 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f39f4062070 con 0x55956ccd6f10 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.272+0000 7f3a06684740 10 client.26557 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.272+0000 7f3a06684740 1 -- 172.21.15.18:0/3167625212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55956cd56240 con 0x55956ccd6f10 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f39eaa00640 1 -- 172.21.15.18:0/3167625212 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f404c070 con 0x55956ccd6f10 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f39eaa00640 1 -- 172.21.15.18:0/3167625212 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f39f4033520 con 0x55956ccd6f10 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f3a06684740 10 client.26557 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f3a06684740 10 client.26557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f3a06684740 20 client.26557 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f3a06684740 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55956cd56710 0x55956cd58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f3a06684740 4 client.26557 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f3a06684740 1 -- 172.21.15.18:0/3167625212 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55956cc438b0 con 0x55956cd56710 2024-04-16T17:08:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.273+0000 7f39fbe00640 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55956cd56710 0x55956cd58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.274+0000 7f39fbe00640 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55956cd56710 0x55956cd58b00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22be05e8c7f455a4 server_cookie=4745069aec568982 in_seq=0 out_seq=0 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.274+0000 7f39eaa00640 10 client.26557 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.274+0000 7f39eaa00640 1 -- 172.21.15.18:0/3167625212 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55956cc438b0 con 0x55956cd56710 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.274+0000 7f39eaa00640 10 client.26557 handle_command_reply: tid=0 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.274+0000 7f3a06684740 1 client.26557 shutdown 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 -- 172.21.15.18:0/3167625212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 msgr2=0x55956ccf5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccf5a00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f39f4002230 tx=0x7f39f4004760 comp rx=0 tx=0).stop 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 -- 172.21.15.18:0/3167625212 shutdown_connections 2024-04-16T17:08:16.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55956cd56710 0x55956cd58b00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956cce0410 0x55956ccf2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55956ccd7830 0x55956ccf5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 --2- 172.21.15.18:0/3167625212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55956ccd6f10 0x55956ccf5a00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 -- 172.21.15.18:0/3167625212 >> 172.21.15.18:0/3167625212 conn(0x55956ccd1640 msgr2=0x55956cccfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f39eaa00640 0 client.26557 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 -- 172.21.15.18:0/3167625212 shutdown_connections 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 1 -- 172.21.15.18:0/3167625212 wait complete. 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.275+0000 7f3a06684740 20 client.26557 trim_cache size 0 max 16384 2024-04-16T17:08:16.278 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:08:16.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 -- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39e00a3da0 msgr2=0x7f39e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39e00a3da0 0x7f39e00a6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55956ccd69e0 tx=0x7f39ec010e30 comp rx=0 tx=0).stop 2024-04-16T17:08:16.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 -- 172.21.15.18:0/1892060795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39fc0743a0 msgr2=0x7f39fc1a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39fc0743a0 0x7f39fc1a4a60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00e790 tx=0x7f39ec00ec50 comp rx=0 tx=0).stop 2024-04-16T17:08:16.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 -- 172.21.15.18:0/1892060795 shutdown_connections 2024-04-16T17:08:16.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39e00a3da0 0x7f39e00a6260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39fc112e10 0x7f39fc19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39fc074cc0 0x7f39fc1a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 --2- 172.21.15.18:0/1892060795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39fc0743a0 0x7f39fc1a4a60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 -- 172.21.15.18:0/1892060795 >> 172.21.15.18:0/1892060795 conn(0x7f39fc0fd950 msgr2=0x7f39fc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 -- 172.21.15.18:0/1892060795 shutdown_connections 2024-04-16T17:08:16.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.276+0000 7f3a02400640 1 -- 172.21.15.18:0/1892060795 wait complete. 2024-04-16T17:08:16.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:08:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 -- 172.21.15.18:0/2972844930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 msgr2=0x7f1ac8104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 --2- 172.21.15.18:0/2972844930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 0x7f1ac8104820 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009900 tx=0x7f1abc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 -- 172.21.15.18:0/2972844930 shutdown_connections 2024-04-16T17:08:16.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 --2- 172.21.15.18:0/2972844930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac8109510 0x7f1ac81110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 --2- 172.21.15.18:0/2972844930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ac8104df0 0x7f1ac8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 --2- 172.21.15.18:0/2972844930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 0x7f1ac8104820 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 -- 172.21.15.18:0/2972844930 >> 172.21.15.18:0/2972844930 conn(0x7f1ac80ffc80 msgr2=0x7f1ac81020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 -- 172.21.15.18:0/2972844930 shutdown_connections 2024-04-16T17:08:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.529+0000 7f1ad0800640 1 -- 172.21.15.18:0/2972844930 wait complete. 2024-04-16T17:08:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.530+0000 7f1ad0800640 1 Processor -- start 2024-04-16T17:08:16.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.530+0000 7f1ad0800640 1 -- start start 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.530+0000 7f1ad0800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 0x7f1ac81a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.530+0000 7f1ad0800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac8109510 0x7f1ac81a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1ad0800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ac819aa20 0x7f1ac819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1ad0800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8114620 con 0x7f1ac819aa20 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1ad0800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81144a0 con 0x7f1ac8104440 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1ad0800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81147a0 con 0x7f1ac8109510 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 0x7f1ac81a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 0x7f1ac81a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35436/0 (socket says 172.21.15.18:35436) 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac8109510 0x7f1ac81a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acde00640 1 -- 172.21.15.18:0/3462748189 learned_addr learned my addr 172.21.15.18:0/3462748189 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1ace800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ac819aa20 0x7f1ac819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acde00640 1 -- 172.21.15.18:0/3462748189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac8109510 msgr2=0x7f1ac81a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acde00640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac8109510 0x7f1ac81a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acde00640 1 -- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ac819aa20 msgr2=0x7f1ac819aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acde00640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ac819aa20 0x7f1ac819aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.531+0000 7f1acde00640 1 -- 172.21.15.18:0/3462748189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac819b540 con 0x7f1ac8104440 2024-04-16T17:08:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.532+0000 7f1acde00640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 0x7f1ac81a0870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1abc02c240 tx=0x7f1abc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.532+0000 7f1ac2a00640 1 -- 172.21.15.18:0/3462748189 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc043020 con 0x7f1ac8104440 2024-04-16T17:08:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.532+0000 7f1ac2a00640 1 -- 172.21.15.18:0/3462748189 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1abc0054e0 con 0x7f1ac8104440 2024-04-16T17:08:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.532+0000 7f1ac2a00640 1 -- 172.21.15.18:0/3462748189 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc003b30 con 0x7f1ac8104440 2024-04-16T17:08:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.532+0000 7f1ad0800640 1 -- 172.21.15.18:0/3462748189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac819b7d0 con 0x7f1ac8104440 2024-04-16T17:08:16.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.532+0000 7f1ad0800640 1 -- 172.21.15.18:0/3462748189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a7a20 con 0x7f1ac8104440 2024-04-16T17:08:16.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.536+0000 7f1ac2a00640 1 -- 172.21.15.18:0/3462748189 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1abc046020 con 0x7f1ac8104440 2024-04-16T17:08:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.537+0000 7f1ad3610740 1 Processor -- start 2024-04-16T17:08:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.537+0000 7f1ad3610740 1 -- start start 2024-04-16T17:08:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a382130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a382670 0x55cb0a38ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a38b250 0x55cb0a38d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ad3610740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cb0a2e7020 con 0x55cb0a382670 2024-04-16T17:08:16.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ad3610740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cb0a28d460 con 0x55cb0a38b250 2024-04-16T17:08:16.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ad3610740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb0a2e2fe0 con 0x55cb0a381d50 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ace800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a382130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ace800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a382130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44068/0 (socket says 172.21.15.18:44068) 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1ace800640 1 -- 172.21.15.18:0/2154949341 learned_addr learned my addr 172.21.15.18:0/2154949341 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1acde00640 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a382670 0x55cb0a38ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.538+0000 7f1acd400640 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a38b250 0x55cb0a38d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1ac2a00640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1acde00640 1 -- 172.21.15.18:0/2154949341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 msgr2=0x55cb0a382130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1acde00640 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a382130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1acde00640 1 -- 172.21.15.18:0/2154949341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a38b250 msgr2=0x55cb0a38d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1acde00640 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a38b250 0x55cb0a38d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1acde00640 1 -- 172.21.15.18:0/2154949341 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0a295040 con 0x55cb0a382670 2024-04-16T17:08:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1acd400640 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a38b250 0x55cb0a38d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1ac2a00640 1 -- 172.21.15.18:0/3462748189 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1abc0f2ad0 con 0x7f1ac8104440 2024-04-16T17:08:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1acde00640 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a382670 0x55cb0a38ad10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0eff20 tx=0x7f1abc04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bf7831e5dbafee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.539+0000 7f1ace800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.540+0000 7f1ab7000640 1 -- 172.21.15.18:0/2154949341 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0be3d0 con 0x55cb0a382670 2024-04-16T17:08:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.540+0000 7f1ab7000640 1 -- 172.21.15.18:0/2154949341 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1abc005690 con 0x55cb0a382670 2024-04-16T17:08:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.540+0000 7f1ab7000640 1 -- 172.21.15.18:0/2154949341 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0be1f0 con 0x55cb0a382670 2024-04-16T17:08:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 -- 172.21.15.18:0/2154949341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a382670 msgr2=0x55cb0a38ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ace800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4013620 tx=0x7f1ac4004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a382670 0x55cb0a38ad10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0eff20 tx=0x7f1abc04f000 comp rx=0 tx=0).stop 2024-04-16T17:08:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 -- 172.21.15.18:0/2154949341 shutdown_connections 2024-04-16T17:08:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a38b250 0x55cb0a38d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a382670 0x55cb0a38ad10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2154949341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a382130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 -- 172.21.15.18:0/2154949341 >> 172.21.15.18:0/2154949341 conn(0x55cb0a37c480 msgr2=0x55cb0a37c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 -- 172.21.15.18:0/2154949341 shutdown_connections 2024-04-16T17:08:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.541+0000 7f1ad3610740 1 -- 172.21.15.18:0/2154949341 wait complete. 2024-04-16T17:08:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.542+0000 7f1ad3610740 1 -- start start 2024-04-16T17:08:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.542+0000 7f1ad3610740 1 Processor -- start 2024-04-16T17:08:16.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 114344 3.97 MB/sec execute 29 sec latency 28.521 ms 2024-04-16T17:08:16.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a388290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3887d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a388d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1ad3610740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cb0a2e7020 con 0x55cb0a38b250 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1ad3610740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cb0a2a0830 con 0x55cb0a381d50 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1ad3610740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb0a38de00 con 0x55cb0a382670 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1ad3610740 10 client.0 _ll_register_callbacks cb 0x55cb0a2a8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1acde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a388290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1acd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3887d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1acd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3887d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44082/0 (socket says 172.21.15.18:44082) 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1acd400640 1 -- 172.21.15.18:0/3455449115 learned_addr learned my addr 172.21.15.18:0/3455449115 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.543+0000 7f1acde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a388290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35454/0 (socket says 172.21.15.18:35454) 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1acd400640 1 -- 172.21.15.18:0/3455449115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 msgr2=0x55cb0a388290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1acd400640 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a388290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1ace800640 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a388d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1acd400640 1 -- 172.21.15.18:0/3455449115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 msgr2=0x55cb0a388d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1acd400640 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a388d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1acd400640 1 -- 172.21.15.18:0/3455449115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0a2a4920 con 0x55cb0a382670 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1ace800640 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a388d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1acd400640 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3887d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8012730 tx=0x7f1ab8012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dd89ed5f9e7d950 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1aab400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1aab400640 1 -- 172.21.15.18:0/3455449115 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab800f9a0 con 0x55cb0a382670 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1aab400640 1 -- 172.21.15.18:0/3455449115 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1ab800fb40 con 0x55cb0a382670 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.544+0000 7f1ad3610740 1 -- 172.21.15.18:0/3455449115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cb0a2f4a40 con 0x55cb0a382670 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.545+0000 7f1aab400640 1 -- 172.21.15.18:0/3455449115 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ab8025050 con 0x55cb0a382670 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.545+0000 7f1ad3610740 10 client.26569 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.545+0000 7f1ad3610740 1 -- 172.21.15.18:0/3455449115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb0a2727f0 con 0x55cb0a382670 2024-04-16T17:08:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.545+0000 7f1aab400640 1 -- 172.21.15.18:0/3455449115 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cb0a2f4a40 con 0x55cb0a382670 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.545+0000 7f1aab400640 1 -- 172.21.15.18:0/3455449115 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ab8010430 con 0x55cb0a382670 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.546+0000 7f1ad3610740 10 client.26569 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.546+0000 7f1ad3610740 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.546+0000 7f1ad3610740 20 client.26569 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.546+0000 7f1ad3610740 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cb0a39b360 0x55cb0a39d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.546+0000 7f1ad3610740 4 client.26569 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.546+0000 7f1ad3610740 1 -- 172.21.15.18:0/3455449115 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb0a2ee8b0 con 0x55cb0a39b360 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.546+0000 7f1acde00640 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cb0a39b360 0x55cb0a39d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.547+0000 7f1acde00640 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cb0a39b360 0x55cb0a39d780 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854881502123a562 server_cookie=d0ddc232da7da27a in_seq=0 out_seq=0 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.547+0000 7f1aab400640 10 client.26569 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.547+0000 7f1aab400640 1 -- 172.21.15.18:0/3455449115 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb0a2ee8b0 con 0x55cb0a39b360 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.547+0000 7f1aab400640 10 client.26569 handle_command_reply: tid=0 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 client.26569 shutdown 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 -- 172.21.15.18:0/3455449115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 msgr2=0x55cb0a3887d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3887d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8012730 tx=0x7f1ab8012bf0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 -- 172.21.15.18:0/3455449115 shutdown_connections 2024-04-16T17:08:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cb0a39b360 0x55cb0a39d780 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a388d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3887d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 --2- 172.21.15.18:0/3455449115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a388290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 -- 172.21.15.18:0/3455449115 >> 172.21.15.18:0/3455449115 conn(0x55cb0a37c480 msgr2=0x55cb0a37e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1aab400640 0 client.26569 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 -- 172.21.15.18:0/3455449115 shutdown_connections 2024-04-16T17:08:16.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 1 -- 172.21.15.18:0/3455449115 wait complete. 2024-04-16T17:08:16.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.548+0000 7f1ad3610740 20 client.26569 trim_cache size 0 max 16384 2024-04-16T17:08:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.565+0000 7f1ad3610740 1 Processor -- start 2024-04-16T17:08:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.565+0000 7f1ad3610740 1 -- start start 2024-04-16T17:08:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.565+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a396b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.565+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a382670 0x55cb0a388b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a3a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1ad3610740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81b4120 con 0x55cb0a38b250 2024-04-16T17:08:16.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1ad3610740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81b19f0 con 0x55cb0a382670 2024-04-16T17:08:16.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1ad3610740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81af840 con 0x55cb0a381d50 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a396b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a396b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44098/0 (socket says 172.21.15.18:44098) 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acde00640 1 -- 172.21.15.18:0/4049737712 learned_addr learned my addr 172.21.15.18:0/4049737712 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acd400640 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a382670 0x55cb0a388b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1ace800640 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a3a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acd400640 1 -- 172.21.15.18:0/4049737712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 msgr2=0x55cb0a396b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acd400640 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a396b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acd400640 1 -- 172.21.15.18:0/4049737712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 msgr2=0x55cb0a3a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acd400640 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a3a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.566+0000 7f1acd400640 1 -- 172.21.15.18:0/4049737712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb09ff93f0 con 0x55cb0a382670 2024-04-16T17:08:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.567+0000 7f1acd400640 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a382670 0x55cb0a388b30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55cb0a389860 tx=0x7f1ab8005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2a279d6a75beae server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.567+0000 7f1ab6600640 1 -- 172.21.15.18:0/4049737712 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8010430 con 0x55cb0a382670 2024-04-16T17:08:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.567+0000 7f1ab6600640 1 -- 172.21.15.18:0/4049737712 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab80105d0 con 0x55cb0a382670 2024-04-16T17:08:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.567+0000 7f1ab6600640 1 -- 172.21.15.18:0/4049737712 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8010bf0 con 0x55cb0a382670 2024-04-16T17:08:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 -- 172.21.15.18:0/4049737712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a382670 msgr2=0x55cb0a388b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a382670 0x55cb0a388b30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55cb0a389860 tx=0x7f1ab8005750 comp rx=0 tx=0).stop 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 -- 172.21.15.18:0/4049737712 shutdown_connections 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a3a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a382670 0x55cb0a388b30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 --2- 172.21.15.18:0/4049737712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a381d50 0x55cb0a396b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 -- 172.21.15.18:0/4049737712 >> 172.21.15.18:0/4049737712 conn(0x55cb0a37c480 msgr2=0x55cb0a37ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 -- 172.21.15.18:0/4049737712 shutdown_connections 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.568+0000 7f1ad3610740 1 -- 172.21.15.18:0/4049737712 wait complete. 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.569+0000 7f1ad3610740 1 -- start start 2024-04-16T17:08:16.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.569+0000 7f1ad3610740 1 Processor -- start 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a393080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1ad3610740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a384a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1ad3610740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81afeb0 con 0x55cb0a38b250 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1ad3610740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81b1320 con 0x55cb0a381d50 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1ad3610740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81b0580 con 0x55cb0a382670 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1ad3610740 10 client.0 _ll_register_callbacks cb 0x7f1ab8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a393080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a393080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35476/0 (socket says 172.21.15.18:35476) 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acde00640 1 -- 172.21.15.18:0/2989081254 learned_addr learned my addr 172.21.15.18:0/2989081254 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acd400640 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1ace800640 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a384a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acde00640 1 -- 172.21.15.18:0/2989081254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 msgr2=0x55cb0a3935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acde00640 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acde00640 1 -- 172.21.15.18:0/2989081254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 msgr2=0x55cb0a384a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acde00640 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a384a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.570+0000 7f1acde00640 1 -- 172.21.15.18:0/2989081254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0a0c37d0 con 0x55cb0a381d50 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.571+0000 7f1acde00640 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a393080 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1abc04fff0 tx=0x7f1abc02ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba0a66b719beb39 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.571+0000 7f1aab400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:16.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.571+0000 7f1aab400640 1 -- 172.21.15.18:0/2989081254 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0be660 con 0x55cb0a381d50 2024-04-16T17:08:16.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.571+0000 7f1aab400640 1 -- 172.21.15.18:0/2989081254 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1abc0ba720 con 0x55cb0a381d50 2024-04-16T17:08:16.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.571+0000 7f1ad3610740 1 -- 172.21.15.18:0/2989081254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab80105d0 con 0x55cb0a381d50 2024-04-16T17:08:16.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.571+0000 7f1aab400640 1 -- 172.21.15.18:0/2989081254 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1abc088050 con 0x55cb0a381d50 2024-04-16T17:08:16.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.571+0000 7f1ad3610740 10 client.26600 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:16.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.571+0000 7f1ad3610740 1 -- 172.21.15.18:0/2989081254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb0a4012c0 con 0x55cb0a381d50 2024-04-16T17:08:16.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.572+0000 7f1aab400640 1 -- 172.21.15.18:0/2989081254 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab80105d0 con 0x55cb0a381d50 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.572+0000 7f1aab400640 1 -- 172.21.15.18:0/2989081254 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1abc003cd0 con 0x55cb0a381d50 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.572+0000 7f1ad3610740 10 client.26600 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.572+0000 7f1ad3610740 10 client.26600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.572+0000 7f1ad3610740 20 client.26600 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.572+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cb0a401710 0x55cb0a403b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.572+0000 7f1ad3610740 4 client.26600 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.572+0000 7f1ad3610740 1 -- 172.21.15.18:0/2989081254 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb0a26a570 con 0x55cb0a401710 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.573+0000 7f1acd400640 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cb0a401710 0x55cb0a403b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.573+0000 7f1acd400640 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cb0a401710 0x55cb0a403b00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847e9e5c17a52e0b server_cookie=7e514a371c7ba45c in_seq=0 out_seq=0 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.573+0000 7f1aab400640 10 client.26600 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.573+0000 7f1aab400640 1 -- 172.21.15.18:0/2989081254 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb0a26a570 con 0x55cb0a401710 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.573+0000 7f1aab400640 10 client.26600 handle_command_reply: tid=0 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.573+0000 7f1ad3610740 1 client.26600 shutdown 2024-04-16T17:08:16.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 -- 172.21.15.18:0/2989081254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 msgr2=0x55cb0a393080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a393080 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1abc04fff0 tx=0x7f1abc02ca90 comp rx=0 tx=0).stop 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 -- 172.21.15.18:0/2989081254 shutdown_connections 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cb0a401710 0x55cb0a403b00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cb0a38b250 0x55cb0a384a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1aab400640 0 client.26600 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0a382670 0x55cb0a3935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 --2- 172.21.15.18:0/2989081254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb0a381d50 0x55cb0a393080 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 -- 172.21.15.18:0/2989081254 >> 172.21.15.18:0/2989081254 conn(0x55cb0a37c480 msgr2=0x55cb0a37b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 -- 172.21.15.18:0/2989081254 shutdown_connections 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 1 -- 172.21.15.18:0/2989081254 wait complete. 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.574+0000 7f1ad3610740 20 client.26600 trim_cache size 0 max 16384 2024-04-16T17:08:16.577 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:08:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 -- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1ab00a3d20 msgr2=0x7f1ab00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4013620 tx=0x7f1ac4004a40 comp rx=0 tx=0).stop 2024-04-16T17:08:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 -- 172.21.15.18:0/3462748189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 msgr2=0x7f1ac81a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 0x7f1ac81a0870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1abc02c240 tx=0x7f1abc0042f0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 -- 172.21.15.18:0/3462748189 shutdown_connections 2024-04-16T17:08:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ac819aa20 0x7f1ac819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac8109510 0x7f1ac81a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 --2- 172.21.15.18:0/3462748189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8104440 0x7f1ac81a0870 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.575+0000 7f1ad0800640 1 -- 172.21.15.18:0/3462748189 >> 172.21.15.18:0/3462748189 conn(0x7f1ac80ffc80 msgr2=0x7f1ac8101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.576+0000 7f1ad0800640 1 -- 172.21.15.18:0/3462748189 shutdown_connections 2024-04-16T17:08:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:16.576+0000 7f1ad0800640 1 -- 172.21.15.18:0/3462748189 wait complete. 2024-04-16T17:08:17.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:16 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2179302067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 -- 172.21.15.18:0/3119026827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bfc102a30 msgr2=0x7f4bfc106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 --2- 172.21.15.18:0/3119026827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bfc102a30 0x7f4bfc106a70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009900 tx=0x7f4be802bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 -- 172.21.15.18:0/3119026827 shutdown_connections 2024-04-16T17:08:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 --2- 172.21.15.18:0/3119026827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc107150 0x7f4bfc10ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 --2- 172.21.15.18:0/3119026827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bfc102a30 0x7f4bfc106a70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 --2- 172.21.15.18:0/3119026827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bfc102080 0x7f4bfc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 -- 172.21.15.18:0/3119026827 >> 172.21.15.18:0/3119026827 conn(0x7f4bfc0fd940 msgr2=0x7f4bfc0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 -- 172.21.15.18:0/3119026827 shutdown_connections 2024-04-16T17:08:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.011+0000 7f4bfb200640 1 -- 172.21.15.18:0/3119026827 wait complete. 2024-04-16T17:08:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.012+0000 7f4bfb200640 1 Processor -- start 2024-04-16T17:08:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bfb200640 1 -- start start 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bfb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc102080 0x7f4bfc074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bfb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bfc107150 0x7f4bfc074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bfb200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bfc075fa0 0x7f4bfc074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bfb200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc112230 con 0x7f4bfc075fa0 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bfb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc1120b0 con 0x7f4bfc107150 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bfb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc1123b0 con 0x7f4bfc102080 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bf9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc102080 0x7f4bfc074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.013+0000 7f4bf9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bfc107150 0x7f4bfc074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bf9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc102080 0x7f4bfc074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34264/0 (socket says 172.21.15.18:34264) 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bf9e00640 1 -- 172.21.15.18:0/656836208 learned_addr learned my addr 172.21.15.18:0/656836208 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bfa800640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bfc075fa0 0x7f4bfc074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bf9e00640 1 -- 172.21.15.18:0/656836208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bfc107150 msgr2=0x7f4bfc074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bfc107150 0x7f4bfc074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bf9e00640 1 -- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bfc075fa0 msgr2=0x7f4bfc074d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bfc075fa0 0x7f4bfc074d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bf9e00640 1 -- 172.21.15.18:0/656836208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bfc1ab440 con 0x7f4bfc102080 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bfa800640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bfc075fa0 0x7f4bfc074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.014+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc102080 0x7f4bfc074300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00e9f0 tx=0x7f4bec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.015+0000 7f4bf2a00640 1 -- 172.21.15.18:0/656836208 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00cd80 con 0x7f4bfc102080 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.015+0000 7f4bf2a00640 1 -- 172.21.15.18:0/656836208 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bec010910 con 0x7f4bfc102080 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.015+0000 7f4bf2a00640 1 -- 172.21.15.18:0/656836208 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec018990 con 0x7f4bfc102080 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.015+0000 7f4bfb200640 1 -- 172.21.15.18:0/656836208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bfc1ab730 con 0x7f4bfc102080 2024-04-16T17:08:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.015+0000 7f4bfb200640 1 -- 172.21.15.18:0/656836208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bfc1abc50 con 0x7f4bfc102080 2024-04-16T17:08:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.019+0000 7f4bf2a00640 1 -- 172.21.15.18:0/656836208 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4bec010ac0 con 0x7f4bfc102080 2024-04-16T17:08:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.019+0000 7f4c03bdd740 1 Processor -- start 2024-04-16T17:08:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.020+0000 7f4c03bdd740 1 -- start start 2024-04-16T17:08:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.020+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6f7f0 0x558590d6e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.020+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d6ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.020+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d7a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.020+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558590cd3780 con 0x558590d6f7f0 2024-04-16T17:08:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.020+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558590c79f40 con 0x558590d78000 2024-04-16T17:08:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.020+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558590ccf740 con 0x558590d6ea80 2024-04-16T17:08:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.020+0000 7f4bfa800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6f7f0 0x558590d6e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bfa800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6f7f0 0x558590d6e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56896/0 (socket says 172.21.15.18:56896) 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bfa800640 1 -- 172.21.15.18:0/2759208942 learned_addr learned my addr 172.21.15.18:0/2759208942 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf9400640 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d7a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf2a00640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4be00a3d00 0x7f4be00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d6ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf9400640 1 -- 172.21.15.18:0/2759208942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 msgr2=0x558590d6ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf9400640 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d6ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf9400640 1 -- 172.21.15.18:0/2759208942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6f7f0 msgr2=0x558590d6e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf2a00640 1 -- 172.21.15.18:0/656836208 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f4bec0d1190 con 0x7f4bfc102080 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf9400640 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6f7f0 0x558590d6e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf9400640 1 -- 172.21.15.18:0/2759208942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585909e6210 con 0x558590d78000 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d6ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.021+0000 7f4bfa800640 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6f7f0 0x558590d6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.022+0000 7f4bf9400640 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d7a3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4be8036040 tx=0x7f4be8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53eab5a55ccd8f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.022+0000 7f4bfa800640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4be00a3d00 0x7f4be00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.022+0000 7f4be7400640 1 -- 172.21.15.18:0/2759208942 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be8045070 con 0x558590d78000 2024-04-16T17:08:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.022+0000 7f4be7400640 1 -- 172.21.15.18:0/2759208942 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be802c650 con 0x558590d78000 2024-04-16T17:08:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.022+0000 7f4be7400640 1 -- 172.21.15.18:0/2759208942 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be8004b40 con 0x558590d78000 2024-04-16T17:08:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.023+0000 7f4c03bdd740 1 -- 172.21.15.18:0/2759208942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 msgr2=0x558590d7a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.023+0000 7f4bfa800640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4be00a3d00 0x7f4be00a61c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40094d0 tx=0x7f4bf4004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.023+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d7a3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4be8036040 tx=0x7f4be8009560 comp rx=0 tx=0).stop 2024-04-16T17:08:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.023+0000 7f4c03bdd740 1 -- 172.21.15.18:0/2759208942 shutdown_connections 2024-04-16T17:08:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.023+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d7a3f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.023+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d6ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.023+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/2759208942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6f7f0 0x558590d6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.023+0000 7f4c03bdd740 1 -- 172.21.15.18:0/2759208942 >> 172.21.15.18:0/2759208942 conn(0x558590d095d0 msgr2=0x558590d69770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.024+0000 7f4c03bdd740 1 -- 172.21.15.18:0/2759208942 shutdown_connections 2024-04-16T17:08:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.024+0000 7f4c03bdd740 1 -- 172.21.15.18:0/2759208942 wait complete. 2024-04-16T17:08:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 1 -- start start 2024-04-16T17:08:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 1 Processor -- start 2024-04-16T17:08:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d74e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d6f7f0 0x558590d753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d78000 0x558590d75900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558590d7acd0 con 0x558590d78000 2024-04-16T17:08:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558590c8d390 con 0x558590d6f7f0 2024-04-16T17:08:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558590d7ae50 con 0x558590d6ea80 2024-04-16T17:08:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.025+0000 7f4c03bdd740 10 client.0 _ll_register_callbacks cb 0x558590c76f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d74e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d74e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34284/0 (socket says 172.21.15.18:34284) 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9e00640 1 -- 172.21.15.18:0/144468804 learned_addr learned my addr 172.21.15.18:0/144468804 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bfa800640 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d78000 0x558590d75900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9400640 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d6f7f0 0x558590d753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9e00640 1 -- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d6f7f0 msgr2=0x558590d753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d6f7f0 0x558590d753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9e00640 1 -- 172.21.15.18:0/144468804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d78000 msgr2=0x558590d75900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d78000 0x558590d75900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9e00640 1 -- 172.21.15.18:0/144468804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558590ab2620 con 0x558590d6ea80 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.026+0000 7f4bf9400640 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d6f7f0 0x558590d753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.027+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d74e80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c6a0 tx=0x7f4bec09d4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398bc09ba5f63f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.027+0000 7f4bdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.027+0000 7f4bdbe00640 1 -- 172.21.15.18:0/144468804 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0ce960 con 0x558590d6ea80 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.027+0000 7f4bdbe00640 1 -- 172.21.15.18:0/144468804 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f4bec0a5700 con 0x558590d6ea80 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.027+0000 7f4c03bdd740 1 -- 172.21.15.18:0/144468804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be802c650 con 0x558590d6ea80 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.027+0000 7f4bdbe00640 1 -- 172.21.15.18:0/144468804 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4bec0a5990 con 0x558590d6ea80 2024-04-16T17:08:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.027+0000 7f4c03bdd740 10 client.26587 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.027+0000 7f4c03bdd740 1 -- 172.21.15.18:0/144468804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558590d86e50 con 0x558590d6ea80 2024-04-16T17:08:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4bdbe00640 1 -- 172.21.15.18:0/144468804 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be802c650 con 0x558590d6ea80 2024-04-16T17:08:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4bdbe00640 1 -- 172.21.15.18:0/144468804 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4bec0b4070 con 0x558590d6ea80 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4c03bdd740 10 client.26587 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4c03bdd740 10 client.26587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4c03bdd740 20 client.26587 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558590d88360 0x558590d8a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4c03bdd740 4 client.26587 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4c03bdd740 1 -- 172.21.15.18:0/144468804 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558590ce11a0 con 0x558590d88360 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.028+0000 7f4bf9400640 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558590d88360 0x558590d8a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.029+0000 7f4bdbe00640 10 client.26587 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.029+0000 7f4bf9400640 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558590d88360 0x558590d8a750 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e257819234a86eb server_cookie=5faf1d9ec4c855b5 in_seq=0 out_seq=0 2024-04-16T17:08:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.030+0000 7f4bdbe00640 1 -- 172.21.15.18:0/144468804 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558590ce11a0 con 0x558590d88360 2024-04-16T17:08:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.030+0000 7f4bdbe00640 10 client.26587 handle_command_reply: tid=0 2024-04-16T17:08:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.030+0000 7f4c03bdd740 1 client.26587 shutdown 2024-04-16T17:08:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.030+0000 7f4c03bdd740 1 -- 172.21.15.18:0/144468804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 msgr2=0x558590d74e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.030+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d74e80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c6a0 tx=0x7f4bec09d4e0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 1 -- 172.21.15.18:0/144468804 shutdown_connections 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558590d88360 0x558590d8a750 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d78000 0x558590d75900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d6f7f0 0x558590d753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/144468804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6ea80 0x558590d74e80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 1 -- 172.21.15.18:0/144468804 >> 172.21.15.18:0/144468804 conn(0x558590d095d0 msgr2=0x558590d6b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4bdbe00640 0 client.26587 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 1 -- 172.21.15.18:0/144468804 shutdown_connections 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 1 -- 172.21.15.18:0/144468804 wait complete. 2024-04-16T17:08:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.031+0000 7f4c03bdd740 20 client.26587 trim_cache size 0 max 16384 2024-04-16T17:08:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.049+0000 7f4c03bdd740 1 Processor -- start 2024-04-16T17:08:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4c03bdd740 1 -- start start 2024-04-16T17:08:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d90060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc1b8220 con 0x558590d6ea80 2024-04-16T17:08:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc1b5af0 con 0x558590d78000 2024-04-16T17:08:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc1b3920 con 0x558590d6f7f0 2024-04-16T17:08:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4bf9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4bf9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d90060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4bf9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d775b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56908/0 (socket says 172.21.15.18:56908) 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4bf9e00640 1 -- 172.21.15.18:0/3564649285 learned_addr learned my addr 172.21.15.18:0/3564649285 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.050+0000 7f4bfa800640 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.051+0000 7f4bf9e00640 1 -- 172.21.15.18:0/3564649285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 msgr2=0x558590d90060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.051+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d90060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.051+0000 7f4bf9e00640 1 -- 172.21.15.18:0/3564649285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 msgr2=0x558590d905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.051+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d905a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.051+0000 7f4bf9e00640 1 -- 172.21.15.18:0/3564649285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558590d8aac0 con 0x558590d6ea80 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.051+0000 7f4bf9400640 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d90060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.051+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d775b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4bec09ca20 tx=0x7f4bec0a1250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737bd3afbf1fff94 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4be6a00640 1 -- 172.21.15.18:0/3564649285 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0ce960 con 0x558590d6ea80 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4be6a00640 1 -- 172.21.15.18:0/3564649285 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bec0bd0e0 con 0x558590d6ea80 2024-04-16T17:08:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4be6a00640 1 -- 172.21.15.18:0/3564649285 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec09c860 con 0x558590d6ea80 2024-04-16T17:08:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4c03bdd740 1 -- 172.21.15.18:0/3564649285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 msgr2=0x558590d775b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d775b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4bec09ca20 tx=0x7f4bec0a1250 comp rx=0 tx=0).stop 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4c03bdd740 1 -- 172.21.15.18:0/3564649285 shutdown_connections 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590d905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d90060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/3564649285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d775b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.052+0000 7f4c03bdd740 1 -- 172.21.15.18:0/3564649285 >> 172.21.15.18:0/3564649285 conn(0x558590d095d0 msgr2=0x558590d6a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.053+0000 7f4c03bdd740 1 -- 172.21.15.18:0/3564649285 shutdown_connections 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.053+0000 7f4c03bdd740 1 -- 172.21.15.18:0/3564649285 wait complete. 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.053+0000 7f4c03bdd740 1 -- start start 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.053+0000 7f4c03bdd740 1 Processor -- start 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d71f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d72460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4c03bdd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590ded8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc1b3fb0 con 0x558590d6ea80 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc1b5420 con 0x558590d78000 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4c03bdd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc1b4680 con 0x558590d6f7f0 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4c03bdd740 10 client.0 _ll_register_callbacks cb 0x7f4bec099090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bf9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d71f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bfa800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590ded8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bf9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d71f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56924/0 (socket says 172.21.15.18:56924) 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bf9e00640 1 -- 172.21.15.18:0/987578881 learned_addr learned my addr 172.21.15.18:0/987578881 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bf9400640 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d72460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bfa800640 1 -- 172.21.15.18:0/987578881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 msgr2=0x558590d72460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bfa800640 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d72460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bfa800640 1 -- 172.21.15.18:0/987578881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 msgr2=0x558590d71f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bfa800640 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d71f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bfa800640 1 -- 172.21.15.18:0/987578881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558590dede10 con 0x558590d78000 2024-04-16T17:08:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.054+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d71f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.055+0000 7f4bfa800640 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590ded8d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x558590d90ae0 tx=0x7f4bf4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd4f7bff0589561 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.055+0000 7f4bdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.055+0000 7f4bdbe00640 1 -- 172.21.15.18:0/987578881 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf402c060 con 0x558590d78000 2024-04-16T17:08:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.055+0000 7f4c03bdd740 1 -- 172.21.15.18:0/987578881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bec0bd0e0 con 0x558590d78000 2024-04-16T17:08:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.055+0000 7f4bdbe00640 1 -- 172.21.15.18:0/987578881 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f4bf4011800 con 0x558590d78000 2024-04-16T17:08:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4bdbe00640 1 -- 172.21.15.18:0/987578881 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4bf4033050 con 0x558590d78000 2024-04-16T17:08:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4c03bdd740 10 client.26618 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4c03bdd740 1 -- 172.21.15.18:0/987578881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558590deefc0 con 0x558590d78000 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4bdbe00640 1 -- 172.21.15.18:0/987578881 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bec0bd0e0 con 0x558590d78000 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4bdbe00640 1 -- 172.21.15.18:0/987578881 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4bf4026070 con 0x558590d78000 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4c03bdd740 10 client.26618 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4c03bdd740 10 client.26618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4c03bdd740 20 client.26618 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558590def710 0x558590df1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4c03bdd740 4 client.26618 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4c03bdd740 1 -- 172.21.15.18:0/987578881 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558590cdb010 con 0x558590def710 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.056+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558590def710 0x558590df1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.057+0000 7f4bf9e00640 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558590def710 0x558590df1b30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef589002814eafde server_cookie=99bbed3f1abc7c18 in_seq=0 out_seq=0 2024-04-16T17:08:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.057+0000 7f4bdbe00640 10 client.26618 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:17.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:16 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2179302067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.208+0000 7f4bdbe00640 1 -- 172.21.15.18:0/987578881 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558590cdb010 con 0x558590def710 2024-04-16T17:08:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.208+0000 7f4bdbe00640 10 client.26618 handle_command_reply: tid=0 2024-04-16T17:08:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.209+0000 7f4c03bdd740 1 client.26618 shutdown 2024-04-16T17:08:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 -- 172.21.15.18:0/987578881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 msgr2=0x558590ded8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590ded8d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x558590d90ae0 tx=0x7f4bf4004ab0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 -- 172.21.15.18:0/987578881 shutdown_connections 2024-04-16T17:08:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558590def710 0x558590df1b30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590d78000 0x558590ded8d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590d6f7f0 0x558590d72460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4bdbe00640 0 client.26618 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 --2- 172.21.15.18:0/987578881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590d6ea80 0x558590d71f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 -- 172.21.15.18:0/987578881 >> 172.21.15.18:0/987578881 conn(0x558590d095d0 msgr2=0x558590d6b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 -- 172.21.15.18:0/987578881 shutdown_connections 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 1 -- 172.21.15.18:0/987578881 wait complete. 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.210+0000 7f4c03bdd740 20 client.26618 trim_cache size 0 max 16384 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:08:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:08:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 -- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4be00a3d00 msgr2=0x7f4be00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4be00a3d00 0x7f4be00a61c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40094d0 tx=0x7f4bf4004a40 comp rx=0 tx=0).stop 2024-04-16T17:08:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 -- 172.21.15.18:0/656836208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc102080 msgr2=0x7f4bfc074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc102080 0x7f4bfc074300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00e9f0 tx=0x7f4bec00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 -- 172.21.15.18:0/656836208 shutdown_connections 2024-04-16T17:08:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4be00a3d00 0x7f4be00a61c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bfc075fa0 0x7f4bfc074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bfc107150 0x7f4bfc074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 --2- 172.21.15.18:0/656836208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc102080 0x7f4bfc074300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.211+0000 7f4bfb200640 1 -- 172.21.15.18:0/656836208 >> 172.21.15.18:0/656836208 conn(0x7f4bfc0fd940 msgr2=0x7f4bfc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.212+0000 7f4bfb200640 1 -- 172.21.15.18:0/656836208 shutdown_connections 2024-04-16T17:08:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:17.212+0000 7f4bfb200640 1 -- 172.21.15.18:0/656836208 wait complete. 2024-04-16T17:08:17.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:09 smithi018 ceph-mon[31776]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-16T17:08:17.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:11 smithi018 ceph-mon[31776]: pgmap v106: 129 pgs: 129 active+clean; 36 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 217 op/s 2024-04-16T17:08:17.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:13 smithi018 ceph-mon[31776]: pgmap v107: 129 pgs: 129 active+clean; 36 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-16T17:08:17.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:15 smithi018 ceph-mon[31776]: pgmap v108: 129 pgs: 129 active+clean; 36 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-04-16T17:08:17.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:16 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2179302067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:17.543 INFO:tasks.workunit.client.0.smithi018.stdout: 1 115438 4.00 MB/sec execute 30 sec latency 65.868 ms 2024-04-16T17:08:17.601 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:08:17.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:08:17.601 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:08:17.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.794+0000 7fb6d3e00640 1 -- 172.21.15.18:0/2301386333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102a40 msgr2=0x7fb6d4106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.794+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/2301386333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102a40 0x7fb6d4106a80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009900 tx=0x7fb6c002bc50 comp rx=0 tx=0).stop 2024-04-16T17:08:17.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.794+0000 7fb6d3e00640 1 -- 172.21.15.18:0/2301386333 shutdown_connections 2024-04-16T17:08:17.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.794+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/2301386333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d4107160 0x7fb6d410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.794+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/2301386333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102a40 0x7fb6d4106a80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.794+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/2301386333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d4102090 0x7fb6d4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.794+0000 7fb6d3e00640 1 -- 172.21.15.18:0/2301386333 >> 172.21.15.18:0/2301386333 conn(0x7fb6d40fd910 msgr2=0x7fb6d40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.795+0000 7fb6d3e00640 1 -- 172.21.15.18:0/2301386333 shutdown_connections 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.795+0000 7fb6d3e00640 1 -- 172.21.15.18:0/2301386333 wait complete. 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.795+0000 7fb6d3e00640 1 Processor -- start 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d3e00640 1 -- start start 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102090 0x7fb6d41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d4107160 0x7fb6d41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d419a7e0 0x7fb6d419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d3e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41122c0 con 0x7fb6d4102090 2024-04-16T17:08:17.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4112140 con 0x7fb6d419a7e0 2024-04-16T17:08:17.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4112440 con 0x7fb6d4107160 2024-04-16T17:08:17.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102090 0x7fb6d41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102090 0x7fb6d41a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56954/0 (socket says 172.21.15.18:56954) 2024-04-16T17:08:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d2a00640 1 -- 172.21.15.18:0/841365227 learned_addr learned my addr 172.21.15.18:0/841365227 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d3400640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d419a7e0 0x7fb6d419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.796+0000 7fb6d2000640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d4107160 0x7fb6d41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6d3400640 1 -- 172.21.15.18:0/841365227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d4107160 msgr2=0x7fb6d41a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6d3400640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d4107160 0x7fb6d41a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6d3400640 1 -- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102090 msgr2=0x7fb6d41a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6d3400640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102090 0x7fb6d41a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6d3400640 1 -- 172.21.15.18:0/841365227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d419b370 con 0x7fb6d419a7e0 2024-04-16T17:08:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102090 0x7fb6d41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6d3400640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d419a7e0 0x7fb6d419ac90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00be20 tx=0x7fb6cc00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6cb400640 1 -- 172.21.15.18:0/841365227 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc019070 con 0x7fb6d419a7e0 2024-04-16T17:08:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.797+0000 7fb6d3e00640 1 -- 172.21.15.18:0/841365227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d419b660 con 0x7fb6d419a7e0 2024-04-16T17:08:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.798+0000 7fb6cb400640 1 -- 172.21.15.18:0/841365227 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6cc014920 con 0x7fb6d419a7e0 2024-04-16T17:08:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.798+0000 7fb6cb400640 1 -- 172.21.15.18:0/841365227 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0129c0 con 0x7fb6d419a7e0 2024-04-16T17:08:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.798+0000 7fb6d3e00640 1 -- 172.21.15.18:0/841365227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41a7860 con 0x7fb6d419a7e0 2024-04-16T17:08:17.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.802+0000 7fb6dc82b740 1 Processor -- start 2024-04-16T17:08:17.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.802+0000 7fb6cb400640 1 -- 172.21.15.18:0/841365227 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb6cc01c070 con 0x7fb6d419a7e0 2024-04-16T17:08:17.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.802+0000 7fb6dc82b740 1 -- start start 2024-04-16T17:08:17.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac3e90 0x55afa5ac4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac47b0 0x55afa5acce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5acd390 0x55afa5acf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55afa5a28cb0 con 0x55afa5ac3e90 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55afa59cf370 con 0x55afa5acd390 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55afa5a24c70 con 0x55afa5ac47b0 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6d2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac3e90 0x55afa5ac4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac47b0 0x55afa5acce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac47b0 0x55afa5acce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34328/0 (socket says 172.21.15.18:34328) 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6d2a00640 1 -- 172.21.15.18:0/780070399 learned_addr learned my addr 172.21.15.18:0/780070399 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6d2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac3e90 0x55afa5ac4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56964/0 (socket says 172.21.15.18:56964) 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6d2a00640 1 -- 172.21.15.18:0/780070399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5acd390 msgr2=0x55afa5acf780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/780070399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5acd390 0x55afa5acf780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.803+0000 7fb6d2a00640 1 -- 172.21.15.18:0/780070399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac3e90 msgr2=0x55afa5ac4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.804+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/780070399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac3e90 0x55afa5ac4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.804+0000 7fb6d2a00640 1 -- 172.21.15.18:0/780070399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa59ee910 con 0x55afa5ac47b0 2024-04-16T17:08:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.804+0000 7fb6d2000640 1 --2- 172.21.15.18:0/780070399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac3e90 0x55afa5ac4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:17.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.804+0000 7fb6cb400640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6b80a3d20 0x7fb6b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.804+0000 7fb6cb400640 1 -- 172.21.15.18:0/841365227 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb6cc0d0270 con 0x7fb6d419a7e0 2024-04-16T17:08:17.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.804+0000 7fb6d2000640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6b80a3d20 0x7fb6b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.805+0000 7fb6d2000640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6b80a3d20 0x7fb6b80a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb6c002c260 tx=0x7fb6c0000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.819+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/780070399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac47b0 0x55afa5acce50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4011650 tx=0x7fb6c4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ce685d81146ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.826+0000 7fb6bfe00640 1 -- 172.21.15.18:0/780070399 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c401e070 con 0x55afa5ac47b0 2024-04-16T17:08:17.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.826+0000 7fb6bfe00640 1 -- 172.21.15.18:0/780070399 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c400fba0 con 0x55afa5ac47b0 2024-04-16T17:08:17.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.826+0000 7fb6bfe00640 1 -- 172.21.15.18:0/780070399 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4019d40 con 0x55afa5ac47b0 2024-04-16T17:08:17.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 -- 172.21.15.18:0/780070399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac47b0 msgr2=0x55afa5acce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/780070399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac47b0 0x55afa5acce50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4011650 tx=0x7fb6c4009f90 comp rx=0 tx=0).stop 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 -- 172.21.15.18:0/780070399 shutdown_connections 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/780070399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5acd390 0x55afa5acf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/780070399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac47b0 0x55afa5acce50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/780070399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac3e90 0x55afa5ac4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 -- 172.21.15.18:0/780070399 >> 172.21.15.18:0/780070399 conn(0x55afa5abe5c0 msgr2=0x55afa5abe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 -- 172.21.15.18:0/780070399 shutdown_connections 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.827+0000 7fb6dc82b740 1 -- 172.21.15.18:0/780070399 wait complete. 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 1 -- start start 2024-04-16T17:08:17.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 1 Processor -- start 2024-04-16T17:08:17.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac3e90 0x55afa5aca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac47b0 0x55afa5aca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5acd390 0x55afa5acaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55afa5ad0180 con 0x55afa5acd390 2024-04-16T17:08:17.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55afa59e2730 con 0x55afa5ac47b0 2024-04-16T17:08:17.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55afa5ad0300 con 0x55afa5ac3e90 2024-04-16T17:08:17.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.828+0000 7fb6dc82b740 10 client.0 _ll_register_callbacks cb 0x55afa59b11e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac47b0 0x55afa5aca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac47b0 0x55afa5aca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54392/0 (socket says 172.21.15.18:54392) 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d2000640 1 -- 172.21.15.18:0/2154993940 learned_addr learned my addr 172.21.15.18:0/2154993940 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5acd390 0x55afa5acaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d3400640 1 -- 172.21.15.18:0/2154993940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac3e90 msgr2=0x55afa5aca460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac3e90 0x55afa5aca460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d3400640 1 -- 172.21.15.18:0/2154993940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac47b0 msgr2=0x55afa5aca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac47b0 0x55afa5aca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d3400640 1 -- 172.21.15.18:0/2154993940 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa58037e0 con 0x55afa5acd390 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.829+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5acd390 0x55afa5acaee0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0cd720 tx=0x7fb6cc015fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4317634a6020b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.830+0000 7fb6bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.830+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2154993940 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0b5060 con 0x55afa5acd390 2024-04-16T17:08:17.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.830+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2154993940 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb6cc014ad0 con 0x55afa5acd390 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.830+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2154993940 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c400fba0 con 0x55afa5acd390 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.830+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2154993940 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6cc0ba050 con 0x55afa5acd390 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.830+0000 7fb6dc82b740 10 client.16431 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.830+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2154993940 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55afa5863f70 con 0x55afa5acd390 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.830+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2154993940 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c400fba0 con 0x55afa5acd390 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.831+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2154993940 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6cc096b90 con 0x55afa5acd390 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.831+0000 7fb6dc82b740 10 client.16431 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.831+0000 7fb6dc82b740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.831+0000 7fb6dc82b740 20 client.16431 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.831+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55afa5add750 0x55afa5adfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.831+0000 7fb6dc82b740 4 client.16431 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:17.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.831+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2154993940 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afa5a366d0 con 0x55afa5add750 2024-04-16T17:08:17.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.841+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55afa5add750 0x55afa5adfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.849+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55afa5add750 0x55afa5adfbf0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c234a981160aca2 server_cookie=c2c3adfe6e9c9db0 in_seq=0 out_seq=0 2024-04-16T17:08:17.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.849+0000 7fb6bcc00640 10 client.16431 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:17.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2154993940 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afa5a366d0 con 0x55afa5add750 2024-04-16T17:08:17.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6bcc00640 10 client.16431 handle_command_reply: tid=0 2024-04-16T17:08:17.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 client.16431 shutdown 2024-04-16T17:08:17.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2154993940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5acd390 msgr2=0x55afa5acaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5acd390 0x55afa5acaee0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0cd720 tx=0x7fb6cc015fb0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2154993940 shutdown_connections 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55afa5add750 0x55afa5adfbf0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5acd390 0x55afa5acaee0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac47b0 0x55afa5aca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2154993940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5ac3e90 0x55afa5aca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2154993940 >> 172.21.15.18:0/2154993940 conn(0x55afa5abe5c0 msgr2=0x55afa5ac0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6bcc00640 0 client.16431 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2154993940 shutdown_connections 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2154993940 wait complete. 2024-04-16T17:08:17.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.850+0000 7fb6dc82b740 20 client.16431 trim_cache size 0 max 16384 2024-04-16T17:08:17.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.870+0000 7fb6dc82b740 1 Processor -- start 2024-04-16T17:08:17.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6dc82b740 1 -- start start 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5accb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5ad9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ae54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b3e20 con 0x55afa5ac47b0 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b16f0 con 0x55afa5ac3e90 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41af540 con 0x55afa5acd390 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ae54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ae54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34346/0 (socket says 172.21.15.18:34346) 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6d3400640 1 -- 172.21.15.18:0/2255685812 learned_addr learned my addr 172.21.15.18:0/2255685812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5accb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.871+0000 7fb6d2000640 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5ad9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.872+0000 7fb6d3400640 1 -- 172.21.15.18:0/2255685812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 msgr2=0x55afa5accb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.872+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5accb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.872+0000 7fb6d3400640 1 -- 172.21.15.18:0/2255685812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 msgr2=0x55afa5ad9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.872+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5ad9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.872+0000 7fb6d3400640 1 -- 172.21.15.18:0/2255685812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa56fb170 con 0x55afa5acd390 2024-04-16T17:08:17.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.879+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ae54c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc09bd20 tx=0x7fb6cc00ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86136533ae66f075 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.880+0000 7fb6bf400640 1 -- 172.21.15.18:0/2255685812 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc07f080 con 0x55afa5acd390 2024-04-16T17:08:17.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.880+0000 7fb6bf400640 1 -- 172.21.15.18:0/2255685812 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6cc0b54d0 con 0x55afa5acd390 2024-04-16T17:08:17.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.880+0000 7fb6bf400640 1 -- 172.21.15.18:0/2255685812 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc014ad0 con 0x55afa5acd390 2024-04-16T17:08:17.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.881+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2255685812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 msgr2=0x55afa5ae54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.881+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ae54c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc09bd20 tx=0x7fb6cc00ce70 comp rx=0 tx=0).stop 2024-04-16T17:08:17.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.881+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2255685812 shutdown_connections 2024-04-16T17:08:17.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.881+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ae54c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.881+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5ad9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.881+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2255685812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5accb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.881+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2255685812 >> 172.21.15.18:0/2255685812 conn(0x55afa5abe5c0 msgr2=0x55afa5ac0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.887+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2255685812 shutdown_connections 2024-04-16T17:08:17.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.887+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2255685812 wait complete. 2024-04-16T17:08:17.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.887+0000 7fb6dc82b740 1 -- start start 2024-04-16T17:08:17.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.887+0000 7fb6dc82b740 1 Processor -- start 2024-04-16T17:08:17.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.888+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5b3c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.888+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5b3c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.888+0000 7fb6dc82b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ad18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.888+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41afbb0 con 0x55afa5ac47b0 2024-04-16T17:08:17.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.888+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b1020 con 0x55afa5ac3e90 2024-04-16T17:08:17.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.888+0000 7fb6dc82b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b0280 con 0x55afa5acd390 2024-04-16T17:08:17.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.888+0000 7fb6dc82b740 10 client.0 _ll_register_callbacks cb 0x7fb6cc09b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:17.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.888+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5b3c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5b3c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54406/0 (socket says 172.21.15.18:54406) 2024-04-16T17:08:17.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2a00640 1 -- 172.21.15.18:0/2653891438 learned_addr learned my addr 172.21.15.18:0/2653891438 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:17.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2000640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5b3c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ad18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2a00640 1 -- 172.21.15.18:0/2653891438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 msgr2=0x55afa5ad18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ad18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2a00640 1 -- 172.21.15.18:0/2653891438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 msgr2=0x55afa5b3c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5b3c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2a00640 1 -- 172.21.15.18:0/2653891438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa5ad1f90 con 0x55afa5ac3e90 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d3400640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ad18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.889+0000 7fb6d2000640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5b3c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6d2a00640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5b3c030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4011650 tx=0x7fb6c4009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9aeaeceeff53014 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:17.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2653891438 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c401e040 con 0x55afa5ac3e90 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2653891438 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb6c4009c10 con 0x55afa5ac3e90 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2653891438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55afa5a30540 con 0x55afa5ac3e90 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2653891438 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6c4037050 con 0x55afa5ac3e90 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6dc82b740 10 client.26633 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2653891438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55afa5b44940 con 0x55afa5ac3e90 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.890+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2653891438 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55afa5a30540 con 0x55afa5ac3e90 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.891+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2653891438 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6c40054c0 con 0x55afa5ac3e90 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.891+0000 7fb6dc82b740 10 client.26633 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.891+0000 7fb6dc82b740 10 client.26633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.891+0000 7fb6dc82b740 20 client.26633 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.891+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55afa5b44ea0 0x55afa5b472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.891+0000 7fb6dc82b740 4 client.26633 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.891+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2653891438 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55afa59acaa0 con 0x55afa5b44ea0 2024-04-16T17:08:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.891+0000 7fb6d2000640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55afa5b44ea0 0x55afa5b472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.892+0000 7fb6d2000640 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55afa5b44ea0 0x55afa5b472c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99cf91f32db23434 server_cookie=e817c1cb3f0ca5e2 in_seq=0 out_seq=0 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.892+0000 7fb6bcc00640 10 client.26633 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6bcc00640 1 -- 172.21.15.18:0/2653891438 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55afa59acaa0 con 0x55afa5b44ea0 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6bcc00640 10 client.26633 handle_command_reply: tid=0 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 client.26633 shutdown 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2653891438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 msgr2=0x55afa5b3c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5b3c030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4011650 tx=0x7fb6c4009390 comp rx=0 tx=0).stop 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2653891438 shutdown_connections 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55afa5b44ea0 0x55afa5b472c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afa5acd390 0x55afa5ad18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55afa5ac47b0 0x55afa5b3c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 --2- 172.21.15.18:0/2653891438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afa5ac3e90 0x55afa5b3c030 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6bcc00640 0 client.26633 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2653891438 >> 172.21.15.18:0/2653891438 conn(0x55afa5abe5c0 msgr2=0x55afa5abf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.893+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2653891438 shutdown_connections 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.894+0000 7fb6dc82b740 1 -- 172.21.15.18:0/2653891438 wait complete. 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.894+0000 7fb6dc82b740 20 client.26633 trim_cache size 0 max 16384 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "9ae63380-3f1d-43e8-aae6-22962bf3e082", 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:08:17.896 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:08:17.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.894+0000 7fb6d3e00640 1 -- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6b80a3d20 msgr2=0x7fb6b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.894+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6b80a3d20 0x7fb6b80a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb6c002c260 tx=0x7fb6c0000bc0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.894+0000 7fb6d3e00640 1 -- 172.21.15.18:0/841365227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d419a7e0 msgr2=0x7fb6d419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:17.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.894+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d419a7e0 0x7fb6d419ac90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00be20 tx=0x7fb6cc00bf10 comp rx=0 tx=0).stop 2024-04-16T17:08:17.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.895+0000 7fb6d3e00640 1 -- 172.21.15.18:0/841365227 shutdown_connections 2024-04-16T17:08:17.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.895+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6b80a3d20 0x7fb6b80a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.895+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d419a7e0 0x7fb6d419ac90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.895+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d4107160 0x7fb6d41a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.895+0000 7fb6d3e00640 1 --2- 172.21.15.18:0/841365227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6d4102090 0x7fb6d41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:17.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.895+0000 7fb6d3e00640 1 -- 172.21.15.18:0/841365227 >> 172.21.15.18:0/841365227 conn(0x7fb6d40fd910 msgr2=0x7fb6d40ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:17.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.895+0000 7fb6d3e00640 1 -- 172.21.15.18:0/841365227 shutdown_connections 2024-04-16T17:08:17.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:17.895+0000 7fb6d3e00640 1 -- 172.21.15.18:0/841365227 wait complete. 2024-04-16T17:08:17.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:08:17.913 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:08:18.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:17 smithi043 ceph-mon[36901]: pgmap v109: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.085+0000 7f84e4000640 1 -- 172.21.15.18:0/1372555125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc104450 msgr2=0x7f84dc104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.085+0000 7f84e4000640 1 --2- 172.21.15.18:0/1372555125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc104450 0x7f84dc104830 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009900 tx=0x7f84d002bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.086+0000 7f84e4000640 1 -- 172.21.15.18:0/1372555125 shutdown_connections 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.086+0000 7f84e4000640 1 --2- 172.21.15.18:0/1372555125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc109520 0x7f84dc1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.086+0000 7f84e4000640 1 --2- 172.21.15.18:0/1372555125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84dc104e00 0x7f84dc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.086+0000 7f84e4000640 1 --2- 172.21.15.18:0/1372555125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc104450 0x7f84dc104830 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.086+0000 7f84e4000640 1 -- 172.21.15.18:0/1372555125 >> 172.21.15.18:0/1372555125 conn(0x7f84dc0ffc90 msgr2=0x7f84dc1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.086+0000 7f84e4000640 1 -- 172.21.15.18:0/1372555125 shutdown_connections 2024-04-16T17:08:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.086+0000 7f84e4000640 1 -- 172.21.15.18:0/1372555125 wait complete. 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e4000640 1 Processor -- start 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e4000640 1 -- start start 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e4000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc104450 0x7f84dc1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e4000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc109520 0x7f84dc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e4000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84dc19a940 0x7f84dc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e4000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1145e0 con 0x7f84dc109520 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e4000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc114460 con 0x7f84dc19a940 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e4000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc114760 con 0x7f84dc104450 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc104450 0x7f84dc1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e0c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc109520 0x7f84dc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc104450 0x7f84dc1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34366/0 (socket says 172.21.15.18:34366) 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.087+0000 7f84e1600640 1 -- 172.21.15.18:0/2312045605 learned_addr learned my addr 172.21.15.18:0/2312045605 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.088+0000 7f84e2000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84dc19a940 0x7f84dc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.088+0000 7f84e1600640 1 -- 172.21.15.18:0/2312045605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84dc19a940 msgr2=0x7f84dc19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.088+0000 7f84e1600640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84dc19a940 0x7f84dc19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.088+0000 7f84e1600640 1 -- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc109520 msgr2=0x7f84dc1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.088+0000 7f84e1600640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc109520 0x7f84dc1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.088+0000 7f84e1600640 1 -- 172.21.15.18:0/2312045605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc19b460 con 0x7f84dc104450 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.088+0000 7f84e0c00640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc109520 0x7f84dc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.088+0000 7f84e1600640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc104450 0x7f84dc1a0740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f84d0006010 tx=0x7f84d002cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:18.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.089+0000 7f84da000640 1 -- 172.21.15.18:0/2312045605 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0041070 con 0x7f84dc104450 2024-04-16T17:08:18.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.089+0000 7f84da000640 1 -- 172.21.15.18:0/2312045605 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d0004450 con 0x7f84dc104450 2024-04-16T17:08:18.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.089+0000 7f84e4000640 1 -- 172.21.15.18:0/2312045605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc19b6f0 con 0x7f84dc104450 2024-04-16T17:08:18.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.089+0000 7f84da000640 1 -- 172.21.15.18:0/2312045605 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0003e00 con 0x7f84dc104450 2024-04-16T17:08:18.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.089+0000 7f84e4000640 1 -- 172.21.15.18:0/2312045605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1a78f0 con 0x7f84dc104450 2024-04-16T17:08:18.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.092+0000 7f84e6d96740 1 Processor -- start 2024-04-16T17:08:18.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.092+0000 7f84e6d96740 1 -- start start 2024-04-16T17:08:18.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.092+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c18590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.092+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c18ad0 0x555e70c21170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.092+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c216b0 0x555e70c23aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.092+0000 7f84e6d96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555e70b7e120 con 0x555e70c181b0 2024-04-16T17:08:18.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.092+0000 7f84e6d96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555e70b24340 con 0x555e70c216b0 2024-04-16T17:08:18.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.092+0000 7f84e6d96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555e70b7a0e0 con 0x555e70c18ad0 2024-04-16T17:08:18.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c216b0 0x555e70c23aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c216b0 0x555e70c23aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54432/0 (socket says 172.21.15.18:54432) 2024-04-16T17:08:18.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 -- 172.21.15.18:0/3691571145 learned_addr learned my addr 172.21.15.18:0/3691571145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:18.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e2000640 1 --2- 172.21.15.18:0/3691571145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c18590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 -- 172.21.15.18:0/3691571145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c18ad0 msgr2=0x555e70c21170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:18.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3691571145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c18ad0 0x555e70c21170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 -- 172.21.15.18:0/3691571145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 msgr2=0x555e70c18590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:18.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3691571145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c18590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 -- 172.21.15.18:0/3691571145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e7095a7d0 con 0x555e70c216b0 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84da000640 1 -- 172.21.15.18:0/2312045605 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84d0044070 con 0x7f84dc104450 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.093+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3691571145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c216b0 0x555e70c23aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f84c8011470 tx=0x7f84c8011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d1d43c9c4270a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.094+0000 7f84ce600640 1 -- 172.21.15.18:0/3691571145 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c801e070 con 0x555e70c216b0 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.094+0000 7f84ce600640 1 -- 172.21.15.18:0/3691571145 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c80035f0 con 0x555e70c216b0 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.094+0000 7f84ce600640 1 -- 172.21.15.18:0/3691571145 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c800f860 con 0x555e70c216b0 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.094+0000 7f84e6d96740 1 -- 172.21.15.18:0/3691571145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c216b0 msgr2=0x555e70c23aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.094+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3691571145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c216b0 0x555e70c23aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f84c8011470 tx=0x7f84c8011930 comp rx=0 tx=0).stop 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84e6d96740 1 -- 172.21.15.18:0/3691571145 shutdown_connections 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3691571145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c216b0 0x555e70c23aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3691571145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c18ad0 0x555e70c21170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3691571145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c18590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84e6d96740 1 -- 172.21.15.18:0/3691571145 >> 172.21.15.18:0/3691571145 conn(0x555e70c128e0 msgr2=0x555e70c12cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:18.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84e6d96740 1 -- 172.21.15.18:0/3691571145 shutdown_connections 2024-04-16T17:08:18.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84e6d96740 1 -- 172.21.15.18:0/3691571145 wait complete. 2024-04-16T17:08:18.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84da000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f84c40a3cb0 0x7f84c40a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84e6d96740 1 -- start start 2024-04-16T17:08:18.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.095+0000 7f84da000640 1 -- 172.21.15.18:0/2312045605 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f84d00f24a0 con 0x7f84dc104450 2024-04-16T17:08:18.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.096+0000 7f84e2000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f84c40a3cb0 0x7f84c40a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.096+0000 7f84e6d96740 1 Processor -- start 2024-04-16T17:08:18.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.096+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.096+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.096+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c216b0 0x555e70c1f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.096+0000 7f84e6d96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555e70c244a0 con 0x555e70c181b0 2024-04-16T17:08:18.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e6d96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555e70b37710 con 0x555e70c18ad0 2024-04-16T17:08:18.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e6d96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555e70c24620 con 0x555e70c216b0 2024-04-16T17:08:18.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e6d96740 10 client.0 _ll_register_callbacks cb 0x555e70b32af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:18.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54442/0 (socket says 172.21.15.18:54442) 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e0c00640 1 -- 172.21.15.18:0/1938657415 learned_addr learned my addr 172.21.15.18:0/1938657415 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e1600640 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e2000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f84c40a3cb0 0x7f84c40a6170 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005fd0 tx=0x7f84d4009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e2000640 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c216b0 0x555e70c1f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e0c00640 1 -- 172.21.15.18:0/1938657415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c216b0 msgr2=0x555e70c1f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e0c00640 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c216b0 0x555e70c1f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e0c00640 1 -- 172.21.15.18:0/1938657415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 msgr2=0x555e70c1e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e0c00640 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e0c00640 1 -- 172.21.15.18:0/1938657415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e708903f0 con 0x555e70c18ad0 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.097+0000 7f84e2000640 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c216b0 0x555e70c1f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.098+0000 7f84e0c00640 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1ec50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f84c801c040 tx=0x7f84c80039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=143fc9fd7ecd7233 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.098+0000 7f84c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.098+0000 7f84c2a00640 1 -- 172.21.15.18:0/1938657415 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c801e040 con 0x555e70c18ad0 2024-04-16T17:08:18.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.098+0000 7f84e6d96740 1 -- 172.21.15.18:0/1938657415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c80035f0 con 0x555e70c18ad0 2024-04-16T17:08:18.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.098+0000 7f84c2a00640 1 -- 172.21.15.18:0/1938657415 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f84c8005040 con 0x555e70c18ad0 2024-04-16T17:08:18.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.098+0000 7f84c2a00640 1 -- 172.21.15.18:0/1938657415 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84c8032050 con 0x555e70c18ad0 2024-04-16T17:08:18.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.098+0000 7f84e6d96740 10 client.26645 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:18.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.098+0000 7f84e6d96740 1 -- 172.21.15.18:0/1938657415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e708500f0 con 0x555e70c18ad0 2024-04-16T17:08:18.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84c2a00640 1 -- 172.21.15.18:0/1938657415 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c80035f0 con 0x555e70c18ad0 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84c2a00640 1 -- 172.21.15.18:0/1938657415 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f84c8021030 con 0x555e70c18ad0 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84e6d96740 10 client.26645 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84e6d96740 10 client.26645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84e6d96740 20 client.26645 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84e6d96740 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555e70c28920 0x555e70c2ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84e6d96740 4 client.26645 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84e6d96740 1 -- 172.21.15.18:0/1938657415 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e70b8bb40 con 0x555e70c28920 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.099+0000 7f84e1600640 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555e70c28920 0x555e70c2ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.100+0000 7f84e1600640 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555e70c28920 0x555e70c2ada0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83f26c9731e131c server_cookie=1dc5bc79ca9b4975 in_seq=0 out_seq=0 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.100+0000 7f84c2a00640 10 client.26645 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.100+0000 7f84c2a00640 1 -- 172.21.15.18:0/1938657415 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e70b8bb40 con 0x555e70c28920 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.100+0000 7f84c2a00640 10 client.26645 handle_command_reply: tid=0 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.100+0000 7f84e6d96740 1 client.26645 shutdown 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 -- 172.21.15.18:0/1938657415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 msgr2=0x555e70c1ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1ec50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f84c801c040 tx=0x7f84c80039d0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 -- 172.21.15.18:0/1938657415 shutdown_connections 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555e70c28920 0x555e70c2ada0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c216b0 0x555e70c1f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1ec50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84c2a00640 0 client.26645 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 --2- 172.21.15.18:0/1938657415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 -- 172.21.15.18:0/1938657415 >> 172.21.15.18:0/1938657415 conn(0x555e70c128e0 msgr2=0x555e70c14630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 -- 172.21.15.18:0/1938657415 shutdown_connections 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 1 -- 172.21.15.18:0/1938657415 wait complete. 2024-04-16T17:08:18.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.101+0000 7f84e6d96740 20 client.26645 trim_cache size 0 max 16384 2024-04-16T17:08:18.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.119+0000 7f84e6d96740 1 Processor -- start 2024-04-16T17:08:18.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e6d96740 1 -- start start 2024-04-16T17:08:18.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c128e0 0x555e70c20e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e6d96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1afba0 con 0x555e70c181b0 2024-04-16T17:08:18.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e6d96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1ad470 con 0x555e70c18ad0 2024-04-16T17:08:18.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e6d96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1ab2c0 con 0x555e70c128e0 2024-04-16T17:08:18.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e0c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e0c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57016/0 (socket says 172.21.15.18:57016) 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e0c00640 1 -- 172.21.15.18:0/3001383249 learned_addr learned my addr 172.21.15.18:0/3001383249 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e1600640 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c128e0 0x555e70c20e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.120+0000 7f84e2000640 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84e0c00640 1 -- 172.21.15.18:0/3001383249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c128e0 msgr2=0x555e70c20e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c128e0 0x555e70c20e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84e0c00640 1 -- 172.21.15.18:0/3001383249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 msgr2=0x555e70c356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c356d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84e0c00640 1 -- 172.21.15.18:0/3001383249 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e70c24380 con 0x555e70c181b0 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84e1600640 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c128e0 0x555e70c20e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1efd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f84c8004bb0 tx=0x7f84c8011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3b24360ca1d0df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84cdc00640 1 -- 172.21.15.18:0/3001383249 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c801e040 con 0x555e70c181b0 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84cdc00640 1 -- 172.21.15.18:0/3001383249 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c8029070 con 0x555e70c181b0 2024-04-16T17:08:18.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.121+0000 7f84cdc00640 1 -- 172.21.15.18:0/3001383249 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8003ca0 con 0x555e70c181b0 2024-04-16T17:08:18.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.122+0000 7f84e6d96740 1 -- 172.21.15.18:0/3001383249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 msgr2=0x555e70c1efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.122+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1efd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f84c8004bb0 tx=0x7f84c8011930 comp rx=0 tx=0).stop 2024-04-16T17:08:18.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.122+0000 7f84e6d96740 1 -- 172.21.15.18:0/3001383249 shutdown_connections 2024-04-16T17:08:18.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.122+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.122+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c181b0 0x555e70c1efd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.122+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3001383249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c128e0 0x555e70c20e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.122+0000 7f84e6d96740 1 -- 172.21.15.18:0/3001383249 >> 172.21.15.18:0/3001383249 conn(0x555e70c2b080 msgr2=0x555e70c34fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:18.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.123+0000 7f84e6d96740 1 -- 172.21.15.18:0/3001383249 shutdown_connections 2024-04-16T17:08:18.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.123+0000 7f84e6d96740 1 -- 172.21.15.18:0/3001383249 wait complete. 2024-04-16T17:08:18.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.123+0000 7f84e6d96740 1 -- start start 2024-04-16T17:08:18.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.123+0000 7f84e6d96740 1 Processor -- start 2024-04-16T17:08:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.124+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c128e0 0x555e70c1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.124+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c181b0 0x555e70c1bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.124+0000 7f84e6d96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.124+0000 7f84e6d96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1ab930 con 0x555e70c128e0 2024-04-16T17:08:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.124+0000 7f84e6d96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1acda0 con 0x555e70c18ad0 2024-04-16T17:08:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.124+0000 7f84e6d96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1ac000 con 0x555e70c181b0 2024-04-16T17:08:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.124+0000 7f84e6d96740 10 client.0 _ll_register_callbacks cb 0x7f84c801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c128e0 0x555e70c1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c181b0 0x555e70c1bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c128e0 0x555e70c1b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57028/0 (socket says 172.21.15.18:57028) 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 -- 172.21.15.18:0/3488811973 learned_addr learned my addr 172.21.15.18:0/3488811973 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e2000640 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 -- 172.21.15.18:0/3488811973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c181b0 msgr2=0x555e70c1bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c181b0 0x555e70c1bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 -- 172.21.15.18:0/3488811973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 msgr2=0x555e70c1c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 -- 172.21.15.18:0/3488811973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e70ca7bf0 con 0x555e70c128e0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c181b0 0x555e70c1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84e1600640 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c128e0 0x555e70c1b600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f84d00babc0 tx=0x7f84d00ba490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32651f7bd05844d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.125+0000 7f84c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.126+0000 7f84c2a00640 1 -- 172.21.15.18:0/3488811973 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00d6070 con 0x555e70c128e0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.126+0000 7f84c2a00640 1 -- 172.21.15.18:0/3488811973 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f84d00bc4f0 con 0x555e70c128e0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.126+0000 7f84e6d96740 1 -- 172.21.15.18:0/3488811973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c8029070 con 0x555e70c128e0 2024-04-16T17:08:18.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.126+0000 7f84c2a00640 1 -- 172.21.15.18:0/3488811973 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84d004b040 con 0x555e70c128e0 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.126+0000 7f84e6d96740 10 client.16455 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.126+0000 7f84e6d96740 1 -- 172.21.15.18:0/3488811973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e70ca8ec0 con 0x555e70c128e0 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.126+0000 7f84c2a00640 1 -- 172.21.15.18:0/3488811973 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c8029070 con 0x555e70c128e0 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84c2a00640 1 -- 172.21.15.18:0/3488811973 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f84d00ba860 con 0x555e70c128e0 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84e6d96740 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84e6d96740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84e6d96740 20 client.16455 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555e70c216b0 0x555e70cab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84e6d96740 4 client.16455 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84e6d96740 1 -- 172.21.15.18:0/3488811973 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e70b859b0 con 0x555e70c216b0 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555e70c216b0 0x555e70cab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:18.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84e0c00640 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555e70c216b0 0x555e70cab5c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1224cdb2172baa server_cookie=594f98178bd75f80 in_seq=0 out_seq=0 2024-04-16T17:08:18.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.127+0000 7f84c2a00640 10 client.16455 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84c2a00640 1 -- 172.21.15.18:0/3488811973 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555e70b859b0 con 0x555e70c216b0 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84c2a00640 10 client.16455 handle_command_reply: tid=0 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 client.16455 shutdown 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 -- 172.21.15.18:0/3488811973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c128e0 msgr2=0x555e70c1b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c128e0 0x555e70c1b600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f84d00babc0 tx=0x7f84d00ba490 comp rx=0 tx=0).stop 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 -- 172.21.15.18:0/3488811973 shutdown_connections 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555e70c216b0 0x555e70cab5c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e70c18ad0 0x555e70c1c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e70c181b0 0x555e70c1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 --2- 172.21.15.18:0/3488811973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555e70c128e0 0x555e70c1b600 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 -- 172.21.15.18:0/3488811973 >> 172.21.15.18:0/3488811973 conn(0x555e70c2b080 msgr2=0x555e70c118d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84c2a00640 0 client.16455 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 -- 172.21.15.18:0/3488811973 shutdown_connections 2024-04-16T17:08:18.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 1 -- 172.21.15.18:0/3488811973 wait complete. 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.128+0000 7f84e6d96740 20 client.16455 trim_cache size 0 max 16384 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout: "9ae63380-3f1d-43e8-aae6-22962bf3e082": { 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "9ae63380-3f1d-43e8-aae6-22962bf3e082", 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:08:18.133 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:08:18.134 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:08:18.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 -- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f84c40a3cb0 msgr2=0x7f84c40a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f84c40a3cb0 0x7f84c40a6170 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005fd0 tx=0x7f84d4009500 comp rx=0 tx=0).stop 2024-04-16T17:08:18.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 -- 172.21.15.18:0/2312045605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc104450 msgr2=0x7f84dc1a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:18.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc104450 0x7f84dc1a0740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f84d0006010 tx=0x7f84d002cb30 comp rx=0 tx=0).stop 2024-04-16T17:08:18.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 -- 172.21.15.18:0/2312045605 shutdown_connections 2024-04-16T17:08:18.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f84c40a3cb0 0x7f84c40a6170 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84dc19a940 0x7f84dc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84dc109520 0x7f84dc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 --2- 172.21.15.18:0/2312045605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc104450 0x7f84dc1a0740 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:18.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 -- 172.21.15.18:0/2312045605 >> 172.21.15.18:0/2312045605 conn(0x7f84dc0ffc90 msgr2=0x7f84dc102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:18.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 -- 172.21.15.18:0/2312045605 shutdown_connections 2024-04-16T17:08:18.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:18.129+0000 7f84e4000640 1 -- 172.21.15.18:0/2312045605 wait complete. 2024-04-16T17:08:18.146 INFO:tasks.cephfs.filesystem:scrub status for tag:9ae63380-3f1d-43e8-aae6-22962bf3e082 - {'path': '/', 'tag': '9ae63380-3f1d-43e8-aae6-22962bf3e082', 'options': 'recursive,force'} 2024-04-16T17:08:18.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:17 smithi022 ceph-mon[36023]: pgmap v109: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-04-16T17:08:18.543 INFO:tasks.workunit.client.0.smithi018.stdout: 1 115903 3.92 MB/sec execute 31 sec latency 84.307 ms 2024-04-16T17:08:19.543 INFO:tasks.workunit.client.0.smithi018.stdout: 1 116176 3.85 MB/sec execute 32 sec latency 106.558 ms 2024-04-16T17:08:19.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:19 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:08:19.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:19 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:19.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:19 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:08:19.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:19 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:19.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:19 smithi022 ceph-mon[36023]: pgmap v110: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-04-16T17:08:19.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:19 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:08:19.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:19 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:08:19.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:19 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:19.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:19 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:08:19.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:19 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:19.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:19 smithi043 ceph-mon[36901]: pgmap v110: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-04-16T17:08:19.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:19 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:08:20.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 117355 4.05 MB/sec execute 33 sec latency 22.217 ms 2024-04-16T17:08:21.454 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:17 smithi018 ceph-mon[31776]: pgmap v109: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-04-16T17:08:21.455 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:19 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:08:21.455 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:19 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:21.455 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:19 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:08:21.455 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:19 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:21.455 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:19 smithi018 ceph-mon[31776]: pgmap v110: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-04-16T17:08:21.455 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:19 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:08:21.455 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:21 smithi018 ceph-mon[31776]: pgmap v111: 129 pgs: 129 active+clean; 35 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-04-16T17:08:21.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 118002 3.96 MB/sec execute 34 sec latency 18.381 ms 2024-04-16T17:08:21.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:21 smithi022 ceph-mon[36023]: pgmap v111: 129 pgs: 129 active+clean; 35 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-04-16T17:08:21.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:21 smithi043 ceph-mon[36901]: pgmap v111: 129 pgs: 129 active+clean; 35 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-04-16T17:08:22.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 119154 3.99 MB/sec execute 35 sec latency 25.524 ms 2024-04-16T17:08:23.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 119707 3.94 MB/sec execute 36 sec latency 92.913 ms 2024-04-16T17:08:23.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:23 smithi022 ceph-mon[36023]: pgmap v112: 129 pgs: 129 active+clean; 35 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-16T17:08:23.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:23 smithi043 ceph-mon[36901]: pgmap v112: 129 pgs: 129 active+clean; 35 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-16T17:08:24.545 INFO:tasks.workunit.client.0.smithi018.stdout: 1 120125 3.96 MB/sec execute 37 sec latency 141.218 ms 2024-04-16T17:08:25.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 121021 4.02 MB/sec execute 38 sec latency 17.265 ms 2024-04-16T17:08:25.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:25 smithi022 ceph-mon[36023]: pgmap v113: 129 pgs: 129 active+clean; 35 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-04-16T17:08:25.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:25 smithi043 ceph-mon[36901]: pgmap v113: 129 pgs: 129 active+clean; 35 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-04-16T17:08:26.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 121775 3.97 MB/sec execute 39 sec latency 17.184 ms 2024-04-16T17:08:27.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 123017 4.00 MB/sec execute 40 sec latency 29.939 ms 2024-04-16T17:08:27.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:27 smithi022 ceph-mon[36023]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-04-16T17:08:27.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:27 smithi043 ceph-mon[36901]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-04-16T17:08:28.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 123539 4.02 MB/sec execute 41 sec latency 63.107 ms 2024-04-16T17:08:29.545 INFO:tasks.workunit.client.0.smithi018.stdout: 1 123916 3.98 MB/sec execute 42 sec latency 47.011 ms 2024-04-16T17:08:29.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:29 smithi022 ceph-mon[36023]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-04-16T17:08:29.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:29 smithi043 ceph-mon[36901]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-04-16T17:08:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 -- 172.21.15.18:0/2924991698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 msgr2=0x7f779c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2924991698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 0x7f779c074790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f778c009900 tx=0x7f778c02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 -- 172.21.15.18:0/2924991698 shutdown_connections 2024-04-16T17:08:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2924991698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f779c10e6d0 0x7f779c110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2924991698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f779c074cd0 0x7f779c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2924991698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 0x7f779c074790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 -- 172.21.15.18:0/2924991698 >> 172.21.15.18:0/2924991698 conn(0x7f779c0fd940 msgr2=0x7f779c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 -- 172.21.15.18:0/2924991698 shutdown_connections 2024-04-16T17:08:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.353+0000 7f77a0e00640 1 -- 172.21.15.18:0/2924991698 wait complete. 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.354+0000 7f77a0e00640 1 Processor -- start 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.354+0000 7f77a0e00640 1 -- start start 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.354+0000 7f77a0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 0x7f779c101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.354+0000 7f77a0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f779c074cd0 0x7f779c102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.354+0000 7f77a0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f779c10e6d0 0x7f779c102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.354+0000 7f77a0e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f779c114430 con 0x7f779c10e6d0 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.354+0000 7f77a0e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1142b0 con 0x7f779c0743b0 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.354+0000 7f77a0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1145b0 con 0x7f779c074cd0 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 0x7f779c101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 0x7f779c101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36510/0 (socket says 172.21.15.18:36510) 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779b400640 1 -- 172.21.15.18:0/2878153691 learned_addr learned my addr 172.21.15.18:0/2878153691 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779aa00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f779c074cd0 0x7f779c102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779be00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f779c10e6d0 0x7f779c102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779b400640 1 -- 172.21.15.18:0/2878153691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f779c074cd0 msgr2=0x7f779c102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779b400640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f779c074cd0 0x7f779c102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779b400640 1 -- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f779c10e6d0 msgr2=0x7f779c102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779b400640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f779c10e6d0 0x7f779c102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779b400640 1 -- 172.21.15.18:0/2878153691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c1ab000 con 0x7f779c0743b0 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.355+0000 7f779be00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f779c10e6d0 0x7f779c102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.356+0000 7f779b400640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 0x7f779c101df0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f778c02c2d0 tx=0x7f778c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.356+0000 7f7793e00640 1 -- 172.21.15.18:0/2878153691 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c041020 con 0x7f779c0743b0 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.356+0000 7f77a0e00640 1 -- 172.21.15.18:0/2878153691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c1ab290 con 0x7f779c0743b0 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.356+0000 7f7793e00640 1 -- 172.21.15.18:0/2878153691 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778c005000 con 0x7f779c0743b0 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.356+0000 7f7793e00640 1 -- 172.21.15.18:0/2878153691 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c003b50 con 0x7f779c0743b0 2024-04-16T17:08:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.356+0000 7f77a0e00640 1 -- 172.21.15.18:0/2878153691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c1ab7d0 con 0x7f779c0743b0 2024-04-16T17:08:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.359+0000 7f77a5228740 1 Processor -- start 2024-04-16T17:08:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.359+0000 7f7793e00640 1 -- 172.21.15.18:0/2878153691 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f778c044020 con 0x7f779c0743b0 2024-04-16T17:08:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.359+0000 7f77a5228740 1 -- start start 2024-04-16T17:08:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f77a5228740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec8d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f77a5228740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec8df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f77a5228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f77a5228740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5627febf2780 con 0x5627fec8da80 2024-04-16T17:08:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f77a5228740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5627feb98f40 con 0x5627fec8e7f0 2024-04-16T17:08:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f77a5228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5627febee740 con 0x5627fec97000 2024-04-16T17:08:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f779b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec8df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f779b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec8df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58418/0 (socket says 172.21.15.18:58418) 2024-04-16T17:08:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f779b400640 1 -- 172.21.15.18:0/913138995 learned_addr learned my addr 172.21.15.18:0/913138995 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f779aa00640 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.360+0000 7f779be00640 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec8d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f779b400640 1 -- 172.21.15.18:0/913138995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 msgr2=0x5627fec993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f779b400640 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f779b400640 1 -- 172.21.15.18:0/913138995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 msgr2=0x5627fec8d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f779b400640 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec8d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f779b400640 1 -- 172.21.15.18:0/913138995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627fe905210 con 0x5627fec8da80 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f779aa00640 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f779be00640 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec8d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f7793e00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f77800a3d80 0x7f77800a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.361+0000 7f779b400640 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec8df00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f778c0ef650 tx=0x7f778c0fa3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3f7c37e6ee6511 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.362+0000 7f7793e00640 1 -- 172.21.15.18:0/2878153691 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f778c0f22c0 con 0x7f779c0743b0 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.362+0000 7f779be00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f77800a3d80 0x7f77800a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.362+0000 7f7790c00640 1 -- 172.21.15.18:0/913138995 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0ba200 con 0x5627fec8da80 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.362+0000 7f7790c00640 1 -- 172.21.15.18:0/913138995 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778c0be660 con 0x5627fec8da80 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.362+0000 7f7790c00640 1 -- 172.21.15.18:0/913138995 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0f5e60 con 0x5627fec8da80 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.362+0000 7f77a5228740 1 -- 172.21.15.18:0/913138995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 msgr2=0x5627fec8df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.362+0000 7f77a5228740 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec8df00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f778c0ef650 tx=0x7f778c0fa3b0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.362+0000 7f779be00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f77800a3d80 0x7f77800a6240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7794005fd0 tx=0x7f7794009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 -- 172.21.15.18:0/913138995 shutdown_connections 2024-04-16T17:08:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec8df00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 --2- 172.21.15.18:0/913138995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec8d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 -- 172.21.15.18:0/913138995 >> 172.21.15.18:0/913138995 conn(0x5627fec285d0 msgr2=0x5627fec88770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 -- 172.21.15.18:0/913138995 shutdown_connections 2024-04-16T17:08:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 -- 172.21.15.18:0/913138995 wait complete. 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 -- start start 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.363+0000 7f77a5228740 1 Processor -- start 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f77a5228740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec93e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f77a5228740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f77a5228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec94900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f77a5228740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5627fec99af0 con 0x5627fec8da80 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f77a5228740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5627febac390 con 0x5627fec8e7f0 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f77a5228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5627fec99c70 con 0x5627fec97000 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f77a5228740 10 client.0 _ll_register_callbacks cb 0x5627feb95f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f779b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec93e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.364+0000 7f779aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec93e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58434/0 (socket says 172.21.15.18:58434) 2024-04-16T17:08:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779b400640 1 -- 172.21.15.18:0/4278199539 learned_addr learned my addr 172.21.15.18:0/4278199539 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779be00640 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec94900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779b400640 1 -- 172.21.15.18:0/4278199539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 msgr2=0x5627fec94900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779b400640 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec94900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779b400640 1 -- 172.21.15.18:0/4278199539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 msgr2=0x5627fec943c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779b400640 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec943c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779b400640 1 -- 172.21.15.18:0/4278199539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627fe9d1620 con 0x5627fec8da80 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.365+0000 7f779aa00640 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.366+0000 7f779b400640 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec93e80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f778c02c2a0 tx=0x7f778c02bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc11d4547cd16c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.366+0000 7f7785600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.366+0000 7f7785600640 1 -- 172.21.15.18:0/4278199539 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0f5e60 con 0x5627fec8da80 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.366+0000 7f7785600640 1 -- 172.21.15.18:0/4278199539 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f778c0bc260 con 0x5627fec8da80 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.366+0000 7f77a5228740 1 -- 172.21.15.18:0/4278199539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778c0be660 con 0x5627fec8da80 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.366+0000 7f7785600640 1 -- 172.21.15.18:0/4278199539 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f778c0dc050 con 0x5627fec8da80 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.366+0000 7f77a5228740 10 client.16467 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.366+0000 7f77a5228740 1 -- 172.21.15.18:0/4278199539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627fec99af0 con 0x5627fec8da80 2024-04-16T17:08:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.367+0000 7f7785600640 1 -- 172.21.15.18:0/4278199539 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778c0be660 con 0x5627fec8da80 2024-04-16T17:08:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.367+0000 7f7785600640 1 -- 172.21.15.18:0/4278199539 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f778c0054d0 con 0x5627fec8da80 2024-04-16T17:08:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.367+0000 7f77a5228740 10 client.16467 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.367+0000 7f77a5228740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.368+0000 7f77a5228740 20 client.16467 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.368+0000 7f77a5228740 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5627feca6cf0 0x5627feca9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.368+0000 7f77a5228740 4 client.16467 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.368+0000 7f77a5228740 1 -- 172.21.15.18:0/4278199539 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627fec001a0 con 0x5627feca6cf0 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.368+0000 7f779aa00640 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5627feca6cf0 0x5627feca9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.368+0000 7f779aa00640 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5627feca6cf0 0x5627feca9170 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485db5a1df09b7a8 server_cookie=bc5f6a1146de6bf9 in_seq=0 out_seq=0 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.368+0000 7f7785600640 10 client.16467 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f7785600640 1 -- 172.21.15.18:0/4278199539 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627fec001a0 con 0x5627feca6cf0 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f7785600640 10 client.16467 handle_command_reply: tid=0 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 client.16467 shutdown 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 -- 172.21.15.18:0/4278199539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 msgr2=0x5627fec93e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec93e80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f778c02c2a0 tx=0x7f778c02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 -- 172.21.15.18:0/4278199539 shutdown_connections 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5627feca6cf0 0x5627feca9170 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec94900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 --2- 172.21.15.18:0/4278199539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec8da80 0x5627fec93e80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f77a5228740 1 -- 172.21.15.18:0/4278199539 >> 172.21.15.18:0/4278199539 conn(0x5627fec285d0 msgr2=0x5627fec8a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.369+0000 7f7785600640 0 client.16467 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.370+0000 7f77a5228740 1 -- 172.21.15.18:0/4278199539 shutdown_connections 2024-04-16T17:08:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.370+0000 7f77a5228740 1 -- 172.21.15.18:0/4278199539 wait complete. 2024-04-16T17:08:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.370+0000 7f77a5228740 20 client.16467 trim_cache size 0 max 16384 2024-04-16T17:08:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.387+0000 7f77a5228740 1 Processor -- start 2024-04-16T17:08:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f77a5228740 1 -- start start 2024-04-16T17:08:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f77a5228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec8e7f0 0x5627fec965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f77a5228740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec97000 0x5627fec94770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f77a5228740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627feca29b0 0x5627fecaea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f77a5228740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1b8240 con 0x5627fec97000 2024-04-16T17:08:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f77a5228740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1b5b10 con 0x5627feca29b0 2024-04-16T17:08:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f77a5228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1b3960 con 0x5627fec8e7f0 2024-04-16T17:08:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f779b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec8e7f0 0x5627fec965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f779be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627feca29b0 0x5627fecaea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f779b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec8e7f0 0x5627fec965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48930/0 (socket says 172.21.15.18:48930) 2024-04-16T17:08:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f779be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627feca29b0 0x5627fecaea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36540/0 (socket says 172.21.15.18:36540) 2024-04-16T17:08:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f779b400640 1 -- 172.21.15.18:0/77084024 learned_addr learned my addr 172.21.15.18:0/77084024 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.388+0000 7f779aa00640 1 --2- 172.21.15.18:0/77084024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec97000 0x5627fec94770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.389+0000 7f779b400640 1 -- 172.21.15.18:0/77084024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627feca29b0 msgr2=0x5627fecaea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.389+0000 7f779b400640 1 --2- 172.21.15.18:0/77084024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627feca29b0 0x5627fecaea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.389+0000 7f779b400640 1 -- 172.21.15.18:0/77084024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec97000 msgr2=0x5627fec94770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.389+0000 7f779b400640 1 --2- 172.21.15.18:0/77084024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec97000 0x5627fec94770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.389+0000 7f779b400640 1 -- 172.21.15.18:0/77084024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627fec8cd50 con 0x5627fec8e7f0 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.389+0000 7f779b400640 1 --2- 172.21.15.18:0/77084024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec8e7f0 0x5627fec965b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f778c0bdef0 tx=0x7f778c005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34a9d1329f24096a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.389+0000 7f7787e00640 1 -- 172.21.15.18:0/77084024 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0f5e60 con 0x5627fec8e7f0 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.389+0000 7f7787e00640 1 -- 172.21.15.18:0/77084024 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778c0be660 con 0x5627fec8e7f0 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f7787e00640 1 -- 172.21.15.18:0/77084024 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0bd8c0 con 0x5627fec8e7f0 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 -- 172.21.15.18:0/77084024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec8e7f0 msgr2=0x5627fec965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 --2- 172.21.15.18:0/77084024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec8e7f0 0x5627fec965b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f778c0bdef0 tx=0x7f778c005440 comp rx=0 tx=0).stop 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 -- 172.21.15.18:0/77084024 shutdown_connections 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 --2- 172.21.15.18:0/77084024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627feca29b0 0x5627fecaea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 --2- 172.21.15.18:0/77084024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627fec97000 0x5627fec94770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 --2- 172.21.15.18:0/77084024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec8e7f0 0x5627fec965b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 -- 172.21.15.18:0/77084024 >> 172.21.15.18:0/77084024 conn(0x5627fec285d0 msgr2=0x5627fec92f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 -- 172.21.15.18:0/77084024 shutdown_connections 2024-04-16T17:08:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.390+0000 7f77a5228740 1 -- 172.21.15.18:0/77084024 wait complete. 2024-04-16T17:08:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 1 -- start start 2024-04-16T17:08:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 1 Processor -- start 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec9f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627feca29b0 0x5627fec90d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1b3fd0 con 0x5627feca29b0 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1b5440 con 0x5627fec8e7f0 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1b46a0 con 0x5627fec97000 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.391+0000 7f77a5228740 10 client.0 _ll_register_callbacks cb 0x7f7788009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec9f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627feca29b0 0x5627fec90d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec9f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36542/0 (socket says 172.21.15.18:36542) 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 -- 172.21.15.18:0/2699339150 learned_addr learned my addr 172.21.15.18:0/2699339150 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 -- 172.21.15.18:0/2699339150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 msgr2=0x5627fec907f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec907f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 -- 172.21.15.18:0/2699339150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627feca29b0 msgr2=0x5627fec90d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627feca29b0 0x5627fec90d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 -- 172.21.15.18:0/2699339150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627fec91560 con 0x5627fec8e7f0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f779b400640 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec9f1d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f778c0bdc60 tx=0x7f778c0fa3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccaa350e425ce278 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.392+0000 7f7785600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.393+0000 7f7785600640 1 -- 172.21.15.18:0/2699339150 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0bdaa0 con 0x5627fec8e7f0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.393+0000 7f7785600640 1 -- 172.21.15.18:0/2699339150 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f778c0bc260 con 0x5627fec8e7f0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.393+0000 7f77a5228740 1 -- 172.21.15.18:0/2699339150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778c0be660 con 0x5627fec8e7f0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.393+0000 7f7785600640 1 -- 172.21.15.18:0/2699339150 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f778c0a0050 con 0x5627fec8e7f0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.393+0000 7f77a5228740 10 client.26666 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.393+0000 7f77a5228740 1 -- 172.21.15.18:0/2699339150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627fec91560 con 0x5627fec8e7f0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.393+0000 7f7785600640 1 -- 172.21.15.18:0/2699339150 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778c0be660 con 0x5627fec8e7f0 2024-04-16T17:08:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.393+0000 7f7785600640 1 -- 172.21.15.18:0/2699339150 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f778c0c6740 con 0x5627fec8e7f0 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.394+0000 7f77a5228740 10 client.26666 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.394+0000 7f77a5228740 10 client.26666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.394+0000 7f77a5228740 20 client.26666 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.394+0000 7f77a5228740 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5627fed0d450 0x5627fed0f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.394+0000 7f77a5228740 4 client.26666 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.394+0000 7f77a5228740 1 -- 172.21.15.18:0/2699339150 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627febfa010 con 0x5627fed0d450 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.394+0000 7f779aa00640 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5627fed0d450 0x5627fed0f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.395+0000 7f779aa00640 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5627fed0d450 0x5627fed0f840 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cabb16a7d6afe501 server_cookie=7bdbbc903f043a90 in_seq=0 out_seq=0 2024-04-16T17:08:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.395+0000 7f7785600640 10 client.26666 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:30.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:23 smithi018 ceph-mon[31776]: pgmap v112: 129 pgs: 129 active+clean; 35 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-16T17:08:30.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:25 smithi018 ceph-mon[31776]: pgmap v113: 129 pgs: 129 active+clean; 35 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-04-16T17:08:30.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:27 smithi018 ceph-mon[31776]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-04-16T17:08:30.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:29 smithi018 ceph-mon[31776]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-04-16T17:08:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f7785600640 1 -- 172.21.15.18:0/2699339150 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627febfa010 con 0x5627fed0d450 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f7785600640 10 client.26666 handle_command_reply: tid=0 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 client.26666 shutdown 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 -- 172.21.15.18:0/2699339150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 msgr2=0x5627fec9f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec9f1d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f778c0bdc60 tx=0x7f778c0fa3b0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 -- 172.21.15.18:0/2699339150 shutdown_connections 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5627fed0d450 0x5627fed0f840 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5627feca29b0 0x5627fec90d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627fec97000 0x5627fec907f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 --2- 172.21.15.18:0/2699339150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627fec8e7f0 0x5627fec9f1d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f77a5228740 1 -- 172.21.15.18:0/2699339150 >> 172.21.15.18:0/2699339150 conn(0x5627fec285d0 msgr2=0x5627feca19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.511+0000 7f7785600640 0 client.26666 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.512+0000 7f77a5228740 1 -- 172.21.15.18:0/2699339150 shutdown_connections 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.512+0000 7f77a5228740 1 -- 172.21.15.18:0/2699339150 wait complete. 2024-04-16T17:08:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.512+0000 7f77a5228740 20 client.26666 trim_cache size 0 max 16384 2024-04-16T17:08:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:08:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:08:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:08:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 -- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f77800a3d80 msgr2=0x7f77800a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f77800a3d80 0x7f77800a6240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7794005fd0 tx=0x7f7794009500 comp rx=0 tx=0).stop 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 -- 172.21.15.18:0/2878153691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 msgr2=0x7f779c101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 0x7f779c101df0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f778c02c2d0 tx=0x7f778c004360 comp rx=0 tx=0).stop 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 -- 172.21.15.18:0/2878153691 shutdown_connections 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f77800a3d80 0x7f77800a6240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f779c10e6d0 0x7f779c102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f779c074cd0 0x7f779c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 --2- 172.21.15.18:0/2878153691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f779c0743b0 0x7f779c101df0 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f778c02c2d0 tx=0x7f778c004360 comp rx=0 tx=0).stop 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 -- 172.21.15.18:0/2878153691 >> 172.21.15.18:0/2878153691 conn(0x7f779c0fd940 msgr2=0x7f779c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 -- 172.21.15.18:0/2878153691 shutdown_connections 2024-04-16T17:08:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:30.513+0000 7f77a0e00640 1 -- 172.21.15.18:0/2878153691 wait complete. 2024-04-16T17:08:30.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 124623 4.00 MB/sec execute 43 sec latency 59.910 ms 2024-04-16T17:08:31.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 125607 4.03 MB/sec execute 44 sec latency 17.068 ms 2024-04-16T17:08:31.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:31 smithi022 ceph-mon[36023]: pgmap v116: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-04-16T17:08:31.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:31 smithi043 ceph-mon[36901]: pgmap v116: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-04-16T17:08:32.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:31 smithi018 ceph-mon[31776]: pgmap v116: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-04-16T17:08:32.544 INFO:tasks.workunit.client.0.smithi018.stdout: 1 126744 4.00 MB/sec execute 45 sec latency 28.064 ms 2024-04-16T17:08:33.545 INFO:tasks.workunit.client.0.smithi018.stdout: 1 127350 4.03 MB/sec execute 46 sec latency 59.997 ms 2024-04-16T17:08:33.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:33 smithi022 ceph-mon[36023]: pgmap v117: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-16T17:08:33.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:33 smithi043 ceph-mon[36901]: pgmap v117: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-16T17:08:34.545 INFO:tasks.workunit.client.0.smithi018.stdout: 1 127687 4.03 MB/sec execute 47 sec latency 104.356 ms 2024-04-16T17:08:35.340 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:33 smithi018 ceph-mon[31776]: pgmap v117: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-16T17:08:35.545 INFO:tasks.workunit.client.0.smithi018.stdout: 1 128259 3.98 MB/sec execute 48 sec latency 20.338 ms 2024-04-16T17:08:35.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:35 smithi022 ceph-mon[36023]: pgmap v118: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1009 KiB/s wr, 81 op/s 2024-04-16T17:08:35.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:35 smithi043 ceph-mon[36901]: pgmap v118: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1009 KiB/s wr, 81 op/s 2024-04-16T17:08:36.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 129272 4.01 MB/sec execute 49 sec latency 17.379 ms 2024-04-16T17:08:37.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 130648 4.07 MB/sec execute 50 sec latency 23.047 ms 2024-04-16T17:08:37.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:37 smithi022 ceph-mon[36023]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-16T17:08:37.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:37 smithi043 ceph-mon[36901]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-16T17:08:38.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 131087 4.03 MB/sec execute 51 sec latency 74.870 ms 2024-04-16T17:08:39.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 131445 4.02 MB/sec execute 52 sec latency 61.038 ms 2024-04-16T17:08:39.596 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:35 smithi018 ceph-mon[31776]: pgmap v118: 129 pgs: 129 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1009 KiB/s wr, 81 op/s 2024-04-16T17:08:39.596 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:37 smithi018 ceph-mon[31776]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-16T17:08:39.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:39 smithi022 ceph-mon[36023]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 881 KiB/s wr, 139 op/s 2024-04-16T17:08:39.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:39 smithi043 ceph-mon[36901]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 881 KiB/s wr, 139 op/s 2024-04-16T17:08:40.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 131885 3.97 MB/sec execute 53 sec latency 30.620 ms 2024-04-16T17:08:41.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 132894 3.99 MB/sec execute 54 sec latency 21.528 ms 2024-04-16T17:08:41.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:41 smithi022 ceph-mon[36023]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-04-16T17:08:41.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:41 smithi043 ceph-mon[36901]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-04-16T17:08:42.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 134191 4.05 MB/sec execute 55 sec latency 27.971 ms 2024-04-16T17:08:43.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 134667 4.02 MB/sec execute 56 sec latency 52.339 ms 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.649+0000 7f0722a00640 1 -- 172.21.15.18:0/799272871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 msgr2=0x7f071c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.649+0000 7f0722a00640 1 --2- 172.21.15.18:0/799272871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 0x7f071c10bde0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0708009900 tx=0x7f070802bce0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.650+0000 7f0722a00640 1 -- 172.21.15.18:0/799272871 shutdown_connections 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.650+0000 7f0722a00640 1 --2- 172.21.15.18:0/799272871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f071c10c400 0x7f071c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.650+0000 7f0722a00640 1 --2- 172.21.15.18:0/799272871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 0x7f071c10bde0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.650+0000 7f0722a00640 1 --2- 172.21.15.18:0/799272871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c102fe0 0x7f071c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.650+0000 7f0722a00640 1 -- 172.21.15.18:0/799272871 >> 172.21.15.18:0/799272871 conn(0x7f071c0fc7e0 msgr2=0x7f071c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.650+0000 7f0722a00640 1 -- 172.21.15.18:0/799272871 shutdown_connections 2024-04-16T17:08:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.650+0000 7f0722a00640 1 -- 172.21.15.18:0/799272871 wait complete. 2024-04-16T17:08:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.651+0000 7f0722a00640 1 Processor -- start 2024-04-16T17:08:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.651+0000 7f0722a00640 1 -- start start 2024-04-16T17:08:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0722a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f071c102fe0 0x7f071c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0722a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 0x7f071c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0722a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c10c400 0x7f071c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0722a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f071c112280 con 0x7f071c103900 2024-04-16T17:08:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0722a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f071c112100 con 0x7f071c10c400 2024-04-16T17:08:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0722a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f071c112400 con 0x7f071c102fe0 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0721600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f071c102fe0 0x7f071c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0721600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f071c102fe0 0x7f071c1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44688/0 (socket says 172.21.15.18:44688) 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0722000640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c10c400 0x7f071c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0721600640 1 -- 172.21.15.18:0/3083094215 learned_addr learned my addr 172.21.15.18:0/3083094215 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.652+0000 7f0720c00640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 0x7f071c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.653+0000 7f0721600640 1 -- 172.21.15.18:0/3083094215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c10c400 msgr2=0x7f071c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.653+0000 7f0721600640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c10c400 0x7f071c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.653+0000 7f0721600640 1 -- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 msgr2=0x7f071c1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.653+0000 7f0721600640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 0x7f071c1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.653+0000 7f0721600640 1 -- 172.21.15.18:0/3083094215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c19af90 con 0x7f071c102fe0 2024-04-16T17:08:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.653+0000 7f0720c00640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 0x7f071c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.653+0000 7f0721600640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f071c102fe0 0x7f071c1a0650 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f071000b450 tx=0x7f071000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.654+0000 7f071a000640 1 -- 172.21.15.18:0/3083094215 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710015020 con 0x7f071c102fe0 2024-04-16T17:08:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.654+0000 7f0722a00640 1 -- 172.21.15.18:0/3083094215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c19b280 con 0x7f071c102fe0 2024-04-16T17:08:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.654+0000 7f071a000640 1 -- 172.21.15.18:0/3083094215 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0710011d70 con 0x7f071c102fe0 2024-04-16T17:08:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.654+0000 7f071a000640 1 -- 172.21.15.18:0/3083094215 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071000f340 con 0x7f071c102fe0 2024-04-16T17:08:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.654+0000 7f0722a00640 1 -- 172.21.15.18:0/3083094215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c1a74a0 con 0x7f071c102fe0 2024-04-16T17:08:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.657+0000 7f0726c53740 1 Processor -- start 2024-04-16T17:08:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0726c53740 1 -- start start 2024-04-16T17:08:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0726c53740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab1027f0 0x55d0ab101540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0726c53740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab101a80 0x55d0ab101f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0726c53740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab10d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0726c53740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d0ab066780 con 0x55d0ab101a80 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0726c53740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d0ab00cf40 con 0x55d0ab1027f0 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0726c53740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d0ab062740 con 0x55d0ab10b000 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0722000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab1027f0 0x55d0ab101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0722000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab1027f0 0x55d0ab101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53786/0 (socket says 172.21.15.18:53786) 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0722000640 1 -- 172.21.15.18:0/2618358459 learned_addr learned my addr 172.21.15.18:0/2618358459 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.658+0000 7f0720c00640 1 --2- 172.21.15.18:0/2618358459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab10d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f0722000640 1 -- 172.21.15.18:0/2618358459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 msgr2=0x55d0ab10d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f0722000640 1 --2- 172.21.15.18:0/2618358459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab10d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f0722000640 1 -- 172.21.15.18:0/2618358459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab101a80 msgr2=0x55d0ab101f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f0722000640 1 --2- 172.21.15.18:0/2618358459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab101a80 0x55d0ab101f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f0722000640 1 -- 172.21.15.18:0/2618358459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0aad79210 con 0x55d0ab1027f0 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f071a000640 1 -- 172.21.15.18:0/3083094215 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0710018020 con 0x7f071c102fe0 2024-04-16T17:08:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f0722000640 1 --2- 172.21.15.18:0/2618358459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab1027f0 0x55d0ab101540 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0714009fd0 tx=0x7f07140118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe10374bbd1791 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f070e600640 1 -- 172.21.15.18:0/2618358459 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714004d20 con 0x55d0ab1027f0 2024-04-16T17:08:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f070e600640 1 -- 172.21.15.18:0/2618358459 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0714004050 con 0x55d0ab1027f0 2024-04-16T17:08:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.659+0000 7f070e600640 1 -- 172.21.15.18:0/2618358459 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714005280 con 0x55d0ab1027f0 2024-04-16T17:08:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f0726c53740 1 -- 172.21.15.18:0/2618358459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab1027f0 msgr2=0x55d0ab101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f0726c53740 1 --2- 172.21.15.18:0/2618358459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab1027f0 0x55d0ab101540 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0714009fd0 tx=0x7f07140118d0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f071a000640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07040a3d00 0x7f07040a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f0726c53740 1 -- 172.21.15.18:0/2618358459 shutdown_connections 2024-04-16T17:08:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f0726c53740 1 --2- 172.21.15.18:0/2618358459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab10d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f0726c53740 1 --2- 172.21.15.18:0/2618358459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab101a80 0x55d0ab101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f0726c53740 1 --2- 172.21.15.18:0/2618358459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab1027f0 0x55d0ab101540 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f0726c53740 1 -- 172.21.15.18:0/2618358459 >> 172.21.15.18:0/2618358459 conn(0x55d0ab09c5d0 msgr2=0x55d0ab0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f071a000640 1 -- 172.21.15.18:0/3083094215 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f07100d04a0 con 0x7f071c102fe0 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.660+0000 7f0721600640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07040a3d00 0x7f07040a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.661+0000 7f0726c53740 1 -- 172.21.15.18:0/2618358459 shutdown_connections 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.661+0000 7f0726c53740 1 -- 172.21.15.18:0/2618358459 wait complete. 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.661+0000 7f0726c53740 1 -- start start 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.661+0000 7f0721600640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07040a3d00 0x7f07040a61c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f07100cdaa0 tx=0x7f0710098600 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.661+0000 7f0726c53740 1 Processor -- start 2024-04-16T17:08:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0726c53740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab101a80 0x55d0ab10aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0726c53740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab107e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0726c53740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab108510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0726c53740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d0ab10dd30 con 0x55d0ab1027f0 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0726c53740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d0ab020390 con 0x55d0ab101a80 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0726c53740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d0ab10deb0 con 0x55d0ab10b000 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0726c53740 10 client.0 _ll_register_callbacks cb 0x55d0ab009f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0722000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab101a80 0x55d0ab10aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0722000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab101a80 0x55d0ab10aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53790/0 (socket says 172.21.15.18:53790) 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0722000640 1 -- 172.21.15.18:0/3191796354 learned_addr learned my addr 172.21.15.18:0/3191796354 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0722000640 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab108510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.662+0000 7f0720c00640 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab107e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.663+0000 7f0720c00640 1 -- 172.21.15.18:0/3191796354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 msgr2=0x55d0ab108510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.663+0000 7f0720c00640 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab108510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.663+0000 7f0720c00640 1 -- 172.21.15.18:0/3191796354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab101a80 msgr2=0x55d0ab10aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.663+0000 7f0720c00640 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab101a80 0x55d0ab10aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.663+0000 7f0720c00640 1 -- 172.21.15.18:0/3191796354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0aae45620 con 0x55d0ab1027f0 2024-04-16T17:08:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.663+0000 7f0722000640 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.663+0000 7f0720c00640 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab107e20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0708009230 tx=0x7f0708005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5174fce4e93ba359 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.663+0000 7f0702a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.664+0000 7f0702a00640 1 -- 172.21.15.18:0/3191796354 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708042070 con 0x55d0ab1027f0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.664+0000 7f0702a00640 1 -- 172.21.15.18:0/3191796354 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f0708005030 con 0x55d0ab1027f0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.664+0000 7f0726c53740 1 -- 172.21.15.18:0/3191796354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0714004050 con 0x55d0ab1027f0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.664+0000 7f0702a00640 1 -- 172.21.15.18:0/3191796354 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f070804f050 con 0x55d0ab1027f0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.664+0000 7f0726c53740 10 client.16482 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.664+0000 7f0726c53740 1 -- 172.21.15.18:0/3191796354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0ab119e00 con 0x55d0ab1027f0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0702a00640 1 -- 172.21.15.18:0/3191796354 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0714004050 con 0x55d0ab1027f0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0702a00640 1 -- 172.21.15.18:0/3191796354 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f07080035f0 con 0x55d0ab1027f0 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0726c53740 10 client.16482 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0726c53740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0726c53740 20 client.16482 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0726c53740 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0ab11b2c0 0x55d0ab11d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0726c53740 4 client.16482 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0726c53740 1 -- 172.21.15.18:0/3191796354 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0ab0741a0 con 0x55d0ab11b2c0 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.665+0000 7f0722000640 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0ab11b2c0 0x55d0ab11d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.666+0000 7f0722000640 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0ab11b2c0 0x55d0ab11d6b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81163843af39a15 server_cookie=2aa12571c8be9655 in_seq=0 out_seq=0 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.666+0000 7f0702a00640 10 client.16482 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0702a00640 1 -- 172.21.15.18:0/3191796354 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0ab0741a0 con 0x55d0ab11b2c0 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0702a00640 10 client.16482 handle_command_reply: tid=0 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 client.16482 shutdown 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 -- 172.21.15.18:0/3191796354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 msgr2=0x55d0ab107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab107e20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0708009230 tx=0x7f0708005d90 comp rx=0 tx=0).stop 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 -- 172.21.15.18:0/3191796354 shutdown_connections 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0ab11b2c0 0x55d0ab11d6b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab10b000 0x55d0ab108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab107e20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 --2- 172.21.15.18:0/3191796354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab101a80 0x55d0ab10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 -- 172.21.15.18:0/3191796354 >> 172.21.15.18:0/3191796354 conn(0x55d0ab09c5d0 msgr2=0x55d0ab0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0702a00640 0 client.16482 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 -- 172.21.15.18:0/3191796354 shutdown_connections 2024-04-16T17:08:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 1 -- 172.21.15.18:0/3191796354 wait complete. 2024-04-16T17:08:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.667+0000 7f0726c53740 20 client.16482 trim_cache size 0 max 16384 2024-04-16T17:08:43.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:43 smithi022 ceph-mon[36023]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 851 KiB/s wr, 137 op/s 2024-04-16T17:08:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.685+0000 7f0726c53740 1 Processor -- start 2024-04-16T17:08:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.685+0000 7f0726c53740 1 -- start start 2024-04-16T17:08:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0726c53740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab116c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0726c53740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab122f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0726c53740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab123490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0726c53740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b3e90 con 0x55d0ab1027f0 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0726c53740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b1760 con 0x55d0ab10b000 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0726c53740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1af5b0 con 0x55d0ab101a80 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0720c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab122f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0720c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab122f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49228/0 (socket says 172.21.15.18:49228) 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0720c00640 1 -- 172.21.15.18:0/4104497426 learned_addr learned my addr 172.21.15.18:0/4104497426 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0722000640 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab123490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0722000640 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab116c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0720c00640 1 -- 172.21.15.18:0/4104497426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 msgr2=0x55d0ab116c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0720c00640 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab116c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0720c00640 1 -- 172.21.15.18:0/4104497426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 msgr2=0x55d0ab123490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0720c00640 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab123490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0720c00640 1 -- 172.21.15.18:0/4104497426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0ab100d50 con 0x55d0ab1027f0 2024-04-16T17:08:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.686+0000 7f0722000640 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab116c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.687+0000 7f0720c00640 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab122f50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f070802c8d0 tx=0x7f0708005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6023e7ae2a21a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.687+0000 7f070dc00640 1 -- 172.21.15.18:0/4104497426 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708042030 con 0x55d0ab1027f0 2024-04-16T17:08:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.687+0000 7f070dc00640 1 -- 172.21.15.18:0/4104497426 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0708009520 con 0x55d0ab1027f0 2024-04-16T17:08:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.687+0000 7f070dc00640 1 -- 172.21.15.18:0/4104497426 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708004dc0 con 0x55d0ab1027f0 2024-04-16T17:08:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 -- 172.21.15.18:0/4104497426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 msgr2=0x55d0ab122f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab122f50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f070802c8d0 tx=0x7f0708005e80 comp rx=0 tx=0).stop 2024-04-16T17:08:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 -- 172.21.15.18:0/4104497426 shutdown_connections 2024-04-16T17:08:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab123490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab122f50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 --2- 172.21.15.18:0/4104497426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab116c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 -- 172.21.15.18:0/4104497426 >> 172.21.15.18:0/4104497426 conn(0x55d0ab09c5d0 msgr2=0x55d0ab1066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 -- 172.21.15.18:0/4104497426 shutdown_connections 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.688+0000 7f0726c53740 1 -- 172.21.15.18:0/4104497426 wait complete. 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 1 -- start start 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 1 Processor -- start 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab104ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab1053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab1807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1afc20 con 0x55d0ab1027f0 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b1090 con 0x55d0ab10b000 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b02f0 con 0x55d0ab101a80 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.689+0000 7f0726c53740 10 client.0 _ll_register_callbacks cb 0x7f071c06a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab1053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab1053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49242/0 (socket says 172.21.15.18:49242) 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 -- 172.21.15.18:0/758580499 learned_addr learned my addr 172.21.15.18:0/758580499 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0722000640 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab104ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0722000640 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab1807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 -- 172.21.15.18:0/758580499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 msgr2=0x55d0ab104ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab104ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 -- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 msgr2=0x55d0ab1807c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab1807c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 -- 172.21.15.18:0/758580499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0ab180d00 con 0x55d0ab1027f0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0722000640 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab1807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0720c00640 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab1053e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f070802c220 tx=0x7f0708003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de5cf761d98b09c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0702a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.690+0000 7f0702a00640 1 -- 172.21.15.18:0/758580499 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708059070 con 0x55d0ab1027f0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0702a00640 1 -- 172.21.15.18:0/758580499 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f07080048c0 con 0x55d0ab1027f0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 1 -- 172.21.15.18:0/758580499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0708009520 con 0x55d0ab1027f0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0702a00640 1 -- 172.21.15.18:0/758580499 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0708004ac0 con 0x55d0ab1027f0 2024-04-16T17:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 10 client.16488 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 1 -- 172.21.15.18:0/758580499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0ab181e50 con 0x55d0ab1027f0 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0702a00640 1 -- 172.21.15.18:0/758580499 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0708009520 con 0x55d0ab1027f0 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0702a00640 1 -- 172.21.15.18:0/758580499 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0708034440 con 0x55d0ab1027f0 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 10 client.16488 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 20 client.16488 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0ab182500 0x55d0ab1848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 4 client.16488 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.691+0000 7f0726c53740 1 -- 172.21.15.18:0/758580499 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0ab06e010 con 0x55d0ab182500 2024-04-16T17:08:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.692+0000 7f0722000640 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0ab182500 0x55d0ab1848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.692+0000 7f0722000640 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0ab182500 0x55d0ab1848f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3133d0e23cdefebe server_cookie=bcd134132221d702 in_seq=0 out_seq=0 2024-04-16T17:08:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.692+0000 7f0702a00640 10 client.16488 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:43.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:43 smithi043 ceph-mon[36901]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 851 KiB/s wr, 137 op/s 2024-04-16T17:08:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.818+0000 7f0702a00640 1 -- 172.21.15.18:0/758580499 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0ab06e010 con 0x55d0ab182500 2024-04-16T17:08:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.818+0000 7f0702a00640 10 client.16488 handle_command_reply: tid=0 2024-04-16T17:08:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 client.16488 shutdown 2024-04-16T17:08:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 -- 172.21.15.18:0/758580499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 msgr2=0x55d0ab1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab1053e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f070802c220 tx=0x7f0708003800 comp rx=0 tx=0).stop 2024-04-16T17:08:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 -- 172.21.15.18:0/758580499 shutdown_connections 2024-04-16T17:08:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0ab182500 0x55d0ab1848f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab10b000 0x55d0ab1807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0ab1027f0 0x55d0ab1053e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 --2- 172.21.15.18:0/758580499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0ab101a80 0x55d0ab104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 -- 172.21.15.18:0/758580499 >> 172.21.15.18:0/758580499 conn(0x55d0ab09c5d0 msgr2=0x55d0ab1094d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0702a00640 0 client.16488 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 -- 172.21.15.18:0/758580499 shutdown_connections 2024-04-16T17:08:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 1 -- 172.21.15.18:0/758580499 wait complete. 2024-04-16T17:08:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.820+0000 7f0726c53740 20 client.16488 trim_cache size 0 max 16384 2024-04-16T17:08:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 -- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07040a3d00 msgr2=0x7f07040a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07040a3d00 0x7f07040a61c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f07100cdaa0 tx=0x7f0710098600 comp rx=0 tx=0).stop 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 -- 172.21.15.18:0/3083094215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f071c102fe0 msgr2=0x7f071c1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f071c102fe0 0x7f071c1a0650 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f071000b450 tx=0x7f071000b910 comp rx=0 tx=0).stop 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 -- 172.21.15.18:0/3083094215 shutdown_connections 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07040a3d00 0x7f07040a61c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c10c400 0x7f071c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f071c103900 0x7f071c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 --2- 172.21.15.18:0/3083094215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f071c102fe0 0x7f071c1a0650 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 -- 172.21.15.18:0/3083094215 >> 172.21.15.18:0/3083094215 conn(0x7f071c0fc7e0 msgr2=0x7f071c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 -- 172.21.15.18:0/3083094215 shutdown_connections 2024-04-16T17:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:43.824+0000 7f0722a00640 1 -- 172.21.15.18:0/3083094215 wait complete. 2024-04-16T17:08:44.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 135041 4.02 MB/sec execute 57 sec latency 94.268 ms 2024-04-16T17:08:45.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 135572 3.96 MB/sec execute 58 sec latency 70.826 ms 2024-04-16T17:08:45.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:45 smithi022 ceph-mon[36023]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 851 KiB/s wr, 137 op/s 2024-04-16T17:08:45.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:45 smithi043 ceph-mon[36901]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 851 KiB/s wr, 137 op/s 2024-04-16T17:08:46.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 136646 3.98 MB/sec execute 59 sec latency 15.267 ms 2024-04-16T17:08:47.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 137748 4.03 MB/sec execute 60 sec latency 34.727 ms 2024-04-16T17:08:47.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:47 smithi022 ceph-mon[36023]: pgmap v124: 129 pgs: 129 active+clean; 37 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-04-16T17:08:47.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:47 smithi043 ceph-mon[36901]: pgmap v124: 129 pgs: 129 active+clean; 37 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-04-16T17:08:48.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:08:48.147 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:08:48.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.446+0000 7f8af7000640 1 -- 172.21.15.18:0/3083942244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af0106310 msgr2=0x7f8af0106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.446+0000 7f8af7000640 1 --2- 172.21.15.18:0/3083942244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af0106310 0x7f8af0106ca0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ca0 tx=0x7f8ae802c140 comp rx=0 tx=0).stop 2024-04-16T17:08:48.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.446+0000 7f8af7000640 1 -- 172.21.15.18:0/3083942244 shutdown_connections 2024-04-16T17:08:48.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.446+0000 7f8af7000640 1 --2- 172.21.15.18:0/3083942244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af0106310 0x7f8af0106ca0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.446+0000 7f8af7000640 1 --2- 172.21.15.18:0/3083942244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8af0101040 0x7f8af0105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.446+0000 7f8af7000640 1 --2- 172.21.15.18:0/3083942244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af0100690 0x7f8af0100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.446+0000 7f8af7000640 1 -- 172.21.15.18:0/3083942244 >> 172.21.15.18:0/3083942244 conn(0x7f8af0075da0 msgr2=0x7f8af00761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:48.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.452+0000 7f8af7000640 1 -- 172.21.15.18:0/3083942244 shutdown_connections 2024-04-16T17:08:48.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.453+0000 7f8af7000640 1 -- 172.21.15.18:0/3083942244 wait complete. 2024-04-16T17:08:48.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.455+0000 7f8af7000640 1 Processor -- start 2024-04-16T17:08:48.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.455+0000 7f8af7000640 1 -- start start 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.455+0000 7f8af7000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af0106310 0x7f8af0073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8af7000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af006bb10 0x7f8af006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8af7000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8af006c4b0 0x7f8af01b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8af7000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0118ab0 con 0x7f8af0106310 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8af7000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0118930 con 0x7f8af006c4b0 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8af7000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0118c30 con 0x7f8af006bb10 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8af5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8af006c4b0 0x7f8af01b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8af5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8af006c4b0 0x7f8af01b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55398/0 (socket says 172.21.15.18:55398) 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8af5000640 1 -- 172.21.15.18:0/853159043 learned_addr learned my addr 172.21.15.18:0/853159043 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8aef400640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af006bb10 0x7f8af006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.456+0000 7f8aefe00640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af0106310 0x7f8af0073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.457+0000 7f8af5000640 1 -- 172.21.15.18:0/853159043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af006bb10 msgr2=0x7f8af006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.457+0000 7f8af5000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af006bb10 0x7f8af006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.457+0000 7f8af5000640 1 -- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af0106310 msgr2=0x7f8af0073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.457+0000 7f8af5000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af0106310 0x7f8af0073b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.457+0000 7f8af5000640 1 -- 172.21.15.18:0/853159043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af01b4480 con 0x7f8af006c4b0 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.458+0000 7f8af5000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8af006c4b0 0x7f8af01b3f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8037040 tx=0x7f8ae8035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.459+0000 7f8aecc00640 1 -- 172.21.15.18:0/853159043 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8039040 con 0x7f8af006c4b0 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.459+0000 7f8aecc00640 1 -- 172.21.15.18:0/853159043 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae8055070 con 0x7f8af006c4b0 2024-04-16T17:08:48.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.459+0000 7f8aecc00640 1 -- 172.21.15.18:0/853159043 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8050450 con 0x7f8af006c4b0 2024-04-16T17:08:48.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.460+0000 7f8af7000640 1 -- 172.21.15.18:0/853159043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af01083e0 con 0x7f8af006c4b0 2024-04-16T17:08:48.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.461+0000 7f8af7000640 1 -- 172.21.15.18:0/853159043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af0108850 con 0x7f8af006c4b0 2024-04-16T17:08:48.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.471+0000 7f8af9c08740 1 Processor -- start 2024-04-16T17:08:48.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.471+0000 7f8aecc00640 1 -- 172.21.15.18:0/853159043 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ae8004040 con 0x7f8af006c4b0 2024-04-16T17:08:48.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.473+0000 7f8af9c08740 1 -- start start 2024-04-16T17:08:48.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.473+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d35161b0 0x5621d3516590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.473+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d351f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.473+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d351f6b0 0x5621d3521aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.473+0000 7f8af9c08740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5621d347c120 con 0x5621d351f6b0 2024-04-16T17:08:48.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.473+0000 7f8af9c08740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5621d3422340 con 0x5621d3516ad0 2024-04-16T17:08:48.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.473+0000 7f8af9c08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5621d34780e0 con 0x5621d35161b0 2024-04-16T17:08:48.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.474+0000 7f8aef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d35161b0 0x5621d3516590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.474+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d351f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.474+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d351f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55412/0 (socket says 172.21.15.18:55412) 2024-04-16T17:08:48.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.474+0000 7f8aefe00640 1 -- 172.21.15.18:0/3104366803 learned_addr learned my addr 172.21.15.18:0/3104366803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:48.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.475+0000 7f8aef400640 1 --2- 172.21.15.18:0/3104366803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d351f6b0 0x5621d3521aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.475+0000 7f8aecc00640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ad40a3ef0 0x7f8ad40a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.475+0000 7f8aecc00640 1 -- 172.21.15.18:0/853159043 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8ae8103cc0 con 0x7f8af006c4b0 2024-04-16T17:08:48.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.475+0000 7f8aefe00640 1 -- 172.21.15.18:0/3104366803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d35161b0 msgr2=0x5621d3516590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:48.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.475+0000 7f8aefe00640 1 --2- 172.21.15.18:0/3104366803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d35161b0 0x5621d3516590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.475+0000 7f8aefe00640 1 -- 172.21.15.18:0/3104366803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d351f6b0 msgr2=0x5621d3521aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:48.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.476+0000 7f8af5000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ad40a3ef0 0x7f8ad40a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.476+0000 7f8aefe00640 1 --2- 172.21.15.18:0/3104366803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d351f6b0 0x5621d3521aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.476+0000 7f8aefe00640 1 -- 172.21.15.18:0/3104366803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d32587d0 con 0x5621d3516ad0 2024-04-16T17:08:48.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.476+0000 7f8aefe00640 1 --2- 172.21.15.18:0/3104366803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d351f170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00a820 tx=0x7f8adc00ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f7f6defb04d818 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:48.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.477+0000 7f8af5000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ad40a3ef0 0x7f8ad40a63b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80cf980 tx=0x7f8ae80095c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:48.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.478+0000 7f8ae5600640 1 -- 172.21.15.18:0/3104366803 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc015070 con 0x5621d3516ad0 2024-04-16T17:08:48.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.479+0000 7f8ae5600640 1 -- 172.21.15.18:0/3104366803 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8adc0113f0 con 0x5621d3516ad0 2024-04-16T17:08:48.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.479+0000 7f8ae5600640 1 -- 172.21.15.18:0/3104366803 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0185c0 con 0x5621d3516ad0 2024-04-16T17:08:48.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 -- 172.21.15.18:0/3104366803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 msgr2=0x5621d351f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 --2- 172.21.15.18:0/3104366803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d351f170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00a820 tx=0x7f8adc00ace0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 -- 172.21.15.18:0/3104366803 shutdown_connections 2024-04-16T17:08:48.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 --2- 172.21.15.18:0/3104366803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d351f6b0 0x5621d3521aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 --2- 172.21.15.18:0/3104366803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d351f170 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 --2- 172.21.15.18:0/3104366803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d35161b0 0x5621d3516590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 -- 172.21.15.18:0/3104366803 >> 172.21.15.18:0/3104366803 conn(0x5621d35108e0 msgr2=0x5621d3510cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:48.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 -- 172.21.15.18:0/3104366803 shutdown_connections 2024-04-16T17:08:48.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.480+0000 7f8af9c08740 1 -- 172.21.15.18:0/3104366803 wait complete. 2024-04-16T17:08:48.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.481+0000 7f8af9c08740 1 -- start start 2024-04-16T17:08:48.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.481+0000 7f8af9c08740 1 Processor -- start 2024-04-16T17:08:48.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 0x5621d351d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8af9c08740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5621d35224a0 con 0x5621d35161b0 2024-04-16T17:08:48.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8af9c08740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5621d3435710 con 0x5621d351f6b0 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8af9c08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5621d3522620 con 0x5621d3516ad0 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8af9c08740 10 client.0 _ll_register_callbacks cb 0x5621d3430af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35274/0 (socket says 172.21.15.18:35274) 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.482+0000 7f8aefe00640 1 -- 172.21.15.18:0/1844147964 learned_addr learned my addr 172.21.15.18:0/1844147964 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8aef400640 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8aefe00640 1 -- 172.21.15.18:0/1844147964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 msgr2=0x5621d351d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8aefe00640 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 0x5621d351d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8aefe00640 1 -- 172.21.15.18:0/1844147964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 msgr2=0x5621d351c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8aefe00640 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8aefe00640 1 -- 172.21.15.18:0/1844147964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d318e3f0 con 0x5621d3516ad0 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8aef400640 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8aefe00640 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351cbc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00c520 tx=0x7f8adc00afc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f500b4162e616cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.483+0000 7f8ada000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8ada000640 1 -- 172.21.15.18:0/1844147964 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc029060 con 0x5621d3516ad0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8af9c08740 1 -- 172.21.15.18:0/1844147964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8adc0113f0 con 0x5621d3516ad0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8ada000640 1 -- 172.21.15.18:0/1844147964 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8adc00cac0 con 0x5621d3516ad0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8ada000640 1 -- 172.21.15.18:0/1844147964 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8adc030050 con 0x5621d3516ad0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8af9c08740 10 client.26662 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8af9c08740 1 -- 172.21.15.18:0/1844147964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621d314e0f0 con 0x5621d3516ad0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8ada000640 1 -- 172.21.15.18:0/1844147964 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8adc0113f0 con 0x5621d3516ad0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8ada000640 1 -- 172.21.15.18:0/1844147964 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8adc01ac20 con 0x5621d3516ad0 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8af9c08740 10 client.26662 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8af9c08740 10 client.26662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.484+0000 7f8af9c08740 20 client.26662 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.485+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5621d352f860 0x5621d3531c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.485+0000 7f8af9c08740 4 client.26662 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:48.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.485+0000 7f8af9c08740 1 -- 172.21.15.18:0/1844147964 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621d3489b40 con 0x5621d352f860 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.485+0000 7f8aef400640 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5621d352f860 0x5621d3531c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.485+0000 7f8aef400640 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5621d352f860 0x5621d3531c50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed56dd0c0e3ccf0e server_cookie=37cd4b0e2ec0da3b in_seq=0 out_seq=0 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.485+0000 7f8ada000640 10 client.26662 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8ada000640 1 -- 172.21.15.18:0/1844147964 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621d3489b40 con 0x5621d352f860 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8ada000640 10 client.26662 handle_command_reply: tid=0 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 client.26662 shutdown 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 -- 172.21.15.18:0/1844147964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 msgr2=0x5621d351cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351cbc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00c520 tx=0x7f8adc00afc0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 -- 172.21.15.18:0/1844147964 shutdown_connections 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5621d352f860 0x5621d3531c50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 0x5621d351d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351cbc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1844147964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8ada000640 0 client.26662 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.486+0000 7f8af9c08740 1 -- 172.21.15.18:0/1844147964 >> 172.21.15.18:0/1844147964 conn(0x5621d35108e0 msgr2=0x5621d3512630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.487+0000 7f8af9c08740 1 -- 172.21.15.18:0/1844147964 shutdown_connections 2024-04-16T17:08:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.487+0000 7f8af9c08740 1 -- 172.21.15.18:0/1844147964 wait complete. 2024-04-16T17:08:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.487+0000 7f8af9c08740 20 client.26662 trim_cache size 0 max 16384 2024-04-16T17:08:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.505+0000 7f8af9c08740 1 Processor -- start 2024-04-16T17:08:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.505+0000 7f8af9c08740 1 -- start start 2024-04-16T17:08:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.505+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.505+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 0x5621d352b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8af9c08740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01b89d0 con 0x5621d35161b0 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8af9c08740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01b62a0 con 0x5621d351f6b0 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8af9c08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8af006d040 con 0x5621d3516ad0 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57468/0 (socket says 172.21.15.18:57468) 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 -- 172.21.15.18:0/4155509773 learned_addr learned my addr 172.21.15.18:0/4155509773 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aefe00640 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 0x5621d352b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 -- 172.21.15.18:0/4155509773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 msgr2=0x5621d351b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 -- 172.21.15.18:0/4155509773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 msgr2=0x5621d352b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 0x5621d352b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.506+0000 7f8aef400640 1 -- 172.21.15.18:0/4155509773 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d3518260 con 0x5621d35161b0 2024-04-16T17:08:48.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.507+0000 7f8aef400640 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000d770 tx=0x7f8ae000d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a27d92613a3e586 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:48.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.507+0000 7f8ae4c00640 1 -- 172.21.15.18:0/4155509773 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000f040 con 0x5621d35161b0 2024-04-16T17:08:48.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.507+0000 7f8ae4c00640 1 -- 172.21.15.18:0/4155509773 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae0010040 con 0x5621d35161b0 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.507+0000 7f8ae4c00640 1 -- 172.21.15.18:0/4155509773 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0018750 con 0x5621d35161b0 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 -- 172.21.15.18:0/4155509773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 msgr2=0x5621d351edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000d770 tx=0x7f8ae000d850 comp rx=0 tx=0).stop 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 -- 172.21.15.18:0/4155509773 shutdown_connections 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d351f6b0 0x5621d352b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d3516ad0 0x5621d351b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 --2- 172.21.15.18:0/4155509773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d35161b0 0x5621d351edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 -- 172.21.15.18:0/4155509773 >> 172.21.15.18:0/4155509773 conn(0x5621d35108e0 msgr2=0x5621d350f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 -- 172.21.15.18:0/4155509773 shutdown_connections 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 -- 172.21.15.18:0/4155509773 wait complete. 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.508+0000 7f8af9c08740 1 -- start start 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8af9c08740 1 Processor -- start 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d3523210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d351f6b0 0x5621d3523750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8af9c08740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d3523c90 0x5621d3594b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8af9c08740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01b4780 con 0x5621d3523c90 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8af9c08740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01b5bd0 con 0x5621d3516ad0 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8af9c08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01b4e30 con 0x5621d351f6b0 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8af9c08740 10 client.0 _ll_register_callbacks cb 0x7f8adc00a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d351f6b0 0x5621d3523750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d351f6b0 0x5621d3523750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35288/0 (socket says 172.21.15.18:35288) 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8aefe00640 1 -- 172.21.15.18:0/1289812703 learned_addr learned my addr 172.21.15.18:0/1289812703 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.509+0000 7f8aef400640 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d3523210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8aef400640 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d3523c90 0x5621d3594b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8aefe00640 1 -- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 msgr2=0x5621d3523210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8aefe00640 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d3523210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8aefe00640 1 -- 172.21.15.18:0/1289812703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d3523c90 msgr2=0x5621d3594b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8aefe00640 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d3523c90 0x5621d3594b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8aefe00640 1 -- 172.21.15.18:0/1289812703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d35950b0 con 0x5621d351f6b0 2024-04-16T17:08:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8aef400640 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d3523210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8aefe00640 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d351f6b0 0x5621d3523750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5621d351bee0 tx=0x7f8adc00a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dc1377eb941d0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8ada000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8ada000640 1 -- 172.21.15.18:0/1289812703 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc029040 con 0x5621d351f6b0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8ada000640 1 -- 172.21.15.18:0/1289812703 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8adc018a10 con 0x5621d351f6b0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.510+0000 7f8af9c08740 1 -- 172.21.15.18:0/1289812703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae0010040 con 0x5621d351f6b0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8ada000640 1 -- 172.21.15.18:0/1289812703 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8adc03a050 con 0x5621d351f6b0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8af9c08740 10 client.26668 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8af9c08740 1 -- 172.21.15.18:0/1289812703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621d3596290 con 0x5621d351f6b0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8ada000640 1 -- 172.21.15.18:0/1289812703 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae0010040 con 0x5621d351f6b0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8ada000640 1 -- 172.21.15.18:0/1289812703 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8adc00c930 con 0x5621d351f6b0 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8af9c08740 10 client.26668 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8af9c08740 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8af9c08740 20 client.26668 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5621d3596a10 0x5621d3598e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8af9c08740 4 client.26668 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:08:48.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.511+0000 7f8af9c08740 1 -- 172.21.15.18:0/1289812703 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5621d34839b0 con 0x5621d3596a10 2024-04-16T17:08:48.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.512+0000 7f8aef400640 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5621d3596a10 0x5621d3598e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.512+0000 7f8aef400640 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5621d3596a10 0x5621d3598e30 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc68a6ca839dab14 server_cookie=1db1954a74baa725 in_seq=0 out_seq=0 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.512+0000 7f8ada000640 10 client.26668 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8ada000640 1 -- 172.21.15.18:0/1289812703 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5621d34839b0 con 0x5621d3596a10 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8ada000640 10 client.26668 handle_command_reply: tid=0 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 client.26668 shutdown 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 -- 172.21.15.18:0/1289812703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d351f6b0 msgr2=0x5621d3523750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d351f6b0 0x5621d3523750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5621d351bee0 tx=0x7f8adc00a1d0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 -- 172.21.15.18:0/1289812703 shutdown_connections 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5621d3596a10 0x5621d3598e30 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5621d3523c90 0x5621d3594b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5621d351f6b0 0x5621d3523750 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8ada000640 0 client.26668 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 --2- 172.21.15.18:0/1289812703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621d3516ad0 0x5621d3523210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 -- 172.21.15.18:0/1289812703 >> 172.21.15.18:0/1289812703 conn(0x5621d35108e0 msgr2=0x5621d3511d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 -- 172.21.15.18:0/1289812703 shutdown_connections 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.514+0000 7f8af9c08740 1 -- 172.21.15.18:0/1289812703 wait complete. 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af9c08740 20 client.26668 trim_cache size 0 max 16384 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:08:48.518 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:08:48.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 -- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ad40a3ef0 msgr2=0x7f8ad40a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ad40a3ef0 0x7f8ad40a63b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80cf980 tx=0x7f8ae80095c0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 -- 172.21.15.18:0/853159043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8af006c4b0 msgr2=0x7f8af01b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8af006c4b0 0x7f8af01b3f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8037040 tx=0x7f8ae8035d10 comp rx=0 tx=0).stop 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 -- 172.21.15.18:0/853159043 shutdown_connections 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ad40a3ef0 0x7f8ad40a63b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8af006c4b0 0x7f8af01b3f40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af006bb10 0x7f8af006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 --2- 172.21.15.18:0/853159043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af0106310 0x7f8af0073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.515+0000 7f8af7000640 1 -- 172.21.15.18:0/853159043 >> 172.21.15.18:0/853159043 conn(0x7f8af0075da0 msgr2=0x7f8af0110bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.516+0000 7f8af7000640 1 -- 172.21.15.18:0/853159043 shutdown_connections 2024-04-16T17:08:48.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.516+0000 7f8af7000640 1 -- 172.21.15.18:0/853159043 wait complete. 2024-04-16T17:08:48.536 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:08:48.536 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:08:48.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 138378 4.05 MB/sec execute 61 sec latency 60.063 ms 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.716+0000 7efeeee00640 1 -- 172.21.15.18:0/2685613154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8104350 msgr2=0x7efee8104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.716+0000 7efeeee00640 1 --2- 172.21.15.18:0/2685613154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8104350 0x7efee8104730 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efed4009900 tx=0x7efed402bc40 comp rx=0 tx=0).stop 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.717+0000 7efeeee00640 1 -- 172.21.15.18:0/2685613154 shutdown_connections 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.717+0000 7efeeee00640 1 --2- 172.21.15.18:0/2685613154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efee8109420 0x7efee8110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.717+0000 7efeeee00640 1 --2- 172.21.15.18:0/2685613154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8104d00 0x7efee8108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.717+0000 7efeeee00640 1 --2- 172.21.15.18:0/2685613154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8104350 0x7efee8104730 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.717+0000 7efeeee00640 1 -- 172.21.15.18:0/2685613154 >> 172.21.15.18:0/2685613154 conn(0x7efee80ffb70 msgr2=0x7efee8101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.717+0000 7efeeee00640 1 -- 172.21.15.18:0/2685613154 shutdown_connections 2024-04-16T17:08:48.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.717+0000 7efeeee00640 1 -- 172.21.15.18:0/2685613154 wait complete. 2024-04-16T17:08:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.718+0000 7efeeee00640 1 Processor -- start 2024-04-16T17:08:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.718+0000 7efeeee00640 1 -- start start 2024-04-16T17:08:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8104350 0x7efee81a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeee00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8109420 0x7efee81a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeee00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efee819a880 0x7efee819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeee00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efee8114520 con 0x7efee819a880 2024-04-16T17:08:48.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeee00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efee81143a0 con 0x7efee8109420 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efee81146a0 con 0x7efee8104350 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8104350 0x7efee81a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8104350 0x7efee81a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35306/0 (socket says 172.21.15.18:35306) 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeda00640 1 -- 172.21.15.18:0/3914549197 learned_addr learned my addr 172.21.15.18:0/3914549197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeed000640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8109420 0x7efee81a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeee400640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efee819a880 0x7efee819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeda00640 1 -- 172.21.15.18:0/3914549197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8109420 msgr2=0x7efee81a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeda00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8109420 0x7efee81a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeda00640 1 -- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efee819a880 msgr2=0x7efee819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeda00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efee819a880 0x7efee819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeeda00640 1 -- 172.21.15.18:0/3914549197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee819b3a0 con 0x7efee8104350 2024-04-16T17:08:48.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.719+0000 7efeed000640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8109420 0x7efee81a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.720+0000 7efeeda00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8104350 0x7efee81a06d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efed4005ab0 tx=0x7efed4004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.720+0000 7efee6000640 1 -- 172.21.15.18:0/3914549197 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4041070 con 0x7efee8104350 2024-04-16T17:08:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.720+0000 7efee6000640 1 -- 172.21.15.18:0/3914549197 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed402cca0 con 0x7efee8104350 2024-04-16T17:08:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.720+0000 7efee6000640 1 -- 172.21.15.18:0/3914549197 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4003e50 con 0x7efee8104350 2024-04-16T17:08:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.720+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee819b630 con 0x7efee8104350 2024-04-16T17:08:48.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.720+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee81a7820 con 0x7efee8104350 2024-04-16T17:08:48.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.722+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee804fe00 con 0x7efee8104350 2024-04-16T17:08:48.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.730+0000 7efee6000640 1 -- 172.21.15.18:0/3914549197 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efed4044070 con 0x7efee8104350 2024-04-16T17:08:48.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.732+0000 7efee6000640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efed00a3f30 0x7efed00a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:48.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.732+0000 7efee6000640 1 -- 172.21.15.18:0/3914549197 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7efed40f26a0 con 0x7efee8104350 2024-04-16T17:08:48.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.732+0000 7efeed000640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efed00a3f30 0x7efed00a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:48.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.733+0000 7efee6000640 1 -- 172.21.15.18:0/3914549197 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efed40f2b20 con 0x7efee8104350 2024-04-16T17:08:48.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:48.733+0000 7efeed000640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efed00a3f30 0x7efed00a63f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efee810a9d0 tx=0x7efedc005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.032+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efee810c650 con 0x7efee8104350 2024-04-16T17:08:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.033+0000 7efee6000640 1 -- 172.21.15.18:0/3914549197 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7efed40be3f0 con 0x7efee8104350 2024-04-16T17:08:49.034 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:08:49.034 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:08:49.035 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:08:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.036+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efed00a3f30 msgr2=0x7efed00a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.036+0000 7efeeee00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efed00a3f30 0x7efed00a63f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efee810a9d0 tx=0x7efedc005f90 comp rx=0 tx=0).stop 2024-04-16T17:08:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.036+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8104350 msgr2=0x7efee81a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.036+0000 7efeeee00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8104350 0x7efee81a06d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efed4005ab0 tx=0x7efed4004360 comp rx=0 tx=0).stop 2024-04-16T17:08:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.037+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 shutdown_connections 2024-04-16T17:08:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.037+0000 7efeeee00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efed00a3f30 0x7efed00a63f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.037+0000 7efeeee00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efee819a880 0x7efee819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.037+0000 7efeeee00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efee8109420 0x7efee81a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.037+0000 7efeeee00640 1 --2- 172.21.15.18:0/3914549197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8104350 0x7efee81a06d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.037+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 >> 172.21.15.18:0/3914549197 conn(0x7efee80ffb70 msgr2=0x7efee8100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.037+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 shutdown_connections 2024-04-16T17:08:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.037+0000 7efeeee00640 1 -- 172.21.15.18:0/3914549197 wait complete. 2024-04-16T17:08:49.063 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:08:49.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 -- 172.21.15.18:0/580011279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 msgr2=0x7ffbd4102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 --2- 172.21.15.18:0/580011279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 0x7ffbd4102610 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009900 tx=0x7ffbc402bc50 comp rx=0 tx=0).stop 2024-04-16T17:08:49.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 -- 172.21.15.18:0/580011279 shutdown_connections 2024-04-16T17:08:49.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 --2- 172.21.15.18:0/580011279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4107300 0x7ffbd410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 --2- 172.21.15.18:0/580011279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd4102be0 0x7ffbd4106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 --2- 172.21.15.18:0/580011279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 0x7ffbd4102610 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 -- 172.21.15.18:0/580011279 >> 172.21.15.18:0/580011279 conn(0x7ffbd40fdab0 msgr2=0x7ffbd40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 -- 172.21.15.18:0/580011279 shutdown_connections 2024-04-16T17:08:49.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.288+0000 7ffbd9600640 1 -- 172.21.15.18:0/580011279 wait complete. 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.289+0000 7ffbd9600640 1 Processor -- start 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.289+0000 7ffbd9600640 1 -- start start 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 0x7ffbd41a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4107300 0x7ffbd41a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd9600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd419a930 0x7ffbd419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd9600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4112410 con 0x7ffbd419a930 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd9600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4112290 con 0x7ffbd4102230 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4112590 con 0x7ffbd4107300 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 0x7ffbd41a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 0x7ffbd41a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55472/0 (socket says 172.21.15.18:55472) 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2a00640 1 -- 172.21.15.18:0/3842911441 learned_addr learned my addr 172.21.15.18:0/3842911441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd3400640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd419a930 0x7ffbd419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2000640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4107300 0x7ffbd41a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2a00640 1 -- 172.21.15.18:0/3842911441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4107300 msgr2=0x7ffbd41a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2a00640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4107300 0x7ffbd41a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2a00640 1 -- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd419a930 msgr2=0x7ffbd419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2a00640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd419a930 0x7ffbd419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.290+0000 7ffbd2a00640 1 -- 172.21.15.18:0/3842911441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd419b450 con 0x7ffbd4102230 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.291+0000 7ffbd2000640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4107300 0x7ffbd41a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.291+0000 7ffbd2a00640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 0x7ffbd41a0780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005ab0 tx=0x7ffbc4004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.291+0000 7ffbcb400640 1 -- 172.21.15.18:0/3842911441 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4041070 con 0x7ffbd4102230 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.291+0000 7ffbcb400640 1 -- 172.21.15.18:0/3842911441 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc402ccb0 con 0x7ffbd4102230 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.291+0000 7ffbcb400640 1 -- 172.21.15.18:0/3842911441 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4003df0 con 0x7ffbd4102230 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.291+0000 7ffbd9600640 1 -- 172.21.15.18:0/3842911441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd419b6e0 con 0x7ffbd4102230 2024-04-16T17:08:49.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.291+0000 7ffbd9600640 1 -- 172.21.15.18:0/3842911441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd41a78e0 con 0x7ffbd4102230 2024-04-16T17:08:49.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.295+0000 7ffbdc3b5740 1 Processor -- start 2024-04-16T17:08:49.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.296+0000 7ffbdc3b5740 1 -- start start 2024-04-16T17:08:49.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.296+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1902d50 0x555ea1903130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.296+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea190bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.296+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea190c250 0x555ea190e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.296+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555ea1868020 con 0x555ea190c250 2024-04-16T17:08:49.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.296+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555ea180e460 con 0x555ea1902d50 2024-04-16T17:08:49.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.296+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555ea1863fe0 con 0x555ea1903670 2024-04-16T17:08:49.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.296+0000 7ffbcb400640 1 -- 172.21.15.18:0/3842911441 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffbc4044070 con 0x7ffbd4102230 2024-04-16T17:08:49.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1902d50 0x555ea1903130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1902d50 0x555ea1903130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55486/0 (socket says 172.21.15.18:55486) 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 -- 172.21.15.18:0/1102265067 learned_addr learned my addr 172.21.15.18:0/1102265067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd2000640 1 --2- 172.21.15.18:0/1102265067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea190c250 0x555ea190e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 -- 172.21.15.18:0/1102265067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 msgr2=0x555ea190bd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 --2- 172.21.15.18:0/1102265067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea190bd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 -- 172.21.15.18:0/1102265067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea190c250 msgr2=0x555ea190e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 --2- 172.21.15.18:0/1102265067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea190c250 0x555ea190e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 -- 172.21.15.18:0/1102265067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea1816040 con 0x555ea1902d50 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.297+0000 7ffbd3400640 1 --2- 172.21.15.18:0/1102265067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1902d50 0x555ea1903130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009f50 tx=0x7ffbcc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1cffa6a1377c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.298+0000 7ffbbf400640 1 -- 172.21.15.18:0/1102265067 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc004b20 con 0x555ea1902d50 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.298+0000 7ffbbf400640 1 -- 172.21.15.18:0/1102265067 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc004cc0 con 0x555ea1902d50 2024-04-16T17:08:49.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.298+0000 7ffbbf400640 1 -- 172.21.15.18:0/1102265067 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc005490 con 0x555ea1902d50 2024-04-16T17:08:49.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.298+0000 7ffbcb400640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbb80a3e20 0x7ffbb80a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.298+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1102265067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1902d50 msgr2=0x555ea1903130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.298+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1102265067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1902d50 0x555ea1903130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009f50 tx=0x7ffbcc0118d0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbcb400640 1 -- 172.21.15.18:0/3842911441 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ffbc40f2040 con 0x7ffbd4102230 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1102265067 shutdown_connections 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1102265067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea190c250 0x555ea190e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1102265067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea190bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1102265067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1902d50 0x555ea1903130 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1102265067 >> 172.21.15.18:0/1102265067 conn(0x555ea18fd480 msgr2=0x555ea18fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbd2a00640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbb80a3e20 0x7ffbb80a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1102265067 shutdown_connections 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.299+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1102265067 wait complete. 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbdc3b5740 1 -- start start 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbd2a00640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbb80a3e20 0x7ffbb80a62e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x555ea1902820 tx=0x7ffbc4000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbdc3b5740 1 Processor -- start 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1909300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1903670 0x555ea1909840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea190c250 0x555ea1909d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555ea190f040 con 0x555ea1902d50 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555ea1821830 con 0x555ea1903670 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.300+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555ea190f1c0 con 0x555ea190c250 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbdc3b5740 10 client.0 _ll_register_callbacks cb 0x555ea1829150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1903670 0x555ea1909840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1903670 0x555ea1909840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55492/0 (socket says 172.21.15.18:55492) 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd2000640 1 -- 172.21.15.18:0/2876038026 learned_addr learned my addr 172.21.15.18:0/2876038026 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd3400640 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1909300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd2000640 1 -- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea190c250 msgr2=0x555ea1909d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd3400640 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea190c250 0x555ea1909d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd2000640 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea190c250 0x555ea1909d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd2000640 1 -- 172.21.15.18:0/2876038026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 msgr2=0x555ea1909300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd2000640 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1909300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd2000640 1 -- 172.21.15.18:0/2876038026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea1825920 con 0x555ea1903670 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.301+0000 7ffbd3400640 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea190c250 0x555ea1909d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.302+0000 7ffbd2000640 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1903670 0x555ea1909840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffbd406b790 tx=0x7ffbc000bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1b91bbd98f2c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.302+0000 7ffbb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.302+0000 7ffbb3e00640 1 -- 172.21.15.18:0/2876038026 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc001b440 con 0x555ea1903670 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.302+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2876038026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc004cc0 con 0x555ea1903670 2024-04-16T17:08:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.302+0000 7ffbb3e00640 1 -- 172.21.15.18:0/2876038026 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ffbc0011440 con 0x555ea1903670 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.302+0000 7ffbb3e00640 1 -- 172.21.15.18:0/2876038026 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffbc0011640 con 0x555ea1903670 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.302+0000 7ffbdc3b5740 10 client.26708 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.302+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2876038026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ea17f37f0 con 0x555ea1903670 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbb3e00640 1 -- 172.21.15.18:0/2876038026 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc004cc0 con 0x555ea1903670 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbb3e00640 1 -- 172.21.15.18:0/2876038026 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffbc0019020 con 0x555ea1903670 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbdc3b5740 10 client.26708 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbdc3b5740 10 client.26708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbdc3b5740 20 client.26708 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x555ea191c6c0 0x555ea191ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbdc3b5740 4 client.26708 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2876038026 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea1875a40 con 0x555ea191c6c0 2024-04-16T17:08:49.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.303+0000 7ffbd3400640 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x555ea191c6c0 0x555ea191ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.304+0000 7ffbd3400640 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x555ea191c6c0 0x555ea191ead0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e980094003c3767 server_cookie=a577f69a37e0cd6f in_seq=0 out_seq=0 2024-04-16T17:08:49.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.304+0000 7ffbb3e00640 10 client.26708 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:08:49.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbb3e00640 1 -- 172.21.15.18:0/2876038026 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ea1875a40 con 0x555ea191c6c0 2024-04-16T17:08:49.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbb3e00640 10 client.26708 handle_command_reply: tid=0 2024-04-16T17:08:49.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 client.26708 shutdown 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2876038026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1903670 msgr2=0x555ea1909840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1903670 0x555ea1909840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffbd406b790 tx=0x7ffbc000bd90 comp rx=0 tx=0).stop 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2876038026 shutdown_connections 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x555ea191c6c0 0x555ea191ead0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea190c250 0x555ea1909d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea1903670 0x555ea1909840 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2876038026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1909300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2876038026 >> 172.21.15.18:0/2876038026 conn(0x555ea18fd480 msgr2=0x555ea18ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbb3e00640 0 client.26708 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2876038026 shutdown_connections 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2876038026 wait complete. 2024-04-16T17:08:49.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.305+0000 7ffbdc3b5740 20 client.26708 trim_cache size 0 max 16384 2024-04-16T17:08:49.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.321+0000 7ffbdc3b5740 1 Processor -- start 2024-04-16T17:08:49.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.321+0000 7ffbdc3b5740 1 -- start start 2024-04-16T17:08:49.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.321+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea190ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.321+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1917f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.321+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 0x555ea18fc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.321+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b4070 con 0x555ea1902d50 2024-04-16T17:08:49.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.321+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b1940 con 0x555ea190c250 2024-04-16T17:08:49.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.321+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41af790 con 0x555ea1903670 2024-04-16T17:08:49.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea190ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1917f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1917f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35350/0 (socket says 172.21.15.18:35350) 2024-04-16T17:08:49.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea190ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57528/0 (socket says 172.21.15.18:57528) 2024-04-16T17:08:49.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 -- 172.21.15.18:0/2081424487 learned_addr learned my addr 172.21.15.18:0/2081424487 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd3400640 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 0x555ea18fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 -- 172.21.15.18:0/2081424487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 msgr2=0x555ea18fc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 0x555ea18fc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 -- 172.21.15.18:0/2081424487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 msgr2=0x555ea190ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea190ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 -- 172.21.15.18:0/2081424487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea157a3f0 con 0x555ea1903670 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd3400640 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea190ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.322+0000 7ffbd2000640 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1917f60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffbc001bed0 tx=0x7ffbc00059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa7a6296e7d680b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.323+0000 7ffbbea00640 1 -- 172.21.15.18:0/2081424487 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc002c020 con 0x555ea1903670 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.323+0000 7ffbbea00640 1 -- 172.21.15.18:0/2081424487 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc00039c0 con 0x555ea1903670 2024-04-16T17:08:49.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.323+0000 7ffbbea00640 1 -- 172.21.15.18:0/2081424487 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc001ba00 con 0x555ea1903670 2024-04-16T17:08:49.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2081424487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 msgr2=0x555ea1917f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1917f60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffbc001bed0 tx=0x7ffbc00059b0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2081424487 shutdown_connections 2024-04-16T17:08:49.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 0x555ea18fc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1917f60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/2081424487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea190ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2081424487 >> 172.21.15.18:0/2081424487 conn(0x555ea18fd480 msgr2=0x555ea18fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2081424487 shutdown_connections 2024-04-16T17:08:49.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/2081424487 wait complete. 2024-04-16T17:08:49.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.324+0000 7ffbdc3b5740 1 -- start start 2024-04-16T17:08:49.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.325+0000 7ffbdc3b5740 1 Processor -- start 2024-04-16T17:08:49.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.325+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1906050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.325+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1906590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.325+0000 7ffbdc3b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 0x555ea1906ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.325+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41afe00 con 0x555ea1902d50 2024-04-16T17:08:49.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.325+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b1270 con 0x555ea190c250 2024-04-16T17:08:49.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.325+0000 7ffbdc3b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b04d0 con 0x555ea1903670 2024-04-16T17:08:49.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.325+0000 7ffbdc3b5740 10 client.0 _ll_register_callbacks cb 0x7ffbc0004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1906050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 0x555ea1906ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1906590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1906050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57534/0 (socket says 172.21.15.18:57534) 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd3400640 1 -- 172.21.15.18:0/1316527965 learned_addr learned my addr 172.21.15.18:0/1316527965 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd2000640 1 -- 172.21.15.18:0/1316527965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 msgr2=0x555ea1906ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd2000640 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 0x555ea1906ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd2000640 1 -- 172.21.15.18:0/1316527965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 msgr2=0x555ea1906050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd2000640 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1906050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.326+0000 7ffbd2000640 1 -- 172.21.15.18:0/1316527965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea16447d0 con 0x555ea1903670 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbd2000640 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1906590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x555ea191b450 tx=0x7ffbc001bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d478f4b35048504 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:49.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbb3e00640 1 -- 172.21.15.18:0/1316527965 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00119f0 con 0x555ea1903670 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1316527965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc00039c0 con 0x555ea1903670 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbb3e00640 1 -- 172.21.15.18:0/1316527965 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ffbc0011b90 con 0x555ea1903670 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbb3e00640 1 -- 172.21.15.18:0/1316527965 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffbc0011db0 con 0x555ea1903670 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbdc3b5740 10 client.26686 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1316527965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ea1982d30 con 0x555ea1903670 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.327+0000 7ffbb3e00640 1 -- 172.21.15.18:0/1316527965 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc00039c0 con 0x555ea1903670 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.328+0000 7ffbb3e00640 1 -- 172.21.15.18:0/1316527965 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffbc0019040 con 0x555ea1903670 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.328+0000 7ffbdc3b5740 10 client.26686 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.328+0000 7ffbdc3b5740 10 client.26686 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.328+0000 7ffbdc3b5740 20 client.26686 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.328+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x555ea19833f0 0x555ea19857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.328+0000 7ffbdc3b5740 4 client.26686 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.328+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1316527965 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ea186f8b0 con 0x555ea19833f0 2024-04-16T17:08:49.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.328+0000 7ffbd3400640 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x555ea19833f0 0x555ea19857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbd3400640 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x555ea19833f0 0x555ea19857e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=506d86b8cb9af593 server_cookie=645b9dd6e44212b0 in_seq=0 out_seq=0 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbb3e00640 10 client.26686 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbb3e00640 1 -- 172.21.15.18:0/1316527965 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ea186f8b0 con 0x555ea19833f0 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbb3e00640 10 client.26686 handle_command_reply: tid=0 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 client.26686 shutdown 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1316527965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 msgr2=0x555ea1906590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1906590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x555ea191b450 tx=0x7ffbc001bc20 comp rx=0 tx=0).stop 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1316527965 shutdown_connections 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x555ea19833f0 0x555ea19857e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ea190c250 0x555ea1906ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea1903670 0x555ea1906590 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 --2- 172.21.15.18:0/1316527965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ea1902d50 0x555ea1906050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbb3e00640 0 client.26686 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1316527965 >> 172.21.15.18:0/1316527965 conn(0x555ea18fd480 msgr2=0x555ea1909b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1316527965 shutdown_connections 2024-04-16T17:08:49.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.329+0000 7ffbdc3b5740 1 -- 172.21.15.18:0/1316527965 wait complete. 2024-04-16T17:08:49.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbdc3b5740 20 client.26686 trim_cache size 0 max 16384 2024-04-16T17:08:49.333 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 -- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbb80a3e20 msgr2=0x7ffbb80a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbb80a3e20 0x7ffbb80a62e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x555ea1902820 tx=0x7ffbc4000bc0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 -- 172.21.15.18:0/3842911441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 msgr2=0x7ffbd41a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 0x7ffbd41a0780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005ab0 tx=0x7ffbc4004340 comp rx=0 tx=0).stop 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 -- 172.21.15.18:0/3842911441 shutdown_connections 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbb80a3e20 0x7ffbb80a62e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd419a930 0x7ffbd419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4107300 0x7ffbd41a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 --2- 172.21.15.18:0/3842911441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4102230 0x7ffbd41a0780 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 -- 172.21.15.18:0/3842911441 >> 172.21.15.18:0/3842911441 conn(0x7ffbd40fdab0 msgr2=0x7ffbd40ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 -- 172.21.15.18:0/3842911441 shutdown_connections 2024-04-16T17:08:49.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.330+0000 7ffbd9600640 1 -- 172.21.15.18:0/3842911441 wait complete. 2024-04-16T17:08:49.348 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:08:49.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3172431263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0e80743a0 msgr2=0x7fe0e8074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3172431263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0e80743a0 0x7fe0e8074780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009900 tx=0x7fe0d802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3172431263 shutdown_connections 2024-04-16T17:08:49.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3172431263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e8112e10 0x7fe0e8115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3172431263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e8074cc0 0x7fe0e81128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3172431263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0e80743a0 0x7fe0e8074780 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3172431263 >> 172.21.15.18:0/3172431263 conn(0x7fe0e80fd950 msgr2=0x7fe0e80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3172431263 shutdown_connections 2024-04-16T17:08:49.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.531+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3172431263 wait complete. 2024-04-16T17:08:49.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.532+0000 7fe0e7e00640 1 Processor -- start 2024-04-16T17:08:49.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.532+0000 7fe0e7e00640 1 -- start start 2024-04-16T17:08:49.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0e80743a0 0x7fe0e819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e8074cc0 0x7fe0e819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e8112e10 0x7fe0e81a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e7e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8118850 con 0x7fe0e80743a0 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e7e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81186d0 con 0x7fe0e8074cc0 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81189d0 con 0x7fe0e8112e10 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0e80743a0 0x7fe0e819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e8074cc0 0x7fe0e819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e8112e10 0x7fe0e81a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e8074cc0 0x7fe0e819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55512/0 (socket says 172.21.15.18:55512) 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e8112e10 0x7fe0e81a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35372/0 (socket says 172.21.15.18:35372) 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.533+0000 7fe0e6000640 1 -- 172.21.15.18:0/3432301445 learned_addr learned my addr 172.21.15.18:0/3432301445 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0e6000640 1 -- 172.21.15.18:0/3432301445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e8112e10 msgr2=0x7fe0e81a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0e6000640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e8112e10 0x7fe0e81a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0e6000640 1 -- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0e80743a0 msgr2=0x7fe0e819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0e6000640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0e80743a0 0x7fe0e819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0e6000640 1 -- 172.21.15.18:0/3432301445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e81a3da0 con 0x7fe0e8074cc0 2024-04-16T17:08:49.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0e6000640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e8074cc0 0x7fe0e819f2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400e9f0 tx=0x7fe0d400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0eca00640 1 -- 172.21.15.18:0/3432301445 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400ccc0 con 0x7fe0e8074cc0 2024-04-16T17:08:49.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3432301445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e81a4090 con 0x7fe0e8074cc0 2024-04-16T17:08:49.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0eca00640 1 -- 172.21.15.18:0/3432301445 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d400ce60 con 0x7fe0e8074cc0 2024-04-16T17:08:49.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3432301445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81ab920 con 0x7fe0e8074cc0 2024-04-16T17:08:49.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.534+0000 7fe0eca00640 1 -- 172.21.15.18:0/3432301445 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4010430 con 0x7fe0e8074cc0 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.537+0000 7fe0f0bd8740 1 Processor -- start 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.537+0000 7fe0eca00640 1 -- 172.21.15.18:0/3432301445 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe0d40046c0 con 0x7fe0e8074cc0 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0f0bd8740 1 -- start start 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad289f10 0x55e7ad28a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad292ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 0x55e7ad295800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e7ad1ef020 con 0x55e7ad28a830 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e7ad195460 con 0x55e7ad293410 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e7ad1eafe0 con 0x55e7ad289f10 2024-04-16T17:08:49.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0e6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad292ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0e6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad292ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57546/0 (socket says 172.21.15.18:57546) 2024-04-16T17:08:49.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad289f10 0x55e7ad28a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.538+0000 7fe0e6a00640 1 -- 172.21.15.18:0/2631467211 learned_addr learned my addr 172.21.15.18:0/2631467211 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0e6000640 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 0x55e7ad295800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0e6a00640 1 -- 172.21.15.18:0/2631467211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad289f10 msgr2=0x55e7ad28a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad289f10 0x55e7ad28a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0e6a00640 1 -- 172.21.15.18:0/2631467211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 msgr2=0x55e7ad295800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 0x55e7ad295800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0e6a00640 1 -- 172.21.15.18:0/2631467211 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ad19d040 con 0x55e7ad28a830 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0e7400640 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad289f10 0x55e7ad28a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0eca00640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.539+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad292ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8000c00 tx=0x7fe0d8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1166415a5dd019 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.540+0000 7fe0d3e00640 1 -- 172.21.15.18:0/2631467211 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8043070 con 0x55e7ad28a830 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.540+0000 7fe0eca00640 1 -- 172.21.15.18:0/3432301445 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe0d40d0f30 con 0x7fe0e8074cc0 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.540+0000 7fe0d3e00640 1 -- 172.21.15.18:0/2631467211 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d802c6f0 con 0x55e7ad28a830 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.540+0000 7fe0d3e00640 1 -- 172.21.15.18:0/2631467211 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8004a00 con 0x55e7ad28a830 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.540+0000 7fe0e7400640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.540+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2631467211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 msgr2=0x55e7ad292ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.540+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad292ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8000c00 tx=0x7fe0d8009560 comp rx=0 tx=0).stop 2024-04-16T17:08:49.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2631467211 shutdown_connections 2024-04-16T17:08:49.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 0x55e7ad295800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad292ed0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2631467211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad289f10 0x55e7ad28a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2631467211 >> 172.21.15.18:0/2631467211 conn(0x55e7ad284640 msgr2=0x55e7ad284a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2631467211 shutdown_connections 2024-04-16T17:08:49.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0e7400640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe0e806b470 tx=0x7fe0e0009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2631467211 wait complete. 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.541+0000 7fe0f0bd8740 1 -- start start 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.542+0000 7fe0f0bd8740 1 Processor -- start 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.542+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad290370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.542+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad28a830 0x55e7ad2908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.542+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad290df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.542+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e7ad2961a0 con 0x55e7ad289f10 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e7ad1a8830 con 0x55e7ad28a830 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e7ad296320 con 0x55e7ad293410 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0f0bd8740 10 client.0 _ll_register_callbacks cb 0x55e7ad1b0150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad290370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad290370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57550/0 (socket says 172.21.15.18:57550) 2024-04-16T17:08:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6a00640 1 -- 172.21.15.18:0/2796259598 learned_addr learned my addr 172.21.15.18:0/2796259598 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6000640 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad28a830 0x55e7ad2908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e7400640 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad290df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6a00640 1 -- 172.21.15.18:0/2796259598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 msgr2=0x55e7ad290df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad290df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6a00640 1 -- 172.21.15.18:0/2796259598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad28a830 msgr2=0x55e7ad2908b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad28a830 0x55e7ad2908b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.543+0000 7fe0e6a00640 1 -- 172.21.15.18:0/2796259598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ad1ac920 con 0x55e7ad289f10 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.544+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad290370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8006010 tx=0x7fe0d80058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12935a32c5e9c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.544+0000 7fe0d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.544+0000 7fe0d0c00640 1 -- 172.21.15.18:0/2796259598 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8043050 con 0x55e7ad289f10 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.544+0000 7fe0d0c00640 1 -- 172.21.15.18:0/2796259598 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe0d8004ee0 con 0x55e7ad289f10 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.544+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2796259598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0d802c6f0 con 0x55e7ad289f10 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.544+0000 7fe0d0c00640 1 -- 172.21.15.18:0/2796259598 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0d8058050 con 0x55e7ad289f10 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.544+0000 7fe0f0bd8740 10 client.16536 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.544+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2796259598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e7ad17a7f0 con 0x55e7ad289f10 2024-04-16T17:08:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0d0c00640 1 -- 172.21.15.18:0/2796259598 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d802c6f0 con 0x55e7ad289f10 2024-04-16T17:08:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0d0c00640 1 -- 172.21.15.18:0/2796259598 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe0d8048b10 con 0x55e7ad289f10 2024-04-16T17:08:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0f0bd8740 10 client.16536 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0f0bd8740 10 client.16536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:08:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0f0bd8740 20 client.16536 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e7ad2a3550 0x55e7ad2a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0f0bd8740 4 client.16536 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2796259598 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ad1fca40 con 0x55e7ad2a3550 2024-04-16T17:08:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.545+0000 7fe0e6000640 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e7ad2a3550 0x55e7ad2a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.549 INFO:tasks.workunit.client.0.smithi018.stdout: 1 138657 4.00 MB/sec execute 62 sec latency 52.263 ms 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.546+0000 7fe0e6000640 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e7ad2a3550 0x55e7ad2a5940 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddd56b5855a2c51e server_cookie=9a0920f332828a6f in_seq=0 out_seq=0 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.546+0000 7fe0d0c00640 10 client.16536 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0d0c00640 1 -- 172.21.15.18:0/2796259598 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7ad1fca40 con 0x55e7ad2a3550 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0d0c00640 10 client.16536 handle_command_reply: tid=0 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 client.16536 shutdown 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2796259598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 msgr2=0x55e7ad290370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad290370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8006010 tx=0x7fe0d80058f0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2796259598 shutdown_connections 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e7ad2a3550 0x55e7ad2a5940 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad290df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad28a830 0x55e7ad2908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/2796259598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad290370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2796259598 >> 172.21.15.18:0/2796259598 conn(0x55e7ad284640 msgr2=0x55e7ad286390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0d0c00640 0 client.16536 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:08:49.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2796259598 shutdown_connections 2024-04-16T17:08:49.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.547+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/2796259598 wait complete. 2024-04-16T17:08:49.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.548+0000 7fe0f0bd8740 20 client.16536 trim_cache size 0 max 16384 2024-04-16T17:08:49.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0f0bd8740 1 Processor -- start 2024-04-16T17:08:49.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0f0bd8740 1 -- start start 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad292aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad28a830 0x55e7ad29f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 0x55e7ad283580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b81b0 con 0x55e7ad289f10 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b5a80 con 0x55e7ad293410 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b38d0 con 0x55e7ad28a830 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.574+0000 7fe0e6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad292aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad292aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57556/0 (socket says 172.21.15.18:57556) 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6a00640 1 -- 172.21.15.18:0/3835514229 learned_addr learned my addr 172.21.15.18:0/3835514229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e7400640 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 0x55e7ad283580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6000640 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad28a830 0x55e7ad29f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6a00640 1 -- 172.21.15.18:0/3835514229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad28a830 msgr2=0x55e7ad29f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad28a830 0x55e7ad29f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6a00640 1 -- 172.21.15.18:0/3835514229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 msgr2=0x55e7ad283580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 0x55e7ad283580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6a00640 1 -- 172.21.15.18:0/3835514229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7acf013f0 con 0x55e7ad289f10 2024-04-16T17:08:49.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.575+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad292aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009900 tx=0x7fe0d8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2915c59804689361 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0d3400640 1 -- 172.21.15.18:0/3835514229 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8043070 con 0x55e7ad289f10 2024-04-16T17:08:49.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0d3400640 1 -- 172.21.15.18:0/3835514229 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d803fe40 con 0x55e7ad289f10 2024-04-16T17:08:49.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0d3400640 1 -- 172.21.15.18:0/3835514229 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8005140 con 0x55e7ad289f10 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/3835514229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 msgr2=0x55e7ad292aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad292aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009900 tx=0x7fe0d8005a20 comp rx=0 tx=0).stop 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/3835514229 shutdown_connections 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad293410 0x55e7ad283580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad28a830 0x55e7ad29f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/3835514229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad289f10 0x55e7ad292aa0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.576+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/3835514229 >> 172.21.15.18:0/3835514229 conn(0x55e7ad284640 msgr2=0x55e7ad282e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.577+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/3835514229 shutdown_connections 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.577+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/3835514229 wait complete. 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.577+0000 7fe0f0bd8740 1 -- start start 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.577+0000 7fe0f0bd8740 1 Processor -- start 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad289f10 0x55e7ad28d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad28d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0f0bd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad28dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b3f40 con 0x55e7ad28a830 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b53b0 con 0x55e7ad289f10 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0f0bd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b4610 con 0x55e7ad293410 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0f0bd8740 10 client.0 _ll_register_callbacks cb 0x7fe0d40d4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad28d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad28d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57564/0 (socket says 172.21.15.18:57564) 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6000640 1 -- 172.21.15.18:0/1069322743 learned_addr learned my addr 172.21.15.18:0/1069322743 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad289f10 0x55e7ad28d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e7400640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad28dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6000640 1 -- 172.21.15.18:0/1069322743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 msgr2=0x55e7ad28dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6000640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad28dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6000640 1 -- 172.21.15.18:0/1069322743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad289f10 msgr2=0x55e7ad28d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6000640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad289f10 0x55e7ad28d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6000640 1 -- 172.21.15.18:0/1069322743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7acfcb7d0 con 0x55e7ad28a830 2024-04-16T17:08:49.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad289f10 0x55e7ad28d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:49.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.578+0000 7fe0e7400640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad28dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:49.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0e6000640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad28d7b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40ce380 tx=0x7fe0d40bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7029d299d3062ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:49.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0d0c00640 1 -- 172.21.15.18:0/1069322743 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d409d220 con 0x55e7ad28a830 2024-04-16T17:08:49.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0d0c00640 1 -- 172.21.15.18:0/1069322743 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe0d40ce550 con 0x55e7ad28a830 2024-04-16T17:08:49.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/1069322743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e7ad1f68b0 con 0x55e7ad28a830 2024-04-16T17:08:49.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0d0c00640 1 -- 172.21.15.18:0/1069322743 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0d40ac050 con 0x55e7ad28a830 2024-04-16T17:08:49.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0f0bd8740 10 client.16542 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/1069322743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e7ad309c40 con 0x55e7ad28a830 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.579+0000 7fe0d0c00640 1 -- 172.21.15.18:0/1069322743 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e7ad1f68b0 con 0x55e7ad28a830 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0d0c00640 1 -- 172.21.15.18:0/1069322743 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe0d40bbce0 con 0x55e7ad28a830 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0f0bd8740 10 client.16542 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0f0bd8740 10 client.16542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0f0bd8740 20 client.16542 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e7ad30a320 0x55e7ad30c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0f0bd8740 4 client.16542 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/1069322743 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7ad172570 con 0x55e7ad30a320 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e7ad30a320 0x55e7ad30c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0e6a00640 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e7ad30a320 0x55e7ad30c740 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb4d93b2f704abc5 server_cookie=d8d7c7211948e174 in_seq=0 out_seq=0 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.580+0000 7fe0d0c00640 10 client.16542 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0d0c00640 1 -- 172.21.15.18:0/1069322743 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7ad172570 con 0x55e7ad30a320 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0d0c00640 10 client.16542 handle_command_reply: tid=0 2024-04-16T17:08:49.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 client.16542 shutdown 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/1069322743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 msgr2=0x55e7ad28d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad28d7b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40ce380 tx=0x7fe0d40bd040 comp rx=0 tx=0).stop 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/1069322743 shutdown_connections 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e7ad30a320 0x55e7ad30c740 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7ad293410 0x55e7ad28dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7ad28a830 0x55e7ad28d7b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 --2- 172.21.15.18:0/1069322743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7ad289f10 0x55e7ad28d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0d0c00640 0 client.16542 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:08:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/1069322743 >> 172.21.15.18:0/1069322743 conn(0x55e7ad284640 msgr2=0x55e7ad283700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/1069322743 shutdown_connections 2024-04-16T17:08:49.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 1 -- 172.21.15.18:0/1069322743 wait complete. 2024-04-16T17:08:49.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0f0bd8740 20 client.16542 trim_cache size 0 max 16384 2024-04-16T17:08:49.588 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:08:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0cc0a3da0 msgr2=0x7fe0cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe0e806b470 tx=0x7fe0e0009500 comp rx=0 tx=0).stop 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3432301445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e8074cc0 msgr2=0x7fe0e819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e8074cc0 0x7fe0e819f2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400e9f0 tx=0x7fe0d400edf0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3432301445 shutdown_connections 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e8112e10 0x7fe0e81a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e8074cc0 0x7fe0e819f2f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 --2- 172.21.15.18:0/3432301445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0e80743a0 0x7fe0e819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3432301445 >> 172.21.15.18:0/3432301445 conn(0x7fe0e80fd950 msgr2=0x7fe0e80fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3432301445 shutdown_connections 2024-04-16T17:08:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.581+0000 7fe0e7e00640 1 -- 172.21.15.18:0/3432301445 wait complete. 2024-04-16T17:08:49.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:08:49.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:49 smithi022 ceph-mon[36023]: pgmap v125: 129 pgs: 129 active+clean; 37 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1008 KiB/s wr, 130 op/s 2024-04-16T17:08:49.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:49 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3914549197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:49.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:49 smithi043 ceph-mon[36901]: pgmap v125: 129 pgs: 129 active+clean; 37 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1008 KiB/s wr, 130 op/s 2024-04-16T17:08:49.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:49 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3914549197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 -- 172.21.15.18:0/3951635203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0170102a20 msgr2=0x7f0170106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 --2- 172.21.15.18:0/3951635203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0170102a20 0x7f0170106a60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f015c009860 tx=0x7f015c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:08:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 -- 172.21.15.18:0/3951635203 shutdown_connections 2024-04-16T17:08:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 --2- 172.21.15.18:0/3951635203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0170107140 0x7f017010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 --2- 172.21.15.18:0/3951635203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0170102a20 0x7f0170106a60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 --2- 172.21.15.18:0/3951635203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0170102070 0x7f0170102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 -- 172.21.15.18:0/3951635203 >> 172.21.15.18:0/3951635203 conn(0x7f01700fd950 msgr2=0x7f01700ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 -- 172.21.15.18:0/3951635203 shutdown_connections 2024-04-16T17:08:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.783+0000 7f0174a00640 1 -- 172.21.15.18:0/3951635203 wait complete. 2024-04-16T17:08:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.784+0000 7f0174a00640 1 Processor -- start 2024-04-16T17:08:49.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f0174a00640 1 -- start start 2024-04-16T17:08:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f0174a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0170102070 0x7f01701a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f0174a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0170107140 0x7f01701a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f0174a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017019a790 0x7f017019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f0174a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01701121b0 con 0x7f0170102070 2024-04-16T17:08:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f0174a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0170112030 con 0x7f0170107140 2024-04-16T17:08:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f0174a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0170112330 con 0x7f017019a790 2024-04-16T17:08:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f016ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0170107140 0x7f01701a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f016ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0170107140 0x7f01701a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55564/0 (socket says 172.21.15.18:55564) 2024-04-16T17:08:49.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.785+0000 7f016ea00640 1 -- 172.21.15.18:0/3158547463 learned_addr learned my addr 172.21.15.18:0/3158547463 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016fe00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017019a790 0x7f017019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016f400640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0170102070 0x7f01701a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016ea00640 1 -- 172.21.15.18:0/3158547463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017019a790 msgr2=0x7f017019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016ea00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017019a790 0x7f017019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016ea00640 1 -- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0170102070 msgr2=0x7f01701a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016ea00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0170102070 0x7f01701a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016ea00640 1 -- 172.21.15.18:0/3158547463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017019b320 con 0x7f0170107140 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016f400640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0170102070 0x7f01701a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f016ea00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0170107140 0x7f01701a0b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f015c037040 tx=0x7f015c004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f0167e00640 1 -- 172.21.15.18:0/3158547463 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c041020 con 0x7f0170107140 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f0167e00640 1 -- 172.21.15.18:0/3158547463 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f015c02c9f0 con 0x7f0170107140 2024-04-16T17:08:49.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f0174a00640 1 -- 172.21.15.18:0/3158547463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017019b5b0 con 0x7f0170107140 2024-04-16T17:08:49.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.786+0000 7f0167e00640 1 -- 172.21.15.18:0/3158547463 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c039040 con 0x7f0170107140 2024-04-16T17:08:49.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.787+0000 7f0174a00640 1 -- 172.21.15.18:0/3158547463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01701a7760 con 0x7f0170107140 2024-04-16T17:08:49.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0178dbe740 1 Processor -- start 2024-04-16T17:08:49.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0178dbe740 1 -- start start 2024-04-16T17:08:49.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0167e00640 1 -- 172.21.15.18:0/3158547463 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f015c044020 con 0x7f0170107140 2024-04-16T17:08:49.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afd52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0178dbe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f33af3a020 con 0x55f33afd5830 2024-04-16T17:08:49.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0178dbe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f33aee0460 con 0x55f33afde410 2024-04-16T17:08:49.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.791+0000 7f0178dbe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f33af35fe0 con 0x55f33afd4f10 2024-04-16T17:08:49.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afd52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afd52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35432/0 (socket says 172.21.15.18:35432) 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016fe00640 1 -- 172.21.15.18:0/3109147531 learned_addr learned my addr 172.21.15.18:0/3109147531 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016f400640 1 --2- 172.21.15.18:0/3109147531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016ea00640 1 -- 172.21.15.18:0/3109147531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 msgr2=0x55f33afd52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016ea00640 1 --2- 172.21.15.18:0/3109147531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afd52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016ea00640 1 -- 172.21.15.18:0/3109147531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 msgr2=0x55f33afdded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016ea00640 1 --2- 172.21.15.18:0/3109147531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.792+0000 7f016ea00640 1 -- 172.21.15.18:0/3109147531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33aee8040 con 0x55f33afde410 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.793+0000 7f016ea00640 1 --2- 172.21.15.18:0/3109147531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe0800 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f015c0ef7d0 tx=0x7f015c0fa450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a43e13333dc7dec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.793+0000 7f015be00640 1 -- 172.21.15.18:0/3109147531 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c0fae60 con 0x55f33afde410 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.793+0000 7f015be00640 1 -- 172.21.15.18:0/3109147531 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f015c0ff030 con 0x55f33afde410 2024-04-16T17:08:49.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.793+0000 7f015be00640 1 -- 172.21.15.18:0/3109147531 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c104750 con 0x55f33afde410 2024-04-16T17:08:49.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0167e00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f01540a3d20 0x7f01540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0178dbe740 1 -- 172.21.15.18:0/3109147531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 msgr2=0x55f33afe0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0178dbe740 1 --2- 172.21.15.18:0/3109147531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe0800 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f015c0ef7d0 tx=0x7f015c0fa450 comp rx=0 tx=0).stop 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0167e00640 1 -- 172.21.15.18:0/3158547463 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f015c0f2320 con 0x7f0170107140 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f016fe00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f01540a3d20 0x7f01540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0178dbe740 1 -- 172.21.15.18:0/3109147531 shutdown_connections 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0178dbe740 1 --2- 172.21.15.18:0/3109147531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe0800 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0178dbe740 1 --2- 172.21.15.18:0/3109147531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0178dbe740 1 --2- 172.21.15.18:0/3109147531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afd52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0178dbe740 1 -- 172.21.15.18:0/3109147531 >> 172.21.15.18:0/3109147531 conn(0x55f33afcf640 msgr2=0x55f33afcfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.794+0000 7f0178dbe740 1 -- 172.21.15.18:0/3109147531 shutdown_connections 2024-04-16T17:08:49.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.795+0000 7f0178dbe740 1 -- 172.21.15.18:0/3109147531 wait complete. 2024-04-16T17:08:49.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.795+0000 7f0178dbe740 1 -- start start 2024-04-16T17:08:49.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.795+0000 7f0178dbe740 1 Processor -- start 2024-04-16T17:08:49.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f016fe00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f01540a3d20 0x7f01540a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0168012640 tx=0x7f0168009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afdb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afdc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f0178dbe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f33afe0fc0 con 0x55f33afd5830 2024-04-16T17:08:49.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f0178dbe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f33aef3830 con 0x55f33afde410 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f0178dbe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f33afe1140 con 0x55f33afd4f10 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f016fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afdc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f016fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afdc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55568/0 (socket says 172.21.15.18:55568) 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f016fe00640 1 -- 172.21.15.18:0/2914336829 learned_addr learned my addr 172.21.15.18:0/2914336829 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.796+0000 7f016f400640 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afdb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f0178dbe740 10 client.0 _ll_register_callbacks cb 0x55f33aefb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f016fe00640 1 -- 172.21.15.18:0/2914336829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 msgr2=0x55f33afdb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f016fe00640 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afdb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f016ea00640 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f016fe00640 1 -- 172.21.15.18:0/2914336829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 msgr2=0x55f33afdbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f016fe00640 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f016fe00640 1 -- 172.21.15.18:0/2914336829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33aef7920 con 0x55f33afde410 2024-04-16T17:08:49.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f016ea00640 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f016fe00640 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afdc090 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0168003f00 tx=0x7f0168004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182afad7fac964e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f0158c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.797+0000 7f0158c00640 1 -- 172.21.15.18:0/2914336829 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016800f8f0 con 0x55f33afde410 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.798+0000 7f0158c00640 1 -- 172.21.15.18:0/2914336829 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f016800fa90 con 0x55f33afde410 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.798+0000 7f0178dbe740 1 -- 172.21.15.18:0/2914336829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015c0ff030 con 0x55f33afde410 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.798+0000 7f0158c00640 1 -- 172.21.15.18:0/2914336829 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f016800fc90 con 0x55f33afde410 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.798+0000 7f0178dbe740 10 client.26744 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.798+0000 7f0178dbe740 1 -- 172.21.15.18:0/2914336829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f33aec57f0 con 0x55f33afde410 2024-04-16T17:08:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.798+0000 7f0158c00640 1 -- 172.21.15.18:0/2914336829 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f015c0ff030 con 0x55f33afde410 2024-04-16T17:08:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.799+0000 7f0158c00640 1 -- 172.21.15.18:0/2914336829 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0168027070 con 0x55f33afde410 2024-04-16T17:08:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.799+0000 7f0178dbe740 10 client.26744 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.799+0000 7f0178dbe740 10 client.26744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.799+0000 7f0178dbe740 20 client.26744 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.799+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f33afee760 0x55f33aff0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.799+0000 7f0178dbe740 4 client.26744 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.799+0000 7f0178dbe740 1 -- 172.21.15.18:0/2914336829 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f33af47a40 con 0x55f33afee760 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.799+0000 7f016f400640 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f33afee760 0x55f33aff0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.800+0000 7f016f400640 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f33afee760 0x55f33aff0bc0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1197673d0f986254 server_cookie=eae2bc806c0401b7 in_seq=0 out_seq=0 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.800+0000 7f0158c00640 10 client.26744 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0158c00640 1 -- 172.21.15.18:0/2914336829 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f33af47a40 con 0x55f33afee760 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0158c00640 10 client.26744 handle_command_reply: tid=0 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 client.26744 shutdown 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 -- 172.21.15.18:0/2914336829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 msgr2=0x55f33afdc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afdc090 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0168003f00 tx=0x7f0168004ab0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 -- 172.21.15.18:0/2914336829 shutdown_connections 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f33afee760 0x55f33aff0bc0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afdc090 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afdbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0158c00640 0 client.26744 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2914336829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afdb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 -- 172.21.15.18:0/2914336829 >> 172.21.15.18:0/2914336829 conn(0x55f33afcf640 msgr2=0x55f33afd1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 -- 172.21.15.18:0/2914336829 shutdown_connections 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 1 -- 172.21.15.18:0/2914336829 wait complete. 2024-04-16T17:08:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.801+0000 7f0178dbe740 20 client.26744 trim_cache size 0 max 16384 2024-04-16T17:08:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f0178dbe740 1 Processor -- start 2024-04-16T17:08:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f0178dbe740 1 -- start start 2024-04-16T17:08:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd4f10 0x55f33afddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd5830 0x55f33afd5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f0178dbe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01701b3e10 con 0x55f33afd4f10 2024-04-16T17:08:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f0178dbe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01701b16e0 con 0x55f33afde410 2024-04-16T17:08:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f0178dbe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01701af530 con 0x55f33afd5830 2024-04-16T17:08:49.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f016fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.818+0000 7f016ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd5830 0x55f33afd5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f016fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55576/0 (socket says 172.21.15.18:55576) 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f016fe00640 1 -- 172.21.15.18:0/799205805 learned_addr learned my addr 172.21.15.18:0/799205805 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f016fe00640 1 -- 172.21.15.18:0/799205805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd5830 msgr2=0x55f33afd5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f016fe00640 1 --2- 172.21.15.18:0/799205805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd5830 0x55f33afd5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f016fe00640 1 -- 172.21.15.18:0/799205805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd4f10 msgr2=0x55f33afddd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f016fe00640 1 --2- 172.21.15.18:0/799205805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd4f10 0x55f33afddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f016fe00640 1 -- 172.21.15.18:0/799205805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33afda5c0 con 0x55f33afde410 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f016fe00640 1 --2- 172.21.15.18:0/799205805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe9c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0168019040 tx=0x7f016801d480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da1c230572360c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.819+0000 7f015b400640 1 -- 172.21.15.18:0/799205805 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016800faf0 con 0x55f33afde410 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.820+0000 7f015b400640 1 -- 172.21.15.18:0/799205805 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f016801d7e0 con 0x55f33afde410 2024-04-16T17:08:49.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.820+0000 7f015b400640 1 -- 172.21.15.18:0/799205805 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680296a0 con 0x55f33afde410 2024-04-16T17:08:49.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.820+0000 7f0178dbe740 1 -- 172.21.15.18:0/799205805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 msgr2=0x55f33afe9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 --2- 172.21.15.18:0/799205805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe9c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0168019040 tx=0x7f016801d480 comp rx=0 tx=0).stop 2024-04-16T17:08:49.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 -- 172.21.15.18:0/799205805 shutdown_connections 2024-04-16T17:08:49.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 --2- 172.21.15.18:0/799205805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe9c90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 --2- 172.21.15.18:0/799205805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd5830 0x55f33afd5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 --2- 172.21.15.18:0/799205805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd4f10 0x55f33afddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 -- 172.21.15.18:0/799205805 >> 172.21.15.18:0/799205805 conn(0x55f33afcf640 msgr2=0x55f33afdbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 -- 172.21.15.18:0/799205805 shutdown_connections 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 -- 172.21.15.18:0/799205805 wait complete. 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.821+0000 7f0178dbe740 1 -- start start 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.822+0000 7f0178dbe740 1 Processor -- start 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.822+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afe6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.822+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afe3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.822+0000 7f0178dbe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.822+0000 7f0178dbe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01701afba0 con 0x55f33afd5830 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.822+0000 7f0178dbe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01701b1010 con 0x55f33afde410 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.822+0000 7f0178dbe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01701b0270 con 0x55f33afd4f10 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.822+0000 7f0178dbe740 10 client.0 _ll_register_callbacks cb 0x7f0168009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afe3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afe6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afe3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57626/0 (socket says 172.21.15.18:57626) 2024-04-16T17:08:49.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 -- 172.21.15.18:0/2643794341 learned_addr learned my addr 172.21.15.18:0/2643794341 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:49.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016fe00640 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 -- 172.21.15.18:0/2643794341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 msgr2=0x55f33afe6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afe6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 -- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 msgr2=0x55f33afe36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 -- 172.21.15.18:0/2643794341 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33ac4c3f0 con 0x55f33afd5830 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016fe00640 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f016ea00640 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afe3170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f015c0ef880 tx=0x7f015c0f25f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df800ed522b969c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.823+0000 7f0158c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.824+0000 7f0158c00640 1 -- 172.21.15.18:0/2643794341 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c0ba2c0 con 0x55f33afd5830 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.824+0000 7f0158c00640 1 -- 172.21.15.18:0/2643794341 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f015c0347d0 con 0x55f33afd5830 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.824+0000 7f0178dbe740 1 -- 172.21.15.18:0/2643794341 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f016801d7e0 con 0x55f33afd5830 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.824+0000 7f0158c00640 1 -- 172.21.15.18:0/2643794341 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f015c0bd980 con 0x55f33afd5830 2024-04-16T17:08:49.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.824+0000 7f0178dbe740 10 client.16554 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.824+0000 7f0178dbe740 1 -- 172.21.15.18:0/2643794341 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f33ad167d0 con 0x55f33afd5830 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.824+0000 7f0158c00640 1 -- 172.21.15.18:0/2643794341 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f016801d7e0 con 0x55f33afd5830 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.825+0000 7f0158c00640 1 -- 172.21.15.18:0/2643794341 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f015c03c030 con 0x55f33afd5830 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.825+0000 7f0178dbe740 10 client.16554 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.825+0000 7f0178dbe740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.825+0000 7f0178dbe740 20 client.16554 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.825+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f33b055cb0 0x55f33b0580d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.825+0000 7f0178dbe740 4 client.16554 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.825+0000 7f0178dbe740 1 -- 172.21.15.18:0/2643794341 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f33af418b0 con 0x55f33b055cb0 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.825+0000 7f016f400640 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f33b055cb0 0x55f33b0580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.826+0000 7f016f400640 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f33b055cb0 0x55f33b0580d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157e16cabc1ccf61 server_cookie=6620f05160809b6a in_seq=0 out_seq=0 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.826+0000 7f0158c00640 10 client.16554 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.826+0000 7f0158c00640 1 -- 172.21.15.18:0/2643794341 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f33af418b0 con 0x55f33b055cb0 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.826+0000 7f0158c00640 10 client.16554 handle_command_reply: tid=0 2024-04-16T17:08:49.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.826+0000 7f0178dbe740 1 client.16554 shutdown 2024-04-16T17:08:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.826+0000 7f0178dbe740 1 -- 172.21.15.18:0/2643794341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 msgr2=0x55f33afe3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.826+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afe3170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f015c0ef880 tx=0x7f015c0f25f0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 1 -- 172.21.15.18:0/2643794341 shutdown_connections 2024-04-16T17:08:49.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f33b055cb0 0x55f33b0580d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f33afde410 0x55f33afe36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f33afd5830 0x55f33afe3170 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0158c00640 0 client.16554 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 1 --2- 172.21.15.18:0/2643794341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f33afd4f10 0x55f33afe6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 1 -- 172.21.15.18:0/2643794341 >> 172.21.15.18:0/2643794341 conn(0x55f33afcf640 msgr2=0x55f33afd09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 1 -- 172.21.15.18:0/2643794341 shutdown_connections 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 1 -- 172.21.15.18:0/2643794341 wait complete. 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.827+0000 7f0178dbe740 20 client.16554 trim_cache size 0 max 16384 2024-04-16T17:08:49.830 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:08:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 -- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f01540a3d20 msgr2=0x7f01540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f01540a3d20 0x7f01540a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0168012640 tx=0x7f0168009500 comp rx=0 tx=0).stop 2024-04-16T17:08:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 -- 172.21.15.18:0/3158547463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0170107140 msgr2=0x7f01701a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0170107140 0x7f01701a0b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f015c037040 tx=0x7f015c004550 comp rx=0 tx=0).stop 2024-04-16T17:08:49.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 -- 172.21.15.18:0/3158547463 shutdown_connections 2024-04-16T17:08:49.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f01540a3d20 0x7f01540a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f017019a790 0x7f017019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0170107140 0x7f01701a0b60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 --2- 172.21.15.18:0/3158547463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0170102070 0x7f01701a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:49.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.828+0000 7f0174a00640 1 -- 172.21.15.18:0/3158547463 >> 172.21.15.18:0/3158547463 conn(0x7f01700fd950 msgr2=0x7f01700fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:49.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.829+0000 7f0174a00640 1 -- 172.21.15.18:0/3158547463 shutdown_connections 2024-04-16T17:08:49.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:49.829+0000 7f0174a00640 1 -- 172.21.15.18:0/3158547463 wait complete. 2024-04-16T17:08:50.546 INFO:tasks.workunit.client.0.smithi018.stdout: 1 139280 3.96 MB/sec execute 63 sec latency 46.198 ms 2024-04-16T17:08:50.854 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:08:50.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:08:50.854 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.100+0000 7f9177200640 1 -- 172.21.15.18:0/2162109041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9178074cd0 msgr2=0x7f917810e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.100+0000 7f9177200640 1 --2- 172.21.15.18:0/2162109041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9178074cd0 0x7f917810e180 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9164009900 tx=0x7f916402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.101+0000 7f9177200640 1 -- 172.21.15.18:0/2162109041 shutdown_connections 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.101+0000 7f9177200640 1 --2- 172.21.15.18:0/2162109041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f917810e6c0 0x7f9178110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.101+0000 7f9177200640 1 --2- 172.21.15.18:0/2162109041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9178074cd0 0x7f917810e180 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.101+0000 7f9177200640 1 --2- 172.21.15.18:0/2162109041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91780743b0 0x7f9178074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.101+0000 7f9177200640 1 -- 172.21.15.18:0/2162109041 >> 172.21.15.18:0/2162109041 conn(0x7f91780fd950 msgr2=0x7f91780ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.101+0000 7f9177200640 1 -- 172.21.15.18:0/2162109041 shutdown_connections 2024-04-16T17:08:51.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.101+0000 7f9177200640 1 -- 172.21.15.18:0/2162109041 wait complete. 2024-04-16T17:08:51.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.102+0000 7f9177200640 1 Processor -- start 2024-04-16T17:08:51.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.102+0000 7f9177200640 1 -- start start 2024-04-16T17:08:51.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9177200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91780743b0 0x7f91781a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9177200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9178074cd0 0x7f91781a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9177200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917810e6c0 0x7f917819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9177200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91781143b0 con 0x7f9178074cd0 2024-04-16T17:08:51.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9177200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9178114230 con 0x7f91780743b0 2024-04-16T17:08:51.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9177200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9178114530 con 0x7f917810e6c0 2024-04-16T17:08:51.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9175e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91780743b0 0x7f91781a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9175400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9178074cd0 0x7f91781a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9175e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91780743b0 0x7f91781a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55608/0 (socket says 172.21.15.18:55608) 2024-04-16T17:08:51.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9175e00640 1 -- 172.21.15.18:0/2746166323 learned_addr learned my addr 172.21.15.18:0/2746166323 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.103+0000 7f9176800640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917810e6c0 0x7f917819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f9175e00640 1 -- 172.21.15.18:0/2746166323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917810e6c0 msgr2=0x7f917819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f9175e00640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917810e6c0 0x7f917819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f9175e00640 1 -- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9178074cd0 msgr2=0x7f91781a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f9175e00640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9178074cd0 0x7f91781a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f9175e00640 1 -- 172.21.15.18:0/2746166323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917819b010 con 0x7f91780743b0 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f9175e00640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91780743b0 0x7f91781a0680 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f916800d950 tx=0x7f916800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f916ea00640 1 -- 172.21.15.18:0/2746166323 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168017030 con 0x7f91780743b0 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f916ea00640 1 -- 172.21.15.18:0/2746166323 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9168004020 con 0x7f91780743b0 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f916ea00640 1 -- 172.21.15.18:0/2746166323 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680047a0 con 0x7f91780743b0 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.104+0000 7f9177200640 1 -- 172.21.15.18:0/2746166323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917819b300 con 0x7f91780743b0 2024-04-16T17:08:51.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.105+0000 7f9177200640 1 -- 172.21.15.18:0/2746166323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91781a74d0 con 0x7f91780743b0 2024-04-16T17:08:51.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.108+0000 7f916ea00640 1 -- 172.21.15.18:0/2746166323 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f91680041d0 con 0x7f91780743b0 2024-04-16T17:08:51.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.109+0000 7f917f829740 1 Processor -- start 2024-04-16T17:08:51.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.109+0000 7f917f829740 1 -- start start 2024-04-16T17:08:51.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f917f829740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dbe90 0x55775b1dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f917f829740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f917f829740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f917f829740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55775b140cb0 con 0x55775b1dbe90 2024-04-16T17:08:51.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f917f829740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55775b0e7370 con 0x55775b1e5390 2024-04-16T17:08:51.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f917f829740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55775b13cc70 con 0x55775b1dc7b0 2024-04-16T17:08:51.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f9176800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dbe90 0x55775b1dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f9175400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f9175400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55622/0 (socket says 172.21.15.18:55622) 2024-04-16T17:08:51.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f9175400640 1 -- 172.21.15.18:0/10269644 learned_addr learned my addr 172.21.15.18:0/10269644 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.110+0000 7f9175e00640 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f916ea00640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f915c0a3da0 0x7f915c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f916ea00640 1 -- 172.21.15.18:0/2746166323 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f91680cfbc0 con 0x7f91780743b0 2024-04-16T17:08:51.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f9176800640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f915c0a3da0 0x7f915c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f9175e00640 1 -- 172.21.15.18:0/10269644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 msgr2=0x55775b1e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f9175e00640 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f9175e00640 1 -- 172.21.15.18:0/10269644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dbe90 msgr2=0x55775b1dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f9175e00640 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dbe90 0x55775b1dc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f9175e00640 1 -- 172.21.15.18:0/10269644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775b106910 con 0x55775b1dc7b0 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.111+0000 7f9176800640 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dbe90 0x55775b1dc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.112+0000 7f9175e00640 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e4e50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f91680ccf00 tx=0x7f9168000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26bd528913f6a1a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.112+0000 7f9163000640 1 -- 172.21.15.18:0/10269644 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916809be50 con 0x55775b1dc7b0 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.112+0000 7f9163000640 1 -- 172.21.15.18:0/10269644 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916809b750 con 0x55775b1dc7b0 2024-04-16T17:08:51.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.112+0000 7f9163000640 1 -- 172.21.15.18:0/10269644 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680a5c40 con 0x55775b1dc7b0 2024-04-16T17:08:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f9176800640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f915c0a3da0 0x7f915c0a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9170005fd0 tx=0x7f9170004ab0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 -- 172.21.15.18:0/10269644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 msgr2=0x55775b1e4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e4e50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f91680ccf00 tx=0x7f9168000bc0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 -- 172.21.15.18:0/10269644 shutdown_connections 2024-04-16T17:08:51.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e4e50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 --2- 172.21.15.18:0/10269644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dbe90 0x55775b1dc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 -- 172.21.15.18:0/10269644 >> 172.21.15.18:0/10269644 conn(0x55775b1d65c0 msgr2=0x55775b1d69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 -- 172.21.15.18:0/10269644 shutdown_connections 2024-04-16T17:08:51.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.113+0000 7f917f829740 1 -- 172.21.15.18:0/10269644 wait complete. 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.114+0000 7f917f829740 1 -- start start 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.114+0000 7f917f829740 1 Processor -- start 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.115+0000 7f917f829740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dbe90 0x55775b1e2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.115+0000 7f917f829740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dc7b0 0x55775b1e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.115+0000 7f917f829740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1e5390 0x55775b1e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.115+0000 7f917f829740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55775b1e8180 con 0x55775b1dc7b0 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.115+0000 7f917f829740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55775b0fa730 con 0x55775b1dbe90 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.115+0000 7f917f829740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55775b1e8300 con 0x55775b1e5390 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.115+0000 7f917f829740 10 client.0 _ll_register_callbacks cb 0x55775b0c91e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dc7b0 0x55775b1e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dc7b0 0x55775b1e2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57674/0 (socket says 172.21.15.18:57674) 2024-04-16T17:08:51.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175400640 1 -- 172.21.15.18:0/230629157 learned_addr learned my addr 172.21.15.18:0/230629157 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175e00640 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dbe90 0x55775b1e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9176800640 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1e5390 0x55775b1e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175400640 1 -- 172.21.15.18:0/230629157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1e5390 msgr2=0x55775b1e2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175400640 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1e5390 0x55775b1e2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175400640 1 -- 172.21.15.18:0/230629157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dbe90 msgr2=0x55775b1e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175400640 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dbe90 0x55775b1e2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9175400640 1 -- 172.21.15.18:0/230629157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775af1b7e0 con 0x55775b1dc7b0 2024-04-16T17:08:51.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.116+0000 7f9176800640 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1e5390 0x55775b1e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:51.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.117+0000 7f9175400640 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dc7b0 0x55775b1e2870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9164036040 tx=0x7f9164004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01d3ad3dd903f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.117+0000 7f9157400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.117+0000 7f9157400640 1 -- 172.21.15.18:0/230629157 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91640046c0 con 0x55775b1dc7b0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.117+0000 7f9157400640 1 -- 172.21.15.18:0/230629157 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9164004860 con 0x55775b1dc7b0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.117+0000 7f917f829740 1 -- 172.21.15.18:0/230629157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f916809b750 con 0x55775b1dc7b0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.118+0000 7f9157400640 1 -- 172.21.15.18:0/230629157 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9164004a60 con 0x55775b1dc7b0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.118+0000 7f917f829740 10 client.16566 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.118+0000 7f917f829740 1 -- 172.21.15.18:0/230629157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55775af7bf70 con 0x55775b1dc7b0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.118+0000 7f9157400640 1 -- 172.21.15.18:0/230629157 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916809b750 con 0x55775b1dc7b0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.119+0000 7f9157400640 1 -- 172.21.15.18:0/230629157 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9164046070 con 0x55775b1dc7b0 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.119+0000 7f917f829740 10 client.16566 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.119+0000 7f917f829740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.119+0000 7f917f829740 20 client.16566 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.119+0000 7f917f829740 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55775b1f5610 0x55775b1f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.119+0000 7f917f829740 4 client.16566 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:51.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.119+0000 7f917f829740 1 -- 172.21.15.18:0/230629157 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55775b14e6d0 con 0x55775b1f5610 2024-04-16T17:08:51.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.119+0000 7f9175e00640 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55775b1f5610 0x55775b1f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.120+0000 7f9175e00640 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55775b1f5610 0x55775b1f7a90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27db43dcb91c1f6b server_cookie=1c3f9206b21a29a7 in_seq=0 out_seq=0 2024-04-16T17:08:51.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.120+0000 7f9157400640 10 client.16566 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:51.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f9157400640 1 -- 172.21.15.18:0/230629157 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55775b14e6d0 con 0x55775b1f5610 2024-04-16T17:08:51.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f9157400640 10 client.16566 handle_command_reply: tid=0 2024-04-16T17:08:51.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 client.16566 shutdown 2024-04-16T17:08:51.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 -- 172.21.15.18:0/230629157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dc7b0 msgr2=0x55775b1e2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dc7b0 0x55775b1e2870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9164036040 tx=0x7f9164004040 comp rx=0 tx=0).stop 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 -- 172.21.15.18:0/230629157 shutdown_connections 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55775b1f5610 0x55775b1f7a90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1e5390 0x55775b1e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1dc7b0 0x55775b1e2870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 --2- 172.21.15.18:0/230629157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dbe90 0x55775b1e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f917f829740 1 -- 172.21.15.18:0/230629157 >> 172.21.15.18:0/230629157 conn(0x55775b1d65c0 msgr2=0x55775b1d8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.122+0000 7f9157400640 0 client.16566 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.123+0000 7f917f829740 1 -- 172.21.15.18:0/230629157 shutdown_connections 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.123+0000 7f917f829740 1 -- 172.21.15.18:0/230629157 wait complete. 2024-04-16T17:08:51.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.123+0000 7f917f829740 20 client.16566 trim_cache size 0 max 16384 2024-04-16T17:08:51.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f917f829740 1 Processor -- start 2024-04-16T17:08:51.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f917f829740 1 -- start start 2024-04-16T17:08:51.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f917f829740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dbe90 0x55775b1e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f917f829740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dc7b0 0x55775b1f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f917f829740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e5390 0x55775b1fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f917f829740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b3f00 con 0x55775b1e5390 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f917f829740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b17d0 con 0x55775b1dc7b0 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f917f829740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91781af620 con 0x55775b1dbe90 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.156+0000 7f9175400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dc7b0 0x55775b1f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9176800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e5390 0x55775b1fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9176800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e5390 0x55775b1fd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57676/0 (socket says 172.21.15.18:57676) 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9176800640 1 -- 172.21.15.18:0/1512960849 learned_addr learned my addr 172.21.15.18:0/1512960849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9175400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dc7b0 0x55775b1f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55634/0 (socket says 172.21.15.18:55634) 2024-04-16T17:08:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9175e00640 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dbe90 0x55775b1e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9175e00640 1 -- 172.21.15.18:0/1512960849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dc7b0 msgr2=0x55775b1f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9175e00640 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dc7b0 0x55775b1f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9175e00640 1 -- 172.21.15.18:0/1512960849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e5390 msgr2=0x55775b1fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9175e00640 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e5390 0x55775b1fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9175e00640 1 -- 172.21.15.18:0/1512960849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775ae13170 con 0x55775b1dbe90 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.157+0000 7f9176800640 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e5390 0x55775b1fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.158+0000 7f9175e00640 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dbe90 0x55775b1e4a60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f91680cffd0 tx=0x7f91680a61c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b502bd45b9a84f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.158+0000 7f9162600640 1 -- 172.21.15.18:0/1512960849 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916809b750 con 0x55775b1dbe90 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.158+0000 7f9162600640 1 -- 172.21.15.18:0/1512960849 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91680cd280 con 0x55775b1dbe90 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.158+0000 7f9162600640 1 -- 172.21.15.18:0/1512960849 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916809bc10 con 0x55775b1dbe90 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 -- 172.21.15.18:0/1512960849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dbe90 msgr2=0x55775b1e4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dbe90 0x55775b1e4a60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f91680cffd0 tx=0x7f91680a61c0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 -- 172.21.15.18:0/1512960849 shutdown_connections 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e5390 0x55775b1fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1dc7b0 0x55775b1f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 --2- 172.21.15.18:0/1512960849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dbe90 0x55775b1e4a60 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f91680cffd0 tx=0x7f91680a61c0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 -- 172.21.15.18:0/1512960849 >> 172.21.15.18:0/1512960849 conn(0x55775b1d65c0 msgr2=0x55775b1e2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 -- 172.21.15.18:0/1512960849 shutdown_connections 2024-04-16T17:08:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 -- 172.21.15.18:0/1512960849 wait complete. 2024-04-16T17:08:51.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.159+0000 7f917f829740 1 -- start start 2024-04-16T17:08:51.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f917f829740 1 Processor -- start 2024-04-16T17:08:51.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f917f829740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f917f829740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f917f829740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e9c20 0x55775b25abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f917f829740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91781afc90 con 0x55775b1e9c20 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f917f829740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b1100 con 0x55775b1e5390 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f917f829740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b0360 con 0x55775b1dc7b0 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f917f829740 10 client.0 _ll_register_callbacks cb 0x7f916402ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f9175e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f9175e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35498/0 (socket says 172.21.15.18:35498) 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.160+0000 7f9175e00640 1 -- 172.21.15.18:0/3147045962 learned_addr learned my addr 172.21.15.18:0/3147045962 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9175400640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9175e00640 1 -- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 msgr2=0x55775b1e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9176800640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e9c20 0x55775b25abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9175e00640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9175e00640 1 -- 172.21.15.18:0/3147045962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e9c20 msgr2=0x55775b25abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9175e00640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e9c20 0x55775b25abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9175e00640 1 -- 172.21.15.18:0/3147045962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775b25b290 con 0x55775b1dc7b0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9175400640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9176800640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e9c20 0x55775b25abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9175e00640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e91a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9168009610 tx=0x7f916809b350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6094d0de145607 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.161+0000 7f9157400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.162+0000 7f9157400640 1 -- 172.21.15.18:0/3147045962 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680ac060 con 0x55775b1dc7b0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.162+0000 7f9157400640 1 -- 172.21.15.18:0/3147045962 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9168097dc0 con 0x55775b1dc7b0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.162+0000 7f917f829740 1 -- 172.21.15.18:0/3147045962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91680cd280 con 0x55775b1dc7b0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.162+0000 7f9157400640 1 -- 172.21.15.18:0/3147045962 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f916809b750 con 0x55775b1dc7b0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.162+0000 7f917f829740 10 client.26740 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.162+0000 7f917f829740 1 -- 172.21.15.18:0/3147045962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55775b25c5b0 con 0x55775b1dc7b0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.162+0000 7f9157400640 1 -- 172.21.15.18:0/3147045962 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91680cd280 con 0x55775b1dc7b0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.163+0000 7f9157400640 1 -- 172.21.15.18:0/3147045962 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f91680a3f30 con 0x55775b1dc7b0 2024-04-16T17:08:51.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.163+0000 7f917f829740 10 client.26740 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:51.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.163+0000 7f917f829740 10 client.26740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:51.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.163+0000 7f917f829740 20 client.26740 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:51.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.163+0000 7f917f829740 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55775b25c5b0 0x55775b25e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.163+0000 7f917f829740 4 client.26740 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:08:51.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.163+0000 7f917f829740 1 -- 172.21.15.18:0/3147045962 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55775b148540 con 0x55775b25c5b0 2024-04-16T17:08:51.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.163+0000 7f9175400640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55775b25c5b0 0x55775b25e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.164+0000 7f9175400640 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55775b25c5b0 0x55775b25e9a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6291bc001156055a server_cookie=70b488e5f63f0056 in_seq=0 out_seq=0 2024-04-16T17:08:51.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.164+0000 7f9157400640 10 client.26740 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f9157400640 1 -- 172.21.15.18:0/3147045962 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55775b148540 con 0x55775b25c5b0 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f9157400640 10 client.26740 handle_command_reply: tid=0 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 client.26740 shutdown 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 -- 172.21.15.18:0/3147045962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 msgr2=0x55775b1e91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e91a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9168009610 tx=0x7f916809b350 comp rx=0 tx=0).stop 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 -- 172.21.15.18:0/3147045962 shutdown_connections 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55775b25c5b0 0x55775b25e9a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55775b1e9c20 0x55775b25abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55775b1e5390 0x55775b1e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 --2- 172.21.15.18:0/3147045962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775b1dc7b0 0x55775b1e91a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.166+0000 7f917f829740 1 -- 172.21.15.18:0/3147045962 >> 172.21.15.18:0/3147045962 conn(0x55775b1d65c0 msgr2=0x55775b1d4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.167+0000 7f9157400640 0 client.26740 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:51.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.167+0000 7f917f829740 1 -- 172.21.15.18:0/3147045962 shutdown_connections 2024-04-16T17:08:51.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.167+0000 7f917f829740 1 -- 172.21.15.18:0/3147045962 wait complete. 2024-04-16T17:08:51.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.167+0000 7f917f829740 20 client.26740 trim_cache size 0 max 16384 2024-04-16T17:08:51.169 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:08:51.169 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:08:51.169 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "eb273c67-6422-4df7-983c-595d6c6e0f90", 2024-04-16T17:08:51.169 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:08:51.169 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:08:51.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 -- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f915c0a3da0 msgr2=0x7f915c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f915c0a3da0 0x7f915c0a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9170005fd0 tx=0x7f9170004ab0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 -- 172.21.15.18:0/2746166323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91780743b0 msgr2=0x7f91781a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91780743b0 0x7f91781a0680 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f916800d950 tx=0x7f916800de10 comp rx=0 tx=0).stop 2024-04-16T17:08:51.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 -- 172.21.15.18:0/2746166323 shutdown_connections 2024-04-16T17:08:51.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f915c0a3da0 0x7f915c0a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917810e6c0 0x7f917819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9178074cd0 0x7f91781a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 --2- 172.21.15.18:0/2746166323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91780743b0 0x7f91781a0680 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.168+0000 7f9177200640 1 -- 172.21.15.18:0/2746166323 >> 172.21.15.18:0/2746166323 conn(0x7f91780fd950 msgr2=0x7f91780fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.169+0000 7f9177200640 1 -- 172.21.15.18:0/2746166323 shutdown_connections 2024-04-16T17:08:51.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.169+0000 7f9177200640 1 -- 172.21.15.18:0/2746166323 wait complete. 2024-04-16T17:08:51.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:08:51.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:08:51.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 -- 172.21.15.18:0/3288087316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c6410e640 msgr2=0x7f8c64110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 --2- 172.21.15.18:0/3288087316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c6410e640 0x7f8c64110a90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009ca0 tx=0x7f8c5c02c180 comp rx=0 tx=0).stop 2024-04-16T17:08:51.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 -- 172.21.15.18:0/3288087316 shutdown_connections 2024-04-16T17:08:51.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 --2- 172.21.15.18:0/3288087316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c6410e640 0x7f8c64110a90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 --2- 172.21.15.18:0/3288087316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c64074d50 0x7f8c6410e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 --2- 172.21.15.18:0/3288087316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c64074430 0x7f8c64074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 -- 172.21.15.18:0/3288087316 >> 172.21.15.18:0/3288087316 conn(0x7f8c640fd950 msgr2=0x7f8c640ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 -- 172.21.15.18:0/3288087316 shutdown_connections 2024-04-16T17:08:51.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.395+0000 7f8c69400640 1 -- 172.21.15.18:0/3288087316 wait complete. 2024-04-16T17:08:51.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c69400640 1 Processor -- start 2024-04-16T17:08:51.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c69400640 1 -- start start 2024-04-16T17:08:51.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c69400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c64074430 0x7f8c641a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c69400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c64074d50 0x7f8c641a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c69400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c6410e640 0x7f8c6419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c69400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641143a0 con 0x7f8c6410e640 2024-04-16T17:08:51.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c69400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64114220 con 0x7f8c64074430 2024-04-16T17:08:51.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c69400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64114520 con 0x7f8c64074d50 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c64074430 0x7f8c641a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c64074430 0x7f8c641a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55664/0 (socket says 172.21.15.18:55664) 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.396+0000 7f8c63e00640 1 -- 172.21.15.18:0/3269210518 learned_addr learned my addr 172.21.15.18:0/3269210518 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c63400640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c64074d50 0x7f8c641a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c68a00640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c6410e640 0x7f8c6419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c63e00640 1 -- 172.21.15.18:0/3269210518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c64074d50 msgr2=0x7f8c641a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c63e00640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c64074d50 0x7f8c641a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c63e00640 1 -- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c6410e640 msgr2=0x7f8c6419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c63e00640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c6410e640 0x7f8c6419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c63e00640 1 -- 172.21.15.18:0/3269210518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6419afc0 con 0x7f8c64074430 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c63400640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c64074d50 0x7f8c641a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.397+0000 7f8c63e00640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c64074430 0x7f8c641a05f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400e9f0 tx=0x7f8c5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.398+0000 7f8c60c00640 1 -- 172.21.15.18:0/3269210518 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400cd80 con 0x7f8c64074430 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.398+0000 7f8c60c00640 1 -- 172.21.15.18:0/3269210518 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c54010910 con 0x7f8c64074430 2024-04-16T17:08:51.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.398+0000 7f8c60c00640 1 -- 172.21.15.18:0/3269210518 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54018990 con 0x7f8c64074430 2024-04-16T17:08:51.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.398+0000 7f8c69400640 1 -- 172.21.15.18:0/3269210518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6419b2b0 con 0x7f8c64074430 2024-04-16T17:08:51.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.398+0000 7f8c69400640 1 -- 172.21.15.18:0/3269210518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a7570 con 0x7f8c64074430 2024-04-16T17:08:51.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c6d61c740 1 Processor -- start 2024-04-16T17:08:51.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c60c00640 1 -- 172.21.15.18:0/3269210518 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8c54010ac0 con 0x7f8c64074430 2024-04-16T17:08:51.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c6d61c740 1 -- start start 2024-04-16T17:08:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3b7390 0x5586ac3b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3c0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3c0890 0x5586ac3c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5586ac31d120 con 0x5586ac3b7cb0 2024-04-16T17:08:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5586ac2c3340 con 0x5586ac3c0890 2024-04-16T17:08:51.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.403+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586ac3190e0 con 0x5586ac3b7390 2024-04-16T17:08:51.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3b7390 0x5586ac3b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3b7390 0x5586ac3b7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35534/0 (socket says 172.21.15.18:35534) 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c68a00640 1 -- 172.21.15.18:0/50978263 learned_addr learned my addr 172.21.15.18:0/50978263 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c63400640 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3c0890 0x5586ac3c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c68a00640 1 -- 172.21.15.18:0/50978263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3c0890 msgr2=0x5586ac3c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c68a00640 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3c0890 0x5586ac3c2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c68a00640 1 -- 172.21.15.18:0/50978263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 msgr2=0x5586ac3c0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c63e00640 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3c0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c68a00640 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3c0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c68a00640 1 -- 172.21.15.18:0/50978263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ac0f97d0 con 0x5586ac3b7390 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.404+0000 7f8c63e00640 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3c0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.405+0000 7f8c68a00640 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3b7390 0x5586ac3b7770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c02c790 tx=0x7f8c5c005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916205312efff3d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.405+0000 7f8c60c00640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c480a3e20 0x7f8c480a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.405+0000 7f8c60c00640 1 -- 172.21.15.18:0/3269210518 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8c540d0e90 con 0x7f8c64074430 2024-04-16T17:08:51.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.405+0000 7f8c63e00640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c480a3e20 0x7f8c480a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.406+0000 7f8c63e00640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c480a3e20 0x7f8c480a62e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5586ac3b6e60 tx=0x7f8c54000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.407+0000 7f8c59200640 1 -- 172.21.15.18:0/50978263 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c043070 con 0x5586ac3b7390 2024-04-16T17:08:51.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.407+0000 7f8c59200640 1 -- 172.21.15.18:0/50978263 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c5c003e50 con 0x5586ac3b7390 2024-04-16T17:08:51.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.407+0000 7f8c59200640 1 -- 172.21.15.18:0/50978263 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c039040 con 0x5586ac3b7390 2024-04-16T17:08:51.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 -- 172.21.15.18:0/50978263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3b7390 msgr2=0x5586ac3b7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3b7390 0x5586ac3b7770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c02c790 tx=0x7f8c5c005f80 comp rx=0 tx=0).stop 2024-04-16T17:08:51.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 -- 172.21.15.18:0/50978263 shutdown_connections 2024-04-16T17:08:51.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3c0890 0x5586ac3c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3c0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/50978263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3b7390 0x5586ac3b7770 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 -- 172.21.15.18:0/50978263 >> 172.21.15.18:0/50978263 conn(0x5586ac3b1ac0 msgr2=0x5586ac3b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 -- 172.21.15.18:0/50978263 shutdown_connections 2024-04-16T17:08:51.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 -- 172.21.15.18:0/50978263 wait complete. 2024-04-16T17:08:51.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.408+0000 7f8c6d61c740 1 -- start start 2024-04-16T17:08:51.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.409+0000 7f8c6d61c740 1 Processor -- start 2024-04-16T17:08:51.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.409+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.409+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.409+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.409+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5586ac3c3620 con 0x5586ac3b7390 2024-04-16T17:08:51.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.409+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5586ac2d6710 con 0x5586ac3b7cb0 2024-04-16T17:08:51.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.409+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586ac3c37a0 con 0x5586ac3c0890 2024-04-16T17:08:51.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.409+0000 7f8c6d61c740 10 client.0 _ll_register_callbacks cb 0x5586ac2d1af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:51.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3bd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57710/0 (socket says 172.21.15.18:57710) 2024-04-16T17:08:51.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 -- 172.21.15.18:0/1184714900 learned_addr learned my addr 172.21.15.18:0/1184714900 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c63400640 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 -- 172.21.15.18:0/1184714900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 msgr2=0x5586ac3be3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3be3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 -- 172.21.15.18:0/1184714900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 msgr2=0x5586ac3bde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.410+0000 7f8c68a00640 1 -- 172.21.15.18:0/1184714900 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ac02f3f0 con 0x5586ac3b7390 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.411+0000 7f8c68a00640 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3bd920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0047a0 tx=0x7f8c5c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8233917d065d563b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.411+0000 7f8c4d600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.411+0000 7f8c4d600640 1 -- 172.21.15.18:0/1184714900 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c039070 con 0x5586ac3b7390 2024-04-16T17:08:51.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.411+0000 7f8c4d600640 1 -- 172.21.15.18:0/1184714900 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8c5c003b80 con 0x5586ac3b7390 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.411+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1184714900 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c5c003e50 con 0x5586ac3b7390 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.411+0000 7f8c4d600640 1 -- 172.21.15.18:0/1184714900 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c5c043040 con 0x5586ac3b7390 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.411+0000 7f8c6d61c740 10 client.16578 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.411+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1184714900 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586abfef0f0 con 0x5586ac3b7390 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c4d600640 1 -- 172.21.15.18:0/1184714900 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c5c003e50 con 0x5586ac3b7390 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c4d600640 1 -- 172.21.15.18:0/1184714900 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c5c048650 con 0x5586ac3b7390 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c6d61c740 10 client.16578 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c6d61c740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c6d61c740 20 client.16578 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5586ac3d0a40 0x5586ac3d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c6d61c740 4 client.16578 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1184714900 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586ac32ab40 con 0x5586ac3d0a40 2024-04-16T17:08:51.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.412+0000 7f8c63400640 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5586ac3d0a40 0x5586ac3d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.413+0000 7f8c63400640 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5586ac3d0a40 0x5586ac3d2e30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a032a62039a916c3 server_cookie=3860e40f60b6981c in_seq=0 out_seq=0 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.413+0000 7f8c4d600640 10 client.16578 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c4d600640 1 -- 172.21.15.18:0/1184714900 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586ac32ab40 con 0x5586ac3d0a40 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c4d600640 10 client.16578 handle_command_reply: tid=0 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 client.16578 shutdown 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1184714900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 msgr2=0x5586ac3bd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3bd920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0047a0 tx=0x7f8c5c004600 comp rx=0 tx=0).stop 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1184714900 shutdown_connections 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5586ac3d0a40 0x5586ac3d2e30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3be3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1184714900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3bd920 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1184714900 >> 172.21.15.18:0/1184714900 conn(0x5586ac3b1ac0 msgr2=0x5586ac3b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c4d600640 0 client.16578 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1184714900 shutdown_connections 2024-04-16T17:08:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.414+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1184714900 wait complete. 2024-04-16T17:08:51.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.415+0000 7f8c6d61c740 20 client.16578 trim_cache size 0 max 16384 2024-04-16T17:08:51.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.434+0000 7f8c6d61c740 1 Processor -- start 2024-04-16T17:08:51.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.434+0000 7f8c6d61c740 1 -- start start 2024-04-16T17:08:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641b3f80 con 0x5586ac3b7390 2024-04-16T17:08:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641b1850 con 0x5586ac3b7cb0 2024-04-16T17:08:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641af660 con 0x5586ac3c0890 2024-04-16T17:08:51.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3c0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57724/0 (socket says 172.21.15.18:57724) 2024-04-16T17:08:51.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c68a00640 1 -- 172.21.15.18:0/3997040623 learned_addr learned my addr 172.21.15.18:0/3997040623 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c63400640 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.435+0000 7f8c68a00640 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.436+0000 7f8c68a00640 1 -- 172.21.15.18:0/3997040623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 msgr2=0x5586ac3b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.436+0000 7f8c68a00640 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.436+0000 7f8c68a00640 1 -- 172.21.15.18:0/3997040623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 msgr2=0x5586ac3bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.436+0000 7f8c68a00640 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.436+0000 7f8c68a00640 1 -- 172.21.15.18:0/3997040623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ac3c3560 con 0x5586ac3b7390 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.436+0000 7f8c63400640 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.436+0000 7f8c68a00640 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3c0050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009a60 tx=0x7f8c5c02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5e253140412eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.436+0000 7f8c4fe00640 1 -- 172.21.15.18:0/3997040623 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c043040 con 0x5586ac3b7390 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.437+0000 7f8c4fe00640 1 -- 172.21.15.18:0/3997040623 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c5c05c020 con 0x5586ac3b7390 2024-04-16T17:08:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.437+0000 7f8c4fe00640 1 -- 172.21.15.18:0/3997040623 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c003b80 con 0x5586ac3b7390 2024-04-16T17:08:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 -- 172.21.15.18:0/3997040623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 msgr2=0x5586ac3c0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3c0050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009a60 tx=0x7f8c5c02c930 comp rx=0 tx=0).stop 2024-04-16T17:08:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 -- 172.21.15.18:0/3997040623 shutdown_connections 2024-04-16T17:08:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7cb0 0x5586ac3bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/3997040623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7390 0x5586ac3c0050 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 -- 172.21.15.18:0/3997040623 >> 172.21.15.18:0/3997040623 conn(0x5586ac3b1ac0 msgr2=0x5586ac3b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 -- 172.21.15.18:0/3997040623 shutdown_connections 2024-04-16T17:08:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 -- 172.21.15.18:0/3997040623 wait complete. 2024-04-16T17:08:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.438+0000 7f8c6d61c740 1 -- start start 2024-04-16T17:08:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.439+0000 7f8c6d61c740 1 Processor -- start 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.439+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7390 0x5586ac3d62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.439+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.439+0000 7f8c6d61c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.439+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641afd10 con 0x5586ac3b7cb0 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.439+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641b1180 con 0x5586ac3b7390 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.439+0000 7f8c6d61c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641b03e0 con 0x5586ac3c0890 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.439+0000 7f8c6d61c740 10 client.0 _ll_register_callbacks cb 0x7f8c50009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7390 0x5586ac3d62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c68a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7390 0x5586ac3d62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55692/0 (socket says 172.21.15.18:55692) 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c68a00640 1 -- 172.21.15.18:0/1029319164 learned_addr learned my addr 172.21.15.18:0/1029319164 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c63400640 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c68a00640 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c63400640 1 -- 172.21.15.18:0/1029319164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 msgr2=0x5586ac3d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c63400640 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3d3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c63400640 1 -- 172.21.15.18:0/1029319164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7390 msgr2=0x5586ac3d62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c63400640 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7390 0x5586ac3d62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c63400640 1 -- 172.21.15.18:0/1029319164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ac3d4400 con 0x5586ac3b7cb0 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c68a00640 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7390 0x5586ac3d62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c63400640 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3d3650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5586ac3c3790 tx=0x7f8c50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c07a55aa3fe1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.440+0000 7f8c4d600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.441+0000 7f8c4d600640 1 -- 172.21.15.18:0/1029319164 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50024030 con 0x5586ac3b7cb0 2024-04-16T17:08:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.441+0000 7f8c4d600640 1 -- 172.21.15.18:0/1029319164 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f8c50010b20 con 0x5586ac3b7cb0 2024-04-16T17:08:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.441+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1029319164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c5c05c020 con 0x5586ac3b7cb0 2024-04-16T17:08:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.441+0000 7f8c4d600640 1 -- 172.21.15.18:0/1029319164 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c50010d20 con 0x5586ac3b7cb0 2024-04-16T17:08:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.441+0000 7f8c6d61c740 10 client.16584 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.441+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1029319164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586ac436d30 con 0x5586ac3b7cb0 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c4d600640 1 -- 172.21.15.18:0/1029319164 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c5c05c020 con 0x5586ac3b7cb0 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c4d600640 1 -- 172.21.15.18:0/1029319164 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8c500295a0 con 0x5586ac3b7cb0 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c6d61c740 10 client.16584 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c6d61c740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c6d61c740 20 client.16584 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5586ac4374d0 0x5586ac4398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c6d61c740 4 client.16584 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1029319164 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586ac3249b0 con 0x5586ac4374d0 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.442+0000 7f8c68a00640 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5586ac4374d0 0x5586ac4398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.443+0000 7f8c68a00640 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5586ac4374d0 0x5586ac4398f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a7cfc77fd78e4c server_cookie=8b6d7af05390f595 in_seq=0 out_seq=0 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.443+0000 7f8c4d600640 10 client.16584 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.443+0000 7f8c4d600640 1 -- 172.21.15.18:0/1029319164 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5586ac3249b0 con 0x5586ac4374d0 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.443+0000 7f8c4d600640 10 client.16584 handle_command_reply: tid=0 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.443+0000 7f8c6d61c740 1 client.16584 shutdown 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.443+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1029319164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 msgr2=0x5586ac3d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.443+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3d3650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5586ac3c3790 tx=0x7f8c50005880 comp rx=0 tx=0).stop 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1029319164 shutdown_connections 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5586ac4374d0 0x5586ac4398f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac3c0890 0x5586ac3d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac3b7cb0 0x5586ac3d3650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c4d600640 0 client.16584 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 1 --2- 172.21.15.18:0/1029319164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac3b7390 0x5586ac3d62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1029319164 >> 172.21.15.18:0/1029319164 conn(0x5586ac3b1ac0 msgr2=0x5586ac3b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1029319164 shutdown_connections 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 1 -- 172.21.15.18:0/1029319164 wait complete. 2024-04-16T17:08:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.444+0000 7f8c6d61c740 20 client.16584 trim_cache size 0 max 16384 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout: "eb273c67-6422-4df7-983c-595d6c6e0f90": { 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "eb273c67-6422-4df7-983c-595d6c6e0f90", 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:08:51.448 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 -- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c480a3e20 msgr2=0x7f8c480a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c480a3e20 0x7f8c480a62e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5586ac3b6e60 tx=0x7f8c54000bc0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 -- 172.21.15.18:0/3269210518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c64074430 msgr2=0x7f8c641a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c64074430 0x7f8c641a05f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400e9f0 tx=0x7f8c5400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 -- 172.21.15.18:0/3269210518 shutdown_connections 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8c480a3e20 0x7f8c480a62e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c6410e640 0x7f8c6419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c64074d50 0x7f8c641a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 --2- 172.21.15.18:0/3269210518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c64074430 0x7f8c641a05f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:51.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 -- 172.21.15.18:0/3269210518 >> 172.21.15.18:0/3269210518 conn(0x7f8c640fd950 msgr2=0x7f8c640fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:51.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 -- 172.21.15.18:0/3269210518 shutdown_connections 2024-04-16T17:08:51.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:08:51.446+0000 7f8c69400640 1 -- 172.21.15.18:0/3269210518 wait complete. 2024-04-16T17:08:51.464 INFO:tasks.cephfs.filesystem:scrub status for tag:eb273c67-6422-4df7-983c-595d6c6e0f90 - {'path': '/', 'tag': 'eb273c67-6422-4df7-983c-595d6c6e0f90', 'options': 'recursive,force'} 2024-04-16T17:08:51.547 INFO:tasks.workunit.client.0.smithi018.stdout: 1 140703 3.99 MB/sec execute 64 sec latency 17.937 ms 2024-04-16T17:08:51.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:51 smithi022 ceph-mon[36023]: pgmap v126: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-16T17:08:51.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:51 smithi043 ceph-mon[36901]: pgmap v126: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-16T17:08:51.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:39 smithi018 ceph-mon[31776]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 881 KiB/s wr, 139 op/s 2024-04-16T17:08:51.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:41 smithi018 ceph-mon[31776]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-04-16T17:08:51.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:43 smithi018 ceph-mon[31776]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 851 KiB/s wr, 137 op/s 2024-04-16T17:08:51.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:45 smithi018 ceph-mon[31776]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 851 KiB/s wr, 137 op/s 2024-04-16T17:08:51.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:47 smithi018 ceph-mon[31776]: pgmap v124: 129 pgs: 129 active+clean; 37 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-04-16T17:08:51.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:49 smithi018 ceph-mon[31776]: pgmap v125: 129 pgs: 129 active+clean; 37 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1008 KiB/s wr, 130 op/s 2024-04-16T17:08:51.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:49 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3914549197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:51.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:51 smithi018 ceph-mon[31776]: pgmap v126: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-16T17:08:52.547 INFO:tasks.workunit.client.0.smithi018.stdout: 1 141716 4.05 MB/sec execute 65 sec latency 23.182 ms 2024-04-16T17:08:52.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:52 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:08:52.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:52 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:52.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:52 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:08:52.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:52 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:52.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:52 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:08:52.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:52 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:52 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:08:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:52 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:53.547 INFO:tasks.workunit.client.0.smithi018.stdout: 1 142122 4.05 MB/sec execute 66 sec latency 83.056 ms 2024-04-16T17:08:53.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:53 smithi022 ceph-mon[36023]: pgmap v127: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-04-16T17:08:53.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:53 smithi043 ceph-mon[36901]: pgmap v127: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-04-16T17:08:54.547 INFO:tasks.workunit.client.0.smithi018.stdout: 1 142325 4.00 MB/sec execute 67 sec latency 88.954 ms 2024-04-16T17:08:54.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:54 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:08:54.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:54 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:08:55.547 INFO:tasks.workunit.client.0.smithi018.stdout: 1 142926 3.95 MB/sec execute 68 sec latency 52.392 ms 2024-04-16T17:08:55.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:55 smithi043 ceph-mon[36901]: pgmap v128: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-04-16T17:08:55.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:52 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:08:55.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:52 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:55.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:52 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:08:55.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:52 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:55.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:53 smithi018 ceph-mon[31776]: pgmap v127: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-04-16T17:08:55.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:54 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:08:55.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:55 smithi018 ceph-mon[31776]: pgmap v128: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-04-16T17:08:55.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:55 smithi022 ceph-mon[36023]: pgmap v128: 129 pgs: 129 active+clean; 38 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-04-16T17:08:56.547 INFO:tasks.workunit.client.0.smithi018.stdout: 1 143893 3.96 MB/sec execute 69 sec latency 48.995 ms 2024-04-16T17:08:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 -- 172.21.15.18:0/2021302472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58101280 msgr2=0x7f4e5810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2021302472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58101280 0x7f4e5810eef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009c10 tx=0x7f4e5002bee0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 -- 172.21.15.18:0/2021302472 shutdown_connections 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2021302472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58101280 0x7f4e5810eef0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2021302472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e581008c0 0x7f4e58100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2021302472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e58107060 0x7f4e58107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 -- 172.21.15.18:0/2021302472 >> 172.21.15.18:0/2021302472 conn(0x7f4e58076240 msgr2=0x7f4e58076650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 -- 172.21.15.18:0/2021302472 shutdown_connections 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.965+0000 7f4e5f000640 1 -- 172.21.15.18:0/2021302472 wait complete. 2024-04-16T17:08:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.967+0000 7f4e5f000640 1 Processor -- start 2024-04-16T17:08:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.967+0000 7f4e5f000640 1 -- start start 2024-04-16T17:08:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.967+0000 7f4e5f000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e581008c0 0x7f4e581a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.967+0000 7f4e5f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e58101280 0x7f4e581a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.967+0000 7f4e5f000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58107060 0x7f4e5819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.967+0000 7f4e5f000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58112480 con 0x7f4e58107060 2024-04-16T17:08:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.967+0000 7f4e5f000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58112300 con 0x7f4e581008c0 2024-04-16T17:08:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.967+0000 7f4e5f000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58112600 con 0x7f4e58101280 2024-04-16T17:08:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e581008c0 0x7f4e581a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e58101280 0x7f4e581a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e581008c0 0x7f4e581a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58918/0 (socket says 172.21.15.18:58918) 2024-04-16T17:08:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57e00640 1 -- 172.21.15.18:0/2882698657 learned_addr learned my addr 172.21.15.18:0/2882698657 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e5d000640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58107060 0x7f4e5819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57e00640 1 -- 172.21.15.18:0/2882698657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e58101280 msgr2=0x7f4e581a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57e00640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e58101280 0x7f4e581a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57e00640 1 -- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58107060 msgr2=0x7f4e5819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57e00640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58107060 0x7f4e5819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e57e00640 1 -- 172.21.15.18:0/2882698657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5819b0a0 con 0x7f4e581008c0 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.968+0000 7f4e5d000640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58107060 0x7f4e5819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.969+0000 7f4e57e00640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e581008c0 0x7f4e581a0770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9f0 tx=0x7f4e4800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.969+0000 7f4e54c00640 1 -- 172.21.15.18:0/2882698657 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800cd80 con 0x7f4e581008c0 2024-04-16T17:08:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.969+0000 7f4e5f000640 1 -- 172.21.15.18:0/2882698657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5819b390 con 0x7f4e581008c0 2024-04-16T17:08:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.969+0000 7f4e54c00640 1 -- 172.21.15.18:0/2882698657 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e48005500 con 0x7f4e581008c0 2024-04-16T17:08:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.969+0000 7f4e54c00640 1 -- 172.21.15.18:0/2882698657 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48010690 con 0x7f4e581008c0 2024-04-16T17:08:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.970+0000 7f4e5f000640 1 -- 172.21.15.18:0/2882698657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a7630 con 0x7f4e581008c0 2024-04-16T17:08:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.973+0000 7f4e61e38740 1 Processor -- start 2024-04-16T17:08:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.973+0000 7f4e61e38740 1 -- start start 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.973+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.973+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bf5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.973+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9c01210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.973+0000 7f4e61e38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bfe9b5a780 con 0x55bfe9bf58a0 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.973+0000 7f4e61e38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bfe9b00f40 con 0x55bfe9bfee20 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.973+0000 7f4e61e38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bfe9b56740 con 0x55bfe9bf6610 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e57400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9c01210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35728/0 (socket says 172.21.15.18:35728) 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 -- 172.21.15.18:0/521364163 learned_addr learned my addr 172.21.15.18:0/521364163 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 -- 172.21.15.18:0/521364163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 msgr2=0x55bfe9c01210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 --2- 172.21.15.18:0/521364163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9c01210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 -- 172.21.15.18:0/521364163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 msgr2=0x55bfe9bf5d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 --2- 172.21.15.18:0/521364163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bf5d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 -- 172.21.15.18:0/521364163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe986d210 con 0x55bfe9bf6610 2024-04-16T17:08:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.974+0000 7f4e5d000640 1 --2- 172.21.15.18:0/521364163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf5360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009be0 tx=0x7f4e50005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f515bf33f924ae90 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.975+0000 7f4e54c00640 1 -- 172.21.15.18:0/2882698657 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e48004f60 con 0x7f4e581008c0 2024-04-16T17:08:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.975+0000 7f4e4d600640 1 -- 172.21.15.18:0/521364163 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50043030 con 0x55bfe9bf6610 2024-04-16T17:08:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.975+0000 7f4e4d600640 1 -- 172.21.15.18:0/521364163 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e50003970 con 0x55bfe9bf6610 2024-04-16T17:08:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.975+0000 7f4e4d600640 1 -- 172.21.15.18:0/521364163 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5002caf0 con 0x55bfe9bf6610 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e61e38740 1 -- 172.21.15.18:0/521364163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf6610 msgr2=0x55bfe9bf5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e61e38740 1 --2- 172.21.15.18:0/521364163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf5360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009be0 tx=0x7f4e50005ec0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e61e38740 1 -- 172.21.15.18:0/521364163 shutdown_connections 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e61e38740 1 --2- 172.21.15.18:0/521364163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9c01210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e61e38740 1 --2- 172.21.15.18:0/521364163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bf5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e61e38740 1 --2- 172.21.15.18:0/521364163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf5360 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e61e38740 1 -- 172.21.15.18:0/521364163 >> 172.21.15.18:0/521364163 conn(0x55bfe9b905d0 msgr2=0x55bfe9bf0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e61e38740 1 -- 172.21.15.18:0/521364163 shutdown_connections 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.976+0000 7f4e54c00640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e3c0a3cc0 0x7f4e3c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.977+0000 7f4e54c00640 1 -- 172.21.15.18:0/2882698657 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f4e480d0ff0 con 0x7f4e581008c0 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.977+0000 7f4e61e38740 1 -- 172.21.15.18:0/521364163 wait complete. 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.977+0000 7f4e57400640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e3c0a3cc0 0x7f4e3c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.977+0000 7f4e61e38740 1 -- start start 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.977+0000 7f4e61e38740 1 Processor -- start 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.978+0000 7f4e57400640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e3c0a3cc0 0x7f4e3c0a6180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55bfe9bf1720 tx=0x7f4e44009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.978+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bfe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.978+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf6610 0x55bfe9bfbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.978+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9bfc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.978+0000 7f4e61e38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bfe9c01af0 con 0x55bfe9bf6610 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.978+0000 7f4e61e38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bfe9b14390 con 0x55bfe9bfee20 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.978+0000 7f4e61e38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bfe9c01c70 con 0x55bfe9bf58a0 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.978+0000 7f4e61e38740 10 client.0 _ll_register_callbacks cb 0x55bfe9afdf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9bfc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e57400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf6610 0x55bfe9bfbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e57e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bfe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9bfc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58950/0 (socket says 172.21.15.18:58950) 2024-04-16T17:08:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 -- 172.21.15.18:0/3980034908 learned_addr learned my addr 172.21.15.18:0/3980034908 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 -- 172.21.15.18:0/3980034908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 msgr2=0x55bfe9bfe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bfe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 -- 172.21.15.18:0/3980034908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf6610 msgr2=0x55bfe9bfbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf6610 0x55bfe9bfbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 -- 172.21.15.18:0/3980034908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe9939620 con 0x55bfe9bfee20 2024-04-16T17:08:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e57e00640 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bfe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e5d000640 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9bfc3a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4e50036040 tx=0x7f4e5002cda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef1ae28bdfdd1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.979+0000 7f4e42000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.980+0000 7f4e42000640 1 -- 172.21.15.18:0/3980034908 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5004c030 con 0x55bfe9bfee20 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.980+0000 7f4e61e38740 1 -- 172.21.15.18:0/3980034908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e50003970 con 0x55bfe9bfee20 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.980+0000 7f4e42000640 1 -- 172.21.15.18:0/3980034908 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f4e50033540 con 0x55bfe9bfee20 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.980+0000 7f4e42000640 1 -- 172.21.15.18:0/3980034908 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e50059050 con 0x55bfe9bfee20 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.980+0000 7f4e61e38740 10 client.26795 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.980+0000 7f4e61e38740 1 -- 172.21.15.18:0/3980034908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfe9c0dbe0 con 0x55bfe9bfee20 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.980+0000 7f4e42000640 1 -- 172.21.15.18:0/3980034908 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e50003970 con 0x55bfe9bfee20 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.980+0000 7f4e42000640 1 -- 172.21.15.18:0/3980034908 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e5003f440 con 0x55bfe9bfee20 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e61e38740 10 client.26795 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e61e38740 10 client.26795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e61e38740 20 client.26795 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e61e38740 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bfe9c0ef30 0x55bfe9c11320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e61e38740 4 client.26795 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e61e38740 1 -- 172.21.15.18:0/3980034908 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe9b681a0 con 0x55bfe9c0ef30 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e57e00640 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bfe9c0ef30 0x55bfe9c11320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e42000640 10 client.26795 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.981+0000 7f4e57e00640 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bfe9c0ef30 0x55bfe9c11320 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488fdeba1890e0cb server_cookie=cc2f57d2bfa37c47 in_seq=0 out_seq=0 2024-04-16T17:08:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.982+0000 7f4e42000640 1 -- 172.21.15.18:0/3980034908 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfe9b681a0 con 0x55bfe9c0ef30 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.982+0000 7f4e42000640 10 client.26795 handle_command_reply: tid=0 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.982+0000 7f4e61e38740 1 client.26795 shutdown 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.982+0000 7f4e61e38740 1 -- 172.21.15.18:0/3980034908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 msgr2=0x55bfe9bfc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.982+0000 7f4e61e38740 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9bfc3a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4e50036040 tx=0x7f4e5002cda0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 1 -- 172.21.15.18:0/3980034908 shutdown_connections 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bfe9c0ef30 0x55bfe9c11320 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bfee20 0x55bfe9bfc3a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf6610 0x55bfe9bfbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e42000640 0 client.26795 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 1 --2- 172.21.15.18:0/3980034908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bfe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 1 -- 172.21.15.18:0/3980034908 >> 172.21.15.18:0/3980034908 conn(0x55bfe9b905d0 msgr2=0x55bfe9bf22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 1 -- 172.21.15.18:0/3980034908 shutdown_connections 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 1 -- 172.21.15.18:0/3980034908 wait complete. 2024-04-16T17:08:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:56.983+0000 7f4e61e38740 20 client.26795 trim_cache size 0 max 16384 2024-04-16T17:08:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.001+0000 7f4e61e38740 1 Processor -- start 2024-04-16T17:08:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.001+0000 7f4e61e38740 1 -- start start 2024-04-16T17:08:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 0x55bfe9c0a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9c16980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bfee20 0x55bfe9c16ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e61e38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b3fc0 con 0x55bfe9bf58a0 2024-04-16T17:08:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e61e38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b1890 con 0x55bfe9bf6610 2024-04-16T17:08:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e61e38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581af6e0 con 0x55bfe9bfee20 2024-04-16T17:08:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e57400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9c16980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e57400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9c16980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58966/0 (socket says 172.21.15.18:58966) 2024-04-16T17:08:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e57400640 1 -- 172.21.15.18:0/1016162411 learned_addr learned my addr 172.21.15.18:0/1016162411 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e57e00640 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 0x55bfe9c0a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e5d000640 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bfee20 0x55bfe9c16ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e57400640 1 -- 172.21.15.18:0/1016162411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bfee20 msgr2=0x55bfe9c16ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e57400640 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bfee20 0x55bfe9c16ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.002+0000 7f4e57400640 1 -- 172.21.15.18:0/1016162411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 msgr2=0x55bfe9c0a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.003+0000 7f4e57400640 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 0x55bfe9c0a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.003+0000 7f4e57400640 1 -- 172.21.15.18:0/1016162411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe9c0bd10 con 0x55bfe9bf6610 2024-04-16T17:08:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.003+0000 7f4e5d000640 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bfee20 0x55bfe9c16ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.003+0000 7f4e57e00640 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 0x55bfe9c0a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.003+0000 7f4e57400640 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9c16980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009530 tx=0x7f4e44016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce971e258c1f4bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.003+0000 7f4e4cc00640 1 -- 172.21.15.18:0/1016162411 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4401cbf0 con 0x55bfe9bf6610 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.003+0000 7f4e4cc00640 1 -- 172.21.15.18:0/1016162411 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4401cd90 con 0x55bfe9bf6610 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.003+0000 7f4e4cc00640 1 -- 172.21.15.18:0/1016162411 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44028460 con 0x55bfe9bf6610 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.004+0000 7f4e61e38740 1 -- 172.21.15.18:0/1016162411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 msgr2=0x55bfe9c16980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.004+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9c16980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009530 tx=0x7f4e44016000 comp rx=0 tx=0).stop 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.005+0000 7f4e61e38740 1 -- 172.21.15.18:0/1016162411 shutdown_connections 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.005+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bfee20 0x55bfe9c16ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.005+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9c16980 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.005+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1016162411 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bf58a0 0x55bfe9c0a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.005+0000 7f4e61e38740 1 -- 172.21.15.18:0/1016162411 >> 172.21.15.18:0/1016162411 conn(0x55bfe9b905d0 msgr2=0x55bfe9bf00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.005+0000 7f4e61e38740 1 -- 172.21.15.18:0/1016162411 shutdown_connections 2024-04-16T17:08:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.005+0000 7f4e61e38740 1 -- 172.21.15.18:0/1016162411 wait complete. 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.005+0000 7f4e61e38740 1 -- start start 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.006+0000 7f4e61e38740 1 Processor -- start 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.006+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bf8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.006+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.006+0000 7f4e61e38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bfee20 0x55bfe9bf9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.006+0000 7f4e61e38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581afd50 con 0x55bfe9bfee20 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.006+0000 7f4e61e38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b11c0 con 0x55bfe9bf6610 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.006+0000 7f4e61e38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b0420 con 0x55bfe9bf58a0 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.006+0000 7f4e61e38740 10 client.0 _ll_register_callbacks cb 0x7f4e44009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58976/0 (socket says 172.21.15.18:58976) 2024-04-16T17:08:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57400640 1 -- 172.21.15.18:0/1501780006 learned_addr learned my addr 172.21.15.18:0/1501780006 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:08:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57e00640 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bf8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57400640 1 -- 172.21.15.18:0/1501780006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 msgr2=0x55bfe9bf8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57400640 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bf8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e5d000640 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bfee20 0x55bfe9bf9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57400640 1 -- 172.21.15.18:0/1501780006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bfee20 msgr2=0x55bfe9bf9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57400640 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bfee20 0x55bfe9bf9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e57400640 1 -- 172.21.15.18:0/1501780006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe9c74390 con 0x55bfe9bf6610 2024-04-16T17:08:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.007+0000 7f4e5d000640 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bfee20 0x55bfe9bf9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.008+0000 7f4e57400640 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf8fe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4e44012010 tx=0x7f4e44003a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19431f6c267bf66e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.008+0000 7f4e42000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.008+0000 7f4e42000640 1 -- 172.21.15.18:0/1501780006 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44005250 con 0x55bfe9bf6610 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.008+0000 7f4e61e38740 1 -- 172.21.15.18:0/1501780006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e4401cd90 con 0x55bfe9bf6610 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.008+0000 7f4e42000640 1 -- 172.21.15.18:0/1501780006 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f4e440053f0 con 0x55bfe9bf6610 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.008+0000 7f4e42000640 1 -- 172.21.15.18:0/1501780006 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e4403e050 con 0x55bfe9bf6610 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.008+0000 7f4e61e38740 10 client.26801 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.008+0000 7f4e61e38740 1 -- 172.21.15.18:0/1501780006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfe9c75660 con 0x55bfe9bf6610 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e42000640 1 -- 172.21.15.18:0/1501780006 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4401cd90 con 0x55bfe9bf6610 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e42000640 1 -- 172.21.15.18:0/1501780006 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e44003de0 con 0x55bfe9bf6610 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e61e38740 10 client.26801 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e61e38740 10 client.26801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e61e38740 20 client.26801 populate_metadata read hostname 'smithi018' 2024-04-16T17:08:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bfe9c75d60 0x55bfe9c78180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e61e38740 4 client.26801 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e61e38740 1 -- 172.21.15.18:0/1501780006 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfe9b62010 con 0x55bfe9c75d60 2024-04-16T17:08:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.009+0000 7f4e57e00640 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bfe9c75d60 0x55bfe9c78180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.010+0000 7f4e57e00640 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bfe9c75d60 0x55bfe9c78180 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4a54c120de01bb server_cookie=2dd9d74bba265c4 in_seq=0 out_seq=0 2024-04-16T17:08:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.010+0000 7f4e42000640 10 client.26801 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e42000640 1 -- 172.21.15.18:0/1501780006 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfe9b62010 con 0x55bfe9c75d60 2024-04-16T17:08:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e42000640 10 client.26801 handle_command_reply: tid=0 2024-04-16T17:08:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 client.26801 shutdown 2024-04-16T17:08:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 -- 172.21.15.18:0/1501780006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 msgr2=0x55bfe9bf8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf8fe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4e44012010 tx=0x7f4e44003a70 comp rx=0 tx=0).stop 2024-04-16T17:08:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 -- 172.21.15.18:0/1501780006 shutdown_connections 2024-04-16T17:08:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bfe9c75d60 0x55bfe9c78180 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bfe9bfee20 0x55bfe9bf9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bfe9bf6610 0x55bfe9bf8fe0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 --2- 172.21.15.18:0/1501780006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfe9bf58a0 0x55bfe9bf8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 -- 172.21.15.18:0/1501780006 >> 172.21.15.18:0/1501780006 conn(0x55bfe9b905d0 msgr2=0x55bfe9bfa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e42000640 0 client.26801 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:08:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 -- 172.21.15.18:0/1501780006 shutdown_connections 2024-04-16T17:08:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.145+0000 7f4e61e38740 1 -- 172.21.15.18:0/1501780006 wait complete. 2024-04-16T17:08:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.146+0000 7f4e61e38740 20 client.26801 trim_cache size 0 max 16384 2024-04-16T17:08:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:08:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:08:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:08:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.147+0000 7f4e5f000640 1 -- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e3c0a3cc0 msgr2=0x7f4e3c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.147+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e3c0a3cc0 0x7f4e3c0a6180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55bfe9bf1720 tx=0x7f4e44009260 comp rx=0 tx=0).stop 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.147+0000 7f4e5f000640 1 -- 172.21.15.18:0/2882698657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e581008c0 msgr2=0x7f4e581a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.147+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e581008c0 0x7f4e581a0770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9f0 tx=0x7f4e4800edf0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.148+0000 7f4e5f000640 1 -- 172.21.15.18:0/2882698657 shutdown_connections 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.148+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e3c0a3cc0 0x7f4e3c0a6180 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.148+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58107060 0x7f4e5819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.148+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e58101280 0x7f4e581a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.148+0000 7f4e5f000640 1 --2- 172.21.15.18:0/2882698657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e581008c0 0x7f4e581a0770 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.148+0000 7f4e5f000640 1 -- 172.21.15.18:0/2882698657 >> 172.21.15.18:0/2882698657 conn(0x7f4e58076240 msgr2=0x7f4e580fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.148+0000 7f4e5f000640 1 -- 172.21.15.18:0/2882698657 shutdown_connections 2024-04-16T17:08:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:08:57.148+0000 7f4e5f000640 1 -- 172.21.15.18:0/2882698657 wait complete. 2024-04-16T17:08:57.547 INFO:tasks.workunit.client.0.smithi018.stdout: 1 145260 4.03 MB/sec execute 70 sec latency 41.347 ms 2024-04-16T17:08:57.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:57 smithi043 ceph-mon[36901]: pgmap v129: 129 pgs: 129 active+clean; 41 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-04-16T17:08:57.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:57 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:08:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:57 smithi022 ceph-mon[36023]: pgmap v129: 129 pgs: 129 active+clean; 41 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-04-16T17:08:57.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:57 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:08:58.207 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:57 smithi018 ceph-mon[31776]: pgmap v129: 129 pgs: 129 active+clean; 41 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-04-16T17:08:58.207 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:57 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:08:58.548 INFO:tasks.workunit.client.0.smithi018.stdout: 1 145674 4.04 MB/sec execute 71 sec latency 261.248 ms 2024-04-16T17:08:59.552 INFO:tasks.workunit.client.0.smithi018.stdout: 1 145877 3.98 MB/sec execute 72 sec latency 54.128 ms 2024-04-16T17:08:59.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:59 smithi043 ceph-mon[36901]: pgmap v130: 129 pgs: 129 active+clean; 41 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-04-16T17:08:59.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:59 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:08:59.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:59 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:08:59.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:08:59 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:08:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:59 smithi022 ceph-mon[36023]: pgmap v130: 129 pgs: 129 active+clean; 41 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-04-16T17:08:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:59 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:08:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:59 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:08:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:08:59 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:09:00.552 INFO:tasks.workunit.client.0.smithi018.stdout: 1 146411 3.94 MB/sec execute 73 sec latency 52.510 ms 2024-04-16T17:09:00.706 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:59 smithi018 ceph-mon[31776]: pgmap v130: 129 pgs: 129 active+clean; 41 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-04-16T17:09:00.707 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:59 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:09:00.707 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:59 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:09:00.707 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:08:59 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:09:01.552 INFO:tasks.workunit.client.0.smithi018.stdout: 1 147622 3.95 MB/sec execute 74 sec latency 17.014 ms 2024-04-16T17:09:01.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:01 smithi043 ceph-mon[36901]: pgmap v131: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-16T17:09:01.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:01 smithi018 ceph-mon[31776]: pgmap v131: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-16T17:09:01.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:01 smithi022 ceph-mon[36023]: pgmap v131: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-16T17:09:02.552 INFO:tasks.workunit.client.0.smithi018.stdout: 1 147911 3.92 MB/sec execute 75 sec latency 59.275 ms 2024-04-16T17:09:03.552 INFO:tasks.workunit.client.0.smithi018.stdout: 1 148043 3.88 MB/sec execute 76 sec latency 720.543 ms 2024-04-16T17:09:03.725 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:03 smithi022 ceph-mon[36023]: pgmap v132: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:09:03.753 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:03 smithi043 ceph-mon[36901]: pgmap v132: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:09:04.552 INFO:tasks.workunit.client.0.smithi018.stdout: 1 148472 3.89 MB/sec execute 77 sec latency 826.870 ms 2024-04-16T17:09:04.641 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:03 smithi018 ceph-mon[31776]: pgmap v132: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:09:05.552 INFO:tasks.workunit.client.0.smithi018.stdout: 1 149420 3.92 MB/sec execute 78 sec latency 18.852 ms 2024-04-16T17:09:05.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:05 smithi043 ceph-mon[36901]: pgmap v133: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:09:05.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:05 smithi022 ceph-mon[36023]: pgmap v133: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:09:06.552 INFO:tasks.workunit.client.0.smithi018.stdout: 1 150207 3.90 MB/sec execute 79 sec latency 12.169 ms 2024-04-16T17:09:07.553 INFO:tasks.workunit.client.0.smithi018.stdout: 1 151408 3.91 MB/sec execute 80 sec latency 21.170 ms 2024-04-16T17:09:07.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:07 smithi043 ceph-mon[36901]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-04-16T17:09:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:07 smithi022 ceph-mon[36023]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-04-16T17:09:08.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:05 smithi018 ceph-mon[31776]: pgmap v133: 129 pgs: 129 active+clean; 34 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:09:08.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:07 smithi018 ceph-mon[31776]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-04-16T17:09:08.554 INFO:tasks.workunit.client.0.smithi018.stdout: 1 151770 3.91 MB/sec execute 81 sec latency 462.512 ms 2024-04-16T17:09:09.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 152779 3.96 MB/sec execute 82 sec latency 633.110 ms 2024-04-16T17:09:09.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:09 smithi043 ceph-mon[36901]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-04-16T17:09:09.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:09 smithi022 ceph-mon[36023]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-04-16T17:09:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 -- 172.21.15.18:0/3034426357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 msgr2=0x7fe78c1128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 --2- 172.21.15.18:0/3034426357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 0x7fe78c1128f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe778009900 tx=0x7fe77802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 -- 172.21.15.18:0/3034426357 shutdown_connections 2024-04-16T17:09:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 --2- 172.21.15.18:0/3034426357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe78c112e30 0x7fe78c115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 --2- 172.21.15.18:0/3034426357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 0x7fe78c1128f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 --2- 172.21.15.18:0/3034426357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe78c0743a0 0x7fe78c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 -- 172.21.15.18:0/3034426357 >> 172.21.15.18:0/3034426357 conn(0x7fe78c0fd910 msgr2=0x7fe78c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 -- 172.21.15.18:0/3034426357 shutdown_connections 2024-04-16T17:09:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.294+0000 7fe78be00640 1 -- 172.21.15.18:0/3034426357 wait complete. 2024-04-16T17:09:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.295+0000 7fe78be00640 1 Processor -- start 2024-04-16T17:09:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78be00640 1 -- start start 2024-04-16T17:09:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe78c0743a0 0x7fe78c1a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 0x7fe78c1a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe78c112e30 0x7fe78c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c118800 con 0x7fe78c112e30 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c118680 con 0x7fe78c074cc0 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c118980 con 0x7fe78c0743a0 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe78c0743a0 0x7fe78c1a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 0x7fe78c1a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe78c0743a0 0x7fe78c1a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39538/0 (socket says 172.21.15.18:39538) 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.296+0000 7fe78aa00640 1 -- 172.21.15.18:0/1389154076 learned_addr learned my addr 172.21.15.18:0/1389154076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.297+0000 7fe78b400640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe78c112e30 0x7fe78c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.297+0000 7fe78a000640 1 -- 172.21.15.18:0/1389154076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe78c0743a0 msgr2=0x7fe78c1a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.297+0000 7fe78a000640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe78c0743a0 0x7fe78c1a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.297+0000 7fe78a000640 1 -- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe78c112e30 msgr2=0x7fe78c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.297+0000 7fe78a000640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe78c112e30 0x7fe78c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.297+0000 7fe78a000640 1 -- 172.21.15.18:0/1389154076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c19f420 con 0x7fe78c074cc0 2024-04-16T17:09:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.297+0000 7fe78b400640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe78c112e30 0x7fe78c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.297+0000 7fe78a000640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 0x7fe78c1a4f90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe778000c00 tx=0x7fe77802cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.298+0000 7fe783400640 1 -- 172.21.15.18:0/1389154076 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778041070 con 0x7fe78c074cc0 2024-04-16T17:09:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.298+0000 7fe783400640 1 -- 172.21.15.18:0/1389154076 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe778004450 con 0x7fe78c074cc0 2024-04-16T17:09:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.298+0000 7fe78be00640 1 -- 172.21.15.18:0/1389154076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c19f6b0 con 0x7fe78c074cc0 2024-04-16T17:09:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.298+0000 7fe783400640 1 -- 172.21.15.18:0/1389154076 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778003e20 con 0x7fe78c074cc0 2024-04-16T17:09:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.298+0000 7fe78be00640 1 -- 172.21.15.18:0/1389154076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1ab7f0 con 0x7fe78c074cc0 2024-04-16T17:09:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.302+0000 7fe79476c740 1 Processor -- start 2024-04-16T17:09:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.302+0000 7fe79476c740 1 -- start start 2024-04-16T17:09:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.302+0000 7fe79476c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07c7f0 0x56307d07b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.302+0000 7fe79476c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d07bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe79476c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d085000 0x56307d0873f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe79476c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56307cfe0780 con 0x56307d07ba80 2024-04-16T17:09:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe79476c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56307cf86f40 con 0x56307d07c7f0 2024-04-16T17:09:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe79476c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56307cfdc740 con 0x56307d085000 2024-04-16T17:09:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07c7f0 0x56307d07b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d07bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07c7f0 0x56307d07b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51870/0 (socket says 172.21.15.18:51870) 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78b400640 1 -- 172.21.15.18:0/3481281077 learned_addr learned my addr 172.21.15.18:0/3481281077 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78b400640 1 -- 172.21.15.18:0/3481281077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d085000 msgr2=0x56307d0873f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78b400640 1 --2- 172.21.15.18:0/3481281077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d085000 0x56307d0873f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78b400640 1 -- 172.21.15.18:0/3481281077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 msgr2=0x56307d07bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78b400640 1 --2- 172.21.15.18:0/3481281077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d07bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78b400640 1 -- 172.21.15.18:0/3481281077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307ccf3210 con 0x56307d07c7f0 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.303+0000 7fe78aa00640 1 --2- 172.21.15.18:0/3481281077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d07bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.304+0000 7fe78b400640 1 --2- 172.21.15.18:0/3481281077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07c7f0 0x56307d07b540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe784009ef0 tx=0x7fe784010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ec86c7df58ac8c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.304+0000 7fe783400640 1 -- 172.21.15.18:0/1389154076 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe778044070 con 0x7fe78c074cc0 2024-04-16T17:09:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.304+0000 7fe777e00640 1 -- 172.21.15.18:0/3481281077 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784016070 con 0x56307d07c7f0 2024-04-16T17:09:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.304+0000 7fe777e00640 1 -- 172.21.15.18:0/3481281077 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78401bb40 con 0x56307d07c7f0 2024-04-16T17:09:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.304+0000 7fe777e00640 1 -- 172.21.15.18:0/3481281077 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784012c40 con 0x56307d07c7f0 2024-04-16T17:09:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 -- 172.21.15.18:0/3481281077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07c7f0 msgr2=0x56307d07b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 --2- 172.21.15.18:0/3481281077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07c7f0 0x56307d07b540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe784009ef0 tx=0x7fe784010740 comp rx=0 tx=0).stop 2024-04-16T17:09:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 -- 172.21.15.18:0/3481281077 shutdown_connections 2024-04-16T17:09:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 --2- 172.21.15.18:0/3481281077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d085000 0x56307d0873f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 --2- 172.21.15.18:0/3481281077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d07bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 --2- 172.21.15.18:0/3481281077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07c7f0 0x56307d07b540 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 -- 172.21.15.18:0/3481281077 >> 172.21.15.18:0/3481281077 conn(0x56307d0165d0 msgr2=0x56307d076770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 -- 172.21.15.18:0/3481281077 shutdown_connections 2024-04-16T17:09:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.305+0000 7fe79476c740 1 -- 172.21.15.18:0/3481281077 wait complete. 2024-04-16T17:09:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.306+0000 7fe79476c740 1 -- start start 2024-04-16T17:09:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.306+0000 7fe79476c740 1 Processor -- start 2024-04-16T17:09:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.306+0000 7fe783400640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe7700a3c90 0x7fe7700a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe783400640 1 -- 172.21.15.18:0/1389154076 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe7780f2550 con 0x7fe78c074cc0 2024-04-16T17:09:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe78b400640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe7700a3c90 0x7fe7700a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe79476c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07ba80 0x56307d081f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe79476c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07c7f0 0x56307d082440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe79476c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d082980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe79476c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56307d087d30 con 0x56307d07c7f0 2024-04-16T17:09:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe79476c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56307cf9a390 con 0x56307d085000 2024-04-16T17:09:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe79476c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56307d087eb0 con 0x56307d07ba80 2024-04-16T17:09:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.307+0000 7fe79476c740 10 client.0 _ll_register_callbacks cb 0x56307cf83f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07ba80 0x56307d081f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07c7f0 0x56307d082440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07ba80 0x56307d081f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39542/0 (socket says 172.21.15.18:39542) 2024-04-16T17:09:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78aa00640 1 -- 172.21.15.18:0/1125872768 learned_addr learned my addr 172.21.15.18:0/1125872768 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78aa00640 1 -- 172.21.15.18:0/1125872768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 msgr2=0x56307d082980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78aa00640 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d082980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78aa00640 1 -- 172.21.15.18:0/1125872768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07c7f0 msgr2=0x56307d082440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78aa00640 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07c7f0 0x56307d082440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.308+0000 7fe78aa00640 1 -- 172.21.15.18:0/1125872768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307cdbf620 con 0x56307d07ba80 2024-04-16T17:09:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.309+0000 7fe78b400640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe7700a3c90 0x7fe7700a6150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe784012720 tx=0x7fe784005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.309+0000 7fe78aa00640 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07ba80 0x56307d081f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00bc20 tx=0x7fe77c004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7c403146c4a320 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.309+0000 7fe774c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.309+0000 7fe774c00640 1 -- 172.21.15.18:0/1125872768 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011440 con 0x56307d07ba80 2024-04-16T17:09:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.309+0000 7fe774c00640 1 -- 172.21.15.18:0/1125872768 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe77c0115e0 con 0x56307d07ba80 2024-04-16T17:09:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.309+0000 7fe79476c740 1 -- 172.21.15.18:0/1125872768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe78401bb40 con 0x56307d07ba80 2024-04-16T17:09:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.310+0000 7fe774c00640 1 -- 172.21.15.18:0/1125872768 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe77c0117e0 con 0x56307d07ba80 2024-04-16T17:09:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.310+0000 7fe79476c740 10 client.26785 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.310+0000 7fe79476c740 1 -- 172.21.15.18:0/1125872768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56307d093f20 con 0x56307d07ba80 2024-04-16T17:09:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe774c00640 1 -- 172.21.15.18:0/1125872768 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78401bb40 con 0x56307d07ba80 2024-04-16T17:09:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe774c00640 1 -- 172.21.15.18:0/1125872768 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe77c025b70 con 0x56307d07ba80 2024-04-16T17:09:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe79476c740 10 client.26785 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe79476c740 10 client.26785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe79476c740 20 client.26785 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe79476c740 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56307d095390 0x56307d097780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe79476c740 4 client.26785 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe79476c740 1 -- 172.21.15.18:0/1125872768 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56307cfee1a0 con 0x56307d095390 2024-04-16T17:09:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.311+0000 7fe78a000640 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56307d095390 0x56307d097780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.312+0000 7fe78a000640 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56307d095390 0x56307d097780 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276792fce4289fda server_cookie=447266f853c3f87e in_seq=0 out_seq=0 2024-04-16T17:09:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.312+0000 7fe774c00640 10 client.26785 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe774c00640 1 -- 172.21.15.18:0/1125872768 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56307cfee1a0 con 0x56307d095390 2024-04-16T17:09:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe774c00640 10 client.26785 handle_command_reply: tid=0 2024-04-16T17:09:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 client.26785 shutdown 2024-04-16T17:09:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 -- 172.21.15.18:0/1125872768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07ba80 msgr2=0x56307d081f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07ba80 0x56307d081f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00bc20 tx=0x7fe77c004560 comp rx=0 tx=0).stop 2024-04-16T17:09:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 -- 172.21.15.18:0/1125872768 shutdown_connections 2024-04-16T17:09:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56307d095390 0x56307d097780 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d082980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07c7f0 0x56307d082440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 --2- 172.21.15.18:0/1125872768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07ba80 0x56307d081f00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe79476c740 1 -- 172.21.15.18:0/1125872768 >> 172.21.15.18:0/1125872768 conn(0x56307d0165d0 msgr2=0x56307d0784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.313+0000 7fe774c00640 0 client.26785 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.315+0000 7fe79476c740 1 -- 172.21.15.18:0/1125872768 shutdown_connections 2024-04-16T17:09:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.315+0000 7fe79476c740 1 -- 172.21.15.18:0/1125872768 wait complete. 2024-04-16T17:09:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.315+0000 7fe79476c740 20 client.26785 trim_cache size 0 max 16384 2024-04-16T17:09:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.344+0000 7fe79476c740 1 Processor -- start 2024-04-16T17:09:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.344+0000 7fe79476c740 1 -- start start 2024-04-16T17:09:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe79476c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07ba80 0x56307d090c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe79476c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 0x56307d081060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe79476c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d085000 0x56307d078270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe79476c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b8320 con 0x56307d085000 2024-04-16T17:09:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe79476c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b5bf0 con 0x56307d07ba80 2024-04-16T17:09:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe79476c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b3a40 con 0x56307d07c7f0 2024-04-16T17:09:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07ba80 0x56307d090c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe78b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d085000 0x56307d078270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07ba80 0x56307d090c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51892/0 (socket says 172.21.15.18:51892) 2024-04-16T17:09:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.345+0000 7fe78aa00640 1 -- 172.21.15.18:0/3897266214 learned_addr learned my addr 172.21.15.18:0/3897266214 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.346+0000 7fe78a000640 1 --2- 172.21.15.18:0/3897266214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 0x56307d081060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.346+0000 7fe78aa00640 1 -- 172.21.15.18:0/3897266214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 msgr2=0x56307d081060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.346+0000 7fe78aa00640 1 --2- 172.21.15.18:0/3897266214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 0x56307d081060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.346+0000 7fe78aa00640 1 -- 172.21.15.18:0/3897266214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d085000 msgr2=0x56307d078270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.346+0000 7fe78aa00640 1 --2- 172.21.15.18:0/3897266214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d085000 0x56307d078270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.346+0000 7fe78aa00640 1 -- 172.21.15.18:0/3897266214 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307d087760 con 0x56307d07ba80 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.346+0000 7fe78aa00640 1 --2- 172.21.15.18:0/3897266214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07ba80 0x56307d090c30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe77c005bc0 tx=0x7fe77c005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf285f9aeac13c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.347+0000 7fe777400640 1 -- 172.21.15.18:0/3897266214 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00b5b0 con 0x56307d07ba80 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.347+0000 7fe777400640 1 -- 172.21.15.18:0/3897266214 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77c011440 con 0x56307d07ba80 2024-04-16T17:09:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.347+0000 7fe777400640 1 -- 172.21.15.18:0/3897266214 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c025470 con 0x56307d07ba80 2024-04-16T17:09:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.348+0000 7fe79476c740 1 -- 172.21.15.18:0/3897266214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07ba80 msgr2=0x56307d090c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.348+0000 7fe79476c740 1 --2- 172.21.15.18:0/3897266214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07ba80 0x56307d090c30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe77c005bc0 tx=0x7fe77c005fb0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.348+0000 7fe79476c740 1 -- 172.21.15.18:0/3897266214 shutdown_connections 2024-04-16T17:09:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.348+0000 7fe79476c740 1 --2- 172.21.15.18:0/3897266214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d085000 0x56307d078270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.348+0000 7fe79476c740 1 --2- 172.21.15.18:0/3897266214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 0x56307d081060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.348+0000 7fe79476c740 1 --2- 172.21.15.18:0/3897266214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d07ba80 0x56307d090c30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.348+0000 7fe79476c740 1 -- 172.21.15.18:0/3897266214 >> 172.21.15.18:0/3897266214 conn(0x56307d0165d0 msgr2=0x56307d077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.349+0000 7fe79476c740 1 -- 172.21.15.18:0/3897266214 shutdown_connections 2024-04-16T17:09:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.349+0000 7fe79476c740 1 -- 172.21.15.18:0/3897266214 wait complete. 2024-04-16T17:09:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.349+0000 7fe79476c740 1 -- start start 2024-04-16T17:09:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.349+0000 7fe79476c740 1 Processor -- start 2024-04-16T17:09:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe79476c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d08d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe79476c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 0x56307d07e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe79476c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d07ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe79476c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b40b0 con 0x56307d07ba80 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe79476c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b5520 con 0x56307d085000 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe79476c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b4780 con 0x56307d07c7f0 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe79476c740 10 client.0 _ll_register_callbacks cb 0x7fe77c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe78b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d07ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d08d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe78b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d07ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51900/0 (socket says 172.21.15.18:51900) 2024-04-16T17:09:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.350+0000 7fe78b400640 1 -- 172.21.15.18:0/471592057 learned_addr learned my addr 172.21.15.18:0/471592057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.351+0000 7fe78a000640 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 0x56307d07e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.351+0000 7fe78b400640 1 -- 172.21.15.18:0/471592057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 msgr2=0x56307d07e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.351+0000 7fe78b400640 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 0x56307d07e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.351+0000 7fe78b400640 1 -- 172.21.15.18:0/471592057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 msgr2=0x56307d08d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.351+0000 7fe78b400640 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d08d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.351+0000 7fe78b400640 1 -- 172.21.15.18:0/471592057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307d07f290 con 0x56307d085000 2024-04-16T17:09:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.351+0000 7fe78b400640 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d07ecd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe784009ef0 tx=0x7fe784010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c7ae9ab0142df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.351+0000 7fe774c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.352+0000 7fe774c00640 1 -- 172.21.15.18:0/471592057 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784016040 con 0x56307d085000 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.352+0000 7fe774c00640 1 -- 172.21.15.18:0/471592057 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fe784012d40 con 0x56307d085000 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.352+0000 7fe79476c740 1 -- 172.21.15.18:0/471592057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77c011440 con 0x56307d085000 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.352+0000 7fe774c00640 1 -- 172.21.15.18:0/471592057 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe78401b460 con 0x56307d085000 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.352+0000 7fe79476c740 10 client.26816 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.352+0000 7fe79476c740 1 -- 172.21.15.18:0/471592057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56307d10c680 con 0x56307d085000 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.352+0000 7fe774c00640 1 -- 172.21.15.18:0/471592057 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77c011440 con 0x56307d085000 2024-04-16T17:09:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.352+0000 7fe774c00640 1 -- 172.21.15.18:0/471592057 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe78400f040 con 0x56307d085000 2024-04-16T17:09:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe79476c740 10 client.26816 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe79476c740 10 client.26816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe79476c740 20 client.26816 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe79476c740 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56307d10cb60 0x56307d10ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe79476c740 4 client.26816 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:09:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe79476c740 1 -- 172.21.15.18:0/471592057 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56307cfe8010 con 0x56307d10cb60 2024-04-16T17:09:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe78aa00640 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56307d10cb60 0x56307d10ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe78aa00640 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56307d10cb60 0x56307d10ef80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba8f441f3d6cc4a server_cookie=7297471f89216b64 in_seq=0 out_seq=0 2024-04-16T17:09:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.353+0000 7fe774c00640 10 client.26816 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:10.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:09 smithi018 ceph-mon[31776]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-04-16T17:09:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe774c00640 1 -- 172.21.15.18:0/471592057 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56307cfe8010 con 0x56307d10cb60 2024-04-16T17:09:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe774c00640 10 client.26816 handle_command_reply: tid=0 2024-04-16T17:09:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 client.26816 shutdown 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 -- 172.21.15.18:0/471592057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 msgr2=0x56307d07ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d07ecd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe784009ef0 tx=0x7fe784010740 comp rx=0 tx=0).stop 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 -- 172.21.15.18:0/471592057 shutdown_connections 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56307d10cb60 0x56307d10ef80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56307d085000 0x56307d07ecd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56307d07c7f0 0x56307d07e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 --2- 172.21.15.18:0/471592057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56307d07ba80 0x56307d08d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 -- 172.21.15.18:0/471592057 >> 172.21.15.18:0/471592057 conn(0x56307d0165d0 msgr2=0x56307d0783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe774c00640 0 client.26816 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 -- 172.21.15.18:0/471592057 shutdown_connections 2024-04-16T17:09:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.509+0000 7fe79476c740 1 -- 172.21.15.18:0/471592057 wait complete. 2024-04-16T17:09:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.510+0000 7fe79476c740 20 client.26816 trim_cache size 0 max 16384 2024-04-16T17:09:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:09:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:09:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:09:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:09:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.510+0000 7fe78be00640 1 -- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe7700a3c90 msgr2=0x7fe7700a6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.510+0000 7fe78be00640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe7700a3c90 0x7fe7700a6150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe784012720 tx=0x7fe784005fd0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 -- 172.21.15.18:0/1389154076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 msgr2=0x7fe78c1a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 0x7fe78c1a4f90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe778000c00 tx=0x7fe77802cc90 comp rx=0 tx=0).stop 2024-04-16T17:09:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 -- 172.21.15.18:0/1389154076 shutdown_connections 2024-04-16T17:09:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe7700a3c90 0x7fe7700a6150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe78c112e30 0x7fe78c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe78c074cc0 0x7fe78c1a4f90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 --2- 172.21.15.18:0/1389154076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe78c0743a0 0x7fe78c1a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 -- 172.21.15.18:0/1389154076 >> 172.21.15.18:0/1389154076 conn(0x7fe78c0fd910 msgr2=0x7fe78c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 -- 172.21.15.18:0/1389154076 shutdown_connections 2024-04-16T17:09:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:10.511+0000 7fe78be00640 1 -- 172.21.15.18:0/1389154076 wait complete. 2024-04-16T17:09:10.554 INFO:tasks.workunit.client.0.smithi018.stdout: 1 153329 3.92 MB/sec execute 83 sec latency 54.761 ms 2024-04-16T17:09:11.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 154361 3.93 MB/sec execute 84 sec latency 20.732 ms 2024-04-16T17:09:11.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:11 smithi043 ceph-mon[36901]: pgmap v136: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-04-16T17:09:11.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:11 smithi022 ceph-mon[36023]: pgmap v136: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-04-16T17:09:12.554 INFO:tasks.workunit.client.0.smithi018.stdout: 1 155547 3.97 MB/sec execute 85 sec latency 17.612 ms 2024-04-16T17:09:13.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 156189 3.98 MB/sec execute 86 sec latency 75.343 ms 2024-04-16T17:09:13.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:13 smithi043 ceph-mon[36901]: pgmap v137: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-16T17:09:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:13 smithi022 ceph-mon[36023]: pgmap v137: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-16T17:09:14.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 156394 3.95 MB/sec execute 87 sec latency 115.914 ms 2024-04-16T17:09:14.961 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:11 smithi018 ceph-mon[31776]: pgmap v136: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-04-16T17:09:14.961 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:13 smithi018 ceph-mon[31776]: pgmap v137: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-16T17:09:15.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 156890 3.92 MB/sec execute 88 sec latency 44.096 ms 2024-04-16T17:09:15.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:15 smithi043 ceph-mon[36901]: pgmap v138: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-16T17:09:15.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:15 smithi022 ceph-mon[36023]: pgmap v138: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-16T17:09:16.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 158029 3.93 MB/sec execute 89 sec latency 14.648 ms 2024-04-16T17:09:17.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 159415 3.98 MB/sec execute 90 sec latency 26.957 ms 2024-04-16T17:09:17.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:17 smithi043 ceph-mon[36901]: pgmap v139: 129 pgs: 129 active+clean; 37 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-16T17:09:17.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:17 smithi022 ceph-mon[36023]: pgmap v139: 129 pgs: 129 active+clean; 37 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-16T17:09:18.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 159866 3.99 MB/sec execute 91 sec latency 89.486 ms 2024-04-16T17:09:19.059 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:15 smithi018 ceph-mon[31776]: pgmap v138: 129 pgs: 129 active+clean; 38 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-16T17:09:19.059 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:17 smithi018 ceph-mon[31776]: pgmap v139: 129 pgs: 129 active+clean; 37 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-16T17:09:19.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 159952 3.95 MB/sec execute 92 sec latency 728.242 ms 2024-04-16T17:09:19.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:19 smithi043 ceph-mon[36901]: pgmap v140: 129 pgs: 129 active+clean; 37 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:09:19.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:19 smithi022 ceph-mon[36023]: pgmap v140: 129 pgs: 129 active+clean; 37 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:09:20.020 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:19 smithi018 ceph-mon[31776]: pgmap v140: 129 pgs: 129 active+clean; 37 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:09:20.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 160608 3.92 MB/sec execute 93 sec latency 53.760 ms 2024-04-16T17:09:21.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:09:21.466 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:09:21.555 INFO:tasks.workunit.client.0.smithi018.stdout: 1 161802 3.93 MB/sec execute 94 sec latency 19.623 ms 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.668+0000 7f115f200640 1 -- 172.21.15.18:0/1106116697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1160074cd0 msgr2=0x7f116010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.668+0000 7f115f200640 1 --2- 172.21.15.18:0/1106116697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1160074cd0 0x7f116010e180 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1148009900 tx=0x7f114802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.669+0000 7f115f200640 1 -- 172.21.15.18:0/1106116697 shutdown_connections 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.669+0000 7f115f200640 1 --2- 172.21.15.18:0/1106116697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116010e6c0 0x7f1160110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.669+0000 7f115f200640 1 --2- 172.21.15.18:0/1106116697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1160074cd0 0x7f116010e180 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.669+0000 7f115f200640 1 --2- 172.21.15.18:0/1106116697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11600743b0 0x7f1160074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.669+0000 7f115f200640 1 -- 172.21.15.18:0/1106116697 >> 172.21.15.18:0/1106116697 conn(0x7f11600fd950 msgr2=0x7f11600ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.669+0000 7f115f200640 1 -- 172.21.15.18:0/1106116697 shutdown_connections 2024-04-16T17:09:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.669+0000 7f115f200640 1 -- 172.21.15.18:0/1106116697 wait complete. 2024-04-16T17:09:21.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.670+0000 7f115f200640 1 Processor -- start 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.670+0000 7f115f200640 1 -- start start 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11600743b0 0x7f11601a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1160074cd0 0x7f11601a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116010e6c0 0x7f116019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1160114400 con 0x7f1160074cd0 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1160114280 con 0x7f11600743b0 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1160114580 con 0x7f116010e6c0 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11600743b0 0x7f11601a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116010e6c0 0x7f116019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11600743b0 0x7f11601a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46702/0 (socket says 172.21.15.18:46702) 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115de00640 1 -- 172.21.15.18:0/2051222645 learned_addr learned my addr 172.21.15.18:0/2051222645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:21.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115d400640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1160074cd0 0x7f11601a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115de00640 1 -- 172.21.15.18:0/2051222645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116010e6c0 msgr2=0x7f116019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115de00640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116010e6c0 0x7f116019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115de00640 1 -- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1160074cd0 msgr2=0x7f11601a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115de00640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1160074cd0 0x7f11601a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115de00640 1 -- 172.21.15.18:0/2051222645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116019af70 con 0x7f11600743b0 2024-04-16T17:09:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.671+0000 7f115d400640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1160074cd0 0x7f11601a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:21.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.672+0000 7f115de00640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11600743b0 0x7f11601a0630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1150009d50 tx=0x7f115000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:21.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.672+0000 7f1156a00640 1 -- 172.21.15.18:0/2051222645 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150003b70 con 0x7f11600743b0 2024-04-16T17:09:21.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.672+0000 7f1156a00640 1 -- 172.21.15.18:0/2051222645 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1150003d10 con 0x7f11600743b0 2024-04-16T17:09:21.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.672+0000 7f1156a00640 1 -- 172.21.15.18:0/2051222645 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150010a20 con 0x7f11600743b0 2024-04-16T17:09:21.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.672+0000 7f115f200640 1 -- 172.21.15.18:0/2051222645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116019b260 con 0x7f11600743b0 2024-04-16T17:09:21.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.672+0000 7f115f200640 1 -- 172.21.15.18:0/2051222645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11601a7480 con 0x7f11600743b0 2024-04-16T17:09:21.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.676+0000 7f1167d8e740 1 Processor -- start 2024-04-16T17:09:21.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.676+0000 7f1167d8e740 1 -- start start 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.676+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544fee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.676+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c544ff7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.676+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.676+0000 7f1167d8e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c544f54120 con 0x55c544feecb0 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.676+0000 7f1167d8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c544efa340 con 0x55c544ff7890 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.676+0000 7f1167d8e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c544f500e0 con 0x55c544fee390 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544fee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46710/0 (socket says 172.21.15.18:46710) 2024-04-16T17:09:21.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115d400640 1 -- 172.21.15.18:0/4248169991 learned_addr learned my addr 172.21.15.18:0/4248169991 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:21.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f1156a00640 1 -- 172.21.15.18:0/2051222645 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1150010bc0 con 0x7f11600743b0 2024-04-16T17:09:21.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115d400640 1 -- 172.21.15.18:0/4248169991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 msgr2=0x55c544fee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:21.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115d400640 1 --2- 172.21.15.18:0/4248169991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544fee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115d400640 1 -- 172.21.15.18:0/4248169991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 msgr2=0x55c544ff7350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:09:21.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115d400640 1 --2- 172.21.15.18:0/4248169991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c544ff7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.677+0000 7f115d400640 1 -- 172.21.15.18:0/4248169991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c544d307d0 con 0x55c544ff7890 2024-04-16T17:09:21.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.678+0000 7f115d400640 1 --2- 172.21.15.18:0/4248169991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff9c80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1148000c00 tx=0x7f114802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9df48639caf864 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:21.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.678+0000 7f114f400640 1 -- 172.21.15.18:0/4248169991 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148004240 con 0x55c544ff7890 2024-04-16T17:09:21.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.678+0000 7f114f400640 1 -- 172.21.15.18:0/4248169991 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114802cdf0 con 0x55c544ff7890 2024-04-16T17:09:21.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.678+0000 7f114f400640 1 -- 172.21.15.18:0/4248169991 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11480048c0 con 0x55c544ff7890 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1156a00640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11440a3da0 0x7f11440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 -- 172.21.15.18:0/4248169991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 msgr2=0x55c544ff9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 --2- 172.21.15.18:0/4248169991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff9c80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1148000c00 tx=0x7f114802c660 comp rx=0 tx=0).stop 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1156a00640 1 -- 172.21.15.18:0/2051222645 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f11500d1270 con 0x7f11600743b0 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f115e800640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11440a3da0 0x7f11440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 -- 172.21.15.18:0/4248169991 shutdown_connections 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 --2- 172.21.15.18:0/4248169991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff9c80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 --2- 172.21.15.18:0/4248169991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c544ff7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 --2- 172.21.15.18:0/4248169991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544fee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 -- 172.21.15.18:0/4248169991 >> 172.21.15.18:0/4248169991 conn(0x55c544fe8ac0 msgr2=0x55c544fe8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 -- 172.21.15.18:0/4248169991 shutdown_connections 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.679+0000 7f1167d8e740 1 -- 172.21.15.18:0/4248169991 wait complete. 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.680+0000 7f115e800640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11440a3da0 0x7f11440a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1158005fd0 tx=0x7f1158005cc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:21.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.680+0000 7f1167d8e740 1 -- start start 2024-04-16T17:09:21.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.680+0000 7f1167d8e740 1 Processor -- start 2024-04-16T17:09:21.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c544ff4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f1167d8e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c544ffa620 con 0x55c544feecb0 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f1167d8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c544f0d710 con 0x55c544ff7890 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f1167d8e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c544ffa7a0 con 0x55c544fee390 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f1167d8e740 10 client.0 _ll_register_callbacks cb 0x55c544f08af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f115e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f115d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c544ff4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f115de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f115e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46716/0 (socket says 172.21.15.18:46716) 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.681+0000 7f115e800640 1 -- 172.21.15.18:0/3517737106 learned_addr learned my addr 172.21.15.18:0/3517737106 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f115e800640 1 -- 172.21.15.18:0/3517737106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 msgr2=0x55c544ff4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f115e800640 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f115e800640 1 -- 172.21.15.18:0/3517737106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 msgr2=0x55c544ff4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f115e800640 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c544ff4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f115e800640 1 -- 172.21.15.18:0/3517737106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c544c663f0 con 0x55c544ff7890 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f115e800640 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff52c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1158003c00 tx=0x7f1158004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e39ab5c4f5c0158c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:21.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f1143e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:21.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f1143e00640 1 -- 172.21.15.18:0/3517737106 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115801fd40 con 0x55c544ff7890 2024-04-16T17:09:21.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.682+0000 7f1167d8e740 1 -- 172.21.15.18:0/3517737106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114802cdf0 con 0x55c544ff7890 2024-04-16T17:09:21.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.683+0000 7f1143e00640 1 -- 172.21.15.18:0/3517737106 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1158011440 con 0x55c544ff7890 2024-04-16T17:09:21.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.683+0000 7f1143e00640 1 -- 172.21.15.18:0/3517737106 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1158011640 con 0x55c544ff7890 2024-04-16T17:09:21.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.683+0000 7f1167d8e740 10 client.26828 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:21.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.683+0000 7f1167d8e740 1 -- 172.21.15.18:0/3517737106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c544c260f0 con 0x55c544ff7890 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.683+0000 7f1143e00640 1 -- 172.21.15.18:0/3517737106 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114802cdf0 con 0x55c544ff7890 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.683+0000 7f1143e00640 1 -- 172.21.15.18:0/3517737106 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1158026070 con 0x55c544ff7890 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.683+0000 7f1167d8e740 10 client.26828 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.683+0000 7f1167d8e740 10 client.26828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.684+0000 7f1167d8e740 20 client.26828 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.684+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c545007bb0 0x55c545009fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.684+0000 7f1167d8e740 4 client.26828 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.684+0000 7f1167d8e740 1 -- 172.21.15.18:0/3517737106 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c544f61b40 con 0x55c545007bb0 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.684+0000 7f115de00640 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c545007bb0 0x55c545009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.684+0000 7f115de00640 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c545007bb0 0x55c545009fe0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb638c68fba414f0 server_cookie=35ea082fd7d7a87a in_seq=0 out_seq=0 2024-04-16T17:09:21.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.684+0000 7f1143e00640 10 client.26828 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:21.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1143e00640 1 -- 172.21.15.18:0/3517737106 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c544f61b40 con 0x55c545007bb0 2024-04-16T17:09:21.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1143e00640 10 client.26828 handle_command_reply: tid=0 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 client.26828 shutdown 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 -- 172.21.15.18:0/3517737106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 msgr2=0x55c544ff52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff52c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1158003c00 tx=0x7f1158004730 comp rx=0 tx=0).stop 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 -- 172.21.15.18:0/3517737106 shutdown_connections 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c545007bb0 0x55c545009fe0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544ff52c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c544ff4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3517737106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1143e00640 0 client.26828 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 -- 172.21.15.18:0/3517737106 >> 172.21.15.18:0/3517737106 conn(0x55c544fe8ac0 msgr2=0x55c544fea810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 -- 172.21.15.18:0/3517737106 shutdown_connections 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.686+0000 7f1167d8e740 1 -- 172.21.15.18:0/3517737106 wait complete. 2024-04-16T17:09:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.687+0000 7f1167d8e740 20 client.26828 trim_cache size 0 max 16384 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.714+0000 7f1167d8e740 1 Processor -- start 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f1167d8e740 1 -- start start 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c545003510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544fe8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f1167d8e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b3ea0 con 0x55c544feecb0 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f1167d8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b1770 con 0x55c544ff7890 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f1167d8e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11601af5c0 con 0x55c544fee390 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f115e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544fe8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f115e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544fe8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46720/0 (socket says 172.21.15.18:46720) 2024-04-16T17:09:21.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f115e800640 1 -- 172.21.15.18:0/616045630 learned_addr learned my addr 172.21.15.18:0/616045630 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f115de00640 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.715+0000 7f115d400640 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c545003510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f115e800640 1 -- 172.21.15.18:0/616045630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 msgr2=0x55c544ff6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f115e800640 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f115e800640 1 -- 172.21.15.18:0/616045630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 msgr2=0x55c545003510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f115e800640 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c545003510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f115e800640 1 -- 172.21.15.18:0/616045630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c544ff9c80 con 0x55c544ff7890 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f115de00640 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f115e800640 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544fe8ea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1158028a70 tx=0x7f11580035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38c3417c5326e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:21.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f114ea00640 1 -- 172.21.15.18:0/616045630 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115803b070 con 0x55c544ff7890 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f114ea00640 1 -- 172.21.15.18:0/616045630 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1158028700 con 0x55c544ff7890 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.716+0000 7f114ea00640 1 -- 172.21.15.18:0/616045630 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158005040 con 0x55c544ff7890 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 -- 172.21.15.18:0/616045630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 msgr2=0x55c544fe8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544fe8ea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1158028a70 tx=0x7f11580035f0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 -- 172.21.15.18:0/616045630 shutdown_connections 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544ff7890 0x55c544fe8ea0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544feecb0 0x55c545003510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 --2- 172.21.15.18:0/616045630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 -- 172.21.15.18:0/616045630 >> 172.21.15.18:0/616045630 conn(0x55c544fe8ac0 msgr2=0x55c544fe7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 -- 172.21.15.18:0/616045630 shutdown_connections 2024-04-16T17:09:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.717+0000 7f1167d8e740 1 -- 172.21.15.18:0/616045630 wait complete. 2024-04-16T17:09:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.718+0000 7f1167d8e740 1 -- start start 2024-04-16T17:09:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.718+0000 7f1167d8e740 1 Processor -- start 2024-04-16T17:09:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544feecb0 0x55c544ff18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f1167d8e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544ff7890 0x55c544ff1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f1167d8e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11601afc30 con 0x55c544ff7890 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f1167d8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b10a0 con 0x55c544feecb0 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f1167d8e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b0300 con 0x55c544fee390 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f1167d8e740 10 client.0 _ll_register_callbacks cb 0x7f1158003a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544feecb0 0x55c544ff18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544feecb0 0x55c544ff18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46728/0 (socket says 172.21.15.18:46728) 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115d400640 1 -- 172.21.15.18:0/3478997012 learned_addr learned my addr 172.21.15.18:0/3478997012 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115e800640 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544ff7890 0x55c544ff1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115de00640 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115d400640 1 -- 172.21.15.18:0/3478997012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 msgr2=0x55c544ff13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115d400640 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115d400640 1 -- 172.21.15.18:0/3478997012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544ff7890 msgr2=0x55c544ff1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115d400640 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544ff7890 0x55c544ff1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.719+0000 7f115d400640 1 -- 172.21.15.18:0/3478997012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54506d120 con 0x55c544feecb0 2024-04-16T17:09:21.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.720+0000 7f115d400640 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544feecb0 0x55c544ff18e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f114802c2d0 tx=0x7f114802c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94dce1468f22961e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.720+0000 7f1143e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.720+0000 7f1143e00640 1 -- 172.21.15.18:0/3478997012 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114802c910 con 0x55c544feecb0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.720+0000 7f1143e00640 1 -- 172.21.15.18:0/3478997012 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f114802cab0 con 0x55c544feecb0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.720+0000 7f1167d8e740 1 -- 172.21.15.18:0/3478997012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1158028700 con 0x55c544feecb0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1143e00640 1 -- 172.21.15.18:0/3478997012 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f114805a050 con 0x55c544feecb0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1167d8e740 10 client.26834 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1167d8e740 1 -- 172.21.15.18:0/3478997012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c54506e2d0 con 0x55c544feecb0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1143e00640 1 -- 172.21.15.18:0/3478997012 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1158028700 con 0x55c544feecb0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1143e00640 1 -- 172.21.15.18:0/3478997012 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f11480345b0 con 0x55c544feecb0 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1167d8e740 10 client.26834 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1167d8e740 10 client.26834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1167d8e740 20 client.26834 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c54506e610 0x55c545070a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1167d8e740 4 client.26834 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.721+0000 7f1167d8e740 1 -- 172.21.15.18:0/3478997012 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c544f5b9b0 con 0x55c54506e610 2024-04-16T17:09:21.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.722+0000 7f115de00640 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c54506e610 0x55c545070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.722+0000 7f115de00640 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c54506e610 0x55c545070a00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7d3104ea4992a6 server_cookie=5ecd751b8a5295f0 in_seq=0 out_seq=0 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.722+0000 7f1143e00640 10 client.26834 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.722+0000 7f1143e00640 1 -- 172.21.15.18:0/3478997012 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c544f5b9b0 con 0x55c54506e610 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.722+0000 7f1143e00640 10 client.26834 handle_command_reply: tid=0 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 client.26834 shutdown 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 -- 172.21.15.18:0/3478997012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544feecb0 msgr2=0x55c544ff18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544feecb0 0x55c544ff18e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f114802c2d0 tx=0x7f114802c340 comp rx=0 tx=0).stop 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 -- 172.21.15.18:0/3478997012 shutdown_connections 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c54506e610 0x55c545070a00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c544ff7890 0x55c544ff1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c544feecb0 0x55c544ff18e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 --2- 172.21.15.18:0/3478997012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c544fee390 0x55c544ff13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 -- 172.21.15.18:0/3478997012 >> 172.21.15.18:0/3478997012 conn(0x55c544fe8ac0 msgr2=0x55c544ff5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:21.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1143e00640 0 client.26834 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:21.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 -- 172.21.15.18:0/3478997012 shutdown_connections 2024-04-16T17:09:21.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.723+0000 7f1167d8e740 1 -- 172.21.15.18:0/3478997012 wait complete. 2024-04-16T17:09:21.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.724+0000 7f1167d8e740 20 client.26834 trim_cache size 0 max 16384 2024-04-16T17:09:21.726 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:09:21.726 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:09:21.726 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:09:21.727 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:09:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.724+0000 7f115f200640 1 -- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11440a3da0 msgr2=0x7f11440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.724+0000 7f115f200640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11440a3da0 0x7f11440a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1158005fd0 tx=0x7f1158005cc0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.724+0000 7f115f200640 1 -- 172.21.15.18:0/2051222645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11600743b0 msgr2=0x7f11601a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.724+0000 7f115f200640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11600743b0 0x7f11601a0630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1150009d50 tx=0x7f115000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.725+0000 7f115f200640 1 -- 172.21.15.18:0/2051222645 shutdown_connections 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.725+0000 7f115f200640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11440a3da0 0x7f11440a6260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.725+0000 7f115f200640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116010e6c0 0x7f116019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.725+0000 7f115f200640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1160074cd0 0x7f11601a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.725+0000 7f115f200640 1 --2- 172.21.15.18:0/2051222645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11600743b0 0x7f11601a0630 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.725+0000 7f115f200640 1 -- 172.21.15.18:0/2051222645 >> 172.21.15.18:0/2051222645 conn(0x7f11600fd950 msgr2=0x7f11600fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.725+0000 7f115f200640 1 -- 172.21.15.18:0/2051222645 shutdown_connections 2024-04-16T17:09:21.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.725+0000 7f115f200640 1 -- 172.21.15.18:0/2051222645 wait complete. 2024-04-16T17:09:21.749 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:09:21.749 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:09:21.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:21 smithi043 ceph-mon[36901]: pgmap v141: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-16T17:09:21.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:21 smithi022 ceph-mon[36023]: pgmap v141: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-16T17:09:21.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.958+0000 7f547b200640 1 -- 172.21.15.18:0/1580600349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f547c10c4b0 msgr2=0x7f547c10e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.958+0000 7f547b200640 1 --2- 172.21.15.18:0/1580600349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f547c10c4b0 0x7f547c10e950 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5474009ca0 tx=0x7f547402c180 comp rx=0 tx=0).stop 2024-04-16T17:09:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.958+0000 7f547b200640 1 -- 172.21.15.18:0/1580600349 shutdown_connections 2024-04-16T17:09:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.958+0000 7f547b200640 1 --2- 172.21.15.18:0/1580600349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f547c10c4b0 0x7f547c10e950 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.958+0000 7f547b200640 1 --2- 172.21.15.18:0/1580600349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f547c1039b0 0x7f547c10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.958+0000 7f547b200640 1 --2- 172.21.15.18:0/1580600349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f547c103090 0x7f547c103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.958+0000 7f547b200640 1 -- 172.21.15.18:0/1580600349 >> 172.21.15.18:0/1580600349 conn(0x7f547c0fc890 msgr2=0x7f547c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.959+0000 7f547b200640 1 -- 172.21.15.18:0/1580600349 shutdown_connections 2024-04-16T17:09:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.959+0000 7f547b200640 1 -- 172.21.15.18:0/1580600349 wait complete. 2024-04-16T17:09:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.959+0000 7f547b200640 1 Processor -- start 2024-04-16T17:09:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f547b200640 1 -- start start 2024-04-16T17:09:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f547b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f547c103090 0x7f547c100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f547b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f547c1039b0 0x7f547c101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f547b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f547c10c4b0 0x7f547c101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f547b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f547c112290 con 0x7f547c1039b0 2024-04-16T17:09:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f547b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f547c112110 con 0x7f547c10c4b0 2024-04-16T17:09:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f547b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f547c112410 con 0x7f547c103090 2024-04-16T17:09:21.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f5479e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f547c103090 0x7f547c100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f5479e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f547c103090 0x7f547c100ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53366/0 (socket says 172.21.15.18:53366) 2024-04-16T17:09:21.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.960+0000 7f5479e00640 1 -- 172.21.15.18:0/1360746860 learned_addr learned my addr 172.21.15.18:0/1360746860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f547a800640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f547c10c4b0 0x7f547c101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f5479400640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f547c1039b0 0x7f547c101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f5479e00640 1 -- 172.21.15.18:0/1360746860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f547c10c4b0 msgr2=0x7f547c101760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f5479e00640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f547c10c4b0 0x7f547c101760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f5479e00640 1 -- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f547c1039b0 msgr2=0x7f547c101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f5479e00640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f547c1039b0 0x7f547c101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f5479e00640 1 -- 172.21.15.18:0/1360746860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c1a6c90 con 0x7f547c103090 2024-04-16T17:09:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f5479400640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f547c1039b0 0x7f547c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:21.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.961+0000 7f5479e00640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f547c103090 0x7f547c100ce0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f546c00e9f0 tx=0x7f546c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:21.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.962+0000 7f5472a00640 1 -- 172.21.15.18:0/1360746860 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c00cd80 con 0x7f547c103090 2024-04-16T17:09:21.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.962+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c1a6f80 con 0x7f547c103090 2024-04-16T17:09:21.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.962+0000 7f5472a00640 1 -- 172.21.15.18:0/1360746860 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f546c010910 con 0x7f547c103090 2024-04-16T17:09:21.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.962+0000 7f5472a00640 1 -- 172.21.15.18:0/1360746860 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c0188f0 con 0x7f547c103090 2024-04-16T17:09:21.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.962+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c1a7440 con 0x7f547c103090 2024-04-16T17:09:21.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.963+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c04fe00 con 0x7f547c103090 2024-04-16T17:09:21.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.965+0000 7f5472a00640 1 -- 172.21.15.18:0/1360746860 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f546c010ac0 con 0x7f547c103090 2024-04-16T17:09:21.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.966+0000 7f5472a00640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54600a3d10 0x7f54600a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:21.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.967+0000 7f5472a00640 1 -- 172.21.15.18:0/1360746860 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f546c0d13c0 con 0x7f547c103090 2024-04-16T17:09:21.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.967+0000 7f5479400640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54600a3d10 0x7f54600a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:21.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.967+0000 7f5479400640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54600a3d10 0x7f54600a61d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f547c102770 tx=0x7f5468005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:21.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:21.969+0000 7f5472a00640 1 -- 172.21.15.18:0/1360746860 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f546c09d650 con 0x7f547c103090 2024-04-16T17:09:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.289+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f547c107550 con 0x7f547c103090 2024-04-16T17:09:22.291 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:09:22.291 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:09:22.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.290+0000 7f5472a00640 1 -- 172.21.15.18:0/1360746860 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7f546c09cf50 con 0x7f547c103090 2024-04-16T17:09:22.291 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.292+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54600a3d10 msgr2=0x7f54600a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.292+0000 7f547b200640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54600a3d10 0x7f54600a61d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f547c102770 tx=0x7f5468005f90 comp rx=0 tx=0).stop 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.292+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f547c103090 msgr2=0x7f547c100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.292+0000 7f547b200640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f547c103090 0x7f547c100ce0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f546c00e9f0 tx=0x7f546c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.293+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 shutdown_connections 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.293+0000 7f547b200640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54600a3d10 0x7f54600a61d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.293+0000 7f547b200640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f547c10c4b0 0x7f547c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.293+0000 7f547b200640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f547c1039b0 0x7f547c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.293+0000 7f547b200640 1 --2- 172.21.15.18:0/1360746860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f547c103090 0x7f547c100ce0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.293+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 >> 172.21.15.18:0/1360746860 conn(0x7f547c0fc890 msgr2=0x7f547c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.293+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 shutdown_connections 2024-04-16T17:09:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.293+0000 7f547b200640 1 -- 172.21.15.18:0/1360746860 wait complete. 2024-04-16T17:09:22.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 -- 172.21.15.18:0/188312124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9958109520 msgr2=0x7f99581110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 --2- 172.21.15.18:0/188312124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9958109520 0x7f99581110f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9950009be0 tx=0x7f995002bef0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 -- 172.21.15.18:0/188312124 shutdown_connections 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 --2- 172.21.15.18:0/188312124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9958109520 0x7f99581110f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 --2- 172.21.15.18:0/188312124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9958104e00 0x7f9958108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 --2- 172.21.15.18:0/188312124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9958104450 0x7f9958104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 -- 172.21.15.18:0/188312124 >> 172.21.15.18:0/188312124 conn(0x7f99580ffc90 msgr2=0x7f99581020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 -- 172.21.15.18:0/188312124 shutdown_connections 2024-04-16T17:09:22.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.509+0000 7f995f800640 1 -- 172.21.15.18:0/188312124 wait complete. 2024-04-16T17:09:22.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.510+0000 7f995f800640 1 Processor -- start 2024-04-16T17:09:22.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995f800640 1 -- start start 2024-04-16T17:09:22.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995f800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9958104450 0x7f99581a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995f800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9958109520 0x7f99581a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995f800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f995819a8a0 0x7f995819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995f800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9958114600 con 0x7f9958109520 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995f800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9958114480 con 0x7f9958104450 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995f800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9958114780 con 0x7f995819a8a0 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9958104450 0x7f99581a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9958104450 0x7f99581a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46770/0 (socket says 172.21.15.18:46770) 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.511+0000 7f995ce00640 1 -- 172.21.15.18:0/703951905 learned_addr learned my addr 172.21.15.18:0/703951905 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.512+0000 7f9957e00640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9958109520 0x7f99581a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.512+0000 7f995d800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f995819a8a0 0x7f995819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.512+0000 7f995ce00640 1 -- 172.21.15.18:0/703951905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f995819a8a0 msgr2=0x7f995819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.512+0000 7f995ce00640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f995819a8a0 0x7f995819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.512+0000 7f995ce00640 1 -- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9958109520 msgr2=0x7f99581a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.512+0000 7f995ce00640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9958109520 0x7f99581a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.512+0000 7f995ce00640 1 -- 172.21.15.18:0/703951905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995819b3c0 con 0x7f9958104450 2024-04-16T17:09:22.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.512+0000 7f995ce00640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9958104450 0x7f99581a06f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9f0 tx=0x7f994c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.513+0000 7f9955600640 1 -- 172.21.15.18:0/703951905 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c00ccc0 con 0x7f9958104450 2024-04-16T17:09:22.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.513+0000 7f995f800640 1 -- 172.21.15.18:0/703951905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995819b6b0 con 0x7f9958104450 2024-04-16T17:09:22.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.513+0000 7f9955600640 1 -- 172.21.15.18:0/703951905 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c010910 con 0x7f9958104450 2024-04-16T17:09:22.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.513+0000 7f9955600640 1 -- 172.21.15.18:0/703951905 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c0188a0 con 0x7f9958104450 2024-04-16T17:09:22.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.513+0000 7f995f800640 1 -- 172.21.15.18:0/703951905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99581a7950 con 0x7f9958104450 2024-04-16T17:09:22.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.516+0000 7f9955600640 1 -- 172.21.15.18:0/703951905 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f994c010430 con 0x7f9958104450 2024-04-16T17:09:22.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.517+0000 7f9962635740 1 Processor -- start 2024-04-16T17:09:22.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.517+0000 7f9962635740 1 -- start start 2024-04-16T17:09:22.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.517+0000 7f9962635740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fcd50 0x5587a87fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.517+0000 7f9962635740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8805d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.517+0000 7f9962635740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a8806250 0x5587a8808640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.517+0000 7f9962635740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5587a8762020 con 0x5587a87fcd50 2024-04-16T17:09:22.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.517+0000 7f9962635740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587a8708460 con 0x5587a87fd670 2024-04-16T17:09:22.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.517+0000 7f9962635740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587a875dfe0 con 0x5587a8806250 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.518+0000 7f995ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8805d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.518+0000 7f995ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8805d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46786/0 (socket says 172.21.15.18:46786) 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.518+0000 7f995ce00640 1 -- 172.21.15.18:0/786624091 learned_addr learned my addr 172.21.15.18:0/786624091 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.518+0000 7f9957e00640 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a8806250 0x5587a8808640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.518+0000 7f995d800640 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fcd50 0x5587a87fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.518+0000 7f9955600640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99400a3da0 0x7f99400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.518+0000 7f995ce00640 1 -- 172.21.15.18:0/786624091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a8806250 msgr2=0x5587a8808640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.518+0000 7f9955600640 1 -- 172.21.15.18:0/703951905 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f994c0d0fc0 con 0x7f9958104450 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f995ce00640 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a8806250 0x5587a8808640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f995ce00640 1 -- 172.21.15.18:0/786624091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fcd50 msgr2=0x5587a87fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f995ce00640 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fcd50 0x5587a87fd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f995ce00640 1 -- 172.21.15.18:0/786624091 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a8710040 con 0x5587a87fd670 2024-04-16T17:09:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f9957e00640 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a8806250 0x5587a8808640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f995d800640 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fcd50 0x5587a87fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f995d800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99400a3da0 0x7f99400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f995ce00640 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8805d10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ce240 tx=0x7f994c009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d99794a376e9f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f994a000640 1 -- 172.21.15.18:0/786624091 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c09c690 con 0x5587a87fd670 2024-04-16T17:09:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f994a000640 1 -- 172.21.15.18:0/786624091 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c09d250 con 0x5587a87fd670 2024-04-16T17:09:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.519+0000 7f994a000640 1 -- 172.21.15.18:0/786624091 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c0d4b60 con 0x5587a87fd670 2024-04-16T17:09:22.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 -- 172.21.15.18:0/786624091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 msgr2=0x5587a8805d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8805d10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ce240 tx=0x7f994c009700 comp rx=0 tx=0).stop 2024-04-16T17:09:22.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f995d800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99400a3da0 0x7f99400a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9950004af0 tx=0x7f9950009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 -- 172.21.15.18:0/786624091 shutdown_connections 2024-04-16T17:09:22.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a8806250 0x5587a8808640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8805d10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 --2- 172.21.15.18:0/786624091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fcd50 0x5587a87fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 -- 172.21.15.18:0/786624091 >> 172.21.15.18:0/786624091 conn(0x5587a87f7480 msgr2=0x5587a87f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 -- 172.21.15.18:0/786624091 shutdown_connections 2024-04-16T17:09:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.520+0000 7f9962635740 1 -- 172.21.15.18:0/786624091 wait complete. 2024-04-16T17:09:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.521+0000 7f9962635740 1 -- start start 2024-04-16T17:09:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.521+0000 7f9962635740 1 Processor -- start 2024-04-16T17:09:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9962635740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a88032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9962635740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8803820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9962635740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 0x5587a8803d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9962635740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5587a8808f80 con 0x5587a8806250 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9962635740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587a871b830 con 0x5587a87fd670 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9962635740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587a8809100 con 0x5587a87fcd50 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9962635740 10 client.0 _ll_register_callbacks cb 0x5587a8723150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9957e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8803820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9957e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8803820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46796/0 (socket says 172.21.15.18:46796) 2024-04-16T17:09:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9957e00640 1 -- 172.21.15.18:0/987013917 learned_addr learned my addr 172.21.15.18:0/987013917 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f995ce00640 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a88032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9957e00640 1 -- 172.21.15.18:0/987013917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 msgr2=0x5587a88032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9957e00640 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a88032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9957e00640 1 -- 172.21.15.18:0/987013917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 msgr2=0x5587a8803d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9957e00640 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 0x5587a8803d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.522+0000 7f9957e00640 1 -- 172.21.15.18:0/987013917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a871f920 con 0x5587a87fd670 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.523+0000 7f9957e00640 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8803820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f99440116c0 tx=0x7f9944011b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdab17cc60a5565b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.523+0000 7f993ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.523+0000 7f993ea00640 1 -- 172.21.15.18:0/987013917 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944004260 con 0x5587a87fd670 2024-04-16T17:09:22.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.524+0000 7f993ea00640 1 -- 172.21.15.18:0/987013917 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f9944004400 con 0x5587a87fd670 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.524+0000 7f9962635740 1 -- 172.21.15.18:0/987013917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994c09d250 con 0x5587a87fd670 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.524+0000 7f993ea00640 1 -- 172.21.15.18:0/987013917 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9944025050 con 0x5587a87fd670 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.524+0000 7f9962635740 10 client.26852 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.524+0000 7f9962635740 1 -- 172.21.15.18:0/987013917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587a86ed7f0 con 0x5587a87fd670 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.524+0000 7f993ea00640 1 -- 172.21.15.18:0/987013917 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c09d250 con 0x5587a87fd670 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.525+0000 7f993ea00640 1 -- 172.21.15.18:0/987013917 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f99440047a0 con 0x5587a87fd670 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.525+0000 7f9962635740 10 client.26852 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.525+0000 7f9962635740 10 client.26852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.525+0000 7f9962635740 20 client.26852 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.525+0000 7f9962635740 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5587a8816630 0x5587a8818a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.525+0000 7f9962635740 4 client.26852 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.525+0000 7f9962635740 1 -- 172.21.15.18:0/987013917 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a876fa40 con 0x5587a8816630 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.526+0000 7f995ce00640 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5587a8816630 0x5587a8818a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.526+0000 7f995ce00640 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5587a8816630 0x5587a8818a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66eea43fdf09aaae server_cookie=21aa51d4631bc91c in_seq=0 out_seq=0 2024-04-16T17:09:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.526+0000 7f993ea00640 10 client.26852 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f993ea00640 1 -- 172.21.15.18:0/987013917 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587a876fa40 con 0x5587a8816630 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f993ea00640 10 client.26852 handle_command_reply: tid=0 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 client.26852 shutdown 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 -- 172.21.15.18:0/987013917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 msgr2=0x5587a8803820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8803820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f99440116c0 tx=0x7f9944011b80 comp rx=0 tx=0).stop 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 -- 172.21.15.18:0/987013917 shutdown_connections 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5587a8816630 0x5587a8818a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 0x5587a8803d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fd670 0x5587a8803820 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 --2- 172.21.15.18:0/987013917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a88032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 -- 172.21.15.18:0/987013917 >> 172.21.15.18:0/987013917 conn(0x5587a87f7480 msgr2=0x5587a87f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f993ea00640 0 client.26852 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 -- 172.21.15.18:0/987013917 shutdown_connections 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.527+0000 7f9962635740 1 -- 172.21.15.18:0/987013917 wait complete. 2024-04-16T17:09:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.528+0000 7f9962635740 20 client.26852 trim_cache size 0 max 16384 2024-04-16T17:09:22.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.548+0000 7f9962635740 1 Processor -- start 2024-04-16T17:09:22.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.548+0000 7f9962635740 1 -- start start 2024-04-16T17:09:22.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f9962635740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fcd50 0x5587a8811e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f9962635740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fd670 0x5587a8803b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f9962635740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 0x5587a881e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f9962635740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99581b4020 con 0x5587a8806250 2024-04-16T17:09:22.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f9962635740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99581b18f0 con 0x5587a87fcd50 2024-04-16T17:09:22.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f9962635740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99581af740 con 0x5587a87fd670 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f9957e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fd670 0x5587a8803b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f995ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fcd50 0x5587a8811e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f995ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fcd50 0x5587a8811e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46810/0 (socket says 172.21.15.18:46810) 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f9957e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fd670 0x5587a8803b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53408/0 (socket says 172.21.15.18:53408) 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f995ce00640 1 -- 172.21.15.18:0/546655152 learned_addr learned my addr 172.21.15.18:0/546655152 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f995d800640 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 0x5587a881e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.549+0000 7f995ce00640 1 -- 172.21.15.18:0/546655152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fd670 msgr2=0x5587a8803b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.550+0000 7f995ce00640 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fd670 0x5587a8803b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.550+0000 7f995ce00640 1 -- 172.21.15.18:0/546655152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 msgr2=0x5587a881e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.550+0000 7f995ce00640 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 0x5587a881e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.550+0000 7f995ce00640 1 -- 172.21.15.18:0/546655152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a84743f0 con 0x5587a87fcd50 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.550+0000 7f9957e00640 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fd670 0x5587a8803b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:22.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.550+0000 7f995d800640 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 0x5587a881e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:22.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.550+0000 7f995ce00640 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fcd50 0x5587a8811e40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f994c0d12b0 tx=0x7f994c0d4a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d631c28963b482 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.551+0000 7f9949600640 1 -- 172.21.15.18:0/546655152 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c09c4b0 con 0x5587a87fcd50 2024-04-16T17:09:22.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.551+0000 7f9949600640 1 -- 172.21.15.18:0/546655152 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c09cb50 con 0x5587a87fcd50 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.551+0000 7f9949600640 1 -- 172.21.15.18:0/546655152 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c0ce580 con 0x5587a87fcd50 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 -- 172.21.15.18:0/546655152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fcd50 msgr2=0x5587a8811e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fcd50 0x5587a8811e40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f994c0d12b0 tx=0x7f994c0d4a70 comp rx=0 tx=0).stop 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 -- 172.21.15.18:0/546655152 shutdown_connections 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a8806250 0x5587a881e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fd670 0x5587a8803b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 --2- 172.21.15.18:0/546655152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a87fcd50 0x5587a8811e40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 -- 172.21.15.18:0/546655152 >> 172.21.15.18:0/546655152 conn(0x5587a87f7480 msgr2=0x5587a87f5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 -- 172.21.15.18:0/546655152 shutdown_connections 2024-04-16T17:09:22.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.552+0000 7f9962635740 1 -- 172.21.15.18:0/546655152 wait complete. 2024-04-16T17:09:22.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.553+0000 7f9962635740 1 -- start start 2024-04-16T17:09:22.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.553+0000 7f9962635740 1 Processor -- start 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.553+0000 7f9962635740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a8874eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.553+0000 7f9962635740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fd670 0x5587a88753f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.553+0000 7f9962635740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a8806250 0x5587a8809530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.553+0000 7f9962635740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99581afdb0 con 0x5587a87fd670 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.553+0000 7f9962635740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99581b1220 con 0x5587a8806250 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.553+0000 7f9962635740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99581b0480 con 0x5587a87fcd50 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a8806250 0x5587a8809530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a8874eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a8806250 0x5587a8809530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46824/0 (socket says 172.21.15.18:46824) 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995d800640 1 -- 172.21.15.18:0/1552422230 learned_addr learned my addr 172.21.15.18:0/1552422230 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f9962635740 10 client.0 _ll_register_callbacks cb 0x7f9944009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f9957e00640 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fd670 0x5587a88753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995ce00640 1 -- 172.21.15.18:0/1552422230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a8806250 msgr2=0x5587a8809530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995ce00640 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a8806250 0x5587a8809530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995ce00640 1 -- 172.21.15.18:0/1552422230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fd670 msgr2=0x5587a88753f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995ce00640 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fd670 0x5587a88753f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995ce00640 1 -- 172.21.15.18:0/1552422230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a853e7d0 con 0x5587a87fcd50 2024-04-16T17:09:22.556 INFO:tasks.workunit.client.0.smithi018.stdout: 1 163303 4.01 MB/sec execute 95 sec latency 10.700 ms 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.554+0000 7f995d800640 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a8806250 0x5587a8809530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f995ce00640 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a8874eb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f994c098fb0 tx=0x7f994c0a1080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d9aceafe6aa335 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f993ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f993ea00640 1 -- 172.21.15.18:0/1552422230 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c09c4b0 con 0x5587a87fcd50 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f993ea00640 1 -- 172.21.15.18:0/1552422230 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f994c0bd0b0 con 0x5587a87fcd50 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f9962635740 1 -- 172.21.15.18:0/1552422230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994c09cb50 con 0x5587a87fcd50 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f993ea00640 1 -- 172.21.15.18:0/1552422230 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f994c09d250 con 0x5587a87fcd50 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f9962635740 10 client.26833 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:22.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f9962635740 1 -- 172.21.15.18:0/1552422230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587a887c450 con 0x5587a87fcd50 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.555+0000 7f993ea00640 1 -- 172.21.15.18:0/1552422230 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c09cb50 con 0x5587a87fcd50 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.556+0000 7f993ea00640 1 -- 172.21.15.18:0/1552422230 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f994c0ce610 con 0x5587a87fcd50 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.556+0000 7f9962635740 10 client.26833 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.556+0000 7f9962635740 10 client.26833 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.556+0000 7f9962635740 20 client.26833 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.556+0000 7f9962635740 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5587a887c6d0 0x5587a887eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.556+0000 7f9962635740 4 client.26833 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.556+0000 7f9962635740 1 -- 172.21.15.18:0/1552422230 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587a87698b0 con 0x5587a887c6d0 2024-04-16T17:09:22.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.557+0000 7f9957e00640 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5587a887c6d0 0x5587a887eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.557+0000 7f9957e00640 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5587a887c6d0 0x5587a887eac0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cdccc44b45b6ebd server_cookie=310d85e2996c5096 in_seq=0 out_seq=0 2024-04-16T17:09:22.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.557+0000 7f993ea00640 10 client.26833 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:09:22.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.557+0000 7f993ea00640 1 -- 172.21.15.18:0/1552422230 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587a87698b0 con 0x5587a887c6d0 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.557+0000 7f993ea00640 10 client.26833 handle_command_reply: tid=0 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.557+0000 7f9962635740 1 client.26833 shutdown 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 -- 172.21.15.18:0/1552422230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 msgr2=0x5587a8874eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a8874eb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f994c098fb0 tx=0x7f994c0a1080 comp rx=0 tx=0).stop 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 -- 172.21.15.18:0/1552422230 shutdown_connections 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5587a887c6d0 0x5587a887eac0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587a8806250 0x5587a8809530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587a87fd670 0x5587a88753f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 --2- 172.21.15.18:0/1552422230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587a87fcd50 0x5587a8874eb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f993ea00640 0 client.26833 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 -- 172.21.15.18:0/1552422230 >> 172.21.15.18:0/1552422230 conn(0x5587a87f7480 msgr2=0x5587a87f64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 -- 172.21.15.18:0/1552422230 shutdown_connections 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 1 -- 172.21.15.18:0/1552422230 wait complete. 2024-04-16T17:09:22.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.558+0000 7f9962635740 20 client.26833 trim_cache size 0 max 16384 2024-04-16T17:09:22.561 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:09:22.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 -- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99400a3da0 msgr2=0x7f99400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99400a3da0 0x7f99400a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9950004af0 tx=0x7f9950009500 comp rx=0 tx=0).stop 2024-04-16T17:09:22.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 -- 172.21.15.18:0/703951905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9958104450 msgr2=0x7f99581a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9958104450 0x7f99581a06f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9f0 tx=0x7f994c00edf0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 -- 172.21.15.18:0/703951905 shutdown_connections 2024-04-16T17:09:22.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99400a3da0 0x7f99400a6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f995819a8a0 0x7f995819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9958109520 0x7f99581a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 --2- 172.21.15.18:0/703951905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9958104450 0x7f99581a06f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 -- 172.21.15.18:0/703951905 >> 172.21.15.18:0/703951905 conn(0x7f99580ffc90 msgr2=0x7f9958100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.559+0000 7f995f800640 1 -- 172.21.15.18:0/703951905 shutdown_connections 2024-04-16T17:09:22.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.560+0000 7f995f800640 1 -- 172.21.15.18:0/703951905 wait complete. 2024-04-16T17:09:22.585 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 -- 172.21.15.18:0/2658449817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48104280 msgr2=0x7ffa48104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 --2- 172.21.15.18:0/2658449817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48104280 0x7ffa48104660 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009900 tx=0x7ffa3802bc40 comp rx=0 tx=0).stop 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 -- 172.21.15.18:0/2658449817 shutdown_connections 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 --2- 172.21.15.18:0/2658449817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa48109350 0x7ffa48110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 --2- 172.21.15.18:0/2658449817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa48104c30 0x7ffa48108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 --2- 172.21.15.18:0/2658449817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48104280 0x7ffa48104660 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 -- 172.21.15.18:0/2658449817 >> 172.21.15.18:0/2658449817 conn(0x7ffa480ffae0 msgr2=0x7ffa48101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 -- 172.21.15.18:0/2658449817 shutdown_connections 2024-04-16T17:09:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.787+0000 7ffa4d400640 1 -- 172.21.15.18:0/2658449817 wait complete. 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.788+0000 7ffa4d400640 1 Processor -- start 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.788+0000 7ffa4d400640 1 -- start start 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa4d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa48104280 0x7ffa481a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa4d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48109350 0x7ffa481a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa4d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa4819a850 0x7ffa4819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa4d400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa48114430 con 0x7ffa48104280 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa4d400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481142b0 con 0x7ffa4819a850 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa4d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481145b0 con 0x7ffa48109350 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48109350 0x7ffa481a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48109350 0x7ffa481a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53424/0 (socket says 172.21.15.18:53424) 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47400640 1 -- 172.21.15.18:0/433692018 learned_addr learned my addr 172.21.15.18:0/433692018 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa4819a850 0x7ffa4819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47e00640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa48104280 0x7ffa481a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47400640 1 -- 172.21.15.18:0/433692018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa4819a850 msgr2=0x7ffa4819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa4819a850 0x7ffa4819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47400640 1 -- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa48104280 msgr2=0x7ffa481a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa48104280 0x7ffa481a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47400640 1 -- 172.21.15.18:0/433692018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4819b3e0 con 0x7ffa48109350 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa47e00640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa48104280 0x7ffa481a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.789+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa4819a850 0x7ffa4819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.790+0000 7ffa47400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48109350 0x7ffa481a0bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400e9a0 tx=0x7ffa3400eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.791+0000 7ffa44c00640 1 -- 172.21.15.18:0/433692018 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400cd80 con 0x7ffa48109350 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.791+0000 7ffa44c00640 1 -- 172.21.15.18:0/433692018 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa340035f0 con 0x7ffa48109350 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.791+0000 7ffa44c00640 1 -- 172.21.15.18:0/433692018 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34010690 con 0x7ffa48109350 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.791+0000 7ffa4d400640 1 -- 172.21.15.18:0/433692018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4819b6d0 con 0x7ffa48109350 2024-04-16T17:09:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.791+0000 7ffa4d400640 1 -- 172.21.15.18:0/433692018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa481a7860 con 0x7ffa48109350 2024-04-16T17:09:22.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.795+0000 7ffa517de740 1 Processor -- start 2024-04-16T17:09:22.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.795+0000 7ffa517de740 1 -- start start 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa517de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dd9d20 0x55dbe2dda100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa517de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 0x55dbe2deb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa517de740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2debc20 0x55dbe2dee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa517de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dbe2d3f020 con 0x55dbe2dda6d0 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa517de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbe2ce5460 con 0x55dbe2dd9d20 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa517de740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dbe2d3afe0 con 0x55dbe2debc20 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa4ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dd9d20 0x55dbe2dda100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa4ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dd9d20 0x55dbe2dda100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46848/0 (socket says 172.21.15.18:46848) 2024-04-16T17:09:22.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.796+0000 7ffa4ca00640 1 -- 172.21.15.18:0/4239574701 learned_addr learned my addr 172.21.15.18:0/4239574701 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.797+0000 7ffa44c00640 1 -- 172.21.15.18:0/433692018 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa340037a0 con 0x7ffa48109350 2024-04-16T17:09:22.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.797+0000 7ffa4ca00640 1 -- 172.21.15.18:0/4239574701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2debc20 msgr2=0x55dbe2dee010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:22.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.797+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/4239574701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2debc20 0x55dbe2dee010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.797+0000 7ffa4ca00640 1 -- 172.21.15.18:0/4239574701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 msgr2=0x55dbe2deb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.797+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/4239574701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 0x55dbe2deb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.797+0000 7ffa4ca00640 1 -- 172.21.15.18:0/4239574701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe2ced040 con 0x55dbe2dd9d20 2024-04-16T17:09:22.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.797+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/4239574701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dd9d20 0x55dbe2dda100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffa400117c0 tx=0x7ffa40011c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e36755439c83eb68 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.797+0000 7ffa3d200640 1 -- 172.21.15.18:0/4239574701 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40004d20 con 0x55dbe2dd9d20 2024-04-16T17:09:22.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.798+0000 7ffa3d200640 1 -- 172.21.15.18:0/4239574701 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40003970 con 0x55dbe2dd9d20 2024-04-16T17:09:22.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.798+0000 7ffa3d200640 1 -- 172.21.15.18:0/4239574701 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400044f0 con 0x55dbe2dd9d20 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.798+0000 7ffa44c00640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa2c0a3da0 0x7ffa2c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.798+0000 7ffa517de740 1 -- 172.21.15.18:0/4239574701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dd9d20 msgr2=0x55dbe2dda100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.798+0000 7ffa517de740 1 --2- 172.21.15.18:0/4239574701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dd9d20 0x55dbe2dda100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffa400117c0 tx=0x7ffa40011c80 comp rx=0 tx=0).stop 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.798+0000 7ffa44c00640 1 -- 172.21.15.18:0/433692018 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ffa340d0fb0 con 0x7ffa48109350 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.799+0000 7ffa517de740 1 -- 172.21.15.18:0/4239574701 shutdown_connections 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.799+0000 7ffa517de740 1 --2- 172.21.15.18:0/4239574701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2debc20 0x55dbe2dee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.799+0000 7ffa517de740 1 --2- 172.21.15.18:0/4239574701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 0x55dbe2deb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.799+0000 7ffa517de740 1 --2- 172.21.15.18:0/4239574701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dd9d20 0x55dbe2dda100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.799+0000 7ffa517de740 1 -- 172.21.15.18:0/4239574701 >> 172.21.15.18:0/4239574701 conn(0x55dbe2dd4640 msgr2=0x55dbe2dd4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.799+0000 7ffa47e00640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa2c0a3da0 0x7ffa2c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.799+0000 7ffa517de740 1 -- 172.21.15.18:0/4239574701 shutdown_connections 2024-04-16T17:09:22.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.799+0000 7ffa517de740 1 -- 172.21.15.18:0/4239574701 wait complete. 2024-04-16T17:09:22.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 1 -- start start 2024-04-16T17:09:22.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa47e00640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa2c0a3da0 0x7ffa2c0a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006010 tx=0x7ffa38000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 1 Processor -- start 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2de9720 0x55dbe2df7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dbe2dee9e0 con 0x55dbe2dda6d0 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbe2cf8830 con 0x55dbe2de9720 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dbe2deeb60 con 0x55dbe2dd9d20 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.800+0000 7ffa517de740 10 client.0 _ll_register_callbacks cb 0x55dbe2d00150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53452/0 (socket says 172.21.15.18:53452) 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47e00640 1 -- 172.21.15.18:0/845794449 learned_addr learned my addr 172.21.15.18:0/845794449 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47400640 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47e00640 1 -- 172.21.15.18:0/845794449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2de9720 msgr2=0x55dbe2df7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2de9720 0x55dbe2df7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47e00640 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2de9720 0x55dbe2df7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47e00640 1 -- 172.21.15.18:0/845794449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 msgr2=0x55dbe2de91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47e00640 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.801+0000 7ffa47e00640 1 -- 172.21.15.18:0/845794449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe2cfc920 con 0x55dbe2dd9d20 2024-04-16T17:09:22.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.802+0000 7ffa47e00640 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de8ca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffa38035000 tx=0x7ffa3802bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b761bba0078453 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.802+0000 7ffa31600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:22.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.802+0000 7ffa31600640 1 -- 172.21.15.18:0/845794449 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3804d020 con 0x55dbe2dd9d20 2024-04-16T17:09:22.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.802+0000 7ffa31600640 1 -- 172.21.15.18:0/845794449 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ffa38005450 con 0x55dbe2dd9d20 2024-04-16T17:09:22.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.802+0000 7ffa517de740 1 -- 172.21.15.18:0/845794449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa40003970 con 0x55dbe2dd9d20 2024-04-16T17:09:22.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.803+0000 7ffa31600640 1 -- 172.21.15.18:0/845794449 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa38058050 con 0x55dbe2dd9d20 2024-04-16T17:09:22.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.803+0000 7ffa517de740 10 client.26842 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:22.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.803+0000 7ffa517de740 1 -- 172.21.15.18:0/845794449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dbe2cca7f0 con 0x55dbe2dd9d20 2024-04-16T17:09:22.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.803+0000 7ffa31600640 1 -- 172.21.15.18:0/845794449 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40003970 con 0x55dbe2dd9d20 2024-04-16T17:09:22.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.803+0000 7ffa31600640 1 -- 172.21.15.18:0/845794449 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa38003950 con 0x55dbe2dd9d20 2024-04-16T17:09:22.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.803+0000 7ffa517de740 10 client.26842 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:22.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.803+0000 7ffa517de740 10 client.26842 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:09:22.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.804+0000 7ffa517de740 20 client.26842 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:22.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.804+0000 7ffa517de740 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55dbe2dfc470 0x55dbe2dfe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.804+0000 7ffa517de740 4 client.26842 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:22.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.804+0000 7ffa517de740 1 -- 172.21.15.18:0/845794449 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe2d4ca40 con 0x55dbe2dfc470 2024-04-16T17:09:22.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.804+0000 7ffa47400640 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55dbe2dfc470 0x55dbe2dfe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.805+0000 7ffa47400640 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55dbe2dfc470 0x55dbe2dfe900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c9cc2d73d055a44 server_cookie=f878a3c98eb345c0 in_seq=0 out_seq=0 2024-04-16T17:09:22.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.805+0000 7ffa31600640 10 client.26842 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:09:22.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.805+0000 7ffa31600640 1 -- 172.21.15.18:0/845794449 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbe2d4ca40 con 0x55dbe2dfc470 2024-04-16T17:09:22.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.805+0000 7ffa31600640 10 client.26842 handle_command_reply: tid=0 2024-04-16T17:09:22.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 client.26842 shutdown 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 -- 172.21.15.18:0/845794449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 msgr2=0x55dbe2de8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de8ca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffa38035000 tx=0x7ffa3802bc40 comp rx=0 tx=0).stop 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 -- 172.21.15.18:0/845794449 shutdown_connections 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55dbe2dfc470 0x55dbe2dfe900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2de9720 0x55dbe2df7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa31600640 0 client.26842 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 --2- 172.21.15.18:0/845794449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de8ca0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 -- 172.21.15.18:0/845794449 >> 172.21.15.18:0/845794449 conn(0x55dbe2dd4640 msgr2=0x55dbe2dd3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 -- 172.21.15.18:0/845794449 shutdown_connections 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 1 -- 172.21.15.18:0/845794449 wait complete. 2024-04-16T17:09:22.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.806+0000 7ffa517de740 20 client.26842 trim_cache size 0 max 16384 2024-04-16T17:09:22.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.822+0000 7ffa517de740 1 Processor -- start 2024-04-16T17:09:22.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.822+0000 7ffa517de740 1 -- start start 2024-04-16T17:09:22.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa517de740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2e04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa517de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2e04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa517de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2df3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa517de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481afad0 con 0x55dbe2de9720 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa517de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481ad3a0 con 0x55dbe2dda6d0 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa517de740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481ab210 con 0x55dbe2dd9d20 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa4ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2df3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa4ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2df3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60772/0 (socket says 172.21.15.18:60772) 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa4ca00640 1 -- 172.21.15.18:0/3369275332 learned_addr learned my addr 172.21.15.18:0/3369275332 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa47e00640 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2e04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.823+0000 7ffa47400640 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2e04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa47e00640 1 -- 172.21.15.18:0/3369275332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 msgr2=0x55dbe2e04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa47e00640 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2e04760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa47e00640 1 -- 172.21.15.18:0/3369275332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 msgr2=0x55dbe2df3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa47e00640 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2df3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa47e00640 1 -- 172.21.15.18:0/3369275332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe2a513f0 con 0x55dbe2dd9d20 2024-04-16T17:09:22.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2df3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:22.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa47400640 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2e04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa47e00640 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2e04220 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009900 tx=0x7ffa3802cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39a2098893edf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.824+0000 7ffa33e00640 1 -- 172.21.15.18:0/3369275332 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3804d070 con 0x55dbe2dd9d20 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.825+0000 7ffa33e00640 1 -- 172.21.15.18:0/3369275332 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa38044070 con 0x55dbe2dd9d20 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.825+0000 7ffa33e00640 1 -- 172.21.15.18:0/3369275332 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38004c00 con 0x55dbe2dd9d20 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.825+0000 7ffa517de740 1 -- 172.21.15.18:0/3369275332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 msgr2=0x55dbe2e04220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.825+0000 7ffa517de740 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2e04220 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009900 tx=0x7ffa3802cc60 comp rx=0 tx=0).stop 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 -- 172.21.15.18:0/3369275332 shutdown_connections 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2df3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2e04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 --2- 172.21.15.18:0/3369275332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2e04220 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 -- 172.21.15.18:0/3369275332 >> 172.21.15.18:0/3369275332 conn(0x55dbe2dd4640 msgr2=0x55dbe2de8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 -- 172.21.15.18:0/3369275332 shutdown_connections 2024-04-16T17:09:22.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 -- 172.21.15.18:0/3369275332 wait complete. 2024-04-16T17:09:22.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 -- start start 2024-04-16T17:09:22.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.826+0000 7ffa517de740 1 Processor -- start 2024-04-16T17:09:22.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa517de740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa517de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa517de740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2de6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa517de740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481ab860 con 0x55dbe2de9720 2024-04-16T17:09:22.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa517de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481accd0 con 0x55dbe2dda6d0 2024-04-16T17:09:22.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46888/0 (socket says 172.21.15.18:46888) 2024-04-16T17:09:22.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa47400640 1 -- 172.21.15.18:0/2766875704 learned_addr learned my addr 172.21.15.18:0/2766875704 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2de6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481abf30 con 0x55dbe2dd9d20 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa47e00640 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.827+0000 7ffa517de740 10 client.0 _ll_register_callbacks cb 0x7ffa34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa47400640 1 -- 172.21.15.18:0/2766875704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 msgr2=0x55dbe2de59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa47400640 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa47400640 1 -- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 msgr2=0x55dbe2de6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa47400640 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2de6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa47400640 1 -- 172.21.15.18:0/2766875704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe2b1b7d0 con 0x55dbe2dda6d0 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa4ca00640 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2de6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa47400640 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de5f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c6a0 tx=0x7ffa3409c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c7ab90ac82be61c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa31600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa31600640 1 -- 172.21.15.18:0/2766875704 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340a57b0 con 0x55dbe2dda6d0 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa31600640 1 -- 172.21.15.18:0/2766875704 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ffa34003b10 con 0x55dbe2dda6d0 2024-04-16T17:09:22.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.828+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa38044070 con 0x55dbe2dda6d0 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa31600640 1 -- 172.21.15.18:0/2766875704 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa340991b0 con 0x55dbe2dda6d0 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa517de740 10 client.26873 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dbe2e62bf0 con 0x55dbe2dda6d0 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa31600640 1 -- 172.21.15.18:0/2766875704 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa38044070 con 0x55dbe2dda6d0 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa31600640 1 -- 172.21.15.18:0/2766875704 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa34016be0 con 0x55dbe2dda6d0 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa517de740 10 client.26873 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa517de740 10 client.26873 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa517de740 20 client.26873 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa517de740 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55dbe2e63320 0x55dbe2e65740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa517de740 4 client.26873 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:22.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.829+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbe2d468b0 con 0x55dbe2e63320 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.830+0000 7ffa47e00640 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55dbe2e63320 0x55dbe2e65740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.830+0000 7ffa47e00640 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55dbe2e63320 0x55dbe2e65740 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc35a60ddb7efee3 server_cookie=455bc18a0090dda2 in_seq=0 out_seq=0 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.830+0000 7ffa31600640 10 client.26873 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa31600640 1 -- 172.21.15.18:0/2766875704 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbe2d468b0 con 0x55dbe2e63320 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa31600640 10 client.26873 handle_command_reply: tid=0 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 client.26873 shutdown 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 msgr2=0x55dbe2de5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de5f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c6a0 tx=0x7ffa3409c5d0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 shutdown_connections 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55dbe2e63320 0x55dbe2e65740 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa31600640 0 client.26873 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbe2de9720 0x55dbe2de6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbe2dda6d0 0x55dbe2de5f00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 --2- 172.21.15.18:0/2766875704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbe2dd9d20 0x55dbe2de59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.831+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 >> 172.21.15.18:0/2766875704 conn(0x55dbe2dd4640 msgr2=0x55dbe2dd2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.832+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 shutdown_connections 2024-04-16T17:09:22.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.832+0000 7ffa517de740 1 -- 172.21.15.18:0/2766875704 wait complete. 2024-04-16T17:09:22.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.832+0000 7ffa517de740 20 client.26873 trim_cache size 0 max 16384 2024-04-16T17:09:22.835 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.832+0000 7ffa4d400640 1 -- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa2c0a3da0 msgr2=0x7ffa2c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.832+0000 7ffa4d400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa2c0a3da0 0x7ffa2c0a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006010 tx=0x7ffa38000bc0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.832+0000 7ffa4d400640 1 -- 172.21.15.18:0/433692018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48109350 msgr2=0x7ffa481a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.832+0000 7ffa4d400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48109350 0x7ffa481a0bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400e9a0 tx=0x7ffa3400eda0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.833+0000 7ffa4d400640 1 -- 172.21.15.18:0/433692018 shutdown_connections 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.833+0000 7ffa4d400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa2c0a3da0 0x7ffa2c0a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.833+0000 7ffa4d400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa4819a850 0x7ffa4819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.833+0000 7ffa4d400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48109350 0x7ffa481a0bd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.833+0000 7ffa4d400640 1 --2- 172.21.15.18:0/433692018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa48104280 0x7ffa481a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.833+0000 7ffa4d400640 1 -- 172.21.15.18:0/433692018 >> 172.21.15.18:0/433692018 conn(0x7ffa480ffae0 msgr2=0x7ffa48100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.833+0000 7ffa4d400640 1 -- 172.21.15.18:0/433692018 shutdown_connections 2024-04-16T17:09:22.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:22.833+0000 7ffa4d400640 1 -- 172.21.15.18:0/433692018 wait complete. 2024-04-16T17:09:22.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:09:22.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:22 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1360746860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:23.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:22 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1360746860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:23.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.038+0000 7f2b63e00640 1 -- 172.21.15.18:0/1534410704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b64102070 msgr2=0x7f2b64102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.038+0000 7f2b63e00640 1 --2- 172.21.15.18:0/1534410704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b64102070 0x7f2b64102450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009860 tx=0x7f2b5402bb70 comp rx=0 tx=0).stop 2024-04-16T17:09:23.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.038+0000 7f2b63e00640 1 -- 172.21.15.18:0/1534410704 shutdown_connections 2024-04-16T17:09:23.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.038+0000 7f2b63e00640 1 --2- 172.21.15.18:0/1534410704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b64107140 0x7f2b6410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.038+0000 7f2b63e00640 1 --2- 172.21.15.18:0/1534410704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b64102a20 0x7f2b64106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.038+0000 7f2b63e00640 1 --2- 172.21.15.18:0/1534410704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b64102070 0x7f2b64102450 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.038+0000 7f2b63e00640 1 -- 172.21.15.18:0/1534410704 >> 172.21.15.18:0/1534410704 conn(0x7f2b640fd950 msgr2=0x7f2b640ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.038+0000 7f2b63e00640 1 -- 172.21.15.18:0/1534410704 shutdown_connections 2024-04-16T17:09:23.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.039+0000 7f2b63e00640 1 -- 172.21.15.18:0/1534410704 wait complete. 2024-04-16T17:09:23.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.040+0000 7f2b63e00640 1 Processor -- start 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.040+0000 7f2b63e00640 1 -- start start 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.040+0000 7f2b63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b64102070 0x7f2b64108d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.040+0000 7f2b63e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b64107140 0x7f2b64109270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.040+0000 7f2b63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b641097b0 0x7f2b641aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.040+0000 7f2b63e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641121d0 con 0x7f2b64107140 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.040+0000 7f2b63e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64112050 con 0x7f2b64102070 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.040+0000 7f2b63e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64112350 con 0x7f2b641097b0 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b62a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b64102070 0x7f2b64108d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b62a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b64102070 0x7f2b64108d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46902/0 (socket says 172.21.15.18:46902) 2024-04-16T17:09:23.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b62a00640 1 -- 172.21.15.18:0/3858927724 learned_addr learned my addr 172.21.15.18:0/3858927724 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b63400640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b641097b0 0x7f2b641aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b62000640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b64107140 0x7f2b64109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b63400640 1 -- 172.21.15.18:0/3858927724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b64102070 msgr2=0x7f2b64108d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b63400640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b64102070 0x7f2b64108d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b63400640 1 -- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b64107140 msgr2=0x7f2b64109270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b63400640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b64107140 0x7f2b64109270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.041+0000 7f2b63400640 1 -- 172.21.15.18:0/3858927724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6419a640 con 0x7f2b641097b0 2024-04-16T17:09:23.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.042+0000 7f2b63400640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b641097b0 0x7f2b641aaea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00ea80 tx=0x7f2b5c00ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.043+0000 7f2b5b400640 1 -- 172.21.15.18:0/3858927724 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00cea0 con 0x7f2b641097b0 2024-04-16T17:09:23.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.043+0000 7f2b5b400640 1 -- 172.21.15.18:0/3858927724 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5c005020 con 0x7f2b641097b0 2024-04-16T17:09:23.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.043+0000 7f2b5b400640 1 -- 172.21.15.18:0/3858927724 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c0106d0 con 0x7f2b641097b0 2024-04-16T17:09:23.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.043+0000 7f2b63e00640 1 -- 172.21.15.18:0/3858927724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6419a8d0 con 0x7f2b641097b0 2024-04-16T17:09:23.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.043+0000 7f2b63e00640 1 -- 172.21.15.18:0/3858927724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6419add0 con 0x7f2b641097b0 2024-04-16T17:09:23.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.047+0000 7f2b6c40d740 1 Processor -- start 2024-04-16T17:09:23.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.048+0000 7f2b5b400640 1 -- 172.21.15.18:0/3858927724 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b5c005540 con 0x7f2b641097b0 2024-04-16T17:09:23.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.048+0000 7f2b6c40d740 1 -- start start 2024-04-16T17:09:23.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.049+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea0022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.049+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea00aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.049+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea00d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.049+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d6e9f67020 con 0x55d6ea00b410 2024-04-16T17:09:23.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.049+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d6e9f0d460 con 0x55d6ea002830 2024-04-16T17:09:23.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.049+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d6e9f62fe0 con 0x55d6ea001f10 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b62000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea00d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b62000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea00d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60822/0 (socket says 172.21.15.18:60822) 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b62000640 1 -- 172.21.15.18:0/1281630631 learned_addr learned my addr 172.21.15.18:0/1281630631 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b62000640 1 -- 172.21.15.18:0/1281630631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 msgr2=0x55d6ea0022f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b62000640 1 --2- 172.21.15.18:0/1281630631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea0022f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b5b400640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b480a3da0 0x7f2b480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b62000640 1 -- 172.21.15.18:0/1281630631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 msgr2=0x55d6ea00aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b62000640 1 --2- 172.21.15.18:0/1281630631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea00aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b62000640 1 -- 172.21.15.18:0/1281630631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e9f15040 con 0x55d6ea00b410 2024-04-16T17:09:23.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.050+0000 7f2b5b400640 1 -- 172.21.15.18:0/3858927724 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2b5c0d1330 con 0x7f2b641097b0 2024-04-16T17:09:23.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.051+0000 7f2b62000640 1 --2- 172.21.15.18:0/1281630631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea00d800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009b80 tx=0x7f2b500117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45303a51ae31ba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.051+0000 7f2b63400640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b480a3da0 0x7f2b480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.051+0000 7f2b4fe00640 1 -- 172.21.15.18:0/1281630631 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50004050 con 0x55d6ea00b410 2024-04-16T17:09:23.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.051+0000 7f2b4fe00640 1 -- 172.21.15.18:0/1281630631 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b50016070 con 0x55d6ea00b410 2024-04-16T17:09:23.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.051+0000 7f2b4fe00640 1 -- 172.21.15.18:0/1281630631 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5001b6d0 con 0x55d6ea00b410 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.052+0000 7f2b6c40d740 1 -- 172.21.15.18:0/1281630631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 msgr2=0x55d6ea00d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.052+0000 7f2b63400640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b480a3da0 0x7f2b480a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c09cbc0 tx=0x7f2b5c005b40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.052+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/1281630631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea00d800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009b80 tx=0x7f2b500117b0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.052+0000 7f2b6c40d740 1 -- 172.21.15.18:0/1281630631 shutdown_connections 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.052+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/1281630631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea00d800 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.052+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/1281630631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea00aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.052+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/1281630631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea0022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.052+0000 7f2b6c40d740 1 -- 172.21.15.18:0/1281630631 >> 172.21.15.18:0/1281630631 conn(0x55d6e9ffc640 msgr2=0x55d6e9ffca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.053+0000 7f2b6c40d740 1 -- 172.21.15.18:0/1281630631 shutdown_connections 2024-04-16T17:09:23.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.053+0000 7f2b6c40d740 1 -- 172.21.15.18:0/1281630631 wait complete. 2024-04-16T17:09:23.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.053+0000 7f2b6c40d740 1 -- start start 2024-04-16T17:09:23.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.053+0000 7f2b6c40d740 1 Processor -- start 2024-04-16T17:09:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.054+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea001f10 0x55d6ea008540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.054+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea002830 0x55d6ea008a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.054+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea008fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.054+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea00e200 con 0x55d6ea001f10 2024-04-16T17:09:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.054+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d6e9f20830 con 0x55d6ea00b410 2024-04-16T17:09:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.054+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea00e380 con 0x55d6ea002830 2024-04-16T17:09:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.054+0000 7f2b6c40d740 10 client.0 _ll_register_callbacks cb 0x55d6e9f28150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.054+0000 7f2b62000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea001f10 0x55d6ea008540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b62000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea001f10 0x55d6ea008540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60826/0 (socket says 172.21.15.18:60826) 2024-04-16T17:09:23.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b62000640 1 -- 172.21.15.18:0/2921797743 learned_addr learned my addr 172.21.15.18:0/2921797743 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b62000640 1 -- 172.21.15.18:0/2921797743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea002830 msgr2=0x55d6ea008a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b62000640 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea002830 0x55d6ea008a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b62000640 1 -- 172.21.15.18:0/2921797743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 msgr2=0x55d6ea008fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b62000640 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea008fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b62000640 1 -- 172.21.15.18:0/2921797743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e9f24920 con 0x55d6ea001f10 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b62000640 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea001f10 0x55d6ea008540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b50003cb0 tx=0x7f2b50003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d9dff7669927aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.055+0000 7f2b4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.056+0000 7f2b4cc00640 1 -- 172.21.15.18:0/2921797743 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b500040b0 con 0x55d6ea001f10 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.056+0000 7f2b4cc00640 1 -- 172.21.15.18:0/2921797743 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2b50004250 con 0x55d6ea001f10 2024-04-16T17:09:23.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.056+0000 7f2b6c40d740 1 -- 172.21.15.18:0/2921797743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b50016070 con 0x55d6ea001f10 2024-04-16T17:09:23.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.056+0000 7f2b4cc00640 1 -- 172.21.15.18:0/2921797743 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b50005020 con 0x55d6ea001f10 2024-04-16T17:09:23.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.057+0000 7f2b6c40d740 10 client.16674 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:23.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.057+0000 7f2b6c40d740 1 -- 172.21.15.18:0/2921797743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d6e9ef27f0 con 0x55d6ea001f10 2024-04-16T17:09:23.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.057+0000 7f2b4cc00640 1 -- 172.21.15.18:0/2921797743 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b50016070 con 0x55d6ea001f10 2024-04-16T17:09:23.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.057+0000 7f2b4cc00640 1 -- 172.21.15.18:0/2921797743 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b5001b470 con 0x55d6ea001f10 2024-04-16T17:09:23.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.058+0000 7f2b6c40d740 10 client.16674 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:23.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.058+0000 7f2b6c40d740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:23.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.058+0000 7f2b6c40d740 20 client.16674 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:23.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.058+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d6ea01b790 0x55d6ea01db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.058+0000 7f2b6c40d740 4 client.16674 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:23.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.058+0000 7f2b6c40d740 1 -- 172.21.15.18:0/2921797743 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6e9f74a40 con 0x55d6ea01b790 2024-04-16T17:09:23.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.062+0000 7f2b62a00640 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d6ea01b790 0x55d6ea01db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.062+0000 7f2b62a00640 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d6ea01b790 0x55d6ea01db80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224d1750dd379540 server_cookie=a6ec90024e155efd in_seq=0 out_seq=0 2024-04-16T17:09:23.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.062+0000 7f2b4cc00640 10 client.16674 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:23.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.063+0000 7f2b4cc00640 1 -- 172.21.15.18:0/2921797743 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6e9f74a40 con 0x55d6ea01b790 2024-04-16T17:09:23.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.063+0000 7f2b4cc00640 10 client.16674 handle_command_reply: tid=0 2024-04-16T17:09:23.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 client.16674 shutdown 2024-04-16T17:09:23.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 -- 172.21.15.18:0/2921797743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea001f10 msgr2=0x55d6ea008540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea001f10 0x55d6ea008540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b50003cb0 tx=0x7f2b50003820 comp rx=0 tx=0).stop 2024-04-16T17:09:23.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 -- 172.21.15.18:0/2921797743 shutdown_connections 2024-04-16T17:09:23.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d6ea01b790 0x55d6ea01db80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea008fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea002830 0x55d6ea008a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/2921797743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea001f10 0x55d6ea008540 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 -- 172.21.15.18:0/2921797743 >> 172.21.15.18:0/2921797743 conn(0x55d6e9ffc640 msgr2=0x55d6e9ffe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b4cc00640 0 client.16674 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:23.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 -- 172.21.15.18:0/2921797743 shutdown_connections 2024-04-16T17:09:23.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 1 -- 172.21.15.18:0/2921797743 wait complete. 2024-04-16T17:09:23.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.064+0000 7f2b6c40d740 20 client.16674 trim_cache size 0 max 16384 2024-04-16T17:09:23.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.094+0000 7f2b6c40d740 1 Processor -- start 2024-04-16T17:09:23.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.094+0000 7f2b6c40d740 1 -- start start 2024-04-16T17:09:23.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.094+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea00ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.094+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea0170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.094+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea023210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.094+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641c0a60 con 0x55d6ea00b410 2024-04-16T17:09:23.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.094+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641be330 con 0x55d6ea002830 2024-04-16T17:09:23.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.094+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6404fdd0 con 0x55d6ea001f10 2024-04-16T17:09:23.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea0170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea0170c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46914/0 (socket says 172.21.15.18:46914) 2024-04-16T17:09:23.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62a00640 1 -- 172.21.15.18:0/3648571794 learned_addr learned my addr 172.21.15.18:0/3648571794 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62000640 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea023210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62000640 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea00ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62a00640 1 -- 172.21.15.18:0/3648571794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 msgr2=0x55d6ea00ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62a00640 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea00ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62a00640 1 -- 172.21.15.18:0/3648571794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 msgr2=0x55d6ea023210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62a00640 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea023210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.095+0000 7f2b62a00640 1 -- 172.21.15.18:0/3648571794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e9c793f0 con 0x55d6ea002830 2024-04-16T17:09:23.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.096+0000 7f2b62a00640 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea0170c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009830 tx=0x7f2b54005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527fac9eae3e5c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.096+0000 7f2b4f400640 1 -- 172.21.15.18:0/3648571794 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54039040 con 0x55d6ea002830 2024-04-16T17:09:23.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.096+0000 7f2b4f400640 1 -- 172.21.15.18:0/3648571794 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b54035d60 con 0x55d6ea002830 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.097+0000 7f2b4f400640 1 -- 172.21.15.18:0/3648571794 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b540052a0 con 0x55d6ea002830 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.097+0000 7f2b6c40d740 1 -- 172.21.15.18:0/3648571794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 msgr2=0x55d6ea0170c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.097+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea0170c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009830 tx=0x7f2b54005d20 comp rx=0 tx=0).stop 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.098+0000 7f2b6c40d740 1 -- 172.21.15.18:0/3648571794 shutdown_connections 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.098+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea00b410 0x55d6ea023210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.098+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea002830 0x55d6ea0170c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.098+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/3648571794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea00ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.098+0000 7f2b6c40d740 1 -- 172.21.15.18:0/3648571794 >> 172.21.15.18:0/3648571794 conn(0x55d6e9ffc640 msgr2=0x55d6ea007570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.098+0000 7f2b6c40d740 1 -- 172.21.15.18:0/3648571794 shutdown_connections 2024-04-16T17:09:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.098+0000 7f2b6c40d740 1 -- 172.21.15.18:0/3648571794 wait complete. 2024-04-16T17:09:23.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.099+0000 7f2b6c40d740 1 -- start start 2024-04-16T17:09:23.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.099+0000 7f2b6c40d740 1 Processor -- start 2024-04-16T17:09:23.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.100+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea0135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.100+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea002830 0x55d6ea013ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.100+0000 7f2b6c40d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea00eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.100+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641a2940 con 0x55d6ea002830 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.100+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641bdc60 con 0x55d6ea00b410 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.100+0000 7f2b6c40d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641bcec0 con 0x55d6ea001f10 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.100+0000 7f2b6c40d740 10 client.0 _ll_register_callbacks cb 0x7f2b50009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea00eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea00eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46928/0 (socket says 172.21.15.18:46928) 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 -- 172.21.15.18:0/4058350112 learned_addr learned my addr 172.21.15.18:0/4058350112 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea0135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62a00640 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea002830 0x55d6ea013ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 -- 172.21.15.18:0/4058350112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 msgr2=0x55d6ea0135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea0135a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 -- 172.21.15.18:0/4058350112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea002830 msgr2=0x55d6ea013ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea002830 0x55d6ea013ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62000640 1 -- 172.21.15.18:0/4058350112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e9d437d0 con 0x55d6ea00b410 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.101+0000 7f2b62a00640 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea002830 0x55d6ea013ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.102+0000 7f2b62000640 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea00eba0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2b50006010 tx=0x7f2b5001b520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=538e7cd79113b919 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.102+0000 7f2b4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:23.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.102+0000 7f2b4cc00640 1 -- 172.21.15.18:0/4058350112 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50012650 con 0x55d6ea00b410 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.102+0000 7f2b4cc00640 1 -- 172.21.15.18:0/4058350112 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f2b500127f0 con 0x55d6ea00b410 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.102+0000 7f2b6c40d740 1 -- 172.21.15.18:0/4058350112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b54035d60 con 0x55d6ea00b410 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.103+0000 7f2b4cc00640 1 -- 172.21.15.18:0/4058350112 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b500129f0 con 0x55d6ea00b410 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.103+0000 7f2b6c40d740 10 client.26885 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.103+0000 7f2b6c40d740 1 -- 172.21.15.18:0/4058350112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d6ea0814c0 con 0x55d6ea00b410 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.103+0000 7f2b4cc00640 1 -- 172.21.15.18:0/4058350112 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b54035d60 con 0x55d6ea00b410 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.103+0000 7f2b4cc00640 1 -- 172.21.15.18:0/4058350112 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b50024ae0 con 0x55d6ea00b410 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.103+0000 7f2b6c40d740 10 client.26885 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.103+0000 7f2b6c40d740 10 client.26885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.104+0000 7f2b6c40d740 20 client.26885 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.104+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d6ea0819b0 0x55d6ea083dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.104+0000 7f2b6c40d740 4 client.26885 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:23.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.104+0000 7f2b6c40d740 1 -- 172.21.15.18:0/4058350112 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6e9f6e8b0 con 0x55d6ea0819b0 2024-04-16T17:09:23.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.110+0000 7f2b62a00640 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d6ea0819b0 0x55d6ea083dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.111+0000 7f2b62a00640 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d6ea0819b0 0x55d6ea083dd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e8c3daa982b6dd server_cookie=61b510b765e6265d in_seq=0 out_seq=0 2024-04-16T17:09:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.111+0000 7f2b4cc00640 10 client.26885 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.111+0000 7f2b4cc00640 1 -- 172.21.15.18:0/4058350112 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6e9f6e8b0 con 0x55d6ea0819b0 2024-04-16T17:09:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.111+0000 7f2b4cc00640 10 client.26885 handle_command_reply: tid=0 2024-04-16T17:09:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.111+0000 7f2b6c40d740 1 client.26885 shutdown 2024-04-16T17:09:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.111+0000 7f2b6c40d740 1 -- 172.21.15.18:0/4058350112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 msgr2=0x55d6ea00eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.111+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea00eba0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2b50006010 tx=0x7f2b5001b520 comp rx=0 tx=0).stop 2024-04-16T17:09:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 1 -- 172.21.15.18:0/4058350112 shutdown_connections 2024-04-16T17:09:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d6ea0819b0 0x55d6ea083dd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6ea00b410 0x55d6ea00eba0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d6ea002830 0x55d6ea013ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 1 --2- 172.21.15.18:0/4058350112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d6ea001f10 0x55d6ea0135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 1 -- 172.21.15.18:0/4058350112 >> 172.21.15.18:0/4058350112 conn(0x55d6e9ffc640 msgr2=0x55d6ea008500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b4cc00640 0 client.26885 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 1 -- 172.21.15.18:0/4058350112 shutdown_connections 2024-04-16T17:09:23.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 1 -- 172.21.15.18:0/4058350112 wait complete. 2024-04-16T17:09:23.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.114+0000 7f2b6c40d740 20 client.26885 trim_cache size 0 max 16384 2024-04-16T17:09:23.116 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:09:23.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 -- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b480a3da0 msgr2=0x7f2b480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b480a3da0 0x7f2b480a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c09cbc0 tx=0x7f2b5c005b40 comp rx=0 tx=0).stop 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 -- 172.21.15.18:0/3858927724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b641097b0 msgr2=0x7f2b641aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b641097b0 0x7f2b641aaea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00ea80 tx=0x7f2b5c00ef40 comp rx=0 tx=0).stop 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 -- 172.21.15.18:0/3858927724 shutdown_connections 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b480a3da0 0x7f2b480a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b641097b0 0x7f2b641aaea0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b64107140 0x7f2b64109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 --2- 172.21.15.18:0/3858927724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b64102070 0x7f2b64108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.115+0000 7f2b63e00640 1 -- 172.21.15.18:0/3858927724 >> 172.21.15.18:0/3858927724 conn(0x7f2b640fd950 msgr2=0x7f2b640fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.116+0000 7f2b63e00640 1 -- 172.21.15.18:0/3858927724 shutdown_connections 2024-04-16T17:09:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:23.116+0000 7f2b63e00640 1 -- 172.21.15.18:0/3858927724 wait complete. 2024-04-16T17:09:23.560 INFO:tasks.workunit.client.0.smithi018.stdout: 1 163613 3.98 MB/sec execute 96 sec latency 82.381 ms 2024-04-16T17:09:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.637+0000 7f07f2200640 1 -- 172.21.15.18:0/2238082847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec104c50 msgr2=0x7f07ec108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.637+0000 7f07f2200640 1 --2- 172.21.15.18:0/2238082847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec104c50 0x7f07ec108c90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009900 tx=0x7f07d802bc40 comp rx=0 tx=0).stop 2024-04-16T17:09:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.638+0000 7f07f2200640 1 -- 172.21.15.18:0/2238082847 shutdown_connections 2024-04-16T17:09:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.638+0000 7f07f2200640 1 --2- 172.21.15.18:0/2238082847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07ec109370 0x7f07ec110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.638+0000 7f07f2200640 1 --2- 172.21.15.18:0/2238082847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec104c50 0x7f07ec108c90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.638+0000 7f07f2200640 1 --2- 172.21.15.18:0/2238082847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ec1042a0 0x7f07ec104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.638+0000 7f07f2200640 1 -- 172.21.15.18:0/2238082847 >> 172.21.15.18:0/2238082847 conn(0x7f07ec0ffb00 msgr2=0x7f07ec101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.638+0000 7f07f2200640 1 -- 172.21.15.18:0/2238082847 shutdown_connections 2024-04-16T17:09:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.638+0000 7f07f2200640 1 -- 172.21.15.18:0/2238082847 wait complete. 2024-04-16T17:09:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.639+0000 7f07f2200640 1 Processor -- start 2024-04-16T17:09:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.639+0000 7f07f2200640 1 -- start start 2024-04-16T17:09:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.639+0000 7f07f2200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec1042a0 0x7f07ec1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.639+0000 7f07f2200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07ec109370 0x7f07ec1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.639+0000 7f07f2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ec19a7b0 0x7f07ec19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.639+0000 7f07f2200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec114430 con 0x7f07ec109370 2024-04-16T17:09:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.639+0000 7f07f2200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1142b0 con 0x7f07ec19a7b0 2024-04-16T17:09:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.639+0000 7f07f2200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1145b0 con 0x7f07ec1042a0 2024-04-16T17:09:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec1042a0 0x7f07ec1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07ec109370 0x7f07ec1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec1042a0 0x7f07ec1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53558/0 (socket says 172.21.15.18:53558) 2024-04-16T17:09:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f0e00640 1 -- 172.21.15.18:0/944455347 learned_addr learned my addr 172.21.15.18:0/944455347 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f1800640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ec19a7b0 0x7f07ec19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f0e00640 1 -- 172.21.15.18:0/944455347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ec19a7b0 msgr2=0x7f07ec19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f0e00640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ec19a7b0 0x7f07ec19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f0e00640 1 -- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07ec109370 msgr2=0x7f07ec1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f0e00640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07ec109370 0x7f07ec1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f0e00640 1 -- 172.21.15.18:0/944455347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ec19b340 con 0x7f07ec1042a0 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07f1800640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ec19a7b0 0x7f07ec19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.640+0000 7f07ebe00640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07ec109370 0x7f07ec1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.641+0000 7f07f0e00640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec1042a0 0x7f07ec1a0640 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07e000ea60 tx=0x7f07e000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.641+0000 7f07e9600640 1 -- 172.21.15.18:0/944455347 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e000cde0 con 0x7f07ec1042a0 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.641+0000 7f07f2200640 1 -- 172.21.15.18:0/944455347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ec19b630 con 0x7f07ec1042a0 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.641+0000 7f07e9600640 1 -- 172.21.15.18:0/944455347 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e00035f0 con 0x7f07ec1042a0 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.641+0000 7f07e9600640 1 -- 172.21.15.18:0/944455347 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0010670 con 0x7f07ec1042a0 2024-04-16T17:09:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.641+0000 7f07f2200640 1 -- 172.21.15.18:0/944455347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ec1a7830 con 0x7f07ec1042a0 2024-04-16T17:09:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.644+0000 7f07f65a8740 1 Processor -- start 2024-04-16T17:09:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07f65a8740 1 -- start start 2024-04-16T17:09:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3834540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 0x55aae3834f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383e000 0x55aae38403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07f65a8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55aae3799780 con 0x55aae3834a80 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07f65a8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aae373ff40 con 0x55aae383e000 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07f65a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aae3795740 con 0x55aae38357f0 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07e9600640 1 -- 172.21.15.18:0/944455347 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f07e00037a0 con 0x7f07ec1042a0 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383e000 0x55aae38403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07f1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3834540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383e000 0x55aae38403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46954/0 (socket says 172.21.15.18:46954) 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.645+0000 7f07ebe00640 1 -- 172.21.15.18:0/2956919833 learned_addr learned my addr 172.21.15.18:0/2956919833 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.646+0000 7f07f0e00640 1 --2- 172.21.15.18:0/2956919833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 0x55aae3834f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.646+0000 7f07ebe00640 1 -- 172.21.15.18:0/2956919833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 msgr2=0x55aae3834540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.646+0000 7f07ebe00640 1 --2- 172.21.15.18:0/2956919833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3834540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.646+0000 7f07ebe00640 1 -- 172.21.15.18:0/2956919833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 msgr2=0x55aae3834f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.646+0000 7f07ebe00640 1 --2- 172.21.15.18:0/2956919833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 0x55aae3834f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.646+0000 7f07ebe00640 1 -- 172.21.15.18:0/2956919833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae34ac210 con 0x55aae383e000 2024-04-16T17:09:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.646+0000 7f07ebe00640 1 --2- 172.21.15.18:0/2956919833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383e000 0x55aae38403f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07ec06b470 tx=0x7f07d80095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbd95d8687d74f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.647+0000 7f07de000640 1 -- 172.21.15.18:0/2956919833 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8043030 con 0x55aae383e000 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.647+0000 7f07de000640 1 -- 172.21.15.18:0/2956919833 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d80046c0 con 0x55aae383e000 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.647+0000 7f07de000640 1 -- 172.21.15.18:0/2956919833 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8034870 con 0x55aae383e000 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.647+0000 7f07e9600640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07d40a3d00 0x7f07d40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.647+0000 7f07e9600640 1 -- 172.21.15.18:0/944455347 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f07e00d0f80 con 0x7f07ec1042a0 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 -- 172.21.15.18:0/2956919833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383e000 msgr2=0x55aae38403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2956919833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383e000 0x55aae38403f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07ec06b470 tx=0x7f07d80095f0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f1800640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07d40a3d00 0x7f07d40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 -- 172.21.15.18:0/2956919833 shutdown_connections 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2956919833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383e000 0x55aae38403f0 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07ec06b470 tx=0x7f07d80095f0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2956919833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 0x55aae3834f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2956919833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3834540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 -- 172.21.15.18:0/2956919833 >> 172.21.15.18:0/2956919833 conn(0x55aae37cf5d0 msgr2=0x55aae382f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 -- 172.21.15.18:0/2956919833 shutdown_connections 2024-04-16T17:09:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 -- 172.21.15.18:0/2956919833 wait complete. 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f65a8740 1 -- start start 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.648+0000 7f07f1800640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07d40a3d00 0x7f07d40a61c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07e4005fd0 tx=0x7f07e4009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.649+0000 7f07f65a8740 1 Processor -- start 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.649+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 0x55aae383af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.649+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae38357f0 0x55aae383b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.649+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae383b9e0 0x55aae384a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.649+0000 7f07f65a8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55aae3840cd0 con 0x55aae3834a80 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.649+0000 7f07f65a8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aae3753390 con 0x55aae38357f0 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.649+0000 7f07f65a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aae3840e50 con 0x55aae383b9e0 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.649+0000 7f07f65a8740 10 client.0 _ll_register_callbacks cb 0x55aae373cf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae38357f0 0x55aae383b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07f0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 0x55aae383af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae38357f0 0x55aae383b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46960/0 (socket says 172.21.15.18:46960) 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 -- 172.21.15.18:0/238223455 learned_addr learned my addr 172.21.15.18:0/238223455 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 -- 172.21.15.18:0/238223455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae383b9e0 msgr2=0x55aae384a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae383b9e0 0x55aae384a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 -- 172.21.15.18:0/238223455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 msgr2=0x55aae383af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 0x55aae383af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 -- 172.21.15.18:0/238223455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae3578620 con 0x55aae38357f0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07ebe00640 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae38357f0 0x55aae383b4a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f07d8036040 tx=0x7f07d8034a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f7b60601bfd5a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.650+0000 7f07d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.651+0000 7f07d2a00640 1 -- 172.21.15.18:0/238223455 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8034e60 con 0x55aae38357f0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.651+0000 7f07d2a00640 1 -- 172.21.15.18:0/238223455 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f07d803f640 con 0x55aae38357f0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.651+0000 7f07f65a8740 1 -- 172.21.15.18:0/238223455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d80046c0 con 0x55aae38357f0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.651+0000 7f07d2a00640 1 -- 172.21.15.18:0/238223455 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07d8004870 con 0x55aae38357f0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.651+0000 7f07f65a8740 10 client.26897 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.651+0000 7f07f65a8740 1 -- 172.21.15.18:0/238223455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aae384d350 con 0x55aae38357f0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.651+0000 7f07d2a00640 1 -- 172.21.15.18:0/238223455 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d80046c0 con 0x55aae38357f0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.652+0000 7f07d2a00640 1 -- 172.21.15.18:0/238223455 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f07d8048740 con 0x55aae38357f0 2024-04-16T17:09:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.652+0000 7f07f65a8740 10 client.26897 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.652+0000 7f07f65a8740 10 client.26897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.652+0000 7f07f65a8740 20 client.26897 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.652+0000 7f07f65a8740 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aae384e6a0 0x55aae3850a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.652+0000 7f07f65a8740 4 client.26897 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.652+0000 7f07f65a8740 1 -- 172.21.15.18:0/238223455 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aae37a71a0 con 0x55aae384e6a0 2024-04-16T17:09:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.652+0000 7f07f0e00640 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aae384e6a0 0x55aae3850a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.653+0000 7f07f0e00640 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aae384e6a0 0x55aae3850a90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c3e29c67982d3 server_cookie=8826509376511c56 in_seq=0 out_seq=0 2024-04-16T17:09:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.653+0000 7f07d2a00640 10 client.26897 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07d2a00640 1 -- 172.21.15.18:0/238223455 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aae37a71a0 con 0x55aae384e6a0 2024-04-16T17:09:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07d2a00640 10 client.26897 handle_command_reply: tid=0 2024-04-16T17:09:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 client.26897 shutdown 2024-04-16T17:09:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 -- 172.21.15.18:0/238223455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae38357f0 msgr2=0x55aae383b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae38357f0 0x55aae383b4a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f07d8036040 tx=0x7f07d8034a60 comp rx=0 tx=0).stop 2024-04-16T17:09:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 -- 172.21.15.18:0/238223455 shutdown_connections 2024-04-16T17:09:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aae384e6a0 0x55aae3850a90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae383b9e0 0x55aae384a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae38357f0 0x55aae383b4a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 --2- 172.21.15.18:0/238223455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae3834a80 0x55aae383af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 -- 172.21.15.18:0/238223455 >> 172.21.15.18:0/238223455 conn(0x55aae37cf5d0 msgr2=0x55aae38314c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07d2a00640 0 client.26897 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.654+0000 7f07f65a8740 1 -- 172.21.15.18:0/238223455 shutdown_connections 2024-04-16T17:09:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.655+0000 7f07f65a8740 1 -- 172.21.15.18:0/238223455 wait complete. 2024-04-16T17:09:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.655+0000 7f07f65a8740 20 client.26897 trim_cache size 0 max 16384 2024-04-16T17:09:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.679+0000 7f07f65a8740 1 Processor -- start 2024-04-16T17:09:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.679+0000 7f07f65a8740 1 -- start start 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.679+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae3834a80 0x55aae384a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.679+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3853ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.679+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae383b9e0 0x55aae3853ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.679+0000 7f07f65a8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b3e60 con 0x55aae383b9e0 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.679+0000 7f07f65a8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b1730 con 0x55aae3834a80 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.679+0000 7f07f65a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1af580 con 0x55aae38357f0 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3853ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3853ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53586/0 (socket says 172.21.15.18:53586) 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 -- 172.21.15.18:0/4169549402 learned_addr learned my addr 172.21.15.18:0/4169549402 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07f1800640 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae383b9e0 0x55aae3853ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07f0e00640 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae3834a80 0x55aae384a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 -- 172.21.15.18:0/4169549402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae3834a80 msgr2=0x55aae384a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae3834a80 0x55aae384a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 -- 172.21.15.18:0/4169549402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae383b9e0 msgr2=0x55aae3853ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae383b9e0 0x55aae3853ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 -- 172.21.15.18:0/4169549402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae3833d50 con 0x55aae38357f0 2024-04-16T17:09:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07f1800640 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae383b9e0 0x55aae3853ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.680+0000 7f07ebe00640 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3853ab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07d8004a30 tx=0x7f07d8005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bf81a700b3f52b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.681+0000 7f07dd600640 1 -- 172.21.15.18:0/4169549402 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d805f070 con 0x55aae38357f0 2024-04-16T17:09:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.681+0000 7f07dd600640 1 -- 172.21.15.18:0/4169549402 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d80046c0 con 0x55aae38357f0 2024-04-16T17:09:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.681+0000 7f07dd600640 1 -- 172.21.15.18:0/4169549402 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8043070 con 0x55aae38357f0 2024-04-16T17:09:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 -- 172.21.15.18:0/4169549402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 msgr2=0x55aae3853ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3853ab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07d8004a30 tx=0x7f07d8005a40 comp rx=0 tx=0).stop 2024-04-16T17:09:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 -- 172.21.15.18:0/4169549402 shutdown_connections 2024-04-16T17:09:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae383b9e0 0x55aae3853ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae38357f0 0x55aae3853ab0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 --2- 172.21.15.18:0/4169549402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae3834a80 0x55aae384a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 -- 172.21.15.18:0/4169549402 >> 172.21.15.18:0/4169549402 conn(0x55aae37cf5d0 msgr2=0x55aae382f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 -- 172.21.15.18:0/4169549402 shutdown_connections 2024-04-16T17:09:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 -- 172.21.15.18:0/4169549402 wait complete. 2024-04-16T17:09:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.682+0000 7f07f65a8740 1 -- start start 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.683+0000 7f07f65a8740 1 Processor -- start 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.683+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae3834a80 0x55aae3851480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.683+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae38357f0 0x55aae38519c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.683+0000 7f07f65a8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383b9e0 0x55aae3851f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.683+0000 7f07f65a8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1afbf0 con 0x55aae38357f0 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.683+0000 7f07f65a8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b1060 con 0x55aae383b9e0 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.683+0000 7f07f65a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b02c0 con 0x55aae3834a80 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f65a8740 10 client.0 _ll_register_callbacks cb 0x7f07e4009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383b9e0 0x55aae3851f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae38357f0 0x55aae38519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383b9e0 0x55aae3851f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46970/0 (socket says 172.21.15.18:46970) 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f1800640 1 -- 172.21.15.18:0/2087865836 learned_addr learned my addr 172.21.15.18:0/2087865836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f0e00640 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae3834a80 0x55aae3851480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f1800640 1 -- 172.21.15.18:0/2087865836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae3834a80 msgr2=0x55aae3851480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f1800640 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae3834a80 0x55aae3851480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f1800640 1 -- 172.21.15.18:0/2087865836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae38357f0 msgr2=0x55aae38519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f1800640 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae38357f0 0x55aae38519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.684+0000 7f07f1800640 1 -- 172.21.15.18:0/2087865836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae38b3880 con 0x55aae383b9e0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.685+0000 7f07f1800640 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383b9e0 0x55aae3851f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f07e4012010 tx=0x7f07e4010e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98aee9ffa4849e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.685+0000 7f07d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.685+0000 7f07d2a00640 1 -- 172.21.15.18:0/2087865836 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e402d060 con 0x55aae383b9e0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.685+0000 7f07d2a00640 1 -- 172.21.15.18:0/2087865836 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f07e4017030 con 0x55aae383b9e0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.685+0000 7f07f65a8740 1 -- 172.21.15.18:0/2087865836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d80046c0 con 0x55aae383b9e0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.685+0000 7f07d2a00640 1 -- 172.21.15.18:0/2087865836 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07e4034050 con 0x55aae383b9e0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.685+0000 7f07f65a8740 10 client.26903 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.685+0000 7f07f65a8740 1 -- 172.21.15.18:0/2087865836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aae38b4bc0 con 0x55aae383b9e0 2024-04-16T17:09:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07d2a00640 1 -- 172.21.15.18:0/2087865836 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d80046c0 con 0x55aae383b9e0 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07d2a00640 1 -- 172.21.15.18:0/2087865836 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f07e401c400 con 0x55aae383b9e0 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07f65a8740 10 client.26903 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07f65a8740 10 client.26903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07f65a8740 20 client.26903 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aae38b5370 0x55aae38b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07f65a8740 4 client.26903 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07f65a8740 1 -- 172.21.15.18:0/2087865836 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aae37a1010 con 0x55aae38b5370 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.686+0000 7f07f0e00640 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aae38b5370 0x55aae38b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.687+0000 7f07f0e00640 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aae38b5370 0x55aae38b7790 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d1706fd314244e server_cookie=58811a57a5ce06d2 in_seq=0 out_seq=0 2024-04-16T17:09:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.687+0000 7f07d2a00640 10 client.26903 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.894+0000 7f07d2a00640 1 -- 172.21.15.18:0/2087865836 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aae37a1010 con 0x55aae38b5370 2024-04-16T17:09:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.894+0000 7f07d2a00640 10 client.26903 handle_command_reply: tid=0 2024-04-16T17:09:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.894+0000 7f07f65a8740 1 client.26903 shutdown 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.894+0000 7f07f65a8740 1 -- 172.21.15.18:0/2087865836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383b9e0 msgr2=0x55aae3851f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.894+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383b9e0 0x55aae3851f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f07e4012010 tx=0x7f07e4010e90 comp rx=0 tx=0).stop 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07f65a8740 1 -- 172.21.15.18:0/2087865836 shutdown_connections 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aae38b5370 0x55aae38b7790 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aae383b9e0 0x55aae3851f00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aae38357f0 0x55aae38519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07f65a8740 1 --2- 172.21.15.18:0/2087865836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aae3834a80 0x55aae3851480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07f65a8740 1 -- 172.21.15.18:0/2087865836 >> 172.21.15.18:0/2087865836 conn(0x55aae37cf5d0 msgr2=0x55aae382df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07d2a00640 0 client.26903 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07f65a8740 1 -- 172.21.15.18:0/2087865836 shutdown_connections 2024-04-16T17:09:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.895+0000 7f07f65a8740 1 -- 172.21.15.18:0/2087865836 wait complete. 2024-04-16T17:09:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.896+0000 7f07f65a8740 20 client.26903 trim_cache size 0 max 16384 2024-04-16T17:09:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:09:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:09:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 -- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07d40a3d00 msgr2=0x7f07d40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07d40a3d00 0x7f07d40a61c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07e4005fd0 tx=0x7f07e4009500 comp rx=0 tx=0).stop 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 -- 172.21.15.18:0/944455347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec1042a0 msgr2=0x7f07ec1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec1042a0 0x7f07ec1a0640 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07e000ea60 tx=0x7f07e000ef20 comp rx=0 tx=0).stop 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 -- 172.21.15.18:0/944455347 shutdown_connections 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f07d40a3d00 0x7f07d40a61c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ec19a7b0 0x7f07ec19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07ec109370 0x7f07ec1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 --2- 172.21.15.18:0/944455347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07ec1042a0 0x7f07ec1a0640 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 -- 172.21.15.18:0/944455347 >> 172.21.15.18:0/944455347 conn(0x7f07ec0ffb00 msgr2=0x7f07ec100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 -- 172.21.15.18:0/944455347 shutdown_connections 2024-04-16T17:09:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:23.897+0000 7f07f2200640 1 -- 172.21.15.18:0/944455347 wait complete. 2024-04-16T17:09:23.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:23 smithi022 ceph-mon[36023]: pgmap v142: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-16T17:09:23.972 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:21 smithi018 ceph-mon[31776]: pgmap v141: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-16T17:09:23.972 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:22 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1360746860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:23.973 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:23 smithi018 ceph-mon[31776]: pgmap v142: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-16T17:09:24.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:23 smithi043 ceph-mon[36901]: pgmap v142: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-16T17:09:24.144 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:09:24.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:09:24.144 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:09:24.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 -- 172.21.15.18:0/269140024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b4104e00 msgr2=0x7fa0b4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/269140024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b4104e00 0x7fa0b4108e40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009900 tx=0x7fa0a002bc60 comp rx=0 tx=0).stop 2024-04-16T17:09:24.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 -- 172.21.15.18:0/269140024 shutdown_connections 2024-04-16T17:09:24.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/269140024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0b4109520 0x7fa0b41110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/269140024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b4104e00 0x7fa0b4108e40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/269140024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b4104450 0x7fa0b4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 -- 172.21.15.18:0/269140024 >> 172.21.15.18:0/269140024 conn(0x7fa0b40ffc70 msgr2=0x7fa0b4102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 -- 172.21.15.18:0/269140024 shutdown_connections 2024-04-16T17:09:24.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.321+0000 7fa0bbe00640 1 -- 172.21.15.18:0/269140024 wait complete. 2024-04-16T17:09:24.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.322+0000 7fa0bbe00640 1 Processor -- start 2024-04-16T17:09:24.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0bbe00640 1 -- start start 2024-04-16T17:09:24.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0b4104450 0x7fa0b41a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b4109520 0x7fa0b41a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b419aa60 0x7fa0b419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4114680 con 0x7fa0b419aa60 2024-04-16T17:09:24.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4114500 con 0x7fa0b4109520 2024-04-16T17:09:24.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4114800 con 0x7fa0b4104450 2024-04-16T17:09:24.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b419aa60 0x7fa0b419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0b4104450 0x7fa0b41a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0b8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b4109520 0x7fa0b41a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0b4104450 0x7fa0b41a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53606/0 (socket says 172.21.15.18:53606) 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0b9400640 1 -- 172.21.15.18:0/858325036 learned_addr learned my addr 172.21.15.18:0/858325036 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.323+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b419aa60 0x7fa0b419af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60912/0 (socket says 172.21.15.18:60912) 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b9400640 1 -- 172.21.15.18:0/858325036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b4109520 msgr2=0x7fa0b41a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b9400640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b4109520 0x7fa0b41a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b9400640 1 -- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b419aa60 msgr2=0x7fa0b419af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b9400640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b419aa60 0x7fa0b419af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b9400640 1 -- 172.21.15.18:0/858325036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b419b580 con 0x7fa0b4104450 2024-04-16T17:09:24.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b9400640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0b4104450 0x7fa0b41a08b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b450 tx=0x7fa0a800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b2000640 1 -- 172.21.15.18:0/858325036 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8015020 con 0x7fa0b4104450 2024-04-16T17:09:24.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b2000640 1 -- 172.21.15.18:0/858325036 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a8011d70 con 0x7fa0b4104450 2024-04-16T17:09:24.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0bbe00640 1 -- 172.21.15.18:0/858325036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b419b870 con 0x7fa0b4104450 2024-04-16T17:09:24.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.324+0000 7fa0b2000640 1 -- 172.21.15.18:0/858325036 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800f340 con 0x7fa0b4104450 2024-04-16T17:09:24.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.325+0000 7fa0bbe00640 1 -- 172.21.15.18:0/858325036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41a7ac0 con 0x7fa0b4104450 2024-04-16T17:09:24.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0be9e0740 1 Processor -- start 2024-04-16T17:09:24.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0be9e0740 1 -- start start 2024-04-16T17:09:24.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0b2000640 1 -- 172.21.15.18:0/858325036 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0a8018020 con 0x7fa0b4104450 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562008327cb0 con 0x5620083c35c0 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5620082ce370 con 0x5620083c2ca0 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.329+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562008323c70 con 0x5620083cc1a0 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:47010/0 (socket says 172.21.15.18:47010) 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9e00640 1 -- 172.21.15.18:0/183862759 learned_addr learned my addr 172.21.15.18:0/183862759 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9400640 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9e00640 1 -- 172.21.15.18:0/183862759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 msgr2=0x5620083ce590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083ce590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9e00640 1 -- 172.21.15.18:0/183862759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 msgr2=0x5620083cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9e00640 1 -- 172.21.15.18:0/183862759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620082ed910 con 0x5620083c2ca0 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.330+0000 7fa0b9400640 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.331+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c3080 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009fd0 tx=0x7fa0ac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490c1101ca320075 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.331+0000 7fa0a6600640 1 -- 172.21.15.18:0/183862759 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac004d20 con 0x5620083c2ca0 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.331+0000 7fa0a6600640 1 -- 172.21.15.18:0/183862759 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ac004050 con 0x5620083c2ca0 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.331+0000 7fa0a6600640 1 -- 172.21.15.18:0/183862759 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac005240 con 0x5620083c2ca0 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.331+0000 7fa0b2000640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.331+0000 7fa0b2000640 1 -- 172.21.15.18:0/858325036 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa0a80d04d0 con 0x7fa0b4104450 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 -- 172.21.15.18:0/183862759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 msgr2=0x5620083c3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c3080 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009fd0 tx=0x7fa0ac0118d0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0b9400640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 -- 172.21.15.18:0/183862759 shutdown_connections 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/183862759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c3080 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 -- 172.21.15.18:0/183862759 >> 172.21.15.18:0/183862759 conn(0x5620083bd3d0 msgr2=0x5620083bd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 -- 172.21.15.18:0/183862759 shutdown_connections 2024-04-16T17:09:24.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0be9e0740 1 -- 172.21.15.18:0/183862759 wait complete. 2024-04-16T17:09:24.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.332+0000 7fa0b9400640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80cda30 tx=0x7fa0a8011000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 1 -- start start 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 1 Processor -- start 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083c9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5620083cef90 con 0x5620083c35c0 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5620082e1730 con 0x5620083c2ca0 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5620083cf110 con 0x5620083cc1a0 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0be9e0740 10 client.0 _ll_register_callbacks cb 0x5620082b01e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.333+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:47014/0 (socket says 172.21.15.18:47014) 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 -- 172.21.15.18:0/1498534555 learned_addr learned my addr 172.21.15.18:0/1498534555 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b8a00640 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 -- 172.21.15.18:0/1498534555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 msgr2=0x5620083c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 -- 172.21.15.18:0/1498534555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 msgr2=0x5620083c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 -- 172.21.15.18:0/1498534555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620081027e0 con 0x5620083c2ca0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c91f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac01b040 tx=0x7fa0ac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4a51b03f49b0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.334+0000 7fa09aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa09aa00640 1 -- 172.21.15.18:0/1498534555 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00f4e0 con 0x5620083c2ca0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa09aa00640 1 -- 172.21.15.18:0/1498534555 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa0ac00f680 con 0x5620083c2ca0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1498534555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0ac004050 con 0x5620083c2ca0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa09aa00640 1 -- 172.21.15.18:0/1498534555 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa0ac035050 con 0x5620083c2ca0 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa0be9e0740 10 client.26912 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:24.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1498534555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562008162f70 con 0x5620083c2ca0 2024-04-16T17:09:24.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa09aa00640 1 -- 172.21.15.18:0/1498534555 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ac004050 con 0x5620083c2ca0 2024-04-16T17:09:24.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa09aa00640 1 -- 172.21.15.18:0/1498534555 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa0ac005450 con 0x5620083c2ca0 2024-04-16T17:09:24.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa0be9e0740 10 client.26912 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:24.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.335+0000 7fa0be9e0740 10 client.26912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:24.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.336+0000 7fa0be9e0740 20 client.26912 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:24.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.336+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5620083dc4e0 0x5620083de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.336+0000 7fa0be9e0740 4 client.26912 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:24.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.336+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1498534555 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620083356d0 con 0x5620083dc4e0 2024-04-16T17:09:24.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.349+0000 7fa0b8a00640 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5620083dc4e0 0x5620083de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.386+0000 7fa09aa00640 10 client.26912 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:24.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.386+0000 7fa0b8a00640 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5620083dc4e0 0x5620083de8d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778ca62c07b24f97 server_cookie=1c59a2240fe1eac9 in_seq=0 out_seq=0 2024-04-16T17:09:24.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa09aa00640 1 -- 172.21.15.18:0/1498534555 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620083356d0 con 0x5620083dc4e0 2024-04-16T17:09:24.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa09aa00640 10 client.26912 handle_command_reply: tid=0 2024-04-16T17:09:24.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 client.26912 shutdown 2024-04-16T17:09:24.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1498534555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 msgr2=0x5620083c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c91f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac01b040 tx=0x7fa0ac009c00 comp rx=0 tx=0).stop 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1498534555 shutdown_connections 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5620083dc4e0 0x5620083de8d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1498534555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083c91f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa09aa00640 0 client.26912 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.389+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1498534555 >> 172.21.15.18:0/1498534555 conn(0x5620083bd3d0 msgr2=0x5620083bf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.390+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1498534555 shutdown_connections 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.390+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1498534555 wait complete. 2024-04-16T17:09:24.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.390+0000 7fa0be9e0740 20 client.26912 trim_cache size 0 max 16384 2024-04-16T17:09:24.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.413+0000 7fa0be9e0740 1 Processor -- start 2024-04-16T17:09:24.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.413+0000 7fa0be9e0740 1 -- start start 2024-04-16T17:09:24.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.413+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083cb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083e4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b41c0 con 0x5620083c35c0 2024-04-16T17:09:24.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b1a90 con 0x5620083c2ca0 2024-04-16T17:09:24.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41af8e0 con 0x5620083cc1a0 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:47030/0 (socket says 172.21.15.18:47030) 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0b9e00640 1 -- 172.21.15.18:0/4116910639 learned_addr learned my addr 172.21.15.18:0/4116910639 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0b8a00640 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0b9e00640 1 -- 172.21.15.18:0/4116910639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 msgr2=0x5620083e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.414+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083e4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.415+0000 7fa0b9e00640 1 -- 172.21.15.18:0/4116910639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 msgr2=0x5620083d7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.415+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.415+0000 7fa0b9e00640 1 -- 172.21.15.18:0/4116910639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562007ffa170 con 0x5620083c2ca0 2024-04-16T17:09:24.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.415+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083cb920 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006010 tx=0x7fa0ac014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef77197b2ab238c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.415+0000 7fa0a5c00640 1 -- 172.21.15.18:0/4116910639 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac003d00 con 0x5620083c2ca0 2024-04-16T17:09:24.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.415+0000 7fa0a5c00640 1 -- 172.21.15.18:0/4116910639 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ac02c070 con 0x5620083c2ca0 2024-04-16T17:09:24.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.415+0000 7fa0a5c00640 1 -- 172.21.15.18:0/4116910639 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac019740 con 0x5620083c2ca0 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 -- 172.21.15.18:0/4116910639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 msgr2=0x5620083cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083cb920 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006010 tx=0x7fa0ac014040 comp rx=0 tx=0).stop 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 -- 172.21.15.18:0/4116910639 shutdown_connections 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083cc1a0 0x5620083e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/4116910639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083c2ca0 0x5620083cb920 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 -- 172.21.15.18:0/4116910639 >> 172.21.15.18:0/4116910639 conn(0x5620083bd3d0 msgr2=0x5620083c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 -- 172.21.15.18:0/4116910639 shutdown_connections 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.416+0000 7fa0be9e0740 1 -- 172.21.15.18:0/4116910639 wait complete. 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.417+0000 7fa0be9e0740 1 -- start start 2024-04-16T17:09:24.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.417+0000 7fa0be9e0740 1 Processor -- start 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.417+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083c2ca0 0x5620083d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.417+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.417+0000 7fa0be9e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083cc1a0 0x5620083d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.417+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41aff50 con 0x5620083c35c0 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.417+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b13c0 con 0x5620083cc1a0 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.417+0000 7fa0be9e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b0620 con 0x5620083c2ca0 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0be9e0740 10 client.0 _ll_register_callbacks cb 0x7fa0ac003c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083cc1a0 0x5620083d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083cc1a0 0x5620083d14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:47038/0 (socket says 172.21.15.18:47038) 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b9e00640 1 -- 172.21.15.18:0/1208902653 learned_addr learned my addr 172.21.15.18:0/1208902653 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083c2ca0 0x5620083d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b8a00640 1 -- 172.21.15.18:0/1208902653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083c2ca0 msgr2=0x5620083d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b8a00640 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083c2ca0 0x5620083d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b8a00640 1 -- 172.21.15.18:0/1208902653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083cc1a0 msgr2=0x5620083d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b8a00640 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083cc1a0 0x5620083d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b8a00640 1 -- 172.21.15.18:0/1208902653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620083d1ce0 con 0x5620083c35c0 2024-04-16T17:09:24.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.418+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083cc1a0 0x5620083d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:24.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.419+0000 7fa0b8a00640 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d48a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0036040 tx=0x7fa0a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caab2874ab30af8d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.419+0000 7fa09aa00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:09:24.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.419+0000 7fa09aa00640 1 -- 172.21.15.18:0/1208902653 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004240 con 0x5620083c35c0 2024-04-16T17:09:24.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.419+0000 7fa09aa00640 1 -- 172.21.15.18:0/1208902653 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fa0a00043e0 con 0x5620083c35c0 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.419+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1208902653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0ac02c070 con 0x5620083c35c0 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.419+0000 7fa09aa00640 1 -- 172.21.15.18:0/1208902653 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa0a0057050 con 0x5620083c35c0 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.419+0000 7fa0be9e0740 10 client.16716 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.419+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1208902653 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620084435c0 con 0x5620083c35c0 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa09aa00640 1 -- 172.21.15.18:0/1208902653 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ac02c070 con 0x5620083c35c0 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa09aa00640 1 -- 172.21.15.18:0/1208902653 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa0a0047330 con 0x5620083c35c0 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa0be9e0740 10 client.16716 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa0be9e0740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa0be9e0740 20 client.16716 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562008443cc0 0x5620084460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa0be9e0740 4 client.16716 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1208902653 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56200832f540 con 0x562008443cc0 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.420+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562008443cc0 0x5620084460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.421+0000 7fa0b9e00640 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562008443cc0 0x5620084460e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f9d167fe607c31 server_cookie=9eca35625fc74465 in_seq=0 out_seq=0 2024-04-16T17:09:24.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.421+0000 7fa09aa00640 10 client.16716 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa09aa00640 1 -- 172.21.15.18:0/1208902653 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56200832f540 con 0x562008443cc0 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa09aa00640 10 client.16716 handle_command_reply: tid=0 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 client.16716 shutdown 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1208902653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 msgr2=0x5620083d48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d48a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0036040 tx=0x7fa0a0000bc0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1208902653 shutdown_connections 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562008443cc0 0x5620084460e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620083cc1a0 0x5620083d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620083c35c0 0x5620083d48a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 --2- 172.21.15.18:0/1208902653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620083c2ca0 0x5620083d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.423+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1208902653 >> 172.21.15.18:0/1208902653 conn(0x5620083bd3d0 msgr2=0x5620083be600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.424+0000 7fa09aa00640 0 client.16716 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.424+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1208902653 shutdown_connections 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.424+0000 7fa0be9e0740 1 -- 172.21.15.18:0/1208902653 wait complete. 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.424+0000 7fa0be9e0740 20 client.16716 trim_cache size 0 max 16384 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "d10ace3b-365a-4928-a036-343ce57cf325", 2024-04-16T17:09:24.426 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:09:24.427 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:09:24.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 -- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa09c0a3d20 msgr2=0x7fa09c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80cda30 tx=0x7fa0a8011000 comp rx=0 tx=0).stop 2024-04-16T17:09:24.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 -- 172.21.15.18:0/858325036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0b4104450 msgr2=0x7fa0b41a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0b4104450 0x7fa0b41a08b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b450 tx=0x7fa0a800b910 comp rx=0 tx=0).stop 2024-04-16T17:09:24.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 -- 172.21.15.18:0/858325036 shutdown_connections 2024-04-16T17:09:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0b419aa60 0x7fa0b419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b4109520 0x7fa0b41a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 --2- 172.21.15.18:0/858325036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0b4104450 0x7fa0b41a08b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 -- 172.21.15.18:0/858325036 >> 172.21.15.18:0/858325036 conn(0x7fa0b40ffc70 msgr2=0x7fa0b4100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 -- 172.21.15.18:0/858325036 shutdown_connections 2024-04-16T17:09:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.425+0000 7fa0bbe00640 1 -- 172.21.15.18:0/858325036 wait complete. 2024-04-16T17:09:24.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:09:24.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:09:24.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 163674 3.94 MB/sec execute 97 sec latency 511.632 ms 2024-04-16T17:09:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 -- 172.21.15.18:0/1922401758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c101570 msgr2=0x7fb56c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 --2- 172.21.15.18:0/1922401758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c101570 0x7fb56c10e230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb558009900 tx=0x7fb55802bc40 comp rx=0 tx=0).stop 2024-04-16T17:09:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 -- 172.21.15.18:0/1922401758 shutdown_connections 2024-04-16T17:09:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 --2- 172.21.15.18:0/1922401758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb56c10e770 0x7fb56c110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 --2- 172.21.15.18:0/1922401758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c101570 0x7fb56c10e230 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 --2- 172.21.15.18:0/1922401758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c100bc0 0x7fb56c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 -- 172.21.15.18:0/1922401758 >> 172.21.15.18:0/1922401758 conn(0x7fb56c077ab0 msgr2=0x7fb56c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 -- 172.21.15.18:0/1922401758 shutdown_connections 2024-04-16T17:09:24.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.637+0000 7fb56be00640 1 -- 172.21.15.18:0/1922401758 wait complete. 2024-04-16T17:09:24.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.639+0000 7fb56be00640 1 Processor -- start 2024-04-16T17:09:24.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.639+0000 7fb56be00640 1 -- start start 2024-04-16T17:09:24.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.639+0000 7fb56be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb56c100bc0 0x7fb56c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.639+0000 7fb56be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c101570 0x7fb56c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c10e770 0x7fb56c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c114460 con 0x7fb56c101570 2024-04-16T17:09:24.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1142e0 con 0x7fb56c100bc0 2024-04-16T17:09:24.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1145e0 con 0x7fb56c10e770 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb56c100bc0 0x7fb56c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb56c100bc0 0x7fb56c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:47062/0 (socket says 172.21.15.18:47062) 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56aa00640 1 -- 172.21.15.18:0/1020210066 learned_addr learned my addr 172.21.15.18:0/1020210066 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56b400640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c10e770 0x7fb56c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56a000640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c101570 0x7fb56c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56aa00640 1 -- 172.21.15.18:0/1020210066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c10e770 msgr2=0x7fb56c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56aa00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c10e770 0x7fb56c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56aa00640 1 -- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c101570 msgr2=0x7fb56c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56aa00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c101570 0x7fb56c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.640+0000 7fb56aa00640 1 -- 172.21.15.18:0/1020210066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56c19aff0 con 0x7fb56c100bc0 2024-04-16T17:09:24.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.641+0000 7fb56aa00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb56c100bc0 0x7fb56c1a06b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00e9f0 tx=0x7fb55c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.641+0000 7fb563400640 1 -- 172.21.15.18:0/1020210066 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c00cd80 con 0x7fb56c100bc0 2024-04-16T17:09:24.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.641+0000 7fb563400640 1 -- 172.21.15.18:0/1020210066 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb55c010910 con 0x7fb56c100bc0 2024-04-16T17:09:24.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.641+0000 7fb56be00640 1 -- 172.21.15.18:0/1020210066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56c19b2e0 con 0x7fb56c100bc0 2024-04-16T17:09:24.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.641+0000 7fb563400640 1 -- 172.21.15.18:0/1020210066 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c018990 con 0x7fb56c100bc0 2024-04-16T17:09:24.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.641+0000 7fb56be00640 1 -- 172.21.15.18:0/1020210066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb56c1a7500 con 0x7fb56c100bc0 2024-04-16T17:09:24.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.645+0000 7fb563400640 1 -- 172.21.15.18:0/1020210066 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb55c010ac0 con 0x7fb56c100bc0 2024-04-16T17:09:24.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.645+0000 7fb57438f740 1 Processor -- start 2024-04-16T17:09:24.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.646+0000 7fb57438f740 1 -- start start 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.646+0000 7fb57438f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869b8390 0x55b1869b8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.646+0000 7fb57438f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 0x55b1869c1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.646+0000 7fb57438f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869c1890 0x55b1869c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.646+0000 7fb57438f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b18691e120 con 0x55b1869c1890 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.646+0000 7fb57438f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b1868c4340 con 0x55b1869b8390 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.646+0000 7fb57438f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b18691a0e0 con 0x55b1869b8cb0 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.646+0000 7fb56b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869b8390 0x55b1869b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869b8390 0x55b1869b8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:47074/0 (socket says 172.21.15.18:47074) 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56b400640 1 -- 172.21.15.18:0/3389535506 learned_addr learned my addr 172.21.15.18:0/3389535506 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56aa00640 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 0x55b1869c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56a000640 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869c1890 0x55b1869c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb563400640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5500a3da0 0x7fb5500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56b400640 1 -- 172.21.15.18:0/3389535506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 msgr2=0x55b1869c1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56b400640 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 0x55b1869c1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56b400640 1 -- 172.21.15.18:0/3389535506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869c1890 msgr2=0x55b1869c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb563400640 1 -- 172.21.15.18:0/1020210066 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb55c0d0ee0 con 0x7fb56c100bc0 2024-04-16T17:09:24.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56b400640 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869c1890 0x55b1869c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.647+0000 7fb56b400640 1 -- 172.21.15.18:0/3389535506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1866fa7d0 con 0x55b1869b8390 2024-04-16T17:09:24.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.648+0000 7fb56b400640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5500a3da0 0x7fb5500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.648+0000 7fb56b400640 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869b8390 0x55b1869b8770 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb564009f30 tx=0x7fb5640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e917965473dd90 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.648+0000 7fb557400640 1 -- 172.21.15.18:0/3389535506 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564019640 con 0x55b1869b8390 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb557400640 1 -- 172.21.15.18:0/3389535506 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb564019cc0 con 0x55b1869b8390 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb557400640 1 -- 172.21.15.18:0/3389535506 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56400fbf0 con 0x55b1869b8390 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb57438f740 1 -- 172.21.15.18:0/3389535506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869b8390 msgr2=0x55b1869b8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb56b400640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5500a3da0 0x7fb5500a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb564020000 tx=0x7fb564021000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb57438f740 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869b8390 0x55b1869b8770 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb564009f30 tx=0x7fb5640118d0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb57438f740 1 -- 172.21.15.18:0/3389535506 shutdown_connections 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb57438f740 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869c1890 0x55b1869c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb57438f740 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 0x55b1869c1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb57438f740 1 --2- 172.21.15.18:0/3389535506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869b8390 0x55b1869b8770 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.649+0000 7fb57438f740 1 -- 172.21.15.18:0/3389535506 >> 172.21.15.18:0/3389535506 conn(0x55b1869b2ac0 msgr2=0x55b1869b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.650+0000 7fb57438f740 1 -- 172.21.15.18:0/3389535506 shutdown_connections 2024-04-16T17:09:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.650+0000 7fb57438f740 1 -- 172.21.15.18:0/3389535506 wait complete. 2024-04-16T17:09:24.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.650+0000 7fb57438f740 1 -- start start 2024-04-16T17:09:24.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.650+0000 7fb57438f740 1 Processor -- start 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb57438f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb57438f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869bee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb57438f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb57438f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b1869c4680 con 0x55b1869b8cb0 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb57438f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b1868d7710 con 0x55b1869c1890 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb57438f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b1869c4800 con 0x55b1869b8390 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb57438f740 10 client.0 _ll_register_callbacks cb 0x55b1868d2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869be8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53676/0 (socket says 172.21.15.18:53676) 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb56aa00640 1 -- 172.21.15.18:0/1746758323 learned_addr learned my addr 172.21.15.18:0/1746758323 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb56b400640 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.651+0000 7fb56a000640 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869bee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb56b400640 1 -- 172.21.15.18:0/1746758323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 msgr2=0x55b1869be8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb56b400640 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869be8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb56b400640 1 -- 172.21.15.18:0/1746758323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 msgr2=0x55b1869bee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb56b400640 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869bee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb56b400640 1 -- 172.21.15.18:0/1746758323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1866303f0 con 0x55b1869c1890 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb56a000640 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869bee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb56b400640 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bf370 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb564005fd0 tx=0x7fb564019b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b830637900e67619 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb54be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.652+0000 7fb54be00640 1 -- 172.21.15.18:0/1746758323 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564033020 con 0x55b1869c1890 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.653+0000 7fb54be00640 1 -- 172.21.15.18:0/1746758323 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb56400f480 con 0x55b1869c1890 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.653+0000 7fb57438f740 1 -- 172.21.15.18:0/1746758323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb564019cc0 con 0x55b1869c1890 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.653+0000 7fb54be00640 1 -- 172.21.15.18:0/1746758323 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb56403e050 con 0x55b1869c1890 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.653+0000 7fb57438f740 10 client.26930 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.653+0000 7fb57438f740 1 -- 172.21.15.18:0/1746758323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b1865f00f0 con 0x55b1869c1890 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.653+0000 7fb54be00640 1 -- 172.21.15.18:0/1746758323 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb564019cc0 con 0x55b1869c1890 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.654+0000 7fb54be00640 1 -- 172.21.15.18:0/1746758323 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb56401bbe0 con 0x55b1869c1890 2024-04-16T17:09:24.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.654+0000 7fb57438f740 10 client.26930 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:24.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.654+0000 7fb57438f740 10 client.26930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:24.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.654+0000 7fb57438f740 20 client.26930 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:24.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.654+0000 7fb57438f740 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b1869d1b30 0x55b1869d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.654+0000 7fb57438f740 4 client.26930 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:24.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.654+0000 7fb57438f740 1 -- 172.21.15.18:0/1746758323 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b18692bb40 con 0x55b1869d1b30 2024-04-16T17:09:24.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.667+0000 7fb56aa00640 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b1869d1b30 0x55b1869d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.675+0000 7fb56aa00640 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b1869d1b30 0x55b1869d3f20 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeea717c87da4419 server_cookie=992d4f03f68a4bd2 in_seq=0 out_seq=0 2024-04-16T17:09:24.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.675+0000 7fb54be00640 10 client.26930 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:24.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.675+0000 7fb54be00640 1 -- 172.21.15.18:0/1746758323 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b18692bb40 con 0x55b1869d1b30 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.675+0000 7fb54be00640 10 client.26930 handle_command_reply: tid=0 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.675+0000 7fb57438f740 1 client.26930 shutdown 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 -- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 msgr2=0x55b1869bf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bf370 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb564005fd0 tx=0x7fb564019b90 comp rx=0 tx=0).stop 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 -- 172.21.15.18:0/1746758323 shutdown_connections 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b1869d1b30 0x55b1869d3f20 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bf370 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869bee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 --2- 172.21.15.18:0/1746758323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869be8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb54be00640 0 client.26930 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 -- 172.21.15.18:0/1746758323 >> 172.21.15.18:0/1746758323 conn(0x55b1869b2ac0 msgr2=0x55b1869b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 -- 172.21.15.18:0/1746758323 shutdown_connections 2024-04-16T17:09:24.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 1 -- 172.21.15.18:0/1746758323 wait complete. 2024-04-16T17:09:24.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.676+0000 7fb57438f740 20 client.26930 trim_cache size 0 max 16384 2024-04-16T17:09:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.696+0000 7fb57438f740 1 Processor -- start 2024-04-16T17:09:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.696+0000 7fb57438f740 1 -- start start 2024-04-16T17:09:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.696+0000 7fb57438f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869c1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.696+0000 7fb57438f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.696+0000 7fb57438f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869b4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.696+0000 7fb57438f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b3f40 con 0x55b1869b8cb0 2024-04-16T17:09:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.696+0000 7fb57438f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b1810 con 0x55b1869c1890 2024-04-16T17:09:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.696+0000 7fb57438f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1af660 con 0x55b1869b8390 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869c1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53688/0 (socket says 172.21.15.18:53688) 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 -- 172.21.15.18:0/1913875436 learned_addr learned my addr 172.21.15.18:0/1913875436 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56a000640 1 --2- 172.21.15.18:0/1913875436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 -- 172.21.15.18:0/1913875436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 msgr2=0x55b1869b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 --2- 172.21.15.18:0/1913875436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 -- 172.21.15.18:0/1913875436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 msgr2=0x55b1869cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 --2- 172.21.15.18:0/1913875436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 -- 172.21.15.18:0/1913875436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1869b8ae0 con 0x55b1869b8390 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.697+0000 7fb56aa00640 1 --2- 172.21.15.18:0/1913875436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869c1020 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0994a0 tx=0x7fb55c0d33f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a91fc02922854185 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.698+0000 7fb556a00640 1 -- 172.21.15.18:0/1913875436 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0a0e80 con 0x55b1869b8390 2024-04-16T17:09:24.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.698+0000 7fb556a00640 1 -- 172.21.15.18:0/1913875436 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb55c0a5740 con 0x55b1869b8390 2024-04-16T17:09:24.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.698+0000 7fb556a00640 1 -- 172.21.15.18:0/1913875436 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c099140 con 0x55b1869b8390 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 -- 172.21.15.18:0/1913875436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 msgr2=0x55b1869c1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 --2- 172.21.15.18:0/1913875436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869c1020 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0994a0 tx=0x7fb55c0d33f0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 -- 172.21.15.18:0/1913875436 shutdown_connections 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 --2- 172.21.15.18:0/1913875436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 --2- 172.21.15.18:0/1913875436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8cb0 0x55b1869cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 --2- 172.21.15.18:0/1913875436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8390 0x55b1869c1020 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 -- 172.21.15.18:0/1913875436 >> 172.21.15.18:0/1913875436 conn(0x55b1869b2ac0 msgr2=0x55b1869bd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 -- 172.21.15.18:0/1913875436 shutdown_connections 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 -- 172.21.15.18:0/1913875436 wait complete. 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 -- start start 2024-04-16T17:09:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.699+0000 7fb57438f740 1 Processor -- start 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb57438f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8390 0x55b1869bb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb57438f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 0x55b1869bbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb57438f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb57438f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1afcd0 con 0x55b1869b8390 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb57438f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b1140 con 0x55b1869c1890 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb57438f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b03a0 con 0x55b1869b8cb0 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb57438f740 10 client.0 _ll_register_callbacks cb 0x7fb564009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8390 0x55b1869bb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb56a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 0x55b1869bbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb56b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8390 0x55b1869bb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:32772/0 (socket says 172.21.15.18:32772) 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.700+0000 7fb56aa00640 1 -- 172.21.15.18:0/4136714424 learned_addr learned my addr 172.21.15.18:0/4136714424 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb56aa00640 1 -- 172.21.15.18:0/4136714424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 msgr2=0x55b1869bbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb56aa00640 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 0x55b1869bbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb56aa00640 1 -- 172.21.15.18:0/4136714424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 msgr2=0x55b1869bc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:24.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb56aa00640 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb56aa00640 1 -- 172.21.15.18:0/4136714424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b186a37270 con 0x55b1869b8390 2024-04-16T17:09:24.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb56b400640 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:24.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb56aa00640 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8390 0x55b1869bb8c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0ce330 tx=0x7fb55c005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45336c195158e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:24.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb54be00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:09:24.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb54be00640 1 -- 172.21.15.18:0/4136714424 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0a0e80 con 0x55b1869b8390 2024-04-16T17:09:24.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb54be00640 1 -- 172.21.15.18:0/4136714424 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb55c0ce570 con 0x55b1869b8390 2024-04-16T17:09:24.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb57438f740 1 -- 172.21.15.18:0/4136714424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb55c0a5740 con 0x55b1869b8390 2024-04-16T17:09:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.701+0000 7fb54be00640 1 -- 172.21.15.18:0/4136714424 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb55c099140 con 0x55b1869b8390 2024-04-16T17:09:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb57438f740 10 client.16734 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb57438f740 1 -- 172.21.15.18:0/4136714424 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b186a38580 con 0x55b1869b8390 2024-04-16T17:09:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb54be00640 1 -- 172.21.15.18:0/4136714424 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb55c0a5740 con 0x55b1869b8390 2024-04-16T17:09:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb54be00640 1 -- 172.21.15.18:0/4136714424 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb55c09c0c0 con 0x55b1869b8390 2024-04-16T17:09:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb57438f740 10 client.16734 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb57438f740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb57438f740 20 client.16734 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb57438f740 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b186a38b90 0x55b186a3afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb57438f740 4 client.16734 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb57438f740 1 -- 172.21.15.18:0/4136714424 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1869259b0 con 0x55b186a38b90 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.702+0000 7fb56a000640 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b186a38b90 0x55b186a3afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.703+0000 7fb56a000640 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b186a38b90 0x55b186a3afb0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf1244c794801ed server_cookie=4a9b503f0af6c256 in_seq=0 out_seq=0 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.703+0000 7fb54be00640 10 client.16734 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.703+0000 7fb54be00640 1 -- 172.21.15.18:0/4136714424 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b1869259b0 con 0x55b186a38b90 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.703+0000 7fb54be00640 10 client.16734 handle_command_reply: tid=0 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.703+0000 7fb57438f740 1 client.16734 shutdown 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.703+0000 7fb57438f740 1 -- 172.21.15.18:0/4136714424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8390 msgr2=0x55b1869bb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.703+0000 7fb57438f740 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8390 0x55b1869bb8c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0ce330 tx=0x7fb55c005c60 comp rx=0 tx=0).stop 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 1 -- 172.21.15.18:0/4136714424 shutdown_connections 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b186a38b90 0x55b186a3afb0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1869c1890 0x55b1869bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b1869b8cb0 0x55b1869bbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb54be00640 0 client.16734 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 1 --2- 172.21.15.18:0/4136714424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b1869b8390 0x55b1869bb8c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 1 -- 172.21.15.18:0/4136714424 >> 172.21.15.18:0/4136714424 conn(0x55b1869b2ac0 msgr2=0x55b1869b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 1 -- 172.21.15.18:0/4136714424 shutdown_connections 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 1 -- 172.21.15.18:0/4136714424 wait complete. 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb57438f740 20 client.16734 trim_cache size 0 max 16384 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stdout: "d10ace3b-365a-4928-a036-343ce57cf325": { 2024-04-16T17:09:24.708 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:09:24.709 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "d10ace3b-365a-4928-a036-343ce57cf325", 2024-04-16T17:09:24.709 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:09:24.709 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:09:24.709 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:09:24.709 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:09:24.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb56be00640 1 -- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5500a3da0 msgr2=0x7fb5500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb56be00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5500a3da0 0x7fb5500a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb564020000 tx=0x7fb564021000 comp rx=0 tx=0).stop 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb56be00640 1 -- 172.21.15.18:0/1020210066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb56c100bc0 msgr2=0x7fb56c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.704+0000 7fb56be00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb56c100bc0 0x7fb56c1a06b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00e9f0 tx=0x7fb55c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.705+0000 7fb56be00640 1 -- 172.21.15.18:0/1020210066 shutdown_connections 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.705+0000 7fb56be00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5500a3da0 0x7fb5500a6260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.705+0000 7fb56be00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c10e770 0x7fb56c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.705+0000 7fb56be00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c101570 0x7fb56c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.705+0000 7fb56be00640 1 --2- 172.21.15.18:0/1020210066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb56c100bc0 0x7fb56c1a06b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.705+0000 7fb56be00640 1 -- 172.21.15.18:0/1020210066 >> 172.21.15.18:0/1020210066 conn(0x7fb56c077ab0 msgr2=0x7fb56c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.705+0000 7fb56be00640 1 -- 172.21.15.18:0/1020210066 shutdown_connections 2024-04-16T17:09:24.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:24.705+0000 7fb56be00640 1 -- 172.21.15.18:0/1020210066 wait complete. 2024-04-16T17:09:24.725 INFO:tasks.cephfs.filesystem:scrub status for tag:d10ace3b-365a-4928-a036-343ce57cf325 - {'path': '/', 'tag': 'd10ace3b-365a-4928-a036-343ce57cf325', 'options': 'recursive,force'} 2024-04-16T17:09:25.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 164404 3.92 MB/sec execute 98 sec latency 50.577 ms 2024-04-16T17:09:25.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:25 smithi022 ceph-mon[36023]: pgmap v143: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-16T17:09:25.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:25 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:09:25.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:25 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:25.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:25 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:09:25.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:25 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:25 smithi043 ceph-mon[36901]: pgmap v143: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-16T17:09:26.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:25 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:09:26.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:25 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:26.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:25 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:09:26.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:25 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:26.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 165673 3.94 MB/sec execute 99 sec latency 11.737 ms 2024-04-16T17:09:27.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 166874 4.00 MB/sec execute 100 sec latency 19.166 ms 2024-04-16T17:09:27.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:27 smithi022 ceph-mon[36023]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-04-16T17:09:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:27 smithi043 ceph-mon[36901]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-04-16T17:09:28.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 167161 3.97 MB/sec execute 101 sec latency 53.665 ms 2024-04-16T17:09:29.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:25 smithi018 ceph-mon[31776]: pgmap v143: 129 pgs: 129 active+clean; 41 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-16T17:09:29.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:25 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:09:29.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:25 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:29.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:25 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:09:29.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:25 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:29.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:27 smithi018 ceph-mon[31776]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-04-16T17:09:29.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 167232 3.94 MB/sec execute 102 sec latency 623.174 ms 2024-04-16T17:09:29.803 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:29 smithi043 ceph-mon[36901]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:09:29.804 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:29 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:09:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:29 smithi022 ceph-mon[36023]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:09:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:29 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:09:30.048 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:29 smithi018 ceph-mon[31776]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:09:30.049 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:29 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:09:30.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 168198 3.95 MB/sec execute 103 sec latency 706.760 ms 2024-04-16T17:09:31.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 169401 3.95 MB/sec execute 104 sec latency 11.737 ms 2024-04-16T17:09:31.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:31 smithi022 ceph-mon[36023]: pgmap v146: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-16T17:09:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:31 smithi043 ceph-mon[36901]: pgmap v146: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-16T17:09:32.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 170529 4.00 MB/sec execute 105 sec latency 11.952 ms 2024-04-16T17:09:33.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 170793 3.97 MB/sec execute 106 sec latency 89.023 ms 2024-04-16T17:09:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:33 smithi022 ceph-mon[36023]: pgmap v147: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:09:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:33 smithi043 ceph-mon[36901]: pgmap v147: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:09:34.561 INFO:tasks.workunit.client.0.smithi018.stdout: 1 171156 3.94 MB/sec execute 107 sec latency 60.006 ms 2024-04-16T17:09:35.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 171757 3.94 MB/sec execute 108 sec latency 68.123 ms 2024-04-16T17:09:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:35 smithi022 ceph-mon[36023]: pgmap v148: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:09:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:35 smithi043 ceph-mon[36901]: pgmap v148: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:09:36.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 172771 3.93 MB/sec execute 109 sec latency 16.026 ms 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.059+0000 7fb5e7e00640 1 -- 172.21.15.18:0/4124095468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 msgr2=0x7fb5e810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.059+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/4124095468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 0x7fb5e810bde0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009860 tx=0x7fb5d402bc10 comp rx=0 tx=0).stop 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.060+0000 7fb5e7e00640 1 -- 172.21.15.18:0/4124095468 shutdown_connections 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.060+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/4124095468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5e810c400 0x7fb5e810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.060+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/4124095468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 0x7fb5e810bde0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.060+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/4124095468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e8102fe0 0x7fb5e81033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.060+0000 7fb5e7e00640 1 -- 172.21.15.18:0/4124095468 >> 172.21.15.18:0/4124095468 conn(0x7fb5e80fc7e0 msgr2=0x7fb5e80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.060+0000 7fb5e7e00640 1 -- 172.21.15.18:0/4124095468 shutdown_connections 2024-04-16T17:09:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.060+0000 7fb5e7e00640 1 -- 172.21.15.18:0/4124095468 wait complete. 2024-04-16T17:09:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.061+0000 7fb5e7e00640 1 Processor -- start 2024-04-16T17:09:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.061+0000 7fb5e7e00640 1 -- start start 2024-04-16T17:09:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e8102fe0 0x7fb5e8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 0x7fb5e8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5e810c400 0x7fb5e819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e7e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81121e0 con 0x7fb5e8102fe0 2024-04-16T17:09:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e7e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8112060 con 0x7fb5e810c400 2024-04-16T17:09:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8112360 con 0x7fb5e8103900 2024-04-16T17:09:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 0x7fb5e8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 0x7fb5e8198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44486/0 (socket says 172.21.15.18:44486) 2024-04-16T17:09:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e6000640 1 -- 172.21.15.18:0/1276465106 learned_addr learned my addr 172.21.15.18:0/1276465106 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.062+0000 7fb5e7400640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5e810c400 0x7fb5e819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.063+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e8102fe0 0x7fb5e8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.063+0000 7fb5e6000640 1 -- 172.21.15.18:0/1276465106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5e810c400 msgr2=0x7fb5e819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.063+0000 7fb5e6000640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5e810c400 0x7fb5e819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.063+0000 7fb5e6000640 1 -- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e8102fe0 msgr2=0x7fb5e8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.063+0000 7fb5e6000640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e8102fe0 0x7fb5e8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.063+0000 7fb5e6000640 1 -- 172.21.15.18:0/1276465106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e819d6c0 con 0x7fb5e8103900 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.063+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e8102fe0 0x7fb5e8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.063+0000 7fb5e6000640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 0x7fb5e8198c60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4037040 tx=0x7fb5d40344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.064+0000 7fb5df400640 1 -- 172.21.15.18:0/1276465106 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4041020 con 0x7fb5e8103900 2024-04-16T17:09:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.064+0000 7fb5df400640 1 -- 172.21.15.18:0/1276465106 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d402ca90 con 0x7fb5e8103900 2024-04-16T17:09:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.064+0000 7fb5df400640 1 -- 172.21.15.18:0/1276465106 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4039040 con 0x7fb5e8103900 2024-04-16T17:09:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.064+0000 7fb5e7e00640 1 -- 172.21.15.18:0/1276465106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e819d950 con 0x7fb5e8103900 2024-04-16T17:09:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.064+0000 7fb5e7e00640 1 -- 172.21.15.18:0/1276465106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e8100b60 con 0x7fb5e8103900 2024-04-16T17:09:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.067+0000 7fb5f041a740 1 Processor -- start 2024-04-16T17:09:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.067+0000 7fb5f041a740 1 -- start start 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595effeb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595effebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff5000 0x5595efff73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5f041a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5595eff50780 con 0x5595effeba80 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5f041a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5595efef6f40 con 0x5595effec7f0 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5f041a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5595eff4c740 con 0x5595efff5000 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595effeb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595effeb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55006/0 (socket says 172.21.15.18:55006) 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e7400640 1 -- 172.21.15.18:0/2752078295 learned_addr learned my addr 172.21.15.18:0/2752078295 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e7400640 1 -- 172.21.15.18:0/2752078295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff5000 msgr2=0x5595efff73f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e7400640 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff5000 0x5595efff73f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e7400640 1 -- 172.21.15.18:0/2752078295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 msgr2=0x5595effebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595effebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e7400640 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595effebf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.068+0000 7fb5e7400640 1 -- 172.21.15.18:0/2752078295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595efc63210 con 0x5595effec7f0 2024-04-16T17:09:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.069+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595effebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.069+0000 7fb5e7400640 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595effeb540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009be0 tx=0x7fb5e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83a98babd1aa138 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.069+0000 7fb5d3e00640 1 -- 172.21.15.18:0/2752078295 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e001e070 con 0x5595effec7f0 2024-04-16T17:09:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.069+0000 7fb5d3e00640 1 -- 172.21.15.18:0/2752078295 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e0009e50 con 0x5595effec7f0 2024-04-16T17:09:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.069+0000 7fb5d3e00640 1 -- 172.21.15.18:0/2752078295 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000ea20 con 0x5595effec7f0 2024-04-16T17:09:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.070+0000 7fb5f041a740 1 -- 172.21.15.18:0/2752078295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 msgr2=0x5595effeb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.070+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595effeb540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009be0 tx=0x7fb5e0011810 comp rx=0 tx=0).stop 2024-04-16T17:09:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.070+0000 7fb5f041a740 1 -- 172.21.15.18:0/2752078295 shutdown_connections 2024-04-16T17:09:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.070+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff5000 0x5595efff73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.070+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595effebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.070+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2752078295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595effeb540 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.070+0000 7fb5f041a740 1 -- 172.21.15.18:0/2752078295 >> 172.21.15.18:0/2752078295 conn(0x5595eff865d0 msgr2=0x5595effe6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.070+0000 7fb5df400640 1 -- 172.21.15.18:0/1276465106 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5d4044020 con 0x7fb5e8103900 2024-04-16T17:09:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.071+0000 7fb5f041a740 1 -- 172.21.15.18:0/2752078295 shutdown_connections 2024-04-16T17:09:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.071+0000 7fb5f041a740 1 -- 172.21.15.18:0/2752078295 wait complete. 2024-04-16T17:09:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.071+0000 7fb5f041a740 1 -- start start 2024-04-16T17:09:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.072+0000 7fb5f041a740 1 Processor -- start 2024-04-16T17:09:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.072+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595efff1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.072+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595efff2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.072+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff2980 0x5595f00099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.072+0000 7fb5f041a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5595efff7d30 con 0x5595effeba80 2024-04-16T17:09:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.072+0000 7fb5f041a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5595eff0a390 con 0x5595effec7f0 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.072+0000 7fb5f041a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5595efff7eb0 con 0x5595efff2980 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.072+0000 7fb5f041a740 10 client.0 _ll_register_callbacks cb 0x5595efef3f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5df400640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5cc0a3ca0 0x7fb5cc0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff2980 0x5595f00099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595efff2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595efff2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55020/0 (socket says 172.21.15.18:55020) 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff2980 0x5595f00099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44500/0 (socket says 172.21.15.18:44500) 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5e6a00640 1 -- 172.21.15.18:0/2766435812 learned_addr learned my addr 172.21.15.18:0/2766435812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5e7400640 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595efff1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5df400640 1 -- 172.21.15.18:0/1276465106 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb5d40f2130 con 0x7fb5e8103900 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5e7400640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5cc0a3ca0 0x7fb5cc0a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.073+0000 7fb5e6a00640 1 -- 172.21.15.18:0/2766435812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff2980 msgr2=0x5595f00099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff2980 0x5595f00099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5e6a00640 1 -- 172.21.15.18:0/2766435812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 msgr2=0x5595efff1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595efff1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5e6a00640 1 -- 172.21.15.18:0/2766435812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595efd2f620 con 0x5595effec7f0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5e7400640 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595efff1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5e6000640 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff2980 0x5595f00099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595efff2440 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009ea0 tx=0x7fb5d8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8f3a314031ac00 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.074+0000 7fb5e7400640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5cc0a3ca0 0x7fb5cc0a6160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0011d10 tx=0x7fb5e0005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.075+0000 7fb5d0c00640 1 -- 172.21.15.18:0/2766435812 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8010980 con 0x5595effec7f0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.075+0000 7fb5f041a740 1 -- 172.21.15.18:0/2766435812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e0009e50 con 0x5595effec7f0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.075+0000 7fb5d0c00640 1 -- 172.21.15.18:0/2766435812 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb5d8010b20 con 0x5595effec7f0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.075+0000 7fb5d0c00640 1 -- 172.21.15.18:0/2766435812 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5d8024050 con 0x5595effec7f0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.075+0000 7fb5f041a740 10 client.26948 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.075+0000 7fb5f041a740 1 -- 172.21.15.18:0/2766435812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595f000cb80 con 0x5595effec7f0 2024-04-16T17:09:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.076+0000 7fb5d0c00640 1 -- 172.21.15.18:0/2766435812 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e0009e50 con 0x5595effec7f0 2024-04-16T17:09:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.076+0000 7fb5d0c00640 1 -- 172.21.15.18:0/2766435812 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb5d800f040 con 0x5595effec7f0 2024-04-16T17:09:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.076+0000 7fb5f041a740 10 client.26948 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.076+0000 7fb5f041a740 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.076+0000 7fb5f041a740 20 client.26948 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.076+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595f000dff0 0x5595f00103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.076+0000 7fb5f041a740 4 client.26948 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.076+0000 7fb5f041a740 1 -- 172.21.15.18:0/2766435812 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595eff5e1a0 con 0x5595f000dff0 2024-04-16T17:09:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.077+0000 7fb5e6000640 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595f000dff0 0x5595f00103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.077+0000 7fb5e6000640 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595f000dff0 0x5595f00103e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f1bbcd82fe2bcd server_cookie=8e693e118531432e in_seq=0 out_seq=0 2024-04-16T17:09:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.077+0000 7fb5d0c00640 10 client.26948 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.079+0000 7fb5d0c00640 1 -- 172.21.15.18:0/2766435812 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595eff5e1a0 con 0x5595f000dff0 2024-04-16T17:09:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.079+0000 7fb5d0c00640 10 client.26948 handle_command_reply: tid=0 2024-04-16T17:09:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.079+0000 7fb5f041a740 1 client.26948 shutdown 2024-04-16T17:09:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 -- 172.21.15.18:0/2766435812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 msgr2=0x5595efff2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595efff2440 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009ea0 tx=0x7fb5d8009f90 comp rx=0 tx=0).stop 2024-04-16T17:09:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 -- 172.21.15.18:0/2766435812 shutdown_connections 2024-04-16T17:09:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595f000dff0 0x5595f00103e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595efff2980 0x5595f00099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595efff2440 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 --2- 172.21.15.18:0/2766435812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595effeba80 0x5595efff1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 -- 172.21.15.18:0/2766435812 >> 172.21.15.18:0/2766435812 conn(0x5595eff865d0 msgr2=0x5595effe84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5d0c00640 0 client.26948 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 -- 172.21.15.18:0/2766435812 shutdown_connections 2024-04-16T17:09:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 1 -- 172.21.15.18:0/2766435812 wait complete. 2024-04-16T17:09:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.080+0000 7fb5f041a740 20 client.26948 trim_cache size 0 max 16384 2024-04-16T17:09:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.100+0000 7fb5f041a740 1 Processor -- start 2024-04-16T17:09:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.100+0000 7fb5f041a740 1 -- start start 2024-04-16T17:09:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.100+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f00094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.100+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f0015cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.100+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.100+0000 7fb5f041a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81af9b0 con 0x5595efff2980 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.100+0000 7fb5f041a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ad280 con 0x5595effec7f0 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.100+0000 7fb5f041a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ab0d0 con 0x5595effeba80 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f0015cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f0015cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55030/0 (socket says 172.21.15.18:55030) 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 -- 172.21.15.18:0/1603053222 learned_addr learned my addr 172.21.15.18:0/1603053222 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e7400640 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f00094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 -- 172.21.15.18:0/1603053222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 msgr2=0x5595f00094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f00094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 -- 172.21.15.18:0/1603053222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 msgr2=0x5595f00161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 -- 172.21.15.18:0/1603053222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595effead50 con 0x5595effec7f0 2024-04-16T17:09:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.101+0000 7fb5e6000640 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f0015cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009830 tx=0x7fb5d4048040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37657396edbb0b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.102+0000 7fb5d3400640 1 -- 172.21.15.18:0/1603053222 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40bd790 con 0x5595effec7f0 2024-04-16T17:09:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.102+0000 7fb5d3400640 1 -- 172.21.15.18:0/1603053222 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d40be530 con 0x5595effec7f0 2024-04-16T17:09:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.102+0000 7fb5d3400640 1 -- 172.21.15.18:0/1603053222 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40bbed0 con 0x5595effec7f0 2024-04-16T17:09:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 -- 172.21.15.18:0/1603053222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 msgr2=0x5595f0015cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f0015cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009830 tx=0x7fb5d4048040 comp rx=0 tx=0).stop 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 -- 172.21.15.18:0/1603053222 shutdown_connections 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f0015cb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 --2- 172.21.15.18:0/1603053222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f00094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 -- 172.21.15.18:0/1603053222 >> 172.21.15.18:0/1603053222 conn(0x5595eff865d0 msgr2=0x5595efff07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 -- 172.21.15.18:0/1603053222 shutdown_connections 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 -- 172.21.15.18:0/1603053222 wait complete. 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.103+0000 7fb5f041a740 1 -- start start 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5f041a740 1 Processor -- start 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f0005cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f00010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5f041a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5f041a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ab740 con 0x5595efff2980 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5f041a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81acbb0 con 0x5595effec7f0 2024-04-16T17:09:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5f041a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81abe10 con 0x5595effeba80 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5f041a740 10 client.0 _ll_register_callbacks cb 0x7fb5d8003a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f00010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.104+0000 7fb5e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f0005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f0005cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44518/0 (socket says 172.21.15.18:44518) 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6a00640 1 -- 172.21.15.18:0/34056682 learned_addr learned my addr 172.21.15.18:0/34056682 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e7400640 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6a00640 1 -- 172.21.15.18:0/34056682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 msgr2=0x5595f00010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f00010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6a00640 1 -- 172.21.15.18:0/34056682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 msgr2=0x5595f00015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6a00640 1 -- 172.21.15.18:0/34056682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f0001df0 con 0x5595effeba80 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6000640 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f00010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e7400640 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5e6a00640 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f0005cf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5595efff34d0 tx=0x7fb5d80058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5743da58f25c996 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.105+0000 7fb5d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.106+0000 7fb5d0c00640 1 -- 172.21.15.18:0/34056682 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8011a10 con 0x5595effeba80 2024-04-16T17:09:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.106+0000 7fb5f041a740 1 -- 172.21.15.18:0/34056682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d40be530 con 0x5595effeba80 2024-04-16T17:09:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.106+0000 7fb5d0c00640 1 -- 172.21.15.18:0/34056682 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb5d8011bb0 con 0x5595effeba80 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.106+0000 7fb5d0c00640 1 -- 172.21.15.18:0/34056682 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5d8011db0 con 0x5595effeba80 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.106+0000 7fb5f041a740 10 client.26923 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.106+0000 7fb5f041a740 1 -- 172.21.15.18:0/34056682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595f00744a0 con 0x5595effeba80 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5d0c00640 1 -- 172.21.15.18:0/34056682 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d40be530 con 0x5595effeba80 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5d0c00640 1 -- 172.21.15.18:0/34056682 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb5d802f070 con 0x5595effeba80 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5f041a740 10 client.26923 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5f041a740 10 client.26923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5f041a740 20 client.26923 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5f041a740 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595f0074c70 0x5595f0077090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5f041a740 4 client.26923 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5f041a740 1 -- 172.21.15.18:0/34056682 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595eff58010 con 0x5595f0074c70 2024-04-16T17:09:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.107+0000 7fb5e6000640 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595f0074c70 0x5595f0077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.108+0000 7fb5e6000640 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595f0074c70 0x5595f0077090 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d8c293be3b044d server_cookie=d98a051c82867f5e in_seq=0 out_seq=0 2024-04-16T17:09:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.108+0000 7fb5d0c00640 10 client.26923 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.251+0000 7fb5d0c00640 1 -- 172.21.15.18:0/34056682 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595eff58010 con 0x5595f0074c70 2024-04-16T17:09:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.251+0000 7fb5d0c00640 10 client.26923 handle_command_reply: tid=0 2024-04-16T17:09:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.251+0000 7fb5f041a740 1 client.26923 shutdown 2024-04-16T17:09:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.251+0000 7fb5f041a740 1 -- 172.21.15.18:0/34056682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 msgr2=0x5595f0005cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.251+0000 7fb5f041a740 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f0005cf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5595efff34d0 tx=0x7fb5d80058d0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 1 -- 172.21.15.18:0/34056682 shutdown_connections 2024-04-16T17:09:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595f0074c70 0x5595f0077090 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595efff2980 0x5595f00015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595effec7f0 0x5595f00010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 1 --2- 172.21.15.18:0/34056682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595effeba80 0x5595f0005cf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 1 -- 172.21.15.18:0/34056682 >> 172.21.15.18:0/34056682 conn(0x5595eff865d0 msgr2=0x5595effe7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5d0c00640 0 client.26923 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 1 -- 172.21.15.18:0/34056682 shutdown_connections 2024-04-16T17:09:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 1 -- 172.21.15.18:0/34056682 wait complete. 2024-04-16T17:09:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.252+0000 7fb5f041a740 20 client.26923 trim_cache size 0 max 16384 2024-04-16T17:09:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:09:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:09:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:09:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 -- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5cc0a3ca0 msgr2=0x7fb5cc0a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5cc0a3ca0 0x7fb5cc0a6160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0011d10 tx=0x7fb5e0005fd0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 -- 172.21.15.18:0/1276465106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 msgr2=0x7fb5e8198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 0x7fb5e8198c60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4037040 tx=0x7fb5d40344c0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 -- 172.21.15.18:0/1276465106 shutdown_connections 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb5cc0a3ca0 0x7fb5cc0a6160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5e810c400 0x7fb5e819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e8103900 0x7fb5e8198c60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 --2- 172.21.15.18:0/1276465106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e8102fe0 0x7fb5e8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 -- 172.21.15.18:0/1276465106 >> 172.21.15.18:0/1276465106 conn(0x7fb5e80fc7e0 msgr2=0x7fb5e80fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 -- 172.21.15.18:0/1276465106 shutdown_connections 2024-04-16T17:09:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:37.256+0000 7fb5e7e00640 1 -- 172.21.15.18:0/1276465106 wait complete. 2024-04-16T17:09:37.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 174064 3.99 MB/sec execute 110 sec latency 52.061 ms 2024-04-16T17:09:37.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:37 smithi022 ceph-mon[36023]: pgmap v149: 129 pgs: 129 active+clean; 43 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-16T17:09:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:37 smithi043 ceph-mon[36901]: pgmap v149: 129 pgs: 129 active+clean; 43 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-16T17:09:38.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 174334 3.96 MB/sec execute 111 sec latency 75.061 ms 2024-04-16T17:09:39.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 174635 3.93 MB/sec execute 112 sec latency 306.242 ms 2024-04-16T17:09:39.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:31 smithi018 ceph-mon[31776]: pgmap v146: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-16T17:09:39.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:33 smithi018 ceph-mon[31776]: pgmap v147: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:09:39.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:35 smithi018 ceph-mon[31776]: pgmap v148: 129 pgs: 129 active+clean; 44 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-04-16T17:09:39.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:37 smithi018 ceph-mon[31776]: pgmap v149: 129 pgs: 129 active+clean; 43 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-16T17:09:39.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:39 smithi022 ceph-mon[36023]: pgmap v150: 129 pgs: 129 active+clean; 43 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 899 KiB/s wr, 81 op/s 2024-04-16T17:09:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:39 smithi043 ceph-mon[36901]: pgmap v150: 129 pgs: 129 active+clean; 43 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 899 KiB/s wr, 81 op/s 2024-04-16T17:09:40.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 175857 3.94 MB/sec execute 113 sec latency 19.258 ms 2024-04-16T17:09:41.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 177171 3.98 MB/sec execute 114 sec latency 28.325 ms 2024-04-16T17:09:41.678 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:39 smithi018 ceph-mon[31776]: pgmap v150: 129 pgs: 129 active+clean; 43 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 899 KiB/s wr, 81 op/s 2024-04-16T17:09:41.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:41 smithi022 ceph-mon[36023]: pgmap v151: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-16T17:09:42.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:41 smithi043 ceph-mon[36901]: pgmap v151: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-16T17:09:42.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 177885 3.99 MB/sec execute 115 sec latency 9.574 ms 2024-04-16T17:09:43.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 178422 3.97 MB/sec execute 116 sec latency 59.315 ms 2024-04-16T17:09:43.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:43 smithi022 ceph-mon[36023]: pgmap v152: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-16T17:09:44.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:43 smithi043 ceph-mon[36901]: pgmap v152: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-16T17:09:44.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 178813 3.96 MB/sec execute 117 sec latency 101.387 ms 2024-04-16T17:09:45.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 179783 3.95 MB/sec execute 118 sec latency 47.648 ms 2024-04-16T17:09:45.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:41 smithi018 ceph-mon[31776]: pgmap v151: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-16T17:09:45.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:43 smithi018 ceph-mon[31776]: pgmap v152: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-16T17:09:45.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:45 smithi018 ceph-mon[31776]: pgmap v153: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-16T17:09:45.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:45 smithi022 ceph-mon[36023]: pgmap v153: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-16T17:09:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:45 smithi043 ceph-mon[36901]: pgmap v153: 129 pgs: 129 active+clean; 42 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-16T17:09:46.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 180934 4.00 MB/sec execute 119 sec latency 26.173 ms 2024-04-16T17:09:47.562 INFO:tasks.workunit.client.0.smithi018.stdout: 1 181498 3.98 MB/sec execute 120 sec latency 8.900 ms 2024-04-16T17:09:47.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:47 smithi022 ceph-mon[36023]: pgmap v154: 129 pgs: 129 active+clean; 42 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-16T17:09:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:47 smithi043 ceph-mon[36901]: pgmap v154: 129 pgs: 129 active+clean; 42 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-16T17:09:48.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 182005 3.97 MB/sec execute 121 sec latency 74.862 ms 2024-04-16T17:09:49.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 182322 3.96 MB/sec execute 122 sec latency 153.752 ms 2024-04-16T17:09:49.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:47 smithi018 ceph-mon[31776]: pgmap v154: 129 pgs: 129 active+clean; 42 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-16T17:09:49.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:49 smithi018 ceph-mon[31776]: pgmap v155: 129 pgs: 129 active+clean; 42 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-16T17:09:49.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:49 smithi022 ceph-mon[36023]: pgmap v155: 129 pgs: 129 active+clean; 42 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-16T17:09:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:49 smithi043 ceph-mon[36901]: pgmap v155: 129 pgs: 129 active+clean; 42 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-16T17:09:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 -- 172.21.15.18:0/3176091805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd864102080 msgr2=0x7fd864102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 --2- 172.21.15.18:0/3176091805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd864102080 0x7fd864102460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009ca0 tx=0x7fd85c02c100 comp rx=0 tx=0).stop 2024-04-16T17:09:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 -- 172.21.15.18:0/3176091805 shutdown_connections 2024-04-16T17:09:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 --2- 172.21.15.18:0/3176091805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd864107150 0x7fd86410ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 --2- 172.21.15.18:0/3176091805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd864102a30 0x7fd864106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 --2- 172.21.15.18:0/3176091805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd864102080 0x7fd864102460 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 -- 172.21.15.18:0/3176091805 >> 172.21.15.18:0/3176091805 conn(0x7fd8640fd940 msgr2=0x7fd8640ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 -- 172.21.15.18:0/3176091805 shutdown_connections 2024-04-16T17:09:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.420+0000 7fd863e00640 1 -- 172.21.15.18:0/3176091805 wait complete. 2024-04-16T17:09:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.423+0000 7fd863e00640 1 Processor -- start 2024-04-16T17:09:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.423+0000 7fd863e00640 1 -- start start 2024-04-16T17:09:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd863e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd864102080 0x7fd864113050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd863e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd864107150 0x7fd864110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd863e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd864110680 0x7fd864110ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd863e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641144e0 con 0x7fd864102080 2024-04-16T17:09:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd863e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd864114360 con 0x7fd864110680 2024-04-16T17:09:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd863e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd864114660 con 0x7fd864107150 2024-04-16T17:09:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd862a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd864102080 0x7fd864113050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd862a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd864102080 0x7fd864113050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37100/0 (socket says 172.21.15.18:37100) 2024-04-16T17:09:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd862a00640 1 -- 172.21.15.18:0/700876615 learned_addr learned my addr 172.21.15.18:0/700876615 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.424+0000 7fd862000640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd864107150 0x7fd864110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.425+0000 7fd863400640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd864110680 0x7fd864110ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.425+0000 7fd862a00640 1 -- 172.21.15.18:0/700876615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd864107150 msgr2=0x7fd864110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.425+0000 7fd862a00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd864107150 0x7fd864110140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.425+0000 7fd862a00640 1 -- 172.21.15.18:0/700876615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd864110680 msgr2=0x7fd864110ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.425+0000 7fd862a00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd864110680 0x7fd864110ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.425+0000 7fd862a00640 1 -- 172.21.15.18:0/700876615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8641ab460 con 0x7fd864102080 2024-04-16T17:09:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.426+0000 7fd862a00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd864102080 0x7fd864113050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd85c02c710 tx=0x7fd85c035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.428+0000 7fd85b400640 1 -- 172.21.15.18:0/700876615 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c039040 con 0x7fd864102080 2024-04-16T17:09:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.428+0000 7fd85b400640 1 -- 172.21.15.18:0/700876615 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c044070 con 0x7fd864102080 2024-04-16T17:09:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.428+0000 7fd85b400640 1 -- 172.21.15.18:0/700876615 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c004540 con 0x7fd864102080 2024-04-16T17:09:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.428+0000 7fd863e00640 1 -- 172.21.15.18:0/700876615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8641ab6f0 con 0x7fd864102080 2024-04-16T17:09:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.428+0000 7fd863e00640 1 -- 172.21.15.18:0/700876615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8641abb90 con 0x7fd864102080 2024-04-16T17:09:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.432+0000 7fd86ca1a740 1 Processor -- start 2024-04-16T17:09:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.432+0000 7fd86ca1a740 1 -- start start 2024-04-16T17:09:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.433+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad333540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.433+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad333a80 0x55d3ad333f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.433+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33d000 0x55d3ad33f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.433+0000 7fd86ca1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d3ad298780 con 0x55d3ad3347f0 2024-04-16T17:09:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.433+0000 7fd86ca1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d3ad23ef40 con 0x55d3ad33d000 2024-04-16T17:09:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.433+0000 7fd86ca1a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d3ad294740 con 0x55d3ad333a80 2024-04-16T17:09:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.433+0000 7fd862000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33d000 0x55d3ad33f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.434+0000 7fd862000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33d000 0x55d3ad33f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55598/0 (socket says 172.21.15.18:55598) 2024-04-16T17:09:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.434+0000 7fd862000640 1 -- 172.21.15.18:0/2276101398 learned_addr learned my addr 172.21.15.18:0/2276101398 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.434+0000 7fd862000640 1 -- 172.21.15.18:0/2276101398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad333a80 msgr2=0x55d3ad333f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:09:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.434+0000 7fd862000640 1 --2- 172.21.15.18:0/2276101398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad333a80 0x55d3ad333f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.434+0000 7fd862000640 1 -- 172.21.15.18:0/2276101398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 msgr2=0x55d3ad333540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.434+0000 7fd862000640 1 --2- 172.21.15.18:0/2276101398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad333540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.434+0000 7fd862000640 1 -- 172.21.15.18:0/2276101398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3acfab210 con 0x55d3ad33d000 2024-04-16T17:09:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.435+0000 7fd862000640 1 --2- 172.21.15.18:0/2276101398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33d000 0x55d3ad33f3f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd8500099d0 tx=0x7fd850009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997ee50ccde645c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.435+0000 7fd84fe00640 1 -- 172.21.15.18:0/2276101398 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850017070 con 0x55d3ad33d000 2024-04-16T17:09:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.435+0000 7fd84fe00640 1 -- 172.21.15.18:0/2276101398 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd850013470 con 0x55d3ad33d000 2024-04-16T17:09:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.435+0000 7fd84fe00640 1 -- 172.21.15.18:0/2276101398 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850012630 con 0x55d3ad33d000 2024-04-16T17:09:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.436+0000 7fd85b400640 1 -- 172.21.15.18:0/700876615 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd85c0046e0 con 0x7fd864102080 2024-04-16T17:09:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.436+0000 7fd86ca1a740 1 -- 172.21.15.18:0/2276101398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33d000 msgr2=0x55d3ad33f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.436+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/2276101398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33d000 0x55d3ad33f3f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd8500099d0 tx=0x7fd850009e90 comp rx=0 tx=0).stop 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.437+0000 7fd86ca1a740 1 -- 172.21.15.18:0/2276101398 shutdown_connections 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.437+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/2276101398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33d000 0x55d3ad33f3f0 secure :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd8500099d0 tx=0x7fd850009e90 comp rx=0 tx=0).stop 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.437+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/2276101398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad333a80 0x55d3ad333f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.437+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/2276101398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad333540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.437+0000 7fd86ca1a740 1 -- 172.21.15.18:0/2276101398 >> 172.21.15.18:0/2276101398 conn(0x55d3ad2ce5d0 msgr2=0x55d3ad32e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.437+0000 7fd86ca1a740 1 -- 172.21.15.18:0/2276101398 shutdown_connections 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.437+0000 7fd86ca1a740 1 -- 172.21.15.18:0/2276101398 wait complete. 2024-04-16T17:09:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.438+0000 7fd86ca1a740 1 -- start start 2024-04-16T17:09:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.439+0000 7fd86ca1a740 1 Processor -- start 2024-04-16T17:09:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.439+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad333a80 0x55d3ad339f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.439+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3347f0 0x55d3ad33a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.439+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad349270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd86ca1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d3ad33fd30 con 0x55d3ad333a80 2024-04-16T17:09:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd86ca1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d3ad252390 con 0x55d3ad33a980 2024-04-16T17:09:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd86ca1a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d3ad33feb0 con 0x55d3ad3347f0 2024-04-16T17:09:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd86ca1a740 10 client.0 _ll_register_callbacks cb 0x55d3ad23bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd85b400640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd8480a3c90 0x7fd8480a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd862000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad349270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd862000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad349270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55610/0 (socket says 172.21.15.18:55610) 2024-04-16T17:09:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd862000640 1 -- 172.21.15.18:0/3973436874 learned_addr learned my addr 172.21.15.18:0/3973436874 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd862a00640 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3347f0 0x55d3ad33a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.440+0000 7fd863400640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd8480a3c90 0x7fd8480a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.441+0000 7fd862000640 1 -- 172.21.15.18:0/3973436874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3347f0 msgr2=0x55d3ad33a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.441+0000 7fd862000640 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3347f0 0x55d3ad33a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.441+0000 7fd862000640 1 -- 172.21.15.18:0/3973436874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad333a80 msgr2=0x55d3ad339f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.441+0000 7fd862000640 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad333a80 0x55d3ad339f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.441+0000 7fd862000640 1 -- 172.21.15.18:0/3973436874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3ad077620 con 0x55d3ad33a980 2024-04-16T17:09:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.442+0000 7fd85b400640 1 -- 172.21.15.18:0/700876615 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd85c0f2e30 con 0x7fd864102080 2024-04-16T17:09:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.442+0000 7fd862a00640 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3347f0 0x55d3ad33a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.443+0000 7fd862000640 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad349270 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55d3ad334520 tx=0x7fd8500047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ceb45c740b7a17c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.443+0000 7fd84cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.444+0000 7fd863400640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd8480a3c90 0x7fd8480a6150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd854006e10 tx=0x7fd854006d60 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.444+0000 7fd84cc00640 1 -- 172.21.15.18:0/3973436874 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85002b060 con 0x55d3ad33a980 2024-04-16T17:09:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.444+0000 7fd84cc00640 1 -- 172.21.15.18:0/3973436874 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd85001c650 con 0x55d3ad33a980 2024-04-16T17:09:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.444+0000 7fd86ca1a740 1 -- 172.21.15.18:0/3973436874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd850013470 con 0x55d3ad33a980 2024-04-16T17:09:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.445+0000 7fd84cc00640 1 -- 172.21.15.18:0/3973436874 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd850017040 con 0x55d3ad33a980 2024-04-16T17:09:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.445+0000 7fd86ca1a740 10 client.26963 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.445+0000 7fd86ca1a740 1 -- 172.21.15.18:0/3973436874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d3ad34c3e0 con 0x55d3ad33a980 2024-04-16T17:09:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.446+0000 7fd84cc00640 1 -- 172.21.15.18:0/3973436874 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd850013470 con 0x55d3ad33a980 2024-04-16T17:09:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.446+0000 7fd84cc00640 1 -- 172.21.15.18:0/3973436874 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd850026600 con 0x55d3ad33a980 2024-04-16T17:09:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.446+0000 7fd86ca1a740 10 client.26963 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.446+0000 7fd86ca1a740 10 client.26963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.446+0000 7fd86ca1a740 20 client.26963 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.446+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3ad34d730 0x55d3ad34fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.446+0000 7fd86ca1a740 4 client.26963 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.446+0000 7fd86ca1a740 1 -- 172.21.15.18:0/3973436874 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3ad2a61a0 con 0x55d3ad34d730 2024-04-16T17:09:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.447+0000 7fd862a00640 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3ad34d730 0x55d3ad34fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.449+0000 7fd862a00640 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3ad34d730 0x55d3ad34fb20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ab7a6d8f38140a server_cookie=40c3085305a42a05 in_seq=0 out_seq=0 2024-04-16T17:09:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.449+0000 7fd84cc00640 10 client.26963 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.449+0000 7fd84cc00640 1 -- 172.21.15.18:0/3973436874 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3ad2a61a0 con 0x55d3ad34d730 2024-04-16T17:09:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.449+0000 7fd84cc00640 10 client.26963 handle_command_reply: tid=0 2024-04-16T17:09:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 client.26963 shutdown 2024-04-16T17:09:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 -- 172.21.15.18:0/3973436874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 msgr2=0x55d3ad349270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad349270 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55d3ad334520 tx=0x7fd8500047d0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 -- 172.21.15.18:0/3973436874 shutdown_connections 2024-04-16T17:09:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3ad34d730 0x55d3ad34fb20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad349270 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3347f0 0x55d3ad33a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/3973436874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad333a80 0x55d3ad339f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 -- 172.21.15.18:0/3973436874 >> 172.21.15.18:0/3973436874 conn(0x55d3ad2ce5d0 msgr2=0x55d3ad3304c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd84cc00640 0 client.26963 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 -- 172.21.15.18:0/3973436874 shutdown_connections 2024-04-16T17:09:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.450+0000 7fd86ca1a740 1 -- 172.21.15.18:0/3973436874 wait complete. 2024-04-16T17:09:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.451+0000 7fd86ca1a740 20 client.26963 trim_cache size 0 max 16384 2024-04-16T17:09:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.486+0000 7fd86ca1a740 1 Processor -- start 2024-04-16T17:09:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.486+0000 7fd86ca1a740 1 -- start start 2024-04-16T17:09:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.486+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad333a80 0x55d3ad355180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.486+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad3556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad33a980 0x55d3ad348cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd862a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad333a80 0x55d3ad355180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd862a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad333a80 0x55d3ad355180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55618/0 (socket says 172.21.15.18:55618) 2024-04-16T17:09:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd862a00640 1 -- 172.21.15.18:0/1765062200 learned_addr learned my addr 172.21.15.18:0/1765062200 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1765062200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd844012820 con 0x55d3ad3347f0 2024-04-16T17:09:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1765062200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd8440100f0 con 0x55d3ad333a80 2024-04-16T17:09:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1765062200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd84400df40 con 0x55d3ad33a980 2024-04-16T17:09:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd863400640 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad33a980 0x55d3ad348cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd862a00640 1 -- 172.21.15.18:0/1765062200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad33a980 msgr2=0x55d3ad348cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd862a00640 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad33a980 0x55d3ad348cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.487+0000 7fd862a00640 1 -- 172.21.15.18:0/1765062200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 msgr2=0x55d3ad3556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.488+0000 7fd862000640 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad3556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.488+0000 7fd862a00640 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad3556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.488+0000 7fd862a00640 1 -- 172.21.15.18:0/1765062200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3ad332d50 con 0x55d3ad333a80 2024-04-16T17:09:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.488+0000 7fd862000640 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad3556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.488+0000 7fd862a00640 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad333a80 0x55d3ad355180 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0bea70 tx=0x7fd85c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f84ecf781e5d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.489+0000 7fd84f400640 1 -- 172.21.15.18:0/1765062200 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0c7750 con 0x55d3ad333a80 2024-04-16T17:09:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.489+0000 7fd84f400640 1 -- 172.21.15.18:0/1765062200 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c009870 con 0x55d3ad333a80 2024-04-16T17:09:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.489+0000 7fd84f400640 1 -- 172.21.15.18:0/1765062200 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0bed40 con 0x55d3ad333a80 2024-04-16T17:09:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.489+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1765062200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad333a80 msgr2=0x55d3ad355180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.489+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad333a80 0x55d3ad355180 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0bea70 tx=0x7fd85c004040 comp rx=0 tx=0).stop 2024-04-16T17:09:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.490+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1765062200 shutdown_connections 2024-04-16T17:09:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.490+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad33a980 0x55d3ad348cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.490+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad3556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.490+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1765062200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad333a80 0x55d3ad355180 secure :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0bea70 tx=0x7fd85c004040 comp rx=0 tx=0).stop 2024-04-16T17:09:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.490+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1765062200 >> 172.21.15.18:0/1765062200 conn(0x55d3ad2ce5d0 msgr2=0x55d3ad32e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.495+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1765062200 shutdown_connections 2024-04-16T17:09:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.495+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1765062200 wait complete. 2024-04-16T17:09:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.495+0000 7fd86ca1a740 1 -- start start 2024-04-16T17:09:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.496+0000 7fd86ca1a740 1 Processor -- start 2024-04-16T17:09:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.496+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad340b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.496+0000 7fd86ca1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad3410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd862000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad3410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd862000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad3410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55620/0 (socket says 172.21.15.18:55620) 2024-04-16T17:09:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd862000640 1 -- 172.21.15.18:0/1700038787 learned_addr learned my addr 172.21.15.18:0/1700038787 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3415f0 0x55d3ad3b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd84400e5b0 con 0x55d3ad3347f0 2024-04-16T17:09:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd84400fa20 con 0x55d3ad33a980 2024-04-16T17:09:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd84400ec80 con 0x55d3ad3415f0 2024-04-16T17:09:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd86ca1a740 10 client.0 _ll_register_callbacks cb 0x7fd850009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd863400640 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3415f0 0x55d3ad3b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd862000640 1 -- 172.21.15.18:0/1700038787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3415f0 msgr2=0x55d3ad3b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd862000640 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3415f0 0x55d3ad3b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.498+0000 7fd862000640 1 -- 172.21.15.18:0/1700038787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 msgr2=0x55d3ad340b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.497+0000 7fd862a00640 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad340b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.498+0000 7fd862000640 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad340b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.498+0000 7fd862000640 1 -- 172.21.15.18:0/1700038787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3ad3b2f30 con 0x55d3ad33a980 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.498+0000 7fd862a00640 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad340b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.498+0000 7fd862000640 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad3410b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd850026a00 tx=0x7fd850026940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa3fd7f181f8d323 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.498+0000 7fd84cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.499+0000 7fd84cc00640 1 -- 172.21.15.18:0/1700038787 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85002b070 con 0x55d3ad33a980 2024-04-16T17:09:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.499+0000 7fd84cc00640 1 -- 172.21.15.18:0/1700038787 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fd850013ce0 con 0x55d3ad33a980 2024-04-16T17:09:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.499+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd85c009870 con 0x55d3ad33a980 2024-04-16T17:09:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.499+0000 7fd84cc00640 1 -- 172.21.15.18:0/1700038787 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd850035080 con 0x55d3ad33a980 2024-04-16T17:09:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.499+0000 7fd86ca1a740 10 client.26969 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d3ad3b4000 con 0x55d3ad33a980 2024-04-16T17:09:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd84cc00640 1 -- 172.21.15.18:0/1700038787 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c009870 con 0x55d3ad33a980 2024-04-16T17:09:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd84cc00640 1 -- 172.21.15.18:0/1700038787 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd850006230 con 0x55d3ad33a980 2024-04-16T17:09:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd86ca1a740 10 client.26969 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd86ca1a740 10 client.26969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd86ca1a740 20 client.26969 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3ad3b4750 0x55d3ad3b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd86ca1a740 4 client.26969 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:09:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.500+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3ad2a0010 con 0x55d3ad3b4750 2024-04-16T17:09:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.501+0000 7fd862a00640 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3ad3b4750 0x55d3ad3b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.501+0000 7fd84cc00640 10 client.26969 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.501+0000 7fd862a00640 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3ad3b4750 0x55d3ad3b6b70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3f3070de0e355e server_cookie=1cc4c218803303be in_seq=0 out_seq=0 2024-04-16T17:09:50.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 183348 3.94 MB/sec execute 123 sec latency 254.064 ms 2024-04-16T17:09:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.649+0000 7fd84cc00640 1 -- 172.21.15.18:0/1700038787 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3ad2a0010 con 0x55d3ad3b4750 2024-04-16T17:09:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.649+0000 7fd84cc00640 10 client.26969 handle_command_reply: tid=0 2024-04-16T17:09:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.649+0000 7fd86ca1a740 1 client.26969 shutdown 2024-04-16T17:09:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.649+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 msgr2=0x55d3ad3410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.649+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad3410b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd850026a00 tx=0x7fd850026940 comp rx=0 tx=0).stop 2024-04-16T17:09:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.650+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 shutdown_connections 2024-04-16T17:09:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.650+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3ad3b4750 0x55d3ad3b6b70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.650+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3ad3415f0 0x55d3ad3b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.650+0000 7fd84cc00640 0 client.26969 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.650+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3ad33a980 0x55d3ad3410b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.650+0000 7fd86ca1a740 1 --2- 172.21.15.18:0/1700038787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3ad3347f0 0x55d3ad340b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.650+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 >> 172.21.15.18:0/1700038787 conn(0x55d3ad2ce5d0 msgr2=0x55d3ad32fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.651+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 shutdown_connections 2024-04-16T17:09:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.651+0000 7fd86ca1a740 1 -- 172.21.15.18:0/1700038787 wait complete. 2024-04-16T17:09:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.651+0000 7fd86ca1a740 20 client.26969 trim_cache size 0 max 16384 2024-04-16T17:09:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:09:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:09:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:09:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 -- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd8480a3c90 msgr2=0x7fd8480a6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd8480a3c90 0x7fd8480a6150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd854006e10 tx=0x7fd854006d60 comp rx=0 tx=0).stop 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 -- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd864102080 msgr2=0x7fd864113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd864102080 0x7fd864113050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd85c02c710 tx=0x7fd85c035d10 comp rx=0 tx=0).stop 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 -- 172.21.15.18:0/700876615 shutdown_connections 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd8480a3c90 0x7fd8480a6150 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd864110680 0x7fd864110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd864107150 0x7fd864110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 --2- 172.21.15.18:0/700876615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd864102080 0x7fd864113050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.654+0000 7fd863e00640 1 -- 172.21.15.18:0/700876615 >> 172.21.15.18:0/700876615 conn(0x7fd8640fd940 msgr2=0x7fd8640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.655+0000 7fd863e00640 1 -- 172.21.15.18:0/700876615 shutdown_connections 2024-04-16T17:09:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:09:50.655+0000 7fd863e00640 1 -- 172.21.15.18:0/700876615 wait complete. 2024-04-16T17:09:51.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 184700 4.01 MB/sec execute 124 sec latency 34.081 ms 2024-04-16T17:09:51.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:51 smithi022 ceph-mon[36023]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-16T17:09:52.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:51 smithi043 ceph-mon[36901]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-16T17:09:52.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 185240 3.98 MB/sec execute 125 sec latency 19.590 ms 2024-04-16T17:09:53.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 185663 3.96 MB/sec execute 126 sec latency 64.102 ms 2024-04-16T17:09:53.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:53 smithi022 ceph-mon[36023]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1019 KiB/s wr, 78 op/s 2024-04-16T17:09:54.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:53 smithi043 ceph-mon[36901]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1019 KiB/s wr, 78 op/s 2024-04-16T17:09:54.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 186047 3.95 MB/sec execute 127 sec latency 87.696 ms 2024-04-16T17:09:54.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:09:54.726 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:09:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 -- 172.21.15.18:0/2115079802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ac101230 msgr2=0x7f73ac101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 --2- 172.21.15.18:0/2115079802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ac101230 0x7f73ac101610 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009930 tx=0x7f73a002bc30 comp rx=0 tx=0).stop 2024-04-16T17:09:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 -- 172.21.15.18:0/2115079802 shutdown_connections 2024-04-16T17:09:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 --2- 172.21.15.18:0/2115079802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73ac1034b0 0x7f73ac10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 --2- 172.21.15.18:0/2115079802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73ac102b10 0x7f73ac102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 --2- 172.21.15.18:0/2115079802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ac101230 0x7f73ac101610 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 -- 172.21.15.18:0/2115079802 >> 172.21.15.18:0/2115079802 conn(0x7f73ac078020 msgr2=0x7f73ac078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:54.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 -- 172.21.15.18:0/2115079802 shutdown_connections 2024-04-16T17:09:54.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.921+0000 7f73b4200640 1 -- 172.21.15.18:0/2115079802 wait complete. 2024-04-16T17:09:54.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.922+0000 7f73b4200640 1 Processor -- start 2024-04-16T17:09:54.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.922+0000 7f73b4200640 1 -- start start 2024-04-16T17:09:54.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b4200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73ac101230 0x7f73ac0fdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b4200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73ac102b10 0x7f73ac0fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b4200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ac1034b0 0x7f73ac0fe920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b4200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1113d0 con 0x7f73ac102b10 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b4200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac111250 con 0x7f73ac101230 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b4200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac111550 con 0x7f73ac1034b0 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73ac101230 0x7f73ac0fdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73ac101230 0x7f73ac0fdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55636/0 (socket says 172.21.15.18:55636) 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b1800640 1 -- 172.21.15.18:0/1556464260 learned_addr learned my addr 172.21.15.18:0/1556464260 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b0e00640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73ac102b10 0x7f73ac0fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b2200640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ac1034b0 0x7f73ac0fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b1800640 1 -- 172.21.15.18:0/1556464260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ac1034b0 msgr2=0x7f73ac0fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b1800640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ac1034b0 0x7f73ac0fe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b1800640 1 -- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73ac102b10 msgr2=0x7f73ac0fe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b1800640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73ac102b10 0x7f73ac0fe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.923+0000 7f73b1800640 1 -- 172.21.15.18:0/1556464260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac1a6ec0 con 0x7f73ac101230 2024-04-16T17:09:54.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.924+0000 7f73b1800640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73ac101230 0x7f73ac0fdea0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009900 tx=0x7f73a0004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:54.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.924+0000 7f73aa000640 1 -- 172.21.15.18:0/1556464260 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0043020 con 0x7f73ac101230 2024-04-16T17:09:54.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.924+0000 7f73aa000640 1 -- 172.21.15.18:0/1556464260 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a00054e0 con 0x7f73ac101230 2024-04-16T17:09:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.924+0000 7f73aa000640 1 -- 172.21.15.18:0/1556464260 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0003bb0 con 0x7f73ac101230 2024-04-16T17:09:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.924+0000 7f73b4200640 1 -- 172.21.15.18:0/1556464260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac1a7150 con 0x7f73ac101230 2024-04-16T17:09:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.924+0000 7f73b4200640 1 -- 172.21.15.18:0/1556464260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1a76a0 con 0x7f73ac101230 2024-04-16T17:09:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.928+0000 7f73b6fd0740 1 Processor -- start 2024-04-16T17:09:54.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.929+0000 7f73b6fd0740 1 -- start start 2024-04-16T17:09:54.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.929+0000 7f73aa000640 1 -- 172.21.15.18:0/1556464260 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73a0046020 con 0x7f73ac101230 2024-04-16T17:09:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.929+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.929+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.929+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a96b0 0x55e92a5abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.929+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e92a506120 con 0x55e92a5a0ad0 2024-04-16T17:09:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.929+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e92a4ac340 con 0x55e92a5a01b0 2024-04-16T17:09:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.929+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e92a5020e0 con 0x55e92a5a96b0 2024-04-16T17:09:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a96b0 0x55e92a5abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55652/0 (socket says 172.21.15.18:55652) 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a96b0 0x55e92a5abaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50202/0 (socket says 172.21.15.18:50202) 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 -- 172.21.15.18:0/3677554491 learned_addr learned my addr 172.21.15.18:0/3677554491 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b1800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 -- 172.21.15.18:0/3677554491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a96b0 msgr2=0x55e92a5abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 --2- 172.21.15.18:0/3677554491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a96b0 0x55e92a5abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 -- 172.21.15.18:0/3677554491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a0ad0 msgr2=0x55e92a5a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 --2- 172.21.15.18:0/3677554491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 -- 172.21.15.18:0/3677554491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92a2e27d0 con 0x55e92a5a01b0 2024-04-16T17:09:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.930+0000 7f73b2200640 1 --2- 172.21.15.18:0/3677554491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a0590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009f40 tx=0x7f73a4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed6977a498b988b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.931+0000 7f739ea00640 1 -- 172.21.15.18:0/3677554491 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4003b70 con 0x55e92a5a01b0 2024-04-16T17:09:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.931+0000 7f739ea00640 1 -- 172.21.15.18:0/3677554491 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4003d10 con 0x55e92a5a01b0 2024-04-16T17:09:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.931+0000 7f739ea00640 1 -- 172.21.15.18:0/3677554491 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4011ae0 con 0x55e92a5a01b0 2024-04-16T17:09:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.931+0000 7f73aa000640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f73940a3d20 0x7f73940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.931+0000 7f73aa000640 1 -- 172.21.15.18:0/1556464260 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f73a00f2c00 con 0x7f73ac101230 2024-04-16T17:09:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b1800640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f73940a3d20 0x7f73940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3677554491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 msgr2=0x55e92a5a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3677554491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a0590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009f40 tx=0x7f73a4010860 comp rx=0 tx=0).stop 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3677554491 shutdown_connections 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3677554491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a96b0 0x55e92a5abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3677554491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3677554491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a0590 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3677554491 >> 172.21.15.18:0/3677554491 conn(0x55e92a59a8e0 msgr2=0x55e92a59acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3677554491 shutdown_connections 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3677554491 wait complete. 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.932+0000 7f73b1800640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f73940a3d20 0x7f73940a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f73a00f0160 tx=0x7f73a00df040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.933+0000 7f73b6fd0740 1 -- start start 2024-04-16T17:09:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.933+0000 7f73b6fd0740 1 Processor -- start 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a01b0 0x55e92a5a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a96b0 0x55e92a5a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e92a5ac4a0 con 0x55e92a5a01b0 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e92a4bf710 con 0x55e92a5a96b0 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e92a5ac620 con 0x55e92a5a0ad0 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b6fd0740 10 client.0 _ll_register_callbacks cb 0x55e92a4baaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50206/0 (socket says 172.21.15.18:50206) 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b0e00640 1 -- 172.21.15.18:0/3806649203 learned_addr learned my addr 172.21.15.18:0/3806649203 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.934+0000 7f73b2200640 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a96b0 0x55e92a5a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f73b2200640 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a01b0 0x55e92a5a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f73b0e00640 1 -- 172.21.15.18:0/3806649203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a96b0 msgr2=0x55e92a5a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f73b0e00640 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a96b0 0x55e92a5a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f73b0e00640 1 -- 172.21.15.18:0/3806649203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a01b0 msgr2=0x55e92a5a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f73b0e00640 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a01b0 0x55e92a5a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f73b0e00640 1 -- 172.21.15.18:0/3806649203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92a2183f0 con 0x55e92a5a0ad0 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f73b2200640 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a01b0 0x55e92a5a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f73b0e00640 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a6d30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7398012700 tx=0x7f7398012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2381313b5fb4222 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f7393400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.935+0000 7f7393400640 1 -- 172.21.15.18:0/3806649203 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398011c70 con 0x55e92a5a0ad0 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.936+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3806649203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a4003d10 con 0x55e92a5a0ad0 2024-04-16T17:09:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.936+0000 7f7393400640 1 -- 172.21.15.18:0/3806649203 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f7398011e10 con 0x55e92a5a0ad0 2024-04-16T17:09:54.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.936+0000 7f7393400640 1 -- 172.21.15.18:0/3806649203 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f739800f460 con 0x55e92a5a0ad0 2024-04-16T17:09:54.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.936+0000 7f73b6fd0740 10 client.26947 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:54.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.936+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3806649203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e92a1d80f0 con 0x55e92a5a0ad0 2024-04-16T17:09:54.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.936+0000 7f7393400640 1 -- 172.21.15.18:0/3806649203 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4003d10 con 0x55e92a5a0ad0 2024-04-16T17:09:54.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.936+0000 7f7393400640 1 -- 172.21.15.18:0/3806649203 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f739801c690 con 0x55e92a5a0ad0 2024-04-16T17:09:54.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.937+0000 7f73b6fd0740 10 client.26947 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.937+0000 7f73b6fd0740 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.937+0000 7f73b6fd0740 20 client.26947 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.937+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e92a5b9b40 0x55e92a5bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.937+0000 7f73b6fd0740 4 client.26947 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.937+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3806649203 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e92a513b40 con 0x55e92a5b9b40 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.937+0000 7f73b2200640 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e92a5b9b40 0x55e92a5bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.938+0000 7f73b2200640 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e92a5b9b40 0x55e92a5bbfc0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b66e357f24aabf server_cookie=9cc7235b38aa774f in_seq=0 out_seq=0 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.938+0000 7f7393400640 10 client.26947 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f7393400640 1 -- 172.21.15.18:0/3806649203 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e92a513b40 con 0x55e92a5b9b40 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f7393400640 10 client.26947 handle_command_reply: tid=0 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 client.26947 shutdown 2024-04-16T17:09:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3806649203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 msgr2=0x55e92a5a6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a6d30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7398012700 tx=0x7f7398012bc0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3806649203 shutdown_connections 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e92a5b9b40 0x55e92a5bbfc0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a96b0 0x55e92a5a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a6d30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/3806649203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a01b0 0x55e92a5a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3806649203 >> 172.21.15.18:0/3806649203 conn(0x55e92a59a8e0 msgr2=0x55e92a59c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f7393400640 0 client.26947 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3806649203 shutdown_connections 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 1 -- 172.21.15.18:0/3806649203 wait complete. 2024-04-16T17:09:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.939+0000 7f73b6fd0740 20 client.26947 trim_cache size 0 max 16384 2024-04-16T17:09:54.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.967+0000 7f73b6fd0740 1 Processor -- start 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b6fd0740 1 -- start start 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 0x55e92a5c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b4070 con 0x55e92a5a96b0 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b1940 con 0x55e92a5a01b0 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1af790 con 0x55e92a5a0ad0 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55656/0 (socket says 172.21.15.18:55656) 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b2200640 1 -- 172.21.15.18:0/1060556623 learned_addr learned my addr 172.21.15.18:0/1060556623 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:54.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.968+0000 7f73b2200640 1 --2- 172.21.15.18:0/1060556623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 0x55e92a5c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f73b0e00640 1 -- 172.21.15.18:0/1060556623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 msgr2=0x55e92a5a8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:54.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f73b0e00640 1 --2- 172.21.15.18:0/1060556623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f73b0e00640 1 -- 172.21.15.18:0/1060556623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 msgr2=0x55e92a5c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:54.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f73b0e00640 1 --2- 172.21.15.18:0/1060556623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 0x55e92a5c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f73b0e00640 1 -- 172.21.15.18:0/1060556623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92a5abe10 con 0x55e92a5a0ad0 2024-04-16T17:09:54.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f73b0e00640 1 --2- 172.21.15.18:0/1060556623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5b5360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f739801cb80 tx=0x7f7398009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7a3bcc33eef1283 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f739e000640 1 -- 172.21.15.18:0/1060556623 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398011c70 con 0x55e92a5a0ad0 2024-04-16T17:09:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f739e000640 1 -- 172.21.15.18:0/1060556623 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739801c900 con 0x55e92a5a0ad0 2024-04-16T17:09:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.969+0000 7f739e000640 1 -- 172.21.15.18:0/1060556623 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398004680 con 0x55e92a5a0ad0 2024-04-16T17:09:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 -- 172.21.15.18:0/1060556623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 msgr2=0x55e92a5b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/1060556623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5b5360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f739801cb80 tx=0x7f7398009fd0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 -- 172.21.15.18:0/1060556623 shutdown_connections 2024-04-16T17:09:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/1060556623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 0x55e92a5c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/1060556623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5b5360 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/1060556623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 -- 172.21.15.18:0/1060556623 >> 172.21.15.18:0/1060556623 conn(0x55e92a59a8e0 msgr2=0x55e92a5a5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 -- 172.21.15.18:0/1060556623 shutdown_connections 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.970+0000 7f73b6fd0740 1 -- 172.21.15.18:0/1060556623 wait complete. 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 1 -- start start 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 1 Processor -- start 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 0x55e92a5a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1afe00 con 0x55e92a5a96b0 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b1270 con 0x55e92a5a01b0 2024-04-16T17:09:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b04d0 con 0x55e92a5a0ad0 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.971+0000 7f73b6fd0740 10 client.0 _ll_register_callbacks cb 0x7f73980097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b2200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 0x55e92a5a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50214/0 (socket says 172.21.15.18:50214) 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b0e00640 1 -- 172.21.15.18:0/2683317074 learned_addr learned my addr 172.21.15.18:0/2683317074 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b2200640 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b0e00640 1 -- 172.21.15.18:0/2683317074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 msgr2=0x55e92a5b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b0e00640 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b0e00640 1 -- 172.21.15.18:0/2683317074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 msgr2=0x55e92a5a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b0e00640 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 0x55e92a5a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.972+0000 7f73b0e00640 1 -- 172.21.15.18:0/2683317074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92a5a3ae0 con 0x55e92a5a0ad0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.973+0000 7f73b0e00640 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a3060 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f73980092e0 tx=0x7f73980055a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe94746d8ebc829 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.973+0000 7f7393400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.973+0000 7f7393400640 1 -- 172.21.15.18:0/2683317074 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398005b90 con 0x55e92a5a0ad0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.973+0000 7f7393400640 1 -- 172.21.15.18:0/2683317074 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f7398009d20 con 0x55e92a5a0ad0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.973+0000 7f73b6fd0740 1 -- 172.21.15.18:0/2683317074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739801c900 con 0x55e92a5a0ad0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.973+0000 7f7393400640 1 -- 172.21.15.18:0/2683317074 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f739801c630 con 0x55e92a5a0ad0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.973+0000 7f73b6fd0740 10 client.26953 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.973+0000 7f73b6fd0740 1 -- 172.21.15.18:0/2683317074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e92a61fe20 con 0x55e92a5a0ad0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f7393400640 1 -- 172.21.15.18:0/2683317074 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739800fae0 con 0x55e92a5a0ad0 2024-04-16T17:09:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f7393400640 1 -- 172.21.15.18:0/2683317074 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f73980107d0 con 0x55e92a5a0ad0 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f73b6fd0740 10 client.26953 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f73b6fd0740 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f73b6fd0740 20 client.26953 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e92a620500 0x55e92a622920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f73b6fd0740 4 client.26953 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f73b6fd0740 1 -- 172.21.15.18:0/2683317074 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f739801c900 con 0x55e92a620500 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.974+0000 7f73b2200640 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e92a620500 0x55e92a622920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b2200640 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e92a620500 0x55e92a622920 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e53e420b218a5b server_cookie=ff022787d0df255 in_seq=0 out_seq=0 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f7393400640 10 client.26953 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f7393400640 1 -- 172.21.15.18:0/2683317074 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f739801c900 con 0x55e92a620500 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f7393400640 10 client.26953 handle_command_reply: tid=0 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 client.26953 shutdown 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 -- 172.21.15.18:0/2683317074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 msgr2=0x55e92a5a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a3060 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f73980092e0 tx=0x7f73980055a0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 -- 172.21.15.18:0/2683317074 shutdown_connections 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e92a620500 0x55e92a622920 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e92a5a96b0 0x55e92a5a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e92a5a0ad0 0x55e92a5a3060 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 --2- 172.21.15.18:0/2683317074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e92a5a01b0 0x55e92a5b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 -- 172.21.15.18:0/2683317074 >> 172.21.15.18:0/2683317074 conn(0x55e92a59a8e0 msgr2=0x55e92a59bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f7393400640 0 client.26953 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 -- 172.21.15.18:0/2683317074 shutdown_connections 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.975+0000 7f73b6fd0740 1 -- 172.21.15.18:0/2683317074 wait complete. 2024-04-16T17:09:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.976+0000 7f73b6fd0740 20 client.26953 trim_cache size 0 max 16384 2024-04-16T17:09:54.981 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:09:54.981 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:09:54.981 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:09:54.981 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.976+0000 7f73b4200640 1 -- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f73940a3d20 msgr2=0x7f73940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.976+0000 7f73b4200640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f73940a3d20 0x7f73940a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f73a00f0160 tx=0x7f73a00df040 comp rx=0 tx=0).stop 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.976+0000 7f73b4200640 1 -- 172.21.15.18:0/1556464260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73ac101230 msgr2=0x7f73ac0fdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.976+0000 7f73b4200640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73ac101230 0x7f73ac0fdea0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009900 tx=0x7f73a0004360 comp rx=0 tx=0).stop 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.977+0000 7f73b4200640 1 -- 172.21.15.18:0/1556464260 shutdown_connections 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.977+0000 7f73b4200640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f73940a3d20 0x7f73940a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.977+0000 7f73b4200640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ac1034b0 0x7f73ac0fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.977+0000 7f73b4200640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f73ac102b10 0x7f73ac0fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.977+0000 7f73b4200640 1 --2- 172.21.15.18:0/1556464260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73ac101230 0x7f73ac0fdea0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.977+0000 7f73b4200640 1 -- 172.21.15.18:0/1556464260 >> 172.21.15.18:0/1556464260 conn(0x7f73ac078020 msgr2=0x7f73ac100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.977+0000 7f73b4200640 1 -- 172.21.15.18:0/1556464260 shutdown_connections 2024-04-16T17:09:54.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:54.977+0000 7f73b4200640 1 -- 172.21.15.18:0/1556464260 wait complete. 2024-04-16T17:09:54.998 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:09:54.998 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:09:55.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 -- 172.21.15.18:0/215670390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 msgr2=0x7fba38110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 --2- 172.21.15.18:0/215670390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 0x7fba38110c70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fba30009ca0 tx=0x7fba3002c190 comp rx=0 tx=0).stop 2024-04-16T17:09:55.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 -- 172.21.15.18:0/215670390 shutdown_connections 2024-04-16T17:09:55.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 --2- 172.21.15.18:0/215670390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 0x7fba38110c70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 --2- 172.21.15.18:0/215670390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba38074cf0 0x7fba3810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 --2- 172.21.15.18:0/215670390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba380743d0 0x7fba380747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 -- 172.21.15.18:0/215670390 >> 172.21.15.18:0/215670390 conn(0x7fba380fdad0 msgr2=0x7fba380ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:55.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 -- 172.21.15.18:0/215670390 shutdown_connections 2024-04-16T17:09:55.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.201+0000 7fba3cc00640 1 -- 172.21.15.18:0/215670390 wait complete. 2024-04-16T17:09:55.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.202+0000 7fba3cc00640 1 Processor -- start 2024-04-16T17:09:55.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba3cc00640 1 -- start start 2024-04-16T17:09:55.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba3cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba380743d0 0x7fba381a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba3cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba38074cf0 0x7fba381a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba3cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 0x7fba3819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba3cc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba38114580 con 0x7fba380743d0 2024-04-16T17:09:55.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba3cc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba38114400 con 0x7fba3810e860 2024-04-16T17:09:55.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba3cc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba38114700 con 0x7fba38074cf0 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba35e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba380743d0 0x7fba381a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba36800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 0x7fba3819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba36800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 0x7fba3819a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55680/0 (socket says 172.21.15.18:55680) 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba35e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba380743d0 0x7fba381a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37214/0 (socket says 172.21.15.18:37214) 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.203+0000 7fba36800640 1 -- 172.21.15.18:0/1505405983 learned_addr learned my addr 172.21.15.18:0/1505405983 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.204+0000 7fba35400640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba38074cf0 0x7fba381a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.204+0000 7fba36800640 1 -- 172.21.15.18:0/1505405983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba38074cf0 msgr2=0x7fba381a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.204+0000 7fba36800640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba38074cf0 0x7fba381a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.204+0000 7fba36800640 1 -- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba380743d0 msgr2=0x7fba381a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.204+0000 7fba36800640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba380743d0 0x7fba381a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.204+0000 7fba36800640 1 -- 172.21.15.18:0/1505405983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3819b0a0 con 0x7fba3810e860 2024-04-16T17:09:55.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.204+0000 7fba36800640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 0x7fba3819a920 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fba30037040 tx=0x7fba30035e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:55.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.204+0000 7fba2ea00640 1 -- 172.21.15.18:0/1505405983 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30039040 con 0x7fba3810e860 2024-04-16T17:09:55.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.205+0000 7fba2ea00640 1 -- 172.21.15.18:0/1505405983 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba30044070 con 0x7fba3810e860 2024-04-16T17:09:55.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.205+0000 7fba2ea00640 1 -- 172.21.15.18:0/1505405983 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300328d0 con 0x7fba3810e860 2024-04-16T17:09:55.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.205+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3819b330 con 0x7fba3810e860 2024-04-16T17:09:55.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.205+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba381a75e0 con 0x7fba3810e860 2024-04-16T17:09:55.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.206+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba38069300 con 0x7fba3810e860 2024-04-16T17:09:55.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.208+0000 7fba2ea00640 1 -- 172.21.15.18:0/1505405983 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba30003d00 con 0x7fba3810e860 2024-04-16T17:09:55.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.210+0000 7fba2ea00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba1c0a3de0 0x7fba1c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.211+0000 7fba2ea00640 1 -- 172.21.15.18:0/1505405983 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fba300f27e0 con 0x7fba3810e860 2024-04-16T17:09:55.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.211+0000 7fba35e00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba1c0a3de0 0x7fba1c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.212+0000 7fba35e00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba1c0a3de0 0x7fba1c0a62a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fba38110de0 tx=0x7fba28005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:55.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.214+0000 7fba2ea00640 1 -- 172.21.15.18:0/1505405983 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba3004d020 con 0x7fba3810e860 2024-04-16T17:09:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.534+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba38109a40 con 0x7fba3810e860 2024-04-16T17:09:55.536 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:09:55.536 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:04:43.395334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/94342022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":94342022},{"type":"v1","addr":"172.21.15.22:6837","nonce":94342022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:09:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-16T17:09:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.535+0000 7fba2ea00640 1 -- 172.21.15.18:0/1505405983 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11848 (secure 0 0 0) 0x7fba300beb80 con 0x7fba3810e860 2024-04-16T17:09:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.538+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba1c0a3de0 msgr2=0x7fba1c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba1c0a3de0 0x7fba1c0a62a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fba38110de0 tx=0x7fba28005f90 comp rx=0 tx=0).stop 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 msgr2=0x7fba3819a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 0x7fba3819a920 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fba30037040 tx=0x7fba30035e50 comp rx=0 tx=0).stop 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 shutdown_connections 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba1c0a3de0 0x7fba1c0a62a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba3810e860 0x7fba3819a920 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba38074cf0 0x7fba381a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 --2- 172.21.15.18:0/1505405983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba380743d0 0x7fba381a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 >> 172.21.15.18:0/1505405983 conn(0x7fba380fdad0 msgr2=0x7fba380ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.539+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 shutdown_connections 2024-04-16T17:09:55.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.543+0000 7fba3cc00640 1 -- 172.21.15.18:0/1505405983 wait complete. 2024-04-16T17:09:55.563 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:09:55.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 187435 3.98 MB/sec execute 128 sec latency 20.767 ms 2024-04-16T17:09:55.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 -- 172.21.15.18:0/733773220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b9c102a20 msgr2=0x7f1b9c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 --2- 172.21.15.18:0/733773220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b9c102a20 0x7f1b9c106a60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009860 tx=0x7f1b8802bb90 comp rx=0 tx=0).stop 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 -- 172.21.15.18:0/733773220 shutdown_connections 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 --2- 172.21.15.18:0/733773220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b9c107140 0x7f1b9c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 --2- 172.21.15.18:0/733773220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b9c102a20 0x7f1b9c106a60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 --2- 172.21.15.18:0/733773220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c102070 0x7f1b9c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 -- 172.21.15.18:0/733773220 >> 172.21.15.18:0/733773220 conn(0x7f1b9c0fd950 msgr2=0x7f1b9c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 -- 172.21.15.18:0/733773220 shutdown_connections 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.749+0000 7f1b9be00640 1 -- 172.21.15.18:0/733773220 wait complete. 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.751+0000 7f1b9be00640 1 Processor -- start 2024-04-16T17:09:55.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.751+0000 7f1b9be00640 1 -- start start 2024-04-16T17:09:55.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.751+0000 7f1b9be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c102070 0x7f1b9c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.751+0000 7f1b9be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b9c107140 0x7f1b9c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b9c19a7d0 0x7f1b9c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c112250 con 0x7f1b9c102070 2024-04-16T17:09:55.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1120d0 con 0x7f1b9c107140 2024-04-16T17:09:55.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1123d0 con 0x7f1b9c19a7d0 2024-04-16T17:09:55.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c102070 0x7f1b9c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c102070 0x7f1b9c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37236/0 (socket says 172.21.15.18:37236) 2024-04-16T17:09:55.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9aa00640 1 -- 172.21.15.18:0/3351323634 learned_addr learned my addr 172.21.15.18:0/3351323634 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:55.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b9c107140 0x7f1b9c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.752+0000 7f1b9b400640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b9c19a7d0 0x7f1b9c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b9aa00640 1 -- 172.21.15.18:0/3351323634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b9c19a7d0 msgr2=0x7f1b9c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b9aa00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b9c19a7d0 0x7f1b9c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b9aa00640 1 -- 172.21.15.18:0/3351323634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b9c107140 msgr2=0x7f1b9c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b9aa00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b9c107140 0x7f1b9c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b9aa00640 1 -- 172.21.15.18:0/3351323634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c19b360 con 0x7f1b9c102070 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b9aa00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c102070 0x7f1b9c1a0660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00e9c0 tx=0x7f1b8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b93400640 1 -- 172.21.15.18:0/3351323634 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00cde0 con 0x7f1b9c102070 2024-04-16T17:09:55.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b93400640 1 -- 172.21.15.18:0/3351323634 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b8c0035f0 con 0x7f1b9c102070 2024-04-16T17:09:55.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.753+0000 7f1b93400640 1 -- 172.21.15.18:0/3351323634 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c010690 con 0x7f1b9c102070 2024-04-16T17:09:55.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.754+0000 7f1b9be00640 1 -- 172.21.15.18:0/3351323634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c19b650 con 0x7f1b9c102070 2024-04-16T17:09:55.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.754+0000 7f1b9be00640 1 -- 172.21.15.18:0/3351323634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c1a77b0 con 0x7f1b9c102070 2024-04-16T17:09:55.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.759+0000 7f1b93400640 1 -- 172.21.15.18:0/3351323634 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1b8c0037a0 con 0x7f1b9c102070 2024-04-16T17:09:55.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.759+0000 7f1ba45b8740 1 Processor -- start 2024-04-16T17:09:55.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.759+0000 7f1ba45b8740 1 -- start start 2024-04-16T17:09:55.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab61b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab62040 0x559d7ab6a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab6d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559d7aabe020 con 0x559d7ab61720 2024-04-16T17:09:55.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559d7aa64460 con 0x559d7ab6ac20 2024-04-16T17:09:55.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d7aab9fe0 con 0x559d7ab62040 2024-04-16T17:09:55.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1b9aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab62040 0x559d7ab6a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab61b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab61b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37250/0 (socket says 172.21.15.18:37250) 2024-04-16T17:09:55.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1b9aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab62040 0x559d7ab6a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50258/0 (socket says 172.21.15.18:50258) 2024-04-16T17:09:55.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.760+0000 7f1b9b400640 1 -- 172.21.15.18:0/990769916 learned_addr learned my addr 172.21.15.18:0/990769916 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:55.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.761+0000 7f1b9a000640 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab6d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.761+0000 7f1b9b400640 1 -- 172.21.15.18:0/990769916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab62040 msgr2=0x559d7ab6a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.761+0000 7f1b9b400640 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab62040 0x559d7ab6a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.761+0000 7f1b9b400640 1 -- 172.21.15.18:0/990769916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 msgr2=0x559d7ab6d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.761+0000 7f1b9b400640 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab6d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.761+0000 7f1b9b400640 1 -- 172.21.15.18:0/990769916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7aa6c040 con 0x559d7ab61720 2024-04-16T17:09:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.761+0000 7f1b9aa00640 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab62040 0x559d7ab6a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.762+0000 7f1b93400640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b800a3da0 0x7f1b800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.762+0000 7f1b9b400640 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab61b00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009f60 tx=0x7f1b94009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a975078c5eb3cf20 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.762+0000 7f1b93400640 1 -- 172.21.15.18:0/3351323634 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1b8c0d0f90 con 0x7f1b9c102070 2024-04-16T17:09:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.762+0000 7f1b87e00640 1 -- 172.21.15.18:0/990769916 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9401a070 con 0x559d7ab61720 2024-04-16T17:09:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.762+0000 7f1b87e00640 1 -- 172.21.15.18:0/990769916 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b940158f0 con 0x559d7ab61720 2024-04-16T17:09:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.762+0000 7f1b87e00640 1 -- 172.21.15.18:0/990769916 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400a880 con 0x559d7ab61720 2024-04-16T17:09:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.762+0000 7f1b9aa00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b800a3da0 0x7f1b800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.763+0000 7f1ba45b8740 1 -- 172.21.15.18:0/990769916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 msgr2=0x559d7ab61b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.763+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab61b00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009f60 tx=0x7f1b94009f90 comp rx=0 tx=0).stop 2024-04-16T17:09:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.763+0000 7f1b9aa00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b800a3da0 0x7f1b800a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0ce3e0 tx=0x7f1b8c000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.764+0000 7f1ba45b8740 1 -- 172.21.15.18:0/990769916 shutdown_connections 2024-04-16T17:09:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.764+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab6d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.764+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab62040 0x559d7ab6a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.764+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/990769916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab61b00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.764+0000 7f1ba45b8740 1 -- 172.21.15.18:0/990769916 >> 172.21.15.18:0/990769916 conn(0x559d7ab5be50 msgr2=0x559d7ab5c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:55.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.764+0000 7f1ba45b8740 1 -- 172.21.15.18:0/990769916 shutdown_connections 2024-04-16T17:09:55.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.765+0000 7f1ba45b8740 1 -- 172.21.15.18:0/990769916 wait complete. 2024-04-16T17:09:55.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.765+0000 7f1ba45b8740 1 -- start start 2024-04-16T17:09:55.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.765+0000 7f1ba45b8740 1 Processor -- start 2024-04-16T17:09:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab67cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab68200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab6ac20 0x559d7ab68740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559d7ab6da10 con 0x559d7ab6ac20 2024-04-16T17:09:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559d7aa77830 con 0x559d7ab62040 2024-04-16T17:09:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d7ab6db90 con 0x559d7ab61720 2024-04-16T17:09:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1ba45b8740 10 client.0 _ll_register_callbacks cb 0x559d7aa7f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab67cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab67cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50272/0 (socket says 172.21.15.18:50272) 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.766+0000 7f1b9b400640 1 -- 172.21.15.18:0/2290708089 learned_addr learned my addr 172.21.15.18:0/2290708089 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.767+0000 7f1b9b400640 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab6ac20 0x559d7ab68740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.767+0000 7f1b9a000640 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab68200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.767+0000 7f1b9b400640 1 -- 172.21.15.18:0/2290708089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 msgr2=0x559d7ab68200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.767+0000 7f1b9b400640 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab68200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.767+0000 7f1b9b400640 1 -- 172.21.15.18:0/2290708089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab6ac20 msgr2=0x559d7ab68740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.767+0000 7f1b9b400640 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab6ac20 0x559d7ab68740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.767+0000 7f1b9b400640 1 -- 172.21.15.18:0/2290708089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7aa7b920 con 0x559d7ab61720 2024-04-16T17:09:55.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.768+0000 7f1b9a000640 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab68200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:55.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.768+0000 7f1b9b400640 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab67cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009d60 tx=0x7f1b9400a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec7792218a61d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:55.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.769+0000 7f1b84c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:55.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.769+0000 7f1b84c00640 1 -- 172.21.15.18:0/2290708089 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94029060 con 0x559d7ab61720 2024-04-16T17:09:55.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.769+0000 7f1b84c00640 1 -- 172.21.15.18:0/2290708089 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1b94015cb0 con 0x559d7ab61720 2024-04-16T17:09:55.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.770+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2290708089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b940158f0 con 0x559d7ab61720 2024-04-16T17:09:55.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.770+0000 7f1b84c00640 1 -- 172.21.15.18:0/2290708089 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b9401a070 con 0x559d7ab61720 2024-04-16T17:09:55.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.770+0000 7f1ba45b8740 10 client.26968 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:55.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.770+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2290708089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d7aa497f0 con 0x559d7ab61720 2024-04-16T17:09:55.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.770+0000 7f1b84c00640 1 -- 172.21.15.18:0/2290708089 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b940158f0 con 0x559d7ab61720 2024-04-16T17:09:55.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.771+0000 7f1b84c00640 1 -- 172.21.15.18:0/2290708089 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b94022a10 con 0x559d7ab61720 2024-04-16T17:09:55.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.771+0000 7f1ba45b8740 10 client.26968 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:55.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.771+0000 7f1ba45b8740 10 client.26968 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:09:55.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.771+0000 7f1ba45b8740 20 client.26968 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:55.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.771+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559d7ab7af50 0x559d7ab7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.772+0000 7f1ba45b8740 4 client.26968 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:55.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.772+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2290708089 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d7aacba40 con 0x559d7ab7af50 2024-04-16T17:09:55.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.772+0000 7f1b9a000640 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559d7ab7af50 0x559d7ab7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.773+0000 7f1b9a000640 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559d7ab7af50 0x559d7ab7d340 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5626b38b3511a7cb server_cookie=d51c5fce9f3dc847 in_seq=0 out_seq=0 2024-04-16T17:09:55.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.773+0000 7f1b84c00640 10 client.26968 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:09:55.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.773+0000 7f1b84c00640 1 -- 172.21.15.18:0/2290708089 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d7aacba40 con 0x559d7ab7af50 2024-04-16T17:09:55.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.773+0000 7f1b84c00640 10 client.26968 handle_command_reply: tid=0 2024-04-16T17:09:55.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.773+0000 7f1ba45b8740 1 client.26968 shutdown 2024-04-16T17:09:55.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2290708089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 msgr2=0x559d7ab67cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab67cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009d60 tx=0x7f1b9400a4a0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2290708089 shutdown_connections 2024-04-16T17:09:55.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559d7ab7af50 0x559d7ab7d340 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1b84c00640 0 client.26968 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:09:55.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab6ac20 0x559d7ab68740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab68200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2290708089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab67cc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2290708089 >> 172.21.15.18:0/2290708089 conn(0x559d7ab5be50 msgr2=0x559d7ab5dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:55.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.774+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2290708089 shutdown_connections 2024-04-16T17:09:55.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.775+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2290708089 wait complete. 2024-04-16T17:09:55.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.775+0000 7f1ba45b8740 20 client.26968 trim_cache size 0 max 16384 2024-04-16T17:09:55.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.803+0000 7f1ba45b8740 1 Processor -- start 2024-04-16T17:09:55.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.803+0000 7f1ba45b8740 1 -- start start 2024-04-16T17:09:55.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.803+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.803+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab62040 0x559d7ab52700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.803+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab68560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.803+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1afb90 con 0x559d7ab62040 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.803+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1ad460 con 0x559d7ab6ac20 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.803+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1ab2b0 con 0x559d7ab61720 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab62040 0x559d7ab52700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab68560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab62040 0x559d7ab52700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37268/0 (socket says 172.21.15.18:37268) 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab68560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55726/0 (socket says 172.21.15.18:55726) 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9a000640 1 -- 172.21.15.18:0/2860352242 learned_addr learned my addr 172.21.15.18:0/2860352242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 --2- 172.21.15.18:0/2860352242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 -- 172.21.15.18:0/2860352242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 msgr2=0x559d7ab768d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 --2- 172.21.15.18:0/2860352242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab768d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 -- 172.21.15.18:0/2860352242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab62040 msgr2=0x559d7ab52700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 --2- 172.21.15.18:0/2860352242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab62040 0x559d7ab52700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 -- 172.21.15.18:0/2860352242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7ab61e70 con 0x559d7ab6ac20 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b9b400640 1 --2- 172.21.15.18:0/2860352242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab68560 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1b94001220 tx=0x7f1b94000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4230e9282974280e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.804+0000 7f1b87400640 1 -- 172.21.15.18:0/2860352242 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94032070 con 0x559d7ab6ac20 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1b87400640 1 -- 172.21.15.18:0/2860352242 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9400cd40 con 0x559d7ab6ac20 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1b87400640 1 -- 172.21.15.18:0/2860352242 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94029040 con 0x559d7ab6ac20 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2860352242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 msgr2=0x559d7ab68560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2860352242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab68560 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1b94001220 tx=0x7f1b94000f40 comp rx=0 tx=0).stop 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2860352242 shutdown_connections 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2860352242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab6ac20 0x559d7ab68560 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2860352242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab62040 0x559d7ab52700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/2860352242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab61720 0x559d7ab768d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.805+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2860352242 >> 172.21.15.18:0/2860352242 conn(0x559d7ab5be50 msgr2=0x559d7ab66460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.806+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2860352242 shutdown_connections 2024-04-16T17:09:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.806+0000 7f1ba45b8740 1 -- 172.21.15.18:0/2860352242 wait complete. 2024-04-16T17:09:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.806+0000 7f1ba45b8740 1 -- start start 2024-04-16T17:09:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.806+0000 7f1ba45b8740 1 Processor -- start 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab72e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab6e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1ba45b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab6ac20 0x559d7ab6e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1ab920 con 0x559d7ab61720 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1acd90 con 0x559d7ab62040 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1ba45b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1abff0 con 0x559d7ab6ac20 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1ba45b8740 10 client.0 _ll_register_callbacks cb 0x7f1b94018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab72e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab72e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37282/0 (socket says 172.21.15.18:37282) 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.807+0000 7f1b9b400640 1 -- 172.21.15.18:0/4024521195 learned_addr learned my addr 172.21.15.18:0/4024521195 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.808+0000 7f1b9b400640 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab6ac20 0x559d7ab6e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.808+0000 7f1b9a000640 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab6e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.808+0000 7f1b9b400640 1 -- 172.21.15.18:0/4024521195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab6ac20 msgr2=0x559d7ab6e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:55.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.808+0000 7f1b9b400640 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab6ac20 0x559d7ab6e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.808+0000 7f1b9b400640 1 -- 172.21.15.18:0/4024521195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 msgr2=0x559d7ab6e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:55.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.808+0000 7f1b9b400640 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab6e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.808+0000 7f1b9b400640 1 -- 172.21.15.18:0/4024521195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7a7d03f0 con 0x559d7ab61720 2024-04-16T17:09:55.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.809+0000 7f1b9b400640 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab72e60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400cf80 tx=0x7f1b9400a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4479c9809d524d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:55.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.809+0000 7f1b84c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:09:55.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.809+0000 7f1b84c00640 1 -- 172.21.15.18:0/4024521195 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9403e060 con 0x559d7ab61720 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.809+0000 7f1ba45b8740 1 -- 172.21.15.18:0/4024521195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9400cd40 con 0x559d7ab61720 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.809+0000 7f1b84c00640 1 -- 172.21.15.18:0/4024521195 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f1b94003a10 con 0x559d7ab61720 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.809+0000 7f1b84c00640 1 -- 172.21.15.18:0/4024521195 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b94045050 con 0x559d7ab61720 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.809+0000 7f1ba45b8740 10 client.16782 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.809+0000 7f1ba45b8740 1 -- 172.21.15.18:0/4024521195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d7a89a7d0 con 0x559d7ab61720 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.810+0000 7f1b84c00640 1 -- 172.21.15.18:0/4024521195 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9400cd40 con 0x559d7ab61720 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.810+0000 7f1b84c00640 1 -- 172.21.15.18:0/4024521195 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b94038030 con 0x559d7ab61720 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.810+0000 7f1ba45b8740 10 client.16782 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.810+0000 7f1ba45b8740 10 client.16782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.810+0000 7f1ba45b8740 20 client.16782 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.810+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559d7ab7af50 0x559d7abe31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.810+0000 7f1ba45b8740 4 client.16782 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.810+0000 7f1ba45b8740 1 -- 172.21.15.18:0/4024521195 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d7aac58b0 con 0x559d7ab7af50 2024-04-16T17:09:55.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.811+0000 7f1b9a000640 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559d7ab7af50 0x559d7abe31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.811+0000 7f1b9a000640 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559d7ab7af50 0x559d7abe31a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8649a617014a1cf0 server_cookie=77ee33308f0e60c8 in_seq=0 out_seq=0 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.811+0000 7f1b84c00640 10 client.16782 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1b84c00640 1 -- 172.21.15.18:0/4024521195 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d7aac58b0 con 0x559d7ab7af50 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1b84c00640 10 client.16782 handle_command_reply: tid=0 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 client.16782 shutdown 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 -- 172.21.15.18:0/4024521195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 msgr2=0x559d7ab72e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab72e60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400cf80 tx=0x7f1b9400a4a0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 -- 172.21.15.18:0/4024521195 shutdown_connections 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x559d7ab7af50 0x559d7abe31a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d7ab6ac20 0x559d7ab6e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d7ab62040 0x559d7ab6e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1b84c00640 0 client.16782 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:09:55.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 --2- 172.21.15.18:0/4024521195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d7ab61720 0x559d7ab72e60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 -- 172.21.15.18:0/4024521195 >> 172.21.15.18:0/4024521195 conn(0x559d7ab5be50 msgr2=0x559d7ab66cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:55.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 -- 172.21.15.18:0/4024521195 shutdown_connections 2024-04-16T17:09:55.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 1 -- 172.21.15.18:0/4024521195 wait complete. 2024-04-16T17:09:55.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.812+0000 7f1ba45b8740 20 client.16782 trim_cache size 0 max 16384 2024-04-16T17:09:55.815 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 -- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b800a3da0 msgr2=0x7f1b800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b800a3da0 0x7f1b800a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0ce3e0 tx=0x7f1b8c000bc0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 -- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c102070 msgr2=0x7f1b9c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c102070 0x7f1b9c1a0660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00e9c0 tx=0x7f1b8c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 -- 172.21.15.18:0/3351323634 shutdown_connections 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b800a3da0 0x7f1b800a6260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b9c19a7d0 0x7f1b9c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b9c107140 0x7f1b9c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 --2- 172.21.15.18:0/3351323634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c102070 0x7f1b9c1a0660 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.813+0000 7f1b9be00640 1 -- 172.21.15.18:0/3351323634 >> 172.21.15.18:0/3351323634 conn(0x7f1b9c0fd950 msgr2=0x7f1b9c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.814+0000 7f1b9be00640 1 -- 172.21.15.18:0/3351323634 shutdown_connections 2024-04-16T17:09:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:55.814+0000 7f1b9be00640 1 -- 172.21.15.18:0/3351323634 wait complete. 2024-04-16T17:09:55.838 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:09:55.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:55 smithi022 ceph-mon[36023]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1019 KiB/s wr, 78 op/s 2024-04-16T17:09:55.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1505405983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:55 smithi043 ceph-mon[36901]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1019 KiB/s wr, 78 op/s 2024-04-16T17:09:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1505405983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:56.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 -- 172.21.15.18:0/688686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91e8107140 msgr2=0x7f91e810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 --2- 172.21.15.18:0/688686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91e8107140 0x7f91e810ed10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009ca0 tx=0x7f91e002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 -- 172.21.15.18:0/688686 shutdown_connections 2024-04-16T17:09:56.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 --2- 172.21.15.18:0/688686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91e8107140 0x7f91e810ed10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 --2- 172.21.15.18:0/688686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e8102a20 0x7f91e8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 --2- 172.21.15.18:0/688686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e8102070 0x7f91e8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 -- 172.21.15.18:0/688686 >> 172.21.15.18:0/688686 conn(0x7f91e80fd950 msgr2=0x7f91e80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 -- 172.21.15.18:0/688686 shutdown_connections 2024-04-16T17:09:56.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.051+0000 7f91e7200640 1 -- 172.21.15.18:0/688686 wait complete. 2024-04-16T17:09:56.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.052+0000 7f91e7200640 1 Processor -- start 2024-04-16T17:09:56.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91e7200640 1 -- start start 2024-04-16T17:09:56.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91e7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91e8102070 0x7f91e81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91e7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e8107140 0x7f91e81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91e7200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e819a710 0x7f91e819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91e7200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8112200 con 0x7f91e819a710 2024-04-16T17:09:56.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91e7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8112080 con 0x7f91e8102070 2024-04-16T17:09:56.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91e7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8112380 con 0x7f91e8107140 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e8107140 0x7f91e81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e8107140 0x7f91e81a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50298/0 (socket says 172.21.15.18:50298) 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.053+0000 7f91dfe00640 1 -- 172.21.15.18:0/2782517250 learned_addr learned my addr 172.21.15.18:0/2782517250 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91e5e00640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91e8102070 0x7f91e81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91e6800640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e819a710 0x7f91e819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91dfe00640 1 -- 172.21.15.18:0/2782517250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91e8102070 msgr2=0x7f91e81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91dfe00640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91e8102070 0x7f91e81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91dfe00640 1 -- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e819a710 msgr2=0x7f91e819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91dfe00640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e819a710 0x7f91e819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91dfe00640 1 -- 172.21.15.18:0/2782517250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e819b310 con 0x7f91e8107140 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91e6800640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e819a710 0x7f91e819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91dfe00640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e8107140 0x7f91e81a0b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f91d000e9f0 tx=0x7f91d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91dea00640 1 -- 172.21.15.18:0/2782517250 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d000ccc0 con 0x7f91e8107140 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.054+0000 7f91dea00640 1 -- 172.21.15.18:0/2782517250 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d0005500 con 0x7f91e8107140 2024-04-16T17:09:56.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.055+0000 7f91e7200640 1 -- 172.21.15.18:0/2782517250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e81a73f0 con 0x7f91e8107140 2024-04-16T17:09:56.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.055+0000 7f91dea00640 1 -- 172.21.15.18:0/2782517250 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0010690 con 0x7f91e8107140 2024-04-16T17:09:56.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.055+0000 7f91e7200640 1 -- 172.21.15.18:0/2782517250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e81a77a0 con 0x7f91e8107140 2024-04-16T17:09:56.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.059+0000 7f91ef9db740 1 Processor -- start 2024-04-16T17:09:56.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.059+0000 7f91ef9db740 1 -- start start 2024-04-16T17:09:56.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.059+0000 7f91dea00640 1 -- 172.21.15.18:0/2782517250 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f91d00046c0 con 0x7f91e8107140 2024-04-16T17:09:56.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db0f10 0x557ed6db12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db1830 0x557ed6db9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6dbc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91ef9db740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557ed6d16020 con 0x557ed6db0f10 2024-04-16T17:09:56.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91ef9db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557ed6cbc460 con 0x557ed6db1830 2024-04-16T17:09:56.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91ef9db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557ed6d11fe0 con 0x557ed6dba410 2024-04-16T17:09:56.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db1830 0x557ed6db9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db0f10 0x557ed6db12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db1830 0x557ed6db9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55762/0 (socket says 172.21.15.18:55762) 2024-04-16T17:09:56.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e5e00640 1 -- 172.21.15.18:0/177334522 learned_addr learned my addr 172.21.15.18:0/177334522 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91dfe00640 1 --2- 172.21.15.18:0/177334522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6dbc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e5e00640 1 -- 172.21.15.18:0/177334522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 msgr2=0x557ed6dbc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e5e00640 1 --2- 172.21.15.18:0/177334522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6dbc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e5e00640 1 -- 172.21.15.18:0/177334522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db0f10 msgr2=0x557ed6db12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e5e00640 1 --2- 172.21.15.18:0/177334522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db0f10 0x557ed6db12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.060+0000 7f91e5e00640 1 -- 172.21.15.18:0/177334522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed6cc4040 con 0x557ed6db1830 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.061+0000 7f91e5e00640 1 --2- 172.21.15.18:0/177334522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db1830 0x557ed6db9ed0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009cf0 tx=0x7f91d8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=482f895ab108544d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.061+0000 7f91d7000640 1 -- 172.21.15.18:0/177334522 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8004230 con 0x557ed6db1830 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.061+0000 7f91d7000640 1 -- 172.21.15.18:0/177334522 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d80043d0 con 0x557ed6db1830 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.061+0000 7f91d7000640 1 -- 172.21.15.18:0/177334522 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80198c0 con 0x557ed6db1830 2024-04-16T17:09:56.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.062+0000 7f91dea00640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91cc0a3d20 0x7f91cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.062+0000 7f91dea00640 1 -- 172.21.15.18:0/2782517250 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f91d00d0ef0 con 0x7f91e8107140 2024-04-16T17:09:56.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.062+0000 7f91e6800640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91cc0a3d20 0x7f91cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.062+0000 7f91ef9db740 1 -- 172.21.15.18:0/177334522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db1830 msgr2=0x557ed6db9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.062+0000 7f91ef9db740 1 --2- 172.21.15.18:0/177334522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db1830 0x557ed6db9ed0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009cf0 tx=0x7f91d8011810 comp rx=0 tx=0).stop 2024-04-16T17:09:56.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.063+0000 7f91ef9db740 1 -- 172.21.15.18:0/177334522 shutdown_connections 2024-04-16T17:09:56.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.063+0000 7f91ef9db740 1 --2- 172.21.15.18:0/177334522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6dbc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.063+0000 7f91ef9db740 1 --2- 172.21.15.18:0/177334522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db1830 0x557ed6db9ed0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.063+0000 7f91ef9db740 1 --2- 172.21.15.18:0/177334522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db0f10 0x557ed6db12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.063+0000 7f91ef9db740 1 -- 172.21.15.18:0/177334522 >> 172.21.15.18:0/177334522 conn(0x557ed6dab640 msgr2=0x557ed6daba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.063+0000 7f91ef9db740 1 -- 172.21.15.18:0/177334522 shutdown_connections 2024-04-16T17:09:56.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.063+0000 7f91e6800640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91cc0a3d20 0x7f91cc0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91e0004af0 tx=0x7f91e0004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.063+0000 7f91ef9db740 1 -- 172.21.15.18:0/177334522 wait complete. 2024-04-16T17:09:56.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.064+0000 7f91ef9db740 1 -- start start 2024-04-16T17:09:56.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.064+0000 7f91ef9db740 1 Processor -- start 2024-04-16T17:09:56.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6db7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6db7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91ef9db740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557ed6dbd200 con 0x557ed6db1830 2024-04-16T17:09:56.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91ef9db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557ed6ccf830 con 0x557ed6db0f10 2024-04-16T17:09:56.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91ef9db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557ed6dbd380 con 0x557ed6dba410 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91ef9db740 10 client.0 _ll_register_callbacks cb 0x557ed6cd7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55770/0 (socket says 172.21.15.18:55770) 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91e5e00640 1 -- 172.21.15.18:0/4016749460 learned_addr learned my addr 172.21.15.18:0/4016749460 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.065+0000 7f91e6800640 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6db7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91e5e00640 1 -- 172.21.15.18:0/4016749460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 msgr2=0x557ed6db7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91e5e00640 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6db7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91e5e00640 1 -- 172.21.15.18:0/4016749460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 msgr2=0x557ed6db7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91e5e00640 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6db7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91e5e00640 1 -- 172.21.15.18:0/4016749460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed6cd3920 con 0x557ed6db0f10 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91e5e00640 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db7410 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f91d8019520 tx=0x7f91d8003d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c61558e62eacc028 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91cb400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91cb400640 1 -- 172.21.15.18:0/4016749460 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8027070 con 0x557ed6db0f10 2024-04-16T17:09:56.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91ef9db740 1 -- 172.21.15.18:0/4016749460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d80043d0 con 0x557ed6db0f10 2024-04-16T17:09:56.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.066+0000 7f91cb400640 1 -- 172.21.15.18:0/4016749460 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f91d8005240 con 0x557ed6db0f10 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.067+0000 7f91cb400640 1 -- 172.21.15.18:0/4016749460 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f91d8033050 con 0x557ed6db0f10 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.067+0000 7f91ef9db740 10 client.27020 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.067+0000 7f91ef9db740 1 -- 172.21.15.18:0/4016749460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557ed6ca17f0 con 0x557ed6db0f10 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.067+0000 7f91cb400640 1 -- 172.21.15.18:0/4016749460 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d80043d0 con 0x557ed6db0f10 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.067+0000 7f91cb400640 1 -- 172.21.15.18:0/4016749460 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f91d80195c0 con 0x557ed6db0f10 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.067+0000 7f91ef9db740 10 client.27020 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.067+0000 7f91ef9db740 10 client.27020 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.068+0000 7f91ef9db740 20 client.27020 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.068+0000 7f91ef9db740 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557ed6dca6f0 0x557ed6dccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.068+0000 7f91ef9db740 4 client.27020 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:56.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.068+0000 7f91ef9db740 1 -- 172.21.15.18:0/4016749460 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed6d23a40 con 0x557ed6dca6f0 2024-04-16T17:09:56.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.068+0000 7f91dfe00640 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557ed6dca6f0 0x557ed6dccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.069+0000 7f91cb400640 10 client.27020 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:09:56.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.069+0000 7f91dfe00640 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557ed6dca6f0 0x557ed6dccae0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f73e8bcff9a3a9df server_cookie=c04d77a6baa21f5f in_seq=0 out_seq=0 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.070+0000 7f91cb400640 1 -- 172.21.15.18:0/4016749460 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ed6d23a40 con 0x557ed6dca6f0 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.070+0000 7f91cb400640 10 client.27020 handle_command_reply: tid=0 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.070+0000 7f91ef9db740 1 client.27020 shutdown 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.070+0000 7f91ef9db740 1 -- 172.21.15.18:0/4016749460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 msgr2=0x557ed6db7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.070+0000 7f91ef9db740 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db7410 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f91d8019520 tx=0x7f91d8003d30 comp rx=0 tx=0).stop 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 1 -- 172.21.15.18:0/4016749460 shutdown_connections 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557ed6dca6f0 0x557ed6dccae0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6db7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6db7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 1 --2- 172.21.15.18:0/4016749460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db7410 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91cb400640 0 client.27020 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:09:56.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 1 -- 172.21.15.18:0/4016749460 >> 172.21.15.18:0/4016749460 conn(0x557ed6dab640 msgr2=0x557ed6dad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 1 -- 172.21.15.18:0/4016749460 shutdown_connections 2024-04-16T17:09:56.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 1 -- 172.21.15.18:0/4016749460 wait complete. 2024-04-16T17:09:56.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.071+0000 7f91ef9db740 20 client.27020 trim_cache size 0 max 16384 2024-04-16T17:09:56.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.086+0000 7f91ef9db740 1 Processor -- start 2024-04-16T17:09:56.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91ef9db740 1 -- start start 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6dc60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6daa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91ef9db740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81b3ee0 con 0x557ed6db1830 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91ef9db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81b17b0 con 0x557ed6db0f10 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91ef9db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81af600 con 0x557ed6dba410 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91dfe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6dc60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55774/0 (socket says 172.21.15.18:55774) 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91e5e00640 1 -- 172.21.15.18:0/1788342493 learned_addr learned my addr 172.21.15.18:0/1788342493 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.087+0000 7f91e6800640 1 --2- 172.21.15.18:0/1788342493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6daa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91e5e00640 1 -- 172.21.15.18:0/1788342493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 msgr2=0x557ed6daa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91e5e00640 1 --2- 172.21.15.18:0/1788342493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6daa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91e5e00640 1 -- 172.21.15.18:0/1788342493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 msgr2=0x557ed6dc60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91e5e00640 1 --2- 172.21.15.18:0/1788342493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6dc60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91e5e00640 1 -- 172.21.15.18:0/1788342493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed6dcce50 con 0x557ed6db0f10 2024-04-16T17:09:56.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91e5e00640 1 --2- 172.21.15.18:0/1788342493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db9b40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f91d8003fa0 tx=0x7f91d8005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe459015191994 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91d6600640 1 -- 172.21.15.18:0/1788342493 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8030020 con 0x557ed6db0f10 2024-04-16T17:09:56.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91d6600640 1 -- 172.21.15.18:0/1788342493 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d8015070 con 0x557ed6db0f10 2024-04-16T17:09:56.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.088+0000 7f91d6600640 1 -- 172.21.15.18:0/1788342493 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8027040 con 0x557ed6db0f10 2024-04-16T17:09:56.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.089+0000 7f91ef9db740 1 -- 172.21.15.18:0/1788342493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 msgr2=0x557ed6db9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.089+0000 7f91ef9db740 1 --2- 172.21.15.18:0/1788342493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db9b40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f91d8003fa0 tx=0x7f91d8005d30 comp rx=0 tx=0).stop 2024-04-16T17:09:56.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.091+0000 7f91ef9db740 1 -- 172.21.15.18:0/1788342493 shutdown_connections 2024-04-16T17:09:56.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.091+0000 7f91ef9db740 1 --2- 172.21.15.18:0/1788342493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6daa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.091+0000 7f91ef9db740 1 --2- 172.21.15.18:0/1788342493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6dc60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.091+0000 7f91ef9db740 1 --2- 172.21.15.18:0/1788342493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6db9b40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.091+0000 7f91ef9db740 1 -- 172.21.15.18:0/1788342493 >> 172.21.15.18:0/1788342493 conn(0x557ed6dab640 msgr2=0x557ed6da9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.091+0000 7f91ef9db740 1 -- 172.21.15.18:0/1788342493 shutdown_connections 2024-04-16T17:09:56.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.091+0000 7f91ef9db740 1 -- 172.21.15.18:0/1788342493 wait complete. 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.091+0000 7f91ef9db740 1 -- start start 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91ef9db740 1 Processor -- start 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6dc2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6dbdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91ef9db740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6dbe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91ef9db740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81afc70 con 0x557ed6db1830 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91ef9db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81b10e0 con 0x557ed6db0f10 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91ef9db740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91e81b0340 con 0x557ed6dba410 2024-04-16T17:09:56.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91ef9db740 10 client.0 _ll_register_callbacks cb 0x7f91d8009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:56.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.092+0000 7f91e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6dc2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6dc2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55788/0 (socket says 172.21.15.18:55788) 2024-04-16T17:09:56.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e5e00640 1 -- 172.21.15.18:0/3903880576 learned_addr learned my addr 172.21.15.18:0/3903880576 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91dfe00640 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6dbdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e6800640 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6dbe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e5e00640 1 -- 172.21.15.18:0/3903880576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 msgr2=0x557ed6dbe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e5e00640 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6dbe0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e5e00640 1 -- 172.21.15.18:0/3903880576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 msgr2=0x557ed6dbdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e5e00640 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6dbdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e5e00640 1 -- 172.21.15.18:0/3903880576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed6db1660 con 0x557ed6db0f10 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91e5e00640 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6dc2650 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f91d8019460 tx=0x7f91d8003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e6a983f16ceceb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91cb400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.093+0000 7f91cb400640 1 -- 172.21.15.18:0/3903880576 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8027070 con 0x557ed6db0f10 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.094+0000 7f91cb400640 1 -- 172.21.15.18:0/3903880576 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f91d8019690 con 0x557ed6db0f10 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.094+0000 7f91ef9db740 1 -- 172.21.15.18:0/3903880576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d8015070 con 0x557ed6db0f10 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.094+0000 7f91cb400640 1 -- 172.21.15.18:0/3903880576 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f91d8049050 con 0x557ed6db0f10 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.094+0000 7f91ef9db740 10 client.27026 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.094+0000 7f91ef9db740 1 -- 172.21.15.18:0/3903880576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557ed6a283f0 con 0x557ed6db0f10 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.094+0000 7f91cb400640 1 -- 172.21.15.18:0/3903880576 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d8015070 con 0x557ed6db0f10 2024-04-16T17:09:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.094+0000 7f91cb400640 1 -- 172.21.15.18:0/3903880576 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f91d80048d0 con 0x557ed6db0f10 2024-04-16T17:09:56.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.095+0000 7f91ef9db740 10 client.27026 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:56.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.095+0000 7f91ef9db740 10 client.27026 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:09:56.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.095+0000 7f91ef9db740 20 client.27026 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:56.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.095+0000 7f91ef9db740 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557ed6e30b10 0x557ed6e32f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.095+0000 7f91ef9db740 4 client.27026 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:56.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.095+0000 7f91ef9db740 1 -- 172.21.15.18:0/3903880576 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ed6d1d8b0 con 0x557ed6e30b10 2024-04-16T17:09:56.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.095+0000 7f91dfe00640 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557ed6e30b10 0x557ed6e32f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.096+0000 7f91dfe00640 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557ed6e30b10 0x557ed6e32f30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=314f9ea1ff34074 server_cookie=faf836890fabb22b in_seq=0 out_seq=0 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.096+0000 7f91cb400640 10 client.27026 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.096+0000 7f91cb400640 1 -- 172.21.15.18:0/3903880576 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ed6d1d8b0 con 0x557ed6e30b10 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.096+0000 7f91cb400640 10 client.27026 handle_command_reply: tid=0 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.096+0000 7f91ef9db740 1 client.27026 shutdown 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 -- 172.21.15.18:0/3903880576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 msgr2=0x557ed6dc2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6dc2650 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f91d8019460 tx=0x7f91d8003e20 comp rx=0 tx=0).stop 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 -- 172.21.15.18:0/3903880576 shutdown_connections 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x557ed6e30b10 0x557ed6e32f30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ed6dba410 0x557ed6dbe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ed6db1830 0x557ed6dbdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 --2- 172.21.15.18:0/3903880576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ed6db0f10 0x557ed6dc2650 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 -- 172.21.15.18:0/3903880576 >> 172.21.15.18:0/3903880576 conn(0x557ed6dab640 msgr2=0x557ed6daa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91cb400640 0 client.27026 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:09:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 -- 172.21.15.18:0/3903880576 shutdown_connections 2024-04-16T17:09:56.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 1 -- 172.21.15.18:0/3903880576 wait complete. 2024-04-16T17:09:56.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.097+0000 7f91ef9db740 20 client.27026 trim_cache size 0 max 16384 2024-04-16T17:09:56.100 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:09:56.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 -- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91cc0a3d20 msgr2=0x7f91cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91cc0a3d20 0x7f91cc0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91e0004af0 tx=0x7f91e0004a40 comp rx=0 tx=0).stop 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 -- 172.21.15.18:0/2782517250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e8107140 msgr2=0x7f91e81a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e8107140 0x7f91e81a0b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f91d000e9f0 tx=0x7f91d000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 -- 172.21.15.18:0/2782517250 shutdown_connections 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91cc0a3d20 0x7f91cc0a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e819a710 0x7f91e819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e8107140 0x7f91e81a0b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 --2- 172.21.15.18:0/2782517250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91e8102070 0x7f91e81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 -- 172.21.15.18:0/2782517250 >> 172.21.15.18:0/2782517250 conn(0x7f91e80fd950 msgr2=0x7f91e80ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 -- 172.21.15.18:0/2782517250 shutdown_connections 2024-04-16T17:09:56.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.098+0000 7f91e7200640 1 -- 172.21.15.18:0/2782517250 wait complete. 2024-04-16T17:09:56.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:09:56.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 -- 172.21.15.18:0/518722080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff38c103900 msgr2=0x7ff38c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 --2- 172.21.15.18:0/518722080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff38c103900 0x7ff38c10bde0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff378009860 tx=0x7ff37802bc00 comp rx=0 tx=0).stop 2024-04-16T17:09:56.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 -- 172.21.15.18:0/518722080 shutdown_connections 2024-04-16T17:09:56.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 --2- 172.21.15.18:0/518722080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c10c400 0x7ff38c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 --2- 172.21.15.18:0/518722080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff38c103900 0x7ff38c10bde0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 --2- 172.21.15.18:0/518722080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff38c102fe0 0x7ff38c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 -- 172.21.15.18:0/518722080 >> 172.21.15.18:0/518722080 conn(0x7ff38c077ab0 msgr2=0x7ff38c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 -- 172.21.15.18:0/518722080 shutdown_connections 2024-04-16T17:09:56.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.323+0000 7ff38b200640 1 -- 172.21.15.18:0/518722080 wait complete. 2024-04-16T17:09:56.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.324+0000 7ff38b200640 1 Processor -- start 2024-04-16T17:09:56.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.324+0000 7ff38b200640 1 -- start start 2024-04-16T17:09:56.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff38b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff38c102fe0 0x7ff38c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff38b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c103900 0x7ff38c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff38b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff38c10c400 0x7ff38c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff38b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c112190 con 0x7ff38c102fe0 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff38b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c112010 con 0x7ff38c10c400 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff38b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c112310 con 0x7ff38c103900 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff389400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c103900 0x7ff38c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff389400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c103900 0x7ff38c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50328/0 (socket says 172.21.15.18:50328) 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff389400640 1 -- 172.21.15.18:0/469486885 learned_addr learned my addr 172.21.15.18:0/469486885 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff389e00640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff38c102fe0 0x7ff38c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.325+0000 7ff38a800640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff38c10c400 0x7ff38c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff389400640 1 -- 172.21.15.18:0/469486885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff38c10c400 msgr2=0x7ff38c19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff389400640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff38c10c400 0x7ff38c19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff389400640 1 -- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff38c102fe0 msgr2=0x7ff38c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff389400640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff38c102fe0 0x7ff38c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff389400640 1 -- 172.21.15.18:0/469486885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c19d740 con 0x7ff38c103900 2024-04-16T17:09:56.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff389400640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c103900 0x7ff38c198c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff378037040 tx=0x7ff3780344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff382a00640 1 -- 172.21.15.18:0/469486885 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378041020 con 0x7ff38c103900 2024-04-16T17:09:56.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff382a00640 1 -- 172.21.15.18:0/469486885 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37802ca80 con 0x7ff38c103900 2024-04-16T17:09:56.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff382a00640 1 -- 172.21.15.18:0/469486885 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378039040 con 0x7ff38c103900 2024-04-16T17:09:56.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.326+0000 7ff38b200640 1 -- 172.21.15.18:0/469486885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c19d9d0 con 0x7ff38c103900 2024-04-16T17:09:56.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.327+0000 7ff38b200640 1 -- 172.21.15.18:0/469486885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c1a52c0 con 0x7ff38c103900 2024-04-16T17:09:56.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.330+0000 7ff382a00640 1 -- 172.21.15.18:0/469486885 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff378044020 con 0x7ff38c103900 2024-04-16T17:09:56.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.330+0000 7ff393e28740 1 Processor -- start 2024-04-16T17:09:56.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.330+0000 7ff393e28740 1 -- start start 2024-04-16T17:09:56.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff393e28740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff393e28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5ac830 0x558f8b5b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff393e28740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5b5410 0x558f8b5b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff393e28740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558f8b511020 con 0x558f8b5abf10 2024-04-16T17:09:56.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff393e28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558f8b4b7460 con 0x558f8b5ac830 2024-04-16T17:09:56.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff393e28740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558f8b50cfe0 con 0x558f8b5b5410 2024-04-16T17:09:56.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff38a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff389400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5b5410 0x558f8b5b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff38a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37370/0 (socket says 172.21.15.18:37370) 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff38a800640 1 -- 172.21.15.18:0/3867144975 learned_addr learned my addr 172.21.15.18:0/3867144975 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.331+0000 7ff389e00640 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5ac830 0x558f8b5b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff38a800640 1 -- 172.21.15.18:0/3867144975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5b5410 msgr2=0x558f8b5b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff38a800640 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5b5410 0x558f8b5b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff38a800640 1 -- 172.21.15.18:0/3867144975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5ac830 msgr2=0x558f8b5b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff38a800640 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5ac830 0x558f8b5b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff38a800640 1 -- 172.21.15.18:0/3867144975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8b4bf040 con 0x558f8b5abf10 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff389e00640 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5ac830 0x558f8b5b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff38a800640 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5ac2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff384009f50 tx=0x7ff3840118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c5db9e76970108 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff382a00640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3700a3d20 0x7ff3700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.332+0000 7ff377400640 1 -- 172.21.15.18:0/3867144975 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384004d20 con 0x558f8b5abf10 2024-04-16T17:09:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.333+0000 7ff382a00640 1 -- 172.21.15.18:0/469486885 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff3780f21f0 con 0x7ff38c103900 2024-04-16T17:09:56.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.333+0000 7ff377400640 1 -- 172.21.15.18:0/3867144975 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384004050 con 0x558f8b5abf10 2024-04-16T17:09:56.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.333+0000 7ff377400640 1 -- 172.21.15.18:0/3867144975 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384005280 con 0x558f8b5abf10 2024-04-16T17:09:56.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.333+0000 7ff389e00640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3700a3d20 0x7ff3700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff389e00640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3700a3d20 0x7ff3700a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff37c017fd0 tx=0x7ff37c019040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 -- 172.21.15.18:0/3867144975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 msgr2=0x558f8b5ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5ac2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff384009f50 tx=0x7ff3840118d0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 -- 172.21.15.18:0/3867144975 shutdown_connections 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5b5410 0x558f8b5b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5ac830 0x558f8b5b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 --2- 172.21.15.18:0/3867144975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5ac2f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 -- 172.21.15.18:0/3867144975 >> 172.21.15.18:0/3867144975 conn(0x558f8b5a6640 msgr2=0x558f8b5a6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 -- 172.21.15.18:0/3867144975 shutdown_connections 2024-04-16T17:09:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.334+0000 7ff393e28740 1 -- 172.21.15.18:0/3867144975 wait complete. 2024-04-16T17:09:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.335+0000 7ff393e28740 1 -- start start 2024-04-16T17:09:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.335+0000 7ff393e28740 1 Processor -- start 2024-04-16T17:09:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff393e28740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff393e28740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff393e28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff393e28740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558f8b5b8200 con 0x558f8b5ac830 2024-04-16T17:09:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff393e28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558f8b4ca830 con 0x558f8b5b5410 2024-04-16T17:09:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff393e28740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558f8b5b8380 con 0x558f8b5abf10 2024-04-16T17:09:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff393e28740 10 client.0 _ll_register_callbacks cb 0x558f8b4d2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50344/0 (socket says 172.21.15.18:50344) 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff389e00640 1 -- 172.21.15.18:0/3738692387 learned_addr learned my addr 172.21.15.18:0/3738692387 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff389400640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.336+0000 7ff38a800640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff389400640 1 -- 172.21.15.18:0/3738692387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 msgr2=0x558f8b5b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff389400640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff389400640 1 -- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 msgr2=0x558f8b5b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff389400640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff389400640 1 -- 172.21.15.18:0/3738692387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f8b4ce920 con 0x558f8b5ac830 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff389e00640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff38a800640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff36be00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:09:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.337+0000 7ff389400640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5b29f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff3780ef640 tx=0x7ff378005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc04ac20bdaa609 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.338+0000 7ff36be00640 1 -- 172.21.15.18:0/3738692387 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780ba130 con 0x558f8b5ac830 2024-04-16T17:09:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.338+0000 7ff36be00640 1 -- 172.21.15.18:0/3738692387 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff3780c6670 con 0x558f8b5ac830 2024-04-16T17:09:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.338+0000 7ff393e28740 1 -- 172.21.15.18:0/3738692387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff384004050 con 0x558f8b5ac830 2024-04-16T17:09:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.338+0000 7ff36be00640 1 -- 172.21.15.18:0/3738692387 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff3780bd7f0 con 0x558f8b5ac830 2024-04-16T17:09:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.338+0000 7ff393e28740 10 client.16806 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.338+0000 7ff393e28740 1 -- 172.21.15.18:0/3738692387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558f8b49c7f0 con 0x558f8b5ac830 2024-04-16T17:09:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff36be00640 1 -- 172.21.15.18:0/3738692387 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384004050 con 0x558f8b5ac830 2024-04-16T17:09:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff36be00640 1 -- 172.21.15.18:0/3738692387 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff37803c030 con 0x558f8b5ac830 2024-04-16T17:09:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff393e28740 10 client.16806 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff393e28740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff393e28740 20 client.16806 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff393e28740 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558f8b5c5770 0x558f8b5c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff393e28740 4 client.16806 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff393e28740 1 -- 172.21.15.18:0/3738692387 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f8b51ea40 con 0x558f8b5c5770 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.339+0000 7ff389e00640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558f8b5c5770 0x558f8b5c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.340+0000 7ff389e00640 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558f8b5c5770 0x558f8b5c7b80 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=618fdb1550b2bf63 server_cookie=2a407d41b6bd6812 in_seq=0 out_seq=0 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.340+0000 7ff36be00640 10 client.16806 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff36be00640 1 -- 172.21.15.18:0/3738692387 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f8b51ea40 con 0x558f8b5c5770 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff36be00640 10 client.16806 handle_command_reply: tid=0 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 client.16806 shutdown 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 -- 172.21.15.18:0/3738692387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 msgr2=0x558f8b5b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5b29f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff3780ef640 tx=0x7ff378005b00 comp rx=0 tx=0).stop 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 -- 172.21.15.18:0/3738692387 shutdown_connections 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558f8b5c5770 0x558f8b5c7b80 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5b29f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 --2- 172.21.15.18:0/3738692387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 -- 172.21.15.18:0/3738692387 >> 172.21.15.18:0/3738692387 conn(0x558f8b5a6640 msgr2=0x558f8b5a8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff36be00640 0 client.16806 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 -- 172.21.15.18:0/3738692387 shutdown_connections 2024-04-16T17:09:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.341+0000 7ff393e28740 1 -- 172.21.15.18:0/3738692387 wait complete. 2024-04-16T17:09:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.342+0000 7ff393e28740 20 client.16806 trim_cache size 0 max 16384 2024-04-16T17:09:56.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.361+0000 7ff393e28740 1 Processor -- start 2024-04-16T17:09:56.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.361+0000 7ff393e28740 1 -- start start 2024-04-16T17:09:56.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.361+0000 7ff393e28740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.361+0000 7ff393e28740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5ac830 0x558f8b5c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.361+0000 7ff393e28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.361+0000 7ff393e28740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1afad0 con 0x558f8b5abf10 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.361+0000 7ff393e28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1ad3a0 con 0x558f8b5b5410 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.361+0000 7ff393e28740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1ab1f0 con 0x558f8b5ac830 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5ac830 0x558f8b5c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5ac830 0x558f8b5c10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50360/0 (socket says 172.21.15.18:50360) 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 -- 172.21.15.18:0/2386283049 learned_addr learned my addr 172.21.15.18:0/2386283049 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff38a800640 1 --2- 172.21.15.18:0/2386283049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 -- 172.21.15.18:0/2386283049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 msgr2=0x558f8b5cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 --2- 172.21.15.18:0/2386283049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 -- 172.21.15.18:0/2386283049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 msgr2=0x558f8b5b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 --2- 172.21.15.18:0/2386283049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 -- 172.21.15.18:0/2386283049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8b5ac660 con 0x558f8b5ac830 2024-04-16T17:09:56.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.362+0000 7ff389400640 1 --2- 172.21.15.18:0/2386283049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5ac830 0x558f8b5c10c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff37803d040 tx=0x7ff3780c6b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ebc01bef7495ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.363+0000 7ff376a00640 1 -- 172.21.15.18:0/2386283049 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780bd9d0 con 0x558f8b5ac830 2024-04-16T17:09:56.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.363+0000 7ff376a00640 1 -- 172.21.15.18:0/2386283049 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff378003e70 con 0x558f8b5ac830 2024-04-16T17:09:56.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.363+0000 7ff376a00640 1 -- 172.21.15.18:0/2386283049 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780bdc60 con 0x558f8b5ac830 2024-04-16T17:09:56.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 -- 172.21.15.18:0/2386283049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5ac830 msgr2=0x558f8b5c10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 --2- 172.21.15.18:0/2386283049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5ac830 0x558f8b5c10c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff37803d040 tx=0x7ff3780c6b00 comp rx=0 tx=0).stop 2024-04-16T17:09:56.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 -- 172.21.15.18:0/2386283049 shutdown_connections 2024-04-16T17:09:56.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 --2- 172.21.15.18:0/2386283049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 --2- 172.21.15.18:0/2386283049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5ac830 0x558f8b5c10c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 --2- 172.21.15.18:0/2386283049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5abf10 0x558f8b5b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 -- 172.21.15.18:0/2386283049 >> 172.21.15.18:0/2386283049 conn(0x558f8b5a6640 msgr2=0x558f8b5ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 -- 172.21.15.18:0/2386283049 shutdown_connections 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 -- 172.21.15.18:0/2386283049 wait complete. 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.364+0000 7ff393e28740 1 -- start start 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff393e28740 1 Processor -- start 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff393e28740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5bd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff393e28740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff393e28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff393e28740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1ab860 con 0x558f8b5ac830 2024-04-16T17:09:56.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff393e28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1accd0 con 0x558f8b5b5410 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff393e28740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1abf30 con 0x558f8b5abf10 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff393e28740 10 client.0 _ll_register_callbacks cb 0x7ff378034a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5bd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5bd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50370/0 (socket says 172.21.15.18:50370) 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff389e00640 1 -- 172.21.15.18:0/1715740671 learned_addr learned my addr 172.21.15.18:0/1715740671 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff389400640 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.365+0000 7ff38a800640 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff389e00640 1 -- 172.21.15.18:0/1715740671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 msgr2=0x558f8b5af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff389e00640 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff389e00640 1 -- 172.21.15.18:0/1715740671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 msgr2=0x558f8b5aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff389e00640 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff389e00640 1 -- 172.21.15.18:0/1715740671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f8b2233f0 con 0x558f8b5abf10 2024-04-16T17:09:56.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff389400640 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff389e00640 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5bd8a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x558f8b5c83f0 tx=0x7ff37c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf96772d60870a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff36be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff36be00640 1 -- 172.21.15.18:0/1715740671 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c023850 con 0x558f8b5abf10 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff36be00640 1 -- 172.21.15.18:0/1715740671 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7ff37c0239f0 con 0x558f8b5abf10 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.366+0000 7ff393e28740 1 -- 172.21.15.18:0/1715740671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff378003e70 con 0x558f8b5abf10 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff36be00640 1 -- 172.21.15.18:0/1715740671 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff37c049050 con 0x558f8b5abf10 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff393e28740 10 client.27010 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff393e28740 1 -- 172.21.15.18:0/1715740671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558f8b2ed7d0 con 0x558f8b5abf10 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff36be00640 1 -- 172.21.15.18:0/1715740671 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff378003e70 con 0x558f8b5abf10 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff36be00640 1 -- 172.21.15.18:0/1715740671 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff37c004940 con 0x558f8b5abf10 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff393e28740 10 client.27010 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff393e28740 10 client.27010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:56.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff393e28740 20 client.27010 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff393e28740 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558f8b62c010 0x558f8b62e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff393e28740 4 client.27010 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff393e28740 1 -- 172.21.15.18:0/1715740671 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f8b5188b0 con 0x558f8b62c010 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.367+0000 7ff389400640 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558f8b62c010 0x558f8b62e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff389400640 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558f8b62c010 0x558f8b62e430 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8787b8bca0c6328d server_cookie=3f495a90548ce37c in_seq=0 out_seq=0 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff36be00640 10 client.27010 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff36be00640 1 -- 172.21.15.18:0/1715740671 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f8b5188b0 con 0x558f8b62c010 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff36be00640 10 client.27010 handle_command_reply: tid=0 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 client.27010 shutdown 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 -- 172.21.15.18:0/1715740671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 msgr2=0x558f8b5bd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5bd8a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x558f8b5c83f0 tx=0x7ff37c01f000 comp rx=0 tx=0).stop 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 -- 172.21.15.18:0/1715740671 shutdown_connections 2024-04-16T17:09:56.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558f8b62c010 0x558f8b62e430 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f8b5b5410 0x558f8b5af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff36be00640 0 client.27010 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f8b5ac830 0x558f8b5aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 --2- 172.21.15.18:0/1715740671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f8b5abf10 0x558f8b5bd8a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 -- 172.21.15.18:0/1715740671 >> 172.21.15.18:0/1715740671 conn(0x558f8b5a6640 msgr2=0x558f8b5a7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.368+0000 7ff393e28740 1 -- 172.21.15.18:0/1715740671 shutdown_connections 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff393e28740 1 -- 172.21.15.18:0/1715740671 wait complete. 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff393e28740 20 client.27010 trim_cache size 0 max 16384 2024-04-16T17:09:56.374 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:09:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 -- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3700a3d20 msgr2=0x7ff3700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3700a3d20 0x7ff3700a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff37c017fd0 tx=0x7ff37c019040 comp rx=0 tx=0).stop 2024-04-16T17:09:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 -- 172.21.15.18:0/469486885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c103900 msgr2=0x7ff38c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c103900 0x7ff38c198c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff378037040 tx=0x7ff3780344c0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 -- 172.21.15.18:0/469486885 shutdown_connections 2024-04-16T17:09:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3700a3d20 0x7ff3700a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff38c10c400 0x7ff38c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c103900 0x7ff38c198c90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 --2- 172.21.15.18:0/469486885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff38c102fe0 0x7ff38c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.369+0000 7ff38b200640 1 -- 172.21.15.18:0/469486885 >> 172.21.15.18:0/469486885 conn(0x7ff38c077ab0 msgr2=0x7ff38c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.370+0000 7ff38b200640 1 -- 172.21.15.18:0/469486885 shutdown_connections 2024-04-16T17:09:56.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:56.370+0000 7ff38b200640 1 -- 172.21.15.18:0/469486885 wait complete. 2024-04-16T17:09:56.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 188476 4.00 MB/sec execute 129 sec latency 24.753 ms 2024-04-16T17:09:57.081 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:51 smithi018 ceph-mon[31776]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-16T17:09:57.081 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:53 smithi018 ceph-mon[31776]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1019 KiB/s wr, 78 op/s 2024-04-16T17:09:57.081 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:55 smithi018 ceph-mon[31776]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1019 KiB/s wr, 78 op/s 2024-04-16T17:09:57.081 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1505405983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:57.387 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:09:57.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:09:57.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:09:57.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 189409 4.01 MB/sec execute 130 sec latency 20.337 ms 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 -- 172.21.15.18:0/3508969377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc574074cf0 msgr2=0x7fc57410e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 --2- 172.21.15.18:0/3508969377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc574074cf0 0x7fc57410e160 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc560009900 tx=0x7fc56002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 -- 172.21.15.18:0/3508969377 shutdown_connections 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 --2- 172.21.15.18:0/3508969377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc57410e6a0 0x7fc574110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 --2- 172.21.15.18:0/3508969377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc574074cf0 0x7fc57410e160 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 --2- 172.21.15.18:0/3508969377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5740743d0 0x7fc5740747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 -- 172.21.15.18:0/3508969377 >> 172.21.15.18:0/3508969377 conn(0x7fc5740fd950 msgr2=0x7fc5740ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 -- 172.21.15.18:0/3508969377 shutdown_connections 2024-04-16T17:09:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.610+0000 7fc578a00640 1 -- 172.21.15.18:0/3508969377 wait complete. 2024-04-16T17:09:57.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.612+0000 7fc578a00640 1 Processor -- start 2024-04-16T17:09:57.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.612+0000 7fc578a00640 1 -- start start 2024-04-16T17:09:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.612+0000 7fc578a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5740743d0 0x7fc5741a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.612+0000 7fc578a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574074cf0 0x7fc5741a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.612+0000 7fc578a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc57410e6a0 0x7fc57419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.612+0000 7fc578a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc574114420 con 0x7fc57410e6a0 2024-04-16T17:09:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.612+0000 7fc578a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741142a0 con 0x7fc5740743d0 2024-04-16T17:09:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.612+0000 7fc578a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741145a0 con 0x7fc574074cf0 2024-04-16T17:09:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5740743d0 0x7fc5741a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5740743d0 0x7fc5741a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35502/0 (socket says 172.21.15.18:35502) 2024-04-16T17:09:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573400640 1 -- 172.21.15.18:0/840423666 learned_addr learned my addr 172.21.15.18:0/840423666 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573e00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc57410e6a0 0x7fc57419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc572a00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574074cf0 0x7fc5741a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573400640 1 -- 172.21.15.18:0/840423666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574074cf0 msgr2=0x7fc5741a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573400640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574074cf0 0x7fc5741a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573400640 1 -- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc57410e6a0 msgr2=0x7fc57419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573400640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc57410e6a0 0x7fc57419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc573400640 1 -- 172.21.15.18:0/840423666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57419af90 con 0x7fc5740743d0 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.613+0000 7fc572a00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574074cf0 0x7fc5741a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.614+0000 7fc573400640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5740743d0 0x7fc5741a0650 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc56400ea60 tx=0x7fc56400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.614+0000 7fc56be00640 1 -- 172.21.15.18:0/840423666 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56400cde0 con 0x7fc5740743d0 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.614+0000 7fc56be00640 1 -- 172.21.15.18:0/840423666 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5640035f0 con 0x7fc5740743d0 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.614+0000 7fc578a00640 1 -- 172.21.15.18:0/840423666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57419b280 con 0x7fc5740743d0 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.614+0000 7fc56be00640 1 -- 172.21.15.18:0/840423666 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564010670 con 0x7fc5740743d0 2024-04-16T17:09:57.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.614+0000 7fc578a00640 1 -- 172.21.15.18:0/840423666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741a74a0 con 0x7fc5740743d0 2024-04-16T17:09:57.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.618+0000 7fc56be00640 1 -- 172.21.15.18:0/840423666 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5640037a0 con 0x7fc5740743d0 2024-04-16T17:09:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.619+0000 7fc57cda9740 1 Processor -- start 2024-04-16T17:09:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.620+0000 7fc56be00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5580a3d20 0x7fc5580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.620+0000 7fc57cda9740 1 -- start start 2024-04-16T17:09:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.620+0000 7fc56be00640 1 -- 172.21.15.18:0/840423666 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc5640d0f80 con 0x7fc5740743d0 2024-04-16T17:09:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.620+0000 7fc573e00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5580a3d20 0x7fc5580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8c9e90 0x56122c8ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8d3390 0x56122c8d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc57cda9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56122c82ecb0 con 0x56122c8c9e90 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc57cda9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56122c7d5370 con 0x56122c8d3390 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc57cda9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56122c82ac70 con 0x56122c8ca7b0 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc572a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc573400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8c9e90 0x56122c8ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc573e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8d3390 0x56122c8d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc572a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38390/0 (socket says 172.21.15.18:38390) 2024-04-16T17:09:57.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.621+0000 7fc572a00640 1 -- 172.21.15.18:0/1028781889 learned_addr learned my addr 172.21.15.18:0/1028781889 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc573e00640 1 -- 172.21.15.18:0/1028781889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 msgr2=0x56122c8d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc573e00640 1 --2- 172.21.15.18:0/1028781889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc573e00640 1 -- 172.21.15.18:0/1028781889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8c9e90 msgr2=0x56122c8ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc573e00640 1 --2- 172.21.15.18:0/1028781889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8c9e90 0x56122c8ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc573e00640 1 -- 172.21.15.18:0/1028781889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122c7f4910 con 0x56122c8d3390 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc572a00640 1 --2- 172.21.15.18:0/1028781889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc573e00640 1 --2- 172.21.15.18:0/1028781889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8d3390 0x56122c8d5780 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc574075320 tx=0x7fc56c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60db56ec3ce454d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc573e00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5580a3d20 0x7fc5580a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc574069e30 tx=0x7fc56c009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc55fe00640 1 -- 172.21.15.18:0/1028781889 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c004b20 con 0x56122c8d3390 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc55fe00640 1 -- 172.21.15.18:0/1028781889 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56c004cc0 con 0x56122c8d3390 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.622+0000 7fc55fe00640 1 -- 172.21.15.18:0/1028781889 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c01a840 con 0x56122c8d3390 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.623+0000 7fc57cda9740 1 -- 172.21.15.18:0/1028781889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8d3390 msgr2=0x56122c8d5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.623+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1028781889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8d3390 0x56122c8d5780 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc574075320 tx=0x7fc56c014000 comp rx=0 tx=0).stop 2024-04-16T17:09:57.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.623+0000 7fc57cda9740 1 -- 172.21.15.18:0/1028781889 shutdown_connections 2024-04-16T17:09:57.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.623+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1028781889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8d3390 0x56122c8d5780 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.623+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1028781889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.623+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1028781889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8c9e90 0x56122c8ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.623+0000 7fc57cda9740 1 -- 172.21.15.18:0/1028781889 >> 172.21.15.18:0/1028781889 conn(0x56122c8c45c0 msgr2=0x56122c8c49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.624+0000 7fc57cda9740 1 -- 172.21.15.18:0/1028781889 shutdown_connections 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.624+0000 7fc57cda9740 1 -- 172.21.15.18:0/1028781889 wait complete. 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.624+0000 7fc57cda9740 1 -- start start 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.624+0000 7fc57cda9740 1 Processor -- start 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc57cda9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56122c8d6180 con 0x56122c8d3390 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc57cda9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56122c7e8730 con 0x56122c8c9e90 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc57cda9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56122c8d6300 con 0x56122c8ca7b0 2024-04-16T17:09:57.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc57cda9740 10 client.0 _ll_register_callbacks cb 0x56122c7b71e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc573e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc573e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41222/0 (socket says 172.21.15.18:41222) 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc573e00640 1 -- 172.21.15.18:0/4243919111 learned_addr learned my addr 172.21.15.18:0/4243919111 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc573400640 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.625+0000 7fc572a00640 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.626+0000 7fc573e00640 1 -- 172.21.15.18:0/4243919111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 msgr2=0x56122c8d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.626+0000 7fc573e00640 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.626+0000 7fc573e00640 1 -- 172.21.15.18:0/4243919111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 msgr2=0x56122c8d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.626+0000 7fc573e00640 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.626+0000 7fc573e00640 1 -- 172.21.15.18:0/4243919111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122c6097e0 con 0x56122c8d3390 2024-04-16T17:09:57.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.626+0000 7fc573e00640 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d0db0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc56c003f30 tx=0x7fc56c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d23976302c2c157 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.626+0000 7fc55cc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:09:57.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.627+0000 7fc55cc00640 1 -- 172.21.15.18:0/4243919111 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c032060 con 0x56122c8d3390 2024-04-16T17:09:57.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.627+0000 7fc55cc00640 1 -- 172.21.15.18:0/4243919111 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc56c005410 con 0x56122c8d3390 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.627+0000 7fc57cda9740 1 -- 172.21.15.18:0/4243919111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56c004cc0 con 0x56122c8d3390 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.627+0000 7fc55cc00640 1 -- 172.21.15.18:0/4243919111 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc56c039050 con 0x56122c8d3390 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.627+0000 7fc57cda9740 10 client.16824 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.627+0000 7fc57cda9740 1 -- 172.21.15.18:0/4243919111 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56122c669f70 con 0x56122c8d3390 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc55cc00640 1 -- 172.21.15.18:0/4243919111 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56c004cc0 con 0x56122c8d3390 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc55cc00640 1 -- 172.21.15.18:0/4243919111 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc56c023030 con 0x56122c8d3390 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc57cda9740 10 client.16824 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc57cda9740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc57cda9740 20 client.16824 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56122c8e3530 0x56122c8e5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc57cda9740 4 client.16824 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc57cda9740 1 -- 172.21.15.18:0/4243919111 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56122c83c6d0 con 0x56122c8e3530 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.628+0000 7fc573400640 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56122c8e3530 0x56122c8e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.629+0000 7fc573400640 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56122c8e3530 0x56122c8e5920 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1cca8fa7fb4e99 server_cookie=2bdbdb972eec6eef in_seq=0 out_seq=0 2024-04-16T17:09:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.629+0000 7fc55cc00640 10 client.16824 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:57.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc55cc00640 1 -- 172.21.15.18:0/4243919111 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56122c83c6d0 con 0x56122c8e3530 2024-04-16T17:09:57.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc55cc00640 10 client.16824 handle_command_reply: tid=0 2024-04-16T17:09:57.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 client.16824 shutdown 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 -- 172.21.15.18:0/4243919111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 msgr2=0x56122c8d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d0db0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc56c003f30 tx=0x7fc56c003f60 comp rx=0 tx=0).stop 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 -- 172.21.15.18:0/4243919111 shutdown_connections 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56122c8e3530 0x56122c8e5920 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d0db0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4243919111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc55cc00640 0 client.16824 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 -- 172.21.15.18:0/4243919111 >> 172.21.15.18:0/4243919111 conn(0x56122c8c45c0 msgr2=0x56122c8c6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.630+0000 7fc57cda9740 1 -- 172.21.15.18:0/4243919111 shutdown_connections 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.631+0000 7fc57cda9740 1 -- 172.21.15.18:0/4243919111 wait complete. 2024-04-16T17:09:57.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.631+0000 7fc57cda9740 20 client.16824 trim_cache size 0 max 16384 2024-04-16T17:09:57.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.654+0000 7fc57cda9740 1 Processor -- start 2024-04-16T17:09:57.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.654+0000 7fc57cda9740 1 -- start start 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.654+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.654+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8ca7b0 0x56122c8c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.654+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8d3390 0x56122c8eb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc57cda9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b3f70 con 0x56122c8ca7b0 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc57cda9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b1840 con 0x56122c8c9e90 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc57cda9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741af690 con 0x56122c8d3390 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35516/0 (socket says 172.21.15.18:35516) 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573400640 1 -- 172.21.15.18:0/4230538531 learned_addr learned my addr 172.21.15.18:0/4230538531 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573e00640 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8d3390 0x56122c8eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc572a00640 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8ca7b0 0x56122c8c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573400640 1 -- 172.21.15.18:0/4230538531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8d3390 msgr2=0x56122c8eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573400640 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8d3390 0x56122c8eb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573400640 1 -- 172.21.15.18:0/4230538531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8ca7b0 msgr2=0x56122c8c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573400640 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8ca7b0 0x56122c8c33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.655+0000 7fc573400640 1 -- 172.21.15.18:0/4230538531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122c501170 con 0x56122c8c9e90 2024-04-16T17:09:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.656+0000 7fc573400640 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d2a60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc564098f70 tx=0x7fc5640bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a16d013171ae29 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.656+0000 7fc55f400640 1 -- 172.21.15.18:0/4230538531 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56409ae10 con 0x56122c8c9e90 2024-04-16T17:09:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.656+0000 7fc55f400640 1 -- 172.21.15.18:0/4230538531 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5640bd0b0 con 0x56122c8c9e90 2024-04-16T17:09:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.656+0000 7fc55f400640 1 -- 172.21.15.18:0/4230538531 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56409bf60 con 0x56122c8c9e90 2024-04-16T17:09:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 -- 172.21.15.18:0/4230538531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 msgr2=0x56122c8d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d2a60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc564098f70 tx=0x7fc5640bd040 comp rx=0 tx=0).stop 2024-04-16T17:09:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 -- 172.21.15.18:0/4230538531 shutdown_connections 2024-04-16T17:09:57.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8d3390 0x56122c8eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8ca7b0 0x56122c8c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 --2- 172.21.15.18:0/4230538531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8d2a60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 -- 172.21.15.18:0/4230538531 >> 172.21.15.18:0/4230538531 conn(0x56122c8c45c0 msgr2=0x56122c8c2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 -- 172.21.15.18:0/4230538531 shutdown_connections 2024-04-16T17:09:57.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.657+0000 7fc57cda9740 1 -- 172.21.15.18:0/4230538531 wait complete. 2024-04-16T17:09:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.658+0000 7fc57cda9740 1 -- start start 2024-04-16T17:09:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.658+0000 7fc57cda9740 1 Processor -- start 2024-04-16T17:09:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc57cda9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc57cda9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741afd00 con 0x56122c8d3390 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc57cda9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b1170 con 0x56122c8c9e90 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc57cda9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b03d0 con 0x56122c8ca7b0 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc57cda9740 10 client.0 _ll_register_callbacks cb 0x7fc564005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc573400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc573400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8db9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35518/0 (socket says 172.21.15.18:35518) 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc573400640 1 -- 172.21.15.18:0/1567480512 learned_addr learned my addr 172.21.15.18:0/1567480512 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc572a00640 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc573e00640 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.659+0000 7fc573400640 1 -- 172.21.15.18:0/1567480512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 msgr2=0x56122c8d8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc573400640 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc573400640 1 -- 172.21.15.18:0/1567480512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 msgr2=0x56122c8d92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc573400640 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc573400640 1 -- 172.21.15.18:0/1567480512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122c8d9b20 con 0x56122c8c9e90 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc573e00640 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc573400640 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8db9e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc56409caa0 tx=0x7fc56409d210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca73ade3b78f9fd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc55cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc55cc00640 1 -- 172.21.15.18:0/1567480512 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564066060 con 0x56122c8c9e90 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc55cc00640 1 -- 172.21.15.18:0/1567480512 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fc5640ce540 con 0x56122c8c9e90 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc57cda9740 1 -- 172.21.15.18:0/1567480512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5640bd0b0 con 0x56122c8c9e90 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc55cc00640 1 -- 172.21.15.18:0/1567480512 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5640ce740 con 0x56122c8c9e90 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc57cda9740 10 client.27062 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.660+0000 7fc57cda9740 1 -- 172.21.15.18:0/1567480512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56122c949ab0 con 0x56122c8c9e90 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc55cc00640 1 -- 172.21.15.18:0/1567480512 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5640bd0b0 con 0x56122c8c9e90 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc55cc00640 1 -- 172.21.15.18:0/1567480512 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc564098a20 con 0x56122c8c9e90 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc57cda9740 10 client.27062 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:57.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc57cda9740 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc57cda9740 20 client.27062 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56122c94a0c0 0x56122c94c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc57cda9740 4 client.27062 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:09:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc57cda9740 1 -- 172.21.15.18:0/1567480512 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56122c836540 con 0x56122c94a0c0 2024-04-16T17:09:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.661+0000 7fc572a00640 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56122c94a0c0 0x56122c94c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.662+0000 7fc572a00640 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56122c94a0c0 0x56122c94c4b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f8f86daf1806483 server_cookie=ffb318100f193eb1 in_seq=0 out_seq=0 2024-04-16T17:09:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.662+0000 7fc55cc00640 10 client.27062 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc55cc00640 1 -- 172.21.15.18:0/1567480512 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56122c836540 con 0x56122c94a0c0 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc55cc00640 10 client.27062 handle_command_reply: tid=0 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 client.27062 shutdown 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 -- 172.21.15.18:0/1567480512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 msgr2=0x56122c8db9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8db9e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc56409caa0 tx=0x7fc56409d210 comp rx=0 tx=0).stop 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 -- 172.21.15.18:0/1567480512 shutdown_connections 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56122c94a0c0 0x56122c94c4b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56122c8d3390 0x56122c8d92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56122c8ca7b0 0x56122c8d8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 --2- 172.21.15.18:0/1567480512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56122c8c9e90 0x56122c8db9e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.663+0000 7fc57cda9740 1 -- 172.21.15.18:0/1567480512 >> 172.21.15.18:0/1567480512 conn(0x56122c8c45c0 msgr2=0x56122c8c3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.664+0000 7fc55cc00640 0 client.27062 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.664+0000 7fc57cda9740 1 -- 172.21.15.18:0/1567480512 shutdown_connections 2024-04-16T17:09:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.664+0000 7fc57cda9740 1 -- 172.21.15.18:0/1567480512 wait complete. 2024-04-16T17:09:57.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.664+0000 7fc57cda9740 20 client.27062 trim_cache size 0 max 16384 2024-04-16T17:09:57.667 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:09:57.667 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:09:57.667 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "09712c08-e701-4b51-bd75-eda41f4a65c0", 2024-04-16T17:09:57.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:09:57.667 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:09:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 -- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5580a3d20 msgr2=0x7fc5580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5580a3d20 0x7fc5580a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc574069e30 tx=0x7fc56c009500 comp rx=0 tx=0).stop 2024-04-16T17:09:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 -- 172.21.15.18:0/840423666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5740743d0 msgr2=0x7fc5741a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5740743d0 0x7fc5741a0650 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc56400ea60 tx=0x7fc56400ef20 comp rx=0 tx=0).stop 2024-04-16T17:09:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 -- 172.21.15.18:0/840423666 shutdown_connections 2024-04-16T17:09:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5580a3d20 0x7fc5580a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc57410e6a0 0x7fc57419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574074cf0 0x7fc5741a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 --2- 172.21.15.18:0/840423666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5740743d0 0x7fc5741a0650 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 -- 172.21.15.18:0/840423666 >> 172.21.15.18:0/840423666 conn(0x7fc5740fd950 msgr2=0x7fc5740fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 -- 172.21.15.18:0/840423666 shutdown_connections 2024-04-16T17:09:57.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.665+0000 7fc578a00640 1 -- 172.21.15.18:0/840423666 wait complete. 2024-04-16T17:09:57.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:09:57.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 -- 172.21.15.18:0/3995948329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba00102070 msgr2=0x7fba00102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 --2- 172.21.15.18:0/3995948329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba00102070 0x7fba00102450 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009900 tx=0x7fb9f402bc50 comp rx=0 tx=0).stop 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 -- 172.21.15.18:0/3995948329 shutdown_connections 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 --2- 172.21.15.18:0/3995948329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba00107140 0x7fba0010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 --2- 172.21.15.18:0/3995948329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba00102a20 0x7fba00106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 --2- 172.21.15.18:0/3995948329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba00102070 0x7fba00102450 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 -- 172.21.15.18:0/3995948329 >> 172.21.15.18:0/3995948329 conn(0x7fba000fd950 msgr2=0x7fba000ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 -- 172.21.15.18:0/3995948329 shutdown_connections 2024-04-16T17:09:57.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.896+0000 7fba06e00640 1 -- 172.21.15.18:0/3995948329 wait complete. 2024-04-16T17:09:57.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.897+0000 7fba06e00640 1 Processor -- start 2024-04-16T17:09:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba06e00640 1 -- start start 2024-04-16T17:09:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba06e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba00102070 0x7fba001a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba06e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba00107140 0x7fba001a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba06e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba0019a860 0x7fba0019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba06e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba00112240 con 0x7fba0019a860 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba06e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba001120c0 con 0x7fba00107140 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba06e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba001123c0 con 0x7fba00102070 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba05a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba00102070 0x7fba001a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.898+0000 7fba05000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba00107140 0x7fba001a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba05000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba00107140 0x7fba001a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35530/0 (socket says 172.21.15.18:35530) 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba05000640 1 -- 172.21.15.18:0/3262166138 learned_addr learned my addr 172.21.15.18:0/3262166138 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba06400640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba0019a860 0x7fba0019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba05000640 1 -- 172.21.15.18:0/3262166138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba00102070 msgr2=0x7fba001a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba05000640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba00102070 0x7fba001a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba05000640 1 -- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba0019a860 msgr2=0x7fba0019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba05000640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba0019a860 0x7fba0019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba05000640 1 -- 172.21.15.18:0/3262166138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0019b3f0 con 0x7fba00107140 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba06400640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba0019a860 0x7fba0019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.899+0000 7fba05000640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba00107140 0x7fba001a0be0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00e9f0 tx=0x7fb9ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.900+0000 7fb9fe000640 1 -- 172.21.15.18:0/3262166138 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00ccc0 con 0x7fba00107140 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.900+0000 7fba06e00640 1 -- 172.21.15.18:0/3262166138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0019b6e0 con 0x7fba00107140 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.900+0000 7fb9fe000640 1 -- 172.21.15.18:0/3262166138 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec005500 con 0x7fba00107140 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.900+0000 7fb9fe000640 1 -- 172.21.15.18:0/3262166138 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec010690 con 0x7fba00107140 2024-04-16T17:09:57.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.900+0000 7fba06e00640 1 -- 172.21.15.18:0/3262166138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001a7870 con 0x7fba00107140 2024-04-16T17:09:57.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.905+0000 7fba0b1a0740 1 Processor -- start 2024-04-16T17:09:57.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.905+0000 7fba0b1a0740 1 -- start start 2024-04-16T17:09:57.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65467580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fb9fe000640 1 -- 172.21.15.18:0/3262166138 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb9ec004f20 con 0x7fba00107140 2024-04-16T17:09:57.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65478b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654790a0 0x559b6547b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559b653cd120 con 0x559b654790a0 2024-04-16T17:09:57.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559b65373340 con 0x559b654671a0 2024-04-16T17:09:57.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b653c90e0 con 0x559b65467b50 2024-04-16T17:09:57.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba05a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65478b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba05a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65478b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38460/0 (socket says 172.21.15.18:38460) 2024-04-16T17:09:57.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba05a00640 1 -- 172.21.15.18:0/2493280137 learned_addr learned my addr 172.21.15.18:0/2493280137 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.906+0000 7fba06400640 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65467580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.907+0000 7fba05000640 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654790a0 0x559b6547b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.907+0000 7fba05a00640 1 -- 172.21.15.18:0/2493280137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 msgr2=0x559b65467580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.907+0000 7fba05a00640 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65467580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.907+0000 7fba05a00640 1 -- 172.21.15.18:0/2493280137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654790a0 msgr2=0x559b6547b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.907+0000 7fba05a00640 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654790a0 0x559b6547b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.908+0000 7fba05a00640 1 -- 172.21.15.18:0/2493280137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b651a97d0 con 0x559b65467b50 2024-04-16T17:09:57.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.908+0000 7fba05000640 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654790a0 0x559b6547b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:57.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.908+0000 7fb9fe000640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9e80a3da0 0x7fb9e80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.908+0000 7fba05a00640 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65478b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4035000 tx=0x7fb9f40095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2148f0ebf25ecc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.908+0000 7fb9fe000640 1 -- 172.21.15.18:0/3262166138 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb9ec0d0f90 con 0x7fba00107140 2024-04-16T17:09:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.908+0000 7fb9f2a00640 1 -- 172.21.15.18:0/2493280137 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4004240 con 0x559b65467b50 2024-04-16T17:09:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.908+0000 7fb9f2a00640 1 -- 172.21.15.18:0/2493280137 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f402c6b0 con 0x559b65467b50 2024-04-16T17:09:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.908+0000 7fb9f2a00640 1 -- 172.21.15.18:0/2493280137 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40051c0 con 0x559b65467b50 2024-04-16T17:09:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.909+0000 7fba06400640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9e80a3da0 0x7fb9e80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.909+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2493280137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 msgr2=0x559b65478b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.909+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65478b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4035000 tx=0x7fb9f40095e0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.909+0000 7fba06400640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9e80a3da0 0x7fb9e80a6260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005fd0 tx=0x7fb9f8009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.910+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2493280137 shutdown_connections 2024-04-16T17:09:57.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.910+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654790a0 0x559b6547b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.910+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65478b60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.910+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2493280137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65467580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.910+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2493280137 >> 172.21.15.18:0/2493280137 conn(0x559b65461ac0 msgr2=0x559b65461ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.910+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2493280137 shutdown_connections 2024-04-16T17:09:57.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.910+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2493280137 wait complete. 2024-04-16T17:09:57.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.910+0000 7fba0b1a0740 1 -- start start 2024-04-16T17:09:57.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.911+0000 7fba0b1a0740 1 Processor -- start 2024-04-16T17:09:57.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.911+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65476100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.911+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65476640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.911+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b65476b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.911+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559b6547be00 con 0x559b65467b50 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.911+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559b65386710 con 0x559b654671a0 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.911+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b6547bf80 con 0x559b654790a0 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.911+0000 7fba0b1a0740 10 client.0 _ll_register_callbacks cb 0x559b65381af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba06400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b65476b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba06400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b65476b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38474/0 (socket says 172.21.15.18:38474) 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba06400640 1 -- 172.21.15.18:0/4141858244 learned_addr learned my addr 172.21.15.18:0/4141858244 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba05a00640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65476100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba05000640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65476640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba06400640 1 -- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 msgr2=0x559b65476100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba06400640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65476100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba06400640 1 -- 172.21.15.18:0/4141858244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 msgr2=0x559b65476640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba06400640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65476640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba06400640 1 -- 172.21.15.18:0/4141858244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b650df3f0 con 0x559b654790a0 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba05000640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65476640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.912+0000 7fba05a00640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65476100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.913+0000 7fba06400640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b65476b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8010b10 tx=0x7fb9f8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d1c0147bec19ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.913+0000 7fb9e7400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.913+0000 7fb9e7400640 1 -- 172.21.15.18:0/4141858244 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f801fc80 con 0x559b654790a0 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.913+0000 7fba0b1a0740 1 -- 172.21.15.18:0/4141858244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559b653dab40 con 0x559b654790a0 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.913+0000 7fb9e7400640 1 -- 172.21.15.18:0/4141858244 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb9f801c3f0 con 0x559b654790a0 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.913+0000 7fb9e7400640 1 -- 172.21.15.18:0/4141858244 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9f801fe20 con 0x559b654790a0 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.913+0000 7fba0b1a0740 10 client.27040 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.913+0000 7fba0b1a0740 1 -- 172.21.15.18:0/4141858244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559b6509f0f0 con 0x559b654790a0 2024-04-16T17:09:57.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.914+0000 7fb9e7400640 1 -- 172.21.15.18:0/4141858244 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559b653dab40 con 0x559b654790a0 2024-04-16T17:09:57.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.914+0000 7fb9e7400640 1 -- 172.21.15.18:0/4141858244 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9f8026070 con 0x559b654790a0 2024-04-16T17:09:57.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.915+0000 7fba0b1a0740 10 client.27040 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:57.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.915+0000 7fba0b1a0740 10 client.27040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:57.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.915+0000 7fba0b1a0740 20 client.27040 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:57.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.915+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b6546c410 0x559b6546e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.915+0000 7fba0b1a0740 4 client.27040 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.915+0000 7fba0b1a0740 1 -- 172.21.15.18:0/4141858244 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b653d49b0 con 0x559b6546c410 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.915+0000 7fba05a00640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b6546c410 0x559b6546e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.916+0000 7fba05a00640 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b6546c410 0x559b6546e860 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28fa5ac5f4d23c91 server_cookie=cbed7007f2f9dbfb in_seq=0 out_seq=0 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.916+0000 7fb9e7400640 10 client.27040 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.916+0000 7fb9e7400640 1 -- 172.21.15.18:0/4141858244 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b653d49b0 con 0x559b6546c410 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.916+0000 7fb9e7400640 10 client.27040 handle_command_reply: tid=0 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 client.27040 shutdown 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 -- 172.21.15.18:0/4141858244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 msgr2=0x559b65476b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b65476b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8010b10 tx=0x7fb9f8004ab0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 -- 172.21.15.18:0/4141858244 shutdown_connections 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b6546c410 0x559b6546e860 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b65476b80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65476640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/4141858244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65476100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fb9e7400640 0 client.27040 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:57.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 -- 172.21.15.18:0/4141858244 >> 172.21.15.18:0/4141858244 conn(0x559b65461ac0 msgr2=0x559b65463810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 -- 172.21.15.18:0/4141858244 shutdown_connections 2024-04-16T17:09:57.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 1 -- 172.21.15.18:0/4141858244 wait complete. 2024-04-16T17:09:57.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.917+0000 7fba0b1a0740 20 client.27040 trim_cache size 0 max 16384 2024-04-16T17:09:57.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:57 smithi022 ceph-mon[36023]: pgmap v159: 129 pgs: 129 active+clean; 43 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-04-16T17:09:57.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.948+0000 7fba0b1a0740 1 Processor -- start 2024-04-16T17:09:57.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.948+0000 7fba0b1a0740 1 -- start start 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65478830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65484d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b6549b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba001afa00 con 0x559b65467b50 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba001ad2d0 con 0x559b654671a0 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba001ab140 con 0x559b654790a0 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65478830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65484d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65478830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35560/0 (socket says 172.21.15.18:35560) 2024-04-16T17:09:57.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05a00640 1 -- 172.21.15.18:0/3601241134 learned_addr learned my addr 172.21.15.18:0/3601241134 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba06400640 1 --2- 172.21.15.18:0/3601241134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b6549b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05a00640 1 -- 172.21.15.18:0/3601241134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 msgr2=0x559b6549b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05a00640 1 --2- 172.21.15.18:0/3601241134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b6549b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05a00640 1 -- 172.21.15.18:0/3601241134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 msgr2=0x559b65484d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05a00640 1 --2- 172.21.15.18:0/3601241134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65484d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.949+0000 7fba05a00640 1 -- 172.21.15.18:0/3601241134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b65463d90 con 0x559b654671a0 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.950+0000 7fba05a00640 1 --2- 172.21.15.18:0/3601241134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65478830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4004a90 tx=0x7fb9f4004ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628757173cb19ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.950+0000 7fb9f2000640 1 -- 172.21.15.18:0/3601241134 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f402cc00 con 0x559b654671a0 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.950+0000 7fb9f2000640 1 -- 172.21.15.18:0/3601241134 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f4053110 con 0x559b654671a0 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.950+0000 7fb9f2000640 1 -- 172.21.15.18:0/3601241134 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4003850 con 0x559b654671a0 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 -- 172.21.15.18:0/3601241134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 msgr2=0x559b65478830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/3601241134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65478830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4004a90 tx=0x7fb9f4004ac0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 -- 172.21.15.18:0/3601241134 shutdown_connections 2024-04-16T17:09:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/3601241134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b654790a0 0x559b6549b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/3601241134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b65467b50 0x559b65484d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/3601241134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654671a0 0x559b65478830 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 -- 172.21.15.18:0/3601241134 >> 172.21.15.18:0/3601241134 conn(0x559b65461ac0 msgr2=0x559b65476950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 -- 172.21.15.18:0/3601241134 shutdown_connections 2024-04-16T17:09:57.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.951+0000 7fba0b1a0740 1 -- 172.21.15.18:0/3601241134 wait complete. 2024-04-16T17:09:57.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.952+0000 7fba0b1a0740 1 -- start start 2024-04-16T17:09:57.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.952+0000 7fba0b1a0740 1 Processor -- start 2024-04-16T17:09:57.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.952+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654671a0 0x559b654734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.952+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65473a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.952+0000 7fba0b1a0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654790a0 0x559b654f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.952+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba001ab790 con 0x559b654671a0 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.952+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba001acc00 con 0x559b654790a0 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.952+0000 7fba0b1a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba001abe60 con 0x559b65467b50 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba0b1a0740 10 client.0 _ll_register_callbacks cb 0x7fb9f8009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654671a0 0x559b654734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654671a0 0x559b654734d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41280/0 (socket says 172.21.15.18:41280) 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05a00640 1 -- 172.21.15.18:0/2330403217 learned_addr learned my addr 172.21.15.18:0/2330403217 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:09:57.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65473a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba06400640 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654790a0 0x559b654f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05000640 1 -- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654790a0 msgr2=0x559b654f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05000640 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654790a0 0x559b654f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05000640 1 -- 172.21.15.18:0/2330403217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654671a0 msgr2=0x559b654734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05000640 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654671a0 0x559b654734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05000640 1 -- 172.21.15.18:0/2330403217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b654f7110 con 0x559b65467b50 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba06400640 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654790a0 0x559b654f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.953+0000 7fba05a00640 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654671a0 0x559b654734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fba05000640 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65473a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0ce2d0 tx=0x7fb9ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=726ac1cf340ab071 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fb9e7400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fb9e7400640 1 -- 172.21.15.18:0/2330403217 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec09c660 con 0x559b65467b50 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fb9e7400640 1 -- 172.21.15.18:0/2330403217 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7fb9ec0046c0 con 0x559b65467b50 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2330403217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f4053110 con 0x559b65467b50 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fb9e7400640 1 -- 172.21.15.18:0/2330403217 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9ec0056b0 con 0x559b65467b50 2024-04-16T17:09:57.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fba0b1a0740 10 client.27046 fetch_fsmap learned FSMap version 25 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2330403217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559b654f83e0 con 0x559b65467b50 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.954+0000 7fb9e7400640 1 -- 172.21.15.18:0/2330403217 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f4053110 con 0x559b65467b50 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.955+0000 7fb9e7400640 1 -- 172.21.15.18:0/2330403217 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9ec0bbce0 con 0x559b65467b50 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.955+0000 7fba0b1a0740 10 client.27046 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.955+0000 7fba0b1a0740 10 client.27046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.955+0000 7fba0b1a0740 20 client.27046 populate_metadata read hostname 'smithi018' 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.955+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b654f8b60 0x559b654faf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.955+0000 7fba0b1a0740 4 client.27046 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.955+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2330403217 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b65350420 con 0x559b654f8b60 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.955+0000 7fba05a00640 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b654f8b60 0x559b654faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba05a00640 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b654f8b60 0x559b654faf80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90eb5a77d355eee2 server_cookie=b6ee6be162225c84 in_seq=0 out_seq=0 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fb9e7400640 10 client.27046 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fb9e7400640 1 -- 172.21.15.18:0/2330403217 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559b65350420 con 0x559b654f8b60 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fb9e7400640 10 client.27046 handle_command_reply: tid=0 2024-04-16T17:09:57.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 client.27046 shutdown 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2330403217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 msgr2=0x559b65473a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65473a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0ce2d0 tx=0x7fb9ec009780 comp rx=0 tx=0).stop 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2330403217 shutdown_connections 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b654f8b60 0x559b654faf80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b654790a0 0x559b654f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b65467b50 0x559b65473a10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 --2- 172.21.15.18:0/2330403217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b654671a0 0x559b654734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2330403217 >> 172.21.15.18:0/2330403217 conn(0x559b65461ac0 msgr2=0x559b65462d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fb9e7400640 0 client.27046 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2330403217 shutdown_connections 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.956+0000 7fba0b1a0740 1 -- 172.21.15.18:0/2330403217 wait complete. 2024-04-16T17:09:57.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.957+0000 7fba0b1a0740 20 client.27046 trim_cache size 0 max 16384 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout: "09712c08-e701-4b51-bd75-eda41f4a65c0": { 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "09712c08-e701-4b51-bd75-eda41f4a65c0", 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:09:57.962 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.957+0000 7fba06e00640 1 -- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9e80a3da0 msgr2=0x7fb9e80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.957+0000 7fba06e00640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9e80a3da0 0x7fb9e80a6260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005fd0 tx=0x7fb9f8009500 comp rx=0 tx=0).stop 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.957+0000 7fba06e00640 1 -- 172.21.15.18:0/3262166138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba00107140 msgr2=0x7fba001a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.957+0000 7fba06e00640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba00107140 0x7fba001a0be0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00e9f0 tx=0x7fb9ec00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.958+0000 7fba06e00640 1 -- 172.21.15.18:0/3262166138 shutdown_connections 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.958+0000 7fba06e00640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9e80a3da0 0x7fb9e80a6260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.958+0000 7fba06e00640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba0019a860 0x7fba0019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.958+0000 7fba06e00640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba00107140 0x7fba001a0be0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.958+0000 7fba06e00640 1 --2- 172.21.15.18:0/3262166138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba00102070 0x7fba001a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:57.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.958+0000 7fba06e00640 1 -- 172.21.15.18:0/3262166138 >> 172.21.15.18:0/3262166138 conn(0x7fba000fd950 msgr2=0x7fba000ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:57.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.958+0000 7fba06e00640 1 -- 172.21.15.18:0/3262166138 shutdown_connections 2024-04-16T17:09:57.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:09:57.958+0000 7fba06e00640 1 -- 172.21.15.18:0/3262166138 wait complete. 2024-04-16T17:09:57.978 INFO:tasks.cephfs.filesystem:scrub status for tag:09712c08-e701-4b51-bd75-eda41f4a65c0 - {'path': '/', 'tag': '09712c08-e701-4b51-bd75-eda41f4a65c0', 'options': 'recursive,force'} 2024-04-16T17:09:58.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:57 smithi043 ceph-mon[36901]: pgmap v159: 129 pgs: 129 active+clean; 43 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-04-16T17:09:58.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 189867 3.98 MB/sec execute 131 sec latency 60.091 ms 2024-04-16T17:09:58.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:58 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:09:58.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:58 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:58.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:58 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:09:58.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:58 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:59.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:58 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:09:59.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:58 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:59.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:58 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:09:59.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:58 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:59.564 INFO:tasks.workunit.client.0.smithi018.stdout: 1 190334 3.96 MB/sec execute 132 sec latency 126.415 ms 2024-04-16T17:09:59.903 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:59 smithi043 ceph-mon[36901]: pgmap v160: 129 pgs: 129 active+clean; 43 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-04-16T17:09:59.903 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:59 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:09:59.903 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:09:59 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:09:59.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:59 smithi022 ceph-mon[36023]: pgmap v160: 129 pgs: 129 active+clean; 43 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-04-16T17:09:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:59 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:09:59.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:09:59 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:10:00.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 191531 4.01 MB/sec execute 133 sec latency 75.105 ms 2024-04-16T17:10:00.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:00 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:10:00.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:00 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:10:00.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:00 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:10:00.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:00 smithi022 ceph-mon[36023]: overall HEALTH_OK 2024-04-16T17:10:01.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:00 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:10:01.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:00 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:10:01.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:00 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:10:01.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:00 smithi043 ceph-mon[36901]: overall HEALTH_OK 2024-04-16T17:10:01.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 192166 3.99 MB/sec execute 134 sec latency 25.329 ms 2024-04-16T17:10:01.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:01 smithi022 ceph-mon[36023]: pgmap v161: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:10:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:01 smithi043 ceph-mon[36901]: pgmap v161: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:10:02.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:57 smithi018 ceph-mon[31776]: pgmap v159: 129 pgs: 129 active+clean; 43 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-04-16T17:10:02.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:58 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:58 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:58 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:58 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:59 smithi018 ceph-mon[31776]: pgmap v160: 129 pgs: 129 active+clean; 43 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:59 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:09:59 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:00 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:00 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:00 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:00 smithi018 ceph-mon[31776]: overall HEALTH_OK 2024-04-16T17:10:02.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:01 smithi018 ceph-mon[31776]: pgmap v161: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:10:02.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 192577 3.98 MB/sec execute 135 sec latency 63.326 ms 2024-04-16T17:10:03.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 192930 3.97 MB/sec execute 136 sec latency 60.058 ms 2024-04-16T17:10:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 -- 172.21.15.18:0/3278943713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb8074cd0 msgr2=0x7efeb810e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 --2- 172.21.15.18:0/3278943713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb8074cd0 0x7efeb810e360 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efea4009860 tx=0x7efea402bc00 comp rx=0 tx=0).stop 2024-04-16T17:10:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 -- 172.21.15.18:0/3278943713 shutdown_connections 2024-04-16T17:10:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 --2- 172.21.15.18:0/3278943713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efeb810e8a0 0x7efeb8110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 --2- 172.21.15.18:0/3278943713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb8074cd0 0x7efeb810e360 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 --2- 172.21.15.18:0/3278943713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb80743b0 0x7efeb8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 -- 172.21.15.18:0/3278943713 >> 172.21.15.18:0/3278943713 conn(0x7efeb80fda90 msgr2=0x7efeb80ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 -- 172.21.15.18:0/3278943713 shutdown_connections 2024-04-16T17:10:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.801+0000 7efec0400640 1 -- 172.21.15.18:0/3278943713 wait complete. 2024-04-16T17:10:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.802+0000 7efec0400640 1 Processor -- start 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.802+0000 7efec0400640 1 -- start start 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efec0400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efeb80743b0 0x7efeb81a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efec0400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb8074cd0 0x7efeb81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efec0400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb810e8a0 0x7efeb819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efec0400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8114590 con 0x7efeb80743b0 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efec0400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8114410 con 0x7efeb810e8a0 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efec0400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8114710 con 0x7efeb8074cd0 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efebe400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb810e8a0 0x7efeb819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efebe400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb810e8a0 0x7efeb819a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35578/0 (socket says 172.21.15.18:35578) 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efebe400640 1 -- 172.21.15.18:0/3647119320 learned_addr learned my addr 172.21.15.18:0/3647119320 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efebda00640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efeb80743b0 0x7efeb81a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.803+0000 7efebd000640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb8074cd0 0x7efeb81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efebe400640 1 -- 172.21.15.18:0/3647119320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb8074cd0 msgr2=0x7efeb81a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efebe400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb8074cd0 0x7efeb81a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efebe400640 1 -- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efeb80743b0 msgr2=0x7efeb81a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efebe400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efeb80743b0 0x7efeb81a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efebe400640 1 -- 172.21.15.18:0/3647119320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb819b090 con 0x7efeb810e8a0 2024-04-16T17:10:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efebe400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb810e8a0 0x7efeb819a910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009fd0 tx=0x7efeb000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efeb6000640 1 -- 172.21.15.18:0/3647119320 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0004b20 con 0x7efeb810e8a0 2024-04-16T17:10:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efec0400640 1 -- 172.21.15.18:0/3647119320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb819b380 con 0x7efeb810e8a0 2024-04-16T17:10:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efeb6000640 1 -- 172.21.15.18:0/3647119320 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb0003e50 con 0x7efeb810e8a0 2024-04-16T17:10:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.804+0000 7efeb6000640 1 -- 172.21.15.18:0/3647119320 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb000b630 con 0x7efeb810e8a0 2024-04-16T17:10:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.805+0000 7efec0400640 1 -- 172.21.15.18:0/3647119320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb81a7620 con 0x7efeb810e8a0 2024-04-16T17:10:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.809+0000 7efec31a8740 1 Processor -- start 2024-04-16T17:10:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.809+0000 7efeb6000640 1 -- 172.21.15.18:0/3647119320 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efeb001d020 con 0x7efeb810e8a0 2024-04-16T17:10:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.809+0000 7efec31a8740 1 -- start start 2024-04-16T17:10:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efec31a8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e450360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efec31a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e450d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efec31a8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e45c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efec31a8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562f3e3b5780 con 0x562f3e459e20 2024-04-16T17:10:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efec31a8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562f3e35bf40 con 0x562f3e451610 2024-04-16T17:10:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efec31a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f3e3b1740 con 0x562f3e4508a0 2024-04-16T17:10:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efebd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e450360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efebd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e450360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35582/0 (socket says 172.21.15.18:35582) 2024-04-16T17:10:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efebd000640 1 -- 172.21.15.18:0/860413811 learned_addr learned my addr 172.21.15.18:0/860413811 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efebda00640 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e450d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.810+0000 7efebd000640 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e45c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efebda00640 1 -- 172.21.15.18:0/860413811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 msgr2=0x562f3e450360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efebda00640 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e450360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efebda00640 1 -- 172.21.15.18:0/860413811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 msgr2=0x562f3e45c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efebda00640 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e45c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efebda00640 1 -- 172.21.15.18:0/860413811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3e0c8210 con 0x562f3e4508a0 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efebda00640 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e450d20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efeac009bf0 tx=0x7efeac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c8a65f6123108df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efeaaa00640 1 -- 172.21.15.18:0/860413811 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac004050 con 0x562f3e4508a0 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efeaaa00640 1 -- 172.21.15.18:0/860413811 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac005500 con 0x562f3e4508a0 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efeb6000640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efea00a3d90 0x7efea00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.811+0000 7efeaaa00640 1 -- 172.21.15.18:0/860413811 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac004dd0 con 0x562f3e4508a0 2024-04-16T17:10:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efeb6000640 1 -- 172.21.15.18:0/3647119320 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7efeb00d0ad0 con 0x7efeb810e8a0 2024-04-16T17:10:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efec31a8740 1 -- 172.21.15.18:0/860413811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 msgr2=0x562f3e450d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efec31a8740 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e450d20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efeac009bf0 tx=0x7efeac011810 comp rx=0 tx=0).stop 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efec31a8740 1 -- 172.21.15.18:0/860413811 shutdown_connections 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efec31a8740 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e45c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efec31a8740 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e450d20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efec31a8740 1 --2- 172.21.15.18:0/860413811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e450360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efec31a8740 1 -- 172.21.15.18:0/860413811 >> 172.21.15.18:0/860413811 conn(0x562f3e3eb5d0 msgr2=0x562f3e44b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.812+0000 7efebd000640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efea00a3d90 0x7efea00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.813+0000 7efec31a8740 1 -- 172.21.15.18:0/860413811 shutdown_connections 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.813+0000 7efec31a8740 1 -- 172.21.15.18:0/860413811 wait complete. 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.813+0000 7efec31a8740 1 -- start start 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.813+0000 7efebd000640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efea00a3d90 0x7efea00a6250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x562f3e44c720 tx=0x7efea40058d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.813+0000 7efec31a8740 1 Processor -- start 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.814+0000 7efec31a8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e4508a0 0x562f3e4598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.814+0000 7efec31a8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e451610 0x562f3e456c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.814+0000 7efec31a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e459e20 0x562f3e4573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.814+0000 7efec31a8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562f3e45cb50 con 0x562f3e451610 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.814+0000 7efec31a8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562f3e36f390 con 0x562f3e4508a0 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.814+0000 7efec31a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f3e45ccd0 con 0x562f3e459e20 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.814+0000 7efec31a8740 10 client.0 _ll_register_callbacks cb 0x562f3e358f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebd000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e451610 0x562f3e456c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebd000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e451610 0x562f3e456c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41308/0 (socket says 172.21.15.18:41308) 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebd000640 1 -- 172.21.15.18:0/2321995752 learned_addr learned my addr 172.21.15.18:0/2321995752 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebda00640 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e4508a0 0x562f3e4598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebe400640 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e459e20 0x562f3e4573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebda00640 1 -- 172.21.15.18:0/2321995752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e459e20 msgr2=0x562f3e4573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebda00640 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e459e20 0x562f3e4573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebda00640 1 -- 172.21.15.18:0/2321995752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e451610 msgr2=0x562f3e456c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebda00640 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e451610 0x562f3e456c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.815+0000 7efebda00640 1 -- 172.21.15.18:0/2321995752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3e194620 con 0x562f3e4508a0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.816+0000 7efebda00640 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e4508a0 0x562f3e4598e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efeac000c00 tx=0x7efeac009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c51937a5e0c72100 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.816+0000 7efe9f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.816+0000 7efe9f400640 1 -- 172.21.15.18:0/2321995752 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac029070 con 0x562f3e4508a0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.816+0000 7efe9f400640 1 -- 172.21.15.18:0/2321995752 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7efeac004f20 con 0x562f3e4508a0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.816+0000 7efec31a8740 1 -- 172.21.15.18:0/2321995752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeac005500 con 0x562f3e4508a0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.817+0000 7efe9f400640 1 -- 172.21.15.18:0/2321995752 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efeac035050 con 0x562f3e4508a0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.817+0000 7efec31a8740 10 client.27089 fetch_fsmap learned FSMap version 25 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.817+0000 7efec31a8740 1 -- 172.21.15.18:0/2321995752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f3e468c70 con 0x562f3e4508a0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.817+0000 7efe9f400640 1 -- 172.21.15.18:0/2321995752 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac0046c0 con 0x562f3e4508a0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.817+0000 7efe9f400640 1 -- 172.21.15.18:0/2321995752 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efeac003cd0 con 0x562f3e4508a0 2024-04-16T17:10:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.818+0000 7efec31a8740 10 client.27089 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:10:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.818+0000 7efec31a8740 10 client.27089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.818+0000 7efec31a8740 20 client.27089 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.818+0000 7efec31a8740 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f3e46a020 0x562f3e46c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.818+0000 7efec31a8740 4 client.27089 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.818+0000 7efec31a8740 1 -- 172.21.15.18:0/2321995752 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efeac005500 con 0x562f3e46a020 2024-04-16T17:10:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.818+0000 7efebd000640 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f3e46a020 0x562f3e46c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.819+0000 7efe9f400640 10 client.27089 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.819+0000 7efebd000640 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f3e46a020 0x562f3e46c410 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa38fa8ddcb1d2d8 server_cookie=bbdf7855a92f98c3 in_seq=0 out_seq=0 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.819+0000 7efe9f400640 1 -- 172.21.15.18:0/2321995752 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7efeac005500 con 0x562f3e46a020 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.819+0000 7efe9f400640 10 client.27089 handle_command_reply: tid=0 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 client.27089 shutdown 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 -- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e4508a0 msgr2=0x562f3e4598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e4508a0 0x562f3e4598e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efeac000c00 tx=0x7efeac009fd0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 -- 172.21.15.18:0/2321995752 shutdown_connections 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f3e46a020 0x562f3e46c410 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e459e20 0x562f3e4573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e451610 0x562f3e456c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 --2- 172.21.15.18:0/2321995752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e4508a0 0x562f3e4598e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 -- 172.21.15.18:0/2321995752 >> 172.21.15.18:0/2321995752 conn(0x562f3e3eb5d0 msgr2=0x562f3e44d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efe9f400640 0 client.27089 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 -- 172.21.15.18:0/2321995752 shutdown_connections 2024-04-16T17:10:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 1 -- 172.21.15.18:0/2321995752 wait complete. 2024-04-16T17:10:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.820+0000 7efec31a8740 20 client.27089 trim_cache size 0 max 16384 2024-04-16T17:10:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.841+0000 7efec31a8740 1 Processor -- start 2024-04-16T17:10:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.841+0000 7efec31a8740 1 -- start start 2024-04-16T17:10:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.841+0000 7efec31a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e465a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.841+0000 7efec31a8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e4593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.841+0000 7efec31a8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e4598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.841+0000 7efec31a8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b4050 con 0x562f3e459e20 2024-04-16T17:10:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.841+0000 7efec31a8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b1920 con 0x562f3e451610 2024-04-16T17:10:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.841+0000 7efec31a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81af770 con 0x562f3e4508a0 2024-04-16T17:10:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e4593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e4593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35608/0 (socket says 172.21.15.18:35608) 2024-04-16T17:10:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebd000640 1 -- 172.21.15.18:0/4276453063 learned_addr learned my addr 172.21.15.18:0/4276453063 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebe400640 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e4598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebda00640 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e465a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebd000640 1 -- 172.21.15.18:0/4276453063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 msgr2=0x562f3e465a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebd000640 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e465a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebd000640 1 -- 172.21.15.18:0/4276453063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 msgr2=0x562f3e4598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebd000640 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e4598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.842+0000 7efebd000640 1 -- 172.21.15.18:0/4276453063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3e457720 con 0x562f3e451610 2024-04-16T17:10:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.847+0000 7efebd000640 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e4593a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efea4005e00 tx=0x7efea4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e95aa7d2d824446 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.847+0000 7efeaa000640 1 -- 172.21.15.18:0/4276453063 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea405a070 con 0x562f3e451610 2024-04-16T17:10:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.847+0000 7efeaa000640 1 -- 172.21.15.18:0/4276453063 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efea402c5a0 con 0x562f3e451610 2024-04-16T17:10:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.847+0000 7efeaa000640 1 -- 172.21.15.18:0/4276453063 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4039040 con 0x562f3e451610 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 -- 172.21.15.18:0/4276453063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 msgr2=0x562f3e4593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e4593a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efea4005e00 tx=0x7efea4005f90 comp rx=0 tx=0).stop 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 -- 172.21.15.18:0/4276453063 shutdown_connections 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e4598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e4593a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 --2- 172.21.15.18:0/4276453063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e465a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 -- 172.21.15.18:0/4276453063 >> 172.21.15.18:0/4276453063 conn(0x562f3e3eb5d0 msgr2=0x562f3e44b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 -- 172.21.15.18:0/4276453063 shutdown_connections 2024-04-16T17:10:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.848+0000 7efec31a8740 1 -- 172.21.15.18:0/4276453063 wait complete. 2024-04-16T17:10:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.849+0000 7efec31a8740 1 -- start start 2024-04-16T17:10:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.849+0000 7efec31a8740 1 Processor -- start 2024-04-16T17:10:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.849+0000 7efec31a8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e4621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.849+0000 7efec31a8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e453f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.849+0000 7efec31a8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e454450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.849+0000 7efec31a8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81afde0 con 0x562f3e459e20 2024-04-16T17:10:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.849+0000 7efec31a8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b1250 con 0x562f3e451610 2024-04-16T17:10:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.849+0000 7efec31a8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b04b0 con 0x562f3e4508a0 2024-04-16T17:10:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efec31a8740 10 client.0 _ll_register_callbacks cb 0x7efeac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e453f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e453f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35612/0 (socket says 172.21.15.18:35612) 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebd000640 1 -- 172.21.15.18:0/2643153954 learned_addr learned my addr 172.21.15.18:0/2643153954 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebda00640 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e4621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebe400640 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e454450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebd000640 1 -- 172.21.15.18:0/2643153954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 msgr2=0x562f3e4621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebd000640 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e4621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebd000640 1 -- 172.21.15.18:0/2643153954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 msgr2=0x562f3e454450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebd000640 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e454450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.850+0000 7efebd000640 1 -- 172.21.15.18:0/2643153954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3e454ac0 con 0x562f3e451610 2024-04-16T17:10:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.851+0000 7efebd000640 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e453f10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efea402bc40 tx=0x7efea4035e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac80316d768766e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.851+0000 7efe9f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.851+0000 7efe9f400640 1 -- 172.21.15.18:0/2643153954 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea405a070 con 0x562f3e451610 2024-04-16T17:10:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.851+0000 7efec31a8740 1 -- 172.21.15.18:0/2643153954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efea402c5a0 con 0x562f3e451610 2024-04-16T17:10:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.851+0000 7efe9f400640 1 -- 172.21.15.18:0/2643153954 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7efea402cb90 con 0x562f3e451610 2024-04-16T17:10:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.851+0000 7efe9f400640 1 -- 172.21.15.18:0/2643153954 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efea4039040 con 0x562f3e451610 2024-04-16T17:10:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.851+0000 7efec31a8740 10 client.27095 fetch_fsmap learned FSMap version 25 2024-04-16T17:10:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.851+0000 7efec31a8740 1 -- 172.21.15.18:0/2643153954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f3e4d0340 con 0x562f3e451610 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efe9f400640 1 -- 172.21.15.18:0/2643153954 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efea402c5a0 con 0x562f3e451610 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efe9f400640 1 -- 172.21.15.18:0/2643153954 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efea403f040 con 0x562f3e451610 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efec31a8740 10 client.27095 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efec31a8740 10 client.27095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efec31a8740 20 client.27095 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efec31a8740 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f3e4d08d0 0x562f3e4d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efec31a8740 4 client.27095 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efec31a8740 1 -- 172.21.15.18:0/2643153954 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f3e3c31a0 con 0x562f3e4d08d0 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efebda00640 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f3e4d08d0 0x562f3e4d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.852+0000 7efebda00640 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f3e4d08d0 0x562f3e4d2cf0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c3bbd5fdb48fed server_cookie=7b94adbbcde3335a in_seq=0 out_seq=0 2024-04-16T17:10:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.853+0000 7efe9f400640 10 client.27095 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:03.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:03 smithi022 ceph-mon[36023]: pgmap v162: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:10:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.996+0000 7efe9f400640 1 -- 172.21.15.18:0/2643153954 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f3e3c31a0 con 0x562f3e4d08d0 2024-04-16T17:10:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.996+0000 7efe9f400640 10 client.27095 handle_command_reply: tid=0 2024-04-16T17:10:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.996+0000 7efec31a8740 1 client.27095 shutdown 2024-04-16T17:10:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 -- 172.21.15.18:0/2643153954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 msgr2=0x562f3e453f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e453f10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efea402bc40 tx=0x7efea4035e50 comp rx=0 tx=0).stop 2024-04-16T17:10:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 -- 172.21.15.18:0/2643153954 shutdown_connections 2024-04-16T17:10:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f3e4d08d0 0x562f3e4d2cf0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f3e459e20 0x562f3e454450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f3e451610 0x562f3e453f10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 --2- 172.21.15.18:0/2643153954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f3e4508a0 0x562f3e4621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 -- 172.21.15.18:0/2643153954 >> 172.21.15.18:0/2643153954 conn(0x562f3e3eb5d0 msgr2=0x562f3e4561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efe9f400640 0 client.27095 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:10:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:10:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:10:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:10:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 -- 172.21.15.18:0/2643153954 shutdown_connections 2024-04-16T17:10:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 1 -- 172.21.15.18:0/2643153954 wait complete. 2024-04-16T17:10:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.997+0000 7efec31a8740 20 client.27095 trim_cache size 0 max 16384 2024-04-16T17:10:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 -- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efea00a3d90 msgr2=0x7efea00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efea00a3d90 0x7efea00a6250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x562f3e44c720 tx=0x7efea40058d0 comp rx=0 tx=0).stop 2024-04-16T17:10:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 -- 172.21.15.18:0/3647119320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb810e8a0 msgr2=0x7efeb819a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb810e8a0 0x7efeb819a910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009fd0 tx=0x7efeb000c800 comp rx=0 tx=0).stop 2024-04-16T17:10:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 -- 172.21.15.18:0/3647119320 shutdown_connections 2024-04-16T17:10:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efea00a3d90 0x7efea00a6250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efeb810e8a0 0x7efeb819a910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeb8074cd0 0x7efeb81a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 --2- 172.21.15.18:0/3647119320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efeb80743b0 0x7efeb81a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 -- 172.21.15.18:0/3647119320 >> 172.21.15.18:0/3647119320 conn(0x7efeb80fda90 msgr2=0x7efeb80ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 -- 172.21.15.18:0/3647119320 shutdown_connections 2024-04-16T17:10:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:03.999+0000 7efec0400640 1 -- 172.21.15.18:0/3647119320 wait complete. 2024-04-16T17:10:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:03 smithi043 ceph-mon[36901]: pgmap v162: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:10:04.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 193189 3.95 MB/sec execute 137 sec latency 72.669 ms 2024-04-16T17:10:05.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 194325 3.96 MB/sec execute 138 sec latency 27.675 ms 2024-04-16T17:10:05.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:05 smithi022 ceph-mon[36023]: pgmap v163: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:10:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:05 smithi043 ceph-mon[36901]: pgmap v163: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:10:06.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 195397 3.99 MB/sec execute 139 sec latency 12.297 ms 2024-04-16T17:10:07.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 196033 3.97 MB/sec execute 140 sec latency 18.687 ms 2024-04-16T17:10:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:07 smithi022 ceph-mon[36023]: pgmap v164: 129 pgs: 129 active+clean; 39 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.8 MiB/s wr, 255 op/s 2024-04-16T17:10:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:07 smithi043 ceph-mon[36901]: pgmap v164: 129 pgs: 129 active+clean; 39 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.8 MiB/s wr, 255 op/s 2024-04-16T17:10:08.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 196531 3.97 MB/sec execute 141 sec latency 119.508 ms 2024-04-16T17:10:09.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 196873 3.94 MB/sec execute 142 sec latency 77.436 ms 2024-04-16T17:10:09.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:03 smithi018 ceph-mon[31776]: pgmap v162: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:10:09.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:05 smithi018 ceph-mon[31776]: pgmap v163: 129 pgs: 129 active+clean; 45 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:10:09.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:07 smithi018 ceph-mon[31776]: pgmap v164: 129 pgs: 129 active+clean; 39 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.8 MiB/s wr, 255 op/s 2024-04-16T17:10:09.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:09 smithi022 ceph-mon[36023]: pgmap v165: 129 pgs: 129 active+clean; 39 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-04-16T17:10:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:09 smithi043 ceph-mon[36901]: pgmap v165: 129 pgs: 129 active+clean; 39 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-04-16T17:10:10.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 198123 3.97 MB/sec execute 143 sec latency 37.331 ms 2024-04-16T17:10:11.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 199076 3.98 MB/sec execute 144 sec latency 10.687 ms 2024-04-16T17:10:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:11 smithi022 ceph-mon[36023]: pgmap v166: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-04-16T17:10:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:11 smithi043 ceph-mon[36901]: pgmap v166: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-04-16T17:10:12.565 INFO:tasks.workunit.client.0.smithi018.stdout: 1 199877 3.97 MB/sec execute 145 sec latency 23.627 ms 2024-04-16T17:10:13.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 200418 3.96 MB/sec execute 146 sec latency 44.806 ms 2024-04-16T17:10:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:13 smithi022 ceph-mon[36023]: pgmap v167: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-16T17:10:13.972 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:09 smithi018 ceph-mon[31776]: pgmap v165: 129 pgs: 129 active+clean; 39 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-04-16T17:10:13.972 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:11 smithi018 ceph-mon[31776]: pgmap v166: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-04-16T17:10:13.972 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:13 smithi018 ceph-mon[31776]: pgmap v167: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-16T17:10:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:13 smithi043 ceph-mon[36901]: pgmap v167: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-16T17:10:14.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 200666 3.94 MB/sec execute 147 sec latency 120.028 ms 2024-04-16T17:10:15.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 201845 3.97 MB/sec execute 148 sec latency 29.939 ms 2024-04-16T17:10:15.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:15 smithi022 ceph-mon[36023]: pgmap v168: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-16T17:10:16.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:15 smithi043 ceph-mon[36901]: pgmap v168: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-16T17:10:16.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 202661 3.98 MB/sec execute 149 sec latency 9.719 ms 2024-04-16T17:10:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 -- 172.21.15.18:0/1143082218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a48074cc0 msgr2=0x7f7a481128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 --2- 172.21.15.18:0/1143082218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a48074cc0 0x7f7a481128f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009900 tx=0x7f7a3402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 -- 172.21.15.18:0/1143082218 shutdown_connections 2024-04-16T17:10:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 --2- 172.21.15.18:0/1143082218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a48112e30 0x7f7a48115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 --2- 172.21.15.18:0/1143082218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a48074cc0 0x7f7a481128f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 --2- 172.21.15.18:0/1143082218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a480743a0 0x7f7a48074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 -- 172.21.15.18:0/1143082218 >> 172.21.15.18:0/1143082218 conn(0x7f7a480fd910 msgr2=0x7f7a480ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 -- 172.21.15.18:0/1143082218 shutdown_connections 2024-04-16T17:10:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.159+0000 7f7a47200640 1 -- 172.21.15.18:0/1143082218 wait complete. 2024-04-16T17:10:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.160+0000 7f7a47200640 1 Processor -- start 2024-04-16T17:10:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.160+0000 7f7a47200640 1 -- start start 2024-04-16T17:10:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a47200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a480743a0 0x7f7a481a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a47200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a48074cc0 0x7f7a481a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a47200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a48112e30 0x7f7a4819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a47200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48118800 con 0x7f7a48112e30 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a47200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48118680 con 0x7f7a480743a0 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a47200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48118980 con 0x7f7a48074cc0 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a45400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a48074cc0 0x7f7a481a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a45e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a480743a0 0x7f7a481a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a45400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a48074cc0 0x7f7a481a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38734/0 (socket says 172.21.15.18:38734) 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a45400640 1 -- 172.21.15.18:0/201157548 learned_addr learned my addr 172.21.15.18:0/201157548 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.161+0000 7f7a46800640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a48112e30 0x7f7a4819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a45e00640 1 -- 172.21.15.18:0/201157548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a48074cc0 msgr2=0x7f7a481a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a45e00640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a48074cc0 0x7f7a481a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a45e00640 1 -- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a48112e30 msgr2=0x7f7a4819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a45e00640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a48112e30 0x7f7a4819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a45e00640 1 -- 172.21.15.18:0/201157548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4819f420 con 0x7f7a480743a0 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a45e00640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a480743a0 0x7f7a481a4a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e9f0 tx=0x7f7a3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a3ea00640 1 -- 172.21.15.18:0/201157548 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800ccc0 con 0x7f7a480743a0 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a3ea00640 1 -- 172.21.15.18:0/201157548 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a38010910 con 0x7f7a480743a0 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a47200640 1 -- 172.21.15.18:0/201157548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4819f710 con 0x7f7a480743a0 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.162+0000 7f7a47200640 1 -- 172.21.15.18:0/201157548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a481ab9d0 con 0x7f7a480743a0 2024-04-16T17:10:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.163+0000 7f7a3ea00640 1 -- 172.21.15.18:0/201157548 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38018960 con 0x7f7a480743a0 2024-04-16T17:10:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.165+0000 7f7a3ea00640 1 -- 172.21.15.18:0/201157548 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a38010ac0 con 0x7f7a480743a0 2024-04-16T17:10:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.166+0000 7f7a4fa6c740 1 Processor -- start 2024-04-16T17:10:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.166+0000 7f7a4fa6c740 1 -- start start 2024-04-16T17:10:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.166+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f17f0 0x55813d8f0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.166+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55813d855780 con 0x55813d8f0a80 2024-04-16T17:10:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55813d7fbf40 con 0x55813d8fa000 2024-04-16T17:10:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55813d851740 con 0x55813d8f17f0 2024-04-16T17:10:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a46800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f17f0 0x55813d8f0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a46800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f17f0 0x55813d8f0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38746/0 (socket says 172.21.15.18:38746) 2024-04-16T17:10:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a46800640 1 -- 172.21.15.18:0/2966017400 learned_addr learned my addr 172.21.15.18:0/2966017400 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a45400640 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a45e00640 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a3ea00640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a2c0a3d00 0x7f7a2c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a46800640 1 -- 172.21.15.18:0/2966017400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 msgr2=0x55813d8fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a46800640 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8fc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a46800640 1 -- 172.21.15.18:0/2966017400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 msgr2=0x55813d8f0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a46800640 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a46800640 1 -- 172.21.15.18:0/2966017400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813d568210 con 0x55813d8f17f0 2024-04-16T17:10:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a45e00640 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.168+0000 7f7a3ea00640 1 -- 172.21.15.18:0/201157548 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f7a380d0e90 con 0x7f7a480743a0 2024-04-16T17:10:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.167+0000 7f7a45400640 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.168+0000 7f7a46800640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a2c0a3d00 0x7f7a2c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.168+0000 7f7a46800640 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f17f0 0x55813d8f0540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7a40010470 tx=0x7f7a40010930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc82cc920c81bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.168+0000 7f7a33400640 1 -- 172.21.15.18:0/2966017400 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4001d400 con 0x55813d8f17f0 2024-04-16T17:10:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.168+0000 7f7a33400640 1 -- 172.21.15.18:0/2966017400 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a4001da80 con 0x55813d8f17f0 2024-04-16T17:10:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.168+0000 7f7a33400640 1 -- 172.21.15.18:0/2966017400 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4001c800 con 0x55813d8f17f0 2024-04-16T17:10:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2966017400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f17f0 msgr2=0x55813d8f0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f17f0 0x55813d8f0540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7a40010470 tx=0x7f7a40010930 comp rx=0 tx=0).stop 2024-04-16T17:10:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a46800640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a2c0a3d00 0x7f7a2c0a61c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7a40021000 tx=0x7f7a4001e000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2966017400 shutdown_connections 2024-04-16T17:10:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2966017400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f17f0 0x55813d8f0540 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2966017400 >> 172.21.15.18:0/2966017400 conn(0x55813d88b5d0 msgr2=0x55813d8eb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2966017400 shutdown_connections 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.169+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2966017400 wait complete. 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.170+0000 7f7a4fa6c740 1 -- start start 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.170+0000 7f7a4fa6c740 1 Processor -- start 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.170+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f0a80 0x55813d8f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.170+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f17f0 0x55813d8f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.170+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8f7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.170+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55813d8fcd30 con 0x55813d8f17f0 2024-04-16T17:10:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.170+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55813d80f390 con 0x55813d8fa000 2024-04-16T17:10:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.170+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55813d8fceb0 con 0x55813d8f0a80 2024-04-16T17:10:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a4fa6c740 10 client.0 _ll_register_callbacks cb 0x55813d7f8f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f0a80 0x55813d8f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f0a80 0x55813d8f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38756/0 (socket says 172.21.15.18:38756) 2024-04-16T17:10:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45e00640 1 -- 172.21.15.18:0/3547351152 learned_addr learned my addr 172.21.15.18:0/3547351152 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45400640 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f17f0 0x55813d8f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a46800640 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45e00640 1 -- 172.21.15.18:0/3547351152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 msgr2=0x55813d8f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45e00640 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8f7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45e00640 1 -- 172.21.15.18:0/3547351152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f17f0 msgr2=0x55813d8f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45e00640 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f17f0 0x55813d8f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.171+0000 7f7a45e00640 1 -- 172.21.15.18:0/3547351152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813d634620 con 0x55813d8f0a80 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a45e00640 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f0a80 0x55813d8f9a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7a380ce230 tx=0x7f7a3800cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4513462cd47d20d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a27e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a27e00640 1 -- 172.21.15.18:0/3547351152 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380bbde0 con 0x55813d8f0a80 2024-04-16T17:10:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a27e00640 1 -- 172.21.15.18:0/3547351152 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f7a3809bf20 con 0x55813d8f0a80 2024-04-16T17:10:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/3547351152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a4001da80 con 0x55813d8f0a80 2024-04-16T17:10:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a27e00640 1 -- 172.21.15.18:0/3547351152 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7a380a0dd0 con 0x55813d8f0a80 2024-04-16T17:10:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a4fa6c740 10 client.27076 fetch_fsmap learned FSMap version 25 2024-04-16T17:10:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/3547351152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55813d908d90 con 0x55813d8f0a80 2024-04-16T17:10:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.172+0000 7f7a27e00640 1 -- 172.21.15.18:0/3547351152 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a4001da80 con 0x55813d8f0a80 2024-04-16T17:10:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a27e00640 1 -- 172.21.15.18:0/3547351152 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7a38018be0 con 0x55813d8f0a80 2024-04-16T17:10:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a4fa6c740 10 client.27076 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:10:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a4fa6c740 10 client.27076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a4fa6c740 20 client.27076 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55813d90a240 0x55813d90c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a4fa6c740 4 client.27076 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/3547351152 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55813d8631a0 con 0x55813d90a240 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a45400640 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55813d90a240 0x55813d90c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a45400640 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55813d90a240 0x55813d90c630 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=815d69b03217ed1d server_cookie=df5b19922e6ed172 in_seq=0 out_seq=0 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.173+0000 7f7a27e00640 10 client.27076 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.174+0000 7f7a27e00640 1 -- 172.21.15.18:0/3547351152 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55813d8631a0 con 0x55813d90a240 2024-04-16T17:10:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.174+0000 7f7a27e00640 10 client.27076 handle_command_reply: tid=0 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.174+0000 7f7a4fa6c740 1 client.27076 shutdown 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/3547351152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f0a80 msgr2=0x55813d8f9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f0a80 0x55813d8f9a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7a380ce230 tx=0x7f7a3800cf20 comp rx=0 tx=0).stop 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/3547351152 shutdown_connections 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55813d90a240 0x55813d90c630 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8fa000 0x55813d8f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f17f0 0x55813d8f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/3547351152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8f0a80 0x55813d8f9a30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a27e00640 0 client.27076 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/3547351152 >> 172.21.15.18:0/3547351152 conn(0x55813d88b5d0 msgr2=0x55813d8ed4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/3547351152 shutdown_connections 2024-04-16T17:10:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/3547351152 wait complete. 2024-04-16T17:10:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.175+0000 7f7a4fa6c740 20 client.27076 trim_cache size 0 max 16384 2024-04-16T17:10:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.207+0000 7f7a4fa6c740 1 Processor -- start 2024-04-16T17:10:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.207+0000 7f7a4fa6c740 1 -- start start 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8ea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d911f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28012820 con 0x55813d8f0a80 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7a280100f0 con 0x55813d8f17f0 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2800df40 con 0x55813d8fa000 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a45400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a45e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a45400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8ea690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58538/0 (socket says 172.21.15.18:58538) 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a45e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38930/0 (socket says 172.21.15.18:38930) 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.208+0000 7f7a45400640 1 -- 172.21.15.18:0/2404705126 learned_addr learned my addr 172.21.15.18:0/2404705126 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.209+0000 7f7a46800640 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d911f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.209+0000 7f7a45400640 1 -- 172.21.15.18:0/2404705126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 msgr2=0x55813d911f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.209+0000 7f7a45400640 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d911f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.209+0000 7f7a45400640 1 -- 172.21.15.18:0/2404705126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 msgr2=0x55813d8f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.209+0000 7f7a45400640 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.209+0000 7f7a45400640 1 -- 172.21.15.18:0/2404705126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813d8f5330 con 0x55813d8f17f0 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.209+0000 7f7a45e00640 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.209+0000 7f7a45400640 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8ea690 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a34000c00 tx=0x7f7a3402e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9585cbd4affd4f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.210+0000 7f7a32a00640 1 -- 172.21.15.18:0/2404705126 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34004380 con 0x55813d8f17f0 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.210+0000 7f7a32a00640 1 -- 172.21.15.18:0/2404705126 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3404f070 con 0x55813d8f17f0 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.210+0000 7f7a32a00640 1 -- 172.21.15.18:0/2404705126 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a340051e0 con 0x55813d8f17f0 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2404705126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 msgr2=0x55813d8ea690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8ea690 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a34000c00 tx=0x7f7a3402e040 comp rx=0 tx=0).stop 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2404705126 shutdown_connections 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d911f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8ea690 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2404705126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2404705126 >> 172.21.15.18:0/2404705126 conn(0x55813d88b5d0 msgr2=0x55813d8e9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2404705126 shutdown_connections 2024-04-16T17:10:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2404705126 wait complete. 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.211+0000 7f7a4fa6c740 1 -- start start 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a4fa6c740 1 Processor -- start 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a4fa6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d96f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2800e5b0 con 0x55813d8f0a80 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2800fa20 con 0x55813d8f17f0 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a4fa6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2800ec80 con 0x55813d8fa000 2024-04-16T17:10:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a4fa6c740 10 client.0 _ll_register_callbacks cb 0x7f7a3809c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a45e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a45400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a46800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d96f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a45e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38938/0 (socket says 172.21.15.18:38938) 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.212+0000 7f7a45e00640 1 -- 172.21.15.18:0/2156218183 learned_addr learned my addr 172.21.15.18:0/2156218183 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.213+0000 7f7a46800640 1 -- 172.21.15.18:0/2156218183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 msgr2=0x55813d8f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.213+0000 7f7a46800640 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.213+0000 7f7a46800640 1 -- 172.21.15.18:0/2156218183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 msgr2=0x55813d8f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.213+0000 7f7a46800640 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.213+0000 7f7a46800640 1 -- 172.21.15.18:0/2156218183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813d96fcd0 con 0x55813d8fa000 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.213+0000 7f7a46800640 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d96f790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7a40010e30 tx=0x7f7a40014f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=527b0aa14e031263 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.213+0000 7f7a27e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.214+0000 7f7a27e00640 1 -- 172.21.15.18:0/2156218183 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40037070 con 0x55813d8fa000 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.214+0000 7f7a27e00640 1 -- 172.21.15.18:0/2156218183 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9466+0+0 (secure 0 0 0) 0x7f7a4001c5b0 con 0x55813d8fa000 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.214+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2156218183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a3404f070 con 0x55813d8fa000 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.214+0000 7f7a27e00640 1 -- 172.21.15.18:0/2156218183 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7a40043050 con 0x55813d8fa000 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.214+0000 7f7a4fa6c740 10 client.27082 fetch_fsmap learned FSMap version 25 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.214+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2156218183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55813d970e80 con 0x55813d8fa000 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.214+0000 7f7a27e00640 1 -- 172.21.15.18:0/2156218183 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3404f070 con 0x55813d8fa000 2024-04-16T17:10:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.215+0000 7f7a27e00640 1 -- 172.21.15.18:0/2156218183 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7a40030040 con 0x55813d8fa000 2024-04-16T17:10:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.215+0000 7f7a4fa6c740 10 client.27082 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:10:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.215+0000 7f7a4fa6c740 10 client.27082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.215+0000 7f7a4fa6c740 20 client.27082 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.215+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55813d9713a0 0x55813d973790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.215+0000 7f7a4fa6c740 4 client.27082 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.215+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2156218183 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55813d85d010 con 0x55813d9713a0 2024-04-16T17:10:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.215+0000 7f7a45e00640 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55813d9713a0 0x55813d973790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.216+0000 7f7a27e00640 10 client.27082 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.216+0000 7f7a45e00640 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55813d9713a0 0x55813d973790 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555139fa3a275006 server_cookie=8d7bd075c178bf22 in_seq=0 out_seq=0 2024-04-16T17:10:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a27e00640 1 -- 172.21.15.18:0/2156218183 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55813d85d010 con 0x55813d9713a0 2024-04-16T17:10:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a27e00640 10 client.27082 handle_command_reply: tid=0 2024-04-16T17:10:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 client.27082 shutdown 2024-04-16T17:10:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2156218183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 msgr2=0x55813d96f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d96f790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7a40010e30 tx=0x7f7a40014f00 comp rx=0 tx=0).stop 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2156218183 shutdown_connections 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55813d9713a0 0x55813d973790 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55813d8fa000 0x55813d96f790 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a27e00640 0 client.27082 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55813d8f17f0 0x55813d8f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 --2- 172.21.15.18:0/2156218183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55813d8f0a80 0x55813d8f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.376+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2156218183 >> 172.21.15.18:0/2156218183 conn(0x55813d88b5d0 msgr2=0x55813d8ea810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.377+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2156218183 shutdown_connections 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.377+0000 7f7a4fa6c740 1 -- 172.21.15.18:0/2156218183 wait complete. 2024-04-16T17:10:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.377+0000 7f7a4fa6c740 20 client.27082 trim_cache size 0 max 16384 2024-04-16T17:10:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:10:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:10:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:10:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:10:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 -- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a2c0a3d00 msgr2=0x7f7a2c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a2c0a3d00 0x7f7a2c0a61c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7a40021000 tx=0x7f7a4001e000 comp rx=0 tx=0).stop 2024-04-16T17:10:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 -- 172.21.15.18:0/201157548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a480743a0 msgr2=0x7f7a481a4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a480743a0 0x7f7a481a4a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e9f0 tx=0x7f7a3800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 -- 172.21.15.18:0/201157548 shutdown_connections 2024-04-16T17:10:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a2c0a3d00 0x7f7a2c0a61c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a48112e30 0x7f7a4819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a48074cc0 0x7f7a481a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 --2- 172.21.15.18:0/201157548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a480743a0 0x7f7a481a4a50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 -- 172.21.15.18:0/201157548 >> 172.21.15.18:0/201157548 conn(0x7f7a480fd910 msgr2=0x7f7a480ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 -- 172.21.15.18:0/201157548 shutdown_connections 2024-04-16T17:10:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:17.378+0000 7f7a47200640 1 -- 172.21.15.18:0/201157548 wait complete. 2024-04-16T17:10:17.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 203657 3.98 MB/sec execute 150 sec latency 22.263 ms 2024-04-16T17:10:17.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:17 smithi022 ceph-mon[36023]: pgmap v169: 129 pgs: 129 active+clean; 47 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-16T17:10:17.924 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:10:17 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setuser ceph since I am not root 2024-04-16T17:10:17.924 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:10:17 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setgroup ceph since I am not root 2024-04-16T17:10:17.924 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:10:17 smithi022 ceph-mds[66065]: -- 172.21.15.22:0/3896830614 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a400a9a40 con 0x556a4007c800 2024-04-16T17:10:17.925 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:10:17 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: starting mds.h at 2024-04-16T17:10:18.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:17 smithi043 ceph-mon[36901]: pgmap v169: 129 pgs: 129 active+clean; 47 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-16T17:10:18.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 204248 3.96 MB/sec execute 151 sec latency 52.272 ms 2024-04-16T17:10:18.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:18 smithi022 ceph-mon[36023]: daemon mds.h restarted 2024-04-16T17:10:18.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:18 smithi022 ceph-mon[36023]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T17:10:18.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:18 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6836/1162334131,v1:172.21.15.22:6837/1162334131] up:boot 2024-04-16T17:10:18.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:18 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:10:18.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:18 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:10:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:18 smithi043 ceph-mon[36901]: daemon mds.h restarted 2024-04-16T17:10:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:18 smithi043 ceph-mon[36901]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T17:10:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:18 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6836/1162334131,v1:172.21.15.22:6837/1162334131] up:boot 2024-04-16T17:10:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:18 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:10:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:18 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:10:19.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 204623 3.95 MB/sec execute 152 sec latency 80.780 ms 2024-04-16T17:10:19.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:19 smithi022 ceph-mon[36023]: pgmap v170: 129 pgs: 129 active+clean; 47 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-04-16T17:10:20.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:19 smithi043 ceph-mon[36901]: pgmap v170: 129 pgs: 129 active+clean; 47 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-04-16T17:10:20.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 205814 3.99 MB/sec execute 153 sec latency 49.016 ms 2024-04-16T17:10:21.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 206452 3.97 MB/sec execute 154 sec latency 17.770 ms 2024-04-16T17:10:21.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:21 smithi022 ceph-mon[36023]: pgmap v172: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:22.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:21 smithi043 ceph-mon[36901]: pgmap v172: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:22.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 207426 3.98 MB/sec execute 155 sec latency 13.974 ms 2024-04-16T17:10:23.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 208067 3.97 MB/sec execute 156 sec latency 52.343 ms 2024-04-16T17:10:23.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:23 smithi022 ceph-mon[36023]: pgmap v173: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:24.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:23 smithi043 ceph-mon[36901]: pgmap v173: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:24.567 INFO:tasks.workunit.client.0.smithi018.stdout: 1 208318 3.95 MB/sec execute 157 sec latency 60.167 ms 2024-04-16T17:10:25.567 INFO:tasks.workunit.client.0.smithi018.stdout: 1 209396 3.98 MB/sec execute 158 sec latency 52.706 ms 2024-04-16T17:10:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:25 smithi043 ceph-mon[36901]: pgmap v174: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:26.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:25 smithi022 ceph-mon[36023]: pgmap v174: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:26.488 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:15 smithi018 ceph-mon[31776]: pgmap v168: 129 pgs: 129 active+clean; 42 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-04-16T17:10:26.488 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:17 smithi018 ceph-mon[31776]: pgmap v169: 129 pgs: 129 active+clean; 47 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-16T17:10:26.488 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:18 smithi018 ceph-mon[31776]: daemon mds.h restarted 2024-04-16T17:10:26.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:18 smithi018 ceph-mon[31776]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T17:10:26.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:18 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6836/1162334131,v1:172.21.15.22:6837/1162334131] up:boot 2024-04-16T17:10:26.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:18 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:10:26.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:18 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:10:26.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:19 smithi018 ceph-mon[31776]: pgmap v170: 129 pgs: 129 active+clean; 47 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-04-16T17:10:26.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:21 smithi018 ceph-mon[31776]: pgmap v172: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:26.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:23 smithi018 ceph-mon[31776]: pgmap v173: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:26.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:25 smithi018 ceph-mon[31776]: pgmap v174: 129 pgs: 129 active+clean; 41 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-04-16T17:10:26.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 209936 3.97 MB/sec execute 159 sec latency 30.073 ms 2024-04-16T17:10:27.567 INFO:tasks.workunit.client.0.smithi018.stdout: 1 210862 3.98 MB/sec execute 160 sec latency 19.692 ms 2024-04-16T17:10:27.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:10:27.979 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:10:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:27 smithi043 ceph-mon[36901]: pgmap v175: 129 pgs: 129 active+clean; 41 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-04-16T17:10:28.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:27 smithi022 ceph-mon[36023]: pgmap v175: 129 pgs: 129 active+clean; 41 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-04-16T17:10:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.178+0000 7f5810c00640 1 -- 172.21.15.18:0/1864621369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c10e6c0 msgr2=0x7f580c110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.178+0000 7f5810c00640 1 --2- 172.21.15.18:0/1864621369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c10e6c0 0x7f580c110ab0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5804009be0 tx=0x7f580402bf80 comp rx=0 tx=0).stop 2024-04-16T17:10:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.178+0000 7f5810c00640 1 -- 172.21.15.18:0/1864621369 shutdown_connections 2024-04-16T17:10:28.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.178+0000 7f5810c00640 1 --2- 172.21.15.18:0/1864621369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c10e6c0 0x7f580c110ab0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.178+0000 7f5810c00640 1 --2- 172.21.15.18:0/1864621369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f580c074cd0 0x7f580c10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.178+0000 7f5810c00640 1 --2- 172.21.15.18:0/1864621369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c0743b0 0x7f580c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.178+0000 7f5810c00640 1 -- 172.21.15.18:0/1864621369 >> 172.21.15.18:0/1864621369 conn(0x7f580c0fd950 msgr2=0x7f580c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.178+0000 7f5810c00640 1 -- 172.21.15.18:0/1864621369 shutdown_connections 2024-04-16T17:10:28.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.179+0000 7f5810c00640 1 -- 172.21.15.18:0/1864621369 wait complete. 2024-04-16T17:10:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.180+0000 7f5810c00640 1 Processor -- start 2024-04-16T17:10:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.180+0000 7f5810c00640 1 -- start start 2024-04-16T17:10:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.180+0000 7f5810c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c0743b0 0x7f580c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.180+0000 7f5810c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c074cd0 0x7f580c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.180+0000 7f5810c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f580c10e6c0 0x7f580c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.180+0000 7f5810c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f580c114410 con 0x7f580c10e6c0 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.180+0000 7f5810c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f580c114290 con 0x7f580c074cd0 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.180+0000 7f5810c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f580c114590 con 0x7f580c0743b0 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c074cd0 0x7f580c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c074cd0 0x7f580c1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37012/0 (socket says 172.21.15.18:37012) 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580aa00640 1 -- 172.21.15.18:0/1683151078 learned_addr learned my addr 172.21.15.18:0/1683151078 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580be00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f580c10e6c0 0x7f580c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580b400640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c0743b0 0x7f580c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580be00640 1 -- 172.21.15.18:0/1683151078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c0743b0 msgr2=0x7f580c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580be00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c0743b0 0x7f580c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580be00640 1 -- 172.21.15.18:0/1683151078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c074cd0 msgr2=0x7f580c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580be00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c074cd0 0x7f580c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580be00640 1 -- 172.21.15.18:0/1683151078 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c19aff0 con 0x7f580c10e6c0 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.181+0000 7f580b400640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c0743b0 0x7f580c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.182+0000 7f580be00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f580c10e6c0 0x7f580c19a730 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f580402c4c0 tx=0x7f5804004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.182+0000 7f5803e00640 1 -- 172.21.15.18:0/1683151078 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804043020 con 0x7f580c10e6c0 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.182+0000 7f5803e00640 1 -- 172.21.15.18:0/1683151078 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58040045f0 con 0x7f580c10e6c0 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.182+0000 7f5810c00640 1 -- 172.21.15.18:0/1683151078 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c19b280 con 0x7f580c10e6c0 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.182+0000 7f5803e00640 1 -- 172.21.15.18:0/1683151078 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580403e6b0 con 0x7f580c10e6c0 2024-04-16T17:10:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.182+0000 7f5810c00640 1 -- 172.21.15.18:0/1683151078 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c1a75a0 con 0x7f580c10e6c0 2024-04-16T17:10:28.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.186+0000 7f5814f76740 1 Processor -- start 2024-04-16T17:10:28.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.187+0000 7f5803e00640 1 -- 172.21.15.18:0/1683151078 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5804046020 con 0x7f580c10e6c0 2024-04-16T17:10:28.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.187+0000 7f5814f76740 1 -- start start 2024-04-16T17:10:28.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f5814f76740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a80d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f5814f76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a816350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f5814f76740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a818c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f5814f76740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56194a773120 con 0x56194a816890 2024-04-16T17:10:28.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f5814f76740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56194a719340 con 0x56194a80d390 2024-04-16T17:10:28.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f5814f76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56194a76f0e0 con 0x56194a80dcb0 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f580aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a80d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f580aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a80d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37016/0 (socket says 172.21.15.18:37016) 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f580aa00640 1 -- 172.21.15.18:0/2862147420 learned_addr learned my addr 172.21.15.18:0/2862147420 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f580b400640 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a816350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.188+0000 7f580aa00640 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a818c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f5803e00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57f00a3d20 0x7f57f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f580aa00640 1 -- 172.21.15.18:0/2862147420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 msgr2=0x56194a816350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f580aa00640 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a816350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f580aa00640 1 -- 172.21.15.18:0/2862147420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 msgr2=0x56194a818c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f580aa00640 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a818c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f580aa00640 1 -- 172.21.15.18:0/2862147420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194a54f7d0 con 0x56194a80d390 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f5803e00640 1 -- 172.21.15.18:0/1683151078 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f58040f30a0 con 0x7f580c10e6c0 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f580aa00640 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a80d770 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009f80 tx=0x7f57f8009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb41c2a4279ff2db server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f580be00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57f00a3d20 0x7f57f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.189+0000 7f57f7e00640 1 -- 172.21.15.18:0/2862147420 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8010710 con 0x56194a80d390 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f57f7e00640 1 -- 172.21.15.18:0/2862147420 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8010d90 con 0x56194a80d390 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f57f7e00640 1 -- 172.21.15.18:0/2862147420 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f801dd80 con 0x56194a80d390 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f5814f76740 1 -- 172.21.15.18:0/2862147420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 msgr2=0x56194a80d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f5814f76740 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a80d770 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009f80 tx=0x7f57f8009790 comp rx=0 tx=0).stop 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f580be00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57f00a3d20 0x7f57f00a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58040bb770 tx=0x7f580402bf80 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f5814f76740 1 -- 172.21.15.18:0/2862147420 shutdown_connections 2024-04-16T17:10:28.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f5814f76740 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a818c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f5814f76740 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a816350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f5814f76740 1 --2- 172.21.15.18:0/2862147420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a80d770 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.190+0000 7f5814f76740 1 -- 172.21.15.18:0/2862147420 >> 172.21.15.18:0/2862147420 conn(0x56194a807ac0 msgr2=0x56194a807ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.191+0000 7f5814f76740 1 -- 172.21.15.18:0/2862147420 shutdown_connections 2024-04-16T17:10:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.191+0000 7f5814f76740 1 -- 172.21.15.18:0/2862147420 wait complete. 2024-04-16T17:10:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.191+0000 7f5814f76740 1 -- start start 2024-04-16T17:10:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.191+0000 7f5814f76740 1 Processor -- start 2024-04-16T17:10:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f5814f76740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a813960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f5814f76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a813ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f5814f76740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a8143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f5814f76740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56194a819680 con 0x56194a816890 2024-04-16T17:10:28.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f5814f76740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56194a72c710 con 0x56194a80d390 2024-04-16T17:10:28.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f5814f76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56194a819800 con 0x56194a80dcb0 2024-04-16T17:10:28.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f5814f76740 10 client.0 _ll_register_callbacks cb 0x56194a727af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:28.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f580aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a813960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f580aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a813960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37032/0 (socket says 172.21.15.18:37032) 2024-04-16T17:10:28.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f580aa00640 1 -- 172.21.15.18:0/2940115436 learned_addr learned my addr 172.21.15.18:0/2940115436 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.192+0000 7f580b400640 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a813ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.193+0000 7f580aa00640 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a8143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.193+0000 7f580b400640 1 -- 172.21.15.18:0/2940115436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 msgr2=0x56194a813960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.193+0000 7f580b400640 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a813960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.193+0000 7f580b400640 1 -- 172.21.15.18:0/2940115436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 msgr2=0x56194a8143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.193+0000 7f580b400640 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a8143e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.193+0000 7f580b400640 1 -- 172.21.15.18:0/2940115436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194a4853f0 con 0x56194a80dcb0 2024-04-16T17:10:28.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.193+0000 7f580b400640 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a813ea0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f57fc012cf0 tx=0x7f57fc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b6956b9d3d5a421 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.193+0000 7f57f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:10:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.194+0000 7f57f4c00640 1 -- 172.21.15.18:0/2940115436 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc012110 con 0x56194a80dcb0 2024-04-16T17:10:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.194+0000 7f5814f76740 1 -- 172.21.15.18:0/2940115436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f8010d90 con 0x56194a80dcb0 2024-04-16T17:10:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.194+0000 7f57f4c00640 1 -- 172.21.15.18:0/2940115436 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f57fc004030 con 0x56194a80dcb0 2024-04-16T17:10:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.194+0000 7f57f4c00640 1 -- 172.21.15.18:0/2940115436 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57fc0122b0 con 0x56194a80dcb0 2024-04-16T17:10:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.194+0000 7f5814f76740 10 client.27097 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.194+0000 7f5814f76740 1 -- 172.21.15.18:0/2940115436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56194a4450f0 con 0x56194a80dcb0 2024-04-16T17:10:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.194+0000 7f57f4c00640 1 -- 172.21.15.18:0/2940115436 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8010d90 con 0x56194a80dcb0 2024-04-16T17:10:28.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.195+0000 7f57f4c00640 1 -- 172.21.15.18:0/2940115436 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f57fc0104c0 con 0x56194a80dcb0 2024-04-16T17:10:28.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.195+0000 7f5814f76740 10 client.27097 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:28.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.195+0000 7f5814f76740 10 client.27097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:28.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.195+0000 7f5814f76740 20 client.27097 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.195+0000 7f5814f76740 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56194a826d20 0x56194a829140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.195+0000 7f5814f76740 4 client.27097 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.195+0000 7f5814f76740 1 -- 172.21.15.18:0/2940115436 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56194a780b40 con 0x56194a826d20 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.195+0000 7f580aa00640 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56194a826d20 0x56194a829140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.196+0000 7f580aa00640 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56194a826d20 0x56194a829140 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e9ece877db82d6 server_cookie=ef6be4f76b5e9dc4 in_seq=0 out_seq=0 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.196+0000 7f57f4c00640 10 client.27097 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.196+0000 7f57f4c00640 1 -- 172.21.15.18:0/2940115436 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56194a780b40 con 0x56194a826d20 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.196+0000 7f57f4c00640 10 client.27097 handle_command_reply: tid=0 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 client.27097 shutdown 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 -- 172.21.15.18:0/2940115436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 msgr2=0x56194a813ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a813ea0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f57fc012cf0 tx=0x7f57fc00f790 comp rx=0 tx=0).stop 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 -- 172.21.15.18:0/2940115436 shutdown_connections 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56194a826d20 0x56194a829140 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a8143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a813ea0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 --2- 172.21.15.18:0/2940115436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a813960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 -- 172.21.15.18:0/2940115436 >> 172.21.15.18:0/2940115436 conn(0x56194a807ac0 msgr2=0x56194a809810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f57f4c00640 0 client.27097 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:28.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 -- 172.21.15.18:0/2940115436 shutdown_connections 2024-04-16T17:10:28.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 1 -- 172.21.15.18:0/2940115436 wait complete. 2024-04-16T17:10:28.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.197+0000 7f5814f76740 20 client.27097 trim_cache size 0 max 16384 2024-04-16T17:10:28.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.216+0000 7f5814f76740 1 Processor -- start 2024-04-16T17:10:28.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.216+0000 7f5814f76740 1 -- start start 2024-04-16T17:10:28.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f5814f76740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a816090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f5814f76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a8128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f5814f76740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a822570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f5814f76740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1b3e70 con 0x56194a816890 2024-04-16T17:10:28.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f5814f76740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1b1740 con 0x56194a80d390 2024-04-16T17:10:28.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f5814f76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1af590 con 0x56194a80dcb0 2024-04-16T17:10:28.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f580aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a822570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f580aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a822570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44478/0 (socket says 172.21.15.18:44478) 2024-04-16T17:10:28.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f580aa00640 1 -- 172.21.15.18:0/1569587396 learned_addr learned my addr 172.21.15.18:0/1569587396 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.217+0000 7f580b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a8128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f580aa00640 1 -- 172.21.15.18:0/1569587396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 msgr2=0x56194a8128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f580aa00640 1 --2- 172.21.15.18:0/1569587396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a8128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f580aa00640 1 -- 172.21.15.18:0/1569587396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 msgr2=0x56194a816090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f580aa00640 1 --2- 172.21.15.18:0/1569587396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a816090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f580aa00640 1 -- 172.21.15.18:0/1569587396 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194a818ff0 con 0x56194a816890 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f580b400640 1 --2- 172.21.15.18:0/1569587396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a8128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f580aa00640 1 --2- 172.21.15.18:0/1569587396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a822570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f57f80125f0 tx=0x7f57f8009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9640b61a0be6572e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f57f7400640 1 -- 172.21.15.18:0/1569587396 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f801de40 con 0x56194a816890 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f57f7400640 1 -- 172.21.15.18:0/1569587396 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8005020 con 0x56194a816890 2024-04-16T17:10:28.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.218+0000 7f57f7400640 1 -- 172.21.15.18:0/1569587396 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f801d640 con 0x56194a816890 2024-04-16T17:10:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 -- 172.21.15.18:0/1569587396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 msgr2=0x56194a822570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 --2- 172.21.15.18:0/1569587396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a822570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f57f80125f0 tx=0x7f57f8009790 comp rx=0 tx=0).stop 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 -- 172.21.15.18:0/1569587396 shutdown_connections 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 --2- 172.21.15.18:0/1569587396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a816890 0x56194a822570 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 --2- 172.21.15.18:0/1569587396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a80dcb0 0x56194a8128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 --2- 172.21.15.18:0/1569587396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a816090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 -- 172.21.15.18:0/1569587396 >> 172.21.15.18:0/1569587396 conn(0x56194a807ac0 msgr2=0x56194a8141b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 -- 172.21.15.18:0/1569587396 shutdown_connections 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.219+0000 7f5814f76740 1 -- 172.21.15.18:0/1569587396 wait complete. 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.220+0000 7f5814f76740 1 -- start start 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.220+0000 7f5814f76740 1 Processor -- start 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.220+0000 7f5814f76740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a8107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.220+0000 7f5814f76740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a80dcb0 0x56194a810cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.220+0000 7f5814f76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a816890 0x56194a811230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.220+0000 7f5814f76740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1afc00 con 0x56194a80dcb0 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.220+0000 7f5814f76740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1b1070 con 0x56194a80d390 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.220+0000 7f5814f76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1b02d0 con 0x56194a816890 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f5814f76740 10 client.0 _ll_register_callbacks cb 0x7f57fc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a80dcb0 0x56194a810cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a80dcb0 0x56194a810cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44488/0 (socket says 172.21.15.18:44488) 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 -- 172.21.15.18:0/1099060638 learned_addr learned my addr 172.21.15.18:0/1099060638 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 -- 172.21.15.18:0/1099060638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a816890 msgr2=0x56194a811230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a816890 0x56194a811230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 -- 172.21.15.18:0/1099060638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 msgr2=0x56194a8107b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a8107b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 -- 172.21.15.18:0/1099060638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194a894c00 con 0x56194a80dcb0 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f580b400640 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a80dcb0 0x56194a810cf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x56194a8131c0 tx=0x7f57fc010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54195e1d4ab74fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.221+0000 7f57f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.222+0000 7f57f4c00640 1 -- 172.21.15.18:0/1099060638 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0128a0 con 0x56194a80dcb0 2024-04-16T17:10:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.222+0000 7f57f4c00640 1 -- 172.21.15.18:0/1099060638 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f57fc0052e0 con 0x56194a80dcb0 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.222+0000 7f5814f76740 1 -- 172.21.15.18:0/1099060638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f8005020 con 0x56194a80dcb0 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.222+0000 7f57f4c00640 1 -- 172.21.15.18:0/1099060638 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57fc0054e0 con 0x56194a80dcb0 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.222+0000 7f5814f76740 10 client.16899 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.222+0000 7f5814f76740 1 -- 172.21.15.18:0/1099060638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56194a895f00 con 0x56194a80dcb0 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.223+0000 7f57f4c00640 1 -- 172.21.15.18:0/1099060638 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8005020 con 0x56194a80dcb0 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.223+0000 7f57f4c00640 1 -- 172.21.15.18:0/1099060638 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f57fc0294c0 con 0x56194a80dcb0 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.223+0000 7f5814f76740 10 client.16899 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.223+0000 7f5814f76740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.223+0000 7f5814f76740 20 client.16899 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.223+0000 7f5814f76740 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56194a8966d0 0x56194a898af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.223+0000 7f5814f76740 4 client.16899 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:10:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.223+0000 7f5814f76740 1 -- 172.21.15.18:0/1099060638 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56194a77a9b0 con 0x56194a8966d0 2024-04-16T17:10:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.231+0000 7f580aa00640 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56194a8966d0 0x56194a898af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.253+0000 7f580aa00640 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56194a8966d0 0x56194a898af0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4d5022c85e243 server_cookie=f61291bd21f5a62d in_seq=0 out_seq=0 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.253+0000 7f57f4c00640 10 client.16899 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f57f4c00640 1 -- 172.21.15.18:0/1099060638 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56194a77a9b0 con 0x56194a8966d0 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f57f4c00640 10 client.16899 handle_command_reply: tid=0 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 client.16899 shutdown 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 -- 172.21.15.18:0/1099060638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a80dcb0 msgr2=0x56194a810cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a80dcb0 0x56194a810cf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x56194a8131c0 tx=0x7f57fc010660 comp rx=0 tx=0).stop 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 -- 172.21.15.18:0/1099060638 shutdown_connections 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56194a8966d0 0x56194a898af0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194a816890 0x56194a811230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56194a80dcb0 0x56194a810cf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 --2- 172.21.15.18:0/1099060638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56194a80d390 0x56194a8107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 -- 172.21.15.18:0/1099060638 >> 172.21.15.18:0/1099060638 conn(0x56194a807ac0 msgr2=0x56194a806290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f57f4c00640 0 client.16899 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:28.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 -- 172.21.15.18:0/1099060638 shutdown_connections 2024-04-16T17:10:28.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.254+0000 7f5814f76740 1 -- 172.21.15.18:0/1099060638 wait complete. 2024-04-16T17:10:28.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5814f76740 20 client.16899 trim_cache size 0 max 16384 2024-04-16T17:10:28.256 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:10:28.257 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:10:28.257 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:10:28.257 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:10:28.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 -- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57f00a3d20 msgr2=0x7f57f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57f00a3d20 0x7f57f00a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58040bb770 tx=0x7f580402bf80 comp rx=0 tx=0).stop 2024-04-16T17:10:28.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 -- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f580c10e6c0 msgr2=0x7f580c19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f580c10e6c0 0x7f580c19a730 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f580402c4c0 tx=0x7f5804004270 comp rx=0 tx=0).stop 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 -- 172.21.15.18:0/1683151078 shutdown_connections 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57f00a3d20 0x7f57f00a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f580c10e6c0 0x7f580c19a730 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f580c074cd0 0x7f580c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 --2- 172.21.15.18:0/1683151078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c0743b0 0x7f580c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.255+0000 7f5810c00640 1 -- 172.21.15.18:0/1683151078 >> 172.21.15.18:0/1683151078 conn(0x7f580c0fd950 msgr2=0x7f580c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.256+0000 7f5810c00640 1 -- 172.21.15.18:0/1683151078 shutdown_connections 2024-04-16T17:10:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.256+0000 7f5810c00640 1 -- 172.21.15.18:0/1683151078 wait complete. 2024-04-16T17:10:28.278 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:10:28.278 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:10:28.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.461+0000 7f7e73e00640 1 -- 172.21.15.18:0/1435930076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e741039b0 msgr2=0x7f7e7410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.461+0000 7f7e73e00640 1 --2- 172.21.15.18:0/1435930076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e741039b0 0x7f7e7410be90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009900 tx=0x7f7e5c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.461+0000 7f7e73e00640 1 -- 172.21.15.18:0/1435930076 shutdown_connections 2024-04-16T17:10:28.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.461+0000 7f7e73e00640 1 --2- 172.21.15.18:0/1435930076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e7410c4b0 0x7f7e7410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.461+0000 7f7e73e00640 1 --2- 172.21.15.18:0/1435930076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e741039b0 0x7f7e7410be90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.461+0000 7f7e73e00640 1 --2- 172.21.15.18:0/1435930076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e74103090 0x7f7e74103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.461+0000 7f7e73e00640 1 -- 172.21.15.18:0/1435930076 >> 172.21.15.18:0/1435930076 conn(0x7f7e740fc890 msgr2=0x7f7e740fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.462+0000 7f7e73e00640 1 -- 172.21.15.18:0/1435930076 shutdown_connections 2024-04-16T17:10:28.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.462+0000 7f7e73e00640 1 -- 172.21.15.18:0/1435930076 wait complete. 2024-04-16T17:10:28.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.463+0000 7f7e73e00640 1 Processor -- start 2024-04-16T17:10:28.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.463+0000 7f7e73e00640 1 -- start start 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e74103090 0x7f7e741a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e741039b0 0x7f7e741a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e7410c4b0 0x7f7e7419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e73e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7e741122c0 con 0x7f7e74103090 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e73e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74112140 con 0x7f7e7410c4b0 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e73e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74112440 con 0x7f7e741039b0 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e72a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e74103090 0x7f7e741a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e72a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e74103090 0x7f7e741a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44494/0 (socket says 172.21.15.18:44494) 2024-04-16T17:10:28.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e72a00640 1 -- 172.21.15.18:0/460222733 learned_addr learned my addr 172.21.15.18:0/460222733 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e72a00640 1 -- 172.21.15.18:0/460222733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e741039b0 msgr2=0x7f7e741a0bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:10:28.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e72a00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e741039b0 0x7f7e741a0bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e72a00640 1 -- 172.21.15.18:0/460222733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e7410c4b0 msgr2=0x7f7e7419a820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:10:28.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e72a00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e7410c4b0 0x7f7e7419a820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.464+0000 7f7e72a00640 1 -- 172.21.15.18:0/460222733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7419aff0 con 0x7f7e74103090 2024-04-16T17:10:28.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.465+0000 7f7e72a00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e74103090 0x7f7e741a06b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400e9f0 tx=0x7f7e6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.465+0000 7f7e6b400640 1 -- 172.21.15.18:0/460222733 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400cd80 con 0x7f7e74103090 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.465+0000 7f7e6b400640 1 -- 172.21.15.18:0/460222733 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e64010910 con 0x7f7e74103090 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.465+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7419b2e0 con 0x7f7e74103090 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.465+0000 7f7e6b400640 1 -- 172.21.15.18:0/460222733 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64018990 con 0x7f7e74103090 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.465+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e741a7500 con 0x7f7e74103090 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.466+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e74069be0 con 0x7f7e74103090 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.469+0000 7f7e6b400640 1 -- 172.21.15.18:0/460222733 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e64010ac0 con 0x7f7e74103090 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.471+0000 7f7e6b400640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7e580a3d90 0x7f7e580a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.471+0000 7f7e6b400640 1 -- 172.21.15.18:0/460222733 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f7e640d13f0 con 0x7f7e74103090 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.471+0000 7f7e6be00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7e580a3d90 0x7f7e580a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.472+0000 7f7e6be00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7e580a3d90 0x7f7e580a6250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c036040 tx=0x7f7e5c005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.474+0000 7f7e6b400640 1 -- 172.21.15.18:0/460222733 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e6409d680 con 0x7f7e74103090 2024-04-16T17:10:28.566 INFO:tasks.workunit.client.0.smithi018.stdout: 1 211378 3.95 MB/sec execute 161 sec latency 52.271 ms 2024-04-16T17:10:28.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.722+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e7404fe00 con 0x7f7e74103090 2024-04-16T17:10:28.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.723+0000 7f7e6b400640 1 -- 172.21.15.18:0/460222733 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11854 (secure 0 0 0) 0x7f7e6409cf80 con 0x7f7e74103090 2024-04-16T17:10:28.724 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:10:28.725 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:10:18.339186+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27119":{"gid":27119,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/1162334131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1162334131},{"type":"v1","addr":"172.21.15.22:6837","nonce":1162334131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:10:28.725 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 26 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7e580a3d90 msgr2=0x7f7e580a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7e580a3d90 0x7f7e580a6250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c036040 tx=0x7f7e5c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e74103090 msgr2=0x7f7e741a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e74103090 0x7f7e741a06b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400e9f0 tx=0x7f7e6400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 shutdown_connections 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7e580a3d90 0x7f7e580a6250 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e7410c4b0 0x7f7e7419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e741039b0 0x7f7e741a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 --2- 172.21.15.18:0/460222733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e74103090 0x7f7e741a06b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 >> 172.21.15.18:0/460222733 conn(0x7f7e740fc890 msgr2=0x7f7e740fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 shutdown_connections 2024-04-16T17:10:28.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.726+0000 7f7e73e00640 1 -- 172.21.15.18:0/460222733 wait complete. 2024-04-16T17:10:28.745 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:10:28.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 -- 172.21.15.18:0/2794284769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc108720 msgr2=0x7f11cc108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 --2- 172.21.15.18:0/2794284769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc108720 0x7f11cc108b80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009900 tx=0x7f11b402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 -- 172.21.15.18:0/2794284769 shutdown_connections 2024-04-16T17:10:28.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 --2- 172.21.15.18:0/2794284769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11cc1090c0 0x7f11cc115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 --2- 172.21.15.18:0/2794284769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc108720 0x7f11cc108b80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 --2- 172.21.15.18:0/2794284769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11cc10a590 0x7f11cc10a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 -- 172.21.15.18:0/2794284769 >> 172.21.15.18:0/2794284769 conn(0x7f11cc077e70 msgr2=0x7f11cc106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 -- 172.21.15.18:0/2794284769 shutdown_connections 2024-04-16T17:10:28.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.923+0000 7f11d1e00640 1 -- 172.21.15.18:0/2794284769 wait complete. 2024-04-16T17:10:28.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.924+0000 7f11d1e00640 1 Processor -- start 2024-04-16T17:10:28.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.924+0000 7f11d1e00640 1 -- start start 2024-04-16T17:10:28.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11cc108720 0x7f11cc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11d1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc1090c0 0x7f11cc19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11d1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11cc10a590 0x7f11cc1a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11d1e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc118af0 con 0x7f11cc1090c0 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11d1e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc118970 con 0x7f11cc108720 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11d1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc118c70 con 0x7f11cc10a590 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11cc108720 0x7f11cc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc1090c0 0x7f11cc19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11cc108720 0x7f11cc19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37102/0 (socket says 172.21.15.18:37102) 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11caa00640 1 -- 172.21.15.18:0/4022796190 learned_addr learned my addr 172.21.15.18:0/4022796190 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.925+0000 7f11cb400640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11cc10a590 0x7f11cc1a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11cb400640 1 -- 172.21.15.18:0/4022796190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11cc108720 msgr2=0x7f11cc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11cb400640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11cc108720 0x7f11cc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11cb400640 1 -- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc1090c0 msgr2=0x7f11cc19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11cb400640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc1090c0 0x7f11cc19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11cb400640 1 -- 172.21.15.18:0/4022796190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc1a3f80 con 0x7f11cc10a590 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11caa00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11cc108720 0x7f11cc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11c3e00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc1090c0 0x7f11cc19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11cb400640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11cc10a590 0x7f11cc1a3850 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f11c400d9c0 tx=0x7f11c400de80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11d0a00640 1 -- 172.21.15.18:0/4022796190 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c400bd10 con 0x7f11cc10a590 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11d0a00640 1 -- 172.21.15.18:0/4022796190 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c4003e50 con 0x7f11cc10a590 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11d0a00640 1 -- 172.21.15.18:0/4022796190 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40106c0 con 0x7f11cc10a590 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.926+0000 7f11d1e00640 1 -- 172.21.15.18:0/4022796190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc1a4270 con 0x7f11cc10a590 2024-04-16T17:10:28.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.927+0000 7f11d1e00640 1 -- 172.21.15.18:0/4022796190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc1abb00 con 0x7f11cc10a590 2024-04-16T17:10:28.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.930+0000 7f11d0a00640 1 -- 172.21.15.18:0/4022796190 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11c4003970 con 0x7f11cc10a590 2024-04-16T17:10:28.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.931+0000 7f11d4c09740 1 Processor -- start 2024-04-16T17:10:28.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.931+0000 7f11d4c09740 1 -- start start 2024-04-16T17:10:28.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 0x55c913a55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a5dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a60640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11d4c09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c9139ba020 con 0x55c913a5e250 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11d4c09740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c913960460 con 0x55c913a54d50 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11d4c09740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c9139b5fe0 con 0x55c913a55670 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a60640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a5dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11d0a00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11b00a3d20 0x7f11b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a5dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48402/0 (socket says 172.21.15.18:48402) 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a60640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44516/0 (socket says 172.21.15.18:44516) 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.932+0000 7f11caa00640 1 -- 172.21.15.18:0/1911048067 learned_addr learned my addr 172.21.15.18:0/1911048067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11c3e00640 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 0x55c913a55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11d0a00640 1 -- 172.21.15.18:0/4022796190 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f11c40d0ed0 con 0x7f11cc10a590 2024-04-16T17:10:28.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11cb400640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11b00a3d20 0x7f11b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11caa00640 1 -- 172.21.15.18:0/1911048067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 msgr2=0x55c913a55130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11caa00640 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 0x55c913a55130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11caa00640 1 -- 172.21.15.18:0/1911048067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 msgr2=0x55c913a60640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11caa00640 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a60640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11caa00640 1 -- 172.21.15.18:0/1911048067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c913968040 con 0x55c913a55670 2024-04-16T17:10:28.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11c3e00640 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 0x55c913a55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:28.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.933+0000 7f11caa00640 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a5dd10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0115f0 tx=0x7f11bc011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff1ff4e49991d107 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11bbe00640 1 -- 172.21.15.18:0/1911048067 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc01e070 con 0x55c913a55670 2024-04-16T17:10:28.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11cb400640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11b00a3d20 0x7f11b00a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f11c40a51a0 tx=0x7f11c4004ab0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11bbe00640 1 -- 172.21.15.18:0/1911048067 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc00fdd0 con 0x55c913a55670 2024-04-16T17:10:28.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11bbe00640 1 -- 172.21.15.18:0/1911048067 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00e3b0 con 0x55c913a55670 2024-04-16T17:10:28.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11d4c09740 1 -- 172.21.15.18:0/1911048067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 msgr2=0x55c913a5dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11d4c09740 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a5dd10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0115f0 tx=0x7f11bc011ab0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11d4c09740 1 -- 172.21.15.18:0/1911048067 shutdown_connections 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11d4c09740 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a60640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.934+0000 7f11d4c09740 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a5dd10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.935+0000 7f11d4c09740 1 --2- 172.21.15.18:0/1911048067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 0x55c913a55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.935+0000 7f11d4c09740 1 -- 172.21.15.18:0/1911048067 >> 172.21.15.18:0/1911048067 conn(0x55c913a4f480 msgr2=0x55c913a4f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.935+0000 7f11d4c09740 1 -- 172.21.15.18:0/1911048067 shutdown_connections 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.935+0000 7f11d4c09740 1 -- 172.21.15.18:0/1911048067 wait complete. 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.935+0000 7f11d4c09740 1 -- start start 2024-04-16T17:10:28.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.935+0000 7f11d4c09740 1 Processor -- start 2024-04-16T17:10:28.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.936+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a54d50 0x55c913a5b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.936+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a5b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.936+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a5e250 0x55c913a5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.936+0000 7f11d4c09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c913a61040 con 0x55c913a54d50 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.936+0000 7f11d4c09740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c913973830 con 0x55c913a55670 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.936+0000 7f11d4c09740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c913a611c0 con 0x55c913a5e250 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.936+0000 7f11d4c09740 10 client.0 _ll_register_callbacks cb 0x55c91397b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.936+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a54d50 0x55c913a5b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a5b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a54d50 0x55c913a5b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44520/0 (socket says 172.21.15.18:44520) 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11c3e00640 1 -- 172.21.15.18:0/3195639096 learned_addr learned my addr 172.21.15.18:0/3195639096 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11caa00640 1 -- 172.21.15.18:0/3195639096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a5e250 msgr2=0x55c913a5bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11caa00640 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a5e250 0x55c913a5bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11caa00640 1 -- 172.21.15.18:0/3195639096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a54d50 msgr2=0x55c913a5b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11caa00640 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a54d50 0x55c913a5b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11caa00640 1 -- 172.21.15.18:0/3195639096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c913977920 con 0x55c913a55670 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11c3e00640 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a54d50 0x55c913a5b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11caa00640 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a5b880 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f11bc011fd0 tx=0x7f11bc005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e2af551540db3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.937+0000 7f11b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.938+0000 7f11b8c00640 1 -- 172.21.15.18:0/3195639096 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc026070 con 0x55c913a55670 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.938+0000 7f11d4c09740 1 -- 172.21.15.18:0/3195639096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11bc00fdd0 con 0x55c913a55670 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.938+0000 7f11b8c00640 1 -- 172.21.15.18:0/3195639096 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f11bc00fad0 con 0x55c913a55670 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.938+0000 7f11b8c00640 1 -- 172.21.15.18:0/3195639096 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11bc032050 con 0x55c913a55670 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.938+0000 7f11d4c09740 10 client.27146 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.938+0000 7f11d4c09740 1 -- 172.21.15.18:0/3195639096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9139457f0 con 0x55c913a55670 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.938+0000 7f11b8c00640 1 -- 172.21.15.18:0/3195639096 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc00fdd0 con 0x55c913a55670 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.939+0000 7f11b8c00640 1 -- 172.21.15.18:0/3195639096 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f11bc004940 con 0x55c913a55670 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.939+0000 7f11d4c09740 10 client.27146 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.939+0000 7f11d4c09740 10 client.27146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:10:28.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.939+0000 7f11d4c09740 20 client.27146 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:28.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.939+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55c913a6e520 0x55c913a70910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.939+0000 7f11d4c09740 4 client.27146 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:28.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.939+0000 7f11d4c09740 1 -- 172.21.15.18:0/3195639096 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9139c7a40 con 0x55c913a6e520 2024-04-16T17:10:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.939+0000 7f11c3e00640 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55c913a6e520 0x55c913a70910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.940+0000 7f11c3e00640 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55c913a6e520 0x55c913a70910 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d902cdb12a312fe server_cookie=c763691c0189131b in_seq=0 out_seq=0 2024-04-16T17:10:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.940+0000 7f11b8c00640 10 client.27146 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:10:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11b8c00640 1 -- 172.21.15.18:0/3195639096 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9139c7a40 con 0x55c913a6e520 2024-04-16T17:10:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11b8c00640 10 client.27146 handle_command_reply: tid=0 2024-04-16T17:10:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 client.27146 shutdown 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 -- 172.21.15.18:0/3195639096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 msgr2=0x55c913a5b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a5b880 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f11bc011fd0 tx=0x7f11bc005d30 comp rx=0 tx=0).stop 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 -- 172.21.15.18:0/3195639096 shutdown_connections 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55c913a6e520 0x55c913a70910 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a5e250 0x55c913a5bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a5b880 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3195639096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a54d50 0x55c913a5b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11b8c00640 0 client.27146 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 -- 172.21.15.18:0/3195639096 >> 172.21.15.18:0/3195639096 conn(0x55c913a4f480 msgr2=0x55c913a511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.941+0000 7f11d4c09740 1 -- 172.21.15.18:0/3195639096 shutdown_connections 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.942+0000 7f11d4c09740 1 -- 172.21.15.18:0/3195639096 wait complete. 2024-04-16T17:10:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.942+0000 7f11d4c09740 20 client.27146 trim_cache size 0 max 16384 2024-04-16T17:10:28.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.958+0000 7f11d4c09740 1 Processor -- start 2024-04-16T17:10:28.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.958+0000 7f11d4c09740 1 -- start start 2024-04-16T17:10:28.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.958+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 0x55c913a5da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.958+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a69f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.958+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a5a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.958+0000 7f11d4c09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b8430 con 0x55c913a5e250 2024-04-16T17:10:28.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.958+0000 7f11d4c09740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b5d00 con 0x55c913a54d50 2024-04-16T17:10:28.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.958+0000 7f11d4c09740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b3b50 con 0x55c913a55670 2024-04-16T17:10:28.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a69f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a5a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a69f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48416/0 (socket says 172.21.15.18:48416) 2024-04-16T17:10:28.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 -- 172.21.15.18:0/1038309430 learned_addr learned my addr 172.21.15.18:0/1038309430 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 -- 172.21.15.18:0/1038309430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 msgr2=0x55c913a5da70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:28.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 --2- 172.21.15.18:0/1038309430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 0x55c913a5da70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 -- 172.21.15.18:0/1038309430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 msgr2=0x55c913a5a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 --2- 172.21.15.18:0/1038309430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a5a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 -- 172.21.15.18:0/1038309430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c913a554a0 con 0x55c913a55670 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.959+0000 7f11caa00640 1 --2- 172.21.15.18:0/1038309430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a69f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f11bc022590 tx=0x7f11bc0225c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1d615b9d504714 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11bb400640 1 -- 172.21.15.18:0/1038309430 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc02f070 con 0x55c913a55670 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11bb400640 1 -- 172.21.15.18:0/1038309430 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc004680 con 0x55c913a55670 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11bb400640 1 -- 172.21.15.18:0/1038309430 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc026050 con 0x55c913a55670 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11d4c09740 1 -- 172.21.15.18:0/1038309430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 msgr2=0x55c913a69f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11d4c09740 1 --2- 172.21.15.18:0/1038309430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a69f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f11bc022590 tx=0x7f11bc0225c0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11d4c09740 1 -- 172.21.15.18:0/1038309430 shutdown_connections 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11d4c09740 1 --2- 172.21.15.18:0/1038309430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a5a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11d4c09740 1 --2- 172.21.15.18:0/1038309430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a55670 0x55c913a69f00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11d4c09740 1 --2- 172.21.15.18:0/1038309430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a54d50 0x55c913a5da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11d4c09740 1 -- 172.21.15.18:0/1038309430 >> 172.21.15.18:0/1038309430 conn(0x55c913a4f480 msgr2=0x55c913a59c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.960+0000 7f11d4c09740 1 -- 172.21.15.18:0/1038309430 shutdown_connections 2024-04-16T17:10:28.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.961+0000 7f11d4c09740 1 -- 172.21.15.18:0/1038309430 wait complete. 2024-04-16T17:10:28.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.961+0000 7f11d4c09740 1 -- start start 2024-04-16T17:10:28.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.961+0000 7f11d4c09740 1 Processor -- start 2024-04-16T17:10:28.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a54d50 0x55c913a669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a58ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11d4c09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11d4c09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b41c0 con 0x55c913a5e250 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11d4c09740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b5630 con 0x55c913a55670 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11d4c09740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b4890 con 0x55c913a54d50 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11d4c09740 10 client.0 _ll_register_callbacks cb 0x7f11bc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44534/0 (socket says 172.21.15.18:44534) 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11c3e00640 1 -- 172.21.15.18:0/3468754511 learned_addr learned my addr 172.21.15.18:0/3468754511 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11caa00640 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a58ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11c3e00640 1 -- 172.21.15.18:0/3468754511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a54d50 msgr2=0x55c913a669a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11c3e00640 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a54d50 0x55c913a669a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11c3e00640 1 -- 172.21.15.18:0/3468754511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 msgr2=0x55c913a58ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11c3e00640 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a58ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11c3e00640 1 -- 172.21.15.18:0/3468754511 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9136cc3f0 con 0x55c913a5e250 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.962+0000 7f11caa00640 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a58ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.963+0000 7f11c3e00640 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a575b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11b402c8c0 tx=0x7f11b402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fdec4b015419a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.963+0000 7f11b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.963+0000 7f11b8c00640 1 -- 172.21.15.18:0/3468754511 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4050060 con 0x55c913a5e250 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.963+0000 7f11b8c00640 1 -- 172.21.15.18:0/3468754511 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f11b4004300 con 0x55c913a5e250 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.963+0000 7f11d4c09740 1 -- 172.21.15.18:0/3468754511 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11bc004680 con 0x55c913a5e250 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.963+0000 7f11b8c00640 1 -- 172.21.15.18:0/3468754511 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11b4057050 con 0x55c913a5e250 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.963+0000 7f11d4c09740 10 client.16923 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.963+0000 7f11d4c09740 1 -- 172.21.15.18:0/3468754511 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9137967d0 con 0x55c913a5e250 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11b8c00640 1 -- 172.21.15.18:0/3468754511 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc004680 con 0x55c913a5e250 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11b8c00640 1 -- 172.21.15.18:0/3468754511 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f11b4003bc0 con 0x55c913a5e250 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11d4c09740 10 client.16923 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11d4c09740 10 client.16923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11d4c09740 20 client.16923 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55c913ad4360 0x55c913ad6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11d4c09740 4 client.16923 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:28.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11d4c09740 1 -- 172.21.15.18:0/3468754511 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9139c18b0 con 0x55c913ad4360 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.964+0000 7f11caa00640 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55c913ad4360 0x55c913ad6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11caa00640 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55c913ad4360 0x55c913ad6780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57bfa10eecd09bf1 server_cookie=2b25f5316571e45e in_seq=0 out_seq=0 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11b8c00640 10 client.16923 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11b8c00640 1 -- 172.21.15.18:0/3468754511 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9139c18b0 con 0x55c913ad4360 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11b8c00640 10 client.16923 handle_command_reply: tid=0 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 client.16923 shutdown 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 -- 172.21.15.18:0/3468754511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 msgr2=0x55c913a575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a575b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11b402c8c0 tx=0x7f11b402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 -- 172.21.15.18:0/3468754511 shutdown_connections 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55c913ad4360 0x55c913ad6780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c913a5e250 0x55c913a575b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c913a55670 0x55c913a58ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 --2- 172.21.15.18:0/3468754511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c913a54d50 0x55c913a669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11d4c09740 1 -- 172.21.15.18:0/3468754511 >> 172.21.15.18:0/3468754511 conn(0x55c913a4f480 msgr2=0x55c913a5a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.965+0000 7f11b8c00640 0 client.16923 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.966+0000 7f11d4c09740 1 -- 172.21.15.18:0/3468754511 shutdown_connections 2024-04-16T17:10:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.966+0000 7f11d4c09740 1 -- 172.21.15.18:0/3468754511 wait complete. 2024-04-16T17:10:28.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.966+0000 7f11d4c09740 20 client.16923 trim_cache size 0 max 16384 2024-04-16T17:10:28.970 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.966+0000 7f11d1e00640 1 -- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11b00a3d20 msgr2=0x7f11b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.966+0000 7f11d1e00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11b00a3d20 0x7f11b00a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f11c40a51a0 tx=0x7f11c4004ab0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.966+0000 7f11d1e00640 1 -- 172.21.15.18:0/4022796190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11cc10a590 msgr2=0x7f11cc1a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.966+0000 7f11d1e00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11cc10a590 0x7f11cc1a3850 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f11c400d9c0 tx=0x7f11c400de80 comp rx=0 tx=0).stop 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.967+0000 7f11d1e00640 1 -- 172.21.15.18:0/4022796190 shutdown_connections 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.967+0000 7f11d1e00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11b00a3d20 0x7f11b00a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.967+0000 7f11d1e00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11cc10a590 0x7f11cc1a3850 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.967+0000 7f11d1e00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11cc1090c0 0x7f11cc19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.967+0000 7f11d1e00640 1 --2- 172.21.15.18:0/4022796190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11cc108720 0x7f11cc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.967+0000 7f11d1e00640 1 -- 172.21.15.18:0/4022796190 >> 172.21.15.18:0/4022796190 conn(0x7f11cc077e70 msgr2=0x7f11cc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.967+0000 7f11d1e00640 1 -- 172.21.15.18:0/4022796190 shutdown_connections 2024-04-16T17:10:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:28.967+0000 7f11d1e00640 1 -- 172.21.15.18:0/4022796190 wait complete. 2024-04-16T17:10:28.988 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:10:29.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 -- 172.21.15.18:0/1276708458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a740743a0 msgr2=0x7f1a74074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 --2- 172.21.15.18:0/1276708458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a740743a0 0x7f1a74074780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009900 tx=0x7f1a6802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 -- 172.21.15.18:0/1276708458 shutdown_connections 2024-04-16T17:10:29.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 --2- 172.21.15.18:0/1276708458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a74112e10 0x7f1a74115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 --2- 172.21.15.18:0/1276708458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a74074cc0 0x7f1a741128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 --2- 172.21.15.18:0/1276708458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a740743a0 0x7f1a74074780 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 -- 172.21.15.18:0/1276708458 >> 172.21.15.18:0/1276708458 conn(0x7f1a740fd950 msgr2=0x7f1a740ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 -- 172.21.15.18:0/1276708458 shutdown_connections 2024-04-16T17:10:29.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.168+0000 7f1a79e00640 1 -- 172.21.15.18:0/1276708458 wait complete. 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.169+0000 7f1a79e00640 1 Processor -- start 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.169+0000 7f1a79e00640 1 -- start start 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.169+0000 7f1a79e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a740743a0 0x7f1a7419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.169+0000 7f1a79e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a74074cc0 0x7f1a7419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a79e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a74112e10 0x7f1a741a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a79e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a74118850 con 0x7f1a74074cc0 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a79e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741186d0 con 0x7f1a74112e10 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a79e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741189d0 con 0x7f1a740743a0 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a73e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a74074cc0 0x7f1a7419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a73e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a74074cc0 0x7f1a7419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44550/0 (socket says 172.21.15.18:44550) 2024-04-16T17:10:29.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a73e00640 1 -- 172.21.15.18:0/2100641536 learned_addr learned my addr 172.21.15.18:0/2100641536 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a78a00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a740743a0 0x7f1a7419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a79400640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a74112e10 0x7f1a741a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a73e00640 1 -- 172.21.15.18:0/2100641536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a740743a0 msgr2=0x7f1a7419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a73e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a740743a0 0x7f1a7419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a73e00640 1 -- 172.21.15.18:0/2100641536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a74112e10 msgr2=0x7f1a741a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a73e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a74112e10 0x7f1a741a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a73e00640 1 -- 172.21.15.18:0/2100641536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a741a3dc0 con 0x7f1a74074cc0 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.170+0000 7f1a79400640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a74112e10 0x7f1a741a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.171+0000 7f1a73e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a74074cc0 0x7f1a7419f310 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000e9f0 tx=0x7f1a6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.171+0000 7f1a71600640 1 -- 172.21.15.18:0/2100641536 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000cd80 con 0x7f1a74074cc0 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.171+0000 7f1a79e00640 1 -- 172.21.15.18:0/2100641536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a741a40b0 con 0x7f1a74074cc0 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.171+0000 7f1a71600640 1 -- 172.21.15.18:0/2100641536 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a60005500 con 0x7f1a74074cc0 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.171+0000 7f1a71600640 1 -- 172.21.15.18:0/2100641536 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60010690 con 0x7f1a74074cc0 2024-04-16T17:10:29.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.171+0000 7f1a79e00640 1 -- 172.21.15.18:0/2100641536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a741ab940 con 0x7f1a74074cc0 2024-04-16T17:10:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.174+0000 7f1a71600640 1 -- 172.21.15.18:0/2100641536 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a600046c0 con 0x7f1a74074cc0 2024-04-16T17:10:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.175+0000 7f1a7e1a1740 1 Processor -- start 2024-04-16T17:10:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.175+0000 7f1a7e1a1740 1 -- start start 2024-04-16T17:10:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.175+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e44f10 0x55ece7e452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.175+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.175+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e4e410 0x55ece7e50800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.175+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ece7daa020 con 0x55ece7e45830 2024-04-16T17:10:29.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.175+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ece7d50460 con 0x55ece7e44f10 2024-04-16T17:10:29.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.175+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ece7da5fe0 con 0x55ece7e4e410 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a79400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e44f10 0x55ece7e452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a79400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e44f10 0x55ece7e452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37164/0 (socket says 172.21.15.18:37164) 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a79400640 1 -- 172.21.15.18:0/2824256878 learned_addr learned my addr 172.21.15.18:0/2824256878 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a78a00640 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a73e00640 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e4e410 0x55ece7e50800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a79400640 1 -- 172.21.15.18:0/2824256878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e4e410 msgr2=0x55ece7e50800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a79400640 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e4e410 0x55ece7e50800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a79400640 1 -- 172.21.15.18:0/2824256878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 msgr2=0x55ece7e4ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a79400640 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a79400640 1 -- 172.21.15.18:0/2824256878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece7d58040 con 0x55ece7e44f10 2024-04-16T17:10:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a78a00640 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.176+0000 7f1a71600640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a5c0a3da0 0x7f1a5c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.177+0000 7f1a71600640 1 -- 172.21.15.18:0/2100641536 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f1a600d0ff0 con 0x7f1a74074cc0 2024-04-16T17:10:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.177+0000 7f1a79400640 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e44f10 0x55ece7e452f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009fa0 tx=0x7f1a6c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b8f9c3a5489a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.177+0000 7f1a78a00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a5c0a3da0 0x7f1a5c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.177+0000 7f1a65c00640 1 -- 172.21.15.18:0/2824256878 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c011700 con 0x55ece7e44f10 2024-04-16T17:10:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.177+0000 7f1a65c00640 1 -- 172.21.15.18:0/2824256878 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a6c011d80 con 0x55ece7e44f10 2024-04-16T17:10:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.177+0000 7f1a65c00640 1 -- 172.21.15.18:0/2824256878 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c01a380 con 0x55ece7e44f10 2024-04-16T17:10:29.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/2824256878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e44f10 msgr2=0x55ece7e452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a78a00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a5c0a3da0 0x7f1a5c0a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a68005ab0 tx=0x7f1a68005a20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e44f10 0x55ece7e452f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009fa0 tx=0x7f1a6c010860 comp rx=0 tx=0).stop 2024-04-16T17:10:29.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/2824256878 shutdown_connections 2024-04-16T17:10:29.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e4e410 0x55ece7e50800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/2824256878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e44f10 0x55ece7e452f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/2824256878 >> 172.21.15.18:0/2824256878 conn(0x55ece7e3f640 msgr2=0x55ece7e3fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/2824256878 shutdown_connections 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.178+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/2824256878 wait complete. 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 1 -- start start 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 1 Processor -- start 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e44f10 0x55ece7e4b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e4bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ece7e51200 con 0x55ece7e45830 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ece7d63830 con 0x55ece7e4e410 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ece7e51380 con 0x55ece7e44f10 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.179+0000 7f1a7e1a1740 10 client.0 _ll_register_callbacks cb 0x55ece7d6b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e44f10 0x55ece7e4b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a73e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e44f10 0x55ece7e4b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48460/0 (socket says 172.21.15.18:48460) 2024-04-16T17:10:29.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a78a00640 1 -- 172.21.15.18:0/4141741806 learned_addr learned my addr 172.21.15.18:0/4141741806 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a73e00640 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44566/0 (socket says 172.21.15.18:44566) 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a79400640 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e4bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a73e00640 1 -- 172.21.15.18:0/4141741806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e44f10 msgr2=0x55ece7e4b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a73e00640 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e44f10 0x55ece7e4b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a73e00640 1 -- 172.21.15.18:0/4141741806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 msgr2=0x55ece7e4bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a73e00640 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e4bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a73e00640 1 -- 172.21.15.18:0/4141741806 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece7d67920 con 0x55ece7e45830 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a79400640 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a73e00640 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4b9a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ce360 tx=0x7f1a60004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e36852a020af19 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.180+0000 7f1a5a000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a5a000640 1 -- 172.21.15.18:0/4141741806 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6009c540 con 0x55ece7e45830 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a5a000640 1 -- 172.21.15.18:0/4141741806 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f1a600d3110 con 0x55ece7e45830 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/4141741806 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a6c011d80 con 0x55ece7e45830 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a5a000640 1 -- 172.21.15.18:0/4141741806 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a600d3310 con 0x55ece7e45830 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a7e1a1740 10 client.16935 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/4141741806 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ece7d357f0 con 0x55ece7e45830 2024-04-16T17:10:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a5a000640 1 -- 172.21.15.18:0/4141741806 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a6c011d80 con 0x55ece7e45830 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a5a000640 1 -- 172.21.15.18:0/4141741806 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1a600a53c0 con 0x55ece7e45830 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a7e1a1740 10 client.16935 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a7e1a1740 10 client.16935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a7e1a1740 20 client.16935 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.181+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55ece7e5e730 0x55ece7e60b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.182+0000 7f1a7e1a1740 4 client.16935 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.182+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/4141741806 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ece7db7a40 con 0x55ece7e5e730 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.182+0000 7f1a78a00640 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55ece7e5e730 0x55ece7e60b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.182+0000 7f1a78a00640 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55ece7e5e730 0x55ece7e60b20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8895d8e2294eaef9 server_cookie=ae8dc77682100bfe in_seq=0 out_seq=0 2024-04-16T17:10:29.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.182+0000 7f1a5a000640 10 client.16935 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:10:29.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a5a000640 1 -- 172.21.15.18:0/4141741806 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ece7db7a40 con 0x55ece7e5e730 2024-04-16T17:10:29.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a5a000640 10 client.16935 handle_command_reply: tid=0 2024-04-16T17:10:29.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 client.16935 shutdown 2024-04-16T17:10:29.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 msgr2=0x55ece7e4b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4b9a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ce360 tx=0x7f1a60004ba0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/4141741806 shutdown_connections 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55ece7e5e730 0x55ece7e60b20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e45830 0x55ece7e4b9a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/4141741806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e44f10 0x55ece7e4b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.183+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/4141741806 >> 172.21.15.18:0/4141741806 conn(0x55ece7e3f640 msgr2=0x55ece7e41390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.184+0000 7f1a5a000640 0 client.16935 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.184+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/4141741806 shutdown_connections 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.184+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/4141741806 wait complete. 2024-04-16T17:10:29.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.184+0000 7f1a7e1a1740 20 client.16935 trim_cache size 0 max 16384 2024-04-16T17:10:29.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.198+0000 7f1a7e1a1740 1 Processor -- start 2024-04-16T17:10:29.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.198+0000 7f1a7e1a1740 1 -- start start 2024-04-16T17:10:29.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.198+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e4db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.198+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e5a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.198+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e41180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.198+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741b82b0 con 0x55ece7e44f10 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.198+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741b5b80 con 0x55ece7e4e410 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.198+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741b39d0 con 0x55ece7e45830 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e4db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e4db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44572/0 (socket says 172.21.15.18:44572) 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a78a00640 1 -- 172.21.15.18:0/1425717978 learned_addr learned my addr 172.21.15.18:0/1425717978 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a73e00640 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e5a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a79400640 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e41180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a78a00640 1 -- 172.21.15.18:0/1425717978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 msgr2=0x55ece7e5a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a78a00640 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e5a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a78a00640 1 -- 172.21.15.18:0/1425717978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 msgr2=0x55ece7e41180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a78a00640 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e41180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a78a00640 1 -- 172.21.15.18:0/1425717978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece7abc3f0 con 0x55ece7e44f10 2024-04-16T17:10:29.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.199+0000 7f1a73e00640 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e5a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:29.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.200+0000 7f1a78a00640 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e4db90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1a6802c2d0 tx=0x7f1a68040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451d9af220dbfab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.200+0000 7f1a65200640 1 -- 172.21.15.18:0/1425717978 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68055b50 con 0x55ece7e44f10 2024-04-16T17:10:29.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.200+0000 7f1a65200640 1 -- 172.21.15.18:0/1425717978 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a6805a070 con 0x55ece7e44f10 2024-04-16T17:10:29.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.200+0000 7f1a65200640 1 -- 172.21.15.18:0/1425717978 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a680049d0 con 0x55ece7e44f10 2024-04-16T17:10:29.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/1425717978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 msgr2=0x55ece7e4db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e4db90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1a6802c2d0 tx=0x7f1a68040000 comp rx=0 tx=0).stop 2024-04-16T17:10:29.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/1425717978 shutdown_connections 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e41180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e5a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/1425717978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e4db90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/1425717978 >> 172.21.15.18:0/1425717978 conn(0x55ece7e3f640 msgr2=0x55ece7e40a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/1425717978 shutdown_connections 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.201+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/1425717978 wait complete. 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 1 -- start start 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 1 Processor -- start 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e47cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e48230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741b4040 con 0x55ece7e44f10 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741b54b0 con 0x55ece7e4e410 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741b4710 con 0x55ece7e45830 2024-04-16T17:10:29.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.202+0000 7f1a7e1a1740 10 client.0 _ll_register_callbacks cb 0x7f1a6000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e47cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e47cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48474/0 (socket says 172.21.15.18:48474) 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a73e00640 1 -- 172.21.15.18:0/698991756 learned_addr learned my addr 172.21.15.18:0/698991756 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a79400640 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e48230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a78a00640 1 -- 172.21.15.18:0/698991756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 msgr2=0x55ece7e47cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a78a00640 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e47cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a78a00640 1 -- 172.21.15.18:0/698991756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 msgr2=0x55ece7e48230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a78a00640 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e48230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.203+0000 7f1a78a00640 1 -- 172.21.15.18:0/698991756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece7b867d0 con 0x55ece7e44f10 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.204+0000 7f1a78a00640 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e567f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a680096c0 tx=0x7f1a6802ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc960a194ac46c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.204+0000 7f1a5a000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.204+0000 7f1a5a000640 1 -- 172.21.15.18:0/698991756 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68063060 con 0x55ece7e44f10 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.204+0000 7f1a5a000640 1 -- 172.21.15.18:0/698991756 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f1a6803a040 con 0x55ece7e44f10 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.204+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/698991756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a6805a070 con 0x55ece7e44f10 2024-04-16T17:10:29.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.204+0000 7f1a5a000640 1 -- 172.21.15.18:0/698991756 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a6806a050 con 0x55ece7e44f10 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.204+0000 7f1a7e1a1740 10 client.16941 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.204+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/698991756 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ece7ec47c0 con 0x55ece7e44f10 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a5a000640 1 -- 172.21.15.18:0/698991756 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a6805a070 con 0x55ece7e44f10 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a5a000640 1 -- 172.21.15.18:0/698991756 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1a68005260 con 0x55ece7e44f10 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a7e1a1740 10 client.16941 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a7e1a1740 10 client.16941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a7e1a1740 20 client.16941 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55ece7ec4eb0 0x55ece7ec72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a7e1a1740 4 client.16941 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/698991756 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ece7db18b0 con 0x55ece7ec4eb0 2024-04-16T17:10:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.205+0000 7f1a73e00640 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55ece7ec4eb0 0x55ece7ec72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a73e00640 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55ece7ec4eb0 0x55ece7ec72d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0dbc1e533c19182 server_cookie=c4aab4d6b3e563b9 in_seq=0 out_seq=0 2024-04-16T17:10:29.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a5a000640 10 client.16941 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a5a000640 1 -- 172.21.15.18:0/698991756 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ece7db18b0 con 0x55ece7ec4eb0 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a5a000640 10 client.16941 handle_command_reply: tid=0 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 client.16941 shutdown 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/698991756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 msgr2=0x55ece7e567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e567f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a680096c0 tx=0x7f1a6802ca40 comp rx=0 tx=0).stop 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/698991756 shutdown_connections 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55ece7ec4eb0 0x55ece7ec72d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ece7e4e410 0x55ece7e48230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ece7e45830 0x55ece7e47cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 --2- 172.21.15.18:0/698991756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ece7e44f10 0x55ece7e567f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/698991756 >> 172.21.15.18:0/698991756 conn(0x55ece7e3f640 msgr2=0x55ece7e3e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.206+0000 7f1a5a000640 0 client.16941 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.207+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/698991756 shutdown_connections 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.207+0000 7f1a7e1a1740 1 -- 172.21.15.18:0/698991756 wait complete. 2024-04-16T17:10:29.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.207+0000 7f1a7e1a1740 20 client.16941 trim_cache size 0 max 16384 2024-04-16T17:10:29.212 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:10:29.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.207+0000 7f1a79e00640 1 -- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a5c0a3da0 msgr2=0x7f1a5c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.207+0000 7f1a79e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a5c0a3da0 0x7f1a5c0a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a68005ab0 tx=0x7f1a68005a20 comp rx=0 tx=0).stop 2024-04-16T17:10:29.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.207+0000 7f1a79e00640 1 -- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a74074cc0 msgr2=0x7f1a7419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.207+0000 7f1a79e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a74074cc0 0x7f1a7419f310 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000e9f0 tx=0x7f1a6000edf0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.208+0000 7f1a79e00640 1 -- 172.21.15.18:0/2100641536 shutdown_connections 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.208+0000 7f1a79e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1a5c0a3da0 0x7f1a5c0a6260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.208+0000 7f1a79e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a74112e10 0x7f1a741a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.208+0000 7f1a79e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a74074cc0 0x7f1a7419f310 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.208+0000 7f1a79e00640 1 --2- 172.21.15.18:0/2100641536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a740743a0 0x7f1a7419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.208+0000 7f1a79e00640 1 -- 172.21.15.18:0/2100641536 >> 172.21.15.18:0/2100641536 conn(0x7f1a740fd950 msgr2=0x7f1a740fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.208+0000 7f1a79e00640 1 -- 172.21.15.18:0/2100641536 shutdown_connections 2024-04-16T17:10:29.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.208+0000 7f1a79e00640 1 -- 172.21.15.18:0/2100641536 wait complete. 2024-04-16T17:10:29.224 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:10:29.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.434+0000 7f0dd7200640 1 -- 172.21.15.18:0/2068158820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd8102070 msgr2=0x7f0dd8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.434+0000 7f0dd7200640 1 --2- 172.21.15.18:0/2068158820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd8102070 0x7f0dd8102450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009900 tx=0x7f0dc802bc50 comp rx=0 tx=0).stop 2024-04-16T17:10:29.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.434+0000 7f0dd7200640 1 -- 172.21.15.18:0/2068158820 shutdown_connections 2024-04-16T17:10:29.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.434+0000 7f0dd7200640 1 --2- 172.21.15.18:0/2068158820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd8107140 0x7f0dd810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.434+0000 7f0dd7200640 1 --2- 172.21.15.18:0/2068158820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dd8102a20 0x7f0dd8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.434+0000 7f0dd7200640 1 --2- 172.21.15.18:0/2068158820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd8102070 0x7f0dd8102450 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.434+0000 7f0dd7200640 1 -- 172.21.15.18:0/2068158820 >> 172.21.15.18:0/2068158820 conn(0x7f0dd80fd950 msgr2=0x7f0dd80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.435+0000 7f0dd7200640 1 -- 172.21.15.18:0/2068158820 shutdown_connections 2024-04-16T17:10:29.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.435+0000 7f0dd7200640 1 -- 172.21.15.18:0/2068158820 wait complete. 2024-04-16T17:10:29.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.437+0000 7f0dd7200640 1 Processor -- start 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.437+0000 7f0dd7200640 1 -- start start 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.437+0000 7f0dd7200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd8102070 0x7f0dd819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.437+0000 7f0dd7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dd8107140 0x7f0dd819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.437+0000 7f0dd7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd81985f0 0x7f0dd8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.437+0000 7f0dd7200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd81121d0 con 0x7f0dd8102070 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.437+0000 7f0dd7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8112050 con 0x7f0dd8107140 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.437+0000 7f0dd7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8112350 con 0x7f0dd81985f0 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dd8107140 0x7f0dd819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd8102070 0x7f0dd819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dd8107140 0x7f0dd819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37196/0 (socket says 172.21.15.18:37196) 2024-04-16T17:10:29.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd81985f0 0x7f0dd8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5400640 1 -- 172.21.15.18:0/4253191239 learned_addr learned my addr 172.21.15.18:0/4253191239 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5400640 1 -- 172.21.15.18:0/4253191239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd81985f0 msgr2=0x7f0dd8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5400640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd81985f0 0x7f0dd8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5400640 1 -- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd8102070 msgr2=0x7f0dd819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5400640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd8102070 0x7f0dd819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd5400640 1 -- 172.21.15.18:0/4253191239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd8199110 con 0x7f0dd8107140 2024-04-16T17:10:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.438+0000 7f0dd6800640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd81985f0 0x7f0dd8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.439+0000 7f0dd5400640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dd8107140 0x7f0dd819e980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4005fd0 tx=0x7f0dc400dd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.439+0000 7f0dcea00640 1 -- 172.21.15.18:0/4253191239 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4017070 con 0x7f0dd8107140 2024-04-16T17:10:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.439+0000 7f0dd7200640 1 -- 172.21.15.18:0/4253191239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd8199400 con 0x7f0dd8107140 2024-04-16T17:10:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.439+0000 7f0dcea00640 1 -- 172.21.15.18:0/4253191239 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc400de50 con 0x7f0dd8107140 2024-04-16T17:10:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.439+0000 7f0dcea00640 1 -- 172.21.15.18:0/4253191239 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4003e00 con 0x7f0dd8107140 2024-04-16T17:10:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.439+0000 7f0dd7200640 1 -- 172.21.15.18:0/4253191239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd81a5650 con 0x7f0dd8107140 2024-04-16T17:10:29.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.443+0000 7f0ddfd97740 1 Processor -- start 2024-04-16T17:10:29.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.443+0000 7f0ddfd97740 1 -- start start 2024-04-16T17:10:29.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495a8f0 0x55fab495acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab49682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 0x55fab496ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fab48c0020 con 0x55fab4968820 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fab4866460 con 0x55fab495a8f0 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fab48bbfe0 con 0x55fab495b2a0 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab49682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab49682e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48500/0 (socket says 172.21.15.18:48500) 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0dd5e00640 1 -- 172.21.15.18:0/46177163 learned_addr learned my addr 172.21.15.18:0/46177163 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0dd6800640 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495a8f0 0x55fab495acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.444+0000 7f0dcea00640 1 -- 172.21.15.18:0/4253191239 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0dc4003fa0 con 0x7f0dd8107140 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.445+0000 7f0dd5e00640 1 -- 172.21.15.18:0/46177163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495a8f0 msgr2=0x55fab495acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.445+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495a8f0 0x55fab495acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.445+0000 7f0dd5e00640 1 -- 172.21.15.18:0/46177163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 msgr2=0x55fab496ac10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:10:29.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.445+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 0x55fab496ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.445+0000 7f0dd5e00640 1 -- 172.21.15.18:0/46177163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fab486e040 con 0x55fab495b2a0 2024-04-16T17:10:29.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.445+0000 7f0dd6800640 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495a8f0 0x55fab495acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:29.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.445+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab49682e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0dc802c190 tx=0x7f0dc8005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dce61b0c9a5eebd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.445+0000 7f0dc3400640 1 -- 172.21.15.18:0/46177163 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc802caa0 con 0x55fab495b2a0 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.446+0000 7f0dc3400640 1 -- 172.21.15.18:0/46177163 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc802cc40 con 0x55fab495b2a0 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.446+0000 7f0dc3400640 1 -- 172.21.15.18:0/46177163 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8005140 con 0x55fab495b2a0 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.446+0000 7f0dcea00640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dbc0a3d20 0x7f0dbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.446+0000 7f0dcea00640 1 -- 172.21.15.18:0/4253191239 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f0dc40d0830 con 0x7f0dd8107140 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.446+0000 7f0dd6800640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dbc0a3d20 0x7f0dbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.446+0000 7f0ddfd97740 1 -- 172.21.15.18:0/46177163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 msgr2=0x55fab49682e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.446+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab49682e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0dc802c190 tx=0x7f0dc8005dd0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 -- 172.21.15.18:0/46177163 shutdown_connections 2024-04-16T17:10:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 0x55fab496ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab49682e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/46177163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495a8f0 0x55fab495acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 -- 172.21.15.18:0/46177163 >> 172.21.15.18:0/46177163 conn(0x55fab4955640 msgr2=0x55fab4955a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 -- 172.21.15.18:0/46177163 shutdown_connections 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 -- 172.21.15.18:0/46177163 wait complete. 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0dd6800640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dbc0a3d20 0x7f0dbc0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00115a0 tx=0x7f0dd0009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 -- start start 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.447+0000 7f0ddfd97740 1 Processor -- start 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4965870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495b2a0 0x55fab4965db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 0x55fab49662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fab496b580 con 0x55fab4968820 2024-04-16T17:10:29.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fab4879830 con 0x55fab495b2a0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fab496b700 con 0x55fab495a8f0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0ddfd97740 10 client.0 _ll_register_callbacks cb 0x55fab4881150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4965870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4965870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48502/0 (socket says 172.21.15.18:48502) 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0dd5e00640 1 -- 172.21.15.18:0/2896184406 learned_addr learned my addr 172.21.15.18:0/2896184406 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0dd6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 0x55fab49662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.448+0000 7f0dd5400640 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495b2a0 0x55fab4965db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0dd5e00640 1 -- 172.21.15.18:0/2896184406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495b2a0 msgr2=0x55fab4965db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495b2a0 0x55fab4965db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0dd5e00640 1 -- 172.21.15.18:0/2896184406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 msgr2=0x55fab49662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 0x55fab49662f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0dd5e00640 1 -- 172.21.15.18:0/2896184406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fab487d920 con 0x55fab495a8f0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0dd6800640 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 0x55fab49662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4965870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80096c0 tx=0x7f0dc802bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c525cf75561e4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0db7e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:10:29.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0db7e00640 1 -- 172.21.15.18:0/2896184406 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8004460 con 0x55fab495a8f0 2024-04-16T17:10:29.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2896184406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc802cc40 con 0x55fab495a8f0 2024-04-16T17:10:29.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.449+0000 7f0db7e00640 1 -- 172.21.15.18:0/2896184406 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f0dc8004ec0 con 0x55fab495a8f0 2024-04-16T17:10:29.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.450+0000 7f0db7e00640 1 -- 172.21.15.18:0/2896184406 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0dc8058050 con 0x55fab495a8f0 2024-04-16T17:10:29.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.450+0000 7f0ddfd97740 10 client.27145 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:29.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.450+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2896184406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fab484b7f0 con 0x55fab495a8f0 2024-04-16T17:10:29.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.450+0000 7f0db7e00640 1 -- 172.21.15.18:0/2896184406 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc802cc40 con 0x55fab495a8f0 2024-04-16T17:10:29.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.451+0000 7f0db7e00640 1 -- 172.21.15.18:0/2896184406 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0dc8003c70 con 0x55fab495a8f0 2024-04-16T17:10:29.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.451+0000 7f0ddfd97740 10 client.27145 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:29.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.451+0000 7f0ddfd97740 10 client.27145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:29.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.451+0000 7f0ddfd97740 20 client.27145 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:29.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.451+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fab4978a00 0x55fab497adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.451+0000 7f0ddfd97740 4 client.27145 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:29.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.451+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2896184406 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fab48cda40 con 0x55fab4978a00 2024-04-16T17:10:29.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.465+0000 7f0dd5400640 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fab4978a00 0x55fab497adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.466+0000 7f0dd5400640 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fab4978a00 0x55fab497adf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef2048aea4bf6d server_cookie=4ef3ff972e537e3b in_seq=0 out_seq=0 2024-04-16T17:10:29.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.466+0000 7f0db7e00640 10 client.27145 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:29.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.467+0000 7f0db7e00640 1 -- 172.21.15.18:0/2896184406 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fab48cda40 con 0x55fab4978a00 2024-04-16T17:10:29.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.467+0000 7f0db7e00640 10 client.27145 handle_command_reply: tid=0 2024-04-16T17:10:29.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.467+0000 7f0ddfd97740 1 client.27145 shutdown 2024-04-16T17:10:29.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2896184406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 msgr2=0x55fab4965870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4965870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80096c0 tx=0x7f0dc802bc50 comp rx=0 tx=0).stop 2024-04-16T17:10:29.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2896184406 shutdown_connections 2024-04-16T17:10:29.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fab4978a00 0x55fab497adf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab4968820 0x55fab49662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab495b2a0 0x55fab4965db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2896184406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4965870 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2896184406 >> 172.21.15.18:0/2896184406 conn(0x55fab4955640 msgr2=0x55fab4957390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0db7e00640 0 client.27145 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:29.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2896184406 shutdown_connections 2024-04-16T17:10:29.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2896184406 wait complete. 2024-04-16T17:10:29.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.468+0000 7f0ddfd97740 20 client.27145 trim_cache size 0 max 16384 2024-04-16T17:10:29.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.496+0000 7f0ddfd97740 1 Processor -- start 2024-04-16T17:10:29.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.497+0000 7f0ddfd97740 1 -- start start 2024-04-16T17:10:29.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.498+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4967fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.498+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495b2a0 0x55fab4957480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.498+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab4964dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.498+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd81afab0 con 0x55fab495b2a0 2024-04-16T17:10:29.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.498+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd81ad380 con 0x55fab4968820 2024-04-16T17:10:29.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.498+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd81ab1d0 con 0x55fab495a8f0 2024-04-16T17:10:29.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.498+0000 7f0dd6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab4964dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.499+0000 7f0dd5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495b2a0 0x55fab4957480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.499+0000 7f0dd6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab4964dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37222/0 (socket says 172.21.15.18:37222) 2024-04-16T17:10:29.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.499+0000 7f0dd6800640 1 -- 172.21.15.18:0/2024268919 learned_addr learned my addr 172.21.15.18:0/2024268919 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.499+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/2024268919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4967fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.499+0000 7f0dd6800640 1 -- 172.21.15.18:0/2024268919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 msgr2=0x55fab4967fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.499+0000 7f0dd6800640 1 --2- 172.21.15.18:0/2024268919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4967fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.500+0000 7f0dd6800640 1 -- 172.21.15.18:0/2024268919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495b2a0 msgr2=0x55fab4957480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.500+0000 7f0dd6800640 1 --2- 172.21.15.18:0/2024268919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495b2a0 0x55fab4957480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.500+0000 7f0dd6800640 1 -- 172.21.15.18:0/2024268919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fab495b040 con 0x55fab4968820 2024-04-16T17:10:29.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.500+0000 7f0dd6800640 1 --2- 172.21.15.18:0/2024268919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab4964dd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00124b0 tx=0x7f0dd0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8792ef45612812f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.501+0000 7f0dc2a00640 1 -- 172.21.15.18:0/2024268919 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd000fda0 con 0x55fab4968820 2024-04-16T17:10:29.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.501+0000 7f0dc2a00640 1 -- 172.21.15.18:0/2024268919 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd0003e50 con 0x55fab4968820 2024-04-16T17:10:29.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.501+0000 7f0dc2a00640 1 -- 172.21.15.18:0/2024268919 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0004490 con 0x55fab4968820 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2024268919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 msgr2=0x55fab4964dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2024268919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab4964dd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00124b0 tx=0x7f0dd0017000 comp rx=0 tx=0).stop 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2024268919 shutdown_connections 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2024268919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab4964dd0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2024268919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495b2a0 0x55fab4957480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/2024268919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495a8f0 0x55fab4967fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2024268919 >> 172.21.15.18:0/2024268919 conn(0x55fab4955640 msgr2=0x55fab4974220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2024268919 shutdown_connections 2024-04-16T17:10:29.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.502+0000 7f0ddfd97740 1 -- 172.21.15.18:0/2024268919 wait complete. 2024-04-16T17:10:29.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.503+0000 7f0ddfd97740 1 -- start start 2024-04-16T17:10:29.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.504+0000 7f0ddfd97740 1 Processor -- start 2024-04-16T17:10:29.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.505+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495a8f0 0x55fab495e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.505+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab495eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.505+0000 7f0ddfd97740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab497ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.506+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd81ab840 con 0x55fab495a8f0 2024-04-16T17:10:29.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.506+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd81accb0 con 0x55fab4968820 2024-04-16T17:10:29.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.506+0000 7f0ddfd97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd81abf10 con 0x55fab495b2a0 2024-04-16T17:10:29.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.506+0000 7f0ddfd97740 10 client.0 _ll_register_callbacks cb 0x7f0dd0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:29.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.506+0000 7f0dd6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab497ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.506+0000 7f0dd6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab497ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37234/0 (socket says 172.21.15.18:37234) 2024-04-16T17:10:29.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.506+0000 7f0dd6800640 1 -- 172.21.15.18:0/990310729 learned_addr learned my addr 172.21.15.18:0/990310729 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:29.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.506+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495a8f0 0x55fab495e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.507+0000 7f0dd5400640 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab495eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.507+0000 7f0dd6800640 1 -- 172.21.15.18:0/990310729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 msgr2=0x55fab495eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.507+0000 7f0dd6800640 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab495eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.507+0000 7f0dd6800640 1 -- 172.21.15.18:0/990310729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495a8f0 msgr2=0x55fab495e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:29.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.507+0000 7f0dd6800640 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495a8f0 0x55fab495e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.507+0000 7f0dd6800640 1 -- 172.21.15.18:0/990310729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fab45d23f0 con 0x55fab4968820 2024-04-16T17:10:29.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.507+0000 7f0dd6800640 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab497ca40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0013010 tx=0x7f0dd0011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5336dc9320fa666 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:29.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.507+0000 7f0db7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:29.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.508+0000 7f0db7e00640 1 -- 172.21.15.18:0/990310729 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0037060 con 0x55fab4968820 2024-04-16T17:10:29.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.508+0000 7f0db7e00640 1 -- 172.21.15.18:0/990310729 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f0dd0026040 con 0x55fab4968820 2024-04-16T17:10:29.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.508+0000 7f0ddfd97740 1 -- 172.21.15.18:0/990310729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd0003e50 con 0x55fab4968820 2024-04-16T17:10:29.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.508+0000 7f0db7e00640 1 -- 172.21.15.18:0/990310729 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0dd0003970 con 0x55fab4968820 2024-04-16T17:10:29.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.510+0000 7f0ddfd97740 10 client.27185 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:29.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.510+0000 7f0ddfd97740 1 -- 172.21.15.18:0/990310729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fab469c7d0 con 0x55fab4968820 2024-04-16T17:10:29.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.510+0000 7f0db7e00640 1 -- 172.21.15.18:0/990310729 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd0003e50 con 0x55fab4968820 2024-04-16T17:10:29.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.511+0000 7f0db7e00640 1 -- 172.21.15.18:0/990310729 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0dd002a940 con 0x55fab4968820 2024-04-16T17:10:29.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.511+0000 7f0ddfd97740 10 client.27185 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:29.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.511+0000 7f0ddfd97740 10 client.27185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:29.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.511+0000 7f0ddfd97740 20 client.27185 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:29.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.511+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fab49e33c0 0x55fab49e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:29.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.511+0000 7f0ddfd97740 4 client.27185 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:29.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.511+0000 7f0ddfd97740 1 -- 172.21.15.18:0/990310729 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fab48c78b0 con 0x55fab49e33c0 2024-04-16T17:10:29.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.512+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fab49e33c0 0x55fab49e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:29.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.513+0000 7f0db7e00640 10 client.27185 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:29.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.514+0000 7f0dd5e00640 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fab49e33c0 0x55fab49e57e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f5d7eee8912431 server_cookie=ae95bfd41875af8d in_seq=0 out_seq=0 2024-04-16T17:10:29.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.514+0000 7f0db7e00640 1 -- 172.21.15.18:0/990310729 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fab48c78b0 con 0x55fab49e33c0 2024-04-16T17:10:29.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.514+0000 7f0db7e00640 10 client.27185 handle_command_reply: tid=0 2024-04-16T17:10:29.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.514+0000 7f0ddfd97740 1 client.27185 shutdown 2024-04-16T17:10:29.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.514+0000 7f0ddfd97740 1 -- 172.21.15.18:0/990310729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 msgr2=0x55fab497ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.514+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab497ca40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0013010 tx=0x7f0dd0011e90 comp rx=0 tx=0).stop 2024-04-16T17:10:29.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.515+0000 7f0ddfd97740 1 -- 172.21.15.18:0/990310729 shutdown_connections 2024-04-16T17:10:29.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.515+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fab49e33c0 0x55fab49e57e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.515+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fab4968820 0x55fab497ca40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.515+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fab495b2a0 0x55fab495eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.515+0000 7f0ddfd97740 1 --2- 172.21.15.18:0/990310729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fab495a8f0 0x55fab495e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.515+0000 7f0db7e00640 0 client.27185 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:29.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.515+0000 7f0ddfd97740 1 -- 172.21.15.18:0/990310729 >> 172.21.15.18:0/990310729 conn(0x55fab4955640 msgr2=0x55fab4965830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.516+0000 7f0ddfd97740 1 -- 172.21.15.18:0/990310729 shutdown_connections 2024-04-16T17:10:29.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.516+0000 7f0ddfd97740 1 -- 172.21.15.18:0/990310729 wait complete. 2024-04-16T17:10:29.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.517+0000 7f0ddfd97740 20 client.27185 trim_cache size 0 max 16384 2024-04-16T17:10:29.518 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:10:29.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.518+0000 7f0dd7200640 1 -- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dbc0a3d20 msgr2=0x7f0dbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.518+0000 7f0dd7200640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dbc0a3d20 0x7f0dbc0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00115a0 tx=0x7f0dd0009500 comp rx=0 tx=0).stop 2024-04-16T17:10:29.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.518+0000 7f0dd7200640 1 -- 172.21.15.18:0/4253191239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dd8107140 msgr2=0x7f0dd819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:29.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.518+0000 7f0dd7200640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dd8107140 0x7f0dd819e980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4005fd0 tx=0x7f0dc400dd20 comp rx=0 tx=0).stop 2024-04-16T17:10:29.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.519+0000 7f0dd7200640 1 -- 172.21.15.18:0/4253191239 shutdown_connections 2024-04-16T17:10:29.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.519+0000 7f0dd7200640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dbc0a3d20 0x7f0dbc0a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.519+0000 7f0dd7200640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd81985f0 0x7f0dd8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.519+0000 7f0dd7200640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dd8107140 0x7f0dd819e980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.519+0000 7f0dd7200640 1 --2- 172.21.15.18:0/4253191239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd8102070 0x7f0dd819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:29.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.519+0000 7f0dd7200640 1 -- 172.21.15.18:0/4253191239 >> 172.21.15.18:0/4253191239 conn(0x7f0dd80fd950 msgr2=0x7f0dd80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:29.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.520+0000 7f0dd7200640 1 -- 172.21.15.18:0/4253191239 shutdown_connections 2024-04-16T17:10:29.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:29.520+0000 7f0dd7200640 1 -- 172.21.15.18:0/4253191239 wait complete. 2024-04-16T17:10:29.568 INFO:tasks.workunit.client.0.smithi018.stdout: 1 211714 3.94 MB/sec execute 162 sec latency 90.374 ms 2024-04-16T17:10:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:29 smithi043 ceph-mon[36901]: pgmap v176: 129 pgs: 129 active+clean; 41 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-04-16T17:10:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:29 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/460222733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:30.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:29 smithi022 ceph-mon[36023]: pgmap v176: 129 pgs: 129 active+clean; 41 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-04-16T17:10:30.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:29 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/460222733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:30.543 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:10:30.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:10:30.543 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:10:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.549+0000 7efdddc00640 1 -- 172.21.15.18:0/1780591090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd8109000 msgr2=0x7efdd8109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.549+0000 7efdddc00640 1 --2- 172.21.15.18:0/1780591090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd8109000 0x7efdd8109460 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009900 tx=0x7efdc402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.549+0000 7efdddc00640 1 -- 172.21.15.18:0/1780591090 shutdown_connections 2024-04-16T17:10:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.549+0000 7efdddc00640 1 --2- 172.21.15.18:0/1780591090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdd81099a0 0x7efdd8117410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.549+0000 7efdddc00640 1 --2- 172.21.15.18:0/1780591090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd8109000 0x7efdd8109460 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.549+0000 7efdddc00640 1 --2- 172.21.15.18:0/1780591090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd810f710 0x7efdd810faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.549+0000 7efdddc00640 1 -- 172.21.15.18:0/1780591090 >> 172.21.15.18:0/1780591090 conn(0x7efdd806d3e0 msgr2=0x7efdd806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.551+0000 7efdddc00640 1 -- 172.21.15.18:0/1780591090 shutdown_connections 2024-04-16T17:10:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.551+0000 7efdddc00640 1 -- 172.21.15.18:0/1780591090 wait complete. 2024-04-16T17:10:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.552+0000 7efdddc00640 1 Processor -- start 2024-04-16T17:10:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.553+0000 7efdddc00640 1 -- start start 2024-04-16T17:10:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.553+0000 7efdddc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd8109000 0x7efdd8117d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.553+0000 7efdddc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdd81099a0 0x7efdd8118260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.553+0000 7efdddc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd810f710 0x7efdd81187a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.553+0000 7efdddc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efdd811cc80 con 0x7efdd81099a0 2024-04-16T17:10:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.553+0000 7efdddc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efdd811cb00 con 0x7efdd810f710 2024-04-16T17:10:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.553+0000 7efdddc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdd811ce00 con 0x7efdd8109000 2024-04-16T17:10:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efddd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd810f710 0x7efdd81187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efddd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd810f710 0x7efdd81187a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37242/0 (socket says 172.21.15.18:37242) 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efddd200640 1 -- 172.21.15.18:0/3604725026 learned_addr learned my addr 172.21.15.18:0/3604725026 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efdd7e00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd8109000 0x7efdd8117d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efdd7400640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdd81099a0 0x7efdd8118260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efddd200640 1 -- 172.21.15.18:0/3604725026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd8109000 msgr2=0x7efdd8117d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efddd200640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd8109000 0x7efdd8117d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efddd200640 1 -- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdd81099a0 msgr2=0x7efdd8118260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efddd200640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdd81099a0 0x7efdd8118260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efddd200640 1 -- 172.21.15.18:0/3604725026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd81b3650 con 0x7efdd810f710 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.554+0000 7efdd7e00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd8109000 0x7efdd8117d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.555+0000 7efddd200640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd810f710 0x7efdd81187a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efdd000eb40 tx=0x7efdd000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.555+0000 7efdd4c00640 1 -- 172.21.15.18:0/3604725026 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0004b20 con 0x7efdd810f710 2024-04-16T17:10:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.555+0000 7efdddc00640 1 -- 172.21.15.18:0/3604725026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd81a2d70 con 0x7efdd810f710 2024-04-16T17:10:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.555+0000 7efdd4c00640 1 -- 172.21.15.18:0/3604725026 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd0004cc0 con 0x7efdd810f710 2024-04-16T17:10:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.555+0000 7efdd4c00640 1 -- 172.21.15.18:0/3604725026 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0010760 con 0x7efdd810f710 2024-04-16T17:10:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.555+0000 7efdddc00640 1 -- 172.21.15.18:0/3604725026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd81a3290 con 0x7efdd810f710 2024-04-16T17:10:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.559+0000 7efde2036740 1 Processor -- start 2024-04-16T17:10:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.561+0000 7efdd4c00640 1 -- 172.21.15.18:0/3604725026 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efdd0010900 con 0x7efdd810f710 2024-04-16T17:10:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.561+0000 7efde2036740 1 -- start start 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.561+0000 7efde2036740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 0x555626a1a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.561+0000 7efde2036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1aa80 0x555626a1af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.561+0000 7efde2036740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a24000 0x555626a263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.561+0000 7efde2036740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55562697f780 con 0x555626a24000 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.561+0000 7efde2036740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555626925f40 con 0x555626a1b7f0 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.561+0000 7efde2036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55562697b740 con 0x555626a1aa80 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1aa80 0x555626a1af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 0x555626a1a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1aa80 0x555626a1af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48540/0 (socket says 172.21.15.18:48540) 2024-04-16T17:10:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7e00640 1 -- 172.21.15.18:0/1903542198 learned_addr learned my addr 172.21.15.18:0/1903542198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7400640 1 --2- 172.21.15.18:0/1903542198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a24000 0x555626a263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7e00640 1 -- 172.21.15.18:0/1903542198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 msgr2=0x555626a1a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7e00640 1 --2- 172.21.15.18:0/1903542198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 0x555626a1a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7e00640 1 -- 172.21.15.18:0/1903542198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a24000 msgr2=0x555626a263f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7e00640 1 --2- 172.21.15.18:0/1903542198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a24000 0x555626a263f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.562+0000 7efdd7e00640 1 -- 172.21.15.18:0/1903542198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555626692210 con 0x555626a1aa80 2024-04-16T17:10:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.563+0000 7efdd7e00640 1 --2- 172.21.15.18:0/1903542198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1aa80 0x555626a1af00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efdc8008c60 tx=0x7efdc800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b35affc369a4e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.563+0000 7efdcd600640 1 -- 172.21.15.18:0/1903542198 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8014070 con 0x555626a1aa80 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.563+0000 7efdcd600640 1 -- 172.21.15.18:0/1903542198 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc80047a0 con 0x555626a1aa80 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.563+0000 7efdcd600640 1 -- 172.21.15.18:0/1903542198 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8010760 con 0x555626a1aa80 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.563+0000 7efdd4c00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efdbc0a3e00 0x7efdbc0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.563+0000 7efdd4c00640 1 -- 172.21.15.18:0/3604725026 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7efdd00d1280 con 0x7efdd810f710 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.564+0000 7efdd7400640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efdbc0a3e00 0x7efdbc0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.564+0000 7efde2036740 1 -- 172.21.15.18:0/1903542198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1aa80 msgr2=0x555626a1af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.564+0000 7efde2036740 1 --2- 172.21.15.18:0/1903542198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1aa80 0x555626a1af00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efdc8008c60 tx=0x7efdc800e720 comp rx=0 tx=0).stop 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.564+0000 7efde2036740 1 -- 172.21.15.18:0/1903542198 shutdown_connections 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.564+0000 7efde2036740 1 --2- 172.21.15.18:0/1903542198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a24000 0x555626a263f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.564+0000 7efde2036740 1 --2- 172.21.15.18:0/1903542198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1aa80 0x555626a1af00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.564+0000 7efde2036740 1 --2- 172.21.15.18:0/1903542198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 0x555626a1a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.564+0000 7efde2036740 1 -- 172.21.15.18:0/1903542198 >> 172.21.15.18:0/1903542198 conn(0x5556269b55d0 msgr2=0x555626a15770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.565+0000 7efde2036740 1 -- 172.21.15.18:0/1903542198 shutdown_connections 2024-04-16T17:10:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.565+0000 7efde2036740 1 -- 172.21.15.18:0/1903542198 wait complete. 2024-04-16T17:10:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.565+0000 7efde2036740 1 -- start start 2024-04-16T17:10:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.566+0000 7efde2036740 1 Processor -- start 2024-04-16T17:10:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.566+0000 7efdd7400640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efdbc0a3e00 0x7efdbc0a62c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006010 tx=0x7efdc4000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.566+0000 7efde2036740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a20e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.566+0000 7efde2036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a213a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efde2036740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efde2036740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555626a26d30 con 0x555626a1aa80 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efde2036740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555626939390 con 0x555626a24000 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efde2036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555626a26eb0 con 0x555626a1b7f0 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efde2036740 10 client.0 _ll_register_callbacks cb 0x555626922f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efddd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efddd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a218e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37264/0 (socket says 172.21.15.18:37264) 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a20e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efddd200640 1 -- 172.21.15.18:0/1212932625 learned_addr learned my addr 172.21.15.18:0/1212932625 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.567+0000 7efdd7400640 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efddd200640 1 -- 172.21.15.18:0/1212932625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 msgr2=0x555626a213a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efddd200640 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a213a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efddd200640 1 -- 172.21.15.18:0/1212932625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 msgr2=0x555626a20e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efddd200640 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a20e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efddd200640 1 -- 172.21.15.18:0/1212932625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55562675e620 con 0x555626a24000 2024-04-16T17:10:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efddd200640 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a218e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efdd00bbfd0 tx=0x7efdd009c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9c039644ef27466 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efdc2000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efdc2000640 1 -- 172.21.15.18:0/1212932625 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd009ce10 con 0x555626a24000 2024-04-16T17:10:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.568+0000 7efdc2000640 1 -- 172.21.15.18:0/1212932625 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7efdd00b6260 con 0x555626a24000 2024-04-16T17:10:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.569+0000 7efde2036740 1 -- 172.21.15.18:0/1212932625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc80047a0 con 0x555626a24000 2024-04-16T17:10:30.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 212803 3.98 MB/sec execute 163 sec latency 41.403 ms 2024-04-16T17:10:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.569+0000 7efdc2000640 1 -- 172.21.15.18:0/1212932625 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdd00bb050 con 0x555626a24000 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.569+0000 7efde2036740 10 client.27197 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.569+0000 7efde2036740 1 -- 172.21.15.18:0/1212932625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555626a32ea0 con 0x555626a24000 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.569+0000 7efdc2000640 1 -- 172.21.15.18:0/1212932625 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc80047a0 con 0x555626a24000 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.570+0000 7efdc2000640 1 -- 172.21.15.18:0/1212932625 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7efdd009d070 con 0x555626a24000 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.570+0000 7efde2036740 10 client.27197 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.570+0000 7efde2036740 10 client.27197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.570+0000 7efde2036740 20 client.27197 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.570+0000 7efde2036740 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555626a34310 0x555626a36700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.570+0000 7efde2036740 4 client.27197 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.570+0000 7efde2036740 1 -- 172.21.15.18:0/1212932625 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55562698d1a0 con 0x555626a34310 2024-04-16T17:10:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.570+0000 7efdd7e00640 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555626a34310 0x555626a36700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.571+0000 7efdd7e00640 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555626a34310 0x555626a36700 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1141646ef34c2870 server_cookie=698a313739231366 in_seq=0 out_seq=0 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.571+0000 7efdc2000640 10 client.27197 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.571+0000 7efdc2000640 1 -- 172.21.15.18:0/1212932625 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55562698d1a0 con 0x555626a34310 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.571+0000 7efdc2000640 10 client.27197 handle_command_reply: tid=0 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 client.27197 shutdown 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 -- 172.21.15.18:0/1212932625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 msgr2=0x555626a218e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a218e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efdd00bbfd0 tx=0x7efdd009c770 comp rx=0 tx=0).stop 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 -- 172.21.15.18:0/1212932625 shutdown_connections 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555626a34310 0x555626a36700 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a218e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a213a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 --2- 172.21.15.18:0/1212932625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a20e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 -- 172.21.15.18:0/1212932625 >> 172.21.15.18:0/1212932625 conn(0x5556269b55d0 msgr2=0x555626a174c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efdc2000640 0 client.27197 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.572+0000 7efde2036740 1 -- 172.21.15.18:0/1212932625 shutdown_connections 2024-04-16T17:10:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.573+0000 7efde2036740 1 -- 172.21.15.18:0/1212932625 wait complete. 2024-04-16T17:10:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.573+0000 7efde2036740 20 client.27197 trim_cache size 0 max 16384 2024-04-16T17:10:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.603+0000 7efde2036740 1 Processor -- start 2024-04-16T17:10:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.603+0000 7efde2036740 1 -- start start 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.603+0000 7efde2036740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a23590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.603+0000 7efde2036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a2fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.603+0000 7efde2036740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a1ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.603+0000 7efde2036740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b8010 con 0x555626a1aa80 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.603+0000 7efde2036740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b58e0 con 0x555626a24000 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.603+0000 7efde2036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b3820 con 0x555626a1b7f0 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a23590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a2fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a2fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48554/0 (socket says 172.21.15.18:48554) 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a23590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44660/0 (socket says 172.21.15.18:44660) 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7400640 1 -- 172.21.15.18:0/340133692 learned_addr learned my addr 172.21.15.18:0/340133692 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efddd200640 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a1ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7e00640 1 -- 172.21.15.18:0/340133692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 msgr2=0x555626a2fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7e00640 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a2fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7e00640 1 -- 172.21.15.18:0/340133692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 msgr2=0x555626a1ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7e00640 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a1ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efdd7e00640 1 -- 172.21.15.18:0/340133692 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555626a36a70 con 0x555626a1aa80 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.604+0000 7efddd200640 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a1ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.605+0000 7efdd7e00640 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a23590 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efdc800ec20 tx=0x7efdc8008310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dde1ec28c26b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.605+0000 7efdccc00640 1 -- 172.21.15.18:0/340133692 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8014040 con 0x555626a1aa80 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.605+0000 7efdccc00640 1 -- 172.21.15.18:0/340133692 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc8005560 con 0x555626a1aa80 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.605+0000 7efdccc00640 1 -- 172.21.15.18:0/340133692 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc80108a0 con 0x555626a1aa80 2024-04-16T17:10:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.606+0000 7efde2036740 1 -- 172.21.15.18:0/340133692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 msgr2=0x555626a23590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.606+0000 7efde2036740 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a23590 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efdc800ec20 tx=0x7efdc8008310 comp rx=0 tx=0).stop 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.606+0000 7efde2036740 1 -- 172.21.15.18:0/340133692 shutdown_connections 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.606+0000 7efde2036740 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a24000 0x555626a1ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.606+0000 7efde2036740 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a1b7f0 0x555626a2fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.606+0000 7efde2036740 1 --2- 172.21.15.18:0/340133692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a23590 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.606+0000 7efde2036740 1 -- 172.21.15.18:0/340133692 >> 172.21.15.18:0/340133692 conn(0x5556269b55d0 msgr2=0x555626a20e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.606+0000 7efde2036740 1 -- 172.21.15.18:0/340133692 shutdown_connections 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.607+0000 7efde2036740 1 -- 172.21.15.18:0/340133692 wait complete. 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.607+0000 7efde2036740 1 -- start start 2024-04-16T17:10:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.607+0000 7efde2036740 1 Processor -- start 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efde2036740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a24a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efde2036740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 0x555626a24fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efde2036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a24000 0x555626a254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efde2036740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b3da0 con 0x555626a1aa80 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efde2036740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b5210 con 0x555626a1b7f0 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efde2036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81b4470 con 0x555626a24000 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efde2036740 10 client.0 _ll_register_callbacks cb 0x7efdd009cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a24a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a24a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44676/0 (socket says 172.21.15.18:44676) 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efdd7e00640 1 -- 172.21.15.18:0/2946577711 learned_addr learned my addr 172.21.15.18:0/2946577711 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efddd200640 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a24000 0x555626a254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.608+0000 7efdd7400640 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 0x555626a24fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdd7e00640 1 -- 172.21.15.18:0/2946577711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a24000 msgr2=0x555626a254e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdd7e00640 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a24000 0x555626a254e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdd7e00640 1 -- 172.21.15.18:0/2946577711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 msgr2=0x555626a24fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdd7e00640 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 0x555626a24fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdd7e00640 1 -- 172.21.15.18:0/2946577711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555626a2a2f0 con 0x555626a1aa80 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdd7e00640 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a24a60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009010 tx=0x7efdc8005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa43425fa2a5837 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdc2000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdc2000640 1 -- 172.21.15.18:0/2946577711 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8014040 con 0x555626a1aa80 2024-04-16T17:10:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efde2036740 1 -- 172.21.15.18:0/2946577711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc8005560 con 0x555626a1aa80 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.609+0000 7efdc2000640 1 -- 172.21.15.18:0/2946577711 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7efdc8005360 con 0x555626a1aa80 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.610+0000 7efdc2000640 1 -- 172.21.15.18:0/2946577711 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdc803b050 con 0x555626a1aa80 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.610+0000 7efde2036740 10 client.16971 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.610+0000 7efde2036740 1 -- 172.21.15.18:0/2946577711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555626a9ab40 con 0x555626a1aa80 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.610+0000 7efdc2000640 1 -- 172.21.15.18:0/2946577711 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc8005560 con 0x555626a1aa80 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.610+0000 7efdc2000640 1 -- 172.21.15.18:0/2946577711 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7efdc8027390 con 0x555626a1aa80 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.611+0000 7efde2036740 10 client.16971 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.611+0000 7efde2036740 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.611+0000 7efde2036740 20 client.16971 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.611+0000 7efde2036740 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555626a9afb0 0x555626a9d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.611+0000 7efde2036740 4 client.16971 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.611+0000 7efde2036740 1 -- 172.21.15.18:0/2946577711 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555626987010 con 0x555626a9afb0 2024-04-16T17:10:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.611+0000 7efdd7400640 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555626a9afb0 0x555626a9d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.612+0000 7efdc2000640 10 client.16971 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.612+0000 7efdd7400640 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555626a9afb0 0x555626a9d3a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c41b6ab2b7bf22 server_cookie=4b99020d264892e6 in_seq=0 out_seq=0 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.755+0000 7efdc2000640 1 -- 172.21.15.18:0/2946577711 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555626987010 con 0x555626a9afb0 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.755+0000 7efdc2000640 10 client.16971 handle_command_reply: tid=0 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.755+0000 7efde2036740 1 client.16971 shutdown 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.755+0000 7efde2036740 1 -- 172.21.15.18:0/2946577711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 msgr2=0x555626a24a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.755+0000 7efde2036740 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a24a60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009010 tx=0x7efdc8005b10 comp rx=0 tx=0).stop 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 1 -- 172.21.15.18:0/2946577711 shutdown_connections 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555626a9afb0 0x555626a9d3a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555626a24000 0x555626a254e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555626a1b7f0 0x555626a24fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 1 --2- 172.21.15.18:0/2946577711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555626a1aa80 0x555626a24a60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 1 -- 172.21.15.18:0/2946577711 >> 172.21.15.18:0/2946577711 conn(0x5556269b55d0 msgr2=0x555626a21610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efdc2000640 0 client.16971 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 1 -- 172.21.15.18:0/2946577711 shutdown_connections 2024-04-16T17:10:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 1 -- 172.21.15.18:0/2946577711 wait complete. 2024-04-16T17:10:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.756+0000 7efde2036740 20 client.16971 trim_cache size 0 max 16384 2024-04-16T17:10:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:10:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:10:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:10:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:10:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 -- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efdbc0a3e00 msgr2=0x7efdbc0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efdbc0a3e00 0x7efdbc0a62c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006010 tx=0x7efdc4000bc0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 -- 172.21.15.18:0/3604725026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd810f710 msgr2=0x7efdd81187a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd810f710 0x7efdd81187a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efdd000eb40 tx=0x7efdd000c2d0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 -- 172.21.15.18:0/3604725026 shutdown_connections 2024-04-16T17:10:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efdbc0a3e00 0x7efdbc0a62c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdd810f710 0x7efdd81187a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdd81099a0 0x7efdd8118260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 --2- 172.21.15.18:0/3604725026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd8109000 0x7efdd8117d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 -- 172.21.15.18:0/3604725026 >> 172.21.15.18:0/3604725026 conn(0x7efdd806d3e0 msgr2=0x7efdd8072600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 -- 172.21.15.18:0/3604725026 shutdown_connections 2024-04-16T17:10:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:30.757+0000 7efdddc00640 1 -- 172.21.15.18:0/3604725026 wait complete. 2024-04-16T17:10:30.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 -- 172.21.15.18:0/2452111887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103b90 msgr2=0x7f33a010c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 --2- 172.21.15.18:0/2452111887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103b90 0x7f33a010c070 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f338c009900 tx=0x7f338c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 -- 172.21.15.18:0/2452111887 shutdown_connections 2024-04-16T17:10:30.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 --2- 172.21.15.18:0/2452111887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a010c5b0 0x7f33a010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 --2- 172.21.15.18:0/2452111887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103b90 0x7f33a010c070 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 --2- 172.21.15.18:0/2452111887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33a0103270 0x7f33a0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 -- 172.21.15.18:0/2452111887 >> 172.21.15.18:0/2452111887 conn(0x7f33a00780b0 msgr2=0x7f33a00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 -- 172.21.15.18:0/2452111887 shutdown_connections 2024-04-16T17:10:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.801+0000 7f33a5c00640 1 -- 172.21.15.18:0/2452111887 wait complete. 2024-04-16T17:10:30.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.803+0000 7f33a5c00640 1 Processor -- start 2024-04-16T17:10:30.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.803+0000 7f33a5c00640 1 -- start start 2024-04-16T17:10:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.803+0000 7f33a5c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103270 0x7f33a01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.803+0000 7f33a5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0103b90 0x7f33a01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.803+0000 7f33a5c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33a010c5b0 0x7f33a019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.803+0000 7f33a5c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01123c0 con 0x7f33a010c5b0 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.803+0000 7f33a5c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0112240 con 0x7f33a0103270 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.803+0000 7f33a5c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0112540 con 0x7f33a0103b90 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103270 0x7f33a01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103270 0x7f33a01a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37312/0 (socket says 172.21.15.18:37312) 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 -- 172.21.15.18:0/4082954782 learned_addr learned my addr 172.21.15.18:0/4082954782 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339e000640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0103b90 0x7f33a01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 -- 172.21.15.18:0/4082954782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0103b90 msgr2=0x7f33a01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0103b90 0x7f33a01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 -- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33a010c5b0 msgr2=0x7f33a019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33a010c5b0 0x7f33a019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 -- 172.21.15.18:0/4082954782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a019afe0 con 0x7f33a0103270 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.804+0000 7f339ea00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103270 0x7f33a01a06b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f339000ea60 tx=0x7f339000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.805+0000 7f3397400640 1 -- 172.21.15.18:0/4082954782 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339000cde0 con 0x7f33a0103270 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.805+0000 7f3397400640 1 -- 172.21.15.18:0/4082954782 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33900035f0 con 0x7f33a0103270 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.805+0000 7f3397400640 1 -- 172.21.15.18:0/4082954782 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390010670 con 0x7f33a0103270 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.805+0000 7f33a5c00640 1 -- 172.21.15.18:0/4082954782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a019b2d0 con 0x7f33a0103270 2024-04-16T17:10:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.805+0000 7f33a5c00640 1 -- 172.21.15.18:0/4082954782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a0104010 con 0x7f33a0103270 2024-04-16T17:10:30.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.809+0000 7f33a896b740 1 Processor -- start 2024-04-16T17:10:30.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.809+0000 7f33a896b740 1 -- start start 2024-04-16T17:10:30.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f33a896b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f33a896b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f10bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f33a896b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 0x56085f10e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f33a896b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56085f067cb0 con 0x56085f102ca0 2024-04-16T17:10:30.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f33a896b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56085f00e370 con 0x56085f1035c0 2024-04-16T17:10:30.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f33a896b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56085f063c70 con 0x56085f10c1a0 2024-04-16T17:10:30.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f339f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f339f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44708/0 (socket says 172.21.15.18:44708) 2024-04-16T17:10:30.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f339f400640 1 -- 172.21.15.18:0/2335475069 learned_addr learned my addr 172.21.15.18:0/2335475069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f339e000640 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 0x56085f10e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.810+0000 7f3397400640 1 -- 172.21.15.18:0/4082954782 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33900037a0 con 0x7f33a0103270 2024-04-16T17:10:30.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.811+0000 7f339f400640 1 -- 172.21.15.18:0/2335475069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 msgr2=0x56085f10e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.811+0000 7f339f400640 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 0x56085f10e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.811+0000 7f339f400640 1 -- 172.21.15.18:0/2335475069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 msgr2=0x56085f10bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.811+0000 7f339ea00640 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f10bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.811+0000 7f339f400640 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f10bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.811+0000 7f339f400640 1 -- 172.21.15.18:0/2335475069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085f02d910 con 0x56085f102ca0 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.811+0000 7f339ea00640 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f10bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.811+0000 7f339f400640 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f103080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3398007c50 tx=0x7f339800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad4cd21067d887d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.812+0000 7f338be00640 1 -- 172.21.15.18:0/2335475069 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398014070 con 0x56085f102ca0 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.812+0000 7f338be00640 1 -- 172.21.15.18:0/2335475069 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339800fb30 con 0x56085f102ca0 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.812+0000 7f338be00640 1 -- 172.21.15.18:0/2335475069 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800eb80 con 0x56085f102ca0 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.812+0000 7f3397400640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f33840a3da0 0x7f33840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.812+0000 7f3397400640 1 -- 172.21.15.18:0/4082954782 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f33900d0f80 con 0x7f33a0103270 2024-04-16T17:10:30.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.813+0000 7f339ea00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f33840a3da0 0x7f33840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.813+0000 7f33a896b740 1 -- 172.21.15.18:0/2335475069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 msgr2=0x56085f103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.813+0000 7f33a896b740 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f103080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3398007c50 tx=0x7f339800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f33a896b740 1 -- 172.21.15.18:0/2335475069 shutdown_connections 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f33a896b740 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 0x56085f10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f33a896b740 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f10bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f33a896b740 1 --2- 172.21.15.18:0/2335475069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f103080 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f33a896b740 1 -- 172.21.15.18:0/2335475069 >> 172.21.15.18:0/2335475069 conn(0x56085f0fd3d0 msgr2=0x56085f0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f33a896b740 1 -- 172.21.15.18:0/2335475069 shutdown_connections 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f339ea00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f33840a3da0 0x7f33840a6260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56085f102770 tx=0x7f339000ef90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f33a896b740 1 -- 172.21.15.18:0/2335475069 wait complete. 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.814+0000 7f33a896b740 1 -- start start 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f33a896b740 1 Processor -- start 2024-04-16T17:10:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f33a896b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f33a896b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f109710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f33a896b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f10c1a0 0x56085f109c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f33a896b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56085f10ef90 con 0x56085f102ca0 2024-04-16T17:10:30.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f33a896b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56085f021730 con 0x56085f10c1a0 2024-04-16T17:10:30.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f33a896b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56085f10f110 con 0x56085f1035c0 2024-04-16T17:10:30.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f33a896b740 10 client.0 _ll_register_callbacks cb 0x56085eff01e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:30.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.815+0000 7f339e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f109710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f109710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48598/0 (socket says 172.21.15.18:48598) 2024-04-16T17:10:30.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339e000640 1 -- 172.21.15.18:0/3652816433 learned_addr learned my addr 172.21.15.18:0/3652816433 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339f400640 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f10c1a0 0x56085f109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339f400640 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339e000640 1 -- 172.21.15.18:0/3652816433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f10c1a0 msgr2=0x56085f109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339e000640 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f10c1a0 0x56085f109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339e000640 1 -- 172.21.15.18:0/3652816433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 msgr2=0x56085f1091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339e000640 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f1091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339e000640 1 -- 172.21.15.18:0/3652816433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085ee427e0 con 0x56085f1035c0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339f400640 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f339e000640 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f109710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f338c009230 tx=0x7f338c0045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca37cd5f823c9c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f3388c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f3388c00640 1 -- 172.21.15.18:0/3652816433 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c044070 con 0x56085f1035c0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f3388c00640 1 -- 172.21.15.18:0/3652816433 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f338c0046c0 con 0x56085f1035c0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.816+0000 7f33a896b740 1 -- 172.21.15.18:0/3652816433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339800fb30 con 0x56085f1035c0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f3388c00640 1 -- 172.21.15.18:0/3652816433 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f338c0048c0 con 0x56085f1035c0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f33a896b740 10 client.27172 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f33a896b740 1 -- 172.21.15.18:0/3652816433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56085eea2f70 con 0x56085f1035c0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f3388c00640 1 -- 172.21.15.18:0/3652816433 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339800fb30 con 0x56085f1035c0 2024-04-16T17:10:30.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f3388c00640 1 -- 172.21.15.18:0/3652816433 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f338c0036f0 con 0x56085f1035c0 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f33a896b740 10 client.27172 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f33a896b740 10 client.27172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f33a896b740 20 client.27172 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f33a896b740 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56085f11c490 0x56085f11e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f33a896b740 4 client.27172 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.817+0000 7f33a896b740 1 -- 172.21.15.18:0/3652816433 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56085f0756d0 con 0x56085f11c490 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.818+0000 7f339f400640 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56085f11c490 0x56085f11e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.818+0000 7f339f400640 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56085f11c490 0x56085f11e8e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea8bd81b5c1b546 server_cookie=33ed0f17d1405d80 in_seq=0 out_seq=0 2024-04-16T17:10:30.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.818+0000 7f3388c00640 10 client.27172 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:30.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f3388c00640 1 -- 172.21.15.18:0/3652816433 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56085f0756d0 con 0x56085f11c490 2024-04-16T17:10:30.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f3388c00640 10 client.27172 handle_command_reply: tid=0 2024-04-16T17:10:30.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 client.27172 shutdown 2024-04-16T17:10:30.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 -- 172.21.15.18:0/3652816433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 msgr2=0x56085f109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f109710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f338c009230 tx=0x7f338c0045f0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 -- 172.21.15.18:0/3652816433 shutdown_connections 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56085f11c490 0x56085f11e8e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f3388c00640 0 client.27172 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f10c1a0 0x56085f109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f109710 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 --2- 172.21.15.18:0/3652816433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 -- 172.21.15.18:0/3652816433 >> 172.21.15.18:0/3652816433 conn(0x56085f0fd3d0 msgr2=0x56085f0ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.819+0000 7f33a896b740 1 -- 172.21.15.18:0/3652816433 shutdown_connections 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.820+0000 7f33a896b740 1 -- 172.21.15.18:0/3652816433 wait complete. 2024-04-16T17:10:30.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.820+0000 7f33a896b740 20 client.27172 trim_cache size 0 max 16384 2024-04-16T17:10:30.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f33a896b740 1 Processor -- start 2024-04-16T17:10:30.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f33a896b740 1 -- start start 2024-04-16T17:10:30.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f33a896b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f10b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f33a896b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f117e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f33a896b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 0x56085f0fef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f33a896b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b8440 con 0x56085f102ca0 2024-04-16T17:10:30.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f33a896b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b5d10 con 0x56085f1035c0 2024-04-16T17:10:30.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f33a896b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b3ba0 con 0x56085f10c1a0 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f339f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f10b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f339e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f117e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f339f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f10b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44726/0 (socket says 172.21.15.18:44726) 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.842+0000 7f339f400640 1 -- 172.21.15.18:0/1541140893 learned_addr learned my addr 172.21.15.18:0/1541140893 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f339f400640 1 --2- 172.21.15.18:0/1541140893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 0x56085f0fef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f339e000640 1 -- 172.21.15.18:0/1541140893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 msgr2=0x56085f0fef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f339e000640 1 --2- 172.21.15.18:0/1541140893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 0x56085f0fef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f339e000640 1 -- 172.21.15.18:0/1541140893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 msgr2=0x56085f10b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f339e000640 1 --2- 172.21.15.18:0/1541140893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f10b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f339e000640 1 -- 172.21.15.18:0/1541140893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085ed3a170 con 0x56085f1035c0 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f339e000640 1 --2- 172.21.15.18:0/1541140893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f117e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f338c003a90 tx=0x7f338c02bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2214b188718fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f338b400640 1 -- 172.21.15.18:0/1541140893 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c044040 con 0x56085f1035c0 2024-04-16T17:10:30.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f338b400640 1 -- 172.21.15.18:0/1541140893 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f338c003d00 con 0x56085f1035c0 2024-04-16T17:10:30.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.843+0000 7f338b400640 1 -- 172.21.15.18:0/1541140893 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c004a60 con 0x56085f1035c0 2024-04-16T17:10:30.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 -- 172.21.15.18:0/1541140893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 msgr2=0x56085f117e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 --2- 172.21.15.18:0/1541140893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f117e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f338c003a90 tx=0x7f338c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 -- 172.21.15.18:0/1541140893 shutdown_connections 2024-04-16T17:10:30.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 --2- 172.21.15.18:0/1541140893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f10c1a0 0x56085f0fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 --2- 172.21.15.18:0/1541140893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f1035c0 0x56085f117e50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 --2- 172.21.15.18:0/1541140893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f102ca0 0x56085f10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 -- 172.21.15.18:0/1541140893 >> 172.21.15.18:0/1541140893 conn(0x56085f0fd3d0 msgr2=0x56085f0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 -- 172.21.15.18:0/1541140893 shutdown_connections 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.844+0000 7f33a896b740 1 -- 172.21.15.18:0/1541140893 wait complete. 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.845+0000 7f33a896b740 1 -- start start 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.845+0000 7f33a896b740 1 Processor -- start 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f33a896b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f102ca0 0x56085f10ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f33a896b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f1104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f33a896b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f10c1a0 0x56085f1109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f33a896b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b41f0 con 0x56085f10c1a0 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f33a896b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b5640 con 0x56085f102ca0 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f33a896b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b48a0 con 0x56085f1035c0 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f33a896b740 10 client.0 _ll_register_callbacks cb 0x7f338c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f339f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f102ca0 0x56085f10ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f339f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f102ca0 0x56085f10ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37352/0 (socket says 172.21.15.18:37352) 2024-04-16T17:10:30.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f339f400640 1 -- 172.21.15.18:0/435420195 learned_addr learned my addr 172.21.15.18:0/435420195 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:30.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.846+0000 7f339f400640 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f10c1a0 0x56085f1109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f339f400640 1 -- 172.21.15.18:0/435420195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 msgr2=0x56085f1104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f339f400640 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f1104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f339f400640 1 -- 172.21.15.18:0/435420195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f10c1a0 msgr2=0x56085f1109e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:30.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f339f400640 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f10c1a0 0x56085f1109e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f339f400640 1 -- 172.21.15.18:0/435420195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085f181c60 con 0x56085f102ca0 2024-04-16T17:10:30.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f339f400640 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f102ca0 0x56085f10ff60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f339800d010 tx=0x7f3398000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=effda6102b4ff8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:30.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f3388c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:30.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f3388c00640 1 -- 172.21.15.18:0/435420195 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398014040 con 0x56085f102ca0 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.847+0000 7f3388c00640 1 -- 172.21.15.18:0/435420195 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f33980265a0 con 0x56085f102ca0 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.848+0000 7f33a896b740 1 -- 172.21.15.18:0/435420195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f338c003d00 con 0x56085f102ca0 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.848+0000 7f3388c00640 1 -- 172.21.15.18:0/435420195 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3398037050 con 0x56085f102ca0 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.848+0000 7f33a896b740 10 client.27218 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.848+0000 7f33a896b740 1 -- 172.21.15.18:0/435420195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56085f182fe0 con 0x56085f102ca0 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.848+0000 7f3388c00640 1 -- 172.21.15.18:0/435420195 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f338c003d00 con 0x56085f102ca0 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f3388c00640 1 -- 172.21.15.18:0/435420195 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3398004040 con 0x56085f102ca0 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f33a896b740 10 client.27218 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f33a896b740 10 client.27218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f33a896b740 20 client.27218 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:30.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f33a896b740 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56085f1833e0 0x56085f185840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f33a896b740 4 client.27218 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f33a896b740 1 -- 172.21.15.18:0/435420195 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56085f06f540 con 0x56085f1833e0 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f339e000640 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56085f1833e0 0x56085f185840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f339e000640 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56085f1833e0 0x56085f185840 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae1363633d79235d server_cookie=5c31e68b2f676bfe in_seq=0 out_seq=0 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.849+0000 7f3388c00640 10 client.27218 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.851+0000 7f3388c00640 1 -- 172.21.15.18:0/435420195 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56085f06f540 con 0x56085f1833e0 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.851+0000 7f3388c00640 10 client.27218 handle_command_reply: tid=0 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.851+0000 7f33a896b740 1 client.27218 shutdown 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.851+0000 7f33a896b740 1 -- 172.21.15.18:0/435420195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f102ca0 msgr2=0x56085f10ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.851+0000 7f33a896b740 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f102ca0 0x56085f10ff60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f339800d010 tx=0x7f3398000f40 comp rx=0 tx=0).stop 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 1 -- 172.21.15.18:0/435420195 shutdown_connections 2024-04-16T17:10:30.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56085f1833e0 0x56085f185840 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56085f10c1a0 0x56085f1109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56085f1035c0 0x56085f1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 1 --2- 172.21.15.18:0/435420195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56085f102ca0 0x56085f10ff60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 1 -- 172.21.15.18:0/435420195 >> 172.21.15.18:0/435420195 conn(0x56085f0fd3d0 msgr2=0x56085f0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f3388c00640 0 client.27218 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 1 -- 172.21.15.18:0/435420195 shutdown_connections 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 1 -- 172.21.15.18:0/435420195 wait complete. 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.852+0000 7f33a896b740 20 client.27218 trim_cache size 0 max 16384 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:10:30.855 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:10:30.856 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "8a449653-c209-4d8b-bfc8-dc4a00ceaeb0", 2024-04-16T17:10:30.856 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:10:30.856 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:10:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 -- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f33840a3da0 msgr2=0x7f33840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f33840a3da0 0x7f33840a6260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56085f102770 tx=0x7f339000ef90 comp rx=0 tx=0).stop 2024-04-16T17:10:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 -- 172.21.15.18:0/4082954782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103270 msgr2=0x7f33a01a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:30.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103270 0x7f33a01a06b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f339000ea60 tx=0x7f339000ef20 comp rx=0 tx=0).stop 2024-04-16T17:10:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 -- 172.21.15.18:0/4082954782 shutdown_connections 2024-04-16T17:10:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f33840a3da0 0x7f33840a6260 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33a010c5b0 0x7f33a019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0103b90 0x7f33a01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 --2- 172.21.15.18:0/4082954782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33a0103270 0x7f33a01a06b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 -- 172.21.15.18:0/4082954782 >> 172.21.15.18:0/4082954782 conn(0x7f33a00780b0 msgr2=0x7f33a0108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 -- 172.21.15.18:0/4082954782 shutdown_connections 2024-04-16T17:10:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:30.853+0000 7f33a5c00640 1 -- 172.21.15.18:0/4082954782 wait complete. 2024-04-16T17:10:30.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:10:30.872 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:10:31.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.118+0000 7ff60e600640 1 -- 172.21.15.18:0/3547438359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff608074cf0 msgr2=0x7ff60810e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.118+0000 7ff60e600640 1 --2- 172.21.15.18:0/3547438359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff608074cf0 0x7ff60810e160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009900 tx=0x7ff5f402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.118+0000 7ff60e600640 1 -- 172.21.15.18:0/3547438359 shutdown_connections 2024-04-16T17:10:31.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.118+0000 7ff60e600640 1 --2- 172.21.15.18:0/3547438359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60810e6a0 0x7ff608110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.118+0000 7ff60e600640 1 --2- 172.21.15.18:0/3547438359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff608074cf0 0x7ff60810e160 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.118+0000 7ff60e600640 1 --2- 172.21.15.18:0/3547438359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6080743d0 0x7ff6080747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.118+0000 7ff60e600640 1 -- 172.21.15.18:0/3547438359 >> 172.21.15.18:0/3547438359 conn(0x7ff6080fd950 msgr2=0x7ff6080ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:31.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.119+0000 7ff60e600640 1 -- 172.21.15.18:0/3547438359 shutdown_connections 2024-04-16T17:10:31.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.119+0000 7ff60e600640 1 -- 172.21.15.18:0/3547438359 wait complete. 2024-04-16T17:10:31.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.119+0000 7ff60e600640 1 Processor -- start 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60e600640 1 -- start start 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6080743d0 0x7ff6081a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff608074cf0 0x7ff6081a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60e600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60810e6a0 0x7ff60819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60e600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff608114450 con 0x7ff608074cf0 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60e600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081142d0 con 0x7ff60810e6a0 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081145d0 con 0x7ff6080743d0 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff607e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff608074cf0 0x7ff6081a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff607e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff608074cf0 0x7ff6081a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44748/0 (socket says 172.21.15.18:44748) 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff607e00640 1 -- 172.21.15.18:0/2521945474 learned_addr learned my addr 172.21.15.18:0/2521945474 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60dc00640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60810e6a0 0x7ff60819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.120+0000 7ff60d200640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6080743d0 0x7ff6081a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.121+0000 7ff607e00640 1 -- 172.21.15.18:0/2521945474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6080743d0 msgr2=0x7ff6081a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.121+0000 7ff607e00640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6080743d0 0x7ff6081a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.121+0000 7ff607e00640 1 -- 172.21.15.18:0/2521945474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60810e6a0 msgr2=0x7ff60819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.121+0000 7ff607e00640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60810e6a0 0x7ff60819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.121+0000 7ff607e00640 1 -- 172.21.15.18:0/2521945474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60819b030 con 0x7ff608074cf0 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.121+0000 7ff607e00640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff608074cf0 0x7ff6081a0bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4000c00 tx=0x7ff5f40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.122+0000 7ff605600640 1 -- 172.21.15.18:0/2521945474 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4042020 con 0x7ff608074cf0 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.122+0000 7ff605600640 1 -- 172.21.15.18:0/2521945474 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f40054e0 con 0x7ff608074cf0 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.122+0000 7ff60e600640 1 -- 172.21.15.18:0/2521945474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60819b2c0 con 0x7ff608074cf0 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.122+0000 7ff605600640 1 -- 172.21.15.18:0/2521945474 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4003af0 con 0x7ff608074cf0 2024-04-16T17:10:31.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.122+0000 7ff60e600640 1 -- 172.21.15.18:0/2521945474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6081a7630 con 0x7ff608074cf0 2024-04-16T17:10:31.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.126+0000 7ff605600640 1 -- 172.21.15.18:0/2521945474 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5f4045020 con 0x7ff608074cf0 2024-04-16T17:10:31.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.126+0000 7ff61286d740 1 Processor -- start 2024-04-16T17:10:31.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.126+0000 7ff61286d740 1 -- start start 2024-04-16T17:10:31.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff61286d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a34770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff61286d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a3d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff61286d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a3fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff61286d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56514899a120 con 0x565148a3d890 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff61286d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565148940340 con 0x565148a34390 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff61286d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651489960e0 con 0x565148a34cb0 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff60dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a34770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff60dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a34770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37368/0 (socket says 172.21.15.18:37368) 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff60dc00640 1 -- 172.21.15.18:0/1420142194 learned_addr learned my addr 172.21.15.18:0/1420142194 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff60d200640 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a3d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff60dc00640 1 -- 172.21.15.18:0/1420142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 msgr2=0x565148a3d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff60dc00640 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a3d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff60dc00640 1 -- 172.21.15.18:0/1420142194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 msgr2=0x565148a3fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.127+0000 7ff607e00640 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a3fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff605600640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f00a3d20 0x7ff5f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff60dc00640 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a3fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff60dc00640 1 -- 172.21.15.18:0/1420142194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651487767d0 con 0x565148a34390 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff60d200640 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a3d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff605600640 1 -- 172.21.15.18:0/2521945474 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7ff5f40f2800 con 0x7ff608074cf0 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff607e00640 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a3fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff60d200640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f00a3d20 0x7ff5f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff60dc00640 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a34770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff600009f10 tx=0x7ff600010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61380993c71aeb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff5fa000640 1 -- 172.21.15.18:0/1420142194 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600016070 con 0x565148a34390 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff5fa000640 1 -- 172.21.15.18:0/1420142194 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff60001bb40 con 0x565148a34390 2024-04-16T17:10:31.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.128+0000 7ff5fa000640 1 -- 172.21.15.18:0/1420142194 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600012c50 con 0x565148a34390 2024-04-16T17:10:31.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.129+0000 7ff61286d740 1 -- 172.21.15.18:0/1420142194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 msgr2=0x565148a34770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.129+0000 7ff60d200640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f00a3d20 0x7ff5f00a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x565148a33e60 tx=0x7ff5fc005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:31.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.129+0000 7ff61286d740 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a34770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff600009f10 tx=0x7ff600010740 comp rx=0 tx=0).stop 2024-04-16T17:10:31.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 -- 172.21.15.18:0/1420142194 shutdown_connections 2024-04-16T17:10:31.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a3fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a3d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 --2- 172.21.15.18:0/1420142194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a34770 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 -- 172.21.15.18:0/1420142194 >> 172.21.15.18:0/1420142194 conn(0x565148a2eac0 msgr2=0x565148a2eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 -- 172.21.15.18:0/1420142194 shutdown_connections 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 -- 172.21.15.18:0/1420142194 wait complete. 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 -- start start 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.130+0000 7ff61286d740 1 Processor -- start 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff61286d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34390 0x565148a3a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff61286d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34cb0 0x565148a3add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff61286d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a3d890 0x565148a3b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff61286d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x565148a40680 con 0x565148a34390 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff61286d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565148953710 con 0x565148a34cb0 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff61286d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565148a40800 con 0x565148a3d890 2024-04-16T17:10:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff61286d740 10 client.0 _ll_register_callbacks cb 0x56514894eaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff60d200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34390 0x565148a3a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff607e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34cb0 0x565148a3add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff60d200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34390 0x565148a3a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44760/0 (socket says 172.21.15.18:44760) 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.131+0000 7ff60d200640 1 -- 172.21.15.18:0/3229983267 learned_addr learned my addr 172.21.15.18:0/3229983267 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff60dc00640 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a3d890 0x565148a3b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff60d200640 1 -- 172.21.15.18:0/3229983267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a3d890 msgr2=0x565148a3b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff60d200640 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a3d890 0x565148a3b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff60d200640 1 -- 172.21.15.18:0/3229983267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34cb0 msgr2=0x565148a3add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff60d200640 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34cb0 0x565148a3add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff60d200640 1 -- 172.21.15.18:0/3229983267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651486ac3f0 con 0x565148a34390 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff60dc00640 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a3d890 0x565148a3b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff60d200640 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34390 0x565148a3a890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc011c40 tx=0x7ff5fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77598df9538b794a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.132+0000 7ff5eea00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:10:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.133+0000 7ff5eea00640 1 -- 172.21.15.18:0/3229983267 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0108d0 con 0x565148a34390 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.133+0000 7ff5eea00640 1 -- 172.21.15.18:0/3229983267 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7ff5fc010a70 con 0x565148a34390 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.133+0000 7ff61286d740 1 -- 172.21.15.18:0/3229983267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff60001bb40 con 0x565148a34390 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.133+0000 7ff5eea00640 1 -- 172.21.15.18:0/3229983267 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5fc010c70 con 0x565148a34390 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.133+0000 7ff61286d740 10 client.16992 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.133+0000 7ff61286d740 1 -- 172.21.15.18:0/3229983267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56514866c0f0 con 0x565148a34390 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.133+0000 7ff5eea00640 1 -- 172.21.15.18:0/3229983267 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff60001bb40 con 0x565148a34390 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.133+0000 7ff5eea00640 1 -- 172.21.15.18:0/3229983267 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff5fc01e410 con 0x565148a34390 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.134+0000 7ff61286d740 10 client.16992 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.134+0000 7ff61286d740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.134+0000 7ff61286d740 20 client.16992 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.134+0000 7ff61286d740 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565148a4dc20 0x565148a50070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.134+0000 7ff61286d740 4 client.16992 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.134+0000 7ff61286d740 1 -- 172.21.15.18:0/3229983267 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651489a7b40 con 0x565148a4dc20 2024-04-16T17:10:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.134+0000 7ff607e00640 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565148a4dc20 0x565148a50070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.134+0000 7ff607e00640 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565148a4dc20 0x565148a50070 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248fadb7176d84b3 server_cookie=e55a0c16e03abf65 in_seq=0 out_seq=0 2024-04-16T17:10:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.135+0000 7ff5eea00640 10 client.16992 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:31.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.135+0000 7ff5eea00640 1 -- 172.21.15.18:0/3229983267 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651489a7b40 con 0x565148a4dc20 2024-04-16T17:10:31.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.135+0000 7ff5eea00640 10 client.16992 handle_command_reply: tid=0 2024-04-16T17:10:31.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.135+0000 7ff61286d740 1 client.16992 shutdown 2024-04-16T17:10:31.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 -- 172.21.15.18:0/3229983267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34390 msgr2=0x565148a3a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34390 0x565148a3a890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc011c40 tx=0x7ff5fc0046c0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 -- 172.21.15.18:0/3229983267 shutdown_connections 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565148a4dc20 0x565148a50070 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a3d890 0x565148a3b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34cb0 0x565148a3add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 --2- 172.21.15.18:0/3229983267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34390 0x565148a3a890 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 -- 172.21.15.18:0/3229983267 >> 172.21.15.18:0/3229983267 conn(0x565148a2eac0 msgr2=0x565148a30810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff5eea00640 0 client.16992 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 -- 172.21.15.18:0/3229983267 shutdown_connections 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.136+0000 7ff61286d740 1 -- 172.21.15.18:0/3229983267 wait complete. 2024-04-16T17:10:31.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.137+0000 7ff61286d740 20 client.16992 trim_cache size 0 max 16384 2024-04-16T17:10:31.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff61286d740 1 Processor -- start 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff61286d740 1 -- start start 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff61286d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a3cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff61286d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a49570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff61286d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff61286d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081b3e70 con 0x565148a3d890 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff61286d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081b1740 con 0x565148a34390 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff61286d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081af590 con 0x565148a34cb0 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.161+0000 7ff60d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a3cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60dc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a3cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37390/0 (socket says 172.21.15.18:37390) 2024-04-16T17:10:31.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60d200640 1 -- 172.21.15.18:0/169301535 learned_addr learned my addr 172.21.15.18:0/169301535 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:31.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff607e00640 1 --2- 172.21.15.18:0/169301535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a49570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60d200640 1 -- 172.21.15.18:0/169301535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 msgr2=0x565148a49570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:31.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60d200640 1 --2- 172.21.15.18:0/169301535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a49570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60d200640 1 -- 172.21.15.18:0/169301535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 msgr2=0x565148a399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:31.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60d200640 1 --2- 172.21.15.18:0/169301535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60d200640 1 -- 172.21.15.18:0/169301535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565148a3fc80 con 0x565148a34390 2024-04-16T17:10:31.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.162+0000 7ff60d200640 1 --2- 172.21.15.18:0/169301535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a3cfc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc005c20 tx=0x7ff5fc01ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4a76f4a866d507 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.163+0000 7ff5f9600640 1 -- 172.21.15.18:0/169301535 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc011d30 con 0x565148a34390 2024-04-16T17:10:31.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.163+0000 7ff5f9600640 1 -- 172.21.15.18:0/169301535 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc03b070 con 0x565148a34390 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.163+0000 7ff5f9600640 1 -- 172.21.15.18:0/169301535 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0107a0 con 0x565148a34390 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 -- 172.21.15.18:0/169301535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 msgr2=0x565148a3cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 --2- 172.21.15.18:0/169301535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a3cfc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc005c20 tx=0x7ff5fc01ef70 comp rx=0 tx=0).stop 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 -- 172.21.15.18:0/169301535 shutdown_connections 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 --2- 172.21.15.18:0/169301535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a3d890 0x565148a399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 --2- 172.21.15.18:0/169301535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34cb0 0x565148a49570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 --2- 172.21.15.18:0/169301535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a34390 0x565148a3cfc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 -- 172.21.15.18:0/169301535 >> 172.21.15.18:0/169301535 conn(0x565148a2eac0 msgr2=0x565148a3a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 -- 172.21.15.18:0/169301535 shutdown_connections 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 -- 172.21.15.18:0/169301535 wait complete. 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.164+0000 7ff61286d740 1 -- start start 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff61286d740 1 Processor -- start 2024-04-16T17:10:31.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff61286d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34390 0x565148a45c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff61286d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34cb0 0x565148a41000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff61286d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a3d890 0x565148a41540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff61286d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081afc00 con 0x565148a34cb0 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff61286d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081b1070 con 0x565148a3d890 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff61286d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081b02d0 con 0x565148a34390 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff61286d740 10 client.0 _ll_register_callbacks cb 0x7ff5fc0118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.165+0000 7ff60dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a3d890 0x565148a41540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff60dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a3d890 0x565148a41540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37392/0 (socket says 172.21.15.18:37392) 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff60dc00640 1 -- 172.21.15.18:0/2938094275 learned_addr learned my addr 172.21.15.18:0/2938094275 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:31.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff60d200640 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34390 0x565148a45c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff607e00640 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34cb0 0x565148a41000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff60dc00640 1 -- 172.21.15.18:0/2938094275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34390 msgr2=0x565148a45c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff60dc00640 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34390 0x565148a45c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff60dc00640 1 -- 172.21.15.18:0/2938094275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34cb0 msgr2=0x565148a41000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff60dc00640 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34cb0 0x565148a41000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff60dc00640 1 -- 172.21.15.18:0/2938094275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565148a41b80 con 0x565148a3d890 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.166+0000 7ff607e00640 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34cb0 0x565148a41000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.167+0000 7ff60dc00640 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a3d890 0x565148a41540 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff600004c80 tx=0x7ff600004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad966f3428a3055 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.167+0000 7ff5eea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.167+0000 7ff5eea00640 1 -- 172.21.15.18:0/2938094275 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60002b060 con 0x565148a3d890 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.167+0000 7ff61286d740 1 -- 172.21.15.18:0/2938094275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5fc03b070 con 0x565148a3d890 2024-04-16T17:10:31.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.167+0000 7ff5eea00640 1 -- 172.21.15.18:0/2938094275 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7ff600005450 con 0x565148a3d890 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff5eea00640 1 -- 172.21.15.18:0/2938094275 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff600016040 con 0x565148a3d890 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff61286d740 10 client.27236 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff61286d740 1 -- 172.21.15.18:0/2938094275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565148ab4060 con 0x565148a3d890 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff5eea00640 1 -- 172.21.15.18:0/2938094275 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc03b070 con 0x565148a3d890 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff5eea00640 1 -- 172.21.15.18:0/2938094275 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff60000f040 con 0x565148a3d890 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff61286d740 10 client.27236 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff61286d740 10 client.27236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff61286d740 20 client.27236 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff61286d740 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565148ab4400 0x565148ab67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff61286d740 4 client.27236 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.168+0000 7ff61286d740 1 -- 172.21.15.18:0/2938094275 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651489a19b0 con 0x565148ab4400 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.169+0000 7ff60d200640 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565148ab4400 0x565148ab67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.169+0000 7ff60d200640 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565148ab4400 0x565148ab67f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f63978c497eb4b server_cookie=7ec7b941d98a1d5e in_seq=0 out_seq=0 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.169+0000 7ff5eea00640 10 client.27236 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:31.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.169+0000 7ff5eea00640 1 -- 172.21.15.18:0/2938094275 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5651489a19b0 con 0x565148ab4400 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.169+0000 7ff5eea00640 10 client.27236 handle_command_reply: tid=0 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.169+0000 7ff61286d740 1 client.27236 shutdown 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 -- 172.21.15.18:0/2938094275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a3d890 msgr2=0x565148a41540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a3d890 0x565148a41540 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff600004c80 tx=0x7ff600004cb0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 -- 172.21.15.18:0/2938094275 shutdown_connections 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565148ab4400 0x565148ab67f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565148a3d890 0x565148a41540 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565148a34cb0 0x565148a41000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 --2- 172.21.15.18:0/2938094275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565148a34390 0x565148a45c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff5eea00640 0 client.27236 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 -- 172.21.15.18:0/2938094275 >> 172.21.15.18:0/2938094275 conn(0x565148a2eac0 msgr2=0x565148a2fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 -- 172.21.15.18:0/2938094275 shutdown_connections 2024-04-16T17:10:31.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 1 -- 172.21.15.18:0/2938094275 wait complete. 2024-04-16T17:10:31.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.170+0000 7ff61286d740 20 client.27236 trim_cache size 0 max 16384 2024-04-16T17:10:31.175 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:10:31.175 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:10:31.175 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:10:31.175 INFO:teuthology.orchestra.run.smithi018.stdout: "8a449653-c209-4d8b-bfc8-dc4a00ceaeb0": { 2024-04-16T17:10:31.175 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:10:31.175 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "8a449653-c209-4d8b-bfc8-dc4a00ceaeb0", 2024-04-16T17:10:31.175 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:10:31.176 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:10:31.176 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:10:31.176 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:10:31.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 -- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f00a3d20 msgr2=0x7ff5f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f00a3d20 0x7ff5f00a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x565148a33e60 tx=0x7ff5fc005f90 comp rx=0 tx=0).stop 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 -- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff608074cf0 msgr2=0x7ff6081a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff608074cf0 0x7ff6081a0bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4000c00 tx=0x7ff5f40043c0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 -- 172.21.15.18:0/2521945474 shutdown_connections 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f00a3d20 0x7ff5f00a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60810e6a0 0x7ff60819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff608074cf0 0x7ff6081a0bf0 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4000c00 tx=0x7ff5f40043c0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 --2- 172.21.15.18:0/2521945474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6080743d0 0x7ff6081a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 -- 172.21.15.18:0/2521945474 >> 172.21.15.18:0/2521945474 conn(0x7ff6080fd950 msgr2=0x7ff6080fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 -- 172.21.15.18:0/2521945474 shutdown_connections 2024-04-16T17:10:31.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:10:31.171+0000 7ff60e600640 1 -- 172.21.15.18:0/2521945474 wait complete. 2024-04-16T17:10:31.178 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:27 smithi018 ceph-mon[31776]: pgmap v175: 129 pgs: 129 active+clean; 41 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-04-16T17:10:31.178 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:29 smithi018 ceph-mon[31776]: pgmap v176: 129 pgs: 129 active+clean; 41 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-04-16T17:10:31.178 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:29 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/460222733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:31.190 INFO:tasks.cephfs.filesystem:scrub status for tag:8a449653-c209-4d8b-bfc8-dc4a00ceaeb0 - {'path': '/', 'tag': '8a449653-c209-4d8b-bfc8-dc4a00ceaeb0', 'options': 'recursive,force'} 2024-04-16T17:10:31.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 213414 3.96 MB/sec execute 164 sec latency 90.639 ms 2024-04-16T17:10:31.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:31 smithi018 ceph-mon[31776]: pgmap v177: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-16T17:10:31.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:31 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:10:31.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:31 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:31.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:31 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:10:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:31 smithi043 ceph-mon[36901]: pgmap v177: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-16T17:10:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:31 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:10:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:31 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:31 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:10:32.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:31 smithi022 ceph-mon[36023]: pgmap v177: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-16T17:10:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:31 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:10:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:31 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:31 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:10:32.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 214326 3.97 MB/sec execute 165 sec latency 31.243 ms 2024-04-16T17:10:33.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:32 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:33.075 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:32 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:33.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 214920 3.95 MB/sec execute 166 sec latency 63.566 ms 2024-04-16T17:10:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:33 smithi043 ceph-mon[36901]: pgmap v178: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-04-16T17:10:34.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:33 smithi022 ceph-mon[36023]: pgmap v178: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-04-16T17:10:34.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 215302 3.94 MB/sec execute 167 sec latency 82.051 ms 2024-04-16T17:10:35.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:34 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:10:35.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:34 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:10:35.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 216417 3.97 MB/sec execute 168 sec latency 42.608 ms 2024-04-16T17:10:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:35 smithi043 ceph-mon[36901]: pgmap v179: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-04-16T17:10:36.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:35 smithi022 ceph-mon[36023]: pgmap v179: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-04-16T17:10:36.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 217081 3.96 MB/sec execute 169 sec latency 17.024 ms 2024-04-16T17:10:37.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 218119 3.97 MB/sec execute 170 sec latency 16.104 ms 2024-04-16T17:10:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:37 smithi043 ceph-mon[36901]: pgmap v180: 129 pgs: 129 active+clean; 42 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-16T17:10:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:37 smithi022 ceph-mon[36023]: pgmap v180: 129 pgs: 129 active+clean; 42 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-16T17:10:38.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 219024 3.96 MB/sec execute 171 sec latency 75.792 ms 2024-04-16T17:10:39.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 219622 3.97 MB/sec execute 172 sec latency 66.842 ms 2024-04-16T17:10:40.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:39 smithi043 ceph-mon[36901]: pgmap v181: 129 pgs: 129 active+clean; 42 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-04-16T17:10:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:39 smithi022 ceph-mon[36023]: pgmap v181: 129 pgs: 129 active+clean; 42 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-04-16T17:10:40.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 220245 3.98 MB/sec execute 173 sec latency 83.831 ms 2024-04-16T17:10:41.570 INFO:tasks.workunit.client.0.smithi018.stdout: 1 220889 3.96 MB/sec execute 174 sec latency 18.023 ms 2024-04-16T17:10:42.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:41 smithi043 ceph-mon[36901]: pgmap v182: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-04-16T17:10:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:41 smithi022 ceph-mon[36023]: pgmap v182: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-04-16T17:10:42.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 222343 3.97 MB/sec execute 175 sec latency 16.528 ms 2024-04-16T17:10:42.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:32 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:42.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:33 smithi018 ceph-mon[31776]: pgmap v178: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-04-16T17:10:42.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:34 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:10:42.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:35 smithi018 ceph-mon[31776]: pgmap v179: 129 pgs: 129 active+clean; 44 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 930 KiB/s wr, 76 op/s 2024-04-16T17:10:42.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:37 smithi018 ceph-mon[31776]: pgmap v180: 129 pgs: 129 active+clean; 42 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-16T17:10:42.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:39 smithi018 ceph-mon[31776]: pgmap v181: 129 pgs: 129 active+clean; 42 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-04-16T17:10:42.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:41 smithi018 ceph-mon[31776]: pgmap v182: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-04-16T17:10:43.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 222955 3.98 MB/sec execute 176 sec latency 48.049 ms 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 -- 172.21.15.18:0/4290145782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9318104470 msgr2=0x7f9318104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 --2- 172.21.15.18:0/4290145782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9318104470 0x7f9318104850 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9304009900 tx=0x7f930402bc30 comp rx=0 tx=0).stop 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 -- 172.21.15.18:0/4290145782 shutdown_connections 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 --2- 172.21.15.18:0/4290145782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9318109540 0x7f9318111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 --2- 172.21.15.18:0/4290145782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9318104e20 0x7f9318108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 --2- 172.21.15.18:0/4290145782 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9318104470 0x7f9318104850 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 -- 172.21.15.18:0/4290145782 >> 172.21.15.18:0/4290145782 conn(0x7f93180ffcf0 msgr2=0x7f9318102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 -- 172.21.15.18:0/4290145782 shutdown_connections 2024-04-16T17:10:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.896+0000 7f9320200640 1 -- 172.21.15.18:0/4290145782 wait complete. 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.897+0000 7f9320200640 1 Processor -- start 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.897+0000 7f9320200640 1 -- start start 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.897+0000 7f9320200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9318104470 0x7f93181a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.897+0000 7f9320200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9318109540 0x7f93181a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.897+0000 7f9320200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f931819a950 0x7f931819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.897+0000 7f9320200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f93181145f0 con 0x7f931819a950 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f9320200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9318114470 con 0x7f9318109540 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f9320200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9318114770 con 0x7f9318104470 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931e200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f931819a950 0x7f931819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931e200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f931819a950 0x7f931819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51194/0 (socket says 172.21.15.18:51194) 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931e200640 1 -- 172.21.15.18:0/3245214519 learned_addr learned my addr 172.21.15.18:0/3245214519 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931ce00640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9318109540 0x7f93181a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931d800640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9318104470 0x7f93181a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931e200640 1 -- 172.21.15.18:0/3245214519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9318104470 msgr2=0x7f93181a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931e200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9318104470 0x7f93181a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931e200640 1 -- 172.21.15.18:0/3245214519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9318109540 msgr2=0x7f93181a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931e200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9318109540 0x7f93181a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931e200640 1 -- 172.21.15.18:0/3245214519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931819b470 con 0x7f931819a950 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.898+0000 7f931ce00640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9318109540 0x7f93181a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.899+0000 7f931e200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f931819a950 0x7f931819ae00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f931000e970 tx=0x7f931000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.899+0000 7f9316000640 1 -- 172.21.15.18:0/3245214519 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931000ccc0 con 0x7f931819a950 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.899+0000 7f9320200640 1 -- 172.21.15.18:0/3245214519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931819b760 con 0x7f931819a950 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.899+0000 7f9320200640 1 -- 172.21.15.18:0/3245214519 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93181a79b0 con 0x7f931819a950 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.899+0000 7f9316000640 1 -- 172.21.15.18:0/3245214519 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931000ce60 con 0x7f931819a950 2024-04-16T17:10:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.899+0000 7f9316000640 1 -- 172.21.15.18:0/3245214519 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310018630 con 0x7f931819a950 2024-04-16T17:10:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9322fde740 1 Processor -- start 2024-04-16T17:10:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9316000640 1 -- 172.21.15.18:0/3245214519 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93100046c0 con 0x7f931819a950 2024-04-16T17:10:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9322fde740 1 -- start start 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9322fde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9322fde740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9322fde740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9322fde740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562ed4313780 con 0x562ed43ae8a0 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9322fde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562ed42b9f40 con 0x562ed43af610 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.903+0000 7f9322fde740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ed430f740 con 0x562ed43b7e20 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.904+0000 7f931ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.904+0000 7f931d800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.904+0000 7f931ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33842/0 (socket says 172.21.15.18:33842) 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.904+0000 7f931ce00640 1 -- 172.21.15.18:0/2058348005 learned_addr learned my addr 172.21.15.18:0/2058348005 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.904+0000 7f931ce00640 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f931ce00640 1 -- 172.21.15.18:0/2058348005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 msgr2=0x562ed43ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f931ce00640 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43ba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f931ce00640 1 -- 172.21.15.18:0/2058348005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 msgr2=0x562ed43aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f931ce00640 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f931ce00640 1 -- 172.21.15.18:0/2058348005 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed4026210 con 0x562ed43af610 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f931d800640 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f931ce00640 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43ae360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9308011ee0 tx=0x7f9308009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=860a0fb064f1ede8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f9316000640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92fc0a3d90 0x7f92fc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f9316000640 1 -- 172.21.15.18:0/3245214519 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f93100d0fc0 con 0x7f931819a950 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.905+0000 7f931d800640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92fc0a3d90 0x7f92fc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.906+0000 7f930ea00640 1 -- 172.21.15.18:0/2058348005 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930800fc20 con 0x562ed43af610 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.906+0000 7f930ea00640 1 -- 172.21.15.18:0/2058348005 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930800fdc0 con 0x562ed43af610 2024-04-16T17:10:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.906+0000 7f930ea00640 1 -- 172.21.15.18:0/2058348005 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080044d0 con 0x562ed43af610 2024-04-16T17:10:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- 172.21.15.18:0/2058348005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 msgr2=0x562ed43ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43ae360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9308011ee0 tx=0x7f9308009f90 comp rx=0 tx=0).stop 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92fc0a3d90 0x7f92fc0a6250 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9304009230 tx=0x7f930402e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- 172.21.15.18:0/2058348005 shutdown_connections 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 --2- 172.21.15.18:0/2058348005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43ae360 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- 172.21.15.18:0/2058348005 >> 172.21.15.18:0/2058348005 conn(0x562ed43495d0 msgr2=0x562ed43a9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- 172.21.15.18:0/2058348005 shutdown_connections 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- 172.21.15.18:0/2058348005 wait complete. 2024-04-16T17:10:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- start start 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 Processor -- start 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562ed43bab50 con 0x562ed43ae8a0 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562ed42cd390 con 0x562ed43af610 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ed43bacd0 con 0x562ed43b7e20 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 10 client.0 _ll_register_callbacks cb 0x562ed42b6f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51224/0 (socket says 172.21.15.18:51224) 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 -- 172.21.15.18:0/2091427735 learned_addr learned my addr 172.21.15.18:0/2091427735 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33846/0 (socket says 172.21.15.18:33846) 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931e200640 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 -- 172.21.15.18:0/2091427735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 msgr2=0x562ed43b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 -- 172.21.15.18:0/2091427735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 msgr2=0x562ed43b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 -- 172.21.15.18:0/2091427735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed40f2620 con 0x562ed43ae8a0 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f931d800640 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43b7840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9304009900 tx=0x7f930402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3529089b5e38a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9303400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9303400640 1 -- 172.21.15.18:0/2091427735 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930404e030 con 0x562ed43ae8a0 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9303400640 1 -- 172.21.15.18:0/2091427735 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f93040048d0 con 0x562ed43ae8a0 2024-04-16T17:10:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- 172.21.15.18:0/2091427735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930800fdc0 con 0x562ed43ae8a0 2024-04-16T17:10:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9303400640 1 -- 172.21.15.18:0/2091427735 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f930405b050 con 0x562ed43ae8a0 2024-04-16T17:10:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 10 client.17007 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.907+0000 7f9322fde740 1 -- 172.21.15.18:0/2091427735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ed43c6c50 con 0x562ed43ae8a0 2024-04-16T17:10:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9303400640 1 -- 172.21.15.18:0/2091427735 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930800fdc0 con 0x562ed43ae8a0 2024-04-16T17:10:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9303400640 1 -- 172.21.15.18:0/2091427735 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9304004ec0 con 0x562ed43ae8a0 2024-04-16T17:10:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9322fde740 10 client.17007 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9322fde740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9322fde740 20 client.17007 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9322fde740 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ed43c81d0 0x562ed43ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9322fde740 4 client.17007 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9322fde740 1 -- 172.21.15.18:0/2091427735 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ed43211a0 con 0x562ed43c81d0 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f931ce00640 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ed43c81d0 0x562ed43ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f931ce00640 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ed43c81d0 0x562ed43ca5c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee8827669f889df0 server_cookie=678948bfadeab84d in_seq=0 out_seq=0 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9303400640 10 client.17007 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9303400640 1 -- 172.21.15.18:0/2091427735 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ed43211a0 con 0x562ed43c81d0 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.912+0000 7f9303400640 10 client.17007 handle_command_reply: tid=0 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 client.17007 shutdown 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 -- 172.21.15.18:0/2091427735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 msgr2=0x562ed43b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43b7840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9304009900 tx=0x7f930402bc30 comp rx=0 tx=0).stop 2024-04-16T17:10:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 -- 172.21.15.18:0/2091427735 shutdown_connections 2024-04-16T17:10:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ed43c81d0 0x562ed43ca5c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 --2- 172.21.15.18:0/2091427735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43b7840 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9303400640 0 client.17007 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 -- 172.21.15.18:0/2091427735 >> 172.21.15.18:0/2091427735 conn(0x562ed43495d0 msgr2=0x562ed43ab2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 -- 172.21.15.18:0/2091427735 shutdown_connections 2024-04-16T17:10:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 1 -- 172.21.15.18:0/2091427735 wait complete. 2024-04-16T17:10:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.915+0000 7f9322fde740 20 client.17007 trim_cache size 0 max 16384 2024-04-16T17:10:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f9322fde740 1 Processor -- start 2024-04-16T17:10:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f9322fde740 1 -- start start 2024-04-16T17:10:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f9322fde740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f9322fde740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43af610 0x562ed43cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f9322fde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43b7e20 0x562ed43d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f9322fde740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f92f8012820 con 0x562ed43ae8a0 2024-04-16T17:10:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f9322fde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f92f80100f0 con 0x562ed43b7e20 2024-04-16T17:10:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f9322fde740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92f800df40 con 0x562ed43af610 2024-04-16T17:10:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43af610 0x562ed43cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43af610 0x562ed43cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40160/0 (socket says 172.21.15.18:40160) 2024-04-16T17:10:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931ce00640 1 -- 172.21.15.18:0/2235730044 learned_addr learned my addr 172.21.15.18:0/2235730044 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931d800640 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931e200640 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43b7e20 0x562ed43d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931e200640 1 -- 172.21.15.18:0/2235730044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43af610 msgr2=0x562ed43cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931e200640 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43af610 0x562ed43cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931e200640 1 -- 172.21.15.18:0/2235730044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 msgr2=0x562ed43c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931e200640 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.937+0000 7f931e200640 1 -- 172.21.15.18:0/2235730044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed43a4e60 con 0x562ed43b7e20 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f931e200640 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43b7e20 0x562ed43d03f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f93100ce300 tx=0x7f931009c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f5eaf8d4534ebb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f930e000640 1 -- 172.21.15.18:0/2235730044 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93100a0f00 con 0x562ed43b7e20 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f930e000640 1 -- 172.21.15.18:0/2235730044 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93100a57c0 con 0x562ed43b7e20 2024-04-16T17:10:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f930e000640 1 -- 172.21.15.18:0/2235730044 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931009bfa0 con 0x562ed43b7e20 2024-04-16T17:10:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 -- 172.21.15.18:0/2235730044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43b7e20 msgr2=0x562ed43d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43b7e20 0x562ed43d03f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f93100ce300 tx=0x7f931009c5d0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 -- 172.21.15.18:0/2235730044 shutdown_connections 2024-04-16T17:10:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43b7e20 0x562ed43d03f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43af610 0x562ed43cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 --2- 172.21.15.18:0/2235730044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 -- 172.21.15.18:0/2235730044 >> 172.21.15.18:0/2235730044 conn(0x562ed43495d0 msgr2=0x562ed43a90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 -- 172.21.15.18:0/2235730044 shutdown_connections 2024-04-16T17:10:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.943+0000 7f9322fde740 1 -- 172.21.15.18:0/2235730044 wait complete. 2024-04-16T17:10:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 -- start start 2024-04-16T17:10:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 Processor -- start 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43bc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f92f800e5b0 con 0x562ed43ae8a0 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f92f800fa20 con 0x562ed43af610 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92f800ec80 con 0x562ed43b7e20 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 10 client.0 _ll_register_callbacks cb 0x7f9304009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931d800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931d800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43bbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51246/0 (socket says 172.21.15.18:51246) 2024-04-16T17:10:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43bc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931d800640 1 -- 172.21.15.18:0/3611385871 learned_addr learned my addr 172.21.15.18:0/3611385871 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931e200640 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931ce00640 1 -- 172.21.15.18:0/3611385871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 msgr2=0x562ed43bc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931ce00640 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43bc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931ce00640 1 -- 172.21.15.18:0/3611385871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 msgr2=0x562ed43bbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931ce00640 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43bbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931ce00640 1 -- 172.21.15.18:0/3611385871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed442d820 con 0x562ed43af610 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931ce00640 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43bc010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9308004140 tx=0x7f93080040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ad5932f8c139a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9303400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9303400640 1 -- 172.21.15.18:0/3611385871 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308031020 con 0x562ed43af610 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9303400640 1 -- 172.21.15.18:0/3611385871 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f930800fde0 con 0x562ed43af610 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 -- 172.21.15.18:0/3611385871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93100a57c0 con 0x562ed43af610 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9303400640 1 -- 172.21.15.18:0/3611385871 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f930803c050 con 0x562ed43af610 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 10 client.27254 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 -- 172.21.15.18:0/3611385871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ed442eb10 con 0x562ed43af610 2024-04-16T17:10:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9303400640 1 -- 172.21.15.18:0/3611385871 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93100a57c0 con 0x562ed43af610 2024-04-16T17:10:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9303400640 1 -- 172.21.15.18:0/3611385871 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f930802d760 con 0x562ed43af610 2024-04-16T17:10:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 10 client.27254 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 10 client.27254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 20 client.27254 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ed442f060 0x562ed4431480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 4 client.27254 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9322fde740 1 -- 172.21.15.18:0/3611385871 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ed431b010 con 0x562ed442f060 2024-04-16T17:10:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931d800640 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ed442f060 0x562ed4431480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f931d800640 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ed442f060 0x562ed4431480 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e1488caf408080 server_cookie=512a8d173c812926 in_seq=0 out_seq=0 2024-04-16T17:10:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:43.944+0000 7f9303400640 10 client.27254 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:44.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:43 smithi043 ceph-mon[36901]: pgmap v183: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 876 KiB/s wr, 129 op/s 2024-04-16T17:10:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.104+0000 7f9303400640 1 -- 172.21.15.18:0/3611385871 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ed431b010 con 0x562ed442f060 2024-04-16T17:10:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.104+0000 7f9303400640 10 client.27254 handle_command_reply: tid=0 2024-04-16T17:10:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.105+0000 7f9322fde740 1 client.27254 shutdown 2024-04-16T17:10:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 -- 172.21.15.18:0/3611385871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 msgr2=0x562ed43bc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43bc010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9308004140 tx=0x7f93080040d0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 -- 172.21.15.18:0/3611385871 shutdown_connections 2024-04-16T17:10:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ed442f060 0x562ed4431480 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ed43b7e20 0x562ed43bc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ed43af610 0x562ed43bc010 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 --2- 172.21.15.18:0/3611385871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ed43ae8a0 0x562ed43bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 -- 172.21.15.18:0/3611385871 >> 172.21.15.18:0/3611385871 conn(0x562ed43495d0 msgr2=0x562ed43b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9303400640 0 client.27254 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 -- 172.21.15.18:0/3611385871 shutdown_connections 2024-04-16T17:10:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.106+0000 7f9322fde740 1 -- 172.21.15.18:0/3611385871 wait complete. 2024-04-16T17:10:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.107+0000 7f9322fde740 20 client.27254 trim_cache size 0 max 16384 2024-04-16T17:10:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:10:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:10:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:10:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:10:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.109+0000 7f9320200640 1 -- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92fc0a3d90 msgr2=0x7f92fc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.109+0000 7f9320200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92fc0a3d90 0x7f92fc0a6250 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9304009230 tx=0x7f930402e040 comp rx=0 tx=0).stop 2024-04-16T17:10:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.109+0000 7f9320200640 1 -- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f931819a950 msgr2=0x7f931819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.109+0000 7f9320200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f931819a950 0x7f931819ae00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f931000e970 tx=0x7f931000ed70 comp rx=0 tx=0).stop 2024-04-16T17:10:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.110+0000 7f9320200640 1 -- 172.21.15.18:0/3245214519 shutdown_connections 2024-04-16T17:10:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.110+0000 7f9320200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92fc0a3d90 0x7f92fc0a6250 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.110+0000 7f9320200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f931819a950 0x7f931819ae00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.110+0000 7f9320200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9318109540 0x7f93181a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.110+0000 7f9320200640 1 --2- 172.21.15.18:0/3245214519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9318104470 0x7f93181a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.110+0000 7f9320200640 1 -- 172.21.15.18:0/3245214519 >> 172.21.15.18:0/3245214519 conn(0x7f93180ffcf0 msgr2=0x7f9318102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.110+0000 7f9320200640 1 -- 172.21.15.18:0/3245214519 shutdown_connections 2024-04-16T17:10:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:44.110+0000 7f9320200640 1 -- 172.21.15.18:0/3245214519 wait complete. 2024-04-16T17:10:44.112 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:43 smithi018 ceph-mon[31776]: pgmap v183: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 876 KiB/s wr, 129 op/s 2024-04-16T17:10:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:43 smithi022 ceph-mon[36023]: pgmap v183: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 876 KiB/s wr, 129 op/s 2024-04-16T17:10:44.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 223276 3.97 MB/sec execute 177 sec latency 104.197 ms 2024-04-16T17:10:45.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 223806 3.97 MB/sec execute 178 sec latency 314.417 ms 2024-04-16T17:10:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:45 smithi043 ceph-mon[36901]: pgmap v184: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 876 KiB/s wr, 130 op/s 2024-04-16T17:10:46.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:45 smithi022 ceph-mon[36023]: pgmap v184: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 876 KiB/s wr, 130 op/s 2024-04-16T17:10:46.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 224426 3.96 MB/sec execute 179 sec latency 48.291 ms 2024-04-16T17:10:46.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:45 smithi018 ceph-mon[31776]: pgmap v184: 129 pgs: 129 active+clean; 45 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 876 KiB/s wr, 130 op/s 2024-04-16T17:10:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:46 smithi022 ceph-mon[36023]: pgmap v185: 129 pgs: 129 active+clean; 39 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.5 MiB/s wr, 230 op/s 2024-04-16T17:10:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:46 smithi043 ceph-mon[36901]: pgmap v185: 129 pgs: 129 active+clean; 39 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.5 MiB/s wr, 230 op/s 2024-04-16T17:10:47.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 225816 3.97 MB/sec execute 180 sec latency 13.570 ms 2024-04-16T17:10:48.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 226412 3.97 MB/sec execute 181 sec latency 45.137 ms 2024-04-16T17:10:49.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 226818 3.97 MB/sec execute 182 sec latency 55.450 ms 2024-04-16T17:10:49.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:49 smithi022 ceph-mon[36023]: pgmap v186: 129 pgs: 129 active+clean; 39 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-04-16T17:10:49.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:49 smithi043 ceph-mon[36901]: pgmap v186: 129 pgs: 129 active+clean; 39 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-04-16T17:10:50.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 227399 3.97 MB/sec execute 183 sec latency 52.411 ms 2024-04-16T17:10:51.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 228024 3.95 MB/sec execute 184 sec latency 53.838 ms 2024-04-16T17:10:51.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:51 smithi022 ceph-mon[36023]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-16T17:10:51.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:51 smithi043 ceph-mon[36901]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-16T17:10:52.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 229424 3.96 MB/sec execute 185 sec latency 14.476 ms 2024-04-16T17:10:53.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 230321 3.98 MB/sec execute 186 sec latency 59.861 ms 2024-04-16T17:10:53.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:53 smithi022 ceph-mon[36023]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-04-16T17:10:53.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:53 smithi043 ceph-mon[36901]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-04-16T17:10:54.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 230684 3.98 MB/sec execute 187 sec latency 60.232 ms 2024-04-16T17:10:55.225 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:46 smithi018 ceph-mon[31776]: pgmap v185: 129 pgs: 129 active+clean; 39 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.5 MiB/s wr, 230 op/s 2024-04-16T17:10:55.225 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:49 smithi018 ceph-mon[31776]: pgmap v186: 129 pgs: 129 active+clean; 39 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-04-16T17:10:55.225 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:51 smithi018 ceph-mon[31776]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-16T17:10:55.225 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:53 smithi018 ceph-mon[31776]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-04-16T17:10:55.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 230945 3.97 MB/sec execute 188 sec latency 92.666 ms 2024-04-16T17:10:55.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:55 smithi022 ceph-mon[36023]: pgmap v189: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-04-16T17:10:55.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:55 smithi043 ceph-mon[36901]: pgmap v189: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-04-16T17:10:56.571 INFO:tasks.workunit.client.0.smithi018.stdout: 1 231583 3.95 MB/sec execute 189 sec latency 28.140 ms 2024-04-16T17:10:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 -- 172.21.15.18:0/1879227637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 msgr2=0x7f82141128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 --2- 172.21.15.18:0/1879227637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 0x7f82141128f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009900 tx=0x7f81fc02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 -- 172.21.15.18:0/1879227637 shutdown_connections 2024-04-16T17:10:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 --2- 172.21.15.18:0/1879227637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214112e30 0x7f8214115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 --2- 172.21.15.18:0/1879227637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 0x7f82141128f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 --2- 172.21.15.18:0/1879227637 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82140743a0 0x7f8214074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 -- 172.21.15.18:0/1879227637 >> 172.21.15.18:0/1879227637 conn(0x7f82140fd910 msgr2=0x7f82140ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 -- 172.21.15.18:0/1879227637 shutdown_connections 2024-04-16T17:10:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.256+0000 7f821a600640 1 -- 172.21.15.18:0/1879227637 wait complete. 2024-04-16T17:10:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.258+0000 7f821a600640 1 Processor -- start 2024-04-16T17:10:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.258+0000 7f821a600640 1 -- start start 2024-04-16T17:10:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.258+0000 7f821a600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82140743a0 0x7f821419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.258+0000 7f821a600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 0x7f821419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f821a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214112e30 0x7f82141a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f821a600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8214118860 con 0x7f82140743a0 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f821a600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82141186e0 con 0x7f8214074cc0 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f821a600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82141189e0 con 0x7f8214112e30 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8213e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 0x7f821419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8219200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82140743a0 0x7f821419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8213e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 0x7f821419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36454/0 (socket says 172.21.15.18:36454) 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8213e00640 1 -- 172.21.15.18:0/1764238140 learned_addr learned my addr 172.21.15.18:0/1764238140 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8219c00640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214112e30 0x7f82141a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8213e00640 1 -- 172.21.15.18:0/1764238140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214112e30 msgr2=0x7f82141a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8213e00640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214112e30 0x7f82141a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8213e00640 1 -- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82140743a0 msgr2=0x7f821419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8213e00640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82140743a0 0x7f821419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.259+0000 7f8213e00640 1 -- 172.21.15.18:0/1764238140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82141a3d10 con 0x7f8214074cc0 2024-04-16T17:10:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.260+0000 7f8213e00640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 0x7f821419f260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f81fc000c00 tx=0x7f81fc0043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.260+0000 7f8211600640 1 -- 172.21.15.18:0/1764238140 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc042020 con 0x7f8214074cc0 2024-04-16T17:10:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.260+0000 7f8211600640 1 -- 172.21.15.18:0/1764238140 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81fc005000 con 0x7f8214074cc0 2024-04-16T17:10:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.260+0000 7f821a600640 1 -- 172.21.15.18:0/1764238140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82141a3fa0 con 0x7f8214074cc0 2024-04-16T17:10:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.260+0000 7f8211600640 1 -- 172.21.15.18:0/1764238140 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc003b20 con 0x7f8214074cc0 2024-04-16T17:10:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.260+0000 7f821a600640 1 -- 172.21.15.18:0/1764238140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82141ab890 con 0x7f8214074cc0 2024-04-16T17:10:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.264+0000 7f821e970740 1 Processor -- start 2024-04-16T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f821e970740 1 -- start start 2024-04-16T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f8211600640 1 -- 172.21.15.18:0/1764238140 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81fc045020 con 0x7f8214074cc0 2024-04-16T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f821e970740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f04540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f821e970740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f04f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f821e970740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f821e970740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5561e1e69780 con 0x5561e1f0e000 2024-04-16T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f821e970740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5561e1e0ff40 con 0x5561e1f04a80 2024-04-16T17:10:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f821e970740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5561e1e65740 con 0x5561e1f057f0 2024-04-16T17:10:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f8219c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f04540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f8219c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f04540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53278/0 (socket says 172.21.15.18:53278) 2024-04-16T17:10:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f8219200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f04f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.265+0000 7f8219c00640 1 -- 172.21.15.18:0/3112313013 learned_addr learned my addr 172.21.15.18:0/3112313013 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f8213e00640 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f8219c00640 1 -- 172.21.15.18:0/3112313013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 msgr2=0x5561e1f04f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f8219c00640 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f04f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f8219c00640 1 -- 172.21.15.18:0/3112313013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 msgr2=0x5561e1f103f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f8219c00640 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f103f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f8219c00640 1 -- 172.21.15.18:0/3112313013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e1b7c210 con 0x5561e1f057f0 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f8219200640 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f04f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f8219c00640 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f04540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b600 tx=0x7f820c00bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0daab48e5012ee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.266+0000 7f820a000640 1 -- 172.21.15.18:0/3112313013 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c003b20 con 0x5561e1f057f0 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f820a000640 1 -- 172.21.15.18:0/3112313013 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820c015070 con 0x5561e1f057f0 2024-04-16T17:10:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f820a000640 1 -- 172.21.15.18:0/3112313013 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c0116a0 con 0x5561e1f057f0 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f821e970740 1 -- 172.21.15.18:0/3112313013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 msgr2=0x5561e1f04540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f821e970740 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f04540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b600 tx=0x7f820c00bac0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f821e970740 1 -- 172.21.15.18:0/3112313013 shutdown_connections 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f821e970740 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f821e970740 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f04f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f821e970740 1 --2- 172.21.15.18:0/3112313013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f04540 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.267+0000 7f821e970740 1 -- 172.21.15.18:0/3112313013 >> 172.21.15.18:0/3112313013 conn(0x5561e1e9f5d0 msgr2=0x5561e1eff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.268+0000 7f821e970740 1 -- 172.21.15.18:0/3112313013 shutdown_connections 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.268+0000 7f8211600640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f81f80a3d30 0x7f81f80a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.268+0000 7f821e970740 1 -- 172.21.15.18:0/3112313013 wait complete. 2024-04-16T17:10:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.268+0000 7f8211600640 1 -- 172.21.15.18:0/1764238140 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f81fc0f2750 con 0x7f8214074cc0 2024-04-16T17:10:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.268+0000 7f8219200640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f81f80a3d30 0x7f81f80a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.268+0000 7f821e970740 1 -- start start 2024-04-16T17:10:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.268+0000 7f821e970740 1 Processor -- start 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.269+0000 7f821e970740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f04a80 0x5561e1f0daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.269+0000 7f821e970740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.269+0000 7f821e970740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f0e000 0x5561e1f0b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.269+0000 7f821e970740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5561e1f10d30 con 0x5561e1f04a80 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.269+0000 7f821e970740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5561e1e23390 con 0x5561e1f0e000 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.269+0000 7f821e970740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5561e1f10eb0 con 0x5561e1f057f0 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.269+0000 7f8219200640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f81f80a3d30 0x7f81f80a61f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f82040099a0 tx=0x7f8204009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f04a80 0x5561e1f0daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f0e000 0x5561e1f0b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f04a80 0x5561e1f0daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55012/0 (socket says 172.21.15.18:55012) 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219200640 1 -- 172.21.15.18:0/1964554681 learned_addr learned my addr 172.21.15.18:0/1964554681 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f821e970740 10 client.0 _ll_register_callbacks cb 0x5561e1e0cf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8213e00640 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219c00640 1 -- 172.21.15.18:0/1964554681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 msgr2=0x5561e1f0ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219c00640 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219c00640 1 -- 172.21.15.18:0/1964554681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f04a80 msgr2=0x5561e1f0daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219c00640 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f04a80 0x5561e1f0daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219c00640 1 -- 172.21.15.18:0/1964554681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e1c48620 con 0x5561e1f0e000 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8213e00640 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8219c00640 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f0e000 0x5561e1f0b520 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f820c004650 tx=0x7f820c01a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d78a888c31f327c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.270+0000 7f8202a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.271+0000 7f8202a00640 1 -- 172.21.15.18:0/1964554681 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c005440 con 0x5561e1f0e000 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.271+0000 7f8202a00640 1 -- 172.21.15.18:0/1964554681 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f820c0038f0 con 0x5561e1f0e000 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.271+0000 7f821e970740 1 -- 172.21.15.18:0/1964554681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f820c015070 con 0x5561e1f0e000 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.271+0000 7f8202a00640 1 -- 172.21.15.18:0/1964554681 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f820c003af0 con 0x5561e1f0e000 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.271+0000 7f821e970740 10 client.27266 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.271+0000 7f821e970740 1 -- 172.21.15.18:0/1964554681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561e1f1ce00 con 0x5561e1f0e000 2024-04-16T17:10:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.271+0000 7f8202a00640 1 -- 172.21.15.18:0/1964554681 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820c015070 con 0x5561e1f0e000 2024-04-16T17:10:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.272+0000 7f8202a00640 1 -- 172.21.15.18:0/1964554681 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f820c023070 con 0x5561e1f0e000 2024-04-16T17:10:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.272+0000 7f821e970740 10 client.27266 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.272+0000 7f821e970740 10 client.27266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.272+0000 7f821e970740 20 client.27266 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.272+0000 7f821e970740 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5561e1f1e200 0x5561e1f205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.272+0000 7f821e970740 4 client.27266 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.272+0000 7f821e970740 1 -- 172.21.15.18:0/1964554681 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561e1e771a0 con 0x5561e1f1e200 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.272+0000 7f8219200640 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5561e1f1e200 0x5561e1f205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.273+0000 7f8219200640 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5561e1f1e200 0x5561e1f205f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f7ada42b73b94a server_cookie=8e6d9edd43fd6085 in_seq=0 out_seq=0 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.273+0000 7f8202a00640 10 client.27266 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f8202a00640 1 -- 172.21.15.18:0/1964554681 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561e1e771a0 con 0x5561e1f1e200 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f8202a00640 10 client.27266 handle_command_reply: tid=0 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 client.27266 shutdown 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 -- 172.21.15.18:0/1964554681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f0e000 msgr2=0x5561e1f0b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f0e000 0x5561e1f0b520 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f820c004650 tx=0x7f820c01a720 comp rx=0 tx=0).stop 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 -- 172.21.15.18:0/1964554681 shutdown_connections 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5561e1f1e200 0x5561e1f205f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f0e000 0x5561e1f0b520 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 --2- 172.21.15.18:0/1964554681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f04a80 0x5561e1f0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 -- 172.21.15.18:0/1964554681 >> 172.21.15.18:0/1964554681 conn(0x5561e1e9f5d0 msgr2=0x5561e1f014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f8202a00640 0 client.27266 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 -- 172.21.15.18:0/1964554681 shutdown_connections 2024-04-16T17:10:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 1 -- 172.21.15.18:0/1964554681 wait complete. 2024-04-16T17:10:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.274+0000 7f821e970740 20 client.27266 trim_cache size 0 max 16384 2024-04-16T17:10:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.293+0000 7f821e970740 1 Processor -- start 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.293+0000 7f821e970740 1 -- start start 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.293+0000 7f821e970740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f19c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.293+0000 7f821e970740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.293+0000 7f821e970740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f0d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.293+0000 7f821e970740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b8220 con 0x5561e1f0e000 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.293+0000 7f821e970740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b5af0 con 0x5561e1f04a80 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.293+0000 7f821e970740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b3940 con 0x5561e1f057f0 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f19c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f19c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36502/0 (socket says 172.21.15.18:36502) 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219200640 1 -- 172.21.15.18:0/371801993 learned_addr learned my addr 172.21.15.18:0/371801993 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8213e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219c00640 1 --2- 172.21.15.18:0/371801993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f0d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219200640 1 -- 172.21.15.18:0/371801993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 msgr2=0x5561e1f0aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219200640 1 --2- 172.21.15.18:0/371801993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219200640 1 -- 172.21.15.18:0/371801993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 msgr2=0x5561e1f0d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219200640 1 --2- 172.21.15.18:0/371801993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f0d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.294+0000 7f8219200640 1 -- 172.21.15.18:0/371801993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e1f10760 con 0x5561e1f04a80 2024-04-16T17:10:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.295+0000 7f8219200640 1 --2- 172.21.15.18:0/371801993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f19c30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82040097f0 tx=0x7f820401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467ba175235cd9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.295+0000 7f8209600640 1 -- 172.21.15.18:0/371801993 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204030070 con 0x5561e1f04a80 2024-04-16T17:10:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.295+0000 7f8209600640 1 -- 172.21.15.18:0/371801993 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820401e430 con 0x5561e1f04a80 2024-04-16T17:10:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.295+0000 7f8209600640 1 -- 172.21.15.18:0/371801993 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820401c680 con 0x5561e1f04a80 2024-04-16T17:10:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.295+0000 7f821e970740 1 -- 172.21.15.18:0/371801993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 msgr2=0x5561e1f19c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.295+0000 7f821e970740 1 --2- 172.21.15.18:0/371801993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f19c30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82040097f0 tx=0x7f820401a000 comp rx=0 tx=0).stop 2024-04-16T17:10:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 -- 172.21.15.18:0/371801993 shutdown_connections 2024-04-16T17:10:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 --2- 172.21.15.18:0/371801993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f0d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 --2- 172.21.15.18:0/371801993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f057f0 0x5561e1f0aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 --2- 172.21.15.18:0/371801993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f04a80 0x5561e1f19c30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 -- 172.21.15.18:0/371801993 >> 172.21.15.18:0/371801993 conn(0x5561e1e9f5d0 msgr2=0x5561e1f0a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 -- 172.21.15.18:0/371801993 shutdown_connections 2024-04-16T17:10:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 -- 172.21.15.18:0/371801993 wait complete. 2024-04-16T17:10:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 -- start start 2024-04-16T17:10:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.296+0000 7f821e970740 1 Processor -- start 2024-04-16T17:10:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f821e970740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f04a80 0x5561e1f16330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f821e970740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f057f0 0x5561e1f076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f821e970740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f07c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f821e970740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b3fb0 con 0x5561e1f0e000 2024-04-16T17:10:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f821e970740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b5420 con 0x5561e1f057f0 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f821e970740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b4680 con 0x5561e1f04a80 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f8219200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f04a80 0x5561e1f16330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f821e970740 10 client.0 _ll_register_callbacks cb 0x7f820c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f8219200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f04a80 0x5561e1f16330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53302/0 (socket says 172.21.15.18:53302) 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f8219200640 1 -- 172.21.15.18:0/464301764 learned_addr learned my addr 172.21.15.18:0/464301764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.297+0000 7f8213e00640 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f057f0 0x5561e1f076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.298+0000 7f8219200640 1 -- 172.21.15.18:0/464301764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f057f0 msgr2=0x5561e1f076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.298+0000 7f8219200640 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f057f0 0x5561e1f076c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.298+0000 7f8219200640 1 -- 172.21.15.18:0/464301764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 msgr2=0x5561e1f07c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.298+0000 7f8219200640 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f07c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.298+0000 7f8219200640 1 -- 172.21.15.18:0/464301764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e1f081c0 con 0x5561e1f04a80 2024-04-16T17:10:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.298+0000 7f8219200640 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f04a80 0x5561e1f16330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f82040049d0 tx=0x7f8204011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744e4c8aafcf00ba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.298+0000 7f8202a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.299+0000 7f8202a00640 1 -- 172.21.15.18:0/464301764 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204030040 con 0x5561e1f04a80 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.299+0000 7f8202a00640 1 -- 172.21.15.18:0/464301764 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f8204004c40 con 0x5561e1f04a80 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.299+0000 7f821e970740 1 -- 172.21.15.18:0/464301764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f820401e430 con 0x5561e1f04a80 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.299+0000 7f8202a00640 1 -- 172.21.15.18:0/464301764 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8204004e40 con 0x5561e1f04a80 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.299+0000 7f821e970740 10 client.27223 fetch_fsmap learned FSMap version 26 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.299+0000 7f821e970740 1 -- 172.21.15.18:0/464301764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561e1f84310 con 0x5561e1f04a80 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.299+0000 7f8202a00640 1 -- 172.21.15.18:0/464301764 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820401e430 con 0x5561e1f04a80 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.300+0000 7f8202a00640 1 -- 172.21.15.18:0/464301764 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8204005040 con 0x5561e1f04a80 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.300+0000 7f821e970740 10 client.27223 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.300+0000 7f821e970740 10 client.27223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.300+0000 7f821e970740 20 client.27223 populate_metadata read hostname 'smithi018' 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.300+0000 7f821e970740 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5561e1f849a0 0x5561e1f86dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.300+0000 7f821e970740 4 client.27223 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.300+0000 7f821e970740 1 -- 172.21.15.18:0/464301764 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561e1e71010 con 0x5561e1f849a0 2024-04-16T17:10:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.300+0000 7f8213e00640 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5561e1f849a0 0x5561e1f86dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.301+0000 7f8202a00640 10 client.27223 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.301+0000 7f8213e00640 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5561e1f849a0 0x5561e1f86dc0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45bd9dc1308aaaec server_cookie=4fc49a8129e604b5 in_seq=0 out_seq=0 2024-04-16T17:10:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.415+0000 7f8202a00640 1 -- 172.21.15.18:0/464301764 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561e1e71010 con 0x5561e1f849a0 2024-04-16T17:10:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.415+0000 7f8202a00640 10 client.27223 handle_command_reply: tid=0 2024-04-16T17:10:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.416+0000 7f821e970740 1 client.27223 shutdown 2024-04-16T17:10:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 -- 172.21.15.18:0/464301764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f04a80 msgr2=0x5561e1f16330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f04a80 0x5561e1f16330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f82040049d0 tx=0x7f8204011dc0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 -- 172.21.15.18:0/464301764 shutdown_connections 2024-04-16T17:10:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5561e1f849a0 0x5561e1f86dc0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5561e1f0e000 0x5561e1f07c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f8202a00640 0 client.27223 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:10:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561e1f057f0 0x5561e1f076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 --2- 172.21.15.18:0/464301764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5561e1f04a80 0x5561e1f16330 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 -- 172.21.15.18:0/464301764 >> 172.21.15.18:0/464301764 conn(0x5561e1e9f5d0 msgr2=0x5561e1f0ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 -- 172.21.15.18:0/464301764 shutdown_connections 2024-04-16T17:10:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 1 -- 172.21.15.18:0/464301764 wait complete. 2024-04-16T17:10:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.419+0000 7f821e970740 20 client.27223 trim_cache size 0 max 16384 2024-04-16T17:10:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:10:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:10:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:10:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:10:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.424+0000 7f821a600640 1 -- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f81f80a3d30 msgr2=0x7f81f80a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.424+0000 7f821a600640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f81f80a3d30 0x7f81f80a61f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f82040099a0 tx=0x7f8204009500 comp rx=0 tx=0).stop 2024-04-16T17:10:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.424+0000 7f821a600640 1 -- 172.21.15.18:0/1764238140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 msgr2=0x7f821419f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.424+0000 7f821a600640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 0x7f821419f260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f81fc000c00 tx=0x7f81fc0043c0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.426+0000 7f821a600640 1 -- 172.21.15.18:0/1764238140 shutdown_connections 2024-04-16T17:10:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.426+0000 7f821a600640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f81f80a3d30 0x7f81f80a61f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.426+0000 7f821a600640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214112e30 0x7f82141a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.426+0000 7f821a600640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8214074cc0 0x7f821419f260 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.426+0000 7f821a600640 1 --2- 172.21.15.18:0/1764238140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82140743a0 0x7f821419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.426+0000 7f821a600640 1 -- 172.21.15.18:0/1764238140 >> 172.21.15.18:0/1764238140 conn(0x7f82140fd910 msgr2=0x7f82140ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.426+0000 7f821a600640 1 -- 172.21.15.18:0/1764238140 shutdown_connections 2024-04-16T17:10:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:10:57.426+0000 7f821a600640 1 -- 172.21.15.18:0/1764238140 wait complete. 2024-04-16T17:10:57.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 232809 3.96 MB/sec execute 190 sec latency 82.728 ms 2024-04-16T17:10:57.603 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:55 smithi018 ceph-mon[31776]: pgmap v189: 129 pgs: 129 active+clean; 42 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-04-16T17:10:57.603 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:57 smithi018 ceph-mon[31776]: pgmap v190: 129 pgs: 129 active+clean; 44 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-04-16T17:10:57.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:57 smithi022 ceph-mon[36023]: pgmap v190: 129 pgs: 129 active+clean; 44 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-04-16T17:10:57.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:57 smithi043 ceph-mon[36901]: pgmap v190: 129 pgs: 129 active+clean; 44 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-04-16T17:10:58.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 233810 3.97 MB/sec execute 191 sec latency 43.093 ms 2024-04-16T17:10:59.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 234147 3.97 MB/sec execute 192 sec latency 60.145 ms 2024-04-16T17:10:59.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:10:59 smithi022 ceph-mon[36023]: pgmap v191: 129 pgs: 129 active+clean; 44 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-04-16T17:10:59.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:10:59 smithi043 ceph-mon[36901]: pgmap v191: 129 pgs: 129 active+clean; 44 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-04-16T17:11:00.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 234328 3.96 MB/sec execute 193 sec latency 115.907 ms 2024-04-16T17:11:00.660 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:10:59 smithi018 ceph-mon[31776]: pgmap v191: 129 pgs: 129 active+clean; 44 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-04-16T17:11:00.660 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:00 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:11:00.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:00 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:11:00.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:00 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:11:01.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:11:01.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 -- 172.21.15.18:0/1450047536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584104450 msgr2=0x7f4584104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 --2- 172.21.15.18:0/1450047536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584104450 0x7f4584104830 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4574009be0 tx=0x7f457402bef0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 -- 172.21.15.18:0/1450047536 shutdown_connections 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 --2- 172.21.15.18:0/1450047536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584109520 0x7f45841110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 --2- 172.21.15.18:0/1450047536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4584104e00 0x7f4584108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 --2- 172.21.15.18:0/1450047536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584104450 0x7f4584104830 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 -- 172.21.15.18:0/1450047536 >> 172.21.15.18:0/1450047536 conn(0x7f45840ffc90 msgr2=0x7f45841020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 -- 172.21.15.18:0/1450047536 shutdown_connections 2024-04-16T17:11:01.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.432+0000 7f4589600640 1 -- 172.21.15.18:0/1450047536 wait complete. 2024-04-16T17:11:01.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.433+0000 7f4589600640 1 Processor -- start 2024-04-16T17:11:01.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.433+0000 7f4589600640 1 -- start start 2024-04-16T17:11:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4589600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584104450 0x7f4584076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4589600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584109520 0x7f4584076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4589600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4584076f90 0x7f45841ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4589600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4584114630 con 0x7f4584076f90 2024-04-16T17:11:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4589600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45841144b0 con 0x7f4584109520 2024-04-16T17:11:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4589600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45841147b0 con 0x7f4584104450 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4582a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584104450 0x7f4584076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4582000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584109520 0x7f4584076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4582a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584104450 0x7f4584076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53326/0 (socket says 172.21.15.18:53326) 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4582a00640 1 -- 172.21.15.18:0/126528741 learned_addr learned my addr 172.21.15.18:0/126528741 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.434+0000 7f4583400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4584076f90 0x7f45841ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f4582a00640 1 -- 172.21.15.18:0/126528741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584109520 msgr2=0x7f4584076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f4582a00640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584109520 0x7f4584076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f4582a00640 1 -- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4584076f90 msgr2=0x7f45841ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f4582a00640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4584076f90 0x7f45841ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f4582a00640 1 -- 172.21.15.18:0/126528741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45841ab6a0 con 0x7f4584104450 2024-04-16T17:11:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f4582a00640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584104450 0x7f4584076510 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4574004af0 tx=0x7f45740336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f457b400640 1 -- 172.21.15.18:0/126528741 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574043020 con 0x7f4584104450 2024-04-16T17:11:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f4589600640 1 -- 172.21.15.18:0/126528741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45841ab930 con 0x7f4584104450 2024-04-16T17:11:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f457b400640 1 -- 172.21.15.18:0/126528741 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45740338a0 con 0x7f4584104450 2024-04-16T17:11:01.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.435+0000 7f457b400640 1 -- 172.21.15.18:0/126528741 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457403e6b0 con 0x7f4584104450 2024-04-16T17:11:01.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.436+0000 7f4589600640 1 -- 172.21.15.18:0/126528741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45841abea0 con 0x7f4584104450 2024-04-16T17:11:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.438+0000 7f457b400640 1 -- 172.21.15.18:0/126528741 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4574046020 con 0x7f4584104450 2024-04-16T17:11:01.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.439+0000 7f458c36d740 1 Processor -- start 2024-04-16T17:11:01.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.439+0000 7f458c36d740 1 -- start start 2024-04-16T17:11:01.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f458c36d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f458c36d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7eec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f458c36d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eeeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f458c36d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7e49120 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f458c36d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7def340 con 0x55c9c7ee31b0 2024-04-16T17:11:01.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f458c36d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7e450e0 con 0x55c9c7eec6b0 2024-04-16T17:11:01.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f4583400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f4583400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36546/0 (socket says 172.21.15.18:36546) 2024-04-16T17:11:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f4583400640 1 -- 172.21.15.18:0/3837934959 learned_addr learned my addr 172.21.15.18:0/3837934959 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.440+0000 7f4582000640 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eeeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f457b400640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45680a3da0 0x7f45680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4583400640 1 -- 172.21.15.18:0/3837934959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 msgr2=0x55c9c7eeeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4583400640 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eeeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4582a00640 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7eec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4583400640 1 -- 172.21.15.18:0/3837934959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee3ad0 msgr2=0x55c9c7eec170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4583400640 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7eec170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4583400640 1 -- 172.21.15.18:0/3837934959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c7c257d0 con 0x55c9c7ee31b0 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4582a00640 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7eec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f457b400640 1 -- 172.21.15.18:0/126528741 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f45740f3090 con 0x7f4584104450 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4583400640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45680a3da0 0x7f45680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.441+0000 7f4583400640 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee3590 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f457c009b90 tx=0x7f457c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc3130ec32444a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.442+0000 7f4573400640 1 -- 172.21.15.18:0/3837934959 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c012590 con 0x55c9c7ee31b0 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.442+0000 7f4573400640 1 -- 172.21.15.18:0/3837934959 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c01c400 con 0x55c9c7ee31b0 2024-04-16T17:11:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.442+0000 7f4573400640 1 -- 172.21.15.18:0/3837934959 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c00f680 con 0x55c9c7ee31b0 2024-04-16T17:11:01.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 -- 172.21.15.18:0/3837934959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 msgr2=0x55c9c7ee3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f4583400640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45680a3da0 0x7f45680a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f457c020000 tx=0x7f457c021000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee3590 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f457c009b90 tx=0x7f457c0117b0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 -- 172.21.15.18:0/3837934959 shutdown_connections 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eeeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7eec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 --2- 172.21.15.18:0/3837934959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee3590 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 -- 172.21.15.18:0/3837934959 >> 172.21.15.18:0/3837934959 conn(0x55c9c7edd8e0 msgr2=0x55c9c7eddcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 -- 172.21.15.18:0/3837934959 shutdown_connections 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 -- 172.21.15.18:0/3837934959 wait complete. 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.443+0000 7f458c36d740 1 -- start start 2024-04-16T17:11:01.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f458c36d740 1 Processor -- start 2024-04-16T17:11:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f458c36d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f458c36d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ee9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f458c36d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f458c36d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7eef4a0 con 0x55c9c7eec6b0 2024-04-16T17:11:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f458c36d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7e02710 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f458c36d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7eef620 con 0x55c9c7ee31b0 2024-04-16T17:11:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f458c36d740 10 client.0 _ll_register_callbacks cb 0x55c9c7dfdaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f4582000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ee9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f4582000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ee9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36556/0 (socket says 172.21.15.18:36556) 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f4582000640 1 -- 172.21.15.18:0/2033920622 learned_addr learned my addr 172.21.15.18:0/2033920622 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f4582a00640 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.444+0000 7f4583400640 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4582000640 1 -- 172.21.15.18:0/2033920622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee31b0 msgr2=0x55c9c7ee96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4582000640 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4582000640 1 -- 172.21.15.18:0/2033920622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 msgr2=0x55c9c7eea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4582000640 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4582000640 1 -- 172.21.15.18:0/2033920622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c7b5b3f0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4582a00640 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4582000640 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ee9c30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f456c005fd0 tx=0x7f456c011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d599b253651c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4567e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4567e00640 1 -- 172.21.15.18:0/2033920622 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c01e060 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f458c36d740 1 -- 172.21.15.18:0/2033920622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f457c01c400 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.445+0000 7f4567e00640 1 -- 172.21.15.18:0/2033920622 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f456c00fbf0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f4567e00640 1 -- 172.21.15.18:0/2033920622 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f456c025050 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f458c36d740 10 client.27284 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f458c36d740 1 -- 172.21.15.18:0/2033920622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9c7b1b0f0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f4567e00640 1 -- 172.21.15.18:0/2033920622 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c01c400 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f4567e00640 1 -- 172.21.15.18:0/2033920622 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f456c019070 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f458c36d740 10 client.27284 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f458c36d740 10 client.27284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f458c36d740 20 client.27284 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f458c36d740 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9c7ef9db0 0x55c9c7efc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f458c36d740 4 client.27284 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.446+0000 7f458c36d740 1 -- 172.21.15.18:0/2033920622 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9c7e56b40 con 0x55c9c7ef9db0 2024-04-16T17:11:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.447+0000 7f4582a00640 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9c7ef9db0 0x55c9c7efc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.447+0000 7f4582a00640 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9c7ef9db0 0x55c9c7efc270 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd7895fdf9e007 server_cookie=561e7b9a7e12f4b in_seq=0 out_seq=0 2024-04-16T17:11:01.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.447+0000 7f4567e00640 10 client.27284 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:01.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f4567e00640 1 -- 172.21.15.18:0/2033920622 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9c7e56b40 con 0x55c9c7ef9db0 2024-04-16T17:11:01.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f4567e00640 10 client.27284 handle_command_reply: tid=0 2024-04-16T17:11:01.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 client.27284 shutdown 2024-04-16T17:11:01.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 -- 172.21.15.18:0/2033920622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 msgr2=0x55c9c7ee9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ee9c30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f456c005fd0 tx=0x7f456c011b30 comp rx=0 tx=0).stop 2024-04-16T17:11:01.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 -- 172.21.15.18:0/2033920622 shutdown_connections 2024-04-16T17:11:01.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9c7ef9db0 0x55c9c7efc270 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ee9c30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 --2- 172.21.15.18:0/2033920622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ee96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 -- 172.21.15.18:0/2033920622 >> 172.21.15.18:0/2033920622 conn(0x55c9c7edd8e0 msgr2=0x55c9c7edf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:01.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f4567e00640 0 client.27284 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:01.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 -- 172.21.15.18:0/2033920622 shutdown_connections 2024-04-16T17:11:01.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 1 -- 172.21.15.18:0/2033920622 wait complete. 2024-04-16T17:11:01.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.448+0000 7f458c36d740 20 client.27284 trim_cache size 0 max 16384 2024-04-16T17:11:01.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.467+0000 7f458c36d740 1 Processor -- start 2024-04-16T17:11:01.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f458c36d740 1 -- start start 2024-04-16T17:11:01.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f458c36d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f458c36d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7edf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f458c36d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 0x55c9c7edc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f458c36d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f45841afc40 con 0x55c9c7ee31b0 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f458c36d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45841ad510 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f458c36d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4584073520 con 0x55c9c7eec6b0 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f4582000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7edf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f4582a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f4582000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7edf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36562/0 (socket says 172.21.15.18:36562) 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f4582000640 1 -- 172.21.15.18:0/846584748 learned_addr learned my addr 172.21.15.18:0/846584748 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.468+0000 7f4583400640 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 0x55c9c7edc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.469+0000 7f4582000640 1 -- 172.21.15.18:0/846584748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 msgr2=0x55c9c7edc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.469+0000 7f4582000640 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 0x55c9c7edc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.469+0000 7f4582000640 1 -- 172.21.15.18:0/846584748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee31b0 msgr2=0x55c9c7ef83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.469+0000 7f4582000640 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.469+0000 7f4582000640 1 -- 172.21.15.18:0/846584748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c7ee3900 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.469+0000 7f4582a00640 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.469+0000 7f4582000640 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7edf5a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f456c009de0 tx=0x7f456c0059f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e8459158bd86351 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.469+0000 7f4572a00640 1 -- 172.21.15.18:0/846584748 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c02d030 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.470+0000 7f4572a00640 1 -- 172.21.15.18:0/846584748 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456c01a3f0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.470+0000 7f4572a00640 1 -- 172.21.15.18:0/846584748 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c004120 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.470+0000 7f458c36d740 1 -- 172.21.15.18:0/846584748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 msgr2=0x55c9c7edf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7edf5a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f456c009de0 tx=0x7f456c0059f0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 -- 172.21.15.18:0/846584748 shutdown_connections 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7eec6b0 0x55c9c7edc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7edf5a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 --2- 172.21.15.18:0/846584748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 -- 172.21.15.18:0/846584748 >> 172.21.15.18:0/846584748 conn(0x55c9c7edd8e0 msgr2=0x55c9c7edc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 -- 172.21.15.18:0/846584748 shutdown_connections 2024-04-16T17:11:01.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 -- 172.21.15.18:0/846584748 wait complete. 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.471+0000 7f458c36d740 1 -- start start 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.472+0000 7f458c36d740 1 Processor -- start 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.472+0000 7f458c36d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.472+0000 7f458c36d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ef4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.472+0000 7f458c36d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eefa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.472+0000 7f458c36d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4584073b90 con 0x55c9c7eec6b0 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.472+0000 7f458c36d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45841ace40 con 0x55c9c7ee31b0 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.472+0000 7f458c36d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45841ac0c0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.472+0000 7f458c36d740 10 client.0 _ll_register_callbacks cb 0x7f456c009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:01.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ef4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36568/0 (socket says 172.21.15.18:36568) 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582a00640 1 -- 172.21.15.18:0/2171374502 learned_addr learned my addr 172.21.15.18:0/2171374502 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582000640 1 -- 172.21.15.18:0/2171374502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 msgr2=0x55c9c7ef48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582000640 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582000640 1 -- 172.21.15.18:0/2171374502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 msgr2=0x55c9c7eefa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582000640 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eefa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582000640 1 -- 172.21.15.18:0/2171374502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c7ef02c0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582a00640 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4582000640 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ef4e20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55c9c7ee2420 tx=0x7f456c0057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7858676409f91b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.473+0000 7f4567e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.474+0000 7f4567e00640 1 -- 172.21.15.18:0/2171374502 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c035070 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.474+0000 7f4567e00640 1 -- 172.21.15.18:0/2171374502 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f456c00fbf0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.474+0000 7f458c36d740 1 -- 172.21.15.18:0/2171374502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456c01a3f0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.474+0000 7f4567e00640 1 -- 172.21.15.18:0/2171374502 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f456c02d040 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f458c36d740 10 client.27241 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:01.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f458c36d740 1 -- 172.21.15.18:0/2171374502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9c7f635d0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f4567e00640 1 -- 172.21.15.18:0/2171374502 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456c01a3f0 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f4567e00640 1 -- 172.21.15.18:0/2171374502 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f456c038020 con 0x55c9c7ee3ad0 2024-04-16T17:11:01.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f458c36d740 10 client.27241 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:01.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f458c36d740 10 client.27241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:01.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f458c36d740 20 client.27241 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:01.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f458c36d740 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9c7f63c80 0x55c9c7f66070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f458c36d740 4 client.27241 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:11:01.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.475+0000 7f458c36d740 1 -- 172.21.15.18:0/2171374502 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9c7e509b0 con 0x55c9c7f63c80 2024-04-16T17:11:01.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.476+0000 7f4582a00640 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9c7f63c80 0x55c9c7f66070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.476+0000 7f4582a00640 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9c7f63c80 0x55c9c7f66070 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5201df76a9e59678 server_cookie=41b58bb164f73e29 in_seq=0 out_seq=0 2024-04-16T17:11:01.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.476+0000 7f4567e00640 10 client.27241 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:01.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.476+0000 7f4567e00640 1 -- 172.21.15.18:0/2171374502 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c9c7e509b0 con 0x55c9c7f63c80 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.476+0000 7f4567e00640 10 client.27241 handle_command_reply: tid=0 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.476+0000 7f458c36d740 1 client.27241 shutdown 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 -- 172.21.15.18:0/2171374502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee3ad0 msgr2=0x55c9c7ef4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ef4e20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55c9c7ee2420 tx=0x7f456c0057e0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 -- 172.21.15.18:0/2171374502 shutdown_connections 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9c7f63c80 0x55c9c7f66070 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9c7eec6b0 0x55c9c7eefa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9c7ee3ad0 0x55c9c7ef4e20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 --2- 172.21.15.18:0/2171374502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9c7ee31b0 0x55c9c7ef48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f4567e00640 0 client.27241 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 -- 172.21.15.18:0/2171374502 >> 172.21.15.18:0/2171374502 conn(0x55c9c7edd8e0 msgr2=0x55c9c7ee9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 -- 172.21.15.18:0/2171374502 shutdown_connections 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 1 -- 172.21.15.18:0/2171374502 wait complete. 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.477+0000 7f458c36d740 20 client.27241 trim_cache size 0 max 16384 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:11:01.480 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 -- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45680a3da0 msgr2=0x7f45680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45680a3da0 0x7f45680a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f457c020000 tx=0x7f457c021000 comp rx=0 tx=0).stop 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 -- 172.21.15.18:0/126528741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584104450 msgr2=0x7f4584076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584104450 0x7f4584076510 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4574004af0 tx=0x7f45740336b0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 -- 172.21.15.18:0/126528741 shutdown_connections 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45680a3da0 0x7f45680a6260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4584076f90 0x7f45841ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584109520 0x7f4584076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 --2- 172.21.15.18:0/126528741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584104450 0x7f4584076510 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 -- 172.21.15.18:0/126528741 >> 172.21.15.18:0/126528741 conn(0x7f45840ffc90 msgr2=0x7f4584101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 -- 172.21.15.18:0/126528741 shutdown_connections 2024-04-16T17:11:01.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.478+0000 7f4589600640 1 -- 172.21.15.18:0/126528741 wait complete. 2024-04-16T17:11:01.500 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:11:01.500 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:11:01.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 234625 3.94 MB/sec execute 194 sec latency 50.278 ms 2024-04-16T17:11:01.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:01 smithi022 ceph-mon[36023]: pgmap v192: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-16T17:11:01.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.747+0000 7fa954600640 1 -- 172.21.15.18:0/48607986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa94c104e10 msgr2=0x7fa94c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.747+0000 7fa954600640 1 --2- 172.21.15.18:0/48607986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa94c104e10 0x7fa94c108e50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa938009900 tx=0x7fa93802bc40 comp rx=0 tx=0).stop 2024-04-16T17:11:01.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.748+0000 7fa954600640 1 -- 172.21.15.18:0/48607986 shutdown_connections 2024-04-16T17:11:01.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.748+0000 7fa954600640 1 --2- 172.21.15.18:0/48607986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c109530 0x7fa94c111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.748+0000 7fa954600640 1 --2- 172.21.15.18:0/48607986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa94c104e10 0x7fa94c108e50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.748+0000 7fa954600640 1 --2- 172.21.15.18:0/48607986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa94c104460 0x7fa94c104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.748+0000 7fa954600640 1 -- 172.21.15.18:0/48607986 >> 172.21.15.18:0/48607986 conn(0x7fa94c0ffcc0 msgr2=0x7fa94c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:01.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.749+0000 7fa954600640 1 -- 172.21.15.18:0/48607986 shutdown_connections 2024-04-16T17:11:01.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.749+0000 7fa954600640 1 -- 172.21.15.18:0/48607986 wait complete. 2024-04-16T17:11:01.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.750+0000 7fa954600640 1 Processor -- start 2024-04-16T17:11:01.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.750+0000 7fa954600640 1 -- start start 2024-04-16T17:11:01.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:01 smithi043 ceph-mon[36901]: pgmap v192: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-16T17:11:01.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa954600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c104460 0x7fa94c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa954600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa94c109530 0x7fa94c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa954600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa94c19a930 0x7fa94c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa954600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1145f0 con 0x7fa94c109530 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa954600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c114470 con 0x7fa94c19a930 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa954600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c114770 con 0x7fa94c104460 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa951c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c104460 0x7fa94c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa951c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c104460 0x7fa94c1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53386/0 (socket says 172.21.15.18:53386) 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa951c00640 1 -- 172.21.15.18:0/234028323 learned_addr learned my addr 172.21.15.18:0/234028323 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.751+0000 7fa951200640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa94c109530 0x7fa94c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.752+0000 7fa951c00640 1 -- 172.21.15.18:0/234028323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa94c19a930 msgr2=0x7fa94c19ade0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.752+0000 7fa951c00640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa94c19a930 0x7fa94c19ade0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.752+0000 7fa951c00640 1 -- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa94c109530 msgr2=0x7fa94c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.752+0000 7fa951c00640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa94c109530 0x7fa94c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:01.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.752+0000 7fa951c00640 1 -- 172.21.15.18:0/234028323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c19b450 con 0x7fa94c104460 2024-04-16T17:11:01.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.752+0000 7fa951c00640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c104460 0x7fa94c1a0780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa94000d950 tx=0x7fa94000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:01.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.752+0000 7fa94a000640 1 -- 172.21.15.18:0/234028323 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940017070 con 0x7fa94c104460 2024-04-16T17:11:01.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.753+0000 7fa94a000640 1 -- 172.21.15.18:0/234028323 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa940009c30 con 0x7fa94c104460 2024-04-16T17:11:01.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.753+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c19b740 con 0x7fa94c104460 2024-04-16T17:11:01.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.753+0000 7fa94a000640 1 -- 172.21.15.18:0/234028323 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94000f6b0 con 0x7fa94c104460 2024-04-16T17:11:01.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.753+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c1a7990 con 0x7fa94c104460 2024-04-16T17:11:01.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.754+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c069300 con 0x7fa94c104460 2024-04-16T17:11:01.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.757+0000 7fa94a000640 1 -- 172.21.15.18:0/234028323 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa940010460 con 0x7fa94c104460 2024-04-16T17:11:01.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.759+0000 7fa94a000640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9340a3d20 0x7fa9340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:01.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.759+0000 7fa94a000640 1 -- 172.21.15.18:0/234028323 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7fa9400d0860 con 0x7fa94c104460 2024-04-16T17:11:01.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.759+0000 7fa951200640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9340a3d20 0x7fa9340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:01.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.760+0000 7fa951200640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9340a3d20 0x7fa9340a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa93802c250 tx=0x7fa93802cf90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:01.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:01.764+0000 7fa94a000640 1 -- 172.21.15.18:0/234028323 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa94009caf0 con 0x7fa94c104460 2024-04-16T17:11:02.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.112+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa94c104e10 con 0x7fa94c104460 2024-04-16T17:11:02.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.113+0000 7fa94a000640 1 -- 172.21.15.18:0/234028323 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11854 (secure 0 0 0) 0x7fa940009de0 con 0x7fa94c104460 2024-04-16T17:11:02.114 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:11:02.114 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:10:18.339186+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27119":{"gid":27119,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/1162334131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1162334131},{"type":"v1","addr":"172.21.15.22:6837","nonce":1162334131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:11:02.114 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 26 2024-04-16T17:11:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9340a3d20 msgr2=0x7fa9340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9340a3d20 0x7fa9340a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa93802c250 tx=0x7fa93802cf90 comp rx=0 tx=0).stop 2024-04-16T17:11:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c104460 msgr2=0x7fa94c1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c104460 0x7fa94c1a0780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa94000d950 tx=0x7fa94000de10 comp rx=0 tx=0).stop 2024-04-16T17:11:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 shutdown_connections 2024-04-16T17:11:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9340a3d20 0x7fa9340a61e0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa93802c250 tx=0x7fa93802cf90 comp rx=0 tx=0).stop 2024-04-16T17:11:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa94c19a930 0x7fa94c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa94c109530 0x7fa94c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 --2- 172.21.15.18:0/234028323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c104460 0x7fa94c1a0780 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.115+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 >> 172.21.15.18:0/234028323 conn(0x7fa94c0ffcc0 msgr2=0x7fa94c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.116+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 shutdown_connections 2024-04-16T17:11:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.116+0000 7fa954600640 1 -- 172.21.15.18:0/234028323 wait complete. 2024-04-16T17:11:02.135 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:11:02.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 -- 172.21.15.18:0/2417373965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4100f60 msgr2=0x7f76f4101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 --2- 172.21.15.18:0/2417373965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4100f60 0x7f76f4101340 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009900 tx=0x7f76e402bc30 comp rx=0 tx=0).stop 2024-04-16T17:11:02.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 -- 172.21.15.18:0/2417373965 shutdown_connections 2024-04-16T17:11:02.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 --2- 172.21.15.18:0/2417373965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f4106030 0x7f76f410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 --2- 172.21.15.18:0/2417373965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4101910 0x7f76f4105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 --2- 172.21.15.18:0/2417373965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4100f60 0x7f76f4101340 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 -- 172.21.15.18:0/2417373965 >> 172.21.15.18:0/2417373965 conn(0x7f76f4077ab0 msgr2=0x7f76f40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 -- 172.21.15.18:0/2417373965 shutdown_connections 2024-04-16T17:11:02.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.348+0000 7f76f8a00640 1 -- 172.21.15.18:0/2417373965 wait complete. 2024-04-16T17:11:02.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.349+0000 7f76f8a00640 1 Processor -- start 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f8a00640 1 -- start start 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4100f60 0x7f76f4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4106030 0x7f76f4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f4196dc0 0x7f76f41aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f8a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4110020 con 0x7f76f4100f60 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f8a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76f410fea0 con 0x7f76f4106030 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41101a0 con 0x7f76f4196dc0 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4100f60 0x7f76f4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4100f60 0x7f76f4196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55118/0 (socket says 172.21.15.18:55118) 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f4196dc0 0x7f76f41aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.350+0000 7f76f3400640 1 -- 172.21.15.18:0/568465200 learned_addr learned my addr 172.21.15.18:0/568465200 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76f2a00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4106030 0x7f76f4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76f3400640 1 -- 172.21.15.18:0/568465200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f4196dc0 msgr2=0x7f76f41aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76f3400640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f4196dc0 0x7f76f41aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76f3400640 1 -- 172.21.15.18:0/568465200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4106030 msgr2=0x7f76f4196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76f3400640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4106030 0x7f76f4196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76f3400640 1 -- 172.21.15.18:0/568465200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f41ab420 con 0x7f76f4100f60 2024-04-16T17:11:02.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76f2a00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4106030 0x7f76f4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:02.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76f3400640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4100f60 0x7f76f4196340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f76e402c240 tx=0x7f76e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.351+0000 7f76ebe00640 1 -- 172.21.15.18:0/568465200 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4043020 con 0x7f76f4100f60 2024-04-16T17:11:02.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.352+0000 7f76ebe00640 1 -- 172.21.15.18:0/568465200 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e40054e0 con 0x7f76f4100f60 2024-04-16T17:11:02.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.352+0000 7f76f8a00640 1 -- 172.21.15.18:0/568465200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f41ab6b0 con 0x7f76f4100f60 2024-04-16T17:11:02.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.352+0000 7f76ebe00640 1 -- 172.21.15.18:0/568465200 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003b20 con 0x7f76f4100f60 2024-04-16T17:11:02.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.352+0000 7f76f8a00640 1 -- 172.21.15.18:0/568465200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41abbd0 con 0x7f76f4100f60 2024-04-16T17:11:02.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.355+0000 7f76fcbfd740 1 Processor -- start 2024-04-16T17:11:02.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.355+0000 7f76ebe00640 1 -- 172.21.15.18:0/568465200 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f76e4046020 con 0x7f76f4100f60 2024-04-16T17:11:02.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76fcbfd740 1 -- start start 2024-04-16T17:11:02.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9726f10 0x55ddf97272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf9732800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ddf968c020 con 0x55ddf9727830 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9632460 con 0x55ddf9726f10 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9687fe0 con 0x55ddf9730410 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76f3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9726f10 0x55ddf97272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf9732800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf9732800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53416/0 (socket says 172.21.15.18:53416) 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76f3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9726f10 0x55ddf97272f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36614/0 (socket says 172.21.15.18:36614) 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.356+0000 7f76f2a00640 1 -- 172.21.15.18:0/2614017476 learned_addr learned my addr 172.21.15.18:0/2614017476 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76f3e00640 1 -- 172.21.15.18:0/2614017476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 msgr2=0x55ddf9732800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76f3e00640 1 --2- 172.21.15.18:0/2614017476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf9732800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76f3e00640 1 -- 172.21.15.18:0/2614017476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 msgr2=0x55ddf972fed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:02.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76f3e00640 1 --2- 172.21.15.18:0/2614017476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76f3e00640 1 -- 172.21.15.18:0/2614017476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf963a040 con 0x55ddf9726f10 2024-04-16T17:11:02.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76ebe00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76d80a3d20 0x7f76d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76f3e00640 1 --2- 172.21.15.18:0/2614017476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9726f10 0x55ddf97272f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f76ec003970 tx=0x7f76ec00b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6968c330142bb03e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76ebe00640 1 -- 172.21.15.18:0/568465200 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f76e40f2b50 con 0x7f76f4100f60 2024-04-16T17:11:02.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.357+0000 7f76dfe00640 1 -- 172.21.15.18:0/2614017476 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004720 con 0x55ddf9726f10 2024-04-16T17:11:02.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.358+0000 7f76dfe00640 1 -- 172.21.15.18:0/2614017476 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec005020 con 0x55ddf9726f10 2024-04-16T17:11:02.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.358+0000 7f76dfe00640 1 -- 172.21.15.18:0/2614017476 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0166b0 con 0x55ddf9726f10 2024-04-16T17:11:02.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.358+0000 7f76f3400640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76d80a3d20 0x7f76d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76f3400640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76d80a3d20 0x7f76d80a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55ddf97269e0 tx=0x7f76e402cdf0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.358+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2614017476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9726f10 msgr2=0x55ddf97272f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2614017476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9726f10 0x55ddf97272f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f76ec003970 tx=0x7f76ec00b710 comp rx=0 tx=0).stop 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2614017476 shutdown_connections 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2614017476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf9732800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2614017476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2614017476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9726f10 0x55ddf97272f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2614017476 >> 172.21.15.18:0/2614017476 conn(0x55ddf9721640 msgr2=0x55ddf9721a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2614017476 shutdown_connections 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2614017476 wait complete. 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.359+0000 7f76fcbfd740 1 -- start start 2024-04-16T17:11:02.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76fcbfd740 1 Processor -- start 2024-04-16T17:11:02.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf972d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf972dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf972e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9733200 con 0x55ddf9726f10 2024-04-16T17:11:02.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9645830 con 0x55ddf9727830 2024-04-16T17:11:02.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9733380 con 0x55ddf9730410 2024-04-16T17:11:02.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76fcbfd740 10 client.0 _ll_register_callbacks cb 0x55ddf964d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:02.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf972dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf972dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36628/0 (socket says 172.21.15.18:36628) 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.360+0000 7f76f2a00640 1 -- 172.21.15.18:0/1808927924 learned_addr learned my addr 172.21.15.18:0/1808927924 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f2a00640 1 -- 172.21.15.18:0/1808927924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 msgr2=0x55ddf972e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f3e00640 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf972e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f2a00640 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf972e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f3e00640 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf972d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f2a00640 1 -- 172.21.15.18:0/1808927924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 msgr2=0x55ddf972d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f2a00640 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf972d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f2a00640 1 -- 172.21.15.18:0/1808927924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf9649920 con 0x55ddf9727830 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f3e00640 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf972d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76f2a00640 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf972dac0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f76e0012700 tx=0x7f76e0012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2cf8676e1ce8801 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76dcc00640 1 -- 172.21.15.18:0/1808927924 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0004260 con 0x55ddf9727830 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.361+0000 7f76dcc00640 1 -- 172.21.15.18:0/1808927924 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f76e0004400 con 0x55ddf9727830 2024-04-16T17:11:02.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.362+0000 7f76fcbfd740 1 -- 172.21.15.18:0/1808927924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ec005020 con 0x55ddf9727830 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.362+0000 7f76dcc00640 1 -- 172.21.15.18:0/1808927924 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f76e00119a0 con 0x55ddf9727830 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.362+0000 7f76fcbfd740 10 client.27305 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.362+0000 7f76fcbfd740 1 -- 172.21.15.18:0/1808927924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddf96177f0 con 0x55ddf9727830 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.362+0000 7f76dcc00640 1 -- 172.21.15.18:0/1808927924 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec005020 con 0x55ddf9727830 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.362+0000 7f76dcc00640 1 -- 172.21.15.18:0/1808927924 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f76e000f440 con 0x55ddf9727830 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.363+0000 7f76fcbfd740 10 client.27305 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.363+0000 7f76fcbfd740 10 client.27305 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.363+0000 7f76fcbfd740 20 client.27305 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.363+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ddf9740900 0x55ddf9742da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.363+0000 7f76fcbfd740 4 client.27305 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.363+0000 7f76fcbfd740 1 -- 172.21.15.18:0/1808927924 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf9699a40 con 0x55ddf9740900 2024-04-16T17:11:02.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.363+0000 7f76f3e00640 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ddf9740900 0x55ddf9742da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.364+0000 7f76dcc00640 10 client.27305 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:11:02.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.364+0000 7f76f3e00640 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ddf9740900 0x55ddf9742da0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85b2b84f19fc0885 server_cookie=ab9ea8384bf4056a in_seq=0 out_seq=0 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.364+0000 7f76dcc00640 1 -- 172.21.15.18:0/1808927924 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddf9699a40 con 0x55ddf9740900 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.364+0000 7f76dcc00640 10 client.27305 handle_command_reply: tid=0 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 client.27305 shutdown 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 -- 172.21.15.18:0/1808927924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 msgr2=0x55ddf972dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf972dac0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f76e0012700 tx=0x7f76e0012bc0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 -- 172.21.15.18:0/1808927924 shutdown_connections 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ddf9740900 0x55ddf9742da0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf972e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76dcc00640 0 client.27305 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf972dac0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/1808927924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf972d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 -- 172.21.15.18:0/1808927924 >> 172.21.15.18:0/1808927924 conn(0x55ddf9721640 msgr2=0x55ddf9723390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 -- 172.21.15.18:0/1808927924 shutdown_connections 2024-04-16T17:11:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.365+0000 7f76fcbfd740 1 -- 172.21.15.18:0/1808927924 wait complete. 2024-04-16T17:11:02.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.366+0000 7f76fcbfd740 20 client.27305 trim_cache size 0 max 16384 2024-04-16T17:11:02.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76fcbfd740 1 Processor -- start 2024-04-16T17:11:02.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76fcbfd740 1 -- start start 2024-04-16T17:11:02.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9726f10 0x55ddf973c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9730410 0x55ddf9720700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b81d0 con 0x55ddf9727830 2024-04-16T17:11:02.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b5aa0 con 0x55ddf9730410 2024-04-16T17:11:02.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b38f0 con 0x55ddf9726f10 2024-04-16T17:11:02.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.381+0000 7f76f3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9730410 0x55ddf9720700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55134/0 (socket says 172.21.15.18:55134) 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f2a00640 1 -- 172.21.15.18:0/661383414 learned_addr learned my addr 172.21.15.18:0/661383414 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f3e00640 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9726f10 0x55ddf973c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f2a00640 1 -- 172.21.15.18:0/661383414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9726f10 msgr2=0x55ddf973c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f2a00640 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9726f10 0x55ddf973c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f2a00640 1 -- 172.21.15.18:0/661383414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9730410 msgr2=0x55ddf9720700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f2a00640 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9730410 0x55ddf9720700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f2a00640 1 -- 172.21.15.18:0/661383414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf972c530 con 0x55ddf9727830 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f3e00640 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9730410 0x55ddf9720700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.382+0000 7f76f2a00640 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fc40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009f50 tx=0x7f76e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaec6d6da494410 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.383+0000 7f76df400640 1 -- 172.21.15.18:0/661383414 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e000f440 con 0x55ddf9727830 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.383+0000 7f76df400640 1 -- 172.21.15.18:0/661383414 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e000f5e0 con 0x55ddf9727830 2024-04-16T17:11:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.383+0000 7f76df400640 1 -- 172.21.15.18:0/661383414 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e00035f0 con 0x55ddf9727830 2024-04-16T17:11:02.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.383+0000 7f76fcbfd740 1 -- 172.21.15.18:0/661383414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 msgr2=0x55ddf972fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.383+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fc40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009f50 tx=0x7f76e0011810 comp rx=0 tx=0).stop 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 -- 172.21.15.18:0/661383414 shutdown_connections 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9730410 0x55ddf9720700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9727830 0x55ddf972fc40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/661383414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9726f10 0x55ddf973c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 -- 172.21.15.18:0/661383414 >> 172.21.15.18:0/661383414 conn(0x55ddf9721640 msgr2=0x55ddf9721a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 -- 172.21.15.18:0/661383414 shutdown_connections 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 -- 172.21.15.18:0/661383414 wait complete. 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 -- start start 2024-04-16T17:11:02.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.385+0000 7f76fcbfd740 1 Processor -- start 2024-04-16T17:11:02.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf9734370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf97348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76fcbfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf97a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b3f60 con 0x55ddf9726f10 2024-04-16T17:11:02.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b53d0 con 0x55ddf9727830 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76fcbfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b4630 con 0x55ddf9730410 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76fcbfd740 10 client.0 _ll_register_callbacks cb 0x7f76e00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76f3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf9734370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76f3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf9734370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55150/0 (socket says 172.21.15.18:55150) 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76f3e00640 1 -- 172.21.15.18:0/2257964251 learned_addr learned my addr 172.21.15.18:0/2257964251 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76f2a00640 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf97348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.386+0000 7f76f3e00640 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf97a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76f2a00640 1 -- 172.21.15.18:0/2257964251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 msgr2=0x55ddf97a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76f2a00640 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf97a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76f2a00640 1 -- 172.21.15.18:0/2257964251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 msgr2=0x55ddf9734370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76f2a00640 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf9734370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76f2a00640 1 -- 172.21.15.18:0/2257964251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf9727660 con 0x55ddf9727830 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76f2a00640 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf97348b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55ddf9726150 tx=0x7f76e000ffc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38e1098018e9fca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76dcc00640 1 -- 172.21.15.18:0/2257964251 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0011d50 con 0x55ddf9727830 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2257964251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e000f5e0 con 0x55ddf9727830 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.387+0000 7f76dcc00640 1 -- 172.21.15.18:0/2257964251 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f76e0018040 con 0x55ddf9727830 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76dcc00640 1 -- 172.21.15.18:0/2257964251 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f76e004d050 con 0x55ddf9727830 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76fcbfd740 10 client.27311 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2257964251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddf939e3f0 con 0x55ddf9727830 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76dcc00640 1 -- 172.21.15.18:0/2257964251 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e000f5e0 con 0x55ddf9727830 2024-04-16T17:11:02.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76dcc00640 1 -- 172.21.15.18:0/2257964251 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f76e002e030 con 0x55ddf9727830 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76fcbfd740 10 client.27311 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76fcbfd740 10 client.27311 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76fcbfd740 20 client.27311 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ddf97a77d0 0x55ddf97a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76fcbfd740 4 client.27311 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.388+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2257964251 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddf96938b0 con 0x55ddf97a77d0 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.389+0000 7f76f3e00640 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ddf97a77d0 0x55ddf97a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.389+0000 7f76f3e00640 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ddf97a77d0 0x55ddf97a9bf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7182bda0b70a823 server_cookie=cfbf44996d67c675 in_seq=0 out_seq=0 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.389+0000 7f76dcc00640 10 client.27311 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.389+0000 7f76dcc00640 1 -- 172.21.15.18:0/2257964251 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddf96938b0 con 0x55ddf97a77d0 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.389+0000 7f76dcc00640 10 client.27311 handle_command_reply: tid=0 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.389+0000 7f76fcbfd740 1 client.27311 shutdown 2024-04-16T17:11:02.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2257964251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 msgr2=0x55ddf97348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf97348b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55ddf9726150 tx=0x7f76e000ffc0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2257964251 shutdown_connections 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ddf97a77d0 0x55ddf97a9bf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddf9730410 0x55ddf97a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ddf9727830 0x55ddf97348b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 --2- 172.21.15.18:0/2257964251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ddf9726f10 0x55ddf9734370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76dcc00640 0 client.27311 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2257964251 >> 172.21.15.18:0/2257964251 conn(0x55ddf9721640 msgr2=0x55ddf972ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2257964251 shutdown_connections 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 1 -- 172.21.15.18:0/2257964251 wait complete. 2024-04-16T17:11:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.390+0000 7f76fcbfd740 20 client.27311 trim_cache size 0 max 16384 2024-04-16T17:11:02.394 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:11:02.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 -- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76d80a3d20 msgr2=0x7f76d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76d80a3d20 0x7f76d80a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55ddf97269e0 tx=0x7f76e402cdf0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 -- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4100f60 msgr2=0x7f76f4196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4100f60 0x7f76f4196340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f76e402c240 tx=0x7f76e40042f0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 -- 172.21.15.18:0/568465200 shutdown_connections 2024-04-16T17:11:02.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76d80a3d20 0x7f76d80a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f4196dc0 0x7f76f41aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76f4106030 0x7f76f4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 --2- 172.21.15.18:0/568465200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76f4100f60 0x7f76f4196340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 -- 172.21.15.18:0/568465200 >> 172.21.15.18:0/568465200 conn(0x7f76f4077ab0 msgr2=0x7f76f40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 -- 172.21.15.18:0/568465200 shutdown_connections 2024-04-16T17:11:02.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.391+0000 7f76f8a00640 1 -- 172.21.15.18:0/568465200 wait complete. 2024-04-16T17:11:02.408 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:11:02.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 235692 3.95 MB/sec execute 195 sec latency 15.285 ms 2024-04-16T17:11:02.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 -- 172.21.15.18:0/201137450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e010c400 msgr2=0x7f94e010e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 --2- 172.21.15.18:0/201137450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e010c400 0x7f94e010e870 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009be0 tx=0x7f94d802bf80 comp rx=0 tx=0).stop 2024-04-16T17:11:02.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 -- 172.21.15.18:0/201137450 shutdown_connections 2024-04-16T17:11:02.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 --2- 172.21.15.18:0/201137450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e010c400 0x7f94e010e870 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 --2- 172.21.15.18:0/201137450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e0103900 0x7f94e010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 --2- 172.21.15.18:0/201137450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94e0102fe0 0x7f94e01033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 -- 172.21.15.18:0/201137450 >> 172.21.15.18:0/201137450 conn(0x7f94e0077ab0 msgr2=0x7f94e00fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 -- 172.21.15.18:0/201137450 shutdown_connections 2024-04-16T17:11:02.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.624+0000 7f94e6e00640 1 -- 172.21.15.18:0/201137450 wait complete. 2024-04-16T17:11:02.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.625+0000 7f94e6e00640 1 Processor -- start 2024-04-16T17:11:02.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e6e00640 1 -- start start 2024-04-16T17:11:02.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e6e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e0102fe0 0x7f94e01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e6e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e0103900 0x7f94e01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e6e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94e010c400 0x7f94e019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e6e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f94e0112190 con 0x7f94e0103900 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e6e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94e0112010 con 0x7f94e0102fe0 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e6e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f94e0112310 con 0x7f94e010c400 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e0102fe0 0x7f94e01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94e010c400 0x7f94e019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e0102fe0 0x7f94e01a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36662/0 (socket says 172.21.15.18:36662) 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e5a00640 1 -- 172.21.15.18:0/1845424399 learned_addr learned my addr 172.21.15.18:0/1845424399 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.626+0000 7f94e5000640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e0103900 0x7f94e01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.627+0000 7f94e6400640 1 -- 172.21.15.18:0/1845424399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e0102fe0 msgr2=0x7f94e01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.627+0000 7f94e6400640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e0102fe0 0x7f94e01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.627+0000 7f94e6400640 1 -- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e0103900 msgr2=0x7f94e01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.627+0000 7f94e6400640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e0103900 0x7f94e01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.627+0000 7f94e6400640 1 -- 172.21.15.18:0/1845424399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e019b030 con 0x7f94e010c400 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.627+0000 7f94e6400640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94e010c400 0x7f94e019a770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f94d802c4c0 tx=0x7f94d8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.627+0000 7f94de000640 1 -- 172.21.15.18:0/1845424399 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8043020 con 0x7f94e010c400 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.628+0000 7f94de000640 1 -- 172.21.15.18:0/1845424399 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d8004670 con 0x7f94e010c400 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.628+0000 7f94e6e00640 1 -- 172.21.15.18:0/1845424399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e019b2c0 con 0x7f94e010c400 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.628+0000 7f94de000640 1 -- 172.21.15.18:0/1845424399 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8034690 con 0x7f94e010c400 2024-04-16T17:11:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.628+0000 7f94e6e00640 1 -- 172.21.15.18:0/1845424399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e01a7630 con 0x7f94e010c400 2024-04-16T17:11:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.632+0000 7f94eb164740 1 Processor -- start 2024-04-16T17:11:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.633+0000 7f94de000640 1 -- 172.21.15.18:0/1845424399 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f94d8046020 con 0x7f94e010c400 2024-04-16T17:11:02.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.633+0000 7f94eb164740 1 -- start start 2024-04-16T17:11:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94eb164740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6c2d20 0x5572eb6c3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94eb164740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94eb164740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6d4c20 0x5572eb6d7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94eb164740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5572eb628020 con 0x5572eb6c2d20 2024-04-16T17:11:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94eb164740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5572eb5ce460 con 0x5572eb6c36d0 2024-04-16T17:11:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94eb164740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5572eb623fe0 con 0x5572eb6d4c20 2024-04-16T17:11:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94e5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6c2d20 0x5572eb6c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94e5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6c2d20 0x5572eb6c3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55174/0 (socket says 172.21.15.18:55174) 2024-04-16T17:11:02.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.634+0000 7f94e5000640 1 -- 172.21.15.18:0/3830782096 learned_addr learned my addr 172.21.15.18:0/3830782096 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.635+0000 7f94e5a00640 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.635+0000 7f94e5000640 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6d4c20 0x5572eb6d7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.635+0000 7f94e5000640 1 -- 172.21.15.18:0/3830782096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 msgr2=0x5572eb6d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.635+0000 7f94e5000640 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.635+0000 7f94e5000640 1 -- 172.21.15.18:0/3830782096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6c2d20 msgr2=0x5572eb6c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.635+0000 7f94e5000640 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6c2d20 0x5572eb6c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.635+0000 7f94e5000640 1 -- 172.21.15.18:0/3830782096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572eb5d6040 con 0x5572eb6d4c20 2024-04-16T17:11:02.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.635+0000 7f94e5000640 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6d4c20 0x5572eb6d7010 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0137d0 tx=0x7f94cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=380d84591ea28985 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.636+0000 7f94d2a00640 1 -- 172.21.15.18:0/3830782096 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc013050 con 0x5572eb6d4c20 2024-04-16T17:11:02.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.636+0000 7f94d2a00640 1 -- 172.21.15.18:0/3830782096 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc0131f0 con 0x5572eb6d4c20 2024-04-16T17:11:02.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.636+0000 7f94d2a00640 1 -- 172.21.15.18:0/3830782096 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc003850 con 0x5572eb6d4c20 2024-04-16T17:11:02.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.636+0000 7f94de000640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f94c80a3d20 0x7f94c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.637+0000 7f94de000640 1 -- 172.21.15.18:0/1845424399 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f94d80f31b0 con 0x7f94e010c400 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.637+0000 7f94eb164740 1 -- 172.21.15.18:0/3830782096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6d4c20 msgr2=0x5572eb6d7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.637+0000 7f94eb164740 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6d4c20 0x5572eb6d7010 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0137d0 tx=0x7f94cc004730 comp rx=0 tx=0).stop 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.637+0000 7f94e5a00640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f94c80a3d20 0x7f94c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.638+0000 7f94e5a00640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f94c80a3d20 0x7f94c80a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5572eb6be8d0 tx=0x7f94d400b480 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.638+0000 7f94eb164740 1 -- 172.21.15.18:0/3830782096 shutdown_connections 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.638+0000 7f94eb164740 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6d4c20 0x5572eb6d7010 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.638+0000 7f94eb164740 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.638+0000 7f94eb164740 1 --2- 172.21.15.18:0/3830782096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6c2d20 0x5572eb6c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.638+0000 7f94eb164740 1 -- 172.21.15.18:0/3830782096 >> 172.21.15.18:0/3830782096 conn(0x5572eb6bd640 msgr2=0x5572eb6bda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.639+0000 7f94eb164740 1 -- 172.21.15.18:0/3830782096 shutdown_connections 2024-04-16T17:11:02.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.639+0000 7f94eb164740 1 -- 172.21.15.18:0/3830782096 wait complete. 2024-04-16T17:11:02.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 1 -- start start 2024-04-16T17:11:02.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 1 Processor -- start 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5572eb6d7980 con 0x5572eb6d4c20 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5572eb5e1830 con 0x5572eb6c36d0 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5572eb6d7b00 con 0x5572eb6c2d20 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.640+0000 7f94eb164740 10 client.0 _ll_register_callbacks cb 0x5572eb5e9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36668/0 (socket says 172.21.15.18:36668) 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5000640 1 -- 172.21.15.18:0/3522364408 learned_addr learned my addr 172.21.15.18:0/3522364408 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e6400640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5a00640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5000640 1 -- 172.21.15.18:0/3522364408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 msgr2=0x5572eb6d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5000640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5000640 1 -- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 msgr2=0x5572eb6d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5000640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5000640 1 -- 172.21.15.18:0/3522364408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572eb5e5920 con 0x5572eb6c36d0 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e5a00640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.641+0000 7f94e6400640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.642+0000 7f94e5000640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d2220 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009530 tx=0x7f94cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62e9b4ae2f3c147 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.642+0000 7f94c7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.642+0000 7f94c7400640 1 -- 172.21.15.18:0/3522364408 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc017070 con 0x5572eb6c36d0 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.642+0000 7f94c7400640 1 -- 172.21.15.18:0/3522364408 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f94cc005020 con 0x5572eb6c36d0 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.642+0000 7f94eb164740 1 -- 172.21.15.18:0/3522364408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94cc0131f0 con 0x5572eb6c36d0 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.642+0000 7f94c7400640 1 -- 172.21.15.18:0/3522364408 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94cc030050 con 0x5572eb6c36d0 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.642+0000 7f94eb164740 10 client.27323 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.642+0000 7f94eb164740 1 -- 172.21.15.18:0/3522364408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572eb5b37f0 con 0x5572eb6c36d0 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94c7400640 1 -- 172.21.15.18:0/3522364408 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc0131f0 con 0x5572eb6c36d0 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94c7400640 1 -- 172.21.15.18:0/3522364408 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f94cc004af0 con 0x5572eb6c36d0 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94eb164740 10 client.27323 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94eb164740 10 client.27323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94eb164740 20 client.27323 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94eb164740 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5572eb6e4ed0 0x5572eb6e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94eb164740 4 client.27323 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94eb164740 1 -- 172.21.15.18:0/3522364408 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572eb635a40 con 0x5572eb6e4ed0 2024-04-16T17:11:02.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.643+0000 7f94e5a00640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5572eb6e4ed0 0x5572eb6e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.644+0000 7f94e5a00640 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5572eb6e4ed0 0x5572eb6e72c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cf3b11cf2156582 server_cookie=53b9855422dbe2c2 in_seq=0 out_seq=0 2024-04-16T17:11:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.645+0000 7f94c7400640 10 client.27323 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:11:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.645+0000 7f94c7400640 1 -- 172.21.15.18:0/3522364408 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572eb635a40 con 0x5572eb6e4ed0 2024-04-16T17:11:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.645+0000 7f94c7400640 10 client.27323 handle_command_reply: tid=0 2024-04-16T17:11:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.645+0000 7f94eb164740 1 client.27323 shutdown 2024-04-16T17:11:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 -- 172.21.15.18:0/3522364408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 msgr2=0x5572eb6d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d2220 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009530 tx=0x7f94cc0046c0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 -- 172.21.15.18:0/3522364408 shutdown_connections 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5572eb6e4ed0 0x5572eb6e72c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6d2220 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 --2- 172.21.15.18:0/3522364408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94c7400640 0 client.27323 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 -- 172.21.15.18:0/3522364408 >> 172.21.15.18:0/3522364408 conn(0x5572eb6bd640 msgr2=0x5572eb6bf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 -- 172.21.15.18:0/3522364408 shutdown_connections 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 1 -- 172.21.15.18:0/3522364408 wait complete. 2024-04-16T17:11:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.646+0000 7f94eb164740 20 client.27323 trim_cache size 0 max 16384 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.661+0000 7f94eb164740 1 Processor -- start 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.661+0000 7f94eb164740 1 -- start start 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.661+0000 7f94eb164740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.661+0000 7f94eb164740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6e0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.661+0000 7f94eb164740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.661+0000 7f94eb164740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f94e01afa80 con 0x5572eb6d4c20 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.661+0000 7f94eb164740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94e01ad350 con 0x5572eb6c36d0 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.661+0000 7f94eb164740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f94e01ab1e0 con 0x5572eb6c2d20 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6e0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53484/0 (socket says 172.21.15.18:53484) 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 -- 172.21.15.18:0/179312591 learned_addr learned my addr 172.21.15.18:0/179312591 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e6400640 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 -- 172.21.15.18:0/179312591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 msgr2=0x5572eb6e0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6e0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 -- 172.21.15.18:0/179312591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 msgr2=0x5572eb6d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 -- 172.21.15.18:0/179312591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572eb6d7380 con 0x5572eb6c2d20 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e6400640 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.662+0000 7f94e5a00640 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d4410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f94d4004010 tx=0x7f94d4019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d7e97556e36535 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.663+0000 7f94d2000640 1 -- 172.21.15.18:0/179312591 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d401eb60 con 0x5572eb6c2d20 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.663+0000 7f94d2000640 1 -- 172.21.15.18:0/179312591 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d402a900 con 0x5572eb6c2d20 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.663+0000 7f94d2000640 1 -- 172.21.15.18:0/179312591 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40047a0 con 0x5572eb6c2d20 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.663+0000 7f94eb164740 1 -- 172.21.15.18:0/179312591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 msgr2=0x5572eb6d4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.663+0000 7f94eb164740 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d4410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f94d4004010 tx=0x7f94d4019000 comp rx=0 tx=0).stop 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 -- 172.21.15.18:0/179312591 shutdown_connections 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6e0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 --2- 172.21.15.18:0/179312591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6d4410 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 -- 172.21.15.18:0/179312591 >> 172.21.15.18:0/179312591 conn(0x5572eb6bd640 msgr2=0x5572eb6bea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 -- 172.21.15.18:0/179312591 shutdown_connections 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 -- 172.21.15.18:0/179312591 wait complete. 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 -- start start 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.664+0000 7f94eb164740 1 Processor -- start 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94eb164740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6dcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94eb164740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94eb164740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94eb164740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f94e01ab830 con 0x5572eb6d4c20 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94eb164740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94e01acc80 con 0x5572eb6c36d0 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94eb164740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f94e01abee0 con 0x5572eb6c2d20 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94eb164740 10 client.0 _ll_register_callbacks cb 0x7f94cc009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94e5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94e5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36688/0 (socket says 172.21.15.18:36688) 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94e5000640 1 -- 172.21.15.18:0/2830715471 learned_addr learned my addr 172.21.15.18:0/2830715471 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94e5a00640 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6dcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.665+0000 7f94e6400640 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94e5000640 1 -- 172.21.15.18:0/2830715471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 msgr2=0x5572eb6dcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94e5000640 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6dcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94e5000640 1 -- 172.21.15.18:0/2830715471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 msgr2=0x5572eb6ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94e5000640 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94e5000640 1 -- 172.21.15.18:0/2830715471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572eb33a3f0 con 0x5572eb6c36d0 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94e5000640 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6ce5b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009340 tx=0x7f94cc005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3be878f74756a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94c7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94c7400640 1 -- 172.21.15.18:0/2830715471 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc017070 con 0x5572eb6c36d0 2024-04-16T17:11:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94c7400640 1 -- 172.21.15.18:0/2830715471 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f94cc003af0 con 0x5572eb6c36d0 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94eb164740 1 -- 172.21.15.18:0/2830715471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d402a900 con 0x5572eb6c36d0 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94c7400640 1 -- 172.21.15.18:0/2830715471 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94cc003cf0 con 0x5572eb6c36d0 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94eb164740 10 client.27329 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.666+0000 7f94eb164740 1 -- 172.21.15.18:0/2830715471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572eb4047d0 con 0x5572eb6c36d0 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94c7400640 1 -- 172.21.15.18:0/2830715471 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d402a900 con 0x5572eb6c36d0 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94c7400640 1 -- 172.21.15.18:0/2830715471 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f94cc004970 con 0x5572eb6c36d0 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94eb164740 10 client.27329 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94eb164740 10 client.27329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94eb164740 20 client.27329 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94eb164740 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5572eb74b4a0 0x5572eb74d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94eb164740 4 client.27329 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94eb164740 1 -- 172.21.15.18:0/2830715471 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572eb62f8b0 con 0x5572eb74b4a0 2024-04-16T17:11:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.667+0000 7f94e5a00640 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5572eb74b4a0 0x5572eb74d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.668+0000 7f94e5a00640 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5572eb74b4a0 0x5572eb74d8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2061e3a273c85757 server_cookie=6c97438c4b4be219 in_seq=0 out_seq=0 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.668+0000 7f94c7400640 10 client.27329 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.668+0000 7f94c7400640 1 -- 172.21.15.18:0/2830715471 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572eb62f8b0 con 0x5572eb74b4a0 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94c7400640 10 client.27329 handle_command_reply: tid=0 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 client.27329 shutdown 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 -- 172.21.15.18:0/2830715471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 msgr2=0x5572eb6ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6ce5b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009340 tx=0x7f94cc005e60 comp rx=0 tx=0).stop 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 -- 172.21.15.18:0/2830715471 shutdown_connections 2024-04-16T17:11:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5572eb74b4a0 0x5572eb74d8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5572eb6d4c20 0x5572eb6ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572eb6c36d0 0x5572eb6ce5b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94c7400640 0 client.27329 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 --2- 172.21.15.18:0/2830715471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572eb6c2d20 0x5572eb6dcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 -- 172.21.15.18:0/2830715471 >> 172.21.15.18:0/2830715471 conn(0x5572eb6bd640 msgr2=0x5572eb6bf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 -- 172.21.15.18:0/2830715471 shutdown_connections 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 1 -- 172.21.15.18:0/2830715471 wait complete. 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.669+0000 7f94eb164740 20 client.27329 trim_cache size 0 max 16384 2024-04-16T17:11:02.674 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:11:02.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 -- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f94c80a3d20 msgr2=0x7f94c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f94c80a3d20 0x7f94c80a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5572eb6be8d0 tx=0x7f94d400b480 comp rx=0 tx=0).stop 2024-04-16T17:11:02.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 -- 172.21.15.18:0/1845424399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94e010c400 msgr2=0x7f94e019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94e010c400 0x7f94e019a770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f94d802c4c0 tx=0x7f94d8004270 comp rx=0 tx=0).stop 2024-04-16T17:11:02.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 -- 172.21.15.18:0/1845424399 shutdown_connections 2024-04-16T17:11:02.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f94c80a3d20 0x7f94c80a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94e010c400 0x7f94e019a770 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94e0103900 0x7f94e01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 --2- 172.21.15.18:0/1845424399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e0102fe0 0x7f94e01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 -- 172.21.15.18:0/1845424399 >> 172.21.15.18:0/1845424399 conn(0x7f94e0077ab0 msgr2=0x7f94e00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 -- 172.21.15.18:0/1845424399 shutdown_connections 2024-04-16T17:11:02.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.670+0000 7f94e6e00640 1 -- 172.21.15.18:0/1845424399 wait complete. 2024-04-16T17:11:02.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:02 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:11:02.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:02 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:11:02.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:02 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:11:02.676 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:02 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/234028323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:02.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:11:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:02 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:11:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:02 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:11:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:02 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:11:02.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:02 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/234028323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.877+0000 7f11bb200640 1 -- 172.21.15.18:0/2236094306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b41061d0 msgr2=0x7f11b4106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.877+0000 7f11bb200640 1 --2- 172.21.15.18:0/2236094306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b41061d0 0x7f11b4106630 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009cd0 tx=0x7f11b002c170 comp rx=0 tx=0).stop 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.878+0000 7f11bb200640 1 -- 172.21.15.18:0/2236094306 shutdown_connections 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.878+0000 7f11bb200640 1 --2- 172.21.15.18:0/2236094306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b41061d0 0x7f11b4106630 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.878+0000 7f11bb200640 1 --2- 172.21.15.18:0/2236094306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b41016f0 0x7f11b4105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.878+0000 7f11bb200640 1 --2- 172.21.15.18:0/2236094306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b4100dd0 0x7f11b41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.878+0000 7f11bb200640 1 -- 172.21.15.18:0/2236094306 >> 172.21.15.18:0/2236094306 conn(0x7f11b4077ab0 msgr2=0x7f11b40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.878+0000 7f11bb200640 1 -- 172.21.15.18:0/2236094306 shutdown_connections 2024-04-16T17:11:02.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.878+0000 7f11bb200640 1 -- 172.21.15.18:0/2236094306 wait complete. 2024-04-16T17:11:02.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.879+0000 7f11bb200640 1 Processor -- start 2024-04-16T17:11:02.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11bb200640 1 -- start start 2024-04-16T17:11:02.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11bb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b4100dd0 0x7f11b419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11bb200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b41016f0 0x7f11b419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11bb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b41061d0 0x7f11b41a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11bb200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41187c0 con 0x7f11b41016f0 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11bb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11b4118640 con 0x7f11b41061d0 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11bb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11b4118940 con 0x7f11b4100dd0 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11b9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b4100dd0 0x7f11b419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11b9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b4100dd0 0x7f11b419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53510/0 (socket says 172.21.15.18:53510) 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11b9e00640 1 -- 172.21.15.18:0/21426071 learned_addr learned my addr 172.21.15.18:0/21426071 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.880+0000 7f11ba800640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b41061d0 0x7f11b41a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.881+0000 7f11b9400640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b41016f0 0x7f11b419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.881+0000 7f11b9e00640 1 -- 172.21.15.18:0/21426071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b41061d0 msgr2=0x7f11b41a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.881+0000 7f11b9e00640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b41061d0 0x7f11b41a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.881+0000 7f11b9e00640 1 -- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b41016f0 msgr2=0x7f11b419f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.881+0000 7f11b9e00640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b41016f0 0x7f11b419f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.881+0000 7f11b9e00640 1 -- 172.21.15.18:0/21426071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b41a3d90 con 0x7f11b4100dd0 2024-04-16T17:11:02.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.881+0000 7f11b9e00640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b4100dd0 0x7f11b419eda0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11a800ea60 tx=0x7f11a800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.882+0000 7f11aea00640 1 -- 172.21.15.18:0/21426071 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a800cde0 con 0x7f11b4100dd0 2024-04-16T17:11:02.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.882+0000 7f11bb200640 1 -- 172.21.15.18:0/21426071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b41a4080 con 0x7f11b4100dd0 2024-04-16T17:11:02.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.882+0000 7f11aea00640 1 -- 172.21.15.18:0/21426071 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a80035f0 con 0x7f11b4100dd0 2024-04-16T17:11:02.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.882+0000 7f11aea00640 1 -- 172.21.15.18:0/21426071 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8010690 con 0x7f11b4100dd0 2024-04-16T17:11:02.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.882+0000 7f11bb200640 1 -- 172.21.15.18:0/21426071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b41ab910 con 0x7f11b4100dd0 2024-04-16T17:11:02.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.885+0000 7f11aea00640 1 -- 172.21.15.18:0/21426071 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11a80037a0 con 0x7f11b4100dd0 2024-04-16T17:11:02.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.886+0000 7f11bf575740 1 Processor -- start 2024-04-16T17:11:02.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.886+0000 7f11bf575740 1 -- start start 2024-04-16T17:11:02.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.886+0000 7f11bf575740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a68f10 0x563a33a692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11bf575740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a69830 0x563a33a71ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11aea00640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f119c0a3da0 0x7f119c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11bf575740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a74800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11bf575740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563a339ce020 con 0x563a33a72410 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11bf575740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563a33974460 con 0x563a33a69830 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11bf575740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a339c9fe0 con 0x563a33a68f10 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11aea00640 1 -- 172.21.15.18:0/21426071 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f11a80d0ea0 con 0x7f11b4100dd0 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11ba800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a68f10 0x563a33a692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11ba800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a68f10 0x563a33a692f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53520/0 (socket says 172.21.15.18:53520) 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11ba800640 1 -- 172.21.15.18:0/2856025860 learned_addr learned my addr 172.21.15.18:0/2856025860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11b9400640 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a69830 0x563a33a71ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.887+0000 7f11ba800640 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a74800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.888+0000 7f11b9e00640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f119c0a3da0 0x7f119c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.888+0000 7f11b9400640 1 -- 172.21.15.18:0/2856025860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a68f10 msgr2=0x563a33a692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.888+0000 7f11b9400640 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a68f10 0x563a33a692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.888+0000 7f11b9400640 1 -- 172.21.15.18:0/2856025860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 msgr2=0x563a33a74800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.888+0000 7f11b9400640 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a74800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.888+0000 7f11b9400640 1 -- 172.21.15.18:0/2856025860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3397c040 con 0x563a33a69830 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.888+0000 7f11b9400640 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a69830 0x563a33a71ed0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009ac0 tx=0x7f11a4009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=988b022e3a4ed4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.888+0000 7f11a3000640 1 -- 172.21.15.18:0/2856025860 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4011d20 con 0x563a33a69830 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.889+0000 7f11a3000640 1 -- 172.21.15.18:0/2856025860 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4012930 con 0x563a33a69830 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.889+0000 7f11a3000640 1 -- 172.21.15.18:0/2856025860 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a400f9a0 con 0x563a33a69830 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.889+0000 7f11b9e00640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f119c0a3da0 0x7f119c0a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f11a80ce240 tx=0x7f11a80045b0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.889+0000 7f11bf575740 1 -- 172.21.15.18:0/2856025860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a69830 msgr2=0x563a33a71ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.889+0000 7f11bf575740 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a69830 0x563a33a71ed0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009ac0 tx=0x7f11a4009f80 comp rx=0 tx=0).stop 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 -- 172.21.15.18:0/2856025860 shutdown_connections 2024-04-16T17:11:02.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a74800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a69830 0x563a33a71ed0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 --2- 172.21.15.18:0/2856025860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a68f10 0x563a33a692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 -- 172.21.15.18:0/2856025860 >> 172.21.15.18:0/2856025860 conn(0x563a33a63640 msgr2=0x563a33a63a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 -- 172.21.15.18:0/2856025860 shutdown_connections 2024-04-16T17:11:02.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 -- 172.21.15.18:0/2856025860 wait complete. 2024-04-16T17:11:02.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 -- start start 2024-04-16T17:11:02.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.890+0000 7f11bf575740 1 Processor -- start 2024-04-16T17:11:02.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11bf575740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a6f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11bf575740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a6f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11bf575740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11bf575740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563a33a75200 con 0x563a33a72410 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11bf575740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563a33987830 con 0x563a33a68f10 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11bf575740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a33a75380 con 0x563a33a69830 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11bf575740 10 client.0 _ll_register_callbacks cb 0x563a3398f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11ba800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a6f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11ba800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a6f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36720/0 (socket says 172.21.15.18:36720) 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11ba800640 1 -- 172.21.15.18:0/258182458 learned_addr learned my addr 172.21.15.18:0/258182458 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11ba800640 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.891+0000 7f11b9400640 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a6f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.892+0000 7f11b9400640 1 -- 172.21.15.18:0/258182458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 msgr2=0x563a33a6f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.892+0000 7f11b9400640 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a6f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.892+0000 7f11b9400640 1 -- 172.21.15.18:0/258182458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 msgr2=0x563a33a6ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.892+0000 7f11b9400640 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.892+0000 7f11b9400640 1 -- 172.21.15.18:0/258182458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3398b920 con 0x563a33a69830 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.892+0000 7f11ba800640 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a6f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.892+0000 7f11b9400640 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a6f9e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009ac0 tx=0x7f11a4012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b30adec4d125f8ca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.892+0000 7f1197400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.893+0000 7f1197400640 1 -- 172.21.15.18:0/258182458 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4028020 con 0x563a33a69830 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.893+0000 7f11bf575740 1 -- 172.21.15.18:0/258182458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a4012930 con 0x563a33a69830 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.893+0000 7f1197400640 1 -- 172.21.15.18:0/258182458 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f11a4004040 con 0x563a33a69830 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.893+0000 7f1197400640 1 -- 172.21.15.18:0/258182458 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11a4033050 con 0x563a33a69830 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.893+0000 7f11bf575740 10 client.27292 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.893+0000 7f11bf575740 1 -- 172.21.15.18:0/258182458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a339597f0 con 0x563a33a69830 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.893+0000 7f1197400640 1 -- 172.21.15.18:0/258182458 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4012930 con 0x563a33a69830 2024-04-16T17:11:02.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.894+0000 7f1197400640 1 -- 172.21.15.18:0/258182458 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f11a400f970 con 0x563a33a69830 2024-04-16T17:11:02.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.894+0000 7f11bf575740 10 client.27292 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:02.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.894+0000 7f11bf575740 10 client.27292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:02.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.894+0000 7f11bf575740 20 client.27292 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:02.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.894+0000 7f11bf575740 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563a33a827f0 0x563a33a84be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.894+0000 7f11bf575740 4 client.27292 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:02.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.894+0000 7f11bf575740 1 -- 172.21.15.18:0/258182458 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a339dba40 con 0x563a33a827f0 2024-04-16T17:11:02.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.894+0000 7f11ba800640 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563a33a827f0 0x563a33a84be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.895+0000 7f1197400640 10 client.27292 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.895+0000 7f11ba800640 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563a33a827f0 0x563a33a84be0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1e90bb30e83255 server_cookie=544185fd6282d0cd in_seq=0 out_seq=0 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.896+0000 7f1197400640 1 -- 172.21.15.18:0/258182458 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a339dba40 con 0x563a33a827f0 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.896+0000 7f1197400640 10 client.27292 handle_command_reply: tid=0 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.896+0000 7f11bf575740 1 client.27292 shutdown 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 -- 172.21.15.18:0/258182458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 msgr2=0x563a33a6f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a6f9e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009ac0 tx=0x7f11a4012680 comp rx=0 tx=0).stop 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 -- 172.21.15.18:0/258182458 shutdown_connections 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563a33a827f0 0x563a33a84be0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a6f9e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 --2- 172.21.15.18:0/258182458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a6f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 -- 172.21.15.18:0/258182458 >> 172.21.15.18:0/258182458 conn(0x563a33a63640 msgr2=0x563a33a65390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f1197400640 0 client.27292 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:02.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 -- 172.21.15.18:0/258182458 shutdown_connections 2024-04-16T17:11:02.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 1 -- 172.21.15.18:0/258182458 wait complete. 2024-04-16T17:11:02.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.897+0000 7f11bf575740 20 client.27292 trim_cache size 0 max 16384 2024-04-16T17:11:02.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.925+0000 7f11bf575740 1 Processor -- start 2024-04-16T17:11:02.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11bf575740 1 -- start start 2024-04-16T17:11:02.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11bf575740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a71bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11bf575740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a7e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11bf575740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11bf575740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41b82b0 con 0x563a33a72410 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11bf575740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41b5b80 con 0x563a33a68f10 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11bf575740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41b39d0 con 0x563a33a69830 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11ba800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a7e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11ba800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55222/0 (socket says 172.21.15.18:55222) 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a7e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53536/0 (socket says 172.21.15.18:53536) 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.926+0000 7f11ba800640 1 -- 172.21.15.18:0/3555791454 learned_addr learned my addr 172.21.15.18:0/3555791454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.927+0000 7f11ba800640 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a71bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.927+0000 7f11b9400640 1 -- 172.21.15.18:0/3555791454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 msgr2=0x563a33a71bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.927+0000 7f11b9400640 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a71bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.927+0000 7f11b9400640 1 -- 172.21.15.18:0/3555791454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 msgr2=0x563a33a6fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.927+0000 7f11b9400640 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.927+0000 7f11b9400640 1 -- 172.21.15.18:0/3555791454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a336e03f0 con 0x563a33a69830 2024-04-16T17:11:02.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.927+0000 7f11ba800640 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a71bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:02.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.927+0000 7f11b9400640 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a7e0c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f11a400fd40 tx=0x7f11a40118f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e1f2e4969b4c75c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.928+0000 7f11a2600640 1 -- 172.21.15.18:0/3555791454 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a403c070 con 0x563a33a69830 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.928+0000 7f11a2600640 1 -- 172.21.15.18:0/3555791454 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4003e20 con 0x563a33a69830 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.928+0000 7f11a2600640 1 -- 172.21.15.18:0/3555791454 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4028030 con 0x563a33a69830 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 -- 172.21.15.18:0/3555791454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 msgr2=0x563a33a7e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a7e0c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f11a400fd40 tx=0x7f11a40118f0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 -- 172.21.15.18:0/3555791454 shutdown_connections 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a72410 0x563a33a6fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a69830 0x563a33a7e0c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 --2- 172.21.15.18:0/3555791454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a71bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 -- 172.21.15.18:0/3555791454 >> 172.21.15.18:0/3555791454 conn(0x563a33a63640 msgr2=0x563a33a6f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 -- 172.21.15.18:0/3555791454 shutdown_connections 2024-04-16T17:11:02.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 -- 172.21.15.18:0/3555791454 wait complete. 2024-04-16T17:11:02.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 -- start start 2024-04-16T17:11:02.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.929+0000 7f11bf575740 1 Processor -- start 2024-04-16T17:11:02.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11bf575740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a75590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11bf575740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a69830 0x563a33a75ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11bf575740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a72410 0x563a33ae6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11bf575740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41b4040 con 0x563a33a69830 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11bf575740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41b54b0 con 0x563a33a68f10 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11bf575740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41b4710 con 0x563a33a72410 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11bf575740 10 client.0 _ll_register_callbacks cb 0x7f11a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11b9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a69830 0x563a33a75ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11b9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a69830 0x563a33a75ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55224/0 (socket says 172.21.15.18:55224) 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11b9400640 1 -- 172.21.15.18:0/2128585161 learned_addr learned my addr 172.21.15.18:0/2128585161 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11ba800640 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a75590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.930+0000 7f11ba800640 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a72410 0x563a33ae6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f11b9400640 1 -- 172.21.15.18:0/2128585161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a72410 msgr2=0x563a33ae6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f11b9400640 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a72410 0x563a33ae6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f11b9400640 1 -- 172.21.15.18:0/2128585161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 msgr2=0x563a33a75590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f11b9400640 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a75590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f11b9400640 1 -- 172.21.15.18:0/2128585161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a337aa7d0 con 0x563a33a69830 2024-04-16T17:11:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f11b9400640 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a69830 0x563a33a75ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f11a400fdc0 tx=0x7f11a40128b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972e76847a16766b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:02.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f1197400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:02.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f1197400640 1 -- 172.21.15.18:0/2128585161 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4028070 con 0x563a33a69830 2024-04-16T17:11:02.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f1197400640 1 -- 172.21.15.18:0/2128585161 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f11a4003b20 con 0x563a33a69830 2024-04-16T17:11:02.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.931+0000 7f11bf575740 1 -- 172.21.15.18:0/2128585161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a4003e20 con 0x563a33a69830 2024-04-16T17:11:02.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.932+0000 7f1197400640 1 -- 172.21.15.18:0/2128585161 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11a404a050 con 0x563a33a69830 2024-04-16T17:11:02.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.932+0000 7f11bf575740 10 client.17094 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:02.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.932+0000 7f11bf575740 1 -- 172.21.15.18:0/2128585161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a33ae7f10 con 0x563a33a69830 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.932+0000 7f1197400640 1 -- 172.21.15.18:0/2128585161 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4003e20 con 0x563a33a69830 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.932+0000 7f1197400640 1 -- 172.21.15.18:0/2128585161 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f11a4005230 con 0x563a33a69830 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.932+0000 7f11bf575740 10 client.17094 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.932+0000 7f11bf575740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.932+0000 7f11bf575740 20 client.17094 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.933+0000 7f11bf575740 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563a33ae8590 0x563a33aea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.933+0000 7f11bf575740 4 client.17094 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.933+0000 7f11bf575740 1 -- 172.21.15.18:0/2128585161 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a339d58b0 con 0x563a33ae8590 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.933+0000 7f11ba800640 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563a33ae8590 0x563a33aea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.933+0000 7f11ba800640 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563a33ae8590 0x563a33aea9b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a42c4ab2213c88 server_cookie=cae0cb81ad217886 in_seq=0 out_seq=0 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.933+0000 7f1197400640 10 client.17094 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f1197400640 1 -- 172.21.15.18:0/2128585161 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a339d58b0 con 0x563a33ae8590 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f1197400640 10 client.17094 handle_command_reply: tid=0 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 client.17094 shutdown 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 -- 172.21.15.18:0/2128585161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a69830 msgr2=0x563a33a75ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a69830 0x563a33a75ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f11a400fdc0 tx=0x7f11a40128b0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 -- 172.21.15.18:0/2128585161 shutdown_connections 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563a33ae8590 0x563a33aea9b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a33a72410 0x563a33ae6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f1197400640 0 client.17094 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a33a69830 0x563a33a75ad0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 --2- 172.21.15.18:0/2128585161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a33a68f10 0x563a33a75590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 -- 172.21.15.18:0/2128585161 >> 172.21.15.18:0/2128585161 conn(0x563a33a63640 msgr2=0x563a33a62700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 -- 172.21.15.18:0/2128585161 shutdown_connections 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.934+0000 7f11bf575740 1 -- 172.21.15.18:0/2128585161 wait complete. 2024-04-16T17:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bf575740 20 client.17094 trim_cache size 0 max 16384 2024-04-16T17:11:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 -- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f119c0a3da0 msgr2=0x7f119c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f119c0a3da0 0x7f119c0a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f11a80ce240 tx=0x7f11a80045b0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 -- 172.21.15.18:0/21426071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b4100dd0 msgr2=0x7f11b419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b4100dd0 0x7f11b419eda0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11a800ea60 tx=0x7f11a800ef20 comp rx=0 tx=0).stop 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 -- 172.21.15.18:0/21426071 shutdown_connections 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f119c0a3da0 0x7f119c0a6260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b41061d0 0x7f11b41a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11b41016f0 0x7f11b419f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.935+0000 7f11bb200640 1 --2- 172.21.15.18:0/21426071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b4100dd0 0x7f11b419eda0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.936+0000 7f11bb200640 1 -- 172.21.15.18:0/21426071 >> 172.21.15.18:0/21426071 conn(0x7f11b4077ab0 msgr2=0x7f11b4103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.936+0000 7f11bb200640 1 -- 172.21.15.18:0/21426071 shutdown_connections 2024-04-16T17:11:02.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:02.936+0000 7f11bb200640 1 -- 172.21.15.18:0/21426071 wait complete. 2024-04-16T17:11:03.573 INFO:tasks.workunit.client.0.smithi018.stdout: 1 236616 3.94 MB/sec execute 196 sec latency 56.747 ms 2024-04-16T17:11:03.955 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:11:03.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:11:03.956 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:11:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:03 smithi043 ceph-mon[36901]: pgmap v193: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-04-16T17:11:04.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.144+0000 7f0ed0a00640 1 -- 172.21.15.18:0/73538520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc107140 msgr2=0x7f0ecc10ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.144+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/73538520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc107140 0x7f0ecc10ed10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009ca0 tx=0x7f0ec402c140 comp rx=0 tx=0).stop 2024-04-16T17:11:04.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.144+0000 7f0ed0a00640 1 -- 172.21.15.18:0/73538520 shutdown_connections 2024-04-16T17:11:04.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.144+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/73538520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc107140 0x7f0ecc10ed10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.145+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/73538520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ecc102a20 0x7f0ecc106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.145+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/73538520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ecc102070 0x7f0ecc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.145+0000 7f0ed0a00640 1 -- 172.21.15.18:0/73538520 >> 172.21.15.18:0/73538520 conn(0x7f0ecc0fd950 msgr2=0x7f0ecc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.145+0000 7f0ed0a00640 1 -- 172.21.15.18:0/73538520 shutdown_connections 2024-04-16T17:11:04.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.145+0000 7f0ed0a00640 1 -- 172.21.15.18:0/73538520 wait complete. 2024-04-16T17:11:04.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.146+0000 7f0ed0a00640 1 Processor -- start 2024-04-16T17:11:04.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.146+0000 7f0ed0a00640 1 -- start start 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ed0a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc102070 0x7f0ecc10e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ed0a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ecc107140 0x7f0ecc10e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ed0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ecc1a3800 0x7f0ecc1a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ed0a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1122f0 con 0x7f0ecc102070 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ed0a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc112170 con 0x7f0ecc107140 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ed0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc112470 con 0x7f0ecc1a3800 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ecb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc102070 0x7f0ecc10e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ecb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc102070 0x7f0ecc10e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55240/0 (socket says 172.21.15.18:55240) 2024-04-16T17:11:04.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.147+0000 7f0ecb400640 1 -- 172.21.15.18:0/2787214728 learned_addr learned my addr 172.21.15.18:0/2787214728 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ecb400640 1 -- 172.21.15.18:0/2787214728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ecc1a3800 msgr2=0x7f0ecc1a3cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:11:04.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ecb400640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ecc1a3800 0x7f0ecc1a3cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ecb400640 1 -- 172.21.15.18:0/2787214728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ecc107140 msgr2=0x7f0ecc10e690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:11:04.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ecb400640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ecc107140 0x7f0ecc10e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ecb400640 1 -- 172.21.15.18:0/2787214728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc1a42f0 con 0x7f0ecc102070 2024-04-16T17:11:04.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ecb400640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc102070 0x7f0ecc10e150 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b450 tx=0x7f0ebc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ec3e00640 1 -- 172.21.15.18:0/2787214728 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc015020 con 0x7f0ecc102070 2024-04-16T17:11:04.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ec3e00640 1 -- 172.21.15.18:0/2787214728 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc011d70 con 0x7f0ecc102070 2024-04-16T17:11:04.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ec3e00640 1 -- 172.21.15.18:0/2787214728 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00f340 con 0x7f0ecc102070 2024-04-16T17:11:04.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.148+0000 7f0ed0a00640 1 -- 172.21.15.18:0/2787214728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc1a45e0 con 0x7f0ecc102070 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.149+0000 7f0ed0a00640 1 -- 172.21.15.18:0/2787214728 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc1abcb0 con 0x7f0ecc102070 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.152+0000 7f0ed4dc2740 1 Processor -- start 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.152+0000 7f0ed4dc2740 1 -- start start 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d17e90 0x55ecd2d18270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 0x55ecd2d20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 0x55ecd2d23780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2c7ccb0 con 0x55ecd2d17e90 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2c23370 con 0x55ecd2d187b0 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2c78c70 con 0x55ecd2d21390 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ecbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d17e90 0x55ecd2d18270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ecbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d17e90 0x55ecd2d18270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55242/0 (socket says 172.21.15.18:55242) 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ecbe00640 1 -- 172.21.15.18:0/2619657680 learned_addr learned my addr 172.21.15.18:0/2619657680 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ecaa00640 1 --2- 172.21.15.18:0/2619657680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 0x55ecd2d23780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.153+0000 7f0ec3e00640 1 -- 172.21.15.18:0/2787214728 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ebc018020 con 0x7f0ecc102070 2024-04-16T17:11:04.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.154+0000 7f0ecbe00640 1 -- 172.21.15.18:0/2619657680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 msgr2=0x55ecd2d23780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.154+0000 7f0ecbe00640 1 --2- 172.21.15.18:0/2619657680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 0x55ecd2d23780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.154+0000 7f0ecbe00640 1 -- 172.21.15.18:0/2619657680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 msgr2=0x55ecd2d20e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:04.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.154+0000 7f0ecbe00640 1 --2- 172.21.15.18:0/2619657680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 0x55ecd2d20e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.154+0000 7f0ecbe00640 1 -- 172.21.15.18:0/2619657680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecd2c42910 con 0x55ecd2d17e90 2024-04-16T17:11:04.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.154+0000 7f0ecbe00640 1 --2- 172.21.15.18:0/2619657680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d17e90 0x55ecd2d18270 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4037040 tx=0x7f0ec4032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f66aa0fdcb5238d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.154+0000 7f0eb7e00640 1 -- 172.21.15.18:0/2619657680 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4041020 con 0x55ecd2d17e90 2024-04-16T17:11:04.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.154+0000 7f0eb7e00640 1 -- 172.21.15.18:0/2619657680 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ec4032cd0 con 0x55ecd2d17e90 2024-04-16T17:11:04.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.155+0000 7f0eb7e00640 1 -- 172.21.15.18:0/2619657680 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4039040 con 0x55ecd2d17e90 2024-04-16T17:11:04.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.155+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2619657680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d17e90 msgr2=0x55ecd2d18270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.155+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/2619657680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d17e90 0x55ecd2d18270 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4037040 tx=0x7f0ec4032430 comp rx=0 tx=0).stop 2024-04-16T17:11:04.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.155+0000 7f0ec3e00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0eb00a3d20 0x7f0eb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.155+0000 7f0ec3e00640 1 -- 172.21.15.18:0/2787214728 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f0ebc0d0670 con 0x7f0ecc102070 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.156+0000 7f0ecbe00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0eb00a3d20 0x7f0eb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.157+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2619657680 shutdown_connections 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.157+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/2619657680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 0x55ecd2d23780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.157+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/2619657680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 0x55ecd2d20e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.157+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/2619657680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d17e90 0x55ecd2d18270 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.157+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2619657680 >> 172.21.15.18:0/2619657680 conn(0x55ecd2d125c0 msgr2=0x55ecd2d129a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.157+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2619657680 shutdown_connections 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.157+0000 7f0ecbe00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0eb00a3d20 0x7f0eb00a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ec402c750 tx=0x7f0ec402e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.157+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2619657680 wait complete. 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.158+0000 7f0ed4dc2740 1 -- start start 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.158+0000 7f0ed4dc2740 1 Processor -- start 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.158+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d17e90 0x55ecd2d1e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.158+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 0x55ecd2d1e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.158+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d21390 0x55ecd2d1ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.158+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2d24120 con 0x55ecd2d21390 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2c36730 con 0x55ecd2d187b0 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2d242a0 con 0x55ecd2d17e90 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ed4dc2740 10 client.0 _ll_register_callbacks cb 0x55ecd2c051e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d21390 0x55ecd2d1ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 0x55ecd2d1e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d21390 0x55ecd2d1ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55250/0 (socket says 172.21.15.18:55250) 2024-04-16T17:11:04.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecbe00640 1 -- 172.21.15.18:0/598574171 learned_addr learned my addr 172.21.15.18:0/598574171 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecaa00640 1 -- 172.21.15.18:0/598574171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d17e90 msgr2=0x55ecd2d1e3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:04.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecaa00640 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d17e90 0x55ecd2d1e3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecaa00640 1 -- 172.21.15.18:0/598574171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d21390 msgr2=0x55ecd2d1ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecaa00640 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d21390 0x55ecd2d1ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.159+0000 7f0ecaa00640 1 -- 172.21.15.18:0/598574171 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecd2a577e0 con 0x55ecd2d187b0 2024-04-16T17:11:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.161+0000 7f0ecaa00640 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 0x55ecd2d1e8e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0fe230 tx=0x7f0eb800e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd08b07b16d5054b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.161+0000 7f0eb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.161+0000 7f0eb4c00640 1 -- 172.21.15.18:0/598574171 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8017070 con 0x55ecd2d187b0 2024-04-16T17:11:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.161+0000 7f0eb4c00640 1 -- 172.21.15.18:0/598574171 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f0eb8013470 con 0x55ecd2d187b0 2024-04-16T17:11:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.161+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/598574171 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ec4032cd0 con 0x55ecd2d187b0 2024-04-16T17:11:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.161+0000 7f0eb4c00640 1 -- 172.21.15.18:0/598574171 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0eb8023050 con 0x55ecd2d187b0 2024-04-16T17:11:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.161+0000 7f0ed4dc2740 10 client.27347 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:04.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.161+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/598574171 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ecd2ab7f70 con 0x55ecd2d187b0 2024-04-16T17:11:04.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.162+0000 7f0eb4c00640 1 -- 172.21.15.18:0/598574171 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ec4032cd0 con 0x55ecd2d187b0 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.162+0000 7f0eb4c00640 1 -- 172.21.15.18:0/598574171 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0eb8012870 con 0x55ecd2d187b0 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.162+0000 7f0ed4dc2740 10 client.27347 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.162+0000 7f0ed4dc2740 10 client.27347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.162+0000 7f0ed4dc2740 20 client.27347 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.162+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ecd2d31710 0x55ecd2d33b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.162+0000 7f0ed4dc2740 4 client.27347 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.162+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/598574171 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecd2c8a6d0 con 0x55ecd2d31710 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.163+0000 7f0ecb400640 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ecd2d31710 0x55ecd2d33b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.163+0000 7f0ecb400640 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ecd2d31710 0x55ecd2d33b90 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6d18c8013b55d9 server_cookie=47bc1afefe319ae4 in_seq=0 out_seq=0 2024-04-16T17:11:04.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.163+0000 7f0eb4c00640 10 client.27347 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0eb4c00640 1 -- 172.21.15.18:0/598574171 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecd2c8a6d0 con 0x55ecd2d31710 2024-04-16T17:11:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0eb4c00640 10 client.27347 handle_command_reply: tid=0 2024-04-16T17:11:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 client.27347 shutdown 2024-04-16T17:11:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/598574171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 msgr2=0x55ecd2d1e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 0x55ecd2d1e8e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0fe230 tx=0x7f0eb800e660 comp rx=0 tx=0).stop 2024-04-16T17:11:04.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/598574171 shutdown_connections 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ecd2d31710 0x55ecd2d33b90 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d21390 0x55ecd2d1ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d187b0 0x55ecd2d1e8e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/598574171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d17e90 0x55ecd2d1e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/598574171 >> 172.21.15.18:0/598574171 conn(0x55ecd2d125c0 msgr2=0x55ecd2d14310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0eb4c00640 0 client.27347 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.164+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/598574171 shutdown_connections 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.165+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/598574171 wait complete. 2024-04-16T17:11:04.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.165+0000 7f0ed4dc2740 20 client.27347 trim_cache size 0 max 16384 2024-04-16T17:11:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:03 smithi022 ceph-mon[36023]: pgmap v193: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-04-16T17:11:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.191+0000 7f0ed4dc2740 1 Processor -- start 2024-04-16T17:11:04.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.191+0000 7f0ed4dc2740 1 -- start start 2024-04-16T17:11:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.191+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d125c0 0x55ecd2d2d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.191+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d1ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.191+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d187b0 0x55ecd2d39460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.191+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b82a0 con 0x55ecd2d125c0 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.191+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b5b70 con 0x55ecd2d17e90 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.191+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b39c0 con 0x55ecd2d187b0 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d125c0 0x55ecd2d2d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d125c0 0x55ecd2d2d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55256/0 (socket says 172.21.15.18:55256) 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 -- 172.21.15.18:0/2171552759 learned_addr learned my addr 172.21.15.18:0/2171552759 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 -- 172.21.15.18:0/2171552759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d187b0 msgr2=0x55ecd2d39460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:04.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 --2- 172.21.15.18:0/2171552759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d187b0 0x55ecd2d39460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 -- 172.21.15.18:0/2171552759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 msgr2=0x55ecd2d1ea70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:04.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 --2- 172.21.15.18:0/2171552759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d1ea70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 -- 172.21.15.18:0/2171552759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecd2d185e0 con 0x55ecd2d125c0 2024-04-16T17:11:04.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.192+0000 7f0ecb400640 1 --2- 172.21.15.18:0/2171552759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d125c0 0x55ecd2d2d010 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc011040 tx=0x7f0ebc010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a38b3d4186a157a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.193+0000 7f0eb7400640 1 -- 172.21.15.18:0/2171552759 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0d0040 con 0x55ecd2d125c0 2024-04-16T17:11:04.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.193+0000 7f0eb7400640 1 -- 172.21.15.18:0/2171552759 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc0a06c0 con 0x55ecd2d125c0 2024-04-16T17:11:04.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.193+0000 7f0eb7400640 1 -- 172.21.15.18:0/2171552759 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0a4af0 con 0x55ecd2d125c0 2024-04-16T17:11:04.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2171552759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d125c0 msgr2=0x55ecd2d2d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/2171552759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d125c0 0x55ecd2d2d010 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc011040 tx=0x7f0ebc010fd0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2171552759 shutdown_connections 2024-04-16T17:11:04.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/2171552759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d187b0 0x55ecd2d39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/2171552759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d1ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/2171552759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d125c0 0x55ecd2d2d010 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2171552759 >> 172.21.15.18:0/2171552759 conn(0x55ecd2d1d020 msgr2=0x55ecd2d1e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2171552759 shutdown_connections 2024-04-16T17:11:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/2171552759 wait complete. 2024-04-16T17:11:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.194+0000 7f0ed4dc2740 1 -- start start 2024-04-16T17:11:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.195+0000 7f0ed4dc2740 1 Processor -- start 2024-04-16T17:11:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.195+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d29a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.195+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d187b0 0x55ecd2d264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.195+0000 7f0ed4dc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 0x55ecd2d26c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.195+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b4030 con 0x55ecd2d187b0 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.195+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b54a0 con 0x55ecd2d17e90 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.195+0000 7f0ed4dc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b4700 con 0x55ecd2d21390 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.195+0000 7f0ed4dc2740 10 client.0 _ll_register_callbacks cb 0x7f0eb8009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d29a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d29a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36800/0 (socket says 172.21.15.18:36800) 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 -- 172.21.15.18:0/1323918724 learned_addr learned my addr 172.21.15.18:0/1323918724 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecaa00640 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d187b0 0x55ecd2d264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 -- 172.21.15.18:0/1323918724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 msgr2=0x55ecd2d26c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 0x55ecd2d26c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 -- 172.21.15.18:0/1323918724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d187b0 msgr2=0x55ecd2d264f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d187b0 0x55ecd2d264f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 -- 172.21.15.18:0/1323918724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecd294f170 con 0x55ecd2d17e90 2024-04-16T17:11:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0ecb400640 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d29a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc09c010 tx=0x7f0ebc09c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2baf31f34a9e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.196+0000 7f0eb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0eb4c00640 1 -- 172.21.15.18:0/1323918724 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0ba060 con 0x55ecd2d17e90 2024-04-16T17:11:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0eb4c00640 1 -- 172.21.15.18:0/1323918724 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f0ebc0cdc30 con 0x55ecd2d17e90 2024-04-16T17:11:04.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/1323918724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ebc0a06c0 con 0x55ecd2d17e90 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0eb4c00640 1 -- 172.21.15.18:0/1323918724 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ebc0d0040 con 0x55ecd2d17e90 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 10 client.27350 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/1323918724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ecd2d98cf0 con 0x55ecd2d17e90 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0eb4c00640 1 -- 172.21.15.18:0/1323918724 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc0a06c0 con 0x55ecd2d17e90 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0eb4c00640 1 -- 172.21.15.18:0/1323918724 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0ebc063070 con 0x55ecd2d17e90 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 10 client.27350 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 10 client.27350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 20 client.27350 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ecd2d994a0 0x55ecd2d9b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 4 client.27350 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.197+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/1323918724 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ecd2c84540 con 0x55ecd2d994a0 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.198+0000 7f0ecaa00640 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ecd2d994a0 0x55ecd2d9b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.198+0000 7f0ecaa00640 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ecd2d994a0 0x55ecd2d9b890 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143530e48f6565ee server_cookie=548293e43f6e0ad6 in_seq=0 out_seq=0 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.198+0000 7f0eb4c00640 10 client.27350 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.199+0000 7f0eb4c00640 1 -- 172.21.15.18:0/1323918724 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ecd2c84540 con 0x55ecd2d994a0 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.199+0000 7f0eb4c00640 10 client.27350 handle_command_reply: tid=0 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.199+0000 7f0ed4dc2740 1 client.27350 shutdown 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/1323918724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 msgr2=0x55ecd2d29a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d29a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc09c010 tx=0x7f0ebc09c490 comp rx=0 tx=0).stop 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/1323918724 shutdown_connections 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ecd2d994a0 0x55ecd2d9b890 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecd2d21390 0x55ecd2d26c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ecd2d187b0 0x55ecd2d264f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0eb4c00640 0 client.27350 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:04.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 --2- 172.21.15.18:0/1323918724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ecd2d17e90 0x55ecd2d29a50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/1323918724 >> 172.21.15.18:0/1323918724 conn(0x55ecd2d1d020 msgr2=0x55ecd2d1ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/1323918724 shutdown_connections 2024-04-16T17:11:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 1 -- 172.21.15.18:0/1323918724 wait complete. 2024-04-16T17:11:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.200+0000 7f0ed4dc2740 20 client.27350 trim_cache size 0 max 16384 2024-04-16T17:11:04.203 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:11:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:11:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "7f06d8ef-a24b-451e-ac93-f8c464e41f6b", 2024-04-16T17:11:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:11:04.204 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:11:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 -- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0eb00a3d20 msgr2=0x7f0eb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0eb00a3d20 0x7f0eb00a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ec402c750 tx=0x7f0ec402e040 comp rx=0 tx=0).stop 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 -- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc102070 msgr2=0x7f0ecc10e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc102070 0x7f0ecc10e150 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b450 tx=0x7f0ebc00b910 comp rx=0 tx=0).stop 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 -- 172.21.15.18:0/2787214728 shutdown_connections 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0eb00a3d20 0x7f0eb00a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ecc1a3800 0x7f0ecc1a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ecc107140 0x7f0ecc10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 --2- 172.21.15.18:0/2787214728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ecc102070 0x7f0ecc10e150 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 -- 172.21.15.18:0/2787214728 >> 172.21.15.18:0/2787214728 conn(0x7f0ecc0fd950 msgr2=0x7f0ecc0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 -- 172.21.15.18:0/2787214728 shutdown_connections 2024-04-16T17:11:04.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.201+0000 7f0ed0a00640 1 -- 172.21.15.18:0/2787214728 wait complete. 2024-04-16T17:11:04.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:11:04.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 -- 172.21.15.18:0/1189855001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 msgr2=0x7f75c8074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 --2- 172.21.15.18:0/1189855001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 0x7f75c8074780 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009900 tx=0x7f75bc02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 -- 172.21.15.18:0/1189855001 shutdown_connections 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 --2- 172.21.15.18:0/1189855001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c8112e10 0x7f75c8115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 --2- 172.21.15.18:0/1189855001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c8074cc0 0x7f75c81128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 --2- 172.21.15.18:0/1189855001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 0x7f75c8074780 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 -- 172.21.15.18:0/1189855001 >> 172.21.15.18:0/1189855001 conn(0x7f75c80fd950 msgr2=0x7f75c80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 -- 172.21.15.18:0/1189855001 shutdown_connections 2024-04-16T17:11:04.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.406+0000 7f75ce800640 1 -- 172.21.15.18:0/1189855001 wait complete. 2024-04-16T17:11:04.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.407+0000 7f75ce800640 1 Processor -- start 2024-04-16T17:11:04.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.407+0000 7f75ce800640 1 -- start start 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75ce800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 0x7f75c819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75ce800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c8074cc0 0x7f75c819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75ce800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c8112e10 0x7f75c81a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75ce800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8118800 con 0x7f75c8112e10 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75ce800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8118680 con 0x7f75c8074cc0 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75ce800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8118980 con 0x7f75c80743a0 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 0x7f75c819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 0x7f75c819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53622/0 (socket says 172.21.15.18:53622) 2024-04-16T17:11:04.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75cd400640 1 -- 172.21.15.18:0/1900020390 learned_addr learned my addr 172.21.15.18:0/1900020390 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75cde00640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c8112e10 0x7f75c81a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.408+0000 7f75cca00640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c8074cc0 0x7f75c819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75cd400640 1 -- 172.21.15.18:0/1900020390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c8074cc0 msgr2=0x7f75c819f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75cd400640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c8074cc0 0x7f75c819f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75cd400640 1 -- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c8112e10 msgr2=0x7f75c81a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75cd400640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c8112e10 0x7f75c81a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75cd400640 1 -- 172.21.15.18:0/1900020390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c81a3d90 con 0x7f75c80743a0 2024-04-16T17:11:04.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75cca00640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c8074cc0 0x7f75c819f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:04.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75cd400640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 0x7f75c819eda0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f75bc036040 tx=0x7f75bc02cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75c6000640 1 -- 172.21.15.18:0/1900020390 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc043020 con 0x7f75c80743a0 2024-04-16T17:11:04.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75c6000640 1 -- 172.21.15.18:0/1900020390 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75bc0054e0 con 0x7f75c80743a0 2024-04-16T17:11:04.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.409+0000 7f75c6000640 1 -- 172.21.15.18:0/1900020390 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc003b60 con 0x7f75c80743a0 2024-04-16T17:11:04.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.410+0000 7f75ce800640 1 -- 172.21.15.18:0/1900020390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c81a4020 con 0x7f75c80743a0 2024-04-16T17:11:04.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.410+0000 7f75ce800640 1 -- 172.21.15.18:0/1900020390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c81ab910 con 0x7f75c80743a0 2024-04-16T17:11:04.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.414+0000 7f75c6000640 1 -- 172.21.15.18:0/1900020390 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f75bc046020 con 0x7f75c80743a0 2024-04-16T17:11:04.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.414+0000 7f75d2b77740 1 Processor -- start 2024-04-16T17:11:04.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75d2b77740 1 -- start start 2024-04-16T17:11:04.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291c390 0x56215291c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291ccb0 0x562152925350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152927c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75d2b77740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562152882120 con 0x56215291ccb0 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75d2b77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562152828340 con 0x56215291c390 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75d2b77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56215287e0e0 con 0x562152925890 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75cde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291c390 0x56215291c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75cde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291c390 0x56215291c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36810/0 (socket says 172.21.15.18:36810) 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75cde00640 1 -- 172.21.15.18:0/1563560506 learned_addr learned my addr 172.21.15.18:0/1563560506 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.415+0000 7f75cd400640 1 --2- 172.21.15.18:0/1563560506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291ccb0 0x562152925350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.416+0000 7f75cde00640 1 -- 172.21.15.18:0/1563560506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 msgr2=0x562152927c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.416+0000 7f75cde00640 1 --2- 172.21.15.18:0/1563560506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152927c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.416+0000 7f75cde00640 1 -- 172.21.15.18:0/1563560506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291ccb0 msgr2=0x562152925350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.416+0000 7f75c6000640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f75b00a3da0 0x7f75b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.416+0000 7f75cde00640 1 --2- 172.21.15.18:0/1563560506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291ccb0 0x562152925350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.416+0000 7f75cde00640 1 -- 172.21.15.18:0/1563560506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215265e7d0 con 0x56215291c390 2024-04-16T17:11:04.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.416+0000 7f75c6000640 1 -- 172.21.15.18:0/1900020390 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f75bc0f2a40 con 0x7f75c80743a0 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.416+0000 7f75cde00640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f75b00a3da0 0x7f75b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.417+0000 7f75cde00640 1 --2- 172.21.15.18:0/1563560506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291c390 0x56215291c770 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009f40 tx=0x7f75c00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf20060ea2723f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.417+0000 7f75ba600640 1 -- 172.21.15.18:0/1563560506 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0004250 con 0x56215291c390 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.417+0000 7f75ba600640 1 -- 172.21.15.18:0/1563560506 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c000fe80 con 0x56215291c390 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.417+0000 7f75ba600640 1 -- 172.21.15.18:0/1563560506 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c001bc00 con 0x56215291c390 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 -- 172.21.15.18:0/1563560506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291c390 msgr2=0x56215291c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 --2- 172.21.15.18:0/1563560506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291c390 0x56215291c770 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009f40 tx=0x7f75c00118d0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75cde00640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f75b00a3da0 0x7f75b00a6260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f75c0020000 tx=0x7f75c0021000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 -- 172.21.15.18:0/1563560506 shutdown_connections 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 --2- 172.21.15.18:0/1563560506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152927c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 --2- 172.21.15.18:0/1563560506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291ccb0 0x562152925350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 --2- 172.21.15.18:0/1563560506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291c390 0x56215291c770 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 -- 172.21.15.18:0/1563560506 >> 172.21.15.18:0/1563560506 conn(0x562152916ac0 msgr2=0x562152916ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 -- 172.21.15.18:0/1563560506 shutdown_connections 2024-04-16T17:11:04.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.418+0000 7f75d2b77740 1 -- 172.21.15.18:0/1563560506 wait complete. 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.419+0000 7f75d2b77740 1 -- start start 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.419+0000 7f75d2b77740 1 Processor -- start 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56215291c390 0x5621529228f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152922e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562152925890 0x562152923370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75d2b77740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562152928680 con 0x562152925890 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75d2b77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56215283b710 con 0x56215291ccb0 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75d2b77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562152928800 con 0x56215291c390 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75d2b77740 10 client.0 _ll_register_callbacks cb 0x562152836af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56215291c390 0x5621529228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75cde00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562152925890 0x562152923370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56215291c390 0x5621529228f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53628/0 (socket says 172.21.15.18:53628) 2024-04-16T17:11:04.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75cd400640 1 -- 172.21.15.18:0/2425652738 learned_addr learned my addr 172.21.15.18:0/2425652738 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.420+0000 7f75cca00640 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152922e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75cca00640 1 -- 172.21.15.18:0/2425652738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56215291c390 msgr2=0x5621529228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75cca00640 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56215291c390 0x5621529228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75cca00640 1 -- 172.21.15.18:0/2425652738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562152925890 msgr2=0x562152923370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75cca00640 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562152925890 0x562152923370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75cca00640 1 -- 172.21.15.18:0/2425652738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621525943f0 con 0x56215291ccb0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75cca00640 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152922e30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f75c8105c20 tx=0x7f75b4011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06ba5c7f4f8447 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75aea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75aea00640 1 -- 172.21.15.18:0/2425652738 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b400fd10 con 0x56215291ccb0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75d2b77740 1 -- 172.21.15.18:0/2425652738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56215288fb40 con 0x56215291ccb0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75aea00640 1 -- 172.21.15.18:0/2425652738 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f75b4005020 con 0x56215291ccb0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75aea00640 1 -- 172.21.15.18:0/2425652738 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75b4024050 con 0x56215291ccb0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75d2b77740 10 client.27362 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.421+0000 7f75d2b77740 1 -- 172.21.15.18:0/2425652738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621525540f0 con 0x56215291ccb0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75aea00640 1 -- 172.21.15.18:0/2425652738 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56215288fb40 con 0x56215291ccb0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75aea00640 1 -- 172.21.15.18:0/2425652738 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f75b4019430 con 0x56215291ccb0 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75d2b77740 10 client.27362 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75d2b77740 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75d2b77740 20 client.27362 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:04.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75d2b77740 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562152935c70 0x5621529380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75d2b77740 4 client.27362 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:04.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75d2b77740 1 -- 172.21.15.18:0/2425652738 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621528899b0 con 0x562152935c70 2024-04-16T17:11:04.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.422+0000 7f75cd400640 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562152935c70 0x5621529380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.423+0000 7f75aea00640 10 client.27362 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:04.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.423+0000 7f75cd400640 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562152935c70 0x5621529380f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b050c0c04d93ce server_cookie=f82d923d8f058efb in_seq=0 out_seq=0 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75aea00640 1 -- 172.21.15.18:0/2425652738 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621528899b0 con 0x562152935c70 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75aea00640 10 client.27362 handle_command_reply: tid=0 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 client.27362 shutdown 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 -- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 msgr2=0x562152922e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152922e30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f75c8105c20 tx=0x7f75b4011d10 comp rx=0 tx=0).stop 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 -- 172.21.15.18:0/2425652738 shutdown_connections 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562152935c70 0x5621529380f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562152925890 0x562152923370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152922e30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 --2- 172.21.15.18:0/2425652738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56215291c390 0x5621529228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 -- 172.21.15.18:0/2425652738 >> 172.21.15.18:0/2425652738 conn(0x562152916ac0 msgr2=0x562152918810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75aea00640 0 client.27362 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:04.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.424+0000 7f75d2b77740 1 -- 172.21.15.18:0/2425652738 shutdown_connections 2024-04-16T17:11:04.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.425+0000 7f75d2b77740 1 -- 172.21.15.18:0/2425652738 wait complete. 2024-04-16T17:11:04.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.425+0000 7f75d2b77740 20 client.27362 trim_cache size 0 max 16384 2024-04-16T17:11:04.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75d2b77740 1 Processor -- start 2024-04-16T17:11:04.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75d2b77740 1 -- start start 2024-04-16T17:11:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x562152925020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152921830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152931570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75d2b77740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b82f0 con 0x56215291c390 2024-04-16T17:11:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75d2b77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b5bc0 con 0x56215291ccb0 2024-04-16T17:11:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75d2b77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b3a10 con 0x562152925890 2024-04-16T17:11:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cd400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x562152925020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152921830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cd400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x562152925020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55306/0 (socket says 172.21.15.18:55306) 2024-04-16T17:11:04.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cd400640 1 -- 172.21.15.18:0/158272336 learned_addr learned my addr 172.21.15.18:0/158272336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cca00640 1 -- 172.21.15.18:0/158272336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 msgr2=0x562152931570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:04.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cca00640 1 --2- 172.21.15.18:0/158272336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152931570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cca00640 1 -- 172.21.15.18:0/158272336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 msgr2=0x562152925020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cca00640 1 --2- 172.21.15.18:0/158272336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x562152925020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cca00640 1 -- 172.21.15.18:0/158272336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215291cae0 con 0x56215291ccb0 2024-04-16T17:11:04.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75cca00640 1 --2- 172.21.15.18:0/158272336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152921830 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75b40198d0 tx=0x7f75b4009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3503b2d239736b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75b9c00640 1 -- 172.21.15.18:0/158272336 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4019430 con 0x56215291ccb0 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75b9c00640 1 -- 172.21.15.18:0/158272336 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b40195d0 con 0x56215291ccb0 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.440+0000 7f75b9c00640 1 -- 172.21.15.18:0/158272336 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b40035f0 con 0x56215291ccb0 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 -- 172.21.15.18:0/158272336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 msgr2=0x562152921830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 --2- 172.21.15.18:0/158272336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152921830 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75b40198d0 tx=0x7f75b4009230 comp rx=0 tx=0).stop 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 -- 172.21.15.18:0/158272336 shutdown_connections 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 --2- 172.21.15.18:0/158272336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152931570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 --2- 172.21.15.18:0/158272336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152921830 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 --2- 172.21.15.18:0/158272336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x562152925020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 -- 172.21.15.18:0/158272336 >> 172.21.15.18:0/158272336 conn(0x562152916ac0 msgr2=0x562152918780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 -- 172.21.15.18:0/158272336 shutdown_connections 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 -- 172.21.15.18:0/158272336 wait complete. 2024-04-16T17:11:04.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.446+0000 7f75d2b77740 1 -- start start 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.447+0000 7f75d2b77740 1 Processor -- start 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x5621529294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152929a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75d2b77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152929f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75d2b77740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b4080 con 0x56215291c390 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75d2b77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b54f0 con 0x56215291ccb0 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75d2b77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b4750 con 0x562152925890 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75d2b77740 10 client.0 _ll_register_callbacks cb 0x7f75b4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cd400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x5621529294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152929a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152929a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36848/0 (socket says 172.21.15.18:36848) 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cca00640 1 -- 172.21.15.18:0/3516273968 learned_addr learned my addr 172.21.15.18:0/3516273968 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cd400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x5621529294c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55312/0 (socket says 172.21.15.18:55312) 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cde00640 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152929f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cca00640 1 -- 172.21.15.18:0/3516273968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 msgr2=0x562152929f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cca00640 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152929f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cca00640 1 -- 172.21.15.18:0/3516273968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 msgr2=0x5621529294c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cca00640 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x5621529294c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.448+0000 7f75cca00640 1 -- 172.21.15.18:0/3516273968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215299b260 con 0x56215291ccb0 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75cde00640 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152929f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75cca00640 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152929a00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75b4019a90 tx=0x7f75b4011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23ec213c3776bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75aea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75aea00640 1 -- 172.21.15.18:0/3516273968 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4019c00 con 0x56215291ccb0 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75d2b77740 1 -- 172.21.15.18:0/3516273968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75b40195d0 con 0x56215291ccb0 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75aea00640 1 -- 172.21.15.18:0/3516273968 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f75b4011020 con 0x56215291ccb0 2024-04-16T17:11:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75aea00640 1 -- 172.21.15.18:0/3516273968 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75b4011220 con 0x56215291ccb0 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75d2b77740 10 client.27368 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.449+0000 7f75d2b77740 1 -- 172.21.15.18:0/3516273968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56215299c550 con 0x56215291ccb0 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75aea00640 1 -- 172.21.15.18:0/3516273968 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b40195d0 con 0x56215291ccb0 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75aea00640 1 -- 172.21.15.18:0/3516273968 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f75b4010430 con 0x56215291ccb0 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75d2b77740 10 client.27368 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75d2b77740 10 client.27368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75d2b77740 20 client.27368 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75d2b77740 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56215299cc70 0x56215299f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75d2b77740 4 client.27368 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75d2b77740 1 -- 172.21.15.18:0/3516273968 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562152805420 con 0x56215299cc70 2024-04-16T17:11:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.450+0000 7f75cd400640 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56215299cc70 0x56215299f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:04.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75aea00640 10 client.27368 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75cd400640 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56215299cc70 0x56215299f090 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7699090597dc2dfc server_cookie=26514cc6adca6f7b in_seq=0 out_seq=0 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75aea00640 1 -- 172.21.15.18:0/3516273968 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562152805420 con 0x56215299cc70 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75aea00640 10 client.27368 handle_command_reply: tid=0 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 client.27368 shutdown 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 -- 172.21.15.18:0/3516273968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 msgr2=0x562152929a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152929a00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75b4019a90 tx=0x7f75b4011d10 comp rx=0 tx=0).stop 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 -- 172.21.15.18:0/3516273968 shutdown_connections 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56215299cc70 0x56215299f090 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562152925890 0x562152929f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56215291ccb0 0x562152929a00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 --2- 172.21.15.18:0/3516273968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56215291c390 0x5621529294c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75aea00640 0 client.27368 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:04.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.451+0000 7f75d2b77740 1 -- 172.21.15.18:0/3516273968 >> 172.21.15.18:0/3516273968 conn(0x562152916ac0 msgr2=0x5621529228b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.452+0000 7f75d2b77740 1 -- 172.21.15.18:0/3516273968 shutdown_connections 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.452+0000 7f75d2b77740 1 -- 172.21.15.18:0/3516273968 wait complete. 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.452+0000 7f75d2b77740 20 client.27368 trim_cache size 0 max 16384 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "7f06d8ef-a24b-451e-ac93-f8c464e41f6b": { 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "7f06d8ef-a24b-451e-ac93-f8c464e41f6b", 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:11:04.456 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:11:04.457 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:11:04.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.452+0000 7f75ce800640 1 -- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f75b00a3da0 msgr2=0x7f75b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.452+0000 7f75ce800640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f75b00a3da0 0x7f75b00a6260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f75c0020000 tx=0x7f75c0021000 comp rx=0 tx=0).stop 2024-04-16T17:11:04.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.452+0000 7f75ce800640 1 -- 172.21.15.18:0/1900020390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 msgr2=0x7f75c819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:04.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.452+0000 7f75ce800640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 0x7f75c819eda0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f75bc036040 tx=0x7f75bc02cbc0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.453+0000 7f75ce800640 1 -- 172.21.15.18:0/1900020390 shutdown_connections 2024-04-16T17:11:04.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.453+0000 7f75ce800640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f75b00a3da0 0x7f75b00a6260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.453+0000 7f75ce800640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c8112e10 0x7f75c81a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.453+0000 7f75ce800640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c8074cc0 0x7f75c819f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.453+0000 7f75ce800640 1 --2- 172.21.15.18:0/1900020390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c80743a0 0x7f75c819eda0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:04.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.453+0000 7f75ce800640 1 -- 172.21.15.18:0/1900020390 >> 172.21.15.18:0/1900020390 conn(0x7f75c80fd950 msgr2=0x7f75c80fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:04.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.453+0000 7f75ce800640 1 -- 172.21.15.18:0/1900020390 shutdown_connections 2024-04-16T17:11:04.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:04.453+0000 7f75ce800640 1 -- 172.21.15.18:0/1900020390 wait complete. 2024-04-16T17:11:04.474 INFO:tasks.cephfs.filesystem:scrub status for tag:7f06d8ef-a24b-451e-ac93-f8c464e41f6b - {'path': '/', 'tag': '7f06d8ef-a24b-451e-ac93-f8c464e41f6b', 'options': 'recursive,force'} 2024-04-16T17:11:04.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 237026 3.94 MB/sec execute 197 sec latency 75.374 ms 2024-04-16T17:11:05.574 INFO:tasks.workunit.client.0.smithi018.stdout: 1 237782 3.95 MB/sec execute 198 sec latency 62.138 ms 2024-04-16T17:11:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:05 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:11:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:05 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:05 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:11:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:05 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:05 smithi043 ceph-mon[36901]: pgmap v194: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-04-16T17:11:06.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:05 smithi043 ceph-mon[36901]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T17:11:06.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:05 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:11:06.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:05 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:06.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:05 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:11:06.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:05 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:05 smithi022 ceph-mon[36023]: pgmap v194: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-04-16T17:11:06.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:05 smithi022 ceph-mon[36023]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T17:11:06.572 INFO:tasks.workunit.client.0.smithi018.stdout: 1 238363 3.94 MB/sec execute 199 sec latency 24.601 ms 2024-04-16T17:11:07.573 INFO:tasks.workunit.client.0.smithi018.stdout: 1 239388 3.95 MB/sec execute 200 sec latency 38.629 ms 2024-04-16T17:11:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:07 smithi043 ceph-mon[36901]: pgmap v196: 129 pgs: 20 active+clean+snaptrim, 54 active+clean, 55 active+clean+snaptrim_wait; 45 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-16T17:11:08.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:07 smithi022 ceph-mon[36023]: pgmap v196: 129 pgs: 20 active+clean+snaptrim, 54 active+clean, 55 active+clean+snaptrim_wait; 45 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-16T17:11:08.580 INFO:tasks.workunit.client.0.smithi018.stdout: 1 240314 3.94 MB/sec execute 201 sec latency 54.501 ms 2024-04-16T17:11:08.975 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:01 smithi018 ceph-mon[31776]: pgmap v192: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-16T17:11:08.975 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:02 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:11:08.975 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:02 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:02 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:02 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/234028323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:03 smithi018 ceph-mon[31776]: pgmap v193: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:05 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:05 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:05 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:05 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:05 smithi018 ceph-mon[31776]: pgmap v194: 129 pgs: 129 active+clean; 42 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:05 smithi018 ceph-mon[31776]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T17:11:08.976 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:07 smithi018 ceph-mon[31776]: pgmap v196: 129 pgs: 20 active+clean+snaptrim, 54 active+clean, 55 active+clean+snaptrim_wait; 45 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-16T17:11:09.579 INFO:tasks.workunit.client.0.smithi018.stdout: 1 240842 3.95 MB/sec execute 202 sec latency 61.104 ms 2024-04-16T17:11:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:09 smithi043 ceph-mon[36901]: pgmap v197: 129 pgs: 20 active+clean+snaptrim, 54 active+clean, 55 active+clean+snaptrim_wait; 45 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-16T17:11:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:09 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:11:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:09 smithi022 ceph-mon[36023]: pgmap v197: 129 pgs: 20 active+clean+snaptrim, 54 active+clean, 55 active+clean+snaptrim_wait; 45 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-16T17:11:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:09 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:11:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.568+0000 7f303f000640 1 -- 172.21.15.18:0/1869503052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038107150 msgr2=0x7f303810ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.568+0000 7f303f000640 1 --2- 172.21.15.18:0/1869503052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038107150 0x7f303810ed20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3030009be0 tx=0x7f303002bef0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.568+0000 7f303f000640 1 -- 172.21.15.18:0/1869503052 shutdown_connections 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.568+0000 7f303f000640 1 --2- 172.21.15.18:0/1869503052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038107150 0x7f303810ed20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.568+0000 7f303f000640 1 --2- 172.21.15.18:0/1869503052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3038102a30 0x7f3038106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.568+0000 7f303f000640 1 --2- 172.21.15.18:0/1869503052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3038102080 0x7f3038102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.568+0000 7f303f000640 1 -- 172.21.15.18:0/1869503052 >> 172.21.15.18:0/1869503052 conn(0x7f30380fd940 msgr2=0x7f30380ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.569+0000 7f303f000640 1 -- 172.21.15.18:0/1869503052 shutdown_connections 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.569+0000 7f303f000640 1 -- 172.21.15.18:0/1869503052 wait complete. 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.570+0000 7f303f000640 1 Processor -- start 2024-04-16T17:11:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.570+0000 7f303f000640 1 -- start start 2024-04-16T17:11:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.570+0000 7f303f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3038102080 0x7f3038198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.570+0000 7f303f000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3038107150 0x7f3038198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.570+0000 7f303f000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f303819cf90 0x7f303819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.570+0000 7f303f000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3038112240 con 0x7f3038107150 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.570+0000 7f303f000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f30381120c0 con 0x7f303819cf90 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.570+0000 7f303f000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30381123c0 con 0x7f3038102080 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3038102080 0x7f3038198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303d200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3038107150 0x7f3038198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3038102080 0x7f3038198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53416/0 (socket says 172.21.15.18:53416) 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303dc00640 1 -- 172.21.15.18:0/3043899211 learned_addr learned my addr 172.21.15.18:0/3043899211 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303e600640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f303819cf90 0x7f303819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303dc00640 1 -- 172.21.15.18:0/3043899211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f303819cf90 msgr2=0x7f303819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303dc00640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f303819cf90 0x7f303819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303dc00640 1 -- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3038107150 msgr2=0x7f3038198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303dc00640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3038107150 0x7f3038198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.571+0000 7f303dc00640 1 -- 172.21.15.18:0/3043899211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303819da80 con 0x7f3038102080 2024-04-16T17:11:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.572+0000 7f303dc00640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3038102080 0x7f3038198720 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f302c00e9c0 tx=0x7f302c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.572+0000 7f3036000640 1 -- 172.21.15.18:0/3043899211 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c00ccc0 con 0x7f3038102080 2024-04-16T17:11:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.572+0000 7f303f000640 1 -- 172.21.15.18:0/3043899211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303819dd70 con 0x7f3038102080 2024-04-16T17:11:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.572+0000 7f3036000640 1 -- 172.21.15.18:0/3043899211 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c005500 con 0x7f3038102080 2024-04-16T17:11:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.572+0000 7f3036000640 1 -- 172.21.15.18:0/3043899211 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c010430 con 0x7f3038102080 2024-04-16T17:11:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.572+0000 7f303f000640 1 -- 172.21.15.18:0/3043899211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3038074550 con 0x7f3038102080 2024-04-16T17:11:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.575+0000 7f3043404740 1 Processor -- start 2024-04-16T17:11:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f3043404740 1 -- start start 2024-04-16T17:11:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f3043404740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f3043404740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f3043404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f3043404740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5600af50d780 con 0x5600af5a97f0 2024-04-16T17:11:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f3043404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5600af4b3f40 con 0x5600af5b2000 2024-04-16T17:11:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f3043404740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5600af509740 con 0x5600af5a8a80 2024-04-16T17:11:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f303e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.576+0000 7f303d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f303e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55332/0 (socket says 172.21.15.18:55332) 2024-04-16T17:11:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f303e600640 1 -- 172.21.15.18:0/2992842563 learned_addr learned my addr 172.21.15.18:0/2992842563 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f3036000640 1 -- 172.21.15.18:0/3043899211 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f302c0046c0 con 0x7f3038102080 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f303d200640 1 -- 172.21.15.18:0/2992842563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 msgr2=0x5600af5a8f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f303d200640 1 --2- 172.21.15.18:0/2992842563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5a8f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f303d200640 1 -- 172.21.15.18:0/2992842563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 msgr2=0x5600af5a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f303d200640 1 --2- 172.21.15.18:0/2992842563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f303d200640 1 -- 172.21.15.18:0/2992842563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600af220210 con 0x5600af5b2000 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.577+0000 7f303d200640 1 --2- 172.21.15.18:0/2992842563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5b43f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f303806b530 tx=0x7f3024005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403318eb3486bd61 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.578+0000 7f302aa00640 1 -- 172.21.15.18:0/2992842563 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024004250 con 0x5600af5b2000 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.578+0000 7f302aa00640 1 -- 172.21.15.18:0/2992842563 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30240043f0 con 0x5600af5b2000 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.578+0000 7f302aa00640 1 -- 172.21.15.18:0/2992842563 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302401a9a0 con 0x5600af5b2000 2024-04-16T17:11:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3043404740 1 -- 172.21.15.18:0/2992842563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 msgr2=0x5600af5b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3043404740 1 --2- 172.21.15.18:0/2992842563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5b43f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f303806b530 tx=0x7f3024005f90 comp rx=0 tx=0).stop 2024-04-16T17:11:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3036000640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f30200a3d00 0x7f30200a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.580 INFO:tasks.workunit.client.0.smithi018.stdout: 1 241466 3.95 MB/sec execute 203 sec latency 111.818 ms 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3036000640 1 -- 172.21.15.18:0/3043899211 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9642+0+0 (secure 0 0 0) 0x7f302c0d0f10 con 0x7f3038102080 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3043404740 1 -- 172.21.15.18:0/2992842563 shutdown_connections 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3043404740 1 --2- 172.21.15.18:0/2992842563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5b43f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3043404740 1 --2- 172.21.15.18:0/2992842563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3043404740 1 --2- 172.21.15.18:0/2992842563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3043404740 1 -- 172.21.15.18:0/2992842563 >> 172.21.15.18:0/2992842563 conn(0x5600af5435d0 msgr2=0x5600af5a3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f3043404740 1 -- 172.21.15.18:0/2992842563 shutdown_connections 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.579+0000 7f303e600640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f30200a3d00 0x7f30200a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.580+0000 7f3043404740 1 -- 172.21.15.18:0/2992842563 wait complete. 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.580+0000 7f3043404740 1 -- start start 2024-04-16T17:11:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.580+0000 7f303e600640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f30200a3d00 0x7f30200a61c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f303002c430 tx=0x7f3030005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.580+0000 7f3043404740 1 Processor -- start 2024-04-16T17:11:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f3043404740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f3043404740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f3043404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f3043404740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5600af5b4d30 con 0x5600af5a97f0 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f3043404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5600af4c7390 con 0x5600af5b2000 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f3043404740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5600af5b4eb0 con 0x5600af5a8a80 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f3043404740 10 client.0 _ll_register_callbacks cb 0x5600af4b0f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f303dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f303d200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f303dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53420/0 (socket says 172.21.15.18:53420) 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.581+0000 7f303dc00640 1 -- 172.21.15.18:0/2682686484 learned_addr learned my addr 172.21.15.18:0/2682686484 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f303e600640 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f303dc00640 1 -- 172.21.15.18:0/2682686484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 msgr2=0x5600af5af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f303dc00640 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f303dc00640 1 -- 172.21.15.18:0/2682686484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 msgr2=0x5600af5af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f303dc00640 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f303dc00640 1 -- 172.21.15.18:0/2682686484 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600af2ec620 con 0x5600af5a8a80 2024-04-16T17:11:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f303dc00640 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5aeea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f302c0ce800 tx=0x7f302c09c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8609df7b4c38df96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f301f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f301f400640 1 -- 172.21.15.18:0/2682686484 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0bd0d0 con 0x5600af5a8a80 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f301f400640 1 -- 172.21.15.18:0/2682686484 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9642+0+0 (secure 0 0 0) 0x7f302c0a5230 con 0x5600af5a8a80 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.582+0000 7f3043404740 1 -- 172.21.15.18:0/2682686484 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30240043f0 con 0x5600af5a8a80 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f301f400640 1 -- 172.21.15.18:0/2682686484 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f302c0a0e00 con 0x5600af5a8a80 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f3043404740 10 client.27322 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f3043404740 1 -- 172.21.15.18:0/2682686484 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600af5c0f10 con 0x5600af5a8a80 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f301f400640 1 -- 172.21.15.18:0/2682686484 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30240043f0 con 0x5600af5a8a80 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f301f400640 1 -- 172.21.15.18:0/2682686484 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f302c0987a0 con 0x5600af5a8a80 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f3043404740 10 client.27322 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f3043404740 10 client.27322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f3043404740 20 client.27322 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f3043404740 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5600af5c23d0 0x5600af5c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f3043404740 4 client.27322 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.583+0000 7f3043404740 1 -- 172.21.15.18:0/2682686484 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600af51b1a0 con 0x5600af5c23d0 2024-04-16T17:11:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.584+0000 7f303d200640 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5600af5c23d0 0x5600af5c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.584+0000 7f303d200640 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5600af5c23d0 0x5600af5c47c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1fccf4f312d13c7 server_cookie=ea1094a424dea5a2 in_seq=0 out_seq=0 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.584+0000 7f301f400640 10 client.27322 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.586+0000 7f301f400640 1 -- 172.21.15.18:0/2682686484 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600af51b1a0 con 0x5600af5c23d0 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.586+0000 7f301f400640 10 client.27322 handle_command_reply: tid=0 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.586+0000 7f3043404740 1 client.27322 shutdown 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 -- 172.21.15.18:0/2682686484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 msgr2=0x5600af5aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5aeea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f302c0ce800 tx=0x7f302c09c470 comp rx=0 tx=0).stop 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 -- 172.21.15.18:0/2682686484 shutdown_connections 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5600af5c23d0 0x5600af5c47c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5b2000 0x5600af5af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 --2- 172.21.15.18:0/2682686484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a8a80 0x5600af5aeea0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f301f400640 0 client.27322 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 -- 172.21.15.18:0/2682686484 >> 172.21.15.18:0/2682686484 conn(0x5600af5435d0 msgr2=0x5600af5a54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 -- 172.21.15.18:0/2682686484 shutdown_connections 2024-04-16T17:11:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 1 -- 172.21.15.18:0/2682686484 wait complete. 2024-04-16T17:11:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.587+0000 7f3043404740 20 client.27322 trim_cache size 0 max 16384 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f3043404740 1 Processor -- start 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f3043404740 1 -- start start 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f3043404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5bdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f3043404740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a97f0 0x5600af5af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f3043404740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5b2000 0x5600af5ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f3043404740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f30381afca0 con 0x5600af5b2000 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f3043404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f30381ad570 con 0x5600af5a8a80 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f3043404740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30381ab3e0 con 0x5600af5a97f0 2024-04-16T17:11:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f303dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5bdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f303dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5bdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57584/0 (socket says 172.21.15.18:57584) 2024-04-16T17:11:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f303dc00640 1 -- 172.21.15.18:0/2198630245 learned_addr learned my addr 172.21.15.18:0/2198630245 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.605+0000 7f303e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5b2000 0x5600af5ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f303d200640 1 --2- 172.21.15.18:0/2198630245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a97f0 0x5600af5af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f303dc00640 1 -- 172.21.15.18:0/2198630245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a97f0 msgr2=0x5600af5af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f303dc00640 1 --2- 172.21.15.18:0/2198630245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a97f0 0x5600af5af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f303dc00640 1 -- 172.21.15.18:0/2198630245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5b2000 msgr2=0x5600af5ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f303dc00640 1 --2- 172.21.15.18:0/2198630245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5b2000 0x5600af5ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f303dc00640 1 -- 172.21.15.18:0/2198630245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600af5a8880 con 0x5600af5a8a80 2024-04-16T17:11:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f303dc00640 1 --2- 172.21.15.18:0/2198630245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5bdc60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f302c099420 tx=0x7f302c098c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcce2c46544452dc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f302a000640 1 -- 172.21.15.18:0/2198630245 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0bd0d0 con 0x5600af5a8a80 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f302a000640 1 -- 172.21.15.18:0/2198630245 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c0a56c0 con 0x5600af5a8a80 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.606+0000 7f302a000640 1 -- 172.21.15.18:0/2198630245 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c09c590 con 0x5600af5a8a80 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 -- 172.21.15.18:0/2198630245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 msgr2=0x5600af5bdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 --2- 172.21.15.18:0/2198630245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5bdc60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f302c099420 tx=0x7f302c098c50 comp rx=0 tx=0).stop 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 -- 172.21.15.18:0/2198630245 shutdown_connections 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 --2- 172.21.15.18:0/2198630245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5b2000 0x5600af5ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 --2- 172.21.15.18:0/2198630245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5a97f0 0x5600af5af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 --2- 172.21.15.18:0/2198630245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5bdc60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 -- 172.21.15.18:0/2198630245 >> 172.21.15.18:0/2198630245 conn(0x5600af5435d0 msgr2=0x5600af5b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 -- 172.21.15.18:0/2198630245 shutdown_connections 2024-04-16T17:11:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.607+0000 7f3043404740 1 -- 172.21.15.18:0/2198630245 wait complete. 2024-04-16T17:11:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.608+0000 7f3043404740 1 -- start start 2024-04-16T17:11:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.608+0000 7f3043404740 1 Processor -- start 2024-04-16T17:11:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f3043404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5ac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f3043404740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f3043404740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5b2000 0x5600af6278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f3043404740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f30381aba30 con 0x5600af5a97f0 2024-04-16T17:11:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f3043404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f30381acea0 con 0x5600af5a8a80 2024-04-16T17:11:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f3043404740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30381ac100 con 0x5600af5b2000 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f3043404740 10 client.0 _ll_register_callbacks cb 0x7f302c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5ac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57588/0 (socket says 172.21.15.18:57588) 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303dc00640 1 -- 172.21.15.18:0/2324641033 learned_addr learned my addr 172.21.15.18:0/2324641033 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303d200640 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303dc00640 1 -- 172.21.15.18:0/2324641033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5b2000 msgr2=0x5600af6278d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303dc00640 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5b2000 0x5600af6278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303dc00640 1 -- 172.21.15.18:0/2324641033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 msgr2=0x5600af5ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303dc00640 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303dc00640 1 -- 172.21.15.18:0/2324641033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600af627e10 con 0x5600af5a8a80 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.609+0000 7f303d200640 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f303dc00640 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5ac280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f302c09ad00 tx=0x7f302c098e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8d2be7324b783 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f301f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f301f400640 1 -- 172.21.15.18:0/2324641033 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c09c590 con 0x5600af5a8a80 2024-04-16T17:11:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f301f400640 1 -- 172.21.15.18:0/2324641033 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9642+0+0 (secure 0 0 0) 0x7f302c09bea0 con 0x5600af5a8a80 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f3043404740 1 -- 172.21.15.18:0/2324641033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302c0a56c0 con 0x5600af5a8a80 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f301f400640 1 -- 172.21.15.18:0/2324641033 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f302c0bd0d0 con 0x5600af5a8a80 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f3043404740 10 client.27386 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f3043404740 1 -- 172.21.15.18:0/2324641033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600af6290c0 con 0x5600af5a8a80 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.610+0000 7f301f400640 1 -- 172.21.15.18:0/2324641033 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c0a56c0 con 0x5600af5a8a80 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.611+0000 7f301f400640 1 -- 172.21.15.18:0/2324641033 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f302c0a1f90 con 0x5600af5a8a80 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.611+0000 7f3043404740 10 client.27386 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.611+0000 7f3043404740 10 client.27386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.611+0000 7f3043404740 20 client.27386 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.611+0000 7f3043404740 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5600af629450 0x5600af62b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.611+0000 7f3043404740 4 client.27386 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.611+0000 7f3043404740 1 -- 172.21.15.18:0/2324641033 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600af515010 con 0x5600af629450 2024-04-16T17:11:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.611+0000 7f303d200640 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5600af629450 0x5600af62b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.612+0000 7f303d200640 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5600af629450 0x5600af62b880 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59244aab56b4ba1c server_cookie=51f6b4f2efea01ea in_seq=0 out_seq=0 2024-04-16T17:11:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.612+0000 7f301f400640 10 client.27386 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f301f400640 1 -- 172.21.15.18:0/2324641033 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600af515010 con 0x5600af629450 2024-04-16T17:11:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f301f400640 10 client.27386 handle_command_reply: tid=0 2024-04-16T17:11:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 client.27386 shutdown 2024-04-16T17:11:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 -- 172.21.15.18:0/2324641033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 msgr2=0x5600af5ac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5ac280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f302c09ad00 tx=0x7f302c098e20 comp rx=0 tx=0).stop 2024-04-16T17:11:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 -- 172.21.15.18:0/2324641033 shutdown_connections 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5600af629450 0x5600af62b880 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600af5b2000 0x5600af6278d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600af5a97f0 0x5600af5ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 --2- 172.21.15.18:0/2324641033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600af5a8a80 0x5600af5ac280 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 -- 172.21.15.18:0/2324641033 >> 172.21.15.18:0/2324641033 conn(0x5600af5435d0 msgr2=0x5600af5b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f301f400640 0 client.27386 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.744+0000 7f3043404740 1 -- 172.21.15.18:0/2324641033 shutdown_connections 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.745+0000 7f3043404740 1 -- 172.21.15.18:0/2324641033 wait complete. 2024-04-16T17:11:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.745+0000 7f3043404740 20 client.27386 trim_cache size 0 max 16384 2024-04-16T17:11:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:11:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:11:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:11:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:11:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 -- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f30200a3d00 msgr2=0x7f30200a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f30200a3d00 0x7f30200a61c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f303002c430 tx=0x7f3030005fd0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 -- 172.21.15.18:0/3043899211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3038102080 msgr2=0x7f3038198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3038102080 0x7f3038198720 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f302c00e9c0 tx=0x7f302c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:11:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 -- 172.21.15.18:0/3043899211 shutdown_connections 2024-04-16T17:11:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f30200a3d00 0x7f30200a61c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f303819cf90 0x7f303819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3038107150 0x7f3038198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 --2- 172.21.15.18:0/3043899211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3038102080 0x7f3038198720 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.747+0000 7f303f000640 1 -- 172.21.15.18:0/3043899211 >> 172.21.15.18:0/3043899211 conn(0x7f30380fd940 msgr2=0x7f30380ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.748+0000 7f303f000640 1 -- 172.21.15.18:0/3043899211 shutdown_connections 2024-04-16T17:11:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:10.749+0000 7f303f000640 1 -- 172.21.15.18:0/3043899211 wait complete. 2024-04-16T17:11:11.099 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:09 smithi018 ceph-mon[31776]: pgmap v197: 129 pgs: 20 active+clean+snaptrim, 54 active+clean, 55 active+clean+snaptrim_wait; 45 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-16T17:11:11.099 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:09 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:11:11.579 INFO:tasks.workunit.client.0.smithi018.stdout: 1 241961 3.94 MB/sec execute 204 sec latency 80.326 ms 2024-04-16T17:11:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:11 smithi043 ceph-mon[36901]: pgmap v198: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-04-16T17:11:12.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:11 smithi022 ceph-mon[36023]: pgmap v198: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-04-16T17:11:12.580 INFO:tasks.workunit.client.0.smithi018.stdout: 1 242857 3.94 MB/sec execute 205 sec latency 21.211 ms 2024-04-16T17:11:13.580 INFO:tasks.workunit.client.0.smithi018.stdout: 1 243635 3.93 MB/sec execute 206 sec latency 82.337 ms 2024-04-16T17:11:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:13 smithi043 ceph-mon[36901]: pgmap v199: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-04-16T17:11:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:13 smithi043 ceph-mon[36901]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T17:11:14.093 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:11 smithi018 ceph-mon[31776]: pgmap v198: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-04-16T17:11:14.093 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:13 smithi018 ceph-mon[31776]: pgmap v199: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-04-16T17:11:14.093 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:13 smithi018 ceph-mon[31776]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T17:11:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:13 smithi022 ceph-mon[36023]: pgmap v199: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-04-16T17:11:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:13 smithi022 ceph-mon[36023]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T17:11:14.580 INFO:tasks.workunit.client.0.smithi018.stdout: 1 244592 3.95 MB/sec execute 207 sec latency 142.775 ms 2024-04-16T17:11:15.580 INFO:tasks.workunit.client.0.smithi018.stdout: 1 245263 3.95 MB/sec execute 208 sec latency 11.736 ms 2024-04-16T17:11:16.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:15 smithi043 ceph-mon[36901]: pgmap v201: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-04-16T17:11:16.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:15 smithi022 ceph-mon[36023]: pgmap v201: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-04-16T17:11:16.585 INFO:tasks.workunit.client.0.smithi018.stdout: 1 246008 3.94 MB/sec execute 209 sec latency 22.327 ms 2024-04-16T17:11:17.585 INFO:tasks.workunit.client.0.smithi018.stdout: 1 247228 3.95 MB/sec execute 210 sec latency 25.606 ms 2024-04-16T17:11:18.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:17 smithi043 ceph-mon[36901]: pgmap v202: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-16T17:11:18.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:17 smithi022 ceph-mon[36023]: pgmap v202: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-16T17:11:18.585 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248042 3.96 MB/sec execute 211 sec latency 83.504 ms 2024-04-16T17:11:19.585 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248363 3.96 MB/sec execute 212 sec latency 77.351 ms 2024-04-16T17:11:20.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:19 smithi043 ceph-mon[36901]: pgmap v203: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-16T17:11:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:19 smithi022 ceph-mon[36023]: pgmap v203: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-16T17:11:20.585 INFO:tasks.workunit.client.0.smithi018.stdout: 1 248715 3.95 MB/sec execute 213 sec latency 60.213 ms 2024-04-16T17:11:21.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:15 smithi018 ceph-mon[31776]: pgmap v201: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-04-16T17:11:21.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:17 smithi018 ceph-mon[31776]: pgmap v202: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-16T17:11:21.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:19 smithi018 ceph-mon[31776]: pgmap v203: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-16T17:11:21.585 INFO:tasks.workunit.client.0.smithi018.stdout: 1 249256 3.93 MB/sec execute 214 sec latency 41.294 ms 2024-04-16T17:11:22.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:21 smithi043 ceph-mon[36901]: pgmap v204: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-16T17:11:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:21 smithi022 ceph-mon[36023]: pgmap v204: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-16T17:11:22.585 INFO:tasks.workunit.client.0.smithi018.stdout: 1 250470 3.94 MB/sec execute 215 sec latency 20.728 ms 2024-04-16T17:11:23.586 INFO:tasks.workunit.client.0.smithi018.stdout: 1 251350 3.95 MB/sec execute 216 sec latency 49.231 ms 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 -- 172.21.15.18:0/1416194161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104e20 msgr2=0x7f3c30108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1416194161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104e20 0x7f3c30108e60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009900 tx=0x7f3c1c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 -- 172.21.15.18:0/1416194161 shutdown_connections 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1416194161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c30109540 0x7f3c30111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1416194161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104e20 0x7f3c30108e60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1416194161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c30104470 0x7f3c30104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 -- 172.21.15.18:0/1416194161 >> 172.21.15.18:0/1416194161 conn(0x7f3c300ffcd0 msgr2=0x7f3c301020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 -- 172.21.15.18:0/1416194161 shutdown_connections 2024-04-16T17:11:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.896+0000 7f3c34e00640 1 -- 172.21.15.18:0/1416194161 wait complete. 2024-04-16T17:11:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.897+0000 7f3c34e00640 1 Processor -- start 2024-04-16T17:11:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.897+0000 7f3c34e00640 1 -- start start 2024-04-16T17:11:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c34e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104470 0x7f3c3019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c34e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c30109540 0x7f3c3019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c34e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3019f530 0x7f3c3019f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c34e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30114600 con 0x7f3c30109540 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c34e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30114480 con 0x7f3c30104470 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c34e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30114780 con 0x7f3c3019f530 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104470 0x7f3c3019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104470 0x7f3c3019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36412/0 (socket says 172.21.15.18:36412) 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2de00640 1 -- 172.21.15.18:0/1673958838 learned_addr learned my addr 172.21.15.18:0/1673958838 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2e800640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3019f530 0x7f3c3019f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2d400640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c30109540 0x7f3c3019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2de00640 1 -- 172.21.15.18:0/1673958838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3019f530 msgr2=0x7f3c3019f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2de00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3019f530 0x7f3c3019f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2de00640 1 -- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c30109540 msgr2=0x7f3c3019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2de00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c30109540 0x7f3c3019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2de00640 1 -- 172.21.15.18:0/1673958838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c301a0020 con 0x7f3c30104470 2024-04-16T17:11:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.898+0000 7f3c2e800640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3019f530 0x7f3c3019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.899+0000 7f3c2de00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104470 0x7f3c3019ac40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000e9f0 tx=0x7f3c2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.899+0000 7f3c26a00640 1 -- 172.21.15.18:0/1673958838 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000cd80 con 0x7f3c30104470 2024-04-16T17:11:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.899+0000 7f3c26a00640 1 -- 172.21.15.18:0/1673958838 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c20010910 con 0x7f3c30104470 2024-04-16T17:11:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.899+0000 7f3c34e00640 1 -- 172.21.15.18:0/1673958838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c301a0310 con 0x7f3c30104470 2024-04-16T17:11:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.899+0000 7f3c26a00640 1 -- 172.21.15.18:0/1673958838 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20018990 con 0x7f3c30104470 2024-04-16T17:11:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.899+0000 7f3c34e00640 1 -- 172.21.15.18:0/1673958838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c301a7b20 con 0x7f3c30104470 2024-04-16T17:11:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.903+0000 7f3c37b6f740 1 Processor -- start 2024-04-16T17:11:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.903+0000 7f3c37b6f740 1 -- start start 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c26a00640 1 -- 172.21.15.18:0/1673958838 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c20010ac0 con 0x7f3c30104470 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309563360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309563d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f30956f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f3094c8780 con 0x55f309564610 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f30946ef40 con 0x55f3095638a0 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f3094c4740 con 0x55f30956ce20 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c2e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309563360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c2d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f30956f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c2e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309563360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38442/0 (socket says 172.21.15.18:38442) 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c2e800640 1 -- 172.21.15.18:0/191469796 learned_addr learned my addr 172.21.15.18:0/191469796 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.904+0000 7f3c2de00640 1 --2- 172.21.15.18:0/191469796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309563d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c2d400640 1 -- 172.21.15.18:0/191469796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 msgr2=0x55f309563d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c2d400640 1 --2- 172.21.15.18:0/191469796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309563d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c2d400640 1 -- 172.21.15.18:0/191469796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 msgr2=0x55f309563360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c2d400640 1 --2- 172.21.15.18:0/191469796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309563360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c2d400640 1 -- 172.21.15.18:0/191469796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3091db210 con 0x55f30956ce20 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c2d400640 1 --2- 172.21.15.18:0/191469796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f30956f210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c036040 tx=0x7f3c1c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=124608f81ed5d415 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c1b400640 1 -- 172.21.15.18:0/191469796 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c042070 con 0x55f30956ce20 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c1b400640 1 -- 172.21.15.18:0/191469796 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c004240 con 0x55f30956ce20 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c1b400640 1 -- 172.21.15.18:0/191469796 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0046c0 con 0x55f30956ce20 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.905+0000 7f3c26a00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3c140a3d90 0x7f3c140a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c26a00640 1 -- 172.21.15.18:0/1673958838 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f3c200d0ff0 con 0x7f3c30104470 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c2e800640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3c140a3d90 0x7f3c140a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c37b6f740 1 -- 172.21.15.18:0/191469796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 msgr2=0x55f30956f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/191469796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f30956f210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c036040 tx=0x7f3c1c005de0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c37b6f740 1 -- 172.21.15.18:0/191469796 shutdown_connections 2024-04-16T17:11:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/191469796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f30956f210 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/191469796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309563d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/191469796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309563360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c37b6f740 1 -- 172.21.15.18:0/191469796 >> 172.21.15.18:0/191469796 conn(0x55f3094fe5d0 msgr2=0x55f30955e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.906+0000 7f3c37b6f740 1 -- 172.21.15.18:0/191469796 shutdown_connections 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.907+0000 7f3c37b6f740 1 -- 172.21.15.18:0/191469796 wait complete. 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.907+0000 7f3c2e800640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3c140a3d90 0x7f3c140a6250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009fd0 tx=0x7f3c28009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.907+0000 7f3c37b6f740 1 -- start start 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.907+0000 7f3c37b6f740 1 Processor -- start 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3095638a0 0x55f30956c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309569c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f30956ce20 0x55f30956a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f30956faf0 con 0x55f309564610 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f309482390 con 0x55f30956ce20 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f30956fc70 con 0x55f3095638a0 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c37b6f740 10 client.0 _ll_register_callbacks cb 0x55f30946bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c2d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309569c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c2d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309569c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38454/0 (socket says 172.21.15.18:38454) 2024-04-16T17:11:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c2d400640 1 -- 172.21.15.18:0/2355251371 learned_addr learned my addr 172.21.15.18:0/2355251371 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.908+0000 7f3c2e800640 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f30956ce20 0x55f30956a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.909+0000 7f3c2d400640 1 -- 172.21.15.18:0/2355251371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3095638a0 msgr2=0x55f30956c8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.909+0000 7f3c2d400640 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3095638a0 0x55f30956c8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.909+0000 7f3c2d400640 1 -- 172.21.15.18:0/2355251371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f30956ce20 msgr2=0x55f30956a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.909+0000 7f3c2d400640 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f30956ce20 0x55f30956a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.909+0000 7f3c2d400640 1 -- 172.21.15.18:0/2355251371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3092a7620 con 0x55f309564610 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.909+0000 7f3c2d400640 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309569c30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c02bc80 tx=0x7f3c1c02c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922deb99c0d84a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.909+0000 7f3c0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.910+0000 7f3c0fe00640 1 -- 172.21.15.18:0/2355251371 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c042050 con 0x55f309564610 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.910+0000 7f3c0fe00640 1 -- 172.21.15.18:0/2355251371 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f3c1c02cc50 con 0x55f309564610 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.910+0000 7f3c37b6f740 1 -- 172.21.15.18:0/2355251371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c1c004240 con 0x55f309564610 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.910+0000 7f3c0fe00640 1 -- 172.21.15.18:0/2355251371 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c1c050480 con 0x55f309564610 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.910+0000 7f3c37b6f740 10 client.17157 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.910+0000 7f3c37b6f740 1 -- 172.21.15.18:0/2355251371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f30957bbc0 con 0x55f309564610 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.910+0000 7f3c0fe00640 1 -- 172.21.15.18:0/2355251371 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c004240 con 0x55f309564610 2024-04-16T17:11:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.911+0000 7f3c0fe00640 1 -- 172.21.15.18:0/2355251371 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3c1c0345d0 con 0x55f309564610 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.911+0000 7f3c37b6f740 10 client.17157 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.911+0000 7f3c37b6f740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.911+0000 7f3c37b6f740 20 client.17157 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.911+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f30957cf10 0x55f30957f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.911+0000 7f3c37b6f740 4 client.17157 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.911+0000 7f3c37b6f740 1 -- 172.21.15.18:0/2355251371 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3094d61a0 con 0x55f30957cf10 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.911+0000 7f3c2de00640 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f30957cf10 0x55f30957f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.912+0000 7f3c0fe00640 10 client.17157 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.912+0000 7f3c2de00640 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f30957cf10 0x55f30957f300 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7395dc419908ee3d server_cookie=f4c5a4c2e4753694 in_seq=0 out_seq=0 2024-04-16T17:11:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c0fe00640 1 -- 172.21.15.18:0/2355251371 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3094d61a0 con 0x55f30957cf10 2024-04-16T17:11:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c0fe00640 10 client.17157 handle_command_reply: tid=0 2024-04-16T17:11:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 client.17157 shutdown 2024-04-16T17:11:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 -- 172.21.15.18:0/2355251371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 msgr2=0x55f309569c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309569c30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c02bc80 tx=0x7f3c1c02c7a0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 -- 172.21.15.18:0/2355251371 shutdown_connections 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f30957cf10 0x55f30957f300 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f30956ce20 0x55f30956a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309569c30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/2355251371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3095638a0 0x55f30956c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c0fe00640 0 client.17157 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 -- 172.21.15.18:0/2355251371 >> 172.21.15.18:0/2355251371 conn(0x55f3094fe5d0 msgr2=0x55f3095602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 -- 172.21.15.18:0/2355251371 shutdown_connections 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 1 -- 172.21.15.18:0/2355251371 wait complete. 2024-04-16T17:11:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.913+0000 7f3c37b6f740 20 client.17157 trim_cache size 0 max 16384 2024-04-16T17:11:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c37b6f740 1 Processor -- start 2024-04-16T17:11:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c37b6f740 1 -- start start 2024-04-16T17:11:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f3095789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309568c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f309584900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b4110 con 0x55f309564610 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b19e0 con 0x55f3095638a0 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301af850 con 0x55f30956ce20 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c2de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f3095789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c2d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309568c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c2de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f3095789f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36430/0 (socket says 172.21.15.18:36430) 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c2de00640 1 -- 172.21.15.18:0/3796606731 learned_addr learned my addr 172.21.15.18:0/3796606731 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c2e800640 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f309584900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.930+0000 7f3c2de00640 1 -- 172.21.15.18:0/3796606731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 msgr2=0x55f309584900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c2de00640 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f309584900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c2de00640 1 -- 172.21.15.18:0/3796606731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 msgr2=0x55f309568c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c2de00640 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309568c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c2de00640 1 -- 172.21.15.18:0/3796606731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f309562b70 con 0x55f3095638a0 2024-04-16T17:11:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c2e800640 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f309584900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c2de00640 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f3095789f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3c2009cb10 tx=0x7f3c200d3500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ba729572890b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c1aa00640 1 -- 172.21.15.18:0/3796606731 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2009cb80 con 0x55f3095638a0 2024-04-16T17:11:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c1aa00640 1 -- 172.21.15.18:0/3796606731 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c200a0f30 con 0x55f3095638a0 2024-04-16T17:11:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.931+0000 7f3c1aa00640 1 -- 172.21.15.18:0/3796606731 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c200bd040 con 0x55f3095638a0 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 -- 172.21.15.18:0/3796606731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 msgr2=0x55f3095789f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f3095789f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3c2009cb10 tx=0x7f3c200d3500 comp rx=0 tx=0).stop 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 -- 172.21.15.18:0/3796606731 shutdown_connections 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f30956ce20 0x55f309584900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f309564610 0x55f309568c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/3796606731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f3095789f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 -- 172.21.15.18:0/3796606731 >> 172.21.15.18:0/3796606731 conn(0x55f3094fe5d0 msgr2=0x55f309568540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 -- 172.21.15.18:0/3796606731 shutdown_connections 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 -- 172.21.15.18:0/3796606731 wait complete. 2024-04-16T17:11:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 -- start start 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.932+0000 7f3c37b6f740 1 Processor -- start 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309574f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f309564610 0x55f309575440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c37b6f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f30956ce20 0x55f309565ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301afea0 con 0x55f30956ce20 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b1310 con 0x55f3095638a0 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c37b6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b0570 con 0x55f309564610 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c37b6f740 10 client.0 _ll_register_callbacks cb 0x7f3c28009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c2e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f30956ce20 0x55f309565ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c2de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309574f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c2e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f30956ce20 0x55f309565ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38480/0 (socket says 172.21.15.18:38480) 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c2e800640 1 -- 172.21.15.18:0/290772920 learned_addr learned my addr 172.21.15.18:0/290772920 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c2de00640 1 -- 172.21.15.18:0/290772920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f309564610 msgr2=0x55f309575440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.933+0000 7f3c2de00640 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f309564610 0x55f309575440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c2de00640 1 -- 172.21.15.18:0/290772920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f30956ce20 msgr2=0x55f309565ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c2de00640 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f30956ce20 0x55f309565ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c2de00640 1 -- 172.21.15.18:0/290772920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3095664d0 con 0x55f3095638a0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c2de00640 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309574f00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3c2009c880 tx=0x7f3c200ce350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e6d8488167bf2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c0fe00640 1 -- 172.21.15.18:0/290772920 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c200b6060 con 0x55f3095638a0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c0fe00640 1 -- 172.21.15.18:0/290772920 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f3c200ce5b0 con 0x55f3095638a0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c37b6f740 1 -- 172.21.15.18:0/290772920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c200a0f30 con 0x55f3095638a0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c0fe00640 1 -- 172.21.15.18:0/290772920 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c2009cb80 con 0x55f3095638a0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c37b6f740 10 client.27404 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c37b6f740 1 -- 172.21.15.18:0/290772920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3095677c0 con 0x55f3095638a0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.934+0000 7f3c0fe00640 1 -- 172.21.15.18:0/290772920 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c200a0f30 con 0x55f3095638a0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.935+0000 7f3c0fe00640 1 -- 172.21.15.18:0/290772920 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3c1c0054d0 con 0x55f3095638a0 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.935+0000 7f3c37b6f740 10 client.27404 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.935+0000 7f3c37b6f740 10 client.27404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.935+0000 7f3c37b6f740 20 client.27404 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.935+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3095e4180 0x55f3095e6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.935+0000 7f3c37b6f740 4 client.27404 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:11:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.935+0000 7f3c37b6f740 1 -- 172.21.15.18:0/290772920 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3094d0010 con 0x55f3095e4180 2024-04-16T17:11:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.935+0000 7f3c2d400640 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3095e4180 0x55f3095e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.936+0000 7f3c2d400640 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3095e4180 0x55f3095e6570 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e37119709ec0d5a server_cookie=ba865ca0ac3ff60f in_seq=0 out_seq=0 2024-04-16T17:11:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:23.936+0000 7f3c0fe00640 10 client.27404 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:24.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:23 smithi043 ceph-mon[36901]: pgmap v205: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-04-16T17:11:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c0fe00640 1 -- 172.21.15.18:0/290772920 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3094d0010 con 0x55f3095e4180 2024-04-16T17:11:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c0fe00640 10 client.27404 handle_command_reply: tid=0 2024-04-16T17:11:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 client.27404 shutdown 2024-04-16T17:11:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 -- 172.21.15.18:0/290772920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 msgr2=0x55f309574f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309574f00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3c2009c880 tx=0x7f3c200ce350 comp rx=0 tx=0).stop 2024-04-16T17:11:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 -- 172.21.15.18:0/290772920 shutdown_connections 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3095e4180 0x55f3095e6570 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f30956ce20 0x55f309565ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f309564610 0x55f309575440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 --2- 172.21.15.18:0/290772920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3095638a0 0x55f309574f00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.076+0000 7f3c37b6f740 1 -- 172.21.15.18:0/290772920 >> 172.21.15.18:0/290772920 conn(0x55f3094fe5d0 msgr2=0x55f30955ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.077+0000 7f3c0fe00640 0 client.27404 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.077+0000 7f3c37b6f740 1 -- 172.21.15.18:0/290772920 shutdown_connections 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.077+0000 7f3c37b6f740 1 -- 172.21.15.18:0/290772920 wait complete. 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.077+0000 7f3c37b6f740 20 client.27404 trim_cache size 0 max 16384 2024-04-16T17:11:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 -- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3c140a3d90 msgr2=0x7f3c140a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3c140a3d90 0x7f3c140a6250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009fd0 tx=0x7f3c28009500 comp rx=0 tx=0).stop 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 -- 172.21.15.18:0/1673958838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104470 msgr2=0x7f3c3019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104470 0x7f3c3019ac40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000e9f0 tx=0x7f3c2000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 -- 172.21.15.18:0/1673958838 shutdown_connections 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3c140a3d90 0x7f3c140a6250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3019f530 0x7f3c3019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c30109540 0x7f3c3019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 --2- 172.21.15.18:0/1673958838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c30104470 0x7f3c3019ac40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.078+0000 7f3c34e00640 1 -- 172.21.15.18:0/1673958838 >> 172.21.15.18:0/1673958838 conn(0x7f3c300ffcd0 msgr2=0x7f3c30102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.080+0000 7f3c34e00640 1 -- 172.21.15.18:0/1673958838 shutdown_connections 2024-04-16T17:11:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:24.080+0000 7f3c34e00640 1 -- 172.21.15.18:0/1673958838 wait complete. 2024-04-16T17:11:24.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:23 smithi022 ceph-mon[36023]: pgmap v205: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-04-16T17:11:24.585 INFO:tasks.workunit.client.0.smithi018.stdout: 1 251838 3.96 MB/sec execute 217 sec latency 67.926 ms 2024-04-16T17:11:25.586 INFO:tasks.workunit.client.0.smithi018.stdout: 1 252161 3.95 MB/sec execute 218 sec latency 121.935 ms 2024-04-16T17:11:26.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:25 smithi022 ceph-mon[36023]: pgmap v206: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-16T17:11:26.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:25 smithi043 ceph-mon[36901]: pgmap v206: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-16T17:11:26.586 INFO:tasks.workunit.client.0.smithi018.stdout: 1 252670 3.93 MB/sec execute 219 sec latency 71.248 ms 2024-04-16T17:11:27.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:26 smithi022 ceph-mon[36023]: pgmap v207: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-04-16T17:11:27.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:26 smithi043 ceph-mon[36901]: pgmap v207: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-04-16T17:11:27.586 INFO:tasks.workunit.client.0.smithi018.stdout: 1 253688 3.94 MB/sec execute 220 sec latency 26.140 ms 2024-04-16T17:11:28.590 INFO:tasks.workunit.client.0.smithi018.stdout: 1 254492 3.93 MB/sec execute 221 sec latency 61.963 ms 2024-04-16T17:11:29.590 INFO:tasks.workunit.client.0.smithi018.stdout: 1 254901 3.93 MB/sec execute 222 sec latency 70.046 ms 2024-04-16T17:11:29.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:29 smithi022 ceph-mon[36023]: pgmap v208: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:11:29.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:29 smithi043 ceph-mon[36901]: pgmap v208: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:11:30.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:21 smithi018 ceph-mon[31776]: pgmap v204: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-16T17:11:30.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:23 smithi018 ceph-mon[31776]: pgmap v205: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-04-16T17:11:30.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:25 smithi018 ceph-mon[31776]: pgmap v206: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-16T17:11:30.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:26 smithi018 ceph-mon[31776]: pgmap v207: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-04-16T17:11:30.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:29 smithi018 ceph-mon[31776]: pgmap v208: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:11:30.590 INFO:tasks.workunit.client.0.smithi018.stdout: 1 255671 3.94 MB/sec execute 223 sec latency 52.837 ms 2024-04-16T17:11:31.590 INFO:tasks.workunit.client.0.smithi018.stdout: 1 256172 3.93 MB/sec execute 224 sec latency 20.609 ms 2024-04-16T17:11:31.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:31 smithi022 ceph-mon[36023]: pgmap v209: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-04-16T17:11:31.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:31 smithi043 ceph-mon[36901]: pgmap v209: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-04-16T17:11:32.591 INFO:tasks.workunit.client.0.smithi018.stdout: 1 257205 3.93 MB/sec execute 225 sec latency 24.972 ms 2024-04-16T17:11:33.591 INFO:tasks.workunit.client.0.smithi018.stdout: 1 258069 3.93 MB/sec execute 226 sec latency 67.277 ms 2024-04-16T17:11:33.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:33 smithi022 ceph-mon[36023]: pgmap v210: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:11:33.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:33 smithi043 ceph-mon[36901]: pgmap v210: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:11:34.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:11:34.475 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:11:34.590 INFO:tasks.workunit.client.0.smithi018.stdout: 1 258590 3.93 MB/sec execute 227 sec latency 65.479 ms 2024-04-16T17:11:34.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 -- 172.21.15.18:0/1697236596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4104c30 msgr2=0x7f66f4108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 --2- 172.21.15.18:0/1697236596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4104c30 0x7f66f4108c70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009900 tx=0x7f66e002bc30 comp rx=0 tx=0).stop 2024-04-16T17:11:34.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 -- 172.21.15.18:0/1697236596 shutdown_connections 2024-04-16T17:11:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 --2- 172.21.15.18:0/1697236596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f4109350 0x7f66f4110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 --2- 172.21.15.18:0/1697236596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4104c30 0x7f66f4108c70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 --2- 172.21.15.18:0/1697236596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f4104280 0x7f66f4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 -- 172.21.15.18:0/1697236596 >> 172.21.15.18:0/1697236596 conn(0x7f66f40ffae0 msgr2=0x7f66f4101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 -- 172.21.15.18:0/1697236596 shutdown_connections 2024-04-16T17:11:34.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.665+0000 7f66f9c00640 1 -- 172.21.15.18:0/1697236596 wait complete. 2024-04-16T17:11:34.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.666+0000 7f66f9c00640 1 Processor -- start 2024-04-16T17:11:34.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f9c00640 1 -- start start 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f4104280 0x7f66f41a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f9c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4109350 0x7f66f41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f9c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f419a820 0x7f66f419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f9c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4114410 con 0x7f66f419a820 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f9c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4114290 con 0x7f66f4109350 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f9c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4114590 con 0x7f66f4104280 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f4104280 0x7f66f41a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4109350 0x7f66f41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f4104280 0x7f66f41a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37850/0 (socket says 172.21.15.18:37850) 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f3e00640 1 -- 172.21.15.18:0/3543611946 learned_addr learned my addr 172.21.15.18:0/3543611946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.667+0000 7f66f9200640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f419a820 0x7f66f419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.668+0000 7f66f3400640 1 -- 172.21.15.18:0/3543611946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f4104280 msgr2=0x7f66f41a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.668+0000 7f66f3400640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f4104280 0x7f66f41a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.668+0000 7f66f3400640 1 -- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f419a820 msgr2=0x7f66f419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.668+0000 7f66f3400640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f419a820 0x7f66f419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.668+0000 7f66f3400640 1 -- 172.21.15.18:0/3543611946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f419b3b0 con 0x7f66f4109350 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.668+0000 7f66f3400640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4109350 0x7f66f41a0bf0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f66e0036040 tx=0x7f66e0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.668+0000 7f66f0c00640 1 -- 172.21.15.18:0/3543611946 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0043020 con 0x7f66f4109350 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.668+0000 7f66f9c00640 1 -- 172.21.15.18:0/3543611946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f419b640 con 0x7f66f4109350 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.669+0000 7f66f9c00640 1 -- 172.21.15.18:0/3543611946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f41a7840 con 0x7f66f4109350 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.669+0000 7f66f0c00640 1 -- 172.21.15.18:0/3543611946 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e0004c80 con 0x7f66f4109350 2024-04-16T17:11:34.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.669+0000 7f66f0c00640 1 -- 172.21.15.18:0/3543611946 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00035f0 con 0x7f66f4109350 2024-04-16T17:11:34.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.672+0000 7f66fdf94740 1 Processor -- start 2024-04-16T17:11:34.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.672+0000 7f66fdf94740 1 -- start start 2024-04-16T17:11:34.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.672+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f7623770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.672+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f762c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.672+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f762ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.672+0000 7f66fdf94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f6f7589120 con 0x55f6f762c890 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.672+0000 7f66fdf94740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f6f752f340 con 0x55f6f7623390 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.672+0000 7f66fdf94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f6f75850e0 con 0x55f6f7623cb0 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f7623770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f7623770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54128/0 (socket says 172.21.15.18:54128) 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 -- 172.21.15.18:0/1802568043 learned_addr learned my addr 172.21.15.18:0/1802568043 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f3e00640 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f762c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 -- 172.21.15.18:0/1802568043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 msgr2=0x55f6f762c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f762c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 -- 172.21.15.18:0/1802568043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 msgr2=0x55f6f762ec80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f762ec80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 -- 172.21.15.18:0/1802568043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6f73657d0 con 0x55f6f7623390 2024-04-16T17:11:34.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f3e00640 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f762c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:34.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f9200640 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f7623770 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009f50 tx=0x7f66ec0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc6124af29de3e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:34.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.673+0000 7f66f0c00640 1 -- 172.21.15.18:0/3543611946 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66e0046020 con 0x7f66f4109350 2024-04-16T17:11:34.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.674+0000 7f66e9600640 1 -- 172.21.15.18:0/1802568043 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec004d20 con 0x55f6f7623390 2024-04-16T17:11:34.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.674+0000 7f66e9600640 1 -- 172.21.15.18:0/1802568043 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ec004050 con 0x55f6f7623390 2024-04-16T17:11:34.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.674+0000 7f66e9600640 1 -- 172.21.15.18:0/1802568043 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec005280 con 0x55f6f7623390 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.674+0000 7f66fdf94740 1 -- 172.21.15.18:0/1802568043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 msgr2=0x55f6f7623770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.674+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f7623770 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009f50 tx=0x7f66ec0118d0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.675+0000 7f66fdf94740 1 -- 172.21.15.18:0/1802568043 shutdown_connections 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.675+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f762ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.675+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f762c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.675+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1802568043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f7623770 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.675+0000 7f66fdf94740 1 -- 172.21.15.18:0/1802568043 >> 172.21.15.18:0/1802568043 conn(0x55f6f761dac0 msgr2=0x55f6f761dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.675+0000 7f66f0c00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66d80a3da0 0x7f66d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.675+0000 7f66f0c00640 1 -- 172.21.15.18:0/3543611946 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f66e00f2300 con 0x7f66f4109350 2024-04-16T17:11:34.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.676+0000 7f66fdf94740 1 -- 172.21.15.18:0/1802568043 shutdown_connections 2024-04-16T17:11:34.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.676+0000 7f66fdf94740 1 -- 172.21.15.18:0/1802568043 wait complete. 2024-04-16T17:11:34.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.676+0000 7f66f3e00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66d80a3da0 0x7f66d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.677+0000 7f66fdf94740 1 -- start start 2024-04-16T17:11:34.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.677+0000 7f66fdf94740 1 Processor -- start 2024-04-16T17:11:34.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.677+0000 7f66f3e00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66d80a3da0 0x7f66d80a6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55f6f7622e60 tx=0x7f66e4005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f76298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7629e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f762a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66fdf94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f6f762f680 con 0x55f6f762c890 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66fdf94740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f6f7542710 con 0x55f6f7623390 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66fdf94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f6f762f800 con 0x55f6f7623cb0 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66fdf94740 10 client.0 _ll_register_callbacks cb 0x55f6f753daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f76298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.678+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f76298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54140/0 (socket says 172.21.15.18:54140) 2024-04-16T17:11:34.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.679+0000 7f66f3e00640 1 -- 172.21.15.18:0/1380122346 learned_addr learned my addr 172.21.15.18:0/1380122346 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:34.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.679+0000 7f66f3400640 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7629e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.679+0000 7f66f9200640 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f762a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66f9200640 1 -- 172.21.15.18:0/1380122346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 msgr2=0x55f6f7629e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66f9200640 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7629e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66f9200640 1 -- 172.21.15.18:0/1380122346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 msgr2=0x55f6f76298e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66f9200640 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f76298e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66f9200640 1 -- 172.21.15.18:0/1380122346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6f729b3f0 con 0x55f6f762c890 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66f3400640 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7629e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66f9200640 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f762a360 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66ec012fd0 tx=0x7f66ec0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61eaa6bd61289650 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66de000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.680+0000 7f66de000640 1 -- 172.21.15.18:0/1380122346 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec029020 con 0x55f6f762c890 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.681+0000 7f66de000640 1 -- 172.21.15.18:0/1380122346 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f66ec025470 con 0x55f6f762c890 2024-04-16T17:11:34.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.681+0000 7f66fdf94740 1 -- 172.21.15.18:0/1380122346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec004050 con 0x55f6f762c890 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.681+0000 7f66de000640 1 -- 172.21.15.18:0/1380122346 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66ec025690 con 0x55f6f762c890 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.681+0000 7f66fdf94740 10 client.17172 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.681+0000 7f66fdf94740 1 -- 172.21.15.18:0/1380122346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6f725b0f0 con 0x55f6f762c890 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66de000640 1 -- 172.21.15.18:0/1380122346 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ec004050 con 0x55f6f762c890 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66de000640 1 -- 172.21.15.18:0/1380122346 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f66ec00f810 con 0x55f6f762c890 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66fdf94740 10 client.17172 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66fdf94740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66fdf94740 20 client.17172 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f6f763cc00 0x55f6f763eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66fdf94740 4 client.17172 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:34.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66fdf94740 1 -- 172.21.15.18:0/1380122346 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6f7596b40 con 0x55f6f763cc00 2024-04-16T17:11:34.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.682+0000 7f66f3e00640 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f6f763cc00 0x55f6f763eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.683+0000 7f66f3e00640 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f6f763cc00 0x55f6f763eff0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d47006c9bbdebeca server_cookie=8906d14e9916044a in_seq=0 out_seq=0 2024-04-16T17:11:34.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.683+0000 7f66de000640 10 client.17172 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:34.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66de000640 1 -- 172.21.15.18:0/1380122346 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6f7596b40 con 0x55f6f763cc00 2024-04-16T17:11:34.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66de000640 10 client.17172 handle_command_reply: tid=0 2024-04-16T17:11:34.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 client.17172 shutdown 2024-04-16T17:11:34.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 -- 172.21.15.18:0/1380122346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 msgr2=0x55f6f762a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f762a360 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66ec012fd0 tx=0x7f66ec0044b0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 -- 172.21.15.18:0/1380122346 shutdown_connections 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f6f763cc00 0x55f6f763eff0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f762a360 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7629e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 --2- 172.21.15.18:0/1380122346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623390 0x55f6f76298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 -- 172.21.15.18:0/1380122346 >> 172.21.15.18:0/1380122346 conn(0x55f6f761dac0 msgr2=0x55f6f761f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66de000640 0 client.17172 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 -- 172.21.15.18:0/1380122346 shutdown_connections 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.684+0000 7f66fdf94740 1 -- 172.21.15.18:0/1380122346 wait complete. 2024-04-16T17:11:34.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.685+0000 7f66fdf94740 20 client.17172 trim_cache size 0 max 16384 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.705+0000 7f66fdf94740 1 Processor -- start 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.705+0000 7f66fdf94740 1 -- start start 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623390 0x55f6f762c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623cb0 0x55f6f76385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f76189b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66fdf94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66d4012950 con 0x55f6f762c890 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66fdf94740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66d4010220 con 0x55f6f7623cb0 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66fdf94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400e090 con 0x55f6f7623390 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623390 0x55f6f762c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623cb0 0x55f6f76385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623cb0 0x55f6f76385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54154/0 (socket says 172.21.15.18:54154) 2024-04-16T17:11:34.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623390 0x55f6f762c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37874/0 (socket says 172.21.15.18:37874) 2024-04-16T17:11:34.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3400640 1 -- 172.21.15.18:0/2876383881 learned_addr learned my addr 172.21.15.18:0/2876383881 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:34.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f9200640 1 --2- 172.21.15.18:0/2876383881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f76189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3400640 1 -- 172.21.15.18:0/2876383881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623390 msgr2=0x55f6f762c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:34.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3400640 1 --2- 172.21.15.18:0/2876383881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623390 0x55f6f762c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3400640 1 -- 172.21.15.18:0/2876383881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 msgr2=0x55f6f76189b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.706+0000 7f66f3400640 1 --2- 172.21.15.18:0/2876383881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f76189b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.707+0000 7f66f3400640 1 -- 172.21.15.18:0/2876383881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6f7623ae0 con 0x55f6f7623cb0 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.707+0000 7f66f3400640 1 --2- 172.21.15.18:0/2876383881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623cb0 0x55f6f76385a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ba8f0 tx=0x7f66e00bdb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0da03d60efe500b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.707+0000 7f66e8c00640 1 -- 172.21.15.18:0/2876383881 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00bdf20 con 0x55f6f7623cb0 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.707+0000 7f66e8c00640 1 -- 172.21.15.18:0/2876383881 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e00c22d0 con 0x55f6f7623cb0 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.707+0000 7f66e8c00640 1 -- 172.21.15.18:0/2876383881 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00d7270 con 0x55f6f7623cb0 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 -- 172.21.15.18:0/2876383881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623cb0 msgr2=0x55f6f76385a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 --2- 172.21.15.18:0/2876383881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623cb0 0x55f6f76385a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ba8f0 tx=0x7f66e00bdb20 comp rx=0 tx=0).stop 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 -- 172.21.15.18:0/2876383881 shutdown_connections 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 --2- 172.21.15.18:0/2876383881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f762c890 0x55f6f76189b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 --2- 172.21.15.18:0/2876383881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f7623cb0 0x55f6f76385a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 --2- 172.21.15.18:0/2876383881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623390 0x55f6f762c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 -- 172.21.15.18:0/2876383881 >> 172.21.15.18:0/2876383881 conn(0x55f6f761dac0 msgr2=0x55f6f761f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:34.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 -- 172.21.15.18:0/2876383881 shutdown_connections 2024-04-16T17:11:34.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.708+0000 7f66fdf94740 1 -- 172.21.15.18:0/2876383881 wait complete. 2024-04-16T17:11:34.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.709+0000 7f66fdf94740 1 -- start start 2024-04-16T17:11:34.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.709+0000 7f66fdf94740 1 Processor -- start 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f7623390 0x55f6f7634ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7630010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f762c890 0x55f6f7630550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400e6e0 con 0x55f6f7623390 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400fb50 con 0x55f6f762c890 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400edb0 con 0x55f6f7623cb0 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 10 client.0 _ll_register_callbacks cb 0x7f66ec009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f7623390 0x55f6f7634ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f7623390 0x55f6f7634ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38354/0 (socket says 172.21.15.18:38354) 2024-04-16T17:11:34.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 -- 172.21.15.18:0/3968435052 learned_addr learned my addr 172.21.15.18:0/3968435052 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3400640 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7630010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 -- 172.21.15.18:0/3968435052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 msgr2=0x55f6f7630010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7630010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 -- 172.21.15.18:0/3968435052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f762c890 msgr2=0x55f6f7630550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f762c890 0x55f6f7630550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 -- 172.21.15.18:0/3968435052 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6f7630b90 con 0x55f6f7623390 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3e00640 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f7623390 0x55f6f7634ca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55f6f763f6f0 tx=0x7f66e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5da8d6967ca720 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 1 -- 172.21.15.18:0/3968435052 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4029910 con 0x55f6f7623390 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 1 -- 172.21.15.18:0/3968435052 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f66e4005020 con 0x55f6f7623390 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- 172.21.15.18:0/3968435052 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e00c22d0 con 0x55f6f7623390 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 1 -- 172.21.15.18:0/3968435052 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66e4039050 con 0x55f6f7623390 2024-04-16T17:11:34.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 10 client.17178 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- 172.21.15.18:0/3968435052 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6f76a2ec0 con 0x55f6f7623390 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 1 -- 172.21.15.18:0/3968435052 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e00c22d0 con 0x55f6f7623390 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 1 -- 172.21.15.18:0/3968435052 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f66e401a040 con 0x55f6f7623390 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 10 client.17178 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 20 client.17178 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f6f76a3640 0x55f6f76a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 4 client.17178 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- 172.21.15.18:0/3968435052 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6f7619180 con 0x55f6f76a3640 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3400640 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f6f76a3640 0x55f6f76a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f3400640 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f6f76a3640 0x55f6f76a5a80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da6c977eb5111f1 server_cookie=96cac1e257d55931 in_seq=0 out_seq=0 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 10 client.17178 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 1 -- 172.21.15.18:0/3968435052 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f6f7619180 con 0x55f6f76a3640 2024-04-16T17:11:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 10 client.17178 handle_command_reply: tid=0 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 client.17178 shutdown 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- 172.21.15.18:0/3968435052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f7623390 msgr2=0x55f6f7634ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f7623390 0x55f6f7634ca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55f6f763f6f0 tx=0x7f66e40046c0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- 172.21.15.18:0/3968435052 shutdown_connections 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f6f76a3640 0x55f6f76a5a80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6f762c890 0x55f6f7630550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6f7623cb0 0x55f6f7630010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 --2- 172.21.15.18:0/3968435052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6f7623390 0x55f6f7634ca0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- 172.21.15.18:0/3968435052 >> 172.21.15.18:0/3968435052 conn(0x55f6f761dac0 msgr2=0x55f6f761ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66de000640 0 client.17178 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- 172.21.15.18:0/3968435052 shutdown_connections 2024-04-16T17:11:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 1 -- 172.21.15.18:0/3968435052 wait complete. 2024-04-16T17:11:34.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66fdf94740 20 client.17178 trim_cache size 0 max 16384 2024-04-16T17:11:34.719 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:11:34.719 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:11:34.719 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:11:34.719 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:11:34.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f9c00640 1 -- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66d80a3da0 msgr2=0x7f66d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f9c00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66d80a3da0 0x7f66d80a6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55f6f7622e60 tx=0x7f66e4005f90 comp rx=0 tx=0).stop 2024-04-16T17:11:34.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f9c00640 1 -- 172.21.15.18:0/3543611946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4109350 msgr2=0x7f66f41a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.710+0000 7f66f9c00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4109350 0x7f66f41a0bf0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f66e0036040 tx=0x7f66e0005e80 comp rx=0 tx=0).stop 2024-04-16T17:11:34.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.715+0000 7f66f9c00640 1 -- 172.21.15.18:0/3543611946 shutdown_connections 2024-04-16T17:11:34.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.715+0000 7f66f9c00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66d80a3da0 0x7f66d80a6260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.715+0000 7f66f9c00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66f419a820 0x7f66f419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.715+0000 7f66f9c00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f4109350 0x7f66f41a0bf0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.715+0000 7f66f9c00640 1 --2- 172.21.15.18:0/3543611946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f4104280 0x7f66f41a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.715+0000 7f66f9c00640 1 -- 172.21.15.18:0/3543611946 >> 172.21.15.18:0/3543611946 conn(0x7f66f40ffae0 msgr2=0x7f66f4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:34.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.715+0000 7f66f9c00640 1 -- 172.21.15.18:0/3543611946 shutdown_connections 2024-04-16T17:11:34.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.715+0000 7f66f9c00640 1 -- 172.21.15.18:0/3543611946 wait complete. 2024-04-16T17:11:34.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:11:34.735 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:11:34.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.907+0000 7f3a31600640 1 -- 172.21.15.18:0/2182003221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c102150 msgr2=0x7f3a2c102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.907+0000 7f3a31600640 1 --2- 172.21.15.18:0/2182003221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c102150 0x7f3a2c102530 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:11:34.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.908+0000 7f3a31600640 1 -- 172.21.15.18:0/2182003221 shutdown_connections 2024-04-16T17:11:34.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.908+0000 7f3a31600640 1 --2- 172.21.15.18:0/2182003221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2c107220 0x7f3a2c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.908+0000 7f3a31600640 1 --2- 172.21.15.18:0/2182003221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a2c102b00 0x7f3a2c106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.908+0000 7f3a31600640 1 --2- 172.21.15.18:0/2182003221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c102150 0x7f3a2c102530 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.908+0000 7f3a31600640 1 -- 172.21.15.18:0/2182003221 >> 172.21.15.18:0/2182003221 conn(0x7f3a2c0fd9f0 msgr2=0x7f3a2c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:34.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.908+0000 7f3a31600640 1 -- 172.21.15.18:0/2182003221 shutdown_connections 2024-04-16T17:11:34.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.908+0000 7f3a31600640 1 -- 172.21.15.18:0/2182003221 wait complete. 2024-04-16T17:11:34.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.921+0000 7f3a31600640 1 Processor -- start 2024-04-16T17:11:34.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.921+0000 7f3a31600640 1 -- start start 2024-04-16T17:11:34.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.921+0000 7f3a31600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2c102150 0x7f3a2c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.921+0000 7f3a31600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c107220 0x7f3a2c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.921+0000 7f3a31600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a2c19a850 0x7f3a2c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.921+0000 7f3a31600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c1122b0 con 0x7f3a2c107220 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.921+0000 7f3a31600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c112130 con 0x7f3a2c19a850 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.921+0000 7f3a31600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c112430 con 0x7f3a2c102150 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c107220 0x7f3a2c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c107220 0x7f3a2c1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38366/0 (socket says 172.21.15.18:38366) 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2b400640 1 -- 172.21.15.18:0/1074226007 learned_addr learned my addr 172.21.15.18:0/1074226007 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2be00640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2c102150 0x7f3a2c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a30c00640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a2c19a850 0x7f3a2c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2be00640 1 -- 172.21.15.18:0/1074226007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a2c19a850 msgr2=0x7f3a2c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2be00640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a2c19a850 0x7f3a2c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2be00640 1 -- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c107220 msgr2=0x7f3a2c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2be00640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c107220 0x7f3a2c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2be00640 1 -- 172.21.15.18:0/1074226007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c19b3e0 con 0x7f3a2c102150 2024-04-16T17:11:34.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.922+0000 7f3a2b400640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c107220 0x7f3a2c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:34.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.923+0000 7f3a2be00640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2c102150 0x7f3a2c1a0690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c036040 tx=0x7f3a1c02cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:34.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.923+0000 7f3a28c00640 1 -- 172.21.15.18:0/1074226007 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c041070 con 0x7f3a2c102150 2024-04-16T17:11:34.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.923+0000 7f3a28c00640 1 -- 172.21.15.18:0/1074226007 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c004450 con 0x7f3a2c102150 2024-04-16T17:11:34.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.923+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2c19b670 con 0x7f3a2c102150 2024-04-16T17:11:34.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.923+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2c1a77d0 con 0x7f3a2c102150 2024-04-16T17:11:34.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.923+0000 7f3a28c00640 1 -- 172.21.15.18:0/1074226007 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c003cd0 con 0x7f3a2c102150 2024-04-16T17:11:34.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.924+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a0c005290 con 0x7f3a2c102150 2024-04-16T17:11:34.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.927+0000 7f3a28c00640 1 -- 172.21.15.18:0/1074226007 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a1c044070 con 0x7f3a2c102150 2024-04-16T17:11:34.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.930+0000 7f3a28c00640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a100a3d10 0x7f3a100a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:34.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.930+0000 7f3a28c00640 1 -- 172.21.15.18:0/1074226007 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f3a1c0f2650 con 0x7f3a2c102150 2024-04-16T17:11:34.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.930+0000 7f3a2b400640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a100a3d10 0x7f3a100a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:34.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.930+0000 7f3a28c00640 1 -- 172.21.15.18:0/1074226007 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a1c0be990 con 0x7f3a2c102150 2024-04-16T17:11:34.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:34.931+0000 7f3a2b400640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a100a3d10 0x7f3a100a61d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0fe0a0 tx=0x7f3a18005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.261+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a0c005580 con 0x7f3a2c102150 2024-04-16T17:11:35.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.262+0000 7f3a28c00640 1 -- 172.21.15.18:0/1074226007 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11854 (secure 0 0 0) 0x7f3a1c02cd10 con 0x7f3a2c102150 2024-04-16T17:11:35.263 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:11:35.263 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:10:18.339186+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27119":{"gid":27119,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/1162334131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1162334131},{"type":"v1","addr":"172.21.15.22:6837","nonce":1162334131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:11:35.263 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 26 2024-04-16T17:11:35.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a100a3d10 msgr2=0x7f3a100a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a100a3d10 0x7f3a100a61d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0fe0a0 tx=0x7f3a18005f90 comp rx=0 tx=0).stop 2024-04-16T17:11:35.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2c102150 msgr2=0x7f3a2c1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2c102150 0x7f3a2c1a0690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c036040 tx=0x7f3a1c02cb20 comp rx=0 tx=0).stop 2024-04-16T17:11:35.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 shutdown_connections 2024-04-16T17:11:35.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a100a3d10 0x7f3a100a61d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a2c19a850 0x7f3a2c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a2c107220 0x7f3a2c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 --2- 172.21.15.18:0/1074226007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2c102150 0x7f3a2c1a0690 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 >> 172.21.15.18:0/1074226007 conn(0x7f3a2c0fd9f0 msgr2=0x7f3a2c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 shutdown_connections 2024-04-16T17:11:35.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.265+0000 7f3a31600640 1 -- 172.21.15.18:0/1074226007 wait complete. 2024-04-16T17:11:35.285 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:11:35.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 -- 172.21.15.18:0/2033263968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4074cf0 msgr2=0x7f70b410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 --2- 172.21.15.18:0/2033263968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4074cf0 0x7f70b410e320 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009900 tx=0x7f70a002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 -- 172.21.15.18:0/2033263968 shutdown_connections 2024-04-16T17:11:35.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 --2- 172.21.15.18:0/2033263968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b410e860 0x7f70b4110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 --2- 172.21.15.18:0/2033263968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4074cf0 0x7f70b410e320 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 --2- 172.21.15.18:0/2033263968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b40743d0 0x7f70b40747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 -- 172.21.15.18:0/2033263968 >> 172.21.15.18:0/2033263968 conn(0x7f70b40fdab0 msgr2=0x7f70b40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 -- 172.21.15.18:0/2033263968 shutdown_connections 2024-04-16T17:11:35.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.505+0000 7f70baa00640 1 -- 172.21.15.18:0/2033263968 wait complete. 2024-04-16T17:11:35.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.507+0000 7f70baa00640 1 Processor -- start 2024-04-16T17:11:35.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.507+0000 7f70baa00640 1 -- start start 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.507+0000 7f70baa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b40743d0 0x7f70b41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.507+0000 7f70baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4074cf0 0x7f70b41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.507+0000 7f70baa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b410e860 0x7f70b419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70baa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4114630 con 0x7f70b40743d0 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70baa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41144b0 con 0x7f70b410e860 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70baa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41147b0 con 0x7f70b4074cf0 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b40743d0 0x7f70b41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b410e860 0x7f70b419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b410e860 0x7f70b419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54190/0 (socket says 172.21.15.18:54190) 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b8a00640 1 -- 172.21.15.18:0/2870183173 learned_addr learned my addr 172.21.15.18:0/2870183173 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b3400640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4074cf0 0x7f70b41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b8a00640 1 -- 172.21.15.18:0/2870183173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4074cf0 msgr2=0x7f70b41a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b8a00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4074cf0 0x7f70b41a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b8a00640 1 -- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b40743d0 msgr2=0x7f70b41a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b8a00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b40743d0 0x7f70b41a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.508+0000 7f70b8a00640 1 -- 172.21.15.18:0/2870183173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b419b130 con 0x7f70b410e860 2024-04-16T17:11:35.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.509+0000 7f70b8a00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b410e860 0x7f70b419a9b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00eb10 tx=0x7f70ac00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.509+0000 7f70b0c00640 1 -- 172.21.15.18:0/2870183173 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac004b20 con 0x7f70b410e860 2024-04-16T17:11:35.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.509+0000 7f70b0c00640 1 -- 172.21.15.18:0/2870183173 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ac004cc0 con 0x7f70b410e860 2024-04-16T17:11:35.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.509+0000 7f70baa00640 1 -- 172.21.15.18:0/2870183173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b419b420 con 0x7f70b410e860 2024-04-16T17:11:35.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.509+0000 7f70b0c00640 1 -- 172.21.15.18:0/2870183173 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac010790 con 0x7f70b410e860 2024-04-16T17:11:35.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.509+0000 7f70baa00640 1 -- 172.21.15.18:0/2870183173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b41a7670 con 0x7f70b410e860 2024-04-16T17:11:35.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.512+0000 7f70b0c00640 1 -- 172.21.15.18:0/2870183173 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f70ac005020 con 0x7f70b410e860 2024-04-16T17:11:35.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.514+0000 7f70bd7cc740 1 Processor -- start 2024-04-16T17:11:35.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.514+0000 7f70b0c00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3da0 0x7f70980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.514+0000 7f70bd7cc740 1 -- start start 2024-04-16T17:11:35.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.514+0000 7f70b0c00640 1 -- 172.21.15.18:0/2870183173 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f70ac0d12b0 con 0x7f70b410e860 2024-04-16T17:11:35.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70b3400640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3da0 0x7f70980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 0x565418d06940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d11e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x565418c63020 con 0x565418d06560 2024-04-16T17:11:35.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565418c09460 con 0x565418d06e80 2024-04-16T17:11:35.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565418c5efe0 con 0x565418d0fa60 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70b3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70b3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54202/0 (socket says 172.21.15.18:54202) 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70b3400640 1 -- 172.21.15.18:0/3634298375 learned_addr learned my addr 172.21.15.18:0/3634298375 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70b8a00640 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d11e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.515+0000 7f70b3e00640 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 0x565418d06940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70b8a00640 1 -- 172.21.15.18:0/3634298375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 msgr2=0x565418d0f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70b8a00640 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70b8a00640 1 -- 172.21.15.18:0/3634298375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 msgr2=0x565418d06940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70b8a00640 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 0x565418d06940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70b8a00640 1 -- 172.21.15.18:0/3634298375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565418c11040 con 0x565418d0fa60 2024-04-16T17:11:35.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70b8a00640 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d11e50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0ce590 tx=0x7f70ac09c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be3fdbb22be4a970 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70b3400640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3da0 0x7f70980a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000c00 tx=0x7f70a00041a0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70a9200640 1 -- 172.21.15.18:0/3634298375 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0b6070 con 0x565418d0fa60 2024-04-16T17:11:35.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70a9200640 1 -- 172.21.15.18:0/3634298375 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ac0a1190 con 0x565418d0fa60 2024-04-16T17:11:35.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.516+0000 7f70a9200640 1 -- 172.21.15.18:0/3634298375 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0a55c0 con 0x565418d0fa60 2024-04-16T17:11:35.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.517+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3634298375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 msgr2=0x565418d11e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.517+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d11e50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0ce590 tx=0x7f70ac09c800 comp rx=0 tx=0).stop 2024-04-16T17:11:35.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.517+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3634298375 shutdown_connections 2024-04-16T17:11:35.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.517+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d11e50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.517+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.517+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3634298375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 0x565418d06940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.517+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3634298375 >> 172.21.15.18:0/3634298375 conn(0x565418d00c90 msgr2=0x565418d01070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.518+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3634298375 shutdown_connections 2024-04-16T17:11:35.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.518+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3634298375 wait complete. 2024-04-16T17:11:35.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.518+0000 7f70bd7cc740 1 -- start start 2024-04-16T17:11:35.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.518+0000 7f70bd7cc740 1 Processor -- start 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.519+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 0x565418d0caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.519+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.519+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d0d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.519+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x565418d127f0 con 0x565418d06560 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.519+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565418c1c830 con 0x565418d06e80 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.519+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565418d12970 con 0x565418d0fa60 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.519+0000 7f70bd7cc740 10 client.0 _ll_register_callbacks cb 0x565418c24150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d0d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d0d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37938/0 (socket says 172.21.15.18:37938) 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 -- 172.21.15.18:0/3328423805 learned_addr learned my addr 172.21.15.18:0/3328423805 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b3400640 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b3e00640 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 0x565418d0caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 -- 172.21.15.18:0/3328423805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 msgr2=0x565418d0d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 -- 172.21.15.18:0/3328423805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 msgr2=0x565418d0caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 0x565418d0caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 -- 172.21.15.18:0/3328423805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565418c20920 con 0x565418d0fa60 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b3400640 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f70b8a00640 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d0d570 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0997b0 tx=0x7f70ac0b6430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80e1165b6f83570b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.520+0000 7f709d600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.521+0000 7f709d600640 1 -- 172.21.15.18:0/3328423805 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0ba070 con 0x565418d0fa60 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.521+0000 7f709d600640 1 -- 172.21.15.18:0/3328423805 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f70ac0a57d0 con 0x565418d0fa60 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.521+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3328423805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70ac0a1190 con 0x565418d0fa60 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.521+0000 7f709d600640 1 -- 172.21.15.18:0/3328423805 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70ac09c230 con 0x565418d0fa60 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.521+0000 7f70bd7cc740 10 client.27370 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.521+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3328423805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565418bee7f0 con 0x565418d0fa60 2024-04-16T17:11:35.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.521+0000 7f709d600640 1 -- 172.21.15.18:0/3328423805 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ac0a1190 con 0x565418d0fa60 2024-04-16T17:11:35.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.522+0000 7f709d600640 1 -- 172.21.15.18:0/3328423805 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f70ac0cece0 con 0x565418d0fa60 2024-04-16T17:11:35.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.522+0000 7f70bd7cc740 10 client.27370 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.522+0000 7f70bd7cc740 10 client.27370 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.522+0000 7f70bd7cc740 20 client.27370 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.522+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x565418d1fc70 0x565418d22060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.522+0000 7f70bd7cc740 4 client.27370 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.522+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3328423805 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565418c70a40 con 0x565418d1fc70 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.523+0000 7f70b3e00640 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x565418d1fc70 0x565418d22060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.523+0000 7f709d600640 10 client.27370 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.523+0000 7f70b3e00640 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x565418d1fc70 0x565418d22060 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1485db106e00f53 server_cookie=3582b7ce5bd88761 in_seq=0 out_seq=0 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.524+0000 7f709d600640 1 -- 172.21.15.18:0/3328423805 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565418c70a40 con 0x565418d1fc70 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.524+0000 7f709d600640 10 client.27370 handle_command_reply: tid=0 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.524+0000 7f70bd7cc740 1 client.27370 shutdown 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3328423805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 msgr2=0x565418d0d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d0d570 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0997b0 tx=0x7f70ac0b6430 comp rx=0 tx=0).stop 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3328423805 shutdown_connections 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x565418d1fc70 0x565418d22060 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d0fa60 0x565418d0d570 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f709d600640 0 client.27370 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:11:35.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/3328423805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06560 0x565418d0caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3328423805 >> 172.21.15.18:0/3328423805 conn(0x565418d00c90 msgr2=0x565418d029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3328423805 shutdown_connections 2024-04-16T17:11:35.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 1 -- 172.21.15.18:0/3328423805 wait complete. 2024-04-16T17:11:35.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.525+0000 7f70bd7cc740 20 client.27370 trim_cache size 0 max 16384 2024-04-16T17:11:35.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.554+0000 7f70bd7cc740 1 Processor -- start 2024-04-16T17:11:35.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.554+0000 7f70bd7cc740 1 -- start start 2024-04-16T17:11:35.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d0f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d0fa60 0x565418d25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41afe60 con 0x565418d0fa60 2024-04-16T17:11:35.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41ad730 con 0x565418d06e80 2024-04-16T17:11:35.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41ab5c0 con 0x565418d06560 2024-04-16T17:11:35.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70b8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d0fa60 0x565418d25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70b8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d0fa60 0x565418d25080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38412/0 (socket says 172.21.15.18:38412) 2024-04-16T17:11:35.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.555+0000 7f70b8a00640 1 -- 172.21.15.18:0/2817128067 learned_addr learned my addr 172.21.15.18:0/2817128067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.556+0000 7f70b3e00640 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d0f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.556+0000 7f70b3400640 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.556+0000 7f70b8a00640 1 -- 172.21.15.18:0/2817128067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 msgr2=0x565418d0f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.556+0000 7f70b8a00640 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d0f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.556+0000 7f70b8a00640 1 -- 172.21.15.18:0/2817128067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 msgr2=0x565418d0d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.556+0000 7f70b8a00640 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.556+0000 7f70b8a00640 1 -- 172.21.15.18:0/2817128067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654189753f0 con 0x565418d0fa60 2024-04-16T17:11:35.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.557+0000 7f70b8a00640 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d0fa60 0x565418d25080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0d1120 tx=0x7f70ac004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2223dc0bb7f82424 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.558+0000 7f709fe00640 1 -- 172.21.15.18:0/2817128067 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0ba070 con 0x565418d0fa60 2024-04-16T17:11:35.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.558+0000 7f709fe00640 1 -- 172.21.15.18:0/2817128067 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ac0a57d0 con 0x565418d0fa60 2024-04-16T17:11:35.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.558+0000 7f709fe00640 1 -- 172.21.15.18:0/2817128067 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac005510 con 0x565418d0fa60 2024-04-16T17:11:35.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.559+0000 7f70bd7cc740 1 -- 172.21.15.18:0/2817128067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d0fa60 msgr2=0x565418d25080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.559+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d0fa60 0x565418d25080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0d1120 tx=0x7f70ac004ec0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.561+0000 7f70bd7cc740 1 -- 172.21.15.18:0/2817128067 shutdown_connections 2024-04-16T17:11:35.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.561+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d0fa60 0x565418d25080 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.561+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d06e80 0x565418d0d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.561+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/2817128067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d0f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.561+0000 7f70bd7cc740 1 -- 172.21.15.18:0/2817128067 >> 172.21.15.18:0/2817128067 conn(0x565418d00c90 msgr2=0x565418d02970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.561+0000 7f70bd7cc740 1 -- 172.21.15.18:0/2817128067 shutdown_connections 2024-04-16T17:11:35.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.561+0000 7f70bd7cc740 1 -- 172.21.15.18:0/2817128067 wait complete. 2024-04-16T17:11:35.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.562+0000 7f70bd7cc740 1 -- start start 2024-04-16T17:11:35.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.564+0000 7f70bd7cc740 1 Processor -- start 2024-04-16T17:11:35.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.565+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.565+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06e80 0x565418d22600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.565+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.566+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d25270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37958/0 (socket says 172.21.15.18:37958) 2024-04-16T17:11:35.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.566+0000 7f70bd7cc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d0fa60 0x565418d22b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.566+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41abc10 con 0x565418d06e80 2024-04-16T17:11:35.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.566+0000 7f70bd7cc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41ad060 con 0x565418d0fa60 2024-04-16T17:11:35.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.566+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41ac2c0 con 0x565418d06560 2024-04-16T17:11:35.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.566+0000 7f70b3e00640 1 -- 172.21.15.18:0/946973501 learned_addr learned my addr 172.21.15.18:0/946973501 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.566+0000 7f70bd7cc740 10 client.0 _ll_register_callbacks cb 0x7f70ac0b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.567+0000 7f70b3e00640 1 -- 172.21.15.18:0/946973501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d0fa60 msgr2=0x565418d22b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.567+0000 7f70b8a00640 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d0fa60 0x565418d22b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.567+0000 7f70b3e00640 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d0fa60 0x565418d22b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.567+0000 7f70b3e00640 1 -- 172.21.15.18:0/946973501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06e80 msgr2=0x565418d22600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.567+0000 7f70b3e00640 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06e80 0x565418d22600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.567+0000 7f70b3e00640 1 -- 172.21.15.18:0/946973501 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565418a3f7d0 con 0x565418d06560 2024-04-16T17:11:35.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.568+0000 7f70b8a00640 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d0fa60 0x565418d22b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.568+0000 7f70b3e00640 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d25270 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004700 tx=0x7f70a4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6f97b540e3c6b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.568+0000 7f709d600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.568+0000 7f709d600640 1 -- 172.21.15.18:0/946973501 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4004250 con 0x565418d06560 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.568+0000 7f709d600640 1 -- 172.21.15.18:0/946973501 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f70a40043f0 con 0x565418d06560 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.568+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70ac0a57d0 con 0x565418d06560 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.569+0000 7f709d600640 1 -- 172.21.15.18:0/946973501 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70a4030050 con 0x565418d06560 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.569+0000 7f70bd7cc740 10 client.27376 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.569+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565418d85ad0 con 0x565418d06560 2024-04-16T17:11:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.569+0000 7f709d600640 1 -- 172.21.15.18:0/946973501 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ac0a57d0 con 0x565418d06560 2024-04-16T17:11:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.570+0000 7f709d600640 1 -- 172.21.15.18:0/946973501 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f70a401f310 con 0x565418d06560 2024-04-16T17:11:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.570+0000 7f70bd7cc740 10 client.27376 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.570+0000 7f70bd7cc740 10 client.27376 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:11:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.570+0000 7f70bd7cc740 20 client.27376 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.570+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x565418d86250 0x565418d886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.570+0000 7f70bd7cc740 4 client.27376 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.570+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565418c6a8b0 con 0x565418d86250 2024-04-16T17:11:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.571+0000 7f70b3400640 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x565418d86250 0x565418d886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.571+0000 7f709d600640 10 client.27376 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:11:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.572+0000 7f70b3400640 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x565418d86250 0x565418d886b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2945afc517174b94 server_cookie=8d108be03e5e6afd in_seq=0 out_seq=0 2024-04-16T17:11:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.572+0000 7f709d600640 1 -- 172.21.15.18:0/946973501 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565418c6a8b0 con 0x565418d86250 2024-04-16T17:11:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.572+0000 7f709d600640 10 client.27376 handle_command_reply: tid=0 2024-04-16T17:11:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.572+0000 7f70bd7cc740 1 client.27376 shutdown 2024-04-16T17:11:35.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.573+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 msgr2=0x565418d25270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.573+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d25270 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004700 tx=0x7f70a4009f90 comp rx=0 tx=0).stop 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.574+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 shutdown_connections 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.574+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x565418d86250 0x565418d886b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.574+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565418d0fa60 0x565418d22b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.574+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565418d06e80 0x565418d22600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.574+0000 7f70bd7cc740 1 --2- 172.21.15.18:0/946973501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565418d06560 0x565418d25270 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.574+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 >> 172.21.15.18:0/946973501 conn(0x565418d00c90 msgr2=0x565418cf6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.574+0000 7f709d600640 0 client.27376 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:11:35.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.574+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 shutdown_connections 2024-04-16T17:11:35.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.575+0000 7f70bd7cc740 1 -- 172.21.15.18:0/946973501 wait complete. 2024-04-16T17:11:35.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.575+0000 7f70bd7cc740 20 client.27376 trim_cache size 0 max 16384 2024-04-16T17:11:35.576 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:11:35.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.577+0000 7f70baa00640 1 -- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3da0 msgr2=0x7f70980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.577+0000 7f70baa00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3da0 0x7f70980a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000c00 tx=0x7f70a00041a0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.577+0000 7f70baa00640 1 -- 172.21.15.18:0/2870183173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b410e860 msgr2=0x7f70b419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.577+0000 7f70baa00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b410e860 0x7f70b419a9b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00eb10 tx=0x7f70ac00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.578+0000 7f70baa00640 1 -- 172.21.15.18:0/2870183173 shutdown_connections 2024-04-16T17:11:35.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.578+0000 7f70baa00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3da0 0x7f70980a6260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.578+0000 7f70baa00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b410e860 0x7f70b419a9b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.578+0000 7f70baa00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4074cf0 0x7f70b41a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.578+0000 7f70baa00640 1 --2- 172.21.15.18:0/2870183173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b40743d0 0x7f70b41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.578+0000 7f70baa00640 1 -- 172.21.15.18:0/2870183173 >> 172.21.15.18:0/2870183173 conn(0x7f70b40fdab0 msgr2=0x7f70b40ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.578+0000 7f70baa00640 1 -- 172.21.15.18:0/2870183173 shutdown_connections 2024-04-16T17:11:35.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.579+0000 7f70baa00640 1 -- 172.21.15.18:0/2870183173 wait complete. 2024-04-16T17:11:35.590 INFO:tasks.workunit.client.0.smithi018.stdout: 1 259280 3.94 MB/sec execute 228 sec latency 76.999 ms 2024-04-16T17:11:35.599 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:11:35.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:35 smithi022 ceph-mon[36023]: pgmap v211: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:11:35.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:35 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1074226007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:35.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:35 smithi043 ceph-mon[36901]: pgmap v211: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:11:35.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:35 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1074226007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 -- 172.21.15.18:0/499388696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074d50 msgr2=0x7f44e410e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 --2- 172.21.15.18:0/499388696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074d50 0x7f44e410e100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009900 tx=0x7f44d002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 -- 172.21.15.18:0/499388696 shutdown_connections 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 --2- 172.21.15.18:0/499388696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e410e640 0x7f44e4110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 --2- 172.21.15.18:0/499388696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074d50 0x7f44e410e100 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 --2- 172.21.15.18:0/499388696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44e4074430 0x7f44e4074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 -- 172.21.15.18:0/499388696 >> 172.21.15.18:0/499388696 conn(0x7f44e40fd950 msgr2=0x7f44e40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 -- 172.21.15.18:0/499388696 shutdown_connections 2024-04-16T17:11:35.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.796+0000 7f44e3200640 1 -- 172.21.15.18:0/499388696 wait complete. 2024-04-16T17:11:35.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e3200640 1 Processor -- start 2024-04-16T17:11:35.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e3200640 1 -- start start 2024-04-16T17:11:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074430 0x7f44e41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e3200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44e4074d50 0x7f44e41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e410e640 0x7f44e419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e3200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41143d0 con 0x7f44e4074d50 2024-04-16T17:11:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4114250 con 0x7f44e4074430 2024-04-16T17:11:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4114550 con 0x7f44e410e640 2024-04-16T17:11:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074430 0x7f44e41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074430 0x7f44e41a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54236/0 (socket says 172.21.15.18:54236) 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.798+0000 7f44e1e00640 1 -- 172.21.15.18:0/1639841900 learned_addr learned my addr 172.21.15.18:0/1639841900 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e1400640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44e4074d50 0x7f44e41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e2800640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e410e640 0x7f44e419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e1e00640 1 -- 172.21.15.18:0/1639841900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e410e640 msgr2=0x7f44e419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e1e00640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e410e640 0x7f44e419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e1e00640 1 -- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44e4074d50 msgr2=0x7f44e41a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e1e00640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44e4074d50 0x7f44e41a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e1e00640 1 -- 172.21.15.18:0/1639841900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e419b070 con 0x7f44e4074430 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e2800640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e410e640 0x7f44e419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e1e00640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074430 0x7f44e41a06a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f44d400e9f0 tx=0x7f44d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44daa00640 1 -- 172.21.15.18:0/1639841900 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d400cd80 con 0x7f44e4074430 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44e3200640 1 -- 172.21.15.18:0/1639841900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e419b360 con 0x7f44e4074430 2024-04-16T17:11:35.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.799+0000 7f44daa00640 1 -- 172.21.15.18:0/1639841900 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d4010910 con 0x7f44e4074430 2024-04-16T17:11:35.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.800+0000 7f44daa00640 1 -- 172.21.15.18:0/1639841900 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40188f0 con 0x7f44e4074430 2024-04-16T17:11:35.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.800+0000 7f44e3200640 1 -- 172.21.15.18:0/1639841900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e41a7620 con 0x7f44e4074430 2024-04-16T17:11:35.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.803+0000 7f44ebd9c740 1 Processor -- start 2024-04-16T17:11:35.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.803+0000 7f44daa00640 1 -- 172.21.15.18:0/1639841900 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44d4010ac0 con 0x7f44e4074430 2024-04-16T17:11:35.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.803+0000 7f44ebd9c740 1 -- start start 2024-04-16T17:11:35.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e868f0 0x55d584e86cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584e986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e98c20 0x55d584e9b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d584dec020 con 0x55d584e872a0 2024-04-16T17:11:35.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d584d92460 con 0x55d584e868f0 2024-04-16T17:11:35.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d584de7fe0 con 0x55d584e98c20 2024-04-16T17:11:35.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44e2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e868f0 0x55d584e86cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44e1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584e986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44e2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e868f0 0x55d584e86cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54248/0 (socket says 172.21.15.18:54248) 2024-04-16T17:11:35.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44e2800640 1 -- 172.21.15.18:0/772821680 learned_addr learned my addr 172.21.15.18:0/772821680 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.804+0000 7f44e1400640 1 --2- 172.21.15.18:0/772821680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e98c20 0x55d584e9b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.805+0000 7f44e2800640 1 -- 172.21.15.18:0/772821680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e98c20 msgr2=0x55d584e9b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.805+0000 7f44e2800640 1 --2- 172.21.15.18:0/772821680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e98c20 0x55d584e9b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.805+0000 7f44e2800640 1 -- 172.21.15.18:0/772821680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 msgr2=0x55d584e986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.805+0000 7f44e2800640 1 --2- 172.21.15.18:0/772821680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584e986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.805+0000 7f44e2800640 1 -- 172.21.15.18:0/772821680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d584d9a040 con 0x55d584e868f0 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.805+0000 7f44daa00640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f44c80a3d20 0x7f44c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.805+0000 7f44e2800640 1 --2- 172.21.15.18:0/772821680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e868f0 0x55d584e86cd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009f10 tx=0x7f44dc00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd4a2515486bda22 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.805+0000 7f44daa00640 1 -- 172.21.15.18:0/1639841900 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f44d40d10b0 con 0x7f44e4074430 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.806+0000 7f44cf400640 1 -- 172.21.15.18:0/772821680 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc003b70 con 0x55d584e868f0 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.806+0000 7f44cf400640 1 -- 172.21.15.18:0/772821680 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc003d10 con 0x55d584e868f0 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.806+0000 7f44cf400640 1 -- 172.21.15.18:0/772821680 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc011a70 con 0x55d584e868f0 2024-04-16T17:11:35.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.806+0000 7f44e1e00640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f44c80a3d20 0x7f44c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.806+0000 7f44ebd9c740 1 -- 172.21.15.18:0/772821680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e868f0 msgr2=0x55d584e86cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.806+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/772821680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e868f0 0x55d584e86cd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009f10 tx=0x7f44dc00f730 comp rx=0 tx=0).stop 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 -- 172.21.15.18:0/772821680 shutdown_connections 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/772821680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e98c20 0x55d584e9b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/772821680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584e986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/772821680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e868f0 0x55d584e86cd0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 -- 172.21.15.18:0/772821680 >> 172.21.15.18:0/772821680 conn(0x55d584e81640 msgr2=0x55d584e81a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 -- 172.21.15.18:0/772821680 shutdown_connections 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 -- 172.21.15.18:0/772821680 wait complete. 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44e1e00640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f44c80a3d20 0x7f44c80a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44d4099040 tx=0x7f44d40d4b60 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 -- start start 2024-04-16T17:11:35.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.807+0000 7f44ebd9c740 1 Processor -- start 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e868f0 0x55d584e95c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e872a0 0x55d584e96190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d584e9b9e0 con 0x55d584e868f0 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d584da5830 con 0x55d584e98c20 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44e2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e868f0 0x55d584e95c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d584e9bb60 con 0x55d584e872a0 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44e2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e868f0 0x55d584e95c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38446/0 (socket says 172.21.15.18:38446) 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44e1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e872a0 0x55d584e96190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44ebd9c740 10 client.0 _ll_register_callbacks cb 0x55d584dad150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:35.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.808+0000 7f44e2800640 1 -- 172.21.15.18:0/4256473518 learned_addr learned my addr 172.21.15.18:0/4256473518 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44e2800640 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44e1400640 1 -- 172.21.15.18:0/4256473518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 msgr2=0x55d584e966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44e1400640 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44e1400640 1 -- 172.21.15.18:0/4256473518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e868f0 msgr2=0x55d584e95c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44e1400640 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e868f0 0x55d584e95c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44e1400640 1 -- 172.21.15.18:0/4256473518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d584da9920 con 0x55d584e872a0 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44e2800640 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44e1400640 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e872a0 0x55d584e96190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d0005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435c049d0ef6f813 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.809+0000 7f44c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.810+0000 7f44c3e00640 1 -- 172.21.15.18:0/4256473518 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d00046c0 con 0x55d584e872a0 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.810+0000 7f44ebd9c740 1 -- 172.21.15.18:0/4256473518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44dc003d10 con 0x55d584e872a0 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.810+0000 7f44c3e00640 1 -- 172.21.15.18:0/4256473518 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f44d0004860 con 0x55d584e872a0 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.810+0000 7f44c3e00640 1 -- 172.21.15.18:0/4256473518 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44d0004a60 con 0x55d584e872a0 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.811+0000 7f44ebd9c740 10 client.27385 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:35.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.811+0000 7f44ebd9c740 1 -- 172.21.15.18:0/4256473518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d584d777f0 con 0x55d584e872a0 2024-04-16T17:11:35.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.811+0000 7f44c3e00640 1 -- 172.21.15.18:0/4256473518 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc003d10 con 0x55d584e872a0 2024-04-16T17:11:35.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.812+0000 7f44c3e00640 1 -- 172.21.15.18:0/4256473518 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f44d0046070 con 0x55d584e872a0 2024-04-16T17:11:35.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.812+0000 7f44ebd9c740 10 client.27385 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:35.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.812+0000 7f44ebd9c740 10 client.27385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:11:35.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.812+0000 7f44ebd9c740 20 client.27385 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:35.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.812+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d584e8bf20 0x55d584e8e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.812+0000 7f44ebd9c740 4 client.27385 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:35.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.812+0000 7f44ebd9c740 1 -- 172.21.15.18:0/4256473518 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d584df9a40 con 0x55d584e8bf20 2024-04-16T17:11:35.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.813+0000 7f44e2800640 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d584e8bf20 0x55d584e8e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.813+0000 7f44e2800640 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d584e8bf20 0x55d584e8e3a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e91456c3c84ae03a server_cookie=fa86c75462f9af0a in_seq=0 out_seq=0 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.813+0000 7f44c3e00640 10 client.27385 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.814+0000 7f44c3e00640 1 -- 172.21.15.18:0/4256473518 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d584df9a40 con 0x55d584e8bf20 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.814+0000 7f44c3e00640 10 client.27385 handle_command_reply: tid=0 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.814+0000 7f44ebd9c740 1 client.27385 shutdown 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.814+0000 7f44ebd9c740 1 -- 172.21.15.18:0/4256473518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e872a0 msgr2=0x55d584e96190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.814+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e872a0 0x55d584e96190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d0005d50 comp rx=0 tx=0).stop 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.814+0000 7f44ebd9c740 1 -- 172.21.15.18:0/4256473518 shutdown_connections 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d584e8bf20 0x55d584e8e3a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e872a0 0x55d584e96190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/4256473518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e868f0 0x55d584e95c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44ebd9c740 1 -- 172.21.15.18:0/4256473518 >> 172.21.15.18:0/4256473518 conn(0x55d584e81640 msgr2=0x55d584e83390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44c3e00640 0 client.27385 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:11:35.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44ebd9c740 1 -- 172.21.15.18:0/4256473518 shutdown_connections 2024-04-16T17:11:35.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44ebd9c740 1 -- 172.21.15.18:0/4256473518 wait complete. 2024-04-16T17:11:35.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.815+0000 7f44ebd9c740 20 client.27385 trim_cache size 0 max 16384 2024-04-16T17:11:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.831+0000 7f44ebd9c740 1 Processor -- start 2024-04-16T17:11:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.831+0000 7f44ebd9c740 1 -- start start 2024-04-16T17:11:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.831+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e98380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.831+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584ea4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.831+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e96320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.831+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41afbd0 con 0x55d584e872a0 2024-04-16T17:11:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.831+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41ad4a0 con 0x55d584e98c20 2024-04-16T17:11:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.831+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41ab330 con 0x55d584e868f0 2024-04-16T17:11:35.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584ea4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e96320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e96320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54254/0 (socket says 172.21.15.18:54254) 2024-04-16T17:11:35.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e2800640 1 -- 172.21.15.18:0/1966563955 learned_addr learned my addr 172.21.15.18:0/1966563955 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e2800640 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e98380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e1400640 1 -- 172.21.15.18:0/1966563955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 msgr2=0x55d584e98380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e1400640 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e98380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e1400640 1 -- 172.21.15.18:0/1966563955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 msgr2=0x55d584e96320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e1400640 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e96320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e1400640 1 -- 172.21.15.18:0/1966563955 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d584e87040 con 0x55d584e872a0 2024-04-16T17:11:35.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.832+0000 7f44e2800640 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e96320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:35.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.833+0000 7f44e1400640 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584ea4910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44d0004f10 tx=0x7f44d002c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1091bc47dd799776 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.833+0000 7f44cea00640 1 -- 172.21.15.18:0/1966563955 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0005850 con 0x55d584e872a0 2024-04-16T17:11:35.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.834+0000 7f44cea00640 1 -- 172.21.15.18:0/1966563955 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d0005470 con 0x55d584e872a0 2024-04-16T17:11:35.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.834+0000 7f44cea00640 1 -- 172.21.15.18:0/1966563955 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d00035f0 con 0x55d584e872a0 2024-04-16T17:11:35.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.834+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1966563955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 msgr2=0x55d584ea4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.834+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584ea4910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44d0004f10 tx=0x7f44d002c3f0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1966563955 shutdown_connections 2024-04-16T17:11:35.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e98c20 0x55d584e96320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e872a0 0x55d584ea4910 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1966563955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e98380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1966563955 >> 172.21.15.18:0/1966563955 conn(0x55d584e81640 msgr2=0x55d584e95c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1966563955 shutdown_connections 2024-04-16T17:11:35.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1966563955 wait complete. 2024-04-16T17:11:35.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 -- start start 2024-04-16T17:11:35.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.835+0000 7f44ebd9c740 1 Processor -- start 2024-04-16T17:11:35.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e92970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e872a0 0x55d584e92eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44ebd9c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e98c20 0x55d584e933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41ab980 con 0x55d584e98c20 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41acdd0 con 0x55d584e872a0 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44ebd9c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41ac030 con 0x55d584e868f0 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44ebd9c740 10 client.0 _ll_register_callbacks cb 0x7f44d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44e2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e92970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44e2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e92970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38000/0 (socket says 172.21.15.18:38000) 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44e2800640 1 -- 172.21.15.18:0/1075532821 learned_addr learned my addr 172.21.15.18:0/1075532821 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44e1400640 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e872a0 0x55d584e92eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.836+0000 7f44e2800640 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e98c20 0x55d584e933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44e1400640 1 -- 172.21.15.18:0/1075532821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 msgr2=0x55d584e92970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44e1400640 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e92970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44e1400640 1 -- 172.21.15.18:0/1075532821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e98c20 msgr2=0x55d584e933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44e1400640 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e98c20 0x55d584e933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44e1400640 1 -- 172.21.15.18:0/1075532821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d584afe3f0 con 0x55d584e872a0 2024-04-16T17:11:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44e1400640 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e872a0 0x55d584e92eb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55d584e87840 tx=0x7f44d0005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78499f3c71fda80 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44c3e00640 1 -- 172.21.15.18:0/1075532821 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0042070 con 0x55d584e872a0 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1075532821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44d0005470 con 0x55d584e872a0 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44c3e00640 1 -- 172.21.15.18:0/1075532821 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f44d0005190 con 0x55d584e872a0 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44c3e00640 1 -- 172.21.15.18:0/1075532821 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44d0005850 con 0x55d584e872a0 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.837+0000 7f44ebd9c740 10 client.27458 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1075532821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d584bc87d0 con 0x55d584e872a0 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44c3e00640 1 -- 172.21.15.18:0/1075532821 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d0005470 con 0x55d584e872a0 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44c3e00640 1 -- 172.21.15.18:0/1075532821 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f44d0055030 con 0x55d584e872a0 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44ebd9c740 10 client.27458 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44ebd9c740 10 client.27458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44ebd9c740 20 client.27458 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d584f17c20 0x55d584f1a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44ebd9c740 4 client.27458 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1075532821 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d584df38b0 con 0x55d584f17c20 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.838+0000 7f44e2800640 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d584f17c20 0x55d584f1a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.839+0000 7f44c3e00640 10 client.27458 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.839+0000 7f44e2800640 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d584f17c20 0x55d584f1a040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77bceb80492097b6 server_cookie=28d207ccc6b493c8 in_seq=0 out_seq=0 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.839+0000 7f44c3e00640 1 -- 172.21.15.18:0/1075532821 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d584df38b0 con 0x55d584f17c20 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.839+0000 7f44c3e00640 10 client.27458 handle_command_reply: tid=0 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.839+0000 7f44ebd9c740 1 client.27458 shutdown 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.839+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1075532821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e872a0 msgr2=0x55d584e92eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.839+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e872a0 0x55d584e92eb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55d584e87840 tx=0x7f44d0005620 comp rx=0 tx=0).stop 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1075532821 shutdown_connections 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d584f17c20 0x55d584f1a040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d584e98c20 0x55d584e933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d584e872a0 0x55d584e92eb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 1 --2- 172.21.15.18:0/1075532821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d584e868f0 0x55d584e92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44c3e00640 0 client.27458 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:11:35.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1075532821 >> 172.21.15.18:0/1075532821 conn(0x55d584e81640 msgr2=0x55d584e80700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1075532821 shutdown_connections 2024-04-16T17:11:35.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 1 -- 172.21.15.18:0/1075532821 wait complete. 2024-04-16T17:11:35.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.840+0000 7f44ebd9c740 20 client.27458 trim_cache size 0 max 16384 2024-04-16T17:11:35.843 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:11:35.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 -- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f44c80a3d20 msgr2=0x7f44c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f44c80a3d20 0x7f44c80a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44d4099040 tx=0x7f44d40d4b60 comp rx=0 tx=0).stop 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 -- 172.21.15.18:0/1639841900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074430 msgr2=0x7f44e41a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074430 0x7f44e41a06a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f44d400e9f0 tx=0x7f44d400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 -- 172.21.15.18:0/1639841900 shutdown_connections 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f44c80a3d20 0x7f44c80a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e410e640 0x7f44e419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f44e4074d50 0x7f44e41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 --2- 172.21.15.18:0/1639841900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4074430 0x7f44e41a06a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 -- 172.21.15.18:0/1639841900 >> 172.21.15.18:0/1639841900 conn(0x7f44e40fd950 msgr2=0x7f44e40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 -- 172.21.15.18:0/1639841900 shutdown_connections 2024-04-16T17:11:35.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:35.841+0000 7f44e3200640 1 -- 172.21.15.18:0/1639841900 wait complete. 2024-04-16T17:11:35.860 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:11:36.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.059+0000 7fb32a000640 1 -- 172.21.15.18:0/3098663178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324102070 msgr2=0x7fb324102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.060+0000 7fb32a000640 1 --2- 172.21.15.18:0/3098663178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324102070 0x7fb324102450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb318009860 tx=0x7fb31802bb70 comp rx=0 tx=0).stop 2024-04-16T17:11:36.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.060+0000 7fb32a000640 1 -- 172.21.15.18:0/3098663178 shutdown_connections 2024-04-16T17:11:36.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.060+0000 7fb32a000640 1 --2- 172.21.15.18:0/3098663178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb324107140 0x7fb32410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.060+0000 7fb32a000640 1 --2- 172.21.15.18:0/3098663178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb324102a20 0x7fb324106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.060+0000 7fb32a000640 1 --2- 172.21.15.18:0/3098663178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324102070 0x7fb324102450 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.060+0000 7fb32a000640 1 -- 172.21.15.18:0/3098663178 >> 172.21.15.18:0/3098663178 conn(0x7fb3240fd910 msgr2=0x7fb3240ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:36.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.060+0000 7fb32a000640 1 -- 172.21.15.18:0/3098663178 shutdown_connections 2024-04-16T17:11:36.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.060+0000 7fb32a000640 1 -- 172.21.15.18:0/3098663178 wait complete. 2024-04-16T17:11:36.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.061+0000 7fb32a000640 1 Processor -- start 2024-04-16T17:11:36.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.061+0000 7fb32a000640 1 -- start start 2024-04-16T17:11:36.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb32a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb324102070 0x7fb3241a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb32a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324107140 0x7fb3241a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb32a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb32419a750 0x7fb32419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb32a000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3241121d0 con 0x7fb324102070 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb32a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb324112050 con 0x7fb32419a750 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb32a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb324112350 con 0x7fb324107140 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb328c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb324102070 0x7fb3241a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb328c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb324102070 0x7fb3241a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38498/0 (socket says 172.21.15.18:38498) 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb328c00640 1 -- 172.21.15.18:0/1545672047 learned_addr learned my addr 172.21.15.18:0/1545672047 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb323e00640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324107140 0x7fb3241a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.062+0000 7fb329600640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb32419a750 0x7fb32419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.063+0000 7fb328c00640 1 -- 172.21.15.18:0/1545672047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324107140 msgr2=0x7fb3241a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.063+0000 7fb328c00640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324107140 0x7fb3241a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.063+0000 7fb328c00640 1 -- 172.21.15.18:0/1545672047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb32419a750 msgr2=0x7fb32419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.063+0000 7fb328c00640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb32419a750 0x7fb32419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.063+0000 7fb328c00640 1 -- 172.21.15.18:0/1545672047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32419b350 con 0x7fb324102070 2024-04-16T17:11:36.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.063+0000 7fb323e00640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324107140 0x7fb3241a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:36.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.063+0000 7fb328c00640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb324102070 0x7fb3241a0690 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb318037040 tx=0x7fb3180045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:36.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.063+0000 7fb321600640 1 -- 172.21.15.18:0/1545672047 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318041020 con 0x7fb324102070 2024-04-16T17:11:36.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.064+0000 7fb32a000640 1 -- 172.21.15.18:0/1545672047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32419b5e0 con 0x7fb324102070 2024-04-16T17:11:36.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.064+0000 7fb321600640 1 -- 172.21.15.18:0/1545672047 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31802c9f0 con 0x7fb324102070 2024-04-16T17:11:36.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.064+0000 7fb321600640 1 -- 172.21.15.18:0/1545672047 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318039040 con 0x7fb324102070 2024-04-16T17:11:36.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.064+0000 7fb32a000640 1 -- 172.21.15.18:0/1545672047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3241a7770 con 0x7fb324102070 2024-04-16T17:11:36.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.067+0000 7fb32e3e1740 1 Processor -- start 2024-04-16T17:11:36.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb32e3e1740 1 -- start start 2024-04-16T17:11:36.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb321600640 1 -- 172.21.15.18:0/1545672047 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb318044020 con 0x7fb324102070 2024-04-16T17:11:36.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd340772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34077830 0x55fd3407fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34080410 0x55fd34082800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fd33fdc020 con 0x55fd34080410 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fd33f82460 con 0x55fd34076f10 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd33fd7fe0 con 0x55fd34077830 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb329600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd340772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb329600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd340772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54294/0 (socket says 172.21.15.18:54294) 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb329600640 1 -- 172.21.15.18:0/1223566784 learned_addr learned my addr 172.21.15.18:0/1223566784 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.068+0000 7fb328c00640 1 --2- 172.21.15.18:0/1223566784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34077830 0x55fd3407fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb329600640 1 -- 172.21.15.18:0/1223566784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34077830 msgr2=0x55fd3407fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb329600640 1 --2- 172.21.15.18:0/1223566784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34077830 0x55fd3407fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb329600640 1 -- 172.21.15.18:0/1223566784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34080410 msgr2=0x55fd34082800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb329600640 1 --2- 172.21.15.18:0/1223566784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34080410 0x55fd34082800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb329600640 1 -- 172.21.15.18:0/1223566784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd33f8a040 con 0x55fd34076f10 2024-04-16T17:11:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb329600640 1 --2- 172.21.15.18:0/1223566784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd340772f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009f50 tx=0x7fb31c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a03637d4111794 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb315c00640 1 -- 172.21.15.18:0/1223566784 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c004d20 con 0x55fd34076f10 2024-04-16T17:11:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb315c00640 1 -- 172.21.15.18:0/1223566784 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31c004050 con 0x55fd34076f10 2024-04-16T17:11:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.069+0000 7fb315c00640 1 -- 172.21.15.18:0/1223566784 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c005260 con 0x55fd34076f10 2024-04-16T17:11:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.070+0000 7fb321600640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb30c0a3da0 0x7fb30c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.070+0000 7fb321600640 1 -- 172.21.15.18:0/1545672047 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fb3180f2220 con 0x7fb324102070 2024-04-16T17:11:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.070+0000 7fb328c00640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb30c0a3da0 0x7fb30c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.070+0000 7fb32e3e1740 1 -- 172.21.15.18:0/1223566784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 msgr2=0x55fd340772f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.070+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/1223566784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd340772f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009f50 tx=0x7fb31c0118d0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.071+0000 7fb32e3e1740 1 -- 172.21.15.18:0/1223566784 shutdown_connections 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.071+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/1223566784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34080410 0x55fd34082800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.071+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/1223566784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34077830 0x55fd3407fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.071+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/1223566784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd340772f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.071+0000 7fb32e3e1740 1 -- 172.21.15.18:0/1223566784 >> 172.21.15.18:0/1223566784 conn(0x55fd34071640 msgr2=0x55fd34071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.071+0000 7fb32e3e1740 1 -- 172.21.15.18:0/1223566784 shutdown_connections 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.071+0000 7fb32e3e1740 1 -- 172.21.15.18:0/1223566784 wait complete. 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.071+0000 7fb328c00640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb30c0a3da0 0x7fb30c0a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3180ef720 tx=0x7fb318005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:36.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.072+0000 7fb32e3e1740 1 -- start start 2024-04-16T17:11:36.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.072+0000 7fb32e3e1740 1 Processor -- start 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd3407d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34077830 0x55fd3407d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd3407deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fd34083200 con 0x55fd34077830 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fd33f95830 con 0x55fd34076f10 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd34083380 con 0x55fd34080410 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb32e3e1740 10 client.0 _ll_register_callbacks cb 0x55fd33f9d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb329600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd3407d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb329600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd3407d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54302/0 (socket says 172.21.15.18:54302) 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb323e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34077830 0x55fd3407d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb329600640 1 -- 172.21.15.18:0/3736196529 learned_addr learned my addr 172.21.15.18:0/3736196529 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb329600640 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd3407deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb323e00640 1 -- 172.21.15.18:0/3736196529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 msgr2=0x55fd3407deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb323e00640 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd3407deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb323e00640 1 -- 172.21.15.18:0/3736196529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 msgr2=0x55fd3407d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb323e00640 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd3407d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb323e00640 1 -- 172.21.15.18:0/3736196529 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd33f99920 con 0x55fd34077830 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.073+0000 7fb329600640 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd3407d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.074+0000 7fb323e00640 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34077830 0x55fd3407d970 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb32419bfa0 tx=0x7fb310011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d7408d00449f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.074+0000 7fb30a000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.074+0000 7fb30a000640 1 -- 172.21.15.18:0/3736196529 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31001e060 con 0x55fd34077830 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.074+0000 7fb30a000640 1 -- 172.21.15.18:0/3736196529 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fb31000fb40 con 0x55fd34077830 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.074+0000 7fb32e3e1740 1 -- 172.21.15.18:0/3736196529 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb31c004050 con 0x55fd34077830 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.075+0000 7fb30a000640 1 -- 172.21.15.18:0/3736196529 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb31000fd80 con 0x55fd34077830 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.075+0000 7fb32e3e1740 10 client.17223 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.075+0000 7fb32e3e1740 1 -- 172.21.15.18:0/3736196529 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd33f677f0 con 0x55fd34077830 2024-04-16T17:11:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.075+0000 7fb30a000640 1 -- 172.21.15.18:0/3736196529 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31c004050 con 0x55fd34077830 2024-04-16T17:11:36.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.075+0000 7fb30a000640 1 -- 172.21.15.18:0/3736196529 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb31000e4d0 con 0x55fd34077830 2024-04-16T17:11:36.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb32e3e1740 10 client.17223 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:36.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb32e3e1740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:36.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb32e3e1740 20 client.17223 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fd34098f60 0x55fd3409b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb32e3e1740 4 client.17223 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb32e3e1740 1 -- 172.21.15.18:0/3736196529 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd33fe9a40 con 0x55fd34098f60 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb329600640 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fd34098f60 0x55fd3409b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb329600640 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fd34098f60 0x55fd3409b350 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa605d33afc6ff2 server_cookie=d97f56de084625c2 in_seq=0 out_seq=0 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.076+0000 7fb30a000640 10 client.17223 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.077+0000 7fb30a000640 1 -- 172.21.15.18:0/3736196529 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd33fe9a40 con 0x55fd34098f60 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.077+0000 7fb30a000640 10 client.17223 handle_command_reply: tid=0 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 client.17223 shutdown 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 -- 172.21.15.18:0/3736196529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34077830 msgr2=0x55fd3407d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34077830 0x55fd3407d970 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb32419bfa0 tx=0x7fb310011db0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 -- 172.21.15.18:0/3736196529 shutdown_connections 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fd34098f60 0x55fd3409b350 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd3407deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34077830 0x55fd3407d970 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/3736196529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34076f10 0x55fd3407d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb30a000640 0 client.17223 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:36.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 -- 172.21.15.18:0/3736196529 >> 172.21.15.18:0/3736196529 conn(0x55fd34071640 msgr2=0x55fd34073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:36.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 -- 172.21.15.18:0/3736196529 shutdown_connections 2024-04-16T17:11:36.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 1 -- 172.21.15.18:0/3736196529 wait complete. 2024-04-16T17:11:36.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.078+0000 7fb32e3e1740 20 client.17223 trim_cache size 0 max 16384 2024-04-16T17:11:36.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.108+0000 7fb32e3e1740 1 Processor -- start 2024-04-16T17:11:36.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.108+0000 7fb32e3e1740 1 -- start start 2024-04-16T17:11:36.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.108+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd3407fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.108+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd34094980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.108+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd340a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.108+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3241afbb0 con 0x55fd34076f10 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.108+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3241ad480 con 0x55fd34077830 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.108+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3241ab350 con 0x55fd34080410 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb323e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd34094980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb329600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd3407fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb323e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd34094980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54312/0 (socket says 172.21.15.18:54312) 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb323e00640 1 -- 172.21.15.18:0/2220857214 learned_addr learned my addr 172.21.15.18:0/2220857214 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb329600640 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd340a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb323e00640 1 -- 172.21.15.18:0/2220857214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 msgr2=0x55fd340a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:36.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb323e00640 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd340a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb323e00640 1 -- 172.21.15.18:0/2220857214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 msgr2=0x55fd3407fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:36.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb323e00640 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd3407fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb323e00640 1 -- 172.21.15.18:0/2220857214 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd33cee3f0 con 0x55fd34077830 2024-04-16T17:11:36.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.109+0000 7fb329600640 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd340a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.110+0000 7fb323e00640 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd34094980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb310000c00 tx=0x7fb310009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a28192bf4814b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.110+0000 7fb315200640 1 -- 172.21.15.18:0/2220857214 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310028080 con 0x55fd34077830 2024-04-16T17:11:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.110+0000 7fb315200640 1 -- 172.21.15.18:0/2220857214 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31000eae0 con 0x55fd34077830 2024-04-16T17:11:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.110+0000 7fb315200640 1 -- 172.21.15.18:0/2220857214 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3100052d0 con 0x55fd34077830 2024-04-16T17:11:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 -- 172.21.15.18:0/2220857214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 msgr2=0x55fd34094980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd34094980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb310000c00 tx=0x7fb310009790 comp rx=0 tx=0).stop 2024-04-16T17:11:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 -- 172.21.15.18:0/2220857214 shutdown_connections 2024-04-16T17:11:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd340a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd34094980 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/2220857214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd3407fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 -- 172.21.15.18:0/2220857214 >> 172.21.15.18:0/2220857214 conn(0x55fd34071640 msgr2=0x55fd34071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 -- 172.21.15.18:0/2220857214 shutdown_connections 2024-04-16T17:11:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.111+0000 7fb32e3e1740 1 -- 172.21.15.18:0/2220857214 wait complete. 2024-04-16T17:11:36.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.112+0000 7fb32e3e1740 1 -- start start 2024-04-16T17:11:36.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.112+0000 7fb32e3e1740 1 Processor -- start 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd34090fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd3408c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb32e3e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd3408c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb3241ab9a0 con 0x55fd34076f10 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3241acdb0 con 0x55fd34077830 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb32e3e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3241ac010 con 0x55fd34080410 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb32e3e1740 10 client.0 _ll_register_callbacks cb 0x7fb310009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb329600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd34090fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb323e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd3408c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb329600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd34090fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38532/0 (socket says 172.21.15.18:38532) 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb329600640 1 -- 172.21.15.18:0/429500863 learned_addr learned my addr 172.21.15.18:0/429500863 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.113+0000 7fb329600640 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd3408c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb323e00640 1 -- 172.21.15.18:0/429500863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 msgr2=0x55fd3408c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:36.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb323e00640 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd3408c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb323e00640 1 -- 172.21.15.18:0/429500863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 msgr2=0x55fd34090fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:36.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb323e00640 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd34090fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb323e00640 1 -- 172.21.15.18:0/429500863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd33db87d0 con 0x55fd34077830 2024-04-16T17:11:36.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb323e00640 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd3408c450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb31000e990 tx=0x7fb31000e710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c758ccc412524a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:36.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb30a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb30a000640 1 -- 172.21.15.18:0/429500863 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310028050 con 0x55fd34077830 2024-04-16T17:11:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb30a000640 1 -- 172.21.15.18:0/429500863 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fb310004020 con 0x55fd34077830 2024-04-16T17:11:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.114+0000 7fb32e3e1740 1 -- 172.21.15.18:0/429500863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb31000eae0 con 0x55fd34077830 2024-04-16T17:11:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb30a000640 1 -- 172.21.15.18:0/429500863 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb31001e040 con 0x55fd34077830 2024-04-16T17:11:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb32e3e1740 10 client.27476 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb32e3e1740 1 -- 172.21.15.18:0/429500863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd340ff070 con 0x55fd34077830 2024-04-16T17:11:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb30a000640 1 -- 172.21.15.18:0/429500863 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31000eae0 con 0x55fd34077830 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb30a000640 1 -- 172.21.15.18:0/429500863 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb31002c450 con 0x55fd34077830 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb32e3e1740 10 client.27476 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb32e3e1740 10 client.27476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb32e3e1740 20 client.27476 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fd340ff6d0 0x55fd34101af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb32e3e1740 4 client.27476 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.115+0000 7fb32e3e1740 1 -- 172.21.15.18:0/429500863 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd33fe38b0 con 0x55fd340ff6d0 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.116+0000 7fb329600640 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fd340ff6d0 0x55fd34101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.116+0000 7fb30a000640 10 client.27476 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:36.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.116+0000 7fb329600640 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fd340ff6d0 0x55fd34101af0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39305e904860053b server_cookie=bedc5581eb76153 in_seq=0 out_seq=0 2024-04-16T17:11:36.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.117+0000 7fb30a000640 1 -- 172.21.15.18:0/429500863 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd33fe38b0 con 0x55fd340ff6d0 2024-04-16T17:11:36.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.117+0000 7fb30a000640 10 client.27476 handle_command_reply: tid=0 2024-04-16T17:11:36.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.117+0000 7fb32e3e1740 1 client.27476 shutdown 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.117+0000 7fb32e3e1740 1 -- 172.21.15.18:0/429500863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 msgr2=0x55fd3408c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.117+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd3408c450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb31000e990 tx=0x7fb31000e710 comp rx=0 tx=0).stop 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 1 -- 172.21.15.18:0/429500863 shutdown_connections 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fd340ff6d0 0x55fd34101af0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd34080410 0x55fd3408c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd34077830 0x55fd3408c450 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 1 --2- 172.21.15.18:0/429500863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fd34076f10 0x55fd34090fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb30a000640 0 client.27476 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 1 -- 172.21.15.18:0/429500863 >> 172.21.15.18:0/429500863 conn(0x55fd34071640 msgr2=0x55fd3406fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 1 -- 172.21.15.18:0/429500863 shutdown_connections 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 1 -- 172.21.15.18:0/429500863 wait complete. 2024-04-16T17:11:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.118+0000 7fb32e3e1740 20 client.27476 trim_cache size 0 max 16384 2024-04-16T17:11:36.121 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:11:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 -- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb30c0a3da0 msgr2=0x7fb30c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb30c0a3da0 0x7fb30c0a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3180ef720 tx=0x7fb318005f90 comp rx=0 tx=0).stop 2024-04-16T17:11:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 -- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb324102070 msgr2=0x7fb3241a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb324102070 0x7fb3241a0690 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb318037040 tx=0x7fb3180045b0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 -- 172.21.15.18:0/1545672047 shutdown_connections 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb30c0a3da0 0x7fb30c0a6260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb32419a750 0x7fb32419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb324107140 0x7fb3241a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 --2- 172.21.15.18:0/1545672047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb324102070 0x7fb3241a0690 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.119+0000 7fb32a000640 1 -- 172.21.15.18:0/1545672047 >> 172.21.15.18:0/1545672047 conn(0x7fb3240fd910 msgr2=0x7fb3240ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.120+0000 7fb32a000640 1 -- 172.21.15.18:0/1545672047 shutdown_connections 2024-04-16T17:11:36.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:36.120+0000 7fb32a000640 1 -- 172.21.15.18:0/1545672047 wait complete. 2024-04-16T17:11:36.591 INFO:tasks.workunit.client.0.smithi018.stdout: 1 260102 3.93 MB/sec execute 229 sec latency 15.017 ms 2024-04-16T17:11:37.137 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:11:37.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:11:37.138 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:11:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 -- 172.21.15.18:0/3953141413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 msgr2=0x7fb95c104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 --2- 172.21.15.18:0/3953141413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 0x7fb95c104850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009860 tx=0x7fb94c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:11:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 -- 172.21.15.18:0/3953141413 shutdown_connections 2024-04-16T17:11:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 --2- 172.21.15.18:0/3953141413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb95c109540 0x7fb95c111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 --2- 172.21.15.18:0/3953141413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb95c104e20 0x7fb95c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 --2- 172.21.15.18:0/3953141413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 0x7fb95c104850 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 -- 172.21.15.18:0/3953141413 >> 172.21.15.18:0/3953141413 conn(0x7fb95c0ffcf0 msgr2=0x7fb95c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 -- 172.21.15.18:0/3953141413 shutdown_connections 2024-04-16T17:11:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.224+0000 7fb960c00640 1 -- 172.21.15.18:0/3953141413 wait complete. 2024-04-16T17:11:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.226+0000 7fb960c00640 1 Processor -- start 2024-04-16T17:11:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.226+0000 7fb960c00640 1 -- start start 2024-04-16T17:11:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.226+0000 7fb960c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 0x7fb95c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.226+0000 7fb960c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb95c109540 0x7fb95c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.226+0000 7fb960c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb95c19a990 0x7fb95c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.226+0000 7fb960c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1145d0 con 0x7fb95c104470 2024-04-16T17:11:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.226+0000 7fb960c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c114450 con 0x7fb95c19a990 2024-04-16T17:11:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.226+0000 7fb960c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c114750 con 0x7fb95c109540 2024-04-16T17:11:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 0x7fb95c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 0x7fb95c1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57966/0 (socket says 172.21.15.18:57966) 2024-04-16T17:11:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959e00640 1 -- 172.21.15.18:0/1687015801 learned_addr learned my addr 172.21.15.18:0/1687015801 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb95a800640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb95c19a990 0x7fb95c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959e00640 1 -- 172.21.15.18:0/1687015801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb95c109540 msgr2=0x7fb95c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959400640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb95c109540 0x7fb95c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959e00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb95c109540 0x7fb95c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959e00640 1 -- 172.21.15.18:0/1687015801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb95c19a990 msgr2=0x7fb95c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959e00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb95c19a990 0x7fb95c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959e00640 1 -- 172.21.15.18:0/1687015801 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c19b4b0 con 0x7fb95c104470 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb959400640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb95c109540 0x7fb95c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.227+0000 7fb95a800640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb95c19a990 0x7fb95c19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.228+0000 7fb959e00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 0x7fb95c1a0790 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb94c037040 tx=0x7fb94c0323b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.228+0000 7fb952a00640 1 -- 172.21.15.18:0/1687015801 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c041020 con 0x7fb95c104470 2024-04-16T17:11:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.228+0000 7fb960c00640 1 -- 172.21.15.18:0/1687015801 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c19b740 con 0x7fb95c104470 2024-04-16T17:11:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.228+0000 7fb952a00640 1 -- 172.21.15.18:0/1687015801 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb94c02c9f0 con 0x7fb95c104470 2024-04-16T17:11:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.228+0000 7fb952a00640 1 -- 172.21.15.18:0/1687015801 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c039040 con 0x7fb95c104470 2024-04-16T17:11:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.228+0000 7fb960c00640 1 -- 172.21.15.18:0/1687015801 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c1a7940 con 0x7fb95c104470 2024-04-16T17:11:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.232+0000 7fb952a00640 1 -- 172.21.15.18:0/1687015801 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb94c044020 con 0x7fb95c104470 2024-04-16T17:11:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.233+0000 7fb9639c5740 1 Processor -- start 2024-04-16T17:11:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.233+0000 7fb9639c5740 1 -- start start 2024-04-16T17:11:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fd610 0x55ef6d7fc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d7fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d808210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb952a00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9400a3e10 0x7fb9400a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb9639c5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ef6d761780 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb9639c5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ef6d707f40 con 0x55ef6d805e20 2024-04-16T17:11:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb9639c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef6d75d740 con 0x55ef6d7fd610 2024-04-16T17:11:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb952a00640 1 -- 172.21.15.18:0/1687015801 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fb94c0f2470 con 0x7fb95c104470 2024-04-16T17:11:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb95a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d808210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb95a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d808210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49782/0 (socket says 172.21.15.18:49782) 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb95a800640 1 -- 172.21.15.18:0/3061420322 learned_addr learned my addr 172.21.15.18:0/3061420322 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.234+0000 7fb95a800640 1 --2- 172.21.15.18:0/3061420322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fd610 0x55ef6d7fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.235+0000 7fb959400640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9400a3e10 0x7fb9400a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.235+0000 7fb95a800640 1 -- 172.21.15.18:0/3061420322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fd610 msgr2=0x55ef6d7fc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.235+0000 7fb95a800640 1 --2- 172.21.15.18:0/3061420322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fd610 0x55ef6d7fc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.235+0000 7fb95a800640 1 -- 172.21.15.18:0/3061420322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 msgr2=0x55ef6d7fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.235+0000 7fb95a800640 1 --2- 172.21.15.18:0/3061420322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d7fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.235+0000 7fb95a800640 1 -- 172.21.15.18:0/3061420322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6d474210 con 0x55ef6d805e20 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.235+0000 7fb95a800640 1 --2- 172.21.15.18:0/3061420322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d808210 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb954012ad0 tx=0x7fb954009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ae27bdb6af79ee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.235+0000 7fb959400640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9400a3e10 0x7fb9400a62d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb948005fd0 tx=0x7fb948005a20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.236+0000 7fb947000640 1 -- 172.21.15.18:0/3061420322 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954003970 con 0x55ef6d805e20 2024-04-16T17:11:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.236+0000 7fb947000640 1 -- 172.21.15.18:0/3061420322 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb954010890 con 0x55ef6d805e20 2024-04-16T17:11:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.236+0000 7fb947000640 1 -- 172.21.15.18:0/3061420322 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954004670 con 0x55ef6d805e20 2024-04-16T17:11:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 -- 172.21.15.18:0/3061420322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 msgr2=0x55ef6d808210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3061420322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d808210 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb954012ad0 tx=0x7fb954009a70 comp rx=0 tx=0).stop 2024-04-16T17:11:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 -- 172.21.15.18:0/3061420322 shutdown_connections 2024-04-16T17:11:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3061420322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d808210 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3061420322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d7fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3061420322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fd610 0x55ef6d7fc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 -- 172.21.15.18:0/3061420322 >> 172.21.15.18:0/3061420322 conn(0x55ef6d7975d0 msgr2=0x55ef6d7f7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 -- 172.21.15.18:0/3061420322 shutdown_connections 2024-04-16T17:11:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.237+0000 7fb9639c5740 1 -- 172.21.15.18:0/3061420322 wait complete. 2024-04-16T17:11:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.238+0000 7fb9639c5740 1 -- start start 2024-04-16T17:11:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.238+0000 7fb9639c5740 1 Processor -- start 2024-04-16T17:11:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d802e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 0x55ef6d803390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 0x55ef6d8038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb9639c5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ef6d808b50 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb9639c5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ef6d71b390 con 0x55ef6d7fd610 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb9639c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef6d808cd0 con 0x55ef6d805e20 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb9639c5740 10 client.0 _ll_register_callbacks cb 0x55ef6d704f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb959e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d802e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb959e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d802e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57982/0 (socket says 172.21.15.18:57982) 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.239+0000 7fb959e00640 1 -- 172.21.15.18:0/3103441450 learned_addr learned my addr 172.21.15.18:0/3103441450 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.240+0000 7fb959e00640 1 -- 172.21.15.18:0/3103441450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 msgr2=0x55ef6d8038d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.240+0000 7fb959e00640 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 0x55ef6d8038d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.240+0000 7fb959e00640 1 -- 172.21.15.18:0/3103441450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 msgr2=0x55ef6d803390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.240+0000 7fb959e00640 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 0x55ef6d803390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.240+0000 7fb959e00640 1 -- 172.21.15.18:0/3103441450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6d540620 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.241+0000 7fb959e00640 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d802e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009830 tx=0x7fb94c0bdad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c061dc75301ceea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.241+0000 7fb93b400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.241+0000 7fb93b400640 1 -- 172.21.15.18:0/3103441450 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0be0b0 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.241+0000 7fb93b400640 1 -- 172.21.15.18:0/3103441450 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fb94c0f2170 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.241+0000 7fb9639c5740 1 -- 172.21.15.18:0/3103441450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb954010890 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.242+0000 7fb93b400640 1 -- 172.21.15.18:0/3103441450 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb94c0c2460 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.242+0000 7fb9639c5740 10 client.17238 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.242+0000 7fb9639c5740 1 -- 172.21.15.18:0/3103441450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef6d80bdc0 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.242+0000 7fb93b400640 1 -- 172.21.15.18:0/3103441450 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb954010890 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.243+0000 7fb93b400640 1 -- 172.21.15.18:0/3103441450 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb94c03c030 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.243+0000 7fb9639c5740 10 client.17238 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.243+0000 7fb9639c5740 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.243+0000 7fb9639c5740 20 client.17238 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.243+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ef6d80d280 0x55ef6d80f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.243+0000 7fb9639c5740 4 client.17238 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.243+0000 7fb9639c5740 1 -- 172.21.15.18:0/3103441450 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef6d76f1a0 con 0x55ef6d80d280 2024-04-16T17:11:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.243+0000 7fb959400640 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ef6d80d280 0x55ef6d80f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.244+0000 7fb959400640 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ef6d80d280 0x55ef6d80f6f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d1e42d6c72a871 server_cookie=a86ab00dc6f08658 in_seq=0 out_seq=0 2024-04-16T17:11:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.244+0000 7fb93b400640 10 client.17238 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb93b400640 1 -- 172.21.15.18:0/3103441450 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef6d76f1a0 con 0x55ef6d80d280 2024-04-16T17:11:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb93b400640 10 client.17238 handle_command_reply: tid=0 2024-04-16T17:11:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 client.17238 shutdown 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 -- 172.21.15.18:0/3103441450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 msgr2=0x55ef6d802e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d802e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009830 tx=0x7fb94c0bdad0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 -- 172.21.15.18:0/3103441450 shutdown_connections 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ef6d80d280 0x55ef6d80f6f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 0x55ef6d8038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 0x55ef6d803390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3103441450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d802e50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb9639c5740 1 -- 172.21.15.18:0/3103441450 >> 172.21.15.18:0/3103441450 conn(0x55ef6d7975d0 msgr2=0x55ef6d7f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.245+0000 7fb93b400640 0 client.17238 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.246+0000 7fb9639c5740 1 -- 172.21.15.18:0/3103441450 shutdown_connections 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.246+0000 7fb9639c5740 1 -- 172.21.15.18:0/3103441450 wait complete. 2024-04-16T17:11:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.246+0000 7fb9639c5740 20 client.17238 trim_cache size 0 max 16384 2024-04-16T17:11:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.278+0000 7fb9639c5740 1 Processor -- start 2024-04-16T17:11:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.278+0000 7fb9639c5740 1 -- start start 2024-04-16T17:11:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.278+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d805580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.278+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 0x55ef6d801fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.278+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 0x55ef6d803520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.278+0000 7fb9639c5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1afe00 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.278+0000 7fb9639c5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1ad6d0 con 0x55ef6d7fd610 2024-04-16T17:11:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.278+0000 7fb9639c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1ab540 con 0x55ef6d805e20 2024-04-16T17:11:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.279+0000 7fb95a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 0x55ef6d803520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.279+0000 7fb959e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d805580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.279+0000 7fb95a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 0x55ef6d803520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44390/0 (socket says 172.21.15.18:44390) 2024-04-16T17:11:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.279+0000 7fb95a800640 1 -- 172.21.15.18:0/3068781104 learned_addr learned my addr 172.21.15.18:0/3068781104 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.279+0000 7fb959e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d805580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57986/0 (socket says 172.21.15.18:57986) 2024-04-16T17:11:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.279+0000 7fb959400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 0x55ef6d801fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.280+0000 7fb959400640 1 -- 172.21.15.18:0/3068781104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 msgr2=0x55ef6d803520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.280+0000 7fb959400640 1 --2- 172.21.15.18:0/3068781104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 0x55ef6d803520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.280+0000 7fb959400640 1 -- 172.21.15.18:0/3068781104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 msgr2=0x55ef6d805580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.280+0000 7fb959400640 1 --2- 172.21.15.18:0/3068781104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d805580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.280+0000 7fb959400640 1 -- 172.21.15.18:0/3068781104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6d808580 con 0x55ef6d7fd610 2024-04-16T17:11:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.280+0000 7fb959400640 1 --2- 172.21.15.18:0/3068781104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 0x55ef6d801fb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55ef6d8026b0 tx=0x7fb948011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1597f39f01e42c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.281+0000 7fb946600640 1 -- 172.21.15.18:0/3068781104 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948004020 con 0x55ef6d7fd610 2024-04-16T17:11:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.281+0000 7fb946600640 1 -- 172.21.15.18:0/3068781104 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb948017070 con 0x55ef6d7fd610 2024-04-16T17:11:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.281+0000 7fb946600640 1 -- 172.21.15.18:0/3068781104 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948004020 con 0x55ef6d7fd610 2024-04-16T17:11:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 -- 172.21.15.18:0/3068781104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 msgr2=0x55ef6d801fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3068781104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 0x55ef6d801fb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55ef6d8026b0 tx=0x7fb948011000 comp rx=0 tx=0).stop 2024-04-16T17:11:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 -- 172.21.15.18:0/3068781104 shutdown_connections 2024-04-16T17:11:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3068781104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d805e20 0x55ef6d803520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3068781104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d7fd610 0x55ef6d801fb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 --2- 172.21.15.18:0/3068781104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d805580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 -- 172.21.15.18:0/3068781104 >> 172.21.15.18:0/3068781104 conn(0x55ef6d7975d0 msgr2=0x55ef6d802e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 -- 172.21.15.18:0/3068781104 shutdown_connections 2024-04-16T17:11:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.282+0000 7fb9639c5740 1 -- 172.21.15.18:0/3068781104 wait complete. 2024-04-16T17:11:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.283+0000 7fb9639c5740 1 -- start start 2024-04-16T17:11:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.283+0000 7fb9639c5740 1 Processor -- start 2024-04-16T17:11:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d815050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fd610 0x55ef6d811d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb9639c5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d812290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb9639c5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1abb90 con 0x55ef6d7fd610 2024-04-16T17:11:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb9639c5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1ad000 con 0x55ef6d805e20 2024-04-16T17:11:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb9639c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1ac260 con 0x55ef6d7fc8a0 2024-04-16T17:11:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb9639c5740 10 client.0 _ll_register_callbacks cb 0x7fb94c0bdef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb95a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d812290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb95a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d812290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49806/0 (socket says 172.21.15.18:49806) 2024-04-16T17:11:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb95a800640 1 -- 172.21.15.18:0/2792411293 learned_addr learned my addr 172.21.15.18:0/2792411293 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.284+0000 7fb959400640 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fd610 0x55ef6d811d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.285+0000 7fb959e00640 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d815050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.285+0000 7fb959400640 1 -- 172.21.15.18:0/2792411293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fc8a0 msgr2=0x55ef6d815050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.285+0000 7fb959400640 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d815050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.285+0000 7fb959400640 1 -- 172.21.15.18:0/2792411293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 msgr2=0x55ef6d812290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.285+0000 7fb959400640 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d812290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.285+0000 7fb959400640 1 -- 172.21.15.18:0/2792411293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6d8127d0 con 0x55ef6d7fd610 2024-04-16T17:11:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.285+0000 7fb95a800640 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d812290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.286+0000 7fb959400640 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fd610 0x55ef6d811d50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55ef6d805c20 tx=0x7fb9480045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572d64116d554194 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.286+0000 7fb93b400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.287+0000 7fb93b400640 1 -- 172.21.15.18:0/2792411293 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948005240 con 0x55ef6d7fd610 2024-04-16T17:11:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.287+0000 7fb93b400640 1 -- 172.21.15.18:0/2792411293 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fb9480053e0 con 0x55ef6d7fd610 2024-04-16T17:11:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.287+0000 7fb9639c5740 1 -- 172.21.15.18:0/2792411293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb948017070 con 0x55ef6d7fd610 2024-04-16T17:11:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.287+0000 7fb93b400640 1 -- 172.21.15.18:0/2792411293 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb94803c050 con 0x55ef6d7fd610 2024-04-16T17:11:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.287+0000 7fb9639c5740 10 client.17244 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.287+0000 7fb9639c5740 1 -- 172.21.15.18:0/2792411293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef6d88de40 con 0x55ef6d7fd610 2024-04-16T17:11:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.288+0000 7fb93b400640 1 -- 172.21.15.18:0/2792411293 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb948017070 con 0x55ef6d7fd610 2024-04-16T17:11:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.288+0000 7fb93b400640 1 -- 172.21.15.18:0/2792411293 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb94800f040 con 0x55ef6d7fd610 2024-04-16T17:11:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.288+0000 7fb9639c5740 10 client.17244 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.288+0000 7fb9639c5740 10 client.17244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.288+0000 7fb9639c5740 20 client.17244 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.288+0000 7fb9639c5740 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ef6d88e4f0 0x55ef6d8908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.288+0000 7fb9639c5740 4 client.17244 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:11:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.288+0000 7fb9639c5740 1 -- 172.21.15.18:0/2792411293 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef6d769010 con 0x55ef6d88e4f0 2024-04-16T17:11:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.289+0000 7fb959e00640 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ef6d88e4f0 0x55ef6d8908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.289+0000 7fb959e00640 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ef6d88e4f0 0x55ef6d8908e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17945ae13b640386 server_cookie=5c7f30d552c2c3b1 in_seq=0 out_seq=0 2024-04-16T17:11:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.289+0000 7fb93b400640 10 client.17244 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.357 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:31 smithi018 ceph-mon[31776]: pgmap v209: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-04-16T17:11:37.357 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:33 smithi018 ceph-mon[31776]: pgmap v210: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:11:37.357 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:35 smithi018 ceph-mon[31776]: pgmap v211: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:11:37.357 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:35 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1074226007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:37.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.409+0000 7fc0d2a00640 1 -- 172.21.15.18:0/106424506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 msgr2=0x7fc0cc104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.409+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/106424506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 0x7fc0cc104660 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009900 tx=0x7fc0b802bc50 comp rx=0 tx=0).stop 2024-04-16T17:11:37.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.409+0000 7fc0d2a00640 1 -- 172.21.15.18:0/106424506 shutdown_connections 2024-04-16T17:11:37.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.409+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/106424506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc109350 0x7fc0cc110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.409+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/106424506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0cc104c30 0x7fc0cc108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.409+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/106424506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 0x7fc0cc104660 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.409+0000 7fc0d2a00640 1 -- 172.21.15.18:0/106424506 >> 172.21.15.18:0/106424506 conn(0x7fc0cc0ffae0 msgr2=0x7fc0cc101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.412+0000 7fc0d2a00640 1 -- 172.21.15.18:0/106424506 shutdown_connections 2024-04-16T17:11:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.412+0000 7fc0d2a00640 1 -- 172.21.15.18:0/106424506 wait complete. 2024-04-16T17:11:37.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.413+0000 7fc0d2a00640 1 Processor -- start 2024-04-16T17:11:37.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.414+0000 7fc0d2a00640 1 -- start start 2024-04-16T17:11:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.414+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 0x7fc0cc11a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.414+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0cc109350 0x7fc0cc110340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.414+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc110880 0x7fc0cc110d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.414+0000 7fc0d2a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc078180 con 0x7fc0cc109350 2024-04-16T17:11:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.414+0000 7fc0d2a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc078000 con 0x7fc0cc104280 2024-04-16T17:11:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.414+0000 7fc0d2a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc078300 con 0x7fc0cc110880 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc110880 0x7fc0cc110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 0x7fc0cc11a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc110880 0x7fc0cc110d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44424/0 (socket says 172.21.15.18:44424) 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d2000640 1 -- 172.21.15.18:0/1217843861 learned_addr learned my addr 172.21.15.18:0/1217843861 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0cc109350 0x7fc0cc110340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d1600640 1 -- 172.21.15.18:0/1217843861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc110880 msgr2=0x7fc0cc110d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d1600640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc110880 0x7fc0cc110d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d1600640 1 -- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0cc109350 msgr2=0x7fc0cc110340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d1600640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0cc109350 0x7fc0cc110340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.415+0000 7fc0d1600640 1 -- 172.21.15.18:0/1217843861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc116060 con 0x7fc0cc104280 2024-04-16T17:11:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.416+0000 7fc0d1600640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 0x7fc0cc11a530 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc0b802c190 tx=0x7fc0b8004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.416+0000 7fc0ca000640 1 -- 172.21.15.18:0/1217843861 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8041070 con 0x7fc0cc104280 2024-04-16T17:11:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.416+0000 7fc0ca000640 1 -- 172.21.15.18:0/1217843861 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b802ccb0 con 0x7fc0cc104280 2024-04-16T17:11:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.416+0000 7fc0ca000640 1 -- 172.21.15.18:0/1217843861 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8041070 con 0x7fc0cc104280 2024-04-16T17:11:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.416+0000 7fc0d2a00640 1 -- 172.21.15.18:0/1217843861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc1162f0 con 0x7fc0cc104280 2024-04-16T17:11:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.417+0000 7fc0d2a00640 1 -- 172.21.15.18:0/1217843861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc116730 con 0x7fc0cc104280 2024-04-16T17:11:37.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.424+0000 7fc0ca000640 1 -- 172.21.15.18:0/1217843861 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc0b8044070 con 0x7fc0cc104280 2024-04-16T17:11:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.426+0000 7fc0d6e11740 1 Processor -- start 2024-04-16T17:11:37.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.428+0000 7fc0d6e11740 1 -- start start 2024-04-16T17:11:37.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.428+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91e7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.428+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91efe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.429+0000 7fc0ca000640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0b40a3fa0 0x7fc0b40a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.429+0000 7fc0ca000640 1 -- 172.21.15.18:0/1217843861 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fc0b80f1fd0 con 0x7fc0cc104280 2024-04-16T17:11:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562ba914bcb0 con 0x562ba91f0390 2024-04-16T17:11:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562ba90f2370 con 0x562ba91e6e90 2024-04-16T17:11:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ba9147c70 con 0x562ba91e77b0 2024-04-16T17:11:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91efe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44434/0 (socket says 172.21.15.18:44434) 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d1600640 1 -- 172.21.15.18:0/1661758070 learned_addr learned my addr 172.21.15.18:0/1661758070 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0b40a3fa0 0x7fc0b40a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.430+0000 7fc0d2000640 1 --2- 172.21.15.18:0/1661758070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.431+0000 7fc0d1600640 1 -- 172.21.15.18:0/1661758070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 msgr2=0x562ba91e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.431+0000 7fc0d1600640 1 --2- 172.21.15.18:0/1661758070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91e7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.431+0000 7fc0d1600640 1 -- 172.21.15.18:0/1661758070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 msgr2=0x562ba91f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.431+0000 7fc0d1600640 1 --2- 172.21.15.18:0/1661758070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.431+0000 7fc0d1600640 1 -- 172.21.15.18:0/1661758070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba9111910 con 0x562ba91e77b0 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.432+0000 7fc0d1600640 1 --2- 172.21.15.18:0/1661758070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91efe50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80ba640 tx=0x7fc0b80bd7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56ba33865a58d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.432+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0b40a3fa0 0x7fc0b40a6460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005e10 tx=0x7fc0c0005da0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.433+0000 7fc0be600640 1 -- 172.21.15.18:0/1661758070 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b804b030 con 0x562ba91e77b0 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.434+0000 7fc0be600640 1 -- 172.21.15.18:0/1661758070 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b80bdc70 con 0x562ba91e77b0 2024-04-16T17:11:37.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.434+0000 7fc0be600640 1 -- 172.21.15.18:0/1661758070 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8003c90 con 0x562ba91e77b0 2024-04-16T17:11:37.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.438+0000 7fc0d6e11740 1 -- 172.21.15.18:0/1661758070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 msgr2=0x562ba91efe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.438+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/1661758070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91efe50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80ba640 tx=0x7fc0b80bd7b0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.438+0000 7fc0d6e11740 1 -- 172.21.15.18:0/1661758070 shutdown_connections 2024-04-16T17:11:37.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.438+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/1661758070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.438+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/1661758070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91efe50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.438+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/1661758070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.439+0000 7fc0d6e11740 1 -- 172.21.15.18:0/1661758070 >> 172.21.15.18:0/1661758070 conn(0x562ba91e15c0 msgr2=0x562ba91e19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.439+0000 7fc0d6e11740 1 -- 172.21.15.18:0/1661758070 shutdown_connections 2024-04-16T17:11:37.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.440+0000 7fc0d6e11740 1 -- 172.21.15.18:0/1661758070 wait complete. 2024-04-16T17:11:37.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.440+0000 7fc0d6e11740 1 -- start start 2024-04-16T17:11:37.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.441+0000 7fc0d6e11740 1 Processor -- start 2024-04-16T17:11:37.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.441+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e6e90 0x562ba91ed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.441+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e77b0 0x562ba91ed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.441+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91ede30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e6e90 0x562ba91ed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e6e90 0x562ba91ed3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44438/0 (socket says 172.21.15.18:44438) 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d1600640 1 -- 172.21.15.18:0/2870097645 learned_addr learned my addr 172.21.15.18:0/2870097645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562ba9147c70 con 0x562ba91f0390 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562ba9105730 con 0x562ba91e77b0 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ba91f2f40 con 0x562ba91e6e90 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e77b0 0x562ba91ed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d6e11740 10 client.0 _ll_register_callbacks cb 0x562ba90d41e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d1600640 1 -- 172.21.15.18:0/2870097645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e77b0 msgr2=0x562ba91ed8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d1600640 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e77b0 0x562ba91ed8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.442+0000 7fc0d1600640 1 -- 172.21.15.18:0/2870097645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 msgr2=0x562ba91ede30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.443+0000 7fc0d1600640 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.443+0000 7fc0d1600640 1 -- 172.21.15.18:0/2870097645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba8f267e0 con 0x562ba91e6e90 2024-04-16T17:11:37.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.443+0000 7fc0d1600640 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e6e90 0x562ba91ed3b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80ba0d0 tx=0x7fc0b80bd8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68162aa96e720e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.443+0000 7fc0b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:11:37.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.443+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2870097645 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b804b070 con 0x562ba91e6e90 2024-04-16T17:11:37.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.443+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2870097645 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fc0b80de0b0 con 0x562ba91e6e90 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.444+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0b80bdc70 con 0x562ba91e6e90 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.444+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2870097645 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0b80de2b0 con 0x562ba91e6e90 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.444+0000 7fc0d6e11740 10 client.27433 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.444+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ba8f86f70 con 0x562ba91e6e90 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.444+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2870097645 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b80bdc70 con 0x562ba91e6e90 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.444+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2870097645 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc0b80c6450 con 0x562ba91e6e90 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.445+0000 7fc0d6e11740 10 client.27433 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.445+0000 7fc0d6e11740 10 client.27433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.445+0000 7fc0d6e11740 20 client.27433 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:37.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.445+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ba92001e0 0x562ba92025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.445+0000 7fc0d6e11740 4 client.27433 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:37.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.445+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ba91596d0 con 0x562ba92001e0 2024-04-16T17:11:37.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.445+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ba92001e0 0x562ba92025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.447+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ba92001e0 0x562ba92025d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da79144c1c667434 server_cookie=bcdf8deffe43a64d in_seq=0 out_seq=0 2024-04-16T17:11:37.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.447+0000 7fc0b2a00640 10 client.27433 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.476+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2870097645 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ba91596d0 con 0x562ba92001e0 2024-04-16T17:11:37.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.476+0000 7fc0b2a00640 10 client.27433 handle_command_reply: tid=0 2024-04-16T17:11:37.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.476+0000 7fc0d6e11740 1 client.27433 shutdown 2024-04-16T17:11:37.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.477+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e6e90 msgr2=0x562ba91ed3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.477+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e6e90 0x562ba91ed3b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80ba0d0 tx=0x7fc0b80bd8e0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.478+0000 7fb93b400640 1 -- 172.21.15.18:0/2792411293 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef6d769010 con 0x55ef6d88e4f0 2024-04-16T17:11:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.478+0000 7fb93b400640 10 client.17244 handle_command_reply: tid=0 2024-04-16T17:11:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.478+0000 7fb9639c5740 1 client.17244 shutdown 2024-04-16T17:11:37.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 shutdown_connections 2024-04-16T17:11:37.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ba92001e0 0x562ba92025d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91ede30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e77b0 0x562ba91ed8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2870097645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e6e90 0x562ba91ed3b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 >> 172.21.15.18:0/2870097645 conn(0x562ba91e15c0 msgr2=0x562ba91e3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0b2a00640 0 client.27433 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fb9639c5740 1 -- 172.21.15.18:0/2792411293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fd610 msgr2=0x55ef6d811d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 shutdown_connections 2024-04-16T17:11:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fb9639c5740 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fd610 0x55ef6d811d50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55ef6d805c20 tx=0x7fb9480045b0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.479+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2870097645 wait complete. 2024-04-16T17:11:37.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.480+0000 7fc0d6e11740 20 client.27433 trim_cache size 0 max 16384 2024-04-16T17:11:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb9639c5740 1 -- 172.21.15.18:0/2792411293 shutdown_connections 2024-04-16T17:11:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb9639c5740 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ef6d88e4f0 0x55ef6d8908e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb9639c5740 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef6d805e20 0x55ef6d812290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb9639c5740 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef6d7fd610 0x55ef6d811d50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb9639c5740 1 --2- 172.21.15.18:0/2792411293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef6d7fc8a0 0x55ef6d815050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb9639c5740 1 -- 172.21.15.18:0/2792411293 >> 172.21.15.18:0/2792411293 conn(0x55ef6d7975d0 msgr2=0x55ef6d801e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb93b400640 0 client.17244 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb9639c5740 1 -- 172.21.15.18:0/2792411293 shutdown_connections 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.481+0000 7fb9639c5740 1 -- 172.21.15.18:0/2792411293 wait complete. 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.482+0000 7fb9639c5740 20 client.17244 trim_cache size 0 max 16384 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:11:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:11:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.485+0000 7fb960c00640 1 -- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9400a3e10 msgr2=0x7fb9400a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.485+0000 7fb960c00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9400a3e10 0x7fb9400a62d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb948005fd0 tx=0x7fb948005a20 comp rx=0 tx=0).stop 2024-04-16T17:11:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.485+0000 7fb960c00640 1 -- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 msgr2=0x7fb95c1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.485+0000 7fb960c00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 0x7fb95c1a0790 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb94c037040 tx=0x7fb94c0323b0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.486+0000 7fb960c00640 1 -- 172.21.15.18:0/1687015801 shutdown_connections 2024-04-16T17:11:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.486+0000 7fb960c00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb9400a3e10 0x7fb9400a62d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.486+0000 7fb960c00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb95c19a990 0x7fb95c19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.486+0000 7fb960c00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb95c109540 0x7fb95c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.486+0000 7fb960c00640 1 --2- 172.21.15.18:0/1687015801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb95c104470 0x7fb95c1a0790 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.486+0000 7fb960c00640 1 -- 172.21.15.18:0/1687015801 >> 172.21.15.18:0/1687015801 conn(0x7fb95c0ffcf0 msgr2=0x7fb95c101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.486+0000 7fb960c00640 1 -- 172.21.15.18:0/1687015801 shutdown_connections 2024-04-16T17:11:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:37.486+0000 7fb960c00640 1 -- 172.21.15.18:0/1687015801 wait complete. 2024-04-16T17:11:37.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.514+0000 7fc0d6e11740 1 Processor -- start 2024-04-16T17:11:37.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d6e11740 1 -- start start 2024-04-16T17:11:37.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91e77b0 0x562ba91e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91f0390 0x562ba9207d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0128b0 con 0x562ba91e77b0 2024-04-16T17:11:37.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac010180 con 0x562ba91e6e90 2024-04-16T17:11:37.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac00dff0 con 0x562ba91f0390 2024-04-16T17:11:37.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49842/0 (socket says 172.21.15.18:49842) 2024-04-16T17:11:37.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.515+0000 7fc0d1600640 1 -- 172.21.15.18:0/3822268281 learned_addr learned my addr 172.21.15.18:0/3822268281 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91e77b0 0x562ba91e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d2000640 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91f0390 0x562ba9207d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d1600640 1 -- 172.21.15.18:0/3822268281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91f0390 msgr2=0x562ba9207d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d1600640 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91f0390 0x562ba9207d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d1600640 1 -- 172.21.15.18:0/3822268281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91e77b0 msgr2=0x562ba91e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d1600640 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91e77b0 0x562ba91e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d1600640 1 -- 172.21.15.18:0/3822268281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba8e1e170 con 0x562ba91e6e90 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91e77b0 0x562ba91e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0d1600640 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91fbce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009900 tx=0x7fc0b80bd7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4397b6e72c745fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0bdc00640 1 -- 172.21.15.18:0/3822268281 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b804b070 con 0x562ba91e6e90 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0bdc00640 1 -- 172.21.15.18:0/3822268281 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b80de0f0 con 0x562ba91e6e90 2024-04-16T17:11:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.516+0000 7fc0bdc00640 1 -- 172.21.15.18:0/3822268281 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80bded0 con 0x562ba91e6e90 2024-04-16T17:11:37.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 -- 172.21.15.18:0/3822268281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 msgr2=0x562ba91fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91fbce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009900 tx=0x7fc0b80bd7b0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 -- 172.21.15.18:0/3822268281 shutdown_connections 2024-04-16T17:11:37.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91f0390 0x562ba9207d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91e77b0 0x562ba91e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/3822268281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91fbce0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 -- 172.21.15.18:0/3822268281 >> 172.21.15.18:0/3822268281 conn(0x562ba91e15c0 msgr2=0x562ba91dfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 -- 172.21.15.18:0/3822268281 shutdown_connections 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.517+0000 7fc0d6e11740 1 -- 172.21.15.18:0/3822268281 wait complete. 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.518+0000 7fc0d6e11740 1 -- start start 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.518+0000 7fc0d6e11740 1 Processor -- start 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.518+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91f8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d6e11740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac00e640 con 0x562ba91f0390 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac00fab0 con 0x562ba91e6e90 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d6e11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac00ed10 con 0x562ba91e77b0 2024-04-16T17:11:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d6e11740 10 client.0 _ll_register_callbacks cb 0x7fc0b80ba000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91f8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49858/0 (socket says 172.21.15.18:49858) 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d1600640 1 -- 172.21.15.18:0/2087550998 learned_addr learned my addr 172.21.15.18:0/2087550998 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d2000640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d1600640 1 -- 172.21.15.18:0/2087550998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 msgr2=0x562ba91f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d1600640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d1600640 1 -- 172.21.15.18:0/2087550998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 msgr2=0x562ba91f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d1600640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d1600640 1 -- 172.21.15.18:0/2087550998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba91f42f0 con 0x562ba91e6e90 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.519+0000 7fc0d2000640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0d1600640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91f8400 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80bd300 tx=0x7fc0b80ba2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548c1ead2434c254 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2087550998 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b804b070 con 0x562ba91e6e90 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2087550998 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7fc0b80be0d0 con 0x562ba91e6e90 2024-04-16T17:11:37.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2087550998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ba9153540 con 0x562ba91e6e90 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2087550998 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0b80bdc70 con 0x562ba91e6e90 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0d6e11740 10 client.27509 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2087550998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ba91f42f0 con 0x562ba91e6e90 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.520+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2087550998 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562ba9153540 con 0x562ba91e6e90 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2087550998 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc0b809eb40 con 0x562ba91e6e90 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0d6e11740 10 client.27509 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0d6e11740 10 client.27509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0d6e11740 20 client.27509 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ba92666f0 0x562ba9268ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0d6e11740 4 client.27509 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2087550998 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ba90cfaa0 con 0x562ba92666f0 2024-04-16T17:11:37.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ba92666f0 0x562ba9268ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0d0c00640 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ba92666f0 0x562ba9268ae0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fca73e4c9e74e server_cookie=ed07d92708677f8b in_seq=0 out_seq=0 2024-04-16T17:11:37.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.521+0000 7fc0b2a00640 10 client.27509 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.522+0000 7fc0b2a00640 1 -- 172.21.15.18:0/2087550998 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ba90cfaa0 con 0x562ba92666f0 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.522+0000 7fc0b2a00640 10 client.27509 handle_command_reply: tid=0 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.522+0000 7fc0d6e11740 1 client.27509 shutdown 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2087550998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 msgr2=0x562ba91f8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91f8400 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80bd300 tx=0x7fc0b80ba2e0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2087550998 shutdown_connections 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562ba92666f0 0x562ba9268ae0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ba91f0390 0x562ba91f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ba91e77b0 0x562ba91f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0b2a00640 0 client.27509 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 --2- 172.21.15.18:0/2087550998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ba91e6e90 0x562ba91f8400 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2087550998 >> 172.21.15.18:0/2087550998 conn(0x562ba91e15c0 msgr2=0x562ba91edc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2087550998 shutdown_connections 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 1 -- 172.21.15.18:0/2087550998 wait complete. 2024-04-16T17:11:37.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.523+0000 7fc0d6e11740 20 client.27509 trim_cache size 0 max 16384 2024-04-16T17:11:37.527 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:11:37.527 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:11:37.527 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "59e445fd-38ef-4ac4-94ab-45f8b9c016cb", 2024-04-16T17:11:37.527 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:11:37.527 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:11:37.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 -- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0b40a3fa0 msgr2=0x7fc0b40a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0b40a3fa0 0x7fc0b40a6460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005e10 tx=0x7fc0c0005da0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 -- 172.21.15.18:0/1217843861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 msgr2=0x7fc0cc11a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 0x7fc0cc11a530 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc0b802c190 tx=0x7fc0b8004430 comp rx=0 tx=0).stop 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 -- 172.21.15.18:0/1217843861 shutdown_connections 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0b40a3fa0 0x7fc0b40a6460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc110880 0x7fc0cc110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0cc109350 0x7fc0cc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 --2- 172.21.15.18:0/1217843861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0cc104280 0x7fc0cc11a530 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 -- 172.21.15.18:0/1217843861 >> 172.21.15.18:0/1217843861 conn(0x7fc0cc0ffae0 msgr2=0x7fc0cc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 -- 172.21.15.18:0/1217843861 shutdown_connections 2024-04-16T17:11:37.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.524+0000 7fc0d2a00640 1 -- 172.21.15.18:0/1217843861 wait complete. 2024-04-16T17:11:37.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:11:37.544 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:11:37.591 INFO:tasks.workunit.client.0.smithi018.stdout: 1 261346 3.94 MB/sec execute 230 sec latency 106.865 ms 2024-04-16T17:11:37.697 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:37 smithi022 ceph-mon[36023]: pgmap v212: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-04-16T17:11:37.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:37 smithi043 ceph-mon[36901]: pgmap v212: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-04-16T17:11:37.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.811+0000 7f5772a00640 1 -- 172.21.15.18:0/4221252398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c074cc0 msgr2=0x7f576c1128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.811+0000 7f5772a00640 1 --2- 172.21.15.18:0/4221252398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c074cc0 0x7f576c1128d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5758009900 tx=0x7f575802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.812+0000 7f5772a00640 1 -- 172.21.15.18:0/4221252398 shutdown_connections 2024-04-16T17:11:37.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.812+0000 7f5772a00640 1 --2- 172.21.15.18:0/4221252398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f576c112e10 0x7f576c115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.812+0000 7f5772a00640 1 --2- 172.21.15.18:0/4221252398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c074cc0 0x7f576c1128d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.812+0000 7f5772a00640 1 --2- 172.21.15.18:0/4221252398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0743a0 0x7f576c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.812+0000 7f5772a00640 1 -- 172.21.15.18:0/4221252398 >> 172.21.15.18:0/4221252398 conn(0x7f576c0fd950 msgr2=0x7f576c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.812+0000 7f5772a00640 1 -- 172.21.15.18:0/4221252398 shutdown_connections 2024-04-16T17:11:37.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.812+0000 7f5772a00640 1 -- 172.21.15.18:0/4221252398 wait complete. 2024-04-16T17:11:37.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.813+0000 7f5772a00640 1 Processor -- start 2024-04-16T17:11:37.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.813+0000 7f5772a00640 1 -- start start 2024-04-16T17:11:37.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.813+0000 7f5772a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c0743a0 0x7f576c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.813+0000 7f5772a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c074cc0 0x7f576c19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.813+0000 7f5772a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f576c112e10 0x7f576c1a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.813+0000 7f5772a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f576c118850 con 0x7f576c112e10 2024-04-16T17:11:37.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.813+0000 7f5772a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1186d0 con 0x7f576c0743a0 2024-04-16T17:11:37.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.813+0000 7f5772a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1189d0 con 0x7f576c074cc0 2024-04-16T17:11:37.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5771600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c0743a0 0x7f576c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5771600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c0743a0 0x7f576c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49866/0 (socket says 172.21.15.18:49866) 2024-04-16T17:11:37.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5772000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f576c112e10 0x7f576c1a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5771600640 1 -- 172.21.15.18:0/1940579884 learned_addr learned my addr 172.21.15.18:0/1940579884 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5770c00640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c074cc0 0x7f576c19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5771600640 1 -- 172.21.15.18:0/1940579884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c074cc0 msgr2=0x7f576c19f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5771600640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c074cc0 0x7f576c19f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5771600640 1 -- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f576c112e10 msgr2=0x7f576c1a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5771600640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f576c112e10 0x7f576c1a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.814+0000 7f5771600640 1 -- 172.21.15.18:0/1940579884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c1a3d80 con 0x7f576c0743a0 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.815+0000 7f5771600640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c0743a0 0x7f576c19ed90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f576000e9f0 tx=0x7f576000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.815+0000 7f576a000640 1 -- 172.21.15.18:0/1940579884 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576000cd80 con 0x7f576c0743a0 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.815+0000 7f576a000640 1 -- 172.21.15.18:0/1940579884 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5760005500 con 0x7f576c0743a0 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.815+0000 7f576a000640 1 -- 172.21.15.18:0/1940579884 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760010690 con 0x7f576c0743a0 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.815+0000 7f5772a00640 1 -- 172.21.15.18:0/1940579884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c1a4070 con 0x7f576c0743a0 2024-04-16T17:11:37.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.815+0000 7f5772a00640 1 -- 172.21.15.18:0/1940579884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c1ab900 con 0x7f576c0743a0 2024-04-16T17:11:37.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.818+0000 7f5776bfe740 1 Processor -- start 2024-04-16T17:11:37.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.818+0000 7f576a000640 1 -- 172.21.15.18:0/1940579884 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f57600046c0 con 0x7f576c0743a0 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5776bfe740 1 -- start start 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b9890 0x55e9441bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5776bfe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e944116120 con 0x55e9441b0cb0 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5776bfe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e9440bc340 con 0x55e9441b9890 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5776bfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9441120e0 con 0x55e9441b0390 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5771600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5772000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5772000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44480/0 (socket says 172.21.15.18:44480) 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.819+0000 7f5772000640 1 -- 172.21.15.18:0/3972195624 learned_addr learned my addr 172.21.15.18:0/3972195624 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.820+0000 7f5770c00640 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b9890 0x55e9441bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.820+0000 7f5772000640 1 -- 172.21.15.18:0/3972195624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b9890 msgr2=0x55e9441bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.820+0000 7f5772000640 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b9890 0x55e9441bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.820+0000 7f5772000640 1 -- 172.21.15.18:0/3972195624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 msgr2=0x55e9441b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.820+0000 7f5772000640 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.820+0000 7f5772000640 1 -- 172.21.15.18:0/3972195624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e943ef27d0 con 0x55e9441b0390 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.820+0000 7f5770c00640 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b9890 0x55e9441bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.820+0000 7f5772000640 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441b0770 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5764009f30 tx=0x7f57640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bcac56bcba3b361 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.821+0000 7f575e600640 1 -- 172.21.15.18:0/3972195624 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764004d20 con 0x55e9441b0390 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.821+0000 7f575e600640 1 -- 172.21.15.18:0/3972195624 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5764004050 con 0x55e9441b0390 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.821+0000 7f575e600640 1 -- 172.21.15.18:0/3972195624 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764005280 con 0x55e9441b0390 2024-04-16T17:11:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.821+0000 7f576a000640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57540a3e70 0x7f57540a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.821+0000 7f576a000640 1 -- 172.21.15.18:0/1940579884 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f57600d0f60 con 0x7f576c0743a0 2024-04-16T17:11:37.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.821+0000 7f5771600640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57540a3e70 0x7f57540a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.822+0000 7f5776bfe740 1 -- 172.21.15.18:0/3972195624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 msgr2=0x55e9441b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.822+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441b0770 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5764009f30 tx=0x7f57640118d0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.822+0000 7f5776bfe740 1 -- 172.21.15.18:0/3972195624 shutdown_connections 2024-04-16T17:11:37.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.822+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b9890 0x55e9441bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.822+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.822+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3972195624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441b0770 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.822+0000 7f5776bfe740 1 -- 172.21.15.18:0/3972195624 >> 172.21.15.18:0/3972195624 conn(0x55e9441aaac0 msgr2=0x55e9441aaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.823+0000 7f5776bfe740 1 -- 172.21.15.18:0/3972195624 shutdown_connections 2024-04-16T17:11:37.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.823+0000 7f5771600640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57540a3e70 0x7f57540a6330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55e9441afe60 tx=0x7f57600d4a10 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.823+0000 7f5776bfe740 1 -- 172.21.15.18:0/3972195624 wait complete. 2024-04-16T17:11:37.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.823+0000 7f5776bfe740 1 -- start start 2024-04-16T17:11:37.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.823+0000 7f5776bfe740 1 Processor -- start 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.824+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0390 0x55e9441b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.824+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.824+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.824+0000 7f5776bfe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e9441bc680 con 0x55e9441b0cb0 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.824+0000 7f5776bfe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e9440cf710 con 0x55e9441b0390 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.824+0000 7f5776bfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9441bc800 con 0x55e9441b9890 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.824+0000 7f5776bfe740 10 client.0 _ll_register_callbacks cb 0x55e9440caaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.825+0000 7f5770c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.825+0000 7f5772000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.825+0000 7f5772000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44496/0 (socket says 172.21.15.18:44496) 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.825+0000 7f5772000640 1 -- 172.21.15.18:0/10702640 learned_addr learned my addr 172.21.15.18:0/10702640 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.825+0000 7f5772000640 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0390 0x55e9441b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.826+0000 7f5770c00640 1 -- 172.21.15.18:0/10702640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 msgr2=0x55e9441b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.826+0000 7f5770c00640 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.826+0000 7f5770c00640 1 -- 172.21.15.18:0/10702640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0390 msgr2=0x55e9441b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.826+0000 7f5770c00640 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0390 0x55e9441b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.826+0000 7f5770c00640 1 -- 172.21.15.18:0/10702640 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e943e283f0 con 0x55e9441b0cb0 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.826+0000 7f5770c00640 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b6dc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f575802c200 tx=0x7f5758005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac13757bb2e8222 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.826+0000 7f5752a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.827+0000 7f5752a00640 1 -- 172.21.15.18:0/10702640 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758043070 con 0x55e9441b0cb0 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.827+0000 7f5752a00640 1 -- 172.21.15.18:0/10702640 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f575802cba0 con 0x55e9441b0cb0 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.827+0000 7f5776bfe740 1 -- 172.21.15.18:0/10702640 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5764004050 con 0x55e9441b0cb0 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.827+0000 7f5752a00640 1 -- 172.21.15.18:0/10702640 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5758050050 con 0x55e9441b0cb0 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.827+0000 7f5776bfe740 10 client.17265 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:37.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.827+0000 7f5776bfe740 1 -- 172.21.15.18:0/10702640 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e943de80f0 con 0x55e9441b0cb0 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.827+0000 7f5752a00640 1 -- 172.21.15.18:0/10702640 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5764004050 con 0x55e9441b0cb0 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.828+0000 7f5752a00640 1 -- 172.21.15.18:0/10702640 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5758005320 con 0x55e9441b0cb0 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.828+0000 7f5776bfe740 10 client.17265 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.828+0000 7f5776bfe740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.828+0000 7f5776bfe740 20 client.17265 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.828+0000 7f5776bfe740 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9441c9bf0 0x55e9441cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.828+0000 7f5776bfe740 4 client.17265 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.828+0000 7f5776bfe740 1 -- 172.21.15.18:0/10702640 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e944123b40 con 0x55e9441c9bf0 2024-04-16T17:11:37.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.828+0000 7f5772000640 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9441c9bf0 0x55e9441cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.829+0000 7f5772000640 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9441c9bf0 0x55e9441cbfe0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c192bfa20c854cc server_cookie=937a80e351600f15 in_seq=0 out_seq=0 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.829+0000 7f5752a00640 10 client.17265 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.829+0000 7f5752a00640 1 -- 172.21.15.18:0/10702640 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e944123b40 con 0x55e9441c9bf0 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.829+0000 7f5752a00640 10 client.17265 handle_command_reply: tid=0 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 client.17265 shutdown 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 -- 172.21.15.18:0/10702640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 msgr2=0x55e9441b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b6dc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f575802c200 tx=0x7f5758005e10 comp rx=0 tx=0).stop 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 -- 172.21.15.18:0/10702640 shutdown_connections 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9441c9bf0 0x55e9441cbfe0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0cb0 0x55e9441b6dc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 --2- 172.21.15.18:0/10702640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0390 0x55e9441b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5752a00640 0 client.17265 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 -- 172.21.15.18:0/10702640 >> 172.21.15.18:0/10702640 conn(0x55e9441aaac0 msgr2=0x55e9441ac810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 -- 172.21.15.18:0/10702640 shutdown_connections 2024-04-16T17:11:37.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 1 -- 172.21.15.18:0/10702640 wait complete. 2024-04-16T17:11:37.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.830+0000 7f5776bfe740 20 client.17265 trim_cache size 0 max 16384 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5776bfe740 1 Processor -- start 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5776bfe740 1 -- start start 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0390 0x55e9441b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441c5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5776bfe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1b8350 con 0x55e9441b0390 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5776bfe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1b5c20 con 0x55e9441b0cb0 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5776bfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1b3a90 con 0x55e9441b9890 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5770c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5772000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0390 0x55e9441b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5770c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49886/0 (socket says 172.21.15.18:49886) 2024-04-16T17:11:37.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.850+0000 7f5770c00640 1 -- 172.21.15.18:0/2650696638 learned_addr learned my addr 172.21.15.18:0/2650696638 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.851+0000 7f5772000640 1 --2- 172.21.15.18:0/2650696638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.851+0000 7f5770c00640 1 -- 172.21.15.18:0/2650696638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 msgr2=0x55e9441b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.851+0000 7f5770c00640 1 --2- 172.21.15.18:0/2650696638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.851+0000 7f5770c00640 1 -- 172.21.15.18:0/2650696638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0390 msgr2=0x55e9441b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.851+0000 7f5770c00640 1 --2- 172.21.15.18:0/2650696638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0390 0x55e9441b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.851+0000 7f5770c00640 1 -- 172.21.15.18:0/2650696638 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9441bbc80 con 0x55e9441b0cb0 2024-04-16T17:11:37.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.851+0000 7f5770c00640 1 --2- 172.21.15.18:0/2650696638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441c5570 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f575802f020 tx=0x7f5758000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639f937f749c8fca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f575dc00640 1 -- 172.21.15.18:0/2650696638 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758000e60 con 0x55e9441b0cb0 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f575dc00640 1 -- 172.21.15.18:0/2650696638 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5758009520 con 0x55e9441b0cb0 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f575dc00640 1 -- 172.21.15.18:0/2650696638 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758043040 con 0x55e9441b0cb0 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f5776bfe740 1 -- 172.21.15.18:0/2650696638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 msgr2=0x55e9441c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f5776bfe740 1 --2- 172.21.15.18:0/2650696638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441c5570 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f575802f020 tx=0x7f5758000bc0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f5776bfe740 1 -- 172.21.15.18:0/2650696638 shutdown_connections 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f5776bfe740 1 --2- 172.21.15.18:0/2650696638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b9890 0x55e9441b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f5776bfe740 1 --2- 172.21.15.18:0/2650696638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441c5570 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f5776bfe740 1 --2- 172.21.15.18:0/2650696638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b0390 0x55e9441b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.852+0000 7f5776bfe740 1 -- 172.21.15.18:0/2650696638 >> 172.21.15.18:0/2650696638 conn(0x55e9441aaac0 msgr2=0x55e9441ac650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.853+0000 7f5776bfe740 1 -- 172.21.15.18:0/2650696638 shutdown_connections 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.853+0000 7f5776bfe740 1 -- 172.21.15.18:0/2650696638 wait complete. 2024-04-16T17:11:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.853+0000 7f5776bfe740 1 -- start start 2024-04-16T17:11:37.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.853+0000 7f5776bfe740 1 Processor -- start 2024-04-16T17:11:37.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5776bfe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b9890 0x55e9441bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5776bfe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1b40e0 con 0x55e9441b9890 2024-04-16T17:11:37.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5776bfe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1b5550 con 0x55e9441b0cb0 2024-04-16T17:11:37.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5776bfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1b47b0 con 0x55e9441b0390 2024-04-16T17:11:37.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5776bfe740 10 client.0 _ll_register_callbacks cb 0x7f5758009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:37.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5770c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5772000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5770c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441bd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49890/0 (socket says 172.21.15.18:49890) 2024-04-16T17:11:37.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5770c00640 1 -- 172.21.15.18:0/3664522682 learned_addr learned my addr 172.21.15.18:0/3664522682 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:37.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.854+0000 7f5772000640 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b9890 0x55e9441bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5772000640 1 -- 172.21.15.18:0/3664522682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 msgr2=0x55e9441bd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5772000640 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441bd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5772000640 1 -- 172.21.15.18:0/3664522682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b9890 msgr2=0x55e9441bd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:37.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5772000640 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b9890 0x55e9441bd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5772000640 1 -- 172.21.15.18:0/3664522682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9441bdbf0 con 0x55e9441b0390 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5770c00640 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5772000640 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441c1cd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5764003fd0 tx=0x7f5764000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22e2619bed9678a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5752a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5752a00640 1 -- 172.21.15.18:0/3664522682 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764004200 con 0x55e9441b0390 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5752a00640 1 -- 172.21.15.18:0/3664522682 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9554+0+0 (secure 0 0 0) 0x7f57640043a0 con 0x55e9441b0390 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.855+0000 7f5776bfe740 1 -- 172.21.15.18:0/3664522682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5758009520 con 0x55e9441b0390 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5752a00640 1 -- 172.21.15.18:0/3664522682 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57640045a0 con 0x55e9441b0390 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5776bfe740 10 client.27463 fetch_fsmap learned FSMap version 26 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5776bfe740 1 -- 172.21.15.18:0/3664522682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e94422ff70 con 0x55e9441b0390 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5752a00640 1 -- 172.21.15.18:0/3664522682 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5758009520 con 0x55e9441b0390 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5752a00640 1 -- 172.21.15.18:0/3664522682 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5764025040 con 0x55e9441b0390 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5776bfe740 10 client.27463 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5776bfe740 10 client.27463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5776bfe740 20 client.27463 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9442304f0 0x55e9442328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5776bfe740 4 client.27463 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.856+0000 7f5776bfe740 1 -- 172.21.15.18:0/3664522682 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e94411d9b0 con 0x55e9442304f0 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5770c00640 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9442304f0 0x55e9442328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5770c00640 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9442304f0 0x55e9442328e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb242f911cd001b6 server_cookie=8602f8c3c22a4804 in_seq=0 out_seq=0 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5752a00640 10 client.27463 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5752a00640 1 -- 172.21.15.18:0/3664522682 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e94411d9b0 con 0x55e9442304f0 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5752a00640 10 client.27463 handle_command_reply: tid=0 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5776bfe740 1 client.27463 shutdown 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5776bfe740 1 -- 172.21.15.18:0/3664522682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 msgr2=0x55e9441c1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441c1cd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5764003fd0 tx=0x7f5764000f40 comp rx=0 tx=0).stop 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5776bfe740 1 -- 172.21.15.18:0/3664522682 shutdown_connections 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.857+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9442304f0 0x55e9442328e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e9441b9890 0x55e9441bd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9441b0cb0 0x55e9441bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5776bfe740 1 --2- 172.21.15.18:0/3664522682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9441b0390 0x55e9441c1cd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5752a00640 0 client.27463 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5776bfe740 1 -- 172.21.15.18:0/3664522682 >> 172.21.15.18:0/3664522682 conn(0x55e9441aaac0 msgr2=0x55e9441abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5776bfe740 1 -- 172.21.15.18:0/3664522682 shutdown_connections 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5776bfe740 1 -- 172.21.15.18:0/3664522682 wait complete. 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5776bfe740 20 client.27463 trim_cache size 0 max 16384 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stdout: "59e445fd-38ef-4ac4-94ab-45f8b9c016cb": { 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "59e445fd-38ef-4ac4-94ab-45f8b9c016cb", 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:11:37.861 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:11:37.862 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:11:37.862 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:11:37.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5772a00640 1 -- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57540a3e70 msgr2=0x7f57540a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5772a00640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57540a3e70 0x7f57540a6330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55e9441afe60 tx=0x7f57600d4a10 comp rx=0 tx=0).stop 2024-04-16T17:11:37.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5772a00640 1 -- 172.21.15.18:0/1940579884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c0743a0 msgr2=0x7f576c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:37.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.858+0000 7f5772a00640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c0743a0 0x7f576c19ed90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f576000e9f0 tx=0x7f576000edf0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.859+0000 7f5772a00640 1 -- 172.21.15.18:0/1940579884 shutdown_connections 2024-04-16T17:11:37.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.859+0000 7f5772a00640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57540a3e70 0x7f57540a6330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.859+0000 7f5772a00640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f576c112e10 0x7f576c1a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.859+0000 7f5772a00640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c074cc0 0x7f576c19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.859+0000 7f5772a00640 1 --2- 172.21.15.18:0/1940579884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c0743a0 0x7f576c19ed90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:37.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.859+0000 7f5772a00640 1 -- 172.21.15.18:0/1940579884 >> 172.21.15.18:0/1940579884 conn(0x7f576c0fd950 msgr2=0x7f576c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:37.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.860+0000 7f5772a00640 1 -- 172.21.15.18:0/1940579884 shutdown_connections 2024-04-16T17:11:37.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:11:37.860+0000 7f5772a00640 1 -- 172.21.15.18:0/1940579884 wait complete. 2024-04-16T17:11:37.877 INFO:tasks.cephfs.filesystem:scrub status for tag:59e445fd-38ef-4ac4-94ab-45f8b9c016cb - {'path': '/', 'tag': '59e445fd-38ef-4ac4-94ab-45f8b9c016cb', 'options': 'recursive,force'} 2024-04-16T17:11:38.174 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:11:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setuser ceph since I am not root 2024-04-16T17:11:38.174 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:11:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setgroup ceph since I am not root 2024-04-16T17:11:38.174 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:11:37 smithi022 ceph-mds[66065]: -- 172.21.15.22:0/655646806 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561050f5da40 con 0x561050f30c00 2024-04-16T17:11:38.174 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:11:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: starting mds.h at 2024-04-16T17:11:38.591 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262186 3.95 MB/sec execute 231 sec latency 36.528 ms 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: daemon mds.h restarted 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6836/678063310,v1:172.21.15.22:6837/678063310] up:boot 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:38.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:38 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:11:38.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:11:38.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:38.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:11:38.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:38.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: daemon mds.h restarted 2024-04-16T17:11:38.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T17:11:38.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6836/678063310,v1:172.21.15.22:6837/678063310] up:boot 2024-04-16T17:11:38.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:38.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:38 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:11:39.591 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262502 3.95 MB/sec execute 232 sec latency 156.510 ms 2024-04-16T17:11:39.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:39 smithi043 ceph-mon[36901]: pgmap v213: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 946 KiB/s wr, 69 op/s 2024-04-16T17:11:39.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:39 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:11:39.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:37 smithi018 ceph-mon[31776]: pgmap v212: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-04-16T17:11:39.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:11:39.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:39.883 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:11:39.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:39.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: daemon mds.h restarted 2024-04-16T17:11:39.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T17:11:39.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6836/678063310,v1:172.21.15.22:6837/678063310] up:boot 2024-04-16T17:11:39.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:39.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:38 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:11:39.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:39 smithi022 ceph-mon[36023]: pgmap v213: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 946 KiB/s wr, 69 op/s 2024-04-16T17:11:39.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:39 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:11:40.591 INFO:tasks.workunit.client.0.smithi018.stdout: 1 262948 3.94 MB/sec execute 233 sec latency 96.752 ms 2024-04-16T17:11:41.591 INFO:tasks.workunit.client.0.smithi018.stdout: 1 263734 3.93 MB/sec execute 234 sec latency 25.284 ms 2024-04-16T17:11:41.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:41 smithi043 ceph-mon[36901]: pgmap v215: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-04-16T17:11:41.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:41 smithi022 ceph-mon[36023]: pgmap v215: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-04-16T17:11:42.592 INFO:tasks.workunit.client.0.smithi018.stdout: 1 264998 3.94 MB/sec execute 235 sec latency 24.284 ms 2024-04-16T17:11:43.598 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266001 3.96 MB/sec execute 236 sec latency 44.944 ms 2024-04-16T17:11:43.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:43 smithi043 ceph-mon[36901]: pgmap v216: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-04-16T17:11:43.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:43 smithi022 ceph-mon[36023]: pgmap v216: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-04-16T17:11:44.598 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266301 3.95 MB/sec execute 237 sec latency 143.470 ms 2024-04-16T17:11:45.598 INFO:tasks.workunit.client.0.smithi018.stdout: 1 266630 3.94 MB/sec execute 238 sec latency 74.591 ms 2024-04-16T17:11:45.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:45 smithi043 ceph-mon[36901]: pgmap v217: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-04-16T17:11:45.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:45 smithi022 ceph-mon[36023]: pgmap v217: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-04-16T17:11:46.256 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:39 smithi018 ceph-mon[31776]: pgmap v213: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 946 KiB/s wr, 69 op/s 2024-04-16T17:11:46.257 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:39 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:11:46.257 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:41 smithi018 ceph-mon[31776]: pgmap v215: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-04-16T17:11:46.257 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:43 smithi018 ceph-mon[31776]: pgmap v216: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-04-16T17:11:46.257 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:45 smithi018 ceph-mon[31776]: pgmap v217: 129 pgs: 129 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-04-16T17:11:46.598 INFO:tasks.workunit.client.0.smithi018.stdout: 1 267548 3.94 MB/sec execute 239 sec latency 26.656 ms 2024-04-16T17:11:47.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 268980 3.96 MB/sec execute 240 sec latency 29.335 ms 2024-04-16T17:11:47.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:47 smithi043 ceph-mon[36901]: pgmap v218: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-04-16T17:11:47.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:47 smithi022 ceph-mon[36023]: pgmap v218: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-04-16T17:11:48.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 269851 3.97 MB/sec execute 241 sec latency 46.764 ms 2024-04-16T17:11:49.598 INFO:tasks.workunit.client.0.smithi018.stdout: 1 270116 3.95 MB/sec execute 242 sec latency 76.579 ms 2024-04-16T17:11:49.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:49 smithi043 ceph-mon[36901]: pgmap v219: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-16T17:11:49.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:49 smithi022 ceph-mon[36023]: pgmap v219: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-16T17:11:50.598 INFO:tasks.workunit.client.0.smithi018.stdout: 1 270415 3.94 MB/sec execute 243 sec latency 89.553 ms 2024-04-16T17:11:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 -- 172.21.15.18:0/133473728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d90103900 msgr2=0x7f7d9010bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 --2- 172.21.15.18:0/133473728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d90103900 0x7f7d9010bde0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009860 tx=0x7f7d7c02bc20 comp rx=0 tx=0).stop 2024-04-16T17:11:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 -- 172.21.15.18:0/133473728 shutdown_connections 2024-04-16T17:11:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 --2- 172.21.15.18:0/133473728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9010c400 0x7f7d9010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 --2- 172.21.15.18:0/133473728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d90103900 0x7f7d9010bde0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 --2- 172.21.15.18:0/133473728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d90102fe0 0x7f7d901033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 -- 172.21.15.18:0/133473728 >> 172.21.15.18:0/133473728 conn(0x7f7d900fc7e0 msgr2=0x7f7d900fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 -- 172.21.15.18:0/133473728 shutdown_connections 2024-04-16T17:11:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.648+0000 7f7d95a00640 1 -- 172.21.15.18:0/133473728 wait complete. 2024-04-16T17:11:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.650+0000 7f7d95a00640 1 Processor -- start 2024-04-16T17:11:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.650+0000 7f7d95a00640 1 -- start start 2024-04-16T17:11:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d90102fe0 0x7f7d901a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d90103900 0x7f7d901a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9010c400 0x7f7d9019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d901121c0 con 0x7f7d90102fe0 2024-04-16T17:11:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90112040 con 0x7f7d90103900 2024-04-16T17:11:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90112340 con 0x7f7d9010c400 2024-04-16T17:11:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9010c400 0x7f7d9019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9010c400 0x7f7d9019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:58688/0 (socket says 172.21.15.18:58688) 2024-04-16T17:11:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.651+0000 7f7d95000640 1 -- 172.21.15.18:0/1490265620 learned_addr learned my addr 172.21.15.18:0/1490265620 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d8f400640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d90103900 0x7f7d901a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d95000640 1 -- 172.21.15.18:0/1490265620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d90103900 msgr2=0x7f7d901a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d95000640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d90103900 0x7f7d901a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d95000640 1 -- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d90102fe0 msgr2=0x7f7d901a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d90102fe0 0x7f7d901a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d95000640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d90102fe0 0x7f7d901a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d95000640 1 -- 172.21.15.18:0/1490265620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9019af50 con 0x7f7d9010c400 2024-04-16T17:11:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d90102fe0 0x7f7d901a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.652+0000 7f7d95000640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9010c400 0x7f7d9019a780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800eaa0 tx=0x7f7d8800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.653+0000 7f7d8cc00640 1 -- 172.21.15.18:0/1490265620 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800cdb0 con 0x7f7d9010c400 2024-04-16T17:11:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.653+0000 7f7d8cc00640 1 -- 172.21.15.18:0/1490265620 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d88005020 con 0x7f7d9010c400 2024-04-16T17:11:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.653+0000 7f7d95a00640 1 -- 172.21.15.18:0/1490265620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9019b240 con 0x7f7d9010c400 2024-04-16T17:11:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.653+0000 7f7d8cc00640 1 -- 172.21.15.18:0/1490265620 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d880116b0 con 0x7f7d9010c400 2024-04-16T17:11:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.654+0000 7f7d95a00640 1 -- 172.21.15.18:0/1490265620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d901a7460 con 0x7f7d9010c400 2024-04-16T17:11:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.658+0000 7f7d99da1740 1 Processor -- start 2024-04-16T17:11:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.659+0000 7f7d8cc00640 1 -- 172.21.15.18:0/1490265620 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d88005560 con 0x7f7d9010c400 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.659+0000 7f7d99da1740 1 -- start start 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.659+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 0x564e9fb4dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.659+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4e290 0x564e9fb4e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.659+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb57810 0x564e9fb59c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.659+0000 7f7d99da1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564e9faaa780 con 0x564e9fb57810 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.660+0000 7f7d99da1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564e9fa50f40 con 0x564e9fb4e290 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.660+0000 7f7d99da1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564e9faa6740 con 0x564e9fb4f000 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.660+0000 7f7d8cc00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d740a3d80 0x7f7d740a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.660+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4e290 0x564e9fb4e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.660+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4e290 0x564e9fb4e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45440/0 (socket says 172.21.15.18:45440) 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8fe00640 1 -- 172.21.15.18:0/3412789019 learned_addr learned my addr 172.21.15.18:0/3412789019 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8cc00640 1 -- 172.21.15.18:0/1490265620 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9466+0+0 (secure 0 0 0) 0x7f7d880d0e30 con 0x7f7d9010c400 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8fe00640 1 -- 172.21.15.18:0/3412789019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 msgr2=0x564e9fb4dd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 0x564e9fb4dd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8f400640 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb57810 0x564e9fb59c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8fe00640 1 -- 172.21.15.18:0/3412789019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb57810 msgr2=0x564e9fb59c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d95000640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d740a3d80 0x7f7d740a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb57810 0x564e9fb59c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8fe00640 1 -- 172.21.15.18:0/3412789019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e9f7bd210 con 0x564e9fb4e290 2024-04-16T17:11:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.661+0000 7f7d8f400640 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb57810 0x564e9fb59c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.662+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4e290 0x564e9fb4e710 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009c30 tx=0x7f7d80010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5af9c4e79bf706fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.662+0000 7f7d95000640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d740a3d80 0x7f7d740a6240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d88098e20 tx=0x7f7d880058d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.662+0000 7f7d85600640 1 -- 172.21.15.18:0/3412789019 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80004020 con 0x564e9fb4e290 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.662+0000 7f7d85600640 1 -- 172.21.15.18:0/3412789019 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d800041c0 con 0x564e9fb4e290 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.662+0000 7f7d85600640 1 -- 172.21.15.18:0/3412789019 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80012690 con 0x564e9fb4e290 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 -- 172.21.15.18:0/3412789019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4e290 msgr2=0x564e9fb4e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4e290 0x564e9fb4e710 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009c30 tx=0x7f7d80010740 comp rx=0 tx=0).stop 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 -- 172.21.15.18:0/3412789019 shutdown_connections 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb57810 0x564e9fb59c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4e290 0x564e9fb4e710 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 --2- 172.21.15.18:0/3412789019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 0x564e9fb4dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 -- 172.21.15.18:0/3412789019 >> 172.21.15.18:0/3412789019 conn(0x564e9fae05d0 msgr2=0x564e9fb48f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 -- 172.21.15.18:0/3412789019 shutdown_connections 2024-04-16T17:11:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.663+0000 7f7d99da1740 1 -- 172.21.15.18:0/3412789019 wait complete. 2024-04-16T17:11:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.664+0000 7f7d99da1740 1 -- start start 2024-04-16T17:11:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.664+0000 7f7d99da1740 1 Processor -- start 2024-04-16T17:11:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.665+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.665+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb54c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.665+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 0x564e9fb55190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.665+0000 7f7d99da1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564e9fb6b550 con 0x564e9fb4e290 2024-04-16T17:11:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.665+0000 7f7d99da1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564e9fa64390 con 0x564e9fb4f000 2024-04-16T17:11:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.665+0000 7f7d99da1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564e9fb6b6d0 con 0x564e9fb57810 2024-04-16T17:11:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.665+0000 7f7d99da1740 10 client.0 _ll_register_callbacks cb 0x564e9fa4df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb54c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb54c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45454/0 (socket says 172.21.15.18:45454) 2024-04-16T17:11:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8fe00640 1 -- 172.21.15.18:0/1662057279 learned_addr learned my addr 172.21.15.18:0/1662057279 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8f400640 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 0x564e9fb55190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8f400640 1 -- 172.21.15.18:0/1662057279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 msgr2=0x564e9fb55190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8f400640 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 0x564e9fb55190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8f400640 1 -- 172.21.15.18:0/1662057279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 msgr2=0x564e9fb54c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8f400640 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb54c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.666+0000 7f7d8f400640 1 -- 172.21.15.18:0/1662057279 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9f889620 con 0x564e9fb4e290 2024-04-16T17:11:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.667+0000 7f7d8f400640 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb54710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009830 tx=0x7f7d7c005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e6d8bc09a33483 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.667+0000 7f7d7a000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:11:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.667+0000 7f7d7a000640 1 -- 172.21.15.18:0/1662057279 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c039050 con 0x564e9fb4e290 2024-04-16T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.667+0000 7f7d7a000640 1 -- 172.21.15.18:0/1662057279 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9466+0+0 (secure 0 0 0) 0x7f7d7c0348d0 con 0x564e9fb4e290 2024-04-16T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.667+0000 7f7d99da1740 1 -- 172.21.15.18:0/1662057279 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d800041c0 con 0x564e9fb4e290 2024-04-16T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.668+0000 7f7d7a000640 1 -- 172.21.15.18:0/1662057279 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c005030 con 0x564e9fb4e290 2024-04-16T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.668+0000 7f7d99da1740 10 client.17277 fetch_fsmap learned FSMap version 27 2024-04-16T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.668+0000 7f7d99da1740 1 -- 172.21.15.18:0/1662057279 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e9fb42cc0 con 0x564e9fb4e290 2024-04-16T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.668+0000 7f7d7a000640 1 -- 172.21.15.18:0/1662057279 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d800041c0 con 0x564e9fb4e290 2024-04-16T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.669+0000 7f7d7a000640 1 -- 172.21.15.18:0/1662057279 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7d7c0046c0 con 0x564e9fb4e290 2024-04-16T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.669+0000 7f7d99da1740 10 client.17277 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:11:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.669+0000 7f7d99da1740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.669+0000 7f7d99da1740 20 client.17277 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.669+0000 7f7d99da1740 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564e9fb44180 0x564e9fb46570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.669+0000 7f7d99da1740 4 client.17277 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.669+0000 7f7d99da1740 1 -- 172.21.15.18:0/1662057279 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e9fab81a0 con 0x564e9fb44180 2024-04-16T17:11:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.669+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564e9fb44180 0x564e9fb46570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.670+0000 7f7d7a000640 10 client.17277 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.670+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564e9fb44180 0x564e9fb46570 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e389d1794c0bf88 server_cookie=732e1c7863e7a2b9 in_seq=0 out_seq=0 2024-04-16T17:11:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.671+0000 7f7d7a000640 1 -- 172.21.15.18:0/1662057279 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e9fab81a0 con 0x564e9fb44180 2024-04-16T17:11:50.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.671+0000 7f7d7a000640 10 client.17277 handle_command_reply: tid=0 2024-04-16T17:11:50.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.671+0000 7f7d99da1740 1 client.17277 shutdown 2024-04-16T17:11:50.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.671+0000 7f7d99da1740 1 -- 172.21.15.18:0/1662057279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 msgr2=0x564e9fb54710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.671+0000 7f7d99da1740 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb54710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009830 tx=0x7f7d7c005cd0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 1 -- 172.21.15.18:0/1662057279 shutdown_connections 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564e9fb44180 0x564e9fb46570 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 0x564e9fb55190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb54c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 1 --2- 172.21.15.18:0/1662057279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb54710 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 1 -- 172.21.15.18:0/1662057279 >> 172.21.15.18:0/1662057279 conn(0x564e9fae05d0 msgr2=0x564e9fb4acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d7a000640 0 client.17277 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 1 -- 172.21.15.18:0/1662057279 shutdown_connections 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 1 -- 172.21.15.18:0/1662057279 wait complete. 2024-04-16T17:11:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.672+0000 7f7d99da1740 20 client.17277 trim_cache size 0 max 16384 2024-04-16T17:11:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.688+0000 7f7d99da1740 1 Processor -- start 2024-04-16T17:11:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d99da1740 1 -- start start 2024-04-16T17:11:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb74450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 0x564e9fb8aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb57810 0x564e9fb8af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d99da1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90073dd0 con 0x564e9fb4e290 2024-04-16T17:11:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d99da1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900716a0 con 0x564e9fb57810 2024-04-16T17:11:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d99da1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9006f510 con 0x564e9fb4f000 2024-04-16T17:11:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb74450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 0x564e9fb8aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb74450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43752/0 (socket says 172.21.15.18:43752) 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d8f400640 1 -- 172.21.15.18:0/3773157886 learned_addr learned my addr 172.21.15.18:0/3773157886 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.689+0000 7f7d8f400640 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb57810 0x564e9fb8af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.690+0000 7f7d8fe00640 1 -- 172.21.15.18:0/3773157886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb57810 msgr2=0x564e9fb8af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.690+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb57810 0x564e9fb8af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.690+0000 7f7d8fe00640 1 -- 172.21.15.18:0/3773157886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 msgr2=0x564e9fb74450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.690+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb74450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.690+0000 7f7d8fe00640 1 -- 172.21.15.18:0/3773157886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e9fb468e0 con 0x564e9fb4f000 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.690+0000 7f7d8f400640 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb57810 0x564e9fb8af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.690+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 0x564e9fb8aa40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d80010fb0 tx=0x7f7d80005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3ae500c035d0326 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.690+0000 7f7d84c00640 1 -- 172.21.15.18:0/3773157886 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80009d20 con 0x564e9fb4f000 2024-04-16T17:11:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d84c00640 1 -- 172.21.15.18:0/3773157886 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d80031070 con 0x564e9fb4f000 2024-04-16T17:11:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d84c00640 1 -- 172.21.15.18:0/3773157886 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8002b2c0 con 0x564e9fb4f000 2024-04-16T17:11:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d99da1740 1 -- 172.21.15.18:0/3773157886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 msgr2=0x564e9fb8aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d99da1740 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 0x564e9fb8aa40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d80010fb0 tx=0x7f7d80005a20 comp rx=0 tx=0).stop 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d99da1740 1 -- 172.21.15.18:0/3773157886 shutdown_connections 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d99da1740 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb57810 0x564e9fb8af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d99da1740 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb4f000 0x564e9fb8aa40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d99da1740 1 --2- 172.21.15.18:0/3773157886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb74450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d99da1740 1 -- 172.21.15.18:0/3773157886 >> 172.21.15.18:0/3773157886 conn(0x564e9fae05d0 msgr2=0x564e9fb3f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.691+0000 7f7d99da1740 1 -- 172.21.15.18:0/3773157886 shutdown_connections 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.692+0000 7f7d99da1740 1 -- 172.21.15.18:0/3773157886 wait complete. 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.692+0000 7f7d99da1740 1 -- start start 2024-04-16T17:11:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.692+0000 7f7d99da1740 1 Processor -- start 2024-04-16T17:11:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb70960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d99da1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 0x564e9fb509c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d99da1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9006fb60 con 0x564e9fb4e290 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d99da1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90070fd0 con 0x564e9fb4f000 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d99da1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90070230 con 0x564e9fb57810 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d99da1740 10 client.0 _ll_register_callbacks cb 0x7f7d9006a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb70960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb70ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45466/0 (socket says 172.21.15.18:45466) 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8fe00640 1 -- 172.21.15.18:0/2263609469 learned_addr learned my addr 172.21.15.18:0/2263609469 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb70960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43768/0 (socket says 172.21.15.18:43768) 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8f400640 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 0x564e9fb509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8fe00640 1 -- 172.21.15.18:0/2263609469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 msgr2=0x564e9fb509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 0x564e9fb509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8fe00640 1 -- 172.21.15.18:0/2263609469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 msgr2=0x564e9fb70960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb70960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.693+0000 7f7d8fe00640 1 -- 172.21.15.18:0/2263609469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9fb50f80 con 0x564e9fb4f000 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.694+0000 7f7d8f400640 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb70960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.694+0000 7f7d8fe00640 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb70ea0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d80010780 tx=0x7f7d80009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62fc14481b406297 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.694+0000 7f7d7a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.694+0000 7f7d7a000640 1 -- 172.21.15.18:0/2263609469 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8003a060 con 0x564e9fb4f000 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.694+0000 7f7d7a000640 1 -- 172.21.15.18:0/2263609469 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9466+0+0 (secure 0 0 0) 0x7f7d80003730 con 0x564e9fb4f000 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.694+0000 7f7d99da1740 1 -- 172.21.15.18:0/2263609469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d80031070 con 0x564e9fb4f000 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d7a000640 1 -- 172.21.15.18:0/2263609469 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d80041050 con 0x564e9fb4f000 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d99da1740 10 client.27551 fetch_fsmap learned FSMap version 27 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d99da1740 1 -- 172.21.15.18:0/2263609469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e9fb52270 con 0x564e9fb4f000 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d7a000640 1 -- 172.21.15.18:0/2263609469 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d80031070 con 0x564e9fb4f000 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d7a000640 1 -- 172.21.15.18:0/2263609469 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7d8000f040 con 0x564e9fb4f000 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d99da1740 10 client.27551 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d99da1740 10 client.27551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d99da1740 20 client.27551 populate_metadata read hostname 'smithi018' 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d99da1740 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564e9fbe82b0 0x564e9fbea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d99da1740 4 client.27551 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:11:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.695+0000 7f7d99da1740 1 -- 172.21.15.18:0/2263609469 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e9fab2010 con 0x564e9fbe82b0 2024-04-16T17:11:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.696+0000 7f7d8f400640 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564e9fbe82b0 0x564e9fbea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.696+0000 7f7d8f400640 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564e9fbe82b0 0x564e9fbea6a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd5e1a67a4ba4d9f server_cookie=f02da50e48abea31 in_seq=0 out_seq=0 2024-04-16T17:11:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.696+0000 7f7d7a000640 10 client.27551 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.860+0000 7f7d7a000640 1 -- 172.21.15.18:0/2263609469 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e9fab2010 con 0x564e9fbe82b0 2024-04-16T17:11:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.860+0000 7f7d7a000640 10 client.27551 handle_command_reply: tid=0 2024-04-16T17:11:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.860+0000 7f7d99da1740 1 client.27551 shutdown 2024-04-16T17:11:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.860+0000 7f7d99da1740 1 -- 172.21.15.18:0/2263609469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 msgr2=0x564e9fb70ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.860+0000 7f7d99da1740 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb70ea0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d80010780 tx=0x7f7d80009460 comp rx=0 tx=0).stop 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.860+0000 7f7d99da1740 1 -- 172.21.15.18:0/2263609469 shutdown_connections 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.860+0000 7f7d99da1740 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564e9fbe82b0 0x564e9fbea6a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.861+0000 7f7d99da1740 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e9fb57810 0x564e9fb509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.861+0000 7f7d99da1740 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e9fb4f000 0x564e9fb70ea0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.861+0000 7f7d99da1740 1 --2- 172.21.15.18:0/2263609469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564e9fb4e290 0x564e9fb70960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.861+0000 7f7d99da1740 1 -- 172.21.15.18:0/2263609469 >> 172.21.15.18:0/2263609469 conn(0x564e9fae05d0 msgr2=0x564e9fb49f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.861+0000 7f7d7a000640 0 client.27551 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.861+0000 7f7d99da1740 1 -- 172.21.15.18:0/2263609469 shutdown_connections 2024-04-16T17:11:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.861+0000 7f7d99da1740 1 -- 172.21.15.18:0/2263609469 wait complete. 2024-04-16T17:11:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.863+0000 7f7d99da1740 20 client.27551 trim_cache size 0 max 16384 2024-04-16T17:11:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:11:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:11:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:11:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:11:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 -- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d740a3d80 msgr2=0x7f7d740a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d740a3d80 0x7f7d740a6240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d88098e20 tx=0x7f7d880058d0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 -- 172.21.15.18:0/1490265620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9010c400 msgr2=0x7f7d9019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9010c400 0x7f7d9019a780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800eaa0 tx=0x7f7d8800ef60 comp rx=0 tx=0).stop 2024-04-16T17:11:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 -- 172.21.15.18:0/1490265620 shutdown_connections 2024-04-16T17:11:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d740a3d80 0x7f7d740a6240 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9010c400 0x7f7d9019a780 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d90103900 0x7f7d901a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 --2- 172.21.15.18:0/1490265620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d90102fe0 0x7f7d901a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 -- 172.21.15.18:0/1490265620 >> 172.21.15.18:0/1490265620 conn(0x7f7d900fc7e0 msgr2=0x7f7d90100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 -- 172.21.15.18:0/1490265620 shutdown_connections 2024-04-16T17:11:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:11:50.864+0000 7f7d95a00640 1 -- 172.21.15.18:0/1490265620 wait complete. 2024-04-16T17:11:50.943 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:47 smithi018 ceph-mon[31776]: pgmap v218: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-04-16T17:11:50.943 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:49 smithi018 ceph-mon[31776]: pgmap v219: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-16T17:11:51.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 271743 3.94 MB/sec execute 244 sec latency 54.682 ms 2024-04-16T17:11:51.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:51 smithi043 ceph-mon[36901]: pgmap v220: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-16T17:11:51.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:51 smithi022 ceph-mon[36023]: pgmap v220: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-16T17:11:52.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 272972 3.96 MB/sec execute 245 sec latency 28.697 ms 2024-04-16T17:11:53.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273570 3.96 MB/sec execute 246 sec latency 63.890 ms 2024-04-16T17:11:53.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:53 smithi043 ceph-mon[36901]: pgmap v221: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-04-16T17:11:53.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:53 smithi022 ceph-mon[36023]: pgmap v221: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-04-16T17:11:54.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 273961 3.95 MB/sec execute 247 sec latency 40.078 ms 2024-04-16T17:11:55.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 274379 3.94 MB/sec execute 248 sec latency 70.015 ms 2024-04-16T17:11:55.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:55 smithi043 ceph-mon[36901]: pgmap v222: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-04-16T17:11:55.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:55 smithi022 ceph-mon[36023]: pgmap v222: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-04-16T17:11:56.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 275560 3.94 MB/sec execute 249 sec latency 52.659 ms 2024-04-16T17:11:57.599 INFO:tasks.workunit.client.0.smithi018.stdout: 1 276773 3.97 MB/sec execute 250 sec latency 29.772 ms 2024-04-16T17:11:57.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:57 smithi043 ceph-mon[36901]: pgmap v223: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-04-16T17:11:57.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:57 smithi022 ceph-mon[36023]: pgmap v223: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-04-16T17:11:58.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 277155 3.96 MB/sec execute 251 sec latency 83.113 ms 2024-04-16T17:11:59.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 277452 3.95 MB/sec execute 252 sec latency 75.223 ms 2024-04-16T17:11:59.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:11:59 smithi043 ceph-mon[36901]: pgmap v224: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-04-16T17:11:59.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:11:59 smithi022 ceph-mon[36023]: pgmap v224: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-04-16T17:12:00.229 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:51 smithi018 ceph-mon[31776]: pgmap v220: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-16T17:12:00.229 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:53 smithi018 ceph-mon[31776]: pgmap v221: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-04-16T17:12:00.229 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:55 smithi018 ceph-mon[31776]: pgmap v222: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-04-16T17:12:00.230 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:57 smithi018 ceph-mon[31776]: pgmap v223: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-04-16T17:12:00.230 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:11:59 smithi018 ceph-mon[31776]: pgmap v224: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-04-16T17:12:00.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 278119 3.95 MB/sec execute 253 sec latency 63.247 ms 2024-04-16T17:12:01.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 278965 3.94 MB/sec execute 254 sec latency 55.201 ms 2024-04-16T17:12:01.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:01 smithi018 ceph-mon[31776]: pgmap v225: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-16T17:12:01.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:01 smithi022 ceph-mon[36023]: pgmap v225: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-16T17:12:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:01 smithi043 ceph-mon[36901]: pgmap v225: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-16T17:12:02.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 279289 3.93 MB/sec execute 255 sec latency 66.044 ms 2024-04-16T17:12:02.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:02 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:12:03.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:02 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:12:03.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 280379 3.95 MB/sec execute 256 sec latency 55.681 ms 2024-04-16T17:12:03.636 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:02 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:12:03.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:03 smithi022 ceph-mon[36023]: pgmap v226: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-04-16T17:12:03.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:03 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:12:03.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:03 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:12:03.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:03 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:12:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:03 smithi043 ceph-mon[36901]: pgmap v226: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-04-16T17:12:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:03 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:12:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:03 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:12:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:03 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:12:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 -- 172.21.15.18:0/78079129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c60107150 msgr2=0x7f4c6010ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 --2- 172.21.15.18:0/78079129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c60107150 0x7f4c6010ed20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009ca0 tx=0x7f4c5802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 -- 172.21.15.18:0/78079129 shutdown_connections 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 --2- 172.21.15.18:0/78079129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c60107150 0x7f4c6010ed20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 --2- 172.21.15.18:0/78079129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60102a30 0x7f4c60106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 --2- 172.21.15.18:0/78079129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c60102080 0x7f4c60102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 -- 172.21.15.18:0/78079129 >> 172.21.15.18:0/78079129 conn(0x7f4c600fd940 msgr2=0x7f4c600ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 -- 172.21.15.18:0/78079129 shutdown_connections 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.015+0000 7f4c67000640 1 -- 172.21.15.18:0/78079129 wait complete. 2024-04-16T17:12:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.016+0000 7f4c67000640 1 Processor -- start 2024-04-16T17:12:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.016+0000 7f4c67000640 1 -- start start 2024-04-16T17:12:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c67000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c60102080 0x7f4c601a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c67000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60107150 0x7f4c601a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c67000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c6019a790 0x7f4c6019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c67000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60112210 con 0x7f4c6019a790 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c67000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60112090 con 0x7f4c60102080 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c67000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60112390 con 0x7f4c60107150 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c60102080 0x7f4c601a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c60102080 0x7f4c601a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:41366/0 (socket says 172.21.15.18:41366) 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c65c00640 1 -- 172.21.15.18:0/1865517638 learned_addr learned my addr 172.21.15.18:0/1865517638 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c65200640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60107150 0x7f4c601a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.017+0000 7f4c66600640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c6019a790 0x7f4c6019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c65c00640 1 -- 172.21.15.18:0/1865517638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60107150 msgr2=0x7f4c601a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60107150 0x7f4c601a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c65c00640 1 -- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c6019a790 msgr2=0x7f4c6019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c6019a790 0x7f4c6019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c65c00640 1 -- 172.21.15.18:0/1865517638 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6019b320 con 0x7f4c60102080 2024-04-16T17:12:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c60102080 0x7f4c601a0580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400e9f0 tx=0x7f4c5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c5e000640 1 -- 172.21.15.18:0/1865517638 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400cd80 con 0x7f4c60102080 2024-04-16T17:12:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c67000640 1 -- 172.21.15.18:0/1865517638 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6019b610 con 0x7f4c60102080 2024-04-16T17:12:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c5e000640 1 -- 172.21.15.18:0/1865517638 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54010910 con 0x7f4c60102080 2024-04-16T17:12:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c67000640 1 -- 172.21.15.18:0/1865517638 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c601a77a0 con 0x7f4c60102080 2024-04-16T17:12:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.018+0000 7f4c5e000640 1 -- 172.21.15.18:0/1865517638 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54018980 con 0x7f4c60102080 2024-04-16T17:12:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.021+0000 7f4c6b23e740 1 Processor -- start 2024-04-16T17:12:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.021+0000 7f4c5e000640 1 -- 172.21.15.18:0/1865517638 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c54010430 con 0x7f4c60102080 2024-04-16T17:12:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.022+0000 7f4c6b23e740 1 -- start start 2024-04-16T17:12:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.022+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f8d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.022+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f8df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.022+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 0x5560b6f993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.022+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5560b6ef2780 con 0x5560b6f8e7f0 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.022+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560b6e98f40 con 0x5560b6f8da80 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.022+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5560b6eee740 con 0x5560b6f97000 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c66600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f8d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f8df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c66600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f8d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36870/0 (socket says 172.21.15.18:36870) 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c66600640 1 -- 172.21.15.18:0/1870093682 learned_addr learned my addr 172.21.15.18:0/1870093682 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c65200640 1 --2- 172.21.15.18:0/1870093682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 0x5560b6f993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c65c00640 1 -- 172.21.15.18:0/1870093682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 msgr2=0x5560b6f993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1870093682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 0x5560b6f993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c65c00640 1 -- 172.21.15.18:0/1870093682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 msgr2=0x5560b6f8d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1870093682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f8d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.023+0000 7f4c65c00640 1 -- 172.21.15.18:0/1870093682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b6c05210 con 0x5560b6f8da80 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.024+0000 7f4c5e000640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c480a3d00 0x7f4c480a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.024+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1870093682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f8df00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4c540ce340 tx=0x7f4c540d96f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957ad6e2917a69d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.024+0000 7f4c5e000640 1 -- 172.21.15.18:0/1865517638 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9466+0+0 (secure 0 0 0) 0x7f4c540d0fe0 con 0x7f4c60102080 2024-04-16T17:12:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.024+0000 7f4c52a00640 1 -- 172.21.15.18:0/1870093682 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54098ee0 con 0x5560b6f8da80 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.024+0000 7f4c52a00640 1 -- 172.21.15.18:0/1870093682 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54010ac0 con 0x5560b6f8da80 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.024+0000 7f4c52a00640 1 -- 172.21.15.18:0/1870093682 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c540d4ba0 con 0x5560b6f8da80 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.024+0000 7f4c66600640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c480a3d00 0x7f4c480a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1870093682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 msgr2=0x5560b6f8df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1870093682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f8df00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4c540ce340 tx=0x7f4c540d96f0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1870093682 shutdown_connections 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1870093682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 0x5560b6f993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1870093682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f8df00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1870093682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f8d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1870093682 >> 172.21.15.18:0/1870093682 conn(0x5560b6f285d0 msgr2=0x5560b6f88770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c66600640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c480a3d00 0x7f4c480a61c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009a60 tx=0x7f4c58005d80 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.025+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1870093682 shutdown_connections 2024-04-16T17:12:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.026+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1870093682 wait complete. 2024-04-16T17:12:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.026+0000 7f4c6b23e740 1 -- start start 2024-04-16T17:12:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.026+0000 7f4c6b23e740 1 Processor -- start 2024-04-16T17:12:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f93e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 0x5560b6f948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5560b6f99c70 con 0x5560b6f8da80 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560b6eac390 con 0x5560b6f97000 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5560b6f99df0 con 0x5560b6f8e7f0 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c6b23e740 10 client.0 _ll_register_callbacks cb 0x5560b6e95f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f93e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c65200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f93e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36876/0 (socket says 172.21.15.18:36876) 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c65c00640 1 -- 172.21.15.18:0/1216448106 learned_addr learned my addr 172.21.15.18:0/1216448106 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.027+0000 7f4c66600640 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 0x5560b6f948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.028+0000 7f4c65c00640 1 -- 172.21.15.18:0/1216448106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 msgr2=0x5560b6f943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.028+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f943a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.028+0000 7f4c65c00640 1 -- 172.21.15.18:0/1216448106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 msgr2=0x5560b6f948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.028+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 0x5560b6f948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.028+0000 7f4c65c00640 1 -- 172.21.15.18:0/1216448106 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b6cd1620 con 0x5560b6f8da80 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.028+0000 7f4c65c00640 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f93e60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4c54099610 tx=0x7f4c54009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35169ff1ec63cbb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.028+0000 7f4c47400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.028+0000 7f4c47400640 1 -- 172.21.15.18:0/1216448106 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5409c5a0 con 0x5560b6f8da80 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c47400640 1 -- 172.21.15.18:0/1216448106 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9466+0+0 (secure 0 0 0) 0x7f4c540ce5c0 con 0x5560b6f8da80 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1216448106 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560b6f001a0 con 0x5560b6f8da80 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c47400640 1 -- 172.21.15.18:0/1216448106 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c5409c780 con 0x5560b6f8da80 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c6b23e740 10 client.17295 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1216448106 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5560b6f99c70 con 0x5560b6f8da80 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c47400640 1 -- 172.21.15.18:0/1216448106 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5560b6f001a0 con 0x5560b6f8da80 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c47400640 1 -- 172.21.15.18:0/1216448106 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4c540d4170 con 0x5560b6f8da80 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c6b23e740 10 client.17295 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.029+0000 7f4c6b23e740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.030+0000 7f4c6b23e740 20 client.17295 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.030+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5560b6fa6f00 0x5560b6fa92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.030+0000 7f4c6b23e740 4 client.17295 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.030+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1216448106 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560b6efa010 con 0x5560b6fa6f00 2024-04-16T17:12:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.030+0000 7f4c65200640 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5560b6fa6f00 0x5560b6fa92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.030+0000 7f4c65200640 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5560b6fa6f00 0x5560b6fa92f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e40556f0250bad67 server_cookie=78613f578f87d984 in_seq=0 out_seq=0 2024-04-16T17:12:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.030+0000 7f4c47400640 10 client.17295 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c47400640 1 -- 172.21.15.18:0/1216448106 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560b6efa010 con 0x5560b6fa6f00 2024-04-16T17:12:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c47400640 10 client.17295 handle_command_reply: tid=0 2024-04-16T17:12:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 client.17295 shutdown 2024-04-16T17:12:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1216448106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 msgr2=0x5560b6f93e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f93e60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4c54099610 tx=0x7f4c54009700 comp rx=0 tx=0).stop 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1216448106 shutdown_connections 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5560b6fa6f00 0x5560b6fa92f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 0x5560b6f948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/1216448106 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f93e60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1216448106 >> 172.21.15.18:0/1216448106 conn(0x5560b6f285d0 msgr2=0x5560b6f8a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.031+0000 7f4c47400640 0 client.17295 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.032+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1216448106 shutdown_connections 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.032+0000 7f4c6b23e740 1 -- 172.21.15.18:0/1216448106 wait complete. 2024-04-16T17:12:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.032+0000 7f4c6b23e740 20 client.17295 trim_cache size 0 max 16384 2024-04-16T17:12:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c6b23e740 1 Processor -- start 2024-04-16T17:12:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c6b23e740 1 -- start start 2024-04-16T17:12:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f96520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f93040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 0x5560b6f94740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601b3ea0 con 0x5560b6f8e7f0 2024-04-16T17:12:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601b1770 con 0x5560b6f8da80 2024-04-16T17:12:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601af5c0 con 0x5560b6f97000 2024-04-16T17:12:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f96520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c66600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 0x5560b6f94740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f96520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:41404/0 (socket says 172.21.15.18:41404) 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.050+0000 7f4c65c00640 1 -- 172.21.15.18:0/3819150801 learned_addr learned my addr 172.21.15.18:0/3819150801 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c65200640 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f93040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c65c00640 1 -- 172.21.15.18:0/3819150801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 msgr2=0x5560b6f94740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c65c00640 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 0x5560b6f94740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c65c00640 1 -- 172.21.15.18:0/3819150801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 msgr2=0x5560b6f93040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c65c00640 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f93040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c65c00640 1 -- 172.21.15.18:0/3819150801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b6f8cd50 con 0x5560b6f8da80 2024-04-16T17:12:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c65200640 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f93040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c65c00640 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f96520 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4c540bd050 tx=0x7f4c54099580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd33ae9b634d7ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c52000640 1 -- 172.21.15.18:0/3819150801 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5409ca10 con 0x5560b6f8da80 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c52000640 1 -- 172.21.15.18:0/3819150801 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54087050 con 0x5560b6f8da80 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.051+0000 7f4c52000640 1 -- 172.21.15.18:0/3819150801 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c540ce5c0 con 0x5560b6f8da80 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 -- 172.21.15.18:0/3819150801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 msgr2=0x5560b6f96520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f96520 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4c540bd050 tx=0x7f4c54099580 comp rx=0 tx=0).stop 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 -- 172.21.15.18:0/3819150801 shutdown_connections 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f97000 0x5560b6f94740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8e7f0 0x5560b6f93040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/3819150801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f8da80 0x5560b6f96520 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 -- 172.21.15.18:0/3819150801 >> 172.21.15.18:0/3819150801 conn(0x5560b6f285d0 msgr2=0x5560b6f89bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 -- 172.21.15.18:0/3819150801 shutdown_connections 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.052+0000 7f4c6b23e740 1 -- 172.21.15.18:0/3819150801 wait complete. 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.053+0000 7f4c6b23e740 1 -- start start 2024-04-16T17:12:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.053+0000 7f4c6b23e740 1 Processor -- start 2024-04-16T17:12:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f9f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f90770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c6b23e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 0x5560b6f90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601afc30 con 0x5560b6f8da80 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601b10a0 con 0x5560b6f97000 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c6b23e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601b0300 con 0x5560b6f8e7f0 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c6b23e740 10 client.0 _ll_register_callbacks cb 0x7f4c54005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f9f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c65200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f90770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c65c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f9f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36906/0 (socket says 172.21.15.18:36906) 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c65200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f90770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41680/0 (socket says 172.21.15.18:41680) 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c65c00640 1 -- 172.21.15.18:0/4224942796 learned_addr learned my addr 172.21.15.18:0/4224942796 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.054+0000 7f4c66600640 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 0x5560b6f90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c65200640 1 -- 172.21.15.18:0/4224942796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 msgr2=0x5560b6f90cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c65200640 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 0x5560b6f90cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c65200640 1 -- 172.21.15.18:0/4224942796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 msgr2=0x5560b6f9f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c65200640 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f9f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c65200640 1 -- 172.21.15.18:0/4224942796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b6f911f0 con 0x5560b6f8e7f0 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c65c00640 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f9f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c65200640 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f90770 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x5560b6f89900 tx=0x7f4c4c004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e75fc91ae8e697 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c47400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c47400640 1 -- 172.21.15.18:0/4224942796 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c003cd0 con 0x5560b6f8e7f0 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c6b23e740 1 -- 172.21.15.18:0/4224942796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c54087050 con 0x5560b6f8e7f0 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.055+0000 7f4c47400640 1 -- 172.21.15.18:0/4224942796 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9466+0+0 (secure 0 0 0) 0x7f4c4c005020 con 0x5560b6f8e7f0 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c47400640 1 -- 172.21.15.18:0/4224942796 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c4c02d050 con 0x5560b6f8e7f0 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c6b23e740 10 client.27496 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c6b23e740 1 -- 172.21.15.18:0/4224942796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5560b700cfb0 con 0x5560b6f8e7f0 2024-04-16T17:12:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c47400640 1 -- 172.21.15.18:0/4224942796 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54087050 con 0x5560b6f8e7f0 2024-04-16T17:12:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c47400640 1 -- 172.21.15.18:0/4224942796 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4c4c0200e0 con 0x5560b6f8e7f0 2024-04-16T17:12:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c6b23e740 10 client.27496 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c6b23e740 10 client.27496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c6b23e740 20 client.27496 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5560b700d6d0 0x5560b700fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c6b23e740 4 client.27496 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.056+0000 7f4c6b23e740 1 -- 172.21.15.18:0/4224942796 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560b6e76530 con 0x5560b700d6d0 2024-04-16T17:12:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.057+0000 7f4c65c00640 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5560b700d6d0 0x5560b700fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.057+0000 7f4c65c00640 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5560b700d6d0 0x5560b700fac0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29474117e4e8c694 server_cookie=54ad865545e8346 in_seq=0 out_seq=0 2024-04-16T17:12:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.057+0000 7f4c47400640 10 client.27496 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:04.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:03 smithi018 ceph-mon[31776]: pgmap v226: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-04-16T17:12:04.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:03 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:12:04.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:03 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:12:04.205 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:03 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:12:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.222+0000 7f4c47400640 1 -- 172.21.15.18:0/4224942796 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560b6e76530 con 0x5560b700d6d0 2024-04-16T17:12:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.222+0000 7f4c47400640 10 client.27496 handle_command_reply: tid=0 2024-04-16T17:12:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.222+0000 7f4c6b23e740 1 client.27496 shutdown 2024-04-16T17:12:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.222+0000 7f4c6b23e740 1 -- 172.21.15.18:0/4224942796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 msgr2=0x5560b6f90770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.222+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f90770 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x5560b6f89900 tx=0x7f4c4c004410 comp rx=0 tx=0).stop 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 1 -- 172.21.15.18:0/4224942796 shutdown_connections 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5560b700d6d0 0x5560b700fac0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560b6f97000 0x5560b6f90cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5560b6f8e7f0 0x5560b6f90770 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 1 --2- 172.21.15.18:0/4224942796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5560b6f8da80 0x5560b6f9f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 1 -- 172.21.15.18:0/4224942796 >> 172.21.15.18:0/4224942796 conn(0x5560b6f285d0 msgr2=0x5560b6f92ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c47400640 0 client.27496 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 1 -- 172.21.15.18:0/4224942796 shutdown_connections 2024-04-16T17:12:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 1 -- 172.21.15.18:0/4224942796 wait complete. 2024-04-16T17:12:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.223+0000 7f4c6b23e740 20 client.27496 trim_cache size 0 max 16384 2024-04-16T17:12:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:12:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:12:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:12:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 -- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c480a3d00 msgr2=0x7f4c480a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c480a3d00 0x7f4c480a61c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009a60 tx=0x7f4c58005d80 comp rx=0 tx=0).stop 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 -- 172.21.15.18:0/1865517638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c60102080 msgr2=0x7f4c601a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c60102080 0x7f4c601a0580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400e9f0 tx=0x7f4c5400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 -- 172.21.15.18:0/1865517638 shutdown_connections 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c480a3d00 0x7f4c480a61c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c6019a790 0x7f4c6019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60107150 0x7f4c601a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 --2- 172.21.15.18:0/1865517638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c60102080 0x7f4c601a0580 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.224+0000 7f4c67000640 1 -- 172.21.15.18:0/1865517638 >> 172.21.15.18:0/1865517638 conn(0x7f4c600fd940 msgr2=0x7f4c600fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.225+0000 7f4c67000640 1 -- 172.21.15.18:0/1865517638 shutdown_connections 2024-04-16T17:12:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:04.225+0000 7f4c67000640 1 -- 172.21.15.18:0/1865517638 wait complete. 2024-04-16T17:12:04.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 280601 3.94 MB/sec execute 257 sec latency 87.806 ms 2024-04-16T17:12:05.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 280714 3.93 MB/sec execute 258 sec latency 79.671 ms 2024-04-16T17:12:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:05 smithi022 ceph-mon[36023]: pgmap v227: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-04-16T17:12:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:05 smithi022 ceph-mon[36023]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T17:12:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:05 smithi043 ceph-mon[36901]: pgmap v227: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-04-16T17:12:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:05 smithi043 ceph-mon[36901]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T17:12:06.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 281211 3.92 MB/sec execute 259 sec latency 49.003 ms 2024-04-16T17:12:06.961 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:05 smithi018 ceph-mon[31776]: pgmap v227: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-04-16T17:12:06.961 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:05 smithi018 ceph-mon[31776]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T17:12:07.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 282442 3.92 MB/sec execute 260 sec latency 16.496 ms 2024-04-16T17:12:07.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:12:07.878 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:12:07.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:07 smithi022 ceph-mon[36023]: pgmap v229: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-16T17:12:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:07 smithi043 ceph-mon[36901]: pgmap v229: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-16T17:12:08.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.081+0000 7ff441e00640 1 -- 172.21.15.18:0/1378968467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 msgr2=0x7ff43c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.081+0000 7ff441e00640 1 --2- 172.21.15.18:0/1378968467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 0x7ff43c10bde0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff428009900 tx=0x7ff42802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.082+0000 7ff441e00640 1 -- 172.21.15.18:0/1378968467 shutdown_connections 2024-04-16T17:12:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.082+0000 7ff441e00640 1 --2- 172.21.15.18:0/1378968467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c10c400 0x7ff43c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.082+0000 7ff441e00640 1 --2- 172.21.15.18:0/1378968467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 0x7ff43c10bde0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.082+0000 7ff441e00640 1 --2- 172.21.15.18:0/1378968467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43c102fe0 0x7ff43c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.082+0000 7ff441e00640 1 -- 172.21.15.18:0/1378968467 >> 172.21.15.18:0/1378968467 conn(0x7ff43c077ab0 msgr2=0x7ff43c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.082+0000 7ff441e00640 1 -- 172.21.15.18:0/1378968467 shutdown_connections 2024-04-16T17:12:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.082+0000 7ff441e00640 1 -- 172.21.15.18:0/1378968467 wait complete. 2024-04-16T17:12:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.083+0000 7ff441e00640 1 Processor -- start 2024-04-16T17:12:08.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.083+0000 7ff441e00640 1 -- start start 2024-04-16T17:12:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff441e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c102fe0 0x7ff43c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff441e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 0x7ff43c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff441e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43c10c400 0x7ff43c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff441e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1121e0 con 0x7ff43c103900 2024-04-16T17:12:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff441e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c112060 con 0x7ff43c102fe0 2024-04-16T17:12:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff441e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c112360 con 0x7ff43c10c400 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff440a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c102fe0 0x7ff43c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff440a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c102fe0 0x7ff43c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32850/0 (socket says 172.21.15.18:32850) 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff440a00640 1 -- 172.21.15.18:0/3301777764 learned_addr learned my addr 172.21.15.18:0/3301777764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff43be00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 0x7ff43c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff441400640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43c10c400 0x7ff43c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff440a00640 1 -- 172.21.15.18:0/3301777764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43c10c400 msgr2=0x7ff43c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff440a00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43c10c400 0x7ff43c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff440a00640 1 -- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 msgr2=0x7ff43c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff440a00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 0x7ff43c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff440a00640 1 -- 172.21.15.18:0/3301777764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c19afb0 con 0x7ff43c102fe0 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff43be00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 0x7ff43c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.084+0000 7ff441400640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43c10c400 0x7ff43c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.085+0000 7ff440a00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c102fe0 0x7ff43c1a0620 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff43000e9f0 tx=0x7ff43000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.085+0000 7ff439600640 1 -- 172.21.15.18:0/3301777764 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43000cd80 con 0x7ff43c102fe0 2024-04-16T17:12:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.085+0000 7ff441e00640 1 -- 172.21.15.18:0/3301777764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c19b2a0 con 0x7ff43c102fe0 2024-04-16T17:12:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.085+0000 7ff439600640 1 -- 172.21.15.18:0/3301777764 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff430010910 con 0x7ff43c102fe0 2024-04-16T17:12:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.085+0000 7ff439600640 1 -- 172.21.15.18:0/3301777764 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300188f0 con 0x7ff43c102fe0 2024-04-16T17:12:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.085+0000 7ff441e00640 1 -- 172.21.15.18:0/3301777764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a7470 con 0x7ff43c102fe0 2024-04-16T17:12:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.089+0000 7ff439600640 1 -- 172.21.15.18:0/3301777764 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff430010ac0 con 0x7ff43c102fe0 2024-04-16T17:12:08.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.089+0000 7ff4461c3740 1 Processor -- start 2024-04-16T17:12:08.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.089+0000 7ff4461c3740 1 -- start start 2024-04-16T17:12:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.089+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab46c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff4461c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56260aaa1120 con 0x56260ab3b390 2024-04-16T17:12:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff4461c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56260aa47340 con 0x56260ab44890 2024-04-16T17:12:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff4461c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56260aa9d0e0 con 0x56260ab3bcb0 2024-04-16T17:12:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff441400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff441400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42124/0 (socket says 172.21.15.18:42124) 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff441400640 1 -- 172.21.15.18:0/2149372207 learned_addr learned my addr 172.21.15.18:0/2149372207 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff43be00640 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab46c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff440a00640 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff441400640 1 -- 172.21.15.18:0/2149372207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 msgr2=0x56260ab44350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff441400640 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab44350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff441400640 1 -- 172.21.15.18:0/2149372207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 msgr2=0x56260ab46c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.090+0000 7ff441400640 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab46c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff441400640 1 -- 172.21.15.18:0/2149372207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260a87d7d0 con 0x56260ab3b390 2024-04-16T17:12:08.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff440a00640 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab44350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff439600640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff4240a3d60 0x7ff4240a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff441400640 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3b770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff434009ee0 tx=0x7ff43400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67f6c3bd53b7a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff439600640 1 -- 172.21.15.18:0/3301777764 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff4300d11c0 con 0x7ff43c102fe0 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff440a00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff4240a3d60 0x7ff4240a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff42dc00640 1 -- 172.21.15.18:0/2149372207 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434011640 con 0x56260ab3b390 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff42dc00640 1 -- 172.21.15.18:0/2149372207 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff434011cc0 con 0x56260ab3b390 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.091+0000 7ff42dc00640 1 -- 172.21.15.18:0/2149372207 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43401be40 con 0x56260ab3b390 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.092+0000 7ff4461c3740 1 -- 172.21.15.18:0/2149372207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 msgr2=0x56260ab3b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.092+0000 7ff4461c3740 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3b770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff434009ee0 tx=0x7ff43400f730 comp rx=0 tx=0).stop 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.092+0000 7ff440a00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff4240a3d60 0x7ff4240a6220 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x56260ab3ae60 tx=0x7ff43001f000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.092+0000 7ff4461c3740 1 -- 172.21.15.18:0/2149372207 shutdown_connections 2024-04-16T17:12:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.092+0000 7ff4461c3740 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab46c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.092+0000 7ff4461c3740 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab44350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.092+0000 7ff4461c3740 1 --2- 172.21.15.18:0/2149372207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3b770 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.092+0000 7ff4461c3740 1 -- 172.21.15.18:0/2149372207 >> 172.21.15.18:0/2149372207 conn(0x56260ab35ac0 msgr2=0x56260ab35ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.093+0000 7ff4461c3740 1 -- 172.21.15.18:0/2149372207 shutdown_connections 2024-04-16T17:12:08.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.093+0000 7ff4461c3740 1 -- 172.21.15.18:0/2149372207 wait complete. 2024-04-16T17:12:08.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.093+0000 7ff4461c3740 1 -- start start 2024-04-16T17:12:08.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.094+0000 7ff4461c3740 1 Processor -- start 2024-04-16T17:12:08.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.094+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 0x56260ab41980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.094+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab3bcb0 0x56260ab41ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.094+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab44890 0x56260ab42400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.094+0000 7ff4461c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56260ab47680 con 0x56260ab44890 2024-04-16T17:12:08.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.094+0000 7ff4461c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56260aa5a710 con 0x56260ab3bcb0 2024-04-16T17:12:08.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.094+0000 7ff4461c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56260ab47800 con 0x56260ab3b390 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.094+0000 7ff4461c3740 10 client.0 _ll_register_callbacks cb 0x56260aa55af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.095+0000 7ff441400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 0x56260ab41980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.095+0000 7ff43be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab3bcb0 0x56260ab41ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.095+0000 7ff43be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab3bcb0 0x56260ab41ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32874/0 (socket says 172.21.15.18:32874) 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.095+0000 7ff43be00640 1 -- 172.21.15.18:0/4025949423 learned_addr learned my addr 172.21.15.18:0/4025949423 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.095+0000 7ff43be00640 1 -- 172.21.15.18:0/4025949423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 msgr2=0x56260ab41980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.095+0000 7ff43be00640 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 0x56260ab41980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.095+0000 7ff441400640 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab44890 0x56260ab42400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.095+0000 7ff43be00640 1 -- 172.21.15.18:0/4025949423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab44890 msgr2=0x56260ab42400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.096+0000 7ff43be00640 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab44890 0x56260ab42400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.096+0000 7ff43be00640 1 -- 172.21.15.18:0/4025949423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260a7b33f0 con 0x56260ab3bcb0 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.096+0000 7ff441400640 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab44890 0x56260ab42400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.096+0000 7ff43be00640 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab3bcb0 0x56260ab41ec0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff428009230 tx=0x7ff428005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d70a496a6ca3f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.096+0000 7ff422000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.096+0000 7ff422000640 1 -- 172.21.15.18:0/4025949423 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42802cdf0 con 0x56260ab3bcb0 2024-04-16T17:12:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.096+0000 7ff4461c3740 1 -- 172.21.15.18:0/4025949423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff434011cc0 con 0x56260ab3bcb0 2024-04-16T17:12:08.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.097+0000 7ff422000640 1 -- 172.21.15.18:0/4025949423 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff428005030 con 0x56260ab3bcb0 2024-04-16T17:12:08.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.097+0000 7ff422000640 1 -- 172.21.15.18:0/4025949423 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff42804e050 con 0x56260ab3bcb0 2024-04-16T17:12:08.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.097+0000 7ff4461c3740 10 client.27581 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:08.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.097+0000 7ff4461c3740 1 -- 172.21.15.18:0/4025949423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56260a7730f0 con 0x56260ab3bcb0 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.097+0000 7ff422000640 1 -- 172.21.15.18:0/4025949423 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff434011cc0 con 0x56260ab3bcb0 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.097+0000 7ff422000640 1 -- 172.21.15.18:0/4025949423 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4280035f0 con 0x56260ab3bcb0 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.097+0000 7ff4461c3740 10 client.27581 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.098+0000 7ff4461c3740 10 client.27581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.098+0000 7ff4461c3740 20 client.27581 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.098+0000 7ff4461c3740 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56260ab54cc0 0x56260ab57140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.098+0000 7ff4461c3740 4 client.27581 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.098+0000 7ff4461c3740 1 -- 172.21.15.18:0/4025949423 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56260aaaeb40 con 0x56260ab54cc0 2024-04-16T17:12:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.098+0000 7ff441400640 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56260ab54cc0 0x56260ab57140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.098+0000 7ff422000640 10 client.27581 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.099+0000 7ff441400640 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56260ab54cc0 0x56260ab57140 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5fc1b8b1d01cb9 server_cookie=ca63da7619aae833 in_seq=0 out_seq=0 2024-04-16T17:12:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.099+0000 7ff422000640 1 -- 172.21.15.18:0/4025949423 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56260aaaeb40 con 0x56260ab54cc0 2024-04-16T17:12:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.099+0000 7ff422000640 10 client.27581 handle_command_reply: tid=0 2024-04-16T17:12:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.099+0000 7ff4461c3740 1 client.27581 shutdown 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 -- 172.21.15.18:0/4025949423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab3bcb0 msgr2=0x56260ab41ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab3bcb0 0x56260ab41ec0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff428009230 tx=0x7ff428005bd0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 -- 172.21.15.18:0/4025949423 shutdown_connections 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56260ab54cc0 0x56260ab57140 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff422000640 0 client.27581 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab44890 0x56260ab42400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab3bcb0 0x56260ab41ec0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 --2- 172.21.15.18:0/4025949423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 0x56260ab41980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 -- 172.21.15.18:0/4025949423 >> 172.21.15.18:0/4025949423 conn(0x56260ab35ac0 msgr2=0x56260ab37810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 -- 172.21.15.18:0/4025949423 shutdown_connections 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 1 -- 172.21.15.18:0/4025949423 wait complete. 2024-04-16T17:12:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.101+0000 7ff4461c3740 20 client.27581 trim_cache size 0 max 16384 2024-04-16T17:12:08.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.140+0000 7ff4461c3740 1 Processor -- start 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.140+0000 7ff4461c3740 1 -- start start 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.140+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 0x56260ab440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.140+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3bcb0 0x56260ab408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab50540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff4461c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1b4190 con 0x56260ab3bcb0 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff4461c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1b1a60 con 0x56260ab44890 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff4461c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1af8d0 con 0x56260ab3b390 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3bcb0 0x56260ab408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3bcb0 0x56260ab408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42142/0 (socket says 172.21.15.18:42142) 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 -- 172.21.15.18:0/413148099 learned_addr learned my addr 172.21.15.18:0/413148099 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff441400640 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 0x56260ab440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff441400640 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab50540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 -- 172.21.15.18:0/413148099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 msgr2=0x56260ab440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 0x56260ab440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 -- 172.21.15.18:0/413148099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 msgr2=0x56260ab50540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab50540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 -- 172.21.15.18:0/413148099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260ab577c0 con 0x56260ab3bcb0 2024-04-16T17:12:08.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.141+0000 7ff43be00640 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3bcb0 0x56260ab408c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff4280092a0 tx=0x7ff4280058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58368684669a10e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.142+0000 7ff42d200640 1 -- 172.21.15.18:0/413148099 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428057070 con 0x56260ab3bcb0 2024-04-16T17:12:08.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.142+0000 7ff42d200640 1 -- 172.21.15.18:0/413148099 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff428003e80 con 0x56260ab3bcb0 2024-04-16T17:12:08.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.142+0000 7ff42d200640 1 -- 172.21.15.18:0/413148099 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280050d0 con 0x56260ab3bcb0 2024-04-16T17:12:08.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 -- 172.21.15.18:0/413148099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3bcb0 msgr2=0x56260ab408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3bcb0 0x56260ab408c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff4280092a0 tx=0x7ff4280058b0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 -- 172.21.15.18:0/413148099 shutdown_connections 2024-04-16T17:12:08.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab50540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3bcb0 0x56260ab408c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 --2- 172.21.15.18:0/413148099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3b390 0x56260ab440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 -- 172.21.15.18:0/413148099 >> 172.21.15.18:0/413148099 conn(0x56260ab35ac0 msgr2=0x56260ab421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 -- 172.21.15.18:0/413148099 shutdown_connections 2024-04-16T17:12:08.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 -- 172.21.15.18:0/413148099 wait complete. 2024-04-16T17:12:08.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.143+0000 7ff4461c3740 1 -- start start 2024-04-16T17:12:08.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.144+0000 7ff4461c3740 1 Processor -- start 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.144+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.144+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab3ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.144+0000 7ff4461c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab3efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.144+0000 7ff4461c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1aff20 con 0x56260ab3b390 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.144+0000 7ff4461c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1b1390 con 0x56260ab44890 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.144+0000 7ff4461c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1b05f0 con 0x56260ab3bcb0 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff4461c3740 10 client.0 _ll_register_callbacks cb 0x7ff430005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff43be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab3ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff43be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab3ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39182/0 (socket says 172.21.15.18:39182) 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff43be00640 1 -- 172.21.15.18:0/1510873218 learned_addr learned my addr 172.21.15.18:0/1510873218 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff441400640 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff441400640 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab3efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff43be00640 1 -- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 msgr2=0x56260ab3efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff43be00640 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab3efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff43be00640 1 -- 172.21.15.18:0/1510873218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 msgr2=0x56260ab3e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff43be00640 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff43be00640 1 -- 172.21.15.18:0/1510873218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260abba280 con 0x56260ab3bcb0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.145+0000 7ff441400640 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab3efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.146+0000 7ff43be00640 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab3ea90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff428003b80 tx=0x7ff428009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99f48d04a50db6d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.146+0000 7ff422000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.146+0000 7ff422000640 1 -- 172.21.15.18:0/1510873218 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428057070 con 0x56260ab3bcb0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.146+0000 7ff422000640 1 -- 172.21.15.18:0/1510873218 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff428005450 con 0x56260ab3bcb0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.146+0000 7ff4461c3740 1 -- 172.21.15.18:0/1510873218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56260aaa89b0 con 0x56260ab3bcb0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.146+0000 7ff422000640 1 -- 172.21.15.18:0/1510873218 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff428005670 con 0x56260ab3bcb0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.146+0000 7ff4461c3740 10 client.27511 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.146+0000 7ff4461c3740 1 -- 172.21.15.18:0/1510873218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56260abbb490 con 0x56260ab3bcb0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff422000640 1 -- 172.21.15.18:0/1510873218 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56260aaa89b0 con 0x56260ab3bcb0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff422000640 1 -- 172.21.15.18:0/1510873218 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff42803f740 con 0x56260ab3bcb0 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff4461c3740 10 client.27511 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff4461c3740 10 client.27511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:08.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff4461c3740 20 client.27511 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:08.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff4461c3740 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56260abbbb30 0x56260abbdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff4461c3740 4 client.27511 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:12:08.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff4461c3740 1 -- 172.21.15.18:0/1510873218 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56260aa24420 con 0x56260abbbb30 2024-04-16T17:12:08.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.147+0000 7ff441400640 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56260abbbb30 0x56260abbdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.148+0000 7ff422000640 10 client.27511 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:08.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.148+0000 7ff441400640 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56260abbbb30 0x56260abbdf20 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4071e52be932c9c6 server_cookie=4c49262d7bafa680 in_seq=0 out_seq=0 2024-04-16T17:12:08.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.148+0000 7ff422000640 1 -- 172.21.15.18:0/1510873218 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56260aa24420 con 0x56260abbbb30 2024-04-16T17:12:08.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.148+0000 7ff422000640 10 client.27511 handle_command_reply: tid=0 2024-04-16T17:12:08.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.148+0000 7ff4461c3740 1 client.27511 shutdown 2024-04-16T17:12:08.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.148+0000 7ff4461c3740 1 -- 172.21.15.18:0/1510873218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 msgr2=0x56260ab3ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.148+0000 7ff4461c3740 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab3ea90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff428003b80 tx=0x7ff428009600 comp rx=0 tx=0).stop 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 1 -- 172.21.15.18:0/1510873218 shutdown_connections 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56260abbbb30 0x56260abbdf20 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56260ab44890 0x56260ab3efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56260ab3bcb0 0x56260ab3ea90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff422000640 0 client.27511 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 1 --2- 172.21.15.18:0/1510873218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56260ab3b390 0x56260ab3e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 1 -- 172.21.15.18:0/1510873218 >> 172.21.15.18:0/1510873218 conn(0x56260ab35ac0 msgr2=0x56260ab36da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 1 -- 172.21.15.18:0/1510873218 shutdown_connections 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 1 -- 172.21.15.18:0/1510873218 wait complete. 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.149+0000 7ff4461c3740 20 client.27511 trim_cache size 0 max 16384 2024-04-16T17:12:08.151 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:12:08.152 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:12:08.152 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:12:08.152 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:12:08.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 -- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff4240a3d60 msgr2=0x7ff4240a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff4240a3d60 0x7ff4240a6220 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x56260ab3ae60 tx=0x7ff43001f000 comp rx=0 tx=0).stop 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 -- 172.21.15.18:0/3301777764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c102fe0 msgr2=0x7ff43c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c102fe0 0x7ff43c1a0620 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff43000e9f0 tx=0x7ff43000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 -- 172.21.15.18:0/3301777764 shutdown_connections 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff4240a3d60 0x7ff4240a6220 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43c10c400 0x7ff43c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff43c103900 0x7ff43c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 --2- 172.21.15.18:0/3301777764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c102fe0 0x7ff43c1a0620 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 -- 172.21.15.18:0/3301777764 >> 172.21.15.18:0/3301777764 conn(0x7ff43c077ab0 msgr2=0x7ff43c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 -- 172.21.15.18:0/3301777764 shutdown_connections 2024-04-16T17:12:08.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.150+0000 7ff441e00640 1 -- 172.21.15.18:0/3301777764 wait complete. 2024-04-16T17:12:08.166 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:12:08.167 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:12:08.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 -- 172.21.15.18:0/2420597051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 msgr2=0x7fc10c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 --2- 172.21.15.18:0/2420597051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 0x7fc10c104730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009900 tx=0x7fc0fc02bc40 comp rx=0 tx=0).stop 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 -- 172.21.15.18:0/2420597051 shutdown_connections 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 --2- 172.21.15.18:0/2420597051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc10c109420 0x7fc10c110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 --2- 172.21.15.18:0/2420597051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc10c104d00 0x7fc10c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 --2- 172.21.15.18:0/2420597051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 0x7fc10c104730 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 -- 172.21.15.18:0/2420597051 >> 172.21.15.18:0/2420597051 conn(0x7fc10c0ffb70 msgr2=0x7fc10c101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 -- 172.21.15.18:0/2420597051 shutdown_connections 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.353+0000 7fc110c00640 1 -- 172.21.15.18:0/2420597051 wait complete. 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.354+0000 7fc110c00640 1 Processor -- start 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.354+0000 7fc110c00640 1 -- start start 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc110c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 0x7fc10c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc110c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc10c109420 0x7fc10c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc110c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc10c19a830 0x7fc10c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc110c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c114530 con 0x7fc10c19a830 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc110c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1143b0 con 0x7fc10c109420 2024-04-16T17:12:08.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc110c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c1146b0 con 0x7fc10c104350 2024-04-16T17:12:08.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc10b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 0x7fc10c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc10b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 0x7fc10c1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39204/0 (socket says 172.21.15.18:39204) 2024-04-16T17:12:08.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc10b400640 1 -- 172.21.15.18:0/324295756 learned_addr learned my addr 172.21.15.18:0/324295756 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc10be00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc10c19a830 0x7fc10c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.355+0000 7fc10aa00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc10c109420 0x7fc10c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc10b400640 1 -- 172.21.15.18:0/324295756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc10c109420 msgr2=0x7fc10c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc10b400640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc10c109420 0x7fc10c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc10b400640 1 -- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc10c19a830 msgr2=0x7fc10c19ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc10b400640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc10c19a830 0x7fc10c19ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc10b400640 1 -- 172.21.15.18:0/324295756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c19b3c0 con 0x7fc10c104350 2024-04-16T17:12:08.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc10be00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc10c19a830 0x7fc10c19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:08.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc10b400640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 0x7fc10c1a06c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005ab0 tx=0x7fc0fc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc103e00640 1 -- 172.21.15.18:0/324295756 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc041070 con 0x7fc10c104350 2024-04-16T17:12:08.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc103e00640 1 -- 172.21.15.18:0/324295756 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0fc02cca0 con 0x7fc10c104350 2024-04-16T17:12:08.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc103e00640 1 -- 172.21.15.18:0/324295756 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc003d90 con 0x7fc10c104350 2024-04-16T17:12:08.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.356+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c19b650 con 0x7fc10c104350 2024-04-16T17:12:08.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.357+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c1a77b0 con 0x7fc10c104350 2024-04-16T17:12:08.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.358+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10c105820 con 0x7fc10c104350 2024-04-16T17:12:08.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.362+0000 7fc103e00640 1 -- 172.21.15.18:0/324295756 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc0fc044070 con 0x7fc10c104350 2024-04-16T17:12:08.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.363+0000 7fc103e00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0f00a3d90 0x7fc0f00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.364+0000 7fc103e00640 1 -- 172.21.15.18:0/324295756 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7fc0fc0f27e0 con 0x7fc10c104350 2024-04-16T17:12:08.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.364+0000 7fc10aa00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0f00a3d90 0x7fc0f00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.364+0000 7fc103e00640 1 -- 172.21.15.18:0/324295756 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0fc0be4e0 con 0x7fc10c104350 2024-04-16T17:12:08.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.365+0000 7fc10aa00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0f00a3d90 0x7fc0f00a6250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80095f0 tx=0x7fc0f8005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 283515 3.94 MB/sec execute 261 sec latency 64.653 ms 2024-04-16T17:12:08.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.632+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc10c10c650 con 0x7fc10c104350 2024-04-16T17:12:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.640+0000 7fc103e00640 1 -- 172.21.15.18:0/324295756 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11851 (secure 0 0 0) 0x7fc0fc0c27e0 con 0x7fc10c104350 2024-04-16T17:12:08.640 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:12:08.640 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:11:38.382274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27445":{"gid":27445,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/678063310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":678063310},{"type":"v1","addr":"172.21.15.22:6837","nonce":678063310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:12:08.640 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 27 2024-04-16T17:12:08.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0f00a3d90 msgr2=0x7fc0f00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0f00a3d90 0x7fc0f00a6250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80095f0 tx=0x7fc0f8005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 msgr2=0x7fc10c1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 0x7fc10c1a06c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005ab0 tx=0x7fc0fc0042f0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 shutdown_connections 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc0f00a3d90 0x7fc0f00a6250 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc10c19a830 0x7fc10c19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc10c109420 0x7fc10c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 --2- 172.21.15.18:0/324295756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc10c104350 0x7fc10c1a06c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.643+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 >> 172.21.15.18:0/324295756 conn(0x7fc10c0ffb70 msgr2=0x7fc10c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.644+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 shutdown_connections 2024-04-16T17:12:08.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.644+0000 7fc110c00640 1 -- 172.21.15.18:0/324295756 wait complete. 2024-04-16T17:12:08.669 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 -- 172.21.15.18:0/913633997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c8074cd0 msgr2=0x7ff7c810e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 --2- 172.21.15.18:0/913633997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c8074cd0 0x7ff7c810e180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009900 tx=0x7ff7b402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 -- 172.21.15.18:0/913633997 shutdown_connections 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 --2- 172.21.15.18:0/913633997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c810e6c0 0x7ff7c8110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 --2- 172.21.15.18:0/913633997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c8074cd0 0x7ff7c810e180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 --2- 172.21.15.18:0/913633997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c80743b0 0x7ff7c8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 -- 172.21.15.18:0/913633997 >> 172.21.15.18:0/913633997 conn(0x7ff7c80fd950 msgr2=0x7ff7c80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 -- 172.21.15.18:0/913633997 shutdown_connections 2024-04-16T17:12:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.898+0000 7ff7cd600640 1 -- 172.21.15.18:0/913633997 wait complete. 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.899+0000 7ff7cd600640 1 Processor -- start 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.899+0000 7ff7cd600640 1 -- start start 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7cd600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c80743b0 0x7ff7c81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7cd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c8074cd0 0x7ff7c81a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7cd600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c810e6c0 0x7ff7c819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7cd600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8114400 con 0x7ff7c80743b0 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7cd600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8114280 con 0x7ff7c8074cd0 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7cd600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8114580 con 0x7ff7c810e6c0 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c80743b0 0x7ff7c81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c8074cd0 0x7ff7c81a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c80743b0 0x7ff7c81a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42192/0 (socket says 172.21.15.18:42192) 2024-04-16T17:12:08.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7e00640 1 -- 172.21.15.18:0/199815868 learned_addr learned my addr 172.21.15.18:0/199815868 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7ccc00640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c810e6c0 0x7ff7c819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7400640 1 -- 172.21.15.18:0/199815868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c810e6c0 msgr2=0x7ff7c819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7400640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c810e6c0 0x7ff7c819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7400640 1 -- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c80743b0 msgr2=0x7ff7c81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7400640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c80743b0 0x7ff7c81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7400640 1 -- 172.21.15.18:0/199815868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c819af50 con 0x7ff7c8074cd0 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.900+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c80743b0 0x7ff7c81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.901+0000 7ff7c7400640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c8074cd0 0x7ff7c81a0b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff7b402c210 tx=0x7ff7b4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.901+0000 7ff7c4c00640 1 -- 172.21.15.18:0/199815868 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4042020 con 0x7ff7c8074cd0 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.901+0000 7ff7cd600640 1 -- 172.21.15.18:0/199815868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c819b1e0 con 0x7ff7c8074cd0 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.901+0000 7ff7c4c00640 1 -- 172.21.15.18:0/199815868 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b402ce00 con 0x7ff7c8074cd0 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.901+0000 7ff7c4c00640 1 -- 172.21.15.18:0/199815868 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4003990 con 0x7ff7c8074cd0 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.901+0000 7ff7cd600640 1 -- 172.21.15.18:0/199815868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c81a7460 con 0x7ff7c8074cd0 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.905+0000 7ff7d19c8740 1 Processor -- start 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.905+0000 7ff7c4c00640 1 -- 172.21.15.18:0/199815868 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff7b4045020 con 0x7ff7c8074cd0 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.905+0000 7ff7d19c8740 1 -- start start 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7dc410 0x55d9eb7de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d9eb738020 con 0x55d9eb7dc410 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d9eb6de460 con 0x55d9eb7d3830 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d9eb733fe0 con 0x55d9eb7d2f10 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7dbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32952/0 (socket says 172.21.15.18:32952) 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7e00640 1 -- 172.21.15.18:0/2720804449 learned_addr learned my addr 172.21.15.18:0/2720804449 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7dc410 0x55d9eb7de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7e00640 1 -- 172.21.15.18:0/2720804449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7d2f10 msgr2=0x55d9eb7d32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7e00640 1 -- 172.21.15.18:0/2720804449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7dc410 msgr2=0x55d9eb7de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7dc410 0x55d9eb7de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7e00640 1 -- 172.21.15.18:0/2720804449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9eb6e6040 con 0x55d9eb7d3830 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.906+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7dc410 0x55d9eb7de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.907+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7dbed0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8011650 tx=0x7ff7b8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd5b9921dbe0c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.907+0000 7ff7bd200640 1 -- 172.21.15.18:0/2720804449 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b801e070 con 0x55d9eb7d3830 2024-04-16T17:12:08.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.907+0000 7ff7bd200640 1 -- 172.21.15.18:0/2720804449 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b800fba0 con 0x55d9eb7d3830 2024-04-16T17:12:08.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.907+0000 7ff7bd200640 1 -- 172.21.15.18:0/2720804449 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8019d20 con 0x55d9eb7d3830 2024-04-16T17:12:08.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.908+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2720804449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 msgr2=0x55d9eb7dbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.908+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7dbed0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8011650 tx=0x7ff7b8009f90 comp rx=0 tx=0).stop 2024-04-16T17:12:08.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.908+0000 7ff7c4c00640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7ac0a3d20 0x7ff7ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.908+0000 7ff7c4c00640 1 -- 172.21.15.18:0/199815868 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff7b40f26c0 con 0x7ff7c8074cd0 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2720804449 shutdown_connections 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7dc410 0x55d9eb7de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7dbed0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2720804449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2720804449 >> 172.21.15.18:0/2720804449 conn(0x55d9eb7cd640 msgr2=0x55d9eb7cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7ccc00640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7ac0a3d20 0x7ff7ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2720804449 shutdown_connections 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2720804449 wait complete. 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.909+0000 7ff7d19c8740 1 -- start start 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7ccc00640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7ac0a3d20 0x7ff7ac0a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00099a0 tx=0x7ff7c0004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7d19c8740 1 Processor -- start 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7d9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d9eb7df200 con 0x55d9eb7d3830 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d9eb6f1830 con 0x55d9eb7d2f10 2024-04-16T17:12:08.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d9eb7df380 con 0x55d9eb7dc410 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.910+0000 7ff7d19c8740 10 client.0 _ll_register_callbacks cb 0x55d9eb6f9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32962/0 (socket says 172.21.15.18:32962) 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7e00640 1 -- 172.21.15.18:0/2682338044 learned_addr learned my addr 172.21.15.18:0/2682338044 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7d9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7e00640 1 -- 172.21.15.18:0/2682338044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 msgr2=0x55d9eb7d9ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7d9ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7e00640 1 -- 172.21.15.18:0/2682338044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 msgr2=0x55d9eb7d9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7d9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7e00640 1 -- 172.21.15.18:0/2682338044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9eb6f5920 con 0x55d9eb7d2f10 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.911+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7d9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.912+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d9570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff7b801c040 tx=0x7ff7b8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f30e7774a96d9c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.912+0000 7ff7b1600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.912+0000 7ff7b1600640 1 -- 172.21.15.18:0/2682338044 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b801e040 con 0x55d9eb7d2f10 2024-04-16T17:12:08.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.912+0000 7ff7b1600640 1 -- 172.21.15.18:0/2682338044 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff7b800fd50 con 0x55d9eb7d2f10 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.912+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2682338044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7b800fba0 con 0x55d9eb7d2f10 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.913+0000 7ff7b1600640 1 -- 172.21.15.18:0/2682338044 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7b8030050 con 0x55d9eb7d2f10 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.913+0000 7ff7d19c8740 10 client.27602 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.913+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2682338044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9eb6c37f0 con 0x55d9eb7d2f10 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.913+0000 7ff7b1600640 1 -- 172.21.15.18:0/2682338044 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b800fba0 con 0x55d9eb7d2f10 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.914+0000 7ff7b1600640 1 -- 172.21.15.18:0/2682338044 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff7b8021070 con 0x55d9eb7d2f10 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.914+0000 7ff7d19c8740 10 client.27602 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.914+0000 7ff7d19c8740 10 client.27602 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.914+0000 7ff7d19c8740 20 client.27602 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.914+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d9eb7ec770 0x55d9eb7eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.914+0000 7ff7d19c8740 4 client.27602 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:08.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.914+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2682338044 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9eb745a40 con 0x55d9eb7ec770 2024-04-16T17:12:08.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.929+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d9eb7ec770 0x55d9eb7eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.959+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d9eb7ec770 0x55d9eb7eeb60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=269f826dc9da6711 server_cookie=2b3ac2e710b93b6a in_seq=0 out_seq=0 2024-04-16T17:12:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.959+0000 7ff7b1600640 10 client.27602 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7b1600640 1 -- 172.21.15.18:0/2682338044 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9eb745a40 con 0x55d9eb7ec770 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7b1600640 10 client.27602 handle_command_reply: tid=0 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 client.27602 shutdown 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2682338044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 msgr2=0x55d9eb7d9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d9570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff7b801c040 tx=0x7ff7b8005dc0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2682338044 shutdown_connections 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d9eb7ec770 0x55d9eb7eeb60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7d9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7d9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2682338044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7d9570 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2682338044 >> 172.21.15.18:0/2682338044 conn(0x55d9eb7cd640 msgr2=0x55d9eb7cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7b1600640 0 client.27602 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:12:08.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2682338044 shutdown_connections 2024-04-16T17:12:08.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.960+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2682338044 wait complete. 2024-04-16T17:12:08.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.961+0000 7ff7d19c8740 20 client.27602 trim_cache size 0 max 16384 2024-04-16T17:12:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.989+0000 7ff7d19c8740 1 Processor -- start 2024-04-16T17:12:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.989+0000 7ff7d19c8740 1 -- start start 2024-04-16T17:12:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7dbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b40d0 con 0x55d9eb7d3830 2024-04-16T17:12:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b19a0 con 0x55d9eb7d2f10 2024-04-16T17:12:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81af810 con 0x55d9eb7dc410 2024-04-16T17:12:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7dbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7c7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7ccc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7dbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32978/0 (socket says 172.21.15.18:32978) 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.990+0000 7ff7c7e00640 1 -- 172.21.15.18:0/3615026639 learned_addr learned my addr 172.21.15.18:0/3615026639 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7c7e00640 1 -- 172.21.15.18:0/3615026639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 msgr2=0x55d9eb7f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/3615026639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7c7e00640 1 -- 172.21.15.18:0/3615026639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 msgr2=0x55d9eb7e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/3615026639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7c7e00640 1 -- 172.21.15.18:0/3615026639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9eb44a3f0 con 0x55d9eb7d2f10 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7c7400640 1 --2- 172.21.15.18:0/3615026639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/3615026639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7dbca0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8000c00 tx=0x7ff7b8011550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24083da2a1eca0bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7b3e00640 1 -- 172.21.15.18:0/3615026639 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8015020 con 0x55d9eb7d2f10 2024-04-16T17:12:08.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7b3e00640 1 -- 172.21.15.18:0/3615026639 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8027070 con 0x55d9eb7d2f10 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.991+0000 7ff7b3e00640 1 -- 172.21.15.18:0/3615026639 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b801e040 con 0x55d9eb7d2f10 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.992+0000 7ff7d19c8740 1 -- 172.21.15.18:0/3615026639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 msgr2=0x55d9eb7dbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.992+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/3615026639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7dbca0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8000c00 tx=0x7ff7b8011550 comp rx=0 tx=0).stop 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 -- 172.21.15.18:0/3615026639 shutdown_connections 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/3615026639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/3615026639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/3615026639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7dbca0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 -- 172.21.15.18:0/3615026639 >> 172.21.15.18:0/3615026639 conn(0x55d9eb7cd640 msgr2=0x55d9eb7cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 -- 172.21.15.18:0/3615026639 shutdown_connections 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 -- 172.21.15.18:0/3615026639 wait complete. 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 -- start start 2024-04-16T17:12:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.993+0000 7ff7d19c8740 1 Processor -- start 2024-04-16T17:12:08.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7e45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7d19c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81afe60 con 0x55d9eb7d2f10 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b12d0 con 0x55d9eb7d3830 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7d19c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b0530 con 0x55d9eb7dc410 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7d19c8740 10 client.0 _ll_register_callbacks cb 0x7ff7b801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7e45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32984/0 (socket says 172.21.15.18:32984) 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7c7400640 1 -- 172.21.15.18:0/2229962276 learned_addr learned my addr 172.21.15.18:0/2229962276 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7c7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7e45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42226/0 (socket says 172.21.15.18:42226) 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7c7400640 1 -- 172.21.15.18:0/2229962276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 msgr2=0x55d9eb7d58a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.994+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7d58a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7c7400640 1 -- 172.21.15.18:0/2229962276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d2f10 msgr2=0x55d9eb7e45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7e45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:08.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7c7400640 1 -- 172.21.15.18:0/2229962276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9eb5147d0 con 0x55d9eb7d3830 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7c7400640 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e4ae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40be240 tx=0x7ff7b40055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec67be236c4ce564 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7b1600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7b1600640 1 -- 172.21.15.18:0/2229962276 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40be2b0 con 0x55d9eb7d3830 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7b1600640 1 -- 172.21.15.18:0/2229962276 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff7b4005250 con 0x55d9eb7d3830 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2229962276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7b8027070 con 0x55d9eb7d3830 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7b1600640 1 -- 172.21.15.18:0/2229962276 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7b40c2660 con 0x55d9eb7d3830 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7d19c8740 10 client.27608 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.995+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2229962276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9eb852370 con 0x55d9eb7d3830 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.996+0000 7ff7b1600640 1 -- 172.21.15.18:0/2229962276 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8027070 con 0x55d9eb7d3830 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.996+0000 7ff7b1600640 1 -- 172.21.15.18:0/2229962276 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff7b40c6a90 con 0x55d9eb7d3830 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.996+0000 7ff7d19c8740 10 client.27608 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.996+0000 7ff7d19c8740 10 client.27608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.996+0000 7ff7d19c8740 20 client.27608 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.996+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d9eb852aa0 0x55d9eb854ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.996+0000 7ff7d19c8740 4 client.27608 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:08.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:08.996+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2229962276 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9eb73f8b0 con 0x55d9eb852aa0 2024-04-16T17:12:08.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.000+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d9eb852aa0 0x55d9eb854ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.000+0000 7ff7c7e00640 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d9eb852aa0 0x55d9eb854ec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=330f6ab0260d9e8c server_cookie=3a9ad3d22b26b1c3 in_seq=0 out_seq=0 2024-04-16T17:12:09.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.000+0000 7ff7b1600640 10 client.27608 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:12:09.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.000+0000 7ff7b1600640 1 -- 172.21.15.18:0/2229962276 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9eb73f8b0 con 0x55d9eb852aa0 2024-04-16T17:12:09.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.000+0000 7ff7b1600640 10 client.27608 handle_command_reply: tid=0 2024-04-16T17:12:09.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 client.27608 shutdown 2024-04-16T17:12:09.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2229962276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 msgr2=0x55d9eb7e4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e4ae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40be240 tx=0x7ff7b40055f0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2229962276 shutdown_connections 2024-04-16T17:12:09.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d9eb852aa0 0x55d9eb854ec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9eb7dc410 0x55d9eb7d58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9eb7d3830 0x55d9eb7e4ae0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7b1600640 0 client.27608 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:12:09.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 --2- 172.21.15.18:0/2229962276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9eb7d2f10 0x55d9eb7e45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2229962276 >> 172.21.15.18:0/2229962276 conn(0x55d9eb7cd640 msgr2=0x55d9eb7d7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2229962276 shutdown_connections 2024-04-16T17:12:09.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 1 -- 172.21.15.18:0/2229962276 wait complete. 2024-04-16T17:12:09.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.001+0000 7ff7d19c8740 20 client.27608 trim_cache size 0 max 16384 2024-04-16T17:12:09.002 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 -- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7ac0a3d20 msgr2=0x7ff7ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7ac0a3d20 0x7ff7ac0a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00099a0 tx=0x7ff7c0004a40 comp rx=0 tx=0).stop 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 -- 172.21.15.18:0/199815868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c8074cd0 msgr2=0x7ff7c81a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c8074cd0 0x7ff7c81a0b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff7b402c210 tx=0x7ff7b4005e80 comp rx=0 tx=0).stop 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 -- 172.21.15.18:0/199815868 shutdown_connections 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff7ac0a3d20 0x7ff7ac0a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c810e6c0 0x7ff7c819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c8074cd0 0x7ff7c81a0b50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 --2- 172.21.15.18:0/199815868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7c80743b0 0x7ff7c81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 -- 172.21.15.18:0/199815868 >> 172.21.15.18:0/199815868 conn(0x7ff7c80fd950 msgr2=0x7ff7c80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.002+0000 7ff7cd600640 1 -- 172.21.15.18:0/199815868 shutdown_connections 2024-04-16T17:12:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.003+0000 7ff7cd600640 1 -- 172.21.15.18:0/199815868 wait complete. 2024-04-16T17:12:09.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 -- 172.21.15.18:0/611130206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102a20 msgr2=0x7fd648106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 --2- 172.21.15.18:0/611130206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102a20 0x7fd648106a60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd634009900 tx=0x7fd63402bc30 comp rx=0 tx=0).stop 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 -- 172.21.15.18:0/611130206 shutdown_connections 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 --2- 172.21.15.18:0/611130206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd648107140 0x7fd64810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 --2- 172.21.15.18:0/611130206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102a20 0x7fd648106a60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 --2- 172.21.15.18:0/611130206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd648102070 0x7fd648102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 -- 172.21.15.18:0/611130206 >> 172.21.15.18:0/611130206 conn(0x7fd6480fd950 msgr2=0x7fd6480ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 -- 172.21.15.18:0/611130206 shutdown_connections 2024-04-16T17:12:09.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.209+0000 7fd64ca00640 1 -- 172.21.15.18:0/611130206 wait complete. 2024-04-16T17:12:09.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.211+0000 7fd64ca00640 1 Processor -- start 2024-04-16T17:12:09.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.211+0000 7fd64ca00640 1 -- start start 2024-04-16T17:12:09.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.211+0000 7fd64ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102070 0x7fd6481a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.211+0000 7fd64ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd648107140 0x7fd6481a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.211+0000 7fd64ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd64819a790 0x7fd64819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.211+0000 7fd64ca00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd6481121d0 con 0x7fd648102070 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.211+0000 7fd64ca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd648112050 con 0x7fd64819a790 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.211+0000 7fd64ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd648112350 con 0x7fd648107140 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102070 0x7fd6481a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102070 0x7fd6481a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42254/0 (socket says 172.21.15.18:42254) 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd646a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd648107140 0x7fd6481a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647400640 1 -- 172.21.15.18:0/3546985454 learned_addr learned my addr 172.21.15.18:0/3546985454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647e00640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd64819a790 0x7fd64819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647400640 1 -- 172.21.15.18:0/3546985454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd648107140 msgr2=0x7fd6481a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647400640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd648107140 0x7fd6481a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647400640 1 -- 172.21.15.18:0/3546985454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd64819a790 msgr2=0x7fd64819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647400640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd64819a790 0x7fd64819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.212+0000 7fd647400640 1 -- 172.21.15.18:0/3546985454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64819b320 con 0x7fd648102070 2024-04-16T17:12:09.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.213+0000 7fd647400640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102070 0x7fd6481a0620 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd63800d770 tx=0x7fd63800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.213+0000 7fd63fe00640 1 -- 172.21.15.18:0/3546985454 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638017070 con 0x7fd648102070 2024-04-16T17:12:09.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.213+0000 7fd63fe00640 1 -- 172.21.15.18:0/3546985454 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd638005500 con 0x7fd648102070 2024-04-16T17:12:09.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.213+0000 7fd63fe00640 1 -- 172.21.15.18:0/3546985454 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638004e00 con 0x7fd648102070 2024-04-16T17:12:09.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.213+0000 7fd64ca00640 1 -- 172.21.15.18:0/3546985454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64819b610 con 0x7fd648102070 2024-04-16T17:12:09.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.213+0000 7fd64ca00640 1 -- 172.21.15.18:0/3546985454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6481a7770 con 0x7fd648102070 2024-04-16T17:12:09.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd650e39740 1 Processor -- start 2024-04-16T17:12:09.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd650e39740 1 -- start start 2024-04-16T17:12:09.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd650e39740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f66342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd650e39740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 0x5647f663ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd650e39740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f663d410 0x5647f663f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd650e39740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5647f6599020 con 0x5647f663d410 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd650e39740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5647f653f460 con 0x5647f6633f10 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd650e39740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5647f6594fe0 con 0x5647f6634830 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd63fe00640 1 -- 172.21.15.18:0/3546985454 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd638004fa0 con 0x7fd648102070 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.217+0000 7fd647e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f66342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f66342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33002/0 (socket says 172.21.15.18:33002) 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647e00640 1 -- 172.21.15.18:0/2750134687 learned_addr learned my addr 172.21.15.18:0/2750134687 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd646a00640 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f663d410 0x5647f663f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647400640 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 0x5647f663ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647e00640 1 -- 172.21.15.18:0/2750134687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 msgr2=0x5647f663ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647e00640 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 0x5647f663ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647e00640 1 -- 172.21.15.18:0/2750134687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f663d410 msgr2=0x5647f663f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647e00640 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f663d410 0x5647f663f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647e00640 1 -- 172.21.15.18:0/2750134687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f6547040 con 0x5647f6633f10 2024-04-16T17:12:09.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.218+0000 7fd647e00640 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f66342f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd640009f90 tx=0x7fd640010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5337448558d5aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.219+0000 7fd633e00640 1 -- 172.21.15.18:0/2750134687 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640011700 con 0x5647f6633f10 2024-04-16T17:12:09.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.219+0000 7fd633e00640 1 -- 172.21.15.18:0/2750134687 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd640011d80 con 0x5647f6633f10 2024-04-16T17:12:09.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.219+0000 7fd633e00640 1 -- 172.21.15.18:0/2750134687 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640018380 con 0x5647f6633f10 2024-04-16T17:12:09.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.219+0000 7fd63fe00640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd62c0a3d20 0x7fd62c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd63fe00640 1 -- 172.21.15.18:0/3546985454 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7fd6380d0630 con 0x7fd648102070 2024-04-16T17:12:09.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 -- 172.21.15.18:0/2750134687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 msgr2=0x5647f66342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f66342f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd640009f90 tx=0x7fd640010860 comp rx=0 tx=0).stop 2024-04-16T17:12:09.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd647400640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd62c0a3d20 0x7fd62c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 -- 172.21.15.18:0/2750134687 shutdown_connections 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f663d410 0x5647f663f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 0x5647f663ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 --2- 172.21.15.18:0/2750134687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f66342f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 -- 172.21.15.18:0/2750134687 >> 172.21.15.18:0/2750134687 conn(0x5647f662e640 msgr2=0x5647f662ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 -- 172.21.15.18:0/2750134687 shutdown_connections 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.220+0000 7fd650e39740 1 -- 172.21.15.18:0/2750134687 wait complete. 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.221+0000 7fd647400640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd62c0a3d20 0x7fd62c0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd6380cd9d0 tx=0x7fd638009c30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.221+0000 7fd650e39740 1 -- start start 2024-04-16T17:12:09.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.221+0000 7fd650e39740 1 Processor -- start 2024-04-16T17:12:09.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd650e39740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f663a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd650e39740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f663aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd650e39740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f663d410 0x5647f663afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd650e39740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5647f6640200 con 0x5647f6634830 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd650e39740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5647f6552830 con 0x5647f6633f10 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd650e39740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5647f6640380 con 0x5647f663d410 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd650e39740 10 client.0 _ll_register_callbacks cb 0x5647f655a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f663a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f663a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33016/0 (socket says 172.21.15.18:33016) 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 -- 172.21.15.18:0/3734820278 learned_addr learned my addr 172.21.15.18:0/3734820278 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd646a00640 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f663aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f663d410 0x5647f663afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 -- 172.21.15.18:0/3734820278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f663d410 msgr2=0x5647f663afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f663d410 0x5647f663afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 -- 172.21.15.18:0/3734820278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 msgr2=0x5647f663aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f663aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.222+0000 7fd647e00640 1 -- 172.21.15.18:0/3734820278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f6556920 con 0x5647f6633f10 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.223+0000 7fd647e00640 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f663a540 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd640019f70 tx=0x7fd640013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e39ae2aa0ce7537 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.223+0000 7fd630c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.223+0000 7fd630c00640 1 -- 172.21.15.18:0/3734820278 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640003970 con 0x5647f6633f10 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.223+0000 7fd630c00640 1 -- 172.21.15.18:0/3734820278 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7fd640003b10 con 0x5647f6633f10 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.223+0000 7fd650e39740 1 -- 172.21.15.18:0/3734820278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd640011d80 con 0x5647f6633f10 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.223+0000 7fd630c00640 1 -- 172.21.15.18:0/3734820278 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd640011500 con 0x5647f6633f10 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.223+0000 7fd650e39740 10 client.27620 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.223+0000 7fd650e39740 1 -- 172.21.15.18:0/3734820278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647f65247f0 con 0x5647f6633f10 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd630c00640 1 -- 172.21.15.18:0/3734820278 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd640011d80 con 0x5647f6633f10 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd630c00640 1 -- 172.21.15.18:0/3734820278 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd64001aad0 con 0x5647f6633f10 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd650e39740 10 client.27620 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd650e39740 10 client.27620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd650e39740 20 client.27620 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd650e39740 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5647f664d780 0x5647f664fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd650e39740 4 client.27620 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd650e39740 1 -- 172.21.15.18:0/3734820278 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647f65a6a40 con 0x5647f664d780 2024-04-16T17:12:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.224+0000 7fd646a00640 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5647f664d780 0x5647f664fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.225+0000 7fd646a00640 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5647f664d780 0x5647f664fb70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6954ad4e472ab6d5 server_cookie=fa05486af9998286 in_seq=0 out_seq=0 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.225+0000 7fd630c00640 10 client.27620 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.225+0000 7fd630c00640 1 -- 172.21.15.18:0/3734820278 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647f65a6a40 con 0x5647f664d780 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.225+0000 7fd630c00640 10 client.27620 handle_command_reply: tid=0 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 client.27620 shutdown 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 -- 172.21.15.18:0/3734820278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 msgr2=0x5647f663a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f663a540 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd640019f70 tx=0x7fd640013040 comp rx=0 tx=0).stop 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 -- 172.21.15.18:0/3734820278 shutdown_connections 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5647f664d780 0x5647f664fb70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f663d410 0x5647f663afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f663aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd630c00640 0 client.27620 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 --2- 172.21.15.18:0/3734820278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f6633f10 0x5647f663a540 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 -- 172.21.15.18:0/3734820278 >> 172.21.15.18:0/3734820278 conn(0x5647f662e640 msgr2=0x5647f6630390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 -- 172.21.15.18:0/3734820278 shutdown_connections 2024-04-16T17:12:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 1 -- 172.21.15.18:0/3734820278 wait complete. 2024-04-16T17:12:09.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.226+0000 7fd650e39740 20 client.27620 trim_cache size 0 max 16384 2024-04-16T17:12:09.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.249+0000 7fd650e39740 1 Processor -- start 2024-04-16T17:12:09.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.249+0000 7fd650e39740 1 -- start start 2024-04-16T17:12:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.249+0000 7fd650e39740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6633f10 0x5647f66490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.249+0000 7fd650e39740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f6655140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.249+0000 7fd650e39740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 0x5647f6655680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.249+0000 7fd650e39740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd6481b4150 con 0x5647f6634830 2024-04-16T17:12:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.249+0000 7fd650e39740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd6481b1a20 con 0x5647f663d410 2024-04-16T17:12:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.249+0000 7fd650e39740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6481af890 con 0x5647f6633f10 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f6655140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f6655140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42280/0 (socket says 172.21.15.18:42280) 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd647e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6633f10 0x5647f66490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 -- 172.21.15.18:0/4123724927 learned_addr learned my addr 172.21.15.18:0/4123724927 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd647e00640 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 0x5647f6655680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 -- 172.21.15.18:0/4123724927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6633f10 msgr2=0x5647f66490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6633f10 0x5647f66490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 -- 172.21.15.18:0/4123724927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 msgr2=0x5647f6655680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 0x5647f6655680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 -- 172.21.15.18:0/4123724927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f62ab3f0 con 0x5647f6634830 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd647e00640 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6633f10 0x5647f66490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.250+0000 7fd646a00640 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f6655140 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd634009900 tx=0x7fd634005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab11dcdd9736494 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd633400640 1 -- 172.21.15.18:0/4123724927 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63404c070 con 0x5647f6634830 2024-04-16T17:12:09.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd633400640 1 -- 172.21.15.18:0/4123724927 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd63402cb60 con 0x5647f6634830 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd633400640 1 -- 172.21.15.18:0/4123724927 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634004920 con 0x5647f6634830 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd650e39740 1 -- 172.21.15.18:0/4123724927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 msgr2=0x5647f6655140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd650e39740 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f6655140 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd634009900 tx=0x7fd634005fd0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd650e39740 1 -- 172.21.15.18:0/4123724927 shutdown_connections 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd650e39740 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 0x5647f6655680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd650e39740 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6634830 0x5647f6655140 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.251+0000 7fd650e39740 1 --2- 172.21.15.18:0/4123724927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6633f10 0x5647f66490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.252+0000 7fd650e39740 1 -- 172.21.15.18:0/4123724927 >> 172.21.15.18:0/4123724927 conn(0x5647f662e640 msgr2=0x5647f662d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.252+0000 7fd650e39740 1 -- 172.21.15.18:0/4123724927 shutdown_connections 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.252+0000 7fd650e39740 1 -- 172.21.15.18:0/4123724927 wait complete. 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.252+0000 7fd650e39740 1 -- start start 2024-04-16T17:12:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.252+0000 7fd650e39740 1 Processor -- start 2024-04-16T17:12:09.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd650e39740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6633f10 0x5647f66458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd650e39740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 0x5647f6642c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd650e39740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 0x5647f66431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd650e39740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd6481afee0 con 0x5647f6633f10 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd650e39740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd6481b1350 con 0x5647f663d410 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd650e39740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6481b05b0 con 0x5647f6634830 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd650e39740 10 client.0 _ll_register_callbacks cb 0x7fd640009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd647e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6633f10 0x5647f66458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd647e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6633f10 0x5647f66458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42292/0 (socket says 172.21.15.18:42292) 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd647e00640 1 -- 172.21.15.18:0/3978772434 learned_addr learned my addr 172.21.15.18:0/3978772434 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd647e00640 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 0x5647f66431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.253+0000 7fd646a00640 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 0x5647f6642c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd647e00640 1 -- 172.21.15.18:0/3978772434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 msgr2=0x5647f6642c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd647e00640 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 0x5647f6642c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd647e00640 1 -- 172.21.15.18:0/3978772434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 msgr2=0x5647f66431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd647e00640 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 0x5647f66431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd647e00640 1 -- 172.21.15.18:0/3978772434 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f63757d0 con 0x5647f6633f10 2024-04-16T17:12:09.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd647e00640 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6633f10 0x5647f66458f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd640019ba0 tx=0x7fd640009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acf678861f1314c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd630c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd630c00640 1 -- 172.21.15.18:0/3978772434 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640003970 con 0x5647f6633f10 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.254+0000 7fd630c00640 1 -- 172.21.15.18:0/3978772434 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7fd640003b10 con 0x5647f6633f10 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.255+0000 7fd650e39740 1 -- 172.21.15.18:0/3978772434 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd63402cb60 con 0x5647f6633f10 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.255+0000 7fd630c00640 1 -- 172.21.15.18:0/3978772434 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd640003d10 con 0x5647f6633f10 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.255+0000 7fd650e39740 10 client.17355 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.255+0000 7fd650e39740 1 -- 172.21.15.18:0/3978772434 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647f66b3760 con 0x5647f6633f10 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.255+0000 7fd630c00640 1 -- 172.21.15.18:0/3978772434 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd63402cb60 con 0x5647f6633f10 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.256+0000 7fd630c00640 1 -- 172.21.15.18:0/3978772434 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd6400117c0 con 0x5647f6633f10 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.256+0000 7fd650e39740 10 client.17355 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.256+0000 7fd650e39740 10 client.17355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.256+0000 7fd650e39740 20 client.17355 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:09.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.256+0000 7fd650e39740 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5647f66b3d10 0x5647f66b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.256+0000 7fd650e39740 4 client.17355 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:09.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.256+0000 7fd650e39740 1 -- 172.21.15.18:0/3978772434 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647f65a08b0 con 0x5647f66b3d10 2024-04-16T17:12:09.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.256+0000 7fd646a00640 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5647f66b3d10 0x5647f66b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.257+0000 7fd646a00640 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5647f66b3d10 0x5647f66b6130 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c704ec470c61398 server_cookie=307c38dd95027d8f in_seq=0 out_seq=0 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.257+0000 7fd630c00640 10 client.17355 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.257+0000 7fd630c00640 1 -- 172.21.15.18:0/3978772434 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647f65a08b0 con 0x5647f66b3d10 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.257+0000 7fd630c00640 10 client.17355 handle_command_reply: tid=0 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.257+0000 7fd650e39740 1 client.17355 shutdown 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.257+0000 7fd650e39740 1 -- 172.21.15.18:0/3978772434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6633f10 msgr2=0x5647f66458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.257+0000 7fd650e39740 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6633f10 0x5647f66458f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd640019ba0 tx=0x7fd640009f90 comp rx=0 tx=0).stop 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.257+0000 7fd650e39740 1 -- 172.21.15.18:0/3978772434 shutdown_connections 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd650e39740 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5647f66b3d10 0x5647f66b6130 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd650e39740 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647f663d410 0x5647f66431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd650e39740 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647f6634830 0x5647f6642c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd650e39740 1 --2- 172.21.15.18:0/3978772434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5647f6633f10 0x5647f66458f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd650e39740 1 -- 172.21.15.18:0/3978772434 >> 172.21.15.18:0/3978772434 conn(0x5647f662e640 msgr2=0x5647f662f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd630c00640 0 client.17355 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:12:09.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd650e39740 1 -- 172.21.15.18:0/3978772434 shutdown_connections 2024-04-16T17:12:09.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd650e39740 1 -- 172.21.15.18:0/3978772434 wait complete. 2024-04-16T17:12:09.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.258+0000 7fd650e39740 20 client.17355 trim_cache size 0 max 16384 2024-04-16T17:12:09.259 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 -- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd62c0a3d20 msgr2=0x7fd62c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd62c0a3d20 0x7fd62c0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd6380cd9d0 tx=0x7fd638009c30 comp rx=0 tx=0).stop 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 -- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102070 msgr2=0x7fd6481a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102070 0x7fd6481a0620 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd63800d770 tx=0x7fd63800dc30 comp rx=0 tx=0).stop 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 -- 172.21.15.18:0/3546985454 shutdown_connections 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd62c0a3d20 0x7fd62c0a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd64819a790 0x7fd64819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd648107140 0x7fd6481a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 --2- 172.21.15.18:0/3546985454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd648102070 0x7fd6481a0620 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 -- 172.21.15.18:0/3546985454 >> 172.21.15.18:0/3546985454 conn(0x7fd6480fd950 msgr2=0x7fd6480feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 -- 172.21.15.18:0/3546985454 shutdown_connections 2024-04-16T17:12:09.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.259+0000 7fd64ca00640 1 -- 172.21.15.18:0/3546985454 wait complete. 2024-04-16T17:12:09.278 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:12:09.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.459+0000 7f595f200640 1 -- 172.21.15.18:0/2334517557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5960102a20 msgr2=0x7f5960106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.459+0000 7f595f200640 1 --2- 172.21.15.18:0/2334517557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5960102a20 0x7f5960106a60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f594c009900 tx=0x7f594c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:12:09.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.460+0000 7f595f200640 1 -- 172.21.15.18:0/2334517557 shutdown_connections 2024-04-16T17:12:09.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.460+0000 7f595f200640 1 --2- 172.21.15.18:0/2334517557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5960107140 0x7f596010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.460+0000 7f595f200640 1 --2- 172.21.15.18:0/2334517557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5960102a20 0x7f5960106a60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.460+0000 7f595f200640 1 --2- 172.21.15.18:0/2334517557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5960102070 0x7f5960102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.460+0000 7f595f200640 1 -- 172.21.15.18:0/2334517557 >> 172.21.15.18:0/2334517557 conn(0x7f59600fd950 msgr2=0x7f59600ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.460+0000 7f595f200640 1 -- 172.21.15.18:0/2334517557 shutdown_connections 2024-04-16T17:12:09.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.460+0000 7f595f200640 1 -- 172.21.15.18:0/2334517557 wait complete. 2024-04-16T17:12:09.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.461+0000 7f595f200640 1 Processor -- start 2024-04-16T17:12:09.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.461+0000 7f595f200640 1 -- start start 2024-04-16T17:12:09.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5960102070 0x7f59601a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5960107140 0x7f59601a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f596019a820 0x7f596019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5960112220 con 0x7f596019a820 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59601120a0 con 0x7f5960107140 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f59601123a0 con 0x7f5960102070 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f596019a820 0x7f596019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f596019a820 0x7f596019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42308/0 (socket says 172.21.15.18:42308) 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595e800640 1 -- 172.21.15.18:0/2307159733 learned_addr learned my addr 172.21.15.18:0/2307159733 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595d400640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5960107140 0x7f59601a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595e800640 1 -- 172.21.15.18:0/2307159733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5960102070 msgr2=0x7f59601a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595e800640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5960102070 0x7f59601a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595e800640 1 -- 172.21.15.18:0/2307159733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5960107140 msgr2=0x7f59601a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595e800640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5960107140 0x7f59601a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.462+0000 7f595e800640 1 -- 172.21.15.18:0/2307159733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596019b3b0 con 0x7f596019a820 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.463+0000 7f595e800640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f596019a820 0x7f596019acd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f595800efd0 tx=0x7f595800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.463+0000 7f5956a00640 1 -- 172.21.15.18:0/2307159733 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958003b70 con 0x7f596019a820 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.463+0000 7f5956a00640 1 -- 172.21.15.18:0/2307159733 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5958003d10 con 0x7f596019a820 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.463+0000 7f595f200640 1 -- 172.21.15.18:0/2307159733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596019b6a0 con 0x7f596019a820 2024-04-16T17:12:09.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.463+0000 7f595f200640 1 -- 172.21.15.18:0/2307159733 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59601a7760 con 0x7f596019a820 2024-04-16T17:12:09.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.463+0000 7f5956a00640 1 -- 172.21.15.18:0/2307159733 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59580109f0 con 0x7f596019a820 2024-04-16T17:12:09.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.467+0000 7f59679fd740 1 Processor -- start 2024-04-16T17:12:09.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.468+0000 7f5956a00640 1 -- 172.21.15.18:0/2307159733 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5958004020 con 0x7f596019a820 2024-04-16T17:12:09.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.468+0000 7f59679fd740 1 -- start start 2024-04-16T17:12:09.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.468+0000 7f59679fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fd52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.468+0000 7f59679fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fdded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.468+0000 7f59679fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.468+0000 7f59679fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f460f3a020 con 0x55f460fd5830 2024-04-16T17:12:09.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.468+0000 7f59679fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f460ee0460 con 0x55f460fde410 2024-04-16T17:12:09.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.468+0000 7f59679fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f460f35fe0 con 0x55f460fd4f10 2024-04-16T17:12:09.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fd52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fd52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39344/0 (socket says 172.21.15.18:39344) 2024-04-16T17:12:09.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 -- 172.21.15.18:0/575720268 learned_addr learned my addr 172.21.15.18:0/575720268 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595de00640 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fdded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 -- 172.21.15.18:0/575720268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 msgr2=0x55f460fe0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 -- 172.21.15.18:0/575720268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 msgr2=0x55f460fdded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fdded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 -- 172.21.15.18:0/575720268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f460ee8040 con 0x55f460fd4f10 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.469+0000 7f595d400640 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fd52f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f594c005c40 tx=0x7f594c005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7b2e9e9276ce39e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.470+0000 7f594b000640 1 -- 172.21.15.18:0/575720268 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c0043b0 con 0x55f460fd4f10 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.470+0000 7f594b000640 1 -- 172.21.15.18:0/575720268 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f594c0046c0 con 0x55f460fd4f10 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.470+0000 7f594b000640 1 -- 172.21.15.18:0/575720268 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c0050e0 con 0x55f460fd4f10 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.470+0000 7f5956a00640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59440a3d20 0x7f59440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.470+0000 7f5956a00640 1 -- 172.21.15.18:0/2307159733 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7f59580d1130 con 0x7f596019a820 2024-04-16T17:12:09.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 -- 172.21.15.18:0/575720268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 msgr2=0x55f460fd52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fd52f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f594c005c40 tx=0x7f594c005b50 comp rx=0 tx=0).stop 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 -- 172.21.15.18:0/575720268 shutdown_connections 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fdded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 --2- 172.21.15.18:0/575720268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fd52f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 -- 172.21.15.18:0/575720268 >> 172.21.15.18:0/575720268 conn(0x55f460fcf640 msgr2=0x55f460fcfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f595de00640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59440a3d20 0x7f59440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 -- 172.21.15.18:0/575720268 shutdown_connections 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.471+0000 7f59679fd740 1 -- 172.21.15.18:0/575720268 wait complete. 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.472+0000 7f59679fd740 1 -- start start 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.472+0000 7f595de00640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59440a3d20 0x7f59440a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55f460fd49e0 tx=0x7f5950009620 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.472+0000 7f59679fd740 1 Processor -- start 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.472+0000 7f59679fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fdb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.472+0000 7f59679fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fdb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f59679fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fdbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f59679fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f460fe11a0 con 0x55f460fd5830 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f59679fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f460ef3830 con 0x55f460fde410 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f59679fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f460fe1320 con 0x55f460fd4f10 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f59679fd740 10 client.0 _ll_register_callbacks cb 0x55f460efb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fdb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fdb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fdb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39354/0 (socket says 172.21.15.18:39354) 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595de00640 1 -- 172.21.15.18:0/3806830104 learned_addr learned my addr 172.21.15.18:0/3806830104 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595de00640 1 -- 172.21.15.18:0/3806830104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 msgr2=0x55f460fdbf10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595de00640 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fdbf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595de00640 1 -- 172.21.15.18:0/3806830104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 msgr2=0x55f460fdb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595de00640 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fdb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.473+0000 7f595de00640 1 -- 172.21.15.18:0/3806830104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f460ef7920 con 0x55f460fd4f10 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.474+0000 7f595de00640 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fdb490 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f59500038a0 tx=0x7f59500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65e6fe26a90bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.474+0000 7f593f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.474+0000 7f593f400640 1 -- 172.21.15.18:0/3806830104 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595000e930 con 0x55f460fd4f10 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.474+0000 7f59679fd740 1 -- 172.21.15.18:0/3806830104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f594c0046c0 con 0x55f460fd4f10 2024-04-16T17:12:09.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.474+0000 7f593f400640 1 -- 172.21.15.18:0/3806830104 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7f595000ead0 con 0x55f460fd4f10 2024-04-16T17:12:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.474+0000 7f593f400640 1 -- 172.21.15.18:0/3806830104 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f595002f050 con 0x55f460fd4f10 2024-04-16T17:12:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.474+0000 7f59679fd740 10 client.27553 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.474+0000 7f59679fd740 1 -- 172.21.15.18:0/3806830104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f460ec57f0 con 0x55f460fd4f10 2024-04-16T17:12:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.475+0000 7f593f400640 1 -- 172.21.15.18:0/3806830104 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f594c0046c0 con 0x55f460fd4f10 2024-04-16T17:12:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.475+0000 7f593f400640 1 -- 172.21.15.18:0/3806830104 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f595001d380 con 0x55f460fd4f10 2024-04-16T17:12:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.475+0000 7f59679fd740 10 client.27553 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.475+0000 7f59679fd740 10 client.27553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.475+0000 7f59679fd740 20 client.27553 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.475+0000 7f59679fd740 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f460fee780 0x55f460ff0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.475+0000 7f59679fd740 4 client.27553 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.475+0000 7f59679fd740 1 -- 172.21.15.18:0/3806830104 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f460f47a40 con 0x55f460fee780 2024-04-16T17:12:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.477+0000 7f595d400640 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f460fee780 0x55f460ff0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.478+0000 7f595d400640 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f460fee780 0x55f460ff0c10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e5dcb8aa67a1b3 server_cookie=65421c7c99b1a199 in_seq=0 out_seq=0 2024-04-16T17:12:09.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.478+0000 7f593f400640 10 client.27553 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:09.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f593f400640 1 -- 172.21.15.18:0/3806830104 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f460f47a40 con 0x55f460fee780 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f593f400640 10 client.27553 handle_command_reply: tid=0 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 client.27553 shutdown 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 -- 172.21.15.18:0/3806830104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 msgr2=0x55f460fdb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fdb490 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f59500038a0 tx=0x7f59500046c0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 -- 172.21.15.18:0/3806830104 shutdown_connections 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f460fee780 0x55f460ff0c10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fdbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fdb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 --2- 172.21.15.18:0/3806830104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fdb490 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f593f400640 0 client.27553 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 -- 172.21.15.18:0/3806830104 >> 172.21.15.18:0/3806830104 conn(0x55f460fcf640 msgr2=0x55f460fd1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 -- 172.21.15.18:0/3806830104 shutdown_connections 2024-04-16T17:12:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 1 -- 172.21.15.18:0/3806830104 wait complete. 2024-04-16T17:12:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.479+0000 7f59679fd740 20 client.27553 trim_cache size 0 max 16384 2024-04-16T17:12:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f59679fd740 1 Processor -- start 2024-04-16T17:12:09.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f59679fd740 1 -- start start 2024-04-16T17:12:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f59679fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd4f10 0x55f460fddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f59679fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd5830 0x55f460fea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f59679fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460ff64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f59679fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f59601b4080 con 0x55f460fd4f10 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f59679fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59601b1950 con 0x55f460fde410 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f59679fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f59601af7c0 con 0x55f460fd5830 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f595de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd4f10 0x55f460fddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.506+0000 7f595de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd4f10 0x55f460fddbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42344/0 (socket says 172.21.15.18:42344) 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595de00640 1 -- 172.21.15.18:0/4167389626 learned_addr learned my addr 172.21.15.18:0/4167389626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595e800640 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460ff64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595de00640 1 -- 172.21.15.18:0/4167389626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd5830 msgr2=0x55f460fea060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595de00640 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd5830 0x55f460fea060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595de00640 1 -- 172.21.15.18:0/4167389626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 msgr2=0x55f460ff64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595de00640 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460ff64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595de00640 1 -- 172.21.15.18:0/4167389626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f460c4c3f0 con 0x55f460fd4f10 2024-04-16T17:12:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595e800640 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460ff64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.507+0000 7f595de00640 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd4f10 0x55f460fddbc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5950003fd0 tx=0x7f59500058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1b554b7a8f7e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.508+0000 7f594a600640 1 -- 172.21.15.18:0/4167389626 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595002c040 con 0x55f460fd4f10 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.508+0000 7f594a600640 1 -- 172.21.15.18:0/4167389626 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595001de20 con 0x55f460fd4f10 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.508+0000 7f594a600640 1 -- 172.21.15.18:0/4167389626 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595000ee30 con 0x55f460fd4f10 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.508+0000 7f59679fd740 1 -- 172.21.15.18:0/4167389626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd4f10 msgr2=0x55f460fddbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.508+0000 7f59679fd740 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd4f10 0x55f460fddbc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5950003fd0 tx=0x7f59500058d0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 -- 172.21.15.18:0/4167389626 shutdown_connections 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460ff64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd5830 0x55f460fea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 --2- 172.21.15.18:0/4167389626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd4f10 0x55f460fddbc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 -- 172.21.15.18:0/4167389626 >> 172.21.15.18:0/4167389626 conn(0x55f460fcf640 msgr2=0x55f460fd1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 -- 172.21.15.18:0/4167389626 shutdown_connections 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 -- 172.21.15.18:0/4167389626 wait complete. 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 -- start start 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.509+0000 7f59679fd740 1 Processor -- start 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f59679fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fe2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f59679fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fe27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f59679fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f59679fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f59601afe10 con 0x55f460fd5830 2024-04-16T17:12:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f59679fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59601b1280 con 0x55f460fde410 2024-04-16T17:12:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f59679fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f59601b04e0 con 0x55f460fd4f10 2024-04-16T17:12:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f59679fd740 10 client.0 _ll_register_callbacks cb 0x7f5950004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:09.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fe27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fe27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42346/0 (socket says 172.21.15.18:42346) 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595d400640 1 -- 172.21.15.18:0/3815502421 learned_addr learned my addr 172.21.15.18:0/3815502421 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595e800640 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595d400640 1 -- 172.21.15.18:0/3815502421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 msgr2=0x55f460fe2270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595d400640 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fe2270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595d400640 1 -- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 msgr2=0x55f460fe2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595d400640 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595d400640 1 -- 172.21.15.18:0/3815502421 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f460d167d0 con 0x55f460fd5830 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.510+0000 7f595e800640 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.511+0000 7f595d400640 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fe27b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f594c009400 tx=0x7f594c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffde91a95ae3b017 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.511+0000 7f593f400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.511+0000 7f593f400640 1 -- 172.21.15.18:0/3815502421 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c05b060 con 0x55f460fd5830 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.511+0000 7f593f400640 1 -- 172.21.15.18:0/3815502421 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7f594c056bc0 con 0x55f460fd5830 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.511+0000 7f59679fd740 1 -- 172.21.15.18:0/3815502421 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595001de20 con 0x55f460fd5830 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.511+0000 7f593f400640 1 -- 172.21.15.18:0/3815502421 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f594c02c880 con 0x55f460fd5830 2024-04-16T17:12:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.511+0000 7f59679fd740 10 client.17373 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.511+0000 7f59679fd740 1 -- 172.21.15.18:0/3815502421 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f461055170 con 0x55f460fd5830 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.512+0000 7f593f400640 1 -- 172.21.15.18:0/3815502421 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595001de20 con 0x55f460fd5830 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.512+0000 7f593f400640 1 -- 172.21.15.18:0/3815502421 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f594c004ee0 con 0x55f460fd5830 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.512+0000 7f59679fd740 10 client.17373 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.512+0000 7f59679fd740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.512+0000 7f59679fd740 20 client.17373 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.512+0000 7f59679fd740 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f461055460 0x55f461057850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.512+0000 7f59679fd740 4 client.17373 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.512+0000 7f59679fd740 1 -- 172.21.15.18:0/3815502421 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f460f418b0 con 0x55f461055460 2024-04-16T17:12:09.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.524+0000 7f595de00640 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f461055460 0x55f461057850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:09.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.546+0000 7f595de00640 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f461055460 0x55f461057850 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2041b722f3b88ac server_cookie=8b7b5e4372e824a1 in_seq=0 out_seq=0 2024-04-16T17:12:09.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.546+0000 7f593f400640 10 client.17373 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f593f400640 1 -- 172.21.15.18:0/3815502421 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f460f418b0 con 0x55f461055460 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f593f400640 10 client.17373 handle_command_reply: tid=0 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 client.17373 shutdown 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 -- 172.21.15.18:0/3815502421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 msgr2=0x55f460fe27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fe27b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f594c009400 tx=0x7f594c02c7e0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 -- 172.21.15.18:0/3815502421 shutdown_connections 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f461055460 0x55f461057850 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f460fde410 0x55f460fe2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f460fd5830 0x55f460fe27b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f593f400640 0 client.17373 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:09.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 --2- 172.21.15.18:0/3815502421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f460fd4f10 0x55f460fe2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 -- 172.21.15.18:0/3815502421 >> 172.21.15.18:0/3815502421 conn(0x55f460fcf640 msgr2=0x55f460fce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 -- 172.21.15.18:0/3815502421 shutdown_connections 2024-04-16T17:12:09.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 1 -- 172.21.15.18:0/3815502421 wait complete. 2024-04-16T17:12:09.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.547+0000 7f59679fd740 20 client.17373 trim_cache size 0 max 16384 2024-04-16T17:12:09.548 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.548+0000 7f595f200640 1 -- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59440a3d20 msgr2=0x7f59440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.548+0000 7f595f200640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59440a3d20 0x7f59440a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55f460fd49e0 tx=0x7f5950009620 comp rx=0 tx=0).stop 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.548+0000 7f595f200640 1 -- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f596019a820 msgr2=0x7f596019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.548+0000 7f595f200640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f596019a820 0x7f596019acd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f595800efd0 tx=0x7f595800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.549+0000 7f595f200640 1 -- 172.21.15.18:0/2307159733 shutdown_connections 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.549+0000 7f595f200640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59440a3d20 0x7f59440a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.549+0000 7f595f200640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f596019a820 0x7f596019acd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.549+0000 7f595f200640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5960107140 0x7f59601a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.549+0000 7f595f200640 1 --2- 172.21.15.18:0/2307159733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5960102070 0x7f59601a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:09.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.549+0000 7f595f200640 1 -- 172.21.15.18:0/2307159733 >> 172.21.15.18:0/2307159733 conn(0x7f59600fd950 msgr2=0x7f59600ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:09.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.549+0000 7f595f200640 1 -- 172.21.15.18:0/2307159733 shutdown_connections 2024-04-16T17:12:09.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:09.549+0000 7f595f200640 1 -- 172.21.15.18:0/2307159733 wait complete. 2024-04-16T17:12:09.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 283752 3.93 MB/sec execute 262 sec latency 128.615 ms 2024-04-16T17:12:09.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:09 smithi022 ceph-mon[36023]: pgmap v230: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-16T17:12:09.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:09 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/324295756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:09 smithi043 ceph-mon[36901]: pgmap v230: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-16T17:12:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:09 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/324295756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:10.565 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:12:10.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:12:10.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:12:10.600 INFO:tasks.workunit.client.0.smithi018.stdout: 1 284186 3.93 MB/sec execute 263 sec latency 52.153 ms 2024-04-16T17:12:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.791+0000 7f41fb200640 1 -- 172.21.15.18:0/4077425628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 msgr2=0x7f41fc074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.791+0000 7f41fb200640 1 --2- 172.21.15.18:0/4077425628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 0x7f41fc074780 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009900 tx=0x7f41ec02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.791+0000 7f41fb200640 1 -- 172.21.15.18:0/4077425628 shutdown_connections 2024-04-16T17:12:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.791+0000 7f41fb200640 1 --2- 172.21.15.18:0/4077425628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc112e10 0x7f41fc115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.791+0000 7f41fb200640 1 --2- 172.21.15.18:0/4077425628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc074cc0 0x7f41fc1128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.791+0000 7f41fb200640 1 --2- 172.21.15.18:0/4077425628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 0x7f41fc074780 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.791+0000 7f41fb200640 1 -- 172.21.15.18:0/4077425628 >> 172.21.15.18:0/4077425628 conn(0x7f41fc0fd950 msgr2=0x7f41fc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:10.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.792+0000 7f41fb200640 1 -- 172.21.15.18:0/4077425628 shutdown_connections 2024-04-16T17:12:10.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.792+0000 7f41fb200640 1 -- 172.21.15.18:0/4077425628 wait complete. 2024-04-16T17:12:10.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.793+0000 7f41fb200640 1 Processor -- start 2024-04-16T17:12:10.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.793+0000 7f41fb200640 1 -- start start 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.793+0000 7f41fb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 0x7f41fc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.793+0000 7f41fb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc074cc0 0x7f41fc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.793+0000 7f41fb200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc112e10 0x7f41fc1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.793+0000 7f41fb200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc118850 con 0x7f41fc112e10 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.793+0000 7f41fb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1186d0 con 0x7f41fc0743a0 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.793+0000 7f41fb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1189d0 con 0x7f41fc074cc0 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 0x7f41fc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 0x7f41fc19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33122/0 (socket says 172.21.15.18:33122) 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 -- 172.21.15.18:0/3994351912 learned_addr learned my addr 172.21.15.18:0/3994351912 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9400640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc074cc0 0x7f41fc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41fa800640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc112e10 0x7f41fc1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 -- 172.21.15.18:0/3994351912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc074cc0 msgr2=0x7f41fc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc074cc0 0x7f41fc19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 -- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc112e10 msgr2=0x7f41fc1a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc112e10 0x7f41fc1a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 -- 172.21.15.18:0/3994351912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc1a3d60 con 0x7f41fc0743a0 2024-04-16T17:12:10.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.794+0000 7f41f9e00640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 0x7f41fc19ed70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000c00 tx=0x7f41ec004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:10.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.795+0000 7f41f2a00640 1 -- 172.21.15.18:0/3994351912 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec042020 con 0x7f41fc0743a0 2024-04-16T17:12:10.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.795+0000 7f41fb200640 1 -- 172.21.15.18:0/3994351912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc1a3ff0 con 0x7f41fc0743a0 2024-04-16T17:12:10.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.795+0000 7f41f2a00640 1 -- 172.21.15.18:0/3994351912 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ec005000 con 0x7f41fc0743a0 2024-04-16T17:12:10.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.795+0000 7f41fb200640 1 -- 172.21.15.18:0/3994351912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc1ab8e0 con 0x7f41fc0743a0 2024-04-16T17:12:10.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.795+0000 7f41f2a00640 1 -- 172.21.15.18:0/3994351912 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec02cb20 con 0x7f41fc0743a0 2024-04-16T17:12:10.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.798+0000 7f4203a57740 1 Processor -- start 2024-04-16T17:12:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.798+0000 7f4203a57740 1 -- start start 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f4203a57740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da4e90 0x5574a8da5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f41f2a00640 1 -- 172.21.15.18:0/3994351912 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41ec045020 con 0x7f41fc0743a0 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f4203a57740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da57b0 0x5574a8dade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f4203a57740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8dae390 0x5574a8db0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f4203a57740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5574a8d09cb0 con 0x5574a8dae390 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f4203a57740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5574a8cb0370 con 0x5574a8da57b0 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f4203a57740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5574a8d05c70 con 0x5574a8da4e90 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f41fa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da4e90 0x5574a8da5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f41fa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da4e90 0x5574a8da5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39406/0 (socket says 172.21.15.18:39406) 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f41fa800640 1 -- 172.21.15.18:0/1548983079 learned_addr learned my addr 172.21.15.18:0/1548983079 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.799+0000 7f41f9400640 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8dae390 0x5574a8db0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41f9e00640 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da57b0 0x5574a8dade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41fa800640 1 -- 172.21.15.18:0/1548983079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da57b0 msgr2=0x5574a8dade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:10.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41fa800640 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da57b0 0x5574a8dade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41fa800640 1 -- 172.21.15.18:0/1548983079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8dae390 msgr2=0x5574a8db0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41fa800640 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8dae390 0x5574a8db0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41fa800640 1 -- 172.21.15.18:0/1548983079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a8ccf910 con 0x5574a8da4e90 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41f9e00640 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da57b0 0x5574a8dade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41f9400640 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8dae390 0x5574a8db0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41fa800640 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da4e90 0x5574a8da5270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009f00 tx=0x7f41f40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76aba669a3c7d8f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.800+0000 7f41f2a00640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e00a3da0 0x7f41e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.801+0000 7f41e7400640 1 -- 172.21.15.18:0/1548983079 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4003b70 con 0x5574a8da4e90 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.801+0000 7f41f2a00640 1 -- 172.21.15.18:0/3994351912 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7f41ec0f1b90 con 0x7f41fc0743a0 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.801+0000 7f41e7400640 1 -- 172.21.15.18:0/1548983079 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4003d10 con 0x5574a8da4e90 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.801+0000 7f41e7400640 1 -- 172.21.15.18:0/1548983079 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4011a40 con 0x5574a8da4e90 2024-04-16T17:12:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.801+0000 7f41f9e00640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e00a3da0 0x7f41e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 -- 172.21.15.18:0/1548983079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da4e90 msgr2=0x5574a8da5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da4e90 0x5574a8da5270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009f00 tx=0x7f41f40107a0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 -- 172.21.15.18:0/1548983079 shutdown_connections 2024-04-16T17:12:10.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8dae390 0x5574a8db0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da57b0 0x5574a8dade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 --2- 172.21.15.18:0/1548983079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da4e90 0x5574a8da5270 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 -- 172.21.15.18:0/1548983079 >> 172.21.15.18:0/1548983079 conn(0x5574a8d9f5c0 msgr2=0x5574a8d9f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f41f9e00640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e00a3da0 0x7f41e00a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5574a8da4960 tx=0x7f41ec02bcc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 -- 172.21.15.18:0/1548983079 shutdown_connections 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 -- 172.21.15.18:0/1548983079 wait complete. 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.802+0000 7f4203a57740 1 -- start start 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f4203a57740 1 Processor -- start 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f4203a57740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8dab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f4203a57740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8dab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f4203a57740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 0x5574a8dabeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f4203a57740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5574a8db1180 con 0x5574a8da57b0 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f4203a57740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5574a8cc3730 con 0x5574a8da4e90 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f4203a57740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5574a8db1300 con 0x5574a8dae390 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f4203a57740 10 client.0 _ll_register_callbacks cb 0x5574a8c921e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f41fa800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8dab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f41fa800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8dab430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33140/0 (socket says 172.21.15.18:33140) 2024-04-16T17:12:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f41fa800640 1 -- 172.21.15.18:0/1941271743 learned_addr learned my addr 172.21.15.18:0/1941271743 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:10.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.803+0000 7f41f9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8dab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41fa800640 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 0x5574a8dabeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41fa800640 1 -- 172.21.15.18:0/1941271743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 msgr2=0x5574a8dabeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41fa800640 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 0x5574a8dabeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41fa800640 1 -- 172.21.15.18:0/1941271743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 msgr2=0x5574a8dab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41fa800640 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8dab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41fa800640 1 -- 172.21.15.18:0/1941271743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a8ae47e0 con 0x5574a8da4e90 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41f9400640 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8dab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41fa800640 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8dab430 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005fd0 tx=0x7f41f4004810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3255e3a800537d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.804+0000 7f41dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.805+0000 7f41dbe00640 1 -- 172.21.15.18:0/1941271743 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4004a40 con 0x5574a8da4e90 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.805+0000 7f41dbe00640 1 -- 172.21.15.18:0/1941271743 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7f41f4004be0 con 0x5574a8da4e90 2024-04-16T17:12:10.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.805+0000 7f4203a57740 1 -- 172.21.15.18:0/1941271743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f4003d10 con 0x5574a8da4e90 2024-04-16T17:12:10.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.805+0000 7f41dbe00640 1 -- 172.21.15.18:0/1941271743 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41f4032050 con 0x5574a8da4e90 2024-04-16T17:12:10.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.805+0000 7f4203a57740 10 client.27647 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:10.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.805+0000 7f4203a57740 1 -- 172.21.15.18:0/1941271743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574a8b44f70 con 0x5574a8da4e90 2024-04-16T17:12:10.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.805+0000 7f41dbe00640 1 -- 172.21.15.18:0/1941271743 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4003d10 con 0x5574a8da4e90 2024-04-16T17:12:10.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.805+0000 7f41dbe00640 1 -- 172.21.15.18:0/1941271743 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f41f401b8d0 con 0x5574a8da4e90 2024-04-16T17:12:10.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f4203a57740 10 client.27647 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:10.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f4203a57740 10 client.27647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:10.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f4203a57740 20 client.27647 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f4203a57740 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574a8dbe660 0x5574a8dc0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f4203a57740 4 client.27647 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f4203a57740 1 -- 172.21.15.18:0/1941271743 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574a8d176d0 con 0x5574a8dbe660 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f41f9400640 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574a8dbe660 0x5574a8dc0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f41dbe00640 10 client.27647 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.806+0000 7f41f9400640 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574a8dbe660 0x5574a8dc0a50 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf796bf1bbaf27 server_cookie=4d104b9d9d68c9ac in_seq=0 out_seq=0 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.807+0000 7f41dbe00640 1 -- 172.21.15.18:0/1941271743 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574a8d176d0 con 0x5574a8dbe660 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.807+0000 7f41dbe00640 10 client.27647 handle_command_reply: tid=0 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.807+0000 7f4203a57740 1 client.27647 shutdown 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 -- 172.21.15.18:0/1941271743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 msgr2=0x5574a8dab430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8dab430 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005fd0 tx=0x7f41f4004810 comp rx=0 tx=0).stop 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 -- 172.21.15.18:0/1941271743 shutdown_connections 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574a8dbe660 0x5574a8dc0a50 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 0x5574a8dabeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8dab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 --2- 172.21.15.18:0/1941271743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8dab430 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f41dbe00640 0 client.27647 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:10.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 -- 172.21.15.18:0/1941271743 >> 172.21.15.18:0/1941271743 conn(0x5574a8d9f5c0 msgr2=0x5574a8da1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:10.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 -- 172.21.15.18:0/1941271743 shutdown_connections 2024-04-16T17:12:10.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.808+0000 7f4203a57740 1 -- 172.21.15.18:0/1941271743 wait complete. 2024-04-16T17:12:10.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.809+0000 7f4203a57740 20 client.27647 trim_cache size 0 max 16384 2024-04-16T17:12:10.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f4203a57740 1 Processor -- start 2024-04-16T17:12:10.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f4203a57740 1 -- start start 2024-04-16T17:12:10.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f4203a57740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da4e90 0x5574a8dba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f4203a57740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da57b0 0x5574a8dabcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f4203a57740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8dae390 0x5574a8dc60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f4203a57740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b84d0 con 0x5574a8da4e90 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f4203a57740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b5da0 con 0x5574a8dae390 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f4203a57740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b3c10 con 0x5574a8da57b0 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f41fa800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da4e90 0x5574a8dba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f41fa800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da4e90 0x5574a8dba0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42404/0 (socket says 172.21.15.18:42404) 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.829+0000 7f41fa800640 1 -- 172.21.15.18:0/1177661745 learned_addr learned my addr 172.21.15.18:0/1177661745 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.830+0000 7f41fa800640 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8dae390 0x5574a8dc60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.830+0000 7f41f9400640 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da57b0 0x5574a8dabcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.830+0000 7f41fa800640 1 -- 172.21.15.18:0/1177661745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da57b0 msgr2=0x5574a8dabcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.830+0000 7f41fa800640 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da57b0 0x5574a8dabcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.830+0000 7f41fa800640 1 -- 172.21.15.18:0/1177661745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8dae390 msgr2=0x5574a8dc60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:10.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.830+0000 7f41fa800640 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8dae390 0x5574a8dc60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.830+0000 7f41fa800640 1 -- 172.21.15.18:0/1177661745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a89dc170 con 0x5574a8da4e90 2024-04-16T17:12:10.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.830+0000 7f41fa800640 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da4e90 0x5574a8dba0d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f41f4002230 tx=0x7f41f4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eff484097c58dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f41e6a00640 1 -- 172.21.15.18:0/1177661745 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4004670 con 0x5574a8da4e90 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f41e6a00640 1 -- 172.21.15.18:0/1177661745 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4003c80 con 0x5574a8da4e90 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f41e6a00640 1 -- 172.21.15.18:0/1177661745 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4004cc0 con 0x5574a8da4e90 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f4203a57740 1 -- 172.21.15.18:0/1177661745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da4e90 msgr2=0x5574a8dba0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f4203a57740 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da4e90 0x5574a8dba0d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f41f4002230 tx=0x7f41f4000f40 comp rx=0 tx=0).stop 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f4203a57740 1 -- 172.21.15.18:0/1177661745 shutdown_connections 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f4203a57740 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8dae390 0x5574a8dc60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f4203a57740 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8da57b0 0x5574a8dabcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f4203a57740 1 --2- 172.21.15.18:0/1177661745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da4e90 0x5574a8dba0d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.831+0000 7f4203a57740 1 -- 172.21.15.18:0/1177661745 >> 172.21.15.18:0/1177661745 conn(0x5574a8d9f5c0 msgr2=0x5574a8da1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.832+0000 7f4203a57740 1 -- 172.21.15.18:0/1177661745 shutdown_connections 2024-04-16T17:12:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.832+0000 7f4203a57740 1 -- 172.21.15.18:0/1177661745 wait complete. 2024-04-16T17:12:10.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.832+0000 7f4203a57740 1 -- start start 2024-04-16T17:12:10.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.832+0000 7f4203a57740 1 Processor -- start 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f4203a57740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8da8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f4203a57740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8da8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f4203a57740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 0x5574a8da8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f4203a57740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b4260 con 0x5574a8da57b0 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f4203a57740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b56d0 con 0x5574a8da4e90 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f4203a57740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b4930 con 0x5574a8dae390 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f4203a57740 10 client.0 _ll_register_callbacks cb 0x7f41f4009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f41fa800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8da8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f41fa800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8da8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33150/0 (socket says 172.21.15.18:33150) 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f41fa800640 1 -- 172.21.15.18:0/1624097685 learned_addr learned my addr 172.21.15.18:0/1624097685 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f41f9400640 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8da8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.833+0000 7f41fa800640 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 0x5574a8da8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41fa800640 1 -- 172.21.15.18:0/1624097685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 msgr2=0x5574a8da8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41fa800640 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 0x5574a8da8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41fa800640 1 -- 172.21.15.18:0/1624097685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 msgr2=0x5574a8da8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41fa800640 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8da8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41fa800640 1 -- 172.21.15.18:0/1624097685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a8e23b00 con 0x5574a8da4e90 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41fa800640 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8da8320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f41f40048a0 tx=0x7f41f40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75587079ad20b1e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41dbe00640 1 -- 172.21.15.18:0/1624097685 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4004b40 con 0x5574a8da4e90 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f41dbe00640 1 -- 172.21.15.18:0/1624097685 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7f41f401b420 con 0x5574a8da4e90 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.834+0000 7f4203a57740 1 -- 172.21.15.18:0/1624097685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f4003c80 con 0x5574a8da4e90 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f41dbe00640 1 -- 172.21.15.18:0/1624097685 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41f402e0c0 con 0x5574a8da4e90 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f4203a57740 10 client.27653 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f4203a57740 1 -- 172.21.15.18:0/1624097685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574a8e24e20 con 0x5574a8da4e90 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f41dbe00640 1 -- 172.21.15.18:0/1624097685 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4003c80 con 0x5574a8da4e90 2024-04-16T17:12:10.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f41dbe00640 1 -- 172.21.15.18:0/1624097685 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f41f4011b50 con 0x5574a8da4e90 2024-04-16T17:12:10.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f4203a57740 10 client.27653 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:10.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f4203a57740 10 client.27653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:10.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f4203a57740 20 client.27653 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:10.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f4203a57740 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574a8e25460 0x5574a8e27880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:10.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f4203a57740 4 client.27653 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:12:10.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.835+0000 7f4203a57740 1 -- 172.21.15.18:0/1624097685 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5574a8d11540 con 0x5574a8e25460 2024-04-16T17:12:10.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.836+0000 7f41f9400640 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574a8e25460 0x5574a8e27880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:10.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.836+0000 7f41f9400640 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574a8e25460 0x5574a8e27880 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee01fde75184a1b1 server_cookie=88eb826017cd27bd in_seq=0 out_seq=0 2024-04-16T17:12:10.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.836+0000 7f41dbe00640 10 client.27653 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:10.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f41dbe00640 1 -- 172.21.15.18:0/1624097685 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5574a8d11540 con 0x5574a8e25460 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f41dbe00640 10 client.27653 handle_command_reply: tid=0 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 client.27653 shutdown 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 -- 172.21.15.18:0/1624097685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 msgr2=0x5574a8da8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8da8320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f41f40048a0 tx=0x7f41f40107a0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 -- 172.21.15.18:0/1624097685 shutdown_connections 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5574a8e25460 0x5574a8e27880 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a8dae390 0x5574a8da8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5574a8da57b0 0x5574a8da8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 --2- 172.21.15.18:0/1624097685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574a8da4e90 0x5574a8da8320 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f4203a57740 1 -- 172.21.15.18:0/1624097685 >> 172.21.15.18:0/1624097685 conn(0x5574a8d9f5c0 msgr2=0x5574a8d9dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.838+0000 7f41dbe00640 0 client.27653 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.839+0000 7f4203a57740 1 -- 172.21.15.18:0/1624097685 shutdown_connections 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.839+0000 7f4203a57740 1 -- 172.21.15.18:0/1624097685 wait complete. 2024-04-16T17:12:10.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.839+0000 7f4203a57740 20 client.27653 trim_cache size 0 max 16384 2024-04-16T17:12:10.841 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:12:10.841 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:12:10.841 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "f06e1071-6d4a-42a6-8f60-ed61202582a9", 2024-04-16T17:12:10.841 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:12:10.841 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 -- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e00a3da0 msgr2=0x7f41e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e00a3da0 0x7f41e00a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5574a8da4960 tx=0x7f41ec02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 -- 172.21.15.18:0/3994351912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 msgr2=0x7f41fc19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 0x7f41fc19ed70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000c00 tx=0x7f41ec004360 comp rx=0 tx=0).stop 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 -- 172.21.15.18:0/3994351912 shutdown_connections 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f41e00a3da0 0x7f41e00a6260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41fc112e10 0x7f41fc1a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41fc074cc0 0x7f41fc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 --2- 172.21.15.18:0/3994351912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41fc0743a0 0x7f41fc19ed70 secure :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000c00 tx=0x7f41ec004360 comp rx=0 tx=0).stop 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 -- 172.21.15.18:0/3994351912 >> 172.21.15.18:0/3994351912 conn(0x7f41fc0fd950 msgr2=0x7f41fc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 -- 172.21.15.18:0/3994351912 shutdown_connections 2024-04-16T17:12:10.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:10.840+0000 7f41fb200640 1 -- 172.21.15.18:0/3994351912 wait complete. 2024-04-16T17:12:10.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:12:10.860 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:12:11.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.160+0000 7ff631a00640 1 -- 172.21.15.18:0/763664769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c1016f0 msgr2=0x7ff62c105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.160+0000 7ff631a00640 1 --2- 172.21.15.18:0/763664769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c1016f0 0x7ff62c105bb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff618009900 tx=0x7ff61802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.161+0000 7ff631a00640 1 -- 172.21.15.18:0/763664769 shutdown_connections 2024-04-16T17:12:11.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.161+0000 7ff631a00640 1 --2- 172.21.15.18:0/763664769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff62c1061d0 0x7ff62c106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.161+0000 7ff631a00640 1 --2- 172.21.15.18:0/763664769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c1016f0 0x7ff62c105bb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.161+0000 7ff631a00640 1 --2- 172.21.15.18:0/763664769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff62c100dd0 0x7ff62c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.161+0000 7ff631a00640 1 -- 172.21.15.18:0/763664769 >> 172.21.15.18:0/763664769 conn(0x7ff62c077ab0 msgr2=0x7ff62c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.161+0000 7ff631a00640 1 -- 172.21.15.18:0/763664769 shutdown_connections 2024-04-16T17:12:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.161+0000 7ff631a00640 1 -- 172.21.15.18:0/763664769 wait complete. 2024-04-16T17:12:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.162+0000 7ff631a00640 1 Processor -- start 2024-04-16T17:12:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.162+0000 7ff631a00640 1 -- start start 2024-04-16T17:12:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff631a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c100dd0 0x7ff62c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff631a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff62c1016f0 0x7ff62c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff631a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff62c1061d0 0x7ff62c1a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff631a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1187f0 con 0x7ff62c1016f0 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff631a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c118670 con 0x7ff62c100dd0 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff631a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c118970 con 0x7ff62c1061d0 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff62be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c100dd0 0x7ff62c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff62be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c100dd0 0x7ff62c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33178/0 (socket says 172.21.15.18:33178) 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.163+0000 7ff62be00640 1 -- 172.21.15.18:0/2685999777 learned_addr learned my addr 172.21.15.18:0/2685999777 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff62b400640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff62c1016f0 0x7ff62c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff62be00640 1 -- 172.21.15.18:0/2685999777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff62c1061d0 msgr2=0x7ff62c1a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff631000640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff62c1061d0 0x7ff62c1a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff62be00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff62c1061d0 0x7ff62c1a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff62be00640 1 -- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff62c1016f0 msgr2=0x7ff62c19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff62be00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff62c1016f0 0x7ff62c19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff62be00640 1 -- 172.21.15.18:0/2685999777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c1a3de0 con 0x7ff62c100dd0 2024-04-16T17:12:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff62b400640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff62c1016f0 0x7ff62c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff631000640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff62c1061d0 0x7ff62c1a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff62be00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c100dd0 0x7ff62c19edf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00e9f0 tx=0x7ff61c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.164+0000 7ff628c00640 1 -- 172.21.15.18:0/2685999777 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00cd80 con 0x7ff62c100dd0 2024-04-16T17:12:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.165+0000 7ff631a00640 1 -- 172.21.15.18:0/2685999777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c1a40d0 con 0x7ff62c100dd0 2024-04-16T17:12:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.165+0000 7ff628c00640 1 -- 172.21.15.18:0/2685999777 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c010910 con 0x7ff62c100dd0 2024-04-16T17:12:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.165+0000 7ff628c00640 1 -- 172.21.15.18:0/2685999777 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c0188f0 con 0x7ff62c100dd0 2024-04-16T17:12:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.165+0000 7ff631a00640 1 -- 172.21.15.18:0/2685999777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1ab960 con 0x7ff62c100dd0 2024-04-16T17:12:11.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.168+0000 7ff628c00640 1 -- 172.21.15.18:0/2685999777 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff61c010ac0 con 0x7ff62c100dd0 2024-04-16T17:12:11.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.169+0000 7ff635ddb740 1 Processor -- start 2024-04-16T17:12:11.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.169+0000 7ff635ddb740 1 -- start start 2024-04-16T17:12:11.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.169+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9cb0 0x5590702f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702f2890 0x5590702f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff635ddb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55907024f120 con 0x5590702f2890 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff635ddb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5590701f5340 con 0x5590702e9cb0 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff635ddb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55907024b0e0 con 0x5590702e9390 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9cb0 0x5590702f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff631000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9cb0 0x5590702f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33184/0 (socket says 172.21.15.18:33184) 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62be00640 1 -- 172.21.15.18:0/2954642937 learned_addr learned my addr 172.21.15.18:0/2954642937 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:11.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62b400640 1 --2- 172.21.15.18:0/2954642937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702f2890 0x5590702f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62be00640 1 -- 172.21.15.18:0/2954642937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 msgr2=0x5590702e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62be00640 1 --2- 172.21.15.18:0/2954642937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62be00640 1 -- 172.21.15.18:0/2954642937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702f2890 msgr2=0x5590702f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62be00640 1 --2- 172.21.15.18:0/2954642937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702f2890 0x5590702f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.170+0000 7ff62be00640 1 -- 172.21.15.18:0/2954642937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55907002b7d0 con 0x5590702e9cb0 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.171+0000 7ff628c00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6100a3da0 0x7ff6100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.171+0000 7ff628c00640 1 -- 172.21.15.18:0/2685999777 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff61c0d1100 con 0x7ff62c100dd0 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.171+0000 7ff62be00640 1 --2- 172.21.15.18:0/2954642937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9cb0 0x5590702f2350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0ce2d0 tx=0x7ff61c0d9780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc5786c7bb661e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.171+0000 7ff631000640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6100a3da0 0x7ff6100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.171+0000 7ff621600640 1 -- 172.21.15.18:0/2954642937 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c098e80 con 0x5590702e9cb0 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.171+0000 7ff621600640 1 -- 172.21.15.18:0/2954642937 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c09d2e0 con 0x5590702e9cb0 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.171+0000 7ff621600640 1 -- 172.21.15.18:0/2954642937 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c09c540 con 0x5590702e9cb0 2024-04-16T17:12:11.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.172+0000 7ff635ddb740 1 -- 172.21.15.18:0/2954642937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9cb0 msgr2=0x5590702f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.172+0000 7ff635ddb740 1 --2- 172.21.15.18:0/2954642937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9cb0 0x5590702f2350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0ce2d0 tx=0x7ff61c0d9780 comp rx=0 tx=0).stop 2024-04-16T17:12:11.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.172+0000 7ff631000640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6100a3da0 0x7ff6100a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff624009370 tx=0x7ff624009520 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:11.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.172+0000 7ff635ddb740 1 -- 172.21.15.18:0/2954642937 shutdown_connections 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.172+0000 7ff635ddb740 1 --2- 172.21.15.18:0/2954642937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702f2890 0x5590702f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.172+0000 7ff635ddb740 1 --2- 172.21.15.18:0/2954642937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9cb0 0x5590702f2350 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.172+0000 7ff635ddb740 1 --2- 172.21.15.18:0/2954642937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.172+0000 7ff635ddb740 1 -- 172.21.15.18:0/2954642937 >> 172.21.15.18:0/2954642937 conn(0x5590702e3ac0 msgr2=0x5590702e3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.173+0000 7ff635ddb740 1 -- 172.21.15.18:0/2954642937 shutdown_connections 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.173+0000 7ff635ddb740 1 -- 172.21.15.18:0/2954642937 wait complete. 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.173+0000 7ff635ddb740 1 -- start start 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.174+0000 7ff635ddb740 1 Processor -- start 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.174+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9390 0x5590702ef950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.174+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702efe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.174+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702f2890 0x5590702f03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.174+0000 7ff635ddb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5590702f5620 con 0x5590702e9cb0 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.174+0000 7ff635ddb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559070208710 con 0x5590702e9390 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.174+0000 7ff635ddb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5590702f57a0 con 0x5590702f2890 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.174+0000 7ff635ddb740 10 client.0 _ll_register_callbacks cb 0x559070203af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9390 0x5590702ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9390 0x5590702ef950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33194/0 (socket says 172.21.15.18:33194) 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 -- 172.21.15.18:0/249856062 learned_addr learned my addr 172.21.15.18:0/249856062 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff631000640 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702f2890 0x5590702f03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62b400640 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 -- 172.21.15.18:0/249856062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702f2890 msgr2=0x5590702f03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702f2890 0x5590702f03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 -- 172.21.15.18:0/249856062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 msgr2=0x5590702efe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702efe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 -- 172.21.15.18:0/249856062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906ff613f0 con 0x5590702e9390 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.175+0000 7ff62be00640 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9390 0x5590702ef950 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0a5320 tx=0x7ff61c099480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b6c14123b8739c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.176+0000 7ff616000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.176+0000 7ff616000640 1 -- 172.21.15.18:0/249856062 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c09c720 con 0x5590702e9390 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.176+0000 7ff616000640 1 -- 172.21.15.18:0/249856062 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff61c09ce40 con 0x5590702e9390 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.176+0000 7ff635ddb740 1 -- 172.21.15.18:0/249856062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff61c09d2e0 con 0x5590702e9390 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.176+0000 7ff616000640 1 -- 172.21.15.18:0/249856062 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff61c09c9b0 con 0x5590702e9390 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.176+0000 7ff635ddb740 10 client.27665 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.176+0000 7ff635ddb740 1 -- 172.21.15.18:0/249856062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55906ff210f0 con 0x5590702e9390 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.176+0000 7ff616000640 1 -- 172.21.15.18:0/249856062 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c09d2e0 con 0x5590702e9390 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.177+0000 7ff616000640 1 -- 172.21.15.18:0/249856062 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff61c0a53c0 con 0x5590702e9390 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.177+0000 7ff635ddb740 10 client.27665 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.177+0000 7ff635ddb740 10 client.27665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:11.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.177+0000 7ff635ddb740 20 client.27665 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.177+0000 7ff635ddb740 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559070302b00 0x559070304ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.177+0000 7ff635ddb740 4 client.27665 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.177+0000 7ff635ddb740 1 -- 172.21.15.18:0/249856062 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55907025cb40 con 0x559070302b00 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.177+0000 7ff62b400640 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559070302b00 0x559070304ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.178+0000 7ff62b400640 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559070302b00 0x559070304ef0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20312b6477a342a server_cookie=37d1fbba9683eff7 in_seq=0 out_seq=0 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.178+0000 7ff616000640 10 client.27665 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.178+0000 7ff616000640 1 -- 172.21.15.18:0/249856062 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55907025cb40 con 0x559070302b00 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.178+0000 7ff616000640 10 client.27665 handle_command_reply: tid=0 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.178+0000 7ff635ddb740 1 client.27665 shutdown 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 -- 172.21.15.18:0/249856062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9390 msgr2=0x5590702ef950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9390 0x5590702ef950 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0a5320 tx=0x7ff61c099480 comp rx=0 tx=0).stop 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 -- 172.21.15.18:0/249856062 shutdown_connections 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559070302b00 0x559070304ef0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702f2890 0x5590702f03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702efe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 --2- 172.21.15.18:0/249856062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702e9390 0x5590702ef950 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff616000640 0 client.27665 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:11.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 -- 172.21.15.18:0/249856062 >> 172.21.15.18:0/249856062 conn(0x5590702e3ac0 msgr2=0x5590702e5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:11.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 -- 172.21.15.18:0/249856062 shutdown_connections 2024-04-16T17:12:11.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 1 -- 172.21.15.18:0/249856062 wait complete. 2024-04-16T17:12:11.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.179+0000 7ff635ddb740 20 client.27665 trim_cache size 0 max 16384 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff635ddb740 1 Processor -- start 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff635ddb740 1 -- start start 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702f2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x5590702e5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff635ddb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b85c0 con 0x5590702e9cb0 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff635ddb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b5e90 con 0x5590702f2890 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff635ddb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b3d00 con 0x5590702e9390 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff62be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff631000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x5590702e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff62be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702f2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39472/0 (socket says 172.21.15.18:39472) 2024-04-16T17:12:11.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff62b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.200+0000 7ff62be00640 1 -- 172.21.15.18:0/3944689516 learned_addr learned my addr 172.21.15.18:0/3944689516 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:11.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff62be00640 1 -- 172.21.15.18:0/3944689516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 msgr2=0x5590702e5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:11.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff62be00640 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x5590702e5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff62be00640 1 -- 172.21.15.18:0/3944689516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 msgr2=0x5590702e3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff62be00640 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702e3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff62be00640 1 -- 172.21.15.18:0/3944689516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590702e9ae0 con 0x5590702e9390 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff62b400640 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff631000640 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x5590702e5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff62be00640 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702f2080 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff61c09c8e0 tx=0x7ff61c0d9780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c001c11866546e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff620c00640 1 -- 172.21.15.18:0/3944689516 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c09c720 con 0x5590702e9390 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.201+0000 7ff620c00640 1 -- 172.21.15.18:0/3944689516 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c09d2e0 con 0x5590702e9390 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff620c00640 1 -- 172.21.15.18:0/3944689516 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c09c540 con 0x5590702e9390 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 -- 172.21.15.18:0/3944689516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 msgr2=0x5590702f2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702f2080 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff61c09c8e0 tx=0x7ff61c0d9780 comp rx=0 tx=0).stop 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 -- 172.21.15.18:0/3944689516 shutdown_connections 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x5590702e5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9cb0 0x5590702e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 --2- 172.21.15.18:0/3944689516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9390 0x5590702f2080 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 -- 172.21.15.18:0/3944689516 >> 172.21.15.18:0/3944689516 conn(0x5590702e3ac0 msgr2=0x5590702e4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:11.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 -- 172.21.15.18:0/3944689516 shutdown_connections 2024-04-16T17:12:11.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.202+0000 7ff635ddb740 1 -- 172.21.15.18:0/3944689516 wait complete. 2024-04-16T17:12:11.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.203+0000 7ff635ddb740 1 -- start start 2024-04-16T17:12:11.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.203+0000 7ff635ddb740 1 Processor -- start 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9390 0x5590703081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9cb0 0x559070305570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff635ddb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x559070305ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff635ddb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b4350 con 0x5590702e9390 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff635ddb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b57c0 con 0x5590702f2890 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff635ddb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b4a20 con 0x5590702e9cb0 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff635ddb740 10 client.0 _ll_register_callbacks cb 0x7ff61c0d9170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff62b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9cb0 0x559070305570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff62b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9cb0 0x559070305570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39474/0 (socket says 172.21.15.18:39474) 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff62b400640 1 -- 172.21.15.18:0/922846124 learned_addr learned my addr 172.21.15.18:0/922846124 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff62be00640 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9390 0x5590703081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.204+0000 7ff631000640 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x559070305ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.205+0000 7ff62b400640 1 -- 172.21.15.18:0/922846124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 msgr2=0x559070305ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.205+0000 7ff62b400640 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x559070305ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.205+0000 7ff62b400640 1 -- 172.21.15.18:0/922846124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9390 msgr2=0x5590703081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.205+0000 7ff62b400640 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9390 0x5590703081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.205+0000 7ff62b400640 1 -- 172.21.15.18:0/922846124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590703061a0 con 0x5590702e9cb0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.205+0000 7ff62b400640 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9cb0 0x559070305570 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff61802c2b0 tx=0x7ff6180041c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae1e3895cbb7f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.205+0000 7ff616000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.206+0000 7ff616000640 1 -- 172.21.15.18:0/922846124 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61804e060 con 0x5590702e9cb0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.206+0000 7ff635ddb740 1 -- 172.21.15.18:0/922846124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff61c09d2e0 con 0x5590702e9cb0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.206+0000 7ff616000640 1 -- 172.21.15.18:0/922846124 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9642+0+0 (secure 0 0 0) 0x7ff61802c950 con 0x5590702e9cb0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.206+0000 7ff616000640 1 -- 172.21.15.18:0/922846124 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6180044c0 con 0x5590702e9cb0 2024-04-16T17:12:11.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.206+0000 7ff635ddb740 10 client.27586 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:11.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.206+0000 7ff635ddb740 1 -- 172.21.15.18:0/922846124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559070368c50 con 0x5590702e9cb0 2024-04-16T17:12:11.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.206+0000 7ff616000640 1 -- 172.21.15.18:0/922846124 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c09d2e0 con 0x5590702e9cb0 2024-04-16T17:12:11.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.207+0000 7ff616000640 1 -- 172.21.15.18:0/922846124 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff618004880 con 0x5590702e9cb0 2024-04-16T17:12:11.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.207+0000 7ff635ddb740 10 client.27586 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:11.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.207+0000 7ff635ddb740 10 client.27586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.207+0000 7ff635ddb740 20 client.27586 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.207+0000 7ff635ddb740 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559070369380 0x55907036b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.207+0000 7ff635ddb740 4 client.27586 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.207+0000 7ff635ddb740 1 -- 172.21.15.18:0/922846124 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590702569b0 con 0x559070369380 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.207+0000 7ff62be00640 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559070369380 0x55907036b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.208+0000 7ff62be00640 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559070369380 0x55907036b7a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9733f1434c14e3 server_cookie=53567250537d98da in_seq=0 out_seq=0 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.208+0000 7ff616000640 10 client.27586 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.208+0000 7ff616000640 1 -- 172.21.15.18:0/922846124 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5590702569b0 con 0x559070369380 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.208+0000 7ff616000640 10 client.27586 handle_command_reply: tid=0 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.208+0000 7ff635ddb740 1 client.27586 shutdown 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff635ddb740 1 -- 172.21.15.18:0/922846124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9cb0 msgr2=0x559070305570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff635ddb740 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9cb0 0x559070305570 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff61802c2b0 tx=0x7ff6180041c0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff635ddb740 1 -- 172.21.15.18:0/922846124 shutdown_connections 2024-04-16T17:12:11.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff635ddb740 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559070369380 0x55907036b7a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff635ddb740 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590702f2890 0x559070305ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff635ddb740 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590702e9cb0 0x559070305570 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff616000640 0 client.27586 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:11.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff635ddb740 1 --2- 172.21.15.18:0/922846124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5590702e9390 0x5590703081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.209+0000 7ff635ddb740 1 -- 172.21.15.18:0/922846124 >> 172.21.15.18:0/922846124 conn(0x5590702e3ac0 msgr2=0x5590702fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:11.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.210+0000 7ff635ddb740 1 -- 172.21.15.18:0/922846124 shutdown_connections 2024-04-16T17:12:11.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.210+0000 7ff635ddb740 1 -- 172.21.15.18:0/922846124 wait complete. 2024-04-16T17:12:11.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.210+0000 7ff635ddb740 20 client.27586 trim_cache size 0 max 16384 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout: "f06e1071-6d4a-42a6-8f60-ed61202582a9": { 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "f06e1071-6d4a-42a6-8f60-ed61202582a9", 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:12:11.213 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 -- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6100a3da0 msgr2=0x7ff6100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6100a3da0 0x7ff6100a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff624009370 tx=0x7ff624009520 comp rx=0 tx=0).stop 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 -- 172.21.15.18:0/2685999777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c100dd0 msgr2=0x7ff62c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c100dd0 0x7ff62c19edf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00e9f0 tx=0x7ff61c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 -- 172.21.15.18:0/2685999777 shutdown_connections 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff6100a3da0 0x7ff6100a6260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff62c1061d0 0x7ff62c1a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff62c1016f0 0x7ff62c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 --2- 172.21.15.18:0/2685999777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62c100dd0 0x7ff62c19edf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 -- 172.21.15.18:0/2685999777 >> 172.21.15.18:0/2685999777 conn(0x7ff62c077ab0 msgr2=0x7ff62c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:11.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 -- 172.21.15.18:0/2685999777 shutdown_connections 2024-04-16T17:12:11.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:11.211+0000 7ff631a00640 1 -- 172.21.15.18:0/2685999777 wait complete. 2024-04-16T17:12:11.235 INFO:tasks.cephfs.filesystem:scrub status for tag:f06e1071-6d4a-42a6-8f60-ed61202582a9 - {'path': '/', 'tag': 'f06e1071-6d4a-42a6-8f60-ed61202582a9', 'options': 'recursive,force'} 2024-04-16T17:12:11.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 285016 3.92 MB/sec execute 264 sec latency 21.902 ms 2024-04-16T17:12:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:11 smithi022 ceph-mon[36023]: pgmap v231: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-04-16T17:12:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:11 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:12:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:11 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:11.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:11 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:12:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:11 smithi043 ceph-mon[36901]: pgmap v231: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-04-16T17:12:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:11 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:12:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:11 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:11 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:12:12.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 286221 3.92 MB/sec execute 265 sec latency 14.955 ms 2024-04-16T17:12:12.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:12 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:13.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:12 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:13.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 287159 3.94 MB/sec execute 266 sec latency 85.375 ms 2024-04-16T17:12:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:13 smithi022 ceph-mon[36023]: pgmap v232: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-16T17:12:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:13 smithi022 ceph-mon[36023]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T17:12:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:13 smithi043 ceph-mon[36901]: pgmap v232: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-16T17:12:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:13 smithi043 ceph-mon[36901]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T17:12:14.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 287531 3.94 MB/sec execute 267 sec latency 91.297 ms 2024-04-16T17:12:14.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:14 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:12:15.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:14 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:12:15.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 287812 3.92 MB/sec execute 268 sec latency 62.610 ms 2024-04-16T17:12:15.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:15 smithi022 ceph-mon[36023]: pgmap v234: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-04-16T17:12:15.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:07 smithi018 ceph-mon[31776]: pgmap v229: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-16T17:12:15.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:09 smithi018 ceph-mon[31776]: pgmap v230: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-16T17:12:15.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:09 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/324295756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:15.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:11 smithi018 ceph-mon[31776]: pgmap v231: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-04-16T17:12:15.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:11 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:12:15.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:11 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:15.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:11 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:12:15.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:12 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:15.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:13 smithi018 ceph-mon[31776]: pgmap v232: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-16T17:12:15.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:13 smithi018 ceph-mon[31776]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T17:12:15.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:14 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:12:15.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:15 smithi018 ceph-mon[31776]: pgmap v234: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-04-16T17:12:16.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:15 smithi043 ceph-mon[36901]: pgmap v234: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-04-16T17:12:16.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 288429 3.92 MB/sec execute 269 sec latency 89.946 ms 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 -- 172.21.15.18:0/2134883080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10cc102c00 msgr2=0x7f10cc106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 --2- 172.21.15.18:0/2134883080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10cc102c00 0x7f10cc106c40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009900 tx=0x7f10b802bc30 comp rx=0 tx=0).stop 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 -- 172.21.15.18:0/2134883080 shutdown_connections 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 --2- 172.21.15.18:0/2134883080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc107320 0x7f10cc10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 --2- 172.21.15.18:0/2134883080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10cc102c00 0x7f10cc106c40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 --2- 172.21.15.18:0/2134883080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10cc102250 0x7f10cc102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 -- 172.21.15.18:0/2134883080 >> 172.21.15.18:0/2134883080 conn(0x7f10cc0fdaf0 msgr2=0x7f10cc0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 -- 172.21.15.18:0/2134883080 shutdown_connections 2024-04-16T17:12:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.386+0000 7f10d1400640 1 -- 172.21.15.18:0/2134883080 wait complete. 2024-04-16T17:12:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.387+0000 7f10d1400640 1 Processor -- start 2024-04-16T17:12:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.387+0000 7f10d1400640 1 -- start start 2024-04-16T17:12:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10d1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc102250 0x7f10cc1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10cc107320 0x7f10cc1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10d1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10cc19a940 0x7f10cc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10d1400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1123e0 con 0x7f10cc19a940 2024-04-16T17:12:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10d1400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc112260 con 0x7f10cc107320 2024-04-16T17:12:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10d1400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc112560 con 0x7f10cc102250 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10cc107320 0x7f10cc1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10cc107320 0x7f10cc1a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54496/0 (socket says 172.21.15.18:54496) 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10ca800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc102250 0x7f10cc1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10c9e00640 1 -- 172.21.15.18:0/2149446991 learned_addr learned my addr 172.21.15.18:0/2149446991 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10cb200640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10cc19a940 0x7f10cc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10c9e00640 1 -- 172.21.15.18:0/2149446991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc102250 msgr2=0x7f10cc1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.388+0000 7f10c9e00640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc102250 0x7f10cc1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10c9e00640 1 -- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10cc19a940 msgr2=0x7f10cc19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10c9e00640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10cc19a940 0x7f10cc19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10c9e00640 1 -- 172.21.15.18:0/2149446991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc19b460 con 0x7f10cc107320 2024-04-16T17:12:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10c9e00640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10cc107320 0x7f10cc1a0cd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f10b802c240 tx=0x7f10b8004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10c3400640 1 -- 172.21.15.18:0/2149446991 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8043020 con 0x7f10cc107320 2024-04-16T17:12:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10d1400640 1 -- 172.21.15.18:0/2149446991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc19b6f0 con 0x7f10cc107320 2024-04-16T17:12:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10c3400640 1 -- 172.21.15.18:0/2149446991 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b80054e0 con 0x7f10cc107320 2024-04-16T17:12:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10c3400640 1 -- 172.21.15.18:0/2149446991 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8003bb0 con 0x7f10cc107320 2024-04-16T17:12:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.389+0000 7f10d1400640 1 -- 172.21.15.18:0/2149446991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1a7990 con 0x7f10cc107320 2024-04-16T17:12:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.394+0000 7f10c3400640 1 -- 172.21.15.18:0/2149446991 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10b8046020 con 0x7f10cc107320 2024-04-16T17:12:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.394+0000 7f10d41b4740 1 Processor -- start 2024-04-16T17:12:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.394+0000 7f10d41b4740 1 -- start start 2024-04-16T17:12:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a5428f2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a5428f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10d41b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a542857780 con 0x55a5428f28a0 2024-04-16T17:12:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10d41b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a5427fdf40 con 0x55a5428fbe20 2024-04-16T17:12:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10d41b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a542853740 con 0x55a5428f3610 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10cb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a5428f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10cb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a5428f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:60614/0 (socket says 172.21.15.18:60614) 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.395+0000 7f10cb200640 1 -- 172.21.15.18:0/87237602 learned_addr learned my addr 172.21.15.18:0/87237602 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10ca800640 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a5428f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10c9e00640 1 -- 172.21.15.18:0/87237602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 msgr2=0x55a5428f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10c9e00640 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a5428f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10c3400640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f10b00a3d90 0x7f10b00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10c3400640 1 -- 172.21.15.18:0/2149446991 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f10b80f2810 con 0x7f10cc107320 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10c9e00640 1 -- 172.21.15.18:0/87237602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 msgr2=0x55a5428f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10c9e00640 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a5428f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10c9e00640 1 -- 172.21.15.18:0/87237602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54256a210 con 0x55a5428fbe20 2024-04-16T17:12:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10ca800640 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a5428f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.396+0000 7f10cb200640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f10b00a3d90 0x7f10b00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.397+0000 7f10c9e00640 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fe210 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f10b80efc00 tx=0x7f10b8004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30e37c320a5031e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.397+0000 7f10b7400640 1 -- 172.21.15.18:0/87237602 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b80ba6f0 con 0x55a5428fbe20 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.397+0000 7f10b7400640 1 -- 172.21.15.18:0/87237602 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b8004c80 con 0x55a5428fbe20 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.397+0000 7f10b7400640 1 -- 172.21.15.18:0/87237602 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b802cb90 con 0x55a5428fbe20 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 -- 172.21.15.18:0/87237602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 msgr2=0x55a5428fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fe210 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f10b80efc00 tx=0x7f10b8004610 comp rx=0 tx=0).stop 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10cb200640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f10b00a3d90 0x7f10b00a6250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f10c4011630 tx=0x7f10c4009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 -- 172.21.15.18:0/87237602 shutdown_connections 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fe210 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a5428f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 --2- 172.21.15.18:0/87237602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a5428f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 -- 172.21.15.18:0/87237602 >> 172.21.15.18:0/87237602 conn(0x55a54288d5d0 msgr2=0x55a5428ed590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 -- 172.21.15.18:0/87237602 shutdown_connections 2024-04-16T17:12:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.398+0000 7f10d41b4740 1 -- 172.21.15.18:0/87237602 wait complete. 2024-04-16T17:12:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 1 -- start start 2024-04-16T17:12:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 1 Processor -- start 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 0x55a5428fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428f3610 0x55a5428f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428fbe20 0x55a5428f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a5427fdf40 con 0x55a5428fbe20 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a542811390 con 0x55a5428f3610 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a5428fe910 con 0x55a5428f28a0 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.399+0000 7f10d41b4740 10 client.0 _ll_register_callbacks cb 0x55a5427faf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428f3610 0x55a5428f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428f3610 0x55a5428f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54520/0 (socket says 172.21.15.18:54520) 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 -- 172.21.15.18:0/1185767762 learned_addr learned my addr 172.21.15.18:0/1185767762 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10ca800640 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 0x55a5428fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 -- 172.21.15.18:0/1185767762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 msgr2=0x55a5428fb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 0x55a5428fb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 -- 172.21.15.18:0/1185767762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428fbe20 msgr2=0x55a5428f93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428fbe20 0x55a5428f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 -- 172.21.15.18:0/1185767762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a542636620 con 0x55a5428f3610 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10c9e00640 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428f3610 0x55a5428f8c70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f10b80096c0 tx=0x7f10b80035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b162d7375ab65fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.400+0000 7f10abe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.401+0000 7f10abe00640 1 -- 172.21.15.18:0/1185767762 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b80bddb0 con 0x55a5428f3610 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.401+0000 7f10abe00640 1 -- 172.21.15.18:0/1185767762 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f10b80efd70 con 0x55a5428f3610 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.401+0000 7f10d41b4740 1 -- 172.21.15.18:0/1185767762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b8004c80 con 0x55a5428f3610 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.401+0000 7f10abe00640 1 -- 172.21.15.18:0/1185767762 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10b80bdf90 con 0x55a5428f3610 2024-04-16T17:12:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.401+0000 7f10d41b4740 10 client.27683 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.401+0000 7f10d41b4740 1 -- 172.21.15.18:0/1185767762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a54290a7c0 con 0x55a5428f3610 2024-04-16T17:12:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.401+0000 7f10abe00640 1 -- 172.21.15.18:0/1185767762 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b8004c80 con 0x55a5428f3610 2024-04-16T17:12:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.402+0000 7f10abe00640 1 -- 172.21.15.18:0/1185767762 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f10b8050060 con 0x55a5428f3610 2024-04-16T17:12:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.402+0000 7f10d41b4740 10 client.27683 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.402+0000 7f10d41b4740 10 client.27683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.402+0000 7f10d41b4740 20 client.27683 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.402+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a54290b990 0x55a54290dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.402+0000 7f10d41b4740 4 client.27683 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.402+0000 7f10d41b4740 1 -- 172.21.15.18:0/1185767762 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5428651a0 con 0x55a54290b990 2024-04-16T17:12:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.403+0000 7f10ca800640 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a54290b990 0x55a54290dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.403+0000 7f10ca800640 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a54290b990 0x55a54290dd80 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba090cf26877c41 server_cookie=610841cac60f4fcf in_seq=0 out_seq=0 2024-04-16T17:12:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.403+0000 7f10abe00640 10 client.27683 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10abe00640 1 -- 172.21.15.18:0/1185767762 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5428651a0 con 0x55a54290b990 2024-04-16T17:12:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10abe00640 10 client.27683 handle_command_reply: tid=0 2024-04-16T17:12:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 client.27683 shutdown 2024-04-16T17:12:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 -- 172.21.15.18:0/1185767762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428f3610 msgr2=0x55a5428f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428f3610 0x55a5428f8c70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f10b80096c0 tx=0x7f10b80035f0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 -- 172.21.15.18:0/1185767762 shutdown_connections 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a54290b990 0x55a54290dd80 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428fbe20 0x55a5428f93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428f3610 0x55a5428f8c70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1185767762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 0x55a5428fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10abe00640 0 client.27683 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.405+0000 7f10d41b4740 1 -- 172.21.15.18:0/1185767762 >> 172.21.15.18:0/1185767762 conn(0x55a54288d5d0 msgr2=0x55a5428ef2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.406+0000 7f10d41b4740 1 -- 172.21.15.18:0/1185767762 shutdown_connections 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.406+0000 7f10d41b4740 1 -- 172.21.15.18:0/1185767762 wait complete. 2024-04-16T17:12:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.406+0000 7f10d41b4740 20 client.27683 trim_cache size 0 max 16384 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10d41b4740 1 Processor -- start 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10d41b4740 1 -- start start 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 0x55a5428ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f3610 0x55a5428fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10d41b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1afe20 con 0x55a5428f3610 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10d41b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1ad6f0 con 0x55a5428fbe20 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10d41b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1ab5c0 con 0x55a5428f28a0 2024-04-16T17:12:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.423+0000 7f10cb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f3610 0x55a5428fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10cb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54522/0 (socket says 172.21.15.18:54522) 2024-04-16T17:12:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f3610 0x55a5428fb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60672/0 (socket says 172.21.15.18:60672) 2024-04-16T17:12:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10cb200640 1 -- 172.21.15.18:0/1262027851 learned_addr learned my addr 172.21.15.18:0/1262027851 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10ca800640 1 --2- 172.21.15.18:0/1262027851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 0x55a5428ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10cb200640 1 -- 172.21.15.18:0/1262027851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 msgr2=0x55a5428ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10cb200640 1 --2- 172.21.15.18:0/1262027851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 0x55a5428ec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10cb200640 1 -- 172.21.15.18:0/1262027851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f3610 msgr2=0x55a5428fb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10cb200640 1 --2- 172.21.15.18:0/1262027851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f3610 0x55a5428fb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10cb200640 1 -- 172.21.15.18:0/1262027851 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5428f1b70 con 0x55a5428fbe20 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.424+0000 7f10cb200640 1 --2- 172.21.15.18:0/1262027851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fb8e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f10c4003c90 tx=0x7f10c4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea04c36615752ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.425+0000 7f10b6a00640 1 -- 172.21.15.18:0/1262027851 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4003cc0 con 0x55a5428fbe20 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.425+0000 7f10b6a00640 1 -- 172.21.15.18:0/1262027851 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c400fa40 con 0x55a5428fbe20 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.425+0000 7f10b6a00640 1 -- 172.21.15.18:0/1262027851 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c401ea40 con 0x55a5428fbe20 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.425+0000 7f10d41b4740 1 -- 172.21.15.18:0/1262027851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 msgr2=0x55a5428fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.425+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1262027851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fb8e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f10c4003c90 tx=0x7f10c4004a40 comp rx=0 tx=0).stop 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.425+0000 7f10d41b4740 1 -- 172.21.15.18:0/1262027851 shutdown_connections 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.425+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1262027851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5428fb8e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.425+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1262027851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f3610 0x55a5428fb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.426+0000 7f10d41b4740 1 --2- 172.21.15.18:0/1262027851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f28a0 0x55a5428ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.426+0000 7f10d41b4740 1 -- 172.21.15.18:0/1262027851 >> 172.21.15.18:0/1262027851 conn(0x55a54288d5d0 msgr2=0x55a5428ebe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.426+0000 7f10d41b4740 1 -- 172.21.15.18:0/1262027851 shutdown_connections 2024-04-16T17:12:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.426+0000 7f10d41b4740 1 -- 172.21.15.18:0/1262027851 wait complete. 2024-04-16T17:12:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.426+0000 7f10d41b4740 1 -- start start 2024-04-16T17:12:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.426+0000 7f10d41b4740 1 Processor -- start 2024-04-16T17:12:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a542900b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a542901070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10d41b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5429015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10d41b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1abc10 con 0x55a5428f28a0 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10d41b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1ad020 con 0x55a5428fbe20 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10d41b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1ac280 con 0x55a5428f3610 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10d41b4740 10 client.0 _ll_register_callbacks cb 0x7f10c4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10ca800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a542900b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a542901070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10ca800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a542900b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60680/0 (socket says 172.21.15.18:60680) 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10ca800640 1 -- 172.21.15.18:0/3617915377 learned_addr learned my addr 172.21.15.18:0/3617915377 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10c9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a542901070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:60656/0 (socket says 172.21.15.18:60656) 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.427+0000 7f10cb200640 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5429015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10c9e00640 1 -- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 msgr2=0x55a5429015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10c9e00640 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5429015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10c9e00640 1 -- 172.21.15.18:0/3617915377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 msgr2=0x55a542900b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10c9e00640 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a542900b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10c9e00640 1 -- 172.21.15.18:0/3617915377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5428ef890 con 0x55a5428f3610 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10ca800640 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a542900b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10cb200640 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5429015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10c9e00640 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a542901070 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f10b80be150 tx=0x7f10b8004ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691f462ae3f3fd81 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10abe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10abe00640 1 -- 172.21.15.18:0/3617915377 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b80be220 con 0x55a5428f3610 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10abe00640 1 -- 172.21.15.18:0/3617915377 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f10b80efce0 con 0x55a5428f3610 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.428+0000 7f10d41b4740 1 -- 172.21.15.18:0/3617915377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10c400fa40 con 0x55a5428f3610 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.429+0000 7f10abe00640 1 -- 172.21.15.18:0/3617915377 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10b80bddb0 con 0x55a5428f3610 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.429+0000 7f10d41b4740 10 client.27604 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.429+0000 7f10d41b4740 1 -- 172.21.15.18:0/3617915377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5428ef890 con 0x55a5428f3610 2024-04-16T17:12:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.429+0000 7f10abe00640 1 -- 172.21.15.18:0/3617915377 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c400fa40 con 0x55a5428f3610 2024-04-16T17:12:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10abe00640 1 -- 172.21.15.18:0/3617915377 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f10b80f2370 con 0x55a5428f3610 2024-04-16T17:12:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10d41b4740 10 client.27604 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10d41b4740 10 client.27604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10d41b4740 20 client.27604 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10d41b4740 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a5429731e0 0x55a5429755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10d41b4740 4 client.27604 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10d41b4740 1 -- 172.21.15.18:0/3617915377 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a54285f010 con 0x55a5429731e0 2024-04-16T17:12:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10ca800640 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a5429731e0 0x55a5429755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10ca800640 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a5429731e0 0x55a5429755d0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678390553ee4e9fe server_cookie=7f11f0f1697d04e5 in_seq=0 out_seq=0 2024-04-16T17:12:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.430+0000 7f10abe00640 10 client.27604 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:17.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 289433 3.92 MB/sec execute 270 sec latency 89.062 ms 2024-04-16T17:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.601+0000 7f10abe00640 1 -- 172.21.15.18:0/3617915377 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a54285f010 con 0x55a5429731e0 2024-04-16T17:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.601+0000 7f10abe00640 10 client.27604 handle_command_reply: tid=0 2024-04-16T17:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 client.27604 shutdown 2024-04-16T17:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 -- 172.21.15.18:0/3617915377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 msgr2=0x55a542901070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a542901070 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f10b80be150 tx=0x7f10b8004ee0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 -- 172.21.15.18:0/3617915377 shutdown_connections 2024-04-16T17:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a5429731e0 0x55a5429755d0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5428fbe20 0x55a5429015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5428f3610 0x55a542901070 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10abe00640 0 client.27604 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 --2- 172.21.15.18:0/3617915377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a5428f28a0 0x55a542900b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.602+0000 7f10d41b4740 1 -- 172.21.15.18:0/3617915377 >> 172.21.15.18:0/3617915377 conn(0x55a54288d5d0 msgr2=0x55a5428ef0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.603+0000 7f10d41b4740 1 -- 172.21.15.18:0/3617915377 shutdown_connections 2024-04-16T17:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.603+0000 7f10d41b4740 1 -- 172.21.15.18:0/3617915377 wait complete. 2024-04-16T17:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.603+0000 7f10d41b4740 20 client.27604 trim_cache size 0 max 16384 2024-04-16T17:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:12:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.604+0000 7f10d1400640 1 -- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f10b00a3d90 msgr2=0x7f10b00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.604+0000 7f10d1400640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f10b00a3d90 0x7f10b00a6250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f10c4011630 tx=0x7f10c4009500 comp rx=0 tx=0).stop 2024-04-16T17:12:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.604+0000 7f10d1400640 1 -- 172.21.15.18:0/2149446991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10cc107320 msgr2=0x7f10cc1a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.604+0000 7f10d1400640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10cc107320 0x7f10cc1a0cd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f10b802c240 tx=0x7f10b8004360 comp rx=0 tx=0).stop 2024-04-16T17:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.605+0000 7f10d1400640 1 -- 172.21.15.18:0/2149446991 shutdown_connections 2024-04-16T17:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.605+0000 7f10d1400640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f10b00a3d90 0x7f10b00a6250 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.605+0000 7f10d1400640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10cc19a940 0x7f10cc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.605+0000 7f10d1400640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10cc107320 0x7f10cc1a0cd0 secure :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f10b802c240 tx=0x7f10b8004360 comp rx=0 tx=0).stop 2024-04-16T17:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.605+0000 7f10d1400640 1 --2- 172.21.15.18:0/2149446991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc102250 0x7f10cc1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.605+0000 7f10d1400640 1 -- 172.21.15.18:0/2149446991 >> 172.21.15.18:0/2149446991 conn(0x7f10cc0fdaf0 msgr2=0x7f10cc0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.605+0000 7f10d1400640 1 -- 172.21.15.18:0/2149446991 shutdown_connections 2024-04-16T17:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:17.605+0000 7f10d1400640 1 -- 172.21.15.18:0/2149446991 wait complete. 2024-04-16T17:12:17.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:17 smithi022 ceph-mon[36023]: pgmap v235: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-16T17:12:18.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:17 smithi043 ceph-mon[36901]: pgmap v235: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-16T17:12:18.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 290326 3.93 MB/sec execute 271 sec latency 61.713 ms 2024-04-16T17:12:18.746 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:17 smithi018 ceph-mon[31776]: pgmap v235: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-16T17:12:19.601 INFO:tasks.workunit.client.0.smithi018.stdout: 1 290825 3.93 MB/sec execute 272 sec latency 93.401 ms 2024-04-16T17:12:19.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:19 smithi022 ceph-mon[36023]: pgmap v236: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-16T17:12:20.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:19 smithi043 ceph-mon[36901]: pgmap v236: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-16T17:12:20.602 INFO:tasks.workunit.client.0.smithi018.stdout: 1 291154 3.92 MB/sec execute 273 sec latency 135.658 ms 2024-04-16T17:12:21.298 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:19 smithi018 ceph-mon[31776]: pgmap v236: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-16T17:12:21.602 INFO:tasks.workunit.client.0.smithi018.stdout: 1 291683 3.91 MB/sec execute 274 sec latency 24.043 ms 2024-04-16T17:12:21.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:21 smithi022 ceph-mon[36023]: pgmap v237: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-04-16T17:12:22.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:21 smithi043 ceph-mon[36901]: pgmap v237: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-04-16T17:12:22.602 INFO:tasks.workunit.client.0.smithi018.stdout: 1 292598 3.91 MB/sec execute 275 sec latency 15.662 ms 2024-04-16T17:12:23.602 INFO:tasks.workunit.client.0.smithi018.stdout: 1 293465 3.91 MB/sec execute 276 sec latency 98.079 ms 2024-04-16T17:12:23.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:23 smithi022 ceph-mon[36023]: pgmap v238: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-16T17:12:24.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:23 smithi043 ceph-mon[36901]: pgmap v238: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-16T17:12:24.602 INFO:tasks.workunit.client.0.smithi018.stdout: 1 293894 3.91 MB/sec execute 277 sec latency 127.282 ms 2024-04-16T17:12:25.602 INFO:tasks.workunit.client.0.smithi018.stdout: 1 294648 3.92 MB/sec execute 278 sec latency 80.881 ms 2024-04-16T17:12:25.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:25 smithi022 ceph-mon[36023]: pgmap v239: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-16T17:12:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:25 smithi043 ceph-mon[36901]: pgmap v239: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-16T17:12:26.602 INFO:tasks.workunit.client.0.smithi018.stdout: 1 295204 3.91 MB/sec execute 279 sec latency 17.285 ms 2024-04-16T17:12:27.602 INFO:tasks.workunit.client.0.smithi018.stdout: 1 296302 3.91 MB/sec execute 280 sec latency 12.544 ms 2024-04-16T17:12:27.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:27 smithi022 ceph-mon[36023]: pgmap v240: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-04-16T17:12:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:27 smithi043 ceph-mon[36901]: pgmap v240: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-04-16T17:12:28.604 INFO:tasks.workunit.client.0.smithi018.stdout: 1 297194 3.92 MB/sec execute 281 sec latency 112.071 ms 2024-04-16T17:12:29.606 INFO:tasks.workunit.client.0.smithi018.stdout: 1 297638 3.91 MB/sec execute 282 sec latency 63.453 ms 2024-04-16T17:12:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:29 smithi022 ceph-mon[36023]: pgmap v241: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-16T17:12:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:29 smithi043 ceph-mon[36901]: pgmap v241: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-16T17:12:30.604 INFO:tasks.workunit.client.0.smithi018.stdout: 1 298249 3.92 MB/sec execute 283 sec latency 73.941 ms 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 -- 172.21.15.18:0/3419319331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8102080 msgr2=0x7f57f8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 --2- 172.21.15.18:0/3419319331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8102080 0x7f57f8102460 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009900 tx=0x7f57ec02bc30 comp rx=0 tx=0).stop 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 -- 172.21.15.18:0/3419319331 shutdown_connections 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 --2- 172.21.15.18:0/3419319331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f8107150 0x7f57f810ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 --2- 172.21.15.18:0/3419319331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8102a30 0x7f57f8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 --2- 172.21.15.18:0/3419319331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8102080 0x7f57f8102460 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 -- 172.21.15.18:0/3419319331 >> 172.21.15.18:0/3419319331 conn(0x7f57f80fd940 msgr2=0x7f57f80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 -- 172.21.15.18:0/3419319331 shutdown_connections 2024-04-16T17:12:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.760+0000 7f57fea00640 1 -- 172.21.15.18:0/3419319331 wait complete. 2024-04-16T17:12:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.761+0000 7f57fea00640 1 Processor -- start 2024-04-16T17:12:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fea00640 1 -- start start 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8102080 0x7f57f81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8107150 0x7f57f81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f819a800 0x7f57f819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fea00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8112260 con 0x7f57f8102080 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81120e0 con 0x7f57f8107150 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81123e0 con 0x7f57f819a800 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8107150 0x7f57f81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8107150 0x7f57f81a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34606/0 (socket says 172.21.15.18:34606) 2024-04-16T17:12:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fcc00640 1 -- 172.21.15.18:0/594315718 learned_addr learned my addr 172.21.15.18:0/594315718 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fd600640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8102080 0x7f57f81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.762+0000 7f57fe000640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f819a800 0x7f57f819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57fcc00640 1 -- 172.21.15.18:0/594315718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f819a800 msgr2=0x7f57f819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57fcc00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f819a800 0x7f57f819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57fcc00640 1 -- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8102080 msgr2=0x7f57f81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57fcc00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8102080 0x7f57f81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57fcc00640 1 -- 172.21.15.18:0/594315718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f819b390 con 0x7f57f8107150 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57fd600640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8102080 0x7f57f81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57fcc00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8107150 0x7f57f81a0b80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e9c0 tx=0x7f57e400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57f6000640 1 -- 172.21.15.18:0/594315718 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e400cd80 con 0x7f57f8107150 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57f6000640 1 -- 172.21.15.18:0/594315718 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e4005500 con 0x7f57f8107150 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.763+0000 7f57f6000640 1 -- 172.21.15.18:0/594315718 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4010690 con 0x7f57f8107150 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.764+0000 7f57fea00640 1 -- 172.21.15.18:0/594315718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f819b680 con 0x7f57f8107150 2024-04-16T17:12:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.764+0000 7f57fea00640 1 -- 172.21.15.18:0/594315718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f81a77c0 con 0x7f57f8107150 2024-04-16T17:12:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.767+0000 7f5802bf0740 1 Processor -- start 2024-04-16T17:12:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.767+0000 7f5802bf0740 1 -- start start 2024-04-16T17:12:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x56139516f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f57f6000640 1 -- 172.21.15.18:0/594315718 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f57e40046c0 con 0x7f57f8107150 2024-04-16T17:12:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x56139516ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x56139517b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f5802bf0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5613950d4780 con 0x5613951707f0 2024-04-16T17:12:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f5802bf0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56139507af40 con 0x56139516fa80 2024-04-16T17:12:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f5802bf0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5613950d0740 con 0x561395179000 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f57fd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x56139516ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f57fd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x56139516ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34612/0 (socket says 172.21.15.18:34612) 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f57fd600640 1 -- 172.21.15.18:0/1792546175 learned_addr learned my addr 172.21.15.18:0/1792546175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.768+0000 7f57fe000640 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x56139516f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.769+0000 7f57fcc00640 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x56139517b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.769+0000 7f57fd600640 1 -- 172.21.15.18:0/1792546175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 msgr2=0x56139517b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.769+0000 7f57fd600640 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x56139517b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.769+0000 7f57fd600640 1 -- 172.21.15.18:0/1792546175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 msgr2=0x56139516f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.769+0000 7f57fd600640 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x56139516f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.769+0000 7f57fd600640 1 -- 172.21.15.18:0/1792546175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561394de7210 con 0x56139516fa80 2024-04-16T17:12:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.769+0000 7f57fcc00640 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x56139517b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.769+0000 7f57fd600640 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x56139516ff00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009230 tx=0x7f57ec009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=335a6e599e9c6a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.770+0000 7f57ea600640 1 -- 172.21.15.18:0/1792546175 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec045070 con 0x56139516fa80 2024-04-16T17:12:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.770+0000 7f57ea600640 1 -- 172.21.15.18:0/1792546175 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57ec02c650 con 0x56139516fa80 2024-04-16T17:12:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.770+0000 7f57ea600640 1 -- 172.21.15.18:0/1792546175 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec005180 con 0x56139516fa80 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.770+0000 7f57f6000640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57e00a3e00 0x7f57e00a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.770+0000 7f57f6000640 1 -- 172.21.15.18:0/594315718 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f57e40d0f60 con 0x7f57f8107150 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.770+0000 7f57fe000640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57e00a3e00 0x7f57e00a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 -- 172.21.15.18:0/1792546175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 msgr2=0x56139516ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x56139516ff00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009230 tx=0x7f57ec009560 comp rx=0 tx=0).stop 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 -- 172.21.15.18:0/1792546175 shutdown_connections 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x56139517b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x56139516ff00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 --2- 172.21.15.18:0/1792546175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x56139516f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 -- 172.21.15.18:0/1792546175 >> 172.21.15.18:0/1792546175 conn(0x56139510a5d0 msgr2=0x56139516a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f57fe000640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57e00a3e00 0x7f57e00a62c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005fd0 tx=0x7f57f0009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 -- 172.21.15.18:0/1792546175 shutdown_connections 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.771+0000 7f5802bf0740 1 -- 172.21.15.18:0/1792546175 wait complete. 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.772+0000 7f5802bf0740 1 -- start start 2024-04-16T17:12:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.772+0000 7f5802bf0740 1 Processor -- start 2024-04-16T17:12:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x561395178a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x561395175e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x561395176550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f5802bf0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56139517bcd0 con 0x5613951707f0 2024-04-16T17:12:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f5802bf0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56139508e390 con 0x56139516fa80 2024-04-16T17:12:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f5802bf0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56139517be50 con 0x561395179000 2024-04-16T17:12:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f5802bf0740 10 client.0 _ll_register_callbacks cb 0x561395077f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f57fd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x561395178a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f57fd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x561395178a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34616/0 (socket says 172.21.15.18:34616) 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f57fd600640 1 -- 172.21.15.18:0/2426140085 learned_addr learned my addr 172.21.15.18:0/2426140085 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.773+0000 7f57fcc00640 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x561395175e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57fd600640 1 -- 172.21.15.18:0/2426140085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 msgr2=0x561395176550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57fd600640 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x561395176550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57fd600640 1 -- 172.21.15.18:0/2426140085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 msgr2=0x561395175e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57fd600640 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x561395175e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57fd600640 1 -- 172.21.15.18:0/2426140085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561394eb3620 con 0x56139516fa80 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57fcc00640 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x561395175e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57fd600640 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x561395178a80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f57ec02c820 tx=0x7f57ec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=226bd23a9fbf5b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57dea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57dea00640 1 -- 172.21.15.18:0/2426140085 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec050070 con 0x56139516fa80 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f5802bf0740 1 -- 172.21.15.18:0/2426140085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ec02c650 con 0x56139516fa80 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57dea00640 1 -- 172.21.15.18:0/2426140085 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f57ec0053f0 con 0x56139516fa80 2024-04-16T17:12:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.774+0000 7f57dea00640 1 -- 172.21.15.18:0/2426140085 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f57ec059050 con 0x56139516fa80 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f5802bf0740 10 client.27701 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f5802bf0740 1 -- 172.21.15.18:0/2426140085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561395187d40 con 0x56139516fa80 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f57dea00640 1 -- 172.21.15.18:0/2426140085 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57ec02c650 con 0x56139516fa80 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f57dea00640 1 -- 172.21.15.18:0/2426140085 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f57ec053020 con 0x56139516fa80 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f5802bf0740 10 client.27701 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f5802bf0740 10 client.27701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f5802bf0740 20 client.27701 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561395189090 0x56139518b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f5802bf0740 4 client.27701 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.775+0000 7f5802bf0740 1 -- 172.21.15.18:0/2426140085 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613950e21a0 con 0x561395189090 2024-04-16T17:12:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.776+0000 7f57fcc00640 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561395189090 0x56139518b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.776+0000 7f57fcc00640 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561395189090 0x56139518b480 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56c6ba1cc6e7ea6 server_cookie=e288fac293edbe3a in_seq=0 out_seq=0 2024-04-16T17:12:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.776+0000 7f57dea00640 10 client.27701 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f57dea00640 1 -- 172.21.15.18:0/2426140085 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613950e21a0 con 0x561395189090 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f57dea00640 10 client.27701 handle_command_reply: tid=0 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 client.27701 shutdown 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 -- 172.21.15.18:0/2426140085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 msgr2=0x561395178a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x561395178a80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f57ec02c820 tx=0x7f57ec009260 comp rx=0 tx=0).stop 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 -- 172.21.15.18:0/2426140085 shutdown_connections 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561395189090 0x56139518b480 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x561395176550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5613951707f0 0x561395175e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2426140085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139516fa80 0x561395178a80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f57dea00640 0 client.27701 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 -- 172.21.15.18:0/2426140085 >> 172.21.15.18:0/2426140085 conn(0x56139510a5d0 msgr2=0x56139516c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 -- 172.21.15.18:0/2426140085 shutdown_connections 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 1 -- 172.21.15.18:0/2426140085 wait complete. 2024-04-16T17:12:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.778+0000 7f5802bf0740 20 client.27701 trim_cache size 0 max 16384 2024-04-16T17:12:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.796+0000 7f5802bf0740 1 Processor -- start 2024-04-16T17:12:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f5802bf0740 1 -- start start 2024-04-16T17:12:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395174e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613951707f0 0x561395178540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x561395178a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f5802bf0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b3f70 con 0x56139516fa80 2024-04-16T17:12:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f5802bf0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b1840 con 0x5613951707f0 2024-04-16T17:12:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f5802bf0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81af6b0 con 0x561395179000 2024-04-16T17:12:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f57fcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613951707f0 0x561395178540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f57fcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613951707f0 0x561395178540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34626/0 (socket says 172.21.15.18:34626) 2024-04-16T17:12:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f57fcc00640 1 -- 172.21.15.18:0/2277743746 learned_addr learned my addr 172.21.15.18:0/2277743746 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.797+0000 7f57fd600640 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395174e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fcc00640 1 -- 172.21.15.18:0/2277743746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 msgr2=0x561395178a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fe000640 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x561395178a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fcc00640 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x561395178a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fcc00640 1 -- 172.21.15.18:0/2277743746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 msgr2=0x561395174e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fcc00640 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395174e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fcc00640 1 -- 172.21.15.18:0/2277743746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139516ed50 con 0x5613951707f0 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fd600640 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395174e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fe000640 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x561395178a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.798+0000 7f57fcc00640 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613951707f0 0x561395178540 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009740 tx=0x7f57e40bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=337e0ce7630ae989 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.799+0000 7f57e9c00640 1 -- 172.21.15.18:0/2277743746 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40b5030 con 0x5613951707f0 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.799+0000 7f57e9c00640 1 -- 172.21.15.18:0/2277743746 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e409ca90 con 0x5613951707f0 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.799+0000 7f57e9c00640 1 -- 172.21.15.18:0/2277743746 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4099100 con 0x5613951707f0 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 -- 172.21.15.18:0/2277743746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613951707f0 msgr2=0x561395178540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613951707f0 0x561395178540 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009740 tx=0x7f57e40bd040 comp rx=0 tx=0).stop 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 -- 172.21.15.18:0/2277743746 shutdown_connections 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561395179000 0x561395178a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613951707f0 0x561395178540 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 --2- 172.21.15.18:0/2277743746 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395174e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 -- 172.21.15.18:0/2277743746 >> 172.21.15.18:0/2277743746 conn(0x56139510a5d0 msgr2=0x561395174720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 -- 172.21.15.18:0/2277743746 shutdown_connections 2024-04-16T17:12:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 -- 172.21.15.18:0/2277743746 wait complete. 2024-04-16T17:12:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.800+0000 7f5802bf0740 1 -- start start 2024-04-16T17:12:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.801+0000 7f5802bf0740 1 Processor -- start 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.801+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395173250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.801+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613951707f0 0x561395173790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.801+0000 7f5802bf0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561395179000 0x5613951ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.801+0000 7f5802bf0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81afd00 con 0x56139516fa80 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.801+0000 7f5802bf0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b1170 con 0x561395179000 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.801+0000 7f5802bf0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b03d0 con 0x5613951707f0 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.801+0000 7f5802bf0740 10 client.0 _ll_register_callbacks cb 0x7f57f0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561395179000 0x5613951ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561395179000 0x5613951ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34628/0 (socket says 172.21.15.18:34628) 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 -- 172.21.15.18:0/3736561618 learned_addr learned my addr 172.21.15.18:0/3736561618 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fd600640 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395173250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fcc00640 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613951707f0 0x561395173790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 -- 172.21.15.18:0/3736561618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613951707f0 msgr2=0x561395173790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613951707f0 0x561395173790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 -- 172.21.15.18:0/3736561618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 msgr2=0x561395173250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395173250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 -- 172.21.15.18:0/3736561618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613951ee7a0 con 0x561395179000 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57fe000640 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561395179000 0x5613951ee260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f57f0012010 tx=0x7f57f0003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8124417ce562b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.802+0000 7f57dea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f57dea00640 1 -- 172.21.15.18:0/3736561618 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f002c060 con 0x561395179000 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f57dea00640 1 -- 172.21.15.18:0/3736561618 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f57f0017030 con 0x561395179000 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f5802bf0740 1 -- 172.21.15.18:0/3736561618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e409ca90 con 0x561395179000 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f57dea00640 1 -- 172.21.15.18:0/3736561618 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f57f0004790 con 0x561395179000 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f5802bf0740 10 client.27707 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f5802bf0740 1 -- 172.21.15.18:0/3736561618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613951efa30 con 0x561395179000 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f57dea00640 1 -- 172.21.15.18:0/3736561618 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e409ca90 con 0x561395179000 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f57dea00640 1 -- 172.21.15.18:0/3736561618 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f57f00276f0 con 0x561395179000 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f5802bf0740 10 client.27707 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f5802bf0740 10 client.27707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.803+0000 7f5802bf0740 20 client.27707 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.804+0000 7f5802bf0740 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5613951f01e0 0x5613951f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.804+0000 7f5802bf0740 4 client.27707 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.804+0000 7f5802bf0740 1 -- 172.21.15.18:0/3736561618 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613950dc010 con 0x5613951f01e0 2024-04-16T17:12:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.804+0000 7f57fd600640 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5613951f01e0 0x5613951f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.805+0000 7f57fd600640 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5613951f01e0 0x5613951f2600 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec31575505f781fb server_cookie=3ebec15c4016fde6 in_seq=0 out_seq=0 2024-04-16T17:12:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.805+0000 7f57dea00640 10 client.27707 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.949+0000 7f57dea00640 1 -- 172.21.15.18:0/3736561618 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613950dc010 con 0x5613951f01e0 2024-04-16T17:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.949+0000 7f57dea00640 10 client.27707 handle_command_reply: tid=0 2024-04-16T17:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.949+0000 7f5802bf0740 1 client.27707 shutdown 2024-04-16T17:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 -- 172.21.15.18:0/3736561618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561395179000 msgr2=0x5613951ee260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561395179000 0x5613951ee260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f57f0012010 tx=0x7f57f0003f20 comp rx=0 tx=0).stop 2024-04-16T17:12:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 -- 172.21.15.18:0/3736561618 shutdown_connections 2024-04-16T17:12:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5613951f01e0 0x5613951f2600 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561395179000 0x5613951ee260 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613951707f0 0x561395173790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 --2- 172.21.15.18:0/3736561618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56139516fa80 0x561395173250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 -- 172.21.15.18:0/3736561618 >> 172.21.15.18:0/3736561618 conn(0x56139510a5d0 msgr2=0x561395184110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f57dea00640 0 client.27707 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 -- 172.21.15.18:0/3736561618 shutdown_connections 2024-04-16T17:12:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 1 -- 172.21.15.18:0/3736561618 wait complete. 2024-04-16T17:12:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.950+0000 7f5802bf0740 20 client.27707 trim_cache size 0 max 16384 2024-04-16T17:12:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:12:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:12:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:12:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:12:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.952+0000 7f57fea00640 1 -- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57e00a3e00 msgr2=0x7f57e00a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.952+0000 7f57fea00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57e00a3e00 0x7f57e00a62c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005fd0 tx=0x7f57f0009500 comp rx=0 tx=0).stop 2024-04-16T17:12:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.952+0000 7f57fea00640 1 -- 172.21.15.18:0/594315718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8107150 msgr2=0x7f57f81a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.952+0000 7f57fea00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8107150 0x7f57f81a0b80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e9c0 tx=0x7f57e400ee80 comp rx=0 tx=0).stop 2024-04-16T17:12:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.953+0000 7f57fea00640 1 -- 172.21.15.18:0/594315718 shutdown_connections 2024-04-16T17:12:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.953+0000 7f57fea00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57e00a3e00 0x7f57e00a62c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.953+0000 7f57fea00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f819a800 0x7f57f819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.953+0000 7f57fea00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f8107150 0x7f57f81a0b80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.953+0000 7f57fea00640 1 --2- 172.21.15.18:0/594315718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57f8102080 0x7f57f81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.953+0000 7f57fea00640 1 -- 172.21.15.18:0/594315718 >> 172.21.15.18:0/594315718 conn(0x7f57f80fd940 msgr2=0x7f57f80ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.953+0000 7f57fea00640 1 -- 172.21.15.18:0/594315718 shutdown_connections 2024-04-16T17:12:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:30.953+0000 7f57fea00640 1 -- 172.21.15.18:0/594315718 wait complete. 2024-04-16T17:12:31.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:21 smithi018 ceph-mon[31776]: pgmap v237: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-04-16T17:12:31.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:23 smithi018 ceph-mon[31776]: pgmap v238: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-16T17:12:31.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:25 smithi018 ceph-mon[31776]: pgmap v239: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-16T17:12:31.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:27 smithi018 ceph-mon[31776]: pgmap v240: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-04-16T17:12:31.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:29 smithi018 ceph-mon[31776]: pgmap v241: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-16T17:12:31.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 299012 3.91 MB/sec execute 284 sec latency 44.923 ms 2024-04-16T17:12:31.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:31 smithi022 ceph-mon[36023]: pgmap v242: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-04-16T17:12:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:31 smithi043 ceph-mon[36901]: pgmap v242: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-04-16T17:12:32.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 300272 3.91 MB/sec execute 285 sec latency 15.854 ms 2024-04-16T17:12:33.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 300996 3.92 MB/sec execute 286 sec latency 68.032 ms 2024-04-16T17:12:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:33 smithi022 ceph-mon[36023]: pgmap v243: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-04-16T17:12:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:33 smithi043 ceph-mon[36901]: pgmap v243: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-04-16T17:12:34.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 301331 3.91 MB/sec execute 287 sec latency 59.874 ms 2024-04-16T17:12:35.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 301907 3.92 MB/sec execute 288 sec latency 37.597 ms 2024-04-16T17:12:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:35 smithi022 ceph-mon[36023]: pgmap v244: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-16T17:12:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:35 smithi043 ceph-mon[36901]: pgmap v244: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-16T17:12:36.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 302539 3.91 MB/sec execute 289 sec latency 21.315 ms 2024-04-16T17:12:37.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 303782 3.91 MB/sec execute 290 sec latency 20.395 ms 2024-04-16T17:12:37.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:37 smithi022 ceph-mon[36023]: pgmap v245: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-04-16T17:12:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:37 smithi043 ceph-mon[36901]: pgmap v245: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-04-16T17:12:38.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 304717 3.92 MB/sec execute 291 sec latency 71.695 ms 2024-04-16T17:12:39.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 305138 3.92 MB/sec execute 292 sec latency 67.227 ms 2024-04-16T17:12:39.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:39 smithi022 ceph-mon[36023]: pgmap v246: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:12:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:39 smithi043 ceph-mon[36901]: pgmap v246: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:12:40.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 305452 3.91 MB/sec execute 293 sec latency 99.022 ms 2024-04-16T17:12:41.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:12:41.235 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 -- 172.21.15.18:0/1309559369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102070 msgr2=0x7f27cc102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 --2- 172.21.15.18:0/1309559369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102070 0x7f27cc102450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009ca0 tx=0x7f27bc02c100 comp rx=0 tx=0).stop 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 -- 172.21.15.18:0/1309559369 shutdown_connections 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 --2- 172.21.15.18:0/1309559369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc107140 0x7f27cc10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 --2- 172.21.15.18:0/1309559369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27cc102a20 0x7f27cc106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 --2- 172.21.15.18:0/1309559369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102070 0x7f27cc102450 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 -- 172.21.15.18:0/1309559369 >> 172.21.15.18:0/1309559369 conn(0x7f27cc0fd950 msgr2=0x7f27cc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 -- 172.21.15.18:0/1309559369 shutdown_connections 2024-04-16T17:12:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.460+0000 7f27d0c00640 1 -- 172.21.15.18:0/1309559369 wait complete. 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.461+0000 7f27d0c00640 1 Processor -- start 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.461+0000 7f27d0c00640 1 -- start start 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27d0c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc102070 0x7f27cc198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27d0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc107140 0x7f27cc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27d0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27cc19d010 0x7f27cc19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27d0c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1121f0 con 0x7f27cc102070 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27d0c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc112070 con 0x7f27cc19d010 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27d0c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc112370 con 0x7f27cc107140 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27cb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc102070 0x7f27cc198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27cc19d010 0x7f27cc19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc107140 0x7f27cc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27cc19d010 0x7f27cc19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54622/0 (socket says 172.21.15.18:54622) 2024-04-16T17:12:41.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.462+0000 7f27cbe00640 1 -- 172.21.15.18:0/2938390918 learned_addr learned my addr 172.21.15.18:0/2938390918 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:41.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.463+0000 7f27cbe00640 1 -- 172.21.15.18:0/2938390918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc107140 msgr2=0x7f27cc198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.463+0000 7f27cbe00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc107140 0x7f27cc198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.463+0000 7f27cbe00640 1 -- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc102070 msgr2=0x7f27cc198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.463+0000 7f27cbe00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc102070 0x7f27cc198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.463+0000 7f27cbe00640 1 -- 172.21.15.18:0/2938390918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc19db00 con 0x7f27cc19d010 2024-04-16T17:12:41.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.463+0000 7f27cbe00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27cc19d010 0x7f27cc19d4c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e970 tx=0x7f27c400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.463+0000 7f27c3e00640 1 -- 172.21.15.18:0/2938390918 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c400ccc0 con 0x7f27cc19d010 2024-04-16T17:12:41.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.464+0000 7f27c3e00640 1 -- 172.21.15.18:0/2938390918 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c400ce60 con 0x7f27cc19d010 2024-04-16T17:12:41.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.464+0000 7f27d0c00640 1 -- 172.21.15.18:0/2938390918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc19ddf0 con 0x7f27cc19d010 2024-04-16T17:12:41.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.464+0000 7f27c3e00640 1 -- 172.21.15.18:0/2938390918 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4010490 con 0x7f27cc19d010 2024-04-16T17:12:41.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.464+0000 7f27d0c00640 1 -- 172.21.15.18:0/2938390918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc1a5590 con 0x7f27cc19d010 2024-04-16T17:12:41.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.468+0000 7f27c3e00640 1 -- 172.21.15.18:0/2938390918 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27c4005020 con 0x7f27cc19d010 2024-04-16T17:12:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27d4e33740 1 Processor -- start 2024-04-16T17:12:41.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27d4e33740 1 -- start start 2024-04-16T17:12:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbeb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 0x5648fcbedc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27d4e33740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5648fcb48120 con 0x5648fcbeb890 2024-04-16T17:12:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27d4e33740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5648fcaee340 con 0x5648fcbe2390 2024-04-16T17:12:41.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27d4e33740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5648fcb440e0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.469+0000 7f27cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbeb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbeb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36732/0 (socket says 172.21.15.18:36732) 2024-04-16T17:12:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27cb400640 1 -- 172.21.15.18:0/2483753487 learned_addr learned my addr 172.21.15.18:0/2483753487 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27caa00640 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 0x5648fcbedc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27cb400640 1 -- 172.21.15.18:0/2483753487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 msgr2=0x5648fcbe2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27cb400640 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27cb400640 1 -- 172.21.15.18:0/2483753487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 msgr2=0x5648fcbedc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27cb400640 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 0x5648fcbedc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27cb400640 1 -- 172.21.15.18:0/2483753487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fc9247d0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27caa00640 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.470+0000 7f27cb400640 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbeb350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f27bc02c710 tx=0x7f27bc0098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b1e3d9490418a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.471+0000 7f27bbe00640 1 -- 172.21.15.18:0/2483753487 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc043070 con 0x5648fcbe2cb0 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.471+0000 7f27bbe00640 1 -- 172.21.15.18:0/2483753487 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27bc003970 con 0x5648fcbe2cb0 2024-04-16T17:12:41.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.471+0000 7f27bbe00640 1 -- 172.21.15.18:0/2483753487 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc039040 con 0x5648fcbe2cb0 2024-04-16T17:12:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27d4e33740 1 -- 172.21.15.18:0/2483753487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 msgr2=0x5648fcbeb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbeb350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f27bc02c710 tx=0x7f27bc0098c0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27c3e00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27d4e33740 1 -- 172.21.15.18:0/2483753487 shutdown_connections 2024-04-16T17:12:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 0x5648fcbedc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbeb350 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2483753487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27d4e33740 1 -- 172.21.15.18:0/2483753487 >> 172.21.15.18:0/2483753487 conn(0x5648fcbdcac0 msgr2=0x5648fcbdcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27c3e00640 1 -- 172.21.15.18:0/2938390918 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f27c40d10a0 con 0x7f27cc19d010 2024-04-16T17:12:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27caa00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.472+0000 7f27d4e33740 1 -- 172.21.15.18:0/2483753487 shutdown_connections 2024-04-16T17:12:41.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.473+0000 7f27d4e33740 1 -- 172.21.15.18:0/2483753487 wait complete. 2024-04-16T17:12:41.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.473+0000 7f27d4e33740 1 -- start start 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.473+0000 7f27caa00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27b00a3d20 0x7f27b00a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5648fcbddc50 tx=0x7f27b4013040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.474+0000 7f27d4e33740 1 Processor -- start 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.474+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.474+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbe8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.474+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 0x5648fcbe93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.474+0000 7f27d4e33740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5648fcbee620 con 0x5648fcbeb890 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.474+0000 7f27d4e33740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5648fcb01710 con 0x5648fcbe2390 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.474+0000 7f27d4e33740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5648fcbee7a0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.474+0000 7f27d4e33740 10 client.0 _ll_register_callbacks cb 0x5648fcafcaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbe8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 0x5648fcbe93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbe8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36734/0 (socket says 172.21.15.18:36734) 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27caa00640 1 -- 172.21.15.18:0/2618207258 learned_addr learned my addr 172.21.15.18:0/2618207258 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27cb400640 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27caa00640 1 -- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 msgr2=0x5648fcbe8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27caa00640 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27caa00640 1 -- 172.21.15.18:0/2618207258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 msgr2=0x5648fcbe93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27caa00640 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 0x5648fcbe93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27caa00640 1 -- 172.21.15.18:0/2618207258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fc85a3f0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.475+0000 7f27cb400640 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.476+0000 7f27caa00640 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbe8e60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f27b4003870 tx=0x7f27b4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48914300cf47f0ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.476+0000 7f27b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.476+0000 7f27b8c00640 1 -- 172.21.15.18:0/2618207258 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b402e060 con 0x5648fcbe2cb0 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.476+0000 7f27b8c00640 1 -- 172.21.15.18:0/2618207258 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f27b4003df0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.476+0000 7f27d4e33740 1 -- 172.21.15.18:0/2618207258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27bc003970 con 0x5648fcbe2cb0 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.477+0000 7f27b8c00640 1 -- 172.21.15.18:0/2618207258 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27b4035050 con 0x5648fcbe2cb0 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.477+0000 7f27d4e33740 10 client.27625 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.477+0000 7f27d4e33740 1 -- 172.21.15.18:0/2618207258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648fc81a0f0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.477+0000 7f27b8c00640 1 -- 172.21.15.18:0/2618207258 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27bc003970 con 0x5648fcbe2cb0 2024-04-16T17:12:41.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.478+0000 7f27b8c00640 1 -- 172.21.15.18:0/2618207258 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27b401baa0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.478+0000 7f27d4e33740 10 client.27625 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.478+0000 7f27d4e33740 10 client.27625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.478+0000 7f27d4e33740 20 client.27625 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.478+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5648fcbfbc70 0x5648fcbfe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.478+0000 7f27d4e33740 4 client.27625 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:41.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.478+0000 7f27d4e33740 1 -- 172.21.15.18:0/2618207258 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648fcb55b40 con 0x5648fcbfbc70 2024-04-16T17:12:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.478+0000 7f27cb400640 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5648fcbfbc70 0x5648fcbfe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.479+0000 7f27cb400640 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5648fcbfbc70 0x5648fcbfe0e0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b08ca8f5832139c0 server_cookie=52dc54b8b30c426c in_seq=0 out_seq=0 2024-04-16T17:12:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.479+0000 7f27b8c00640 10 client.27625 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27b8c00640 1 -- 172.21.15.18:0/2618207258 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648fcb55b40 con 0x5648fcbfbc70 2024-04-16T17:12:41.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27b8c00640 10 client.27625 handle_command_reply: tid=0 2024-04-16T17:12:41.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 client.27625 shutdown 2024-04-16T17:12:41.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 -- 172.21.15.18:0/2618207258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 msgr2=0x5648fcbe8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbe8e60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f27b4003870 tx=0x7f27b4005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 -- 172.21.15.18:0/2618207258 shutdown_connections 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5648fcbfbc70 0x5648fcbfe0e0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbeb890 0x5648fcbe93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2cb0 0x5648fcbe8e60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2618207258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2390 0x5648fcbe8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27d4e33740 1 -- 172.21.15.18:0/2618207258 >> 172.21.15.18:0/2618207258 conn(0x5648fcbdcac0 msgr2=0x5648fcbde810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.480+0000 7f27b8c00640 0 client.27625 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.481+0000 7f27d4e33740 1 -- 172.21.15.18:0/2618207258 shutdown_connections 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.481+0000 7f27d4e33740 1 -- 172.21.15.18:0/2618207258 wait complete. 2024-04-16T17:12:41.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.481+0000 7f27d4e33740 20 client.27625 trim_cache size 0 max 16384 2024-04-16T17:12:41.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.505+0000 7f27d4e33740 1 Processor -- start 2024-04-16T17:12:41.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.505+0000 7f27d4e33740 1 -- start start 2024-04-16T17:12:41.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.505+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2390 0x5648fcbeb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.505+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2cb0 0x5648fcbf74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbeb890 0x5648fcbfdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27d4e33740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1afbe0 con 0x5648fcbe2390 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27d4e33740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1ad4b0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27d4e33740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1ab320 con 0x5648fcbeb890 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2cb0 0x5648fcbf74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2cb0 0x5648fcbf74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54640/0 (socket says 172.21.15.18:54640) 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27caa00640 1 -- 172.21.15.18:0/1047217232 learned_addr learned my addr 172.21.15.18:0/1047217232 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27cb400640 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2390 0x5648fcbeb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27cbe00640 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbeb890 0x5648fcbfdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27caa00640 1 -- 172.21.15.18:0/1047217232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbeb890 msgr2=0x5648fcbfdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27caa00640 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbeb890 0x5648fcbfdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27caa00640 1 -- 172.21.15.18:0/1047217232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2390 msgr2=0x5648fcbeb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27caa00640 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2390 0x5648fcbeb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.506+0000 7f27caa00640 1 -- 172.21.15.18:0/1047217232 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fcbe78d0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.507+0000 7f27caa00640 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2cb0 0x5648fcbf74e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f27b400f460 tx=0x7f27b4004000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c7f24722c683404 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.507+0000 7f27bb400640 1 -- 172.21.15.18:0/1047217232 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b403e070 con 0x5648fcbe2cb0 2024-04-16T17:12:41.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.507+0000 7f27bb400640 1 -- 172.21.15.18:0/1047217232 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b4005520 con 0x5648fcbe2cb0 2024-04-16T17:12:41.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.507+0000 7f27bb400640 1 -- 172.21.15.18:0/1047217232 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4004990 con 0x5648fcbe2cb0 2024-04-16T17:12:41.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 -- 172.21.15.18:0/1047217232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2cb0 msgr2=0x5648fcbf74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2cb0 0x5648fcbf74e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f27b400f460 tx=0x7f27b4004000 comp rx=0 tx=0).stop 2024-04-16T17:12:41.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 -- 172.21.15.18:0/1047217232 shutdown_connections 2024-04-16T17:12:41.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbeb890 0x5648fcbfdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbe2cb0 0x5648fcbf74e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 --2- 172.21.15.18:0/1047217232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2390 0x5648fcbeb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 -- 172.21.15.18:0/1047217232 >> 172.21.15.18:0/1047217232 conn(0x5648fcbdcac0 msgr2=0x5648fcbdb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 -- 172.21.15.18:0/1047217232 shutdown_connections 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.508+0000 7f27d4e33740 1 -- 172.21.15.18:0/1047217232 wait complete. 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 1 -- start start 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 1 Processor -- start 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2390 0x5648fcbef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2cb0 0x5648fcbefa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbeb890 0x5648fcbeff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1ab970 con 0x5648fcbe2cb0 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1acde0 con 0x5648fcbeb890 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1ac040 con 0x5648fcbe2390 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.509+0000 7f27d4e33740 10 client.0 _ll_register_callbacks cb 0x7f27b40044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2390 0x5648fcbef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27caa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2cb0 0x5648fcbefa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2390 0x5648fcbef4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36764/0 (socket says 172.21.15.18:36764) 2024-04-16T17:12:41.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27cb400640 1 -- 172.21.15.18:0/2562169099 learned_addr learned my addr 172.21.15.18:0/2562169099 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:41.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27cbe00640 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbeb890 0x5648fcbeff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27caa00640 1 -- 172.21.15.18:0/2562169099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2390 msgr2=0x5648fcbef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27caa00640 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2390 0x5648fcbef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27caa00640 1 -- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbeb890 msgr2=0x5648fcbeff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27caa00640 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbeb890 0x5648fcbeff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27caa00640 1 -- 172.21.15.18:0/2562169099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fcbfe450 con 0x5648fcbe2cb0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27cbe00640 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbeb890 0x5648fcbeff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.510+0000 7f27cb400640 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2390 0x5648fcbef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.511+0000 7f27caa00640 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2cb0 0x5648fcbefa10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x5648fcbe99b0 tx=0x7f27b4004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d72b47f75c16a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.511+0000 7f27b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.511+0000 7f27b8c00640 1 -- 172.21.15.18:0/2562169099 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b403e050 con 0x5648fcbe2cb0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.511+0000 7f27d4e33740 1 -- 172.21.15.18:0/2562169099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b4005520 con 0x5648fcbe2cb0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.511+0000 7f27b8c00640 1 -- 172.21.15.18:0/2562169099 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f27b401b3c0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.511+0000 7f27b8c00640 1 -- 172.21.15.18:0/2562169099 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27b402e040 con 0x5648fcbe2cb0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.511+0000 7f27d4e33740 10 client.17460 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.511+0000 7f27d4e33740 1 -- 172.21.15.18:0/2562169099 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648fcc62450 con 0x5648fcbe2cb0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27b8c00640 1 -- 172.21.15.18:0/2562169099 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b4005520 con 0x5648fcbe2cb0 2024-04-16T17:12:41.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27b8c00640 1 -- 172.21.15.18:0/2562169099 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27b400f6c0 con 0x5648fcbe2cb0 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27d4e33740 10 client.17460 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27d4e33740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27d4e33740 20 client.17460 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5648fcc62a00 0x5648fcc64df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27d4e33740 4 client.17460 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27d4e33740 1 -- 172.21.15.18:0/2562169099 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648fcb4f9b0 con 0x5648fcc62a00 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.512+0000 7f27cb400640 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5648fcc62a00 0x5648fcc64df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.513+0000 7f27cb400640 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5648fcc62a00 0x5648fcc64df0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7598758a82399a67 server_cookie=6b715d65119164a6 in_seq=0 out_seq=0 2024-04-16T17:12:41.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.513+0000 7f27b8c00640 10 client.17460 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:41.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.514+0000 7f27b8c00640 1 -- 172.21.15.18:0/2562169099 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5648fcb4f9b0 con 0x5648fcc62a00 2024-04-16T17:12:41.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.514+0000 7f27b8c00640 10 client.17460 handle_command_reply: tid=0 2024-04-16T17:12:41.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.514+0000 7f27d4e33740 1 client.17460 shutdown 2024-04-16T17:12:41.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.514+0000 7f27d4e33740 1 -- 172.21.15.18:0/2562169099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2cb0 msgr2=0x5648fcbefa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.514+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2cb0 0x5648fcbefa10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x5648fcbe99b0 tx=0x7f27b4004070 comp rx=0 tx=0).stop 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.514+0000 7f27d4e33740 1 -- 172.21.15.18:0/2562169099 shutdown_connections 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.514+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5648fcc62a00 0x5648fcc64df0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.514+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5648fcbeb890 0x5648fcbeff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.515+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5648fcbe2cb0 0x5648fcbefa10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.515+0000 7f27d4e33740 1 --2- 172.21.15.18:0/2562169099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648fcbe2390 0x5648fcbef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.515+0000 7f27d4e33740 1 -- 172.21.15.18:0/2562169099 >> 172.21.15.18:0/2562169099 conn(0x5648fcbdcac0 msgr2=0x5648fcbe90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.515+0000 7f27b8c00640 0 client.17460 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.515+0000 7f27d4e33740 1 -- 172.21.15.18:0/2562169099 shutdown_connections 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.515+0000 7f27d4e33740 1 -- 172.21.15.18:0/2562169099 wait complete. 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.515+0000 7f27d4e33740 20 client.17460 trim_cache size 0 max 16384 2024-04-16T17:12:41.517 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:12:41.518 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:12:41.518 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:12:41.518 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:12:41.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 -- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27b00a3d20 msgr2=0x7f27b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27b00a3d20 0x7f27b00a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5648fcbddc50 tx=0x7f27b4013040 comp rx=0 tx=0).stop 2024-04-16T17:12:41.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 -- 172.21.15.18:0/2938390918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27cc19d010 msgr2=0x7f27cc19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27cc19d010 0x7f27cc19d4c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e970 tx=0x7f27c400ed70 comp rx=0 tx=0).stop 2024-04-16T17:12:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 -- 172.21.15.18:0/2938390918 shutdown_connections 2024-04-16T17:12:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27cc19d010 0x7f27cc19d4c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc107140 0x7f27cc198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 --2- 172.21.15.18:0/2938390918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc102070 0x7f27cc198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 -- 172.21.15.18:0/2938390918 >> 172.21.15.18:0/2938390918 conn(0x7f27cc0fd950 msgr2=0x7f27cc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 -- 172.21.15.18:0/2938390918 shutdown_connections 2024-04-16T17:12:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.516+0000 7f27d0c00640 1 -- 172.21.15.18:0/2938390918 wait complete. 2024-04-16T17:12:41.534 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:12:41.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:12:41.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 306207 3.91 MB/sec execute 294 sec latency 16.094 ms 2024-04-16T17:12:41.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.736+0000 7f28ba200640 1 -- 172.21.15.18:0/62700187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4074d50 msgr2=0x7f28b410e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.736+0000 7f28ba200640 1 --2- 172.21.15.18:0/62700187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4074d50 0x7f28b410e1c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009900 tx=0x7f28a002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.737+0000 7f28ba200640 1 -- 172.21.15.18:0/62700187 shutdown_connections 2024-04-16T17:12:41.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.737+0000 7f28ba200640 1 --2- 172.21.15.18:0/62700187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28b410e700 0x7f28b4110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.737+0000 7f28ba200640 1 --2- 172.21.15.18:0/62700187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4074d50 0x7f28b410e1c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.737+0000 7f28ba200640 1 --2- 172.21.15.18:0/62700187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28b4074430 0x7f28b4074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.737+0000 7f28ba200640 1 -- 172.21.15.18:0/62700187 >> 172.21.15.18:0/62700187 conn(0x7f28b40fd9f0 msgr2=0x7f28b40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.737+0000 7f28ba200640 1 -- 172.21.15.18:0/62700187 shutdown_connections 2024-04-16T17:12:41.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.737+0000 7f28ba200640 1 -- 172.21.15.18:0/62700187 wait complete. 2024-04-16T17:12:41.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.738+0000 7f28ba200640 1 Processor -- start 2024-04-16T17:12:41.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.738+0000 7f28ba200640 1 -- start start 2024-04-16T17:12:41.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.738+0000 7f28ba200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28b4074430 0x7f28b41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.738+0000 7f28ba200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28b4074d50 0x7f28b41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.738+0000 7f28ba200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b410e700 0x7f28b419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.738+0000 7f28ba200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41144c0 con 0x7f28b4074d50 2024-04-16T17:12:41.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.738+0000 7f28ba200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4114340 con 0x7f28b410e700 2024-04-16T17:12:41.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.738+0000 7f28ba200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4114640 con 0x7f28b4074430 2024-04-16T17:12:41.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28b4074d50 0x7f28b41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b410e700 0x7f28b419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28b4074d50 0x7f28b41a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58198/0 (socket says 172.21.15.18:58198) 2024-04-16T17:12:41.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b3e00640 1 -- 172.21.15.18:0/2671938910 learned_addr learned my addr 172.21.15.18:0/2671938910 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:41.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b8e00640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28b4074430 0x7f28b41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b9800640 1 -- 172.21.15.18:0/2671938910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28b4074430 msgr2=0x7f28b41a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b9800640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28b4074430 0x7f28b41a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b9800640 1 -- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28b4074d50 msgr2=0x7f28b41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b9800640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28b4074d50 0x7f28b41a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.739+0000 7f28b9800640 1 -- 172.21.15.18:0/2671938910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b419af90 con 0x7f28b410e700 2024-04-16T17:12:41.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.740+0000 7f28b9800640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b410e700 0x7f28b419a7c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009fd0 tx=0x7f28ac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.740+0000 7f28b1600640 1 -- 172.21.15.18:0/2671938910 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00ce20 con 0x7f28b410e700 2024-04-16T17:12:41.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.740+0000 7f28b1600640 1 -- 172.21.15.18:0/2671938910 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28ac005020 con 0x7f28b410e700 2024-04-16T17:12:41.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.740+0000 7f28b1600640 1 -- 172.21.15.18:0/2671938910 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac010720 con 0x7f28b410e700 2024-04-16T17:12:41.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.740+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b419b280 con 0x7f28b410e700 2024-04-16T17:12:41.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.740+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a74a0 con 0x7f28b410e700 2024-04-16T17:12:41.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.741+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b4075ae0 con 0x7f28b410e700 2024-04-16T17:12:41.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.743+0000 7f28b1600640 1 -- 172.21.15.18:0/2671938910 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f28ac005540 con 0x7f28b410e700 2024-04-16T17:12:41.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.745+0000 7f28b1600640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f289c0a3d90 0x7f289c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.745+0000 7f28b1600640 1 -- 172.21.15.18:0/2671938910 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f28ac0d1240 con 0x7f28b410e700 2024-04-16T17:12:41.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.745+0000 7f28b8e00640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f289c0a3d90 0x7f289c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.746+0000 7f28b8e00640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f289c0a3d90 0x7f289c0a6250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f28a80095f0 tx=0x7f28a8005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:41.749+0000 7f28b1600640 1 -- 172.21.15.18:0/2671938910 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28ac09d470 con 0x7f28b410e700 2024-04-16T17:12:41.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:41 smithi022 ceph-mon[36023]: pgmap v247: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-16T17:12:42.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:41 smithi043 ceph-mon[36901]: pgmap v247: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-16T17:12:42.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.017+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b402d620 con 0x7f28b410e700 2024-04-16T17:12:42.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.018+0000 7f28b1600640 1 -- 172.21.15.18:0/2671938910 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11851 (secure 0 0 0) 0x7f28ac09cd70 con 0x7f28b410e700 2024-04-16T17:12:42.019 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:12:42.019 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:11:38.382274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27445":{"gid":27445,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/678063310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":678063310},{"type":"v1","addr":"172.21.15.22:6837","nonce":678063310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:12:42.020 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 27 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f289c0a3d90 msgr2=0x7f289c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f289c0a3d90 0x7f289c0a6250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f28a80095f0 tx=0x7f28a8005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b410e700 msgr2=0x7f28b419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b410e700 0x7f28b419a7c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009fd0 tx=0x7f28ac00ef10 comp rx=0 tx=0).stop 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 shutdown_connections 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f289c0a3d90 0x7f289c0a6250 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b410e700 0x7f28b419a7c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28b4074d50 0x7f28b41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 --2- 172.21.15.18:0/2671938910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28b4074430 0x7f28b41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.022+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 >> 172.21.15.18:0/2671938910 conn(0x7f28b40fd9f0 msgr2=0x7f28b40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.023+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 shutdown_connections 2024-04-16T17:12:42.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.023+0000 7f28ba200640 1 -- 172.21.15.18:0/2671938910 wait complete. 2024-04-16T17:12:42.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:12:42.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 -- 172.21.15.18:0/3293708687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8102070 msgr2=0x7fa3e8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/3293708687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8102070 0x7fa3e8102450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009900 tx=0x7fa3d802bc50 comp rx=0 tx=0).stop 2024-04-16T17:12:42.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 -- 172.21.15.18:0/3293708687 shutdown_connections 2024-04-16T17:12:42.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/3293708687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3e8107140 0x7fa3e810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/3293708687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3e8102a20 0x7fa3e8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/3293708687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8102070 0x7fa3e8102450 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 -- 172.21.15.18:0/3293708687 >> 172.21.15.18:0/3293708687 conn(0x7fa3e80fd950 msgr2=0x7fa3e80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 -- 172.21.15.18:0/3293708687 shutdown_connections 2024-04-16T17:12:42.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.260+0000 7fa3ecc00640 1 -- 172.21.15.18:0/3293708687 wait complete. 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.261+0000 7fa3ecc00640 1 Processor -- start 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.261+0000 7fa3ecc00640 1 -- start start 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3ecc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3e8102070 0x7fa3e8074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3ecc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8107140 0x7fa3e8074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3ecc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3e8074d80 0x7fa3e81a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3ecc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e8112220 con 0x7fa3e8102070 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3ecc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81120a0 con 0x7fa3e8107140 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3ecc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81123a0 con 0x7fa3e8074d80 2024-04-16T17:12:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3e8102070 0x7fa3e8074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8107140 0x7fa3e8074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3e8102070 0x7fa3e8074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58224/0 (socket says 172.21.15.18:58224) 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3e7400640 1 -- 172.21.15.18:0/2841410812 learned_addr learned my addr 172.21.15.18:0/2841410812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8107140 0x7fa3e8074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54688/0 (socket says 172.21.15.18:54688) 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.262+0000 7fa3e7e00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3e8074d80 0x7fa3e81a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3e7400640 1 -- 172.21.15.18:0/2841410812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3e8074d80 msgr2=0x7fa3e81a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3e7400640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3e8074d80 0x7fa3e81a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3e7400640 1 -- 172.21.15.18:0/2841410812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8107140 msgr2=0x7fa3e8074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3e7400640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8107140 0x7fa3e8074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3e7400640 1 -- 172.21.15.18:0/2841410812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e81a70b0 con 0x7fa3e8102070 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3e7400640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3e8102070 0x7fa3e8074300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005ab0 tx=0x7fa3d8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3dfe00640 1 -- 172.21.15.18:0/2841410812 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8041070 con 0x7fa3e8102070 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3dfe00640 1 -- 172.21.15.18:0/2841410812 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d802ccb0 con 0x7fa3e8102070 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3ecc00640 1 -- 172.21.15.18:0/2841410812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e81a7340 con 0x7fa3e8102070 2024-04-16T17:12:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.263+0000 7fa3dfe00640 1 -- 172.21.15.18:0/2841410812 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8003df0 con 0x7fa3e8102070 2024-04-16T17:12:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.264+0000 7fa3ecc00640 1 -- 172.21.15.18:0/2841410812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e81a78e0 con 0x7fa3e8102070 2024-04-16T17:12:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.267+0000 7fa3dfe00640 1 -- 172.21.15.18:0/2841410812 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa3d8044070 con 0x7fa3e8102070 2024-04-16T17:12:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.267+0000 7fa3f0e51740 1 Processor -- start 2024-04-16T17:12:42.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.268+0000 7fa3f0e51740 1 -- start start 2024-04-16T17:12:42.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.268+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812c8f10 0x5625812c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.268+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c9830 0x5625812d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.268+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812d2410 0x5625812d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.268+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56258122e020 con 0x5625812c8f10 2024-04-16T17:12:42.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.268+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625811d4460 con 0x5625812d2410 2024-04-16T17:12:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.268+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562581229fe0 con 0x5625812c9830 2024-04-16T17:12:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.268+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812c8f10 0x5625812c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812d2410 0x5625812d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812d2410 0x5625812d4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54704/0 (socket says 172.21.15.18:54704) 2024-04-16T17:12:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e6a00640 1 -- 172.21.15.18:0/1082310044 learned_addr learned my addr 172.21.15.18:0/1082310044 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e7400640 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c9830 0x5625812d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e6a00640 1 -- 172.21.15.18:0/1082310044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c9830 msgr2=0x5625812d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c9830 0x5625812d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e6a00640 1 -- 172.21.15.18:0/1082310044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812c8f10 msgr2=0x5625812c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812c8f10 0x5625812c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e6a00640 1 -- 172.21.15.18:0/1082310044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625811dc040 con 0x5625812d2410 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.269+0000 7fa3e7e00640 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812c8f10 0x5625812c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.270+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812d2410 0x5625812d4800 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009c60 tx=0x7fa3d4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdb99fb5748f3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.270+0000 7fa3dfe00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3cc0a3da0 0x7fa3cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.270+0000 7fa3d3e00640 1 -- 172.21.15.18:0/1082310044 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4004230 con 0x5625812d2410 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.270+0000 7fa3dfe00640 1 -- 172.21.15.18:0/2841410812 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fa3d80f24c0 con 0x7fa3e8102070 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.270+0000 7fa3d3e00640 1 -- 172.21.15.18:0/1082310044 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d40043d0 con 0x5625812d2410 2024-04-16T17:12:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.270+0000 7fa3d3e00640 1 -- 172.21.15.18:0/1082310044 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4004dc0 con 0x5625812d2410 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.270+0000 7fa3e7e00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3cc0a3da0 0x7fa3cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 -- 172.21.15.18:0/1082310044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812d2410 msgr2=0x5625812d4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812d2410 0x5625812d4800 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009c60 tx=0x7fa3d4011810 comp rx=0 tx=0).stop 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 -- 172.21.15.18:0/1082310044 shutdown_connections 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812d2410 0x5625812d4800 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c9830 0x5625812d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/1082310044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812c8f10 0x5625812c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 -- 172.21.15.18:0/1082310044 >> 172.21.15.18:0/1082310044 conn(0x5625812c3640 msgr2=0x5625812c3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3e7e00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3cc0a3da0 0x7fa3cc0a6260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005fd0 tx=0x7fa3e0009600 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 -- 172.21.15.18:0/1082310044 shutdown_connections 2024-04-16T17:12:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.271+0000 7fa3f0e51740 1 -- 172.21.15.18:0/1082310044 wait complete. 2024-04-16T17:12:42.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.272+0000 7fa3f0e51740 1 -- start start 2024-04-16T17:12:42.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.272+0000 7fa3f0e51740 1 Processor -- start 2024-04-16T17:12:42.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.272+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.272+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.272+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.272+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5625812d5200 con 0x5625812d2410 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625811e7830 con 0x5625812c9830 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5625812d5380 con 0x5625812c8f10 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3f0e51740 10 client.0 _ll_register_callbacks cb 0x5625811ef150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36832/0 (socket says 172.21.15.18:36832) 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 -- 172.21.15.18:0/2520747992 learned_addr learned my addr 172.21.15.18:0/2520747992 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 -- 172.21.15.18:0/2520747992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 msgr2=0x5625812cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 -- 172.21.15.18:0/2520747992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 msgr2=0x5625812cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 -- 172.21.15.18:0/2520747992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625811eb920 con 0x5625812c8f10 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7e00640 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:42.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.273+0000 7fa3e7400640 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812cf3f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8000c00 tx=0x7fa3d8004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1645cb4f2736cbfb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.274+0000 7fa3d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.274+0000 7fa3d0c00640 1 -- 172.21.15.18:0/2520747992 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d804b070 con 0x5625812c8f10 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.274+0000 7fa3d0c00640 1 -- 172.21.15.18:0/2520747992 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fa3d80c68e0 con 0x5625812c8f10 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.274+0000 7fa3f0e51740 1 -- 172.21.15.18:0/2520747992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d40043d0 con 0x5625812c8f10 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.274+0000 7fa3d0c00640 1 -- 172.21.15.18:0/2520747992 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3d80bdcf0 con 0x5625812c8f10 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.274+0000 7fa3f0e51740 10 client.27649 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.274+0000 7fa3f0e51740 1 -- 172.21.15.18:0/2520747992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5625811b97f0 con 0x5625812c8f10 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.274+0000 7fa3d0c00640 1 -- 172.21.15.18:0/2520747992 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d40043d0 con 0x5625812c8f10 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.275+0000 7fa3d0c00640 1 -- 172.21.15.18:0/2520747992 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d80c2130 con 0x5625812c8f10 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.275+0000 7fa3f0e51740 10 client.27649 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.275+0000 7fa3f0e51740 10 client.27649 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.275+0000 7fa3f0e51740 20 client.27649 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.275+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5625812e2630 0x5625812e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.275+0000 7fa3f0e51740 4 client.27649 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:42.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.275+0000 7fa3f0e51740 1 -- 172.21.15.18:0/2520747992 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56258123ba40 con 0x5625812e2630 2024-04-16T17:12:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.275+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5625812e2630 0x5625812e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.276+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5625812e2630 0x5625812e4a80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c63320d2f496d1c9 server_cookie=5c085fa1e35cefd4 in_seq=0 out_seq=0 2024-04-16T17:12:42.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.276+0000 7fa3d0c00640 10 client.27649 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.276+0000 7fa3d0c00640 1 -- 172.21.15.18:0/2520747992 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56258123ba40 con 0x5625812e2630 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.276+0000 7fa3d0c00640 10 client.27649 handle_command_reply: tid=0 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 client.27649 shutdown 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 -- 172.21.15.18:0/2520747992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 msgr2=0x5625812cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812cf3f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8000c00 tx=0x7fa3d8004530 comp rx=0 tx=0).stop 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 -- 172.21.15.18:0/2520747992 shutdown_connections 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5625812e2630 0x5625812e4a80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/2520747992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812cf3f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 -- 172.21.15.18:0/2520747992 >> 172.21.15.18:0/2520747992 conn(0x5625812c3640 msgr2=0x5625812c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3d0c00640 0 client.27649 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:12:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 -- 172.21.15.18:0/2520747992 shutdown_connections 2024-04-16T17:12:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 1 -- 172.21.15.18:0/2520747992 wait complete. 2024-04-16T17:12:42.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.277+0000 7fa3f0e51740 20 client.27649 trim_cache size 0 max 16384 2024-04-16T17:12:42.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.306+0000 7fa3f0e51740 1 Processor -- start 2024-04-16T17:12:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.307+0000 7fa3f0e51740 1 -- start start 2024-04-16T17:12:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.307+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.307+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.307+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812c2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.307+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b3f40 con 0x5625812d2410 2024-04-16T17:12:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.307+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b1810 con 0x5625812c9830 2024-04-16T17:12:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.307+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81af680 con 0x5625812c8f10 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.307+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812c2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58250/0 (socket says 172.21.15.18:58250) 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7e00640 1 -- 172.21.15.18:0/860769591 learned_addr learned my addr 172.21.15.18:0/860769591 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7400640 1 -- 172.21.15.18:0/860769591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 msgr2=0x5625812de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7400640 1 --2- 172.21.15.18:0/860769591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812de0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7400640 1 -- 172.21.15.18:0/860769591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 msgr2=0x5625812c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7400640 1 --2- 172.21.15.18:0/860769591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7400640 1 -- 172.21.15.18:0/860769591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562580f403f0 con 0x5625812c8f10 2024-04-16T17:12:42.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/860769591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:42.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.308+0000 7fa3e7400640 1 --2- 172.21.15.18:0/860769591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812d1b20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80bab80 tx=0x7fa3d8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b14488fb501143e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3d3400640 1 -- 172.21.15.18:0/860769591 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d804b050 con 0x5625812c8f10 2024-04-16T17:12:42.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3d3400640 1 -- 172.21.15.18:0/860769591 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8004c80 con 0x5625812c8f10 2024-04-16T17:12:42.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3d3400640 1 -- 172.21.15.18:0/860769591 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80efc30 con 0x5625812c8f10 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3f0e51740 1 -- 172.21.15.18:0/860769591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 msgr2=0x5625812d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/860769591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812d1b20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80bab80 tx=0x7fa3d8009260 comp rx=0 tx=0).stop 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3f0e51740 1 -- 172.21.15.18:0/860769591 shutdown_connections 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/860769591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/860769591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/860769591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812d1b20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3f0e51740 1 -- 172.21.15.18:0/860769591 >> 172.21.15.18:0/860769591 conn(0x5625812c3640 msgr2=0x5625812c1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.309+0000 7fa3f0e51740 1 -- 172.21.15.18:0/860769591 shutdown_connections 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.310+0000 7fa3f0e51740 1 -- 172.21.15.18:0/860769591 wait complete. 2024-04-16T17:12:42.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.310+0000 7fa3f0e51740 1 -- start start 2024-04-16T17:12:42.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.310+0000 7fa3f0e51740 1 Processor -- start 2024-04-16T17:12:42.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3f0e51740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81afcd0 con 0x5625812d2410 2024-04-16T17:12:42.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b1140 con 0x5625812c9830 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3f0e51740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b03a0 con 0x5625812c8f10 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3f0e51740 10 client.0 _ll_register_callbacks cb 0x7fa3d8005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812da7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36848/0 (socket says 172.21.15.18:36848) 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54740/0 (socket says 172.21.15.18:54740) 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.311+0000 7fa3e7400640 1 -- 172.21.15.18:0/3456611247 learned_addr learned my addr 172.21.15.18:0/3456611247 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3e7e00640 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3e7400640 1 -- 172.21.15.18:0/3456611247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 msgr2=0x5625812cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3e7400640 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3e7400640 1 -- 172.21.15.18:0/3456611247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 msgr2=0x5625812cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3e7400640 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3e7400640 1 -- 172.21.15.18:0/3456611247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258100a7d0 con 0x5625812c8f10 2024-04-16T17:12:42.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3e7e00640 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3e7400640 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812da7f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80038d0 tx=0x7fa3d80be040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5100c1b6cd848f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.312+0000 7fa3d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3d0c00640 1 -- 172.21.15.18:0/3456611247 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d804b050 con 0x5625812c8f10 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3d0c00640 1 -- 172.21.15.18:0/3456611247 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fa3d80d2ba0 con 0x5625812c8f10 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3f0e51740 1 -- 172.21.15.18:0/3456611247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d8004c80 con 0x5625812c8f10 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3d0c00640 1 -- 172.21.15.18:0/3456611247 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3d80035f0 con 0x5625812c8f10 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3f0e51740 10 client.27655 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3f0e51740 1 -- 172.21.15.18:0/3456611247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56258100a7d0 con 0x5625812c8f10 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3d0c00640 1 -- 172.21.15.18:0/3456611247 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8004c80 con 0x5625812c8f10 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3d0c00640 1 -- 172.21.15.18:0/3456611247 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d80bd070 con 0x5625812c8f10 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3f0e51740 10 client.27655 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3f0e51740 10 client.27655 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3f0e51740 20 client.27655 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.313+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x562581348ad0 0x56258134aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.314+0000 7fa3f0e51740 4 client.27655 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:42.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.314+0000 7fa3f0e51740 1 -- 172.21.15.18:0/3456611247 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625812e4e10 con 0x562581348ad0 2024-04-16T17:12:42.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.314+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x562581348ad0 0x56258134aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.314+0000 7fa3e6a00640 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x562581348ad0 0x56258134aec0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=406e4fe292c547a9 server_cookie=72199ff418449ad in_seq=0 out_seq=0 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.314+0000 7fa3d0c00640 10 client.27655 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3d0c00640 1 -- 172.21.15.18:0/3456611247 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625812e4e10 con 0x562581348ad0 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3d0c00640 10 client.27655 handle_command_reply: tid=0 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 client.27655 shutdown 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 -- 172.21.15.18:0/3456611247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 msgr2=0x5625812da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812da7f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80038d0 tx=0x7fa3d80be040 comp rx=0 tx=0).stop 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 -- 172.21.15.18:0/3456611247 shutdown_connections 2024-04-16T17:12:42.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x562581348ad0 0x56258134aec0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5625812d2410 0x5625812cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625812c9830 0x5625812cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3d0c00640 0 client.27655 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 --2- 172.21.15.18:0/3456611247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625812c8f10 0x5625812da7f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 -- 172.21.15.18:0/3456611247 >> 172.21.15.18:0/3456611247 conn(0x5625812c3640 msgr2=0x5625812c2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 -- 172.21.15.18:0/3456611247 shutdown_connections 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.315+0000 7fa3f0e51740 1 -- 172.21.15.18:0/3456611247 wait complete. 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.316+0000 7fa3f0e51740 20 client.27655 trim_cache size 0 max 16384 2024-04-16T17:12:42.319 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:12:42.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.316+0000 7fa3ecc00640 1 -- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3cc0a3da0 msgr2=0x7fa3cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.316+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3cc0a3da0 0x7fa3cc0a6260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005fd0 tx=0x7fa3e0009600 comp rx=0 tx=0).stop 2024-04-16T17:12:42.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.316+0000 7fa3ecc00640 1 -- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3e8102070 msgr2=0x7fa3e8074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.316+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3e8102070 0x7fa3e8074300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005ab0 tx=0x7fa3d8004340 comp rx=0 tx=0).stop 2024-04-16T17:12:42.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.317+0000 7fa3ecc00640 1 -- 172.21.15.18:0/2841410812 shutdown_connections 2024-04-16T17:12:42.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.317+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3cc0a3da0 0x7fa3cc0a6260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.317+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3e8074d80 0x7fa3e81a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.317+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3e8107140 0x7fa3e8074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.317+0000 7fa3ecc00640 1 --2- 172.21.15.18:0/2841410812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3e8102070 0x7fa3e8074300 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.317+0000 7fa3ecc00640 1 -- 172.21.15.18:0/2841410812 >> 172.21.15.18:0/2841410812 conn(0x7fa3e80fd950 msgr2=0x7fa3e80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.317+0000 7fa3ecc00640 1 -- 172.21.15.18:0/2841410812 shutdown_connections 2024-04-16T17:12:42.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.317+0000 7fa3ecc00640 1 -- 172.21.15.18:0/2841410812 wait complete. 2024-04-16T17:12:42.340 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:12:42.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 -- 172.21.15.18:0/1479092052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248102070 msgr2=0x7fa248102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 --2- 172.21.15.18:0/1479092052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248102070 0x7fa248102450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa238009900 tx=0x7fa23802bc50 comp rx=0 tx=0).stop 2024-04-16T17:12:42.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 -- 172.21.15.18:0/1479092052 shutdown_connections 2024-04-16T17:12:42.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 --2- 172.21.15.18:0/1479092052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa248107140 0x7fa24810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 --2- 172.21.15.18:0/1479092052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa248102a20 0x7fa248106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 --2- 172.21.15.18:0/1479092052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248102070 0x7fa248102450 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 -- 172.21.15.18:0/1479092052 >> 172.21.15.18:0/1479092052 conn(0x7fa2480fd950 msgr2=0x7fa2480ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 -- 172.21.15.18:0/1479092052 shutdown_connections 2024-04-16T17:12:42.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.546+0000 7fa24da00640 1 -- 172.21.15.18:0/1479092052 wait complete. 2024-04-16T17:12:42.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.547+0000 7fa24da00640 1 Processor -- start 2024-04-16T17:12:42.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.547+0000 7fa24da00640 1 -- start start 2024-04-16T17:12:42.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.547+0000 7fa24da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa248102070 0x7fa248198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.547+0000 7fa24da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248107140 0x7fa248198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.547+0000 7fa24da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24819cfc0 0x7fa24819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.547+0000 7fa24da00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa248112220 con 0x7fa248102070 2024-04-16T17:12:42.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.547+0000 7fa24da00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481120a0 con 0x7fa248107140 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.547+0000 7fa24da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481123a0 con 0x7fa24819cfc0 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248107140 0x7fa248198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa248102070 0x7fa248198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248107140 0x7fa248198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54762/0 (socket says 172.21.15.18:54762) 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 -- 172.21.15.18:0/1995223476 learned_addr learned my addr 172.21.15.18:0/1995223476 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa24d000640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24819cfc0 0x7fa24819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 -- 172.21.15.18:0/1995223476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24819cfc0 msgr2=0x7fa24819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24819cfc0 0x7fa24819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 -- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa248102070 msgr2=0x7fa248198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa248102070 0x7fa248198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 -- 172.21.15.18:0/1995223476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24819dab0 con 0x7fa248107140 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247e00640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa248102070 0x7fa248198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:42.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.548+0000 7fa247400640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248107140 0x7fa248198c40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa23400e9a0 tx=0x7fa23400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.549+0000 7fa244c00640 1 -- 172.21.15.18:0/1995223476 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23400cd80 con 0x7fa248107140 2024-04-16T17:12:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.549+0000 7fa24da00640 1 -- 172.21.15.18:0/1995223476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24819dda0 con 0x7fa248107140 2024-04-16T17:12:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.549+0000 7fa244c00640 1 -- 172.21.15.18:0/1995223476 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2340035f0 con 0x7fa248107140 2024-04-16T17:12:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.549+0000 7fa244c00640 1 -- 172.21.15.18:0/1995223476 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234010670 con 0x7fa248107140 2024-04-16T17:12:42.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.549+0000 7fa24da00640 1 -- 172.21.15.18:0/1995223476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481a5590 con 0x7fa248107140 2024-04-16T17:12:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.552+0000 7fa251ddc740 1 Processor -- start 2024-04-16T17:12:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.552+0000 7fa251ddc740 1 -- start start 2024-04-16T17:12:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.552+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87ed20 0x5589af87f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.552+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af88c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.552+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af88c820 0x5589af88ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.552+0000 7fa251ddc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5589af7e4020 con 0x5589af88c820 2024-04-16T17:12:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.552+0000 7fa251ddc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5589af78a460 con 0x5589af87f6d0 2024-04-16T17:12:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.552+0000 7fa251ddc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589af7dffe0 con 0x5589af87ed20 2024-04-16T17:12:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa247e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af88c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa247e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af88c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54772/0 (socket says 172.21.15.18:54772) 2024-04-16T17:12:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa247e00640 1 -- 172.21.15.18:0/3005988594 learned_addr learned my addr 172.21.15.18:0/3005988594 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa24d000640 1 --2- 172.21.15.18:0/3005988594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87ed20 0x5589af87f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa247e00640 1 -- 172.21.15.18:0/3005988594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87ed20 msgr2=0x5589af87f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa247e00640 1 --2- 172.21.15.18:0/3005988594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87ed20 0x5589af87f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa247e00640 1 -- 172.21.15.18:0/3005988594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af88c820 msgr2=0x5589af88ec10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa247e00640 1 --2- 172.21.15.18:0/3005988594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af88c820 0x5589af88ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.553+0000 7fa247e00640 1 -- 172.21.15.18:0/3005988594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589af792040 con 0x5589af87f6d0 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.554+0000 7fa247e00640 1 --2- 172.21.15.18:0/3005988594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af88c2e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa238035000 tx=0x7fa238009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abac02a32514e966 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.554+0000 7fa244c00640 1 -- 172.21.15.18:0/1995223476 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa2340037a0 con 0x7fa248107140 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.554+0000 7fa23d600640 1 -- 172.21.15.18:0/3005988594 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238004a70 con 0x5589af87f6d0 2024-04-16T17:12:42.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.554+0000 7fa23d600640 1 -- 172.21.15.18:0/3005988594 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa238004c10 con 0x5589af87f6d0 2024-04-16T17:12:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.554+0000 7fa23d600640 1 -- 172.21.15.18:0/3005988594 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238005360 con 0x5589af87f6d0 2024-04-16T17:12:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.555+0000 7fa251ddc740 1 -- 172.21.15.18:0/3005988594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 msgr2=0x5589af88c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.555+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3005988594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af88c2e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa238035000 tx=0x7fa238009670 comp rx=0 tx=0).stop 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.555+0000 7fa251ddc740 1 -- 172.21.15.18:0/3005988594 shutdown_connections 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.555+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3005988594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af88c820 0x5589af88ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.555+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3005988594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af88c2e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.555+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3005988594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87ed20 0x5589af87f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.555+0000 7fa251ddc740 1 -- 172.21.15.18:0/3005988594 >> 172.21.15.18:0/3005988594 conn(0x5589af879640 msgr2=0x5589af879a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.556+0000 7fa251ddc740 1 -- 172.21.15.18:0/3005988594 shutdown_connections 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.556+0000 7fa251ddc740 1 -- 172.21.15.18:0/3005988594 wait complete. 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.556+0000 7fa244c00640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa22c0a3c30 0x7fa22c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.556+0000 7fa251ddc740 1 -- start start 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.556+0000 7fa244c00640 1 -- 172.21.15.18:0/1995223476 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fa2340d0ea0 con 0x7fa248107140 2024-04-16T17:12:42.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.556+0000 7fa251ddc740 1 Processor -- start 2024-04-16T17:12:42.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.556+0000 7fa24d000640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa22c0a3c30 0x7fa22c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af889a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af889f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af88a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa251ddc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5589af88f580 con 0x5589af87ed20 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa251ddc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5589af79d830 con 0x5589af87f6d0 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa251ddc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589af88f700 con 0x5589af88c820 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa251ddc740 10 client.0 _ll_register_callbacks cb 0x5589af7a5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa24d000640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa22c0a3c30 0x7fa22c0a60f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa240005fd0 tx=0x7fa240009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa24d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af88a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa24d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af88a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36886/0 (socket says 172.21.15.18:36886) 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa24d000640 1 -- 172.21.15.18:0/3757849626 learned_addr learned my addr 172.21.15.18:0/3757849626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.557+0000 7fa247e00640 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af889a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa247400640 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af889f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa24d000640 1 -- 172.21.15.18:0/3757849626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 msgr2=0x5589af889f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa24d000640 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af889f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa24d000640 1 -- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 msgr2=0x5589af889a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa24d000640 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af889a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa24d000640 1 -- 172.21.15.18:0/3757849626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589af7a1920 con 0x5589af88c820 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa247e00640 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af889a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa24d000640 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af88a490 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa2400099c0 tx=0x7fa240004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9245aa09ee28a679 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.558+0000 7fa232000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.559+0000 7fa232000640 1 -- 172.21.15.18:0/3757849626 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240004b20 con 0x5589af88c820 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.559+0000 7fa251ddc740 1 -- 172.21.15.18:0/3757849626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa238004c10 con 0x5589af88c820 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.559+0000 7fa232000640 1 -- 172.21.15.18:0/3757849626 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fa240004cc0 con 0x5589af88c820 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.559+0000 7fa232000640 1 -- 172.21.15.18:0/3757849626 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa240030050 con 0x5589af88c820 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.559+0000 7fa251ddc740 10 client.27667 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:42.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.559+0000 7fa251ddc740 1 -- 172.21.15.18:0/3757849626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589af76f7f0 con 0x5589af88c820 2024-04-16T17:12:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.559+0000 7fa232000640 1 -- 172.21.15.18:0/3757849626 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa238004c10 con 0x5589af88c820 2024-04-16T17:12:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.560+0000 7fa232000640 1 -- 172.21.15.18:0/3757849626 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa240026030 con 0x5589af88c820 2024-04-16T17:12:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.560+0000 7fa251ddc740 10 client.27667 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.560+0000 7fa251ddc740 10 client.27667 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:12:42.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.560+0000 7fa251ddc740 20 client.27667 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.560+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5589af8a1180 0x5589af8a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.560+0000 7fa251ddc740 4 client.27667 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.560+0000 7fa251ddc740 1 -- 172.21.15.18:0/3757849626 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589af7f1a40 con 0x5589af8a1180 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.560+0000 7fa247e00640 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5589af8a1180 0x5589af8a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.561+0000 7fa247e00640 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5589af8a1180 0x5589af8a35b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92251a853b4e5c8b server_cookie=b31f053ca6a7fc5 in_seq=0 out_seq=0 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.561+0000 7fa232000640 10 client.27667 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa232000640 1 -- 172.21.15.18:0/3757849626 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589af7f1a40 con 0x5589af8a1180 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa232000640 10 client.27667 handle_command_reply: tid=0 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 client.27667 shutdown 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 -- 172.21.15.18:0/3757849626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 msgr2=0x5589af88a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af88a490 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa2400099c0 tx=0x7fa240004ab0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 -- 172.21.15.18:0/3757849626 shutdown_connections 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5589af8a1180 0x5589af8a35b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af88a490 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87f6d0 0x5589af889f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 --2- 172.21.15.18:0/3757849626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af889a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa251ddc740 1 -- 172.21.15.18:0/3757849626 >> 172.21.15.18:0/3757849626 conn(0x5589af879640 msgr2=0x5589af87b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.562+0000 7fa232000640 0 client.27667 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:12:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.563+0000 7fa251ddc740 1 -- 172.21.15.18:0/3757849626 shutdown_connections 2024-04-16T17:12:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.563+0000 7fa251ddc740 1 -- 172.21.15.18:0/3757849626 wait complete. 2024-04-16T17:12:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.563+0000 7fa251ddc740 20 client.27667 trim_cache size 0 max 16384 2024-04-16T17:12:42.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.581+0000 7fa251ddc740 1 Processor -- start 2024-04-16T17:12:42.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.581+0000 7fa251ddc740 1 -- start start 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.581+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af8a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87f6d0 0x5589af8a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af88c820 0x5589af877e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa251ddc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481afb10 con 0x5589af87ed20 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa251ddc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ad3e0 con 0x5589af88c820 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa251ddc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ab250 con 0x5589af87f6d0 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa24d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af88c820 0x5589af877e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa24d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af88c820 0x5589af877e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54798/0 (socket says 172.21.15.18:54798) 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa24d000640 1 -- 172.21.15.18:0/2940705131 learned_addr learned my addr 172.21.15.18:0/2940705131 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa247400640 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87f6d0 0x5589af8a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa247e00640 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af8a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa24d000640 1 -- 172.21.15.18:0/2940705131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87f6d0 msgr2=0x5589af8a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa24d000640 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87f6d0 0x5589af8a93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa24d000640 1 -- 172.21.15.18:0/2940705131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 msgr2=0x5589af8a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa24d000640 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af8a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.582+0000 7fa24d000640 1 -- 172.21.15.18:0/2940705131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589af4f63f0 con 0x5589af88c820 2024-04-16T17:12:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.583+0000 7fa24d000640 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af88c820 0x5589af877e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa240018040 tx=0x7fa240003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f946c05342e7c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.583+0000 7fa23cc00640 1 -- 172.21.15.18:0/2940705131 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400059a0 con 0x5589af88c820 2024-04-16T17:12:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.583+0000 7fa23cc00640 1 -- 172.21.15.18:0/2940705131 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240004b20 con 0x5589af88c820 2024-04-16T17:12:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.583+0000 7fa23cc00640 1 -- 172.21.15.18:0/2940705131 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24001e500 con 0x5589af88c820 2024-04-16T17:12:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 -- 172.21.15.18:0/2940705131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af88c820 msgr2=0x5589af877e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af88c820 0x5589af877e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa240018040 tx=0x7fa240003df0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 -- 172.21.15.18:0/2940705131 shutdown_connections 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af88c820 0x5589af877e70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af87f6d0 0x5589af8a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 --2- 172.21.15.18:0/2940705131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87ed20 0x5589af8a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 -- 172.21.15.18:0/2940705131 >> 172.21.15.18:0/2940705131 conn(0x5589af879640 msgr2=0x5589af888bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 -- 172.21.15.18:0/2940705131 shutdown_connections 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.584+0000 7fa251ddc740 1 -- 172.21.15.18:0/2940705131 wait complete. 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 1 -- start start 2024-04-16T17:12:42.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 1 Processor -- start 2024-04-16T17:12:42.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87ed20 0x5589af89d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87f6d0 0x5589af89d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af886280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ab8a0 con 0x5589af87f6d0 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481acd10 con 0x5589af87ed20 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481abf70 con 0x5589af88c820 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.585+0000 7fa251ddc740 10 client.0 _ll_register_callbacks cb 0x7fa240009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87ed20 0x5589af89d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87ed20 0x5589af89d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54808/0 (socket says 172.21.15.18:54808) 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 -- 172.21.15.18:0/1504538088 learned_addr learned my addr 172.21.15.18:0/1504538088 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247400640 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87f6d0 0x5589af89d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa24d000640 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af886280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 -- 172.21.15.18:0/1504538088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 msgr2=0x5589af886280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af886280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 -- 172.21.15.18:0/1504538088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87f6d0 msgr2=0x5589af89d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87f6d0 0x5589af89d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 -- 172.21.15.18:0/1504538088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589af5c07d0 con 0x5589af87ed20 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa24d000640 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af886280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa247e00640 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87ed20 0x5589af89d410 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa238036040 tx=0x7fa23802ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb565ec7f99548 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.586+0000 7fa232000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.587+0000 7fa232000640 1 -- 172.21.15.18:0/1504538088 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2380535c0 con 0x5589af87ed20 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.587+0000 7fa232000640 1 -- 172.21.15.18:0/1504538088 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fa238053760 con 0x5589af87ed20 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.587+0000 7fa251ddc740 1 -- 172.21.15.18:0/1504538088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa240004b20 con 0x5589af87ed20 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.587+0000 7fa232000640 1 -- 172.21.15.18:0/1504538088 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa238063050 con 0x5589af87ed20 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.587+0000 7fa251ddc740 10 client.27767 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.587+0000 7fa251ddc740 1 -- 172.21.15.18:0/1504538088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589af906d90 con 0x5589af87ed20 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.587+0000 7fa232000640 1 -- 172.21.15.18:0/1504538088 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240004b20 con 0x5589af87ed20 2024-04-16T17:12:42.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.588+0000 7fa232000640 1 -- 172.21.15.18:0/1504538088 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa2380344d0 con 0x5589af87ed20 2024-04-16T17:12:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.588+0000 7fa251ddc740 10 client.27767 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.588+0000 7fa251ddc740 10 client.27767 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:12:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.588+0000 7fa251ddc740 20 client.27767 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.588+0000 7fa251ddc740 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5589af907080 0x5589af909470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.588+0000 7fa251ddc740 4 client.27767 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.588+0000 7fa251ddc740 1 -- 172.21.15.18:0/1504538088 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589af8835a0 con 0x5589af907080 2024-04-16T17:12:42.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.588+0000 7fa247400640 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5589af907080 0x5589af909470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa247400640 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5589af907080 0x5589af909470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78dde9ad7ebb18e0 server_cookie=1f4eabf5cebcf8e0 in_seq=0 out_seq=0 2024-04-16T17:12:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa232000640 10 client.27767 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:12:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa232000640 1 -- 172.21.15.18:0/1504538088 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589af8835a0 con 0x5589af907080 2024-04-16T17:12:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa232000640 10 client.27767 handle_command_reply: tid=0 2024-04-16T17:12:42.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa251ddc740 1 client.27767 shutdown 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa251ddc740 1 -- 172.21.15.18:0/1504538088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87ed20 msgr2=0x5589af89d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa251ddc740 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87ed20 0x5589af89d410 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa238036040 tx=0x7fa23802ce00 comp rx=0 tx=0).stop 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa251ddc740 1 -- 172.21.15.18:0/1504538088 shutdown_connections 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.589+0000 7fa251ddc740 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5589af907080 0x5589af909470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa251ddc740 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589af88c820 0x5589af886280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa251ddc740 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589af87f6d0 0x5589af89d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa232000640 0 client.27767 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa251ddc740 1 --2- 172.21.15.18:0/1504538088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589af87ed20 0x5589af89d410 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa251ddc740 1 -- 172.21.15.18:0/1504538088 >> 172.21.15.18:0/1504538088 conn(0x5589af879640 msgr2=0x5589af881d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa251ddc740 1 -- 172.21.15.18:0/1504538088 shutdown_connections 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa251ddc740 1 -- 172.21.15.18:0/1504538088 wait complete. 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa251ddc740 20 client.27767 trim_cache size 0 max 16384 2024-04-16T17:12:42.593 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:12:42.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa24da00640 1 -- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa22c0a3c30 msgr2=0x7fa22c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.590+0000 7fa24da00640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa22c0a3c30 0x7fa22c0a60f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa240005fd0 tx=0x7fa240009500 comp rx=0 tx=0).stop 2024-04-16T17:12:42.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 -- 172.21.15.18:0/1995223476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248107140 msgr2=0x7fa248198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248107140 0x7fa248198c40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa23400e9a0 tx=0x7fa23400ee60 comp rx=0 tx=0).stop 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 -- 172.21.15.18:0/1995223476 shutdown_connections 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa22c0a3c30 0x7fa22c0a60f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa24819cfc0 0x7fa24819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa248107140 0x7fa248198c40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 --2- 172.21.15.18:0/1995223476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa248102070 0x7fa248198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 -- 172.21.15.18:0/1995223476 >> 172.21.15.18:0/1995223476 conn(0x7fa2480fd950 msgr2=0x7fa2480feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 -- 172.21.15.18:0/1995223476 shutdown_connections 2024-04-16T17:12:42.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.591+0000 7fa24da00640 1 -- 172.21.15.18:0/1995223476 wait complete. 2024-04-16T17:12:42.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 307551 3.91 MB/sec execute 295 sec latency 18.603 ms 2024-04-16T17:12:42.609 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 -- 172.21.15.18:0/4104810195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c28074cf0 msgr2=0x7f2c2810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 --2- 172.21.15.18:0/4104810195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c28074cf0 0x7f2c2810e320 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009900 tx=0x7f2c1402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 -- 172.21.15.18:0/4104810195 shutdown_connections 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 --2- 172.21.15.18:0/4104810195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2810e860 0x7f2c28110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 --2- 172.21.15.18:0/4104810195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c28074cf0 0x7f2c2810e320 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 --2- 172.21.15.18:0/4104810195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c280743d0 0x7f2c280747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 -- 172.21.15.18:0/4104810195 >> 172.21.15.18:0/4104810195 conn(0x7f2c280fdab0 msgr2=0x7f2c280ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 -- 172.21.15.18:0/4104810195 shutdown_connections 2024-04-16T17:12:42.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.874+0000 7f2c30400640 1 -- 172.21.15.18:0/4104810195 wait complete. 2024-04-16T17:12:42.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.875+0000 7f2c30400640 1 Processor -- start 2024-04-16T17:12:42.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.875+0000 7f2c30400640 1 -- start start 2024-04-16T17:12:42.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.875+0000 7f2c30400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c280743d0 0x7f2c28196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c30400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c28074cf0 0x7f2c28197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c30400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2810e860 0x7f2c2819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c30400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28104840 con 0x7f2c280743d0 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c30400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281046c0 con 0x7f2c2810e860 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c30400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281049c0 con 0x7f2c28074cf0 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c280743d0 0x7f2c28196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c280743d0 0x7f2c28196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58302/0 (socket says 172.21.15.18:58302) 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2da00640 1 -- 172.21.15.18:0/4026893269 learned_addr learned my addr 172.21.15.18:0/4026893269 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2d000640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c28074cf0 0x7f2c28197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2e400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2810e860 0x7f2c2819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2da00640 1 -- 172.21.15.18:0/4026893269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c28074cf0 msgr2=0x7f2c28197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2da00640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c28074cf0 0x7f2c28197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2da00640 1 -- 172.21.15.18:0/4026893269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2810e860 msgr2=0x7f2c2819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2da00640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2810e860 0x7f2c2819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2da00640 1 -- 172.21.15.18:0/4026893269 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2819b720 con 0x7f2c280743d0 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.876+0000 7f2c2e400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2810e860 0x7f2c2819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.877+0000 7f2c2da00640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c280743d0 0x7f2c28196b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009fd0 tx=0x7f2c1c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.877+0000 7f2c26000640 1 -- 172.21.15.18:0/4026893269 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c01c760 con 0x7f2c280743d0 2024-04-16T17:12:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.877+0000 7f2c26000640 1 -- 172.21.15.18:0/4026893269 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c1c022070 con 0x7f2c280743d0 2024-04-16T17:12:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.877+0000 7f2c30400640 1 -- 172.21.15.18:0/4026893269 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2819ba10 con 0x7f2c280743d0 2024-04-16T17:12:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.877+0000 7f2c26000640 1 -- 172.21.15.18:0/4026893269 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c01dc30 con 0x7f2c280743d0 2024-04-16T17:12:42.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.877+0000 7f2c30400640 1 -- 172.21.15.18:0/4026893269 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c281a32a0 con 0x7f2c280743d0 2024-04-16T17:12:42.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.881+0000 7f2c26000640 1 -- 172.21.15.18:0/4026893269 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c1c01d3f0 con 0x7f2c280743d0 2024-04-16T17:12:42.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.882+0000 7f2c33181740 1 Processor -- start 2024-04-16T17:12:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.882+0000 7f2c33181740 1 -- start start 2024-04-16T17:12:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c26000640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c100a3d20 0x7f2c100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c33181740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e957796130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c33181740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c33181740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e95779f250 0x55e9577a1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c33181740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e9576fb020 con 0x55e95779f250 2024-04-16T17:12:42.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c33181740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e9576a1460 con 0x55e957796670 2024-04-16T17:12:42.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c33181740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9576f6fe0 con 0x55e957795d50 2024-04-16T17:12:42.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c26000640 1 -- 172.21.15.18:0/4026893269 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f2c1c0d8f60 con 0x7f2c280743d0 2024-04-16T17:12:42.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c2da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e957796130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c2e400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e95779f250 0x55e9577a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c2d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c2e400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e95779f250 0x55e9577a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58304/0 (socket says 172.21.15.18:58304) 2024-04-16T17:12:42.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c2d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54830/0 (socket says 172.21.15.18:54830) 2024-04-16T17:12:42.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c2e400640 1 -- 172.21.15.18:0/4108198191 learned_addr learned my addr 172.21.15.18:0/4108198191 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.883+0000 7f2c2e400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c100a3d20 0x7f2c100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.884+0000 7f2c2da00640 1 -- 172.21.15.18:0/4108198191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 msgr2=0x55e95779ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.884+0000 7f2c2da00640 1 --2- 172.21.15.18:0/4108198191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.884+0000 7f2c2da00640 1 -- 172.21.15.18:0/4108198191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e95779f250 msgr2=0x55e9577a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.884+0000 7f2c2da00640 1 --2- 172.21.15.18:0/4108198191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e95779f250 0x55e9577a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.884+0000 7f2c2da00640 1 -- 172.21.15.18:0/4108198191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9576a9040 con 0x55e957795d50 2024-04-16T17:12:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.884+0000 7f2c2da00640 1 --2- 172.21.15.18:0/4108198191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e957796130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0d6240 tx=0x7f2c1c0a5190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ca6593ca7e7bec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.884+0000 7f2c2e400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c100a3d20 0x7f2c100a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c20005c40 tx=0x7f2c20009f40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.884+0000 7f2c1aa00640 1 -- 172.21.15.18:0/4108198191 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c0a8e40 con 0x55e957795d50 2024-04-16T17:12:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.885+0000 7f2c1aa00640 1 -- 172.21.15.18:0/4108198191 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c1c0ad700 con 0x55e957795d50 2024-04-16T17:12:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.885+0000 7f2c1aa00640 1 -- 172.21.15.18:0/4108198191 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c0a4140 con 0x55e957795d50 2024-04-16T17:12:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 -- 172.21.15.18:0/4108198191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 msgr2=0x55e957796130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 --2- 172.21.15.18:0/4108198191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e957796130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0d6240 tx=0x7f2c1c0a5190 comp rx=0 tx=0).stop 2024-04-16T17:12:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 -- 172.21.15.18:0/4108198191 shutdown_connections 2024-04-16T17:12:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 --2- 172.21.15.18:0/4108198191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e95779f250 0x55e9577a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 --2- 172.21.15.18:0/4108198191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 --2- 172.21.15.18:0/4108198191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e957796130 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 -- 172.21.15.18:0/4108198191 >> 172.21.15.18:0/4108198191 conn(0x55e957790480 msgr2=0x55e957790860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 -- 172.21.15.18:0/4108198191 shutdown_connections 2024-04-16T17:12:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.886+0000 7f2c33181740 1 -- 172.21.15.18:0/4108198191 wait complete. 2024-04-16T17:12:42.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.887+0000 7f2c33181740 1 -- start start 2024-04-16T17:12:42.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.887+0000 7f2c33181740 1 Processor -- start 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c33181740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e95779c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c33181740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e95779c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c33181740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e95779cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c33181740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e9577a2040 con 0x55e957796670 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c33181740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e9576b4830 con 0x55e95779f250 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c33181740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9577a21c0 con 0x55e957795d50 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c33181740 10 client.0 _ll_register_callbacks cb 0x55e9576bc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c2da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e95779c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c2da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e95779c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36940/0 (socket says 172.21.15.18:36940) 2024-04-16T17:12:42.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c2da00640 1 -- 172.21.15.18:0/187920548 learned_addr learned my addr 172.21.15.18:0/187920548 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c2d000640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e95779c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c2e400640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e95779cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.888+0000 7f2c2da00640 1 -- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 msgr2=0x55e95779cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c2da00640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e95779cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c2da00640 1 -- 172.21.15.18:0/187920548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 msgr2=0x55e95779c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c2da00640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e95779c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c2da00640 1 -- 172.21.15.18:0/187920548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9576b8920 con 0x55e957795d50 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c2d000640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e95779c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:42.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c2e400640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e95779cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c2da00640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e95779c330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0a4790 tx=0x7f2c1c01cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a3d0fb574b11997 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c0f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.889+0000 7f2c0f400640 1 -- 172.21.15.18:0/187920548 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c0a1250 con 0x55e957795d50 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.890+0000 7f2c0f400640 1 -- 172.21.15.18:0/187920548 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f2c1c0a40f0 con 0x55e957795d50 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.890+0000 7f2c33181740 1 -- 172.21.15.18:0/187920548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c1c0ad700 con 0x55e957795d50 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.890+0000 7f2c0f400640 1 -- 172.21.15.18:0/187920548 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c1c0a8e40 con 0x55e957795d50 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.890+0000 7f2c33181740 10 client.27682 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.890+0000 7f2c33181740 1 -- 172.21.15.18:0/187920548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9576867f0 con 0x55e957795d50 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.890+0000 7f2c0f400640 1 -- 172.21.15.18:0/187920548 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c1c0ad700 con 0x55e957795d50 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.891+0000 7f2c0f400640 1 -- 172.21.15.18:0/187920548 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2c1c08f0b0 con 0x55e957795d50 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.891+0000 7f2c33181740 10 client.27682 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.891+0000 7f2c33181740 10 client.27682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:42.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.891+0000 7f2c33181740 20 client.27682 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:42.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.891+0000 7f2c33181740 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9577af520 0x55e9577b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.892+0000 7f2c33181740 4 client.27682 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:42.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.892+0000 7f2c33181740 1 -- 172.21.15.18:0/187920548 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e957708a40 con 0x55e9577af520 2024-04-16T17:12:42.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.892+0000 7f2c2d000640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9577af520 0x55e9577b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.892+0000 7f2c2d000640 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9577af520 0x55e9577b1910 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a32723e230cc728 server_cookie=e40569ecaec85fe4 in_seq=0 out_seq=0 2024-04-16T17:12:42.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.892+0000 7f2c0f400640 10 client.27682 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:42.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c0f400640 1 -- 172.21.15.18:0/187920548 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e957708a40 con 0x55e9577af520 2024-04-16T17:12:42.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c0f400640 10 client.27682 handle_command_reply: tid=0 2024-04-16T17:12:42.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 client.27682 shutdown 2024-04-16T17:12:42.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 -- 172.21.15.18:0/187920548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 msgr2=0x55e95779c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e95779c330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0a4790 tx=0x7f2c1c01cf00 comp rx=0 tx=0).stop 2024-04-16T17:12:42.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 -- 172.21.15.18:0/187920548 shutdown_connections 2024-04-16T17:12:42.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e9577af520 0x55e9577b1910 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e95779cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c0f400640 0 client.27682 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:42.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e95779c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 --2- 172.21.15.18:0/187920548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e95779c330 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.894+0000 7f2c33181740 1 -- 172.21.15.18:0/187920548 >> 172.21.15.18:0/187920548 conn(0x55e957790480 msgr2=0x55e9577921d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.895+0000 7f2c33181740 1 -- 172.21.15.18:0/187920548 shutdown_connections 2024-04-16T17:12:42.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.895+0000 7f2c33181740 1 -- 172.21.15.18:0/187920548 wait complete. 2024-04-16T17:12:42.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.895+0000 7f2c33181740 20 client.27682 trim_cache size 0 max 16384 2024-04-16T17:12:42.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c33181740 1 Processor -- start 2024-04-16T17:12:42.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c33181740 1 -- start start 2024-04-16T17:12:42.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c33181740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957795d50 0x55e95779ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c33181740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c33181740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e95779f250 0x55e9577aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c33181740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281aff20 con 0x55e957795d50 2024-04-16T17:12:42.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c33181740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281ad7f0 con 0x55e957796670 2024-04-16T17:12:42.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c33181740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281ab660 con 0x55e95779f250 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c2da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957795d50 0x55e95779ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.925+0000 7f2c2d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54852/0 (socket says 172.21.15.18:54852) 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2d000640 1 -- 172.21.15.18:0/1174898520 learned_addr learned my addr 172.21.15.18:0/1174898520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2e400640 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e95779f250 0x55e9577aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2d000640 1 -- 172.21.15.18:0/1174898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e95779f250 msgr2=0x55e9577aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2d000640 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e95779f250 0x55e9577aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2d000640 1 -- 172.21.15.18:0/1174898520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957795d50 msgr2=0x55e95779ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2d000640 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957795d50 0x55e95779ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2d000640 1 -- 172.21.15.18:0/1174898520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9577964a0 con 0x55e957796670 2024-04-16T17:12:42.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2e400640 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e95779f250 0x55e9577aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c2d000640 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779b490 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009230 tx=0x7f2c14009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10fc30fff5f97d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.926+0000 7f2c1a000640 1 -- 172.21.15.18:0/1174898520 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1404b020 con 0x55e957796670 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.927+0000 7f2c1a000640 1 -- 172.21.15.18:0/1174898520 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c1402c920 con 0x55e957796670 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.927+0000 7f2c1a000640 1 -- 172.21.15.18:0/1174898520 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14047800 con 0x55e957796670 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 -- 172.21.15.18:0/1174898520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 msgr2=0x55e95779b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779b490 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009230 tx=0x7f2c14009260 comp rx=0 tx=0).stop 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 -- 172.21.15.18:0/1174898520 shutdown_connections 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e95779f250 0x55e9577aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e957796670 0x55e95779b490 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 --2- 172.21.15.18:0/1174898520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957795d50 0x55e95779ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 -- 172.21.15.18:0/1174898520 >> 172.21.15.18:0/1174898520 conn(0x55e957790480 msgr2=0x55e9577aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 -- 172.21.15.18:0/1174898520 shutdown_connections 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 -- 172.21.15.18:0/1174898520 wait complete. 2024-04-16T17:12:42.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.928+0000 7f2c33181740 1 -- start start 2024-04-16T17:12:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.929+0000 7f2c33181740 1 Processor -- start 2024-04-16T17:12:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.929+0000 7f2c33181740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e9577b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.929+0000 7f2c33181740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e9577b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.929+0000 7f2c33181740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e9577b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.929+0000 7f2c33181740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281abcb0 con 0x55e957796670 2024-04-16T17:12:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.929+0000 7f2c33181740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281ad120 con 0x55e95779f250 2024-04-16T17:12:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.929+0000 7f2c33181740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281ac380 con 0x55e957795d50 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e9577b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e9577b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36956/0 (socket says 172.21.15.18:36956) 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 -- 172.21.15.18:0/466939745 learned_addr learned my addr 172.21.15.18:0/466939745 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2d000640 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e9577b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c33181740 10 client.0 _ll_register_callbacks cb 0x7f2c20011740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2e400640 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e9577b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 -- 172.21.15.18:0/466939745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 msgr2=0x55e9577b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e9577b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 -- 172.21.15.18:0/466939745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 msgr2=0x55e9577b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e9577b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 -- 172.21.15.18:0/466939745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95740d3f0 con 0x55e957795d50 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2d000640 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e9577b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c2da00640 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e9577b2620 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0a1460 tx=0x7f2c1c0a0db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19ca645605d95ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.930+0000 7f2c0f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c0f400640 1 -- 172.21.15.18:0/466939745 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c0ad700 con 0x55e957795d50 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c0f400640 1 -- 172.21.15.18:0/466939745 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f2c1c08f0b0 con 0x55e957795d50 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c33181740 1 -- 172.21.15.18:0/466939745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c1402c920 con 0x55e957795d50 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c0f400640 1 -- 172.21.15.18:0/466939745 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c1c0a07e0 con 0x55e957795d50 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c33181740 10 client.27688 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c33181740 1 -- 172.21.15.18:0/466939745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9574d77d0 con 0x55e957795d50 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c0f400640 1 -- 172.21.15.18:0/466939745 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c1402c920 con 0x55e957795d50 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c0f400640 1 -- 172.21.15.18:0/466939745 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2c1c0d6990 con 0x55e957795d50 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c33181740 10 client.27688 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c33181740 10 client.27688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.931+0000 7f2c33181740 20 client.27688 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.932+0000 7f2c33181740 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e957815e10 0x55e957818200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.932+0000 7f2c33181740 4 client.27688 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.932+0000 7f2c33181740 1 -- 172.21.15.18:0/466939745 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9577028b0 con 0x55e957815e10 2024-04-16T17:12:42.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.932+0000 7f2c2d000640 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e957815e10 0x55e957818200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c2d000640 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e957815e10 0x55e957818200 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae2e3a5e0162a67 server_cookie=d40ea9394cd8c121 in_seq=0 out_seq=0 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c0f400640 10 client.27688 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c0f400640 1 -- 172.21.15.18:0/466939745 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9577028b0 con 0x55e957815e10 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c0f400640 10 client.27688 handle_command_reply: tid=0 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 client.27688 shutdown 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 -- 172.21.15.18:0/466939745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 msgr2=0x55e9577b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e9577b2620 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0a1460 tx=0x7f2c1c0a0db0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 -- 172.21.15.18:0/466939745 shutdown_connections 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e957815e10 0x55e957818200 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95779f250 0x55e9577b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e957796670 0x55e9577b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c0f400640 0 client.27688 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 --2- 172.21.15.18:0/466939745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e957795d50 0x55e9577b2620 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 -- 172.21.15.18:0/466939745 >> 172.21.15.18:0/466939745 conn(0x55e957790480 msgr2=0x55e95778ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 -- 172.21.15.18:0/466939745 shutdown_connections 2024-04-16T17:12:42.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 1 -- 172.21.15.18:0/466939745 wait complete. 2024-04-16T17:12:42.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.933+0000 7f2c33181740 20 client.27688 trim_cache size 0 max 16384 2024-04-16T17:12:42.937 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:12:42.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 -- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c100a3d20 msgr2=0x7f2c100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c100a3d20 0x7f2c100a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c20005c40 tx=0x7f2c20009f40 comp rx=0 tx=0).stop 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 -- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c280743d0 msgr2=0x7f2c28196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c280743d0 0x7f2c28196b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009fd0 tx=0x7f2c1c007650 comp rx=0 tx=0).stop 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 -- 172.21.15.18:0/4026893269 shutdown_connections 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c100a3d20 0x7f2c100a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2810e860 0x7f2c2819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c28074cf0 0x7f2c28197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 --2- 172.21.15.18:0/4026893269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c280743d0 0x7f2c28196b20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.934+0000 7f2c30400640 1 -- 172.21.15.18:0/4026893269 >> 172.21.15.18:0/4026893269 conn(0x7f2c280fdab0 msgr2=0x7f2c280ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.935+0000 7f2c30400640 1 -- 172.21.15.18:0/4026893269 shutdown_connections 2024-04-16T17:12:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:42.935+0000 7f2c30400640 1 -- 172.21.15.18:0/4026893269 wait complete. 2024-04-16T17:12:43.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:42 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2671938910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:43.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:42 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2671938910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:43.605 INFO:tasks.workunit.client.0.smithi018.stdout: 1 308668 3.93 MB/sec execute 296 sec latency 59.193 ms 2024-04-16T17:12:43.952 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:12:43.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:12:43.953 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:12:44.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:43 smithi043 ceph-mon[36901]: pgmap v248: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:12:44.001 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:31 smithi018 ceph-mon[31776]: pgmap v242: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-04-16T17:12:44.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:33 smithi018 ceph-mon[31776]: pgmap v243: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-04-16T17:12:44.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:35 smithi018 ceph-mon[31776]: pgmap v244: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-16T17:12:44.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:37 smithi018 ceph-mon[31776]: pgmap v245: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-04-16T17:12:44.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:39 smithi018 ceph-mon[31776]: pgmap v246: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:12:44.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:41 smithi018 ceph-mon[31776]: pgmap v247: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-16T17:12:44.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:42 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2671938910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 -- 172.21.15.18:0/2106216681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed38102fe0 msgr2=0x7fed381033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 --2- 172.21.15.18:0/2106216681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed38102fe0 0x7fed381033c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fed28009900 tx=0x7fed2802bce0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 -- 172.21.15.18:0/2106216681 shutdown_connections 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 --2- 172.21.15.18:0/2106216681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed3810c400 0x7fed3810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 --2- 172.21.15.18:0/2106216681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed38103900 0x7fed3810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 --2- 172.21.15.18:0/2106216681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed38102fe0 0x7fed381033c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 -- 172.21.15.18:0/2106216681 >> 172.21.15.18:0/2106216681 conn(0x7fed380fc7e0 msgr2=0x7fed380fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 -- 172.21.15.18:0/2106216681 shutdown_connections 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.088+0000 7fed3ca00640 1 -- 172.21.15.18:0/2106216681 wait complete. 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.089+0000 7fed3ca00640 1 Processor -- start 2024-04-16T17:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.089+0000 7fed3ca00640 1 -- start start 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed3ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed38102fe0 0x7fed381a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed3ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed38103900 0x7fed381a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed3ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed3810c400 0x7fed3819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed3ca00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed38112260 con 0x7fed3810c400 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed3ca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed381120e0 con 0x7fed38103900 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed3ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed381123e0 con 0x7fed38102fe0 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed37e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed3810c400 0x7fed3819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed37e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed3810c400 0x7fed3819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58340/0 (socket says 172.21.15.18:58340) 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed38102fe0 0x7fed381a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed37e00640 1 -- 172.21.15.18:0/699922022 learned_addr learned my addr 172.21.15.18:0/699922022 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.090+0000 7fed36a00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed38103900 0x7fed381a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed36a00640 1 -- 172.21.15.18:0/699922022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed38102fe0 msgr2=0x7fed381a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed36a00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed38102fe0 0x7fed381a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed36a00640 1 -- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed3810c400 msgr2=0x7fed3819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed36a00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed3810c400 0x7fed3819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed36a00640 1 -- 172.21.15.18:0/699922022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3819af70 con 0x7fed38103900 2024-04-16T17:12:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed36a00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed38103900 0x7fed381a0b70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fed2400b410 tx=0x7fed2400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed2fe00640 1 -- 172.21.15.18:0/699922022 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24015020 con 0x7fed38103900 2024-04-16T17:12:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed2fe00640 1 -- 172.21.15.18:0/699922022 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24011d10 con 0x7fed38103900 2024-04-16T17:12:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed3ca00640 1 -- 172.21.15.18:0/699922022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3819b260 con 0x7fed38103900 2024-04-16T17:12:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed2fe00640 1 -- 172.21.15.18:0/699922022 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2400f2e0 con 0x7fed38103900 2024-04-16T17:12:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.091+0000 7fed3ca00640 1 -- 172.21.15.18:0/699922022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed381a7480 con 0x7fed38103900 2024-04-16T17:12:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.096+0000 7fed40d92740 1 Processor -- start 2024-04-16T17:12:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.096+0000 7fed40d92740 1 -- start start 2024-04-16T17:12:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed40d92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x55688423f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed40d92740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55688423fa80 0x55688423ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed40d92740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x55688424b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed40d92740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5568841a4780 con 0x55688423fa80 2024-04-16T17:12:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed40d92740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55688414af40 con 0x556884249000 2024-04-16T17:12:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed40d92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5568841a0740 con 0x5568842407f0 2024-04-16T17:12:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed2fe00640 1 -- 172.21.15.18:0/699922022 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed24018020 con 0x7fed38103900 2024-04-16T17:12:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed37400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55688423fa80 0x55688423ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed37e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x55688423f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed37400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55688423fa80 0x55688423ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58348/0 (socket says 172.21.15.18:58348) 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed37400640 1 -- 172.21.15.18:0/3763402454 learned_addr learned my addr 172.21.15.18:0/3763402454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.097+0000 7fed36a00640 1 --2- 172.21.15.18:0/3763402454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x55688424b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.098+0000 7fed37e00640 1 -- 172.21.15.18:0/3763402454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 msgr2=0x55688424b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.098+0000 7fed37e00640 1 --2- 172.21.15.18:0/3763402454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x55688424b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.098+0000 7fed37e00640 1 -- 172.21.15.18:0/3763402454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55688423fa80 msgr2=0x55688423ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.098+0000 7fed37e00640 1 --2- 172.21.15.18:0/3763402454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55688423fa80 0x55688423ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.098+0000 7fed37e00640 1 -- 172.21.15.18:0/3763402454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556883eb7210 con 0x5568842407f0 2024-04-16T17:12:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.098+0000 7fed37e00640 1 --2- 172.21.15.18:0/3763402454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x55688423f540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fed30009f50 tx=0x7fed30011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bf64aa5fd65da80 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.099+0000 7fed23e00640 1 -- 172.21.15.18:0/3763402454 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30004d20 con 0x5568842407f0 2024-04-16T17:12:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.099+0000 7fed23e00640 1 -- 172.21.15.18:0/3763402454 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed300129f0 con 0x5568842407f0 2024-04-16T17:12:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.099+0000 7fed23e00640 1 -- 172.21.15.18:0/3763402454 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed300055f0 con 0x5568842407f0 2024-04-16T17:12:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.099+0000 7fed2fe00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed1c0a3d00 0x7fed1c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed2fe00640 1 -- 172.21.15.18:0/699922022 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fed240d0360 con 0x7fed38103900 2024-04-16T17:12:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 -- 172.21.15.18:0/3763402454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 msgr2=0x55688423f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 --2- 172.21.15.18:0/3763402454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x55688423f540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fed30009f50 tx=0x7fed30011870 comp rx=0 tx=0).stop 2024-04-16T17:12:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 -- 172.21.15.18:0/3763402454 shutdown_connections 2024-04-16T17:12:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 --2- 172.21.15.18:0/3763402454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x55688424b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 --2- 172.21.15.18:0/3763402454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55688423fa80 0x55688423ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 --2- 172.21.15.18:0/3763402454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x55688423f540 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 -- 172.21.15.18:0/3763402454 >> 172.21.15.18:0/3763402454 conn(0x5568841da5d0 msgr2=0x55688423a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 -- 172.21.15.18:0/3763402454 shutdown_connections 2024-04-16T17:12:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed40d92740 1 -- 172.21.15.18:0/3763402454 wait complete. 2024-04-16T17:12:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.100+0000 7fed37400640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed1c0a3d00 0x7fed1c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.101+0000 7fed40d92740 1 -- start start 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.101+0000 7fed40d92740 1 Processor -- start 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.101+0000 7fed37400640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed1c0a3d00 0x7fed1c0a61c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed28036040 tx=0x7fed280095f0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed40d92740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55688423fa80 0x556884248a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed40d92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x556884245da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed40d92740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x5568842463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed40d92740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55688424bd30 con 0x556884249000 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed40d92740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55688415e390 con 0x55688423fa80 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed40d92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55688424beb0 con 0x5568842407f0 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed40d92740 10 client.0 _ll_register_callbacks cb 0x556884147f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed36a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x556884245da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed37400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55688423fa80 0x556884248a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed37400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55688423fa80 0x556884248a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54904/0 (socket says 172.21.15.18:54904) 2024-04-16T17:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.102+0000 7fed37400640 1 -- 172.21.15.18:0/2680479423 learned_addr learned my addr 172.21.15.18:0/2680479423 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.103+0000 7fed37e00640 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x5568842463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.103+0000 7fed37400640 1 -- 172.21.15.18:0/2680479423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 msgr2=0x556884245da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.103+0000 7fed37400640 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x556884245da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.103+0000 7fed37400640 1 -- 172.21.15.18:0/2680479423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 msgr2=0x5568842463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.103+0000 7fed37400640 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x5568842463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.103+0000 7fed37400640 1 -- 172.21.15.18:0/2680479423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556883f83620 con 0x55688423fa80 2024-04-16T17:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.103+0000 7fed36a00640 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x556884245da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.104+0000 7fed37400640 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55688423fa80 0x556884248a10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fed2802c8d0 tx=0x7fed28009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812db4dfbac987b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.104+0000 7fed20c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.104+0000 7fed20c00640 1 -- 172.21.15.18:0/2680479423 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2804f070 con 0x55688423fa80 2024-04-16T17:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.104+0000 7fed40d92740 1 -- 172.21.15.18:0/2680479423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed300129f0 con 0x55688423fa80 2024-04-16T17:12:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.107+0000 7fed20c00640 1 -- 172.21.15.18:0/2680479423 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fed280048c0 con 0x55688423fa80 2024-04-16T17:12:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.107+0000 7fed20c00640 1 -- 172.21.15.18:0/2680479423 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed28004ac0 con 0x55688423fa80 2024-04-16T17:12:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.107+0000 7fed40d92740 10 client.27794 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.107+0000 7fed40d92740 1 -- 172.21.15.18:0/2680479423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556884257df0 con 0x55688423fa80 2024-04-16T17:12:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.107+0000 7fed20c00640 1 -- 172.21.15.18:0/2680479423 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed300129f0 con 0x55688423fa80 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.108+0000 7fed20c00640 1 -- 172.21.15.18:0/2680479423 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fed280035f0 con 0x55688423fa80 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.109+0000 7fed40d92740 10 client.27794 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.109+0000 7fed40d92740 10 client.27794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.109+0000 7fed40d92740 20 client.27794 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.109+0000 7fed40d92740 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556884259260 0x55688425b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.109+0000 7fed40d92740 4 client.27794 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.109+0000 7fed40d92740 1 -- 172.21.15.18:0/2680479423 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568841b21a0 con 0x556884259260 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.109+0000 7fed36a00640 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556884259260 0x55688425b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.110+0000 7fed36a00640 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556884259260 0x55688425b650 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e8498272c6d599 server_cookie=e60c3e34c36296c8 in_seq=0 out_seq=0 2024-04-16T17:12:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.110+0000 7fed20c00640 10 client.27794 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.111+0000 7fed20c00640 1 -- 172.21.15.18:0/2680479423 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568841b21a0 con 0x556884259260 2024-04-16T17:12:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.111+0000 7fed20c00640 10 client.27794 handle_command_reply: tid=0 2024-04-16T17:12:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.111+0000 7fed40d92740 1 client.27794 shutdown 2024-04-16T17:12:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.111+0000 7fed40d92740 1 -- 172.21.15.18:0/2680479423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55688423fa80 msgr2=0x556884248a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.111+0000 7fed40d92740 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55688423fa80 0x556884248a10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fed2802c8d0 tx=0x7fed28009260 comp rx=0 tx=0).stop 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed40d92740 1 -- 172.21.15.18:0/2680479423 shutdown_connections 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed40d92740 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556884259260 0x55688425b650 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed40d92740 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x5568842463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed40d92740 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568842407f0 0x556884245da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed40d92740 1 --2- 172.21.15.18:0/2680479423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55688423fa80 0x556884248a10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed40d92740 1 -- 172.21.15.18:0/2680479423 >> 172.21.15.18:0/2680479423 conn(0x5568841da5d0 msgr2=0x556884239810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed20c00640 0 client.27794 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed40d92740 1 -- 172.21.15.18:0/2680479423 shutdown_connections 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.112+0000 7fed40d92740 1 -- 172.21.15.18:0/2680479423 wait complete. 2024-04-16T17:12:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.113+0000 7fed40d92740 20 client.27794 trim_cache size 0 max 16384 2024-04-16T17:12:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.136+0000 7fed40d92740 1 Processor -- start 2024-04-16T17:12:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.136+0000 7fed40d92740 1 -- start start 2024-04-16T17:12:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.136+0000 7fed40d92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884245d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed40d92740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568842407f0 0x5568842484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed40d92740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x556884248a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed40d92740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed381b4010 con 0x556884249000 2024-04-16T17:12:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed40d92740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed381b18e0 con 0x5568842407f0 2024-04-16T17:12:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed40d92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed381af750 con 0x55688423fa80 2024-04-16T17:12:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed36a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568842407f0 0x5568842484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed36a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568842407f0 0x5568842484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54906/0 (socket says 172.21.15.18:54906) 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed36a00640 1 -- 172.21.15.18:0/2068856986 learned_addr learned my addr 172.21.15.18:0/2068856986 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed37400640 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884245d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.137+0000 7fed37e00640 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x556884248a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed36a00640 1 -- 172.21.15.18:0/2068856986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 msgr2=0x556884245d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed36a00640 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884245d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed36a00640 1 -- 172.21.15.18:0/2068856986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 msgr2=0x556884248a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed36a00640 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x556884248a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed36a00640 1 -- 172.21.15.18:0/2068856986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688423ed50 con 0x5568842407f0 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed37e00640 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x556884248a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed36a00640 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568842407f0 0x5568842484d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fed2400bdd0 tx=0x7fed240d3e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f6bcfe320aef5a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed23400640 1 -- 172.21.15.18:0/2068856986 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24010ea0 con 0x5568842407f0 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed23400640 1 -- 172.21.15.18:0/2068856986 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2409c700 con 0x5568842407f0 2024-04-16T17:12:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.138+0000 7fed23400640 1 -- 172.21.15.18:0/2068856986 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2409a300 con 0x5568842407f0 2024-04-16T17:12:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.139+0000 7fed40d92740 1 -- 172.21.15.18:0/2068856986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568842407f0 msgr2=0x5568842484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.139+0000 7fed40d92740 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568842407f0 0x5568842484d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fed2400bdd0 tx=0x7fed240d3e30 comp rx=0 tx=0).stop 2024-04-16T17:12:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.139+0000 7fed40d92740 1 -- 172.21.15.18:0/2068856986 shutdown_connections 2024-04-16T17:12:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.139+0000 7fed40d92740 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556884249000 0x556884248a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.139+0000 7fed40d92740 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568842407f0 0x5568842484d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.139+0000 7fed40d92740 1 --2- 172.21.15.18:0/2068856986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884245d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.139+0000 7fed40d92740 1 -- 172.21.15.18:0/2068856986 >> 172.21.15.18:0/2068856986 conn(0x5568841da5d0 msgr2=0x556884245650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.139+0000 7fed40d92740 1 -- 172.21.15.18:0/2068856986 shutdown_connections 2024-04-16T17:12:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.140+0000 7fed40d92740 1 -- 172.21.15.18:0/2068856986 wait complete. 2024-04-16T17:12:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.140+0000 7fed40d92740 1 -- start start 2024-04-16T17:12:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.140+0000 7fed40d92740 1 Processor -- start 2024-04-16T17:12:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.141+0000 7fed40d92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884242d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.141+0000 7fed40d92740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568842407f0 0x5568842432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.141+0000 7fed40d92740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x5568842437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.141+0000 7fed40d92740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed381afda0 con 0x5568842407f0 2024-04-16T17:12:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.141+0000 7fed40d92740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed381b1210 con 0x556884249000 2024-04-16T17:12:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.141+0000 7fed40d92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed381b0470 con 0x55688423fa80 2024-04-16T17:12:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.141+0000 7fed40d92740 10 client.0 _ll_register_callbacks cb 0x7fed2802c930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884242d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x5568842437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884242d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36988/0 (socket says 172.21.15.18:36988) 2024-04-16T17:12:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37400640 1 -- 172.21.15.18:0/4135886279 learned_addr learned my addr 172.21.15.18:0/4135886279 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x5568842437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54916/0 (socket says 172.21.15.18:54916) 2024-04-16T17:12:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed36a00640 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568842407f0 0x5568842432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37e00640 1 -- 172.21.15.18:0/4135886279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 msgr2=0x556884242d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37e00640 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884242d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37e00640 1 -- 172.21.15.18:0/4135886279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568842407f0 msgr2=0x5568842432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37e00640 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568842407f0 0x5568842432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.142+0000 7fed37e00640 1 -- 172.21.15.18:0/4135886279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568842be9d0 con 0x556884249000 2024-04-16T17:12:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.143+0000 7fed37e00640 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x5568842437f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fed30011d70 tx=0x7fed30009c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d76163691ad5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.143+0000 7fed20c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.143+0000 7fed20c00640 1 -- 172.21.15.18:0/4135886279 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30029020 con 0x556884249000 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.143+0000 7fed40d92740 1 -- 172.21.15.18:0/4135886279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed2409c700 con 0x556884249000 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.143+0000 7fed20c00640 1 -- 172.21.15.18:0/4135886279 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7fed30005260 con 0x556884249000 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.143+0000 7fed20c00640 1 -- 172.21.15.18:0/4135886279 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed30034050 con 0x556884249000 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.143+0000 7fed40d92740 10 client.27800 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed40d92740 1 -- 172.21.15.18:0/4135886279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568842bfce0 con 0x556884249000 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed20c00640 1 -- 172.21.15.18:0/4135886279 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2409c700 con 0x556884249000 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed20c00640 1 -- 172.21.15.18:0/4135886279 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fed3000f040 con 0x556884249000 2024-04-16T17:12:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed40d92740 10 client.27800 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed40d92740 10 client.27800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed40d92740 20 client.27800 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed40d92740 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568842c0120 0x5568842c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed40d92740 4 client.27800 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed40d92740 1 -- 172.21.15.18:0/4135886279 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568841ac010 con 0x5568842c0120 2024-04-16T17:12:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.144+0000 7fed37400640 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568842c0120 0x5568842c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.145+0000 7fed37400640 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568842c0120 0x5568842c2540 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f570663f78ad3e server_cookie=a37901f92cffb1d9 in_seq=0 out_seq=0 2024-04-16T17:12:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.145+0000 7fed20c00640 10 client.27800 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:43 smithi022 ceph-mon[36023]: pgmap v248: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:12:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.184+0000 7f963ae00640 1 -- 172.21.15.18:0/2314286197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f963411b7e0 msgr2=0x7f963411dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.184+0000 7f963ae00640 1 --2- 172.21.15.18:0/2314286197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f963411b7e0 0x7f963411dbd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9624009900 tx=0x7f962402bc30 comp rx=0 tx=0).stop 2024-04-16T17:12:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.185+0000 7f963ae00640 1 -- 172.21.15.18:0/2314286197 shutdown_connections 2024-04-16T17:12:44.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.185+0000 7f963ae00640 1 --2- 172.21.15.18:0/2314286197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f963411b7e0 0x7f963411dbd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.185+0000 7f963ae00640 1 --2- 172.21.15.18:0/2314286197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9634110bc0 0x7f963410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.185+0000 7f963ae00640 1 --2- 172.21.15.18:0/2314286197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9634110750 0x7f963410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.185+0000 7f963ae00640 1 -- 172.21.15.18:0/2314286197 >> 172.21.15.18:0/2314286197 conn(0x7f963406d0a0 msgr2=0x7f963406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.187+0000 7f963ae00640 1 -- 172.21.15.18:0/2314286197 shutdown_connections 2024-04-16T17:12:44.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.187+0000 7f963ae00640 1 -- 172.21.15.18:0/2314286197 wait complete. 2024-04-16T17:12:44.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.192+0000 7f963ae00640 1 Processor -- start 2024-04-16T17:12:44.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.192+0000 7f963ae00640 1 -- start start 2024-04-16T17:12:44.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.195+0000 7f963ae00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9634110750 0x7f96341a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.195+0000 7f963ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9634110bc0 0x7f96341a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.195+0000 7f963ae00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f963411b7e0 0x7f96341abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.196+0000 7f9633400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9634110bc0 0x7f96341a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.196+0000 7f9633400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9634110bc0 0x7f96341a7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37006/0 (socket says 172.21.15.18:37006) 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.196+0000 7f9633400640 1 -- 172.21.15.18:0/2676765117 learned_addr learned my addr 172.21.15.18:0/2676765117 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.196+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9634121160 con 0x7f963411b7e0 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.196+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9634120fe0 con 0x7f9634110750 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.196+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96341212e0 con 0x7f9634110bc0 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.196+0000 7f9633e00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9634110750 0x7f96341a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.197+0000 7f9638e00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f963411b7e0 0x7f96341abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.197+0000 7f9633400640 1 -- 172.21.15.18:0/2676765117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9634110750 msgr2=0x7f96341a75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.197+0000 7f9633400640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9634110750 0x7f96341a75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.197+0000 7f9633400640 1 -- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f963411b7e0 msgr2=0x7f96341abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.197+0000 7f9633400640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f963411b7e0 0x7f96341abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.197+0000 7f9633400640 1 -- 172.21.15.18:0/2676765117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96341ac5d0 con 0x7f9634110bc0 2024-04-16T17:12:44.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.197+0000 7f9638e00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f963411b7e0 0x7f96341abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:44.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.198+0000 7f9633400640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9634110bc0 0x7f96341a7b20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f962c00ebc0 tx=0x7f962c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.199+0000 7f9630c00640 1 -- 172.21.15.18:0/2676765117 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c010630 con 0x7f9634110bc0 2024-04-16T17:12:44.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.199+0000 7f9630c00640 1 -- 172.21.15.18:0/2676765117 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c010cb0 con 0x7f9634110bc0 2024-04-16T17:12:44.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.199+0000 7f9630c00640 1 -- 172.21.15.18:0/2676765117 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c018bd0 con 0x7f9634110bc0 2024-04-16T17:12:44.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.199+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96341ac8c0 con 0x7f9634110bc0 2024-04-16T17:12:44.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.199+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96341b4150 con 0x7f9634110bc0 2024-04-16T17:12:44.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.209+0000 7f963dbc7740 1 Processor -- start 2024-04-16T17:12:44.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.212+0000 7f9630c00640 1 -- 172.21.15.18:0/2676765117 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f962c0107d0 con 0x7f9634110bc0 2024-04-16T17:12:44.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.213+0000 7f963dbc7740 1 -- start start 2024-04-16T17:12:44.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.214+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef87ca0 0x562f6ef88080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.214+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef885c0 0x562f6ef90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.214+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef911a0 0x562f6ef93590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.214+0000 7f963dbc7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562f6eeeccb0 con 0x562f6ef885c0 2024-04-16T17:12:44.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.214+0000 7f963dbc7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562f6ee93370 con 0x562f6ef87ca0 2024-04-16T17:12:44.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.214+0000 7f963dbc7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f6eee8c70 con 0x562f6ef911a0 2024-04-16T17:12:44.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.214+0000 7f9630c00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f96180a3ea0 0x7f96180a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.215+0000 7f9630c00640 1 -- 172.21.15.18:0/2676765117 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f962c0d1250 con 0x7f9634110bc0 2024-04-16T17:12:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.215+0000 7f9633400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef911a0 0x562f6ef93590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.215+0000 7f9633400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef911a0 0x562f6ef93590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37020/0 (socket says 172.21.15.18:37020) 2024-04-16T17:12:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.215+0000 7f9633400640 1 -- 172.21.15.18:0/3770373422 learned_addr learned my addr 172.21.15.18:0/3770373422 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.215+0000 7f9633400640 1 -- 172.21.15.18:0/3770373422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef87ca0 msgr2=0x562f6ef88080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:12:44.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.215+0000 7f9633400640 1 --2- 172.21.15.18:0/3770373422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef87ca0 0x562f6ef88080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.216+0000 7f9633400640 1 -- 172.21.15.18:0/3770373422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef885c0 msgr2=0x562f6ef90c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.216+0000 7f9633400640 1 --2- 172.21.15.18:0/3770373422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef885c0 0x562f6ef90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.216+0000 7f9633400640 1 -- 172.21.15.18:0/3770373422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6eeb2910 con 0x562f6ef911a0 2024-04-16T17:12:44.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.216+0000 7f9633400640 1 --2- 172.21.15.18:0/3770373422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef911a0 0x562f6ef93590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f962c0997b0 tx=0x7f962c09c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=885c8d1cc6f8b727 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.217 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:43 smithi018 ceph-mon[31776]: pgmap v248: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:12:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.218+0000 7f9629600640 1 -- 172.21.15.18:0/3770373422 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0a0070 con 0x562f6ef911a0 2024-04-16T17:12:44.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.218+0000 7f9629600640 1 -- 172.21.15.18:0/3770373422 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c0a1190 con 0x562f6ef911a0 2024-04-16T17:12:44.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.219+0000 7f963dbc7740 1 -- 172.21.15.18:0/3770373422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef911a0 msgr2=0x562f6ef93590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.219+0000 7f9629600640 1 -- 172.21.15.18:0/3770373422 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0a0070 con 0x562f6ef911a0 2024-04-16T17:12:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.219+0000 7f963dbc7740 1 --2- 172.21.15.18:0/3770373422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef911a0 0x562f6ef93590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f962c0997b0 tx=0x7f962c09c740 comp rx=0 tx=0).stop 2024-04-16T17:12:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.219+0000 7f963dbc7740 1 -- 172.21.15.18:0/3770373422 shutdown_connections 2024-04-16T17:12:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.219+0000 7f963dbc7740 1 --2- 172.21.15.18:0/3770373422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef911a0 0x562f6ef93590 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.219+0000 7f963dbc7740 1 --2- 172.21.15.18:0/3770373422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef885c0 0x562f6ef90c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.219+0000 7f963dbc7740 1 --2- 172.21.15.18:0/3770373422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef87ca0 0x562f6ef88080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.219+0000 7f963dbc7740 1 -- 172.21.15.18:0/3770373422 >> 172.21.15.18:0/3770373422 conn(0x562f6ef823d0 msgr2=0x562f6ef827b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.220+0000 7f963dbc7740 1 -- 172.21.15.18:0/3770373422 shutdown_connections 2024-04-16T17:12:44.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.220+0000 7f963dbc7740 1 -- 172.21.15.18:0/3770373422 wait complete. 2024-04-16T17:12:44.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.220+0000 7f9638e00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f96180a3ea0 0x7f96180a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.220+0000 7f963dbc7740 1 -- start start 2024-04-16T17:12:44.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.221+0000 7f963dbc7740 1 Processor -- start 2024-04-16T17:12:44.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.225+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef87ca0 0x562f6ef8e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.225+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef885c0 0x562f6ef8e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.225+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef911a0 0x562f6ef8ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.225+0000 7f963dbc7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562f6ef93f80 con 0x562f6ef87ca0 2024-04-16T17:12:44.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.225+0000 7f963dbc7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562f6eea6730 con 0x562f6ef911a0 2024-04-16T17:12:44.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.225+0000 7f963dbc7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f6ef94100 con 0x562f6ef885c0 2024-04-16T17:12:44.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.225+0000 7f963dbc7740 10 client.0 _ll_register_callbacks cb 0x562f6ee751e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:44.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.226+0000 7f9638e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef911a0 0x562f6ef8ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.226+0000 7f9633400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef885c0 0x562f6ef8e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.226+0000 7f9638e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef911a0 0x562f6ef8ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54960/0 (socket says 172.21.15.18:54960) 2024-04-16T17:12:44.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.226+0000 7f9638e00640 1 -- 172.21.15.18:0/507267476 learned_addr learned my addr 172.21.15.18:0/507267476 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.226+0000 7f9633400640 1 -- 172.21.15.18:0/507267476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef911a0 msgr2=0x562f6ef8ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.226+0000 7f9633400640 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef911a0 0x562f6ef8ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.226+0000 7f9633400640 1 -- 172.21.15.18:0/507267476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef87ca0 msgr2=0x562f6ef8e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.226+0000 7f9633400640 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef87ca0 0x562f6ef8e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.227+0000 7f9633400640 1 -- 172.21.15.18:0/507267476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6ecc77e0 con 0x562f6ef885c0 2024-04-16T17:12:44.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.227+0000 7f9633400640 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef885c0 0x562f6ef8e6f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f962c0997b0 tx=0x7f962c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6fc54c46454cd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.227+0000 7f961e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:44.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.228+0000 7f961e000640 1 -- 172.21.15.18:0/507267476 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0b5030 con 0x562f6ef885c0 2024-04-16T17:12:44.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.227+0000 7f9638e00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f96180a3ea0 0x7f96180a6360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9624005c40 tx=0x7f962402df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.229+0000 7f963dbc7740 1 -- 172.21.15.18:0/507267476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962c0a1190 con 0x562f6ef885c0 2024-04-16T17:12:44.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.229+0000 7f961e000640 1 -- 172.21.15.18:0/507267476 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f962c0b6270 con 0x562f6ef885c0 2024-04-16T17:12:44.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.230+0000 7f961e000640 1 -- 172.21.15.18:0/507267476 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f962c0a0070 con 0x562f6ef885c0 2024-04-16T17:12:44.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.230+0000 7f963dbc7740 10 client.27715 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:44.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.230+0000 7f963dbc7740 1 -- 172.21.15.18:0/507267476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f6ed27f70 con 0x562f6ef885c0 2024-04-16T17:12:44.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.230+0000 7f961e000640 1 -- 172.21.15.18:0/507267476 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c0a1190 con 0x562f6ef885c0 2024-04-16T17:12:44.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.230+0000 7f961e000640 1 -- 172.21.15.18:0/507267476 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f962c0cece0 con 0x562f6ef885c0 2024-04-16T17:12:44.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.231+0000 7f963dbc7740 10 client.27715 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:44.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.231+0000 7f963dbc7740 10 client.27715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:44.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.231+0000 7f963dbc7740 20 client.27715 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:44.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.231+0000 7f963dbc7740 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f6efa1160 0x562f6efa3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.231+0000 7f963dbc7740 4 client.27715 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:44.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.231+0000 7f963dbc7740 1 -- 172.21.15.18:0/507267476 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f6eefa6d0 con 0x562f6efa1160 2024-04-16T17:12:44.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.235+0000 7f9633e00640 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f6efa1160 0x562f6efa3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.237+0000 7f9633e00640 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f6efa1160 0x562f6efa3550 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3533292a8f661c server_cookie=a403cc04da81fc3a in_seq=0 out_seq=0 2024-04-16T17:12:44.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.237+0000 7f961e000640 10 client.27715 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed20c00640 1 -- 172.21.15.18:0/4135886279 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568841ac010 con 0x5568842c0120 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed20c00640 10 client.27800 handle_command_reply: tid=0 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 client.27800 shutdown 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 -- 172.21.15.18:0/4135886279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 msgr2=0x5568842437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x5568842437f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fed30011d70 tx=0x7fed30009c10 comp rx=0 tx=0).stop 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 -- 172.21.15.18:0/4135886279 shutdown_connections 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568842c0120 0x5568842c2540 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556884249000 0x5568842437f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568842407f0 0x5568842432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 --2- 172.21.15.18:0/4135886279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55688423fa80 0x556884242d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed40d92740 1 -- 172.21.15.18:0/4135886279 >> 172.21.15.18:0/4135886279 conn(0x5568841da5d0 msgr2=0x556884238f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.307+0000 7fed20c00640 0 client.27800 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.307+0000 7f961e000640 1 -- 172.21.15.18:0/507267476 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f6eefa6d0 con 0x562f6efa1160 2024-04-16T17:12:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.307+0000 7f961e000640 10 client.27715 handle_command_reply: tid=0 2024-04-16T17:12:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.308+0000 7fed40d92740 1 -- 172.21.15.18:0/4135886279 shutdown_connections 2024-04-16T17:12:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 client.27715 shutdown 2024-04-16T17:12:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.308+0000 7fed40d92740 1 -- 172.21.15.18:0/4135886279 wait complete. 2024-04-16T17:12:44.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 -- 172.21.15.18:0/507267476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef885c0 msgr2=0x562f6ef8e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef885c0 0x562f6ef8e6f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f962c0997b0 tx=0x7f962c000f40 comp rx=0 tx=0).stop 2024-04-16T17:12:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.308+0000 7fed40d92740 20 client.27800 trim_cache size 0 max 16384 2024-04-16T17:12:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 -- 172.21.15.18:0/507267476 shutdown_connections 2024-04-16T17:12:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f6efa1160 0x562f6efa3550 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef911a0 0x562f6ef8ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef885c0 0x562f6ef8e6f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 --2- 172.21.15.18:0/507267476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef87ca0 0x562f6ef8e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f963dbc7740 1 -- 172.21.15.18:0/507267476 >> 172.21.15.18:0/507267476 conn(0x562f6ef823d0 msgr2=0x562f6ef84120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.308+0000 7f961e000640 0 client.27715 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:12:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:12:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:12:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:12:44.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.309+0000 7f963dbc7740 1 -- 172.21.15.18:0/507267476 shutdown_connections 2024-04-16T17:12:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 -- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed1c0a3d00 msgr2=0x7fed1c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed1c0a3d00 0x7fed1c0a61c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed28036040 tx=0x7fed280095f0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 -- 172.21.15.18:0/699922022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed38103900 msgr2=0x7fed381a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed38103900 0x7fed381a0b70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fed2400b410 tx=0x7fed2400b8d0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.309+0000 7f963dbc7740 1 -- 172.21.15.18:0/507267476 wait complete. 2024-04-16T17:12:44.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.309+0000 7f963dbc7740 20 client.27715 trim_cache size 0 max 16384 2024-04-16T17:12:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 -- 172.21.15.18:0/699922022 shutdown_connections 2024-04-16T17:12:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed1c0a3d00 0x7fed1c0a61c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed3810c400 0x7fed3819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed38103900 0x7fed381a0b70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 --2- 172.21.15.18:0/699922022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed38102fe0 0x7fed381a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.309+0000 7fed3ca00640 1 -- 172.21.15.18:0/699922022 >> 172.21.15.18:0/699922022 conn(0x7fed380fc7e0 msgr2=0x7fed38100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.311+0000 7fed3ca00640 1 -- 172.21.15.18:0/699922022 shutdown_connections 2024-04-16T17:12:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:44.311+0000 7fed3ca00640 1 -- 172.21.15.18:0/699922022 wait complete. 2024-04-16T17:12:44.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f963dbc7740 1 Processor -- start 2024-04-16T17:12:44.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f963dbc7740 1 -- start start 2024-04-16T17:12:44.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6ef908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f963dbc7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f96341c0d40 con 0x562f6ef911a0 2024-04-16T17:12:44.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f963dbc7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f96341be610 con 0x562f6ef885c0 2024-04-16T17:12:44.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f963dbc7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96341bc480 con 0x562f6ef87ca0 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f9633e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6ef908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f9633400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f9633400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54968/0 (socket says 172.21.15.18:54968) 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.334+0000 7f9633400640 1 -- 172.21.15.18:0/2428440314 learned_addr learned my addr 172.21.15.18:0/2428440314 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.335+0000 7f9638e00640 1 --2- 172.21.15.18:0/2428440314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.335+0000 7f9633400640 1 -- 172.21.15.18:0/2428440314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 msgr2=0x562f6ef908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.335+0000 7f9633400640 1 --2- 172.21.15.18:0/2428440314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6ef908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.335+0000 7f9633400640 1 -- 172.21.15.18:0/2428440314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 msgr2=0x562f6efa6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.335+0000 7f9633400640 1 --2- 172.21.15.18:0/2428440314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.335+0000 7f9633400640 1 -- 172.21.15.18:0/2428440314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6ebbf170 con 0x562f6ef885c0 2024-04-16T17:12:44.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.335+0000 7f9633400640 1 --2- 172.21.15.18:0/2428440314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa6570 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f962c00cba0 tx=0x7f962c09c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed814dbfd22fce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.335+0000 7f9628c00640 1 -- 172.21.15.18:0/2428440314 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0b5070 con 0x562f6ef885c0 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f9628c00640 1 -- 172.21.15.18:0/2428440314 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c0a1190 con 0x562f6ef885c0 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f9628c00640 1 -- 172.21.15.18:0/2428440314 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c010e60 con 0x562f6ef885c0 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f963dbc7740 1 -- 172.21.15.18:0/2428440314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 msgr2=0x562f6efa6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2428440314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa6570 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f962c00cba0 tx=0x7f962c09c740 comp rx=0 tx=0).stop 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f963dbc7740 1 -- 172.21.15.18:0/2428440314 shutdown_connections 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2428440314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2428440314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa6570 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2428440314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6ef908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.336+0000 7f963dbc7740 1 -- 172.21.15.18:0/2428440314 >> 172.21.15.18:0/2428440314 conn(0x562f6ef823d0 msgr2=0x562f6ef840b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.337+0000 7f963dbc7740 1 -- 172.21.15.18:0/2428440314 shutdown_connections 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.337+0000 7f963dbc7740 1 -- 172.21.15.18:0/2428440314 wait complete. 2024-04-16T17:12:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.337+0000 7f963dbc7740 1 -- start start 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.337+0000 7f963dbc7740 1 Processor -- start 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6efa4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f963dbc7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f963dbc7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f96341bcad0 con 0x562f6ef911a0 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f963dbc7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f96341bdf40 con 0x562f6ef885c0 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f963dbc7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96341bd1a0 con 0x562f6ef87ca0 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f963dbc7740 10 client.0 _ll_register_callbacks cb 0x7f962c0a5a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f9633e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6efa4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f9633e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6efa4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37040/0 (socket says 172.21.15.18:37040) 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f9633e00640 1 -- 172.21.15.18:0/2915736751 learned_addr learned my addr 172.21.15.18:0/2915736751 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f9638e00640 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.338+0000 7f9633400640 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.339+0000 7f9638e00640 1 -- 172.21.15.18:0/2915736751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 msgr2=0x562f6efa4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.339+0000 7f9638e00640 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6efa4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.339+0000 7f9638e00640 1 -- 172.21.15.18:0/2915736751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 msgr2=0x562f6efa4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.339+0000 7f9638e00640 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.339+0000 7f9638e00640 1 -- 172.21.15.18:0/2915736751 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6f006590 con 0x562f6ef911a0 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.339+0000 7f9638e00640 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa4cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9624009850 tx=0x7f962402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a645076562cf703 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.339+0000 7f961e000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.339+0000 7f961e000640 1 -- 172.21.15.18:0/2915736751 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962404f020 con 0x562f6ef911a0 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.340+0000 7f961e000640 1 -- 172.21.15.18:0/2915736751 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f962403a040 con 0x562f6ef911a0 2024-04-16T17:12:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.340+0000 7f963dbc7740 1 -- 172.21.15.18:0/2915736751 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962c0a1190 con 0x562f6ef911a0 2024-04-16T17:12:44.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.340+0000 7f961e000640 1 -- 172.21.15.18:0/2915736751 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f962405a050 con 0x562f6ef911a0 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.340+0000 7f963dbc7740 10 client.17535 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.340+0000 7f963dbc7740 1 -- 172.21.15.18:0/2915736751 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f6f0078b0 con 0x562f6ef911a0 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.340+0000 7f961e000640 1 -- 172.21.15.18:0/2915736751 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c0a1190 con 0x562f6ef911a0 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.341+0000 7f961e000640 1 -- 172.21.15.18:0/2915736751 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f962404a040 con 0x562f6ef911a0 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.341+0000 7f963dbc7740 10 client.17535 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.341+0000 7f963dbc7740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.341+0000 7f963dbc7740 20 client.17535 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.341+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f6f008080 0x562f6f00a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.341+0000 7f963dbc7740 4 client.17535 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.341+0000 7f963dbc7740 1 -- 172.21.15.18:0/2915736751 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562f6eef4540 con 0x562f6f008080 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.341+0000 7f9633e00640 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f6f008080 0x562f6f00a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.342+0000 7f9633e00640 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f6f008080 0x562f6f00a4a0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c6d67b893bdf53 server_cookie=a64900aeec767758 in_seq=0 out_seq=0 2024-04-16T17:12:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.342+0000 7f961e000640 10 client.17535 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.343+0000 7f961e000640 1 -- 172.21.15.18:0/2915736751 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562f6eef4540 con 0x562f6f008080 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.343+0000 7f961e000640 10 client.17535 handle_command_reply: tid=0 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.343+0000 7f963dbc7740 1 client.17535 shutdown 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.343+0000 7f963dbc7740 1 -- 172.21.15.18:0/2915736751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 msgr2=0x562f6efa4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.343+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa4cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9624009850 tx=0x7f962402bc30 comp rx=0 tx=0).stop 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 1 -- 172.21.15.18:0/2915736751 shutdown_connections 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f6f008080 0x562f6f00a4a0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f6ef911a0 0x562f6efa4cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f6ef885c0 0x562f6efa4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 1 --2- 172.21.15.18:0/2915736751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6ef87ca0 0x562f6efa4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 1 -- 172.21.15.18:0/2915736751 >> 172.21.15.18:0/2915736751 conn(0x562f6ef823d0 msgr2=0x562f6ef80ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f961e000640 0 client.17535 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 1 -- 172.21.15.18:0/2915736751 shutdown_connections 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 1 -- 172.21.15.18:0/2915736751 wait complete. 2024-04-16T17:12:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.344+0000 7f963dbc7740 20 client.17535 trim_cache size 0 max 16384 2024-04-16T17:12:44.346 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:12:44.346 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:12:44.346 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "c57b0c1f-8660-4966-b9fe-bc6a7c3f0349", 2024-04-16T17:12:44.346 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:12:44.346 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f96180a3ea0 msgr2=0x7f96180a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f96180a3ea0 0x7f96180a6360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9624005c40 tx=0x7f962402df90 comp rx=0 tx=0).stop 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9634110bc0 msgr2=0x7f96341a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9634110bc0 0x7f96341a7b20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f962c00ebc0 tx=0x7f962c00c6a0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 shutdown_connections 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f96180a3ea0 0x7f96180a6360 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f963411b7e0 0x7f96341abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9634110bc0 0x7f96341a7b20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 --2- 172.21.15.18:0/2676765117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9634110750 0x7f96341a75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 >> 172.21.15.18:0/2676765117 conn(0x7f963406d0a0 msgr2=0x7f963411a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 shutdown_connections 2024-04-16T17:12:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.345+0000 7f963ae00640 1 -- 172.21.15.18:0/2676765117 wait complete. 2024-04-16T17:12:44.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:12:44.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:12:44.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.594+0000 7f8a97e00640 1 -- 172.21.15.18:0/3532960888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a980743a0 msgr2=0x7f8a98074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.594+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3532960888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a980743a0 0x7f8a98074780 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009900 tx=0x7f8a8802bcf0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.594+0000 7f8a97e00640 1 -- 172.21.15.18:0/3532960888 shutdown_connections 2024-04-16T17:12:44.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.594+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3532960888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a98112e10 0x7f8a98115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.594+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3532960888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a98074cc0 0x7f8a981128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.594+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3532960888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a980743a0 0x7f8a98074780 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.594+0000 7f8a97e00640 1 -- 172.21.15.18:0/3532960888 >> 172.21.15.18:0/3532960888 conn(0x7f8a980fd950 msgr2=0x7f8a980ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.595+0000 7f8a97e00640 1 -- 172.21.15.18:0/3532960888 shutdown_connections 2024-04-16T17:12:44.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.595+0000 7f8a97e00640 1 -- 172.21.15.18:0/3532960888 wait complete. 2024-04-16T17:12:44.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.595+0000 7f8a97e00640 1 Processor -- start 2024-04-16T17:12:44.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a97e00640 1 -- start start 2024-04-16T17:12:44.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980743a0 0x7f8a9819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a98074cc0 0x7f8a9819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a98112e10 0x7f8a981a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a97e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981188a0 con 0x7f8a98112e10 2024-04-16T17:12:44.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a97e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98118720 con 0x7f8a98074cc0 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a97e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98118a20 con 0x7f8a980743a0 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980743a0 0x7f8a9819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980743a0 0x7f8a9819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37054/0 (socket says 172.21.15.18:37054) 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a96a00640 1 -- 172.21.15.18:0/3844969499 learned_addr learned my addr 172.21.15.18:0/3844969499 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a96000640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a98074cc0 0x7f8a9819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.596+0000 7f8a97400640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a98112e10 0x7f8a981a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.597+0000 7f8a96a00640 1 -- 172.21.15.18:0/3844969499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a98074cc0 msgr2=0x7f8a9819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.597+0000 7f8a96a00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a98074cc0 0x7f8a9819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.597+0000 7f8a96a00640 1 -- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a98112e10 msgr2=0x7f8a981a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.597+0000 7f8a96a00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a98112e10 0x7f8a981a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.597+0000 7f8a96a00640 1 -- 172.21.15.18:0/3844969499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a981a3db0 con 0x7f8a980743a0 2024-04-16T17:12:44.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.597+0000 7f8a96000640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a98074cc0 0x7f8a9819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.597+0000 7f8a96a00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980743a0 0x7f8a9819edc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8a88000c00 tx=0x7f8a88004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.598+0000 7f8a8f400640 1 -- 172.21.15.18:0/3844969499 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88042020 con 0x7f8a980743a0 2024-04-16T17:12:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.598+0000 7f8a8f400640 1 -- 172.21.15.18:0/3844969499 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a880054e0 con 0x7f8a980743a0 2024-04-16T17:12:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.598+0000 7f8a8f400640 1 -- 172.21.15.18:0/3844969499 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88003be0 con 0x7f8a980743a0 2024-04-16T17:12:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.598+0000 7f8a97e00640 1 -- 172.21.15.18:0/3844969499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a981a4040 con 0x7f8a980743a0 2024-04-16T17:12:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.598+0000 7f8a97e00640 1 -- 172.21.15.18:0/3844969499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a981ab930 con 0x7f8a980743a0 2024-04-16T17:12:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.602+0000 7f8aa0991740 1 Processor -- start 2024-04-16T17:12:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.602+0000 7f8aa0991740 1 -- start start 2024-04-16T17:12:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8a8f400640 1 -- 172.21.15.18:0/3844969499 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a88045020 con 0x7f8a980743a0 2024-04-16T17:12:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ecf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233ed8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ed8890 0x563233edac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8aa0991740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563233e35120 con 0x563233ecfcb0 2024-04-16T17:12:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8aa0991740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563233ddb340 con 0x563233ecf390 2024-04-16T17:12:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8aa0991740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563233e310e0 con 0x563233ed8890 2024-04-16T17:12:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8a97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ecf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8a97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ecf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55006/0 (socket says 172.21.15.18:55006) 2024-04-16T17:12:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8a97400640 1 -- 172.21.15.18:0/1869457417 learned_addr learned my addr 172.21.15.18:0/1869457417 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.603+0000 7f8a96a00640 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233ed8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.604+0000 7f8a97400640 1 -- 172.21.15.18:0/1869457417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ed8890 msgr2=0x563233edac80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:12:44.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.604+0000 7f8a97400640 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ed8890 0x563233edac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.604+0000 7f8a97400640 1 -- 172.21.15.18:0/1869457417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 msgr2=0x563233ed8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.604+0000 7f8a97400640 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233ed8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.604+0000 7f8a97400640 1 -- 172.21.15.18:0/1869457417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563233c117d0 con 0x563233ecf390 2024-04-16T17:12:44.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.604+0000 7f8a96a00640 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233ed8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.604+0000 7f8a97400640 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ecf770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009f30 tx=0x7f8a9000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d19f7bb9346ddca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.605+0000 7f8a83e00640 1 -- 172.21.15.18:0/1869457417 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9001a070 con 0x563233ecf390 2024-04-16T17:12:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.605+0000 7f8a83e00640 1 -- 172.21.15.18:0/1869457417 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a900158f0 con 0x563233ecf390 2024-04-16T17:12:44.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.605+0000 7f8a83e00640 1 -- 172.21.15.18:0/1869457417 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000a9a0 con 0x563233ecf390 2024-04-16T17:12:44.606 INFO:tasks.workunit.client.0.smithi018.stdout: 1 308892 3.93 MB/sec execute 297 sec latency 91.199 ms 2024-04-16T17:12:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.605+0000 7f8a8f400640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8a7c0a3e70 0x7f8a7c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 -- 172.21.15.18:0/1869457417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 msgr2=0x563233ecf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ecf770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009f30 tx=0x7f8a9000c8b0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 -- 172.21.15.18:0/1869457417 shutdown_connections 2024-04-16T17:12:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ed8890 0x563233edac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233ed8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 --2- 172.21.15.18:0/1869457417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ecf770 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 -- 172.21.15.18:0/1869457417 >> 172.21.15.18:0/1869457417 conn(0x563233ec9ac0 msgr2=0x563233ec9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8a8f400640 1 -- 172.21.15.18:0/3844969499 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f8a880f2830 con 0x7f8a980743a0 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8a96a00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8a7c0a3e70 0x7f8a7c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 -- 172.21.15.18:0/1869457417 shutdown_connections 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.606+0000 7f8aa0991740 1 -- 172.21.15.18:0/1869457417 wait complete. 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.607+0000 7f8a96a00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8a7c0a3e70 0x7f8a7c0a6330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x563233ecee60 tx=0x7f8a880ba040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.607+0000 7f8aa0991740 1 -- start start 2024-04-16T17:12:44.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.607+0000 7f8aa0991740 1 Processor -- start 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ed5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecfcb0 0x563233ed5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ed8890 0x563233ed63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8aa0991740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563233edb680 con 0x563233ed8890 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8aa0991740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563233dee710 con 0x563233ecf390 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8aa0991740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563233edb800 con 0x563233ecfcb0 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8aa0991740 10 client.0 _ll_register_callbacks cb 0x563233de9af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ed5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ed5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55012/0 (socket says 172.21.15.18:55012) 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 -- 172.21.15.18:0/2842635012 learned_addr learned my addr 172.21.15.18:0/2842635012 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ed8890 0x563233ed63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a96000640 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecfcb0 0x563233ed5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 -- 172.21.15.18:0/2842635012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecfcb0 msgr2=0x563233ed5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecfcb0 0x563233ed5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 -- 172.21.15.18:0/2842635012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ed8890 msgr2=0x563233ed63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ed8890 0x563233ed63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.608+0000 7f8a97400640 1 -- 172.21.15.18:0/2842635012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563233b473f0 con 0x563233ecf390 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.609+0000 7f8a97400640 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ed5920 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009d40 tx=0x7f8a9000a4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfcdbab4f8060d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.609+0000 7f8a80c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.609+0000 7f8a80c00640 1 -- 172.21.15.18:0/2842635012 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90028060 con 0x563233ecf390 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.609+0000 7f8a80c00640 1 -- 172.21.15.18:0/2842635012 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f8a90015640 con 0x563233ecf390 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.609+0000 7f8aa0991740 1 -- 172.21.15.18:0/2842635012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a900158f0 con 0x563233ecf390 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.609+0000 7f8a80c00640 1 -- 172.21.15.18:0/2842635012 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a9001a040 con 0x563233ecf390 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8aa0991740 10 client.27827 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:44.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8aa0991740 1 -- 172.21.15.18:0/2842635012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563233b070f0 con 0x563233ecf390 2024-04-16T17:12:44.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8a80c00640 1 -- 172.21.15.18:0/2842635012 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a900158f0 con 0x563233ecf390 2024-04-16T17:12:44.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8a80c00640 1 -- 172.21.15.18:0/2842635012 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8a90020070 con 0x563233ecf390 2024-04-16T17:12:44.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8aa0991740 10 client.27827 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:44.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8aa0991740 10 client.27827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:44.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8aa0991740 20 client.27827 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:44.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8aa0991740 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563233ee8c40 0x563233eeb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8aa0991740 4 client.27827 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:44.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.610+0000 7f8aa0991740 1 -- 172.21.15.18:0/2842635012 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563233e42b40 con 0x563233ee8c40 2024-04-16T17:12:44.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.611+0000 7f8a96000640 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563233ee8c40 0x563233eeb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.611+0000 7f8a96000640 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563233ee8c40 0x563233eeb030 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376b6127f7c249b5 server_cookie=ddf4392621daacca in_seq=0 out_seq=0 2024-04-16T17:12:44.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.611+0000 7f8a80c00640 10 client.27827 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8a80c00640 1 -- 172.21.15.18:0/2842635012 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563233e42b40 con 0x563233ee8c40 2024-04-16T17:12:44.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8a80c00640 10 client.27827 handle_command_reply: tid=0 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 client.27827 shutdown 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 -- 172.21.15.18:0/2842635012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 msgr2=0x563233ed5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ed5920 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009d40 tx=0x7f8a9000a4a0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 -- 172.21.15.18:0/2842635012 shutdown_connections 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563233ee8c40 0x563233eeb030 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ed8890 0x563233ed63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecfcb0 0x563233ed5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 --2- 172.21.15.18:0/2842635012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233ed5920 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 -- 172.21.15.18:0/2842635012 >> 172.21.15.18:0/2842635012 conn(0x563233ec9ac0 msgr2=0x563233ecb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8a80c00640 0 client.27827 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 -- 172.21.15.18:0/2842635012 shutdown_connections 2024-04-16T17:12:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.612+0000 7f8aa0991740 1 -- 172.21.15.18:0/2842635012 wait complete. 2024-04-16T17:12:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.613+0000 7f8aa0991740 20 client.27827 trim_cache size 0 max 16384 2024-04-16T17:12:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.632+0000 7f8aa0991740 1 Processor -- start 2024-04-16T17:12:44.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.632+0000 7f8aa0991740 1 -- start start 2024-04-16T17:12:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.632+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ec9ac0 0x563233ed8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.632+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecf390 0x563233ed4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.632+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecfcb0 0x563233eee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.632+0000 7f8aa0991740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981b82e0 con 0x563233ec9ac0 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.632+0000 7f8aa0991740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981b5bb0 con 0x563233ecfcb0 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.632+0000 7f8aa0991740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981b3a20 con 0x563233ecf390 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a96000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecf390 0x563233ed4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a96000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecf390 0x563233ed4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37084/0 (socket says 172.21.15.18:37084) 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a96000640 1 -- 172.21.15.18:0/3942575738 learned_addr learned my addr 172.21.15.18:0/3942575738 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a97400640 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ec9ac0 0x563233ed8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a96000640 1 -- 172.21.15.18:0/3942575738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecfcb0 msgr2=0x563233eee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a97400640 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecfcb0 0x563233eee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a96000640 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecfcb0 0x563233eee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a96000640 1 -- 172.21.15.18:0/3942575738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ec9ac0 msgr2=0x563233ed8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a96000640 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ec9ac0 0x563233ed8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a96000640 1 -- 172.21.15.18:0/3942575738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563233ecfae0 con 0x563233ecf390 2024-04-16T17:12:44.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.633+0000 7f8a97400640 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecfcb0 0x563233eee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:44.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.634+0000 7f8a96000640 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecf390 0x563233ed4860 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x563233ed5180 tx=0x7f8a84005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb9470ab2e93636 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.634+0000 7f8a83400640 1 -- 172.21.15.18:0/3942575738 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84014a60 con 0x563233ecf390 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.634+0000 7f8a83400640 1 -- 172.21.15.18:0/3942575738 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a84014c00 con 0x563233ecf390 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.634+0000 7f8a83400640 1 -- 172.21.15.18:0/3942575738 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84005020 con 0x563233ecf390 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 -- 172.21.15.18:0/3942575738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecf390 msgr2=0x563233ed4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecf390 0x563233ed4860 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x563233ed5180 tx=0x7f8a84005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 -- 172.21.15.18:0/3942575738 shutdown_connections 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecfcb0 0x563233eee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ecf390 0x563233ed4860 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3942575738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ec9ac0 0x563233ed8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 -- 172.21.15.18:0/3942575738 >> 172.21.15.18:0/3942575738 conn(0x563233ed12f0 msgr2=0x563233ed6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 -- 172.21.15.18:0/3942575738 shutdown_connections 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 -- 172.21.15.18:0/3942575738 wait complete. 2024-04-16T17:12:44.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.635+0000 7f8aa0991740 1 -- start start 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8aa0991740 1 Processor -- start 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ec9ac0 0x563233eec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233eec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8aa0991740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233f4de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8aa0991740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981b4070 con 0x563233ecfcb0 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8aa0991740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981b54e0 con 0x563233ecf390 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8aa0991740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981b4740 con 0x563233ec9ac0 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8aa0991740 10 client.0 _ll_register_callbacks cb 0x7f8a90018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8a96000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233eec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8a96000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233eec6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55030/0 (socket says 172.21.15.18:55030) 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8a97400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ec9ac0 0x563233eec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.636+0000 7f8a96000640 1 -- 172.21.15.18:0/3699141602 learned_addr learned my addr 172.21.15.18:0/3699141602 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a97400640 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233f4de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a96000640 1 -- 172.21.15.18:0/3699141602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ec9ac0 msgr2=0x563233eec180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a96000640 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ec9ac0 0x563233eec180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a96000640 1 -- 172.21.15.18:0/3699141602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 msgr2=0x563233f4de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a96000640 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233f4de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a96000640 1 -- 172.21.15.18:0/3699141602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563233f4e360 con 0x563233ecf390 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a96000640 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233eec6c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8a8401b690 tx=0x7f8a8401b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d01206d4deb85e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a80c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a80c00640 1 -- 172.21.15.18:0/3699141602 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8401b6f0 con 0x563233ecf390 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8aa0991740 1 -- 172.21.15.18:0/3699141602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a84014c00 con 0x563233ecf390 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.637+0000 7f8a80c00640 1 -- 172.21.15.18:0/3699141602 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f8a8401b890 con 0x563233ecf390 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8a80c00640 1 -- 172.21.15.18:0/3699141602 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a84004020 con 0x563233ecf390 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8aa0991740 10 client.27830 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8aa0991740 1 -- 172.21.15.18:0/3699141602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563233f4f550 con 0x563233ecf390 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8a80c00640 1 -- 172.21.15.18:0/3699141602 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a84014c00 con 0x563233ecf390 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8a80c00640 1 -- 172.21.15.18:0/3699141602 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8a8402c020 con 0x563233ecf390 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8aa0991740 10 client.27830 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8aa0991740 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:44.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8aa0991740 20 client.27830 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:44.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563233ed8890 0x563233f51ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:44.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8aa0991740 4 client.27830 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:12:44.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.638+0000 7f8aa0991740 1 -- 172.21.15.18:0/3699141602 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563233e3c9b0 con 0x563233ed8890 2024-04-16T17:12:44.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.639+0000 7f8a97400640 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563233ed8890 0x563233f51ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.639+0000 7f8a80c00640 10 client.27830 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.639+0000 7f8a97400640 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563233ed8890 0x563233f51ba0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab57152317273ff server_cookie=d03bd99cd1aba478 in_seq=0 out_seq=0 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.639+0000 7f8a80c00640 1 -- 172.21.15.18:0/3699141602 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563233e3c9b0 con 0x563233ed8890 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.639+0000 7f8a80c00640 10 client.27830 handle_command_reply: tid=0 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.639+0000 7f8aa0991740 1 client.27830 shutdown 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.639+0000 7f8aa0991740 1 -- 172.21.15.18:0/3699141602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 msgr2=0x563233eec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.639+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233eec6c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8a8401b690 tx=0x7f8a8401b330 comp rx=0 tx=0).stop 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 1 -- 172.21.15.18:0/3699141602 shutdown_connections 2024-04-16T17:12:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563233ed8890 0x563233f51ba0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563233ecfcb0 0x563233f4de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563233ecf390 0x563233eec6c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 1 --2- 172.21.15.18:0/3699141602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563233ec9ac0 0x563233eec180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 1 -- 172.21.15.18:0/3699141602 >> 172.21.15.18:0/3699141602 conn(0x563233ed12f0 msgr2=0x563233ed58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8a80c00640 0 client.27830 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 1 -- 172.21.15.18:0/3699141602 shutdown_connections 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 1 -- 172.21.15.18:0/3699141602 wait complete. 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8aa0991740 20 client.27830 trim_cache size 0 max 16384 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:12:44.643 INFO:teuthology.orchestra.run.smithi018.stdout: "c57b0c1f-8660-4966-b9fe-bc6a7c3f0349": { 2024-04-16T17:12:44.644 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:12:44.644 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "c57b0c1f-8660-4966-b9fe-bc6a7c3f0349", 2024-04-16T17:12:44.644 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:12:44.644 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:12:44.644 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:12:44.644 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:12:44.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.640+0000 7f8a97e00640 1 -- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8a7c0a3e70 msgr2=0x7f8a7c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8a7c0a3e70 0x7f8a7c0a6330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x563233ecee60 tx=0x7f8a880ba040 comp rx=0 tx=0).stop 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 -- 172.21.15.18:0/3844969499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980743a0 msgr2=0x7f8a9819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980743a0 0x7f8a9819edc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8a88000c00 tx=0x7f8a88004360 comp rx=0 tx=0).stop 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 -- 172.21.15.18:0/3844969499 shutdown_connections 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8a7c0a3e70 0x7f8a7c0a6330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a98112e10 0x7f8a981a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a98074cc0 0x7f8a9819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 --2- 172.21.15.18:0/3844969499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980743a0 0x7f8a9819edc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 -- 172.21.15.18:0/3844969499 >> 172.21.15.18:0/3844969499 conn(0x7f8a980fd950 msgr2=0x7f8a980fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 -- 172.21.15.18:0/3844969499 shutdown_connections 2024-04-16T17:12:44.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:12:44.641+0000 7f8a97e00640 1 -- 172.21.15.18:0/3844969499 wait complete. 2024-04-16T17:12:44.662 INFO:tasks.cephfs.filesystem:scrub status for tag:c57b0c1f-8660-4966-b9fe-bc6a7c3f0349 - {'path': '/', 'tag': 'c57b0c1f-8660-4966-b9fe-bc6a7c3f0349', 'options': 'recursive,force'} 2024-04-16T17:12:45.606 INFO:tasks.workunit.client.0.smithi018.stdout: 1 309163 3.91 MB/sec execute 298 sec latency 75.329 ms 2024-04-16T17:12:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:45 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:12:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:45 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:45 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:12:46.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:45 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:46.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:45 smithi043 ceph-mon[36901]: pgmap v249: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:12:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:45 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:12:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:45 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:46.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:45 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:12:46.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:45 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:46.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:45 smithi022 ceph-mon[36023]: pgmap v249: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:12:46.267 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:45 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:12:46.267 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:45 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:46.268 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:45 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:12:46.268 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:45 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:46.268 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:45 smithi018 ceph-mon[31776]: pgmap v249: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-16T17:12:46.606 INFO:tasks.workunit.client.0.smithi018.stdout: 1 310049 3.92 MB/sec execute 299 sec latency 15.864 ms 2024-04-16T17:12:47.606 INFO:tasks.workunit.client.0.smithi018.stdout: 1 311222 3.91 MB/sec execute 300 sec latency 12.844 ms 2024-04-16T17:12:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:47 smithi043 ceph-mon[36901]: pgmap v250: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-04-16T17:12:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:47 smithi022 ceph-mon[36023]: pgmap v250: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-04-16T17:12:48.606 INFO:tasks.workunit.client.0.smithi018.stdout: 1 312251 3.93 MB/sec execute 301 sec latency 48.336 ms 2024-04-16T17:12:49.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:47 smithi018 ceph-mon[31776]: pgmap v250: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-04-16T17:12:49.606 INFO:tasks.workunit.client.0.smithi018.stdout: 1 312455 3.92 MB/sec execute 302 sec latency 194.842 ms 2024-04-16T17:12:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:49 smithi043 ceph-mon[36901]: pgmap v251: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-16T17:12:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:49 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:12:50.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:49 smithi022 ceph-mon[36023]: pgmap v251: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-16T17:12:50.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:49 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:12:50.612 INFO:tasks.workunit.client.0.smithi018.stdout: 1 312694 3.91 MB/sec execute 303 sec latency 85.459 ms 2024-04-16T17:12:51.229 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:49 smithi018 ceph-mon[31776]: pgmap v251: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-16T17:12:51.229 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:49 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:12:51.612 INFO:tasks.workunit.client.0.smithi018.stdout: 1 313590 3.92 MB/sec execute 304 sec latency 44.756 ms 2024-04-16T17:12:52.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:51 smithi043 ceph-mon[36901]: pgmap v252: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-04-16T17:12:52.023 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:51 smithi018 ceph-mon[31776]: pgmap v252: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-04-16T17:12:52.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:51 smithi022 ceph-mon[36023]: pgmap v252: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-04-16T17:12:52.612 INFO:tasks.workunit.client.0.smithi018.stdout: 1 314709 3.91 MB/sec execute 305 sec latency 45.045 ms 2024-04-16T17:12:53.612 INFO:tasks.workunit.client.0.smithi018.stdout: 1 315779 3.93 MB/sec execute 306 sec latency 67.051 ms 2024-04-16T17:12:54.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:53 smithi043 ceph-mon[36901]: pgmap v253: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-04-16T17:12:54.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:53 smithi022 ceph-mon[36023]: pgmap v253: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-04-16T17:12:54.355 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:53 smithi018 ceph-mon[31776]: pgmap v253: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-04-16T17:12:54.612 INFO:tasks.workunit.client.0.smithi018.stdout: 1 315981 3.92 MB/sec execute 307 sec latency 268.119 ms 2024-04-16T17:12:55.612 INFO:tasks.workunit.client.0.smithi018.stdout: 1 316239 3.91 MB/sec execute 308 sec latency 55.525 ms 2024-04-16T17:12:55.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:55 smithi018 ceph-mon[31776]: pgmap v254: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-04-16T17:12:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:55 smithi043 ceph-mon[36901]: pgmap v254: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-04-16T17:12:56.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:55 smithi022 ceph-mon[36023]: pgmap v254: 129 pgs: 129 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-04-16T17:12:56.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 317140 3.91 MB/sec execute 309 sec latency 19.848 ms 2024-04-16T17:12:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.455+0000 7f987ae00640 1 -- 172.21.15.18:0/2098413509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98740743b0 msgr2=0x7f9874074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.455+0000 7f987ae00640 1 --2- 172.21.15.18:0/2098413509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98740743b0 0x7f9874074790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9868009900 tx=0x7f986802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.456+0000 7f987ae00640 1 -- 172.21.15.18:0/2098413509 shutdown_connections 2024-04-16T17:12:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.456+0000 7f987ae00640 1 --2- 172.21.15.18:0/2098413509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f987410e6d0 0x7f9874110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.456+0000 7f987ae00640 1 --2- 172.21.15.18:0/2098413509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9874074cd0 0x7f987410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.456+0000 7f987ae00640 1 --2- 172.21.15.18:0/2098413509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98740743b0 0x7f9874074790 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.456+0000 7f987ae00640 1 -- 172.21.15.18:0/2098413509 >> 172.21.15.18:0/2098413509 conn(0x7f98740fd940 msgr2=0x7f98740ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.456+0000 7f987ae00640 1 -- 172.21.15.18:0/2098413509 shutdown_connections 2024-04-16T17:12:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.456+0000 7f987ae00640 1 -- 172.21.15.18:0/2098413509 wait complete. 2024-04-16T17:12:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.457+0000 7f987ae00640 1 Processor -- start 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.457+0000 7f987ae00640 1 -- start start 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f987ae00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9874074cd0 0x7f98741a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f987ae00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987410e6d0 0x7f98741a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f987ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f987419a950 0x7f987419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f987ae00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9874114430 con 0x7f987410e6d0 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f987ae00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98741142b0 con 0x7f9874074cd0 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f987ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98741145b0 con 0x7f987419a950 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f9879a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9874074cd0 0x7f98741a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f9879a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9874074cd0 0x7f98741a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50090/0 (socket says 172.21.15.18:50090) 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f9879a00640 1 -- 172.21.15.18:0/118422155 learned_addr learned my addr 172.21.15.18:0/118422155 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.458+0000 7f9879000640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987410e6d0 0x7f98741a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f987a400640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f987419a950 0x7f987419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f9879a00640 1 -- 172.21.15.18:0/118422155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f987419a950 msgr2=0x7f987419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f9879a00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f987419a950 0x7f987419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f9879a00640 1 -- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987410e6d0 msgr2=0x7f98741a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f9879a00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987410e6d0 0x7f98741a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f9879a00640 1 -- 172.21.15.18:0/118422155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987419b470 con 0x7f9874074cd0 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f987a400640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f987419a950 0x7f987419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f9879a00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9874074cd0 0x7f98741a07a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f986802c250 tx=0x7f9868004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.459+0000 7f9872000640 1 -- 172.21.15.18:0/118422155 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868041020 con 0x7f9874074cd0 2024-04-16T17:12:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.460+0000 7f9872000640 1 -- 172.21.15.18:0/118422155 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9868005000 con 0x7f9874074cd0 2024-04-16T17:12:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.460+0000 7f9872000640 1 -- 172.21.15.18:0/118422155 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868003a10 con 0x7f9874074cd0 2024-04-16T17:12:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.460+0000 7f987ae00640 1 -- 172.21.15.18:0/118422155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987419b700 con 0x7f9874074cd0 2024-04-16T17:12:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.460+0000 7f987ae00640 1 -- 172.21.15.18:0/118422155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98741a7900 con 0x7f9874074cd0 2024-04-16T17:12:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.463+0000 7f9872000640 1 -- 172.21.15.18:0/118422155 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9868044020 con 0x7f9874074cd0 2024-04-16T17:12:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.464+0000 7f987f1ba740 1 Processor -- start 2024-04-16T17:12:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.464+0000 7f987f1ba740 1 -- start start 2024-04-16T17:12:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987f1ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5622a0314780 con 0x5622a03b07f0 2024-04-16T17:12:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987f1ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5622a02baf40 con 0x5622a03b9000 2024-04-16T17:12:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987f1ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5622a0310740 con 0x5622a03afa80 2024-04-16T17:12:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f9879000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f9879000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03bb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50094/0 (socket says 172.21.15.18:50094) 2024-04-16T17:12:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53906/0 (socket says 172.21.15.18:53906) 2024-04-16T17:12:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f987a400640 1 -- 172.21.15.18:0/213587390 learned_addr learned my addr 172.21.15.18:0/213587390 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.465+0000 7f9879a00640 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.466+0000 7f9872000640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f985c0a3de0 0x7f985c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.466+0000 7f9879a00640 1 -- 172.21.15.18:0/213587390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 msgr2=0x5622a03bb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.466+0000 7f9879a00640 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03bb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.466+0000 7f9879a00640 1 -- 172.21.15.18:0/213587390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 msgr2=0x5622a03af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.466+0000 7f9879a00640 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.466+0000 7f9879a00640 1 -- 172.21.15.18:0/213587390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a0027210 con 0x5622a03afa80 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.466+0000 7f987a400640 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.466+0000 7f9872000640 1 -- 172.21.15.18:0/118422155 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f98680f2360 con 0x7f9874074cd0 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.467+0000 7f9879a00640 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03aff00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f98680ef750 tx=0x7f98680b3430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8b35424e7e8b077 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.467+0000 7f9866a00640 1 -- 172.21.15.18:0/213587390 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680bdd70 con 0x5622a03afa80 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.467+0000 7f9866a00640 1 -- 172.21.15.18:0/213587390 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9868088ab0 con 0x5622a03afa80 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.467+0000 7f9866a00640 1 -- 172.21.15.18:0/213587390 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680bd900 con 0x5622a03afa80 2024-04-16T17:12:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.467+0000 7f987a400640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f985c0a3de0 0x7f985c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 -- 172.21.15.18:0/213587390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 msgr2=0x5622a03aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03aff00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f98680ef750 tx=0x7f98680b3430 comp rx=0 tx=0).stop 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 -- 172.21.15.18:0/213587390 shutdown_connections 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03aff00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 --2- 172.21.15.18:0/213587390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 -- 172.21.15.18:0/213587390 >> 172.21.15.18:0/213587390 conn(0x5622a034a5d0 msgr2=0x5622a03aa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987a400640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f985c0a3de0 0x7f985c0a62a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f986c0099a0 tx=0x7f986c004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 -- 172.21.15.18:0/213587390 shutdown_connections 2024-04-16T17:12:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.468+0000 7f987f1ba740 1 -- 172.21.15.18:0/213587390 wait complete. 2024-04-16T17:12:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.469+0000 7f987f1ba740 1 -- start start 2024-04-16T17:12:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.469+0000 7f987f1ba740 1 Processor -- start 2024-04-16T17:12:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f987f1ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5622a0310740 con 0x5622a03b07f0 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f987f1ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5622a02ce390 con 0x5622a03b9000 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f987f1ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5622a03bbaf0 con 0x5622a03afa80 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f987f1ba740 10 client.0 _ll_register_callbacks cb 0x5622a02b7f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47892/0 (socket says 172.21.15.18:47892) 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879a00640 1 -- 172.21.15.18:0/1798084447 learned_addr learned my addr 172.21.15.18:0/1798084447 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f987a400640 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879000640 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879a00640 1 -- 172.21.15.18:0/1798084447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 msgr2=0x5622a03b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879a00640 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879a00640 1 -- 172.21.15.18:0/1798084447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 msgr2=0x5622a03b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879a00640 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.470+0000 7f9879a00640 1 -- 172.21.15.18:0/1798084447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a00f3620 con 0x5622a03afa80 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.471+0000 7f9879a00640 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b8a60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9868036040 tx=0x7f98680be6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28862144d9cc582a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.471+0000 7f985b400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.471+0000 7f985b400640 1 -- 172.21.15.18:0/1798084447 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680bdae0 con 0x5622a03afa80 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.471+0000 7f985b400640 1 -- 172.21.15.18:0/1798084447 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f98680d2700 con 0x5622a03afa80 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.471+0000 7f987f1ba740 1 -- 172.21.15.18:0/1798084447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9868088ab0 con 0x5622a03afa80 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.471+0000 7f985b400640 1 -- 172.21.15.18:0/1798084447 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98680bd900 con 0x5622a03afa80 2024-04-16T17:12:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.471+0000 7f987f1ba740 10 client.27745 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.471+0000 7f987f1ba740 1 -- 172.21.15.18:0/1798084447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622a03c7a20 con 0x5622a03afa80 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f985b400640 1 -- 172.21.15.18:0/1798084447 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9868088ab0 con 0x5622a03afa80 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f985b400640 1 -- 172.21.15.18:0/1798084447 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f98680ba610 con 0x5622a03afa80 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f987f1ba740 10 client.27745 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f987f1ba740 10 client.27745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f987f1ba740 20 client.27745 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f987f1ba740 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5622a03c8de0 0x5622a03cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f987f1ba740 4 client.27745 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f987f1ba740 1 -- 172.21.15.18:0/1798084447 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622a03221a0 con 0x5622a03c8de0 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.472+0000 7f9879000640 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5622a03c8de0 0x5622a03cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.473+0000 7f9879000640 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5622a03c8de0 0x5622a03cb1d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9348961fb7d027df server_cookie=291a044e31b9d601 in_seq=0 out_seq=0 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.473+0000 7f985b400640 10 client.27745 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.473+0000 7f985b400640 1 -- 172.21.15.18:0/1798084447 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622a03221a0 con 0x5622a03c8de0 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f985b400640 10 client.27745 handle_command_reply: tid=0 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 client.27745 shutdown 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 -- 172.21.15.18:0/1798084447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 msgr2=0x5622a03b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b8a60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9868036040 tx=0x7f98680be6a0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 -- 172.21.15.18:0/1798084447 shutdown_connections 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5622a03c8de0 0x5622a03cb1d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 --2- 172.21.15.18:0/1798084447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b8a60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f985b400640 0 client.27745 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 -- 172.21.15.18:0/1798084447 >> 172.21.15.18:0/1798084447 conn(0x5622a034a5d0 msgr2=0x5622a03ac4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 -- 172.21.15.18:0/1798084447 shutdown_connections 2024-04-16T17:12:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 1 -- 172.21.15.18:0/1798084447 wait complete. 2024-04-16T17:12:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.474+0000 7f987f1ba740 20 client.27745 trim_cache size 0 max 16384 2024-04-16T17:12:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.497+0000 7f987f1ba740 1 Processor -- start 2024-04-16T17:12:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f987f1ba740 1 -- start start 2024-04-16T17:12:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f987f1ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f98741b45a0 con 0x5622a03b07f0 2024-04-16T17:12:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f987f1ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98741b1e70 con 0x5622a03b9000 2024-04-16T17:12:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f987f1ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98741afce0 con 0x5622a03afa80 2024-04-16T17:12:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f9879000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f987a400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.498+0000 7f9879000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03a9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53922/0 (socket says 172.21.15.18:53922) 2024-04-16T17:12:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f9879000640 1 -- 172.21.15.18:0/2477568693 learned_addr learned my addr 172.21.15.18:0/2477568693 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f9879a00640 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f9879000640 1 -- 172.21.15.18:0/2477568693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 msgr2=0x5622a03b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f9879000640 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f9879000640 1 -- 172.21.15.18:0/2477568693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 msgr2=0x5622a03b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f9879000640 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f9879000640 1 -- 172.21.15.18:0/2477568693 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a03b6880 con 0x5622a03b07f0 2024-04-16T17:12:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f987a400640 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.499+0000 7f9879000640 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03a9690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x5622a03b0520 tx=0x7f9860005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a44ab187ba2b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.500+0000 7f9866000640 1 -- 172.21.15.18:0/2477568693 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860004230 con 0x5622a03b07f0 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.500+0000 7f9866000640 1 -- 172.21.15.18:0/2477568693 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98600043d0 con 0x5622a03b07f0 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.500+0000 7f9866000640 1 -- 172.21.15.18:0/2477568693 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860004a00 con 0x5622a03b07f0 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 -- 172.21.15.18:0/2477568693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 msgr2=0x5622a03a9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03a9690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x5622a03b0520 tx=0x7f9860005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 -- 172.21.15.18:0/2477568693 shutdown_connections 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b9000 0x5622a03b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b07f0 0x5622a03a9690 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2477568693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 -- 172.21.15.18:0/2477568693 >> 172.21.15.18:0/2477568693 conn(0x5622a034a5d0 msgr2=0x5622a03a8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 -- 172.21.15.18:0/2477568693 shutdown_connections 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 -- 172.21.15.18:0/2477568693 wait complete. 2024-04-16T17:12:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.501+0000 7f987f1ba740 1 -- start start 2024-04-16T17:12:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.502+0000 7f987f1ba740 1 Processor -- start 2024-04-16T17:12:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.502+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.502+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b07f0 0x5622a03bcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.502+0000 7f987f1ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b9000 0x5622a03bd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.502+0000 7f987f1ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f98741b0330 con 0x5622a03b9000 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.502+0000 7f987f1ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98741b17a0 con 0x5622a03b07f0 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.502+0000 7f987f1ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98741b0a00 con 0x5622a03afa80 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.502+0000 7f987f1ba740 10 client.0 _ll_register_callbacks cb 0x7f986c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b07f0 0x5622a03bcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03bc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47908/0 (socket says 172.21.15.18:47908) 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879a00640 1 -- 172.21.15.18:0/2674379737 learned_addr learned my addr 172.21.15.18:0/2674379737 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b07f0 0x5622a03bcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50112/0 (socket says 172.21.15.18:50112) 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f987a400640 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b9000 0x5622a03bd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879000640 1 -- 172.21.15.18:0/2674379737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 msgr2=0x5622a03bc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879000640 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03bc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879000640 1 -- 172.21.15.18:0/2674379737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b9000 msgr2=0x5622a03bd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879000640 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b9000 0x5622a03bd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.503+0000 7f9879000640 1 -- 172.21.15.18:0/2674379737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a042e5f0 con 0x5622a03b07f0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.504+0000 7f9879000640 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b07f0 0x5622a03bcee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5622a03cb690 tx=0x7f9860005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889670c477337df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.504+0000 7f985b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.504+0000 7f985b400640 1 -- 172.21.15.18:0/2674379737 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986002c020 con 0x5622a03b07f0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.504+0000 7f987f1ba740 1 -- 172.21.15.18:0/2674379737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98600043d0 con 0x5622a03b07f0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.504+0000 7f985b400640 1 -- 172.21.15.18:0/2674379737 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9554+0+0 (secure 0 0 0) 0x7f986001e480 con 0x5622a03b07f0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.504+0000 7f985b400640 1 -- 172.21.15.18:0/2674379737 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9860037050 con 0x5622a03b07f0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.504+0000 7f987f1ba740 10 client.27848 fetch_fsmap learned FSMap version 27 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.504+0000 7f987f1ba740 1 -- 172.21.15.18:0/2674379737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622a042f8f0 con 0x5622a03b07f0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f985b400640 1 -- 172.21.15.18:0/2674379737 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98600043d0 con 0x5622a03b07f0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f985b400640 1 -- 172.21.15.18:0/2674379737 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f98600283f0 con 0x5622a03b07f0 2024-04-16T17:12:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f987f1ba740 10 client.27848 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:12:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f987f1ba740 10 client.27848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f987f1ba740 20 client.27848 populate_metadata read hostname 'smithi018' 2024-04-16T17:12:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5622a0430070 0x5622a0432490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f987f1ba740 4 client.27848 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f987f1ba740 1 -- 172.21.15.18:0/2674379737 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622a031c010 con 0x5622a0430070 2024-04-16T17:12:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.505+0000 7f9879a00640 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5622a0430070 0x5622a0432490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.506+0000 7f9879a00640 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5622a0430070 0x5622a0432490 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c68855a5275931b server_cookie=a57a38e06071c7ad in_seq=0 out_seq=0 2024-04-16T17:12:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.506+0000 7f985b400640 10 client.27848 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:57.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 318134 3.91 MB/sec execute 310 sec latency 83.955 ms 2024-04-16T17:12:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.645+0000 7f985b400640 1 -- 172.21.15.18:0/2674379737 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622a031c010 con 0x5622a0430070 2024-04-16T17:12:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.645+0000 7f985b400640 10 client.27848 handle_command_reply: tid=0 2024-04-16T17:12:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.645+0000 7f987f1ba740 1 client.27848 shutdown 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 -- 172.21.15.18:0/2674379737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b07f0 msgr2=0x5622a03bcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b07f0 0x5622a03bcee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5622a03cb690 tx=0x7f9860005250 comp rx=0 tx=0).stop 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 -- 172.21.15.18:0/2674379737 shutdown_connections 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5622a0430070 0x5622a0432490 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622a03b9000 0x5622a03bd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5622a03b07f0 0x5622a03bcee0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 --2- 172.21.15.18:0/2674379737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622a03afa80 0x5622a03bc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 -- 172.21.15.18:0/2674379737 >> 172.21.15.18:0/2674379737 conn(0x5622a034a5d0 msgr2=0x5622a03a9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f985b400640 0 client.27848 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 -- 172.21.15.18:0/2674379737 shutdown_connections 2024-04-16T17:12:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.646+0000 7f987f1ba740 1 -- 172.21.15.18:0/2674379737 wait complete. 2024-04-16T17:12:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.647+0000 7f987f1ba740 20 client.27848 trim_cache size 0 max 16384 2024-04-16T17:12:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:12:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:12:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:12:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:12:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 -- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f985c0a3de0 msgr2=0x7f985c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f985c0a3de0 0x7f985c0a62a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f986c0099a0 tx=0x7f986c004a40 comp rx=0 tx=0).stop 2024-04-16T17:12:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 -- 172.21.15.18:0/118422155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9874074cd0 msgr2=0x7f98741a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9874074cd0 0x7f98741a07a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f986802c250 tx=0x7f9868004340 comp rx=0 tx=0).stop 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 -- 172.21.15.18:0/118422155 shutdown_connections 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f985c0a3de0 0x7f985c0a62a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f987419a950 0x7f987419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987410e6d0 0x7f98741a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 --2- 172.21.15.18:0/118422155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9874074cd0 0x7f98741a07a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 -- 172.21.15.18:0/118422155 >> 172.21.15.18:0/118422155 conn(0x7f98740fd940 msgr2=0x7f98740fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.648+0000 7f987ae00640 1 -- 172.21.15.18:0/118422155 shutdown_connections 2024-04-16T17:12:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:12:57.649+0000 7f987ae00640 1 -- 172.21.15.18:0/118422155 wait complete. 2024-04-16T17:12:58.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:57 smithi043 ceph-mon[36901]: pgmap v255: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-16T17:12:58.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:57 smithi022 ceph-mon[36023]: pgmap v255: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-16T17:12:58.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 319132 3.92 MB/sec execute 311 sec latency 111.179 ms 2024-04-16T17:12:59.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 319485 3.92 MB/sec execute 312 sec latency 66.967 ms 2024-04-16T17:13:00.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:12:59 smithi043 ceph-mon[36901]: pgmap v256: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-04-16T17:13:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:12:59 smithi022 ceph-mon[36023]: pgmap v256: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-04-16T17:13:00.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 319905 3.91 MB/sec execute 313 sec latency 53.347 ms 2024-04-16T17:13:01.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 320776 3.91 MB/sec execute 314 sec latency 26.804 ms 2024-04-16T17:13:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:01 smithi043 ceph-mon[36901]: pgmap v257: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-04-16T17:13:02.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:01 smithi022 ceph-mon[36023]: pgmap v257: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-04-16T17:13:02.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 321853 3.91 MB/sec execute 315 sec latency 18.027 ms 2024-04-16T17:13:03.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 322759 3.93 MB/sec execute 316 sec latency 52.177 ms 2024-04-16T17:13:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:03 smithi043 ceph-mon[36901]: pgmap v258: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:13:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:03 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:13:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:03 smithi022 ceph-mon[36023]: pgmap v258: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:13:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:03 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:13:04.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 323014 3.92 MB/sec execute 317 sec latency 57.759 ms 2024-04-16T17:13:05.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:04 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:13:05.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:04 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:13:05.074 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:04 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:13:05.074 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:04 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:13:05.485 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:57 smithi018 ceph-mon[31776]: pgmap v255: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-16T17:13:05.485 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:12:59 smithi018 ceph-mon[31776]: pgmap v256: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-04-16T17:13:05.485 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:01 smithi018 ceph-mon[31776]: pgmap v257: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-04-16T17:13:05.485 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:03 smithi018 ceph-mon[31776]: pgmap v258: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-16T17:13:05.485 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:03 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:13:05.485 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:04 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:13:05.486 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:04 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:13:05.613 INFO:tasks.workunit.client.0.smithi018.stdout: 1 323119 3.91 MB/sec execute 318 sec latency 89.970 ms 2024-04-16T17:13:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:05 smithi043 ceph-mon[36901]: pgmap v259: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:13:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:05 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:13:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:05 smithi043 ceph-mon[36901]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T17:13:06.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:05 smithi022 ceph-mon[36023]: pgmap v259: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:13:06.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:05 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:13:06.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:05 smithi022 ceph-mon[36023]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T17:13:06.614 INFO:tasks.workunit.client.0.smithi018.stdout: 1 323517 3.90 MB/sec execute 319 sec latency 50.518 ms 2024-04-16T17:13:06.861 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:05 smithi018 ceph-mon[31776]: pgmap v259: 129 pgs: 129 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:13:06.861 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:05 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:13:06.861 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:05 smithi018 ceph-mon[31776]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T17:13:07.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.90 MB/sec execute 320 sec latency 253.511 ms 2024-04-16T17:13:08.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:07 smithi022 ceph-mon[36023]: pgmap v261: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:13:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:07 smithi018 ceph-mon[31776]: pgmap v261: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:13:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:07 smithi043 ceph-mon[36901]: pgmap v261: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:13:08.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.89 MB/sec execute 321 sec latency 1253.586 ms 2024-04-16T17:13:09.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.87 MB/sec execute 322 sec latency 2253.691 ms 2024-04-16T17:13:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:09 smithi022 ceph-mon[36023]: pgmap v262: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:13:10.232 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:09 smithi018 ceph-mon[31776]: pgmap v262: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:13:10.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:09 smithi043 ceph-mon[36901]: pgmap v262: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:13:10.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.86 MB/sec execute 323 sec latency 3253.803 ms 2024-04-16T17:13:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 -- 172.21.15.18:0/830528293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c4074cc0 msgr2=0x7fe5c41128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 --2- 172.21.15.18:0/830528293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c4074cc0 0x7fe5c41128f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009900 tx=0x7fe5b002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 -- 172.21.15.18:0/830528293 shutdown_connections 2024-04-16T17:13:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 --2- 172.21.15.18:0/830528293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5c4112e30 0x7fe5c4115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 --2- 172.21.15.18:0/830528293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c4074cc0 0x7fe5c41128f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 --2- 172.21.15.18:0/830528293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c40743a0 0x7fe5c4074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 -- 172.21.15.18:0/830528293 >> 172.21.15.18:0/830528293 conn(0x7fe5c40fd910 msgr2=0x7fe5c40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 -- 172.21.15.18:0/830528293 shutdown_connections 2024-04-16T17:13:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.785+0000 7fe5c3200640 1 -- 172.21.15.18:0/830528293 wait complete. 2024-04-16T17:13:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.786+0000 7fe5c3200640 1 Processor -- start 2024-04-16T17:13:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.786+0000 7fe5c3200640 1 -- start start 2024-04-16T17:13:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.786+0000 7fe5c3200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c40743a0 0x7fe5c419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5c4074cc0 0x7fe5c419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c4112e30 0x7fe5c41a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c3200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4118800 con 0x7fe5c40743a0 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4118680 con 0x7fe5c4074cc0 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4118980 con 0x7fe5c4112e30 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c40743a0 0x7fe5c419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c40743a0 0x7fe5c419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56034/0 (socket says 172.21.15.18:56034) 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1e00640 1 -- 172.21.15.18:0/2341350167 learned_addr learned my addr 172.21.15.18:0/2341350167 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1400640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5c4074cc0 0x7fe5c419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c2800640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c4112e30 0x7fe5c41a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1e00640 1 -- 172.21.15.18:0/2341350167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c4112e30 msgr2=0x7fe5c41a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1e00640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c4112e30 0x7fe5c41a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1e00640 1 -- 172.21.15.18:0/2341350167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5c4074cc0 msgr2=0x7fe5c419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1e00640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5c4074cc0 0x7fe5c419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.787+0000 7fe5c1e00640 1 -- 172.21.15.18:0/2341350167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41a3d70 con 0x7fe5c40743a0 2024-04-16T17:13:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.788+0000 7fe5c1e00640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c40743a0 0x7fe5c419ed80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400d950 tx=0x7fe5b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.788+0000 7fe5baa00640 1 -- 172.21.15.18:0/2341350167 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4017030 con 0x7fe5c40743a0 2024-04-16T17:13:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.788+0000 7fe5baa00640 1 -- 172.21.15.18:0/2341350167 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b4004020 con 0x7fe5c40743a0 2024-04-16T17:13:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.788+0000 7fe5baa00640 1 -- 172.21.15.18:0/2341350167 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40047a0 con 0x7fe5c40743a0 2024-04-16T17:13:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.788+0000 7fe5c3200640 1 -- 172.21.15.18:0/2341350167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c41a4060 con 0x7fe5c40743a0 2024-04-16T17:13:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.788+0000 7fe5c3200640 1 -- 172.21.15.18:0/2341350167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41ab8f0 con 0x7fe5c40743a0 2024-04-16T17:13:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.791+0000 7fe5cbbfc740 1 Processor -- start 2024-04-16T17:13:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.791+0000 7fe5baa00640 1 -- 172.21.15.18:0/2341350167 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe5b40041d0 con 0x7fe5c40743a0 2024-04-16T17:13:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5cbbfc740 1 -- start start 2024-04-16T17:13:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeb7f0 0x5633c2eea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 0x5633c2eeaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 0x5633c2ef63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5633c2e4f780 con 0x5633c2eeb7f0 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5633c2df5f40 con 0x5633c2eeaa80 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5633c2e4b740 con 0x5633c2ef4000 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeb7f0 0x5633c2eea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeb7f0 0x5633c2eea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56044/0 (socket says 172.21.15.18:56044) 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5c2800640 1 -- 172.21.15.18:0/279302875 learned_addr learned my addr 172.21.15.18:0/279302875 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5c1e00640 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 0x5633c2eeaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.792+0000 7fe5c1400640 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 0x5633c2ef63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.793+0000 7fe5c2800640 1 -- 172.21.15.18:0/279302875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 msgr2=0x5633c2ef63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.793+0000 7fe5c2800640 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 0x5633c2ef63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.793+0000 7fe5c2800640 1 -- 172.21.15.18:0/279302875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 msgr2=0x5633c2eeaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.793+0000 7fe5c2800640 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 0x5633c2eeaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.793+0000 7fe5c2800640 1 -- 172.21.15.18:0/279302875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c2b62210 con 0x5633c2eeb7f0 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.793+0000 7fe5baa00640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5a80a3d80 0x7fe5a80a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.793+0000 7fe5c2800640 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeb7f0 0x5633c2eea540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009f00 tx=0x7fe5bc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8995459027c1244b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.793+0000 7fe5baa00640 1 -- 172.21.15.18:0/2341350167 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7fe5b40d0610 con 0x7fe5c40743a0 2024-04-16T17:13:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.794+0000 7fe5c1e00640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5a80a3d80 0x7fe5a80a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.794+0000 7fe5af400640 1 -- 172.21.15.18:0/279302875 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc003b70 con 0x5633c2eeb7f0 2024-04-16T17:13:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.794+0000 7fe5af400640 1 -- 172.21.15.18:0/279302875 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc003d10 con 0x5633c2eeb7f0 2024-04-16T17:13:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.794+0000 7fe5af400640 1 -- 172.21.15.18:0/279302875 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc011a40 con 0x5633c2eeb7f0 2024-04-16T17:13:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.794+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/279302875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeb7f0 msgr2=0x5633c2eea540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.794+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeb7f0 0x5633c2eea540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009f00 tx=0x7fe5bc0107a0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.794+0000 7fe5c1e00640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5a80a3d80 0x7fe5a80a6240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40cd9b0 tx=0x7fe5b40d6550 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/279302875 shutdown_connections 2024-04-16T17:13:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 0x5633c2ef63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 0x5633c2eeaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/279302875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeb7f0 0x5633c2eea540 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/279302875 >> 172.21.15.18:0/279302875 conn(0x5633c2e855d0 msgr2=0x5633c2ee5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/279302875 shutdown_connections 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/279302875 wait complete. 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 -- start start 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.795+0000 7fe5cbbfc740 1 Processor -- start 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeaa80 0x5633c2ef3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5633c2ef6d30 con 0x5633c2ef4000 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5633c2e09390 con 0x5633c2eeb7f0 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5633c2ef6eb0 con 0x5633c2eeaa80 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5cbbfc740 10 client.0 _ll_register_callbacks cb 0x5633c2df2f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56056/0 (socket says 172.21.15.18:56056) 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5c2800640 1 -- 172.21.15.18:0/2884421544 learned_addr learned my addr 172.21.15.18:0/2884421544 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.796+0000 7fe5c2800640 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeaa80 0x5633c2ef3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5c1400640 1 -- 172.21.15.18:0/2884421544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeaa80 msgr2=0x5633c2ef3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5c1400640 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeaa80 0x5633c2ef3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5c1400640 1 -- 172.21.15.18:0/2884421544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 msgr2=0x5633c2ef1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5c1400640 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5c1400640 1 -- 172.21.15.18:0/2884421544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c2c2e620 con 0x5633c2eeb7f0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5c1400640 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef0e40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0000c00 tx=0x7fe5b0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e85a7cfb78d9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2884421544 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0004350 con 0x5633c2eeb7f0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2884421544 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7fe5b00046c0 con 0x5633c2eeb7f0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2884421544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5bc003d10 con 0x5633c2eeb7f0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2884421544 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5b00048c0 con 0x5633c2eeb7f0 2024-04-16T17:13:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.797+0000 7fe5cbbfc740 10 client.27860 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2884421544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633c2f02e20 con 0x5633c2eeb7f0 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2884421544 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc003d10 con 0x5633c2eeb7f0 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2884421544 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe5b00035f0 con 0x5633c2eeb7f0 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5cbbfc740 10 client.27860 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5cbbfc740 10 client.27860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5cbbfc740 20 client.27860 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5633c2f042e0 0x5633c2f066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5cbbfc740 4 client.27860 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.798+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2884421544 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633c2e5d1a0 con 0x5633c2f042e0 2024-04-16T17:13:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.799+0000 7fe5c2800640 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5633c2f042e0 0x5633c2f066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.799+0000 7fe5c2800640 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5633c2f042e0 0x5633c2f066d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb72fb9785fe0dc2 server_cookie=8ad8881d7040cafe in_seq=0 out_seq=0 2024-04-16T17:13:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.799+0000 7fe5a3e00640 10 client.27860 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2884421544 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633c2e5d1a0 con 0x5633c2f042e0 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5a3e00640 10 client.27860 handle_command_reply: tid=0 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 client.27860 shutdown 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2884421544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 msgr2=0x5633c2ef0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef0e40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0000c00 tx=0x7fe5b0005dc0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2884421544 shutdown_connections 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5633c2f042e0 0x5633c2f066d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5a3e00640 0 client.27860 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef0e40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2884421544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeaa80 0x5633c2ef3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.800+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2884421544 >> 172.21.15.18:0/2884421544 conn(0x5633c2e855d0 msgr2=0x5633c2ee74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.801+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2884421544 shutdown_connections 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.801+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2884421544 wait complete. 2024-04-16T17:13:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.801+0000 7fe5cbbfc740 20 client.27860 trim_cache size 0 max 16384 2024-04-16T17:13:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.833+0000 7fe5cbbfc740 1 Processor -- start 2024-04-16T17:13:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.833+0000 7fe5cbbfc740 1 -- start start 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.833+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 0x5633c2effc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.833+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.833+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.833+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b8550 con 0x5633c2ef4000 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.833+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b5e20 con 0x5633c2eeaa80 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.833+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b3c90 con 0x5633c2eeb7f0 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 0x5633c2effc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57370/0 (socket says 172.21.15.18:57370) 2024-04-16T17:13:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56068/0 (socket says 172.21.15.18:56068) 2024-04-16T17:13:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c2800640 1 -- 172.21.15.18:0/3864725494 learned_addr learned my addr 172.21.15.18:0/3864725494 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c1400640 1 -- 172.21.15.18:0/3864725494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 msgr2=0x5633c2effc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c1400640 1 --2- 172.21.15.18:0/3864725494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 0x5633c2effc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c1400640 1 -- 172.21.15.18:0/3864725494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 msgr2=0x5633c2ef3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c1400640 1 --2- 172.21.15.18:0/3864725494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c1400640 1 -- 172.21.15.18:0/3864725494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c2ee9d50 con 0x5633c2eeb7f0 2024-04-16T17:13:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.834+0000 7fe5c2800640 1 --2- 172.21.15.18:0/3864725494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.835+0000 7fe5c1400640 1 --2- 172.21.15.18:0/3864725494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef3570 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0003b80 tx=0x7fe5b0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad57b53e7d1c1e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.835+0000 7fe5aea00640 1 -- 172.21.15.18:0/3864725494 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b005c070 con 0x5633c2eeb7f0 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.835+0000 7fe5aea00640 1 -- 172.21.15.18:0/3864725494 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0004350 con 0x5633c2eeb7f0 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.835+0000 7fe5aea00640 1 -- 172.21.15.18:0/3864725494 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0004a60 con 0x5633c2eeb7f0 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/3864725494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeb7f0 msgr2=0x5633c2ef3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/3864725494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef3570 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0003b80 tx=0x7fe5b0005fd0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/3864725494 shutdown_connections 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/3864725494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2ef4000 0x5633c2ef3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/3864725494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef3570 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/3864725494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeaa80 0x5633c2effc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/3864725494 >> 172.21.15.18:0/3864725494 conn(0x5633c2e855d0 msgr2=0x5633c2eef710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/3864725494 shutdown_connections 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/3864725494 wait complete. 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.836+0000 7fe5cbbfc740 1 -- start start 2024-04-16T17:13:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5cbbfc740 1 Processor -- start 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeaa80 0x5633c2ef7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5cbbfc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 0x5633c2ef86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b42e0 con 0x5633c2eeaa80 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b5750 con 0x5633c2eeb7f0 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5cbbfc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b49b0 con 0x5633c2ef4000 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5cbbfc740 10 client.0 _ll_register_callbacks cb 0x7fe5b0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeaa80 0x5633c2ef7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56648/0 (socket says 172.21.15.18:56648) 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.837+0000 7fe5c1400640 1 -- 172.21.15.18:0/2242802665 learned_addr learned my addr 172.21.15.18:0/2242802665 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5c2800640 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 0x5633c2ef86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5c1400640 1 -- 172.21.15.18:0/2242802665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 msgr2=0x5633c2ef86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5c1400640 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 0x5633c2ef86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5c1400640 1 -- 172.21.15.18:0/2242802665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeaa80 msgr2=0x5633c2ef7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5c1400640 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeaa80 0x5633c2ef7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5c1400640 1 -- 172.21.15.18:0/2242802665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c2ee7a70 con 0x5633c2eeb7f0 2024-04-16T17:13:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5c1400640 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef81b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe5b002c200 tx=0x7fe5b0003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c863b4f8f2711904 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2242802665 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b005c070 con 0x5633c2eeb7f0 2024-04-16T17:13:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2242802665 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7fe5b0004c70 con 0x5633c2eeb7f0 2024-04-16T17:13:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.838+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2242802665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0004350 con 0x5633c2eeb7f0 2024-04-16T17:13:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2242802665 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5b0004ef0 con 0x5633c2eeb7f0 2024-04-16T17:13:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5cbbfc740 10 client.27866 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2242802665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633c2f6a9d0 con 0x5633c2eeb7f0 2024-04-16T17:13:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2242802665 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0004350 con 0x5633c2eeb7f0 2024-04-16T17:13:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2242802665 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe5b004b720 con 0x5633c2eeb7f0 2024-04-16T17:13:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5cbbfc740 10 client.27866 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5cbbfc740 10 client.27866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5cbbfc740 20 client.27866 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5633c2f6b000 0x5633c2f6d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5cbbfc740 4 client.27866 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:13:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.839+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2242802665 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633c2e57010 con 0x5633c2f6b000 2024-04-16T17:13:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.840+0000 7fe5c2800640 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5633c2f6b000 0x5633c2f6d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.840+0000 7fe5a3e00640 10 client.27866 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:10.840+0000 7fe5c2800640 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5633c2f6b000 0x5633c2f6d420 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242676ea21fb569 server_cookie=c582331d37cdf877 in_seq=0 out_seq=0 2024-04-16T17:13:11.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.85 MB/sec execute 324 sec latency 4253.915 ms 2024-04-16T17:13:12.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:11 smithi022 ceph-mon[36023]: pgmap v263: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-04-16T17:13:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:11 smithi018 ceph-mon[31776]: pgmap v263: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-04-16T17:13:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:11 smithi043 ceph-mon[36901]: pgmap v263: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-04-16T17:13:12.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.84 MB/sec execute 325 sec latency 5254.033 ms 2024-04-16T17:13:13.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.83 MB/sec execute 326 sec latency 6254.150 ms 2024-04-16T17:13:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:13 smithi022 ceph-mon[36023]: pgmap v264: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-04-16T17:13:14.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:13 smithi022 ceph-mon[36023]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T17:13:14.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:13 smithi018 ceph-mon[31776]: pgmap v264: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-04-16T17:13:14.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:13 smithi018 ceph-mon[31776]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T17:13:14.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:13 smithi043 ceph-mon[36901]: pgmap v264: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-04-16T17:13:14.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:13 smithi043 ceph-mon[36901]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T17:13:14.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.82 MB/sec execute 327 sec latency 7254.315 ms 2024-04-16T17:13:14.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:13:14.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:13:14.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 -- 172.21.15.18:0/1064837906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204104260 msgr2=0x7f7204104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:14.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 --2- 172.21.15.18:0/1064837906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204104260 0x7f7204104640 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009900 tx=0x7f71f802bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 -- 172.21.15.18:0/1064837906 shutdown_connections 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 --2- 172.21.15.18:0/1064837906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7204109330 0x7f7204110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 --2- 172.21.15.18:0/1064837906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7204104c10 0x7f7204108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 --2- 172.21.15.18:0/1064837906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204104260 0x7f7204104640 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 -- 172.21.15.18:0/1064837906 >> 172.21.15.18:0/1064837906 conn(0x7f72040ffaa0 msgr2=0x7f7204101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 -- 172.21.15.18:0/1064837906 shutdown_connections 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.875+0000 7f720a200640 1 -- 172.21.15.18:0/1064837906 wait complete. 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.876+0000 7f720a200640 1 Processor -- start 2024-04-16T17:13:14.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.876+0000 7f720a200640 1 -- start start 2024-04-16T17:13:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f720a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7204104260 0x7f72041a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f720a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204109330 0x7f72041a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f720a200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f720419a820 0x7f720419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f720a200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7204114440 con 0x7f720419a820 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f720a200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72041142c0 con 0x7f7204104260 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f720a200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72041145c0 con 0x7f7204109330 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f7209800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f720419a820 0x7f720419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f7203e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204109330 0x7f72041a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f7208e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7204104260 0x7f72041a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f7203e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204109330 0x7f72041a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:57390/0 (socket says 172.21.15.18:57390) 2024-04-16T17:13:14.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.877+0000 7f7203e00640 1 -- 172.21.15.18:0/3253400314 learned_addr learned my addr 172.21.15.18:0/3253400314 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7208e00640 1 -- 172.21.15.18:0/3253400314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204109330 msgr2=0x7f72041a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:14.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7208e00640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204109330 0x7f72041a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7208e00640 1 -- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f720419a820 msgr2=0x7f720419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7208e00640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f720419a820 0x7f720419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7208e00640 1 -- 172.21.15.18:0/3253400314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720419b3b0 con 0x7f7204104260 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7208e00640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7204104260 0x7f72041a0660 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f71f802c170 tx=0x7f71f8004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7201600640 1 -- 172.21.15.18:0/3253400314 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8043020 con 0x7f7204104260 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f720a200640 1 -- 172.21.15.18:0/3253400314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720419b640 con 0x7f7204104260 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7201600640 1 -- 172.21.15.18:0/3253400314 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f80054e0 con 0x7f7204104260 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.878+0000 7f7201600640 1 -- 172.21.15.18:0/3253400314 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8003b80 con 0x7f7204104260 2024-04-16T17:13:14.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.879+0000 7f720a200640 1 -- 172.21.15.18:0/3253400314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72041a77f0 con 0x7f7204104260 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f720e573740 1 Processor -- start 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f720e573740 1 -- start start 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f7201600640 1 -- 172.21.15.18:0/3253400314 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71f8046020 con 0x7f7204104260 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f720e573740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c319770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f720e573740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c322350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f720e573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c324c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f720e573740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d70c27f120 con 0x55d70c319cb0 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f720e573740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d70c225340 con 0x55d70c319390 2024-04-16T17:13:14.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.882+0000 7f720e573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d70c27b0e0 con 0x55d70c322890 2024-04-16T17:13:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c319770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c319770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56674/0 (socket says 172.21.15.18:56674) 2024-04-16T17:13:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 -- 172.21.15.18:0/3692459733 learned_addr learned my addr 172.21.15.18:0/3692459733 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7208e00640 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c322350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7203e00640 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c324c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 -- 172.21.15.18:0/3692459733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 msgr2=0x55d70c324c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c324c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 -- 172.21.15.18:0/3692459733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 msgr2=0x55d70c322350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c322350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 -- 172.21.15.18:0/3692459733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d70c05b7d0 con 0x55d70c319390 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7208e00640 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c322350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.883+0000 7f7209800640 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c319770 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009f50 tx=0x7f71fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5051589257d7ead5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.884+0000 7f71f6000640 1 -- 172.21.15.18:0/3692459733 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004d20 con 0x55d70c319390 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.884+0000 7f71f6000640 1 -- 172.21.15.18:0/3692459733 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc004050 con 0x55d70c319390 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.884+0000 7f7201600640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71ec0a3d20 0x7f71ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.884+0000 7f71f6000640 1 -- 172.21.15.18:0/3692459733 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc005280 con 0x55d70c319390 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.884+0000 7f7201600640 1 -- 172.21.15.18:0/3253400314 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f71f80f2a30 con 0x7f7204104260 2024-04-16T17:13:14.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.884+0000 7f7208e00640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71ec0a3d20 0x7f71ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.884+0000 7f720e573740 1 -- 172.21.15.18:0/3692459733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 msgr2=0x55d70c319770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.884+0000 7f720e573740 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c319770 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009f50 tx=0x7f71fc0118d0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 -- 172.21.15.18:0/3692459733 shutdown_connections 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c324c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c322350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 --2- 172.21.15.18:0/3692459733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c319770 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 -- 172.21.15.18:0/3692459733 >> 172.21.15.18:0/3692459733 conn(0x55d70c313ac0 msgr2=0x55d70c313ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 -- 172.21.15.18:0/3692459733 shutdown_connections 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f7208e00640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71ec0a3d20 0x7f71ec0a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f71f80eff30 tx=0x7f71f802bc30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:14.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 -- 172.21.15.18:0/3692459733 wait complete. 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 -- start start 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.885+0000 7f720e573740 1 Processor -- start 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f720e573740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c31f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f720e573740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f720e573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c320310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f720e573740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d70c325680 con 0x55d70c319cb0 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f720e573740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d70c238710 con 0x55d70c319390 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f720e573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d70c325800 con 0x55d70c322890 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f720e573740 10 client.0 _ll_register_callbacks cb 0x55d70c233af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f7203e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f7209800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c31f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f7203e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56118/0 (socket says 172.21.15.18:56118) 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f7203e00640 1 -- 172.21.15.18:0/2578717234 learned_addr learned my addr 172.21.15.18:0/2578717234 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.886+0000 7f7209800640 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c320310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f7203e00640 1 -- 172.21.15.18:0/2578717234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 msgr2=0x55d70c320310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:14.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f7203e00640 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c320310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f7203e00640 1 -- 172.21.15.18:0/2578717234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 msgr2=0x55d70c31f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f7203e00640 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c31f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f7203e00640 1 -- 172.21.15.18:0/2578717234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70bf913f0 con 0x55d70c319cb0 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f7203e00640 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31fdd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f71f0012700 tx=0x7f71f0012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cecb0535aa70824 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f71eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f71eaa00640 1 -- 172.21.15.18:0/2578717234 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004050 con 0x55d70c319cb0 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f71eaa00640 1 -- 172.21.15.18:0/2578717234 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f71f00041f0 con 0x55d70c319cb0 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.887+0000 7f720e573740 1 -- 172.21.15.18:0/2578717234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71fc004050 con 0x55d70c319cb0 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f71eaa00640 1 -- 172.21.15.18:0/2578717234 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71f00043f0 con 0x55d70c319cb0 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f720e573740 10 client.17601 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:14.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f720e573740 1 -- 172.21.15.18:0/2578717234 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d70bf510f0 con 0x55d70c319cb0 2024-04-16T17:13:14.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f71eaa00640 1 -- 172.21.15.18:0/2578717234 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc004050 con 0x55d70c319cb0 2024-04-16T17:13:14.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f71eaa00640 1 -- 172.21.15.18:0/2578717234 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f71f0003a30 con 0x55d70c319cb0 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f720e573740 10 client.17601 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f720e573740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f720e573740 20 client.17601 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f720e573740 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d70c332c60 0x55d70c335050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f720e573740 4 client.17601 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.888+0000 7f720e573740 1 -- 172.21.15.18:0/2578717234 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d70c28cb40 con 0x55d70c332c60 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.889+0000 7f7209800640 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d70c332c60 0x55d70c335050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.889+0000 7f7209800640 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d70c332c60 0x55d70c335050 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31231f2f2d43c0e server_cookie=da4f5d1db79ab518 in_seq=0 out_seq=0 2024-04-16T17:13:14.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:14.889+0000 7f71eaa00640 10 client.17601 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:15.615 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.80 MB/sec execute 328 sec latency 8254.385 ms 2024-04-16T17:13:16.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:15 smithi022 ceph-mon[36023]: pgmap v266: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-04-16T17:13:16.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:15 smithi018 ceph-mon[31776]: pgmap v266: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-04-16T17:13:16.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:15 smithi043 ceph-mon[36901]: pgmap v266: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-04-16T17:13:16.616 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.79 MB/sec execute 329 sec latency 9254.615 ms 2024-04-16T17:13:17.616 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.78 MB/sec execute 330 sec latency 10254.758 ms 2024-04-16T17:13:18.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:17 smithi022 ceph-mon[36023]: pgmap v267: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-04-16T17:13:18.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:17 smithi018 ceph-mon[31776]: pgmap v267: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-04-16T17:13:18.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:17 smithi043 ceph-mon[36901]: pgmap v267: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-04-16T17:13:18.616 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324198 3.77 MB/sec execute 331 sec latency 11254.876 ms 2024-04-16T17:13:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.850+0000 7fe5a3e00640 1 -- 172.21.15.18:0/2242802665 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633c2e57010 con 0x5633c2f6b000 2024-04-16T17:13:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.851+0000 7fe5a3e00640 10 client.27866 handle_command_reply: tid=0 2024-04-16T17:13:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.851+0000 7fe5cbbfc740 1 client.27866 shutdown 2024-04-16T17:13:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2242802665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 msgr2=0x5633c2ef81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef81b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe5b002c200 tx=0x7fe5b0003e30 comp rx=0 tx=0).stop 2024-04-16T17:13:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2242802665 shutdown_connections 2024-04-16T17:13:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5633c2f6b000 0x5633c2f6d420 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633c2ef4000 0x5633c2ef86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5633c2eeb7f0 0x5633c2ef81b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5cbbfc740 1 --2- 172.21.15.18:0/2242802665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5633c2eeaa80 0x5633c2ef7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2242802665 >> 172.21.15.18:0/2242802665 conn(0x5633c2e855d0 msgr2=0x5633c2ee6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.852+0000 7fe5a3e00640 0 client.27866 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.853+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2242802665 shutdown_connections 2024-04-16T17:13:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.853+0000 7fe5cbbfc740 1 -- 172.21.15.18:0/2242802665 wait complete. 2024-04-16T17:13:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.853+0000 7fe5cbbfc740 20 client.27866 trim_cache size 0 max 16384 2024-04-16T17:13:18.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.853+0000 7f71eaa00640 1 -- 172.21.15.18:0/2578717234 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d70c28cb40 con 0x55d70c332c60 2024-04-16T17:13:18.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.853+0000 7f71eaa00640 10 client.17601 handle_command_reply: tid=0 2024-04-16T17:13:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:13:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.854+0000 7fe5c3200640 1 -- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5a80a3d80 msgr2=0x7fe5a80a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.854+0000 7fe5c3200640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5a80a3d80 0x7fe5a80a6240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40cd9b0 tx=0x7fe5b40d6550 comp rx=0 tx=0).stop 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.854+0000 7fe5c3200640 1 -- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c40743a0 msgr2=0x7fe5c419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.854+0000 7fe5c3200640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c40743a0 0x7fe5c419ed80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400d950 tx=0x7fe5b400de10 comp rx=0 tx=0).stop 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.855+0000 7fe5c3200640 1 -- 172.21.15.18:0/2341350167 shutdown_connections 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.855+0000 7fe5c3200640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe5a80a3d80 0x7fe5a80a6240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.855+0000 7fe5c3200640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c4112e30 0x7fe5c41a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.855+0000 7fe5c3200640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5c4074cc0 0x7fe5c419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.855+0000 7fe5c3200640 1 --2- 172.21.15.18:0/2341350167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5c40743a0 0x7fe5c419ed80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.855+0000 7fe5c3200640 1 -- 172.21.15.18:0/2341350167 >> 172.21.15.18:0/2341350167 conn(0x7fe5c40fd910 msgr2=0x7fe5c40ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:18.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.855+0000 7f720e573740 1 client.17601 shutdown 2024-04-16T17:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.855+0000 7fe5c3200640 1 -- 172.21.15.18:0/2341350167 shutdown_connections 2024-04-16T17:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:18.855+0000 7fe5c3200640 1 -- 172.21.15.18:0/2341350167 wait complete. 2024-04-16T17:13:18.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f720e573740 1 -- 172.21.15.18:0/2578717234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 msgr2=0x55d70c31fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:18.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f720e573740 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31fdd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f71f0012700 tx=0x7f71f0012b00 comp rx=0 tx=0).stop 2024-04-16T17:13:18.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f720e573740 1 -- 172.21.15.18:0/2578717234 shutdown_connections 2024-04-16T17:13:18.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f720e573740 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d70c332c60 0x55d70c335050 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f71eaa00640 0 client.17601 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:18.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f720e573740 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c320310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f720e573740 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31fdd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f720e573740 1 --2- 172.21.15.18:0/2578717234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c31f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.856+0000 7f720e573740 1 -- 172.21.15.18:0/2578717234 >> 172.21.15.18:0/2578717234 conn(0x55d70c313ac0 msgr2=0x55d70c315810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:18.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.857+0000 7f720e573740 1 -- 172.21.15.18:0/2578717234 shutdown_connections 2024-04-16T17:13:18.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.857+0000 7f720e573740 1 -- 172.21.15.18:0/2578717234 wait complete. 2024-04-16T17:13:18.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.857+0000 7f720e573740 20 client.17601 trim_cache size 0 max 16384 2024-04-16T17:13:18.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.884+0000 7f720e573740 1 Processor -- start 2024-04-16T17:13:18.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.884+0000 7f720e573740 1 -- start start 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f720e573740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c321fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f720e573740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f720e573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c32e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f720e573740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b3f20 con 0x55d70c319cb0 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f720e573740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b17f0 con 0x55d70c319390 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f720e573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72041af660 con 0x55d70c322890 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f7209800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c321fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f7209800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c321fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59852/0 (socket says 172.21.15.18:59852) 2024-04-16T17:13:18.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f7209800640 1 -- 172.21.15.18:0/2008421739 learned_addr learned my addr 172.21.15.18:0/2008421739 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:18.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f7209800640 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c32e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:18.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.885+0000 7f7203e00640 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:18.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f7209800640 1 -- 172.21.15.18:0/2008421739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 msgr2=0x55d70c32e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:18.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f7209800640 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c32e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f7209800640 1 -- 172.21.15.18:0/2008421739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 msgr2=0x55d70c31e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:18.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f7209800640 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f7209800640 1 -- 172.21.15.18:0/2008421739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d70c324c80 con 0x55d70c319390 2024-04-16T17:13:18.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f7203e00640 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:18.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f7209800640 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c321fc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005cc0 tx=0x7f71fc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb0df514b6a0377 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:18.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f71f5600640 1 -- 172.21.15.18:0/2008421739 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004480 con 0x55d70c319390 2024-04-16T17:13:18.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f71f5600640 1 -- 172.21.15.18:0/2008421739 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc003e50 con 0x55d70c319390 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.886+0000 7f71f5600640 1 -- 172.21.15.18:0/2008421739 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc027440 con 0x55d70c319390 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.887+0000 7f720e573740 1 -- 172.21.15.18:0/2008421739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 msgr2=0x55d70c321fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.887+0000 7f720e573740 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c321fc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005cc0 tx=0x7f71fc005fd0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.887+0000 7f720e573740 1 -- 172.21.15.18:0/2008421739 shutdown_connections 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.887+0000 7f720e573740 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c322890 0x55d70c32e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.887+0000 7f720e573740 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c319cb0 0x55d70c31e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.887+0000 7f720e573740 1 --2- 172.21.15.18:0/2008421739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319390 0x55d70c321fc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.887+0000 7f720e573740 1 -- 172.21.15.18:0/2008421739 >> 172.21.15.18:0/2008421739 conn(0x55d70c313ac0 msgr2=0x55d70c3200e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.888+0000 7f720e573740 1 -- 172.21.15.18:0/2008421739 shutdown_connections 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.888+0000 7f720e573740 1 -- 172.21.15.18:0/2008421739 wait complete. 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.888+0000 7f720e573740 1 -- start start 2024-04-16T17:13:18.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.888+0000 7f720e573740 1 Processor -- start 2024-04-16T17:13:18.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f720e573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c319390 0x55d70c3268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:18.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f720e573740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319cb0 0x55d70c326e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f720e573740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c322890 0x55d70c398190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f720e573740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f72041afcb0 con 0x55d70c322890 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f720e573740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b1120 con 0x55d70c319cb0 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f720e573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b0380 con 0x55d70c319390 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f720e573740 10 client.0 _ll_register_callbacks cb 0x7f71f00096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f7203e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319cb0 0x55d70c326e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f7203e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319cb0 0x55d70c326e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59866/0 (socket says 172.21.15.18:59866) 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f7209800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c319390 0x55d70c3268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f7203e00640 1 -- 172.21.15.18:0/2986280119 learned_addr learned my addr 172.21.15.18:0/2986280119 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.889+0000 7f7209800640 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c322890 0x55d70c398190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f7203e00640 1 -- 172.21.15.18:0/2986280119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c319390 msgr2=0x55d70c3268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f7203e00640 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c319390 0x55d70c3268d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f7203e00640 1 -- 172.21.15.18:0/2986280119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c322890 msgr2=0x55d70c398190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f7203e00640 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c322890 0x55d70c398190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f7203e00640 1 -- 172.21.15.18:0/2986280119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70c3986d0 con 0x55d70c319cb0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f7203e00640 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319cb0 0x55d70c326e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f71f00058c0 tx=0x7f71f000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3eb76ecffee6d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f71eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f71eaa00640 1 -- 172.21.15.18:0/2986280119 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f002d070 con 0x55d70c319cb0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f71eaa00640 1 -- 172.21.15.18:0/2986280119 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f71f00109a0 con 0x55d70c319cb0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.890+0000 7f720e573740 1 -- 172.21.15.18:0/2986280119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71fc003e50 con 0x55d70c319cb0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f71eaa00640 1 -- 172.21.15.18:0/2986280119 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71f0004210 con 0x55d70c319cb0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f720e573740 10 client.27884 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f720e573740 1 -- 172.21.15.18:0/2986280119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d70c399820 con 0x55d70c319cb0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f71eaa00640 1 -- 172.21.15.18:0/2986280119 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc003e50 con 0x55d70c319cb0 2024-04-16T17:13:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f71eaa00640 1 -- 172.21.15.18:0/2986280119 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f71f0003820 con 0x55d70c319cb0 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f720e573740 10 client.27884 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f720e573740 10 client.27884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f720e573740 20 client.27884 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f720e573740 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d70c332c60 0x55d70c39bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f720e573740 4 client.27884 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.891+0000 7f720e573740 1 -- 172.21.15.18:0/2986280119 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d70c2869b0 con 0x55d70c332c60 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.892+0000 7f7209800640 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d70c332c60 0x55d70c39bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.892+0000 7f7209800640 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d70c332c60 0x55d70c39bfe0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dda0515f1a183b2 server_cookie=c84341014db7ab32 in_seq=0 out_seq=0 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.892+0000 7f71eaa00640 10 client.27884 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.892+0000 7f71eaa00640 1 -- 172.21.15.18:0/2986280119 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d70c2869b0 con 0x55d70c332c60 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.892+0000 7f71eaa00640 10 client.27884 handle_command_reply: tid=0 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.892+0000 7f720e573740 1 client.27884 shutdown 2024-04-16T17:13:18.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 -- 172.21.15.18:0/2986280119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319cb0 msgr2=0x55d70c326e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319cb0 0x55d70c326e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f71f00058c0 tx=0x7f71f000f3c0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 -- 172.21.15.18:0/2986280119 shutdown_connections 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d70c332c60 0x55d70c39bfe0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d70c322890 0x55d70c398190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d70c319cb0 0x55d70c326e10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 --2- 172.21.15.18:0/2986280119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d70c319390 0x55d70c3268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f71eaa00640 0 client.27884 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 -- 172.21.15.18:0/2986280119 >> 172.21.15.18:0/2986280119 conn(0x55d70c313ac0 msgr2=0x55d70c314e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 -- 172.21.15.18:0/2986280119 shutdown_connections 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 1 -- 172.21.15.18:0/2986280119 wait complete. 2024-04-16T17:13:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.893+0000 7f720e573740 20 client.27884 trim_cache size 0 max 16384 2024-04-16T17:13:18.897 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:13:18.897 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:13:18.897 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:13:18.897 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:13:18.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 -- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71ec0a3d20 msgr2=0x7f71ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:18.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71ec0a3d20 0x7f71ec0a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f71f80eff30 tx=0x7f71f802bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 -- 172.21.15.18:0/3253400314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7204104260 msgr2=0x7f72041a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7204104260 0x7f72041a0660 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f71f802c170 tx=0x7f71f8004360 comp rx=0 tx=0).stop 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 -- 172.21.15.18:0/3253400314 shutdown_connections 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f71ec0a3d20 0x7f71ec0a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f720419a820 0x7f720419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7204109330 0x7f72041a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 --2- 172.21.15.18:0/3253400314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7204104260 0x7f72041a0660 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 -- 172.21.15.18:0/3253400314 >> 172.21.15.18:0/3253400314 conn(0x7f72040ffaa0 msgr2=0x7f7204101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 -- 172.21.15.18:0/3253400314 shutdown_connections 2024-04-16T17:13:18.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:18.894+0000 7f720a200640 1 -- 172.21.15.18:0/3253400314 wait complete. 2024-04-16T17:13:18.918 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:13:18.918 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:13:19.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 -- 172.21.15.18:0/1422145819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 msgr2=0x7f351c0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 --2- 172.21.15.18:0/1422145819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 0x7f351c0696e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3510009900 tx=0x7f351002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 -- 172.21.15.18:0/1422145819 shutdown_connections 2024-04-16T17:13:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 --2- 172.21.15.18:0/1422145819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f351c10ed00 0x7f351c1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 --2- 172.21.15.18:0/1422145819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351c069c20 0x7f351c10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 --2- 172.21.15.18:0/1422145819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 0x7f351c0696e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 -- 172.21.15.18:0/1422145819 >> 172.21.15.18:0/1422145819 conn(0x7f351c077cf0 msgr2=0x7f351c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 -- 172.21.15.18:0/1422145819 shutdown_connections 2024-04-16T17:13:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.134+0000 7f3523600640 1 -- 172.21.15.18:0/1422145819 wait complete. 2024-04-16T17:13:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.135+0000 7f3523600640 1 Processor -- start 2024-04-16T17:13:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.135+0000 7f3523600640 1 -- start start 2024-04-16T17:13:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3523600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 0x7f351c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3523600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351c069c20 0x7f351c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3523600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f351c10ed00 0x7f351c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3523600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f351c114620 con 0x7f351c069300 2024-04-16T17:13:19.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3523600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1144a0 con 0x7f351c10ed00 2024-04-16T17:13:19.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3523600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1147a0 con 0x7f351c069c20 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3520c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 0x7f351c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3520c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 0x7f351c1a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41996/0 (socket says 172.21.15.18:41996) 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3520c00640 1 -- 172.21.15.18:0/1261850927 learned_addr learned my addr 172.21.15.18:0/1261850927 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f351be00640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351c069c20 0x7f351c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3521600640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f351c10ed00 0x7f351c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3520c00640 1 -- 172.21.15.18:0/1261850927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351c069c20 msgr2=0x7f351c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3520c00640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351c069c20 0x7f351c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3520c00640 1 -- 172.21.15.18:0/1261850927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f351c10ed00 msgr2=0x7f351c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3520c00640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f351c10ed00 0x7f351c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.136+0000 7f3520c00640 1 -- 172.21.15.18:0/1261850927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c19b080 con 0x7f351c069300 2024-04-16T17:13:19.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.137+0000 7f3520c00640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 0x7f351c1a0750 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3510000c00 tx=0x7f35100043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.137+0000 7f3519600640 1 -- 172.21.15.18:0/1261850927 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510042020 con 0x7f351c069300 2024-04-16T17:13:19.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.137+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c19b310 con 0x7f351c069300 2024-04-16T17:13:19.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.137+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c1a7610 con 0x7f351c069300 2024-04-16T17:13:19.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.138+0000 7f3519600640 1 -- 172.21.15.18:0/1261850927 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35100053e0 con 0x7f351c069300 2024-04-16T17:13:19.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.138+0000 7f3519600640 1 -- 172.21.15.18:0/1261850927 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510003a10 con 0x7f351c069300 2024-04-16T17:13:19.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.139+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c068eb0 con 0x7f351c069300 2024-04-16T17:13:19.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.142+0000 7f3519600640 1 -- 172.21.15.18:0/1261850927 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3510045020 con 0x7f351c069300 2024-04-16T17:13:19.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.143+0000 7f3519600640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35040a3d20 0x7f35040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.143+0000 7f3519600640 1 -- 172.21.15.18:0/1261850927 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f35100f2230 con 0x7f351c069300 2024-04-16T17:13:19.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.144+0000 7f351be00640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35040a3d20 0x7f35040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.145+0000 7f351be00640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35040a3d20 0x7f35040a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f351c102310 tx=0x7f350800b410 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.151+0000 7f3519600640 1 -- 172.21.15.18:0/1261850927 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f351004c030 con 0x7f351c069300 2024-04-16T17:13:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.489+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f351c1a7900 con 0x7f351c069300 2024-04-16T17:13:19.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.490+0000 7f3519600640 1 -- 172.21.15.18:0/1261850927 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11851 (secure 0 0 0) 0x7f35100be570 con 0x7f351c069300 2024-04-16T17:13:19.491 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:13:19.491 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:11:38.382274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27445":{"gid":27445,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/678063310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":678063310},{"type":"v1","addr":"172.21.15.22:6837","nonce":678063310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:13:19.491 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 27 2024-04-16T17:13:19.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35040a3d20 msgr2=0x7f35040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35040a3d20 0x7f35040a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f351c102310 tx=0x7f350800b410 comp rx=0 tx=0).stop 2024-04-16T17:13:19.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 msgr2=0x7f351c1a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 0x7f351c1a0750 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3510000c00 tx=0x7f35100043c0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 shutdown_connections 2024-04-16T17:13:19.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35040a3d20 0x7f35040a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f351c10ed00 0x7f351c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351c069c20 0x7f351c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.493+0000 7f3523600640 1 --2- 172.21.15.18:0/1261850927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f351c069300 0x7f351c1a0750 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.494+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 >> 172.21.15.18:0/1261850927 conn(0x7f351c077cf0 msgr2=0x7f351c1100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.494+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 shutdown_connections 2024-04-16T17:13:19.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.494+0000 7f3523600640 1 -- 172.21.15.18:0/1261850927 wait complete. 2024-04-16T17:13:19.511 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:13:19.616 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324371 3.76 MB/sec execute 332 sec latency 11487.701 ms 2024-04-16T17:13:19.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- 172.21.15.18:0/1636222841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b60101510 msgr2=0x7f4b6010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 --2- 172.21.15.18:0/1636222841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b60101510 0x7f4b6010e1d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009900 tx=0x7f4b4c02bc50 comp rx=0 tx=0).stop 2024-04-16T17:13:19.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- 172.21.15.18:0/1636222841 shutdown_connections 2024-04-16T17:13:19.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 --2- 172.21.15.18:0/1636222841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b6010e710 0x7f4b60110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 --2- 172.21.15.18:0/1636222841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b60101510 0x7f4b6010e1d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 --2- 172.21.15.18:0/1636222841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b60100b60 0x7f4b60100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- 172.21.15.18:0/1636222841 >> 172.21.15.18:0/1636222841 conn(0x7f4b60077ab0 msgr2=0x7f4b600fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- 172.21.15.18:0/1636222841 shutdown_connections 2024-04-16T17:13:19.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- 172.21.15.18:0/1636222841 wait complete. 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 Processor -- start 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- start start 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b60100b60 0x7f4b601a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b60101510 0x7f4b601a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b6010e710 0x7f4b6019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601144f0 con 0x7f4b6010e710 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60114370 con 0x7f4b60100b60 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60114670 con 0x7f4b60101510 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b6010e710 0x7f4b6019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b6010e710 0x7f4b6019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42020/0 (socket says 172.21.15.18:42020) 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 -- 172.21.15.18:0/3055967351 learned_addr learned my addr 172.21.15.18:0/3055967351 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:19.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5de00640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b60100b60 0x7f4b601a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5d400640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b60101510 0x7f4b601a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 -- 172.21.15.18:0/3055967351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b60101510 msgr2=0x7f4b601a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b60101510 0x7f4b601a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 -- 172.21.15.18:0/3055967351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b60100b60 msgr2=0x7f4b601a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b60100b60 0x7f4b601a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 -- 172.21.15.18:0/3055967351 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6019b0b0 con 0x7f4b6010e710 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5e800640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b6010e710 0x7f4b6019a7f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800dac0 tx=0x7f4b5800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b56a00640 1 -- 172.21.15.18:0/3055967351 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58017030 con 0x7f4b6010e710 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- 172.21.15.18:0/3055967351 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6019b3a0 con 0x7f4b6010e710 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b56a00640 1 -- 172.21.15.18:0/3055967351 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b58003e50 con 0x7f4b6010e710 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b56a00640 1 -- 172.21.15.18:0/3055967351 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58010460 con 0x7f4b6010e710 2024-04-16T17:13:19.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.692+0000 7f4b5f200640 1 -- 172.21.15.18:0/3055967351 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b601a7660 con 0x7f4b6010e710 2024-04-16T17:13:19.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b67992740 1 Processor -- start 2024-04-16T17:13:19.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b56a00640 1 -- 172.21.15.18:0/3055967351 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b5801a070 con 0x7f4b6010e710 2024-04-16T17:13:19.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b67992740 1 -- start start 2024-04-16T17:13:19.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b67992740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b67992740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d1ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b67992740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 0x55dc8d202800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b67992740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dc8d15c020 con 0x55dc8d1f7830 2024-04-16T17:13:19.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b67992740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dc8d102460 con 0x55dc8d1f6f10 2024-04-16T17:13:19.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.697+0000 7f4b67992740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc8d157fe0 con 0x55dc8d200410 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59910/0 (socket says 172.21.15.18:59910) 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 -- 172.21.15.18:0/2249129564 learned_addr learned my addr 172.21.15.18:0/2249129564 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 0x55dc8d202800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5de00640 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d1ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 -- 172.21.15.18:0/2249129564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 msgr2=0x55dc8d202800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 0x55dc8d202800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 -- 172.21.15.18:0/2249129564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 msgr2=0x55dc8d1ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d1ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 -- 172.21.15.18:0/2249129564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8d10a040 con 0x55dc8d1f6f10 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5de00640 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d1ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:19.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5d400640 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1f72f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009230 tx=0x7f4b4c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55e04fe91bbcb309 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b56a00640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4b440a3d20 0x7f4b440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b4b000640 1 -- 172.21.15.18:0/2249129564 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c02c8a0 con 0x55dc8d1f6f10 2024-04-16T17:13:19.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b4b000640 1 -- 172.21.15.18:0/2249129564 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b4c004ba0 con 0x55dc8d1f6f10 2024-04-16T17:13:19.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b4b000640 1 -- 172.21.15.18:0/2249129564 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c005480 con 0x55dc8d1f6f10 2024-04-16T17:13:19.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b56a00640 1 -- 172.21.15.18:0/3055967351 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f4b580d0930 con 0x7f4b6010e710 2024-04-16T17:13:19.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.702+0000 7f4b5de00640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4b440a3d20 0x7f4b440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- 172.21.15.18:0/2249129564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 msgr2=0x55dc8d1f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1f72f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009230 tx=0x7f4b4c009560 comp rx=0 tx=0).stop 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- 172.21.15.18:0/2249129564 shutdown_connections 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4b440a3d20 0x7f4b440a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55dc8d1f69e0 tx=0x7f4b50008040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 0x55dc8d202800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d1ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 --2- 172.21.15.18:0/2249129564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1f72f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- 172.21.15.18:0/2249129564 >> 172.21.15.18:0/2249129564 conn(0x55dc8d1f1640 msgr2=0x55dc8d1f1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- 172.21.15.18:0/2249129564 shutdown_connections 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- 172.21.15.18:0/2249129564 wait complete. 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- start start 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 Processor -- start 2024-04-16T17:13:19.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f7830 0x55dc8d1fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d200410 0x55dc8d1fded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dc8d2031a0 con 0x55dc8d200410 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dc8d115830 con 0x55dc8d1f6f10 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc8d203320 con 0x55dc8d1f7830 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 10 client.0 _ll_register_callbacks cb 0x55dc8d11d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59920/0 (socket says 172.21.15.18:59920) 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 -- 172.21.15.18:0/614438233 learned_addr learned my addr 172.21.15.18:0/614438233 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5e800640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d200410 0x55dc8d1fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5d400640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f7830 0x55dc8d1fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 -- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f7830 msgr2=0x55dc8d1fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:19.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f7830 0x55dc8d1fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 -- 172.21.15.18:0/614438233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d200410 msgr2=0x55dc8d1fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:19.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d200410 0x55dc8d1fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 -- 172.21.15.18:0/614438233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8d119920 con 0x55dc8d1f6f10 2024-04-16T17:13:19.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5e800640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d200410 0x55dc8d1fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:19.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5d400640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f7830 0x55dc8d1fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:19.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b5de00640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1fd450 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4b500156c0 tx=0x7f4b500046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=920da5b3b3bf21be server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b3f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:19.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b3f400640 1 -- 172.21.15.18:0/614438233 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50025020 con 0x55dc8d1f6f10 2024-04-16T17:13:19.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b3f400640 1 -- 172.21.15.18:0/614438233 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f4b500144a0 con 0x55dc8d1f6f10 2024-04-16T17:13:19.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- 172.21.15.18:0/614438233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b4c004ba0 con 0x55dc8d1f6f10 2024-04-16T17:13:19.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b3f400640 1 -- 172.21.15.18:0/614438233 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b50030050 con 0x55dc8d1f6f10 2024-04-16T17:13:19.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 10 client.27899 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:19.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- 172.21.15.18:0/614438233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc8d0e77f0 con 0x55dc8d1f6f10 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b3f400640 1 -- 172.21.15.18:0/614438233 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b4c004ba0 con 0x55dc8d1f6f10 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b3f400640 1 -- 172.21.15.18:0/614438233 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4b50021a30 con 0x55dc8d1f6f10 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 10 client.27899 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 10 client.27899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 20 client.27899 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55dc8d2107b0 0x55dc8d212c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 4 client.27899 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.705+0000 7f4b67992740 1 -- 172.21.15.18:0/614438233 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc8d169a40 con 0x55dc8d2107b0 2024-04-16T17:13:19.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.710+0000 7f4b5d400640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55dc8d2107b0 0x55dc8d212c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.710+0000 7f4b3f400640 10 client.27899 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:13:19.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b5d400640 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55dc8d2107b0 0x55dc8d212c70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ac517c6982411a3 server_cookie=9fb22bde55f90d0 in_seq=0 out_seq=0 2024-04-16T17:13:19.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b3f400640 1 -- 172.21.15.18:0/614438233 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc8d169a40 con 0x55dc8d2107b0 2024-04-16T17:13:19.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b3f400640 10 client.27899 handle_command_reply: tid=0 2024-04-16T17:13:19.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 client.27899 shutdown 2024-04-16T17:13:19.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 -- 172.21.15.18:0/614438233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 msgr2=0x55dc8d1fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1fd450 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4b500156c0 tx=0x7f4b500046e0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 -- 172.21.15.18:0/614438233 shutdown_connections 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55dc8d2107b0 0x55dc8d212c70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d200410 0x55dc8d1fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f7830 0x55dc8d1fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 --2- 172.21.15.18:0/614438233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1fd450 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b3f400640 0 client.27899 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 -- 172.21.15.18:0/614438233 >> 172.21.15.18:0/614438233 conn(0x55dc8d1f1640 msgr2=0x55dc8d1f3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 -- 172.21.15.18:0/614438233 shutdown_connections 2024-04-16T17:13:19.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 1 -- 172.21.15.18:0/614438233 wait complete. 2024-04-16T17:13:19.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.712+0000 7f4b67992740 20 client.27899 trim_cache size 0 max 16384 2024-04-16T17:13:19.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.746+0000 7f4b67992740 1 Processor -- start 2024-04-16T17:13:19.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.746+0000 7f4b67992740 1 -- start start 2024-04-16T17:13:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.746+0000 7f4b67992740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.746+0000 7f4b67992740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d20c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.746+0000 7f4b67992740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 0x55dc8d218540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.746+0000 7f4b67992740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601b4020 con 0x55dc8d1f7830 2024-04-16T17:13:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.746+0000 7f4b67992740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601b18f0 con 0x55dc8d1f6f10 2024-04-16T17:13:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.746+0000 7f4b67992740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601af760 con 0x55dc8d200410 2024-04-16T17:13:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.747+0000 7f4b5de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.747+0000 7f4b5de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1ffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59922/0 (socket says 172.21.15.18:59922) 2024-04-16T17:13:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.747+0000 7f4b5de00640 1 -- 172.21.15.18:0/2608041579 learned_addr learned my addr 172.21.15.18:0/2608041579 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.747+0000 7f4b5e800640 1 --2- 172.21.15.18:0/2608041579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 0x55dc8d218540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.748+0000 7f4b5de00640 1 -- 172.21.15.18:0/2608041579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 msgr2=0x55dc8d218540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.748+0000 7f4b5de00640 1 --2- 172.21.15.18:0/2608041579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 0x55dc8d218540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.748+0000 7f4b5de00640 1 -- 172.21.15.18:0/2608041579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 msgr2=0x55dc8d20c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.748+0000 7f4b5de00640 1 --2- 172.21.15.18:0/2608041579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d20c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.748+0000 7f4b5de00640 1 -- 172.21.15.18:0/2608041579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8ce6e3f0 con 0x55dc8d1f6f10 2024-04-16T17:13:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.748+0000 7f4b5de00640 1 --2- 172.21.15.18:0/2608041579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1ffb80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4b500116a0 tx=0x7f4b50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400c1b7d3592316d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.749+0000 7f4b4a600640 1 -- 172.21.15.18:0/2608041579 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50025050 con 0x55dc8d1f6f10 2024-04-16T17:13:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.749+0000 7f4b4a600640 1 -- 172.21.15.18:0/2608041579 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b5001a030 con 0x55dc8d1f6f10 2024-04-16T17:13:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.749+0000 7f4b4a600640 1 -- 172.21.15.18:0/2608041579 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50020e10 con 0x55dc8d1f6f10 2024-04-16T17:13:19.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.750+0000 7f4b67992740 1 -- 172.21.15.18:0/2608041579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 msgr2=0x55dc8d1ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.750+0000 7f4b67992740 1 --2- 172.21.15.18:0/2608041579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1ffb80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4b500116a0 tx=0x7f4b50004750 comp rx=0 tx=0).stop 2024-04-16T17:13:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.750+0000 7f4b67992740 1 -- 172.21.15.18:0/2608041579 shutdown_connections 2024-04-16T17:13:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.750+0000 7f4b67992740 1 --2- 172.21.15.18:0/2608041579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d200410 0x55dc8d218540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.750+0000 7f4b67992740 1 --2- 172.21.15.18:0/2608041579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d20c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.750+0000 7f4b67992740 1 --2- 172.21.15.18:0/2608041579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d1f6f10 0x55dc8d1ffb80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.750+0000 7f4b67992740 1 -- 172.21.15.18:0/2608041579 >> 172.21.15.18:0/2608041579 conn(0x55dc8d1f1640 msgr2=0x55dc8d1f1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.751+0000 7f4b67992740 1 -- 172.21.15.18:0/2608041579 shutdown_connections 2024-04-16T17:13:19.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.751+0000 7f4b67992740 1 -- 172.21.15.18:0/2608041579 wait complete. 2024-04-16T17:13:19.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.752+0000 7f4b67992740 1 -- start start 2024-04-16T17:13:19.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.752+0000 7f4b67992740 1 Processor -- start 2024-04-16T17:13:19.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.753+0000 7f4b67992740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f6f10 0x55dc8d208720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b67992740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d205a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b67992740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d200410 0x55dc8d205f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b67992740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601afdb0 con 0x55dc8d1f7830 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b67992740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601b1220 con 0x55dc8d200410 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b67992740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601b0480 con 0x55dc8d1f6f10 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b67992740 10 client.0 _ll_register_callbacks cb 0x7f4b500097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b5e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d200410 0x55dc8d205f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b5e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d200410 0x55dc8d205f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59936/0 (socket says 172.21.15.18:59936) 2024-04-16T17:13:19.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.754+0000 7f4b5e800640 1 -- 172.21.15.18:0/4228036113 learned_addr learned my addr 172.21.15.18:0/4228036113 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:19.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.755+0000 7f4b5e800640 1 -- 172.21.15.18:0/4228036113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f6f10 msgr2=0x55dc8d208720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:13:19.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.755+0000 7f4b5e800640 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f6f10 0x55dc8d208720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.755+0000 7f4b5e800640 1 -- 172.21.15.18:0/4228036113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 msgr2=0x55dc8d205a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:19.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.755+0000 7f4b5e800640 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d205a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.755+0000 7f4b5e800640 1 -- 172.21.15.18:0/4228036113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8cf387d0 con 0x55dc8d200410 2024-04-16T17:13:19.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.756+0000 7f4b5e800640 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d200410 0x55dc8d205f80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4b580cdcd0 tx=0x7f4b5809cd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26494a3c122cd3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.756+0000 7f4b3f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:19.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.756+0000 7f4b3f400640 1 -- 172.21.15.18:0/4228036113 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58098b30 con 0x55dc8d200410 2024-04-16T17:13:19.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.756+0000 7f4b3f400640 1 -- 172.21.15.18:0/4228036113 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f4b5800bc30 con 0x55dc8d200410 2024-04-16T17:13:19.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.756+0000 7f4b67992740 1 -- 172.21.15.18:0/4228036113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b5001a030 con 0x55dc8d200410 2024-04-16T17:13:19.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b3f400640 1 -- 172.21.15.18:0/4228036113 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b58098cd0 con 0x55dc8d200410 2024-04-16T17:13:19.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b67992740 10 client.27905 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:19.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b67992740 1 -- 172.21.15.18:0/4228036113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc8d27f5c0 con 0x55dc8d200410 2024-04-16T17:13:19.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b3f400640 1 -- 172.21.15.18:0/4228036113 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b580bb2d0 con 0x55dc8d200410 2024-04-16T17:13:19.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b3f400640 1 -- 172.21.15.18:0/4228036113 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4b58024060 con 0x55dc8d200410 2024-04-16T17:13:19.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b67992740 10 client.27905 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:19.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b67992740 10 client.27905 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:13:19.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b67992740 20 client.27905 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:19.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b67992740 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55dc8d27fbc0 0x55dc8d282040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b67992740 4 client.27905 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:19.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.757+0000 7f4b67992740 1 -- 172.21.15.18:0/4228036113 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4b5001a030 con 0x55dc8d27fbc0 2024-04-16T17:13:19.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.765+0000 7f4b5de00640 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55dc8d27fbc0 0x55dc8d282040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.781+0000 7f4b5de00640 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55dc8d27fbc0 0x55dc8d282040 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf3c0e8822cf305f server_cookie=dd259d1dac493801 in_seq=0 out_seq=0 2024-04-16T17:13:19.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.781+0000 7f4b3f400640 10 client.27905 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:13:19.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.788+0000 7f4b3f400640 1 -- 172.21.15.18:0/4228036113 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4b5001a030 con 0x55dc8d27fbc0 2024-04-16T17:13:19.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.788+0000 7f4b3f400640 10 client.27905 handle_command_reply: tid=0 2024-04-16T17:13:19.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.788+0000 7f4b67992740 1 client.27905 shutdown 2024-04-16T17:13:19.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 -- 172.21.15.18:0/4228036113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d200410 msgr2=0x55dc8d205f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d200410 0x55dc8d205f80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4b580cdcd0 tx=0x7f4b5809cd00 comp rx=0 tx=0).stop 2024-04-16T17:13:19.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 -- 172.21.15.18:0/4228036113 shutdown_connections 2024-04-16T17:13:19.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55dc8d27fbc0 0x55dc8d282040 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc8d200410 0x55dc8d205f80 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dc8d1f7830 0x55dc8d205a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 --2- 172.21.15.18:0/4228036113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc8d1f6f10 0x55dc8d208720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 -- 172.21.15.18:0/4228036113 >> 172.21.15.18:0/4228036113 conn(0x55dc8d1f1640 msgr2=0x55dc8d1f2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b3f400640 0 client.27905 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:13:19.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 -- 172.21.15.18:0/4228036113 shutdown_connections 2024-04-16T17:13:19.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 1 -- 172.21.15.18:0/4228036113 wait complete. 2024-04-16T17:13:19.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.789+0000 7f4b67992740 20 client.27905 trim_cache size 0 max 16384 2024-04-16T17:13:19.791 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:13:19.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 -- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4b440a3d20 msgr2=0x7f4b440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4b440a3d20 0x7f4b440a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55dc8d1f69e0 tx=0x7f4b50008040 comp rx=0 tx=0).stop 2024-04-16T17:13:19.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 -- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b6010e710 msgr2=0x7f4b6019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b6010e710 0x7f4b6019a7f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800dac0 tx=0x7f4b5800df80 comp rx=0 tx=0).stop 2024-04-16T17:13:19.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 -- 172.21.15.18:0/3055967351 shutdown_connections 2024-04-16T17:13:19.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4b440a3d20 0x7f4b440a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b6010e710 0x7f4b6019a7f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b60101510 0x7f4b601a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 --2- 172.21.15.18:0/3055967351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b60100b60 0x7f4b601a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 -- 172.21.15.18:0/3055967351 >> 172.21.15.18:0/3055967351 conn(0x7f4b60077ab0 msgr2=0x7f4b600ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.790+0000 7f4b5f200640 1 -- 172.21.15.18:0/3055967351 shutdown_connections 2024-04-16T17:13:19.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.791+0000 7f4b5f200640 1 -- 172.21.15.18:0/3055967351 wait complete. 2024-04-16T17:13:19.808 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:13:19.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 -- 172.21.15.18:0/2106945565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbf0104440 msgr2=0x7fbbf0104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2106945565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbf0104440 0x7fbbf0104820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009900 tx=0x7fbbe002bc40 comp rx=0 tx=0).stop 2024-04-16T17:13:19.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 -- 172.21.15.18:0/2106945565 shutdown_connections 2024-04-16T17:13:19.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2106945565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbf0109510 0x7fbbf01110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2106945565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbf0104df0 0x7fbbf0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2106945565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbf0104440 0x7fbbf0104820 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 -- 172.21.15.18:0/2106945565 >> 172.21.15.18:0/2106945565 conn(0x7fbbf00ffc80 msgr2=0x7fbbf01020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 -- 172.21.15.18:0/2106945565 shutdown_connections 2024-04-16T17:13:19.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.993+0000 7fbbf5800640 1 -- 172.21.15.18:0/2106945565 wait complete. 2024-04-16T17:13:19.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.994+0000 7fbbf5800640 1 Processor -- start 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.994+0000 7fbbf5800640 1 -- start start 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbf5800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbf0104440 0x7fbbf01a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbf5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbf0109510 0x7fbbf01a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbf5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbf019a9d0 0x7fbbf019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbf5800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01145c0 con 0x7fbbf0104440 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbf5800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0114440 con 0x7fbbf0109510 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbf5800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0114740 con 0x7fbbf019a9d0 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbf0104440 0x7fbbf01a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbf0104440 0x7fbbf01a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42066/0 (socket says 172.21.15.18:42066) 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbeea00640 1 -- 172.21.15.18:0/2625095527 learned_addr learned my addr 172.21.15.18:0/2625095527 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbee000640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbf0109510 0x7fbbf01a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbef400640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbf019a9d0 0x7fbbf019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:19.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbeea00640 1 -- 172.21.15.18:0/2625095527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbf019a9d0 msgr2=0x7fbbf019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbeea00640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbf019a9d0 0x7fbbf019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbeea00640 1 -- 172.21.15.18:0/2625095527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbf0109510 msgr2=0x7fbbf01a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbeea00640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbf0109510 0x7fbbf01a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.995+0000 7fbbeea00640 1 -- 172.21.15.18:0/2625095527 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf019b4f0 con 0x7fbbf0104440 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.996+0000 7fbbeea00640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbf0104440 0x7fbbf01a0820 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0006010 tx=0x7fbbe002cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.996+0000 7fbbe7400640 1 -- 172.21.15.18:0/2625095527 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0041070 con 0x7fbbf0104440 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.996+0000 7fbbe7400640 1 -- 172.21.15.18:0/2625095527 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe0004450 con 0x7fbbf0104440 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.996+0000 7fbbe7400640 1 -- 172.21.15.18:0/2625095527 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0003e30 con 0x7fbbf0104440 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.996+0000 7fbbf5800640 1 -- 172.21.15.18:0/2625095527 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf019b780 con 0x7fbbf0104440 2024-04-16T17:13:19.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:19.996+0000 7fbbf5800640 1 -- 172.21.15.18:0/2625095527 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf01a79d0 con 0x7fbbf0104440 2024-04-16T17:13:20.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.000+0000 7fbbe7400640 1 -- 172.21.15.18:0/2625095527 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbe0044070 con 0x7fbbf0104440 2024-04-16T17:13:20.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.000+0000 7fbbf85b8740 1 Processor -- start 2024-04-16T17:13:20.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.000+0000 7fbbf85b8740 1 -- start start 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa33c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa33c670 0x55a0fa344d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa347640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a0fa2a1020 con 0x55a0fa33bd50 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a0fa247460 con 0x55a0fa33c670 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a0fa29cfe0 con 0x55a0fa345250 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbef400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa33c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbef400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa33c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42082/0 (socket says 172.21.15.18:42082) 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbef400640 1 -- 172.21.15.18:0/165385954 learned_addr learned my addr 172.21.15.18:0/165385954 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbeea00640 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa33c670 0x55a0fa344d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbe7400640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbd40a3da0 0x7fbbd40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.001+0000 7fbbee000640 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa347640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbe7400640 1 -- 172.21.15.18:0/2625095527 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fbbe00f2980 con 0x7fbbf0104440 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbef400640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbd40a3da0 0x7fbbd40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbee000640 1 -- 172.21.15.18:0/165385954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa33c670 msgr2=0x55a0fa344d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbee000640 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa33c670 0x55a0fa344d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbee000640 1 -- 172.21.15.18:0/165385954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 msgr2=0x55a0fa33c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbee000640 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa33c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbee000640 1 -- 172.21.15.18:0/165385954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0fa24f040 con 0x55a0fa345250 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbef400640 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa33c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbee000640 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa347640 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbbf019c1c0 tx=0x7fbbdc0113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4dca94252690d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbef400640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbd40a3da0 0x7fbbd40a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80049d0 tx=0x7fbbe8004ab0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.002+0000 7fbbdbe00640 1 -- 172.21.15.18:0/165385954 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc011de0 con 0x55a0fa345250 2024-04-16T17:13:20.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.003+0000 7fbbdbe00640 1 -- 172.21.15.18:0/165385954 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc012930 con 0x55a0fa345250 2024-04-16T17:13:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.003+0000 7fbbdbe00640 1 -- 172.21.15.18:0/165385954 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc00fad0 con 0x55a0fa345250 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.003+0000 7fbbf85b8740 1 -- 172.21.15.18:0/165385954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 msgr2=0x55a0fa347640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.003+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa347640 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbbf019c1c0 tx=0x7fbbdc0113e0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 -- 172.21.15.18:0/165385954 shutdown_connections 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa347640 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa33c670 0x55a0fa344d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/165385954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa33c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 -- 172.21.15.18:0/165385954 >> 172.21.15.18:0/165385954 conn(0x55a0fa336480 msgr2=0x55a0fa336860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 -- 172.21.15.18:0/165385954 shutdown_connections 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 -- 172.21.15.18:0/165385954 wait complete. 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 -- start start 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.004+0000 7fbbf85b8740 1 Processor -- start 2024-04-16T17:13:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa342170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa33c670 0x55a0fa3426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa345250 0x55a0fa342bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a0fa348040 con 0x55a0fa33bd50 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a0fa25a830 con 0x55a0fa345250 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a0fa3481c0 con 0x55a0fa33c670 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbf85b8740 10 client.0 _ll_register_callbacks cb 0x55a0fa262150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa342170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa342170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42096/0 (socket says 172.21.15.18:42096) 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbeea00640 1 -- 172.21.15.18:0/1079432069 learned_addr learned my addr 172.21.15.18:0/1079432069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.005+0000 7fbbee000640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa33c670 0x55a0fa3426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbef400640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa345250 0x55a0fa342bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbeea00640 1 -- 172.21.15.18:0/1079432069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa33c670 msgr2=0x55a0fa3426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbeea00640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa33c670 0x55a0fa3426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbeea00640 1 -- 172.21.15.18:0/1079432069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa345250 msgr2=0x55a0fa342bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbeea00640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa345250 0x55a0fa342bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbeea00640 1 -- 172.21.15.18:0/1079432069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0fa25e920 con 0x55a0fa33bd50 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbee000640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa33c670 0x55a0fa3426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbef400640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa345250 0x55a0fa342bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbeea00640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa342170 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0003fd0 tx=0x7fbbe0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307d019efbf789ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbd8c00640 1 -- 172.21.15.18:0/1079432069 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe004b040 con 0x55a0fa33bd50 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbd8c00640 1 -- 172.21.15.18:0/1079432069 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fbbe00bea20 con 0x55a0fa33bd50 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.006+0000 7fbbf85b8740 1 -- 172.21.15.18:0/1079432069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc012930 con 0x55a0fa33bd50 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbd8c00640 1 -- 172.21.15.18:0/1079432069 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbe00be390 con 0x55a0fa33bd50 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbf85b8740 10 client.17634 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:20.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbf85b8740 1 -- 172.21.15.18:0/1079432069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0fa22c7f0 con 0x55a0fa33bd50 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbd8c00640 1 -- 172.21.15.18:0/1079432069 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc012930 con 0x55a0fa33bd50 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbd8c00640 1 -- 172.21.15.18:0/1079432069 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbbe00c6da0 con 0x55a0fa33bd50 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbf85b8740 10 client.17634 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbf85b8740 10 client.17634 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbf85b8740 20 client.17634 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a0fa355460 0x55a0fa357890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbf85b8740 4 client.17634 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.007+0000 7fbbf85b8740 1 -- 172.21.15.18:0/1079432069 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0fa2aea40 con 0x55a0fa355460 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.008+0000 7fbbee000640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a0fa355460 0x55a0fa357890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.008+0000 7fbbd8c00640 10 client.17634 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:13:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.008+0000 7fbbee000640 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a0fa355460 0x55a0fa357890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97482d70aa205c63 server_cookie=57bcfd05921c54af in_seq=0 out_seq=0 2024-04-16T17:13:20.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.009+0000 7fbbd8c00640 1 -- 172.21.15.18:0/1079432069 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0fa2aea40 con 0x55a0fa355460 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.009+0000 7fbbd8c00640 10 client.17634 handle_command_reply: tid=0 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.009+0000 7fbbf85b8740 1 client.17634 shutdown 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.009+0000 7fbbf85b8740 1 -- 172.21.15.18:0/1079432069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 msgr2=0x55a0fa342170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.009+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa342170 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0003fd0 tx=0x7fbbe0005bd0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 1 -- 172.21.15.18:0/1079432069 shutdown_connections 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a0fa355460 0x55a0fa357890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa345250 0x55a0fa342bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa33c670 0x55a0fa3426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/1079432069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33bd50 0x55a0fa342170 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbd8c00640 0 client.17634 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 1 -- 172.21.15.18:0/1079432069 >> 172.21.15.18:0/1079432069 conn(0x55a0fa336480 msgr2=0x55a0fa3381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 1 -- 172.21.15.18:0/1079432069 shutdown_connections 2024-04-16T17:13:20.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 1 -- 172.21.15.18:0/1079432069 wait complete. 2024-04-16T17:13:20.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.010+0000 7fbbf85b8740 20 client.17634 trim_cache size 0 max 16384 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.035+0000 7fbbf85b8740 1 Processor -- start 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.035+0000 7fbbf85b8740 1 -- start start 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa3448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa350f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 0x55a0fa35d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01b4070 con 0x55a0fa33c670 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01b1940 con 0x55a0fa35d170 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01af7b0 con 0x55a0fa345250 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa3448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa3448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42104/0 (socket says 172.21.15.18:42104) 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbeea00640 1 -- 172.21.15.18:0/3711588582 learned_addr learned my addr 172.21.15.18:0/3711588582 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbef400640 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 0x55a0fa35d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.036+0000 7fbbee000640 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa350f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbeea00640 1 -- 172.21.15.18:0/3711588582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 msgr2=0x55a0fa350f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbeea00640 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa350f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbeea00640 1 -- 172.21.15.18:0/3711588582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 msgr2=0x55a0fa35d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbeea00640 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 0x55a0fa35d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbeea00640 1 -- 172.21.15.18:0/3711588582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f9fb33f0 con 0x55a0fa33c670 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbee000640 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa350f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbeea00640 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa3448a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0000c00 tx=0x7fbbe00ba1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96976126fea53d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbdb400640 1 -- 172.21.15.18:0/3711588582 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00dc070 con 0x55a0fa33c670 2024-04-16T17:13:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbdb400640 1 -- 172.21.15.18:0/3711588582 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe00c7230 con 0x55a0fa33c670 2024-04-16T17:13:20.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.037+0000 7fbbdb400640 1 -- 172.21.15.18:0/3711588582 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe004b070 con 0x55a0fa33c670 2024-04-16T17:13:20.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 -- 172.21.15.18:0/3711588582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 msgr2=0x55a0fa3448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa3448a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0000c00 tx=0x7fbbe00ba1a0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 -- 172.21.15.18:0/3711588582 shutdown_connections 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 0x55a0fa35d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa350f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/3711588582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa3448a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 -- 172.21.15.18:0/3711588582 >> 172.21.15.18:0/3711588582 conn(0x55a0fa336480 msgr2=0x55a0fa336860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 -- 172.21.15.18:0/3711588582 shutdown_connections 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.038+0000 7fbbf85b8740 1 -- 172.21.15.18:0/3711588582 wait complete. 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.039+0000 7fbbf85b8740 1 -- start start 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.039+0000 7fbbf85b8740 1 Processor -- start 2024-04-16T17:13:20.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa33f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa33f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbf85b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 0x55a0fa33fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01afe00 con 0x55a0fa33c670 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01b1270 con 0x55a0fa35d170 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbf85b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01b04d0 con 0x55a0fa345250 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbf85b8740 10 client.0 _ll_register_callbacks cb 0x7fbbe0005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa33f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa33f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42114/0 (socket says 172.21.15.18:42114) 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbeea00640 1 -- 172.21.15.18:0/719611499 learned_addr learned my addr 172.21.15.18:0/719611499 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbee000640 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa33f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbeea00640 1 -- 172.21.15.18:0/719611499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 msgr2=0x55a0fa33f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbeea00640 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa33f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbeea00640 1 -- 172.21.15.18:0/719611499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 msgr2=0x55a0fa33fb80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.040+0000 7fbbeea00640 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 0x55a0fa33fb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.041+0000 7fbbeea00640 1 -- 172.21.15.18:0/719611499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0fa07d7d0 con 0x55a0fa33c670 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.041+0000 7fbbeea00640 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa33f100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005940 tx=0x7fbbe00ba910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48928ee8797f99bf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.041+0000 7fbbd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.041+0000 7fbbd8c00640 1 -- 172.21.15.18:0/719611499 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00dc070 con 0x55a0fa33c670 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.041+0000 7fbbd8c00640 1 -- 172.21.15.18:0/719611499 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fbbe00bea20 con 0x55a0fa33c670 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.041+0000 7fbbf85b8740 1 -- 172.21.15.18:0/719611499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe00c7230 con 0x55a0fa33c670 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.041+0000 7fbbd8c00640 1 -- 172.21.15.18:0/719611499 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbe004b070 con 0x55a0fa33c670 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.042+0000 7fbbf85b8740 10 client.17640 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.042+0000 7fbbf85b8740 1 -- 172.21.15.18:0/719611499 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0fa07d7d0 con 0x55a0fa33c670 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.042+0000 7fbbd8c00640 1 -- 172.21.15.18:0/719611499 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe00c7230 con 0x55a0fa33c670 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.043+0000 7fbbd8c00640 1 -- 172.21.15.18:0/719611499 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbbe00bd480 con 0x55a0fa33c670 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.043+0000 7fbbf85b8740 10 client.17640 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.043+0000 7fbbf85b8740 10 client.17640 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.043+0000 7fbbf85b8740 20 client.17640 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.043+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a0fa3bc040 0x55a0fa3be430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.043+0000 7fbbf85b8740 4 client.17640 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.043+0000 7fbbf85b8740 1 -- 172.21.15.18:0/719611499 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0fa2a88b0 con 0x55a0fa3bc040 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.043+0000 7fbbee000640 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a0fa3bc040 0x55a0fa3be430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbee000640 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a0fa3bc040 0x55a0fa3be430 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4795b2684b921a16 server_cookie=cbdba9c755638bc4 in_seq=0 out_seq=0 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbd8c00640 10 client.17640 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbd8c00640 1 -- 172.21.15.18:0/719611499 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0fa2a88b0 con 0x55a0fa3bc040 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbd8c00640 10 client.17640 handle_command_reply: tid=0 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 client.17640 shutdown 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 -- 172.21.15.18:0/719611499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 msgr2=0x55a0fa33f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa33f100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005940 tx=0x7fbbe00ba910 comp rx=0 tx=0).stop 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 -- 172.21.15.18:0/719611499 shutdown_connections 2024-04-16T17:13:20.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a0fa3bc040 0x55a0fa3be430 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0fa35d170 0x55a0fa33fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0fa345250 0x55a0fa33f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 --2- 172.21.15.18:0/719611499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a0fa33c670 0x55a0fa33f100 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbd8c00640 0 client.17640 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:13:20.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.044+0000 7fbbf85b8740 1 -- 172.21.15.18:0/719611499 >> 172.21.15.18:0/719611499 conn(0x55a0fa336480 msgr2=0x55a0fa334c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.045+0000 7fbbf85b8740 1 -- 172.21.15.18:0/719611499 shutdown_connections 2024-04-16T17:13:20.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.045+0000 7fbbf85b8740 1 -- 172.21.15.18:0/719611499 wait complete. 2024-04-16T17:13:20.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.045+0000 7fbbf85b8740 20 client.17640 trim_cache size 0 max 16384 2024-04-16T17:13:20.051 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 -- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbd40a3da0 msgr2=0x7fbbd40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbd40a3da0 0x7fbbd40a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80049d0 tx=0x7fbbe8004ab0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 -- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbf0104440 msgr2=0x7fbbf01a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbf0104440 0x7fbbf01a0820 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0006010 tx=0x7fbbe002cb40 comp rx=0 tx=0).stop 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 -- 172.21.15.18:0/2625095527 shutdown_connections 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbd40a3da0 0x7fbbd40a6260 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbf019a9d0 0x7fbbf019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbf0109510 0x7fbbf01a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 --2- 172.21.15.18:0/2625095527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbf0104440 0x7fbbf01a0820 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 -- 172.21.15.18:0/2625095527 >> 172.21.15.18:0/2625095527 conn(0x7fbbf00ffc80 msgr2=0x7fbbf0100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 -- 172.21.15.18:0/2625095527 shutdown_connections 2024-04-16T17:13:20.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.046+0000 7fbbf5800640 1 -- 172.21.15.18:0/2625095527 wait complete. 2024-04-16T17:13:20.066 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:13:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:19 smithi022 ceph-mon[36023]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-04-16T17:13:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:19 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1261850927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:20.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:19 smithi043 ceph-mon[36901]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-04-16T17:13:20.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:19 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1261850927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 -- 172.21.15.18:0/2569755761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b0109520 msgr2=0x7f97b01110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 --2- 172.21.15.18:0/2569755761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b0109520 0x7f97b01110f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009ca0 tx=0x7f97a802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 -- 172.21.15.18:0/2569755761 shutdown_connections 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 --2- 172.21.15.18:0/2569755761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b0109520 0x7f97b01110f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 --2- 172.21.15.18:0/2569755761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0104e00 0x7f97b0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 --2- 172.21.15.18:0/2569755761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97b0104450 0x7f97b0104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 -- 172.21.15.18:0/2569755761 >> 172.21.15.18:0/2569755761 conn(0x7f97b00ffc90 msgr2=0x7f97b01020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 -- 172.21.15.18:0/2569755761 shutdown_connections 2024-04-16T17:13:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.255+0000 7f97b6800640 1 -- 172.21.15.18:0/2569755761 wait complete. 2024-04-16T17:13:20.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.256+0000 7f97b6800640 1 Processor -- start 2024-04-16T17:13:20.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.256+0000 7f97b6800640 1 -- start start 2024-04-16T17:13:20.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97b6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97b0104450 0x7f97b01a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97b6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0109520 0x7f97b01a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97b6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019a8f0 0x7f97b019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97b6800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01145b0 con 0x7f97b0104450 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97b6800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f97b0114430 con 0x7f97b0109520 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97b6800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97b0114730 con 0x7f97b019a8f0 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0109520 0x7f97b01a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97af400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97b0104450 0x7f97b01a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0109520 0x7f97b01a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60006/0 (socket says 172.21.15.18:60006) 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97aea00640 1 -- 172.21.15.18:0/2570311547 learned_addr learned my addr 172.21.15.18:0/2570311547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97afe00640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019a8f0 0x7f97b019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97aea00640 1 -- 172.21.15.18:0/2570311547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019a8f0 msgr2=0x7f97b019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97aea00640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019a8f0 0x7f97b019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97aea00640 1 -- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97b0104450 msgr2=0x7f97b01a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97aea00640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97b0104450 0x7f97b01a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.257+0000 7f97aea00640 1 -- 172.21.15.18:0/2570311547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b019b410 con 0x7f97b0109520 2024-04-16T17:13:20.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.258+0000 7f97aea00640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0109520 0x7f97b01a0c30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f979c005fd0 tx=0x7f979c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.258+0000 7f97a7e00640 1 -- 172.21.15.18:0/2570311547 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c017070 con 0x7f97b0109520 2024-04-16T17:13:20.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.258+0000 7f97a7e00640 1 -- 172.21.15.18:0/2570311547 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979c009c30 con 0x7f97b0109520 2024-04-16T17:13:20.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.258+0000 7f97a7e00640 1 -- 172.21.15.18:0/2570311547 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c010460 con 0x7f97b0109520 2024-04-16T17:13:20.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.258+0000 7f97b6800640 1 -- 172.21.15.18:0/2570311547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b019b700 con 0x7f97b0109520 2024-04-16T17:13:20.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.258+0000 7f97b6800640 1 -- 172.21.15.18:0/2570311547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b01a7950 con 0x7f97b0109520 2024-04-16T17:13:20.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.261+0000 7f97b95dd740 1 Processor -- start 2024-04-16T17:13:20.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97b95dd740 1 -- start start 2024-04-16T17:13:20.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a850a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8512d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 0x5629a8515640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97b95dd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5629a846f020 con 0x5629a850a670 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97b95dd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629a8415460 con 0x5629a8513250 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97b95dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629a846afe0 con 0x5629a8509d50 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97a7e00640 1 -- 172.21.15.18:0/2570311547 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f979c003e70 con 0x7f97b0109520 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a850a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a850a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43926/0 (socket says 172.21.15.18:43926) 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.262+0000 7f97afe00640 1 -- 172.21.15.18:0/3556185523 learned_addr learned my addr 172.21.15.18:0/3556185523 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.263+0000 7f97af400640 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8512d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.263+0000 7f97afe00640 1 -- 172.21.15.18:0/3556185523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 msgr2=0x5629a8515640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.263+0000 7f97afe00640 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 0x5629a8515640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.263+0000 7f97afe00640 1 -- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 msgr2=0x5629a8512d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.263+0000 7f97afe00640 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8512d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.263+0000 7f97afe00640 1 -- 172.21.15.18:0/3556185523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a841d040 con 0x5629a8509d50 2024-04-16T17:13:20.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.263+0000 7f97afe00640 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a850a130 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97a802c700 tx=0x7f97a8032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8967d3bf59604ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.264+0000 7f97a4c00640 1 -- 172.21.15.18:0/3556185523 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8043020 con 0x5629a8509d50 2024-04-16T17:13:20.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.264+0000 7f97a4c00640 1 -- 172.21.15.18:0/3556185523 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a802cca0 con 0x5629a8509d50 2024-04-16T17:13:20.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.264+0000 7f97a4c00640 1 -- 172.21.15.18:0/3556185523 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8039040 con 0x5629a8509d50 2024-04-16T17:13:20.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.264+0000 7f97a7e00640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f97940a3da0 0x7f97940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.264+0000 7f97a7e00640 1 -- 172.21.15.18:0/2570311547 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f979c0d07a0 con 0x7f97b0109520 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.264+0000 7f97af400640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f97940a3da0 0x7f97940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 -- 172.21.15.18:0/3556185523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 msgr2=0x5629a850a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a850a130 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97a802c700 tx=0x7f97a8032340 comp rx=0 tx=0).stop 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 -- 172.21.15.18:0/3556185523 shutdown_connections 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 0x5629a8515640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8512d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a850a130 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 -- 172.21.15.18:0/3556185523 >> 172.21.15.18:0/3556185523 conn(0x5629a8504480 msgr2=0x5629a8504860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 -- 172.21.15.18:0/3556185523 shutdown_connections 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97b95dd740 1 -- 172.21.15.18:0/3556185523 wait complete. 2024-04-16T17:13:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.265+0000 7f97af400640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f97940a3da0 0x7f97940a6260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5629a8509820 tx=0x7f97a0005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 1 -- start start 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 1 Processor -- start 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a8509d50 0x5629a85102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a850a670 0x5629a8510830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 0x5629a8510d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5629a8515fe0 con 0x5629a8509d50 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629a8428830 con 0x5629a850a670 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629a8516160 con 0x5629a8513250 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.266+0000 7f97b95dd740 10 client.0 _ll_register_callbacks cb 0x5629a8430150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97af400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a8509d50 0x5629a85102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97af400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a8509d50 0x5629a85102f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42150/0 (socket says 172.21.15.18:42150) 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97af400640 1 -- 172.21.15.18:0/3441322131 learned_addr learned my addr 172.21.15.18:0/3441322131 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97aea00640 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a850a670 0x5629a8510830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97afe00640 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 0x5629a8510d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97aea00640 1 -- 172.21.15.18:0/3441322131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 msgr2=0x5629a8510d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97aea00640 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 0x5629a8510d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97aea00640 1 -- 172.21.15.18:0/3441322131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a8509d50 msgr2=0x5629a85102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97aea00640 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a8509d50 0x5629a85102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97aea00640 1 -- 172.21.15.18:0/3441322131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a842c920 con 0x5629a850a670 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f97aea00640 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a850a670 0x5629a8510830 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f979c0cdae0 tx=0x7f979c0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ed2494ae48517c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.267+0000 7f9799600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f9799600640 1 -- 172.21.15.18:0/3441322131 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c09c330 con 0x5629a850a670 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f9799600640 1 -- 172.21.15.18:0/3441322131 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f979c0b56c0 con 0x5629a850a670 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 1 -- 172.21.15.18:0/3441322131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a802cca0 con 0x5629a850a670 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f9799600640 1 -- 172.21.15.18:0/3441322131 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f979c0ba050 con 0x5629a850a670 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 10 client.27929 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:20.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 1 -- 172.21.15.18:0/3441322131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629a83fa7f0 con 0x5629a850a670 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f9799600640 1 -- 172.21.15.18:0/3441322131 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a802cca0 con 0x5629a850a670 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f9799600640 1 -- 172.21.15.18:0/3441322131 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f979c0b5250 con 0x5629a850a670 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 10 client.27929 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 10 client.27929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 20 client.27929 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629a8523550 0x5629a8525940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 4 client.27929 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.268+0000 7f97b95dd740 1 -- 172.21.15.18:0/3441322131 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629a847ca40 con 0x5629a8523550 2024-04-16T17:13:20.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.269+0000 7f97af400640 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629a8523550 0x5629a8525940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.269+0000 7f9799600640 10 client.27929 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.269+0000 7f97af400640 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629a8523550 0x5629a8525940 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e27f74b2b4286b server_cookie=434df56e5921876e in_seq=0 out_seq=0 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f9799600640 1 -- 172.21.15.18:0/3441322131 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629a847ca40 con 0x5629a8523550 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f9799600640 10 client.27929 handle_command_reply: tid=0 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 client.27929 shutdown 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 -- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a850a670 msgr2=0x5629a8510830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a850a670 0x5629a8510830 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f979c0cdae0 tx=0x7f979c0056d0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 -- 172.21.15.18:0/3441322131 shutdown_connections 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629a8523550 0x5629a8525940 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 0x5629a8510d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a850a670 0x5629a8510830 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f9799600640 0 client.27929 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:20.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3441322131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a8509d50 0x5629a85102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.270+0000 7f97b95dd740 1 -- 172.21.15.18:0/3441322131 >> 172.21.15.18:0/3441322131 conn(0x5629a8504480 msgr2=0x5629a85061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.271+0000 7f97b95dd740 1 -- 172.21.15.18:0/3441322131 shutdown_connections 2024-04-16T17:13:20.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.271+0000 7f97b95dd740 1 -- 172.21.15.18:0/3441322131 wait complete. 2024-04-16T17:13:20.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.271+0000 7f97b95dd740 20 client.27929 trim_cache size 0 max 16384 2024-04-16T17:13:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.285+0000 7f97b95dd740 1 Processor -- start 2024-04-16T17:13:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.285+0000 7f97b95dd740 1 -- start start 2024-04-16T17:13:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.285+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8509d50 0x5629a851eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8510b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 0x5629a852b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97b95dd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01b4170 con 0x5629a850a670 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97b95dd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01b1a40 con 0x5629a8509d50 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97b95dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01af8b0 con 0x5629a8513250 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97aea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8510b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97aea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8510b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42158/0 (socket says 172.21.15.18:42158) 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97aea00640 1 -- 172.21.15.18:0/32427264 learned_addr learned my addr 172.21.15.18:0/32427264 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97af400640 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8509d50 0x5629a851eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97afe00640 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 0x5629a852b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97aea00640 1 -- 172.21.15.18:0/32427264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 msgr2=0x5629a852b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97aea00640 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 0x5629a852b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97aea00640 1 -- 172.21.15.18:0/32427264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8509d50 msgr2=0x5629a851eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97aea00640 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8509d50 0x5629a851eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97aea00640 1 -- 172.21.15.18:0/32427264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a81813f0 con 0x5629a850a670 2024-04-16T17:13:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.286+0000 7f97af400640 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8509d50 0x5629a851eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.287+0000 7f97aea00640 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8510b90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f979c09c2c0 tx=0x7f979c003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10c15891c86bb85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.287+0000 7f979be00640 1 -- 172.21.15.18:0/32427264 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c080030 con 0x5629a850a670 2024-04-16T17:13:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.287+0000 7f979be00640 1 -- 172.21.15.18:0/32427264 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979c0a4fa0 con 0x5629a850a670 2024-04-16T17:13:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.287+0000 7f979be00640 1 -- 172.21.15.18:0/32427264 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0989a0 con 0x5629a850a670 2024-04-16T17:13:20.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.287+0000 7f97b95dd740 1 -- 172.21.15.18:0/32427264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 msgr2=0x5629a8510b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8510b90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f979c09c2c0 tx=0x7f979c003880 comp rx=0 tx=0).stop 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 -- 172.21.15.18:0/32427264 shutdown_connections 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8513250 0x5629a852b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a8510b90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 --2- 172.21.15.18:0/32427264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8509d50 0x5629a851eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 -- 172.21.15.18:0/32427264 >> 172.21.15.18:0/32427264 conn(0x5629a8504480 msgr2=0x5629a8506160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 -- 172.21.15.18:0/32427264 shutdown_connections 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 -- 172.21.15.18:0/32427264 wait complete. 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 -- start start 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.288+0000 7f97b95dd740 1 Processor -- start 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a851b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a850ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97b95dd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 0x5629a850d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97b95dd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01aff00 con 0x5629a850a670 2024-04-16T17:13:20.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97b95dd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01b1370 con 0x5629a8513250 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97b95dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01b05d0 con 0x5629a8509d50 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97b95dd740 10 client.0 _ll_register_callbacks cb 0x7f979c00df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97aea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a850ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97aea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a850ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42174/0 (socket says 172.21.15.18:42174) 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97aea00640 1 -- 172.21.15.18:0/3236296688 learned_addr learned my addr 172.21.15.18:0/3236296688 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97af400640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a851b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.289+0000 7f97afe00640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 0x5629a850d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97aea00640 1 -- 172.21.15.18:0/3236296688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 msgr2=0x5629a851b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97aea00640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a851b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97aea00640 1 -- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 msgr2=0x5629a850d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97aea00640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 0x5629a850d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97aea00640 1 -- 172.21.15.18:0/3236296688 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a824b7d0 con 0x5629a850a670 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97af400640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a851b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97afe00640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 0x5629a850d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97aea00640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a850ccf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f979c098010 tx=0x7f979c098bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e59984f3eecb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f9799600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f9799600640 1 -- 172.21.15.18:0/3236296688 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c080030 con 0x5629a850a670 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f9799600640 1 -- 172.21.15.18:0/3236296688 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f979c0d0380 con 0x5629a850a670 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97b95dd740 1 -- 172.21.15.18:0/3236296688 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f979c0a4fa0 con 0x5629a850a670 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f9799600640 1 -- 172.21.15.18:0/3236296688 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f979c0d0600 con 0x5629a850a670 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97b95dd740 10 client.17658 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.290+0000 7f97b95dd740 1 -- 172.21.15.18:0/3236296688 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629a85898d0 con 0x5629a850a670 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f9799600640 1 -- 172.21.15.18:0/3236296688 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979c0a4fa0 con 0x5629a850a670 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f9799600640 1 -- 172.21.15.18:0/3236296688 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f979c09b780 con 0x5629a850a670 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f97b95dd740 10 client.17658 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f97b95dd740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f97b95dd740 20 client.17658 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629a8589c60 0x5629a858c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f97b95dd740 4 client.17658 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f97b95dd740 1 -- 172.21.15.18:0/3236296688 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629a84768b0 con 0x5629a8589c60 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.291+0000 7f97af400640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629a8589c60 0x5629a858c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97af400640 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629a8589c60 0x5629a858c080 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c07b9728e219100 server_cookie=3645ca27669f60e3 in_seq=0 out_seq=0 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f9799600640 10 client.17658 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f9799600640 1 -- 172.21.15.18:0/3236296688 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629a84768b0 con 0x5629a8589c60 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f9799600640 10 client.17658 handle_command_reply: tid=0 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 client.17658 shutdown 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 -- 172.21.15.18:0/3236296688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 msgr2=0x5629a850ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a850ccf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f979c098010 tx=0x7f979c098bd0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 -- 172.21.15.18:0/3236296688 shutdown_connections 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629a8589c60 0x5629a858c080 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8513250 0x5629a850d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629a850a670 0x5629a850ccf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 --2- 172.21.15.18:0/3236296688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629a8509d50 0x5629a851b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f9799600640 0 client.17658 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 -- 172.21.15.18:0/3236296688 >> 172.21.15.18:0/3236296688 conn(0x5629a8504480 msgr2=0x5629a8502c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.292+0000 7f97b95dd740 1 -- 172.21.15.18:0/3236296688 shutdown_connections 2024-04-16T17:13:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b95dd740 1 -- 172.21.15.18:0/3236296688 wait complete. 2024-04-16T17:13:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b95dd740 20 client.17658 trim_cache size 0 max 16384 2024-04-16T17:13:20.298 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:13:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 -- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f97940a3da0 msgr2=0x7f97940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f97940a3da0 0x7f97940a6260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5629a8509820 tx=0x7f97a0005f90 comp rx=0 tx=0).stop 2024-04-16T17:13:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 -- 172.21.15.18:0/2570311547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0109520 msgr2=0x7f97b01a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0109520 0x7f97b01a0c30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f979c005fd0 tx=0x7f979c00dde0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 -- 172.21.15.18:0/2570311547 shutdown_connections 2024-04-16T17:13:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f97940a3da0 0x7f97940a6260 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019a8f0 0x7f97b019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0109520 0x7f97b01a0c30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 --2- 172.21.15.18:0/2570311547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97b0104450 0x7f97b01a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.293+0000 7f97b6800640 1 -- 172.21.15.18:0/2570311547 >> 172.21.15.18:0/2570311547 conn(0x7f97b00ffc90 msgr2=0x7f97b0101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.294+0000 7f97b6800640 1 -- 172.21.15.18:0/2570311547 shutdown_connections 2024-04-16T17:13:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:20.294+0000 7f97b6800640 1 -- 172.21.15.18:0/2570311547 wait complete. 2024-04-16T17:13:20.616 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324571 3.75 MB/sec execute 333 sec latency 268.870 ms 2024-04-16T17:13:21.311 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:13:21.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:13:21.311 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:13:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.503+0000 7f991ae00640 1 -- 172.21.15.18:0/2088448490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9914074cf0 msgr2=0x7f991410e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.503+0000 7f991ae00640 1 --2- 172.21.15.18:0/2088448490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9914074cf0 0x7f991410e160 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9900009900 tx=0x7f990002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.503+0000 7f991ae00640 1 -- 172.21.15.18:0/2088448490 shutdown_connections 2024-04-16T17:13:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.503+0000 7f991ae00640 1 --2- 172.21.15.18:0/2088448490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f991410e6a0 0x7f9914110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.503+0000 7f991ae00640 1 --2- 172.21.15.18:0/2088448490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9914074cf0 0x7f991410e160 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.503+0000 7f991ae00640 1 --2- 172.21.15.18:0/2088448490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99140743d0 0x7f99140747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.503+0000 7f991ae00640 1 -- 172.21.15.18:0/2088448490 >> 172.21.15.18:0/2088448490 conn(0x7f99140fd950 msgr2=0x7f99140ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.504+0000 7f991ae00640 1 -- 172.21.15.18:0/2088448490 shutdown_connections 2024-04-16T17:13:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.504+0000 7f991ae00640 1 -- 172.21.15.18:0/2088448490 wait complete. 2024-04-16T17:13:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f991ae00640 1 Processor -- start 2024-04-16T17:13:21.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f991ae00640 1 -- start start 2024-04-16T17:13:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f991ae00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99140743d0 0x7f99141a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f991ae00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9914074cf0 0x7f99141a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f991ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f991410e6a0 0x7f991419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f991ae00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9914114420 con 0x7f99140743d0 2024-04-16T17:13:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f991ae00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99141142a0 con 0x7f9914074cf0 2024-04-16T17:13:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f991ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99141145a0 con 0x7f991410e6a0 2024-04-16T17:13:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f9919a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99140743d0 0x7f99141a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f9919a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99140743d0 0x7f99141a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42184/0 (socket says 172.21.15.18:42184) 2024-04-16T17:13:21.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.505+0000 7f9919a00640 1 -- 172.21.15.18:0/1279269157 learned_addr learned my addr 172.21.15.18:0/1279269157 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f991a400640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f991410e6a0 0x7f991419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9919a00640 1 -- 172.21.15.18:0/1279269157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f991410e6a0 msgr2=0x7f991419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9919a00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f991410e6a0 0x7f991419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9919a00640 1 -- 172.21.15.18:0/1279269157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9914074cf0 msgr2=0x7f99141a0b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:13:21.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9919a00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9914074cf0 0x7f99141a0b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9919a00640 1 -- 172.21.15.18:0/1279269157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991419af70 con 0x7f99140743d0 2024-04-16T17:13:21.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9919a00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99140743d0 0x7f99141a0630 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f990800ea60 tx=0x7f990800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9912000640 1 -- 172.21.15.18:0/1279269157 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800cde0 con 0x7f99140743d0 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9912000640 1 -- 172.21.15.18:0/1279269157 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99080035f0 con 0x7f99140743d0 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.506+0000 7f9912000640 1 -- 172.21.15.18:0/1279269157 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908010670 con 0x7f99140743d0 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.507+0000 7f991ae00640 1 -- 172.21.15.18:0/1279269157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991419b260 con 0x7f99140743d0 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.507+0000 7f991ae00640 1 -- 172.21.15.18:0/1279269157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99141a7480 con 0x7f99140743d0 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.510+0000 7f9912000640 1 -- 172.21.15.18:0/1279269157 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99080037a0 con 0x7f99140743d0 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.510+0000 7f991f1b8740 1 Processor -- start 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f991f1b8740 1 -- start start 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b5c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b64e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b67780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f991f1b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560bb7ac0cb0 con 0x560bb7b5be90 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f991f1b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560bb7a67370 con 0x560bb7b5c7b0 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f991f1b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560bb7abcc70 con 0x560bb7b65390 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f9919a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b64e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f991a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b5c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f9919a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b64e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60080/0 (socket says 172.21.15.18:60080) 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.511+0000 7f9919a00640 1 -- 172.21.15.18:0/382467565 learned_addr learned my addr 172.21.15.18:0/382467565 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f9919000640 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b67780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f991a400640 1 -- 172.21.15.18:0/382467565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 msgr2=0x560bb7b67780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f991a400640 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b67780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f991a400640 1 -- 172.21.15.18:0/382467565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 msgr2=0x560bb7b64e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f991a400640 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b64e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f991a400640 1 -- 172.21.15.18:0/382467565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bb7a86910 con 0x560bb7b5be90 2024-04-16T17:13:21.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f9919a00640 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b64e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f9912000640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f98fc0a3da0 0x7f98fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.512+0000 7f991a400640 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b5c270 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9914075320 tx=0x7f990c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb372e83fac45f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.513+0000 7f9912000640 1 -- 172.21.15.18:0/1279269157 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f99080d0fe0 con 0x7f99140743d0 2024-04-16T17:13:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.513+0000 7f9906a00640 1 -- 172.21.15.18:0/382467565 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c016070 con 0x560bb7b5be90 2024-04-16T17:13:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.513+0000 7f9906a00640 1 -- 172.21.15.18:0/382467565 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990c005020 con 0x560bb7b5be90 2024-04-16T17:13:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.513+0000 7f9906a00640 1 -- 172.21.15.18:0/382467565 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c0126b0 con 0x560bb7b5be90 2024-04-16T17:13:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.513+0000 7f9919a00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f98fc0a3da0 0x7f98fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 -- 172.21.15.18:0/382467565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 msgr2=0x560bb7b5c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f9919a00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f98fc0a3da0 0x7f98fc0a6260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99080ce380 tx=0x7f99080d4b30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b5c270 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9914075320 tx=0x7f990c010740 comp rx=0 tx=0).stop 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 -- 172.21.15.18:0/382467565 shutdown_connections 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b67780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b64e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 --2- 172.21.15.18:0/382467565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b5c270 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 -- 172.21.15.18:0/382467565 >> 172.21.15.18:0/382467565 conn(0x560bb7b565c0 msgr2=0x560bb7b569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 -- 172.21.15.18:0/382467565 shutdown_connections 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.514+0000 7f991f1b8740 1 -- 172.21.15.18:0/382467565 wait complete. 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.515+0000 7f991f1b8740 1 -- start start 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.515+0000 7f991f1b8740 1 Processor -- start 2024-04-16T17:13:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.515+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b62430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.515+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b62970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.515+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b62eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.515+0000 7f991f1b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560bb7b68180 con 0x560bb7b5be90 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.515+0000 7f991f1b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560bb7a7a730 con 0x560bb7b5c7b0 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.515+0000 7f991f1b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560bb7b68300 con 0x560bb7b65390 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991f1b8740 10 client.0 _ll_register_callbacks cb 0x560bb7a491e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b62430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b62430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42212/0 (socket says 172.21.15.18:42212) 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 -- 172.21.15.18:0/1842767803 learned_addr learned my addr 172.21.15.18:0/1842767803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b62eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f9919000640 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b62970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 -- 172.21.15.18:0/1842767803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 msgr2=0x560bb7b62eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b62eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 -- 172.21.15.18:0/1842767803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 msgr2=0x560bb7b62970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b62970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.516+0000 7f991a400640 1 -- 172.21.15.18:0/1842767803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bb789b7e0 con 0x560bb7b5be90 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.517+0000 7f991a400640 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b62430 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f990c004e90 tx=0x7f990c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8071f91ff36f70e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.517+0000 7f98fb400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.517+0000 7f98fb400640 1 -- 172.21.15.18:0/1842767803 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c022040 con 0x560bb7b5be90 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.517+0000 7f98fb400640 1 -- 172.21.15.18:0/1842767803 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f990c0055e0 con 0x560bb7b5be90 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.517+0000 7f991f1b8740 1 -- 172.21.15.18:0/1842767803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f990c005020 con 0x560bb7b5be90 2024-04-16T17:13:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f98fb400640 1 -- 172.21.15.18:0/1842767803 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f990c016040 con 0x560bb7b5be90 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f991f1b8740 10 client.17670 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f991f1b8740 1 -- 172.21.15.18:0/1842767803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bb78fbf70 con 0x560bb7b5be90 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f98fb400640 1 -- 172.21.15.18:0/1842767803 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990c005020 con 0x560bb7b5be90 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f98fb400640 1 -- 172.21.15.18:0/1842767803 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f990c00f040 con 0x560bb7b5be90 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f991f1b8740 10 client.17670 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f991f1b8740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f991f1b8740 20 client.17670 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f991f1b8740 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bb7b75660 0x560bb7b77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f991f1b8740 4 client.17670 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.518+0000 7f991f1b8740 1 -- 172.21.15.18:0/1842767803 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bb7ace6d0 con 0x560bb7b75660 2024-04-16T17:13:21.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.519+0000 7f9919000640 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bb7b75660 0x560bb7b77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.519+0000 7f9919000640 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bb7b75660 0x560bb7b77a50 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a445a9f52228fac server_cookie=839e43f8888a21e2 in_seq=0 out_seq=0 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.519+0000 7f98fb400640 10 client.17670 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f98fb400640 1 -- 172.21.15.18:0/1842767803 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bb7ace6d0 con 0x560bb7b75660 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f98fb400640 10 client.17670 handle_command_reply: tid=0 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 client.17670 shutdown 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 -- 172.21.15.18:0/1842767803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 msgr2=0x560bb7b62430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b62430 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f990c004e90 tx=0x7f990c0099e0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 -- 172.21.15.18:0/1842767803 shutdown_connections 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bb7b75660 0x560bb7b77a50 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b62eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b62970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 --2- 172.21.15.18:0/1842767803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b62430 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f991f1b8740 1 -- 172.21.15.18:0/1842767803 >> 172.21.15.18:0/1842767803 conn(0x560bb7b565c0 msgr2=0x560bb7b58310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.520+0000 7f98fb400640 0 client.17670 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.521+0000 7f991f1b8740 1 -- 172.21.15.18:0/1842767803 shutdown_connections 2024-04-16T17:13:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.521+0000 7f991f1b8740 1 -- 172.21.15.18:0/1842767803 wait complete. 2024-04-16T17:13:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.521+0000 7f991f1b8740 20 client.17670 trim_cache size 0 max 16384 2024-04-16T17:13:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991f1b8740 1 Processor -- start 2024-04-16T17:13:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991f1b8740 1 -- start start 2024-04-16T17:13:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b64b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b71040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b7d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991f1b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99141b40d0 con 0x560bb7b5be90 2024-04-16T17:13:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991f1b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99141b19a0 con 0x560bb7b5c7b0 2024-04-16T17:13:21.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991f1b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99141af810 con 0x560bb7b65390 2024-04-16T17:13:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b64b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.556+0000 7f991a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b64b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42214/0 (socket says 172.21.15.18:42214) 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.557+0000 7f991a400640 1 -- 172.21.15.18:0/4258066193 learned_addr learned my addr 172.21.15.18:0/4258066193 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.557+0000 7f991a400640 1 --2- 172.21.15.18:0/4258066193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b7d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.557+0000 7f991a400640 1 -- 172.21.15.18:0/4258066193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 msgr2=0x560bb7b7d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.557+0000 7f991a400640 1 --2- 172.21.15.18:0/4258066193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b7d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.557+0000 7f991a400640 1 -- 172.21.15.18:0/4258066193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 msgr2=0x560bb7b71040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.557+0000 7f991a400640 1 --2- 172.21.15.18:0/4258066193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b71040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.557+0000 7f991a400640 1 -- 172.21.15.18:0/4258066193 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bb7793170 con 0x560bb7b5be90 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.557+0000 7f991a400640 1 --2- 172.21.15.18:0/4258066193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b64b60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f990c009720 tx=0x7f990c004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2440b2e39a1327 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.558+0000 7f9906000640 1 -- 172.21.15.18:0/4258066193 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c004ec0 con 0x560bb7b5be90 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.558+0000 7f9906000640 1 -- 172.21.15.18:0/4258066193 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990c01a6a0 con 0x560bb7b5be90 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.558+0000 7f9906000640 1 -- 172.21.15.18:0/4258066193 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c0056b0 con 0x560bb7b5be90 2024-04-16T17:13:21.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 -- 172.21.15.18:0/4258066193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 msgr2=0x560bb7b64b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 --2- 172.21.15.18:0/4258066193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b64b60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f990c009720 tx=0x7f990c004b80 comp rx=0 tx=0).stop 2024-04-16T17:13:21.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 -- 172.21.15.18:0/4258066193 shutdown_connections 2024-04-16T17:13:21.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 --2- 172.21.15.18:0/4258066193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b65390 0x560bb7b7d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 --2- 172.21.15.18:0/4258066193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b71040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 --2- 172.21.15.18:0/4258066193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b5be90 0x560bb7b64b60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 -- 172.21.15.18:0/4258066193 >> 172.21.15.18:0/4258066193 conn(0x560bb7b565c0 msgr2=0x560bb7b62c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 -- 172.21.15.18:0/4258066193 shutdown_connections 2024-04-16T17:13:21.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 -- 172.21.15.18:0/4258066193 wait complete. 2024-04-16T17:13:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.559+0000 7f991f1b8740 1 -- start start 2024-04-16T17:13:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.560+0000 7f991f1b8740 1 Processor -- start 2024-04-16T17:13:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.560+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b5be90 0x560bb7b6d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.560+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b6a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.560+0000 7f991f1b8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b65390 0x560bb7b6aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.560+0000 7f991f1b8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99141afe60 con 0x560bb7b65390 2024-04-16T17:13:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.560+0000 7f991f1b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99141b12d0 con 0x560bb7b5c7b0 2024-04-16T17:13:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.560+0000 7f991f1b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99141b0530 con 0x560bb7b5be90 2024-04-16T17:13:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.560+0000 7f991f1b8740 10 client.0 _ll_register_callbacks cb 0x7f990c009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:21.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b65390 0x560bb7b6aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b65390 0x560bb7b6aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42216/0 (socket says 172.21.15.18:42216) 2024-04-16T17:13:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 -- 172.21.15.18:0/907800560 learned_addr learned my addr 172.21.15.18:0/907800560 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f9919000640 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b6a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 -- 172.21.15.18:0/907800560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b5be90 msgr2=0x560bb7b6d740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:13:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b5be90 0x560bb7b6d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 -- 172.21.15.18:0/907800560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 msgr2=0x560bb7b6a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b6a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 -- 172.21.15.18:0/907800560 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bb7b6afd0 con 0x560bb7b65390 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f991a400640 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b65390 0x560bb7b6aa60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f990c01c040 tx=0x7f990c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff3ae784ebb7e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.561+0000 7f98fb400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.562+0000 7f98fb400640 1 -- 172.21.15.18:0/907800560 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c0097d0 con 0x560bb7b65390 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.562+0000 7f98fb400640 1 -- 172.21.15.18:0/907800560 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f990c01ab20 con 0x560bb7b65390 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.562+0000 7f991f1b8740 1 -- 172.21.15.18:0/907800560 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f990c01a6a0 con 0x560bb7b65390 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.562+0000 7f98fb400640 1 -- 172.21.15.18:0/907800560 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f990c047050 con 0x560bb7b65390 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.562+0000 7f991f1b8740 10 client.17676 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.562+0000 7f991f1b8740 1 -- 172.21.15.18:0/907800560 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bb7bdc660 con 0x560bb7b65390 2024-04-16T17:13:21.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.562+0000 7f98fb400640 1 -- 172.21.15.18:0/907800560 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990c01a6a0 con 0x560bb7b65390 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.563+0000 7f98fb400640 1 -- 172.21.15.18:0/907800560 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f990c035070 con 0x560bb7b65390 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.563+0000 7f991f1b8740 10 client.17676 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.563+0000 7f991f1b8740 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.563+0000 7f991f1b8740 20 client.17676 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.563+0000 7f991f1b8740 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bb7bdcc70 0x560bb7bdf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.563+0000 7f991f1b8740 4 client.17676 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.563+0000 7f991f1b8740 1 -- 172.21.15.18:0/907800560 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560bb7ac8540 con 0x560bb7bdcc70 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.563+0000 7f9919000640 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bb7bdcc70 0x560bb7bdf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.564+0000 7f98fb400640 10 client.17676 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.564+0000 7f9919000640 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bb7bdcc70 0x560bb7bdf060 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75188cff694140b6 server_cookie=e36c8193c3afaa8d in_seq=0 out_seq=0 2024-04-16T17:13:21.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f98fb400640 1 -- 172.21.15.18:0/907800560 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560bb7ac8540 con 0x560bb7bdcc70 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f98fb400640 10 client.17676 handle_command_reply: tid=0 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 client.17676 shutdown 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 -- 172.21.15.18:0/907800560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b65390 msgr2=0x560bb7b6aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b65390 0x560bb7b6aa60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f990c01c040 tx=0x7f990c005cf0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 -- 172.21.15.18:0/907800560 shutdown_connections 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bb7bdcc70 0x560bb7bdf060 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bb7b65390 0x560bb7b6aa60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bb7b5c7b0 0x560bb7b6a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 --2- 172.21.15.18:0/907800560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bb7b5be90 0x560bb7b6d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.565+0000 7f991f1b8740 1 -- 172.21.15.18:0/907800560 >> 172.21.15.18:0/907800560 conn(0x560bb7b565c0 msgr2=0x560bb7b578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.566+0000 7f98fb400640 0 client.17676 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:21.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.566+0000 7f991f1b8740 1 -- 172.21.15.18:0/907800560 shutdown_connections 2024-04-16T17:13:21.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.566+0000 7f991f1b8740 1 -- 172.21.15.18:0/907800560 wait complete. 2024-04-16T17:13:21.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.566+0000 7f991f1b8740 20 client.17676 trim_cache size 0 max 16384 2024-04-16T17:13:21.574 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:13:21.574 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:13:21.574 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "28d949af-2201-4a8d-a6e8-8785cc1e9468", 2024-04-16T17:13:21.574 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:13:21.575 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:13:21.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 -- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f98fc0a3da0 msgr2=0x7f98fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f98fc0a3da0 0x7f98fc0a6260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99080ce380 tx=0x7f99080d4b30 comp rx=0 tx=0).stop 2024-04-16T17:13:21.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 -- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99140743d0 msgr2=0x7f99141a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99140743d0 0x7f99141a0630 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f990800ea60 tx=0x7f990800ef20 comp rx=0 tx=0).stop 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 -- 172.21.15.18:0/1279269157 shutdown_connections 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f98fc0a3da0 0x7f98fc0a6260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f991410e6a0 0x7f991419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9914074cf0 0x7f99141a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 --2- 172.21.15.18:0/1279269157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99140743d0 0x7f99141a0630 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 -- 172.21.15.18:0/1279269157 >> 172.21.15.18:0/1279269157 conn(0x7f99140fd950 msgr2=0x7f99140fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 -- 172.21.15.18:0/1279269157 shutdown_connections 2024-04-16T17:13:21.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.567+0000 7f991ae00640 1 -- 172.21.15.18:0/1279269157 wait complete. 2024-04-16T17:13:21.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:13:21.586 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:13:21.618 INFO:tasks.workunit.client.0.smithi018.stdout: 1 324879 3.74 MB/sec execute 334 sec latency 324.428 ms 2024-04-16T17:13:21.805 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:19 smithi018 ceph-mon[31776]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-04-16T17:13:21.806 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:19 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1261850927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:21.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 -- 172.21.15.18:0/232522601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40c10e6c0 msgr2=0x7fa40c110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 --2- 172.21.15.18:0/232522601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40c10e6c0 0x7fa40c110ab0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa404009ca0 tx=0x7fa40402c180 comp rx=0 tx=0).stop 2024-04-16T17:13:21.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 -- 172.21.15.18:0/232522601 shutdown_connections 2024-04-16T17:13:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 --2- 172.21.15.18:0/232522601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40c10e6c0 0x7fa40c110ab0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 --2- 172.21.15.18:0/232522601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c074cd0 0x7fa40c10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 --2- 172.21.15.18:0/232522601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa40c0743b0 0x7fa40c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 -- 172.21.15.18:0/232522601 >> 172.21.15.18:0/232522601 conn(0x7fa40c0fd950 msgr2=0x7fa40c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 -- 172.21.15.18:0/232522601 shutdown_connections 2024-04-16T17:13:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.807+0000 7fa412a00640 1 -- 172.21.15.18:0/232522601 wait complete. 2024-04-16T17:13:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.808+0000 7fa412a00640 1 Processor -- start 2024-04-16T17:13:21.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.808+0000 7fa412a00640 1 -- start start 2024-04-16T17:13:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa412a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0743b0 0x7fa40c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa412a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa40c074cd0 0x7fa40c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa412a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40c10e6c0 0x7fa40c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa412a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1143a0 con 0x7fa40c074cd0 2024-04-16T17:13:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa412a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c114220 con 0x7fa40c0743b0 2024-04-16T17:13:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa412a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c114520 con 0x7fa40c10e6c0 2024-04-16T17:13:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa411600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0743b0 0x7fa40c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa410c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa40c074cd0 0x7fa40c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa411600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0743b0 0x7fa40c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60108/0 (socket says 172.21.15.18:60108) 2024-04-16T17:13:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa411600640 1 -- 172.21.15.18:0/951313191 learned_addr learned my addr 172.21.15.18:0/951313191 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.809+0000 7fa412000640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40c10e6c0 0x7fa40c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.810+0000 7fa411600640 1 -- 172.21.15.18:0/951313191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40c10e6c0 msgr2=0x7fa40c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.810+0000 7fa411600640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40c10e6c0 0x7fa40c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.810+0000 7fa411600640 1 -- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa40c074cd0 msgr2=0x7fa40c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.810+0000 7fa411600640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa40c074cd0 0x7fa40c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.810+0000 7fa411600640 1 -- 172.21.15.18:0/951313191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c19af80 con 0x7fa40c0743b0 2024-04-16T17:13:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.820+0000 7fa411600640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0743b0 0x7fa40c1a0600 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa40000d950 tx=0x7fa40000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.820+0000 7fa40a000640 1 -- 172.21.15.18:0/951313191 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400017070 con 0x7fa40c0743b0 2024-04-16T17:13:21.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.820+0000 7fa412a00640 1 -- 172.21.15.18:0/951313191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c19b270 con 0x7fa40c0743b0 2024-04-16T17:13:21.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.821+0000 7fa40a000640 1 -- 172.21.15.18:0/951313191 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa400009c30 con 0x7fa40c0743b0 2024-04-16T17:13:21.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.821+0000 7fa412a00640 1 -- 172.21.15.18:0/951313191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1a7580 con 0x7fa40c0743b0 2024-04-16T17:13:21.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.821+0000 7fa40a000640 1 -- 172.21.15.18:0/951313191 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400020060 con 0x7fa40c0743b0 2024-04-16T17:13:21.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.824+0000 7fa416d8f740 1 Processor -- start 2024-04-16T17:13:21.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.824+0000 7fa416d8f740 1 -- start start 2024-04-16T17:13:21.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa40a000640 1 -- 172.21.15.18:0/951313191 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4000109d0 con 0x7fa40c0743b0 2024-04-16T17:13:21.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebd4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebdd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebdfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa416d8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556dbeb3a120 con 0x556dbebdd890 2024-04-16T17:13:21.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa416d8f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556dbeae0340 con 0x556dbebd4390 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa416d8f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556dbeb360e0 con 0x556dbebd4cb0 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa412000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebd4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa411600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebdd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa412000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebd4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60110/0 (socket says 172.21.15.18:60110) 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa412000640 1 -- 172.21.15.18:0/3020076691 learned_addr learned my addr 172.21.15.18:0/3020076691 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa410c00640 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebdfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa412000640 1 -- 172.21.15.18:0/3020076691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 msgr2=0x556dbebdd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa412000640 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebdd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa412000640 1 -- 172.21.15.18:0/3020076691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 msgr2=0x556dbebdfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa412000640 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebdfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.825+0000 7fa412000640 1 -- 172.21.15.18:0/3020076691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbe9167d0 con 0x556dbebd4390 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.826+0000 7fa410c00640 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebdfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.826+0000 7fa412000640 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebd4770 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa40402c790 tx=0x7fa4040098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d8cb3db4ab718fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.826+0000 7fa3fe600640 1 -- 172.21.15.18:0/3020076691 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404043070 con 0x556dbebd4390 2024-04-16T17:13:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.826+0000 7fa3fe600640 1 -- 172.21.15.18:0/3020076691 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa404003e50 con 0x556dbebd4390 2024-04-16T17:13:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.826+0000 7fa3fe600640 1 -- 172.21.15.18:0/3020076691 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404039040 con 0x556dbebd4390 2024-04-16T17:13:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa40a000640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3f40a3da0 0x7fa3f40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa40a000640 1 -- 172.21.15.18:0/951313191 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fa4000d0390 con 0x7fa40c0743b0 2024-04-16T17:13:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa416d8f740 1 -- 172.21.15.18:0/3020076691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 msgr2=0x556dbebd4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebd4770 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa40402c790 tx=0x7fa4040098c0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa416d8f740 1 -- 172.21.15.18:0/3020076691 shutdown_connections 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebdfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebdd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3020076691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebd4770 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.827+0000 7fa416d8f740 1 -- 172.21.15.18:0/3020076691 >> 172.21.15.18:0/3020076691 conn(0x556dbebceac0 msgr2=0x556dbebceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.828+0000 7fa416d8f740 1 -- 172.21.15.18:0/3020076691 shutdown_connections 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.828+0000 7fa411600640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3f40a3da0 0x7fa3f40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.828+0000 7fa416d8f740 1 -- 172.21.15.18:0/3020076691 wait complete. 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.828+0000 7fa416d8f740 1 -- start start 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.829+0000 7fa411600640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3f40a3da0 0x7fa3f40a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa4000cd780 tx=0x7fa4000d28a0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.829+0000 7fa416d8f740 1 Processor -- start 2024-04-16T17:13:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.829+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebda8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.829+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4cb0 0x556dbebdae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.829+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebdd890 0x556dbebdb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.829+0000 7fa416d8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556dbebe8e30 con 0x556dbebd4cb0 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.830+0000 7fa416d8f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556dbeaf3710 con 0x556dbebd4390 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.830+0000 7fa416d8f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556dbebe8fb0 con 0x556dbebdd890 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.830+0000 7fa416d8f740 10 client.0 _ll_register_callbacks cb 0x556dbeaeeaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.830+0000 7fa410c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4cb0 0x556dbebdae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.830+0000 7fa412000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebda8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.830+0000 7fa410c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4cb0 0x556dbebdae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42252/0 (socket says 172.21.15.18:42252) 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.830+0000 7fa410c00640 1 -- 172.21.15.18:0/4058634237 learned_addr learned my addr 172.21.15.18:0/4058634237 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.830+0000 7fa412000640 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebdd890 0x556dbebdb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa412000640 1 -- 172.21.15.18:0/4058634237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebdd890 msgr2=0x556dbebdb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa412000640 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebdd890 0x556dbebdb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa412000640 1 -- 172.21.15.18:0/4058634237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4cb0 msgr2=0x556dbebdae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa412000640 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4cb0 0x556dbebdae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa412000640 1 -- 172.21.15.18:0/4058634237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbe84c3f0 con 0x556dbebd4390 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa410c00640 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4cb0 0x556dbebdae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa412000640 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebda8e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa404009ca0 tx=0x7fa404004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b3417b14543f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa3f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa3f2a00640 1 -- 172.21.15.18:0/4058634237 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404053060 con 0x556dbebd4390 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa416d8f740 1 -- 172.21.15.18:0/4058634237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa404003e50 con 0x556dbebd4390 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.831+0000 7fa3f2a00640 1 -- 172.21.15.18:0/4058634237 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fa4040326f0 con 0x556dbebd4390 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa3f2a00640 1 -- 172.21.15.18:0/4058634237 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa404039040 con 0x556dbebd4390 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa416d8f740 10 client.27959 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa416d8f740 1 -- 172.21.15.18:0/4058634237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556dbe80c0f0 con 0x556dbebd4390 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa3f2a00640 1 -- 172.21.15.18:0/4058634237 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa404003e50 con 0x556dbebd4390 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa3f2a00640 1 -- 172.21.15.18:0/4058634237 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa4040489b0 con 0x556dbebd4390 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa416d8f740 10 client.27959 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa416d8f740 10 client.27959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa416d8f740 20 client.27959 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:21.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa416d8f740 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556dbebf61e0 0x556dbebf85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa416d8f740 4 client.27959 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.832+0000 7fa416d8f740 1 -- 172.21.15.18:0/4058634237 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dbeb47b40 con 0x556dbebf61e0 2024-04-16T17:13:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.833+0000 7fa410c00640 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556dbebf61e0 0x556dbebf85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.833+0000 7fa410c00640 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556dbebf61e0 0x556dbebf85d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c5b860f6819c29 server_cookie=de43601ab9ab0d48 in_seq=0 out_seq=0 2024-04-16T17:13:21.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.833+0000 7fa3f2a00640 10 client.27959 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.834+0000 7fa3f2a00640 1 -- 172.21.15.18:0/4058634237 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556dbeb47b40 con 0x556dbebf61e0 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.834+0000 7fa3f2a00640 10 client.27959 handle_command_reply: tid=0 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.834+0000 7fa416d8f740 1 client.27959 shutdown 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 -- 172.21.15.18:0/4058634237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 msgr2=0x556dbebda8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebda8e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa404009ca0 tx=0x7fa404004590 comp rx=0 tx=0).stop 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 -- 172.21.15.18:0/4058634237 shutdown_connections 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556dbebf61e0 0x556dbebf85d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebdd890 0x556dbebdb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4cb0 0x556dbebdae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 --2- 172.21.15.18:0/4058634237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebda8e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 -- 172.21.15.18:0/4058634237 >> 172.21.15.18:0/4058634237 conn(0x556dbebceac0 msgr2=0x556dbebd0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa3f2a00640 0 client.27959 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 -- 172.21.15.18:0/4058634237 shutdown_connections 2024-04-16T17:13:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 1 -- 172.21.15.18:0/4058634237 wait complete. 2024-04-16T17:13:21.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.835+0000 7fa416d8f740 20 client.27959 trim_cache size 0 max 16384 2024-04-16T17:13:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.855+0000 7fa416d8f740 1 Processor -- start 2024-04-16T17:13:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.855+0000 7fa416d8f740 1 -- start start 2024-04-16T17:13:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4390 0x556dbebdd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebd7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebdd890 0x556dbebd80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa416d8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1afb80 con 0x556dbebd4390 2024-04-16T17:13:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa416d8f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1ad450 con 0x556dbebdd890 2024-04-16T17:13:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa416d8f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1ab2c0 con 0x556dbebd4cb0 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa412000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4390 0x556dbebdd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa410c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebd7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa412000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4390 0x556dbebdd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42254/0 (socket says 172.21.15.18:42254) 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa412000640 1 -- 172.21.15.18:0/543128648 learned_addr learned my addr 172.21.15.18:0/543128648 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.856+0000 7fa412000640 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebdd890 0x556dbebd80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa410c00640 1 -- 172.21.15.18:0/543128648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebdd890 msgr2=0x556dbebd80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa410c00640 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebdd890 0x556dbebd80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa410c00640 1 -- 172.21.15.18:0/543128648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4390 msgr2=0x556dbebdd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa410c00640 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4390 0x556dbebdd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa410c00640 1 -- 172.21.15.18:0/543128648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbebd4ae0 con 0x556dbebd4cb0 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa412000640 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebdd890 0x556dbebd80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa410c00640 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebd7b60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x556dbebdbeb0 tx=0x7fa3f8014840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d3e0d96d4e4e45b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa3fdc00640 1 -- 172.21.15.18:0/543128648 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80142d0 con 0x556dbebd4cb0 2024-04-16T17:13:21.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa3fdc00640 1 -- 172.21.15.18:0/543128648 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f80035f0 con 0x556dbebd4cb0 2024-04-16T17:13:21.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.857+0000 7fa3fdc00640 1 -- 172.21.15.18:0/543128648 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f801e2b0 con 0x556dbebd4cb0 2024-04-16T17:13:21.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 -- 172.21.15.18:0/543128648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 msgr2=0x556dbebd7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebd7b60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x556dbebdbeb0 tx=0x7fa3f8014840 comp rx=0 tx=0).stop 2024-04-16T17:13:21.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 -- 172.21.15.18:0/543128648 shutdown_connections 2024-04-16T17:13:21.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebdd890 0x556dbebd80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebd7b60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 --2- 172.21.15.18:0/543128648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebd4390 0x556dbebdd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 -- 172.21.15.18:0/543128648 >> 172.21.15.18:0/543128648 conn(0x556dbebceac0 msgr2=0x556dbebceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 -- 172.21.15.18:0/543128648 shutdown_connections 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.858+0000 7fa416d8f740 1 -- 172.21.15.18:0/543128648 wait complete. 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.859+0000 7fa416d8f740 1 -- start start 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.859+0000 7fa416d8f740 1 Processor -- start 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.859+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebfbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.859+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebf8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.859+0000 7fa416d8f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebf9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.859+0000 7fa416d8f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1ab910 con 0x556dbebdd890 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.859+0000 7fa416d8f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1acd80 con 0x556dbebd4390 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.859+0000 7fa416d8f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1abfe0 con 0x556dbebd4cb0 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa416d8f740 10 client.0 _ll_register_callbacks cb 0x7fa4000d3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa412000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebfbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa410c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebf8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa412000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebf9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa410c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebf8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44068/0 (socket says 172.21.15.18:44068) 2024-04-16T17:13:21.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa410c00640 1 -- 172.21.15.18:0/3241282053 learned_addr learned my addr 172.21.15.18:0/3241282053 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa412000640 1 -- 172.21.15.18:0/3241282053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 msgr2=0x556dbebf8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa412000640 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebf8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa412000640 1 -- 172.21.15.18:0/3241282053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 msgr2=0x556dbebfbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa412000640 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebfbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa412000640 1 -- 172.21.15.18:0/3241282053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbec5b3c0 con 0x556dbebdd890 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.860+0000 7fa410c00640 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebf8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.861+0000 7fa412000640 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebf9680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x556dbebca650 tx=0x7fa404009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f7973d43f94615 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.861+0000 7fa3f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.861+0000 7fa3f2a00640 1 -- 172.21.15.18:0/3241282053 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404053040 con 0x556dbebdd890 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.861+0000 7fa3f2a00640 1 -- 172.21.15.18:0/3241282053 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fa40403fd00 con 0x556dbebdd890 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.861+0000 7fa416d8f740 1 -- 172.21.15.18:0/3241282053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3f80035f0 con 0x556dbebdd890 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.862+0000 7fa3f2a00640 1 -- 172.21.15.18:0/3241282053 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa404043040 con 0x556dbebdd890 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.862+0000 7fa416d8f740 10 client.17691 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.862+0000 7fa416d8f740 1 -- 172.21.15.18:0/3241282053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556dbec5c550 con 0x556dbebdd890 2024-04-16T17:13:21.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa3f2a00640 1 -- 172.21.15.18:0/3241282053 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f80035f0 con 0x556dbebdd890 2024-04-16T17:13:21.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa3f2a00640 1 -- 172.21.15.18:0/3241282053 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa404063060 con 0x556dbebdd890 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa416d8f740 10 client.17691 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa416d8f740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa416d8f740 20 client.17691 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556dbec5cbb0 0x556dbec5efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa416d8f740 4 client.17691 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa416d8f740 1 -- 172.21.15.18:0/3241282053 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556dbeb419b0 con 0x556dbec5cbb0 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.863+0000 7fa410c00640 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556dbec5cbb0 0x556dbec5efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.864+0000 7fa3f2a00640 10 client.17691 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.864+0000 7fa410c00640 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556dbec5cbb0 0x556dbec5efa0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb860def2de517cf server_cookie=f1111a51ef6d5b1d in_seq=0 out_seq=0 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.864+0000 7fa3f2a00640 1 -- 172.21.15.18:0/3241282053 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556dbeb419b0 con 0x556dbec5cbb0 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.864+0000 7fa3f2a00640 10 client.17691 handle_command_reply: tid=0 2024-04-16T17:13:21.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.864+0000 7fa416d8f740 1 client.17691 shutdown 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.864+0000 7fa416d8f740 1 -- 172.21.15.18:0/3241282053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 msgr2=0x556dbebf9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.864+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebf9680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x556dbebca650 tx=0x7fa404009360 comp rx=0 tx=0).stop 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 1 -- 172.21.15.18:0/3241282053 shutdown_connections 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556dbec5cbb0 0x556dbec5efa0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556dbebdd890 0x556dbebf9680 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556dbebd4cb0 0x556dbebf8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 1 --2- 172.21.15.18:0/3241282053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dbebd4390 0x556dbebfbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 1 -- 172.21.15.18:0/3241282053 >> 172.21.15.18:0/3241282053 conn(0x556dbebceac0 msgr2=0x556dbebcd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa3f2a00640 0 client.17691 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 1 -- 172.21.15.18:0/3241282053 shutdown_connections 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 1 -- 172.21.15.18:0/3241282053 wait complete. 2024-04-16T17:13:21.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.865+0000 7fa416d8f740 20 client.17691 trim_cache size 0 max 16384 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout: "28d949af-2201-4a8d-a6e8-8785cc1e9468": { 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "28d949af-2201-4a8d-a6e8-8785cc1e9468", 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:13:21.868 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 -- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3f40a3da0 msgr2=0x7fa3f40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3f40a3da0 0x7fa3f40a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa4000cd780 tx=0x7fa4000d28a0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 -- 172.21.15.18:0/951313191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0743b0 msgr2=0x7fa40c1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0743b0 0x7fa40c1a0600 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa40000d950 tx=0x7fa40000de10 comp rx=0 tx=0).stop 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 -- 172.21.15.18:0/951313191 shutdown_connections 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa3f40a3da0 0x7fa3f40a6260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40c10e6c0 0x7fa40c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa40c074cd0 0x7fa40c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 --2- 172.21.15.18:0/951313191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0743b0 0x7fa40c1a0600 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 -- 172.21.15.18:0/951313191 >> 172.21.15.18:0/951313191 conn(0x7fa40c0fd950 msgr2=0x7fa40c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 -- 172.21.15.18:0/951313191 shutdown_connections 2024-04-16T17:13:21.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:21.866+0000 7fa412a00640 1 -- 172.21.15.18:0/951313191 wait complete. 2024-04-16T17:13:21.887 INFO:tasks.cephfs.filesystem:scrub status for tag:28d949af-2201-4a8d-a6e8-8785cc1e9468 - {'path': '/', 'tag': '28d949af-2201-4a8d-a6e8-8785cc1e9468', 'options': 'recursive,force'} 2024-04-16T17:13:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:21 smithi022 ceph-mon[36023]: pgmap v269: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 721 KiB/s wr, 87 op/s 2024-04-16T17:13:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:21 smithi043 ceph-mon[36901]: pgmap v269: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 721 KiB/s wr, 87 op/s 2024-04-16T17:13:22.618 INFO:tasks.workunit.client.0.smithi018.stdout: 1 325568 3.74 MB/sec execute 335 sec latency 112.413 ms 2024-04-16T17:13:23.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:22 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:13:23.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:22 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:23.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:22 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:13:23.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:22 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:23.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:22 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:13:23.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:22 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:23.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:22 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:13:23.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:22 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:23.618 INFO:tasks.workunit.client.0.smithi018.stdout: 1 326670 3.75 MB/sec execute 336 sec latency 9.288 ms 2024-04-16T17:13:24.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:23 smithi022 ceph-mon[36023]: pgmap v270: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 769 KiB/s wr, 85 op/s 2024-04-16T17:13:24.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:21 smithi018 ceph-mon[31776]: pgmap v269: 129 pgs: 129 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 721 KiB/s wr, 87 op/s 2024-04-16T17:13:24.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:22 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:13:24.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:23 smithi043 ceph-mon[36901]: pgmap v270: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 769 KiB/s wr, 85 op/s 2024-04-16T17:13:24.587 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:22 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:24.588 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:22 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:13:24.588 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:22 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:24.588 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:23 smithi018 ceph-mon[31776]: pgmap v270: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 769 KiB/s wr, 85 op/s 2024-04-16T17:13:24.618 INFO:tasks.workunit.client.0.smithi018.stdout: 1 327316 3.75 MB/sec execute 337 sec latency 52.839 ms 2024-04-16T17:13:25.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:24 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:13:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:24 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:13:25.619 INFO:tasks.workunit.client.0.smithi018.stdout: 1 327955 3.75 MB/sec execute 338 sec latency 63.460 ms 2024-04-16T17:13:26.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:25 smithi022 ceph-mon[36023]: pgmap v271: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 702 KiB/s wr, 77 op/s 2024-04-16T17:13:26.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:25 smithi043 ceph-mon[36901]: pgmap v271: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 702 KiB/s wr, 77 op/s 2024-04-16T17:13:26.618 INFO:tasks.workunit.client.0.smithi018.stdout: 1 328182 3.74 MB/sec execute 339 sec latency 96.933 ms 2024-04-16T17:13:27.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:26 smithi022 ceph-mon[36023]: pgmap v272: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-16T17:13:27.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:26 smithi043 ceph-mon[36901]: pgmap v272: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-16T17:13:27.619 INFO:tasks.workunit.client.0.smithi018.stdout: 1 328755 3.73 MB/sec execute 340 sec latency 52.629 ms 2024-04-16T17:13:28.619 INFO:tasks.workunit.client.0.smithi018.stdout: 1 329977 3.75 MB/sec execute 341 sec latency 12.045 ms 2024-04-16T17:13:29.618 INFO:tasks.workunit.client.0.smithi018.stdout: 1 330505 3.75 MB/sec execute 342 sec latency 21.547 ms 2024-04-16T17:13:29.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:29 smithi043 ceph-mon[36901]: pgmap v273: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 729 KiB/s wr, 64 op/s 2024-04-16T17:13:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:29 smithi022 ceph-mon[36023]: pgmap v273: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 729 KiB/s wr, 64 op/s 2024-04-16T17:13:30.186 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:24 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:13:30.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:25 smithi018 ceph-mon[31776]: pgmap v271: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 702 KiB/s wr, 77 op/s 2024-04-16T17:13:30.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:26 smithi018 ceph-mon[31776]: pgmap v272: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-16T17:13:30.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:29 smithi018 ceph-mon[31776]: pgmap v273: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 729 KiB/s wr, 64 op/s 2024-04-16T17:13:30.620 INFO:tasks.workunit.client.0.smithi018.stdout: 1 331121 3.74 MB/sec execute 343 sec latency 59.780 ms 2024-04-16T17:13:31.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 331447 3.74 MB/sec execute 344 sec latency 75.464 ms 2024-04-16T17:13:31.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:31 smithi043 ceph-mon[36901]: pgmap v274: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-16T17:13:31.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:31 smithi022 ceph-mon[36023]: pgmap v274: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-16T17:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 -- 172.21.15.18:0/4010383911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5480102fe0 msgr2=0x7f54801033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 --2- 172.21.15.18:0/4010383911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5480102fe0 0x7f54801033c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5474009900 tx=0x7f547402bce0 comp rx=0 tx=0).stop 2024-04-16T17:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 -- 172.21.15.18:0/4010383911 shutdown_connections 2024-04-16T17:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 --2- 172.21.15.18:0/4010383911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f548010c400 0x7f548010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 --2- 172.21.15.18:0/4010383911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5480103900 0x7f548010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 --2- 172.21.15.18:0/4010383911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5480102fe0 0x7f54801033c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 -- 172.21.15.18:0/4010383911 >> 172.21.15.18:0/4010383911 conn(0x7f54800fc7e0 msgr2=0x7f54800fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 -- 172.21.15.18:0/4010383911 shutdown_connections 2024-04-16T17:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.994+0000 7f5486600640 1 -- 172.21.15.18:0/4010383911 wait complete. 2024-04-16T17:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.995+0000 7f5486600640 1 Processor -- start 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.995+0000 7f5486600640 1 -- start start 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f5486600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5480102fe0 0x7f54801a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f5486600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5480103900 0x7f54801a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f5486600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548010c400 0x7f548019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f5486600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5480112260 con 0x7f5480103900 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f5486600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54801120e0 con 0x7f5480102fe0 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f5486600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54801123e0 con 0x7f548010c400 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f547fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5480103900 0x7f54801a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f547fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5480103900 0x7f54801a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42334/0 (socket says 172.21.15.18:42334) 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f547fe00640 1 -- 172.21.15.18:0/2474094189 learned_addr learned my addr 172.21.15.18:0/2474094189 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f5485c00640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548010c400 0x7f548019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f547fe00640 1 -- 172.21.15.18:0/2474094189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548010c400 msgr2=0x7f548019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f547fe00640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548010c400 0x7f548019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f547fe00640 1 -- 172.21.15.18:0/2474094189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5480102fe0 msgr2=0x7f54801a0610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.996+0000 7f547fe00640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5480102fe0 0x7f54801a0610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.997+0000 7f547fe00640 1 -- 172.21.15.18:0/2474094189 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548019af50 con 0x7f5480103900 2024-04-16T17:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.997+0000 7f5485c00640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548010c400 0x7f548019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.997+0000 7f547fe00640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5480103900 0x7f54801a0b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f546c00e9f0 tx=0x7f546c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.997+0000 7f547d600640 1 -- 172.21.15.18:0/2474094189 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c00ccc0 con 0x7f5480103900 2024-04-16T17:13:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.997+0000 7f547d600640 1 -- 172.21.15.18:0/2474094189 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f546c010910 con 0x7f5480103900 2024-04-16T17:13:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.997+0000 7f5486600640 1 -- 172.21.15.18:0/2474094189 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548019b240 con 0x7f5480103900 2024-04-16T17:13:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.997+0000 7f5486600640 1 -- 172.21.15.18:0/2474094189 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54801a7460 con 0x7f5480103900 2024-04-16T17:13:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:31.998+0000 7f547d600640 1 -- 172.21.15.18:0/2474094189 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c018920 con 0x7f5480103900 2024-04-16T17:13:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.001+0000 7f548a831740 1 Processor -- start 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.001+0000 7f548a831740 1 -- start start 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.001+0000 7f548a831740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3324540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.001+0000 7f548a831740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc3324a80 0x55dfc3324f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.001+0000 7f548a831740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc332e000 0x55dfc33303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.001+0000 7f548a831740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dfc3289780 con 0x55dfc33257f0 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.001+0000 7f548a831740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dfc322ff40 con 0x55dfc3324a80 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.001+0000 7f548a831740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfc3285740 con 0x55dfc332e000 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3324540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc3324a80 0x55dfc3324f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc3324a80 0x55dfc3324f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57050/0 (socket says 172.21.15.18:57050) 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3324540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42342/0 (socket says 172.21.15.18:42342) 2024-04-16T17:13:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 -- 172.21.15.18:0/476861248 learned_addr learned my addr 172.21.15.18:0/476861248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 -- 172.21.15.18:0/476861248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc332e000 msgr2=0x55dfc33303f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 --2- 172.21.15.18:0/476861248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc332e000 0x55dfc33303f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 -- 172.21.15.18:0/476861248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 msgr2=0x55dfc3324540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 --2- 172.21.15.18:0/476861248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3324540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 -- 172.21.15.18:0/476861248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc2f9c210 con 0x55dfc3324a80 2024-04-16T17:13:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.002+0000 7f5485200640 1 --2- 172.21.15.18:0/476861248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc3324a80 0x55dfc3324f00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5474005940 tx=0x7f5474005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f29158777e4c83fe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.003+0000 7f5472000640 1 -- 172.21.15.18:0/476861248 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474042020 con 0x55dfc3324a80 2024-04-16T17:13:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.003+0000 7f5472000640 1 -- 172.21.15.18:0/476861248 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5474004040 con 0x55dfc3324a80 2024-04-16T17:13:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.003+0000 7f5472000640 1 -- 172.21.15.18:0/476861248 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474003a80 con 0x55dfc3324a80 2024-04-16T17:13:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.003+0000 7f547d600640 1 -- 172.21.15.18:0/2474094189 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f546c010430 con 0x7f5480103900 2024-04-16T17:13:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 -- 172.21.15.18:0/476861248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc3324a80 msgr2=0x55dfc3324f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 --2- 172.21.15.18:0/476861248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc3324a80 0x55dfc3324f00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5474005940 tx=0x7f5474005dd0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 -- 172.21.15.18:0/476861248 shutdown_connections 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 --2- 172.21.15.18:0/476861248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc332e000 0x55dfc33303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 --2- 172.21.15.18:0/476861248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc3324a80 0x55dfc3324f00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 --2- 172.21.15.18:0/476861248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3324540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 -- 172.21.15.18:0/476861248 >> 172.21.15.18:0/476861248 conn(0x55dfc32bf5d0 msgr2=0x55dfc331f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 -- 172.21.15.18:0/476861248 shutdown_connections 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f548a831740 1 -- 172.21.15.18:0/476861248 wait complete. 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.004+0000 7f547d600640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54680a3c90 0x7f54680a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.005+0000 7f548a831740 1 -- start start 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.005+0000 7f547d600640 1 -- 172.21.15.18:0/2474094189 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f546c0d0d60 con 0x7f5480103900 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.005+0000 7f5485c00640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54680a3c90 0x7f54680a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.005+0000 7f548a831740 1 Processor -- start 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f548a831740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc332da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f548a831740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc332ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f548a831740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc332b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f548a831740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dfc3330cd0 con 0x55dfc3324a80 2024-04-16T17:13:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f548a831740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dfc3243390 con 0x55dfc332e000 2024-04-16T17:13:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f548a831740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfc3330e50 con 0x55dfc33257f0 2024-04-16T17:13:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f548a831740 10 client.0 _ll_register_callbacks cb 0x55dfc322cf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f5485200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc332da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f5485200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc332da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42354/0 (socket says 172.21.15.18:42354) 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f5485c00640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc332b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f5485200640 1 -- 172.21.15.18:0/2393362923 learned_addr learned my addr 172.21.15.18:0/2393362923 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.006+0000 7f5485c00640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54680a3c90 0x7f54680a6150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f547800a7f0 tx=0x7f547800d040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f547fe00640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc332ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5485c00640 1 -- 172.21.15.18:0/2393362923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 msgr2=0x55dfc332ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5485c00640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc332ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5485c00640 1 -- 172.21.15.18:0/2393362923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 msgr2=0x55dfc332da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5485c00640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc332da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5485c00640 1 -- 172.21.15.18:0/2393362923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc3068620 con 0x55dfc332e000 2024-04-16T17:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5485200640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc332da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f547fe00640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc332ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5485c00640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc332b520 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5478011580 tx=0x7f5478004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e3f2d5e32b312f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5466a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5466a00640 1 -- 172.21.15.18:0/2393362923 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478026060 con 0x55dfc332e000 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f548a831740 1 -- 172.21.15.18:0/2393362923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5474004040 con 0x55dfc332e000 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.007+0000 7f5466a00640 1 -- 172.21.15.18:0/2393362923 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f54780168f0 con 0x55dfc332e000 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.008+0000 7f5466a00640 1 -- 172.21.15.18:0/2393362923 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f547802d050 con 0x55dfc332e000 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.008+0000 7f548a831740 10 client.27971 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.008+0000 7f548a831740 1 -- 172.21.15.18:0/2393362923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dfc333ce50 con 0x55dfc332e000 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.008+0000 7f5466a00640 1 -- 172.21.15.18:0/2393362923 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5474004040 con 0x55dfc332e000 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.008+0000 7f5466a00640 1 -- 172.21.15.18:0/2393362923 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54780173c0 con 0x55dfc332e000 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.008+0000 7f548a831740 10 client.27971 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.009+0000 7f548a831740 10 client.27971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.009+0000 7f548a831740 20 client.27971 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.009+0000 7f548a831740 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfc333e380 0x55dfc33407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.009+0000 7f548a831740 4 client.27971 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.009+0000 7f548a831740 1 -- 172.21.15.18:0/2393362923 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfc32971a0 con 0x55dfc333e380 2024-04-16T17:13:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.009+0000 7f5485200640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfc333e380 0x55dfc33407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.010+0000 7f5485200640 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfc333e380 0x55dfc33407b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4c13a866105bb1 server_cookie=efe55de2b75bd32e in_seq=0 out_seq=0 2024-04-16T17:13:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.010+0000 7f5466a00640 10 client.27971 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.011+0000 7f5466a00640 1 -- 172.21.15.18:0/2393362923 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfc32971a0 con 0x55dfc333e380 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.011+0000 7f5466a00640 10 client.27971 handle_command_reply: tid=0 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.011+0000 7f548a831740 1 client.27971 shutdown 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 -- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 msgr2=0x55dfc332b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc332b520 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5478011580 tx=0x7f5478004a60 comp rx=0 tx=0).stop 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 -- 172.21.15.18:0/2393362923 shutdown_connections 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfc333e380 0x55dfc33407b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f5466a00640 0 client.27971 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc332b520 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc332ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 --2- 172.21.15.18:0/2393362923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc332da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 -- 172.21.15.18:0/2393362923 >> 172.21.15.18:0/2393362923 conn(0x55dfc32bf5d0 msgr2=0x55dfc33214c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 -- 172.21.15.18:0/2393362923 shutdown_connections 2024-04-16T17:13:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 1 -- 172.21.15.18:0/2393362923 wait complete. 2024-04-16T17:13:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.012+0000 7f548a831740 20 client.27971 trim_cache size 0 max 16384 2024-04-16T17:13:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.029+0000 7f548a831740 1 Processor -- start 2024-04-16T17:13:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f548a831740 1 -- start start 2024-04-16T17:13:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f548a831740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc3324a80 0x55dfc3339c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f548a831740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3346110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f548a831740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc3346650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f548a831740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5460012940 con 0x55dfc33257f0 2024-04-16T17:13:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f548a831740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5460010210 con 0x55dfc332e000 2024-04-16T17:13:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f548a831740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f546000e080 con 0x55dfc3324a80 2024-04-16T17:13:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f547fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3346110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f5485c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc3346650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f5485200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc3324a80 0x55dfc3339c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f5485200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc3324a80 0x55dfc3339c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:60438/0 (socket says 172.21.15.18:60438) 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.030+0000 7f5485200640 1 -- 172.21.15.18:0/2162533825 learned_addr learned my addr 172.21.15.18:0/2162533825 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.031+0000 7f5485200640 1 -- 172.21.15.18:0/2162533825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 msgr2=0x55dfc3346650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.031+0000 7f5485200640 1 --2- 172.21.15.18:0/2162533825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc3346650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.031+0000 7f5485200640 1 -- 172.21.15.18:0/2162533825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 msgr2=0x55dfc3346110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.031+0000 7f5485200640 1 --2- 172.21.15.18:0/2162533825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3346110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.031+0000 7f5485200640 1 -- 172.21.15.18:0/2162533825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc3340e30 con 0x55dfc3324a80 2024-04-16T17:13:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.031+0000 7f5485200640 1 --2- 172.21.15.18:0/2162533825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc3324a80 0x55dfc3339c60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5474009900 tx=0x7f54740058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0aeec5e5588155d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.031+0000 7f5471600640 1 -- 172.21.15.18:0/2162533825 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474042040 con 0x55dfc3324a80 2024-04-16T17:13:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f5471600640 1 -- 172.21.15.18:0/2162533825 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5474050070 con 0x55dfc3324a80 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f5471600640 1 -- 172.21.15.18:0/2162533825 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474050530 con 0x55dfc3324a80 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f548a831740 1 -- 172.21.15.18:0/2162533825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc3324a80 msgr2=0x55dfc3339c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f548a831740 1 --2- 172.21.15.18:0/2162533825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc3324a80 0x55dfc3339c60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5474009900 tx=0x7f54740058d0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f548a831740 1 -- 172.21.15.18:0/2162533825 shutdown_connections 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f548a831740 1 --2- 172.21.15.18:0/2162533825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc3346650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f548a831740 1 --2- 172.21.15.18:0/2162533825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc33257f0 0x55dfc3346110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f548a831740 1 --2- 172.21.15.18:0/2162533825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc3324a80 0x55dfc3339c60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f548a831740 1 -- 172.21.15.18:0/2162533825 >> 172.21.15.18:0/2162533825 conn(0x55dfc32bf5d0 msgr2=0x55dfc331f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.032+0000 7f548a831740 1 -- 172.21.15.18:0/2162533825 shutdown_connections 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 -- 172.21.15.18:0/2162533825 wait complete. 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 -- start start 2024-04-16T17:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 Processor -- start 2024-04-16T17:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc3327c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc3328160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc33286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f546000e6d0 con 0x55dfc3324a80 2024-04-16T17:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f546000fb40 con 0x55dfc332e000 2024-04-16T17:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.033+0000 7f548a831740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f546000eda0 con 0x55dfc33257f0 2024-04-16T17:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f548a831740 10 client.0 _ll_register_callbacks cb 0x7f5478011eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc3327c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc3327c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42370/0 (socket says 172.21.15.18:42370) 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485200640 1 -- 172.21.15.18:0/2186799981 learned_addr learned my addr 172.21.15.18:0/2186799981 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485c00640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc33286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f547fe00640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc3328160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485c00640 1 -- 172.21.15.18:0/2186799981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 msgr2=0x55dfc3328160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485c00640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc3328160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485c00640 1 -- 172.21.15.18:0/2186799981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 msgr2=0x55dfc3327c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485c00640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc3327c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485c00640 1 -- 172.21.15.18:0/2186799981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc3323d50 con 0x55dfc332e000 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f547fe00640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc3328160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.034+0000 7f5485200640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc3327c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f5485c00640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc33286a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5478017790 tx=0x7f5478018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf7957b87102902 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f5466a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f5466a00640 1 -- 172.21.15.18:0/2186799981 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478017c00 con 0x55dfc332e000 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f5466a00640 1 -- 172.21.15.18:0/2186799981 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f5478003a10 con 0x55dfc332e000 2024-04-16T17:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f548a831740 1 -- 172.21.15.18:0/2186799981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5474050070 con 0x55dfc332e000 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f5466a00640 1 -- 172.21.15.18:0/2186799981 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5478041050 con 0x55dfc332e000 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f548a831740 10 client.27977 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f548a831740 1 -- 172.21.15.18:0/2186799981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dfc33a4b90 con 0x55dfc332e000 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.035+0000 7f5466a00640 1 -- 172.21.15.18:0/2186799981 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5474050070 con 0x55dfc332e000 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.036+0000 7f5466a00640 1 -- 172.21.15.18:0/2186799981 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54780040b0 con 0x55dfc332e000 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.036+0000 7f548a831740 10 client.27977 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.036+0000 7f548a831740 10 client.27977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.036+0000 7f548a831740 20 client.27977 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.036+0000 7f548a831740 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfc33a5310 0x55dfc33a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.036+0000 7f548a831740 4 client.27977 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.036+0000 7f548a831740 1 -- 172.21.15.18:0/2186799981 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfc3291010 con 0x55dfc33a5310 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.036+0000 7f5485200640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfc33a5310 0x55dfc33a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.037+0000 7f5485200640 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfc33a5310 0x55dfc33a7730 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97fbd9504f036a36 server_cookie=caa0bdf053ac17b5 in_seq=0 out_seq=0 2024-04-16T17:13:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.037+0000 7f5466a00640 10 client.27977 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.160+0000 7f5466a00640 1 -- 172.21.15.18:0/2186799981 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfc3291010 con 0x55dfc33a5310 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.160+0000 7f5466a00640 10 client.27977 handle_command_reply: tid=0 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 client.27977 shutdown 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 -- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 msgr2=0x55dfc33286a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc33286a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5478017790 tx=0x7f5478018000 comp rx=0 tx=0).stop 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 -- 172.21.15.18:0/2186799981 shutdown_connections 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dfc33a5310 0x55dfc33a7730 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfc332e000 0x55dfc33286a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfc33257f0 0x55dfc3328160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 --2- 172.21.15.18:0/2186799981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dfc3324a80 0x55dfc3327c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 -- 172.21.15.18:0/2186799981 >> 172.21.15.18:0/2186799981 conn(0x55dfc32bf5d0 msgr2=0x55dfc3329b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f5466a00640 0 client.27977 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 -- 172.21.15.18:0/2186799981 shutdown_connections 2024-04-16T17:13:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.161+0000 7f548a831740 1 -- 172.21.15.18:0/2186799981 wait complete. 2024-04-16T17:13:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.162+0000 7f548a831740 20 client.27977 trim_cache size 0 max 16384 2024-04-16T17:13:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:13:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:13:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:13:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:13:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 -- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54680a3c90 msgr2=0x7f54680a6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54680a3c90 0x7f54680a6150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f547800a7f0 tx=0x7f547800d040 comp rx=0 tx=0).stop 2024-04-16T17:13:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 -- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5480103900 msgr2=0x7f54801a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5480103900 0x7f54801a0b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f546c00e9f0 tx=0x7f546c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 -- 172.21.15.18:0/2474094189 shutdown_connections 2024-04-16T17:13:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54680a3c90 0x7f54680a6150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548010c400 0x7f548019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5480103900 0x7f54801a0b50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 --2- 172.21.15.18:0/2474094189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5480102fe0 0x7f54801a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 -- 172.21.15.18:0/2474094189 >> 172.21.15.18:0/2474094189 conn(0x7f54800fc7e0 msgr2=0x7f5480100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 -- 172.21.15.18:0/2474094189 shutdown_connections 2024-04-16T17:13:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:32.163+0000 7f5486600640 1 -- 172.21.15.18:0/2474094189 wait complete. 2024-04-16T17:13:32.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 331746 3.73 MB/sec execute 345 sec latency 98.357 ms 2024-04-16T17:13:32.708 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:31 smithi018 ceph-mon[31776]: pgmap v274: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-16T17:13:33.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 332780 3.74 MB/sec execute 346 sec latency 23.388 ms 2024-04-16T17:13:33.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:33 smithi043 ceph-mon[36901]: pgmap v275: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-04-16T17:13:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:33 smithi022 ceph-mon[36023]: pgmap v275: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-04-16T17:13:34.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 333727 3.75 MB/sec execute 347 sec latency 12.802 ms 2024-04-16T17:13:35.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 334218 3.74 MB/sec execute 348 sec latency 30.853 ms 2024-04-16T17:13:35.725 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:35 smithi022 ceph-mon[36023]: pgmap v276: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 997 KiB/s wr, 78 op/s 2024-04-16T17:13:35.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:35 smithi043 ceph-mon[36901]: pgmap v276: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 997 KiB/s wr, 78 op/s 2024-04-16T17:13:35.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:33 smithi018 ceph-mon[31776]: pgmap v275: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-04-16T17:13:35.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:35 smithi018 ceph-mon[31776]: pgmap v276: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 997 KiB/s wr, 78 op/s 2024-04-16T17:13:36.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 334341 3.73 MB/sec execute 349 sec latency 727.667 ms 2024-04-16T17:13:37.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 335123 3.73 MB/sec execute 350 sec latency 31.926 ms 2024-04-16T17:13:37.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:37 smithi043 ceph-mon[36901]: pgmap v277: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:13:37.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:37 smithi022 ceph-mon[36023]: pgmap v277: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:13:38.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 336035 3.73 MB/sec execute 351 sec latency 32.968 ms 2024-04-16T17:13:39.621 INFO:tasks.workunit.client.0.smithi018.stdout: 1 337235 3.75 MB/sec execute 352 sec latency 11.502 ms 2024-04-16T17:13:39.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:39 smithi043 ceph-mon[36901]: pgmap v278: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 751 KiB/s wr, 101 op/s 2024-04-16T17:13:39.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:39 smithi022 ceph-mon[36023]: pgmap v278: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 751 KiB/s wr, 101 op/s 2024-04-16T17:13:40.623 INFO:tasks.workunit.client.0.smithi018.stdout: 1 337685 3.74 MB/sec execute 353 sec latency 23.965 ms 2024-04-16T17:13:41.623 INFO:tasks.workunit.client.0.smithi018.stdout: 1 338035 3.73 MB/sec execute 354 sec latency 97.019 ms 2024-04-16T17:13:41.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:41 smithi043 ceph-mon[36901]: pgmap v279: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-04-16T17:13:41.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:41 smithi022 ceph-mon[36023]: pgmap v279: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-04-16T17:13:42.623 INFO:tasks.workunit.client.0.smithi018.stdout: 1 338405 3.73 MB/sec execute 355 sec latency 75.965 ms 2024-04-16T17:13:43.623 INFO:tasks.workunit.client.0.smithi018.stdout: 1 339085 3.72 MB/sec execute 356 sec latency 30.724 ms 2024-04-16T17:13:43.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:43 smithi043 ceph-mon[36901]: pgmap v280: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 860 KiB/s wr, 106 op/s 2024-04-16T17:13:43.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:37 smithi018 ceph-mon[31776]: pgmap v277: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:13:43.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:39 smithi018 ceph-mon[31776]: pgmap v278: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 751 KiB/s wr, 101 op/s 2024-04-16T17:13:43.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:41 smithi018 ceph-mon[31776]: pgmap v279: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-04-16T17:13:43.781 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:43 smithi018 ceph-mon[31776]: pgmap v280: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 860 KiB/s wr, 106 op/s 2024-04-16T17:13:43.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:43 smithi022 ceph-mon[36023]: pgmap v280: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 860 KiB/s wr, 106 op/s 2024-04-16T17:13:44.623 INFO:tasks.workunit.client.0.smithi018.stdout: 1 339968 3.73 MB/sec execute 357 sec latency 44.662 ms 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 -- 172.21.15.18:0/3524590126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec10e8b0 msgr2=0x7facec110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 --2- 172.21.15.18:0/3524590126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec10e8b0 0x7facec110ca0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7face4009900 tx=0x7face402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 -- 172.21.15.18:0/3524590126 shutdown_connections 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 --2- 172.21.15.18:0/3524590126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec10e8b0 0x7facec110ca0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 --2- 172.21.15.18:0/3524590126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facec074cd0 0x7facec10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 --2- 172.21.15.18:0/3524590126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facec0743b0 0x7facec074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 -- 172.21.15.18:0/3524590126 >> 172.21.15.18:0/3524590126 conn(0x7facec0fdac0 msgr2=0x7facec0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 -- 172.21.15.18:0/3524590126 shutdown_connections 2024-04-16T17:13:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.305+0000 7facf1600640 1 -- 172.21.15.18:0/3524590126 wait complete. 2024-04-16T17:13:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.307+0000 7facf1600640 1 Processor -- start 2024-04-16T17:13:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.307+0000 7facf1600640 1 -- start start 2024-04-16T17:13:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.307+0000 7facf1600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facec0743b0 0x7facec1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7facf1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec074cd0 0x7facec1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7facf1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facec10e8b0 0x7facec19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7facf1600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7facec1145e0 con 0x7facec0743b0 2024-04-16T17:13:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7facf1600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7facec114460 con 0x7facec074cd0 2024-04-16T17:13:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7facf1600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7facec114760 con 0x7facec10e8b0 2024-04-16T17:13:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facec0743b0 0x7facec1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facec0743b0 0x7facec1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47118/0 (socket says 172.21.15.18:47118) 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceaa00640 1 -- 172.21.15.18:0/1539359619 learned_addr learned my addr 172.21.15.18:0/1539359619 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7facea000640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec074cd0 0x7facec1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceb400640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facec10e8b0 0x7facec19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceaa00640 1 -- 172.21.15.18:0/1539359619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facec10e8b0 msgr2=0x7facec19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceaa00640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facec10e8b0 0x7facec19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceaa00640 1 -- 172.21.15.18:0/1539359619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec074cd0 msgr2=0x7facec1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceaa00640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec074cd0 0x7facec1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.308+0000 7faceaa00640 1 -- 172.21.15.18:0/1539359619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec19b100 con 0x7facec0743b0 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.309+0000 7facea000640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec074cd0 0x7facec1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.309+0000 7faceaa00640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facec0743b0 0x7facec1a0780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7facdc00ebc0 tx=0x7facdc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.309+0000 7face3400640 1 -- 172.21.15.18:0/1539359619 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc010630 con 0x7facec0743b0 2024-04-16T17:13:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.309+0000 7face3400640 1 -- 172.21.15.18:0/1539359619 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facdc010cb0 con 0x7facec0743b0 2024-04-16T17:13:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.309+0000 7facf1600640 1 -- 172.21.15.18:0/1539359619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facec19b3f0 con 0x7facec0743b0 2024-04-16T17:13:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.309+0000 7face3400640 1 -- 172.21.15.18:0/1539359619 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc018bd0 con 0x7facec0743b0 2024-04-16T17:13:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.309+0000 7facf1600640 1 -- 172.21.15.18:0/1539359619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facec1a7640 con 0x7facec0743b0 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7facf4215740 1 Processor -- start 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7face3400640 1 -- 172.21.15.18:0/1539359619 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7facdc018420 con 0x7facec0743b0 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7facf4215740 1 -- start start 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7facf4215740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab96166360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7facf4215740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab961668a0 0x55ab96166d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7facf4215740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96172210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7facf4215740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ab960cb780 con 0x55ab9616fe20 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7facf4215740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ab96071f40 con 0x55ab961668a0 2024-04-16T17:13:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.313+0000 7facf4215740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ab960c7740 con 0x55ab96167610 2024-04-16T17:13:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7faceaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab961668a0 0x55ab96166d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7facea000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96172210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7faceaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab961668a0 0x55ab96166d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40836/0 (socket says 172.21.15.18:40836) 2024-04-16T17:13:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7facea000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96172210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47126/0 (socket says 172.21.15.18:47126) 2024-04-16T17:13:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7faceaa00640 1 -- 172.21.15.18:0/3648396878 learned_addr learned my addr 172.21.15.18:0/3648396878 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7faceb400640 1 --2- 172.21.15.18:0/3648396878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab96166360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7faceaa00640 1 -- 172.21.15.18:0/3648396878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 msgr2=0x55ab96166360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7faceaa00640 1 --2- 172.21.15.18:0/3648396878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab96166360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.314+0000 7faceaa00640 1 -- 172.21.15.18:0/3648396878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 msgr2=0x55ab96172210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7faceaa00640 1 --2- 172.21.15.18:0/3648396878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96172210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7faceaa00640 1 -- 172.21.15.18:0/3648396878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab95dde210 con 0x55ab961668a0 2024-04-16T17:13:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7face3400640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7facd00a3d10 0x7facd00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7faceaa00640 1 --2- 172.21.15.18:0/3648396878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab961668a0 0x55ab96166d20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7facdc0ce300 tx=0x7facdc0d9730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2503f6c03a6a3cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7face3400640 1 -- 172.21.15.18:0/1539359619 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7facdc0d10e0 con 0x7facec0743b0 2024-04-16T17:13:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7facd7400640 1 -- 172.21.15.18:0/3648396878 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc098f60 con 0x55ab961668a0 2024-04-16T17:13:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7faceb400640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7facd00a3d10 0x7facd00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7facd7400640 1 -- 172.21.15.18:0/3648396878 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facdc09ccc0 con 0x55ab961668a0 2024-04-16T17:13:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.315+0000 7facd7400640 1 -- 172.21.15.18:0/3648396878 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0d4c80 con 0x55ab961668a0 2024-04-16T17:13:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.316+0000 7facf4215740 1 -- 172.21.15.18:0/3648396878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab961668a0 msgr2=0x55ab96166d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.316+0000 7facf4215740 1 --2- 172.21.15.18:0/3648396878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab961668a0 0x55ab96166d20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7facdc0ce300 tx=0x7facdc0d9730 comp rx=0 tx=0).stop 2024-04-16T17:13:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7facf4215740 1 -- 172.21.15.18:0/3648396878 shutdown_connections 2024-04-16T17:13:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7facf4215740 1 --2- 172.21.15.18:0/3648396878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96172210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7facf4215740 1 --2- 172.21.15.18:0/3648396878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab961668a0 0x55ab96166d20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7facf4215740 1 --2- 172.21.15.18:0/3648396878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab96166360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7facf4215740 1 -- 172.21.15.18:0/3648396878 >> 172.21.15.18:0/3648396878 conn(0x55ab961015d0 msgr2=0x55ab96161590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7facf4215740 1 -- 172.21.15.18:0/3648396878 shutdown_connections 2024-04-16T17:13:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7faceb400640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7facd00a3d10 0x7facd00a61d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7face4005ab0 tx=0x7face402e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7facf4215740 1 -- 172.21.15.18:0/3648396878 wait complete. 2024-04-16T17:13:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.317+0000 7facf4215740 1 -- start start 2024-04-16T17:13:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.318+0000 7facf4215740 1 Processor -- start 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.318+0000 7facf4215740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab9616f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.318+0000 7facf4215740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab96167610 0x55ab9616cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.318+0000 7facf4215740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 0x55ab9616d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.318+0000 7facf4215740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ab96172b50 con 0x55ab96167610 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.318+0000 7facf4215740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ab96085390 con 0x55ab9616fe20 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.318+0000 7facf4215740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ab96172cd0 con 0x55ab961668a0 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7facf4215740 10 client.0 _ll_register_callbacks cb 0x55ab9606ef60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab9616f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 0x55ab9616d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab9616f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47164/0 (socket says 172.21.15.18:47164) 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 -- 172.21.15.18:0/2496371361 learned_addr learned my addr 172.21.15.18:0/2496371361 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7facea000640 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab96167610 0x55ab9616cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 -- 172.21.15.18:0/2496371361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 msgr2=0x55ab9616d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 0x55ab9616d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 -- 172.21.15.18:0/2496371361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab96167610 msgr2=0x55ab9616cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab96167610 0x55ab9616cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 -- 172.21.15.18:0/2496371361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab95eaa620 con 0x55ab961668a0 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faceaa00640 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab9616f8b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7facdc09cc50 tx=0x7facdc0d4700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761824287d7f7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.319+0000 7faccbe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.320+0000 7faccbe00640 1 -- 172.21.15.18:0/2496371361 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc010e60 con 0x55ab961668a0 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.320+0000 7faccbe00640 1 -- 172.21.15.18:0/2496371361 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7facdc0ce640 con 0x55ab961668a0 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.320+0000 7facf4215740 1 -- 172.21.15.18:0/2496371361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facdc09ccc0 con 0x55ab961668a0 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.320+0000 7faccbe00640 1 -- 172.21.15.18:0/2496371361 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7facdc09c620 con 0x55ab961668a0 2024-04-16T17:13:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.320+0000 7facf4215740 10 client.27901 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.320+0000 7facf4215740 1 -- 172.21.15.18:0/2496371361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab96172b50 con 0x55ab961668a0 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.320+0000 7faccbe00640 1 -- 172.21.15.18:0/2496371361 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facdc09ccc0 con 0x55ab961668a0 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.320+0000 7faccbe00640 1 -- 172.21.15.18:0/2496371361 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7facdc09cf20 con 0x55ab961668a0 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.321+0000 7facf4215740 10 client.27901 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.321+0000 7facf4215740 10 client.27901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.321+0000 7facf4215740 20 client.27901 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.321+0000 7facf4215740 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab9617fe30 0x55ab96182220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.321+0000 7facf4215740 4 client.27901 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.321+0000 7facf4215740 1 -- 172.21.15.18:0/2496371361 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab960d91a0 con 0x55ab9617fe30 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.321+0000 7facea000640 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab9617fe30 0x55ab96182220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.322+0000 7facea000640 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab9617fe30 0x55ab96182220 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff55e0d2c1035f7 server_cookie=851ff290d45e635d in_seq=0 out_seq=0 2024-04-16T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.322+0000 7faccbe00640 10 client.27901 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.322+0000 7faccbe00640 1 -- 172.21.15.18:0/2496371361 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab960d91a0 con 0x55ab9617fe30 2024-04-16T17:13:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.322+0000 7faccbe00640 10 client.27901 handle_command_reply: tid=0 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.322+0000 7facf4215740 1 client.27901 shutdown 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 -- 172.21.15.18:0/2496371361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 msgr2=0x55ab9616f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab9616f8b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7facdc09cc50 tx=0x7facdc0d4700 comp rx=0 tx=0).stop 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 -- 172.21.15.18:0/2496371361 shutdown_connections 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab9617fe30 0x55ab96182220 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 0x55ab9616d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab96167610 0x55ab9616cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 --2- 172.21.15.18:0/2496371361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab9616f8b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 -- 172.21.15.18:0/2496371361 >> 172.21.15.18:0/2496371361 conn(0x55ab961015d0 msgr2=0x55ab961632e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7faccbe00640 0 client.27901 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 -- 172.21.15.18:0/2496371361 shutdown_connections 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 1 -- 172.21.15.18:0/2496371361 wait complete. 2024-04-16T17:13:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.323+0000 7facf4215740 20 client.27901 trim_cache size 0 max 16384 2024-04-16T17:13:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.341+0000 7facf4215740 1 Processor -- start 2024-04-16T17:13:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.341+0000 7facf4215740 1 -- start start 2024-04-16T17:13:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.341+0000 7facf4215740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab961668a0 0x55ab9616cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.341+0000 7facf4215740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab9617bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.341+0000 7facf4215740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 0x55ab9616f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.341+0000 7facf4215740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faccc0128f0 con 0x55ab961668a0 2024-04-16T17:13:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.341+0000 7facf4215740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faccc0101c0 con 0x55ab9616fe20 2024-04-16T17:13:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.341+0000 7facf4215740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faccc00e030 con 0x55ab96167610 2024-04-16T17:13:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7faceaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab961668a0 0x55ab9616cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7faceb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 0x55ab9616f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7faceaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab961668a0 0x55ab9616cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47156/0 (socket says 172.21.15.18:47156) 2024-04-16T17:13:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7faceaa00640 1 -- 172.21.15.18:0/3273704219 learned_addr learned my addr 172.21.15.18:0/3273704219 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7facea000640 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab9617bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7facea000640 1 -- 172.21.15.18:0/3273704219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 msgr2=0x55ab9616f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7facea000640 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 0x55ab9616f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7facea000640 1 -- 172.21.15.18:0/3273704219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab961668a0 msgr2=0x55ab9616cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7facea000640 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab961668a0 0x55ab9616cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7facea000640 1 -- 172.21.15.18:0/3273704219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab96165b70 con 0x55ab96167610 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.342+0000 7faceaa00640 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab961668a0 0x55ab9616cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.343+0000 7facea000640 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab9617bae0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x55ab9616c4a0 tx=0x7facd800b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb3ec0073c9db15 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.343+0000 7facd6a00640 1 -- 172.21.15.18:0/3273704219 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8004220 con 0x55ab96167610 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.343+0000 7facd6a00640 1 -- 172.21.15.18:0/3273704219 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd8023070 con 0x55ab96167610 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.343+0000 7facd6a00640 1 -- 172.21.15.18:0/3273704219 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80047a0 con 0x55ab96167610 2024-04-16T17:13:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 -- 172.21.15.18:0/3273704219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 msgr2=0x55ab9617bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab9617bae0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x55ab9616c4a0 tx=0x7facd800b390 comp rx=0 tx=0).stop 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 -- 172.21.15.18:0/3273704219 shutdown_connections 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab9616fe20 0x55ab9616f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab96167610 0x55ab9617bae0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 --2- 172.21.15.18:0/3273704219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab961668a0 0x55ab9616cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 -- 172.21.15.18:0/3273704219 >> 172.21.15.18:0/3273704219 conn(0x55ab961015d0 msgr2=0x55ab9616c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 -- 172.21.15.18:0/3273704219 shutdown_connections 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 -- 172.21.15.18:0/3273704219 wait complete. 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 -- start start 2024-04-16T17:13:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.344+0000 7facf4215740 1 Processor -- start 2024-04-16T17:13:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7facf4215740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab96177ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7facf4215740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab96167610 0x55ab96178530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7facf4215740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96174fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7facf4215740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faccc00e680 con 0x55ab9616fe20 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7facf4215740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faccc00faf0 con 0x55ab96167610 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7facf4215740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faccc00ed50 con 0x55ab961668a0 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7facf4215740 10 client.0 _ll_register_callbacks cb 0x7facd800ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7faceb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96174fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7faceaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab96177ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7faceb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96174fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47168/0 (socket says 172.21.15.18:47168) 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7faceb400640 1 -- 172.21.15.18:0/2365390440 learned_addr learned my addr 172.21.15.18:0/2365390440 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.345+0000 7facea000640 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab96167610 0x55ab96178530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faceaa00640 1 -- 172.21.15.18:0/2365390440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab96167610 msgr2=0x55ab96178530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faceaa00640 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab96167610 0x55ab96178530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faceaa00640 1 -- 172.21.15.18:0/2365390440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 msgr2=0x55ab96174fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faceaa00640 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96174fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faceaa00640 1 -- 172.21.15.18:0/2365390440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab961757d0 con 0x55ab961668a0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faceaa00640 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab96177ff0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7facdc09af70 tx=0x7facdc09d1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420bede2b22ef551 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faccbe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faccbe00640 1 -- 172.21.15.18:0/2365390440 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc010e60 con 0x55ab961668a0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faccbe00640 1 -- 172.21.15.18:0/2365390440 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7facdc09c110 con 0x55ab961668a0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7facf4215740 1 -- 172.21.15.18:0/2365390440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd8023070 con 0x55ab961668a0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7faccbe00640 1 -- 172.21.15.18:0/2365390440 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7facdc09c800 con 0x55ab961668a0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7facf4215740 10 client.27907 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.346+0000 7facf4215740 1 -- 172.21.15.18:0/2365390440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab961e6d30 con 0x55ab961668a0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.347+0000 7faccbe00640 1 -- 172.21.15.18:0/2365390440 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd8023070 con 0x55ab961668a0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.347+0000 7faccbe00640 1 -- 172.21.15.18:0/2365390440 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7facdc0ceb10 con 0x55ab961668a0 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.348+0000 7facf4215740 10 client.27907 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.348+0000 7facf4215740 10 client.27907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.348+0000 7facf4215740 20 client.27907 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.348+0000 7facf4215740 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab961e7140 0x55ab961e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.348+0000 7facf4215740 4 client.27907 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:13:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.348+0000 7facf4215740 1 -- 172.21.15.18:0/2365390440 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab960d3010 con 0x55ab961e7140 2024-04-16T17:13:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.349+0000 7facea000640 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab961e7140 0x55ab961e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.349+0000 7facea000640 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab961e7140 0x55ab961e9580 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4857843db49403 server_cookie=db730cce4ed2d0c8 in_seq=0 out_seq=0 2024-04-16T17:13:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.349+0000 7faccbe00640 10 client.27907 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.492+0000 7faccbe00640 1 -- 172.21.15.18:0/2365390440 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab960d3010 con 0x55ab961e7140 2024-04-16T17:13:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.492+0000 7faccbe00640 10 client.27907 handle_command_reply: tid=0 2024-04-16T17:13:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.492+0000 7facf4215740 1 client.27907 shutdown 2024-04-16T17:13:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.492+0000 7facf4215740 1 -- 172.21.15.18:0/2365390440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 msgr2=0x55ab96177ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.492+0000 7facf4215740 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab96177ff0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7facdc09af70 tx=0x7facdc09d1b0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 1 -- 172.21.15.18:0/2365390440 shutdown_connections 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab961e7140 0x55ab961e9580 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab9616fe20 0x55ab96174fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab96167610 0x55ab96178530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 1 --2- 172.21.15.18:0/2365390440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab961668a0 0x55ab96177ff0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 1 -- 172.21.15.18:0/2365390440 >> 172.21.15.18:0/2365390440 conn(0x55ab961015d0 msgr2=0x55ab9615fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7faccbe00640 0 client.27907 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 1 -- 172.21.15.18:0/2365390440 shutdown_connections 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 1 -- 172.21.15.18:0/2365390440 wait complete. 2024-04-16T17:13:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.493+0000 7facf4215740 20 client.27907 trim_cache size 0 max 16384 2024-04-16T17:13:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:13:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:13:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:13:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 -- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7facd00a3d10 msgr2=0x7facd00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7facd00a3d10 0x7facd00a61d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7face4005ab0 tx=0x7face402e040 comp rx=0 tx=0).stop 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 -- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facec0743b0 msgr2=0x7facec1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facec0743b0 0x7facec1a0780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7facdc00ebc0 tx=0x7facdc00c6a0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 -- 172.21.15.18:0/1539359619 shutdown_connections 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7facd00a3d10 0x7facd00a61d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facec10e8b0 0x7facec19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facec074cd0 0x7facec1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 --2- 172.21.15.18:0/1539359619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7facec0743b0 0x7facec1a0780 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.494+0000 7facf1600640 1 -- 172.21.15.18:0/1539359619 >> 172.21.15.18:0/1539359619 conn(0x7facec0fdac0 msgr2=0x7facec0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.495+0000 7facf1600640 1 -- 172.21.15.18:0/1539359619 shutdown_connections 2024-04-16T17:13:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:45.495+0000 7facf1600640 1 -- 172.21.15.18:0/1539359619 wait complete. 2024-04-16T17:13:45.625 INFO:tasks.workunit.client.0.smithi018.stdout: 1 340813 3.74 MB/sec execute 358 sec latency 75.013 ms 2024-04-16T17:13:45.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:45 smithi022 ceph-mon[36023]: pgmap v281: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 853 KiB/s wr, 104 op/s 2024-04-16T17:13:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:45 smithi043 ceph-mon[36901]: pgmap v281: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 853 KiB/s wr, 104 op/s 2024-04-16T17:13:46.625 INFO:tasks.workunit.client.0.smithi018.stdout: 1 341053 3.73 MB/sec execute 359 sec latency 91.328 ms 2024-04-16T17:13:47.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 341377 3.72 MB/sec execute 360 sec latency 69.764 ms 2024-04-16T17:13:47.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:47 smithi022 ceph-mon[36023]: pgmap v282: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:13:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:47 smithi043 ceph-mon[36901]: pgmap v282: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:13:48.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 342065 3.72 MB/sec execute 361 sec latency 44.930 ms 2024-04-16T17:13:49.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 342915 3.72 MB/sec execute 362 sec latency 34.860 ms 2024-04-16T17:13:49.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:49 smithi022 ceph-mon[36023]: pgmap v283: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 885 KiB/s wr, 68 op/s 2024-04-16T17:13:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:49 smithi043 ceph-mon[36901]: pgmap v283: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 885 KiB/s wr, 68 op/s 2024-04-16T17:13:50.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 343991 3.73 MB/sec execute 363 sec latency 24.303 ms 2024-04-16T17:13:51.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 344292 3.73 MB/sec execute 364 sec latency 172.541 ms 2024-04-16T17:13:51.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:13:51.888 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:13:51.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:51 smithi022 ceph-mon[36023]: pgmap v284: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:13:52.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:51 smithi043 ceph-mon[36901]: pgmap v284: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:13:52.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 -- 172.21.15.18:0/1069437812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98102070 msgr2=0x7f2b98102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 --2- 172.21.15.18:0/1069437812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98102070 0x7f2b98102450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009900 tx=0x7f2b8802bc50 comp rx=0 tx=0).stop 2024-04-16T17:13:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 -- 172.21.15.18:0/1069437812 shutdown_connections 2024-04-16T17:13:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 --2- 172.21.15.18:0/1069437812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b98107140 0x7f2b9810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 --2- 172.21.15.18:0/1069437812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b98102a20 0x7f2b98106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 --2- 172.21.15.18:0/1069437812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98102070 0x7f2b98102450 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 -- 172.21.15.18:0/1069437812 >> 172.21.15.18:0/1069437812 conn(0x7f2b980fd950 msgr2=0x7f2b980ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 -- 172.21.15.18:0/1069437812 shutdown_connections 2024-04-16T17:13:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.068+0000 7f2b97e00640 1 -- 172.21.15.18:0/1069437812 wait complete. 2024-04-16T17:13:52.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.069+0000 7f2b97e00640 1 Processor -- start 2024-04-16T17:13:52.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.070+0000 7f2b97e00640 1 -- start start 2024-04-16T17:13:52.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.070+0000 7f2b97e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b98102070 0x7f2b981a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.070+0000 7f2b97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98107140 0x7f2b981a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.070+0000 7f2b97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9819a7f0 0x7f2b9819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.070+0000 7f2b97e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981121d0 con 0x7f2b98102070 2024-04-16T17:13:52.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.070+0000 7f2b97e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98112050 con 0x7f2b98107140 2024-04-16T17:13:52.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.070+0000 7f2b97e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98112350 con 0x7f2b9819a7f0 2024-04-16T17:13:52.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b98102070 0x7f2b981a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98107140 0x7f2b981a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98107140 0x7f2b981a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44676/0 (socket says 172.21.15.18:44676) 2024-04-16T17:13:52.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 -- 172.21.15.18:0/4279077485 learned_addr learned my addr 172.21.15.18:0/4279077485 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b97400640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9819a7f0 0x7f2b9819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 -- 172.21.15.18:0/4279077485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9819a7f0 msgr2=0x7f2b9819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9819a7f0 0x7f2b9819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 -- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b98102070 msgr2=0x7f2b981a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b98102070 0x7f2b981a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 -- 172.21.15.18:0/4279077485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9819b380 con 0x7f2b98107140 2024-04-16T17:13:52.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b97400640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9819a7f0 0x7f2b9819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:52.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96a00640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b98102070 0x7f2b981a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:52.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b96000640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98107140 0x7f2b981a0bc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400d770 tx=0x7f2b8400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b8f400640 1 -- 172.21.15.18:0/4279077485 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84017070 con 0x7f2b98107140 2024-04-16T17:13:52.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b8f400640 1 -- 172.21.15.18:0/4279077485 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b84009c30 con 0x7f2b98107140 2024-04-16T17:13:52.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b8f400640 1 -- 172.21.15.18:0/4279077485 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8400f620 con 0x7f2b98107140 2024-04-16T17:13:52.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b97e00640 1 -- 172.21.15.18:0/4279077485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9819b670 con 0x7f2b98107140 2024-04-16T17:13:52.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b97e00640 1 -- 172.21.15.18:0/4279077485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b981a78a0 con 0x7f2b98107140 2024-04-16T17:13:52.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2ba0391740 1 Processor -- start 2024-04-16T17:13:52.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2ba0391740 1 -- start start 2024-04-16T17:13:52.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e3e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893e49c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2ba0391740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d893da4120 con 0x55d893e47890 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2ba0391740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d893d4a340 con 0x55d893e3e390 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2ba0391740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d893da00e0 con 0x55d893e3ecb0 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e3e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e3e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44690/0 (socket says 172.21.15.18:44690) 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.071+0000 7f2b97400640 1 -- 172.21.15.18:0/1994889215 learned_addr learned my addr 172.21.15.18:0/1994889215 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b96a00640 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b97400640 1 -- 172.21.15.18:0/1994889215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 msgr2=0x55d893e47350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b97400640 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e47350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b97400640 1 -- 172.21.15.18:0/1994889215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 msgr2=0x55d893e49c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b97400640 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893e49c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b97400640 1 -- 172.21.15.18:0/1994889215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d893b807d0 con 0x55d893e3e390 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b96a00640 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e47350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:52.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b97400640 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e3e770 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009fd0 tx=0x7f2b90010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f475d206f48c17ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b83400640 1 -- 172.21.15.18:0/1994889215 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90011700 con 0x55d893e3e390 2024-04-16T17:13:52.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b83400640 1 -- 172.21.15.18:0/1994889215 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b90011d80 con 0x55d893e3e390 2024-04-16T17:13:52.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b83400640 1 -- 172.21.15.18:0/1994889215 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000e3c0 con 0x55d893e3e390 2024-04-16T17:13:52.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2b8f400640 1 -- 172.21.15.18:0/4279077485 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b8401a070 con 0x7f2b98107140 2024-04-16T17:13:52.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 -- 172.21.15.18:0/1994889215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 msgr2=0x55d893e3e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e3e770 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009fd0 tx=0x7f2b90010860 comp rx=0 tx=0).stop 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 -- 172.21.15.18:0/1994889215 shutdown_connections 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893e49c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e47350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 --2- 172.21.15.18:0/1994889215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e3e770 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 -- 172.21.15.18:0/1994889215 >> 172.21.15.18:0/1994889215 conn(0x55d893e38ac0 msgr2=0x55d893e38ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 -- 172.21.15.18:0/1994889215 shutdown_connections 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.077+0000 7f2ba0391740 1 -- 172.21.15.18:0/1994889215 wait complete. 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- start start 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 Processor -- start 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3e390 0x55d893e44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e44e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d893e4a680 con 0x55d893e3e390 2024-04-16T17:13:52.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d893d5d710 con 0x55d893e47890 2024-04-16T17:13:52.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d893e4a800 con 0x55d893e3ecb0 2024-04-16T17:13:52.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 10 client.0 _ll_register_callbacks cb 0x55d893d58af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:52.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b8f400640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b7c0a3cb0 0x7f2b7c0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44694/0 (socket says 172.21.15.18:44694) 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 -- 172.21.15.18:0/3290506017 learned_addr learned my addr 172.21.15.18:0/3290506017 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b8f400640 1 -- 172.21.15.18:0/4279077485 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f2b840d06b0 con 0x7f2b98107140 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96a00640 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e44e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 -- 172.21.15.18:0/3290506017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 msgr2=0x55d893e44e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e44e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b97400640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b7c0a3cb0 0x7f2b7c0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 -- 172.21.15.18:0/3290506017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3e390 msgr2=0x55d893e44950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3e390 0x55d893e44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 -- 172.21.15.18:0/3290506017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d893ab63f0 con 0x55d893e47890 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96000640 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e453d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2b840a4980 tx=0x7f2b8409c0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d47aa2702c95aed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:52.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 1 -- 172.21.15.18:0/3290506017 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84014020 con 0x55d893e47890 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- 172.21.15.18:0/3290506017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b90011d80 con 0x55d893e47890 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 1 -- 172.21.15.18:0/3290506017 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f2b840a0120 con 0x55d893e47890 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 1 -- 172.21.15.18:0/3290506017 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2b84021070 con 0x55d893e47890 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 10 client.28007 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- 172.21.15.18:0/3290506017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d893a760f0 con 0x55d893e47890 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 1 -- 172.21.15.18:0/3290506017 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b90011d80 con 0x55d893e47890 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 1 -- 172.21.15.18:0/3290506017 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2b840b0ce0 con 0x55d893e47890 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 10 client.28007 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 10 client.28007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 20 client.28007 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d893e57cd0 0x55d893e5a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 4 client.28007 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- 172.21.15.18:0/3290506017 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d893db1b40 con 0x55d893e57cd0 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b97400640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b7c0a3cb0 0x7f2b7c0a6170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2b9001bc20 tx=0x7f2b90013040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96a00640 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d893e57cd0 0x55d893e5a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b96a00640 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d893e57cd0 0x55d893e5a0f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5922ac1e821abe58 server_cookie=5fd8f5ffba9d98c in_seq=0 out_seq=0 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 10 client.28007 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 1 -- 172.21.15.18:0/3290506017 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d893db1b40 con 0x55d893e57cd0 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 10 client.28007 handle_command_reply: tid=0 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 client.28007 shutdown 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- 172.21.15.18:0/3290506017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 msgr2=0x55d893e453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e453d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2b840a4980 tx=0x7f2b8409c0d0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- 172.21.15.18:0/3290506017 shutdown_connections 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d893e57cd0 0x55d893e5a0f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2b77e00640 0 client.28007 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e453d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e44e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3290506017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3e390 0x55d893e44950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- 172.21.15.18:0/3290506017 >> 172.21.15.18:0/3290506017 conn(0x55d893e38ac0 msgr2=0x55d893e3a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- 172.21.15.18:0/3290506017 shutdown_connections 2024-04-16T17:13:52.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 1 -- 172.21.15.18:0/3290506017 wait complete. 2024-04-16T17:13:52.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.079+0000 7f2ba0391740 20 client.28007 trim_cache size 0 max 16384 2024-04-16T17:13:52.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 Processor -- start 2024-04-16T17:13:52.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- start start 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3e390 0x55d893e53570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3ecb0 0x55d893e43ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981b40e0 con 0x55d893e3ecb0 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981b19b0 con 0x55d893e47890 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981af820 con 0x55d893e3e390 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3ecb0 0x55d893e43ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3ecb0 0x55d893e43ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52742/0 (socket says 172.21.15.18:52742) 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 -- 172.21.15.18:0/1918056204 learned_addr learned my addr 172.21.15.18:0/1918056204 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96a00640 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3e390 0x55d893e53570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b97400640 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 -- 172.21.15.18:0/1918056204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3e390 msgr2=0x55d893e53570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3e390 0x55d893e53570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 -- 172.21.15.18:0/1918056204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 msgr2=0x55d893e451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 -- 172.21.15.18:0/1918056204 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d893e3eae0 con 0x55d893e3ecb0 2024-04-16T17:13:52.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96a00640 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3e390 0x55d893e53570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:52.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b96000640 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3ecb0 0x55d893e43ab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b84098c10 tx=0x7f2b840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76ac1d6a610dd13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b82a00640 1 -- 172.21.15.18:0/1918056204 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84014070 con 0x55d893e3ecb0 2024-04-16T17:13:52.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b82a00640 1 -- 172.21.15.18:0/1918056204 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b84009de0 con 0x55d893e3ecb0 2024-04-16T17:13:52.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2b82a00640 1 -- 172.21.15.18:0/1918056204 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840d04a0 con 0x55d893e3ecb0 2024-04-16T17:13:52.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- 172.21.15.18:0/1918056204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3ecb0 msgr2=0x55d893e43ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3ecb0 0x55d893e43ab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b84098c10 tx=0x7f2b840057f0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- 172.21.15.18:0/1918056204 shutdown_connections 2024-04-16T17:13:52.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e47890 0x55d893e451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e3ecb0 0x55d893e43ab0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 --2- 172.21.15.18:0/1918056204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3e390 0x55d893e53570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- 172.21.15.18:0/1918056204 >> 172.21.15.18:0/1918056204 conn(0x55d893e38ac0 msgr2=0x55d893e37290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- 172.21.15.18:0/1918056204 shutdown_connections 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- 172.21.15.18:0/1918056204 wait complete. 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 -- start start 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.101+0000 7f2ba0391740 1 Processor -- start 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2ba0391740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893ebc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2ba0391740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981afe70 con 0x55d893e47890 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2ba0391740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981b12e0 con 0x55d893e3e390 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2ba0391740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981b0540 con 0x55d893e3ecb0 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2ba0391740 10 client.0 _ll_register_callbacks cb 0x7f2b84097ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893ebc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893ebc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52748/0 (socket says 172.21.15.18:52748) 2024-04-16T17:13:52.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b97400640 1 -- 172.21.15.18:0/3077618669 learned_addr learned my addr 172.21.15.18:0/3077618669 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e4ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44720/0 (socket says 172.21.15.18:44720) 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96000640 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96a00640 1 -- 172.21.15.18:0/3077618669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 msgr2=0x55d893e4b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96a00640 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e4b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96a00640 1 -- 172.21.15.18:0/3077618669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 msgr2=0x55d893ebc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96a00640 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893ebc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96a00640 1 -- 172.21.15.18:0/3077618669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d893ebc790 con 0x55d893e3e390 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b97400640 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893ebc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b96a00640 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e4ac30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000c00 tx=0x7f2b88009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe37704b5a80ef39 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b77e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:52.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b77e00640 1 -- 172.21.15.18:0/3077618669 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8804b070 con 0x55d893e3e390 2024-04-16T17:13:52.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2b77e00640 1 -- 172.21.15.18:0/3077618669 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f2b88004240 con 0x55d893e3e390 2024-04-16T17:13:52.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.107+0000 7f2ba0391740 1 -- 172.21.15.18:0/3077618669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b84009de0 con 0x55d893e3e390 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b77e00640 1 -- 172.21.15.18:0/3077618669 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2b88057050 con 0x55d893e3e390 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 10 client.28013 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 -- 172.21.15.18:0/3077618669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d893ebda50 con 0x55d893e3e390 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b77e00640 1 -- 172.21.15.18:0/3077618669 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b84009de0 con 0x55d893e3e390 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b77e00640 1 -- 172.21.15.18:0/3077618669 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2b88047730 con 0x55d893e3e390 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 10 client.28013 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 10 client.28013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 20 client.28013 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d893e57cd0 0x55d893ec0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 4 client.28013 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 -- 172.21.15.18:0/3077618669 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d893dab9b0 con 0x55d893e57cd0 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b96000640 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d893e57cd0 0x55d893ec0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b96000640 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d893e57cd0 0x55d893ec0130 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d4d38da3c8f295 server_cookie=a2ea524e7fdcaa7e in_seq=0 out_seq=0 2024-04-16T17:13:52.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b77e00640 10 client.28013 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b77e00640 1 -- 172.21.15.18:0/3077618669 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d893dab9b0 con 0x55d893e57cd0 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b77e00640 10 client.28013 handle_command_reply: tid=0 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 client.28013 shutdown 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 -- 172.21.15.18:0/3077618669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 msgr2=0x55d893e4ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e4ac30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000c00 tx=0x7f2b88009680 comp rx=0 tx=0).stop 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 -- 172.21.15.18:0/3077618669 shutdown_connections 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d893e57cd0 0x55d893ec0130 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d893e47890 0x55d893ebc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d893e3ecb0 0x55d893e4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b77e00640 0 client.28013 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 --2- 172.21.15.18:0/3077618669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d893e3e390 0x55d893e4ac30 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 -- 172.21.15.18:0/3077618669 >> 172.21.15.18:0/3077618669 conn(0x55d893e38ac0 msgr2=0x55d893e37b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 -- 172.21.15.18:0/3077618669 shutdown_connections 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 1 -- 172.21.15.18:0/3077618669 wait complete. 2024-04-16T17:13:52.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2ba0391740 20 client.28013 trim_cache size 0 max 16384 2024-04-16T17:13:52.115 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:13:52.115 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:13:52.115 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:13:52.115 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 -- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b7c0a3cb0 msgr2=0x7f2b7c0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b7c0a3cb0 0x7f2b7c0a6170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2b9001bc20 tx=0x7f2b90013040 comp rx=0 tx=0).stop 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 -- 172.21.15.18:0/4279077485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98107140 msgr2=0x7f2b981a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98107140 0x7f2b981a0bc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400d770 tx=0x7f2b8400dc30 comp rx=0 tx=0).stop 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 -- 172.21.15.18:0/4279077485 shutdown_connections 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b7c0a3cb0 0x7f2b7c0a6170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9819a7f0 0x7f2b9819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b98107140 0x7f2b981a0bc0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 --2- 172.21.15.18:0/4279077485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b98102070 0x7f2b981a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 -- 172.21.15.18:0/4279077485 >> 172.21.15.18:0/4279077485 conn(0x7f2b980fd950 msgr2=0x7f2b980fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 -- 172.21.15.18:0/4279077485 shutdown_connections 2024-04-16T17:13:52.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.109+0000 7f2b97e00640 1 -- 172.21.15.18:0/4279077485 wait complete. 2024-04-16T17:13:52.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:13:52.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:13:52.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2774898876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fb4102150 msgr2=0x7f1fb4102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/2774898876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fb4102150 0x7f1fb4102530 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009900 tx=0x7f1fa802bc50 comp rx=0 tx=0).stop 2024-04-16T17:13:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2774898876 shutdown_connections 2024-04-16T17:13:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/2774898876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fb4107220 0x7f1fb410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/2774898876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fb4102b00 0x7f1fb4106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/2774898876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fb4102150 0x7f1fb4102530 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2774898876 >> 172.21.15.18:0/2774898876 conn(0x7f1fb40fd9f0 msgr2=0x7f1fb40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2774898876 shutdown_connections 2024-04-16T17:13:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.319+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2774898876 wait complete. 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.320+0000 7f1fbaa00640 1 Processor -- start 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.320+0000 7f1fbaa00640 1 -- start start 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.320+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fb4102150 0x7f1fb4074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.320+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fb4107220 0x7f1fb4074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.320+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fb4074d80 0x7f1fb41a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.320+0000 7f1fbaa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41122b0 con 0x7f1fb4107220 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.320+0000 7f1fbaa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4112130 con 0x7f1fb4102150 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.320+0000 7f1fbaa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4112430 con 0x7f1fb4074d80 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fb4107220 0x7f1fb4074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fb4107220 0x7f1fb4074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52758/0 (socket says 172.21.15.18:52758) 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 -- 172.21.15.18:0/1508741135 learned_addr learned my addr 172.21.15.18:0/1508741135 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb9600640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fb4102150 0x7f1fb4074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fba000640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fb4074d80 0x7f1fb41a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 -- 172.21.15.18:0/1508741135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fb4074d80 msgr2=0x7f1fb41a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fb4074d80 0x7f1fb41a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 -- 172.21.15.18:0/1508741135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fb4102150 msgr2=0x7f1fb4074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fb4102150 0x7f1fb4074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 -- 172.21.15.18:0/1508741135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb41a7100 con 0x7f1fb4107220 2024-04-16T17:13:52.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.321+0000 7f1fb8c00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fb4107220 0x7f1fb4074840 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000e9a0 tx=0x7f1fa000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.322+0000 7f1fb2000640 1 -- 172.21.15.18:0/1508741135 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000cd80 con 0x7f1fb4107220 2024-04-16T17:13:52.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.322+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb41a73f0 con 0x7f1fb4107220 2024-04-16T17:13:52.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.322+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb41a7990 con 0x7f1fb4107220 2024-04-16T17:13:52.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.322+0000 7f1fb2000640 1 -- 172.21.15.18:0/1508741135 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa00035f0 con 0x7f1fb4107220 2024-04-16T17:13:52.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.322+0000 7f1fb2000640 1 -- 172.21.15.18:0/1508741135 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0010670 con 0x7f1fb4107220 2024-04-16T17:13:52.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.324+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f94005290 con 0x7f1fb4107220 2024-04-16T17:13:52.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.327+0000 7f1fb2000640 1 -- 172.21.15.18:0/1508741135 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1fa0004000 con 0x7f1fb4107220 2024-04-16T17:13:52.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.329+0000 7f1fb2000640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f9c0a3d90 0x7f1f9c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.330+0000 7f1fb2000640 1 -- 172.21.15.18:0/1508741135 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f1fa00d0e20 con 0x7f1fb4107220 2024-04-16T17:13:52.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.330+0000 7f1fb9600640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f9c0a3d90 0x7f1f9c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.331+0000 7f1fb9600640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f9c0a3d90 0x7f1f9c0a6250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8006010 tx=0x7f1fa8005b50 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.335+0000 7f1fb2000640 1 -- 172.21.15.18:0/1508741135 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fa009d050 con 0x7f1fb4107220 2024-04-16T17:13:52.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 344479 3.72 MB/sec execute 365 sec latency 91.980 ms 2024-04-16T17:13:52.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.702+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f94005850 con 0x7f1fb4107220 2024-04-16T17:13:52.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.704+0000 7f1fb2000640 1 -- 172.21.15.18:0/1508741135 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11851 (secure 0 0 0) 0x7f1fa009c950 con 0x7f1fb4107220 2024-04-16T17:13:52.704 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:13:52.705 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:11:38.382274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27445":{"gid":27445,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/678063310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":678063310},{"type":"v1","addr":"172.21.15.22:6837","nonce":678063310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:13:52.705 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 27 2024-04-16T17:13:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f9c0a3d90 msgr2=0x7f1f9c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f9c0a3d90 0x7f1f9c0a6250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8006010 tx=0x7f1fa8005b50 comp rx=0 tx=0).stop 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fb4107220 msgr2=0x7f1fb4074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fb4107220 0x7f1fb4074840 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000e9a0 tx=0x7f1fa000ee60 comp rx=0 tx=0).stop 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 shutdown_connections 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f9c0a3d90 0x7f1f9c0a6250 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fb4074d80 0x7f1fb41a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fb4107220 0x7f1fb4074840 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 --2- 172.21.15.18:0/1508741135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fb4102150 0x7f1fb4074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 >> 172.21.15.18:0/1508741135 conn(0x7f1fb40fd9f0 msgr2=0x7f1fb40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 shutdown_connections 2024-04-16T17:13:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.707+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1508741135 wait complete. 2024-04-16T17:13:52.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:13:52.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 -- 172.21.15.18:0/4023584541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad70109350 msgr2=0x7fad70110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 --2- 172.21.15.18:0/4023584541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad70109350 0x7fad70110f20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fad68009be0 tx=0x7fad6802bef0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 -- 172.21.15.18:0/4023584541 shutdown_connections 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 --2- 172.21.15.18:0/4023584541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad70109350 0x7fad70110f20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 --2- 172.21.15.18:0/4023584541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad70104c30 0x7fad70108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 --2- 172.21.15.18:0/4023584541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad70104280 0x7fad70104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 -- 172.21.15.18:0/4023584541 >> 172.21.15.18:0/4023584541 conn(0x7fad700ffae0 msgr2=0x7fad70101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 -- 172.21.15.18:0/4023584541 shutdown_connections 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.920+0000 7fad6f200640 1 -- 172.21.15.18:0/4023584541 wait complete. 2024-04-16T17:13:52.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.921+0000 7fad6f200640 1 Processor -- start 2024-04-16T17:13:52.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.921+0000 7fad6f200640 1 -- start start 2024-04-16T17:13:52.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad70104280 0x7fad701a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad70109350 0x7fad701a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad7019a6e0 0x7fad7019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fad70114440 con 0x7fad7019a6e0 2024-04-16T17:13:52.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad701142c0 con 0x7fad70109350 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad701145c0 con 0x7fad70104280 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad70104280 0x7fad701a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad70104280 0x7fad701a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42234/0 (socket says 172.21.15.18:42234) 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6de00640 1 -- 172.21.15.18:0/883415422 learned_addr learned my addr 172.21.15.18:0/883415422 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad67e00640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad70109350 0x7fad701a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.922+0000 7fad6e800640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad7019a6e0 0x7fad7019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad6de00640 1 -- 172.21.15.18:0/883415422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad70109350 msgr2=0x7fad701a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad6de00640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad70109350 0x7fad701a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad6de00640 1 -- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad7019a6e0 msgr2=0x7fad7019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad6de00640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad7019a6e0 0x7fad7019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad6de00640 1 -- 172.21.15.18:0/883415422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7019b2e0 con 0x7fad70104280 2024-04-16T17:13:52.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad6de00640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad70104280 0x7fad701a0620 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fad6000d950 tx=0x7fad6000dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad66a00640 1 -- 172.21.15.18:0/883415422 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60017070 con 0x7fad70104280 2024-04-16T17:13:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad66a00640 1 -- 172.21.15.18:0/883415422 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6000de80 con 0x7fad70104280 2024-04-16T17:13:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.923+0000 7fad66a00640 1 -- 172.21.15.18:0/883415422 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6000f620 con 0x7fad70104280 2024-04-16T17:13:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.924+0000 7fad6f200640 1 -- 172.21.15.18:0/883415422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad701a73c0 con 0x7fad70104280 2024-04-16T17:13:52.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.924+0000 7fad6f200640 1 -- 172.21.15.18:0/883415422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad701a77c0 con 0x7fad70104280 2024-04-16T17:13:52.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.928+0000 7fad77bfa740 1 Processor -- start 2024-04-16T17:13:52.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.928+0000 7fad77bfa740 1 -- start start 2024-04-16T17:13:52.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.928+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398baf8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf86d0 0x56398bb096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb0c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad77bfa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56398ba5d020 con 0x56398baf7d20 2024-04-16T17:13:52.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad77bfa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56398ba03460 con 0x56398bb09c20 2024-04-16T17:13:52.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad77bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56398ba58fe0 con 0x56398baf86d0 2024-04-16T17:13:52.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad67e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb0c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad6e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398baf8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad67e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb0c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44772/0 (socket says 172.21.15.18:44772) 2024-04-16T17:13:52.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad67e00640 1 -- 172.21.15.18:0/3730774887 learned_addr learned my addr 172.21.15.18:0/3730774887 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad66a00640 1 -- 172.21.15.18:0/883415422 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad60009c30 con 0x7fad70104280 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad67e00640 1 -- 172.21.15.18:0/3730774887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf86d0 msgr2=0x56398bb096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad67e00640 1 --2- 172.21.15.18:0/3730774887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf86d0 0x56398bb096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad67e00640 1 -- 172.21.15.18:0/3730774887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 msgr2=0x56398baf8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.929+0000 7fad67e00640 1 --2- 172.21.15.18:0/3730774887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398baf8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.930+0000 7fad67e00640 1 -- 172.21.15.18:0/3730774887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56398ba0b040 con 0x56398bb09c20 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.930+0000 7fad67e00640 1 --2- 172.21.15.18:0/3730774887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb0c010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fad58009c10 tx=0x7fad58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8868c28689d565d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.930+0000 7fad5f400640 1 -- 172.21.15.18:0/3730774887 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58004230 con 0x56398bb09c20 2024-04-16T17:13:52.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.930+0000 7fad5f400640 1 -- 172.21.15.18:0/3730774887 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad580043d0 con 0x56398bb09c20 2024-04-16T17:13:52.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.930+0000 7fad5f400640 1 -- 172.21.15.18:0/3730774887 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58019880 con 0x56398bb09c20 2024-04-16T17:13:52.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 -- 172.21.15.18:0/3730774887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 msgr2=0x56398bb0c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 --2- 172.21.15.18:0/3730774887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb0c010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fad58009c10 tx=0x7fad58011810 comp rx=0 tx=0).stop 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 -- 172.21.15.18:0/3730774887 shutdown_connections 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 --2- 172.21.15.18:0/3730774887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb0c010 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 --2- 172.21.15.18:0/3730774887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf86d0 0x56398bb096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 --2- 172.21.15.18:0/3730774887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398baf8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 -- 172.21.15.18:0/3730774887 >> 172.21.15.18:0/3730774887 conn(0x56398baf2640 msgr2=0x56398baf2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 -- 172.21.15.18:0/3730774887 shutdown_connections 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.931+0000 7fad77bfa740 1 -- 172.21.15.18:0/3730774887 wait complete. 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.932+0000 7fad77bfa740 1 -- start start 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.932+0000 7fad77bfa740 1 Processor -- start 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.932+0000 7fad66a00640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fad540a3cb0 0x7fad540a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.932+0000 7fad66a00640 1 -- 172.21.15.18:0/883415422 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fad600d0920 con 0x7fad70104280 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.932+0000 7fad6e800640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fad540a3cb0 0x7fad540a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf7d20 0x56398bb06d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb07270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad77bfa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56398bb0c9e0 con 0x56398baf86d0 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad77bfa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56398ba16830 con 0x56398bb09c20 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad77bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56398bb0cb60 con 0x56398baf7d20 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad77bfa740 10 client.0 _ll_register_callbacks cb 0x56398ba1e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad6e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad6e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44778/0 (socket says 172.21.15.18:44778) 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad6e800640 1 -- 172.21.15.18:0/2661884208 learned_addr learned my addr 172.21.15.18:0/2661884208 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad67e00640 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb07270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.933+0000 7fad6de00640 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf7d20 0x56398bb06d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.934+0000 7fad6de00640 1 -- 172.21.15.18:0/2661884208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 msgr2=0x56398bb077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.934+0000 7fad6de00640 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb077b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.934+0000 7fad6de00640 1 -- 172.21.15.18:0/2661884208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 msgr2=0x56398bb07270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.934+0000 7fad6de00640 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb07270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.934+0000 7fad6de00640 1 -- 172.21.15.18:0/2661884208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56398ba1a920 con 0x56398baf7d20 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.934+0000 7fad67e00640 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.934+0000 7fad6e800640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fad540a3cb0 0x7fad540a6170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fad68005f00 tx=0x7fad68033bd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad6de00640 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf7d20 0x56398bb06d30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fad600cdc00 tx=0x7fad6009cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a30c4208677bc99 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad53e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad53e00640 1 -- 172.21.15.18:0/2661884208 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60020040 con 0x56398baf7d20 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad53e00640 1 -- 172.21.15.18:0/2661884208 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fad600b56c0 con 0x56398baf7d20 2024-04-16T17:13:52.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad77bfa740 1 -- 172.21.15.18:0/2661884208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad580043d0 con 0x56398baf7d20 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad53e00640 1 -- 172.21.15.18:0/2661884208 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad600a0800 con 0x56398baf7d20 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad77bfa740 10 client.27940 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad77bfa740 1 -- 172.21.15.18:0/2661884208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56398b9e87f0 con 0x56398baf7d20 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.935+0000 7fad53e00640 1 -- 172.21.15.18:0/2661884208 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad600a50c0 con 0x56398baf7d20 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.936+0000 7fad53e00640 1 -- 172.21.15.18:0/2661884208 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fad600b5bd0 con 0x56398baf7d20 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.936+0000 7fad77bfa740 10 client.27940 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.936+0000 7fad77bfa740 10 client.27940 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.936+0000 7fad77bfa740 20 client.27940 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.936+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56398bb1a0a0 0x56398bb1c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.936+0000 7fad77bfa740 4 client.27940 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.936+0000 7fad77bfa740 1 -- 172.21.15.18:0/2661884208 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fad580043d0 con 0x56398bb1a0a0 2024-04-16T17:13:52.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.936+0000 7fad67e00640 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56398bb1a0a0 0x56398bb1c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.937+0000 7fad67e00640 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56398bb1a0a0 0x56398bb1c4b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d457b30e423959 server_cookie=959442ea1b1b7a9d in_seq=0 out_seq=0 2024-04-16T17:13:52.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.937+0000 7fad53e00640 10 client.27940 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad53e00640 1 -- 172.21.15.18:0/2661884208 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fad580043d0 con 0x56398bb1a0a0 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad53e00640 10 client.27940 handle_command_reply: tid=0 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 client.27940 shutdown 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 -- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf7d20 msgr2=0x56398bb06d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf7d20 0x56398bb06d30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fad600cdc00 tx=0x7fad6009cb50 comp rx=0 tx=0).stop 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 -- 172.21.15.18:0/2661884208 shutdown_connections 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56398bb1a0a0 0x56398bb1c4b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad53e00640 0 client.27940 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398bb09c20 0x56398bb077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2661884208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398baf7d20 0x56398bb06d30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 -- 172.21.15.18:0/2661884208 >> 172.21.15.18:0/2661884208 conn(0x56398baf2640 msgr2=0x56398baf4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 -- 172.21.15.18:0/2661884208 shutdown_connections 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.938+0000 7fad77bfa740 1 -- 172.21.15.18:0/2661884208 wait complete. 2024-04-16T17:13:52.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.939+0000 7fad77bfa740 20 client.27940 trim_cache size 0 max 16384 2024-04-16T17:13:52.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.955+0000 7fad77bfa740 1 Processor -- start 2024-04-16T17:13:52.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.955+0000 7fad77bfa740 1 -- start start 2024-04-16T17:13:52.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf7d20 0x56398bb09460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb15910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 0x56398bb21da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad77bfa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fad701afb20 con 0x56398baf86d0 2024-04-16T17:13:52.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad77bfa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad701ad3f0 con 0x56398baf7d20 2024-04-16T17:13:52.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad77bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad701ab260 con 0x56398bb09c20 2024-04-16T17:13:52.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf7d20 0x56398bb09460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf7d20 0x56398bb09460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44786/0 (socket says 172.21.15.18:44786) 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6de00640 1 -- 172.21.15.18:0/2358106518 learned_addr learned my addr 172.21.15.18:0/2358106518 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad67e00640 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb15910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6e800640 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 0x56398bb21da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6de00640 1 -- 172.21.15.18:0/2358106518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 msgr2=0x56398bb21da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6de00640 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 0x56398bb21da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6de00640 1 -- 172.21.15.18:0/2358106518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 msgr2=0x56398bb15910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6de00640 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb15910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad6de00640 1 -- 172.21.15.18:0/2358106518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56398b76f3f0 con 0x56398baf7d20 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.956+0000 7fad67e00640 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb15910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.957+0000 7fad6de00640 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf7d20 0x56398bb09460 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fad600988b0 tx=0x7fad600b5f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8288ac45abd2255 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.957+0000 7fad5ea00640 1 -- 172.21.15.18:0/2358106518 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60014020 con 0x56398baf7d20 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.957+0000 7fad5ea00640 1 -- 172.21.15.18:0/2358106518 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad600b5bd0 con 0x56398baf7d20 2024-04-16T17:13:52.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.957+0000 7fad5ea00640 1 -- 172.21.15.18:0/2358106518 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60020070 con 0x56398baf7d20 2024-04-16T17:13:52.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 -- 172.21.15.18:0/2358106518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf7d20 msgr2=0x56398bb09460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf7d20 0x56398bb09460 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fad600988b0 tx=0x7fad600b5f90 comp rx=0 tx=0).stop 2024-04-16T17:13:52.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 -- 172.21.15.18:0/2358106518 shutdown_connections 2024-04-16T17:13:52.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 0x56398bb21da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf86d0 0x56398bb15910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2358106518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf7d20 0x56398bb09460 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 -- 172.21.15.18:0/2358106518 >> 172.21.15.18:0/2358106518 conn(0x56398baf2640 msgr2=0x56398baf4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 -- 172.21.15.18:0/2358106518 shutdown_connections 2024-04-16T17:13:52.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.958+0000 7fad77bfa740 1 -- 172.21.15.18:0/2358106518 wait complete. 2024-04-16T17:13:52.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.959+0000 7fad77bfa740 1 -- start start 2024-04-16T17:13:52.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.959+0000 7fad77bfa740 1 Processor -- start 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398bb12010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf86d0 0x56398bb03580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad77bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 0x56398bb03ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad77bfa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fad701ab8b0 con 0x56398baf7d20 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad77bfa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad701acd20 con 0x56398baf86d0 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad77bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad701abf80 con 0x56398bb09c20 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad77bfa740 10 client.0 _ll_register_callbacks cb 0x7fad6009bf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad6de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398bb12010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad6de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398bb12010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52824/0 (socket says 172.21.15.18:52824) 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad6de00640 1 -- 172.21.15.18:0/2082024252 learned_addr learned my addr 172.21.15.18:0/2082024252 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad6e800640 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 0x56398bb03ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.960+0000 7fad67e00640 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf86d0 0x56398bb03580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad6de00640 1 -- 172.21.15.18:0/2082024252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 msgr2=0x56398bb03ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad6de00640 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 0x56398bb03ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad6de00640 1 -- 172.21.15.18:0/2082024252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf86d0 msgr2=0x56398bb03580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad6de00640 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf86d0 0x56398bb03580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad6de00640 1 -- 172.21.15.18:0/2082024252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56398b8397d0 con 0x56398baf7d20 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad53e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad6de00640 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398bb12010 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fad6009c3e0 tx=0x7fad6009c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5c9be6ec900955 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad53e00640 1 -- 172.21.15.18:0/2082024252 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60014030 con 0x56398baf7d20 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad53e00640 1 -- 172.21.15.18:0/2082024252 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fad6009b8a0 con 0x56398baf7d20 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad77bfa740 1 -- 172.21.15.18:0/2082024252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad600b5bd0 con 0x56398baf7d20 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.961+0000 7fad53e00640 1 -- 172.21.15.18:0/2082024252 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad60084050 con 0x56398baf7d20 2024-04-16T17:13:52.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad77bfa740 10 client.17763 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:52.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad77bfa740 1 -- 172.21.15.18:0/2082024252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56398b8397d0 con 0x56398baf7d20 2024-04-16T17:13:52.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad53e00640 1 -- 172.21.15.18:0/2082024252 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad600b5bd0 con 0x56398baf7d20 2024-04-16T17:13:52.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad53e00640 1 -- 172.21.15.18:0/2082024252 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fad6009b430 con 0x56398baf7d20 2024-04-16T17:13:52.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad77bfa740 10 client.17763 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:52.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad77bfa740 10 client.17763 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad77bfa740 20 client.17763 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56398bb1a0a0 0x56398bb824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad77bfa740 4 client.17763 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad77bfa740 1 -- 172.21.15.18:0/2082024252 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56398ba6aa40 con 0x56398bb1a0a0 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.962+0000 7fad67e00640 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56398bb1a0a0 0x56398bb824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad67e00640 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56398bb1a0a0 0x56398bb824f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59d22ca8839f572 server_cookie=70a378c4362c6f in_seq=0 out_seq=0 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad53e00640 10 client.17763 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad53e00640 1 -- 172.21.15.18:0/2082024252 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56398ba6aa40 con 0x56398bb1a0a0 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad53e00640 10 client.17763 handle_command_reply: tid=0 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 client.17763 shutdown 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 -- 172.21.15.18:0/2082024252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 msgr2=0x56398bb12010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398bb12010 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fad6009c3e0 tx=0x7fad6009c040 comp rx=0 tx=0).stop 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 -- 172.21.15.18:0/2082024252 shutdown_connections 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56398bb1a0a0 0x56398bb824f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56398bb09c20 0x56398bb03ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad53e00640 0 client.17763 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:13:52.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56398baf86d0 0x56398bb03580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 --2- 172.21.15.18:0/2082024252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56398baf7d20 0x56398bb12010 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.963+0000 7fad77bfa740 1 -- 172.21.15.18:0/2082024252 >> 172.21.15.18:0/2082024252 conn(0x56398baf2640 msgr2=0x56398baf3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.964+0000 7fad77bfa740 1 -- 172.21.15.18:0/2082024252 shutdown_connections 2024-04-16T17:13:52.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.964+0000 7fad77bfa740 1 -- 172.21.15.18:0/2082024252 wait complete. 2024-04-16T17:13:52.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.964+0000 7fad77bfa740 20 client.17763 trim_cache size 0 max 16384 2024-04-16T17:13:52.967 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.964+0000 7fad6f200640 1 -- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fad540a3cb0 msgr2=0x7fad540a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.964+0000 7fad6f200640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fad540a3cb0 0x7fad540a6170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fad68005f00 tx=0x7fad68033bd0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.964+0000 7fad6f200640 1 -- 172.21.15.18:0/883415422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad70104280 msgr2=0x7fad701a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.964+0000 7fad6f200640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad70104280 0x7fad701a0620 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fad6000d950 tx=0x7fad6000dd50 comp rx=0 tx=0).stop 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.965+0000 7fad6f200640 1 -- 172.21.15.18:0/883415422 shutdown_connections 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.965+0000 7fad6f200640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fad540a3cb0 0x7fad540a6170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.965+0000 7fad6f200640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad7019a6e0 0x7fad7019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.965+0000 7fad6f200640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad70109350 0x7fad701a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.965+0000 7fad6f200640 1 --2- 172.21.15.18:0/883415422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad70104280 0x7fad701a0620 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:52.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.965+0000 7fad6f200640 1 -- 172.21.15.18:0/883415422 >> 172.21.15.18:0/883415422 conn(0x7fad700ffae0 msgr2=0x7fad70100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:52.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.965+0000 7fad6f200640 1 -- 172.21.15.18:0/883415422 shutdown_connections 2024-04-16T17:13:52.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:52.965+0000 7fad6f200640 1 -- 172.21.15.18:0/883415422 wait complete. 2024-04-16T17:13:52.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:13:53.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.201+0000 7f027be00640 1 -- 172.21.15.18:0/2475400244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c104280 msgr2=0x7f027c104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.201+0000 7f027be00640 1 --2- 172.21.15.18:0/2475400244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c104280 0x7f027c104660 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f026c009ca0 tx=0x7f026c02c100 comp rx=0 tx=0).stop 2024-04-16T17:13:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.202+0000 7f027be00640 1 -- 172.21.15.18:0/2475400244 shutdown_connections 2024-04-16T17:13:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.202+0000 7f027be00640 1 --2- 172.21.15.18:0/2475400244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f027c109350 0x7f027c110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.202+0000 7f027be00640 1 --2- 172.21.15.18:0/2475400244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f027c104c30 0x7f027c108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.202+0000 7f027be00640 1 --2- 172.21.15.18:0/2475400244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c104280 0x7f027c104660 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.202+0000 7f027be00640 1 -- 172.21.15.18:0/2475400244 >> 172.21.15.18:0/2475400244 conn(0x7f027c0ffae0 msgr2=0x7f027c101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.202+0000 7f027be00640 1 -- 172.21.15.18:0/2475400244 shutdown_connections 2024-04-16T17:13:53.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.202+0000 7f027be00640 1 -- 172.21.15.18:0/2475400244 wait complete. 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027be00640 1 Processor -- start 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027be00640 1 -- start start 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f027c104280 0x7f027c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f027c109350 0x7f027c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c19a760 0x7f027c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1143e0 con 0x7f027c19a760 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f027c114260 con 0x7f027c109350 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f027c114560 con 0x7f027c104280 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f027c104280 0x7f027c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f0272000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f027c109350 0x7f027c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f027c104280 0x7f027c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42278/0 (socket says 172.21.15.18:42278) 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.203+0000 7f027aa00640 1 -- 172.21.15.18:0/3570122631 learned_addr learned my addr 172.21.15.18:0/3570122631 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f027b400640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c19a760 0x7f027c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f0272000640 1 -- 172.21.15.18:0/3570122631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f027c104280 msgr2=0x7f027c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f0272000640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f027c104280 0x7f027c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f0272000640 1 -- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c19a760 msgr2=0x7f027c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f0272000640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c19a760 0x7f027c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f0272000640 1 -- 172.21.15.18:0/3570122631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c19b360 con 0x7f027c109350 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f027b400640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c19a760 0x7f027c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f0272000640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f027c109350 0x7f027c1a0be0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f026400e9c0 tx=0x7f026400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.204+0000 7f0280a00640 1 -- 172.21.15.18:0/3570122631 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026400ccc0 con 0x7f027c109350 2024-04-16T17:13:53.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.205+0000 7f027be00640 1 -- 172.21.15.18:0/3570122631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c1a7440 con 0x7f027c109350 2024-04-16T17:13:53.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.205+0000 7f0280a00640 1 -- 172.21.15.18:0/3570122631 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0264010910 con 0x7f027c109350 2024-04-16T17:13:53.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.205+0000 7f0280a00640 1 -- 172.21.15.18:0/3570122631 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264018880 con 0x7f027c109350 2024-04-16T17:13:53.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.205+0000 7f027be00640 1 -- 172.21.15.18:0/3570122631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c1a78a0 con 0x7f027c109350 2024-04-16T17:13:53.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.208+0000 7f0280a00640 1 -- 172.21.15.18:0/3570122631 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0264010ac0 con 0x7f027c109350 2024-04-16T17:13:53.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.208+0000 7f0284a06740 1 Processor -- start 2024-04-16T17:13:53.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.208+0000 7f0284a06740 1 -- start start 2024-04-16T17:13:53.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f0284a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817013f10 0x5598170142f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f0284a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f0284a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x55981701f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f0284a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559816f79020 con 0x55981701d410 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f0284a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559816f1f460 con 0x559817013f10 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f0284a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559816f74fe0 con 0x559817014830 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f027b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817013f10 0x5598170142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f027b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817013f10 0x5598170142f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44830/0 (socket says 172.21.15.18:44830) 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f027b400640 1 -- 172.21.15.18:0/3125313189 learned_addr learned my addr 172.21.15.18:0/3125313189 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f027aa00640 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.209+0000 7f0272000640 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x55981701f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f027b400640 1 -- 172.21.15.18:0/3125313189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 msgr2=0x55981701ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f027b400640 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f027b400640 1 -- 172.21.15.18:0/3125313189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 msgr2=0x55981701f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f027b400640 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x55981701f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f027b400640 1 -- 172.21.15.18:0/3125313189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559816f27040 con 0x559817013f10 2024-04-16T17:13:53.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f0272000640 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x55981701f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:53.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f0280a00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f02600a3d20 0x7f02600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f0280a00640 1 -- 172.21.15.18:0/3570122631 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f02640d0ff0 con 0x7f027c109350 2024-04-16T17:13:53.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f027aa00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f02600a3d20 0x7f02600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.210+0000 7f027b400640 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817013f10 0x5598170142f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0274009c10 tx=0x7f02740117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ec0a02875dfeb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.211+0000 7f026be00640 1 -- 172.21.15.18:0/3125313189 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274012450 con 0x559817013f10 2024-04-16T17:13:53.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.211+0000 7f026be00640 1 -- 172.21.15.18:0/3125313189 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0274012ad0 con 0x559817013f10 2024-04-16T17:13:53.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.211+0000 7f026be00640 1 -- 172.21.15.18:0/3125313189 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027400fc10 con 0x559817013f10 2024-04-16T17:13:53.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.211+0000 7f0284a06740 1 -- 172.21.15.18:0/3125313189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817013f10 msgr2=0x5598170142f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.211+0000 7f0284a06740 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817013f10 0x5598170142f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0274009c10 tx=0x7f02740117b0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.211+0000 7f027aa00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f02600a3d20 0x7f02600a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5598170139e0 tx=0x7f026c005d80 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 -- 172.21.15.18:0/3125313189 shutdown_connections 2024-04-16T17:13:53.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x55981701f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 --2- 172.21.15.18:0/3125313189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817013f10 0x5598170142f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 -- 172.21.15.18:0/3125313189 >> 172.21.15.18:0/3125313189 conn(0x55981700e640 msgr2=0x55981700ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 -- 172.21.15.18:0/3125313189 shutdown_connections 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 -- 172.21.15.18:0/3125313189 wait complete. 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 -- start start 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.212+0000 7f0284a06740 1 Processor -- start 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0284a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0284a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0284a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981701af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0284a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559817020200 con 0x559817013f10 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0284a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559816f32830 con 0x55981701d410 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0284a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559817020380 con 0x559817014830 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0284a06740 10 client.0 _ll_register_callbacks cb 0x559816f3a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0272000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f027b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981701af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0272000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42288/0 (socket says 172.21.15.18:42288) 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.213+0000 7f0272000640 1 -- 172.21.15.18:0/1194553389 learned_addr learned my addr 172.21.15.18:0/1194553389 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0272000640 1 -- 172.21.15.18:0/1194553389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 msgr2=0x55981701af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0272000640 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981701af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0272000640 1 -- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 msgr2=0x55981701a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f027aa00640 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0272000640 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0272000640 1 -- 172.21.15.18:0/1194553389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559816f36920 con 0x559817014830 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f027aa00640 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f027b400640 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981701af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0272000640 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701aa50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f026409cb10 tx=0x7f026409ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7195fb949540f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0268c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0268c00640 1 -- 172.21.15.18:0/1194553389 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026409cb80 con 0x559817014830 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0268c00640 1 -- 172.21.15.18:0/1194553389 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f02640d30e0 con 0x559817014830 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.214+0000 7f0284a06740 1 -- 172.21.15.18:0/1194553389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0274012ad0 con 0x559817014830 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.215+0000 7f0268c00640 1 -- 172.21.15.18:0/1194553389 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02640a0f30 con 0x559817014830 2024-04-16T17:13:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.215+0000 7f0284a06740 10 client.27958 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.215+0000 7f0284a06740 1 -- 172.21.15.18:0/1194553389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559816f047f0 con 0x559817014830 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.215+0000 7f0268c00640 1 -- 172.21.15.18:0/1194553389 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0274012ad0 con 0x559817014830 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.215+0000 7f0268c00640 1 -- 172.21.15.18:0/1194553389 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f02640988d0 con 0x559817014830 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.215+0000 7f0284a06740 10 client.27958 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.215+0000 7f0284a06740 10 client.27958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.215+0000 7f0284a06740 20 client.27958 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.216+0000 7f0284a06740 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55981702d8c0 0x55981702fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.216+0000 7f0284a06740 4 client.27958 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.216+0000 7f0284a06740 1 -- 172.21.15.18:0/1194553389 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559816f86a40 con 0x55981702d8c0 2024-04-16T17:13:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.216+0000 7f027aa00640 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55981702d8c0 0x55981702fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.216+0000 7f027aa00640 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55981702d8c0 0x55981702fcb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50bb2fec9fa33945 server_cookie=e811b08422738d7 in_seq=0 out_seq=0 2024-04-16T17:13:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.216+0000 7f0268c00640 10 client.27958 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:13:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.217+0000 7f0268c00640 1 -- 172.21.15.18:0/1194553389 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559816f86a40 con 0x55981702d8c0 2024-04-16T17:13:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.217+0000 7f0268c00640 10 client.27958 handle_command_reply: tid=0 2024-04-16T17:13:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.217+0000 7f0284a06740 1 client.27958 shutdown 2024-04-16T17:13:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 -- 172.21.15.18:0/1194553389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 msgr2=0x55981701aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701aa50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f026409cb10 tx=0x7f026409ca10 comp rx=0 tx=0).stop 2024-04-16T17:13:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 -- 172.21.15.18:0/1194553389 shutdown_connections 2024-04-16T17:13:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55981702d8c0 0x55981702fcb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981701af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x55981701aa50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0268c00640 0 client.27958 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:13:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 --2- 172.21.15.18:0/1194553389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 -- 172.21.15.18:0/1194553389 >> 172.21.15.18:0/1194553389 conn(0x55981700e640 msgr2=0x559817010390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 -- 172.21.15.18:0/1194553389 shutdown_connections 2024-04-16T17:13:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 1 -- 172.21.15.18:0/1194553389 wait complete. 2024-04-16T17:13:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.218+0000 7f0284a06740 20 client.27958 trim_cache size 0 max 16384 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.233+0000 7f0284a06740 1 Processor -- start 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.233+0000 7f0284a06740 1 -- start start 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.233+0000 7f0284a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.233+0000 7f0284a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x5598170290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.233+0000 7f0284a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981700d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.233+0000 7f0284a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1b4000 con 0x559817013f10 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.233+0000 7f0284a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1b18d0 con 0x55981701d410 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.233+0000 7f0284a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1af740 con 0x559817014830 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981700d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52880/0 (socket says 172.21.15.18:52880) 2024-04-16T17:13:53.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027aa00640 1 -- 172.21.15.18:0/3905114492 learned_addr learned my addr 172.21.15.18:0/3905114492 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027b400640 1 -- 172.21.15.18:0/3905114492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 msgr2=0x5598170290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027b400640 1 --2- 172.21.15.18:0/3905114492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x5598170290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027b400640 1 -- 172.21.15.18:0/3905114492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 msgr2=0x55981701cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027b400640 1 --2- 172.21.15.18:0/3905114492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027b400640 1 -- 172.21.15.18:0/3905114492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559817014660 con 0x55981701d410 2024-04-16T17:13:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.234+0000 7f027b400640 1 --2- 172.21.15.18:0/3905114492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981700d580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f027401c040 tx=0x7f027400f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e26e1dcecd190396 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.235+0000 7f026b400640 1 -- 172.21.15.18:0/3905114492 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027400f560 con 0x55981701d410 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.235+0000 7f026b400640 1 -- 172.21.15.18:0/3905114492 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0274009470 con 0x55981701d410 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.235+0000 7f026b400640 1 -- 172.21.15.18:0/3905114492 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740052a0 con 0x55981701d410 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.235+0000 7f0284a06740 1 -- 172.21.15.18:0/3905114492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 msgr2=0x55981700d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.235+0000 7f0284a06740 1 --2- 172.21.15.18:0/3905114492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981700d580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f027401c040 tx=0x7f027400f7b0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 -- 172.21.15.18:0/3905114492 shutdown_connections 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 --2- 172.21.15.18:0/3905114492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55981701d410 0x55981700d580 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 --2- 172.21.15.18:0/3905114492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817014830 0x5598170290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 --2- 172.21.15.18:0/3905114492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559817013f10 0x55981701cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 -- 172.21.15.18:0/3905114492 >> 172.21.15.18:0/3905114492 conn(0x55981700e640 msgr2=0x55981700ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 -- 172.21.15.18:0/3905114492 shutdown_connections 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 -- 172.21.15.18:0/3905114492 wait complete. 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 -- start start 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.236+0000 7f0284a06740 1 Processor -- start 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0284a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817013f10 0x559817020f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0284a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817014830 0x559817021450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0284a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x559817021990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0284a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1afd90 con 0x55981701d410 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0284a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1b1200 con 0x559817014830 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0284a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1b0460 con 0x559817013f10 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0284a06740 10 client.0 _ll_register_callbacks cb 0x7f0264098e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:53.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0272000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817014830 0x559817021450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0272000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817014830 0x559817021450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44872/0 (socket says 172.21.15.18:44872) 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0272000640 1 -- 172.21.15.18:0/4061559302 learned_addr learned my addr 172.21.15.18:0/4061559302 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f027b400640 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x559817021990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f027aa00640 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817013f10 0x559817020f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0272000640 1 -- 172.21.15.18:0/4061559302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817013f10 msgr2=0x559817020f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0272000640 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817013f10 0x559817020f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0272000640 1 -- 172.21.15.18:0/4061559302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 msgr2=0x559817021990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0272000640 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x559817021990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.237+0000 7f0272000640 1 -- 172.21.15.18:0/4061559302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559816c8b3f0 con 0x559817014830 2024-04-16T17:13:53.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.238+0000 7f0272000640 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817014830 0x559817021450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f026400c6a0 tx=0x7f0264098ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830baf465c13ce32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.238+0000 7f0268c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.238+0000 7f0268c00640 1 -- 172.21.15.18:0/4061559302 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026409cb80 con 0x559817014830 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.238+0000 7f0268c00640 1 -- 172.21.15.18:0/4061559302 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f026409c1d0 con 0x559817014830 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.238+0000 7f0284a06740 1 -- 172.21.15.18:0/4061559302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0274009470 con 0x559817014830 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.238+0000 7f0268c00640 1 -- 172.21.15.18:0/4061559302 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0264098ae0 con 0x559817014830 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.238+0000 7f0284a06740 10 client.28055 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.238+0000 7f0284a06740 1 -- 172.21.15.18:0/4061559302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559816d557d0 con 0x559817014830 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f0268c00640 1 -- 172.21.15.18:0/4061559302 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0274009470 con 0x559817014830 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f0268c00640 1 -- 172.21.15.18:0/4061559302 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f02640a20c0 con 0x559817014830 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f0284a06740 10 client.28055 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f0284a06740 10 client.28055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:13:53.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f0284a06740 20 client.28055 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:53.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f0284a06740 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55981709cc20 0x55981709f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f0284a06740 4 client.28055 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:53.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f0284a06740 1 -- 172.21.15.18:0/4061559302 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559816f808b0 con 0x55981709cc20 2024-04-16T17:13:53.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.239+0000 7f027aa00640 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55981709cc20 0x55981709f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.240+0000 7f0268c00640 10 client.28055 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:13:53.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.240+0000 7f027aa00640 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55981709cc20 0x55981709f040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d7732d1c376e90d server_cookie=17a2c4a51fada097 in_seq=0 out_seq=0 2024-04-16T17:13:53.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.240+0000 7f0268c00640 1 -- 172.21.15.18:0/4061559302 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559816f808b0 con 0x55981709cc20 2024-04-16T17:13:53.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.240+0000 7f0268c00640 10 client.28055 handle_command_reply: tid=0 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.240+0000 7f0284a06740 1 client.28055 shutdown 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 -- 172.21.15.18:0/4061559302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817014830 msgr2=0x559817021450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817014830 0x559817021450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f026400c6a0 tx=0x7f0264098ef0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 -- 172.21.15.18:0/4061559302 shutdown_connections 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55981709cc20 0x55981709f040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981701d410 0x559817021990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0268c00640 0 client.28055 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559817014830 0x559817021450 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 --2- 172.21.15.18:0/4061559302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559817013f10 0x559817020f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 -- 172.21.15.18:0/4061559302 >> 172.21.15.18:0/4061559302 conn(0x55981700e640 msgr2=0x55981700d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 -- 172.21.15.18:0/4061559302 shutdown_connections 2024-04-16T17:13:53.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 1 -- 172.21.15.18:0/4061559302 wait complete. 2024-04-16T17:13:53.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.241+0000 7f0284a06740 20 client.28055 trim_cache size 0 max 16384 2024-04-16T17:13:53.247 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:13:53.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 -- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f02600a3d20 msgr2=0x7f02600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f02600a3d20 0x7f02600a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5598170139e0 tx=0x7f026c005d80 comp rx=0 tx=0).stop 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 -- 172.21.15.18:0/3570122631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f027c109350 msgr2=0x7f027c1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f027c109350 0x7f027c1a0be0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f026400e9c0 tx=0x7f026400ee80 comp rx=0 tx=0).stop 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 -- 172.21.15.18:0/3570122631 shutdown_connections 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f02600a3d20 0x7f02600a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f027c19a760 0x7f027c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f027c109350 0x7f027c1a0be0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 --2- 172.21.15.18:0/3570122631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f027c104280 0x7f027c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 -- 172.21.15.18:0/3570122631 >> 172.21.15.18:0/3570122631 conn(0x7f027c0ffae0 msgr2=0x7f027c100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.242+0000 7f027be00640 1 -- 172.21.15.18:0/3570122631 shutdown_connections 2024-04-16T17:13:53.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.243+0000 7f027be00640 1 -- 172.21.15.18:0/3570122631 wait complete. 2024-04-16T17:13:53.262 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 -- 172.21.15.18:0/1830425692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c410e770 msgr2=0x7f50c4110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 --2- 172.21.15.18:0/1830425692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c410e770 0x7f50c4110b60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009ca0 tx=0x7f50bc02c0f0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 -- 172.21.15.18:0/1830425692 shutdown_connections 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 --2- 172.21.15.18:0/1830425692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c410e770 0x7f50c4110b60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 --2- 172.21.15.18:0/1830425692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4101570 0x7f50c410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 --2- 172.21.15.18:0/1830425692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c4100bc0 0x7f50c4100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 -- 172.21.15.18:0/1830425692 >> 172.21.15.18:0/1830425692 conn(0x7f50c4077ab0 msgr2=0x7f50c40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 -- 172.21.15.18:0/1830425692 shutdown_connections 2024-04-16T17:13:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.464+0000 7f50c9600640 1 -- 172.21.15.18:0/1830425692 wait complete. 2024-04-16T17:13:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.465+0000 7f50c9600640 1 Processor -- start 2024-04-16T17:13:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.465+0000 7f50c9600640 1 -- start start 2024-04-16T17:13:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.465+0000 7f50c9600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4100bc0 0x7f50c41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.465+0000 7f50c9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4101570 0x7f50c41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.465+0000 7f50c9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c410e770 0x7f50c419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.465+0000 7f50c9600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41143b0 con 0x7f50c4100bc0 2024-04-16T17:13:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.465+0000 7f50c9600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4114230 con 0x7f50c410e770 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.465+0000 7f50c9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4114530 con 0x7f50c4101570 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c410e770 0x7f50c419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4101570 0x7f50c41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c410e770 0x7f50c419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44890/0 (socket says 172.21.15.18:44890) 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4100bc0 0x7f50c41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c8c00640 1 -- 172.21.15.18:0/2580574060 learned_addr learned my addr 172.21.15.18:0/2580574060 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c8c00640 1 -- 172.21.15.18:0/2580574060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4101570 msgr2=0x7f50c41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c8c00640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4101570 0x7f50c41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c8c00640 1 -- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4100bc0 msgr2=0x7f50c41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c8c00640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4100bc0 0x7f50c41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c8c00640 1 -- 172.21.15.18:0/2580574060 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c419b010 con 0x7f50c410e770 2024-04-16T17:13:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.466+0000 7f50c3400640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4101570 0x7f50c41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.467+0000 7f50c8c00640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c410e770 0x7f50c419a750 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f50bc005cc0 tx=0x7f50bc032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.467+0000 7f50c0c00640 1 -- 172.21.15.18:0/2580574060 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc041020 con 0x7f50c410e770 2024-04-16T17:13:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.467+0000 7f50c0c00640 1 -- 172.21.15.18:0/2580574060 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc034cd0 con 0x7f50c410e770 2024-04-16T17:13:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.467+0000 7f50c9600640 1 -- 172.21.15.18:0/2580574060 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c419b2a0 con 0x7f50c410e770 2024-04-16T17:13:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.467+0000 7f50c0c00640 1 -- 172.21.15.18:0/2580574060 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc039040 con 0x7f50c410e770 2024-04-16T17:13:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.467+0000 7f50c9600640 1 -- 172.21.15.18:0/2580574060 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c41a7430 con 0x7f50c410e770 2024-04-16T17:13:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.470+0000 7f50c0c00640 1 -- 172.21.15.18:0/2580574060 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f50bc044020 con 0x7f50c410e770 2024-04-16T17:13:53.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.471+0000 7f50cd9d7740 1 Processor -- start 2024-04-16T17:13:53.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.471+0000 7f50cd9d7740 1 -- start start 2024-04-16T17:13:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.471+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dccb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.471+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.471+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.471+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55589dc30020 con 0x55589dccaf10 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.471+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55589dbd6460 con 0x55589dccb830 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.471+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55589dc2bfe0 con 0x55589dcd4410 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44892/0 (socket says 172.21.15.18:44892) 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3e00640 1 -- 172.21.15.18:0/3914303538 learned_addr learned my addr 172.21.15.18:0/3914303538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3400640 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dccb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3400640 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3e00640 1 -- 172.21.15.18:0/3914303538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 msgr2=0x55589dcd6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3e00640 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3e00640 1 -- 172.21.15.18:0/3914303538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 msgr2=0x55589dccb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3e00640 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dccb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3e00640 1 -- 172.21.15.18:0/3914303538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589dbde040 con 0x55589dccb830 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.472+0000 7f50c3400640 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.473+0000 7f50c0c00640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50a80a3d20 0x7f50a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.473+0000 7f50c3e00640 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd3ed0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009cf0 tx=0x7f50b4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209525c7bd72a6e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.473+0000 7f50c0c00640 1 -- 172.21.15.18:0/2580574060 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f50bc0f2830 con 0x7f50c410e770 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.473+0000 7f50b9200640 1 -- 172.21.15.18:0/3914303538 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4004230 con 0x55589dccb830 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.473+0000 7f50c3400640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50a80a3d20 0x7f50a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.473+0000 7f50b9200640 1 -- 172.21.15.18:0/3914303538 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b40043d0 con 0x55589dccb830 2024-04-16T17:13:53.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.473+0000 7f50b9200640 1 -- 172.21.15.18:0/3914303538 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4019960 con 0x55589dccb830 2024-04-16T17:13:53.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3914303538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 msgr2=0x55589dcd3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd3ed0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009cf0 tx=0x7f50b4011810 comp rx=0 tx=0).stop 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3914303538 shutdown_connections 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd3ed0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3914303538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dccb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3914303538 >> 172.21.15.18:0/3914303538 conn(0x55589dcc5640 msgr2=0x55589dcc5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50c3400640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50a80a3d20 0x7f50a80a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55589dcc67d0 tx=0x7f50b00096c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3914303538 shutdown_connections 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3914303538 wait complete. 2024-04-16T17:13:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 -- start start 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.474+0000 7f50cd9d7740 1 Processor -- start 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcd14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55589dcd7200 con 0x55589dccaf10 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55589dbe9830 con 0x55589dccb830 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55589dcd7380 con 0x55589dcd4410 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50cd9d7740 10 client.0 _ll_register_callbacks cb 0x55589dbf1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50c8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50c8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42342/0 (socket says 172.21.15.18:42342) 2024-04-16T17:13:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50c8c00640 1 -- 172.21.15.18:0/3721229516 learned_addr learned my addr 172.21.15.18:0/3721229516 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.475+0000 7f50c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44902/0 (socket says 172.21.15.18:44902) 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50c3e00640 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcd14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50c8c00640 1 -- 172.21.15.18:0/3721229516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 msgr2=0x55589dcd19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50c8c00640 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50c8c00640 1 -- 172.21.15.18:0/3721229516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 msgr2=0x55589dcd14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50c8c00640 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcd14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50c8c00640 1 -- 172.21.15.18:0/3721229516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589dbed920 con 0x55589dcd4410 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50c3400640 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50c8c00640 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd1f20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0c6bb0 tx=0x7f50bc0be070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16a1c1e15134470 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50ad600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50ad600640 1 -- 172.21.15.18:0/3721229516 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0be470 con 0x55589dcd4410 2024-04-16T17:13:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.476+0000 7f50ad600640 1 -- 172.21.15.18:0/3721229516 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f50bc0c6c50 con 0x55589dcd4410 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.477+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3721229516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b40043d0 con 0x55589dcd4410 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.477+0000 7f50ad600640 1 -- 172.21.15.18:0/3721229516 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50bc0c2820 con 0x55589dcd4410 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.477+0000 7f50cd9d7740 10 client.27970 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.477+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3721229516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55589dbbb7f0 con 0x55589dcd4410 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.477+0000 7f50ad600640 1 -- 172.21.15.18:0/3721229516 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b40043d0 con 0x55589dcd4410 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.477+0000 7f50ad600640 1 -- 172.21.15.18:0/3721229516 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f50bc03c030 con 0x55589dcd4410 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.478+0000 7f50cd9d7740 10 client.27970 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.478+0000 7f50cd9d7740 10 client.27970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.478+0000 7f50cd9d7740 20 client.27970 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.478+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55589dce46e0 0x55589dce6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.478+0000 7f50cd9d7740 4 client.27970 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:53.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.478+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3721229516 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55589dc3da40 con 0x55589dce46e0 2024-04-16T17:13:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.478+0000 7f50c3e00640 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55589dce46e0 0x55589dce6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.479+0000 7f50c3e00640 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55589dce46e0 0x55589dce6b40 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc88c2c2fb66e57 server_cookie=c023aad9cac0c9f in_seq=0 out_seq=0 2024-04-16T17:13:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.479+0000 7f50ad600640 10 client.27970 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.480+0000 7f50ad600640 1 -- 172.21.15.18:0/3721229516 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55589dc3da40 con 0x55589dce46e0 2024-04-16T17:13:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.480+0000 7f50ad600640 10 client.27970 handle_command_reply: tid=0 2024-04-16T17:13:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.480+0000 7f50cd9d7740 1 client.27970 shutdown 2024-04-16T17:13:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.480+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3721229516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 msgr2=0x55589dcd1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.480+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd1f20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0c6bb0 tx=0x7f50bc0be070 comp rx=0 tx=0).stop 2024-04-16T17:13:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.480+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3721229516 shutdown_connections 2024-04-16T17:13:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.480+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55589dce46e0 0x55589dce6b40 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.480+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcd1f20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.481+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcd19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.481+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/3721229516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcd14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.481+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3721229516 >> 172.21.15.18:0/3721229516 conn(0x55589dcc5640 msgr2=0x55589dcc7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.481+0000 7f50ad600640 0 client.27970 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:53.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.481+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3721229516 shutdown_connections 2024-04-16T17:13:53.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.481+0000 7f50cd9d7740 1 -- 172.21.15.18:0/3721229516 wait complete. 2024-04-16T17:13:53.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.481+0000 7f50cd9d7740 20 client.27970 trim_cache size 0 max 16384 2024-04-16T17:13:53.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.510+0000 7f50cd9d7740 1 Processor -- start 2024-04-16T17:13:53.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50cd9d7740 1 -- start start 2024-04-16T17:13:53.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcd3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dce00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcc7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41b3f80 con 0x55589dccaf10 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41b1850 con 0x55589dccb830 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41af6c0 con 0x55589dcd4410 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dce00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dce00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44904/0 (socket says 172.21.15.18:44904) 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50c3400640 1 -- 172.21.15.18:0/2286571282 learned_addr learned my addr 172.21.15.18:0/2286571282 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.511+0000 7f50c3e00640 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcd3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50c8c00640 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcc7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50c3400640 1 -- 172.21.15.18:0/2286571282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 msgr2=0x55589dcc7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50c3400640 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcc7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50c3400640 1 -- 172.21.15.18:0/2286571282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 msgr2=0x55589dcd3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50c3400640 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcd3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50c3400640 1 -- 172.21.15.18:0/2286571282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589dccb660 con 0x55589dccb830 2024-04-16T17:13:53.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50c8c00640 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcc7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50c3400640 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dce00c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f50c419bfa0 tx=0x7f50b0018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63301a364983de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.512+0000 7f50afe00640 1 -- 172.21.15.18:0/2286571282 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0003da0 con 0x55589dccb830 2024-04-16T17:13:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50afe00640 1 -- 172.21.15.18:0/2286571282 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b0012020 con 0x55589dccb830 2024-04-16T17:13:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50afe00640 1 -- 172.21.15.18:0/2286571282 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0005570 con 0x55589dccb830 2024-04-16T17:13:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50cd9d7740 1 -- 172.21.15.18:0/2286571282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 msgr2=0x55589dce00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dce00c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f50c419bfa0 tx=0x7f50b0018000 comp rx=0 tx=0).stop 2024-04-16T17:13:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50cd9d7740 1 -- 172.21.15.18:0/2286571282 shutdown_connections 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dcc7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dce00c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/2286571282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcd3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.513+0000 7f50cd9d7740 1 -- 172.21.15.18:0/2286571282 >> 172.21.15.18:0/2286571282 conn(0x55589dcc5640 msgr2=0x55589dcc6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.514+0000 7f50cd9d7740 1 -- 172.21.15.18:0/2286571282 shutdown_connections 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.514+0000 7f50cd9d7740 1 -- 172.21.15.18:0/2286571282 wait complete. 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.514+0000 7f50cd9d7740 1 -- start start 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.514+0000 7f50cd9d7740 1 Processor -- start 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcdc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcdcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50cd9d7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dccdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41afd10 con 0x55589dccaf10 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41b1180 con 0x55589dccb830 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50cd9d7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41b03e0 con 0x55589dcd4410 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50cd9d7740 10 client.0 _ll_register_callbacks cb 0x7f50bc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcdc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcdcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcdc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52948/0 (socket says 172.21.15.18:52948) 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.515+0000 7f50c3e00640 1 -- 172.21.15.18:0/1285055971 learned_addr learned my addr 172.21.15.18:0/1285055971 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:53.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50c8c00640 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dccdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50c3e00640 1 -- 172.21.15.18:0/1285055971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 msgr2=0x55589dccdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50c3e00640 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dccdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50c3e00640 1 -- 172.21.15.18:0/1285055971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 msgr2=0x55589dcdcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:53.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50c3e00640 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcdcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50c3e00640 1 -- 172.21.15.18:0/1285055971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589d9423f0 con 0x55589dccaf10 2024-04-16T17:13:53.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50c8c00640 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dccdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:53.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50c3e00640 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcdc5d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55589dcca9e0 tx=0x7f50b40194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e2317350b27c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:53.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.516+0000 7f50ad600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:53.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.520+0000 7f50ad600640 1 -- 172.21.15.18:0/1285055971 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b402bd60 con 0x55589dccaf10 2024-04-16T17:13:53.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.520+0000 7f50ad600640 1 -- 172.21.15.18:0/1285055971 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f50b4005040 con 0x55589dccaf10 2024-04-16T17:13:53.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.520+0000 7f50cd9d7740 1 -- 172.21.15.18:0/1285055971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b0012020 con 0x55589dccaf10 2024-04-16T17:13:53.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50ad600640 1 -- 172.21.15.18:0/1285055971 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50b403c050 con 0x55589dccaf10 2024-04-16T17:13:53.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50cd9d7740 10 client.17796 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:53.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50cd9d7740 1 -- 172.21.15.18:0/1285055971 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55589da0c7d0 con 0x55589dccaf10 2024-04-16T17:13:53.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50ad600640 1 -- 172.21.15.18:0/1285055971 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b0012020 con 0x55589dccaf10 2024-04-16T17:13:53.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50ad600640 1 -- 172.21.15.18:0/1285055971 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f50b40035f0 con 0x55589dccaf10 2024-04-16T17:13:53.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50cd9d7740 10 client.17796 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:53.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50cd9d7740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:53.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50cd9d7740 20 client.17796 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:53.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55589dd4ab60 0x55589dd4cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:53.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50cd9d7740 4 client.17796 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:53.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.521+0000 7f50cd9d7740 1 -- 172.21.15.18:0/1285055971 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55589dc378b0 con 0x55589dd4ab60 2024-04-16T17:13:53.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.522+0000 7f50c3400640 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55589dd4ab60 0x55589dd4cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:53.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.522+0000 7f50c3400640 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55589dd4ab60 0x55589dd4cf80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237f78867d865217 server_cookie=2ac8079939b62c68 in_seq=0 out_seq=0 2024-04-16T17:13:53.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.522+0000 7f50ad600640 10 client.17796 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:53.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.524+0000 7f50ad600640 1 -- 172.21.15.18:0/1285055971 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55589dc378b0 con 0x55589dd4ab60 2024-04-16T17:13:53.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.524+0000 7f50ad600640 10 client.17796 handle_command_reply: tid=0 2024-04-16T17:13:53.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.524+0000 7f50cd9d7740 1 client.17796 shutdown 2024-04-16T17:13:53.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 -- 172.21.15.18:0/1285055971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 msgr2=0x55589dcdc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcdc5d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55589dcca9e0 tx=0x7f50b40194c0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 -- 172.21.15.18:0/1285055971 shutdown_connections 2024-04-16T17:13:53.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55589dd4ab60 0x55589dd4cf80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589dcd4410 0x55589dccdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55589dccb830 0x55589dcdcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 --2- 172.21.15.18:0/1285055971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55589dccaf10 0x55589dcdc5d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 -- 172.21.15.18:0/1285055971 >> 172.21.15.18:0/1285055971 conn(0x55589dcc5640 msgr2=0x55589dcc7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50ad600640 0 client.17796 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:53.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 -- 172.21.15.18:0/1285055971 shutdown_connections 2024-04-16T17:13:53.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 1 -- 172.21.15.18:0/1285055971 wait complete. 2024-04-16T17:13:53.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.525+0000 7f50cd9d7740 20 client.17796 trim_cache size 0 max 16384 2024-04-16T17:13:53.527 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:13:53.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 -- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50a80a3d20 msgr2=0x7f50a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50a80a3d20 0x7f50a80a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55589dcc67d0 tx=0x7f50b00096c0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 -- 172.21.15.18:0/2580574060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c410e770 msgr2=0x7f50c419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:53.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c410e770 0x7f50c419a750 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f50bc005cc0 tx=0x7f50bc032f20 comp rx=0 tx=0).stop 2024-04-16T17:13:53.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 -- 172.21.15.18:0/2580574060 shutdown_connections 2024-04-16T17:13:53.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f50a80a3d20 0x7f50a80a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c410e770 0x7f50c419a750 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4101570 0x7f50c41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 --2- 172.21.15.18:0/2580574060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4100bc0 0x7f50c41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:53.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.526+0000 7f50c9600640 1 -- 172.21.15.18:0/2580574060 >> 172.21.15.18:0/2580574060 conn(0x7f50c4077ab0 msgr2=0x7f50c40ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:53.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.527+0000 7f50c9600640 1 -- 172.21.15.18:0/2580574060 shutdown_connections 2024-04-16T17:13:53.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:53.527+0000 7f50c9600640 1 -- 172.21.15.18:0/2580574060 wait complete. 2024-04-16T17:13:53.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 345492 3.72 MB/sec execute 366 sec latency 32.190 ms 2024-04-16T17:13:53.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:53 smithi022 ceph-mon[36023]: pgmap v285: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 636 KiB/s wr, 108 op/s 2024-04-16T17:13:53.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:53 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1508741135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:54.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:53 smithi043 ceph-mon[36901]: pgmap v285: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 636 KiB/s wr, 108 op/s 2024-04-16T17:13:54.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:53 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1508741135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:54.550 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:13:54.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:13:54.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:13:54.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 346208 3.71 MB/sec execute 367 sec latency 29.196 ms 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 -- 172.21.15.18:0/1308986743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0104280 msgr2=0x7ff3b0104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 --2- 172.21.15.18:0/1308986743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0104280 0x7ff3b0104660 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009900 tx=0x7ff3a002bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 -- 172.21.15.18:0/1308986743 shutdown_connections 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 --2- 172.21.15.18:0/1308986743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0109350 0x7ff3b0110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 --2- 172.21.15.18:0/1308986743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3b0104c30 0x7ff3b0108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 --2- 172.21.15.18:0/1308986743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0104280 0x7ff3b0104660 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 -- 172.21.15.18:0/1308986743 >> 172.21.15.18:0/1308986743 conn(0x7ff3b00ffae0 msgr2=0x7ff3b0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 -- 172.21.15.18:0/1308986743 shutdown_connections 2024-04-16T17:13:54.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.783+0000 7ff3af200640 1 -- 172.21.15.18:0/1308986743 wait complete. 2024-04-16T17:13:54.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.784+0000 7ff3af200640 1 Processor -- start 2024-04-16T17:13:54.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.784+0000 7ff3af200640 1 -- start start 2024-04-16T17:13:54.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.784+0000 7ff3af200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3b0104280 0x7ff3b01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.784+0000 7ff3af200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0109350 0x7ff3b01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.784+0000 7ff3af200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b019a780 0x7ff3b019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.784+0000 7ff3af200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0114460 con 0x7ff3b0104280 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.784+0000 7ff3af200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01142e0 con 0x7ff3b019a780 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.784+0000 7ff3af200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01145e0 con 0x7ff3b0109350 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0109350 0x7ff3b01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3b0104280 0x7ff3b01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0109350 0x7ff3b01a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42380/0 (socket says 172.21.15.18:42380) 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ad400640 1 -- 172.21.15.18:0/3015721809 learned_addr learned my addr 172.21.15.18:0/3015721809 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ae800640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b019a780 0x7ff3b019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ad400640 1 -- 172.21.15.18:0/3015721809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b019a780 msgr2=0x7ff3b019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ad400640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b019a780 0x7ff3b019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ad400640 1 -- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3b0104280 msgr2=0x7ff3b01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ad400640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3b0104280 0x7ff3b01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.785+0000 7ff3ad400640 1 -- 172.21.15.18:0/3015721809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b019b310 con 0x7ff3b0109350 2024-04-16T17:13:54.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.786+0000 7ff3ad400640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0109350 0x7ff3b01a0b50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00e9c0 tx=0x7ff39c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:54.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.786+0000 7ff3a6a00640 1 -- 172.21.15.18:0/3015721809 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00cd80 con 0x7ff3b0109350 2024-04-16T17:13:54.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.786+0000 7ff3a6a00640 1 -- 172.21.15.18:0/3015721809 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c005500 con 0x7ff3b0109350 2024-04-16T17:13:54.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.786+0000 7ff3a6a00640 1 -- 172.21.15.18:0/3015721809 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c010690 con 0x7ff3b0109350 2024-04-16T17:13:54.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.786+0000 7ff3af200640 1 -- 172.21.15.18:0/3015721809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b019b600 con 0x7ff3b0109350 2024-04-16T17:13:54.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.786+0000 7ff3af200640 1 -- 172.21.15.18:0/3015721809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b01a7830 con 0x7ff3b0109350 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.790+0000 7ff3b767b740 1 Processor -- start 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.790+0000 7ff3b767b740 1 -- start start 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.790+0000 7ff3a6a00640 1 -- 172.21.15.18:0/3015721809 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff39c0046c0 con 0x7ff3b0109350 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda44ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4507e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda450a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3b767b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cda4463cb0 con 0x55cda44fee90 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3b767b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cda440a370 con 0x55cda44ff7b0 2024-04-16T17:13:54.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3b767b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cda445fc70 con 0x55cda4508390 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4507e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3ae800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda44ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4507e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44928/0 (socket says 172.21.15.18:44928) 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3ade00640 1 -- 172.21.15.18:0/3298576892 learned_addr learned my addr 172.21.15.18:0/3298576892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.791+0000 7ff3ad400640 1 --2- 172.21.15.18:0/3298576892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda450a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff3ae800640 1 -- 172.21.15.18:0/3298576892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 msgr2=0x55cda450a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff3ae800640 1 --2- 172.21.15.18:0/3298576892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda450a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff3ae800640 1 -- 172.21.15.18:0/3298576892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 msgr2=0x55cda4507e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff3ae800640 1 --2- 172.21.15.18:0/3298576892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4507e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff3ae800640 1 -- 172.21.15.18:0/3298576892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cda4429910 con 0x55cda44fee90 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff3ae800640 1 --2- 172.21.15.18:0/3298576892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda44ff270 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009f00 tx=0x7ff3a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4da67da0aa5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:54.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff39b000640 1 -- 172.21.15.18:0/3298576892 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8004d20 con 0x55cda44fee90 2024-04-16T17:13:54.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff39b000640 1 -- 172.21.15.18:0/3298576892 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a8004050 con 0x55cda44fee90 2024-04-16T17:13:54.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.792+0000 7ff39b000640 1 -- 172.21.15.18:0/3298576892 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8005280 con 0x55cda44fee90 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3a6a00640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3940a3da0 0x7ff3940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3b767b740 1 -- 172.21.15.18:0/3298576892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 msgr2=0x55cda44ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3b767b740 1 --2- 172.21.15.18:0/3298576892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda44ff270 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009f00 tx=0x7ff3a8011810 comp rx=0 tx=0).stop 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3a6a00640 1 -- 172.21.15.18:0/3015721809 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7ff39c0d0f60 con 0x7ff3b0109350 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3ade00640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3940a3da0 0x7ff3940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3b767b740 1 -- 172.21.15.18:0/3298576892 shutdown_connections 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3b767b740 1 --2- 172.21.15.18:0/3298576892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda450a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3b767b740 1 --2- 172.21.15.18:0/3298576892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4507e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3b767b740 1 --2- 172.21.15.18:0/3298576892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda44ff270 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3b767b740 1 -- 172.21.15.18:0/3298576892 >> 172.21.15.18:0/3298576892 conn(0x55cda44f95c0 msgr2=0x55cda44f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:54.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.793+0000 7ff3b767b740 1 -- 172.21.15.18:0/3298576892 shutdown_connections 2024-04-16T17:13:54.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.794+0000 7ff3b767b740 1 -- 172.21.15.18:0/3298576892 wait complete. 2024-04-16T17:13:54.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.794+0000 7ff3b767b740 1 -- start start 2024-04-16T17:13:54.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.794+0000 7ff3ade00640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3940a3da0 0x7ff3940a6260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00096c0 tx=0x7ff3a002df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:54.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3b767b740 1 Processor -- start 2024-04-16T17:13:54.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda4505340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4505880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda4505dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3b767b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cda450b180 con 0x55cda44fee90 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3b767b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cda441d730 con 0x55cda44ff7b0 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3b767b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cda450b300 con 0x55cda4508390 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3b767b740 10 client.0 _ll_register_callbacks cb 0x55cda43ec1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3ae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda4505dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3ae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda4505dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42398/0 (socket says 172.21.15.18:42398) 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3ae800640 1 -- 172.21.15.18:0/1998601803 learned_addr learned my addr 172.21.15.18:0/1998601803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.795+0000 7ff3ad400640 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4505880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff3ade00640 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda4505340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff3ae800640 1 -- 172.21.15.18:0/1998601803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 msgr2=0x55cda4505880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff3ae800640 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4505880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff3ae800640 1 -- 172.21.15.18:0/1998601803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 msgr2=0x55cda4505340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff3ae800640 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda4505340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff3ae800640 1 -- 172.21.15.18:0/1998601803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cda423e7e0 con 0x55cda4508390 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff3ae800640 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda4505dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff3a801b040 tx=0x7ff3a80044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c653e680c337be91 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:54.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff38f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.796+0000 7ff38f400640 1 -- 172.21.15.18:0/1998601803 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8029020 con 0x55cda4508390 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.797+0000 7ff38f400640 1 -- 172.21.15.18:0/1998601803 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7ff3a8025470 con 0x55cda4508390 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.797+0000 7ff3b767b740 1 -- 172.21.15.18:0/1998601803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a8004050 con 0x55cda4508390 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.797+0000 7ff38f400640 1 -- 172.21.15.18:0/1998601803 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3a8003e40 con 0x55cda4508390 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.797+0000 7ff3b767b740 10 client.27988 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.797+0000 7ff3b767b740 1 -- 172.21.15.18:0/1998601803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cda429ef70 con 0x55cda4508390 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.797+0000 7ff38f400640 1 -- 172.21.15.18:0/1998601803 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a8004050 con 0x55cda4508390 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.797+0000 7ff38f400640 1 -- 172.21.15.18:0/1998601803 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff3a80199c0 con 0x55cda4508390 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.798+0000 7ff3b767b740 10 client.27988 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.798+0000 7ff3b767b740 10 client.27988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.798+0000 7ff3b767b740 20 client.27988 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.798+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cda4520f50 0x55cda4523340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.798+0000 7ff3b767b740 4 client.27988 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.798+0000 7ff3b767b740 1 -- 172.21.15.18:0/1998601803 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cda44716d0 con 0x55cda4520f50 2024-04-16T17:13:54.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.798+0000 7ff3ade00640 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cda4520f50 0x55cda4523340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.799+0000 7ff38f400640 10 client.27988 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.799+0000 7ff3ade00640 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cda4520f50 0x55cda4523340 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4782c2da9583729c server_cookie=781352fa990def52 in_seq=0 out_seq=0 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff38f400640 1 -- 172.21.15.18:0/1998601803 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cda44716d0 con 0x55cda4520f50 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff38f400640 10 client.27988 handle_command_reply: tid=0 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 client.27988 shutdown 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 -- 172.21.15.18:0/1998601803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 msgr2=0x55cda4505dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda4505dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff3a801b040 tx=0x7ff3a80044b0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 -- 172.21.15.18:0/1998601803 shutdown_connections 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cda4520f50 0x55cda4523340 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda4508390 0x55cda4505dc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44ff7b0 0x55cda4505880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1998601803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44fee90 0x55cda4505340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 -- 172.21.15.18:0/1998601803 >> 172.21.15.18:0/1998601803 conn(0x55cda44f95c0 msgr2=0x55cda44fb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:54.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff38f400640 0 client.27988 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:54.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 -- 172.21.15.18:0/1998601803 shutdown_connections 2024-04-16T17:13:54.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 1 -- 172.21.15.18:0/1998601803 wait complete. 2024-04-16T17:13:54.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.800+0000 7ff3b767b740 20 client.27988 trim_cache size 0 max 16384 2024-04-16T17:13:54.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3b767b740 1 Processor -- start 2024-04-16T17:13:54.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3b767b740 1 -- start start 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44fee90 0x55cda4507a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44ff7b0 0x55cda451c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda4508390 0x55cda4505a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3b767b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01afaa0 con 0x55cda4508390 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3b767b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01ad370 con 0x55cda44fee90 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3b767b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01ab1e0 con 0x55cda44ff7b0 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44fee90 0x55cda4507a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3ad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44ff7b0 0x55cda451c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.840+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44fee90 0x55cda4507a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44936/0 (socket says 172.21.15.18:44936) 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44ff7b0 0x55cda451c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42402/0 (socket says 172.21.15.18:42402) 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ade00640 1 -- 172.21.15.18:0/2208844483 learned_addr learned my addr 172.21.15.18:0/2208844483 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ae800640 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda4508390 0x55cda4505a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ade00640 1 -- 172.21.15.18:0/2208844483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44ff7b0 msgr2=0x55cda451c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ade00640 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44ff7b0 0x55cda451c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ade00640 1 -- 172.21.15.18:0/2208844483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda4508390 msgr2=0x55cda4505a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ade00640 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda4508390 0x55cda4505a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ade00640 1 -- 172.21.15.18:0/2208844483 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cda4136170 con 0x55cda44fee90 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ad400640 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44ff7b0 0x55cda451c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ae800640 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda4508390 0x55cda4505a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.841+0000 7ff3ade00640 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44fee90 0x55cda4507a70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff3a002c240 tx=0x7ff3a0042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76eb703c5a7f2937 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.842+0000 7ff39a600640 1 -- 172.21.15.18:0/2208844483 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0046070 con 0x55cda44fee90 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.842+0000 7ff39a600640 1 -- 172.21.15.18:0/2208844483 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a0004ba0 con 0x55cda44fee90 2024-04-16T17:13:54.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.842+0000 7ff39a600640 1 -- 172.21.15.18:0/2208844483 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0055530 con 0x55cda44fee90 2024-04-16T17:13:54.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.842+0000 7ff3b767b740 1 -- 172.21.15.18:0/2208844483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44fee90 msgr2=0x55cda4507a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.842+0000 7ff3b767b740 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44fee90 0x55cda4507a70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff3a002c240 tx=0x7ff3a0042000 comp rx=0 tx=0).stop 2024-04-16T17:13:54.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 -- 172.21.15.18:0/2208844483 shutdown_connections 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda4508390 0x55cda4505a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44ff7b0 0x55cda451c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 --2- 172.21.15.18:0/2208844483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda44fee90 0x55cda4507a70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 -- 172.21.15.18:0/2208844483 >> 172.21.15.18:0/2208844483 conn(0x55cda44f95c0 msgr2=0x55cda4505300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 -- 172.21.15.18:0/2208844483 shutdown_connections 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 -- 172.21.15.18:0/2208844483 wait complete. 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 -- start start 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.843+0000 7ff3b767b740 1 Processor -- start 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44fee90 0x55cda4514110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44ff7b0 0x55cda4514650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3b767b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda4508390 0x55cda457f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3b767b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01ab830 con 0x55cda44ff7b0 2024-04-16T17:13:54.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3b767b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01acca0 con 0x55cda4508390 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3b767b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01abf00 con 0x55cda44fee90 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3b767b740 10 client.0 _ll_register_callbacks cb 0x7ff3a8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44fee90 0x55cda4514110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44fee90 0x55cda4514110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42412/0 (socket says 172.21.15.18:42412) 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3ade00640 1 -- 172.21.15.18:0/1979672325 learned_addr learned my addr 172.21.15.18:0/1979672325 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.844+0000 7ff3ad400640 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44ff7b0 0x55cda4514650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.845+0000 7ff3ae800640 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda4508390 0x55cda457f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.845+0000 7ff3ade00640 1 -- 172.21.15.18:0/1979672325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda4508390 msgr2=0x55cda457f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.845+0000 7ff3ade00640 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda4508390 0x55cda457f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.845+0000 7ff3ade00640 1 -- 172.21.15.18:0/1979672325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44ff7b0 msgr2=0x55cda4514650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.845+0000 7ff3ade00640 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44ff7b0 0x55cda4514650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.845+0000 7ff3ade00640 1 -- 172.21.15.18:0/1979672325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cda457fa80 con 0x55cda44fee90 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.845+0000 7ff3ade00640 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44fee90 0x55cda4514110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0036040 tx=0x7ff3a002cb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c509360bab54dc6c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.845+0000 7ff38f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.846+0000 7ff38f400640 1 -- 172.21.15.18:0/1979672325 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0061070 con 0x55cda44fee90 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.846+0000 7ff38f400640 1 -- 172.21.15.18:0/1979672325 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7ff3a0004ec0 con 0x55cda44fee90 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.846+0000 7ff3b767b740 1 -- 172.21.15.18:0/1979672325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a0004ba0 con 0x55cda44fee90 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.846+0000 7ff38f400640 1 -- 172.21.15.18:0/1979672325 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3a00050c0 con 0x55cda44fee90 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.846+0000 7ff3b767b740 10 client.27994 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.846+0000 7ff3b767b740 1 -- 172.21.15.18:0/1979672325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cda4586e70 con 0x55cda44fee90 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.846+0000 7ff38f400640 1 -- 172.21.15.18:0/1979672325 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a0004ba0 con 0x55cda44fee90 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.847+0000 7ff38f400640 1 -- 172.21.15.18:0/1979672325 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff3a00558a0 con 0x55cda44fee90 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.847+0000 7ff3b767b740 10 client.27994 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.847+0000 7ff3b767b740 10 client.27994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.847+0000 7ff3b767b740 20 client.27994 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:54.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.847+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cda45875a0 0x55cda45899c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.847+0000 7ff3b767b740 4 client.27994 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.847+0000 7ff3b767b740 1 -- 172.21.15.18:0/1979672325 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cda446b540 con 0x55cda45875a0 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.847+0000 7ff3ad400640 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cda45875a0 0x55cda45899c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.848+0000 7ff3ad400640 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cda45875a0 0x55cda45899c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b21835b5de91e9d server_cookie=d1b92072db21a1db in_seq=0 out_seq=0 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.848+0000 7ff38f400640 10 client.27994 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff38f400640 1 -- 172.21.15.18:0/1979672325 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cda446b540 con 0x55cda45875a0 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff38f400640 10 client.27994 handle_command_reply: tid=0 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 client.27994 shutdown 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 -- 172.21.15.18:0/1979672325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44fee90 msgr2=0x55cda4514110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44fee90 0x55cda4514110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0036040 tx=0x7ff3a002cb00 comp rx=0 tx=0).stop 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 -- 172.21.15.18:0/1979672325 shutdown_connections 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cda45875a0 0x55cda45899c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cda4508390 0x55cda457f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cda44ff7b0 0x55cda4514650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff38f400640 0 client.27994 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:54.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 --2- 172.21.15.18:0/1979672325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cda44fee90 0x55cda4514110 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 -- 172.21.15.18:0/1979672325 >> 172.21.15.18:0/1979672325 conn(0x55cda44f95c0 msgr2=0x55cda44fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:54.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 -- 172.21.15.18:0/1979672325 shutdown_connections 2024-04-16T17:13:54.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.849+0000 7ff3b767b740 1 -- 172.21.15.18:0/1979672325 wait complete. 2024-04-16T17:13:54.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.850+0000 7ff3b767b740 20 client.27994 trim_cache size 0 max 16384 2024-04-16T17:13:54.852 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:13:54.852 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:13:54.852 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "6d6e160c-63a7-4169-aaa1-18a70158c994", 2024-04-16T17:13:54.853 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:13:54.853 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:13:54.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.850+0000 7ff3af200640 1 -- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3940a3da0 msgr2=0x7ff3940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.850+0000 7ff3af200640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3940a3da0 0x7ff3940a6260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00096c0 tx=0x7ff3a002df90 comp rx=0 tx=0).stop 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.850+0000 7ff3af200640 1 -- 172.21.15.18:0/3015721809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0109350 msgr2=0x7ff3b01a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.850+0000 7ff3af200640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0109350 0x7ff3b01a0b50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00e9c0 tx=0x7ff39c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.851+0000 7ff3af200640 1 -- 172.21.15.18:0/3015721809 shutdown_connections 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.851+0000 7ff3af200640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff3940a3da0 0x7ff3940a6260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.851+0000 7ff3af200640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b019a780 0x7ff3b019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.851+0000 7ff3af200640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b0109350 0x7ff3b01a0b50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.851+0000 7ff3af200640 1 --2- 172.21.15.18:0/3015721809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3b0104280 0x7ff3b01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.851+0000 7ff3af200640 1 -- 172.21.15.18:0/3015721809 >> 172.21.15.18:0/3015721809 conn(0x7ff3b00ffae0 msgr2=0x7ff3b0101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.851+0000 7ff3af200640 1 -- 172.21.15.18:0/3015721809 shutdown_connections 2024-04-16T17:13:54.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:54.851+0000 7ff3af200640 1 -- 172.21.15.18:0/3015721809 wait complete. 2024-04-16T17:13:54.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:13:54.872 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:13:55.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 -- 172.21.15.18:0/337845945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c074cf0 msgr2=0x7f5d2c10e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 --2- 172.21.15.18:0/337845945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c074cf0 0x7f5d2c10e160 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009860 tx=0x7f5d1802bc10 comp rx=0 tx=0).stop 2024-04-16T17:13:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 -- 172.21.15.18:0/337845945 shutdown_connections 2024-04-16T17:13:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 --2- 172.21.15.18:0/337845945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2c10e6a0 0x7f5d2c110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 --2- 172.21.15.18:0/337845945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c074cf0 0x7f5d2c10e160 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 --2- 172.21.15.18:0/337845945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c0743d0 0x7f5d2c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 -- 172.21.15.18:0/337845945 >> 172.21.15.18:0/337845945 conn(0x7f5d2c0fd950 msgr2=0x7f5d2c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 -- 172.21.15.18:0/337845945 shutdown_connections 2024-04-16T17:13:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.069+0000 7f5d30c00640 1 -- 172.21.15.18:0/337845945 wait complete. 2024-04-16T17:13:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.070+0000 7f5d30c00640 1 Processor -- start 2024-04-16T17:13:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.071+0000 7f5d30c00640 1 -- start start 2024-04-16T17:13:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.071+0000 7f5d30c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2c0743d0 0x7f5d2c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.071+0000 7f5d30c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c074cf0 0x7f5d2c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.071+0000 7f5d30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c10e6a0 0x7f5d2c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.071+0000 7f5d30c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c114400 con 0x7f5d2c074cf0 2024-04-16T17:13:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.071+0000 7f5d30c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c114280 con 0x7f5d2c0743d0 2024-04-16T17:13:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.071+0000 7f5d30c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c114580 con 0x7f5d2c10e6a0 2024-04-16T17:13:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2c0743d0 0x7f5d2c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2c0743d0 0x7f5d2c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44948/0 (socket says 172.21.15.18:44948) 2024-04-16T17:13:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 -- 172.21.15.18:0/1219678915 learned_addr learned my addr 172.21.15.18:0/1219678915 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c074cf0 0x7f5d2c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2be00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c10e6a0 0x7f5d2c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 -- 172.21.15.18:0/1219678915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c10e6a0 msgr2=0x7f5d2c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c10e6a0 0x7f5d2c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 -- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c074cf0 msgr2=0x7f5d2c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c074cf0 0x7f5d2c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 -- 172.21.15.18:0/1219678915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c19af70 con 0x7f5d2c0743d0 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.072+0000 7f5d2b400640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2c0743d0 0x7f5d2c1a0630 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e9c0 tx=0x7f5d1c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.073+0000 7f5d23e00640 1 -- 172.21.15.18:0/1219678915 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00cde0 con 0x7f5d2c0743d0 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.073+0000 7f5d23e00640 1 -- 172.21.15.18:0/1219678915 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c0035f0 con 0x7f5d2c0743d0 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.073+0000 7f5d30c00640 1 -- 172.21.15.18:0/1219678915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c19b260 con 0x7f5d2c0743d0 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.073+0000 7f5d23e00640 1 -- 172.21.15.18:0/1219678915 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c010690 con 0x7f5d2c0743d0 2024-04-16T17:13:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.073+0000 7f5d30c00640 1 -- 172.21.15.18:0/1219678915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c1a7480 con 0x7f5d2c0743d0 2024-04-16T17:13:55.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.076+0000 7f5d34f7b740 1 Processor -- start 2024-04-16T17:13:55.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d34f7b740 1 -- start start 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc1f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 0x55dbddc28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc28890 0x55dbddc2ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dbddb85120 con 0x55dbddc28890 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbddb2b340 con 0x55dbddc1f390 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dbddb810e0 con 0x55dbddc1fcb0 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d2be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc1f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d2be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc1f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44960/0 (socket says 172.21.15.18:44960) 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.077+0000 7f5d2be00640 1 -- 172.21.15.18:0/2427918124 learned_addr learned my addr 172.21.15.18:0/2427918124 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc28890 0x55dbddc2ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d2be00640 1 -- 172.21.15.18:0/2427918124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 msgr2=0x55dbddc28350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d2be00640 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 0x55dbddc28350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d2be00640 1 -- 172.21.15.18:0/2427918124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc28890 msgr2=0x55dbddc2ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d2be00640 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc28890 0x55dbddc2ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d2be00640 1 -- 172.21.15.18:0/2427918124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdd9617d0 con 0x55dbddc1f390 2024-04-16T17:13:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc28890 0x55dbddc2ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d23e00640 1 -- 172.21.15.18:0/1219678915 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d1c0037a0 con 0x7f5d2c0743d0 2024-04-16T17:13:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d2be00640 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc1f770 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009ec0 tx=0x7f5d240107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86df48d3bed19d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d17e00640 1 -- 172.21.15.18:0/2427918124 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24004d20 con 0x55dbddc1f390 2024-04-16T17:13:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d17e00640 1 -- 172.21.15.18:0/2427918124 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d24003e50 con 0x55dbddc1f390 2024-04-16T17:13:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.078+0000 7f5d17e00640 1 -- 172.21.15.18:0/2427918124 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24005280 con 0x55dbddc1f390 2024-04-16T17:13:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.079+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2427918124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 msgr2=0x55dbddc1f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.079+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc1f770 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009ec0 tx=0x7f5d240107a0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.079+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2427918124 shutdown_connections 2024-04-16T17:13:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.079+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc28890 0x55dbddc2ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.079+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 0x55dbddc28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.079+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2427918124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc1f770 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.079+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2427918124 >> 172.21.15.18:0/2427918124 conn(0x55dbddc19ac0 msgr2=0x55dbddc19ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.080+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2427918124 shutdown_connections 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.080+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2427918124 wait complete. 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.080+0000 7f5d23e00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d100a3c30 0x7f5d100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.080+0000 7f5d23e00640 1 -- 172.21.15.18:0/1219678915 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f5d1c0d0fe0 con 0x7f5d2c0743d0 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.080+0000 7f5d34f7b740 1 -- start start 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.080+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d100a3c30 0x7f5d100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d34f7b740 1 Processor -- start 2024-04-16T17:13:55.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc1f390 0x55dbddc25980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc25ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc28890 0x55dbddc26400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dbddc2b680 con 0x55dbddc1f390 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbddb3e710 con 0x55dbddc1fcb0 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dbddc2b800 con 0x55dbddc28890 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d100a3c30 0x7f5d100a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5d1802c220 tx=0x7f5d180058d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.081+0000 7f5d34f7b740 10 client.0 _ll_register_callbacks cb 0x55dbddb39af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc1f390 0x55dbddc25980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc25ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc1f390 0x55dbddc25980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53068/0 (socket says 172.21.15.18:53068) 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2b400640 1 -- 172.21.15.18:0/3706507165 learned_addr learned my addr 172.21.15.18:0/3706507165 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2be00640 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc28890 0x55dbddc26400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2b400640 1 -- 172.21.15.18:0/3706507165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc28890 msgr2=0x55dbddc26400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2b400640 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc28890 0x55dbddc26400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2b400640 1 -- 172.21.15.18:0/3706507165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 msgr2=0x55dbddc25ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2b400640 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc25ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2b400640 1 -- 172.21.15.18:0/3706507165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbdd8973f0 con 0x55dbddc1f390 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.082+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.083+0000 7f5d2b400640 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc1f390 0x55dbddc25980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0ce920 tx=0x7f5d1c09c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e8d579628eb41a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.083+0000 7f5d14c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.083+0000 7f5d14c00640 1 -- 172.21.15.18:0/3706507165 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0a0f20 con 0x55dbddc1f390 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.083+0000 7f5d14c00640 1 -- 172.21.15.18:0/3706507165 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f5d1c0991e0 con 0x55dbddc1f390 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.083+0000 7f5d34f7b740 1 -- 172.21.15.18:0/3706507165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d24003e50 con 0x55dbddc1f390 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.083+0000 7f5d14c00640 1 -- 172.21.15.18:0/3706507165 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d1c0a57e0 con 0x55dbddc1f390 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.083+0000 7f5d34f7b740 10 client.17823 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.083+0000 7f5d34f7b740 1 -- 172.21.15.18:0/3706507165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbdd8570f0 con 0x55dbddc1f390 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d14c00640 1 -- 172.21.15.18:0/3706507165 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d24003e50 con 0x55dbddc1f390 2024-04-16T17:13:55.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d14c00640 1 -- 172.21.15.18:0/3706507165 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5d1c0b7020 con 0x55dbddc1f390 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d34f7b740 10 client.17823 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d34f7b740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d34f7b740 20 client.17823 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dbddc38b90 0x55dbddc3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d34f7b740 4 client.17823 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d34f7b740 1 -- 172.21.15.18:0/3706507165 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbddb92b40 con 0x55dbddc38b90 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.084+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dbddc38b90 0x55dbddc3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.085+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dbddc38b90 0x55dbddc3af80 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c911f78c50ec567 server_cookie=bbfcefee45e73476 in_seq=0 out_seq=0 2024-04-16T17:13:55.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.085+0000 7f5d14c00640 10 client.17823 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:55.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.086+0000 7f5d14c00640 1 -- 172.21.15.18:0/3706507165 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbddb92b40 con 0x55dbddc38b90 2024-04-16T17:13:55.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.086+0000 7f5d14c00640 10 client.17823 handle_command_reply: tid=0 2024-04-16T17:13:55.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 client.17823 shutdown 2024-04-16T17:13:55.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 -- 172.21.15.18:0/3706507165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc1f390 msgr2=0x55dbddc25980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc1f390 0x55dbddc25980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0ce920 tx=0x7f5d1c09c770 comp rx=0 tx=0).stop 2024-04-16T17:13:55.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 -- 172.21.15.18:0/3706507165 shutdown_connections 2024-04-16T17:13:55.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dbddc38b90 0x55dbddc3af80 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc28890 0x55dbddc26400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d14c00640 0 client.17823 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:55.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/3706507165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc1f390 0x55dbddc25980 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.087+0000 7f5d34f7b740 1 -- 172.21.15.18:0/3706507165 >> 172.21.15.18:0/3706507165 conn(0x55dbddc19ac0 msgr2=0x55dbddc1b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:55.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.088+0000 7f5d34f7b740 1 -- 172.21.15.18:0/3706507165 shutdown_connections 2024-04-16T17:13:55.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.088+0000 7f5d34f7b740 1 -- 172.21.15.18:0/3706507165 wait complete. 2024-04-16T17:13:55.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.088+0000 7f5d34f7b740 20 client.17823 trim_cache size 0 max 16384 2024-04-16T17:13:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.111+0000 7f5d34f7b740 1 Processor -- start 2024-04-16T17:13:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.111+0000 7f5d34f7b740 1 -- start start 2024-04-16T17:13:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.111+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.111+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1f390 0x55dbddc248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.111+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc34540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.111+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1b3f30 con 0x55dbddc19ac0 2024-04-16T17:13:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.111+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1b1800 con 0x55dbddc1fcb0 2024-04-16T17:13:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.111+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1af670 con 0x55dbddc1f390 2024-04-16T17:13:55.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53084/0 (socket says 172.21.15.18:53084) 2024-04-16T17:13:55.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2b400640 1 -- 172.21.15.18:0/2923172402 learned_addr learned my addr 172.21.15.18:0/2923172402 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2be00640 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc34540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1f390 0x55dbddc248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2b400640 1 -- 172.21.15.18:0/2923172402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1f390 msgr2=0x55dbddc248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2b400640 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1f390 0x55dbddc248e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2b400640 1 -- 172.21.15.18:0/2923172402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 msgr2=0x55dbddc34540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2b400640 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc34540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.112+0000 7f5d2b400640 1 -- 172.21.15.18:0/2923172402 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbddc1fae0 con 0x55dbddc19ac0 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.113+0000 7f5d2b400640 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc280b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c098fd0 tx=0x7f5d1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8650e463b8434494 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.113+0000 7f5d17400640 1 -- 172.21.15.18:0/2923172402 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0988c0 con 0x55dbddc19ac0 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.113+0000 7f5d17400640 1 -- 172.21.15.18:0/2923172402 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c0a0f20 con 0x55dbddc19ac0 2024-04-16T17:13:55.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.113+0000 7f5d17400640 1 -- 172.21.15.18:0/2923172402 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c09bfc0 con 0x55dbddc19ac0 2024-04-16T17:13:55.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2923172402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 msgr2=0x55dbddc280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc280b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c098fd0 tx=0x7f5d1c004730 comp rx=0 tx=0).stop 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2923172402 shutdown_connections 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1fcb0 0x55dbddc34540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1f390 0x55dbddc248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2923172402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc280b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2923172402 >> 172.21.15.18:0/2923172402 conn(0x55dbddc21390 msgr2=0x55dbddc18b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2923172402 shutdown_connections 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.114+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2923172402 wait complete. 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 1 -- start start 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 1 Processor -- start 2024-04-16T17:13:55.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc22d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc23250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 0x55dbddc9e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1afcc0 con 0x55dbddc19ac0 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1b1130 con 0x55dbddc1f390 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1b0390 con 0x55dbddc1fcb0 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.115+0000 7f5d34f7b740 10 client.0 _ll_register_callbacks cb 0x7f5d1c004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc22d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc22d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53092/0 (socket says 172.21.15.18:53092) 2024-04-16T17:13:55.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2b400640 1 -- 172.21.15.18:0/2814834294 learned_addr learned my addr 172.21.15.18:0/2814834294 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2be00640 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 0x55dbddc9e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc23250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2b400640 1 -- 172.21.15.18:0/2814834294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 msgr2=0x55dbddc9e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2b400640 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 0x55dbddc9e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2b400640 1 -- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 msgr2=0x55dbddc23250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2b400640 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc23250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2b400640 1 -- 172.21.15.18:0/2814834294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbddc3b2f0 con 0x55dbddc19ac0 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.116+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc23250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d2b400640 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc22d10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c6a0 tx=0x7f5d1c09cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619bedd981cab9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d14c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d14c00640 1 -- 172.21.15.18:0/2814834294 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c098c40 con 0x55dbddc19ac0 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d14c00640 1 -- 172.21.15.18:0/2814834294 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f5d1c09c1d0 con 0x55dbddc19ac0 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2814834294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1c0a0f20 con 0x55dbddc19ac0 2024-04-16T17:13:55.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d14c00640 1 -- 172.21.15.18:0/2814834294 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d1c0988c0 con 0x55dbddc19ac0 2024-04-16T17:13:55.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d34f7b740 10 client.17829 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:55.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2814834294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbddc9f610 con 0x55dbddc19ac0 2024-04-16T17:13:55.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.117+0000 7f5d14c00640 1 -- 172.21.15.18:0/2814834294 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c0a0f20 con 0x55dbddc19ac0 2024-04-16T17:13:55.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.118+0000 7f5d14c00640 1 -- 172.21.15.18:0/2814834294 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5d1c0b7020 con 0x55dbddc19ac0 2024-04-16T17:13:55.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.118+0000 7f5d34f7b740 10 client.17829 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:55.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.118+0000 7f5d34f7b740 10 client.17829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:55.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.118+0000 7f5d34f7b740 20 client.17829 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:55.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.118+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dbddc28890 0x55dbddca1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:55.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.118+0000 7f5d34f7b740 4 client.17829 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:13:55.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.118+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2814834294 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbddb8c9b0 con 0x55dbddc28890 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.118+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dbddc28890 0x55dbddca1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d14c00640 10 client.17829 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d2aa00640 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dbddc28890 0x55dbddca1af0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484d78838b117146 server_cookie=8f0347fa84ca5c26 in_seq=0 out_seq=0 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d14c00640 1 -- 172.21.15.18:0/2814834294 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dbddb8c9b0 con 0x55dbddc28890 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d14c00640 10 client.17829 handle_command_reply: tid=0 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d34f7b740 1 client.17829 shutdown 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2814834294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 msgr2=0x55dbddc22d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc22d10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c6a0 tx=0x7f5d1c09cc30 comp rx=0 tx=0).stop 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2814834294 shutdown_connections 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dbddc28890 0x55dbddca1af0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbddc1fcb0 0x55dbddc9e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbddc1f390 0x55dbddc23250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d34f7b740 1 --2- 172.21.15.18:0/2814834294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dbddc19ac0 0x55dbddc22d10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.120+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2814834294 >> 172.21.15.18:0/2814834294 conn(0x55dbddc21390 msgr2=0x55dbddc1ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.119+0000 7f5d14c00640 0 client.17829 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:55.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.120+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2814834294 shutdown_connections 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.120+0000 7f5d34f7b740 1 -- 172.21.15.18:0/2814834294 wait complete. 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.120+0000 7f5d34f7b740 20 client.17829 trim_cache size 0 max 16384 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout: "6d6e160c-63a7-4169-aaa1-18a70158c994": { 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "6d6e160c-63a7-4169-aaa1-18a70158c994", 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:13:55.123 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:13:55.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 -- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d100a3c30 msgr2=0x7f5d100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d100a3c30 0x7f5d100a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5d1802c220 tx=0x7f5d180058d0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 -- 172.21.15.18:0/1219678915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2c0743d0 msgr2=0x7f5d2c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:55.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2c0743d0 0x7f5d2c1a0630 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e9c0 tx=0x7f5d1c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:13:55.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 -- 172.21.15.18:0/1219678915 shutdown_connections 2024-04-16T17:13:55.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d100a3c30 0x7f5d100a60f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c10e6a0 0x7f5d2c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d2c074cf0 0x7f5d2c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 --2- 172.21.15.18:0/1219678915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2c0743d0 0x7f5d2c1a0630 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:55.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 -- 172.21.15.18:0/1219678915 >> 172.21.15.18:0/1219678915 conn(0x7f5d2c0fd950 msgr2=0x7f5d2c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:55.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 -- 172.21.15.18:0/1219678915 shutdown_connections 2024-04-16T17:13:55.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:13:55.121+0000 7f5d30c00640 1 -- 172.21.15.18:0/1219678915 wait complete. 2024-04-16T17:13:55.142 INFO:tasks.cephfs.filesystem:scrub status for tag:6d6e160c-63a7-4169-aaa1-18a70158c994 - {'path': '/', 'tag': '6d6e160c-63a7-4169-aaa1-18a70158c994', 'options': 'recursive,force'} 2024-04-16T17:13:55.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 347441 3.73 MB/sec execute 368 sec latency 40.221 ms 2024-04-16T17:13:55.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:55 smithi022 ceph-mon[36023]: pgmap v286: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 633 KiB/s wr, 107 op/s 2024-04-16T17:13:55.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:55 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:13:55.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:55 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:55.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:55 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:13:55.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:55 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:55 smithi043 ceph-mon[36901]: pgmap v286: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 633 KiB/s wr, 107 op/s 2024-04-16T17:13:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:55 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:13:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:55 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:55 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:13:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:55 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:56.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:45 smithi018 ceph-mon[31776]: pgmap v281: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 853 KiB/s wr, 104 op/s 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:47 smithi018 ceph-mon[31776]: pgmap v282: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:49 smithi018 ceph-mon[31776]: pgmap v283: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 885 KiB/s wr, 68 op/s 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:51 smithi018 ceph-mon[31776]: pgmap v284: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:53 smithi018 ceph-mon[31776]: pgmap v285: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 636 KiB/s wr, 108 op/s 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:53 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1508741135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:55 smithi018 ceph-mon[31776]: pgmap v286: 129 pgs: 129 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 633 KiB/s wr, 107 op/s 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:55 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:55 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:55 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:13:56.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:55 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:56.628 INFO:tasks.workunit.client.0.smithi018.stdout: 1 347833 3.73 MB/sec execute 369 sec latency 88.340 ms 2024-04-16T17:13:57.629 INFO:tasks.workunit.client.0.smithi018.stdout: 1 348007 3.72 MB/sec execute 370 sec latency 82.842 ms 2024-04-16T17:13:57.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:57 smithi022 ceph-mon[36023]: pgmap v287: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 929 KiB/s wr, 142 op/s 2024-04-16T17:13:58.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:57 smithi043 ceph-mon[36901]: pgmap v287: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 929 KiB/s wr, 142 op/s 2024-04-16T17:13:58.629 INFO:tasks.workunit.client.0.smithi018.stdout: 1 348669 3.72 MB/sec execute 371 sec latency 27.172 ms 2024-04-16T17:13:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.647+0000 7fb84e000640 1 -- 172.21.15.18:0/795088400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848107fb0 msgr2=0x7fb848108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.647+0000 7fb84e000640 1 --2- 172.21.15.18:0/795088400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848107fb0 0x7fb848108390 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009930 tx=0x7fb83c02bc70 comp rx=0 tx=0).stop 2024-04-16T17:13:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.647+0000 7fb84e000640 1 -- 172.21.15.18:0/795088400 shutdown_connections 2024-04-16T17:13:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.647+0000 7fb84e000640 1 --2- 172.21.15.18:0/795088400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8481010e0 0x7fb84810ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.647+0000 7fb84e000640 1 --2- 172.21.15.18:0/795088400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb848100720 0x7fb848100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.647+0000 7fb84e000640 1 --2- 172.21.15.18:0/795088400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848107fb0 0x7fb848108390 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.647+0000 7fb84e000640 1 -- 172.21.15.18:0/795088400 >> 172.21.15.18:0/795088400 conn(0x7fb848076070 msgr2=0x7fb848076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.647+0000 7fb84e000640 1 -- 172.21.15.18:0/795088400 shutdown_connections 2024-04-16T17:13:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.648+0000 7fb84e000640 1 -- 172.21.15.18:0/795088400 wait complete. 2024-04-16T17:13:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.649+0000 7fb84e000640 1 Processor -- start 2024-04-16T17:13:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.649+0000 7fb84e000640 1 -- start start 2024-04-16T17:13:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848100720 0x7fb848101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8481010e0 0x7fb848101e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84e000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb848107fb0 0x7fb848102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84e000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481122a0 con 0x7fb848107fb0 2024-04-16T17:13:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84e000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb848112120 con 0x7fb848100720 2024-04-16T17:13:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84e000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb848112420 con 0x7fb8481010e0 2024-04-16T17:13:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848100720 0x7fb848101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848100720 0x7fb848101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44932/0 (socket says 172.21.15.18:44932) 2024-04-16T17:13:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb84cc00640 1 -- 172.21.15.18:0/1836282257 learned_addr learned my addr 172.21.15.18:0/1836282257 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.650+0000 7fb847e00640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8481010e0 0x7fb848101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.651+0000 7fb84d600640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb848107fb0 0x7fb848102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.651+0000 7fb84cc00640 1 -- 172.21.15.18:0/1836282257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8481010e0 msgr2=0x7fb848101e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.651+0000 7fb84cc00640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8481010e0 0x7fb848101e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.651+0000 7fb84cc00640 1 -- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb848107fb0 msgr2=0x7fb848102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.651+0000 7fb84cc00640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb848107fb0 0x7fb848102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.651+0000 7fb84cc00640 1 -- 172.21.15.18:0/1836282257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8481a6d80 con 0x7fb848100720 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.651+0000 7fb847e00640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8481010e0 0x7fb848101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.651+0000 7fb84cc00640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848100720 0x7fb848101900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005ab0 tx=0x7fb83c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.652+0000 7fb845600640 1 -- 172.21.15.18:0/1836282257 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c043020 con 0x7fb848100720 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.652+0000 7fb84e000640 1 -- 172.21.15.18:0/1836282257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8481a7010 con 0x7fb848100720 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.652+0000 7fb845600640 1 -- 172.21.15.18:0/1836282257 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c0054e0 con 0x7fb848100720 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.652+0000 7fb845600640 1 -- 172.21.15.18:0/1836282257 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0039b0 con 0x7fb848100720 2024-04-16T17:13:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.652+0000 7fb84e000640 1 -- 172.21.15.18:0/1836282257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481a75b0 con 0x7fb848100720 2024-04-16T17:13:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.655+0000 7fb845600640 1 -- 172.21.15.18:0/1836282257 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb83c046020 con 0x7fb848100720 2024-04-16T17:13:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.656+0000 7fb845600640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8300a3e00 0x7fb8300a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.657+0000 7fb845600640 1 -- 172.21.15.18:0/1836282257 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fb83c0f2c20 con 0x7fb848100720 2024-04-16T17:13:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.657+0000 7fb85241d740 1 Processor -- start 2024-04-16T17:13:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.657+0000 7fb847e00640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8300a3e00 0x7fb8300a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.657+0000 7fb85241d740 1 -- start start 2024-04-16T17:13:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb85241d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb85241d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162e8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb85241d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162f1810 0x55ab162f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb85241d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ab16244780 con 0x55ab162e9000 2024-04-16T17:13:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb85241d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ab161eaf40 con 0x55ab162e8290 2024-04-16T17:13:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb85241d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ab16240740 con 0x55ab162f1810 2024-04-16T17:13:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb847e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb847e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162e8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44938/0 (socket says 172.21.15.18:44938) 2024-04-16T17:13:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.658+0000 7fb847e00640 1 -- 172.21.15.18:0/1955585888 learned_addr learned my addr 172.21.15.18:0/1955585888 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb84cc00640 1 -- 172.21.15.18:0/1955585888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162f1810 msgr2=0x55ab162f3c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb84cc00640 1 --2- 172.21.15.18:0/1955585888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162f1810 0x55ab162f3c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb84cc00640 1 -- 172.21.15.18:0/1955585888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 msgr2=0x55ab162e8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb84cc00640 1 --2- 172.21.15.18:0/1955585888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162e8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb84cc00640 1 -- 172.21.15.18:0/1955585888 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab15f57210 con 0x55ab162e9000 2024-04-16T17:13:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb847e00640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8300a3e00 0x7fb8300a62c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb848103390 tx=0x7fb834005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb84cc00640 1 --2- 172.21.15.18:0/1955585888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162e7d50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0f0010 tx=0x7fb83c02bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff24b6b18a3ed20 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb839c00640 1 -- 172.21.15.18:0/1955585888 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0be860 con 0x55ab162e9000 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb839c00640 1 -- 172.21.15.18:0/1955585888 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c0c2c10 con 0x55ab162e9000 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.659+0000 7fb839c00640 1 -- 172.21.15.18:0/1955585888 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0f0180 con 0x55ab162e9000 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.660+0000 7fb85241d740 1 -- 172.21.15.18:0/1955585888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 msgr2=0x55ab162e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.660+0000 7fb85241d740 1 --2- 172.21.15.18:0/1955585888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162e7d50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0f0010 tx=0x7fb83c02bc70 comp rx=0 tx=0).stop 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.660+0000 7fb85241d740 1 -- 172.21.15.18:0/1955585888 shutdown_connections 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.660+0000 7fb85241d740 1 --2- 172.21.15.18:0/1955585888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162f1810 0x55ab162f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.660+0000 7fb85241d740 1 --2- 172.21.15.18:0/1955585888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162e8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.660+0000 7fb85241d740 1 --2- 172.21.15.18:0/1955585888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162e7d50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.660+0000 7fb85241d740 1 -- 172.21.15.18:0/1955585888 >> 172.21.15.18:0/1955585888 conn(0x55ab1627a5d0 msgr2=0x55ab162e2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.661+0000 7fb85241d740 1 -- 172.21.15.18:0/1955585888 shutdown_connections 2024-04-16T17:13:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.661+0000 7fb85241d740 1 -- 172.21.15.18:0/1955585888 wait complete. 2024-04-16T17:13:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.661+0000 7fb85241d740 1 -- start start 2024-04-16T17:13:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.661+0000 7fb85241d740 1 Processor -- start 2024-04-16T17:13:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb85241d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb85241d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162eec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb85241d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb85241d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ab162f4300 con 0x55ab162e9000 2024-04-16T17:13:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb85241d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ab161fe390 con 0x55ab162f1810 2024-04-16T17:13:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb85241d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ab162f4480 con 0x55ab162e8290 2024-04-16T17:13:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb85241d740 10 client.0 _ll_register_callbacks cb 0x55ab161e7f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb847e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162ee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53830/0 (socket says 172.21.15.18:53830) 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb84cc00640 1 -- 172.21.15.18:0/3635824915 learned_addr learned my addr 172.21.15.18:0/3635824915 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.662+0000 7fb84d600640 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.663+0000 7fb84d600640 1 -- 172.21.15.18:0/3635824915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 msgr2=0x55ab162ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.663+0000 7fb84d600640 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.663+0000 7fb84d600640 1 -- 172.21.15.18:0/3635824915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 msgr2=0x55ab162eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.663+0000 7fb84d600640 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.663+0000 7fb84d600640 1 -- 172.21.15.18:0/3635824915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab16023620 con 0x55ab162f1810 2024-04-16T17:13:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.663+0000 7fb84d600640 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162ef1a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb840005fd0 tx=0x7fb840010c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9baa2d3a14586da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.663+0000 7fb82e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.664+0000 7fb82e000640 1 -- 172.21.15.18:0/3635824915 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840017070 con 0x55ab162f1810 2024-04-16T17:13:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.664+0000 7fb82e000640 1 -- 172.21.15.18:0/3635824915 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fb84000ecc0 con 0x55ab162f1810 2024-04-16T17:13:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.664+0000 7fb85241d740 1 -- 172.21.15.18:0/3635824915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83c0c2c10 con 0x55ab162f1810 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.664+0000 7fb82e000640 1 -- 172.21.15.18:0/3635824915 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb840003970 con 0x55ab162f1810 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.664+0000 7fb85241d740 10 client.28121 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.664+0000 7fb85241d740 1 -- 172.21.15.18:0/3635824915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab16300420 con 0x55ab162f1810 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.664+0000 7fb82e000640 1 -- 172.21.15.18:0/3635824915 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb840015070 con 0x55ab162f1810 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.665+0000 7fb82e000640 1 -- 172.21.15.18:0/3635824915 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb840011440 con 0x55ab162f1810 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.665+0000 7fb85241d740 10 client.28121 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.665+0000 7fb85241d740 10 client.28121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.665+0000 7fb85241d740 20 client.28121 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.665+0000 7fb85241d740 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab16301930 0x55ab16303d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.665+0000 7fb85241d740 4 client.28121 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.665+0000 7fb85241d740 1 -- 172.21.15.18:0/3635824915 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb83c0c2c10 con 0x55ab16301930 2024-04-16T17:13:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.666+0000 7fb84cc00640 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab16301930 0x55ab16303d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.666+0000 7fb84cc00640 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab16301930 0x55ab16303d20 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a2bd3553df22e3 server_cookie=10c85c1c610d889b in_seq=0 out_seq=0 2024-04-16T17:13:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.666+0000 7fb82e000640 10 client.28121 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.667+0000 7fb82e000640 1 -- 172.21.15.18:0/3635824915 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fb83c0c2c10 con 0x55ab16301930 2024-04-16T17:13:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.667+0000 7fb82e000640 10 client.28121 handle_command_reply: tid=0 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.667+0000 7fb85241d740 1 client.28121 shutdown 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 -- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 msgr2=0x55ab162ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162ef1a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb840005fd0 tx=0x7fb840010c20 comp rx=0 tx=0).stop 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 -- 172.21.15.18:0/3635824915 shutdown_connections 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab16301930 0x55ab16303d20 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162ef1a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 --2- 172.21.15.18:0/3635824915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 -- 172.21.15.18:0/3635824915 >> 172.21.15.18:0/3635824915 conn(0x55ab1627a5d0 msgr2=0x55ab162e4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb82e000640 0 client.28121 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 -- 172.21.15.18:0/3635824915 shutdown_connections 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 1 -- 172.21.15.18:0/3635824915 wait complete. 2024-04-16T17:13:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.668+0000 7fb85241d740 20 client.28121 trim_cache size 0 max 16384 2024-04-16T17:13:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.687+0000 7fb85241d740 1 Processor -- start 2024-04-16T17:13:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.687+0000 7fb85241d740 1 -- start start 2024-04-16T17:13:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.687+0000 7fb85241d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.687+0000 7fb85241d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.687+0000 7fb85241d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162d9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.687+0000 7fb85241d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481afbd0 con 0x55ab162e9000 2024-04-16T17:13:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.687+0000 7fb85241d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ad4a0 con 0x55ab162f1810 2024-04-16T17:13:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.687+0000 7fb85241d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ab310 con 0x55ab162e8290 2024-04-16T17:13:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162d9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162d9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44970/0 (socket says 172.21.15.18:44970) 2024-04-16T17:13:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84d600640 1 -- 172.21.15.18:0/894957547 learned_addr learned my addr 172.21.15.18:0/894957547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84cc00640 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb847e00640 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84d600640 1 -- 172.21.15.18:0/894957547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 msgr2=0x55ab162fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84d600640 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84d600640 1 -- 172.21.15.18:0/894957547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 msgr2=0x55ab162eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84d600640 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.688+0000 7fb84d600640 1 -- 172.21.15.18:0/894957547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab16304090 con 0x55ab162f1810 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.689+0000 7fb84d600640 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162d9690 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb840000f80 tx=0x7fb840010fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c103b45ab05fc822 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.689+0000 7fb839200640 1 -- 172.21.15.18:0/894957547 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84002d070 con 0x55ab162f1810 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.689+0000 7fb839200640 1 -- 172.21.15.18:0/894957547 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb840015070 con 0x55ab162f1810 2024-04-16T17:13:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.689+0000 7fb839200640 1 -- 172.21.15.18:0/894957547 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840017030 con 0x55ab162f1810 2024-04-16T17:13:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.690+0000 7fb85241d740 1 -- 172.21.15.18:0/894957547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 msgr2=0x55ab162d9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.690+0000 7fb85241d740 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162d9690 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb840000f80 tx=0x7fb840010fb0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.690+0000 7fb85241d740 1 -- 172.21.15.18:0/894957547 shutdown_connections 2024-04-16T17:13:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.690+0000 7fb85241d740 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162f1810 0x55ab162d9690 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.690+0000 7fb85241d740 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162e9000 0x55ab162eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.690+0000 7fb85241d740 1 --2- 172.21.15.18:0/894957547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e8290 0x55ab162fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.690+0000 7fb85241d740 1 -- 172.21.15.18:0/894957547 >> 172.21.15.18:0/894957547 conn(0x55ab1627a5d0 msgr2=0x55ab162d8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.690+0000 7fb85241d740 1 -- 172.21.15.18:0/894957547 shutdown_connections 2024-04-16T17:13:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.691+0000 7fb85241d740 1 -- 172.21.15.18:0/894957547 wait complete. 2024-04-16T17:13:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.691+0000 7fb85241d740 1 -- start start 2024-04-16T17:13:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.691+0000 7fb85241d740 1 Processor -- start 2024-04-16T17:13:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb85241d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162f5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb85241d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e9000 0x55ab162f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb85241d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162f1810 0x55ab162f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb85241d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ab960 con 0x55ab162f1810 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb85241d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481acdd0 con 0x55ab162e8290 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb85241d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ac030 con 0x55ab162e9000 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb85241d740 10 client.0 _ll_register_callbacks cb 0x7fb8400099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162f5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44972/0 (socket says 172.21.15.18:44972) 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb84cc00640 1 -- 172.21.15.18:0/157490185 learned_addr learned my addr 172.21.15.18:0/157490185 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb84d600640 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162f1810 0x55ab162f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb847e00640 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e9000 0x55ab162f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb84cc00640 1 -- 172.21.15.18:0/157490185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e9000 msgr2=0x55ab162f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.692+0000 7fb84cc00640 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e9000 0x55ab162f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.693+0000 7fb84cc00640 1 -- 172.21.15.18:0/157490185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162f1810 msgr2=0x55ab162f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.693+0000 7fb84cc00640 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162f1810 0x55ab162f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.693+0000 7fb84cc00640 1 -- 172.21.15.18:0/157490185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab16367030 con 0x55ab162e8290 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.693+0000 7fb84cc00640 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162f5150 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009900 tx=0x7fb83c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c561d3f39dc039d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.693+0000 7fb82e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:13:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.693+0000 7fb82e000640 1 -- 172.21.15.18:0/157490185 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0c2c10 con 0x55ab162e8290 2024-04-16T17:13:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.693+0000 7fb82e000640 1 -- 172.21.15.18:0/157490185 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fb83c0baed0 con 0x55ab162e8290 2024-04-16T17:13:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.693+0000 7fb85241d740 1 -- 172.21.15.18:0/157490185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb840015070 con 0x55ab162e8290 2024-04-16T17:13:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb82e000640 1 -- 172.21.15.18:0/157490185 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb83c0a6050 con 0x55ab162e8290 2024-04-16T17:13:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb85241d740 10 client.28127 fetch_fsmap learned FSMap version 27 2024-04-16T17:13:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb85241d740 1 -- 172.21.15.18:0/157490185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab16368360 con 0x55ab162e8290 2024-04-16T17:13:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb82e000640 1 -- 172.21.15.18:0/157490185 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb840015070 con 0x55ab162e8290 2024-04-16T17:13:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb82e000640 1 -- 172.21.15.18:0/157490185 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb83c0beac0 con 0x55ab162e8290 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb85241d740 10 client.28127 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb85241d740 10 client.28127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb85241d740 20 client.28127 populate_metadata read hostname 'smithi018' 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb85241d740 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab163688a0 0x55ab1636ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb85241d740 4 client.28127 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.694+0000 7fb85241d740 1 -- 172.21.15.18:0/157490185 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab162521a0 con 0x55ab163688a0 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.695+0000 7fb847e00640 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab163688a0 0x55ab1636ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.696+0000 7fb82e000640 10 client.28127 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.696+0000 7fb847e00640 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab163688a0 0x55ab1636ac90 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999a7d60f4d34a24 server_cookie=1d22eac74a3c53b5 in_seq=0 out_seq=0 2024-04-16T17:13:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.885+0000 7fb82e000640 1 -- 172.21.15.18:0/157490185 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab162521a0 con 0x55ab163688a0 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.885+0000 7fb82e000640 10 client.28127 handle_command_reply: tid=0 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.885+0000 7fb85241d740 1 client.28127 shutdown 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 -- 172.21.15.18:0/157490185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 msgr2=0x55ab162f5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162f5150 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009900 tx=0x7fb83c009560 comp rx=0 tx=0).stop 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 -- 172.21.15.18:0/157490185 shutdown_connections 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab163688a0 0x55ab1636ac90 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab162f1810 0x55ab162f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab162e9000 0x55ab162f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 --2- 172.21.15.18:0/157490185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab162e8290 0x55ab162f5150 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb82e000640 0 client.28127 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 -- 172.21.15.18:0/157490185 >> 172.21.15.18:0/157490185 conn(0x55ab1627a5d0 msgr2=0x55ab162e4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 -- 172.21.15.18:0/157490185 shutdown_connections 2024-04-16T17:13:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 1 -- 172.21.15.18:0/157490185 wait complete. 2024-04-16T17:13:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.886+0000 7fb85241d740 20 client.28127 trim_cache size 0 max 16384 2024-04-16T17:13:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:13:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:13:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:13:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:13:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.887+0000 7fb84e000640 1 -- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8300a3e00 msgr2=0x7fb8300a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.887+0000 7fb84e000640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8300a3e00 0x7fb8300a62c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb848103390 tx=0x7fb834005f90 comp rx=0 tx=0).stop 2024-04-16T17:13:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.887+0000 7fb84e000640 1 -- 172.21.15.18:0/1836282257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848100720 msgr2=0x7fb848101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.887+0000 7fb84e000640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848100720 0x7fb848101900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005ab0 tx=0x7fb83c0042f0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.888+0000 7fb84e000640 1 -- 172.21.15.18:0/1836282257 shutdown_connections 2024-04-16T17:13:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.888+0000 7fb84e000640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb8300a3e00 0x7fb8300a62c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.888+0000 7fb84e000640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb848107fb0 0x7fb848102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.888+0000 7fb84e000640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8481010e0 0x7fb848101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.888+0000 7fb84e000640 1 --2- 172.21.15.18:0/1836282257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb848100720 0x7fb848101900 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.888+0000 7fb84e000640 1 -- 172.21.15.18:0/1836282257 >> 172.21.15.18:0/1836282257 conn(0x7fb848076070 msgr2=0x7fb8480ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.888+0000 7fb84e000640 1 -- 172.21.15.18:0/1836282257 shutdown_connections 2024-04-16T17:13:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:13:58.888+0000 7fb84e000640 1 -- 172.21.15.18:0/1836282257 wait complete. 2024-04-16T17:13:59.629 INFO:tasks.workunit.client.0.smithi018.stdout: 1 349389 3.71 MB/sec execute 372 sec latency 48.337 ms 2024-04-16T17:13:59.803 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:59 smithi043 ceph-mon[36901]: pgmap v288: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 661 KiB/s wr, 112 op/s 2024-04-16T17:13:59.803 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:13:59 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:13:59.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:59 smithi022 ceph-mon[36023]: pgmap v288: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 661 KiB/s wr, 112 op/s 2024-04-16T17:13:59.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:13:59 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:14:00.629 INFO:tasks.workunit.client.0.smithi018.stdout: 1 350516 3.72 MB/sec execute 373 sec latency 46.382 ms 2024-04-16T17:14:01.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:57 smithi018 ceph-mon[31776]: pgmap v287: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 929 KiB/s wr, 142 op/s 2024-04-16T17:14:01.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:59 smithi018 ceph-mon[31776]: pgmap v288: 129 pgs: 129 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 661 KiB/s wr, 112 op/s 2024-04-16T17:14:01.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:13:59 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:14:01.629 INFO:tasks.workunit.client.0.smithi018.stdout: 1 350852 3.72 MB/sec execute 374 sec latency 111.800 ms 2024-04-16T17:14:01.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:01 smithi018 ceph-mon[31776]: pgmap v289: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-04-16T17:14:01.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:01 smithi022 ceph-mon[36023]: pgmap v289: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-04-16T17:14:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:01 smithi043 ceph-mon[36901]: pgmap v289: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-04-16T17:14:02.629 INFO:tasks.workunit.client.0.smithi018.stdout: 1 351099 3.72 MB/sec execute 375 sec latency 97.143 ms 2024-04-16T17:14:03.629 INFO:tasks.workunit.client.0.smithi018.stdout: 1 351444 3.71 MB/sec execute 376 sec latency 67.360 ms 2024-04-16T17:14:03.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:03 smithi022 ceph-mon[36023]: pgmap v290: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 832 KiB/s wr, 134 op/s 2024-04-16T17:14:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:03 smithi043 ceph-mon[36901]: pgmap v290: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 832 KiB/s wr, 134 op/s 2024-04-16T17:14:04.633 INFO:tasks.workunit.client.0.smithi018.stdout: 1 351947 3.70 MB/sec execute 377 sec latency 26.795 ms 2024-04-16T17:14:05.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:03 smithi018 ceph-mon[31776]: pgmap v290: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 832 KiB/s wr, 134 op/s 2024-04-16T17:14:05.633 INFO:tasks.workunit.client.0.smithi018.stdout: 1 352137 3.69 MB/sec execute 378 sec latency 125.252 ms 2024-04-16T17:14:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:05 smithi018 ceph-mon[31776]: pgmap v291: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 830 KiB/s wr, 132 op/s 2024-04-16T17:14:05.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:05 smithi018 ceph-mon[31776]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T17:14:05.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:05 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:14:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:05 smithi022 ceph-mon[36023]: pgmap v291: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 830 KiB/s wr, 132 op/s 2024-04-16T17:14:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:05 smithi022 ceph-mon[36023]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T17:14:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:05 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:14:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:05 smithi043 ceph-mon[36901]: pgmap v291: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 830 KiB/s wr, 132 op/s 2024-04-16T17:14:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:05 smithi043 ceph-mon[36901]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T17:14:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:05 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:14:06.633 INFO:tasks.workunit.client.0.smithi018.stdout: 1 352336 3.69 MB/sec execute 379 sec latency 147.594 ms 2024-04-16T17:14:07.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 352611 3.69 MB/sec execute 380 sec latency 129.595 ms 2024-04-16T17:14:07.835 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:07 smithi018 ceph-mon[31776]: pgmap v293: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 195 op/s 2024-04-16T17:14:07.835 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:07 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:14:07.835 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:07 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:14:07.835 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:07 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:14:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:07 smithi022 ceph-mon[36023]: pgmap v293: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 195 op/s 2024-04-16T17:14:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:07 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:14:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:07 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:14:07.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:07 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:14:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:07 smithi043 ceph-mon[36901]: pgmap v293: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 195 op/s 2024-04-16T17:14:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:07 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:14:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:07 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:14:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:07 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:14:08.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 353086 3.68 MB/sec execute 381 sec latency 74.712 ms 2024-04-16T17:14:09.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 354078 3.69 MB/sec execute 382 sec latency 26.473 ms 2024-04-16T17:14:09.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:09 smithi022 ceph-mon[36023]: pgmap v294: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 195 op/s 2024-04-16T17:14:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:09 smithi043 ceph-mon[36901]: pgmap v294: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 195 op/s 2024-04-16T17:14:10.554 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:09 smithi018 ceph-mon[31776]: pgmap v294: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.3 MiB/s wr, 195 op/s 2024-04-16T17:14:10.633 INFO:tasks.workunit.client.0.smithi018.stdout: 1 354972 3.69 MB/sec execute 383 sec latency 14.556 ms 2024-04-16T17:14:11.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 355235 3.68 MB/sec execute 384 sec latency 97.282 ms 2024-04-16T17:14:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:11 smithi022 ceph-mon[36023]: pgmap v295: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:14:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:11 smithi043 ceph-mon[36901]: pgmap v295: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:14:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.051+0000 7f6b49e00640 1 -- 172.21.15.18:0/1553292621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44104470 msgr2=0x7f6b44104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.051+0000 7f6b49e00640 1 --2- 172.21.15.18:0/1553292621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44104470 0x7f6b44104850 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009860 tx=0x7f6b3402bb70 comp rx=0 tx=0).stop 2024-04-16T17:14:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.051+0000 7f6b49e00640 1 -- 172.21.15.18:0/1553292621 shutdown_connections 2024-04-16T17:14:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.051+0000 7f6b49e00640 1 --2- 172.21.15.18:0/1553292621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b44109540 0x7f6b44111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.051+0000 7f6b49e00640 1 --2- 172.21.15.18:0/1553292621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44104e20 0x7f6b44108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.051+0000 7f6b49e00640 1 --2- 172.21.15.18:0/1553292621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44104470 0x7f6b44104850 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.051+0000 7f6b49e00640 1 -- 172.21.15.18:0/1553292621 >> 172.21.15.18:0/1553292621 conn(0x7f6b440ffcf0 msgr2=0x7f6b44102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.051+0000 7f6b49e00640 1 -- 172.21.15.18:0/1553292621 shutdown_connections 2024-04-16T17:14:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.052+0000 7f6b49e00640 1 -- 172.21.15.18:0/1553292621 wait complete. 2024-04-16T17:14:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.053+0000 7f6b49e00640 1 Processor -- start 2024-04-16T17:14:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.053+0000 7f6b49e00640 1 -- start start 2024-04-16T17:14:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b49e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b44104470 0x7f6b441a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b49e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44109540 0x7f6b441a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b49e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4419aa10 0x7f6b4419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b49e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441145d0 con 0x7f6b44104470 2024-04-16T17:14:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b49e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44114450 con 0x7f6b44109540 2024-04-16T17:14:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b49e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44114750 con 0x7f6b4419aa10 2024-04-16T17:14:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b42000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44109540 0x7f6b441a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b44104470 0x7f6b441a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b42000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44109540 0x7f6b441a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55602/0 (socket says 172.21.15.18:55602) 2024-04-16T17:14:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b42000640 1 -- 172.21.15.18:0/90654074 learned_addr learned my addr 172.21.15.18:0/90654074 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b43400640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4419aa10 0x7f6b4419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.054+0000 7f6b42000640 1 -- 172.21.15.18:0/90654074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4419aa10 msgr2=0x7f6b4419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b42000640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4419aa10 0x7f6b4419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b42000640 1 -- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b44104470 msgr2=0x7f6b441a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b42000640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b44104470 0x7f6b441a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b42000640 1 -- 172.21.15.18:0/90654074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4419b530 con 0x7f6b44109540 2024-04-16T17:14:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b43400640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4419aa10 0x7f6b4419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b42000640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44109540 0x7f6b441a0d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000e9f0 tx=0x7f6b3000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b48a00640 1 -- 172.21.15.18:0/90654074 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3000ccc0 con 0x7f6b44109540 2024-04-16T17:14:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b48a00640 1 -- 172.21.15.18:0/90654074 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b30010910 con 0x7f6b44109540 2024-04-16T17:14:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b48a00640 1 -- 172.21.15.18:0/90654074 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b300188c0 con 0x7f6b44109540 2024-04-16T17:14:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.055+0000 7f6b49e00640 1 -- 172.21.15.18:0/90654074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4419b820 con 0x7f6b44109540 2024-04-16T17:14:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.056+0000 7f6b49e00640 1 -- 172.21.15.18:0/90654074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b441a79d0 con 0x7f6b44109540 2024-04-16T17:14:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b4cbaf740 1 Processor -- start 2024-04-16T17:14:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b4cbaf740 1 -- start start 2024-04-16T17:14:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b48a00640 1 -- 172.21.15.18:0/90654074 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6b30010ac0 con 0x7f6b44109540 2024-04-16T17:14:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c16610 0x557522c15360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c15d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c21210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557522b7a780 con 0x557522c16610 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557522b20f40 con 0x557522c1ee20 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.061+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557522b76740 con 0x557522c158a0 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b43400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c16610 0x557522c15360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b43400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c16610 0x557522c15360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59274/0 (socket says 172.21.15.18:59274) 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b43400640 1 -- 172.21.15.18:0/2773857608 learned_addr learned my addr 172.21.15.18:0/2773857608 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b42000640 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c21210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b42a00640 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c15d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b43400640 1 -- 172.21.15.18:0/2773857608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 msgr2=0x557522c15d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b43400640 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c15d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b43400640 1 -- 172.21.15.18:0/2773857608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 msgr2=0x557522c21210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b43400640 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c21210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b43400640 1 -- 172.21.15.18:0/2773857608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55752288d210 con 0x557522c16610 2024-04-16T17:14:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.062+0000 7f6b42a00640 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c15d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.063+0000 7f6b43400640 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c16610 0x557522c15360 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c011790 tx=0x7f6b3c011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e643aaa42fe4fdbc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.063+0000 7f6b48a00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6b280a3d90 0x7f6b280a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.063+0000 7f6b48a00640 1 -- 172.21.15.18:0/90654074 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9642+0+0 (secure 0 0 0) 0x7f6b300d0e90 con 0x7f6b44109540 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.063+0000 7f6b2fe00640 1 -- 172.21.15.18:0/2773857608 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004d20 con 0x557522c16610 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.063+0000 7f6b2fe00640 1 -- 172.21.15.18:0/2773857608 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3c003970 con 0x557522c16610 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.063+0000 7f6b2fe00640 1 -- 172.21.15.18:0/2773857608 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004500 con 0x557522c16610 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.064+0000 7f6b42a00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6b280a3d90 0x7f6b280a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.064+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/2773857608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c16610 msgr2=0x557522c15360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.064+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c16610 0x557522c15360 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c011790 tx=0x7f6b3c011c50 comp rx=0 tx=0).stop 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.064+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/2773857608 shutdown_connections 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.064+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c21210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.064+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c15d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.064+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/2773857608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c16610 0x557522c15360 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.064+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/2773857608 >> 172.21.15.18:0/2773857608 conn(0x557522bb05d0 msgr2=0x557522c10590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.065+0000 7f6b42a00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6b280a3d90 0x7f6b280a6250 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009830 tx=0x7f6b340056d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.066+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/2773857608 shutdown_connections 2024-04-16T17:14:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.066+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/2773857608 wait complete. 2024-04-16T17:14:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.066+0000 7f6b4cbaf740 1 -- start start 2024-04-16T17:14:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.067+0000 7f6b4cbaf740 1 Processor -- start 2024-04-16T17:14:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.067+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c1e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.067+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 0x557522c1bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c1c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557522c21b50 con 0x557522c158a0 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557522b34390 con 0x557522c1ee20 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557522c21cd0 con 0x557522c16610 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c1e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b4cbaf740 10 client.0 _ll_register_callbacks cb 0x557522b1df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c1e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59290/0 (socket says 172.21.15.18:59290) 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42a00640 1 -- 172.21.15.18:0/3277702201 learned_addr learned my addr 172.21.15.18:0/3277702201 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42000640 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 0x557522c1bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b43400640 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c1c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42a00640 1 -- 172.21.15.18:0/3277702201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 msgr2=0x557522c1bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42a00640 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 0x557522c1bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42a00640 1 -- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 msgr2=0x557522c1c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42a00640 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c1c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b42a00640 1 -- 172.21.15.18:0/3277702201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557522959620 con 0x557522c158a0 2024-04-16T17:14:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.068+0000 7f6b43400640 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c1c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.069+0000 7f6b42a00640 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c1e8d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009860 tx=0x7f6b3402bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6188108fd1cfe444 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.069+0000 7f6b2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:14:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.069+0000 7f6b2cc00640 1 -- 172.21.15.18:0/3277702201 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3404d020 con 0x557522c158a0 2024-04-16T17:14:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.069+0000 7f6b2cc00640 1 -- 172.21.15.18:0/3277702201 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9642+0+0 (secure 0 0 0) 0x7f6b3402c710 con 0x557522c158a0 2024-04-16T17:14:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.069+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/3277702201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b3c003970 con 0x557522c158a0 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.069+0000 7f6b2cc00640 1 -- 172.21.15.18:0/3277702201 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b34058050 con 0x557522c158a0 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b4cbaf740 10 client.17859 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/3277702201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557522c36240 con 0x557522c158a0 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b2cc00640 1 -- 172.21.15.18:0/3277702201 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3c003970 con 0x557522c158a0 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b2cc00640 1 -- 172.21.15.18:0/3277702201 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6b340035f0 con 0x557522c158a0 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b4cbaf740 10 client.17859 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b4cbaf740 10 client.17859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b4cbaf740 20 client.17859 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557522c37750 0x557522c39b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b4cbaf740 4 client.17859 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.070+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/3277702201 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557522b881a0 con 0x557522c37750 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.071+0000 7f6b42000640 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557522c37750 0x557522c39b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.071+0000 7f6b42000640 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557522c37750 0x557522c39b40 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38d4cda352ecc07 server_cookie=935362dd41b44f31 in_seq=0 out_seq=0 2024-04-16T17:14:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.071+0000 7f6b2cc00640 10 client.17859 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b2cc00640 1 -- 172.21.15.18:0/3277702201 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557522b881a0 con 0x557522c37750 2024-04-16T17:14:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b2cc00640 10 client.17859 handle_command_reply: tid=0 2024-04-16T17:14:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 client.17859 shutdown 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/3277702201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 msgr2=0x557522c1e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c1e8d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009860 tx=0x7f6b3402bb70 comp rx=0 tx=0).stop 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/3277702201 shutdown_connections 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557522c37750 0x557522c39b40 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c1c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 0x557522c1bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/3277702201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c1e8d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/3277702201 >> 172.21.15.18:0/3277702201 conn(0x557522bb05d0 msgr2=0x557522c122e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b2cc00640 0 client.17859 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.072+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/3277702201 shutdown_connections 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.073+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/3277702201 wait complete. 2024-04-16T17:14:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.073+0000 7f6b4cbaf740 20 client.17859 trim_cache size 0 max 16384 2024-04-16T17:14:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.092+0000 7f6b4cbaf740 1 Processor -- start 2024-04-16T17:14:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.092+0000 7f6b4cbaf740 1 -- start start 2024-04-16T17:14:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.092+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c3f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.092+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 0x557522c3f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.092+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c18ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.092+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441aff20 con 0x557522c158a0 2024-04-16T17:14:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.092+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441ad7f0 con 0x557522c1ee20 2024-04-16T17:14:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.092+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441ab680 con 0x557522c16610 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c3f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c18ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b42000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 0x557522c3f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c18ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55624/0 (socket says 172.21.15.18:55624) 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 -- 172.21.15.18:0/717938183 learned_addr learned my addr 172.21.15.18:0/717938183 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 -- 172.21.15.18:0/717938183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 msgr2=0x557522c3f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 --2- 172.21.15.18:0/717938183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 0x557522c3f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 -- 172.21.15.18:0/717938183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 msgr2=0x557522c3f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 --2- 172.21.15.18:0/717938183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c3f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 -- 172.21.15.18:0/717938183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557522c14b70 con 0x557522c1ee20 2024-04-16T17:14:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.093+0000 7f6b43400640 1 --2- 172.21.15.18:0/717938183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c18ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009540 tx=0x7f6b3c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf14475abc63548f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.094+0000 7f6b2f400640 1 -- 172.21.15.18:0/717938183 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c027020 con 0x557522c1ee20 2024-04-16T17:14:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.094+0000 7f6b2f400640 1 -- 172.21.15.18:0/717938183 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3c01e040 con 0x557522c1ee20 2024-04-16T17:14:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.094+0000 7f6b2f400640 1 -- 172.21.15.18:0/717938183 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c01d070 con 0x557522c1ee20 2024-04-16T17:14:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.095+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/717938183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 msgr2=0x557522c18ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.095+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/717938183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c18ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009540 tx=0x7f6b3c000f40 comp rx=0 tx=0).stop 2024-04-16T17:14:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.095+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/717938183 shutdown_connections 2024-04-16T17:14:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.095+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/717938183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c1ee20 0x557522c18ec0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.095+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/717938183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c16610 0x557522c3f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.095+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/717938183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c158a0 0x557522c3f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.095+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/717938183 >> 172.21.15.18:0/717938183 conn(0x557522bb05d0 msgr2=0x557522c1a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.095+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/717938183 shutdown_connections 2024-04-16T17:14:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.096+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/717938183 wait complete. 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.096+0000 7f6b4cbaf740 1 -- start start 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.096+0000 7f6b4cbaf740 1 Processor -- start 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c2b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c16610 0x557522c2bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b4cbaf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c1ee20 0x557522c9cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441abcd0 con 0x557522c1ee20 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441ad120 con 0x557522c16610 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b4cbaf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441ac380 con 0x557522c158a0 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b4cbaf740 10 client.0 _ll_register_callbacks cb 0x7f6b3c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c16610 0x557522c2bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c16610 0x557522c2bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55632/0 (socket says 172.21.15.18:55632) 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42000640 1 -- 172.21.15.18:0/1177250093 learned_addr learned my addr 172.21.15.18:0/1177250093 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42a00640 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b43400640 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c1ee20 0x557522c9cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42a00640 1 -- 172.21.15.18:0/1177250093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c16610 msgr2=0x557522c2bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42a00640 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c16610 0x557522c2bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42a00640 1 -- 172.21.15.18:0/1177250093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c1ee20 msgr2=0x557522c9cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42a00640 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c1ee20 0x557522c9cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.097+0000 7f6b42a00640 1 -- 172.21.15.18:0/1177250093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557522c12890 con 0x557522c158a0 2024-04-16T17:14:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.098+0000 7f6b42a00640 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c2b600 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402c0b0 tx=0x7f6b34003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca7ddba46fca0b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.098+0000 7f6b2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.098+0000 7f6b2cc00640 1 -- 172.21.15.18:0/1177250093 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3404d040 con 0x557522c158a0 2024-04-16T17:14:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.098+0000 7f6b2cc00640 1 -- 172.21.15.18:0/1177250093 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9642+0+0 (secure 0 0 0) 0x7f6b3402c510 con 0x557522c158a0 2024-04-16T17:14:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.098+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/1177250093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b3c01e040 con 0x557522c158a0 2024-04-16T17:14:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.098+0000 7f6b2cc00640 1 -- 172.21.15.18:0/1177250093 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b3406d050 con 0x557522c158a0 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b4cbaf740 10 client.28042 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/1177250093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557522c9e160 con 0x557522c158a0 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b2cc00640 1 -- 172.21.15.18:0/1177250093 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3c01e040 con 0x557522c158a0 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b2cc00640 1 -- 172.21.15.18:0/1177250093 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6b34032ce0 con 0x557522c158a0 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b4cbaf740 10 client.28042 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b4cbaf740 10 client.28042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b4cbaf740 20 client.28042 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557522c9e850 0x557522ca0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b4cbaf740 4 client.28042 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.099+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/1177250093 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557522b82010 con 0x557522c9e850 2024-04-16T17:14:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.100+0000 7f6b42000640 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557522c9e850 0x557522ca0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.100+0000 7f6b42000640 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557522c9e850 0x557522ca0c70 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4387e89813f343 server_cookie=a646ef06dcd42320 in_seq=0 out_seq=0 2024-04-16T17:14:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.100+0000 7f6b2cc00640 10 client.28042 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.220+0000 7f6b2cc00640 1 -- 172.21.15.18:0/1177250093 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557522b82010 con 0x557522c9e850 2024-04-16T17:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.220+0000 7f6b2cc00640 10 client.28042 handle_command_reply: tid=0 2024-04-16T17:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.220+0000 7f6b4cbaf740 1 client.28042 shutdown 2024-04-16T17:14:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/1177250093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 msgr2=0x557522c2b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c2b600 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402c0b0 tx=0x7f6b34003820 comp rx=0 tx=0).stop 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/1177250093 shutdown_connections 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557522c9e850 0x557522ca0c70 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557522c1ee20 0x557522c9cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557522c16610 0x557522c2bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 --2- 172.21.15.18:0/1177250093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557522c158a0 0x557522c2b600 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b2cc00640 0 client.28042 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/1177250093 >> 172.21.15.18:0/1177250093 conn(0x557522bb05d0 msgr2=0x557522c117c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/1177250093 shutdown_connections 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 1 -- 172.21.15.18:0/1177250093 wait complete. 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.223+0000 7f6b4cbaf740 20 client.28042 trim_cache size 0 max 16384 2024-04-16T17:14:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:14:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:14:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:14:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:14:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 -- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6b280a3d90 msgr2=0x7f6b280a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6b280a3d90 0x7f6b280a6250 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009830 tx=0x7f6b340056d0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 -- 172.21.15.18:0/90654074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44109540 msgr2=0x7f6b441a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44109540 0x7f6b441a0d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000e9f0 tx=0x7f6b3000edf0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 -- 172.21.15.18:0/90654074 shutdown_connections 2024-04-16T17:14:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6b280a3d90 0x7f6b280a6250 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4419aa10 0x7f6b4419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b44109540 0x7f6b441a0d50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 --2- 172.21.15.18:0/90654074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b44104470 0x7f6b441a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 -- 172.21.15.18:0/90654074 >> 172.21.15.18:0/90654074 conn(0x7f6b440ffcf0 msgr2=0x7f6b44101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 -- 172.21.15.18:0/90654074 shutdown_connections 2024-04-16T17:14:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:12.224+0000 7f6b49e00640 1 -- 172.21.15.18:0/90654074 wait complete. 2024-04-16T17:14:12.234 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:11 smithi018 ceph-mon[31776]: pgmap v295: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:14:12.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 356153 3.69 MB/sec execute 385 sec latency 77.368 ms 2024-04-16T17:14:13.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 356843 3.68 MB/sec execute 386 sec latency 41.559 ms 2024-04-16T17:14:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:13 smithi022 ceph-mon[36023]: pgmap v296: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2024-04-16T17:14:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:13 smithi022 ceph-mon[36023]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T17:14:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:13 smithi043 ceph-mon[36901]: pgmap v296: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2024-04-16T17:14:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:13 smithi043 ceph-mon[36901]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T17:14:14.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 357973 3.69 MB/sec execute 387 sec latency 21.126 ms 2024-04-16T17:14:15.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:13 smithi018 ceph-mon[31776]: pgmap v296: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2024-04-16T17:14:15.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:13 smithi018 ceph-mon[31776]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T17:14:15.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 358686 3.70 MB/sec execute 388 sec latency 25.244 ms 2024-04-16T17:14:15.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:15 smithi022 ceph-mon[36023]: pgmap v298: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-04-16T17:14:16.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:15 smithi043 ceph-mon[36901]: pgmap v298: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-04-16T17:14:16.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 359004 3.69 MB/sec execute 389 sec latency 74.389 ms 2024-04-16T17:14:17.063 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:15 smithi018 ceph-mon[31776]: pgmap v298: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-04-16T17:14:17.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 359779 3.69 MB/sec execute 390 sec latency 191.525 ms 2024-04-16T17:14:17.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:17 smithi022 ceph-mon[36023]: pgmap v299: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-04-16T17:14:18.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:17 smithi043 ceph-mon[36901]: pgmap v299: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-04-16T17:14:18.634 INFO:tasks.workunit.client.0.smithi018.stdout: 1 360658 3.69 MB/sec execute 391 sec latency 31.724 ms 2024-04-16T17:14:18.731 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:17 smithi018 ceph-mon[31776]: pgmap v299: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-04-16T17:14:19.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 361951 3.70 MB/sec execute 392 sec latency 14.596 ms 2024-04-16T17:14:19.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:19 smithi022 ceph-mon[36023]: pgmap v300: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-04-16T17:14:20.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:19 smithi043 ceph-mon[36901]: pgmap v300: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-04-16T17:14:20.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 362531 3.70 MB/sec execute 393 sec latency 27.357 ms 2024-04-16T17:14:21.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 362862 3.69 MB/sec execute 394 sec latency 74.824 ms 2024-04-16T17:14:21.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:21 smithi022 ceph-mon[36023]: pgmap v301: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-04-16T17:14:22.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:21 smithi043 ceph-mon[36901]: pgmap v301: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-04-16T17:14:22.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 363201 3.69 MB/sec execute 395 sec latency 114.300 ms 2024-04-16T17:14:23.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 363882 3.68 MB/sec execute 396 sec latency 54.662 ms 2024-04-16T17:14:23.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:23 smithi022 ceph-mon[36023]: pgmap v302: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-04-16T17:14:24.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:23 smithi043 ceph-mon[36901]: pgmap v302: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-04-16T17:14:24.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 365094 3.70 MB/sec execute 397 sec latency 23.171 ms 2024-04-16T17:14:25.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:14:25.144 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 -- 172.21.15.18:0/2267422484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 msgr2=0x7f5924104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 --2- 172.21.15.18:0/2267422484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 0x7f5924104680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5914009900 tx=0x7f591402bc40 comp rx=0 tx=0).stop 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 -- 172.21.15.18:0/2267422484 shutdown_connections 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 --2- 172.21.15.18:0/2267422484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5924109370 0x7f5924110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 --2- 172.21.15.18:0/2267422484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5924104c50 0x7f5924108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 --2- 172.21.15.18:0/2267422484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 0x7f5924104680 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 -- 172.21.15.18:0/2267422484 >> 172.21.15.18:0/2267422484 conn(0x7f59240ffb00 msgr2=0x7f5924101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 -- 172.21.15.18:0/2267422484 shutdown_connections 2024-04-16T17:14:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.363+0000 7f5923200640 1 -- 172.21.15.18:0/2267422484 wait complete. 2024-04-16T17:14:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.364+0000 7f5923200640 1 Processor -- start 2024-04-16T17:14:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5923200640 1 -- start start 2024-04-16T17:14:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5923200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 0x7f5924110cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5923200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5924109370 0x7f592410ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5923200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f592410f350 0x7f592410f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f2bdea00640 1 -- 172.21.15.18:0/1076153068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104e00 msgr2=0x7f2bd8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f2bdea00640 1 --2- 172.21.15.18:0/1076153068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104e00 0x7f2bd8108e40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009900 tx=0x7f2bc402bc40 comp rx=0 tx=0).stop 2024-04-16T17:14:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5923200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5924076ba0 con 0x7f592410f350 2024-04-16T17:14:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5923200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5924076a20 con 0x7f5924109370 2024-04-16T17:14:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5923200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5924076d20 con 0x7f59241042a0 2024-04-16T17:14:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5921e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 0x7f5924110cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5921e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 0x7f5924110cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41496/0 (socket says 172.21.15.18:41496) 2024-04-16T17:14:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f5921e00640 1 -- 172.21.15.18:0/2761331802 learned_addr learned my addr 172.21.15.18:0/2761331802 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f5921400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5924109370 0x7f592410ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f2bdea00640 1 -- 172.21.15.18:0/1076153068 shutdown_connections 2024-04-16T17:14:25.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f2bdea00640 1 --2- 172.21.15.18:0/1076153068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd8109520 0x7f2bd81110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f2bdea00640 1 --2- 172.21.15.18:0/1076153068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104e00 0x7f2bd8108e40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.365+0000 7f2bdea00640 1 --2- 172.21.15.18:0/1076153068 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd8104450 0x7f2bd8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f2bdea00640 1 -- 172.21.15.18:0/1076153068 >> 172.21.15.18:0/1076153068 conn(0x7f2bd80ffc90 msgr2=0x7f2bd81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f2bdea00640 1 -- 172.21.15.18:0/1076153068 shutdown_connections 2024-04-16T17:14:25.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f2bdea00640 1 -- 172.21.15.18:0/1076153068 wait complete. 2024-04-16T17:14:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f5921e00640 1 -- 172.21.15.18:0/2761331802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5924109370 msgr2=0x7f592410ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f5921e00640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5924109370 0x7f592410ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f5921e00640 1 -- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f592410f350 msgr2=0x7f592410f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f5921e00640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f592410f350 0x7f592410f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f5921e00640 1 -- 172.21.15.18:0/2761331802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59241ab410 con 0x7f59241042a0 2024-04-16T17:14:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.366+0000 7f5921e00640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 0x7f5924110cc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5914005ab0 tx=0x7f59140042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.367+0000 7f591aa00640 1 -- 172.21.15.18:0/2761331802 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914041070 con 0x7f59241042a0 2024-04-16T17:14:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.367+0000 7f591aa00640 1 -- 172.21.15.18:0/2761331802 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591402cca0 con 0x7f59241042a0 2024-04-16T17:14:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.367+0000 7f5923200640 1 -- 172.21.15.18:0/2761331802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59241ab6a0 con 0x7f59241042a0 2024-04-16T17:14:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.367+0000 7f591aa00640 1 -- 172.21.15.18:0/2761331802 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914003d90 con 0x7f59241042a0 2024-04-16T17:14:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.367+0000 7f5923200640 1 -- 172.21.15.18:0/2761331802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59241abbe0 con 0x7f59241042a0 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.367+0000 7f2bdea00640 1 Processor -- start 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.367+0000 7f2bdea00640 1 -- start start 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.368+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104450 0x7f2bd81a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.368+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd8109520 0x7f2bd81a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd819a920 0x7f2bd819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdea00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81145e0 con 0x7f2bd8109520 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8114460 con 0x7f2bd819a920 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8114760 con 0x7f2bd8104450 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd819a920 0x7f2bd819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd819a920 0x7f2bd819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54854/0 (socket says 172.21.15.18:54854) 2024-04-16T17:14:25.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdca00640 1 -- 172.21.15.18:0/2407888122 learned_addr learned my addr 172.21.15.18:0/2407888122 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104450 0x7f2bd81a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdca00640 1 -- 172.21.15.18:0/2407888122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104450 msgr2=0x7f2bd81a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdca00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104450 0x7f2bd81a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bdca00640 1 -- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd8109520 msgr2=0x7f2bd81a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.369+0000 7f2bd7400640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd8109520 0x7f2bd81a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.370+0000 7f2bdca00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd8109520 0x7f2bd81a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.370+0000 7f2bdca00640 1 -- 172.21.15.18:0/2407888122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd819b440 con 0x7f2bd819a920 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.370+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104450 0x7f2bd81a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.370+0000 7f2bd7400640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd8109520 0x7f2bd81a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.370+0000 7f2bdca00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd819a920 0x7f2bd819add0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e970 tx=0x7f2bc800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.370+0000 7f2bd4c00640 1 -- 172.21.15.18:0/2407888122 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ccc0 con 0x7f2bd819a920 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.370+0000 7f2bd4c00640 1 -- 172.21.15.18:0/2407888122 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc800ce60 con 0x7f2bd819a920 2024-04-16T17:14:25.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.370+0000 7f2bd4c00640 1 -- 172.21.15.18:0/2407888122 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8010690 con 0x7f2bd819a920 2024-04-16T17:14:25.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.371+0000 7f2bdea00640 1 -- 172.21.15.18:0/2407888122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd819b730 con 0x7f2bd819a920 2024-04-16T17:14:25.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.371+0000 7f2bdea00640 1 -- 172.21.15.18:0/2407888122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81a7930 con 0x7f2bd819a920 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.371+0000 7f592ba1b740 1 Processor -- start 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.372+0000 7f592ba1b740 1 -- start start 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.372+0000 7f591aa00640 1 -- 172.21.15.18:0/2761331802 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5914044070 con 0x7f59241042a0 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.372+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c67f0 0x55e7063c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.372+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.372+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063cf000 0x55e7063d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.372+0000 7f592ba1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e70632a780 con 0x55e7063cf000 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.372+0000 7f592ba1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e7062d0f40 con 0x55e7063c67f0 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.373+0000 7f592ba1b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e706326740 con 0x55e7063c5a80 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.373+0000 7f5922800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c67f0 0x55e7063c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.373+0000 7f5922800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c67f0 0x55e7063c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54858/0 (socket says 172.21.15.18:54858) 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.373+0000 7f5922800640 1 -- 172.21.15.18:0/3552372801 learned_addr learned my addr 172.21.15.18:0/3552372801 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.373+0000 7f5921400640 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063cf000 0x55e7063d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.374+0000 7f5922800640 1 -- 172.21.15.18:0/3552372801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 msgr2=0x55e7063c5f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.374+0000 7f5922800640 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063c5f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.374+0000 7f5922800640 1 -- 172.21.15.18:0/3552372801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063cf000 msgr2=0x55e7063d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.374+0000 7f5922800640 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063cf000 0x55e7063d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.374+0000 7f5922800640 1 -- 172.21.15.18:0/3552372801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70603d210 con 0x55e7063c67f0 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.374+0000 7f5921400640 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063cf000 0x55e7063d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.374+0000 7f5922800640 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c67f0 0x55e7063c5540 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f591c005c40 tx=0x7f591c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559825e59960c33c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.374+0000 7f591aa00640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59080a3e00 0x7f59080a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.375+0000 7f591aa00640 1 -- 172.21.15.18:0/2761331802 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f59140f2520 con 0x7f59241042a0 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.375+0000 7f590f000640 1 -- 172.21.15.18:0/3552372801 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c003b70 con 0x55e7063c67f0 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.375+0000 7f590f000640 1 -- 172.21.15.18:0/3552372801 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c003d10 con 0x55e7063c67f0 2024-04-16T17:14:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.375+0000 7f590f000640 1 -- 172.21.15.18:0/3552372801 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0109d0 con 0x55e7063c67f0 2024-04-16T17:14:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.375+0000 7f5921e00640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59080a3e00 0x7f59080a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f592ba1b740 1 -- 172.21.15.18:0/3552372801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c67f0 msgr2=0x55e7063c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f592ba1b740 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c67f0 0x55e7063c5540 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f591c005c40 tx=0x7f591c00e720 comp rx=0 tx=0).stop 2024-04-16T17:14:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f5921e00640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59080a3e00 0x7f59080a62c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55e7063c6f50 tx=0x7f5914009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f2bd4c00640 1 -- 172.21.15.18:0/2407888122 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bc80046c0 con 0x7f2bd819a920 2024-04-16T17:14:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f592ba1b740 1 -- 172.21.15.18:0/3552372801 shutdown_connections 2024-04-16T17:14:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f592ba1b740 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063cf000 0x55e7063d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f592ba1b740 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f592ba1b740 1 --2- 172.21.15.18:0/3552372801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c67f0 0x55e7063c5540 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.376+0000 7f592ba1b740 1 -- 172.21.15.18:0/3552372801 >> 172.21.15.18:0/3552372801 conn(0x55e7063605d0 msgr2=0x55e7063c0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.377+0000 7f592ba1b740 1 -- 172.21.15.18:0/3552372801 shutdown_connections 2024-04-16T17:14:25.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.377+0000 7f2be17e5740 1 Processor -- start 2024-04-16T17:14:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.377+0000 7f592ba1b740 1 -- 172.21.15.18:0/3552372801 wait complete. 2024-04-16T17:14:25.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.377+0000 7f2be17e5740 1 -- start start 2024-04-16T17:14:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.377+0000 7f592ba1b740 1 -- start start 2024-04-16T17:14:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f592ba1b740 1 Processor -- start 2024-04-16T17:14:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d9672fb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672fbec0 0x55d9672fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f2be17e5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d967250120 con 0x55d9672f29c0 2024-04-16T17:14:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f2be17e5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d9671f6340 con 0x55d9672f32e0 2024-04-16T17:14:25.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f2be17e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d96724c0e0 con 0x55d9672fbec0 2024-04-16T17:14:25.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f2bd4c00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f2bd4c00640 1 -- 172.21.15.18:0/2407888122 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f2bc80d1150 con 0x7f2bd819a920 2024-04-16T17:14:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.378+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f592ba1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e7063e2d40 con 0x55e7063c67f0 2024-04-16T17:14:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f592ba1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e7062e4390 con 0x55e7063cf000 2024-04-16T17:14:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f592ba1b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e7063e2ec0 con 0x55e7063c5a80 2024-04-16T17:14:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f592ba1b740 10 client.0 _ll_register_callbacks cb 0x55e7062cdf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f5922800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f5922800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41514/0 (socket says 172.21.15.18:41514) 2024-04-16T17:14:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f5922800640 1 -- 172.21.15.18:0/256038445 learned_addr learned my addr 172.21.15.18:0/256038445 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f5922800640 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d9672fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d9672fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54870/0 (socket says 172.21.15.18:54870) 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7e00640 1 -- 172.21.15.18:0/1288006677 learned_addr learned my addr 172.21.15.18:0/1288006677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7400640 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672fbec0 0x55d9672fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7400640 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bdca00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7e00640 1 -- 172.21.15.18:0/1288006677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672fbec0 msgr2=0x55d9672fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672fbec0 0x55d9672fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7e00640 1 -- 172.21.15.18:0/1288006677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 msgr2=0x55d9672f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.379+0000 7f2bd7e00640 1 -- 172.21.15.18:0/1288006677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96702c7d0 con 0x55d9672f32e0 2024-04-16T17:14:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f5921400640 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f5922800640 1 -- 172.21.15.18:0/256038445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 msgr2=0x55e7063cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f5922800640 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f5922800640 1 -- 172.21.15.18:0/256038445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 msgr2=0x55e7063cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d9672fb980 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000f390 tx=0x7f2bd000f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346988daa78a17b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f5922800640 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f2bdca00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80990e0 tx=0x7f2bc80049b0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f2bcd200640 1 -- 172.21.15.18:0/1288006677 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0022710 con 0x55d9672f32e0 2024-04-16T17:14:25.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f2bcd200640 1 -- 172.21.15.18:0/1288006677 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0022d90 con 0x55d9672f32e0 2024-04-16T17:14:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f5922800640 1 -- 172.21.15.18:0/256038445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e706109620 con 0x55e7063cf000 2024-04-16T17:14:25.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.380+0000 7f2bcd200640 1 -- 172.21.15.18:0/1288006677 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00213e0 con 0x55d9672f32e0 2024-04-16T17:14:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f5922800640 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063cc980 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f591c010670 tx=0x7f591c0098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b105024abbb7db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f5903400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f5903400640 1 -- 172.21.15.18:0/256038445 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c020040 con 0x55e7063cf000 2024-04-16T17:14:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f5903400640 1 -- 172.21.15.18:0/256038445 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f591c004d20 con 0x55e7063cf000 2024-04-16T17:14:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f592ba1b740 1 -- 172.21.15.18:0/256038445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591c003d10 con 0x55e7063cf000 2024-04-16T17:14:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f5903400640 1 -- 172.21.15.18:0/256038445 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f591c030050 con 0x55e7063cf000 2024-04-16T17:14:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f592ba1b740 10 client.28163 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f592ba1b740 1 -- 172.21.15.18:0/256038445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e7063eeef0 con 0x55e7063cf000 2024-04-16T17:14:25.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f2be17e5740 1 -- 172.21.15.18:0/1288006677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 msgr2=0x55d9672fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.381+0000 7f2be17e5740 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d9672fb980 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000f390 tx=0x7f2bd000f850 comp rx=0 tx=0).stop 2024-04-16T17:14:25.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f2be17e5740 1 -- 172.21.15.18:0/1288006677 shutdown_connections 2024-04-16T17:14:25.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f2be17e5740 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672fbec0 0x55d9672fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f2be17e5740 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d9672fb980 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f2be17e5740 1 --2- 172.21.15.18:0/1288006677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f2be17e5740 1 -- 172.21.15.18:0/1288006677 >> 172.21.15.18:0/1288006677 conn(0x55d9672ed0f0 msgr2=0x55d9672ed4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f2be17e5740 1 -- 172.21.15.18:0/1288006677 shutdown_connections 2024-04-16T17:14:25.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f2be17e5740 1 -- 172.21.15.18:0/1288006677 wait complete. 2024-04-16T17:14:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f5903400640 1 -- 172.21.15.18:0/256038445 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c003d10 con 0x55e7063cf000 2024-04-16T17:14:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f5903400640 1 -- 172.21.15.18:0/256038445 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f591c026070 con 0x55e7063cf000 2024-04-16T17:14:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f592ba1b740 10 client.28163 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f592ba1b740 10 client.28163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f592ba1b740 20 client.28163 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f592ba1b740 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e7063f0300 0x55e7063f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f592ba1b740 4 client.28163 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.382+0000 7f592ba1b740 1 -- 172.21.15.18:0/256038445 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7063381a0 con 0x55e7063f0300 2024-04-16T17:14:25.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 1 -- start start 2024-04-16T17:14:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f5921400640 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e7063f0300 0x55e7063f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 1 Processor -- start 2024-04-16T17:14:25.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672ea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f32e0 0x55d9672ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672fa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d9672e43f0 con 0x55d9672f29c0 2024-04-16T17:14:25.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d967209710 con 0x55d9672fbec0 2024-04-16T17:14:25.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d9672e4570 con 0x55d9672f32e0 2024-04-16T17:14:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f5921400640 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e7063f0300 0x55e7063f26f0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c51f573e04c10b4 server_cookie=e8e0fb1af18b27d1 in_seq=0 out_seq=0 2024-04-16T17:14:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f5903400640 10 client.28163 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:25.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.383+0000 7f2be17e5740 10 client.0 _ll_register_callbacks cb 0x55d967204af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:25.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672ea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f32e0 0x55d9672ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672ea170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58768/0 (socket says 172.21.15.18:58768) 2024-04-16T17:14:25.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f2bd7400640 1 -- 172.21.15.18:0/3452908621 learned_addr learned my addr 172.21.15.18:0/3452908621 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672fa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f5903400640 1 -- 172.21.15.18:0/256038445 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7063381a0 con 0x55e7063f0300 2024-04-16T17:14:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f5903400640 10 client.28163 handle_command_reply: tid=0 2024-04-16T17:14:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f592ba1b740 1 client.28163 shutdown 2024-04-16T17:14:25.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f2bd7e00640 1 -- 172.21.15.18:0/3452908621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 msgr2=0x55d9672fa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672fa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.384+0000 7f2bd7e00640 1 -- 172.21.15.18:0/3452908621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 msgr2=0x55d9672ea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 -- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 msgr2=0x55e7063cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063cc980 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f591c010670 tx=0x7f591c0098d0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672ea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f2bd7e00640 1 -- 172.21.15.18:0/3452908621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d966f623f0 con 0x55d9672f32e0 2024-04-16T17:14:25.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 -- 172.21.15.18:0/256038445 shutdown_connections 2024-04-16T17:14:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e7063f0300 0x55e7063f26f0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063cc980 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 --2- 172.21.15.18:0/256038445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 -- 172.21.15.18:0/256038445 >> 172.21.15.18:0/256038445 conn(0x55e7063605d0 msgr2=0x55e7063c24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f5903400640 0 client.28163 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:25.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f32e0 0x55d9672ea6b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0000f80 tx=0x7f2bd002c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39cbaed5ef8fab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f2bc1600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 -- 172.21.15.18:0/256038445 shutdown_connections 2024-04-16T17:14:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 1 -- 172.21.15.18:0/256038445 wait complete. 2024-04-16T17:14:25.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f2bc1600640 1 -- 172.21.15.18:0/3452908621 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0003d80 con 0x55d9672f32e0 2024-04-16T17:14:25.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f2be17e5740 1 -- 172.21.15.18:0/3452908621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd0022d90 con 0x55d9672f32e0 2024-04-16T17:14:25.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f2bc1600640 1 -- 172.21.15.18:0/3452908621 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f2bd0022510 con 0x55d9672f32e0 2024-04-16T17:14:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.385+0000 7f592ba1b740 20 client.28163 trim_cache size 0 max 16384 2024-04-16T17:14:25.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.386+0000 7f2bc1600640 1 -- 172.21.15.18:0/3452908621 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bd0044050 con 0x55d9672f32e0 2024-04-16T17:14:25.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.386+0000 7f2be17e5740 10 client.28060 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.386+0000 7f2be17e5740 1 -- 172.21.15.18:0/3452908621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d966f220f0 con 0x55d9672f32e0 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.386+0000 7f2bc1600640 1 -- 172.21.15.18:0/3452908621 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0022d90 con 0x55d9672f32e0 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.386+0000 7f2bc1600640 1 -- 172.21.15.18:0/3452908621 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2bd0004160 con 0x55d9672f32e0 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2be17e5740 10 client.28060 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2be17e5740 10 client.28060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2be17e5740 20 client.28060 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d967303a20 0x55d967305e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2be17e5740 4 client.28060 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2be17e5740 1 -- 172.21.15.18:0/3452908621 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96725db40 con 0x55d967303a20 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d967303a20 0x55d967305e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d967303a20 0x55d967305e10 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd075ded2935225d server_cookie=33a11d04de6caab4 in_seq=0 out_seq=0 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.387+0000 7f2bc1600640 10 client.28060 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.388+0000 7f2bc1600640 1 -- 172.21.15.18:0/3452908621 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d96725db40 con 0x55d967303a20 2024-04-16T17:14:25.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.388+0000 7f2bc1600640 10 client.28060 handle_command_reply: tid=0 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.388+0000 7f2be17e5740 1 client.28060 shutdown 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2be17e5740 1 -- 172.21.15.18:0/3452908621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f32e0 msgr2=0x55d9672ea6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f32e0 0x55d9672ea6b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0000f80 tx=0x7f2bd002c640 comp rx=0 tx=0).stop 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2be17e5740 1 -- 172.21.15.18:0/3452908621 shutdown_connections 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d967303a20 0x55d967305e10 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672fa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f32e0 0x55d9672ea6b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2bc1600640 0 client.28060 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3452908621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f29c0 0x55d9672ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.389+0000 7f2be17e5740 1 -- 172.21.15.18:0/3452908621 >> 172.21.15.18:0/3452908621 conn(0x55d9672ed0f0 msgr2=0x55d9672eee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.390+0000 7f2be17e5740 1 -- 172.21.15.18:0/3452908621 shutdown_connections 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.390+0000 7f2be17e5740 1 -- 172.21.15.18:0/3452908621 wait complete. 2024-04-16T17:14:25.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.390+0000 7f2be17e5740 20 client.28060 trim_cache size 0 max 16384 2024-04-16T17:14:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.420+0000 7f592ba1b740 1 Processor -- start 2024-04-16T17:14:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.420+0000 7f592ba1b740 1 -- start start 2024-04-16T17:14:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c5a80 0x55e7063ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f5922800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c5a80 0x55e7063ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f5921400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f5922800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c5a80 0x55e7063ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54898/0 (socket says 172.21.15.18:54898) 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f5922800640 1 -- 172.21.15.18:0/2089361957 learned_addr learned my addr 172.21.15.18:0/2089361957 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f5921400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58784/0 (socket says 172.21.15.18:58784) 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f592ba1b740 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063cf000 0x55e7063eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f592ba1b740 1 -- 172.21.15.18:0/2089361957 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f59241afbe0 con 0x55e7063c67f0 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f592ba1b740 1 -- 172.21.15.18:0/2089361957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59241ad4b0 con 0x55e7063c5a80 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.421+0000 7f592ba1b740 1 -- 172.21.15.18:0/2089361957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f592411b910 con 0x55e7063cf000 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.422+0000 7f5922800640 1 -- 172.21.15.18:0/2089361957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063cf000 msgr2=0x55e7063eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.422+0000 7f5922800640 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063cf000 0x55e7063eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.422+0000 7f5922800640 1 -- 172.21.15.18:0/2089361957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 msgr2=0x55e7063cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.422+0000 7f5922800640 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.422+0000 7f5922800640 1 -- 172.21.15.18:0/2089361957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7063c2a70 con 0x55e7063c5a80 2024-04-16T17:14:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.422+0000 7f5921400640 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.422+0000 7f5922800640 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c5a80 0x55e7063ce630 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f591c018860 tx=0x7f591c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1da466a3b20dc31 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.422+0000 7f590e600640 1 -- 172.21.15.18:0/2089361957 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c02d070 con 0x55e7063c5a80 2024-04-16T17:14:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.423+0000 7f590e600640 1 -- 172.21.15.18:0/2089361957 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c009480 con 0x55e7063c5a80 2024-04-16T17:14:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.423+0000 7f590e600640 1 -- 172.21.15.18:0/2089361957 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c003c80 con 0x55e7063c5a80 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 -- 172.21.15.18:0/2089361957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c5a80 msgr2=0x55e7063ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c5a80 0x55e7063ce630 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f591c018860 tx=0x7f591c009f70 comp rx=0 tx=0).stop 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 -- 172.21.15.18:0/2089361957 shutdown_connections 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063cf000 0x55e7063eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 --2- 172.21.15.18:0/2089361957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063c5a80 0x55e7063ce630 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 -- 172.21.15.18:0/2089361957 >> 172.21.15.18:0/2089361957 conn(0x55e7063605d0 msgr2=0x55e7063eb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 -- 172.21.15.18:0/2089361957 shutdown_connections 2024-04-16T17:14:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 -- 172.21.15.18:0/2089361957 wait complete. 2024-04-16T17:14:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.424+0000 7f592ba1b740 1 -- start start 2024-04-16T17:14:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.425+0000 7f592ba1b740 1 Processor -- start 2024-04-16T17:14:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.425+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063f5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.425+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.425+0000 7f592ba1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.425+0000 7f592ba1b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f592411bf60 con 0x55e7063c67f0 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.425+0000 7f592ba1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59241acde0 con 0x55e7063cf000 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.425+0000 7f592ba1b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f59241ac040 con 0x55e7063c5a80 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.425+0000 7f592ba1b740 10 client.0 _ll_register_callbacks cb 0x7f591c004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063f5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41540/0 (socket says 172.21.15.18:41540) 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 -- 172.21.15.18:0/1703952992 learned_addr learned my addr 172.21.15.18:0/1703952992 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5921400640 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 -- 172.21.15.18:0/1703952992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 msgr2=0x55e7063f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 -- 172.21.15.18:0/1703952992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 msgr2=0x55e7063f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063f2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5922800640 1 -- 172.21.15.18:0/1703952992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7063f3730 con 0x55e7063c5a80 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.426+0000 7f5921400640 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.427+0000 7f5922800640 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063f5850 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f591c018570 tx=0x7f591c012040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8c9a67c9a8ec82 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.427+0000 7f5903400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.427+0000 7f5903400640 1 -- 172.21.15.18:0/1703952992 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c02d070 con 0x55e7063c5a80 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.427+0000 7f592ba1b740 1 -- 172.21.15.18:0/1703952992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591c009480 con 0x55e7063c5a80 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.427+0000 7f5903400640 1 -- 172.21.15.18:0/1703952992 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f591c003c80 con 0x55e7063c5a80 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.427+0000 7f5903400640 1 -- 172.21.15.18:0/1703952992 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f591c03a040 con 0x55e7063c5a80 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.427+0000 7f592ba1b740 10 client.28063 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.427+0000 7f592ba1b740 1 -- 172.21.15.18:0/1703952992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e706456250 con 0x55e7063c5a80 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.428+0000 7f5903400640 1 -- 172.21.15.18:0/1703952992 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c009480 con 0x55e7063c5a80 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.428+0000 7f5903400640 1 -- 172.21.15.18:0/1703952992 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f591c040070 con 0x55e7063c5a80 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.428+0000 7f592ba1b740 10 client.28063 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.428+0000 7f592ba1b740 10 client.28063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.428+0000 7f592ba1b740 20 client.28063 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.428+0000 7f592ba1b740 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e706456850 0x55e706458c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.428+0000 7f592ba1b740 4 client.28063 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:14:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.428+0000 7f592ba1b740 1 -- 172.21.15.18:0/1703952992 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e706332010 con 0x55e706456850 2024-04-16T17:14:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.429+0000 7f5921400640 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e706456850 0x55e706458c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2be17e5740 1 Processor -- start 2024-04-16T17:14:25.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2be17e5740 1 -- start start 2024-04-16T17:14:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f5903400640 10 client.28063 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f5921400640 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e706456850 0x55e706458c70 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5008e04bad8b55d server_cookie=295e8de957f3ee15 in_seq=0 out_seq=0 2024-04-16T17:14:25.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d96730b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d96730b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672fbec0 0x55d9672fa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2be17e5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81afe60 con 0x55d9672fbec0 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2be17e5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81ad730 con 0x55d9672f32e0 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2be17e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81ab5a0 con 0x55d9672f29c0 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d96730b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d96730b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54914/0 (socket says 172.21.15.18:54914) 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2bd7e00640 1 -- 172.21.15.18:0/3847164606 learned_addr learned my addr 172.21.15.18:0/3847164606 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.430+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d96730b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672fbec0 0x55d9672fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bd7e00640 1 -- 172.21.15.18:0/3847164606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 msgr2=0x55d96730b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d96730b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bd7e00640 1 -- 172.21.15.18:0/3847164606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672fbec0 msgr2=0x55d9672fa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672fbec0 0x55d9672fa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bd7e00640 1 -- 172.21.15.18:0/3847164606 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9672e8510 con 0x55d9672f32e0 2024-04-16T17:14:25.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672fbec0 0x55d9672fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d96730b920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000f890 tx=0x7f2bd00021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa92c5eb89ddfc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bc3e00640 1 -- 172.21.15.18:0/3847164606 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd004c030 con 0x55d9672f32e0 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bc3e00640 1 -- 172.21.15.18:0/3847164606 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd003b070 con 0x55d9672f32e0 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.431+0000 7f2bc3e00640 1 -- 172.21.15.18:0/3847164606 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd002c910 con 0x55d9672f32e0 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.432+0000 7f2be17e5740 1 -- 172.21.15.18:0/3847164606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 msgr2=0x55d96730b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.432+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d96730b920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000f890 tx=0x7f2bd00021d0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.432+0000 7f2be17e5740 1 -- 172.21.15.18:0/3847164606 shutdown_connections 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.432+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672fbec0 0x55d9672fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.432+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672f32e0 0x55d96730b920 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.432+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3847164606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d96730b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.432+0000 7f2be17e5740 1 -- 172.21.15.18:0/3847164606 >> 172.21.15.18:0/3847164606 conn(0x55d9672ed0f0 msgr2=0x55d9672ed4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.433+0000 7f2be17e5740 1 -- 172.21.15.18:0/3847164606 shutdown_connections 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.433+0000 7f2be17e5740 1 -- 172.21.15.18:0/3847164606 wait complete. 2024-04-16T17:14:25.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.433+0000 7f2be17e5740 1 -- start start 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.433+0000 7f2be17e5740 1 Processor -- start 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d9672fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f32e0 0x55d9672f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2be17e5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672f6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2be17e5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81abbf0 con 0x55d9672f32e0 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2be17e5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81ad060 con 0x55d9672fbec0 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2be17e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81ac2c0 con 0x55d9672f29c0 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2be17e5740 10 client.0 _ll_register_callbacks cb 0x7f2bd0026070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d9672fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d9672fa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41562/0 (socket says 172.21.15.18:41562) 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2bd7400640 1 -- 172.21.15.18:0/3468808034 learned_addr learned my addr 172.21.15.18:0/3468808034 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.434+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f32e0 0x55d9672f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.435+0000 7f2bd7400640 1 -- 172.21.15.18:0/3468808034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 msgr2=0x55d9672fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.435+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d9672fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.435+0000 7f2bd7400640 1 -- 172.21.15.18:0/3468808034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f32e0 msgr2=0x55d9672f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:25.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.435+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f32e0 0x55d9672f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.435+0000 7f2bd7400640 1 -- 172.21.15.18:0/3468808034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9672f6880 con 0x55d9672fbec0 2024-04-16T17:14:25.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.435+0000 7f2bd7400640 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672f6140 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55d9672f6c80 tx=0x7f2bc402e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed817ca8ea3c499c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.435+0000 7f2bc1600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.436+0000 7f2bc1600640 1 -- 172.21.15.18:0/3468808034 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc402cc00 con 0x55d9672fbec0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.436+0000 7f2bc1600640 1 -- 172.21.15.18:0/3468808034 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f2bc403a040 con 0x55d9672fbec0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.436+0000 7f2be17e5740 1 -- 172.21.15.18:0/3468808034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd003b070 con 0x55d9672fbec0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2bc1600640 1 -- 172.21.15.18:0/3468808034 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bc4060050 con 0x55d9672fbec0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2be17e5740 10 client.28178 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2be17e5740 1 -- 172.21.15.18:0/3468808034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d967369b80 con 0x55d9672fbec0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2bc1600640 1 -- 172.21.15.18:0/3468808034 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd003b070 con 0x55d9672fbec0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2bc1600640 1 -- 172.21.15.18:0/3468808034 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2bc4003ad0 con 0x55d9672fbec0 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2be17e5740 10 client.28178 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2be17e5740 10 client.28178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2be17e5740 20 client.28178 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d96736a2b0 0x55d96736c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2be17e5740 4 client.28178 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:14:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.437+0000 7f2be17e5740 1 -- 172.21.15.18:0/3468808034 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9672579b0 con 0x55d96736a2b0 2024-04-16T17:14:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.438+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d96736a2b0 0x55d96736c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.438+0000 7f2bd7e00640 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d96736a2b0 0x55d96736c6d0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e95b2dab1cf9ba server_cookie=f8d340c3073852bc in_seq=0 out_seq=0 2024-04-16T17:14:25.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.438+0000 7f2bc1600640 10 client.28178 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.598+0000 7f5903400640 1 -- 172.21.15.18:0/1703952992 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e706332010 con 0x55e706456850 2024-04-16T17:14:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.598+0000 7f5903400640 10 client.28063 handle_command_reply: tid=0 2024-04-16T17:14:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.598+0000 7f592ba1b740 1 client.28063 shutdown 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.598+0000 7f2bc1600640 1 -- 172.21.15.18:0/3468808034 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d9672579b0 con 0x55d96736a2b0 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.598+0000 7f2bc1600640 10 client.28178 handle_command_reply: tid=0 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.598+0000 7f2be17e5740 1 client.28178 shutdown 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 -- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 msgr2=0x55d9672f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672f6140 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55d9672f6c80 tx=0x7f2bc402e040 comp rx=0 tx=0).stop 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 -- 172.21.15.18:0/3468808034 shutdown_connections 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d96736a2b0 0x55d96736c6d0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9672fbec0 0x55d9672f6140 secure :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55d9672f6c80 tx=0x7f2bc402e040 comp rx=0 tx=0).stop 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d9672f32e0 0x55d9672f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 --2- 172.21.15.18:0/3468808034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9672f29c0 0x55d9672fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 -- 172.21.15.18:0/3468808034 >> 172.21.15.18:0/3468808034 conn(0x55d9672ed0f0 msgr2=0x55d9672e8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2bc1600640 0 client.28178 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 -- 172.21.15.18:0/3468808034 shutdown_connections 2024-04-16T17:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f592ba1b740 1 -- 172.21.15.18:0/1703952992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 msgr2=0x55e7063f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f592ba1b740 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063f5850 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f591c018570 tx=0x7f591c012040 comp rx=0 tx=0).stop 2024-04-16T17:14:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 1 -- 172.21.15.18:0/3468808034 wait complete. 2024-04-16T17:14:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f2be17e5740 20 client.28178 trim_cache size 0 max 16384 2024-04-16T17:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f592ba1b740 1 -- 172.21.15.18:0/1703952992 shutdown_connections 2024-04-16T17:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f592ba1b740 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e706456850 0x55e706458c70 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f592ba1b740 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7063cf000 0x55e7063f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f592ba1b740 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e7063c67f0 0x55e7063f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f592ba1b740 1 --2- 172.21.15.18:0/1703952992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7063c5a80 0x55e7063f5850 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.599+0000 7f592ba1b740 1 -- 172.21.15.18:0/1703952992 >> 172.21.15.18:0/1703952992 conn(0x55e7063605d0 msgr2=0x55e7063eb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.600+0000 7f5903400640 0 client.28063 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.600+0000 7f592ba1b740 1 -- 172.21.15.18:0/1703952992 shutdown_connections 2024-04-16T17:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.600+0000 7f592ba1b740 1 -- 172.21.15.18:0/1703952992 wait complete. 2024-04-16T17:14:25.603 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:14:25.603 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:14:25.603 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:14:25.603 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.600+0000 7f592ba1b740 20 client.28063 trim_cache size 0 max 16384 2024-04-16T17:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 -- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3da0 msgr2=0x7f2bbc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80990e0 tx=0x7f2bc80049b0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 -- 172.21.15.18:0/2407888122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd819a920 msgr2=0x7f2bd819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd819a920 0x7f2bd819add0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e970 tx=0x7f2bc800ed70 comp rx=0 tx=0).stop 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 -- 172.21.15.18:0/2407888122 shutdown_connections 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd819a920 0x7f2bd819add0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd8109520 0x7f2bd81a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 --2- 172.21.15.18:0/2407888122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8104450 0x7f2bd81a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 -- 172.21.15.18:0/2407888122 >> 172.21.15.18:0/2407888122 conn(0x7f2bd80ffc90 msgr2=0x7f2bd8102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 -- 172.21.15.18:0/2407888122 shutdown_connections 2024-04-16T17:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f5923200640 1 -- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59080a3e00 msgr2=0x7f59080a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f5923200640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59080a3e00 0x7f59080a62c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55e7063c6f50 tx=0x7f5914009260 comp rx=0 tx=0).stop 2024-04-16T17:14:25.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.601+0000 7f2bdea00640 1 -- 172.21.15.18:0/2407888122 wait complete. 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 -- 172.21.15.18:0/2761331802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 msgr2=0x7f5924110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 0x7f5924110cc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5914005ab0 tx=0x7f59140042f0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 -- 172.21.15.18:0/2761331802 shutdown_connections 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f59080a3e00 0x7f59080a62c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f592410f350 0x7f592410f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5924109370 0x7f592410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 --2- 172.21.15.18:0/2761331802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59241042a0 0x7f5924110cc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 -- 172.21.15.18:0/2761331802 >> 172.21.15.18:0/2761331802 conn(0x7f59240ffb00 msgr2=0x7f5924101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 -- 172.21.15.18:0/2761331802 shutdown_connections 2024-04-16T17:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:25.602+0000 7f5923200640 1 -- 172.21.15.18:0/2761331802 wait complete. 2024-04-16T17:14:25.630 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:14:25.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:14:25.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 365740 3.70 MB/sec execute 398 sec latency 84.360 ms 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 -- 172.21.15.18:0/3002812917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8107340 msgr2=0x7f0bd810ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 --2- 172.21.15.18:0/3002812917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8107340 0x7f0bd810ef10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009be0 tx=0x7f0bd002bef0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 -- 172.21.15.18:0/3002812917 shutdown_connections 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 --2- 172.21.15.18:0/3002812917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8107340 0x7f0bd810ef10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 --2- 172.21.15.18:0/3002812917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd8102c20 0x7f0bd8106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 --2- 172.21.15.18:0/3002812917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bd8102270 0x7f0bd8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 -- 172.21.15.18:0/3002812917 >> 172.21.15.18:0/3002812917 conn(0x7f0bd80fdab0 msgr2=0x7f0bd80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 -- 172.21.15.18:0/3002812917 shutdown_connections 2024-04-16T17:14:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.819+0000 7f0bdea00640 1 -- 172.21.15.18:0/3002812917 wait complete. 2024-04-16T17:14:25.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.820+0000 7f0bdea00640 1 Processor -- start 2024-04-16T17:14:25.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.820+0000 7f0bdea00640 1 -- start start 2024-04-16T17:14:25.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bdea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd8102270 0x7f0bd8074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bdea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bd8107340 0x7f0bd8074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bdea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8074d80 0x7f0bd81a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bdea00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81123e0 con 0x7f0bd8107340 2024-04-16T17:14:25.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bdea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8112260 con 0x7f0bd8074d80 2024-04-16T17:14:25.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bdea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8112560 con 0x7f0bd8102270 2024-04-16T17:14:25.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bd7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bd8107340 0x7f0bd8074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd8102270 0x7f0bd8074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bdca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8074d80 0x7f0bd81a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd8102270 0x7f0bd8074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41568/0 (socket says 172.21.15.18:41568) 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bd7e00640 1 -- 172.21.15.18:0/707668940 learned_addr learned my addr 172.21.15.18:0/707668940 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.821+0000 7f0bdca00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8074d80 0x7f0bd81a6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54944/0 (socket says 172.21.15.18:54944) 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.822+0000 7f0bd7e00640 1 -- 172.21.15.18:0/707668940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8074d80 msgr2=0x7f0bd81a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.822+0000 7f0bd7e00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8074d80 0x7f0bd81a6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.822+0000 7f0bd7e00640 1 -- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bd8107340 msgr2=0x7f0bd8074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.822+0000 7f0bd7e00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bd8107340 0x7f0bd8074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.822+0000 7f0bd7e00640 1 -- 172.21.15.18:0/707668940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd81a7170 con 0x7f0bd8102270 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.822+0000 7f0bd7e00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd8102270 0x7f0bd8074300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800d950 tx=0x7f0bc800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.823+0000 7f0bd4c00640 1 -- 172.21.15.18:0/707668940 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8017030 con 0x7f0bd8102270 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.823+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd81a7460 con 0x7f0bd8102270 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.823+0000 7f0bd4c00640 1 -- 172.21.15.18:0/707668940 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8004020 con 0x7f0bd8102270 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.823+0000 7f0bd4c00640 1 -- 172.21.15.18:0/707668940 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80047a0 con 0x7f0bd8102270 2024-04-16T17:14:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.823+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd81a7a00 con 0x7f0bd8102270 2024-04-16T17:14:25.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.824+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd8068eb0 con 0x7f0bd8102270 2024-04-16T17:14:25.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.828+0000 7f0bd4c00640 1 -- 172.21.15.18:0/707668940 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0bc80041d0 con 0x7f0bd8102270 2024-04-16T17:14:25.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.829+0000 7f0bd4c00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0bbc0a3d50 0x7f0bbc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:25.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.829+0000 7f0bd4c00640 1 -- 172.21.15.18:0/707668940 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f0bc80d06a0 con 0x7f0bd8102270 2024-04-16T17:14:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.830+0000 7f0bd7400640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0bbc0a3d50 0x7f0bbc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.830+0000 7f0bd7400640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0bbc0a3d50 0x7f0bbc0a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40096c0 tx=0x7f0bc4009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:25.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:25.834+0000 7f0bd4c00640 1 -- 172.21.15.18:0/707668940 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bc809c8d0 con 0x7f0bd8102270 2024-04-16T17:14:25.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:25 smithi022 ceph-mon[36023]: pgmap v303: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-16T17:14:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:25 smithi043 ceph-mon[36901]: pgmap v303: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-16T17:14:26.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.081+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bd810a570 con 0x7f0bd8102270 2024-04-16T17:14:26.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.082+0000 7f0bd4c00640 1 -- 172.21.15.18:0/707668940 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11851 (secure 0 0 0) 0x7f0bc809c1d0 con 0x7f0bd8102270 2024-04-16T17:14:26.083 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:14:26.083 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:11:38.382274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27445":{"gid":27445,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/678063310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":678063310},{"type":"v1","addr":"172.21.15.22:6837","nonce":678063310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:14:26.083 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 27 2024-04-16T17:14:26.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.084+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0bbc0a3d50 msgr2=0x7f0bbc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.084+0000 7f0bdea00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0bbc0a3d50 0x7f0bbc0a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40096c0 tx=0x7f0bc4009260 comp rx=0 tx=0).stop 2024-04-16T17:14:26.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.084+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd8102270 msgr2=0x7f0bd8074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.084+0000 7f0bdea00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd8102270 0x7f0bd8074300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800d950 tx=0x7f0bc800de10 comp rx=0 tx=0).stop 2024-04-16T17:14:26.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.085+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 shutdown_connections 2024-04-16T17:14:26.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.085+0000 7f0bdea00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0bbc0a3d50 0x7f0bbc0a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.085+0000 7f0bdea00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd8074d80 0x7f0bd81a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.085+0000 7f0bdea00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bd8107340 0x7f0bd8074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.085+0000 7f0bdea00640 1 --2- 172.21.15.18:0/707668940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd8102270 0x7f0bd8074300 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.085+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 >> 172.21.15.18:0/707668940 conn(0x7f0bd80fdab0 msgr2=0x7f0bd80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.085+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 shutdown_connections 2024-04-16T17:14:26.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.085+0000 7f0bdea00640 1 -- 172.21.15.18:0/707668940 wait complete. 2024-04-16T17:14:26.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:14:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.285+0000 7f9faae00640 1 -- 172.21.15.18:0/3667466154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104c30 msgr2=0x7f9fa4108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.285+0000 7f9faae00640 1 --2- 172.21.15.18:0/3667466154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104c30 0x7f9fa4108c70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009860 tx=0x7f9f9002bb90 comp rx=0 tx=0).stop 2024-04-16T17:14:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.285+0000 7f9faae00640 1 -- 172.21.15.18:0/3667466154 shutdown_connections 2024-04-16T17:14:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.285+0000 7f9faae00640 1 --2- 172.21.15.18:0/3667466154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4109350 0x7f9fa4110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.286+0000 7f9faae00640 1 --2- 172.21.15.18:0/3667466154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104c30 0x7f9fa4108c70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.286+0000 7f9faae00640 1 --2- 172.21.15.18:0/3667466154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fa4104280 0x7f9fa4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.286+0000 7f9faae00640 1 -- 172.21.15.18:0/3667466154 >> 172.21.15.18:0/3667466154 conn(0x7f9fa40ffae0 msgr2=0x7f9fa4101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.286+0000 7f9faae00640 1 -- 172.21.15.18:0/3667466154 shutdown_connections 2024-04-16T17:14:26.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.286+0000 7f9faae00640 1 -- 172.21.15.18:0/3667466154 wait complete. 2024-04-16T17:14:26.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.287+0000 7f9faae00640 1 Processor -- start 2024-04-16T17:14:26.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.287+0000 7f9faae00640 1 -- start start 2024-04-16T17:14:26.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9faae00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104280 0x7f9fa41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9faae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4109350 0x7f9fa41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9faae00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fa419a840 0x7f9fa419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9faae00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4114460 con 0x7f9fa419a840 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9faae00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41142e0 con 0x7f9fa4104280 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9faae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41145e0 con 0x7f9fa4109350 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104280 0x7f9fa41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4109350 0x7f9fa41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9faa400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fa419a840 0x7f9fa419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104280 0x7f9fa41a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54962/0 (socket says 172.21.15.18:54962) 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9a00640 1 -- 172.21.15.18:0/3528985860 learned_addr learned my addr 172.21.15.18:0/3528985860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9a00640 1 -- 172.21.15.18:0/3528985860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4109350 msgr2=0x7f9fa41a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9a00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4109350 0x7f9fa41a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9a00640 1 -- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fa419a840 msgr2=0x7f9fa419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9a00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fa419a840 0x7f9fa419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.288+0000 7f9fa9a00640 1 -- 172.21.15.18:0/3528985860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa419b3d0 con 0x7f9fa4104280 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.289+0000 7f9fa9000640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4109350 0x7f9fa41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:26.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.289+0000 7f9fa9a00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104280 0x7f9fa41a0680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800e9c0 tx=0x7f9f9800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.289+0000 7f9fa2000640 1 -- 172.21.15.18:0/3528985860 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800ccc0 con 0x7f9fa4104280 2024-04-16T17:14:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.289+0000 7f9fa2000640 1 -- 172.21.15.18:0/3528985860 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f98005500 con 0x7f9fa4104280 2024-04-16T17:14:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.289+0000 7f9faae00640 1 -- 172.21.15.18:0/3528985860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa419b6c0 con 0x7f9fa4104280 2024-04-16T17:14:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.289+0000 7f9fa2000640 1 -- 172.21.15.18:0/3528985860 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98010430 con 0x7f9fa4104280 2024-04-16T17:14:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.289+0000 7f9faae00640 1 -- 172.21.15.18:0/3528985860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa41a7870 con 0x7f9fa4104280 2024-04-16T17:14:26.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.293+0000 7f9faf065740 1 Processor -- start 2024-04-16T17:14:26.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9faf065740 1 -- start start 2024-04-16T17:14:26.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9faf065740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5c9b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9faf065740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9faf065740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9faf065740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5641a5c00020 con 0x5641a5c9af10 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9faf065740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5641a5ba6460 con 0x5641a5c9b830 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9faf065740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5641a5bfbfe0 con 0x5641a5ca4410 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9fa2000640 1 -- 172.21.15.18:0/3528985860 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9f980046c0 con 0x7f9fa4104280 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9fa9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9faa400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5c9b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9fa9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41606/0 (socket says 172.21.15.18:41606) 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.294+0000 7f9fa9000640 1 -- 172.21.15.18:0/3903526302 learned_addr learned my addr 172.21.15.18:0/3903526302 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.295+0000 7f9fa9a00640 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.295+0000 7f9faa400640 1 -- 172.21.15.18:0/3903526302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 msgr2=0x5641a5ca6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.295+0000 7f9faa400640 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.295+0000 7f9faa400640 1 -- 172.21.15.18:0/3903526302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 msgr2=0x5641a5ca3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.295+0000 7f9faa400640 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.295+0000 7f9faa400640 1 -- 172.21.15.18:0/3903526302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a5bae040 con 0x5641a5c9af10 2024-04-16T17:14:26.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.295+0000 7f9fa9a00640 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.295+0000 7f9faa400640 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5c9b2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009fb0 tx=0x7f9f9c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e6a5b6c37b57c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.296+0000 7f9f96a00640 1 -- 172.21.15.18:0/3903526302 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c013790 con 0x5641a5c9af10 2024-04-16T17:14:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.296+0000 7f9f96a00640 1 -- 172.21.15.18:0/3903526302 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9c019070 con 0x5641a5c9af10 2024-04-16T17:14:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.296+0000 7f9f96a00640 1 -- 172.21.15.18:0/3903526302 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c014d60 con 0x5641a5c9af10 2024-04-16T17:14:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.296+0000 7f9fa2000640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9f8c0a3da0 0x7f9f8c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.296+0000 7f9fa2000640 1 -- 172.21.15.18:0/3528985860 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f9f980d0ec0 con 0x7f9fa4104280 2024-04-16T17:14:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 -- 172.21.15.18:0/3903526302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 msgr2=0x5641a5c9b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5c9b2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009fb0 tx=0x7f9f9c009650 comp rx=0 tx=0).stop 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 -- 172.21.15.18:0/3903526302 shutdown_connections 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 --2- 172.21.15.18:0/3903526302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5c9b2f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 -- 172.21.15.18:0/3903526302 >> 172.21.15.18:0/3903526302 conn(0x5641a5c95640 msgr2=0x5641a5c95a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9fa9a00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9f8c0a3da0 0x7f9f8c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 -- 172.21.15.18:0/3903526302 shutdown_connections 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.297+0000 7f9faf065740 1 -- 172.21.15.18:0/3903526302 wait complete. 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.298+0000 7f9fa9a00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9f8c0a3da0 0x7f9f8c0a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5641a5c9a9e0 tx=0x7f9f980bd040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.298+0000 7f9faf065740 1 -- start start 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.298+0000 7f9faf065740 1 Processor -- start 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.298+0000 7f9faf065740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5ca1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faf065740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faf065740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faf065740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5641a5ca7200 con 0x5641a5c9af10 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faf065740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5641a5bb9830 con 0x5641a5c9b830 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faf065740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5641a5ca7380 con 0x5641a5ca4410 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faf065740 10 client.0 _ll_register_callbacks cb 0x5641a5bc1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faa400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5ca1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faa400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5ca1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58852/0 (socket says 172.21.15.18:58852) 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faa400640 1 -- 172.21.15.18:0/64019549 learned_addr learned my addr 172.21.15.18:0/64019549 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9fa9000640 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9faa400640 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9fa9000640 1 -- 172.21.15.18:0/64019549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 msgr2=0x5641a5ca1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9fa9000640 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9fa9000640 1 -- 172.21.15.18:0/64019549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 msgr2=0x5641a5ca1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9fa9000640 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5ca1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.299+0000 7f9fa9000640 1 -- 172.21.15.18:0/64019549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a5bbd920 con 0x5641a5c9b830 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.300+0000 7f9faa400640 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5ca1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.300+0000 7f9fa9000640 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca19b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009830 tx=0x7f9f90004510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e72f4a2d86a478ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.300+0000 7f9f8b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.300+0000 7f9f8b400640 1 -- 172.21.15.18:0/64019549 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90044020 con 0x5641a5c9b830 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.300+0000 7f9faf065740 1 -- 172.21.15.18:0/64019549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9c019070 con 0x5641a5c9b830 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.300+0000 7f9f8b400640 1 -- 172.21.15.18:0/64019549 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f9f90034640 con 0x5641a5c9b830 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9f8b400640 1 -- 172.21.15.18:0/64019549 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f9004f050 con 0x5641a5c9b830 2024-04-16T17:14:26.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9faf065740 10 client.28196 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9faf065740 1 -- 172.21.15.18:0/64019549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5641a5b8b7f0 con 0x5641a5c9b830 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9f8b400640 1 -- 172.21.15.18:0/64019549 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f9c019070 con 0x5641a5c9b830 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9f8b400640 1 -- 172.21.15.18:0/64019549 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9f900329a0 con 0x5641a5c9b830 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9faf065740 10 client.28196 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9faf065740 10 client.28196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9faf065740 20 client.28196 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9faf065740 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641a5cbcfa0 0x5641a5cbf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9faf065740 4 client.28196 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.301+0000 7f9faf065740 1 -- 172.21.15.18:0/64019549 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641a5c0da40 con 0x5641a5cbcfa0 2024-04-16T17:14:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.302+0000 7f9faa400640 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641a5cbcfa0 0x5641a5cbf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.302+0000 7f9faa400640 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641a5cbcfa0 0x5641a5cbf3f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7100de971c2958ce server_cookie=8923f9c90d3a6751 in_seq=0 out_seq=0 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.302+0000 7f9f8b400640 10 client.28196 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.303+0000 7f9f8b400640 1 -- 172.21.15.18:0/64019549 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641a5c0da40 con 0x5641a5cbcfa0 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.303+0000 7f9f8b400640 10 client.28196 handle_command_reply: tid=0 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.303+0000 7f9faf065740 1 client.28196 shutdown 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 -- 172.21.15.18:0/64019549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 msgr2=0x5641a5ca19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca19b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009830 tx=0x7f9f90004510 comp rx=0 tx=0).stop 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 -- 172.21.15.18:0/64019549 shutdown_connections 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641a5cbcfa0 0x5641a5cbf3f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5ca4410 0x5641a5ca1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca19b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 --2- 172.21.15.18:0/64019549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5c9af10 0x5641a5ca1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9f8b400640 0 client.28196 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:14:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 -- 172.21.15.18:0/64019549 >> 172.21.15.18:0/64019549 conn(0x5641a5c95640 msgr2=0x5641a5c97390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 -- 172.21.15.18:0/64019549 shutdown_connections 2024-04-16T17:14:26.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 1 -- 172.21.15.18:0/64019549 wait complete. 2024-04-16T17:14:26.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.304+0000 7f9faf065740 20 client.28196 trim_cache size 0 max 16384 2024-04-16T17:14:26.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.319+0000 7f9faf065740 1 Processor -- start 2024-04-16T17:14:26.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.319+0000 7f9faf065740 1 -- start start 2024-04-16T17:14:26.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.319+0000 7f9faf065740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9af10 0x5641a5ca3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.319+0000 7f9faf065740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9b830 0x5641a5cb00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.319+0000 7f9faf065740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 0x5641a5c97180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.319+0000 7f9faf065740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41afc30 con 0x5641a5ca4410 2024-04-16T17:14:26.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.319+0000 7f9faf065740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41ad500 con 0x5641a5c9af10 2024-04-16T17:14:26.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.319+0000 7f9faf065740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41ab370 con 0x5641a5c9b830 2024-04-16T17:14:26.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9faa400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9af10 0x5641a5ca3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9faa400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9af10 0x5641a5ca3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54986/0 (socket says 172.21.15.18:54986) 2024-04-16T17:14:26.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9faa400640 1 -- 172.21.15.18:0/843687269 learned_addr learned my addr 172.21.15.18:0/843687269 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9fa9000640 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9b830 0x5641a5cb00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9faa400640 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 0x5641a5c97180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9fa9000640 1 -- 172.21.15.18:0/843687269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9af10 msgr2=0x5641a5ca3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9fa9000640 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9af10 0x5641a5ca3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9fa9000640 1 -- 172.21.15.18:0/843687269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 msgr2=0x5641a5c97180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9fa9000640 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 0x5641a5c97180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9fa9000640 1 -- 172.21.15.18:0/843687269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a59123f0 con 0x5641a5c9b830 2024-04-16T17:14:26.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.320+0000 7f9faa400640 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9af10 0x5641a5ca3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:26.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.321+0000 7f9fa9000640 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9b830 0x5641a5cb00f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9f90032d90 tx=0x7f9f90009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f21f69de13ceff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.321+0000 7f9f96000640 1 -- 172.21.15.18:0/843687269 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90000e60 con 0x5641a5c9b830 2024-04-16T17:14:26.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.321+0000 7f9f96000640 1 -- 172.21.15.18:0/843687269 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f90032640 con 0x5641a5c9b830 2024-04-16T17:14:26.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.321+0000 7f9f96000640 1 -- 172.21.15.18:0/843687269 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90044030 con 0x5641a5c9b830 2024-04-16T17:14:26.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 -- 172.21.15.18:0/843687269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9b830 msgr2=0x5641a5cb00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9b830 0x5641a5cb00f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9f90032d90 tx=0x7f9f90009550 comp rx=0 tx=0).stop 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 -- 172.21.15.18:0/843687269 shutdown_connections 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 0x5641a5c97180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9b830 0x5641a5cb00f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 --2- 172.21.15.18:0/843687269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9af10 0x5641a5ca3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 -- 172.21.15.18:0/843687269 >> 172.21.15.18:0/843687269 conn(0x5641a5c95640 msgr2=0x5641a5c96a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 -- 172.21.15.18:0/843687269 shutdown_connections 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 -- 172.21.15.18:0/843687269 wait complete. 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 -- start start 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.322+0000 7f9faf065740 1 Processor -- start 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faf065740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9af10 0x5641a5cac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faf065740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faf065740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 0x5641a5ca8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faf065740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41ab9c0 con 0x5641a5ca4410 2024-04-16T17:14:26.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faf065740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41ace30 con 0x5641a5c9b830 2024-04-16T17:14:26.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faf065740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41ac090 con 0x5641a5c9af10 2024-04-16T17:14:26.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faf065740 10 client.0 _ll_register_callbacks cb 0x7f9f98005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:26.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9fa9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faa400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9af10 0x5641a5cac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9fa9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54996/0 (socket says 172.21.15.18:54996) 2024-04-16T17:14:26.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9faa400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 0x5641a5ca8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.323+0000 7f9fa9000640 1 -- 172.21.15.18:0/2509419357 learned_addr learned my addr 172.21.15.18:0/2509419357 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9fa9000640 1 -- 172.21.15.18:0/2509419357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9af10 msgr2=0x5641a5cac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9fa9000640 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9af10 0x5641a5cac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9fa9000640 1 -- 172.21.15.18:0/2509419357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 msgr2=0x5641a5ca8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9fa9000640 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 0x5641a5ca8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9fa9000640 1 -- 172.21.15.18:0/2509419357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a59dc7d0 con 0x5641a5c9b830 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9fa9000640 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca7d00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009860 tx=0x7f9f90032aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e05bf68beb6a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9f8b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9f8b400640 1 -- 172.21.15.18:0/2509419357 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90061060 con 0x5641a5c9b830 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9faf065740 1 -- 172.21.15.18:0/2509419357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f90032640 con 0x5641a5c9b830 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.324+0000 7f9f8b400640 1 -- 172.21.15.18:0/2509419357 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f9f90034650 con 0x5641a5c9b830 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.325+0000 7f9f8b400640 1 -- 172.21.15.18:0/2509419357 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f90068050 con 0x5641a5c9b830 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.325+0000 7f9faf065740 10 client.28202 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.325+0000 7f9faf065740 1 -- 172.21.15.18:0/2509419357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5641a5d232e0 con 0x5641a5c9b830 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.325+0000 7f9f8b400640 1 -- 172.21.15.18:0/2509419357 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f90032640 con 0x5641a5c9b830 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.325+0000 7f9f8b400640 1 -- 172.21.15.18:0/2509419357 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9f9002c530 con 0x5641a5c9b830 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.325+0000 7f9faf065740 10 client.28202 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.325+0000 7f9faf065740 10 client.28202 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.326+0000 7f9faf065740 20 client.28202 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.326+0000 7f9faf065740 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641a5d239b0 0x5641a5d25dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.326+0000 7f9faf065740 4 client.28202 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.326+0000 7f9faf065740 1 -- 172.21.15.18:0/2509419357 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641a5c078b0 con 0x5641a5d239b0 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.326+0000 7f9faa400640 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641a5d239b0 0x5641a5d25dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.326+0000 7f9faa400640 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641a5d239b0 0x5641a5d25dd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4de7d8fea9104b93 server_cookie=d4236a4bf1a5080f in_seq=0 out_seq=0 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.326+0000 7f9f8b400640 10 client.28202 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9f8b400640 1 -- 172.21.15.18:0/2509419357 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641a5c078b0 con 0x5641a5d239b0 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9f8b400640 10 client.28202 handle_command_reply: tid=0 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 client.28202 shutdown 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 -- 172.21.15.18:0/2509419357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 msgr2=0x5641a5ca7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca7d00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009860 tx=0x7f9f90032aa0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 -- 172.21.15.18:0/2509419357 shutdown_connections 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641a5d239b0 0x5641a5d25dd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641a5ca4410 0x5641a5ca8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641a5c9b830 0x5641a5ca7d00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9f8b400640 0 client.28202 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 --2- 172.21.15.18:0/2509419357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641a5c9af10 0x5641a5cac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 -- 172.21.15.18:0/2509419357 >> 172.21.15.18:0/2509419357 conn(0x5641a5c95640 msgr2=0x5641a5c97300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 -- 172.21.15.18:0/2509419357 shutdown_connections 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 1 -- 172.21.15.18:0/2509419357 wait complete. 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.327+0000 7f9faf065740 20 client.28202 trim_cache size 0 max 16384 2024-04-16T17:14:26.330 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:14:26.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 -- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9f8c0a3da0 msgr2=0x7f9f8c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9f8c0a3da0 0x7f9f8c0a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5641a5c9a9e0 tx=0x7f9f980bd040 comp rx=0 tx=0).stop 2024-04-16T17:14:26.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 -- 172.21.15.18:0/3528985860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104280 msgr2=0x7f9fa41a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104280 0x7f9fa41a0680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800e9c0 tx=0x7f9f9800ee80 comp rx=0 tx=0).stop 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 -- 172.21.15.18:0/3528985860 shutdown_connections 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9f8c0a3da0 0x7f9f8c0a6260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9fa419a840 0x7f9fa419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa4109350 0x7f9fa41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 --2- 172.21.15.18:0/3528985860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4104280 0x7f9fa41a0680 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 -- 172.21.15.18:0/3528985860 >> 172.21.15.18:0/3528985860 conn(0x7f9fa40ffae0 msgr2=0x7f9fa4100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 -- 172.21.15.18:0/3528985860 shutdown_connections 2024-04-16T17:14:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.328+0000 7f9faae00640 1 -- 172.21.15.18:0/3528985860 wait complete. 2024-04-16T17:14:26.346 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:14:26.478 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:19 smithi018 ceph-mon[31776]: pgmap v300: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-04-16T17:14:26.478 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:21 smithi018 ceph-mon[31776]: pgmap v301: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-04-16T17:14:26.478 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:23 smithi018 ceph-mon[31776]: pgmap v302: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-04-16T17:14:26.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 -- 172.21.15.18:0/1667309018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1881090c0 msgr2=0x7fa188115570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 --2- 172.21.15.18:0/1667309018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1881090c0 0x7fa188115570 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa180009ca0 tx=0x7fa18002c180 comp rx=0 tx=0).stop 2024-04-16T17:14:26.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 -- 172.21.15.18:0/1667309018 shutdown_connections 2024-04-16T17:14:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 --2- 172.21.15.18:0/1667309018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1881090c0 0x7fa188115570 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 --2- 172.21.15.18:0/1667309018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa188108720 0x7fa188108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 --2- 172.21.15.18:0/1667309018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa18810a590 0x7fa18810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 -- 172.21.15.18:0/1667309018 >> 172.21.15.18:0/1667309018 conn(0x7fa188077e70 msgr2=0x7fa188106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 -- 172.21.15.18:0/1667309018 shutdown_connections 2024-04-16T17:14:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.546+0000 7fa18e800640 1 -- 172.21.15.18:0/1667309018 wait complete. 2024-04-16T17:14:26.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.547+0000 7fa18e800640 1 Processor -- start 2024-04-16T17:14:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.547+0000 7fa18e800640 1 -- start start 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa18e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa188108720 0x7fa18819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa18e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1881090c0 0x7fa18819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa18e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa18810a590 0x7fa1881a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa18e800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa188118af0 con 0x7fa1881090c0 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa18e800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa188118970 con 0x7fa188108720 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa18e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa188118c70 con 0x7fa18810a590 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa186a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1881090c0 0x7fa18819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa187400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa188108720 0x7fa18819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa186a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1881090c0 0x7fa18819f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58872/0 (socket says 172.21.15.18:58872) 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa186a00640 1 -- 172.21.15.18:0/1227967908 learned_addr learned my addr 172.21.15.18:0/1227967908 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.548+0000 7fa187e00640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa18810a590 0x7fa1881a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.549+0000 7fa187400640 1 -- 172.21.15.18:0/1227967908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa18810a590 msgr2=0x7fa1881a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.549+0000 7fa187400640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa18810a590 0x7fa1881a3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.549+0000 7fa187400640 1 -- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1881090c0 msgr2=0x7fa18819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.549+0000 7fa187400640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1881090c0 0x7fa18819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.549+0000 7fa187400640 1 -- 172.21.15.18:0/1227967908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1881a3eb0 con 0x7fa188108720 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.549+0000 7fa186a00640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1881090c0 0x7fa18819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.549+0000 7fa187400640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa188108720 0x7fa18819ef10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa17800e9f0 tx=0x7fa17800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.550+0000 7fa17fe00640 1 -- 172.21.15.18:0/1227967908 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17800cd80 con 0x7fa188108720 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.550+0000 7fa17fe00640 1 -- 172.21.15.18:0/1227967908 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1780035f0 con 0x7fa188108720 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.550+0000 7fa18e800640 1 -- 172.21.15.18:0/1227967908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1881a41a0 con 0x7fa188108720 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.550+0000 7fa17fe00640 1 -- 172.21.15.18:0/1227967908 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178010690 con 0x7fa188108720 2024-04-16T17:14:26.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.550+0000 7fa18e800640 1 -- 172.21.15.18:0/1227967908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1881aba30 con 0x7fa188108720 2024-04-16T17:14:26.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa191451740 1 Processor -- start 2024-04-16T17:14:26.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa191451740 1 -- start start 2024-04-16T17:14:26.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa17fe00640 1 -- 172.21.15.18:0/1227967908 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa178003b30 con 0x7fa188108720 2024-04-16T17:14:26.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa191451740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf22130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa191451740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa191451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf2b250 0x55e1ccf2d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa191451740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e1cce87020 con 0x55e1ccf21d50 2024-04-16T17:14:26.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa191451740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e1cce2d460 con 0x55e1ccf22670 2024-04-16T17:14:26.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.554+0000 7fa191451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e1cce82fe0 con 0x55e1ccf2b250 2024-04-16T17:14:26.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa187400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa186a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf2b250 0x55e1ccf2d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa186a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf2b250 0x55e1ccf2d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41652/0 (socket says 172.21.15.18:41652) 2024-04-16T17:14:26.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa187400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55026/0 (socket says 172.21.15.18:55026) 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa186a00640 1 -- 172.21.15.18:0/1578102777 learned_addr learned my addr 172.21.15.18:0/1578102777 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa187e00640 1 --2- 172.21.15.18:0/1578102777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf22130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa187400640 1 -- 172.21.15.18:0/1578102777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf2b250 msgr2=0x55e1ccf2d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa187400640 1 --2- 172.21.15.18:0/1578102777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf2b250 0x55e1ccf2d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa187400640 1 -- 172.21.15.18:0/1578102777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 msgr2=0x55e1ccf22130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa187400640 1 --2- 172.21.15.18:0/1578102777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf22130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.555+0000 7fa187400640 1 -- 172.21.15.18:0/1578102777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cce35040 con 0x55e1ccf22670 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.556+0000 7fa187400640 1 --2- 172.21.15.18:0/1578102777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ad10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa1780ce200 tx=0x7fa1780d94b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb9a7bd0c839932 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.556+0000 7fa17cc00640 1 -- 172.21.15.18:0/1578102777 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1780e4070 con 0x55e1ccf22670 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.556+0000 7fa17cc00640 1 -- 172.21.15.18:0/1578102777 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1780dfae0 con 0x55e1ccf22670 2024-04-16T17:14:26.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.556+0000 7fa17cc00640 1 -- 172.21.15.18:0/1578102777 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1780de950 con 0x55e1ccf22670 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa17fe00640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa16c0a3d20 0x7fa16c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa17fe00640 1 -- 172.21.15.18:0/1227967908 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fa1780d0f20 con 0x7fa188108720 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa187e00640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa16c0a3d20 0x7fa16c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa191451740 1 -- 172.21.15.18:0/1578102777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 msgr2=0x55e1ccf2ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa191451740 1 --2- 172.21.15.18:0/1578102777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ad10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa1780ce200 tx=0x7fa1780d94b0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa191451740 1 -- 172.21.15.18:0/1578102777 shutdown_connections 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa191451740 1 --2- 172.21.15.18:0/1578102777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf2b250 0x55e1ccf2d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa191451740 1 --2- 172.21.15.18:0/1578102777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ad10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa191451740 1 --2- 172.21.15.18:0/1578102777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf22130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.557+0000 7fa191451740 1 -- 172.21.15.18:0/1578102777 >> 172.21.15.18:0/1578102777 conn(0x55e1ccf1c480 msgr2=0x55e1ccf1c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.558+0000 7fa191451740 1 -- 172.21.15.18:0/1578102777 shutdown_connections 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.558+0000 7fa191451740 1 -- 172.21.15.18:0/1578102777 wait complete. 2024-04-16T17:14:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.558+0000 7fa187e00640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa16c0a3d20 0x7fa16c0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa180005e30 tx=0x7fa180005d80 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.558+0000 7fa191451740 1 -- start start 2024-04-16T17:14:26.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa191451740 1 Processor -- start 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa191451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf21d50 0x55e1ccf28280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa191451740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf287c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa191451740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf28d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa191451740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e1ccf2e040 con 0x55e1ccf2b250 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa191451740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e1cce40830 con 0x55e1ccf22670 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa191451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e1ccf2e1c0 con 0x55e1ccf21d50 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa191451740 10 client.0 _ll_register_callbacks cb 0x55e1cce48150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.559+0000 7fa186a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf287c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa186a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf287c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55040/0 (socket says 172.21.15.18:55040) 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa186a00640 1 -- 172.21.15.18:0/964418770 learned_addr learned my addr 172.21.15.18:0/964418770 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa187e00640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf28d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa187400640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf21d50 0x55e1ccf28280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa186a00640 1 -- 172.21.15.18:0/964418770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf21d50 msgr2=0x55e1ccf28280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa186a00640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf21d50 0x55e1ccf28280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa186a00640 1 -- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 msgr2=0x55e1ccf28d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa186a00640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf28d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa186a00640 1 -- 172.21.15.18:0/964418770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cce44920 con 0x55e1ccf22670 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa187400640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf21d50 0x55e1ccf28280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa187e00640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf28d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa186a00640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf287c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa174012d20 tx=0x7fa17400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7483403b57d0c375 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.560+0000 7fa171600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.561+0000 7fa171600640 1 -- 172.21.15.18:0/964418770 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174004020 con 0x55e1ccf22670 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.561+0000 7fa171600640 1 -- 172.21.15.18:0/964418770 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fa1740041c0 con 0x55e1ccf22670 2024-04-16T17:14:26.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.561+0000 7fa191451740 1 -- 172.21.15.18:0/964418770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1780dfae0 con 0x55e1ccf22670 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.561+0000 7fa171600640 1 -- 172.21.15.18:0/964418770 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa174025050 con 0x55e1ccf22670 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.561+0000 7fa191451740 10 client.28214 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.561+0000 7fa191451740 1 -- 172.21.15.18:0/964418770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1cce127f0 con 0x55e1ccf22670 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.561+0000 7fa171600640 1 -- 172.21.15.18:0/964418770 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1780dfae0 con 0x55e1ccf22670 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.562+0000 7fa171600640 1 -- 172.21.15.18:0/964418770 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa17401c470 con 0x55e1ccf22670 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.562+0000 7fa191451740 10 client.28214 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.562+0000 7fa191451740 10 client.28214 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.562+0000 7fa191451740 20 client.28214 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.562+0000 7fa191451740 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e1ccf3b700 0x55e1ccf3db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.562+0000 7fa191451740 4 client.28214 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:26.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.562+0000 7fa191451740 1 -- 172.21.15.18:0/964418770 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1cce94a40 con 0x55e1ccf3b700 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.562+0000 7fa187400640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e1ccf3b700 0x55e1ccf3db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.563+0000 7fa187400640 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e1ccf3b700 0x55e1ccf3db60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59b8dd3188a1a770 server_cookie=99426060c483e7ca in_seq=0 out_seq=0 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.563+0000 7fa171600640 10 client.28214 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa171600640 1 -- 172.21.15.18:0/964418770 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1cce94a40 con 0x55e1ccf3b700 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa171600640 10 client.28214 handle_command_reply: tid=0 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 client.28214 shutdown 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 -- 172.21.15.18:0/964418770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 msgr2=0x55e1ccf287c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf287c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa174012d20 tx=0x7fa17400f790 comp rx=0 tx=0).stop 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 -- 172.21.15.18:0/964418770 shutdown_connections 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e1ccf3b700 0x55e1ccf3db60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf28d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf22670 0x55e1ccf287c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 --2- 172.21.15.18:0/964418770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf21d50 0x55e1ccf28280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa171600640 0 client.28214 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:14:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 -- 172.21.15.18:0/964418770 >> 172.21.15.18:0/964418770 conn(0x55e1ccf1c480 msgr2=0x55e1ccf1e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 -- 172.21.15.18:0/964418770 shutdown_connections 2024-04-16T17:14:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 1 -- 172.21.15.18:0/964418770 wait complete. 2024-04-16T17:14:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.564+0000 7fa191451740 20 client.28214 trim_cache size 0 max 16384 2024-04-16T17:14:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa191451740 1 Processor -- start 2024-04-16T17:14:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa191451740 1 -- start start 2024-04-16T17:14:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa191451740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa191451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 0x55e1ccf36f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa191451740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf2b250 0x55e1ccf1b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa191451740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881b84d0 con 0x55e1ccf21d50 2024-04-16T17:14:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa191451740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881b5da0 con 0x55e1ccf2b250 2024-04-16T17:14:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa191451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881b3c10 con 0x55e1ccf22670 2024-04-16T17:14:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa187400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa187400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58902/0 (socket says 172.21.15.18:58902) 2024-04-16T17:14:26.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.580+0000 7fa187400640 1 -- 172.21.15.18:0/2464345698 learned_addr learned my addr 172.21.15.18:0/2464345698 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa186a00640 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 0x55e1ccf36f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa187e00640 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf2b250 0x55e1ccf1b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa186a00640 1 -- 172.21.15.18:0/2464345698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf2b250 msgr2=0x55e1ccf1b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa186a00640 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf2b250 0x55e1ccf1b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa186a00640 1 -- 172.21.15.18:0/2464345698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 msgr2=0x55e1ccf2a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa186a00640 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa186a00640 1 -- 172.21.15.18:0/2464345698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ccf224a0 con 0x55e1ccf22670 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa187400640 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa186a00640 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 0x55e1ccf36f30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa17401ca30 tx=0x7fa174012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520a9695221f20d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.581+0000 7fa173e00640 1 -- 172.21.15.18:0/2464345698 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174005c50 con 0x55e1ccf22670 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa173e00640 1 -- 172.21.15.18:0/2464345698 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17401c470 con 0x55e1ccf22670 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa173e00640 1 -- 172.21.15.18:0/2464345698 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1740035f0 con 0x55e1ccf22670 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 -- 172.21.15.18:0/2464345698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 msgr2=0x55e1ccf36f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 0x55e1ccf36f30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa17401ca30 tx=0x7fa174012990 comp rx=0 tx=0).stop 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 -- 172.21.15.18:0/2464345698 shutdown_connections 2024-04-16T17:14:26.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf2b250 0x55e1ccf1b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 0x55e1ccf36f30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 --2- 172.21.15.18:0/2464345698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 -- 172.21.15.18:0/2464345698 >> 172.21.15.18:0/2464345698 conn(0x55e1ccf1c480 msgr2=0x55e1ccf1ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 -- 172.21.15.18:0/2464345698 shutdown_connections 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.582+0000 7fa191451740 1 -- 172.21.15.18:0/2464345698 wait complete. 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.583+0000 7fa191451740 1 -- start start 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.583+0000 7fa191451740 1 Processor -- start 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa191451740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa191451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa191451740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf99ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa191451740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881b4260 con 0x55e1ccf2b250 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa191451740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881b56d0 con 0x55e1ccf21d50 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa191451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881b4930 con 0x55e1ccf22670 2024-04-16T17:14:26.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa191451740 10 client.0 _ll_register_callbacks cb 0x7fa174009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:26.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf99ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55048/0 (socket says 172.21.15.18:55048) 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187400640 1 -- 172.21.15.18:0/1344375036 learned_addr learned my addr 172.21.15.18:0/1344375036 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187400640 1 -- 172.21.15.18:0/1344375036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 msgr2=0x55e1ccf2ff20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187400640 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187400640 1 -- 172.21.15.18:0/1344375036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 msgr2=0x55e1ccf99ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187400640 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf99ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187400640 1 -- 172.21.15.18:0/1344375036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ccb993f0 con 0x55e1ccf21d50 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.584+0000 7fa187e00640 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf99ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:26.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa187400640 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2f9e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa178099420 tx=0x7fa1780d4a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd383a6fb86495c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa171600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa171600640 1 -- 172.21.15.18:0/1344375036 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17809c820 con 0x55e1ccf21d50 2024-04-16T17:14:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa171600640 1 -- 172.21.15.18:0/1344375036 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fa17809ad50 con 0x55e1ccf21d50 2024-04-16T17:14:26.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa191451740 1 -- 172.21.15.18:0/1344375036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa17401c470 con 0x55e1ccf21d50 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa171600640 1 -- 172.21.15.18:0/1344375036 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1780d9e60 con 0x55e1ccf21d50 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa191451740 10 client.28220 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa191451740 1 -- 172.21.15.18:0/1344375036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1ccc637d0 con 0x55e1ccf21d50 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa171600640 1 -- 172.21.15.18:0/1344375036 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17401c470 con 0x55e1ccf21d50 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa171600640 1 -- 172.21.15.18:0/1344375036 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa1780990c0 con 0x55e1ccf21d50 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa191451740 10 client.28220 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa191451740 10 client.28220 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.585+0000 7fa191451740 20 client.28220 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.586+0000 7fa191451740 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e1ccfa2750 0x55e1ccfa4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.586+0000 7fa191451740 4 client.28220 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.586+0000 7fa191451740 1 -- 172.21.15.18:0/1344375036 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1cce8e8b0 con 0x55e1ccfa2750 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.586+0000 7fa186a00640 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e1ccfa2750 0x55e1ccfa4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.586+0000 7fa186a00640 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e1ccfa2750 0x55e1ccfa4b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15a2e75a1d7a45dc server_cookie=dac6f5730d203d3 in_seq=0 out_seq=0 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.586+0000 7fa171600640 10 client.28220 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa171600640 1 -- 172.21.15.18:0/1344375036 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1cce8e8b0 con 0x55e1ccfa2750 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa171600640 10 client.28220 handle_command_reply: tid=0 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 client.28220 shutdown 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 -- 172.21.15.18:0/1344375036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf21d50 msgr2=0x55e1ccf2f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2f9e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa178099420 tx=0x7fa1780d4a60 comp rx=0 tx=0).stop 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 -- 172.21.15.18:0/1344375036 shutdown_connections 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e1ccfa2750 0x55e1ccfa4b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e1ccf2b250 0x55e1ccf99ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1ccf22670 0x55e1ccf2ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa171600640 0 client.28220 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:14:26.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 --2- 172.21.15.18:0/1344375036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1ccf21d50 0x55e1ccf2f9e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 -- 172.21.15.18:0/1344375036 >> 172.21.15.18:0/1344375036 conn(0x55e1ccf1c480 msgr2=0x55e1ccf28ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 -- 172.21.15.18:0/1344375036 shutdown_connections 2024-04-16T17:14:26.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 1 -- 172.21.15.18:0/1344375036 wait complete. 2024-04-16T17:14:26.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.587+0000 7fa191451740 20 client.28220 trim_cache size 0 max 16384 2024-04-16T17:14:26.595 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:14:26.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 -- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa16c0a3d20 msgr2=0x7fa16c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa16c0a3d20 0x7fa16c0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa180005e30 tx=0x7fa180005d80 comp rx=0 tx=0).stop 2024-04-16T17:14:26.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 -- 172.21.15.18:0/1227967908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa188108720 msgr2=0x7fa18819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa188108720 0x7fa18819ef10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa17800e9f0 tx=0x7fa17800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 -- 172.21.15.18:0/1227967908 shutdown_connections 2024-04-16T17:14:26.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa16c0a3d20 0x7fa16c0a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa18810a590 0x7fa1881a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1881090c0 0x7fa18819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 --2- 172.21.15.18:0/1227967908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa188108720 0x7fa18819ef10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.588+0000 7fa18e800640 1 -- 172.21.15.18:0/1227967908 >> 172.21.15.18:0/1227967908 conn(0x7fa188077e70 msgr2=0x7fa188078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.589+0000 7fa18e800640 1 -- 172.21.15.18:0/1227967908 shutdown_connections 2024-04-16T17:14:26.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.589+0000 7fa18e800640 1 -- 172.21.15.18:0/1227967908 wait complete. 2024-04-16T17:14:26.612 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:14:26.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 366001 3.69 MB/sec execute 399 sec latency 105.440 ms 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.803+0000 7fb01ca00640 1 -- 172.21.15.18:0/877363826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 msgr2=0x7fb018076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.803+0000 7fb01ca00640 1 --2- 172.21.15.18:0/877363826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 0x7fb018076910 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb008009930 tx=0x7fb00802bc30 comp rx=0 tx=0).stop 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.803+0000 7fb01ca00640 1 -- 172.21.15.18:0/877363826 shutdown_connections 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.803+0000 7fb01ca00640 1 --2- 172.21.15.18:0/877363826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb018112cb0 0x7fb0181150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.803+0000 7fb01ca00640 1 --2- 172.21.15.18:0/877363826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb018076e50 0x7fb018112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.803+0000 7fb01ca00640 1 --2- 172.21.15.18:0/877363826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 0x7fb018076910 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.803+0000 7fb01ca00640 1 -- 172.21.15.18:0/877363826 >> 172.21.15.18:0/877363826 conn(0x7fb0180ffae0 msgr2=0x7fb018101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.803+0000 7fb01ca00640 1 -- 172.21.15.18:0/877363826 shutdown_connections 2024-04-16T17:14:26.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.804+0000 7fb01ca00640 1 -- 172.21.15.18:0/877363826 wait complete. 2024-04-16T17:14:26.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.805+0000 7fb01ca00640 1 Processor -- start 2024-04-16T17:14:26.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.805+0000 7fb01ca00640 1 -- start start 2024-04-16T17:14:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.805+0000 7fb01ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 0x7fb01819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.805+0000 7fb01ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb018076e50 0x7fb01819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.805+0000 7fb01ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb018112cb0 0x7fb0181a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.805+0000 7fb01ca00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb018118850 con 0x7fb018076e50 2024-04-16T17:14:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.805+0000 7fb01ca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181186d0 con 0x7fb018112cb0 2024-04-16T17:14:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.805+0000 7fb01ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181189d0 con 0x7fb018076530 2024-04-16T17:14:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb016800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 0x7fb01819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb016800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 0x7fb01819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41688/0 (socket says 172.21.15.18:41688) 2024-04-16T17:14:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb016800640 1 -- 172.21.15.18:0/3126132217 learned_addr learned my addr 172.21.15.18:0/3126132217 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb015e00640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb018076e50 0x7fb01819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb017200640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb018112cb0 0x7fb0181a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb016800640 1 -- 172.21.15.18:0/3126132217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb018112cb0 msgr2=0x7fb0181a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb016800640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb018112cb0 0x7fb0181a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb016800640 1 -- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb018076e50 msgr2=0x7fb01819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb016800640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb018076e50 0x7fb01819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.806+0000 7fb016800640 1 -- 172.21.15.18:0/3126132217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0181a3d40 con 0x7fb018076530 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.807+0000 7fb016800640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 0x7fb01819ed50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb008009900 tx=0x7fb008004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.807+0000 7fb00f400640 1 -- 172.21.15.18:0/3126132217 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008043020 con 0x7fb018076530 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.807+0000 7fb01ca00640 1 -- 172.21.15.18:0/3126132217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0181a3fd0 con 0x7fb018076530 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.807+0000 7fb00f400640 1 -- 172.21.15.18:0/3126132217 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0080054e0 con 0x7fb018076530 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.807+0000 7fb00f400640 1 -- 172.21.15.18:0/3126132217 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008003bb0 con 0x7fb018076530 2024-04-16T17:14:26.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.808+0000 7fb01ca00640 1 -- 172.21.15.18:0/3126132217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0181ab8c0 con 0x7fb018076530 2024-04-16T17:14:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.811+0000 7fb00f400640 1 -- 172.21.15.18:0/3126132217 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb008046020 con 0x7fb018076530 2024-04-16T17:14:26.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.814+0000 7fb020001740 1 Processor -- start 2024-04-16T17:14:26.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.814+0000 7fb020001740 1 -- start start 2024-04-16T17:14:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.814+0000 7fb00f400640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faffc0a3d20 0x7faffc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.815+0000 7fb00f400640 1 -- 172.21.15.18:0/3126132217 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fb0080f2b50 con 0x7fb018076530 2024-04-16T17:14:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.815+0000 7fb017200640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faffc0a3d20 0x7faffc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb020001740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a30402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb020001740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3048ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb020001740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a304b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb020001740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f7a2fa5020 con 0x55f7a303ff10 2024-04-16T17:14:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb020001740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f7a2f4b460 con 0x55f7a3040830 2024-04-16T17:14:26.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb020001740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f7a2fa0fe0 con 0x55f7a3049410 2024-04-16T17:14:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb016800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a30402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb016800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a30402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58946/0 (socket says 172.21.15.18:58946) 2024-04-16T17:14:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb016800640 1 -- 172.21.15.18:0/3668391425 learned_addr learned my addr 172.21.15.18:0/3668391425 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.816+0000 7fb017200640 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a304b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.817+0000 7fb015e00640 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3048ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.817+0000 7fb016800640 1 -- 172.21.15.18:0/3668391425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 msgr2=0x55f7a304b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.817+0000 7fb016800640 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a304b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.817+0000 7fb016800640 1 -- 172.21.15.18:0/3668391425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 msgr2=0x55f7a3048ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.817+0000 7fb016800640 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3048ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.817+0000 7fb016800640 1 -- 172.21.15.18:0/3668391425 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a2f53040 con 0x55f7a303ff10 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.817+0000 7fb015e00640 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3048ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.818+0000 7fb017200640 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a304b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.818+0000 7fb016800640 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a30402f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb0080babf0 tx=0x7fb0080035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52feebe070683e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.818+0000 7fb017200640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faffc0a3d20 0x7faffc0a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb018069ba0 tx=0x7fb010009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.818+0000 7fb003400640 1 -- 172.21.15.18:0/3668391425 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008005690 con 0x55f7a303ff10 2024-04-16T17:14:26.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.818+0000 7fb003400640 1 -- 172.21.15.18:0/3668391425 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0080bee90 con 0x55f7a303ff10 2024-04-16T17:14:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.818+0000 7fb003400640 1 -- 172.21.15.18:0/3668391425 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0080f00b0 con 0x55f7a303ff10 2024-04-16T17:14:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.819+0000 7fb020001740 1 -- 172.21.15.18:0/3668391425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 msgr2=0x55f7a30402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.819+0000 7fb020001740 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a30402f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb0080babf0 tx=0x7fb0080035f0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.820+0000 7fb020001740 1 -- 172.21.15.18:0/3668391425 shutdown_connections 2024-04-16T17:14:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.820+0000 7fb020001740 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a304b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.820+0000 7fb020001740 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3048ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.820+0000 7fb020001740 1 --2- 172.21.15.18:0/3668391425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a30402f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.820+0000 7fb020001740 1 -- 172.21.15.18:0/3668391425 >> 172.21.15.18:0/3668391425 conn(0x55f7a303a640 msgr2=0x55f7a303aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.820+0000 7fb020001740 1 -- 172.21.15.18:0/3668391425 shutdown_connections 2024-04-16T17:14:26.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.820+0000 7fb020001740 1 -- 172.21.15.18:0/3668391425 wait complete. 2024-04-16T17:14:26.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.821+0000 7fb020001740 1 -- start start 2024-04-16T17:14:26.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.821+0000 7fb020001740 1 Processor -- start 2024-04-16T17:14:26.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.822+0000 7fb020001740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3046400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.822+0000 7fb020001740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3046940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.822+0000 7fb020001740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a3046e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.822+0000 7fb020001740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f7a304bfc0 con 0x55f7a303ff10 2024-04-16T17:14:26.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.822+0000 7fb020001740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f7a2f5e830 con 0x55f7a3040830 2024-04-16T17:14:26.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.822+0000 7fb020001740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f7a304c140 con 0x55f7a3049410 2024-04-16T17:14:26.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.822+0000 7fb020001740 10 client.0 _ll_register_callbacks cb 0x55f7a2f66150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:26.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.823+0000 7fb017200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a3046e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.823+0000 7fb015e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3046940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.823+0000 7fb015e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3046940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55088/0 (socket says 172.21.15.18:55088) 2024-04-16T17:14:26.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.823+0000 7fb017200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a3046e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:41708/0 (socket says 172.21.15.18:41708) 2024-04-16T17:14:26.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.823+0000 7fb015e00640 1 -- 172.21.15.18:0/768382765 learned_addr learned my addr 172.21.15.18:0/768382765 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.823+0000 7fb016800640 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3046400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.824+0000 7fb016800640 1 -- 172.21.15.18:0/768382765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 msgr2=0x55f7a3046e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.824+0000 7fb016800640 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a3046e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.824+0000 7fb016800640 1 -- 172.21.15.18:0/768382765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 msgr2=0x55f7a3046940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.824+0000 7fb016800640 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3046940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.824+0000 7fb016800640 1 -- 172.21.15.18:0/768382765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a2f62920 con 0x55f7a303ff10 2024-04-16T17:14:26.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.824+0000 7fb017200640 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a3046e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-16T17:14:26.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.824+0000 7fb016800640 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3046400 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb0080babf0 tx=0x7fb008003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e1901073edc912 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.824+0000 7faff7e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:14:26.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.825+0000 7faff7e00640 1 -- 172.21.15.18:0/768382765 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0080db020 con 0x55f7a303ff10 2024-04-16T17:14:26.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.825+0000 7faff7e00640 1 -- 172.21.15.18:0/768382765 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fb00802cb90 con 0x55f7a303ff10 2024-04-16T17:14:26.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.825+0000 7fb020001740 1 -- 172.21.15.18:0/768382765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0080bee90 con 0x55f7a303ff10 2024-04-16T17:14:26.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.825+0000 7faff7e00640 1 -- 172.21.15.18:0/768382765 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb008005690 con 0x55f7a303ff10 2024-04-16T17:14:26.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.825+0000 7fb020001740 10 client.17946 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:26.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.825+0000 7fb020001740 1 -- 172.21.15.18:0/768382765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7a2f307f0 con 0x55f7a303ff10 2024-04-16T17:14:26.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.827+0000 7faff7e00640 1 -- 172.21.15.18:0/768382765 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0080bee90 con 0x55f7a303ff10 2024-04-16T17:14:26.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.827+0000 7faff7e00640 1 -- 172.21.15.18:0/768382765 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb0080c7080 con 0x55f7a303ff10 2024-04-16T17:14:26.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.827+0000 7fb020001740 10 client.17946 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:26.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.827+0000 7fb020001740 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:26.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.827+0000 7fb020001740 20 client.17946 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:26.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.827+0000 7fb020001740 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7a3059270 0x55f7a305b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.827+0000 7fb020001740 4 client.17946 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:26.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.827+0000 7fb020001740 1 -- 172.21.15.18:0/768382765 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7a2fb2a40 con 0x55f7a3059270 2024-04-16T17:14:26.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.828+0000 7fb015e00640 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7a3059270 0x55f7a305b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.829+0000 7fb015e00640 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7a3059270 0x55f7a305b660 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c68367c8a02dadf server_cookie=aa07cad0a30c0d5e in_seq=0 out_seq=0 2024-04-16T17:14:26.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.829+0000 7faff7e00640 10 client.17946 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:26.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.829+0000 7faff7e00640 1 -- 172.21.15.18:0/768382765 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7a2fb2a40 con 0x55f7a3059270 2024-04-16T17:14:26.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.829+0000 7faff7e00640 10 client.17946 handle_command_reply: tid=0 2024-04-16T17:14:26.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 client.17946 shutdown 2024-04-16T17:14:26.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 -- 172.21.15.18:0/768382765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 msgr2=0x55f7a3046400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3046400 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb0080babf0 tx=0x7fb008003f80 comp rx=0 tx=0).stop 2024-04-16T17:14:26.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 -- 172.21.15.18:0/768382765 shutdown_connections 2024-04-16T17:14:26.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7a3059270 0x55f7a305b660 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3049410 0x55f7a3046e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3046940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 --2- 172.21.15.18:0/768382765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3046400 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7fb020001740 1 -- 172.21.15.18:0/768382765 >> 172.21.15.18:0/768382765 conn(0x55f7a303a640 msgr2=0x55f7a303c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.830+0000 7faff7e00640 0 client.17946 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:26.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.831+0000 7fb020001740 1 -- 172.21.15.18:0/768382765 shutdown_connections 2024-04-16T17:14:26.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.831+0000 7fb020001740 1 -- 172.21.15.18:0/768382765 wait complete. 2024-04-16T17:14:26.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.831+0000 7fb020001740 20 client.17946 trim_cache size 0 max 16384 2024-04-16T17:14:26.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.856+0000 7fb020001740 1 Processor -- start 2024-04-16T17:14:26.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.856+0000 7fb020001740 1 -- start start 2024-04-16T17:14:26.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a303ff10 0x55f7a3048ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3054d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a3049410 0x55f7a3060d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b8390 con 0x55f7a3049410 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b5c60 con 0x55f7a3040830 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b3ad0 con 0x55f7a303ff10 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3054d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb017200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a3049410 0x55f7a3060d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3054d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51298/0 (socket says 172.21.15.18:51298) 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 -- 172.21.15.18:0/1919992787 learned_addr learned my addr 172.21.15.18:0/1919992787 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 -- 172.21.15.18:0/1919992787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a303ff10 msgr2=0x55f7a3048ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 --2- 172.21.15.18:0/1919992787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a303ff10 0x55f7a3048ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 -- 172.21.15.18:0/1919992787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a3049410 msgr2=0x55f7a3060d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 --2- 172.21.15.18:0/1919992787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a3049410 0x55f7a3060d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 -- 172.21.15.18:0/1919992787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a2cb73f0 con 0x55f7a3040830 2024-04-16T17:14:26.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb015e00640 1 --2- 172.21.15.18:0/1919992787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3054d20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb004000c00 tx=0x7fb004005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1050a8f1fdeecb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb002a00640 1 -- 172.21.15.18:0/1919992787 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004004250 con 0x55f7a3040830 2024-04-16T17:14:26.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb002a00640 1 -- 172.21.15.18:0/1919992787 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0040043f0 con 0x55f7a3040830 2024-04-16T17:14:26.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb002a00640 1 -- 172.21.15.18:0/1919992787 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004020110 con 0x55f7a3040830 2024-04-16T17:14:26.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- 172.21.15.18:0/1919992787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 msgr2=0x55f7a3054d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 --2- 172.21.15.18:0/1919992787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3054d20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb004000c00 tx=0x7fb004005f90 comp rx=0 tx=0).stop 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- 172.21.15.18:0/1919992787 shutdown_connections 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 --2- 172.21.15.18:0/1919992787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a3049410 0x55f7a3060d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 --2- 172.21.15.18:0/1919992787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3040830 0x55f7a3054d20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 --2- 172.21.15.18:0/1919992787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a303ff10 0x55f7a3048ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- 172.21.15.18:0/1919992787 >> 172.21.15.18:0/1919992787 conn(0x55f7a303a640 msgr2=0x55f7a30416f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- 172.21.15.18:0/1919992787 shutdown_connections 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- 172.21.15.18:0/1919992787 wait complete. 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.857+0000 7fb020001740 1 -- start start 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 Processor -- start 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3051300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3040830 0x55f7a304c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3049410 0x55f7a304ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b4120 con 0x55f7a303ff10 2024-04-16T17:14:26.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b5590 con 0x55f7a3049410 2024-04-16T17:14:26.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181b47f0 con 0x55f7a3040830 2024-04-16T17:14:26.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 10 client.0 _ll_register_callbacks cb 0x7fb0080c2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:26.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb016800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3051300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb015e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3040830 0x55f7a304c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb016800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3051300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53448/0 (socket says 172.21.15.18:53448) 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb016800640 1 -- 172.21.15.18:0/357467639 learned_addr learned my addr 172.21.15.18:0/357467639 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb015e00640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3040830 0x55f7a304c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36462/0 (socket says 172.21.15.18:36462) 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb017200640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3049410 0x55f7a304ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb015e00640 1 -- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3049410 msgr2=0x55f7a304ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb015e00640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3049410 0x55f7a304ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb015e00640 1 -- 172.21.15.18:0/357467639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 msgr2=0x55f7a3051300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb015e00640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3051300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb015e00640 1 -- 172.21.15.18:0/357467639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a2d817d0 con 0x55f7a3040830 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb016800640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3051300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb017200640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3049410 0x55f7a304ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb015e00640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3040830 0x55f7a304c790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55f7a303a5d0 tx=0x7fb004004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da52a9c768cb663 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7faff7e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7faff7e00640 1 -- 172.21.15.18:0/357467639 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00402e060 con 0x55f7a3040830 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 -- 172.21.15.18:0/357467639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0040043f0 con 0x55f7a3040830 2024-04-16T17:14:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7faff7e00640 1 -- 172.21.15.18:0/357467639 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fb004020310 con 0x55f7a3040830 2024-04-16T17:14:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7faff7e00640 1 -- 172.21.15.18:0/357467639 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb004035050 con 0x55f7a3040830 2024-04-16T17:14:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 10 client.28111 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 -- 172.21.15.18:0/357467639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7a30bf120 con 0x55f7a3040830 2024-04-16T17:14:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7faff7e00640 1 -- 172.21.15.18:0/357467639 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0040043f0 con 0x55f7a3040830 2024-04-16T17:14:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7faff7e00640 1 -- 172.21.15.18:0/357467639 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb004003b70 con 0x55f7a3040830 2024-04-16T17:14:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 10 client.28111 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 10 client.28111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 20 client.28111 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7a30bf850 0x55f7a30c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 4 client.28111 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.860+0000 7fb020001740 1 -- 172.21.15.18:0/357467639 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7a2fac8b0 con 0x55f7a30bf850 2024-04-16T17:14:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb016800640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7a30bf850 0x55f7a30c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb016800640 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7a30bf850 0x55f7a30c1c70 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4819a47af695cd1c server_cookie=b4308ff0efb63ea in_seq=0 out_seq=0 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7faff7e00640 10 client.28111 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7faff7e00640 1 -- 172.21.15.18:0/357467639 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7a2fac8b0 con 0x55f7a30bf850 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7faff7e00640 10 client.28111 handle_command_reply: tid=0 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 client.28111 shutdown 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 -- 172.21.15.18:0/357467639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3040830 msgr2=0x55f7a304c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3040830 0x55f7a304c790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55f7a303a5d0 tx=0x7fb004004730 comp rx=0 tx=0).stop 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 -- 172.21.15.18:0/357467639 shutdown_connections 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7a30bf850 0x55f7a30c1c70 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7a3049410 0x55f7a304ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7a3040830 0x55f7a304c790 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 --2- 172.21.15.18:0/357467639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f7a303ff10 0x55f7a3051300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7faff7e00640 0 client.28111 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 -- 172.21.15.18:0/357467639 >> 172.21.15.18:0/357467639 conn(0x55f7a303a640 msgr2=0x55f7a3038e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 -- 172.21.15.18:0/357467639 shutdown_connections 2024-04-16T17:14:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 1 -- 172.21.15.18:0/357467639 wait complete. 2024-04-16T17:14:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.873+0000 7fb020001740 20 client.28111 trim_cache size 0 max 16384 2024-04-16T17:14:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:14:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 -- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faffc0a3d20 msgr2=0x7faffc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faffc0a3d20 0x7faffc0a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb018069ba0 tx=0x7fb010009500 comp rx=0 tx=0).stop 2024-04-16T17:14:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 -- 172.21.15.18:0/3126132217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 msgr2=0x7fb01819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 0x7fb01819ed50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb008009900 tx=0x7fb008004360 comp rx=0 tx=0).stop 2024-04-16T17:14:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 -- 172.21.15.18:0/3126132217 shutdown_connections 2024-04-16T17:14:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faffc0a3d20 0x7faffc0a61e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb018112cb0 0x7fb0181a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb018076e50 0x7fb01819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 --2- 172.21.15.18:0/3126132217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb018076530 0x7fb01819ed50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 -- 172.21.15.18:0/3126132217 >> 172.21.15.18:0/3126132217 conn(0x7fb0180ffae0 msgr2=0x7fb018101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 -- 172.21.15.18:0/3126132217 shutdown_connections 2024-04-16T17:14:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:26.874+0000 7fb01ca00640 1 -- 172.21.15.18:0/3126132217 wait complete. 2024-04-16T17:14:26.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:26 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/707668940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:27.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:26 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/707668940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:27.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 366371 3.68 MB/sec execute 400 sec latency 105.880 ms 2024-04-16T17:14:27.901 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:14:27.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:14:27.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:14:27.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:27 smithi022 ceph-mon[36023]: pgmap v304: 129 pgs: 129 active+clean; 51 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-04-16T17:14:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:27 smithi043 ceph-mon[36901]: pgmap v304: 129 pgs: 129 active+clean; 51 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-04-16T17:14:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.125+0000 7ff023e00640 1 -- 172.21.15.18:0/1815968151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 msgr2=0x7ff01c102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.125+0000 7ff023e00640 1 --2- 172.21.15.18:0/1815968151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 0x7ff01c102610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff010009860 tx=0x7ff01002bb70 comp rx=0 tx=0).stop 2024-04-16T17:14:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.126+0000 7ff023e00640 1 -- 172.21.15.18:0/1815968151 shutdown_connections 2024-04-16T17:14:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.126+0000 7ff023e00640 1 --2- 172.21.15.18:0/1815968151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff01c107300 0x7ff01c10eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.126+0000 7ff023e00640 1 --2- 172.21.15.18:0/1815968151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff01c102be0 0x7ff01c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.126+0000 7ff023e00640 1 --2- 172.21.15.18:0/1815968151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 0x7ff01c102610 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.126+0000 7ff023e00640 1 -- 172.21.15.18:0/1815968151 >> 172.21.15.18:0/1815968151 conn(0x7ff01c0fdab0 msgr2=0x7ff01c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.126+0000 7ff023e00640 1 -- 172.21.15.18:0/1815968151 shutdown_connections 2024-04-16T17:14:28.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.126+0000 7ff023e00640 1 -- 172.21.15.18:0/1815968151 wait complete. 2024-04-16T17:14:28.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.127+0000 7ff023e00640 1 Processor -- start 2024-04-16T17:14:28.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.127+0000 7ff023e00640 1 -- start start 2024-04-16T17:14:28.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.127+0000 7ff023e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 0x7ff01c074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.127+0000 7ff023e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff01c107300 0x7ff01c074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.127+0000 7ff023e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff01c074d80 0x7ff01c1a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.127+0000 7ff023e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1123c0 con 0x7ff01c074d80 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.127+0000 7ff023e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c112240 con 0x7ff01c107300 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.127+0000 7ff023e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c112540 con 0x7ff01c102230 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff020a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff01c107300 0x7ff01c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff021e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff01c074d80 0x7ff01c1a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff020a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff01c107300 0x7ff01c074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51306/0 (socket says 172.21.15.18:51306) 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff020a00640 1 -- 172.21.15.18:0/641328429 learned_addr learned my addr 172.21.15.18:0/641328429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff021400640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 0x7ff01c074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff021e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff01c074d80 0x7ff01c1a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53470/0 (socket says 172.21.15.18:53470) 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff021400640 1 -- 172.21.15.18:0/641328429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff01c107300 msgr2=0x7ff01c074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff021400640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff01c107300 0x7ff01c074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff021400640 1 -- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff01c074d80 msgr2=0x7ff01c1a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff021400640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff01c074d80 0x7ff01c1a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.128+0000 7ff021400640 1 -- 172.21.15.18:0/641328429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c1a71f0 con 0x7ff01c102230 2024-04-16T17:14:28.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.129+0000 7ff021400640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 0x7ff01c074300 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff01002c0b0 tx=0x7ff0100045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.129+0000 7ff01a000640 1 -- 172.21.15.18:0/641328429 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010041020 con 0x7ff01c102230 2024-04-16T17:14:28.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.129+0000 7ff01a000640 1 -- 172.21.15.18:0/641328429 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff01002c9f0 con 0x7ff01c102230 2024-04-16T17:14:28.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.129+0000 7ff01a000640 1 -- 172.21.15.18:0/641328429 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010039040 con 0x7ff01c102230 2024-04-16T17:14:28.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.129+0000 7ff023e00640 1 -- 172.21.15.18:0/641328429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c1a7480 con 0x7ff01c102230 2024-04-16T17:14:28.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.129+0000 7ff023e00640 1 -- 172.21.15.18:0/641328429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1a7920 con 0x7ff01c102230 2024-04-16T17:14:28.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.132+0000 7ff026b95740 1 Processor -- start 2024-04-16T17:14:28.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff026b95740 1 -- start start 2024-04-16T17:14:28.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff026b95740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506826080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff026b95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x55950682ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff026b95740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x559506831590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff026b95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55950678acb0 con 0x5595068265c0 2024-04-16T17:14:28.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff026b95740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559506731370 con 0x55950682f1a0 2024-04-16T17:14:28.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff026b95740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559506786c70 con 0x559506825ca0 2024-04-16T17:14:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff021e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506826080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff021e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506826080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36486/0 (socket says 172.21.15.18:36486) 2024-04-16T17:14:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff021e00640 1 -- 172.21.15.18:0/4224305161 learned_addr learned my addr 172.21.15.18:0/4224305161 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.133+0000 7ff020a00640 1 --2- 172.21.15.18:0/4224305161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x559506831590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.134+0000 7ff021e00640 1 -- 172.21.15.18:0/4224305161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 msgr2=0x559506831590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.134+0000 7ff021e00640 1 --2- 172.21.15.18:0/4224305161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x559506831590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.134+0000 7ff021e00640 1 -- 172.21.15.18:0/4224305161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 msgr2=0x55950682ec60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:14:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.134+0000 7ff021e00640 1 --2- 172.21.15.18:0/4224305161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x55950682ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.134+0000 7ff021e00640 1 -- 172.21.15.18:0/4224305161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559506750910 con 0x559506825ca0 2024-04-16T17:14:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.134+0000 7ff01a000640 1 -- 172.21.15.18:0/641328429 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff010044020 con 0x7ff01c102230 2024-04-16T17:14:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.134+0000 7ff021e00640 1 --2- 172.21.15.18:0/4224305161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506826080 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff014009f50 tx=0x7ff0140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ca00da3ed3f9ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.135+0000 7ff00e600640 1 -- 172.21.15.18:0/4224305161 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014004d20 con 0x559506825ca0 2024-04-16T17:14:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.135+0000 7ff00e600640 1 -- 172.21.15.18:0/4224305161 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff014004050 con 0x559506825ca0 2024-04-16T17:14:28.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.135+0000 7ff00e600640 1 -- 172.21.15.18:0/4224305161 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014005260 con 0x559506825ca0 2024-04-16T17:14:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 -- 172.21.15.18:0/4224305161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 msgr2=0x559506826080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 --2- 172.21.15.18:0/4224305161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506826080 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff014009f50 tx=0x7ff0140118d0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 -- 172.21.15.18:0/4224305161 shutdown_connections 2024-04-16T17:14:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 --2- 172.21.15.18:0/4224305161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x559506831590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 --2- 172.21.15.18:0/4224305161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x55950682ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 --2- 172.21.15.18:0/4224305161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506826080 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff01a000640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff0040a3dd0 0x7ff0040a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 -- 172.21.15.18:0/4224305161 >> 172.21.15.18:0/4224305161 conn(0x5595068203d0 msgr2=0x5595068207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff01a000640 1 -- 172.21.15.18:0/641328429 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7ff0100f21f0 con 0x7ff01c102230 2024-04-16T17:14:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 -- 172.21.15.18:0/4224305161 shutdown_connections 2024-04-16T17:14:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff021e00640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff0040a3dd0 0x7ff0040a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.136+0000 7ff026b95740 1 -- 172.21.15.18:0/4224305161 wait complete. 2024-04-16T17:14:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.137+0000 7ff026b95740 1 -- start start 2024-04-16T17:14:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.137+0000 7ff021e00640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff0040a3dd0 0x7ff0040a6290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff014009d60 tx=0x7ff014004720 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.137+0000 7ff026b95740 1 Processor -- start 2024-04-16T17:14:28.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff026b95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559506825ca0 0x55950682c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff026b95740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595068265c0 0x55950682c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff026b95740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x55950682ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff026b95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559506831f90 con 0x559506825ca0 2024-04-16T17:14:28.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff026b95740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559506744730 con 0x55950682f1a0 2024-04-16T17:14:28.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff026b95740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559506832110 con 0x5595068265c0 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff026b95740 10 client.0 _ll_register_callbacks cb 0x5595067131e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff021400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559506825ca0 0x55950682c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff021400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559506825ca0 0x55950682c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53482/0 (socket says 172.21.15.18:53482) 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.138+0000 7ff021400640 1 -- 172.21.15.18:0/4162487868 learned_addr learned my addr 172.21.15.18:0/4162487868 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff021e00640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x55950682ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff020a00640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595068265c0 0x55950682c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff021e00640 1 -- 172.21.15.18:0/4162487868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595068265c0 msgr2=0x55950682c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff021e00640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595068265c0 0x55950682c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff021e00640 1 -- 172.21.15.18:0/4162487868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559506825ca0 msgr2=0x55950682c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff021e00640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559506825ca0 0x55950682c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff021e00640 1 -- 172.21.15.18:0/4162487868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595065657e0 con 0x55950682f1a0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff020a00640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595068265c0 0x55950682c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff021400640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559506825ca0 0x55950682c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.139+0000 7ff021e00640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x55950682ccb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff01401b040 tx=0x7ff014005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13cd6d03582aa37a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.140+0000 7ff002a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.140+0000 7ff002a00640 1 -- 172.21.15.18:0/4162487868 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014034020 con 0x55950682f1a0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.140+0000 7ff026b95740 1 -- 172.21.15.18:0/4162487868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff014004050 con 0x55950682f1a0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.140+0000 7ff002a00640 1 -- 172.21.15.18:0/4162487868 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7ff01400f480 con 0x55950682f1a0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.140+0000 7ff002a00640 1 -- 172.21.15.18:0/4162487868 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff01403f050 con 0x55950682f1a0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.140+0000 7ff026b95740 10 client.28250 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.140+0000 7ff026b95740 1 -- 172.21.15.18:0/4162487868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595065c5f70 con 0x55950682f1a0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff002a00640 1 -- 172.21.15.18:0/4162487868 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff014004050 con 0x55950682f1a0 2024-04-16T17:14:28.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff002a00640 1 -- 172.21.15.18:0/4162487868 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff0140236a0 con 0x55950682f1a0 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff026b95740 10 client.28250 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff026b95740 10 client.28250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff026b95740 20 client.28250 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff026b95740 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55950683f520 0x559506841910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff026b95740 4 client.28250 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff026b95740 1 -- 172.21.15.18:0/4162487868 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595067986d0 con 0x55950683f520 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.141+0000 7ff021400640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55950683f520 0x559506841910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.142+0000 7ff002a00640 10 client.28250 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:28.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.142+0000 7ff021400640 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55950683f520 0x559506841910 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1a66c76839ec12b server_cookie=1d053a5e10619a3d in_seq=0 out_seq=0 2024-04-16T17:14:28.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff002a00640 1 -- 172.21.15.18:0/4162487868 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595067986d0 con 0x55950683f520 2024-04-16T17:14:28.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff002a00640 10 client.28250 handle_command_reply: tid=0 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 client.28250 shutdown 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 -- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 msgr2=0x55950682ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x55950682ccb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff01401b040 tx=0x7ff014005bd0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 -- 172.21.15.18:0/4162487868 shutdown_connections 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55950683f520 0x559506841910 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x55950682ccb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595068265c0 0x55950682c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff002a00640 0 client.28250 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 --2- 172.21.15.18:0/4162487868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559506825ca0 0x55950682c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 -- 172.21.15.18:0/4162487868 >> 172.21.15.18:0/4162487868 conn(0x5595068203d0 msgr2=0x559506822120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 -- 172.21.15.18:0/4162487868 shutdown_connections 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 1 -- 172.21.15.18:0/4162487868 wait complete. 2024-04-16T17:14:28.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.143+0000 7ff026b95740 20 client.28250 trim_cache size 0 max 16384 2024-04-16T17:14:28.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.169+0000 7ff026b95740 1 Processor -- start 2024-04-16T17:14:28.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.169+0000 7ff026b95740 1 -- start start 2024-04-16T17:14:28.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.169+0000 7ff026b95740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x55950682e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff026b95740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595068265c0 0x55950683aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff026b95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55950682f1a0 0x559506847130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff026b95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b4120 con 0x55950682f1a0 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff026b95740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b19f0 con 0x5595068265c0 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff026b95740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1af860 con 0x559506825ca0 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x55950682e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x55950682e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36508/0 (socket says 172.21.15.18:36508) 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021400640 1 -- 172.21.15.18:0/690197796 learned_addr learned my addr 172.21.15.18:0/690197796 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff020a00640 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595068265c0 0x55950683aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021e00640 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55950682f1a0 0x559506847130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021400640 1 -- 172.21.15.18:0/690197796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595068265c0 msgr2=0x55950683aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021400640 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595068265c0 0x55950683aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021400640 1 -- 172.21.15.18:0/690197796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55950682f1a0 msgr2=0x559506847130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021400640 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55950682f1a0 0x559506847130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021400640 1 -- 172.21.15.18:0/690197796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950645d170 con 0x559506825ca0 2024-04-16T17:14:28.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.170+0000 7ff021e00640 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55950682f1a0 0x559506847130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:28.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.171+0000 7ff021400640 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x55950682e960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff0100bddc0 tx=0x7ff01003f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a29fbdc8c070cea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.171+0000 7ff00dc00640 1 -- 172.21.15.18:0/690197796 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100ba0d0 con 0x559506825ca0 2024-04-16T17:14:28.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.171+0000 7ff00dc00640 1 -- 172.21.15.18:0/690197796 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0100be530 con 0x559506825ca0 2024-04-16T17:14:28.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.171+0000 7ff00dc00640 1 -- 172.21.15.18:0/690197796 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010034860 con 0x559506825ca0 2024-04-16T17:14:28.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 -- 172.21.15.18:0/690197796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 msgr2=0x55950682e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x55950682e960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff0100bddc0 tx=0x7ff01003f040 comp rx=0 tx=0).stop 2024-04-16T17:14:28.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 -- 172.21.15.18:0/690197796 shutdown_connections 2024-04-16T17:14:28.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55950682f1a0 0x559506847130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595068265c0 0x55950683aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 --2- 172.21.15.18:0/690197796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x55950682e960 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 -- 172.21.15.18:0/690197796 >> 172.21.15.18:0/690197796 conn(0x5595068203d0 msgr2=0x55950682ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 -- 172.21.15.18:0/690197796 shutdown_connections 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.172+0000 7ff026b95740 1 -- 172.21.15.18:0/690197796 wait complete. 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.173+0000 7ff026b95740 1 -- start start 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.173+0000 7ff026b95740 1 Processor -- start 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.173+0000 7ff026b95740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506829540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff026b95740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x559506829a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff026b95740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x5595068a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff026b95740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1afeb0 con 0x5595068265c0 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff026b95740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b1320 con 0x55950682f1a0 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff026b95740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b0580 con 0x559506825ca0 2024-04-16T17:14:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff026b95740 10 client.0 _ll_register_callbacks cb 0x7ff014004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff020a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x559506829a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff020a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x559506829a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53496/0 (socket says 172.21.15.18:53496) 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff020a00640 1 -- 172.21.15.18:0/2725750524 learned_addr learned my addr 172.21.15.18:0/2725750524 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff021e00640 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x5595068a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff021400640 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506829540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff020a00640 1 -- 172.21.15.18:0/2725750524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 msgr2=0x559506829540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff020a00640 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506829540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff020a00640 1 -- 172.21.15.18:0/2725750524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 msgr2=0x5595068a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff020a00640 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x5595068a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff020a00640 1 -- 172.21.15.18:0/2725750524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595068a4ee0 con 0x5595068265c0 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.174+0000 7ff021400640 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506829540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.175+0000 7ff020a00640 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x559506829a80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff01c075d90 tx=0x7ff008003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1964a7ba58ed471c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.175+0000 7ff002a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.175+0000 7ff002a00640 1 -- 172.21.15.18:0/2725750524 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008010de0 con 0x5595068265c0 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.175+0000 7ff002a00640 1 -- 172.21.15.18:0/2725750524 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7ff0080047a0 con 0x5595068265c0 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.175+0000 7ff026b95740 1 -- 172.21.15.18:0/2725750524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0100be530 con 0x5595068265c0 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.175+0000 7ff002a00640 1 -- 172.21.15.18:0/2725750524 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff008028050 con 0x5595068265c0 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.175+0000 7ff026b95740 10 client.17964 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.175+0000 7ff026b95740 1 -- 172.21.15.18:0/2725750524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595068a60f0 con 0x5595068265c0 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff002a00640 1 -- 172.21.15.18:0/2725750524 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0100be530 con 0x5595068265c0 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff002a00640 1 -- 172.21.15.18:0/2725750524 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff00801c680 con 0x5595068265c0 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff026b95740 10 client.17964 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff026b95740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff026b95740 20 client.17964 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff026b95740 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55950683f520 0x5595068a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff026b95740 4 client.17964 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:14:28.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff026b95740 1 -- 172.21.15.18:0/2725750524 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559506792540 con 0x55950683f520 2024-04-16T17:14:28.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.176+0000 7ff021400640 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55950683f520 0x5595068a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.177+0000 7ff021400640 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55950683f520 0x5595068a8900 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4af78db65afebc server_cookie=c7cdef782316881 in_seq=0 out_seq=0 2024-04-16T17:14:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.177+0000 7ff002a00640 10 client.17964 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.177+0000 7ff002a00640 1 -- 172.21.15.18:0/2725750524 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559506792540 con 0x55950683f520 2024-04-16T17:14:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.177+0000 7ff002a00640 10 client.17964 handle_command_reply: tid=0 2024-04-16T17:14:28.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 client.17964 shutdown 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 -- 172.21.15.18:0/2725750524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 msgr2=0x559506829a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x559506829a80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff01c075d90 tx=0x7ff008003e40 comp rx=0 tx=0).stop 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 -- 172.21.15.18:0/2725750524 shutdown_connections 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55950683f520 0x5595068a8900 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55950682f1a0 0x5595068a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595068265c0 0x559506829a80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 --2- 172.21.15.18:0/2725750524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559506825ca0 0x559506829540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 -- 172.21.15.18:0/2725750524 >> 172.21.15.18:0/2725750524 conn(0x5595068203d0 msgr2=0x5595068216d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff002a00640 0 client.17964 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 -- 172.21.15.18:0/2725750524 shutdown_connections 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 1 -- 172.21.15.18:0/2725750524 wait complete. 2024-04-16T17:14:28.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.178+0000 7ff026b95740 20 client.17964 trim_cache size 0 max 16384 2024-04-16T17:14:28.183 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:14:28.183 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:14:28.183 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "5b55b586-ca03-4122-b3c6-7679a4c0dc32", 2024-04-16T17:14:28.183 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:14:28.183 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 -- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff0040a3dd0 msgr2=0x7ff0040a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff0040a3dd0 0x7ff0040a6290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff014009d60 tx=0x7ff014004720 comp rx=0 tx=0).stop 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 -- 172.21.15.18:0/641328429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 msgr2=0x7ff01c074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 0x7ff01c074300 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff01002c0b0 tx=0x7ff0100045b0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 -- 172.21.15.18:0/641328429 shutdown_connections 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff0040a3dd0 0x7ff0040a6290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff01c074d80 0x7ff01c1a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff01c107300 0x7ff01c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 --2- 172.21.15.18:0/641328429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c102230 0x7ff01c074300 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.179+0000 7ff023e00640 1 -- 172.21.15.18:0/641328429 >> 172.21.15.18:0/641328429 conn(0x7ff01c0fdab0 msgr2=0x7ff01c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.180+0000 7ff023e00640 1 -- 172.21.15.18:0/641328429 shutdown_connections 2024-04-16T17:14:28.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.180+0000 7ff023e00640 1 -- 172.21.15.18:0/641328429 wait complete. 2024-04-16T17:14:28.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:14:28.202 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:14:28.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 -- 172.21.15.18:0/3839397124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac104280 msgr2=0x7f9cac104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3839397124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac104280 0x7f9cac104660 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009900 tx=0x7f9c9c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:14:28.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 -- 172.21.15.18:0/3839397124 shutdown_connections 2024-04-16T17:14:28.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3839397124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac109350 0x7f9cac110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3839397124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cac104c30 0x7f9cac108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3839397124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac104280 0x7f9cac104660 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 -- 172.21.15.18:0/3839397124 >> 172.21.15.18:0/3839397124 conn(0x7f9cac0ffae0 msgr2=0x7f9cac101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 -- 172.21.15.18:0/3839397124 shutdown_connections 2024-04-16T17:14:28.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.406+0000 7f9cabe00640 1 -- 172.21.15.18:0/3839397124 wait complete. 2024-04-16T17:14:28.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.407+0000 7f9cabe00640 1 Processor -- start 2024-04-16T17:14:28.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.407+0000 7f9cabe00640 1 -- start start 2024-04-16T17:14:28.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.407+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cac104280 0x7f9cac1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac109350 0x7f9cac1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac19a780 0x7f9cac19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9cabe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1143e0 con 0x7f9cac19a780 2024-04-16T17:14:28.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9cabe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac114260 con 0x7f9cac109350 2024-04-16T17:14:28.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9cabe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac114560 con 0x7f9cac104280 2024-04-16T17:14:28.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caa000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac109350 0x7f9cac1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cac104280 0x7f9cac1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caa000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac109350 0x7f9cac1a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51366/0 (socket says 172.21.15.18:51366) 2024-04-16T17:14:28.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caa000640 1 -- 172.21.15.18:0/3919197998 learned_addr learned my addr 172.21.15.18:0/3919197998 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9cab400640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac19a780 0x7f9cac19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caa000640 1 -- 172.21.15.18:0/3919197998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cac104280 msgr2=0x7f9cac1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caa000640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cac104280 0x7f9cac1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caa000640 1 -- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac19a780 msgr2=0x7f9cac19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caa000640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac19a780 0x7f9cac19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9caa000640 1 -- 172.21.15.18:0/3919197998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac19b310 con 0x7f9cac109350 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.408+0000 7f9cab400640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac19a780 0x7f9cac19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.409+0000 7f9caa000640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac109350 0x7f9cac1a0b50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800d770 tx=0x7f9c9800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.409+0000 7f9ca3400640 1 -- 172.21.15.18:0/3919197998 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98017070 con 0x7f9cac109350 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.409+0000 7f9ca3400640 1 -- 172.21.15.18:0/3919197998 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c98009c30 con 0x7f9cac109350 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.409+0000 7f9cabe00640 1 -- 172.21.15.18:0/3919197998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac19b600 con 0x7f9cac109350 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.409+0000 7f9ca3400640 1 -- 172.21.15.18:0/3919197998 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800f620 con 0x7f9cac109350 2024-04-16T17:14:28.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.409+0000 7f9cabe00640 1 -- 172.21.15.18:0/3919197998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac1a7830 con 0x7f9cac109350 2024-04-16T17:14:28.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.412+0000 7f9cb4a2a740 1 Processor -- start 2024-04-16T17:14:28.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.413+0000 7f9cb4a2a740 1 -- start start 2024-04-16T17:14:28.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.413+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.413+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685bf5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.413+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 0x55b685bf8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.413+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b685b4a120 con 0x55b685be4cb0 2024-04-16T17:14:28.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.413+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b685af0340 con 0x55b685bf60a0 2024-04-16T17:14:28.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.413+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b685b460e0 con 0x55b685be4390 2024-04-16T17:14:28.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9caaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685bf5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9cab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9caaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685bf5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53514/0 (socket says 172.21.15.18:53514) 2024-04-16T17:14:28.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9caaa00640 1 -- 172.21.15.18:0/1774877735 learned_addr learned my addr 172.21.15.18:0/1774877735 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9ca3400640 1 -- 172.21.15.18:0/3919197998 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9c9801a070 con 0x7f9cac109350 2024-04-16T17:14:28.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9cab400640 1 -- 172.21.15.18:0/1774877735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 msgr2=0x55b685bf8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9cab400640 1 --2- 172.21.15.18:0/1774877735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 0x55b685bf8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9cab400640 1 -- 172.21.15.18:0/1774877735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 msgr2=0x55b685bf5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9cab400640 1 --2- 172.21.15.18:0/1774877735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685bf5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.414+0000 7f9cab400640 1 -- 172.21.15.18:0/1774877735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6859267d0 con 0x55b685be4390 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.415+0000 7f9caaa00640 1 --2- 172.21.15.18:0/1774877735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685bf5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.415+0000 7f9cab400640 1 --2- 172.21.15.18:0/1774877735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be4770 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009f00 tx=0x7f9ca400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c172c7a5a75d8f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.415+0000 7f9c97e00640 1 -- 172.21.15.18:0/1774877735 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4015070 con 0x55b685be4390 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.415+0000 7f9c97e00640 1 -- 172.21.15.18:0/1774877735 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ca401ab40 con 0x55b685be4390 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.415+0000 7f9c97e00640 1 -- 172.21.15.18:0/1774877735 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4011c30 con 0x55b685be4390 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9ca3400640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9c900a3da0 0x7f9c900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/1774877735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 msgr2=0x55b685be4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/1774877735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be4770 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009f00 tx=0x7f9ca400f730 comp rx=0 tx=0).stop 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/1774877735 shutdown_connections 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/1774877735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 0x55b685bf8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/1774877735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685bf5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9ca3400640 1 -- 172.21.15.18:0/3919197998 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f9c980d0710 con 0x7f9cac109350 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/1774877735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be4770 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/1774877735 >> 172.21.15.18:0/1774877735 conn(0x55b685bdeac0 msgr2=0x55b685bdeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/1774877735 shutdown_connections 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9caaa00640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9c900a3da0 0x7f9c900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.416+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/1774877735 wait complete. 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.417+0000 7f9cb4a2a740 1 -- start start 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.417+0000 7f9cb4a2a740 1 Processor -- start 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.417+0000 7f9caaa00640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9c900a3da0 0x7f9c900a6260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0096c0 tx=0x7f9c9c005e80 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4390 0x55b685bf3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4cb0 0x55b685bf36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685bf60a0 0x55b685bf3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b685bf8e90 con 0x55b685be4390 2024-04-16T17:14:28.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b685b03710 con 0x55b685be4cb0 2024-04-16T17:14:28.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b685bf9010 con 0x55b685bf60a0 2024-04-16T17:14:28.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cb4a2a740 10 client.0 _ll_register_callbacks cb 0x55b685afeaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685bf60a0 0x55b685bf3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9caaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4390 0x55b685bf3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685bf60a0 0x55b685bf3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36556/0 (socket says 172.21.15.18:36556) 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9cab400640 1 -- 172.21.15.18:0/3317110270 learned_addr learned my addr 172.21.15.18:0/3317110270 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.418+0000 7f9caa000640 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4cb0 0x55b685bf36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9cab400640 1 -- 172.21.15.18:0/3317110270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4cb0 msgr2=0x55b685bf36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9cab400640 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4cb0 0x55b685bf36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9cab400640 1 -- 172.21.15.18:0/3317110270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4390 msgr2=0x55b685bf3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9cab400640 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4390 0x55b685bf3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9cab400640 1 -- 172.21.15.18:0/3317110270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68585c3f0 con 0x55b685bf60a0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9cab400640 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685bf60a0 0x55b685bf3bf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006010 tx=0x7f9ca4011660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef83358be057e649 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9c94c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9c94c00640 1 -- 172.21.15.18:0/3317110270 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4015040 con 0x55b685bf60a0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9c94c00640 1 -- 172.21.15.18:0/3317110270 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f9ca40119b0 con 0x55b685bf60a0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.419+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/3317110270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ca401ab40 con 0x55b685bf60a0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9c94c00640 1 -- 172.21.15.18:0/3317110270 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ca4030050 con 0x55b685bf60a0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9cb4a2a740 10 client.28141 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/3317110270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b68581c0f0 con 0x55b685bf60a0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9c94c00640 1 -- 172.21.15.18:0/3317110270 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ca401ab40 con 0x55b685bf60a0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9c94c00640 1 -- 172.21.15.18:0/3317110270 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9ca402b030 con 0x55b685bf60a0 2024-04-16T17:14:28.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9cb4a2a740 10 client.28141 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:28.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9cb4a2a740 10 client.28141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:28.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9cb4a2a740 20 client.28141 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:28.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b685c06480 0x55b685c08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9cb4a2a740 4 client.28141 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:28.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.420+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/3317110270 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b685b57b40 con 0x55b685c06480 2024-04-16T17:14:28.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.421+0000 7f9caaa00640 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b685c06480 0x55b685c08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.421+0000 7f9caaa00640 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b685c06480 0x55b685c08870 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810b4b6e14e2bd3 server_cookie=e85f32ce62881baf in_seq=0 out_seq=0 2024-04-16T17:14:28.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.421+0000 7f9c94c00640 10 client.28141 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:28.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.422+0000 7f9c94c00640 1 -- 172.21.15.18:0/3317110270 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b685b57b40 con 0x55b685c06480 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.422+0000 7f9c94c00640 10 client.28141 handle_command_reply: tid=0 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.422+0000 7f9cb4a2a740 1 client.28141 shutdown 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.422+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/3317110270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685bf60a0 msgr2=0x55b685bf3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.422+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685bf60a0 0x55b685bf3bf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006010 tx=0x7f9ca4011660 comp rx=0 tx=0).stop 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/3317110270 shutdown_connections 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b685c06480 0x55b685c08870 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685bf60a0 0x55b685bf3bf0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4cb0 0x55b685bf36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/3317110270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4390 0x55b685bf3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9c94c00640 0 client.28141 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/3317110270 >> 172.21.15.18:0/3317110270 conn(0x55b685bdeac0 msgr2=0x55b685be0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/3317110270 shutdown_connections 2024-04-16T17:14:28.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/3317110270 wait complete. 2024-04-16T17:14:28.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.423+0000 7f9cb4a2a740 20 client.28141 trim_cache size 0 max 16384 2024-04-16T17:14:28.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.446+0000 7f9cb4a2a740 1 Processor -- start 2024-04-16T17:14:28.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.446+0000 7f9cb4a2a740 1 -- start start 2024-04-16T17:14:28.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4390 0x55b685bf58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4cb0 0x55b685c01d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685bf60a0 0x55b685bdd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1afc90 con 0x55b685bf60a0 2024-04-16T17:14:28.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1ad560 con 0x55b685be4390 2024-04-16T17:14:28.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1ab3d0 con 0x55b685be4cb0 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9caaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4390 0x55b685bf58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9caa000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4cb0 0x55b685c01d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9cab400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685bf60a0 0x55b685bdd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9caa000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4cb0 0x55b685c01d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36564/0 (socket says 172.21.15.18:36564) 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9caaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4390 0x55b685bf58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51396/0 (socket says 172.21.15.18:51396) 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.447+0000 7f9caa000640 1 -- 172.21.15.18:0/65640865 learned_addr learned my addr 172.21.15.18:0/65640865 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9caa000640 1 -- 172.21.15.18:0/65640865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4390 msgr2=0x55b685bf58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9caa000640 1 --2- 172.21.15.18:0/65640865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4390 0x55b685bf58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9caa000640 1 -- 172.21.15.18:0/65640865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685bf60a0 msgr2=0x55b685bdd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9caa000640 1 --2- 172.21.15.18:0/65640865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685bf60a0 0x55b685bdd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9caa000640 1 -- 172.21.15.18:0/65640865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b685be9120 con 0x55b685be4cb0 2024-04-16T17:14:28.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9cab400640 1 --2- 172.21.15.18:0/65640865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685bf60a0 0x55b685bdd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:28.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9caa000640 1 --2- 172.21.15.18:0/65640865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4cb0 0x55b685c01d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9c980986a0 tx=0x7f9c98005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bae5eeea2fbbdc30 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9c97400640 1 -- 172.21.15.18:0/65640865 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98021040 con 0x55b685be4cb0 2024-04-16T17:14:28.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9c97400640 1 -- 172.21.15.18:0/65640865 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c980a05f0 con 0x55b685be4cb0 2024-04-16T17:14:28.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.448+0000 7f9c97400640 1 -- 172.21.15.18:0/65640865 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980d0260 con 0x55b685be4cb0 2024-04-16T17:14:28.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.449+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/65640865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4cb0 msgr2=0x55b685c01d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.449+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/65640865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4cb0 0x55b685c01d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9c980986a0 tx=0x7f9c98005b20 comp rx=0 tx=0).stop 2024-04-16T17:14:28.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.449+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/65640865 shutdown_connections 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.449+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/65640865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685bf60a0 0x55b685bdd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.449+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/65640865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4cb0 0x55b685c01d80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.449+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/65640865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685be4390 0x55b685bf58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.449+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/65640865 >> 172.21.15.18:0/65640865 conn(0x55b685bdeac0 msgr2=0x55b685bdd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.450+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/65640865 shutdown_connections 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.450+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/65640865 wait complete. 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.450+0000 7f9cb4a2a740 1 -- start start 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.450+0000 7f9cb4a2a740 1 Processor -- start 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685be7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cb4a2a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 0x55b685be80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1aba20 con 0x55b685be4cb0 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1ace90 con 0x55b685bf60a0 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cb4a2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1ac0f0 con 0x55b685be4390 2024-04-16T17:14:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cb4a2a740 10 client.0 _ll_register_callbacks cb 0x7f9c9c005dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:28.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cab400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 0x55b685be80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9caaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cab400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 0x55b685be80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51410/0 (socket says 172.21.15.18:51410) 2024-04-16T17:14:28.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9cab400640 1 -- 172.21.15.18:0/315761229 learned_addr learned my addr 172.21.15.18:0/315761229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:28.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9caa000640 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685be7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9caaa00640 1 -- 172.21.15.18:0/315761229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 msgr2=0x55b685be80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9caaa00640 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 0x55b685be80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9caaa00640 1 -- 172.21.15.18:0/315761229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 msgr2=0x55b685be7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.451+0000 7f9caaa00640 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685be7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.452+0000 7f9caaa00640 1 -- 172.21.15.18:0/315761229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b685c6b710 con 0x55b685be4390 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.452+0000 7f9caa000640 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685be7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.452+0000 7f9caaa00640 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be7650 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55b685be3e60 tx=0x7f9c9c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97028870f28faa1b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.452+0000 7f9c94c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.452+0000 7f9c94c00640 1 -- 172.21.15.18:0/315761229 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c004ad0 con 0x55b685be4390 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.452+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/315761229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c980a05f0 con 0x55b685be4390 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.452+0000 7f9c94c00640 1 -- 172.21.15.18:0/315761229 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f9c9c055330 con 0x55b685be4390 2024-04-16T17:14:28.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9c94c00640 1 -- 172.21.15.18:0/315761229 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c9c065050 con 0x55b685be4390 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9cb4a2a740 10 client.28147 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/315761229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b685c6ca60 con 0x55b685be4390 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9c94c00640 1 -- 172.21.15.18:0/315761229 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c980a05f0 con 0x55b685be4390 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9c94c00640 1 -- 172.21.15.18:0/315761229 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9c9c0054c0 con 0x55b685be4390 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9cb4a2a740 10 client.28147 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9cb4a2a740 10 client.28147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9cb4a2a740 20 client.28147 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b685c6d160 0x55b685c6f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9cb4a2a740 4 client.28147 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.453+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/315761229 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b685b519b0 con 0x55b685c6d160 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.454+0000 7f9caa000640 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b685c6d160 0x55b685c6f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.454+0000 7f9caa000640 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b685c6d160 0x55b685c6f580 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3325ea7aaff32eac server_cookie=f3f95db1a10ab633 in_seq=0 out_seq=0 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.454+0000 7f9c94c00640 10 client.28147 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.454+0000 7f9c94c00640 1 -- 172.21.15.18:0/315761229 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b685b519b0 con 0x55b685c6d160 2024-04-16T17:14:28.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.454+0000 7f9c94c00640 10 client.28147 handle_command_reply: tid=0 2024-04-16T17:14:28.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.454+0000 7f9cb4a2a740 1 client.28147 shutdown 2024-04-16T17:14:28.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/315761229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 msgr2=0x55b685be7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be7650 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55b685be3e60 tx=0x7f9c9c009260 comp rx=0 tx=0).stop 2024-04-16T17:14:28.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/315761229 shutdown_connections 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b685c6d160 0x55b685c6f580 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b685bf60a0 0x55b685be80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b685be4cb0 0x55b685be7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 --2- 172.21.15.18:0/315761229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b685be4390 0x55b685be7650 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9c94c00640 0 client.28147 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/315761229 >> 172.21.15.18:0/315761229 conn(0x55b685bdeac0 msgr2=0x55b685bddb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/315761229 shutdown_connections 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 1 -- 172.21.15.18:0/315761229 wait complete. 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.455+0000 7f9cb4a2a740 20 client.28147 trim_cache size 0 max 16384 2024-04-16T17:14:28.458 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout: "5b55b586-ca03-4122-b3c6-7679a4c0dc32": { 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "5b55b586-ca03-4122-b3c6-7679a4c0dc32", 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:14:28.459 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 -- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9c900a3da0 msgr2=0x7f9c900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9c900a3da0 0x7f9c900a6260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0096c0 tx=0x7f9c9c005e80 comp rx=0 tx=0).stop 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 -- 172.21.15.18:0/3919197998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac109350 msgr2=0x7f9cac1a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac109350 0x7f9cac1a0b50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800d770 tx=0x7f9c9800dc30 comp rx=0 tx=0).stop 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 -- 172.21.15.18:0/3919197998 shutdown_connections 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9c900a3da0 0x7f9c900a6260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cac19a780 0x7f9cac19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cac109350 0x7f9cac1a0b50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 --2- 172.21.15.18:0/3919197998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cac104280 0x7f9cac1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.456+0000 7f9cabe00640 1 -- 172.21.15.18:0/3919197998 >> 172.21.15.18:0/3919197998 conn(0x7f9cac0ffae0 msgr2=0x7f9cac100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:28.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.457+0000 7f9cabe00640 1 -- 172.21.15.18:0/3919197998 shutdown_connections 2024-04-16T17:14:28.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:28.457+0000 7f9cabe00640 1 -- 172.21.15.18:0/3919197998 wait complete. 2024-04-16T17:14:28.477 INFO:tasks.cephfs.filesystem:scrub status for tag:5b55b586-ca03-4122-b3c6-7679a4c0dc32 - {'path': '/', 'tag': '5b55b586-ca03-4122-b3c6-7679a4c0dc32', 'options': 'recursive,force'} 2024-04-16T17:14:28.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 366999 3.68 MB/sec execute 401 sec latency 41.941 ms 2024-04-16T17:14:29.474 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:25 smithi018 ceph-mon[31776]: pgmap v303: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-16T17:14:29.474 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:26 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/707668940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:29.474 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:27 smithi018 ceph-mon[31776]: pgmap v304: 129 pgs: 129 active+clean; 51 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-04-16T17:14:29.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 367874 3.68 MB/sec execute 402 sec latency 29.264 ms 2024-04-16T17:14:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:29 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:14:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:29 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:29 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:14:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:29 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:29 smithi022 ceph-mon[36023]: pgmap v305: 129 pgs: 129 active+clean; 51 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-04-16T17:14:29.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:29 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:14:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:29 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:14:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:29 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:29 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:14:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:29 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:29 smithi043 ceph-mon[36901]: pgmap v305: 129 pgs: 129 active+clean; 51 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-04-16T17:14:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:29 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:14:30.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 369029 3.70 MB/sec execute 403 sec latency 14.740 ms 2024-04-16T17:14:31.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 369292 3.69 MB/sec execute 404 sec latency 83.941 ms 2024-04-16T17:14:31.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:31 smithi022 ceph-mon[36023]: pgmap v306: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-04-16T17:14:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:31 smithi043 ceph-mon[36901]: pgmap v306: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-04-16T17:14:32.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:29 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:14:32.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:29 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:32.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:29 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:14:32.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:29 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:32.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:29 smithi018 ceph-mon[31776]: pgmap v305: 129 pgs: 129 active+clean; 51 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-04-16T17:14:32.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:29 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:14:32.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:31 smithi018 ceph-mon[31776]: pgmap v306: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-04-16T17:14:32.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 369441 3.68 MB/sec execute 405 sec latency 90.234 ms 2024-04-16T17:14:33.635 INFO:tasks.workunit.client.0.smithi018.stdout: 1 370121 3.68 MB/sec execute 406 sec latency 40.323 ms 2024-04-16T17:14:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:33 smithi022 ceph-mon[36023]: pgmap v307: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-04-16T17:14:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:33 smithi043 ceph-mon[36901]: pgmap v307: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-04-16T17:14:34.636 INFO:tasks.workunit.client.0.smithi018.stdout: 1 370865 3.68 MB/sec execute 407 sec latency 47.721 ms 2024-04-16T17:14:35.636 INFO:tasks.workunit.client.0.smithi018.stdout: 1 371896 3.69 MB/sec execute 408 sec latency 29.958 ms 2024-04-16T17:14:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:35 smithi022 ceph-mon[36023]: pgmap v308: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:35 smithi043 ceph-mon[36901]: pgmap v308: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:36.636 INFO:tasks.workunit.client.0.smithi018.stdout: 1 372409 3.69 MB/sec execute 409 sec latency 59.867 ms 2024-04-16T17:14:37.636 INFO:tasks.workunit.client.0.smithi018.stdout: 1 372679 3.68 MB/sec execute 410 sec latency 82.328 ms 2024-04-16T17:14:37.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:37 smithi022 ceph-mon[36023]: pgmap v309: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:14:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:37 smithi043 ceph-mon[36901]: pgmap v309: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:14:38.636 INFO:tasks.workunit.client.0.smithi018.stdout: 1 373232 3.68 MB/sec execute 411 sec latency 77.603 ms 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 -- 172.21.15.18:0/3326120802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c102a40 msgr2=0x7f891c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 --2- 172.21.15.18:0/3326120802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c102a40 0x7f891c106a80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8904009860 tx=0x7f890402bb70 comp rx=0 tx=0).stop 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 -- 172.21.15.18:0/3326120802 shutdown_connections 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 --2- 172.21.15.18:0/3326120802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c107160 0x7f891c10ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 --2- 172.21.15.18:0/3326120802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c102a40 0x7f891c106a80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 --2- 172.21.15.18:0/3326120802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c102090 0x7f891c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 -- 172.21.15.18:0/3326120802 >> 172.21.15.18:0/3326120802 conn(0x7f891c0fd910 msgr2=0x7f891c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 -- 172.21.15.18:0/3326120802 shutdown_connections 2024-04-16T17:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.762+0000 7f891b200640 1 -- 172.21.15.18:0/3326120802 wait complete. 2024-04-16T17:14:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.763+0000 7f891b200640 1 Processor -- start 2024-04-16T17:14:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.763+0000 7f891b200640 1 -- start start 2024-04-16T17:14:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f891b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c102090 0x7f891c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f891b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c107160 0x7f891c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f891b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19a770 0x7f891c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f891b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f891c1121d0 con 0x7f891c102090 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f891b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f891c112050 con 0x7f891c107160 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f891b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f891c112350 con 0x7f891c19a770 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8911400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c107160 0x7f891c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8919e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c102090 0x7f891c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8911400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c107160 0x7f891c1a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52302/0 (socket says 172.21.15.18:52302) 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8911400640 1 -- 172.21.15.18:0/2073556663 learned_addr learned my addr 172.21.15.18:0/2073556663 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f891a800640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19a770 0x7f891c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8911400640 1 -- 172.21.15.18:0/2073556663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19a770 msgr2=0x7f891c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8911400640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19a770 0x7f891c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8911400640 1 -- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c102090 msgr2=0x7f891c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8911400640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c102090 0x7f891c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f8911400640 1 -- 172.21.15.18:0/2073556663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c19b370 con 0x7f891c107160 2024-04-16T17:14:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.764+0000 7f891a800640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19a770 0x7f891c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.765+0000 7f8911400640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c107160 0x7f891c1a0ba0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8904037040 tx=0x7f89040045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.765+0000 7f8913400640 1 -- 172.21.15.18:0/2073556663 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904041020 con 0x7f891c107160 2024-04-16T17:14:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.765+0000 7f8913400640 1 -- 172.21.15.18:0/2073556663 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890402c9f0 con 0x7f891c107160 2024-04-16T17:14:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.765+0000 7f891b200640 1 -- 172.21.15.18:0/2073556663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c19b600 con 0x7f891c107160 2024-04-16T17:14:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.765+0000 7f8913400640 1 -- 172.21.15.18:0/2073556663 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904039040 con 0x7f891c107160 2024-04-16T17:14:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.765+0000 7f891b200640 1 -- 172.21.15.18:0/2073556663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c1a76f0 con 0x7f891c107160 2024-04-16T17:14:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.768+0000 7f8913400640 1 -- 172.21.15.18:0/2073556663 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8904044020 con 0x7f891c107160 2024-04-16T17:14:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.769+0000 7f8923e45740 1 Processor -- start 2024-04-16T17:14:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.770+0000 7f8923e45740 1 -- start start 2024-04-16T17:14:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.770+0000 7f8913400640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d00 0x7f89000a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.770+0000 7f8913400640 1 -- 172.21.15.18:0/2073556663 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f89040f2430 con 0x7f891c107160 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.770+0000 7f8923e45740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c8e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.770+0000 7f8923e45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c8ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.770+0000 7f8923e45740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c9a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.771+0000 7f8923e45740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557347bf3780 con 0x557347c8f7f0 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.771+0000 7f891a800640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d00 0x7f89000a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.771+0000 7f8923e45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557347b99f40 con 0x557347c8ea80 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.771+0000 7f8919e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c8e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.771+0000 7f8923e45740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557347bef740 con 0x557347c98000 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.771+0000 7f8919e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c8e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56394/0 (socket says 172.21.15.18:56394) 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.771+0000 7f8919e00640 1 -- 172.21.15.18:0/3761022279 learned_addr learned my addr 172.21.15.18:0/3761022279 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.771+0000 7f8911400640 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c8ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f8911400640 1 -- 172.21.15.18:0/3761022279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 msgr2=0x557347c9a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f891a800640 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c9a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f8911400640 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c9a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f8911400640 1 -- 172.21.15.18:0/3761022279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 msgr2=0x557347c8e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f8911400640 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c8e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f8911400640 1 -- 172.21.15.18:0/3761022279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557347906210 con 0x557347c8ea80 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f8919e00640 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c8e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f891a800640 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c9a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f8911400640 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c8ef00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f89040ef820 tx=0x7f8904032820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17220e2f1d511f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f890b400640 1 -- 172.21.15.18:0/3761022279 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89040efe60 con 0x557347c8ea80 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f890b400640 1 -- 172.21.15.18:0/3761022279 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89040be070 con 0x557347c8ea80 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.772+0000 7f890b400640 1 -- 172.21.15.18:0/3761022279 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89040bc370 con 0x557347c8ea80 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.773+0000 7f8923e45740 1 -- 172.21.15.18:0/3761022279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 msgr2=0x557347c8ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.773+0000 7f8923e45740 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c8ef00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f89040ef820 tx=0x7f8904032820 comp rx=0 tx=0).stop 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.773+0000 7f891a800640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d00 0x7f89000a61c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x557347c8a900 tx=0x7f8914009c10 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.773+0000 7f8923e45740 1 -- 172.21.15.18:0/3761022279 shutdown_connections 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.773+0000 7f8923e45740 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c9a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.773+0000 7f8923e45740 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c8ef00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.773+0000 7f8923e45740 1 --2- 172.21.15.18:0/3761022279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c8e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.773+0000 7f8923e45740 1 -- 172.21.15.18:0/3761022279 >> 172.21.15.18:0/3761022279 conn(0x557347c295d0 msgr2=0x557347c89770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.775+0000 7f8923e45740 1 -- 172.21.15.18:0/3761022279 shutdown_connections 2024-04-16T17:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.775+0000 7f8923e45740 1 -- 172.21.15.18:0/3761022279 wait complete. 2024-04-16T17:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.775+0000 7f8923e45740 1 -- start start 2024-04-16T17:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.775+0000 7f8923e45740 1 Processor -- start 2024-04-16T17:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8923e45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c94ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8923e45740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c8f7f0 0x557347c95420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8923e45740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c98000 0x557347c95960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8923e45740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557347cabb00 con 0x557347c98000 2024-04-16T17:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8923e45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557347bad390 con 0x557347c8ea80 2024-04-16T17:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8923e45740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557347cabc80 con 0x557347c8f7f0 2024-04-16T17:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8923e45740 10 client.0 _ll_register_callbacks cb 0x557347b96f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8919e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c94ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8919e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c94ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52310/0 (socket says 172.21.15.18:52310) 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8919e00640 1 -- 172.21.15.18:0/1850380528 learned_addr learned my addr 172.21.15.18:0/1850380528 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f891a800640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c98000 0x557347c95960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.776+0000 7f8911400640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c8f7f0 0x557347c95420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f8911400640 1 -- 172.21.15.18:0/1850380528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 msgr2=0x557347c94ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f8911400640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c94ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f8911400640 1 -- 172.21.15.18:0/1850380528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c98000 msgr2=0x557347c95960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f8911400640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c98000 0x557347c95960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f8911400640 1 -- 172.21.15.18:0/1850380528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573479d2620 con 0x557347c8f7f0 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f8919e00640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c94ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f891a800640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c98000 0x557347c95960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f8911400640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c8f7f0 0x557347c95420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x557347cab990 tx=0x7f890402c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=647b73c737effbd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f88ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f88ffe00640 1 -- 172.21.15.18:0/1850380528 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89040d6070 con 0x557347c8f7f0 2024-04-16T17:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f8923e45740 1 -- 172.21.15.18:0/1850380528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89040be070 con 0x557347c8f7f0 2024-04-16T17:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.777+0000 7f88ffe00640 1 -- 172.21.15.18:0/1850380528 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f89040ef990 con 0x557347c8f7f0 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f88ffe00640 1 -- 172.21.15.18:0/1850380528 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89040efe60 con 0x557347c8f7f0 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f8923e45740 10 client.28156 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f8923e45740 1 -- 172.21.15.18:0/1850380528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557347cabb00 con 0x557347c8f7f0 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f88ffe00640 1 -- 172.21.15.18:0/1850380528 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89040be070 con 0x557347c8f7f0 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f88ffe00640 1 -- 172.21.15.18:0/1850380528 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f890403c030 con 0x557347c8f7f0 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f8923e45740 10 client.28156 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f8923e45740 10 client.28156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f8923e45740 20 client.28156 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f8923e45740 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557347cb8d80 0x557347cbb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f8923e45740 4 client.28156 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.778+0000 7f8923e45740 1 -- 172.21.15.18:0/1850380528 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557347c011a0 con 0x557347cb8d80 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.779+0000 7f8919e00640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557347cb8d80 0x557347cbb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.779+0000 7f88ffe00640 10 client.28156 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.779+0000 7f8919e00640 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557347cb8d80 0x557347cbb170 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b151d9e5a42c2fea server_cookie=6f81825f01280715 in_seq=0 out_seq=0 2024-04-16T17:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.780+0000 7f88ffe00640 1 -- 172.21.15.18:0/1850380528 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557347c011a0 con 0x557347cb8d80 2024-04-16T17:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.780+0000 7f88ffe00640 10 client.28156 handle_command_reply: tid=0 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 client.28156 shutdown 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 -- 172.21.15.18:0/1850380528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c8f7f0 msgr2=0x557347c95420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c8f7f0 0x557347c95420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x557347cab990 tx=0x7f890402c390 comp rx=0 tx=0).stop 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 -- 172.21.15.18:0/1850380528 shutdown_connections 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557347cb8d80 0x557347cbb170 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c98000 0x557347c95960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c8f7f0 0x557347c95420 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 --2- 172.21.15.18:0/1850380528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c94ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f88ffe00640 0 client.28156 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 -- 172.21.15.18:0/1850380528 >> 172.21.15.18:0/1850380528 conn(0x557347c295d0 msgr2=0x557347c8b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 -- 172.21.15.18:0/1850380528 shutdown_connections 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 1 -- 172.21.15.18:0/1850380528 wait complete. 2024-04-16T17:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.781+0000 7f8923e45740 20 client.28156 trim_cache size 0 max 16384 2024-04-16T17:14:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.797+0000 7f8923e45740 1 Processor -- start 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8923e45740 1 -- start start 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8923e45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c97610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8923e45740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c94040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8923e45740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c95780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8923e45740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f891c1ab6e0 con 0x557347c8f7f0 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8923e45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f891c1a8fb0 con 0x557347c8ea80 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8923e45740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f891c19bc50 con 0x557347c98000 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8919e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c97610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8911400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c94040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8919e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c97610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52320/0 (socket says 172.21.15.18:52320) 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f8919e00640 1 -- 172.21.15.18:0/2619901097 learned_addr learned my addr 172.21.15.18:0/2619901097 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.798+0000 7f891a800640 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c95780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f8919e00640 1 -- 172.21.15.18:0/2619901097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 msgr2=0x557347c95780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f8919e00640 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c95780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f8919e00640 1 -- 172.21.15.18:0/2619901097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 msgr2=0x557347c94040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f8919e00640 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c94040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f8919e00640 1 -- 172.21.15.18:0/2619901097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557347c8dd50 con 0x557347c8ea80 2024-04-16T17:14:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f8911400640 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c94040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f8919e00640 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c97610 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f890c005bc0 tx=0x7f890c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7cf7bfb6e6ee6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f890aa00640 1 -- 172.21.15.18:0/2619901097 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c012110 con 0x557347c8ea80 2024-04-16T17:14:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f890aa00640 1 -- 172.21.15.18:0/2619901097 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c025070 con 0x557347c8ea80 2024-04-16T17:14:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.799+0000 7f890aa00640 1 -- 172.21.15.18:0/2619901097 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c003af0 con 0x557347c8ea80 2024-04-16T17:14:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 -- 172.21.15.18:0/2619901097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 msgr2=0x557347c97610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c97610 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f890c005bc0 tx=0x7f890c005f90 comp rx=0 tx=0).stop 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 -- 172.21.15.18:0/2619901097 shutdown_connections 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347c95780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347c94040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 --2- 172.21.15.18:0/2619901097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347c97610 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 -- 172.21.15.18:0/2619901097 >> 172.21.15.18:0/2619901097 conn(0x557347c295d0 msgr2=0x557347c87f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 -- 172.21.15.18:0/2619901097 shutdown_connections 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.800+0000 7f8923e45740 1 -- 172.21.15.18:0/2619901097 wait complete. 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.801+0000 7f8923e45740 1 -- start start 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.801+0000 7f8923e45740 1 Processor -- start 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8923e45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347cac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8923e45740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347cac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8923e45740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347cb0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8923e45740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f891c073d80 con 0x557347c8f7f0 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8923e45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f891c1a88e0 con 0x557347c8ea80 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8923e45740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f891c1a7b40 con 0x557347c98000 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8923e45740 10 client.0 _ll_register_callbacks cb 0x7f890c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8919e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347cac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8919e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347cac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52334/0 (socket says 172.21.15.18:52334) 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8919e00640 1 -- 172.21.15.18:0/4212832026 learned_addr learned my addr 172.21.15.18:0/4212832026 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f891a800640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347cb0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f8911400640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347cac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f891a800640 1 -- 172.21.15.18:0/4212832026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 msgr2=0x557347cac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f891a800640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347cac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f891a800640 1 -- 172.21.15.18:0/4212832026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 msgr2=0x557347cac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.802+0000 7f891a800640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347cac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.803+0000 7f891a800640 1 -- 172.21.15.18:0/4212832026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557347cb0f90 con 0x557347c98000 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.803+0000 7f8919e00640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347cac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.803+0000 7f8911400640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347cac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.803+0000 7f891a800640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347cb0950 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8914011010 tx=0x7f8914004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55f45132de5b95f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.803+0000 7f88ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.803+0000 7f88ffe00640 1 -- 172.21.15.18:0/4212832026 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891402d060 con 0x557347c98000 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.803+0000 7f88ffe00640 1 -- 172.21.15.18:0/4212832026 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f891400e800 con 0x557347c98000 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.803+0000 7f8923e45740 1 -- 172.21.15.18:0/4212832026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890c025070 con 0x557347c98000 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f88ffe00640 1 -- 172.21.15.18:0/4212832026 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8914034050 con 0x557347c98000 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f8923e45740 10 client.28162 fetch_fsmap learned FSMap version 27 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f8923e45740 1 -- 172.21.15.18:0/4212832026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557347d1ee30 con 0x557347c98000 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f88ffe00640 1 -- 172.21.15.18:0/4212832026 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c025070 con 0x557347c98000 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f88ffe00640 1 -- 172.21.15.18:0/4212832026 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f891400b040 con 0x557347c98000 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f8923e45740 10 client.28162 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f8923e45740 10 client.28162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f8923e45740 20 client.28162 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f8923e45740 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557347d1f5b0 0x557347d219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f8923e45740 4 client.28162 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:14:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.804+0000 7f8923e45740 1 -- 172.21.15.18:0/4212832026 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557347bfb010 con 0x557347d1f5b0 2024-04-16T17:14:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.805+0000 7f8919e00640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557347d1f5b0 0x557347d219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.805+0000 7f8919e00640 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557347d1f5b0 0x557347d219d0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b7463c09eb20d9c server_cookie=4a0984b545aa4ddc in_seq=0 out_seq=0 2024-04-16T17:14:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.805+0000 7f88ffe00640 10 client.28162 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.968+0000 7f88ffe00640 1 -- 172.21.15.18:0/4212832026 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557347bfb010 con 0x557347d1f5b0 2024-04-16T17:14:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.968+0000 7f88ffe00640 10 client.28162 handle_command_reply: tid=0 2024-04-16T17:14:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.968+0000 7f8923e45740 1 client.28162 shutdown 2024-04-16T17:14:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.968+0000 7f8923e45740 1 -- 172.21.15.18:0/4212832026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 msgr2=0x557347cb0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.968+0000 7f8923e45740 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347cb0950 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8914011010 tx=0x7f8914004a40 comp rx=0 tx=0).stop 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.968+0000 7f8923e45740 1 -- 172.21.15.18:0/4212832026 shutdown_connections 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f8923e45740 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557347d1f5b0 0x557347d219d0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f8923e45740 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557347c98000 0x557347cb0950 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f8923e45740 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557347c8f7f0 0x557347cac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f8923e45740 1 --2- 172.21.15.18:0/4212832026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557347c8ea80 0x557347cac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f8923e45740 1 -- 172.21.15.18:0/4212832026 >> 172.21.15.18:0/4212832026 conn(0x557347c295d0 msgr2=0x557347c88810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f88ffe00640 0 client.28162 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f8923e45740 1 -- 172.21.15.18:0/4212832026 shutdown_connections 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f8923e45740 1 -- 172.21.15.18:0/4212832026 wait complete. 2024-04-16T17:14:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.969+0000 7f8923e45740 20 client.28162 trim_cache size 0 max 16384 2024-04-16T17:14:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:14:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:14:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:14:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:14:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 -- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d00 msgr2=0x7f89000a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d00 0x7f89000a61c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x557347c8a900 tx=0x7f8914009c10 comp rx=0 tx=0).stop 2024-04-16T17:14:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 -- 172.21.15.18:0/2073556663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c107160 msgr2=0x7f891c1a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c107160 0x7f891c1a0ba0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8904037040 tx=0x7f89040045b0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 -- 172.21.15.18:0/2073556663 shutdown_connections 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d00 0x7f89000a61c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19a770 0x7f891c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c107160 0x7f891c1a0ba0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 --2- 172.21.15.18:0/2073556663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c102090 0x7f891c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.970+0000 7f891b200640 1 -- 172.21.15.18:0/2073556663 >> 172.21.15.18:0/2073556663 conn(0x7f891c0fd910 msgr2=0x7f891c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.971+0000 7f891b200640 1 -- 172.21.15.18:0/2073556663 shutdown_connections 2024-04-16T17:14:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:38.972+0000 7f891b200640 1 -- 172.21.15.18:0/2073556663 wait complete. 2024-04-16T17:14:39.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:33 smithi018 ceph-mon[31776]: pgmap v307: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-04-16T17:14:39.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:35 smithi018 ceph-mon[31776]: pgmap v308: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:39.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:37 smithi018 ceph-mon[31776]: pgmap v309: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-16T17:14:39.636 INFO:tasks.workunit.client.0.smithi018.stdout: 1 373992 3.68 MB/sec execute 412 sec latency 114.285 ms 2024-04-16T17:14:39.668 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:14:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setuser ceph since I am not root 2024-04-16T17:14:39.668 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:14:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setgroup ceph since I am not root 2024-04-16T17:14:39.668 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:14:39 smithi022 ceph-mds[66065]: -- 172.21.15.22:0/1844358869 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56239d36da40 con 0x56239d340800 2024-04-16T17:14:39.668 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:14:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: starting mds.h at 2024-04-16T17:14:39.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:39 smithi022 ceph-mon[36023]: pgmap v310: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 922 KiB/s wr, 134 op/s 2024-04-16T17:14:39.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:39 smithi022 ceph-mon[36023]: daemon mds.h restarted 2024-04-16T17:14:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:39 smithi043 ceph-mon[36901]: pgmap v310: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 922 KiB/s wr, 134 op/s 2024-04-16T17:14:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:39 smithi043 ceph-mon[36901]: daemon mds.h restarted 2024-04-16T17:14:40.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:39 smithi018 ceph-mon[31776]: pgmap v310: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 922 KiB/s wr, 134 op/s 2024-04-16T17:14:40.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:39 smithi018 ceph-mon[31776]: daemon mds.h restarted 2024-04-16T17:14:40.636 INFO:tasks.workunit.client.0.smithi018.stdout: 1 374833 3.67 MB/sec execute 413 sec latency 37.303 ms 2024-04-16T17:14:40.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:40 smithi022 ceph-mon[36023]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T17:14:40.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:40 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:14:40.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:40 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6836/257552546,v1:172.21.15.22:6837/257552546] up:boot 2024-04-16T17:14:40.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:40 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:14:40.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:40 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:14:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:40 smithi043 ceph-mon[36901]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T17:14:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:40 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:14:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:40 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6836/257552546,v1:172.21.15.22:6837/257552546] up:boot 2024-04-16T17:14:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:40 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:14:41.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:40 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:14:41.636 INFO:tasks.workunit.client.0.smithi018.stdout: 1 375351 3.68 MB/sec execute 414 sec latency 93.569 ms 2024-04-16T17:14:42.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:41 smithi043 ceph-mon[36901]: pgmap v312: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 865 KiB/s wr, 138 op/s 2024-04-16T17:14:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:41 smithi022 ceph-mon[36023]: pgmap v312: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 865 KiB/s wr, 138 op/s 2024-04-16T17:14:42.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:40 smithi018 ceph-mon[31776]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T17:14:42.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:40 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:14:42.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:40 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6836/257552546,v1:172.21.15.22:6837/257552546] up:boot 2024-04-16T17:14:42.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:40 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:14:42.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:40 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:14:42.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:41 smithi018 ceph-mon[31776]: pgmap v312: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 865 KiB/s wr, 138 op/s 2024-04-16T17:14:42.638 INFO:tasks.workunit.client.0.smithi018.stdout: 1 375738 3.67 MB/sec execute 415 sec latency 127.535 ms 2024-04-16T17:14:43.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 376397 3.68 MB/sec execute 416 sec latency 44.953 ms 2024-04-16T17:14:44.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:43 smithi043 ceph-mon[36901]: pgmap v313: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 846 KiB/s wr, 131 op/s 2024-04-16T17:14:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:43 smithi022 ceph-mon[36023]: pgmap v313: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 846 KiB/s wr, 131 op/s 2024-04-16T17:14:44.173 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:43 smithi018 ceph-mon[31776]: pgmap v313: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 846 KiB/s wr, 131 op/s 2024-04-16T17:14:44.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 377095 3.67 MB/sec execute 417 sec latency 35.769 ms 2024-04-16T17:14:45.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 377788 3.67 MB/sec execute 418 sec latency 33.689 ms 2024-04-16T17:14:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:45 smithi043 ceph-mon[36901]: pgmap v314: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 846 KiB/s wr, 131 op/s 2024-04-16T17:14:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:45 smithi022 ceph-mon[36023]: pgmap v314: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 846 KiB/s wr, 131 op/s 2024-04-16T17:14:46.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 378300 3.67 MB/sec execute 419 sec latency 44.553 ms 2024-04-16T17:14:47.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 378566 3.66 MB/sec execute 420 sec latency 151.697 ms 2024-04-16T17:14:47.785 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:45 smithi018 ceph-mon[31776]: pgmap v314: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 846 KiB/s wr, 131 op/s 2024-04-16T17:14:47.785 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:47 smithi018 ceph-mon[31776]: pgmap v315: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:47 smithi043 ceph-mon[36901]: pgmap v315: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:47 smithi022 ceph-mon[36023]: pgmap v315: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:48.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 379580 3.67 MB/sec execute 421 sec latency 414.954 ms 2024-04-16T17:14:49.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 380229 3.67 MB/sec execute 422 sec latency 19.032 ms 2024-04-16T17:14:49.910 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:49 smithi018 ceph-mon[31776]: pgmap v316: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:49 smithi043 ceph-mon[36901]: pgmap v316: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:50.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:49 smithi022 ceph-mon[36023]: pgmap v316: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:14:50.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 381086 3.67 MB/sec execute 423 sec latency 33.435 ms 2024-04-16T17:14:51.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 381422 3.66 MB/sec execute 424 sec latency 89.748 ms 2024-04-16T17:14:52.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:51 smithi043 ceph-mon[36901]: pgmap v317: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 -- 172.21.15.18:0/791212594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d4074cd0 msgr2=0x7f13d410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 --2- 172.21.15.18:0/791212594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d4074cd0 0x7f13d410e190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009860 tx=0x7f13c002bc20 comp rx=0 tx=0).stop 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 -- 172.21.15.18:0/791212594 shutdown_connections 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 --2- 172.21.15.18:0/791212594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d410e6d0 0x7f13d4110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 --2- 172.21.15.18:0/791212594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d4074cd0 0x7f13d410e190 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 --2- 172.21.15.18:0/791212594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d40743b0 0x7f13d4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 -- 172.21.15.18:0/791212594 >> 172.21.15.18:0/791212594 conn(0x7f13d40fd940 msgr2=0x7f13d40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 -- 172.21.15.18:0/791212594 shutdown_connections 2024-04-16T17:14:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.110+0000 7f13d9800640 1 -- 172.21.15.18:0/791212594 wait complete. 2024-04-16T17:14:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.111+0000 7f13d9800640 1 Processor -- start 2024-04-16T17:14:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.111+0000 7f13d9800640 1 -- start start 2024-04-16T17:14:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d40743b0 0x7f13d41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d9800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d4074cd0 0x7f13d41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d410e6d0 0x7f13d419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d9800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41143c0 con 0x7f13d4074cd0 2024-04-16T17:14:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d9800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4114240 con 0x7f13d40743b0 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d9800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4114540 con 0x7f13d410e6d0 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d4074cd0 0x7f13d41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d4074cd0 0x7f13d41a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42524/0 (socket says 172.21.15.18:42524) 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d3400640 1 -- 172.21.15.18:0/935580253 learned_addr learned my addr 172.21.15.18:0/935580253 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d3e00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d40743b0 0x7f13d41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.112+0000 7f13d8e00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d410e6d0 0x7f13d419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d3e00640 1 -- 172.21.15.18:0/935580253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d410e6d0 msgr2=0x7f13d419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d3e00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d410e6d0 0x7f13d419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d3e00640 1 -- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d4074cd0 msgr2=0x7f13d41a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d3e00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d4074cd0 0x7f13d41a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d3e00640 1 -- 172.21.15.18:0/935580253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d419b070 con 0x7f13d40743b0 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d3400640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d4074cd0 0x7f13d41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d8e00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d410e6d0 0x7f13d419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d3e00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d40743b0 0x7f13d41a06a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f13c400e9f0 tx=0x7f13c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d0c00640 1 -- 172.21.15.18:0/935580253 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c400ccc0 con 0x7f13d40743b0 2024-04-16T17:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d0c00640 1 -- 172.21.15.18:0/935580253 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c4010910 con 0x7f13d40743b0 2024-04-16T17:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d9800640 1 -- 172.21.15.18:0/935580253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d419b360 con 0x7f13d40743b0 2024-04-16T17:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d0c00640 1 -- 172.21.15.18:0/935580253 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4018960 con 0x7f13d40743b0 2024-04-16T17:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.113+0000 7f13d9800640 1 -- 172.21.15.18:0/935580253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d41a7620 con 0x7f13d40743b0 2024-04-16T17:14:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.117+0000 7f13d0c00640 1 -- 172.21.15.18:0/935580253 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f13c4010ac0 con 0x7f13d40743b0 2024-04-16T17:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13dda2b740 1 Processor -- start 2024-04-16T17:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13dda2b740 1 -- start start 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 0x55873ddfc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873ddfcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873de06000 0x55873de083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13dda2b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55873dd61780 con 0x55873ddfd7f0 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13dda2b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55873dd07f40 con 0x55873ddfca80 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13dda2b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55873dd5d740 con 0x55873de06000 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.118+0000 7f13d0c00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f13b80a3d00 0x7f13b80a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873ddfcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873ddfcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54290/0 (socket says 172.21.15.18:54290) 2024-04-16T17:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 -- 172.21.15.18:0/1792333430 learned_addr learned my addr 172.21.15.18:0/1792333430 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d8e00640 1 --2- 172.21.15.18:0/1792333430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 0x55873ddfc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d0c00640 1 -- 172.21.15.18:0/935580253 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f13c40d0ff0 con 0x7f13d40743b0 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d8e00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f13b80a3d00 0x7f13b80a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 -- 172.21.15.18:0/1792333430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873de06000 msgr2=0x55873de083f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 --2- 172.21.15.18:0/1792333430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873de06000 0x55873de083f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 -- 172.21.15.18:0/1792333430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 msgr2=0x55873ddfc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 --2- 172.21.15.18:0/1792333430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 0x55873ddfc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 -- 172.21.15.18:0/1792333430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873da74210 con 0x55873ddfca80 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.119+0000 7f13d3e00640 1 --2- 172.21.15.18:0/1792333430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873ddfcf00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f13c400cf10 tx=0x7f13c4013330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1ed89875c25aae server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.120+0000 7f13c9600640 1 -- 172.21.15.18:0/1792333430 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4026040 con 0x55873ddfca80 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.120+0000 7f13c9600640 1 -- 172.21.15.18:0/1792333430 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c409d280 con 0x55873ddfca80 2024-04-16T17:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.120+0000 7f13c9600640 1 -- 172.21.15.18:0/1792333430 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40ce5b0 con 0x55873ddfca80 2024-04-16T17:14:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 -- 172.21.15.18:0/1792333430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 msgr2=0x55873ddfcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13d8e00640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f13b80a3d00 0x7f13b80a61c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009590 tx=0x7f13cc0095c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1792333430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873ddfcf00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f13c400cf10 tx=0x7f13c4013330 comp rx=0 tx=0).stop 2024-04-16T17:14:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 -- 172.21.15.18:0/1792333430 shutdown_connections 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1792333430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873de06000 0x55873de083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1792333430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873ddfcf00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1792333430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 0x55873ddfc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 -- 172.21.15.18:0/1792333430 >> 172.21.15.18:0/1792333430 conn(0x55873dd975d0 msgr2=0x55873ddf7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 -- 172.21.15.18:0/1792333430 shutdown_connections 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 -- 172.21.15.18:0/1792333430 wait complete. 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.121+0000 7f13dda2b740 1 -- start start 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13dda2b740 1 Processor -- start 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfca80 0x55873de05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de02e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de03550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13dda2b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55873de08cd0 con 0x55873ddfca80 2024-04-16T17:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13dda2b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55873dd1b390 con 0x55873de06000 2024-04-16T17:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13dda2b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55873de08e50 con 0x55873ddfd7f0 2024-04-16T17:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13dda2b740 10 client.0 _ll_register_callbacks cb 0x55873dd04f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfca80 0x55873de05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13d8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de03550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.122+0000 7f13d8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de03550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54296/0 (socket says 172.21.15.18:54296) 2024-04-16T17:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d8e00640 1 -- 172.21.15.18:0/119466954 learned_addr learned my addr 172.21.15.18:0/119466954 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d3e00640 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfca80 0x55873de05a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42542/0 (socket says 172.21.15.18:42542) 2024-04-16T17:14:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d3400640 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de02e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d8e00640 1 -- 172.21.15.18:0/119466954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 msgr2=0x55873de02e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d8e00640 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de02e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d8e00640 1 -- 172.21.15.18:0/119466954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfca80 msgr2=0x55873de05a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d8e00640 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfca80 0x55873de05a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d8e00640 1 -- 172.21.15.18:0/119466954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873db40620 con 0x55873de06000 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13d8e00640 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de03550 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00de30 tx=0x7f13cc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b70f77fbd5075fd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.123+0000 7f13be000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.124+0000 7f13be000640 1 -- 172.21.15.18:0/119466954 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc023070 con 0x55873de06000 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.124+0000 7f13dda2b740 1 -- 172.21.15.18:0/119466954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c409d280 con 0x55873de06000 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.124+0000 7f13be000640 1 -- 172.21.15.18:0/119466954 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f13cc003970 con 0x55873de06000 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.124+0000 7f13be000640 1 -- 172.21.15.18:0/119466954 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13cc003bb0 con 0x55873de06000 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.124+0000 7f13dda2b740 10 client.28307 fetch_fsmap learned FSMap version 29 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.124+0000 7f13dda2b740 1 -- 172.21.15.18:0/119466954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55873de14e00 con 0x55873de06000 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.124+0000 7f13be000640 1 -- 172.21.15.18:0/119466954 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c409d280 con 0x55873de06000 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.124+0000 7f13be000640 1 -- 172.21.15.18:0/119466954 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f13cc01c040 con 0x55873de06000 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.125+0000 7f13dda2b740 10 client.28307 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.125+0000 7f13dda2b740 10 client.28307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.125+0000 7f13dda2b740 20 client.28307 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.125+0000 7f13dda2b740 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55873de162c0 0x55873de186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.125+0000 7f13dda2b740 4 client.28307 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.125+0000 7f13dda2b740 1 -- 172.21.15.18:0/119466954 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873dd6f1a0 con 0x55873de162c0 2024-04-16T17:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.132+0000 7f13d3e00640 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55873de162c0 0x55873de186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.170+0000 7f13be000640 10 client.28307 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.170+0000 7f13d3e00640 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55873de162c0 0x55873de186b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce9f9c956751287 server_cookie=1dfed43d92457458 in_seq=0 out_seq=0 2024-04-16T17:14:52.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:51 smithi022 ceph-mon[36023]: pgmap v317: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13be000640 1 -- 172.21.15.18:0/119466954 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55873dd6f1a0 con 0x55873de162c0 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13be000640 10 client.28307 handle_command_reply: tid=0 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 client.28307 shutdown 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 -- 172.21.15.18:0/119466954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 msgr2=0x55873de03550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de03550 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00de30 tx=0x7f13cc004a40 comp rx=0 tx=0).stop 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 -- 172.21.15.18:0/119466954 shutdown_connections 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55873de162c0 0x55873de186b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de03550 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de02e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13be000640 0 client.28307 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 --2- 172.21.15.18:0/119466954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfca80 0x55873de05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 -- 172.21.15.18:0/119466954 >> 172.21.15.18:0/119466954 conn(0x55873dd975d0 msgr2=0x55873ddf94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 -- 172.21.15.18:0/119466954 shutdown_connections 2024-04-16T17:14:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.185+0000 7f13dda2b740 1 -- 172.21.15.18:0/119466954 wait complete. 2024-04-16T17:14:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.186+0000 7f13dda2b740 20 client.28307 trim_cache size 0 max 16384 2024-04-16T17:14:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.202+0000 7f13dda2b740 1 Processor -- start 2024-04-16T17:14:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.202+0000 7f13dda2b740 1 -- start start 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.202+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873de11bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.202+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de01e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.202+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873de06000 0x55873de05540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.202+0000 7f13dda2b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b3f20 con 0x55873de06000 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.202+0000 7f13dda2b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b17f0 con 0x55873ddfca80 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.202+0000 7f13dda2b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41af640 con 0x55873ddfd7f0 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de01e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873de11bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de01e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40290/0 (socket says 172.21.15.18:40290) 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3400640 1 -- 172.21.15.18:0/817741404 learned_addr learned my addr 172.21.15.18:0/817741404 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3e00640 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873de11bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54306/0 (socket says 172.21.15.18:54306) 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d8e00640 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873de06000 0x55873de05540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3400640 1 -- 172.21.15.18:0/817741404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 msgr2=0x55873de11bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3400640 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873de11bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3400640 1 -- 172.21.15.18:0/817741404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873de06000 msgr2=0x55873de05540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3400640 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873de06000 0x55873de05540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d3400640 1 -- 172.21.15.18:0/817741404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873ddfbd50 con 0x55873ddfd7f0 2024-04-16T17:14:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.203+0000 7f13d8e00640 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873de06000 0x55873de05540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.204+0000 7f13d3400640 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de01e30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f13c0037040 tx=0x7f13c00056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a41c2491d6fdd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.204+0000 7f13c8c00640 1 -- 172.21.15.18:0/817741404 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0045070 con 0x55873ddfd7f0 2024-04-16T17:14:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.204+0000 7f13c8c00640 1 -- 172.21.15.18:0/817741404 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c002ce50 con 0x55873ddfd7f0 2024-04-16T17:14:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.204+0000 7f13c8c00640 1 -- 172.21.15.18:0/817741404 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0039040 con 0x55873ddfd7f0 2024-04-16T17:14:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 -- 172.21.15.18:0/817741404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 msgr2=0x55873de01e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de01e30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f13c0037040 tx=0x7f13c00056d0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 -- 172.21.15.18:0/817741404 shutdown_connections 2024-04-16T17:14:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873de06000 0x55873de05540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfd7f0 0x55873de01e30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 --2- 172.21.15.18:0/817741404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873ddfca80 0x55873de11bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 -- 172.21.15.18:0/817741404 >> 172.21.15.18:0/817741404 conn(0x55873dd975d0 msgr2=0x55873de01720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 -- 172.21.15.18:0/817741404 shutdown_connections 2024-04-16T17:14:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 -- 172.21.15.18:0/817741404 wait complete. 2024-04-16T17:14:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.205+0000 7f13dda2b740 1 -- start start 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13dda2b740 1 Processor -- start 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfca80 0x55873ddffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 0x55873de00360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13dda2b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de7b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13dda2b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41afcb0 con 0x55873ddfd7f0 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13dda2b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b1120 con 0x55873de06000 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13dda2b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b0380 con 0x55873ddfca80 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13dda2b740 10 client.0 _ll_register_callbacks cb 0x7f13cc010e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfca80 0x55873ddffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13d3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 0x55873de00360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfca80 0x55873ddffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40294/0 (socket says 172.21.15.18:40294) 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.206+0000 7f13d3e00640 1 -- 172.21.15.18:0/1090936511 learned_addr learned my addr 172.21.15.18:0/1090936511 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13d8e00640 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de7b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13d3e00640 1 -- 172.21.15.18:0/1090936511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 msgr2=0x55873de7b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13d3e00640 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de7b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13d3e00640 1 -- 172.21.15.18:0/1090936511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 msgr2=0x55873de00360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13d3e00640 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 0x55873de00360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13d3e00640 1 -- 172.21.15.18:0/1090936511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873de00bf0 con 0x55873ddfca80 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13d3e00640 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfca80 0x55873ddffe20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f13c4098fe0 tx=0x7f13c400cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31b0fc006e1dcfea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13be000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13be000640 1 -- 172.21.15.18:0/1090936511 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c409d280 con 0x55873ddfca80 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13dda2b740 1 -- 172.21.15.18:0/1090936511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c002ce50 con 0x55873ddfca80 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.207+0000 7f13be000640 1 -- 172.21.15.18:0/1090936511 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f13c40ce5b0 con 0x55873ddfca80 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13be000640 1 -- 172.21.15.18:0/1090936511 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13c40bd0d0 con 0x55873ddfca80 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13dda2b740 10 client.28183 fetch_fsmap learned FSMap version 29 2024-04-16T17:14:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13dda2b740 1 -- 172.21.15.18:0/1090936511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55873de7ce30 con 0x55873ddfca80 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13be000640 1 -- 172.21.15.18:0/1090936511 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c002ce50 con 0x55873ddfca80 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13be000640 1 -- 172.21.15.18:0/1090936511 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f13c409c060 con 0x55873ddfca80 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13dda2b740 10 client.28183 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13dda2b740 10 client.28183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13dda2b740 20 client.28183 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.208+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55873de7d170 0x55873de7f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.209+0000 7f13dda2b740 4 client.28183 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.209+0000 7f13dda2b740 1 -- 172.21.15.18:0/1090936511 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55873dd69010 con 0x55873de7d170 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.209+0000 7f13d3400640 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55873de7d170 0x55873de7f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.209+0000 7f13d3400640 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55873de7d170 0x55873de7f560 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5baa9c8ed0060 server_cookie=62f390fb3d9a074c in_seq=0 out_seq=0 2024-04-16T17:14:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.209+0000 7f13be000640 10 client.28183 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13be000640 1 -- 172.21.15.18:0/1090936511 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55873dd69010 con 0x55873de7d170 2024-04-16T17:14:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13be000640 10 client.28183 handle_command_reply: tid=0 2024-04-16T17:14:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 client.28183 shutdown 2024-04-16T17:14:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 -- 172.21.15.18:0/1090936511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfca80 msgr2=0x55873ddffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfca80 0x55873ddffe20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f13c4098fe0 tx=0x7f13c400cf60 comp rx=0 tx=0).stop 2024-04-16T17:14:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 -- 172.21.15.18:0/1090936511 shutdown_connections 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55873de7d170 0x55873de7f560 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55873de06000 0x55873de7b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55873ddfd7f0 0x55873de00360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 --2- 172.21.15.18:0/1090936511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55873ddfca80 0x55873ddffe20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.387+0000 7f13dda2b740 1 -- 172.21.15.18:0/1090936511 >> 172.21.15.18:0/1090936511 conn(0x55873dd975d0 msgr2=0x55873ddf5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.388+0000 7f13be000640 0 client.28183 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.388+0000 7f13dda2b740 1 -- 172.21.15.18:0/1090936511 shutdown_connections 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.388+0000 7f13dda2b740 1 -- 172.21.15.18:0/1090936511 wait complete. 2024-04-16T17:14:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:14:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:14:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:14:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:14:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.388+0000 7f13dda2b740 20 client.28183 trim_cache size 0 max 16384 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 -- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f13b80a3d00 msgr2=0x7f13b80a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f13b80a3d00 0x7f13b80a61c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009590 tx=0x7f13cc0095c0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 -- 172.21.15.18:0/935580253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d40743b0 msgr2=0x7f13d41a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d40743b0 0x7f13d41a06a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f13c400e9f0 tx=0x7f13c400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 -- 172.21.15.18:0/935580253 shutdown_connections 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f13b80a3d00 0x7f13b80a61c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d410e6d0 0x7f13d419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13d4074cd0 0x7f13d41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 --2- 172.21.15.18:0/935580253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d40743b0 0x7f13d41a06a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 -- 172.21.15.18:0/935580253 >> 172.21.15.18:0/935580253 conn(0x7f13d40fd940 msgr2=0x7f13d40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 -- 172.21.15.18:0/935580253 shutdown_connections 2024-04-16T17:14:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:14:52.389+0000 7f13d9800640 1 -- 172.21.15.18:0/935580253 wait complete. 2024-04-16T17:14:52.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 381626 3.66 MB/sec execute 425 sec latency 108.871 ms 2024-04-16T17:14:52.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:51 smithi018 ceph-mon[31776]: pgmap v317: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-04-16T17:14:53.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 382572 3.66 MB/sec execute 426 sec latency 137.529 ms 2024-04-16T17:14:54.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:53 smithi043 ceph-mon[36901]: pgmap v318: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-04-16T17:14:54.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:53 smithi022 ceph-mon[36023]: pgmap v318: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-04-16T17:14:54.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 383492 3.67 MB/sec execute 427 sec latency 15.864 ms 2024-04-16T17:14:55.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 384230 3.67 MB/sec execute 428 sec latency 22.686 ms 2024-04-16T17:14:56.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:55 smithi043 ceph-mon[36901]: pgmap v319: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-04-16T17:14:56.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:53 smithi018 ceph-mon[31776]: pgmap v318: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-04-16T17:14:56.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:55 smithi018 ceph-mon[31776]: pgmap v319: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-04-16T17:14:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:55 smithi022 ceph-mon[36023]: pgmap v319: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-04-16T17:14:56.639 INFO:tasks.workunit.client.0.smithi018.stdout: 1 384663 3.67 MB/sec execute 429 sec latency 46.377 ms 2024-04-16T17:14:57.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 384965 3.66 MB/sec execute 430 sec latency 94.610 ms 2024-04-16T17:14:58.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:57 smithi043 ceph-mon[36901]: pgmap v320: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:14:58.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:57 smithi022 ceph-mon[36023]: pgmap v320: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:14:58.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:14:58.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:14:58.640 INFO:tasks.workunit.client.0.smithi018.stdout: 1 385710 3.66 MB/sec execute 431 sec latency 108.312 ms 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 -- 172.21.15.18:0/3593810195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3558100fb0 msgr2=0x7f355810c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 --2- 172.21.15.18:0/3593810195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3558100fb0 0x7f355810c030 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3544009900 tx=0x7f354402bc30 comp rx=0 tx=0).stop 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 -- 172.21.15.18:0/3593810195 shutdown_connections 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 --2- 172.21.15.18:0/3593810195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355810c600 0x7f355810ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 --2- 172.21.15.18:0/3593810195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3558100fb0 0x7f355810c030 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 --2- 172.21.15.18:0/3593810195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3558100690 0x7f3558100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 -- 172.21.15.18:0/3593810195 >> 172.21.15.18:0/3593810195 conn(0x7f3558075da0 msgr2=0x7f35580761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 -- 172.21.15.18:0/3593810195 shutdown_connections 2024-04-16T17:14:58.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.684+0000 7f355e600640 1 -- 172.21.15.18:0/3593810195 wait complete. 2024-04-16T17:14:58.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f355e600640 1 Processor -- start 2024-04-16T17:14:58.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f355e600640 1 -- start start 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f355e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3558100690 0x7f35581a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f355e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3558100fb0 0x7f35581a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f355e600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f355810c600 0x7f355819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f355e600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3558112390 con 0x7f3558100690 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f355e600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3558112210 con 0x7f355810c600 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f355e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3558112510 con 0x7f3558100fb0 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.686+0000 7f3556a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3558100fb0 0x7f35581a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3556a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3558100fb0 0x7f35581a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38700/0 (socket says 172.21.15.18:38700) 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3556a00640 1 -- 172.21.15.18:0/1551118167 learned_addr learned my addr 172.21.15.18:0/1551118167 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3557400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3558100690 0x7f35581a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3557e00640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f355810c600 0x7f355819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3556a00640 1 -- 172.21.15.18:0/1551118167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f355810c600 msgr2=0x7f355819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3556a00640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f355810c600 0x7f355819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3556a00640 1 -- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3558100690 msgr2=0x7f35581a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3556a00640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3558100690 0x7f35581a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3556a00640 1 -- 172.21.15.18:0/1551118167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355819b030 con 0x7f3558100fb0 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3557e00640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f355810c600 0x7f355819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.687+0000 7f3556a00640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3558100fb0 0x7f35581a0c40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3544036040 tx=0x7f354402cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.688+0000 7f354fe00640 1 -- 172.21.15.18:0/1551118167 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544041070 con 0x7f3558100fb0 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.688+0000 7f354fe00640 1 -- 172.21.15.18:0/1551118167 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3544004450 con 0x7f3558100fb0 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.688+0000 7f354fe00640 1 -- 172.21.15.18:0/1551118167 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544003dc0 con 0x7f3558100fb0 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.688+0000 7f355e600640 1 -- 172.21.15.18:0/1551118167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355819b2c0 con 0x7f3558100fb0 2024-04-16T17:14:58.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.688+0000 7f355e600640 1 -- 172.21.15.18:0/1551118167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35581a75c0 con 0x7f3558100fb0 2024-04-16T17:14:58.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f35613ab740 1 Processor -- start 2024-04-16T17:14:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f35613ab740 1 -- start start 2024-04-16T17:14:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f35613ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f35613ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf2ad0 0x560b9fcfb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f35613ab740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcfdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f354fe00640 1 -- 172.21.15.18:0/1551118167 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3544044070 con 0x7f3558100fb0 2024-04-16T17:14:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f35613ab740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560b9fc58120 con 0x560b9fcfb6b0 2024-04-16T17:14:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f35613ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560b9fbfe340 con 0x560b9fcf21b0 2024-04-16T17:14:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.692+0000 7f35613ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560b9fc540e0 con 0x560b9fcf2ad0 2024-04-16T17:14:58.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54932/0 (socket says 172.21.15.18:54932) 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557e00640 1 -- 172.21.15.18:0/4231805489 learned_addr learned my addr 172.21.15.18:0/4231805489 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3556a00640 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcfdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557400640 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf2ad0 0x560b9fcfb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557e00640 1 -- 172.21.15.18:0/4231805489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf2ad0 msgr2=0x560b9fcfb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557e00640 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf2ad0 0x560b9fcfb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557e00640 1 -- 172.21.15.18:0/4231805489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 msgr2=0x560b9fcfdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557e00640 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcfdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.693+0000 7f3557e00640 1 -- 172.21.15.18:0/4231805489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9fa347d0 con 0x560b9fcf21b0 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.694+0000 7f3557e00640 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf2590 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3550010750 tx=0x7f3550010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fe2d4454a3e216 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.694+0000 7f354cc00640 1 -- 172.21.15.18:0/4231805489 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355000ead0 con 0x560b9fcf21b0 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.694+0000 7f354cc00640 1 -- 172.21.15.18:0/4231805489 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3550010e60 con 0x560b9fcf21b0 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.694+0000 7f354cc00640 1 -- 172.21.15.18:0/4231805489 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550011640 con 0x560b9fcf21b0 2024-04-16T17:14:58.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.694+0000 7f354fe00640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f353c0a3da0 0x7f353c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f354fe00640 1 -- 172.21.15.18:0/1551118167 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f35440f2510 con 0x7f3558100fb0 2024-04-16T17:14:58.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f3557400640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f353c0a3da0 0x7f353c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 -- 172.21.15.18:0/4231805489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 msgr2=0x560b9fcf2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf2590 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3550010750 tx=0x7f3550010c10 comp rx=0 tx=0).stop 2024-04-16T17:14:58.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 -- 172.21.15.18:0/4231805489 shutdown_connections 2024-04-16T17:14:58.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcfdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf2ad0 0x560b9fcfb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 --2- 172.21.15.18:0/4231805489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf2590 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 -- 172.21.15.18:0/4231805489 >> 172.21.15.18:0/4231805489 conn(0x560b9fcec8e0 msgr2=0x560b9fceccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 -- 172.21.15.18:0/4231805489 shutdown_connections 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f3557400640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f353c0a3da0 0x7f353c0a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x560b9fcf1c80 tx=0x7f3548005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.695+0000 7f35613ab740 1 -- 172.21.15.18:0/4231805489 wait complete. 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.696+0000 7f35613ab740 1 -- start start 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.696+0000 7f35613ab740 1 Processor -- start 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.696+0000 7f35613ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.696+0000 7f35613ab740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 0x560b9fcf8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f35613ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f35613ab740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560b9fcfe4a0 con 0x560b9fcf2ad0 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f35613ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560b9fc11710 con 0x560b9fcf21b0 2024-04-16T17:14:58.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f35613ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560b9fcfe620 con 0x560b9fcfb6b0 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f35613ab740 10 client.0 _ll_register_callbacks cb 0x560b9fc0caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54944/0 (socket says 172.21.15.18:54944) 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557400640 1 -- 172.21.15.18:0/1606326067 learned_addr learned my addr 172.21.15.18:0/1606326067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3556a00640 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 0x560b9fcf8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557e00640 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557400640 1 -- 172.21.15.18:0/1606326067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcfb6b0 msgr2=0x560b9fcf91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557400640 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557400640 1 -- 172.21.15.18:0/1606326067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 msgr2=0x560b9fcf8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557400640 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 0x560b9fcf8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557400640 1 -- 172.21.15.18:0/1606326067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9f96a3f0 con 0x560b9fcf21b0 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.697+0000 7f3557e00640 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.698+0000 7f3557400640 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf8720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f35480045e0 tx=0x7f3548004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65f78c7526a6c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:58.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.698+0000 7f3541600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:58.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.698+0000 7f3541600640 1 -- 172.21.15.18:0/1606326067 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548029440 con 0x560b9fcf21b0 2024-04-16T17:14:58.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.698+0000 7f35613ab740 1 -- 172.21.15.18:0/1606326067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3550010e60 con 0x560b9fcf21b0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.698+0000 7f3541600640 1 -- 172.21.15.18:0/1606326067 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f354801f400 con 0x560b9fcf21b0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.699+0000 7f3541600640 1 -- 172.21.15.18:0/1606326067 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f354801f600 con 0x560b9fcf21b0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.699+0000 7f35613ab740 10 client.28325 fetch_fsmap learned FSMap version 29 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.699+0000 7f35613ab740 1 -- 172.21.15.18:0/1606326067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b9f92a0f0 con 0x560b9fcf21b0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.699+0000 7f3541600640 1 -- 172.21.15.18:0/1606326067 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3550010e60 con 0x560b9fcf21b0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.699+0000 7f3541600640 1 -- 172.21.15.18:0/1606326067 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f3548027020 con 0x560b9fcf21b0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f35613ab740 10 client.28325 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f35613ab740 10 client.28325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f35613ab740 20 client.28325 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f35613ab740 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b9fd0bab0 0x560b9fd0df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f35613ab740 4 client.28325 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f35613ab740 1 -- 172.21.15.18:0/1606326067 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b9fc65b40 con 0x560b9fd0bab0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f3556a00640 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b9fd0bab0 0x560b9fd0df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f3556a00640 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b9fd0bab0 0x560b9fd0df00 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55df26450317bda0 server_cookie=c5db6371f5517065 in_seq=0 out_seq=0 2024-04-16T17:14:58.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.700+0000 7f3541600640 10 client.28325 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f3541600640 1 -- 172.21.15.18:0/1606326067 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b9fc65b40 con 0x560b9fd0bab0 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f3541600640 10 client.28325 handle_command_reply: tid=0 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 client.28325 shutdown 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 -- 172.21.15.18:0/1606326067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 msgr2=0x560b9fcf8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf8720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f35480045e0 tx=0x7f3548004730 comp rx=0 tx=0).stop 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 -- 172.21.15.18:0/1606326067 shutdown_connections 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b9fd0bab0 0x560b9fd0df00 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 0x560b9fcf8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 --2- 172.21.15.18:0/1606326067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf21b0 0x560b9fcf8720 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f35613ab740 1 -- 172.21.15.18:0/1606326067 >> 172.21.15.18:0/1606326067 conn(0x560b9fcec8e0 msgr2=0x560b9fcee630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.701+0000 7f3541600640 0 client.28325 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.702+0000 7f35613ab740 1 -- 172.21.15.18:0/1606326067 shutdown_connections 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.702+0000 7f35613ab740 1 -- 172.21.15.18:0/1606326067 wait complete. 2024-04-16T17:14:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.702+0000 7f35613ab740 20 client.28325 trim_cache size 0 max 16384 2024-04-16T17:14:58.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.731+0000 7f35613ab740 1 Processor -- start 2024-04-16T17:14:58.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.731+0000 7f35613ab740 1 -- start start 2024-04-16T17:14:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.731+0000 7f35613ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fcfae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.731+0000 7f35613ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf2ad0 0x560b9fd07390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.731+0000 7f35613ab740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.731+0000 7f35613ab740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f35581b4020 con 0x560b9fcfb6b0 2024-04-16T17:14:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.731+0000 7f35613ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f35581b18f0 con 0x560b9fcf2ad0 2024-04-16T17:14:58.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.731+0000 7f35613ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35581af740 con 0x560b9fcf21b0 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf2ad0 0x560b9fd07390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf2ad0 0x560b9fd07390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54948/0 (socket says 172.21.15.18:54948) 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 -- 172.21.15.18:0/2349684244 learned_addr learned my addr 172.21.15.18:0/2349684244 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3557e00640 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3557400640 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fcfae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 -- 172.21.15.18:0/2349684244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 msgr2=0x560b9fcfae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fcfae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 -- 172.21.15.18:0/2349684244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 msgr2=0x560b9fcf7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 -- 172.21.15.18:0/2349684244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9fcf6c70 con 0x560b9fcf2ad0 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3557e00640 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3556a00640 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf2ad0 0x560b9fd07390 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f354402cfd0 tx=0x7f3544004ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8f0f11c01404f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.732+0000 7f3543e00640 1 -- 172.21.15.18:0/2349684244 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354404b040 con 0x560b9fcf2ad0 2024-04-16T17:14:58.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f3543e00640 1 -- 172.21.15.18:0/2349684244 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35440f5e10 con 0x560b9fcf2ad0 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f3543e00640 1 -- 172.21.15.18:0/2349684244 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35440c6e20 con 0x560b9fcf2ad0 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f35613ab740 1 -- 172.21.15.18:0/2349684244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf2ad0 msgr2=0x560b9fd07390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f35613ab740 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf2ad0 0x560b9fd07390 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f354402cfd0 tx=0x7f3544004ee0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f35613ab740 1 -- 172.21.15.18:0/2349684244 shutdown_connections 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f35613ab740 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcfb6b0 0x560b9fcf7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f35613ab740 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcf2ad0 0x560b9fd07390 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f35613ab740 1 --2- 172.21.15.18:0/2349684244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fcfae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f35613ab740 1 -- 172.21.15.18:0/2349684244 >> 172.21.15.18:0/2349684244 conn(0x560b9fcec8e0 msgr2=0x560b9fceb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.733+0000 7f35613ab740 1 -- 172.21.15.18:0/2349684244 shutdown_connections 2024-04-16T17:14:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.734+0000 7f35613ab740 1 -- 172.21.15.18:0/2349684244 wait complete. 2024-04-16T17:14:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.734+0000 7f35613ab740 1 -- start start 2024-04-16T17:14:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.734+0000 7f35613ab740 1 Processor -- start 2024-04-16T17:14:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.735+0000 7f35613ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fd03c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.735+0000 7f35613ab740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 0x560b9fcfefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.735+0000 7f35613ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcfb6b0 0x560b9fcff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.735+0000 7f35613ab740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f35581afdb0 con 0x560b9fcf2ad0 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.735+0000 7f35613ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f35581b1220 con 0x560b9fcfb6b0 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.735+0000 7f35613ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35581b0480 con 0x560b9fcf21b0 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.735+0000 7f35613ab740 10 client.0 _ll_register_callbacks cb 0x7f3548009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fd03c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcfb6b0 0x560b9fcff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fd03c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38752/0 (socket says 172.21.15.18:38752) 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcfb6b0 0x560b9fcff4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54960/0 (socket says 172.21.15.18:54960) 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 -- 172.21.15.18:0/2575628527 learned_addr learned my addr 172.21.15.18:0/2575628527 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3556a00640 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 0x560b9fcfefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 -- 172.21.15.18:0/2575628527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcfb6b0 msgr2=0x560b9fcff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcfb6b0 0x560b9fcff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 -- 172.21.15.18:0/2575628527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 msgr2=0x560b9fcfefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 0x560b9fcfefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 -- 172.21.15.18:0/2575628527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9fcffd20 con 0x560b9fcf21b0 2024-04-16T17:14:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3557400640 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fd03c40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3548029cb0 tx=0x7f3548004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf004338587c28f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.736+0000 7f3541600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.737+0000 7f3541600640 1 -- 172.21.15.18:0/2575628527 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480043d0 con 0x560b9fcf21b0 2024-04-16T17:14:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.737+0000 7f3541600640 1 -- 172.21.15.18:0/2575628527 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f354801f400 con 0x560b9fcf21b0 2024-04-16T17:14:58.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.737+0000 7f35613ab740 1 -- 172.21.15.18:0/2575628527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35440f5e10 con 0x560b9fcf21b0 2024-04-16T17:14:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.740+0000 7f3541600640 1 -- 172.21.15.18:0/2575628527 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f354801f9b0 con 0x560b9fcf21b0 2024-04-16T17:14:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.740+0000 7f35613ab740 10 client.28201 fetch_fsmap learned FSMap version 29 2024-04-16T17:14:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.740+0000 7f35613ab740 1 -- 172.21.15.18:0/2575628527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b9fd7a850 con 0x560b9fcf21b0 2024-04-16T17:14:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f3541600640 1 -- 172.21.15.18:0/2575628527 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35440f5e10 con 0x560b9fcf21b0 2024-04-16T17:14:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f3541600640 1 -- 172.21.15.18:0/2575628527 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f3548027040 con 0x560b9fcf21b0 2024-04-16T17:14:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f35613ab740 10 client.28201 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:14:58.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f35613ab740 10 client.28201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f35613ab740 20 client.28201 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f35613ab740 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b9fd7aff0 0x560b9fd7d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f35613ab740 4 client.28201 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:14:58.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f35613ab740 1 -- 172.21.15.18:0/2575628527 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b9fc5f9b0 con 0x560b9fd7aff0 2024-04-16T17:14:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.741+0000 7f3556a00640 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b9fd7aff0 0x560b9fd7d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.742+0000 7f3556a00640 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b9fd7aff0 0x560b9fd7d410 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4177813517a5d9 server_cookie=ed5cb9488fa16e11 in_seq=0 out_seq=0 2024-04-16T17:14:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.742+0000 7f3541600640 10 client.28201 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.742+0000 7f3541600640 1 -- 172.21.15.18:0/2575628527 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560b9fc5f9b0 con 0x560b9fd7aff0 2024-04-16T17:14:58.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.742+0000 7f3541600640 10 client.28201 handle_command_reply: tid=0 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.742+0000 7f35613ab740 1 client.28201 shutdown 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.742+0000 7f35613ab740 1 -- 172.21.15.18:0/2575628527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 msgr2=0x560b9fd03c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.742+0000 7f35613ab740 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fd03c40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3548029cb0 tx=0x7f3548004690 comp rx=0 tx=0).stop 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 1 -- 172.21.15.18:0/2575628527 shutdown_connections 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b9fd7aff0 0x560b9fd7d410 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9fcfb6b0 0x560b9fcff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b9fcf2ad0 0x560b9fcfefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 1 --2- 172.21.15.18:0/2575628527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b9fcf21b0 0x560b9fd03c40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f3541600640 0 client.28201 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 1 -- 172.21.15.18:0/2575628527 >> 172.21.15.18:0/2575628527 conn(0x560b9fcec8e0 msgr2=0x560b9fceb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 1 -- 172.21.15.18:0/2575628527 shutdown_connections 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 1 -- 172.21.15.18:0/2575628527 wait complete. 2024-04-16T17:14:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.743+0000 7f35613ab740 20 client.28201 trim_cache size 0 max 16384 2024-04-16T17:14:58.744 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:14:58.744 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:14:58.744 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:14:58.744 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:14:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 -- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f353c0a3da0 msgr2=0x7f353c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f353c0a3da0 0x7f353c0a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x560b9fcf1c80 tx=0x7f3548005f90 comp rx=0 tx=0).stop 2024-04-16T17:14:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 -- 172.21.15.18:0/1551118167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3558100fb0 msgr2=0x7f35581a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3558100fb0 0x7f35581a0c40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3544036040 tx=0x7f354402cb20 comp rx=0 tx=0).stop 2024-04-16T17:14:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 -- 172.21.15.18:0/1551118167 shutdown_connections 2024-04-16T17:14:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f353c0a3da0 0x7f353c0a6260 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f355810c600 0x7f355819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3558100fb0 0x7f35581a0c40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 --2- 172.21.15.18:0/1551118167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3558100690 0x7f35581a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 -- 172.21.15.18:0/1551118167 >> 172.21.15.18:0/1551118167 conn(0x7f3558075da0 msgr2=0x7f35580ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 -- 172.21.15.18:0/1551118167 shutdown_connections 2024-04-16T17:14:58.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.744+0000 7f355e600640 1 -- 172.21.15.18:0/1551118167 wait complete. 2024-04-16T17:14:58.759 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:14:58.760 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:14:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 -- 172.21.15.18:0/1942305752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18dc0743b0 msgr2=0x7f18dc074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 --2- 172.21.15.18:0/1942305752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18dc0743b0 0x7f18dc074790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009900 tx=0x7f18cc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 -- 172.21.15.18:0/1942305752 shutdown_connections 2024-04-16T17:14:58.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 --2- 172.21.15.18:0/1942305752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18dc10e890 0x7f18dc110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 --2- 172.21.15.18:0/1942305752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18dc074cd0 0x7f18dc10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 --2- 172.21.15.18:0/1942305752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18dc0743b0 0x7f18dc074790 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 -- 172.21.15.18:0/1942305752 >> 172.21.15.18:0/1942305752 conn(0x7f18dc0fdae0 msgr2=0x7f18dc0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:58.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 -- 172.21.15.18:0/1942305752 shutdown_connections 2024-04-16T17:14:58.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.972+0000 7f18e1c00640 1 -- 172.21.15.18:0/1942305752 wait complete. 2024-04-16T17:14:58.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.973+0000 7f18e1c00640 1 Processor -- start 2024-04-16T17:14:58.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.973+0000 7f18e1c00640 1 -- start start 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.973+0000 7f18e1c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18dc0743b0 0x7f18dc10e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18e1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18dc074cd0 0x7f18dc10c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18e1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18dc10e890 0x7f18dc10cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18e1c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1027e0 con 0x7f18dc0743b0 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18e1c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc102660 con 0x7f18dc074cd0 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18e1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc102960 con 0x7f18dc10e890 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18da000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18dc074cd0 0x7f18dc10c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18da000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18dc074cd0 0x7f18dc10c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54982/0 (socket says 172.21.15.18:54982) 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18da000640 1 -- 172.21.15.18:0/2688626017 learned_addr learned my addr 172.21.15.18:0/2688626017 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18db400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18dc10e890 0x7f18dc10cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18daa00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18dc0743b0 0x7f18dc10e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18da000640 1 -- 172.21.15.18:0/2688626017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18dc10e890 msgr2=0x7f18dc10cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18da000640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18dc10e890 0x7f18dc10cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18da000640 1 -- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18dc0743b0 msgr2=0x7f18dc10e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:58.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18da000640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18dc0743b0 0x7f18dc10e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:58.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18da000640 1 -- 172.21.15.18:0/2688626017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc1ab280 con 0x7f18dc074cd0 2024-04-16T17:14:58.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.974+0000 7f18db400640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18dc10e890 0x7f18dc10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:58.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.975+0000 7f18da000640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18dc074cd0 0x7f18dc10c410 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9f0 tx=0x7f18c800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:58.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.975+0000 7f18d3400640 1 -- 172.21.15.18:0/2688626017 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c800ccc0 con 0x7f18dc074cd0 2024-04-16T17:14:58.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.975+0000 7f18e1c00640 1 -- 172.21.15.18:0/2688626017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc1ab510 con 0x7f18dc074cd0 2024-04-16T17:14:58.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.975+0000 7f18d3400640 1 -- 172.21.15.18:0/2688626017 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c800ce60 con 0x7f18dc074cd0 2024-04-16T17:14:58.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.975+0000 7f18d3400640 1 -- 172.21.15.18:0/2688626017 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8010690 con 0x7f18dc074cd0 2024-04-16T17:14:58.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.975+0000 7f18e1c00640 1 -- 172.21.15.18:0/2688626017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1ab9e0 con 0x7f18dc074cd0 2024-04-16T17:14:58.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.977+0000 7f18d0c00640 1 -- 172.21.15.18:0/2688626017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18dc069300 con 0x7f18dc074cd0 2024-04-16T17:14:58.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.980+0000 7f18d3400640 1 -- 172.21.15.18:0/2688626017 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18c80046c0 con 0x7f18dc074cd0 2024-04-16T17:14:58.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.982+0000 7f18d3400640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18c00a3d20 0x7f18c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:58.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.982+0000 7f18d3400640 1 -- 172.21.15.18:0/2688626017 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f18c80d0f70 con 0x7f18dc074cd0 2024-04-16T17:14:58.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.982+0000 7f18daa00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18c00a3d20 0x7f18c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:58.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.983+0000 7f18daa00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18c00a3d20 0x7f18c00a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005ab0 tx=0x7f18cc005a20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:58.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:58.987+0000 7f18d3400640 1 -- 172.21.15.18:0/2688626017 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18c809d260 con 0x7f18dc074cd0 2024-04-16T17:14:59.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.267+0000 7f18d0c00640 1 -- 172.21.15.18:0/2688626017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18dc105f30 con 0x7f18dc074cd0 2024-04-16T17:14:59.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.268+0000 7f18d3400640 1 -- 172.21.15.18:0/2688626017 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11851 (secure 0 0 0) 0x7f18c809cb60 con 0x7f18dc074cd0 2024-04-16T17:14:59.267 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:14:59.268 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:14:39.680633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28295":{"gid":28295,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/257552546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":257552546},{"type":"v1","addr":"172.21.15.22:6837","nonce":257552546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:14:59.268 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-16T17:14:59.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.270+0000 7f18d0c00640 1 -- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18c00a3d20 msgr2=0x7f18c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.270+0000 7f18d0c00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18c00a3d20 0x7f18c00a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005ab0 tx=0x7f18cc005a20 comp rx=0 tx=0).stop 2024-04-16T17:14:59.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.270+0000 7f18d0c00640 1 -- 172.21.15.18:0/2688626017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18dc074cd0 msgr2=0x7f18dc10c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.270+0000 7f18d0c00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18dc074cd0 0x7f18dc10c410 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9f0 tx=0x7f18c800edf0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.271+0000 7f18d0c00640 1 -- 172.21.15.18:0/2688626017 shutdown_connections 2024-04-16T17:14:59.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.271+0000 7f18d0c00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18c00a3d20 0x7f18c00a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.271+0000 7f18d0c00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18dc10e890 0x7f18dc10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.271+0000 7f18d0c00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18dc074cd0 0x7f18dc10c410 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.271+0000 7f18d0c00640 1 --2- 172.21.15.18:0/2688626017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18dc0743b0 0x7f18dc10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.271+0000 7f18d0c00640 1 -- 172.21.15.18:0/2688626017 >> 172.21.15.18:0/2688626017 conn(0x7f18dc0fdae0 msgr2=0x7f18dc0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.271+0000 7f18d0c00640 1 -- 172.21.15.18:0/2688626017 shutdown_connections 2024-04-16T17:14:59.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.271+0000 7f18d0c00640 1 -- 172.21.15.18:0/2688626017 wait complete. 2024-04-16T17:14:59.289 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 -- 172.21.15.18:0/852334177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8103900 msgr2=0x7f93f810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 --2- 172.21.15.18:0/852334177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8103900 0x7f93f810bde0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009900 tx=0x7f93e402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 -- 172.21.15.18:0/852334177 shutdown_connections 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 --2- 172.21.15.18:0/852334177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93f810c400 0x7f93f810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 --2- 172.21.15.18:0/852334177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8103900 0x7f93f810bde0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 --2- 172.21.15.18:0/852334177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93f8102fe0 0x7f93f81033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 -- 172.21.15.18:0/852334177 >> 172.21.15.18:0/852334177 conn(0x7f93f8077ab0 msgr2=0x7f93f80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 -- 172.21.15.18:0/852334177 shutdown_connections 2024-04-16T17:14:59.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.494+0000 7f93fe200640 1 -- 172.21.15.18:0/852334177 wait complete. 2024-04-16T17:14:59.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.496+0000 7f93fe200640 1 Processor -- start 2024-04-16T17:14:59.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.496+0000 7f93fe200640 1 -- start start 2024-04-16T17:14:59.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.496+0000 7f93fe200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8102fe0 0x7f93f8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.496+0000 7f93fe200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93f8103900 0x7f93f8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.496+0000 7f93fe200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93f810c400 0x7f93f819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.496+0000 7f93fe200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112200 con 0x7f93f8103900 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.496+0000 7f93fe200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112080 con 0x7f93f810c400 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.496+0000 7f93fe200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112380 con 0x7f93f8102fe0 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8102fe0 0x7f93f8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8102fe0 0x7f93f8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38780/0 (socket says 172.21.15.18:38780) 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 -- 172.21.15.18:0/1805510414 learned_addr learned my addr 172.21.15.18:0/1805510414 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93f7e00640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93f8103900 0x7f93f8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fd800640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93f810c400 0x7f93f819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 -- 172.21.15.18:0/1805510414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93f810c400 msgr2=0x7f93f819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93f810c400 0x7f93f819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 -- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93f8103900 msgr2=0x7f93f8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93f8103900 0x7f93f8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 -- 172.21.15.18:0/1805510414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f819d750 con 0x7f93f8102fe0 2024-04-16T17:14:59.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.497+0000 7f93fce00640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8102fe0 0x7f93f8198760 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e9f0 tx=0x7f93ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.498+0000 7f93f5600640 1 -- 172.21.15.18:0/1805510414 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00cd80 con 0x7f93f8102fe0 2024-04-16T17:14:59.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.498+0000 7f93f5600640 1 -- 172.21.15.18:0/1805510414 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93ec010910 con 0x7f93f8102fe0 2024-04-16T17:14:59.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.498+0000 7f93f5600640 1 -- 172.21.15.18:0/1805510414 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec018990 con 0x7f93f8102fe0 2024-04-16T17:14:59.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.498+0000 7f93fe200640 1 -- 172.21.15.18:0/1805510414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f819da40 con 0x7f93f8102fe0 2024-04-16T17:14:59.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.498+0000 7f93fe200640 1 -- 172.21.15.18:0/1805510414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f8100b60 con 0x7f93f8102fe0 2024-04-16T17:14:59.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.501+0000 7f93f5600640 1 -- 172.21.15.18:0/1805510414 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93ec010ac0 con 0x7f93f8102fe0 2024-04-16T17:14:59.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.502+0000 7f940241a740 1 Processor -- start 2024-04-16T17:14:59.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.502+0000 7f940241a740 1 -- start start 2024-04-16T17:14:59.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f93f5600640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f93e00a3da0 0x7f93e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f940241a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e932f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f940241a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e9bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f940241a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 0x556ea0e9e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f940241a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556ea0df8020 con 0x556ea0e93830 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f940241a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556ea0d9e460 con 0x556ea0e9c410 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f940241a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556ea0df3fe0 con 0x556ea0e92f10 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f93f5600640 1 -- 172.21.15.18:0/1805510414 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f93ec0d0e90 con 0x7f93f8102fe0 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f93fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e932f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f93fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e932f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38784/0 (socket says 172.21.15.18:38784) 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f93fce00640 1 -- 172.21.15.18:0/2107300370 learned_addr learned my addr 172.21.15.18:0/2107300370 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f93f7e00640 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e9bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f93fd800640 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 0x556ea0e9e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.503+0000 7f93fd800640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f93e00a3da0 0x7f93e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.504+0000 7f93fce00640 1 -- 172.21.15.18:0/2107300370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 msgr2=0x556ea0e9e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.504+0000 7f93fce00640 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 0x556ea0e9e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.504+0000 7f93fce00640 1 -- 172.21.15.18:0/2107300370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 msgr2=0x556ea0e9bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.504+0000 7f93fce00640 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e9bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.504+0000 7f93fce00640 1 -- 172.21.15.18:0/2107300370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea0da6040 con 0x556ea0e92f10 2024-04-16T17:14:59.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.504+0000 7f93fce00640 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e932f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0a5160 tx=0x7f93ec09cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8f0924c90a15cf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.504+0000 7f93fd800640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f93e00a3da0 0x7f93e00a6260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x556ea0e8e7d0 tx=0x7f93f0009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.504+0000 7f93ea000640 1 -- 172.21.15.18:0/2107300370 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0a5690 con 0x556ea0e92f10 2024-04-16T17:14:59.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.505+0000 7f93ea000640 1 -- 172.21.15.18:0/2107300370 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93ec09add0 con 0x556ea0e92f10 2024-04-16T17:14:59.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.505+0000 7f93ea000640 1 -- 172.21.15.18:0/2107300370 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0a5460 con 0x556ea0e92f10 2024-04-16T17:14:59.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 -- 172.21.15.18:0/2107300370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 msgr2=0x556ea0e932f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e932f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0a5160 tx=0x7f93ec09cb90 comp rx=0 tx=0).stop 2024-04-16T17:14:59.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 -- 172.21.15.18:0/2107300370 shutdown_connections 2024-04-16T17:14:59.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 0x556ea0e9e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e9bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 --2- 172.21.15.18:0/2107300370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e932f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 -- 172.21.15.18:0/2107300370 >> 172.21.15.18:0/2107300370 conn(0x556ea0e8d640 msgr2=0x556ea0e8da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 -- 172.21.15.18:0/2107300370 shutdown_connections 2024-04-16T17:14:59.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.506+0000 7f940241a740 1 -- 172.21.15.18:0/2107300370 wait complete. 2024-04-16T17:14:59.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.507+0000 7f940241a740 1 -- start start 2024-04-16T17:14:59.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.507+0000 7f940241a740 1 Processor -- start 2024-04-16T17:14:59.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f940241a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e99550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f940241a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e99a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f940241a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 0x556ea0e99fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f940241a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556ea0e9f200 con 0x556ea0e93830 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f940241a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556ea0db1830 con 0x556ea0e9c410 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f940241a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556ea0e9f380 con 0x556ea0e92f10 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f940241a740 10 client.0 _ll_register_callbacks cb 0x556ea0db9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f93fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e99550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f93fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e99550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38792/0 (socket says 172.21.15.18:38792) 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.508+0000 7f93fce00640 1 -- 172.21.15.18:0/1140385720 learned_addr learned my addr 172.21.15.18:0/1140385720 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93fd800640 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 0x556ea0e99fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93f7e00640 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e99a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93fce00640 1 -- 172.21.15.18:0/1140385720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 msgr2=0x556ea0e99fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93fce00640 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 0x556ea0e99fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93fce00640 1 -- 172.21.15.18:0/1140385720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 msgr2=0x556ea0e99a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93fce00640 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e99a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93fce00640 1 -- 172.21.15.18:0/1140385720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ea0db5920 con 0x556ea0e92f10 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93f7e00640 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e99a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93fce00640 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e99550 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009350 tx=0x7f93ec0969b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=181cd424b8e49c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93dea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:59.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93dea00640 1 -- 172.21.15.18:0/1140385720 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0ce450 con 0x556ea0e92f10 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f93dea00640 1 -- 172.21.15.18:0/1140385720 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f93ec018b30 con 0x556ea0e92f10 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.509+0000 7f940241a740 1 -- 172.21.15.18:0/1140385720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ec09add0 con 0x556ea0e92f10 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f93dea00640 1 -- 172.21.15.18:0/1140385720 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93ec0a5690 con 0x556ea0e92f10 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f940241a740 10 client.28219 fetch_fsmap learned FSMap version 29 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f940241a740 1 -- 172.21.15.18:0/1140385720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ea0d837f0 con 0x556ea0e92f10 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f93dea00640 1 -- 172.21.15.18:0/1140385720 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93ec09add0 con 0x556ea0e92f10 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f93dea00640 1 -- 172.21.15.18:0/1140385720 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f93ec0ba020 con 0x556ea0e92f10 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f940241a740 10 client.28219 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f940241a740 10 client.28219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f940241a740 20 client.28219 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f940241a740 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556ea0eac730 0x556ea0eaeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f940241a740 4 client.28219 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:59.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.510+0000 7f940241a740 1 -- 172.21.15.18:0/1140385720 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ea0e05a40 con 0x556ea0eac730 2024-04-16T17:14:59.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.511+0000 7f93f7e00640 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556ea0eac730 0x556ea0eaeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.511+0000 7f93f7e00640 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556ea0eac730 0x556ea0eaeb20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef31ae26907dd21e server_cookie=1da5837f4422a16b in_seq=0 out_seq=0 2024-04-16T17:14:59.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.511+0000 7f93dea00640 10 client.28219 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f93dea00640 1 -- 172.21.15.18:0/1140385720 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ea0e05a40 con 0x556ea0eac730 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f93dea00640 10 client.28219 handle_command_reply: tid=0 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 client.28219 shutdown 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 -- 172.21.15.18:0/1140385720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 msgr2=0x556ea0e99550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e99550 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009350 tx=0x7f93ec0969b0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 -- 172.21.15.18:0/1140385720 shutdown_connections 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556ea0eac730 0x556ea0eaeb20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e9c410 0x556ea0e99fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e93830 0x556ea0e99a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 --2- 172.21.15.18:0/1140385720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e99550 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f93dea00640 0 client.28219 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.512+0000 7f940241a740 1 -- 172.21.15.18:0/1140385720 >> 172.21.15.18:0/1140385720 conn(0x556ea0e8d640 msgr2=0x556ea0e8f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.513+0000 7f940241a740 1 -- 172.21.15.18:0/1140385720 shutdown_connections 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.513+0000 7f940241a740 1 -- 172.21.15.18:0/1140385720 wait complete. 2024-04-16T17:14:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.513+0000 7f940241a740 20 client.28219 trim_cache size 0 max 16384 2024-04-16T17:14:59.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.528+0000 7f940241a740 1 Processor -- start 2024-04-16T17:14:59.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f940241a740 1 -- start start 2024-04-16T17:14:59.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f940241a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e9bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f940241a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e93830 0x556ea0e98e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f940241a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e9c410 0x556ea0e8be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f940241a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81afb40 con 0x556ea0e9c410 2024-04-16T17:14:59.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f940241a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ad410 con 0x556ea0e93830 2024-04-16T17:14:59.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f940241a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ab260 con 0x556ea0e92f10 2024-04-16T17:14:59.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e93830 0x556ea0e98e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e93830 0x556ea0e98e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55034/0 (socket says 172.21.15.18:55034) 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f93f7e00640 1 -- 172.21.15.18:0/3980539218 learned_addr learned my addr 172.21.15.18:0/3980539218 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f93fd800640 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e9c410 0x556ea0e8be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.529+0000 7f93fce00640 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e9bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.530+0000 7f93f7e00640 1 -- 172.21.15.18:0/3980539218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 msgr2=0x556ea0e9bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.530+0000 7f93f7e00640 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e9bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.530+0000 7f93f7e00640 1 -- 172.21.15.18:0/3980539218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e9c410 msgr2=0x556ea0e8be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.530+0000 7f93f7e00640 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e9c410 0x556ea0e8be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.530+0000 7f93f7e00640 1 -- 172.21.15.18:0/3980539218 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea0b0a3f0 con 0x556ea0e93830 2024-04-16T17:14:59.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.530+0000 7f93fce00640 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e9bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:59.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.530+0000 7f93fd800640 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e9c410 0x556ea0e8be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:59.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.530+0000 7f93f7e00640 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e93830 0x556ea0e98e50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f93e402c2b0 tx=0x7f93e4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68937a41b2349c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.531+0000 7f93e9600640 1 -- 172.21.15.18:0/3980539218 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e404f070 con 0x556ea0e93830 2024-04-16T17:14:59.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.531+0000 7f93e9600640 1 -- 172.21.15.18:0/3980539218 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e402ce00 con 0x556ea0e93830 2024-04-16T17:14:59.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.531+0000 7f93e9600640 1 -- 172.21.15.18:0/3980539218 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4004b00 con 0x556ea0e93830 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.532+0000 7f940241a740 1 -- 172.21.15.18:0/3980539218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e93830 msgr2=0x556ea0e98e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.532+0000 7f940241a740 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e93830 0x556ea0e98e50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f93e402c2b0 tx=0x7f93e4005a20 comp rx=0 tx=0).stop 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.532+0000 7f940241a740 1 -- 172.21.15.18:0/3980539218 shutdown_connections 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.532+0000 7f940241a740 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0e9c410 0x556ea0e8be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.532+0000 7f940241a740 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e93830 0x556ea0e98e50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.532+0000 7f940241a740 1 --2- 172.21.15.18:0/3980539218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e92f10 0x556ea0e9bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.532+0000 7f940241a740 1 -- 172.21.15.18:0/3980539218 >> 172.21.15.18:0/3980539218 conn(0x556ea0e8d640 msgr2=0x556ea0ea7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.533+0000 7f940241a740 1 -- 172.21.15.18:0/3980539218 shutdown_connections 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.533+0000 7f940241a740 1 -- 172.21.15.18:0/3980539218 wait complete. 2024-04-16T17:14:59.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.533+0000 7f940241a740 1 -- start start 2024-04-16T17:14:59.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.533+0000 7f940241a740 1 Processor -- start 2024-04-16T17:14:59.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f940241a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e92f10 0x556ea0eaf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f940241a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e9c410 0x556ea0eafb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f940241a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0eb00b0 0x556ea0f11910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f940241a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ab8d0 con 0x556ea0eb00b0 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f940241a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81acd40 con 0x556ea0e92f10 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f940241a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81abfa0 con 0x556ea0e9c410 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f940241a740 10 client.0 _ll_register_callbacks cb 0x7f93f00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e9c410 0x556ea0eafb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e9c410 0x556ea0eafb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38818/0 (socket says 172.21.15.18:38818) 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f93f7e00640 1 -- 172.21.15.18:0/2228268454 learned_addr learned my addr 172.21.15.18:0/2228268454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f93fce00640 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e92f10 0x556ea0eaf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.534+0000 7f93fd800640 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0eb00b0 0x556ea0f11910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93f7e00640 1 -- 172.21.15.18:0/2228268454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e92f10 msgr2=0x556ea0eaf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93f7e00640 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e92f10 0x556ea0eaf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93f7e00640 1 -- 172.21.15.18:0/2228268454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0eb00b0 msgr2=0x556ea0f11910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93f7e00640 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0eb00b0 0x556ea0f11910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93f7e00640 1 -- 172.21.15.18:0/2228268454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ea0bd47d0 con 0x556ea0e9c410 2024-04-16T17:14:59.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93fce00640 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e92f10 0x556ea0eaf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93f7e00640 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e9c410 0x556ea0eafb70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f93e4036040 tx=0x7f93e4009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1deeb4a1cd724f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93dea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.535+0000 7f93dea00640 1 -- 172.21.15.18:0/2228268454 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4057060 con 0x556ea0e9c410 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.536+0000 7f93dea00640 1 -- 172.21.15.18:0/2228268454 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f93e402c920 con 0x556ea0e9c410 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.536+0000 7f940241a740 1 -- 172.21.15.18:0/2228268454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e402ce00 con 0x556ea0e9c410 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.536+0000 7f93dea00640 1 -- 172.21.15.18:0/2228268454 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93e405e050 con 0x556ea0e9c410 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.536+0000 7f940241a740 10 client.28225 fetch_fsmap learned FSMap version 29 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.536+0000 7f940241a740 1 -- 172.21.15.18:0/2228268454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ea0f12f40 con 0x556ea0e9c410 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.537+0000 7f93dea00640 1 -- 172.21.15.18:0/2228268454 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e402ce00 con 0x556ea0e9c410 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.537+0000 7f93dea00640 1 -- 172.21.15.18:0/2228268454 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f93e4005480 con 0x556ea0e9c410 2024-04-16T17:14:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.537+0000 7f940241a740 10 client.28225 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:14:59.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.537+0000 7f940241a740 10 client.28225 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:14:59.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.537+0000 7f940241a740 20 client.28225 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:59.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.537+0000 7f940241a740 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556ea0f135c0 0x556ea0f159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.537+0000 7f940241a740 4 client.28225 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:59.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.537+0000 7f940241a740 1 -- 172.21.15.18:0/2228268454 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ea0e8cd20 con 0x556ea0f135c0 2024-04-16T17:14:59.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.538+0000 7f93fce00640 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556ea0f135c0 0x556ea0f159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f93fce00640 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556ea0f135c0 0x556ea0f159e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a10c8528f5c4e997 server_cookie=6c8d24c79aa06fc2 in_seq=0 out_seq=0 2024-04-16T17:14:59.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f93dea00640 10 client.28225 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:14:59.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f93dea00640 1 -- 172.21.15.18:0/2228268454 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ea0e8cd20 con 0x556ea0f135c0 2024-04-16T17:14:59.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f93dea00640 10 client.28225 handle_command_reply: tid=0 2024-04-16T17:14:59.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 client.28225 shutdown 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 -- 172.21.15.18:0/2228268454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e9c410 msgr2=0x556ea0eafb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e9c410 0x556ea0eafb70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f93e4036040 tx=0x7f93e4009680 comp rx=0 tx=0).stop 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 -- 172.21.15.18:0/2228268454 shutdown_connections 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x556ea0f135c0 0x556ea0f159e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556ea0eb00b0 0x556ea0f11910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ea0e9c410 0x556ea0eafb70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 --2- 172.21.15.18:0/2228268454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ea0e92f10 0x556ea0eaf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 -- 172.21.15.18:0/2228268454 >> 172.21.15.18:0/2228268454 conn(0x556ea0e8d640 msgr2=0x556ea0e8e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f93dea00640 0 client.28225 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 -- 172.21.15.18:0/2228268454 shutdown_connections 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 1 -- 172.21.15.18:0/2228268454 wait complete. 2024-04-16T17:14:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.539+0000 7f940241a740 20 client.28225 trim_cache size 0 max 16384 2024-04-16T17:14:59.540 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:14:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 -- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f93e00a3da0 msgr2=0x7f93e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f93e00a3da0 0x7f93e00a6260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x556ea0e8e7d0 tx=0x7f93f0009500 comp rx=0 tx=0).stop 2024-04-16T17:14:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 -- 172.21.15.18:0/1805510414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8102fe0 msgr2=0x7f93f8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8102fe0 0x7f93f8198760 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e9f0 tx=0x7f93ec00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 -- 172.21.15.18:0/1805510414 shutdown_connections 2024-04-16T17:14:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f93e00a3da0 0x7f93e00a6260 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93f810c400 0x7f93f819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93f8103900 0x7f93f8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 --2- 172.21.15.18:0/1805510414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93f8102fe0 0x7f93f8198760 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.540+0000 7f93fe200640 1 -- 172.21.15.18:0/1805510414 >> 172.21.15.18:0/1805510414 conn(0x7f93f8077ab0 msgr2=0x7f93f80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.541+0000 7f93fe200640 1 -- 172.21.15.18:0/1805510414 shutdown_connections 2024-04-16T17:14:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.541+0000 7f93fe200640 1 -- 172.21.15.18:0/1805510414 wait complete. 2024-04-16T17:14:59.558 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:14:59.640 INFO:tasks.workunit.client.0.smithi018.stdout: 1 386977 3.67 MB/sec execute 432 sec latency 16.721 ms 2024-04-16T17:14:59.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 -- 172.21.15.18:0/3868628104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23881016f0 msgr2=0x7f2388105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 --2- 172.21.15.18:0/3868628104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23881016f0 0x7f2388105bb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2374009900 tx=0x7f237402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 -- 172.21.15.18:0/3868628104 shutdown_connections 2024-04-16T17:14:59.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 --2- 172.21.15.18:0/3868628104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23881061d0 0x7f2388106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 --2- 172.21.15.18:0/3868628104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23881016f0 0x7f2388105bb0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 --2- 172.21.15.18:0/3868628104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2388100dd0 0x7f23881011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 -- 172.21.15.18:0/3868628104 >> 172.21.15.18:0/3868628104 conn(0x7f2388077ab0 msgr2=0x7f23880fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 -- 172.21.15.18:0/3868628104 shutdown_connections 2024-04-16T17:14:59.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.772+0000 7f2387200640 1 -- 172.21.15.18:0/3868628104 wait complete. 2024-04-16T17:14:59.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.773+0000 7f2387200640 1 Processor -- start 2024-04-16T17:14:59.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.773+0000 7f2387200640 1 -- start start 2024-04-16T17:14:59.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.773+0000 7f2387200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2388100dd0 0x7f238819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2387200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23881016f0 0x7f238819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2387200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23881061d0 0x7f23881a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2387200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23881187f0 con 0x7f23881016f0 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2387200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2388118670 con 0x7f2388100dd0 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2387200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2388118970 con 0x7f23881061d0 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2388100dd0 0x7f238819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23881016f0 0x7f238819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2388100dd0 0x7f238819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55052/0 (socket says 172.21.15.18:55052) 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385e00640 1 -- 172.21.15.18:0/2047108603 learned_addr learned my addr 172.21.15.18:0/2047108603 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2386800640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23881061d0 0x7f23881a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385e00640 1 -- 172.21.15.18:0/2047108603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23881061d0 msgr2=0x7f23881a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385e00640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23881061d0 0x7f23881a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385e00640 1 -- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23881016f0 msgr2=0x7f238819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385e00640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23881016f0 0x7f238819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.774+0000 7f2385e00640 1 -- 172.21.15.18:0/2047108603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23881a3e00 con 0x7f2388100dd0 2024-04-16T17:14:59.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.775+0000 7f2385e00640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2388100dd0 0x7f238819ee10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f237800e9f0 tx=0x7f237800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.775+0000 7f237ea00640 1 -- 172.21.15.18:0/2047108603 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237800cd80 con 0x7f2388100dd0 2024-04-16T17:14:59.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.775+0000 7f237ea00640 1 -- 172.21.15.18:0/2047108603 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2378010910 con 0x7f2388100dd0 2024-04-16T17:14:59.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.775+0000 7f2387200640 1 -- 172.21.15.18:0/2047108603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23881a40f0 con 0x7f2388100dd0 2024-04-16T17:14:59.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.775+0000 7f237ea00640 1 -- 172.21.15.18:0/2047108603 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378018990 con 0x7f2388100dd0 2024-04-16T17:14:59.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.775+0000 7f2387200640 1 -- 172.21.15.18:0/2047108603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881ab980 con 0x7f2388100dd0 2024-04-16T17:14:59.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.779+0000 7f238f7ad740 1 Processor -- start 2024-04-16T17:14:59.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.779+0000 7f238f7ad740 1 -- start start 2024-04-16T17:14:59.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.779+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.779+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.779+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685fc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.779+0000 7f238f7ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555a68556020 con 0x555a685fa410 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.779+0000 7f238f7ad740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555a684fc460 con 0x555a685f1830 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.779+0000 7f238f7ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555a68551fe0 con 0x555a685f0f10 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2386800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2386800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38852/0 (socket says 172.21.15.18:38852) 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2386800640 1 -- 172.21.15.18:0/1112614256 learned_addr learned my addr 172.21.15.18:0/1112614256 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2385400640 1 --2- 172.21.15.18:0/1112614256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685fc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2385400640 1 -- 172.21.15.18:0/1112614256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 msgr2=0x555a685f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2385400640 1 --2- 172.21.15.18:0/1112614256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2385400640 1 -- 172.21.15.18:0/1112614256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 msgr2=0x555a685f9ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2385400640 1 --2- 172.21.15.18:0/1112614256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f9ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f2385400640 1 -- 172.21.15.18:0/1112614256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a68504040 con 0x555a685fa410 2024-04-16T17:14:59.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.780+0000 7f237ea00640 1 -- 172.21.15.18:0/2047108603 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2378010ac0 con 0x7f2388100dd0 2024-04-16T17:14:59.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.781+0000 7f2385400640 1 --2- 172.21.15.18:0/1112614256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685fc800 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f237402c210 tx=0x7f2374009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f5d162593758e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.781+0000 7f2373000640 1 -- 172.21.15.18:0/1112614256 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374043070 con 0x555a685fa410 2024-04-16T17:14:59.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.781+0000 7f2373000640 1 -- 172.21.15.18:0/1112614256 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237402c6f0 con 0x555a685fa410 2024-04-16T17:14:59.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.781+0000 7f2373000640 1 -- 172.21.15.18:0/1112614256 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740049b0 con 0x555a685fa410 2024-04-16T17:14:59.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f237ea00640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f236c0a3d20 0x7f236c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 -- 172.21.15.18:0/1112614256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 msgr2=0x555a685fc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 --2- 172.21.15.18:0/1112614256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685fc800 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f237402c210 tx=0x7f2374009560 comp rx=0 tx=0).stop 2024-04-16T17:14:59.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f237ea00640 1 -- 172.21.15.18:0/2047108603 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f23780d0fe0 con 0x7f2388100dd0 2024-04-16T17:14:59.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 -- 172.21.15.18:0/1112614256 shutdown_connections 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f2386800640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f236c0a3d20 0x7f236c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 --2- 172.21.15.18:0/1112614256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685fc800 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 --2- 172.21.15.18:0/1112614256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 --2- 172.21.15.18:0/1112614256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 -- 172.21.15.18:0/1112614256 >> 172.21.15.18:0/1112614256 conn(0x555a685eb640 msgr2=0x555a685eba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 -- 172.21.15.18:0/1112614256 shutdown_connections 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.782+0000 7f238f7ad740 1 -- 172.21.15.18:0/1112614256 wait complete. 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.783+0000 7f238f7ad740 1 -- start start 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.783+0000 7f2386800640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f236c0a3d20 0x7f236c0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2380005fd0 tx=0x7f2380009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.783+0000 7f238f7ad740 1 Processor -- start 2024-04-16T17:14:59.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f238f7ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555a685fd1a0 con 0x555a685fa410 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f238f7ad740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555a6850f830 con 0x555a685f1830 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f238f7ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555a685fd320 con 0x555a685f0f10 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f238f7ad740 10 client.0 _ll_register_callbacks cb 0x555a68517150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55054/0 (socket says 172.21.15.18:55054) 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385400640 1 -- 172.21.15.18:0/4214687435 learned_addr learned my addr 172.21.15.18:0/4214687435 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2386800640 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385e00640 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385400640 1 -- 172.21.15.18:0/4214687435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 msgr2=0x555a685f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385400640 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385400640 1 -- 172.21.15.18:0/4214687435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 msgr2=0x555a685f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385400640 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.784+0000 7f2385400640 1 -- 172.21.15.18:0/4214687435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a68513920 con 0x555a685f1830 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.785+0000 7f2385400640 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f7a50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2374006010 tx=0x7f2374004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d6be92b39851618 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.785+0000 7f2367400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.785+0000 7f2367400640 1 -- 172.21.15.18:0/4214687435 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374043050 con 0x555a685f1830 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.785+0000 7f238f7ad740 1 -- 172.21.15.18:0/4214687435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f237402c6f0 con 0x555a685f1830 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.785+0000 7f2367400640 1 -- 172.21.15.18:0/4214687435 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f23740050e0 con 0x555a685f1830 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.785+0000 7f2367400640 1 -- 172.21.15.18:0/4214687435 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2374059050 con 0x555a685f1830 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.785+0000 7f238f7ad740 10 client.28364 fetch_fsmap learned FSMap version 29 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.785+0000 7f238f7ad740 1 -- 172.21.15.18:0/4214687435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a684e17f0 con 0x555a685f1830 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.786+0000 7f2367400640 1 -- 172.21.15.18:0/4214687435 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237402c6f0 con 0x555a685f1830 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.786+0000 7f2367400640 1 -- 172.21.15.18:0/4214687435 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2374048b90 con 0x555a685f1830 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.786+0000 7f238f7ad740 10 client.28364 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.786+0000 7f238f7ad740 10 client.28364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.786+0000 7f238f7ad740 20 client.28364 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.786+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x555a6860a6a0 0x555a6860ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.786+0000 7f238f7ad740 4 client.28364 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.786+0000 7f238f7ad740 1 -- 172.21.15.18:0/4214687435 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a68563a40 con 0x555a6860a6a0 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.787+0000 7f2385e00640 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x555a6860a6a0 0x555a6860ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.787+0000 7f2385e00640 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x555a6860a6a0 0x555a6860ca90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6170122603b05d2d server_cookie=8a3081ae4463df96 in_seq=0 out_seq=0 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.787+0000 7f2367400640 10 client.28364 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.788+0000 7f2367400640 1 -- 172.21.15.18:0/4214687435 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a68563a40 con 0x555a6860a6a0 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.788+0000 7f2367400640 10 client.28364 handle_command_reply: tid=0 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 client.28364 shutdown 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 -- 172.21.15.18:0/4214687435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 msgr2=0x555a685f7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f7a50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2374006010 tx=0x7f2374004960 comp rx=0 tx=0).stop 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 -- 172.21.15.18:0/4214687435 shutdown_connections 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x555a6860a6a0 0x555a6860ca90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a685f7a50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f2367400640 0 client.28364 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4214687435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 -- 172.21.15.18:0/4214687435 >> 172.21.15.18:0/4214687435 conn(0x555a685eb640 msgr2=0x555a685ed390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 -- 172.21.15.18:0/4214687435 shutdown_connections 2024-04-16T17:14:59.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.789+0000 7f238f7ad740 1 -- 172.21.15.18:0/4214687435 wait complete. 2024-04-16T17:14:59.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.790+0000 7f238f7ad740 20 client.28364 trim_cache size 0 max 16384 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.806+0000 7f238f7ad740 1 Processor -- start 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f238f7ad740 1 -- start start 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a68606060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f238f7ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23881b82b0 con 0x555a685fa410 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f238f7ad740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f23881b5b80 con 0x555a685f1830 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f238f7ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23881b39d0 con 0x555a685f0f10 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f2385e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f2385e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38868/0 (socket says 172.21.15.18:38868) 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f2385e00640 1 -- 172.21.15.18:0/4263390496 learned_addr learned my addr 172.21.15.18:0/4263390496 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f2385400640 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a68606060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.807+0000 7f2386800640 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2385400640 1 -- 172.21.15.18:0/4263390496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 msgr2=0x555a685f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2385400640 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2385400640 1 -- 172.21.15.18:0/4263390496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 msgr2=0x555a685eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2385400640 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685eba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2385400640 1 -- 172.21.15.18:0/4263390496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a685f64c0 con 0x555a685f1830 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2385e00640 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2385400640 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a68606060 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2374005ec0 tx=0x7f2374004d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e723bda41a67d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2372600640 1 -- 172.21.15.18:0/4263390496 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374043030 con 0x555a685f1830 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2372600640 1 -- 172.21.15.18:0/4263390496 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237403f560 con 0x555a685f1830 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.808+0000 7f2372600640 1 -- 172.21.15.18:0/4263390496 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740053f0 con 0x555a685f1830 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 -- 172.21.15.18:0/4263390496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 msgr2=0x555a68606060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a68606060 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2374005ec0 tx=0x7f2374004d60 comp rx=0 tx=0).stop 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 -- 172.21.15.18:0/4263390496 shutdown_connections 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685fa410 0x555a685eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f1830 0x555a68606060 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 --2- 172.21.15.18:0/4263390496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685f0f10 0x555a685f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 -- 172.21.15.18:0/4263390496 >> 172.21.15.18:0/4263390496 conn(0x555a685eb640 msgr2=0x555a685ea700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 -- 172.21.15.18:0/4263390496 shutdown_connections 2024-04-16T17:14:59.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.809+0000 7f238f7ad740 1 -- 172.21.15.18:0/4263390496 wait complete. 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.810+0000 7f238f7ad740 1 -- start start 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.810+0000 7f238f7ad740 1 Processor -- start 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f0f10 0x555a685fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685f1830 0x555a685fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f238f7ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685fa410 0x555a685fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f238f7ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23881b4040 con 0x555a685f1830 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f238f7ad740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f23881b54b0 con 0x555a685f0f10 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f238f7ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23881b4710 con 0x555a685fa410 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f238f7ad740 10 client.0 _ll_register_callbacks cb 0x7f23800099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f2385e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f0f10 0x555a685fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f2385e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f0f10 0x555a685fdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55072/0 (socket says 172.21.15.18:55072) 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f2385e00640 1 -- 172.21.15.18:0/3216250649 learned_addr learned my addr 172.21.15.18:0/3216250649 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.811+0000 7f2386800640 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685fa410 0x555a685fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2385400640 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685f1830 0x555a685fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2385e00640 1 -- 172.21.15.18:0/3216250649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685fa410 msgr2=0x555a685fe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2385e00640 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685fa410 0x555a685fe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2385e00640 1 -- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685f1830 msgr2=0x555a685fe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2385e00640 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685f1830 0x555a685fe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2385e00640 1 -- 172.21.15.18:0/3216250649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a682683f0 con 0x555a685f0f10 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2385400640 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685f1830 0x555a685fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2385e00640 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f0f10 0x555a685fdeb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f23780bd010 tx=0x7f2378018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aabee66b0b408f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2367400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2367400640 1 -- 172.21.15.18:0/3216250649 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780b5030 con 0x555a685f0f10 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f238f7ad740 1 -- 172.21.15.18:0/3216250649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f237403f560 con 0x555a685f0f10 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2367400640 1 -- 172.21.15.18:0/3216250649 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f23780bd0b0 con 0x555a685f0f10 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f2367400640 1 -- 172.21.15.18:0/3216250649 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23780a57e0 con 0x555a685f0f10 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f238f7ad740 10 client.28370 fetch_fsmap learned FSMap version 29 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.812+0000 7f238f7ad740 1 -- 172.21.15.18:0/3216250649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a683327d0 con 0x555a685f0f10 2024-04-16T17:14:59.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f2367400640 1 -- 172.21.15.18:0/3216250649 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237403f560 con 0x555a685f0f10 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f2367400640 1 -- 172.21.15.18:0/3216250649 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f23780b7020 con 0x555a685f0f10 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f238f7ad740 10 client.28370 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f238f7ad740 10 client.28370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f238f7ad740 20 client.28370 populate_metadata read hostname 'smithi018' 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f238f7ad740 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x555a68670f40 0x555a68673360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f238f7ad740 4 client.28370 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f238f7ad740 1 -- 172.21.15.18:0/3216250649 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a6855d8b0 con 0x555a68670f40 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.813+0000 7f2385400640 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x555a68670f40 0x555a68673360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:59.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.814+0000 7f2385400640 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x555a68670f40 0x555a68673360 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9a3df5704adb1f server_cookie=e88e5ea9139b9888 in_seq=0 out_seq=0 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.814+0000 7f2367400640 10 client.28370 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.814+0000 7f2367400640 1 -- 172.21.15.18:0/3216250649 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a6855d8b0 con 0x555a68670f40 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.814+0000 7f2367400640 10 client.28370 handle_command_reply: tid=0 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.814+0000 7f238f7ad740 1 client.28370 shutdown 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 -- 172.21.15.18:0/3216250649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f0f10 msgr2=0x555a685fdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f0f10 0x555a685fdeb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f23780bd010 tx=0x7f2378018420 comp rx=0 tx=0).stop 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 -- 172.21.15.18:0/3216250649 shutdown_connections 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x555a68670f40 0x555a68673360 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555a685fa410 0x555a685fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555a685f1830 0x555a685fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 --2- 172.21.15.18:0/3216250649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a685f0f10 0x555a685fdeb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f2367400640 0 client.28370 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 -- 172.21.15.18:0/3216250649 >> 172.21.15.18:0/3216250649 conn(0x555a685eb640 msgr2=0x555a685ea700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 -- 172.21.15.18:0/3216250649 shutdown_connections 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 1 -- 172.21.15.18:0/3216250649 wait complete. 2024-04-16T17:14:59.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.815+0000 7f238f7ad740 20 client.28370 trim_cache size 0 max 16384 2024-04-16T17:14:59.817 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:14:59.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 -- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f236c0a3d20 msgr2=0x7f236c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f236c0a3d20 0x7f236c0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2380005fd0 tx=0x7f2380009500 comp rx=0 tx=0).stop 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 -- 172.21.15.18:0/2047108603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2388100dd0 msgr2=0x7f238819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2388100dd0 0x7f238819ee10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f237800e9f0 tx=0x7f237800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 -- 172.21.15.18:0/2047108603 shutdown_connections 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f236c0a3d20 0x7f236c0a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23881061d0 0x7f23881a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23881016f0 0x7f238819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 --2- 172.21.15.18:0/2047108603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2388100dd0 0x7f238819ee10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 -- 172.21.15.18:0/2047108603 >> 172.21.15.18:0/2047108603 conn(0x7f2388077ab0 msgr2=0x7f2388103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 -- 172.21.15.18:0/2047108603 shutdown_connections 2024-04-16T17:14:59.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:14:59.816+0000 7f2387200640 1 -- 172.21.15.18:0/2047108603 wait complete. 2024-04-16T17:14:59.840 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:15:00.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:59 smithi043 ceph-mon[36901]: pgmap v321: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-16T17:15:00.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:14:59 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2688626017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 -- 172.21.15.18:0/2365072301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c107300 msgr2=0x7f265c10eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 --2- 172.21.15.18:0/2365072301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c107300 0x7f265c10eed0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2654009ca0 tx=0x7f265402c100 comp rx=0 tx=0).stop 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 -- 172.21.15.18:0/2365072301 shutdown_connections 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 --2- 172.21.15.18:0/2365072301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c107300 0x7f265c10eed0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 --2- 172.21.15.18:0/2365072301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c102be0 0x7f265c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 --2- 172.21.15.18:0/2365072301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f265c102230 0x7f265c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 -- 172.21.15.18:0/2365072301 >> 172.21.15.18:0/2365072301 conn(0x7f265c0fdab0 msgr2=0x7f265c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 -- 172.21.15.18:0/2365072301 shutdown_connections 2024-04-16T17:15:00.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.047+0000 7f2663200640 1 -- 172.21.15.18:0/2365072301 wait complete. 2024-04-16T17:15:00.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.049+0000 7f2663200640 1 Processor -- start 2024-04-16T17:15:00.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.049+0000 7f2663200640 1 -- start start 2024-04-16T17:15:00.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2663200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102230 0x7f265c074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2663200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c107300 0x7f265c074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2663200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f265c074d80 0x7f265c1a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2663200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1123c0 con 0x7f265c074d80 2024-04-16T17:15:00.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2663200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f265c112240 con 0x7f265c102230 2024-04-16T17:15:00.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2663200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f265c112540 con 0x7f265c107300 2024-04-16T17:15:00.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2661200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f265c074d80 0x7f265c1a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f265be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102230 0x7f265c074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f265b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c107300 0x7f265c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2661200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f265c074d80 0x7f265c1a6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48982/0 (socket says 172.21.15.18:48982) 2024-04-16T17:15:00.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f2661200640 1 -- 172.21.15.18:0/3698860949 learned_addr learned my addr 172.21.15.18:0/3698860949 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:00.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f265be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102230 0x7f265c074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55094/0 (socket says 172.21.15.18:55094) 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f265b400640 1 -- 172.21.15.18:0/3698860949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102230 msgr2=0x7f265c074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f265b400640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102230 0x7f265c074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f265b400640 1 -- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f265c074d80 msgr2=0x7f265c1a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f265b400640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f265c074d80 0x7f265c1a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.050+0000 7f265b400640 1 -- 172.21.15.18:0/3698860949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c1a7230 con 0x7f265c107300 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.051+0000 7f265b400640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c107300 0x7f265c074840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f264800e9c0 tx=0x7f264800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.051+0000 7f2658c00640 1 -- 172.21.15.18:0/3698860949 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264800ccc0 con 0x7f265c107300 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.051+0000 7f2663200640 1 -- 172.21.15.18:0/3698860949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c1a7520 con 0x7f265c107300 2024-04-16T17:15:00.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.051+0000 7f2658c00640 1 -- 172.21.15.18:0/3698860949 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648005020 con 0x7f265c107300 2024-04-16T17:15:00.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.051+0000 7f2658c00640 1 -- 172.21.15.18:0/3698860949 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648004a00 con 0x7f265c107300 2024-04-16T17:15:00.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.051+0000 7f2663200640 1 -- 172.21.15.18:0/3698860949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c1a7a20 con 0x7f265c107300 2024-04-16T17:15:00.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.054+0000 7f2665fb0740 1 Processor -- start 2024-04-16T17:15:00.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f2665fb0740 1 -- start start 2024-04-16T17:15:00.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e41d50 0x563b02e42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e4ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 0x563b02e4d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f2665fb0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563b02da7020 con 0x563b02e41d50 2024-04-16T17:15:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f2665fb0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563b02d4d460 con 0x563b02e4b250 2024-04-16T17:15:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f2665fb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563b02da2fe0 con 0x563b02e42670 2024-04-16T17:15:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f2661200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e41d50 0x563b02e42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f265be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e4ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f265be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e4ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38908/0 (socket says 172.21.15.18:38908) 2024-04-16T17:15:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.055+0000 7f265be00640 1 -- 172.21.15.18:0/875224572 learned_addr learned my addr 172.21.15.18:0/875224572 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.056+0000 7f265be00640 1 -- 172.21.15.18:0/875224572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 msgr2=0x563b02e4d640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:15:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.056+0000 7f265be00640 1 --2- 172.21.15.18:0/875224572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 0x563b02e4d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.056+0000 7f265be00640 1 -- 172.21.15.18:0/875224572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e41d50 msgr2=0x563b02e42130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.056+0000 7f265be00640 1 --2- 172.21.15.18:0/875224572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e41d50 0x563b02e42130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.056+0000 7f265be00640 1 -- 172.21.15.18:0/875224572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b02d55040 con 0x563b02e42670 2024-04-16T17:15:00.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.056+0000 7f265be00640 1 --2- 172.21.15.18:0/875224572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e4ad10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f264c00bd10 tx=0x7f264c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebee3d529c42b57d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:00.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.056+0000 7f2651600640 1 -- 172.21.15.18:0/875224572 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c004030 con 0x563b02e42670 2024-04-16T17:15:00.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.056+0000 7f2658c00640 1 -- 172.21.15.18:0/3698860949 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f26480051d0 con 0x7f265c107300 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2651600640 1 -- 172.21.15.18:0/875224572 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c0041d0 con 0x563b02e42670 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2651600640 1 -- 172.21.15.18:0/875224572 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0117e0 con 0x563b02e42670 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2665fb0740 1 -- 172.21.15.18:0/875224572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 msgr2=0x563b02e4ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2665fb0740 1 --2- 172.21.15.18:0/875224572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e4ad10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f264c00bd10 tx=0x7f264c0107a0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2665fb0740 1 -- 172.21.15.18:0/875224572 shutdown_connections 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2665fb0740 1 --2- 172.21.15.18:0/875224572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 0x563b02e4d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2665fb0740 1 --2- 172.21.15.18:0/875224572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e4ad10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2665fb0740 1 --2- 172.21.15.18:0/875224572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e41d50 0x563b02e42130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.057+0000 7f2665fb0740 1 -- 172.21.15.18:0/875224572 >> 172.21.15.18:0/875224572 conn(0x563b02e3c480 msgr2=0x563b02e3c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.058+0000 7f2665fb0740 1 -- 172.21.15.18:0/875224572 shutdown_connections 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.058+0000 7f2665fb0740 1 -- 172.21.15.18:0/875224572 wait complete. 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.058+0000 7f2665fb0740 1 -- start start 2024-04-16T17:15:00.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.058+0000 7f2665fb0740 1 Processor -- start 2024-04-16T17:15:00.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e41d50 0x563b02e48370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e42670 0x563b02e488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 0x563b02e48df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2665fb0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563b02e4e040 con 0x563b02e42670 2024-04-16T17:15:00.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2665fb0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563b02d60830 con 0x563b02e4b250 2024-04-16T17:15:00.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2665fb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563b02e4e1c0 con 0x563b02e41d50 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2665fb0740 10 client.0 _ll_register_callbacks cb 0x563b02d68150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2658c00640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f26400a3c40 0x7f26400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2661200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e41d50 0x563b02e48370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f265be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e42670 0x563b02e488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f265be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e42670 0x563b02e488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49006/0 (socket says 172.21.15.18:49006) 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f265be00640 1 -- 172.21.15.18:0/923058914 learned_addr learned my addr 172.21.15.18:0/923058914 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f265b400640 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 0x563b02e48df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.059+0000 7f2658c00640 1 -- 172.21.15.18:0/3698860949 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f26480d0af0 con 0x7f265c107300 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f265be00640 1 -- 172.21.15.18:0/923058914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e41d50 msgr2=0x563b02e48370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f265be00640 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e41d50 0x563b02e48370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f265be00640 1 -- 172.21.15.18:0/923058914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 msgr2=0x563b02e48df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f265be00640 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 0x563b02e48df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f265be00640 1 -- 172.21.15.18:0/923058914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b02d64920 con 0x563b02e42670 2024-04-16T17:15:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f2661200640 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e41d50 0x563b02e48370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:00.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f2661200640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f26400a3c40 0x7f26400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f265be00640 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e42670 0x563b02e488b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f264c00c010 tx=0x7f264c003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b312cb5264499 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:00.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.060+0000 7f2646000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:00.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.061+0000 7f2646000640 1 -- 172.21.15.18:0/923058914 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c029060 con 0x563b02e42670 2024-04-16T17:15:00.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.061+0000 7f2646000640 1 -- 172.21.15.18:0/923058914 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f264c0035f0 con 0x563b02e42670 2024-04-16T17:15:00.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.061+0000 7f2665fb0740 1 -- 172.21.15.18:0/923058914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f264c0041d0 con 0x563b02e42670 2024-04-16T17:15:00.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.061+0000 7f2661200640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f26400a3c40 0x7f26400a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2654009a60 tx=0x7f265402e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.061+0000 7f2646000640 1 -- 172.21.15.18:0/923058914 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f264c004000 con 0x563b02e42670 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.061+0000 7f2665fb0740 10 client.18084 fetch_fsmap learned FSMap version 29 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.061+0000 7f2665fb0740 1 -- 172.21.15.18:0/923058914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b02d327f0 con 0x563b02e42670 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f2646000640 1 -- 172.21.15.18:0/923058914 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c0041d0 con 0x563b02e42670 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f2646000640 1 -- 172.21.15.18:0/923058914 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f264c004900 con 0x563b02e42670 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f2665fb0740 10 client.18084 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f2665fb0740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f2665fb0740 20 client.18084 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f2665fb0740 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563b02e5b660 0x563b02e5da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f2665fb0740 4 client.18084 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f2665fb0740 1 -- 172.21.15.18:0/923058914 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b02db4a40 con 0x563b02e5b660 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.062+0000 7f265b400640 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563b02e5b660 0x563b02e5da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.063+0000 7f265b400640 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563b02e5b660 0x563b02e5da50 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4bb218ea07b8af server_cookie=aa35a12ca4fd1825 in_seq=0 out_seq=0 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.063+0000 7f2646000640 10 client.18084 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2646000640 1 -- 172.21.15.18:0/923058914 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b02db4a40 con 0x563b02e5b660 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2646000640 10 client.18084 handle_command_reply: tid=0 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 client.18084 shutdown 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 -- 172.21.15.18:0/923058914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e42670 msgr2=0x563b02e488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e42670 0x563b02e488b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f264c00c010 tx=0x7f264c003dd0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 -- 172.21.15.18:0/923058914 shutdown_connections 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563b02e5b660 0x563b02e5da50 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e4b250 0x563b02e48df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e42670 0x563b02e488b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 --2- 172.21.15.18:0/923058914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e41d50 0x563b02e48370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2646000640 0 client.18084 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 -- 172.21.15.18:0/923058914 >> 172.21.15.18:0/923058914 conn(0x563b02e3c480 msgr2=0x563b02e3e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:00.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 -- 172.21.15.18:0/923058914 shutdown_connections 2024-04-16T17:15:00.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.064+0000 7f2665fb0740 1 -- 172.21.15.18:0/923058914 wait complete. 2024-04-16T17:15:00.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.065+0000 7f2665fb0740 20 client.18084 trim_cache size 0 max 16384 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.080+0000 7f2665fb0740 1 Processor -- start 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.080+0000 7f2665fb0740 1 -- start start 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.080+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e4aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.080+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e56f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.080+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e47530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.080+0000 7f2665fb0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1b3fa0 con 0x563b02e4b250 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.080+0000 7f2665fb0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1b1870 con 0x563b02e41d50 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.080+0000 7f2665fb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1af6c0 con 0x563b02e42670 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f2661200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e47530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e56f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e4aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e56f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38932/0 (socket says 172.21.15.18:38932) 2024-04-16T17:15:00.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e4aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55120/0 (socket says 172.21.15.18:55120) 2024-04-16T17:15:00.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265b400640 1 -- 172.21.15.18:0/1303000367 learned_addr learned my addr 172.21.15.18:0/1303000367 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:00.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265be00640 1 -- 172.21.15.18:0/1303000367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 msgr2=0x563b02e56f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:00.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265be00640 1 --2- 172.21.15.18:0/1303000367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e56f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265be00640 1 -- 172.21.15.18:0/1303000367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 msgr2=0x563b02e47530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:00.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265be00640 1 --2- 172.21.15.18:0/1303000367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e47530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265be00640 1 -- 172.21.15.18:0/1303000367 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b02e57e80 con 0x563b02e41d50 2024-04-16T17:15:00.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f2661200640 1 --2- 172.21.15.18:0/1303000367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e47530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.081+0000 7f265be00640 1 --2- 172.21.15.18:0/1303000367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e4aaa0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f264c001eb0 tx=0x7f264c003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=201487e7c77b639f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2650c00640 1 -- 172.21.15.18:0/1303000367 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c029070 con 0x563b02e41d50 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2650c00640 1 -- 172.21.15.18:0/1303000367 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c00b7e0 con 0x563b02e41d50 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2650c00640 1 -- 172.21.15.18:0/1303000367 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0049b0 con 0x563b02e41d50 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2665fb0740 1 -- 172.21.15.18:0/1303000367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 msgr2=0x563b02e4aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2665fb0740 1 --2- 172.21.15.18:0/1303000367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e4aaa0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f264c001eb0 tx=0x7f264c003d60 comp rx=0 tx=0).stop 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2665fb0740 1 -- 172.21.15.18:0/1303000367 shutdown_connections 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2665fb0740 1 --2- 172.21.15.18:0/1303000367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e47530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2665fb0740 1 --2- 172.21.15.18:0/1303000367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e56f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2665fb0740 1 --2- 172.21.15.18:0/1303000367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e4aaa0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.082+0000 7f2665fb0740 1 -- 172.21.15.18:0/1303000367 >> 172.21.15.18:0/1303000367 conn(0x563b02e3c480 msgr2=0x563b02e3c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.083+0000 7f2665fb0740 1 -- 172.21.15.18:0/1303000367 shutdown_connections 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.083+0000 7f2665fb0740 1 -- 172.21.15.18:0/1303000367 wait complete. 2024-04-16T17:15:00.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.083+0000 7f2665fb0740 1 -- start start 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.083+0000 7f2665fb0740 1 Processor -- start 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e53410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e53950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f2665fb0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e44840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f2665fb0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1afd30 con 0x563b02e4b250 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f2665fb0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1b11a0 con 0x563b02e41d50 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f2665fb0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1b0400 con 0x563b02e42670 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f2665fb0740 10 client.0 _ll_register_callbacks cb 0x7f2648005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e53410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f2661200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e44840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e53410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55122/0 (socket says 172.21.15.18:55122) 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265be00640 1 -- 172.21.15.18:0/653720550 learned_addr learned my addr 172.21.15.18:0/653720550 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265b400640 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e53950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265be00640 1 -- 172.21.15.18:0/653720550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 msgr2=0x563b02e53950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265be00640 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e53950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265be00640 1 -- 172.21.15.18:0/653720550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 msgr2=0x563b02e44840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265be00640 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e44840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.084+0000 7f265be00640 1 -- 172.21.15.18:0/653720550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b02ab93f0 con 0x563b02e41d50 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.085+0000 7f265be00640 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e53410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f264c005d80 tx=0x7f264c003c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=703f6026e0877ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.085+0000 7f2646000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.085+0000 7f2646000640 1 -- 172.21.15.18:0/653720550 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c029070 con 0x563b02e41d50 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.085+0000 7f2646000640 1 -- 172.21.15.18:0/653720550 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f264c0035f0 con 0x563b02e41d50 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.085+0000 7f2665fb0740 1 -- 172.21.15.18:0/653720550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f264c00b7e0 con 0x563b02e41d50 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.085+0000 7f2646000640 1 -- 172.21.15.18:0/653720550 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f264c046050 con 0x563b02e41d50 2024-04-16T17:15:00.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.085+0000 7f2665fb0740 10 client.28385 fetch_fsmap learned FSMap version 29 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.085+0000 7f2665fb0740 1 -- 172.21.15.18:0/653720550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b02b837d0 con 0x563b02e41d50 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f2646000640 1 -- 172.21.15.18:0/653720550 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c00b7e0 con 0x563b02e41d50 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f2646000640 1 -- 172.21.15.18:0/653720550 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f264c004ec0 con 0x563b02e41d50 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f2665fb0740 10 client.28385 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f2665fb0740 10 client.28385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f2665fb0740 20 client.28385 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f2665fb0740 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563b02ec15a0 0x563b02ec3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f2665fb0740 4 client.28385 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f2665fb0740 1 -- 172.21.15.18:0/653720550 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b02dae8b0 con 0x563b02ec15a0 2024-04-16T17:15:00.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.086+0000 7f265b400640 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563b02ec15a0 0x563b02ec3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:00.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f265b400640 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563b02ec15a0 0x563b02ec3990 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc36fa1309e4b6b server_cookie=dc6ca049cd6fa5fb in_seq=0 out_seq=0 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2646000640 10 client.28385 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2646000640 1 -- 172.21.15.18:0/653720550 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b02dae8b0 con 0x563b02ec15a0 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2646000640 10 client.28385 handle_command_reply: tid=0 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 client.28385 shutdown 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 -- 172.21.15.18:0/653720550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 msgr2=0x563b02e53410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e53410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f264c005d80 tx=0x7f264c003c70 comp rx=0 tx=0).stop 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 -- 172.21.15.18:0/653720550 shutdown_connections 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563b02ec15a0 0x563b02ec3990 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563b02e4b250 0x563b02e44840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b02e42670 0x563b02e53950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 --2- 172.21.15.18:0/653720550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b02e41d50 0x563b02e53410 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 -- 172.21.15.18:0/653720550 >> 172.21.15.18:0/653720550 conn(0x563b02e3c480 msgr2=0x563b02e48310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2646000640 0 client.28385 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.087+0000 7f2665fb0740 1 -- 172.21.15.18:0/653720550 shutdown_connections 2024-04-16T17:15:00.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.088+0000 7f2665fb0740 1 -- 172.21.15.18:0/653720550 wait complete. 2024-04-16T17:15:00.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.088+0000 7f2665fb0740 20 client.28385 trim_cache size 0 max 16384 2024-04-16T17:15:00.091 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.088+0000 7f2663200640 1 -- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f26400a3c40 msgr2=0x7f26400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.088+0000 7f2663200640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f26400a3c40 0x7f26400a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2654009a60 tx=0x7f265402e040 comp rx=0 tx=0).stop 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.088+0000 7f2663200640 1 -- 172.21.15.18:0/3698860949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c107300 msgr2=0x7f265c074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.088+0000 7f2663200640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c107300 0x7f265c074840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f264800e9c0 tx=0x7f264800ee80 comp rx=0 tx=0).stop 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.089+0000 7f2663200640 1 -- 172.21.15.18:0/3698860949 shutdown_connections 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.089+0000 7f2663200640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f26400a3c40 0x7f26400a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.089+0000 7f2663200640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f265c074d80 0x7f265c1a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.089+0000 7f2663200640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f265c107300 0x7f265c074840 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.089+0000 7f2663200640 1 --2- 172.21.15.18:0/3698860949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102230 0x7f265c074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.089+0000 7f2663200640 1 -- 172.21.15.18:0/3698860949 >> 172.21.15.18:0/3698860949 conn(0x7f265c0fdab0 msgr2=0x7f265c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.089+0000 7f2663200640 1 -- 172.21.15.18:0/3698860949 shutdown_connections 2024-04-16T17:15:00.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:00.089+0000 7f2663200640 1 -- 172.21.15.18:0/3698860949 wait complete. 2024-04-16T17:15:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:59 smithi022 ceph-mon[36023]: pgmap v321: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-16T17:15:00.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:14:59 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2688626017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:00.640 INFO:tasks.workunit.client.0.smithi018.stdout: 1 387726 3.67 MB/sec execute 433 sec latency 19.737 ms 2024-04-16T17:15:01.108 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:15:01.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:15:01.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:15:01.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- 172.21.15.18:0/390035974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c102070 msgr2=0x7f010c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 --2- 172.21.15.18:0/390035974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c102070 0x7f010c102450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009900 tx=0x7f00fc02bc50 comp rx=0 tx=0).stop 2024-04-16T17:15:01.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- 172.21.15.18:0/390035974 shutdown_connections 2024-04-16T17:15:01.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 --2- 172.21.15.18:0/390035974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010c107140 0x7f010c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 --2- 172.21.15.18:0/390035974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c102a20 0x7f010c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 --2- 172.21.15.18:0/390035974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c102070 0x7f010c102450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- 172.21.15.18:0/390035974 >> 172.21.15.18:0/390035974 conn(0x7f010c0fd950 msgr2=0x7f010c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- 172.21.15.18:0/390035974 shutdown_connections 2024-04-16T17:15:01.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- 172.21.15.18:0/390035974 wait complete. 2024-04-16T17:15:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 Processor -- start 2024-04-16T17:15:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- start start 2024-04-16T17:15:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c102070 0x7f010c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c107140 0x7f010c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010c19a860 0x7f010c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1122c0 con 0x7f010c107140 2024-04-16T17:15:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f010c112140 con 0x7f010c102070 2024-04-16T17:15:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f010b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f010c112440 con 0x7f010c19a860 2024-04-16T17:15:01.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c107140 0x7f010c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c102070 0x7f010c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c107140 0x7f010c1a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49038/0 (socket says 172.21.15.18:49038) 2024-04-16T17:15:01.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109400640 1 -- 172.21.15.18:0/3781810951 learned_addr learned my addr 172.21.15.18:0/3781810951 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109e00640 1 -- 172.21.15.18:0/3781810951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010c19a860 msgr2=0x7f010c19ad10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:15:01.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109e00640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010c19a860 0x7f010c19ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109e00640 1 -- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c107140 msgr2=0x7f010c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109e00640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c107140 0x7f010c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109e00640 1 -- 172.21.15.18:0/3781810951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c19b380 con 0x7f010c102070 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.304+0000 7f0109400640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c107140 0x7f010c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.311+0000 7f0109e00640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c102070 0x7f010c1a06b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f00fc02c190 tx=0x7f00fc004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.312+0000 7f0102a00640 1 -- 172.21.15.18:0/3781810951 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc041070 con 0x7f010c102070 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.312+0000 7f010b200640 1 -- 172.21.15.18:0/3781810951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c19b610 con 0x7f010c102070 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.312+0000 7f010b200640 1 -- 172.21.15.18:0/3781810951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c1a78b0 con 0x7f010c102070 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.312+0000 7f0102a00640 1 -- 172.21.15.18:0/3781810951 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00fc004c80 con 0x7f010c102070 2024-04-16T17:15:01.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.312+0000 7f0102a00640 1 -- 172.21.15.18:0/3781810951 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc04b060 con 0x7f010c102070 2024-04-16T17:15:01.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f01137e9740 1 Processor -- start 2024-04-16T17:15:01.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f01137e9740 1 -- start start 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f01137e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e5e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f01137e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e66e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f01137e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e69780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f01137e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559891dc2cb0 con 0x559891e5de90 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f01137e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559891d69370 con 0x559891e67390 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f01137e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559891dbec70 con 0x559891e5e7b0 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f010a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e5e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f010a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e5e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49052/0 (socket says 172.21.15.18:49052) 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f010a800640 1 -- 172.21.15.18:0/3307500815 learned_addr learned my addr 172.21.15.18:0/3307500815 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f0109400640 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e69780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f0102a00640 1 -- 172.21.15.18:0/3781810951 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f00fc044070 con 0x7f010c102070 2024-04-16T17:15:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f0109400640 1 -- 172.21.15.18:0/3307500815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 msgr2=0x559891e66e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:15:01.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f0109400640 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e66e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f0109400640 1 -- 172.21.15.18:0/3307500815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 msgr2=0x559891e5e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f0109400640 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e5e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f0109400640 1 -- 172.21.15.18:0/3307500815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559891d88910 con 0x559891e67390 2024-04-16T17:15:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f010a800640 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e5e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f0109400640 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e69780 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f010c108b00 tx=0x7f00f80119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb7e08b6f111a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f00f7000640 1 -- 172.21.15.18:0/3307500815 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f801e070 con 0x559891e67390 2024-04-16T17:15:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f00f7000640 1 -- 172.21.15.18:0/3307500815 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f80035f0 con 0x559891e67390 2024-04-16T17:15:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.313+0000 7f00f7000640 1 -- 172.21.15.18:0/3307500815 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f800f850 con 0x559891e67390 2024-04-16T17:15:01.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f0102a00640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00f00a3da0 0x7f00f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f0102a00640 1 -- 172.21.15.18:0/3781810951 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f00fc0f23b0 con 0x7f010c102070 2024-04-16T17:15:01.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- 172.21.15.18:0/3307500815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 msgr2=0x559891e69780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e69780 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f010c108b00 tx=0x7f00f80119a0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00f00a3da0 0x7f00f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- 172.21.15.18:0/3307500815 shutdown_connections 2024-04-16T17:15:01.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e69780 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e66e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 --2- 172.21.15.18:0/3307500815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e5e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- 172.21.15.18:0/3307500815 >> 172.21.15.18:0/3307500815 conn(0x559891e585c0 msgr2=0x559891e589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- 172.21.15.18:0/3307500815 shutdown_connections 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- 172.21.15.18:0/3307500815 wait complete. 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- start start 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 Processor -- start 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00f00a3da0 0x7f00f00a6260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0104009b30 tx=0x7f010400b040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e5de90 0x559891e64410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e64950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e67390 0x559891e64e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559891e6a180 con 0x559891e67390 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559891d7c730 con 0x559891e5de90 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559891e6a300 con 0x559891e5e7b0 2024-04-16T17:15:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 10 client.0 _ll_register_callbacks cb 0x559891d4b1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e67390 0x559891e64e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f0109400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e64950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e67390 0x559891e64e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49058/0 (socket says 172.21.15.18:49058) 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f0109400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e64950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38984/0 (socket says 172.21.15.18:38984) 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 -- 172.21.15.18:0/3627470824 learned_addr learned my addr 172.21.15.18:0/3627470824 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 -- 172.21.15.18:0/3627470824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 msgr2=0x559891e64950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e64950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 -- 172.21.15.18:0/3627470824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e5de90 msgr2=0x559891e64410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e5de90 0x559891e64410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 -- 172.21.15.18:0/3627470824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559891b9d7e0 con 0x559891e67390 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f010a800640 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e67390 0x559891e64e90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0104015a50 tx=0x7f0104004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a664eeafc0ed0e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f00eb400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:01.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f00eb400640 1 -- 172.21.15.18:0/3627470824 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104013d90 con 0x559891e67390 2024-04-16T17:15:01.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f00eb400640 1 -- 172.21.15.18:0/3627470824 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f010401f3f0 con 0x559891e67390 2024-04-16T17:15:01.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- 172.21.15.18:0/3627470824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f80035f0 con 0x559891e67390 2024-04-16T17:15:01.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f00eb400640 1 -- 172.21.15.18:0/3627470824 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f010401f5f0 con 0x559891e67390 2024-04-16T17:15:01.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 10 client.18102 fetch_fsmap learned FSMap version 29 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f01137e9740 1 -- 172.21.15.18:0/3627470824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559891bfdf70 con 0x559891e67390 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.318+0000 7f00eb400640 1 -- 172.21.15.18:0/3627470824 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f80035f0 con 0x559891e67390 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f00eb400640 1 -- 172.21.15.18:0/3627470824 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f0104027750 con 0x559891e67390 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f01137e9740 10 client.18102 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f01137e9740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f01137e9740 20 client.18102 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f01137e9740 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559891e77910 0x559891e79d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f01137e9740 4 client.18102 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f01137e9740 1 -- 172.21.15.18:0/3627470824 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559891dd06d0 con 0x559891e77910 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f0109e00640 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559891e77910 0x559891e79d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f0109e00640 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559891e77910 0x559891e79d90 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70488dc395aec9 server_cookie=f2cfbc20fd550fbd in_seq=0 out_seq=0 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f00eb400640 10 client.18102 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f00eb400640 1 -- 172.21.15.18:0/3627470824 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559891dd06d0 con 0x559891e77910 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.324+0000 7f00eb400640 10 client.18102 handle_command_reply: tid=0 2024-04-16T17:15:01.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 client.18102 shutdown 2024-04-16T17:15:01.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 -- 172.21.15.18:0/3627470824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e67390 msgr2=0x559891e64e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e67390 0x559891e64e90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0104015a50 tx=0x7f0104004ad0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 -- 172.21.15.18:0/3627470824 shutdown_connections 2024-04-16T17:15:01.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559891e77910 0x559891e79d90 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e67390 0x559891e64e90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e64950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 --2- 172.21.15.18:0/3627470824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e5de90 0x559891e64410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f00eb400640 0 client.18102 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:01.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 -- 172.21.15.18:0/3627470824 >> 172.21.15.18:0/3627470824 conn(0x559891e585c0 msgr2=0x559891e5a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 -- 172.21.15.18:0/3627470824 shutdown_connections 2024-04-16T17:15:01.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 1 -- 172.21.15.18:0/3627470824 wait complete. 2024-04-16T17:15:01.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.326+0000 7f01137e9740 20 client.18102 trim_cache size 0 max 16384 2024-04-16T17:15:01.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f01137e9740 1 Processor -- start 2024-04-16T17:15:01.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f01137e9740 1 -- start start 2024-04-16T17:15:01.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f01137e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f01137e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f01137e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e5a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f01137e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec012830 con 0x559891e5de90 2024-04-16T17:15:01.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f01137e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec010100 con 0x559891e67390 2024-04-16T17:15:01.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f01137e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec00df50 con 0x559891e5e7b0 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f0109400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f0109400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39000/0 (socket says 172.21.15.18:39000) 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f0109400640 1 -- 172.21.15.18:0/3648964236 learned_addr learned my addr 172.21.15.18:0/3648964236 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.362+0000 7f0109e00640 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.363+0000 7f010a800640 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e5a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.363+0000 7f0109400640 1 -- 172.21.15.18:0/3648964236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 msgr2=0x559891e5a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.363+0000 7f0109400640 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e5a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.363+0000 7f0109400640 1 -- 172.21.15.18:0/3648964236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 msgr2=0x559891e66b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.363+0000 7f0109400640 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e66b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.363+0000 7f0109400640 1 -- 172.21.15.18:0/3648964236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559891e5e5e0 con 0x559891e5e7b0 2024-04-16T17:15:01.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.364+0000 7f0109400640 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e730d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f00f801c040 tx=0x7f00f8009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7709a10c563438d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.365+0000 7f00f6600640 1 -- 172.21.15.18:0/3648964236 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f801e040 con 0x559891e5e7b0 2024-04-16T17:15:01.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.365+0000 7f00f6600640 1 -- 172.21.15.18:0/3648964236 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f8029070 con 0x559891e5e7b0 2024-04-16T17:15:01.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.365+0000 7f00f6600640 1 -- 172.21.15.18:0/3648964236 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8004a20 con 0x559891e5e7b0 2024-04-16T17:15:01.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.365+0000 7f01137e9740 1 -- 172.21.15.18:0/3648964236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 msgr2=0x559891e730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.365+0000 7f01137e9740 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e730d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f00f801c040 tx=0x7f00f8009590 comp rx=0 tx=0).stop 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 -- 172.21.15.18:0/3648964236 shutdown_connections 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e5a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e730d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 --2- 172.21.15.18:0/3648964236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e66b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 -- 172.21.15.18:0/3648964236 >> 172.21.15.18:0/3648964236 conn(0x559891e585c0 msgr2=0x559891e599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 -- 172.21.15.18:0/3648964236 shutdown_connections 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 -- 172.21.15.18:0/3648964236 wait complete. 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 -- start start 2024-04-16T17:15:01.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.366+0000 7f01137e9740 1 Processor -- start 2024-04-16T17:15:01.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f01137e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f01137e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e61830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f01137e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e61d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f01137e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec00e5c0 con 0x559891e5de90 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f01137e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec00fa30 con 0x559891e67390 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f01137e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec00ec90 con 0x559891e5e7b0 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f01137e9740 10 client.0 _ll_register_callbacks cb 0x7f01040156d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f0109400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e61830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f0109e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f0109400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e61830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39004/0 (socket says 172.21.15.18:39004) 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f0109400640 1 -- 172.21.15.18:0/359464665 learned_addr learned my addr 172.21.15.18:0/359464665 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.367+0000 7f010a800640 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e61d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f0109e00640 1 -- 172.21.15.18:0/359464665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 msgr2=0x559891e61830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f0109e00640 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e61830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f0109e00640 1 -- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 msgr2=0x559891e61d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f0109e00640 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e61d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f0109e00640 1 -- 172.21.15.18:0/359464665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559891a95170 con 0x559891e5de90 2024-04-16T17:15:01.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f010a800640 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e61d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:01.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f0109e00640 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e612f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0ef860 tx=0x7f00fc0f27a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9798f6f61899ac27 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f00eb400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:01.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.368+0000 7f00eb400640 1 -- 172.21.15.18:0/359464665 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0d6070 con 0x559891e5de90 2024-04-16T17:15:01.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f00eb400640 1 -- 172.21.15.18:0/359464665 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7f00fc0f2120 con 0x559891e5de90 2024-04-16T17:15:01.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f01137e9740 1 -- 172.21.15.18:0/359464665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f8029070 con 0x559891e5de90 2024-04-16T17:15:01.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f00eb400640 1 -- 172.21.15.18:0/359464665 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00fc0be7b0 con 0x559891e5de90 2024-04-16T17:15:01.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f01137e9740 10 client.18108 fetch_fsmap learned FSMap version 29 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f01137e9740 1 -- 172.21.15.18:0/359464665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559891ede2f0 con 0x559891e5de90 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f00eb400640 1 -- 172.21.15.18:0/359464665 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f8029070 con 0x559891e5de90 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f00eb400640 1 -- 172.21.15.18:0/359464665 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f00fc0ef9d0 con 0x559891e5de90 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f01137e9740 10 client.18108 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f01137e9740 10 client.18108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.369+0000 7f01137e9740 20 client.18108 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.370+0000 7f01137e9740 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559891edea40 0x559891ee0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.370+0000 7f01137e9740 4 client.18108 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.370+0000 7f01137e9740 1 -- 172.21.15.18:0/359464665 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559891dca540 con 0x559891edea40 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.370+0000 7f0109400640 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559891edea40 0x559891ee0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.373+0000 7f0109400640 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559891edea40 0x559891ee0e60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbc24f81f50a925 server_cookie=bb83ca3aa497f6f in_seq=0 out_seq=0 2024-04-16T17:15:01.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.373+0000 7f00eb400640 10 client.18108 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:01.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.375+0000 7f00eb400640 1 -- 172.21.15.18:0/359464665 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559891dca540 con 0x559891edea40 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.375+0000 7f00eb400640 10 client.18108 handle_command_reply: tid=0 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.375+0000 7f01137e9740 1 client.18108 shutdown 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 -- 172.21.15.18:0/359464665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 msgr2=0x559891e612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e612f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0ef860 tx=0x7f00fc0f27a0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 -- 172.21.15.18:0/359464665 shutdown_connections 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559891edea40 0x559891ee0e60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559891e67390 0x559891e61d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f00eb400640 0 client.18108 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559891e5e7b0 0x559891e61830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 --2- 172.21.15.18:0/359464665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559891e5de90 0x559891e612f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 -- 172.21.15.18:0/359464665 >> 172.21.15.18:0/359464665 conn(0x559891e585c0 msgr2=0x559891e57560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 -- 172.21.15.18:0/359464665 shutdown_connections 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 1 -- 172.21.15.18:0/359464665 wait complete. 2024-04-16T17:15:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.376+0000 7f01137e9740 20 client.18108 trim_cache size 0 max 16384 2024-04-16T17:15:01.377 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:15:01.377 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:15:01.377 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "dce8f4db-3bcc-4008-8f0e-63dd38b611fc", 2024-04-16T17:15:01.377 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:15:01.377 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:15:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.377+0000 7f010b200640 1 -- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00f00a3da0 msgr2=0x7f00f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.377+0000 7f010b200640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00f00a3da0 0x7f00f00a6260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0104009b30 tx=0x7f010400b040 comp rx=0 tx=0).stop 2024-04-16T17:15:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.377+0000 7f010b200640 1 -- 172.21.15.18:0/3781810951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c102070 msgr2=0x7f010c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.377+0000 7f010b200640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c102070 0x7f010c1a06b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f00fc02c190 tx=0x7f00fc004360 comp rx=0 tx=0).stop 2024-04-16T17:15:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.378+0000 7f010b200640 1 -- 172.21.15.18:0/3781810951 shutdown_connections 2024-04-16T17:15:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.378+0000 7f010b200640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00f00a3da0 0x7f00f00a6260 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.378+0000 7f010b200640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010c19a860 0x7f010c19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.378+0000 7f010b200640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c107140 0x7f010c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.378+0000 7f010b200640 1 --2- 172.21.15.18:0/3781810951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c102070 0x7f010c1a06b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.378+0000 7f010b200640 1 -- 172.21.15.18:0/3781810951 >> 172.21.15.18:0/3781810951 conn(0x7f010c0fd950 msgr2=0x7f010c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.379+0000 7f010b200640 1 -- 172.21.15.18:0/3781810951 shutdown_connections 2024-04-16T17:15:01.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.379+0000 7f010b200640 1 -- 172.21.15.18:0/3781810951 wait complete. 2024-04-16T17:15:01.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:15:01.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:15:01.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.596+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1174577734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a0102a20 msgr2=0x7fd4a0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.596+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1174577734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a0102a20 0x7fd4a0106a60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009900 tx=0x7fd48c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:01.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.597+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1174577734 shutdown_connections 2024-04-16T17:15:01.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.597+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1174577734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4a0107140 0x7fd4a010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.597+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1174577734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a0102a20 0x7fd4a0106a60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.597+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1174577734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a0102070 0x7fd4a0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.597+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1174577734 >> 172.21.15.18:0/1174577734 conn(0x7fd4a00fd950 msgr2=0x7fd4a00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.597+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1174577734 shutdown_connections 2024-04-16T17:15:01.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.597+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1174577734 wait complete. 2024-04-16T17:15:01.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6c00640 1 Processor -- start 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6c00640 1 -- start start 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a0102070 0x7fd4a01a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4a0107140 0x7fd4a01a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a019a7f0 0x7fd4a019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01121b0 con 0x7fd4a0107140 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a0112030 con 0x7fd4a019a7f0 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a0112330 con 0x7fd4a0102070 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.598+0000 7fd4a6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a019a7f0 0x7fd4a019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a0102070 0x7fd4a01a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a019a7f0 0x7fd4a019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55198/0 (socket says 172.21.15.18:55198) 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a6200640 1 -- 172.21.15.18:0/1210552702 learned_addr learned my addr 172.21.15.18:0/1210552702 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4a0107140 0x7fd4a01a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a6200640 1 -- 172.21.15.18:0/1210552702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a0102070 msgr2=0x7fd4a01a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a6200640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a0102070 0x7fd4a01a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a6200640 1 -- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4a0107140 msgr2=0x7fd4a01a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a6200640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4a0107140 0x7fd4a01a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a6200640 1 -- 172.21.15.18:0/1210552702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a019b380 con 0x7fd4a019a7f0 2024-04-16T17:15:01.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.599+0000 7fd4a6200640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a019a7f0 0x7fd4a019aca0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd49800eb10 tx=0x7fd49800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.600+0000 7fd49e000640 1 -- 172.21.15.18:0/1210552702 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498004b20 con 0x7fd4a019a7f0 2024-04-16T17:15:01.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.600+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1210552702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a019b670 con 0x7fd4a019a7f0 2024-04-16T17:15:01.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.600+0000 7fd49e000640 1 -- 172.21.15.18:0/1210552702 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd498004cc0 con 0x7fd4a019a7f0 2024-04-16T17:15:01.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.600+0000 7fd49e000640 1 -- 172.21.15.18:0/1210552702 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498010760 con 0x7fd4a019a7f0 2024-04-16T17:15:01.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.600+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1210552702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a01a7820 con 0x7fd4a019a7f0 2024-04-16T17:15:01.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.603+0000 7fd4aae3f740 1 Processor -- start 2024-04-16T17:15:01.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.604+0000 7fd49e000640 1 -- 172.21.15.18:0/1210552702 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd498005020 con 0x7fd4a019a7f0 2024-04-16T17:15:01.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.604+0000 7fd4aae3f740 1 -- start start 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b68b92f120 con 0x55b68b9c9cb0 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b68b8d5340 con 0x55b68b9c9390 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b68b92b0e0 con 0x55b68b9d2890 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a5800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a5800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49108/0 (socket says 172.21.15.18:49108) 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a5800640 1 -- 172.21.15.18:0/1208563187 learned_addr learned my addr 172.21.15.18:0/1208563187 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a5800640 1 -- 172.21.15.18:0/1208563187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 msgr2=0x55b68b9d4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a5800640 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a5800640 1 -- 172.21.15.18:0/1208563187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 msgr2=0x55b68b9c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a5800640 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a5800640 1 -- 172.21.15.18:0/1208563187 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68b70b7d0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.605+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:01.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.606+0000 7fd49e000640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4880a3d20 0x7fd4880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.606+0000 7fd4a5800640 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9d2350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd494009cf0 tx=0x7fd494011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353ecf5b08443b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.606+0000 7fd49e000640 1 -- 172.21.15.18:0/1210552702 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7fd4980d10b0 con 0x7fd4a019a7f0 2024-04-16T17:15:01.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.606+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4880a3d20 0x7fd4880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.606+0000 7fd492a00640 1 -- 172.21.15.18:0/1208563187 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49401e070 con 0x55b68b9c9cb0 2024-04-16T17:15:01.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.606+0000 7fd492a00640 1 -- 172.21.15.18:0/1208563187 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd494019b10 con 0x55b68b9c9cb0 2024-04-16T17:15:01.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.606+0000 7fd492a00640 1 -- 172.21.15.18:0/1208563187 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49400ea60 con 0x55b68b9c9cb0 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1208563187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 msgr2=0x55b68b9d2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9d2350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd494009cf0 tx=0x7fd494011810 comp rx=0 tx=0).stop 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4880a3d20 0x7fd4880a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009230 tx=0x7fd48c005dd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1208563187 shutdown_connections 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9d2350 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/1208563187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1208563187 >> 172.21.15.18:0/1208563187 conn(0x55b68b9c3ac0 msgr2=0x55b68b9c3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.607+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1208563187 shutdown_connections 2024-04-16T17:15:01.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.608+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1208563187 wait complete. 2024-04-16T17:15:01.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.608+0000 7fd4aae3f740 1 -- start start 2024-04-16T17:15:01.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.608+0000 7fd4aae3f740 1 Processor -- start 2024-04-16T17:15:01.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9390 0x55b68b9cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9cb0 0x55b68b9cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b68b9d56d0 con 0x55b68b9c9390 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b68b8e8710 con 0x55b68b9c9cb0 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b68b9d5850 con 0x55b68b9d2890 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4aae3f740 10 client.0 _ll_register_callbacks cb 0x55b68b8e3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a5800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9390 0x55b68b9cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a5800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9390 0x55b68b9cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49124/0 (socket says 172.21.15.18:49124) 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a5800640 1 -- 172.21.15.18:0/2973251904 learned_addr learned my addr 172.21.15.18:0/2973251904 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9cb0 0x55b68b9cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a6200640 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a5800640 1 -- 172.21.15.18:0/2973251904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 msgr2=0x55b68b9d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a5800640 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a5800640 1 -- 172.21.15.18:0/2973251904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9cb0 msgr2=0x55b68b9cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a5800640 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9cb0 0x55b68b9cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.609+0000 7fd4a5800640 1 -- 172.21.15.18:0/2973251904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68b6413f0 con 0x55b68b9c9390 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.610+0000 7fd4a6200640 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.610+0000 7fd4a5800640 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9390 0x55b68b9cf8c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd49401b000 tx=0x7fd494019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd66db52620a9b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.610+0000 7fd487400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.610+0000 7fd487400640 1 -- 172.21.15.18:0/2973251904 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49401e050 con 0x55b68b9c9390 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.610+0000 7fd487400640 1 -- 172.21.15.18:0/2973251904 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7fd494005240 con 0x55b68b9c9390 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.610+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2973251904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd494019b10 con 0x55b68b9c9390 2024-04-16T17:15:01.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.610+0000 7fd487400640 1 -- 172.21.15.18:0/2973251904 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd494033050 con 0x55b68b9c9390 2024-04-16T17:15:01.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.610+0000 7fd4aae3f740 10 client.18120 fetch_fsmap learned FSMap version 29 2024-04-16T17:15:01.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2973251904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b68b6010f0 con 0x55b68b9c9390 2024-04-16T17:15:01.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd487400640 1 -- 172.21.15.18:0/2973251904 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd494019b10 con 0x55b68b9c9390 2024-04-16T17:15:01.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd487400640 1 -- 172.21.15.18:0/2973251904 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd4940046c0 con 0x55b68b9c9390 2024-04-16T17:15:01.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd4aae3f740 10 client.18120 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:15:01.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd4aae3f740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:01.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd4aae3f740 20 client.18120 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:01.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b68b9e2c00 0x55b68b9e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd4aae3f740 4 client.18120 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2973251904 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b68b93cb40 con 0x55b68b9e2c00 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.611+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b68b9e2c00 0x55b68b9e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.612+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b68b9e2c00 0x55b68b9e4ff0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8d8f22652d4b1 server_cookie=988a74cef1ce18d9 in_seq=0 out_seq=0 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.612+0000 7fd487400640 10 client.18120 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd487400640 1 -- 172.21.15.18:0/2973251904 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b68b93cb40 con 0x55b68b9e2c00 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd487400640 10 client.18120 handle_command_reply: tid=0 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 client.18120 shutdown 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2973251904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9390 msgr2=0x55b68b9cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9390 0x55b68b9cf8c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd49401b000 tx=0x7fd494019660 comp rx=0 tx=0).stop 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2973251904 shutdown_connections 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b68b9e2c00 0x55b68b9e4ff0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9cb0 0x55b68b9cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2973251904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9390 0x55b68b9cf8c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2973251904 >> 172.21.15.18:0/2973251904 conn(0x55b68b9c3ac0 msgr2=0x55b68b9c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd487400640 0 client.18120 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:01.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2973251904 shutdown_connections 2024-04-16T17:15:01.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2973251904 wait complete. 2024-04-16T17:15:01.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.613+0000 7fd4aae3f740 20 client.18120 trim_cache size 0 max 16384 2024-04-16T17:15:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4aae3f740 1 Processor -- start 2024-04-16T17:15:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4aae3f740 1 -- start start 2024-04-16T17:15:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9c9cb0 0x55b68b9de590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9d2890 0x55b68b9cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01b3f10 con 0x55b68b9d2890 2024-04-16T17:15:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01b17e0 con 0x55b68b9c9390 2024-04-16T17:15:01.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01af630 con 0x55b68b9c9cb0 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4a6200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9d2890 0x55b68b9cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4a5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4a4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9c9cb0 0x55b68b9de590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4a4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9c9cb0 0x55b68b9de590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39068/0 (socket says 172.21.15.18:39068) 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4a4e00640 1 -- 172.21.15.18:0/1239650953 learned_addr learned my addr 172.21.15.18:0/1239650953 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.633+0000 7fd4a6200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9d2890 0x55b68b9cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49140/0 (socket says 172.21.15.18:49140) 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd4a4e00640 1 -- 172.21.15.18:0/1239650953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 msgr2=0x55b68b9d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1239650953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd4a4e00640 1 -- 172.21.15.18:0/1239650953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9d2890 msgr2=0x55b68b9cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1239650953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9d2890 0x55b68b9cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd4a4e00640 1 -- 172.21.15.18:0/1239650953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68b9e52f0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/1239650953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9c9cb0 0x55b68b9de590 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009900 tx=0x7fd48c041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbda4c9c22515aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd492000640 1 -- 172.21.15.18:0/1239650953 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c054b50 con 0x55b68b9c9cb0 2024-04-16T17:15:01.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd492000640 1 -- 172.21.15.18:0/1239650953 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd48c054cf0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.634+0000 7fd492000640 1 -- 172.21.15.18:0/1239650953 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c004920 con 0x55b68b9c9cb0 2024-04-16T17:15:01.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1239650953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9c9cb0 msgr2=0x55b68b9de590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/1239650953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9c9cb0 0x55b68b9de590 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009900 tx=0x7fd48c041040 comp rx=0 tx=0).stop 2024-04-16T17:15:01.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1239650953 shutdown_connections 2024-04-16T17:15:01.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/1239650953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9d2890 0x55b68b9cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/1239650953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9c9cb0 0x55b68b9de590 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/1239650953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1239650953 >> 172.21.15.18:0/1239650953 conn(0x55b68b9c3ac0 msgr2=0x55b68b9ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1239650953 shutdown_connections 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.635+0000 7fd4aae3f740 1 -- 172.21.15.18:0/1239650953 wait complete. 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.636+0000 7fd4aae3f740 1 -- start start 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.636+0000 7fd4aae3f740 1 Processor -- start 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.636+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9da860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9dada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4aae3f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01afca0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01b1110 con 0x55b68b9c9390 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4aae3f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01b0370 con 0x55b68b9d2890 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4aae3f740 10 client.0 _ll_register_callbacks cb 0x7fd49401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a4e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a4e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9dada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49146/0 (socket says 172.21.15.18:49146) 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a4e00640 1 -- 172.21.15.18:0/2662309248 learned_addr learned my addr 172.21.15.18:0/2662309248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a5800640 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9da860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a6200640 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a4e00640 1 -- 172.21.15.18:0/2662309248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 msgr2=0x55b68b9d5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a4e00640 1 -- 172.21.15.18:0/2662309248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 msgr2=0x55b68b9da860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9da860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a4e00640 1 -- 172.21.15.18:0/2662309248 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68b9c9ae0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.637+0000 7fd4a6200640 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.638+0000 7fd4a4e00640 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9dada0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd48c057040 tx=0x7fd48c0547a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc35a32dda058a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.638+0000 7fd487400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.638+0000 7fd487400640 1 -- 172.21.15.18:0/2662309248 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c062060 con 0x55b68b9c9cb0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.638+0000 7fd487400640 1 -- 172.21.15.18:0/2662309248 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9466+0+0 (secure 0 0 0) 0x7fd48c0549c0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.638+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2662309248 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd48c054cf0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.638+0000 7fd487400640 1 -- 172.21.15.18:0/2662309248 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd48c004040 con 0x55b68b9c9cb0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.638+0000 7fd4aae3f740 10 client.18126 fetch_fsmap learned FSMap version 29 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.638+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2662309248 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b68ba48910 con 0x55b68b9c9cb0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd487400640 1 -- 172.21.15.18:0/2662309248 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd48c054cf0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd487400640 1 -- 172.21.15.18:0/2662309248 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd48c0050a0 con 0x55b68b9c9cb0 2024-04-16T17:15:01.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd4aae3f740 10 client.18126 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:15:01.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd4aae3f740 10 client.18126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:01.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd4aae3f740 20 client.18126 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:01.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b68ba49060 0x55b68ba4b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:01.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd4aae3f740 4 client.18126 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:15:01.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2662309248 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b68b9369b0 con 0x55b68ba49060 2024-04-16T17:15:01.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.639+0000 7fd4a5800640 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b68ba49060 0x55b68ba4b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:01.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd487400640 10 client.18126 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:01.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4a5800640 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b68ba49060 0x55b68ba4b480 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897c76ac6eb6ebef server_cookie=b996aff7419d4f9a in_seq=0 out_seq=0 2024-04-16T17:15:01.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd487400640 1 -- 172.21.15.18:0/2662309248 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b68b9369b0 con 0x55b68ba49060 2024-04-16T17:15:01.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd487400640 10 client.18126 handle_command_reply: tid=0 2024-04-16T17:15:01.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 client.18126 shutdown 2024-04-16T17:15:01.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2662309248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 msgr2=0x55b68b9dada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9dada0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd48c057040 tx=0x7fd48c0547a0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2662309248 shutdown_connections 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b68ba49060 0x55b68ba4b480 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd487400640 0 client.18126 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b68b9d2890 0x55b68b9d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b68b9c9cb0 0x55b68b9dada0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 --2- 172.21.15.18:0/2662309248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b68b9c9390 0x55b68b9da860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2662309248 >> 172.21.15.18:0/2662309248 conn(0x55b68b9c3ac0 msgr2=0x55b68b9cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.640+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2662309248 shutdown_connections 2024-04-16T17:15:01.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4aae3f740 1 -- 172.21.15.18:0/2662309248 wait complete. 2024-04-16T17:15:01.643 INFO:tasks.workunit.client.0.smithi018.stdout: 1 388111 3.67 MB/sec execute 434 sec latency 107.639 ms 2024-04-16T17:15:01.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4aae3f740 20 client.18126 trim_cache size 0 max 16384 2024-04-16T17:15:01.644 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:15:01.644 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:15:01.644 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:15:01.644 INFO:teuthology.orchestra.run.smithi018.stdout: "dce8f4db-3bcc-4008-8f0e-63dd38b611fc": { 2024-04-16T17:15:01.644 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:15:01.644 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "dce8f4db-3bcc-4008-8f0e-63dd38b611fc", 2024-04-16T17:15:01.645 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:15:01.645 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:15:01.645 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:15:01.645 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:15:01.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4880a3d20 msgr2=0x7fd4880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4880a3d20 0x7fd4880a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009230 tx=0x7fd48c005dd0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1210552702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a019a7f0 msgr2=0x7fd4a019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a019a7f0 0x7fd4a019aca0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd49800eb10 tx=0x7fd49800c2d0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1210552702 shutdown_connections 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd4880a3d20 0x7fd4880a61e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4a019a7f0 0x7fd4a019aca0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4a0107140 0x7fd4a01a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 --2- 172.21.15.18:0/1210552702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a0102070 0x7fd4a01a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.641+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1210552702 >> 172.21.15.18:0/1210552702 conn(0x7fd4a00fd950 msgr2=0x7fd4a00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.642+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1210552702 shutdown_connections 2024-04-16T17:15:01.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:01.642+0000 7fd4a6c00640 1 -- 172.21.15.18:0/1210552702 wait complete. 2024-04-16T17:15:01.660 INFO:tasks.cephfs.filesystem:scrub status for tag:dce8f4db-3bcc-4008-8f0e-63dd38b611fc - {'path': '/', 'tag': 'dce8f4db-3bcc-4008-8f0e-63dd38b611fc', 'options': 'recursive,force'} 2024-04-16T17:15:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:01 smithi043 ceph-mon[36901]: pgmap v322: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-04-16T17:15:02.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:01 smithi022 ceph-mon[36023]: pgmap v322: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-04-16T17:15:02.640 INFO:tasks.workunit.client.0.smithi018.stdout: 1 388382 3.66 MB/sec execute 435 sec latency 156.833 ms 2024-04-16T17:15:03.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:02 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:15:03.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:02 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:03.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:02 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:15:03.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:02 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:03.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:02 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:15:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:02 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:02 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:15:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:02 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:03.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:57 smithi018 ceph-mon[31776]: pgmap v320: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-16T17:15:03.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:59 smithi018 ceph-mon[31776]: pgmap v321: 129 pgs: 129 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-16T17:15:03.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:14:59 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2688626017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:03.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:01 smithi018 ceph-mon[31776]: pgmap v322: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-04-16T17:15:03.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:02 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:15:03.575 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:02 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:03.575 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:02 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:15:03.575 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:02 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:03.640 INFO:tasks.workunit.client.0.smithi018.stdout: 1 388950 3.65 MB/sec execute 436 sec latency 63.208 ms 2024-04-16T17:15:03.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:03 smithi018 ceph-mon[31776]: pgmap v323: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:15:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:03 smithi022 ceph-mon[36023]: pgmap v323: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:15:04.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:03 smithi043 ceph-mon[36901]: pgmap v323: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-16T17:15:04.640 INFO:tasks.workunit.client.0.smithi018.stdout: 1 389279 3.65 MB/sec execute 437 sec latency 80.016 ms 2024-04-16T17:15:05.074 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:04 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:15:05.210 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:04 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:15:05.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:04 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:15:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.590+0000 7f81ab800640 1 -- 172.21.15.18:0/447982133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4104470 msgr2=0x7f81a4104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.590+0000 7f81ab800640 1 --2- 172.21.15.18:0/447982133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4104470 0x7f81a4104850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8198009860 tx=0x7f819802bb90 comp rx=0 tx=0).stop 2024-04-16T17:15:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.590+0000 7f81ab800640 1 -- 172.21.15.18:0/447982133 shutdown_connections 2024-04-16T17:15:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.590+0000 7f81ab800640 1 --2- 172.21.15.18:0/447982133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a4109540 0x7f81a4111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.590+0000 7f81ab800640 1 --2- 172.21.15.18:0/447982133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81a4104e20 0x7f81a4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.590+0000 7f81ab800640 1 --2- 172.21.15.18:0/447982133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4104470 0x7f81a4104850 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.590+0000 7f81ab800640 1 -- 172.21.15.18:0/447982133 >> 172.21.15.18:0/447982133 conn(0x7f81a40ffcf0 msgr2=0x7f81a4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.591+0000 7f81ab800640 1 -- 172.21.15.18:0/447982133 shutdown_connections 2024-04-16T17:15:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.591+0000 7f81ab800640 1 -- 172.21.15.18:0/447982133 wait complete. 2024-04-16T17:15:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.594+0000 7f81ab800640 1 Processor -- start 2024-04-16T17:15:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.594+0000 7f81ab800640 1 -- start start 2024-04-16T17:15:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.595+0000 7f81ab800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81a4104470 0x7f81a41128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.595+0000 7f81ab800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a4109540 0x7f81a4112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.595+0000 7f81ab800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4113350 0x7f81a4119930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.595+0000 7f81ab800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f81a40781c0 con 0x7f81a4113350 2024-04-16T17:15:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.595+0000 7f81ab800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4078040 con 0x7f81a4104470 2024-04-16T17:15:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.595+0000 7f81ab800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4078340 con 0x7f81a4109540 2024-04-16T17:15:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81a4104470 0x7f81a41128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81a4104470 0x7f81a41128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55256/0 (socket says 172.21.15.18:55256) 2024-04-16T17:15:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a8e00640 1 -- 172.21.15.18:0/4186905160 learned_addr learned my addr 172.21.15.18:0/4186905160 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a3e00640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a4109540 0x7f81a4112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a9800640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4113350 0x7f81a4119930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a8e00640 1 -- 172.21.15.18:0/4186905160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a4109540 msgr2=0x7f81a4112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a8e00640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a4109540 0x7f81a4112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a8e00640 1 -- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4113350 msgr2=0x7f81a4119930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a8e00640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4113350 0x7f81a4119930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.596+0000 7f81a8e00640 1 -- 172.21.15.18:0/4186905160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a411a100 con 0x7f81a4104470 2024-04-16T17:15:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.597+0000 7f81a8e00640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81a4104470 0x7f81a41128d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8198037040 tx=0x7f8198032460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.598+0000 7f81a1600640 1 -- 172.21.15.18:0/4186905160 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198041020 con 0x7f81a4104470 2024-04-16T17:15:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.598+0000 7f81a1600640 1 -- 172.21.15.18:0/4186905160 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819802ca10 con 0x7f81a4104470 2024-04-16T17:15:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.598+0000 7f81a1600640 1 -- 172.21.15.18:0/4186905160 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198039040 con 0x7f81a4104470 2024-04-16T17:15:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.598+0000 7f81ab800640 1 -- 172.21.15.18:0/4186905160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a411a390 con 0x7f81a4104470 2024-04-16T17:15:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.598+0000 7f81ab800640 1 -- 172.21.15.18:0/4186905160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a411a830 con 0x7f81a4104470 2024-04-16T17:15:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.604+0000 7f81a1600640 1 -- 172.21.15.18:0/4186905160 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8198044020 con 0x7f81a4104470 2024-04-16T17:15:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.604+0000 7f81ae56b740 1 Processor -- start 2024-04-16T17:15:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.605+0000 7f81ae56b740 1 -- start start 2024-04-16T17:15:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.605+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4eb610 0x5568cb4ea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.605+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4ea8a0 0x5568cb4ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.605+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.605+0000 7f81ae56b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5568cb44f780 con 0x5568cb4eb610 2024-04-16T17:15:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.605+0000 7f81ae56b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5568cb3f5f40 con 0x5568cb4f3e20 2024-04-16T17:15:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.605+0000 7f81ae56b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5568cb44b740 con 0x5568cb4ea8a0 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a9800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4eb610 0x5568cb4ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4ea8a0 0x5568cb4ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a9800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4eb610 0x5568cb4ea360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49168/0 (socket says 172.21.15.18:49168) 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a9800640 1 -- 172.21.15.18:0/138742562 learned_addr learned my addr 172.21.15.18:0/138742562 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a3e00640 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a9800640 1 -- 172.21.15.18:0/138742562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4ea8a0 msgr2=0x5568cb4ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a9800640 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4ea8a0 0x5568cb4ead20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a9800640 1 -- 172.21.15.18:0/138742562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 msgr2=0x5568cb4f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a9800640 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a8e00640 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4ea8a0 0x5568cb4ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.606+0000 7f81a9800640 1 -- 172.21.15.18:0/138742562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568cb162210 con 0x5568cb4eb610 2024-04-16T17:15:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.607+0000 7f81a1600640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f818c0a3e60 0x7f818c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.607+0000 7f81a1600640 1 -- 172.21.15.18:0/4186905160 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9642+0+0 (secure 0 0 0) 0x7f81980f2550 con 0x7f81a4104470 2024-04-16T17:15:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.607+0000 7f81a9800640 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4eb610 0x5568cb4ea360 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f819c007d90 tx=0x7f819c00c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73796384ff059223 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.607+0000 7f81a8e00640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f818c0a3e60 0x7f818c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.608+0000 7f8196000640 1 -- 172.21.15.18:0/138742562 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c015020 con 0x5568cb4eb610 2024-04-16T17:15:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.608+0000 7f8196000640 1 -- 172.21.15.18:0/138742562 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c011b10 con 0x5568cb4eb610 2024-04-16T17:15:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.608+0000 7f8196000640 1 -- 172.21.15.18:0/138742562 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c010ab0 con 0x5568cb4eb610 2024-04-16T17:15:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.609+0000 7f81ae56b740 1 -- 172.21.15.18:0/138742562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4eb610 msgr2=0x5568cb4ea360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.609+0000 7f81ae56b740 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4eb610 0x5568cb4ea360 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f819c007d90 tx=0x7f819c00c560 comp rx=0 tx=0).stop 2024-04-16T17:15:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.609+0000 7f81ae56b740 1 -- 172.21.15.18:0/138742562 shutdown_connections 2024-04-16T17:15:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.609+0000 7f81ae56b740 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.609+0000 7f81ae56b740 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4ea8a0 0x5568cb4ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.609+0000 7f81ae56b740 1 --2- 172.21.15.18:0/138742562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4eb610 0x5568cb4ea360 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.609+0000 7f81ae56b740 1 -- 172.21.15.18:0/138742562 >> 172.21.15.18:0/138742562 conn(0x5568cb4855d0 msgr2=0x5568cb4e5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.610+0000 7f81a8e00640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f818c0a3e60 0x7f818c0a6320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f81980efa00 tx=0x7f81980f8210 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.611+0000 7f81ae56b740 1 -- 172.21.15.18:0/138742562 shutdown_connections 2024-04-16T17:15:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.611+0000 7f81ae56b740 1 -- 172.21.15.18:0/138742562 wait complete. 2024-04-16T17:15:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.611+0000 7f81ae56b740 1 -- start start 2024-04-16T17:15:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.612+0000 7f81ae56b740 1 Processor -- start 2024-04-16T17:15:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.612+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4f3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81ae56b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5568cb4f6b50 con 0x5568cb4ea8a0 2024-04-16T17:15:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81ae56b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5568cb409390 con 0x5568cb4f3e20 2024-04-16T17:15:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81ae56b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5568cb4f6cd0 con 0x5568cb4eb610 2024-04-16T17:15:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81ae56b740 10 client.0 _ll_register_callbacks cb 0x5568cb3f2f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4f0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39118/0 (socket says 172.21.15.18:39118) 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.613+0000 7f81a3e00640 1 -- 172.21.15.18:0/3139033705 learned_addr learned my addr 172.21.15.18:0/3139033705 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.614+0000 7f81a9800640 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.614+0000 7f81a9800640 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.614+0000 7f81a3e00640 1 -- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 msgr2=0x5568cb4f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.614+0000 7f81a3e00640 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.614+0000 7f81a3e00640 1 -- 172.21.15.18:0/3139033705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 msgr2=0x5568cb4f3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.614+0000 7f81a3e00640 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4f3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.614+0000 7f81a3e00640 1 -- 172.21.15.18:0/3139033705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568cb22e620 con 0x5568cb4eb610 2024-04-16T17:15:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.614+0000 7f81a9800640 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.615+0000 7f81a3e00640 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4f0bf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8190009dc0 tx=0x7f81900117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd01792ee37ce3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.615+0000 7f818aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:15:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.616+0000 7f818aa00640 1 -- 172.21.15.18:0/3139033705 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190005220 con 0x5568cb4eb610 2024-04-16T17:15:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.616+0000 7f818aa00640 1 -- 172.21.15.18:0/3139033705 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9642+0+0 (secure 0 0 0) 0x7f81900053c0 con 0x5568cb4eb610 2024-04-16T17:15:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.616+0000 7f81ae56b740 1 -- 172.21.15.18:0/3139033705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f819c011b10 con 0x5568cb4eb610 2024-04-16T17:15:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.616+0000 7f818aa00640 1 -- 172.21.15.18:0/3139033705 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f81900055c0 con 0x5568cb4eb610 2024-04-16T17:15:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.616+0000 7f81ae56b740 10 client.28294 fetch_fsmap learned FSMap version 29 2024-04-16T17:15:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.616+0000 7f81ae56b740 1 -- 172.21.15.18:0/3139033705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568cb502c20 con 0x5568cb4eb610 2024-04-16T17:15:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.617+0000 7f818aa00640 1 -- 172.21.15.18:0/3139033705 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c011b10 con 0x5568cb4eb610 2024-04-16T17:15:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.617+0000 7f818aa00640 1 -- 172.21.15.18:0/3139033705 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f819000f040 con 0x5568cb4eb610 2024-04-16T17:15:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.618+0000 7f81ae56b740 10 client.28294 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:15:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.618+0000 7f81ae56b740 10 client.28294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.618+0000 7f81ae56b740 20 client.28294 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.618+0000 7f81ae56b740 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568cb5040e0 0x5568cb506530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.618+0000 7f81ae56b740 4 client.28294 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.618+0000 7f81ae56b740 1 -- 172.21.15.18:0/3139033705 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568cb45d1a0 con 0x5568cb5040e0 2024-04-16T17:15:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.618+0000 7f81a9800640 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568cb5040e0 0x5568cb506530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.620+0000 7f81a9800640 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568cb5040e0 0x5568cb506530 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde1a5f55b2a7005 server_cookie=37fbfa389fe1fa8e in_seq=0 out_seq=0 2024-04-16T17:15:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.620+0000 7f818aa00640 10 client.28294 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.621+0000 7f818aa00640 1 -- 172.21.15.18:0/3139033705 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568cb45d1a0 con 0x5568cb5040e0 2024-04-16T17:15:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.621+0000 7f818aa00640 10 client.28294 handle_command_reply: tid=0 2024-04-16T17:15:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.621+0000 7f81ae56b740 1 client.28294 shutdown 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.621+0000 7f81ae56b740 1 -- 172.21.15.18:0/3139033705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 msgr2=0x5568cb4f0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.621+0000 7f81ae56b740 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4f0bf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8190009dc0 tx=0x7f81900117b0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 1 -- 172.21.15.18:0/3139033705 shutdown_connections 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568cb5040e0 0x5568cb506530 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4f0bf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 1 --2- 172.21.15.18:0/3139033705 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 1 -- 172.21.15.18:0/3139033705 >> 172.21.15.18:0/3139033705 conn(0x5568cb4855d0 msgr2=0x5568cb4e72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f818aa00640 0 client.28294 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 1 -- 172.21.15.18:0/3139033705 shutdown_connections 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 1 -- 172.21.15.18:0/3139033705 wait complete. 2024-04-16T17:15:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.622+0000 7f81ae56b740 20 client.28294 trim_cache size 0 max 16384 2024-04-16T17:15:05.640 INFO:tasks.workunit.client.0.smithi018.stdout: 1 389870 3.65 MB/sec execute 438 sec latency 66.750 ms 2024-04-16T17:15:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.652+0000 7f81ae56b740 1 Processor -- start 2024-04-16T17:15:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.653+0000 7f81ae56b740 1 -- start start 2024-04-16T17:15:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.653+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.653+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb50be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.653+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb50c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.653+0000 7f81ae56b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41c0ef0 con 0x5568cb4ea8a0 2024-04-16T17:15:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.653+0000 7f81ae56b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41be7c0 con 0x5568cb4f3e20 2024-04-16T17:15:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.653+0000 7f81ae56b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41bc630 con 0x5568cb4eb610 2024-04-16T17:15:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.654+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb50be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.654+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb50be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39124/0 (socket says 172.21.15.18:39124) 2024-04-16T17:15:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.654+0000 7f81a3e00640 1 -- 172.21.15.18:0/4208923511 learned_addr learned my addr 172.21.15.18:0/4208923511 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.654+0000 7f81a3e00640 1 -- 172.21.15.18:0/4208923511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 msgr2=0x5568cb50c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.655+0000 7f81a3e00640 1 --2- 172.21.15.18:0/4208923511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb50c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.655+0000 7f81a3e00640 1 -- 172.21.15.18:0/4208923511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 msgr2=0x5568cb4ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.655+0000 7f81a3e00640 1 --2- 172.21.15.18:0/4208923511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.655+0000 7f81a3e00640 1 -- 172.21.15.18:0/4208923511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568cb4e9b70 con 0x5568cb4eb610 2024-04-16T17:15:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.655+0000 7f81a3e00640 1 --2- 172.21.15.18:0/4208923511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb50be40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8190009d90 tx=0x7f8190005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46aa22c77d310bd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.656+0000 7f8195600640 1 -- 172.21.15.18:0/4208923511 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819002d070 con 0x5568cb4eb610 2024-04-16T17:15:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.656+0000 7f8195600640 1 -- 172.21.15.18:0/4208923511 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81900057f0 con 0x5568cb4eb610 2024-04-16T17:15:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.656+0000 7f8195600640 1 -- 172.21.15.18:0/4208923511 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819001ad30 con 0x5568cb4eb610 2024-04-16T17:15:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.657+0000 7f81ae56b740 1 -- 172.21.15.18:0/4208923511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 msgr2=0x5568cb50be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.657+0000 7f81ae56b740 1 --2- 172.21.15.18:0/4208923511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb50be40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8190009d90 tx=0x7f8190005d70 comp rx=0 tx=0).stop 2024-04-16T17:15:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.657+0000 7f81ae56b740 1 -- 172.21.15.18:0/4208923511 shutdown_connections 2024-04-16T17:15:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.657+0000 7f81ae56b740 1 --2- 172.21.15.18:0/4208923511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb50c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.657+0000 7f81ae56b740 1 --2- 172.21.15.18:0/4208923511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb50be40 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.657+0000 7f81ae56b740 1 --2- 172.21.15.18:0/4208923511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.657+0000 7f81ae56b740 1 -- 172.21.15.18:0/4208923511 >> 172.21.15.18:0/4208923511 conn(0x5568cb4855d0 msgr2=0x5568cb4ef540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.658+0000 7f81ae56b740 1 -- 172.21.15.18:0/4208923511 shutdown_connections 2024-04-16T17:15:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.658+0000 7f81ae56b740 1 -- 172.21.15.18:0/4208923511 wait complete. 2024-04-16T17:15:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.659+0000 7f81ae56b740 1 -- start start 2024-04-16T17:15:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.659+0000 7f81ae56b740 1 Processor -- start 2024-04-16T17:15:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4ee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81ae56b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81ae56b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41bcc80 con 0x5568cb4ea8a0 2024-04-16T17:15:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81ae56b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41be0f0 con 0x5568cb4f3e20 2024-04-16T17:15:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81ae56b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41bd350 con 0x5568cb4eb610 2024-04-16T17:15:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81ae56b740 10 client.0 _ll_register_callbacks cb 0x7f8190009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.662+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4ee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39130/0 (socket says 172.21.15.18:39130) 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.663+0000 7f81a3e00640 1 -- 172.21.15.18:0/669471451 learned_addr learned my addr 172.21.15.18:0/669471451 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.663+0000 7f81a9800640 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.663+0000 7f81a9800640 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.663+0000 7f81a3e00640 1 -- 172.21.15.18:0/669471451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 msgr2=0x5568cb4ee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.663+0000 7f81a3e00640 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4ee6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.663+0000 7f81a3e00640 1 -- 172.21.15.18:0/669471451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 msgr2=0x5568cb4edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.663+0000 7f81a3e00640 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.663+0000 7f81a3e00640 1 -- 172.21.15.18:0/669471451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568cb569980 con 0x5568cb4eb610 2024-04-16T17:15:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.664+0000 7f81a9800640 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.664+0000 7f81a3e00640 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4ee170 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8190000c00 tx=0x7f8190012870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5ce16e58368054 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.664+0000 7f818aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:15:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.665+0000 7f818aa00640 1 -- 172.21.15.18:0/669471451 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819002d070 con 0x5568cb4eb610 2024-04-16T17:15:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.665+0000 7f818aa00640 1 -- 172.21.15.18:0/669471451 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9642+0+0 (secure 0 0 0) 0x7f819001a930 con 0x5568cb4eb610 2024-04-16T17:15:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.665+0000 7f81ae56b740 1 -- 172.21.15.18:0/669471451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81900057f0 con 0x5568cb4eb610 2024-04-16T17:15:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.666+0000 7f818aa00640 1 -- 172.21.15.18:0/669471451 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f819001ab90 con 0x5568cb4eb610 2024-04-16T17:15:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.666+0000 7f81ae56b740 10 client.28300 fetch_fsmap learned FSMap version 29 2024-04-16T17:15:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.666+0000 7f81ae56b740 1 -- 172.21.15.18:0/669471451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568cb56ac90 con 0x5568cb4eb610 2024-04-16T17:15:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.666+0000 7f818aa00640 1 -- 172.21.15.18:0/669471451 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81900057f0 con 0x5568cb4eb610 2024-04-16T17:15:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.668+0000 7f818aa00640 1 -- 172.21.15.18:0/669471451 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f8190039020 con 0x5568cb4eb610 2024-04-16T17:15:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.668+0000 7f81ae56b740 10 client.28300 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:15:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.668+0000 7f81ae56b740 10 client.28300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.669+0000 7f81ae56b740 20 client.28300 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.669+0000 7f81ae56b740 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568cb56b2a0 0x5568cb56d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.669+0000 7f81ae56b740 4 client.28300 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:15:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.669+0000 7f81ae56b740 1 -- 172.21.15.18:0/669471451 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568cb5068c0 con 0x5568cb56b2a0 2024-04-16T17:15:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.669+0000 7f81a9800640 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568cb56b2a0 0x5568cb56d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.671+0000 7f81a9800640 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568cb56b2a0 0x5568cb56d690 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60ce1aaf737287b server_cookie=5a1d8e3f2c56ffe4 in_seq=0 out_seq=0 2024-04-16T17:15:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.671+0000 7f818aa00640 10 client.28300 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.940+0000 7f818aa00640 1 -- 172.21.15.18:0/669471451 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568cb5068c0 con 0x5568cb56b2a0 2024-04-16T17:15:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.940+0000 7f818aa00640 10 client.28300 handle_command_reply: tid=0 2024-04-16T17:15:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.940+0000 7f81ae56b740 1 client.28300 shutdown 2024-04-16T17:15:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f81ae56b740 1 -- 172.21.15.18:0/669471451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 msgr2=0x5568cb4ee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f81ae56b740 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4ee170 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8190000c00 tx=0x7f8190012870 comp rx=0 tx=0).stop 2024-04-16T17:15:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f81ae56b740 1 -- 172.21.15.18:0/669471451 shutdown_connections 2024-04-16T17:15:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f81ae56b740 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5568cb56b2a0 0x5568cb56d690 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f81ae56b740 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568cb4f3e20 0x5568cb4ee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f81ae56b740 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568cb4eb610 0x5568cb4ee170 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f81ae56b740 1 --2- 172.21.15.18:0/669471451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5568cb4ea8a0 0x5568cb4edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f81ae56b740 1 -- 172.21.15.18:0/669471451 >> 172.21.15.18:0/669471451 conn(0x5568cb4855d0 msgr2=0x5568cb4e3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.941+0000 7f818aa00640 0 client.28300 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.943+0000 7f81ae56b740 1 -- 172.21.15.18:0/669471451 shutdown_connections 2024-04-16T17:15:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.943+0000 7f81ae56b740 1 -- 172.21.15.18:0/669471451 wait complete. 2024-04-16T17:15:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.943+0000 7f81ae56b740 20 client.28300 trim_cache size 0 max 16384 2024-04-16T17:15:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:15:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:15:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:15:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:15:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.946+0000 7f81ab800640 1 -- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f818c0a3e60 msgr2=0x7f818c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.946+0000 7f81ab800640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f818c0a3e60 0x7f818c0a6320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f81980efa00 tx=0x7f81980f8210 comp rx=0 tx=0).stop 2024-04-16T17:15:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.946+0000 7f81ab800640 1 -- 172.21.15.18:0/4186905160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81a4104470 msgr2=0x7f81a41128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.946+0000 7f81ab800640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81a4104470 0x7f81a41128d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8198037040 tx=0x7f8198032460 comp rx=0 tx=0).stop 2024-04-16T17:15:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.948+0000 7f81ab800640 1 -- 172.21.15.18:0/4186905160 shutdown_connections 2024-04-16T17:15:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.948+0000 7f81ab800640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f818c0a3e60 0x7f818c0a6320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.948+0000 7f81ab800640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4113350 0x7f81a4119930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.948+0000 7f81ab800640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a4109540 0x7f81a4112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.948+0000 7f81ab800640 1 --2- 172.21.15.18:0/4186905160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81a4104470 0x7f81a41128d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.948+0000 7f81ab800640 1 -- 172.21.15.18:0/4186905160 >> 172.21.15.18:0/4186905160 conn(0x7f81a40ffcf0 msgr2=0x7f81a4079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.949+0000 7f81ab800640 1 -- 172.21.15.18:0/4186905160 shutdown_connections 2024-04-16T17:15:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:05.950+0000 7f81ab800640 1 -- 172.21.15.18:0/4186905160 wait complete. 2024-04-16T17:15:06.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:05 smithi022 ceph-mon[36023]: pgmap v324: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-04-16T17:15:06.138 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:05 smithi022 ceph-mon[36023]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T17:15:06.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:05 smithi043 ceph-mon[36901]: pgmap v324: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-04-16T17:15:06.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:05 smithi043 ceph-mon[36901]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T17:15:06.423 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:15:06 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setuser ceph since I am not root 2024-04-16T17:15:06.423 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:15:06 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setgroup ceph since I am not root 2024-04-16T17:15:06.423 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:15:06 smithi022 ceph-mds[66065]: -- 172.21.15.22:0/1004700168 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565403209a40 con 0x5654031dd000 2024-04-16T17:15:06.423 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:15:06 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: starting mds.h at 2024-04-16T17:15:06.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 390199 3.65 MB/sec execute 439 sec latency 107.919 ms 2024-04-16T17:15:07.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:06 smithi022 ceph-mon[36023]: daemon mds.h restarted 2024-04-16T17:15:07.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:06 smithi043 ceph-mon[36901]: daemon mds.h restarted 2024-04-16T17:15:07.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 390422 3.65 MB/sec execute 440 sec latency 61.235 ms 2024-04-16T17:15:08.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:07 smithi022 ceph-mon[36023]: pgmap v326: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-04-16T17:15:08.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:07 smithi022 ceph-mon[36023]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T17:15:08.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:07 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6836/801048775,v1:172.21.15.22:6837/801048775] up:boot 2024-04-16T17:15:08.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:07 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:15:08.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:07 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:15:08.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:07 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:15:08.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:07 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:15:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:07 smithi043 ceph-mon[36901]: pgmap v326: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-04-16T17:15:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:07 smithi043 ceph-mon[36901]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T17:15:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:07 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6836/801048775,v1:172.21.15.22:6837/801048775] up:boot 2024-04-16T17:15:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:07 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:15:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:07 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:15:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:07 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:15:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:07 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:15:08.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 390839 3.64 MB/sec execute 441 sec latency 63.663 ms 2024-04-16T17:15:09.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:08 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:08 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:08 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:08 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:08 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:08 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:08 smithi022 ceph-mon[36023]: pgmap v328: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 894 KiB/s wr, 115 op/s 2024-04-16T17:15:09.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:08 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:08 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:08 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:08 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:08 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:08 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:09.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:08 smithi043 ceph-mon[36901]: pgmap v328: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 894 KiB/s wr, 115 op/s 2024-04-16T17:15:09.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 391781 3.64 MB/sec execute 442 sec latency 28.284 ms 2024-04-16T17:15:10.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 392569 3.64 MB/sec execute 443 sec latency 37.795 ms 2024-04-16T17:15:11.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:11.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:11.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: pgmap v329: 129 pgs: 6 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 105 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:15:11.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:10 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: pgmap v329: 129 pgs: 6 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 105 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:11.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:15:11.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:10 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:15:11.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 393091 3.64 MB/sec execute 444 sec latency 68.990 ms 2024-04-16T17:15:12.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:12.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:11 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:11 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:12.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 393487 3.64 MB/sec execute 445 sec latency 74.451 ms 2024-04-16T17:15:13.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: pgmap v330: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:15:13.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:12 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: pgmap v330: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:13.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:15:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:15:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:15:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:15:13.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:12 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:13.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 394166 3.64 MB/sec execute 446 sec latency 30.278 ms 2024-04-16T17:15:14.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 395042 3.64 MB/sec execute 447 sec latency 19.585 ms 2024-04-16T17:15:14.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:14 smithi043 ceph-mon[36901]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T17:15:14.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:14 smithi022 ceph-mon[36023]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T17:15:15.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 395858 3.64 MB/sec execute 448 sec latency 28.329 ms 2024-04-16T17:15:15.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:15 smithi043 ceph-mon[36901]: pgmap v332: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 741 KiB/s wr, 117 op/s 2024-04-16T17:15:15.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:15 smithi022 ceph-mon[36023]: pgmap v332: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 741 KiB/s wr, 117 op/s 2024-04-16T17:15:16.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 396216 3.64 MB/sec execute 449 sec latency 82.181 ms 2024-04-16T17:15:17.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:05 smithi018 ceph-mon[31776]: pgmap v324: 129 pgs: 129 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-04-16T17:15:17.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:05 smithi018 ceph-mon[31776]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T17:15:17.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:06 smithi018 ceph-mon[31776]: daemon mds.h restarted 2024-04-16T17:15:17.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:07 smithi018 ceph-mon[31776]: pgmap v326: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-04-16T17:15:17.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:07 smithi018 ceph-mon[31776]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:07 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6836/801048775,v1:172.21.15.22:6837/801048775] up:boot 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:07 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:07 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:07 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:07 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:08 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:08 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:08 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:08 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:08 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:08 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:08 smithi018 ceph-mon[31776]: pgmap v328: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 894 KiB/s wr, 115 op/s 2024-04-16T17:15:17.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: pgmap v329: 129 pgs: 6 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 105 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.151 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.151 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:15:17.151 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:10 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:15:17.151 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.151 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:11 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: pgmap v330: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.152 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:12 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:14 smithi018 ceph-mon[31776]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T17:15:17.153 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:15 smithi018 ceph-mon[31776]: pgmap v332: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 741 KiB/s wr, 117 op/s 2024-04-16T17:15:17.641 INFO:tasks.workunit.client.0.smithi018.stdout: 1 396721 3.64 MB/sec execute 450 sec latency 93.282 ms 2024-04-16T17:15:17.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:17 smithi043 ceph-mon[36901]: pgmap v333: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-04-16T17:15:17.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:17 smithi022 ceph-mon[36023]: pgmap v333: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-04-16T17:15:18.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 397622 3.64 MB/sec execute 451 sec latency 37.570 ms 2024-04-16T17:15:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 -- 172.21.15.18:0/2257032927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc074d50 msgr2=0x7fbbdc10e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 --2- 172.21.15.18:0/2257032927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc074d50 0x7fbbdc10e120 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009860 tx=0x7fbbc402bc00 comp rx=0 tx=0).stop 2024-04-16T17:15:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 -- 172.21.15.18:0/2257032927 shutdown_connections 2024-04-16T17:15:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 --2- 172.21.15.18:0/2257032927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc10e660 0x7fbbdc110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 --2- 172.21.15.18:0/2257032927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc074d50 0x7fbbdc10e120 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 --2- 172.21.15.18:0/2257032927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc074430 0x7fbbdc074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 -- 172.21.15.18:0/2257032927 >> 172.21.15.18:0/2257032927 conn(0x7fbbdc0fd910 msgr2=0x7fbbdc0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 -- 172.21.15.18:0/2257032927 shutdown_connections 2024-04-16T17:15:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.092+0000 7fbbe1800640 1 -- 172.21.15.18:0/2257032927 wait complete. 2024-04-16T17:15:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.094+0000 7fbbe1800640 1 Processor -- start 2024-04-16T17:15:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.094+0000 7fbbe1800640 1 -- start start 2024-04-16T17:15:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.094+0000 7fbbe1800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc074430 0x7fbbdc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.094+0000 7fbbe1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc074d50 0x7fbbdc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.094+0000 7fbbe1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc10e660 0x7fbbdc19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.094+0000 7fbbe1800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1143f0 con 0x7fbbdc074430 2024-04-16T17:15:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.094+0000 7fbbe1800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc114270 con 0x7fbbdc10e660 2024-04-16T17:15:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.094+0000 7fbbe1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc114570 con 0x7fbbdc074d50 2024-04-16T17:15:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbdbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc074430 0x7fbbdc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbd3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc074d50 0x7fbbdc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbdbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc074430 0x7fbbdc1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36414/0 (socket says 172.21.15.18:36414) 2024-04-16T17:15:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbdbe00640 1 -- 172.21.15.18:0/3600372473 learned_addr learned my addr 172.21.15.18:0/3600372473 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbe0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc10e660 0x7fbbdc19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbe0e00640 1 -- 172.21.15.18:0/3600372473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc074d50 msgr2=0x7fbbdc1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc074d50 0x7fbbdc1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbe0e00640 1 -- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc074430 msgr2=0x7fbbdc1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc074430 0x7fbbdc1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbe0e00640 1 -- 172.21.15.18:0/3600372473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc19b050 con 0x7fbbdc10e660 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc074d50 0x7fbbdc1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc074430 0x7fbbdc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.095+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc10e660 0x7fbbdc19a790 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009fd0 tx=0x7fbbd400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.096+0000 7fbbd9600640 1 -- 172.21.15.18:0/3600372473 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400ce20 con 0x7fbbdc10e660 2024-04-16T17:15:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.096+0000 7fbbd9600640 1 -- 172.21.15.18:0/3600372473 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd4005020 con 0x7fbbdc10e660 2024-04-16T17:15:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.096+0000 7fbbe1800640 1 -- 172.21.15.18:0/3600372473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc19b340 con 0x7fbbdc10e660 2024-04-16T17:15:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.096+0000 7fbbd9600640 1 -- 172.21.15.18:0/3600372473 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4010720 con 0x7fbbdc10e660 2024-04-16T17:15:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.096+0000 7fbbe1800640 1 -- 172.21.15.18:0/3600372473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc1a7600 con 0x7fbbdc10e660 2024-04-16T17:15:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.099+0000 7fbbe5bbb740 1 Processor -- start 2024-04-16T17:15:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.099+0000 7fbbd9600640 1 -- 172.21.15.18:0/3600372473 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbd4005540 con 0x7fbbdc10e660 2024-04-16T17:15:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.099+0000 7fbbe5bbb740 1 -- start start 2024-04-16T17:15:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b27540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b27f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 0x565219b333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x565219a8c780 con 0x565219b27a80 2024-04-16T17:15:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565219a32f40 con 0x565219b287f0 2024-04-16T17:15:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565219a88740 con 0x565219b31000 2024-04-16T17:15:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbd3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b27540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbdbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b27f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbdbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b27f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36422/0 (socket says 172.21.15.18:36422) 2024-04-16T17:15:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbdbe00640 1 -- 172.21.15.18:0/1053896563 learned_addr learned my addr 172.21.15.18:0/1053896563 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.100+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/1053896563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 0x565219b333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbdbe00640 1 -- 172.21.15.18:0/1053896563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 msgr2=0x565219b333f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/1053896563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 0x565219b333f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbdbe00640 1 -- 172.21.15.18:0/1053896563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 msgr2=0x565219b27540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/1053896563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b27540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbdbe00640 1 -- 172.21.15.18:0/1053896563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521979f210 con 0x565219b27a80 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/1053896563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b27f00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc011600 tx=0x7fbbcc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d60b5a4fb0029d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbd9600640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3d00 0x7fbbc00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbd1600640 1 -- 172.21.15.18:0/1053896563 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc01e070 con 0x565219b27a80 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbd1600640 1 -- 172.21.15.18:0/1053896563 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbcc00fbd0 con 0x565219b27a80 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbd1600640 1 -- 172.21.15.18:0/1053896563 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc019cd0 con 0x565219b27a80 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.101+0000 7fbbd9600640 1 -- 172.21.15.18:0/3600372473 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fbbd40d12e0 con 0x7fbbdc10e660 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.102+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3d00 0x7fbbc00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.102+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1053896563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 msgr2=0x565219b27f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.102+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1053896563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b27f00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc011600 tx=0x7fbbcc011ac0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.102+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1053896563 shutdown_connections 2024-04-16T17:15:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.102+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1053896563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 0x565219b333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.102+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1053896563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b27f00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.102+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1053896563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b27540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.102+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1053896563 >> 172.21.15.18:0/1053896563 conn(0x565219ac25d0 msgr2=0x565219b22770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.103+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3d00 0x7fbbc00a61c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbbc402c210 tx=0x7fbbc40056d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.103+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1053896563 shutdown_connections 2024-04-16T17:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.103+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1053896563 wait complete. 2024-04-16T17:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.103+0000 7fbbe5bbb740 1 -- start start 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbe5bbb740 1 Processor -- start 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b30aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b287f0 0x565219b2de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b2e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x565219b33d30 con 0x565219b27a80 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565219a46390 con 0x565219b31000 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565219b33eb0 con 0x565219b287f0 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbe5bbb740 10 client.0 _ll_register_callbacks cb 0x565219a2ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbdbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b30aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbd3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b287f0 0x565219b2de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.104+0000 7fbbd3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b287f0 0x565219b2de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59196/0 (socket says 172.21.15.18:59196) 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbd3e00640 1 -- 172.21.15.18:0/1003687953 learned_addr learned my addr 172.21.15.18:0/1003687953 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b2e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbdbe00640 1 -- 172.21.15.18:0/1003687953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b287f0 msgr2=0x565219b2de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b287f0 0x565219b2de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbdbe00640 1 -- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 msgr2=0x565219b2e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b2e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbdbe00640 1 -- 172.21.15.18:0/1003687953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521986b620 con 0x565219b27a80 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b2e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b30aa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc011600 tx=0x7fbbcc00ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7cb4fd2cac8640 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbca000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.105+0000 7fbbca000640 1 -- 172.21.15.18:0/1003687953 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc027060 con 0x565219b27a80 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.106+0000 7fbbca000640 1 -- 172.21.15.18:0/1003687953 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fbbcc019a10 con 0x565219b27a80 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.106+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1003687953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbcc00fbd0 con 0x565219b27a80 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.106+0000 7fbbca000640 1 -- 172.21.15.18:0/1003687953 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbbcc01e040 con 0x565219b27a80 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.106+0000 7fbbe5bbb740 10 client.18153 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.106+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1003687953 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565219b3fe00 con 0x565219b27a80 2024-04-16T17:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.106+0000 7fbbca000640 1 -- 172.21.15.18:0/1003687953 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbcc00fbd0 con 0x565219b27a80 2024-04-16T17:15:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.106+0000 7fbbca000640 1 -- 172.21.15.18:0/1003687953 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fbbcc021070 con 0x565219b27a80 2024-04-16T17:15:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.107+0000 7fbbe5bbb740 10 client.18153 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.107+0000 7fbbe5bbb740 10 client.18153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.107+0000 7fbbe5bbb740 20 client.18153 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.107+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565219b41200 0x565219b435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.107+0000 7fbbe5bbb740 4 client.18153 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.107+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1003687953 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565219a9a1a0 con 0x565219b41200 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.107+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565219b41200 0x565219b435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.108+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565219b41200 0x565219b435f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=405e7e4275ed8db3 server_cookie=438b922672233e83 in_seq=0 out_seq=0 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.108+0000 7fbbca000640 10 client.18153 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.109+0000 7fbbca000640 1 -- 172.21.15.18:0/1003687953 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565219a9a1a0 con 0x565219b41200 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.109+0000 7fbbca000640 10 client.18153 handle_command_reply: tid=0 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.109+0000 7fbbe5bbb740 1 client.18153 shutdown 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.109+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1003687953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 msgr2=0x565219b30aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.109+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b30aa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc011600 tx=0x7fbbcc00ffb0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1003687953 shutdown_connections 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565219b41200 0x565219b435f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b2e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b287f0 0x565219b2de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1003687953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b30aa0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1003687953 >> 172.21.15.18:0/1003687953 conn(0x565219ac25d0 msgr2=0x565219b244c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbca000640 0 client.18153 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1003687953 shutdown_connections 2024-04-16T17:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1003687953 wait complete. 2024-04-16T17:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.110+0000 7fbbe5bbb740 20 client.18153 trim_cache size 0 max 16384 2024-04-16T17:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.129+0000 7fbbe5bbb740 1 Processor -- start 2024-04-16T17:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe5bbb740 1 -- start start 2024-04-16T17:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b27a80 0x565219b2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b287f0 0x565219b3cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b30560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b3f30 con 0x565219b287f0 2024-04-16T17:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b1800 con 0x565219b31000 2024-04-16T17:15:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1af670 con 0x565219b27a80 2024-04-16T17:15:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbd3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b287f0 0x565219b3cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b30560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b30560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54276/0 (socket says 172.21.15.18:54276) 2024-04-16T17:15:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe0e00640 1 -- 172.21.15.18:0/3957860468 learned_addr learned my addr 172.21.15.18:0/3957860468 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbd3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b287f0 0x565219b3cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36438/0 (socket says 172.21.15.18:36438) 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/3957860468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b27a80 0x565219b2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.130+0000 7fbbe0e00640 1 -- 172.21.15.18:0/3957860468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b27a80 msgr2=0x565219b2ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.131+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/3957860468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b27a80 0x565219b2ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.131+0000 7fbbe0e00640 1 -- 172.21.15.18:0/3957860468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b287f0 msgr2=0x565219b3cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.131+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/3957860468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b287f0 0x565219b3cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.131+0000 7fbbe0e00640 1 -- 172.21.15.18:0/3957860468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565219b2e890 con 0x565219b31000 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.131+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/3957860468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b30560 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c760 tx=0x7fbbd4005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde643fd595dc378 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.131+0000 7fbbd0c00640 1 -- 172.21.15.18:0/3957860468 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40b5070 con 0x565219b31000 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.131+0000 7fbbd0c00640 1 -- 172.21.15.18:0/3957860468 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd40a5a80 con 0x565219b31000 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.131+0000 7fbbd0c00640 1 -- 172.21.15.18:0/3957860468 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40a5830 con 0x565219b31000 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/3957860468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 msgr2=0x565219b30560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/3957860468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b30560 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c760 tx=0x7fbbd4005940 comp rx=0 tx=0).stop 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/3957860468 shutdown_connections 2024-04-16T17:15:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/3957860468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b31000 0x565219b30560 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/3957860468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b287f0 0x565219b3cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/3957860468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b27a80 0x565219b2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/3957860468 >> 172.21.15.18:0/3957860468 conn(0x565219ac25d0 msgr2=0x565219b2d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/3957860468 shutdown_connections 2024-04-16T17:15:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.132+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/3957860468 wait complete. 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 1 -- start start 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 1 Processor -- start 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b2bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b39140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 0x565219b2fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1afcc0 con 0x565219b27a80 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b1130 con 0x565219b287f0 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b0390 con 0x565219b31000 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbe5bbb740 10 client.0 _ll_register_callbacks cb 0x7fbbcc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbd3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b39140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbdbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b2bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbd3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b39140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54280/0 (socket says 172.21.15.18:54280) 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.133+0000 7fbbd3e00640 1 -- 172.21.15.18:0/1845352090 learned_addr learned my addr 172.21.15.18:0/1845352090 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbe0e00640 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 0x565219b2fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbd3e00640 1 -- 172.21.15.18:0/1845352090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 msgr2=0x565219b2fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 0x565219b2fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbd3e00640 1 -- 172.21.15.18:0/1845352090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 msgr2=0x565219b2bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b2bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbd3e00640 1 -- 172.21.15.18:0/1845352090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565219b30320 con 0x565219b287f0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbd3e00640 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b39140 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009830 tx=0x7fbbc402bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d02f4fc363817f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbca000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbca000640 1 -- 172.21.15.18:0/1845352090 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc405c070 con 0x565219b287f0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.134+0000 7fbbca000640 1 -- 172.21.15.18:0/1845352090 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fbbc402c860 con 0x565219b287f0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1845352090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd40a5a80 con 0x565219b287f0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbca000640 1 -- 172.21.15.18:0/1845352090 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbbc4068050 con 0x565219b287f0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 10 client.28457 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1845352090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565219ba7be0 con 0x565219b287f0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbca000640 1 -- 172.21.15.18:0/1845352090 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd40a5a80 con 0x565219b287f0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbca000640 1 -- 172.21.15.18:0/1845352090 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fbbc403c050 con 0x565219b287f0 2024-04-16T17:15:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 10 client.28457 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 10 client.28457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 20 client.28457 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565219ba8340 0x565219baa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 4 client.28457 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:15:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.135+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1845352090 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565219a94010 con 0x565219ba8340 2024-04-16T17:15:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.136+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565219ba8340 0x565219baa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.136+0000 7fbbdbe00640 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565219ba8340 0x565219baa760 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c040ed34cf15cd server_cookie=32180df94a1b6d4e in_seq=0 out_seq=0 2024-04-16T17:15:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.136+0000 7fbbca000640 10 client.28457 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:19.256 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:17 smithi018 ceph-mon[31776]: pgmap v333: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-04-16T17:15:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbca000640 1 -- 172.21.15.18:0/1845352090 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565219a94010 con 0x565219ba8340 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbca000640 10 client.28457 handle_command_reply: tid=0 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 client.28457 shutdown 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1845352090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 msgr2=0x565219b39140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b39140 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009830 tx=0x7fbbc402bc00 comp rx=0 tx=0).stop 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1845352090 shutdown_connections 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x565219ba8340 0x565219baa760 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565219b31000 0x565219b2fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565219b287f0 0x565219b39140 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 --2- 172.21.15.18:0/1845352090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565219b27a80 0x565219b2bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1845352090 >> 172.21.15.18:0/1845352090 conn(0x565219ac25d0 msgr2=0x565219b20f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.318+0000 7fbbca000640 0 client.28457 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.319+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1845352090 shutdown_connections 2024-04-16T17:15:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.319+0000 7fbbe5bbb740 1 -- 172.21.15.18:0/1845352090 wait complete. 2024-04-16T17:15:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.319+0000 7fbbe5bbb740 20 client.28457 trim_cache size 0 max 16384 2024-04-16T17:15:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:15:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:15:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:15:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:15:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 -- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3d00 msgr2=0x7fbbc00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3d00 0x7fbbc00a61c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbbc402c210 tx=0x7fbbc40056d0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 -- 172.21.15.18:0/3600372473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc10e660 msgr2=0x7fbbdc19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc10e660 0x7fbbdc19a790 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009fd0 tx=0x7fbbd400ef10 comp rx=0 tx=0).stop 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 -- 172.21.15.18:0/3600372473 shutdown_connections 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3d00 0x7fbbc00a61c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc10e660 0x7fbbdc19a790 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc074d50 0x7fbbdc1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 --2- 172.21.15.18:0/3600372473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc074430 0x7fbbdc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 -- 172.21.15.18:0/3600372473 >> 172.21.15.18:0/3600372473 conn(0x7fbbdc0fd910 msgr2=0x7fbbdc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.321+0000 7fbbe1800640 1 -- 172.21.15.18:0/3600372473 shutdown_connections 2024-04-16T17:15:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:19.322+0000 7fbbe1800640 1 -- 172.21.15.18:0/3600372473 wait complete. 2024-04-16T17:15:19.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 398313 3.64 MB/sec execute 452 sec latency 91.003 ms 2024-04-16T17:15:19.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:19 smithi043 ceph-mon[36901]: pgmap v334: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-04-16T17:15:19.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:19 smithi022 ceph-mon[36023]: pgmap v334: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-04-16T17:15:20.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 399085 3.64 MB/sec execute 453 sec latency 28.594 ms 2024-04-16T17:15:21.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 399487 3.63 MB/sec execute 454 sec latency 91.972 ms 2024-04-16T17:15:21.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:21 smithi043 ceph-mon[36901]: pgmap v335: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-04-16T17:15:21.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:21 smithi022 ceph-mon[36023]: pgmap v335: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-04-16T17:15:22.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 399806 3.63 MB/sec execute 455 sec latency 76.604 ms 2024-04-16T17:15:23.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 401014 3.64 MB/sec execute 456 sec latency 38.892 ms 2024-04-16T17:15:23.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:23 smithi043 ceph-mon[36901]: pgmap v336: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:15:23.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:23 smithi022 ceph-mon[36023]: pgmap v336: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:15:24.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 401581 3.64 MB/sec execute 457 sec latency 21.173 ms 2024-04-16T17:15:25.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 402417 3.64 MB/sec execute 458 sec latency 37.024 ms 2024-04-16T17:15:25.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:25 smithi022 ceph-mon[36023]: pgmap v337: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:15:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:25 smithi043 ceph-mon[36901]: pgmap v337: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:15:26.236 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:19 smithi018 ceph-mon[31776]: pgmap v334: 129 pgs: 129 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-04-16T17:15:26.237 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:21 smithi018 ceph-mon[31776]: pgmap v335: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-04-16T17:15:26.237 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:23 smithi018 ceph-mon[31776]: pgmap v336: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:15:26.237 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:25 smithi018 ceph-mon[31776]: pgmap v337: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-04-16T17:15:26.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 402776 3.64 MB/sec execute 459 sec latency 59.970 ms 2024-04-16T17:15:27.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 403059 3.63 MB/sec execute 460 sec latency 118.990 ms 2024-04-16T17:15:27.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:27 smithi022 ceph-mon[36023]: pgmap v338: 129 pgs: 129 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-04-16T17:15:28.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:27 smithi043 ceph-mon[36901]: pgmap v338: 129 pgs: 129 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-04-16T17:15:28.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 404060 3.64 MB/sec execute 461 sec latency 52.140 ms 2024-04-16T17:15:29.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 404858 3.64 MB/sec execute 462 sec latency 17.028 ms 2024-04-16T17:15:29.802 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:29 smithi043 ceph-mon[36901]: pgmap v339: 129 pgs: 129 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-04-16T17:15:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:29 smithi022 ceph-mon[36023]: pgmap v339: 129 pgs: 129 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-04-16T17:15:30.143 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:27 smithi018 ceph-mon[31776]: pgmap v338: 129 pgs: 129 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-04-16T17:15:30.143 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:29 smithi018 ceph-mon[31776]: pgmap v339: 129 pgs: 129 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-04-16T17:15:30.642 INFO:tasks.workunit.client.0.smithi018.stdout: 1 405576 3.64 MB/sec execute 463 sec latency 26.227 ms 2024-04-16T17:15:31.647 INFO:tasks.workunit.client.0.smithi018.stdout: 1 405959 3.64 MB/sec execute 464 sec latency 97.455 ms 2024-04-16T17:15:31.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:15:31.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 -- 172.21.15.18:0/2528094792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f500743d0 msgr2=0x7f2f500747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 --2- 172.21.15.18:0/2528094792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f500743d0 0x7f2f500747b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009900 tx=0x7f2f4002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 -- 172.21.15.18:0/2528094792 shutdown_connections 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 --2- 172.21.15.18:0/2528094792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5010e6a0 0x7f2f50110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 --2- 172.21.15.18:0/2528094792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f50074cf0 0x7f2f5010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 --2- 172.21.15.18:0/2528094792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f500743d0 0x7f2f500747b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 -- 172.21.15.18:0/2528094792 >> 172.21.15.18:0/2528094792 conn(0x7f2f500fd950 msgr2=0x7f2f500ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 -- 172.21.15.18:0/2528094792 shutdown_connections 2024-04-16T17:15:31.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.860+0000 7f2f55400640 1 -- 172.21.15.18:0/2528094792 wait complete. 2024-04-16T17:15:31.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.861+0000 7f2f55400640 1 Processor -- start 2024-04-16T17:15:31.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.861+0000 7f2f55400640 1 -- start start 2024-04-16T17:15:31.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f55400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f500743d0 0x7f2f501a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f55400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f50074cf0 0x7f2f501a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f55400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5010e6a0 0x7f2f5019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f55400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50114470 con 0x7f2f50074cf0 2024-04-16T17:15:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f55400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501142f0 con 0x7f2f5010e6a0 2024-04-16T17:15:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f55400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501145f0 con 0x7f2f500743d0 2024-04-16T17:15:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f4fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f500743d0 0x7f2f501a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f4fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f500743d0 0x7f2f501a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55916/0 (socket says 172.21.15.18:55916) 2024-04-16T17:15:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f4fe00640 1 -- 172.21.15.18:0/580402558 learned_addr learned my addr 172.21.15.18:0/580402558 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f4f400640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f50074cf0 0x7f2f501a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f54a00640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5010e6a0 0x7f2f5019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f4fe00640 1 -- 172.21.15.18:0/580402558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5010e6a0 msgr2=0x7f2f5019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f4fe00640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5010e6a0 0x7f2f5019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f4fe00640 1 -- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f50074cf0 msgr2=0x7f2f501a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.862+0000 7f2f4fe00640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f50074cf0 0x7f2f501a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.863+0000 7f2f4fe00640 1 -- 172.21.15.18:0/580402558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5019afa0 con 0x7f2f500743d0 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.863+0000 7f2f4fe00640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f500743d0 0x7f2f501a0610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000c00 tx=0x7f2f40004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.863+0000 7f2f4cc00640 1 -- 172.21.15.18:0/580402558 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40042020 con 0x7f2f500743d0 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.863+0000 7f2f4cc00640 1 -- 172.21.15.18:0/580402558 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f400054e0 con 0x7f2f500743d0 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.863+0000 7f2f55400640 1 -- 172.21.15.18:0/580402558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5019b230 con 0x7f2f500743d0 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.863+0000 7f2f4cc00640 1 -- 172.21.15.18:0/580402558 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40003b80 con 0x7f2f500743d0 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.863+0000 7f2f55400640 1 -- 172.21.15.18:0/580402558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f501a7460 con 0x7f2f500743d0 2024-04-16T17:15:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f59808740 1 Processor -- start 2024-04-16T17:15:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f59808740 1 -- start start 2024-04-16T17:15:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f59808740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89390 0x555ceee89770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f59808740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89cb0 0x555ceee92350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f59808740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee94c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f59808740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555ceedef120 con 0x555ceee89cb0 2024-04-16T17:15:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f59808740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555ceed95340 con 0x555ceee92890 2024-04-16T17:15:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f59808740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555ceedeb0e0 con 0x555ceee89390 2024-04-16T17:15:31.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89390 0x555ceee89770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89390 0x555ceee89770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:55918/0 (socket says 172.21.15.18:55918) 2024-04-16T17:15:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f54a00640 1 -- 172.21.15.18:0/2254559391 learned_addr learned my addr 172.21.15.18:0/2254559391 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f4f400640 1 --2- 172.21.15.18:0/2254559391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee94c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f4cc00640 1 -- 172.21.15.18:0/580402558 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f40045020 con 0x7f2f500743d0 2024-04-16T17:15:31.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f54a00640 1 -- 172.21.15.18:0/2254559391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 msgr2=0x555ceee94c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f54a00640 1 --2- 172.21.15.18:0/2254559391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee94c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f54a00640 1 -- 172.21.15.18:0/2254559391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89cb0 msgr2=0x555ceee92350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f54a00640 1 --2- 172.21.15.18:0/2254559391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89cb0 0x555ceee92350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f54a00640 1 -- 172.21.15.18:0/2254559391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ceebcb7d0 con 0x555ceee89390 2024-04-16T17:15:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f4cc00640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2f340a3da0 0x7f2f340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f4cc00640 1 -- 172.21.15.18:0/580402558 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f2f400f2250 con 0x7f2f500743d0 2024-04-16T17:15:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f4fe00640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2f340a3da0 0x7f2f340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.866+0000 7f2f4fe00640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2f340a3da0 0x7f2f340a6260 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x555ceee88e60 tx=0x7f2f400035f0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f54a00640 1 --2- 172.21.15.18:0/2254559391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89390 0x555ceee89770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009f30 tx=0x7f2f480118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a893a98e4a79819 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f45200640 1 -- 172.21.15.18:0/2254559391 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48004d20 con 0x555ceee89390 2024-04-16T17:15:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f45200640 1 -- 172.21.15.18:0/2254559391 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f48004050 con 0x555ceee89390 2024-04-16T17:15:31.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f45200640 1 -- 172.21.15.18:0/2254559391 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48005260 con 0x555ceee89390 2024-04-16T17:15:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- 172.21.15.18:0/2254559391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89390 msgr2=0x555ceee89770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 --2- 172.21.15.18:0/2254559391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89390 0x555ceee89770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009f30 tx=0x7f2f480118d0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- 172.21.15.18:0/2254559391 shutdown_connections 2024-04-16T17:15:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 --2- 172.21.15.18:0/2254559391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee94c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 --2- 172.21.15.18:0/2254559391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89cb0 0x555ceee92350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 --2- 172.21.15.18:0/2254559391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89390 0x555ceee89770 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- 172.21.15.18:0/2254559391 >> 172.21.15.18:0/2254559391 conn(0x555ceee83ac0 msgr2=0x555ceee83ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:31.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- 172.21.15.18:0/2254559391 shutdown_connections 2024-04-16T17:15:31.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- 172.21.15.18:0/2254559391 wait complete. 2024-04-16T17:15:31.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- start start 2024-04-16T17:15:31.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 Processor -- start 2024-04-16T17:15:31.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee89390 0x555ceee8f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee8fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee92890 0x555ceee903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555ceee95680 con 0x555ceee92890 2024-04-16T17:15:31.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555ceeda8710 con 0x555ceee89390 2024-04-16T17:15:31.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555ceee95800 con 0x555ceee89cb0 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 10 client.0 _ll_register_callbacks cb 0x555ceeda3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee89390 0x555ceee8f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f4f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee8fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee89390 0x555ceee8f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57272/0 (socket says 172.21.15.18:57272) 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f54a00640 1 -- 172.21.15.18:0/1621684400 learned_addr learned my addr 172.21.15.18:0/1621684400 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f54a00640 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee92890 0x555ceee903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f4f400640 1 -- 172.21.15.18:0/1621684400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee89390 msgr2=0x555ceee8f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f4f400640 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee89390 0x555ceee8f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f4f400640 1 -- 172.21.15.18:0/1621684400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee92890 msgr2=0x555ceee903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f4f400640 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee92890 0x555ceee903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f4f400640 1 -- 172.21.15.18:0/1621684400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ceeb013f0 con 0x555ceee89cb0 2024-04-16T17:15:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f54a00640 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee89390 0x555ceee8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f4f400640 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee8fe90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f5010fa00 tx=0x7f2f3c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5458b5d746b333e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f39600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:15:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f39600640 1 -- 172.21.15.18:0/1621684400 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00f8f0 con 0x555ceee89cb0 2024-04-16T17:15:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f39600640 1 -- 172.21.15.18:0/1621684400 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f2f3c00fa90 con 0x555ceee89cb0 2024-04-16T17:15:31.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- 172.21.15.18:0/1621684400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f48004050 con 0x555ceee89cb0 2024-04-16T17:15:31.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f39600640 1 -- 172.21.15.18:0/1621684400 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f3c00fc90 con 0x555ceee89cb0 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 10 client.28336 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.874+0000 7f2f59808740 1 -- 172.21.15.18:0/1621684400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ceeac10f0 con 0x555ceee89cb0 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f39600640 1 -- 172.21.15.18:0/1621684400 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f48004050 con 0x555ceee89cb0 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f39600640 1 -- 172.21.15.18:0/1621684400 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2f3c01c420 con 0x555ceee89cb0 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f59808740 10 client.28336 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f59808740 10 client.28336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f59808740 20 client.28336 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f59808740 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555ceeea2da0 0x555ceeea51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f59808740 4 client.28336 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f59808740 1 -- 172.21.15.18:0/1621684400 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ceedfcb40 con 0x555ceeea2da0 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f54a00640 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555ceeea2da0 0x555ceeea51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f54a00640 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555ceeea2da0 0x555ceeea51c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12426c6344d4a20 server_cookie=73d7e72577b2c069 in_seq=0 out_seq=0 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f39600640 10 client.28336 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f39600640 1 -- 172.21.15.18:0/1621684400 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ceedfcb40 con 0x555ceeea2da0 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f39600640 10 client.28336 handle_command_reply: tid=0 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.879+0000 7f2f59808740 1 client.28336 shutdown 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 -- 172.21.15.18:0/1621684400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 msgr2=0x555ceee8fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee8fe90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f5010fa00 tx=0x7f2f3c012b60 comp rx=0 tx=0).stop 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 -- 172.21.15.18:0/1621684400 shutdown_connections 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555ceeea2da0 0x555ceeea51c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee92890 0x555ceee903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee8fe90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 --2- 172.21.15.18:0/1621684400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee89390 0x555ceee8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 -- 172.21.15.18:0/1621684400 >> 172.21.15.18:0/1621684400 conn(0x555ceee83ac0 msgr2=0x555ceee85810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f39600640 0 client.28336 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 -- 172.21.15.18:0/1621684400 shutdown_connections 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 1 -- 172.21.15.18:0/1621684400 wait complete. 2024-04-16T17:15:31.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.881+0000 7f2f59808740 20 client.28336 trim_cache size 0 max 16384 2024-04-16T17:15:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f59808740 1 Processor -- start 2024-04-16T17:15:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f59808740 1 -- start start 2024-04-16T17:15:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f59808740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f59808740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f59808740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f59808740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501b3f10 con 0x555ceee89390 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f59808740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501b17e0 con 0x555ceee92890 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f59808740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501af650 con 0x555ceee89cb0 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49242/0 (socket says 172.21.15.18:49242) 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f54a00640 1 -- 172.21.15.18:0/534724714 learned_addr learned my addr 172.21.15.18:0/534724714 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.902+0000 7f2f4f400640 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.903+0000 7f2f54a00640 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.903+0000 7f2f54a00640 1 -- 172.21.15.18:0/534724714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 msgr2=0x555ceee901f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.903+0000 7f2f54a00640 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee901f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.903+0000 7f2f54a00640 1 -- 172.21.15.18:0/534724714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 msgr2=0x555ceee829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:31.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.903+0000 7f2f54a00640 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee829a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.903+0000 7f2f54a00640 1 -- 172.21.15.18:0/534724714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ceee9ea80 con 0x555ceee89390 2024-04-16T17:15:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.903+0000 7f2f4f400640 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:31.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.903+0000 7f2f54a00640 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9e540 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2f48014040 tx=0x7f2f48004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ed8fd9294c1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.904+0000 7f2f3be00640 1 -- 172.21.15.18:0/534724714 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48004590 con 0x555ceee89390 2024-04-16T17:15:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.904+0000 7f2f3be00640 1 -- 172.21.15.18:0/534724714 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f4802d130 con 0x555ceee89390 2024-04-16T17:15:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.904+0000 7f2f3be00640 1 -- 172.21.15.18:0/534724714 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480286c0 con 0x555ceee89390 2024-04-16T17:15:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 -- 172.21.15.18:0/534724714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 msgr2=0x555ceee9e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9e540 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2f48014040 tx=0x7f2f48004970 comp rx=0 tx=0).stop 2024-04-16T17:15:31.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 -- 172.21.15.18:0/534724714 shutdown_connections 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 --2- 172.21.15.18:0/534724714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9e540 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 -- 172.21.15.18:0/534724714 >> 172.21.15.18:0/534724714 conn(0x555ceee83ac0 msgr2=0x555ceee82290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 -- 172.21.15.18:0/534724714 shutdown_connections 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 -- 172.21.15.18:0/534724714 wait complete. 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.905+0000 7f2f59808740 1 -- start start 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f59808740 1 Processor -- start 2024-04-16T17:15:31.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f59808740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f59808740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee9aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f59808740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee95f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f59808740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501afca0 con 0x555ceee89390 2024-04-16T17:15:31.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f59808740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501b1110 con 0x555ceee92890 2024-04-16T17:15:31.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f59808740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501b0370 con 0x555ceee89cb0 2024-04-16T17:15:31.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f59808740 10 client.0 _ll_register_callbacks cb 0x7f2f3c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:31.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.906+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49250/0 (socket says 172.21.15.18:49250) 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f54a00640 1 -- 172.21.15.18:0/636527796 learned_addr learned my addr 172.21.15.18:0/636527796 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f54a00640 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee95f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f54a00640 1 -- 172.21.15.18:0/636527796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 msgr2=0x555ceee9aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f4f400640 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee9aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f54a00640 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee9aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f54a00640 1 -- 172.21.15.18:0/636527796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 msgr2=0x555ceee95f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f54a00640 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee95f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f54a00640 1 -- 172.21.15.18:0/636527796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ceee96710 con 0x555ceee89390 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.907+0000 7f2f4f400640 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee9aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.908+0000 7f2f54a00640 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9a9b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2f48003da0 tx=0x7f2f480021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f314b509df5e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.908+0000 7f2f39600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.908+0000 7f2f39600640 1 -- 172.21.15.18:0/636527796 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4803e060 con 0x555ceee89390 2024-04-16T17:15:31.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.908+0000 7f2f39600640 1 -- 172.21.15.18:0/636527796 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f2f48004be0 con 0x555ceee89390 2024-04-16T17:15:31.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.908+0000 7f2f59808740 1 -- 172.21.15.18:0/636527796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4802d130 con 0x555ceee89390 2024-04-16T17:15:31.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f39600640 1 -- 172.21.15.18:0/636527796 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f48045050 con 0x555ceee89390 2024-04-16T17:15:31.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f59808740 10 client.18174 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:31.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f59808740 1 -- 172.21.15.18:0/636527796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ceef11700 con 0x555ceee89390 2024-04-16T17:15:31.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f39600640 1 -- 172.21.15.18:0/636527796 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f4802d130 con 0x555ceee89390 2024-04-16T17:15:31.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f39600640 1 -- 172.21.15.18:0/636527796 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2f48025040 con 0x555ceee89390 2024-04-16T17:15:31.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f59808740 10 client.18174 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:31.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f59808740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:31.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f59808740 20 client.18174 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:31.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f59808740 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555ceef11bf0 0x555ceef14010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:31.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f59808740 4 client.18174 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.909+0000 7f2f59808740 1 -- 172.21.15.18:0/636527796 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ceee7f180 con 0x555ceef11bf0 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.911+0000 7f2f4f400640 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555ceef11bf0 0x555ceef14010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.911+0000 7f2f4f400640 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555ceef11bf0 0x555ceef14010 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba350654f5f92a server_cookie=1671f960cf8f11b7 in_seq=0 out_seq=0 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f39600640 10 client.18174 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f39600640 1 -- 172.21.15.18:0/636527796 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ceee7f180 con 0x555ceef11bf0 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f39600640 10 client.18174 handle_command_reply: tid=0 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 client.18174 shutdown 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 -- 172.21.15.18:0/636527796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 msgr2=0x555ceee9a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9a9b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2f48003da0 tx=0x7f2f480021d0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 -- 172.21.15.18:0/636527796 shutdown_connections 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555ceef11bf0 0x555ceef14010 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555ceee92890 0x555ceee95f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ceee89cb0 0x555ceee9aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 --2- 172.21.15.18:0/636527796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555ceee89390 0x555ceee9a9b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 -- 172.21.15.18:0/636527796 >> 172.21.15.18:0/636527796 conn(0x555ceee83ac0 msgr2=0x555ceee82b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f39600640 0 client.18174 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 -- 172.21.15.18:0/636527796 shutdown_connections 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.912+0000 7f2f59808740 1 -- 172.21.15.18:0/636527796 wait complete. 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.913+0000 7f2f59808740 20 client.18174 trim_cache size 0 max 16384 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:15:31.915 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:15:31.916 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:15:31.916 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:15:31.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.913+0000 7f2f55400640 1 -- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2f340a3da0 msgr2=0x7f2f340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.913+0000 7f2f55400640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2f340a3da0 0x7f2f340a6260 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x555ceee88e60 tx=0x7f2f400035f0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.913+0000 7f2f55400640 1 -- 172.21.15.18:0/580402558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f500743d0 msgr2=0x7f2f501a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.913+0000 7f2f55400640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f500743d0 0x7f2f501a0610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000c00 tx=0x7f2f40004360 comp rx=0 tx=0).stop 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.914+0000 7f2f55400640 1 -- 172.21.15.18:0/580402558 shutdown_connections 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.914+0000 7f2f55400640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2f340a3da0 0x7f2f340a6260 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.914+0000 7f2f55400640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f5010e6a0 0x7f2f5019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.914+0000 7f2f55400640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f50074cf0 0x7f2f501a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.914+0000 7f2f55400640 1 --2- 172.21.15.18:0/580402558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f500743d0 0x7f2f501a0610 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.914+0000 7f2f55400640 1 -- 172.21.15.18:0/580402558 >> 172.21.15.18:0/580402558 conn(0x7f2f500fd950 msgr2=0x7f2f500fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.914+0000 7f2f55400640 1 -- 172.21.15.18:0/580402558 shutdown_connections 2024-04-16T17:15:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:31.914+0000 7f2f55400640 1 -- 172.21.15.18:0/580402558 wait complete. 2024-04-16T17:15:31.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:31 smithi022 ceph-mon[36023]: pgmap v340: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-16T17:15:31.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:15:31.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:15:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:31 smithi043 ceph-mon[36901]: pgmap v340: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-16T17:15:32.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 -- 172.21.15.18:0/475451342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104d00 msgr2=0x7f04e0108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 --2- 172.21.15.18:0/475451342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104d00 0x7f04e0108d40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009860 tx=0x7f04cc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:15:32.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 -- 172.21.15.18:0/475451342 shutdown_connections 2024-04-16T17:15:32.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 --2- 172.21.15.18:0/475451342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e0109420 0x7f04e0110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 --2- 172.21.15.18:0/475451342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104d00 0x7f04e0108d40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 --2- 172.21.15.18:0/475451342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04e0104350 0x7f04e0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 -- 172.21.15.18:0/475451342 >> 172.21.15.18:0/475451342 conn(0x7f04e00ffb70 msgr2=0x7f04e0101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 -- 172.21.15.18:0/475451342 shutdown_connections 2024-04-16T17:15:32.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.121+0000 7f04e5000640 1 -- 172.21.15.18:0/475451342 wait complete. 2024-04-16T17:15:32.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.122+0000 7f04e5000640 1 Processor -- start 2024-04-16T17:15:32.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.122+0000 7f04e5000640 1 -- start start 2024-04-16T17:15:32.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04e5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104350 0x7f04e0076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04e5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04e0109420 0x7f04e0076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04e5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e0076f90 0x7f04e01aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04e5000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0114490 con 0x7f04e0109420 2024-04-16T17:15:32.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04e5000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0114310 con 0x7f04e0104350 2024-04-16T17:15:32.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04e5000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0114610 con 0x7f04e0076f90 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104350 0x7f04e0076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04dea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04e0109420 0x7f04e0076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e0076f90 0x7f04e01aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104350 0x7f04e0076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57304/0 (socket says 172.21.15.18:57304) 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.123+0000 7f04df400640 1 -- 172.21.15.18:0/4088632158 learned_addr learned my addr 172.21.15.18:0/4088632158 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04df400640 1 -- 172.21.15.18:0/4088632158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e0076f90 msgr2=0x7f04e01aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04df400640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e0076f90 0x7f04e01aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04df400640 1 -- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04e0109420 msgr2=0x7f04e0076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04df400640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04e0109420 0x7f04e0076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04df400640 1 -- 172.21.15.18:0/4088632158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e01ab500 con 0x7f04e0104350 2024-04-16T17:15:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04df400640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104350 0x7f04e0076510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f04d000d950 tx=0x7f04d000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04d7e00640 1 -- 172.21.15.18:0/4088632158 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0017030 con 0x7f04e0104350 2024-04-16T17:15:32.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04d7e00640 1 -- 172.21.15.18:0/4088632158 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0004020 con 0x7f04e0104350 2024-04-16T17:15:32.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04d7e00640 1 -- 172.21.15.18:0/4088632158 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00047a0 con 0x7f04e0104350 2024-04-16T17:15:32.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.124+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e01ab790 con 0x7f04e0104350 2024-04-16T17:15:32.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.125+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01abc00 con 0x7f04e0104350 2024-04-16T17:15:32.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.128+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e0105820 con 0x7f04e0104350 2024-04-16T17:15:32.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.128+0000 7f04d7e00640 1 -- 172.21.15.18:0/4088632158 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f04d00041d0 con 0x7f04e0104350 2024-04-16T17:15:32.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.129+0000 7f04d7e00640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f04c40a3d10 0x7f04c40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.129+0000 7f04d7e00640 1 -- 172.21.15.18:0/4088632158 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f04d00d05b0 con 0x7f04e0104350 2024-04-16T17:15:32.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.130+0000 7f04dea00640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f04c40a3d10 0x7f04c40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.131+0000 7f04dea00640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f04c40a3d10 0x7f04c40a61d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009830 tx=0x7f04cc0056d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.134+0000 7f04d7e00640 1 -- 172.21.15.18:0/4088632158 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04d009c840 con 0x7f04e0104350 2024-04-16T17:15:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 -- 172.21.15.18:0/2366663112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 msgr2=0x7f0b6810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 --2- 172.21.15.18:0/2366663112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 0x7f0b6810bde0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009900 tx=0x7f0b5402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 -- 172.21.15.18:0/2366663112 shutdown_connections 2024-04-16T17:15:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 --2- 172.21.15.18:0/2366663112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b6810c400 0x7f0b6810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 --2- 172.21.15.18:0/2366663112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 0x7f0b6810bde0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 --2- 172.21.15.18:0/2366663112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b68102fe0 0x7f0b681033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 -- 172.21.15.18:0/2366663112 >> 172.21.15.18:0/2366663112 conn(0x7f0b680fc7e0 msgr2=0x7f0b680fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 -- 172.21.15.18:0/2366663112 shutdown_connections 2024-04-16T17:15:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.481+0000 7f0b6da00640 1 -- 172.21.15.18:0/2366663112 wait complete. 2024-04-16T17:15:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.482+0000 7f0b6da00640 1 Processor -- start 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.483+0000 7f0b6da00640 1 -- start start 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.483+0000 7f0b6da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b68102fe0 0x7f0b681026a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.483+0000 7f0b6da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 0x7f0b68100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.483+0000 7f0b6da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6810c400 0x7f0b68101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.483+0000 7f0b6da00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68112210 con 0x7f0b68102fe0 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.483+0000 7f0b6da00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68112090 con 0x7f0b68103900 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.483+0000 7f0b6da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68112390 con 0x7f0b6810c400 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 0x7f0b68100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 0x7f0b68100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57310/0 (socket says 172.21.15.18:57310) 2024-04-16T17:15:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 -- 172.21.15.18:0/3776670764 learned_addr learned my addr 172.21.15.18:0/3776670764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b6d000640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6810c400 0x7f0b68101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 -- 172.21.15.18:0/3776670764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6810c400 msgr2=0x7f0b68101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6810c400 0x7f0b68101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 -- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b68102fe0 msgr2=0x7f0b681026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b68102fe0 0x7f0b681026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 -- 172.21.15.18:0/3776670764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b681018a0 con 0x7f0b68103900 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b6d000640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6810c400 0x7f0b68101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.484+0000 7f0b67400640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 0x7f0b68100c30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0b54000c00 tx=0x7f0b54004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.485+0000 7f0b64c00640 1 -- 172.21.15.18:0/3776670764 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54042020 con 0x7f0b68103900 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.485+0000 7f0b64c00640 1 -- 172.21.15.18:0/3776670764 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b540054e0 con 0x7f0b68103900 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.485+0000 7f0b64c00640 1 -- 172.21.15.18:0/3776670764 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54003b20 con 0x7f0b68103900 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.485+0000 7f0b6da00640 1 -- 172.21.15.18:0/3776670764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b681af730 con 0x7f0b68103900 2024-04-16T17:15:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.485+0000 7f0b6da00640 1 -- 172.21.15.18:0/3776670764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b681afb40 con 0x7f0b68103900 2024-04-16T17:15:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.489+0000 7f0b71d73740 1 Processor -- start 2024-04-16T17:15:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.489+0000 7f0b71d73740 1 -- start start 2024-04-16T17:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.489+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b98c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.489+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b98cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b9983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b71d73740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d75b8f1780 con 0x55d75b996000 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b71d73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d75b897f40 con 0x55d75b98d7f0 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b71d73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d75b8ed740 con 0x55d75b98ca80 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b64c00640 1 -- 172.21.15.18:0/3776670764 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b54045020 con 0x7f0b68103900 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b6d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b98c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b6d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b98c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57326/0 (socket says 172.21.15.18:57326) 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b6d000640 1 -- 172.21.15.18:0/3522088600 learned_addr learned my addr 172.21.15.18:0/3522088600 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b67e00640 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b98cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b6d000640 1 -- 172.21.15.18:0/3522088600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 msgr2=0x55d75b98cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b6d000640 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b98cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b6d000640 1 -- 172.21.15.18:0/3522088600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 msgr2=0x55d75b9983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b6d000640 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b9983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b6d000640 1 -- 172.21.15.18:0/3522088600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75b604210 con 0x55d75b98d7f0 2024-04-16T17:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.490+0000 7f0b67e00640 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b98cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.491+0000 7f0b6d000640 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b98c540 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009e10 tx=0x7f0b6000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce39982495840c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.491+0000 7f0b5d600640 1 -- 172.21.15.18:0/3522088600 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60004d20 con 0x55d75b98d7f0 2024-04-16T17:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.491+0000 7f0b5d600640 1 -- 172.21.15.18:0/3522088600 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b60005500 con 0x55d75b98d7f0 2024-04-16T17:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.491+0000 7f0b5d600640 1 -- 172.21.15.18:0/3522088600 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600118d0 con 0x55d75b98d7f0 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 -- 172.21.15.18:0/3522088600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 msgr2=0x55d75b98c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b98c540 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009e10 tx=0x7f0b6000f730 comp rx=0 tx=0).stop 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 -- 172.21.15.18:0/3522088600 shutdown_connections 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b9983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b98cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 --2- 172.21.15.18:0/3522088600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b98c540 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 -- 172.21.15.18:0/3522088600 >> 172.21.15.18:0/3522088600 conn(0x55d75b9275d0 msgr2=0x55d75b987770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 -- 172.21.15.18:0/3522088600 shutdown_connections 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b71d73740 1 -- 172.21.15.18:0/3522088600 wait complete. 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b64c00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b4c0a3c90 0x7f0b4c0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.492+0000 7f0b64c00640 1 -- 172.21.15.18:0/3776670764 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f0b540f27e0 con 0x7f0b68103900 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 1 -- start start 2024-04-16T17:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b67e00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b4c0a3c90 0x7f0b4c0a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 1 Processor -- start 2024-04-16T17:15:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b992e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b9933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d75b998d30 con 0x55d75b996000 2024-04-16T17:15:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d75b8ab390 con 0x55d75b98d7f0 2024-04-16T17:15:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d75b998eb0 con 0x55d75b98ca80 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.493+0000 7f0b71d73740 10 client.0 _ll_register_callbacks cb 0x55d75b894f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b6d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b992e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b6d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49298/0 (socket says 172.21.15.18:49298) 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b6d000640 1 -- 172.21.15.18:0/2934165980 learned_addr learned my addr 172.21.15.18:0/2934165980 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b67e00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b4c0a3c90 0x7f0b4c0a6150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55d75b98df50 tx=0x7f0b580097b0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b67400640 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b9933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b6d000640 1 -- 172.21.15.18:0/2934165980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 msgr2=0x55d75b992e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b6d000640 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b992e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b6d000640 1 -- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 msgr2=0x55d75b9933c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b6d000640 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b9933c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b6d000640 1 -- 172.21.15.18:0/2934165980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75b6d0620 con 0x55d75b996000 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.494+0000 7f0b67400640 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b9933c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:15:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.495+0000 7f0b6d000640 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0b600058e0 tx=0x7f0b60005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9da405fa845204 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.495+0000 7f0b52000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.495+0000 7f0b52000640 1 -- 172.21.15.18:0/2934165980 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60029060 con 0x55d75b996000 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.495+0000 7f0b52000640 1 -- 172.21.15.18:0/2934165980 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f0b60005210 con 0x55d75b996000 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.495+0000 7f0b71d73740 1 -- 172.21.15.18:0/2934165980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b60005500 con 0x55d75b996000 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.495+0000 7f0b52000640 1 -- 172.21.15.18:0/2934165980 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0b60030050 con 0x55d75b996000 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.495+0000 7f0b71d73740 10 client.18186 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.495+0000 7f0b71d73740 1 -- 172.21.15.18:0/2934165980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d75b9ad6c0 con 0x55d75b996000 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b52000640 1 -- 172.21.15.18:0/2934165980 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b60005500 con 0x55d75b996000 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b52000640 1 -- 172.21.15.18:0/2934165980 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0b60019430 con 0x55d75b996000 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b71d73740 10 client.18186 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b71d73740 10 client.18186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b71d73740 20 client.18186 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d75b9aeb20 0x55d75b9b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b71d73740 4 client.18186 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b71d73740 1 -- 172.21.15.18:0/2934165980 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d75b8ff1a0 con 0x55d75b9aeb20 2024-04-16T17:15:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.496+0000 7f0b67e00640 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d75b9aeb20 0x55d75b9b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.497+0000 7f0b67e00640 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d75b9aeb20 0x55d75b9b0f10 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70dae3a6a201bf6 server_cookie=3fd215eef3f468d9 in_seq=0 out_seq=0 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.497+0000 7f0b52000640 10 client.18186 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b52000640 1 -- 172.21.15.18:0/2934165980 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d75b8ff1a0 con 0x55d75b9aeb20 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b52000640 10 client.18186 handle_command_reply: tid=0 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 client.18186 shutdown 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 -- 172.21.15.18:0/2934165980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 msgr2=0x55d75b993900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0b600058e0 tx=0x7f0b60005910 comp rx=0 tx=0).stop 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 -- 172.21.15.18:0/2934165980 shutdown_connections 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d75b9aeb20 0x55d75b9b0f10 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993900 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b52000640 0 client.18186 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b9933c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2934165980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b992e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 -- 172.21.15.18:0/2934165980 >> 172.21.15.18:0/2934165980 conn(0x55d75b9275d0 msgr2=0x55d75b9894c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.498+0000 7f0b71d73740 1 -- 172.21.15.18:0/2934165980 shutdown_connections 2024-04-16T17:15:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.499+0000 7f0b71d73740 1 -- 172.21.15.18:0/2934165980 wait complete. 2024-04-16T17:15:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.499+0000 7f0b71d73740 20 client.18186 trim_cache size 0 max 16384 2024-04-16T17:15:32.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.509+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04e010c650 con 0x7f04e0104350 2024-04-16T17:15:32.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.510+0000 7f04d7e00640 1 -- 172.21.15.18:0/4088632158 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11851 (secure 0 0 0) 0x7f04d009c140 con 0x7f04e0104350 2024-04-16T17:15:32.511 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:15:32.511 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:15:06.832807+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28439":{"gid":28439,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/801048775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":801048775},{"type":"v1","addr":"172.21.15.22:6837","nonce":801048775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:15:32.511 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-16T17:15:32.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.512+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f04c40a3d10 msgr2=0x7f04c40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.512+0000 7f04e5000640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f04c40a3d10 0x7f04c40a61d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009830 tx=0x7f04cc0056d0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104350 msgr2=0x7f04e0076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104350 0x7f04e0076510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f04d000d950 tx=0x7f04d000de10 comp rx=0 tx=0).stop 2024-04-16T17:15:32.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 shutdown_connections 2024-04-16T17:15:32.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f04c40a3d10 0x7f04c40a61d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e0076f90 0x7f04e01aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04e0109420 0x7f04e0076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 --2- 172.21.15.18:0/4088632158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e0104350 0x7f04e0076510 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 >> 172.21.15.18:0/4088632158 conn(0x7f04e00ffb70 msgr2=0x7f04e0100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 shutdown_connections 2024-04-16T17:15:32.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.513+0000 7f04e5000640 1 -- 172.21.15.18:0/4088632158 wait complete. 2024-04-16T17:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.527+0000 7f0b71d73740 1 Processor -- start 2024-04-16T17:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.527+0000 7f0b71d73740 1 -- start start 2024-04-16T17:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.527+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b9a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.527+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b989690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.528+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.528+0000 7f0b71d73740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681b8370 con 0x55d75b996000 2024-04-16T17:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.528+0000 7f0b71d73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681b5c40 con 0x55d75b98d7f0 2024-04-16T17:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.528+0000 7f0b71d73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681b3ad0 con 0x55d75b98ca80 2024-04-16T17:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.528+0000 7f0b6d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.528+0000 7f0b67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b9a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.528+0000 7f0b6d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49308/0 (socket says 172.21.15.18:49308) 2024-04-16T17:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.528+0000 7f0b6d000640 1 -- 172.21.15.18:0/912228408 learned_addr learned my addr 172.21.15.18:0/912228408 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.529+0000 7f0b67e00640 1 -- 172.21.15.18:0/912228408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 msgr2=0x55d75b989690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.529+0000 7f0b67e00640 1 --2- 172.21.15.18:0/912228408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b989690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.529+0000 7f0b67e00640 1 -- 172.21.15.18:0/912228408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 msgr2=0x55d75b993740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.529+0000 7f0b67e00640 1 --2- 172.21.15.18:0/912228408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.529+0000 7f0b67e00640 1 -- 172.21.15.18:0/912228408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75b986f90 con 0x55d75b98ca80 2024-04-16T17:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.529+0000 7f0b6d000640 1 --2- 172.21.15.18:0/912228408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:32.531 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.529+0000 7f0b67e00640 1 --2- 172.21.15.18:0/912228408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b9a1c30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55d75b995e00 tx=0x7f0b58014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3b4b2c649babaf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.533+0000 7f0b5cc00640 1 -- 172.21.15.18:0/912228408 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5802c070 con 0x55d75b98ca80 2024-04-16T17:15:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.533+0000 7f0b5cc00640 1 -- 172.21.15.18:0/912228408 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b58027730 con 0x55d75b98ca80 2024-04-16T17:15:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.533+0000 7f0b5cc00640 1 -- 172.21.15.18:0/912228408 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58020d70 con 0x55d75b98ca80 2024-04-16T17:15:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 -- 172.21.15.18:0/912228408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 msgr2=0x55d75b9a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 --2- 172.21.15.18:0/912228408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b9a1c30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55d75b995e00 tx=0x7f0b58014000 comp rx=0 tx=0).stop 2024-04-16T17:15:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 -- 172.21.15.18:0/912228408 shutdown_connections 2024-04-16T17:15:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 --2- 172.21.15.18:0/912228408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b996000 0x55d75b993740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 --2- 172.21.15.18:0/912228408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98d7f0 0x55d75b989690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 --2- 172.21.15.18:0/912228408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b98ca80 0x55d75b9a1c30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 -- 172.21.15.18:0/912228408 >> 172.21.15.18:0/912228408 conn(0x55d75b9275d0 msgr2=0x55d75b991f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 -- 172.21.15.18:0/912228408 shutdown_connections 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.534+0000 7f0b71d73740 1 -- 172.21.15.18:0/912228408 wait complete. 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 1 -- start start 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 1 Processor -- start 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98ca80 0x55d75b98fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b98d7f0 0x55d75b990250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b996000 0x55d75b990790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681b4120 con 0x55d75b98d7f0 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681b5570 con 0x55d75b98ca80 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681b47d0 con 0x55d75b996000 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.535+0000 7f0b71d73740 10 client.0 _ll_register_callbacks cb 0x7f0b60009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b98d7f0 0x55d75b990250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b98d7f0 0x55d75b990250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49318/0 (socket says 172.21.15.18:49318) 2024-04-16T17:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67400640 1 -- 172.21.15.18:0/2621560983 learned_addr learned my addr 172.21.15.18:0/2621560983 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67e00640 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98ca80 0x55d75b98fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b6d000640 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b996000 0x55d75b990790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67400640 1 -- 172.21.15.18:0/2621560983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b996000 msgr2=0x55d75b990790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67400640 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b996000 0x55d75b990790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67400640 1 -- 172.21.15.18:0/2621560983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98ca80 msgr2=0x55d75b98fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67400640 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98ca80 0x55d75b98fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b67400640 1 -- 172.21.15.18:0/2621560983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75ba13f00 con 0x55d75b98d7f0 2024-04-16T17:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.536+0000 7f0b6d000640 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b996000 0x55d75b990790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.537+0000 7f0b67400640 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b98d7f0 0x55d75b990250 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005ec0 tx=0x7f0b540be2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d6c49a14b3116d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.537+0000 7f0b52000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.540+0000 7f0b52000640 1 -- 172.21.15.18:0/2621560983 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b540beb20 con 0x55d75b98d7f0 2024-04-16T17:15:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.540+0000 7f0b52000640 1 -- 172.21.15.18:0/2621560983 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f0b5402cc50 con 0x55d75b98d7f0 2024-04-16T17:15:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.540+0000 7f0b71d73740 1 -- 172.21.15.18:0/2621560983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b58027730 con 0x55d75b98d7f0 2024-04-16T17:15:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.541+0000 7f0b52000640 1 -- 172.21.15.18:0/2621560983 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0b540be420 con 0x55d75b98d7f0 2024-04-16T17:15:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.541+0000 7f0b71d73740 10 client.18192 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.541+0000 7f0b71d73740 1 -- 172.21.15.18:0/2621560983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d75ba151d0 con 0x55d75b98d7f0 2024-04-16T17:15:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.541+0000 7f0b52000640 1 -- 172.21.15.18:0/2621560983 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b58027730 con 0x55d75b98d7f0 2024-04-16T17:15:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.542+0000 7f0b52000640 1 -- 172.21.15.18:0/2621560983 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0b540f2bd0 con 0x55d75b98d7f0 2024-04-16T17:15:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.542+0000 7f0b71d73740 10 client.18192 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.542+0000 7f0b71d73740 10 client.18192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.542+0000 7f0b71d73740 20 client.18192 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.542+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d75ba15920 0x55d75ba17d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.542+0000 7f0b71d73740 4 client.18192 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:15:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.542+0000 7f0b71d73740 1 -- 172.21.15.18:0/2621560983 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d75b8f9010 con 0x55d75ba15920 2024-04-16T17:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.546+0000 7f0b67e00640 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d75ba15920 0x55d75ba17d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.547+0000 7f0b67e00640 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d75ba15920 0x55d75ba17d40 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0db2c1aa706576 server_cookie=1e9afd6d287cab33 in_seq=0 out_seq=0 2024-04-16T17:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.547+0000 7f0b52000640 10 client.18192 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:32.647 INFO:tasks.workunit.client.0.smithi018.stdout: 1 406176 3.63 MB/sec execute 465 sec latency 210.842 ms 2024-04-16T17:15:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b52000640 1 -- 172.21.15.18:0/2621560983 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d75b8f9010 con 0x55d75ba15920 2024-04-16T17:15:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b52000640 10 client.18192 handle_command_reply: tid=0 2024-04-16T17:15:32.741 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b71d73740 1 client.18192 shutdown 2024-04-16T17:15:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b71d73740 1 -- 172.21.15.18:0/2621560983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b98d7f0 msgr2=0x55d75b990250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b98d7f0 0x55d75b990250 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005ec0 tx=0x7f0b540be2b0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b71d73740 1 -- 172.21.15.18:0/2621560983 shutdown_connections 2024-04-16T17:15:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d75ba15920 0x55d75ba17d40 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d75b996000 0x55d75b990790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d75b98d7f0 0x55d75b990250 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.741+0000 7f0b71d73740 1 --2- 172.21.15.18:0/2621560983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75b98ca80 0x55d75b98fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.742+0000 7f0b71d73740 1 -- 172.21.15.18:0/2621560983 >> 172.21.15.18:0/2621560983 conn(0x55d75b9275d0 msgr2=0x55d75b991680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.742+0000 7f0b52000640 0 client.18192 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.742+0000 7f0b71d73740 1 -- 172.21.15.18:0/2621560983 shutdown_connections 2024-04-16T17:15:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.742+0000 7f0b71d73740 1 -- 172.21.15.18:0/2621560983 wait complete. 2024-04-16T17:15:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b71d73740 20 client.18192 trim_cache size 0 max 16384 2024-04-16T17:15:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:15:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:15:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:15:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:15:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 -- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b4c0a3c90 msgr2=0x7f0b4c0a6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b4c0a3c90 0x7f0b4c0a6150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55d75b98df50 tx=0x7f0b580097b0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 -- 172.21.15.18:0/3776670764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 msgr2=0x7f0b68100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 0x7f0b68100c30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0b54000c00 tx=0x7f0b54004360 comp rx=0 tx=0).stop 2024-04-16T17:15:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 -- 172.21.15.18:0/3776670764 shutdown_connections 2024-04-16T17:15:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b4c0a3c90 0x7f0b4c0a6150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6810c400 0x7f0b68101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b68103900 0x7f0b68100c30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 --2- 172.21.15.18:0/3776670764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b68102fe0 0x7f0b681026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.743+0000 7f0b6da00640 1 -- 172.21.15.18:0/3776670764 >> 172.21.15.18:0/3776670764 conn(0x7f0b680fc7e0 msgr2=0x7f0b680fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.744+0000 7f0b6da00640 1 -- 172.21.15.18:0/3776670764 shutdown_connections 2024-04-16T17:15:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:32.744+0000 7f0b6da00640 1 -- 172.21.15.18:0/3776670764 wait complete. 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 -- 172.21.15.18:0/2637766079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d4074cd0 msgr2=0x7f28d410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 --2- 172.21.15.18:0/2637766079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d4074cd0 0x7f28d410e180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009900 tx=0x7f28c002bd10 comp rx=0 tx=0).stop 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 -- 172.21.15.18:0/2637766079 shutdown_connections 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 --2- 172.21.15.18:0/2637766079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d410e6c0 0x7f28d4110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 --2- 172.21.15.18:0/2637766079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d4074cd0 0x7f28d410e180 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 --2- 172.21.15.18:0/2637766079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d40743b0 0x7f28d4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 -- 172.21.15.18:0/2637766079 >> 172.21.15.18:0/2637766079 conn(0x7f28d40fd950 msgr2=0x7f28d40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 -- 172.21.15.18:0/2637766079 shutdown_connections 2024-04-16T17:15:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.820+0000 7f28da200640 1 -- 172.21.15.18:0/2637766079 wait complete. 2024-04-16T17:15:32.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.821+0000 7f28da200640 1 Processor -- start 2024-04-16T17:15:32.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.821+0000 7f28da200640 1 -- start start 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28da200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d40743b0 0x7f28d41a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28da200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d4074cd0 0x7f28d41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28da200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d410e6c0 0x7f28d419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28da200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4114530 con 0x7f28d40743b0 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28da200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28d41143b0 con 0x7f28d4074cd0 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28da200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28d41146b0 con 0x7f28d410e6c0 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28d8e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d40743b0 0x7f28d41a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d4074cd0 0x7f28d41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d4074cd0 0x7f28d41a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57372/0 (socket says 172.21.15.18:57372) 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.822+0000 7f28d3e00640 1 -- 172.21.15.18:0/3621228506 learned_addr learned my addr 172.21.15.18:0/3621228506 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.823+0000 7f28d3e00640 1 -- 172.21.15.18:0/3621228506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d410e6c0 msgr2=0x7f28d419a850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.823+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d410e6c0 0x7f28d419a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.823+0000 7f28d3e00640 1 -- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d40743b0 msgr2=0x7f28d41a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.823+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d40743b0 0x7f28d41a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.823+0000 7f28d3e00640 1 -- 172.21.15.18:0/3621228506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d419b020 con 0x7f28d4074cd0 2024-04-16T17:15:32.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.823+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d40743b0 0x7f28d41a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:32.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.830+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d4074cd0 0x7f28d41a0c20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f28c002c320 tx=0x7f28c002c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.830+0000 7f28d1600640 1 -- 172.21.15.18:0/3621228506 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0042020 con 0x7f28d4074cd0 2024-04-16T17:15:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.830+0000 7f28da200640 1 -- 172.21.15.18:0/3621228506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d419b2b0 con 0x7f28d4074cd0 2024-04-16T17:15:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.830+0000 7f28da200640 1 -- 172.21.15.18:0/3621228506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d41a7530 con 0x7f28d4074cd0 2024-04-16T17:15:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.830+0000 7f28d1600640 1 -- 172.21.15.18:0/3621228506 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c0004c80 con 0x7f28d4074cd0 2024-04-16T17:15:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.830+0000 7f28d1600640 1 -- 172.21.15.18:0/3621228506 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0003830 con 0x7f28d4074cd0 2024-04-16T17:15:32.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.833+0000 7f28de5c8740 1 Processor -- start 2024-04-16T17:15:32.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28de5c8740 1 -- start start 2024-04-16T17:15:32.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a3a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a3a830 0x5631f8a42ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a43410 0x5631f8a45800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28de5c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5631f899f020 con 0x5631f8a3a830 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28de5c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5631f8945460 con 0x5631f8a39f10 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28de5c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5631f899afe0 con 0x5631f8a43410 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28d8e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a3a830 0x5631f8a42ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28d8e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a3a830 0x5631f8a42ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49354/0 (socket says 172.21.15.18:49354) 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28d8e00640 1 -- 172.21.15.18:0/287441873 learned_addr learned my addr 172.21.15.18:0/287441873 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28d9800640 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a3a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.834+0000 7f28d1600640 1 -- 172.21.15.18:0/3621228506 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f28c0045020 con 0x7f28d4074cd0 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28d9800640 1 -- 172.21.15.18:0/287441873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a43410 msgr2=0x5631f8a45800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28d9800640 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a43410 0x5631f8a45800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28d9800640 1 -- 172.21.15.18:0/287441873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a3a830 msgr2=0x5631f8a42ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28d9800640 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a3a830 0x5631f8a42ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28d9800640 1 -- 172.21.15.18:0/287441873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631f894d040 con 0x5631f8a39f10 2024-04-16T17:15:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28d8e00640 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a3a830 0x5631f8a42ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28d9800640 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a3a2f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009ce0 tx=0x7f28cc00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef42cb305753834 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28c6000640 1 -- 172.21.15.18:0/287441873 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc018070 con 0x5631f8a39f10 2024-04-16T17:15:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28c6000640 1 -- 172.21.15.18:0/287441873 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc011a00 con 0x5631f8a39f10 2024-04-16T17:15:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.835+0000 7f28c6000640 1 -- 172.21.15.18:0/287441873 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc013a80 con 0x5631f8a39f10 2024-04-16T17:15:32.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28d1600640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28bc0a3da0 0x7f28bc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28d1600640 1 -- 172.21.15.18:0/3621228506 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f28c00f2290 con 0x7f28d4074cd0 2024-04-16T17:15:32.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 -- 172.21.15.18:0/287441873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 msgr2=0x5631f8a3a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a3a2f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009ce0 tx=0x7f28cc00b7e0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28bc0a3da0 0x7f28bc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 -- 172.21.15.18:0/287441873 shutdown_connections 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a43410 0x5631f8a45800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a3a830 0x5631f8a42ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 --2- 172.21.15.18:0/287441873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a3a2f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 -- 172.21.15.18:0/287441873 >> 172.21.15.18:0/287441873 conn(0x5631f8a34640 msgr2=0x5631f8a34a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 -- 172.21.15.18:0/287441873 shutdown_connections 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.836+0000 7f28de5c8740 1 -- 172.21.15.18:0/287441873 wait complete. 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.837+0000 7f28de5c8740 1 -- start start 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.837+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28bc0a3da0 0x7f28bc0a6260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f28c80092d0 tx=0x7f28c80090e0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.837+0000 7f28de5c8740 1 Processor -- start 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 0x5631f8a40f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28de5c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5631f8a46200 con 0x5631f8a43410 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28de5c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5631f8958830 con 0x5631f8a39f10 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28de5c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5631f8a46380 con 0x5631f8a3a830 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28de5c8740 10 client.0 _ll_register_callbacks cb 0x5631f8960150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56058/0 (socket says 172.21.15.18:56058) 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d3e00640 1 -- 172.21.15.18:0/3230793811 learned_addr learned my addr 172.21.15.18:0/3230793811 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d9800640 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 0x5631f8a40f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d3e00640 1 -- 172.21.15.18:0/3230793811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 msgr2=0x5631f8a404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d3e00640 1 -- 172.21.15.18:0/3230793811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 msgr2=0x5631f8a40f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 0x5631f8a40f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.838+0000 7f28d3e00640 1 -- 172.21.15.18:0/3230793811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631f895c920 con 0x5631f8a3a830 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a409e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f28c002ce90 tx=0x7f28c00bd8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccfc99040354a9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28baa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28baa00640 1 -- 172.21.15.18:0/3230793811 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00d7060 con 0x5631f8a3a830 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28baa00640 1 -- 172.21.15.18:0/3230793811 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f28c00d7200 con 0x5631f8a3a830 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28de5c8740 1 -- 172.21.15.18:0/3230793811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28cc011a00 con 0x5631f8a3a830 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28baa00640 1 -- 172.21.15.18:0/3230793811 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f28c00ef9f0 con 0x5631f8a3a830 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28de5c8740 10 client.28363 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28de5c8740 1 -- 172.21.15.18:0/3230793811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5631f892a7f0 con 0x5631f8a3a830 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.839+0000 7f28baa00640 1 -- 172.21.15.18:0/3230793811 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc011a00 con 0x5631f8a3a830 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.840+0000 7f28baa00640 1 -- 172.21.15.18:0/3230793811 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f28c00bd320 con 0x5631f8a3a830 2024-04-16T17:15:32.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.840+0000 7f28de5c8740 10 client.28363 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:32.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.840+0000 7f28de5c8740 10 client.28363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:15:32.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.840+0000 7f28de5c8740 20 client.28363 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:32.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.840+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5631f8a538f0 0x5631f8a55d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.840+0000 7f28de5c8740 4 client.28363 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:32.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.840+0000 7f28de5c8740 1 -- 172.21.15.18:0/3230793811 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631f89aca40 con 0x5631f8a538f0 2024-04-16T17:15:32.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.840+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5631f8a538f0 0x5631f8a55d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.841+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5631f8a538f0 0x5631f8a55d10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24fa3f5c3ec30dd2 server_cookie=b6c7872cefdbe2af in_seq=0 out_seq=0 2024-04-16T17:15:32.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.841+0000 7f28baa00640 10 client.28363 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:15:32.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.841+0000 7f28baa00640 1 -- 172.21.15.18:0/3230793811 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631f89aca40 con 0x5631f8a538f0 2024-04-16T17:15:32.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.841+0000 7f28baa00640 10 client.28363 handle_command_reply: tid=0 2024-04-16T17:15:32.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.841+0000 7f28de5c8740 1 client.28363 shutdown 2024-04-16T17:15:32.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 -- 172.21.15.18:0/3230793811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 msgr2=0x5631f8a409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a409e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f28c002ce90 tx=0x7f28c00bd8b0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 -- 172.21.15.18:0/3230793811 shutdown_connections 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5631f8a538f0 0x5631f8a55d10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 0x5631f8a40f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a409e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3230793811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 -- 172.21.15.18:0/3230793811 >> 172.21.15.18:0/3230793811 conn(0x5631f8a34640 msgr2=0x5631f8a36390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28baa00640 0 client.28363 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 -- 172.21.15.18:0/3230793811 shutdown_connections 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 1 -- 172.21.15.18:0/3230793811 wait complete. 2024-04-16T17:15:32.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.842+0000 7f28de5c8740 20 client.28363 trim_cache size 0 max 16384 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.857+0000 7f28de5c8740 1 Processor -- start 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28de5c8740 1 -- start start 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a39f10 0x5631f8a4f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a33580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a43410 0x5631f8a40d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28de5c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4012980 con 0x5631f8a39f10 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28de5c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4010250 con 0x5631f8a43410 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28de5c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400e0c0 con 0x5631f8a3a830 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28d8e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a39f10 0x5631f8a4f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a33580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a33580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56074/0 (socket says 172.21.15.18:56074) 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28d3e00640 1 -- 172.21.15.18:0/942982458 learned_addr learned my addr 172.21.15.18:0/942982458 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28d9800640 1 --2- 172.21.15.18:0/942982458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a43410 0x5631f8a40d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.858+0000 7f28d3e00640 1 -- 172.21.15.18:0/942982458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a43410 msgr2=0x5631f8a40d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.859+0000 7f28d3e00640 1 --2- 172.21.15.18:0/942982458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a43410 0x5631f8a40d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.859+0000 7f28d3e00640 1 -- 172.21.15.18:0/942982458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a39f10 msgr2=0x5631f8a4f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.859+0000 7f28d3e00640 1 --2- 172.21.15.18:0/942982458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a39f10 0x5631f8a4f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.859+0000 7f28d3e00640 1 -- 172.21.15.18:0/942982458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631f86b13f0 con 0x5631f8a3a830 2024-04-16T17:15:32.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.859+0000 7f28d3e00640 1 --2- 172.21.15.18:0/942982458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a33580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f28c002cdf0 tx=0x7f28c00be050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec174b622a5e3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.859+0000 7f28c5600640 1 -- 172.21.15.18:0/942982458 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00bda10 con 0x5631f8a3a830 2024-04-16T17:15:32.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.859+0000 7f28c5600640 1 -- 172.21.15.18:0/942982458 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c00c2280 con 0x5631f8a3a830 2024-04-16T17:15:32.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.859+0000 7f28c5600640 1 -- 172.21.15.18:0/942982458 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00ef7f0 con 0x5631f8a3a830 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 -- 172.21.15.18:0/942982458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 msgr2=0x5631f8a33580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 --2- 172.21.15.18:0/942982458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a33580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f28c002cdf0 tx=0x7f28c00be050 comp rx=0 tx=0).stop 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 -- 172.21.15.18:0/942982458 shutdown_connections 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 --2- 172.21.15.18:0/942982458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a43410 0x5631f8a40d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 --2- 172.21.15.18:0/942982458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a33580 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 --2- 172.21.15.18:0/942982458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a39f10 0x5631f8a4f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 -- 172.21.15.18:0/942982458 >> 172.21.15.18:0/942982458 conn(0x5631f8a34640 msgr2=0x5631f8a32e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 -- 172.21.15.18:0/942982458 shutdown_connections 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 -- 172.21.15.18:0/942982458 wait complete. 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.860+0000 7f28de5c8740 1 -- start start 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28de5c8740 1 Processor -- start 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28de5c8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 0x5631f8a3d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28de5c8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400e710 con 0x5631f8a43410 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28de5c8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400fb80 con 0x5631f8a39f10 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28de5c8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400ede0 con 0x5631f8a3a830 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28de5c8740 10 client.0 _ll_register_callbacks cb 0x7f28c0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28d9800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 0x5631f8a3d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28d8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a3ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56082/0 (socket says 172.21.15.18:56082) 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.861+0000 7f28d3e00640 1 -- 172.21.15.18:0/3055611275 learned_addr learned my addr 172.21.15.18:0/3055611275 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28d8e00640 1 -- 172.21.15.18:0/3055611275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 msgr2=0x5631f8a3ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a3ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28d8e00640 1 -- 172.21.15.18:0/3055611275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 msgr2=0x5631f8a3d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 0x5631f8a3d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28d8e00640 1 -- 172.21.15.18:0/3055611275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631f877b7d0 con 0x5631f8a39f10 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28d8e00640 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a4ba10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x5631f8a56570 tx=0x7f28c8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b860ba48cb4e8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28baa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.862+0000 7f28baa00640 1 -- 172.21.15.18:0/3055611275 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8031070 con 0x5631f8a39f10 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.863+0000 7f28baa00640 1 -- 172.21.15.18:0/3055611275 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f28c802a240 con 0x5631f8a39f10 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.863+0000 7f28de5c8740 1 -- 172.21.15.18:0/3055611275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c00c2280 con 0x5631f8a39f10 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.863+0000 7f28baa00640 1 -- 172.21.15.18:0/3055611275 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f28c802a440 con 0x5631f8a39f10 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.863+0000 7f28de5c8740 10 client.28508 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:32.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.863+0000 7f28de5c8740 1 -- 172.21.15.18:0/3055611275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5631f8ab9d60 con 0x5631f8a39f10 2024-04-16T17:15:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.863+0000 7f28baa00640 1 -- 172.21.15.18:0/3055611275 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c00c2280 con 0x5631f8a39f10 2024-04-16T17:15:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.864+0000 7f28baa00640 1 -- 172.21.15.18:0/3055611275 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f28c8020040 con 0x5631f8a39f10 2024-04-16T17:15:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.864+0000 7f28de5c8740 10 client.28508 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.864+0000 7f28de5c8740 10 client.28508 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.864+0000 7f28de5c8740 20 client.28508 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.864+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5631f8aba480 0x5631f8abc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.864+0000 7f28de5c8740 4 client.28508 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.864+0000 7f28de5c8740 1 -- 172.21.15.18:0/3055611275 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631f89a68b0 con 0x5631f8aba480 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.864+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5631f8aba480 0x5631f8abc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28baa00640 10 client.28508 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28d3e00640 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5631f8aba480 0x5631f8abc870 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48cc06faf28dc6d3 server_cookie=cc1971fd84b30d9f in_seq=0 out_seq=0 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28baa00640 1 -- 172.21.15.18:0/3055611275 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631f89a68b0 con 0x5631f8aba480 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28baa00640 10 client.28508 handle_command_reply: tid=0 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 client.28508 shutdown 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 -- 172.21.15.18:0/3055611275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 msgr2=0x5631f8a4ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a4ba10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x5631f8a56570 tx=0x7f28c8014000 comp rx=0 tx=0).stop 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 -- 172.21.15.18:0/3055611275 shutdown_connections 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5631f8aba480 0x5631f8abc870 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631f8a43410 0x5631f8a3d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631f8a3a830 0x5631f8a3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 --2- 172.21.15.18:0/3055611275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f8a39f10 0x5631f8a4ba10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28baa00640 0 client.28508 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:15:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 -- 172.21.15.18:0/3055611275 >> 172.21.15.18:0/3055611275 conn(0x5631f8a34640 msgr2=0x5631f8a33700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 -- 172.21.15.18:0/3055611275 shutdown_connections 2024-04-16T17:15:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.865+0000 7f28de5c8740 1 -- 172.21.15.18:0/3055611275 wait complete. 2024-04-16T17:15:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28de5c8740 20 client.28508 trim_cache size 0 max 16384 2024-04-16T17:15:32.870 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 -- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28bc0a3da0 msgr2=0x7f28bc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28bc0a3da0 0x7f28bc0a6260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f28c80092d0 tx=0x7f28c80090e0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 -- 172.21.15.18:0/3621228506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d4074cd0 msgr2=0x7f28d41a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d4074cd0 0x7f28d41a0c20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f28c002c320 tx=0x7f28c002c740 comp rx=0 tx=0).stop 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 -- 172.21.15.18:0/3621228506 shutdown_connections 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f28bc0a3da0 0x7f28bc0a6260 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d410e6c0 0x7f28d419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d4074cd0 0x7f28d41a0c20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 --2- 172.21.15.18:0/3621228506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28d40743b0 0x7f28d41a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:32.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.866+0000 7f28da200640 1 -- 172.21.15.18:0/3621228506 >> 172.21.15.18:0/3621228506 conn(0x7f28d40fd950 msgr2=0x7f28d40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:32.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.867+0000 7f28da200640 1 -- 172.21.15.18:0/3621228506 shutdown_connections 2024-04-16T17:15:32.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:32.867+0000 7f28da200640 1 -- 172.21.15.18:0/3621228506 wait complete. 2024-04-16T17:15:32.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:15:32.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:32 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4088632158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:33.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:32 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4088632158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.127+0000 7f00bbe00640 1 -- 172.21.15.18:0/1910501002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 msgr2=0x7f00bc101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.127+0000 7f00bbe00640 1 --2- 172.21.15.18:0/1910501002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 0x7f00bc101340 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009900 tx=0x7f00ac02bc40 comp rx=0 tx=0).stop 2024-04-16T17:15:33.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.128+0000 7f00bbe00640 1 -- 172.21.15.18:0/1910501002 shutdown_connections 2024-04-16T17:15:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.128+0000 7f00bbe00640 1 --2- 172.21.15.18:0/1910501002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc106030 0x7f00bc10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.128+0000 7f00bbe00640 1 --2- 172.21.15.18:0/1910501002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc101910 0x7f00bc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.128+0000 7f00bbe00640 1 --2- 172.21.15.18:0/1910501002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 0x7f00bc101340 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.128+0000 7f00bbe00640 1 -- 172.21.15.18:0/1910501002 >> 172.21.15.18:0/1910501002 conn(0x7f00bc077ab0 msgr2=0x7f00bc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.128+0000 7f00bbe00640 1 -- 172.21.15.18:0/1910501002 shutdown_connections 2024-04-16T17:15:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.128+0000 7f00bbe00640 1 -- 172.21.15.18:0/1910501002 wait complete. 2024-04-16T17:15:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.129+0000 7f00bbe00640 1 Processor -- start 2024-04-16T17:15:33.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.130+0000 7f00bbe00640 1 -- start start 2024-04-16T17:15:33.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.130+0000 7f00bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 0x7f00bc10e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.130+0000 7f00bbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc106030 0x7f00bc10ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.130+0000 7f00bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc10f010 0x7f00bc1a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.130+0000 7f00bbe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc112290 con 0x7f00bc106030 2024-04-16T17:15:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.130+0000 7f00bbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc112110 con 0x7f00bc100f60 2024-04-16T17:15:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.130+0000 7f00bbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc112410 con 0x7f00bc10f010 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00baa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 0x7f00bc10e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc106030 0x7f00bc10ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00baa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 0x7f00bc10e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57430/0 (socket says 172.21.15.18:57430) 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00baa00640 1 -- 172.21.15.18:0/3991066754 learned_addr learned my addr 172.21.15.18:0/3991066754 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00bb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc10f010 0x7f00bc1a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00baa00640 1 -- 172.21.15.18:0/3991066754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc10f010 msgr2=0x7f00bc1a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00baa00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc10f010 0x7f00bc1a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00baa00640 1 -- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc106030 msgr2=0x7f00bc10ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00baa00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc106030 0x7f00bc10ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.131+0000 7f00baa00640 1 -- 172.21.15.18:0/3991066754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc1a7000 con 0x7f00bc100f60 2024-04-16T17:15:33.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.132+0000 7f00bb400640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc10f010 0x7f00bc1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:33.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.132+0000 7f00baa00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 0x7f00bc10e590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000c00 tx=0x7f00ac02c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.132+0000 7f00c0a00640 1 -- 172.21.15.18:0/3991066754 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac041070 con 0x7f00bc100f60 2024-04-16T17:15:33.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.133+0000 7f00c0a00640 1 -- 172.21.15.18:0/3991066754 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ac0054e0 con 0x7f00bc100f60 2024-04-16T17:15:33.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.133+0000 7f00bbe00640 1 -- 172.21.15.18:0/3991066754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1a7290 con 0x7f00bc100f60 2024-04-16T17:15:33.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.133+0000 7f00c0a00640 1 -- 172.21.15.18:0/3991066754 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac034440 con 0x7f00bc100f60 2024-04-16T17:15:33.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.133+0000 7f00bbe00640 1 -- 172.21.15.18:0/3991066754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a7780 con 0x7f00bc100f60 2024-04-16T17:15:33.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.137+0000 7f00c0a00640 1 -- 172.21.15.18:0/3991066754 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f00ac044070 con 0x7f00bc100f60 2024-04-16T17:15:33.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.139+0000 7f00c4bd8740 1 Processor -- start 2024-04-16T17:15:33.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.139+0000 7f00c4bd8740 1 -- start start 2024-04-16T17:15:33.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.140+0000 7f00c0a00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00a00a3ea0 0x7f00a00a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.140+0000 7f00c0a00640 1 -- 172.21.15.18:0/3991066754 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f00ac0f2640 con 0x7f00bc100f60 2024-04-16T17:15:33.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.140+0000 7f00bb400640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00a00a3ea0 0x7f00a00a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728bb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728bb830 0x563c728c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728c4410 0x563c728c6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563c72820020 con 0x563c728baf10 2024-04-16T17:15:33.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563c727c6460 con 0x563c728c4410 2024-04-16T17:15:33.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563c7281bfe0 con 0x563c728bb830 2024-04-16T17:15:33.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728bb830 0x563c728c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00baa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728bb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728bb830 0x563c728c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56096/0 (socket says 172.21.15.18:56096) 2024-04-16T17:15:33.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00b3e00640 1 -- 172.21.15.18:0/456797461 learned_addr learned my addr 172.21.15.18:0/456797461 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.141+0000 7f00b3e00640 1 -- 172.21.15.18:0/456797461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728c4410 msgr2=0x563c728c6800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.142+0000 7f00bb400640 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728c4410 0x563c728c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.142+0000 7f00b3e00640 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728c4410 0x563c728c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.142+0000 7f00b3e00640 1 -- 172.21.15.18:0/456797461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 msgr2=0x563c728bb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.142+0000 7f00b3e00640 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728bb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.142+0000 7f00b3e00640 1 -- 172.21.15.18:0/456797461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c727ce040 con 0x563c728bb830 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.142+0000 7f00baa00640 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728bb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.142+0000 7f00bb400640 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728c4410 0x563c728c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.142+0000 7f00b3e00640 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728bb830 0x563c728c3ed0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f00a400ac70 tx=0x7f00a400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc7fb18bd4c7cb9b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.143+0000 7f00abe00640 1 -- 172.21.15.18:0/456797461 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a401d070 con 0x563c728bb830 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.143+0000 7f00abe00640 1 -- 172.21.15.18:0/456797461 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a4018b20 con 0x563c728bb830 2024-04-16T17:15:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.143+0000 7f00abe00640 1 -- 172.21.15.18:0/456797461 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4010bf0 con 0x563c728bb830 2024-04-16T17:15:33.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 -- 172.21.15.18:0/456797461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728bb830 msgr2=0x563c728c3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00bb400640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00a00a3ea0 0x7f00a00a6360 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x563c728b67d0 tx=0x7f00b4009650 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728bb830 0x563c728c3ed0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f00a400ac70 tx=0x7f00a400f790 comp rx=0 tx=0).stop 2024-04-16T17:15:33.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 -- 172.21.15.18:0/456797461 shutdown_connections 2024-04-16T17:15:33.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728c4410 0x563c728c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728bb830 0x563c728c3ed0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/456797461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728bb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 -- 172.21.15.18:0/456797461 >> 172.21.15.18:0/456797461 conn(0x563c728b5640 msgr2=0x563c728b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 -- 172.21.15.18:0/456797461 shutdown_connections 2024-04-16T17:15:33.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.144+0000 7f00c4bd8740 1 -- 172.21.15.18:0/456797461 wait complete. 2024-04-16T17:15:33.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.145+0000 7f00c4bd8740 1 -- start start 2024-04-16T17:15:33.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.145+0000 7f00c4bd8740 1 Processor -- start 2024-04-16T17:15:33.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563c728c7200 con 0x563c728baf10 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563c727d9830 con 0x563c728bb830 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563c728c7380 con 0x563c728c4410 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00c4bd8740 10 client.0 _ll_register_callbacks cb 0x563c727e1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728c1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57446/0 (socket says 172.21.15.18:57446) 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00b3e00640 1 -- 172.21.15.18:0/2926372304 learned_addr learned my addr 172.21.15.18:0/2926372304 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00baa00640 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00bb400640 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00b3e00640 1 -- 172.21.15.18:0/2926372304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 msgr2=0x563c728c1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00b3e00640 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.146+0000 7f00b3e00640 1 -- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 msgr2=0x563c728c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.147+0000 7f00b3e00640 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.147+0000 7f00b3e00640 1 -- 172.21.15.18:0/2926372304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c727dd920 con 0x563c728bb830 2024-04-16T17:15:33.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.147+0000 7f00baa00640 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.147+0000 7f00b3e00640 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728c1a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f00a400a840 tx=0x7f00a4010be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbab150deb3da173 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.147+0000 7f00a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.147+0000 7f00a8c00640 1 -- 172.21.15.18:0/2926372304 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a401d050 con 0x563c728bb830 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.147+0000 7f00a8c00640 1 -- 172.21.15.18:0/2926372304 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f00a4018640 con 0x563c728bb830 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.147+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2926372304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a4018b20 con 0x563c728bb830 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.148+0000 7f00a8c00640 1 -- 172.21.15.18:0/2926372304 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f00a4018840 con 0x563c728bb830 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.148+0000 7f00c4bd8740 10 client.28517 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.148+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2926372304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c727ab7f0 con 0x563c728bb830 2024-04-16T17:15:33.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.148+0000 7f00a8c00640 1 -- 172.21.15.18:0/2926372304 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a4018b20 con 0x563c728bb830 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.149+0000 7f00a8c00640 1 -- 172.21.15.18:0/2926372304 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f00a4004750 con 0x563c728bb830 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.149+0000 7f00c4bd8740 10 client.28517 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.149+0000 7f00c4bd8740 10 client.28517 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.149+0000 7f00c4bd8740 20 client.28517 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.149+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x563c728d4730 0x563c728d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.149+0000 7f00c4bd8740 4 client.28517 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.149+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2926372304 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c7282da40 con 0x563c728d4730 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.149+0000 7f00baa00640 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x563c728d4730 0x563c728d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.150+0000 7f00baa00640 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x563c728d4730 0x563c728d6b20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ddf2d98b6da94a6 server_cookie=96c17c7f16a55fe4 in_seq=0 out_seq=0 2024-04-16T17:15:33.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.150+0000 7f00a8c00640 10 client.28517 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00a8c00640 1 -- 172.21.15.18:0/2926372304 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c7282da40 con 0x563c728d4730 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00a8c00640 10 client.28517 handle_command_reply: tid=0 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 client.28517 shutdown 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2926372304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 msgr2=0x563c728c1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728c1a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f00a400a840 tx=0x7f00a4010be0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2926372304 shutdown_connections 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x563c728d4730 0x563c728d6b20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00a8c00640 0 client.28517 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728c1a00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2926372304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2926372304 >> 172.21.15.18:0/2926372304 conn(0x563c728b5640 msgr2=0x563c728b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2926372304 shutdown_connections 2024-04-16T17:15:33.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2926372304 wait complete. 2024-04-16T17:15:33.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.151+0000 7f00c4bd8740 20 client.28517 trim_cache size 0 max 16384 2024-04-16T17:15:33.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.180+0000 7f00c4bd8740 1 Processor -- start 2024-04-16T17:15:33.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.180+0000 7f00c4bd8740 1 -- start start 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b3f70 con 0x563c728baf10 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b1840 con 0x563c728bb830 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1af650 con 0x563c728c4410 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00bb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00bb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56116/0 (socket says 172.21.15.18:56116) 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00bb400640 1 -- 172.21.15.18:0/199522267 learned_addr learned my addr 172.21.15.18:0/199522267 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57448/0 (socket says 172.21.15.18:57448) 2024-04-16T17:15:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00baa00640 1 --2- 172.21.15.18:0/199522267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00bb400640 1 -- 172.21.15.18:0/199522267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 msgr2=0x563c728d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.181+0000 7f00bb400640 1 --2- 172.21.15.18:0/199522267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.182+0000 7f00bb400640 1 -- 172.21.15.18:0/199522267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 msgr2=0x563c728c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.182+0000 7f00bb400640 1 --2- 172.21.15.18:0/199522267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.182+0000 7f00bb400640 1 -- 172.21.15.18:0/199522267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c725323f0 con 0x563c728c4410 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.182+0000 7f00bb400640 1 --2- 172.21.15.18:0/199522267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c0620 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f00b40128c0 tx=0x7f00b400f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6078a331a1061f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.182+0000 7f00ab400640 1 -- 172.21.15.18:0/199522267 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4029070 con 0x563c728c4410 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.182+0000 7f00ab400640 1 -- 172.21.15.18:0/199522267 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b4024400 con 0x563c728c4410 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.182+0000 7f00ab400640 1 -- 172.21.15.18:0/199522267 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4019e60 con 0x563c728c4410 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.183+0000 7f00c4bd8740 1 -- 172.21.15.18:0/199522267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 msgr2=0x563c728c0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.183+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/199522267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c0620 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f00b40128c0 tx=0x7f00b400f000 comp rx=0 tx=0).stop 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.183+0000 7f00c4bd8740 1 -- 172.21.15.18:0/199522267 shutdown_connections 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.183+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/199522267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728c4410 0x563c728c0620 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.183+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/199522267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.183+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/199522267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728baf10 0x563c728c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.183+0000 7f00c4bd8740 1 -- 172.21.15.18:0/199522267 >> 172.21.15.18:0/199522267 conn(0x563c728b5640 msgr2=0x563c728c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.183+0000 7f00c4bd8740 1 -- 172.21.15.18:0/199522267 shutdown_connections 2024-04-16T17:15:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.184+0000 7f00c4bd8740 1 -- 172.21.15.18:0/199522267 wait complete. 2024-04-16T17:15:33.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.184+0000 7f00c4bd8740 1 -- start start 2024-04-16T17:15:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.184+0000 7f00c4bd8740 1 Processor -- start 2024-04-16T17:15:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728baf10 0x563c728be080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00c4bd8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728c4410 0x563c728beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1afd00 con 0x563c728c4410 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b1170 con 0x563c728bb830 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00c4bd8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b03d0 con 0x563c728baf10 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00c4bd8740 10 client.0 _ll_register_callbacks cb 0x7f00a401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728baf10 0x563c728be080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728baf10 0x563c728be080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56118/0 (socket says 172.21.15.18:56118) 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00baa00640 1 -- 172.21.15.18:0/2712580951 learned_addr learned my addr 172.21.15.18:0/2712580951 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00b3e00640 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00bb400640 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728c4410 0x563c728beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00baa00640 1 -- 172.21.15.18:0/2712580951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 msgr2=0x563c728be5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00baa00640 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728be5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.185+0000 7f00baa00640 1 -- 172.21.15.18:0/2712580951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728c4410 msgr2=0x563c728beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00baa00640 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728c4410 0x563c728beb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00baa00640 1 -- 172.21.15.18:0/2712580951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c725fc7d0 con 0x563c728baf10 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00baa00640 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728baf10 0x563c728be080 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0be070 tx=0x7f00ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23eae4cd14fbb266 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00a8c00640 1 -- 172.21.15.18:0/2712580951 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac04b040 con 0x563c728baf10 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00a8c00640 1 -- 172.21.15.18:0/2712580951 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f00ac0c6ac0 con 0x563c728baf10 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2712580951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b4024400 con 0x563c728baf10 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00a8c00640 1 -- 172.21.15.18:0/2712580951 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f00ac005690 con 0x563c728baf10 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00c4bd8740 10 client.28384 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.186+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2712580951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c7293ab90 con 0x563c728baf10 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00a8c00640 1 -- 172.21.15.18:0/2712580951 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b4024400 con 0x563c728baf10 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00a8c00640 1 -- 172.21.15.18:0/2712580951 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f00a4019ce0 con 0x563c728baf10 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00c4bd8740 10 client.28384 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00c4bd8740 10 client.28384 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00c4bd8740 20 client.28384 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x563c7293b260 0x563c7293d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00c4bd8740 4 client.28384 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:33.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2712580951 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c728278b0 con 0x563c7293b260 2024-04-16T17:15:33.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.187+0000 7f00b3e00640 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x563c7293b260 0x563c7293d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.188+0000 7f00b3e00640 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x563c7293b260 0x563c7293d680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d53d8e7023b99a server_cookie=6cdd2a31a4873eb8 in_seq=0 out_seq=0 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.188+0000 7f00a8c00640 10 client.28384 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.188+0000 7f00a8c00640 1 -- 172.21.15.18:0/2712580951 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c728278b0 con 0x563c7293b260 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.188+0000 7f00a8c00640 10 client.28384 handle_command_reply: tid=0 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.188+0000 7f00c4bd8740 1 client.28384 shutdown 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.188+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2712580951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728baf10 msgr2=0x563c728be080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.188+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728baf10 0x563c728be080 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0be070 tx=0x7f00ac009260 comp rx=0 tx=0).stop 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.188+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2712580951 shutdown_connections 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x563c7293b260 0x563c7293d680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563c728c4410 0x563c728beb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c728bb830 0x563c728be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00a8c00640 0 client.28384 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:15:33.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00c4bd8740 1 --2- 172.21.15.18:0/2712580951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563c728baf10 0x563c728be080 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2712580951 >> 172.21.15.18:0/2712580951 conn(0x563c728b5640 msgr2=0x563c728b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2712580951 shutdown_connections 2024-04-16T17:15:33.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00c4bd8740 1 -- 172.21.15.18:0/2712580951 wait complete. 2024-04-16T17:15:33.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.189+0000 7f00c4bd8740 20 client.28384 trim_cache size 0 max 16384 2024-04-16T17:15:33.192 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 -- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00a00a3ea0 msgr2=0x7f00a00a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00a00a3ea0 0x7f00a00a6360 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x563c728b67d0 tx=0x7f00b4009650 comp rx=0 tx=0).stop 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 -- 172.21.15.18:0/3991066754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 msgr2=0x7f00bc10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 0x7f00bc10e590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000c00 tx=0x7f00ac02c760 comp rx=0 tx=0).stop 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 -- 172.21.15.18:0/3991066754 shutdown_connections 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00a00a3ea0 0x7f00a00a6360 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc10f010 0x7f00bc1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00bc106030 0x7f00bc10ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 --2- 172.21.15.18:0/3991066754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00bc100f60 0x7f00bc10e590 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 -- 172.21.15.18:0/3991066754 >> 172.21.15.18:0/3991066754 conn(0x7f00bc077ab0 msgr2=0x7f00bc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 -- 172.21.15.18:0/3991066754 shutdown_connections 2024-04-16T17:15:33.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.190+0000 7f00bbe00640 1 -- 172.21.15.18:0/3991066754 wait complete. 2024-04-16T17:15:33.211 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:15:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.403+0000 7fc5af200640 1 -- 172.21.15.18:0/3578361781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8102040 msgr2=0x7fc5a8102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.403+0000 7fc5af200640 1 --2- 172.21.15.18:0/3578361781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8102040 0x7fc5a8102420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009900 tx=0x7fc59c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.404+0000 7fc5af200640 1 -- 172.21.15.18:0/3578361781 shutdown_connections 2024-04-16T17:15:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.404+0000 7fc5af200640 1 --2- 172.21.15.18:0/3578361781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a8107110 0x7fc5a810ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.404+0000 7fc5af200640 1 --2- 172.21.15.18:0/3578361781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5a81029f0 0x7fc5a8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.404+0000 7fc5af200640 1 --2- 172.21.15.18:0/3578361781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8102040 0x7fc5a8102420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.404+0000 7fc5af200640 1 -- 172.21.15.18:0/3578361781 >> 172.21.15.18:0/3578361781 conn(0x7fc5a80fd8e0 msgr2=0x7fc5a80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.404+0000 7fc5af200640 1 -- 172.21.15.18:0/3578361781 shutdown_connections 2024-04-16T17:15:33.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.404+0000 7fc5af200640 1 -- 172.21.15.18:0/3578361781 wait complete. 2024-04-16T17:15:33.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5af200640 1 Processor -- start 2024-04-16T17:15:33.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5af200640 1 -- start start 2024-04-16T17:15:33.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5af200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5a8102040 0x7fc5a819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5af200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8107110 0x7fc5a819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5af200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a819f330 0x7fc5a819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5af200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a81121a0 con 0x7fc5a819f330 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5af200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a8112020 con 0x7fc5a8107110 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5af200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a8112320 con 0x7fc5a8102040 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.405+0000 7fc5ade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5a8102040 0x7fc5a819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8107110 0x7fc5a819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5a8102040 0x7fc5a819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56132/0 (socket says 172.21.15.18:56132) 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ade00640 1 -- 172.21.15.18:0/276634766 learned_addr learned my addr 172.21.15.18:0/276634766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ae800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a819f330 0x7fc5a819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8107110 0x7fc5a819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57474/0 (socket says 172.21.15.18:57474) 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ae800640 1 -- 172.21.15.18:0/276634766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5a8102040 msgr2=0x7fc5a819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ae800640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5a8102040 0x7fc5a819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ae800640 1 -- 172.21.15.18:0/276634766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8107110 msgr2=0x7fc5a819af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ae800640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8107110 0x7fc5a819af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ae800640 1 -- 172.21.15.18:0/276634766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a819fe20 con 0x7fc5a819f330 2024-04-16T17:15:33.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ad400640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8107110 0x7fc5a819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:33.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.406+0000 7fc5ade00640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5a8102040 0x7fc5a819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:33.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.407+0000 7fc5ae800640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a819f330 0x7fc5a819f7e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009fd0 tx=0x7fc5a400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.407+0000 7fc5a2a00640 1 -- 172.21.15.18:0/276634766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400ce20 con 0x7fc5a819f330 2024-04-16T17:15:33.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.407+0000 7fc5a2a00640 1 -- 172.21.15.18:0/276634766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a4005020 con 0x7fc5a819f330 2024-04-16T17:15:33.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.407+0000 7fc5a2a00640 1 -- 172.21.15.18:0/276634766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4010720 con 0x7fc5a819f330 2024-04-16T17:15:33.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.407+0000 7fc5af200640 1 -- 172.21.15.18:0/276634766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a81a0110 con 0x7fc5a819f330 2024-04-16T17:15:33.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.407+0000 7fc5af200640 1 -- 172.21.15.18:0/276634766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a81a7880 con 0x7fc5a819f330 2024-04-16T17:15:33.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.411+0000 7fc5a2a00640 1 -- 172.21.15.18:0/276634766 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5a4005540 con 0x7fc5a819f330 2024-04-16T17:15:33.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.412+0000 7fc5b35d2740 1 Processor -- start 2024-04-16T17:15:33.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.412+0000 7fc5b35d2740 1 -- start start 2024-04-16T17:15:33.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.412+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 0x55dda46b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dda4611020 con 0x55dda46b5410 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dda45b7460 con 0x55dda46abf10 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dda460cfe0 con 0x55dda46ac830 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57482/0 (socket says 172.21.15.18:57482) 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 -- 172.21.15.18:0/1714578646 learned_addr learned my addr 172.21.15.18:0/1714578646 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5a2a00640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5900a3da0 0x7fc5900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 0x55dda46b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ade00640 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5a2a00640 1 -- 172.21.15.18:0/276634766 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fc5a40d0e30 con 0x7fc5a819f330 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 -- 172.21.15.18:0/1714578646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 msgr2=0x55dda46b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 -- 172.21.15.18:0/1714578646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 msgr2=0x55dda46b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 0x55dda46b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.413+0000 7fc5ad400640 1 -- 172.21.15.18:0/1714578646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda45bf040 con 0x55dda46abf10 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.414+0000 7fc5ad400640 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46ac2f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc594011780 tx=0x7fc594005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2f2c980174e1775 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.414+0000 7fc5ae800640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5900a3da0 0x7fc5900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.414+0000 7fc59b000640 1 -- 172.21.15.18:0/1714578646 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594011050 con 0x55dda46abf10 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.414+0000 7fc59b000640 1 -- 172.21.15.18:0/1714578646 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5940111f0 con 0x55dda46abf10 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.414+0000 7fc59b000640 1 -- 172.21.15.18:0/1714578646 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594005260 con 0x55dda46abf10 2024-04-16T17:15:33.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1714578646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 msgr2=0x55dda46ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46ac2f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc594011780 tx=0x7fc594005850 comp rx=0 tx=0).stop 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5ae800640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5900a3da0 0x7fc5900a6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc5a401a040 tx=0x7fc5a4005f70 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1714578646 shutdown_connections 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 0x55dda46b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/1714578646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46ac2f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1714578646 >> 172.21.15.18:0/1714578646 conn(0x55dda46a6640 msgr2=0x55dda46a6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1714578646 shutdown_connections 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.415+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1714578646 wait complete. 2024-04-16T17:15:33.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.416+0000 7fc5b35d2740 1 -- start start 2024-04-16T17:15:33.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.416+0000 7fc5b35d2740 1 Processor -- start 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 0x55dda46b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dda46b8200 con 0x55dda46b5410 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dda45ca830 con 0x55dda46abf10 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dda46b8380 con 0x55dda46ac830 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5b35d2740 10 client.0 _ll_register_callbacks cb 0x55dda45d2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57484/0 (socket says 172.21.15.18:57484) 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5ad400640 1 -- 172.21.15.18:0/3711588707 learned_addr learned my addr 172.21.15.18:0/3711588707 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5ad400640 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 0x55dda46b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.417+0000 7fc5ade00640 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc5ad400640 1 -- 172.21.15.18:0/3711588707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 msgr2=0x55dda46b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc5ad400640 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc5ad400640 1 -- 172.21.15.18:0/3711588707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 msgr2=0x55dda46b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc5ad400640 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 0x55dda46b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc5ad400640 1 -- 172.21.15.18:0/3711588707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda45ce920 con 0x55dda46abf10 2024-04-16T17:15:33.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc5ade00640 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:15:33.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc5ad400640 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46b24d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc59400f790 tx=0x7fc59400fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d436a9145055c798 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc58f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:15:33.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc58f400640 1 -- 172.21.15.18:0/3711588707 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5940050e0 con 0x55dda46abf10 2024-04-16T17:15:33.420 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:31 smithi018 ceph-mon[31776]: pgmap v340: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-16T17:15:33.420 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:32 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4088632158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc5b35d2740 1 -- 172.21.15.18:0/3711588707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5940111f0 con 0x55dda46abf10 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.418+0000 7fc58f400640 1 -- 172.21.15.18:0/3711588707 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fc594005280 con 0x55dda46abf10 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc58f400640 1 -- 172.21.15.18:0/3711588707 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc594004050 con 0x55dda46abf10 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc5b35d2740 10 client.28535 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc5b35d2740 1 -- 172.21.15.18:0/3711588707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dda459c7f0 con 0x55dda46abf10 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc58f400640 1 -- 172.21.15.18:0/3711588707 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5940111f0 con 0x55dda46abf10 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc58f400640 1 -- 172.21.15.18:0/3711588707 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc5940035f0 con 0x55dda46abf10 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc5b35d2740 10 client.28535 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc5b35d2740 10 client.28535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc5b35d2740 20 client.28535 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.419+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dda46c5820 0x55dda46c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.420+0000 7fc5b35d2740 4 client.28535 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.420+0000 7fc5b35d2740 1 -- 172.21.15.18:0/3711588707 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dda461ea40 con 0x55dda46c5820 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.420+0000 7fc5ade00640 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dda46c5820 0x55dda46c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.421+0000 7fc5ade00640 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dda46c5820 0x55dda46c7c10 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746f189201b18fe1 server_cookie=d468e3349298f50b in_seq=0 out_seq=0 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.421+0000 7fc58f400640 10 client.28535 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.421+0000 7fc58f400640 1 -- 172.21.15.18:0/3711588707 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dda461ea40 con 0x55dda46c5820 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc58f400640 10 client.28535 handle_command_reply: tid=0 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 client.28535 shutdown 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 -- 172.21.15.18:0/3711588707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 msgr2=0x55dda46b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46b24d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc59400f790 tx=0x7fc59400fda0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 -- 172.21.15.18:0/3711588707 shutdown_connections 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dda46c5820 0x55dda46c7c10 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46b5410 0x55dda46b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/3711588707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46abf10 0x55dda46b24d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 -- 172.21.15.18:0/3711588707 >> 172.21.15.18:0/3711588707 conn(0x55dda46a6640 msgr2=0x55dda46a8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc58f400640 0 client.28535 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 -- 172.21.15.18:0/3711588707 shutdown_connections 2024-04-16T17:15:33.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 1 -- 172.21.15.18:0/3711588707 wait complete. 2024-04-16T17:15:33.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.422+0000 7fc5b35d2740 20 client.28535 trim_cache size 0 max 16384 2024-04-16T17:15:33.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.439+0000 7fc5b35d2740 1 Processor -- start 2024-04-16T17:15:33.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5b35d2740 1 -- start start 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46abf10 0x55dda46b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46ac830 0x55dda46c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46b5410 0x55dda46cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a81b3f60 con 0x55dda46abf10 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a81b1830 con 0x55dda46ac830 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a81af6a0 con 0x55dda46b5410 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5ade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46ac830 0x55dda46c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46abf10 0x55dda46b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5ade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46ac830 0x55dda46c10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57488/0 (socket says 172.21.15.18:57488) 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46abf10 0x55dda46b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49474/0 (socket says 172.21.15.18:49474) 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.440+0000 7fc5ade00640 1 -- 172.21.15.18:0/1519401019 learned_addr learned my addr 172.21.15.18:0/1519401019 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc5ad400640 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46b5410 0x55dda46cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc5ad400640 1 -- 172.21.15.18:0/1519401019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46b5410 msgr2=0x55dda46cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc5ad400640 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46b5410 0x55dda46cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc5ad400640 1 -- 172.21.15.18:0/1519401019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46ac830 msgr2=0x55dda46c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc5ad400640 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46ac830 0x55dda46c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc5ad400640 1 -- 172.21.15.18:0/1519401019 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda43233f0 con 0x55dda46abf10 2024-04-16T17:15:33.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc5ade00640 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46ac830 0x55dda46c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:33.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc5ad400640 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46abf10 0x55dda46b4c00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc594021040 tx=0x7fc594011570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0da0c3f3362cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc59a600640 1 -- 172.21.15.18:0/1519401019 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594009b50 con 0x55dda46abf10 2024-04-16T17:15:33.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc59a600640 1 -- 172.21.15.18:0/1519401019 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc594003c00 con 0x55dda46abf10 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.441+0000 7fc59a600640 1 -- 172.21.15.18:0/1519401019 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594011020 con 0x55dda46abf10 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.442+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1519401019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46abf10 msgr2=0x55dda46b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.442+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46abf10 0x55dda46b4c00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc594021040 tx=0x7fc594011570 comp rx=0 tx=0).stop 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.442+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1519401019 shutdown_connections 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.442+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46b5410 0x55dda46cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.442+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46ac830 0x55dda46c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.442+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/1519401019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46abf10 0x55dda46b4c00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.442+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1519401019 >> 172.21.15.18:0/1519401019 conn(0x55dda46a6640 msgr2=0x55dda46b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.443+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1519401019 shutdown_connections 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.443+0000 7fc5b35d2740 1 -- 172.21.15.18:0/1519401019 wait complete. 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.443+0000 7fc5b35d2740 1 -- start start 2024-04-16T17:15:33.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.443+0000 7fc5b35d2740 1 Processor -- start 2024-04-16T17:15:33.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.444+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.444+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46b5410 0x55dda46bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.444+0000 7fc5b35d2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46aec60 0x55dda46af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.444+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a81afcf0 con 0x55dda46aec60 2024-04-16T17:15:33.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.444+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a81b1160 con 0x55dda46b5410 2024-04-16T17:15:33.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.444+0000 7fc5b35d2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a81b03c0 con 0x55dda46ac830 2024-04-16T17:15:33.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.444+0000 7fc5b35d2740 10 client.0 _ll_register_callbacks cb 0x7fc5940115e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:33.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.444+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46bd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56158/0 (socket says 172.21.15.18:56158) 2024-04-16T17:15:33.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 -- 172.21.15.18:0/748033757 learned_addr learned my addr 172.21.15.18:0/748033757 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:33.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46aec60 0x55dda46af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 -- 172.21.15.18:0/748033757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46b5410 msgr2=0x55dda46bdb10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46b5410 0x55dda46bdb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 -- 172.21.15.18:0/748033757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46aec60 msgr2=0x55dda46af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46aec60 0x55dda46af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 -- 172.21.15.18:0/748033757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda43ed7d0 con 0x55dda46ac830 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc5ad400640 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46bd5d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc594003a10 tx=0x7fc5940035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0248f3a1ff5a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.445+0000 7fc58f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.446+0000 7fc58f400640 1 -- 172.21.15.18:0/748033757 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594009b50 con 0x55dda46ac830 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.446+0000 7fc58f400640 1 -- 172.21.15.18:0/748033757 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fc59401e430 con 0x55dda46ac830 2024-04-16T17:15:33.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.446+0000 7fc5b35d2740 1 -- 172.21.15.18:0/748033757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc594003c00 con 0x55dda46ac830 2024-04-16T17:15:33.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.447+0000 7fc58f400640 1 -- 172.21.15.18:0/748033757 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc594005740 con 0x55dda46ac830 2024-04-16T17:15:33.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.448+0000 7fc5b35d2740 10 client.28399 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:33.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.448+0000 7fc5b35d2740 1 -- 172.21.15.18:0/748033757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dda472ba40 con 0x55dda46ac830 2024-04-16T17:15:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.452+0000 7fc58f400640 1 -- 172.21.15.18:0/748033757 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc594003c00 con 0x55dda46ac830 2024-04-16T17:15:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.453+0000 7fc58f400640 1 -- 172.21.15.18:0/748033757 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc59401a070 con 0x55dda46ac830 2024-04-16T17:15:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.453+0000 7fc5b35d2740 10 client.28399 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:33.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.453+0000 7fc5b35d2740 10 client.28399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.453+0000 7fc5b35d2740 20 client.28399 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.453+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dda472c0a0 0x55dda472e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.453+0000 7fc5b35d2740 4 client.28399 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:33.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.453+0000 7fc5b35d2740 1 -- 172.21.15.18:0/748033757 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dda46188b0 con 0x55dda472c0a0 2024-04-16T17:15:33.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.455+0000 7fc5ade00640 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dda472c0a0 0x55dda472e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:33.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.459+0000 7fc5ade00640 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dda472c0a0 0x55dda472e4c0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb332953eb2f50f server_cookie=a42d34ee972a0e09 in_seq=0 out_seq=0 2024-04-16T17:15:33.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.459+0000 7fc58f400640 10 client.28399 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:33.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.462+0000 7fc58f400640 1 -- 172.21.15.18:0/748033757 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dda46188b0 con 0x55dda472c0a0 2024-04-16T17:15:33.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.462+0000 7fc58f400640 10 client.28399 handle_command_reply: tid=0 2024-04-16T17:15:33.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.462+0000 7fc5b35d2740 1 client.28399 shutdown 2024-04-16T17:15:33.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.465+0000 7fc5b35d2740 1 -- 172.21.15.18:0/748033757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 msgr2=0x55dda46bd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.465+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46bd5d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc594003a10 tx=0x7fc5940035f0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.466+0000 7fc5b35d2740 1 -- 172.21.15.18:0/748033757 shutdown_connections 2024-04-16T17:15:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.466+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55dda472c0a0 0x55dda472e4c0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.466+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dda46aec60 0x55dda46af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.466+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dda46b5410 0x55dda46bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.466+0000 7fc5b35d2740 1 --2- 172.21.15.18:0/748033757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dda46ac830 0x55dda46bd5d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.466+0000 7fc5b35d2740 1 -- 172.21.15.18:0/748033757 >> 172.21.15.18:0/748033757 conn(0x55dda46a6640 msgr2=0x55dda46a4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.466+0000 7fc58f400640 0 client.28399 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.467+0000 7fc5b35d2740 1 -- 172.21.15.18:0/748033757 shutdown_connections 2024-04-16T17:15:33.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.467+0000 7fc5b35d2740 1 -- 172.21.15.18:0/748033757 wait complete. 2024-04-16T17:15:33.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.467+0000 7fc5b35d2740 20 client.28399 trim_cache size 0 max 16384 2024-04-16T17:15:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:15:33.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.469+0000 7fc5af200640 1 -- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5900a3da0 msgr2=0x7fc5900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.469+0000 7fc5af200640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5900a3da0 0x7fc5900a6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc5a401a040 tx=0x7fc5a4005f70 comp rx=0 tx=0).stop 2024-04-16T17:15:33.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.469+0000 7fc5af200640 1 -- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a819f330 msgr2=0x7fc5a819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:33.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.469+0000 7fc5af200640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a819f330 0x7fc5a819f7e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009fd0 tx=0x7fc5a400ef10 comp rx=0 tx=0).stop 2024-04-16T17:15:33.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.470+0000 7fc5af200640 1 -- 172.21.15.18:0/276634766 shutdown_connections 2024-04-16T17:15:33.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.470+0000 7fc5af200640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc5900a3da0 0x7fc5900a6260 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.470+0000 7fc5af200640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5a819f330 0x7fc5a819f7e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.470+0000 7fc5af200640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a8107110 0x7fc5a819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.470+0000 7fc5af200640 1 --2- 172.21.15.18:0/276634766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5a8102040 0x7fc5a819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.470+0000 7fc5af200640 1 -- 172.21.15.18:0/276634766 >> 172.21.15.18:0/276634766 conn(0x7fc5a80fd8e0 msgr2=0x7fc5a80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:33.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.472+0000 7fc5af200640 1 -- 172.21.15.18:0/276634766 shutdown_connections 2024-04-16T17:15:33.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:33.472+0000 7fc5af200640 1 -- 172.21.15.18:0/276634766 wait complete. 2024-04-16T17:15:33.647 INFO:tasks.workunit.client.0.smithi018.stdout: 1 406869 3.63 MB/sec execute 466 sec latency 156.921 ms 2024-04-16T17:15:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:33 smithi022 ceph-mon[36023]: pgmap v341: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1005 KiB/s wr, 93 op/s 2024-04-16T17:15:34.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:33 smithi043 ceph-mon[36901]: pgmap v341: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1005 KiB/s wr, 93 op/s 2024-04-16T17:15:34.502 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:15:34.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:15:34.502 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:15:34.647 INFO:tasks.workunit.client.0.smithi018.stdout: 1 408086 3.64 MB/sec execute 467 sec latency 30.329 ms 2024-04-16T17:15:34.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.732+0000 7fe2e6a00640 1 -- 172.21.15.18:0/1989620232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0104c30 msgr2=0x7fe2e0108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.732+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/1989620232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0104c30 0x7fe2e0108c70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009900 tx=0x7fe2d402bc40 comp rx=0 tx=0).stop 2024-04-16T17:15:34.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.732+0000 7fe2e6a00640 1 -- 172.21.15.18:0/1989620232 shutdown_connections 2024-04-16T17:15:34.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.732+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/1989620232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0109350 0x7fe2e0110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.732+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/1989620232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0104c30 0x7fe2e0108c70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.732+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/1989620232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2e0104280 0x7fe2e0104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.732+0000 7fe2e6a00640 1 -- 172.21.15.18:0/1989620232 >> 172.21.15.18:0/1989620232 conn(0x7fe2e00ffae0 msgr2=0x7fe2e0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:34.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.733+0000 7fe2e6a00640 1 -- 172.21.15.18:0/1989620232 shutdown_connections 2024-04-16T17:15:34.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.733+0000 7fe2e6a00640 1 -- 172.21.15.18:0/1989620232 wait complete. 2024-04-16T17:15:34.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.734+0000 7fe2e6a00640 1 Processor -- start 2024-04-16T17:15:34.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.734+0000 7fe2e6a00640 1 -- start start 2024-04-16T17:15:34.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0104280 0x7fe2e0076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2e0109350 0x7fe2e0076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0076f90 0x7fe2e01aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e6a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e0114410 con 0x7fe2e0109350 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e6a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e0114290 con 0x7fe2e0076f90 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e6a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e0114590 con 0x7fe2e0104280 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2e0109350 0x7fe2e0076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0104280 0x7fe2e0076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2e0109350 0x7fe2e0076a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49500/0 (socket says 172.21.15.18:49500) 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e4c00640 1 -- 172.21.15.18:0/2532529176 learned_addr learned my addr 172.21.15.18:0/2532529176 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.735+0000 7fe2e6000640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0076f90 0x7fe2e01aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e5600640 1 -- 172.21.15.18:0/2532529176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0076f90 msgr2=0x7fe2e01aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e5600640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0076f90 0x7fe2e01aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e5600640 1 -- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2e0109350 msgr2=0x7fe2e0076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e5600640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2e0109350 0x7fe2e0076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e5600640 1 -- 172.21.15.18:0/2532529176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e01ab4f0 con 0x7fe2e0104280 2024-04-16T17:15:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2e0109350 0x7fe2e0076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e6000640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0076f90 0x7fe2e01aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e5600640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0104280 0x7fe2e0076510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800eb00 tx=0x7fe2d800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2de000640 1 -- 172.21.15.18:0/2532529176 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8010430 con 0x7fe2e0104280 2024-04-16T17:15:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2de000640 1 -- 172.21.15.18:0/2532529176 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d8010ab0 con 0x7fe2e0104280 2024-04-16T17:15:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2de000640 1 -- 172.21.15.18:0/2532529176 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8017bb0 con 0x7fe2e0104280 2024-04-16T17:15:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.736+0000 7fe2e6a00640 1 -- 172.21.15.18:0/2532529176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e01ab7e0 con 0x7fe2e0104280 2024-04-16T17:15:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.737+0000 7fe2e6a00640 1 -- 172.21.15.18:0/2532529176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e01abcb0 con 0x7fe2e0104280 2024-04-16T17:15:34.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.740+0000 7fe2de000640 1 -- 172.21.15.18:0/2532529176 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe2d8010c60 con 0x7fe2e0104280 2024-04-16T17:15:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.741+0000 7fe2eac1a740 1 Processor -- start 2024-04-16T17:15:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.741+0000 7fe2eac1a740 1 -- start start 2024-04-16T17:15:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.741+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101398d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.741+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x561013995e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.741+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561013996390 0x561013998780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5610138f1cb0 con 0x561013996390 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561013898370 con 0x56101398d7b0 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5610138edc70 con 0x56101398ce90 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2de000640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe2c80a3d20 0x7fe2c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101398d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101398d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56186/0 (socket says 172.21.15.18:56186) 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2e6000640 1 -- 172.21.15.18:0/2921511653 learned_addr learned my addr 172.21.15.18:0/2921511653 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2e5600640 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x561013995e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2de000640 1 -- 172.21.15.18:0/2532529176 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fe2d80d1620 con 0x7fe2e0104280 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561013996390 0x561013998780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.742+0000 7fe2e6000640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe2c80a3d20 0x7fe2c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e6000640 1 -- 172.21.15.18:0/2921511653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 msgr2=0x561013995e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e6000640 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x561013995e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e6000640 1 -- 172.21.15.18:0/2921511653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561013996390 msgr2=0x561013998780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e6000640 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561013996390 0x561013998780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e6000640 1 -- 172.21.15.18:0/2921511653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610138b7910 con 0x56101398ce90 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e5600640 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x561013995e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561013996390 0x561013998780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e6000640 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101398d270 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc012ac0 tx=0x7fe2cc012f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c9ec626e577e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2e6000640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe2c80a3d20 0x7fe2c80a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00e7e0 tx=0x7fe2cc01f000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2d2600640 1 -- 172.21.15.18:0/2921511653 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc01ee40 con 0x56101398ce90 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2d2600640 1 -- 172.21.15.18:0/2921511653 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2cc00ec30 con 0x56101398ce90 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.743+0000 7fe2d2600640 1 -- 172.21.15.18:0/2921511653 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00f620 con 0x56101398ce90 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 -- 172.21.15.18:0/2921511653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 msgr2=0x56101398d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101398d270 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc012ac0 tx=0x7fe2cc012f80 comp rx=0 tx=0).stop 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 -- 172.21.15.18:0/2921511653 shutdown_connections 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561013996390 0x561013998780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x561013995e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/2921511653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101398d270 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 -- 172.21.15.18:0/2921511653 >> 172.21.15.18:0/2921511653 conn(0x5610139875c0 msgr2=0x5610139879a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:34.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 -- 172.21.15.18:0/2921511653 shutdown_connections 2024-04-16T17:15:34.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.744+0000 7fe2eac1a740 1 -- 172.21.15.18:0/2921511653 wait complete. 2024-04-16T17:15:34.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.745+0000 7fe2eac1a740 1 -- start start 2024-04-16T17:15:34.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.745+0000 7fe2eac1a740 1 Processor -- start 2024-04-16T17:15:34.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398ce90 0x561013993420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x561013993960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x561013993ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5610139990c0 con 0x56101398d7b0 2024-04-16T17:15:34.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610138ab730 con 0x56101398ce90 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561013999240 con 0x561013996390 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2eac1a740 10 client.0 _ll_register_callbacks cb 0x56101387a1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398ce90 0x561013993420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398ce90 0x561013993420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57534/0 (socket says 172.21.15.18:57534) 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e5600640 1 -- 172.21.15.18:0/1229159211 learned_addr learned my addr 172.21.15.18:0/1229159211 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e6000640 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x561013993ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x561013993960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e5600640 1 -- 172.21.15.18:0/1229159211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 msgr2=0x561013993ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e5600640 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x561013993ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e5600640 1 -- 172.21.15.18:0/1229159211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 msgr2=0x561013993960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e5600640 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x561013993960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e5600640 1 -- 172.21.15.18:0/1229159211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610136cc7e0 con 0x56101398ce90 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.746+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x561013993960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2e5600640 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398ce90 0x561013993420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8005fd0 tx=0x7fe2d809cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f5ab5efbb6832e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1229159211 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d80b5030 con 0x56101398ce90 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1229159211 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fe2d8012e50 con 0x56101398ce90 2024-04-16T17:15:34.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1229159211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2cc00ec30 con 0x56101398ce90 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1229159211 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe2d809d150 con 0x56101398ce90 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2eac1a740 10 client.28550 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1229159211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56101372cf70 con 0x56101398ce90 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.747+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1229159211 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2cc00ec30 con 0x56101398ce90 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.748+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1229159211 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe2d80a1160 con 0x56101398ce90 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.748+0000 7fe2eac1a740 10 client.28550 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.748+0000 7fe2eac1a740 10 client.28550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.748+0000 7fe2eac1a740 20 client.28550 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.748+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5610139a6600 0x5610139a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.748+0000 7fe2eac1a740 4 client.28550 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.748+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1229159211 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610138ff6d0 con 0x5610139a6600 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.748+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5610139a6600 0x5610139a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.749+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5610139a6600 0x5610139a8a50 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b86d44593999c89 server_cookie=4f4bcd97b61f4c3a in_seq=0 out_seq=0 2024-04-16T17:15:34.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.749+0000 7fe2c6a00640 10 client.28550 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:34.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1229159211 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610138ff6d0 con 0x5610139a6600 2024-04-16T17:15:34.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2c6a00640 10 client.28550 handle_command_reply: tid=0 2024-04-16T17:15:34.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 client.28550 shutdown 2024-04-16T17:15:34.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1229159211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398ce90 msgr2=0x561013993420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398ce90 0x561013993420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8005fd0 tx=0x7fe2d809cd50 comp rx=0 tx=0).stop 2024-04-16T17:15:34.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1229159211 shutdown_connections 2024-04-16T17:15:34.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5610139a6600 0x5610139a8a50 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x561013993ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x561013993960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1229159211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398ce90 0x561013993420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1229159211 >> 172.21.15.18:0/1229159211 conn(0x5610139875c0 msgr2=0x561013989310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:34.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2c6a00640 0 client.28550 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:34.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1229159211 shutdown_connections 2024-04-16T17:15:34.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1229159211 wait complete. 2024-04-16T17:15:34.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.750+0000 7fe2eac1a740 20 client.28550 trim_cache size 0 max 16384 2024-04-16T17:15:34.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.772+0000 7fe2eac1a740 1 Processor -- start 2024-04-16T17:15:34.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2eac1a740 1 -- start start 2024-04-16T17:15:34.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398ce90 0x561013995b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x5610139a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x5610139ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b8410 con 0x56101398ce90 2024-04-16T17:15:34.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b5ce0 con 0x56101398d7b0 2024-04-16T17:15:34.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b3b50 con 0x561013996390 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2e5600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398ce90 0x561013995b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x5610139ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x5610139ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56212/0 (socket says 172.21.15.18:56212) 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2e6000640 1 -- 172.21.15.18:0/1838675359 learned_addr learned my addr 172.21.15.18:0/1838675359 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.773+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x5610139a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2e4c00640 1 -- 172.21.15.18:0/1838675359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 msgr2=0x5610139ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x5610139ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2e4c00640 1 -- 172.21.15.18:0/1838675359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398ce90 msgr2=0x561013995b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398ce90 0x561013995b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2e4c00640 1 -- 172.21.15.18:0/1838675359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610135c4170 con 0x56101398d7b0 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2e6000640 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x5610139ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2e5600640 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398ce90 0x561013995b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x5610139a1f80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4006010 tx=0x7fe2d402e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb48b33476bb0baf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:34.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2d1c00640 1 -- 172.21.15.18:0/1838675359 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4048dc0 con 0x56101398d7b0 2024-04-16T17:15:34.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2d1c00640 1 -- 172.21.15.18:0/1838675359 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d402c890 con 0x56101398d7b0 2024-04-16T17:15:34.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.774+0000 7fe2d1c00640 1 -- 172.21.15.18:0/1838675359 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d40487c0 con 0x56101398d7b0 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.775+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1838675359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 msgr2=0x5610139a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.775+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x5610139a1f80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4006010 tx=0x7fe2d402e040 comp rx=0 tx=0).stop 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.775+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1838675359 shutdown_connections 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.775+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561013996390 0x5610139ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.775+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56101398d7b0 0x5610139a1f80 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.775+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1838675359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398ce90 0x561013995b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.775+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1838675359 >> 172.21.15.18:0/1838675359 conn(0x5610139875c0 msgr2=0x561013986560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.776+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1838675359 shutdown_connections 2024-04-16T17:15:34.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.776+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1838675359 wait complete. 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.776+0000 7fe2eac1a740 1 -- start start 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.776+0000 7fe2eac1a740 1 Processor -- start 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101399e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x56101399a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2eac1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561013996390 0x56101399adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b41a0 con 0x56101398d7b0 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b5610 con 0x561013996390 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2eac1a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b4870 con 0x56101398ce90 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2eac1a740 10 client.0 _ll_register_callbacks cb 0x7fe2d809cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x56101399a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561013996390 0x56101399adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x56101399a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49534/0 (socket says 172.21.15.18:49534) 2024-04-16T17:15:34.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e4c00640 1 -- 172.21.15.18:0/1583994353 learned_addr learned my addr 172.21.15.18:0/1583994353 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:34.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e5600640 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101399e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e4c00640 1 -- 172.21.15.18:0/1583994353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 msgr2=0x56101399e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:34.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101399e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e4c00640 1 -- 172.21.15.18:0/1583994353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561013996390 msgr2=0x56101399adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:34.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561013996390 0x56101399adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.777+0000 7fe2e4c00640 1 -- 172.21.15.18:0/1583994353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101399b330 con 0x56101398d7b0 2024-04-16T17:15:34.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.778+0000 7fe2e4c00640 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x56101399a8b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009850 tx=0x7fe2d4048440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1102e0e5e3eb0796 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:34.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.778+0000 7fe2c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.778+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1583994353 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4005160 con 0x56101398d7b0 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.778+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1583994353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2d402c890 con 0x56101398d7b0 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.778+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1583994353 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fe2d4005300 con 0x56101398d7b0 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.778+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1583994353 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe2d402cd10 con 0x56101398d7b0 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.778+0000 7fe2eac1a740 10 client.18261 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.778+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1583994353 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561013a0d8f0 con 0x56101398d7b0 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.779+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1583994353 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d402c890 con 0x56101398d7b0 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.779+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1583994353 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe2d4034540 con 0x56101398d7b0 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.779+0000 7fe2eac1a740 10 client.18261 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.779+0000 7fe2eac1a740 10 client.18261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.779+0000 7fe2eac1a740 20 client.18261 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.779+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561013a0dff0 0x561013a10410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.780+0000 7fe2eac1a740 4 client.18261 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:15:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.780+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1583994353 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5610138f9540 con 0x561013a0dff0 2024-04-16T17:15:34.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.781+0000 7fe2e5600640 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561013a0dff0 0x561013a10410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:34.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.781+0000 7fe2e5600640 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561013a0dff0 0x561013a10410 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47300b15299042b0 server_cookie=6a098cc209cbab17 in_seq=0 out_seq=0 2024-04-16T17:15:34.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.781+0000 7fe2c6a00640 10 client.18261 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:34.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.783+0000 7fe2c6a00640 1 -- 172.21.15.18:0/1583994353 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5610138f9540 con 0x561013a0dff0 2024-04-16T17:15:34.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.783+0000 7fe2c6a00640 10 client.18261 handle_command_reply: tid=0 2024-04-16T17:15:34.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 client.18261 shutdown 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1583994353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 msgr2=0x56101399a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x56101399a8b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009850 tx=0x7fe2d4048440 comp rx=0 tx=0).stop 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1583994353 shutdown_connections 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561013a0dff0 0x561013a10410 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561013996390 0x56101399adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56101398d7b0 0x56101399a8b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 --2- 172.21.15.18:0/1583994353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101398ce90 0x56101399e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1583994353 >> 172.21.15.18:0/1583994353 conn(0x5610139875c0 msgr2=0x5610139933e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:34.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2c6a00640 0 client.18261 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:34.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1583994353 shutdown_connections 2024-04-16T17:15:34.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 1 -- 172.21.15.18:0/1583994353 wait complete. 2024-04-16T17:15:34.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.784+0000 7fe2eac1a740 20 client.18261 trim_cache size 0 max 16384 2024-04-16T17:15:34.786 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:15:34.786 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:15:34.786 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "6d6b3576-eb6b-4978-b28a-fdf9015a072e", 2024-04-16T17:15:34.786 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:15:34.786 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 -- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe2c80a3d20 msgr2=0x7fe2c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe2c80a3d20 0x7fe2c80a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00e7e0 tx=0x7fe2cc01f000 comp rx=0 tx=0).stop 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 -- 172.21.15.18:0/2532529176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0104280 msgr2=0x7fe2e0076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0104280 0x7fe2e0076510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800eb00 tx=0x7fe2d800c2d0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 -- 172.21.15.18:0/2532529176 shutdown_connections 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe2c80a3d20 0x7fe2c80a61e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0076f90 0x7fe2e01aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2e0109350 0x7fe2e0076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 --2- 172.21.15.18:0/2532529176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0104280 0x7fe2e0076510 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.785+0000 7fe2e6a00640 1 -- 172.21.15.18:0/2532529176 >> 172.21.15.18:0/2532529176 conn(0x7fe2e00ffae0 msgr2=0x7fe2e0101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.786+0000 7fe2e6a00640 1 -- 172.21.15.18:0/2532529176 shutdown_connections 2024-04-16T17:15:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:34.786+0000 7fe2e6a00640 1 -- 172.21.15.18:0/2532529176 wait complete. 2024-04-16T17:15:34.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:15:34.806 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:15:35.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 -- 172.21.15.18:0/2502090368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 msgr2=0x7f7c40112770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:35.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 --2- 172.21.15.18:0/2502090368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 0x7f7c40112770 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009930 tx=0x7f7c2c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:35.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 -- 172.21.15.18:0/2502090368 shutdown_connections 2024-04-16T17:15:35.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 --2- 172.21.15.18:0/2502090368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c40112cb0 0x7f7c401150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 --2- 172.21.15.18:0/2502090368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 0x7f7c40112770 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 --2- 172.21.15.18:0/2502090368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c40076530 0x7f7c40076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 -- 172.21.15.18:0/2502090368 >> 172.21.15.18:0/2502090368 conn(0x7f7c400ffae0 msgr2=0x7f7c40101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:35.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 -- 172.21.15.18:0/2502090368 shutdown_connections 2024-04-16T17:15:35.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.033+0000 7f7c44e00640 1 -- 172.21.15.18:0/2502090368 wait complete. 2024-04-16T17:15:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.034+0000 7f7c44e00640 1 Processor -- start 2024-04-16T17:15:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.034+0000 7f7c44e00640 1 -- start start 2024-04-16T17:15:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.034+0000 7f7c44e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c40076530 0x7f7c4019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.034+0000 7f7c44e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 0x7f7c4019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.034+0000 7f7c44e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c40112cb0 0x7f7c401a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.034+0000 7f7c44e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7c40118870 con 0x7f7c40076530 2024-04-16T17:15:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.034+0000 7f7c44e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401186f0 con 0x7f7c40076e50 2024-04-16T17:15:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.034+0000 7f7c44e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401189f0 con 0x7f7c40112cb0 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 0x7f7c4019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 0x7f7c4019f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57554/0 (socket says 172.21.15.18:57554) 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3ea00640 1 -- 172.21.15.18:0/204558007 learned_addr learned my addr 172.21.15.18:0/204558007 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3f400640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c40076530 0x7f7c4019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3ea00640 1 -- 172.21.15.18:0/204558007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c40112cb0 msgr2=0x7f7c401a3660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c40112cb0 0x7f7c401a3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3ea00640 1 -- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c40076530 msgr2=0x7f7c4019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c40076530 0x7f7c4019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.035+0000 7f7c3ea00640 1 -- 172.21.15.18:0/204558007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c401a3d90 con 0x7f7c40076e50 2024-04-16T17:15:35.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.036+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 0x7f7c4019f330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009900 tx=0x7f7c2c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:35.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.036+0000 7f7c37e00640 1 -- 172.21.15.18:0/204558007 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c043020 con 0x7f7c40076e50 2024-04-16T17:15:35.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.036+0000 7f7c37e00640 1 -- 172.21.15.18:0/204558007 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2c0054e0 con 0x7f7c40076e50 2024-04-16T17:15:35.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.036+0000 7f7c37e00640 1 -- 172.21.15.18:0/204558007 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c003bb0 con 0x7f7c40076e50 2024-04-16T17:15:35.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.036+0000 7f7c44e00640 1 -- 172.21.15.18:0/204558007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c401a4020 con 0x7f7c40076e50 2024-04-16T17:15:35.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.036+0000 7f7c44e00640 1 -- 172.21.15.18:0/204558007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c401ab910 con 0x7f7c40076e50 2024-04-16T17:15:35.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.041+0000 7f7c4905e740 1 Processor -- start 2024-04-16T17:15:35.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.041+0000 7f7c4905e740 1 -- start start 2024-04-16T17:15:35.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.041+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a531fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.041+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fccb0 0x558a53205350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.041+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a53205890 0x558a53207c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.041+0000 7f7c4905e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558a53162120 con 0x558a531fccb0 2024-04-16T17:15:35.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.041+0000 7f7c4905e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a53108340 con 0x558a53205890 2024-04-16T17:15:35.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.041+0000 7f7c4905e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558a5315e0e0 con 0x558a531fc390 2024-04-16T17:15:35.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a531fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fccb0 0x558a53205350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fccb0 0x558a53205350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49566/0 (socket says 172.21.15.18:49566) 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3f400640 1 -- 172.21.15.18:0/3071616431 learned_addr learned my addr 172.21.15.18:0/3071616431 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3f400640 1 -- 172.21.15.18:0/3071616431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 msgr2=0x558a531fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3f400640 1 --2- 172.21.15.18:0/3071616431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a531fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3f400640 1 -- 172.21.15.18:0/3071616431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a53205890 msgr2=0x558a53207c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3f400640 1 --2- 172.21.15.18:0/3071616431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a53205890 0x558a53207c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.042+0000 7f7c3f400640 1 -- 172.21.15.18:0/3071616431 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a52f3e7d0 con 0x558a531fccb0 2024-04-16T17:15:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.043+0000 7f7c3f400640 1 --2- 172.21.15.18:0/3071616431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fccb0 0x558a53205350 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009d20 tx=0x7f7c30011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e68d524ef876a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:35.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.043+0000 7f7c34c00640 1 -- 172.21.15.18:0/3071616431 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3001e070 con 0x558a531fccb0 2024-04-16T17:15:35.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.043+0000 7f7c34c00640 1 -- 172.21.15.18:0/3071616431 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c30009e10 con 0x558a531fccb0 2024-04-16T17:15:35.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.043+0000 7f7c34c00640 1 -- 172.21.15.18:0/3071616431 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000ea80 con 0x558a531fccb0 2024-04-16T17:15:35.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c37e00640 1 -- 172.21.15.18:0/204558007 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c2c046020 con 0x7f7c40076e50 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 -- 172.21.15.18:0/3071616431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fccb0 msgr2=0x558a53205350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3071616431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fccb0 0x558a53205350 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009d20 tx=0x7f7c30011810 comp rx=0 tx=0).stop 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 -- 172.21.15.18:0/3071616431 shutdown_connections 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3071616431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a53205890 0x558a53207c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3071616431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fccb0 0x558a53205350 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3071616431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a531fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 -- 172.21.15.18:0/3071616431 >> 172.21.15.18:0/3071616431 conn(0x558a531f6ac0 msgr2=0x558a531f6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 -- 172.21.15.18:0/3071616431 shutdown_connections 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.044+0000 7f7c4905e740 1 -- 172.21.15.18:0/3071616431 wait complete. 2024-04-16T17:15:35.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.045+0000 7f7c4905e740 1 -- start start 2024-04-16T17:15:35.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.045+0000 7f7c4905e740 1 Processor -- start 2024-04-16T17:15:35.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.046+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a53202970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.046+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53202eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.046+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a532033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.046+0000 7f7c4905e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558a53208680 con 0x558a53205890 2024-04-16T17:15:35.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.046+0000 7f7c4905e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a5311b710 con 0x558a531fccb0 2024-04-16T17:15:35.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.046+0000 7f7c4905e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558a53208800 con 0x558a531fc390 2024-04-16T17:15:35.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.046+0000 7f7c4905e740 10 client.0 _ll_register_callbacks cb 0x558a53116af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:35.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.046+0000 7f7c37e00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7c240a3cb0 0x7f7c240a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53202eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c37e00640 1 -- 172.21.15.18:0/204558007 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f7c2c0f2ad0 con 0x7f7c40076e50 2024-04-16T17:15:35.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a532033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53202eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57560/0 (socket says 172.21.15.18:57560) 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3f400640 1 -- 172.21.15.18:0/4113779580 learned_addr learned my addr 172.21.15.18:0/4113779580 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a532033f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49568/0 (socket says 172.21.15.18:49568) 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3fe00640 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a53202970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3fe00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7c240a3cb0 0x7f7c240a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3ea00640 1 -- 172.21.15.18:0/4113779580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 msgr2=0x558a53202970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a53202970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3ea00640 1 -- 172.21.15.18:0/4113779580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 msgr2=0x558a53202eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53202eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.047+0000 7f7c3ea00640 1 -- 172.21.15.18:0/4113779580 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a52e743f0 con 0x558a53205890 2024-04-16T17:15:35.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.048+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a532033f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0eff20 tx=0x7f7c2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b345c6ae86e37f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:35.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.048+0000 7f7c29600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:35.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.048+0000 7f7c3fe00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7c240a3cb0 0x7f7c240a6170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c40069ba0 tx=0x7f7c38009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:35.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.048+0000 7f7c29600640 1 -- 172.21.15.18:0/4113779580 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c004c80 con 0x558a53205890 2024-04-16T17:15:35.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.048+0000 7f7c29600640 1 -- 172.21.15.18:0/4113779580 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f7c2c02cb90 con 0x558a53205890 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.048+0000 7f7c4905e740 1 -- 172.21.15.18:0/4113779580 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c30009e10 con 0x558a53205890 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.048+0000 7f7c29600640 1 -- 172.21.15.18:0/4113779580 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c2c005690 con 0x558a53205890 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c4905e740 10 client.18273 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c4905e740 1 -- 172.21.15.18:0/4113779580 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558a52e340f0 con 0x558a53205890 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c29600640 1 -- 172.21.15.18:0/4113779580 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c30009e10 con 0x558a53205890 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c29600640 1 -- 172.21.15.18:0/4113779580 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7c2c0f0090 con 0x558a53205890 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c4905e740 10 client.18273 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c4905e740 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c4905e740 20 client.18273 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a53215c00 0x558a53218020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c4905e740 4 client.18273 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:35.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.049+0000 7f7c4905e740 1 -- 172.21.15.18:0/4113779580 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a5316fb40 con 0x558a53215c00 2024-04-16T17:15:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.050+0000 7f7c3f400640 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a53215c00 0x558a53218020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.050+0000 7f7c3f400640 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a53215c00 0x558a53218020 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6962704dd96936 server_cookie=4894b5bc93fd4f09 in_seq=0 out_seq=0 2024-04-16T17:15:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.050+0000 7f7c29600640 10 client.18273 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.051+0000 7f7c29600640 1 -- 172.21.15.18:0/4113779580 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a5316fb40 con 0x558a53215c00 2024-04-16T17:15:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.051+0000 7f7c29600640 10 client.18273 handle_command_reply: tid=0 2024-04-16T17:15:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.051+0000 7f7c4905e740 1 client.18273 shutdown 2024-04-16T17:15:35.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c4905e740 1 -- 172.21.15.18:0/4113779580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 msgr2=0x558a532033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:35.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a532033f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0eff20 tx=0x7f7c2c009260 comp rx=0 tx=0).stop 2024-04-16T17:15:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c4905e740 1 -- 172.21.15.18:0/4113779580 shutdown_connections 2024-04-16T17:15:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a53215c00 0x558a53218020 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a532033f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53202eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4113779580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a53202970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c29600640 0 client.18273 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.052+0000 7f7c4905e740 1 -- 172.21.15.18:0/4113779580 >> 172.21.15.18:0/4113779580 conn(0x558a531f6ac0 msgr2=0x558a531f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:35.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.053+0000 7f7c4905e740 1 -- 172.21.15.18:0/4113779580 shutdown_connections 2024-04-16T17:15:35.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.053+0000 7f7c4905e740 1 -- 172.21.15.18:0/4113779580 wait complete. 2024-04-16T17:15:35.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.053+0000 7f7c4905e740 20 client.18273 trim_cache size 0 max 16384 2024-04-16T17:15:35.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.081+0000 7f7c4905e740 1 Processor -- start 2024-04-16T17:15:35.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.081+0000 7f7c4905e740 1 -- start start 2024-04-16T17:15:35.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.081+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a532050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.081+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53211540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a531f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c4905e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b8300 con 0x558a53205890 2024-04-16T17:15:35.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c4905e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b5bd0 con 0x558a531fccb0 2024-04-16T17:15:35.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c4905e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b3a40 con 0x558a531fc390 2024-04-16T17:15:35.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53211540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a532050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53211540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57572/0 (socket says 172.21.15.18:57572) 2024-04-16T17:15:35.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3ea00640 1 -- 172.21.15.18:0/4166989929 learned_addr learned my addr 172.21.15.18:0/4166989929 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3fe00640 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a531f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3ea00640 1 -- 172.21.15.18:0/4166989929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 msgr2=0x558a532050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a532050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3ea00640 1 -- 172.21.15.18:0/4166989929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 msgr2=0x558a531f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a531f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3ea00640 1 -- 172.21.15.18:0/4166989929 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a53200ec0 con 0x558a531fccb0 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.082+0000 7f7c3fe00640 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a531f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.083+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53211540 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0be470 tx=0x7f7c2c0ba410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72009d89c4cce513 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.083+0000 7f7c2be00640 1 -- 172.21.15.18:0/4166989929 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c004c80 con 0x558a531fccb0 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.083+0000 7f7c2be00640 1 -- 172.21.15.18:0/4166989929 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2c0c71d0 con 0x558a531fccb0 2024-04-16T17:15:35.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.083+0000 7f7c2be00640 1 -- 172.21.15.18:0/4166989929 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c02cb90 con 0x558a531fccb0 2024-04-16T17:15:35.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 -- 172.21.15.18:0/4166989929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 msgr2=0x558a53211540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:35.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53211540 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0be470 tx=0x7f7c2c0ba410 comp rx=0 tx=0).stop 2024-04-16T17:15:35.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 -- 172.21.15.18:0/4166989929 shutdown_connections 2024-04-16T17:15:35.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a53205890 0x558a531f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53211540 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 --2- 172.21.15.18:0/4166989929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a531fc390 0x558a532050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 -- 172.21.15.18:0/4166989929 >> 172.21.15.18:0/4166989929 conn(0x558a531f6ac0 msgr2=0x558a53202930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:35.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 -- 172.21.15.18:0/4166989929 shutdown_connections 2024-04-16T17:15:35.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 -- 172.21.15.18:0/4166989929 wait complete. 2024-04-16T17:15:35.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 -- start start 2024-04-16T17:15:35.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.084+0000 7f7c4905e740 1 Processor -- start 2024-04-16T17:15:35.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fc390 0x558a532098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53209df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c4905e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a53205890 0x558a5327b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c4905e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b4090 con 0x558a531fc390 2024-04-16T17:15:35.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c4905e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b5500 con 0x558a531fccb0 2024-04-16T17:15:35.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c4905e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b4760 con 0x558a53205890 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c4905e740 10 client.0 _ll_register_callbacks cb 0x7f7c2c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c3fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a53205890 0x558a5327b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.085+0000 7f7c3f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fc390 0x558a532098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fc390 0x558a532098b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49578/0 (socket says 172.21.15.18:49578) 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3f400640 1 -- 172.21.15.18:0/3904640177 learned_addr learned my addr 172.21.15.18:0/3904640177 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53209df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3f400640 1 -- 172.21.15.18:0/3904640177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a53205890 msgr2=0x558a5327b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3f400640 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a53205890 0x558a5327b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3f400640 1 -- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 msgr2=0x558a53209df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3f400640 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53209df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3f400640 1 -- 172.21.15.18:0/3904640177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a5327b6c0 con 0x558a531fc390 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3fe00640 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a53205890 0x558a5327b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53209df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.086+0000 7f7c3f400640 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fc390 0x558a532098b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000e670 tx=0x7f7c30009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b9e74ce91ba862 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.087+0000 7f7c29600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.087+0000 7f7c29600640 1 -- 172.21.15.18:0/3904640177 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3001e040 con 0x558a531fc390 2024-04-16T17:15:35.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.087+0000 7f7c29600640 1 -- 172.21.15.18:0/3904640177 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f7c30009e10 con 0x558a531fc390 2024-04-16T17:15:35.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.087+0000 7f7c4905e740 1 -- 172.21.15.18:0/3904640177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c2c0c71d0 con 0x558a531fc390 2024-04-16T17:15:35.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.087+0000 7f7c29600640 1 -- 172.21.15.18:0/3904640177 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c3001e040 con 0x558a531fc390 2024-04-16T17:15:35.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.088+0000 7f7c4905e740 10 client.18276 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.088+0000 7f7c4905e740 1 -- 172.21.15.18:0/3904640177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558a5327c7e0 con 0x558a531fc390 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.088+0000 7f7c29600640 1 -- 172.21.15.18:0/3904640177 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2c0c71d0 con 0x558a531fc390 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.089+0000 7f7c29600640 1 -- 172.21.15.18:0/3904640177 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7c3000ec10 con 0x558a531fc390 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.089+0000 7f7c4905e740 10 client.18276 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.089+0000 7f7c4905e740 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.089+0000 7f7c4905e740 20 client.18276 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.089+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a5327cda0 0x558a5327f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.089+0000 7f7c4905e740 4 client.18276 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.089+0000 7f7c4905e740 1 -- 172.21.15.18:0/3904640177 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a531699b0 con 0x558a5327cda0 2024-04-16T17:15:35.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.089+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a5327cda0 0x558a5327f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:35.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c3ea00640 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a5327cda0 0x558a5327f1c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ee5f81bdc70cfe9 server_cookie=822f3b644d0da7fc in_seq=0 out_seq=0 2024-04-16T17:15:35.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c29600640 10 client.18276 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c29600640 1 -- 172.21.15.18:0/3904640177 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558a531699b0 con 0x558a5327cda0 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c29600640 10 client.18276 handle_command_reply: tid=0 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c4905e740 1 client.18276 shutdown 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c4905e740 1 -- 172.21.15.18:0/3904640177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fc390 msgr2=0x558a532098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fc390 0x558a532098b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000e670 tx=0x7f7c30009cb0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c4905e740 1 -- 172.21.15.18:0/3904640177 shutdown_connections 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a5327cda0 0x558a5327f1c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.090+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a53205890 0x558a5327b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.091+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a531fccb0 0x558a53209df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.091+0000 7f7c4905e740 1 --2- 172.21.15.18:0/3904640177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a531fc390 0x558a532098b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.091+0000 7f7c4905e740 1 -- 172.21.15.18:0/3904640177 >> 172.21.15.18:0/3904640177 conn(0x558a531f6ac0 msgr2=0x558a531f5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.091+0000 7f7c29600640 0 client.18276 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.091+0000 7f7c4905e740 1 -- 172.21.15.18:0/3904640177 shutdown_connections 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.091+0000 7f7c4905e740 1 -- 172.21.15.18:0/3904640177 wait complete. 2024-04-16T17:15:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.091+0000 7f7c4905e740 20 client.18276 trim_cache size 0 max 16384 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout: "6d6b3576-eb6b-4978-b28a-fdf9015a072e": { 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "6d6b3576-eb6b-4978-b28a-fdf9015a072e", 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:15:35.093 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 -- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7c240a3cb0 msgr2=0x7f7c240a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7c240a3cb0 0x7f7c240a6170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c40069ba0 tx=0x7f7c38009500 comp rx=0 tx=0).stop 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 -- 172.21.15.18:0/204558007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 msgr2=0x7f7c4019f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 0x7f7c4019f330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009900 tx=0x7f7c2c004360 comp rx=0 tx=0).stop 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 -- 172.21.15.18:0/204558007 shutdown_connections 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7c240a3cb0 0x7f7c240a6170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c40112cb0 0x7f7c401a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c40076e50 0x7f7c4019f330 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 --2- 172.21.15.18:0/204558007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c40076530 0x7f7c4019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 -- 172.21.15.18:0/204558007 >> 172.21.15.18:0/204558007 conn(0x7f7c400ffae0 msgr2=0x7f7c40101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 -- 172.21.15.18:0/204558007 shutdown_connections 2024-04-16T17:15:35.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:15:35.092+0000 7f7c44e00640 1 -- 172.21.15.18:0/204558007 wait complete. 2024-04-16T17:15:35.109 INFO:tasks.cephfs.filesystem:scrub status for tag:6d6b3576-eb6b-4978-b28a-fdf9015a072e - {'path': '/', 'tag': '6d6b3576-eb6b-4978-b28a-fdf9015a072e', 'options': 'recursive,force'} 2024-04-16T17:15:35.648 INFO:tasks.workunit.client.0.smithi018.stdout: 1 408738 3.64 MB/sec execute 468 sec latency 22.621 ms 2024-04-16T17:15:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:35 smithi022 ceph-mon[36023]: pgmap v342: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 942 KiB/s wr, 78 op/s 2024-04-16T17:15:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:35 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:15:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:35 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:35 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:15:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:35 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:35 smithi043 ceph-mon[36901]: pgmap v342: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 942 KiB/s wr, 78 op/s 2024-04-16T17:15:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:35 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:15:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:35 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:35 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:15:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:35 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:36.648 INFO:tasks.workunit.client.0.smithi018.stdout: 1 409165 3.63 MB/sec execute 469 sec latency 89.509 ms 2024-04-16T17:15:37.649 INFO:tasks.workunit.client.0.smithi018.stdout: 1 409490 3.63 MB/sec execute 470 sec latency 67.567 ms 2024-04-16T17:15:37.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:37 smithi022 ceph-mon[36023]: pgmap v343: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-16T17:15:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:37 smithi043 ceph-mon[36901]: pgmap v343: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-16T17:15:38.649 INFO:tasks.workunit.client.0.smithi018.stdout: 1 409966 3.62 MB/sec execute 471 sec latency 89.489 ms 2024-04-16T17:15:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:33 smithi018 ceph-mon[31776]: pgmap v341: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1005 KiB/s wr, 93 op/s 2024-04-16T17:15:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:35 smithi018 ceph-mon[31776]: pgmap v342: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 942 KiB/s wr, 78 op/s 2024-04-16T17:15:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:35 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:15:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:35 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:35 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:15:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:35 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:37 smithi018 ceph-mon[31776]: pgmap v343: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-16T17:15:39.649 INFO:tasks.workunit.client.0.smithi018.stdout: 1 411155 3.63 MB/sec execute 472 sec latency 52.826 ms 2024-04-16T17:15:39.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:39 smithi022 ceph-mon[36023]: pgmap v344: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 977 KiB/s wr, 146 op/s 2024-04-16T17:15:39.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:39 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:15:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:39 smithi043 ceph-mon[36901]: pgmap v344: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 977 KiB/s wr, 146 op/s 2024-04-16T17:15:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:39 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:15:40.649 INFO:tasks.workunit.client.0.smithi018.stdout: 1 411901 3.64 MB/sec execute 473 sec latency 14.178 ms 2024-04-16T17:15:41.578 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:39 smithi018 ceph-mon[31776]: pgmap v344: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 977 KiB/s wr, 146 op/s 2024-04-16T17:15:41.578 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:39 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:15:41.649 INFO:tasks.workunit.client.0.smithi018.stdout: 1 412355 3.63 MB/sec execute 474 sec latency 75.647 ms 2024-04-16T17:15:41.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:41 smithi022 ceph-mon[36023]: pgmap v345: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-16T17:15:42.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:41 smithi043 ceph-mon[36901]: pgmap v345: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-16T17:15:42.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 412713 3.63 MB/sec execute 475 sec latency 74.433 ms 2024-04-16T17:15:43.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 413099 3.63 MB/sec execute 476 sec latency 61.137 ms 2024-04-16T17:15:43.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:43 smithi022 ceph-mon[36023]: pgmap v346: 129 pgs: 129 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1014 KiB/s wr, 145 op/s 2024-04-16T17:15:44.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:43 smithi043 ceph-mon[36901]: pgmap v346: 129 pgs: 129 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1014 KiB/s wr, 145 op/s 2024-04-16T17:15:44.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 413949 3.62 MB/sec execute 477 sec latency 46.280 ms 2024-04-16T17:15:45.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 415075 3.64 MB/sec execute 478 sec latency 28.085 ms 2024-04-16T17:15:45.736 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:41 smithi018 ceph-mon[31776]: pgmap v345: 129 pgs: 129 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-16T17:15:45.736 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:43 smithi018 ceph-mon[31776]: pgmap v346: 129 pgs: 129 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1014 KiB/s wr, 145 op/s 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 -- 172.21.15.18:0/2563284919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc920102260 msgr2=0x7fc920102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 --2- 172.21.15.18:0/2563284919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc920102260 0x7fc920102640 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc914009900 tx=0x7fc91402bc50 comp rx=0 tx=0).stop 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 -- 172.21.15.18:0/2563284919 shutdown_connections 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 --2- 172.21.15.18:0/2563284919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc920107330 0x7fc92010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 --2- 172.21.15.18:0/2563284919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc920102c10 0x7fc920106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 --2- 172.21.15.18:0/2563284919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc920102260 0x7fc920102640 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 -- 172.21.15.18:0/2563284919 >> 172.21.15.18:0/2563284919 conn(0x7fc9200fdac0 msgr2=0x7fc9200ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 -- 172.21.15.18:0/2563284919 shutdown_connections 2024-04-16T17:15:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.882+0000 7fc928600640 1 -- 172.21.15.18:0/2563284919 wait complete. 2024-04-16T17:15:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.883+0000 7fc928600640 1 Processor -- start 2024-04-16T17:15:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc928600640 1 -- start start 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc928600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc920102260 0x7fc92019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc928600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc920107330 0x7fc92019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc928600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc920198800 0x7fc920198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc928600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201123e0 con 0x7fc920102260 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc928600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc920112260 con 0x7fc920198800 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc928600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc920112560 con 0x7fc920107330 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc926600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc920198800 0x7fc920198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc926600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc920198800 0x7fc920198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38048/0 (socket says 172.21.15.18:38048) 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc925c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc920102260 0x7fc92019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc926600640 1 -- 172.21.15.18:0/3264583462 learned_addr learned my addr 172.21.15.18:0/3264583462 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.884+0000 7fc925200640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc920107330 0x7fc92019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.885+0000 7fc926600640 1 -- 172.21.15.18:0/3264583462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc920107330 msgr2=0x7fc92019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.885+0000 7fc926600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc920107330 0x7fc92019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.885+0000 7fc926600640 1 -- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc920102260 msgr2=0x7fc92019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.885+0000 7fc926600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc920102260 0x7fc92019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.885+0000 7fc926600640 1 -- 172.21.15.18:0/3264583462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc920199390 con 0x7fc920198800 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.886+0000 7fc926600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc920198800 0x7fc920198cb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc91800efd0 tx=0x7fc91800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.886+0000 7fc91e000640 1 -- 172.21.15.18:0/3264583462 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918003b70 con 0x7fc920198800 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.886+0000 7fc91e000640 1 -- 172.21.15.18:0/3264583462 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc918003d10 con 0x7fc920198800 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.886+0000 7fc91e000640 1 -- 172.21.15.18:0/3264583462 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9180109f0 con 0x7fc920198800 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.886+0000 7fc928600640 1 -- 172.21.15.18:0/3264583462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc920199680 con 0x7fc920198800 2024-04-16T17:15:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.886+0000 7fc928600640 1 -- 172.21.15.18:0/3264583462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc920074560 con 0x7fc920198800 2024-04-16T17:15:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.890+0000 7fc92b3b1740 1 Processor -- start 2024-04-16T17:15:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.890+0000 7fc91e000640 1 -- 172.21.15.18:0/3264583462 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc918004020 con 0x7fc920198800 2024-04-16T17:15:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.890+0000 7fc92b3b1740 1 -- start start 2024-04-16T17:15:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad8325c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad8325cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83268210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ad831c1780 con 0x55ad8325d610 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ad83167f40 con 0x55ad83265e20 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ad831bd740 con 0x55ad8325c8a0 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc925200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad8325c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc925200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad8325c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53556/0 (socket says 172.21.15.18:53556) 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc925200640 1 -- 172.21.15.18:0/1648546438 learned_addr learned my addr 172.21.15.18:0/1648546438 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc925200640 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83268210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.891+0000 7fc925c00640 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad8325cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc925200640 1 -- 172.21.15.18:0/1648546438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 msgr2=0x55ad8325cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc925200640 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad8325cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc925200640 1 -- 172.21.15.18:0/1648546438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 msgr2=0x55ad83268210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc925200640 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83268210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc925200640 1 -- 172.21.15.18:0/1648546438 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad82ed4210 con 0x55ad8325d610 2024-04-16T17:15:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc91e000640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9080a3d10 0x7fc9080a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc925200640 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad8325c360 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0099d0 tx=0x7fc90c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26027cfa42acb88d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc91e000640 1 -- 172.21.15.18:0/3264583462 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fc9180d0ff0 con 0x7fc920198800 2024-04-16T17:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.892+0000 7fc925c00640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9080a3d10 0x7fc9080a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.893+0000 7fc912a00640 1 -- 172.21.15.18:0/1648546438 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c004050 con 0x55ad8325d610 2024-04-16T17:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.893+0000 7fc912a00640 1 -- 172.21.15.18:0/1648546438 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc90c005500 con 0x55ad8325d610 2024-04-16T17:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.893+0000 7fc912a00640 1 -- 172.21.15.18:0/1648546438 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0049c0 con 0x55ad8325d610 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.893+0000 7fc92b3b1740 1 -- 172.21.15.18:0/1648546438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 msgr2=0x55ad8325c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.893+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad8325c360 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0099d0 tx=0x7fc90c005f90 comp rx=0 tx=0).stop 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.893+0000 7fc925c00640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9080a3d10 0x7fc9080a61d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc914006010 tx=0x7fc914005b50 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 -- 172.21.15.18:0/1648546438 shutdown_connections 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83268210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad8325cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/1648546438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad8325c360 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 -- 172.21.15.18:0/1648546438 >> 172.21.15.18:0/1648546438 conn(0x55ad831f75d0 msgr2=0x55ad83257590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 -- 172.21.15.18:0/1648546438 shutdown_connections 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 -- 172.21.15.18:0/1648546438 wait complete. 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 -- start start 2024-04-16T17:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.894+0000 7fc92b3b1740 1 Processor -- start 2024-04-16T17:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad832658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad83262c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83263370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ad83268b50 con 0x55ad8325d610 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ad8317b390 con 0x55ad83265e20 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ad83268cd0 con 0x55ad8325c8a0 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc92b3b1740 10 client.0 _ll_register_callbacks cb 0x55ad83164f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc925c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad832658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc925c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad832658a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37260/0 (socket says 172.21.15.18:37260) 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc925c00640 1 -- 172.21.15.18:0/2419797086 learned_addr learned my addr 172.21.15.18:0/2419797086 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc925200640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad83262c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.895+0000 7fc926600640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83263370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.896+0000 7fc925c00640 1 -- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 msgr2=0x55ad83263370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.896+0000 7fc925c00640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83263370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.896+0000 7fc925c00640 1 -- 172.21.15.18:0/2419797086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 msgr2=0x55ad83262c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.896+0000 7fc925c00640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad83262c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.896+0000 7fc925c00640 1 -- 172.21.15.18:0/2419797086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad82fa0620 con 0x55ad8325c8a0 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.896+0000 7fc926600640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83263370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.896+0000 7fc925200640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad83262c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.897+0000 7fc925c00640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad832658a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc914009900 tx=0x7fc914009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a9e1a807944bdc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.897+0000 7fc907400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:15:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.897+0000 7fc907400640 1 -- 172.21.15.18:0/2419797086 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914050070 con 0x55ad8325c8a0 2024-04-16T17:15:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.897+0000 7fc907400640 1 -- 172.21.15.18:0/2419797086 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fc914004ad0 con 0x55ad8325c8a0 2024-04-16T17:15:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.897+0000 7fc92b3b1740 1 -- 172.21.15.18:0/2419797086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc90c005500 con 0x55ad8325c8a0 2024-04-16T17:15:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.897+0000 7fc907400640 1 -- 172.21.15.18:0/2419797086 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc91405c050 con 0x55ad8325c8a0 2024-04-16T17:15:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.897+0000 7fc92b3b1740 10 client.28444 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.897+0000 7fc92b3b1740 1 -- 172.21.15.18:0/2419797086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ad83274c20 con 0x55ad8325c8a0 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc907400640 1 -- 172.21.15.18:0/2419797086 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc90c005500 con 0x55ad8325c8a0 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc907400640 1 -- 172.21.15.18:0/2419797086 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc914003850 con 0x55ad8325c8a0 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc92b3b1740 10 client.28444 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc92b3b1740 10 client.28444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc92b3b1740 20 client.28444 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ad832760e0 0x55ad83278510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc92b3b1740 4 client.28444 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc92b3b1740 1 -- 172.21.15.18:0/2419797086 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad831cf1a0 con 0x55ad832760e0 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.898+0000 7fc925200640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ad832760e0 0x55ad83278510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.899+0000 7fc907400640 10 client.28444 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.899+0000 7fc925200640 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ad832760e0 0x55ad83278510 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a93f682a5eb0a41 server_cookie=19e4f567a5c75d1 in_seq=0 out_seq=0 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc907400640 1 -- 172.21.15.18:0/2419797086 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad831cf1a0 con 0x55ad832760e0 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc907400640 10 client.28444 handle_command_reply: tid=0 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 client.28444 shutdown 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 -- 172.21.15.18:0/2419797086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 msgr2=0x55ad832658a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad832658a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc914009900 tx=0x7fc914009260 comp rx=0 tx=0).stop 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 -- 172.21.15.18:0/2419797086 shutdown_connections 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ad832760e0 0x55ad83278510 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad83263370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad83262c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/2419797086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad832658a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 -- 172.21.15.18:0/2419797086 >> 172.21.15.18:0/2419797086 conn(0x55ad831f75d0 msgr2=0x55ad832592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc907400640 0 client.28444 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 -- 172.21.15.18:0/2419797086 shutdown_connections 2024-04-16T17:15:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 1 -- 172.21.15.18:0/2419797086 wait complete. 2024-04-16T17:15:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.901+0000 7fc92b3b1740 20 client.28444 trim_cache size 0 max 16384 2024-04-16T17:15:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc92b3b1740 1 Processor -- start 2024-04-16T17:15:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc92b3b1740 1 -- start start 2024-04-16T17:15:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad83271a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad832568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad8327de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201afd30 con 0x55ad8325d610 2024-04-16T17:15:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201ad600 con 0x55ad83265e20 2024-04-16T17:15:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201ab470 con 0x55ad8325c8a0 2024-04-16T17:15:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad832568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad832568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53584/0 (socket says 172.21.15.18:53584) 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925200640 1 -- 172.21.15.18:0/601296709 learned_addr learned my addr 172.21.15.18:0/601296709 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad83271a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc926600640 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad8327de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925c00640 1 -- 172.21.15.18:0/601296709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 msgr2=0x55ad8327de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925c00640 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad8327de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925c00640 1 -- 172.21.15.18:0/601296709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 msgr2=0x55ad832568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925c00640 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad832568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925c00640 1 -- 172.21.15.18:0/601296709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad83251ac0 con 0x55ad8325c8a0 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc926600640 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad8327de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc925c00640 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad83271a50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc914003e10 tx=0x7fc91402c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637d6bb0f05e20d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc912000640 1 -- 172.21.15.18:0/601296709 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914064070 con 0x55ad8325c8a0 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc912000640 1 -- 172.21.15.18:0/601296709 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc914003e90 con 0x55ad8325c8a0 2024-04-16T17:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.917+0000 7fc912000640 1 -- 172.21.15.18:0/601296709 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914050050 con 0x55ad8325c8a0 2024-04-16T17:15:45.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:45 smithi022 ceph-mon[36023]: pgmap v347: 129 pgs: 129 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 978 KiB/s wr, 140 op/s 2024-04-16T17:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- 172.21.15.18:0/601296709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 msgr2=0x55ad83271a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad83271a50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc914003e10 tx=0x7fc91402c6b0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- 172.21.15.18:0/601296709 shutdown_connections 2024-04-16T17:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad83265e20 0x55ad8327de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad8325d610 0x55ad832568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/601296709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad83271a50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- 172.21.15.18:0/601296709 >> 172.21.15.18:0/601296709 conn(0x55ad831f75d0 msgr2=0x55ad83259110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- 172.21.15.18:0/601296709 shutdown_connections 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- 172.21.15.18:0/601296709 wait complete. 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- start start 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 Processor -- start 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad8326e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad8325d610 0x55ad8325f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad83265e20 0x55ad8325fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201abac0 con 0x55ad83265e20 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201acf30 con 0x55ad8325d610 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.922+0000 7fc92b3b1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201ac190 con 0x55ad8325c8a0 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 10 client.0 _ll_register_callbacks cb 0x7fc914009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc926600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad83265e20 0x55ad8325fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc925c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad8326e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc926600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad83265e20 0x55ad8325fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53600/0 (socket says 172.21.15.18:53600) 2024-04-16T17:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc926600640 1 -- 172.21.15.18:0/3684843470 learned_addr learned my addr 172.21.15.18:0/3684843470 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc925200640 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad8325d610 0x55ad8325f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc925200640 1 -- 172.21.15.18:0/3684843470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 msgr2=0x55ad8326e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc925200640 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad8326e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc925200640 1 -- 172.21.15.18:0/3684843470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad83265e20 msgr2=0x55ad8325fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc925200640 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad83265e20 0x55ad8325fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc925200640 1 -- 172.21.15.18:0/3684843470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad832600b0 con 0x55ad8325d610 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc926600640 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad83265e20 0x55ad8325fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc925200640 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad8325d610 0x55ad8325f500 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0058c0 tx=0x7fc90c010930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1274b0630dd615 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc907400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc907400640 1 -- 172.21.15.18:0/3684843470 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c005200 con 0x55ad8325d610 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc907400640 1 -- 172.21.15.18:0/3684843470 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fc90c0053a0 con 0x55ad8325d610 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 1 -- 172.21.15.18:0/3684843470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc914003e90 con 0x55ad8325d610 2024-04-16T17:15:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc907400640 1 -- 172.21.15.18:0/3684843470 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc90c0055c0 con 0x55ad8325d610 2024-04-16T17:15:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 10 client.28589 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 1 -- 172.21.15.18:0/3684843470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ad832dc530 con 0x55ad8325d610 2024-04-16T17:15:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc907400640 1 -- 172.21.15.18:0/3684843470 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc914003e90 con 0x55ad8325d610 2024-04-16T17:15:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc907400640 1 -- 172.21.15.18:0/3684843470 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc90c01f040 con 0x55ad8325d610 2024-04-16T17:15:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 10 client.28589 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 10 client.28589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 20 client.28589 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ad832dc9e0 0x55ad832dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 4 client.28589 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:15:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.924+0000 7fc92b3b1740 1 -- 172.21.15.18:0/3684843470 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad831c9010 con 0x55ad832dc9e0 2024-04-16T17:15:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.939+0000 7fc925c00640 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ad832dc9e0 0x55ad832dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.947+0000 7fc925c00640 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ad832dc9e0 0x55ad832dee00 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ce6328a267ad4e server_cookie=d4e9c2beaa3d3173 in_seq=0 out_seq=0 2024-04-16T17:15:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:45.947+0000 7fc907400640 10 client.28589 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:45 smithi043 ceph-mon[36901]: pgmap v347: 129 pgs: 129 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 978 KiB/s wr, 140 op/s 2024-04-16T17:15:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc907400640 1 -- 172.21.15.18:0/3684843470 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad831c9010 con 0x55ad832dc9e0 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc907400640 10 client.28589 handle_command_reply: tid=0 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 client.28589 shutdown 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 -- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad8325d610 msgr2=0x55ad8325f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad8325d610 0x55ad8325f500 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0058c0 tx=0x7fc90c010930 comp rx=0 tx=0).stop 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 -- 172.21.15.18:0/3684843470 shutdown_connections 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ad832dc9e0 0x55ad832dee00 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ad83265e20 0x55ad8325fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad8325d610 0x55ad8325f500 secure :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0058c0 tx=0x7fc90c010930 comp rx=0 tx=0).stop 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 --2- 172.21.15.18:0/3684843470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad8325c8a0 0x55ad8326e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 -- 172.21.15.18:0/3684843470 >> 172.21.15.18:0/3684843470 conn(0x55ad831f75d0 msgr2=0x55ad83270a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc907400640 0 client.28589 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 -- 172.21.15.18:0/3684843470 shutdown_connections 2024-04-16T17:15:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 1 -- 172.21.15.18:0/3684843470 wait complete. 2024-04-16T17:15:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc92b3b1740 20 client.28589 trim_cache size 0 max 16384 2024-04-16T17:15:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:15:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:15:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:15:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:15:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 -- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9080a3d10 msgr2=0x7fc9080a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9080a3d10 0x7fc9080a61d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc914006010 tx=0x7fc914005b50 comp rx=0 tx=0).stop 2024-04-16T17:15:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 -- 172.21.15.18:0/3264583462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc920198800 msgr2=0x7fc920198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc920198800 0x7fc920198cb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc91800efd0 tx=0x7fc91800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 -- 172.21.15.18:0/3264583462 shutdown_connections 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9080a3d10 0x7fc9080a61d0 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc914006010 tx=0x7fc914005b50 comp rx=0 tx=0).stop 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc920198800 0x7fc920198cb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc920107330 0x7fc92019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 --2- 172.21.15.18:0/3264583462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc920102260 0x7fc92019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 -- 172.21.15.18:0/3264583462 >> 172.21.15.18:0/3264583462 conn(0x7fc9200fdac0 msgr2=0x7fc9200fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 -- 172.21.15.18:0/3264583462 shutdown_connections 2024-04-16T17:15:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:46.112+0000 7fc928600640 1 -- 172.21.15.18:0/3264583462 wait complete. 2024-04-16T17:15:46.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 415401 3.63 MB/sec execute 479 sec latency 83.948 ms 2024-04-16T17:15:47.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 415643 3.62 MB/sec execute 480 sec latency 62.344 ms 2024-04-16T17:15:47.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:47 smithi022 ceph-mon[36023]: pgmap v348: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-16T17:15:48.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:47 smithi043 ceph-mon[36901]: pgmap v348: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-16T17:15:48.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:45 smithi018 ceph-mon[31776]: pgmap v347: 129 pgs: 129 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 978 KiB/s wr, 140 op/s 2024-04-16T17:15:48.022 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:47 smithi018 ceph-mon[31776]: pgmap v348: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-16T17:15:48.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 416201 3.62 MB/sec execute 481 sec latency 49.246 ms 2024-04-16T17:15:49.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 416819 3.62 MB/sec execute 482 sec latency 54.714 ms 2024-04-16T17:15:49.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:49 smithi022 ceph-mon[36023]: pgmap v349: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.0 MiB/s wr, 65 op/s 2024-04-16T17:15:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:49 smithi043 ceph-mon[36901]: pgmap v349: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.0 MiB/s wr, 65 op/s 2024-04-16T17:15:50.650 INFO:tasks.workunit.client.0.smithi018.stdout: 1 417917 3.63 MB/sec execute 483 sec latency 45.060 ms 2024-04-16T17:15:51.651 INFO:tasks.workunit.client.0.smithi018.stdout: 1 418530 3.63 MB/sec execute 484 sec latency 92.864 ms 2024-04-16T17:15:51.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:51 smithi022 ceph-mon[36023]: pgmap v350: 129 pgs: 129 active+clean; 50 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-04-16T17:15:52.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:51 smithi043 ceph-mon[36901]: pgmap v350: 129 pgs: 129 active+clean; 50 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-04-16T17:15:52.657 INFO:tasks.workunit.client.0.smithi018.stdout: 1 418787 3.63 MB/sec execute 485 sec latency 143.074 ms 2024-04-16T17:15:53.657 INFO:tasks.workunit.client.0.smithi018.stdout: 1 419260 3.62 MB/sec execute 486 sec latency 44.602 ms 2024-04-16T17:15:53.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:53 smithi022 ceph-mon[36023]: pgmap v351: 129 pgs: 129 active+clean; 46 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-16T17:15:54.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:53 smithi043 ceph-mon[36901]: pgmap v351: 129 pgs: 129 active+clean; 46 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-16T17:15:54.657 INFO:tasks.workunit.client.0.smithi018.stdout: 1 420089 3.62 MB/sec execute 487 sec latency 69.207 ms 2024-04-16T17:15:55.657 INFO:tasks.workunit.client.0.smithi018.stdout: 1 420984 3.62 MB/sec execute 488 sec latency 43.852 ms 2024-04-16T17:15:55.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:55 smithi022 ceph-mon[36023]: pgmap v352: 129 pgs: 129 active+clean; 46 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-04-16T17:15:56.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:55 smithi043 ceph-mon[36901]: pgmap v352: 129 pgs: 129 active+clean; 46 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-04-16T17:15:56.657 INFO:tasks.workunit.client.0.smithi018.stdout: 1 421511 3.62 MB/sec execute 489 sec latency 89.415 ms 2024-04-16T17:15:57.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:49 smithi018 ceph-mon[31776]: pgmap v349: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.0 MiB/s wr, 65 op/s 2024-04-16T17:15:57.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:51 smithi018 ceph-mon[31776]: pgmap v350: 129 pgs: 129 active+clean; 50 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-04-16T17:15:57.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:53 smithi018 ceph-mon[31776]: pgmap v351: 129 pgs: 129 active+clean; 46 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-16T17:15:57.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:55 smithi018 ceph-mon[31776]: pgmap v352: 129 pgs: 129 active+clean; 46 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-04-16T17:15:57.657 INFO:tasks.workunit.client.0.smithi018.stdout: 1 421896 3.62 MB/sec execute 490 sec latency 75.756 ms 2024-04-16T17:15:57.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:57 smithi022 ceph-mon[36023]: pgmap v353: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-04-16T17:15:58.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:57 smithi043 ceph-mon[36901]: pgmap v353: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-04-16T17:15:58.657 INFO:tasks.workunit.client.0.smithi018.stdout: 1 422530 3.62 MB/sec execute 491 sec latency 82.002 ms 2024-04-16T17:15:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.256+0000 7f55d2400640 1 -- 172.21.15.18:0/3144701285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc109540 msgr2=0x7f55cc111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.256+0000 7f55d2400640 1 --2- 172.21.15.18:0/3144701285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc109540 0x7f55cc111110 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009ca0 tx=0x7f55c402c100 comp rx=0 tx=0).stop 2024-04-16T17:15:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.256+0000 7f55d2400640 1 -- 172.21.15.18:0/3144701285 shutdown_connections 2024-04-16T17:15:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.256+0000 7f55d2400640 1 --2- 172.21.15.18:0/3144701285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc109540 0x7f55cc111110 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.256+0000 7f55d2400640 1 --2- 172.21.15.18:0/3144701285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55cc104e20 0x7f55cc108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.256+0000 7f55d2400640 1 --2- 172.21.15.18:0/3144701285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55cc104470 0x7f55cc104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.256+0000 7f55d2400640 1 -- 172.21.15.18:0/3144701285 >> 172.21.15.18:0/3144701285 conn(0x7f55cc0ffcd0 msgr2=0x7f55cc1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.256+0000 7f55d2400640 1 -- 172.21.15.18:0/3144701285 shutdown_connections 2024-04-16T17:15:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.257+0000 7f55d2400640 1 -- 172.21.15.18:0/3144701285 wait complete. 2024-04-16T17:15:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55d2400640 1 Processor -- start 2024-04-16T17:15:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55d2400640 1 -- start start 2024-04-16T17:15:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55d2400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc104470 0x7f55cc1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55d2400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55cc109540 0x7f55cc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55d2400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55cc19a940 0x7f55cc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55d2400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1145e0 con 0x7f55cc109540 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55d2400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc114460 con 0x7f55cc104470 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55d2400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc114760 con 0x7f55cc19a940 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc104470 0x7f55cc1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55caa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55cc109540 0x7f55cc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc104470 0x7f55cc1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35716/0 (socket says 172.21.15.18:35716) 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.258+0000 7f55cb400640 1 -- 172.21.15.18:0/2846447015 learned_addr learned my addr 172.21.15.18:0/2846447015 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.259+0000 7f55cbe00640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55cc19a940 0x7f55cc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.259+0000 7f55caa00640 1 -- 172.21.15.18:0/2846447015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55cc19a940 msgr2=0x7f55cc19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.259+0000 7f55caa00640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55cc19a940 0x7f55cc19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.259+0000 7f55caa00640 1 -- 172.21.15.18:0/2846447015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc104470 msgr2=0x7f55cc1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.259+0000 7f55caa00640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc104470 0x7f55cc1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.259+0000 7f55caa00640 1 -- 172.21.15.18:0/2846447015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc19b460 con 0x7f55cc109540 2024-04-16T17:15:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.259+0000 7f55caa00640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55cc109540 0x7f55cc1a0c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f55b800e9c0 tx=0x7f55b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.260+0000 7f55c3e00640 1 -- 172.21.15.18:0/2846447015 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b800ccc0 con 0x7f55cc109540 2024-04-16T17:15:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.260+0000 7f55c3e00640 1 -- 172.21.15.18:0/2846447015 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8010910 con 0x7f55cc109540 2024-04-16T17:15:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.260+0000 7f55c3e00640 1 -- 172.21.15.18:0/2846447015 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80188c0 con 0x7f55cc109540 2024-04-16T17:15:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.260+0000 7f55d2400640 1 -- 172.21.15.18:0/2846447015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc19b750 con 0x7f55cc109540 2024-04-16T17:15:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.260+0000 7f55d2400640 1 -- 172.21.15.18:0/2846447015 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc1a79a0 con 0x7f55cc109540 2024-04-16T17:15:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.263+0000 7f55d51c3740 1 Processor -- start 2024-04-16T17:15:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55d51c3740 1 -- start start 2024-04-16T17:15:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55c3e00640 1 -- 172.21.15.18:0/2846447015 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f55b8010430 con 0x7f55cc109540 2024-04-16T17:15:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e20a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55d51c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fdc6d71780 con 0x55fdc6e1e630 2024-04-16T17:15:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55d51c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fdc6d17f40 con 0x55fdc6e0d610 2024-04-16T17:15:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55d51c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fdc6d6d740 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35724/0 (socket says 172.21.15.18:35724) 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.264+0000 7f55cbe00640 1 -- 172.21.15.18:0/3133446245 learned_addr learned my addr 172.21.15.18:0/3133446245 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55cb400640 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55caa00640 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e20a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55cbe00640 1 -- 172.21.15.18:0/3133446245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 msgr2=0x55fdc6e0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55cbe00640 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e0cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55cbe00640 1 -- 172.21.15.18:0/3133446245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 msgr2=0x55fdc6e20a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55cbe00640 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e20a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55cbe00640 1 -- 172.21.15.18:0/3133446245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc6a84210 con 0x55fdc6e0d610 2024-04-16T17:15:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55cb400640 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.265+0000 7f55cbe00640 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0c360 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f55c402c710 tx=0x7f55c4005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42678c80aed5a95b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.266+0000 7f55c0c00640 1 -- 172.21.15.18:0/3133446245 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4043070 con 0x55fdc6e0d610 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.266+0000 7f55c0c00640 1 -- 172.21.15.18:0/3133446245 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c4032c90 con 0x55fdc6e0d610 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.266+0000 7f55c0c00640 1 -- 172.21.15.18:0/3133446245 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4039040 con 0x55fdc6e0d610 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.266+0000 7f55c3e00640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55b00a3d90 0x7f55b00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.266+0000 7f55c3e00640 1 -- 172.21.15.18:0/2846447015 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f55b80d0ed0 con 0x7f55cc109540 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.266+0000 7f55cb400640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55b00a3d90 0x7f55b00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55d51c3740 1 -- 172.21.15.18:0/3133446245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 msgr2=0x55fdc6e0c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55d51c3740 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0c360 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f55c402c710 tx=0x7f55c4005f80 comp rx=0 tx=0).stop 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55d51c3740 1 -- 172.21.15.18:0/3133446245 shutdown_connections 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55d51c3740 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e20a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55d51c3740 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55d51c3740 1 --2- 172.21.15.18:0/3133446245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0c360 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55d51c3740 1 -- 172.21.15.18:0/3133446245 >> 172.21.15.18:0/3133446245 conn(0x55fdc6da75d0 msgr2=0x55fdc6e07590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55cb400640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55b00a3d90 0x7f55b00a6250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55fdc6e0dd70 tx=0x7f55bc00b4a0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.267+0000 7f55d51c3740 1 -- 172.21.15.18:0/3133446245 shutdown_connections 2024-04-16T17:15:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.268+0000 7f55d51c3740 1 -- 172.21.15.18:0/3133446245 wait complete. 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.268+0000 7f55d51c3740 1 -- start start 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.268+0000 7f55d51c3740 1 Processor -- start 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e1e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e1b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55d51c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fdc6e21300 con 0x55fdc6e1e630 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55d51c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fdc6d2b390 con 0x55fdc6e0d610 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55d51c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fdc6e21480 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e1b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55d51c3740 10 client.0 _ll_register_callbacks cb 0x55fdc6d14f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e1e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e1b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35738/0 (socket says 172.21.15.18:35738) 2024-04-16T17:15:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.269+0000 7f55caa00640 1 -- 172.21.15.18:0/2625048539 learned_addr learned my addr 172.21.15.18:0/2625048539 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55cb400640 1 -- 172.21.15.18:0/2625048539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 msgr2=0x55fdc6e1b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55cb400640 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e1b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55cbe00640 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55cb400640 1 -- 172.21.15.18:0/2625048539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 msgr2=0x55fdc6e1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55cb400640 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55cb400640 1 -- 172.21.15.18:0/2625048539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc6b50620 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55cbe00640 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55cb400640 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e1e0b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005fd0 tx=0x7f55bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772dbaef59e25a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.270+0000 7f55b5600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.271+0000 7f55b5600640 1 -- 172.21.15.18:0/2625048539 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc01b870 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.271+0000 7f55d51c3740 1 -- 172.21.15.18:0/2625048539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c4032c90 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.271+0000 7f55b5600640 1 -- 172.21.15.18:0/2625048539 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f55bc01ba10 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.271+0000 7f55b5600640 1 -- 172.21.15.18:0/2625048539 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55bc02f050 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.271+0000 7f55d51c3740 10 client.28459 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.271+0000 7f55d51c3740 1 -- 172.21.15.18:0/2625048539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fdc6e2d480 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.271+0000 7f55b5600640 1 -- 172.21.15.18:0/2625048539 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c4032c90 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.272+0000 7f55b5600640 1 -- 172.21.15.18:0/2625048539 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f55bc019040 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.272+0000 7f55d51c3740 10 client.28459 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.272+0000 7f55d51c3740 10 client.28459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.272+0000 7f55d51c3740 20 client.28459 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.272+0000 7f55d51c3740 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fdc6e2e8f0 0x55fdc6e30ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.272+0000 7f55d51c3740 4 client.28459 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.272+0000 7f55d51c3740 1 -- 172.21.15.18:0/2625048539 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdc6d7f1a0 con 0x55fdc6e2e8f0 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.272+0000 7f55caa00640 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fdc6e2e8f0 0x55fdc6e30ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.273+0000 7f55caa00640 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fdc6e2e8f0 0x55fdc6e30ce0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2770eff531706793 server_cookie=33a2c9813a2c3c33 in_seq=0 out_seq=0 2024-04-16T17:15:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.273+0000 7f55b5600640 10 client.28459 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.276+0000 7f55b5600640 1 -- 172.21.15.18:0/2625048539 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fdc6d7f1a0 con 0x55fdc6e2e8f0 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.276+0000 7f55b5600640 10 client.28459 handle_command_reply: tid=0 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.276+0000 7f55d51c3740 1 client.28459 shutdown 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 -- 172.21.15.18:0/2625048539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 msgr2=0x55fdc6e1e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e1e0b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005fd0 tx=0x7f55bc0046c0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 -- 172.21.15.18:0/2625048539 shutdown_connections 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fdc6e2e8f0 0x55fdc6e30ce0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e1b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55b5600640 0 client.28459 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 --2- 172.21.15.18:0/2625048539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e1e0b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 -- 172.21.15.18:0/2625048539 >> 172.21.15.18:0/2625048539 conn(0x55fdc6da75d0 msgr2=0x55fdc6e092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 -- 172.21.15.18:0/2625048539 shutdown_connections 2024-04-16T17:15:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 1 -- 172.21.15.18:0/2625048539 wait complete. 2024-04-16T17:15:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.277+0000 7f55d51c3740 20 client.28459 trim_cache size 0 max 16384 2024-04-16T17:15:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.309+0000 7f55d51c3740 1 Processor -- start 2024-04-16T17:15:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.309+0000 7f55d51c3740 1 -- start start 2024-04-16T17:15:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e2a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55d51c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1afda0 con 0x55fdc6e1e630 2024-04-16T17:15:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55d51c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ad670 con 0x55fdc6e0d610 2024-04-16T17:15:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55d51c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ab4e0 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e2a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e2a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50154/0 (socket says 172.21.15.18:50154) 2024-04-16T17:15:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.310+0000 7f55cb400640 1 -- 172.21.15.18:0/1716825115 learned_addr learned my addr 172.21.15.18:0/1716825115 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.311+0000 7f55cbe00640 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.311+0000 7f55caa00640 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.311+0000 7f55cb400640 1 -- 172.21.15.18:0/1716825115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 msgr2=0x55fdc6e0bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.311+0000 7f55cb400640 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.311+0000 7f55cb400640 1 -- 172.21.15.18:0/1716825115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 msgr2=0x55fdc6e1a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.311+0000 7f55cb400640 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.311+0000 7f55cb400640 1 -- 172.21.15.18:0/1716825115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc6e1bef0 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.311+0000 7f55cb400640 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e2a200 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02d040 tx=0x7f55bc00b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b083cd86788443 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.312+0000 7f55b7e00640 1 -- 172.21.15.18:0/1716825115 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc004510 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.312+0000 7f55b7e00640 1 -- 172.21.15.18:0/1716825115 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc038070 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.312+0000 7f55b7e00640 1 -- 172.21.15.18:0/1716825115 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc004730 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 -- 172.21.15.18:0/1716825115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 msgr2=0x55fdc6e2a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e2a200 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02d040 tx=0x7f55bc00b5b0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 -- 172.21.15.18:0/1716825115 shutdown_connections 2024-04-16T17:15:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e1a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0d610 0x55fdc6e0bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1716825115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e2a200 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 -- 172.21.15.18:0/1716825115 >> 172.21.15.18:0/1716825115 conn(0x55fdc6da75d0 msgr2=0x55fdc6e19d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 -- 172.21.15.18:0/1716825115 shutdown_connections 2024-04-16T17:15:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.313+0000 7f55d51c3740 1 -- 172.21.15.18:0/1716825115 wait complete. 2024-04-16T17:15:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.314+0000 7f55d51c3740 1 -- start start 2024-04-16T17:15:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.314+0000 7f55d51c3740 1 Processor -- start 2024-04-16T17:15:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0d610 0x55fdc6e18800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55d51c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e18d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55d51c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1abb30 con 0x55fdc6e1e630 2024-04-16T17:15:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55d51c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1acfa0 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55d51c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ac200 con 0x55fdc6e0d610 2024-04-16T17:15:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55d51c3740 10 client.0 _ll_register_callbacks cb 0x7f55bc005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0d610 0x55fdc6e18800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.315+0000 7f55caa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0d610 0x55fdc6e18800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50170/0 (socket says 172.21.15.18:50170) 2024-04-16T17:15:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55caa00640 1 -- 172.21.15.18:0/1371099689 learned_addr learned my addr 172.21.15.18:0/1371099689 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:15:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55cb400640 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55cb400640 1 -- 172.21.15.18:0/1371099689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0d610 msgr2=0x55fdc6e18800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55cb400640 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0d610 0x55fdc6e18800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55cb400640 1 -- 172.21.15.18:0/1371099689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 msgr2=0x55fdc6e18d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55cb400640 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e18d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55cb400640 1 -- 172.21.15.18:0/1371099689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc6e93f70 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55cb400640 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e182c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f55bc026db0 tx=0x7f55bc012df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf82d8c650c30e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.316+0000 7f55b5600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55b5600640 1 -- 172.21.15.18:0/1371099689 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc010c80 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55d51c3740 1 -- 172.21.15.18:0/1371099689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55bc038070 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55b5600640 1 -- 172.21.15.18:0/1371099689 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f55bc010e20 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55b5600640 1 -- 172.21.15.18:0/1371099689 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55bc049050 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55d51c3740 10 client.28607 fetch_fsmap learned FSMap version 31 2024-04-16T17:15:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55d51c3740 1 -- 172.21.15.18:0/1371099689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fdc6e95260 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55b5600640 1 -- 172.21.15.18:0/1371099689 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc038070 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55b5600640 1 -- 172.21.15.18:0/1371099689 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f55bc019070 con 0x55fdc6e0c8a0 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55d51c3740 10 client.28607 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.317+0000 7f55d51c3740 10 client.28607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.318+0000 7f55d51c3740 20 client.28607 populate_metadata read hostname 'smithi018' 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.318+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fdc6e95860 0x55fdc6e97c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.318+0000 7f55d51c3740 4 client.28607 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.318+0000 7f55d51c3740 1 -- 172.21.15.18:0/1371099689 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdc6d79010 con 0x55fdc6e95860 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.318+0000 7f55caa00640 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fdc6e95860 0x55fdc6e97c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.318+0000 7f55caa00640 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fdc6e95860 0x55fdc6e97c50 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592d8e1916e3404c server_cookie=87ae2d32d97c3ed0 in_seq=0 out_seq=0 2024-04-16T17:15:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.318+0000 7f55b5600640 10 client.28607 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:59.481 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:57 smithi018 ceph-mon[31776]: pgmap v353: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-04-16T17:15:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.502+0000 7f55b5600640 1 -- 172.21.15.18:0/1371099689 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fdc6d79010 con 0x55fdc6e95860 2024-04-16T17:15:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.502+0000 7f55b5600640 10 client.28607 handle_command_reply: tid=0 2024-04-16T17:15:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.502+0000 7f55d51c3740 1 client.28607 shutdown 2024-04-16T17:15:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.503+0000 7f55d51c3740 1 -- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0c8a0 msgr2=0x55fdc6e182c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.503+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e182c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f55bc026db0 tx=0x7f55bc012df0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55d51c3740 1 -- 172.21.15.18:0/1371099689 shutdown_connections 2024-04-16T17:15:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fdc6e95860 0x55fdc6e97c50 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fdc6e1e630 0x55fdc6e18d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdc6e0d610 0x55fdc6e18800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55d51c3740 1 --2- 172.21.15.18:0/1371099689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fdc6e0c8a0 0x55fdc6e182c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55d51c3740 1 -- 172.21.15.18:0/1371099689 >> 172.21.15.18:0/1371099689 conn(0x55fdc6da75d0 msgr2=0x55fdc6e066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55b5600640 0 client.28607 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:15:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55d51c3740 1 -- 172.21.15.18:0/1371099689 shutdown_connections 2024-04-16T17:15:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.504+0000 7f55d51c3740 1 -- 172.21.15.18:0/1371099689 wait complete. 2024-04-16T17:15:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.505+0000 7f55d51c3740 20 client.28607 trim_cache size 0 max 16384 2024-04-16T17:15:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:15:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:15:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:15:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:15:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 -- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55b00a3d90 msgr2=0x7f55b00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55b00a3d90 0x7f55b00a6250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55fdc6e0dd70 tx=0x7f55bc00b4a0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 -- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55cc109540 msgr2=0x7f55cc1a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55cc109540 0x7f55cc1a0c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f55b800e9c0 tx=0x7f55b800ee80 comp rx=0 tx=0).stop 2024-04-16T17:15:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 -- 172.21.15.18:0/2846447015 shutdown_connections 2024-04-16T17:15:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f55b00a3d90 0x7f55b00a6250 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55cc19a940 0x7f55cc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55cc109540 0x7f55cc1a0c80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 --2- 172.21.15.18:0/2846447015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55cc104470 0x7f55cc1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.506+0000 7f55d2400640 1 -- 172.21.15.18:0/2846447015 >> 172.21.15.18:0/2846447015 conn(0x7f55cc0ffcd0 msgr2=0x7f55cc102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.507+0000 7f55d2400640 1 -- 172.21.15.18:0/2846447015 shutdown_connections 2024-04-16T17:15:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:15:59.507+0000 7f55d2400640 1 -- 172.21.15.18:0/2846447015 wait complete. 2024-04-16T17:15:59.658 INFO:tasks.workunit.client.0.smithi018.stdout: 1 423170 3.62 MB/sec execute 492 sec latency 67.281 ms 2024-04-16T17:15:59.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:15:59 smithi022 ceph-mon[36023]: pgmap v354: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1006 KiB/s wr, 126 op/s 2024-04-16T17:15:59.923 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:15:59 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setuser ceph since I am not root 2024-04-16T17:15:59.924 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:15:59 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ignoring --setgroup ceph since I am not root 2024-04-16T17:15:59.924 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:15:59 smithi022 ceph-mds[66065]: -- 172.21.15.22:0/1621103800 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c323073a40 con 0x55c323047000 2024-04-16T17:15:59.924 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:15:59 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: starting mds.h at 2024-04-16T17:16:00.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:15:59 smithi043 ceph-mon[36901]: pgmap v354: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1006 KiB/s wr, 126 op/s 2024-04-16T17:16:00.657 INFO:tasks.workunit.client.0.smithi018.stdout: 1 423909 3.62 MB/sec execute 493 sec latency 39.767 ms 2024-04-16T17:16:00.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:00 smithi022 ceph-mon[36023]: daemon mds.h restarted 2024-04-16T17:16:01.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:00 smithi043 ceph-mon[36901]: daemon mds.h restarted 2024-04-16T17:16:01.658 INFO:tasks.workunit.client.0.smithi018.stdout: 1 424304 3.61 MB/sec execute 494 sec latency 82.312 ms 2024-04-16T17:16:01.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:01 smithi022 ceph-mon[36023]: pgmap v355: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-04-16T17:16:01.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:01 smithi022 ceph-mon[36023]: osdmap e109: 12 total, 12 up, 12 in 2024-04-16T17:16:01.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:01 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:16:01.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:01 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6836/3815032359,v1:172.21.15.22:6837/3815032359] up:boot 2024-04-16T17:16:01.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:01 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:16:01.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:01 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:16:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:01 smithi043 ceph-mon[36901]: pgmap v355: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-04-16T17:16:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:01 smithi043 ceph-mon[36901]: osdmap e109: 12 total, 12 up, 12 in 2024-04-16T17:16:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:01 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:16:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:01 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6836/3815032359,v1:172.21.15.22:6837/3815032359] up:boot 2024-04-16T17:16:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:01 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:16:02.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:01 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:16:02.334 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:15:59 smithi018 ceph-mon[31776]: pgmap v354: 129 pgs: 129 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1006 KiB/s wr, 126 op/s 2024-04-16T17:16:02.334 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:00 smithi018 ceph-mon[31776]: daemon mds.h restarted 2024-04-16T17:16:02.334 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:01 smithi018 ceph-mon[31776]: pgmap v355: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-04-16T17:16:02.334 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:01 smithi018 ceph-mon[31776]: osdmap e109: 12 total, 12 up, 12 in 2024-04-16T17:16:02.334 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:01 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:16:02.334 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:01 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6836/3815032359,v1:172.21.15.22:6837/3815032359] up:boot 2024-04-16T17:16:02.334 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:01 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:16:02.334 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:01 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:16:02.658 INFO:tasks.workunit.client.0.smithi018.stdout: 1 424420 3.60 MB/sec execute 495 sec latency 74.121 ms 2024-04-16T17:16:03.658 INFO:tasks.workunit.client.0.smithi018.stdout: 1 424604 3.60 MB/sec execute 496 sec latency 82.538 ms 2024-04-16T17:16:03.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:03 smithi022 ceph-mon[36023]: pgmap v357: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-16T17:16:03.970 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:03 smithi018 ceph-mon[31776]: pgmap v357: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-16T17:16:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:03 smithi043 ceph-mon[36901]: pgmap v357: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-16T17:16:04.658 INFO:tasks.workunit.client.0.smithi018.stdout: 1 425807 3.61 MB/sec execute 497 sec latency 37.588 ms 2024-04-16T17:16:05.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:05.111 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:05.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.431+0000 7fcf32200640 1 -- 172.21.15.18:0/1344926358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c0743a0 msgr2=0x7fcf2c074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.431+0000 7fcf32200640 1 --2- 172.21.15.18:0/1344926358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c0743a0 0x7fcf2c074780 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009900 tx=0x7fcf1802bcf0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.431+0000 7fcf32200640 1 -- 172.21.15.18:0/1344926358 shutdown_connections 2024-04-16T17:16:05.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.431+0000 7fcf32200640 1 --2- 172.21.15.18:0/1344926358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf2c112fe0 0x7fcf2c1153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.431+0000 7fcf32200640 1 --2- 172.21.15.18:0/1344926358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c074cc0 0x7fcf2c112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.431+0000 7fcf32200640 1 --2- 172.21.15.18:0/1344926358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c0743a0 0x7fcf2c074780 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.431+0000 7fcf32200640 1 -- 172.21.15.18:0/1344926358 >> 172.21.15.18:0/1344926358 conn(0x7fcf2c0fdac0 msgr2=0x7fcf2c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:05.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.432+0000 7fcf32200640 1 -- 172.21.15.18:0/1344926358 shutdown_connections 2024-04-16T17:16:05.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.432+0000 7fcf32200640 1 -- 172.21.15.18:0/1344926358 wait complete. 2024-04-16T17:16:05.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.433+0000 7fcf32200640 1 Processor -- start 2024-04-16T17:16:05.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.434+0000 7fcf32200640 1 -- start start 2024-04-16T17:16:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.436+0000 7fcf32200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0743a0 0x7fcf2c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.436+0000 7fcf32200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf2c074cc0 0x7fcf2c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.436+0000 7fcf32200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c112fe0 0x7fcf2c1a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.436+0000 7fcf32200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c118a90 con 0x7fcf2c074cc0 2024-04-16T17:16:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.436+0000 7fcf32200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c118910 con 0x7fcf2c112fe0 2024-04-16T17:16:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.436+0000 7fcf32200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c118c10 con 0x7fcf2c0743a0 2024-04-16T17:16:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.437+0000 7fcf2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0743a0 0x7fcf2c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.437+0000 7fcf2aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf2c074cc0 0x7fcf2c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.437+0000 7fcf2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0743a0 0x7fcf2c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50198/0 (socket says 172.21.15.18:50198) 2024-04-16T17:16:05.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.437+0000 7fcf2b400640 1 -- 172.21.15.18:0/1618180599 learned_addr learned my addr 172.21.15.18:0/1618180599 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:05.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.437+0000 7fcf2be00640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c112fe0 0x7fcf2c1a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.438+0000 7fcf2be00640 1 -- 172.21.15.18:0/1618180599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0743a0 msgr2=0x7fcf2c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.438+0000 7fcf2be00640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0743a0 0x7fcf2c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.438+0000 7fcf2be00640 1 -- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf2c074cc0 msgr2=0x7fcf2c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.438+0000 7fcf2be00640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf2c074cc0 0x7fcf2c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.438+0000 7fcf2be00640 1 -- 172.21.15.18:0/1618180599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c1a3f80 con 0x7fcf2c112fe0 2024-04-16T17:16:05.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.438+0000 7fcf2be00640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c112fe0 0x7fcf2c1a3850 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400ca80 tx=0x7fcf2400cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:05.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.439+0000 7fcf23e00640 1 -- 172.21.15.18:0/1618180599 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400ae20 con 0x7fcf2c112fe0 2024-04-16T17:16:05.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.439+0000 7fcf23e00640 1 -- 172.21.15.18:0/1618180599 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2400f930 con 0x7fcf2c112fe0 2024-04-16T17:16:05.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.439+0000 7fcf23e00640 1 -- 172.21.15.18:0/1618180599 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24018ad0 con 0x7fcf2c112fe0 2024-04-16T17:16:05.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.439+0000 7fcf32200640 1 -- 172.21.15.18:0/1618180599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c1a4270 con 0x7fcf2c112fe0 2024-04-16T17:16:05.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.439+0000 7fcf32200640 1 -- 172.21.15.18:0/1618180599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c1abb00 con 0x7fcf2c112fe0 2024-04-16T17:16:05.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.445+0000 7fcf3500e740 1 Processor -- start 2024-04-16T17:16:05.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.446+0000 7fcf23e00640 1 -- 172.21.15.18:0/1618180599 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf2400fae0 con 0x7fcf2c112fe0 2024-04-16T17:16:05.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.446+0000 7fcf3500e740 1 -- start start 2024-04-16T17:16:05.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.447+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.447+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa2ad0 0x557e60fab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.447+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 0x557e60fadaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.447+0000 7fcf2aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.447+0000 7fcf2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa2ad0 0x557e60fab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.447+0000 7fcf2aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35776/0 (socket says 172.21.15.18:35776) 2024-04-16T17:16:05.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.447+0000 7fcf2aa00640 1 -- 172.21.15.18:0/310285260 learned_addr learned my addr 172.21.15.18:0/310285260 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:05.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.447+0000 7fcf2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa2ad0 0x557e60fab170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50206/0 (socket says 172.21.15.18:50206) 2024-04-16T17:16:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.448+0000 7fcf3500e740 1 -- 172.21.15.18:0/310285260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557e60f08120 con 0x557e60fab6b0 2024-04-16T17:16:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.448+0000 7fcf3500e740 1 -- 172.21.15.18:0/310285260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e60eae340 con 0x557e60fa21b0 2024-04-16T17:16:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.448+0000 7fcf3500e740 1 -- 172.21.15.18:0/310285260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557e60f040e0 con 0x557e60fa2ad0 2024-04-16T17:16:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.448+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 0x557e60fadaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.449+0000 7fcf23e00640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcf100a3da0 0x7fcf100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.450+0000 7fcf23e00640 1 -- 172.21.15.18:0/1618180599 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fcf24043020 con 0x7fcf2c112fe0 2024-04-16T17:16:05.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.450+0000 7fcf2aa00640 1 -- 172.21.15.18:0/310285260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa2ad0 msgr2=0x557e60fab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.450+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa2ad0 0x557e60fab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.450+0000 7fcf2aa00640 1 -- 172.21.15.18:0/310285260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 msgr2=0x557e60fadaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.450+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 0x557e60fadaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.450+0000 7fcf2aa00640 1 -- 172.21.15.18:0/310285260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e60ce47d0 con 0x557e60fa21b0 2024-04-16T17:16:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.450+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa2590 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c016500 tx=0x7fcf1c00a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81e65b026d52e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.451+0000 7fcf17e00640 1 -- 172.21.15.18:0/310285260 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c018430 con 0x557e60fa21b0 2024-04-16T17:16:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.451+0000 7fcf17e00640 1 -- 172.21.15.18:0/310285260 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c018ab0 con 0x557e60fa21b0 2024-04-16T17:16:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.451+0000 7fcf17e00640 1 -- 172.21.15.18:0/310285260 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0178f0 con 0x557e60fa21b0 2024-04-16T17:16:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.451+0000 7fcf2b400640 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa2ad0 0x557e60fab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:05.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.451+0000 7fcf2be00640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcf100a3da0 0x7fcf100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 -- 172.21.15.18:0/310285260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 msgr2=0x557e60fa2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa2590 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c016500 tx=0x7fcf1c00a2b0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 -- 172.21.15.18:0/310285260 shutdown_connections 2024-04-16T17:16:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 0x557e60fadaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa2ad0 0x557e60fab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 --2- 172.21.15.18:0/310285260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa2590 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 -- 172.21.15.18:0/310285260 >> 172.21.15.18:0/310285260 conn(0x557e60f9c8e0 msgr2=0x557e60f9ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf2be00640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcf100a3da0 0x7fcf100a6260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf24007310 tx=0x7fcf2400a890 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 -- 172.21.15.18:0/310285260 shutdown_connections 2024-04-16T17:16:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.452+0000 7fcf3500e740 1 -- 172.21.15.18:0/310285260 wait complete. 2024-04-16T17:16:05.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.453+0000 7fcf3500e740 1 -- start start 2024-04-16T17:16:05.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.453+0000 7fcf3500e740 1 Processor -- start 2024-04-16T17:16:05.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.454+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.454+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 0x557e60fa8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.454+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fa9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.454+0000 7fcf3500e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557e60fae440 con 0x557e60fa2ad0 2024-04-16T17:16:05.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.454+0000 7fcf3500e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e60ec1710 con 0x557e60fa21b0 2024-04-16T17:16:05.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.454+0000 7fcf3500e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557e60fae5c0 con 0x557e60fab6b0 2024-04-16T17:16:05.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.454+0000 7fcf3500e740 10 client.0 _ll_register_callbacks cb 0x557e60ebcaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:05.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.455+0000 7fcf2aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.455+0000 7fcf2aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fa9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.455+0000 7fcf2aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fa9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50212/0 (socket says 172.21.15.18:50212) 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.455+0000 7fcf2aa00640 1 -- 172.21.15.18:0/2788757558 learned_addr learned my addr 172.21.15.18:0/2788757558 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.456+0000 7fcf2aa00640 1 -- 172.21.15.18:0/2788757558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 msgr2=0x557e60fa8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.456+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.456+0000 7fcf2aa00640 1 -- 172.21.15.18:0/2788757558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 msgr2=0x557e60fa8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.456+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 0x557e60fa8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.456+0000 7fcf2aa00640 1 -- 172.21.15.18:0/2788757558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e60c1a3f0 con 0x557e60fab6b0 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.456+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fa9200 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c006f60 tx=0x7fcf1c006d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a314058a6656c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.456+0000 7fcf14c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.457+0000 7fcf14c00640 1 -- 172.21.15.18:0/2788757558 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c017e40 con 0x557e60fab6b0 2024-04-16T17:16:05.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.457+0000 7fcf14c00640 1 -- 172.21.15.18:0/2788757558 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fcf1c018870 con 0x557e60fab6b0 2024-04-16T17:16:05.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.457+0000 7fcf3500e740 1 -- 172.21.15.18:0/2788757558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf1c018ab0 con 0x557e60fab6b0 2024-04-16T17:16:05.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.457+0000 7fcf14c00640 1 -- 172.21.15.18:0/2788757558 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcf1c035050 con 0x557e60fab6b0 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.458+0000 7fcf3500e740 10 client.28480 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.458+0000 7fcf3500e740 1 -- 172.21.15.18:0/2788757558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e60bda0f0 con 0x557e60fab6b0 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.458+0000 7fcf14c00640 1 -- 172.21.15.18:0/2788757558 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c018ab0 con 0x557e60fab6b0 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.459+0000 7fcf14c00640 1 -- 172.21.15.18:0/2788757558 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcf1c024040 con 0x557e60fab6b0 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.459+0000 7fcf3500e740 10 client.28480 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.459+0000 7fcf3500e740 10 client.28480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.459+0000 7fcf3500e740 20 client.28480 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.460+0000 7fcf3500e740 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e60fbb7a0 0x557e60fbdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.460+0000 7fcf3500e740 4 client.28480 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.460+0000 7fcf3500e740 1 -- 172.21.15.18:0/2788757558 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e60f15b40 con 0x557e60fbb7a0 2024-04-16T17:16:05.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.460+0000 7fcf2b400640 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e60fbb7a0 0x557e60fbdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.463+0000 7fcf2b400640 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e60fbb7a0 0x557e60fbdb90 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7dec1976377ea36 server_cookie=c123eee5b7a5d571 in_seq=0 out_seq=0 2024-04-16T17:16:05.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.463+0000 7fcf14c00640 10 client.28480 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:05.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.464+0000 7fcf14c00640 1 -- 172.21.15.18:0/2788757558 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e60f15b40 con 0x557e60fbb7a0 2024-04-16T17:16:05.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.464+0000 7fcf14c00640 10 client.28480 handle_command_reply: tid=0 2024-04-16T17:16:05.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.464+0000 7fcf3500e740 1 client.28480 shutdown 2024-04-16T17:16:05.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.465+0000 7fcf3500e740 1 -- 172.21.15.18:0/2788757558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 msgr2=0x557e60fa9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.465+0000 7fcf3500e740 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fa9200 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c006f60 tx=0x7fcf1c006d50 comp rx=0 tx=0).stop 2024-04-16T17:16:05.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 1 -- 172.21.15.18:0/2788757558 shutdown_connections 2024-04-16T17:16:05.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e60fbb7a0 0x557e60fbdb90 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fa9200 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 0x557e60fa8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf14c00640 0 client.28480 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:05.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 1 --2- 172.21.15.18:0/2788757558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60fa8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 1 -- 172.21.15.18:0/2788757558 >> 172.21.15.18:0/2788757558 conn(0x557e60f9c8e0 msgr2=0x557e60f9e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:05.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 1 -- 172.21.15.18:0/2788757558 shutdown_connections 2024-04-16T17:16:05.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 1 -- 172.21.15.18:0/2788757558 wait complete. 2024-04-16T17:16:05.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.466+0000 7fcf3500e740 20 client.28480 trim_cache size 0 max 16384 2024-04-16T17:16:05.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.508+0000 7fcf3500e740 1 Processor -- start 2024-04-16T17:16:05.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.509+0000 7fcf3500e740 1 -- start start 2024-04-16T17:16:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.509+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60faaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.509+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 0x557e60f9b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.509+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fc3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.509+0000 7fcf3500e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b8670 con 0x557e60fa2ad0 2024-04-16T17:16:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.509+0000 7fcf3500e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b5f40 con 0x557e60fa21b0 2024-04-16T17:16:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.509+0000 7fcf3500e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b3db0 con 0x557e60fab6b0 2024-04-16T17:16:05.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.509+0000 7fcf2aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60faaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60faaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35792/0 (socket says 172.21.15.18:35792) 2024-04-16T17:16:05.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 -- 172.21.15.18:0/1867469175 learned_addr learned my addr 172.21.15.18:0/1867469175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fc3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2b400640 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 0x557e60f9b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 -- 172.21.15.18:0/1867469175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 msgr2=0x557e60fc3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fc3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 -- 172.21.15.18:0/1867469175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 msgr2=0x557e60f9b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 0x557e60f9b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 -- 172.21.15.18:0/1867469175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e60fa2900 con 0x557e60fa21b0 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.510+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60faaeb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c016500 tx=0x7fcf1c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9494083733012a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.511+0000 7fcf17400640 1 -- 172.21.15.18:0/1867469175 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c018870 con 0x557e60fa21b0 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.511+0000 7fcf17400640 1 -- 172.21.15.18:0/1867469175 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c02c070 con 0x557e60fa21b0 2024-04-16T17:16:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.511+0000 7fcf17400640 1 -- 172.21.15.18:0/1867469175 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c004310 con 0x557e60fa21b0 2024-04-16T17:16:05.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.512+0000 7fcf3500e740 1 -- 172.21.15.18:0/1867469175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 msgr2=0x557e60faaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.512+0000 7fcf3500e740 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60faaeb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c016500 tx=0x7fcf1c008040 comp rx=0 tx=0).stop 2024-04-16T17:16:05.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.512+0000 7fcf3500e740 1 -- 172.21.15.18:0/1867469175 shutdown_connections 2024-04-16T17:16:05.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.512+0000 7fcf3500e740 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fab6b0 0x557e60fc3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.512+0000 7fcf3500e740 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fa2ad0 0x557e60f9b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.512+0000 7fcf3500e740 1 --2- 172.21.15.18:0/1867469175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa21b0 0x557e60faaeb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.512+0000 7fcf3500e740 1 -- 172.21.15.18:0/1867469175 >> 172.21.15.18:0/1867469175 conn(0x557e60f9c8e0 msgr2=0x557e60f9e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:05.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.513+0000 7fcf3500e740 1 -- 172.21.15.18:0/1867469175 shutdown_connections 2024-04-16T17:16:05.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.513+0000 7fcf3500e740 1 -- 172.21.15.18:0/1867469175 wait complete. 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.513+0000 7fcf3500e740 1 -- start start 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.513+0000 7fcf3500e740 1 Processor -- start 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.514+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa21b0 0x557e60fb3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa2ad0 0x557e60fb0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf3500e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 0x557e60fb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf3500e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b4400 con 0x557e60fab6b0 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf3500e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b5870 con 0x557e60fa2ad0 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf3500e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b4ad0 con 0x557e60fa21b0 2024-04-16T17:16:05.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf3500e740 10 client.0 _ll_register_callbacks cb 0x7fcf1c00ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf2b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa2ad0 0x557e60fb0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf2b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa2ad0 0x557e60fb0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35804/0 (socket says 172.21.15.18:35804) 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf2b400640 1 -- 172.21.15.18:0/234321967 learned_addr learned my addr 172.21.15.18:0/234321967 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.515+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa21b0 0x557e60fb3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.516+0000 7fcf2b400640 1 -- 172.21.15.18:0/234321967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa21b0 msgr2=0x557e60fb3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.516+0000 7fcf2b400640 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa21b0 0x557e60fb3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.516+0000 7fcf2b400640 1 -- 172.21.15.18:0/234321967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 msgr2=0x557e60fb1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.516+0000 7fcf2b400640 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 0x557e60fb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.516+0000 7fcf2b400640 1 -- 172.21.15.18:0/234321967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e60fb1af0 con 0x557e60fa2ad0 2024-04-16T17:16:05.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.517+0000 7fcf2b400640 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa2ad0 0x557e60fb0ec0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcf180096c0 tx=0x7fcf18009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd5d10129c5b362 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:05.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.517+0000 7fcf14c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:05.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.518+0000 7fcf14c00640 1 -- 172.21.15.18:0/234321967 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18046e30 con 0x557e60fa2ad0 2024-04-16T17:16:05.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.518+0000 7fcf14c00640 1 -- 172.21.15.18:0/234321967 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fcf18004330 con 0x557e60fa2ad0 2024-04-16T17:16:05.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.518+0000 7fcf3500e740 1 -- 172.21.15.18:0/234321967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf1c02c070 con 0x557e60fa2ad0 2024-04-16T17:16:05.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.519+0000 7fcf14c00640 1 -- 172.21.15.18:0/234321967 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcf1802c940 con 0x557e60fa2ad0 2024-04-16T17:16:05.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.519+0000 7fcf3500e740 10 client.28631 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:05.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.519+0000 7fcf3500e740 1 -- 172.21.15.18:0/234321967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e61021950 con 0x557e60fa2ad0 2024-04-16T17:16:05.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.519+0000 7fcf14c00640 1 -- 172.21.15.18:0/234321967 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c02c070 con 0x557e60fa2ad0 2024-04-16T17:16:05.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.519+0000 7fcf14c00640 1 -- 172.21.15.18:0/234321967 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcf18046310 con 0x557e60fa2ad0 2024-04-16T17:16:05.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.521+0000 7fcf3500e740 10 client.28631 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:05.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.521+0000 7fcf3500e740 10 client.28631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:05.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.521+0000 7fcf3500e740 20 client.28631 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:05.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.521+0000 7fcf3500e740 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e610220c0 0x557e610244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.521+0000 7fcf3500e740 4 client.28631 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:05.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.521+0000 7fcf3500e740 1 -- 172.21.15.18:0/234321967 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e60f0f9b0 con 0x557e610220c0 2024-04-16T17:16:05.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.522+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e610220c0 0x557e610244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.523+0000 7fcf2aa00640 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e610220c0 0x557e610244e0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689302c674ec1464 server_cookie=e78cac1f6ac19f41 in_seq=0 out_seq=0 2024-04-16T17:16:05.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.524+0000 7fcf14c00640 10 client.28631 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:05.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.524+0000 7fcf14c00640 1 -- 172.21.15.18:0/234321967 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e60f0f9b0 con 0x557e610220c0 2024-04-16T17:16:05.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.524+0000 7fcf14c00640 10 client.28631 handle_command_reply: tid=0 2024-04-16T17:16:05.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.524+0000 7fcf3500e740 1 client.28631 shutdown 2024-04-16T17:16:05.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.525+0000 7fcf3500e740 1 -- 172.21.15.18:0/234321967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa2ad0 msgr2=0x557e60fb0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.525+0000 7fcf3500e740 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa2ad0 0x557e60fb0ec0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcf180096c0 tx=0x7fcf18009650 comp rx=0 tx=0).stop 2024-04-16T17:16:05.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 1 -- 172.21.15.18:0/234321967 shutdown_connections 2024-04-16T17:16:05.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e610220c0 0x557e610244e0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf14c00640 0 client.28631 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:05.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e60fab6b0 0x557e60fb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e60fa2ad0 0x557e60fb0ec0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 1 --2- 172.21.15.18:0/234321967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e60fa21b0 0x557e60fb3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 1 -- 172.21.15.18:0/234321967 >> 172.21.15.18:0/234321967 conn(0x557e60f9c8e0 msgr2=0x557e60fa9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:05.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 1 -- 172.21.15.18:0/234321967 shutdown_connections 2024-04-16T17:16:05.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 1 -- 172.21.15.18:0/234321967 wait complete. 2024-04-16T17:16:05.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.526+0000 7fcf3500e740 20 client.28631 trim_cache size 0 max 16384 2024-04-16T17:16:05.528 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:16:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:16:05.528 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:16:05.528 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:16:05.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 -- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcf100a3da0 msgr2=0x7fcf100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcf100a3da0 0x7fcf100a6260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf24007310 tx=0x7fcf2400a890 comp rx=0 tx=0).stop 2024-04-16T17:16:05.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 -- 172.21.15.18:0/1618180599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c112fe0 msgr2=0x7fcf2c1a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c112fe0 0x7fcf2c1a3850 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400ca80 tx=0x7fcf2400cf40 comp rx=0 tx=0).stop 2024-04-16T17:16:05.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 -- 172.21.15.18:0/1618180599 shutdown_connections 2024-04-16T17:16:05.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcf100a3da0 0x7fcf100a6260 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf2c112fe0 0x7fcf2c1a3850 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf2c074cc0 0x7fcf2c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 --2- 172.21.15.18:0/1618180599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0743a0 0x7fcf2c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.529+0000 7fcf32200640 1 -- 172.21.15.18:0/1618180599 >> 172.21.15.18:0/1618180599 conn(0x7fcf2c0fdac0 msgr2=0x7fcf2c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:05.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.530+0000 7fcf32200640 1 -- 172.21.15.18:0/1618180599 shutdown_connections 2024-04-16T17:16:05.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.530+0000 7fcf32200640 1 -- 172.21.15.18:0/1618180599 wait complete. 2024-04-16T17:16:05.557 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:16:05.557 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:16:05.664 INFO:tasks.workunit.client.0.smithi018.stdout: 1 426164 3.61 MB/sec execute 498 sec latency 57.943 ms 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 -- 172.21.15.18:0/2550922609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4112e60 msgr2=0x7fbfc4115250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 --2- 172.21.15.18:0/2550922609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4112e60 0x7fbfc4115250 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009cd0 tx=0x7fbfbc02c0f0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 -- 172.21.15.18:0/2550922609 shutdown_connections 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 --2- 172.21.15.18:0/2550922609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4112e60 0x7fbfc4115250 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 --2- 172.21.15.18:0/2550922609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfc4076e50 0x7fbfc4112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 --2- 172.21.15.18:0/2550922609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc4076530 0x7fbfc4076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 -- 172.21.15.18:0/2550922609 >> 172.21.15.18:0/2550922609 conn(0x7fbfc40ffcb0 msgr2=0x7fbfc41020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 -- 172.21.15.18:0/2550922609 shutdown_connections 2024-04-16T17:16:05.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.862+0000 7fbfcc600640 1 -- 172.21.15.18:0/2550922609 wait complete. 2024-04-16T17:16:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.864+0000 7fbfcc600640 1 Processor -- start 2024-04-16T17:16:05.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.864+0000 7fbfcc600640 1 -- start start 2024-04-16T17:16:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.865+0000 7fbfcc600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfc4076530 0x7fbfc41124b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfcc600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4076e50 0x7fbfc410b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfcc600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc4112e60 0x7fbfc410ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfcc600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41189c0 con 0x7fbfc4076530 2024-04-16T17:16:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfcc600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4118840 con 0x7fbfc4076e50 2024-04-16T17:16:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfcc600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4118b40 con 0x7fbfc4112e60 2024-04-16T17:16:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfca600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc4112e60 0x7fbfc410ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfc9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4076e50 0x7fbfc410b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfc9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4076e50 0x7fbfc410b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35820/0 (socket says 172.21.15.18:35820) 2024-04-16T17:16:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfca600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc4112e60 0x7fbfc410ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50250/0 (socket says 172.21.15.18:50250) 2024-04-16T17:16:05.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.866+0000 7fbfc9200640 1 -- 172.21.15.18:0/1242968942 learned_addr learned my addr 172.21.15.18:0/1242968942 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:05.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.867+0000 7fbfc9200640 1 -- 172.21.15.18:0/1242968942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc4112e60 msgr2=0x7fbfc410ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.867+0000 7fbfc9200640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc4112e60 0x7fbfc410ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.867+0000 7fbfc9200640 1 -- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfc4076530 msgr2=0x7fbfc41124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:05.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.867+0000 7fbfc9200640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfc4076530 0x7fbfc41124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:05.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.867+0000 7fbfc9200640 1 -- 172.21.15.18:0/1242968942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc410c0b0 con 0x7fbfc4076e50 2024-04-16T17:16:05.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.867+0000 7fbfc9200640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4076e50 0x7fbfc410b530 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000d980 tx=0x7fbfb000de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:05.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.867+0000 7fbfca600640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc4112e60 0x7fbfc410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-16T17:16:05.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.868+0000 7fbfc2000640 1 -- 172.21.15.18:0/1242968942 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0017070 con 0x7fbfc4076e50 2024-04-16T17:16:05.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.868+0000 7fbfc2000640 1 -- 172.21.15.18:0/1242968942 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb0009c30 con 0x7fbfc4076e50 2024-04-16T17:16:05.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.868+0000 7fbfc2000640 1 -- 172.21.15.18:0/1242968942 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000f620 con 0x7fbfc4076e50 2024-04-16T17:16:05.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.868+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc410ffd0 con 0x7fbfc4076e50 2024-04-16T17:16:05.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.868+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc41104d0 con 0x7fbfc4076e50 2024-04-16T17:16:05.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.870+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc410c340 con 0x7fbfc4076e50 2024-04-16T17:16:05.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.876+0000 7fbfc2000640 1 -- 172.21.15.18:0/1242968942 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbfb0010460 con 0x7fbfc4076e50 2024-04-16T17:16:05.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.881+0000 7fbfc2000640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfac0a3ef0 0x7fbfac0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:05.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.881+0000 7fbfc2000640 1 -- 172.21.15.18:0/1242968942 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fbfb00d0590 con 0x7fbfc4076e50 2024-04-16T17:16:05.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.882+0000 7fbfc9c00640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfac0a3ef0 0x7fbfac0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:05.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.883+0000 7fbfc2000640 1 -- 172.21.15.18:0/1242968942 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfb009c770 con 0x7fbfc4076e50 2024-04-16T17:16:05.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:05.883+0000 7fbfc9c00640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfac0a3ef0 0x7fbfac0a63b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb8005c70 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:05 smithi022 ceph-mon[36023]: pgmap v358: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-16T17:16:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:05 smithi022 ceph-mon[36023]: osdmap e110: 12 total, 12 up, 12 in 2024-04-16T17:16:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:05 smithi043 ceph-mon[36901]: pgmap v358: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-16T17:16:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:05 smithi043 ceph-mon[36901]: osdmap e110: 12 total, 12 up, 12 in 2024-04-16T17:16:06.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:05 smithi018 ceph-mon[31776]: pgmap v358: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-04-16T17:16:06.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:05 smithi018 ceph-mon[31776]: osdmap e110: 12 total, 12 up, 12 in 2024-04-16T17:16:06.372 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:16:06.372 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:16:00.629944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:16:06.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.370+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfc410ca60 con 0x7fbfc4076e50 2024-04-16T17:16:06.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.371+0000 7fbfc2000640 1 -- 172.21.15.18:0/1242968942 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11854 (secure 0 0 0) 0x7fbfb009c070 con 0x7fbfc4076e50 2024-04-16T17:16:06.372 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-16T17:16:06.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfac0a3ef0 msgr2=0x7fbfac0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfac0a3ef0 0x7fbfac0a63b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb8005c70 comp rx=0 tx=0).stop 2024-04-16T17:16:06.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4076e50 msgr2=0x7fbfc410b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4076e50 0x7fbfc410b530 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000d980 tx=0x7fbfb000de40 comp rx=0 tx=0).stop 2024-04-16T17:16:06.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 shutdown_connections 2024-04-16T17:16:06.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfac0a3ef0 0x7fbfac0a63b0 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb8005c70 comp rx=0 tx=0).stop 2024-04-16T17:16:06.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc4112e60 0x7fbfc410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfc4076e50 0x7fbfc410b530 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 --2- 172.21.15.18:0/1242968942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfc4076530 0x7fbfc41124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.376+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 >> 172.21.15.18:0/1242968942 conn(0x7fbfc40ffcb0 msgr2=0x7fbfc4114750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:06.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.379+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 shutdown_connections 2024-04-16T17:16:06.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.380+0000 7fbfcc600640 1 -- 172.21.15.18:0/1242968942 wait complete. 2024-04-16T17:16:06.410 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:16:06.665 INFO:tasks.workunit.client.0.smithi018.stdout: 1 426447 3.60 MB/sec execute 499 sec latency 115.898 ms 2024-04-16T17:16:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.724+0000 7f2c26600640 1 -- 172.21.15.18:0/3901452992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c201091e0 msgr2=0x7f2c201095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.724+0000 7f2c26600640 1 --2- 172.21.15.18:0/3901452992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c201091e0 0x7f2c201095c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009cd0 tx=0x7f2c1802c0e0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.724+0000 7f2c26600640 1 -- 172.21.15.18:0/3901452992 shutdown_connections 2024-04-16T17:16:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.724+0000 7f2c26600640 1 --2- 172.21.15.18:0/3901452992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c20115150 0x7f2c20117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.724+0000 7f2c26600640 1 --2- 172.21.15.18:0/3901452992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c20109b00 0x7f2c20114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.724+0000 7f2c26600640 1 --2- 172.21.15.18:0/3901452992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c201091e0 0x7f2c201095c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.724+0000 7f2c26600640 1 -- 172.21.15.18:0/3901452992 >> 172.21.15.18:0/3901452992 conn(0x7f2c2006d0a0 msgr2=0x7f2c2006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.724+0000 7f2c26600640 1 -- 172.21.15.18:0/3901452992 shutdown_connections 2024-04-16T17:16:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.725+0000 7f2c26600640 1 -- 172.21.15.18:0/3901452992 wait complete. 2024-04-16T17:16:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.726+0000 7f2c26600640 1 Processor -- start 2024-04-16T17:16:06.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.726+0000 7f2c26600640 1 -- start start 2024-04-16T17:16:06.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.727+0000 7f2c26600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c201091e0 0x7f2c20114b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.727+0000 7f2c26600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c20109b00 0x7f2c20111be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c26600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20115150 0x7f2c201121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c26600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2011ce00 con 0x7f2c201091e0 2024-04-16T17:16:06.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c26600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2011cc80 con 0x7f2c20115150 2024-04-16T17:16:06.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c26600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2011cf80 con 0x7f2c20109b00 2024-04-16T17:16:06.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c20109b00 0x7f2c20111be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c20109b00 0x7f2c20111be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50274/0 (socket says 172.21.15.18:50274) 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c1ea00640 1 -- 172.21.15.18:0/3730775779 learned_addr learned my addr 172.21.15.18:0/3730775779 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c1ea00640 1 -- 172.21.15.18:0/3730775779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20115150 msgr2=0x7f2c201121b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20115150 0x7f2c201121b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c1ea00640 1 -- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c201091e0 msgr2=0x7f2c20114b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c201091e0 0x7f2c20114b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.728+0000 7f2c1ea00640 1 -- 172.21.15.18:0/3730775779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c201b3b40 con 0x7f2c20109b00 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.729+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c20109b00 0x7f2c20111be0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00b4a0 tx=0x7f2c0c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:06.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.729+0000 7f2c17e00640 1 -- 172.21.15.18:0/3730775779 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c014070 con 0x7f2c20109b00 2024-04-16T17:16:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.730+0000 7f2c26600640 1 -- 172.21.15.18:0/3730775779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c201b3d10 con 0x7f2c20109b00 2024-04-16T17:16:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.730+0000 7f2c26600640 1 -- 172.21.15.18:0/3730775779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c201b4270 con 0x7f2c20109b00 2024-04-16T17:16:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.730+0000 7f2c17e00640 1 -- 172.21.15.18:0/3730775779 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c0c010af0 con 0x7f2c20109b00 2024-04-16T17:16:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.730+0000 7f2c17e00640 1 -- 172.21.15.18:0/3730775779 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c017d10 con 0x7f2c20109b00 2024-04-16T17:16:06.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.734+0000 7f2c293b0740 1 Processor -- start 2024-04-16T17:16:06.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.735+0000 7f2c293b0740 1 -- start start 2024-04-16T17:16:06.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.735+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a5d50 0x558af65a6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.735+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65a6670 0x558af65aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.735+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 0x558af65b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.735+0000 7f2c293b0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558af650b020 con 0x558af65a5d50 2024-04-16T17:16:06.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.735+0000 7f2c293b0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558af64b1460 con 0x558af65a6670 2024-04-16T17:16:06.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.735+0000 7f2c293b0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558af6506fe0 con 0x558af65af250 2024-04-16T17:16:06.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.736+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a5d50 0x558af65a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.736+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a5d50 0x558af65a6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49484/0 (socket says 172.21.15.18:49484) 2024-04-16T17:16:06.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.736+0000 7f2c1fe00640 1 -- 172.21.15.18:0/4059257528 learned_addr learned my addr 172.21.15.18:0/4059257528 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:06.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.737+0000 7f2c1fe00640 1 -- 172.21.15.18:0/4059257528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 msgr2=0x558af65b1640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:06.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.737+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/4059257528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 0x558af65b1640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.737+0000 7f2c1fe00640 1 -- 172.21.15.18:0/4059257528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65a6670 msgr2=0x558af65aed10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:06.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.737+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/4059257528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65a6670 0x558af65aed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.737+0000 7f2c1fe00640 1 -- 172.21.15.18:0/4059257528 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af64b9040 con 0x558af65a5d50 2024-04-16T17:16:06.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.741+0000 7f2c17e00640 1 -- 172.21.15.18:0/3730775779 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c0c017070 con 0x7f2c20109b00 2024-04-16T17:16:06.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.741+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/4059257528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a5d50 0x558af65a6130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009b70 tx=0x7f2c1000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2d27bd34648a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:06.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.742+0000 7f2c14c00640 1 -- 172.21.15.18:0/4059257528 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10009e60 con 0x558af65a5d50 2024-04-16T17:16:06.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.743+0000 7f2c14c00640 1 -- 172.21.15.18:0/4059257528 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c10005500 con 0x558af65a5d50 2024-04-16T17:16:06.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.743+0000 7f2c14c00640 1 -- 172.21.15.18:0/4059257528 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10010690 con 0x558af65a5d50 2024-04-16T17:16:06.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.743+0000 7f2c17e00640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c040a3da0 0x7f2c040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c17e00640 1 -- 172.21.15.18:0/3730775779 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f2c0c0cfc40 con 0x7f2c20109b00 2024-04-16T17:16:06.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c293b0740 1 -- 172.21.15.18:0/4059257528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a5d50 msgr2=0x558af65a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c293b0740 1 --2- 172.21.15.18:0/4059257528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a5d50 0x558af65a6130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009b70 tx=0x7f2c1000e720 comp rx=0 tx=0).stop 2024-04-16T17:16:06.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c293b0740 1 -- 172.21.15.18:0/4059257528 shutdown_connections 2024-04-16T17:16:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c293b0740 1 --2- 172.21.15.18:0/4059257528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 0x558af65b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c293b0740 1 --2- 172.21.15.18:0/4059257528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65a6670 0x558af65aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c293b0740 1 --2- 172.21.15.18:0/4059257528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a5d50 0x558af65a6130 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c293b0740 1 -- 172.21.15.18:0/4059257528 >> 172.21.15.18:0/4059257528 conn(0x558af65a0480 msgr2=0x558af65a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.744+0000 7f2c1f400640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c040a3da0 0x7f2c040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.745+0000 7f2c293b0740 1 -- 172.21.15.18:0/4059257528 shutdown_connections 2024-04-16T17:16:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.745+0000 7f2c293b0740 1 -- 172.21.15.18:0/4059257528 wait complete. 2024-04-16T17:16:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.745+0000 7f2c293b0740 1 -- start start 2024-04-16T17:16:06.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.745+0000 7f2c1f400640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c040a3da0 0x7f2c040a6260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c18037040 tx=0x7f2c180098a0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:06.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.745+0000 7f2c293b0740 1 Processor -- start 2024-04-16T17:16:06.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a5d50 0x558af65ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65af250 0x558af65accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558af65b2040 con 0x558af65a6670 2024-04-16T17:16:06.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558af64c4830 con 0x558af65af250 2024-04-16T17:16:06.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558af65b21c0 con 0x558af65a5d50 2024-04-16T17:16:06.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 10 client.0 _ll_register_callbacks cb 0x558af64cc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:06.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a5d50 0x558af65ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a5d50 0x558af65ac250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50294/0 (socket says 172.21.15.18:50294) 2024-04-16T17:16:06.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 -- 172.21.15.18:0/2207219905 learned_addr learned my addr 172.21.15.18:0/2207219905 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:06.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65af250 0x558af65accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 -- 172.21.15.18:0/2207219905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65af250 msgr2=0x558af65accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:06.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65af250 0x558af65accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 -- 172.21.15.18:0/2207219905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 msgr2=0x558af65ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:06.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 -- 172.21.15.18:0/2207219905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af64c8920 con 0x558af65a5d50 2024-04-16T17:16:06.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:06.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65af250 0x558af65accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c1f400640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a5d50 0x558af65ac250 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2c18036000 tx=0x7f2c180095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=affc11755bceee2c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c09600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c09600640 1 -- 172.21.15.18:0/2207219905 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1804f060 con 0x558af65a5d50 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c09600640 1 -- 172.21.15.18:0/2207219905 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f2c18003970 con 0x558af65a5d50 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 1 -- 172.21.15.18:0/2207219905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c10005500 con 0x558af65a5d50 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c09600640 1 -- 172.21.15.18:0/2207219905 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2c18039040 con 0x558af65a5d50 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 10 client.28498 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c293b0740 1 -- 172.21.15.18:0/2207219905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558af64967f0 con 0x558af65a5d50 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c09600640 1 -- 172.21.15.18:0/2207219905 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c10005500 con 0x558af65a5d50 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.746+0000 7f2c09600640 1 -- 172.21.15.18:0/2207219905 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2c18052050 con 0x558af65a5d50 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 10 client.28498 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 10 client.28498 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:06.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 20 client.28498 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:06.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558af65bf620 0x558af65c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 4 client.28498 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:06.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 -- 172.21.15.18:0/2207219905 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558af6518a40 con 0x558af65bf620 2024-04-16T17:16:06.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558af65bf620 0x558af65c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558af65bf620 0x558af65c1ae0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9624d5f7c026273e server_cookie=309ba86bb42e66bb in_seq=0 out_seq=0 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c09600640 10 client.28498 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c09600640 1 -- 172.21.15.18:0/2207219905 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558af6518a40 con 0x558af65bf620 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c09600640 10 client.28498 handle_command_reply: tid=0 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 client.28498 shutdown 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 -- 172.21.15.18:0/2207219905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a5d50 msgr2=0x558af65ac250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a5d50 0x558af65ac250 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2c18036000 tx=0x7f2c180095c0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 -- 172.21.15.18:0/2207219905 shutdown_connections 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558af65bf620 0x558af65c1ae0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65af250 0x558af65accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 --2- 172.21.15.18:0/2207219905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a5d50 0x558af65ac250 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2c18036000 tx=0x7f2c180095c0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c09600640 0 client.28498 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 -- 172.21.15.18:0/2207219905 >> 172.21.15.18:0/2207219905 conn(0x558af65a0480 msgr2=0x558af65a21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 -- 172.21.15.18:0/2207219905 shutdown_connections 2024-04-16T17:16:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.751+0000 7f2c293b0740 1 -- 172.21.15.18:0/2207219905 wait complete. 2024-04-16T17:16:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.755+0000 7f2c293b0740 20 client.28498 trim_cache size 0 max 16384 2024-04-16T17:16:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.784+0000 7f2c293b0740 1 Processor -- start 2024-04-16T17:16:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.784+0000 7f2c293b0740 1 -- start start 2024-04-16T17:16:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.784+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.784+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 0x558af65baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.784+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af65c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af65c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af65c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:35868/0 (socket says 172.21.15.18:35868) 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c293b0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00012ac0 con 0x558af65a6670 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c293b0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00010390 con 0x558af65c73b0 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c293b0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0000e200 con 0x558af65af250 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 0x558af65baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 0x558af65baf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50296/0 (socket says 172.21.15.18:50296) 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1ea00640 1 -- 172.21.15.18:0/1819732322 learned_addr learned my addr 172.21.15.18:0/1819732322 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1f400640 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1fe00640 1 -- 172.21.15.18:0/1819732322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 msgr2=0x558af65baf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 0x558af65baf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1fe00640 1 -- 172.21.15.18:0/1819732322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 msgr2=0x558af65ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1fe00640 1 -- 172.21.15.18:0/1819732322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af621d3f0 con 0x558af65c73b0 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.785+0000 7f2c1f400640 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.786+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af65c7810 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2c10004c60 tx=0x7f2c1000eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11ce17941ba3e0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.787+0000 7f2c0be00640 1 -- 172.21.15.18:0/1819732322 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10027020 con 0x558af65c73b0 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.787+0000 7f2c0be00640 1 -- 172.21.15.18:0/1819732322 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c10004f80 con 0x558af65c73b0 2024-04-16T17:16:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.787+0000 7f2c0be00640 1 -- 172.21.15.18:0/1819732322 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1001d040 con 0x558af65c73b0 2024-04-16T17:16:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.788+0000 7f2c293b0740 1 -- 172.21.15.18:0/1819732322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 msgr2=0x558af65c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.788+0000 7f2c293b0740 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af65c7810 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2c10004c60 tx=0x7f2c1000eed0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.789+0000 7f2c293b0740 1 -- 172.21.15.18:0/1819732322 shutdown_connections 2024-04-16T17:16:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.789+0000 7f2c293b0740 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af65c7810 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.789+0000 7f2c293b0740 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65af250 0x558af65baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.789+0000 7f2c293b0740 1 --2- 172.21.15.18:0/1819732322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65a6670 0x558af65ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.789+0000 7f2c293b0740 1 -- 172.21.15.18:0/1819732322 >> 172.21.15.18:0/1819732322 conn(0x558af65a0480 msgr2=0x558af65a2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:06.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.791+0000 7f2c293b0740 1 -- 172.21.15.18:0/1819732322 shutdown_connections 2024-04-16T17:16:06.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.791+0000 7f2c293b0740 1 -- 172.21.15.18:0/1819732322 wait complete. 2024-04-16T17:16:06.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.791+0000 7f2c293b0740 1 -- start start 2024-04-16T17:16:06.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.791+0000 7f2c293b0740 1 Processor -- start 2024-04-16T17:16:06.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.792+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a6670 0x558af65a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.792+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65af250 0x558af65a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.792+0000 7f2c293b0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af6624c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.792+0000 7f2c293b0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0000e850 con 0x558af65af250 2024-04-16T17:16:06.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.792+0000 7f2c293b0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0000fcc0 con 0x558af65c73b0 2024-04-16T17:16:06.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.792+0000 7f2c293b0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0000ef20 con 0x558af65a6670 2024-04-16T17:16:06.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.792+0000 7f2c293b0740 10 client.0 _ll_register_callbacks cb 0x7f2c18037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:06.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a6670 0x558af65a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a6670 0x558af65a92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:50304/0 (socket says 172.21.15.18:50304) 2024-04-16T17:16:06.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 -- 172.21.15.18:0/299458265 learned_addr learned my addr 172.21.15.18:0/299458265 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:06.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65af250 0x558af65a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af6624c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 -- 172.21.15.18:0/299458265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 msgr2=0x558af6624c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:06.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af6624c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 -- 172.21.15.18:0/299458265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65af250 msgr2=0x558af65a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65af250 0x558af65a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 -- 172.21.15.18:0/299458265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af62e77d0 con 0x558af65a6670 2024-04-16T17:16:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1fe00640 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af6624c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c1f400640 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a6670 0x558af65a92c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2c18042040 tx=0x7f2c18003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d30ae5a194b77ba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c09600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c09600640 1 -- 172.21.15.18:0/299458265 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18058020 con 0x558af65a6670 2024-04-16T17:16:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c09600640 1 -- 172.21.15.18:0/299458265 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f2c18004040 con 0x558af65a6670 2024-04-16T17:16:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c293b0740 1 -- 172.21.15.18:0/299458265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c10004f80 con 0x558af65a6670 2024-04-16T17:16:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c09600640 1 -- 172.21.15.18:0/299458265 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2c1804f070 con 0x558af65a6670 2024-04-16T17:16:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c293b0740 10 client.28504 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c293b0740 1 -- 172.21.15.18:0/299458265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558af66260a0 con 0x558af65a6670 2024-04-16T17:16:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c09600640 1 -- 172.21.15.18:0/299458265 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c10004f80 con 0x558af65a6670 2024-04-16T17:16:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.793+0000 7f2c09600640 1 -- 172.21.15.18:0/299458265 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2c18052030 con 0x558af65a6670 2024-04-16T17:16:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.798+0000 7f2c293b0740 10 client.28504 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.798+0000 7f2c293b0740 10 client.28504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.798+0000 7f2c293b0740 20 client.28504 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.798+0000 7f2c293b0740 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558af66267f0 0x558af6628c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.798+0000 7f2c293b0740 4 client.28504 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.798+0000 7f2c293b0740 1 -- 172.21.15.18:0/299458265 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558af65128b0 con 0x558af66267f0 2024-04-16T17:16:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.798+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558af66267f0 0x558af6628c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:06.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.801+0000 7f2c1ea00640 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558af66267f0 0x558af6628c10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1c470a0286dd5e8 server_cookie=8364a873fd4e5fda in_seq=0 out_seq=0 2024-04-16T17:16:06.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.801+0000 7f2c09600640 10 client.28504 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c09600640 1 -- 172.21.15.18:0/299458265 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558af65128b0 con 0x558af66267f0 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c09600640 10 client.28504 handle_command_reply: tid=0 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c293b0740 1 client.28504 shutdown 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c293b0740 1 -- 172.21.15.18:0/299458265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a6670 msgr2=0x558af65a92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c293b0740 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a6670 0x558af65a92c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2c18042040 tx=0x7f2c18003b80 comp rx=0 tx=0).stop 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c293b0740 1 -- 172.21.15.18:0/299458265 shutdown_connections 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c293b0740 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558af66267f0 0x558af6628c10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c293b0740 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af65c73b0 0x558af6624c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c293b0740 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558af65af250 0x558af65a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.802+0000 7f2c293b0740 1 --2- 172.21.15.18:0/299458265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558af65a6670 0x558af65a92c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.803+0000 7f2c293b0740 1 -- 172.21.15.18:0/299458265 >> 172.21.15.18:0/299458265 conn(0x558af65a0480 msgr2=0x558af659ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:06.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.803+0000 7f2c09600640 0 client.28504 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:06.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.803+0000 7f2c293b0740 1 -- 172.21.15.18:0/299458265 shutdown_connections 2024-04-16T17:16:06.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.803+0000 7f2c293b0740 1 -- 172.21.15.18:0/299458265 wait complete. 2024-04-16T17:16:06.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.803+0000 7f2c293b0740 20 client.28504 trim_cache size 0 max 16384 2024-04-16T17:16:06.805 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:06.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 -- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c040a3da0 msgr2=0x7f2c040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c040a3da0 0x7f2c040a6260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c18037040 tx=0x7f2c180098a0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 -- 172.21.15.18:0/3730775779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c20109b00 msgr2=0x7f2c20111be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:06.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c20109b00 0x7f2c20111be0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00b4a0 tx=0x7f2c0c00b960 comp rx=0 tx=0).stop 2024-04-16T17:16:06.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 -- 172.21.15.18:0/3730775779 shutdown_connections 2024-04-16T17:16:06.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c040a3da0 0x7f2c040a6260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20115150 0x7f2c201121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c20109b00 0x7f2c20111be0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 --2- 172.21.15.18:0/3730775779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c201091e0 0x7f2c20114b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:06.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 -- 172.21.15.18:0/3730775779 >> 172.21.15.18:0/3730775779 conn(0x7f2c2006d0a0 msgr2=0x7f2c20070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:06.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 -- 172.21.15.18:0/3730775779 shutdown_connections 2024-04-16T17:16:06.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:06.804+0000 7f2c26600640 1 -- 172.21.15.18:0/3730775779 wait complete. 2024-04-16T17:16:06.836 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:16:06.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:06 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1242968942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:06.987 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:06 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1242968942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:07.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:06 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1242968942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:07.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.104+0000 7fe81f200640 1 -- 172.21.15.18:0/2459240020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 msgr2=0x7fe82010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.104+0000 7fe81f200640 1 --2- 172.21.15.18:0/2459240020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 0x7fe82010dc00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe818009ca0 tx=0x7fe81802c140 comp rx=0 tx=0).stop 2024-04-16T17:16:07.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.104+0000 7fe81f200640 1 -- 172.21.15.18:0/2459240020 shutdown_connections 2024-04-16T17:16:07.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.104+0000 7fe81f200640 1 --2- 172.21.15.18:0/2459240020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 0x7fe82010dc00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.104+0000 7fe81f200640 1 --2- 172.21.15.18:0/2459240020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe820101910 0x7fe820105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.104+0000 7fe81f200640 1 --2- 172.21.15.18:0/2459240020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe820100f60 0x7fe820101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.104+0000 7fe81f200640 1 -- 172.21.15.18:0/2459240020 >> 172.21.15.18:0/2459240020 conn(0x7fe820077ab0 msgr2=0x7fe8200fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.104+0000 7fe81f200640 1 -- 172.21.15.18:0/2459240020 shutdown_connections 2024-04-16T17:16:07.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.105+0000 7fe81f200640 1 -- 172.21.15.18:0/2459240020 wait complete. 2024-04-16T17:16:07.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.106+0000 7fe81f200640 1 Processor -- start 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.106+0000 7fe81f200640 1 -- start start 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.106+0000 7fe81f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe820100f60 0x7fe8201a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.106+0000 7fe81f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 0x7fe8201a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.106+0000 7fe81f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe82019a8a0 0x7fe82019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.106+0000 7fe81f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201111e0 con 0x7fe820106030 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.106+0000 7fe81f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe820111060 con 0x7fe82019a8a0 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.106+0000 7fe81f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe820111360 con 0x7fe820100f60 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe820100f60 0x7fe8201a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe820100f60 0x7fe8201a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35198/0 (socket says 172.21.15.18:35198) 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81de00640 1 -- 172.21.15.18:0/811282176 learned_addr learned my addr 172.21.15.18:0/811282176 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81e800640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe82019a8a0 0x7fe82019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81d400640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 0x7fe8201a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81de00640 1 -- 172.21.15.18:0/811282176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe82019a8a0 msgr2=0x7fe82019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81de00640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe82019a8a0 0x7fe82019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81de00640 1 -- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 msgr2=0x7fe8201a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81de00640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 0x7fe8201a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81de00640 1 -- 172.21.15.18:0/811282176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82019b3c0 con 0x7fe820100f60 2024-04-16T17:16:07.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.107+0000 7fe81d400640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 0x7fe8201a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:07.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.108+0000 7fe81de00640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe820100f60 0x7fe8201a06f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b450 tx=0x7fe81000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.108+0000 7fe816a00640 1 -- 172.21.15.18:0/811282176 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810015020 con 0x7fe820100f60 2024-04-16T17:16:07.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.108+0000 7fe816a00640 1 -- 172.21.15.18:0/811282176 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe810011d70 con 0x7fe820100f60 2024-04-16T17:16:07.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.108+0000 7fe81f200640 1 -- 172.21.15.18:0/811282176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82019b6b0 con 0x7fe820100f60 2024-04-16T17:16:07.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.108+0000 7fe816a00640 1 -- 172.21.15.18:0/811282176 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81000f340 con 0x7fe820100f60 2024-04-16T17:16:07.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.108+0000 7fe81f200640 1 -- 172.21.15.18:0/811282176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8201a7950 con 0x7fe820100f60 2024-04-16T17:16:07.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe827655740 1 Processor -- start 2024-04-16T17:16:07.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe827655740 1 -- start start 2024-04-16T17:16:07.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe827655740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9df10 0x560523f9e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe827655740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9e830 0x560523fa6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe827655740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 0x560523fa9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe827655740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560523f03020 con 0x560523fa7410 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe827655740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560523ea9460 con 0x560523f9e830 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe827655740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560523efefe0 con 0x560523f9df10 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe81e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9df10 0x560523f9e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.112+0000 7fe81e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9df10 0x560523f9e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35202/0 (socket says 172.21.15.18:35202) 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe81e800640 1 -- 172.21.15.18:0/2630882678 learned_addr learned my addr 172.21.15.18:0/2630882678 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe81d400640 1 --2- 172.21.15.18:0/2630882678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 0x560523fa9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe81e800640 1 -- 172.21.15.18:0/2630882678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9e830 msgr2=0x560523fa6ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe81e800640 1 --2- 172.21.15.18:0/2630882678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9e830 0x560523fa6ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe81e800640 1 -- 172.21.15.18:0/2630882678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 msgr2=0x560523fa9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe81e800640 1 --2- 172.21.15.18:0/2630882678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 0x560523fa9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe81e800640 1 -- 172.21.15.18:0/2630882678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560523eb1040 con 0x560523f9df10 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe816a00640 1 -- 172.21.15.18:0/811282176 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe810018020 con 0x7fe820100f60 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe81e800640 1 --2- 172.21.15.18:0/2630882678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9df10 0x560523f9e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe81802c750 tx=0x7fe818005f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=550f3a88e15c6473 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.113+0000 7fe80b000640 1 -- 172.21.15.18:0/2630882678 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818043070 con 0x560523f9df10 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe80b000640 1 -- 172.21.15.18:0/2630882678 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe818003970 con 0x560523f9df10 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe80b000640 1 -- 172.21.15.18:0/2630882678 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818039040 con 0x560523f9df10 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe827655740 1 -- 172.21.15.18:0/2630882678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9df10 msgr2=0x560523f9e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe827655740 1 --2- 172.21.15.18:0/2630882678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9df10 0x560523f9e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe81802c750 tx=0x7fe818005f60 comp rx=0 tx=0).stop 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe827655740 1 -- 172.21.15.18:0/2630882678 shutdown_connections 2024-04-16T17:16:07.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe827655740 1 --2- 172.21.15.18:0/2630882678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 0x560523fa9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe827655740 1 --2- 172.21.15.18:0/2630882678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9e830 0x560523fa6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe827655740 1 --2- 172.21.15.18:0/2630882678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9df10 0x560523f9e2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe827655740 1 -- 172.21.15.18:0/2630882678 >> 172.21.15.18:0/2630882678 conn(0x560523f98640 msgr2=0x560523f98a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.114+0000 7fe816a00640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe8040a3da0 0x7fe8040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.115+0000 7fe816a00640 1 -- 172.21.15.18:0/811282176 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fe8100d05d0 con 0x7fe820100f60 2024-04-16T17:16:07.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.115+0000 7fe827655740 1 -- 172.21.15.18:0/2630882678 shutdown_connections 2024-04-16T17:16:07.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.115+0000 7fe81d400640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe8040a3da0 0x7fe8040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.115+0000 7fe827655740 1 -- 172.21.15.18:0/2630882678 wait complete. 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.115+0000 7fe827655740 1 -- start start 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.115+0000 7fe827655740 1 Processor -- start 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe81d400640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe8040a3da0 0x7fe8040a6260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x560523f997d0 tx=0x7fe80c008040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe827655740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fa4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe827655740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fa4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe827655740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 0x560523fa4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe827655740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560523faa1a0 con 0x560523fa7410 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe827655740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560523ebc830 con 0x560523f9df10 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe827655740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560523faa320 con 0x560523f9e830 2024-04-16T17:16:07.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe81de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fa4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe827655740 10 client.0 _ll_register_callbacks cb 0x560523ec4150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe81de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fa4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56432/0 (socket says 172.21.15.18:56432) 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe81de00640 1 -- 172.21.15.18:0/3402084161 learned_addr learned my addr 172.21.15.18:0/3402084161 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe81d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fa4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.116+0000 7fe81e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 0x560523fa4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe81de00640 1 -- 172.21.15.18:0/3402084161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 msgr2=0x560523fa4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe81de00640 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fa4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe81de00640 1 -- 172.21.15.18:0/3402084161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 msgr2=0x560523fa4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe81de00640 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 0x560523fa4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe81de00640 1 -- 172.21.15.18:0/3402084161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560523ec0920 con 0x560523f9df10 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe81de00640 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fa4510 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe8100cdec0 tx=0x7fe81009be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc1e42425cab0378 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe7ff400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe7ff400640 1 -- 172.21.15.18:0/3402084161 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100a05d0 con 0x560523f9df10 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.117+0000 7fe7ff400640 1 -- 172.21.15.18:0/3402084161 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fe8100a0170 con 0x560523f9df10 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.118+0000 7fe827655740 1 -- 172.21.15.18:0/3402084161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe818003970 con 0x560523f9df10 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.118+0000 7fe7ff400640 1 -- 172.21.15.18:0/3402084161 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe8100a0390 con 0x560523f9df10 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.118+0000 7fe827655740 10 client.28655 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.118+0000 7fe827655740 1 -- 172.21.15.18:0/3402084161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560523e8e7f0 con 0x560523f9df10 2024-04-16T17:16:07.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.118+0000 7fe7ff400640 1 -- 172.21.15.18:0/3402084161 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe818003970 con 0x560523f9df10 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.118+0000 7fe7ff400640 1 -- 172.21.15.18:0/3402084161 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe81009b670 con 0x560523f9df10 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.119+0000 7fe827655740 10 client.28655 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.119+0000 7fe827655740 10 client.28655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.119+0000 7fe827655740 20 client.28655 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.119+0000 7fe827655740 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x560523fae740 0x560523fb0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.119+0000 7fe827655740 4 client.28655 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.119+0000 7fe827655740 1 -- 172.21.15.18:0/3402084161 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560523f10a40 con 0x560523fae740 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.119+0000 7fe81d400640 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x560523fae740 0x560523fb0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.120+0000 7fe81d400640 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x560523fae740 0x560523fb0bc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3d88528e8bd9095 server_cookie=dd41cce8a0e3e9cc in_seq=0 out_seq=0 2024-04-16T17:16:07.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.120+0000 7fe7ff400640 10 client.28655 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:07.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.121+0000 7fe7ff400640 1 -- 172.21.15.18:0/3402084161 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560523f10a40 con 0x560523fae740 2024-04-16T17:16:07.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.121+0000 7fe7ff400640 10 client.28655 handle_command_reply: tid=0 2024-04-16T17:16:07.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.121+0000 7fe827655740 1 client.28655 shutdown 2024-04-16T17:16:07.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 -- 172.21.15.18:0/3402084161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 msgr2=0x560523fa4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fa4510 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe8100cdec0 tx=0x7fe81009be90 comp rx=0 tx=0).stop 2024-04-16T17:16:07.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 -- 172.21.15.18:0/3402084161 shutdown_connections 2024-04-16T17:16:07.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x560523fae740 0x560523fb0bc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523fa7410 0x560523fa4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fa4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 --2- 172.21.15.18:0/3402084161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fa4510 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe7ff400640 0 client.28655 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 -- 172.21.15.18:0/3402084161 >> 172.21.15.18:0/3402084161 conn(0x560523f98640 msgr2=0x560523f9a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 -- 172.21.15.18:0/3402084161 shutdown_connections 2024-04-16T17:16:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.122+0000 7fe827655740 1 -- 172.21.15.18:0/3402084161 wait complete. 2024-04-16T17:16:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.123+0000 7fe827655740 20 client.28655 trim_cache size 0 max 16384 2024-04-16T17:16:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.143+0000 7fe827655740 1 Processor -- start 2024-04-16T17:16:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe827655740 1 -- start start 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe827655740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9df10 0x560523fa6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe827655740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fbb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe827655740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523fa7410 0x560523fd1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe827655740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201afd60 con 0x560523f9df10 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe827655740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201ad630 con 0x560523fa7410 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe827655740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201ab4a0 con 0x560523f9e830 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe81e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523fa7410 0x560523fd1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe81e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523fa7410 0x560523fd1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56442/0 (socket says 172.21.15.18:56442) 2024-04-16T17:16:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe81e800640 1 -- 172.21.15.18:0/1207276048 learned_addr learned my addr 172.21.15.18:0/1207276048 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.144+0000 7fe81d400640 1 --2- 172.21.15.18:0/1207276048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fbb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.145+0000 7fe81d400640 1 -- 172.21.15.18:0/1207276048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523fa7410 msgr2=0x560523fd1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.145+0000 7fe81d400640 1 --2- 172.21.15.18:0/1207276048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523fa7410 0x560523fd1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.145+0000 7fe81d400640 1 -- 172.21.15.18:0/1207276048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9df10 msgr2=0x560523fa6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.145+0000 7fe81d400640 1 --2- 172.21.15.18:0/1207276048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9df10 0x560523fa6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.145+0000 7fe81d400640 1 -- 172.21.15.18:0/1207276048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560523c153f0 con 0x560523f9e830 2024-04-16T17:16:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.145+0000 7fe81d400640 1 --2- 172.21.15.18:0/1207276048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fbb6a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x560523fb1430 tx=0x7fe80c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d2a26c80007ed61 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.145+0000 7fe80a600640 1 -- 172.21.15.18:0/1207276048 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00fc90 con 0x560523f9e830 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.146+0000 7fe80a600640 1 -- 172.21.15.18:0/1207276048 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe80c02c070 con 0x560523f9e830 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.146+0000 7fe80a600640 1 -- 172.21.15.18:0/1207276048 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c024670 con 0x560523f9e830 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.146+0000 7fe827655740 1 -- 172.21.15.18:0/1207276048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 msgr2=0x560523fbb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.147+0000 7fe827655740 1 --2- 172.21.15.18:0/1207276048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fbb6a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x560523fb1430 tx=0x7fe80c012000 comp rx=0 tx=0).stop 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.147+0000 7fe827655740 1 -- 172.21.15.18:0/1207276048 shutdown_connections 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.147+0000 7fe827655740 1 --2- 172.21.15.18:0/1207276048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523fa7410 0x560523fd1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.147+0000 7fe827655740 1 --2- 172.21.15.18:0/1207276048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523f9e830 0x560523fbb6a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.147+0000 7fe827655740 1 --2- 172.21.15.18:0/1207276048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9df10 0x560523fa6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.147+0000 7fe827655740 1 -- 172.21.15.18:0/1207276048 >> 172.21.15.18:0/1207276048 conn(0x560523f98640 msgr2=0x560523fbaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.147+0000 7fe827655740 1 -- 172.21.15.18:0/1207276048 shutdown_connections 2024-04-16T17:16:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.147+0000 7fe827655740 1 -- 172.21.15.18:0/1207276048 wait complete. 2024-04-16T17:16:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 1 -- start start 2024-04-16T17:16:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 1 Processor -- start 2024-04-16T17:16:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fb6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9e830 0x560523fb3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523fa7410 0x560523fb39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201abaf0 con 0x560523f9e830 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201acf60 con 0x560523f9df10 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201ac1c0 con 0x560523fa7410 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.148+0000 7fe827655740 10 client.0 _ll_register_callbacks cb 0x7fe810010890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.149+0000 7fe81de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fb6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.149+0000 7fe81e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523fa7410 0x560523fb39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.149+0000 7fe81d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9e830 0x560523fb3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.149+0000 7fe81e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523fa7410 0x560523fb39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35224/0 (socket says 172.21.15.18:35224) 2024-04-16T17:16:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.149+0000 7fe81e800640 1 -- 172.21.15.18:0/2062809299 learned_addr learned my addr 172.21.15.18:0/2062809299 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.150+0000 7fe81d400640 1 -- 172.21.15.18:0/2062809299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523fa7410 msgr2=0x560523fb39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.150+0000 7fe81d400640 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523fa7410 0x560523fb39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.150+0000 7fe81d400640 1 -- 172.21.15.18:0/2062809299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 msgr2=0x560523fb6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.150+0000 7fe81d400640 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fb6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.150+0000 7fe81d400640 1 -- 172.21.15.18:0/2062809299 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560523cdf7d0 con 0x560523f9e830 2024-04-16T17:16:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.150+0000 7fe81de00640 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fb6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.150+0000 7fe81d400640 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9e830 0x560523fb3470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x560523f9e600 tx=0x7fe80c026980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957d5b54dbfb4a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.150+0000 7fe7ff400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:16:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.151+0000 7fe7ff400640 1 -- 172.21.15.18:0/2062809299 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c036020 con 0x560523f9e830 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.151+0000 7fe827655740 1 -- 172.21.15.18:0/2062809299 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe80c02c070 con 0x560523f9e830 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.151+0000 7fe7ff400640 1 -- 172.21.15.18:0/2062809299 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fe80c0262f0 con 0x560523f9e830 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.151+0000 7fe7ff400640 1 -- 172.21.15.18:0/2062809299 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe80c0264f0 con 0x560523f9e830 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.151+0000 7fe827655740 10 client.18345 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.151+0000 7fe827655740 1 -- 172.21.15.18:0/2062809299 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56052402f380 con 0x560523f9e830 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.152+0000 7fe7ff400640 1 -- 172.21.15.18:0/2062809299 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe80c02c070 con 0x560523f9e830 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.153+0000 7fe7ff400640 1 -- 172.21.15.18:0/2062809299 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe80c024520 con 0x560523f9e830 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.153+0000 7fe827655740 10 client.18345 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.153+0000 7fe827655740 10 client.18345 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.153+0000 7fe827655740 20 client.18345 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.153+0000 7fe827655740 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56052402faa0 0x560524031e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.153+0000 7fe827655740 4 client.18345 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:07.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.153+0000 7fe827655740 1 -- 172.21.15.18:0/2062809299 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560523f0a8b0 con 0x56052402faa0 2024-04-16T17:16:07.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.154+0000 7fe81de00640 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56052402faa0 0x560524031e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.155+0000 7fe7ff400640 10 client.18345 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.155+0000 7fe81de00640 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56052402faa0 0x560524031e90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fed920c8468deb01 server_cookie=4c57181e2285a793 in_seq=0 out_seq=0 2024-04-16T17:16:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.155+0000 7fe7ff400640 1 -- 172.21.15.18:0/2062809299 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560523f0a8b0 con 0x56052402faa0 2024-04-16T17:16:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.155+0000 7fe7ff400640 10 client.18345 handle_command_reply: tid=0 2024-04-16T17:16:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.155+0000 7fe827655740 1 client.18345 shutdown 2024-04-16T17:16:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.155+0000 7fe827655740 1 -- 172.21.15.18:0/2062809299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9e830 msgr2=0x560523fb3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.155+0000 7fe827655740 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9e830 0x560523fb3470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x560523f9e600 tx=0x7fe80c026980 comp rx=0 tx=0).stop 2024-04-16T17:16:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 1 -- 172.21.15.18:0/2062809299 shutdown_connections 2024-04-16T17:16:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x56052402faa0 0x560524031e90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560523fa7410 0x560523fb39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560523f9e830 0x560523fb3470 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 1 --2- 172.21.15.18:0/2062809299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560523f9df10 0x560523fb6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe7ff400640 0 client.18345 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 1 -- 172.21.15.18:0/2062809299 >> 172.21.15.18:0/2062809299 conn(0x560523f98640 msgr2=0x560523f96e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 1 -- 172.21.15.18:0/2062809299 shutdown_connections 2024-04-16T17:16:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 1 -- 172.21.15.18:0/2062809299 wait complete. 2024-04-16T17:16:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.156+0000 7fe827655740 20 client.18345 trim_cache size 0 max 16384 2024-04-16T17:16:07.158 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.157+0000 7fe81f200640 1 -- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe8040a3da0 msgr2=0x7fe8040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.157+0000 7fe81f200640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe8040a3da0 0x7fe8040a6260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x560523f997d0 tx=0x7fe80c008040 comp rx=0 tx=0).stop 2024-04-16T17:16:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.157+0000 7fe81f200640 1 -- 172.21.15.18:0/811282176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe820100f60 msgr2=0x7fe8201a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.157+0000 7fe81f200640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe820100f60 0x7fe8201a06f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b450 tx=0x7fe81000b910 comp rx=0 tx=0).stop 2024-04-16T17:16:07.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.158+0000 7fe81f200640 1 -- 172.21.15.18:0/811282176 shutdown_connections 2024-04-16T17:16:07.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.158+0000 7fe81f200640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe8040a3da0 0x7fe8040a6260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.158+0000 7fe81f200640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe82019a8a0 0x7fe82019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.158+0000 7fe81f200640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe820106030 0x7fe8201a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.158+0000 7fe81f200640 1 --2- 172.21.15.18:0/811282176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe820100f60 0x7fe8201a06f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.158+0000 7fe81f200640 1 -- 172.21.15.18:0/811282176 >> 172.21.15.18:0/811282176 conn(0x7fe820077ab0 msgr2=0x7fe820069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.158+0000 7fe81f200640 1 -- 172.21.15.18:0/811282176 shutdown_connections 2024-04-16T17:16:07.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.158+0000 7fe81f200640 1 -- 172.21.15.18:0/811282176 wait complete. 2024-04-16T17:16:07.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:16:07.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 -- 172.21.15.18:0/1796415723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c104c30 msgr2=0x7f696c108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 --2- 172.21.15.18:0/1796415723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c104c30 0x7f696c108c70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6958009900 tx=0x7f695802bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:07.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 -- 172.21.15.18:0/1796415723 shutdown_connections 2024-04-16T17:16:07.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 --2- 172.21.15.18:0/1796415723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f696c109350 0x7f696c110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 --2- 172.21.15.18:0/1796415723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c104c30 0x7f696c108c70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 --2- 172.21.15.18:0/1796415723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f696c104280 0x7f696c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 -- 172.21.15.18:0/1796415723 >> 172.21.15.18:0/1796415723 conn(0x7f696c0ffae0 msgr2=0x7f696c101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 -- 172.21.15.18:0/1796415723 shutdown_connections 2024-04-16T17:16:07.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.406+0000 7f696be00640 1 -- 172.21.15.18:0/1796415723 wait complete. 2024-04-16T17:16:07.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.407+0000 7f696be00640 1 Processor -- start 2024-04-16T17:16:07.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.408+0000 7f696be00640 1 -- start start 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f696c104280 0x7f696c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f696c109350 0x7f696c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c19a830 0x7f696c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f696c114430 con 0x7f696c109350 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1142b0 con 0x7f696c19a830 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1145b0 con 0x7f696c104280 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f696c109350 0x7f696c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f696c104280 0x7f696c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f696c109350 0x7f696c1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39828/0 (socket says 172.21.15.18:39828) 2024-04-16T17:16:07.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696b400640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c19a830 0x7f696c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696a000640 1 -- 172.21.15.18:0/4094870348 learned_addr learned my addr 172.21.15.18:0/4094870348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.409+0000 7f696aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f696c104280 0x7f696c1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35242/0 (socket says 172.21.15.18:35242) 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f696b400640 1 -- 172.21.15.18:0/4094870348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f696c104280 msgr2=0x7f696c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f696b400640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f696c104280 0x7f696c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f696b400640 1 -- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f696c109350 msgr2=0x7f696c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f696b400640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f696c109350 0x7f696c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f696b400640 1 -- 172.21.15.18:0/4094870348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c19b3c0 con 0x7f696c19a830 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f696b400640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c19a830 0x7f696c19ace0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6964016fd0 tx=0x7f696400ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f6970a00640 1 -- 172.21.15.18:0/4094870348 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964004b20 con 0x7f696c19a830 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f6970a00640 1 -- 172.21.15.18:0/4094870348 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6964004cc0 con 0x7f696c19a830 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f6970a00640 1 -- 172.21.15.18:0/4094870348 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696401ec40 con 0x7f696c19a830 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.410+0000 7f696be00640 1 -- 172.21.15.18:0/4094870348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c19b6b0 con 0x7f696c19a830 2024-04-16T17:16:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.411+0000 7f696be00640 1 -- 172.21.15.18:0/4094870348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c076770 con 0x7f696c19a830 2024-04-16T17:16:07.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.414+0000 7f6974b66740 1 Processor -- start 2024-04-16T17:16:07.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f6970a00640 1 -- 172.21.15.18:0/4094870348 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f696400b6f0 con 0x7f696c19a830 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f6974b66740 1 -- start start 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f6974b66740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f6974b66740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f10830 0x55adc2f18ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f6974b66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f19410 0x55adc2f1b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f6974b66740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55adc2e75020 con 0x55adc2f0ff10 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f6974b66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55adc2e1b460 con 0x55adc2f19410 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f6974b66740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adc2e70fe0 con 0x55adc2f10830 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f696aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f10830 0x55adc2f18ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f696aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f10830 0x55adc2f18ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35244/0 (socket says 172.21.15.18:35244) 2024-04-16T17:16:07.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.415+0000 7f696aa00640 1 -- 172.21.15.18:0/3112001938 learned_addr learned my addr 172.21.15.18:0/3112001938 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696a000640 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 -- 172.21.15.18:0/3112001938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f19410 msgr2=0x55adc2f1b800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696a000640 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f19410 0x55adc2f1b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f19410 0x55adc2f1b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 -- 172.21.15.18:0/3112001938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 msgr2=0x55adc2f102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 -- 172.21.15.18:0/3112001938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc2e23040 con 0x55adc2f10830 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696a000640 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f19410 0x55adc2f1b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:16:07.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f10830 0x55adc2f18ed0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f695c009d20 tx=0x7f695c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d13e9b6d85af5d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6970a00640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69500a3d20 0x7f69500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6957e00640 1 -- 172.21.15.18:0/3112001938 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c01e070 con 0x55adc2f10830 2024-04-16T17:16:07.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6957e00640 1 -- 172.21.15.18:0/3112001938 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695c019b10 con 0x55adc2f10830 2024-04-16T17:16:07.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6957e00640 1 -- 172.21.15.18:0/3112001938 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c00eae0 con 0x55adc2f10830 2024-04-16T17:16:07.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6970a00640 1 -- 172.21.15.18:0/4094870348 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f69640d9110 con 0x7f696c19a830 2024-04-16T17:16:07.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696a000640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69500a3d20 0x7f69500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- 172.21.15.18:0/3112001938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f10830 msgr2=0x55adc2f18ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f10830 0x55adc2f18ed0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f695c009d20 tx=0x7f695c011810 comp rx=0 tx=0).stop 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- 172.21.15.18:0/3112001938 shutdown_connections 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f19410 0x55adc2f1b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f10830 0x55adc2f18ed0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 --2- 172.21.15.18:0/3112001938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- 172.21.15.18:0/3112001938 >> 172.21.15.18:0/3112001938 conn(0x55adc2f0a640 msgr2=0x55adc2f0aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696a000640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69500a3d20 0x7f69500a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f696c10a9e0 tx=0x7f6958005dd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- 172.21.15.18:0/3112001938 shutdown_connections 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- 172.21.15.18:0/3112001938 wait complete. 2024-04-16T17:16:07.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- start start 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 Processor -- start 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f16920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f16e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55adc2f1c200 con 0x55adc2f19410 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55adc2e2e830 con 0x55adc2f10830 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adc2f1c380 con 0x55adc2f0ff10 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6974b66740 10 client.0 _ll_register_callbacks cb 0x55adc2e36150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f163e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35254/0 (socket says 172.21.15.18:35254) 2024-04-16T17:16:07.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 -- 172.21.15.18:0/4271807208 learned_addr learned my addr 172.21.15.18:0/4271807208 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696a000640 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f16920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696b400640 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f16e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 -- 172.21.15.18:0/4271807208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 msgr2=0x55adc2f16920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f16920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 -- 172.21.15.18:0/4271807208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 msgr2=0x55adc2f16e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f16e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 -- 172.21.15.18:0/4271807208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc2e32920 con 0x55adc2f0ff10 2024-04-16T17:16:07.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f696aa00640 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f163e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f695c01c040 tx=0x7f695c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70414d735d843947 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6954c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:07.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.416+0000 7f6954c00640 1 -- 172.21.15.18:0/4271807208 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c027070 con 0x55adc2f0ff10 2024-04-16T17:16:07.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6954c00640 1 -- 172.21.15.18:0/4271807208 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f695c019630 con 0x55adc2f0ff10 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 1 -- 172.21.15.18:0/4271807208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695c019b10 con 0x55adc2f0ff10 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6954c00640 1 -- 172.21.15.18:0/4271807208 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f695c033050 con 0x55adc2f0ff10 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 10 client.28531 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 1 -- 172.21.15.18:0/4271807208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55adc2e007f0 con 0x55adc2f0ff10 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6954c00640 1 -- 172.21.15.18:0/4271807208 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695c019b10 con 0x55adc2f0ff10 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6954c00640 1 -- 172.21.15.18:0/4271807208 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f695c0035f0 con 0x55adc2f0ff10 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 10 client.28531 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 10 client.28531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 20 client.28531 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55adc2f29590 0x55adc2f2b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 4 client.28531 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f6974b66740 1 -- 172.21.15.18:0/4271807208 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc2e82a40 con 0x55adc2f29590 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.422+0000 7f696a000640 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55adc2f29590 0x55adc2f2b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f696a000640 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55adc2f29590 0x55adc2f2b980 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b414796f0811acfe server_cookie=e74f8c3e91c644f8 in_seq=0 out_seq=0 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6954c00640 10 client.28531 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6954c00640 1 -- 172.21.15.18:0/4271807208 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55adc2e82a40 con 0x55adc2f29590 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6954c00640 10 client.28531 handle_command_reply: tid=0 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 client.28531 shutdown 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 -- 172.21.15.18:0/4271807208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 msgr2=0x55adc2f163e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f163e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f695c01c040 tx=0x7f695c0045e0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 -- 172.21.15.18:0/4271807208 shutdown_connections 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55adc2f29590 0x55adc2f2b980 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f16e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f16920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 --2- 172.21.15.18:0/4271807208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f163e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 -- 172.21.15.18:0/4271807208 >> 172.21.15.18:0/4271807208 conn(0x55adc2f0a640 msgr2=0x55adc2f0c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6954c00640 0 client.28531 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:07.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 -- 172.21.15.18:0/4271807208 shutdown_connections 2024-04-16T17:16:07.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 1 -- 172.21.15.18:0/4271807208 wait complete. 2024-04-16T17:16:07.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.425+0000 7f6974b66740 20 client.28531 trim_cache size 0 max 16384 2024-04-16T17:16:07.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.441+0000 7f6974b66740 1 Processor -- start 2024-04-16T17:16:07.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.441+0000 7f6974b66740 1 -- start start 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.441+0000 7f6974b66740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f18b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.441+0000 7f6974b66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f250c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.441+0000 7f6974b66740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f19410 0x55adc2f16ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f6974b66740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1afdc0 con 0x55adc2f0ff10 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f6974b66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1ad690 con 0x55adc2f10830 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f6974b66740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1ab500 con 0x55adc2f19410 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f18b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f250c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f18b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39846/0 (socket says 172.21.15.18:39846) 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 -- 172.21.15.18:0/3101496344 learned_addr learned my addr 172.21.15.18:0/3101496344 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696b400640 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f19410 0x55adc2f16ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 -- 172.21.15.18:0/3101496344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f19410 msgr2=0x55adc2f16ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f19410 0x55adc2f16ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 -- 172.21.15.18:0/3101496344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 msgr2=0x55adc2f250c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f250c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 -- 172.21.15.18:0/3101496344 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc2b873f0 con 0x55adc2f0ff10 2024-04-16T17:16:07.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696a000640 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f250c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:07.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.442+0000 7f696aa00640 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f18b10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f695c009250 tx=0x7f695c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60534c0d78004dce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.443+0000 7f6957400640 1 -- 172.21.15.18:0/3101496344 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c027070 con 0x55adc2f0ff10 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.443+0000 7f6957400640 1 -- 172.21.15.18:0/3101496344 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695c00e530 con 0x55adc2f0ff10 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.443+0000 7f6957400640 1 -- 172.21.15.18:0/3101496344 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c01e040 con 0x55adc2f0ff10 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.443+0000 7f6974b66740 1 -- 172.21.15.18:0/3101496344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 msgr2=0x55adc2f18b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.443+0000 7f6974b66740 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f18b10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f695c009250 tx=0x7f695c009ed0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 -- 172.21.15.18:0/3101496344 shutdown_connections 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f19410 0x55adc2f16ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f250c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 --2- 172.21.15.18:0/3101496344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f0ff10 0x55adc2f18b10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 -- 172.21.15.18:0/3101496344 >> 172.21.15.18:0/3101496344 conn(0x55adc2f0a640 msgr2=0x55adc2f163a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 -- 172.21.15.18:0/3101496344 shutdown_connections 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 -- 172.21.15.18:0/3101496344 wait complete. 2024-04-16T17:16:07.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 -- start start 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.444+0000 7f6974b66740 1 Processor -- start 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f6974b66740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f21860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f6974b66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f1cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f6974b66740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f1d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f6974b66740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1abb50 con 0x55adc2f19410 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f6974b66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1acfc0 con 0x55adc2f10830 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f6974b66740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1ac220 con 0x55adc2f0ff10 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f6974b66740 10 client.0 _ll_register_callbacks cb 0x7f695c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f1d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f1d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39854/0 (socket says 172.21.15.18:39854) 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696b400640 1 -- 172.21.15.18:0/2159510322 learned_addr learned my addr 172.21.15.18:0/2159510322 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f1cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696a000640 1 -- 172.21.15.18:0/2159510322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 msgr2=0x55adc2f21860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:07.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696a000640 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f21860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696a000640 1 -- 172.21.15.18:0/2159510322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 msgr2=0x55adc2f1d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:07.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696a000640 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f1d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696a000640 1 -- 172.21.15.18:0/2159510322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc2c517d0 con 0x55adc2f10830 2024-04-16T17:16:07.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.445+0000 7f696b400640 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f1d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.447+0000 7f696a000640 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f1cbd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6958006010 tx=0x7f695802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a19c2bb3a189920 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.447+0000 7f6954c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.447+0000 7f6954c00640 1 -- 172.21.15.18:0/2159510322 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695802c940 con 0x55adc2f10830 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.447+0000 7f6974b66740 1 -- 172.21.15.18:0/2159510322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695c00e530 con 0x55adc2f10830 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.447+0000 7f6954c00640 1 -- 172.21.15.18:0/2159510322 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f695802cae0 con 0x55adc2f10830 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6954c00640 1 -- 172.21.15.18:0/2159510322 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f695802cce0 con 0x55adc2f10830 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6974b66740 10 client.28676 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6974b66740 1 -- 172.21.15.18:0/2159510322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55adc2f8f720 con 0x55adc2f10830 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6954c00640 1 -- 172.21.15.18:0/2159510322 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695c00e530 con 0x55adc2f10830 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6954c00640 1 -- 172.21.15.18:0/2159510322 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6958004ec0 con 0x55adc2f10830 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6974b66740 10 client.28676 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6974b66740 10 client.28676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6974b66740 20 client.28676 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6974b66740 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55adc2f8fe80 0x55adc2f922a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6974b66740 4 client.28676 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.448+0000 7f6974b66740 1 -- 172.21.15.18:0/2159510322 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adc2e7c8b0 con 0x55adc2f8fe80 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.449+0000 7f696aa00640 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55adc2f8fe80 0x55adc2f922a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.449+0000 7f696aa00640 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55adc2f8fe80 0x55adc2f922a0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d593155300710e8e server_cookie=f80f2ae6775a2ab6 in_seq=0 out_seq=0 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.449+0000 7f6954c00640 10 client.28676 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.449+0000 7f6954c00640 1 -- 172.21.15.18:0/2159510322 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adc2e7c8b0 con 0x55adc2f8fe80 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.449+0000 7f6954c00640 10 client.28676 handle_command_reply: tid=0 2024-04-16T17:16:07.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.449+0000 7f6974b66740 1 client.28676 shutdown 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 -- 172.21.15.18:0/2159510322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 msgr2=0x55adc2f1cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f1cbd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6958006010 tx=0x7f695802bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 -- 172.21.15.18:0/2159510322 shutdown_connections 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55adc2f8fe80 0x55adc2f922a0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adc2f19410 0x55adc2f1d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adc2f10830 0x55adc2f1cbd0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 --2- 172.21.15.18:0/2159510322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adc2f0ff10 0x55adc2f21860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6954c00640 0 client.28676 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 -- 172.21.15.18:0/2159510322 >> 172.21.15.18:0/2159510322 conn(0x55adc2f0a640 msgr2=0x55adc2f09700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 -- 172.21.15.18:0/2159510322 shutdown_connections 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 1 -- 172.21.15.18:0/2159510322 wait complete. 2024-04-16T17:16:07.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.450+0000 7f6974b66740 20 client.28676 trim_cache size 0 max 16384 2024-04-16T17:16:07.455 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 -- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69500a3d20 msgr2=0x7f69500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69500a3d20 0x7f69500a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f696c10a9e0 tx=0x7f6958005dd0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 -- 172.21.15.18:0/4094870348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c19a830 msgr2=0x7f696c19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c19a830 0x7f696c19ace0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6964016fd0 tx=0x7f696400ca50 comp rx=0 tx=0).stop 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 -- 172.21.15.18:0/4094870348 shutdown_connections 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69500a3d20 0x7f69500a61e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f696c19a830 0x7f696c19ace0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f696c109350 0x7f696c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 --2- 172.21.15.18:0/4094870348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f696c104280 0x7f696c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 -- 172.21.15.18:0/4094870348 >> 172.21.15.18:0/4094870348 conn(0x7f696c0ffae0 msgr2=0x7f696c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 -- 172.21.15.18:0/4094870348 shutdown_connections 2024-04-16T17:16:07.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:07.451+0000 7f696be00640 1 -- 172.21.15.18:0/4094870348 wait complete. 2024-04-16T17:16:07.664 INFO:tasks.workunit.client.0.smithi018.stdout: 1 426711 3.60 MB/sec execute 500 sec latency 72.940 ms 2024-04-16T17:16:07.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:07 smithi018 ceph-mon[31776]: pgmap v360: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-04-16T17:16:07.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:07 smithi022 ceph-mon[36023]: pgmap v360: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-04-16T17:16:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:07 smithi043 ceph-mon[36901]: pgmap v360: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-04-16T17:16:08.468 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:16:08.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:16:08.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:16:08.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 427126 3.60 MB/sec execute 501 sec latency 89.554 ms 2024-04-16T17:16:08.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 -- 172.21.15.18:0/81404651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf00074cd0 msgr2=0x7fcf0010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 --2- 172.21.15.18:0/81404651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf00074cd0 0x7fcf0010e180 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fceec009900 tx=0x7fceec02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 -- 172.21.15.18:0/81404651 shutdown_connections 2024-04-16T17:16:08.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 --2- 172.21.15.18:0/81404651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0010e6c0 0x7fcf00110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 --2- 172.21.15.18:0/81404651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf00074cd0 0x7fcf0010e180 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 --2- 172.21.15.18:0/81404651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf000743b0 0x7fcf00074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 -- 172.21.15.18:0/81404651 >> 172.21.15.18:0/81404651 conn(0x7fcf000fd950 msgr2=0x7fcf000ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 -- 172.21.15.18:0/81404651 shutdown_connections 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.668+0000 7fcf06200640 1 -- 172.21.15.18:0/81404651 wait complete. 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.669+0000 7fcf06200640 1 Processor -- start 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.669+0000 7fcf06200640 1 -- start start 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.669+0000 7fcf06200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf000743b0 0x7fcf001a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.669+0000 7fcf06200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf00074cd0 0x7fcf001a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.669+0000 7fcf06200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0010e6c0 0x7fcf0019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.669+0000 7fcf06200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00114420 con 0x7fcf00074cd0 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.669+0000 7fcf06200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001142a0 con 0x7fcf000743b0 2024-04-16T17:16:08.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.669+0000 7fcf06200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001145a0 con 0x7fcf0010e6c0 2024-04-16T17:16:08.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fceffe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf00074cd0 0x7fcf001a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fceffe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf00074cd0 0x7fcf001a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39868/0 (socket says 172.21.15.18:39868) 2024-04-16T17:16:08.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fceffe00640 1 -- 172.21.15.18:0/1555121857 learned_addr learned my addr 172.21.15.18:0/1555121857 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fcf05800640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0010e6c0 0x7fcf0019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fceffe00640 1 -- 172.21.15.18:0/1555121857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0010e6c0 msgr2=0x7fcf0019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fceffe00640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0010e6c0 0x7fcf0019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fceffe00640 1 -- 172.21.15.18:0/1555121857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf000743b0 msgr2=0x7fcf001a0680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:16:08.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fceffe00640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf000743b0 0x7fcf001a0680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.670+0000 7fceffe00640 1 -- 172.21.15.18:0/1555121857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0019b000 con 0x7fcf00074cd0 2024-04-16T17:16:08.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.671+0000 7fceffe00640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf00074cd0 0x7fcf001a0bc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fceec000c00 tx=0x7fceec004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.671+0000 7fcefd600640 1 -- 172.21.15.18:0/1555121857 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec042020 con 0x7fcf00074cd0 2024-04-16T17:16:08.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.671+0000 7fcf06200640 1 -- 172.21.15.18:0/1555121857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0019b290 con 0x7fcf00074cd0 2024-04-16T17:16:08.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.671+0000 7fcefd600640 1 -- 172.21.15.18:0/1555121857 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceec0054e0 con 0x7fcf00074cd0 2024-04-16T17:16:08.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.671+0000 7fcefd600640 1 -- 172.21.15.18:0/1555121857 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec003b20 con 0x7fcf00074cd0 2024-04-16T17:16:08.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.671+0000 7fcf06200640 1 -- 172.21.15.18:0/1555121857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf001a7600 con 0x7fcf00074cd0 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.676+0000 7fcefd600640 1 -- 172.21.15.18:0/1555121857 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fceec045020 con 0x7fcf00074cd0 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.676+0000 7fcf0a584740 1 Processor -- start 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.676+0000 7fcf0a584740 1 -- start start 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.676+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c1e90 0x55d4035c2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.676+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.676+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d4035cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.677+0000 7fcf0a584740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d403526cb0 con 0x55d4035c1e90 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.677+0000 7fcf0a584740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d4034cd370 con 0x55d4035cb390 2024-04-16T17:16:08.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.677+0000 7fcf0a584740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d403522c70 con 0x55d4035c27b0 2024-04-16T17:16:08.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.677+0000 7fcf04e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.677+0000 7fceffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d4035cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.677+0000 7fceffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d4035cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56534/0 (socket says 172.21.15.18:56534) 2024-04-16T17:16:08.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.677+0000 7fceffe00640 1 -- 172.21.15.18:0/2537633332 learned_addr learned my addr 172.21.15.18:0/2537633332 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fcefd600640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcee80a3d20 0x7fcee80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fcf05800640 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c1e90 0x55d4035c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fceffe00640 1 -- 172.21.15.18:0/2537633332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 msgr2=0x55d4035cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fceffe00640 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fceffe00640 1 -- 172.21.15.18:0/2537633332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c1e90 msgr2=0x55d4035c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fceffe00640 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c1e90 0x55d4035c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fceffe00640 1 -- 172.21.15.18:0/2537633332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4034ec910 con 0x55d4035cb390 2024-04-16T17:16:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fcf05800640 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c1e90 0x55d4035c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fcefd600640 1 -- 172.21.15.18:0/1555121857 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fceec0f2550 con 0x7fcf00074cd0 2024-04-16T17:16:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.678+0000 7fcf05800640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcee80a3d20 0x7fcee80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.679+0000 7fceffe00640 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d4035cd780 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fceec0ef860 tx=0x7fceec004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5868975713b23c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.679+0000 7fcef2000640 1 -- 172.21.15.18:0/2537633332 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec0bdc80 con 0x55d4035cb390 2024-04-16T17:16:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.679+0000 7fcef2000640 1 -- 172.21.15.18:0/2537633332 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceec0be840 con 0x55d4035cb390 2024-04-16T17:16:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.679+0000 7fcef2000640 1 -- 172.21.15.18:0/2537633332 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec0bdaa0 con 0x55d4035cb390 2024-04-16T17:16:08.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf05800640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcee80a3d20 0x7fcee80a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005fd0 tx=0x7fcef8014040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 -- 172.21.15.18:0/2537633332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 msgr2=0x55d4035cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d4035cd780 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fceec0ef860 tx=0x7fceec004610 comp rx=0 tx=0).stop 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 -- 172.21.15.18:0/2537633332 shutdown_connections 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d4035cd780 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 --2- 172.21.15.18:0/2537633332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c1e90 0x55d4035c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 -- 172.21.15.18:0/2537633332 >> 172.21.15.18:0/2537633332 conn(0x55d4035bc5c0 msgr2=0x55d4035bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 -- 172.21.15.18:0/2537633332 shutdown_connections 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.680+0000 7fcf0a584740 1 -- 172.21.15.18:0/2537633332 wait complete. 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.681+0000 7fcf0a584740 1 -- start start 2024-04-16T17:16:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.681+0000 7fcf0a584740 1 Processor -- start 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c27b0 0x55d4035c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 0x55d4035c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf0a584740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d4034cd370 con 0x55d4035cb390 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf0a584740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d4034e0730 con 0x55d4035c27b0 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf0a584740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d4035cdf40 con 0x55d4035c1e90 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf0a584740 10 client.0 _ll_register_callbacks cb 0x55d4034af1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fceffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c27b0 0x55d4035c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fceffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c27b0 0x55d4035c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56548/0 (socket says 172.21.15.18:56548) 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fceffe00640 1 -- 172.21.15.18:0/4091136287 learned_addr learned my addr 172.21.15.18:0/4091136287 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf05800640 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 0x55d4035c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.682+0000 7fcf04e00640 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fceffe00640 1 -- 172.21.15.18:0/4091136287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 msgr2=0x55d4035c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fceffe00640 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fceffe00640 1 -- 172.21.15.18:0/4091136287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 msgr2=0x55d4035c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fceffe00640 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 0x55d4035c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fceffe00640 1 -- 172.21.15.18:0/4091136287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4033017e0 con 0x55d4035c27b0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fceffe00640 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c27b0 0x55d4035c89e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fceec0be9f0 tx=0x7fceec005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=325f58d7da06ed68 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fcee6a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fcee6a00640 1 -- 172.21.15.18:0/4091136287 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec0bdaa0 con 0x55d4035c27b0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fcee6a00640 1 -- 172.21.15.18:0/4091136287 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fceec0910d0 con 0x55d4035c27b0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fcf0a584740 1 -- 172.21.15.18:0/4091136287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceec0be840 con 0x55d4035c27b0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.683+0000 7fcee6a00640 1 -- 172.21.15.18:0/4091136287 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fceec0bdc80 con 0x55d4035c27b0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcf0a584740 10 client.28685 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcf0a584740 1 -- 172.21.15.18:0/4091136287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d403361f70 con 0x55d4035c27b0 2024-04-16T17:16:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcee6a00640 1 -- 172.21.15.18:0/4091136287 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceec0be840 con 0x55d4035c27b0 2024-04-16T17:16:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcee6a00640 1 -- 172.21.15.18:0/4091136287 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fceec0b9e90 con 0x55d4035c27b0 2024-04-16T17:16:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcf0a584740 10 client.28685 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcf0a584740 10 client.28685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcf0a584740 20 client.28685 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcf0a584740 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d4035db2f0 0x55d4035dd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcf0a584740 4 client.28685 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.684+0000 7fcf0a584740 1 -- 172.21.15.18:0/4091136287 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4035346d0 con 0x55d4035db2f0 2024-04-16T17:16:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.685+0000 7fcf04e00640 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d4035db2f0 0x55d4035dd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.685+0000 7fcf04e00640 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d4035db2f0 0x55d4035dd6e0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52ec2c3ad310da3 server_cookie=fab6e80f11a294a4 in_seq=0 out_seq=0 2024-04-16T17:16:08.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.685+0000 7fcee6a00640 10 client.28685 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:08.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcee6a00640 1 -- 172.21.15.18:0/4091136287 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4035346d0 con 0x55d4035db2f0 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcee6a00640 10 client.28685 handle_command_reply: tid=0 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 client.28685 shutdown 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 -- 172.21.15.18:0/4091136287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c27b0 msgr2=0x55d4035c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c27b0 0x55d4035c89e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fceec0be9f0 tx=0x7fceec005d50 comp rx=0 tx=0).stop 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 -- 172.21.15.18:0/4091136287 shutdown_connections 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d4035db2f0 0x55d4035dd6e0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 0x55d4035c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c27b0 0x55d4035c89e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 --2- 172.21.15.18:0/4091136287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.686+0000 7fcf0a584740 1 -- 172.21.15.18:0/4091136287 >> 172.21.15.18:0/4091136287 conn(0x55d4035bc5c0 msgr2=0x55d4035be310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.687+0000 7fcee6a00640 0 client.28685 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.687+0000 7fcf0a584740 1 -- 172.21.15.18:0/4091136287 shutdown_connections 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.687+0000 7fcf0a584740 1 -- 172.21.15.18:0/4091136287 wait complete. 2024-04-16T17:16:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.687+0000 7fcf0a584740 20 client.28685 trim_cache size 0 max 16384 2024-04-16T17:16:08.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fcf0a584740 1 Processor -- start 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fcf0a584740 1 -- start start 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c1e90 0x55d4035cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035d6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 0x55d4035bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fcf0a584740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001b4050 con 0x55d4035cb390 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fcf0a584740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001b1920 con 0x55d4035c1e90 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fcf0a584740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001af770 con 0x55d4035c27b0 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.706+0000 7fceffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035d6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcf04e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c1e90 0x55d4035cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fceffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035d6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35312/0 (socket says 172.21.15.18:35312) 2024-04-16T17:16:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fceffe00640 1 -- 172.21.15.18:0/2705841514 learned_addr learned my addr 172.21.15.18:0/2705841514 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcf05800640 1 --2- 172.21.15.18:0/2705841514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 0x55d4035bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcf04e00640 1 -- 172.21.15.18:0/2705841514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 msgr2=0x55d4035d6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcf04e00640 1 --2- 172.21.15.18:0/2705841514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035d6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcf04e00640 1 -- 172.21.15.18:0/2705841514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 msgr2=0x55d4035bb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcf04e00640 1 --2- 172.21.15.18:0/2705841514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 0x55d4035bb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcf04e00640 1 -- 172.21.15.18:0/2705841514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4035c25e0 con 0x55d4035c1e90 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcf04e00640 1 --2- 172.21.15.18:0/2705841514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c1e90 0x55d4035cabd0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fceec0be080 tx=0x7fcef4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad6c2d9e7cba743 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.707+0000 7fcef1600640 1 -- 172.21.15.18:0/2705841514 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4014190 con 0x55d4035c1e90 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcef1600640 1 -- 172.21.15.18:0/2705841514 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef401f020 con 0x55d4035c1e90 2024-04-16T17:16:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcef1600640 1 -- 172.21.15.18:0/2705841514 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef401b590 con 0x55d4035c1e90 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 -- 172.21.15.18:0/2705841514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c1e90 msgr2=0x55d4035cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 --2- 172.21.15.18:0/2705841514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c1e90 0x55d4035cabd0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fceec0be080 tx=0x7fcef4005f90 comp rx=0 tx=0).stop 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 -- 172.21.15.18:0/2705841514 shutdown_connections 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 --2- 172.21.15.18:0/2705841514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035cb390 0x55d4035bb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 --2- 172.21.15.18:0/2705841514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c27b0 0x55d4035d6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 --2- 172.21.15.18:0/2705841514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035c1e90 0x55d4035cabd0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 -- 172.21.15.18:0/2705841514 >> 172.21.15.18:0/2705841514 conn(0x55d4035bc5c0 msgr2=0x55d4035bacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 -- 172.21.15.18:0/2705841514 shutdown_connections 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.708+0000 7fcf0a584740 1 -- 172.21.15.18:0/2705841514 wait complete. 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.709+0000 7fcf0a584740 1 -- start start 2024-04-16T17:16:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.709+0000 7fcf0a584740 1 Processor -- start 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.709+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c27b0 0x55d4035cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf0a584740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d403640710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf0a584740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001afde0 con 0x55d4035c27b0 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf0a584740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001b1250 con 0x55d4035cb390 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf0a584740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001b04b0 con 0x55d4035c1e90 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf0a584740 10 client.0 _ll_register_callbacks cb 0x7fceec005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf04e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fceffe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c27b0 0x55d4035cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf04e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35322/0 (socket says 172.21.15.18:35322) 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf04e00640 1 -- 172.21.15.18:0/3614232089 learned_addr learned my addr 172.21.15.18:0/3614232089 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf05800640 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d403640710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf04e00640 1 -- 172.21.15.18:0/3614232089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 msgr2=0x55d403640710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf04e00640 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d403640710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf04e00640 1 -- 172.21.15.18:0/3614232089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c27b0 msgr2=0x55d4035cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf04e00640 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c27b0 0x55d4035cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fcf04e00640 1 -- 172.21.15.18:0/3614232089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4031f9170 con 0x55d4035c1e90 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.710+0000 7fceffe00640 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c27b0 0x55d4035cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.711+0000 7fcf04e00640 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035cf390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009700 tx=0x7fcef4014220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23c88fe9f99effa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.711+0000 7fcee6a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.711+0000 7fcee6a00640 1 -- 172.21.15.18:0/3614232089 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4014bf0 con 0x55d4035c1e90 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.711+0000 7fcf0a584740 1 -- 172.21.15.18:0/3614232089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef401f020 con 0x55d4035c1e90 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.711+0000 7fcee6a00640 1 -- 172.21.15.18:0/3614232089 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fcef4005020 con 0x55d4035c1e90 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.711+0000 7fcee6a00640 1 -- 172.21.15.18:0/3614232089 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcef4034050 con 0x55d4035c1e90 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.711+0000 7fcf0a584740 10 client.28552 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.711+0000 7fcf0a584740 1 -- 172.21.15.18:0/3614232089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d403641e50 con 0x55d4035c1e90 2024-04-16T17:16:08.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fcee6a00640 1 -- 172.21.15.18:0/3614232089 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef401f020 con 0x55d4035c1e90 2024-04-16T17:16:08.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fcee6a00640 1 -- 172.21.15.18:0/3614232089 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcef401b330 con 0x55d4035c1e90 2024-04-16T17:16:08.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fcf0a584740 10 client.28552 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:08.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fcf0a584740 10 client.28552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fcf0a584740 20 client.28552 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fcf0a584740 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d403642470 0x55d403644860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fcf0a584740 4 client.28552 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fcf0a584740 1 -- 172.21.15.18:0/3614232089 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d40352e540 con 0x55d403642470 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.712+0000 7fceffe00640 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d403642470 0x55d403644860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.713+0000 7fceffe00640 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d403642470 0x55d403644860 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb146ecca89e0d69 server_cookie=bd10f17a527a4c09 in_seq=0 out_seq=0 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.713+0000 7fcee6a00640 10 client.28552 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcee6a00640 1 -- 172.21.15.18:0/3614232089 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d40352e540 con 0x55d403642470 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcee6a00640 10 client.28552 handle_command_reply: tid=0 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 client.28552 shutdown 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 -- 172.21.15.18:0/3614232089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 msgr2=0x55d4035cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035cf390 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009700 tx=0x7fcef4014220 comp rx=0 tx=0).stop 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 -- 172.21.15.18:0/3614232089 shutdown_connections 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d403642470 0x55d403644860 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4035cb390 0x55d403640710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d4035c27b0 0x55d4035cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 --2- 172.21.15.18:0/3614232089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4035c1e90 0x55d4035cf390 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcee6a00640 0 client.28552 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 -- 172.21.15.18:0/3614232089 >> 172.21.15.18:0/3614232089 conn(0x55d4035bc5c0 msgr2=0x55d4035c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.715+0000 7fcf0a584740 1 -- 172.21.15.18:0/3614232089 shutdown_connections 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.716+0000 7fcf0a584740 1 -- 172.21.15.18:0/3614232089 wait complete. 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.716+0000 7fcf0a584740 20 client.28552 trim_cache size 0 max 16384 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "29e6f4e8-011b-4284-b2ab-67262bd90c68", 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:16:08.718 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:16:08.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 -- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcee80a3d20 msgr2=0x7fcee80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcee80a3d20 0x7fcee80a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005fd0 tx=0x7fcef8014040 comp rx=0 tx=0).stop 2024-04-16T17:16:08.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 -- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf00074cd0 msgr2=0x7fcf001a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf00074cd0 0x7fcf001a0bc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fceec000c00 tx=0x7fceec004360 comp rx=0 tx=0).stop 2024-04-16T17:16:08.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 -- 172.21.15.18:0/1555121857 shutdown_connections 2024-04-16T17:16:08.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcee80a3d20 0x7fcee80a61e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0010e6c0 0x7fcf0019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf00074cd0 0x7fcf001a0bc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 --2- 172.21.15.18:0/1555121857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf000743b0 0x7fcf001a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 -- 172.21.15.18:0/1555121857 >> 172.21.15.18:0/1555121857 conn(0x7fcf000fd950 msgr2=0x7fcf000fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 -- 172.21.15.18:0/1555121857 shutdown_connections 2024-04-16T17:16:08.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.717+0000 7fcf06200640 1 -- 172.21.15.18:0/1555121857 wait complete. 2024-04-16T17:16:08.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:08.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.939+0000 7f66a5a00640 1 -- 172.21.15.18:0/3109617824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a010c400 msgr2=0x7f66a010e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.939+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3109617824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a010c400 0x7f66a010e870 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6698009900 tx=0x7f669802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.939+0000 7f66a5a00640 1 -- 172.21.15.18:0/3109617824 shutdown_connections 2024-04-16T17:16:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.939+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3109617824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a010c400 0x7f66a010e870 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.939+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3109617824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a0103900 0x7f66a010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.939+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3109617824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a0102fe0 0x7f66a01033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.939+0000 7f66a5a00640 1 -- 172.21.15.18:0/3109617824 >> 172.21.15.18:0/3109617824 conn(0x7f66a0077ab0 msgr2=0x7f66a00fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.939+0000 7f66a5a00640 1 -- 172.21.15.18:0/3109617824 shutdown_connections 2024-04-16T17:16:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.940+0000 7f66a5a00640 1 -- 172.21.15.18:0/3109617824 wait complete. 2024-04-16T17:16:08.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.940+0000 7f66a5a00640 1 Processor -- start 2024-04-16T17:16:08.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f66a5a00640 1 -- start start 2024-04-16T17:16:08.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f66a5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a0102fe0 0x7f66a0100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f66a5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a0103900 0x7f66a0101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f66a5a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a010c400 0x7f66a01016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f66a5a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01121e0 con 0x7f66a010c400 2024-04-16T17:16:08.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f66a5a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0112060 con 0x7f66a0102fe0 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f66a5a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0112360 con 0x7f66a0103900 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f669fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a0102fe0 0x7f66a0100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f669fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a0102fe0 0x7f66a0100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56584/0 (socket says 172.21.15.18:56584) 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f669fe00640 1 -- 172.21.15.18:0/3259237815 learned_addr learned my addr 172.21.15.18:0/3259237815 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f6697e00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a0103900 0x7f66a0101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.941+0000 7f66a5000640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a010c400 0x7f66a01016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669fe00640 1 -- 172.21.15.18:0/3259237815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a0103900 msgr2=0x7f66a0101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669fe00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a0103900 0x7f66a0101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669fe00640 1 -- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a010c400 msgr2=0x7f66a01016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669fe00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a010c400 0x7f66a01016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669fe00640 1 -- 172.21.15.18:0/3259237815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a01a6be0 con 0x7f66a0102fe0 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669fe00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a0102fe0 0x7f66a0100c30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f669000ebc0 tx=0x7f669000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669d600640 1 -- 172.21.15.18:0/3259237815 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690010630 con 0x7f66a0102fe0 2024-04-16T17:16:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669d600640 1 -- 172.21.15.18:0/3259237815 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6690010cb0 con 0x7f66a0102fe0 2024-04-16T17:16:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f669d600640 1 -- 172.21.15.18:0/3259237815 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690018bd0 con 0x7f66a0102fe0 2024-04-16T17:16:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.942+0000 7f66a5a00640 1 -- 172.21.15.18:0/3259237815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a01a6ed0 con 0x7f66a0102fe0 2024-04-16T17:16:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.943+0000 7f66a5a00640 1 -- 172.21.15.18:0/3259237815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a73e0 con 0x7f66a0102fe0 2024-04-16T17:16:08.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.946+0000 7f669d600640 1 -- 172.21.15.18:0/3259237815 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66900107d0 con 0x7f66a0102fe0 2024-04-16T17:16:08.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.947+0000 7f66a9e19740 1 Processor -- start 2024-04-16T17:16:08.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.947+0000 7f66a9e19740 1 -- start start 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.947+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.947+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 0x559a63bbb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bbb890 0x559a63bbdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a9e19740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559a63b18120 con 0x559a63bbb890 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a9e19740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559a63abe340 con 0x559a63bb2390 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a9e19740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a63b140e0 con 0x559a63bb2cb0 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56590/0 (socket says 172.21.15.18:56590) 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a5000640 1 -- 172.21.15.18:0/2020308320 learned_addr learned my addr 172.21.15.18:0/2020308320 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f669fe00640 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 0x559a63bbb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a5000640 1 -- 172.21.15.18:0/2020308320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 msgr2=0x559a63bbb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f6697e00640 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bbb890 0x559a63bbdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a5000640 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 0x559a63bbb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a5000640 1 -- 172.21.15.18:0/2020308320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bbb890 msgr2=0x559a63bbdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a5000640 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bbb890 0x559a63bbdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f66a5000640 1 -- 172.21.15.18:0/2020308320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a638f47d0 con 0x559a63bb2390 2024-04-16T17:16:08.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f6697e00640 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bbb890 0x559a63bbdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:16:08.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.948+0000 7f669d600640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3da0 0x7f66840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.949+0000 7f669d600640 1 -- 172.21.15.18:0/3259237815 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f66900d12d0 con 0x7f66a0102fe0 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.949+0000 7f66a5000640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3da0 0x7f66840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.949+0000 7f66a5000640 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb2770 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6698009900 tx=0x7f6698005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43671949bdbf377b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.949+0000 7f6695600640 1 -- 172.21.15.18:0/2020308320 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669802c910 con 0x559a63bb2390 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.949+0000 7f6695600640 1 -- 172.21.15.18:0/2020308320 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6698042050 con 0x559a63bb2390 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.949+0000 7f6695600640 1 -- 172.21.15.18:0/2020308320 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698050070 con 0x559a63bb2390 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.950+0000 7f66a9e19740 1 -- 172.21.15.18:0/2020308320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 msgr2=0x559a63bb2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.950+0000 7f66a9e19740 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb2770 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6698009900 tx=0x7f6698005de0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.950+0000 7f66a5000640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3da0 0x7f66840a6260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6698049000 tx=0x7f669804c000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.951+0000 7f66a9e19740 1 -- 172.21.15.18:0/2020308320 shutdown_connections 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.951+0000 7f66a9e19740 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bbb890 0x559a63bbdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.951+0000 7f66a9e19740 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 0x559a63bbb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.951+0000 7f66a9e19740 1 --2- 172.21.15.18:0/2020308320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb2770 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.951+0000 7f66a9e19740 1 -- 172.21.15.18:0/2020308320 >> 172.21.15.18:0/2020308320 conn(0x559a63bacac0 msgr2=0x559a63bacea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.951+0000 7f66a9e19740 1 -- 172.21.15.18:0/2020308320 shutdown_connections 2024-04-16T17:16:08.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.951+0000 7f66a9e19740 1 -- 172.21.15.18:0/2020308320 wait complete. 2024-04-16T17:16:08.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.952+0000 7f66a9e19740 1 -- start start 2024-04-16T17:16:08.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.952+0000 7f66a9e19740 1 Processor -- start 2024-04-16T17:16:08.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.953+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.953+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bb8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.953+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 0x559a63bb92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.953+0000 7f669fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f669fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56592/0 (socket says 172.21.15.18:56592) 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f669fe00640 1 -- 172.21.15.18:0/3235787849 learned_addr learned my addr 172.21.15.18:0/3235787849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559a63bbe680 con 0x559a63bb2cb0 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559a63ad1710 con 0x559a63bb2390 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a63bbe800 con 0x559a63bbb890 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f66a9e19740 10 client.0 _ll_register_callbacks cb 0x559a63accaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:08.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f66a5000640 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 0x559a63bb92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f669fe00640 1 -- 172.21.15.18:0/3235787849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 msgr2=0x559a63bb92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f6697e00640 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bb8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f669fe00640 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 0x559a63bb92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f669fe00640 1 -- 172.21.15.18:0/3235787849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 msgr2=0x559a63bb8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.954+0000 7f669fe00640 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bb8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.955+0000 7f669fe00640 1 -- 172.21.15.18:0/3235787849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a6382a3f0 con 0x559a63bb2390 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.955+0000 7f6697e00640 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bb8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.955+0000 7f669fe00640 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb8860 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f669000eb90 tx=0x7f669000ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb414a0b29f284b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.955+0000 7f668e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.955+0000 7f668e000640 1 -- 172.21.15.18:0/3235787849 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690010e60 con 0x559a63bb2390 2024-04-16T17:16:08.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.955+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6698042050 con 0x559a63bb2390 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f668e000640 1 -- 172.21.15.18:0/3235787849 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f669009c200 con 0x559a63bb2390 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f668e000640 1 -- 172.21.15.18:0/3235787849 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f669009cb80 con 0x559a63bb2390 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f66a9e19740 10 client.28703 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a637ea0f0 con 0x559a63bb2390 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f668e000640 1 -- 172.21.15.18:0/3235787849 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6698042050 con 0x559a63bb2390 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f668e000640 1 -- 172.21.15.18:0/3235787849 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6690087070 con 0x559a63bb2390 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f66a9e19740 10 client.28703 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f66a9e19740 10 client.28703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f66a9e19740 20 client.28703 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f66a9e19740 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a63bd44b0 0x559a63bd6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f66a9e19740 4 client.28703 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.956+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a63b25b40 con 0x559a63bd44b0 2024-04-16T17:16:08.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.957+0000 7f6697e00640 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a63bd44b0 0x559a63bd6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.957+0000 7f6697e00640 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a63bd44b0 0x559a63bd6940 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a52c745d7e7bf2 server_cookie=572cff90b9ca72ef in_seq=0 out_seq=0 2024-04-16T17:16:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.957+0000 7f668e000640 10 client.28703 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.958+0000 7f668e000640 1 -- 172.21.15.18:0/3235787849 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a63b25b40 con 0x559a63bd44b0 2024-04-16T17:16:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.958+0000 7f668e000640 10 client.28703 handle_command_reply: tid=0 2024-04-16T17:16:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.958+0000 7f66a9e19740 1 client.28703 shutdown 2024-04-16T17:16:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 msgr2=0x559a63bb8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb8860 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f669000eb90 tx=0x7f669000ef80 comp rx=0 tx=0).stop 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 shutdown_connections 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a63bd44b0 0x559a63bd6940 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 0x559a63bb92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bb8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f668e000640 0 client.28703 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 --2- 172.21.15.18:0/3235787849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bb8860 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 >> 172.21.15.18:0/3235787849 conn(0x559a63bacac0 msgr2=0x559a63bae810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 shutdown_connections 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 1 -- 172.21.15.18:0/3235787849 wait complete. 2024-04-16T17:16:08.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.959+0000 7f66a9e19740 20 client.28703 trim_cache size 0 max 16384 2024-04-16T17:16:08.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.979+0000 7f66a9e19740 1 Processor -- start 2024-04-16T17:16:08.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.979+0000 7f66a9e19740 1 -- start start 2024-04-16T17:16:08.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bbaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bcfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 0x559a63bb77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f66a9e19740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01afc60 con 0x559a63bb2cb0 2024-04-16T17:16:08.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f66a9e19740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ad530 con 0x559a63bb2390 2024-04-16T17:16:08.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f66a9e19740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ab3a0 con 0x559a63bbb890 2024-04-16T17:16:08.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f669fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bbaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f669fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bbaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56604/0 (socket says 172.21.15.18:56604) 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f669fe00640 1 -- 172.21.15.18:0/69849963 learned_addr learned my addr 172.21.15.18:0/69849963 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f6697e00640 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bcfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f66a5000640 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 0x559a63bb77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f669fe00640 1 -- 172.21.15.18:0/69849963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 msgr2=0x559a63bb77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f669fe00640 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 0x559a63bb77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f669fe00640 1 -- 172.21.15.18:0/69849963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 msgr2=0x559a63bcfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f669fe00640 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bcfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f669fe00640 1 -- 172.21.15.18:0/69849963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a63bb6db0 con 0x559a63bb2390 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.980+0000 7f6697e00640 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bcfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.981+0000 7f669fe00640 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bbaf90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f669009cab0 tx=0x7f6690018670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f519ef769f0ca94d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.981+0000 7f6694c00640 1 -- 172.21.15.18:0/69849963 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690010e60 con 0x559a63bb2390 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.981+0000 7f6694c00640 1 -- 172.21.15.18:0/69849963 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66900a0020 con 0x559a63bb2390 2024-04-16T17:16:08.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.981+0000 7f6694c00640 1 -- 172.21.15.18:0/69849963 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669009c710 con 0x559a63bb2390 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 -- 172.21.15.18:0/69849963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 msgr2=0x559a63bbaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bbaf90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f669009cab0 tx=0x7f6690018670 comp rx=0 tx=0).stop 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 -- 172.21.15.18:0/69849963 shutdown_connections 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bbb890 0x559a63bb77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2cb0 0x559a63bcfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 --2- 172.21.15.18:0/69849963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bb2390 0x559a63bbaf90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 -- 172.21.15.18:0/69849963 >> 172.21.15.18:0/69849963 conn(0x559a63bacac0 msgr2=0x559a63babb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 -- 172.21.15.18:0/69849963 shutdown_connections 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 -- 172.21.15.18:0/69849963 wait complete. 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 -- start start 2024-04-16T17:16:08.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.982+0000 7f66a9e19740 1 Processor -- start 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2390 0x559a63bc8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 0x559a63bc8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f66a9e19740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bbb890 0x559a63c32a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f66a9e19740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ab9f0 con 0x559a63bb2390 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f66a9e19740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ace60 con 0x559a63bbb890 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f66a9e19740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ac0c0 con 0x559a63bb2cb0 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f66a9e19740 10 client.0 _ll_register_callbacks cb 0x7f66900099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f669fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2390 0x559a63bc8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f66a5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bbb890 0x559a63c32a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f669fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2390 0x559a63bc8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39946/0 (socket says 172.21.15.18:39946) 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f669fe00640 1 -- 172.21.15.18:0/519418167 learned_addr learned my addr 172.21.15.18:0/519418167 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.983+0000 7f6697e00640 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 0x559a63bc8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f66a5000640 1 -- 172.21.15.18:0/519418167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 msgr2=0x559a63bc8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f66a5000640 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 0x559a63bc8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f66a5000640 1 -- 172.21.15.18:0/519418167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2390 msgr2=0x559a63bc8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f66a5000640 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2390 0x559a63bc8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f66a5000640 1 -- 172.21.15.18:0/519418167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a63c33240 con 0x559a63bbb890 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f66a5000640 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bbb890 0x559a63c32a40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f669802dfd0 tx=0x7f6698009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f612368e73327f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f668e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f668e000640 1 -- 172.21.15.18:0/519418167 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698005510 con 0x559a63bbb890 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f668e000640 1 -- 172.21.15.18:0/519418167 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f66980046c0 con 0x559a63bbb890 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.984+0000 7f66a9e19740 1 -- 172.21.15.18:0/519418167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66900a0020 con 0x559a63bbb890 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f668e000640 1 -- 172.21.15.18:0/519418167 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6698067050 con 0x559a63bbb890 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f66a9e19740 10 client.28709 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f66a9e19740 1 -- 172.21.15.18:0/519418167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a63c3b610 con 0x559a63bbb890 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f668e000640 1 -- 172.21.15.18:0/519418167 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66900a0020 con 0x559a63bbb890 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f668e000640 1 -- 172.21.15.18:0/519418167 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f669805e470 con 0x559a63bbb890 2024-04-16T17:16:08.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f66a9e19740 10 client.28709 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f66a9e19740 10 client.28709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f66a9e19740 20 client.28709 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f66a9e19740 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a63c3b900 0x559a63c3dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f66a9e19740 4 client.28709 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f66a9e19740 1 -- 172.21.15.18:0/519418167 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a63b1f9b0 con 0x559a63c3b900 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.985+0000 7f669fe00640 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a63c3b900 0x559a63c3dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.986+0000 7f669fe00640 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a63c3b900 0x559a63c3dcf0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c57d6850437323 server_cookie=b4cf3f51d1520a5c in_seq=0 out_seq=0 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.986+0000 7f668e000640 10 client.28709 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f668e000640 1 -- 172.21.15.18:0/519418167 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a63b1f9b0 con 0x559a63c3b900 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f668e000640 10 client.28709 handle_command_reply: tid=0 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 client.28709 shutdown 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 -- 172.21.15.18:0/519418167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bbb890 msgr2=0x559a63c32a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bbb890 0x559a63c32a40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f669802dfd0 tx=0x7f6698009670 comp rx=0 tx=0).stop 2024-04-16T17:16:08.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 -- 172.21.15.18:0/519418167 shutdown_connections 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a63c3b900 0x559a63c3dcf0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a63bbb890 0x559a63c32a40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a63bb2cb0 0x559a63bc8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f668e000640 0 client.28709 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 --2- 172.21.15.18:0/519418167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a63bb2390 0x559a63bc8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 -- 172.21.15.18:0/519418167 >> 172.21.15.18:0/519418167 conn(0x559a63bacac0 msgr2=0x559a63badef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 -- 172.21.15.18:0/519418167 shutdown_connections 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 1 -- 172.21.15.18:0/519418167 wait complete. 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.987+0000 7f66a9e19740 20 client.28709 trim_cache size 0 max 16384 2024-04-16T17:16:08.991 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:16:08.992 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:16:08.992 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:16:08.992 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:16:08.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 -- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3da0 msgr2=0x7f66840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3da0 0x7f66840a6260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6698049000 tx=0x7f669804c000 comp rx=0 tx=0).stop 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 -- 172.21.15.18:0/3259237815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a0102fe0 msgr2=0x7f66a0100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a0102fe0 0x7f66a0100c30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f669000ebc0 tx=0x7f669000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 -- 172.21.15.18:0/3259237815 shutdown_connections 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3da0 0x7f66840a6260 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66a010c400 0x7f66a01016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a0103900 0x7f66a0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 --2- 172.21.15.18:0/3259237815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66a0102fe0 0x7f66a0100c30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.988+0000 7f66a5a00640 1 -- 172.21.15.18:0/3259237815 >> 172.21.15.18:0/3259237815 conn(0x7f66a0077ab0 msgr2=0x7f66a00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.989+0000 7f66a5a00640 1 -- 172.21.15.18:0/3259237815 shutdown_connections 2024-04-16T17:16:08.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:08.989+0000 7f66a5a00640 1 -- 172.21.15.18:0/3259237815 wait complete. 2024-04-16T17:16:09.006 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:16:09.006 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 -- 172.21.15.18:0/332712659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102ae0 msgr2=0x7f4038106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 --2- 172.21.15.18:0/332712659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102ae0 0x7f4038106b20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4024009900 tx=0x7f402402bc50 comp rx=0 tx=0).stop 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 -- 172.21.15.18:0/332712659 shutdown_connections 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 --2- 172.21.15.18:0/332712659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4038107200 0x7f403810edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 --2- 172.21.15.18:0/332712659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102ae0 0x7f4038106b20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 --2- 172.21.15.18:0/332712659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4038102130 0x7f4038102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 -- 172.21.15.18:0/332712659 >> 172.21.15.18:0/332712659 conn(0x7f40380fd9f0 msgr2=0x7f40380ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 -- 172.21.15.18:0/332712659 shutdown_connections 2024-04-16T17:16:09.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.221+0000 7f403e800640 1 -- 172.21.15.18:0/332712659 wait complete. 2024-04-16T17:16:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.222+0000 7f403e800640 1 Processor -- start 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.222+0000 7f403e800640 1 -- start start 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102130 0x7f40381a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403e800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4038107200 0x7f40381a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403819a860 0x7f403819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403e800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f40381122c0 con 0x7f4038107200 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403e800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4038112140 con 0x7f4038102130 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4038112440 con 0x7f403819a860 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102130 0x7f40381a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102130 0x7f40381a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56632/0 (socket says 172.21.15.18:56632) 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403d400640 1 -- 172.21.15.18:0/839345229 learned_addr learned my addr 172.21.15.18:0/839345229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403de00640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403819a860 0x7f403819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.223+0000 7f403ca00640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4038107200 0x7f40381a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403d400640 1 -- 172.21.15.18:0/839345229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403819a860 msgr2=0x7f403819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403d400640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403819a860 0x7f403819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403d400640 1 -- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4038107200 msgr2=0x7f40381a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403d400640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4038107200 0x7f40381a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403d400640 1 -- 172.21.15.18:0/839345229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403819b380 con 0x7f4038102130 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403de00640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403819a860 0x7f403819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403d400640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102130 0x7f40381a06b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f402c00e9f0 tx=0x7f402c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f4036000640 1 -- 172.21.15.18:0/839345229 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c00cd80 con 0x7f4038102130 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f4036000640 1 -- 172.21.15.18:0/839345229 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f402c010910 con 0x7f4038102130 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403819b670 con 0x7f4038102130 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f4036000640 1 -- 172.21.15.18:0/839345229 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c018990 con 0x7f4038102130 2024-04-16T17:16:09.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.224+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40381a78c0 con 0x7f4038102130 2024-04-16T17:16:09.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.226+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403804fe00 con 0x7f4038102130 2024-04-16T17:16:09.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.229+0000 7f4036000640 1 -- 172.21.15.18:0/839345229 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f402c010ac0 con 0x7f4038102130 2024-04-16T17:16:09.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.231+0000 7f4036000640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40200a3d10 0x7f40200a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.232+0000 7f4036000640 1 -- 172.21.15.18:0/839345229 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f402c0d12a0 con 0x7f4038102130 2024-04-16T17:16:09.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.232+0000 7f403ca00640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40200a3d10 0x7f40200a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.232+0000 7f4036000640 1 -- 172.21.15.18:0/839345229 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f402c09ce30 con 0x7f4038102130 2024-04-16T17:16:09.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.233+0000 7f403ca00640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40200a3d10 0x7f40200a61d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f402402c260 tx=0x7f4024005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:09.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.577+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4038102ae0 con 0x7f4038102130 2024-04-16T17:16:09.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.578+0000 7f4036000640 1 -- 172.21.15.18:0/839345229 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11854 (secure 0 0 0) 0x7f402c0a1130 con 0x7f4038102130 2024-04-16T17:16:09.579 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:16:09.579 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:16:00.629944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:16:09.579 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-16T17:16:09.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40200a3d10 msgr2=0x7f40200a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40200a3d10 0x7f40200a61d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f402402c260 tx=0x7f4024005fd0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102130 msgr2=0x7f40381a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102130 0x7f40381a06b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f402c00e9f0 tx=0x7f402c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 shutdown_connections 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f40200a3d10 0x7f40200a61d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403819a860 0x7f403819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4038107200 0x7f40381a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 --2- 172.21.15.18:0/839345229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4038102130 0x7f40381a06b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 >> 172.21.15.18:0/839345229 conn(0x7f40380fd9f0 msgr2=0x7f40380fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 shutdown_connections 2024-04-16T17:16:09.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.581+0000 7f403e800640 1 -- 172.21.15.18:0/839345229 wait complete. 2024-04-16T17:16:09.601 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:16:09.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 428010 3.59 MB/sec execute 502 sec latency 74.686 ms 2024-04-16T17:16:09.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.810+0000 7ffbda200640 1 -- 172.21.15.18:0/1218111165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4101910 msgr2=0x7ffbd4105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.810+0000 7ffbda200640 1 --2- 172.21.15.18:0/1218111165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4101910 0x7ffbd4105950 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009900 tx=0x7ffbc002bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:09.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.811+0000 7ffbda200640 1 -- 172.21.15.18:0/1218111165 shutdown_connections 2024-04-16T17:16:09.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.811+0000 7ffbda200640 1 --2- 172.21.15.18:0/1218111165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd4106030 0x7ffbd410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.811+0000 7ffbda200640 1 --2- 172.21.15.18:0/1218111165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4101910 0x7ffbd4105950 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.811+0000 7ffbda200640 1 --2- 172.21.15.18:0/1218111165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd4100f60 0x7ffbd4101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.811+0000 7ffbda200640 1 -- 172.21.15.18:0/1218111165 >> 172.21.15.18:0/1218111165 conn(0x7ffbd4077ab0 msgr2=0x7ffbd40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:09.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.811+0000 7ffbda200640 1 -- 172.21.15.18:0/1218111165 shutdown_connections 2024-04-16T17:16:09.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.811+0000 7ffbda200640 1 -- 172.21.15.18:0/1218111165 wait complete. 2024-04-16T17:16:09.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.812+0000 7ffbda200640 1 Processor -- start 2024-04-16T17:16:09.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.812+0000 7ffbda200640 1 -- start start 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.812+0000 7ffbda200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd4100f60 0x7ffbd410cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.812+0000 7ffbda200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4106030 0x7ffbd410d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.812+0000 7ffbda200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd41a36c0 0x7ffbd41a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.812+0000 7ffbda200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41121c0 con 0x7ffbd4100f60 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.812+0000 7ffbda200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4112040 con 0x7ffbd41a36c0 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.812+0000 7ffbda200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4112340 con 0x7ffbd4106030 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd41a36c0 0x7ffbd41a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd41a36c0 0x7ffbd41a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56652/0 (socket says 172.21.15.18:56652) 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd9800640 1 -- 172.21.15.18:0/4268196206 learned_addr learned my addr 172.21.15.18:0/4268196206 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd4100f60 0x7ffbd410cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4106030 0x7ffbd410d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd9800640 1 -- 172.21.15.18:0/4268196206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4106030 msgr2=0x7ffbd410d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd9800640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4106030 0x7ffbd410d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd9800640 1 -- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd4100f60 msgr2=0x7ffbd410cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd9800640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd4100f60 0x7ffbd410cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.813+0000 7ffbd9800640 1 -- 172.21.15.18:0/4268196206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41a4180 con 0x7ffbd41a36c0 2024-04-16T17:16:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.814+0000 7ffbd9800640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd41a36c0 0x7ffbd41a3b40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00eb10 tx=0x7ffbcc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:09.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.814+0000 7ffbd1600640 1 -- 172.21.15.18:0/4268196206 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc004b20 con 0x7ffbd41a36c0 2024-04-16T17:16:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.814+0000 7ffbd1600640 1 -- 172.21.15.18:0/4268196206 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc004cc0 con 0x7ffbd41a36c0 2024-04-16T17:16:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.814+0000 7ffbd1600640 1 -- 172.21.15.18:0/4268196206 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc010760 con 0x7ffbd41a36c0 2024-04-16T17:16:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.814+0000 7ffbda200640 1 -- 172.21.15.18:0/4268196206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd41a4470 con 0x7ffbd41a36c0 2024-04-16T17:16:09.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.814+0000 7ffbda200640 1 -- 172.21.15.18:0/4268196206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd41abc60 con 0x7ffbd41a36c0 2024-04-16T17:16:09.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.818+0000 7ffbd1600640 1 -- 172.21.15.18:0/4268196206 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffbcc005020 con 0x7ffbd41a36c0 2024-04-16T17:16:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.819+0000 7ffbde483740 1 Processor -- start 2024-04-16T17:16:09.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.819+0000 7ffbde483740 1 -- start start 2024-04-16T17:16:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.819+0000 7ffbde483740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.819+0000 7ffbde483740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.819+0000 7ffbde483740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 0x55b50d0e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbde483740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b50d042020 con 0x55b50d0e6410 2024-04-16T17:16:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbde483740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b50cfe8460 con 0x55b50d0dcf10 2024-04-16T17:16:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbde483740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b50d03dfe0 con 0x55b50d0dd830 2024-04-16T17:16:09.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbd1600640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbbc0a3da0 0x7ffbbc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbd3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbd1600640 1 -- 172.21.15.18:0/4268196206 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7ffbcc0d1170 con 0x7ffbd41a36c0 2024-04-16T17:16:09.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35404/0 (socket says 172.21.15.18:35404) 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbd8e00640 1 -- 172.21.15.18:0/2061928164 learned_addr learned my addr 172.21.15.18:0/2061928164 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.820+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 0x55b50d0e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbd9800640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbbc0a3da0 0x7ffbbc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbd8e00640 1 -- 172.21.15.18:0/2061928164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 msgr2=0x55b50d0dd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0dd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbd8e00640 1 -- 172.21.15.18:0/2061928164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 msgr2=0x55b50d0e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 0x55b50d0e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbd8e00640 1 -- 172.21.15.18:0/2061928164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50cff0040 con 0x55b50d0dd830 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 0x55b50d0e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e5ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009cf0 tx=0x7ffbc8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d175d57a5850cc41 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:09.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbc6000640 1 -- 172.21.15.18:0/2061928164 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8004230 con 0x55b50d0dd830 2024-04-16T17:16:09.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbc6000640 1 -- 172.21.15.18:0/2061928164 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc80043d0 con 0x55b50d0dd830 2024-04-16T17:16:09.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.821+0000 7ffbc6000640 1 -- 172.21.15.18:0/2061928164 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8019960 con 0x55b50d0dd830 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.822+0000 7ffbde483740 1 -- 172.21.15.18:0/2061928164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 msgr2=0x55b50d0e5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.822+0000 7ffbde483740 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e5ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009cf0 tx=0x7ffbc8011810 comp rx=0 tx=0).stop 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.822+0000 7ffbd9800640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbbc0a3da0 0x7ffbbc0a6260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0ce460 tx=0x7ffbcc005b40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 -- 172.21.15.18:0/2061928164 shutdown_connections 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 0x55b50d0e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e5ed0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 --2- 172.21.15.18:0/2061928164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 -- 172.21.15.18:0/2061928164 >> 172.21.15.18:0/2061928164 conn(0x55b50d0d7640 msgr2=0x55b50d0d7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 -- 172.21.15.18:0/2061928164 shutdown_connections 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 -- 172.21.15.18:0/2061928164 wait complete. 2024-04-16T17:16:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 -- start start 2024-04-16T17:16:09.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.823+0000 7ffbde483740 1 Processor -- start 2024-04-16T17:16:09.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbde483740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbde483740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbde483740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 0x55b50d0e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbde483740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b50d0e9200 con 0x55b50d0e6410 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbde483740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b50cffb830 con 0x55b50d0dcf10 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbde483740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b50d0e9380 con 0x55b50d0dd830 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbde483740 10 client.0 _ll_register_callbacks cb 0x55b50d003150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35420/0 (socket says 172.21.15.18:35420) 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbd8e00640 1 -- 172.21.15.18:0/2938087230 learned_addr learned my addr 172.21.15.18:0/2938087230 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbd8e00640 1 -- 172.21.15.18:0/2938087230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 msgr2=0x55b50d0e34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0e34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbd8e00640 1 -- 172.21.15.18:0/2938087230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 msgr2=0x55b50d0e3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.824+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 0x55b50d0e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.825+0000 7ffbd8e00640 1 -- 172.21.15.18:0/2938087230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50cfff920 con 0x55b50d0dd830 2024-04-16T17:16:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.825+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e3a20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80195c0 tx=0x7ffbc8005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cb75cd92184827b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.825+0000 7ffbbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.825+0000 7ffbbaa00640 1 -- 172.21.15.18:0/2938087230 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8027070 con 0x55b50d0dd830 2024-04-16T17:16:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.826+0000 7ffbbaa00640 1 -- 172.21.15.18:0/2938087230 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7ffbc80037f0 con 0x55b50d0dd830 2024-04-16T17:16:09.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.826+0000 7ffbde483740 1 -- 172.21.15.18:0/2938087230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc80043d0 con 0x55b50d0dd830 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.826+0000 7ffbbaa00640 1 -- 172.21.15.18:0/2938087230 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffbc8033050 con 0x55b50d0dd830 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.826+0000 7ffbde483740 10 client.28588 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.826+0000 7ffbde483740 1 -- 172.21.15.18:0/2938087230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b50cfcd7f0 con 0x55b50d0dd830 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.826+0000 7ffbbaa00640 1 -- 172.21.15.18:0/2938087230 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc80043d0 con 0x55b50d0dd830 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.827+0000 7ffbbaa00640 1 -- 172.21.15.18:0/2938087230 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ffbc8004ba0 con 0x55b50d0dd830 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.827+0000 7ffbde483740 10 client.28588 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.827+0000 7ffbde483740 10 client.28588 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.827+0000 7ffbde483740 20 client.28588 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:09.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.827+0000 7ffbde483740 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b50d0f6770 0x55b50d0f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.827+0000 7ffbde483740 4 client.28588 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.827+0000 7ffbde483740 1 -- 172.21.15.18:0/2938087230 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b50d04fa40 con 0x55b50d0f6770 2024-04-16T17:16:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.827+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b50d0f6770 0x55b50d0f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.828+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b50d0f6770 0x55b50d0f8b60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8dae56bb2ac91fed server_cookie=a237a0f7db75e1a0 in_seq=0 out_seq=0 2024-04-16T17:16:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.828+0000 7ffbbaa00640 10 client.28588 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.828+0000 7ffbbaa00640 1 -- 172.21.15.18:0/2938087230 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b50d04fa40 con 0x55b50d0f6770 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.828+0000 7ffbbaa00640 10 client.28588 handle_command_reply: tid=0 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 client.28588 shutdown 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 -- 172.21.15.18:0/2938087230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 msgr2=0x55b50d0e3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e3a20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80195c0 tx=0x7ffbc8005270 comp rx=0 tx=0).stop 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 -- 172.21.15.18:0/2938087230 shutdown_connections 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b50d0f6770 0x55b50d0f8b60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0e6410 0x55b50d0e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dd830 0x55b50d0e3a20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbbaa00640 0 client.28588 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:09.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 --2- 172.21.15.18:0/2938087230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 -- 172.21.15.18:0/2938087230 >> 172.21.15.18:0/2938087230 conn(0x55b50d0d7640 msgr2=0x55b50d0d9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:09.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 -- 172.21.15.18:0/2938087230 shutdown_connections 2024-04-16T17:16:09.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 1 -- 172.21.15.18:0/2938087230 wait complete. 2024-04-16T17:16:09.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.829+0000 7ffbde483740 20 client.28588 trim_cache size 0 max 16384 2024-04-16T17:16:09.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.859+0000 7ffbde483740 1 Processor -- start 2024-04-16T17:16:09.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.859+0000 7ffbde483740 1 -- start start 2024-04-16T17:16:09.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.859+0000 7ffbde483740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0d7640 0x55b50d0e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.859+0000 7ffbde483740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.859+0000 7ffbde483740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0dd830 0x55b50d0d6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.859+0000 7ffbde483740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b8550 con 0x55b50d0dd830 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.859+0000 7ffbde483740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b5e20 con 0x55b50d0dcf10 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.859+0000 7ffbde483740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b3c90 con 0x55b50d0d7640 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56668/0 (socket says 172.21.15.18:56668) 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd8e00640 1 -- 172.21.15.18:0/1657912008 learned_addr learned my addr 172.21.15.18:0/1657912008 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0d7640 0x55b50d0e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0dd830 0x55b50d0d6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd8e00640 1 -- 172.21.15.18:0/1657912008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0d7640 msgr2=0x55b50d0e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0d7640 0x55b50d0e5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd8e00640 1 -- 172.21.15.18:0/1657912008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0dd830 msgr2=0x55b50d0d6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0dd830 0x55b50d0d6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd8e00640 1 -- 172.21.15.18:0/1657912008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50d0e24b0 con 0x55b50d0dcf10 2024-04-16T17:16:09.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.860+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0d7640 0x55b50d0e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.861+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0f20c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc80198c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fb7d4483cc6e5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.861+0000 7ffbc5600640 1 -- 172.21.15.18:0/1657912008 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc803c070 con 0x55b50d0dcf10 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.861+0000 7ffbc5600640 1 -- 172.21.15.18:0/1657912008 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8015030 con 0x55b50d0dcf10 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.861+0000 7ffbc5600640 1 -- 172.21.15.18:0/1657912008 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8027070 con 0x55b50d0dcf10 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 -- 172.21.15.18:0/1657912008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 msgr2=0x55b50d0f20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0f20c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc80198c0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 -- 172.21.15.18:0/1657912008 shutdown_connections 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0dd830 0x55b50d0d6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dcf10 0x55b50d0f20c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 --2- 172.21.15.18:0/1657912008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0d7640 0x55b50d0e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 -- 172.21.15.18:0/1657912008 >> 172.21.15.18:0/1657912008 conn(0x55b50d0f8ed0 msgr2=0x55b50d0d7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 -- 172.21.15.18:0/1657912008 shutdown_connections 2024-04-16T17:16:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.862+0000 7ffbde483740 1 -- 172.21.15.18:0/1657912008 wait complete. 2024-04-16T17:16:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.863+0000 7ffbde483740 1 -- start start 2024-04-16T17:16:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.863+0000 7ffbde483740 1 Processor -- start 2024-04-16T17:16:09.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbde483740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0d7640 0x55b50d0e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbde483740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dcf10 0x55b50d0e0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbde483740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dd830 0x55b50d0e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbde483740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b42e0 con 0x55b50d0d7640 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbde483740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b5750 con 0x55b50d0dd830 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbde483740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b49b0 con 0x55b50d0dcf10 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbde483740 10 client.0 _ll_register_callbacks cb 0x7ffbc8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dcf10 0x55b50d0e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0d7640 0x55b50d0e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dcf10 0x55b50d0e0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35432/0 (socket says 172.21.15.18:35432) 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd8e00640 1 -- 172.21.15.18:0/3814619206 learned_addr learned my addr 172.21.15.18:0/3814619206 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dd830 0x55b50d0e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd8e00640 1 -- 172.21.15.18:0/3814619206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dd830 msgr2=0x55b50d0e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dd830 0x55b50d0e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd8e00640 1 -- 172.21.15.18:0/3814619206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0d7640 msgr2=0x55b50d0e01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0d7640 0x55b50d0e01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd8e00640 1 -- 172.21.15.18:0/3814619206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50cd543f0 con 0x55b50d0dcf10 2024-04-16T17:16:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.864+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0d7640 0x55b50d0e01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.865+0000 7ffbd8e00640 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dcf10 0x55b50d0e0730 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009250 tx=0x7ffbc8009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff275765e31c4d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.865+0000 7ffbbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.865+0000 7ffbbaa00640 1 -- 172.21.15.18:0/3814619206 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc803c070 con 0x55b50d0dcf10 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.865+0000 7ffbde483740 1 -- 172.21.15.18:0/3814619206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc8015030 con 0x55b50d0dcf10 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.865+0000 7ffbbaa00640 1 -- 172.21.15.18:0/3814619206 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7ffbc8019c80 con 0x55b50d0dcf10 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.865+0000 7ffbbaa00640 1 -- 172.21.15.18:0/3814619206 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffbc8049050 con 0x55b50d0dcf10 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbde483740 10 client.28594 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbde483740 1 -- 172.21.15.18:0/3814619206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b50ce1e7d0 con 0x55b50d0dcf10 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbbaa00640 1 -- 172.21.15.18:0/3814619206 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8015030 con 0x55b50d0dcf10 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbbaa00640 1 -- 172.21.15.18:0/3814619206 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ffbc8018760 con 0x55b50d0dcf10 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbde483740 10 client.28594 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbde483740 10 client.28594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbde483740 20 client.28594 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbde483740 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b50d0e6410 0x55b50d15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbde483740 4 client.28594 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbde483740 1 -- 172.21.15.18:0/3814619206 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b50d0498b0 con 0x55b50d0e6410 2024-04-16T17:16:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.866+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b50d0e6410 0x55b50d15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbd3e00640 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b50d0e6410 0x55b50d15f0f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=984b1517f181c8b2 server_cookie=ffab6e7b211afa97 in_seq=0 out_seq=0 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbbaa00640 10 client.28594 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbbaa00640 1 -- 172.21.15.18:0/3814619206 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b50d0498b0 con 0x55b50d0e6410 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbbaa00640 10 client.28594 handle_command_reply: tid=0 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 client.28594 shutdown 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 -- 172.21.15.18:0/3814619206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dcf10 msgr2=0x55b50d0e0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dcf10 0x55b50d0e0730 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009250 tx=0x7ffbc8009b70 comp rx=0 tx=0).stop 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 -- 172.21.15.18:0/3814619206 shutdown_connections 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55b50d0e6410 0x55b50d15f0f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b50d0dd830 0x55b50d0e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b50d0dcf10 0x55b50d0e0730 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 --2- 172.21.15.18:0/3814619206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b50d0d7640 0x55b50d0e01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbbaa00640 0 client.28594 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:09.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.867+0000 7ffbde483740 1 -- 172.21.15.18:0/3814619206 >> 172.21.15.18:0/3814619206 conn(0x55b50d0f8ed0 msgr2=0x55b50d0d92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:09.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.868+0000 7ffbde483740 1 -- 172.21.15.18:0/3814619206 shutdown_connections 2024-04-16T17:16:09.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.868+0000 7ffbde483740 1 -- 172.21.15.18:0/3814619206 wait complete. 2024-04-16T17:16:09.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.868+0000 7ffbde483740 20 client.28594 trim_cache size 0 max 16384 2024-04-16T17:16:09.871 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.868+0000 7ffbda200640 1 -- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbbc0a3da0 msgr2=0x7ffbbc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.868+0000 7ffbda200640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbbc0a3da0 0x7ffbbc0a6260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0ce460 tx=0x7ffbcc005b40 comp rx=0 tx=0).stop 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.868+0000 7ffbda200640 1 -- 172.21.15.18:0/4268196206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd41a36c0 msgr2=0x7ffbd41a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.868+0000 7ffbda200640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd41a36c0 0x7ffbd41a3b40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00eb10 tx=0x7ffbcc00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.869+0000 7ffbda200640 1 -- 172.21.15.18:0/4268196206 shutdown_connections 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.869+0000 7ffbda200640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffbbc0a3da0 0x7ffbbc0a6260 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.869+0000 7ffbda200640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbd41a36c0 0x7ffbd41a3b40 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.869+0000 7ffbda200640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd4106030 0x7ffbd410d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.869+0000 7ffbda200640 1 --2- 172.21.15.18:0/4268196206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffbd4100f60 0x7ffbd410cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.869+0000 7ffbda200640 1 -- 172.21.15.18:0/4268196206 >> 172.21.15.18:0/4268196206 conn(0x7ffbd4077ab0 msgr2=0x7ffbd4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.869+0000 7ffbda200640 1 -- 172.21.15.18:0/4268196206 shutdown_connections 2024-04-16T17:16:09.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:09.869+0000 7ffbda200640 1 -- 172.21.15.18:0/4268196206 wait complete. 2024-04-16T17:16:09.892 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:16:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:09 smithi043 ceph-mon[36901]: pgmap v361: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 778 KiB/s wr, 114 op/s 2024-04-16T17:16:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:09 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:16:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:09 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:09 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:16:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:09 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:09 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:16:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:09 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/839345229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.149+0000 7faf39a00640 1 -- 172.21.15.18:0/4149032053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 msgr2=0x7faf341011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.149+0000 7faf39a00640 1 --2- 172.21.15.18:0/4149032053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 0x7faf341011b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009ca0 tx=0x7faf2c02c180 comp rx=0 tx=0).stop 2024-04-16T17:16:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.150+0000 7faf39a00640 1 -- 172.21.15.18:0/4149032053 shutdown_connections 2024-04-16T17:16:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.150+0000 7faf39a00640 1 --2- 172.21.15.18:0/4149032053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf341061d0 0x7faf34106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.150+0000 7faf39a00640 1 --2- 172.21.15.18:0/4149032053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf341016f0 0x7faf34105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.150+0000 7faf39a00640 1 --2- 172.21.15.18:0/4149032053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 0x7faf341011b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.150+0000 7faf39a00640 1 -- 172.21.15.18:0/4149032053 >> 172.21.15.18:0/4149032053 conn(0x7faf34077ab0 msgr2=0x7faf340fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.150+0000 7faf39a00640 1 -- 172.21.15.18:0/4149032053 shutdown_connections 2024-04-16T17:16:10.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.150+0000 7faf39a00640 1 -- 172.21.15.18:0/4149032053 wait complete. 2024-04-16T17:16:10.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.151+0000 7faf39a00640 1 Processor -- start 2024-04-16T17:16:10.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.151+0000 7faf39a00640 1 -- start start 2024-04-16T17:16:10.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf39a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 0x7faf341a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf39a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf341016f0 0x7faf341a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf39a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf341061d0 0x7faf3419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf39a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faf341187c0 con 0x7faf341016f0 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf39a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf34118640 con 0x7faf34100dd0 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf39a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf34118940 con 0x7faf341061d0 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 0x7faf341a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf33400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf341016f0 0x7faf341a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 0x7faf341a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56700/0 (socket says 172.21.15.18:56700) 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf33e00640 1 -- 172.21.15.18:0/3186175319 learned_addr learned my addr 172.21.15.18:0/3186175319 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.152+0000 7faf39000640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf341061d0 0x7faf3419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.153+0000 7faf33e00640 1 -- 172.21.15.18:0/3186175319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf341061d0 msgr2=0x7faf3419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.153+0000 7faf33e00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf341061d0 0x7faf3419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.153+0000 7faf33e00640 1 -- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf341016f0 msgr2=0x7faf341a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.153+0000 7faf33e00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf341016f0 0x7faf341a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.153+0000 7faf33e00640 1 -- 172.21.15.18:0/3186175319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3419f3e0 con 0x7faf34100dd0 2024-04-16T17:16:10.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.153+0000 7faf39000640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf341061d0 0x7faf3419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:10.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.153+0000 7faf33e00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 0x7faf341a4a60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7faf2c005e30 tx=0x7faf2c0346d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.153+0000 7faf30c00640 1 -- 172.21.15.18:0/3186175319 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c039050 con 0x7faf34100dd0 2024-04-16T17:16:10.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.154+0000 7faf39a00640 1 -- 172.21.15.18:0/3186175319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3419f670 con 0x7faf34100dd0 2024-04-16T17:16:10.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.154+0000 7faf30c00640 1 -- 172.21.15.18:0/3186175319 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf2c045070 con 0x7faf34100dd0 2024-04-16T17:16:10.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.154+0000 7faf30c00640 1 -- 172.21.15.18:0/3186175319 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c004440 con 0x7faf34100dd0 2024-04-16T17:16:10.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.154+0000 7faf39a00640 1 -- 172.21.15.18:0/3186175319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341ab800 con 0x7faf34100dd0 2024-04-16T17:16:10.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.157+0000 7faf3dc60740 1 Processor -- start 2024-04-16T17:16:10.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.158+0000 7faf3dc60740 1 -- start start 2024-04-16T17:16:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.158+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141aaf10 0x5587141ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.158+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141b4410 0x5587141b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf3dc60740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558714110020 con 0x5587141aaf10 2024-04-16T17:16:10.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf3dc60740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587140b6460 con 0x5587141b4410 2024-04-16T17:16:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf3dc60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55871410bfe0 con 0x5587141ab830 2024-04-16T17:16:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf39000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141aaf10 0x5587141ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf33400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141b4410 0x5587141b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf33400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141b4410 0x5587141b6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56706/0 (socket says 172.21.15.18:56706) 2024-04-16T17:16:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf33400640 1 -- 172.21.15.18:0/2111345655 learned_addr learned my addr 172.21.15.18:0/2111345655 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf30c00640 1 -- 172.21.15.18:0/3186175319 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faf2c0045e0 con 0x7faf34100dd0 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf33400640 1 -- 172.21.15.18:0/2111345655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 msgr2=0x5587141b3ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf33400640 1 --2- 172.21.15.18:0/2111345655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141b3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf33400640 1 -- 172.21.15.18:0/2111345655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141aaf10 msgr2=0x5587141ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.159+0000 7faf33400640 1 --2- 172.21.15.18:0/2111345655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141aaf10 0x5587141ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.160+0000 7faf33400640 1 -- 172.21.15.18:0/2111345655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587140be040 con 0x5587141b4410 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.160+0000 7faf33400640 1 --2- 172.21.15.18:0/2111345655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141b4410 0x5587141b6800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7faf20009c90 tx=0x7faf20011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601cc1bfff76aaaf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.160+0000 7faf29600640 1 -- 172.21.15.18:0/2111345655 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2001e070 con 0x5587141b4410 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.160+0000 7faf29600640 1 -- 172.21.15.18:0/2111345655 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf20019b10 con 0x5587141b4410 2024-04-16T17:16:10.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.160+0000 7faf29600640 1 -- 172.21.15.18:0/2111345655 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2000eac0 con 0x5587141b4410 2024-04-16T17:16:10.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf30c00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faf180a3d20 0x7faf180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf3dc60740 1 -- 172.21.15.18:0/2111345655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141b4410 msgr2=0x5587141b6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf3dc60740 1 --2- 172.21.15.18:0/2111345655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141b4410 0x5587141b6800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7faf20009c90 tx=0x7faf20011810 comp rx=0 tx=0).stop 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf30c00640 1 -- 172.21.15.18:0/3186175319 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7faf2c0f3190 con 0x7faf34100dd0 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf3dc60740 1 -- 172.21.15.18:0/2111345655 shutdown_connections 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf3dc60740 1 --2- 172.21.15.18:0/2111345655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141b4410 0x5587141b6800 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf3dc60740 1 --2- 172.21.15.18:0/2111345655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf3dc60740 1 --2- 172.21.15.18:0/2111345655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141aaf10 0x5587141ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf3dc60740 1 -- 172.21.15.18:0/2111345655 >> 172.21.15.18:0/2111345655 conn(0x5587141a5640 msgr2=0x5587141a5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.161+0000 7faf39000640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faf180a3d20 0x7faf180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.162+0000 7faf3dc60740 1 -- 172.21.15.18:0/2111345655 shutdown_connections 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.162+0000 7faf3dc60740 1 -- 172.21.15.18:0/2111345655 wait complete. 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.162+0000 7faf3dc60740 1 -- start start 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.162+0000 7faf39000640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faf180a3d20 0x7faf180a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faf24009c70 tx=0x7faf24005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.162+0000 7faf3dc60740 1 Processor -- start 2024-04-16T17:16:10.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf3dc60740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5587141b7200 con 0x5587141b4410 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf3dc60740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587140c9830 con 0x5587141aaf10 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf3dc60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587141b7380 con 0x5587141ab830 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf3dc60740 10 client.0 _ll_register_callbacks cb 0x5587140d1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.163+0000 7faf33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56718/0 (socket says 172.21.15.18:56718) 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf33e00640 1 -- 172.21.15.18:0/3006794567 learned_addr learned my addr 172.21.15.18:0/3006794567 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35470/0 (socket says 172.21.15.18:35470) 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf39000640 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf33e00640 1 -- 172.21.15.18:0/3006794567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 msgr2=0x5587141b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf33e00640 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf33e00640 1 -- 172.21.15.18:0/3006794567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 msgr2=0x5587141b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf33e00640 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.164+0000 7faf33e00640 1 -- 172.21.15.18:0/3006794567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587140cd920 con 0x5587141aaf10 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.165+0000 7faf33e00640 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b1500 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0f0a80 tx=0x7faf2c0be9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=784841e7be9e88b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.165+0000 7faf1e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.165+0000 7faf1e000640 1 -- 172.21.15.18:0/3006794567 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0c2070 con 0x5587141aaf10 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.165+0000 7faf1e000640 1 -- 172.21.15.18:0/3006794567 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7faf2c0f0bd0 con 0x5587141aaf10 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.165+0000 7faf3dc60740 1 -- 172.21.15.18:0/3006794567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf20019b10 con 0x5587141aaf10 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.165+0000 7faf1e000640 1 -- 172.21.15.18:0/3006794567 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faf2c0f0e50 con 0x5587141aaf10 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.165+0000 7faf3dc60740 10 client.28745 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.165+0000 7faf3dc60740 1 -- 172.21.15.18:0/3006794567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55871409b7f0 con 0x5587141aaf10 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf1e000640 1 -- 172.21.15.18:0/3006794567 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf20019b10 con 0x5587141aaf10 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf1e000640 1 -- 172.21.15.18:0/3006794567 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faf2c0d8080 con 0x5587141aaf10 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf3dc60740 10 client.28745 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf3dc60740 10 client.28745 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf3dc60740 20 client.28745 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:10.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf3dc60740 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5587141c4840 0x5587141c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf3dc60740 4 client.28745 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:10.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf3dc60740 1 -- 172.21.15.18:0/3006794567 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55871411da40 con 0x5587141c4840 2024-04-16T17:16:10.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.166+0000 7faf33400640 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5587141c4840 0x5587141c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.167+0000 7faf33400640 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5587141c4840 0x5587141c6c30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb0e038b9de12c11 server_cookie=7297d0cfcfbe89ea in_seq=0 out_seq=0 2024-04-16T17:16:10.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.167+0000 7faf1e000640 10 client.28745 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:10.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.168+0000 7faf1e000640 1 -- 172.21.15.18:0/3006794567 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55871411da40 con 0x5587141c4840 2024-04-16T17:16:10.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.168+0000 7faf1e000640 10 client.28745 handle_command_reply: tid=0 2024-04-16T17:16:10.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.168+0000 7faf3dc60740 1 client.28745 shutdown 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.168+0000 7faf3dc60740 1 -- 172.21.15.18:0/3006794567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 msgr2=0x5587141b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.168+0000 7faf3dc60740 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b1500 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0f0a80 tx=0x7faf2c0be9e0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 1 -- 172.21.15.18:0/3006794567 shutdown_connections 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5587141c4840 0x5587141c6c30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf1e000640 0 client.28745 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 1 --2- 172.21.15.18:0/3006794567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b1500 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 1 -- 172.21.15.18:0/3006794567 >> 172.21.15.18:0/3006794567 conn(0x5587141a5640 msgr2=0x5587141a7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 1 -- 172.21.15.18:0/3006794567 shutdown_connections 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 1 -- 172.21.15.18:0/3006794567 wait complete. 2024-04-16T17:16:10.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.169+0000 7faf3dc60740 20 client.28745 trim_cache size 0 max 16384 2024-04-16T17:16:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:09 smithi022 ceph-mon[36023]: pgmap v361: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 778 KiB/s wr, 114 op/s 2024-04-16T17:16:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:09 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:16:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:09 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:09 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:16:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:09 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:09 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:16:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:09 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/839345229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:10.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.195+0000 7faf3dc60740 1 Processor -- start 2024-04-16T17:16:10.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.195+0000 7faf3dc60740 1 -- start start 2024-04-16T17:16:10.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf3dc60740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b84b0 con 0x5587141b4410 2024-04-16T17:16:10.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf3dc60740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b5d80 con 0x5587141aaf10 2024-04-16T17:16:10.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf3dc60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b3bf0 con 0x5587141ab830 2024-04-16T17:16:10.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56722/0 (socket says 172.21.15.18:56722) 2024-04-16T17:16:10.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf33e00640 1 -- 172.21.15.18:0/511596296 learned_addr learned my addr 172.21.15.18:0/511596296 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf33400640 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.196+0000 7faf39000640 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf33e00640 1 -- 172.21.15.18:0/511596296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 msgr2=0x5587141c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf33e00640 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf33e00640 1 -- 172.21.15.18:0/511596296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 msgr2=0x5587141b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf33e00640 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf33e00640 1 -- 172.21.15.18:0/511596296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558713e223f0 con 0x5587141aaf10 2024-04-16T17:16:10.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf33e00640 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b3c30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0f0640 tx=0x7faf2c0bea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6a6f950ddfdba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf28c00640 1 -- 172.21.15.18:0/511596296 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0c3190 con 0x5587141aaf10 2024-04-16T17:16:10.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf28c00640 1 -- 172.21.15.18:0/511596296 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf2c03a040 con 0x5587141aaf10 2024-04-16T17:16:10.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.197+0000 7faf28c00640 1 -- 172.21.15.18:0/511596296 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0be4b0 con 0x5587141aaf10 2024-04-16T17:16:10.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.198+0000 7faf3dc60740 1 -- 172.21.15.18:0/511596296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 msgr2=0x5587141b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.198+0000 7faf3dc60740 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b3c30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0f0640 tx=0x7faf2c0bea50 comp rx=0 tx=0).stop 2024-04-16T17:16:10.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.198+0000 7faf3dc60740 1 -- 172.21.15.18:0/511596296 shutdown_connections 2024-04-16T17:16:10.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.198+0000 7faf3dc60740 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.198+0000 7faf3dc60740 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.198+0000 7faf3dc60740 1 --2- 172.21.15.18:0/511596296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141b3c30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.198+0000 7faf3dc60740 1 -- 172.21.15.18:0/511596296 >> 172.21.15.18:0/511596296 conn(0x5587141a5640 msgr2=0x5587141afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.199+0000 7faf3dc60740 1 -- 172.21.15.18:0/511596296 shutdown_connections 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.199+0000 7faf3dc60740 1 -- 172.21.15.18:0/511596296 wait complete. 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.199+0000 7faf3dc60740 1 -- start start 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.199+0000 7faf3dc60740 1 Processor -- start 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141bcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf3dc60740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf3dc60740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b4240 con 0x5587141b4410 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf3dc60740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b56b0 con 0x5587141aaf10 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf3dc60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b4910 con 0x5587141ab830 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf3dc60740 10 client.0 _ll_register_callbacks cb 0x7faf2c0bb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141bcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141bcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56730/0 (socket says 172.21.15.18:56730) 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf33e00640 1 -- 172.21.15.18:0/661389341 learned_addr learned my addr 172.21.15.18:0/661389341 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf39000640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.200+0000 7faf33400640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf33400640 1 -- 172.21.15.18:0/661389341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 msgr2=0x5587141bcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf33400640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141bcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf33400640 1 -- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 msgr2=0x5587141af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf33400640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf33400640 1 -- 172.21.15.18:0/661389341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558713eec7d0 con 0x5587141ab830 2024-04-16T17:16:10.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf33e00640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141bcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:10.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf39000640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:10.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf33400640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141aea80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faf20011d10 tx=0x7faf2000e640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0e9aa7815ca6787 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.201+0000 7faf1e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:10.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.202+0000 7faf1e000640 1 -- 172.21.15.18:0/661389341 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2001e040 con 0x5587141ab830 2024-04-16T17:16:10.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.202+0000 7faf1e000640 1 -- 172.21.15.18:0/661389341 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7faf200052c0 con 0x5587141ab830 2024-04-16T17:16:10.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.202+0000 7faf3dc60740 1 -- 172.21.15.18:0/661389341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf2c03a040 con 0x5587141ab830 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.202+0000 7faf1e000640 1 -- 172.21.15.18:0/661389341 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faf200054c0 con 0x5587141ab830 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.202+0000 7faf3dc60740 10 client.28609 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.202+0000 7faf3dc60740 1 -- 172.21.15.18:0/661389341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558714229fc0 con 0x5587141ab830 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.202+0000 7faf1e000640 1 -- 172.21.15.18:0/661389341 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf2c03a040 con 0x5587141ab830 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.203+0000 7faf1e000640 1 -- 172.21.15.18:0/661389341 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faf2000ebe0 con 0x5587141ab830 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.203+0000 7faf3dc60740 10 client.28609 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.203+0000 7faf3dc60740 10 client.28609 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.203+0000 7faf3dc60740 20 client.28609 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.203+0000 7faf3dc60740 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5587141c4840 0x55871422c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.203+0000 7faf3dc60740 4 client.28609 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.203+0000 7faf3dc60740 1 -- 172.21.15.18:0/661389341 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587141178b0 con 0x5587141c4840 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.203+0000 7faf33e00640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5587141c4840 0x55871422c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf33e00640 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5587141c4840 0x55871422c570 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4080d9891491455d server_cookie=700d3cd4a9f60e1b in_seq=0 out_seq=0 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf1e000640 10 client.28609 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf1e000640 1 -- 172.21.15.18:0/661389341 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587141178b0 con 0x5587141c4840 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf1e000640 10 client.28609 handle_command_reply: tid=0 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf3dc60740 1 client.28609 shutdown 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf3dc60740 1 -- 172.21.15.18:0/661389341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 msgr2=0x5587141aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf3dc60740 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141aea80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faf20011d10 tx=0x7faf2000e640 comp rx=0 tx=0).stop 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf3dc60740 1 -- 172.21.15.18:0/661389341 shutdown_connections 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf3dc60740 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5587141c4840 0x55871422c570 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf3dc60740 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587141b4410 0x5587141af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.204+0000 7faf3dc60740 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587141ab830 0x5587141aea80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.205+0000 7faf3dc60740 1 --2- 172.21.15.18:0/661389341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587141aaf10 0x5587141bcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.205+0000 7faf1e000640 0 client.28609 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.205+0000 7faf3dc60740 1 -- 172.21.15.18:0/661389341 >> 172.21.15.18:0/661389341 conn(0x5587141a5640 msgr2=0x5587141a6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.205+0000 7faf3dc60740 1 -- 172.21.15.18:0/661389341 shutdown_connections 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.205+0000 7faf3dc60740 1 -- 172.21.15.18:0/661389341 wait complete. 2024-04-16T17:16:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.205+0000 7faf3dc60740 20 client.28609 trim_cache size 0 max 16384 2024-04-16T17:16:10.208 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:10.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 -- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faf180a3d20 msgr2=0x7faf180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faf180a3d20 0x7faf180a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faf24009c70 tx=0x7faf24005f90 comp rx=0 tx=0).stop 2024-04-16T17:16:10.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 -- 172.21.15.18:0/3186175319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 msgr2=0x7faf341a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 0x7faf341a4a60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7faf2c005e30 tx=0x7faf2c0346d0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 -- 172.21.15.18:0/3186175319 shutdown_connections 2024-04-16T17:16:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faf180a3d20 0x7faf180a61e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf341061d0 0x7faf3419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf341016f0 0x7faf341a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 --2- 172.21.15.18:0/3186175319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf34100dd0 0x7faf341a4a60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 -- 172.21.15.18:0/3186175319 >> 172.21.15.18:0/3186175319 conn(0x7faf34077ab0 msgr2=0x7faf34103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 -- 172.21.15.18:0/3186175319 shutdown_connections 2024-04-16T17:16:10.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.206+0000 7faf39a00640 1 -- 172.21.15.18:0/3186175319 wait complete. 2024-04-16T17:16:10.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:16:10.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 -- 172.21.15.18:0/183334945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafe00743b0 msgr2=0x7fafe0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 --2- 172.21.15.18:0/183334945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafe00743b0 0x7fafe0074790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009900 tx=0x7fafd002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 -- 172.21.15.18:0/183334945 shutdown_connections 2024-04-16T17:16:10.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 --2- 172.21.15.18:0/183334945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafe010e6c0 0x7fafe0110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 --2- 172.21.15.18:0/183334945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafe0074cd0 0x7fafe010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 --2- 172.21.15.18:0/183334945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafe00743b0 0x7fafe0074790 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 -- 172.21.15.18:0/183334945 >> 172.21.15.18:0/183334945 conn(0x7fafe00fd950 msgr2=0x7fafe00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 -- 172.21.15.18:0/183334945 shutdown_connections 2024-04-16T17:16:10.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.427+0000 7fafe5600640 1 -- 172.21.15.18:0/183334945 wait complete. 2024-04-16T17:16:10.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.428+0000 7fafe5600640 1 Processor -- start 2024-04-16T17:16:10.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.428+0000 7fafe5600640 1 -- start start 2024-04-16T17:16:10.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafe5600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafe00743b0 0x7fafe01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafe5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafe0074cd0 0x7fafe01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafe5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafe010e6c0 0x7fafe019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafe5600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0114410 con 0x7fafe00743b0 2024-04-16T17:16:10.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafe5600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0114290 con 0x7fafe0074cd0 2024-04-16T17:16:10.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafe5600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0114590 con 0x7fafe010e6c0 2024-04-16T17:16:10.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdfe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafe00743b0 0x7fafe01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdfe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafe00743b0 0x7fafe01a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40086/0 (socket says 172.21.15.18:40086) 2024-04-16T17:16:10.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdfe00640 1 -- 172.21.15.18:0/3549150012 learned_addr learned my addr 172.21.15.18:0/3549150012 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafe4c00640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafe010e6c0 0x7fafe019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdf400640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafe0074cd0 0x7fafe01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdfe00640 1 -- 172.21.15.18:0/3549150012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafe010e6c0 msgr2=0x7fafe019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdfe00640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafe010e6c0 0x7fafe019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdfe00640 1 -- 172.21.15.18:0/3549150012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafe0074cd0 msgr2=0x7fafe01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdfe00640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafe0074cd0 0x7fafe01a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.429+0000 7fafdfe00640 1 -- 172.21.15.18:0/3549150012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe019aff0 con 0x7fafe00743b0 2024-04-16T17:16:10.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.430+0000 7fafdfe00640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafe00743b0 0x7fafe01a0670 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fafd0000c00 tx=0x7fafd00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.430+0000 7fafdcc00640 1 -- 172.21.15.18:0/3549150012 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0042020 con 0x7fafe00743b0 2024-04-16T17:16:10.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.430+0000 7fafdcc00640 1 -- 172.21.15.18:0/3549150012 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd0005000 con 0x7fafe00743b0 2024-04-16T17:16:10.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.430+0000 7fafdcc00640 1 -- 172.21.15.18:0/3549150012 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0003b00 con 0x7fafe00743b0 2024-04-16T17:16:10.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.430+0000 7fafe5600640 1 -- 172.21.15.18:0/3549150012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe019b280 con 0x7fafe00743b0 2024-04-16T17:16:10.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.430+0000 7fafe5600640 1 -- 172.21.15.18:0/3549150012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01a75f0 con 0x7fafe00743b0 2024-04-16T17:16:10.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.433+0000 7fafe97e1740 1 Processor -- start 2024-04-16T17:16:10.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe97e1740 1 -- start start 2024-04-16T17:16:10.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eb830 0x5565e06f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06f4410 0x5565e06f6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe97e1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5565e0650020 con 0x5565e06eaf10 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe97e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5565e05f6460 con 0x5565e06eb830 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe97e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565e064bfe0 con 0x5565e06f4410 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafdcc00640 1 -- 172.21.15.18:0/3549150012 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fafd0045020 con 0x7fafe00743b0 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06eb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40092/0 (socket says 172.21.15.18:40092) 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.434+0000 7fafe4c00640 1 -- 172.21.15.18:0/3194602926 learned_addr learned my addr 172.21.15.18:0/3194602926 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.435+0000 7fafdfe00640 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eb830 0x5565e06f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.435+0000 7fafdf400640 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06f4410 0x5565e06f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.435+0000 7fafe4c00640 1 -- 172.21.15.18:0/3194602926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06f4410 msgr2=0x5565e06f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.435+0000 7fafe4c00640 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06f4410 0x5565e06f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.435+0000 7fafe4c00640 1 -- 172.21.15.18:0/3194602926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eb830 msgr2=0x5565e06f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.435+0000 7fafe4c00640 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eb830 0x5565e06f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.435+0000 7fafe4c00640 1 -- 172.21.15.18:0/3194602926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e05fe040 con 0x5565e06eaf10 2024-04-16T17:16:10.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.435+0000 7fafe4c00640 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06eb2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009f50 tx=0x7fafd80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e7a57314a21086 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.436+0000 7fafd5200640 1 -- 172.21.15.18:0/3194602926 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8004d20 con 0x5565e06eaf10 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.436+0000 7fafd5200640 1 -- 172.21.15.18:0/3194602926 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd8004050 con 0x5565e06eaf10 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.436+0000 7fafd5200640 1 -- 172.21.15.18:0/3194602926 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8005280 con 0x5565e06eaf10 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.436+0000 7fafdcc00640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fafc40a3d20 0x7fafc40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.436+0000 7fafdcc00640 1 -- 172.21.15.18:0/3549150012 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fafd00f28f0 con 0x7fafe00743b0 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafdfe00640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fafc40a3d20 0x7fafc40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 -- 172.21.15.18:0/3194602926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 msgr2=0x5565e06eb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06eb2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009f50 tx=0x7fafd80118d0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 -- 172.21.15.18:0/3194602926 shutdown_connections 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06f4410 0x5565e06f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eb830 0x5565e06f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 --2- 172.21.15.18:0/3194602926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06eb2f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 -- 172.21.15.18:0/3194602926 >> 172.21.15.18:0/3194602926 conn(0x5565e06e5640 msgr2=0x5565e06e5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 -- 172.21.15.18:0/3194602926 shutdown_connections 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafe97e1740 1 -- 172.21.15.18:0/3194602926 wait complete. 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.437+0000 7fafdfe00640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fafc40a3d20 0x7fafc40a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5565e06ea9e0 tx=0x7fafd00f8530 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 1 -- start start 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 1 Processor -- start 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eaf10 0x5565e06f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06f4410 0x5565e06f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5565e06f7200 con 0x5565e06f4410 2024-04-16T17:16:10.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5565e0609830 con 0x5565e06eaf10 2024-04-16T17:16:10.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565e06f7380 con 0x5565e06eb830 2024-04-16T17:16:10.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.438+0000 7fafe97e1740 10 client.0 _ll_register_callbacks cb 0x5565e0611150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:10.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eaf10 0x5565e06f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eaf10 0x5565e06f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56758/0 (socket says 172.21.15.18:56758) 2024-04-16T17:16:10.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafe4c00640 1 -- 172.21.15.18:0/1303418176 learned_addr learned my addr 172.21.15.18:0/1303418176 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafdf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafe4c00640 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06f4410 0x5565e06f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafdf400640 1 -- 172.21.15.18:0/1303418176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eaf10 msgr2=0x5565e06f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafdf400640 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eaf10 0x5565e06f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafdf400640 1 -- 172.21.15.18:0/1303418176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06f4410 msgr2=0x5565e06f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafdf400640 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06f4410 0x5565e06f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafdf400640 1 -- 172.21.15.18:0/1303418176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e060d920 con 0x5565e06eb830 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.439+0000 7fafe4c00640 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eaf10 0x5565e06f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.440+0000 7fafdf400640 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06f1970 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fafe019bf80 tx=0x7fafcc012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56a722f71335dfe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.440+0000 7fafc9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.440+0000 7fafc9600640 1 -- 172.21.15.18:0/1303418176 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc01f060 con 0x5565e06eb830 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.440+0000 7fafe97e1740 1 -- 172.21.15.18:0/1303418176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd8004050 con 0x5565e06eb830 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.440+0000 7fafc9600640 1 -- 172.21.15.18:0/1303418176 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fafcc010930 con 0x5565e06eb830 2024-04-16T17:16:10.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.440+0000 7fafc9600640 1 -- 172.21.15.18:0/1303418176 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fafcc026050 con 0x5565e06eb830 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.440+0000 7fafe97e1740 10 client.28621 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.440+0000 7fafe97e1740 1 -- 172.21.15.18:0/1303418176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5565e05db7f0 con 0x5565e06eb830 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafc9600640 1 -- 172.21.15.18:0/1303418176 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd8004050 con 0x5565e06eb830 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafc9600640 1 -- 172.21.15.18:0/1303418176 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fafcc019030 con 0x5565e06eb830 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafe97e1740 10 client.28621 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafe97e1740 10 client.28621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafe97e1740 20 client.28621 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafe97e1740 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5565e0704820 0x5565e0706c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafe97e1740 4 client.28621 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafe97e1740 1 -- 172.21.15.18:0/1303418176 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e065da40 con 0x5565e0704820 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.441+0000 7fafe4c00640 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5565e0704820 0x5565e0706c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.442+0000 7fafe4c00640 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5565e0704820 0x5565e0706c40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13929107a8951865 server_cookie=51535bdaaa1ab6ad in_seq=0 out_seq=0 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.442+0000 7fafc9600640 10 client.28621 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.442+0000 7fafc9600640 1 -- 172.21.15.18:0/1303418176 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565e065da40 con 0x5565e0704820 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.442+0000 7fafc9600640 10 client.28621 handle_command_reply: tid=0 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.442+0000 7fafe97e1740 1 client.28621 shutdown 2024-04-16T17:16:10.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 -- 172.21.15.18:0/1303418176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 msgr2=0x5565e06f1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06f1970 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fafe019bf80 tx=0x7fafcc012820 comp rx=0 tx=0).stop 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 -- 172.21.15.18:0/1303418176 shutdown_connections 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5565e0704820 0x5565e0706c40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06f4410 0x5565e06f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06f1970 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafc9600640 0 client.28621 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 --2- 172.21.15.18:0/1303418176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06eaf10 0x5565e06f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 -- 172.21.15.18:0/1303418176 >> 172.21.15.18:0/1303418176 conn(0x5565e06e5640 msgr2=0x5565e06e7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 -- 172.21.15.18:0/1303418176 shutdown_connections 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 1 -- 172.21.15.18:0/1303418176 wait complete. 2024-04-16T17:16:10.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.443+0000 7fafe97e1740 20 client.28621 trim_cache size 0 max 16384 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.458+0000 7fafe97e1740 1 Processor -- start 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.458+0000 7fafe97e1740 1 -- start start 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.458+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eaf10 0x5565e06f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.458+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eb830 0x5565e0700120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.458+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06e07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.458+0000 7fafe97e1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01b4160 con 0x5565e06eb830 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.458+0000 7fafe97e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01b1a30 con 0x5565e06f4410 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.458+0000 7fafe97e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01af8a0 con 0x5565e06eaf10 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06e07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafdf400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eb830 0x5565e0700120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06e07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56772/0 (socket says 172.21.15.18:56772) 2024-04-16T17:16:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafe4c00640 1 -- 172.21.15.18:0/543029932 learned_addr learned my addr 172.21.15.18:0/543029932 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafe4c00640 1 --2- 172.21.15.18:0/543029932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eaf10 0x5565e06f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafdf400640 1 -- 172.21.15.18:0/543029932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eaf10 msgr2=0x5565e06f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafdf400640 1 --2- 172.21.15.18:0/543029932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eaf10 0x5565e06f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafdf400640 1 -- 172.21.15.18:0/543029932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 msgr2=0x5565e06e07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafdf400640 1 --2- 172.21.15.18:0/543029932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06e07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.459+0000 7fafdf400640 1 -- 172.21.15.18:0/543029932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e06f03e0 con 0x5565e06eb830 2024-04-16T17:16:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.460+0000 7fafdf400640 1 --2- 172.21.15.18:0/543029932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eb830 0x5565e0700120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00fe50 tx=0x7fafcc005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839fcd51d244609d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.460+0000 7fafcbe00640 1 -- 172.21.15.18:0/543029932 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc005850 con 0x5565e06eb830 2024-04-16T17:16:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.460+0000 7fafcbe00640 1 -- 172.21.15.18:0/543029932 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc02f070 con 0x5565e06eb830 2024-04-16T17:16:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.460+0000 7fafcbe00640 1 -- 172.21.15.18:0/543029932 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc010730 con 0x5565e06eb830 2024-04-16T17:16:10.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 -- 172.21.15.18:0/543029932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eb830 msgr2=0x5565e0700120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 --2- 172.21.15.18:0/543029932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eb830 0x5565e0700120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00fe50 tx=0x7fafcc005710 comp rx=0 tx=0).stop 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 -- 172.21.15.18:0/543029932 shutdown_connections 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 --2- 172.21.15.18:0/543029932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 --2- 172.21.15.18:0/543029932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eb830 0x5565e0700120 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 --2- 172.21.15.18:0/543029932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eaf10 0x5565e06f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 -- 172.21.15.18:0/543029932 >> 172.21.15.18:0/543029932 conn(0x5565e06e5640 msgr2=0x5565e06e4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 -- 172.21.15.18:0/543029932 shutdown_connections 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 -- 172.21.15.18:0/543029932 wait complete. 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.461+0000 7fafe97e1740 1 -- start start 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe97e1740 1 Processor -- start 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06ee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06ee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe97e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe97e1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01afef0 con 0x5565e06eaf10 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe97e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01b1360 con 0x5565e06f4410 2024-04-16T17:16:10.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe97e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01b05c0 con 0x5565e06eb830 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe97e1740 10 client.0 _ll_register_callbacks cb 0x7fafcc00ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06ee300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40110/0 (socket says 172.21.15.18:40110) 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafdf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe4c00640 1 -- 172.21.15.18:0/2570054861 learned_addr learned my addr 172.21.15.18:0/2570054861 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.462+0000 7fafe4c00640 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafdf400640 1 -- 172.21.15.18:0/2570054861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 msgr2=0x5565e06eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafdf400640 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafdf400640 1 -- 172.21.15.18:0/2570054861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 msgr2=0x5565e06ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafdf400640 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafdf400640 1 -- 172.21.15.18:0/2570054861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e03623f0 con 0x5565e06eb830 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafe4c00640 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafdf400640 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06ee840 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x5565e06f26b0 tx=0x7fafcc005c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ec1509f50245466 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafc9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:10.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafc9600640 1 -- 172.21.15.18:0/2570054861 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc037020 con 0x5565e06eb830 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafe97e1740 1 -- 172.21.15.18:0/2570054861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafcc02f070 con 0x5565e06eb830 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.463+0000 7fafc9600640 1 -- 172.21.15.18:0/2570054861 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fafcc005040 con 0x5565e06eb830 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafc9600640 1 -- 172.21.15.18:0/2570054861 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fafcc01f070 con 0x5565e06eb830 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe97e1740 10 client.28627 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe97e1740 1 -- 172.21.15.18:0/2570054861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5565e042c7d0 con 0x5565e06eb830 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafc9600640 1 -- 172.21.15.18:0/2570054861 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc02f070 con 0x5565e06eb830 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafc9600640 1 -- 172.21.15.18:0/2570054861 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fafcc019040 con 0x5565e06eb830 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe97e1740 10 client.28627 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe97e1740 10 client.28627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe97e1740 20 client.28627 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe97e1740 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5565e076b540 0x5565e076d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe97e1740 4 client.28627 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe97e1740 1 -- 172.21.15.18:0/2570054861 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565e06578b0 con 0x5565e076b540 2024-04-16T17:16:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.464+0000 7fafe4c00640 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5565e076b540 0x5565e076d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe4c00640 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5565e076b540 0x5565e076d960 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d8a6626bc7de7b server_cookie=4973bbcaf48e7b9f in_seq=0 out_seq=0 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafc9600640 10 client.28627 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafc9600640 1 -- 172.21.15.18:0/2570054861 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565e06578b0 con 0x5565e076b540 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafc9600640 10 client.28627 handle_command_reply: tid=0 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 client.28627 shutdown 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 -- 172.21.15.18:0/2570054861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 msgr2=0x5565e06ee840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06ee840 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x5565e06f26b0 tx=0x7fafcc005c10 comp rx=0 tx=0).stop 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 -- 172.21.15.18:0/2570054861 shutdown_connections 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5565e076b540 0x5565e076d960 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565e06f4410 0x5565e06eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e06eb830 0x5565e06ee840 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 --2- 172.21.15.18:0/2570054861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565e06eaf10 0x5565e06ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafc9600640 0 client.28627 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 -- 172.21.15.18:0/2570054861 >> 172.21.15.18:0/2570054861 conn(0x5565e06e5640 msgr2=0x5565e06e3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 -- 172.21.15.18:0/2570054861 shutdown_connections 2024-04-16T17:16:10.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.465+0000 7fafe97e1740 1 -- 172.21.15.18:0/2570054861 wait complete. 2024-04-16T17:16:10.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.466+0000 7fafe97e1740 20 client.28627 trim_cache size 0 max 16384 2024-04-16T17:16:10.470 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:10.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.466+0000 7fafe5600640 1 -- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fafc40a3d20 msgr2=0x7fafc40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.466+0000 7fafe5600640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fafc40a3d20 0x7fafc40a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5565e06ea9e0 tx=0x7fafd00f8530 comp rx=0 tx=0).stop 2024-04-16T17:16:10.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.466+0000 7fafe5600640 1 -- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafe00743b0 msgr2=0x7fafe01a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:10.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.466+0000 7fafe5600640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafe00743b0 0x7fafe01a0670 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fafd0000c00 tx=0x7fafd00043c0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.467+0000 7fafe5600640 1 -- 172.21.15.18:0/3549150012 shutdown_connections 2024-04-16T17:16:10.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.467+0000 7fafe5600640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fafc40a3d20 0x7fafc40a61e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.467+0000 7fafe5600640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafe010e6c0 0x7fafe019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.467+0000 7fafe5600640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafe0074cd0 0x7fafe01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.467+0000 7fafe5600640 1 --2- 172.21.15.18:0/3549150012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fafe00743b0 0x7fafe01a0670 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:10.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.467+0000 7fafe5600640 1 -- 172.21.15.18:0/3549150012 >> 172.21.15.18:0/3549150012 conn(0x7fafe00fd950 msgr2=0x7fafe00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:10.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.467+0000 7fafe5600640 1 -- 172.21.15.18:0/3549150012 shutdown_connections 2024-04-16T17:16:10.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:10.467+0000 7fafe5600640 1 -- 172.21.15.18:0/3549150012 wait complete. 2024-04-16T17:16:10.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 429365 3.61 MB/sec execute 503 sec latency 18.723 ms 2024-04-16T17:16:11.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:09 smithi018 ceph-mon[31776]: pgmap v361: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 778 KiB/s wr, 114 op/s 2024-04-16T17:16:11.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:09 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:16:11.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:09 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:11.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:09 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:16:11.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:09 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:11.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:09 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:16:11.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:09 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/839345229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:11.483 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:16:11.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:16:11.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:16:11.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 429745 3.60 MB/sec execute 504 sec latency 34.927 ms 2024-04-16T17:16:11.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 -- 172.21.15.18:0/3293450249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc102240 msgr2=0x7f0adc102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/3293450249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc102240 0x7f0adc102620 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009900 tx=0x7f0acc02bc50 comp rx=0 tx=0).stop 2024-04-16T17:16:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 -- 172.21.15.18:0/3293450249 shutdown_connections 2024-04-16T17:16:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/3293450249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0adc107310 0x7f0adc10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/3293450249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0adc102bf0 0x7f0adc106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/3293450249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc102240 0x7f0adc102620 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 -- 172.21.15.18:0/3293450249 >> 172.21.15.18:0/3293450249 conn(0x7f0adc0fdac0 msgr2=0x7f0adc0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 -- 172.21.15.18:0/3293450249 shutdown_connections 2024-04-16T17:16:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.694+0000 7f0ae0e00640 1 -- 172.21.15.18:0/3293450249 wait complete. 2024-04-16T17:16:11.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.696+0000 7f0ae0e00640 1 Processor -- start 2024-04-16T17:16:11.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.696+0000 7f0ae0e00640 1 -- start start 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.696+0000 7f0ae0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0adc102240 0x7f0adc1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ae0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc107310 0x7f0adc1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ae0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0adc19a9f0 0x7f0adc19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ae0e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc112470 con 0x7f0adc107310 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ae0e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1122f0 con 0x7f0adc19a9f0 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ae0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1125f0 con 0x7f0adc102240 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ad9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc107310 0x7f0adc1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ada800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0adc19a9f0 0x7f0adc19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0adc102240 0x7f0adc1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ada800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0adc19a9f0 0x7f0adc19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56816/0 (socket says 172.21.15.18:56816) 2024-04-16T17:16:11.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ad9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc107310 0x7f0adc1a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40130/0 (socket says 172.21.15.18:40130) 2024-04-16T17:16:11.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ada800640 1 -- 172.21.15.18:0/466645764 learned_addr learned my addr 172.21.15.18:0/466645764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.697+0000 7f0ad9400640 1 -- 172.21.15.18:0/466645764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0adc102240 msgr2=0x7f0adc1a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad9400640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0adc102240 0x7f0adc1a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad9400640 1 -- 172.21.15.18:0/466645764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0adc19a9f0 msgr2=0x7f0adc19aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad9400640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0adc19a9f0 0x7f0adc19aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad9400640 1 -- 172.21.15.18:0/466645764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc19b510 con 0x7f0adc107310 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ada800640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0adc19a9f0 0x7f0adc19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad9400640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc107310 0x7f0adc1a0d30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009fd0 tx=0x7f0ac800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad2a00640 1 -- 172.21.15.18:0/466645764 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8015020 con 0x7f0adc107310 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad2a00640 1 -- 172.21.15.18:0/466645764 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ac8011d10 con 0x7f0adc107310 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad2a00640 1 -- 172.21.15.18:0/466645764 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac800f2e0 con 0x7f0adc107310 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae0e00640 1 -- 172.21.15.18:0/466645764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc19b800 con 0x7f0adc107310 2024-04-16T17:16:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae0e00640 1 -- 172.21.15.18:0/466645764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc1a79b0 con 0x7f0adc107310 2024-04-16T17:16:11.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae3a4b740 1 Processor -- start 2024-04-16T17:16:11.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae3a4b740 1 -- start start 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f855ca0 0x559a3f856080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f8565c0 0x559a3f85ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f861590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559a3f7bacb0 con 0x559a3f85f1a0 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559a3f761370 con 0x559a3f8565c0 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a3f7b6c70 con 0x559a3f855ca0 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.698+0000 7f0ad2a00640 1 -- 172.21.15.18:0/466645764 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ac8018020 con 0x7f0adc107310 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f8565c0 0x559a3f85ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ada800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f855ca0 0x559a3f856080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f8565c0 0x559a3f85ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56820/0 (socket says 172.21.15.18:56820) 2024-04-16T17:16:11.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 -- 172.21.15.18:0/588910870 learned_addr learned my addr 172.21.15.18:0/588910870 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:11.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9400640 1 --2- 172.21.15.18:0/588910870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f861590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 -- 172.21.15.18:0/588910870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f855ca0 msgr2=0x559a3f856080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:11.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/588910870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f855ca0 0x559a3f856080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 -- 172.21.15.18:0/588910870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 msgr2=0x559a3f861590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:11.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/588910870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f861590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 -- 172.21.15.18:0/588910870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3f780910 con 0x559a3f8565c0 2024-04-16T17:16:11.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/588910870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f8565c0 0x559a3f85ec60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009230 tx=0x7f0acc005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54be6a15ef5549f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:11.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ac7400640 1 -- 172.21.15.18:0/588910870 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc042070 con 0x559a3f8565c0 2024-04-16T17:16:11.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ac7400640 1 -- 172.21.15.18:0/588910870 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0acc004240 con 0x559a3f8565c0 2024-04-16T17:16:11.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ac7400640 1 -- 172.21.15.18:0/588910870 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0051c0 con 0x559a3f8565c0 2024-04-16T17:16:11.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad2a00640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0ac00a3e20 0x7f0ac00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ad2a00640 1 -- 172.21.15.18:0/466645764 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f0ac80d0310 con 0x7f0adc107310 2024-04-16T17:16:11.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ada800640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0ac00a3e20 0x7f0ac00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/588910870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f8565c0 msgr2=0x559a3f85ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/588910870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f8565c0 0x559a3f85ec60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009230 tx=0x7f0acc005e80 comp rx=0 tx=0).stop 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/588910870 shutdown_connections 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/588910870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f861590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/588910870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f8565c0 0x559a3f85ec60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/588910870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f855ca0 0x559a3f856080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/588910870 >> 172.21.15.18:0/588910870 conn(0x559a3f8503d0 msgr2=0x559a3f8507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ada800640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0ac00a3e20 0x7f0ac00a62e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009270 tx=0x7f0ad4009680 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.703+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/588910870 shutdown_connections 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/588910870 wait complete. 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 -- start start 2024-04-16T17:16:11.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 Processor -- start 2024-04-16T17:16:11.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f85c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f85cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559a3f861f30 con 0x559a3f85f1a0 2024-04-16T17:16:11.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559a3f774730 con 0x559a3f855ca0 2024-04-16T17:16:11.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a3f8620b0 con 0x559a3f8565c0 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.707+0000 7f0ae3a4b740 10 client.0 _ll_register_callbacks cb 0x559a3f7431e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f85c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f85c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56832/0 (socket says 172.21.15.18:56832) 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9e00640 1 -- 172.21.15.18:0/1820289057 learned_addr learned my addr 172.21.15.18:0/1820289057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ada800640 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f85cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9400640 1 -- 172.21.15.18:0/1820289057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 msgr2=0x559a3f85c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9400640 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f85c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9400640 1 -- 172.21.15.18:0/1820289057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 msgr2=0x559a3f85cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9400640 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f85cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9400640 1 -- 172.21.15.18:0/1820289057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3f5957e0 con 0x559a3f8565c0 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ada800640 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f85cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f85c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:11.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.708+0000 7f0ad9400640 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85c720 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80cd6b0 tx=0x7f0ac8010f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=590faac0c2e980be server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:11.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.709+0000 7f0abbe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:11.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.709+0000 7f0abbe00640 1 -- 172.21.15.18:0/1820289057 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac809ba40 con 0x559a3f8565c0 2024-04-16T17:16:11.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.709+0000 7f0abbe00640 1 -- 172.21.15.18:0/1820289057 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f0ac8098570 con 0x559a3f8565c0 2024-04-16T17:16:11.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.709+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/1820289057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0acc004240 con 0x559a3f8565c0 2024-04-16T17:16:11.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.709+0000 7f0abbe00640 1 -- 172.21.15.18:0/1820289057 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0ac809b860 con 0x559a3f8565c0 2024-04-16T17:16:11.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.709+0000 7f0ae3a4b740 10 client.28636 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:11.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.709+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/1820289057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a3f5f5f70 con 0x559a3f8565c0 2024-04-16T17:16:11.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0abbe00640 1 -- 172.21.15.18:0/1820289057 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0acc004240 con 0x559a3f8565c0 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0abbe00640 1 -- 172.21.15.18:0/1820289057 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0ac80d3ba0 con 0x559a3f8565c0 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 10 client.28636 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 10 client.28636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 20 client.28636 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a3f86f610 0x559a3f871a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 4 client.28636 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/1820289057 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a3f7c86d0 con 0x559a3f86f610 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a3f86f610 0x559a3f871a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a3f86f610 0x559a3f871a30 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a3983ad6308cfb server_cookie=5563ec7850e56285 in_seq=0 out_seq=0 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0abbe00640 10 client.28636 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0abbe00640 1 -- 172.21.15.18:0/1820289057 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a3f7c86d0 con 0x559a3f86f610 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0abbe00640 10 client.28636 handle_command_reply: tid=0 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 client.28636 shutdown 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/1820289057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 msgr2=0x559a3f85c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85c720 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80cd6b0 tx=0x7f0ac8010f10 comp rx=0 tx=0).stop 2024-04-16T17:16:11.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/1820289057 shutdown_connections 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a3f86f610 0x559a3f871a30 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f85cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85c720 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/1820289057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f85c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/1820289057 >> 172.21.15.18:0/1820289057 conn(0x559a3f8503d0 msgr2=0x559a3f852120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0abbe00640 0 client.28636 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/1820289057 shutdown_connections 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/1820289057 wait complete. 2024-04-16T17:16:11.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.710+0000 7f0ae3a4b740 20 client.28636 trim_cache size 0 max 16384 2024-04-16T17:16:11.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.742+0000 7f0ae3a4b740 1 Processor -- start 2024-04-16T17:16:11.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.742+0000 7f0ae3a4b740 1 -- start start 2024-04-16T17:16:11.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.742+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f86ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.742+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.742+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f8772e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.742+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1b41a0 con 0x559a3f85f1a0 2024-04-16T17:16:11.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.742+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1b1a70 con 0x559a3f855ca0 2024-04-16T17:16:11.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.742+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1af8e0 con 0x559a3f8565c0 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f86ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f86ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56834/0 (socket says 172.21.15.18:56834) 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ada800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f8772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 -- 172.21.15.18:0/2572651209 learned_addr learned my addr 172.21.15.18:0/2572651209 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 -- 172.21.15.18:0/2572651209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 msgr2=0x559a3f85ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/2572651209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 -- 172.21.15.18:0/2572651209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 msgr2=0x559a3f8772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/2572651209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f8772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 -- 172.21.15.18:0/2572651209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3f48d170 con 0x559a3f855ca0 2024-04-16T17:16:11.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ada800640 1 --2- 172.21.15.18:0/2572651209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f8772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:11.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.743+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/2572651209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f86ae20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0acc004a90 tx=0x7f0acc004ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9ea3645d7b6e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:11.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.744+0000 7f0ac6a00640 1 -- 172.21.15.18:0/2572651209 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc042040 con 0x559a3f855ca0 2024-04-16T17:16:11.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.744+0000 7f0ac6a00640 1 -- 172.21.15.18:0/2572651209 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0acc004b30 con 0x559a3f855ca0 2024-04-16T17:16:11.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.744+0000 7f0ac6a00640 1 -- 172.21.15.18:0/2572651209 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc054450 con 0x559a3f855ca0 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.744+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2572651209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 msgr2=0x559a3f86ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.744+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2572651209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f86ae20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0acc004a90 tx=0x7f0acc004ac0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2572651209 shutdown_connections 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2572651209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f85f1a0 0x559a3f8772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2572651209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f8565c0 0x559a3f85ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2572651209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f86ae20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2572651209 >> 172.21.15.18:0/2572651209 conn(0x559a3f8503d0 msgr2=0x559a3f84f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2572651209 shutdown_connections 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2572651209 wait complete. 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 -- start start 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.745+0000 7f0ae3a4b740 1 Processor -- start 2024-04-16T17:16:11.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f8634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f8565c0 0x559a3f8639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ae3a4b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f85f1a0 0x559a3f8d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1aff30 con 0x559a3f8565c0 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1b13a0 con 0x559a3f855ca0 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ae3a4b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1b0600 con 0x559a3f85f1a0 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ae3a4b740 10 client.0 _ll_register_callbacks cb 0x7f0ac8011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f8634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f8634a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56848/0 (socket says 172.21.15.18:56848) 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9e00640 1 -- 172.21.15.18:0/2396234298 learned_addr learned my addr 172.21.15.18:0/2396234298 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9400640 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f8565c0 0x559a3f8639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ada800640 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f85f1a0 0x559a3f8d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9e00640 1 -- 172.21.15.18:0/2396234298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f85f1a0 msgr2=0x559a3f8d4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f85f1a0 0x559a3f8d4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9e00640 1 -- 172.21.15.18:0/2396234298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f8565c0 msgr2=0x559a3f8639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f8565c0 0x559a3f8639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9e00640 1 -- 172.21.15.18:0/2396234298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3f8d50c0 con 0x559a3f855ca0 2024-04-16T17:16:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.746+0000 7f0ad9400640 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f8565c0 0x559a3f8639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:11.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.747+0000 7f0ad9e00640 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f8634a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0acc054040 tx=0x7f0acc02dfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c746c60751ce2feb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:11.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.747+0000 7f0abbe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.747+0000 7f0abbe00640 1 -- 172.21.15.18:0/2396234298 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc058030 con 0x559a3f855ca0 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.747+0000 7f0abbe00640 1 -- 172.21.15.18:0/2396234298 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f0acc02cab0 con 0x559a3f855ca0 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2396234298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0acc004b30 con 0x559a3f855ca0 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0abbe00640 1 -- 172.21.15.18:0/2396234298 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0acc042050 con 0x559a3f855ca0 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0ae3a4b740 10 client.28787 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2396234298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a3f8d6400 con 0x559a3f855ca0 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0abbe00640 1 -- 172.21.15.18:0/2396234298 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0acc004b30 con 0x559a3f855ca0 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0abbe00640 1 -- 172.21.15.18:0/2396234298 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0acc054570 con 0x559a3f855ca0 2024-04-16T17:16:11.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0ae3a4b740 10 client.28787 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:11.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0ae3a4b740 10 client.28787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:11.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.748+0000 7f0ae3a4b740 20 client.28787 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:11.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.749+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a3f8d6620 0x559a3f8d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.749+0000 7f0ae3a4b740 4 client.28787 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:16:11.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.749+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2396234298 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a3f7c2540 con 0x559a3f8d6620 2024-04-16T17:16:11.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.749+0000 7f0ad9400640 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a3f8d6620 0x559a3f8d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.749+0000 7f0ad9400640 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a3f8d6620 0x559a3f8d8a10 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50c395eb6e74b6d server_cookie=c0acbb5795cf43eb in_seq=0 out_seq=0 2024-04-16T17:16:11.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.749+0000 7f0abbe00640 10 client.28787 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0abbe00640 1 -- 172.21.15.18:0/2396234298 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a3f7c2540 con 0x559a3f8d6620 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0abbe00640 10 client.28787 handle_command_reply: tid=0 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 client.28787 shutdown 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2396234298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 msgr2=0x559a3f8634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f8634a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0acc054040 tx=0x7f0acc02dfb0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2396234298 shutdown_connections 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559a3f8d6620 0x559a3f8d8a10 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a3f85f1a0 0x559a3f8d4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559a3f8565c0 0x559a3f8639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 --2- 172.21.15.18:0/2396234298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a3f855ca0 0x559a3f8634a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2396234298 >> 172.21.15.18:0/2396234298 conn(0x559a3f8503d0 msgr2=0x559a3f851590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0abbe00640 0 client.28787 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:11.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2396234298 shutdown_connections 2024-04-16T17:16:11.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.751+0000 7f0ae3a4b740 1 -- 172.21.15.18:0/2396234298 wait complete. 2024-04-16T17:16:11.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.752+0000 7f0ae3a4b740 20 client.28787 trim_cache size 0 max 16384 2024-04-16T17:16:11.755 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:16:11.755 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:16:11.755 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "892b6895-de2d-4270-a294-299a3da81a04", 2024-04-16T17:16:11.755 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:16:11.755 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.752+0000 7f0ae0e00640 1 -- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0ac00a3e20 msgr2=0x7f0ac00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.752+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0ac00a3e20 0x7f0ac00a62e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009270 tx=0x7f0ad4009680 comp rx=0 tx=0).stop 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.752+0000 7f0ae0e00640 1 -- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc107310 msgr2=0x7f0adc1a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.752+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc107310 0x7f0adc1a0d30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009fd0 tx=0x7f0ac800b870 comp rx=0 tx=0).stop 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.752+0000 7f0ae0e00640 1 -- 172.21.15.18:0/466645764 shutdown_connections 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.753+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0ac00a3e20 0x7f0ac00a62e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.753+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0adc19a9f0 0x7f0adc19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.753+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0adc107310 0x7f0adc1a0d30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.753+0000 7f0ae0e00640 1 --2- 172.21.15.18:0/466645764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0adc102240 0x7f0adc1a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.753+0000 7f0ae0e00640 1 -- 172.21.15.18:0/466645764 >> 172.21.15.18:0/466645764 conn(0x7f0adc0fdac0 msgr2=0x7f0adc0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:11.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.753+0000 7f0ae0e00640 1 -- 172.21.15.18:0/466645764 shutdown_connections 2024-04-16T17:16:11.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.753+0000 7f0ae0e00640 1 -- 172.21.15.18:0/466645764 wait complete. 2024-04-16T17:16:11.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:11.774 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.991+0000 7fa7ece00640 1 -- 172.21.15.18:0/1615141225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e810c710 msgr2=0x7fa7e81091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.991+0000 7fa7ece00640 1 --2- 172.21.15.18:0/1615141225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e810c710 0x7fa7e81091e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009900 tx=0x7fa7d802bc80 comp rx=0 tx=0).stop 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.991+0000 7fa7ece00640 1 -- 172.21.15.18:0/1615141225 shutdown_connections 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.991+0000 7fa7ece00640 1 --2- 172.21.15.18:0/1615141225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e8117090 0x7fa7e8119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.991+0000 7fa7ece00640 1 --2- 172.21.15.18:0/1615141225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8109720 0x7fa7e8109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.991+0000 7fa7ece00640 1 --2- 172.21.15.18:0/1615141225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e810c710 0x7fa7e81091e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.991+0000 7fa7ece00640 1 -- 172.21.15.18:0/1615141225 >> 172.21.15.18:0/1615141225 conn(0x7fa7e806d0a0 msgr2=0x7fa7e806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.992+0000 7fa7ece00640 1 -- 172.21.15.18:0/1615141225 shutdown_connections 2024-04-16T17:16:11.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.992+0000 7fa7ece00640 1 -- 172.21.15.18:0/1615141225 wait complete. 2024-04-16T17:16:11.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.993+0000 7fa7ece00640 1 Processor -- start 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.993+0000 7fa7ece00640 1 -- start start 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.993+0000 7fa7ece00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e8109720 0x7fa7e81a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.993+0000 7fa7ece00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e810c710 0x7fa7e81a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.993+0000 7fa7ece00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8117090 0x7fa7e81a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.993+0000 7fa7ece00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e811cec0 con 0x7fa7e8117090 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.993+0000 7fa7ece00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e811cd40 con 0x7fa7e8109720 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.993+0000 7fa7ece00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e811d040 con 0x7fa7e810c710 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8117090 0x7fa7e81a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8117090 0x7fa7e81a7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40174/0 (socket says 172.21.15.18:40174) 2024-04-16T17:16:11.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e6800640 1 -- 172.21.15.18:0/275159552 learned_addr learned my addr 172.21.15.18:0/275159552 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:11.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e8109720 0x7fa7e81a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e6800640 1 -- 172.21.15.18:0/275159552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e810c710 msgr2=0x7fa7e81a38e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e6800640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e810c710 0x7fa7e81a38e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e6800640 1 -- 172.21.15.18:0/275159552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e8109720 msgr2=0x7fa7e81a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e6800640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e8109720 0x7fa7e81a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e6800640 1 -- 172.21.15.18:0/275159552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e81a8390 con 0x7fa7e8117090 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.994+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e8109720 0x7fa7e81a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.995+0000 7fa7e6800640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8117090 0x7fa7e81a7c60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000b9e0 tx=0x7fa7e000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.995+0000 7fa7dea00640 1 -- 172.21.15.18:0/275159552 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0019070 con 0x7fa7e8117090 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.995+0000 7fa7dea00640 1 -- 172.21.15.18:0/275159552 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e0014440 con 0x7fa7e8117090 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.995+0000 7fa7ece00640 1 -- 172.21.15.18:0/275159552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e81a8680 con 0x7fa7e8117090 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.995+0000 7fa7dea00640 1 -- 172.21.15.18:0/275159552 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00125c0 con 0x7fa7e8117090 2024-04-16T17:16:11.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:11.995+0000 7fa7ece00640 1 -- 172.21.15.18:0/275159552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81aff10 con 0x7fa7e8117090 2024-04-16T17:16:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.000+0000 7fa7efa24740 1 Processor -- start 2024-04-16T17:16:12.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:11 smithi043 ceph-mon[36901]: pgmap v362: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1008 KiB/s wr, 124 op/s 2024-04-16T17:16:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.001+0000 7fa7dea00640 1 -- 172.21.15.18:0/275159552 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa7e001c070 con 0x7fa7e8117090 2024-04-16T17:16:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.002+0000 7fa7efa24740 1 -- start start 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.002+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce42339c0 0x563ce4233da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.002+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce423c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.002+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce423cec0 0x563ce423f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7efa24740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563ce4191120 con 0x563ce42342e0 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7efa24740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563ce4137340 con 0x563ce423cec0 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7efa24740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ce418d0e0 con 0x563ce42339c0 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce42339c0 0x563ce4233da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce423c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce42339c0 0x563ce4233da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35620/0 (socket says 172.21.15.18:35620) 2024-04-16T17:16:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5400640 1 -- 172.21.15.18:0/2806691148 learned_addr learned my addr 172.21.15.18:0/2806691148 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7dea00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa7cc0a3e20 0x7fa7cc0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5400640 1 --2- 172.21.15.18:0/2806691148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce423cec0 0x563ce423f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7dea00640 1 -- 172.21.15.18:0/275159552 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fa7e00d00d0 con 0x7fa7e8117090 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5e00640 1 -- 172.21.15.18:0/2806691148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce42339c0 msgr2=0x563ce4233da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/2806691148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce42339c0 0x563ce4233da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5e00640 1 -- 172.21.15.18:0/2806691148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce423cec0 msgr2=0x563ce423f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/2806691148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce423cec0 0x563ce423f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.003+0000 7fa7e5e00640 1 -- 172.21.15.18:0/2806691148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce3f6d7d0 con 0x563ce42342e0 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.004+0000 7fa7e6800640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa7cc0a3e20 0x7fa7cc0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.004+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/2806691148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce423c980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa7d802c1c0 tx=0x7fa7d802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f73932e7bbb21eb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.004+0000 7fa7e6800640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa7cc0a3e20 0x7fa7cc0a62e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0098690 tx=0x7fa7e000c590 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.005+0000 7fa7d3400640 1 -- 172.21.15.18:0/2806691148 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8044070 con 0x563ce42342e0 2024-04-16T17:16:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.005+0000 7fa7d3400640 1 -- 172.21.15.18:0/2806691148 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d8005000 con 0x563ce42342e0 2024-04-16T17:16:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.005+0000 7fa7d3400640 1 -- 172.21.15.18:0/2806691148 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80049b0 con 0x563ce42342e0 2024-04-16T17:16:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 -- 172.21.15.18:0/2806691148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 msgr2=0x563ce423c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 --2- 172.21.15.18:0/2806691148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce423c980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa7d802c1c0 tx=0x7fa7d802c890 comp rx=0 tx=0).stop 2024-04-16T17:16:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 -- 172.21.15.18:0/2806691148 shutdown_connections 2024-04-16T17:16:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 --2- 172.21.15.18:0/2806691148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce423cec0 0x563ce423f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 --2- 172.21.15.18:0/2806691148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce423c980 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 --2- 172.21.15.18:0/2806691148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce42339c0 0x563ce4233da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 -- 172.21.15.18:0/2806691148 >> 172.21.15.18:0/2806691148 conn(0x563ce422e0f0 msgr2=0x563ce422e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 -- 172.21.15.18:0/2806691148 shutdown_connections 2024-04-16T17:16:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.006+0000 7fa7efa24740 1 -- 172.21.15.18:0/2806691148 wait complete. 2024-04-16T17:16:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.007+0000 7fa7efa24740 1 -- start start 2024-04-16T17:16:12.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.007+0000 7fa7efa24740 1 Processor -- start 2024-04-16T17:16:12.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42339c0 0x563ce423a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42342e0 0x563ce423a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce423aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7efa24740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563ce423fc50 con 0x563ce42339c0 2024-04-16T17:16:12.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7efa24740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563ce414a710 con 0x563ce42342e0 2024-04-16T17:16:12.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7efa24740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ce423fdd0 con 0x563ce423cec0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7efa24740 10 client.0 _ll_register_callbacks cb 0x563ce4145af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42339c0 0x563ce423a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42342e0 0x563ce423a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42339c0 0x563ce423a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40192/0 (socket says 172.21.15.18:40192) 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5400640 1 -- 172.21.15.18:0/3978015584 learned_addr learned my addr 172.21.15.18:0/3978015584 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5400640 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce423aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5e00640 1 -- 172.21.15.18:0/3978015584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 msgr2=0x563ce423aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce423aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5e00640 1 -- 172.21.15.18:0/3978015584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42339c0 msgr2=0x563ce423a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42339c0 0x563ce423a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.008+0000 7fa7e5e00640 1 -- 172.21.15.18:0/3978015584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce3ea33f0 con 0x563ce42342e0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.009+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42342e0 0x563ce423a570 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8036040 tx=0x7fa7d8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b0d3bcbf775706 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.009+0000 7fa7c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.009+0000 7fa7c7e00640 1 -- 172.21.15.18:0/3978015584 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d804f070 con 0x563ce42342e0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.009+0000 7fa7efa24740 1 -- 172.21.15.18:0/3978015584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d8005000 con 0x563ce42342e0 2024-04-16T17:16:12.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.009+0000 7fa7c7e00640 1 -- 172.21.15.18:0/3978015584 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fa7d8004b40 con 0x563ce42342e0 2024-04-16T17:16:12.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7c7e00640 1 -- 172.21.15.18:0/3978015584 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa7d805b050 con 0x563ce42342e0 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7efa24740 10 client.28796 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7efa24740 1 -- 172.21.15.18:0/3978015584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ce3e630f0 con 0x563ce42342e0 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7c7e00640 1 -- 172.21.15.18:0/3978015584 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d8005000 con 0x563ce42342e0 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7c7e00640 1 -- 172.21.15.18:0/3978015584 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa7d8034440 con 0x563ce42342e0 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7efa24740 10 client.28796 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7efa24740 10 client.28796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7efa24740 20 client.28796 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.010+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ce424d1c0 0x563ce424f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.011+0000 7fa7efa24740 4 client.28796 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.011+0000 7fa7efa24740 1 -- 172.21.15.18:0/3978015584 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ce419eb40 con 0x563ce424d1c0 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.011+0000 7fa7e5400640 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ce424d1c0 0x563ce424f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.011+0000 7fa7e5400640 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ce424d1c0 0x563ce424f5b0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68372c1e5bae86c server_cookie=11027f1131bec103 in_seq=0 out_seq=0 2024-04-16T17:16:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.011+0000 7fa7c7e00640 10 client.28796 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:12.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.012+0000 7fa7c7e00640 1 -- 172.21.15.18:0/3978015584 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ce419eb40 con 0x563ce424d1c0 2024-04-16T17:16:12.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.012+0000 7fa7c7e00640 10 client.28796 handle_command_reply: tid=0 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.012+0000 7fa7efa24740 1 client.28796 shutdown 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.012+0000 7fa7efa24740 1 -- 172.21.15.18:0/3978015584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42342e0 msgr2=0x563ce423a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.012+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42342e0 0x563ce423a570 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8036040 tx=0x7fa7d8005fd0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.012+0000 7fa7efa24740 1 -- 172.21.15.18:0/3978015584 shutdown_connections 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ce424d1c0 0x563ce424f5b0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce423aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42342e0 0x563ce423a570 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7c7e00640 0 client.28796 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3978015584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42339c0 0x563ce423a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7efa24740 1 -- 172.21.15.18:0/3978015584 >> 172.21.15.18:0/3978015584 conn(0x563ce422e0f0 msgr2=0x563ce422fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7efa24740 1 -- 172.21.15.18:0/3978015584 shutdown_connections 2024-04-16T17:16:12.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7efa24740 1 -- 172.21.15.18:0/3978015584 wait complete. 2024-04-16T17:16:12.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.013+0000 7fa7efa24740 20 client.28796 trim_cache size 0 max 16384 2024-04-16T17:16:12.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7efa24740 1 Processor -- start 2024-04-16T17:16:12.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7efa24740 1 -- start start 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce4248b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4239570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce4224750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7efa24740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81b8680 con 0x563ce42342e0 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7efa24740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81b5f50 con 0x563ce42339c0 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7efa24740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81b3dc0 con 0x563ce423cec0 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce4248b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce4248b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56898/0 (socket says 172.21.15.18:56898) 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7e5400640 1 -- 172.21.15.18:0/3770865357 learned_addr learned my addr 172.21.15.18:0/3770865357 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.033+0000 7fa7e5400640 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce4224750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.034+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4239570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.034+0000 7fa7e5400640 1 -- 172.21.15.18:0/3770865357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 msgr2=0x563ce4224750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.034+0000 7fa7e5400640 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce4224750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.034+0000 7fa7e5400640 1 -- 172.21.15.18:0/3770865357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 msgr2=0x563ce4239570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.034+0000 7fa7e5400640 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4239570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.034+0000 7fa7e5400640 1 -- 172.21.15.18:0/3770865357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce4238fe0 con 0x563ce42339c0 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.034+0000 7fa7e5400640 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce4248b10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006fd0 tx=0x7fa7d401c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320e7853dfc27aed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.035+0000 7fa7d2a00640 1 -- 172.21.15.18:0/3770865357 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400f050 con 0x563ce42339c0 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.035+0000 7fa7d2a00640 1 -- 172.21.15.18:0/3770865357 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d4010040 con 0x563ce42339c0 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.035+0000 7fa7d2a00640 1 -- 172.21.15.18:0/3770865357 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4021e40 con 0x563ce42339c0 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.035+0000 7fa7efa24740 1 -- 172.21.15.18:0/3770865357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 msgr2=0x563ce4248b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.035+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce4248b10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006fd0 tx=0x7fa7d401c040 comp rx=0 tx=0).stop 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 -- 172.21.15.18:0/3770865357 shutdown_connections 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce4224750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4239570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 --2- 172.21.15.18:0/3770865357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce4248b10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 -- 172.21.15.18:0/3770865357 >> 172.21.15.18:0/3770865357 conn(0x563ce422e0f0 msgr2=0x563ce4224040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 -- 172.21.15.18:0/3770865357 shutdown_connections 2024-04-16T17:16:12.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 -- 172.21.15.18:0/3770865357 wait complete. 2024-04-16T17:16:12.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 -- start start 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.036+0000 7fa7efa24740 1 Processor -- start 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce42409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4240f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7efa24740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce4241440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7efa24740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81b4410 con 0x563ce42342e0 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7efa24740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81b5880 con 0x563ce42339c0 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7efa24740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81b4ae0 con 0x563ce423cec0 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7efa24740 10 client.0 _ll_register_callbacks cb 0x7fa7d4009150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce42409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce42409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56914/0 (socket says 172.21.15.18:56914) 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7e5400640 1 -- 172.21.15.18:0/4166953306 learned_addr learned my addr 172.21.15.18:0/4166953306 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4240f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.037+0000 7fa7e5400640 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce4241440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7e5e00640 1 -- 172.21.15.18:0/4166953306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 msgr2=0x563ce4241440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce4241440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7e5e00640 1 -- 172.21.15.18:0/4166953306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 msgr2=0x563ce42409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce42409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7e5e00640 1 -- 172.21.15.18:0/4166953306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce42b23c0 con 0x563ce42342e0 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7e5400640 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce42409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7e5e00640 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4240f00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80349b0 tx=0x7fa7d80056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1a1c74301e554e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.038+0000 7fa7c7e00640 1 -- 172.21.15.18:0/4166953306 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8058070 con 0x563ce42342e0 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.039+0000 7fa7c7e00640 1 -- 172.21.15.18:0/4166953306 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7fa7d8034c30 con 0x563ce42342e0 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.039+0000 7fa7efa24740 1 -- 172.21.15.18:0/4166953306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d4010040 con 0x563ce42342e0 2024-04-16T17:16:12.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.039+0000 7fa7c7e00640 1 -- 172.21.15.18:0/4166953306 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa7d804f040 con 0x563ce42342e0 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.039+0000 7fa7efa24740 10 client.18483 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.039+0000 7fa7efa24740 1 -- 172.21.15.18:0/4166953306 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ce42b3630 con 0x563ce42342e0 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.039+0000 7fa7c7e00640 1 -- 172.21.15.18:0/4166953306 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d4010040 con 0x563ce42342e0 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.040+0000 7fa7c7e00640 1 -- 172.21.15.18:0/4166953306 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa7d8049450 con 0x563ce42342e0 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.040+0000 7fa7efa24740 10 client.18483 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.040+0000 7fa7efa24740 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.040+0000 7fa7efa24740 20 client.18483 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.040+0000 7fa7efa24740 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ce42b3de0 0x563ce42b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.040+0000 7fa7efa24740 4 client.18483 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:12.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.040+0000 7fa7efa24740 1 -- 172.21.15.18:0/4166953306 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ce41989b0 con 0x563ce42b3de0 2024-04-16T17:16:12.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.040+0000 7fa7e5400640 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ce42b3de0 0x563ce42b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7e5400640 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ce42b3de0 0x563ce42b6200 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8a4aae2232ad03 server_cookie=cada3fa651abb88c in_seq=0 out_seq=0 2024-04-16T17:16:12.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7c7e00640 10 client.18483 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:12.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7c7e00640 1 -- 172.21.15.18:0/4166953306 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563ce41989b0 con 0x563ce42b3de0 2024-04-16T17:16:12.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7c7e00640 10 client.18483 handle_command_reply: tid=0 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 client.18483 shutdown 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 -- 172.21.15.18:0/4166953306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 msgr2=0x563ce4240f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4240f00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80349b0 tx=0x7fa7d80056f0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 -- 172.21.15.18:0/4166953306 shutdown_connections 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ce42b3de0 0x563ce42b6200 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ce423cec0 0x563ce4241440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ce42342e0 0x563ce4240f00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 --2- 172.21.15.18:0/4166953306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ce42339c0 0x563ce42409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7c7e00640 0 client.18483 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 -- 172.21.15.18:0/4166953306 >> 172.21.15.18:0/4166953306 conn(0x563ce422e0f0 msgr2=0x563ce423a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 -- 172.21.15.18:0/4166953306 shutdown_connections 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.041+0000 7fa7efa24740 1 -- 172.21.15.18:0/4166953306 wait complete. 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7efa24740 20 client.18483 trim_cache size 0 max 16384 2024-04-16T17:16:12.043 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout: "892b6895-de2d-4270-a294-299a3da81a04": { 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "892b6895-de2d-4270-a294-299a3da81a04", 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:16:12.044 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 -- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa7cc0a3e20 msgr2=0x7fa7cc0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa7cc0a3e20 0x7fa7cc0a62e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0098690 tx=0x7fa7e000c590 comp rx=0 tx=0).stop 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 -- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8117090 msgr2=0x7fa7e81a7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8117090 0x7fa7e81a7c60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000b9e0 tx=0x7fa7e000bea0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 -- 172.21.15.18:0/275159552 shutdown_connections 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa7cc0a3e20 0x7fa7cc0a62e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7e8117090 0x7fa7e81a7c60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e810c710 0x7fa7e81a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 --2- 172.21.15.18:0/275159552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7e8109720 0x7fa7e81a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.042+0000 7fa7ece00640 1 -- 172.21.15.18:0/275159552 >> 172.21.15.18:0/275159552 conn(0x7fa7e806d0a0 msgr2=0x7fa7e8117470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.043+0000 7fa7ece00640 1 -- 172.21.15.18:0/275159552 shutdown_connections 2024-04-16T17:16:12.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:12.043+0000 7fa7ece00640 1 -- 172.21.15.18:0/275159552 wait complete. 2024-04-16T17:16:12.060 INFO:tasks.cephfs.filesystem:scrub status for tag:892b6895-de2d-4270-a294-299a3da81a04 - {'path': '/', 'tag': '892b6895-de2d-4270-a294-299a3da81a04', 'options': 'recursive,force'} 2024-04-16T17:16:12.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:11 smithi022 ceph-mon[36023]: pgmap v362: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1008 KiB/s wr, 124 op/s 2024-04-16T17:16:12.605 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:11 smithi018 ceph-mon[31776]: pgmap v362: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1008 KiB/s wr, 124 op/s 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- 172.21.15.18:0/783604265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124104e20 msgr2=0x7f3124108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 --2- 172.21.15.18:0/783604265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124104e20 0x7f3124108e60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3110009900 tx=0x7f311002bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- 172.21.15.18:0/783604265 shutdown_connections 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 --2- 172.21.15.18:0/783604265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3124109540 0x7f3124111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 --2- 172.21.15.18:0/783604265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124104e20 0x7f3124108e60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 --2- 172.21.15.18:0/783604265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3124104470 0x7f3124104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- 172.21.15.18:0/783604265 >> 172.21.15.18:0/783604265 conn(0x7f31240ffcf0 msgr2=0x7f3124102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- 172.21.15.18:0/783604265 shutdown_connections 2024-04-16T17:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- 172.21.15.18:0/783604265 wait complete. 2024-04-16T17:16:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 Processor -- start 2024-04-16T17:16:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- start start 2024-04-16T17:16:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3124104470 0x7f31241a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124109540 0x7f31241a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f312419a980 0x7f312419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3124114620 con 0x7f3124109540 2024-04-16T17:16:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f31241144a0 con 0x7f312419a980 2024-04-16T17:16:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312c000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31241147a0 con 0x7f3124104470 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f3128c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124109540 0x7f31241a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f312419a980 0x7f312419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f3128c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124109540 0x7f31241a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40240/0 (socket says 172.21.15.18:40240) 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f312419a980 0x7f312419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56930/0 (socket says 172.21.15.18:56930) 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f3128c00640 1 -- 172.21.15.18:0/2774205156 learned_addr learned my addr 172.21.15.18:0/2774205156 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312a000640 1 -- 172.21.15.18:0/2774205156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3124104470 msgr2=0x7f31241a0780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312a000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3124104470 0x7f31241a0780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312a000640 1 -- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124109540 msgr2=0x7f31241a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312a000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124109540 0x7f31241a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312a000640 1 -- 172.21.15.18:0/2774205156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312419b4a0 con 0x7f312419a980 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.650+0000 7f312a000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f312419a980 0x7f312419ae30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f311c009fd0 tx=0x7f311c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.655+0000 7f3122000640 1 -- 172.21.15.18:0/2774205156 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c00ce20 con 0x7f312419a980 2024-04-16T17:16:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.655+0000 7f3122000640 1 -- 172.21.15.18:0/2774205156 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f311c005020 con 0x7f312419a980 2024-04-16T17:16:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.655+0000 7f312c000640 1 -- 172.21.15.18:0/2774205156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312419b790 con 0x7f312419a980 2024-04-16T17:16:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.655+0000 7f3122000640 1 -- 172.21.15.18:0/2774205156 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c010720 con 0x7f312419a980 2024-04-16T17:16:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.655+0000 7f312c000640 1 -- 172.21.15.18:0/2774205156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31241a7990 con 0x7f312419a980 2024-04-16T17:16:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.658+0000 7f312edf3740 1 Processor -- start 2024-04-16T17:16:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.659+0000 7f3122000640 1 -- 172.21.15.18:0/2774205156 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f311c005540 con 0x7f312419a980 2024-04-16T17:16:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.660+0000 7f312edf3740 1 -- start start 2024-04-16T17:16:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.660+0000 7f312edf3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe9610 0x560672fe8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.660+0000 7f312edf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672fe8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.660+0000 7f312edf3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560672ff4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.660+0000 7f312edf3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560672f4d780 con 0x560672fe9610 2024-04-16T17:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.660+0000 7f312edf3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560672ef3f40 con 0x560672ff1e20 2024-04-16T17:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.660+0000 7f312edf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560672f49740 con 0x560672fe88a0 2024-04-16T17:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3129600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672fe8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3128c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe9610 0x560672fe8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3129600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672fe8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35674/0 (socket says 172.21.15.18:35674) 2024-04-16T17:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3129600640 1 -- 172.21.15.18:0/420503930 learned_addr learned my addr 172.21.15.18:0/420503930 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3129600640 1 -- 172.21.15.18:0/420503930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 msgr2=0x560672ff4210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3129600640 1 --2- 172.21.15.18:0/420503930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560672ff4210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3129600640 1 -- 172.21.15.18:0/420503930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe9610 msgr2=0x560672fe8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3129600640 1 --2- 172.21.15.18:0/420503930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe9610 0x560672fe8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.661+0000 7f3129600640 1 -- 172.21.15.18:0/420503930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560672c60210 con 0x560672fe88a0 2024-04-16T17:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.662+0000 7f3122000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f310c0a3d90 0x7f310c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.662+0000 7f3122000640 1 -- 172.21.15.18:0/2774205156 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f311c0d1340 con 0x7f312419a980 2024-04-16T17:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.662+0000 7f3128c00640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f310c0a3d90 0x7f310c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.663+0000 7f3128c00640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f310c0a3d90 0x7f310c0a6250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3124103450 tx=0x7f311002df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.665+0000 7f3129600640 1 --2- 172.21.15.18:0/420503930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672fe8d20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f311800ba50 tx=0x7f311800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13053ba92d0e648 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.665+0000 7f3116600640 1 -- 172.21.15.18:0/420503930 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118016070 con 0x560672fe88a0 2024-04-16T17:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.665+0000 7f3116600640 1 -- 172.21.15.18:0/420503930 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3118005500 con 0x560672fe88a0 2024-04-16T17:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.665+0000 7f3116600640 1 -- 172.21.15.18:0/420503930 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118010460 con 0x560672fe88a0 2024-04-16T17:16:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 -- 172.21.15.18:0/420503930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 msgr2=0x560672fe8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 --2- 172.21.15.18:0/420503930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672fe8d20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f311800ba50 tx=0x7f311800bf10 comp rx=0 tx=0).stop 2024-04-16T17:16:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 -- 172.21.15.18:0/420503930 shutdown_connections 2024-04-16T17:16:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 --2- 172.21.15.18:0/420503930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560672ff4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 --2- 172.21.15.18:0/420503930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672fe8d20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 --2- 172.21.15.18:0/420503930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe9610 0x560672fe8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 -- 172.21.15.18:0/420503930 >> 172.21.15.18:0/420503930 conn(0x560672f835d0 msgr2=0x560672fe3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 -- 172.21.15.18:0/420503930 shutdown_connections 2024-04-16T17:16:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.666+0000 7f312edf3740 1 -- 172.21.15.18:0/420503930 wait complete. 2024-04-16T17:16:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.667+0000 7f312edf3740 1 -- start start 2024-04-16T17:16:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.667+0000 7f312edf3740 1 Processor -- start 2024-04-16T17:16:12.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 430133 3.60 MB/sec execute 505 sec latency 47.422 ms 2024-04-16T17:16:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f312edf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672feecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f312edf3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672fe9610 0x560672fef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f312edf3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672ff1e20 0x560672fef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3128c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672fe9610 0x560672fef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3128c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672fe9610 0x560672fef1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56944/0 (socket says 172.21.15.18:56944) 2024-04-16T17:16:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3128c00640 1 -- 172.21.15.18:0/2052724813 learned_addr learned my addr 172.21.15.18:0/2052724813 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3129600640 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672feecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560672ff4b50 con 0x560672ff1e20 2024-04-16T17:16:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560672f07390 con 0x560672fe9610 2024-04-16T17:16:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560672ff4cd0 con 0x560672fe88a0 2024-04-16T17:16:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f312edf3740 10 client.0 _ll_register_callbacks cb 0x560672ef0f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f312a000640 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672ff1e20 0x560672fef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3129600640 1 -- 172.21.15.18:0/2052724813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672fe9610 msgr2=0x560672fef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3129600640 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672fe9610 0x560672fef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3129600640 1 -- 172.21.15.18:0/2052724813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672ff1e20 msgr2=0x560672fef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3129600640 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672ff1e20 0x560672fef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.668+0000 7f3129600640 1 -- 172.21.15.18:0/2052724813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560672d2c620 con 0x560672fe88a0 2024-04-16T17:16:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f3129600640 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672feecb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f311800ba50 tx=0x7f311800b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59fa3f5690c0f565 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f310aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f310aa00640 1 -- 172.21.15.18:0/2052724813 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118016030 con 0x560672fe88a0 2024-04-16T17:16:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f310aa00640 1 -- 172.21.15.18:0/2052724813 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f31180051e0 con 0x560672fe88a0 2024-04-16T17:16:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3118005500 con 0x560672fe88a0 2024-04-16T17:16:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f310aa00640 1 -- 172.21.15.18:0/2052724813 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f31180048d0 con 0x560672fe88a0 2024-04-16T17:16:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 10 client.28666 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560673009480 con 0x560672fe88a0 2024-04-16T17:16:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f310aa00640 1 -- 172.21.15.18:0/2052724813 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3118005500 con 0x560672fe88a0 2024-04-16T17:16:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f310aa00640 1 -- 172.21.15.18:0/2052724813 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3118022070 con 0x560672fe88a0 2024-04-16T17:16:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 10 client.28666 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 10 client.28666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 20 client.28666 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56067300a890 0x56067300cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 4 client.28666 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560672f5b1a0 con 0x56067300a890 2024-04-16T17:16:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.677+0000 7f3128c00640 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56067300a890 0x56067300cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.679+0000 7f3128c00640 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56067300a890 0x56067300cc80 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26ea881b43ac4be server_cookie=d24aa1a1fa4c0706 in_seq=0 out_seq=0 2024-04-16T17:16:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.679+0000 7f310aa00640 10 client.28666 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f310aa00640 1 -- 172.21.15.18:0/2052724813 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560672f5b1a0 con 0x56067300a890 2024-04-16T17:16:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f310aa00640 10 client.28666 handle_command_reply: tid=0 2024-04-16T17:16:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 client.28666 shutdown 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 msgr2=0x560672feecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672feecb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f311800ba50 tx=0x7f311800b7d0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 shutdown_connections 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56067300a890 0x56067300cc80 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672ff1e20 0x560672fef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672fe9610 0x560672fef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 --2- 172.21.15.18:0/2052724813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe88a0 0x560672feecb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 >> 172.21.15.18:0/2052724813 conn(0x560672f835d0 msgr2=0x560672fe52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f310aa00640 0 client.28666 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 shutdown_connections 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 1 -- 172.21.15.18:0/2052724813 wait complete. 2024-04-16T17:16:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.683+0000 7f312edf3740 20 client.28666 trim_cache size 0 max 16384 2024-04-16T17:16:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.717+0000 7f312edf3740 1 Processor -- start 2024-04-16T17:16:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.717+0000 7f312edf3740 1 -- start start 2024-04-16T17:16:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.717+0000 7f312edf3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560672ff13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.717+0000 7f312edf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 0x560673012430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.717+0000 7f312edf3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560673012970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.717+0000 7f312edf3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f31241afee0 con 0x560672fe88a0 2024-04-16T17:16:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.717+0000 7f312edf3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f31241ad7b0 con 0x560672ff1e20 2024-04-16T17:16:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.717+0000 7f312edf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31241ab5e0 con 0x560672fe9610 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560672ff13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f312a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560673012970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560672ff13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40270/0 (socket says 172.21.15.18:40270) 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 -- 172.21.15.18:0/2514644078 learned_addr learned my addr 172.21.15.18:0/2514644078 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3128c00640 1 --2- 172.21.15.18:0/2514644078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 0x560673012430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 -- 172.21.15.18:0/2514644078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 msgr2=0x560673012430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 --2- 172.21.15.18:0/2514644078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 0x560673012430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 -- 172.21.15.18:0/2514644078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 msgr2=0x560673012970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 --2- 172.21.15.18:0/2514644078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560673012970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 -- 172.21.15.18:0/2514644078 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560672fe7b70 con 0x560672fe88a0 2024-04-16T17:16:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.718+0000 7f3129600640 1 --2- 172.21.15.18:0/2514644078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560672ff13e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f311800d830 tx=0x7f31180035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33969800f97b39e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.719+0000 7f3115c00640 1 -- 172.21.15.18:0/2514644078 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31180048d0 con 0x560672fe88a0 2024-04-16T17:16:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.719+0000 7f3115c00640 1 -- 172.21.15.18:0/2514644078 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3118003980 con 0x560672fe88a0 2024-04-16T17:16:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.719+0000 7f3115c00640 1 -- 172.21.15.18:0/2514644078 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118005630 con 0x560672fe88a0 2024-04-16T17:16:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 -- 172.21.15.18:0/2514644078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 msgr2=0x560672ff13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 --2- 172.21.15.18:0/2514644078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560672ff13e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f311800d830 tx=0x7f31180035f0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 -- 172.21.15.18:0/2514644078 shutdown_connections 2024-04-16T17:16:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 --2- 172.21.15.18:0/2514644078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560673012970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 --2- 172.21.15.18:0/2514644078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 0x560673012430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 --2- 172.21.15.18:0/2514644078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560672ff13e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 -- 172.21.15.18:0/2514644078 >> 172.21.15.18:0/2514644078 conn(0x560672f835d0 msgr2=0x560672fed5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 -- 172.21.15.18:0/2514644078 shutdown_connections 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 -- 172.21.15.18:0/2514644078 wait complete. 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.720+0000 7f312edf3740 1 -- start start 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.721+0000 7f312edf3740 1 Processor -- start 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.721+0000 7f312edf3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560673002b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.721+0000 7f312edf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 0x560672febf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.721+0000 7f312edf3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560672fec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.721+0000 7f312edf3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f31241abc70 con 0x560672fe88a0 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.721+0000 7f312edf3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f31241ad0e0 con 0x560672ff1e20 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.721+0000 7f312edf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31241ac340 con 0x560672fe9610 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.721+0000 7f312edf3740 10 client.0 _ll_register_callbacks cb 0x7f311800b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560673002b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f312a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560672fec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560673002b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40284/0 (socket says 172.21.15.18:40284) 2024-04-16T17:16:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 -- 172.21.15.18:0/3213078571 learned_addr learned my addr 172.21.15.18:0/3213078571 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3128c00640 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 0x560672febf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 -- 172.21.15.18:0/3213078571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 msgr2=0x560672febf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 0x560672febf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 -- 172.21.15.18:0/3213078571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 msgr2=0x560672fec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560672fec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 -- 172.21.15.18:0/3213078571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560672feccc0 con 0x560672fe88a0 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f312a000640 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560672fec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f3129600640 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560673002b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f31180059c0 tx=0x7f3118004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218ef9515ce92a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.722+0000 7f310aa00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:16:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.723+0000 7f310aa00640 1 -- 172.21.15.18:0/3213078571 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118040060 con 0x560672fe88a0 2024-04-16T17:16:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.723+0000 7f310aa00640 1 -- 172.21.15.18:0/3213078571 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9642+0+0 (secure 0 0 0) 0x7f311800da20 con 0x560672fe88a0 2024-04-16T17:16:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.723+0000 7f312edf3740 1 -- 172.21.15.18:0/3213078571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3118003980 con 0x560672fe88a0 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.723+0000 7f310aa00640 1 -- 172.21.15.18:0/3213078571 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3118047050 con 0x560672fe88a0 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.723+0000 7f312edf3740 10 client.18501 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.723+0000 7f312edf3740 1 -- 172.21.15.18:0/3213078571 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560673070d00 con 0x560672fe88a0 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.723+0000 7f310aa00640 1 -- 172.21.15.18:0/3213078571 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3118003980 con 0x560672fe88a0 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.724+0000 7f310aa00640 1 -- 172.21.15.18:0/3213078571 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3118022050 con 0x560672fe88a0 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.724+0000 7f312edf3740 10 client.18501 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.724+0000 7f312edf3740 10 client.18501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.724+0000 7f312edf3740 20 client.18501 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.724+0000 7f312edf3740 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606730712b0 0x5606730736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.724+0000 7f312edf3740 4 client.18501 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.724+0000 7f312edf3740 1 -- 172.21.15.18:0/3213078571 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560672f55010 con 0x5606730712b0 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.724+0000 7f3128c00640 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606730712b0 0x5606730736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.725+0000 7f3128c00640 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606730712b0 0x5606730736a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155760b3e1be6793 server_cookie=7325f675b6195e78 in_seq=0 out_seq=0 2024-04-16T17:16:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.725+0000 7f310aa00640 10 client.18501 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f310aa00640 1 -- 172.21.15.18:0/3213078571 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560672f55010 con 0x5606730712b0 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f310aa00640 10 client.18501 handle_command_reply: tid=0 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 client.18501 shutdown 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 -- 172.21.15.18:0/3213078571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 msgr2=0x560673002b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560673002b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f31180059c0 tx=0x7f3118004950 comp rx=0 tx=0).stop 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 -- 172.21.15.18:0/3213078571 shutdown_connections 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606730712b0 0x5606730736a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560672ff1e20 0x560672fec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560672fe9610 0x560672febf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 --2- 172.21.15.18:0/3213078571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560672fe88a0 0x560673002b70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f310aa00640 0 client.18501 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.881+0000 7f312edf3740 1 -- 172.21.15.18:0/3213078571 >> 172.21.15.18:0/3213078571 conn(0x560672f835d0 msgr2=0x560672fe1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.882+0000 7f312edf3740 1 -- 172.21.15.18:0/3213078571 shutdown_connections 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.882+0000 7f312edf3740 1 -- 172.21.15.18:0/3213078571 wait complete. 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.882+0000 7f312edf3740 20 client.18501 trim_cache size 0 max 16384 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:16:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.882+0000 7f312c000640 1 -- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f310c0a3d90 msgr2=0x7f310c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.882+0000 7f312c000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f310c0a3d90 0x7f310c0a6250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3124103450 tx=0x7f311002df90 comp rx=0 tx=0).stop 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.882+0000 7f312c000640 1 -- 172.21.15.18:0/2774205156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f312419a980 msgr2=0x7f312419ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.882+0000 7f312c000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f312419a980 0x7f312419ae30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f311c009fd0 tx=0x7f311c00ef10 comp rx=0 tx=0).stop 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.883+0000 7f312c000640 1 -- 172.21.15.18:0/2774205156 shutdown_connections 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.883+0000 7f312c000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f310c0a3d90 0x7f310c0a6250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.883+0000 7f312c000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f312419a980 0x7f312419ae30 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.883+0000 7f312c000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3124109540 0x7f31241a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.883+0000 7f312c000640 1 --2- 172.21.15.18:0/2774205156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3124104470 0x7f31241a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.883+0000 7f312c000640 1 -- 172.21.15.18:0/2774205156 >> 172.21.15.18:0/2774205156 conn(0x7f31240ffcf0 msgr2=0x7f3124102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.883+0000 7f312c000640 1 -- 172.21.15.18:0/2774205156 shutdown_connections 2024-04-16T17:16:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:12.883+0000 7f312c000640 1 -- 172.21.15.18:0/2774205156 wait complete. 2024-04-16T17:16:13.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:12 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:16:13.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:12 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:13.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:12 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:16:13.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:12 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:13.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:12 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:16:13.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:12 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:13.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:12 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:16:13.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:12 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:13.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 430468 3.59 MB/sec execute 506 sec latency 58.163 ms 2024-04-16T17:16:14.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:13 smithi043 ceph-mon[36901]: pgmap v363: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-04-16T17:16:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:13 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:16:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:13 smithi043 ceph-mon[36901]: osdmap e111: 12 total, 12 up, 12 in 2024-04-16T17:16:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:13 smithi022 ceph-mon[36023]: pgmap v363: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-04-16T17:16:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:13 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:16:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:13 smithi022 ceph-mon[36023]: osdmap e111: 12 total, 12 up, 12 in 2024-04-16T17:16:14.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 431147 3.59 MB/sec execute 507 sec latency 68.923 ms 2024-04-16T17:16:15.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:14 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:16:15.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:14 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:16:15.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:14 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:16:15.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:14 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:16:15.049 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:12 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:16:15.049 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:12 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:15.049 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:12 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:16:15.049 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:12 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:15.050 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:13 smithi018 ceph-mon[31776]: pgmap v363: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-04-16T17:16:15.050 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:13 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:16:15.050 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:13 smithi018 ceph-mon[31776]: osdmap e111: 12 total, 12 up, 12 in 2024-04-16T17:16:15.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:14 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:16:15.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:14 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:16:15.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:14 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:16:15.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:14 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:16:15.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 432389 3.60 MB/sec execute 508 sec latency 28.037 ms 2024-04-16T17:16:16.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:15 smithi043 ceph-mon[36901]: pgmap v365: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-04-16T17:16:16.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:15 smithi022 ceph-mon[36023]: pgmap v365: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-04-16T17:16:16.667 INFO:tasks.workunit.client.0.smithi018.stdout: 1 433027 3.61 MB/sec execute 509 sec latency 64.724 ms 2024-04-16T17:16:17.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 433267 3.60 MB/sec execute 510 sec latency 76.947 ms 2024-04-16T17:16:17.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:17 smithi022 ceph-mon[36023]: pgmap v366: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-04-16T17:16:18.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:17 smithi043 ceph-mon[36901]: pgmap v366: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-04-16T17:16:18.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 433705 3.59 MB/sec execute 511 sec latency 66.900 ms 2024-04-16T17:16:19.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 434461 3.59 MB/sec execute 512 sec latency 61.627 ms 2024-04-16T17:16:20.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:19 smithi043 ceph-mon[36901]: pgmap v367: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-04-16T17:16:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:19 smithi022 ceph-mon[36023]: pgmap v367: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-04-16T17:16:20.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 435642 3.60 MB/sec execute 513 sec latency 39.595 ms 2024-04-16T17:16:21.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 436343 3.60 MB/sec execute 514 sec latency 86.093 ms 2024-04-16T17:16:22.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:21 smithi043 ceph-mon[36901]: pgmap v368: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-16T17:16:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:21 smithi022 ceph-mon[36023]: pgmap v368: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-16T17:16:22.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 436572 3.60 MB/sec execute 515 sec latency 105.610 ms 2024-04-16T17:16:23.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 436932 3.59 MB/sec execute 516 sec latency 82.333 ms 2024-04-16T17:16:24.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:23 smithi043 ceph-mon[36901]: pgmap v369: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-04-16T17:16:24.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:23 smithi022 ceph-mon[36023]: pgmap v369: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-04-16T17:16:24.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 437819 3.60 MB/sec execute 517 sec latency 35.798 ms 2024-04-16T17:16:25.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 438649 3.59 MB/sec execute 518 sec latency 57.518 ms 2024-04-16T17:16:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:25 smithi043 ceph-mon[36901]: pgmap v370: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.019+0000 7fd12de00640 1 -- 172.21.15.18:0/1913122585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd128104e20 msgr2=0x7fd128108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.019+0000 7fd12de00640 1 --2- 172.21.15.18:0/1913122585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd128104e20 0x7fd128108e60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd114009900 tx=0x7fd11402bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.019+0000 7fd12de00640 1 -- 172.21.15.18:0/1913122585 shutdown_connections 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.019+0000 7fd12de00640 1 --2- 172.21.15.18:0/1913122585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128109540 0x7fd128111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.019+0000 7fd12de00640 1 --2- 172.21.15.18:0/1913122585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd128104e20 0x7fd128108e60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.019+0000 7fd12de00640 1 --2- 172.21.15.18:0/1913122585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd128104470 0x7fd128104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.019+0000 7fd12de00640 1 -- 172.21.15.18:0/1913122585 >> 172.21.15.18:0/1913122585 conn(0x7fd1280ffcf0 msgr2=0x7fd128102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.019+0000 7fd12de00640 1 -- 172.21.15.18:0/1913122585 shutdown_connections 2024-04-16T17:16:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.020+0000 7fd12de00640 1 -- 172.21.15.18:0/1913122585 wait complete. 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.020+0000 7fd12de00640 1 Processor -- start 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd12de00640 1 -- start start 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd12de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128104470 0x7fd1281a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd12de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd128109540 0x7fd1281a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd12de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd12819a940 0x7fd12819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd12de00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd128114600 con 0x7fd12819a940 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd12de00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd128114480 con 0x7fd128109540 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd12de00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd128114780 con 0x7fd128104470 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd126a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128104470 0x7fd1281a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd126a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128104470 0x7fd1281a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:45218/0 (socket says 172.21.15.18:45218) 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd126a00640 1 -- 172.21.15.18:0/3676292781 learned_addr learned my addr 172.21.15.18:0/3676292781 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd127400640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd12819a940 0x7fd12819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.021+0000 7fd126000640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd128109540 0x7fd1281a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.022+0000 7fd126a00640 1 -- 172.21.15.18:0/3676292781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd128109540 msgr2=0x7fd1281a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.022+0000 7fd126a00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd128109540 0x7fd1281a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.022+0000 7fd126a00640 1 -- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd12819a940 msgr2=0x7fd12819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.022+0000 7fd126a00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd12819a940 0x7fd12819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.022+0000 7fd126a00640 1 -- 172.21.15.18:0/3676292781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12819b460 con 0x7fd128104470 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.022+0000 7fd126a00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128104470 0x7fd1281a0740 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd11800e9f0 tx=0x7fd11800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.022+0000 7fd12ca00640 1 -- 172.21.15.18:0/3676292781 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11800cd80 con 0x7fd128104470 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.022+0000 7fd12de00640 1 -- 172.21.15.18:0/3676292781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12819b750 con 0x7fd128104470 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.023+0000 7fd12ca00640 1 -- 172.21.15.18:0/3676292781 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd118010910 con 0x7fd128104470 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.023+0000 7fd12ca00640 1 -- 172.21.15.18:0/3676292781 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1180188f0 con 0x7fd128104470 2024-04-16T17:16:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.023+0000 7fd12de00640 1 -- 172.21.15.18:0/3676292781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1281a7950 con 0x7fd128104470 2024-04-16T17:16:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.027+0000 7fd130a06740 1 Processor -- start 2024-04-16T17:16:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.027+0000 7fd130a06740 1 -- start start 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.027+0000 7fd12ca00640 1 -- 172.21.15.18:0/3676292781 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd118010ac0 con 0x7fd128104470 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.027+0000 7fd130a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d04360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.027+0000 7fd130a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.027+0000 7fd130a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 0x555651d10210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.027+0000 7fd130a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555651c69780 con 0x555651d048a0 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd130a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555651c0ff40 con 0x555651d0de20 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd130a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555651c65740 con 0x555651d05610 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd126a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd127400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d04360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd126a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d04d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45902/0 (socket says 172.21.15.18:45902) 2024-04-16T17:16:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd127400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d04360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:45232/0 (socket says 172.21.15.18:45232) 2024-04-16T17:16:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd126a00640 1 -- 172.21.15.18:0/3424121448 learned_addr learned my addr 172.21.15.18:0/3424121448 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd127400640 1 -- 172.21.15.18:0/3424121448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 msgr2=0x555651d10210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd127400640 1 --2- 172.21.15.18:0/3424121448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 0x555651d10210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd127400640 1 -- 172.21.15.18:0/3424121448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 msgr2=0x555651d04d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd127400640 1 --2- 172.21.15.18:0/3424121448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d04d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.028+0000 7fd127400640 1 -- 172.21.15.18:0/3424121448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55565197c210 con 0x555651d05610 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.029+0000 7fd127400640 1 --2- 172.21.15.18:0/3424121448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d04360 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd120009f00 tx=0x7fd120011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf4bb3d5a8f5a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.029+0000 7fd12ca00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd10c0a3d10 0x7fd10c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.029+0000 7fd113e00640 1 -- 172.21.15.18:0/3424121448 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120004d20 con 0x555651d05610 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.029+0000 7fd113e00640 1 -- 172.21.15.18:0/3424121448 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd120004050 con 0x555651d05610 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.029+0000 7fd113e00640 1 -- 172.21.15.18:0/3424121448 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120005280 con 0x555651d05610 2024-04-16T17:16:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.029+0000 7fd12ca00640 1 -- 172.21.15.18:0/3676292781 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fd1180d0fb0 con 0x7fd128104470 2024-04-16T17:16:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.030+0000 7fd126a00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd10c0a3d10 0x7fd10c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.030+0000 7fd130a06740 1 -- 172.21.15.18:0/3424121448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 msgr2=0x555651d04360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.030+0000 7fd130a06740 1 --2- 172.21.15.18:0/3424121448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d04360 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd120009f00 tx=0x7fd120011810 comp rx=0 tx=0).stop 2024-04-16T17:16:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.030+0000 7fd130a06740 1 -- 172.21.15.18:0/3424121448 shutdown_connections 2024-04-16T17:16:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.030+0000 7fd130a06740 1 --2- 172.21.15.18:0/3424121448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 0x555651d10210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.030+0000 7fd130a06740 1 --2- 172.21.15.18:0/3424121448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d04d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.030+0000 7fd130a06740 1 --2- 172.21.15.18:0/3424121448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d04360 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.030+0000 7fd130a06740 1 -- 172.21.15.18:0/3424121448 >> 172.21.15.18:0/3424121448 conn(0x555651c9f5d0 msgr2=0x555651cff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.031+0000 7fd130a06740 1 -- 172.21.15.18:0/3424121448 shutdown_connections 2024-04-16T17:16:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.031+0000 7fd126a00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd10c0a3d10 0x7fd10c0a61d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd11809cb80 tx=0x7fd1180d4b00 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.031+0000 7fd130a06740 1 -- 172.21.15.18:0/3424121448 wait complete. 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.032+0000 7fd130a06740 1 -- start start 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.032+0000 7fd130a06740 1 Processor -- start 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.032+0000 7fd130a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d048a0 0x555651d0d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd130a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d0ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd130a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d0b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd130a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555651d10b50 con 0x555651d0de20 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd130a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555651c23390 con 0x555651d048a0 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd130a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555651d10cd0 con 0x555651d05610 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd130a06740 10 client.0 _ll_register_callbacks cb 0x555651c0cf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd126000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d0ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd127400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d048a0 0x555651d0d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd126000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d0ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:45242/0 (socket says 172.21.15.18:45242) 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd126000640 1 -- 172.21.15.18:0/1849581624 learned_addr learned my addr 172.21.15.18:0/1849581624 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd127400640 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d0b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd126000640 1 -- 172.21.15.18:0/1849581624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d048a0 msgr2=0x555651d0d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd126000640 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d048a0 0x555651d0d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd126000640 1 -- 172.21.15.18:0/1849581624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 msgr2=0x555651d0b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd126000640 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d0b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd126000640 1 -- 172.21.15.18:0/1849581624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555651a48620 con 0x555651d05610 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.033+0000 7fd127400640 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d0b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.034+0000 7fd126000640 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d0ac00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd114000c00 tx=0x7fd1140045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d268371c28a70b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.034+0000 7fd110c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.034+0000 7fd110c00640 1 -- 172.21.15.18:0/1849581624 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114045070 con 0x555651d05610 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.034+0000 7fd110c00640 1 -- 172.21.15.18:0/1849581624 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fd11402ca90 con 0x555651d05610 2024-04-16T17:16:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.034+0000 7fd130a06740 1 -- 172.21.15.18:0/1849581624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd120004050 con 0x555651d05610 2024-04-16T17:16:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.035+0000 7fd110c00640 1 -- 172.21.15.18:0/1849581624 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd114051050 con 0x555651d05610 2024-04-16T17:16:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.035+0000 7fd130a06740 10 client.28684 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.035+0000 7fd130a06740 1 -- 172.21.15.18:0/1849581624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555651d1cbe0 con 0x555651d05610 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.035+0000 7fd110c00640 1 -- 172.21.15.18:0/1849581624 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd120004050 con 0x555651d05610 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.035+0000 7fd110c00640 1 -- 172.21.15.18:0/1849581624 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd114005280 con 0x555651d05610 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.036+0000 7fd130a06740 10 client.28684 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.036+0000 7fd130a06740 10 client.28684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.036+0000 7fd130a06740 20 client.28684 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.036+0000 7fd130a06740 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555651d1e0a0 0x555651d20490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.036+0000 7fd130a06740 4 client.28684 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.036+0000 7fd130a06740 1 -- 172.21.15.18:0/1849581624 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555651c771a0 con 0x555651d1e0a0 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.036+0000 7fd127400640 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555651d1e0a0 0x555651d20490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.037+0000 7fd127400640 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555651d1e0a0 0x555651d20490 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10fd3a9cfe319b76 server_cookie=93ec16bf91e4f521 in_seq=0 out_seq=0 2024-04-16T17:16:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.037+0000 7fd110c00640 10 client.28684 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.038+0000 7fd110c00640 1 -- 172.21.15.18:0/1849581624 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555651c771a0 con 0x555651d1e0a0 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.038+0000 7fd110c00640 10 client.28684 handle_command_reply: tid=0 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.038+0000 7fd130a06740 1 client.28684 shutdown 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.038+0000 7fd130a06740 1 -- 172.21.15.18:0/1849581624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 msgr2=0x555651d0ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.038+0000 7fd130a06740 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d0ac00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd114000c00 tx=0x7fd1140045d0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 1 -- 172.21.15.18:0/1849581624 shutdown_connections 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555651d1e0a0 0x555651d20490 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d0b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d0ac00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 1 --2- 172.21.15.18:0/1849581624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d048a0 0x555651d0d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 1 -- 172.21.15.18:0/1849581624 >> 172.21.15.18:0/1849581624 conn(0x555651c9f5d0 msgr2=0x555651d012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd110c00640 0 client.28684 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 1 -- 172.21.15.18:0/1849581624 shutdown_connections 2024-04-16T17:16:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 1 -- 172.21.15.18:0/1849581624 wait complete. 2024-04-16T17:16:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.039+0000 7fd130a06740 20 client.28684 trim_cache size 0 max 16384 2024-04-16T17:16:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.072+0000 7fd130a06740 1 Processor -- start 2024-04-16T17:16:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.072+0000 7fd130a06740 1 -- start start 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.072+0000 7fd130a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d0aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.072+0000 7fd130a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d19a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.072+0000 7fd130a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 0x555651d25d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.072+0000 7fd130a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281b4180 con 0x555651d048a0 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.072+0000 7fd130a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281b1a50 con 0x555651d0de20 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.072+0000 7fd130a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281af8c0 con 0x555651d05610 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d0aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d0aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45926/0 (socket says 172.21.15.18:45926) 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 -- 172.21.15.18:0/1372876736 learned_addr learned my addr 172.21.15.18:0/1372876736 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd126000640 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d19a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 0x555651d25d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 -- 172.21.15.18:0/1372876736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 msgr2=0x555651d19a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d19a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 -- 172.21.15.18:0/1372876736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 msgr2=0x555651d25d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 0x555651d25d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.073+0000 7fd127400640 1 -- 172.21.15.18:0/1372876736 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555651d03b70 con 0x555651d048a0 2024-04-16T17:16:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.074+0000 7fd127400640 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d0aba0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd120014040 tx=0x7fd120003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bbbb7ff9396a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.074+0000 7fd113400640 1 -- 172.21.15.18:0/1372876736 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12002b020 con 0x555651d048a0 2024-04-16T17:16:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.074+0000 7fd113400640 1 -- 172.21.15.18:0/1372876736 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd120004530 con 0x555651d048a0 2024-04-16T17:16:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.074+0000 7fd113400640 1 -- 172.21.15.18:0/1372876736 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120027bc0 con 0x555651d048a0 2024-04-16T17:16:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 -- 172.21.15.18:0/1372876736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 msgr2=0x555651d0aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d0aba0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd120014040 tx=0x7fd120003970 comp rx=0 tx=0).stop 2024-04-16T17:16:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 -- 172.21.15.18:0/1372876736 shutdown_connections 2024-04-16T17:16:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d0de20 0x555651d25d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d05610 0x555651d19a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 --2- 172.21.15.18:0/1372876736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d048a0 0x555651d0aba0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 -- 172.21.15.18:0/1372876736 >> 172.21.15.18:0/1372876736 conn(0x555651c9f5d0 msgr2=0x555651d0a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 -- 172.21.15.18:0/1372876736 shutdown_connections 2024-04-16T17:16:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.075+0000 7fd130a06740 1 -- 172.21.15.18:0/1372876736 wait complete. 2024-04-16T17:16:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.076+0000 7fd130a06740 1 -- start start 2024-04-16T17:16:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.076+0000 7fd130a06740 1 Processor -- start 2024-04-16T17:16:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd130a06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d048a0 0x555651d11ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd130a06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d05610 0x555651d12010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd130a06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d12550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd130a06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281aff10 con 0x555651d0de20 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd130a06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281b1380 con 0x555651d05610 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd130a06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281b05e0 con 0x555651d048a0 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd130a06740 10 client.0 _ll_register_callbacks cb 0x7fd1200093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd127400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d048a0 0x555651d11ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd126000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d05610 0x555651d12010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd127400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d048a0 0x555651d11ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:45262/0 (socket says 172.21.15.18:45262) 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd127400640 1 -- 172.21.15.18:0/4092488884 learned_addr learned my addr 172.21.15.18:0/4092488884 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.077+0000 7fd127400640 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d12550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd126000640 1 -- 172.21.15.18:0/4092488884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d048a0 msgr2=0x555651d11ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd126000640 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d048a0 0x555651d11ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd126000640 1 -- 172.21.15.18:0/4092488884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 msgr2=0x555651d12550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd126000640 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d12550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd126000640 1 -- 172.21.15.18:0/4092488884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555651d83730 con 0x555651d05610 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd126000640 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d05610 0x555651d12010 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd114005770 tx=0x7fd1140058c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa615cd712e4924 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd110c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd110c00640 1 -- 172.21.15.18:0/4092488884 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114045070 con 0x555651d05610 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd130a06740 1 -- 172.21.15.18:0/4092488884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd120004530 con 0x555651d05610 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.078+0000 7fd110c00640 1 -- 172.21.15.18:0/4092488884 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fd114003b30 con 0x555651d05610 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd110c00640 1 -- 172.21.15.18:0/4092488884 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd1140043e0 con 0x555651d05610 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd130a06740 10 client.28832 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd130a06740 1 -- 172.21.15.18:0/4092488884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555651d84a20 con 0x555651d05610 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd110c00640 1 -- 172.21.15.18:0/4092488884 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd120004530 con 0x555651d05610 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd110c00640 1 -- 172.21.15.18:0/4092488884 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd114004750 con 0x555651d05610 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd130a06740 10 client.28832 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd130a06740 10 client.28832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd130a06740 20 client.28832 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd130a06740 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555651d851c0 0x555651d875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd130a06740 4 client.28832 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:16:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd130a06740 1 -- 172.21.15.18:0/4092488884 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555651c71010 con 0x555651d851c0 2024-04-16T17:16:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.079+0000 7fd127400640 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555651d851c0 0x555651d875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.080+0000 7fd127400640 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555651d851c0 0x555651d875e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e70511e45fb75c9 server_cookie=1794a3a5e0b599d3 in_seq=0 out_seq=0 2024-04-16T17:16:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.080+0000 7fd110c00640 10 client.28832 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:26.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:25 smithi022 ceph-mon[36023]: pgmap v370: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-04-16T17:16:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.254+0000 7fd110c00640 1 -- 172.21.15.18:0/4092488884 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555651c71010 con 0x555651d851c0 2024-04-16T17:16:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.254+0000 7fd110c00640 10 client.28832 handle_command_reply: tid=0 2024-04-16T17:16:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.254+0000 7fd130a06740 1 client.28832 shutdown 2024-04-16T17:16:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.254+0000 7fd130a06740 1 -- 172.21.15.18:0/4092488884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d05610 msgr2=0x555651d12010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.254+0000 7fd130a06740 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d05610 0x555651d12010 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd114005770 tx=0x7fd1140058c0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 1 -- 172.21.15.18:0/4092488884 shutdown_connections 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555651d851c0 0x555651d875e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555651d0de20 0x555651d12550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555651d05610 0x555651d12010 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd110c00640 0 client.28832 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 1 --2- 172.21.15.18:0/4092488884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555651d048a0 0x555651d11ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 1 -- 172.21.15.18:0/4092488884 >> 172.21.15.18:0/4092488884 conn(0x555651c9f5d0 msgr2=0x555651cfde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 1 -- 172.21.15.18:0/4092488884 shutdown_connections 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 1 -- 172.21.15.18:0/4092488884 wait complete. 2024-04-16T17:16:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.255+0000 7fd130a06740 20 client.28832 trim_cache size 0 max 16384 2024-04-16T17:16:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:16:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:16:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:16:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.256+0000 7fd12de00640 1 -- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd10c0a3d10 msgr2=0x7fd10c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.256+0000 7fd12de00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd10c0a3d10 0x7fd10c0a61d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd11809cb80 tx=0x7fd1180d4b00 comp rx=0 tx=0).stop 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.256+0000 7fd12de00640 1 -- 172.21.15.18:0/3676292781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128104470 msgr2=0x7fd1281a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.256+0000 7fd12de00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128104470 0x7fd1281a0740 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd11800e9f0 tx=0x7fd11800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.256+0000 7fd12de00640 1 -- 172.21.15.18:0/3676292781 shutdown_connections 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.256+0000 7fd12de00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd10c0a3d10 0x7fd10c0a61d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.256+0000 7fd12de00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd12819a940 0x7fd12819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.256+0000 7fd12de00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd128109540 0x7fd1281a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.257+0000 7fd12de00640 1 --2- 172.21.15.18:0/3676292781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd128104470 0x7fd1281a0740 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.257+0000 7fd12de00640 1 -- 172.21.15.18:0/3676292781 >> 172.21.15.18:0/3676292781 conn(0x7fd1280ffcf0 msgr2=0x7fd128102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.257+0000 7fd12de00640 1 -- 172.21.15.18:0/3676292781 shutdown_connections 2024-04-16T17:16:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:26.257+0000 7fd12de00640 1 -- 172.21.15.18:0/3676292781 wait complete. 2024-04-16T17:16:26.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 439392 3.60 MB/sec execute 519 sec latency 88.442 ms 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:14 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:14 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:14 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:14 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:15 smithi018 ceph-mon[31776]: pgmap v365: 129 pgs: 129 active+clean; 43 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:17 smithi018 ceph-mon[31776]: pgmap v366: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:19 smithi018 ceph-mon[31776]: pgmap v367: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:21 smithi018 ceph-mon[31776]: pgmap v368: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:23 smithi018 ceph-mon[31776]: pgmap v369: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-04-16T17:16:27.002 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:25 smithi018 ceph-mon[31776]: pgmap v370: 129 pgs: 129 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-04-16T17:16:27.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 439854 3.60 MB/sec execute 520 sec latency 96.258 ms 2024-04-16T17:16:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:27 smithi043 ceph-mon[36901]: pgmap v371: 129 pgs: 129 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-16T17:16:28.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:27 smithi022 ceph-mon[36023]: pgmap v371: 129 pgs: 129 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-16T17:16:28.668 INFO:tasks.workunit.client.0.smithi018.stdout: 1 440234 3.60 MB/sec execute 521 sec latency 70.424 ms 2024-04-16T17:16:29.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 440834 3.59 MB/sec execute 522 sec latency 27.885 ms 2024-04-16T17:16:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:29 smithi043 ceph-mon[36901]: pgmap v372: 129 pgs: 129 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 75 op/s 2024-04-16T17:16:30.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:29 smithi022 ceph-mon[36023]: pgmap v372: 129 pgs: 129 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 75 op/s 2024-04-16T17:16:30.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 441631 3.59 MB/sec execute 523 sec latency 34.623 ms 2024-04-16T17:16:31.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 442253 3.59 MB/sec execute 524 sec latency 76.355 ms 2024-04-16T17:16:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:31 smithi043 ceph-mon[36901]: pgmap v373: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-04-16T17:16:32.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:31 smithi022 ceph-mon[36023]: pgmap v373: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-04-16T17:16:32.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 442837 3.59 MB/sec execute 525 sec latency 59.612 ms 2024-04-16T17:16:33.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 443413 3.60 MB/sec execute 526 sec latency 74.548 ms 2024-04-16T17:16:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:33 smithi043 ceph-mon[36901]: pgmap v374: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-04-16T17:16:34.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:33 smithi022 ceph-mon[36023]: pgmap v374: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-04-16T17:16:34.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 443970 3.59 MB/sec execute 527 sec latency 22.006 ms 2024-04-16T17:16:35.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 444829 3.59 MB/sec execute 528 sec latency 54.079 ms 2024-04-16T17:16:36.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:35 smithi043 ceph-mon[36901]: pgmap v375: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-04-16T17:16:36.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:35 smithi022 ceph-mon[36023]: pgmap v375: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-04-16T17:16:36.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 445180 3.59 MB/sec execute 529 sec latency 116.432 ms 2024-04-16T17:16:37.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 445441 3.58 MB/sec execute 530 sec latency 59.595 ms 2024-04-16T17:16:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:37 smithi022 ceph-mon[36023]: pgmap v376: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-16T17:16:38.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:37 smithi043 ceph-mon[36901]: pgmap v376: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-16T17:16:38.671 INFO:tasks.workunit.client.0.smithi018.stdout: 1 446164 3.59 MB/sec execute 531 sec latency 45.031 ms 2024-04-16T17:16:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.396+0000 7faaca000640 1 -- 172.21.15.18:0/821546361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac41061b0 msgr2=0x7faac4106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.396+0000 7faaca000640 1 --2- 172.21.15.18:0/821546361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac41061b0 0x7faac4106640 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faabc009ca0 tx=0x7faabc02c1d0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.396+0000 7faaca000640 1 -- 172.21.15.18:0/821546361 shutdown_connections 2024-04-16T17:16:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.396+0000 7faaca000640 1 --2- 172.21.15.18:0/821546361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac41061b0 0x7faac4106640 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.396+0000 7faaca000640 1 --2- 172.21.15.18:0/821546361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faac41016d0 0x7faac4105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.396+0000 7faaca000640 1 --2- 172.21.15.18:0/821546361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faac4100db0 0x7faac4101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.396+0000 7faaca000640 1 -- 172.21.15.18:0/821546361 >> 172.21.15.18:0/821546361 conn(0x7faac40fc820 msgr2=0x7faac40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.396+0000 7faaca000640 1 -- 172.21.15.18:0/821546361 shutdown_connections 2024-04-16T17:16:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.397+0000 7faaca000640 1 -- 172.21.15.18:0/821546361 wait complete. 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.397+0000 7faaca000640 1 Processor -- start 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faaca000640 1 -- start start 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faaca000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faac4100db0 0x7faac41a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faaca000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac41016d0 0x7faac41a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faaca000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faac41061b0 0x7faac419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faaca000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faac41187b0 con 0x7faac4100db0 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faaca000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faac4118630 con 0x7faac41061b0 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faaca000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faac4118930 con 0x7faac41016d0 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faac8c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faac4100db0 0x7faac41a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faac8c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faac4100db0 0x7faac41a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50398/0 (socket says 172.21.15.18:50398) 2024-04-16T17:16:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faac8c00640 1 -- 172.21.15.18:0/2198025696 learned_addr learned my addr 172.21.15.18:0/2198025696 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.398+0000 7faac3e00640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac41016d0 0x7faac41a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.399+0000 7faac9600640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faac41061b0 0x7faac419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.399+0000 7faac8c00640 1 -- 172.21.15.18:0/2198025696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac41016d0 msgr2=0x7faac41a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.399+0000 7faac8c00640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac41016d0 0x7faac41a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.399+0000 7faac8c00640 1 -- 172.21.15.18:0/2198025696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faac41061b0 msgr2=0x7faac419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.399+0000 7faac8c00640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faac41061b0 0x7faac419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.399+0000 7faac8c00640 1 -- 172.21.15.18:0/2198025696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac419f210 con 0x7faac4100db0 2024-04-16T17:16:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.399+0000 7faac8c00640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faac4100db0 0x7faac41a48e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faab800e9f0 tx=0x7faab800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.400+0000 7faac1600640 1 -- 172.21.15.18:0/2198025696 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab800cd80 con 0x7faac4100db0 2024-04-16T17:16:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.400+0000 7faaca000640 1 -- 172.21.15.18:0/2198025696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac419f500 con 0x7faac4100db0 2024-04-16T17:16:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.400+0000 7faac1600640 1 -- 172.21.15.18:0/2198025696 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab8010910 con 0x7faac4100db0 2024-04-16T17:16:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.400+0000 7faac1600640 1 -- 172.21.15.18:0/2198025696 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80188f0 con 0x7faac4100db0 2024-04-16T17:16:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.400+0000 7faaca000640 1 -- 172.21.15.18:0/2198025696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac41ab7a0 con 0x7faac4100db0 2024-04-16T17:16:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.405+0000 7faace1f5740 1 Processor -- start 2024-04-16T17:16:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.405+0000 7faac1600640 1 -- 172.21.15.18:0/2198025696 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faab8010ac0 con 0x7faac4100db0 2024-04-16T17:16:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.406+0000 7faace1f5740 1 -- start start 2024-04-16T17:16:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.406+0000 7faace1f5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c2a7f0 0x55f270c29540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.406+0000 7faace1f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c29f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.406+0000 7faace1f5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c33000 0x55f270c353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.406+0000 7faace1f5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f270b8e780 con 0x55f270c33000 2024-04-16T17:16:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.406+0000 7faace1f5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f270b34f40 con 0x55f270c2a7f0 2024-04-16T17:16:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.406+0000 7faace1f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f270b8a740 con 0x55f270c29a80 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.406+0000 7faac8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c29f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c2a7f0 0x55f270c29540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c29f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37218/0 (socket says 172.21.15.18:37218) 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac8c00640 1 -- 172.21.15.18:0/1105884340 learned_addr learned my addr 172.21.15.18:0/1105884340 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac3e00640 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c33000 0x55f270c353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac9600640 1 -- 172.21.15.18:0/1105884340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 msgr2=0x55f270c29f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac9600640 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c29f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac9600640 1 -- 172.21.15.18:0/1105884340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c33000 msgr2=0x55f270c353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac9600640 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c33000 0x55f270c353f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac9600640 1 -- 172.21.15.18:0/1105884340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2708a1210 con 0x55f270c2a7f0 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.407+0000 7faac8c00640 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c29f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.408+0000 7faac9600640 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c2a7f0 0x55f270c29540 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7faabc02c7e0 tx=0x7faabc009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97601badf01ec12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.408+0000 7faab5c00640 1 -- 172.21.15.18:0/1105884340 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc043070 con 0x55f270c2a7f0 2024-04-16T17:16:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.408+0000 7faab5c00640 1 -- 172.21.15.18:0/1105884340 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc003e50 con 0x55f270c2a7f0 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.408+0000 7faab5c00640 1 -- 172.21.15.18:0/1105884340 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc039040 con 0x55f270c2a7f0 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.408+0000 7faac1600640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faaac0a3d80 0x7faaac0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faac1600640 1 -- 172.21.15.18:0/2198025696 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7faab80d0f50 con 0x7faac4100db0 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faac8c00640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faaac0a3d80 0x7faaac0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faace1f5740 1 -- 172.21.15.18:0/1105884340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c2a7f0 msgr2=0x55f270c29540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faace1f5740 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c2a7f0 0x55f270c29540 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7faabc02c7e0 tx=0x7faabc009900 comp rx=0 tx=0).stop 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faace1f5740 1 -- 172.21.15.18:0/1105884340 shutdown_connections 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faace1f5740 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c33000 0x55f270c353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faace1f5740 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c29f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faace1f5740 1 --2- 172.21.15.18:0/1105884340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c2a7f0 0x55f270c29540 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.409+0000 7faace1f5740 1 -- 172.21.15.18:0/1105884340 >> 172.21.15.18:0/1105884340 conn(0x55f270bc45d0 msgr2=0x55f270c24770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.410+0000 7faace1f5740 1 -- 172.21.15.18:0/1105884340 shutdown_connections 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.410+0000 7faac8c00640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faaac0a3d80 0x7faaac0a6240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faab809cac0 tx=0x7faab80d4a00 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.410+0000 7faace1f5740 1 -- 172.21.15.18:0/1105884340 wait complete. 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.410+0000 7faace1f5740 1 -- start start 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faace1f5740 1 Processor -- start 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faace1f5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c29a80 0x55f270c2fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faace1f5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2a7f0 0x55f270c303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faace1f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c33000 0x55f270c30900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faace1f5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f270c35cd0 con 0x55f270c2a7f0 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faace1f5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f270b48390 con 0x55f270c29a80 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faace1f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f270c35e50 con 0x55f270c33000 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faace1f5740 10 client.0 _ll_register_callbacks cb 0x55f270b31f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faac3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2a7f0 0x55f270c303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faac9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c29a80 0x55f270c2fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faac3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2a7f0 0x55f270c303c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50416/0 (socket says 172.21.15.18:50416) 2024-04-16T17:16:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.411+0000 7faac3e00640 1 -- 172.21.15.18:0/555813985 learned_addr learned my addr 172.21.15.18:0/555813985 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faac9600640 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c33000 0x55f270c30900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faac3e00640 1 -- 172.21.15.18:0/555813985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c33000 msgr2=0x55f270c30900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faac3e00640 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c33000 0x55f270c30900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faac3e00640 1 -- 172.21.15.18:0/555813985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c29a80 msgr2=0x55f270c2fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faac3e00640 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c29a80 0x55f270c2fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faac3e00640 1 -- 172.21.15.18:0/555813985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27096d620 con 0x55f270c2a7f0 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faac9600640 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c29a80 0x55f270c2fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faac3e00640 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2a7f0 0x55f270c303c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faab0012700 tx=0x7faab0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ae67b26b7d1cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.412+0000 7faaaa000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:16:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.413+0000 7faaaa000640 1 -- 172.21.15.18:0/555813985 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab000f920 con 0x55f270c2a7f0 2024-04-16T17:16:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.413+0000 7faace1f5740 1 -- 172.21.15.18:0/555813985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faabc003e50 con 0x55f270c2a7f0 2024-04-16T17:16:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.413+0000 7faaaa000640 1 -- 172.21.15.18:0/555813985 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7faab000fac0 con 0x55f270c2a7f0 2024-04-16T17:16:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.413+0000 7faaaa000640 1 -- 172.21.15.18:0/555813985 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faab0025050 con 0x55f270c2a7f0 2024-04-16T17:16:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.413+0000 7faace1f5740 10 client.18531 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.413+0000 7faace1f5740 1 -- 172.21.15.18:0/555813985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f270c41eb0 con 0x55f270c2a7f0 2024-04-16T17:16:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.414+0000 7faaaa000640 1 -- 172.21.15.18:0/555813985 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc003e50 con 0x55f270c2a7f0 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.414+0000 7faaaa000640 1 -- 172.21.15.18:0/555813985 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faab00051e0 con 0x55f270c2a7f0 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.414+0000 7faace1f5740 10 client.18531 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.414+0000 7faace1f5740 10 client.18531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.414+0000 7faace1f5740 20 client.18531 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.414+0000 7faace1f5740 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f270c43320 0x55f270c45710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.414+0000 7faace1f5740 4 client.18531 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.414+0000 7faace1f5740 1 -- 172.21.15.18:0/555813985 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f270b9c1a0 con 0x55f270c43320 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.415+0000 7faac9600640 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f270c43320 0x55f270c45710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.416+0000 7faac9600640 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f270c43320 0x55f270c45710 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7542e6c4a16f02d3 server_cookie=58c967363a0a1aed in_seq=0 out_seq=0 2024-04-16T17:16:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.416+0000 7faaaa000640 10 client.18531 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.417+0000 7faaaa000640 1 -- 172.21.15.18:0/555813985 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f270b9c1a0 con 0x55f270c43320 2024-04-16T17:16:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.417+0000 7faaaa000640 10 client.18531 handle_command_reply: tid=0 2024-04-16T17:16:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.417+0000 7faace1f5740 1 client.18531 shutdown 2024-04-16T17:16:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 -- 172.21.15.18:0/555813985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2a7f0 msgr2=0x55f270c303c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2a7f0 0x55f270c303c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faab0012700 tx=0x7faab0012bc0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 -- 172.21.15.18:0/555813985 shutdown_connections 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f270c43320 0x55f270c45710 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c33000 0x55f270c30900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2a7f0 0x55f270c303c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 --2- 172.21.15.18:0/555813985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c29a80 0x55f270c2fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 -- 172.21.15.18:0/555813985 >> 172.21.15.18:0/555813985 conn(0x55f270bc45d0 msgr2=0x55f270c264c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faaaa000640 0 client.18531 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 -- 172.21.15.18:0/555813985 shutdown_connections 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 1 -- 172.21.15.18:0/555813985 wait complete. 2024-04-16T17:16:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.418+0000 7faace1f5740 20 client.18531 trim_cache size 0 max 16384 2024-04-16T17:16:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.440+0000 7faace1f5740 1 Processor -- start 2024-04-16T17:16:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.440+0000 7faace1f5740 1 -- start start 2024-04-16T17:16:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.440+0000 7faace1f5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c29a80 0x55f270c325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.440+0000 7faace1f5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c3ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.440+0000 7faace1f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c2efe0 0x55f270c30550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.440+0000 7faace1f5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b83a0 con 0x55f270c29a80 2024-04-16T17:16:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.440+0000 7faace1f5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b5c70 con 0x55f270c33000 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.440+0000 7faace1f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b3ae0 con 0x55f270c2efe0 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c3ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c3ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54204/0 (socket says 172.21.15.18:54204) 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 -- 172.21.15.18:0/3137097906 learned_addr learned my addr 172.21.15.18:0/3137097906 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac9600640 1 --2- 172.21.15.18:0/3137097906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c29a80 0x55f270c325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 -- 172.21.15.18:0/3137097906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c2efe0 msgr2=0x55f270c30550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 --2- 172.21.15.18:0/3137097906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c2efe0 0x55f270c30550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 -- 172.21.15.18:0/3137097906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c29a80 msgr2=0x55f270c325b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 --2- 172.21.15.18:0/3137097906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c29a80 0x55f270c325b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 -- 172.21.15.18:0/3137097906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f270c28d50 con 0x55f270c33000 2024-04-16T17:16:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faac3e00640 1 --2- 172.21.15.18:0/3137097906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c3ebd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7faab0012040 tx=0x7faab00093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18187c977f0d48c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faab5200640 1 -- 172.21.15.18:0/3137097906 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab002e070 con 0x55f270c33000 2024-04-16T17:16:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faab5200640 1 -- 172.21.15.18:0/3137097906 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab001c4f0 con 0x55f270c33000 2024-04-16T17:16:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.441+0000 7faab5200640 1 -- 172.21.15.18:0/3137097906 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00053a0 con 0x55f270c33000 2024-04-16T17:16:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 -- 172.21.15.18:0/3137097906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 msgr2=0x55f270c3ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 --2- 172.21.15.18:0/3137097906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c3ebd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7faab0012040 tx=0x7faab00093f0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 -- 172.21.15.18:0/3137097906 shutdown_connections 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 --2- 172.21.15.18:0/3137097906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c2efe0 0x55f270c30550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 --2- 172.21.15.18:0/3137097906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c3ebd0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 --2- 172.21.15.18:0/3137097906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c29a80 0x55f270c325b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 -- 172.21.15.18:0/3137097906 >> 172.21.15.18:0/3137097906 conn(0x55f270bc45d0 msgr2=0x55f270c2fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 -- 172.21.15.18:0/3137097906 shutdown_connections 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.442+0000 7faace1f5740 1 -- 172.21.15.18:0/3137097906 wait complete. 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.443+0000 7faace1f5740 1 -- start start 2024-04-16T17:16:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.443+0000 7faace1f5740 1 Processor -- start 2024-04-16T17:16:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faace1f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c2ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faace1f5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2efe0 0x55f270c2d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faace1f5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c2d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faace1f5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b4130 con 0x55f270c2efe0 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faace1f5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b55a0 con 0x55f270c33000 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faace1f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b4800 con 0x55f270c29a80 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faace1f5740 10 client.0 _ll_register_callbacks cb 0x7faab0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faac3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2efe0 0x55f270c2d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faac3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2efe0 0x55f270c2d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50424/0 (socket says 172.21.15.18:50424) 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faac3e00640 1 -- 172.21.15.18:0/3621546983 learned_addr learned my addr 172.21.15.18:0/3621546983 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faac9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c2ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.444+0000 7faac9600640 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c2d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.445+0000 7faac3e00640 1 -- 172.21.15.18:0/3621546983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 msgr2=0x55f270c2ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.445+0000 7faac3e00640 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c2ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.445+0000 7faac3e00640 1 -- 172.21.15.18:0/3621546983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 msgr2=0x55f270c2d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.445+0000 7faac3e00640 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c2d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.445+0000 7faac3e00640 1 -- 172.21.15.18:0/3621546983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f270ca8b80 con 0x55f270c2efe0 2024-04-16T17:16:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.445+0000 7faac3e00640 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2efe0 0x55f270c2d370 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faab00058c0 tx=0x7faab0005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10f5b858a498e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.445+0000 7faaaa000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:16:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.446+0000 7faaaa000640 1 -- 172.21.15.18:0/3621546983 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab002e030 con 0x55f270c2efe0 2024-04-16T17:16:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.446+0000 7faaaa000640 1 -- 172.21.15.18:0/3621546983 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7faab0018040 con 0x55f270c2efe0 2024-04-16T17:16:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.446+0000 7faace1f5740 1 -- 172.21.15.18:0/3621546983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab001c4f0 con 0x55f270c2efe0 2024-04-16T17:16:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faaaa000640 1 -- 172.21.15.18:0/3621546983 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faab003b050 con 0x55f270c2efe0 2024-04-16T17:16:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faace1f5740 10 client.18537 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faace1f5740 1 -- 172.21.15.18:0/3621546983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f270ca9e90 con 0x55f270c2efe0 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faaaa000640 1 -- 172.21.15.18:0/3621546983 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab001c4f0 con 0x55f270c2efe0 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faaaa000640 1 -- 172.21.15.18:0/3621546983 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faab00035f0 con 0x55f270c2efe0 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faace1f5740 10 client.18537 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faace1f5740 10 client.18537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faace1f5740 20 client.18537 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faace1f5740 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f270caa4f0 0x55f270cac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faace1f5740 4 client.18537 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.447+0000 7faace1f5740 1 -- 172.21.15.18:0/3621546983 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f270b96010 con 0x55f270caa4f0 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.448+0000 7faac9600640 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f270caa4f0 0x55f270cac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.448+0000 7faac9600640 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f270caa4f0 0x55f270cac8e0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28bdd83aecc1d70 server_cookie=508c6a23a2e831fb in_seq=0 out_seq=0 2024-04-16T17:16:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.448+0000 7faaaa000640 10 client.18537 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faaaa000640 1 -- 172.21.15.18:0/3621546983 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f270b96010 con 0x55f270caa4f0 2024-04-16T17:16:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faaaa000640 10 client.18537 handle_command_reply: tid=0 2024-04-16T17:16:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 client.18537 shutdown 2024-04-16T17:16:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 -- 172.21.15.18:0/3621546983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2efe0 msgr2=0x55f270c2d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2efe0 0x55f270c2d370 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faab00058c0 tx=0x7faab0005f70 comp rx=0 tx=0).stop 2024-04-16T17:16:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 -- 172.21.15.18:0/3621546983 shutdown_connections 2024-04-16T17:16:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f270caa4f0 0x55f270cac8e0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f270c33000 0x55f270c2d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f270c2efe0 0x55f270c2d370 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 --2- 172.21.15.18:0/3621546983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f270c29a80 0x55f270c2ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faace1f5740 1 -- 172.21.15.18:0/3621546983 >> 172.21.15.18:0/3621546983 conn(0x55f270bc45d0 msgr2=0x55f270c306d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.618+0000 7faaaa000640 0 client.18537 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.619+0000 7faace1f5740 1 -- 172.21.15.18:0/3621546983 shutdown_connections 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.619+0000 7faace1f5740 1 -- 172.21.15.18:0/3621546983 wait complete. 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.619+0000 7faace1f5740 20 client.18537 trim_cache size 0 max 16384 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:16:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:16:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:16:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:16:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.620+0000 7faaca000640 1 -- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faaac0a3d80 msgr2=0x7faaac0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.620+0000 7faaca000640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faaac0a3d80 0x7faaac0a6240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faab809cac0 tx=0x7faab80d4a00 comp rx=0 tx=0).stop 2024-04-16T17:16:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.620+0000 7faaca000640 1 -- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faac4100db0 msgr2=0x7faac41a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.620+0000 7faaca000640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faac4100db0 0x7faac41a48e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faab800e9f0 tx=0x7faab800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.620+0000 7faaca000640 1 -- 172.21.15.18:0/2198025696 shutdown_connections 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.620+0000 7faaca000640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7faaac0a3d80 0x7faaac0a6240 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.620+0000 7faaca000640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faac41061b0 0x7faac419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.621+0000 7faaca000640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac41016d0 0x7faac41a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.621+0000 7faaca000640 1 --2- 172.21.15.18:0/2198025696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faac4100db0 0x7faac41a48e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.621+0000 7faaca000640 1 -- 172.21.15.18:0/2198025696 >> 172.21.15.18:0/2198025696 conn(0x7faac40fc820 msgr2=0x7faac4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.621+0000 7faaca000640 1 -- 172.21.15.18:0/2198025696 shutdown_connections 2024-04-16T17:16:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:39.621+0000 7faaca000640 1 -- 172.21.15.18:0/2198025696 wait complete. 2024-04-16T17:16:39.669 INFO:tasks.workunit.client.0.smithi018.stdout: 1 447191 3.59 MB/sec execute 532 sec latency 43.147 ms 2024-04-16T17:16:39.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:27 smithi018 ceph-mon[31776]: pgmap v371: 129 pgs: 129 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-16T17:16:39.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:29 smithi018 ceph-mon[31776]: pgmap v372: 129 pgs: 129 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 75 op/s 2024-04-16T17:16:39.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:31 smithi018 ceph-mon[31776]: pgmap v373: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-04-16T17:16:39.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:33 smithi018 ceph-mon[31776]: pgmap v374: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-04-16T17:16:39.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:35 smithi018 ceph-mon[31776]: pgmap v375: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-04-16T17:16:39.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:37 smithi018 ceph-mon[31776]: pgmap v376: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-16T17:16:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:39 smithi022 ceph-mon[36023]: pgmap v377: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-04-16T17:16:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:39 smithi043 ceph-mon[36901]: pgmap v377: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-04-16T17:16:40.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 447813 3.59 MB/sec execute 533 sec latency 28.028 ms 2024-04-16T17:16:41.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 448420 3.59 MB/sec execute 534 sec latency 38.169 ms 2024-04-16T17:16:42.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:42.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:41 smithi022 ceph-mon[36023]: pgmap v378: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-04-16T17:16:42.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:41 smithi043 ceph-mon[36901]: pgmap v378: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-04-16T17:16:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.251+0000 7f4be0e00640 1 -- 172.21.15.18:0/3420623479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc104df0 msgr2=0x7f4bdc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.251+0000 7f4be0e00640 1 --2- 172.21.15.18:0/3420623479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc104df0 0x7f4bdc108e30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009860 tx=0x7f4bc802bb90 comp rx=0 tx=0).stop 2024-04-16T17:16:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.251+0000 7f4be0e00640 1 -- 172.21.15.18:0/3420623479 shutdown_connections 2024-04-16T17:16:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.251+0000 7f4be0e00640 1 --2- 172.21.15.18:0/3420623479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bdc109510 0x7f4bdc1110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.251+0000 7f4be0e00640 1 --2- 172.21.15.18:0/3420623479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc104df0 0x7f4bdc108e30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.251+0000 7f4be0e00640 1 --2- 172.21.15.18:0/3420623479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bdc104440 0x7f4bdc104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.251+0000 7f4be0e00640 1 -- 172.21.15.18:0/3420623479 >> 172.21.15.18:0/3420623479 conn(0x7f4bdc0ffc80 msgr2=0x7f4bdc1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.252+0000 7f4be0e00640 1 -- 172.21.15.18:0/3420623479 shutdown_connections 2024-04-16T17:16:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.252+0000 7f4be0e00640 1 -- 172.21.15.18:0/3420623479 wait complete. 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.253+0000 7f4be0e00640 1 Processor -- start 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.253+0000 7f4be0e00640 1 -- start start 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.253+0000 7f4be0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bdc104440 0x7f4bdc1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.253+0000 7f4be0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bdc109510 0x7f4bdc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.253+0000 7f4be0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc19a940 0x7f4bdc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.253+0000 7f4be0e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc114580 con 0x7f4bdc109510 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.253+0000 7f4be0e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc114400 con 0x7f4bdc19a940 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.253+0000 7f4be0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc114700 con 0x7f4bdc104440 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bd9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bdc109510 0x7f4bdc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bd9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bdc109510 0x7f4bdc1a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50438/0 (socket says 172.21.15.18:50438) 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bd9400640 1 -- 172.21.15.18:0/958808143 learned_addr learned my addr 172.21.15.18:0/958808143 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bda800640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc19a940 0x7f4bdc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bdc104440 0x7f4bdc1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bda800640 1 -- 172.21.15.18:0/958808143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bdc104440 msgr2=0x7f4bdc1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bda800640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bdc104440 0x7f4bdc1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bda800640 1 -- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bdc109510 msgr2=0x7f4bdc1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bda800640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bdc109510 0x7f4bdc1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bda800640 1 -- 172.21.15.18:0/958808143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bdc19b460 con 0x7f4bdc19a940 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.254+0000 7f4bda800640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc19a940 0x7f4bdc19adf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400e940 tx=0x7f4bd400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.255+0000 7f4bd2a00640 1 -- 172.21.15.18:0/958808143 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4004d20 con 0x7f4bdc19a940 2024-04-16T17:16:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.255+0000 7f4bd2a00640 1 -- 172.21.15.18:0/958808143 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd4005020 con 0x7f4bdc19a940 2024-04-16T17:16:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.255+0000 7f4bd2a00640 1 -- 172.21.15.18:0/958808143 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40116b0 con 0x7f4bdc19a940 2024-04-16T17:16:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.255+0000 7f4be0e00640 1 -- 172.21.15.18:0/958808143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bdc19b750 con 0x7f4bdc19a940 2024-04-16T17:16:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.255+0000 7f4be0e00640 1 -- 172.21.15.18:0/958808143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bdc1a7950 con 0x7f4bdc19a940 2024-04-16T17:16:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.258+0000 7f4be3b73740 1 Processor -- start 2024-04-16T17:16:42.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.260+0000 7f4bd2a00640 1 -- 172.21.15.18:0/958808143 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4bd40054d0 con 0x7f4bdc19a940 2024-04-16T17:16:42.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.260+0000 7f4be3b73740 1 -- start start 2024-04-16T17:16:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.260+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041388590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.260+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562041388ad0 0x562041391170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.260+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620413916b0 0x562041393aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4be3b73740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5620412ee120 con 0x5620413881b0 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4be3b73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562041294340 con 0x562041388ad0 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4be3b73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5620412ea0e0 con 0x5620413916b0 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041388590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562041388ad0 0x562041391170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562041388ad0 0x562041391170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54226/0 (socket says 172.21.15.18:54226) 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9e00640 1 -- 172.21.15.18:0/822441506 learned_addr learned my addr 172.21.15.18:0/822441506 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9400640 1 --2- 172.21.15.18:0/822441506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620413916b0 0x562041393aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9e00640 1 -- 172.21.15.18:0/822441506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620413916b0 msgr2=0x562041393aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/822441506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620413916b0 0x562041393aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9e00640 1 -- 172.21.15.18:0/822441506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 msgr2=0x562041388590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/822441506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041388590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.261+0000 7f4bd9e00640 1 -- 172.21.15.18:0/822441506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620410ca7d0 con 0x562041388ad0 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.262+0000 7f4bd2a00640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4bc00a3da0 0x7f4bc00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.262+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/822441506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562041388ad0 0x562041391170 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009c80 tx=0x7f4bcc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d589579581883d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.262+0000 7f4bd9400640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4bc00a3da0 0x7f4bc00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.262+0000 7f4bc7400640 1 -- 172.21.15.18:0/822441506 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc004220 con 0x562041388ad0 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.262+0000 7f4bd2a00640 1 -- 172.21.15.18:0/958808143 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f4bd40d0c20 con 0x7f4bdc19a940 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.262+0000 7f4bc7400640 1 -- 172.21.15.18:0/822441506 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bcc0043c0 con 0x562041388ad0 2024-04-16T17:16:42.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.262+0000 7f4bc7400640 1 -- 172.21.15.18:0/822441506 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc0198f0 con 0x562041388ad0 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 -- 172.21.15.18:0/822441506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562041388ad0 msgr2=0x562041391170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 --2- 172.21.15.18:0/822441506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562041388ad0 0x562041391170 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009c80 tx=0x7f4bcc011810 comp rx=0 tx=0).stop 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4bd9400640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4bc00a3da0 0x7f4bc00a6260 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc19c0e0 tx=0x7f4bc8005d20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 -- 172.21.15.18:0/822441506 shutdown_connections 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 --2- 172.21.15.18:0/822441506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5620413916b0 0x562041393aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 --2- 172.21.15.18:0/822441506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562041388ad0 0x562041391170 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 --2- 172.21.15.18:0/822441506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041388590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 -- 172.21.15.18:0/822441506 >> 172.21.15.18:0/822441506 conn(0x5620413828e0 msgr2=0x562041382cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 -- 172.21.15.18:0/822441506 shutdown_connections 2024-04-16T17:16:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.263+0000 7f4be3b73740 1 -- 172.21.15.18:0/822441506 wait complete. 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 1 -- start start 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 1 Processor -- start 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x56204138e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5620413944a0 con 0x5620413916b0 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5620412a7710 con 0x5620413881b0 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562041394620 con 0x562041388ad0 2024-04-16T17:16:42.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.264+0000 7f4be3b73740 10 client.0 _ll_register_callbacks cb 0x5620412a2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x56204138e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x56204138e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54228/0 (socket says 172.21.15.18:54228) 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 -- 172.21.15.18:0/3375082221 learned_addr learned my addr 172.21.15.18:0/3375082221 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bda800640 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9400640 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 -- 172.21.15.18:0/3375082221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 msgr2=0x56204138ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 -- 172.21.15.18:0/3375082221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 msgr2=0x56204138f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 -- 172.21.15.18:0/3375082221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620410003f0 con 0x5620413881b0 2024-04-16T17:16:42.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9400640 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x56204138e700 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc019470 tx=0x7f4bcc009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa467c44c9f9b526 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.265+0000 7f4bbbe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.266+0000 7f4bbbe00640 1 -- 172.21.15.18:0/3375082221 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc028070 con 0x5620413881b0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.266+0000 7f4bbbe00640 1 -- 172.21.15.18:0/3375082221 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f4bcc0046c0 con 0x5620413881b0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.266+0000 7f4be3b73740 1 -- 172.21.15.18:0/3375082221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bcc0043c0 con 0x5620413881b0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.266+0000 7f4bbbe00640 1 -- 172.21.15.18:0/3375082221 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4bcc034050 con 0x5620413881b0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.266+0000 7f4be3b73740 10 client.28862 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.266+0000 7f4be3b73740 1 -- 172.21.15.18:0/3375082221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562040fc00f0 con 0x5620413881b0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.266+0000 7f4bbbe00640 1 -- 172.21.15.18:0/3375082221 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bcc0043c0 con 0x5620413881b0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.266+0000 7f4bbbe00640 1 -- 172.21.15.18:0/3375082221 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4bcc0039f0 con 0x5620413881b0 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4be3b73740 10 client.28862 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4be3b73740 10 client.28862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4be3b73740 20 client.28862 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4be3b73740 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5620413a19a0 0x5620413a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4be3b73740 4 client.28862 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4be3b73740 1 -- 172.21.15.18:0/3375082221 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620412fbb40 con 0x5620413a19a0 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4bd9400640 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5620413a19a0 0x5620413a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4bd9400640 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5620413a19a0 0x5620413a3d90 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe076217357a94f server_cookie=c0290eac8e63141 in_seq=0 out_seq=0 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.267+0000 7f4bbbe00640 10 client.28862 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4bbbe00640 1 -- 172.21.15.18:0/3375082221 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620412fbb40 con 0x5620413a19a0 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4bbbe00640 10 client.28862 handle_command_reply: tid=0 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 client.28862 shutdown 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 -- 172.21.15.18:0/3375082221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 msgr2=0x56204138e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x56204138e700 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc019470 tx=0x7f4bcc009f60 comp rx=0 tx=0).stop 2024-04-16T17:16:42.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 -- 172.21.15.18:0/3375082221 shutdown_connections 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5620413a19a0 0x5620413a3d90 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 --2- 172.21.15.18:0/3375082221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x56204138e700 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4bbbe00640 0 client.28862 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.268+0000 7f4be3b73740 1 -- 172.21.15.18:0/3375082221 >> 172.21.15.18:0/3375082221 conn(0x5620413828e0 msgr2=0x562041384630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.269+0000 7f4be3b73740 1 -- 172.21.15.18:0/3375082221 shutdown_connections 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.269+0000 7f4be3b73740 1 -- 172.21.15.18:0/3375082221 wait complete. 2024-04-16T17:16:42.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.269+0000 7f4be3b73740 20 client.28862 trim_cache size 0 max 16384 2024-04-16T17:16:42.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4be3b73740 1 Processor -- start 2024-04-16T17:16:42.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4be3b73740 1 -- start start 2024-04-16T17:16:42.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041390e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204139d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413916b0 0x56204138d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4be3b73740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc1b41d0 con 0x5620413881b0 2024-04-16T17:16:42.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4be3b73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc1b1aa0 con 0x5620413916b0 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4be3b73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc1af910 con 0x562041388ad0 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041390e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041390e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50456/0 (socket says 172.21.15.18:50456) 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4bd9e00640 1 -- 172.21.15.18:0/1646298275 learned_addr learned my addr 172.21.15.18:0/1646298275 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.288+0000 7f4bda800640 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413916b0 0x56204138d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.289+0000 7f4bda800640 1 -- 172.21.15.18:0/1646298275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 msgr2=0x56204139d360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.289+0000 7f4bda800640 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204139d360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.289+0000 7f4bda800640 1 -- 172.21.15.18:0/1646298275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 msgr2=0x562041390e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.289+0000 7f4bda800640 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041390e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.289+0000 7f4bda800640 1 -- 172.21.15.18:0/1646298275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620413a4100 con 0x5620413916b0 2024-04-16T17:16:42.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.289+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041390e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:42.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.293+0000 7f4bda800640 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413916b0 0x56204138d860 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40cdf00 tx=0x7f4bd4005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9b9461efe72eda8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:42.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4bc6a00640 1 -- 172.21.15.18:0/1646298275 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd401f040 con 0x5620413916b0 2024-04-16T17:16:42.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4bc6a00640 1 -- 172.21.15.18:0/1646298275 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd40a53c0 con 0x5620413916b0 2024-04-16T17:16:42.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4bc6a00640 1 -- 172.21.15.18:0/1646298275 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd409bc50 con 0x5620413916b0 2024-04-16T17:16:42.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4be3b73740 1 -- 172.21.15.18:0/1646298275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413916b0 msgr2=0x56204138d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4be3b73740 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413916b0 0x56204138d860 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40cdf00 tx=0x7f4bd4005840 comp rx=0 tx=0).stop 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4be3b73740 1 -- 172.21.15.18:0/1646298275 shutdown_connections 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4be3b73740 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413916b0 0x56204138d860 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4be3b73740 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204139d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4be3b73740 1 --2- 172.21.15.18:0/1646298275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413881b0 0x562041390e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.294+0000 7f4be3b73740 1 -- 172.21.15.18:0/1646298275 >> 172.21.15.18:0/1646298275 conn(0x5620413828e0 msgr2=0x562041381040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.295+0000 7f4be3b73740 1 -- 172.21.15.18:0/1646298275 shutdown_connections 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.295+0000 7f4be3b73740 1 -- 172.21.15.18:0/1646298275 wait complete. 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.295+0000 7f4be3b73740 1 -- start start 2024-04-16T17:16:42.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.295+0000 7f4be3b73740 1 Processor -- start 2024-04-16T17:16:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x562041399a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4be3b73740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4be3b73740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc1aff60 con 0x5620413916b0 2024-04-16T17:16:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4be3b73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc1b13d0 con 0x5620413881b0 2024-04-16T17:16:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4be3b73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc1b0630 con 0x562041388ad0 2024-04-16T17:16:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4be3b73740 10 client.0 _ll_register_callbacks cb 0x7f4bcc009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x562041399a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4bda800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4bd9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x562041399a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54248/0 (socket says 172.21.15.18:54248) 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4bda800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50464/0 (socket says 172.21.15.18:50464) 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4bd9e00640 1 -- 172.21.15.18:0/523312678 learned_addr learned my addr 172.21.15.18:0/523312678 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.296+0000 7f4bd9400640 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bda800640 1 -- 172.21.15.18:0/523312678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 msgr2=0x56204138af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bda800640 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bda800640 1 -- 172.21.15.18:0/523312678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 msgr2=0x562041399a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bda800640 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x562041399a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bda800640 1 -- 172.21.15.18:0/523312678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56204138b9f0 con 0x5620413916b0 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bda800640 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138b4b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40cdf30 tx=0x7f4bd40a0bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6aea0cb4ac5b7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bbbe00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bbbe00640 1 -- 172.21.15.18:0/523312678 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd401f040 con 0x5620413916b0 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4bbbe00640 1 -- 172.21.15.18:0/523312678 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f4bd409bc50 con 0x5620413916b0 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.297+0000 7f4be3b73740 1 -- 172.21.15.18:0/523312678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd40a53c0 con 0x5620413916b0 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4bbbe00640 1 -- 172.21.15.18:0/523312678 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4bd409be50 con 0x5620413916b0 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4be3b73740 10 client.18555 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4be3b73740 1 -- 172.21.15.18:0/523312678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562041407af0 con 0x5620413916b0 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4bbbe00640 1 -- 172.21.15.18:0/523312678 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd40a53c0 con 0x5620413916b0 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4bbbe00640 1 -- 172.21.15.18:0/523312678 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4bd40b4020 con 0x5620413916b0 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4be3b73740 10 client.18555 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4be3b73740 10 client.18555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:42.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4be3b73740 20 client.18555 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4be3b73740 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562041408100 0x56204140a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4be3b73740 4 client.18555 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.298+0000 7f4be3b73740 1 -- 172.21.15.18:0/523312678 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620412f59b0 con 0x562041408100 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562041408100 0x56204140a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4bd9e00640 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562041408100 0x56204140a4f0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c032a3617bac2c server_cookie=5e469b8361fea5d0 in_seq=0 out_seq=0 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4bbbe00640 10 client.18555 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4bbbe00640 1 -- 172.21.15.18:0/523312678 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5620412f59b0 con 0x562041408100 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4bbbe00640 10 client.18555 handle_command_reply: tid=0 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 client.18555 shutdown 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 -- 172.21.15.18:0/523312678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 msgr2=0x56204138b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138b4b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40cdf30 tx=0x7f4bd40a0bc0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 -- 172.21.15.18:0/523312678 shutdown_connections 2024-04-16T17:16:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562041408100 0x56204140a4f0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5620413916b0 0x56204138b4b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562041388ad0 0x56204138af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 --2- 172.21.15.18:0/523312678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620413881b0 0x562041399a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.299+0000 7f4be3b73740 1 -- 172.21.15.18:0/523312678 >> 172.21.15.18:0/523312678 conn(0x5620413828e0 msgr2=0x56204138e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.300+0000 7f4bbbe00640 0 client.18555 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.300+0000 7f4be3b73740 1 -- 172.21.15.18:0/523312678 shutdown_connections 2024-04-16T17:16:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.300+0000 7f4be3b73740 1 -- 172.21.15.18:0/523312678 wait complete. 2024-04-16T17:16:42.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.300+0000 7f4be3b73740 20 client.18555 trim_cache size 0 max 16384 2024-04-16T17:16:42.307 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:16:42.307 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:16:42.307 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:16:42.307 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.300+0000 7f4be0e00640 1 -- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4bc00a3da0 msgr2=0x7f4bc00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.300+0000 7f4be0e00640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4bc00a3da0 0x7f4bc00a6260 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc19c0e0 tx=0x7f4bc8005d20 comp rx=0 tx=0).stop 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.300+0000 7f4be0e00640 1 -- 172.21.15.18:0/958808143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc19a940 msgr2=0x7f4bdc19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.300+0000 7f4be0e00640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc19a940 0x7f4bdc19adf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400e940 tx=0x7f4bd400ee00 comp rx=0 tx=0).stop 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.301+0000 7f4be0e00640 1 -- 172.21.15.18:0/958808143 shutdown_connections 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.301+0000 7f4be0e00640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4bc00a3da0 0x7f4bc00a6260 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.301+0000 7f4be0e00640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bdc19a940 0x7f4bdc19adf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.301+0000 7f4be0e00640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4bdc109510 0x7f4bdc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.301+0000 7f4be0e00640 1 --2- 172.21.15.18:0/958808143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bdc104440 0x7f4bdc1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.301+0000 7f4be0e00640 1 -- 172.21.15.18:0/958808143 >> 172.21.15.18:0/958808143 conn(0x7f4bdc0ffc80 msgr2=0x7f4bdc101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.301+0000 7f4be0e00640 1 -- 172.21.15.18:0/958808143 shutdown_connections 2024-04-16T17:16:42.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.301+0000 7f4be0e00640 1 -- 172.21.15.18:0/958808143 wait complete. 2024-04-16T17:16:42.321 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:16:42.321 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:16:42.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 -- 172.21.15.18:0/4068078697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c104350 msgr2=0x7f0f6c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 --2- 172.21.15.18:0/4068078697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c104350 0x7f0f6c104730 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009900 tx=0x7f0f5c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:42.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 -- 172.21.15.18:0/4068078697 shutdown_connections 2024-04-16T17:16:42.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 --2- 172.21.15.18:0/4068078697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6c109420 0x7f0f6c110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 --2- 172.21.15.18:0/4068078697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c104d00 0x7f0f6c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 --2- 172.21.15.18:0/4068078697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c104350 0x7f0f6c104730 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 -- 172.21.15.18:0/4068078697 >> 172.21.15.18:0/4068078697 conn(0x7f0f6c0ffb70 msgr2=0x7f0f6c101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:42.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 -- 172.21.15.18:0/4068078697 shutdown_connections 2024-04-16T17:16:42.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.516+0000 7f0f6b200640 1 -- 172.21.15.18:0/4068078697 wait complete. 2024-04-16T17:16:42.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.517+0000 7f0f6b200640 1 Processor -- start 2024-04-16T17:16:42.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f6b200640 1 -- start start 2024-04-16T17:16:42.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f6b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c104350 0x7f0f6c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f6b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6c109420 0x7f0f6c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f6b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c19a840 0x7f0f6c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f6b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c114500 con 0x7f0f6c109420 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f6b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c114380 con 0x7f0f6c19a840 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f6b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c114680 con 0x7f0f6c104350 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f69e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c104350 0x7f0f6c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f69e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c104350 0x7f0f6c1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37334/0 (socket says 172.21.15.18:37334) 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.518+0000 7f0f69e00640 1 -- 172.21.15.18:0/1061226480 learned_addr learned my addr 172.21.15.18:0/1061226480 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f6a800640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c19a840 0x7f0f6c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f69400640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6c109420 0x7f0f6c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f69e00640 1 -- 172.21.15.18:0/1061226480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c19a840 msgr2=0x7f0f6c19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f69e00640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c19a840 0x7f0f6c19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f69e00640 1 -- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6c109420 msgr2=0x7f0f6c1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f69e00640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6c109420 0x7f0f6c1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f69e00640 1 -- 172.21.15.18:0/1061226480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c19b3d0 con 0x7f0f6c104350 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f69400640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6c109420 0x7f0f6c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.519+0000 7f0f69e00640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c104350 0x7f0f6c1a06d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c02c240 tx=0x7f0f5c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.520+0000 7f0f62a00640 1 -- 172.21.15.18:0/1061226480 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c043020 con 0x7f0f6c104350 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.520+0000 7f0f62a00640 1 -- 172.21.15.18:0/1061226480 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f5c0054e0 con 0x7f0f6c104350 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.520+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c19b660 con 0x7f0f6c104350 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.520+0000 7f0f62a00640 1 -- 172.21.15.18:0/1061226480 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c003b00 con 0x7f0f6c104350 2024-04-16T17:16:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.520+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c1a7840 con 0x7f0f6c104350 2024-04-16T17:16:42.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.521+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c105820 con 0x7f0f6c104350 2024-04-16T17:16:42.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.524+0000 7f0f62a00640 1 -- 172.21.15.18:0/1061226480 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f5c046020 con 0x7f0f6c104350 2024-04-16T17:16:42.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.526+0000 7f0f62a00640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f500a3d10 0x7f0f500a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:42.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.527+0000 7f0f62a00640 1 -- 172.21.15.18:0/1061226480 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f0f5c0f2b70 con 0x7f0f6c104350 2024-04-16T17:16:42.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.527+0000 7f0f69400640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f500a3d10 0x7f0f500a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:42.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.527+0000 7f0f69400640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f500a3d10 0x7f0f500a61d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c10a9d0 tx=0x7f0f58005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:42.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.530+0000 7f0f62a00640 1 -- 172.21.15.18:0/1061226480 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f5c04d020 con 0x7f0f6c104350 2024-04-16T17:16:42.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 448638 3.58 MB/sec execute 535 sec latency 84.722 ms 2024-04-16T17:16:42.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.878+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f6c10c650 con 0x7f0f6c104350 2024-04-16T17:16:42.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.880+0000 7f0f62a00640 1 -- 172.21.15.18:0/1061226480 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11854 (secure 0 0 0) 0x7f0f5c0beeb0 con 0x7f0f6c104350 2024-04-16T17:16:42.880 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:16:42.880 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:16:00.629944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:16:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-16T17:16:42.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f500a3d10 msgr2=0x7f0f500a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f500a3d10 0x7f0f500a61d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c10a9d0 tx=0x7f0f58005f90 comp rx=0 tx=0).stop 2024-04-16T17:16:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c104350 msgr2=0x7f0f6c1a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c104350 0x7f0f6c1a06d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c02c240 tx=0x7f0f5c0042f0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 shutdown_connections 2024-04-16T17:16:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f500a3d10 0x7f0f500a61d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c19a840 0x7f0f6c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6c109420 0x7f0f6c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.881+0000 7f0f6b200640 1 --2- 172.21.15.18:0/1061226480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c104350 0x7f0f6c1a06d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.882+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 >> 172.21.15.18:0/1061226480 conn(0x7f0f6c0ffb70 msgr2=0x7f0f6c101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.882+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 shutdown_connections 2024-04-16T17:16:42.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:42.882+0000 7f0f6b200640 1 -- 172.21.15.18:0/1061226480 wait complete. 2024-04-16T17:16:42.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:16:43.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.092+0000 7fc14a800640 1 -- 172.21.15.18:0/3048771117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc144102a20 msgr2=0x7fc144106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.092+0000 7fc14a800640 1 --2- 172.21.15.18:0/3048771117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc144102a20 0x7fc144106a60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc130009900 tx=0x7fc13002bc40 comp rx=0 tx=0).stop 2024-04-16T17:16:43.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.093+0000 7fc14a800640 1 -- 172.21.15.18:0/3048771117 shutdown_connections 2024-04-16T17:16:43.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.093+0000 7fc14a800640 1 --2- 172.21.15.18:0/3048771117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc144107140 0x7fc14410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.093+0000 7fc14a800640 1 --2- 172.21.15.18:0/3048771117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc144102a20 0x7fc144106a60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.093+0000 7fc14a800640 1 --2- 172.21.15.18:0/3048771117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc144102070 0x7fc144102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.093+0000 7fc14a800640 1 -- 172.21.15.18:0/3048771117 >> 172.21.15.18:0/3048771117 conn(0x7fc1440fd950 msgr2=0x7fc1440ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.093+0000 7fc14a800640 1 -- 172.21.15.18:0/3048771117 shutdown_connections 2024-04-16T17:16:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.093+0000 7fc14a800640 1 -- 172.21.15.18:0/3048771117 wait complete. 2024-04-16T17:16:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc14a800640 1 Processor -- start 2024-04-16T17:16:43.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc14a800640 1 -- start start 2024-04-16T17:16:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc14a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc144102070 0x7fc144198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc14a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc144107140 0x7fc144198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc14a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14419cff0 0x7fc14419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc14a800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc144112200 con 0x7fc144102070 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc14a800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc144112080 con 0x7fc144107140 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc14a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc144112380 con 0x7fc14419cff0 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.094+0000 7fc149400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc144102070 0x7fc144198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14419cff0 0x7fc14419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14419cff0 0x7fc14419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37352/0 (socket says 172.21.15.18:37352) 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc144102070 0x7fc144198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50496/0 (socket says 172.21.15.18:50496) 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 -- 172.21.15.18:0/1392612437 learned_addr learned my addr 172.21.15.18:0/1392612437 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc148a00640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc144107140 0x7fc144198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 -- 172.21.15.18:0/1392612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc144107140 msgr2=0x7fc144198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc144107140 0x7fc144198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 -- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc144102070 msgr2=0x7fc144198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc144102070 0x7fc144198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 -- 172.21.15.18:0/1392612437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14419dae0 con 0x7fc14419cff0 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc148a00640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc144107140 0x7fc144198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.095+0000 7fc149e00640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14419cff0 0x7fc14419d4a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00efd0 tx=0x7fc13c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.096+0000 7fc142000640 1 -- 172.21.15.18:0/1392612437 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c003b70 con 0x7fc14419cff0 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.096+0000 7fc142000640 1 -- 172.21.15.18:0/1392612437 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13c003d10 con 0x7fc14419cff0 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.096+0000 7fc14a800640 1 -- 172.21.15.18:0/1392612437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14419ddd0 con 0x7fc14419cff0 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.096+0000 7fc142000640 1 -- 172.21.15.18:0/1392612437 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0109f0 con 0x7fc14419cff0 2024-04-16T17:16:43.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.096+0000 7fc14a800640 1 -- 172.21.15.18:0/1392612437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1441a5590 con 0x7fc14419cff0 2024-04-16T17:16:43.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.100+0000 7fc142000640 1 -- 172.21.15.18:0/1392612437 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc13c004020 con 0x7fc14419cff0 2024-04-16T17:16:43.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.100+0000 7fc14ea16740 1 Processor -- start 2024-04-16T17:16:43.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.100+0000 7fc14ea16740 1 -- start start 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af91052cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af910602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af91060820 0x55af91062c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc14ea16740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55af90fb8020 con 0x55af91060820 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc14ea16740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55af90f5e460 con 0x55af910528f0 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc14ea16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55af90fb3fe0 con 0x55af910532a0 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc148a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af91052cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc148a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af91052cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54292/0 (socket says 172.21.15.18:54292) 2024-04-16T17:16:43.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc148a00640 1 -- 172.21.15.18:0/3227334922 learned_addr learned my addr 172.21.15.18:0/3227334922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc149400640 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af910602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.101+0000 7fc148a00640 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af91060820 0x55af91062c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc149400640 1 -- 172.21.15.18:0/3227334922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 msgr2=0x55af91052cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc149400640 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af91052cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc149400640 1 -- 172.21.15.18:0/3227334922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af91060820 msgr2=0x55af91062c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc149400640 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af91060820 0x55af91062c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc149400640 1 -- 172.21.15.18:0/3227334922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af90f66040 con 0x55af910532a0 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc142000640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc12c0a3de0 0x7fc12c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc149400640 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af910602e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc138009c20 tx=0x7fc138011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a1ab581b69b609a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc142000640 1 -- 172.21.15.18:0/1392612437 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fc13c0d0fe0 con 0x7fc14419cff0 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.102+0000 7fc148a00640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc12c0a3de0 0x7fc12c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.103+0000 7fc136600640 1 -- 172.21.15.18:0/3227334922 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13801e070 con 0x55af910532a0 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.103+0000 7fc136600640 1 -- 172.21.15.18:0/3227334922 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138009dd0 con 0x55af910532a0 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.103+0000 7fc136600640 1 -- 172.21.15.18:0/3227334922 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380187a0 con 0x55af910532a0 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 -- 172.21.15.18:0/3227334922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 msgr2=0x55af910602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af910602e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc138009c20 tx=0x7fc138011810 comp rx=0 tx=0).stop 2024-04-16T17:16:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc148a00640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc12c0a3de0 0x7fc12c0a62a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc13002ca00 tx=0x7fc130045040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 -- 172.21.15.18:0/3227334922 shutdown_connections 2024-04-16T17:16:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af91060820 0x55af91062c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af910602e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3227334922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af91052cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 -- 172.21.15.18:0/3227334922 >> 172.21.15.18:0/3227334922 conn(0x55af9104d640 msgr2=0x55af9104da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 -- 172.21.15.18:0/3227334922 shutdown_connections 2024-04-16T17:16:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 -- 172.21.15.18:0/3227334922 wait complete. 2024-04-16T17:16:43.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 -- start start 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.104+0000 7fc14ea16740 1 Processor -- start 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.105+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af9105d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.105+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af9105dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.105+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af91060820 0x55af9105e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.105+0000 7fc14ea16740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55af910635e0 con 0x55af910528f0 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc14ea16740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55af90f71830 con 0x55af91060820 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc14ea16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55af91063760 con 0x55af910532a0 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af91060820 0x55af9105e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af9105d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af91060820 0x55af9105e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54306/0 (socket says 172.21.15.18:54306) 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149e00640 1 -- 172.21.15.18:0/3273301645 learned_addr learned my addr 172.21.15.18:0/3273301645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc148a00640 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af9105dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc14ea16740 10 client.0 _ll_register_callbacks cb 0x55af90f79150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149e00640 1 -- 172.21.15.18:0/3273301645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 msgr2=0x55af9105dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149e00640 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af9105dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149e00640 1 -- 172.21.15.18:0/3273301645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 msgr2=0x55af9105d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149e00640 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af9105d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149e00640 1 -- 172.21.15.18:0/3273301645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af90f75920 con 0x55af91060820 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.106+0000 7fc149400640 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af9105d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.107+0000 7fc149e00640 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af91060820 0x55af9105e320 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc13c09c810 tx=0x7fc13c09c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82661d2bf29b804 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.107+0000 7fc12aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.107+0000 7fc12aa00640 1 -- 172.21.15.18:0/3273301645 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0b6060 con 0x55af91060820 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.107+0000 7fc14ea16740 1 -- 172.21.15.18:0/3273301645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc138009dd0 con 0x55af91060820 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.107+0000 7fc12aa00640 1 -- 172.21.15.18:0/3273301645 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fc13c004310 con 0x55af91060820 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.107+0000 7fc12aa00640 1 -- 172.21.15.18:0/3273301645 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc13c09cb10 con 0x55af91060820 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.107+0000 7fc14ea16740 10 client.28886 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.107+0000 7fc14ea16740 1 -- 172.21.15.18:0/3273301645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55af90f437f0 con 0x55af91060820 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.108+0000 7fc12aa00640 1 -- 172.21.15.18:0/3273301645 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138009dd0 con 0x55af91060820 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.108+0000 7fc12aa00640 1 -- 172.21.15.18:0/3273301645 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc13c0b8070 con 0x55af91060820 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.108+0000 7fc14ea16740 10 client.28886 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.108+0000 7fc14ea16740 10 client.28886 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.108+0000 7fc14ea16740 20 client.28886 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:43.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.108+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55af910569d0 0x55af91056e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.108+0000 7fc14ea16740 4 client.28886 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:43.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.108+0000 7fc14ea16740 1 -- 172.21.15.18:0/3273301645 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af90fc5a40 con 0x55af910569d0 2024-04-16T17:16:43.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.109+0000 7fc149400640 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55af910569d0 0x55af91056e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.109+0000 7fc149400640 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55af910569d0 0x55af91056e50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d809e0dd00ef84f9 server_cookie=45f5ca907fb5cda1 in_seq=0 out_seq=0 2024-04-16T17:16:43.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.109+0000 7fc12aa00640 10 client.28886 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:43.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.110+0000 7fc12aa00640 1 -- 172.21.15.18:0/3273301645 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af90fc5a40 con 0x55af910569d0 2024-04-16T17:16:43.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.110+0000 7fc12aa00640 10 client.28886 handle_command_reply: tid=0 2024-04-16T17:16:43.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.110+0000 7fc14ea16740 1 client.28886 shutdown 2024-04-16T17:16:43.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.110+0000 7fc14ea16740 1 -- 172.21.15.18:0/3273301645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af91060820 msgr2=0x55af9105e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.110+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af91060820 0x55af9105e320 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc13c09c810 tx=0x7fc13c09c780 comp rx=0 tx=0).stop 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 1 -- 172.21.15.18:0/3273301645 shutdown_connections 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55af910569d0 0x55af91056e50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af91060820 0x55af9105e320 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af910532a0 0x55af9105dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 1 --2- 172.21.15.18:0/3273301645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af9105d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc12aa00640 0 client.28886 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 1 -- 172.21.15.18:0/3273301645 >> 172.21.15.18:0/3273301645 conn(0x55af9104d640 msgr2=0x55af9104f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 1 -- 172.21.15.18:0/3273301645 shutdown_connections 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 1 -- 172.21.15.18:0/3273301645 wait complete. 2024-04-16T17:16:43.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.111+0000 7fc14ea16740 20 client.28886 trim_cache size 0 max 16384 2024-04-16T17:16:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.128+0000 7fc14ea16740 1 Processor -- start 2024-04-16T17:16:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.128+0000 7fc14ea16740 1 -- start start 2024-04-16T17:16:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.128+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af9105ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.128+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910532a0 0x55af9105ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.128+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af9105b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.128+0000 7fc14ea16740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441afad0 con 0x55af910532a0 2024-04-16T17:16:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.128+0000 7fc14ea16740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441ad3a0 con 0x55af910528f0 2024-04-16T17:16:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.128+0000 7fc14ea16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441ab210 con 0x55af91060820 2024-04-16T17:16:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc148a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910532a0 0x55af9105ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc148a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910532a0 0x55af9105ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50520/0 (socket says 172.21.15.18:50520) 2024-04-16T17:16:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc148a00640 1 -- 172.21.15.18:0/1356888613 learned_addr learned my addr 172.21.15.18:0/1356888613 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc149400640 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af9105ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc149e00640 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af9105b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc149400640 1 -- 172.21.15.18:0/1356888613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 msgr2=0x55af9105b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc149400640 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af9105b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc149400640 1 -- 172.21.15.18:0/1356888613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910532a0 msgr2=0x55af9105ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc149400640 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910532a0 0x55af9105ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc149400640 1 -- 172.21.15.18:0/1356888613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af90cca3f0 con 0x55af910528f0 2024-04-16T17:16:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.129+0000 7fc149e00640 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af9105b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.130+0000 7fc149400640 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af9105ffd0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc138009bf0 tx=0x7fc138009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dd1df7a04066781 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.130+0000 7fc135c00640 1 -- 172.21.15.18:0/1356888613 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138035070 con 0x55af910528f0 2024-04-16T17:16:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.130+0000 7fc135c00640 1 -- 172.21.15.18:0/1356888613 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138005040 con 0x55af910528f0 2024-04-16T17:16:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.130+0000 7fc135c00640 1 -- 172.21.15.18:0/1356888613 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138019650 con 0x55af910528f0 2024-04-16T17:16:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 -- 172.21.15.18:0/1356888613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 msgr2=0x55af9105ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af9105ffd0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc138009bf0 tx=0x7fc138009ee0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 -- 172.21.15.18:0/1356888613 shutdown_connections 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af9105b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910532a0 0x55af9105ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 --2- 172.21.15.18:0/1356888613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910528f0 0x55af9105ffd0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 -- 172.21.15.18:0/1356888613 >> 172.21.15.18:0/1356888613 conn(0x55af9104d640 msgr2=0x55af9106c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 -- 172.21.15.18:0/1356888613 shutdown_connections 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 -- 172.21.15.18:0/1356888613 wait complete. 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.131+0000 7fc14ea16740 1 -- start start 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.132+0000 7fc14ea16740 1 Processor -- start 2024-04-16T17:16:43.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.132+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af910700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.132+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910532a0 0x55af910705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.132+0000 7fc14ea16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af91070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.132+0000 7fc14ea16740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441ab860 con 0x55af910528f0 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.132+0000 7fc14ea16740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441accd0 con 0x55af910532a0 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.132+0000 7fc14ea16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441abf30 con 0x55af91060820 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.132+0000 7fc14ea16740 10 client.0 _ll_register_callbacks cb 0x7fc13c009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc149400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af910700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc148a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910532a0 0x55af910705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc149e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af91070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc148a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910532a0 0x55af910705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54336/0 (socket says 172.21.15.18:54336) 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc148a00640 1 -- 172.21.15.18:0/2737090146 learned_addr learned my addr 172.21.15.18:0/2737090146 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc149e00640 1 -- 172.21.15.18:0/2737090146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910532a0 msgr2=0x55af910705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc149e00640 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910532a0 0x55af910705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc149e00640 1 -- 172.21.15.18:0/2737090146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 msgr2=0x55af910700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc149e00640 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af910700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc149e00640 1 -- 172.21.15.18:0/2737090146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af90d947d0 con 0x55af91060820 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc149e00640 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af91070b30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0a5250 tx=0x7fc13c004850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84cd3d7d6a032cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.133+0000 7fc12aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc12aa00640 1 -- 172.21.15.18:0/2737090146 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0b6040 con 0x55af91060820 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc12aa00640 1 -- 172.21.15.18:0/2737090146 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fc13c09bf60 con 0x55af91060820 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 1 -- 172.21.15.18:0/2737090146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc138005040 con 0x55af91060820 2024-04-16T17:16:43.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc12aa00640 1 -- 172.21.15.18:0/2737090146 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc13c0a0ec0 con 0x55af91060820 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 10 client.28744 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 1 -- 172.21.15.18:0/2737090146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55af910d2c80 con 0x55af91060820 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc12aa00640 1 -- 172.21.15.18:0/2737090146 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138005040 con 0x55af91060820 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc12aa00640 1 -- 172.21.15.18:0/2737090146 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc13c0b8020 con 0x55af91060820 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 10 client.28744 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 10 client.28744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 20 client.28744 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55af910d3210 0x55af910d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 4 client.28744 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:43.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.134+0000 7fc14ea16740 1 -- 172.21.15.18:0/2737090146 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af90fbf8b0 con 0x55af910d3210 2024-04-16T17:16:43.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.135+0000 7fc149400640 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55af910d3210 0x55af910d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.135+0000 7fc149400640 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55af910d3210 0x55af910d5600 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dab6a63eafe9f0d server_cookie=65095ffa00d45c26 in_seq=0 out_seq=0 2024-04-16T17:16:43.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.135+0000 7fc12aa00640 10 client.28744 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:43.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.135+0000 7fc12aa00640 1 -- 172.21.15.18:0/2737090146 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af90fbf8b0 con 0x55af910d3210 2024-04-16T17:16:43.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.135+0000 7fc12aa00640 10 client.28744 handle_command_reply: tid=0 2024-04-16T17:16:43.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.135+0000 7fc14ea16740 1 client.28744 shutdown 2024-04-16T17:16:43.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 -- 172.21.15.18:0/2737090146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 msgr2=0x55af91070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af91070b30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0a5250 tx=0x7fc13c004850 comp rx=0 tx=0).stop 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 -- 172.21.15.18:0/2737090146 shutdown_connections 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55af910d3210 0x55af910d5600 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af91060820 0x55af91070b30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55af910532a0 0x55af910705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 --2- 172.21.15.18:0/2737090146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55af910528f0 0x55af910700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 -- 172.21.15.18:0/2737090146 >> 172.21.15.18:0/2737090146 conn(0x55af9104d640 msgr2=0x55af9104be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc12aa00640 0 client.28744 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 -- 172.21.15.18:0/2737090146 shutdown_connections 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 1 -- 172.21.15.18:0/2737090146 wait complete. 2024-04-16T17:16:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.136+0000 7fc14ea16740 20 client.28744 trim_cache size 0 max 16384 2024-04-16T17:16:43.144 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:43.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 -- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc12c0a3de0 msgr2=0x7fc12c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc12c0a3de0 0x7fc12c0a62a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc13002ca00 tx=0x7fc130045040 comp rx=0 tx=0).stop 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 -- 172.21.15.18:0/1392612437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14419cff0 msgr2=0x7fc14419d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14419cff0 0x7fc14419d4a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00efd0 tx=0x7fc13c00c6a0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 -- 172.21.15.18:0/1392612437 shutdown_connections 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc12c0a3de0 0x7fc12c0a62a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14419cff0 0x7fc14419d4a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc144107140 0x7fc144198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 --2- 172.21.15.18:0/1392612437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc144102070 0x7fc144198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 -- 172.21.15.18:0/1392612437 >> 172.21.15.18:0/1392612437 conn(0x7fc1440fd950 msgr2=0x7fc1440fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 -- 172.21.15.18:0/1392612437 shutdown_connections 2024-04-16T17:16:43.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.137+0000 7fc14a800640 1 -- 172.21.15.18:0/1392612437 wait complete. 2024-04-16T17:16:43.154 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:16:43.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 -- 172.21.15.18:0/2068911228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c102070 msgr2=0x7f5e8c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 --2- 172.21.15.18:0/2068911228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c102070 0x7f5e8c102450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009900 tx=0x7f5e8002bc80 comp rx=0 tx=0).stop 2024-04-16T17:16:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 -- 172.21.15.18:0/2068911228 shutdown_connections 2024-04-16T17:16:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 --2- 172.21.15.18:0/2068911228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8c107140 0x7f5e8c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 --2- 172.21.15.18:0/2068911228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e8c102a20 0x7f5e8c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 --2- 172.21.15.18:0/2068911228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c102070 0x7f5e8c102450 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 -- 172.21.15.18:0/2068911228 >> 172.21.15.18:0/2068911228 conn(0x7f5e8c0fd950 msgr2=0x7f5e8c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 -- 172.21.15.18:0/2068911228 shutdown_connections 2024-04-16T17:16:43.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.336+0000 7f5e92400640 1 -- 172.21.15.18:0/2068911228 wait complete. 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.337+0000 7f5e92400640 1 Processor -- start 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.337+0000 7f5e92400640 1 -- start start 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.337+0000 7f5e92400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e8c102070 0x7f5e8c1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.337+0000 7f5e92400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c107140 0x7f5e8c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.337+0000 7f5e92400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8c19a8d0 0x7f5e8c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.337+0000 7f5e92400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1122f0 con 0x7f5e8c107140 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.337+0000 7f5e92400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c112170 con 0x7f5e8c102070 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.337+0000 7f5e92400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c112470 con 0x7f5e8c19a8d0 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c107140 0x7f5e8c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c107140 0x7f5e8c1a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50560/0 (socket says 172.21.15.18:50560) 2024-04-16T17:16:43.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 -- 172.21.15.18:0/3204539345 learned_addr learned my addr 172.21.15.18:0/3204539345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e91000640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e8c102070 0x7f5e8c1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 -- 172.21.15.18:0/3204539345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8c19a8d0 msgr2=0x7f5e8c19ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:16:43.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8c19a8d0 0x7f5e8c19ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 -- 172.21.15.18:0/3204539345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e8c102070 msgr2=0x7f5e8c1a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e8c102070 0x7f5e8c1a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 -- 172.21.15.18:0/3204539345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c19b3f0 con 0x7f5e8c107140 2024-04-16T17:16:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e91000640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e8c102070 0x7f5e8c1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.338+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c107140 0x7f5e8c1a0c60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800b470 tx=0x7f5e7800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.339+0000 7f5e89600640 1 -- 172.21.15.18:0/3204539345 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78015020 con 0x7f5e8c107140 2024-04-16T17:16:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.339+0000 7f5e92400640 1 -- 172.21.15.18:0/3204539345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c19b6e0 con 0x7f5e8c107140 2024-04-16T17:16:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.339+0000 7f5e89600640 1 -- 172.21.15.18:0/3204539345 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e78011d70 con 0x7f5e8c107140 2024-04-16T17:16:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.339+0000 7f5e89600640 1 -- 172.21.15.18:0/3204539345 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800f340 con 0x7f5e8c107140 2024-04-16T17:16:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.339+0000 7f5e92400640 1 -- 172.21.15.18:0/3204539345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1a7980 con 0x7f5e8c107140 2024-04-16T17:16:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.342+0000 7f5e965e0740 1 Processor -- start 2024-04-16T17:16:43.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e965e0740 1 -- start start 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f0984f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984f830 0x558f09857ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 0x558f0985a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e965e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558f097b4020 con 0x558f0984f830 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e965e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558f0975a460 con 0x558f09858410 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e965e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558f097affe0 con 0x558f0984ef10 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e91000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984f830 0x558f09857ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e91000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984f830 0x558f09857ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50564/0 (socket says 172.21.15.18:50564) 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.343+0000 7f5e91000640 1 -- 172.21.15.18:0/934392016 learned_addr learned my addr 172.21.15.18:0/934392016 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.344+0000 7f5e91000640 1 -- 172.21.15.18:0/934392016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 msgr2=0x558f0984f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.344+0000 7f5e91000640 1 --2- 172.21.15.18:0/934392016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f0984f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.344+0000 7f5e91000640 1 -- 172.21.15.18:0/934392016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 msgr2=0x558f0985a800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:16:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.344+0000 7f5e91000640 1 --2- 172.21.15.18:0/934392016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 0x558f0985a800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.344+0000 7f5e91000640 1 -- 172.21.15.18:0/934392016 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f09762040 con 0x558f0984f830 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.344+0000 7f5e89600640 1 -- 172.21.15.18:0/3204539345 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e78018020 con 0x7f5e8c107140 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.344+0000 7f5e91000640 1 --2- 172.21.15.18:0/934392016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984f830 0x558f09857ed0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009900 tx=0x7f5e8002c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291a157d3a89da97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.345+0000 7f5e7e000640 1 -- 172.21.15.18:0/934392016 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80045070 con 0x558f0984f830 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.345+0000 7f5e7e000640 1 -- 172.21.15.18:0/934392016 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e80004a70 con 0x558f0984f830 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.345+0000 7f5e7e000640 1 -- 172.21.15.18:0/934392016 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80005380 con 0x558f0984f830 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.345+0000 7f5e965e0740 1 -- 172.21.15.18:0/934392016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984f830 msgr2=0x558f09857ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.345+0000 7f5e965e0740 1 --2- 172.21.15.18:0/934392016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984f830 0x558f09857ed0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009900 tx=0x7f5e8002c620 comp rx=0 tx=0).stop 2024-04-16T17:16:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e965e0740 1 -- 172.21.15.18:0/934392016 shutdown_connections 2024-04-16T17:16:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e965e0740 1 --2- 172.21.15.18:0/934392016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 0x558f0985a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e965e0740 1 --2- 172.21.15.18:0/934392016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984f830 0x558f09857ed0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e965e0740 1 --2- 172.21.15.18:0/934392016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f0984f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e965e0740 1 -- 172.21.15.18:0/934392016 >> 172.21.15.18:0/934392016 conn(0x558f09849640 msgr2=0x558f09849a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e965e0740 1 -- 172.21.15.18:0/934392016 shutdown_connections 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e965e0740 1 -- 172.21.15.18:0/934392016 wait complete. 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e89600640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5e740a3c30 0x7f5e740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e89600640 1 -- 172.21.15.18:0/3204539345 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f5e780d0390 con 0x7f5e8c107140 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.346+0000 7f5e965e0740 1 -- start start 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e965e0740 1 Processor -- start 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e91000640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5e740a3c30 0x7f5e740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984ef10 0x558f09855550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984f830 0x558f09855a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 0x558f09855fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e965e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558f0985b1a0 con 0x558f0984ef10 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e965e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558f0976d830 con 0x558f09858410 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e965e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558f0985b320 con 0x558f0984f830 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.347+0000 7f5e965e0740 10 client.0 _ll_register_callbacks cb 0x558f09775150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.348+0000 7f5e91000640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5e740a3c30 0x7f5e740a60f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5e80004480 tx=0x7f5e8002bc80 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.348+0000 7f5e91000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984ef10 0x558f09855550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.348+0000 7f5e91000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984ef10 0x558f09855550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50568/0 (socket says 172.21.15.18:50568) 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.348+0000 7f5e91000640 1 -- 172.21.15.18:0/3854595710 learned_addr learned my addr 172.21.15.18:0/3854595710 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.348+0000 7f5e91a00640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 0x558f09855fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e91000640 1 -- 172.21.15.18:0/3854595710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984f830 msgr2=0x558f09855a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984f830 0x558f09855a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e91000640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984f830 0x558f09855a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e91000640 1 -- 172.21.15.18:0/3854595710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 msgr2=0x558f09855fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e91000640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 0x558f09855fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e91000640 1 -- 172.21.15.18:0/3854595710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f09771920 con 0x558f0984ef10 2024-04-16T17:16:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e91a00640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 0x558f09855fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984f830 0x558f09855a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:16:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e91000640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984ef10 0x558f09855550 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x558f0984e9e0 tx=0x7f5e80005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3fd353d39770e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.349+0000 7f5e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:16:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.350+0000 7f5e72a00640 1 -- 172.21.15.18:0/3854595710 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8004e070 con 0x558f0984ef10 2024-04-16T17:16:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.350+0000 7f5e72a00640 1 -- 172.21.15.18:0/3854595710 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f5e800346e0 con 0x558f0984ef10 2024-04-16T17:16:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.350+0000 7f5e965e0740 1 -- 172.21.15.18:0/3854595710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e80004a70 con 0x558f0984ef10 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.350+0000 7f5e72a00640 1 -- 172.21.15.18:0/3854595710 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5e80045040 con 0x558f0984ef10 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.350+0000 7f5e965e0740 10 client.18591 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.350+0000 7f5e965e0740 1 -- 172.21.15.18:0/3854595710 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558f0973f7f0 con 0x558f0984ef10 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.350+0000 7f5e72a00640 1 -- 172.21.15.18:0/3854595710 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e80004a70 con 0x558f0984ef10 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.350+0000 7f5e72a00640 1 -- 172.21.15.18:0/3854595710 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5e80055ab0 con 0x558f0984ef10 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.351+0000 7f5e965e0740 10 client.18591 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.351+0000 7f5e965e0740 10 client.18591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.351+0000 7f5e965e0740 20 client.18591 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:43.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.351+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x558f098686b0 0x558f0986aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.351+0000 7f5e965e0740 4 client.18591 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.351+0000 7f5e965e0740 1 -- 172.21.15.18:0/3854595710 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f097c1a40 con 0x558f098686b0 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.351+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x558f098686b0 0x558f0986aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.352+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x558f098686b0 0x558f0986aaa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76e28cf64ba0a4f2 server_cookie=5274be1b13081ffb in_seq=0 out_seq=0 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.352+0000 7f5e72a00640 10 client.18591 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.352+0000 7f5e72a00640 1 -- 172.21.15.18:0/3854595710 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f5e7807ae50 con 0x558f098686b0 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.352+0000 7f5e72a00640 10 client.18591 handle_command_reply: tid=0 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.352+0000 7f5e965e0740 1 client.18591 shutdown 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 -- 172.21.15.18:0/3854595710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984ef10 msgr2=0x558f09855550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984ef10 0x558f09855550 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x558f0984e9e0 tx=0x7f5e80005fd0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 -- 172.21.15.18:0/3854595710 shutdown_connections 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x558f098686b0 0x558f0986aaa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09858410 0x558f09855fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984f830 0x558f09855a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3854595710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f0984ef10 0x558f09855550 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e72a00640 0 client.18591 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 -- 172.21.15.18:0/3854595710 >> 172.21.15.18:0/3854595710 conn(0x558f09849640 msgr2=0x558f0984b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 -- 172.21.15.18:0/3854595710 shutdown_connections 2024-04-16T17:16:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 1 -- 172.21.15.18:0/3854595710 wait complete. 2024-04-16T17:16:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.353+0000 7f5e965e0740 20 client.18591 trim_cache size 0 max 16384 2024-04-16T17:16:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.368+0000 7f5e965e0740 1 Processor -- start 2024-04-16T17:16:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.368+0000 7f5e965e0740 1 -- start start 2024-04-16T17:16:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.368+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f09857c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.368+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09864090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.368+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f09870100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.368+0000 7f5e965e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1b4040 con 0x558f09858410 2024-04-16T17:16:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.368+0000 7f5e965e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1b1910 con 0x558f0984f830 2024-04-16T17:16:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.368+0000 7f5e965e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1af780 con 0x558f0984ef10 2024-04-16T17:16:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09864090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e91a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f09870100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09864090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54362/0 (socket says 172.21.15.18:54362) 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e8be00640 1 -- 172.21.15.18:0/208103213 learned_addr learned my addr 172.21.15.18:0/208103213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e91000640 1 --2- 172.21.15.18:0/208103213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f09857c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e8be00640 1 -- 172.21.15.18:0/208103213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 msgr2=0x558f09857c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e8be00640 1 --2- 172.21.15.18:0/208103213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f09857c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e8be00640 1 -- 172.21.15.18:0/208103213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 msgr2=0x558f09870100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e8be00640 1 --2- 172.21.15.18:0/208103213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f09870100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.369+0000 7f5e8be00640 1 -- 172.21.15.18:0/208103213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f094c63f0 con 0x558f0984f830 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.370+0000 7f5e8be00640 1 --2- 172.21.15.18:0/208103213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09864090 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5e780093f0 tx=0x7f5e78010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2e915a83d91c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.370+0000 7f5e7d600640 1 -- 172.21.15.18:0/208103213 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7809a0d0 con 0x558f0984f830 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.370+0000 7f5e7d600640 1 -- 172.21.15.18:0/208103213 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558f097c1a40 con 0x558f0984f830 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.370+0000 7f5e7d600640 1 -- 172.21.15.18:0/208103213 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780986a0 con 0x558f0984f830 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 -- 172.21.15.18:0/208103213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 msgr2=0x558f09864090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 --2- 172.21.15.18:0/208103213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09864090 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5e780093f0 tx=0x7f5e78010890 comp rx=0 tx=0).stop 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 -- 172.21.15.18:0/208103213 shutdown_connections 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 --2- 172.21.15.18:0/208103213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f09870100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 --2- 172.21.15.18:0/208103213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09864090 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 --2- 172.21.15.18:0/208103213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f09857c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 -- 172.21.15.18:0/208103213 >> 172.21.15.18:0/208103213 conn(0x558f09849640 msgr2=0x558f09855da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 -- 172.21.15.18:0/208103213 shutdown_connections 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 -- 172.21.15.18:0/208103213 wait complete. 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 -- start start 2024-04-16T17:16:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.371+0000 7f5e965e0740 1 Processor -- start 2024-04-16T17:16:43.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f098605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09860ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e965e0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f0985d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e965e0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1afdd0 con 0x558f09858410 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e965e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1b1240 con 0x558f0984f830 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e965e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1b04a0 con 0x558f0984ef10 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e965e0740 10 client.0 _ll_register_callbacks cb 0x7f5e800095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09860ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e91a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f0985d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09860ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54364/0 (socket says 172.21.15.18:54364) 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e8be00640 1 -- 172.21.15.18:0/3763751588 learned_addr learned my addr 172.21.15.18:0/3763751588 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.372+0000 7f5e91000640 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f098605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e91000640 1 -- 172.21.15.18:0/3763751588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 msgr2=0x558f09860ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e91000640 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09860ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e91000640 1 -- 172.21.15.18:0/3763751588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 msgr2=0x558f0985d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e91000640 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f0985d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e91000640 1 -- 172.21.15.18:0/3763751588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f095907d0 con 0x558f0984ef10 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e91a00640 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f0985d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09860ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e91000640 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f098605a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5e800096c0 tx=0x7f5e80005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2be6e0325577566e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.373+0000 7f5e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.374+0000 7f5e72a00640 1 -- 172.21.15.18:0/3763751588 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80065070 con 0x558f0984ef10 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.374+0000 7f5e72a00640 1 -- 172.21.15.18:0/3763751588 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f5e8004aa70 con 0x558f0984ef10 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.374+0000 7f5e965e0740 1 -- 172.21.15.18:0/3763751588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f097c1a40 con 0x558f0984ef10 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.374+0000 7f5e72a00640 1 -- 172.21.15.18:0/3763751588 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5e8004e040 con 0x558f0984ef10 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.374+0000 7f5e965e0740 10 client.28750 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.374+0000 7f5e965e0740 1 -- 172.21.15.18:0/3763751588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558f098cf3e0 con 0x558f0984ef10 2024-04-16T17:16:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.374+0000 7f5e72a00640 1 -- 172.21.15.18:0/3763751588 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558f097c1a40 con 0x558f0984ef10 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.375+0000 7f5e72a00640 1 -- 172.21.15.18:0/3763751588 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5e80055960 con 0x558f0984ef10 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.375+0000 7f5e965e0740 10 client.28750 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.375+0000 7f5e965e0740 10 client.28750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.375+0000 7f5e965e0740 20 client.28750 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.375+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x558f098cfaf0 0x558f098d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.375+0000 7f5e965e0740 4 client.28750 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.375+0000 7f5e965e0740 1 -- 172.21.15.18:0/3763751588 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f097bb8b0 con 0x558f098cfaf0 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.375+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x558f098cfaf0 0x558f098d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e8be00640 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x558f098cfaf0 0x558f098d1ee0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3c1a6956e3b1b3b server_cookie=99b98366dde01a00 in_seq=0 out_seq=0 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e72a00640 10 client.28750 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e72a00640 1 -- 172.21.15.18:0/3763751588 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f097bb8b0 con 0x558f098cfaf0 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e72a00640 10 client.28750 handle_command_reply: tid=0 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 client.28750 shutdown 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 -- 172.21.15.18:0/3763751588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 msgr2=0x558f098605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f098605a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5e800096c0 tx=0x7f5e80005e50 comp rx=0 tx=0).stop 2024-04-16T17:16:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 -- 172.21.15.18:0/3763751588 shutdown_connections 2024-04-16T17:16:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x558f098cfaf0 0x558f098d1ee0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558f09858410 0x558f0985d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f0984f830 0x558f09860ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 --2- 172.21.15.18:0/3763751588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558f0984ef10 0x558f098605a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e965e0740 1 -- 172.21.15.18:0/3763751588 >> 172.21.15.18:0/3763751588 conn(0x558f09849640 msgr2=0x558f09847e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.376+0000 7f5e72a00640 0 client.28750 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:16:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.377+0000 7f5e965e0740 1 -- 172.21.15.18:0/3763751588 shutdown_connections 2024-04-16T17:16:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.377+0000 7f5e965e0740 1 -- 172.21.15.18:0/3763751588 wait complete. 2024-04-16T17:16:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.377+0000 7f5e965e0740 20 client.28750 trim_cache size 0 max 16384 2024-04-16T17:16:43.381 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 -- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5e740a3c30 msgr2=0x7f5e740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5e740a3c30 0x7f5e740a60f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5e80004480 tx=0x7f5e8002bc80 comp rx=0 tx=0).stop 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 -- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c107140 msgr2=0x7f5e8c1a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c107140 0x7f5e8c1a0c60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800b470 tx=0x7f5e7800b930 comp rx=0 tx=0).stop 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 -- 172.21.15.18:0/3204539345 shutdown_connections 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5e740a3c30 0x7f5e740a60f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8c19a8d0 0x7f5e8c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e8c107140 0x7f5e8c1a0c60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 --2- 172.21.15.18:0/3204539345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e8c102070 0x7f5e8c1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.378+0000 7f5e92400640 1 -- 172.21.15.18:0/3204539345 >> 172.21.15.18:0/3204539345 conn(0x7f5e8c0fd950 msgr2=0x7f5e8c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.379+0000 7f5e92400640 1 -- 172.21.15.18:0/3204539345 shutdown_connections 2024-04-16T17:16:43.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.379+0000 7f5e92400640 1 -- 172.21.15.18:0/3204539345 wait complete. 2024-04-16T17:16:43.399 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:16:43.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.600+0000 7f6f6be00640 1 -- 172.21.15.18:0/26324680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 msgr2=0x7f6f6c10ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.600+0000 7f6f6be00640 1 --2- 172.21.15.18:0/26324680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 0x7f6f6c10ed10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009860 tx=0x7f6f5c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:16:43.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.600+0000 7f6f6be00640 1 -- 172.21.15.18:0/26324680 shutdown_connections 2024-04-16T17:16:43.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.600+0000 7f6f6be00640 1 --2- 172.21.15.18:0/26324680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 0x7f6f6c10ed10 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.600+0000 7f6f6be00640 1 --2- 172.21.15.18:0/26324680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c102a20 0x7f6f6c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.600+0000 7f6f6be00640 1 --2- 172.21.15.18:0/26324680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f6c102070 0x7f6f6c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.600+0000 7f6f6be00640 1 -- 172.21.15.18:0/26324680 >> 172.21.15.18:0/26324680 conn(0x7f6f6c0fd950 msgr2=0x7f6f6c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.601+0000 7f6f6be00640 1 -- 172.21.15.18:0/26324680 shutdown_connections 2024-04-16T17:16:43.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.601+0000 7f6f6be00640 1 -- 172.21.15.18:0/26324680 wait complete. 2024-04-16T17:16:43.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.601+0000 7f6f6be00640 1 Processor -- start 2024-04-16T17:16:43.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6be00640 1 -- start start 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c102070 0x7f6f6c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 0x7f6f6c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f6c19a780 0x7f6f6c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c112200 con 0x7f6f6c19a780 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c112080 con 0x7f6f6c107140 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c112380 con 0x7f6f6c102070 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 0x7f6f6c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 0x7f6f6c1a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54378/0 (socket says 172.21.15.18:54378) 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6a000640 1 -- 172.21.15.18:0/3235381998 learned_addr learned my addr 172.21.15.18:0/3235381998 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.602+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c102070 0x7f6f6c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.603+0000 7f6f6b400640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f6c19a780 0x7f6f6c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.603+0000 7f6f6a000640 1 -- 172.21.15.18:0/3235381998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c102070 msgr2=0x7f6f6c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.603+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c102070 0x7f6f6c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.603+0000 7f6f6a000640 1 -- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f6c19a780 msgr2=0x7f6f6c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.603+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f6c19a780 0x7f6f6c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.603+0000 7f6f6a000640 1 -- 172.21.15.18:0/3235381998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c19b360 con 0x7f6f6c107140 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.603+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 0x7f6f6c1a0b70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400e9f0 tx=0x7f6f5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.604+0000 7f6f63400640 1 -- 172.21.15.18:0/3235381998 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400ccc0 con 0x7f6f6c107140 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.604+0000 7f6f63400640 1 -- 172.21.15.18:0/3235381998 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f54005500 con 0x7f6f6c107140 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.604+0000 7f6f6be00640 1 -- 172.21.15.18:0/3235381998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c1a7440 con 0x7f6f6c107140 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.604+0000 7f6f63400640 1 -- 172.21.15.18:0/3235381998 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54010690 con 0x7f6f6c107140 2024-04-16T17:16:43.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.604+0000 7f6f6be00640 1 -- 172.21.15.18:0/3235381998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c1a77f0 con 0x7f6f6c107140 2024-04-16T17:16:43.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.608+0000 7f6f747b4740 1 Processor -- start 2024-04-16T17:16:43.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.608+0000 7f6f747b4740 1 -- start start 2024-04-16T17:16:43.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f63400640 1 -- 172.21.15.18:0/3235381998 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f540046c0 con 0x7f6f6c107140 2024-04-16T17:16:43.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117af10 0x55f2d117b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117b830 0x55f2d1183ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 0x55f2d1186800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f747b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f2d10e0020 con 0x55f2d117b830 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f747b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f2d1086460 con 0x55f2d117af10 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f747b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f2d10dbfe0 con 0x55f2d1184410 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117af10 0x55f2d117b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f6aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117b830 0x55f2d1183ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117af10 0x55f2d117b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54388/0 (socket says 172.21.15.18:54388) 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f6b400640 1 -- 172.21.15.18:0/1582428041 learned_addr learned my addr 172.21.15.18:0/1582428041 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.609+0000 7f6f6a000640 1 --2- 172.21.15.18:0/1582428041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 0x55f2d1186800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f6b400640 1 -- 172.21.15.18:0/1582428041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 msgr2=0x55f2d1186800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f6b400640 1 --2- 172.21.15.18:0/1582428041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 0x55f2d1186800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f6b400640 1 -- 172.21.15.18:0/1582428041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117b830 msgr2=0x55f2d1183ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f6b400640 1 --2- 172.21.15.18:0/1582428041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117b830 0x55f2d1183ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f6b400640 1 -- 172.21.15.18:0/1582428041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d108e040 con 0x55f2d117af10 2024-04-16T17:16:43.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f6b400640 1 --2- 172.21.15.18:0/1582428041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117af10 0x55f2d117b2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009860 tx=0x7f6f5c005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0527f66af0e0fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f5be00640 1 -- 172.21.15.18:0/1582428041 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c039040 con 0x55f2d117af10 2024-04-16T17:16:43.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f5be00640 1 -- 172.21.15.18:0/1582428041 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5c034cb0 con 0x55f2d117af10 2024-04-16T17:16:43.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.610+0000 7f6f5be00640 1 -- 172.21.15.18:0/1582428041 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0052a0 con 0x55f2d117af10 2024-04-16T17:16:43.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.611+0000 7f6f63400640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6f500a3da0 0x7f6f500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.611+0000 7f6f63400640 1 -- 172.21.15.18:0/3235381998 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f6f540d0f10 con 0x7f6f6c107140 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.611+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6f500a3da0 0x7f6f500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.612+0000 7f6f747b4740 1 -- 172.21.15.18:0/1582428041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117af10 msgr2=0x55f2d117b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.612+0000 7f6f747b4740 1 --2- 172.21.15.18:0/1582428041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117af10 0x55f2d117b2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009860 tx=0x7f6f5c005da0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.612+0000 7f6f747b4740 1 -- 172.21.15.18:0/1582428041 shutdown_connections 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.612+0000 7f6f747b4740 1 --2- 172.21.15.18:0/1582428041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 0x55f2d1186800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.612+0000 7f6f747b4740 1 --2- 172.21.15.18:0/1582428041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117b830 0x55f2d1183ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.612+0000 7f6f747b4740 1 --2- 172.21.15.18:0/1582428041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117af10 0x55f2d117b2f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.612+0000 7f6f747b4740 1 -- 172.21.15.18:0/1582428041 >> 172.21.15.18:0/1582428041 conn(0x55f2d1175640 msgr2=0x55f2d1175a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.613+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6f500a3da0 0x7f6f500a6260 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55f2d117a9e0 tx=0x7f6f64010040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.613+0000 7f6f747b4740 1 -- 172.21.15.18:0/1582428041 shutdown_connections 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.613+0000 7f6f747b4740 1 -- 172.21.15.18:0/1582428041 wait complete. 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.613+0000 7f6f747b4740 1 -- start start 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.614+0000 7f6f747b4740 1 Processor -- start 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.614+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d11814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.614+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117b830 0x55f2d11819e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.614+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d1184410 0x55f2d1181f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.614+0000 7f6f747b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f2d11871a0 con 0x55f2d117af10 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.614+0000 7f6f747b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f2d1099830 con 0x55f2d1184410 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.614+0000 7f6f747b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f2d1187320 con 0x55f2d117b830 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f747b4740 10 client.0 _ll_register_callbacks cb 0x55f2d10a1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:43.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d1184410 0x55f2d1181f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d1184410 0x55f2d1181f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54390/0 (socket says 172.21.15.18:54390) 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6b400640 1 -- 172.21.15.18:0/3180472667 learned_addr learned my addr 172.21.15.18:0/3180472667 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117b830 0x55f2d11819e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d11814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6a000640 1 -- 172.21.15.18:0/3180472667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d1184410 msgr2=0x55f2d1181f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d1184410 0x55f2d1181f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6a000640 1 -- 172.21.15.18:0/3180472667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 msgr2=0x55f2d11814a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d11814a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6a000640 1 -- 172.21.15.18:0/3180472667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d109d920 con 0x55f2d117b830 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6b400640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d1184410 0x55f2d1181f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.615+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d11814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.616+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117b830 0x55f2d11819e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6f540ce1f0 tx=0x7f6f54098df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756b7ba4f158ed62 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.616+0000 7f6f58c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:43.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.616+0000 7f6f58c00640 1 -- 172.21.15.18:0/3180472667 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f540056b0 con 0x55f2d117b830 2024-04-16T17:16:43.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.616+0000 7f6f747b4740 1 -- 172.21.15.18:0/3180472667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5c034cb0 con 0x55f2d117b830 2024-04-16T17:16:43.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.616+0000 7f6f58c00640 1 -- 172.21.15.18:0/3180472667 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f6f540a5220 con 0x55f2d117b830 2024-04-16T17:16:43.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.616+0000 7f6f58c00640 1 -- 172.21.15.18:0/3180472667 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6f540ce670 con 0x55f2d117b830 2024-04-16T17:16:43.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.617+0000 7f6f747b4740 10 client.28762 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:43.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.617+0000 7f6f747b4740 1 -- 172.21.15.18:0/3180472667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f2d106b7f0 con 0x55f2d117b830 2024-04-16T17:16:43.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.617+0000 7f6f58c00640 1 -- 172.21.15.18:0/3180472667 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5c034cb0 con 0x55f2d117b830 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.617+0000 7f6f58c00640 1 -- 172.21.15.18:0/3180472667 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6f540d4020 con 0x55f2d117b830 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.618+0000 7f6f747b4740 10 client.28762 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.618+0000 7f6f747b4740 10 client.28762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.618+0000 7f6f747b4740 20 client.28762 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.618+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f2d1194820 0x55f2d1196c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.618+0000 7f6f747b4740 4 client.28762 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.618+0000 7f6f747b4740 1 -- 172.21.15.18:0/3180472667 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2d10eda40 con 0x55f2d1194820 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.618+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f2d1194820 0x55f2d1196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.619+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f2d1194820 0x55f2d1196c30 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf4e313e4fd3e1c server_cookie=9e02430a54f82e28 in_seq=0 out_seq=0 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.619+0000 7f6f58c00640 10 client.28762 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:43.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f58c00640 1 -- 172.21.15.18:0/3180472667 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2d10eda40 con 0x55f2d1194820 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f58c00640 10 client.28762 handle_command_reply: tid=0 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 client.28762 shutdown 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 -- 172.21.15.18:0/3180472667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117b830 msgr2=0x55f2d11819e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117b830 0x55f2d11819e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6f540ce1f0 tx=0x7f6f54098df0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 -- 172.21.15.18:0/3180472667 shutdown_connections 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f2d1194820 0x55f2d1196c30 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d1184410 0x55f2d1181f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117b830 0x55f2d11819e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3180472667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d11814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f58c00640 0 client.28762 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 -- 172.21.15.18:0/3180472667 >> 172.21.15.18:0/3180472667 conn(0x55f2d1175640 msgr2=0x55f2d1177390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 -- 172.21.15.18:0/3180472667 shutdown_connections 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 1 -- 172.21.15.18:0/3180472667 wait complete. 2024-04-16T17:16:43.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.620+0000 7f6f747b4740 20 client.28762 trim_cache size 0 max 16384 2024-04-16T17:16:43.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.635+0000 7f6f747b4740 1 Processor -- start 2024-04-16T17:16:43.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.635+0000 7f6f747b4740 1 -- start start 2024-04-16T17:16:43.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.635+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117af10 0x55f2d1183bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.635+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d1190090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.635+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d1184410 0x55f2d1177180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.635+0000 7f6f747b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1b3e80 con 0x55f2d1184410 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.635+0000 7f6f747b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1b1750 con 0x55f2d117b830 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.635+0000 7f6f747b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1af5c0 con 0x55f2d117af10 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d1190090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d1190090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54392/0 (socket says 172.21.15.18:54392) 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6a000640 1 -- 172.21.15.18:0/3752512268 learned_addr learned my addr 172.21.15.18:0/3752512268 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117af10 0x55f2d1183bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6a000640 1 -- 172.21.15.18:0/3752512268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117af10 msgr2=0x55f2d1183bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117af10 0x55f2d1183bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6a000640 1 -- 172.21.15.18:0/3752512268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d1184410 msgr2=0x55f2d1177180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:16:43.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6b400640 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d1184410 0x55f2d1177180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d1184410 0x55f2d1177180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.636+0000 7f6f6a000640 1 -- 172.21.15.18:0/3752512268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d0df23f0 con 0x55f2d117b830 2024-04-16T17:16:43.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.637+0000 7f6f6a000640 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d1190090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400c6a0 tx=0x7f6f5409af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e80aa04885982e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.637+0000 7f6f5b400640 1 -- 172.21.15.18:0/3752512268 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5409be90 con 0x55f2d117b830 2024-04-16T17:16:43.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.637+0000 7f6f5b400640 1 -- 172.21.15.18:0/3752512268 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5409d140 con 0x55f2d117b830 2024-04-16T17:16:43.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.637+0000 7f6f5b400640 1 -- 172.21.15.18:0/3752512268 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5409c580 con 0x55f2d117b830 2024-04-16T17:16:43.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 -- 172.21.15.18:0/3752512268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 msgr2=0x55f2d1190090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d1190090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400c6a0 tx=0x7f6f5409af50 comp rx=0 tx=0).stop 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 -- 172.21.15.18:0/3752512268 shutdown_connections 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d1184410 0x55f2d1177180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d1190090 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 --2- 172.21.15.18:0/3752512268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d117af10 0x55f2d1183bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 -- 172.21.15.18:0/3752512268 >> 172.21.15.18:0/3752512268 conn(0x55f2d1175640 msgr2=0x55f2d1176a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 -- 172.21.15.18:0/3752512268 shutdown_connections 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 -- 172.21.15.18:0/3752512268 wait complete. 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.638+0000 7f6f747b4740 1 -- start start 2024-04-16T17:16:43.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.639+0000 7f6f747b4740 1 Processor -- start 2024-04-16T17:16:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.639+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d118c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.639+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d118ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.639+0000 7f6f747b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 0x55f2d1187a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.639+0000 7f6f747b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1afc10 con 0x55f2d117af10 2024-04-16T17:16:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.639+0000 7f6f747b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1b1080 con 0x55f2d117b830 2024-04-16T17:16:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.639+0000 7f6f747b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1b02e0 con 0x55f2d1184410 2024-04-16T17:16:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.639+0000 7f6f747b4740 10 client.0 _ll_register_callbacks cb 0x7f6f540096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d118ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d118ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54400/0 (socket says 172.21.15.18:54400) 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 -- 172.21.15.18:0/2128792660 learned_addr learned my addr 172.21.15.18:0/2128792660 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6b400640 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 0x55f2d1187a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d118c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 -- 172.21.15.18:0/2128792660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 msgr2=0x55f2d1187a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 0x55f2d1187a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 -- 172.21.15.18:0/2128792660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 msgr2=0x55f2d118c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d118c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 -- 172.21.15.18:0/2128792660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d0ebc7d0 con 0x55f2d117b830 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d118c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f6a000640 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d118ca80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400e9c0 tx=0x7f6f540a55c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e864ac9ab170b4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.640+0000 7f6f58c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.641+0000 7f6f58c00640 1 -- 172.21.15.18:0/2128792660 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f540ce680 con 0x55f2d117b830 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.641+0000 7f6f58c00640 1 -- 172.21.15.18:0/2128792660 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f6f54098990 con 0x55f2d117b830 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.641+0000 7f6f747b4740 1 -- 172.21.15.18:0/2128792660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5409d140 con 0x55f2d117b830 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.641+0000 7f6f58c00640 1 -- 172.21.15.18:0/2128792660 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6f540ad050 con 0x55f2d117b830 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.641+0000 7f6f747b4740 10 client.28922 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.641+0000 7f6f747b4740 1 -- 172.21.15.18:0/2128792660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f2d1202fa0 con 0x55f2d117b830 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f58c00640 1 -- 172.21.15.18:0/2128792660 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5409d140 con 0x55f2d117b830 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f58c00640 1 -- 172.21.15.18:0/2128792660 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6f540a1f80 con 0x55f2d117b830 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f747b4740 10 client.28922 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f747b4740 10 client.28922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f747b4740 20 client.28922 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f747b4740 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f2d1203470 0x55f2d1205860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f747b4740 4 client.28922 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f747b4740 1 -- 172.21.15.18:0/2128792660 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2d10e78b0 con 0x55f2d1203470 2024-04-16T17:16:43.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.642+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f2d1203470 0x55f2d1205860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.643+0000 7f6f6aa00640 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f2d1203470 0x55f2d1205860 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704615b2515e5882 server_cookie=1b5eadc35c49a2bd in_seq=0 out_seq=0 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.643+0000 7f6f58c00640 10 client.28922 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.643+0000 7f6f58c00640 1 -- 172.21.15.18:0/2128792660 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2d10e78b0 con 0x55f2d1203470 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.643+0000 7f6f58c00640 10 client.28922 handle_command_reply: tid=0 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 client.28922 shutdown 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 -- 172.21.15.18:0/2128792660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 msgr2=0x55f2d118ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d118ca80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400e9c0 tx=0x7f6f540a55c0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 -- 172.21.15.18:0/2128792660 shutdown_connections 2024-04-16T17:16:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f2d1203470 0x55f2d1205860 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2d1184410 0x55f2d1187a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d117b830 0x55f2d118ca80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f58c00640 0 client.28922 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 --2- 172.21.15.18:0/2128792660 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2d117af10 0x55f2d118c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 -- 172.21.15.18:0/2128792660 >> 172.21.15.18:0/2128792660 conn(0x55f2d1175640 msgr2=0x55f2d1177300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 -- 172.21.15.18:0/2128792660 shutdown_connections 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 1 -- 172.21.15.18:0/2128792660 wait complete. 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.644+0000 7f6f747b4740 20 client.28922 trim_cache size 0 max 16384 2024-04-16T17:16:43.648 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:16:43.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 -- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6f500a3da0 msgr2=0x7f6f500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6f500a3da0 0x7f6f500a6260 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55f2d117a9e0 tx=0x7f6f64010040 comp rx=0 tx=0).stop 2024-04-16T17:16:43.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 -- 172.21.15.18:0/3235381998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 msgr2=0x7f6f6c1a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 0x7f6f6c1a0b70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400e9f0 tx=0x7f6f5400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 -- 172.21.15.18:0/3235381998 shutdown_connections 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6f500a3da0 0x7f6f500a6260 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f6c19a780 0x7f6f6c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f6c107140 0x7f6f6c1a0b70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 --2- 172.21.15.18:0/3235381998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c102070 0x7f6f6c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 -- 172.21.15.18:0/3235381998 >> 172.21.15.18:0/3235381998 conn(0x7f6f6c0fd950 msgr2=0x7f6f6c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 -- 172.21.15.18:0/3235381998 shutdown_connections 2024-04-16T17:16:43.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:43.645+0000 7f6f6be00640 1 -- 172.21.15.18:0/3235381998 wait complete. 2024-04-16T17:16:43.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 449206 3.58 MB/sec execute 536 sec latency 74.715 ms 2024-04-16T17:16:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:43 smithi022 ceph-mon[36023]: pgmap v379: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 947 KiB/s wr, 125 op/s 2024-04-16T17:16:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:43 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1061226480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:44.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:43 smithi043 ceph-mon[36901]: pgmap v379: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 947 KiB/s wr, 125 op/s 2024-04-16T17:16:44.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:43 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1061226480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:44.666 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:16:44.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:16:44.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:16:44.674 INFO:tasks.workunit.client.0.smithi018.stdout: 1 450436 3.59 MB/sec execute 537 sec latency 13.444 ms 2024-04-16T17:16:44.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 -- 172.21.15.18:0/552709223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6220103900 msgr2=0x7f622010bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 --2- 172.21.15.18:0/552709223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6220103900 0x7f622010bde0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6208009900 tx=0x7f620802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 -- 172.21.15.18:0/552709223 shutdown_connections 2024-04-16T17:16:44.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 --2- 172.21.15.18:0/552709223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622010c400 0x7f622010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 --2- 172.21.15.18:0/552709223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6220103900 0x7f622010bde0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 --2- 172.21.15.18:0/552709223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6220102fe0 0x7f62201033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 -- 172.21.15.18:0/552709223 >> 172.21.15.18:0/552709223 conn(0x7f6220077ab0 msgr2=0x7f62200fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:44.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 -- 172.21.15.18:0/552709223 shutdown_connections 2024-04-16T17:16:44.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.888+0000 7f6225a00640 1 -- 172.21.15.18:0/552709223 wait complete. 2024-04-16T17:16:44.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.889+0000 7f6225a00640 1 Processor -- start 2024-04-16T17:16:44.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.889+0000 7f6225a00640 1 -- start start 2024-04-16T17:16:44.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6225a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6220102fe0 0x7f6220198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6225a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6220103900 0x7f6220198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6225a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f622010c400 0x7f622019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6225a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f62201121e0 con 0x7f622010c400 2024-04-16T17:16:44.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6225a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6220112060 con 0x7f6220103900 2024-04-16T17:16:44.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6225a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6220112360 con 0x7f6220102fe0 2024-04-16T17:16:44.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6217e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6220103900 0x7f6220198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6225000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f622010c400 0x7f622019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6217e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6220103900 0x7f6220198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54426/0 (socket says 172.21.15.18:54426) 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6217e00640 1 -- 172.21.15.18:0/2177271023 learned_addr learned my addr 172.21.15.18:0/2177271023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f621fe00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6220102fe0 0x7f6220198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.890+0000 7f6217e00640 1 -- 172.21.15.18:0/2177271023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6220102fe0 msgr2=0x7f6220198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f6217e00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6220102fe0 0x7f6220198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f6217e00640 1 -- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f622010c400 msgr2=0x7f622019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f6217e00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f622010c400 0x7f622019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f6217e00640 1 -- 172.21.15.18:0/2177271023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622019d750 con 0x7f6220103900 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f6225000640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f622010c400 0x7f622019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:44.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f6217e00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6220103900 0x7f6220198ca0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6208000c00 tx=0x7f6208004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f621d600640 1 -- 172.21.15.18:0/2177271023 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208042020 con 0x7f6220103900 2024-04-16T17:16:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f6225a00640 1 -- 172.21.15.18:0/2177271023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622019d9e0 con 0x7f6220103900 2024-04-16T17:16:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f621d600640 1 -- 172.21.15.18:0/2177271023 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6208005000 con 0x7f6220103900 2024-04-16T17:16:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.891+0000 7f621d600640 1 -- 172.21.15.18:0/2177271023 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208003a80 con 0x7f6220103900 2024-04-16T17:16:44.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.892+0000 7f6225a00640 1 -- 172.21.15.18:0/2177271023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62201a52d0 con 0x7f6220103900 2024-04-16T17:16:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.895+0000 7f6229dc2740 1 Processor -- start 2024-04-16T17:16:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.895+0000 7f621d600640 1 -- 172.21.15.18:0/2177271023 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6208045020 con 0x7f6220103900 2024-04-16T17:16:44.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.895+0000 7f6229dc2740 1 -- start start 2024-04-16T17:16:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.895+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 0x5555d34d9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6229dc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5555d342ecb0 con 0x5555d34ce2a0 2024-04-16T17:16:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6229dc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5555d33d5370 con 0x5555d34d77a0 2024-04-16T17:16:44.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6229dc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5555d342ac70 con 0x5555d34cebc0 2024-04-16T17:16:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6225000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6225000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50662/0 (socket says 172.21.15.18:50662) 2024-04-16T17:16:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6225000640 1 -- 172.21.15.18:0/2450885209 learned_addr learned my addr 172.21.15.18:0/2450885209 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f621fe00640 1 --2- 172.21.15.18:0/2450885209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6225000640 1 -- 172.21.15.18:0/2450885209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 msgr2=0x5555d34d7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6225000640 1 --2- 172.21.15.18:0/2450885209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34d7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6225000640 1 -- 172.21.15.18:0/2450885209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 msgr2=0x5555d34d9b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:16:44.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.896+0000 7f6225000640 1 --2- 172.21.15.18:0/2450885209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 0x5555d34d9b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.897+0000 7f6225000640 1 -- 172.21.15.18:0/2450885209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d33f4910 con 0x5555d34ce2a0 2024-04-16T17:16:44.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.897+0000 7f621d600640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f62040a3da0 0x7f62040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.897+0000 7f621d600640 1 -- 172.21.15.18:0/2177271023 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f62080f2710 con 0x7f6220103900 2024-04-16T17:16:44.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.897+0000 7f6225000640 1 --2- 172.21.15.18:0/2450885209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34ce680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6218011790 tx=0x7f6218011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c801d55740b68dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:44.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.897+0000 7f621fe00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f62040a3da0 0x7f62040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.897+0000 7f6215600640 1 -- 172.21.15.18:0/2450885209 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218004d20 con 0x5555d34ce2a0 2024-04-16T17:16:44.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.897+0000 7f6215600640 1 -- 172.21.15.18:0/2450885209 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6218003970 con 0x5555d34ce2a0 2024-04-16T17:16:44.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.897+0000 7f6215600640 1 -- 172.21.15.18:0/2450885209 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62180045d0 con 0x5555d34ce2a0 2024-04-16T17:16:44.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.898+0000 7f621fe00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f62040a3da0 0x7f62040a6260 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5555d34cdd70 tx=0x7f6210005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:44.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.898+0000 7f6229dc2740 1 -- 172.21.15.18:0/2450885209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 msgr2=0x5555d34ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.898+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2450885209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34ce680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6218011790 tx=0x7f6218011c50 comp rx=0 tx=0).stop 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.898+0000 7f6229dc2740 1 -- 172.21.15.18:0/2450885209 shutdown_connections 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.898+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2450885209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 0x5555d34d9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.898+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2450885209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.898+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2450885209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34ce680 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.898+0000 7f6229dc2740 1 -- 172.21.15.18:0/2450885209 >> 172.21.15.18:0/2450885209 conn(0x5555d34c89d0 msgr2=0x5555d34c8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.899+0000 7f6229dc2740 1 -- 172.21.15.18:0/2450885209 shutdown_connections 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.899+0000 7f6229dc2740 1 -- 172.21.15.18:0/2450885209 wait complete. 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.899+0000 7f6229dc2740 1 -- start start 2024-04-16T17:16:44.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.899+0000 7f6229dc2740 1 Processor -- start 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34ce2a0 0x5555d34d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34d77a0 0x5555d34d5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6229dc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5555d34da590 con 0x5555d34d77a0 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6229dc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5555d33e8730 con 0x5555d34cebc0 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6229dc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5555d34da710 con 0x5555d34ce2a0 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6229dc2740 10 client.0 _ll_register_callbacks cb 0x5555d33b71e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6217e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.900+0000 7f6217e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34d4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54442/0 (socket says 172.21.15.18:54442) 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.901+0000 7f6217e00640 1 -- 172.21.15.18:0/3038636257 learned_addr learned my addr 172.21.15.18:0/3038636257 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.901+0000 7f6225000640 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34d77a0 0x5555d34d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.901+0000 7f6217e00640 1 -- 172.21.15.18:0/3038636257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34ce2a0 msgr2=0x5555d34d4800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:16:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.901+0000 7f6217e00640 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34ce2a0 0x5555d34d4800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.901+0000 7f6217e00640 1 -- 172.21.15.18:0/3038636257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34d77a0 msgr2=0x5555d34d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.901+0000 7f6217e00640 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34d77a0 0x5555d34d5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.901+0000 7f6217e00640 1 -- 172.21.15.18:0/3038636257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d32097e0 con 0x5555d34cebc0 2024-04-16T17:16:44.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.901+0000 7f6225000640 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34d77a0 0x5555d34d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.902+0000 7f6217e00640 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34d4d40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6208005ab0 tx=0x7f62080bdf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8cf0a07fe1677c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.902+0000 7f620e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.902+0000 7f620e000640 1 -- 172.21.15.18:0/3038636257 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62080be350 con 0x5555d34cebc0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.902+0000 7f620e000640 1 -- 172.21.15.18:0/3038636257 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f62080bc650 con 0x5555d34cebc0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.902+0000 7f6229dc2740 1 -- 172.21.15.18:0/3038636257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6218003970 con 0x5555d34cebc0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.902+0000 7f620e000640 1 -- 172.21.15.18:0/3038636257 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62080c2700 con 0x5555d34cebc0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.902+0000 7f6229dc2740 10 client.28931 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.902+0000 7f6229dc2740 1 -- 172.21.15.18:0/3038636257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5555d3269f70 con 0x5555d34cebc0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f620e000640 1 -- 172.21.15.18:0/3038636257 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6218003970 con 0x5555d34cebc0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f620e000640 1 -- 172.21.15.18:0/3038636257 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f62080be5b0 con 0x5555d34cebc0 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f6229dc2740 10 client.28931 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f6229dc2740 10 client.28931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f6229dc2740 20 client.28931 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5555d34e7b70 0x5555d34e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f6229dc2740 4 client.28931 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f6229dc2740 1 -- 172.21.15.18:0/3038636257 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d343c6d0 con 0x5555d34e7b70 2024-04-16T17:16:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.903+0000 7f621fe00640 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5555d34e7b70 0x5555d34e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.904+0000 7f621fe00640 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5555d34e7b70 0x5555d34e9fd0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfbd51d14db2864 server_cookie=1a62f5c2445dee71 in_seq=0 out_seq=0 2024-04-16T17:16:44.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.904+0000 7f620e000640 10 client.28931 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:44.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f620e000640 1 -- 172.21.15.18:0/3038636257 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555d343c6d0 con 0x5555d34e7b70 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f620e000640 10 client.28931 handle_command_reply: tid=0 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 client.28931 shutdown 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 -- 172.21.15.18:0/3038636257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 msgr2=0x5555d34d4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34d4d40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6208005ab0 tx=0x7f62080bdf10 comp rx=0 tx=0).stop 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 -- 172.21.15.18:0/3038636257 shutdown_connections 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5555d34e7b70 0x5555d34e9fd0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34d77a0 0x5555d34d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34d4d40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3038636257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34ce2a0 0x5555d34d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f620e000640 0 client.28931 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.906+0000 7f6229dc2740 1 -- 172.21.15.18:0/3038636257 >> 172.21.15.18:0/3038636257 conn(0x5555d34c89d0 msgr2=0x5555d34ca720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:44.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.907+0000 7f6229dc2740 1 -- 172.21.15.18:0/3038636257 shutdown_connections 2024-04-16T17:16:44.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.907+0000 7f6229dc2740 1 -- 172.21.15.18:0/3038636257 wait complete. 2024-04-16T17:16:44.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.907+0000 7f6229dc2740 20 client.28931 trim_cache size 0 max 16384 2024-04-16T17:16:44.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.946+0000 7f6229dc2740 1 Processor -- start 2024-04-16T17:16:44.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.946+0000 7f6229dc2740 1 -- start start 2024-04-16T17:16:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.946+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.946+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34e3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.946+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 0x5555d34ca530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.946+0000 7f6229dc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f62201afb40 con 0x5555d34ce2a0 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.946+0000 7f6229dc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62201ad410 con 0x5555d34d77a0 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.946+0000 7f6229dc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62201ab280 con 0x5555d34cebc0 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f621fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6217e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6225000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 0x5555d34ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6217e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34e3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37528/0 (socket says 172.21.15.18:37528) 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f621fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34d6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50682/0 (socket says 172.21.15.18:50682) 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6217e00640 1 -- 172.21.15.18:0/3109430850 learned_addr learned my addr 172.21.15.18:0/3109430850 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6217e00640 1 -- 172.21.15.18:0/3109430850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 msgr2=0x5555d34ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6217e00640 1 --2- 172.21.15.18:0/3109430850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 0x5555d34ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6217e00640 1 -- 172.21.15.18:0/3109430850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 msgr2=0x5555d34d6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6217e00640 1 --2- 172.21.15.18:0/3109430850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34d6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f6217e00640 1 -- 172.21.15.18:0/3109430850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d3101170 con 0x5555d34cebc0 2024-04-16T17:16:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.947+0000 7f621fe00640 1 --2- 172.21.15.18:0/3109430850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34d6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.948+0000 7f6217e00640 1 --2- 172.21.15.18:0/3109430850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34e3480 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f620802c8b0 tx=0x7f6208005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75cc6b04a7dbfde0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.948+0000 7f6214c00640 1 -- 172.21.15.18:0/3109430850 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620809e070 con 0x5555d34cebc0 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.948+0000 7f6214c00640 1 -- 172.21.15.18:0/3109430850 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62080c6fc0 con 0x5555d34cebc0 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.948+0000 7f6214c00640 1 -- 172.21.15.18:0/3109430850 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62080be5b0 con 0x5555d34cebc0 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 -- 172.21.15.18:0/3109430850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 msgr2=0x5555d34e3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3109430850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34e3480 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f620802c8b0 tx=0x7f6208005fd0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 -- 172.21.15.18:0/3109430850 shutdown_connections 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3109430850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34d77a0 0x5555d34ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3109430850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34cebc0 0x5555d34e3480 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 --2- 172.21.15.18:0/3109430850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34d6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 -- 172.21.15.18:0/3109430850 >> 172.21.15.18:0/3109430850 conn(0x5555d34c89d0 msgr2=0x5555d34c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:44.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 -- 172.21.15.18:0/3109430850 shutdown_connections 2024-04-16T17:16:44.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.949+0000 7f6229dc2740 1 -- 172.21.15.18:0/3109430850 wait complete. 2024-04-16T17:16:44.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.950+0000 7f6229dc2740 1 -- start start 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.950+0000 7f6229dc2740 1 Processor -- start 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6229dc2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34d77a0 0x5555d34dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6229dc2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f62201ab8d0 con 0x5555d34ce2a0 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6229dc2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62201acd40 con 0x5555d34cebc0 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6229dc2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62201abfa0 con 0x5555d34d77a0 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6229dc2740 10 client.0 _ll_register_callbacks cb 0x7f62080ba5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:44.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6217e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.951+0000 7f6217e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34dba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54464/0 (socket says 172.21.15.18:54464) 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f6217e00640 1 -- 172.21.15.18:0/2555853795 learned_addr learned my addr 172.21.15.18:0/2555853795 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f6225000640 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34d77a0 0x5555d34dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f621fe00640 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f6217e00640 1 -- 172.21.15.18:0/2555853795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34d77a0 msgr2=0x5555d34dbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f6217e00640 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34d77a0 0x5555d34dbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f6217e00640 1 -- 172.21.15.18:0/2555853795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 msgr2=0x5555d34db4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f6217e00640 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34db4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f6217e00640 1 -- 172.21.15.18:0/2555853795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d354d240 con 0x5555d34cebc0 2024-04-16T17:16:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f6217e00640 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34dba10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f62080ba030 tx=0x7f6208009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3acc99cd31d95d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.952+0000 7f620e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.953+0000 7f620e000640 1 -- 172.21.15.18:0/2555853795 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620809e070 con 0x5555d34cebc0 2024-04-16T17:16:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.953+0000 7f620e000640 1 -- 172.21.15.18:0/2555853795 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f62080f2260 con 0x5555d34cebc0 2024-04-16T17:16:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.954+0000 7f6229dc2740 1 -- 172.21.15.18:0/2555853795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62080c6fc0 con 0x5555d34cebc0 2024-04-16T17:16:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.954+0000 7f620e000640 1 -- 172.21.15.18:0/2555853795 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62080be350 con 0x5555d34cebc0 2024-04-16T17:16:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.954+0000 7f6229dc2740 10 client.28937 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.954+0000 7f6229dc2740 1 -- 172.21.15.18:0/2555853795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5555d354d240 con 0x5555d34cebc0 2024-04-16T17:16:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.954+0000 7f620e000640 1 -- 172.21.15.18:0/2555853795 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62080c6fc0 con 0x5555d34cebc0 2024-04-16T17:16:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.955+0000 7f620e000640 1 -- 172.21.15.18:0/2555853795 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f62080bd7a0 con 0x5555d34cebc0 2024-04-16T17:16:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.955+0000 7f6229dc2740 10 client.28937 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.955+0000 7f6229dc2740 10 client.28937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.955+0000 7f6229dc2740 20 client.28937 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.955+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5555d354e890 0x5555d3550cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.955+0000 7f6229dc2740 4 client.28937 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:16:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.955+0000 7f6229dc2740 1 -- 172.21.15.18:0/2555853795 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555d3436540 con 0x5555d354e890 2024-04-16T17:16:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.955+0000 7f621fe00640 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5555d354e890 0x5555d3550cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.956+0000 7f621fe00640 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5555d354e890 0x5555d3550cb0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb016dfdc9f110e server_cookie=fd566bb2f41b90bd in_seq=0 out_seq=0 2024-04-16T17:16:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.956+0000 7f620e000640 10 client.28937 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.958+0000 7f620e000640 1 -- 172.21.15.18:0/2555853795 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555d3436540 con 0x5555d354e890 2024-04-16T17:16:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f620e000640 10 client.28937 handle_command_reply: tid=0 2024-04-16T17:16:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 client.28937 shutdown 2024-04-16T17:16:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 -- 172.21.15.18:0/2555853795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 msgr2=0x5555d34dba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34dba10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f62080ba030 tx=0x7f6208009260 comp rx=0 tx=0).stop 2024-04-16T17:16:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 -- 172.21.15.18:0/2555853795 shutdown_connections 2024-04-16T17:16:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5555d354e890 0x5555d3550cb0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555d34d77a0 0x5555d34dbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555d34cebc0 0x5555d34dba10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 --2- 172.21.15.18:0/2555853795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5555d34ce2a0 0x5555d34db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 -- 172.21.15.18:0/2555853795 >> 172.21.15.18:0/2555853795 conn(0x5555d34c89d0 msgr2=0x5555d34d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f620e000640 0 client.28937 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 -- 172.21.15.18:0/2555853795 shutdown_connections 2024-04-16T17:16:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.959+0000 7f6229dc2740 1 -- 172.21.15.18:0/2555853795 wait complete. 2024-04-16T17:16:44.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.960+0000 7f6229dc2740 20 client.28937 trim_cache size 0 max 16384 2024-04-16T17:16:44.961 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:16:44.961 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:16:44.961 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "01d3a598-e812-4bf2-81ec-fbbdecb49120", 2024-04-16T17:16:44.961 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:16:44.961 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 -- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f62040a3da0 msgr2=0x7f62040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f62040a3da0 0x7f62040a6260 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5555d34cdd70 tx=0x7f6210005f90 comp rx=0 tx=0).stop 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 -- 172.21.15.18:0/2177271023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6220103900 msgr2=0x7f6220198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6220103900 0x7f6220198ca0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6208000c00 tx=0x7f6208004360 comp rx=0 tx=0).stop 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 -- 172.21.15.18:0/2177271023 shutdown_connections 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f62040a3da0 0x7f62040a6260 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f622010c400 0x7f622019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6220103900 0x7f6220198ca0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 --2- 172.21.15.18:0/2177271023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6220102fe0 0x7f6220198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 -- 172.21.15.18:0/2177271023 >> 172.21.15.18:0/2177271023 conn(0x7f6220077ab0 msgr2=0x7f62200fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:44.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 -- 172.21.15.18:0/2177271023 shutdown_connections 2024-04-16T17:16:44.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:44.961+0000 7f6225a00640 1 -- 172.21.15.18:0/2177271023 wait complete. 2024-04-16T17:16:44.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:44.985 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 -- 172.21.15.18:0/344755369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc076530 msgr2=0x7fbbdc076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 --2- 172.21.15.18:0/344755369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc076530 0x7fbbdc076910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009890 tx=0x7fbbcc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 -- 172.21.15.18:0/344755369 shutdown_connections 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 --2- 172.21.15.18:0/344755369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc112e70 0x7fbbdc115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 --2- 172.21.15.18:0/344755369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc076e50 0x7fbbdc112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 --2- 172.21.15.18:0/344755369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc076530 0x7fbbdc076910 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 -- 172.21.15.18:0/344755369 >> 172.21.15.18:0/344755369 conn(0x7fbbdc0ffc80 msgr2=0x7fbbdc1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 -- 172.21.15.18:0/344755369 shutdown_connections 2024-04-16T17:16:45.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.237+0000 7fbbe1600640 1 -- 172.21.15.18:0/344755369 wait complete. 2024-04-16T17:16:45.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.238+0000 7fbbe1600640 1 Processor -- start 2024-04-16T17:16:45.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.238+0000 7fbbe1600640 1 -- start start 2024-04-16T17:16:45.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbe1600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc076530 0x7fbbdc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbe1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc076e50 0x7fbbdc19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbe1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc112e70 0x7fbbdc1a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbe1600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1189d0 con 0x7fbbdc076530 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbe1600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc118850 con 0x7fbbdc076e50 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbe1600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc118b50 con 0x7fbbdc112e70 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbda000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc076e50 0x7fbbdc19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbda000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc076e50 0x7fbbdc19f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54478/0 (socket says 172.21.15.18:54478) 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbda000640 1 -- 172.21.15.18:0/200227221 learned_addr learned my addr 172.21.15.18:0/200227221 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbdb400640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc112e70 0x7fbbdc1a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc076530 0x7fbbdc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbda000640 1 -- 172.21.15.18:0/200227221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc112e70 msgr2=0x7fbbdc1a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbda000640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc112e70 0x7fbbdc1a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbda000640 1 -- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc076530 msgr2=0x7fbbdc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbda000640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc076530 0x7fbbdc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.239+0000 7fbbda000640 1 -- 172.21.15.18:0/200227221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc1a3f40 con 0x7fbbdc076e50 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.240+0000 7fbbda000640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc076e50 0x7fbbdc19f490 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800d950 tx=0x7fbbc800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:45.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.240+0000 7fbbd3400640 1 -- 172.21.15.18:0/200227221 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8017070 con 0x7fbbdc076e50 2024-04-16T17:16:45.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.241+0000 7fbbd3400640 1 -- 172.21.15.18:0/200227221 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbc8009c30 con 0x7fbbdc076e50 2024-04-16T17:16:45.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.241+0000 7fbbd3400640 1 -- 172.21.15.18:0/200227221 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800f6b0 con 0x7fbbdc076e50 2024-04-16T17:16:45.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.241+0000 7fbbe1600640 1 -- 172.21.15.18:0/200227221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc1a4230 con 0x7fbbdc076e50 2024-04-16T17:16:45.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.241+0000 7fbbe1600640 1 -- 172.21.15.18:0/200227221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc1abac0 con 0x7fbbdc076e50 2024-04-16T17:16:45.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.244+0000 7fbbe43ba740 1 Processor -- start 2024-04-16T17:16:45.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbe43ba740 1 -- start start 2024-04-16T17:16:45.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a291b0 0x55f381a29590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a29ad0 0x55f381a32170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 0x55f381a34aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f38198f120 con 0x55f381a29ad0 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f381935340 con 0x55f381a291b0 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f38198b0e0 con 0x55f381a326b0 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbdb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a291b0 0x55f381a29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.245+0000 7fbbdb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a291b0 0x55f381a29590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54494/0 (socket says 172.21.15.18:54494) 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdb400640 1 -- 172.21.15.18:0/2201952237 learned_addr learned my addr 172.21.15.18:0/2201952237 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbd3400640 1 -- 172.21.15.18:0/200227221 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbc80109d0 con 0x7fbbdc076e50 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a29ad0 0x55f381a32170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdb400640 1 -- 172.21.15.18:0/2201952237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 msgr2=0x55f381a34aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdb400640 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 0x55f381a34aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdb400640 1 -- 172.21.15.18:0/2201952237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a29ad0 msgr2=0x55f381a32170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdb400640 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a29ad0 0x55f381a32170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdb400640 1 -- 172.21.15.18:0/2201952237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38176b7d0 con 0x55f381a291b0 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a29ad0 0x55f381a32170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:16:45.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.246+0000 7fbbdb400640 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a291b0 0x55f381a29590 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e480 tx=0x7fbbd40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25391664f8f6d1ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:45.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.247+0000 7fbbc7400640 1 -- 172.21.15.18:0/2201952237 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400d9d0 con 0x55f381a291b0 2024-04-16T17:16:45.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.247+0000 7fbbc7400640 1 -- 172.21.15.18:0/2201952237 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd401c070 con 0x55f381a291b0 2024-04-16T17:16:45.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.247+0000 7fbbc7400640 1 -- 172.21.15.18:0/2201952237 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4016380 con 0x55f381a291b0 2024-04-16T17:16:45.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.248+0000 7fbbe43ba740 1 -- 172.21.15.18:0/2201952237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a291b0 msgr2=0x55f381a29590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.248+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a291b0 0x55f381a29590 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e480 tx=0x7fbbd40095c0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.248+0000 7fbbd3400640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3da0 0x7fbbc00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.248+0000 7fbbd3400640 1 -- 172.21.15.18:0/200227221 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fbbc80d0890 con 0x7fbbdc076e50 2024-04-16T17:16:45.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.248+0000 7fbbdb400640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3da0 0x7fbbc00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.249+0000 7fbbe43ba740 1 -- 172.21.15.18:0/2201952237 shutdown_connections 2024-04-16T17:16:45.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.249+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 0x55f381a34aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.249+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a29ad0 0x55f381a32170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.249+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/2201952237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a291b0 0x55f381a29590 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.249+0000 7fbbe43ba740 1 -- 172.21.15.18:0/2201952237 >> 172.21.15.18:0/2201952237 conn(0x55f381a238e0 msgr2=0x55f381a23cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:45.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.249+0000 7fbbe43ba740 1 -- 172.21.15.18:0/2201952237 shutdown_connections 2024-04-16T17:16:45.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.249+0000 7fbbe43ba740 1 -- 172.21.15.18:0/2201952237 wait complete. 2024-04-16T17:16:45.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbdb400640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3da0 0x7fbbc00a6260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009e20 tx=0x7fbbd4009b30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:45.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 1 -- start start 2024-04-16T17:16:45.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 1 Processor -- start 2024-04-16T17:16:45.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a2f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a2fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 0x55f381a301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f381a354a0 con 0x55f381a291b0 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f381948710 con 0x55f381a29ad0 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f381a35620 con 0x55f381a326b0 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.250+0000 7fbbe43ba740 10 client.0 _ll_register_callbacks cb 0x55f381943af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.251+0000 7fbbdaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a2f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.251+0000 7fbbdaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a2f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50718/0 (socket says 172.21.15.18:50718) 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.251+0000 7fbbdaa00640 1 -- 172.21.15.18:0/772227603 learned_addr learned my addr 172.21.15.18:0/772227603 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.251+0000 7fbbda000640 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a2fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.251+0000 7fbbdb400640 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 0x55f381a301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbda000640 1 -- 172.21.15.18:0/772227603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 msgr2=0x55f381a301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbda000640 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 0x55f381a301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbda000640 1 -- 172.21.15.18:0/772227603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 msgr2=0x55f381a2f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbda000640 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a2f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbda000640 1 -- 172.21.15.18:0/772227603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3816a13f0 con 0x55f381a29ad0 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a2f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbda000640 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a2fc80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80cdb70 tx=0x7fbbc809bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=399d1f0f0e5065bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbbbe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbbbe00640 1 -- 172.21.15.18:0/772227603 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80b0e60 con 0x55f381a29ad0 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbbbe00640 1 -- 172.21.15.18:0/772227603 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fbbc80d0400 con 0x55f381a29ad0 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.252+0000 7fbbe43ba740 1 -- 172.21.15.18:0/772227603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd401c070 con 0x55f381a29ad0 2024-04-16T17:16:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbbbe00640 1 -- 172.21.15.18:0/772227603 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbbc80cdff0 con 0x55f381a29ad0 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbe43ba740 10 client.28949 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbe43ba740 1 -- 172.21.15.18:0/772227603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f3816610f0 con 0x55f381a29ad0 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbbbe00640 1 -- 172.21.15.18:0/772227603 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd401c070 con 0x55f381a29ad0 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbbbe00640 1 -- 172.21.15.18:0/772227603 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbbc8023060 con 0x55f381a29ad0 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbe43ba740 10 client.28949 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbe43ba740 10 client.28949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbe43ba740 20 client.28949 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f381a42a00 0x55f381a44df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbe43ba740 4 client.28949 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbe43ba740 1 -- 172.21.15.18:0/772227603 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f38199cb40 con 0x55f381a42a00 2024-04-16T17:16:45.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.253+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f381a42a00 0x55f381a44df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.254+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f381a42a00 0x55f381a44df0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e6a25e61c3a3d0 server_cookie=3ef11dd613f8275e in_seq=0 out_seq=0 2024-04-16T17:16:45.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.254+0000 7fbbbbe00640 10 client.28949 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:45.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbbbe00640 1 -- 172.21.15.18:0/772227603 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f38199cb40 con 0x55f381a42a00 2024-04-16T17:16:45.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbbbe00640 10 client.28949 handle_command_reply: tid=0 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 client.28949 shutdown 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 -- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 msgr2=0x55f381a2fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a2fc80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80cdb70 tx=0x7fbbc809bde0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 -- 172.21.15.18:0/772227603 shutdown_connections 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f381a42a00 0x55f381a44df0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a326b0 0x55f381a301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a2fc80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/772227603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a2f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 -- 172.21.15.18:0/772227603 >> 172.21.15.18:0/772227603 conn(0x55f381a238e0 msgr2=0x55f381a25630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbbbe00640 0 client.28949 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 -- 172.21.15.18:0/772227603 shutdown_connections 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.255+0000 7fbbe43ba740 1 -- 172.21.15.18:0/772227603 wait complete. 2024-04-16T17:16:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.256+0000 7fbbe43ba740 20 client.28949 trim_cache size 0 max 16384 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.277+0000 7fbbe43ba740 1 Processor -- start 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.277+0000 7fbbe43ba740 1 -- start start 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.277+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a238e0 0x55f381a3e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.277+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a23cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.277+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a29ad0 0x55f381a2ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.277+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b8650 con 0x55f381a291b0 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.277+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b5f20 con 0x55f381a238e0 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.277+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b3d90 con 0x55f381a29ad0 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a238e0 0x55f381a3e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a29ad0 0x55f381a2ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a29ad0 0x55f381a2ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37596/0 (socket says 172.21.15.18:37596) 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdb400640 1 -- 172.21.15.18:0/990107360 learned_addr learned my addr 172.21.15.18:0/990107360 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a238e0 0x55f381a3e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54510/0 (socket says 172.21.15.18:54510) 2024-04-16T17:16:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbda000640 1 --2- 172.21.15.18:0/990107360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a23cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdaa00640 1 -- 172.21.15.18:0/990107360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a29ad0 msgr2=0x55f381a2ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/990107360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a29ad0 0x55f381a2ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdaa00640 1 -- 172.21.15.18:0/990107360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 msgr2=0x55f381a23cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/990107360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a23cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdaa00640 1 -- 172.21.15.18:0/990107360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f381a45160 con 0x55f381a238e0 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/990107360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a238e0 0x55f381a3e360 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009250 tx=0x7fbbcc0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f42698f253bc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.278+0000 7fbbc6a00640 1 -- 172.21.15.18:0/990107360 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc039040 con 0x55f381a238e0 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbc6a00640 1 -- 172.21.15.18:0/990107360 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbcc04b070 con 0x55f381a238e0 2024-04-16T17:16:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbc6a00640 1 -- 172.21.15.18:0/990107360 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc02cbc0 con 0x55f381a238e0 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 -- 172.21.15.18:0/990107360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a238e0 msgr2=0x55f381a3e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/990107360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a238e0 0x55f381a3e360 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009250 tx=0x7fbbcc0056d0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 -- 172.21.15.18:0/990107360 shutdown_connections 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/990107360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a29ad0 0x55f381a2ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/990107360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a291b0 0x55f381a23cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/990107360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a238e0 0x55f381a3e360 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 -- 172.21.15.18:0/990107360 >> 172.21.15.18:0/990107360 conn(0x55f381a2b110 msgr2=0x55f381a22040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 -- 172.21.15.18:0/990107360 shutdown_connections 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.279+0000 7fbbe43ba740 1 -- 172.21.15.18:0/990107360 wait complete. 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 1 -- start start 2024-04-16T17:16:45.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 1 Processor -- start 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a238e0 0x55f381a3a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a291b0 0x55f381a35e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a36370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b43e0 con 0x55f381a238e0 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b5850 con 0x55f381a29ad0 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1b4ab0 con 0x55f381a291b0 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.280+0000 7fbbe43ba740 10 client.0 _ll_register_callbacks cb 0x7fbbc809c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a36370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a36370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54526/0 (socket says 172.21.15.18:54526) 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 -- 172.21.15.18:0/438989750 learned_addr learned my addr 172.21.15.18:0/438989750 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a238e0 0x55f381a3a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbda000640 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a291b0 0x55f381a35e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 -- 172.21.15.18:0/438989750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a291b0 msgr2=0x55f381a35e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a291b0 0x55f381a35e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 -- 172.21.15.18:0/438989750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a238e0 msgr2=0x55f381a3a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a238e0 0x55f381a3a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 -- 172.21.15.18:0/438989750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f381a36930 con 0x55f381a29ad0 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbdb400640 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a36370 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009ba0 tx=0x7fbbd4017b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5928a9db14ad8c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.281+0000 7fbbbbe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.282+0000 7fbbbbe00640 1 -- 172.21.15.18:0/438989750 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400dea0 con 0x55f381a29ad0 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.282+0000 7fbbe43ba740 1 -- 172.21.15.18:0/438989750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbcc04b070 con 0x55f381a29ad0 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.282+0000 7fbbbbe00640 1 -- 172.21.15.18:0/438989750 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7fbbd4017420 con 0x55f381a29ad0 2024-04-16T17:16:45.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.282+0000 7fbbbbe00640 1 -- 172.21.15.18:0/438989750 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbbd4017620 con 0x55f381a29ad0 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.282+0000 7fbbe43ba740 10 client.28955 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.282+0000 7fbbe43ba740 1 -- 172.21.15.18:0/438989750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f381aa8c00 con 0x55f381a29ad0 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.282+0000 7fbbbbe00640 1 -- 172.21.15.18:0/438989750 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbcc04b070 con 0x55f381a29ad0 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.283+0000 7fbbbbe00640 1 -- 172.21.15.18:0/438989750 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbbd400b040 con 0x55f381a29ad0 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.283+0000 7fbbe43ba740 10 client.28955 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.283+0000 7fbbe43ba740 10 client.28955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.283+0000 7fbbe43ba740 20 client.28955 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.283+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f381a326b0 0x55f381aab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.283+0000 7fbbe43ba740 4 client.28955 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.283+0000 7fbbe43ba740 1 -- 172.21.15.18:0/438989750 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3819969b0 con 0x55f381a326b0 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.283+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f381a326b0 0x55f381aab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.284+0000 7fbbdaa00640 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f381a326b0 0x55f381aab2a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a1ad0f5a4d5d16 server_cookie=6f0abdc198369894 in_seq=0 out_seq=0 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.284+0000 7fbbbbe00640 10 client.28955 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.284+0000 7fbbbbe00640 1 -- 172.21.15.18:0/438989750 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f3819969b0 con 0x55f381a326b0 2024-04-16T17:16:45.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.284+0000 7fbbbbe00640 10 client.28955 handle_command_reply: tid=0 2024-04-16T17:16:45.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.284+0000 7fbbe43ba740 1 client.28955 shutdown 2024-04-16T17:16:45.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 -- 172.21.15.18:0/438989750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 msgr2=0x55f381a36370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a36370 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009ba0 tx=0x7fbbd4017b10 comp rx=0 tx=0).stop 2024-04-16T17:16:45.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 -- 172.21.15.18:0/438989750 shutdown_connections 2024-04-16T17:16:45.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f381a326b0 0x55f381aab2a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f381a29ad0 0x55f381a36370 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f381a291b0 0x55f381a35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 --2- 172.21.15.18:0/438989750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f381a238e0 0x55f381a3a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 -- 172.21.15.18:0/438989750 >> 172.21.15.18:0/438989750 conn(0x55f381a2b110 msgr2=0x55f381a24d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbbbe00640 0 client.28955 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 -- 172.21.15.18:0/438989750 shutdown_connections 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 1 -- 172.21.15.18:0/438989750 wait complete. 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.285+0000 7fbbe43ba740 20 client.28955 trim_cache size 0 max 16384 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stdout: "01d3a598-e812-4bf2-81ec-fbbdecb49120": { 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "01d3a598-e812-4bf2-81ec-fbbdecb49120", 2024-04-16T17:16:45.288 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:16:45.289 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:16:45.289 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:16:45.289 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:16:45.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 -- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3da0 msgr2=0x7fbbc00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3da0 0x7fbbc00a6260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009e20 tx=0x7fbbd4009b30 comp rx=0 tx=0).stop 2024-04-16T17:16:45.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 -- 172.21.15.18:0/200227221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc076e50 msgr2=0x7fbbdc19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:45.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc076e50 0x7fbbdc19f490 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800d950 tx=0x7fbbc800de10 comp rx=0 tx=0).stop 2024-04-16T17:16:45.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 -- 172.21.15.18:0/200227221 shutdown_connections 2024-04-16T17:16:45.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbbc00a3da0 0x7fbbc00a6260 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbdc112e70 0x7fbbdc1a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbdc076e50 0x7fbbdc19f490 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 --2- 172.21.15.18:0/200227221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbdc076530 0x7fbbdc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:45.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.286+0000 7fbbe1600640 1 -- 172.21.15.18:0/200227221 >> 172.21.15.18:0/200227221 conn(0x7fbbdc0ffc80 msgr2=0x7fbbdc114560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:45.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.287+0000 7fbbe1600640 1 -- 172.21.15.18:0/200227221 shutdown_connections 2024-04-16T17:16:45.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:16:45.287+0000 7fbbe1600640 1 -- 172.21.15.18:0/200227221 wait complete. 2024-04-16T17:16:45.304 INFO:tasks.cephfs.filesystem:scrub status for tag:01d3a598-e812-4bf2-81ec-fbbdecb49120 - {'path': '/', 'tag': '01d3a598-e812-4bf2-81ec-fbbdecb49120', 'options': 'recursive,force'} 2024-04-16T17:16:45.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 451088 3.59 MB/sec execute 538 sec latency 37.000 ms 2024-04-16T17:16:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:45 smithi022 ceph-mon[36023]: pgmap v380: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 839 KiB/s wr, 110 op/s 2024-04-16T17:16:46.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:45 smithi043 ceph-mon[36901]: pgmap v380: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 839 KiB/s wr, 110 op/s 2024-04-16T17:16:46.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:39 smithi018 ceph-mon[31776]: pgmap v377: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-04-16T17:16:46.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:41 smithi018 ceph-mon[31776]: pgmap v378: 129 pgs: 129 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-04-16T17:16:46.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:43 smithi018 ceph-mon[31776]: pgmap v379: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 947 KiB/s wr, 125 op/s 2024-04-16T17:16:46.324 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:43 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1061226480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:46.324 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:45 smithi018 ceph-mon[31776]: pgmap v380: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 839 KiB/s wr, 110 op/s 2024-04-16T17:16:46.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 451616 3.59 MB/sec execute 539 sec latency 55.650 ms 2024-04-16T17:16:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:46 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:16:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:46 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:46 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:16:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:46 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:46 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:16:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:46 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:46 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:16:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:46 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:47.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 451991 3.59 MB/sec execute 540 sec latency 116.998 ms 2024-04-16T17:16:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:47 smithi022 ceph-mon[36023]: pgmap v381: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-16T17:16:48.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:47 smithi043 ceph-mon[36901]: pgmap v381: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-16T17:16:48.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 452334 3.58 MB/sec execute 541 sec latency 85.158 ms 2024-04-16T17:16:49.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 453466 3.59 MB/sec execute 542 sec latency 34.730 ms 2024-04-16T17:16:50.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:49 smithi022 ceph-mon[36023]: pgmap v382: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 870 KiB/s wr, 125 op/s 2024-04-16T17:16:50.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:49 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:16:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:49 smithi043 ceph-mon[36901]: pgmap v382: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 870 KiB/s wr, 125 op/s 2024-04-16T17:16:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:49 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:16:50.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 454400 3.59 MB/sec execute 543 sec latency 51.718 ms 2024-04-16T17:16:51.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:46 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:16:51.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:46 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:51.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:46 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:16:51.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:46 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:51.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:47 smithi018 ceph-mon[31776]: pgmap v381: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-16T17:16:51.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:49 smithi018 ceph-mon[31776]: pgmap v382: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 870 KiB/s wr, 125 op/s 2024-04-16T17:16:51.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:49 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:16:51.670 INFO:tasks.workunit.client.0.smithi018.stdout: 1 454822 3.59 MB/sec execute 544 sec latency 60.077 ms 2024-04-16T17:16:52.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:51 smithi022 ceph-mon[36023]: pgmap v383: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 996 KiB/s wr, 136 op/s 2024-04-16T17:16:52.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:51 smithi043 ceph-mon[36901]: pgmap v383: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 996 KiB/s wr, 136 op/s 2024-04-16T17:16:52.671 INFO:tasks.workunit.client.0.smithi018.stdout: 1 455110 3.58 MB/sec execute 545 sec latency 77.666 ms 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 -- 172.21.15.18:0/4195596050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c1042a0 msgr2=0x7f549c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 --2- 172.21.15.18:0/4195596050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c1042a0 0x7f549c104680 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5490009900 tx=0x7f549002bc50 comp rx=0 tx=0).stop 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 -- 172.21.15.18:0/4195596050 shutdown_connections 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 --2- 172.21.15.18:0/4195596050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f549c109370 0x7f549c110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 --2- 172.21.15.18:0/4195596050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c104c50 0x7f549c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 --2- 172.21.15.18:0/4195596050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c1042a0 0x7f549c104680 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 -- 172.21.15.18:0/4195596050 >> 172.21.15.18:0/4195596050 conn(0x7f549c0ffb00 msgr2=0x7f549c101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 -- 172.21.15.18:0/4195596050 shutdown_connections 2024-04-16T17:16:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.754+0000 7f54a2c00640 1 -- 172.21.15.18:0/4195596050 wait complete. 2024-04-16T17:16:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.755+0000 7f54a2c00640 1 Processor -- start 2024-04-16T17:16:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.755+0000 7f54a2c00640 1 -- start start 2024-04-16T17:16:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.755+0000 7f54a2c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1042a0 0x7f549c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.755+0000 7f54a2c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c109370 0x7f549c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.755+0000 7f54a2c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f549c19a870 0x7f549c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.755+0000 7f54a2c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1144f0 con 0x7f549c109370 2024-04-16T17:16:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.755+0000 7f54a2c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f549c114370 con 0x7f549c1042a0 2024-04-16T17:16:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.755+0000 7f54a2c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f549c114670 con 0x7f549c19a870 2024-04-16T17:16:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1042a0 0x7f549c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1042a0 0x7f549c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57776/0 (socket says 172.21.15.18:57776) 2024-04-16T17:16:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a1800640 1 -- 172.21.15.18:0/1856286681 learned_addr learned my addr 172.21.15.18:0/1856286681 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a0e00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c109370 0x7f549c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a2200640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f549c19a870 0x7f549c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a0e00640 1 -- 172.21.15.18:0/1856286681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f549c19a870 msgr2=0x7f549c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a0e00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f549c19a870 0x7f549c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a0e00640 1 -- 172.21.15.18:0/1856286681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1042a0 msgr2=0x7f549c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a0e00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1042a0 0x7f549c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.756+0000 7f54a0e00640 1 -- 172.21.15.18:0/1856286681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c19b400 con 0x7f549c109370 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.757+0000 7f54a0e00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c109370 0x7f549c1a0bf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f548800ea30 tx=0x7f548800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.757+0000 7f549a000640 1 -- 172.21.15.18:0/1856286681 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548800ce40 con 0x7f549c109370 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.757+0000 7f549a000640 1 -- 172.21.15.18:0/1856286681 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5488010910 con 0x7f549c109370 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.757+0000 7f54a2c00640 1 -- 172.21.15.18:0/1856286681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c19b6f0 con 0x7f549c109370 2024-04-16T17:16:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.757+0000 7f549a000640 1 -- 172.21.15.18:0/1856286681 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488018ac0 con 0x7f549c109370 2024-04-16T17:16:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.757+0000 7f54a2c00640 1 -- 172.21.15.18:0/1856286681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c1a7880 con 0x7f549c109370 2024-04-16T17:16:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.761+0000 7f549a000640 1 -- 172.21.15.18:0/1856286681 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5488010ac0 con 0x7f549c109370 2024-04-16T17:16:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.761+0000 7f54a6f7f740 1 Processor -- start 2024-04-16T17:16:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.761+0000 7f54a6f7f740 1 -- start start 2024-04-16T17:16:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564213518780 con 0x5642135bd000 2024-04-16T17:16:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5642134bef40 con 0x5642135b47f0 2024-04-16T17:16:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564213514740 con 0x5642135b3a80 2024-04-16T17:16:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135bf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42626/0 (socket says 172.21.15.18:42626) 2024-04-16T17:16:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a0e00640 1 -- 172.21.15.18:0/1885926644 learned_addr learned my addr 172.21.15.18:0/1885926644 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f549a000640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54840a3d00 0x7f54840a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.762+0000 7f54a1800640 1 --2- 172.21.15.18:0/1885926644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f549a000640 1 -- 172.21.15.18:0/1856286681 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f54880d1230 con 0x7f549c109370 2024-04-16T17:16:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f54a0e00640 1 -- 172.21.15.18:0/1885926644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 msgr2=0x5642135b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f54a0e00640 1 --2- 172.21.15.18:0/1885926644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f54a0e00640 1 -- 172.21.15.18:0/1885926644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 msgr2=0x5642135b3540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:16:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f54a0e00640 1 --2- 172.21.15.18:0/1885926644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135b3540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f54a2200640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54840a3d00 0x7f54840a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f54a0e00640 1 -- 172.21.15.18:0/1885926644 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56421322b210 con 0x5642135bd000 2024-04-16T17:16:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f54a0e00640 1 --2- 172.21.15.18:0/1885926644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135bf3f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f54880a54a0 tx=0x7f5488009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48be195fcdf08551 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.763+0000 7f548ea00640 1 -- 172.21.15.18:0/1885926644 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548809cd60 con 0x5642135bd000 2024-04-16T17:16:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.764+0000 7f548ea00640 1 -- 172.21.15.18:0/1885926644 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54880a1110 con 0x5642135bd000 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.764+0000 7f548ea00640 1 -- 172.21.15.18:0/1885926644 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54880ce990 con 0x5642135bd000 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.764+0000 7f54a6f7f740 1 -- 172.21.15.18:0/1885926644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 msgr2=0x5642135bf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.764+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/1885926644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135bf3f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f54880a54a0 tx=0x7f5488009700 comp rx=0 tx=0).stop 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.764+0000 7f54a2200640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54840a3d00 0x7f54840a61c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f549400a4c0 tx=0x7f549400d040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.765+0000 7f54a6f7f740 1 -- 172.21.15.18:0/1885926644 shutdown_connections 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.765+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/1885926644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135bf3f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.765+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/1885926644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.765+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/1885926644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.765+0000 7f54a6f7f740 1 -- 172.21.15.18:0/1885926644 >> 172.21.15.18:0/1885926644 conn(0x56421354e5d0 msgr2=0x5642135ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.765+0000 7f54a6f7f740 1 -- 172.21.15.18:0/1885926644 shutdown_connections 2024-04-16T17:16:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.765+0000 7f54a6f7f740 1 -- 172.21.15.18:0/1885926644 wait complete. 2024-04-16T17:16:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.765+0000 7f54a6f7f740 1 -- start start 2024-04-16T17:16:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.766+0000 7f54a6f7f740 1 Processor -- start 2024-04-16T17:16:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.766+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.766+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135ba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.766+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.766+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5642135bfcd0 con 0x5642135bd000 2024-04-16T17:16:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.766+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5642134d2390 con 0x5642135b47f0 2024-04-16T17:16:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.766+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5642135bfe50 con 0x5642135b3a80 2024-04-16T17:16:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a6f7f740 10 client.0 _ll_register_callbacks cb 0x5642134bbf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a2200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135ba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135ba400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57794/0 (socket says 172.21.15.18:57794) 2024-04-16T17:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a2200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135ba940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42640/0 (socket says 172.21.15.18:42640) 2024-04-16T17:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a0e00640 1 -- 172.21.15.18:0/175158584 learned_addr learned my addr 172.21.15.18:0/175158584 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a1800640 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a0e00640 1 -- 172.21.15.18:0/175158584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 msgr2=0x5642135b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a0e00640 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a0e00640 1 -- 172.21.15.18:0/175158584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 msgr2=0x5642135ba940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a0e00640 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135ba940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.767+0000 7f54a0e00640 1 -- 172.21.15.18:0/175158584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642132f7620 con 0x5642135b47f0 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.768+0000 7f54a0e00640 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135ba400 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5488098e00 tx=0x7f54880d1160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821aff0dec53f366 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.768+0000 7f5483400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.768+0000 7f5483400640 1 -- 172.21.15.18:0/175158584 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54880a59d0 con 0x5642135b47f0 2024-04-16T17:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.768+0000 7f5483400640 1 -- 172.21.15.18:0/175158584 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f54880bd0b0 con 0x5642135b47f0 2024-04-16T17:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.768+0000 7f54a6f7f740 1 -- 172.21.15.18:0/175158584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54880a1110 con 0x5642135b47f0 2024-04-16T17:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.768+0000 7f5483400640 1 -- 172.21.15.18:0/175158584 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54880bd2b0 con 0x5642135b47f0 2024-04-16T17:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.768+0000 7f54a6f7f740 10 client.28964 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.769+0000 7f54a6f7f740 1 -- 172.21.15.18:0/175158584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5642135cbe40 con 0x5642135b47f0 2024-04-16T17:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.769+0000 7f5483400640 1 -- 172.21.15.18:0/175158584 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54880a1110 con 0x5642135b47f0 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.770+0000 7f5483400640 1 -- 172.21.15.18:0/175158584 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f548809be50 con 0x5642135b47f0 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.770+0000 7f54a6f7f740 10 client.28964 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.770+0000 7f54a6f7f740 10 client.28964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.770+0000 7f54a6f7f740 20 client.28964 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.770+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5642135cd190 0x5642135cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.770+0000 7f54a6f7f740 4 client.28964 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.770+0000 7f54a6f7f740 1 -- 172.21.15.18:0/175158584 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642135261a0 con 0x5642135cd190 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.770+0000 7f54a1800640 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5642135cd190 0x5642135cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.771+0000 7f54a1800640 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5642135cd190 0x5642135cf580 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b58f18e2b6dada server_cookie=d29df0bf33dcbcbd in_seq=0 out_seq=0 2024-04-16T17:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.771+0000 7f5483400640 10 client.28964 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f5483400640 1 -- 172.21.15.18:0/175158584 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642135261a0 con 0x5642135cd190 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f5483400640 10 client.28964 handle_command_reply: tid=0 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 client.28964 shutdown 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 -- 172.21.15.18:0/175158584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 msgr2=0x5642135ba400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135ba400 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5488098e00 tx=0x7f54880d1160 comp rx=0 tx=0).stop 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 -- 172.21.15.18:0/175158584 shutdown_connections 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5642135cd190 0x5642135cf580 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135ba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135ba400 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/175158584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 -- 172.21.15.18:0/175158584 >> 172.21.15.18:0/175158584 conn(0x56421354e5d0 msgr2=0x5642135b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f5483400640 0 client.28964 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 -- 172.21.15.18:0/175158584 shutdown_connections 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.772+0000 7f54a6f7f740 1 -- 172.21.15.18:0/175158584 wait complete. 2024-04-16T17:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.773+0000 7f54a6f7f740 20 client.28964 trim_cache size 0 max 16384 2024-04-16T17:16:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.789+0000 7f54a6f7f740 1 Processor -- start 2024-04-16T17:16:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.789+0000 7f54a6f7f740 1 -- start start 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.789+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.789+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135b47f0 0x5642135ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.789+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135bd000 0x5642135d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.789+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b3fd0 con 0x5642135b47f0 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.789+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b18a0 con 0x5642135bd000 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.789+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1af710 con 0x5642135b3a80 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135b47f0 0x5642135ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135b47f0 0x5642135ba760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42644/0 (socket says 172.21.15.18:42644) 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 -- 172.21.15.18:0/3313439443 learned_addr learned my addr 172.21.15.18:0/3313439443 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a2200640 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135bd000 0x5642135d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 -- 172.21.15.18:0/3313439443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 msgr2=0x5642135c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 -- 172.21.15.18:0/3313439443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135bd000 msgr2=0x5642135d4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135bd000 0x5642135d4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 -- 172.21.15.18:0/3313439443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642135a8c20 con 0x5642135b47f0 2024-04-16T17:16:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a2200640 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135bd000 0x5642135d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.790+0000 7f54a0e00640 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135b47f0 0x5642135ba760 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f54880991c0 tx=0x7f548800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2489dac51afef7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.791+0000 7f548e000640 1 -- 172.21.15.18:0/3313439443 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54880b62a0 con 0x5642135b47f0 2024-04-16T17:16:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.791+0000 7f548e000640 1 -- 172.21.15.18:0/3313439443 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54880a1110 con 0x5642135b47f0 2024-04-16T17:16:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.791+0000 7f548e000640 1 -- 172.21.15.18:0/3313439443 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548809bd10 con 0x5642135b47f0 2024-04-16T17:16:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3313439443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135b47f0 msgr2=0x5642135ba760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135b47f0 0x5642135ba760 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f54880991c0 tx=0x7f548800c890 comp rx=0 tx=0).stop 2024-04-16T17:16:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3313439443 shutdown_connections 2024-04-16T17:16:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135bd000 0x5642135d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135b47f0 0x5642135ba760 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3313439443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3313439443 >> 172.21.15.18:0/3313439443 conn(0x56421354e5d0 msgr2=0x5642135acf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3313439443 shutdown_connections 2024-04-16T17:16:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3313439443 wait complete. 2024-04-16T17:16:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.792+0000 7f54a6f7f740 1 -- start start 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.793+0000 7f54a6f7f740 1 Processor -- start 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.793+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.793+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.793+0000 7f54a6f7f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.793+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1afd60 con 0x5642135bd000 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.793+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b11d0 con 0x5642135b47f0 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.793+0000 7f54a6f7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b0430 con 0x5642135b3a80 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.793+0000 7f54a6f7f740 10 client.0 _ll_register_callbacks cb 0x7f5488005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38296/0 (socket says 172.21.15.18:38296) 2024-04-16T17:16:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 -- 172.21.15.18:0/3891371974 learned_addr learned my addr 172.21.15.18:0/3891371974 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a2200640 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 -- 172.21.15.18:0/3891371974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 msgr2=0x5642135c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 -- 172.21.15.18:0/3891371974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 msgr2=0x5642135c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 -- 172.21.15.18:0/3891371974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642135b2d50 con 0x5642135b3a80 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a2200640 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f54a1800640 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c0dd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f549002e010 tx=0x7f5490004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950f7d8b83bb1144 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.794+0000 7f5483400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:16:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.797+0000 7f5483400640 1 -- 172.21.15.18:0/3891371974 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490047bc0 con 0x5642135b3a80 2024-04-16T17:16:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.797+0000 7f5483400640 1 -- 172.21.15.18:0/3891371974 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9554+0+0 (secure 0 0 0) 0x7f5490047d60 con 0x5642135b3a80 2024-04-16T17:16:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.797+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3891371974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54880a1110 con 0x5642135b3a80 2024-04-16T17:16:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.797+0000 7f5483400640 1 -- 172.21.15.18:0/3891371974 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5490055050 con 0x5642135b3a80 2024-04-16T17:16:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a6f7f740 10 client.28810 fetch_fsmap learned FSMap version 33 2024-04-16T17:16:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3891371974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564213633800 con 0x5642135b3a80 2024-04-16T17:16:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f5483400640 1 -- 172.21.15.18:0/3891371974 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54880a1110 con 0x5642135b3a80 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f5483400640 1 -- 172.21.15.18:0/3891371974 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5490047560 con 0x5642135b3a80 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a6f7f740 10 client.28810 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a6f7f740 10 client.28810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a6f7f740 20 client.28810 populate_metadata read hostname 'smithi018' 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564213633f80 0x5642136363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a6f7f740 4 client.28810 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3891371974 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564213520010 con 0x564213633f80 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.798+0000 7f54a0e00640 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564213633f80 0x5642136363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.799+0000 7f54a0e00640 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564213633f80 0x5642136363a0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d4d77859ca6162 server_cookie=e0e574e8ed2a7d87 in_seq=0 out_seq=0 2024-04-16T17:16:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.799+0000 7f5483400640 10 client.28810 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:52.881 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:51 smithi018 ceph-mon[31776]: pgmap v383: 129 pgs: 129 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 996 KiB/s wr, 136 op/s 2024-04-16T17:16:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.973+0000 7f5483400640 1 -- 172.21.15.18:0/3891371974 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564213520010 con 0x564213633f80 2024-04-16T17:16:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.973+0000 7f5483400640 10 client.28810 handle_command_reply: tid=0 2024-04-16T17:16:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.973+0000 7f54a6f7f740 1 client.28810 shutdown 2024-04-16T17:16:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.973+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3891371974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 msgr2=0x5642135c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.973+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c0dd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f549002e010 tx=0x7f5490004410 comp rx=0 tx=0).stop 2024-04-16T17:16:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.974+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3891371974 shutdown_connections 2024-04-16T17:16:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.974+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564213633f80 0x5642136363a0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.974+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642135bd000 0x5642135c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.974+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642135b47f0 0x5642135c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.974+0000 7f54a6f7f740 1 --2- 172.21.15.18:0/3891371974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642135b3a80 0x5642135c0dd0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.974+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3891371974 >> 172.21.15.18:0/3891371974 conn(0x56421354e5d0 msgr2=0x5642135acf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.974+0000 7f5483400640 0 client.28810 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:16:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.974+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3891371974 shutdown_connections 2024-04-16T17:16:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.977+0000 7f54a6f7f740 1 -- 172.21.15.18:0/3891371974 wait complete. 2024-04-16T17:16:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:16:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:16:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:16:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:16:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.978+0000 7f54a6f7f740 20 client.28810 trim_cache size 0 max 16384 2024-04-16T17:16:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 -- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54840a3d00 msgr2=0x7f54840a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54840a3d00 0x7f54840a61c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f549400a4c0 tx=0x7f549400d040 comp rx=0 tx=0).stop 2024-04-16T17:16:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 -- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c109370 msgr2=0x7f549c1a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c109370 0x7f549c1a0bf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f548800ea30 tx=0x7f548800eef0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 -- 172.21.15.18:0/1856286681 shutdown_connections 2024-04-16T17:16:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54840a3d00 0x7f54840a61c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f549c19a870 0x7f549c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f549c109370 0x7f549c1a0bf0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 --2- 172.21.15.18:0/1856286681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f549c1042a0 0x7f549c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 -- 172.21.15.18:0/1856286681 >> 172.21.15.18:0/1856286681 conn(0x7f549c0ffb00 msgr2=0x7f549c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.981+0000 7f54a2c00640 1 -- 172.21.15.18:0/1856286681 shutdown_connections 2024-04-16T17:16:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:16:52.982+0000 7f54a2c00640 1 -- 172.21.15.18:0/1856286681 wait complete. 2024-04-16T17:16:53.671 INFO:tasks.workunit.client.0.smithi018.stdout: 1 455489 3.58 MB/sec execute 546 sec latency 138.387 ms 2024-04-16T17:16:54.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:53 smithi022 ceph-mon[36023]: pgmap v384: 129 pgs: 129 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-04-16T17:16:54.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:53 smithi043 ceph-mon[36901]: pgmap v384: 129 pgs: 129 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-04-16T17:16:54.671 INFO:tasks.workunit.client.0.smithi018.stdout: 1 456200 3.58 MB/sec execute 547 sec latency 33.928 ms 2024-04-16T17:16:55.671 INFO:tasks.workunit.client.0.smithi018.stdout: 1 457334 3.58 MB/sec execute 548 sec latency 63.777 ms 2024-04-16T17:16:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:55 smithi022 ceph-mon[36023]: pgmap v385: 129 pgs: 129 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 888 KiB/s wr, 76 op/s 2024-04-16T17:16:56.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:55 smithi043 ceph-mon[36901]: pgmap v385: 129 pgs: 129 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 888 KiB/s wr, 76 op/s 2024-04-16T17:16:56.671 INFO:tasks.workunit.client.0.smithi018.stdout: 1 457866 3.59 MB/sec execute 549 sec latency 62.037 ms 2024-04-16T17:16:57.678 INFO:tasks.workunit.client.0.smithi018.stdout: 1 458040 3.58 MB/sec execute 550 sec latency 57.901 ms 2024-04-16T17:16:58.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:57 smithi022 ceph-mon[36023]: pgmap v386: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-04-16T17:16:58.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:57 smithi043 ceph-mon[36901]: pgmap v386: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-04-16T17:16:58.679 INFO:tasks.workunit.client.0.smithi018.stdout: 1 458344 3.57 MB/sec execute 551 sec latency 110.108 ms 2024-04-16T17:16:59.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:53 smithi018 ceph-mon[31776]: pgmap v384: 129 pgs: 129 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-04-16T17:16:59.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:55 smithi018 ceph-mon[31776]: pgmap v385: 129 pgs: 129 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 888 KiB/s wr, 76 op/s 2024-04-16T17:16:59.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:57 smithi018 ceph-mon[31776]: pgmap v386: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-04-16T17:16:59.678 INFO:tasks.workunit.client.0.smithi018.stdout: 1 458821 3.60 MB/sec execute 552 sec latency 98.685 ms 2024-04-16T17:17:00.072 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:16:59 smithi018 ceph-mon[31776]: pgmap v387: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 814 KiB/s wr, 119 op/s 2024-04-16T17:17:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:16:59 smithi022 ceph-mon[36023]: pgmap v387: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 814 KiB/s wr, 119 op/s 2024-04-16T17:17:00.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:16:59 smithi043 ceph-mon[36901]: pgmap v387: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 814 KiB/s wr, 119 op/s 2024-04-16T17:17:00.678 INFO:tasks.workunit.client.0.smithi018.stdout: 1 459575 3.60 MB/sec execute 553 sec latency 85.529 ms 2024-04-16T17:17:01.679 INFO:tasks.workunit.client.0.smithi018.stdout: 1 459987 3.60 MB/sec execute 554 sec latency 65.344 ms 2024-04-16T17:17:02.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:01 smithi022 ceph-mon[36023]: pgmap v388: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-16T17:17:02.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:01 smithi043 ceph-mon[36901]: pgmap v388: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-16T17:17:02.679 INFO:tasks.workunit.client.0.smithi018.stdout: 1 460320 3.60 MB/sec execute 555 sec latency 60.851 ms 2024-04-16T17:17:03.679 INFO:tasks.workunit.client.0.smithi018.stdout: 1 460616 3.59 MB/sec execute 556 sec latency 59.823 ms 2024-04-16T17:17:03.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:01 smithi018 ceph-mon[31776]: pgmap v388: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-16T17:17:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:03 smithi018 ceph-mon[31776]: pgmap v389: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-04-16T17:17:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:03 smithi022 ceph-mon[36023]: pgmap v389: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-04-16T17:17:04.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:03 smithi043 ceph-mon[36901]: pgmap v389: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-04-16T17:17:04.679 INFO:tasks.workunit.client.0.smithi018.stdout: 1 460629 3.58 MB/sec execute 557 sec latency 593.914 ms 2024-04-16T17:17:04.896 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:04 smithi018 ceph-mon[31776]: osdmap e112: 12 total, 12 up, 12 in 2024-04-16T17:17:05.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:04 smithi022 ceph-mon[36023]: osdmap e112: 12 total, 12 up, 12 in 2024-04-16T17:17:05.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:04 smithi043 ceph-mon[36901]: osdmap e112: 12 total, 12 up, 12 in 2024-04-16T17:17:05.679 INFO:tasks.workunit.client.0.smithi018.stdout: 1 460633 3.58 MB/sec execute 558 sec latency 808.338 ms 2024-04-16T17:17:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3144406859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 msgr2=0x7fd5a4106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3144406859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 0x7fd5a4106660 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009ca0 tx=0x7fd59c02c1e0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3144406859 shutdown_connections 2024-04-16T17:17:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3144406859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 0x7fd5a4106660 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3144406859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5a41016f0 0x7fd5a4105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3144406859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5a4100dd0 0x7fd5a41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3144406859 >> 172.21.15.18:0/3144406859 conn(0x7fd5a40fc7e0 msgr2=0x7fd5a40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3144406859 shutdown_connections 2024-04-16T17:17:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.145+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3144406859 wait complete. 2024-04-16T17:17:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.147+0000 7fd5a8a00640 1 Processor -- start 2024-04-16T17:17:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.147+0000 7fd5a8a00640 1 -- start start 2024-04-16T17:17:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.147+0000 7fd5a8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5a4100dd0 0x7fd5a41a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.147+0000 7fd5a8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5a41016f0 0x7fd5a41a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.147+0000 7fd5a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 0x7fd5a419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.147+0000 7fd5a8a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41187d0 con 0x7fd5a4100dd0 2024-04-16T17:17:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.147+0000 7fd5a8a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4118650 con 0x7fd5a41016f0 2024-04-16T17:17:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.147+0000 7fd5a8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4118950 con 0x7fd5a41061d0 2024-04-16T17:17:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5a41016f0 0x7fd5a41a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 0x7fd5a419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5a41016f0 0x7fd5a41a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48856/0 (socket says 172.21.15.18:48856) 2024-04-16T17:17:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 -- 172.21.15.18:0/3259300710 learned_addr learned my addr 172.21.15.18:0/3259300710 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a2800640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5a4100dd0 0x7fd5a41a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 -- 172.21.15.18:0/3259300710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 msgr2=0x7fd5a419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 0x7fd5a419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 -- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5a4100dd0 msgr2=0x7fd5a41a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5a4100dd0 0x7fd5a41a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 -- 172.21.15.18:0/3259300710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a419f230 con 0x7fd5a41016f0 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a3200640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 0x7fd5a419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.148+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5a41016f0 0x7fd5a41a4e40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e9f0 tx=0x7fd59000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.149+0000 7fd59b400640 1 -- 172.21.15.18:0/3259300710 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59000ccc0 con 0x7fd5a41016f0 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.149+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3259300710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a419f520 con 0x7fd5a41016f0 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.149+0000 7fd59b400640 1 -- 172.21.15.18:0/3259300710 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd590010910 con 0x7fd5a41016f0 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.149+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3259300710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a41ab7c0 con 0x7fd5a41016f0 2024-04-16T17:17:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.149+0000 7fd59b400640 1 -- 172.21.15.18:0/3259300710 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900188e0 con 0x7fd5a41016f0 2024-04-16T17:17:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.153+0000 7fd5ac1c0740 1 Processor -- start 2024-04-16T17:17:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5ac1c0740 1 -- start start 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbaca80 0x56324bbacf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56324bb11780 con 0x56324bbb6000 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56324bab7f40 con 0x56324bbaca80 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56324bb0d740 con 0x56324bbad7f0 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd59b400640 1 -- 172.21.15.18:0/3259300710 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd590010ac0 con 0x7fd5a41016f0 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5a3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5a3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53382/0 (socket says 172.21.15.18:53382) 2024-04-16T17:17:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.154+0000 7fd5a3200640 1 -- 172.21.15.18:0/1169681013 learned_addr learned my addr 172.21.15.18:0/1169681013 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.155+0000 7fd5a3200640 1 -- 172.21.15.18:0/1169681013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbaca80 msgr2=0x56324bbacf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.155+0000 7fd5a3200640 1 --2- 172.21.15.18:0/1169681013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbaca80 0x56324bbacf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.155+0000 7fd5a3200640 1 -- 172.21.15.18:0/1169681013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 msgr2=0x56324bbb83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.155+0000 7fd5a3200640 1 --2- 172.21.15.18:0/1169681013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.155+0000 7fd5a3200640 1 -- 172.21.15.18:0/1169681013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324b824210 con 0x56324bbad7f0 2024-04-16T17:17:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.155+0000 7fd5a3200640 1 --2- 172.21.15.18:0/1169681013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbac540 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd59c02c7f0 tx=0x7fd59c005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b15a90231bf2563a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.156+0000 7fd58f400640 1 -- 172.21.15.18:0/1169681013 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c043070 con 0x56324bbad7f0 2024-04-16T17:17:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.156+0000 7fd58f400640 1 -- 172.21.15.18:0/1169681013 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59c003e50 con 0x56324bbad7f0 2024-04-16T17:17:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.156+0000 7fd58f400640 1 -- 172.21.15.18:0/1169681013 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c039040 con 0x56324bbad7f0 2024-04-16T17:17:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.156+0000 7fd59b400640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd5880a3d00 0x7fd5880a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.156+0000 7fd59b400640 1 -- 172.21.15.18:0/3259300710 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9642+0+0 (secure 0 0 0) 0x7fd5900d0ef0 con 0x7fd5a41016f0 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.156+0000 7fd5a2800640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd5880a3d00 0x7fd5880a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1169681013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 msgr2=0x56324bbac540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1169681013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbac540 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd59c02c7f0 tx=0x7fd59c005f80 comp rx=0 tx=0).stop 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5a2800640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd5880a3d00 0x7fd5880a61c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x56324bbadf50 tx=0x7fd5940097f0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1169681013 shutdown_connections 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1169681013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1169681013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbaca80 0x56324bbacf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1169681013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbac540 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1169681013 >> 172.21.15.18:0/1169681013 conn(0x56324bb475d0 msgr2=0x56324bba7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1169681013 shutdown_connections 2024-04-16T17:17:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.157+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1169681013 wait complete. 2024-04-16T17:17:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.158+0000 7fd5ac1c0740 1 -- start start 2024-04-16T17:17:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.158+0000 7fd5ac1c0740 1 Processor -- start 2024-04-16T17:17:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.158+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.158+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbb3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53398/0 (socket says 172.21.15.18:53398) 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbb3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbb3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48878/0 (socket says 172.21.15.18:48878) 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a2800640 1 -- 172.21.15.18:0/2197957049 learned_addr learned my addr 172.21.15.18:0/2197957049 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56324bbb8cd0 con 0x56324bbb6000 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56324bacb390 con 0x56324bbad7f0 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56324bbb8e50 con 0x56324bbaca80 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a3200640 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5ac1c0740 10 client.0 _ll_register_callbacks cb 0x56324bab4f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a1e00640 1 -- 172.21.15.18:0/2197957049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 msgr2=0x56324bbb2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a1e00640 1 -- 172.21.15.18:0/2197957049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 msgr2=0x56324bbb3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a1e00640 1 -- 172.21.15.18:0/2197957049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324b8f0620 con 0x56324bbad7f0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.159+0000 7fd5a2800640 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.160+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbb3420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ce1e0 tx=0x7fd59009c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150951b1597fc90a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.160+0000 7fd583e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.160+0000 7fd583e00640 1 -- 172.21.15.18:0/2197957049 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59009c080 con 0x56324bbad7f0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.160+0000 7fd583e00640 1 -- 172.21.15.18:0/2197957049 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9642+0+0 (secure 0 0 0) 0x7fd590010430 con 0x56324bbad7f0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.160+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd59c003e50 con 0x56324bbad7f0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.160+0000 7fd583e00640 1 -- 172.21.15.18:0/2197957049 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd59009ca30 con 0x56324bbad7f0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.160+0000 7fd5ac1c0740 10 client.28979 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.160+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56324bbc4e40 con 0x56324bbad7f0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.161+0000 7fd583e00640 1 -- 172.21.15.18:0/2197957049 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59c003e50 con 0x56324bbad7f0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.161+0000 7fd583e00640 1 -- 172.21.15.18:0/2197957049 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd590098780 con 0x56324bbad7f0 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.161+0000 7fd5ac1c0740 10 client.28979 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.161+0000 7fd5ac1c0740 10 client.28979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.161+0000 7fd5ac1c0740 20 client.28979 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.161+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56324bbc62b0 0x56324bbc86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.161+0000 7fd5ac1c0740 4 client.28979 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.161+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56324bb1f1a0 con 0x56324bbc62b0 2024-04-16T17:17:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.162+0000 7fd5a2800640 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56324bbc62b0 0x56324bbc86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.162+0000 7fd5a2800640 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56324bbc62b0 0x56324bbc86a0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c22bd1aa8d03bca server_cookie=cd20d52a18957f0a in_seq=0 out_seq=0 2024-04-16T17:17:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.162+0000 7fd583e00640 10 client.28979 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd583e00640 1 -- 172.21.15.18:0/2197957049 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56324bb1f1a0 con 0x56324bbc62b0 2024-04-16T17:17:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd583e00640 10 client.28979 handle_command_reply: tid=0 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 client.28979 shutdown 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 msgr2=0x56324bbb3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbb3420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ce1e0 tx=0x7fd59009c5f0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 shutdown_connections 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56324bbc62b0 0x56324bbc86a0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbb3420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd583e00640 0 client.28979 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/2197957049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.163+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 >> 172.21.15.18:0/2197957049 conn(0x56324bb475d0 msgr2=0x56324bba94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.164+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 shutdown_connections 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.164+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/2197957049 wait complete. 2024-04-16T17:17:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.164+0000 7fd5ac1c0740 20 client.28979 trim_cache size 0 max 16384 2024-04-16T17:17:06.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:05 smithi022 ceph-mon[36023]: pgmap v391: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 169 op/s 2024-04-16T17:17:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.181+0000 7fd5ac1c0740 1 Processor -- start 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.181+0000 7fd5ac1c0740 1 -- start start 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbc1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41b8480 con 0x56324bbb6000 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41b5d50 con 0x56324bbad7f0 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41b3bc0 con 0x56324bbaca80 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53404/0 (socket says 172.21.15.18:53404) 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a2800640 1 -- 172.21.15.18:0/1913949483 learned_addr learned my addr 172.21.15.18:0/1913949483 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a3200640 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbc1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a2800640 1 -- 172.21.15.18:0/1913949483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 msgr2=0x56324bbc1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a2800640 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbc1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a2800640 1 -- 172.21.15.18:0/1913949483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 msgr2=0x56324bbb2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a2800640 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.182+0000 7fd5a2800640 1 -- 172.21.15.18:0/1913949483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324bbabd50 con 0x56324bbaca80 2024-04-16T17:17:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.183+0000 7fd5a2800640 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb5610 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd594009b10 tx=0x7fd594014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c0067a34d290b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.183+0000 7fd58ea00640 1 -- 172.21.15.18:0/1913949483 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59402d070 con 0x56324bbaca80 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.183+0000 7fd58ea00640 1 -- 172.21.15.18:0/1913949483 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd594028870 con 0x56324bbaca80 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.183+0000 7fd58ea00640 1 -- 172.21.15.18:0/1913949483 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59401fce0 con 0x56324bbaca80 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1913949483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 msgr2=0x56324bbb5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb5610 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd594009b10 tx=0x7fd594014000 comp rx=0 tx=0).stop 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1913949483 shutdown_connections 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbb6000 0x56324bbb2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbad7f0 0x56324bbc1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1913949483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbaca80 0x56324bbb5610 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1913949483 >> 172.21.15.18:0/1913949483 conn(0x56324bb475d0 msgr2=0x56324bbb2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1913949483 shutdown_connections 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1913949483 wait complete. 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 -- start start 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.184+0000 7fd5ac1c0740 1 Processor -- start 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbaca80 0x56324bbafa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbaffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5ac1c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbb6000 0x56324bbb04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41b4210 con 0x56324bbaca80 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41b5680 con 0x56324bbb6000 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5ac1c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41b48e0 con 0x56324bbad7f0 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5ac1c0740 10 client.0 _ll_register_callbacks cb 0x7fd59000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbaffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5a2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbaca80 0x56324bbafa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbaffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53406/0 (socket says 172.21.15.18:53406) 2024-04-16T17:17:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5a1e00640 1 -- 172.21.15.18:0/1074635836 learned_addr learned my addr 172.21.15.18:0/1074635836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5a3200640 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbb6000 0x56324bbb04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5a1e00640 1 -- 172.21.15.18:0/1074635836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbb6000 msgr2=0x56324bbb04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.185+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbb6000 0x56324bbb04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd5a1e00640 1 -- 172.21.15.18:0/1074635836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbaca80 msgr2=0x56324bbafa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbaca80 0x56324bbafa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd5a1e00640 1 -- 172.21.15.18:0/1074635836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324bba9a70 con 0x56324bbad7f0 2024-04-16T17:17:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd5a1e00640 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbaffa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd59000cf10 tx=0x7fd59000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db049ea4acaca94 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd583e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd583e00640 1 -- 172.21.15.18:0/1074635836 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59009c800 con 0x56324bbad7f0 2024-04-16T17:17:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd583e00640 1 -- 172.21.15.18:0/1074635836 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9642+0+0 (secure 0 0 0) 0x7fd59009b8f0 con 0x56324bbad7f0 2024-04-16T17:17:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1074635836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd594028870 con 0x56324bbad7f0 2024-04-16T17:17:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd583e00640 1 -- 172.21.15.18:0/1074635836 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd590098cd0 con 0x56324bbad7f0 2024-04-16T17:17:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd5ac1c0740 10 client.28828 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.186+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1074635836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56324bc2c830 con 0x56324bbad7f0 2024-04-16T17:17:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.187+0000 7fd583e00640 1 -- 172.21.15.18:0/1074635836 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd594028870 con 0x56324bbad7f0 2024-04-16T17:17:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.187+0000 7fd583e00640 1 -- 172.21.15.18:0/1074635836 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd5900a1f70 con 0x56324bbad7f0 2024-04-16T17:17:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.187+0000 7fd5ac1c0740 10 client.28828 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.187+0000 7fd5ac1c0740 10 client.28828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.187+0000 7fd5ac1c0740 20 client.28828 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.187+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56324bc2cfe0 0x56324bc2f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.187+0000 7fd5ac1c0740 4 client.28828 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:17:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.187+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1074635836 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56324bb19010 con 0x56324bc2cfe0 2024-04-16T17:17:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.225+0000 7fd5a2800640 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56324bc2cfe0 0x56324bc2f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.226+0000 7fd5a2800640 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56324bc2cfe0 0x56324bc2f400 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca7a1cd70413c89 server_cookie=fa842aba0c159c9 in_seq=0 out_seq=0 2024-04-16T17:17:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.226+0000 7fd583e00640 10 client.28828 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:06.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:05 smithi043 ceph-mon[36901]: pgmap v391: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 169 op/s 2024-04-16T17:17:06.262 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:05 smithi018 ceph-mon[31776]: pgmap v391: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 169 op/s 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.381+0000 7fd583e00640 1 -- 172.21.15.18:0/1074635836 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56324bb19010 con 0x56324bc2cfe0 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.381+0000 7fd583e00640 10 client.28828 handle_command_reply: tid=0 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.381+0000 7fd5ac1c0740 1 client.28828 shutdown 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1074635836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 msgr2=0x56324bbaffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbaffa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd59000cf10 tx=0x7fd59000cf60 comp rx=0 tx=0).stop 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1074635836 shutdown_connections 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56324bc2cfe0 0x56324bc2f400 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56324bbb6000 0x56324bbb04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56324bbad7f0 0x56324bbaffa0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 --2- 172.21.15.18:0/1074635836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56324bbaca80 0x56324bbafa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1074635836 >> 172.21.15.18:0/1074635836 conn(0x56324bb475d0 msgr2=0x56324bbb1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd583e00640 0 client.28828 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1074635836 shutdown_connections 2024-04-16T17:17:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.382+0000 7fd5ac1c0740 1 -- 172.21.15.18:0/1074635836 wait complete. 2024-04-16T17:17:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.383+0000 7fd5ac1c0740 20 client.28828 trim_cache size 0 max 16384 2024-04-16T17:17:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:17:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:17:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:17:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:17:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.383+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd5880a3d00 msgr2=0x7fd5880a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.383+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd5880a3d00 0x7fd5880a61c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x56324bbadf50 tx=0x7fd5940097f0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.383+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3259300710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5a41016f0 msgr2=0x7fd5a41a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.383+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5a41016f0 0x7fd5a41a4e40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e9f0 tx=0x7fd59000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.384+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3259300710 shutdown_connections 2024-04-16T17:17:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.384+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd5880a3d00 0x7fd5880a61c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.384+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a41061d0 0x7fd5a419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.384+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5a41016f0 0x7fd5a41a4e40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.384+0000 7fd5a8a00640 1 --2- 172.21.15.18:0/3259300710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5a4100dd0 0x7fd5a41a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.384+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3259300710 >> 172.21.15.18:0/3259300710 conn(0x7fd5a40fc7e0 msgr2=0x7fd5a40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.384+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3259300710 shutdown_connections 2024-04-16T17:17:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:06.384+0000 7fd5a8a00640 1 -- 172.21.15.18:0/3259300710 wait complete. 2024-04-16T17:17:06.682 INFO:tasks.workunit.client.0.smithi018.stdout: 1 461234 3.58 MB/sec execute 559 sec latency 82.567 ms 2024-04-16T17:17:07.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 461674 3.58 MB/sec execute 560 sec latency 63.592 ms 2024-04-16T17:17:08.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:07 smithi022 ceph-mon[36023]: pgmap v392: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-04-16T17:17:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:07 smithi043 ceph-mon[36901]: pgmap v392: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-04-16T17:17:08.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 462132 3.58 MB/sec execute 561 sec latency 77.523 ms 2024-04-16T17:17:09.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 462638 3.58 MB/sec execute 562 sec latency 20.877 ms 2024-04-16T17:17:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:09 smithi022 ceph-mon[36023]: pgmap v393: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-04-16T17:17:10.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:09 smithi043 ceph-mon[36901]: pgmap v393: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-04-16T17:17:10.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 463462 3.58 MB/sec execute 563 sec latency 30.774 ms 2024-04-16T17:17:11.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 464228 3.57 MB/sec execute 564 sec latency 60.243 ms 2024-04-16T17:17:12.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:11 smithi022 ceph-mon[36023]: pgmap v394: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-16T17:17:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:11 smithi043 ceph-mon[36901]: pgmap v394: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-16T17:17:12.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 464510 3.57 MB/sec execute 565 sec latency 53.531 ms 2024-04-16T17:17:13.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 465169 3.58 MB/sec execute 566 sec latency 75.089 ms 2024-04-16T17:17:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:13 smithi022 ceph-mon[36023]: pgmap v395: 129 pgs: 129 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-16T17:17:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:13 smithi022 ceph-mon[36023]: osdmap e113: 12 total, 12 up, 12 in 2024-04-16T17:17:14.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:13 smithi043 ceph-mon[36901]: pgmap v395: 129 pgs: 129 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-16T17:17:14.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:13 smithi043 ceph-mon[36901]: osdmap e113: 12 total, 12 up, 12 in 2024-04-16T17:17:14.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 465911 3.58 MB/sec execute 567 sec latency 18.130 ms 2024-04-16T17:17:15.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:14 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:17:15.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:14 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:17:15.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:17:15.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:17:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 -- 172.21.15.18:0/2682211174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa46c0743b0 msgr2=0x7fa46c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 --2- 172.21.15.18:0/2682211174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa46c0743b0 0x7fa46c074790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009900 tx=0x7fa45c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 -- 172.21.15.18:0/2682211174 shutdown_connections 2024-04-16T17:17:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 --2- 172.21.15.18:0/2682211174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c10e6c0 0x7fa46c110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 --2- 172.21.15.18:0/2682211174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa46c074cd0 0x7fa46c10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 --2- 172.21.15.18:0/2682211174 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa46c0743b0 0x7fa46c074790 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 -- 172.21.15.18:0/2682211174 >> 172.21.15.18:0/2682211174 conn(0x7fa46c0fd950 msgr2=0x7fa46c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:15.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 -- 172.21.15.18:0/2682211174 shutdown_connections 2024-04-16T17:17:15.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.548+0000 7fa470a00640 1 -- 172.21.15.18:0/2682211174 wait complete. 2024-04-16T17:17:15.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.549+0000 7fa470a00640 1 Processor -- start 2024-04-16T17:17:15.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.549+0000 7fa470a00640 1 -- start start 2024-04-16T17:17:15.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa470a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c0743b0 0x7fa46c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa470a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa46c074cd0 0x7fa46c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa470a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa46c10e6c0 0x7fa46c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa470a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1143d0 con 0x7fa46c10e6c0 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa470a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c114250 con 0x7fa46c074cd0 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa470a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c114550 con 0x7fa46c0743b0 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa469e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa46c074cd0 0x7fa46c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa469e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa46c074cd0 0x7fa46c1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46174/0 (socket says 172.21.15.18:46174) 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa469e00640 1 -- 172.21.15.18:0/2128818787 learned_addr learned my addr 172.21.15.18:0/2128818787 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa46b200640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa46c10e6c0 0x7fa46c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa46a800640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c0743b0 0x7fa46c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa469e00640 1 -- 172.21.15.18:0/2128818787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c0743b0 msgr2=0x7fa46c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa469e00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c0743b0 0x7fa46c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa469e00640 1 -- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa46c10e6c0 msgr2=0x7fa46c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.550+0000 7fa469e00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa46c10e6c0 0x7fa46c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.551+0000 7fa469e00640 1 -- 172.21.15.18:0/2128818787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c19afd0 con 0x7fa46c074cd0 2024-04-16T17:17:15.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.551+0000 7fa469e00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa46c074cd0 0x7fa46c1a0b90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa45800ea60 tx=0x7fa45800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:15.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.551+0000 7fa463400640 1 -- 172.21.15.18:0/2128818787 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45800cde0 con 0x7fa46c074cd0 2024-04-16T17:17:15.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.551+0000 7fa463400640 1 -- 172.21.15.18:0/2128818787 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4580035f0 con 0x7fa46c074cd0 2024-04-16T17:17:15.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.551+0000 7fa470a00640 1 -- 172.21.15.18:0/2128818787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46c19b2c0 con 0x7fa46c074cd0 2024-04-16T17:17:15.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.551+0000 7fa463400640 1 -- 172.21.15.18:0/2128818787 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458010670 con 0x7fa46c074cd0 2024-04-16T17:17:15.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.551+0000 7fa470a00640 1 -- 172.21.15.18:0/2128818787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa46c1a75d0 con 0x7fa46c074cd0 2024-04-16T17:17:15.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.555+0000 7fa47422b740 1 Processor -- start 2024-04-16T17:17:15.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.555+0000 7fa47422b740 1 -- start start 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa47422b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9177770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa47422b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf9180350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa47422b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9180890 0x560cf9182c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa47422b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560cf90dd120 con 0x560cf9177390 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa47422b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560cf9083340 con 0x560cf9180890 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa47422b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560cf90d90e0 con 0x560cf9177cb0 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf9180350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9177770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf9180350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52488/0 (socket says 172.21.15.18:52488) 2024-04-16T17:17:15.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46a800640 1 -- 172.21.15.18:0/76525171 learned_addr learned my addr 172.21.15.18:0/76525171 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:15.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa463400640 1 -- 172.21.15.18:0/2128818787 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4580037a0 con 0x7fa46c074cd0 2024-04-16T17:17:15.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46a800640 1 -- 172.21.15.18:0/76525171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9180890 msgr2=0x560cf9182c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:17:15.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46a800640 1 --2- 172.21.15.18:0/76525171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9180890 0x560cf9182c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46a800640 1 -- 172.21.15.18:0/76525171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 msgr2=0x560cf9177770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:15.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46a800640 1 --2- 172.21.15.18:0/76525171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9177770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.556+0000 7fa46a800640 1 -- 172.21.15.18:0/76525171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cf8eb97d0 con 0x560cf9177cb0 2024-04-16T17:17:15.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.557+0000 7fa46a800640 1 --2- 172.21.15.18:0/76525171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf9180350 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa45c000c00 tx=0x7fa45c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530f05aab23f3e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:15.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.557+0000 7fa457400640 1 -- 172.21.15.18:0/76525171 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c042070 con 0x560cf9177cb0 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.557+0000 7fa457400640 1 -- 172.21.15.18:0/76525171 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa45c02cb20 con 0x560cf9177cb0 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.557+0000 7fa457400640 1 -- 172.21.15.18:0/76525171 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c0046c0 con 0x560cf9177cb0 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa463400640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa4500a3da0 0x7fa4500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa463400640 1 -- 172.21.15.18:0/2128818787 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fa4580d0f70 con 0x7fa46c074cd0 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa47422b740 1 -- 172.21.15.18:0/76525171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 msgr2=0x560cf9180350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa47422b740 1 --2- 172.21.15.18:0/76525171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf9180350 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa45c000c00 tx=0x7fa45c005dc0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa47422b740 1 -- 172.21.15.18:0/76525171 shutdown_connections 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa47422b740 1 --2- 172.21.15.18:0/76525171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9180890 0x560cf9182c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa46b200640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa4500a3da0 0x7fa4500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa47422b740 1 --2- 172.21.15.18:0/76525171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf9180350 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa47422b740 1 --2- 172.21.15.18:0/76525171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9177770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.558+0000 7fa47422b740 1 -- 172.21.15.18:0/76525171 >> 172.21.15.18:0/76525171 conn(0x560cf9171ac0 msgr2=0x560cf9171ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.559+0000 7fa47422b740 1 -- 172.21.15.18:0/76525171 shutdown_connections 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.559+0000 7fa47422b740 1 -- 172.21.15.18:0/76525171 wait complete. 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.559+0000 7fa47422b740 1 -- start start 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.559+0000 7fa46b200640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa4500a3da0 0x7fa4500a6260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa464005fd0 tx=0x7fa464009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.559+0000 7fa47422b740 1 Processor -- start 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa47422b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf917d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa47422b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf917de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa47422b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 0x560cf917e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa47422b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560cf9183620 con 0x560cf9177390 2024-04-16T17:17:15.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa47422b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560cf9096710 con 0x560cf9177cb0 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa47422b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560cf91837a0 con 0x560cf9180890 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa47422b740 10 client.0 _ll_register_callbacks cb 0x560cf9091af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa46a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf917d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa46a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf917d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36932/0 (socket says 172.21.15.18:36932) 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa46a800640 1 -- 172.21.15.18:0/1575994645 learned_addr learned my addr 172.21.15.18:0/1575994645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa469e00640 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf917de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.560+0000 7fa46b200640 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 0x560cf917e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa46a800640 1 -- 172.21.15.18:0/1575994645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 msgr2=0x560cf917e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa46a800640 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 0x560cf917e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa46a800640 1 -- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 msgr2=0x560cf917de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa46a800640 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf917de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa46a800640 1 -- 172.21.15.18:0/1575994645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cf8def3f0 con 0x560cf9177390 2024-04-16T17:17:15.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa469e00640 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf917de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:15.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa46a800640 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf917d950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa45c005300 tx=0x7fa45c034a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecbe8cfdf157605 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:15.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa44be00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:17:15.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa44be00640 1 -- 172.21.15.18:0/1575994645 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c042050 con 0x560cf9177390 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.561+0000 7fa44be00640 1 -- 172.21.15.18:0/1575994645 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fa45c005500 con 0x560cf9177390 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.562+0000 7fa47422b740 1 -- 172.21.15.18:0/1575994645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa45c02cb20 con 0x560cf9177390 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.562+0000 7fa44be00640 1 -- 172.21.15.18:0/1575994645 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa45c034c40 con 0x560cf9177390 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.562+0000 7fa47422b740 10 client.18687 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.562+0000 7fa47422b740 1 -- 172.21.15.18:0/1575994645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560cf8daf0f0 con 0x560cf9177390 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.562+0000 7fa44be00640 1 -- 172.21.15.18:0/1575994645 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa45c02cb20 con 0x560cf9177390 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.563+0000 7fa44be00640 1 -- 172.21.15.18:0/1575994645 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa45c0046c0 con 0x560cf9177390 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.563+0000 7fa47422b740 10 client.18687 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.563+0000 7fa47422b740 10 client.18687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.563+0000 7fa47422b740 20 client.18687 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:15.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.563+0000 7fa47422b740 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cf9199310 0x560cf919b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.563+0000 7fa47422b740 4 client.18687 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.563+0000 7fa47422b740 1 -- 172.21.15.18:0/1575994645 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cf90eab40 con 0x560cf9199310 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.563+0000 7fa469e00640 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cf9199310 0x560cf919b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.564+0000 7fa469e00640 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cf9199310 0x560cf919b700 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1e81c9335897 server_cookie=8734bd7c9237d1b8 in_seq=0 out_seq=0 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.564+0000 7fa44be00640 10 client.18687 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.564+0000 7fa44be00640 1 -- 172.21.15.18:0/1575994645 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cf90eab40 con 0x560cf9199310 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.564+0000 7fa44be00640 10 client.18687 handle_command_reply: tid=0 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 client.18687 shutdown 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 -- 172.21.15.18:0/1575994645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 msgr2=0x560cf917d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf917d950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa45c005300 tx=0x7fa45c034a50 comp rx=0 tx=0).stop 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 -- 172.21.15.18:0/1575994645 shutdown_connections 2024-04-16T17:17:15.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cf9199310 0x560cf919b700 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 0x560cf917e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf917de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 --2- 172.21.15.18:0/1575994645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf917d950 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 -- 172.21.15.18:0/1575994645 >> 172.21.15.18:0/1575994645 conn(0x560cf9171ac0 msgr2=0x560cf9173810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:15.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa44be00640 0 client.18687 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:15.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 -- 172.21.15.18:0/1575994645 shutdown_connections 2024-04-16T17:17:15.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 1 -- 172.21.15.18:0/1575994645 wait complete. 2024-04-16T17:17:15.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.565+0000 7fa47422b740 20 client.18687 trim_cache size 0 max 16384 2024-04-16T17:17:15.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.588+0000 7fa47422b740 1 Processor -- start 2024-04-16T17:17:15.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.588+0000 7fa47422b740 1 -- start start 2024-04-16T17:17:15.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.588+0000 7fa47422b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9180080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.588+0000 7fa47422b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf9194b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.588+0000 7fa47422b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 0x560cf917d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.588+0000 7fa47422b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1afb50 con 0x560cf9177390 2024-04-16T17:17:15.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.588+0000 7fa47422b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1ad420 con 0x560cf9177cb0 2024-04-16T17:17:15.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.588+0000 7fa47422b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1ab290 con 0x560cf9180890 2024-04-16T17:17:15.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa46a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9180080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa469e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf9194b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa46a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9180080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36934/0 (socket says 172.21.15.18:36934) 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa46a800640 1 -- 172.21.15.18:0/4005808584 learned_addr learned my addr 172.21.15.18:0/4005808584 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa46b200640 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 0x560cf917d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa469e00640 1 -- 172.21.15.18:0/4005808584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 msgr2=0x560cf917d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa469e00640 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 0x560cf917d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa469e00640 1 -- 172.21.15.18:0/4005808584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 msgr2=0x560cf9180080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa469e00640 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9180080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa469e00640 1 -- 172.21.15.18:0/4005808584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cf9182fb0 con 0x560cf9177cb0 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.589+0000 7fa46a800640 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9180080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.590+0000 7fa469e00640 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf9194b40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa4580ce2b0 tx=0x7fa458003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6d85aa59113d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:15.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.590+0000 7fa456a00640 1 -- 172.21.15.18:0/4005808584 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4580b5070 con 0x560cf9177cb0 2024-04-16T17:17:15.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.590+0000 7fa456a00640 1 -- 172.21.15.18:0/4005808584 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4580a0eb0 con 0x560cf9177cb0 2024-04-16T17:17:15.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.590+0000 7fa456a00640 1 -- 172.21.15.18:0/4005808584 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4580ce530 con 0x560cf9177cb0 2024-04-16T17:17:15.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 -- 172.21.15.18:0/4005808584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 msgr2=0x560cf9194b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf9194b40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa4580ce2b0 tx=0x7fa458003f30 comp rx=0 tx=0).stop 2024-04-16T17:17:15.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 -- 172.21.15.18:0/4005808584 shutdown_connections 2024-04-16T17:17:15.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9180890 0x560cf917d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177cb0 0x560cf9194b40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 --2- 172.21.15.18:0/4005808584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9177390 0x560cf9180080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 -- 172.21.15.18:0/4005808584 >> 172.21.15.18:0/4005808584 conn(0x560cf9171ac0 msgr2=0x560cf917d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:15.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 -- 172.21.15.18:0/4005808584 shutdown_connections 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 -- 172.21.15.18:0/4005808584 wait complete. 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 -- start start 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.591+0000 7fa47422b740 1 Processor -- start 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa47422b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177390 0x560cf919ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa47422b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf919be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa47422b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9180890 0x560cf919c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa47422b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1ab8e0 con 0x560cf9180890 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa47422b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1acd50 con 0x560cf9177390 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa47422b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1abfb0 con 0x560cf9177cb0 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa47422b740 10 client.0 _ll_register_callbacks cb 0x7fa4640099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa469e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf919be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa469e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf919be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52512/0 (socket says 172.21.15.18:52512) 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa469e00640 1 -- 172.21.15.18:0/1450119847 learned_addr learned my addr 172.21.15.18:0/1450119847 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa46a800640 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177390 0x560cf919ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.592+0000 7fa46b200640 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9180890 0x560cf919c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa469e00640 1 -- 172.21.15.18:0/1450119847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177390 msgr2=0x560cf919ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa469e00640 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177390 0x560cf919ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa469e00640 1 -- 172.21.15.18:0/1450119847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9180890 msgr2=0x560cf919c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa469e00640 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9180890 0x560cf919c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa469e00640 1 -- 172.21.15.18:0/1450119847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cf919cbd0 con 0x560cf9177cb0 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa46b200640 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9180890 0x560cf919c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa469e00640 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf919be60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa458003ee0 tx=0x7fa458005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa0208994f00db74 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa44be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa44be00640 1 -- 172.21.15.18:0/1450119847 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4580b5070 con 0x560cf9177cb0 2024-04-16T17:17:15.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa47422b740 1 -- 172.21.15.18:0/1450119847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4580a0eb0 con 0x560cf9177cb0 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.593+0000 7fa44be00640 1 -- 172.21.15.18:0/1450119847 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fa45809ae00 con 0x560cf9177cb0 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa44be00640 1 -- 172.21.15.18:0/1450119847 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa458003b30 con 0x560cf9177cb0 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa47422b740 10 client.28846 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa47422b740 1 -- 172.21.15.18:0/1450119847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560cf91ff4f0 con 0x560cf9177cb0 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa44be00640 1 -- 172.21.15.18:0/1450119847 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4580a0eb0 con 0x560cf9177cb0 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa44be00640 1 -- 172.21.15.18:0/1450119847 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa45809bf50 con 0x560cf9177cb0 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa47422b740 10 client.28846 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa47422b740 10 client.28846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa47422b740 20 client.28846 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa47422b740 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cf91ffba0 0x560cf9201fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa47422b740 4 client.28846 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:17:15.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.594+0000 7fa47422b740 1 -- 172.21.15.18:0/1450119847 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560cf90e49b0 con 0x560cf91ffba0 2024-04-16T17:17:15.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.595+0000 7fa46a800640 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cf91ffba0 0x560cf9201fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.595+0000 7fa46a800640 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cf91ffba0 0x560cf9201fc0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d263d737e801467c server_cookie=a9c743ae8b3e9748 in_seq=0 out_seq=0 2024-04-16T17:17:15.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.595+0000 7fa44be00640 10 client.28846 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:15.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.595+0000 7fa44be00640 1 -- 172.21.15.18:0/1450119847 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560cf90e49b0 con 0x560cf91ffba0 2024-04-16T17:17:15.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.595+0000 7fa44be00640 10 client.28846 handle_command_reply: tid=0 2024-04-16T17:17:15.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.595+0000 7fa47422b740 1 client.28846 shutdown 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 -- 172.21.15.18:0/1450119847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 msgr2=0x560cf919be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf919be60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa458003ee0 tx=0x7fa458005b20 comp rx=0 tx=0).stop 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 -- 172.21.15.18:0/1450119847 shutdown_connections 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cf91ffba0 0x560cf9201fc0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cf9180890 0x560cf919c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cf9177cb0 0x560cf919be60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 --2- 172.21.15.18:0/1450119847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cf9177390 0x560cf919ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 -- 172.21.15.18:0/1450119847 >> 172.21.15.18:0/1450119847 conn(0x560cf9171ac0 msgr2=0x560cf917e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa44be00640 0 client.28846 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 -- 172.21.15.18:0/1450119847 shutdown_connections 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 1 -- 172.21.15.18:0/1450119847 wait complete. 2024-04-16T17:17:15.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.596+0000 7fa47422b740 20 client.28846 trim_cache size 0 max 16384 2024-04-16T17:17:15.602 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:17:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:17:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:17:15.602 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:17:15.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 -- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa4500a3da0 msgr2=0x7fa4500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa4500a3da0 0x7fa4500a6260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa464005fd0 tx=0x7fa464009500 comp rx=0 tx=0).stop 2024-04-16T17:17:15.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 -- 172.21.15.18:0/2128818787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa46c074cd0 msgr2=0x7fa46c1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa46c074cd0 0x7fa46c1a0b90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa45800ea60 tx=0x7fa45800ef20 comp rx=0 tx=0).stop 2024-04-16T17:17:15.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 -- 172.21.15.18:0/2128818787 shutdown_connections 2024-04-16T17:17:15.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa4500a3da0 0x7fa4500a6260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa46c10e6c0 0x7fa46c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa46c074cd0 0x7fa46c1a0b90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 --2- 172.21.15.18:0/2128818787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa46c0743b0 0x7fa46c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 -- 172.21.15.18:0/2128818787 >> 172.21.15.18:0/2128818787 conn(0x7fa46c0fd950 msgr2=0x7fa46c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:15.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 -- 172.21.15.18:0/2128818787 shutdown_connections 2024-04-16T17:17:15.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.597+0000 7fa470a00640 1 -- 172.21.15.18:0/2128818787 wait complete. 2024-04-16T17:17:15.619 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:17:15.619 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:17:15.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 466871 3.58 MB/sec execute 568 sec latency 28.129 ms 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.833+0000 7f085e000640 1 -- 172.21.15.18:0/3915558393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0858074cd0 msgr2=0x7f085810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.833+0000 7f085e000640 1 --2- 172.21.15.18:0/3915558393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0858074cd0 0x7f085810e340 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0844009900 tx=0x7f084402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.834+0000 7f085e000640 1 -- 172.21.15.18:0/3915558393 shutdown_connections 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.834+0000 7f085e000640 1 --2- 172.21.15.18:0/3915558393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f085810e880 0x7f0858110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.834+0000 7f085e000640 1 --2- 172.21.15.18:0/3915558393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0858074cd0 0x7f085810e340 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.834+0000 7f085e000640 1 --2- 172.21.15.18:0/3915558393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08580743b0 0x7f0858074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.834+0000 7f085e000640 1 -- 172.21.15.18:0/3915558393 >> 172.21.15.18:0/3915558393 conn(0x7f08580fdad0 msgr2=0x7f08580ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.834+0000 7f085e000640 1 -- 172.21.15.18:0/3915558393 shutdown_connections 2024-04-16T17:17:15.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.834+0000 7f085e000640 1 -- 172.21.15.18:0/3915558393 wait complete. 2024-04-16T17:17:15.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.835+0000 7f085e000640 1 Processor -- start 2024-04-16T17:17:15.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.835+0000 7f085e000640 1 -- start start 2024-04-16T17:17:15.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f085e000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08580743b0 0x7f0858101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f085e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858074cd0 0x7f08581024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f085e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f085810e880 0x7f0858102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f085e000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f08581145c0 con 0x7f08580743b0 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f085e000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0858114440 con 0x7f085810e880 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f085e000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0858114740 con 0x7f0858074cd0 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f0857400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08580743b0 0x7f0858101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f0857e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f085810e880 0x7f0858102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f0857400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08580743b0 0x7f0858101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36962/0 (socket says 172.21.15.18:36962) 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f0857400640 1 -- 172.21.15.18:0/2252640918 learned_addr learned my addr 172.21.15.18:0/2252640918 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.836+0000 7f0856a00640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858074cd0 0x7f08581024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.837+0000 7f0857e00640 1 -- 172.21.15.18:0/2252640918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858074cd0 msgr2=0x7f08581024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.837+0000 7f0857e00640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858074cd0 0x7f08581024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.837+0000 7f0857e00640 1 -- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08580743b0 msgr2=0x7f0858101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.837+0000 7f0857e00640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08580743b0 0x7f0858101fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.837+0000 7f0857e00640 1 -- 172.21.15.18:0/2252640918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08581ab1a0 con 0x7f085810e880 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.837+0000 7f0857e00640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f085810e880 0x7f0858102a20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f085000ea80 tx=0x7f085000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.837+0000 7f084fe00640 1 -- 172.21.15.18:0/2252640918 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085000cea0 con 0x7f085810e880 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.837+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08581ab490 con 0x7f085810e880 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.838+0000 7f084fe00640 1 -- 172.21.15.18:0/2252640918 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0850005020 con 0x7f085810e880 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.838+0000 7f084fe00640 1 -- 172.21.15.18:0/2252640918 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500186d0 con 0x7f085810e880 2024-04-16T17:17:15.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.838+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08581aba00 con 0x7f085810e880 2024-04-16T17:17:15.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.839+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0858075a60 con 0x7f085810e880 2024-04-16T17:17:15.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.841+0000 7f084fe00640 1 -- 172.21.15.18:0/2252640918 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f08500054f0 con 0x7f085810e880 2024-04-16T17:17:15.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.842+0000 7f084fe00640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f083c0a3da0 0x7f083c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:15.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.843+0000 7f084fe00640 1 -- 172.21.15.18:0/2252640918 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f08500d1270 con 0x7f085810e880 2024-04-16T17:17:15.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.843+0000 7f0857400640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f083c0a3da0 0x7f083c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:15.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.850+0000 7f084fe00640 1 -- 172.21.15.18:0/2252640918 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f085009d4a0 con 0x7f085810e880 2024-04-16T17:17:15.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:15.850+0000 7f0857400640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f083c0a3da0 0x7f083c0a6260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0848009c90 tx=0x7f0848005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:15 smithi022 ceph-mon[36023]: pgmap v397: 129 pgs: 129 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-16T17:17:16.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:15 smithi043 ceph-mon[36901]: pgmap v397: 129 pgs: 129 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-16T17:17:16.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.285+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0858109a00 con 0x7f085810e880 2024-04-16T17:17:16.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.286+0000 7f084fe00640 1 -- 172.21.15.18:0/2252640918 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11854 (secure 0 0 0) 0x7f085009cda0 con 0x7f085810e880 2024-04-16T17:17:16.287 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:17:16.287 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:16:00.629944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:17:16.287 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-16T17:17:16.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f083c0a3da0 msgr2=0x7f083c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f083c0a3da0 0x7f083c0a6260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0848009c90 tx=0x7f0848005f90 comp rx=0 tx=0).stop 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f085810e880 msgr2=0x7f0858102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f085810e880 0x7f0858102a20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f085000ea80 tx=0x7f085000ef40 comp rx=0 tx=0).stop 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 shutdown_connections 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f083c0a3da0 0x7f083c0a6260 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f085810e880 0x7f0858102a20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858074cd0 0x7f08581024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 --2- 172.21.15.18:0/2252640918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08580743b0 0x7f0858101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 >> 172.21.15.18:0/2252640918 conn(0x7f08580fdad0 msgr2=0x7f08580ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 shutdown_connections 2024-04-16T17:17:16.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.289+0000 7f085e000640 1 -- 172.21.15.18:0/2252640918 wait complete. 2024-04-16T17:17:16.314 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:17:16.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 -- 172.21.15.18:0/3372315698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8112e30 msgr2=0x7f7aa8115220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 --2- 172.21.15.18:0/3372315698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8112e30 0x7f7aa8115220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009ca0 tx=0x7f7aa002c180 comp rx=0 tx=0).stop 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 -- 172.21.15.18:0/3372315698 shutdown_connections 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 --2- 172.21.15.18:0/3372315698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8112e30 0x7f7aa8115220 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 --2- 172.21.15.18:0/3372315698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7aa8074cc0 0x7f7aa81128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 --2- 172.21.15.18:0/3372315698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa80743a0 0x7f7aa8074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 -- 172.21.15.18:0/3372315698 >> 172.21.15.18:0/3372315698 conn(0x7f7aa80fd910 msgr2=0x7f7aa80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 -- 172.21.15.18:0/3372315698 shutdown_connections 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.532+0000 7f7aade00640 1 -- 172.21.15.18:0/3372315698 wait complete. 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.533+0000 7f7aade00640 1 Processor -- start 2024-04-16T17:17:16.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.533+0000 7f7aade00640 1 -- start start 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7aa80743a0 0x7f7aa81a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aade00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8074cc0 0x7f7aa81a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8112e30 0x7f7aa819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aade00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81187f0 con 0x7f7aa8074cc0 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aade00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8118670 con 0x7f7aa8112e30 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aade00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8118970 con 0x7f7aa80743a0 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aa7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8074cc0 0x7f7aa81a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aaca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7aa80743a0 0x7f7aa81a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aa7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8074cc0 0x7f7aa81a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36980/0 (socket says 172.21.15.18:36980) 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aa7e00640 1 -- 172.21.15.18:0/3389413035 learned_addr learned my addr 172.21.15.18:0/3389413035 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.534+0000 7f7aad400640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8112e30 0x7f7aa819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aaca00640 1 -- 172.21.15.18:0/3389413035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8112e30 msgr2=0x7f7aa819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8112e30 0x7f7aa819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aaca00640 1 -- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8074cc0 msgr2=0x7f7aa81a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8074cc0 0x7f7aa81a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aaca00640 1 -- 172.21.15.18:0/3389413035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa819f3f0 con 0x7f7aa80743a0 2024-04-16T17:17:16.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7aa80743a0 0x7f7aa81a4a70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e9f0 tx=0x7f7a9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aa5600640 1 -- 172.21.15.18:0/3389413035 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00cd80 con 0x7f7aa80743a0 2024-04-16T17:17:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aade00640 1 -- 172.21.15.18:0/3389413035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa819f6e0 con 0x7f7aa80743a0 2024-04-16T17:17:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aa5600640 1 -- 172.21.15.18:0/3389413035 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0035f0 con 0x7f7aa80743a0 2024-04-16T17:17:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aa5600640 1 -- 172.21.15.18:0/3389413035 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c010670 con 0x7f7aa80743a0 2024-04-16T17:17:16.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.535+0000 7f7aade00640 1 -- 172.21.15.18:0/3389413035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa81ab9f0 con 0x7f7aa80743a0 2024-04-16T17:17:16.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.538+0000 7f7ab21ad740 1 Processor -- start 2024-04-16T17:17:16.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.538+0000 7f7aa5600640 1 -- 172.21.15.18:0/3389413035 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a9c0037a0 con 0x7f7aa80743a0 2024-04-16T17:17:16.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7ab21ad740 1 -- start start 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611230f10 0x5566112312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611239ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x55661123c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556611196020 con 0x556611230f10 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55661113c460 con 0x556611231830 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556611191fe0 con 0x55661123a410 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7aad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611230f10 0x5566112312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7aad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611230f10 0x5566112312f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36986/0 (socket says 172.21.15.18:36986) 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7aad400640 1 -- 172.21.15.18:0/3684703314 learned_addr learned my addr 172.21.15.18:0/3684703314 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611239ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.539+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x55661123c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.540+0000 7f7aad400640 1 -- 172.21.15.18:0/3684703314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 msgr2=0x55661123c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.540+0000 7f7aad400640 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x55661123c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.540+0000 7f7aad400640 1 -- 172.21.15.18:0/3684703314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 msgr2=0x556611239ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.540+0000 7f7aad400640 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611239ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.540+0000 7f7aad400640 1 -- 172.21.15.18:0/3684703314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556611144040 con 0x556611230f10 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.540+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611239ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.541+0000 7f7aad400640 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611230f10 0x5566112312f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7aa806b480 tx=0x7f7aa0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f67dd055a206804 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.541+0000 7f7a99c00640 1 -- 172.21.15.18:0/3684703314 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0043070 con 0x556611230f10 2024-04-16T17:17:16.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.541+0000 7f7a99c00640 1 -- 172.21.15.18:0/3684703314 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa0003e50 con 0x556611230f10 2024-04-16T17:17:16.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.541+0000 7f7a99c00640 1 -- 172.21.15.18:0/3684703314 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0039040 con 0x556611230f10 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3684703314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611230f10 msgr2=0x5566112312f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611230f10 0x5566112312f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7aa806b480 tx=0x7f7aa0009900 comp rx=0 tx=0).stop 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3684703314 shutdown_connections 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x55661123c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611239ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3684703314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611230f10 0x5566112312f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3684703314 >> 172.21.15.18:0/3684703314 conn(0x55661122b640 msgr2=0x55661122ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3684703314 shutdown_connections 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7aa5600640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a900a3d50 0x7f7a900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3684703314 wait complete. 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7aa5600640 1 -- 172.21.15.18:0/3389413035 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f7a9c0d0e40 con 0x7f7aa80743a0 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.542+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a900a3d50 0x7f7a900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7ab21ad740 1 -- start start 2024-04-16T17:17:16.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7ab21ad740 1 Processor -- start 2024-04-16T17:17:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a900a3d50 0x7f7a900a6210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55661122c7d0 tx=0x7f7a94005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x5566112374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611231830 0x556611237a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611237f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55661123d1a0 con 0x55661123a410 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55661114f830 con 0x556611230f10 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.543+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55661123d320 con 0x556611231830 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7ab21ad740 10 client.0 _ll_register_callbacks cb 0x556611157150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x5566112374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611237f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x5566112374d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46244/0 (socket says 172.21.15.18:46244) 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 -- 172.21.15.18:0/2650949935 learned_addr learned my addr 172.21.15.18:0/2650949935 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611231830 0x556611237a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 -- 172.21.15.18:0/2650949935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611231830 msgr2=0x556611237a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611231830 0x556611237a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 -- 172.21.15.18:0/2650949935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 msgr2=0x556611237f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611237f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 -- 172.21.15.18:0/2650949935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556611153920 con 0x556611230f10 2024-04-16T17:17:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7aaca00640 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x5566112374d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e9c0 tx=0x7f7a9c005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5046717c4070dc6e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.544+0000 7f7a8e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7a8e000640 1 -- 172.21.15.18:0/2650949935 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c09c510 con 0x556611230f10 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7a8e000640 1 -- 172.21.15.18:0/2650949935 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f7a9c0ce400 con 0x556611230f10 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2650949935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa0003e50 con 0x556611230f10 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7a8e000640 1 -- 172.21.15.18:0/2650949935 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c09c330 con 0x556611230f10 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7ab21ad740 10 client.29018 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2650949935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5566111217f0 con 0x556611230f10 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7a8e000640 1 -- 172.21.15.18:0/2650949935 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa0003e50 con 0x556611230f10 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7a8e000640 1 -- 172.21.15.18:0/2650949935 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7a9c0d2070 con 0x556611230f10 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7ab21ad740 10 client.29018 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.545+0000 7f7ab21ad740 10 client.29018 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.546+0000 7f7ab21ad740 20 client.29018 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.546+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55661124a7c0 0x55661124cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.546+0000 7f7ab21ad740 4 client.29018 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.546+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2650949935 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566111a3a40 con 0x55661124a7c0 2024-04-16T17:17:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.546+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55661124a7c0 0x55661124cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.546+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55661124a7c0 0x55661124cc60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84d46a7e17e79637 server_cookie=4b4786dadcb0601f in_seq=0 out_seq=0 2024-04-16T17:17:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.546+0000 7f7a8e000640 10 client.29018 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.547+0000 7f7a8e000640 1 -- 172.21.15.18:0/2650949935 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566111a3a40 con 0x55661124a7c0 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.547+0000 7f7a8e000640 10 client.29018 handle_command_reply: tid=0 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.547+0000 7f7ab21ad740 1 client.29018 shutdown 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.547+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2650949935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 msgr2=0x5566112374d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.547+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x5566112374d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e9c0 tx=0x7f7a9c005bb0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2650949935 shutdown_connections 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55661124a7c0 0x55661124cc60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611237f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611231830 0x556611237a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2650949935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x5566112374d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7a8e000640 0 client.29018 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2650949935 >> 172.21.15.18:0/2650949935 conn(0x55661122b640 msgr2=0x55661122d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2650949935 shutdown_connections 2024-04-16T17:17:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2650949935 wait complete. 2024-04-16T17:17:16.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.548+0000 7f7ab21ad740 20 client.29018 trim_cache size 0 max 16384 2024-04-16T17:17:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.576+0000 7f7ab21ad740 1 Processor -- start 2024-04-16T17:17:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.576+0000 7f7ab21ad740 1 -- start start 2024-04-16T17:17:16.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x556611239c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611231830 0x556611246060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x556611252530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b8360 con 0x556611231830 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b5c30 con 0x556611230f10 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b3aa0 con 0x55661123a410 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x556611252530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aaca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x556611239c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x556611252530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52574/0 (socket says 172.21.15.18:52574) 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aad400640 1 -- 172.21.15.18:0/2534251639 learned_addr learned my addr 172.21.15.18:0/2534251639 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611231830 0x556611246060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aad400640 1 -- 172.21.15.18:0/2534251639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 msgr2=0x556611239c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aad400640 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x556611239c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aad400640 1 -- 172.21.15.18:0/2534251639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611231830 msgr2=0x556611246060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aad400640 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611231830 0x556611246060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aad400640 1 -- 172.21.15.18:0/2534251639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556610ea83f0 con 0x55661123a410 2024-04-16T17:17:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.577+0000 7f7aaca00640 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x556611239c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.578+0000 7f7aad400640 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x556611252530 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0005cc0 tx=0x7f7aa002c180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e02438cf538e10e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.578+0000 7f7a99200640 1 -- 172.21.15.18:0/2534251639 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0043040 con 0x55661123a410 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.578+0000 7f7a99200640 1 -- 172.21.15.18:0/2534251639 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa004c070 con 0x55661123a410 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.578+0000 7f7a99200640 1 -- 172.21.15.18:0/2534251639 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0039040 con 0x55661123a410 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2534251639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 msgr2=0x556611252530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x556611252530 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0005cc0 tx=0x7f7aa002c180 comp rx=0 tx=0).stop 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2534251639 shutdown_connections 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55661123a410 0x556611252530 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556611231830 0x556611246060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/2534251639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611230f10 0x556611239c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2534251639 >> 172.21.15.18:0/2534251639 conn(0x55661122b640 msgr2=0x55661122a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2534251639 shutdown_connections 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 -- 172.21.15.18:0/2534251639 wait complete. 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 -- start start 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.579+0000 7f7ab21ad740 1 Processor -- start 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611230f10 0x556611234500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611234a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7ab21ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611234fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b40f0 con 0x55661123a410 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b5560 con 0x556611231830 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7ab21ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b47c0 con 0x556611230f10 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7ab21ad740 10 client.0 _ll_register_callbacks cb 0x7f7a9c003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7aad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611234fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7aad400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611234fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37006/0 (socket says 172.21.15.18:37006) 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7aad400640 1 -- 172.21.15.18:0/3647877632 learned_addr learned my addr 172.21.15.18:0/3647877632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611230f10 0x556611234500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.580+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611234a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7aaca00640 1 -- 172.21.15.18:0/3647877632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 msgr2=0x556611234a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611234a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7aaca00640 1 -- 172.21.15.18:0/3647877632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 msgr2=0x556611234fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611234fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7aaca00640 1 -- 172.21.15.18:0/3647877632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556610f727d0 con 0x556611230f10 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7aad400640 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611234fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7aaca00640 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611230f10 0x556611234500 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c09c960 tx=0x7f7a9c005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1e14537118bda23 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7a8e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7a8e000640 1 -- 172.21.15.18:0/3647877632 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c09c510 con 0x556611230f10 2024-04-16T17:17:16.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3647877632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa004c070 con 0x556611230f10 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7a8e000640 1 -- 172.21.15.18:0/3647877632 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f7a9c0872b0 con 0x556611230f10 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7a8e000640 1 -- 172.21.15.18:0/3647877632 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c098c70 con 0x556611230f10 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.581+0000 7f7ab21ad740 10 client.28870 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3647877632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5566112b11a0 con 0x556611230f10 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7a8e000640 1 -- 172.21.15.18:0/3647877632 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa004c070 con 0x556611230f10 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7a8e000640 1 -- 172.21.15.18:0/3647877632 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7a9c09be20 con 0x556611230f10 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7ab21ad740 10 client.28870 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7ab21ad740 10 client.28870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7ab21ad740 20 client.28870 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5566112b18c0 0x5566112b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7ab21ad740 4 client.28870 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3647877632 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55661119d8b0 con 0x5566112b18c0 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.582+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5566112b18c0 0x5566112b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7aa7e00640 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5566112b18c0 0x5566112b3cb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aec41071591995c4 server_cookie=df1a785e8942ddcd in_seq=0 out_seq=0 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7a8e000640 10 client.28870 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7a8e000640 1 -- 172.21.15.18:0/3647877632 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55661119d8b0 con 0x5566112b18c0 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7a8e000640 10 client.28870 handle_command_reply: tid=0 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 client.28870 shutdown 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611230f10 msgr2=0x556611234500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611230f10 0x556611234500 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c09c960 tx=0x7f7a9c005b20 comp rx=0 tx=0).stop 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3647877632 shutdown_connections 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5566112b18c0 0x5566112b3cb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55661123a410 0x556611234fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7a8e000640 0 client.28870 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556611231830 0x556611234a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 --2- 172.21.15.18:0/3647877632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556611230f10 0x556611234500 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3647877632 >> 172.21.15.18:0/3647877632 conn(0x55661122b640 msgr2=0x55661122ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3647877632 shutdown_connections 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.583+0000 7f7ab21ad740 1 -- 172.21.15.18:0/3647877632 wait complete. 2024-04-16T17:17:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7ab21ad740 20 client.28870 trim_cache size 0 max 16384 2024-04-16T17:17:16.588 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:17:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 -- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a900a3d50 msgr2=0x7f7a900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a900a3d50 0x7f7a900a6210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55661122c7d0 tx=0x7f7a94005f90 comp rx=0 tx=0).stop 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 -- 172.21.15.18:0/3389413035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7aa80743a0 msgr2=0x7f7aa81a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7aa80743a0 0x7f7aa81a4a70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e9f0 tx=0x7f7a9c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 -- 172.21.15.18:0/3389413035 shutdown_connections 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7a900a3d50 0x7f7a900a6210 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8112e30 0x7f7aa819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7aa8074cc0 0x7f7aa81a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 --2- 172.21.15.18:0/3389413035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7aa80743a0 0x7f7aa81a4a70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 -- 172.21.15.18:0/3389413035 >> 172.21.15.18:0/3389413035 conn(0x7f7aa80fd910 msgr2=0x7f7aa80ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 -- 172.21.15.18:0/3389413035 shutdown_connections 2024-04-16T17:17:16.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.584+0000 7f7aade00640 1 -- 172.21.15.18:0/3389413035 wait complete. 2024-04-16T17:17:16.604 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:17:16.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 467321 3.57 MB/sec execute 569 sec latency 82.594 ms 2024-04-16T17:17:16.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 -- 172.21.15.18:0/3603020949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074d50 msgr2=0x7fd0cc10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 --2- 172.21.15.18:0/3603020949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074d50 0x7fd0cc10e2c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009860 tx=0x7fd0b402bc00 comp rx=0 tx=0).stop 2024-04-16T17:17:16.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 -- 172.21.15.18:0/3603020949 shutdown_connections 2024-04-16T17:17:16.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 --2- 172.21.15.18:0/3603020949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0cc10e800 0x7fd0cc110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 --2- 172.21.15.18:0/3603020949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074d50 0x7fd0cc10e2c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 --2- 172.21.15.18:0/3603020949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0cc074430 0x7fd0cc074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 -- 172.21.15.18:0/3603020949 >> 172.21.15.18:0/3603020949 conn(0x7fd0cc0fdab0 msgr2=0x7fd0cc0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 -- 172.21.15.18:0/3603020949 shutdown_connections 2024-04-16T17:17:16.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.790+0000 7fd0d2800640 1 -- 172.21.15.18:0/3603020949 wait complete. 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.791+0000 7fd0d2800640 1 Processor -- start 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.791+0000 7fd0d2800640 1 -- start start 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0d2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074430 0x7fd0cc1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0d2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0cc074d50 0x7fd0cc1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0d2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0cc10e800 0x7fd0cc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0d2800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc114590 con 0x7fd0cc074d50 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0d2800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc114410 con 0x7fd0cc10e800 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0d2800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc114710 con 0x7fd0cc074430 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074430 0x7fd0cc1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074430 0x7fd0cc1a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52594/0 (socket says 172.21.15.18:52594) 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cb400640 1 -- 172.21.15.18:0/4294445042 learned_addr learned my addr 172.21.15.18:0/4294445042 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0cc10e800 0x7fd0cc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0cc074d50 0x7fd0cc1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cb400640 1 -- 172.21.15.18:0/4294445042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0cc10e800 msgr2=0x7fd0cc19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cb400640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0cc10e800 0x7fd0cc19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cb400640 1 -- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0cc074d50 msgr2=0x7fd0cc1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cb400640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0cc074d50 0x7fd0cc1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.792+0000 7fd0cb400640 1 -- 172.21.15.18:0/4294445042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc19b090 con 0x7fd0cc074430 2024-04-16T17:17:16.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.793+0000 7fd0cb400640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074430 0x7fd0cc1a0760 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00e9f0 tx=0x7fd0bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.793+0000 7fd0c8c00640 1 -- 172.21.15.18:0/4294445042 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00cd80 con 0x7fd0cc074430 2024-04-16T17:17:16.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.793+0000 7fd0c8c00640 1 -- 172.21.15.18:0/4294445042 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0bc010910 con 0x7fd0cc074430 2024-04-16T17:17:16.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.793+0000 7fd0d2800640 1 -- 172.21.15.18:0/4294445042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc19b380 con 0x7fd0cc074430 2024-04-16T17:17:16.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.793+0000 7fd0c8c00640 1 -- 172.21.15.18:0/4294445042 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc018990 con 0x7fd0cc074430 2024-04-16T17:17:16.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.793+0000 7fd0d2800640 1 -- 172.21.15.18:0/4294445042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc1a7620 con 0x7fd0cc074430 2024-04-16T17:17:16.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.797+0000 7fd0d544f740 1 Processor -- start 2024-04-16T17:17:16.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.797+0000 7fd0d544f740 1 -- start start 2024-04-16T17:17:16.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d71130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d79d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 0x55c5c3d7c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0d544f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c5c3cd6020 con 0x55c5c3d7a250 2024-04-16T17:17:16.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0d544f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c5c3c7c460 con 0x55c5c3d70d50 2024-04-16T17:17:16.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0d544f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c5c3cd1fe0 con 0x55c5c3d71670 2024-04-16T17:17:16.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0c8c00640 1 -- 172.21.15.18:0/4294445042 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd0bc010ac0 con 0x7fd0cc074430 2024-04-16T17:17:16.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d71130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d71130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46268/0 (socket says 172.21.15.18:46268) 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cbe00640 1 -- 172.21.15.18:0/960591074 learned_addr learned my addr 172.21.15.18:0/960591074 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cb400640 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d79d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cbe00640 1 -- 172.21.15.18:0/960591074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 msgr2=0x55c5c3d79d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d79d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cbe00640 1 -- 172.21.15.18:0/960591074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 msgr2=0x55c5c3d7c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 0x55c5c3d7c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cbe00640 1 -- 172.21.15.18:0/960591074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c3c84040 con 0x55c5c3d70d50 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.798+0000 7fd0cb400640 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d79d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.799+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d71130 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e480 tx=0x7fd0c40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6ddc5b43885c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.799+0000 7fd0c0c00640 1 -- 172.21.15.18:0/960591074 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4003970 con 0x55c5c3d70d50 2024-04-16T17:17:16.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.799+0000 7fd0c0c00640 1 -- 172.21.15.18:0/960591074 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c400d7d0 con 0x55c5c3d70d50 2024-04-16T17:17:16.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0c0c00640 1 -- 172.21.15.18:0/960591074 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4004470 con 0x55c5c3d70d50 2024-04-16T17:17:16.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0c8c00640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0b00a3d20 0x7fd0b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0c8c00640 1 -- 172.21.15.18:0/4294445042 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fd0bc0d0fa0 con 0x7fd0cc074430 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0d544f740 1 -- 172.21.15.18:0/960591074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 msgr2=0x55c5c3d71130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0d544f740 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d71130 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e480 tx=0x7fd0c40095c0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0d544f740 1 -- 172.21.15.18:0/960591074 shutdown_connections 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0d544f740 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 0x55c5c3d7c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0d544f740 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d79d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0d544f740 1 --2- 172.21.15.18:0/960591074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d71130 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0d544f740 1 -- 172.21.15.18:0/960591074 >> 172.21.15.18:0/960591074 conn(0x55c5c3d6b480 msgr2=0x55c5c3d6b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.800+0000 7fd0cb400640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0b00a3d20 0x7fd0b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.801+0000 7fd0d544f740 1 -- 172.21.15.18:0/960591074 shutdown_connections 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.801+0000 7fd0d544f740 1 -- 172.21.15.18:0/960591074 wait complete. 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.801+0000 7fd0d544f740 1 -- start start 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.801+0000 7fd0cb400640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0b00a3d20 0x7fd0b00a61e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55c5c3d70820 tx=0x7fd0bc000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.801+0000 7fd0d544f740 1 Processor -- start 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d77240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d77780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 0x55c5c3d77cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0d544f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c5c3d7d090 con 0x55c5c3d7a250 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0d544f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c5c3c8f830 con 0x55c5c3d70d50 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0d544f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c5c3d7d210 con 0x55c5c3d71670 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0d544f740 10 client.0 _ll_register_callbacks cb 0x55c5c3c97150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d77240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d77240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46270/0 (socket says 172.21.15.18:46270) 2024-04-16T17:17:16.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d77780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0cbe00640 1 -- 172.21.15.18:0/1408286417 learned_addr learned my addr 172.21.15.18:0/1408286417 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.802+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 0x55c5c3d77cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.803+0000 7fd0c3e00640 1 -- 172.21.15.18:0/1408286417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 msgr2=0x55c5c3d77240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.803+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d77240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.803+0000 7fd0c3e00640 1 -- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 msgr2=0x55c5c3d77cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.803+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 0x55c5c3d77cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.803+0000 7fd0c3e00640 1 -- 172.21.15.18:0/1408286417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c3c93920 con 0x55c5c3d71670 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.803+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 0x55c5c3d77cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.803+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d77780 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4037040 tx=0x7fd0b4005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd334c9e710c1094 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.803+0000 7fd0b9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:16.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.804+0000 7fd0b9600640 1 -- 172.21.15.18:0/1408286417 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4041020 con 0x55c5c3d71670 2024-04-16T17:17:16.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.804+0000 7fd0d544f740 1 -- 172.21.15.18:0/1408286417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c400d7d0 con 0x55c5c3d71670 2024-04-16T17:17:16.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.804+0000 7fd0b9600640 1 -- 172.21.15.18:0/1408286417 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fd0b4034440 con 0x55c5c3d71670 2024-04-16T17:17:16.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.804+0000 7fd0b9600640 1 -- 172.21.15.18:0/1408286417 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd0b404c050 con 0x55c5c3d71670 2024-04-16T17:17:16.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.804+0000 7fd0d544f740 10 client.28879 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:16.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.804+0000 7fd0d544f740 1 -- 172.21.15.18:0/1408286417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c5c3c617f0 con 0x55c5c3d71670 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.804+0000 7fd0b9600640 1 -- 172.21.15.18:0/1408286417 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c400d7d0 con 0x55c5c3d71670 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.804+0000 7fd0b9600640 1 -- 172.21.15.18:0/1408286417 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd0b40035f0 con 0x55c5c3d71670 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0d544f740 10 client.28879 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0d544f740 10 client.28879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0d544f740 20 client.28879 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c5c3d92d40 0x55c5c3d95130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0d544f740 4 client.28879 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0d544f740 1 -- 172.21.15.18:0/1408286417 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5c3ce3a40 con 0x55c5c3d92d40 2024-04-16T17:17:16.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c5c3d92d40 0x55c5c3d95130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c5c3d92d40 0x55c5c3d95130 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff44802a5766a3c3 server_cookie=47d52ca4d9786554 in_seq=0 out_seq=0 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.805+0000 7fd0b9600640 10 client.28879 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.806+0000 7fd0b9600640 1 -- 172.21.15.18:0/1408286417 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5c3ce3a40 con 0x55c5c3d92d40 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.806+0000 7fd0b9600640 10 client.28879 handle_command_reply: tid=0 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.806+0000 7fd0d544f740 1 client.28879 shutdown 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.806+0000 7fd0d544f740 1 -- 172.21.15.18:0/1408286417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 msgr2=0x55c5c3d77780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.806+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d77780 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4037040 tx=0x7fd0b4005b20 comp rx=0 tx=0).stop 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 1 -- 172.21.15.18:0/1408286417 shutdown_connections 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c5c3d92d40 0x55c5c3d95130 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d7a250 0x55c5c3d77cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d71670 0x55c5c3d77780 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1408286417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d77240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 1 -- 172.21.15.18:0/1408286417 >> 172.21.15.18:0/1408286417 conn(0x55c5c3d6b480 msgr2=0x55c5c3d6d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0b9600640 0 client.28879 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 1 -- 172.21.15.18:0/1408286417 shutdown_connections 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 1 -- 172.21.15.18:0/1408286417 wait complete. 2024-04-16T17:17:16.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.807+0000 7fd0d544f740 20 client.28879 trim_cache size 0 max 16384 2024-04-16T17:17:16.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0d544f740 1 Processor -- start 2024-04-16T17:17:16.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0d544f740 1 -- start start 2024-04-16T17:17:16.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d79970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 0x55c5c3d9a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0d544f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1afda0 con 0x55c5c3d71670 2024-04-16T17:17:16.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0d544f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1ad670 con 0x55c5c3d70d50 2024-04-16T17:17:16.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0d544f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1ab4e0 con 0x55c5c3d7a250 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d79970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d79970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46276/0 (socket says 172.21.15.18:46276) 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d8e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37070/0 (socket says 172.21.15.18:37070) 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0cbe00640 1 -- 172.21.15.18:0/3325411447 learned_addr learned my addr 172.21.15.18:0/3325411447 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.822+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/3325411447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 0x55c5c3d9a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0c3e00640 1 -- 172.21.15.18:0/3325411447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 msgr2=0x55c5c3d9a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/3325411447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 0x55c5c3d9a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0c3e00640 1 -- 172.21.15.18:0/3325411447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 msgr2=0x55c5c3d79970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/3325411447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d79970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0c3e00640 1 -- 172.21.15.18:0/3325411447 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c3d714a0 con 0x55c5c3d71670 2024-04-16T17:17:16.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/3325411447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d8e7e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4003a90 tx=0x7fd0b402bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6543bddcf39284 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0bbe00640 1 -- 172.21.15.18:0/3325411447 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4056070 con 0x55c5c3d71670 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0bbe00640 1 -- 172.21.15.18:0/3325411447 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b4009450 con 0x55c5c3d71670 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.823+0000 7fd0bbe00640 1 -- 172.21.15.18:0/3325411447 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4041030 con 0x55c5c3d71670 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.824+0000 7fd0d544f740 1 -- 172.21.15.18:0/3325411447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 msgr2=0x55c5c3d8e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.824+0000 7fd0d544f740 1 --2- 172.21.15.18:0/3325411447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d8e7e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4003a90 tx=0x7fd0b402bc00 comp rx=0 tx=0).stop 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.824+0000 7fd0d544f740 1 -- 172.21.15.18:0/3325411447 shutdown_connections 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.824+0000 7fd0d544f740 1 --2- 172.21.15.18:0/3325411447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 0x55c5c3d9a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.824+0000 7fd0d544f740 1 --2- 172.21.15.18:0/3325411447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d8e7e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.824+0000 7fd0d544f740 1 --2- 172.21.15.18:0/3325411447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d79970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.824+0000 7fd0d544f740 1 -- 172.21.15.18:0/3325411447 >> 172.21.15.18:0/3325411447 conn(0x55c5c3d6b480 msgr2=0x55c5c3d72610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.824+0000 7fd0d544f740 1 -- 172.21.15.18:0/3325411447 shutdown_connections 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.825+0000 7fd0d544f740 1 -- 172.21.15.18:0/3325411447 wait complete. 2024-04-16T17:17:16.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.825+0000 7fd0d544f740 1 -- start start 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.825+0000 7fd0d544f740 1 Processor -- start 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d8afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d73b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0d544f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 0x55c5c3d740d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0d544f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1abb30 con 0x55c5c3d71670 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0d544f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1acfa0 con 0x55c5c3d70d50 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0d544f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1ac200 con 0x55c5c3d7a250 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0d544f740 10 client.0 _ll_register_callbacks cb 0x7fd0bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d73b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0c3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d73b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37082/0 (socket says 172.21.15.18:37082) 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0c3e00640 1 -- 172.21.15.18:0/1234095923 learned_addr learned my addr 172.21.15.18:0/1234095923 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d8afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 0x55c5c3d740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0c3e00640 1 -- 172.21.15.18:0/1234095923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 msgr2=0x55c5c3d740d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 0x55c5c3d740d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0c3e00640 1 -- 172.21.15.18:0/1234095923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 msgr2=0x55c5c3d8afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:16.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d8afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.826+0000 7fd0c3e00640 1 -- 172.21.15.18:0/1234095923 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c39e83f0 con 0x55c5c3d71670 2024-04-16T17:17:16.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.827+0000 7fd0c3e00640 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d73b90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd0b402ce00 tx=0x7fd0b4004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef12da43c1374fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:16.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.827+0000 7fd0b9600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.827+0000 7fd0b9600640 1 -- 172.21.15.18:0/1234095923 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4041070 con 0x55c5c3d71670 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.827+0000 7fd0d544f740 1 -- 172.21.15.18:0/1234095923 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b4009450 con 0x55c5c3d71670 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.827+0000 7fd0b9600640 1 -- 172.21.15.18:0/1234095923 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fd0b4003660 con 0x55c5c3d71670 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.827+0000 7fd0b9600640 1 -- 172.21.15.18:0/1234095923 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd0b4056040 con 0x55c5c3d71670 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.827+0000 7fd0d544f740 10 client.18729 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.827+0000 7fd0d544f740 1 -- 172.21.15.18:0/1234095923 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c5c3ab27d0 con 0x55c5c3d71670 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0b9600640 1 -- 172.21.15.18:0/1234095923 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b4009450 con 0x55c5c3d71670 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0b9600640 1 -- 172.21.15.18:0/1234095923 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd0b402c870 con 0x55c5c3d71670 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0d544f740 10 client.18729 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:16.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0d544f740 10 client.18729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0d544f740 20 client.18729 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c5c3df9290 0x55c5c3dfb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0d544f740 4 client.18729 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0d544f740 1 -- 172.21.15.18:0/1234095923 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5c3cdd8b0 con 0x55c5c3df9290 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.828+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c5c3df9290 0x55c5c3dfb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0cbe00640 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c5c3df9290 0x55c5c3dfb680 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d0304cd4e55e17f server_cookie=eeff2c5ea97af61a in_seq=0 out_seq=0 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0b9600640 10 client.18729 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0b9600640 1 -- 172.21.15.18:0/1234095923 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5c3cdd8b0 con 0x55c5c3df9290 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0b9600640 10 client.18729 handle_command_reply: tid=0 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 client.18729 shutdown 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 -- 172.21.15.18:0/1234095923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 msgr2=0x55c5c3d73b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d73b90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd0b402ce00 tx=0x7fd0b4004540 comp rx=0 tx=0).stop 2024-04-16T17:17:16.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 -- 172.21.15.18:0/1234095923 shutdown_connections 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55c5c3df9290 0x55c5c3dfb680 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5c3d7a250 0x55c5c3d740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c5c3d71670 0x55c5c3d73b90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 --2- 172.21.15.18:0/1234095923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c5c3d70d50 0x55c5c3d8afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0b9600640 0 client.18729 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.829+0000 7fd0d544f740 1 -- 172.21.15.18:0/1234095923 >> 172.21.15.18:0/1234095923 conn(0x55c5c3d6b480 msgr2=0x55c5c3d771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.830+0000 7fd0d544f740 1 -- 172.21.15.18:0/1234095923 shutdown_connections 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.830+0000 7fd0d544f740 1 -- 172.21.15.18:0/1234095923 wait complete. 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.830+0000 7fd0d544f740 20 client.18729 trim_cache size 0 max 16384 2024-04-16T17:17:16.833 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:17:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.830+0000 7fd0d2800640 1 -- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0b00a3d20 msgr2=0x7fd0b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.830+0000 7fd0d2800640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0b00a3d20 0x7fd0b00a61e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55c5c3d70820 tx=0x7fd0bc000bc0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.830+0000 7fd0d2800640 1 -- 172.21.15.18:0/4294445042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074430 msgr2=0x7fd0cc1a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.830+0000 7fd0d2800640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074430 0x7fd0cc1a0760 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00e9f0 tx=0x7fd0bc00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.831+0000 7fd0d2800640 1 -- 172.21.15.18:0/4294445042 shutdown_connections 2024-04-16T17:17:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.831+0000 7fd0d2800640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0b00a3d20 0x7fd0b00a61e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.831+0000 7fd0d2800640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0cc10e800 0x7fd0cc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.831+0000 7fd0d2800640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd0cc074d50 0x7fd0cc1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.831+0000 7fd0d2800640 1 --2- 172.21.15.18:0/4294445042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc074430 0x7fd0cc1a0760 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:16.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.831+0000 7fd0d2800640 1 -- 172.21.15.18:0/4294445042 >> 172.21.15.18:0/4294445042 conn(0x7fd0cc0fdab0 msgr2=0x7fd0cc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:16.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.831+0000 7fd0d2800640 1 -- 172.21.15.18:0/4294445042 shutdown_connections 2024-04-16T17:17:16.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:16.831+0000 7fd0d2800640 1 -- 172.21.15.18:0/4294445042 wait complete. 2024-04-16T17:17:16.852 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:17:17.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 -- 172.21.15.18:0/3717427224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 msgr2=0x7f80341033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 --2- 172.21.15.18:0/3717427224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 0x7f80341033c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8028009900 tx=0x7f802802bce0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 -- 172.21.15.18:0/3717427224 shutdown_connections 2024-04-16T17:17:17.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 --2- 172.21.15.18:0/3717427224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803410c400 0x7f803410e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 --2- 172.21.15.18:0/3717427224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8034103900 0x7f803410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 --2- 172.21.15.18:0/3717427224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 0x7f80341033c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 -- 172.21.15.18:0/3717427224 >> 172.21.15.18:0/3717427224 conn(0x7f8034077ab0 msgr2=0x7f80340fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:17.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 -- 172.21.15.18:0/3717427224 shutdown_connections 2024-04-16T17:17:17.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.042+0000 7f803ae00640 1 -- 172.21.15.18:0/3717427224 wait complete. 2024-04-16T17:17:17.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.043+0000 7f803ae00640 1 Processor -- start 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.043+0000 7f803ae00640 1 -- start start 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f803ae00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 0x7f8034100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f803ae00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8034103900 0x7f8034101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f803ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803410c400 0x7f80341016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f803ae00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8034112250 con 0x7f8034102fe0 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f803ae00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80341120d0 con 0x7f8034103900 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f803ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80341123d0 con 0x7f803410c400 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 0x7f8034100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 0x7f8034100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36400/0 (socket says 172.21.15.18:36400) 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039a00640 1 -- 172.21.15.18:0/2276959712 learned_addr learned my addr 172.21.15.18:0/2276959712 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f803a400640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803410c400 0x7f80341016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039a00640 1 -- 172.21.15.18:0/2276959712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803410c400 msgr2=0x7f80341016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039000640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8034103900 0x7f8034101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039a00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803410c400 0x7f80341016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039a00640 1 -- 172.21.15.18:0/2276959712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8034103900 msgr2=0x7f8034101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039a00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8034103900 0x7f8034101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039a00640 1 -- 172.21.15.18:0/2276959712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80341a6c90 con 0x7f8034102fe0 2024-04-16T17:17:17.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.044+0000 7f8039000640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8034103900 0x7f8034101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:17.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.045+0000 7f8039a00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 0x7f8034100c30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8028006010 tx=0x7f802802cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:17.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.045+0000 7f8032000640 1 -- 172.21.15.18:0/2276959712 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028041070 con 0x7f8034102fe0 2024-04-16T17:17:17.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.045+0000 7f8032000640 1 -- 172.21.15.18:0/2276959712 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8028004450 con 0x7f8034102fe0 2024-04-16T17:17:17.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.045+0000 7f8032000640 1 -- 172.21.15.18:0/2276959712 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028003d80 con 0x7f8034102fe0 2024-04-16T17:17:17.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.045+0000 7f803ae00640 1 -- 172.21.15.18:0/2276959712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80341a6f20 con 0x7f8034102fe0 2024-04-16T17:17:17.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.045+0000 7f803ae00640 1 -- 172.21.15.18:0/2276959712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80341a7390 con 0x7f8034102fe0 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803f1e9740 1 Processor -- start 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803f1e9740 1 -- start start 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f8032000640 1 -- 172.21.15.18:0/2276959712 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8028044070 con 0x7f8034102fe0 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30f1410 0x55d3a30f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803f1e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d3a304d020 con 0x55d3a30e7f10 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803f1e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d3a2ff3460 con 0x55d3a30e8830 2024-04-16T17:17:17.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803f1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d3a3048fe0 con 0x55d3a30f1410 2024-04-16T17:17:17.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30e82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36410/0 (socket says 172.21.15.18:36410) 2024-04-16T17:17:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 -- 172.21.15.18:0/3632519299 learned_addr learned my addr 172.21.15.18:0/3632519299 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 -- 172.21.15.18:0/3632519299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30f1410 msgr2=0x55d3a30f3800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:17:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 --2- 172.21.15.18:0/3632519299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30f1410 0x55d3a30f3800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 -- 172.21.15.18:0/3632519299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 msgr2=0x55d3a30f0ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:17:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 --2- 172.21.15.18:0/3632519299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30f0ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 -- 172.21.15.18:0/3632519299 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a2ffb040 con 0x55d3a30e7f10 2024-04-16T17:17:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f803a400640 1 --2- 172.21.15.18:0/3632519299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30e82f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f802c009fd0 tx=0x7f802c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd2aa9bac36fb97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f8026a00640 1 -- 172.21.15.18:0/3632519299 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c004d20 con 0x55d3a30e7f10 2024-04-16T17:17:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f8026a00640 1 -- 172.21.15.18:0/3632519299 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802c004050 con 0x55d3a30e7f10 2024-04-16T17:17:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f8026a00640 1 -- 172.21.15.18:0/3632519299 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c005280 con 0x55d3a30e7f10 2024-04-16T17:17:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f8032000640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f801c0a3da0 0x7f801c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.046+0000 7f8032000640 1 -- 172.21.15.18:0/2276959712 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f80280f26c0 con 0x7f8034102fe0 2024-04-16T17:17:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f8039a00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f801c0a3da0 0x7f801c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- 172.21.15.18:0/3632519299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 msgr2=0x55d3a30e82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3632519299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30e82f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f802c009fd0 tx=0x7f802c0118d0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- 172.21.15.18:0/3632519299 shutdown_connections 2024-04-16T17:17:17.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3632519299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30f1410 0x55d3a30f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3632519299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3632519299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30e82f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- 172.21.15.18:0/3632519299 >> 172.21.15.18:0/3632519299 conn(0x55d3a30e2640 msgr2=0x55d3a30e2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:17.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- 172.21.15.18:0/3632519299 shutdown_connections 2024-04-16T17:17:17.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- 172.21.15.18:0/3632519299 wait complete. 2024-04-16T17:17:17.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f8039a00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f801c0a3da0 0x7f801c0a6260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55d3a30e79e0 tx=0x7f802802c500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:17.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- start start 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 Processor -- start 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30f1410 0x55d3a30eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d3a30f4200 con 0x55d3a30e7f10 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d3a3006830 con 0x55d3a30f1410 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d3a30f4380 con 0x55d3a30e8830 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803f1e9740 10 client.0 _ll_register_callbacks cb 0x55d3a300e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30ee3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36426/0 (socket says 172.21.15.18:36426) 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 -- 172.21.15.18:0/2648507432 learned_addr learned my addr 172.21.15.18:0/2648507432 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f8039000640 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 -- 172.21.15.18:0/2648507432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 msgr2=0x55d3a30ee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:17.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30ee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 -- 172.21.15.18:0/2648507432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30f1410 msgr2=0x55d3a30eee50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30f1410 0x55d3a30eee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 -- 172.21.15.18:0/2648507432 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a300a920 con 0x55d3a30e7f10 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f803a400640 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30ee3d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f802c004430 tx=0x7f802c004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533f82348ffddcaa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.051+0000 7f801b400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f801b400640 1 -- 172.21.15.18:0/2648507432 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c029070 con 0x55d3a30e7f10 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f801b400640 1 -- 172.21.15.18:0/2648507432 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f802c00f4e0 con 0x55d3a30e7f10 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 1 -- 172.21.15.18:0/2648507432 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f802c004050 con 0x55d3a30e7f10 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f801b400640 1 -- 172.21.15.18:0/2648507432 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f802c035050 con 0x55d3a30e7f10 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 10 client.18741 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 1 -- 172.21.15.18:0/2648507432 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d3a2fd87f0 con 0x55d3a30e7f10 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f801b400640 1 -- 172.21.15.18:0/2648507432 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802c004050 con 0x55d3a30e7f10 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f801b400640 1 -- 172.21.15.18:0/2648507432 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f802c0235f0 con 0x55d3a30e7f10 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 10 client.18741 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:17.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 10 client.18741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:17.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 20 client.18741 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:17.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3a3101650 0x55d3a3103a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 4 client.18741 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:17.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.054+0000 7f803f1e9740 1 -- 172.21.15.18:0/2648507432 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3a305aa40 con 0x55d3a3101650 2024-04-16T17:17:17.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.062+0000 7f8039000640 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3a3101650 0x55d3a3103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.062+0000 7f8039000640 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3a3101650 0x55d3a3103a40 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6312d245c50c0675 server_cookie=6046e3389ac542d5 in_seq=0 out_seq=0 2024-04-16T17:17:17.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.062+0000 7f801b400640 10 client.18741 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:17.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f801b400640 1 -- 172.21.15.18:0/2648507432 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3a305aa40 con 0x55d3a3101650 2024-04-16T17:17:17.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f801b400640 10 client.18741 handle_command_reply: tid=0 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 client.18741 shutdown 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 -- 172.21.15.18:0/2648507432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 msgr2=0x55d3a30ee3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30ee3d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f802c004430 tx=0x7f802c004460 comp rx=0 tx=0).stop 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 -- 172.21.15.18:0/2648507432 shutdown_connections 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3a3101650 0x55d3a3103a40 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30f1410 0x55d3a30eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 --2- 172.21.15.18:0/2648507432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30e7f10 0x55d3a30ee3d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 -- 172.21.15.18:0/2648507432 >> 172.21.15.18:0/2648507432 conn(0x55d3a30e2640 msgr2=0x55d3a30e4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f801b400640 0 client.18741 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 -- 172.21.15.18:0/2648507432 shutdown_connections 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 1 -- 172.21.15.18:0/2648507432 wait complete. 2024-04-16T17:17:17.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.063+0000 7f803f1e9740 20 client.18741 trim_cache size 0 max 16384 2024-04-16T17:17:17.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.077+0000 7f803f1e9740 1 Processor -- start 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803f1e9740 1 -- start start 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e7f10 0x55d3a30f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 0x55d3a3109270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803f1e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80341b3fc0 con 0x55d3a30f1410 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803f1e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80341b1890 con 0x55d3a30e8830 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803f1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80341af700 con 0x55d3a30e7f10 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e7f10 0x55d3a30f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f8039000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f8039000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30fd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56346/0 (socket says 172.21.15.18:56346) 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e7f10 0x55d3a30f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37620/0 (socket says 172.21.15.18:37620) 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f8039000640 1 -- 172.21.15.18:0/864608925 learned_addr learned my addr 172.21.15.18:0/864608925 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.078+0000 7f803a400640 1 --2- 172.21.15.18:0/864608925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 0x55d3a3109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8039000640 1 -- 172.21.15.18:0/864608925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e7f10 msgr2=0x55d3a30f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:17.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8039000640 1 --2- 172.21.15.18:0/864608925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e7f10 0x55d3a30f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8039000640 1 -- 172.21.15.18:0/864608925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 msgr2=0x55d3a3109270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:17.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8039000640 1 --2- 172.21.15.18:0/864608925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 0x55d3a3109270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8039000640 1 -- 172.21.15.18:0/864608925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a30e8660 con 0x55d3a30e8830 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8039000640 1 --2- 172.21.15.18:0/864608925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30fd0f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55d3a30edc30 tx=0x7f802000ae10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51270475ec9105f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8026000640 1 -- 172.21.15.18:0/864608925 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802001a490 con 0x55d3a30e8830 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8026000640 1 -- 172.21.15.18:0/864608925 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802001e030 con 0x55d3a30e8830 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.079+0000 7f8026000640 1 -- 172.21.15.18:0/864608925 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200223d0 con 0x55d3a30e8830 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- 172.21.15.18:0/864608925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 msgr2=0x55d3a30fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 --2- 172.21.15.18:0/864608925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30fd0f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55d3a30edc30 tx=0x7f802000ae10 comp rx=0 tx=0).stop 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- 172.21.15.18:0/864608925 shutdown_connections 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 --2- 172.21.15.18:0/864608925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 0x55d3a3109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 --2- 172.21.15.18:0/864608925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e8830 0x55d3a30fd0f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 --2- 172.21.15.18:0/864608925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e7f10 0x55d3a30f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- 172.21.15.18:0/864608925 >> 172.21.15.18:0/864608925 conn(0x55d3a30e2640 msgr2=0x55d3a30ed400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- 172.21.15.18:0/864608925 shutdown_connections 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- 172.21.15.18:0/864608925 wait complete. 2024-04-16T17:17:17.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- start start 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 Processor -- start 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e7f10 0x55d3a30f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 0x55d3a30f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f80341afd50 con 0x55d3a30f1410 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80341b11c0 con 0x55d3a30e7f10 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80341b0420 con 0x55d3a30e8830 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 10 client.0 _ll_register_callbacks cb 0x7f802c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e7f10 0x55d3a30f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e7f10 0x55d3a30f9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56348/0 (socket says 172.21.15.18:56348) 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 -- 172.21.15.18:0/3846729686 learned_addr learned my addr 172.21.15.18:0/3846729686 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f8039000640 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 0x55d3a30f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 -- 172.21.15.18:0/3846729686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 msgr2=0x55d3a30f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 -- 172.21.15.18:0/3846729686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 msgr2=0x55d3a30f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 0x55d3a30f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 -- 172.21.15.18:0/3846729686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a2d5f3f0 con 0x55d3a30e7f10 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f8039000640 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803a400640 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e7f10 0x55d3a30f9540 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f802c006010 tx=0x7f802c004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0760a13e94aee92 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f801b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f801b400640 1 -- 172.21.15.18:0/3846729686 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c029070 con 0x55d3a30e7f10 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f801b400640 1 -- 172.21.15.18:0/3846729686 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f802c002030 con 0x55d3a30e7f10 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- 172.21.15.18:0/3846729686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f802001e030 con 0x55d3a30e7f10 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f801b400640 1 -- 172.21.15.18:0/3846729686 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f802c003c40 con 0x55d3a30e7f10 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 10 client.29048 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- 172.21.15.18:0/3846729686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d3a2e297d0 con 0x55d3a30e7f10 2024-04-16T17:17:17.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f801b400640 1 -- 172.21.15.18:0/3846729686 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802001e030 con 0x55d3a30e7f10 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f801b400640 1 -- 172.21.15.18:0/3846729686 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f802c02e440 con 0x55d3a30e7f10 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 10 client.29048 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 10 client.29048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 20 client.29048 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3a3167930 0x55d3a3169d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 4 client.29048 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f803f1e9740 1 -- 172.21.15.18:0/3846729686 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3a30548b0 con 0x55d3a3167930 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f8039000640 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3a3167930 0x55d3a3169d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f8039000640 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3a3167930 0x55d3a3169d50 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cde8f425ba5220c server_cookie=94ae92c64f9464ef in_seq=0 out_seq=0 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f801b400640 10 client.29048 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.080+0000 7f801b400640 1 -- 172.21.15.18:0/3846729686 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3a30548b0 con 0x55d3a3167930 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f801b400640 10 client.29048 handle_command_reply: tid=0 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 client.29048 shutdown 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 -- 172.21.15.18:0/3846729686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e7f10 msgr2=0x55d3a30f9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e7f10 0x55d3a30f9540 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f802c006010 tx=0x7f802c004d50 comp rx=0 tx=0).stop 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 -- 172.21.15.18:0/3846729686 shutdown_connections 2024-04-16T17:17:17.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d3a3167930 0x55d3a3169d50 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d3a30f1410 0x55d3a30f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d3a30e8830 0x55d3a30f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 --2- 172.21.15.18:0/3846729686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a30e7f10 0x55d3a30f9540 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 -- 172.21.15.18:0/3846729686 >> 172.21.15.18:0/3846729686 conn(0x55d3a30e2640 msgr2=0x55d3a30e0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f801b400640 0 client.29048 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 -- 172.21.15.18:0/3846729686 shutdown_connections 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 1 -- 172.21.15.18:0/3846729686 wait complete. 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803f1e9740 20 client.29048 trim_cache size 0 max 16384 2024-04-16T17:17:17.090 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 -- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f801c0a3da0 msgr2=0x7f801c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f801c0a3da0 0x7f801c0a6260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55d3a30e79e0 tx=0x7f802802c500 comp rx=0 tx=0).stop 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 -- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 msgr2=0x7f8034100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 0x7f8034100c30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8028006010 tx=0x7f802802cbe0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 -- 172.21.15.18:0/2276959712 shutdown_connections 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f801c0a3da0 0x7f801c0a6260 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803410c400 0x7f80341016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8034103900 0x7f8034101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 --2- 172.21.15.18:0/2276959712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8034102fe0 0x7f8034100c30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:17.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 -- 172.21.15.18:0/2276959712 >> 172.21.15.18:0/2276959712 conn(0x7f8034077ab0 msgr2=0x7f80340fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:17.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 -- 172.21.15.18:0/2276959712 shutdown_connections 2024-04-16T17:17:17.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:17.085+0000 7f803ae00640 1 -- 172.21.15.18:0/2276959712 wait complete. 2024-04-16T17:17:17.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:16 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:17:17.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:16 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:17:17.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:16 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:17:17.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:16 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2252640918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:17.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:16 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:17:17.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:16 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:17:17.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:16 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:17:17.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:16 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2252640918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:17.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 467629 3.57 MB/sec execute 570 sec latency 84.311 ms 2024-04-16T17:17:18.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:17:18.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:17:18.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:17:18.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:17 smithi022 ceph-mon[36023]: pgmap v398: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:17:18.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:17 smithi043 ceph-mon[36901]: pgmap v398: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 -- 172.21.15.18:0/8614072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574104c30 msgr2=0x7ff574108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 --2- 172.21.15.18:0/8614072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574104c30 0x7ff574108c70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff560009860 tx=0x7ff56002bb70 comp rx=0 tx=0).stop 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 -- 172.21.15.18:0/8614072 shutdown_connections 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 --2- 172.21.15.18:0/8614072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff574109350 0x7ff574110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 --2- 172.21.15.18:0/8614072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574104c30 0x7ff574108c70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 --2- 172.21.15.18:0/8614072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff574104280 0x7ff574104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 -- 172.21.15.18:0/8614072 >> 172.21.15.18:0/8614072 conn(0x7ff5740ffae0 msgr2=0x7ff574101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 -- 172.21.15.18:0/8614072 shutdown_connections 2024-04-16T17:17:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 -- 172.21.15.18:0/8614072 wait complete. 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.306+0000 7ff57a800640 1 Processor -- start 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 -- start start 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff574104280 0x7ff5741a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574109350 0x7ff5741a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57419a7a0 0x7ff57419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741143e0 con 0x7ff574109350 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff574114260 con 0x7ff574104280 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff574114560 con 0x7ff57419a7a0 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574109350 0x7ff5741a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff579400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff574104280 0x7ff5741a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574109350 0x7ff5741a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36468/0 (socket says 172.21.15.18:36468) 2024-04-16T17:17:18.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 -- 172.21.15.18:0/1822189948 learned_addr learned my addr 172.21.15.18:0/1822189948 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 -- 172.21.15.18:0/1822189948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57419a7a0 msgr2=0x7ff57419ac50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:17:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57419a7a0 0x7ff57419ac50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 -- 172.21.15.18:0/1822189948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff574104280 msgr2=0x7ff5741a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff574104280 0x7ff5741a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 -- 172.21.15.18:0/1822189948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57419b330 con 0x7ff574109350 2024-04-16T17:17:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff579400640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff574104280 0x7ff5741a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:18.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff578a00640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574109350 0x7ff5741a0b70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff560037040 tx=0x7ff5600045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff572000640 1 -- 172.21.15.18:0/1822189948 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560041020 con 0x7ff574109350 2024-04-16T17:17:18.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff572000640 1 -- 172.21.15.18:0/1822189948 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56002c9f0 con 0x7ff574109350 2024-04-16T17:17:18.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff572000640 1 -- 172.21.15.18:0/1822189948 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560039040 con 0x7ff574109350 2024-04-16T17:17:18.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 -- 172.21.15.18:0/1822189948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57419b5c0 con 0x7ff574109350 2024-04-16T17:17:18.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.312+0000 7ff57a800640 1 -- 172.21.15.18:0/1822189948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741a7770 con 0x7ff574109350 2024-04-16T17:17:18.315 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:07 smithi018 ceph-mon[31776]: pgmap v392: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:09 smithi018 ceph-mon[31776]: pgmap v393: 129 pgs: 9 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 94 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:11 smithi018 ceph-mon[31776]: pgmap v394: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:13 smithi018 ceph-mon[31776]: pgmap v395: 129 pgs: 129 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:13 smithi018 ceph-mon[31776]: osdmap e113: 12 total, 12 up, 12 in 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:14 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:15 smithi018 ceph-mon[31776]: pgmap v397: 129 pgs: 129 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:16 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:16 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:16 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:16 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2252640918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:18.316 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:17 smithi018 ceph-mon[31776]: pgmap v398: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:17:18.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.318+0000 7ff57ebee740 1 Processor -- start 2024-04-16T17:17:18.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.318+0000 7ff57ebee740 1 -- start start 2024-04-16T17:17:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff572000640 1 -- 172.21.15.18:0/1822189948 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff560044020 con 0x7ff574109350 2024-04-16T17:17:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a429270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a431e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a434780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff57ebee740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c21a38dcb0 con 0x55c21a428e90 2024-04-16T17:17:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff57ebee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c21a334370 con 0x55c21a4297b0 2024-04-16T17:17:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff57ebee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c21a389c70 con 0x55c21a432390 2024-04-16T17:17:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff579400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a431e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff579e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a429270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff579400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a431e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56392/0 (socket says 172.21.15.18:56392) 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff579400640 1 -- 172.21.15.18:0/1005494466 learned_addr learned my addr 172.21.15.18:0/1005494466 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff578a00640 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a434780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.319+0000 7ff579400640 1 -- 172.21.15.18:0/1005494466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 msgr2=0x55c21a434780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff579400640 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a434780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff579400640 1 -- 172.21.15.18:0/1005494466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 msgr2=0x55c21a429270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff579400640 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a429270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff579400640 1 -- 172.21.15.18:0/1005494466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21a353910 con 0x55c21a4297b0 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff578a00640 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a434780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff579e00640 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a429270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff579400640 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a431e50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff568009fd0 tx=0x7ff5680118e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf3afd2f16b2186 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff566600640 1 -- 172.21.15.18:0/1005494466 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56801e070 con 0x55c21a4297b0 2024-04-16T17:17:18.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff566600640 1 -- 172.21.15.18:0/1005494466 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56800f9c0 con 0x55c21a4297b0 2024-04-16T17:17:18.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.320+0000 7ff566600640 1 -- 172.21.15.18:0/1005494466 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568019b20 con 0x55c21a4297b0 2024-04-16T17:17:18.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.321+0000 7ff572000640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff55c0a3d20 0x7ff55c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.321+0000 7ff572000640 1 -- 172.21.15.18:0/1822189948 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7ff5600f2530 con 0x7ff574109350 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.321+0000 7ff579e00640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff55c0a3d20 0x7ff55c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.321+0000 7ff57ebee740 1 -- 172.21.15.18:0/1005494466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 msgr2=0x55c21a431e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.321+0000 7ff57ebee740 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a431e50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff568009fd0 tx=0x7ff5680118e0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff57ebee740 1 -- 172.21.15.18:0/1005494466 shutdown_connections 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff57ebee740 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a434780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff57ebee740 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a431e50 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff57ebee740 1 --2- 172.21.15.18:0/1005494466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a429270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff57ebee740 1 -- 172.21.15.18:0/1005494466 >> 172.21.15.18:0/1005494466 conn(0x55c21a4235c0 msgr2=0x55c21a4239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff579e00640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff55c0a3d20 0x7ff55c0a61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005fd0 tx=0x7ff56c009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff57ebee740 1 -- 172.21.15.18:0/1005494466 shutdown_connections 2024-04-16T17:17:18.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff57ebee740 1 -- 172.21.15.18:0/1005494466 wait complete. 2024-04-16T17:17:18.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.322+0000 7ff57ebee740 1 -- start start 2024-04-16T17:17:18.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.323+0000 7ff57ebee740 1 Processor -- start 2024-04-16T17:17:18.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.323+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a42f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.323+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a42f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.323+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a42fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.323+0000 7ff57ebee740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c21a435180 con 0x55c21a428e90 2024-04-16T17:17:18.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.323+0000 7ff57ebee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c21a347730 con 0x55c21a4297b0 2024-04-16T17:17:18.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.323+0000 7ff57ebee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c21a435300 con 0x55c21a432390 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.323+0000 7ff57ebee740 10 client.0 _ll_register_callbacks cb 0x55c21a3161e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a42fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a42fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37670/0 (socket says 172.21.15.18:37670) 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 -- 172.21.15.18:0/2939429185 learned_addr learned my addr 172.21.15.18:0/2939429185 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579400640 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a42f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff578a00640 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a42f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 -- 172.21.15.18:0/2939429185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 msgr2=0x55c21a42f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a42f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 -- 172.21.15.18:0/2939429185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 msgr2=0x55c21a42f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a42f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 -- 172.21.15.18:0/2939429185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21a1687e0 con 0x55c21a432390 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579400640 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a42f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff579e00640 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a42fea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff56c003f50 tx=0x7ff56c013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a86a0930f44ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.324+0000 7ff55aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:18.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.325+0000 7ff55aa00640 1 -- 172.21.15.18:0/2939429185 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00f890 con 0x55c21a432390 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.325+0000 7ff57ebee740 1 -- 172.21.15.18:0/2939429185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff56800f9c0 con 0x55c21a432390 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.325+0000 7ff55aa00640 1 -- 172.21.15.18:0/2939429185 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7ff56c00fa30 con 0x55c21a432390 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.325+0000 7ff55aa00640 1 -- 172.21.15.18:0/2939429185 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff56c02f050 con 0x55c21a432390 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.325+0000 7ff57ebee740 10 client.28906 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.325+0000 7ff57ebee740 1 -- 172.21.15.18:0/2939429185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c21a1c8f70 con 0x55c21a432390 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.325+0000 7ff55aa00640 1 -- 172.21.15.18:0/2939429185 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56800f9c0 con 0x55c21a432390 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.326+0000 7ff55aa00640 1 -- 172.21.15.18:0/2939429185 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff56c00e3b0 con 0x55c21a432390 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.326+0000 7ff57ebee740 10 client.28906 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.326+0000 7ff57ebee740 10 client.28906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.326+0000 7ff57ebee740 20 client.28906 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.326+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c21a4426d0 0x55c21a444b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.326+0000 7ff57ebee740 4 client.28906 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.326+0000 7ff57ebee740 1 -- 172.21.15.18:0/2939429185 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c21a39b6d0 con 0x55c21a4426d0 2024-04-16T17:17:18.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.326+0000 7ff579400640 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c21a4426d0 0x55c21a444b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.327+0000 7ff579400640 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c21a4426d0 0x55c21a444b10 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36bd5e723a12a00c server_cookie=f1586da783b0329d in_seq=0 out_seq=0 2024-04-16T17:17:18.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.327+0000 7ff55aa00640 10 client.28906 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff55aa00640 1 -- 172.21.15.18:0/2939429185 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c21a39b6d0 con 0x55c21a4426d0 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff55aa00640 10 client.28906 handle_command_reply: tid=0 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 client.28906 shutdown 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 -- 172.21.15.18:0/2939429185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 msgr2=0x55c21a42fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a42fea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff56c003f50 tx=0x7ff56c013040 comp rx=0 tx=0).stop 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 -- 172.21.15.18:0/2939429185 shutdown_connections 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c21a4426d0 0x55c21a444b10 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a432390 0x55c21a42fea0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a42f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2939429185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a42f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 -- 172.21.15.18:0/2939429185 >> 172.21.15.18:0/2939429185 conn(0x55c21a4235c0 msgr2=0x55c21a425310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff55aa00640 0 client.28906 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 -- 172.21.15.18:0/2939429185 shutdown_connections 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 1 -- 172.21.15.18:0/2939429185 wait complete. 2024-04-16T17:17:18.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.328+0000 7ff57ebee740 20 client.28906 trim_cache size 0 max 16384 2024-04-16T17:17:18.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.348+0000 7ff57ebee740 1 Processor -- start 2024-04-16T17:17:18.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.348+0000 7ff57ebee740 1 -- start start 2024-04-16T17:17:18.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a428e90 0x55c21a43e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a4223e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a432390 0x55c21a44a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff57ebee740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b4010 con 0x55c21a432390 2024-04-16T17:17:18.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff57ebee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b18e0 con 0x55c21a4297b0 2024-04-16T17:17:18.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff57ebee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741af750 con 0x55c21a428e90 2024-04-16T17:17:18.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a432390 0x55c21a44a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a432390 0x55c21a44a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36482/0 (socket says 172.21.15.18:36482) 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579e00640 1 -- 172.21.15.18:0/2292071792 learned_addr learned my addr 172.21.15.18:0/2292071792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff578a00640 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a4223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579400640 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a428e90 0x55c21a43e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579e00640 1 -- 172.21.15.18:0/2292071792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a428e90 msgr2=0x55c21a43e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579e00640 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a428e90 0x55c21a43e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579e00640 1 -- 172.21.15.18:0/2292071792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 msgr2=0x55c21a4223e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579e00640 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a4223e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.349+0000 7ff579e00640 1 -- 172.21.15.18:0/2292071792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21a060170 con 0x55c21a432390 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.350+0000 7ff579e00640 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a432390 0x55c21a44a3e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005fa0 tx=0x7ff56c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f9eb1b6990d4d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.350+0000 7ff565c00640 1 -- 172.21.15.18:0/2292071792 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00e3b0 con 0x55c21a432390 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.350+0000 7ff565c00640 1 -- 172.21.15.18:0/2292071792 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56c037020 con 0x55c21a432390 2024-04-16T17:17:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.350+0000 7ff565c00640 1 -- 172.21.15.18:0/2292071792 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00f690 con 0x55c21a432390 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 -- 172.21.15.18:0/2292071792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a432390 msgr2=0x55c21a44a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a432390 0x55c21a44a3e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005fa0 tx=0x7ff56c005cf0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 -- 172.21.15.18:0/2292071792 shutdown_connections 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a432390 0x55c21a44a3e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a4297b0 0x55c21a4223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 --2- 172.21.15.18:0/2292071792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a428e90 0x55c21a43e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 -- 172.21.15.18:0/2292071792 >> 172.21.15.18:0/2292071792 conn(0x55c21a4235c0 msgr2=0x55c21a421cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 -- 172.21.15.18:0/2292071792 shutdown_connections 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.351+0000 7ff57ebee740 1 -- 172.21.15.18:0/2292071792 wait complete. 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 -- start start 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 Processor -- start 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a435f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a4297b0 0x55c21a4364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a432390 0x55c21a4369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741afda0 con 0x55c21a428e90 2024-04-16T17:17:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b1210 con 0x55c21a432390 2024-04-16T17:17:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b0470 con 0x55c21a4297b0 2024-04-16T17:17:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 10 client.0 _ll_register_callbacks cb 0x7ff56c003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:18.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a432390 0x55c21a4369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a432390 0x55c21a4369e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56402/0 (socket says 172.21.15.18:56402) 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 -- 172.21.15.18:0/713509952 learned_addr learned my addr 172.21.15.18:0/713509952 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff578a00640 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a4297b0 0x55c21a4364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579400640 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a435f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 -- 172.21.15.18:0/713509952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a4297b0 msgr2=0x55c21a4364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a4297b0 0x55c21a4364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 -- 172.21.15.18:0/713509952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 msgr2=0x55c21a435f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a435f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 -- 172.21.15.18:0/713509952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21a4a7c50 con 0x55c21a432390 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579400640 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a435f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff579e00640 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a432390 0x55c21a4369e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff56c039040 tx=0x7ff56c00e5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ab87ef4225e20b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff55aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff55aa00640 1 -- 172.21.15.18:0/713509952 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00e970 con 0x55c21a432390 2024-04-16T17:17:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 -- 172.21.15.18:0/713509952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff56c037020 con 0x55c21a432390 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff55aa00640 1 -- 172.21.15.18:0/713509952 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7ff56c004aa0 con 0x55c21a432390 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff55aa00640 1 -- 172.21.15.18:0/713509952 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff56c049050 con 0x55c21a432390 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 10 client.29066 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 -- 172.21.15.18:0/713509952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c21a4a8f70 con 0x55c21a432390 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff55aa00640 1 -- 172.21.15.18:0/713509952 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56c037020 con 0x55c21a432390 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff55aa00640 1 -- 172.21.15.18:0/713509952 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff56c044030 con 0x55c21a432390 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 10 client.29066 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 10 client.29066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 20 client.29066 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c21a4a95d0 0x55c21a4ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 4 client.29066 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:17:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.352+0000 7ff57ebee740 1 -- 172.21.15.18:0/713509952 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c21a395540 con 0x55c21a4a95d0 2024-04-16T17:17:18.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.360+0000 7ff579400640 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c21a4a95d0 0x55c21a4ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.360+0000 7ff579400640 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c21a4a95d0 0x55c21a4ab9f0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b11805467bb0567 server_cookie=baacd837689db337 in_seq=0 out_seq=0 2024-04-16T17:17:18.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.360+0000 7ff55aa00640 10 client.29066 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:18.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.361+0000 7ff55aa00640 1 -- 172.21.15.18:0/713509952 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c21a395540 con 0x55c21a4a95d0 2024-04-16T17:17:18.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.361+0000 7ff55aa00640 10 client.29066 handle_command_reply: tid=0 2024-04-16T17:17:18.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.361+0000 7ff57ebee740 1 client.29066 shutdown 2024-04-16T17:17:18.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff57ebee740 1 -- 172.21.15.18:0/713509952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a432390 msgr2=0x55c21a4369e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff57ebee740 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a432390 0x55c21a4369e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff56c039040 tx=0x7ff56c00e5e0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff57ebee740 1 -- 172.21.15.18:0/713509952 shutdown_connections 2024-04-16T17:17:18.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff57ebee740 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c21a4a95d0 0x55c21a4ab9f0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff57ebee740 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c21a432390 0x55c21a4369e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff57ebee740 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c21a4297b0 0x55c21a4364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff57ebee740 1 --2- 172.21.15.18:0/713509952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c21a428e90 0x55c21a435f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff57ebee740 1 -- 172.21.15.18:0/713509952 >> 172.21.15.18:0/713509952 conn(0x55c21a4235c0 msgr2=0x55c21a42fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.362+0000 7ff55aa00640 0 client.29066 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:18.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.367+0000 7ff57ebee740 1 -- 172.21.15.18:0/713509952 shutdown_connections 2024-04-16T17:17:18.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.367+0000 7ff57ebee740 1 -- 172.21.15.18:0/713509952 wait complete. 2024-04-16T17:17:18.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.368+0000 7ff57ebee740 20 client.29066 trim_cache size 0 max 16384 2024-04-16T17:17:18.368 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:17:18.368 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:17:18.368 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "be2405a6-9890-468f-98cc-f379f22ba599", 2024-04-16T17:17:18.368 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:17:18.368 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:17:18.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 -- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff55c0a3d20 msgr2=0x7ff55c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff55c0a3d20 0x7ff55c0a61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005fd0 tx=0x7ff56c009500 comp rx=0 tx=0).stop 2024-04-16T17:17:18.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 -- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574109350 msgr2=0x7ff5741a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574109350 0x7ff5741a0b70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff560037040 tx=0x7ff5600045b0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 -- 172.21.15.18:0/1822189948 shutdown_connections 2024-04-16T17:17:18.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff55c0a3d20 0x7ff55c0a61e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57419a7a0 0x7ff57419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff574109350 0x7ff5741a0b70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 --2- 172.21.15.18:0/1822189948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff574104280 0x7ff5741a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 -- 172.21.15.18:0/1822189948 >> 172.21.15.18:0/1822189948 conn(0x7ff5740ffae0 msgr2=0x7ff574100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 -- 172.21.15.18:0/1822189948 shutdown_connections 2024-04-16T17:17:18.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.369+0000 7ff57a800640 1 -- 172.21.15.18:0/1822189948 wait complete. 2024-04-16T17:17:18.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:17:18.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.581+0000 7f36b7200640 1 -- 172.21.15.18:0/3057725525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b8102a20 msgr2=0x7f36b8106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.581+0000 7f36b7200640 1 --2- 172.21.15.18:0/3057725525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b8102a20 0x7f36b8106a60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009900 tx=0x7f36a402bc40 comp rx=0 tx=0).stop 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.581+0000 7f36b7200640 1 -- 172.21.15.18:0/3057725525 shutdown_connections 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.581+0000 7f36b7200640 1 --2- 172.21.15.18:0/3057725525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36b8107140 0x7f36b810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.581+0000 7f36b7200640 1 --2- 172.21.15.18:0/3057725525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b8102a20 0x7f36b8106a60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.581+0000 7f36b7200640 1 --2- 172.21.15.18:0/3057725525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b8102070 0x7f36b8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.581+0000 7f36b7200640 1 -- 172.21.15.18:0/3057725525 >> 172.21.15.18:0/3057725525 conn(0x7f36b80fd950 msgr2=0x7f36b80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.581+0000 7f36b7200640 1 -- 172.21.15.18:0/3057725525 shutdown_connections 2024-04-16T17:17:18.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.582+0000 7f36b7200640 1 -- 172.21.15.18:0/3057725525 wait complete. 2024-04-16T17:17:18.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.582+0000 7f36b7200640 1 Processor -- start 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b7200640 1 -- start start 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b8102070 0x7f36b8198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b7200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36b8107140 0x7f36b8198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b819cff0 0x7f36b819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b7200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8112200 con 0x7f36b8107140 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8112080 con 0x7f36b819cff0 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8112380 con 0x7f36b8102070 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36b8107140 0x7f36b8198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.583+0000 7f36b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b8102070 0x7f36b8198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36b8107140 0x7f36b8198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36516/0 (socket says 172.21.15.18:36516) 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b5400640 1 -- 172.21.15.18:0/407900739 learned_addr learned my addr 172.21.15.18:0/407900739 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b6800640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b819cff0 0x7f36b819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b5400640 1 -- 172.21.15.18:0/407900739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b8102070 msgr2=0x7f36b8198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b5400640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b8102070 0x7f36b8198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b5400640 1 -- 172.21.15.18:0/407900739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b819cff0 msgr2=0x7f36b819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b5400640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b819cff0 0x7f36b819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b5400640 1 -- 172.21.15.18:0/407900739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b819dae0 con 0x7f36b8107140 2024-04-16T17:17:18.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.584+0000 7f36b5400640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36b8107140 0x7f36b8198cc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f36a402c250 tx=0x7f36a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.585+0000 7f36aea00640 1 -- 172.21.15.18:0/407900739 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40044e0 con 0x7f36b8107140 2024-04-16T17:17:18.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.585+0000 7f36aea00640 1 -- 172.21.15.18:0/407900739 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a402cca0 con 0x7f36b8107140 2024-04-16T17:17:18.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.585+0000 7f36aea00640 1 -- 172.21.15.18:0/407900739 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40035f0 con 0x7f36b8107140 2024-04-16T17:17:18.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.585+0000 7f36b7200640 1 -- 172.21.15.18:0/407900739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b819dd70 con 0x7f36b8107140 2024-04-16T17:17:18.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.585+0000 7f36b7200640 1 -- 172.21.15.18:0/407900739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b80744c0 con 0x7f36b8107140 2024-04-16T17:17:18.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.589+0000 7f36bf7a2740 1 Processor -- start 2024-04-16T17:17:18.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.589+0000 7f36bf7a2740 1 -- start start 2024-04-16T17:17:18.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.589+0000 7f36aea00640 1 -- 172.21.15.18:0/407900739 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f36a403e6b0 con 0x7f36b8107140 2024-04-16T17:17:18.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b10513d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b105146350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b105148c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b1050a3120 con 0x55b10513d390 2024-04-16T17:17:18.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b105049340 con 0x55b10513dcb0 2024-04-16T17:17:18.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b10509f0e0 con 0x55b105146890 2024-04-16T17:17:18.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b105146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36b6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b10513d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b105146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56436/0 (socket says 172.21.15.18:56436) 2024-04-16T17:17:18.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.590+0000 7f36b5e00640 1 -- 172.21.15.18:0/2946926168 learned_addr learned my addr 172.21.15.18:0/2946926168 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36b5400640 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b105148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36b5e00640 1 -- 172.21.15.18:0/2946926168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 msgr2=0x55b105148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36b5e00640 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b105148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36b5e00640 1 -- 172.21.15.18:0/2946926168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 msgr2=0x55b10513d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36b5e00640 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b10513d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36b5e00640 1 -- 172.21.15.18:0/2946926168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b104e7f7d0 con 0x55b10513dcb0 2024-04-16T17:17:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36b5400640 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b105148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36b5e00640 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b105146350 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009cf0 tx=0x7f36a8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab87bf200c1fd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.591+0000 7f36a3000640 1 -- 172.21.15.18:0/2946926168 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004230 con 0x55b10513dcb0 2024-04-16T17:17:18.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.592+0000 7f36a3000640 1 -- 172.21.15.18:0/2946926168 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a80043d0 con 0x55b10513dcb0 2024-04-16T17:17:18.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.592+0000 7f36aea00640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f369c0a3e20 0x7f369c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.592+0000 7f36a3000640 1 -- 172.21.15.18:0/2946926168 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8019960 con 0x55b10513dcb0 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.592+0000 7f36aea00640 1 -- 172.21.15.18:0/407900739 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f36a40f6300 con 0x7f36b8107140 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.592+0000 7f36b6800640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f369c0a3e20 0x7f369c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 -- 172.21.15.18:0/2946926168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 msgr2=0x55b105146350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b105146350 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009cf0 tx=0x7f36a8011810 comp rx=0 tx=0).stop 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 -- 172.21.15.18:0/2946926168 shutdown_connections 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b105148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b105146350 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/2946926168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b10513d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 -- 172.21.15.18:0/2946926168 >> 172.21.15.18:0/2946926168 conn(0x55b105137ac0 msgr2=0x55b105137ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 -- 172.21.15.18:0/2946926168 shutdown_connections 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36b6800640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f369c0a3e20 0x7f369c0a62e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f36b0005fd0 tx=0x7f36b0009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.593+0000 7f36bf7a2740 1 -- 172.21.15.18:0/2946926168 wait complete. 2024-04-16T17:17:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 1 -- start start 2024-04-16T17:17:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 1 Processor -- start 2024-04-16T17:17:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105143990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 0x55b105143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105144410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b105149680 con 0x55b10513dcb0 2024-04-16T17:17:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b10505c710 con 0x55b105146890 2024-04-16T17:17:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b105149800 con 0x55b10513d390 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.594+0000 7f36bf7a2740 10 client.0 _ll_register_callbacks cb 0x55b105057af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105143990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37734/0 (socket says 172.21.15.18:37734) 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 -- 172.21.15.18:0/312252663 learned_addr learned my addr 172.21.15.18:0/312252663 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5400640 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 0x55b105143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b6800640 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 -- 172.21.15.18:0/312252663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 msgr2=0x55b105144410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105144410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 -- 172.21.15.18:0/312252663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 msgr2=0x55b105143ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 0x55b105143ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 -- 172.21.15.18:0/312252663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b104db53f0 con 0x55b10513d390 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f36b5e00640 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105143990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f36a80195c0 tx=0x7f36a8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21c2fa190857bc28 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.595+0000 7f3697400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.596+0000 7f3697400640 1 -- 172.21.15.18:0/312252663 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8027070 con 0x55b10513d390 2024-04-16T17:17:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.596+0000 7f36bf7a2740 1 -- 172.21.15.18:0/312252663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a80043d0 con 0x55b10513d390 2024-04-16T17:17:18.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.596+0000 7f3697400640 1 -- 172.21.15.18:0/312252663 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f36a8005240 con 0x55b10513d390 2024-04-16T17:17:18.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.596+0000 7f3697400640 1 -- 172.21.15.18:0/312252663 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f36a8033050 con 0x55b10513d390 2024-04-16T17:17:18.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.596+0000 7f36bf7a2740 10 client.28921 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:18.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.596+0000 7f36bf7a2740 1 -- 172.21.15.18:0/312252663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b104d750f0 con 0x55b10513d390 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.597+0000 7f3697400640 1 -- 172.21.15.18:0/312252663 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a80043d0 con 0x55b10513d390 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.597+0000 7f3697400640 1 -- 172.21.15.18:0/312252663 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f36a80046c0 con 0x55b10513d390 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.598+0000 7f36bf7a2740 10 client.28921 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.598+0000 7f36bf7a2740 10 client.28921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.598+0000 7f36bf7a2740 20 client.28921 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.598+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b105156c30 0x55b105159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.598+0000 7f36bf7a2740 4 client.28921 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.598+0000 7f36bf7a2740 1 -- 172.21.15.18:0/312252663 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1050b0b40 con 0x55b105156c30 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.598+0000 7f36b5400640 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b105156c30 0x55b105159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.598+0000 7f36b5400640 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b105156c30 0x55b105159020 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf35e4b0f99e1333 server_cookie=ef302b5b4a20226f in_seq=0 out_seq=0 2024-04-16T17:17:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.599+0000 7f3697400640 10 client.28921 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:18.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.599+0000 7f3697400640 1 -- 172.21.15.18:0/312252663 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1050b0b40 con 0x55b105156c30 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.599+0000 7f3697400640 10 client.28921 handle_command_reply: tid=0 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.599+0000 7f36bf7a2740 1 client.28921 shutdown 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.599+0000 7f36bf7a2740 1 -- 172.21.15.18:0/312252663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 msgr2=0x55b105143990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.599+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105143990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f36a80195c0 tx=0x7f36a8003820 comp rx=0 tx=0).stop 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 1 -- 172.21.15.18:0/312252663 shutdown_connections 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b105156c30 0x55b105159020 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105144410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 0x55b105143ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/312252663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105143990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 1 -- 172.21.15.18:0/312252663 >> 172.21.15.18:0/312252663 conn(0x55b105137ac0 msgr2=0x55b105139810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f3697400640 0 client.28921 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 1 -- 172.21.15.18:0/312252663 shutdown_connections 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 1 -- 172.21.15.18:0/312252663 wait complete. 2024-04-16T17:17:18.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.600+0000 7f36bf7a2740 20 client.28921 trim_cache size 0 max 16384 2024-04-16T17:17:18.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.619+0000 7f36bf7a2740 1 Processor -- start 2024-04-16T17:17:18.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.619+0000 7f36bf7a2740 1 -- start start 2024-04-16T17:17:18.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.619+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105152570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.619+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 0x55b105143140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.619+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105139560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.619+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81afb50 con 0x55b10513dcb0 2024-04-16T17:17:18.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.619+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81ad420 con 0x55b105146890 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.619+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81ab290 con 0x55b10513d390 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37750/0 (socket says 172.21.15.18:37750) 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 -- 172.21.15.18:0/3170370725 learned_addr learned my addr 172.21.15.18:0/3170370725 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56462/0 (socket says 172.21.15.18:56462) 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5400640 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 0x55b105143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 -- 172.21.15.18:0/3170370725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 msgr2=0x55b105139560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105139560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 -- 172.21.15.18:0/3170370725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 msgr2=0x55b105143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 0x55b105143140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 -- 172.21.15.18:0/3170370725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b105159390 con 0x55b10513d390 2024-04-16T17:17:18.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b6800640 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105139560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:18.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.620+0000 7f36b5e00640 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105152570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009cf0 tx=0x7f36a8005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a54dfdd93174227a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.621+0000 7f36a2600640 1 -- 172.21.15.18:0/3170370725 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8030070 con 0x55b10513d390 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.621+0000 7f36a2600640 1 -- 172.21.15.18:0/3170370725 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a8003890 con 0x55b10513d390 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.621+0000 7f36a2600640 1 -- 172.21.15.18:0/3170370725 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8027030 con 0x55b10513d390 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 -- 172.21.15.18:0/3170370725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 msgr2=0x55b105152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105152570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009cf0 tx=0x7f36a8005b90 comp rx=0 tx=0).stop 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 -- 172.21.15.18:0/3170370725 shutdown_connections 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b105146890 0x55b105139560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513dcb0 0x55b105143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/3170370725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b10513d390 0x55b105152570 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 -- 172.21.15.18:0/3170370725 >> 172.21.15.18:0/3170370725 conn(0x55b105137ac0 msgr2=0x55b105138e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 -- 172.21.15.18:0/3170370725 shutdown_connections 2024-04-16T17:17:18.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.622+0000 7f36bf7a2740 1 -- 172.21.15.18:0/3170370725 wait complete. 2024-04-16T17:17:18.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.623+0000 7f36bf7a2740 1 -- start start 2024-04-16T17:17:18.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.623+0000 7f36bf7a2740 1 Processor -- start 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b105149c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b10514a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36bf7a2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b1051c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81ab8e0 con 0x55b10513d390 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81acd50 con 0x55b10513dcb0 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36bf7a2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81abfb0 con 0x55b105146890 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36bf7a2740 10 client.0 _ll_register_callbacks cb 0x7f36a8009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36b5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b10514a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36b5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b10514a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56476/0 (socket says 172.21.15.18:56476) 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.624+0000 7f36b5400640 1 -- 172.21.15.18:0/1781697477 learned_addr learned my addr 172.21.15.18:0/1781697477 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:18.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.625+0000 7f36b6800640 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b1051c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.625+0000 7f36b5e00640 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b105149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.625+0000 7f36b5400640 1 -- 172.21.15.18:0/1781697477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 msgr2=0x55b1051c3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.625+0000 7f36b5400640 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b1051c3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.625+0000 7f36b5400640 1 -- 172.21.15.18:0/1781697477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 msgr2=0x55b105149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.625+0000 7f36b5400640 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b105149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.625+0000 7f36b5400640 1 -- 172.21.15.18:0/1781697477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1051b50b0 con 0x55b10513dcb0 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.626+0000 7f36b5e00640 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b105149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.626+0000 7f36b5400640 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b10514a1b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f36a40c1cf0 tx=0x7f36a403ea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78808ce71d44b39f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.626+0000 7f3697400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.626+0000 7f3697400640 1 -- 172.21.15.18:0/1781697477 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40c1ff0 con 0x55b10513dcb0 2024-04-16T17:17:18.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.626+0000 7f3697400640 1 -- 172.21.15.18:0/1781697477 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f36a40be660 con 0x55b10513dcb0 2024-04-16T17:17:18.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.626+0000 7f36bf7a2740 1 -- 172.21.15.18:0/1781697477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a8003890 con 0x55b10513dcb0 2024-04-16T17:17:18.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.627+0000 7f3697400640 1 -- 172.21.15.18:0/1781697477 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f36a40c62f0 con 0x55b10513dcb0 2024-04-16T17:17:18.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.627+0000 7f36bf7a2740 10 client.29084 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:18.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.627+0000 7f36bf7a2740 1 -- 172.21.15.18:0/1781697477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b1051b6400 con 0x55b10513dcb0 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.627+0000 7f3697400640 1 -- 172.21.15.18:0/1781697477 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a8003890 con 0x55b10513dcb0 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.627+0000 7f3697400640 1 -- 172.21.15.18:0/1781697477 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f36a403c030 con 0x55b10513dcb0 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f36bf7a2740 10 client.29084 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f36bf7a2740 10 client.29084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f36bf7a2740 20 client.29084 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b1051b6c00 0x55b1051b9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f36bf7a2740 4 client.29084 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f36bf7a2740 1 -- 172.21.15.18:0/1781697477 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1050aa9b0 con 0x55b1051b6c00 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f36b5e00640 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b1051b6c00 0x55b1051b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f3697400640 10 client.29084 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:18.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.628+0000 7f36b5e00640 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b1051b6c00 0x55b1051b9020 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=527c7061057b7a3f server_cookie=340c4b9dd6406e5c in_seq=0 out_seq=0 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.629+0000 7f3697400640 1 -- 172.21.15.18:0/1781697477 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b1050aa9b0 con 0x55b1051b6c00 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f3697400640 10 client.29084 handle_command_reply: tid=0 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 client.29084 shutdown 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 -- 172.21.15.18:0/1781697477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 msgr2=0x55b10514a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b10514a1b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f36a40c1cf0 tx=0x7f36a403ea50 comp rx=0 tx=0).stop 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 -- 172.21.15.18:0/1781697477 shutdown_connections 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b1051b6c00 0x55b1051b9020 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b105146890 0x55b1051c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b10513dcb0 0x55b10514a1b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f3697400640 0 client.29084 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 --2- 172.21.15.18:0/1781697477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b10513d390 0x55b105149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 -- 172.21.15.18:0/1781697477 >> 172.21.15.18:0/1781697477 conn(0x55b105137ac0 msgr2=0x55b1051396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.630+0000 7f36bf7a2740 1 -- 172.21.15.18:0/1781697477 shutdown_connections 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.631+0000 7f36bf7a2740 1 -- 172.21.15.18:0/1781697477 wait complete. 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.631+0000 7f36bf7a2740 20 client.29084 trim_cache size 0 max 16384 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stdout: "be2405a6-9890-468f-98cc-f379f22ba599": { 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "be2405a6-9890-468f-98cc-f379f22ba599", 2024-04-16T17:17:18.632 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:17:18.633 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:17:18.633 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:17:18.633 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:17:18.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.632+0000 7f36b7200640 1 -- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f369c0a3e20 msgr2=0x7f369c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.632+0000 7f36b7200640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f369c0a3e20 0x7f369c0a62e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f36b0005fd0 tx=0x7f36b0009500 comp rx=0 tx=0).stop 2024-04-16T17:17:18.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.632+0000 7f36b7200640 1 -- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36b8107140 msgr2=0x7f36b8198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.632+0000 7f36b7200640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36b8107140 0x7f36b8198cc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f36a402c250 tx=0x7f36a40042f0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.633+0000 7f36b7200640 1 -- 172.21.15.18:0/407900739 shutdown_connections 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.633+0000 7f36b7200640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f369c0a3e20 0x7f369c0a62e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.633+0000 7f36b7200640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b819cff0 0x7f36b819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.633+0000 7f36b7200640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36b8107140 0x7f36b8198cc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.633+0000 7f36b7200640 1 --2- 172.21.15.18:0/407900739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36b8102070 0x7f36b8198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.633+0000 7f36b7200640 1 -- 172.21.15.18:0/407900739 >> 172.21.15.18:0/407900739 conn(0x7f36b80fd950 msgr2=0x7f36b80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.633+0000 7f36b7200640 1 -- 172.21.15.18:0/407900739 shutdown_connections 2024-04-16T17:17:18.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:18.633+0000 7f36b7200640 1 -- 172.21.15.18:0/407900739 wait complete. 2024-04-16T17:17:18.657 INFO:tasks.cephfs.filesystem:scrub status for tag:be2405a6-9890-468f-98cc-f379f22ba599 - {'path': '/', 'tag': 'be2405a6-9890-468f-98cc-f379f22ba599', 'options': 'recursive,force'} 2024-04-16T17:17:18.680 INFO:tasks.workunit.client.0.smithi018.stdout: 1 468345 3.57 MB/sec execute 571 sec latency 45.511 ms 2024-04-16T17:17:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 -- 172.21.15.18:0/2391499566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4109540 msgr2=0x7fd9a4111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 --2- 172.21.15.18:0/2391499566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4109540 0x7fd9a4111110 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009be0 tx=0x7fd99c02bef0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 -- 172.21.15.18:0/2391499566 shutdown_connections 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 --2- 172.21.15.18:0/2391499566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4109540 0x7fd9a4111110 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 --2- 172.21.15.18:0/2391499566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a4104e20 0x7fd9a4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 --2- 172.21.15.18:0/2391499566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9a4104470 0x7fd9a4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 -- 172.21.15.18:0/2391499566 >> 172.21.15.18:0/2391499566 conn(0x7fd9a40ffcf0 msgr2=0x7fd9a4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 -- 172.21.15.18:0/2391499566 shutdown_connections 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.534+0000 7fd9aa600640 1 -- 172.21.15.18:0/2391499566 wait complete. 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.535+0000 7fd9aa600640 1 Processor -- start 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.535+0000 7fd9aa600640 1 -- start start 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9aa600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a4104470 0x7fd9a4110f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9aa600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9a4109540 0x7fd9a410e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9aa600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a410e610 0x7fd9a410eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9aa600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41145e0 con 0x7fd9a4109540 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9aa600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4114460 con 0x7fd9a4104470 2024-04-16T17:17:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9aa600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4114760 con 0x7fd9a410e610 2024-04-16T17:17:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a4104470 0x7fd9a4110f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a4104470 0x7fd9a4110f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56494/0 (socket says 172.21.15.18:56494) 2024-04-16T17:17:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3400640 1 -- 172.21.15.18:0/3558412212 learned_addr learned my addr 172.21.15.18:0/3558412212 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a410e610 0x7fd9a410eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9a4109540 0x7fd9a410e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3400640 1 -- 172.21.15.18:0/3558412212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a410e610 msgr2=0x7fd9a410eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3400640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a410e610 0x7fd9a410eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3400640 1 -- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9a4109540 msgr2=0x7fd9a410e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3400640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9a4109540 0x7fd9a410e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.536+0000 7fd9a3400640 1 -- 172.21.15.18:0/3558412212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a41ab510 con 0x7fd9a4104470 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.537+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9a4109540 0x7fd9a410e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.537+0000 7fd9a3400640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a4104470 0x7fd9a4110f90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd99400e9c0 tx=0x7fd99400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.537+0000 7fd99be00640 1 -- 172.21.15.18:0/3558412212 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99400ccc0 con 0x7fd9a4104470 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.537+0000 7fd9aa600640 1 -- 172.21.15.18:0/3558412212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a41ab800 con 0x7fd9a4104470 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.537+0000 7fd99be00640 1 -- 172.21.15.18:0/3558412212 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd994005500 con 0x7fd9a4104470 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.537+0000 7fd99be00640 1 -- 172.21.15.18:0/3558412212 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994010430 con 0x7fd9a4104470 2024-04-16T17:17:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.537+0000 7fd9aa600640 1 -- 172.21.15.18:0/3558412212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41abd50 con 0x7fd9a4104470 2024-04-16T17:17:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.540+0000 7fd9ad3e7740 1 Processor -- start 2024-04-16T17:17:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.540+0000 7fd9ad3e7740 1 -- start start 2024-04-16T17:17:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7d6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559b7a73b780 con 0x559b7a7dfe20 2024-04-16T17:17:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559b7a6e1f40 con 0x559b7a7d7610 2024-04-16T17:17:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b7a737740 con 0x559b7a7d68a0 2024-04-16T17:17:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9a2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7d6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56510/0 (socket says 172.21.15.18:56510) 2024-04-16T17:17:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.541+0000 7fd9a3e00640 1 -- 172.21.15.18:0/1466316398 learned_addr learned my addr 172.21.15.18:0/1466316398 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.542+0000 7fd9a3e00640 1 -- 172.21.15.18:0/1466316398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 msgr2=0x559b7a7d6d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:17:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.542+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/1466316398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7d6d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.542+0000 7fd9a3e00640 1 -- 172.21.15.18:0/1466316398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 msgr2=0x559b7a7e2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.542+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/1466316398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.542+0000 7fd9a3e00640 1 -- 172.21.15.18:0/1466316398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b7a44e210 con 0x559b7a7d7610 2024-04-16T17:17:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.542+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/1466316398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7d6360 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009be0 tx=0x7fd99c005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6ec02afa5be7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.542+0000 7fd998c00640 1 -- 172.21.15.18:0/1466316398 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c043030 con 0x559b7a7d7610 2024-04-16T17:17:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.543+0000 7fd998c00640 1 -- 172.21.15.18:0/1466316398 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c033c50 con 0x559b7a7d7610 2024-04-16T17:17:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.543+0000 7fd998c00640 1 -- 172.21.15.18:0/1466316398 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c02c890 con 0x559b7a7d7610 2024-04-16T17:17:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.543+0000 7fd99be00640 1 -- 172.21.15.18:0/3558412212 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9940046c0 con 0x7fd9a4104470 2024-04-16T17:17:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.543+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/1466316398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 msgr2=0x559b7a7d6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.543+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/1466316398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7d6360 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009be0 tx=0x7fd99c005c60 comp rx=0 tx=0).stop 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.544+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/1466316398 shutdown_connections 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.544+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/1466316398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.544+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/1466316398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7d6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.544+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/1466316398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7d6360 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.544+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/1466316398 >> 172.21.15.18:0/1466316398 conn(0x559b7a7715d0 msgr2=0x559b7a7d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.544+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/1466316398 shutdown_connections 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.545+0000 7fd99be00640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9880a3c20 0x7fd9880a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.545+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/1466316398 wait complete. 2024-04-16T17:17:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.545+0000 7fd99be00640 1 -- 172.21.15.18:0/3558412212 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fd9940d0fa0 con 0x7fd9a4104470 2024-04-16T17:17:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.545+0000 7fd9ad3e7740 1 -- start start 2024-04-16T17:17:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.546+0000 7fd9ad3e7740 1 Processor -- start 2024-04-16T17:17:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.546+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9880a3c20 0x7fd9880a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7df8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7dd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559b7a7e2af0 con 0x559b7a7dfe20 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559b7a6f5390 con 0x559b7a7d7610 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b7a7e2c70 con 0x559b7a7d68a0 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9ad3e7740 10 client.0 _ll_register_callbacks cb 0x559b7a6def60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7dccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56518/0 (socket says 172.21.15.18:56518) 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.547+0000 7fd9a2a00640 1 -- 172.21.15.18:0/35585403 learned_addr learned my addr 172.21.15.18:0/35585403 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9880a3c20 0x7fd9880a60e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x559b7a7d2720 tx=0x7fd990005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a3400640 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7df8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7dd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a2a00640 1 -- 172.21.15.18:0/35585403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 msgr2=0x559b7a7df8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7df8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a2a00640 1 -- 172.21.15.18:0/35585403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 msgr2=0x559b7a7dd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7dd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a2a00640 1 -- 172.21.15.18:0/35585403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b7a51a620 con 0x559b7a7d7610 2024-04-16T17:17:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7dd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7dccb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd9900118f0 tx=0x7fd9900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9219131b104603 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.548+0000 7fd98d600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.549+0000 7fd98d600640 1 -- 172.21.15.18:0/35585403 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990005220 con 0x559b7a7d7610 2024-04-16T17:17:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.549+0000 7fd98d600640 1 -- 172.21.15.18:0/35585403 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fd9900053c0 con 0x559b7a7d7610 2024-04-16T17:17:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.549+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/35585403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c033c50 con 0x559b7a7d7610 2024-04-16T17:17:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.549+0000 7fd98d600640 1 -- 172.21.15.18:0/35585403 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd9900055c0 con 0x559b7a7d7610 2024-04-16T17:17:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.549+0000 7fd9ad3e7740 10 client.29096 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.549+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/35585403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b7a7eec40 con 0x559b7a7d7610 2024-04-16T17:17:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.551+0000 7fd98d600640 1 -- 172.21.15.18:0/35585403 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c033c50 con 0x559b7a7d7610 2024-04-16T17:17:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.551+0000 7fd98d600640 1 -- 172.21.15.18:0/35585403 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd9900279c0 con 0x559b7a7d7610 2024-04-16T17:17:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.551+0000 7fd9ad3e7740 10 client.29096 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.551+0000 7fd9ad3e7740 10 client.29096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.551+0000 7fd9ad3e7740 20 client.29096 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.551+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b7a7f0150 0x559b7a7f2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.551+0000 7fd9ad3e7740 4 client.29096 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.551+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/35585403 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b7a7491a0 con 0x559b7a7f0150 2024-04-16T17:17:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.552+0000 7fd9a3400640 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b7a7f0150 0x559b7a7f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.552+0000 7fd9a3400640 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b7a7f0150 0x559b7a7f2540 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e79be2f5b840b9d server_cookie=c976cf378864abaf in_seq=0 out_seq=0 2024-04-16T17:17:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.552+0000 7fd98d600640 10 client.29096 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.553+0000 7fd98d600640 1 -- 172.21.15.18:0/35585403 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b7a7491a0 con 0x559b7a7f0150 2024-04-16T17:17:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.553+0000 7fd98d600640 10 client.29096 handle_command_reply: tid=0 2024-04-16T17:17:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 client.29096 shutdown 2024-04-16T17:17:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/35585403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 msgr2=0x559b7a7dccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7dccb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd9900118f0 tx=0x7fd9900046c0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/35585403 shutdown_connections 2024-04-16T17:17:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b7a7f0150 0x559b7a7f2540 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7dd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7dccb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/35585403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7df8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd98d600640 0 client.29096 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/35585403 >> 172.21.15.18:0/35585403 conn(0x559b7a7715d0 msgr2=0x559b7a7d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/35585403 shutdown_connections 2024-04-16T17:17:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.554+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/35585403 wait complete. 2024-04-16T17:17:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.555+0000 7fd9ad3e7740 20 client.29096 trim_cache size 0 max 16384 2024-04-16T17:17:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.588+0000 7fd9ad3e7740 1 Processor -- start 2024-04-16T17:17:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.589+0000 7fd9ad3e7740 1 -- start start 2024-04-16T17:17:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.589+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d68a0 0x559b7a7eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.589+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d7610 0x559b7a7f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.589+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7f8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.589+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b8570 con 0x559b7a7dfe20 2024-04-16T17:17:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.589+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b5e40 con 0x559b7a7d68a0 2024-04-16T17:17:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.589+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b3cb0 con 0x559b7a7d7610 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.590+0000 7fd9a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d68a0 0x559b7a7eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.590+0000 7fd9a3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.590+0000 7fd9a2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d7610 0x559b7a7f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.590+0000 7fd9a3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36590/0 (socket says 172.21.15.18:36590) 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.590+0000 7fd9a3e00640 1 -- 172.21.15.18:0/260336204 learned_addr learned my addr 172.21.15.18:0/260336204 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.590+0000 7fd9a3400640 1 -- 172.21.15.18:0/260336204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d7610 msgr2=0x559b7a7f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.590+0000 7fd9a3400640 1 --2- 172.21.15.18:0/260336204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d7610 0x559b7a7f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.590+0000 7fd9a3400640 1 -- 172.21.15.18:0/260336204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 msgr2=0x559b7a7f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.591+0000 7fd9a3400640 1 --2- 172.21.15.18:0/260336204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.591+0000 7fd9a3400640 1 -- 172.21.15.18:0/260336204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b7a7f28b0 con 0x559b7a7d68a0 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.591+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/260336204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.591+0000 7fd9a3400640 1 --2- 172.21.15.18:0/260336204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d68a0 0x559b7a7eb9f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd9940ce2e0 tx=0x7fd9940bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbe2dcf606a922bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.591+0000 7fd98fe00640 1 -- 172.21.15.18:0/260336204 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99409de60 con 0x559b7a7d68a0 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.591+0000 7fd98fe00640 1 -- 172.21.15.18:0/260336204 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9940a0ee0 con 0x559b7a7d68a0 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.591+0000 7fd98fe00640 1 -- 172.21.15.18:0/260336204 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9940991a0 con 0x559b7a7d68a0 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.592+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/260336204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d68a0 msgr2=0x559b7a7eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.592+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/260336204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d68a0 0x559b7a7eb9f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd9940ce2e0 tx=0x7fd9940bd040 comp rx=0 tx=0).stop 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.592+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/260336204 shutdown_connections 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.592+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/260336204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.592+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/260336204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d7610 0x559b7a7f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.592+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/260336204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d68a0 0x559b7a7eb9f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.592+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/260336204 >> 172.21.15.18:0/260336204 conn(0x559b7a7715d0 msgr2=0x559b7a7d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.593+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/260336204 shutdown_connections 2024-04-16T17:17:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.593+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/260336204 wait complete. 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.593+0000 7fd9ad3e7740 1 -- start start 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.593+0000 7fd9ad3e7740 1 Processor -- start 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9ad3e7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b4300 con 0x559b7a7dfe20 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b5770 con 0x559b7a7d7610 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9ad3e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b49d0 con 0x559b7a7d68a0 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9ad3e7740 10 client.0 _ll_register_callbacks cb 0x7fd990009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7e40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56534/0 (socket says 172.21.15.18:56534) 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9a2a00640 1 -- 172.21.15.18:0/3846661308 learned_addr learned my addr 172.21.15.18:0/3846661308 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9a3400640 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9a2a00640 1 -- 172.21.15.18:0/3846661308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 msgr2=0x559b7a7e3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7e3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.594+0000 7fd9a2a00640 1 -- 172.21.15.18:0/3846661308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 msgr2=0x559b7a7e4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.595+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.595+0000 7fd9a2a00640 1 -- 172.21.15.18:0/3846661308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b7a8558d0 con 0x559b7a7d7610 2024-04-16T17:17:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.595+0000 7fd9a3e00640 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.595+0000 7fd9a2a00640 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7e40d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd990016040 tx=0x7fd990027bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc35c90f6d71db3a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.595+0000 7fd98d600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.595+0000 7fd98d600640 1 -- 172.21.15.18:0/3846661308 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99001f870 con 0x559b7a7d7610 2024-04-16T17:17:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.595+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/3846661308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9940a0ee0 con 0x559b7a7d7610 2024-04-16T17:17:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.595+0000 7fd98d600640 1 -- 172.21.15.18:0/3846661308 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fd99001fa10 con 0x559b7a7d7610 2024-04-16T17:17:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd98d600640 1 -- 172.21.15.18:0/3846661308 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd990011db0 con 0x559b7a7d7610 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd9ad3e7740 10 client.29102 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/3846661308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b7a856ba0 con 0x559b7a7d7610 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd98d600640 1 -- 172.21.15.18:0/3846661308 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9940a0ee0 con 0x559b7a7d7610 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd98d600640 1 -- 172.21.15.18:0/3846661308 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd990034430 con 0x559b7a7d7610 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd9ad3e7740 10 client.29102 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd9ad3e7740 10 client.29102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd9ad3e7740 20 client.29102 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b7a857260 0x559b7a859650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd9ad3e7740 4 client.29102 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.596+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/3846661308 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b7a743010 con 0x559b7a857260 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.597+0000 7fd9a3400640 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b7a857260 0x559b7a859650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.597+0000 7fd9a3400640 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b7a857260 0x559b7a859650 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b141ac255815b65a server_cookie=c56c039a57143382 in_seq=0 out_seq=0 2024-04-16T17:17:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.597+0000 7fd98d600640 10 client.29102 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:19.681 INFO:tasks.workunit.client.0.smithi018.stdout: 1 469278 3.58 MB/sec execute 572 sec latency 45.737 ms 2024-04-16T17:17:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.769+0000 7fd98d600640 1 -- 172.21.15.18:0/3846661308 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b7a743010 con 0x559b7a857260 2024-04-16T17:17:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.769+0000 7fd98d600640 10 client.29102 handle_command_reply: tid=0 2024-04-16T17:17:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.769+0000 7fd9ad3e7740 1 client.29102 shutdown 2024-04-16T17:17:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/3846661308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 msgr2=0x559b7a7e40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7e40d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd990016040 tx=0x7fd990027bb0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/3846661308 shutdown_connections 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559b7a857260 0x559b7a859650 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b7a7dfe20 0x559b7a7e4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b7a7d7610 0x559b7a7e40d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 --2- 172.21.15.18:0/3846661308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b7a7d68a0 0x559b7a7e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/3846661308 >> 172.21.15.18:0/3846661308 conn(0x559b7a7715d0 msgr2=0x559b7a7cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd98d600640 0 client.29102 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/3846661308 shutdown_connections 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 1 -- 172.21.15.18:0/3846661308 wait complete. 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.770+0000 7fd9ad3e7740 20 client.29102 trim_cache size 0 max 16384 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:17:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 -- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9880a3c20 msgr2=0x7fd9880a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9880a3c20 0x7fd9880a60e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x559b7a7d2720 tx=0x7fd990005f90 comp rx=0 tx=0).stop 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 -- 172.21.15.18:0/3558412212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a4104470 msgr2=0x7fd9a4110f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a4104470 0x7fd9a4110f90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd99400e9c0 tx=0x7fd99400ee80 comp rx=0 tx=0).stop 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 -- 172.21.15.18:0/3558412212 shutdown_connections 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd9880a3c20 0x7fd9880a60e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a410e610 0x7fd9a410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9a4109540 0x7fd9a410e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 --2- 172.21.15.18:0/3558412212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a4104470 0x7fd9a4110f90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.771+0000 7fd9aa600640 1 -- 172.21.15.18:0/3558412212 >> 172.21.15.18:0/3558412212 conn(0x7fd9a40ffcf0 msgr2=0x7fd9a4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.772+0000 7fd9aa600640 1 -- 172.21.15.18:0/3558412212 shutdown_connections 2024-04-16T17:17:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:19.772+0000 7fd9aa600640 1 -- 172.21.15.18:0/3558412212 wait complete. 2024-04-16T17:17:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:19 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:17:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:19 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:19 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:17:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:19 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:19 smithi022 ceph-mon[36023]: pgmap v399: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:17:20.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:19 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:17:20.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:19 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:17:20.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:19 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:20.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:19 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:17:20.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:19 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:20.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:19 smithi043 ceph-mon[36901]: pgmap v399: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:17:20.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:19 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:17:20.553 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:19 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:17:20.681 INFO:tasks.workunit.client.0.smithi018.stdout: 1 469916 3.57 MB/sec execute 573 sec latency 107.631 ms 2024-04-16T17:17:21.593 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:19 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:21.593 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:19 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:17:21.593 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:19 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:21.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:19 smithi018 ceph-mon[31776]: pgmap v399: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-16T17:17:21.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:19 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:17:21.681 INFO:tasks.workunit.client.0.smithi018.stdout: 1 470613 3.58 MB/sec execute 574 sec latency 49.410 ms 2024-04-16T17:17:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:21 smithi022 ceph-mon[36023]: pgmap v400: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1015 KiB/s wr, 137 op/s 2024-04-16T17:17:22.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:21 smithi043 ceph-mon[36901]: pgmap v400: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1015 KiB/s wr, 137 op/s 2024-04-16T17:17:22.634 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:21 smithi018 ceph-mon[31776]: pgmap v400: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1015 KiB/s wr, 137 op/s 2024-04-16T17:17:22.681 INFO:tasks.workunit.client.0.smithi018.stdout: 1 470911 3.57 MB/sec execute 575 sec latency 111.683 ms 2024-04-16T17:17:23.681 INFO:tasks.workunit.client.0.smithi018.stdout: 1 471256 3.57 MB/sec execute 576 sec latency 71.982 ms 2024-04-16T17:17:24.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:23 smithi022 ceph-mon[36023]: pgmap v401: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-04-16T17:17:24.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:23 smithi043 ceph-mon[36901]: pgmap v401: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-04-16T17:17:24.681 INFO:tasks.workunit.client.0.smithi018.stdout: 1 472275 3.57 MB/sec execute 577 sec latency 47.387 ms 2024-04-16T17:17:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:24 smithi043 ceph-mon[36901]: pgmap v402: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-04-16T17:17:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:24 smithi022 ceph-mon[36023]: pgmap v402: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-04-16T17:17:25.681 INFO:tasks.workunit.client.0.smithi018.stdout: 1 472978 3.58 MB/sec execute 578 sec latency 14.032 ms 2024-04-16T17:17:25.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:23 smithi018 ceph-mon[31776]: pgmap v401: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-04-16T17:17:25.829 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:24 smithi018 ceph-mon[31776]: pgmap v402: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-04-16T17:17:26.686 INFO:tasks.workunit.client.0.smithi018.stdout: 1 473630 3.57 MB/sec execute 579 sec latency 52.302 ms 2024-04-16T17:17:27.686 INFO:tasks.workunit.client.0.smithi018.stdout: 1 473972 3.57 MB/sec execute 580 sec latency 81.738 ms 2024-04-16T17:17:27.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:27 smithi022 ceph-mon[36023]: pgmap v403: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-16T17:17:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:27 smithi043 ceph-mon[36901]: pgmap v403: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-16T17:17:28.687 INFO:tasks.workunit.client.0.smithi018.stdout: 1 474211 3.57 MB/sec execute 581 sec latency 66.095 ms 2024-04-16T17:17:29.686 INFO:tasks.workunit.client.0.smithi018.stdout: 1 475017 3.56 MB/sec execute 582 sec latency 43.019 ms 2024-04-16T17:17:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:29 smithi022 ceph-mon[36023]: pgmap v404: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:17:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:29 smithi043 ceph-mon[36901]: pgmap v404: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:17:30.686 INFO:tasks.workunit.client.0.smithi018.stdout: 1 476263 3.58 MB/sec execute 583 sec latency 13.173 ms 2024-04-16T17:17:31.686 INFO:tasks.workunit.client.0.smithi018.stdout: 1 476619 3.57 MB/sec execute 584 sec latency 89.478 ms 2024-04-16T17:17:31.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:31 smithi022 ceph-mon[36023]: pgmap v405: 129 pgs: 129 active+clean; 40 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-16T17:17:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:31 smithi043 ceph-mon[36901]: pgmap v405: 129 pgs: 129 active+clean; 40 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-16T17:17:32.686 INFO:tasks.workunit.client.0.smithi018.stdout: 1 477001 3.57 MB/sec execute 585 sec latency 69.941 ms 2024-04-16T17:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 -- 172.21.15.18:0/847199704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f00743d0 msgr2=0x7fb2f00747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/847199704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f00743d0 0x7fb2f00747b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009860 tx=0x7fb2e402bc10 comp rx=0 tx=0).stop 2024-04-16T17:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 -- 172.21.15.18:0/847199704 shutdown_connections 2024-04-16T17:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/847199704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2f010e6b0 0x7fb2f0110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/847199704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2f0074cf0 0x7fb2f010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/847199704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f00743d0 0x7fb2f00747b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 -- 172.21.15.18:0/847199704 >> 172.21.15.18:0/847199704 conn(0x7fb2f00fd940 msgr2=0x7fb2f00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 -- 172.21.15.18:0/847199704 shutdown_connections 2024-04-16T17:17:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.907+0000 7fb2f6e00640 1 -- 172.21.15.18:0/847199704 wait complete. 2024-04-16T17:17:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.908+0000 7fb2f6e00640 1 Processor -- start 2024-04-16T17:17:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.908+0000 7fb2f6e00640 1 -- start start 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f6e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f00743d0 0x7fb2f01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f6e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2f0074cf0 0x7fb2f01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f6e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2f010e6b0 0x7fb2f019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f6e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0114420 con 0x7fb2f0074cf0 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f6e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01142a0 con 0x7fb2f010e6b0 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f6e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01145a0 con 0x7fb2f00743d0 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2f0074cf0 0x7fb2f01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2f0074cf0 0x7fb2f01a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46646/0 (socket says 172.21.15.18:46646) 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f5000640 1 -- 172.21.15.18:0/1703483119 learned_addr learned my addr 172.21.15.18:0/1703483119 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.909+0000 7fb2f6400640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2f010e6b0 0x7fb2f019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.910+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f00743d0 0x7fb2f01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.910+0000 7fb2f5000640 1 -- 172.21.15.18:0/1703483119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f00743d0 msgr2=0x7fb2f01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.910+0000 7fb2f5000640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f00743d0 0x7fb2f01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.910+0000 7fb2f5000640 1 -- 172.21.15.18:0/1703483119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2f010e6b0 msgr2=0x7fb2f019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.910+0000 7fb2f5000640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2f010e6b0 0x7fb2f019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.910+0000 7fb2f5000640 1 -- 172.21.15.18:0/1703483119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f019afb0 con 0x7fb2f0074cf0 2024-04-16T17:17:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.910+0000 7fb2f6400640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2f010e6b0 0x7fb2f019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.910+0000 7fb2f5000640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2f0074cf0 0x7fb2f01a0b60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00e9c0 tx=0x7fb2dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.911+0000 7fb2ee000640 1 -- 172.21.15.18:0/1703483119 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00ccc0 con 0x7fb2f0074cf0 2024-04-16T17:17:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.911+0000 7fb2f6e00640 1 -- 172.21.15.18:0/1703483119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f019b2a0 con 0x7fb2f0074cf0 2024-04-16T17:17:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.911+0000 7fb2f6e00640 1 -- 172.21.15.18:0/1703483119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a7470 con 0x7fb2f0074cf0 2024-04-16T17:17:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.911+0000 7fb2ee000640 1 -- 172.21.15.18:0/1703483119 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2dc005500 con 0x7fb2f0074cf0 2024-04-16T17:17:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.911+0000 7fb2ee000640 1 -- 172.21.15.18:0/1703483119 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc010430 con 0x7fb2f0074cf0 2024-04-16T17:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.915+0000 7fb2fb17c740 1 Processor -- start 2024-04-16T17:17:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.915+0000 7fb2fb17c740 1 -- start start 2024-04-16T17:17:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.915+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68906d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c68906df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c6890793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c688fd2780 con 0x55c68906da80 2024-04-16T17:17:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c688f78f40 con 0x55c68906e7f0 2024-04-16T17:17:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c688fce740 con 0x55c689077000 2024-04-16T17:17:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2f6400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68906d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2f6400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68906d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33722/0 (socket says 172.21.15.18:33722) 2024-04-16T17:17:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2f6400640 1 -- 172.21.15.18:0/3500979548 learned_addr learned my addr 172.21.15.18:0/3500979548 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2ee000640 1 -- 172.21.15.18:0/1703483119 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2dc0046c0 con 0x7fb2f0074cf0 2024-04-16T17:17:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.916+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c68906df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.917+0000 7fb2f5000640 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c6890793f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.917+0000 7fb2f5a00640 1 -- 172.21.15.18:0/3500979548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 msgr2=0x55c6890793f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.917+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c6890793f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.917+0000 7fb2f5a00640 1 -- 172.21.15.18:0/3500979548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 msgr2=0x55c68906d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.917+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68906d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.917+0000 7fb2f5a00640 1 -- 172.21.15.18:0/3500979548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c688ce5210 con 0x55c68906da80 2024-04-16T17:17:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.917+0000 7fb2f5000640 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c6890793f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.917+0000 7fb2f6400640 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68906d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.918+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c68906df00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009830 tx=0x7fb2e4035b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef3a42c9f71e401 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.918+0000 7fb2e2a00640 1 -- 172.21.15.18:0/3500979548 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4039040 con 0x55c68906da80 2024-04-16T17:17:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.918+0000 7fb2e2a00640 1 -- 172.21.15.18:0/3500979548 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e4034440 con 0x55c68906da80 2024-04-16T17:17:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.918+0000 7fb2e2a00640 1 -- 172.21.15.18:0/3500979548 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4005460 con 0x55c68906da80 2024-04-16T17:17:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.919+0000 7fb2ee000640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2d80a3d80 0x7fb2d80a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.919+0000 7fb2ee000640 1 -- 172.21.15.18:0/1703483119 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fb2dc0d0df0 con 0x7fb2f0074cf0 2024-04-16T17:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.919+0000 7fb2f6400640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2d80a3d80 0x7fb2d80a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.919+0000 7fb2fb17c740 1 -- 172.21.15.18:0/3500979548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 msgr2=0x55c68906df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.919+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c68906df00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009830 tx=0x7fb2e4035b60 comp rx=0 tx=0).stop 2024-04-16T17:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.920+0000 7fb2fb17c740 1 -- 172.21.15.18:0/3500979548 shutdown_connections 2024-04-16T17:17:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.920+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c6890793f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.920+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c68906df00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.920+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/3500979548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68906d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.920+0000 7fb2fb17c740 1 -- 172.21.15.18:0/3500979548 >> 172.21.15.18:0/3500979548 conn(0x55c6890085d0 msgr2=0x55c689068770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.920+0000 7fb2f6400640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2d80a3d80 0x7fb2d80a6240 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009fd0 tx=0x7fb2e8009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.921+0000 7fb2fb17c740 1 -- 172.21.15.18:0/3500979548 shutdown_connections 2024-04-16T17:17:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.921+0000 7fb2fb17c740 1 -- 172.21.15.18:0/3500979548 wait complete. 2024-04-16T17:17:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.922+0000 7fb2fb17c740 1 -- start start 2024-04-16T17:17:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.922+0000 7fb2fb17c740 1 Processor -- start 2024-04-16T17:17:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689076a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689073de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c689074520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c689079cd0 con 0x55c68906e7f0 2024-04-16T17:17:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c688f8c390 con 0x55c68906da80 2024-04-16T17:17:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c689079e50 con 0x55c689077000 2024-04-16T17:17:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2fb17c740 10 client.0 _ll_register_callbacks cb 0x55c688f75f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689076a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33730/0 (socket says 172.21.15.18:33730) 2024-04-16T17:17:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2f5a00640 1 -- 172.21.15.18:0/2882654293 learned_addr learned my addr 172.21.15.18:0/2882654293 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2f6400640 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c689074520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.923+0000 7fb2f5000640 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2f5a00640 1 -- 172.21.15.18:0/2882654293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 msgr2=0x55c689074520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c689074520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2f5a00640 1 -- 172.21.15.18:0/2882654293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 msgr2=0x55c689073de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689073de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2f5a00640 1 -- 172.21.15.18:0/2882654293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c688db1620 con 0x55c68906da80 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689076a50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb2e402c150 tx=0x7fb2e4034a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f03273c33fec575 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2d7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2d7400640 1 -- 172.21.15.18:0/2882654293 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e404c070 con 0x55c68906da80 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2d7400640 1 -- 172.21.15.18:0/2882654293 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fb2e40055f0 con 0x55c68906da80 2024-04-16T17:17:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.924+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2882654293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e4034440 con 0x55c68906da80 2024-04-16T17:17:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2d7400640 1 -- 172.21.15.18:0/2882654293 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb2e4058050 con 0x55c68906da80 2024-04-16T17:17:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2fb17c740 10 client.29114 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2882654293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c689085d60 con 0x55c68906da80 2024-04-16T17:17:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2d7400640 1 -- 172.21.15.18:0/2882654293 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e4034440 con 0x55c68906da80 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2d7400640 1 -- 172.21.15.18:0/2882654293 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb2e40046c0 con 0x55c68906da80 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2fb17c740 10 client.29114 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2fb17c740 10 client.29114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2fb17c740 20 client.29114 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c6890870b0 0x55c6890894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2fb17c740 4 client.29114 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.925+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2882654293 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c688fe01a0 con 0x55c6890870b0 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.926+0000 7fb2f5000640 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c6890870b0 0x55c6890894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.926+0000 7fb2d7400640 10 client.29114 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.926+0000 7fb2f5000640 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c6890870b0 0x55c6890894a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8f02700f97bbf9 server_cookie=b6d3337eec69a86f in_seq=0 out_seq=0 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2d7400640 1 -- 172.21.15.18:0/2882654293 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c688fe01a0 con 0x55c6890870b0 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2d7400640 10 client.29114 handle_command_reply: tid=0 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2fb17c740 1 client.29114 shutdown 2024-04-16T17:17:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2882654293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 msgr2=0x55c689076a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689076a50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb2e402c150 tx=0x7fb2e4034a10 comp rx=0 tx=0).stop 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2882654293 shutdown_connections 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c6890870b0 0x55c6890894a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c689074520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.928+0000 7fb2d7400640 0 client.29114 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.927+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689073de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.928+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2882654293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689076a50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.928+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2882654293 >> 172.21.15.18:0/2882654293 conn(0x55c6890085d0 msgr2=0x55c68906a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.928+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2882654293 shutdown_connections 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.928+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2882654293 wait complete. 2024-04-16T17:17:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.928+0000 7fb2fb17c740 20 client.29114 trim_cache size 0 max 16384 2024-04-16T17:17:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.944+0000 7fb2fb17c740 1 Processor -- start 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2fb17c740 1 -- start start 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c689082bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68908ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c68908f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0012950 con 0x55c68906da80 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0010220 con 0x55c68906e7f0 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d000e090 con 0x55c689077000 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68908ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68908ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33742/0 (socket says 172.21.15.18:33742) 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c689082bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5000640 1 -- 172.21.15.18:0/2803601723 learned_addr learned my addr 172.21.15.18:0/2803601723 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f6400640 1 --2- 172.21.15.18:0/2803601723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c68908f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5000640 1 -- 172.21.15.18:0/2803601723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 msgr2=0x55c68908f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5000640 1 --2- 172.21.15.18:0/2803601723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c68908f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5000640 1 -- 172.21.15.18:0/2803601723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 msgr2=0x55c689082bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5000640 1 --2- 172.21.15.18:0/2803601723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c689082bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.945+0000 7fb2f5000640 1 -- 172.21.15.18:0/2803601723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c689079e30 con 0x55c68906e7f0 2024-04-16T17:17:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.946+0000 7fb2f5000640 1 --2- 172.21.15.18:0/2803601723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68908ead0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00cf80 tx=0x7fb2dc0bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee194a0d4437d51f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.946+0000 7fb2e2000640 1 -- 172.21.15.18:0/2803601723 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc09c4c0 con 0x55c68906e7f0 2024-04-16T17:17:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.946+0000 7fb2e2000640 1 -- 172.21.15.18:0/2803601723 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2dc09d080 con 0x55c68906e7f0 2024-04-16T17:17:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.946+0000 7fb2e2000640 1 -- 172.21.15.18:0/2803601723 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0056b0 con 0x55c68906e7f0 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2803601723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 msgr2=0x55c68908ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2803601723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68908ead0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00cf80 tx=0x7fb2dc0bd040 comp rx=0 tx=0).stop 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2803601723 shutdown_connections 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2803601723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c68908f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2803601723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906e7f0 0x55c68908ead0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/2803601723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906da80 0x55c689082bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2803601723 >> 172.21.15.18:0/2803601723 conn(0x55c6890085d0 msgr2=0x55c6890682a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2803601723 shutdown_connections 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 -- 172.21.15.18:0/2803601723 wait complete. 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.947+0000 7fb2fb17c740 1 -- start start 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.948+0000 7fb2fb17c740 1 Processor -- start 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.948+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689075e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.948+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689076350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.948+0000 7fb2fb17c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c68907d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.948+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d000e6e0 con 0x55c68906e7f0 2024-04-16T17:17:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.948+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d000fb50 con 0x55c68906da80 2024-04-16T17:17:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.948+0000 7fb2fb17c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d000edb0 con 0x55c689077000 2024-04-16T17:17:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.948+0000 7fb2fb17c740 10 client.0 _ll_register_callbacks cb 0x7fb2e8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689076350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689075e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33746/0 (socket says 172.21.15.18:33746) 2024-04-16T17:17:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 -- 172.21.15.18:0/315038926 learned_addr learned my addr 172.21.15.18:0/315038926 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f6400640 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c68907d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 -- 172.21.15.18:0/315038926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 msgr2=0x55c68907d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c68907d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 -- 172.21.15.18:0/315038926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 msgr2=0x55c689076350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689076350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 -- 172.21.15.18:0/315038926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c68907dbb0 con 0x55c68906da80 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5000640 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689076350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2f5a00640 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689075e10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4000c00 tx=0x7fb2e40048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed6fb007c9607e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2d7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.949+0000 7fb2d7400640 1 -- 172.21.15.18:0/315038926 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e404c040 con 0x55c68906da80 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 1 -- 172.21.15.18:0/315038926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2dc09d080 con 0x55c68906da80 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2d7400640 1 -- 172.21.15.18:0/315038926 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fb2e4034d50 con 0x55c68906da80 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2d7400640 1 -- 172.21.15.18:0/315038926 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb2e4039040 con 0x55c68906da80 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 10 client.29120 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 1 -- 172.21.15.18:0/315038926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c6890eda90 con 0x55c68906da80 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2d7400640 1 -- 172.21.15.18:0/315038926 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2dc09d080 con 0x55c68906da80 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2d7400640 1 -- 172.21.15.18:0/315038926 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb2e4004040 con 0x55c68906da80 2024-04-16T17:17:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 10 client.29120 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 10 client.29120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 20 client.29120 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c6890ee010 0x55c6890f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 4 client.29120 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:17:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.950+0000 7fb2fb17c740 1 -- 172.21.15.18:0/315038926 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c688fda010 con 0x55c6890ee010 2024-04-16T17:17:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.951+0000 7fb2f5000640 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c6890ee010 0x55c6890f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.951+0000 7fb2f5000640 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c6890ee010 0x55c6890f0400 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4366e7cca45df259 server_cookie=5c6c6d80b3620fb1 in_seq=0 out_seq=0 2024-04-16T17:17:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:32.951+0000 7fb2d7400640 10 client.29120 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.107+0000 7fb2d7400640 1 -- 172.21.15.18:0/315038926 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c688fda010 con 0x55c6890ee010 2024-04-16T17:17:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.107+0000 7fb2d7400640 10 client.29120 handle_command_reply: tid=0 2024-04-16T17:17:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.107+0000 7fb2fb17c740 1 client.29120 shutdown 2024-04-16T17:17:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 -- 172.21.15.18:0/315038926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 msgr2=0x55c689075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689075e10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4000c00 tx=0x7fb2e40048f0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 -- 172.21.15.18:0/315038926 shutdown_connections 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c6890ee010 0x55c6890f0400 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c689077000 0x55c68907d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c68906e7f0 0x55c689076350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 --2- 172.21.15.18:0/315038926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c68906da80 0x55c689075e10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 -- 172.21.15.18:0/315038926 >> 172.21.15.18:0/315038926 conn(0x55c6890085d0 msgr2=0x55c689073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2d7400640 0 client.29120 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 -- 172.21.15.18:0/315038926 shutdown_connections 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 1 -- 172.21.15.18:0/315038926 wait complete. 2024-04-16T17:17:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.109+0000 7fb2fb17c740 20 client.29120 trim_cache size 0 max 16384 2024-04-16T17:17:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:17:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:17:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:17:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.110+0000 7fb2f6e00640 1 -- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2d80a3d80 msgr2=0x7fb2d80a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.110+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2d80a3d80 0x7fb2d80a6240 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009fd0 tx=0x7fb2e8009500 comp rx=0 tx=0).stop 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.110+0000 7fb2f6e00640 1 -- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2f0074cf0 msgr2=0x7fb2f01a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.110+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2f0074cf0 0x7fb2f01a0b60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00e9c0 tx=0x7fb2dc00ee80 comp rx=0 tx=0).stop 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.111+0000 7fb2f6e00640 1 -- 172.21.15.18:0/1703483119 shutdown_connections 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.111+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2d80a3d80 0x7fb2d80a6240 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.111+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2f010e6b0 0x7fb2f019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.111+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2f0074cf0 0x7fb2f01a0b60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.111+0000 7fb2f6e00640 1 --2- 172.21.15.18:0/1703483119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f00743d0 0x7fb2f01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.111+0000 7fb2f6e00640 1 -- 172.21.15.18:0/1703483119 >> 172.21.15.18:0/1703483119 conn(0x7fb2f00fd940 msgr2=0x7fb2f00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.111+0000 7fb2f6e00640 1 -- 172.21.15.18:0/1703483119 shutdown_connections 2024-04-16T17:17:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:33.111+0000 7fb2f6e00640 1 -- 172.21.15.18:0/1703483119 wait complete. 2024-04-16T17:17:33.686 INFO:tasks.workunit.client.0.smithi018.stdout: 1 477309 3.56 MB/sec execute 586 sec latency 55.503 ms 2024-04-16T17:17:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:33 smithi022 ceph-mon[36023]: pgmap v406: 129 pgs: 129 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-04-16T17:17:34.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:33 smithi043 ceph-mon[36901]: pgmap v406: 129 pgs: 129 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-04-16T17:17:34.686 INFO:tasks.workunit.client.0.smithi018.stdout: 1 478053 3.56 MB/sec execute 587 sec latency 33.310 ms 2024-04-16T17:17:35.687 INFO:tasks.workunit.client.0.smithi018.stdout: 1 479041 3.57 MB/sec execute 588 sec latency 26.550 ms 2024-04-16T17:17:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:35 smithi022 ceph-mon[36023]: pgmap v407: 129 pgs: 129 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 955 KiB/s wr, 72 op/s 2024-04-16T17:17:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:35 smithi043 ceph-mon[36901]: pgmap v407: 129 pgs: 129 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 955 KiB/s wr, 72 op/s 2024-04-16T17:17:36.687 INFO:tasks.workunit.client.0.smithi018.stdout: 1 479965 3.57 MB/sec execute 589 sec latency 75.683 ms 2024-04-16T17:17:37.687 INFO:tasks.workunit.client.0.smithi018.stdout: 1 480184 3.57 MB/sec execute 590 sec latency 45.807 ms 2024-04-16T17:17:37.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:37 smithi022 ceph-mon[36023]: pgmap v408: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-04-16T17:17:37.964 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:27 smithi018 ceph-mon[31776]: pgmap v403: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-16T17:17:37.964 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:29 smithi018 ceph-mon[31776]: pgmap v404: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-16T17:17:37.964 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:31 smithi018 ceph-mon[31776]: pgmap v405: 129 pgs: 129 active+clean; 40 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-16T17:17:37.964 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:33 smithi018 ceph-mon[31776]: pgmap v406: 129 pgs: 129 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-04-16T17:17:37.964 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:35 smithi018 ceph-mon[31776]: pgmap v407: 129 pgs: 129 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 955 KiB/s wr, 72 op/s 2024-04-16T17:17:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:37 smithi043 ceph-mon[36901]: pgmap v408: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-04-16T17:17:38.687 INFO:tasks.workunit.client.0.smithi018.stdout: 1 480423 3.56 MB/sec execute 591 sec latency 85.629 ms 2024-04-16T17:17:39.687 INFO:tasks.workunit.client.0.smithi018.stdout: 1 481213 3.56 MB/sec execute 592 sec latency 31.742 ms 2024-04-16T17:17:39.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:39 smithi022 ceph-mon[36023]: pgmap v409: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 2.4 MiB/s wr, 111 op/s 2024-04-16T17:17:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:39 smithi043 ceph-mon[36901]: pgmap v409: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 2.4 MiB/s wr, 111 op/s 2024-04-16T17:17:40.687 INFO:tasks.workunit.client.0.smithi018.stdout: 1 482053 3.56 MB/sec execute 593 sec latency 33.167 ms 2024-04-16T17:17:40.734 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:37 smithi018 ceph-mon[31776]: pgmap v408: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-04-16T17:17:40.734 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:39 smithi018 ceph-mon[31776]: pgmap v409: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 2.4 MiB/s wr, 111 op/s 2024-04-16T17:17:41.691 INFO:tasks.workunit.client.0.smithi018.stdout: 1 483041 3.57 MB/sec execute 594 sec latency 57.625 ms 2024-04-16T17:17:41.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:41 smithi022 ceph-mon[36023]: pgmap v410: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 137 op/s 2024-04-16T17:17:42.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:41 smithi043 ceph-mon[36901]: pgmap v410: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 137 op/s 2024-04-16T17:17:42.691 INFO:tasks.workunit.client.0.smithi018.stdout: 1 483374 3.57 MB/sec execute 595 sec latency 100.958 ms 2024-04-16T17:17:42.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:41 smithi018 ceph-mon[31776]: pgmap v410: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 137 op/s 2024-04-16T17:17:43.691 INFO:tasks.workunit.client.0.smithi018.stdout: 1 483665 3.56 MB/sec execute 596 sec latency 46.557 ms 2024-04-16T17:17:43.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:43 smithi022 ceph-mon[36023]: pgmap v411: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-04-16T17:17:44.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:43 smithi043 ceph-mon[36901]: pgmap v411: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-04-16T17:17:44.691 INFO:tasks.workunit.client.0.smithi018.stdout: 1 484197 3.56 MB/sec execute 597 sec latency 53.070 ms 2024-04-16T17:17:45.692 INFO:tasks.workunit.client.0.smithi018.stdout: 1 484914 3.56 MB/sec execute 598 sec latency 31.056 ms 2024-04-16T17:17:45.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:45 smithi022 ceph-mon[36023]: pgmap v412: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 2.3 MiB/s wr, 108 op/s 2024-04-16T17:17:46.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:45 smithi043 ceph-mon[36901]: pgmap v412: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 2.3 MiB/s wr, 108 op/s 2024-04-16T17:17:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 -- 172.21.15.18:0/4234202695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b00743a0 msgr2=0x7fa1b0074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 --2- 172.21.15.18:0/4234202695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b00743a0 0x7fa1b0074780 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009860 tx=0x7fa1a002bc20 comp rx=0 tx=0).stop 2024-04-16T17:17:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 -- 172.21.15.18:0/4234202695 shutdown_connections 2024-04-16T17:17:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 --2- 172.21.15.18:0/4234202695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1b0112e30 0x7fa1b0115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 --2- 172.21.15.18:0/4234202695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b0074cc0 0x7fa1b01128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 --2- 172.21.15.18:0/4234202695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b00743a0 0x7fa1b0074780 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 -- 172.21.15.18:0/4234202695 >> 172.21.15.18:0/4234202695 conn(0x7fa1b00fd910 msgr2=0x7fa1b00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 -- 172.21.15.18:0/4234202695 shutdown_connections 2024-04-16T17:17:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.249+0000 7fa1afe00640 1 -- 172.21.15.18:0/4234202695 wait complete. 2024-04-16T17:17:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.250+0000 7fa1afe00640 1 Processor -- start 2024-04-16T17:17:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.251+0000 7fa1afe00640 1 -- start start 2024-04-16T17:17:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.251+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b00743a0 0x7fa1b01a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.251+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1b0074cc0 0x7fa1b01a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.251+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b0112e30 0x7fa1b019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.251+0000 7fa1afe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0118800 con 0x7fa1b0074cc0 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.251+0000 7fa1afe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0118680 con 0x7fa1b0112e30 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.251+0000 7fa1afe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0118980 con 0x7fa1b00743a0 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b00743a0 0x7fa1b01a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b0112e30 0x7fa1b019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b00743a0 0x7fa1b01a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59790/0 (socket says 172.21.15.18:59790) 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 -- 172.21.15.18:0/3431404271 learned_addr learned my addr 172.21.15.18:0/3431404271 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b0112e30 0x7fa1b019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36028/0 (socket says 172.21.15.18:36028) 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1ae000640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1b0074cc0 0x7fa1b01a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 -- 172.21.15.18:0/3431404271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b0112e30 msgr2=0x7fa1b019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b0112e30 0x7fa1b019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 -- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1b0074cc0 msgr2=0x7fa1b01a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1b0074cc0 0x7fa1b01a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 -- 172.21.15.18:0/3431404271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b019f420 con 0x7fa1b00743a0 2024-04-16T17:17:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.252+0000 7fa1aea00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b00743a0 0x7fa1b01a4a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0037040 tx=0x7fa1a00346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.253+0000 7fa1a7400640 1 -- 172.21.15.18:0/3431404271 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0041020 con 0x7fa1b00743a0 2024-04-16T17:17:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.253+0000 7fa1a7400640 1 -- 172.21.15.18:0/3431404271 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a002ce50 con 0x7fa1b00743a0 2024-04-16T17:17:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.253+0000 7fa1a7400640 1 -- 172.21.15.18:0/3431404271 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0039040 con 0x7fa1b00743a0 2024-04-16T17:17:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.253+0000 7fa1afe00640 1 -- 172.21.15.18:0/3431404271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b019f6b0 con 0x7fa1b00743a0 2024-04-16T17:17:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.253+0000 7fa1afe00640 1 -- 172.21.15.18:0/3431404271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b01ab7f0 con 0x7fa1b00743a0 2024-04-16T17:17:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.257+0000 7fa1a7400640 1 -- 172.21.15.18:0/3431404271 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa1a0044020 con 0x7fa1b00743a0 2024-04-16T17:17:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.257+0000 7fa1b7ffd740 1 Processor -- start 2024-04-16T17:17:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.257+0000 7fa1b7ffd740 1 -- start start 2024-04-16T17:17:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b23540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b23f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 0x562b99b2f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562b99a88780 con 0x562b99b247f0 2024-04-16T17:17:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562b99a2ef40 con 0x562b99b23a80 2024-04-16T17:17:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562b99a84740 con 0x562b99b2d000 2024-04-16T17:17:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1af400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b23540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b23f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b23f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36042/0 (socket says 172.21.15.18:36042) 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1aea00640 1 -- 172.21.15.18:0/3791209250 learned_addr learned my addr 172.21.15.18:0/3791209250 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.258+0000 7fa1ae000640 1 --2- 172.21.15.18:0/3791209250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 0x562b99b2f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.259+0000 7fa1af400640 1 -- 172.21.15.18:0/3791209250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 msgr2=0x562b99b2f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.259+0000 7fa1af400640 1 --2- 172.21.15.18:0/3791209250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 0x562b99b2f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.259+0000 7fa1af400640 1 -- 172.21.15.18:0/3791209250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 msgr2=0x562b99b23f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.259+0000 7fa1af400640 1 --2- 172.21.15.18:0/3791209250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b23f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.259+0000 7fa1af400640 1 -- 172.21.15.18:0/3791209250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9979b210 con 0x562b99b247f0 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.259+0000 7fa1a7400640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1940a3d00 0x7fa1940a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.259+0000 7fa1a7400640 1 -- 172.21.15.18:0/3431404271 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fa1a00f21f0 con 0x7fa1b00743a0 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.259+0000 7fa1aea00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1940a3d00 0x7fa1940a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.260+0000 7fa1af400640 1 --2- 172.21.15.18:0/3791209250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b23540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005fd0 tx=0x7fa1a8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9cf5fcd7415a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.260+0000 7fa19b400640 1 -- 172.21.15.18:0/3791209250 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8016070 con 0x562b99b247f0 2024-04-16T17:17:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.260+0000 7fa19b400640 1 -- 172.21.15.18:0/3791209250 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8004d20 con 0x562b99b247f0 2024-04-16T17:17:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.260+0000 7fa19b400640 1 -- 172.21.15.18:0/3791209250 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8012810 con 0x562b99b247f0 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.261+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/3791209250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 msgr2=0x562b99b23540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.261+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/3791209250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b23540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005fd0 tx=0x7fa1a8010740 comp rx=0 tx=0).stop 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.261+0000 7fa1aea00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1940a3d00 0x7fa1940a61c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00bdb30 tx=0x7fa1a0005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.261+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/3791209250 shutdown_connections 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.261+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/3791209250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 0x562b99b2f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.261+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/3791209250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b23f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.261+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/3791209250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b23540 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.261+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/3791209250 >> 172.21.15.18:0/3791209250 conn(0x562b99abe5d0 msgr2=0x562b99b1e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.262+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/3791209250 shutdown_connections 2024-04-16T17:17:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.262+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/3791209250 wait complete. 2024-04-16T17:17:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.262+0000 7fa1b7ffd740 1 -- start start 2024-04-16T17:17:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1b7ffd740 1 Processor -- start 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b2cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b29e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 0x562b99b2a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562b99b2fd30 con 0x562b99b247f0 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562b99a42390 con 0x562b99b23a80 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562b99b2feb0 con 0x562b99b2d000 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1b7ffd740 10 client.0 _ll_register_callbacks cb 0x562b99a2bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.263+0000 7fa1af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b2cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b2cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36056/0 (socket says 172.21.15.18:36056) 2024-04-16T17:17:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1af400640 1 -- 172.21.15.18:0/1263202086 learned_addr learned my addr 172.21.15.18:0/1263202086 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1af400640 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 0x562b99b2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1ae000640 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1af400640 1 -- 172.21.15.18:0/1263202086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 msgr2=0x562b99b2a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1af400640 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 0x562b99b2a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1af400640 1 -- 172.21.15.18:0/1263202086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 msgr2=0x562b99b29e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1af400640 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b29e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1af400640 1 -- 172.21.15.18:0/1263202086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b99867620 con 0x562b99b23a80 2024-04-16T17:17:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.264+0000 7fa1ae000640 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b29e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.265+0000 7fa1af400640 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b2cab0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009c70 tx=0x7fa1a8012cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996affa019bf2d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.265+0000 7fa18fe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.265+0000 7fa18fe00640 1 -- 172.21.15.18:0/1263202086 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8016070 con 0x562b99b23a80 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.265+0000 7fa18fe00640 1 -- 172.21.15.18:0/1263202086 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fa1a8003b80 con 0x562b99b23a80 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.265+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1263202086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a8004d20 con 0x562b99b23a80 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.265+0000 7fa18fe00640 1 -- 172.21.15.18:0/1263202086 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa1a802f050 con 0x562b99b23a80 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.265+0000 7fa1b7ffd740 10 client.29132 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.265+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1263202086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562b99b3be50 con 0x562b99b23a80 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.266+0000 7fa18fe00640 1 -- 172.21.15.18:0/1263202086 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8004d20 con 0x562b99b23a80 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.266+0000 7fa18fe00640 1 -- 172.21.15.18:0/1263202086 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa1a8028040 con 0x562b99b23a80 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.266+0000 7fa1b7ffd740 10 client.29132 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.266+0000 7fa1b7ffd740 10 client.29132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.266+0000 7fa1b7ffd740 20 client.29132 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.266+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b99b3d260 0x562b99b3f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.266+0000 7fa1b7ffd740 4 client.29132 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.266+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1263202086 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b99a961a0 con 0x562b99b3d260 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.267+0000 7fa1ae000640 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b99b3d260 0x562b99b3f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.267+0000 7fa1ae000640 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b99b3d260 0x562b99b3f650 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ad106de7727784 server_cookie=9ff7c9fb89fec67a in_seq=0 out_seq=0 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.267+0000 7fa18fe00640 10 client.29132 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa18fe00640 1 -- 172.21.15.18:0/1263202086 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b99a961a0 con 0x562b99b3d260 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa18fe00640 10 client.29132 handle_command_reply: tid=0 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 client.29132 shutdown 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1263202086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 msgr2=0x562b99b2cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b2cab0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009c70 tx=0x7fa1a8012cd0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1263202086 shutdown_connections 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b99b3d260 0x562b99b3f650 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b2d000 0x562b99b2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b29e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1263202086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b23a80 0x562b99b2cab0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1263202086 >> 172.21.15.18:0/1263202086 conn(0x562b99abe5d0 msgr2=0x562b99b204c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa18fe00640 0 client.29132 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1263202086 shutdown_connections 2024-04-16T17:17:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.268+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1263202086 wait complete. 2024-04-16T17:17:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.269+0000 7fa1b7ffd740 20 client.29132 trim_cache size 0 max 16384 2024-04-16T17:17:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1b7ffd740 1 Processor -- start 2024-04-16T17:17:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1b7ffd740 1 -- start start 2024-04-16T17:17:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b23a80 0x562b99b38c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 0x562b99b2c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b83f0 con 0x562b99b247f0 2024-04-16T17:17:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b5cc0 con 0x562b99b2d000 2024-04-16T17:17:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b3b30 con 0x562b99b23a80 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b23a80 0x562b99b38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b23a80 0x562b99b38c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59806/0 (socket says 172.21.15.18:59806) 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1af400640 1 -- 172.21.15.18:0/1981380321 learned_addr learned my addr 172.21.15.18:0/1981380321 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.285+0000 7fa1ae000640 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.286+0000 7fa1af400640 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 0x562b99b2c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.286+0000 7fa1af400640 1 -- 172.21.15.18:0/1981380321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 msgr2=0x562b99b2c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.286+0000 7fa1af400640 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 0x562b99b2c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.286+0000 7fa1af400640 1 -- 172.21.15.18:0/1981380321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 msgr2=0x562b99b28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.286+0000 7fa1af400640 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b28d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.286+0000 7fa1af400640 1 -- 172.21.15.18:0/1981380321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b99b1ecb0 con 0x562b99b23a80 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.286+0000 7fa1ae000640 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.286+0000 7fa1af400640 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b23a80 0x562b99b38c60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8015040 tx=0x7fa1a802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db3e83525d6ed46 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.287+0000 7fa19aa00640 1 -- 172.21.15.18:0/1981380321 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a803a070 con 0x562b99b23a80 2024-04-16T17:17:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.287+0000 7fa19aa00640 1 -- 172.21.15.18:0/1981380321 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8012630 con 0x562b99b23a80 2024-04-16T17:17:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.287+0000 7fa19aa00640 1 -- 172.21.15.18:0/1981380321 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8016070 con 0x562b99b23a80 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1981380321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b23a80 msgr2=0x562b99b38c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b23a80 0x562b99b38c60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8015040 tx=0x7fa1a802a410 comp rx=0 tx=0).stop 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1981380321 shutdown_connections 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 0x562b99b2c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b247f0 0x562b99b28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/1981380321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b23a80 0x562b99b38c60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1981380321 >> 172.21.15.18:0/1981380321 conn(0x562b99abe5d0 msgr2=0x562b99b28600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1981380321 shutdown_connections 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/1981380321 wait complete. 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.288+0000 7fa1b7ffd740 1 -- start start 2024-04-16T17:17:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.289+0000 7fa1b7ffd740 1 Processor -- start 2024-04-16T17:17:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.289+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b23a80 0x562b99b35510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.289+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b247f0 0x562b99b267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.289+0000 7fa1b7ffd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 0x562b99b26cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.289+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b4180 con 0x562b99b23a80 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.289+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b55f0 con 0x562b99b2d000 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.289+0000 7fa1b7ffd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01b4850 con 0x562b99b247f0 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1b7ffd740 10 client.0 _ll_register_callbacks cb 0x7fa1a8009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1af400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b23a80 0x562b99b35510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1af400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b23a80 0x562b99b35510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34012/0 (socket says 172.21.15.18:34012) 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1af400640 1 -- 172.21.15.18:0/561843685 learned_addr learned my addr 172.21.15.18:0/561843685 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1ae000640 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b247f0 0x562b99b267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1af400640 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 0x562b99b26cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1ae000640 1 -- 172.21.15.18:0/561843685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 msgr2=0x562b99b26cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1ae000640 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 0x562b99b26cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1ae000640 1 -- 172.21.15.18:0/561843685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b23a80 msgr2=0x562b99b35510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1ae000640 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b23a80 0x562b99b35510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1ae000640 1 -- 172.21.15.18:0/561843685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b99b2a8f0 con 0x562b99b247f0 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.290+0000 7fa1af400640 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b23a80 0x562b99b35510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.291+0000 7fa1ae000640 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b247f0 0x562b99b267b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x562b99b296a0 tx=0x7fa19c004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=901e5f6cb39e7a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.291+0000 7fa18fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.291+0000 7fa18fe00640 1 -- 172.21.15.18:0/561843685 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c01d430 con 0x562b99b247f0 2024-04-16T17:17:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.291+0000 7fa18fe00640 1 -- 172.21.15.18:0/561843685 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fa19c01d5d0 con 0x562b99b247f0 2024-04-16T17:17:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.291+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/561843685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a8012630 con 0x562b99b247f0 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa18fe00640 1 -- 172.21.15.18:0/561843685 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa19c02d050 con 0x562b99b247f0 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa1b7ffd740 10 client.28978 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/561843685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562b99ba3290 con 0x562b99b247f0 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa18fe00640 1 -- 172.21.15.18:0/561843685 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8012630 con 0x562b99b247f0 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa18fe00640 1 -- 172.21.15.18:0/561843685 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa19c026410 con 0x562b99b247f0 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa1b7ffd740 10 client.28978 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa1b7ffd740 10 client.28978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa1b7ffd740 20 client.28978 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.292+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b99ba39e0 0x562b99ba5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.293+0000 7fa1b7ffd740 4 client.28978 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.293+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/561843685 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b99a90010 con 0x562b99ba39e0 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.293+0000 7fa1af400640 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b99ba39e0 0x562b99ba5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.293+0000 7fa1af400640 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b99ba39e0 0x562b99ba5e00 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f146fa8510cd7ec server_cookie=8345c756a72f34b3 in_seq=0 out_seq=0 2024-04-16T17:17:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.293+0000 7fa18fe00640 10 client.28978 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa18fe00640 1 -- 172.21.15.18:0/561843685 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b99a90010 con 0x562b99ba39e0 2024-04-16T17:17:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa18fe00640 10 client.28978 handle_command_reply: tid=0 2024-04-16T17:17:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 client.28978 shutdown 2024-04-16T17:17:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/561843685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b247f0 msgr2=0x562b99b267b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b247f0 0x562b99b267b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x562b99b296a0 tx=0x7fa19c004410 comp rx=0 tx=0).stop 2024-04-16T17:17:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/561843685 shutdown_connections 2024-04-16T17:17:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b99ba39e0 0x562b99ba5e00 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b99b2d000 0x562b99b26cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b99b247f0 0x562b99b267b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 --2- 172.21.15.18:0/561843685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b99b23a80 0x562b99b35510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.438+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/561843685 >> 172.21.15.18:0/561843685 conn(0x562b99abe5d0 msgr2=0x562b99b28e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.439+0000 7fa18fe00640 0 client.28978 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.439+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/561843685 shutdown_connections 2024-04-16T17:17:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.439+0000 7fa1b7ffd740 1 -- 172.21.15.18:0/561843685 wait complete. 2024-04-16T17:17:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.439+0000 7fa1b7ffd740 20 client.28978 trim_cache size 0 max 16384 2024-04-16T17:17:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:17:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.440+0000 7fa1afe00640 1 -- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1940a3d00 msgr2=0x7fa1940a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.440+0000 7fa1afe00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1940a3d00 0x7fa1940a61c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00bdb30 tx=0x7fa1a0005f90 comp rx=0 tx=0).stop 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.440+0000 7fa1afe00640 1 -- 172.21.15.18:0/3431404271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b00743a0 msgr2=0x7fa1b01a4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.440+0000 7fa1afe00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b00743a0 0x7fa1b01a4a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0037040 tx=0x7fa1a00346f0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.441+0000 7fa1afe00640 1 -- 172.21.15.18:0/3431404271 shutdown_connections 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.441+0000 7fa1afe00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1940a3d00 0x7fa1940a61c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.441+0000 7fa1afe00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1b0112e30 0x7fa1b019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.441+0000 7fa1afe00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1b0074cc0 0x7fa1b01a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.441+0000 7fa1afe00640 1 --2- 172.21.15.18:0/3431404271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b00743a0 0x7fa1b01a4a50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.441+0000 7fa1afe00640 1 -- 172.21.15.18:0/3431404271 >> 172.21.15.18:0/3431404271 conn(0x7fa1b00fd910 msgr2=0x7fa1b00ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.442+0000 7fa1afe00640 1 -- 172.21.15.18:0/3431404271 shutdown_connections 2024-04-16T17:17:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:46.442+0000 7fa1afe00640 1 -- 172.21.15.18:0/3431404271 wait complete. 2024-04-16T17:17:46.632 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:43 smithi018 ceph-mon[31776]: pgmap v411: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-04-16T17:17:46.632 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:45 smithi018 ceph-mon[31776]: pgmap v412: 129 pgs: 129 active+clean; 61 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 2.3 MiB/s wr, 108 op/s 2024-04-16T17:17:46.692 INFO:tasks.workunit.client.0.smithi018.stdout: 1 485679 3.56 MB/sec execute 599 sec latency 82.882 ms 2024-04-16T17:17:47.692 INFO:tasks.workunit.client.0.smithi018.stdout: 1 cleanup 600 sec 2024-04-16T17:17:47.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:47 smithi022 ceph-mon[36023]: pgmap v413: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-04-16T17:17:48.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:47 smithi043 ceph-mon[36901]: pgmap v413: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-04-16T17:17:48.017 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:47 smithi018 ceph-mon[31776]: pgmap v413: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-04-16T17:17:48.027 INFO:tasks.workunit.client.0.smithi018.stdout: 0 cleanup 600 sec 2024-04-16T17:17:48.027 INFO:tasks.workunit.client.0.smithi018.stdout: 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Operation Count AvgLat MaxLat 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: ---------------------------------------- 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: NTCreateX 67981 3.376 11487.685 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Close 49971 0.029 7.467 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Rename 2880 9.944 147.580 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Unlink 13716 4.112 127.264 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Deltree 2 154.587 210.505 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Mkdir 1 0.276 0.276 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Qpathinfo 61648 2.185 727.654 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Qfileinfo 10820 0.034 14.793 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Qfsinfo 11287 0.558 29.914 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Sfileinfo 5554 0.722 59.358 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: Find 23822 2.427 314.409 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: WriteX 33973 0.780 808.327 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: ReadX 106529 0.183 59.188 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: LockX 222 1.302 25.921 2024-04-16T17:17:48.028 INFO:tasks.workunit.client.0.smithi018.stdout: UnlockX 222 1.056 37.226 2024-04-16T17:17:48.029 INFO:tasks.workunit.client.0.smithi018.stdout: Flush 4772 6.619 306.229 2024-04-16T17:17:48.029 INFO:tasks.workunit.client.0.smithi018.stdout: 2024-04-16T17:17:48.029 INFO:tasks.workunit.client.0.smithi018.stdout:Throughput 3.55833 MB/sec 1 clients 1 procs max_latency=11487.701 ms 2024-04-16T17:17:48.029 INFO:tasks.workunit.client.0.smithi018.stderr:+ dbench 10 2024-04-16T17:17:48.030 INFO:tasks.workunit.client.0.smithi018.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-16T17:17:48.030 INFO:tasks.workunit.client.0.smithi018.stdout: 2024-04-16T17:17:48.030 INFO:tasks.workunit.client.0.smithi018.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-16T17:17:48.035 INFO:tasks.workunit.client.0.smithi018.stdout:9 of 10 processes prepared for launch 0 sec 2024-04-16T17:17:48.135 INFO:tasks.workunit.client.0.smithi018.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-16T17:17:48.135 INFO:tasks.workunit.client.0.smithi018.stdout:releasing clients 2024-04-16T17:17:48.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:17:48.658 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:17:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.951+0000 7f614ae00640 1 -- 172.21.15.18:0/1987875069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144109350 msgr2=0x7f6144110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:48.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.951+0000 7f614ae00640 1 --2- 172.21.15.18:0/1987875069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144109350 0x7f6144110f20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f613c009ca0 tx=0x7f613c02c140 comp rx=0 tx=0).stop 2024-04-16T17:17:48.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.951+0000 7f614ae00640 1 -- 172.21.15.18:0/1987875069 shutdown_connections 2024-04-16T17:17:48.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.951+0000 7f614ae00640 1 --2- 172.21.15.18:0/1987875069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144109350 0x7f6144110f20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.951+0000 7f614ae00640 1 --2- 172.21.15.18:0/1987875069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6144104c30 0x7f6144108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.951+0000 7f614ae00640 1 --2- 172.21.15.18:0/1987875069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6144104280 0x7f6144104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.951+0000 7f614ae00640 1 -- 172.21.15.18:0/1987875069 >> 172.21.15.18:0/1987875069 conn(0x7f61440ffae0 msgr2=0x7f6144101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:48.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.952+0000 7f614ae00640 1 -- 172.21.15.18:0/1987875069 shutdown_connections 2024-04-16T17:17:48.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.952+0000 7f614ae00640 1 -- 172.21.15.18:0/1987875069 wait complete. 2024-04-16T17:17:48.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.954+0000 7f614ae00640 1 Processor -- start 2024-04-16T17:17:48.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.954+0000 7f614ae00640 1 -- start start 2024-04-16T17:17:48.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.954+0000 7f614ae00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6144104280 0x7f61441a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.954+0000 7f614ae00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6144109350 0x7f61441a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f614ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614419a7f0 0x7f614419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f614ae00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f61441143f0 con 0x7f6144104280 2024-04-16T17:17:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f614ae00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6144114270 con 0x7f6144109350 2024-04-16T17:17:48.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f614ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6144114570 con 0x7f614419a7f0 2024-04-16T17:17:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f6149000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6144109350 0x7f61441a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f614a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614419a7f0 0x7f614419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f614a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614419a7f0 0x7f614419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39000/0 (socket says 172.21.15.18:39000) 2024-04-16T17:17:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f614a400640 1 -- 172.21.15.18:0/3663567708 learned_addr learned my addr 172.21.15.18:0/3663567708 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f6149a00640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6144104280 0x7f61441a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f6149000640 1 -- 172.21.15.18:0/3663567708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614419a7f0 msgr2=0x7f614419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:48.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f6149000640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614419a7f0 0x7f614419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f6149000640 1 -- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6144104280 msgr2=0x7f61441a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:48.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f6149000640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6144104280 0x7f61441a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f6149000640 1 -- 172.21.15.18:0/3663567708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614419b380 con 0x7f6144109350 2024-04-16T17:17:48.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.955+0000 7f614a400640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614419a7f0 0x7f614419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:48.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.956+0000 7f6149000640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6144109350 0x7f61441a0bc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f613000e9c0 tx=0x7f613000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:48.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.956+0000 7f6142000640 1 -- 172.21.15.18:0/3663567708 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613000ccc0 con 0x7f6144109350 2024-04-16T17:17:48.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.956+0000 7f614ae00640 1 -- 172.21.15.18:0/3663567708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614419b670 con 0x7f6144109350 2024-04-16T17:17:48.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.956+0000 7f614ae00640 1 -- 172.21.15.18:0/3663567708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61441a78a0 con 0x7f6144109350 2024-04-16T17:17:48.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.956+0000 7f6142000640 1 -- 172.21.15.18:0/3663567708 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6130010910 con 0x7f6144109350 2024-04-16T17:17:48.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.957+0000 7f6142000640 1 -- 172.21.15.18:0/3663567708 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130018880 con 0x7f6144109350 2024-04-16T17:17:48.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.963+0000 7f6142000640 1 -- 172.21.15.18:0/3663567708 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6130018a20 con 0x7f6144109350 2024-04-16T17:17:48.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.964+0000 7f614f02b740 1 Processor -- start 2024-04-16T17:17:48.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.965+0000 7f614f02b740 1 -- start start 2024-04-16T17:17:48.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.966+0000 7f614f02b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 0x55b041908770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.965+0000 7f6142000640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f612c0a3d60 0x7f612c0a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.966+0000 7f614f02b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041911350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.966+0000 7f614f02b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041913c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.966+0000 7f614f02b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b04186e120 con 0x55b041908390 2024-04-16T17:17:48.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.966+0000 7f614f02b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b041814340 con 0x55b041908cb0 2024-04-16T17:17:48.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.966+0000 7f614f02b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b04186a0e0 con 0x55b041911890 2024-04-16T17:17:48.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.966+0000 7f6142000640 1 -- 172.21.15.18:0/3663567708 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f61300d0fb0 con 0x7f6144109350 2024-04-16T17:17:48.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.968+0000 7f614a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041913c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.968+0000 7f614a400640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f612c0a3d60 0x7f612c0a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.968+0000 7f6149000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041911350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.968+0000 7f6149a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 0x55b041908770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.968+0000 7f6149000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041911350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37204/0 (socket says 172.21.15.18:37204) 2024-04-16T17:17:48.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.968+0000 7f6149000640 1 -- 172.21.15.18:0/994739443 learned_addr learned my addr 172.21.15.18:0/994739443 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:48.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.969+0000 7f614a400640 1 -- 172.21.15.18:0/994739443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 msgr2=0x55b041911350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:48.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.969+0000 7f614a400640 1 --2- 172.21.15.18:0/994739443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041911350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.969+0000 7f614a400640 1 -- 172.21.15.18:0/994739443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 msgr2=0x55b041908770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:48.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.969+0000 7f614a400640 1 --2- 172.21.15.18:0/994739443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 0x55b041908770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.969+0000 7f614a400640 1 -- 172.21.15.18:0/994739443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04164a7d0 con 0x55b041911890 2024-04-16T17:17:48.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.970+0000 7f614a400640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f612c0a3d60 0x7f612c0a6220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f613c005fe0 tx=0x7f613c005ef0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:48.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.970+0000 7f614a400640 1 --2- 172.21.15.18:0/994739443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041913c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f613c005e30 tx=0x7f613c047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a64e658b30a232f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:48.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.970+0000 7f6136a00640 1 -- 172.21.15.18:0/994739443 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c039040 con 0x55b041911890 2024-04-16T17:17:48.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.970+0000 7f6136a00640 1 -- 172.21.15.18:0/994739443 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f613c04fe40 con 0x55b041911890 2024-04-16T17:17:48.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.970+0000 7f6136a00640 1 -- 172.21.15.18:0/994739443 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c039040 con 0x55b041911890 2024-04-16T17:17:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.977+0000 7f614f02b740 1 -- 172.21.15.18:0/994739443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 msgr2=0x55b041913c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.977+0000 7f614f02b740 1 --2- 172.21.15.18:0/994739443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041913c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f613c005e30 tx=0x7f613c047000 comp rx=0 tx=0).stop 2024-04-16T17:17:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.978+0000 7f614f02b740 1 -- 172.21.15.18:0/994739443 shutdown_connections 2024-04-16T17:17:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.978+0000 7f614f02b740 1 --2- 172.21.15.18:0/994739443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041913c80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.978+0000 7f614f02b740 1 --2- 172.21.15.18:0/994739443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041911350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.978+0000 7f614f02b740 1 --2- 172.21.15.18:0/994739443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 0x55b041908770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.978+0000 7f614f02b740 1 -- 172.21.15.18:0/994739443 >> 172.21.15.18:0/994739443 conn(0x55b041902ac0 msgr2=0x55b041902ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:48.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.978+0000 7f614f02b740 1 -- 172.21.15.18:0/994739443 shutdown_connections 2024-04-16T17:17:48.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.979+0000 7f614f02b740 1 -- 172.21.15.18:0/994739443 wait complete. 2024-04-16T17:17:48.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.979+0000 7f614f02b740 1 -- start start 2024-04-16T17:17:48.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.980+0000 7f614f02b740 1 Processor -- start 2024-04-16T17:17:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f614f02b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 0x55b04190e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f614f02b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b04190edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f614f02b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f614f02b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b041914620 con 0x55b041911890 2024-04-16T17:17:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f614f02b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b041827710 con 0x55b041908cb0 2024-04-16T17:17:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f614f02b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b0419147a0 con 0x55b041908390 2024-04-16T17:17:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f614f02b740 10 client.0 _ll_register_callbacks cb 0x55b041822af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f6149000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b04190edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f6149000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b04190edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37208/0 (socket says 172.21.15.18:37208) 2024-04-16T17:17:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.982+0000 7f6149000640 1 -- 172.21.15.18:0/3853639995 learned_addr learned my addr 172.21.15.18:0/3853639995 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.983+0000 7f6149000640 1 -- 172.21.15.18:0/3853639995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 msgr2=0x55b04190e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.983+0000 7f6149a00640 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 0x55b04190e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.983+0000 7f6149000640 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 0x55b04190e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.983+0000 7f6149000640 1 -- 172.21.15.18:0/3853639995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 msgr2=0x55b04190f330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:17:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.983+0000 7f6149000640 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.983+0000 7f6149000640 1 -- 172.21.15.18:0/3853639995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0415803f0 con 0x55b041908cb0 2024-04-16T17:17:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.983+0000 7f6149a00640 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 0x55b04190e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.984+0000 7f6149000640 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b04190edf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f613009da00 tx=0x7f61300be1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f7cfac80e8a80a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.984+0000 7f612b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.985+0000 7f612b400640 1 -- 172.21.15.18:0/3853639995 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61300a6450 con 0x55b041908cb0 2024-04-16T17:17:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.985+0000 7f612b400640 1 -- 172.21.15.18:0/3853639995 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f6130010ac0 con 0x55b041908cb0 2024-04-16T17:17:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.985+0000 7f614f02b740 1 -- 172.21.15.18:0/3853639995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b04187bb40 con 0x55b041908cb0 2024-04-16T17:17:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.985+0000 7f612b400640 1 -- 172.21.15.18:0/3853639995 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f61300a66e0 con 0x55b041908cb0 2024-04-16T17:17:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.985+0000 7f614f02b740 10 client.29150 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:48.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.985+0000 7f614f02b740 1 -- 172.21.15.18:0/3853639995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b0415400f0 con 0x55b041908cb0 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.985+0000 7f612b400640 1 -- 172.21.15.18:0/3853639995 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b04187bb40 con 0x55b041908cb0 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.986+0000 7f612b400640 1 -- 172.21.15.18:0/3853639995 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f61300ba050 con 0x55b041908cb0 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.986+0000 7f614f02b740 10 client.29150 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.986+0000 7f614f02b740 10 client.29150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.986+0000 7f614f02b740 20 client.29150 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.986+0000 7f614f02b740 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b041921d60 0x55b0419241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.986+0000 7f614f02b740 4 client.29150 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.986+0000 7f614f02b740 1 -- 172.21.15.18:0/3853639995 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0418759b0 con 0x55b041921d60 2024-04-16T17:17:48.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.987+0000 7f6149a00640 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b041921d60 0x55b0419241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:48.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.988+0000 7f6149a00640 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b041921d60 0x55b0419241e0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd638cb7afccab3d server_cookie=a11aa080b1d6d916 in_seq=0 out_seq=0 2024-04-16T17:17:48.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.988+0000 7f612b400640 10 client.29150 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:48.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.989+0000 7f612b400640 1 -- 172.21.15.18:0/3853639995 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0418759b0 con 0x55b041921d60 2024-04-16T17:17:48.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.989+0000 7f612b400640 10 client.29150 handle_command_reply: tid=0 2024-04-16T17:17:48.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.989+0000 7f614f02b740 1 client.29150 shutdown 2024-04-16T17:17:48.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 -- 172.21.15.18:0/3853639995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 msgr2=0x55b04190edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:48.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b04190edf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f613009da00 tx=0x7f61300be1e0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 -- 172.21.15.18:0/3853639995 shutdown_connections 2024-04-16T17:17:48.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b041921d60 0x55b0419241e0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b04190edf0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 --2- 172.21.15.18:0/3853639995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 0x55b04190e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:48.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 -- 172.21.15.18:0/3853639995 >> 172.21.15.18:0/3853639995 conn(0x55b041902ac0 msgr2=0x55b041904810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:48.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f612b400640 0 client.29150 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:48.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.990+0000 7f614f02b740 1 -- 172.21.15.18:0/3853639995 shutdown_connections 2024-04-16T17:17:48.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.991+0000 7f614f02b740 1 -- 172.21.15.18:0/3853639995 wait complete. 2024-04-16T17:17:48.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:48.991+0000 7f614f02b740 20 client.29150 trim_cache size 0 max 16384 2024-04-16T17:17:49.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.030+0000 7f614f02b740 1 Processor -- start 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.032+0000 7f614f02b740 1 -- start start 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.032+0000 7f614f02b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 0x55b04191d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.033+0000 7f614f02b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041902ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.033+0000 7f614f02b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.033+0000 7f614f02b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f61240128a0 con 0x55b041911890 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.033+0000 7f614f02b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6124010170 con 0x55b041908cb0 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.033+0000 7f614f02b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f612400dfe0 con 0x55b041908390 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.033+0000 7f614a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.033+0000 7f614a400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47364/0 (socket says 172.21.15.18:47364) 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.033+0000 7f614a400640 1 -- 172.21.15.18:0/1576215827 learned_addr learned my addr 172.21.15.18:0/1576215827 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.034+0000 7f614a400640 1 -- 172.21.15.18:0/1576215827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 msgr2=0x55b04191d570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:17:49.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.034+0000 7f614a400640 1 --2- 172.21.15.18:0/1576215827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 0x55b04191d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.034+0000 7f614a400640 1 -- 172.21.15.18:0/1576215827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 msgr2=0x55b041902ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:17:49.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.034+0000 7f614a400640 1 --2- 172.21.15.18:0/1576215827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041902ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.034+0000 7f614a400640 1 -- 172.21.15.18:0/1576215827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04190f690 con 0x55b041911890 2024-04-16T17:17:49.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.035+0000 7f614a400640 1 --2- 172.21.15.18:0/1576215827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f613c009a60 tx=0x7f613c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6338c474baf21532 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:49.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.035+0000 7f6136000640 1 -- 172.21.15.18:0/1576215827 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c039040 con 0x55b041911890 2024-04-16T17:17:49.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.036+0000 7f6136000640 1 -- 172.21.15.18:0/1576215827 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f613c053030 con 0x55b041911890 2024-04-16T17:17:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.036+0000 7f6136000640 1 -- 172.21.15.18:0/1576215827 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c034c50 con 0x55b041911890 2024-04-16T17:17:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.036+0000 7f614f02b740 1 -- 172.21.15.18:0/1576215827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 msgr2=0x55b04190f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.036+0000 7f614f02b740 1 --2- 172.21.15.18:0/1576215827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f613c009a60 tx=0x7f613c0095c0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.037+0000 7f614f02b740 1 -- 172.21.15.18:0/1576215827 shutdown_connections 2024-04-16T17:17:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.037+0000 7f614f02b740 1 --2- 172.21.15.18:0/1576215827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041911890 0x55b04190f150 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.037+0000 7f614f02b740 1 --2- 172.21.15.18:0/1576215827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041902ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.037+0000 7f614f02b740 1 --2- 172.21.15.18:0/1576215827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041908390 0x55b04191d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.037+0000 7f614f02b740 1 -- 172.21.15.18:0/1576215827 >> 172.21.15.18:0/1576215827 conn(0x55b041902ac0 msgr2=0x55b041901290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.037+0000 7f614f02b740 1 -- 172.21.15.18:0/1576215827 shutdown_connections 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.037+0000 7f614f02b740 1 -- 172.21.15.18:0/1576215827 wait complete. 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.038+0000 7f614f02b740 1 -- start start 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.038+0000 7f614f02b740 1 Processor -- start 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614f02b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 0x55b0419154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614f02b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041915a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614f02b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041915f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614f02b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f612400e630 con 0x55b041908390 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614f02b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f612400faa0 con 0x55b041908cb0 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614f02b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f612400ed00 con 0x55b041911890 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614f02b740 10 client.0 _ll_register_callbacks cb 0x7f613009d3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041915f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041915f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39050/0 (socket says 172.21.15.18:39050) 2024-04-16T17:17:49.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f614a400640 1 -- 172.21.15.18:0/640025529 learned_addr learned my addr 172.21.15.18:0/640025529 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.039+0000 7f6149000640 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041915a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.040+0000 7f6149a00640 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 0x55b0419154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.040+0000 7f614a400640 1 -- 172.21.15.18:0/640025529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 msgr2=0x55b041915a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.040+0000 7f614a400640 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041915a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.040+0000 7f614a400640 1 -- 172.21.15.18:0/640025529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 msgr2=0x55b0419154e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.040+0000 7f614a400640 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 0x55b0419154e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.040+0000 7f614a400640 1 -- 172.21.15.18:0/640025529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b041998330 con 0x55b041911890 2024-04-16T17:17:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.041+0000 7f614a400640 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041915f60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f613c02e010 tx=0x7f613c034890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851ede7f46c9a1cd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.041+0000 7f612b400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.042+0000 7f612b400640 1 -- 172.21.15.18:0/640025529 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c05d060 con 0x55b041911890 2024-04-16T17:17:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.042+0000 7f612b400640 1 -- 172.21.15.18:0/640025529 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f613c034990 con 0x55b041911890 2024-04-16T17:17:49.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.043+0000 7f614f02b740 1 -- 172.21.15.18:0/640025529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f613c053030 con 0x55b041911890 2024-04-16T17:17:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.044+0000 7f612b400640 1 -- 172.21.15.18:0/640025529 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f613c04d070 con 0x55b041911890 2024-04-16T17:17:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.044+0000 7f614f02b740 10 client.28990 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.044+0000 7f614f02b740 1 -- 172.21.15.18:0/640025529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b041999620 con 0x55b041911890 2024-04-16T17:17:49.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.044+0000 7f612b400640 1 -- 172.21.15.18:0/640025529 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f613c053030 con 0x55b041911890 2024-04-16T17:17:49.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.045+0000 7f612b400640 1 -- 172.21.15.18:0/640025529 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f613c0045a0 con 0x55b041911890 2024-04-16T17:17:49.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.045+0000 7f614f02b740 10 client.28990 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:49.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.045+0000 7f614f02b740 10 client.28990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:49.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.045+0000 7f614f02b740 20 client.28990 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:49.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.045+0000 7f614f02b740 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b041999840 0x55b04199bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.045+0000 7f614f02b740 4 client.28990 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:17:49.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.045+0000 7f614f02b740 1 -- 172.21.15.18:0/640025529 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0417f1420 con 0x55b041999840 2024-04-16T17:17:49.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.046+0000 7f6149a00640 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b041999840 0x55b04199bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.047+0000 7f6149a00640 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b041999840 0x55b04199bc30 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a36dbcdd656063b5 server_cookie=f8abad1e3b3a7787 in_seq=0 out_seq=0 2024-04-16T17:17:49.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.047+0000 7f612b400640 10 client.28990 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:49.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f612b400640 1 -- 172.21.15.18:0/640025529 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0417f1420 con 0x55b041999840 2024-04-16T17:17:49.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f612b400640 10 client.28990 handle_command_reply: tid=0 2024-04-16T17:17:49.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 client.28990 shutdown 2024-04-16T17:17:49.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 -- 172.21.15.18:0/640025529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 msgr2=0x55b041915f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041915f60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f613c02e010 tx=0x7f613c034890 comp rx=0 tx=0).stop 2024-04-16T17:17:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 -- 172.21.15.18:0/640025529 shutdown_connections 2024-04-16T17:17:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b041999840 0x55b04199bc30 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f612b400640 0 client.28990 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b041911890 0x55b041915f60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b041908cb0 0x55b041915a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 --2- 172.21.15.18:0/640025529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b041908390 0x55b0419154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.053+0000 7f614f02b740 1 -- 172.21.15.18:0/640025529 >> 172.21.15.18:0/640025529 conn(0x55b041902ac0 msgr2=0x55b041901b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.055+0000 7f614f02b740 1 -- 172.21.15.18:0/640025529 shutdown_connections 2024-04-16T17:17:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.055+0000 7f614f02b740 1 -- 172.21.15.18:0/640025529 wait complete. 2024-04-16T17:17:49.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.055+0000 7f614f02b740 20 client.28990 trim_cache size 0 max 16384 2024-04-16T17:17:49.054 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:17:49.055 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:17:49.055 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:17:49.055 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:17:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.057+0000 7f614ae00640 1 -- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f612c0a3d60 msgr2=0x7f612c0a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.057+0000 7f614ae00640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f612c0a3d60 0x7f612c0a6220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f613c005fe0 tx=0x7f613c005ef0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.057+0000 7f614ae00640 1 -- 172.21.15.18:0/3663567708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6144109350 msgr2=0x7f61441a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.057+0000 7f614ae00640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6144109350 0x7f61441a0bc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f613000e9c0 tx=0x7f613000ee80 comp rx=0 tx=0).stop 2024-04-16T17:17:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.058+0000 7f614ae00640 1 -- 172.21.15.18:0/3663567708 shutdown_connections 2024-04-16T17:17:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.058+0000 7f614ae00640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f612c0a3d60 0x7f612c0a6220 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.058+0000 7f614ae00640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614419a7f0 0x7f614419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.058+0000 7f614ae00640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6144109350 0x7f61441a0bc0 secure :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f613000e9c0 tx=0x7f613000ee80 comp rx=0 tx=0).stop 2024-04-16T17:17:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.058+0000 7f614ae00640 1 --2- 172.21.15.18:0/3663567708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6144104280 0x7f61441a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.058+0000 7f614ae00640 1 -- 172.21.15.18:0/3663567708 >> 172.21.15.18:0/3663567708 conn(0x7f61440ffae0 msgr2=0x7f6144101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:49.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.060+0000 7f614ae00640 1 -- 172.21.15.18:0/3663567708 shutdown_connections 2024-04-16T17:17:49.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.062+0000 7f614ae00640 1 -- 172.21.15.18:0/3663567708 wait complete. 2024-04-16T17:17:49.108 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:17:49.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:17:49.135 INFO:tasks.workunit.client.0.smithi018.stdout: 10 110 49.90 MB/sec warmup 1 sec latency 701.415 ms 2024-04-16T17:17:49.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.382+0000 7f503aa00640 1 -- 172.21.15.18:0/4256573203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5034106280 msgr2=0x7f5034106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.382+0000 7f503aa00640 1 --2- 172.21.15.18:0/4256573203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5034106280 0x7f5034106710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f502c009ca0 tx=0x7f502c02c1b0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.382+0000 7f503aa00640 1 -- 172.21.15.18:0/4256573203 shutdown_connections 2024-04-16T17:17:49.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.382+0000 7f503aa00640 1 --2- 172.21.15.18:0/4256573203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5034106280 0x7f5034106710 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.382+0000 7f503aa00640 1 --2- 172.21.15.18:0/4256573203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50341017a0 0x7f5034105c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.382+0000 7f503aa00640 1 --2- 172.21.15.18:0/4256573203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5034100e80 0x7f5034101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.383+0000 7f503aa00640 1 -- 172.21.15.18:0/4256573203 >> 172.21.15.18:0/4256573203 conn(0x7f50340fc890 msgr2=0x7f50340fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:49.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.383+0000 7f503aa00640 1 -- 172.21.15.18:0/4256573203 shutdown_connections 2024-04-16T17:17:49.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.383+0000 7f503aa00640 1 -- 172.21.15.18:0/4256573203 wait complete. 2024-04-16T17:17:49.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.384+0000 7f503aa00640 1 Processor -- start 2024-04-16T17:17:49.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.384+0000 7f503aa00640 1 -- start start 2024-04-16T17:17:49.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.384+0000 7f503aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5034100e80 0x7f503419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.384+0000 7f503aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50341017a0 0x7f503419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.384+0000 7f503aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5034106280 0x7f50341a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503aa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5034118910 con 0x7f5034100e80 2024-04-16T17:17:49.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503aa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5034118790 con 0x7f50341017a0 2024-04-16T17:17:49.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5034118a90 con 0x7f5034106280 2024-04-16T17:17:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5034106280 0x7f50341a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f5039600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5034100e80 0x7f503419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5034106280 0x7f50341a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39066/0 (socket says 172.21.15.18:39066) 2024-04-16T17:17:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503a000640 1 -- 172.21.15.18:0/2599147398 learned_addr learned my addr 172.21.15.18:0/2599147398 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f5038c00640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50341017a0 0x7f503419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503a000640 1 -- 172.21.15.18:0/2599147398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50341017a0 msgr2=0x7f503419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503a000640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50341017a0 0x7f503419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503a000640 1 -- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5034100e80 msgr2=0x7f503419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503a000640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5034100e80 0x7f503419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.385+0000 7f503a000640 1 -- 172.21.15.18:0/2599147398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50341a3de0 con 0x7f5034106280 2024-04-16T17:17:49.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.386+0000 7f503a000640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5034106280 0x7f50341a36b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f502c005e30 tx=0x7f502c035e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:49.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.386+0000 7f5032000640 1 -- 172.21.15.18:0/2599147398 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c039040 con 0x7f5034106280 2024-04-16T17:17:49.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.386+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50341a4070 con 0x7f5034106280 2024-04-16T17:17:49.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.386+0000 7f5032000640 1 -- 172.21.15.18:0/2599147398 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c044070 con 0x7f5034106280 2024-04-16T17:17:49.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.386+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50341ab960 con 0x7f5034106280 2024-04-16T17:17:49.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.386+0000 7f5032000640 1 -- 172.21.15.18:0/2599147398 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c0328a0 con 0x7f5034106280 2024-04-16T17:17:49.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.388+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503404fe00 con 0x7f5034106280 2024-04-16T17:17:49.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.391+0000 7f5032000640 1 -- 172.21.15.18:0/2599147398 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f502c003b70 con 0x7f5034106280 2024-04-16T17:17:49.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.393+0000 7f5032000640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f501c0a3d90 0x7f501c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:49.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.394+0000 7f5032000640 1 -- 172.21.15.18:0/2599147398 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f502c0f2e80 con 0x7f5034106280 2024-04-16T17:17:49.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.394+0000 7f5039600640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f501c0a3d90 0x7f501c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:49.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.396+0000 7f5039600640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f501c0a3d90 0x7f501c0a6250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5028006fd0 tx=0x7f50280090e0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:49.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.396+0000 7f5032000640 1 -- 172.21.15.18:0/2599147398 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f502c0bf1c0 con 0x7f5034106280 2024-04-16T17:17:49.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.751+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50341abb30 con 0x7f5034106280 2024-04-16T17:17:49.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.752+0000 7f5032000640 1 -- 172.21.15.18:0/2599147398 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11854 (secure 0 0 0) 0x7f502c0beac0 con 0x7f5034106280 2024-04-16T17:17:49.752 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:17:49.752 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:16:00.629944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:17:49.752 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-16T17:17:49.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f501c0a3d90 msgr2=0x7f501c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f501c0a3d90 0x7f501c0a6250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5028006fd0 tx=0x7f50280090e0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5034106280 msgr2=0x7f50341a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:49.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5034106280 0x7f50341a36b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f502c005e30 tx=0x7f502c035e50 comp rx=0 tx=0).stop 2024-04-16T17:17:49.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 shutdown_connections 2024-04-16T17:17:49.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f501c0a3d90 0x7f501c0a6250 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5034106280 0x7f50341a36b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50341017a0 0x7f503419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 --2- 172.21.15.18:0/2599147398 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5034100e80 0x7f503419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:49.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 >> 172.21.15.18:0/2599147398 conn(0x7f50340fc890 msgr2=0x7f5034103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:49.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 shutdown_connections 2024-04-16T17:17:49.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:49.755+0000 7f503aa00640 1 -- 172.21.15.18:0/2599147398 wait complete. 2024-04-16T17:17:49.776 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:17:49.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:49 smithi022 ceph-mon[36023]: pgmap v414: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 887 KiB/s wr, 66 op/s 2024-04-16T17:17:50.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:49 smithi043 ceph-mon[36901]: pgmap v414: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 887 KiB/s wr, 66 op/s 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 -- 172.21.15.18:0/214165580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f2410c400 msgr2=0x7f7f2410e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 --2- 172.21.15.18:0/214165580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f2410c400 0x7f7f2410e870 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009900 tx=0x7f7f1c02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 -- 172.21.15.18:0/214165580 shutdown_connections 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 --2- 172.21.15.18:0/214165580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f2410c400 0x7f7f2410e870 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 --2- 172.21.15.18:0/214165580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f24103900 0x7f7f2410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 --2- 172.21.15.18:0/214165580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f24102fe0 0x7f7f241033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 -- 172.21.15.18:0/214165580 >> 172.21.15.18:0/214165580 conn(0x7f7f24077ab0 msgr2=0x7f7f240fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 -- 172.21.15.18:0/214165580 shutdown_connections 2024-04-16T17:17:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.076+0000 7f7f23200640 1 -- 172.21.15.18:0/214165580 wait complete. 2024-04-16T17:17:50.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.078+0000 7f7f23200640 1 Processor -- start 2024-04-16T17:17:50.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.078+0000 7f7f23200640 1 -- start start 2024-04-16T17:17:50.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.078+0000 7f7f23200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f24102fe0 0x7f7f241a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f23200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24103900 0x7f7f241a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f23200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2410c400 0x7f7f2419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f23200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24112190 con 0x7f7f2410c400 2024-04-16T17:17:50.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f23200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24112010 con 0x7f7f24102fe0 2024-04-16T17:17:50.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f23200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24112310 con 0x7f7f24103900 2024-04-16T17:17:50.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f24102fe0 0x7f7f241a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f24102fe0 0x7f7f241a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37276/0 (socket says 172.21.15.18:37276) 2024-04-16T17:17:50.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f21e00640 1 -- 172.21.15.18:0/1122989952 learned_addr learned my addr 172.21.15.18:0/1122989952 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f19400640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24103900 0x7f7f241a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f22800640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2410c400 0x7f7f2419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f21e00640 1 -- 172.21.15.18:0/1122989952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24103900 msgr2=0x7f7f241a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f21e00640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24103900 0x7f7f241a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f21e00640 1 -- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2410c400 msgr2=0x7f7f2419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f21e00640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2410c400 0x7f7f2419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f21e00640 1 -- 172.21.15.18:0/1122989952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2419af00 con 0x7f7f24102fe0 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.079+0000 7f7f22800640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2410c400 0x7f7f2419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.080+0000 7f7f21e00640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f24102fe0 0x7f7f241a0580 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c920 tx=0x7f7f1400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.080+0000 7f7f1b400640 1 -- 172.21.15.18:0/1122989952 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14007c70 con 0x7f7f24102fe0 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.080+0000 7f7f23200640 1 -- 172.21.15.18:0/1122989952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2419b1f0 con 0x7f7f24102fe0 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.080+0000 7f7f1b400640 1 -- 172.21.15.18:0/1122989952 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f14007e10 con 0x7f7f24102fe0 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.080+0000 7f7f1b400640 1 -- 172.21.15.18:0/1122989952 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14018890 con 0x7f7f24102fe0 2024-04-16T17:17:50.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.081+0000 7f7f23200640 1 -- 172.21.15.18:0/1122989952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241a7500 con 0x7f7f24102fe0 2024-04-16T17:17:50.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.084+0000 7f7f1b400640 1 -- 172.21.15.18:0/1122989952 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7f14018b30 con 0x7f7f24102fe0 2024-04-16T17:17:50.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.084+0000 7f7f2bf8c740 1 Processor -- start 2024-04-16T17:17:50.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.085+0000 7f7f2bf8c740 1 -- start start 2024-04-16T17:17:50.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.085+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.085+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.085+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.085+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ff1f347020 con 0x55ff1f3eb410 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.085+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ff1f2ed460 con 0x55ff1f3e1f10 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.085+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ff1f342fe0 con 0x55ff1f3e2830 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f19400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f1b400640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f040a3e20 0x7f7f040a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f19400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3ed800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47440/0 (socket says 172.21.15.18:47440) 2024-04-16T17:17:50.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f19400640 1 -- 172.21.15.18:0/1364050235 learned_addr learned my addr 172.21.15.18:0/1364050235 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f1b400640 1 -- 172.21.15.18:0/1122989952 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f7f140d0cf0 con 0x7f7f24102fe0 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f21e00640 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f22800640 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f21e00640 1 -- 172.21.15.18:0/1364050235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 msgr2=0x55ff1f3e22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f21e00640 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.086+0000 7f7f21e00640 1 -- 172.21.15.18:0/1364050235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 msgr2=0x55ff1f3ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.087+0000 7f7f21e00640 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.087+0000 7f7f21e00640 1 -- 172.21.15.18:0/1364050235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1f2f5040 con 0x55ff1f3e2830 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.087+0000 7f7f19400640 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:50.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.087+0000 7f7f21e00640 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3eaed0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7f140cdfd0 tx=0x7f7f1409c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddfe2d0e3c8ae5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.087+0000 7f7f22800640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f040a3e20 0x7f7f040a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.087+0000 7f7f13400640 1 -- 172.21.15.18:0/1364050235 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1409c820 con 0x55ff1f3e2830 2024-04-16T17:17:50.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.087+0000 7f7f13400640 1 -- 172.21.15.18:0/1364050235 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f140a0bd0 con 0x55ff1f3e2830 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.087+0000 7f7f13400640 1 -- 172.21.15.18:0/1364050235 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14098e90 con 0x55ff1f3e2830 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.088+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/1364050235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 msgr2=0x55ff1f3eaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.088+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3eaed0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7f140cdfd0 tx=0x7f7f1409c420 comp rx=0 tx=0).stop 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.088+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/1364050235 shutdown_connections 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.088+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.088+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3eaed0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.088+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/1364050235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.088+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/1364050235 >> 172.21.15.18:0/1364050235 conn(0x55ff1f3dc640 msgr2=0x55ff1f3dca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.089+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/1364050235 shutdown_connections 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.089+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/1364050235 wait complete. 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.089+0000 7f7f2bf8c740 1 -- start start 2024-04-16T17:17:50.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.089+0000 7f7f2bf8c740 1 Processor -- start 2024-04-16T17:17:50.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.090+0000 7f7f22800640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f040a3e20 0x7f7f040a62e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c02bcc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.090+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.090+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.090+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3eb410 0x55ff1f3e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.090+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ff1f3ff210 con 0x55ff1f3e1f10 2024-04-16T17:17:50.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.090+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ff1f300830 con 0x55ff1f3e2830 2024-04-16T17:17:50.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.090+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ff1f3ff390 con 0x55ff1f3eb410 2024-04-16T17:17:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f2bf8c740 10 client.0 _ll_register_callbacks cb 0x55ff1f308150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47452/0 (socket says 172.21.15.18:47452) 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f21e00640 1 -- 172.21.15.18:0/3712808700 learned_addr learned my addr 172.21.15.18:0/3712808700 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f22800640 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3eb410 0x55ff1f3e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f21e00640 1 -- 172.21.15.18:0/3712808700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3eb410 msgr2=0x55ff1f3e8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f21e00640 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3eb410 0x55ff1f3e8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f21e00640 1 -- 172.21.15.18:0/3712808700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e2830 msgr2=0x55ff1f3e8a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f21e00640 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e8a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.091+0000 7f7f21e00640 1 -- 172.21.15.18:0/3712808700 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1f304920 con 0x55ff1f3e1f10 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.092+0000 7f7f21e00640 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e84f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7f1409c7b0 tx=0x7f7f14099140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2064f6c507a90ba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.092+0000 7f7f0be00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.092+0000 7f7f0be00640 1 -- 172.21.15.18:0/3712808700 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1409c820 con 0x55ff1f3e1f10 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.092+0000 7f7f0be00640 1 -- 172.21.15.18:0/3712808700 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f7f140a5000 con 0x55ff1f3e1f10 2024-04-16T17:17:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.092+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3712808700 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f140a0bd0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.092+0000 7f7f0be00640 1 -- 172.21.15.18:0/3712808700 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f140a5240 con 0x55ff1f3e1f10 2024-04-16T17:17:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f2bf8c740 10 client.18849 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3712808700 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ff1f2d27f0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f0be00640 1 -- 172.21.15.18:0/3712808700 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f140a0bd0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f0be00640 1 -- 172.21.15.18:0/3712808700 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7f1409b6e0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f2bf8c740 10 client.18849 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f2bf8c740 10 client.18849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f2bf8c740 20 client.18849 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ff1f40c530 0x55ff1f40e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f2bf8c740 4 client.18849 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.093+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3712808700 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff1f354a40 con 0x55ff1f40c530 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.094+0000 7f7f19400640 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ff1f40c530 0x55ff1f40e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.094+0000 7f7f0be00640 10 client.18849 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:17:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.094+0000 7f7f19400640 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ff1f40c530 0x55ff1f40e920 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8320ce18b0f46ea1 server_cookie=1e633c2d107b0e7a in_seq=0 out_seq=0 2024-04-16T17:17:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.096+0000 7f7f0be00640 1 -- 172.21.15.18:0/3712808700 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff1f354a40 con 0x55ff1f40c530 2024-04-16T17:17:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.096+0000 7f7f0be00640 10 client.18849 handle_command_reply: tid=0 2024-04-16T17:17:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.096+0000 7f7f2bf8c740 1 client.18849 shutdown 2024-04-16T17:17:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.096+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3712808700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3e1f10 msgr2=0x55ff1f3e84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.096+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e84f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7f1409c7b0 tx=0x7f7f14099140 comp rx=0 tx=0).stop 2024-04-16T17:17:50.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.097+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3712808700 shutdown_connections 2024-04-16T17:17:50.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.097+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ff1f40c530 0x55ff1f40e920 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.097+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3eb410 0x55ff1f3e8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.097+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.097+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3712808700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3e84f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.097+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3712808700 >> 172.21.15.18:0/3712808700 conn(0x55ff1f3dc640 msgr2=0x55ff1f3de390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.099+0000 7f7f0be00640 0 client.18849 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:17:50.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.099+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3712808700 shutdown_connections 2024-04-16T17:17:50.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.103+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3712808700 wait complete. 2024-04-16T17:17:50.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.103+0000 7f7f2bf8c740 20 client.18849 trim_cache size 0 max 16384 2024-04-16T17:17:50.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.120+0000 7f7f2bf8c740 1 Processor -- start 2024-04-16T17:17:50.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.120+0000 7f7f2bf8c740 1 -- start start 2024-04-16T17:17:50.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.120+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3eac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.120+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.120+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.120+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241ab610 con 0x55ff1f3eb410 2024-04-16T17:17:50.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.120+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241a8ee0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.120+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24069bc0 con 0x55ff1f3e2830 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3eac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39110/0 (socket says 172.21.15.18:39110) 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3eac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37308/0 (socket says 172.21.15.18:37308) 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 -- 172.21.15.18:0/3222968849 learned_addr learned my addr 172.21.15.18:0/3222968849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f22800640 1 --2- 172.21.15.18:0/3222968849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 -- 172.21.15.18:0/3222968849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 msgr2=0x55ff1f3eac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 --2- 172.21.15.18:0/3222968849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3eac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 -- 172.21.15.18:0/3222968849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 msgr2=0x55ff1f3e8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 --2- 172.21.15.18:0/3222968849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3e8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 -- 172.21.15.18:0/3222968849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1f3e74a0 con 0x55ff1f3e2830 2024-04-16T17:17:50.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f19400640 1 --2- 172.21.15.18:0/3222968849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e6dc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009b20 tx=0x7f7f0c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21dc292ff2b7dc42 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f12a00640 1 -- 172.21.15.18:0/3222968849 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c004570 con 0x55ff1f3e2830 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f12a00640 1 -- 172.21.15.18:0/3222968849 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f0c01e920 con 0x55ff1f3e2830 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.121+0000 7f7f12a00640 1 -- 172.21.15.18:0/3222968849 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0057f0 con 0x55ff1f3e2830 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3222968849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 msgr2=0x55ff1f3e6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3222968849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e6dc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009b20 tx=0x7f7f0c009f90 comp rx=0 tx=0).stop 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3222968849 shutdown_connections 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3222968849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f3e8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3222968849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f3e6dc0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3222968849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f3eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3222968849 >> 172.21.15.18:0/3222968849 conn(0x55ff1f3dc640 msgr2=0x55ff1f3db700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3222968849 shutdown_connections 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.122+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3222968849 wait complete. 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.123+0000 7f7f2bf8c740 1 -- start start 2024-04-16T17:17:50.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.123+0000 7f7f2bf8c740 1 Processor -- start 2024-04-16T17:17:50.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.123+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f40ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f40f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f2bf8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f40f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24073fc0 con 0x55ff1f3eb410 2024-04-16T17:17:50.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241a8810 con 0x55ff1f3e1f10 2024-04-16T17:17:50.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f2bf8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241a7a70 con 0x55ff1f3e2830 2024-04-16T17:17:50.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f2bf8c740 10 client.0 _ll_register_callbacks cb 0x7f7f0c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f40ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f40ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37314/0 (socket says 172.21.15.18:37314) 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f21e00640 1 -- 172.21.15.18:0/3233974076 learned_addr learned my addr 172.21.15.18:0/3233974076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f19400640 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f40f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f22800640 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f40f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f21e00640 1 -- 172.21.15.18:0/3233974076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 msgr2=0x55ff1f40f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f21e00640 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f40f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.124+0000 7f7f21e00640 1 -- 172.21.15.18:0/3233974076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 msgr2=0x55ff1f40f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.125+0000 7f7f21e00640 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f40f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.125+0000 7f7f21e00640 1 -- 172.21.15.18:0/3233974076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1f0593f0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.125+0000 7f7f22800640 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f40f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.125+0000 7f7f21e00640 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f40ef70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400fee0 tx=0x7f7f14000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=124c6691e292b0a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.125+0000 7f7f0be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.125+0000 7f7f0be00640 1 -- 172.21.15.18:0/3233974076 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140b9070 con 0x55ff1f3e1f10 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.125+0000 7f7f0be00640 1 -- 172.21.15.18:0/3233974076 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f7f140a1f20 con 0x55ff1f3e1f10 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3233974076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff1f34e8b0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f0be00640 1 -- 172.21.15.18:0/3233974076 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f1409b8e0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 10 client.29174 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3233974076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ff1f1237d0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f0be00640 1 -- 172.21.15.18:0/3233974076 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ff1f34e8b0 con 0x55ff1f3e1f10 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f0be00640 1 -- 172.21.15.18:0/3233974076 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7f1409db50 con 0x55ff1f3e1f10 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 10 client.29174 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 10 client.29174 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 20 client.29174 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ff1f47b360 0x55ff1f47d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 4 client.29174 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.126+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3233974076 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff1f2ca570 con 0x55ff1f47b360 2024-04-16T17:17:50.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.127+0000 7f7f19400640 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ff1f47b360 0x55ff1f47d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.127+0000 7f7f0be00640 10 client.29174 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.127+0000 7f7f19400640 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ff1f47b360 0x55ff1f47d750 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=904294cd45668b6c server_cookie=a64f309298c27ea in_seq=0 out_seq=0 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.127+0000 7f7f0be00640 1 -- 172.21.15.18:0/3233974076 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff1f2ca570 con 0x55ff1f47b360 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.127+0000 7f7f0be00640 10 client.29174 handle_command_reply: tid=0 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.127+0000 7f7f2bf8c740 1 client.29174 shutdown 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.128+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3233974076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 msgr2=0x55ff1f40ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.128+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f40ef70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400fee0 tx=0x7f7f14000bc0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3233974076 shutdown_connections 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55ff1f47b360 0x55ff1f47d750 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ff1f3eb410 0x55ff1f40f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1f3e2830 0x55ff1f40f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 1 --2- 172.21.15.18:0/3233974076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff1f3e1f10 0x55ff1f40ef70 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f0be00640 0 client.29174 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3233974076 >> 172.21.15.18:0/3233974076 conn(0x55ff1f3dc640 msgr2=0x55ff1f407ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3233974076 shutdown_connections 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 1 -- 172.21.15.18:0/3233974076 wait complete. 2024-04-16T17:17:50.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f2bf8c740 20 client.29174 trim_cache size 0 max 16384 2024-04-16T17:17:50.130 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:17:50.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f23200640 1 -- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f040a3e20 msgr2=0x7f7f040a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f23200640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f040a3e20 0x7f7f040a62e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f23200640 1 -- 172.21.15.18:0/1122989952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f24102fe0 msgr2=0x7f7f241a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.129+0000 7f7f23200640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f24102fe0 0x7f7f241a0580 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c920 tx=0x7f7f1400cde0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.130+0000 7f7f23200640 1 -- 172.21.15.18:0/1122989952 shutdown_connections 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.130+0000 7f7f23200640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7f040a3e20 0x7f7f040a62e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.130+0000 7f7f23200640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2410c400 0x7f7f2419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.130+0000 7f7f23200640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24103900 0x7f7f241a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.130+0000 7f7f23200640 1 --2- 172.21.15.18:0/1122989952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f24102fe0 0x7f7f241a0580 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.130+0000 7f7f23200640 1 -- 172.21.15.18:0/1122989952 >> 172.21.15.18:0/1122989952 conn(0x7f7f24077ab0 msgr2=0x7f7f24100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.130+0000 7f7f23200640 1 -- 172.21.15.18:0/1122989952 shutdown_connections 2024-04-16T17:17:50.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.130+0000 7f7f23200640 1 -- 172.21.15.18:0/1122989952 wait complete. 2024-04-16T17:17:50.135 INFO:tasks.workunit.client.0.smithi018.stdout: 10 407 72.09 MB/sec warmup 2 sec latency 134.006 ms 2024-04-16T17:17:50.148 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:17:50.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.373+0000 7fc757000640 1 -- 172.21.15.18:0/3719997104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750102070 msgr2=0x7fc750102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.373+0000 7fc757000640 1 --2- 172.21.15.18:0/3719997104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750102070 0x7fc750102450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc744009900 tx=0x7fc74402bc40 comp rx=0 tx=0).stop 2024-04-16T17:17:50.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.374+0000 7fc757000640 1 -- 172.21.15.18:0/3719997104 shutdown_connections 2024-04-16T17:17:50.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.374+0000 7fc757000640 1 --2- 172.21.15.18:0/3719997104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc750107140 0x7fc75010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.374+0000 7fc757000640 1 --2- 172.21.15.18:0/3719997104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc750102a20 0x7fc750106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.374+0000 7fc757000640 1 --2- 172.21.15.18:0/3719997104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750102070 0x7fc750102450 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.374+0000 7fc757000640 1 -- 172.21.15.18:0/3719997104 >> 172.21.15.18:0/3719997104 conn(0x7fc7500fd950 msgr2=0x7fc7500ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.374+0000 7fc757000640 1 -- 172.21.15.18:0/3719997104 shutdown_connections 2024-04-16T17:17:50.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.374+0000 7fc757000640 1 -- 172.21.15.18:0/3719997104 wait complete. 2024-04-16T17:17:50.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.375+0000 7fc757000640 1 Processor -- start 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.375+0000 7fc757000640 1 -- start start 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc757000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc750102070 0x7fc75010e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc757000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750107140 0x7fc75010e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc757000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7501a3640 0x7fc7501a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc757000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc750112220 con 0x7fc750102070 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc757000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501120a0 con 0x7fc750107140 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc757000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501123a0 con 0x7fc7501a3640 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc755200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750107140 0x7fc75010e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc755c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc750102070 0x7fc75010e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc755200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750107140 0x7fc75010e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37330/0 (socket says 172.21.15.18:37330) 2024-04-16T17:17:50.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.376+0000 7fc755200640 1 -- 172.21.15.18:0/1462539978 learned_addr learned my addr 172.21.15.18:0/1462539978 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.377+0000 7fc755200640 1 -- 172.21.15.18:0/1462539978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7501a3640 msgr2=0x7fc7501a3af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:17:50.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.377+0000 7fc755200640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7501a3640 0x7fc7501a3af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.377+0000 7fc755200640 1 -- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc750102070 msgr2=0x7fc75010e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.377+0000 7fc755200640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc750102070 0x7fc75010e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.377+0000 7fc755200640 1 -- 172.21.15.18:0/1462539978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7501a4130 con 0x7fc750107140 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.377+0000 7fc755c00640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc750102070 0x7fc75010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.377+0000 7fc755200640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750107140 0x7fc75010e6b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00e9c0 tx=0x7fc73c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.378+0000 7fc74e000640 1 -- 172.21.15.18:0/1462539978 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c00cd80 con 0x7fc750107140 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.378+0000 7fc74e000640 1 -- 172.21.15.18:0/1462539978 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc73c005500 con 0x7fc750107140 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.378+0000 7fc74e000640 1 -- 172.21.15.18:0/1462539978 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c010690 con 0x7fc750107140 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.378+0000 7fc757000640 1 -- 172.21.15.18:0/1462539978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7501a4420 con 0x7fc750107140 2024-04-16T17:17:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.378+0000 7fc757000640 1 -- 172.21.15.18:0/1462539978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7501979c0 con 0x7fc750107140 2024-04-16T17:17:50.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.382+0000 7fc75b3aa740 1 Processor -- start 2024-04-16T17:17:50.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.382+0000 7fc75b3aa740 1 -- start start 2024-04-16T17:17:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.383+0000 7fc74e000640 1 -- 172.21.15.18:0/1462539978 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc73c0046c0 con 0x7fc750107140 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.383+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b6f10 0x559b1c0b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.384+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 0x559b1c0bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.384+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0c2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.384+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559b1c01c020 con 0x559b1c0b6f10 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.384+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559b1bfc2460 con 0x559b1c0c0410 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.384+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b1c017fe0 con 0x559b1c0b7830 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.384+0000 7fc755200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.384+0000 7fc755200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0c2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37342/0 (socket says 172.21.15.18:37342) 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.384+0000 7fc755200640 1 -- 172.21.15.18:0/744053649 learned_addr learned my addr 172.21.15.18:0/744053649 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.385+0000 7fc755200640 1 -- 172.21.15.18:0/744053649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 msgr2=0x559b1c0bfed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:17:50.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.385+0000 7fc755200640 1 --2- 172.21.15.18:0/744053649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 0x559b1c0bfed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.385+0000 7fc755200640 1 -- 172.21.15.18:0/744053649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b6f10 msgr2=0x559b1c0b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.385+0000 7fc755200640 1 --2- 172.21.15.18:0/744053649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b6f10 0x559b1c0b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.385+0000 7fc755200640 1 -- 172.21.15.18:0/744053649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1bfca040 con 0x559b1c0c0410 2024-04-16T17:17:50.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.386+0000 7fc755200640 1 --2- 172.21.15.18:0/744053649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0c2800 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0ce2c0 tx=0x7fc73c0d91a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15da18f6ec254d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.386+0000 7fc74e000640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7380a3d20 0x7fc7380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.387+0000 7fc74e000640 1 -- 172.21.15.18:0/1462539978 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fc73c0d10a0 con 0x7fc750107140 2024-04-16T17:17:50.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.387+0000 7fc742a00640 1 -- 172.21.15.18:0/744053649 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0e4070 con 0x559b1c0c0410 2024-04-16T17:17:50.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.387+0000 7fc742a00640 1 -- 172.21.15.18:0/744053649 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc73c0d32e0 con 0x559b1c0c0410 2024-04-16T17:17:50.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.387+0000 7fc742a00640 1 -- 172.21.15.18:0/744053649 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c09c530 con 0x559b1c0c0410 2024-04-16T17:17:50.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.387+0000 7fc756600640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7380a3d20 0x7fc7380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc75b3aa740 1 -- 172.21.15.18:0/744053649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 msgr2=0x559b1c0c2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/744053649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0c2800 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0ce2c0 tx=0x7fc73c0d91a0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc75b3aa740 1 -- 172.21.15.18:0/744053649 shutdown_connections 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/744053649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0c2800 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/744053649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 0x559b1c0bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/744053649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b6f10 0x559b1c0b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc75b3aa740 1 -- 172.21.15.18:0/744053649 >> 172.21.15.18:0/744053649 conn(0x559b1c0b1640 msgr2=0x559b1c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc75b3aa740 1 -- 172.21.15.18:0/744053649 shutdown_connections 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.388+0000 7fc756600640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7380a3d20 0x7fc7380a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc748005fd0 tx=0x7fc748009880 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.389+0000 7fc75b3aa740 1 -- 172.21.15.18:0/744053649 wait complete. 2024-04-16T17:17:50.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.389+0000 7fc75b3aa740 1 -- start start 2024-04-16T17:17:50.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.389+0000 7fc75b3aa740 1 Processor -- start 2024-04-16T17:17:50.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.390+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b6f10 0x559b1c0bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.390+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 0x559b1c0bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.390+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0bdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.390+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559b1c0c3200 con 0x559b1c0b6f10 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.390+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559b1bfd5830 con 0x559b1c0c0410 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.390+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b1c0c3380 con 0x559b1c0b7830 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.390+0000 7fc75b3aa740 10 client.0 _ll_register_callbacks cb 0x559b1bfdd150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.390+0000 7fc756600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0bdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.391+0000 7fc756600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0bdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37356/0 (socket says 172.21.15.18:37356) 2024-04-16T17:17:50.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.391+0000 7fc756600640 1 -- 172.21.15.18:0/3445767775 learned_addr learned my addr 172.21.15.18:0/3445767775 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.391+0000 7fc755200640 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 0x559b1c0bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.391+0000 7fc756600640 1 -- 172.21.15.18:0/3445767775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 msgr2=0x559b1c0bd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.391+0000 7fc756600640 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 0x559b1c0bd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.391+0000 7fc756600640 1 -- 172.21.15.18:0/3445767775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b6f10 msgr2=0x559b1c0bd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.391+0000 7fc756600640 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b6f10 0x559b1c0bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.391+0000 7fc756600640 1 -- 172.21.15.18:0/3445767775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1bfd9920 con 0x559b1c0c0410 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.392+0000 7fc756600640 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0bdf30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc748003d40 tx=0x7fc748004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43508cfdada143a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.392+0000 7fc737400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.392+0000 7fc737400640 1 -- 172.21.15.18:0/3445767775 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748021070 con 0x559b1c0c0410 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.392+0000 7fc737400640 1 -- 172.21.15.18:0/3445767775 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fc74801d840 con 0x559b1c0c0410 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.392+0000 7fc75b3aa740 1 -- 172.21.15.18:0/3445767775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc73c0d32e0 con 0x559b1c0c0410 2024-04-16T17:17:50.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.392+0000 7fc737400640 1 -- 172.21.15.18:0/3445767775 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc74802b050 con 0x559b1c0c0410 2024-04-16T17:17:50.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.393+0000 7fc75b3aa740 10 client.29186 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:50.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.393+0000 7fc75b3aa740 1 -- 172.21.15.18:0/3445767775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b1bfa77f0 con 0x559b1c0c0410 2024-04-16T17:17:50.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.393+0000 7fc737400640 1 -- 172.21.15.18:0/3445767775 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc73c0d32e0 con 0x559b1c0c0410 2024-04-16T17:17:50.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.394+0000 7fc737400640 1 -- 172.21.15.18:0/3445767775 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc748026b60 con 0x559b1c0c0410 2024-04-16T17:17:50.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.394+0000 7fc75b3aa740 10 client.29186 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:50.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.394+0000 7fc75b3aa740 10 client.29186 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:17:50.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.394+0000 7fc75b3aa740 20 client.29186 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:50.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.394+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559b1c0d08a0 0x559b1c0d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.394+0000 7fc75b3aa740 4 client.29186 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:50.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.394+0000 7fc75b3aa740 1 -- 172.21.15.18:0/3445767775 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b1c029a40 con 0x559b1c0d08a0 2024-04-16T17:17:50.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.394+0000 7fc755c00640 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559b1c0d08a0 0x559b1c0d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.395+0000 7fc755c00640 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559b1c0d08a0 0x559b1c0d2cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3749e57ecbf40a5 server_cookie=3832259c67beb1ae in_seq=0 out_seq=0 2024-04-16T17:17:50.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.395+0000 7fc737400640 10 client.29186 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:17:50.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.396+0000 7fc737400640 1 -- 172.21.15.18:0/3445767775 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b1c029a40 con 0x559b1c0d08a0 2024-04-16T17:17:50.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.396+0000 7fc737400640 10 client.29186 handle_command_reply: tid=0 2024-04-16T17:17:50.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.396+0000 7fc75b3aa740 1 client.29186 shutdown 2024-04-16T17:17:50.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 -- 172.21.15.18:0/3445767775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 msgr2=0x559b1c0bdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0bdf30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc748003d40 tx=0x7fc748004a40 comp rx=0 tx=0).stop 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 -- 172.21.15.18:0/3445767775 shutdown_connections 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559b1c0d08a0 0x559b1c0d2cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0bdf30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b7830 0x559b1c0bd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/3445767775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b6f10 0x559b1c0bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc737400640 0 client.29186 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 -- 172.21.15.18:0/3445767775 >> 172.21.15.18:0/3445767775 conn(0x559b1c0b1640 msgr2=0x559b1c0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 -- 172.21.15.18:0/3445767775 shutdown_connections 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 1 -- 172.21.15.18:0/3445767775 wait complete. 2024-04-16T17:17:50.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.397+0000 7fc75b3aa740 20 client.29186 trim_cache size 0 max 16384 2024-04-16T17:17:50.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.420+0000 7fc75b3aa740 1 Processor -- start 2024-04-16T17:17:50.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.420+0000 7fc75b3aa740 1 -- start start 2024-04-16T17:17:50.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b6f10 0x559b1c0bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0d85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501c0ce0 con 0x559b1c0b7830 2024-04-16T17:17:50.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501be5b0 con 0x559b1c0c0410 2024-04-16T17:17:50.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501a0170 con 0x559b1c0b6f10 2024-04-16T17:17:50.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc755c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b6f10 0x559b1c0bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc756600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0d85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc755c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b6f10 0x559b1c0bfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39134/0 (socket says 172.21.15.18:39134) 2024-04-16T17:17:50.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc755c00640 1 -- 172.21.15.18:0/1082456164 learned_addr learned my addr 172.21.15.18:0/1082456164 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.421+0000 7fc755200640 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.422+0000 7fc756600640 1 -- 172.21.15.18:0/1082456164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b6f10 msgr2=0x559b1c0bfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.422+0000 7fc756600640 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b6f10 0x559b1c0bfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.422+0000 7fc756600640 1 -- 172.21.15.18:0/1082456164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 msgr2=0x559b1c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.422+0000 7fc756600640 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.422+0000 7fc756600640 1 -- 172.21.15.18:0/1082456164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1c0b7660 con 0x559b1c0c0410 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.422+0000 7fc755c00640 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b6f10 0x559b1c0bfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.422+0000 7fc756600640 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0d85c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc74800ebb0 tx=0x7fc748004890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8377372e01ea684 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.422+0000 7fc742000640 1 -- 172.21.15.18:0/1082456164 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748021070 con 0x559b1c0c0410 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.423+0000 7fc742000640 1 -- 172.21.15.18:0/1082456164 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74801db20 con 0x559b1c0c0410 2024-04-16T17:17:50.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.423+0000 7fc742000640 1 -- 172.21.15.18:0/1082456164 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748010e20 con 0x559b1c0c0410 2024-04-16T17:17:50.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.423+0000 7fc75b3aa740 1 -- 172.21.15.18:0/1082456164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 msgr2=0x559b1c0d85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.423+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0d85c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc74800ebb0 tx=0x7fc748004890 comp rx=0 tx=0).stop 2024-04-16T17:17:50.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.424+0000 7fc75b3aa740 1 -- 172.21.15.18:0/1082456164 shutdown_connections 2024-04-16T17:17:50.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.424+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0c0410 0x559b1c0d85c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.424+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.424+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/1082456164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0b6f10 0x559b1c0bfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.424+0000 7fc75b3aa740 1 -- 172.21.15.18:0/1082456164 >> 172.21.15.18:0/1082456164 conn(0x559b1c0b1640 msgr2=0x559b1c0b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.424+0000 7fc75b3aa740 1 -- 172.21.15.18:0/1082456164 shutdown_connections 2024-04-16T17:17:50.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.424+0000 7fc75b3aa740 1 -- 172.21.15.18:0/1082456164 wait complete. 2024-04-16T17:17:50.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.424+0000 7fc75b3aa740 1 -- start start 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc75b3aa740 1 Processor -- start 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0b6f10 0x559b1c0c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc75b3aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0c0410 0x559b1c0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501bca70 con 0x559b1c0b7830 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501bdee0 con 0x559b1c0b6f10 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc75b3aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501bd140 con 0x559b1c0c0410 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc75b3aa740 10 client.0 _ll_register_callbacks cb 0x7fc748014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc755c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0b6f10 0x559b1c0c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc755c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0b6f10 0x559b1c0c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37386/0 (socket says 172.21.15.18:37386) 2024-04-16T17:17:50.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.425+0000 7fc755c00640 1 -- 172.21.15.18:0/11094994 learned_addr learned my addr 172.21.15.18:0/11094994 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc756600640 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0c0410 0x559b1c0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc755c00640 1 -- 172.21.15.18:0/11094994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0c0410 msgr2=0x559b1c0c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc755c00640 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0c0410 0x559b1c0c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc755c00640 1 -- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 msgr2=0x559b1c0c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc755200640 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc755c00640 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc755c00640 1 -- 172.21.15.18:0/11094994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1bd2e3f0 con 0x559b1c0b6f10 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc755200640 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc755c00640 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0b6f10 0x559b1c0c4030 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc744009340 tx=0x7fc74402bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6950d9d850da34b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.426+0000 7fc737400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.427+0000 7fc737400640 1 -- 172.21.15.18:0/11094994 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74404b060 con 0x559b1c0b6f10 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.427+0000 7fc737400640 1 -- 172.21.15.18:0/11094994 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fc744045d50 con 0x559b1c0b6f10 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.427+0000 7fc75b3aa740 1 -- 172.21.15.18:0/11094994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc74801db20 con 0x559b1c0b6f10 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.427+0000 7fc737400640 1 -- 172.21.15.18:0/11094994 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc744052050 con 0x559b1c0b6f10 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.427+0000 7fc75b3aa740 10 client.29192 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:50.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.427+0000 7fc75b3aa740 1 -- 172.21.15.18:0/11094994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b1bdf87d0 con 0x559b1c0b6f10 2024-04-16T17:17:50.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.428+0000 7fc737400640 1 -- 172.21.15.18:0/11094994 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74801db20 con 0x559b1c0b6f10 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.428+0000 7fc737400640 1 -- 172.21.15.18:0/11094994 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc744004f60 con 0x559b1c0b6f10 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.428+0000 7fc75b3aa740 10 client.29192 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.428+0000 7fc75b3aa740 10 client.29192 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.428+0000 7fc75b3aa740 20 client.29192 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.428+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559b1c1374f0 0x559b1c1398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.428+0000 7fc75b3aa740 4 client.29192 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.428+0000 7fc75b3aa740 1 -- 172.21.15.18:0/11094994 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b1c0238b0 con 0x559b1c1374f0 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.429+0000 7fc755200640 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559b1c1374f0 0x559b1c1398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc755200640 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559b1c1374f0 0x559b1c1398e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c82cac13b8fb15f4 server_cookie=71d474491eecb51b in_seq=0 out_seq=0 2024-04-16T17:17:50.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc737400640 10 client.29192 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:17:50.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc737400640 1 -- 172.21.15.18:0/11094994 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b1c0238b0 con 0x559b1c1374f0 2024-04-16T17:17:50.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc737400640 10 client.29192 handle_command_reply: tid=0 2024-04-16T17:17:50.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc75b3aa740 1 client.29192 shutdown 2024-04-16T17:17:50.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc75b3aa740 1 -- 172.21.15.18:0/11094994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0b6f10 msgr2=0x559b1c0c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0b6f10 0x559b1c0c4030 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc744009340 tx=0x7fc74402bc40 comp rx=0 tx=0).stop 2024-04-16T17:17:50.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc75b3aa740 1 -- 172.21.15.18:0/11094994 shutdown_connections 2024-04-16T17:17:50.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.430+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559b1c1374f0 0x559b1c1398e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.431+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b1c0c0410 0x559b1c0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.431+0000 7fc737400640 0 client.29192 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:17:50.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.431+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559b1c0b7830 0x559b1c0c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.431+0000 7fc75b3aa740 1 --2- 172.21.15.18:0/11094994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b1c0b6f10 0x559b1c0c4030 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.431+0000 7fc75b3aa740 1 -- 172.21.15.18:0/11094994 >> 172.21.15.18:0/11094994 conn(0x559b1c0b1640 msgr2=0x559b1c0b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.432+0000 7fc75b3aa740 1 -- 172.21.15.18:0/11094994 shutdown_connections 2024-04-16T17:17:50.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.432+0000 7fc75b3aa740 1 -- 172.21.15.18:0/11094994 wait complete. 2024-04-16T17:17:50.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.432+0000 7fc75b3aa740 20 client.29192 trim_cache size 0 max 16384 2024-04-16T17:17:50.432 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:17:50.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.433+0000 7fc757000640 1 -- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7380a3d20 msgr2=0x7fc7380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.433+0000 7fc757000640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7380a3d20 0x7fc7380a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc748005fd0 tx=0x7fc748009880 comp rx=0 tx=0).stop 2024-04-16T17:17:50.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.433+0000 7fc757000640 1 -- 172.21.15.18:0/1462539978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750107140 msgr2=0x7fc75010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.433+0000 7fc757000640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750107140 0x7fc75010e6b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00e9c0 tx=0x7fc73c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:17:50.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.434+0000 7fc757000640 1 -- 172.21.15.18:0/1462539978 shutdown_connections 2024-04-16T17:17:50.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.434+0000 7fc757000640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7380a3d20 0x7fc7380a61e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.434+0000 7fc757000640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7501a3640 0x7fc7501a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.434+0000 7fc757000640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc750107140 0x7fc75010e6b0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.434+0000 7fc757000640 1 --2- 172.21.15.18:0/1462539978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc750102070 0x7fc75010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.434+0000 7fc757000640 1 -- 172.21.15.18:0/1462539978 >> 172.21.15.18:0/1462539978 conn(0x7fc7500fd950 msgr2=0x7fc7500ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.434+0000 7fc757000640 1 -- 172.21.15.18:0/1462539978 shutdown_connections 2024-04-16T17:17:50.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.434+0000 7fc757000640 1 -- 172.21.15.18:0/1462539978 wait complete. 2024-04-16T17:17:50.452 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:17:50.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.725+0000 7f3b45800640 1 -- 172.21.15.18:0/2814576171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4011b6a0 msgr2=0x7f3b4011da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.725+0000 7f3b45800640 1 --2- 172.21.15.18:0/2814576171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4011b6a0 0x7f3b4011da90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3b3801acb0 tx=0x7f3b3803d130 comp rx=0 tx=0).stop 2024-04-16T17:17:50.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.726+0000 7f3b45800640 1 -- 172.21.15.18:0/2814576171 shutdown_connections 2024-04-16T17:17:50.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.726+0000 7f3b45800640 1 --2- 172.21.15.18:0/2814576171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4011b6a0 0x7f3b4011da90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.726+0000 7f3b45800640 1 --2- 172.21.15.18:0/2814576171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4006e3e0 0x7f3b4011b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.726+0000 7f3b45800640 1 --2- 172.21.15.18:0/2814576171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4006dac0 0x7f3b4006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.726+0000 7f3b45800640 1 -- 172.21.15.18:0/2814576171 >> 172.21.15.18:0/2814576171 conn(0x7f3b4006d1b0 msgr2=0x7f3b4010b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.729+0000 7f3b45800640 1 -- 172.21.15.18:0/2814576171 shutdown_connections 2024-04-16T17:17:50.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.729+0000 7f3b45800640 1 -- 172.21.15.18:0/2814576171 wait complete. 2024-04-16T17:17:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.730+0000 7f3b45800640 1 Processor -- start 2024-04-16T17:17:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.730+0000 7f3b45800640 1 -- start start 2024-04-16T17:17:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.730+0000 7f3b45800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4006dac0 0x7f3b4011add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.730+0000 7f3b45800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4006e3e0 0x7f3b40116eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.730+0000 7f3b45800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4011b6a0 0x7f3b401173f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.730+0000 7f3b45800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010f300 con 0x7f3b4011b6a0 2024-04-16T17:17:50.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.730+0000 7f3b45800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010f180 con 0x7f3b4006e3e0 2024-04-16T17:17:50.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.730+0000 7f3b45800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010f480 con 0x7f3b4006dac0 2024-04-16T17:17:50.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4006e3e0 0x7f3b40116eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4006dac0 0x7f3b4011add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4006dac0 0x7f3b4011add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39148/0 (socket says 172.21.15.18:39148) 2024-04-16T17:17:50.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3ea00640 1 -- 172.21.15.18:0/4240989690 learned_addr learned my addr 172.21.15.18:0/4240989690 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4006e3e0 0x7f3b40116eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37392/0 (socket says 172.21.15.18:37392) 2024-04-16T17:17:50.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3f400640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4011b6a0 0x7f3b401173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3ea00640 1 -- 172.21.15.18:0/4240989690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4006e3e0 msgr2=0x7f3b40116eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3ea00640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4006e3e0 0x7f3b40116eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3ea00640 1 -- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4011b6a0 msgr2=0x7f3b401173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3ea00640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4011b6a0 0x7f3b401173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.731+0000 7f3b3ea00640 1 -- 172.21.15.18:0/4240989690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b40117b20 con 0x7f3b4006dac0 2024-04-16T17:17:50.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.732+0000 7f3b3ea00640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4006dac0 0x7f3b4011add0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00e9f0 tx=0x7f3b2c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.732+0000 7f3b37400640 1 -- 172.21.15.18:0/4240989690 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00ccc0 con 0x7f3b4006dac0 2024-04-16T17:17:50.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.732+0000 7f3b37400640 1 -- 172.21.15.18:0/4240989690 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c00ce60 con 0x7f3b4006dac0 2024-04-16T17:17:50.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.732+0000 7f3b37400640 1 -- 172.21.15.18:0/4240989690 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c010690 con 0x7f3b4006dac0 2024-04-16T17:17:50.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.733+0000 7f3b45800640 1 -- 172.21.15.18:0/4240989690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b401ab570 con 0x7f3b4006dac0 2024-04-16T17:17:50.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.733+0000 7f3b45800640 1 -- 172.21.15.18:0/4240989690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b401ab980 con 0x7f3b4006dac0 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.740+0000 7f3b48563740 1 Processor -- start 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.740+0000 7f3b48563740 1 -- start start 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.740+0000 7f3b48563740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac6946130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.740+0000 7f3b48563740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac694ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.741+0000 7f3b48563740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac694f250 0x563ac6951640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.741+0000 7f3b48563740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563ac68ab020 con 0x563ac6945d50 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.741+0000 7f3b48563740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563ac6851460 con 0x563ac6946670 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.741+0000 7f3b48563740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ac68a6fe0 con 0x563ac694f250 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.741+0000 7f3b37400640 1 -- 172.21.15.18:0/4240989690 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3b2c0046c0 con 0x7f3b4006dac0 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.741+0000 7f3b3f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac6946130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.741+0000 7f3b3f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac6946130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47534/0 (socket says 172.21.15.18:47534) 2024-04-16T17:17:50.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.741+0000 7f3b3f400640 1 -- 172.21.15.18:0/1159727329 learned_addr learned my addr 172.21.15.18:0/1159727329 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.743+0000 7f3b3f400640 1 -- 172.21.15.18:0/1159727329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac694f250 msgr2=0x563ac6951640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.743+0000 7f3b3f400640 1 --2- 172.21.15.18:0/1159727329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac694f250 0x563ac6951640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.743+0000 7f3b3f400640 1 -- 172.21.15.18:0/1159727329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 msgr2=0x563ac694ed10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:17:50.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.743+0000 7f3b3f400640 1 --2- 172.21.15.18:0/1159727329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac694ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.743+0000 7f3b3f400640 1 -- 172.21.15.18:0/1159727329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ac6859040 con 0x563ac6945d50 2024-04-16T17:17:50.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.743+0000 7f3b3f400640 1 --2- 172.21.15.18:0/1159727329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac6946130 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3b38048040 tx=0x7f3b38043b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fa343dcb109949 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.743+0000 7f3b37400640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3b240a3e20 0x7f3b240a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.744+0000 7f3b37400640 1 -- 172.21.15.18:0/4240989690 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f3b2c0d1010 con 0x7f3b4006dac0 2024-04-16T17:17:50.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.744+0000 7f3b2be00640 1 -- 172.21.15.18:0/1159727329 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3804a050 con 0x563ac6945d50 2024-04-16T17:17:50.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.744+0000 7f3b2be00640 1 -- 172.21.15.18:0/1159727329 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b38056070 con 0x563ac6945d50 2024-04-16T17:17:50.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.744+0000 7f3b2be00640 1 -- 172.21.15.18:0/1159727329 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380042a0 con 0x563ac6945d50 2024-04-16T17:17:50.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.746+0000 7f3b48563740 1 -- 172.21.15.18:0/1159727329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 msgr2=0x563ac6946130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.746+0000 7f3b48563740 1 --2- 172.21.15.18:0/1159727329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac6946130 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3b38048040 tx=0x7f3b38043b40 comp rx=0 tx=0).stop 2024-04-16T17:17:50.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.746+0000 7f3b3ea00640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3b240a3e20 0x7f3b240a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.749+0000 7f3b3ea00640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3b240a3e20 0x7f3b240a62e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x563ac6945820 tx=0x7f3b2c005720 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.750+0000 7f3b48563740 1 -- 172.21.15.18:0/1159727329 shutdown_connections 2024-04-16T17:17:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.750+0000 7f3b48563740 1 --2- 172.21.15.18:0/1159727329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac694f250 0x563ac6951640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.750+0000 7f3b48563740 1 --2- 172.21.15.18:0/1159727329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac694ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.750+0000 7f3b48563740 1 --2- 172.21.15.18:0/1159727329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac6946130 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.750+0000 7f3b48563740 1 -- 172.21.15.18:0/1159727329 >> 172.21.15.18:0/1159727329 conn(0x563ac6940480 msgr2=0x563ac6940860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.751+0000 7f3b48563740 1 -- 172.21.15.18:0/1159727329 shutdown_connections 2024-04-16T17:17:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.751+0000 7f3b48563740 1 -- 172.21.15.18:0/1159727329 wait complete. 2024-04-16T17:17:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.752+0000 7f3b48563740 1 -- start start 2024-04-16T17:17:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.752+0000 7f3b48563740 1 Processor -- start 2024-04-16T17:17:50.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.753+0000 7f3b48563740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.753+0000 7f3b48563740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac694c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.753+0000 7f3b3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.753+0000 7f3b3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39166/0 (socket says 172.21.15.18:39166) 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.754+0000 7f3b3f400640 1 -- 172.21.15.18:0/2811923020 learned_addr learned my addr 172.21.15.18:0/2811923020 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.754+0000 7f3b3e000640 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac694c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.754+0000 7f3b48563740 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 0x563ac694cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.754+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563ac6951fe0 con 0x563ac694f250 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.754+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563ac6864830 con 0x563ac6946670 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.754+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ac6952160 con 0x563ac6945d50 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.754+0000 7f3b48563740 10 client.0 _ll_register_callbacks cb 0x563ac686c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.755+0000 7f3b3e000640 1 -- 172.21.15.18:0/2811923020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 msgr2=0x563ac694c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.755+0000 7f3b3e000640 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.755+0000 7f3b3e000640 1 -- 172.21.15.18:0/2811923020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 msgr2=0x563ac694cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.755+0000 7f3b3e000640 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 0x563ac694cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.755+0000 7f3b3e000640 1 -- 172.21.15.18:0/2811923020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ac6868920 con 0x563ac6946670 2024-04-16T17:17:50.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.755+0000 7f3b3e000640 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac694c890 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x563ac6941610 tx=0x7f3b3000e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f80b0305580fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.755+0000 7f3b28c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:50.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.756+0000 7f3b28c00640 1 -- 172.21.15.18:0/2811923020 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30017070 con 0x563ac6946670 2024-04-16T17:17:50.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.756+0000 7f3b28c00640 1 -- 172.21.15.18:0/2811923020 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f3b30013680 con 0x563ac6946670 2024-04-16T17:17:50.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.756+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b38056070 con 0x563ac6946670 2024-04-16T17:17:50.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.757+0000 7f3b28c00640 1 -- 172.21.15.18:0/2811923020 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3b30023050 con 0x563ac6946670 2024-04-16T17:17:50.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.757+0000 7f3b48563740 10 client.29201 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:50.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.757+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ac68367f0 con 0x563ac6946670 2024-04-16T17:17:50.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.759+0000 7f3b28c00640 1 -- 172.21.15.18:0/2811923020 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b38056070 con 0x563ac6946670 2024-04-16T17:17:50.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.759+0000 7f3b28c00640 1 -- 172.21.15.18:0/2811923020 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3b30012920 con 0x563ac6946670 2024-04-16T17:17:50.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.760+0000 7f3b48563740 10 client.29201 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:50.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.760+0000 7f3b48563740 10 client.29201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:50.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.760+0000 7f3b48563740 20 client.29201 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:50.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.760+0000 7f3b48563740 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ac695f630 0x563ac6961a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.761+0000 7f3b48563740 4 client.29201 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:50.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.761+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ac68b8a40 con 0x563ac695f630 2024-04-16T17:17:50.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.761+0000 7f3b3f400640 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ac695f630 0x563ac6961a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.764+0000 7f3b3f400640 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ac695f630 0x563ac6961a50 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3003018b6ebb73 server_cookie=a27b8a992fa50f15 in_seq=0 out_seq=0 2024-04-16T17:17:50.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.764+0000 7f3b28c00640 10 client.29201 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:50.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.764+0000 7f3b28c00640 1 -- 172.21.15.18:0/2811923020 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ac68b8a40 con 0x563ac695f630 2024-04-16T17:17:50.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.764+0000 7f3b28c00640 10 client.29201 handle_command_reply: tid=0 2024-04-16T17:17:50.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.765+0000 7f3b48563740 1 client.29201 shutdown 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.766+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 msgr2=0x563ac694c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.766+0000 7f3b48563740 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac694c890 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x563ac6941610 tx=0x7f3b3000e8e0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.766+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 shutdown_connections 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.766+0000 7f3b48563740 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ac695f630 0x563ac6961a50 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.766+0000 7f3b48563740 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 0x563ac694cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.766+0000 7f3b48563740 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac694c890 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.766+0000 7f3b28c00640 0 client.29201 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.766+0000 7f3b48563740 1 --2- 172.21.15.18:0/2811923020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.767+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 >> 172.21.15.18:0/2811923020 conn(0x563ac6940480 msgr2=0x563ac69421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.767+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 shutdown_connections 2024-04-16T17:17:50.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.767+0000 7f3b48563740 1 -- 172.21.15.18:0/2811923020 wait complete. 2024-04-16T17:17:50.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.767+0000 7f3b48563740 20 client.29201 trim_cache size 0 max 16384 2024-04-16T17:17:50.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.795+0000 7f3b48563740 1 Processor -- start 2024-04-16T17:17:50.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.795+0000 7f3b48563740 1 -- start start 2024-04-16T17:17:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.795+0000 7f3b48563740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.795+0000 7f3b48563740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac695aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.795+0000 7f3b48563740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 0x563ac69672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.795+0000 7f3b48563740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401b8590 con 0x563ac694f250 2024-04-16T17:17:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.796+0000 7f3b48563740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401b5e60 con 0x563ac6946670 2024-04-16T17:17:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.796+0000 7f3b48563740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401b3cd0 con 0x563ac6945d50 2024-04-16T17:17:50.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.796+0000 7f3b3e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac695aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.796+0000 7f3b3e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac695aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37422/0 (socket says 172.21.15.18:37422) 2024-04-16T17:17:50.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.796+0000 7f3b3e000640 1 -- 172.21.15.18:0/2631727493 learned_addr learned my addr 172.21.15.18:0/2631727493 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.796+0000 7f3b3f400640 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 0x563ac69672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.796+0000 7f3b3f400640 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.797+0000 7f3b3e000640 1 -- 172.21.15.18:0/2631727493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 msgr2=0x563ac694ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.797+0000 7f3b3e000640 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.797+0000 7f3b3e000640 1 -- 172.21.15.18:0/2631727493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 msgr2=0x563ac69672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.797+0000 7f3b3e000640 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 0x563ac69672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.797+0000 7f3b3e000640 1 -- 172.21.15.18:0/2631727493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ac65bd3f0 con 0x563ac6946670 2024-04-16T17:17:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.797+0000 7f3b3f400640 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.798+0000 7f3b3e000640 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac695aea0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3b30019000 tx=0x7f3b30001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a68d0af2546e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.798+0000 7f3b2b400640 1 -- 172.21.15.18:0/2631727493 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30020070 con 0x563ac6946670 2024-04-16T17:17:50.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.798+0000 7f3b2b400640 1 -- 172.21.15.18:0/2631727493 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30012c40 con 0x563ac6946670 2024-04-16T17:17:50.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.799+0000 7f3b48563740 1 -- 172.21.15.18:0/2631727493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 msgr2=0x563ac695aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.799+0000 7f3b48563740 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac695aea0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3b30019000 tx=0x7f3b30001e50 comp rx=0 tx=0).stop 2024-04-16T17:17:50.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.799+0000 7f3b48563740 1 -- 172.21.15.18:0/2631727493 shutdown_connections 2024-04-16T17:17:50.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.799+0000 7f3b48563740 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac694f250 0x563ac69672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.799+0000 7f3b48563740 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac6946670 0x563ac695aea0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.799+0000 7f3b48563740 1 --2- 172.21.15.18:0/2631727493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6945d50 0x563ac694ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.799+0000 7f3b48563740 1 -- 172.21.15.18:0/2631727493 >> 172.21.15.18:0/2631727493 conn(0x563ac6940480 msgr2=0x563ac694b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.800+0000 7f3b48563740 1 -- 172.21.15.18:0/2631727493 shutdown_connections 2024-04-16T17:17:50.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.800+0000 7f3b48563740 1 -- 172.21.15.18:0/2631727493 wait complete. 2024-04-16T17:17:50.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.800+0000 7f3b48563740 1 -- start start 2024-04-16T17:17:50.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.800+0000 7f3b48563740 1 Processor -- start 2024-04-16T17:17:50.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.801+0000 7f3b48563740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac69575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.801+0000 7f3b48563740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6946670 0x563ac6952970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.801+0000 7f3b48563740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac694f250 0x563ac6952eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.801+0000 7f3b48563740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401b4320 con 0x563ac6945d50 2024-04-16T17:17:50.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.801+0000 7f3b48563740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401b5790 con 0x563ac694f250 2024-04-16T17:17:50.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.801+0000 7f3b48563740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401b49f0 con 0x563ac6946670 2024-04-16T17:17:50.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.801+0000 7f3b48563740 10 client.0 _ll_register_callbacks cb 0x7f3b300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:50.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6946670 0x563ac6952970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6946670 0x563ac6952970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39184/0 (socket says 172.21.15.18:39184) 2024-04-16T17:17:50.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3e000640 1 -- 172.21.15.18:0/4174090221 learned_addr learned my addr 172.21.15.18:0/4174090221 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:50.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3f400640 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac694f250 0x563ac6952eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3f400640 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac69575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3e000640 1 -- 172.21.15.18:0/4174090221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac694f250 msgr2=0x563ac6952eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3e000640 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac694f250 0x563ac6952eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3e000640 1 -- 172.21.15.18:0/4174090221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 msgr2=0x563ac69575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3e000640 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac69575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.802+0000 7f3b3e000640 1 -- 172.21.15.18:0/4174090221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ac66877d0 con 0x563ac6946670 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.803+0000 7f3b3e000640 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6946670 0x563ac6952970 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3b300091f0 tx=0x7f3b3000e3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=976b6276b86fc243 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.803+0000 7f3b28c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.803+0000 7f3b28c00640 1 -- 172.21.15.18:0/4174090221 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30020040 con 0x563ac6946670 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.803+0000 7f3b28c00640 1 -- 172.21.15.18:0/4174090221 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f3b30025d20 con 0x563ac6946670 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.804+0000 7f3b48563740 1 -- 172.21.15.18:0/4174090221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b30012c40 con 0x563ac6946670 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.804+0000 7f3b28c00640 1 -- 172.21.15.18:0/4174090221 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3b30024e80 con 0x563ac6946670 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.804+0000 7f3b48563740 10 client.29038 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.804+0000 7f3b48563740 1 -- 172.21.15.18:0/4174090221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ac69c57c0 con 0x563ac6946670 2024-04-16T17:17:50.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.804+0000 7f3b28c00640 1 -- 172.21.15.18:0/4174090221 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b30012c40 con 0x563ac6946670 2024-04-16T17:17:50.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.805+0000 7f3b28c00640 1 -- 172.21.15.18:0/4174090221 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3b30012920 con 0x563ac6946670 2024-04-16T17:17:50.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.805+0000 7f3b48563740 10 client.29038 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:50.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.805+0000 7f3b48563740 10 client.29038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:50.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.805+0000 7f3b48563740 20 client.29038 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:50.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.805+0000 7f3b48563740 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ac69c5e20 0x563ac69c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:50.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.805+0000 7f3b48563740 4 client.29038 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:50.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.805+0000 7f3b48563740 1 -- 172.21.15.18:0/4174090221 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ac68b28b0 con 0x563ac69c5e20 2024-04-16T17:17:50.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.805+0000 7f3b3f400640 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ac69c5e20 0x563ac69c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:50.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.806+0000 7f3b28c00640 10 client.29038 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:50.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.806+0000 7f3b3f400640 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ac69c5e20 0x563ac69c8240 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e8da7ffb1f5643 server_cookie=65705c97d1ddfa96 in_seq=0 out_seq=0 2024-04-16T17:17:50.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b28c00640 1 -- 172.21.15.18:0/4174090221 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ac68b28b0 con 0x563ac69c5e20 2024-04-16T17:17:50.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b28c00640 10 client.29038 handle_command_reply: tid=0 2024-04-16T17:17:50.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 client.29038 shutdown 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 -- 172.21.15.18:0/4174090221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6946670 msgr2=0x563ac6952970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6946670 0x563ac6952970 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3b300091f0 tx=0x7f3b3000e3e0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 -- 172.21.15.18:0/4174090221 shutdown_connections 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ac69c5e20 0x563ac69c8240 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ac694f250 0x563ac6952eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ac6946670 0x563ac6952970 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 --2- 172.21.15.18:0/4174090221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ac6945d50 0x563ac69575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 -- 172.21.15.18:0/4174090221 >> 172.21.15.18:0/4174090221 conn(0x563ac6940480 msgr2=0x563ac693ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b28c00640 0 client.29038 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.811+0000 7f3b48563740 1 -- 172.21.15.18:0/4174090221 shutdown_connections 2024-04-16T17:17:50.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.812+0000 7f3b48563740 1 -- 172.21.15.18:0/4174090221 wait complete. 2024-04-16T17:17:50.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.812+0000 7f3b48563740 20 client.29038 trim_cache size 0 max 16384 2024-04-16T17:17:50.812 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:17:50.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.812+0000 7f3b45800640 1 -- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3b240a3e20 msgr2=0x7f3b240a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.812+0000 7f3b45800640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3b240a3e20 0x7f3b240a62e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x563ac6945820 tx=0x7f3b2c005720 comp rx=0 tx=0).stop 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.812+0000 7f3b45800640 1 -- 172.21.15.18:0/4240989690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4006dac0 msgr2=0x7f3b4011add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.812+0000 7f3b45800640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4006dac0 0x7f3b4011add0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00e9f0 tx=0x7f3b2c00edf0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.813+0000 7f3b45800640 1 -- 172.21.15.18:0/4240989690 shutdown_connections 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.813+0000 7f3b45800640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3b240a3e20 0x7f3b240a62e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.813+0000 7f3b45800640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b4011b6a0 0x7f3b401173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.813+0000 7f3b45800640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4006e3e0 0x7f3b40116eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.813+0000 7f3b45800640 1 --2- 172.21.15.18:0/4240989690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4006dac0 0x7f3b4011add0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.813+0000 7f3b45800640 1 -- 172.21.15.18:0/4240989690 >> 172.21.15.18:0/4240989690 conn(0x7f3b4006d1b0 msgr2=0x7f3b4011c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.813+0000 7f3b45800640 1 -- 172.21.15.18:0/4240989690 shutdown_connections 2024-04-16T17:17:50.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:50.813+0000 7f3b45800640 1 -- 172.21.15.18:0/4240989690 wait complete. 2024-04-16T17:17:50.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:50 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2599147398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:51.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:50 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2599147398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:51.136 INFO:tasks.workunit.client.0.smithi018.stdout: 10 744 69.25 MB/sec warmup 3 sec latency 204.900 ms 2024-04-16T17:17:51.605 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:49 smithi018 ceph-mon[31776]: pgmap v414: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 887 KiB/s wr, 66 op/s 2024-04-16T17:17:51.605 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:50 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2599147398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:51.835 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:17:51.836 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:17:52.322 INFO:tasks.workunit.client.0.smithi018.stdout: 10 884 50.52 MB/sec warmup 4 sec latency 847.246 ms 2024-04-16T17:17:52.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.659+0000 7fc07a600640 1 -- 172.21.15.18:0/4256534552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074103900 msgr2=0x7fc07410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.659+0000 7fc07a600640 1 --2- 172.21.15.18:0/4256534552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074103900 0x7fc07410bde0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc060009900 tx=0x7fc06002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.659+0000 7fc07a600640 1 -- 172.21.15.18:0/4256534552 shutdown_connections 2024-04-16T17:17:52.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.659+0000 7fc07a600640 1 --2- 172.21.15.18:0/4256534552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07410c400 0x7fc07410e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.659+0000 7fc07a600640 1 --2- 172.21.15.18:0/4256534552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074103900 0x7fc07410bde0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.659+0000 7fc07a600640 1 --2- 172.21.15.18:0/4256534552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc074102fe0 0x7fc0741033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.659+0000 7fc07a600640 1 -- 172.21.15.18:0/4256534552 >> 172.21.15.18:0/4256534552 conn(0x7fc074077ab0 msgr2=0x7fc0740fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:52.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.660+0000 7fc07a600640 1 -- 172.21.15.18:0/4256534552 shutdown_connections 2024-04-16T17:17:52.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.660+0000 7fc07a600640 1 -- 172.21.15.18:0/4256534552 wait complete. 2024-04-16T17:17:52.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.661+0000 7fc07a600640 1 Processor -- start 2024-04-16T17:17:52.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.661+0000 7fc07a600640 1 -- start start 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc07a600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074102fe0 0x7fc0741a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc07a600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc074103900 0x7fc0741a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc07a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07410c400 0x7fc07419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc07a600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc074112280 con 0x7fc074103900 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc07a600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc074112100 con 0x7fc074102fe0 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc07a600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc074112400 con 0x7fc07410c400 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc079200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074102fe0 0x7fc0741a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc073e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc074103900 0x7fc0741a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc079200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074102fe0 0x7fc0741a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37448/0 (socket says 172.21.15.18:37448) 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc079200640 1 -- 172.21.15.18:0/1140287091 learned_addr learned my addr 172.21.15.18:0/1140287091 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:52.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.662+0000 7fc079c00640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07410c400 0x7fc07419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.663+0000 7fc079c00640 1 -- 172.21.15.18:0/1140287091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074102fe0 msgr2=0x7fc0741a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.663+0000 7fc079c00640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074102fe0 0x7fc0741a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.663+0000 7fc079c00640 1 -- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc074103900 msgr2=0x7fc0741a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.663+0000 7fc079c00640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc074103900 0x7fc0741a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.663+0000 7fc079c00640 1 -- 172.21.15.18:0/1140287091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07419afd0 con 0x7fc07410c400 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.663+0000 7fc079c00640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07410c400 0x7fc07419a800 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009fd0 tx=0x7fc06c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.663+0000 7fc071600640 1 -- 172.21.15.18:0/1140287091 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00ce20 con 0x7fc07410c400 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.664+0000 7fc07a600640 1 -- 172.21.15.18:0/1140287091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07419b2c0 con 0x7fc07410c400 2024-04-16T17:17:52.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.664+0000 7fc071600640 1 -- 172.21.15.18:0/1140287091 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c005020 con 0x7fc07410c400 2024-04-16T17:17:52.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.664+0000 7fc071600640 1 -- 172.21.15.18:0/1140287091 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c010720 con 0x7fc07410c400 2024-04-16T17:17:52.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.664+0000 7fc07a600640 1 -- 172.21.15.18:0/1140287091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07406ba20 con 0x7fc07410c400 2024-04-16T17:17:52.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.668+0000 7fc071600640 1 -- 172.21.15.18:0/1140287091 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc06c005540 con 0x7fc07410c400 2024-04-16T17:17:52.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.668+0000 7fc07e84e740 1 Processor -- start 2024-04-16T17:17:52.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.669+0000 7fc071600640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc05c0a3e20 0x7fc05c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.669+0000 7fc071600640 1 -- 172.21.15.18:0/1140287091 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fc06c0d1360 con 0x7fc07410c400 2024-04-16T17:17:52.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.670+0000 7fc07e84e740 1 -- start start 2024-04-16T17:17:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x556296143270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562961437b0 0x55629614be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x55629614e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc07e84e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5562960a7cb0 con 0x5562961437b0 2024-04-16T17:17:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc07e84e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55629604e370 con 0x556296142e90 2024-04-16T17:17:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc07e84e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5562960a3c70 con 0x55629614c390 2024-04-16T17:17:52.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc073e00640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc05c0a3e20 0x7fc05c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc079c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x55629614e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc079200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x556296143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc079c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x55629614e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39226/0 (socket says 172.21.15.18:39226) 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc079c00640 1 -- 172.21.15.18:0/1959748294 learned_addr learned my addr 172.21.15.18:0/1959748294 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.671+0000 7fc073e00640 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562961437b0 0x55629614be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.672+0000 7fc079c00640 1 -- 172.21.15.18:0/1959748294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 msgr2=0x556296143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.672+0000 7fc079c00640 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x556296143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.672+0000 7fc079c00640 1 -- 172.21.15.18:0/1959748294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562961437b0 msgr2=0x55629614be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.672+0000 7fc079c00640 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562961437b0 0x55629614be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.672+0000 7fc079c00640 1 -- 172.21.15.18:0/1959748294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629606d910 con 0x55629614c390 2024-04-16T17:17:52.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.672+0000 7fc073e00640 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562961437b0 0x55629614be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:52.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:52 smithi022 ceph-mon[36023]: pgmap v415: 129 pgs: 129 active+clean; 56 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-04-16T17:17:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.673+0000 7fc073e00640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc05c0a3e20 0x7fc05c0a62e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc060006010 tx=0x7fc060004120 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.673+0000 7fc079c00640 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x55629614e780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc06c005fd0 tx=0x7fc06c09cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=686930d2826587b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.674+0000 7fc066000640 1 -- 172.21.15.18:0/1959748294 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b6070 con 0x55629614c390 2024-04-16T17:17:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.674+0000 7fc066000640 1 -- 172.21.15.18:0/1959748294 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c09b190 con 0x55629614c390 2024-04-16T17:17:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.674+0000 7fc066000640 1 -- 172.21.15.18:0/1959748294 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0ce8c0 con 0x55629614c390 2024-04-16T17:17:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.675+0000 7fc07e84e740 1 -- 172.21.15.18:0/1959748294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 msgr2=0x55629614e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.675+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x55629614e780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc06c005fd0 tx=0x7fc06c09cf50 comp rx=0 tx=0).stop 2024-04-16T17:17:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.675+0000 7fc07e84e740 1 -- 172.21.15.18:0/1959748294 shutdown_connections 2024-04-16T17:17:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.675+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x55629614e780 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.675+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562961437b0 0x55629614be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.675+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1959748294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x556296143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.676+0000 7fc07e84e740 1 -- 172.21.15.18:0/1959748294 >> 172.21.15.18:0/1959748294 conn(0x55629613d5c0 msgr2=0x55629613d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:52.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.677+0000 7fc07e84e740 1 -- 172.21.15.18:0/1959748294 shutdown_connections 2024-04-16T17:17:52.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.677+0000 7fc07e84e740 1 -- 172.21.15.18:0/1959748294 wait complete. 2024-04-16T17:17:52.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.678+0000 7fc07e84e740 1 -- start start 2024-04-16T17:17:52.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.678+0000 7fc07e84e740 1 Processor -- start 2024-04-16T17:17:52.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.678+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.678+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562961437b0 0x556296149a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296149f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc07e84e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55629614f180 con 0x556296142e90 2024-04-16T17:17:52.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc07e84e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556296061730 con 0x5562961437b0 2024-04-16T17:17:52.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc07e84e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55629614f300 con 0x55629614c390 2024-04-16T17:17:52.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc07e84e740 10 client.0 _ll_register_callbacks cb 0x5562960301e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:52.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc073e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562961437b0 0x556296149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc073e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562961437b0 0x556296149a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37462/0 (socket says 172.21.15.18:37462) 2024-04-16T17:17:52.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc073e00640 1 -- 172.21.15.18:0/1283821696 learned_addr learned my addr 172.21.15.18:0/1283821696 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:52.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.679+0000 7fc079200640 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.680+0000 7fc073e00640 1 -- 172.21.15.18:0/1283821696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 msgr2=0x556296149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:52.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.680+0000 7fc073e00640 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.680+0000 7fc073e00640 1 -- 172.21.15.18:0/1283821696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 msgr2=0x5562961494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:52.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.680+0000 7fc073e00640 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.680+0000 7fc073e00640 1 -- 172.21.15.18:0/1283821696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556295e827e0 con 0x5562961437b0 2024-04-16T17:17:52.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.682+0000 7fc073e00640 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562961437b0 0x556296149a10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc060009900 tx=0x7fc06002bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=806ea9fcc0fba7c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:52.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.682+0000 7fc05aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:52.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.683+0000 7fc05aa00640 1 -- 172.21.15.18:0/1283821696 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060062060 con 0x5562961437b0 2024-04-16T17:17:52.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.683+0000 7fc05aa00640 1 -- 172.21.15.18:0/1283821696 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fc06004a040 con 0x5562961437b0 2024-04-16T17:17:52.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.683+0000 7fc07e84e740 1 -- 172.21.15.18:0/1283821696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc06c09b190 con 0x5562961437b0 2024-04-16T17:17:52.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.683+0000 7fc05aa00640 1 -- 172.21.15.18:0/1283821696 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc060005310 con 0x5562961437b0 2024-04-16T17:17:52.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.683+0000 7fc07e84e740 10 client.29219 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:52.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.683+0000 7fc07e84e740 1 -- 172.21.15.18:0/1283821696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556295ee2f70 con 0x5562961437b0 2024-04-16T17:17:52.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc05aa00640 1 -- 172.21.15.18:0/1283821696 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c09b190 con 0x5562961437b0 2024-04-16T17:17:52.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc05aa00640 1 -- 172.21.15.18:0/1283821696 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc060050040 con 0x5562961437b0 2024-04-16T17:17:52.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc07e84e740 10 client.29219 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:52.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc07e84e740 10 client.29219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:52.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc07e84e740 20 client.29219 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:52.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55629615c520 0x55629615e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc07e84e740 4 client.29219 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:52.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc07e84e740 1 -- 172.21.15.18:0/1283821696 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562960b56d0 con 0x55629615c520 2024-04-16T17:17:52.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.684+0000 7fc079200640 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55629615c520 0x55629615e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.685+0000 7fc079200640 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55629615c520 0x55629615e9c0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b88aa6b04b6b3e2 server_cookie=3931cb9c03966d0f in_seq=0 out_seq=0 2024-04-16T17:17:52.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.685+0000 7fc05aa00640 10 client.29219 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.689+0000 7fc05aa00640 1 -- 172.21.15.18:0/1283821696 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562960b56d0 con 0x55629615c520 2024-04-16T17:17:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.689+0000 7fc05aa00640 10 client.29219 handle_command_reply: tid=0 2024-04-16T17:17:52.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.689+0000 7fc07e84e740 1 client.29219 shutdown 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.690+0000 7fc07e84e740 1 -- 172.21.15.18:0/1283821696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562961437b0 msgr2=0x556296149a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.690+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562961437b0 0x556296149a10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc060009900 tx=0x7fc06002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.690+0000 7fc07e84e740 1 -- 172.21.15.18:0/1283821696 shutdown_connections 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.690+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55629615c520 0x55629615e9c0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.690+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.690+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562961437b0 0x556296149a10 secure :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc060009900 tx=0x7fc06002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.690+0000 7fc07e84e740 1 --2- 172.21.15.18:0/1283821696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.691+0000 7fc05aa00640 0 client.29219 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:52.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.691+0000 7fc07e84e740 1 -- 172.21.15.18:0/1283821696 >> 172.21.15.18:0/1283821696 conn(0x55629613d5c0 msgr2=0x55629613f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.691+0000 7fc07e84e740 1 -- 172.21.15.18:0/1283821696 shutdown_connections 2024-04-16T17:17:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.691+0000 7fc07e84e740 1 -- 172.21.15.18:0/1283821696 wait complete. 2024-04-16T17:17:52.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.692+0000 7fc07e84e740 20 client.29219 trim_cache size 0 max 16384 2024-04-16T17:17:52.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.725+0000 7fc07e84e740 1 Processor -- start 2024-04-16T17:17:52.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.726+0000 7fc07e84e740 1 -- start start 2024-04-16T17:17:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.727+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x55629614bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.727+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.727+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55629613f100 0x556296164290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.728+0000 7fc079200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x55629614bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.728+0000 7fc079200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x55629614bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37468/0 (socket says 172.21.15.18:37468) 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.728+0000 7fc073e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.728+0000 7fc073e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39244/0 (socket says 172.21.15.18:39244) 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.728+0000 7fc079c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55629613f100 0x556296164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.728+0000 7fc079c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55629613f100 0x556296164290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47614/0 (socket says 172.21.15.18:47614) 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.728+0000 7fc079200640 1 -- 172.21.15.18:0/3058413597 learned_addr learned my addr 172.21.15.18:0/3058413597 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.728+0000 7fc07e84e740 1 -- 172.21.15.18:0/3058413597 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741a74e0 con 0x55629613f100 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.729+0000 7fc07e84e740 1 -- 172.21.15.18:0/3058413597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc074071a50 con 0x556296142e90 2024-04-16T17:17:52.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.729+0000 7fc07e84e740 1 -- 172.21.15.18:0/3058413597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc07406f8e0 con 0x55629614c390 2024-04-16T17:17:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.729+0000 7fc073e00640 1 -- 172.21.15.18:0/3058413597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 msgr2=0x55629614bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.729+0000 7fc073e00640 1 --2- 172.21.15.18:0/3058413597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x55629614bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.729+0000 7fc073e00640 1 -- 172.21.15.18:0/3058413597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55629613f100 msgr2=0x556296164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.729+0000 7fc073e00640 1 --2- 172.21.15.18:0/3058413597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55629613f100 0x556296164290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.730+0000 7fc073e00640 1 -- 172.21.15.18:0/3058413597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629614eaf0 con 0x55629614c390 2024-04-16T17:17:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.730+0000 7fc079200640 1 --2- 172.21.15.18:0/3058413597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x55629614bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:52.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.730+0000 7fc073e00640 1 --2- 172.21.15.18:0/3058413597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296158040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55629614a560 tx=0x7fc060009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab57e5e0020ccfb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:52.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.730+0000 7fc065600640 1 -- 172.21.15.18:0/3058413597 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060062040 con 0x55629614c390 2024-04-16T17:17:52.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.730+0000 7fc065600640 1 -- 172.21.15.18:0/3058413597 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc060003f00 con 0x55629614c390 2024-04-16T17:17:52.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.730+0000 7fc065600640 1 -- 172.21.15.18:0/3058413597 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06006c080 con 0x55629614c390 2024-04-16T17:17:52.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.731+0000 7fc07e84e740 1 -- 172.21.15.18:0/3058413597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 msgr2=0x556296158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.731+0000 7fc07e84e740 1 --2- 172.21.15.18:0/3058413597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296158040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55629614a560 tx=0x7fc060009800 comp rx=0 tx=0).stop 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.732+0000 7fc07e84e740 1 -- 172.21.15.18:0/3058413597 shutdown_connections 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.732+0000 7fc07e84e740 1 --2- 172.21.15.18:0/3058413597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55629613f100 0x556296164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.732+0000 7fc07e84e740 1 --2- 172.21.15.18:0/3058413597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x556296158040 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.732+0000 7fc07e84e740 1 --2- 172.21.15.18:0/3058413597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556296142e90 0x55629614bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.732+0000 7fc07e84e740 1 -- 172.21.15.18:0/3058413597 >> 172.21.15.18:0/3058413597 conn(0x55629613d5c0 msgr2=0x55629613e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.732+0000 7fc07e84e740 1 -- 172.21.15.18:0/3058413597 shutdown_connections 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.732+0000 7fc07e84e740 1 -- 172.21.15.18:0/3058413597 wait complete. 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.733+0000 7fc07e84e740 1 -- start start 2024-04-16T17:17:52.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.733+0000 7fc07e84e740 1 Processor -- start 2024-04-16T17:17:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.734+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629613f100 0x5562961bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.734+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.734+0000 7fc07e84e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x5562961508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc079200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629613f100 0x5562961bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc079200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629613f100 0x5562961bae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37482/0 (socket says 172.21.15.18:37482) 2024-04-16T17:17:52.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc073e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc073e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961bb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47630/0 (socket says 172.21.15.18:47630) 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc07e84e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc07406ff30 con 0x556296142e90 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc079200640 1 -- 172.21.15.18:0/2875918115 learned_addr learned my addr 172.21.15.18:0/2875918115 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc074071380 con 0x55629613f100 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0740705e0 con 0x55629614c390 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc07e84e740 10 client.0 _ll_register_callbacks cb 0x7fc060004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.735+0000 7fc079c00640 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x5562961508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.736+0000 7fc079c00640 1 -- 172.21.15.18:0/2875918115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629613f100 msgr2=0x5562961bae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.736+0000 7fc079c00640 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629613f100 0x5562961bae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.736+0000 7fc079c00640 1 -- 172.21.15.18:0/2875918115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 msgr2=0x5562961bb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:52.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.736+0000 7fc079c00640 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961bb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.736+0000 7fc079c00640 1 -- 172.21.15.18:0/2875918115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556295d7a170 con 0x55629614c390 2024-04-16T17:17:52.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.736+0000 7fc079200640 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629613f100 0x5562961bae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:52.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.737+0000 7fc079c00640 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x5562961508a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00c760 tx=0x7fc06c005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9e22761b5b1763 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:52.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.737+0000 7fc05aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:52.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.738+0000 7fc05aa00640 1 -- 172.21.15.18:0/2875918115 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b6040 con 0x55629614c390 2024-04-16T17:17:52.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.738+0000 7fc05aa00640 1 -- 172.21.15.18:0/2875918115 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7fc06c0b61e0 con 0x55629614c390 2024-04-16T17:17:52.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.739+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc060003f00 con 0x55629614c390 2024-04-16T17:17:52.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.739+0000 7fc05aa00640 1 -- 172.21.15.18:0/2875918115 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc06c099130 con 0x55629614c390 2024-04-16T17:17:52.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.740+0000 7fc07e84e740 10 client.29053 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:52.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.740+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5562961c3550 con 0x55629614c390 2024-04-16T17:17:52.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.740+0000 7fc05aa00640 1 -- 172.21.15.18:0/2875918115 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc060003f00 con 0x55629614c390 2024-04-16T17:17:52.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.740+0000 7fc05aa00640 1 -- 172.21.15.18:0/2875918115 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc06c09c370 con 0x55629614c390 2024-04-16T17:17:52.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.741+0000 7fc07e84e740 10 client.29053 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:52.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.741+0000 7fc07e84e740 10 client.29053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:52.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.741+0000 7fc07e84e740 20 client.29053 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:52.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.741+0000 7fc07e84e740 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562961c3780 0x5562961c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:52.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.741+0000 7fc07e84e740 4 client.29053 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:17:52.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.741+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5562960af540 con 0x5562961c3780 2024-04-16T17:17:52.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.742+0000 7fc079200640 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562961c3780 0x5562961c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:52.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.743+0000 7fc079200640 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562961c3780 0x5562961c5b70 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6624f647f0b7b9d5 server_cookie=c8757f31921076eb in_seq=0 out_seq=0 2024-04-16T17:17:52.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.743+0000 7fc05aa00640 10 client.29053 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:52.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.748+0000 7fc05aa00640 1 -- 172.21.15.18:0/2875918115 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5562960af540 con 0x5562961c3780 2024-04-16T17:17:52.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.748+0000 7fc05aa00640 10 client.29053 handle_command_reply: tid=0 2024-04-16T17:17:52.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.748+0000 7fc07e84e740 1 client.29053 shutdown 2024-04-16T17:17:52.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.748+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 msgr2=0x5562961508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.748+0000 7fc07e84e740 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x5562961508a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00c760 tx=0x7fc06c005890 comp rx=0 tx=0).stop 2024-04-16T17:17:52.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.750+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 shutdown_connections 2024-04-16T17:17:52.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.750+0000 7fc07e84e740 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562961c3780 0x5562961c5b70 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.750+0000 7fc07e84e740 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55629614c390 0x5562961508a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.750+0000 7fc07e84e740 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556296142e90 0x5562961bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.750+0000 7fc07e84e740 1 --2- 172.21.15.18:0/2875918115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629613f100 0x5562961bae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.750+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 >> 172.21.15.18:0/2875918115 conn(0x55629613d5c0 msgr2=0x556296149490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:52.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.750+0000 7fc05aa00640 0 client.29053 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:52.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:52 smithi043 ceph-mon[36901]: pgmap v415: 129 pgs: 129 active+clean; 56 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-04-16T17:17:52.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.752+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 shutdown_connections 2024-04-16T17:17:52.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.752+0000 7fc07e84e740 1 -- 172.21.15.18:0/2875918115 wait complete. 2024-04-16T17:17:52.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.752+0000 7fc07e84e740 20 client.29053 trim_cache size 0 max 16384 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "22001623-f413-4128-9ec8-c15042e9e617", 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.754+0000 7fc07a600640 1 -- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc05c0a3e20 msgr2=0x7fc05c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.754+0000 7fc07a600640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc05c0a3e20 0x7fc05c0a62e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc060006010 tx=0x7fc060004120 comp rx=0 tx=0).stop 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.754+0000 7fc07a600640 1 -- 172.21.15.18:0/1140287091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07410c400 msgr2=0x7fc07419a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:52.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.754+0000 7fc07a600640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07410c400 0x7fc07419a800 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009fd0 tx=0x7fc06c00ef10 comp rx=0 tx=0).stop 2024-04-16T17:17:52.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.755+0000 7fc07a600640 1 -- 172.21.15.18:0/1140287091 shutdown_connections 2024-04-16T17:17:52.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.755+0000 7fc07a600640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc05c0a3e20 0x7fc05c0a62e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.755+0000 7fc07a600640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07410c400 0x7fc07419a800 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.755+0000 7fc07a600640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc074103900 0x7fc0741a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.755+0000 7fc07a600640 1 --2- 172.21.15.18:0/1140287091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074102fe0 0x7fc0741a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:52.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.755+0000 7fc07a600640 1 -- 172.21.15.18:0/1140287091 >> 172.21.15.18:0/1140287091 conn(0x7fc074077ab0 msgr2=0x7fc074100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:52.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.756+0000 7fc07a600640 1 -- 172.21.15.18:0/1140287091 shutdown_connections 2024-04-16T17:17:52.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:52.756+0000 7fc07a600640 1 -- 172.21.15.18:0/1140287091 wait complete. 2024-04-16T17:17:52.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:17:52.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:17:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 -- 172.21.15.18:0/4257893774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 msgr2=0x7f14a0104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 --2- 172.21.15.18:0/4257893774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 0x7f14a0104660 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1490009860 tx=0x7f149002bb70 comp rx=0 tx=0).stop 2024-04-16T17:17:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 -- 172.21.15.18:0/4257893774 shutdown_connections 2024-04-16T17:17:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 --2- 172.21.15.18:0/4257893774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a0109350 0x7f14a0110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 --2- 172.21.15.18:0/4257893774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a0104c30 0x7f14a0108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 --2- 172.21.15.18:0/4257893774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 0x7f14a0104660 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 -- 172.21.15.18:0/4257893774 >> 172.21.15.18:0/4257893774 conn(0x7f14a00ffae0 msgr2=0x7f14a0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 -- 172.21.15.18:0/4257893774 shutdown_connections 2024-04-16T17:17:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.004+0000 7f14a5800640 1 -- 172.21.15.18:0/4257893774 wait complete. 2024-04-16T17:17:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.005+0000 7f14a5800640 1 Processor -- start 2024-04-16T17:17:53.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.005+0000 7f14a5800640 1 -- start start 2024-04-16T17:17:53.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 0x7f14a01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a5800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a0109350 0x7f14a01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a019a780 0x7f14a019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a5800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01143e0 con 0x7f14a0109350 2024-04-16T17:17:53.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a5800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14a0114260 con 0x7f14a019a780 2024-04-16T17:17:53.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a5800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14a0114560 con 0x7f14a0104280 2024-04-16T17:17:53.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f149fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 0x7f14a01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a019a780 0x7f14a019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a019a780 0x7f14a019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37502/0 (socket says 172.21.15.18:37502) 2024-04-16T17:17:53.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f14a4e00640 1 -- 172.21.15.18:0/4049747949 learned_addr learned my addr 172.21.15.18:0/4049747949 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:53.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.006+0000 7f149f400640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a0109350 0x7f14a01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.009+0000 7f14a4e00640 1 -- 172.21.15.18:0/4049747949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 msgr2=0x7f14a01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.009+0000 7f14a4e00640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 0x7f14a01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.009+0000 7f14a4e00640 1 -- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a0109350 msgr2=0x7f14a01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.009+0000 7f14a4e00640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a0109350 0x7f14a01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.009+0000 7f14a4e00640 1 -- 172.21.15.18:0/4049747949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a019b310 con 0x7f14a019a780 2024-04-16T17:17:53.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.009+0000 7f14a4e00640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a019a780 0x7f14a019ac30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1498009fd0 tx=0x7f149800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:53.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.009+0000 7f149cc00640 1 -- 172.21.15.18:0/4049747949 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498017070 con 0x7f14a019a780 2024-04-16T17:17:53.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.010+0000 7f149cc00640 1 -- 172.21.15.18:0/4049747949 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1498003e50 con 0x7f14a019a780 2024-04-16T17:17:53.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.010+0000 7f149cc00640 1 -- 172.21.15.18:0/4049747949 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980106c0 con 0x7f14a019a780 2024-04-16T17:17:53.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.011+0000 7f149f400640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a0109350 0x7f14a01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:53.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.011+0000 7f14a5800640 1 -- 172.21.15.18:0/4049747949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a019b600 con 0x7f14a019a780 2024-04-16T17:17:53.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.011+0000 7f14a5800640 1 -- 172.21.15.18:0/4049747949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a01a7790 con 0x7f14a019a780 2024-04-16T17:17:53.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.011+0000 7f149fe00640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 0x7f14a01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:53.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.016+0000 7f14a9b9b740 1 Processor -- start 2024-04-16T17:17:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.019+0000 7f149cc00640 1 -- 172.21.15.18:0/4049747949 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1498003970 con 0x7f14a019a780 2024-04-16T17:17:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.020+0000 7f14a9b9b740 1 -- start start 2024-04-16T17:17:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.020+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280968150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.020+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x557280979b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.020+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x55728097c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.020+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5572808ce120 con 0x557280968720 2024-04-16T17:17:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.020+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557280874340 con 0x557280967d70 2024-04-16T17:17:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.020+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5572808ca0e0 con 0x55728097a0a0 2024-04-16T17:17:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x557280979b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x557280979b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47658/0 (socket says 172.21.15.18:47658) 2024-04-16T17:17:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280968150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149fe00640 1 -- 172.21.15.18:0/788644150 learned_addr learned my addr 172.21.15.18:0/788644150 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149f400640 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x55728097c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149f400640 1 -- 172.21.15.18:0/788644150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 msgr2=0x55728097c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149f400640 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x55728097c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149f400640 1 -- 172.21.15.18:0/788644150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 msgr2=0x557280979b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149cc00640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14840a3da0 0x7f14840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149f400640 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x557280979b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149f400640 1 -- 172.21.15.18:0/788644150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572806aa7d0 con 0x557280967d70 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.021+0000 7f149fe00640 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x557280979b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.022+0000 7f149cc00640 1 -- 172.21.15.18:0/4049747949 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f14980d0560 con 0x7f14a019a780 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.022+0000 7f149f400640 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280968150 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f14a010a8e0 tx=0x7f148c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6001f1763f20d8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.022+0000 7f149f400640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14840a3da0 0x7f14840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.022+0000 7f1495600640 1 -- 172.21.15.18:0/788644150 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c004230 con 0x557280967d70 2024-04-16T17:17:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.022+0000 7f1495600640 1 -- 172.21.15.18:0/788644150 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f148c0043d0 con 0x557280967d70 2024-04-16T17:17:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.022+0000 7f1495600640 1 -- 172.21.15.18:0/788644150 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c010c80 con 0x557280967d70 2024-04-16T17:17:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f14a9b9b740 1 -- 172.21.15.18:0/788644150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 msgr2=0x557280968150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280968150 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f14a010a8e0 tx=0x7f148c009790 comp rx=0 tx=0).stop 2024-04-16T17:17:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f149f400640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14840a3da0 0x7f14840a6260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f148c011010 tx=0x7f148c010f00 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f14a9b9b740 1 -- 172.21.15.18:0/788644150 shutdown_connections 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x55728097c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x557280979b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/788644150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280968150 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f14a9b9b740 1 -- 172.21.15.18:0/788644150 >> 172.21.15.18:0/788644150 conn(0x557280962ac0 msgr2=0x557280962ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.023+0000 7f14a9b9b740 1 -- 172.21.15.18:0/788644150 shutdown_connections 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.024+0000 7f14a9b9b740 1 -- 172.21.15.18:0/788644150 wait complete. 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.024+0000 7f14a9b9b740 1 -- start start 2024-04-16T17:17:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.024+0000 7f14a9b9b740 1 Processor -- start 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.024+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280977080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55728097a0a0 0x557280977b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55728097ce60 con 0x55728097a0a0 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557280887710 con 0x557280967d70 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55728097cfe0 con 0x557280968720 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f14a9b9b740 10 client.0 _ll_register_callbacks cb 0x557280882af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809775c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39286/0 (socket says 172.21.15.18:39286) 2024-04-16T17:17:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149f400640 1 -- 172.21.15.18:0/3750192725 learned_addr learned my addr 172.21.15.18:0/3750192725 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f14a4e00640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55728097a0a0 0x557280977b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149fe00640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280977080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149f400640 1 -- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 msgr2=0x557280977080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149f400640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280977080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149f400640 1 -- 172.21.15.18:0/3750192725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55728097a0a0 msgr2=0x557280977b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149f400640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55728097a0a0 0x557280977b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149f400640 1 -- 172.21.15.18:0/3750192725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572805e03f0 con 0x557280968720 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f14a4e00640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55728097a0a0 0x557280977b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.025+0000 7f149fe00640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280977080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.026+0000 7f149f400640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809775c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f148c01e780 tx=0x7f148c00f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54fdd42474ef5dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.026+0000 7f148a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.026+0000 7f148a000640 1 -- 172.21.15.18:0/3750192725 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c010550 con 0x557280968720 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.026+0000 7f148a000640 1 -- 172.21.15.18:0/3750192725 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f148c010b90 con 0x557280968720 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.026+0000 7f14a9b9b740 1 -- 172.21.15.18:0/3750192725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f148c0043d0 con 0x557280968720 2024-04-16T17:17:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.027+0000 7f148a000640 1 -- 172.21.15.18:0/3750192725 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f148c00e900 con 0x557280968720 2024-04-16T17:17:53.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.027+0000 7f14a9b9b740 10 client.29065 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:53.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.027+0000 7f14a9b9b740 1 -- 172.21.15.18:0/3750192725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5572805a00f0 con 0x557280968720 2024-04-16T17:17:53.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.027+0000 7f148a000640 1 -- 172.21.15.18:0/3750192725 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f148c0043d0 con 0x557280968720 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.028+0000 7f148a000640 1 -- 172.21.15.18:0/3750192725 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f148c01d030 con 0x557280968720 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.028+0000 7f14a9b9b740 10 client.29065 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.028+0000 7f14a9b9b740 10 client.29065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.028+0000 7f14a9b9b740 20 client.29065 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.028+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55728098a2a0 0x55728098c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.028+0000 7f14a9b9b740 4 client.29065 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.028+0000 7f14a9b9b740 1 -- 172.21.15.18:0/3750192725 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572808dbb40 con 0x55728098a2a0 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.029+0000 7f149fe00640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55728098a2a0 0x55728098c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.029+0000 7f149fe00640 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55728098a2a0 0x55728098c690 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb0e6bf9137b50e server_cookie=d47eb94345728278 in_seq=0 out_seq=0 2024-04-16T17:17:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.029+0000 7f148a000640 10 client.29065 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:53.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.031+0000 7f148a000640 1 -- 172.21.15.18:0/3750192725 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572808dbb40 con 0x55728098a2a0 2024-04-16T17:17:53.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.031+0000 7f148a000640 10 client.29065 handle_command_reply: tid=0 2024-04-16T17:17:53.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.031+0000 7f14a9b9b740 1 client.29065 shutdown 2024-04-16T17:17:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 -- 172.21.15.18:0/3750192725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 msgr2=0x5572809775c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809775c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f148c01e780 tx=0x7f148c00f000 comp rx=0 tx=0).stop 2024-04-16T17:17:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 -- 172.21.15.18:0/3750192725 shutdown_connections 2024-04-16T17:17:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55728098a2a0 0x55728098c690 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55728097a0a0 0x557280977b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809775c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/3750192725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280977080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 -- 172.21.15.18:0/3750192725 >> 172.21.15.18:0/3750192725 conn(0x557280962ac0 msgr2=0x557280964810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:53.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f148a000640 0 client.29065 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:53.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 -- 172.21.15.18:0/3750192725 shutdown_connections 2024-04-16T17:17:53.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 1 -- 172.21.15.18:0/3750192725 wait complete. 2024-04-16T17:17:53.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.032+0000 7f14a9b9b740 20 client.29065 trim_cache size 0 max 16384 2024-04-16T17:17:53.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.061+0000 7f14a9b9b740 1 Processor -- start 2024-04-16T17:17:53.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.061+0000 7f14a9b9b740 1 -- start start 2024-04-16T17:17:53.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280985d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x5572809619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x557280991e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01afc00 con 0x557280968720 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01ad4d0 con 0x557280967d70 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01ab360 con 0x55728097a0a0 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x557280991e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f149fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280985d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x557280991e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39292/0 (socket says 172.21.15.18:39292) 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f149fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280985d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37524/0 (socket says 172.21.15.18:37524) 2024-04-16T17:17:53.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a4e00640 1 -- 172.21.15.18:0/2099932752 learned_addr learned my addr 172.21.15.18:0/2099932752 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a4e00640 1 -- 172.21.15.18:0/2099932752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 msgr2=0x557280985d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a4e00640 1 --2- 172.21.15.18:0/2099932752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280985d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.062+0000 7f14a4e00640 1 -- 172.21.15.18:0/2099932752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 msgr2=0x5572809619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:53.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.063+0000 7f14a4e00640 1 --2- 172.21.15.18:0/2099932752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x5572809619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.063+0000 7f14a4e00640 1 -- 172.21.15.18:0/2099932752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557280971e70 con 0x55728097a0a0 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.063+0000 7f14a4e00640 1 --2- 172.21.15.18:0/2099932752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x557280991e20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f149809c020 tx=0x7f149809bb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22a732ff93a43f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.063+0000 7f1494c00640 1 -- 172.21.15.18:0/2099932752 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980b5070 con 0x55728097a0a0 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.063+0000 7f1494c00640 1 -- 172.21.15.18:0/2099932752 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14980a0440 con 0x55728097a0a0 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.063+0000 7f1494c00640 1 -- 172.21.15.18:0/2099932752 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498020040 con 0x55728097a0a0 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 -- 172.21.15.18:0/2099932752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 msgr2=0x557280991e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/2099932752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x557280991e20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f149809c020 tx=0x7f149809bb00 comp rx=0 tx=0).stop 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 -- 172.21.15.18:0/2099932752 shutdown_connections 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/2099932752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55728097a0a0 0x557280991e20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/2099932752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280968720 0x5572809619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/2099932752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557280967d70 0x557280985d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 -- 172.21.15.18:0/2099932752 >> 172.21.15.18:0/2099932752 conn(0x557280962ac0 msgr2=0x557280961290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 -- 172.21.15.18:0/2099932752 shutdown_connections 2024-04-16T17:17:53.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.064+0000 7f14a9b9b740 1 -- 172.21.15.18:0/2099932752 wait complete. 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.065+0000 7f14a9b9b740 1 -- start start 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.065+0000 7f14a9b9b740 1 Processor -- start 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.065+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280967d70 0x5572809822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.065+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.065+0000 7f14a9b9b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55728097a0a0 0x55728097f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.065+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01ab9b0 con 0x557280967d70 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.065+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01ace00 con 0x55728097a0a0 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a9b9b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01ac060 con 0x557280968720 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a9b9b740 10 client.0 _ll_register_callbacks cb 0x7f148c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55728097a0a0 0x55728097f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55728097a0a0 0x55728097f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37536/0 (socket says 172.21.15.18:37536) 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a4e00640 1 -- 172.21.15.18:0/229371843 learned_addr learned my addr 172.21.15.18:0/229371843 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f149f400640 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a4e00640 1 -- 172.21.15.18:0/229371843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 msgr2=0x5572809827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a4e00640 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a4e00640 1 -- 172.21.15.18:0/229371843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280967d70 msgr2=0x5572809822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a4e00640 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280967d70 0x5572809822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f14a4e00640 1 -- 172.21.15.18:0/229371843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728097f9f0 con 0x55728097a0a0 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.066+0000 7f149f400640 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.067+0000 7f14a4e00640 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55728097a0a0 0x55728097f270 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f149809bd90 tx=0x7f1498005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8487adcdbd2ab963 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.067+0000 7f148a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:17:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.067+0000 7f148a000640 1 -- 172.21.15.18:0/229371843 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498020040 con 0x55728097a0a0 2024-04-16T17:17:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.067+0000 7f148a000640 1 -- 172.21.15.18:0/229371843 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7f14980d0150 con 0x55728097a0a0 2024-04-16T17:17:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.067+0000 7f14a9b9b740 1 -- 172.21.15.18:0/229371843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14980a0440 con 0x55728097a0a0 2024-04-16T17:17:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.068+0000 7f148a000640 1 -- 172.21.15.18:0/229371843 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f14980aa050 con 0x55728097a0a0 2024-04-16T17:17:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.069+0000 7f14a9b9b740 10 client.29240 fetch_fsmap learned FSMap version 33 2024-04-16T17:17:53.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.069+0000 7f14a9b9b740 1 -- 172.21.15.18:0/229371843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557280a02140 con 0x55728097a0a0 2024-04-16T17:17:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.071+0000 7f148a000640 1 -- 172.21.15.18:0/229371843 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14980a0440 con 0x55728097a0a0 2024-04-16T17:17:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.071+0000 7f148a000640 1 -- 172.21.15.18:0/229371843 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1498023060 con 0x55728097a0a0 2024-04-16T17:17:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.071+0000 7f14a9b9b740 10 client.29240 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:17:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.071+0000 7f14a9b9b740 10 client.29240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.071+0000 7f14a9b9b740 20 client.29240 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:53.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.071+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557280a02830 0x557280a04c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:53.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.071+0000 7f14a9b9b740 4 client.29240 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:17:53.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.071+0000 7f14a9b9b740 1 -- 172.21.15.18:0/229371843 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572808d59b0 con 0x557280a02830 2024-04-16T17:17:53.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.072+0000 7f149fe00640 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557280a02830 0x557280a04c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:53.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.073+0000 7f148a000640 10 client.29240 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:53.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.073+0000 7f149fe00640 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557280a02830 0x557280a04c50 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882ea7423d15764f server_cookie=74b3b8b1b37a050 in_seq=0 out_seq=0 2024-04-16T17:17:53.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f148a000640 1 -- 172.21.15.18:0/229371843 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5572808d59b0 con 0x557280a02830 2024-04-16T17:17:53.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f148a000640 10 client.29240 handle_command_reply: tid=0 2024-04-16T17:17:53.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 client.29240 shutdown 2024-04-16T17:17:53.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 -- 172.21.15.18:0/229371843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55728097a0a0 msgr2=0x55728097f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55728097a0a0 0x55728097f270 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f149809bd90 tx=0x7f1498005ea0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 -- 172.21.15.18:0/229371843 shutdown_connections 2024-04-16T17:17:53.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557280a02830 0x557280a04c50 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55728097a0a0 0x55728097f270 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557280968720 0x5572809827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 --2- 172.21.15.18:0/229371843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557280967d70 0x5572809822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f14a9b9b740 1 -- 172.21.15.18:0/229371843 >> 172.21.15.18:0/229371843 conn(0x557280962ac0 msgr2=0x557280961b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:53.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.077+0000 7f148a000640 0 client.29240 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:53.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.078+0000 7f14a9b9b740 1 -- 172.21.15.18:0/229371843 shutdown_connections 2024-04-16T17:17:53.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.078+0000 7f14a9b9b740 1 -- 172.21.15.18:0/229371843 wait complete. 2024-04-16T17:17:53.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.078+0000 7f14a9b9b740 20 client.29240 trim_cache size 0 max 16384 2024-04-16T17:17:53.078 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:17:53.078 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (331 inodes in the stack)", 2024-04-16T17:17:53.078 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:17:53.078 INFO:teuthology.orchestra.run.smithi018.stdout: "22001623-f413-4128-9ec8-c15042e9e617": { 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "22001623-f413-4128-9ec8-c15042e9e617", 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.079+0000 7f14a5800640 1 -- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14840a3da0 msgr2=0x7f14840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.079+0000 7f14a5800640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14840a3da0 0x7f14840a6260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f148c011010 tx=0x7f148c010f00 comp rx=0 tx=0).stop 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.079+0000 7f14a5800640 1 -- 172.21.15.18:0/4049747949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a019a780 msgr2=0x7f14a019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:53.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.079+0000 7f14a5800640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a019a780 0x7f14a019ac30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1498009fd0 tx=0x7f149800b260 comp rx=0 tx=0).stop 2024-04-16T17:17:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.080+0000 7f14a5800640 1 -- 172.21.15.18:0/4049747949 shutdown_connections 2024-04-16T17:17:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.080+0000 7f14a5800640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14840a3da0 0x7f14840a6260 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.080+0000 7f14a5800640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a019a780 0x7f14a019ac30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.080+0000 7f14a5800640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a0109350 0x7f14a01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.080+0000 7f14a5800640 1 --2- 172.21.15.18:0/4049747949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14a0104280 0x7f14a01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.080+0000 7f14a5800640 1 -- 172.21.15.18:0/4049747949 >> 172.21.15.18:0/4049747949 conn(0x7f14a00ffae0 msgr2=0x7f14a0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.080+0000 7f14a5800640 1 -- 172.21.15.18:0/4049747949 shutdown_connections 2024-04-16T17:17:53.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:17:53.081+0000 7f14a5800640 1 -- 172.21.15.18:0/4049747949 wait complete. 2024-04-16T17:17:53.097 INFO:tasks.cephfs.filesystem:scrub status for tag:22001623-f413-4128-9ec8-c15042e9e617 - {'path': '/', 'tag': '22001623-f413-4128-9ec8-c15042e9e617', 'options': 'recursive,force'} 2024-04-16T17:17:53.320 INFO:tasks.workunit.client.0.smithi018.stdout: 10 1148 43.35 MB/sec warmup 5 sec latency 1017.899 ms 2024-04-16T17:17:53.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:53 smithi022 ceph-mon[36023]: pgmap v416: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-04-16T17:17:53.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:53 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:17:53.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:53 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:53.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:53 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:17:53.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:53 smithi043 ceph-mon[36901]: pgmap v416: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-04-16T17:17:53.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:53 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:17:53.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:53 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:53.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:53 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:17:54.320 INFO:tasks.workunit.client.0.smithi018.stdout: 10 1666 41.73 MB/sec warmup 6 sec latency 205.223 ms 2024-04-16T17:17:54.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:54 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:54.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:54 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:17:54.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:54 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:54.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:54 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:17:55.320 INFO:tasks.workunit.client.0.smithi018.stdout: 10 2305 38.45 MB/sec warmup 7 sec latency 76.399 ms 2024-04-16T17:17:55.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:55 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:17:55.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:55 smithi022 ceph-mon[36023]: pgmap v417: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1024 KiB/s wr, 121 op/s 2024-04-16T17:17:55.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:55 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:17:55.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:55 smithi043 ceph-mon[36901]: pgmap v417: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1024 KiB/s wr, 121 op/s 2024-04-16T17:17:56.320 INFO:tasks.workunit.client.0.smithi018.stdout: 10 2874 40.62 MB/sec warmup 8 sec latency 168.368 ms 2024-04-16T17:17:56.742 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:52 smithi018 ceph-mon[31776]: pgmap v415: 129 pgs: 129 active+clean; 56 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-04-16T17:17:56.742 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:53 smithi018 ceph-mon[31776]: pgmap v416: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-04-16T17:17:56.742 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:53 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:17:56.742 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:53 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:56.742 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:53 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:17:56.742 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:54 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:17:56.743 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:54 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:17:56.743 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:55 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:17:56.743 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:55 smithi018 ceph-mon[31776]: pgmap v417: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1024 KiB/s wr, 121 op/s 2024-04-16T17:17:57.320 INFO:tasks.workunit.client.0.smithi018.stdout: 10 3292 39.52 MB/sec warmup 9 sec latency 288.400 ms 2024-04-16T17:17:57.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:57 smithi022 ceph-mon[36023]: pgmap v418: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-04-16T17:17:58.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:57 smithi043 ceph-mon[36901]: pgmap v418: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-04-16T17:17:58.320 INFO:tasks.workunit.client.0.smithi018.stdout: 10 3610 37.74 MB/sec warmup 10 sec latency 217.346 ms 2024-04-16T17:17:59.320 INFO:tasks.workunit.client.0.smithi018.stdout: 10 3809 34.67 MB/sec warmup 11 sec latency 214.803 ms 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 -- 172.21.15.18:0/3880683020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1fc10e880 msgr2=0x7ff1fc110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 --2- 172.21.15.18:0/3880683020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1fc10e880 0x7ff1fc110c90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009be0 tx=0x7ff1f402bf80 comp rx=0 tx=0).stop 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 -- 172.21.15.18:0/3880683020 shutdown_connections 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 --2- 172.21.15.18:0/3880683020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1fc10e880 0x7ff1fc110c90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 --2- 172.21.15.18:0/3880683020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1fc074cf0 0x7ff1fc10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 --2- 172.21.15.18:0/3880683020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1fc0743d0 0x7ff1fc0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 -- 172.21.15.18:0/3880683020 >> 172.21.15.18:0/3880683020 conn(0x7ff1fc0fdad0 msgr2=0x7ff1fc0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 -- 172.21.15.18:0/3880683020 shutdown_connections 2024-04-16T17:17:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.602+0000 7ff201800640 1 -- 172.21.15.18:0/3880683020 wait complete. 2024-04-16T17:17:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.603+0000 7ff201800640 1 Processor -- start 2024-04-16T17:17:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.604+0000 7ff201800640 1 -- start start 2024-04-16T17:17:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.604+0000 7ff201800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1fc0743d0 0x7ff1fc1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.604+0000 7ff201800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1fc074cf0 0x7ff1fc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.604+0000 7ff201800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1fc10e880 0x7ff1fc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.604+0000 7ff201800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc114610 con 0x7ff1fc0743d0 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.604+0000 7ff201800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc114490 con 0x7ff1fc074cf0 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.604+0000 7ff201800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc114790 con 0x7ff1fc10e880 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1fc074cf0 0x7ff1fc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1fc074cf0 0x7ff1fc1a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45838/0 (socket says 172.21.15.18:45838) 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 -- 172.21.15.18:0/852719824 learned_addr learned my addr 172.21.15.18:0/852719824 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fb400640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1fc10e880 0x7ff1fc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1faa00640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1fc0743d0 0x7ff1fc1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 -- 172.21.15.18:0/852719824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1fc10e880 msgr2=0x7ff1fc19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1fc10e880 0x7ff1fc19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 -- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1fc0743d0 msgr2=0x7ff1fc1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1fc0743d0 0x7ff1fc1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 -- 172.21.15.18:0/852719824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc19b150 con 0x7ff1fc074cf0 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1faa00640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1fc0743d0 0x7ff1fc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.605+0000 7ff1fa000640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1fc074cf0 0x7ff1fc1a0d10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9c0 tx=0x7ff1e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.607+0000 7ff1f3400640 1 -- 172.21.15.18:0/852719824 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800cde0 con 0x7ff1fc074cf0 2024-04-16T17:17:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.608+0000 7ff1f3400640 1 -- 172.21.15.18:0/852719824 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e80035f0 con 0x7ff1fc074cf0 2024-04-16T17:17:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.608+0000 7ff201800640 1 -- 172.21.15.18:0/852719824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc19b440 con 0x7ff1fc074cf0 2024-04-16T17:17:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.608+0000 7ff1f3400640 1 -- 172.21.15.18:0/852719824 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8010690 con 0x7ff1fc074cf0 2024-04-16T17:17:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.608+0000 7ff201800640 1 -- 172.21.15.18:0/852719824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc101ed0 con 0x7ff1fc074cf0 2024-04-16T17:17:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.611+0000 7ff204411740 1 Processor -- start 2024-04-16T17:17:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.611+0000 7ff204411740 1 -- start start 2024-04-16T17:17:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.611+0000 7ff204411740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d47360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.611+0000 7ff204411740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d478a0 0x556735d47d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.611+0000 7ff204411740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d53210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.611+0000 7ff204411740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556735cac780 con 0x556735d478a0 2024-04-16T17:17:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.611+0000 7ff204411740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556735c52f40 con 0x556735d48610 2024-04-16T17:17:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.611+0000 7ff204411740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556735ca8740 con 0x556735d50e20 2024-04-16T17:17:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d47360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d47360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45848/0 (socket says 172.21.15.18:45848) 2024-04-16T17:17:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1fb400640 1 -- 172.21.15.18:0/2776384540 learned_addr learned my addr 172.21.15.18:0/2776384540 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1faa00640 1 --2- 172.21.15.18:0/2776384540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d478a0 0x556735d47d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1fb400640 1 -- 172.21.15.18:0/2776384540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 msgr2=0x556735d53210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:17:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1fb400640 1 --2- 172.21.15.18:0/2776384540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d53210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1fb400640 1 -- 172.21.15.18:0/2776384540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d478a0 msgr2=0x556735d47d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1fb400640 1 --2- 172.21.15.18:0/2776384540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d478a0 0x556735d47d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.612+0000 7ff1fb400640 1 -- 172.21.15.18:0/2776384540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567359bf210 con 0x556735d48610 2024-04-16T17:17:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.613+0000 7ff1f3400640 1 -- 172.21.15.18:0/852719824 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1e8004020 con 0x7ff1fc074cf0 2024-04-16T17:17:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.613+0000 7ff1fb400640 1 --2- 172.21.15.18:0/2776384540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d47360 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc075320 tx=0x7ff1f4033470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df10acf783581b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.613+0000 7ff1e7e00640 1 -- 172.21.15.18:0/2776384540 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4043030 con 0x556735d48610 2024-04-16T17:17:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.613+0000 7ff1e7e00640 1 -- 172.21.15.18:0/2776384540 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f4003e50 con 0x556735d48610 2024-04-16T17:17:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.613+0000 7ff1e7e00640 1 -- 172.21.15.18:0/2776384540 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4004290 con 0x556735d48610 2024-04-16T17:17:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.614+0000 7ff204411740 1 -- 172.21.15.18:0/2776384540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 msgr2=0x556735d47360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.614+0000 7ff204411740 1 --2- 172.21.15.18:0/2776384540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d47360 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc075320 tx=0x7ff1f4033470 comp rx=0 tx=0).stop 2024-04-16T17:17:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.614+0000 7ff1f3400640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff1e00a3d90 0x7ff1e00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.614+0000 7ff1f3400640 1 -- 172.21.15.18:0/852719824 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7ff1e80d0f60 con 0x7ff1fc074cf0 2024-04-16T17:17:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.615+0000 7ff204411740 1 -- 172.21.15.18:0/2776384540 shutdown_connections 2024-04-16T17:17:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.615+0000 7ff204411740 1 --2- 172.21.15.18:0/2776384540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d53210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.615+0000 7ff204411740 1 --2- 172.21.15.18:0/2776384540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d478a0 0x556735d47d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.615+0000 7ff204411740 1 --2- 172.21.15.18:0/2776384540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d47360 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.615+0000 7ff204411740 1 -- 172.21.15.18:0/2776384540 >> 172.21.15.18:0/2776384540 conn(0x556735ce25d0 msgr2=0x556735d42590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.615+0000 7ff1fb400640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff1e00a3d90 0x7ff1e00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.616+0000 7ff204411740 1 -- 172.21.15.18:0/2776384540 shutdown_connections 2024-04-16T17:17:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.617+0000 7ff1fb400640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff1e00a3d90 0x7ff1e00a6250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff1f402ced0 tx=0x7ff1f4009940 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.617+0000 7ff204411740 1 -- 172.21.15.18:0/2776384540 wait complete. 2024-04-16T17:17:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.618+0000 7ff204411740 1 -- start start 2024-04-16T17:17:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.618+0000 7ff204411740 1 Processor -- start 2024-04-16T17:17:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff204411740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d478a0 0x556735d4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff204411740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d4e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff204411740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d4e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff204411740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556735d53af0 con 0x556735d48610 2024-04-16T17:17:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff204411740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556735c66390 con 0x556735d50e20 2024-04-16T17:17:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff204411740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556735d53c70 con 0x556735d478a0 2024-04-16T17:17:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff204411740 10 client.0 _ll_register_callbacks cb 0x556735c4ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff1faa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d478a0 0x556735d4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff1faa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d478a0 0x556735d4dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42616/0 (socket says 172.21.15.18:42616) 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.619+0000 7ff1faa00640 1 -- 172.21.15.18:0/704279927 learned_addr learned my addr 172.21.15.18:0/704279927 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1faa00640 1 -- 172.21.15.18:0/704279927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 msgr2=0x556735d4e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1fa000640 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d4e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1faa00640 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d4e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1faa00640 1 -- 172.21.15.18:0/704279927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 msgr2=0x556735d4e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1faa00640 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d4e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1faa00640 1 -- 172.21.15.18:0/704279927 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556735a8b620 con 0x556735d478a0 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1fa000640 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d4e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1faa00640 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d478a0 0x556735d4dcc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0126a0 tx=0x7ff1ec012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2449fe6a5a947129 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.620+0000 7ff1e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.621+0000 7ff1e4c00640 1 -- 172.21.15.18:0/704279927 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec004460 con 0x556735d478a0 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.621+0000 7ff1e4c00640 1 -- 172.21.15.18:0/704279927 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7ff1ec010950 con 0x556735d478a0 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.621+0000 7ff204411740 1 -- 172.21.15.18:0/704279927 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f4003e50 con 0x556735d478a0 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.621+0000 7ff1e4c00640 1 -- 172.21.15.18:0/704279927 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff1ec010b50 con 0x556735d478a0 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.621+0000 7ff204411740 10 client.29077 fetch_fsmap learned FSMap version 34 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.621+0000 7ff204411740 1 -- 172.21.15.18:0/704279927 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556735d5fca0 con 0x556735d478a0 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.622+0000 7ff1e4c00640 1 -- 172.21.15.18:0/704279927 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f4003e50 con 0x556735d478a0 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.622+0000 7ff1e4c00640 1 -- 172.21.15.18:0/704279927 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff1ec019070 con 0x556735d478a0 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.622+0000 7ff204411740 10 client.29077 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.622+0000 7ff204411740 10 client.29077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.622+0000 7ff204411740 20 client.29077 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.622+0000 7ff204411740 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556735d61160 0x556735d63570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.622+0000 7ff204411740 4 client.29077 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.622+0000 7ff204411740 1 -- 172.21.15.18:0/704279927 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556735cba1a0 con 0x556735d61160 2024-04-16T17:17:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.623+0000 7ff1fa000640 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556735d61160 0x556735d63570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.623+0000 7ff1fa000640 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556735d61160 0x556735d63570 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee17dbd388668bff server_cookie=e1029749de2e10af in_seq=0 out_seq=0 2024-04-16T17:17:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.623+0000 7ff1e4c00640 10 client.29077 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff1e4c00640 1 -- 172.21.15.18:0/704279927 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556735cba1a0 con 0x556735d61160 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff1e4c00640 10 client.29077 handle_command_reply: tid=0 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 client.29077 shutdown 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 -- 172.21.15.18:0/704279927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d478a0 msgr2=0x556735d4dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d478a0 0x556735d4dcc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0126a0 tx=0x7ff1ec012b60 comp rx=0 tx=0).stop 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 -- 172.21.15.18:0/704279927 shutdown_connections 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556735d61160 0x556735d63570 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d4e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d4e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 --2- 172.21.15.18:0/704279927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d478a0 0x556735d4dcc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 -- 172.21.15.18:0/704279927 >> 172.21.15.18:0/704279927 conn(0x556735ce25d0 msgr2=0x556735d442e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff1e4c00640 0 client.29077 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.627+0000 7ff204411740 1 -- 172.21.15.18:0/704279927 shutdown_connections 2024-04-16T17:17:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.628+0000 7ff204411740 1 -- 172.21.15.18:0/704279927 wait complete. 2024-04-16T17:17:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.628+0000 7ff204411740 20 client.29077 trim_cache size 0 max 16384 2024-04-16T17:17:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.651+0000 7ff204411740 1 Processor -- start 2024-04-16T17:17:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.651+0000 7ff204411740 1 -- start start 2024-04-16T17:17:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff204411740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff204411740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d5c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff204411740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d4e590 0x556735d4cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff204411740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b85a0 con 0x556735d48610 2024-04-16T17:17:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff204411740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b5e10 con 0x556735d50e20 2024-04-16T17:17:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff204411740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b3c80 con 0x556735d4e590 2024-04-16T17:17:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff1fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d4e590 0x556735d4cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff1fa000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d5c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff1fa000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d5c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45860/0 (socket says 172.21.15.18:45860) 2024-04-16T17:17:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff1fa000640 1 -- 172.21.15.18:0/3484716463 learned_addr learned my addr 172.21.15.18:0/3484716463 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.652+0000 7ff1faa00640 1 --2- 172.21.15.18:0/3484716463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.653+0000 7ff1fa000640 1 -- 172.21.15.18:0/3484716463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d4e590 msgr2=0x556735d4cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.653+0000 7ff1fa000640 1 --2- 172.21.15.18:0/3484716463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d4e590 0x556735d4cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.653+0000 7ff1fa000640 1 -- 172.21.15.18:0/3484716463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 msgr2=0x556735d503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.653+0000 7ff1fa000640 1 --2- 172.21.15.18:0/3484716463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.653+0000 7ff1fa000640 1 -- 172.21.15.18:0/3484716463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556735d46b70 con 0x556735d50e20 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.653+0000 7ff1fa000640 1 --2- 172.21.15.18:0/3484716463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d5c9f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff1e809ca20 tx=0x7ff1e800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1283ab6cbe9556 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.653+0000 7ff1e7400640 1 -- 172.21.15.18:0/3484716463 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80a5700 con 0x556735d50e20 2024-04-16T17:17:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.654+0000 7ff1e7400640 1 -- 172.21.15.18:0/3484716463 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e8004310 con 0x556735d50e20 2024-04-16T17:17:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.654+0000 7ff1e7400640 1 -- 172.21.15.18:0/3484716463 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80037a0 con 0x556735d50e20 2024-04-16T17:17:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 -- 172.21.15.18:0/3484716463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 msgr2=0x556735d5c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 --2- 172.21.15.18:0/3484716463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d5c9f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff1e809ca20 tx=0x7ff1e800c850 comp rx=0 tx=0).stop 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 -- 172.21.15.18:0/3484716463 shutdown_connections 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 --2- 172.21.15.18:0/3484716463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d4e590 0x556735d4cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 --2- 172.21.15.18:0/3484716463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d50e20 0x556735d5c9f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 --2- 172.21.15.18:0/3484716463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d48610 0x556735d503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 -- 172.21.15.18:0/3484716463 >> 172.21.15.18:0/3484716463 conn(0x556735ce25d0 msgr2=0x556735d4c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 -- 172.21.15.18:0/3484716463 shutdown_connections 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.655+0000 7ff204411740 1 -- 172.21.15.18:0/3484716463 wait complete. 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.656+0000 7ff204411740 1 -- start start 2024-04-16T17:17:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.656+0000 7ff204411740 1 Processor -- start 2024-04-16T17:17:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.657+0000 7ff204411740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.657+0000 7ff1faa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1faa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d59330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45868/0 (socket says 172.21.15.18:45868) 2024-04-16T17:17:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff204411740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d4e590 0x556735d4a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff204411740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d4abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff204411740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b42d0 con 0x556735d4e590 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff204411740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b5740 con 0x556735d48610 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff204411740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b49a0 con 0x556735d50e20 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff204411740 10 client.0 _ll_register_callbacks cb 0x7ff1ec009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d4abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d4abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42634/0 (socket says 172.21.15.18:42634) 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fb400640 1 -- 172.21.15.18:0/3112502145 learned_addr learned my addr 172.21.15.18:0/3112502145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fa000640 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d4e590 0x556735d4a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fb400640 1 -- 172.21.15.18:0/3112502145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 msgr2=0x556735d59330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fb400640 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d59330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fb400640 1 -- 172.21.15.18:0/3112502145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d4e590 msgr2=0x556735d4a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fb400640 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d4e590 0x556735d4a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fb400640 1 -- 172.21.15.18:0/3112502145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556735d4b3c0 con 0x556735d50e20 2024-04-16T17:17:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.658+0000 7ff1fa000640 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d4e590 0x556735d4a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.659+0000 7ff1fb400640 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d4abe0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40041a0 tx=0x7ff1f4042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f5421d2fba217f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.659+0000 7ff1e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.659+0000 7ff1e4c00640 1 -- 172.21.15.18:0/3112502145 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4058070 con 0x556735d50e20 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.659+0000 7ff1e4c00640 1 -- 172.21.15.18:0/3112502145 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9554+0+0 (secure 0 0 0) 0x7ff1f4004450 con 0x556735d50e20 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.660+0000 7ff204411740 1 -- 172.21.15.18:0/3112502145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e8004310 con 0x556735d50e20 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.660+0000 7ff1e4c00640 1 -- 172.21.15.18:0/3112502145 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff1f4062050 con 0x556735d50e20 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.660+0000 7ff204411740 10 client.29083 fetch_fsmap learned FSMap version 34 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.660+0000 7ff204411740 1 -- 172.21.15.18:0/3112502145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556735dc7710 con 0x556735d50e20 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.660+0000 7ff1e4c00640 1 -- 172.21.15.18:0/3112502145 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e8004310 con 0x556735d50e20 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.661+0000 7ff1e4c00640 1 -- 172.21.15.18:0/3112502145 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff1f4053040 con 0x556735d50e20 2024-04-16T17:17:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.661+0000 7ff204411740 10 client.29083 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:17:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.661+0000 7ff204411740 10 client.29083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.661+0000 7ff204411740 20 client.29083 populate_metadata read hostname 'smithi018' 2024-04-16T17:17:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.661+0000 7ff204411740 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556735dc7aa0 0x556735dc9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.661+0000 7ff204411740 4 client.29083 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:17:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.661+0000 7ff204411740 1 -- 172.21.15.18:0/3112502145 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556735cb4010 con 0x556735dc7aa0 2024-04-16T17:17:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.661+0000 7ff1faa00640 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556735dc7aa0 0x556735dc9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.662+0000 7ff1faa00640 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556735dc7aa0 0x556735dc9e90 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53911e4b6a74c0a4 server_cookie=477a1512a213dec3 in_seq=0 out_seq=0 2024-04-16T17:17:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:17:59.662+0000 7ff1e4c00640 10 client.29083 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:17:59.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:17:59 smithi022 ceph-mon[36023]: pgmap v419: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-04-16T17:18:00.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:17:59 smithi043 ceph-mon[36901]: pgmap v419: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-04-16T17:18:00.072 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:57 smithi018 ceph-mon[31776]: pgmap v418: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-04-16T17:18:00.073 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:17:59 smithi018 ceph-mon[31776]: pgmap v419: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-04-16T17:18:00.320 INFO:tasks.workunit.client.0.smithi018.stdout: 10 3932 31.96 MB/sec warmup 12 sec latency 359.282 ms 2024-04-16T17:18:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.005+0000 7ff1e4c00640 1 -- 172.21.15.18:0/3112502145 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556735cb4010 con 0x556735dc7aa0 2024-04-16T17:18:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.005+0000 7ff1e4c00640 10 client.29083 handle_command_reply: tid=0 2024-04-16T17:18:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 client.29083 shutdown 2024-04-16T17:18:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 -- 172.21.15.18:0/3112502145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 msgr2=0x556735d4abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d4abe0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40041a0 tx=0x7ff1f4042000 comp rx=0 tx=0).stop 2024-04-16T17:18:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 -- 172.21.15.18:0/3112502145 shutdown_connections 2024-04-16T17:18:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x556735dc7aa0 0x556735dc9e90 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556735d50e20 0x556735d4abe0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556735d4e590 0x556735d4a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 --2- 172.21.15.18:0/3112502145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556735d48610 0x556735d59330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.006+0000 7ff204411740 1 -- 172.21.15.18:0/3112502145 >> 172.21.15.18:0/3112502145 conn(0x556735ce25d0 msgr2=0x556735d43940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.007+0000 7ff1e4c00640 0 client.29083 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.007+0000 7ff204411740 1 -- 172.21.15.18:0/3112502145 shutdown_connections 2024-04-16T17:18:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.007+0000 7ff204411740 1 -- 172.21.15.18:0/3112502145 wait complete. 2024-04-16T17:18:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.007+0000 7ff204411740 20 client.29083 trim_cache size 0 max 16384 2024-04-16T17:18:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:18:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:18:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:18:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.008+0000 7ff201800640 1 -- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff1e00a3d90 msgr2=0x7ff1e00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.008+0000 7ff201800640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff1e00a3d90 0x7ff1e00a6250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff1f402ced0 tx=0x7ff1f4009940 comp rx=0 tx=0).stop 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.008+0000 7ff201800640 1 -- 172.21.15.18:0/852719824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1fc074cf0 msgr2=0x7ff1fc1a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.008+0000 7ff201800640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1fc074cf0 0x7ff1fc1a0d10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9c0 tx=0x7ff1e800ee80 comp rx=0 tx=0).stop 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.009+0000 7ff201800640 1 -- 172.21.15.18:0/852719824 shutdown_connections 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.009+0000 7ff201800640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff1e00a3d90 0x7ff1e00a6250 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.009+0000 7ff201800640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1fc10e880 0x7ff1fc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.009+0000 7ff201800640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1fc074cf0 0x7ff1fc1a0d10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.009+0000 7ff201800640 1 --2- 172.21.15.18:0/852719824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1fc0743d0 0x7ff1fc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.009+0000 7ff201800640 1 -- 172.21.15.18:0/852719824 >> 172.21.15.18:0/852719824 conn(0x7ff1fc0fdad0 msgr2=0x7ff1fc0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.010+0000 7ff201800640 1 -- 172.21.15.18:0/852719824 shutdown_connections 2024-04-16T17:18:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:01.010+0000 7ff201800640 1 -- 172.21.15.18:0/852719824 wait complete. 2024-04-16T17:18:01.322 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4048 29.59 MB/sec warmup 13 sec latency 457.682 ms 2024-04-16T17:18:01.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:01 smithi022 ceph-mon[36023]: pgmap v420: 129 pgs: 129 active+clean; 76 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.7 MiB/s wr, 462 op/s 2024-04-16T17:18:02.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:01 smithi043 ceph-mon[36901]: pgmap v420: 129 pgs: 129 active+clean; 76 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.7 MiB/s wr, 462 op/s 2024-04-16T17:18:02.082 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:01 smithi018 ceph-mon[31776]: pgmap v420: 129 pgs: 129 active+clean; 76 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.7 MiB/s wr, 462 op/s 2024-04-16T17:18:02.322 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4224 27.95 MB/sec warmup 14 sec latency 537.706 ms 2024-04-16T17:18:03.322 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4443 26.44 MB/sec warmup 15 sec latency 522.364 ms 2024-04-16T17:18:03.639 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:03 smithi018 ceph-mon[31776]: pgmap v421: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.6 MiB/s wr, 487 op/s 2024-04-16T17:18:03.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:03 smithi022 ceph-mon[36023]: pgmap v421: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.6 MiB/s wr, 487 op/s 2024-04-16T17:18:04.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:03 smithi043 ceph-mon[36901]: pgmap v421: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.6 MiB/s wr, 487 op/s 2024-04-16T17:18:04.322 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4474 24.99 MB/sec warmup 16 sec latency 637.468 ms 2024-04-16T17:18:05.322 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4480 23.63 MB/sec warmup 17 sec latency 700.514 ms 2024-04-16T17:18:05.714 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:05 smithi018 ceph-mon[31776]: pgmap v422: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.3 MiB/s wr, 455 op/s 2024-04-16T17:18:05.714 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:05 smithi018 ceph-mon[31776]: osdmap e114: 12 total, 12 up, 12 in 2024-04-16T17:18:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:05 smithi022 ceph-mon[36023]: pgmap v422: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.3 MiB/s wr, 455 op/s 2024-04-16T17:18:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:05 smithi022 ceph-mon[36023]: osdmap e114: 12 total, 12 up, 12 in 2024-04-16T17:18:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:05 smithi043 ceph-mon[36901]: pgmap v422: 129 pgs: 129 active+clean; 77 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.3 MiB/s wr, 455 op/s 2024-04-16T17:18:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:05 smithi043 ceph-mon[36901]: osdmap e114: 12 total, 12 up, 12 in 2024-04-16T17:18:06.322 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4493 22.41 MB/sec warmup 18 sec latency 896.114 ms 2024-04-16T17:18:07.322 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4577 21.63 MB/sec warmup 19 sec latency 1122.487 ms 2024-04-16T17:18:07.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:07 smithi022 ceph-mon[36023]: pgmap v424: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 83 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 854 op/s 2024-04-16T17:18:08.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:07 smithi043 ceph-mon[36901]: pgmap v424: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 83 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 854 op/s 2024-04-16T17:18:08.323 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4779 20.83 MB/sec warmup 20 sec latency 615.247 ms 2024-04-16T17:18:09.323 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4926 20.29 MB/sec warmup 21 sec latency 755.790 ms 2024-04-16T17:18:09.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:09 smithi022 ceph-mon[36023]: pgmap v425: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 83 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 854 op/s 2024-04-16T17:18:10.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:09 smithi043 ceph-mon[36901]: pgmap v425: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 83 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 854 op/s 2024-04-16T17:18:10.323 INFO:tasks.workunit.client.0.smithi018.stdout: 10 4959 19.52 MB/sec warmup 22 sec latency 1464.917 ms 2024-04-16T17:18:11.323 INFO:tasks.workunit.client.0.smithi018.stdout: 10 5208 19.50 MB/sec warmup 23 sec latency 1295.168 ms 2024-04-16T17:18:11.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:07 smithi018 ceph-mon[31776]: pgmap v424: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 83 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 854 op/s 2024-04-16T17:18:11.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:09 smithi018 ceph-mon[31776]: pgmap v425: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 83 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 854 op/s 2024-04-16T17:18:11.699 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:11 smithi018 ceph-mon[31776]: pgmap v426: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 MiB/s wr, 685 op/s 2024-04-16T17:18:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:11 smithi022 ceph-mon[36023]: pgmap v426: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 MiB/s wr, 685 op/s 2024-04-16T17:18:12.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:11 smithi043 ceph-mon[36901]: pgmap v426: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 MiB/s wr, 685 op/s 2024-04-16T17:18:12.324 INFO:tasks.workunit.client.0.smithi018.stdout: 10 5497 19.40 MB/sec warmup 24 sec latency 199.037 ms 2024-04-16T17:18:13.325 INFO:tasks.workunit.client.0.smithi018.stdout: 10 5954 19.24 MB/sec warmup 25 sec latency 86.255 ms 2024-04-16T17:18:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:13 smithi022 ceph-mon[36023]: pgmap v427: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 24 MiB/s wr, 640 op/s 2024-04-16T17:18:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:13 smithi022 ceph-mon[36023]: osdmap e115: 12 total, 12 up, 12 in 2024-04-16T17:18:14.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:13 smithi043 ceph-mon[36901]: pgmap v427: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 24 MiB/s wr, 640 op/s 2024-04-16T17:18:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:13 smithi043 ceph-mon[36901]: osdmap e115: 12 total, 12 up, 12 in 2024-04-16T17:18:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.196+0000 7f4e85200640 1 -- 172.21.15.18:0/3492523064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e80108e00 msgr2=0x7f4e801152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.196+0000 7f4e85200640 1 --2- 172.21.15.18:0/3492523064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e80108e00 0x7f4e801152c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009900 tx=0x7f4e7002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.200+0000 7f4e85200640 1 -- 172.21.15.18:0/3492523064 shutdown_connections 2024-04-16T17:18:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.200+0000 7f4e85200640 1 --2- 172.21.15.18:0/3492523064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e80108e00 0x7f4e801152c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.200+0000 7f4e85200640 1 --2- 172.21.15.18:0/3492523064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e80108460 0x7f4e801088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.200+0000 7f4e85200640 1 --2- 172.21.15.18:0/3492523064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8010d3d0 0x7f4e8010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.200+0000 7f4e85200640 1 -- 172.21.15.18:0/3492523064 >> 172.21.15.18:0/3492523064 conn(0x7f4e80077bb0 msgr2=0x7f4e80079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.204+0000 7f4e85200640 1 -- 172.21.15.18:0/3492523064 shutdown_connections 2024-04-16T17:18:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.204+0000 7f4e85200640 1 -- 172.21.15.18:0/3492523064 wait complete. 2024-04-16T17:18:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.205+0000 7f4e85200640 1 Processor -- start 2024-04-16T17:18:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.205+0000 7f4e85200640 1 -- start start 2024-04-16T17:18:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.206+0000 7f4e85200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e80108460 0x7f4e801a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.206+0000 7f4e85200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e80108e00 0x7f4e801a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.206+0000 7f4e85200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8010d3d0 0x7f4e8019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.206+0000 7f4e85200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80118840 con 0x7f4e8010d3d0 2024-04-16T17:18:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.206+0000 7f4e85200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801186c0 con 0x7f4e80108e00 2024-04-16T17:18:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.206+0000 7f4e85200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801189c0 con 0x7f4e80108460 2024-04-16T17:18:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.207+0000 7f4e7ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e80108e00 0x7f4e801a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.207+0000 7f4e7ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e80108e00 0x7f4e801a4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58174/0 (socket says 172.21.15.18:58174) 2024-04-16T17:18:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.207+0000 7f4e7ea00640 1 -- 172.21.15.18:0/2056472224 learned_addr learned my addr 172.21.15.18:0/2056472224 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.207+0000 7f4e7f400640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e80108460 0x7f4e801a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.208+0000 7f4e7ea00640 1 -- 172.21.15.18:0/2056472224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e80108460 msgr2=0x7f4e801a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.208+0000 7f4e7ea00640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e80108460 0x7f4e801a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.208+0000 7f4e7ea00640 1 -- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8010d3d0 msgr2=0x7f4e8019ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.208+0000 7f4e7ea00640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8010d3d0 0x7f4e8019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.208+0000 7f4e7ea00640 1 -- 172.21.15.18:0/2056472224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8019f1a0 con 0x7f4e80108e00 2024-04-16T17:18:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.209+0000 7f4e7ea00640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e80108e00 0x7f4e801a4db0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c920 tx=0x7f4e6c00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.209+0000 7f4e77e00640 1 -- 172.21.15.18:0/2056472224 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c007c70 con 0x7f4e80108e00 2024-04-16T17:18:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.209+0000 7f4e77e00640 1 -- 172.21.15.18:0/2056472224 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c007e10 con 0x7f4e80108e00 2024-04-16T17:18:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.209+0000 7f4e77e00640 1 -- 172.21.15.18:0/2056472224 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c018890 con 0x7f4e80108e00 2024-04-16T17:18:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.209+0000 7f4e85200640 1 -- 172.21.15.18:0/2056472224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8019f490 con 0x7f4e80108e00 2024-04-16T17:18:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.210+0000 7f4e85200640 1 -- 172.21.15.18:0/2056472224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801ab730 con 0x7f4e80108e00 2024-04-16T17:18:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.218+0000 7f4e895a6740 1 Processor -- start 2024-04-16T17:18:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.221+0000 7f4e895a6740 1 -- start start 2024-04-16T17:18:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.221+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f407f0 0x562bb4f3f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.221+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f3ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.222+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f49000 0x562bb4f4b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.222+0000 7f4e895a6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562bb4ea4780 con 0x562bb4f407f0 2024-04-16T17:18:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.222+0000 7f4e895a6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562bb4e4af40 con 0x562bb4f49000 2024-04-16T17:18:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.222+0000 7f4e895a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562bb4ea0740 con 0x562bb4f3fa80 2024-04-16T17:18:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.222+0000 7f4e7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f3ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.222+0000 7f4e7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f3ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51962/0 (socket says 172.21.15.18:51962) 2024-04-16T17:18:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.222+0000 7f4e7f400640 1 -- 172.21.15.18:0/1196502155 learned_addr learned my addr 172.21.15.18:0/1196502155 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.223+0000 7f4e7f400640 1 -- 172.21.15.18:0/1196502155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f49000 msgr2=0x562bb4f4b3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.223+0000 7f4e7f400640 1 --2- 172.21.15.18:0/1196502155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f49000 0x562bb4f4b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.223+0000 7f4e7f400640 1 -- 172.21.15.18:0/1196502155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f407f0 msgr2=0x562bb4f3f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.223+0000 7f4e7f400640 1 --2- 172.21.15.18:0/1196502155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f407f0 0x562bb4f3f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.223+0000 7f4e7f400640 1 -- 172.21.15.18:0/1196502155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb4bb7210 con 0x562bb4f3fa80 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.223+0000 7f4e7f400640 1 --2- 172.21.15.18:0/1196502155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f3ff00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800add0 tx=0x7f4e7800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21eda8d1ec16a728 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.223+0000 7f4e74c00640 1 -- 172.21.15.18:0/1196502155 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78004b20 con 0x562bb4f3fa80 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.223+0000 7f4e74c00640 1 -- 172.21.15.18:0/1196502155 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78004cc0 con 0x562bb4f3fa80 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.224+0000 7f4e74c00640 1 -- 172.21.15.18:0/1196502155 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e780056c0 con 0x562bb4f3fa80 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.225+0000 7f4e895a6740 1 -- 172.21.15.18:0/1196502155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 msgr2=0x562bb4f3ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.225+0000 7f4e895a6740 1 --2- 172.21.15.18:0/1196502155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f3ff00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800add0 tx=0x7f4e7800f730 comp rx=0 tx=0).stop 2024-04-16T17:18:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.228+0000 7f4e77e00640 1 -- 172.21.15.18:0/2056472224 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e6c018a70 con 0x7f4e80108e00 2024-04-16T17:18:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.230+0000 7f4e895a6740 1 -- 172.21.15.18:0/1196502155 shutdown_connections 2024-04-16T17:18:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.230+0000 7f4e895a6740 1 --2- 172.21.15.18:0/1196502155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f49000 0x562bb4f4b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.230+0000 7f4e895a6740 1 --2- 172.21.15.18:0/1196502155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f3ff00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.230+0000 7f4e895a6740 1 --2- 172.21.15.18:0/1196502155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f407f0 0x562bb4f3f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.230+0000 7f4e895a6740 1 -- 172.21.15.18:0/1196502155 >> 172.21.15.18:0/1196502155 conn(0x562bb4eda5d0 msgr2=0x562bb4f3a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.231+0000 7f4e77e00640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e640a3d80 0x7f4e640a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.231+0000 7f4e77e00640 1 -- 172.21.15.18:0/2056472224 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9566+0+0 (secure 0 0 0) 0x7f4e6c0d0d60 con 0x7f4e80108e00 2024-04-16T17:18:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.233+0000 7f4e7fe00640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e640a3d80 0x7f4e640a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.233+0000 7f4e895a6740 1 -- 172.21.15.18:0/1196502155 shutdown_connections 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.234+0000 7f4e895a6740 1 -- 172.21.15.18:0/1196502155 wait complete. 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.234+0000 7f4e895a6740 1 -- start start 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.235+0000 7f4e895a6740 1 Processor -- start 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.235+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f3fa80 0x562bb4f48ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.235+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 0x562bb4f45e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.235+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f49000 0x562bb4f46590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.235+0000 7f4e895a6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562bb4f4bd30 con 0x562bb4f3fa80 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.235+0000 7f4e895a6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562bb4e5e390 con 0x562bb4f407f0 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.235+0000 7f4e895a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562bb4f4beb0 con 0x562bb4f49000 2024-04-16T17:18:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.235+0000 7f4e895a6740 10 client.0 _ll_register_callbacks cb 0x562bb4e47f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.237+0000 7f4e7fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f49000 0x562bb4f46590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.237+0000 7f4e7fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f49000 0x562bb4f46590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51970/0 (socket says 172.21.15.18:51970) 2024-04-16T17:18:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.237+0000 7f4e7fe00640 1 -- 172.21.15.18:0/3918332913 learned_addr learned my addr 172.21.15.18:0/3918332913 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.237+0000 7f4e7fe00640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e640a3d80 0x7f4e640a6240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e70036040 tx=0x7f4e70009230 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.238+0000 7f4e7fe00640 1 -- 172.21.15.18:0/3918332913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 msgr2=0x562bb4f45e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.238+0000 7f4e7fe00640 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 0x562bb4f45e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.238+0000 7f4e7fe00640 1 -- 172.21.15.18:0/3918332913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f3fa80 msgr2=0x562bb4f48ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.238+0000 7f4e7fe00640 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f3fa80 0x562bb4f48ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.238+0000 7f4e7fe00640 1 -- 172.21.15.18:0/3918332913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb4c83620 con 0x562bb4f49000 2024-04-16T17:18:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.238+0000 7f4e7fe00640 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f49000 0x562bb4f46590 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e70035000 tx=0x7f4e70005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=163df6c5a330be0e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.239+0000 7f4e69600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.239+0000 7f4e69600640 1 -- 172.21.15.18:0/3918332913 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70044050 con 0x562bb4f49000 2024-04-16T17:18:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.239+0000 7f4e69600640 1 -- 172.21.15.18:0/3918332913 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9566+0+0 (secure 0 0 0) 0x7f4e70005260 con 0x562bb4f49000 2024-04-16T17:18:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.240+0000 7f4e895a6740 1 -- 172.21.15.18:0/3918332913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e78004cc0 con 0x562bb4f49000 2024-04-16T17:18:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.240+0000 7f4e69600640 1 -- 172.21.15.18:0/3918332913 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4e70005460 con 0x562bb4f49000 2024-04-16T17:18:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.243+0000 7f4e895a6740 10 client.29095 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.243+0000 7f4e895a6740 1 -- 172.21.15.18:0/3918332913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562bb4f68eb0 con 0x562bb4f49000 2024-04-16T17:18:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.243+0000 7f4e69600640 1 -- 172.21.15.18:0/3918332913 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78004cc0 con 0x562bb4f49000 2024-04-16T17:18:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.244+0000 7f4e69600640 1 -- 172.21.15.18:0/3918332913 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4e70003950 con 0x562bb4f49000 2024-04-16T17:18:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.245+0000 7f4e895a6740 10 client.29095 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.246+0000 7f4e895a6740 10 client.29095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.246+0000 7f4e895a6740 20 client.29095 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.246+0000 7f4e895a6740 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562bb4f6a3c0 0x562bb4f6c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.246+0000 7f4e895a6740 4 client.29095 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.247+0000 7f4e895a6740 1 -- 172.21.15.18:0/3918332913 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bb4eb21a0 con 0x562bb4f6a3c0 2024-04-16T17:18:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.248+0000 7f4e7f400640 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562bb4f6a3c0 0x562bb4f6c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.250+0000 7f4e7f400640 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562bb4f6a3c0 0x562bb4f6c7f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fdf6a1849fa219 server_cookie=944f453aef5997ba in_seq=0 out_seq=0 2024-04-16T17:18:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.250+0000 7f4e69600640 10 client.29095 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.251+0000 7f4e69600640 1 -- 172.21.15.18:0/3918332913 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bb4eb21a0 con 0x562bb4f6a3c0 2024-04-16T17:18:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.251+0000 7f4e69600640 10 client.29095 handle_command_reply: tid=0 2024-04-16T17:18:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.251+0000 7f4e895a6740 1 client.29095 shutdown 2024-04-16T17:18:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e895a6740 1 -- 172.21.15.18:0/3918332913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f49000 msgr2=0x562bb4f46590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e895a6740 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f49000 0x562bb4f46590 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e70035000 tx=0x7f4e70005fd0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e895a6740 1 -- 172.21.15.18:0/3918332913 shutdown_connections 2024-04-16T17:18:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e895a6740 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562bb4f6a3c0 0x562bb4f6c7f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e895a6740 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f49000 0x562bb4f46590 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e895a6740 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 0x562bb4f45e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e895a6740 1 --2- 172.21.15.18:0/3918332913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f3fa80 0x562bb4f48ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e895a6740 1 -- 172.21.15.18:0/3918332913 >> 172.21.15.18:0/3918332913 conn(0x562bb4eda5d0 msgr2=0x562bb4f3c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.253+0000 7f4e69600640 0 client.29095 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.254+0000 7f4e895a6740 1 -- 172.21.15.18:0/3918332913 shutdown_connections 2024-04-16T17:18:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.254+0000 7f4e895a6740 1 -- 172.21.15.18:0/3918332913 wait complete. 2024-04-16T17:18:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.254+0000 7f4e895a6740 20 client.29095 trim_cache size 0 max 16384 2024-04-16T17:18:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.287+0000 7f4e895a6740 1 Processor -- start 2024-04-16T17:18:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.287+0000 7f4e895a6740 1 -- start start 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.287+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f54cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.287+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 0x562bb4f48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.287+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 0x562bb4f48ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.287+0000 7f4e895a6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801afae0 con 0x562bb4f49000 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.287+0000 7f4e895a6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801ad350 con 0x562bb4f407f0 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e895a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80073ad0 con 0x562bb4f3fa80 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f54cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f54cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51978/0 (socket says 172.21.15.18:51978) 2024-04-16T17:18:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7f400640 1 -- 172.21.15.18:0/1848313988 learned_addr learned my addr 172.21.15.18:0/1848313988 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7ea00640 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 0x562bb4f48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7f400640 1 -- 172.21.15.18:0/1848313988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 msgr2=0x562bb4f48580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7f400640 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 0x562bb4f48580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7f400640 1 -- 172.21.15.18:0/1848313988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 msgr2=0x562bb4f48ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7f400640 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 0x562bb4f48ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7f400640 1 -- 172.21.15.18:0/1848313988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb4f443a0 con 0x562bb4f3fa80 2024-04-16T17:18:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.288+0000 7f4e7ea00640 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 0x562bb4f48580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.289+0000 7f4e7f400640 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f54cb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x562bb4f40f50 tx=0x7f4e780051c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510d77b7d2291ced server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.289+0000 7f4e6be00640 1 -- 172.21.15.18:0/1848313988 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e780049e0 con 0x562bb4f3fa80 2024-04-16T17:18:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.289+0000 7f4e6be00640 1 -- 172.21.15.18:0/1848313988 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7802b030 con 0x562bb4f3fa80 2024-04-16T17:18:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.289+0000 7f4e6be00640 1 -- 172.21.15.18:0/1848313988 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78011650 con 0x562bb4f3fa80 2024-04-16T17:18:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.290+0000 7f4e895a6740 1 -- 172.21.15.18:0/1848313988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 msgr2=0x562bb4f54cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.290+0000 7f4e895a6740 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f54cb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x562bb4f40f50 tx=0x7f4e780051c0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.291+0000 7f4e895a6740 1 -- 172.21.15.18:0/1848313988 shutdown_connections 2024-04-16T17:18:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.291+0000 7f4e895a6740 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 0x562bb4f48ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.291+0000 7f4e895a6740 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f407f0 0x562bb4f48580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.291+0000 7f4e895a6740 1 --2- 172.21.15.18:0/1848313988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f3fa80 0x562bb4f54cb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.291+0000 7f4e895a6740 1 -- 172.21.15.18:0/1848313988 >> 172.21.15.18:0/1848313988 conn(0x562bb4eda5d0 msgr2=0x562bb4f3a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.291+0000 7f4e895a6740 1 -- 172.21.15.18:0/1848313988 shutdown_connections 2024-04-16T17:18:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.291+0000 7f4e895a6740 1 -- 172.21.15.18:0/1848313988 wait complete. 2024-04-16T17:18:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.292+0000 7f4e895a6740 1 -- start start 2024-04-16T17:18:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.292+0000 7f4e895a6740 1 Processor -- start 2024-04-16T17:18:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.293+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f3fa80 0x562bb4fcd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.293+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f407f0 0x562bb4fcdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.293+0000 7f4e895a6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 0x562bb4f511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.293+0000 7f4e895a6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801ab900 con 0x562bb4f49000 2024-04-16T17:18:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.293+0000 7f4e895a6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801acc80 con 0x562bb4f3fa80 2024-04-16T17:18:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.293+0000 7f4e895a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801abee0 con 0x562bb4f407f0 2024-04-16T17:18:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.293+0000 7f4e895a6740 10 client.0 _ll_register_callbacks cb 0x7f4e70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.294+0000 7f4e7ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f407f0 0x562bb4fcdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 0x562bb4f511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f3fa80 0x562bb4fcd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f3fa80 0x562bb4fcd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58192/0 (socket says 172.21.15.18:58192) 2024-04-16T17:18:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7f400640 1 -- 172.21.15.18:0/233363269 learned_addr learned my addr 172.21.15.18:0/233363269 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7f400640 1 -- 172.21.15.18:0/233363269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f407f0 msgr2=0x562bb4fcdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7f400640 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f407f0 0x562bb4fcdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7f400640 1 -- 172.21.15.18:0/233363269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 msgr2=0x562bb4f511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7f400640 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 0x562bb4f511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.295+0000 7f4e7f400640 1 -- 172.21.15.18:0/233363269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb4fce720 con 0x562bb4f3fa80 2024-04-16T17:18:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.296+0000 7f4e7f400640 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f3fa80 0x562bb4fcd970 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800ad80 tx=0x7f4e7800ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29536ab9a1bd460 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.296+0000 7f4e69600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.296+0000 7f4e69600640 1 -- 172.21.15.18:0/233363269 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78005370 con 0x562bb4f3fa80 2024-04-16T17:18:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.296+0000 7f4e69600640 1 -- 172.21.15.18:0/233363269 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9566+0+0 (secure 0 0 0) 0x7f4e78005510 con 0x562bb4f3fa80 2024-04-16T17:18:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.296+0000 7f4e895a6740 1 -- 172.21.15.18:0/233363269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e7802b030 con 0x562bb4f3fa80 2024-04-16T17:18:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.297+0000 7f4e69600640 1 -- 172.21.15.18:0/233363269 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4e78041050 con 0x562bb4f3fa80 2024-04-16T17:18:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.297+0000 7f4e895a6740 10 client.29261 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.297+0000 7f4e895a6740 1 -- 172.21.15.18:0/233363269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562bb4f42f80 con 0x562bb4f3fa80 2024-04-16T17:18:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.297+0000 7f4e69600640 1 -- 172.21.15.18:0/233363269 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78003e50 con 0x562bb4f3fa80 2024-04-16T17:18:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.298+0000 7f4e69600640 1 -- 172.21.15.18:0/233363269 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4e780049e0 con 0x562bb4f3fa80 2024-04-16T17:18:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.298+0000 7f4e895a6740 10 client.29261 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.298+0000 7f4e895a6740 10 client.29261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.298+0000 7f4e895a6740 20 client.29261 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.298+0000 7f4e895a6740 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562bb4fe1a40 0x562bb4fe3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.298+0000 7f4e895a6740 4 client.29261 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:18:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.298+0000 7f4e895a6740 1 -- 172.21.15.18:0/233363269 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4e7802b030 con 0x562bb4fe1a40 2024-04-16T17:18:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.299+0000 7f4e7ea00640 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562bb4fe1a40 0x562bb4fe3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.299+0000 7f4e7ea00640 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562bb4fe1a40 0x562bb4fe3e30 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a280da72ebd730 server_cookie=d975757e7717219d in_seq=0 out_seq=0 2024-04-16T17:18:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.299+0000 7f4e69600640 10 client.29261 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:14.325 INFO:tasks.workunit.client.0.smithi018.stdout: 10 6280 19.28 MB/sec warmup 26 sec latency 104.925 ms 2024-04-16T17:18:14.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:13 smithi018 ceph-mon[31776]: pgmap v427: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 24 MiB/s wr, 640 op/s 2024-04-16T17:18:14.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:13 smithi018 ceph-mon[31776]: osdmap e115: 12 total, 12 up, 12 in 2024-04-16T17:18:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e69600640 1 -- 172.21.15.18:0/233363269 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4e7802b030 con 0x562bb4fe1a40 2024-04-16T17:18:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e69600640 10 client.29261 handle_command_reply: tid=0 2024-04-16T17:18:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 client.29261 shutdown 2024-04-16T17:18:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 -- 172.21.15.18:0/233363269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f3fa80 msgr2=0x562bb4fcd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f3fa80 0x562bb4fcd970 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800ad80 tx=0x7f4e7800ff40 comp rx=0 tx=0).stop 2024-04-16T17:18:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 -- 172.21.15.18:0/233363269 shutdown_connections 2024-04-16T17:18:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562bb4fe1a40 0x562bb4fe3e30 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562bb4f49000 0x562bb4f511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bb4f407f0 0x562bb4fcdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 --2- 172.21.15.18:0/233363269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562bb4f3fa80 0x562bb4fcd970 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e895a6740 1 -- 172.21.15.18:0/233363269 >> 172.21.15.18:0/233363269 conn(0x562bb4eda5d0 msgr2=0x562bb4f38f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.892+0000 7f4e69600640 0 client.29261 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.893+0000 7f4e895a6740 1 -- 172.21.15.18:0/233363269 shutdown_connections 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.893+0000 7f4e895a6740 1 -- 172.21.15.18:0/233363269 wait complete. 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.893+0000 7f4e895a6740 20 client.29261 trim_cache size 0 max 16384 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:18:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:18:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:18:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:18:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 -- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e640a3d80 msgr2=0x7f4e640a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e640a3d80 0x7f4e640a6240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e70036040 tx=0x7f4e70009230 comp rx=0 tx=0).stop 2024-04-16T17:18:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 -- 172.21.15.18:0/2056472224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e80108e00 msgr2=0x7f4e801a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e80108e00 0x7f4e801a4db0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c920 tx=0x7f4e6c00cde0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 -- 172.21.15.18:0/2056472224 shutdown_connections 2024-04-16T17:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e640a3d80 0x7f4e640a6240 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8010d3d0 0x7f4e8019ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e80108e00 0x7f4e801a4db0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 --2- 172.21.15.18:0/2056472224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e80108460 0x7f4e801a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.894+0000 7f4e85200640 1 -- 172.21.15.18:0/2056472224 >> 172.21.15.18:0/2056472224 conn(0x7f4e80077bb0 msgr2=0x7f4e80078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.895+0000 7f4e85200640 1 -- 172.21.15.18:0/2056472224 shutdown_connections 2024-04-16T17:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:14.895+0000 7f4e85200640 1 -- 172.21.15.18:0/2056472224 wait complete. 2024-04-16T17:18:15.325 INFO:tasks.workunit.client.0.smithi018.stdout: 10 6545 19.38 MB/sec warmup 27 sec latency 255.669 ms 2024-04-16T17:18:16.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:15 smithi043 ceph-mon[36901]: pgmap v429: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 24 MiB/s wr, 647 op/s 2024-04-16T17:18:16.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:15 smithi022 ceph-mon[36023]: pgmap v429: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 24 MiB/s wr, 647 op/s 2024-04-16T17:18:16.325 INFO:tasks.workunit.client.0.smithi018.stdout: 10 6931 19.59 MB/sec warmup 28 sec latency 75.217 ms 2024-04-16T17:18:16.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:15 smithi018 ceph-mon[31776]: pgmap v429: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 24 MiB/s wr, 647 op/s 2024-04-16T17:18:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:16 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:16 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:16 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.326 INFO:tasks.workunit.client.0.smithi018.stdout: 10 6996 19.26 MB/sec warmup 29 sec latency 770.659 ms 2024-04-16T17:18:18.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:17 smithi043 ceph-mon[36901]: pgmap v430: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 23 MiB/s wr, 588 op/s 2024-04-16T17:18:18.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:17 smithi022 ceph-mon[36023]: pgmap v430: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 23 MiB/s wr, 588 op/s 2024-04-16T17:18:18.326 INFO:tasks.workunit.client.0.smithi018.stdout: 10 7323 19.10 MB/sec warmup 30 sec latency 697.776 ms 2024-04-16T17:18:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:18 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:18:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:18 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:18:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:18 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:18:19.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:18 smithi043 ceph-mon[36901]: osdmap e116: 12 total, 12 up, 12 in 2024-04-16T17:18:19.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:18 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:18:19.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:18 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:18:19.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:18 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:18:19.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:18 smithi022 ceph-mon[36023]: osdmap e116: 12 total, 12 up, 12 in 2024-04-16T17:18:19.326 INFO:tasks.workunit.client.0.smithi018.stdout: 10 7626 19.03 MB/sec warmup 31 sec latency 50.361 ms 2024-04-16T17:18:20.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:19 smithi043 ceph-mon[36901]: pgmap v432: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.1 MiB/s wr, 508 op/s 2024-04-16T17:18:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:19 smithi022 ceph-mon[36023]: pgmap v432: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.1 MiB/s wr, 508 op/s 2024-04-16T17:18:20.326 INFO:tasks.workunit.client.0.smithi018.stdout: 10 7859 18.65 MB/sec warmup 32 sec latency 181.620 ms 2024-04-16T17:18:21.327 INFO:tasks.workunit.client.0.smithi018.stdout: 10 8099 18.39 MB/sec warmup 33 sec latency 335.664 ms 2024-04-16T17:18:22.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:21 smithi043 ceph-mon[36901]: pgmap v433: 129 pgs: 129 active+clean; 275 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 538 op/s 2024-04-16T17:18:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:21 smithi022 ceph-mon[36023]: pgmap v433: 129 pgs: 129 active+clean; 275 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 538 op/s 2024-04-16T17:18:22.327 INFO:tasks.workunit.client.0.smithi018.stdout: 10 8314 18.02 MB/sec warmup 34 sec latency 457.570 ms 2024-04-16T17:18:22.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:17 smithi018 ceph-mon[31776]: pgmap v430: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 23 MiB/s wr, 588 op/s 2024-04-16T17:18:22.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:18 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:18:22.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:18 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:18:22.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:18 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:18:22.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:18 smithi018 ceph-mon[31776]: osdmap e116: 12 total, 12 up, 12 in 2024-04-16T17:18:22.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:19 smithi018 ceph-mon[31776]: pgmap v432: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.1 MiB/s wr, 508 op/s 2024-04-16T17:18:22.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:21 smithi018 ceph-mon[31776]: pgmap v433: 129 pgs: 129 active+clean; 275 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 538 op/s 2024-04-16T17:18:23.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:18:23.098 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:18:23.327 INFO:tasks.workunit.client.0.smithi018.stdout: 10 8758 18.18 MB/sec warmup 35 sec latency 164.212 ms 2024-04-16T17:18:23.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 -- 172.21.15.18:0/3534836349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 msgr2=0x7fbb38106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 --2- 172.21.15.18:0/3534836349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 0x7fbb38106630 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009ca0 tx=0x7fbb3002c190 comp rx=0 tx=0).stop 2024-04-16T17:18:23.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 -- 172.21.15.18:0/3534836349 shutdown_connections 2024-04-16T17:18:23.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 --2- 172.21.15.18:0/3534836349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 0x7fbb38106630 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 --2- 172.21.15.18:0/3534836349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb381016f0 0x7fbb38105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 --2- 172.21.15.18:0/3534836349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb38100dd0 0x7fbb381011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 -- 172.21.15.18:0/3534836349 >> 172.21.15.18:0/3534836349 conn(0x7fbb38077ab0 msgr2=0x7fbb380fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:23.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 -- 172.21.15.18:0/3534836349 shutdown_connections 2024-04-16T17:18:23.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.361+0000 7fbb3d200640 1 -- 172.21.15.18:0/3534836349 wait complete. 2024-04-16T17:18:23.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.362+0000 7fbb3d200640 1 Processor -- start 2024-04-16T17:18:23.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.362+0000 7fbb3d200640 1 -- start start 2024-04-16T17:18:23.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb3d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb38100dd0 0x7fbb380737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb3d200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb381016f0 0x7fbb3806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb3d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 0x7fbb3806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb3d200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381187a0 con 0x7fbb381016f0 2024-04-16T17:18:23.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb3d200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38118620 con 0x7fbb381061d0 2024-04-16T17:18:23.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb3d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38118920 con 0x7fbb38100dd0 2024-04-16T17:18:23.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 0x7fbb3806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 0x7fbb3806cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38208/0 (socket says 172.21.15.18:38208) 2024-04-16T17:18:23.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb37e00640 1 -- 172.21.15.18:0/222527985 learned_addr learned my addr 172.21.15.18:0/222527985 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:23.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.363+0000 7fbb37400640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb38100dd0 0x7fbb380737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb37e00640 1 -- 172.21.15.18:0/222527985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb38100dd0 msgr2=0x7fbb380737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb37e00640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb38100dd0 0x7fbb380737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb37e00640 1 -- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb381016f0 msgr2=0x7fbb3806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb37e00640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb381016f0 0x7fbb3806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb37e00640 1 -- 172.21.15.18:0/222527985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3806d380 con 0x7fbb381061d0 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb37e00640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 0x7fbb3806cdf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbb30037040 tx=0x7fbb300346c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb2fe00640 1 -- 172.21.15.18:0/222527985 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30039050 con 0x7fbb381061d0 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb2fe00640 1 -- 172.21.15.18:0/222527985 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb30045070 con 0x7fbb381061d0 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.364+0000 7fbb2fe00640 1 -- 172.21.15.18:0/222527985 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30004500 con 0x7fbb381061d0 2024-04-16T17:18:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.365+0000 7fbb3d200640 1 -- 172.21.15.18:0/222527985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3806d610 con 0x7fbb381061d0 2024-04-16T17:18:23.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.365+0000 7fbb3d200640 1 -- 172.21.15.18:0/222527985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb38071540 con 0x7fbb381061d0 2024-04-16T17:18:23.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.369+0000 7fbb415fe740 1 Processor -- start 2024-04-16T17:18:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.372+0000 7fbb2fe00640 1 -- 172.21.15.18:0/222527985 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb300046a0 con 0x7fbb381061d0 2024-04-16T17:18:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.372+0000 7fbb415fe740 1 -- start start 2024-04-16T17:18:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.372+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdb390 0x5629b1cdb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.372+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.372+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.372+0000 7fbb415fe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5629b1c41120 con 0x5629b1cdb390 2024-04-16T17:18:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.372+0000 7fbb415fe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629b1be7340 con 0x5629b1ce4890 2024-04-16T17:18:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.372+0000 7fbb415fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629b1c3d0e0 con 0x5629b1cdbcb0 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb36a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36434/0 (socket says 172.21.15.18:36434) 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb37400640 1 -- 172.21.15.18:0/2721020616 learned_addr learned my addr 172.21.15.18:0/2721020616 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb36a00640 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38214/0 (socket says 172.21.15.18:38214) 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb36a00640 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdb390 0x5629b1cdb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb36a00640 1 -- 172.21.15.18:0/2721020616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 msgr2=0x5629b1ce4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb36a00640 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb36a00640 1 -- 172.21.15.18:0/2721020616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdb390 msgr2=0x5629b1cdb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb36a00640 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdb390 0x5629b1cdb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb36a00640 1 -- 172.21.15.18:0/2721020616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b1a1d7d0 con 0x5629b1ce4890 2024-04-16T17:18:23.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.373+0000 7fbb37400640 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:23.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.374+0000 7fbb36a00640 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce6c80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbb3806e0b0 tx=0x7fbb240096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aee12f13c837bff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.374+0000 7fbb2cc00640 1 -- 172.21.15.18:0/2721020616 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24018030 con 0x5629b1ce4890 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.374+0000 7fbb2cc00640 1 -- 172.21.15.18:0/2721020616 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb24010e20 con 0x5629b1ce4890 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.374+0000 7fbb2fe00640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbb1c0a3da0 0x7fbb1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.374+0000 7fbb2cc00640 1 -- 172.21.15.18:0/2721020616 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24011440 con 0x5629b1ce4890 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.374+0000 7fbb2fe00640 1 -- 172.21.15.18:0/222527985 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fbb300f30d0 con 0x7fbb381061d0 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.375+0000 7fbb37400640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbb1c0a3da0 0x7fbb1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.375+0000 7fbb415fe740 1 -- 172.21.15.18:0/2721020616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 msgr2=0x5629b1ce6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.375+0000 7fbb415fe740 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce6c80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbb3806e0b0 tx=0x7fbb240096e0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.375+0000 7fbb37400640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbb1c0a3da0 0x7fbb1c0a6260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5629b1cdae60 tx=0x7fbb2800b390 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.376+0000 7fbb415fe740 1 -- 172.21.15.18:0/2721020616 shutdown_connections 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.376+0000 7fbb415fe740 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce6c80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.376+0000 7fbb415fe740 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.376+0000 7fbb415fe740 1 --2- 172.21.15.18:0/2721020616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdb390 0x5629b1cdb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.376+0000 7fbb415fe740 1 -- 172.21.15.18:0/2721020616 >> 172.21.15.18:0/2721020616 conn(0x5629b1cd5ac0 msgr2=0x5629b1cd5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:23.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.376+0000 7fbb415fe740 1 -- 172.21.15.18:0/2721020616 shutdown_connections 2024-04-16T17:18:23.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.376+0000 7fbb415fe740 1 -- 172.21.15.18:0/2721020616 wait complete. 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.376+0000 7fbb415fe740 1 -- start start 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb415fe740 1 Processor -- start 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb415fe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5629b1ce7680 con 0x5629b1cdbcb0 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb415fe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629b1bfa710 con 0x5629b1ce4890 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb415fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629b1ce7800 con 0x5629b1cdb390 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb415fe740 10 client.0 _ll_register_callbacks cb 0x5629b1bf5af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:23.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36448/0 (socket says 172.21.15.18:36448) 2024-04-16T17:18:23.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.377+0000 7fbb37400640 1 -- 172.21.15.18:0/3690184043 learned_addr learned my addr 172.21.15.18:0/3690184043 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:23.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.378+0000 7fbb37e00640 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.378+0000 7fbb37400640 1 -- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 msgr2=0x5629b1ce2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:23.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.378+0000 7fbb37400640 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.378+0000 7fbb37400640 1 -- 172.21.15.18:0/3690184043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 msgr2=0x5629b1ce1e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.379+0000 7fbb37400640 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.379+0000 7fbb37400640 1 -- 172.21.15.18:0/3690184043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b19533f0 con 0x5629b1cdb390 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.379+0000 7fbb37e00640 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.379+0000 7fbb37400640 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce1910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbb28003da0 tx=0x7fbb28004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=411d3613451ea3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.379+0000 7fbb21600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.379+0000 7fbb21600640 1 -- 172.21.15.18:0/3690184043 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28028440 con 0x5629b1cdb390 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.379+0000 7fbb415fe740 1 -- 172.21.15.18:0/3690184043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb24010e20 con 0x5629b1cdb390 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.379+0000 7fbb21600640 1 -- 172.21.15.18:0/3690184043 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fbb28011440 con 0x5629b1cdb390 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.380+0000 7fbb21600640 1 -- 172.21.15.18:0/3690184043 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbb28011640 con 0x5629b1cdb390 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.380+0000 7fbb415fe740 10 client.29113 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.380+0000 7fbb415fe740 1 -- 172.21.15.18:0/3690184043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5629b19130f0 con 0x5629b1cdb390 2024-04-16T17:18:23.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb21600640 1 -- 172.21.15.18:0/3690184043 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb24010e20 con 0x5629b1cdb390 2024-04-16T17:18:23.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb21600640 1 -- 172.21.15.18:0/3690184043 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fbb28026070 con 0x5629b1cdb390 2024-04-16T17:18:23.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb415fe740 10 client.29113 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:23.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb415fe740 10 client.29113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:23.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb415fe740 20 client.29113 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:23.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb415fe740 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629b1cf4ea0 0x5629b1cf7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb415fe740 4 client.29113 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:23.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb415fe740 1 -- 172.21.15.18:0/3690184043 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629b1c4eb40 con 0x5629b1cf4ea0 2024-04-16T17:18:23.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.381+0000 7fbb36a00640 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629b1cf4ea0 0x5629b1cf7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.382+0000 7fbb36a00640 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629b1cf4ea0 0x5629b1cf7320 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43f6cfacf19edb52 server_cookie=b4d59a2a93a51677 in_seq=0 out_seq=0 2024-04-16T17:18:23.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.382+0000 7fbb21600640 10 client.29113 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:23.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.390+0000 7fbb21600640 1 -- 172.21.15.18:0/3690184043 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629b1c4eb40 con 0x5629b1cf4ea0 2024-04-16T17:18:23.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.390+0000 7fbb21600640 10 client.29113 handle_command_reply: tid=0 2024-04-16T17:18:23.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.390+0000 7fbb415fe740 1 client.29113 shutdown 2024-04-16T17:18:23.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb415fe740 1 -- 172.21.15.18:0/3690184043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 msgr2=0x5629b1ce1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb415fe740 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce1910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbb28003da0 tx=0x7fbb28004730 comp rx=0 tx=0).stop 2024-04-16T17:18:23.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb415fe740 1 -- 172.21.15.18:0/3690184043 shutdown_connections 2024-04-16T17:18:23.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb415fe740 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629b1cf4ea0 0x5629b1cf7320 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb415fe740 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1ce2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb415fe740 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 0x5629b1ce1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb415fe740 1 --2- 172.21.15.18:0/3690184043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce1910 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb21600640 0 client.29113 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:23.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.391+0000 7fbb415fe740 1 -- 172.21.15.18:0/3690184043 >> 172.21.15.18:0/3690184043 conn(0x5629b1cd5ac0 msgr2=0x5629b1cd7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:23.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.392+0000 7fbb415fe740 1 -- 172.21.15.18:0/3690184043 shutdown_connections 2024-04-16T17:18:23.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.392+0000 7fbb415fe740 1 -- 172.21.15.18:0/3690184043 wait complete. 2024-04-16T17:18:23.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.392+0000 7fbb415fe740 20 client.29113 trim_cache size 0 max 16384 2024-04-16T17:18:23.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb415fe740 1 Processor -- start 2024-04-16T17:18:23.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb415fe740 1 -- start start 2024-04-16T17:18:23.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 0x5629b1cf0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1cd7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb415fe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381c0c10 con 0x5629b1cdbcb0 2024-04-16T17:18:23.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb415fe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381be480 con 0x5629b1ce4890 2024-04-16T17:18:23.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb415fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381bc2f0 con 0x5629b1cdb390 2024-04-16T17:18:23.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1cd7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.426+0000 7fbb37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.427+0000 7fbb37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36462/0 (socket says 172.21.15.18:36462) 2024-04-16T17:18:23.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.427+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1cd7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38224/0 (socket says 172.21.15.18:38224) 2024-04-16T17:18:23.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.427+0000 7fbb37400640 1 -- 172.21.15.18:0/1605381608 learned_addr learned my addr 172.21.15.18:0/1605381608 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:23.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.427+0000 7fbb37e00640 1 -- 172.21.15.18:0/1605381608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 msgr2=0x5629b1ce4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:23.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.427+0000 7fbb37e00640 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.427+0000 7fbb37e00640 1 -- 172.21.15.18:0/1605381608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 msgr2=0x5629b1cf0540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:23.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.428+0000 7fbb36a00640 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 0x5629b1cf0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.428+0000 7fbb37e00640 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 0x5629b1cf0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.428+0000 7fbb37e00640 1 -- 172.21.15.18:0/1605381608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b1cda710 con 0x5629b1ce4890 2024-04-16T17:18:23.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.428+0000 7fbb36a00640 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 0x5629b1cf0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:23.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.428+0000 7fbb37400640 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:23.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.428+0000 7fbb37e00640 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1cd7600 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbb300f0530 tx=0x7fbb30004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f4150ef5562113 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:23.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.428+0000 7fbb23e00640 1 -- 172.21.15.18:0/1605381608 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300bed80 con 0x5629b1ce4890 2024-04-16T17:18:23.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.429+0000 7fbb23e00640 1 -- 172.21.15.18:0/1605381608 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3003a040 con 0x5629b1ce4890 2024-04-16T17:18:23.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.429+0000 7fbb23e00640 1 -- 172.21.15.18:0/1605381608 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300f0800 con 0x5629b1ce4890 2024-04-16T17:18:23.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 -- 172.21.15.18:0/1605381608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 msgr2=0x5629b1cd7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1cd7600 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbb300f0530 tx=0x7fbb30004990 comp rx=0 tx=0).stop 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 -- 172.21.15.18:0/1605381608 shutdown_connections 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1ce4890 0x5629b1cd7600 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1cdbcb0 0x5629b1cf0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1605381608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdb390 0x5629b1ce4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 -- 172.21.15.18:0/1605381608 >> 172.21.15.18:0/1605381608 conn(0x5629b1cd5ac0 msgr2=0x5629b1cd6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 -- 172.21.15.18:0/1605381608 shutdown_connections 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.430+0000 7fbb415fe740 1 -- 172.21.15.18:0/1605381608 wait complete. 2024-04-16T17:18:23.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.431+0000 7fbb415fe740 1 -- start start 2024-04-16T17:18:23.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.432+0000 7fbb415fe740 1 Processor -- start 2024-04-16T17:18:23.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.432+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1cdb390 0x5629b1cecee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.432+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1cde3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.432+0000 7fbb415fe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1ce4890 0x5629b1cde920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.433+0000 7fbb36a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1cde3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.433+0000 7fbb36a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1cde3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36466/0 (socket says 172.21.15.18:36466) 2024-04-16T17:18:23.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.433+0000 7fbb37400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1cdb390 0x5629b1cecee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.433+0000 7fbb37400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1cdb390 0x5629b1cecee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38236/0 (socket says 172.21.15.18:38236) 2024-04-16T17:18:23.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.433+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1ce4890 0x5629b1cde920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.434+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1ce4890 0x5629b1cde920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58402/0 (socket says 172.21.15.18:58402) 2024-04-16T17:18:23.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.434+0000 7fbb36a00640 1 -- 172.21.15.18:0/1246283592 learned_addr learned my addr 172.21.15.18:0/1246283592 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:23.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.434+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381bc940 con 0x5629b1ce4890 2024-04-16T17:18:23.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.434+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381bddb0 con 0x5629b1cdb390 2024-04-16T17:18:23.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.434+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381bd010 con 0x5629b1cdbcb0 2024-04-16T17:18:23.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.435+0000 7fbb37400640 1 -- 172.21.15.18:0/1246283592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 msgr2=0x5629b1cde3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:23.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.435+0000 7fbb37400640 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1cde3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.435+0000 7fbb37400640 1 -- 172.21.15.18:0/1246283592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1ce4890 msgr2=0x5629b1cde920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:23.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.435+0000 7fbb415fe740 10 client.0 _ll_register_callbacks cb 0x7fbb28005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:23.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.435+0000 7fbb37400640 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1ce4890 0x5629b1cde920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.435+0000 7fbb37400640 1 -- 172.21.15.18:0/1246283592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b1cdf150 con 0x5629b1cdb390 2024-04-16T17:18:23.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.436+0000 7fbb37e00640 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1ce4890 0x5629b1cde920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:23.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.436+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb3003a040 con 0x5629b1cdb390 2024-04-16T17:18:23.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.436+0000 7fbb37400640 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1cdb390 0x5629b1cecee0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbb28028de0 tx=0x7fbb2801c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5545e76f6bcea2d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:23.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.436+0000 7fbb21600640 10 client.29276 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:23.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.436+0000 7fbb21600640 1 -- 172.21.15.18:0/1246283592 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2803b070 con 0x5629b1cdb390 2024-04-16T17:18:23.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.436+0000 7fbb21600640 1 -- 172.21.15.18:0/1246283592 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fbb2801cde0 con 0x5629b1cdb390 2024-04-16T17:18:23.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb21600640 1 -- 172.21.15.18:0/1246283592 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbb28005040 con 0x5629b1cdb390 2024-04-16T17:18:23.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb415fe740 10 client.29276 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:23.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5629b1d5b3a0 con 0x5629b1cdb390 2024-04-16T17:18:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb21600640 1 -- 172.21.15.18:0/1246283592 <== mon.2 v2:172.21.15.22:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fbb2803e020 con 0x5629b1cdb390 2024-04-16T17:18:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb415fe740 10 client.29276 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb415fe740 10 client.29276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb415fe740 20 client.29276 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629b1d5ba70 0x5629b1d5de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb415fe740 4 client.29276 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:18:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.437+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629b1c489b0 con 0x5629b1d5ba70 2024-04-16T17:18:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.438+0000 7fbb36a00640 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629b1d5ba70 0x5629b1d5de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.438+0000 7fbb36a00640 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629b1d5ba70 0x5629b1d5de90 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbdda702fa8c41e2 server_cookie=3c8820a0a80c92ee in_seq=0 out_seq=0 2024-04-16T17:18:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.438+0000 7fbb21600640 10 client.29276 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:23.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb21600640 1 -- 172.21.15.18:0/1246283592 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5629b1c489b0 con 0x5629b1d5ba70 2024-04-16T17:18:23.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb21600640 10 client.29276 handle_command_reply: tid=0 2024-04-16T17:18:23.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 client.29276 shutdown 2024-04-16T17:18:23.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1cdb390 msgr2=0x5629b1cecee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1cdb390 0x5629b1cecee0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbb28028de0 tx=0x7fbb2801c750 comp rx=0 tx=0).stop 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 shutdown_connections 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5629b1d5ba70 0x5629b1d5de90 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5629b1ce4890 0x5629b1cde920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb21600640 0 client.29276 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629b1cdbcb0 0x5629b1cde3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 --2- 172.21.15.18:0/1246283592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629b1cdb390 0x5629b1cecee0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 >> 172.21.15.18:0/1246283592 conn(0x5629b1cd5ac0 msgr2=0x5629b1cd4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 shutdown_connections 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 1 -- 172.21.15.18:0/1246283592 wait complete. 2024-04-16T17:18:23.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.449+0000 7fbb415fe740 20 client.29276 trim_cache size 0 max 16384 2024-04-16T17:18:23.451 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:18:23.451 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:18:23.451 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:18:23.451 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.450+0000 7fbb3d200640 1 -- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbb1c0a3da0 msgr2=0x7fbb1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.450+0000 7fbb3d200640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbb1c0a3da0 0x7fbb1c0a6260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5629b1cdae60 tx=0x7fbb2800b390 comp rx=0 tx=0).stop 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.450+0000 7fbb3d200640 1 -- 172.21.15.18:0/222527985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 msgr2=0x7fbb3806cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.450+0000 7fbb3d200640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 0x7fbb3806cdf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbb30037040 tx=0x7fbb300346c0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.451+0000 7fbb3d200640 1 -- 172.21.15.18:0/222527985 shutdown_connections 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.451+0000 7fbb3d200640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbb1c0a3da0 0x7fbb1c0a6260 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.451+0000 7fbb3d200640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb381061d0 0x7fbb3806cdf0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.451+0000 7fbb3d200640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbb381016f0 0x7fbb3806c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.451+0000 7fbb3d200640 1 --2- 172.21.15.18:0/222527985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb38100dd0 0x7fbb380737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.451+0000 7fbb3d200640 1 -- 172.21.15.18:0/222527985 >> 172.21.15.18:0/222527985 conn(0x7fbb38077ab0 msgr2=0x7fbb38103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.451+0000 7fbb3d200640 1 -- 172.21.15.18:0/222527985 shutdown_connections 2024-04-16T17:18:23.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.451+0000 7fbb3d200640 1 -- 172.21.15.18:0/222527985 wait complete. 2024-04-16T17:18:23.472 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:18:23.472 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:18:23.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.736+0000 7fc6c7400640 1 -- 172.21.15.18:0/3986553793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0110820 msgr2=0x7fc6c0109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.736+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3986553793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0110820 0x7fc6c0109000 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009ca0 tx=0x7fc6b802c140 comp rx=0 tx=0).stop 2024-04-16T17:18:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.737+0000 7fc6c7400640 1 -- 172.21.15.18:0/3986553793 shutdown_connections 2024-04-16T17:18:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.737+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3986553793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c011b7e0 0x7fc6c011dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.737+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3986553793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6c0109540 0x7fc6c01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.737+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3986553793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0110820 0x7fc6c0109000 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.737+0000 7fc6c7400640 1 -- 172.21.15.18:0/3986553793 >> 172.21.15.18:0/3986553793 conn(0x7fc6c006e390 msgr2=0x7fc6c006e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.739+0000 7fc6c7400640 1 -- 172.21.15.18:0/3986553793 shutdown_connections 2024-04-16T17:18:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.740+0000 7fc6c7400640 1 -- 172.21.15.18:0/3986553793 wait complete. 2024-04-16T17:18:23.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.741+0000 7fc6c7400640 1 Processor -- start 2024-04-16T17:18:23.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.741+0000 7fc6c7400640 1 -- start start 2024-04-16T17:18:23.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.742+0000 7fc6c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0109540 0x7fc6c01ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.743+0000 7fc6c7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c0110820 0x7fc6c01adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.743+0000 7fc6c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6c011b7e0 0x7fc6c01a7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.743+0000 7fc6c4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0109540 0x7fc6c01ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.743+0000 7fc6c4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0109540 0x7fc6c01ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38260/0 (socket says 172.21.15.18:38260) 2024-04-16T17:18:23.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.743+0000 7fc6c5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6c011b7e0 0x7fc6c01a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.743+0000 7fc6c5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6c011b7e0 0x7fc6c01a7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36478/0 (socket says 172.21.15.18:36478) 2024-04-16T17:18:23.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.743+0000 7fc6c5400640 1 -- 172.21.15.18:0/3642961530 learned_addr learned my addr 172.21.15.18:0/3642961530 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:23.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.744+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c01213b0 con 0x7fc6c0110820 2024-04-16T17:18:23.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.744+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0121230 con 0x7fc6c0109540 2024-04-16T17:18:23.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.744+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0121530 con 0x7fc6c011b7e0 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.744+0000 7fc6c5400640 1 -- 172.21.15.18:0/3642961530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0109540 msgr2=0x7fc6c01ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.744+0000 7fc6c5400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0109540 0x7fc6c01ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.744+0000 7fc6c5400640 1 -- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c0110820 msgr2=0x7fc6c01adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.744+0000 7fc6c5400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c0110820 0x7fc6c01adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.744+0000 7fc6c5400640 1 -- 172.21.15.18:0/3642961530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c01a7f50 con 0x7fc6c011b7e0 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.745+0000 7fc6c5400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6c011b7e0 0x7fc6c01a7690 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b3c0 tx=0x7fc6ac00b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.745+0000 7fc6bd600640 1 -- 172.21.15.18:0/3642961530 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac011830 con 0x7fc6c011b7e0 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.745+0000 7fc6bd600640 1 -- 172.21.15.18:0/3642961530 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac016070 con 0x7fc6c011b7e0 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.745+0000 7fc6bd600640 1 -- 172.21.15.18:0/3642961530 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac00f2e0 con 0x7fc6c011b7e0 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.745+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c01a8240 con 0x7fc6c011b7e0 2024-04-16T17:18:23.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.746+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01b4370 con 0x7fc6c011b7e0 2024-04-16T17:18:23.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.749+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a0005290 con 0x7fc6c011b7e0 2024-04-16T17:18:23.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.751+0000 7fc6bd600640 1 -- 172.21.15.18:0/3642961530 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6ac00f480 con 0x7fc6c011b7e0 2024-04-16T17:18:23.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.756+0000 7fc6bd600640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6a40a3e70 0x7fc6a40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:23.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.757+0000 7fc6c4a00640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6a40a3e70 0x7fc6a40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:23.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.757+0000 7fc6bd600640 1 -- 172.21.15.18:0/3642961530 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fc6ac0d0090 con 0x7fc6c011b7e0 2024-04-16T17:18:23.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.758+0000 7fc6bd600640 1 -- 172.21.15.18:0/3642961530 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6ac09c3e0 con 0x7fc6c011b7e0 2024-04-16T17:18:23.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:23.758+0000 7fc6c4a00640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6a40a3e70 0x7fc6a40a6330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8006010 tx=0x7fc6b8009a20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:24.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:23 smithi043 ceph-mon[36901]: pgmap v434: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 507 op/s 2024-04-16T17:18:24.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:23 smithi022 ceph-mon[36023]: pgmap v434: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 507 op/s 2024-04-16T17:18:24.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.289+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6a0005850 con 0x7fc6c011b7e0 2024-04-16T17:18:24.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.293+0000 7fc6bd600640 1 -- 172.21.15.18:0/3642961530 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11857 (secure 0 0 0) 0x7fc6ac09bce0 con 0x7fc6c011b7e0 2024-04-16T17:18:24.297 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:18:24.297 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:17:53.822480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:18:24.298 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 34 2024-04-16T17:18:24.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.301+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6a40a3e70 msgr2=0x7fc6a40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.301+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6a40a3e70 0x7fc6a40a6330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8006010 tx=0x7fc6b8009a20 comp rx=0 tx=0).stop 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.301+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6c011b7e0 msgr2=0x7fc6c01a7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.301+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6c011b7e0 0x7fc6c01a7690 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b3c0 tx=0x7fc6ac00b880 comp rx=0 tx=0).stop 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.302+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 shutdown_connections 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.302+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc6a40a3e70 0x7fc6a40a6330 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.302+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6c011b7e0 0x7fc6c01a7690 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.302+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c0110820 0x7fc6c01adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.302+0000 7fc6c7400640 1 --2- 172.21.15.18:0/3642961530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c0109540 0x7fc6c01ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.302+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 >> 172.21.15.18:0/3642961530 conn(0x7fc6c006e390 msgr2=0x7fc6c01197d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:24.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.306+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 shutdown_connections 2024-04-16T17:18:24.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.306+0000 7fc6c7400640 1 -- 172.21.15.18:0/3642961530 wait complete. 2024-04-16T17:18:24.329 INFO:tasks.workunit.client.0.smithi018.stdout: 10 9242 18.22 MB/sec warmup 36 sec latency 256.536 ms 2024-04-16T17:18:24.345 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:18:24.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.646+0000 7f14c7200640 1 -- 172.21.15.18:0/1716322495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c810c400 msgr2=0x7f14c810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.646+0000 7f14c7200640 1 --2- 172.21.15.18:0/1716322495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c810c400 0x7f14c810e870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009ca0 tx=0x7f14c002c180 comp rx=0 tx=0).stop 2024-04-16T17:18:24.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.647+0000 7f14c7200640 1 -- 172.21.15.18:0/1716322495 shutdown_connections 2024-04-16T17:18:24.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.647+0000 7f14c7200640 1 --2- 172.21.15.18:0/1716322495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c810c400 0x7f14c810e870 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.647+0000 7f14c7200640 1 --2- 172.21.15.18:0/1716322495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c8103900 0x7f14c810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.647+0000 7f14c7200640 1 --2- 172.21.15.18:0/1716322495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14c8102fe0 0x7f14c81033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.647+0000 7f14c7200640 1 -- 172.21.15.18:0/1716322495 >> 172.21.15.18:0/1716322495 conn(0x7f14c8077ab0 msgr2=0x7f14c80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:24.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.648+0000 7f14c7200640 1 -- 172.21.15.18:0/1716322495 shutdown_connections 2024-04-16T17:18:24.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.648+0000 7f14c7200640 1 -- 172.21.15.18:0/1716322495 wait complete. 2024-04-16T17:18:24.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.649+0000 7f14c7200640 1 Processor -- start 2024-04-16T17:18:24.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c7200640 1 -- start start 2024-04-16T17:18:24.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c8102fe0 0x7f14c8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c7200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c8103900 0x7f14c8100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14c810c400 0x7f14c8101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c7200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81121b0 con 0x7f14c8103900 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8112030 con 0x7f14c810c400 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8112330 con 0x7f14c8102fe0 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c8102fe0 0x7f14c8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c8102fe0 0x7f14c8102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36496/0 (socket says 172.21.15.18:36496) 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.650+0000 7f14c5e00640 1 -- 172.21.15.18:0/1526367603 learned_addr learned my addr 172.21.15.18:0/1526367603 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14c6800640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14c810c400 0x7f14c8101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14c5400640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c8103900 0x7f14c8100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14c5e00640 1 -- 172.21.15.18:0/1526367603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14c810c400 msgr2=0x7f14c8101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14c5e00640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14c810c400 0x7f14c8101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14c5e00640 1 -- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c8103900 msgr2=0x7f14c8100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14c5e00640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c8103900 0x7f14c8100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14c5e00640 1 -- 172.21.15.18:0/1526367603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c81018a0 con 0x7f14c8102fe0 2024-04-16T17:18:24.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14c5e00640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c8102fe0 0x7f14c8102650 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f14b800e9f0 tx=0x7f14b800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:24.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.651+0000 7f14bea00640 1 -- 172.21.15.18:0/1526367603 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b800cd80 con 0x7f14c8102fe0 2024-04-16T17:18:24.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.652+0000 7f14bea00640 1 -- 172.21.15.18:0/1526367603 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14b8005500 con 0x7f14c8102fe0 2024-04-16T17:18:24.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.652+0000 7f14c7200640 1 -- 172.21.15.18:0/1526367603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c81af6d0 con 0x7f14c8102fe0 2024-04-16T17:18:24.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.652+0000 7f14c7200640 1 -- 172.21.15.18:0/1526367603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c81afb40 con 0x7f14c8102fe0 2024-04-16T17:18:24.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.653+0000 7f14bea00640 1 -- 172.21.15.18:0/1526367603 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8005220 con 0x7f14c8102fe0 2024-04-16T17:18:24.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.657+0000 7f14cff6d740 1 Processor -- start 2024-04-16T17:18:24.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.657+0000 7f14bea00640 1 -- 172.21.15.18:0/1526367603 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14b8004680 con 0x7f14c8102fe0 2024-04-16T17:18:24.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.658+0000 7f14cff6d740 1 -- start start 2024-04-16T17:18:24.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.658+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9d9ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.658+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.658+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9db1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.658+0000 7f14cff6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fad9d02020 con 0x55fad9d9d2a0 2024-04-16T17:18:24.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.658+0000 7f14cff6d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fad9ca8460 con 0x55fad9d9c8f0 2024-04-16T17:18:24.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.658+0000 7f14cff6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fad9cfdfe0 con 0x55fad9daec20 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.659+0000 7f14c6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9d9ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.659+0000 7f14c6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9d9ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38284/0 (socket says 172.21.15.18:38284) 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.659+0000 7f14c5e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.659+0000 7f14c6800640 1 -- 172.21.15.18:0/2382474242 learned_addr learned my addr 172.21.15.18:0/2382474242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.660+0000 7f14c6800640 1 -- 172.21.15.18:0/2382474242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 msgr2=0x55fad9db1010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.660+0000 7f14c6800640 1 --2- 172.21.15.18:0/2382474242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9db1010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.660+0000 7f14c6800640 1 -- 172.21.15.18:0/2382474242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 msgr2=0x55fad9dae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.660+0000 7f14c6800640 1 --2- 172.21.15.18:0/2382474242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.660+0000 7f14c6800640 1 -- 172.21.15.18:0/2382474242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad9cb0040 con 0x55fad9d9c8f0 2024-04-16T17:18:24.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.660+0000 7f14c5e00640 1 --2- 172.21.15.18:0/2382474242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:24.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.661+0000 7f14bea00640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14ac0a3da0 0x7f14ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.661+0000 7f14bea00640 1 -- 172.21.15.18:0/1526367603 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f14b80d0e30 con 0x7f14c8102fe0 2024-04-16T17:18:24.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.661+0000 7f14c5e00640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14ac0a3da0 0x7f14ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.662+0000 7f14c6800640 1 --2- 172.21.15.18:0/2382474242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9d9ccd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f14c002c790 tx=0x7f14c0009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2311de3048186c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:24.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.663+0000 7f14c5e00640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14ac0a3da0 0x7f14ac0a6260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f14b80a50b0 tx=0x7f14b80d4860 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:24.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.668+0000 7f14b3400640 1 -- 172.21.15.18:0/2382474242 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0043020 con 0x55fad9d9c8f0 2024-04-16T17:18:24.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.668+0000 7f14b3400640 1 -- 172.21.15.18:0/2382474242 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c0003e50 con 0x55fad9d9c8f0 2024-04-16T17:18:24.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.668+0000 7f14b3400640 1 -- 172.21.15.18:0/2382474242 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0039040 con 0x55fad9d9c8f0 2024-04-16T17:18:24.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.669+0000 7f14cff6d740 1 -- 172.21.15.18:0/2382474242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 msgr2=0x55fad9d9ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.669+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2382474242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9d9ccd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f14c002c790 tx=0x7f14c0009800 comp rx=0 tx=0).stop 2024-04-16T17:18:24.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.669+0000 7f14cff6d740 1 -- 172.21.15.18:0/2382474242 shutdown_connections 2024-04-16T17:18:24.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.669+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2382474242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9db1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.669+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2382474242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.669+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2382474242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9d9ccd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.669+0000 7f14cff6d740 1 -- 172.21.15.18:0/2382474242 >> 172.21.15.18:0/2382474242 conn(0x55fad9d97640 msgr2=0x55fad9d97a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:24.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.670+0000 7f14cff6d740 1 -- 172.21.15.18:0/2382474242 shutdown_connections 2024-04-16T17:18:24.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.670+0000 7f14cff6d740 1 -- 172.21.15.18:0/2382474242 wait complete. 2024-04-16T17:18:24.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.670+0000 7f14cff6d740 1 -- start start 2024-04-16T17:18:24.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.671+0000 7f14cff6d740 1 Processor -- start 2024-04-16T17:18:24.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.672+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dabd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.672+0000 7f14c6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dabd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.673+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.673+0000 7f14c6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dabd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38296/0 (socket says 172.21.15.18:38296) 2024-04-16T17:18:24.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.673+0000 7f14c6800640 1 -- 172.21.15.18:0/2010300290 learned_addr learned my addr 172.21.15.18:0/2010300290 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:24.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.673+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9dac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.673+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fad9db1980 con 0x55fad9d9d2a0 2024-04-16T17:18:24.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.673+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fad9cbb830 con 0x55fad9d9c8f0 2024-04-16T17:18:24.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.673+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fad9db1b00 con 0x55fad9daec20 2024-04-16T17:18:24.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.673+0000 7f14c5400640 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.674+0000 7f14cff6d740 10 client.0 _ll_register_callbacks cb 0x55fad9cc3150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.676+0000 7f14c5400640 1 -- 172.21.15.18:0/2010300290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 msgr2=0x55fad9dac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.676+0000 7f14c5400640 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9dac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.676+0000 7f14c5400640 1 -- 172.21.15.18:0/2010300290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 msgr2=0x55fad9dabd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.676+0000 7f14c5400640 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dabd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.676+0000 7f14c5400640 1 -- 172.21.15.18:0/2010300290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad9cbf920 con 0x55fad9d9d2a0 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.676+0000 7f14c5400640 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dac290 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f14b4005fd0 tx=0x7f14b400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb407433fc57b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.678+0000 7f14a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.678+0000 7f14a7e00640 1 -- 172.21.15.18:0/2010300290 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4016070 con 0x55fad9d9d2a0 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.678+0000 7f14a7e00640 1 -- 172.21.15.18:0/2010300290 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f14b4005220 con 0x55fad9d9d2a0 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.678+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c0003e50 con 0x55fad9d9d2a0 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.678+0000 7f14a7e00640 1 -- 172.21.15.18:0/2010300290 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f14b401f050 con 0x55fad9d9d2a0 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.679+0000 7f14cff6d740 10 client.18918 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.679+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fad9c8d7f0 con 0x55fad9d9d2a0 2024-04-16T17:18:24.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.680+0000 7f14a7e00640 1 -- 172.21.15.18:0/2010300290 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c0003e50 con 0x55fad9d9d2a0 2024-04-16T17:18:24.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.681+0000 7f14a7e00640 1 -- 172.21.15.18:0/2010300290 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f14b400f040 con 0x55fad9d9d2a0 2024-04-16T17:18:24.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.681+0000 7f14cff6d740 10 client.18918 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:24.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.681+0000 7f14cff6d740 10 client.18918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:18:24.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.681+0000 7f14cff6d740 20 client.18918 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:24.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.681+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55fad9dbf560 0x55fad9dc19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.681+0000 7f14cff6d740 4 client.18918 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:24.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.681+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad9d0fa40 con 0x55fad9dbf560 2024-04-16T17:18:24.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.682+0000 7f14c6800640 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55fad9dbf560 0x55fad9dc19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.682+0000 7f14c6800640 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55fad9dbf560 0x55fad9dc19c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc5116c92e1d253a server_cookie=d2947ed6dbd817a4 in_seq=0 out_seq=0 2024-04-16T17:18:24.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.682+0000 7f14a7e00640 10 client.18918 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:18:24.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.686+0000 7f14a7e00640 1 -- 172.21.15.18:0/2010300290 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fad9d0fa40 con 0x55fad9dbf560 2024-04-16T17:18:24.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.686+0000 7f14a7e00640 10 client.18918 handle_command_reply: tid=0 2024-04-16T17:18:24.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.686+0000 7f14cff6d740 1 client.18918 shutdown 2024-04-16T17:18:24.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.692+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 msgr2=0x55fad9dac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.692+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dac290 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f14b4005fd0 tx=0x7f14b400e6c0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.693+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 shutdown_connections 2024-04-16T17:18:24.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.693+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55fad9dbf560 0x55fad9dc19c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.693+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9dac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.693+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9d2a0 0x55fad9dac290 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.693+0000 7f14cff6d740 1 --2- 172.21.15.18:0/2010300290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dabd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.693+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 >> 172.21.15.18:0/2010300290 conn(0x55fad9d97640 msgr2=0x55fad9d99390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:24.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.693+0000 7f14a7e00640 0 client.18918 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:18:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.697+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 shutdown_connections 2024-04-16T17:18:24.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.697+0000 7f14cff6d740 1 -- 172.21.15.18:0/2010300290 wait complete. 2024-04-16T17:18:24.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.700+0000 7f14cff6d740 20 client.18918 trim_cache size 0 max 16384 2024-04-16T17:18:24.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.730+0000 7f14cff6d740 1 Processor -- start 2024-04-16T17:18:24.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.731+0000 7f14cff6d740 1 -- start start 2024-04-16T17:18:24.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.732+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.732+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9d96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.732+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9dc72d0 0x55fad9dc7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.732+0000 7f14cff6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b8520 con 0x55fad9dc72d0 2024-04-16T17:18:24.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.732+0000 7f14cff6d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b5d90 con 0x55fad9d9c8f0 2024-04-16T17:18:24.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.732+0000 7f14cff6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b3bc0 con 0x55fad9daec20 2024-04-16T17:18:24.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.733+0000 7f14c6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9dc72d0 0x55fad9dc7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.733+0000 7f14c6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.733+0000 7f14c6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9dc72d0 0x55fad9dc7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58484/0 (socket says 172.21.15.18:58484) 2024-04-16T17:18:24.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.733+0000 7f14c6800640 1 -- 172.21.15.18:0/144746348 learned_addr learned my addr 172.21.15.18:0/144746348 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:24.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.735+0000 7f14c5400640 1 --2- 172.21.15.18:0/144746348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9d96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.735+0000 7f14c5400640 1 -- 172.21.15.18:0/144746348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 msgr2=0x55fad9dba880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:24.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.735+0000 7f14c5400640 1 --2- 172.21.15.18:0/144746348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dba880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.735+0000 7f14c5400640 1 -- 172.21.15.18:0/144746348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9dc72d0 msgr2=0x55fad9dc7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:24.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.735+0000 7f14c5400640 1 --2- 172.21.15.18:0/144746348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9dc72d0 0x55fad9dc7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.735+0000 7f14c5400640 1 -- 172.21.15.18:0/144746348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad9a143f0 con 0x55fad9daec20 2024-04-16T17:18:24.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.736+0000 7f14c5400640 1 --2- 172.21.15.18:0/144746348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9d96580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f14b4022df0 tx=0x7f14b4005600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a54e0662b5ec4cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:24.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.736+0000 7f14b2a00640 1 -- 172.21.15.18:0/144746348 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b401f050 con 0x55fad9daec20 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.736+0000 7f14b2a00640 1 -- 172.21.15.18:0/144746348 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c0003e50 con 0x55fad9daec20 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.736+0000 7f14b2a00640 1 -- 172.21.15.18:0/144746348 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b40057f0 con 0x55fad9daec20 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.740+0000 7f14cff6d740 1 -- 172.21.15.18:0/144746348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 msgr2=0x55fad9d96580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.740+0000 7f14cff6d740 1 --2- 172.21.15.18:0/144746348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9d96580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f14b4022df0 tx=0x7f14b4005600 comp rx=0 tx=0).stop 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.741+0000 7f14cff6d740 1 -- 172.21.15.18:0/144746348 shutdown_connections 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.741+0000 7f14cff6d740 1 --2- 172.21.15.18:0/144746348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9dc72d0 0x55fad9dc7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.742+0000 7f14cff6d740 1 --2- 172.21.15.18:0/144746348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9daec20 0x55fad9d96580 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.742+0000 7f14cff6d740 1 --2- 172.21.15.18:0/144746348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9d9c8f0 0x55fad9dba880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.742+0000 7f14cff6d740 1 -- 172.21.15.18:0/144746348 >> 172.21.15.18:0/144746348 conn(0x55fad9d97640 msgr2=0x55fad9d95e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:24.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.743+0000 7f14cff6d740 1 -- 172.21.15.18:0/144746348 shutdown_connections 2024-04-16T17:18:24.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.744+0000 7f14cff6d740 1 -- 172.21.15.18:0/144746348 wait complete. 2024-04-16T17:18:24.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.745+0000 7f14cff6d740 1 -- start start 2024-04-16T17:18:24.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.745+0000 7f14cff6d740 1 Processor -- start 2024-04-16T17:18:24.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.747+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9c8f0 0x55fad9db6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.747+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9daec20 0x55fad9db6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.747+0000 7f14c6800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9c8f0 0x55fad9db6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14cff6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9dc72d0 0x55fad9db1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14cff6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b4250 con 0x55fad9d9c8f0 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.747+0000 7f14c5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9daec20 0x55fad9db6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14c5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9daec20 0x55fad9db6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38318/0 (socket says 172.21.15.18:38318) 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14c5400640 1 -- 172.21.15.18:0/250496468 learned_addr learned my addr 172.21.15.18:0/250496468 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b56c0 con 0x55fad9daec20 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b4920 con 0x55fad9dc72d0 2024-04-16T17:18:24.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14cff6d740 10 client.0 _ll_register_callbacks cb 0x7f14b4005cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:24.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14c6800640 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9dc72d0 0x55fad9db1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14c5400640 1 -- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9dc72d0 msgr2=0x55fad9db1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:24.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14c5400640 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9dc72d0 0x55fad9db1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14c5400640 1 -- 172.21.15.18:0/250496468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9c8f0 msgr2=0x55fad9db6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:24.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14c5400640 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9c8f0 0x55fad9db6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.748+0000 7f14c5400640 1 -- 172.21.15.18:0/250496468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad9ade7d0 con 0x55fad9daec20 2024-04-16T17:18:24.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.749+0000 7f14c6800640 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9dc72d0 0x55fad9db1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:24.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.749+0000 7f14c5400640 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9daec20 0x55fad9db6fe0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f14b40093c0 tx=0x7f14b400cfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a123916f47b68990 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:24.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.749+0000 7f14a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:24.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.749+0000 7f14a7e00640 1 -- 172.21.15.18:0/250496468 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b401f070 con 0x55fad9daec20 2024-04-16T17:18:24.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.749+0000 7f14a7e00640 1 -- 172.21.15.18:0/250496468 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f14b4005670 con 0x55fad9daec20 2024-04-16T17:18:24.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.749+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c0003e50 con 0x55fad9daec20 2024-04-16T17:18:24.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.750+0000 7f14a7e00640 1 -- 172.21.15.18:0/250496468 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f14b400ee60 con 0x55fad9daec20 2024-04-16T17:18:24.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.750+0000 7f14cff6d740 10 client.29300 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:24.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.750+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fad9e253e0 con 0x55fad9daec20 2024-04-16T17:18:24.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.750+0000 7f14a7e00640 1 -- 172.21.15.18:0/250496468 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c0003e50 con 0x55fad9daec20 2024-04-16T17:18:24.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.751+0000 7f14a7e00640 1 -- 172.21.15.18:0/250496468 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f14b400f040 con 0x55fad9daec20 2024-04-16T17:18:24.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.751+0000 7f14cff6d740 10 client.29300 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:24.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.751+0000 7f14cff6d740 10 client.29300 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:18:24.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.751+0000 7f14cff6d740 20 client.29300 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:24.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.751+0000 7f14cff6d740 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55fad9e25a10 0x55fad9e27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:24.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.751+0000 7f14cff6d740 4 client.29300 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:18:24.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.751+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fad9d098b0 con 0x55fad9e25a10 2024-04-16T17:18:24.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.752+0000 7f14c6800640 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55fad9e25a10 0x55fad9e27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:24.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.752+0000 7f14c6800640 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55fad9e25a10 0x55fad9e27e30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a025899280426d4f server_cookie=137ce44b8aecf5f7 in_seq=0 out_seq=0 2024-04-16T17:18:24.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14a7e00640 10 client.29300 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:18:24.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14a7e00640 1 -- 172.21.15.18:0/250496468 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fad9d098b0 con 0x55fad9e25a10 2024-04-16T17:18:24.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14a7e00640 10 client.29300 handle_command_reply: tid=0 2024-04-16T17:18:24.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 client.29300 shutdown 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9daec20 msgr2=0x55fad9db6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9daec20 0x55fad9db6fe0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f14b40093c0 tx=0x7f14b400cfd0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 shutdown_connections 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55fad9e25a10 0x55fad9e27e30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fad9dc72d0 0x55fad9db1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fad9daec20 0x55fad9db6fe0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 --2- 172.21.15.18:0/250496468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fad9d9c8f0 0x55fad9db6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 >> 172.21.15.18:0/250496468 conn(0x55fad9d97640 msgr2=0x55fad9daa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:24.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.753+0000 7f14a7e00640 0 client.29300 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:18:24.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.755+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 shutdown_connections 2024-04-16T17:18:24.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.756+0000 7f14cff6d740 1 -- 172.21.15.18:0/250496468 wait complete. 2024-04-16T17:18:24.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.756+0000 7f14cff6d740 20 client.29300 trim_cache size 0 max 16384 2024-04-16T17:18:24.760 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:18:24.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.762+0000 7f14c7200640 1 -- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14ac0a3da0 msgr2=0x7f14ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.762+0000 7f14c7200640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14ac0a3da0 0x7f14ac0a6260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f14b80a50b0 tx=0x7f14b80d4860 comp rx=0 tx=0).stop 2024-04-16T17:18:24.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.762+0000 7f14c7200640 1 -- 172.21.15.18:0/1526367603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c8102fe0 msgr2=0x7f14c8102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:24.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.762+0000 7f14c7200640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c8102fe0 0x7f14c8102650 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f14b800e9f0 tx=0x7f14b800edf0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.765+0000 7f14c7200640 1 -- 172.21.15.18:0/1526367603 shutdown_connections 2024-04-16T17:18:24.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.765+0000 7f14c7200640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14ac0a3da0 0x7f14ac0a6260 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.765+0000 7f14c7200640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14c810c400 0x7f14c8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.765+0000 7f14c7200640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c8103900 0x7f14c8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.765+0000 7f14c7200640 1 --2- 172.21.15.18:0/1526367603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14c8102fe0 0x7f14c8102650 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:24.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.765+0000 7f14c7200640 1 -- 172.21.15.18:0/1526367603 >> 172.21.15.18:0/1526367603 conn(0x7f14c8077ab0 msgr2=0x7f14c80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:24.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.766+0000 7f14c7200640 1 -- 172.21.15.18:0/1526367603 shutdown_connections 2024-04-16T17:18:24.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:24.766+0000 7f14c7200640 1 -- 172.21.15.18:0/1526367603 wait complete. 2024-04-16T17:18:24.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:18:25.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:24 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3642961530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:25.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 -- 172.21.15.18:0/2987824765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c102be0 msgr2=0x7f460c106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 --2- 172.21.15.18:0/2987824765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c102be0 0x7f460c106c20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f802bc30 comp rx=0 tx=0).stop 2024-04-16T17:18:25.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 -- 172.21.15.18:0/2987824765 shutdown_connections 2024-04-16T17:18:25.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 --2- 172.21.15.18:0/2987824765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f460c107300 0x7f460c10eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 --2- 172.21.15.18:0/2987824765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c102be0 0x7f460c106c20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 --2- 172.21.15.18:0/2987824765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c102230 0x7f460c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 -- 172.21.15.18:0/2987824765 >> 172.21.15.18:0/2987824765 conn(0x7f460c0fdab0 msgr2=0x7f460c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 -- 172.21.15.18:0/2987824765 shutdown_connections 2024-04-16T17:18:25.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.101+0000 7f4614000640 1 -- 172.21.15.18:0/2987824765 wait complete. 2024-04-16T17:18:25.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.102+0000 7f4614000640 1 Processor -- start 2024-04-16T17:18:25.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.102+0000 7f4614000640 1 -- start start 2024-04-16T17:18:25.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4614000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f460c102230 0x7f460c1a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4614000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c107300 0x7f460c1a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4614000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c10df30 0x7f460c10e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4614000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1123c0 con 0x7f460c102230 2024-04-16T17:18:25.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4614000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f460c112240 con 0x7f460c107300 2024-04-16T17:18:25.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4614000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f460c112540 con 0x7f460c10df30 2024-04-16T17:18:25.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4612000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c10df30 0x7f460c10e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4612000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c10df30 0x7f460c10e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36538/0 (socket says 172.21.15.18:36538) 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4612000640 1 -- 172.21.15.18:0/280116229 learned_addr learned my addr 172.21.15.18:0/280116229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4610c00640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c107300 0x7f460c1a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.103+0000 7f4611600640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f460c102230 0x7f460c1a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f4612000640 1 -- 172.21.15.18:0/280116229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c107300 msgr2=0x7f460c1a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f4612000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c107300 0x7f460c1a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f4612000640 1 -- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f460c102230 msgr2=0x7f460c1a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f4612000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f460c102230 0x7f460c1a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f4612000640 1 -- 172.21.15.18:0/280116229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c10eb10 con 0x7f460c10df30 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f4611600640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f460c102230 0x7f460c1a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f4612000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c10df30 0x7f460c10e3e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f460400eb10 tx=0x7f460400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f460a000640 1 -- 172.21.15.18:0/280116229 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604004b20 con 0x7f460c10df30 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f460a000640 1 -- 172.21.15.18:0/280116229 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4604004cc0 con 0x7f460c10df30 2024-04-16T17:18:25.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f460a000640 1 -- 172.21.15.18:0/280116229 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604010760 con 0x7f460c10df30 2024-04-16T17:18:25.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.104+0000 7f4614000640 1 -- 172.21.15.18:0/280116229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460c1aba30 con 0x7f460c10df30 2024-04-16T17:18:25.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.105+0000 7f4614000640 1 -- 172.21.15.18:0/280116229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460c1abde0 con 0x7f460c10df30 2024-04-16T17:18:25.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.109+0000 7f4616c5f740 1 Processor -- start 2024-04-16T17:18:25.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.113+0000 7f460a000640 1 -- 172.21.15.18:0/280116229 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4604005020 con 0x7f460c10df30 2024-04-16T17:18:25.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.115+0000 7f460a000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45f40a3e20 0x7f45f40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.115+0000 7f460a000640 1 -- 172.21.15.18:0/280116229 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f46040d1010 con 0x7f460c10df30 2024-04-16T17:18:25.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.116+0000 7f4610c00640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45f40a3e20 0x7f45f40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.117+0000 7f4616c5f740 1 -- start start 2024-04-16T17:18:25.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.117+0000 7f4610c00640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45f40a3e20 0x7f45f40a62e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f45f802c240 tx=0x7f45f802df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.117+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c1d50 0x5567124c2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.117+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c2670 0x5567124cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.117+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 0x5567124cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.117+0000 7f4616c5f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556712427020 con 0x5567124cb250 2024-04-16T17:18:25.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.117+0000 7f4616c5f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5567123cd460 con 0x5567124c1d50 2024-04-16T17:18:25.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.117+0000 7f4616c5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556712422fe0 con 0x5567124c2670 2024-04-16T17:18:25.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.118+0000 7f4610c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c2670 0x5567124cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.118+0000 7f4610c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c2670 0x5567124cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36544/0 (socket says 172.21.15.18:36544) 2024-04-16T17:18:25.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.118+0000 7f4610c00640 1 -- 172.21.15.18:0/3653632287 learned_addr learned my addr 172.21.15.18:0/3653632287 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.118+0000 7f4610c00640 1 -- 172.21.15.18:0/3653632287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c1d50 msgr2=0x5567124c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.118+0000 7f4610c00640 1 --2- 172.21.15.18:0/3653632287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c1d50 0x5567124c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.118+0000 7f4610c00640 1 -- 172.21.15.18:0/3653632287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 msgr2=0x5567124cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.118+0000 7f4610c00640 1 --2- 172.21.15.18:0/3653632287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 0x5567124cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.118+0000 7f4610c00640 1 -- 172.21.15.18:0/3653632287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567123d5040 con 0x5567124c2670 2024-04-16T17:18:25.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.119+0000 7f4610c00640 1 --2- 172.21.15.18:0/3653632287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c2670 0x5567124cad10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10a09a08fd750ce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.119+0000 7f45fe600640 1 -- 172.21.15.18:0/3653632287 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004450 con 0x5567124c2670 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.119+0000 7f45fe600640 1 -- 172.21.15.18:0/3653632287 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f8051070 con 0x5567124c2670 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.119+0000 7f45fe600640 1 -- 172.21.15.18:0/3653632287 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8005140 con 0x5567124c2670 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.120+0000 7f4616c5f740 1 -- 172.21.15.18:0/3653632287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c2670 msgr2=0x5567124cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.120+0000 7f4616c5f740 1 --2- 172.21.15.18:0/3653632287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c2670 0x5567124cad10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f803f000 comp rx=0 tx=0).stop 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.120+0000 7f4616c5f740 1 -- 172.21.15.18:0/3653632287 shutdown_connections 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.120+0000 7f4616c5f740 1 --2- 172.21.15.18:0/3653632287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 0x5567124cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.120+0000 7f4616c5f740 1 --2- 172.21.15.18:0/3653632287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c2670 0x5567124cad10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.120+0000 7f4616c5f740 1 --2- 172.21.15.18:0/3653632287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c1d50 0x5567124c2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.120+0000 7f4616c5f740 1 -- 172.21.15.18:0/3653632287 >> 172.21.15.18:0/3653632287 conn(0x5567124bc480 msgr2=0x5567124bc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.121+0000 7f4616c5f740 1 -- 172.21.15.18:0/3653632287 shutdown_connections 2024-04-16T17:18:25.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.121+0000 7f4616c5f740 1 -- 172.21.15.18:0/3653632287 wait complete. 2024-04-16T17:18:25.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.121+0000 7f4616c5f740 1 -- start start 2024-04-16T17:18:25.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.121+0000 7f4616c5f740 1 Processor -- start 2024-04-16T17:18:25.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.123+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c1d50 0x5567124c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.123+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.123+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 0x5567124c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.123+0000 7f4616c5f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5567124cdfe0 con 0x5567124cb250 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.124+0000 7f4616c5f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5567123e0830 con 0x5567124c2670 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.124+0000 7f4616c5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5567124ce160 con 0x5567124c1d50 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.124+0000 7f4616c5f740 10 client.0 _ll_register_callbacks cb 0x5567123e8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.124+0000 7f4611600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c1d50 0x5567124c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.124+0000 7f4611600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c1d50 0x5567124c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36554/0 (socket says 172.21.15.18:36554) 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.124+0000 7f4611600640 1 -- 172.21.15.18:0/2815621561 learned_addr learned my addr 172.21.15.18:0/2815621561 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.124+0000 7f4610c00640 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.124+0000 7f4612000640 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 0x5567124c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.125+0000 7f4612000640 1 -- 172.21.15.18:0/2815621561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c1d50 msgr2=0x5567124c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.125+0000 7f4612000640 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c1d50 0x5567124c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.126+0000 7f4612000640 1 -- 172.21.15.18:0/2815621561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 msgr2=0x5567124c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.126+0000 7f4612000640 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.126+0000 7f4612000640 1 -- 172.21.15.18:0/2815621561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567123e4920 con 0x5567124cb250 2024-04-16T17:18:25.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.126+0000 7f4610c00640 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:25.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.126+0000 7f4612000640 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 0x5567124c8d30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f46040ce300 tx=0x7f460409ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bac842d27af30 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.127+0000 7f45f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:18:25.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.127+0000 7f45f2a00640 1 -- 172.21.15.18:0/2815621561 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040b6060 con 0x5567124cb250 2024-04-16T17:18:25.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.127+0000 7f45f2a00640 1 -- 172.21.15.18:0/2815621561 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f46040a5330 con 0x5567124cb250 2024-04-16T17:18:25.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.128+0000 7f4616c5f740 1 -- 172.21.15.18:0/2815621561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f8051070 con 0x5567124cb250 2024-04-16T17:18:25.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.130+0000 7f45f2a00640 1 -- 172.21.15.18:0/2815621561 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f46040ce780 con 0x5567124cb250 2024-04-16T17:18:25.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.131+0000 7f4616c5f740 10 client.18930 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:25.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.132+0000 7f4616c5f740 1 -- 172.21.15.18:0/2815621561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5567123b27f0 con 0x5567124cb250 2024-04-16T17:18:25.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.134+0000 7f45f2a00640 1 -- 172.21.15.18:0/2815621561 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f8051070 con 0x5567124cb250 2024-04-16T17:18:25.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.134+0000 7f45f2a00640 1 -- 172.21.15.18:0/2815621561 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f460409bfa0 con 0x5567124cb250 2024-04-16T17:18:25.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.134+0000 7f4616c5f740 10 client.18930 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:25.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.134+0000 7f4616c5f740 10 client.18930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:18:25.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.134+0000 7f4616c5f740 20 client.18930 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:25.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.134+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567124dbb40 0x5567124ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.135+0000 7f4611600640 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567124dbb40 0x5567124ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.135+0000 7f4616c5f740 4 client.18930 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:25.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.135+0000 7f4616c5f740 1 -- 172.21.15.18:0/2815621561 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556712434a40 con 0x5567124dbb40 2024-04-16T17:18:25.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.136+0000 7f4611600640 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567124dbb40 0x5567124ddf80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a384a7862629379 server_cookie=e37aaa8286b9ccb2 in_seq=0 out_seq=0 2024-04-16T17:18:25.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.136+0000 7f45f2a00640 10 client.18930 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:18:25.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.137+0000 7f45f2a00640 1 -- 172.21.15.18:0/2815621561 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556712434a40 con 0x5567124dbb40 2024-04-16T17:18:25.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.137+0000 7f45f2a00640 10 client.18930 handle_command_reply: tid=0 2024-04-16T17:18:25.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.137+0000 7f4616c5f740 1 client.18930 shutdown 2024-04-16T17:18:25.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 -- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 msgr2=0x5567124c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 0x5567124c8d30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f46040ce300 tx=0x7f460409ca10 comp rx=0 tx=0).stop 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 -- 172.21.15.18:0/2815621561 shutdown_connections 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567124dbb40 0x5567124ddf80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124cb250 0x5567124c8d30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2815621561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124c1d50 0x5567124c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 -- 172.21.15.18:0/2815621561 >> 172.21.15.18:0/2815621561 conn(0x5567124bc480 msgr2=0x5567124be1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f45f2a00640 0 client.18930 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 -- 172.21.15.18:0/2815621561 shutdown_connections 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 1 -- 172.21.15.18:0/2815621561 wait complete. 2024-04-16T17:18:25.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.138+0000 7f4616c5f740 20 client.18930 trim_cache size 0 max 16384 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.170+0000 7f4616c5f740 1 Processor -- start 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.170+0000 7f4616c5f740 1 -- start start 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.170+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.170+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.171+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124e3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.171+0000 7f4616c5f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0129f0 con 0x5567124c1d50 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.171+0000 7f4616c5f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec010260 con 0x5567124c2670 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.171+0000 7f4616c5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00e0d0 con 0x5567124cb250 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.172+0000 7f4612000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.172+0000 7f4612000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124e3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36568/0 (socket says 172.21.15.18:36568) 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.172+0000 7f4612000640 1 -- 172.21.15.18:0/2577437562 learned_addr learned my addr 172.21.15.18:0/2577437562 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.172+0000 7f4610c00640 1 --2- 172.21.15.18:0/2577437562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.173+0000 7f4612000640 1 -- 172.21.15.18:0/2577437562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 msgr2=0x5567124c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.173+0000 7f4612000640 1 --2- 172.21.15.18:0/2577437562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.173+0000 7f4612000640 1 -- 172.21.15.18:0/2577437562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 msgr2=0x5567124d6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.173+0000 7f4612000640 1 --2- 172.21.15.18:0/2577437562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.173+0000 7f4612000640 1 -- 172.21.15.18:0/2577437562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567121393f0 con 0x5567124cb250 2024-04-16T17:18:25.176 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:24 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3642961530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.174+0000 7f4612000640 1 --2- 172.21.15.18:0/2577437562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124e3850 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f460409cae0 tx=0x7f460409bbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb894e79592a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.175+0000 7f45fdc00640 1 -- 172.21.15.18:0/2577437562 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604080080 con 0x5567124cb250 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.175+0000 7f45fdc00640 1 -- 172.21.15.18:0/2577437562 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f460409bd90 con 0x5567124cb250 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.175+0000 7f45fdc00640 1 -- 172.21.15.18:0/2577437562 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040a57c0 con 0x5567124cb250 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.175+0000 7f4616c5f740 1 -- 172.21.15.18:0/2577437562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 msgr2=0x5567124e3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.175+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2577437562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124e3850 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f460409cae0 tx=0x7f460409bbd0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.176+0000 7f4616c5f740 1 -- 172.21.15.18:0/2577437562 shutdown_connections 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.176+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2577437562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124e3850 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.176+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2577437562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.176+0000 7f4616c5f740 1 --2- 172.21.15.18:0/2577437562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.176+0000 7f4616c5f740 1 -- 172.21.15.18:0/2577437562 >> 172.21.15.18:0/2577437562 conn(0x5567124bc480 msgr2=0x5567124be160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.176+0000 7f4616c5f740 1 -- 172.21.15.18:0/2577437562 shutdown_connections 2024-04-16T17:18:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.176+0000 7f4616c5f740 1 -- 172.21.15.18:0/2577437562 wait complete. 2024-04-16T17:18:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.177+0000 7f4616c5f740 1 -- start start 2024-04-16T17:18:25.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.177+0000 7f4616c5f740 1 Processor -- start 2024-04-16T17:18:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.177+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.177+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.177+0000 7f4616c5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4611600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4611600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58564/0 (socket says 172.21.15.18:58564) 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4610c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4610c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38388/0 (socket says 172.21.15.18:38388) 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4612000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4612000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36578/0 (socket says 172.21.15.18:36578) 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4611600640 1 -- 172.21.15.18:0/4163882753 learned_addr learned my addr 172.21.15.18:0/4163882753 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00e720 con 0x5567124c1d50 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00fb90 con 0x5567124c2670 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.178+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec00edf0 con 0x5567124cb250 2024-04-16T17:18:25.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.179+0000 7f4616c5f740 10 client.0 _ll_register_callbacks cb 0x7f4604005b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.180+0000 7f4610c00640 1 -- 172.21.15.18:0/4163882753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 msgr2=0x5567124cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.181+0000 7f4610c00640 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.184+0000 7f4610c00640 1 -- 172.21.15.18:0/4163882753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 msgr2=0x5567124d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.184+0000 7f4610c00640 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.184+0000 7f4610c00640 1 -- 172.21.15.18:0/4163882753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567122037d0 con 0x5567124c2670 2024-04-16T17:18:25.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.184+0000 7f4612000640 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:25.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.184+0000 7f4611600640 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:25.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.185+0000 7f4610c00640 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124ceb30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f45f8000c00 tx=0x7f45f80048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7e4d895e302520 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.185+0000 7f45f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:25.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.185+0000 7f45f2a00640 1 -- 172.21.15.18:0/4163882753 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f80050e0 con 0x5567124c2670 2024-04-16T17:18:25.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.186+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f460409bd90 con 0x5567124c2670 2024-04-16T17:18:25.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.186+0000 7f45f2a00640 1 -- 172.21.15.18:0/4163882753 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f45f8004ee0 con 0x5567124c2670 2024-04-16T17:18:25.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.186+0000 7f45f2a00640 1 -- 172.21.15.18:0/4163882753 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f45f80050e0 con 0x5567124c2670 2024-04-16T17:18:25.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.187+0000 7f4616c5f740 10 client.29315 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:25.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.187+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556712541f40 con 0x5567124c2670 2024-04-16T17:18:25.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.187+0000 7f45f2a00640 1 -- 172.21.15.18:0/4163882753 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f460409bd90 con 0x5567124c2670 2024-04-16T17:18:25.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.188+0000 7f45f2a00640 1 -- 172.21.15.18:0/4163882753 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f45f8003b00 con 0x5567124c2670 2024-04-16T17:18:25.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.189+0000 7f4616c5f740 10 client.29315 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:25.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.189+0000 7f4616c5f740 10 client.29315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:18:25.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.190+0000 7f4616c5f740 20 client.29315 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:25.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.190+0000 7f4616c5f740 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556712542230 0x556712544620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.190+0000 7f4616c5f740 4 client.29315 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:18:25.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.190+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55671242e8b0 con 0x556712542230 2024-04-16T17:18:25.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.191+0000 7f4611600640 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556712542230 0x556712544620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.191+0000 7f4611600640 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556712542230 0x556712544620 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd7e9dfc1174b00e server_cookie=90a859b8f695fa67 in_seq=0 out_seq=0 2024-04-16T17:18:25.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.192+0000 7f45f2a00640 10 client.29315 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:18:25.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.192+0000 7f45f2a00640 1 -- 172.21.15.18:0/4163882753 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55671242e8b0 con 0x556712542230 2024-04-16T17:18:25.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.192+0000 7f45f2a00640 10 client.29315 handle_command_reply: tid=0 2024-04-16T17:18:25.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.192+0000 7f4616c5f740 1 client.29315 shutdown 2024-04-16T17:18:25.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.193+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 msgr2=0x5567124ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.193+0000 7f4616c5f740 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124ceb30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f45f8000c00 tx=0x7f45f80048f0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.196+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 shutdown_connections 2024-04-16T17:18:25.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.197+0000 7f4616c5f740 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x556712542230 0x556712544620 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.197+0000 7f4616c5f740 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567124cb250 0x5567124cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.197+0000 7f45f2a00640 0 client.29315 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:18:25.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.197+0000 7f4616c5f740 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567124c2670 0x5567124ceb30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.197+0000 7f4616c5f740 1 --2- 172.21.15.18:0/4163882753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567124c1d50 0x5567124d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.197+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 >> 172.21.15.18:0/4163882753 conn(0x5567124bc480 msgr2=0x5567124bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.197+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 shutdown_connections 2024-04-16T17:18:25.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.197+0000 7f4616c5f740 1 -- 172.21.15.18:0/4163882753 wait complete. 2024-04-16T17:18:25.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.198+0000 7f4616c5f740 20 client.29315 trim_cache size 0 max 16384 2024-04-16T17:18:25.200 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:18:25.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.202+0000 7f4614000640 1 -- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45f40a3e20 msgr2=0x7f45f40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.202+0000 7f4614000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45f40a3e20 0x7f45f40a62e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f45f802c240 tx=0x7f45f802df90 comp rx=0 tx=0).stop 2024-04-16T17:18:25.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.202+0000 7f4614000640 1 -- 172.21.15.18:0/280116229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c10df30 msgr2=0x7f460c10e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.202+0000 7f4614000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c10df30 0x7f460c10e3e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f460400eb10 tx=0x7f460400c2d0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.204+0000 7f4614000640 1 -- 172.21.15.18:0/280116229 shutdown_connections 2024-04-16T17:18:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.205+0000 7f4614000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45f40a3e20 0x7f45f40a62e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.205+0000 7f4614000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c10df30 0x7f460c10e3e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.205+0000 7f4614000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c107300 0x7f460c1a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.205+0000 7f4614000640 1 --2- 172.21.15.18:0/280116229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f460c102230 0x7f460c1a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.205+0000 7f4614000640 1 -- 172.21.15.18:0/280116229 >> 172.21.15.18:0/280116229 conn(0x7f460c0fdab0 msgr2=0x7f460c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.205+0000 7f4614000640 1 -- 172.21.15.18:0/280116229 shutdown_connections 2024-04-16T17:18:25.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.205+0000 7f4614000640 1 -- 172.21.15.18:0/280116229 wait complete. 2024-04-16T17:18:25.238 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:18:25.330 INFO:tasks.workunit.client.0.smithi018.stdout: 10 9811 18.55 MB/sec warmup 37 sec latency 143.621 ms 2024-04-16T17:18:25.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.512+0000 7f1820400640 1 -- 172.21.15.18:0/1599225607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1818074cc0 msgr2=0x7f1818112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.512+0000 7f1820400640 1 --2- 172.21.15.18:0/1599225607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1818074cc0 0x7f1818112aa0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1804009900 tx=0x7f180402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.513+0000 7f1820400640 1 -- 172.21.15.18:0/1599225607 shutdown_connections 2024-04-16T17:18:25.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.513+0000 7f1820400640 1 --2- 172.21.15.18:0/1599225607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1818112fe0 0x7f18181153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.513+0000 7f1820400640 1 --2- 172.21.15.18:0/1599225607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1818074cc0 0x7f1818112aa0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.513+0000 7f1820400640 1 --2- 172.21.15.18:0/1599225607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18180743a0 0x7f1818074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.513+0000 7f1820400640 1 -- 172.21.15.18:0/1599225607 >> 172.21.15.18:0/1599225607 conn(0x7f18180fdac0 msgr2=0x7f18180ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.513+0000 7f1820400640 1 -- 172.21.15.18:0/1599225607 shutdown_connections 2024-04-16T17:18:25.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.513+0000 7f1820400640 1 -- 172.21.15.18:0/1599225607 wait complete. 2024-04-16T17:18:25.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.514+0000 7f1820400640 1 Processor -- start 2024-04-16T17:18:25.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.514+0000 7f1820400640 1 -- start start 2024-04-16T17:18:25.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f1820400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18180743a0 0x7f181819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f1820400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1818074cc0 0x7f181819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f181da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18180743a0 0x7f181819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f181da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18180743a0 0x7f181819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38398/0 (socket says 172.21.15.18:38398) 2024-04-16T17:18:25.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f1820400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1818112fe0 0x7f18181a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f1820400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18181189e0 con 0x7f1818074cc0 2024-04-16T17:18:25.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f1820400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1818118860 con 0x7f18180743a0 2024-04-16T17:18:25.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f1820400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1818118b60 con 0x7f1818112fe0 2024-04-16T17:18:25.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.515+0000 7f181da00640 1 -- 172.21.15.18:0/3234433898 learned_addr learned my addr 172.21.15.18:0/3234433898 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.516+0000 7f181d000640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1818074cc0 0x7f181819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.516+0000 7f181e400640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1818112fe0 0x7f18181a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.516+0000 7f181da00640 1 -- 172.21.15.18:0/3234433898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1818112fe0 msgr2=0x7f18181a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.516+0000 7f181da00640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1818112fe0 0x7f18181a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.516+0000 7f181da00640 1 -- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1818074cc0 msgr2=0x7f181819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.516+0000 7f181da00640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1818074cc0 0x7f181819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.516+0000 7f181da00640 1 -- 172.21.15.18:0/3234433898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18181a3ef0 con 0x7f18180743a0 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.516+0000 7f181d000640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1818074cc0 0x7f181819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:25.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.517+0000 7f181da00640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18180743a0 0x7f181819ef00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f180c00ea60 tx=0x7f180c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.517+0000 7f1816000640 1 -- 172.21.15.18:0/3234433898 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c00cde0 con 0x7f18180743a0 2024-04-16T17:18:25.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.517+0000 7f1816000640 1 -- 172.21.15.18:0/3234433898 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c0035f0 con 0x7f18180743a0 2024-04-16T17:18:25.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.517+0000 7f1820400640 1 -- 172.21.15.18:0/3234433898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18181a41e0 con 0x7f18180743a0 2024-04-16T17:18:25.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.517+0000 7f1820400640 1 -- 172.21.15.18:0/3234433898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18181aba70 con 0x7f18180743a0 2024-04-16T17:18:25.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.518+0000 7f1816000640 1 -- 172.21.15.18:0/3234433898 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c010670 con 0x7f18180743a0 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f18231ab740 1 Processor -- start 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f18231ab740 1 -- start start 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f18231ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f18231ab740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efb3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f18231ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f18231ab740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d95ef10020 con 0x55d95efab670 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f18231ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d95eeb6460 con 0x55d95efaad50 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f18231ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d95ef0bfe0 con 0x55d95efb4250 2024-04-16T17:18:25.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f181e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f181e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38400/0 (socket says 172.21.15.18:38400) 2024-04-16T17:18:25.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f181e400640 1 -- 172.21.15.18:0/194514772 learned_addr learned my addr 172.21.15.18:0/194514772 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.522+0000 7f1816000640 1 -- 172.21.15.18:0/3234433898 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f180c004020 con 0x7f18180743a0 2024-04-16T17:18:25.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181d000640 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181da00640 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efb3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181e400640 1 -- 172.21.15.18:0/194514772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 msgr2=0x55d95efb6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181e400640 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181e400640 1 -- 172.21.15.18:0/194514772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 msgr2=0x55d95efb3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181e400640 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efb3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181e400640 1 -- 172.21.15.18:0/194514772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95eebe040 con 0x55d95efaad50 2024-04-16T17:18:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181d000640 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.523+0000 7f181da00640 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efb3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.524+0000 7f181e400640 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efab130 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1810009f30 tx=0x7f18100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2d918eeea8698cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.524+0000 7f180aa00640 1 -- 172.21.15.18:0/194514772 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810004d20 con 0x55d95efaad50 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.524+0000 7f180aa00640 1 -- 172.21.15.18:0/194514772 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1810004050 con 0x55d95efaad50 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.524+0000 7f180aa00640 1 -- 172.21.15.18:0/194514772 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810005260 con 0x55d95efaad50 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.524+0000 7f1816000640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18000a3e70 0x7f18000a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.524+0000 7f1816000640 1 -- 172.21.15.18:0/3234433898 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f180c0d0d20 con 0x7f18180743a0 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.525+0000 7f181da00640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18000a3e70 0x7f18000a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.525+0000 7f18231ab740 1 -- 172.21.15.18:0/194514772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 msgr2=0x55d95efab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.525+0000 7f18231ab740 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efab130 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1810009f30 tx=0x7f18100118d0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.525+0000 7f18231ab740 1 -- 172.21.15.18:0/194514772 shutdown_connections 2024-04-16T17:18:25.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.525+0000 7f18231ab740 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.525+0000 7f18231ab740 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efb3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.525+0000 7f18231ab740 1 --2- 172.21.15.18:0/194514772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efab130 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.525+0000 7f18231ab740 1 -- 172.21.15.18:0/194514772 >> 172.21.15.18:0/194514772 conn(0x55d95efa5480 msgr2=0x55d95efa5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.526+0000 7f181da00640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18000a3e70 0x7f18000a6330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55d95efaa820 tx=0x7f180c00ef90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.526+0000 7f18231ab740 1 -- 172.21.15.18:0/194514772 shutdown_connections 2024-04-16T17:18:25.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.526+0000 7f18231ab740 1 -- 172.21.15.18:0/194514772 wait complete. 2024-04-16T17:18:25.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.527+0000 7f18231ab740 1 -- start start 2024-04-16T17:18:25.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.527+0000 7f18231ab740 1 Processor -- start 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f18231ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efaad50 0x55d95efb11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f18231ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efab670 0x55d95efb1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f18231ab740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efb4250 0x55d95efb1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f18231ab740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d95efc8050 con 0x55d95efb4250 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f18231ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d95eec9830 con 0x55d95efab670 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f18231ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d95efc81d0 con 0x55d95efaad50 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f18231ab740 10 client.0 _ll_register_callbacks cb 0x55d95eed1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f181d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efab670 0x55d95efb1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f181d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efab670 0x55d95efb1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38404/0 (socket says 172.21.15.18:38404) 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.528+0000 7f181d000640 1 -- 172.21.15.18:0/886266953 learned_addr learned my addr 172.21.15.18:0/886266953 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.529+0000 7f181d000640 1 -- 172.21.15.18:0/886266953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efaad50 msgr2=0x55d95efb11d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:18:25.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.529+0000 7f181d000640 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efaad50 0x55d95efb11d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.529+0000 7f181d000640 1 -- 172.21.15.18:0/886266953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efb4250 msgr2=0x55d95efb1c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:25.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.529+0000 7f181e400640 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efb4250 0x55d95efb1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.529+0000 7f181d000640 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efb4250 0x55d95efb1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.529+0000 7f181d000640 1 -- 172.21.15.18:0/886266953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95eecd920 con 0x55d95efab670 2024-04-16T17:18:25.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f181e400640 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efb4250 0x55d95efb1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f181d000640 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efab670 0x55d95efb1710 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f180402c210 tx=0x7f1804009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59df33c532046a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f17ff400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f17ff400640 1 -- 172.21.15.18:0/886266953 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804042070 con 0x55d95efab670 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f18231ab740 1 -- 172.21.15.18:0/886266953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1810004050 con 0x55d95efab670 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f17ff400640 1 -- 172.21.15.18:0/886266953 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f1804004450 con 0x55d95efab670 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f17ff400640 1 -- 172.21.15.18:0/886266953 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f180404e050 con 0x55d95efab670 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f18231ab740 10 client.29327 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.530+0000 7f18231ab740 1 -- 172.21.15.18:0/886266953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d95ee9b7f0 con 0x55d95efab670 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f17ff400640 1 -- 172.21.15.18:0/886266953 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1810004050 con 0x55d95efab670 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f17ff400640 1 -- 172.21.15.18:0/886266953 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f18040035f0 con 0x55d95efab670 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f18231ab740 10 client.29327 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f18231ab740 10 client.29327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f18231ab740 20 client.29327 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f18231ab740 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d95efd5580 0x55d95efd79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f18231ab740 4 client.29327 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f18231ab740 1 -- 172.21.15.18:0/886266953 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d95ef1da40 con 0x55d95efd5580 2024-04-16T17:18:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.531+0000 7f181e400640 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d95efd5580 0x55d95efd79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.532+0000 7f181e400640 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d95efd5580 0x55d95efd79e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ac9e9a4f430d server_cookie=6db7627c8e6a69a in_seq=0 out_seq=0 2024-04-16T17:18:25.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.532+0000 7f17ff400640 10 client.29327 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:25.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.534+0000 7f17ff400640 1 -- 172.21.15.18:0/886266953 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d95ef1da40 con 0x55d95efd5580 2024-04-16T17:18:25.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.534+0000 7f17ff400640 10 client.29327 handle_command_reply: tid=0 2024-04-16T17:18:25.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.534+0000 7f18231ab740 1 client.29327 shutdown 2024-04-16T17:18:25.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f18231ab740 1 -- 172.21.15.18:0/886266953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efab670 msgr2=0x55d95efb1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f18231ab740 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efab670 0x55d95efb1710 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f180402c210 tx=0x7f1804009660 comp rx=0 tx=0).stop 2024-04-16T17:18:25.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f18231ab740 1 -- 172.21.15.18:0/886266953 shutdown_connections 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f18231ab740 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d95efd5580 0x55d95efd79e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f18231ab740 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efb4250 0x55d95efb1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f18231ab740 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efab670 0x55d95efb1710 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f18231ab740 1 --2- 172.21.15.18:0/886266953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efaad50 0x55d95efb11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f18231ab740 1 -- 172.21.15.18:0/886266953 >> 172.21.15.18:0/886266953 conn(0x55d95efa5480 msgr2=0x55d95efa71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.535+0000 7f17ff400640 0 client.29327 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.536+0000 7f18231ab740 1 -- 172.21.15.18:0/886266953 shutdown_connections 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.536+0000 7f18231ab740 1 -- 172.21.15.18:0/886266953 wait complete. 2024-04-16T17:18:25.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.536+0000 7f18231ab740 20 client.29327 trim_cache size 0 max 16384 2024-04-16T17:18:25.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.562+0000 7f18231ab740 1 Processor -- start 2024-04-16T17:18:25.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.562+0000 7f18231ab740 1 -- start start 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.563+0000 7f18231ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efb3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.563+0000 7f18231ab740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efd0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.563+0000 7f18231ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.563+0000 7f18231ab740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17f8012950 con 0x55d95efab670 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.563+0000 7f18231ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17f80101c0 con 0x55d95efaad50 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.563+0000 7f18231ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17f800e030 con 0x55d95efb4250 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.564+0000 7f181e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efb3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.564+0000 7f181e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efb3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38414/0 (socket says 172.21.15.18:38414) 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.564+0000 7f181e400640 1 -- 172.21.15.18:0/35051555 learned_addr learned my addr 172.21.15.18:0/35051555 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.564+0000 7f181e400640 1 --2- 172.21.15.18:0/35051555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.565+0000 7f181e400640 1 -- 172.21.15.18:0/35051555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 msgr2=0x55d95efb01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.565+0000 7f181e400640 1 --2- 172.21.15.18:0/35051555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.565+0000 7f181e400640 1 -- 172.21.15.18:0/35051555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 msgr2=0x55d95efd0f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:25.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.565+0000 7f181e400640 1 --2- 172.21.15.18:0/35051555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efd0f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.565+0000 7f181e400640 1 -- 172.21.15.18:0/35051555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95ec223f0 con 0x55d95efaad50 2024-04-16T17:18:25.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.565+0000 7f181e400640 1 --2- 172.21.15.18:0/35051555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efb3900 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1810009bb0 tx=0x7f18100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5776d42096e75335 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.566+0000 7f180a000640 1 -- 172.21.15.18:0/35051555 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810003b70 con 0x55d95efaad50 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.566+0000 7f180a000640 1 -- 172.21.15.18:0/35051555 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1810003d10 con 0x55d95efaad50 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.566+0000 7f180a000640 1 -- 172.21.15.18:0/35051555 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100053b0 con 0x55d95efaad50 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.567+0000 7f18231ab740 1 -- 172.21.15.18:0/35051555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 msgr2=0x55d95efb3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.567+0000 7f18231ab740 1 --2- 172.21.15.18:0/35051555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efb3900 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1810009bb0 tx=0x7f18100118d0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.567+0000 7f18231ab740 1 -- 172.21.15.18:0/35051555 shutdown_connections 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.567+0000 7f18231ab740 1 --2- 172.21.15.18:0/35051555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efb01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.567+0000 7f18231ab740 1 --2- 172.21.15.18:0/35051555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efd0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.567+0000 7f18231ab740 1 --2- 172.21.15.18:0/35051555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efb3900 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.567+0000 7f18231ab740 1 -- 172.21.15.18:0/35051555 >> 172.21.15.18:0/35051555 conn(0x55d95efa5480 msgr2=0x55d95efafaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.567+0000 7f18231ab740 1 -- 172.21.15.18:0/35051555 shutdown_connections 2024-04-16T17:18:25.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.568+0000 7f18231ab740 1 -- 172.21.15.18:0/35051555 wait complete. 2024-04-16T17:18:25.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.568+0000 7f18231ab740 1 -- start start 2024-04-16T17:18:25.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.569+0000 7f18231ab740 1 Processor -- start 2024-04-16T17:18:25.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.569+0000 7f18231ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efcd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.569+0000 7f18231ab740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.569+0000 7f18231ab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efaedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.569+0000 7f18231ab740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17f800e680 con 0x55d95efab670 2024-04-16T17:18:25.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.569+0000 7f18231ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17f800faf0 con 0x55d95efaad50 2024-04-16T17:18:25.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.569+0000 7f18231ab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17f800ed50 con 0x55d95efb4250 2024-04-16T17:18:25.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.569+0000 7f18231ab740 10 client.0 _ll_register_callbacks cb 0x7f180c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:25.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.570+0000 7f181e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efcd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.570+0000 7f181e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efcd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38424/0 (socket says 172.21.15.18:38424) 2024-04-16T17:18:25.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.570+0000 7f181e400640 1 -- 172.21.15.18:0/2854962630 learned_addr learned my addr 172.21.15.18:0/2854962630 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:25.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.570+0000 7f181e400640 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efaedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.570+0000 7f181d000640 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.571+0000 7f181e400640 1 -- 172.21.15.18:0/2854962630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 msgr2=0x55d95efaedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.571+0000 7f181e400640 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efaedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.571+0000 7f181e400640 1 -- 172.21.15.18:0/2854962630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 msgr2=0x55d95efae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:25.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.571+0000 7f181e400640 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.571+0000 7f181e400640 1 -- 172.21.15.18:0/2854962630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95ecec7d0 con 0x55d95efaad50 2024-04-16T17:18:25.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.571+0000 7f181e400640 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efcd920 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1810004920 tx=0x7f1810009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24355cf045fc55d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:25.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.571+0000 7f17ff400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:25.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.572+0000 7f17ff400640 1 -- 172.21.15.18:0/2854962630 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100055d0 con 0x55d95efaad50 2024-04-16T17:18:25.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.572+0000 7f17ff400640 1 -- 172.21.15.18:0/2854962630 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f1810005770 con 0x55d95efaad50 2024-04-16T17:18:25.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.572+0000 7f18231ab740 1 -- 172.21.15.18:0/2854962630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1810003d10 con 0x55d95efaad50 2024-04-16T17:18:25.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.572+0000 7f17ff400640 1 -- 172.21.15.18:0/2854962630 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1810005990 con 0x55d95efaad50 2024-04-16T17:18:25.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.573+0000 7f18231ab740 10 client.29333 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:25.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.573+0000 7f18231ab740 1 -- 172.21.15.18:0/2854962630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d95f04bda0 con 0x55d95efaad50 2024-04-16T17:18:25.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.573+0000 7f17ff400640 1 -- 172.21.15.18:0/2854962630 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1810003d10 con 0x55d95efaad50 2024-04-16T17:18:25.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.573+0000 7f17ff400640 1 -- 172.21.15.18:0/2854962630 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f181000f720 con 0x55d95efaad50 2024-04-16T17:18:25.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.574+0000 7f18231ab740 10 client.29333 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:25.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.574+0000 7f18231ab740 10 client.29333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:25.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.574+0000 7f18231ab740 20 client.29333 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:25.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.574+0000 7f18231ab740 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d95f04c1d0 0x55d95f04e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:25.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.574+0000 7f18231ab740 4 client.29333 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:18:25.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.574+0000 7f18231ab740 1 -- 172.21.15.18:0/2854962630 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d95ef178b0 con 0x55d95f04c1d0 2024-04-16T17:18:25.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.574+0000 7f181d000640 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d95f04c1d0 0x55d95f04e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:25.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.575+0000 7f181d000640 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d95f04c1d0 0x55d95f04e5f0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bfaa8f71b01c96 server_cookie=1e37d8ec50261a83 in_seq=0 out_seq=0 2024-04-16T17:18:25.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.575+0000 7f17ff400640 10 client.29333 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:25.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f17ff400640 1 -- 172.21.15.18:0/2854962630 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d95ef178b0 con 0x55d95f04c1d0 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f17ff400640 10 client.29333 handle_command_reply: tid=0 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 client.29333 shutdown 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 -- 172.21.15.18:0/2854962630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 msgr2=0x55d95efcd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efcd920 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1810004920 tx=0x7f1810009230 comp rx=0 tx=0).stop 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 -- 172.21.15.18:0/2854962630 shutdown_connections 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d95f04c1d0 0x55d95f04e5f0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d95efb4250 0x55d95efaedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d95efab670 0x55d95efae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 --2- 172.21.15.18:0/2854962630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d95efaad50 0x55d95efcd920 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f18231ab740 1 -- 172.21.15.18:0/2854962630 >> 172.21.15.18:0/2854962630 conn(0x55d95efa5480 msgr2=0x55d95efa6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.579+0000 7f17ff400640 0 client.29333 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.580+0000 7f18231ab740 1 -- 172.21.15.18:0/2854962630 shutdown_connections 2024-04-16T17:18:25.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.580+0000 7f18231ab740 1 -- 172.21.15.18:0/2854962630 wait complete. 2024-04-16T17:18:25.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.580+0000 7f18231ab740 20 client.29333 trim_cache size 0 max 16384 2024-04-16T17:18:25.582 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:18:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.582+0000 7f1820400640 1 -- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18000a3e70 msgr2=0x7f18000a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.582+0000 7f1820400640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18000a3e70 0x7f18000a6330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55d95efaa820 tx=0x7f180c00ef90 comp rx=0 tx=0).stop 2024-04-16T17:18:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.582+0000 7f1820400640 1 -- 172.21.15.18:0/3234433898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18180743a0 msgr2=0x7f181819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.582+0000 7f1820400640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18180743a0 0x7f181819ef00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f180c00ea60 tx=0x7f180c00ef20 comp rx=0 tx=0).stop 2024-04-16T17:18:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.583+0000 7f1820400640 1 -- 172.21.15.18:0/3234433898 shutdown_connections 2024-04-16T17:18:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.583+0000 7f1820400640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18000a3e70 0x7f18000a6330 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.583+0000 7f1820400640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1818112fe0 0x7f18181a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.583+0000 7f1820400640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1818074cc0 0x7f181819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.583+0000 7f1820400640 1 --2- 172.21.15.18:0/3234433898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18180743a0 0x7f181819ef00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.583+0000 7f1820400640 1 -- 172.21.15.18:0/3234433898 >> 172.21.15.18:0/3234433898 conn(0x7f18180fdac0 msgr2=0x7f18180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.583+0000 7f1820400640 1 -- 172.21.15.18:0/3234433898 shutdown_connections 2024-04-16T17:18:25.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:25.583+0000 7f1820400640 1 -- 172.21.15.18:0/3234433898 wait complete. 2024-04-16T17:18:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:25 smithi043 ceph-mon[36901]: pgmap v435: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-04-16T17:18:26.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:25 smithi022 ceph-mon[36023]: pgmap v435: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-04-16T17:18:26.329 INFO:tasks.workunit.client.0.smithi018.stdout: 10 10367 19.33 MB/sec warmup 38 sec latency 48.198 ms 2024-04-16T17:18:26.606 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:18:26.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:18:26.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:18:26.823 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:23 smithi018 ceph-mon[31776]: pgmap v434: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 507 op/s 2024-04-16T17:18:26.823 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:24 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3642961530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:26.823 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:25 smithi018 ceph-mon[31776]: pgmap v435: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-04-16T17:18:26.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.847+0000 7f82d8c00640 1 -- 172.21.15.18:0/1962339333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d4110580 msgr2=0x7f82d4110960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.847+0000 7f82d8c00640 1 --2- 172.21.15.18:0/1962339333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d4110580 0x7f82d4110960 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009ca0 tx=0x7f82c402c0f0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.847+0000 7f82d8c00640 1 -- 172.21.15.18:0/1962339333 shutdown_connections 2024-04-16T17:18:26.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.847+0000 7f82d8c00640 1 --2- 172.21.15.18:0/1962339333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82d410ec00 0x7f82d411d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.847+0000 7f82d8c00640 1 --2- 172.21.15.18:0/1962339333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d410e0e0 0x7f82d410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.847+0000 7f82d8c00640 1 --2- 172.21.15.18:0/1962339333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d4110580 0x7f82d4110960 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.847+0000 7f82d8c00640 1 -- 172.21.15.18:0/1962339333 >> 172.21.15.18:0/1962339333 conn(0x7f82d406cf10 msgr2=0x7f82d406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:26.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.848+0000 7f82d8c00640 1 -- 172.21.15.18:0/1962339333 shutdown_connections 2024-04-16T17:18:26.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.848+0000 7f82d8c00640 1 -- 172.21.15.18:0/1962339333 wait complete. 2024-04-16T17:18:26.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.849+0000 7f82d8c00640 1 Processor -- start 2024-04-16T17:18:26.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.849+0000 7f82d8c00640 1 -- start start 2024-04-16T17:18:26.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.850+0000 7f82d8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82d410e0e0 0x7f82d41a7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.850+0000 7f82d8c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d410ec00 0x7f82d41a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.850+0000 7f82d8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d4110580 0x7f82d41abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.850+0000 7f82d8c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4121030 con 0x7f82d410ec00 2024-04-16T17:18:26.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.850+0000 7f82d8c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4120eb0 con 0x7f82d410e0e0 2024-04-16T17:18:26.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.850+0000 7f82d8c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41211b0 con 0x7f82d4110580 2024-04-16T17:18:26.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.850+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d410ec00 0x7f82d41a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.850+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d410ec00 0x7f82d41a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41400/0 (socket says 172.21.15.18:41400) 2024-04-16T17:18:26.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.851+0000 7f82d2a00640 1 -- 172.21.15.18:0/3652087213 learned_addr learned my addr 172.21.15.18:0/3652087213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:26.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.851+0000 7f82d3400640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82d410e0e0 0x7f82d41a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.851+0000 7f82d2a00640 1 -- 172.21.15.18:0/3652087213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d4110580 msgr2=0x7f82d41abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.851+0000 7f82d2a00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d4110580 0x7f82d41abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.851+0000 7f82d2a00640 1 -- 172.21.15.18:0/3652087213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82d410e0e0 msgr2=0x7f82d41a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.851+0000 7f82d2a00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82d410e0e0 0x7f82d41a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.851+0000 7f82d2a00640 1 -- 172.21.15.18:0/3652087213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d41ac520 con 0x7f82d410ec00 2024-04-16T17:18:26.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.851+0000 7f82d2a00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d410ec00 0x7f82d41a7a70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f82c000b4a0 tx=0x7f82c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:26.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.853+0000 7f82cbe00640 1 -- 172.21.15.18:0/3652087213 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0014070 con 0x7f82d410ec00 2024-04-16T17:18:26.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.853+0000 7f82cbe00640 1 -- 172.21.15.18:0/3652087213 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c0010af0 con 0x7f82d410ec00 2024-04-16T17:18:26.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.853+0000 7f82cbe00640 1 -- 172.21.15.18:0/3652087213 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0017ce0 con 0x7f82d410ec00 2024-04-16T17:18:26.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.853+0000 7f82d8c00640 1 -- 172.21.15.18:0/3652087213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d41ac810 con 0x7f82d410ec00 2024-04-16T17:18:26.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.853+0000 7f82d8c00640 1 -- 172.21.15.18:0/3652087213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d41b40a0 con 0x7f82d410ec00 2024-04-16T17:18:26.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.859+0000 7f82cbe00640 1 -- 172.21.15.18:0/3652087213 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f82c0017070 con 0x7f82d410ec00 2024-04-16T17:18:26.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.860+0000 7f82cbe00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f82b80a3e20 0x7f82b80a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.860+0000 7f82cbe00640 1 -- 172.21.15.18:0/3652087213 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f82c00cfbd0 con 0x7f82d410ec00 2024-04-16T17:18:26.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.861+0000 7f82dd031740 1 Processor -- start 2024-04-16T17:18:26.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.862+0000 7f82dd031740 1 -- start start 2024-04-16T17:18:26.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.862+0000 7f82d3400640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f82b80a3e20 0x7f82b80a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.863+0000 7f82dd031740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 0x557dd78c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.863+0000 7f82dd031740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c37b0 0x557dd78cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.863+0000 7f82dd031740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.863+0000 7f82dd031740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557dd7827cb0 con 0x557dd78c2e90 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.863+0000 7f82dd031740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557dd77ce370 con 0x557dd78cc390 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.863+0000 7f82dd031740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557dd7823c70 con 0x557dd78c37b0 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c37b0 0x557dd78cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c37b0 0x557dd78cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59770/0 (socket says 172.21.15.18:59770) 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d2a00640 1 -- 172.21.15.18:0/570622144 learned_addr learned my addr 172.21.15.18:0/570622144 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d3e00640 1 --2- 172.21.15.18:0/570622144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d3400640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f82b80a3e20 0x7f82b80a62e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f82c402c700 tx=0x7f82c4032590 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d2a00640 1 -- 172.21.15.18:0/570622144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 msgr2=0x557dd78ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d2a00640 1 --2- 172.21.15.18:0/570622144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d2a00640 1 -- 172.21.15.18:0/570622144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 msgr2=0x557dd78c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.864+0000 7f82d2a00640 1 --2- 172.21.15.18:0/570622144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 0x557dd78c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.865+0000 7f82d2a00640 1 -- 172.21.15.18:0/570622144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd77ed910 con 0x557dd78c37b0 2024-04-16T17:18:26.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.865+0000 7f82d2a00640 1 --2- 172.21.15.18:0/570622144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c37b0 0x557dd78cbe50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f82c0097c80 tx=0x7f82c000bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd9e781ce7638dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:26.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.865+0000 7f82bfe00640 1 -- 172.21.15.18:0/570622144 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c001e050 con 0x557dd78c37b0 2024-04-16T17:18:26.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.865+0000 7f82bfe00640 1 -- 172.21.15.18:0/570622144 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c00a4490 con 0x557dd78c37b0 2024-04-16T17:18:26.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.865+0000 7f82bfe00640 1 -- 172.21.15.18:0/570622144 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c009cb50 con 0x557dd78c37b0 2024-04-16T17:18:26.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.867+0000 7f82dd031740 1 -- 172.21.15.18:0/570622144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c37b0 msgr2=0x557dd78cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.867+0000 7f82dd031740 1 --2- 172.21.15.18:0/570622144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c37b0 0x557dd78cbe50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f82c0097c80 tx=0x7f82c000bf90 comp rx=0 tx=0).stop 2024-04-16T17:18:26.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.867+0000 7f82dd031740 1 -- 172.21.15.18:0/570622144 shutdown_connections 2024-04-16T17:18:26.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.867+0000 7f82dd031740 1 --2- 172.21.15.18:0/570622144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.867+0000 7f82dd031740 1 --2- 172.21.15.18:0/570622144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c37b0 0x557dd78cbe50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.867+0000 7f82dd031740 1 --2- 172.21.15.18:0/570622144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 0x557dd78c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.867+0000 7f82dd031740 1 -- 172.21.15.18:0/570622144 >> 172.21.15.18:0/570622144 conn(0x557dd78bd5c0 msgr2=0x557dd78bd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:26.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.868+0000 7f82dd031740 1 -- 172.21.15.18:0/570622144 shutdown_connections 2024-04-16T17:18:26.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.868+0000 7f82dd031740 1 -- 172.21.15.18:0/570622144 wait complete. 2024-04-16T17:18:26.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.869+0000 7f82dd031740 1 -- start start 2024-04-16T17:18:26.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.869+0000 7f82dd031740 1 Processor -- start 2024-04-16T17:18:26.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.870+0000 7f82dd031740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82dd031740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78c9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82dd031740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82dd031740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557dd78cf120 con 0x557dd78c37b0 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82dd031740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557dd77e1730 con 0x557dd78cc390 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82dd031740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557dd78cf2a0 con 0x557dd78c2e90 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82dd031740 10 client.0 _ll_register_callbacks cb 0x557dd77b01e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41424/0 (socket says 172.21.15.18:41424) 2024-04-16T17:18:26.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.871+0000 7f82d2a00640 1 -- 172.21.15.18:0/982538212 learned_addr learned my addr 172.21.15.18:0/982538212 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.872+0000 7f82d2a00640 1 -- 172.21.15.18:0/982538212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 msgr2=0x557dd78c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.872+0000 7f82d2a00640 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.872+0000 7f82d2a00640 1 -- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 msgr2=0x557dd78c9e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.872+0000 7f82d3e00640 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.872+0000 7f82d2a00640 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.872+0000 7f82d2a00640 1 -- 172.21.15.18:0/982538212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd76027e0 con 0x557dd78c37b0 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.873+0000 7f82d3400640 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.873+0000 7f82d3e00640 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.873+0000 7f82d2a00640 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78c9930 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f82c00a3f60 tx=0x7f82c0017ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79d0109aaedd88c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.873+0000 7f82bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.873+0000 7f82bcc00640 1 -- 172.21.15.18:0/982538212 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00b8070 con 0x557dd78c37b0 2024-04-16T17:18:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.874+0000 7f82bcc00640 1 -- 172.21.15.18:0/982538212 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f82c00cd140 con 0x557dd78c37b0 2024-04-16T17:18:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.874+0000 7f82dd031740 1 -- 172.21.15.18:0/982538212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c00a4490 con 0x557dd78c37b0 2024-04-16T17:18:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.874+0000 7f82bcc00640 1 -- 172.21.15.18:0/982538212 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82c001e020 con 0x557dd78c37b0 2024-04-16T17:18:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.875+0000 7f82dd031740 10 client.18948 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.875+0000 7f82dd031740 1 -- 172.21.15.18:0/982538212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557dd7662f70 con 0x557dd78c37b0 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.875+0000 7f82bcc00640 1 -- 172.21.15.18:0/982538212 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c0081030 con 0x557dd78c37b0 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.875+0000 7f82bcc00640 1 -- 172.21.15.18:0/982538212 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f82c009ace0 con 0x557dd78c37b0 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.875+0000 7f82dd031740 10 client.18948 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.876+0000 7f82dd031740 10 client.18948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.876+0000 7f82dd031740 20 client.18948 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.876+0000 7f82dd031740 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557dd78dc5d0 0x557dd78de9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.876+0000 7f82dd031740 4 client.18948 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.876+0000 7f82dd031740 1 -- 172.21.15.18:0/982538212 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dd78356d0 con 0x557dd78dc5d0 2024-04-16T17:18:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.877+0000 7f82d3400640 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557dd78dc5d0 0x557dd78de9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.877+0000 7f82d3400640 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557dd78dc5d0 0x557dd78de9c0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750f0586862d2cb2 server_cookie=2b565c3c382a0d18 in_seq=0 out_seq=0 2024-04-16T17:18:26.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.877+0000 7f82bcc00640 10 client.18948 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.880+0000 7f82bcc00640 1 -- 172.21.15.18:0/982538212 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557dd78356d0 con 0x557dd78dc5d0 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.880+0000 7f82bcc00640 10 client.18948 handle_command_reply: tid=0 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.880+0000 7f82dd031740 1 client.18948 shutdown 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 -- 172.21.15.18:0/982538212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 msgr2=0x557dd78c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78c9930 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f82c00a3f60 tx=0x7f82c0017ea0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 -- 172.21.15.18:0/982538212 shutdown_connections 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557dd78dc5d0 0x557dd78de9c0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82bcc00640 0 client.18948 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd78c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78c9930 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 --2- 172.21.15.18:0/982538212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 -- 172.21.15.18:0/982538212 >> 172.21.15.18:0/982538212 conn(0x557dd78bd5c0 msgr2=0x557dd78bf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:26.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 -- 172.21.15.18:0/982538212 shutdown_connections 2024-04-16T17:18:26.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 1 -- 172.21.15.18:0/982538212 wait complete. 2024-04-16T17:18:26.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.881+0000 7f82dd031740 20 client.18948 trim_cache size 0 max 16384 2024-04-16T17:18:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.921+0000 7f82dd031740 1 Processor -- start 2024-04-16T17:18:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.921+0000 7f82dd031740 1 -- start start 2024-04-16T17:18:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.921+0000 7f82dd031740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 0x557dd78c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.921+0000 7f82dd031740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78c37b0 0x557dd78d8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.921+0000 7f82dd031740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78cc390 0x557dd78c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.921+0000 7f82dd031740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f82b40129a0 con 0x557dd78c2e90 2024-04-16T17:18:26.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.921+0000 7f82dd031740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4010210 con 0x557dd78c37b0 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.921+0000 7f82dd031740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400e080 con 0x557dd78cc390 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78c37b0 0x557dd78d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78c37b0 0x557dd78d8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54682/0 (socket says 172.21.15.18:54682) 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d2a00640 1 -- 172.21.15.18:0/4108376809 learned_addr learned my addr 172.21.15.18:0/4108376809 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d3400640 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 0x557dd78c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d3e00640 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78cc390 0x557dd78c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d2a00640 1 -- 172.21.15.18:0/4108376809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78cc390 msgr2=0x557dd78c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d2a00640 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78cc390 0x557dd78c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d2a00640 1 -- 172.21.15.18:0/4108376809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 msgr2=0x557dd78c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d2a00640 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 0x557dd78c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d2a00640 1 -- 172.21.15.18:0/4108376809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd78c35e0 con 0x557dd78c37b0 2024-04-16T17:18:26.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d3400640 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 0x557dd78c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.922+0000 7f82d3e00640 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78cc390 0x557dd78c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.923+0000 7f82d2a00640 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78c37b0 0x557dd78d8040 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f82c009b520 tx=0x7f82c00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19ae04bb78ecf67 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.923+0000 7f82bf400640 1 -- 172.21.15.18:0/4108376809 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00b8070 con 0x557dd78c37b0 2024-04-16T17:18:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.923+0000 7f82bf400640 1 -- 172.21.15.18:0/4108376809 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c0081020 con 0x557dd78c37b0 2024-04-16T17:18:26.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.923+0000 7f82bf400640 1 -- 172.21.15.18:0/4108376809 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c009cdb0 con 0x557dd78c37b0 2024-04-16T17:18:26.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 -- 172.21.15.18:0/4108376809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78c37b0 msgr2=0x557dd78d8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78c37b0 0x557dd78d8040 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f82c009b520 tx=0x7f82c00046c0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 -- 172.21.15.18:0/4108376809 shutdown_connections 2024-04-16T17:18:26.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78cc390 0x557dd78c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78c37b0 0x557dd78d8040 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 --2- 172.21.15.18:0/4108376809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c2e90 0x557dd78c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 -- 172.21.15.18:0/4108376809 >> 172.21.15.18:0/4108376809 conn(0x557dd78bd5c0 msgr2=0x557dd78bc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 -- 172.21.15.18:0/4108376809 shutdown_connections 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 -- 172.21.15.18:0/4108376809 wait complete. 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.924+0000 7f82dd031740 1 -- start start 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.925+0000 7f82dd031740 1 Processor -- start 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.925+0000 7f82dd031740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.925+0000 7f82dd031740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.925+0000 7f82dd031740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd793b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.925+0000 7f82dd031740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400e6d0 con 0x557dd78c37b0 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.925+0000 7f82dd031740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400fb40 con 0x557dd78cc390 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.925+0000 7f82dd031740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400eda0 con 0x557dd78c2e90 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.925+0000 7f82dd031740 10 client.0 _ll_register_callbacks cb 0x7f82c00ccf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd793b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd793b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54696/0 (socket says 172.21.15.18:54696) 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3e00640 1 -- 172.21.15.18:0/2472588930 learned_addr learned my addr 172.21.15.18:0/2472588930 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3400640 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d2a00640 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3400640 1 -- 172.21.15.18:0/2472588930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 msgr2=0x557dd793b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:26.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3400640 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd793b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3400640 1 -- 172.21.15.18:0/2472588930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 msgr2=0x557dd78d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3400640 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d3400640 1 -- 172.21.15.18:0/2472588930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd74fa170 con 0x557dd78c2e90 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.926+0000 7f82d2a00640 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82d3400640 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78d0840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f82c4000f80 tx=0x7f82c4003c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54a8b018cb0ba15 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82bcc00640 1 -- 172.21.15.18:0/2472588930 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4039070 con 0x557dd78c2e90 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82bcc00640 1 -- 172.21.15.18:0/2472588930 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f82c4004040 con 0x557dd78c2e90 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82dd031740 1 -- 172.21.15.18:0/2472588930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c0081020 con 0x557dd78c2e90 2024-04-16T17:18:26.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82bcc00640 1 -- 172.21.15.18:0/2472588930 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82c4003e40 con 0x557dd78c2e90 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82dd031740 10 client.29173 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82dd031740 1 -- 172.21.15.18:0/2472588930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557dd7943020 con 0x557dd78c2e90 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.927+0000 7f82bcc00640 1 -- 172.21.15.18:0/2472588930 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c4059070 con 0x557dd78c2e90 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.928+0000 7f82bcc00640 1 -- 172.21.15.18:0/2472588930 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f82c404b040 con 0x557dd78c2e90 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.928+0000 7f82dd031740 10 client.29173 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.928+0000 7f82dd031740 10 client.29173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.928+0000 7f82dd031740 20 client.29173 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.928+0000 7f82dd031740 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557dd7943820 0x557dd7945c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.928+0000 7f82dd031740 4 client.29173 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.928+0000 7f82dd031740 1 -- 172.21.15.18:0/2472588930 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f82c0081020 con 0x557dd7943820 2024-04-16T17:18:26.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.928+0000 7f82d2a00640 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557dd7943820 0x557dd7945c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.929+0000 7f82d2a00640 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557dd7943820 0x557dd7945c40 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c81586bcdc6c6e server_cookie=728720dcc0e2549b in_seq=0 out_seq=0 2024-04-16T17:18:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.929+0000 7f82bcc00640 10 client.29173 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82bcc00640 1 -- 172.21.15.18:0/2472588930 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f82c0081020 con 0x557dd7943820 2024-04-16T17:18:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82bcc00640 10 client.29173 handle_command_reply: tid=0 2024-04-16T17:18:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 client.29173 shutdown 2024-04-16T17:18:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 -- 172.21.15.18:0/2472588930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 msgr2=0x557dd78d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78d0840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f82c4000f80 tx=0x7f82c4003c50 comp rx=0 tx=0).stop 2024-04-16T17:18:26.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 -- 172.21.15.18:0/2472588930 shutdown_connections 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557dd7943820 0x557dd7945c40 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dd78cc390 0x557dd793b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557dd78c37b0 0x557dd78d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82bcc00640 0 client.29173 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 --2- 172.21.15.18:0/2472588930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557dd78c2e90 0x557dd78d0840 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 -- 172.21.15.18:0/2472588930 >> 172.21.15.18:0/2472588930 conn(0x557dd78bd5c0 msgr2=0x557dd78be780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 -- 172.21.15.18:0/2472588930 shutdown_connections 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.931+0000 7f82dd031740 1 -- 172.21.15.18:0/2472588930 wait complete. 2024-04-16T17:18:26.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.932+0000 7f82dd031740 20 client.29173 trim_cache size 0 max 16384 2024-04-16T17:18:26.936 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:18:26.936 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:18:26.936 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "d9270267-05e9-40ea-a232-390b6c2226be", 2024-04-16T17:18:26.936 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:18:26.936 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:18:26.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.936+0000 7f82d8c00640 1 -- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f82b80a3e20 msgr2=0x7f82b80a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.936+0000 7f82d8c00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f82b80a3e20 0x7f82b80a62e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f82c402c700 tx=0x7f82c4032590 comp rx=0 tx=0).stop 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.936+0000 7f82d8c00640 1 -- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d410ec00 msgr2=0x7f82d41a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.936+0000 7f82d8c00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d410ec00 0x7f82d41a7a70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f82c000b4a0 tx=0x7f82c000b960 comp rx=0 tx=0).stop 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.937+0000 7f82d8c00640 1 -- 172.21.15.18:0/3652087213 shutdown_connections 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.937+0000 7f82d8c00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f82b80a3e20 0x7f82b80a62e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.937+0000 7f82d8c00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d4110580 0x7f82d41abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.937+0000 7f82d8c00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82d410ec00 0x7f82d41a7a70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.937+0000 7f82d8c00640 1 --2- 172.21.15.18:0/3652087213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82d410e0e0 0x7f82d41a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.937+0000 7f82d8c00640 1 -- 172.21.15.18:0/3652087213 >> 172.21.15.18:0/3652087213 conn(0x7f82d406cf10 msgr2=0x7f82d40713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.937+0000 7f82d8c00640 1 -- 172.21.15.18:0/3652087213 shutdown_connections 2024-04-16T17:18:26.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:26.937+0000 7f82d8c00640 1 -- 172.21.15.18:0/3652087213 wait complete. 2024-04-16T17:18:26.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:18:26.957 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:18:27.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.219+0000 7fdd38000640 1 -- 172.21.15.18:0/1784419891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 msgr2=0x7fdd301035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.219+0000 7fdd38000640 1 --2- 172.21.15.18:0/1784419891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 0x7fdd301035f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009900 tx=0x7fdd2402bcf0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.221+0000 7fdd38000640 1 -- 172.21.15.18:0/1784419891 shutdown_connections 2024-04-16T17:18:27.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.221+0000 7fdd38000640 1 --2- 172.21.15.18:0/1784419891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd3010c5e0 0x7fdd3010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.221+0000 7fdd38000640 1 --2- 172.21.15.18:0/1784419891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd30103b30 0x7fdd3010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.221+0000 7fdd38000640 1 --2- 172.21.15.18:0/1784419891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 0x7fdd301035f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.221+0000 7fdd38000640 1 -- 172.21.15.18:0/1784419891 >> 172.21.15.18:0/1784419891 conn(0x7fdd30078110 msgr2=0x7fdd30078520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:27.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.223+0000 7fdd38000640 1 -- 172.21.15.18:0/1784419891 shutdown_connections 2024-04-16T17:18:27.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.223+0000 7fdd38000640 1 -- 172.21.15.18:0/1784419891 wait complete. 2024-04-16T17:18:27.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.224+0000 7fdd38000640 1 Processor -- start 2024-04-16T17:18:27.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.224+0000 7fdd38000640 1 -- start start 2024-04-16T17:18:27.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.224+0000 7fdd38000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 0x7fdd30104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.224+0000 7fdd38000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd30103b30 0x7fdd301045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.224+0000 7fdd38000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd3010c5e0 0x7fdd30104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.224+0000 7fdd38000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd30112460 con 0x7fdd3010c5e0 2024-04-16T17:18:27.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd38000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd301122e0 con 0x7fdd30103210 2024-04-16T17:18:27.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd38000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd301125e0 con 0x7fdd30103b30 2024-04-16T17:18:27.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd35600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 0x7fdd30104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd35600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 0x7fdd30104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54726/0 (socket says 172.21.15.18:54726) 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd35600640 1 -- 172.21.15.18:0/1964251221 learned_addr learned my addr 172.21.15.18:0/1964251221 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd34c00640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd30103b30 0x7fdd301045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd36000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd3010c5e0 0x7fdd30104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd35600640 1 -- 172.21.15.18:0/1964251221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd30103b30 msgr2=0x7fdd301045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd35600640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd30103b30 0x7fdd301045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd35600640 1 -- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd3010c5e0 msgr2=0x7fdd30104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd35600640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd3010c5e0 0x7fdd30104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd35600640 1 -- 172.21.15.18:0/1964251221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3019a870 con 0x7fdd30103210 2024-04-16T17:18:27.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.225+0000 7fdd36000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd3010c5e0 0x7fdd30104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:27.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.226+0000 7fdd35600640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 0x7fdd30104080 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdd2402c300 tx=0x7fdd240045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:27.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.226+0000 7fdd2e000640 1 -- 172.21.15.18:0/1964251221 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24043020 con 0x7fdd30103210 2024-04-16T17:18:27.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.226+0000 7fdd38000640 1 -- 172.21.15.18:0/1964251221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3019ab00 con 0x7fdd30103210 2024-04-16T17:18:27.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.226+0000 7fdd2e000640 1 -- 172.21.15.18:0/1964251221 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd240054e0 con 0x7fdd30103210 2024-04-16T17:18:27.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.226+0000 7fdd2e000640 1 -- 172.21.15.18:0/1964251221 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24003da0 con 0x7fdd30103210 2024-04-16T17:18:27.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.227+0000 7fdd38000640 1 -- 172.21.15.18:0/1964251221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3019b0a0 con 0x7fdd30103210 2024-04-16T17:18:27.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.230+0000 7fdd3ad9a740 1 Processor -- start 2024-04-16T17:18:27.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.231+0000 7fdd3ad9a740 1 -- start start 2024-04-16T17:18:27.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.231+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a33318d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.231+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318dad0 0x55a333196170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.231+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 0x55a333198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.231+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a3330f3120 con 0x55a3331966b0 2024-04-16T17:18:27.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.231+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a333099340 con 0x55a33318dad0 2024-04-16T17:18:27.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.231+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a3330ef0e0 con 0x55a33318d1b0 2024-04-16T17:18:27.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.232+0000 7fdd36000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a33318d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.232+0000 7fdd36000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a33318d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59830/0 (socket says 172.21.15.18:59830) 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.232+0000 7fdd36000640 1 -- 172.21.15.18:0/3075062623 learned_addr learned my addr 172.21.15.18:0/3075062623 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.232+0000 7fdd34c00640 1 --2- 172.21.15.18:0/3075062623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 0x55a333198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.233+0000 7fdd2e000640 1 -- 172.21.15.18:0/1964251221 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd24046020 con 0x7fdd30103210 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.233+0000 7fdd34c00640 1 -- 172.21.15.18:0/3075062623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 msgr2=0x55a33318d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.233+0000 7fdd34c00640 1 --2- 172.21.15.18:0/3075062623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a33318d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.233+0000 7fdd34c00640 1 -- 172.21.15.18:0/3075062623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318dad0 msgr2=0x55a333196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.233+0000 7fdd34c00640 1 --2- 172.21.15.18:0/3075062623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318dad0 0x55a333196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.233+0000 7fdd34c00640 1 -- 172.21.15.18:0/3075062623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a332ecf7d0 con 0x55a3331966b0 2024-04-16T17:18:27.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.233+0000 7fdd34c00640 1 --2- 172.21.15.18:0/3075062623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 0x55a333198aa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd301058d0 tx=0x7fdd1c00bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255e1cf2f60560df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:27.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.234+0000 7fdd22600640 1 -- 172.21.15.18:0/3075062623 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00be10 con 0x55a3331966b0 2024-04-16T17:18:27.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.234+0000 7fdd22600640 1 -- 172.21.15.18:0/3075062623 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd1c019070 con 0x55a3331966b0 2024-04-16T17:18:27.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.234+0000 7fdd22600640 1 -- 172.21.15.18:0/3075062623 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0123e0 con 0x55a3331966b0 2024-04-16T17:18:27.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.235+0000 7fdd2e000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd180a3e60 0x7fdd180a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.235+0000 7fdd2e000640 1 -- 172.21.15.18:0/1964251221 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fdd240f2da0 con 0x7fdd30103210 2024-04-16T17:18:27.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.236+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3075062623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 msgr2=0x55a333198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.236+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3075062623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 0x55a333198aa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd301058d0 tx=0x7fdd1c00bbc0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.236+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3075062623 shutdown_connections 2024-04-16T17:18:27.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.236+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3075062623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 0x55a333198aa0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.236+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3075062623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318dad0 0x55a333196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.236+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3075062623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a33318d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.236+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3075062623 >> 172.21.15.18:0/3075062623 conn(0x55a3331878e0 msgr2=0x55a333187cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:27.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.236+0000 7fdd36000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd180a3e60 0x7fdd180a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.238+0000 7fdd36000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd180a3e60 0x7fdd180a6320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd28010430 tx=0x7fdd28008730 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:27.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.238+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3075062623 shutdown_connections 2024-04-16T17:18:27.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.238+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3075062623 wait complete. 2024-04-16T17:18:27.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.239+0000 7fdd3ad9a740 1 -- start start 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.239+0000 7fdd3ad9a740 1 Processor -- start 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.239+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318d1b0 0x55a3331936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.239+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a3331966b0 0x55a333193bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.239+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a333194130 0x55a3331b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.239+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a3331aa4b0 con 0x55a333194130 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.239+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a3330ac710 con 0x55a33318d1b0 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.239+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a3331aa630 con 0x55a3331966b0 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd3ad9a740 10 client.0 _ll_register_callbacks cb 0x55a3330a7af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd35600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318d1b0 0x55a3331936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd34c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a3331966b0 0x55a333193bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd35600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318d1b0 0x55a3331936b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54752/0 (socket says 172.21.15.18:54752) 2024-04-16T17:18:27.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd34c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a3331966b0 0x55a333193bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59838/0 (socket says 172.21.15.18:59838) 2024-04-16T17:18:27.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd35600640 1 -- 172.21.15.18:0/1573207303 learned_addr learned my addr 172.21.15.18:0/1573207303 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:27.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd35600640 1 -- 172.21.15.18:0/1573207303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a3331966b0 msgr2=0x55a333193bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:27.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd35600640 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a3331966b0 0x55a333193bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd35600640 1 -- 172.21.15.18:0/1573207303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a333194130 msgr2=0x55a3331b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:27.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd35600640 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a333194130 0x55a3331b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.240+0000 7fdd35600640 1 -- 172.21.15.18:0/1573207303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a332e053f0 con 0x55a33318d1b0 2024-04-16T17:18:27.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.241+0000 7fdd35600640 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318d1b0 0x55a3331936b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdd24004650 tx=0x7fdd240bad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36d9f3b662b791d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:27.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.241+0000 7fdd16a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:27.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.241+0000 7fdd16a00640 1 -- 172.21.15.18:0/1573207303 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd240bcb30 con 0x55a33318d1b0 2024-04-16T17:18:27.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.241+0000 7fdd16a00640 1 -- 172.21.15.18:0/1573207303 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fdd240bec50 con 0x55a33318d1b0 2024-04-16T17:18:27.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.241+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/1573207303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd1c019070 con 0x55a33318d1b0 2024-04-16T17:18:27.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.241+0000 7fdd16a00640 1 -- 172.21.15.18:0/1573207303 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdd240f0520 con 0x55a33318d1b0 2024-04-16T17:18:27.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.241+0000 7fdd3ad9a740 10 client.29357 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.241+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/1573207303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a332dc50f0 con 0x55a33318d1b0 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd16a00640 1 -- 172.21.15.18:0/1573207303 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd1c019070 con 0x55a33318d1b0 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd16a00640 1 -- 172.21.15.18:0/1573207303 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fdd240c71d0 con 0x55a33318d1b0 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd3ad9a740 10 client.29357 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd3ad9a740 10 client.29357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd3ad9a740 20 client.29357 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a3331b7ee0 0x55a3331ba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd3ad9a740 4 client.29357 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/1573207303 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a333100b40 con 0x55a3331b7ee0 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.242+0000 7fdd34c00640 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a3331b7ee0 0x55a3331ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.243+0000 7fdd34c00640 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a3331b7ee0 0x55a3331ba360 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0766d5cfc39836 server_cookie=c4ef37ee56c84f4a in_seq=0 out_seq=0 2024-04-16T17:18:27.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.243+0000 7fdd16a00640 10 client.29357 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:27.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.246+0000 7fdd16a00640 1 -- 172.21.15.18:0/1573207303 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a333100b40 con 0x55a3331b7ee0 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.246+0000 7fdd16a00640 10 client.29357 handle_command_reply: tid=0 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 client.29357 shutdown 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/1573207303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318d1b0 msgr2=0x55a3331936b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318d1b0 0x55a3331936b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdd24004650 tx=0x7fdd240bad30 comp rx=0 tx=0).stop 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/1573207303 shutdown_connections 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a3331b7ee0 0x55a3331ba360 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a333194130 0x55a3331b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a3331966b0 0x55a333193bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/1573207303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a33318d1b0 0x55a3331936b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/1573207303 >> 172.21.15.18:0/1573207303 conn(0x55a3331878e0 msgr2=0x55a333189630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:27.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd16a00640 0 client.29357 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:27.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/1573207303 shutdown_connections 2024-04-16T17:18:27.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.247+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/1573207303 wait complete. 2024-04-16T17:18:27.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.248+0000 7fdd3ad9a740 20 client.29357 trim_cache size 0 max 16384 2024-04-16T17:18:27.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.280+0000 7fdd3ad9a740 1 Processor -- start 2024-04-16T17:18:27.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.281+0000 7fdd3ad9a740 1 -- start start 2024-04-16T17:18:27.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.281+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3331878e0 0x55a333191f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.281+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a33318d1b0 0x55a333186750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.281+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a333194130 0x55a3331bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.281+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100129a0 con 0x55a33318d1b0 2024-04-16T17:18:27.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.281+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10010210 con 0x55a3331878e0 2024-04-16T17:18:27.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.281+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1000e080 con 0x55a333194130 2024-04-16T17:18:27.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd35600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3331878e0 0x55a333191f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd35600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3331878e0 0x55a333191f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54768/0 (socket says 172.21.15.18:54768) 2024-04-16T17:18:27.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd35600640 1 -- 172.21.15.18:0/2807381630 learned_addr learned my addr 172.21.15.18:0/2807381630 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd36000640 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a333194130 0x55a3331bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd34c00640 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a33318d1b0 0x55a333186750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd35600640 1 -- 172.21.15.18:0/2807381630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a333194130 msgr2=0x55a3331bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd35600640 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a333194130 0x55a3331bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd35600640 1 -- 172.21.15.18:0/2807381630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a33318d1b0 msgr2=0x55a333186750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd35600640 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a33318d1b0 0x55a333186750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd35600640 1 -- 172.21.15.18:0/2807381630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33318c410 con 0x55a3331878e0 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.282+0000 7fdd34c00640 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a33318d1b0 0x55a333186750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.283+0000 7fdd35600640 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3331878e0 0x55a333191f80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdd240bb2e0 tx=0x7fdd24005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b015de1e1736bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.283+0000 7fdd21c00640 1 -- 172.21.15.18:0/2807381630 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd240d9020 con 0x55a3331878e0 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.283+0000 7fdd21c00640 1 -- 172.21.15.18:0/2807381630 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd240a2020 con 0x55a3331878e0 2024-04-16T17:18:27.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.283+0000 7fdd21c00640 1 -- 172.21.15.18:0/2807381630 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd240ddbb0 con 0x55a3331878e0 2024-04-16T17:18:27.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.285+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/2807381630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3331878e0 msgr2=0x55a333191f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.285+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3331878e0 0x55a333191f80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdd240bb2e0 tx=0x7fdd24005fd0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.285+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/2807381630 shutdown_connections 2024-04-16T17:18:27.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.285+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a333194130 0x55a3331bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.285+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a33318d1b0 0x55a333186750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.285+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/2807381630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3331878e0 0x55a333191f80 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.285+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/2807381630 >> 172.21.15.18:0/2807381630 conn(0x55a333193d50 msgr2=0x55a333186040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:27.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.286+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/2807381630 shutdown_connections 2024-04-16T17:18:27.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.286+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/2807381630 wait complete. 2024-04-16T17:18:27.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.287+0000 7fdd3ad9a740 1 -- start start 2024-04-16T17:18:27.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.287+0000 7fdd3ad9a740 1 Processor -- start 2024-04-16T17:18:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.289+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a3331af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.289+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a333194130 0x55a3331afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd3ad9a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 0x55a3331abe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1000e6d0 con 0x55a3331966b0 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1000fb40 con 0x55a333194130 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd3ad9a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1000eda0 con 0x55a33318d1b0 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd3ad9a740 10 client.0 _ll_register_callbacks cb 0x7fdd24009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd35600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a3331af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd35600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a3331af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59858/0 (socket says 172.21.15.18:59858) 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd35600640 1 -- 172.21.15.18:0/3537768755 learned_addr learned my addr 172.21.15.18:0/3537768755 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.290+0000 7fdd34c00640 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a333194130 0x55a3331afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.291+0000 7fdd34c00640 1 -- 172.21.15.18:0/3537768755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 msgr2=0x55a3331af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.291+0000 7fdd34c00640 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a3331af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.291+0000 7fdd34c00640 1 -- 172.21.15.18:0/3537768755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 msgr2=0x55a3331abe00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.291+0000 7fdd34c00640 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 0x55a3331abe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.291+0000 7fdd34c00640 1 -- 172.21.15.18:0/3537768755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3331ac600 con 0x55a333194130 2024-04-16T17:18:27.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.292+0000 7fdd34c00640 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a333194130 0x55a3331afe80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000c00 tx=0x7fdd1c006b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4819fac022ff210 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:27.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.292+0000 7fdd16a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:27.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.292+0000 7fdd16a00640 1 -- 172.21.15.18:0/3537768755 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c014aa0 con 0x55a333194130 2024-04-16T17:18:27.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.292+0000 7fdd16a00640 1 -- 172.21.15.18:0/3537768755 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fdd1c025b50 con 0x55a333194130 2024-04-16T17:18:27.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.293+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3537768755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd240a2020 con 0x55a333194130 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.293+0000 7fdd16a00640 1 -- 172.21.15.18:0/3537768755 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdd1c025d50 con 0x55a333194130 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.293+0000 7fdd3ad9a740 10 client.29363 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.293+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3537768755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a333199120 con 0x55a333194130 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd16a00640 1 -- 172.21.15.18:0/3537768755 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd240a2020 con 0x55a333194130 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd16a00640 1 -- 172.21.15.18:0/3537768755 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fdd1c013070 con 0x55a333194130 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd3ad9a740 10 client.29363 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd3ad9a740 10 client.29363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd3ad9a740 20 client.29363 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a333199620 0x55a33319ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd3ad9a740 4 client.29363 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3537768755 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a3330fa9b0 con 0x55a333199620 2024-04-16T17:18:27.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.294+0000 7fdd35600640 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a333199620 0x55a33319ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:27.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.295+0000 7fdd16a00640 10 client.29363 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:27.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.295+0000 7fdd35600640 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a333199620 0x55a33319ba40 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574e8ab46dafd5d4 server_cookie=dbe0b2dd0a7e3ce9 in_seq=0 out_seq=0 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.297+0000 7fdd16a00640 1 -- 172.21.15.18:0/3537768755 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55a3330fa9b0 con 0x55a333199620 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.297+0000 7fdd16a00640 10 client.29363 handle_command_reply: tid=0 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.297+0000 7fdd3ad9a740 1 client.29363 shutdown 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a333194130 msgr2=0x55a3331afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a333194130 0x55a3331afe80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000c00 tx=0x7fdd1c006b70 comp rx=0 tx=0).stop 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3537768755 shutdown_connections 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a333199620 0x55a33319ba40 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3331966b0 0x55a3331abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a333194130 0x55a3331afe80 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 --2- 172.21.15.18:0/3537768755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a33318d1b0 0x55a3331af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3537768755 >> 172.21.15.18:0/3537768755 conn(0x55a333193d50 msgr2=0x55a333186040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd16a00640 0 client.29363 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.298+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3537768755 shutdown_connections 2024-04-16T17:18:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.299+0000 7fdd3ad9a740 1 -- 172.21.15.18:0/3537768755 wait complete. 2024-04-16T17:18:27.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.299+0000 7fdd3ad9a740 20 client.29363 trim_cache size 0 max 16384 2024-04-16T17:18:27.300 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:18:27.300 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (581 inodes in the stack)", 2024-04-16T17:18:27.301 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:18:27.301 INFO:teuthology.orchestra.run.smithi018.stdout: "d9270267-05e9-40ea-a232-390b6c2226be": { 2024-04-16T17:18:27.301 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:18:27.301 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "d9270267-05e9-40ea-a232-390b6c2226be", 2024-04-16T17:18:27.301 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:18:27.301 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:18:27.301 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:18:27.301 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:18:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.300+0000 7fdd38000640 1 -- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd180a3e60 msgr2=0x7fdd180a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.300+0000 7fdd38000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd180a3e60 0x7fdd180a6320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd28010430 tx=0x7fdd28008730 comp rx=0 tx=0).stop 2024-04-16T17:18:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.300+0000 7fdd38000640 1 -- 172.21.15.18:0/1964251221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 msgr2=0x7fdd30104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.300+0000 7fdd38000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 0x7fdd30104080 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdd2402c300 tx=0x7fdd240045e0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.302+0000 7fdd38000640 1 -- 172.21.15.18:0/1964251221 shutdown_connections 2024-04-16T17:18:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.302+0000 7fdd38000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdd180a3e60 0x7fdd180a6320 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.302+0000 7fdd38000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd3010c5e0 0x7fdd30104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.302+0000 7fdd38000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd30103b30 0x7fdd301045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.302+0000 7fdd38000640 1 --2- 172.21.15.18:0/1964251221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd30103210 0x7fdd30104080 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:27.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.302+0000 7fdd38000640 1 -- 172.21.15.18:0/1964251221 >> 172.21.15.18:0/1964251221 conn(0x7fdd30078110 msgr2=0x7fdd30101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:27.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.302+0000 7fdd38000640 1 -- 172.21.15.18:0/1964251221 shutdown_connections 2024-04-16T17:18:27.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:27.303+0000 7fdd38000640 1 -- 172.21.15.18:0/1964251221 wait complete. 2024-04-16T17:18:27.330 INFO:tasks.workunit.client.0.smithi018.stdout: 10 10688 19.42 MB/sec warmup 39 sec latency 187.542 ms 2024-04-16T17:18:27.342 INFO:tasks.cephfs.filesystem:scrub status for tag:d9270267-05e9-40ea-a232-390b6c2226be - {'path': '/', 'tag': 'd9270267-05e9-40ea-a232-390b6c2226be', 'options': 'recursive,force'} 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: pgmap v436: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:18:28.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:27 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 -- 172.21.15.18:0/2724241186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 msgr2=0x7f87a0112960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 --2- 172.21.15.18:0/2724241186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 0x7f87a0112960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f878c009930 tx=0x7f878c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 -- 172.21.15.18:0/2724241186 shutdown_connections 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 --2- 172.21.15.18:0/2724241186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a0112ea0 0x7f87a0115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 --2- 172.21.15.18:0/2724241186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 0x7f87a0112960 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 --2- 172.21.15.18:0/2724241186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a0076530 0x7f87a0076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 -- 172.21.15.18:0/2724241186 >> 172.21.15.18:0/2724241186 conn(0x7f87a00ffcf0 msgr2=0x7f87a0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 -- 172.21.15.18:0/2724241186 shutdown_connections 2024-04-16T17:18:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.054+0000 7f87a5c00640 1 -- 172.21.15.18:0/2724241186 wait complete. 2024-04-16T17:18:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.056+0000 7f87a5c00640 1 Processor -- start 2024-04-16T17:18:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.056+0000 7f87a5c00640 1 -- start start 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f87a5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a0076530 0x7f87a019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f87a5c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 0x7f87a019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f87a5c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a0112ea0 0x7f87a01a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f87a5c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0118a00 con 0x7f87a0112ea0 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f87a5c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0118880 con 0x7f87a0076e50 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f87a5c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0118b80 con 0x7f87a0076530 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a0076530 0x7f87a019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a0076530 0x7f87a019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59876/0 (socket says 172.21.15.18:59876) 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879ea00640 1 -- 172.21.15.18:0/1183000456 learned_addr learned my addr 172.21.15.18:0/1183000456 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879e000640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 0x7f87a019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879f400640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a0112ea0 0x7f87a01a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879ea00640 1 -- 172.21.15.18:0/1183000456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 msgr2=0x7f87a019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879ea00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 0x7f87a019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879ea00640 1 -- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a0112ea0 msgr2=0x7f87a01a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879ea00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a0112ea0 0x7f87a01a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879ea00640 1 -- 172.21.15.18:0/1183000456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a01a3f40 con 0x7f87a0076530 2024-04-16T17:18:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879f400640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a0112ea0 0x7f87a01a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.057+0000 7f879e000640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 0x7f87a019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.058+0000 7f879ea00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a0076530 0x7f87a019efa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f879000ea60 tx=0x7f879000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.058+0000 7f8797400640 1 -- 172.21.15.18:0/1183000456 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879000cde0 con 0x7f87a0076530 2024-04-16T17:18:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.058+0000 7f8797400640 1 -- 172.21.15.18:0/1183000456 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87900035f0 con 0x7f87a0076530 2024-04-16T17:18:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.058+0000 7f8797400640 1 -- 172.21.15.18:0/1183000456 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790010670 con 0x7f87a0076530 2024-04-16T17:18:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.060+0000 7f87a5c00640 1 -- 172.21.15.18:0/1183000456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a01a4230 con 0x7f87a0076530 2024-04-16T17:18:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.060+0000 7f87a5c00640 1 -- 172.21.15.18:0/1183000456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01abac0 con 0x7f87a0076530 2024-04-16T17:18:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f87a880f740 1 Processor -- start 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f87a880f740 1 -- start start 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f8797400640 1 -- 172.21.15.18:0/1183000456 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f87900037a0 con 0x7f87a0076530 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f87a880f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfa7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f87a880f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfa7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f87a880f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfb3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f87a880f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557eebf0c780 con 0x557eebfb0e20 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f87a880f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557eebeb2f40 con 0x557eebfa8610 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.063+0000 7f87a880f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557eebf08740 con 0x557eebfa78a0 2024-04-16T17:18:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfa7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfa7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54818/0 (socket says 172.21.15.18:54818) 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879f400640 1 -- 172.21.15.18:0/2910637694 learned_addr learned my addr 172.21.15.18:0/2910637694 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879ea00640 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfa7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879e000640 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfb3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879f400640 1 -- 172.21.15.18:0/2910637694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 msgr2=0x557eebfa7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879f400640 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfa7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879f400640 1 -- 172.21.15.18:0/2910637694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 msgr2=0x557eebfb3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879f400640 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfb3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879f400640 1 -- 172.21.15.18:0/2910637694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eebc1f210 con 0x557eebfa8610 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879e000640 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfb3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.064+0000 7f879ea00640 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfa7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.065+0000 7f879f400640 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfa7360 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8798009e70 tx=0x7f879800e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646ce21cbf2fa0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.065+0000 7f878be00640 1 -- 172.21.15.18:0/2910637694 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798016070 con 0x557eebfa8610 2024-04-16T17:18:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.065+0000 7f878be00640 1 -- 172.21.15.18:0/2910637694 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879800ee90 con 0x557eebfa8610 2024-04-16T17:18:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.065+0000 7f878be00640 1 -- 172.21.15.18:0/2910637694 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798011760 con 0x557eebfa8610 2024-04-16T17:18:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f8797400640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87840a3d90 0x7f87840a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f87a880f740 1 -- 172.21.15.18:0/2910637694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 msgr2=0x557eebfa7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f87a880f740 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfa7360 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8798009e70 tx=0x7f879800e6c0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f87a880f740 1 -- 172.21.15.18:0/2910637694 shutdown_connections 2024-04-16T17:18:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f87a880f740 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfb3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f87a880f740 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfa7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f87a880f740 1 --2- 172.21.15.18:0/2910637694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfa7360 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f87a880f740 1 -- 172.21.15.18:0/2910637694 >> 172.21.15.18:0/2910637694 conn(0x557eebf425d0 msgr2=0x557eebfa2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f8797400640 1 -- 172.21.15.18:0/1183000456 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f87900d12f0 con 0x7f87a0076530 2024-04-16T17:18:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f87a880f740 1 -- 172.21.15.18:0/2910637694 shutdown_connections 2024-04-16T17:18:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.066+0000 7f879ea00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87840a3d90 0x7f87840a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.067+0000 7f87a880f740 1 -- 172.21.15.18:0/2910637694 wait complete. 2024-04-16T17:18:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.067+0000 7f879ea00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87840a3d90 0x7f87840a6250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x557eebfa8d70 tx=0x7f8790003f30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.067+0000 7f87a880f740 1 -- start start 2024-04-16T17:18:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.068+0000 7f87a880f740 1 Processor -- start 2024-04-16T17:18:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.068+0000 7f87a880f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfb0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.068+0000 7f87a880f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfadc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.068+0000 7f87a880f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.068+0000 7f87a880f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557eebfb3b50 con 0x557eebfb0e20 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.068+0000 7f87a880f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557eebec6390 con 0x557eebfa8610 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.068+0000 7f87a880f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557eebfb3cd0 con 0x557eebfa78a0 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.069+0000 7f87a880f740 10 client.0 _ll_register_callbacks cb 0x557eebeaff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.069+0000 7f879f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfb0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.069+0000 7f879f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfb0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:59890/0 (socket says 172.21.15.18:59890) 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.069+0000 7f879f400640 1 -- 172.21.15.18:0/2649562537 learned_addr learned my addr 172.21.15.18:0/2649562537 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.069+0000 7f879f400640 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.069+0000 7f879e000640 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfadc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.069+0000 7f879f400640 1 -- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 msgr2=0x557eebfadc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f879f400640 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfadc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f879f400640 1 -- 172.21.15.18:0/2649562537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 msgr2=0x557eebfae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f879f400640 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f879f400640 1 -- 172.21.15.18:0/2649562537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eebceb620 con 0x557eebfa78a0 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f879e000640 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfadc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f879f400640 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfb0880 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8798005b40 tx=0x7f8798005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2be34c517fc27460 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f8788c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f8788c00640 1 -- 172.21.15.18:0/2649562537 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798016070 con 0x557eebfa78a0 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.070+0000 7f87a880f740 1 -- 172.21.15.18:0/2649562537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f879800ee90 con 0x557eebfa78a0 2024-04-16T17:18:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.071+0000 7f8788c00640 1 -- 172.21.15.18:0/2649562537 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f8798005510 con 0x557eebfa78a0 2024-04-16T17:18:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.071+0000 7f8788c00640 1 -- 172.21.15.18:0/2649562537 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8798030050 con 0x557eebfa78a0 2024-04-16T17:18:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.071+0000 7f87a880f740 10 client.29200 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.071+0000 7f87a880f740 1 -- 172.21.15.18:0/2649562537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557eebfbfc00 con 0x557eebfa78a0 2024-04-16T17:18:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.071+0000 7f8788c00640 1 -- 172.21.15.18:0/2649562537 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879800ee90 con 0x557eebfa78a0 2024-04-16T17:18:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.073+0000 7f8788c00640 1 -- 172.21.15.18:0/2649562537 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f879800f040 con 0x557eebfa78a0 2024-04-16T17:18:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.073+0000 7f87a880f740 10 client.29200 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.074+0000 7f87a880f740 10 client.29200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.074+0000 7f87a880f740 20 client.29200 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.074+0000 7f87a880f740 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557eebfc1070 0x557eebfc3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.074+0000 7f87a880f740 4 client.29200 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.074+0000 7f87a880f740 1 -- 172.21.15.18:0/2649562537 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eebf1a1a0 con 0x557eebfc1070 2024-04-16T17:18:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.074+0000 7f879e000640 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557eebfc1070 0x557eebfc3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.075+0000 7f879e000640 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557eebfc1070 0x557eebfc3460 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b92484e535a9ea server_cookie=df5983e05e5f75fc in_seq=0 out_seq=0 2024-04-16T17:18:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.075+0000 7f8788c00640 10 client.29200 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f8788c00640 1 -- 172.21.15.18:0/2649562537 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eebf1a1a0 con 0x557eebfc1070 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f8788c00640 10 client.29200 handle_command_reply: tid=0 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 client.29200 shutdown 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 -- 172.21.15.18:0/2649562537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 msgr2=0x557eebfb0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfb0880 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8798005b40 tx=0x7f8798005a40 comp rx=0 tx=0).stop 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 -- 172.21.15.18:0/2649562537 shutdown_connections 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557eebfc1070 0x557eebfc3460 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfb0e20 0x557eebfae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfadc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 --2- 172.21.15.18:0/2649562537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfa78a0 0x557eebfb0880 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 -- 172.21.15.18:0/2649562537 >> 172.21.15.18:0/2649562537 conn(0x557eebf425d0 msgr2=0x557eebfa42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f8788c00640 0 client.29200 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 -- 172.21.15.18:0/2649562537 shutdown_connections 2024-04-16T17:18:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.081+0000 7f87a880f740 1 -- 172.21.15.18:0/2649562537 wait complete. 2024-04-16T17:18:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.082+0000 7f87a880f740 20 client.29200 trim_cache size 0 max 16384 2024-04-16T17:18:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f87a880f740 1 Processor -- start 2024-04-16T17:18:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f87a880f740 1 -- start start 2024-04-16T17:18:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f87a880f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfbca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f87a880f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfacbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f87a880f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfb0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f87a880f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b8580 con 0x557eebfa78a0 2024-04-16T17:18:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f87a880f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b5df0 con 0x557eebfa8610 2024-04-16T17:18:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f87a880f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b3cf0 con 0x557eebfb0e20 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f879e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfacbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f879f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfbca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f879e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfacbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54840/0 (socket says 172.21.15.18:54840) 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.099+0000 7f879e000640 1 -- 172.21.15.18:0/3995599532 learned_addr learned my addr 172.21.15.18:0/3995599532 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f879f400640 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfb0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f879e000640 1 -- 172.21.15.18:0/3995599532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 msgr2=0x557eebfb0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f879e000640 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfb0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f879e000640 1 -- 172.21.15.18:0/3995599532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 msgr2=0x557eebfbca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f879e000640 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfbca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f879e000640 1 -- 172.21.15.18:0/3995599532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eebfa6b70 con 0x557eebfa8610 2024-04-16T17:18:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f879f400640 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfb0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f879e000640 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfacbf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f878c005ab0 tx=0x7f878c02c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=822a4de7461a0ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.100+0000 7f878b400640 1 -- 172.21.15.18:0/3995599532 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c02cd40 con 0x557eebfa8610 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.101+0000 7f878b400640 1 -- 172.21.15.18:0/3995599532 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c005040 con 0x557eebfa8610 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.101+0000 7f878b400640 1 -- 172.21.15.18:0/3995599532 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c004b60 con 0x557eebfa8610 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.101+0000 7f87a880f740 1 -- 172.21.15.18:0/3995599532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 msgr2=0x557eebfacbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.101+0000 7f87a880f740 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfacbf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f878c005ab0 tx=0x7f878c02c6d0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.102+0000 7f87a880f740 1 -- 172.21.15.18:0/3995599532 shutdown_connections 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.102+0000 7f87a880f740 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfb0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.102+0000 7f87a880f740 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfacbf0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.102+0000 7f87a880f740 1 --2- 172.21.15.18:0/3995599532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfbca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.102+0000 7f87a880f740 1 -- 172.21.15.18:0/3995599532 >> 172.21.15.18:0/3995599532 conn(0x557eebf425d0 msgr2=0x557eebfac4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.102+0000 7f87a880f740 1 -- 172.21.15.18:0/3995599532 shutdown_connections 2024-04-16T17:18:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.102+0000 7f87a880f740 1 -- 172.21.15.18:0/3995599532 wait complete. 2024-04-16T17:18:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.102+0000 7f87a880f740 1 -- start start 2024-04-16T17:18:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.103+0000 7f87a880f740 1 Processor -- start 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.103+0000 7f87a880f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.103+0000 7f87a880f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfaa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.103+0000 7f87a880f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfaa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.103+0000 7f87a880f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b42b0 con 0x557eebfa78a0 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.103+0000 7f87a880f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b5720 con 0x557eebfa8610 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.103+0000 7f87a880f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b4980 con 0x557eebfb0e20 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.103+0000 7f87a880f740 10 client.0 _ll_register_callbacks cb 0x7f8798009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfb9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41538/0 (socket says 172.21.15.18:41538) 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 -- 172.21.15.18:0/3686719803 learned_addr learned my addr 172.21.15.18:0/3686719803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfaa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879e000640 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfaa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 -- 172.21.15.18:0/3686719803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 msgr2=0x557eebfaa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfaa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 -- 172.21.15.18:0/3686719803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 msgr2=0x557eebfaa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfaa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.104+0000 7f879f400640 1 -- 172.21.15.18:0/3686719803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eebfaaf30 con 0x557eebfa78a0 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.105+0000 7f879f400640 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfb9150 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8798005a10 tx=0x7f87980092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1d799955c27734 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.105+0000 7f8788c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.105+0000 7f8788c00640 1 -- 172.21.15.18:0/3686719803 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798016040 con 0x557eebfa78a0 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.105+0000 7f8788c00640 1 -- 172.21.15.18:0/3686719803 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f8798005540 con 0x557eebfa78a0 2024-04-16T17:18:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.106+0000 7f87a880f740 1 -- 172.21.15.18:0/3686719803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878c005040 con 0x557eebfa78a0 2024-04-16T17:18:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.106+0000 7f8788c00640 1 -- 172.21.15.18:0/3686719803 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f87980041d0 con 0x557eebfa78a0 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f87a880f740 10 client.18969 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f87a880f740 1 -- 172.21.15.18:0/3686719803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557eec0270f0 con 0x557eebfa78a0 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f8788c00640 1 -- 172.21.15.18:0/3686719803 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c005040 con 0x557eebfa78a0 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f8788c00640 1 -- 172.21.15.18:0/3686719803 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8798011470 con 0x557eebfa78a0 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f87a880f740 10 client.18969 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f87a880f740 10 client.18969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f87a880f740 20 client.18969 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f87a880f740 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557eec027780 0x557eec029ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f87a880f740 4 client.18969 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:18:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.108+0000 7f87a880f740 1 -- 172.21.15.18:0/3686719803 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557eebf14010 con 0x557eec027780 2024-04-16T17:18:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.109+0000 7f879e000640 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557eec027780 0x557eec029ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.109+0000 7f8788c00640 10 client.18969 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:28.109+0000 7f879e000640 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557eec027780 0x557eec029ba0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1069bac7d924a84d server_cookie=3951220b15e03645 in_seq=0 out_seq=0 2024-04-16T17:18:28.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: pgmap v436: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-16T17:18:28.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:18:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:18:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:18:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:18:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:18:28.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:27 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.333 INFO:tasks.workunit.client.0.smithi018.stdout: 10 10890 19.32 MB/sec warmup 40 sec latency 224.320 ms 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: pgmap v436: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:18:28.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:27 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:29.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:28 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.011+0000 7f8788c00640 1 -- 172.21.15.18:0/3686719803 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557eebf14010 con 0x557eec027780 2024-04-16T17:18:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.011+0000 7f8788c00640 10 client.18969 handle_command_reply: tid=0 2024-04-16T17:18:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.012+0000 7f87a880f740 1 client.18969 shutdown 2024-04-16T17:18:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.012+0000 7f87a880f740 1 -- 172.21.15.18:0/3686719803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 msgr2=0x557eebfb9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.012+0000 7f87a880f740 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfb9150 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8798005a10 tx=0x7f87980092c0 comp rx=0 tx=0).stop 2024-04-16T17:18:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.013+0000 7f87a880f740 1 -- 172.21.15.18:0/3686719803 shutdown_connections 2024-04-16T17:18:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.013+0000 7f87a880f740 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557eec027780 0x557eec029ba0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.013+0000 7f87a880f740 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eebfb0e20 0x557eebfaa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.013+0000 7f87a880f740 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557eebfa8610 0x557eebfaa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.013+0000 7f87a880f740 1 --2- 172.21.15.18:0/3686719803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557eebfa78a0 0x557eebfb9150 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.013+0000 7f87a880f740 1 -- 172.21.15.18:0/3686719803 >> 172.21.15.18:0/3686719803 conn(0x557eebf425d0 msgr2=0x557eebfa37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.013+0000 7f8788c00640 0 client.18969 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.013+0000 7f87a880f740 1 -- 172.21.15.18:0/3686719803 shutdown_connections 2024-04-16T17:18:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.014+0000 7f87a880f740 1 -- 172.21.15.18:0/3686719803 wait complete. 2024-04-16T17:18:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.014+0000 7f87a880f740 20 client.18969 trim_cache size 0 max 16384 2024-04-16T17:18:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:18:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:18:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:18:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 -- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87840a3d90 msgr2=0x7f87840a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87840a3d90 0x7f87840a6250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x557eebfa8d70 tx=0x7f8790003f30 comp rx=0 tx=0).stop 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 -- 172.21.15.18:0/1183000456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a0076530 msgr2=0x7f87a019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a0076530 0x7f87a019efa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f879000ea60 tx=0x7f879000ef20 comp rx=0 tx=0).stop 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 -- 172.21.15.18:0/1183000456 shutdown_connections 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87840a3d90 0x7f87840a6250 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x557eebfa8d70 tx=0x7f8790003f30 comp rx=0 tx=0).stop 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a0112ea0 0x7f87a01a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a0076e50 0x7f87a019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 --2- 172.21.15.18:0/1183000456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a0076530 0x7f87a019efa0 secure :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f879000ea60 tx=0x7f879000ef20 comp rx=0 tx=0).stop 2024-04-16T17:18:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.018+0000 7f87a5c00640 1 -- 172.21.15.18:0/1183000456 >> 172.21.15.18:0/1183000456 conn(0x7f87a00ffcf0 msgr2=0x7f87a0114630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.020+0000 7f87a5c00640 1 -- 172.21.15.18:0/1183000456 shutdown_connections 2024-04-16T17:18:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:29.020+0000 7f87a5c00640 1 -- 172.21.15.18:0/1183000456 wait complete. 2024-04-16T17:18:29.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:28 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:29.331 INFO:tasks.workunit.client.0.smithi018.stdout: 10 11035 18.97 MB/sec warmup 41 sec latency 425.053 ms 2024-04-16T17:18:30.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:29 smithi043 ceph-mon[36901]: pgmap v437: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-16T17:18:30.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:29 smithi022 ceph-mon[36023]: pgmap v437: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-16T17:18:30.330 INFO:tasks.workunit.client.0.smithi018.stdout: 10 11247 18.63 MB/sec warmup 42 sec latency 117.738 ms 2024-04-16T17:18:31.331 INFO:tasks.workunit.client.0.smithi018.stdout: 10 11476 18.37 MB/sec warmup 43 sec latency 125.985 ms 2024-04-16T17:18:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:31 smithi043 ceph-mon[36901]: pgmap v438: 129 pgs: 129 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-04-16T17:18:32.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:31 smithi022 ceph-mon[36023]: pgmap v438: 129 pgs: 129 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-04-16T17:18:32.331 INFO:tasks.workunit.client.0.smithi018.stdout: 10 11778 18.24 MB/sec warmup 44 sec latency 64.861 ms 2024-04-16T17:18:33.331 INFO:tasks.workunit.client.0.smithi018.stdout: 10 12303 18.34 MB/sec warmup 45 sec latency 86.093 ms 2024-04-16T17:18:34.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:33 smithi022 ceph-mon[36023]: pgmap v439: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 490 op/s 2024-04-16T17:18:34.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:33 smithi043 ceph-mon[36901]: pgmap v439: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 490 op/s 2024-04-16T17:18:34.331 INFO:tasks.workunit.client.0.smithi018.stdout: 10 12822 18.51 MB/sec warmup 46 sec latency 215.537 ms 2024-04-16T17:18:35.332 INFO:tasks.workunit.client.0.smithi018.stdout: 10 13287 18.69 MB/sec warmup 47 sec latency 177.781 ms 2024-04-16T17:18:36.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:35 smithi022 ceph-mon[36023]: pgmap v440: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 469 op/s 2024-04-16T17:18:36.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:35 smithi043 ceph-mon[36901]: pgmap v440: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 469 op/s 2024-04-16T17:18:36.333 INFO:tasks.workunit.client.0.smithi018.stdout: 10 13776 19.00 MB/sec warmup 48 sec latency 154.461 ms 2024-04-16T17:18:37.333 INFO:tasks.workunit.client.0.smithi018.stdout: 10 14135 19.16 MB/sec warmup 49 sec latency 173.419 ms 2024-04-16T17:18:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:37 smithi022 ceph-mon[36023]: pgmap v441: 129 pgs: 129 active+clean; 298 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-04-16T17:18:38.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:37 smithi043 ceph-mon[36901]: pgmap v441: 129 pgs: 129 active+clean; 298 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-04-16T17:18:38.332 INFO:tasks.workunit.client.0.smithi018.stdout: 10 14404 19.12 MB/sec warmup 50 sec latency 145.856 ms 2024-04-16T17:18:39.333 INFO:tasks.workunit.client.0.smithi018.stdout: 10 14660 18.97 MB/sec warmup 51 sec latency 104.312 ms 2024-04-16T17:18:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:39 smithi022 ceph-mon[36023]: pgmap v442: 129 pgs: 129 active+clean; 298 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 487 op/s 2024-04-16T17:18:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:39 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:18:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:39 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:18:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:39 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:18:40.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:39 smithi043 ceph-mon[36901]: pgmap v442: 129 pgs: 129 active+clean; 298 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 487 op/s 2024-04-16T17:18:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:39 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:18:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:39 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:18:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:39 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:18:40.333 INFO:tasks.workunit.client.0.smithi018.stdout: 10 14983 18.85 MB/sec warmup 52 sec latency 88.645 ms 2024-04-16T17:18:41.333 INFO:tasks.workunit.client.0.smithi018.stdout: 10 15250 18.74 MB/sec warmup 53 sec latency 218.738 ms 2024-04-16T17:18:41.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:28 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:18:41.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:29 smithi018 ceph-mon[31776]: pgmap v437: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-16T17:18:41.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:31 smithi018 ceph-mon[31776]: pgmap v438: 129 pgs: 129 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-04-16T17:18:41.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:33 smithi018 ceph-mon[31776]: pgmap v439: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 490 op/s 2024-04-16T17:18:41.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:35 smithi018 ceph-mon[31776]: pgmap v440: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 469 op/s 2024-04-16T17:18:41.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:37 smithi018 ceph-mon[31776]: pgmap v441: 129 pgs: 129 active+clean; 298 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 575 op/s 2024-04-16T17:18:41.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:39 smithi018 ceph-mon[31776]: pgmap v442: 129 pgs: 129 active+clean; 298 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 487 op/s 2024-04-16T17:18:41.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:39 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:18:41.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:39 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:18:41.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:39 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:18:41.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:41 smithi018 ceph-mon[31776]: pgmap v443: 129 pgs: 129 active+clean; 307 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 560 op/s 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 -- 172.21.15.18:0/3932237123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 msgr2=0x7f3a38115220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 --2- 172.21.15.18:0/3932237123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 0x7f3a38115220 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009be0 tx=0x7f3a3002bf80 comp rx=0 tx=0).stop 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 -- 172.21.15.18:0/3932237123 shutdown_connections 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 --2- 172.21.15.18:0/3932237123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 0x7f3a38115220 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 --2- 172.21.15.18:0/3932237123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a38074cc0 0x7f3a381128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 --2- 172.21.15.18:0/3932237123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a380743a0 0x7f3a38074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 -- 172.21.15.18:0/3932237123 >> 172.21.15.18:0/3932237123 conn(0x7f3a380fd910 msgr2=0x7f3a380ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 -- 172.21.15.18:0/3932237123 shutdown_connections 2024-04-16T17:18:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.150+0000 7f3a3da00640 1 -- 172.21.15.18:0/3932237123 wait complete. 2024-04-16T17:18:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.151+0000 7f3a3da00640 1 Processor -- start 2024-04-16T17:18:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.151+0000 7f3a3da00640 1 -- start start 2024-04-16T17:18:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3da00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a380743a0 0x7f3a381a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a38074cc0 0x7f3a381a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 0x7f3a3819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3da00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38118840 con 0x7f3a380743a0 2024-04-16T17:18:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3da00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381186c0 con 0x7f3a38112e30 2024-04-16T17:18:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381189c0 con 0x7f3a38074cc0 2024-04-16T17:18:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a37e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a380743a0 0x7f3a381a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 0x7f3a3819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a38074cc0 0x7f3a381a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 0x7f3a3819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:57578/0 (socket says 172.21.15.18:57578) 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.152+0000 7f3a3d000640 1 -- 172.21.15.18:0/2157940902 learned_addr learned my addr 172.21.15.18:0/2157940902 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a3d000640 1 -- 172.21.15.18:0/2157940902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a38074cc0 msgr2=0x7f3a381a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a3d000640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a38074cc0 0x7f3a381a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a3d000640 1 -- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a380743a0 msgr2=0x7f3a381a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a3d000640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a380743a0 0x7f3a381a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a3d000640 1 -- 172.21.15.18:0/2157940902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3819f2c0 con 0x7f3a38112e30 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a3d000640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 0x7f3a3819eb40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3a3002c4c0 tx=0x7f3a30004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a35600640 1 -- 172.21.15.18:0/2157940902 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30043020 con 0x7f3a38112e30 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a3da00640 1 -- 172.21.15.18:0/2157940902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3819f550 con 0x7f3a38112e30 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.153+0000 7f3a3da00640 1 -- 172.21.15.18:0/2157940902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381ab800 con 0x7f3a38112e30 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.154+0000 7f3a35600640 1 -- 172.21.15.18:0/2157940902 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a300045f0 con 0x7f3a38112e30 2024-04-16T17:18:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.154+0000 7f3a35600640 1 -- 172.21.15.18:0/2157940902 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3003e450 con 0x7f3a38112e30 2024-04-16T17:18:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.156+0000 7f3a41db2740 1 Processor -- start 2024-04-16T17:18:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.157+0000 7f3a35600640 1 -- 172.21.15.18:0/2157940902 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a30046020 con 0x7f3a38112e30 2024-04-16T17:18:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.158+0000 7f3a41db2740 1 -- start start 2024-04-16T17:18:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.158+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.158+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.158+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 0x56025c6b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.158+0000 7f3a41db2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56025c60a780 con 0x56025c6a67f0 2024-04-16T17:18:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.158+0000 7f3a41db2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56025c5b0f40 con 0x56025c6a5a80 2024-04-16T17:18:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.158+0000 7f3a41db2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56025c606740 con 0x56025c6af000 2024-04-16T17:18:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.158+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49430/0 (socket says 172.21.15.18:49430) 2024-04-16T17:18:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a2fe00640 1 -- 172.21.15.18:0/1999627837 learned_addr learned my addr 172.21.15.18:0/1999627837 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a37e00640 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 0x56025c6b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a37e00640 1 -- 172.21.15.18:0/1999627837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 msgr2=0x56025c6b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a37e00640 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 0x56025c6b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a37e00640 1 -- 172.21.15.18:0/1999627837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 msgr2=0x56025c6a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a37e00640 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a37e00640 1 -- 172.21.15.18:0/1999627837 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56025c31d210 con 0x56025c6a5a80 2024-04-16T17:18:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a35600640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a1c0a3d80 0x7f3a1c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.159+0000 7f3a37e00640 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6a5f00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009c20 tx=0x7f3a28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53f59704f173320 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.160+0000 7f3a35600640 1 -- 172.21.15.18:0/2157940902 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f3a300f2a40 con 0x7f3a38112e30 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.160+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a1c0a3d80 0x7f3a1c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.160+0000 7f3a2d600640 1 -- 172.21.15.18:0/1999627837 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2801e070 con 0x56025c6a5a80 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.160+0000 7f3a2d600640 1 -- 172.21.15.18:0/1999627837 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a28009dd0 con 0x56025c6a5a80 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.160+0000 7f3a2d600640 1 -- 172.21.15.18:0/1999627837 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28019410 con 0x56025c6a5a80 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 -- 172.21.15.18:0/1999627837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 msgr2=0x56025c6a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a1c0a3d80 0x7f3a1c0a6240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x56025c6a1900 tx=0x7f3a2000b670 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6a5f00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009c20 tx=0x7f3a28011810 comp rx=0 tx=0).stop 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 -- 172.21.15.18:0/1999627837 shutdown_connections 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 0x56025c6b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6a5f00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 --2- 172.21.15.18:0/1999627837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 -- 172.21.15.18:0/1999627837 >> 172.21.15.18:0/1999627837 conn(0x56025c6405d0 msgr2=0x56025c6a0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 -- 172.21.15.18:0/1999627837 shutdown_connections 2024-04-16T17:18:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 -- 172.21.15.18:0/1999627837 wait complete. 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.161+0000 7f3a41db2740 1 -- start start 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a41db2740 1 Processor -- start 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 0x56025c6abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a41db2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56025c6b1d30 con 0x56025c6af000 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a41db2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56025c5c4390 con 0x56025c6a67f0 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a41db2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56025c6b1eb0 con 0x56025c6a5a80 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a41db2740 10 client.0 _ll_register_callbacks cb 0x56025c5adf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a3d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a37e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a37e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42676/0 (socket says 172.21.15.18:42676) 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a37e00640 1 -- 172.21.15.18:0/3248950463 learned_addr learned my addr 172.21.15.18:0/3248950463 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a3d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49444/0 (socket says 172.21.15.18:49444) 2024-04-16T17:18:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.162+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 0x56025c6abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.163+0000 7f3a37e00640 1 -- 172.21.15.18:0/3248950463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 msgr2=0x56025c6abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.163+0000 7f3a37e00640 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 0x56025c6abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.163+0000 7f3a37e00640 1 -- 172.21.15.18:0/3248950463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 msgr2=0x56025c6ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.163+0000 7f3a37e00640 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.163+0000 7f3a37e00640 1 -- 172.21.15.18:0/3248950463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56025c3e9620 con 0x56025c6a5a80 2024-04-16T17:18:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:41 smithi022 ceph-mon[36023]: pgmap v443: 129 pgs: 129 active+clean; 307 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 560 op/s 2024-04-16T17:18:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.191+0000 7f3a37e00640 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6aea30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3a28003890 tx=0x7f3a28005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a15a8587f683f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.191+0000 7f3a26000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.198+0000 7f3a26000640 1 -- 172.21.15.18:0/3248950463 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28026020 con 0x56025c6a5a80 2024-04-16T17:18:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.198+0000 7f3a26000640 1 -- 172.21.15.18:0/3248950463 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f3a280046c0 con 0x56025c6a5a80 2024-04-16T17:18:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.198+0000 7f3a41db2740 1 -- 172.21.15.18:0/3248950463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a28009dd0 con 0x56025c6a5a80 2024-04-16T17:18:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.213+0000 7f3a26000640 1 -- 172.21.15.18:0/3248950463 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3a28031050 con 0x56025c6a5a80 2024-04-16T17:18:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.213+0000 7f3a41db2740 10 client.29215 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.213+0000 7f3a41db2740 1 -- 172.21.15.18:0/3248950463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56025c6bde20 con 0x56025c6a5a80 2024-04-16T17:18:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.220+0000 7f3a26000640 1 -- 172.21.15.18:0/3248950463 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a28009dd0 con 0x56025c6a5a80 2024-04-16T17:18:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.221+0000 7f3a26000640 1 -- 172.21.15.18:0/3248950463 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3a28021030 con 0x56025c6a5a80 2024-04-16T17:18:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.221+0000 7f3a41db2740 10 client.29215 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.221+0000 7f3a41db2740 10 client.29215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.221+0000 7f3a41db2740 20 client.29215 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.221+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56025c6bf220 0x56025c6c1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.221+0000 7f3a41db2740 4 client.29215 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.221+0000 7f3a41db2740 1 -- 172.21.15.18:0/3248950463 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56025c6181a0 con 0x56025c6bf220 2024-04-16T17:18:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.221+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56025c6bf220 0x56025c6c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.222+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56025c6bf220 0x56025c6c1610 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe00f4c7e9b5335 server_cookie=5a8905174937e2c2 in_seq=0 out_seq=0 2024-04-16T17:18:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.222+0000 7f3a26000640 10 client.29215 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a26000640 1 -- 172.21.15.18:0/3248950463 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56025c6181a0 con 0x56025c6bf220 2024-04-16T17:18:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a26000640 10 client.29215 handle_command_reply: tid=0 2024-04-16T17:18:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 client.29215 shutdown 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 -- 172.21.15.18:0/3248950463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 msgr2=0x56025c6aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6aea30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3a28003890 tx=0x7f3a28005740 comp rx=0 tx=0).stop 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 -- 172.21.15.18:0/3248950463 shutdown_connections 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56025c6bf220 0x56025c6c1610 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 0x56025c6abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3248950463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6aea30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a26000640 0 client.29215 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 -- 172.21.15.18:0/3248950463 >> 172.21.15.18:0/3248950463 conn(0x56025c6405d0 msgr2=0x56025c6a24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 -- 172.21.15.18:0/3248950463 shutdown_connections 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 1 -- 172.21.15.18:0/3248950463 wait complete. 2024-04-16T17:18:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.223+0000 7f3a41db2740 20 client.29215 trim_cache size 0 max 16384 2024-04-16T17:18:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a41db2740 1 Processor -- start 2024-04-16T17:18:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a41db2740 1 -- start start 2024-04-16T17:18:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 0x56025c6b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a41db2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14012990 con 0x56025c6a67f0 2024-04-16T17:18:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a41db2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14010200 con 0x56025c6a5a80 2024-04-16T17:18:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a41db2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1400e070 con 0x56025c6af000 2024-04-16T17:18:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.239+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a37e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49458/0 (socket says 172.21.15.18:49458) 2024-04-16T17:18:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a2fe00640 1 -- 172.21.15.18:0/3940126041 learned_addr learned my addr 172.21.15.18:0/3940126041 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a3d000640 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 0x56025c6b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a2fe00640 1 -- 172.21.15.18:0/3940126041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 msgr2=0x56025c6b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 0x56025c6b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a2fe00640 1 -- 172.21.15.18:0/3940126041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 msgr2=0x56025c6bacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6bacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a2fe00640 1 -- 172.21.15.18:0/3940126041 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56025c6a4d50 con 0x56025c6a67f0 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a37e00640 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6bacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.240+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6b03f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x56025c6a6520 tx=0x7f3a20011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47bf254b8d8a3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.241+0000 7f3a2cc00640 1 -- 172.21.15.18:0/3940126041 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2002c530 con 0x56025c6a67f0 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.241+0000 7f3a2cc00640 1 -- 172.21.15.18:0/3940126041 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a20031070 con 0x56025c6a67f0 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.241+0000 7f3a2cc00640 1 -- 172.21.15.18:0/3940126041 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20020400 con 0x56025c6a67f0 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.241+0000 7f3a41db2740 1 -- 172.21.15.18:0/3940126041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 msgr2=0x56025c6b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.241+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6b03f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x56025c6a6520 tx=0x7f3a20011000 comp rx=0 tx=0).stop 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.241+0000 7f3a41db2740 1 -- 172.21.15.18:0/3940126041 shutdown_connections 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.241+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6af000 0x56025c6b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.242+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6a67f0 0x56025c6b03f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.242+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3940126041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a5a80 0x56025c6bacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.242+0000 7f3a41db2740 1 -- 172.21.15.18:0/3940126041 >> 172.21.15.18:0/3940126041 conn(0x56025c6405d0 msgr2=0x56025c6a02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.242+0000 7f3a41db2740 1 -- 172.21.15.18:0/3940126041 shutdown_connections 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.242+0000 7f3a41db2740 1 -- 172.21.15.18:0/3940126041 wait complete. 2024-04-16T17:18:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.242+0000 7f3a41db2740 1 -- start start 2024-04-16T17:18:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.242+0000 7f3a41db2740 1 Processor -- start 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6b7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 0x56025c6a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a41db2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6a9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a41db2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1400e6c0 con 0x56025c6af000 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a41db2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1400fb30 con 0x56025c6a67f0 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a41db2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1400ed90 con 0x56025c6a5a80 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a41db2740 10 client.0 _ll_register_callbacks cb 0x7f3a2801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a37e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a37e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6b7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42688/0 (socket says 172.21.15.18:42688) 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a3d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a37e00640 1 -- 172.21.15.18:0/3455645970 learned_addr learned my addr 172.21.15.18:0/3455645970 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 0x56025c6a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a37e00640 1 -- 172.21.15.18:0/3455645970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 msgr2=0x56025c6a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a37e00640 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 0x56025c6a9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a37e00640 1 -- 172.21.15.18:0/3455645970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 msgr2=0x56025c6a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a37e00640 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a37e00640 1 -- 172.21.15.18:0/3455645970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56025c6a9c70 con 0x56025c6a5a80 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.243+0000 7f3a3d000640 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.244+0000 7f3a37e00640 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6b7500 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3a28014020 tx=0x7f3a28009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d54df03e8a9f015 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.244+0000 7f3a26000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.244+0000 7f3a26000640 1 -- 172.21.15.18:0/3455645970 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28026040 con 0x56025c6a5a80 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.244+0000 7f3a41db2740 1 -- 172.21.15.18:0/3455645970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a20031070 con 0x56025c6a5a80 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.244+0000 7f3a26000640 1 -- 172.21.15.18:0/3455645970 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f3a28009d10 con 0x56025c6a5a80 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.244+0000 7f3a26000640 1 -- 172.21.15.18:0/3455645970 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3a2801e040 con 0x56025c6a5a80 2024-04-16T17:18:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.244+0000 7f3a41db2740 10 client.29221 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.244+0000 7f3a41db2740 1 -- 172.21.15.18:0/3455645970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56025c7256e0 con 0x56025c6a5a80 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a26000640 1 -- 172.21.15.18:0/3455645970 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a20031070 con 0x56025c6a5a80 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a26000640 1 -- 172.21.15.18:0/3455645970 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3a28021040 con 0x56025c6a5a80 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a41db2740 10 client.29221 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a41db2740 10 client.29221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a41db2740 20 client.29221 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56025c725d70 0x56025c728190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a41db2740 4 client.29221 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a41db2740 1 -- 172.21.15.18:0/3455645970 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56025c612010 con 0x56025c725d70 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.245+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56025c725d70 0x56025c728190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.246+0000 7f3a2fe00640 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56025c725d70 0x56025c728190 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0542f8e0caf31b3 server_cookie=c735a29187b4cf63 in_seq=0 out_seq=0 2024-04-16T17:18:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:42.246+0000 7f3a26000640 10 client.29221 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:42.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:41 smithi043 ceph-mon[36901]: pgmap v443: 129 pgs: 129 active+clean; 307 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 560 op/s 2024-04-16T17:18:42.334 INFO:tasks.workunit.client.0.smithi018.stdout: 10 15286 18.45 MB/sec warmup 54 sec latency 832.024 ms 2024-04-16T17:18:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.062+0000 7f3a26000640 1 -- 172.21.15.18:0/3455645970 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56025c612010 con 0x56025c725d70 2024-04-16T17:18:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.062+0000 7f3a26000640 10 client.29221 handle_command_reply: tid=0 2024-04-16T17:18:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.062+0000 7f3a41db2740 1 client.29221 shutdown 2024-04-16T17:18:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.063+0000 7f3a41db2740 1 -- 172.21.15.18:0/3455645970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 msgr2=0x56025c6b7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.063+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6b7500 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3a28014020 tx=0x7f3a28009640 comp rx=0 tx=0).stop 2024-04-16T17:18:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.064+0000 7f3a41db2740 1 -- 172.21.15.18:0/3455645970 shutdown_connections 2024-04-16T17:18:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.064+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56025c725d70 0x56025c728190 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.064+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56025c6af000 0x56025c6a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.064+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56025c6a67f0 0x56025c6a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.064+0000 7f3a41db2740 1 --2- 172.21.15.18:0/3455645970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56025c6a5a80 0x56025c6b7500 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.064+0000 7f3a41db2740 1 -- 172.21.15.18:0/3455645970 >> 172.21.15.18:0/3455645970 conn(0x56025c6405d0 msgr2=0x56025c6ab620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.065+0000 7f3a26000640 0 client.29221 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.065+0000 7f3a41db2740 1 -- 172.21.15.18:0/3455645970 shutdown_connections 2024-04-16T17:18:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.065+0000 7f3a41db2740 1 -- 172.21.15.18:0/3455645970 wait complete. 2024-04-16T17:18:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.065+0000 7f3a41db2740 20 client.29221 trim_cache size 0 max 16384 2024-04-16T17:18:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:18:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:18:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:18:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:18:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 -- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a1c0a3d80 msgr2=0x7f3a1c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a1c0a3d80 0x7f3a1c0a6240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x56025c6a1900 tx=0x7f3a2000b670 comp rx=0 tx=0).stop 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 -- 172.21.15.18:0/2157940902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 msgr2=0x7f3a3819eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 0x7f3a3819eb40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3a3002c4c0 tx=0x7f3a30004270 comp rx=0 tx=0).stop 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 -- 172.21.15.18:0/2157940902 shutdown_connections 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3a1c0a3d80 0x7f3a1c0a6240 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a38112e30 0x7f3a3819eb40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a38074cc0 0x7f3a381a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 --2- 172.21.15.18:0/2157940902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a380743a0 0x7f3a381a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.069+0000 7f3a3da00640 1 -- 172.21.15.18:0/2157940902 >> 172.21.15.18:0/2157940902 conn(0x7f3a380fd910 msgr2=0x7f3a380ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.074+0000 7f3a3da00640 1 -- 172.21.15.18:0/2157940902 shutdown_connections 2024-04-16T17:18:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:43.074+0000 7f3a3da00640 1 -- 172.21.15.18:0/2157940902 wait complete. 2024-04-16T17:18:43.334 INFO:tasks.workunit.client.0.smithi018.stdout: 10 15443 18.25 MB/sec warmup 55 sec latency 1098.710 ms 2024-04-16T17:18:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:43 smithi022 ceph-mon[36023]: pgmap v444: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 271 op/s 2024-04-16T17:18:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:43 smithi043 ceph-mon[36901]: pgmap v444: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 271 op/s 2024-04-16T17:18:44.335 INFO:tasks.workunit.client.0.smithi018.stdout: 10 15908 18.36 MB/sec warmup 56 sec latency 51.633 ms 2024-04-16T17:18:45.335 INFO:tasks.workunit.client.0.smithi018.stdout: 10 16423 18.37 MB/sec warmup 57 sec latency 43.915 ms 2024-04-16T17:18:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:45 smithi022 ceph-mon[36023]: pgmap v445: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-04-16T17:18:46.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:45 smithi043 ceph-mon[36901]: pgmap v445: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-04-16T17:18:46.337 INFO:tasks.workunit.client.0.smithi018.stdout: 10 16749 18.27 MB/sec warmup 58 sec latency 345.032 ms 2024-04-16T17:18:47.080 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:43 smithi018 ceph-mon[31776]: pgmap v444: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 271 op/s 2024-04-16T17:18:47.080 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:45 smithi018 ceph-mon[31776]: pgmap v445: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-04-16T17:18:47.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 17081 18.38 MB/sec warmup 59 sec latency 462.370 ms 2024-04-16T17:18:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:47 smithi022 ceph-mon[36023]: pgmap v446: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 514 op/s 2024-04-16T17:18:48.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:47 smithi043 ceph-mon[36901]: pgmap v446: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 514 op/s 2024-04-16T17:18:48.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 17483 18.56 MB/sec warmup 60 sec latency 110.824 ms 2024-04-16T17:18:49.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 17903 18.77 MB/sec warmup 61 sec latency 178.957 ms 2024-04-16T17:18:50.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:49 smithi022 ceph-mon[36023]: pgmap v447: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 408 op/s 2024-04-16T17:18:50.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:49 smithi043 ceph-mon[36901]: pgmap v447: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 408 op/s 2024-04-16T17:18:50.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 18163 18.74 MB/sec warmup 62 sec latency 218.768 ms 2024-04-16T17:18:51.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 18348 18.53 MB/sec warmup 63 sec latency 221.405 ms 2024-04-16T17:18:51.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:47 smithi018 ceph-mon[31776]: pgmap v446: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 514 op/s 2024-04-16T17:18:51.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:49 smithi018 ceph-mon[31776]: pgmap v447: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 408 op/s 2024-04-16T17:18:52.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:51 smithi022 ceph-mon[36023]: pgmap v448: 129 pgs: 129 active+clean; 309 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 532 op/s 2024-04-16T17:18:52.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:51 smithi043 ceph-mon[36901]: pgmap v448: 129 pgs: 129 active+clean; 309 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 532 op/s 2024-04-16T17:18:52.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 18505 18.31 MB/sec warmup 64 sec latency 283.501 ms 2024-04-16T17:18:53.337 INFO:tasks.workunit.client.0.smithi018.stdout: 10 18736 18.15 MB/sec warmup 65 sec latency 242.624 ms 2024-04-16T17:18:54.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:53 smithi022 ceph-mon[36023]: pgmap v449: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-04-16T17:18:54.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:53 smithi043 ceph-mon[36901]: pgmap v449: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-04-16T17:18:54.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 19094 18.05 MB/sec warmup 66 sec latency 222.104 ms 2024-04-16T17:18:55.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 19616 18.20 MB/sec warmup 67 sec latency 50.355 ms 2024-04-16T17:18:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:55 smithi022 ceph-mon[36023]: pgmap v450: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-04-16T17:18:56.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:55 smithi043 ceph-mon[36901]: pgmap v450: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-04-16T17:18:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 -- 172.21.15.18:0/1808826142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24101042a0 msgr2=0x7f2410104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 --2- 172.21.15.18:0/1808826142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24101042a0 0x7f2410104680 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2400009900 tx=0x7f240002bc80 comp rx=0 tx=0).stop 2024-04-16T17:18:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 -- 172.21.15.18:0/1808826142 shutdown_connections 2024-04-16T17:18:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 --2- 172.21.15.18:0/1808826142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2410109370 0x7f2410110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 --2- 172.21.15.18:0/1808826142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2410104c50 0x7f2410108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 --2- 172.21.15.18:0/1808826142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24101042a0 0x7f2410104680 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 -- 172.21.15.18:0/1808826142 >> 172.21.15.18:0/1808826142 conn(0x7f24100ffb00 msgr2=0x7f2410101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 -- 172.21.15.18:0/1808826142 shutdown_connections 2024-04-16T17:18:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.290+0000 7f240fe00640 1 -- 172.21.15.18:0/1808826142 wait complete. 2024-04-16T17:18:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.291+0000 7f240fe00640 1 Processor -- start 2024-04-16T17:18:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.291+0000 7f240fe00640 1 -- start start 2024-04-16T17:18:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24101042a0 0x7f24101a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2410109370 0x7f24101a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241019a8d0 0x7f241019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240fe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2410114570 con 0x7f24101042a0 2024-04-16T17:18:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240fe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24101143f0 con 0x7f241019a8d0 2024-04-16T17:18:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24101146f0 con 0x7f2410109370 2024-04-16T17:18:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2410109370 0x7f24101a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2410109370 0x7f24101a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48734/0 (socket says 172.21.15.18:48734) 2024-04-16T17:18:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240e000640 1 -- 172.21.15.18:0/2801563675 learned_addr learned my addr 172.21.15.18:0/2801563675 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240ea00640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24101042a0 0x7f24101a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.292+0000 7f240f400640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241019a8d0 0x7f241019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.293+0000 7f240e000640 1 -- 172.21.15.18:0/2801563675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241019a8d0 msgr2=0x7f241019ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.293+0000 7f240e000640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241019a8d0 0x7f241019ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.293+0000 7f240e000640 1 -- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24101042a0 msgr2=0x7f24101a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.293+0000 7f240e000640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24101042a0 0x7f24101a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.293+0000 7f240e000640 1 -- 172.21.15.18:0/2801563675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241019b3f0 con 0x7f2410109370 2024-04-16T17:18:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.294+0000 7f240e000640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2410109370 0x7f24101a0c10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b9c0 tx=0x7f23fc00be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.294+0000 7f2407400640 1 -- 172.21.15.18:0/2801563675 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc019070 con 0x7f2410109370 2024-04-16T17:18:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.294+0000 7f240fe00640 1 -- 172.21.15.18:0/2801563675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241019b6e0 con 0x7f2410109370 2024-04-16T17:18:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.294+0000 7f240fe00640 1 -- 172.21.15.18:0/2801563675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a7930 con 0x7f2410109370 2024-04-16T17:18:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.294+0000 7f2407400640 1 -- 172.21.15.18:0/2801563675 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23fc00cba0 con 0x7f2410109370 2024-04-16T17:18:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.294+0000 7f2407400640 1 -- 172.21.15.18:0/2801563675 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0126b0 con 0x7f2410109370 2024-04-16T17:18:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.301+0000 7f24189f4740 1 Processor -- start 2024-04-16T17:18:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.301+0000 7f24189f4740 1 -- start start 2024-04-16T17:18:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f2407400640 1 -- 172.21.15.18:0/2801563675 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23fc01c070 con 0x7f2410109370 2024-04-16T17:18:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f24189f4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c97f0 0x564f5f4c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f24189f4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c8a80 0x564f5f4c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f24189f4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f24189f4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564f5f42d780 con 0x564f5f4da810 2024-04-16T17:18:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f24189f4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564f5f3d3f40 con 0x564f5f4c8a80 2024-04-16T17:18:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f24189f4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f5f429740 con 0x564f5f4c97f0 2024-04-16T17:18:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f240ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c8a80 0x564f5f4c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f240ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c8a80 0x564f5f4c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38752/0 (socket says 172.21.15.18:38752) 2024-04-16T17:18:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.302+0000 7f240ea00640 1 -- 172.21.15.18:0/2717846505 learned_addr learned my addr 172.21.15.18:0/2717846505 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.303+0000 7f240f400640 1 --2- 172.21.15.18:0/2717846505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c97f0 0x564f5f4c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.303+0000 7f2407400640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f23f40a3d80 0x7f23f40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.303+0000 7f240ea00640 1 -- 172.21.15.18:0/2717846505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c97f0 msgr2=0x564f5f4c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.303+0000 7f240ea00640 1 --2- 172.21.15.18:0/2717846505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c97f0 0x564f5f4c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.303+0000 7f240ea00640 1 -- 172.21.15.18:0/2717846505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 msgr2=0x564f5f4dcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.303+0000 7f240ea00640 1 --2- 172.21.15.18:0/2717846505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.303+0000 7f240ea00640 1 -- 172.21.15.18:0/2717846505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f5f140210 con 0x564f5f4c8a80 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.304+0000 7f240ea00640 1 --2- 172.21.15.18:0/2717846505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c8a80 0x564f5f4c8f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2400036040 tx=0x7f240002c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0b8e73433d5deb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.304+0000 7f2407400640 1 -- 172.21.15.18:0/2801563675 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f23fc0cff90 con 0x7f2410109370 2024-04-16T17:18:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.305+0000 7f23fbe00640 1 -- 172.21.15.18:0/2717846505 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400042030 con 0x564f5f4c8a80 2024-04-16T17:18:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.305+0000 7f23fbe00640 1 -- 172.21.15.18:0/2717846505 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240002c900 con 0x564f5f4c8a80 2024-04-16T17:18:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.305+0000 7f23fbe00640 1 -- 172.21.15.18:0/2717846505 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000046c0 con 0x564f5f4c8a80 2024-04-16T17:18:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.305+0000 7f240f400640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f23f40a3d80 0x7f23f40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f24189f4740 1 -- 172.21.15.18:0/2717846505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c8a80 msgr2=0x564f5f4c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f24189f4740 1 --2- 172.21.15.18:0/2717846505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c8a80 0x564f5f4c8f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2400036040 tx=0x7f240002c750 comp rx=0 tx=0).stop 2024-04-16T17:18:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f24189f4740 1 -- 172.21.15.18:0/2717846505 shutdown_connections 2024-04-16T17:18:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f24189f4740 1 --2- 172.21.15.18:0/2717846505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4dcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f24189f4740 1 --2- 172.21.15.18:0/2717846505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c8a80 0x564f5f4c8f00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f24189f4740 1 --2- 172.21.15.18:0/2717846505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c97f0 0x564f5f4c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f24189f4740 1 -- 172.21.15.18:0/2717846505 >> 172.21.15.18:0/2717846505 conn(0x564f5f4635d0 msgr2=0x564f5f4c3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f240f400640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f23f40a3d80 0x7f23f40a6240 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f240800a4c0 tx=0x7f240800c040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.306+0000 7f24189f4740 1 -- 172.21.15.18:0/2717846505 shutdown_connections 2024-04-16T17:18:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.307+0000 7f24189f4740 1 -- 172.21.15.18:0/2717846505 wait complete. 2024-04-16T17:18:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.307+0000 7f24189f4740 1 -- start start 2024-04-16T17:18:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.307+0000 7f24189f4740 1 Processor -- start 2024-04-16T17:18:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f24189f4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f24189f4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4c97f0 0x564f5f4d7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f24189f4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4da810 0x564f5f4d81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f24189f4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564f5f4dd4e0 con 0x564f5f4c97f0 2024-04-16T17:18:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f24189f4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564f5f3e7390 con 0x564f5f4da810 2024-04-16T17:18:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f24189f4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f5f4dd660 con 0x564f5f4c8a80 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f24189f4740 10 client.0 _ll_register_callbacks cb 0x564f5f3d0f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f240ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f240ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48752/0 (socket says 172.21.15.18:48752) 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f240ea00640 1 -- 172.21.15.18:0/1188713920 learned_addr learned my addr 172.21.15.18:0/1188713920 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f240f400640 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4da810 0x564f5f4d81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f240ea00640 1 -- 172.21.15.18:0/1188713920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4da810 msgr2=0x564f5f4d81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f240ea00640 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4da810 0x564f5f4d81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f240ea00640 1 -- 172.21.15.18:0/1188713920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4c97f0 msgr2=0x564f5f4d7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.308+0000 7f240ea00640 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4c97f0 0x564f5f4d7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f240ea00640 1 -- 172.21.15.18:0/1188713920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f5f20c620 con 0x564f5f4c8a80 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f240ea00640 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d7720 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f240002f020 tx=0x7f240002cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87cba9a0f8aec260 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f23f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f23f8c00640 1 -- 172.21.15.18:0/1188713920 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240004b070 con 0x564f5f4c8a80 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f24189f4740 1 -- 172.21.15.18:0/1188713920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240002c900 con 0x564f5f4c8a80 2024-04-16T17:18:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f23f8c00640 1 -- 172.21.15.18:0/1188713920 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f2400004380 con 0x564f5f4c8a80 2024-04-16T17:18:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f23f8c00640 1 -- 172.21.15.18:0/1188713920 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2400057050 con 0x564f5f4c8a80 2024-04-16T17:18:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f24189f4740 10 client.29233 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.309+0000 7f24189f4740 1 -- 172.21.15.18:0/1188713920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564f5f4dd4e0 con 0x564f5f4c8a80 2024-04-16T17:18:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f23f8c00640 1 -- 172.21.15.18:0/1188713920 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240002c900 con 0x564f5f4c8a80 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f23f8c00640 1 -- 172.21.15.18:0/1188713920 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2400003710 con 0x564f5f4c8a80 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f24189f4740 10 client.29233 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f24189f4740 10 client.29233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f24189f4740 20 client.29233 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f24189f4740 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564f5f4ea890 0x564f5f4ecc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f24189f4740 4 client.29233 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f24189f4740 1 -- 172.21.15.18:0/1188713920 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f5f43b1a0 con 0x564f5f4ea890 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.310+0000 7f240e000640 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564f5f4ea890 0x564f5f4ecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.311+0000 7f23f8c00640 10 client.29233 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.311+0000 7f240e000640 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564f5f4ea890 0x564f5f4ecc80 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3adbcd2efd891c42 server_cookie=a5cf852f9f268580 in_seq=0 out_seq=0 2024-04-16T17:18:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.318+0000 7f23f8c00640 1 -- 172.21.15.18:0/1188713920 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f5f43b1a0 con 0x564f5f4ea890 2024-04-16T17:18:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.318+0000 7f23f8c00640 10 client.29233 handle_command_reply: tid=0 2024-04-16T17:18:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.318+0000 7f24189f4740 1 client.29233 shutdown 2024-04-16T17:18:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.319+0000 7f24189f4740 1 -- 172.21.15.18:0/1188713920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 msgr2=0x564f5f4d7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.319+0000 7f24189f4740 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d7720 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f240002f020 tx=0x7f240002cce0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.319+0000 7f24189f4740 1 -- 172.21.15.18:0/1188713920 shutdown_connections 2024-04-16T17:18:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.319+0000 7f24189f4740 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564f5f4ea890 0x564f5f4ecc80 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.319+0000 7f24189f4740 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4da810 0x564f5f4d81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.319+0000 7f24189f4740 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4c97f0 0x564f5f4d7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.319+0000 7f24189f4740 1 --2- 172.21.15.18:0/1188713920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d7720 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.319+0000 7f24189f4740 1 -- 172.21.15.18:0/1188713920 >> 172.21.15.18:0/1188713920 conn(0x564f5f4635d0 msgr2=0x564f5f4c54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.320+0000 7f23f8c00640 0 client.29233 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.320+0000 7f24189f4740 1 -- 172.21.15.18:0/1188713920 shutdown_connections 2024-04-16T17:18:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.321+0000 7f24189f4740 1 -- 172.21.15.18:0/1188713920 wait complete. 2024-04-16T17:18:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.321+0000 7f24189f4740 20 client.29233 trim_cache size 0 max 16384 2024-04-16T17:18:56.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 20214 18.37 MB/sec warmup 68 sec latency 50.275 ms 2024-04-16T17:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.345+0000 7f24189f4740 1 Processor -- start 2024-04-16T17:18:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.345+0000 7f24189f4740 1 -- start start 2024-04-16T17:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f24189f4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f24189f4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c97f0 0x564f5f4e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f24189f4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f24189f4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0012990 con 0x564f5f4da810 2024-04-16T17:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f24189f4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0010200 con 0x564f5f4c97f0 2024-04-16T17:18:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f24189f4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23f000e070 con 0x564f5f4c8a80 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f240ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f240ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48764/0 (socket says 172.21.15.18:48764) 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.346+0000 7f240ea00640 1 -- 172.21.15.18:0/1874645139 learned_addr learned my addr 172.21.15.18:0/1874645139 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.347+0000 7f240ea00640 1 -- 172.21.15.18:0/1874645139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c97f0 msgr2=0x564f5f4e6440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.347+0000 7f240ea00640 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c97f0 0x564f5f4e6440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.347+0000 7f240ea00640 1 -- 172.21.15.18:0/1874645139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 msgr2=0x564f5f4d6700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.347+0000 7f240f400640 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.347+0000 7f240ea00640 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.347+0000 7f240ea00640 1 -- 172.21.15.18:0/1874645139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f5f4d5c70 con 0x564f5f4c8a80 2024-04-16T17:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.347+0000 7f240f400640 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.348+0000 7f240ea00640 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d9e50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2400000c00 tx=0x7f24000095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ab7f1b079dae43b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.348+0000 7f23fb400640 1 -- 172.21.15.18:0/1874645139 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240004b040 con 0x564f5f4c8a80 2024-04-16T17:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.348+0000 7f23fb400640 1 -- 172.21.15.18:0/1874645139 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2400005310 con 0x564f5f4c8a80 2024-04-16T17:18:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.348+0000 7f23fb400640 1 -- 172.21.15.18:0/1874645139 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400042040 con 0x564f5f4c8a80 2024-04-16T17:18:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.349+0000 7f24189f4740 1 -- 172.21.15.18:0/1874645139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 msgr2=0x564f5f4d9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.349+0000 7f24189f4740 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d9e50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2400000c00 tx=0x7f24000095a0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.350+0000 7f24189f4740 1 -- 172.21.15.18:0/1874645139 shutdown_connections 2024-04-16T17:18:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.350+0000 7f24189f4740 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.350+0000 7f24189f4740 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c97f0 0x564f5f4e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.350+0000 7f24189f4740 1 --2- 172.21.15.18:0/1874645139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4d9e50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.351+0000 7f24189f4740 1 -- 172.21.15.18:0/1874645139 >> 172.21.15.18:0/1874645139 conn(0x564f5f4635d0 msgr2=0x564f5f4d5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.351+0000 7f24189f4740 1 -- 172.21.15.18:0/1874645139 shutdown_connections 2024-04-16T17:18:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.351+0000 7f24189f4740 1 -- 172.21.15.18:0/1874645139 wait complete. 2024-04-16T17:18:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.353+0000 7f24189f4740 1 -- start start 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.353+0000 7f24189f4740 1 Processor -- start 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.353+0000 7f24189f4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4e2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.353+0000 7f24189f4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c97f0 0x564f5f4e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.353+0000 7f24189f4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.353+0000 7f24189f4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23f000e6c0 con 0x564f5f4da810 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f24189f4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f23f000fb30 con 0x564f5f4c97f0 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f24189f4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23f000ed90 con 0x564f5f4c8a80 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f24189f4740 10 client.0 _ll_register_callbacks cb 0x7f240002c620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4e2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48770/0 (socket says 172.21.15.18:48770) 2024-04-16T17:18:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240ea00640 1 -- 172.21.15.18:0/3697571526 learned_addr learned my addr 172.21.15.18:0/3697571526 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240ea00640 1 -- 172.21.15.18:0/3697571526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c97f0 msgr2=0x564f5f4e2e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240ea00640 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c97f0 0x564f5f4e2e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240ea00640 1 -- 172.21.15.18:0/3697571526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 msgr2=0x564f5f4d49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240f400640 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240ea00640 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.354+0000 7f240ea00640 1 -- 172.21.15.18:0/3697571526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f5f4d4f90 con 0x564f5f4c8a80 2024-04-16T17:18:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.355+0000 7f240ea00640 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4e2950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2400004740 tx=0x7f240002cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3992a6682f8c5210 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.355+0000 7f23f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.355+0000 7f23f8c00640 1 -- 172.21.15.18:0/3697571526 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240004b070 con 0x564f5f4c8a80 2024-04-16T17:18:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.355+0000 7f23f8c00640 1 -- 172.21.15.18:0/3697571526 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f24000035f0 con 0x564f5f4c8a80 2024-04-16T17:18:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.356+0000 7f24189f4740 1 -- 172.21.15.18:0/3697571526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2400005310 con 0x564f5f4c8a80 2024-04-16T17:18:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.357+0000 7f23f8c00640 1 -- 172.21.15.18:0/3697571526 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2400042040 con 0x564f5f4c8a80 2024-04-16T17:18:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.358+0000 7f24189f4740 10 client.29239 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.358+0000 7f24189f4740 1 -- 172.21.15.18:0/3697571526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564f5f4db430 con 0x564f5f4c8a80 2024-04-16T17:18:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.360+0000 7f23f8c00640 1 -- 172.21.15.18:0/3697571526 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2400005310 con 0x564f5f4c8a80 2024-04-16T17:18:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.360+0000 7f23f8c00640 1 -- 172.21.15.18:0/3697571526 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2400034440 con 0x564f5f4c8a80 2024-04-16T17:18:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.360+0000 7f24189f4740 10 client.29239 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.360+0000 7f24189f4740 10 client.29239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.360+0000 7f24189f4740 20 client.29239 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.360+0000 7f24189f4740 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564f5f4c9fa0 0x564f5f4cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.360+0000 7f24189f4740 4 client.29239 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:18:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.360+0000 7f24189f4740 1 -- 172.21.15.18:0/3697571526 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f5f435010 con 0x564f5f4c9fa0 2024-04-16T17:18:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.361+0000 7f240e000640 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564f5f4c9fa0 0x564f5f4cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.361+0000 7f240e000640 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564f5f4c9fa0 0x564f5f4cc3c0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf67d4d575f914c server_cookie=3f809525a77e536c in_seq=0 out_seq=0 2024-04-16T17:18:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:56.361+0000 7f23f8c00640 10 client.29239 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:56.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:51 smithi018 ceph-mon[31776]: pgmap v448: 129 pgs: 129 active+clean; 309 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 532 op/s 2024-04-16T17:18:56.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:53 smithi018 ceph-mon[31776]: pgmap v449: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-04-16T17:18:56.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:55 smithi018 ceph-mon[31776]: pgmap v450: 129 pgs: 129 active+clean; 308 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-04-16T17:18:57.336 INFO:tasks.workunit.client.0.smithi018.stdout: 10 20396 18.28 MB/sec warmup 69 sec latency 563.626 ms 2024-04-16T17:18:57.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:18:57.343 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:18:57.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.678+0000 7f8010e00640 1 -- 172.21.15.18:0/2805746646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c104450 msgr2=0x7f800c104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:57.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.678+0000 7f8010e00640 1 --2- 172.21.15.18:0/2805746646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c104450 0x7f800c104830 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009860 tx=0x7f7ffc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:18:57.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.680+0000 7f8010e00640 1 -- 172.21.15.18:0/2805746646 shutdown_connections 2024-04-16T17:18:57.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.680+0000 7f8010e00640 1 --2- 172.21.15.18:0/2805746646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f800c109520 0x7f800c1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.680+0000 7f8010e00640 1 --2- 172.21.15.18:0/2805746646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f800c104e00 0x7f800c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.680+0000 7f8010e00640 1 --2- 172.21.15.18:0/2805746646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c104450 0x7f800c104830 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.680+0000 7f8010e00640 1 -- 172.21.15.18:0/2805746646 >> 172.21.15.18:0/2805746646 conn(0x7f800c0ffc90 msgr2=0x7f800c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:57.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.681+0000 7f8010e00640 1 -- 172.21.15.18:0/2805746646 shutdown_connections 2024-04-16T17:18:57.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.681+0000 7f8010e00640 1 -- 172.21.15.18:0/2805746646 wait complete. 2024-04-16T17:18:57.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.682+0000 7f8010e00640 1 Processor -- start 2024-04-16T17:18:57.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.683+0000 7f8010e00640 1 -- start start 2024-04-16T17:18:57.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.684+0000 7f8010e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f800c104450 0x7f800c111000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.684+0000 7f8010e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f800c109520 0x7f800c10e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.684+0000 7f8010e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c10e630 0x7f800c10ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.684+0000 7f8010e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f800c114650 con 0x7f800c109520 2024-04-16T17:18:57.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.684+0000 7f8010e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1144d0 con 0x7f800c104450 2024-04-16T17:18:57.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.684+0000 7f8010e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1147d0 con 0x7f800c10e630 2024-04-16T17:18:57.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.685+0000 7f8009e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f800c104450 0x7f800c111000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:57.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.685+0000 7f8009e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f800c104450 0x7f800c111000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54272/0 (socket says 172.21.15.18:54272) 2024-04-16T17:18:57.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.685+0000 7f8009e00640 1 -- 172.21.15.18:0/1868287862 learned_addr learned my addr 172.21.15.18:0/1868287862 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:57.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.685+0000 7f800a800640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c10e630 0x7f800c10ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:57.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.686+0000 7f8009e00640 1 -- 172.21.15.18:0/1868287862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c10e630 msgr2=0x7f800c10ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:57.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.686+0000 7f8009e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c10e630 0x7f800c10ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.686+0000 7f8009e00640 1 -- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f800c109520 msgr2=0x7f800c10e0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:18:57.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.686+0000 7f8009e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f800c109520 0x7f800c10e0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.686+0000 7f8009e00640 1 -- 172.21.15.18:0/1868287862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c072cb0 con 0x7f800c104450 2024-04-16T17:18:57.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.686+0000 7f8009e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f800c104450 0x7f800c111000 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc037040 tx=0x7f7ffc0323b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:57.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.686+0000 7f8002a00640 1 -- 172.21.15.18:0/1868287862 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc041020 con 0x7f800c104450 2024-04-16T17:18:57.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.687+0000 7f8002a00640 1 -- 172.21.15.18:0/1868287862 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ffc02c9f0 con 0x7f800c104450 2024-04-16T17:18:57.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.687+0000 7f8002a00640 1 -- 172.21.15.18:0/1868287862 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc039040 con 0x7f800c104450 2024-04-16T17:18:57.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.687+0000 7f8010e00640 1 -- 172.21.15.18:0/1868287862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c072f40 con 0x7f800c104450 2024-04-16T17:18:57.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.688+0000 7f8010e00640 1 -- 172.21.15.18:0/1868287862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c0733b0 con 0x7f800c104450 2024-04-16T17:18:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.696+0000 7f8002a00640 1 -- 172.21.15.18:0/1868287862 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ffc044020 con 0x7f800c104450 2024-04-16T17:18:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.697+0000 7f8013b89740 1 Processor -- start 2024-04-16T17:18:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.697+0000 7f8013b89740 1 -- start start 2024-04-16T17:18:57.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.697+0000 7f8013b89740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8013b89740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 0x56110f8b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8013b89740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8013b89740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56110f813120 con 0x56110f8adad0 2024-04-16T17:18:57.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8013b89740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56110f7b9340 con 0x56110f8b66b0 2024-04-16T17:18:57.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8013b89740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56110f80f0e0 con 0x56110f8ad1b0 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8003e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8003e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54278/0 (socket says 172.21.15.18:54278) 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8003e00640 1 -- 172.21.15.18:0/1380544846 learned_addr learned my addr 172.21.15.18:0/1380544846 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8003e00640 1 -- 172.21.15.18:0/1380544846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 msgr2=0x56110f8ad590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8003e00640 1 --2- 172.21.15.18:0/1380544846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8ad590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8003e00640 1 -- 172.21.15.18:0/1380544846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 msgr2=0x56110f8b6170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8003e00640 1 --2- 172.21.15.18:0/1380544846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 0x56110f8b6170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.698+0000 7f8003e00640 1 -- 172.21.15.18:0/1380544846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110f5ef7d0 con 0x56110f8b66b0 2024-04-16T17:18:57.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.699+0000 7f8003e00640 1 --2- 172.21.15.18:0/1380544846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b8aa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009a30 tx=0x7f7ff4009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aed715ac9a6bfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.699+0000 7f7ffb400640 1 -- 172.21.15.18:0/1380544846 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400ecb0 con 0x56110f8b66b0 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.700+0000 7f7ffb400640 1 -- 172.21.15.18:0/1380544846 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff400ee50 con 0x56110f8b66b0 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.700+0000 7f7ffb400640 1 -- 172.21.15.18:0/1380544846 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400ecb0 con 0x56110f8b66b0 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.701+0000 7f8013b89740 1 -- 172.21.15.18:0/1380544846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 msgr2=0x56110f8b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.701+0000 7f8013b89740 1 --2- 172.21.15.18:0/1380544846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b8aa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009a30 tx=0x7f7ff4009ef0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.701+0000 7f8002a00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7ff00a3e20 0x7f7ff00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.701+0000 7f8002a00640 1 -- 172.21.15.18:0/1868287862 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f7ffc0f1a60 con 0x7f800c104450 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.702+0000 7f8013b89740 1 -- 172.21.15.18:0/1380544846 shutdown_connections 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.702+0000 7f8013b89740 1 --2- 172.21.15.18:0/1380544846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b8aa0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.702+0000 7f8013b89740 1 --2- 172.21.15.18:0/1380544846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 0x56110f8b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.702+0000 7f8013b89740 1 --2- 172.21.15.18:0/1380544846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.702+0000 7f8013b89740 1 -- 172.21.15.18:0/1380544846 >> 172.21.15.18:0/1380544846 conn(0x56110f8a78e0 msgr2=0x56110f8a7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:57.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.703+0000 7f800a800640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7ff00a3e20 0x7f7ff00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:57.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.703+0000 7f8013b89740 1 -- 172.21.15.18:0/1380544846 shutdown_connections 2024-04-16T17:18:57.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.703+0000 7f8013b89740 1 -- 172.21.15.18:0/1380544846 wait complete. 2024-04-16T17:18:57.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.704+0000 7f8013b89740 1 -- start start 2024-04-16T17:18:57.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.704+0000 7f8013b89740 1 Processor -- start 2024-04-16T17:18:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.706+0000 7f8013b89740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.706+0000 7f8013b89740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 0x56110f8b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.707+0000 7f8013b89740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.707+0000 7f800a800640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7ff00a3e20 0x7f7ff00a62e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8004009f60 tx=0x7f800400d040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.707+0000 7f8009e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.707+0000 7f8013b89740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56110f8b94a0 con 0x56110f8adad0 2024-04-16T17:18:57.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.707+0000 7f8013b89740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56110f7cc710 con 0x56110f8b66b0 2024-04-16T17:18:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.707+0000 7f8013b89740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56110f8b9620 con 0x56110f8ad1b0 2024-04-16T17:18:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.707+0000 7f8009e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54612/0 (socket says 172.21.15.18:54612) 2024-04-16T17:18:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.707+0000 7f8009e00640 1 -- 172.21.15.18:0/1420547817 learned_addr learned my addr 172.21.15.18:0/1420547817 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.708+0000 7f8013b89740 10 client.0 _ll_register_callbacks cb 0x56110f7c7af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.708+0000 7f8003e00640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 0x56110f8b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.708+0000 7f800a800640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.709+0000 7f8009e00640 1 -- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 msgr2=0x56110f8b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.709+0000 7f8009e00640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.709+0000 7f8009e00640 1 -- 172.21.15.18:0/1420547817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 msgr2=0x56110f8b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.709+0000 7f8009e00640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 0x56110f8b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.709+0000 7f8009e00640 1 -- 172.21.15.18:0/1420547817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110f5253f0 con 0x56110f8ad1b0 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.709+0000 7f8003e00640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 0x56110f8b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.709+0000 7f800a800640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.709+0000 7f8009e00640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8b3780 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc02c180 tx=0x7f7ffc0bd010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f2b36127f37c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:57.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.710+0000 7f7fefe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.710+0000 7f7fefe00640 1 -- 172.21.15.18:0/1420547817 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0d6070 con 0x56110f8ad1b0 2024-04-16T17:18:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.710+0000 7f7fefe00640 1 -- 172.21.15.18:0/1420547817 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f7ffc0eefd0 con 0x56110f8ad1b0 2024-04-16T17:18:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.710+0000 7f8013b89740 1 -- 172.21.15.18:0/1420547817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff400ee50 con 0x56110f8ad1b0 2024-04-16T17:18:57.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.710+0000 7f7fefe00640 1 -- 172.21.15.18:0/1420547817 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7ffc0ef1d0 con 0x56110f8ad1b0 2024-04-16T17:18:57.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.714+0000 7f8013b89740 10 client.29248 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:57.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.714+0000 7f8013b89740 1 -- 172.21.15.18:0/1420547817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56110f4e50f0 con 0x56110f8ad1b0 2024-04-16T17:18:57.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.715+0000 7f7fefe00640 1 -- 172.21.15.18:0/1420547817 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff400ee50 con 0x56110f8ad1b0 2024-04-16T17:18:57.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.715+0000 7f7fefe00640 1 -- 172.21.15.18:0/1420547817 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7ffc03c030 con 0x56110f8ad1b0 2024-04-16T17:18:57.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.715+0000 7f8013b89740 10 client.29248 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:57.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.715+0000 7f8013b89740 10 client.29248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:57.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.716+0000 7f8013b89740 20 client.29248 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:57.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.716+0000 7f8013b89740 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56110f8c6980 0x56110f8c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:57.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.716+0000 7f8013b89740 4 client.29248 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:57.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.716+0000 7f8013b89740 1 -- 172.21.15.18:0/1420547817 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56110f820b40 con 0x56110f8c6980 2024-04-16T17:18:57.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.717+0000 7f8003e00640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56110f8c6980 0x56110f8c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:57.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.718+0000 7f8003e00640 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56110f8c6980 0x56110f8c8db0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0e7cc99c0e8f85 server_cookie=d5ef9e8df569eafb in_seq=0 out_seq=0 2024-04-16T17:18:57.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:57.718+0000 7f7fefe00640 10 client.29248 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.108+0000 7f23f8c00640 1 -- 172.21.15.18:0/3697571526 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f5f435010 con 0x564f5f4c9fa0 2024-04-16T17:18:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.108+0000 7f23f8c00640 10 client.29239 handle_command_reply: tid=0 2024-04-16T17:18:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.108+0000 7f24189f4740 1 client.29239 shutdown 2024-04-16T17:18:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.108+0000 7f24189f4740 1 -- 172.21.15.18:0/3697571526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 msgr2=0x564f5f4e2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.108+0000 7f24189f4740 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4e2950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2400004740 tx=0x7f240002cef0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.108+0000 7f7fefe00640 1 -- 172.21.15.18:0/1420547817 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56110f820b40 con 0x56110f8c6980 2024-04-16T17:18:58.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.108+0000 7f7fefe00640 10 client.29248 handle_command_reply: tid=0 2024-04-16T17:18:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.109+0000 7f24189f4740 1 -- 172.21.15.18:0/3697571526 shutdown_connections 2024-04-16T17:18:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.109+0000 7f24189f4740 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564f5f4c9fa0 0x564f5f4cc3c0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.109+0000 7f24189f4740 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f5f4da810 0x564f5f4d49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.109+0000 7f24189f4740 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f5f4c97f0 0x564f5f4e2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.109+0000 7f24189f4740 1 --2- 172.21.15.18:0/3697571526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f5f4c8a80 0x564f5f4e2950 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.109+0000 7f24189f4740 1 -- 172.21.15.18:0/3697571526 >> 172.21.15.18:0/3697571526 conn(0x564f5f4635d0 msgr2=0x564f5f4d6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.109+0000 7f23f8c00640 0 client.29239 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:58.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.109+0000 7f8013b89740 1 client.29248 shutdown 2024-04-16T17:18:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.110+0000 7f24189f4740 1 -- 172.21.15.18:0/3697571526 shutdown_connections 2024-04-16T17:18:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.110+0000 7f24189f4740 1 -- 172.21.15.18:0/3697571526 wait complete. 2024-04-16T17:18:58.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f8013b89740 1 -- 172.21.15.18:0/1420547817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 msgr2=0x56110f8b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f8013b89740 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8b3780 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc02c180 tx=0x7f7ffc0bd010 comp rx=0 tx=0).stop 2024-04-16T17:18:58.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f8013b89740 1 -- 172.21.15.18:0/1420547817 shutdown_connections 2024-04-16T17:18:58.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f8013b89740 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56110f8c6980 0x56110f8c8db0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f8013b89740 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8b66b0 0x56110f8b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f8013b89740 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8adad0 0x56110f8b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f8013b89740 1 --2- 172.21.15.18:0/1420547817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8ad1b0 0x56110f8b3780 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f8013b89740 1 -- 172.21.15.18:0/1420547817 >> 172.21.15.18:0/1420547817 conn(0x56110f8a78e0 msgr2=0x56110f8a9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:58.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.112+0000 7f7fefe00640 0 client.29248 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:58.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.113+0000 7f8013b89740 1 -- 172.21.15.18:0/1420547817 shutdown_connections 2024-04-16T17:18:58.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.113+0000 7f8013b89740 1 -- 172.21.15.18:0/1420547817 wait complete. 2024-04-16T17:18:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.113+0000 7f24189f4740 20 client.29239 trim_cache size 0 max 16384 2024-04-16T17:18:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.113+0000 7f8013b89740 20 client.29248 trim_cache size 0 max 16384 2024-04-16T17:18:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:18:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:18:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:18:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:18:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.115+0000 7f240fe00640 1 -- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f23f40a3d80 msgr2=0x7f23f40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.115+0000 7f240fe00640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f23f40a3d80 0x7f23f40a6240 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f240800a4c0 tx=0x7f240800c040 comp rx=0 tx=0).stop 2024-04-16T17:18:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.115+0000 7f240fe00640 1 -- 172.21.15.18:0/2801563675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2410109370 msgr2=0x7f24101a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.115+0000 7f240fe00640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2410109370 0x7f24101a0c10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b9c0 tx=0x7f23fc00be80 comp rx=0 tx=0).stop 2024-04-16T17:18:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.116+0000 7f240fe00640 1 -- 172.21.15.18:0/2801563675 shutdown_connections 2024-04-16T17:18:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.116+0000 7f240fe00640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f23f40a3d80 0x7f23f40a6240 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.116+0000 7f240fe00640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f241019a8d0 0x7f241019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.116+0000 7f240fe00640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2410109370 0x7f24101a0c10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.116+0000 7f240fe00640 1 --2- 172.21.15.18:0/2801563675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24101042a0 0x7f24101a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.116+0000 7f240fe00640 1 -- 172.21.15.18:0/2801563675 >> 172.21.15.18:0/2801563675 conn(0x7f24100ffb00 msgr2=0x7f2410101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.117+0000 7f240fe00640 1 -- 172.21.15.18:0/2801563675 shutdown_connections 2024-04-16T17:18:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:18:58.117+0000 7f240fe00640 1 -- 172.21.15.18:0/2801563675 wait complete. 2024-04-16T17:18:58.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.161+0000 7f8013b89740 1 Processor -- start 2024-04-16T17:18:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.161+0000 7f8013b89740 1 -- start start 2024-04-16T17:18:58.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.161+0000 7f8013b89740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.161+0000 7f8013b89740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8c2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.161+0000 7f8013b89740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 0x56110f8b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.161+0000 7f8013b89740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1c9740 con 0x56110f8ad1b0 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.161+0000 7f8013b89740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1c6fb0 con 0x56110f8adad0 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.161+0000 7f8013b89740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1c4e20 con 0x56110f8b66b0 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.162+0000 7f8003e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.162+0000 7f8003e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8c2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54298/0 (socket says 172.21.15.18:54298) 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.162+0000 7f8003e00640 1 -- 172.21.15.18:0/1387440986 learned_addr learned my addr 172.21.15.18:0/1387440986 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:58.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.162+0000 7f800a800640 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 0x56110f8b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.163+0000 7f8009e00640 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.163+0000 7f8003e00640 1 -- 172.21.15.18:0/1387440986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 msgr2=0x56110f8b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.163+0000 7f8003e00640 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 0x56110f8b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.163+0000 7f8003e00640 1 -- 172.21.15.18:0/1387440986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 msgr2=0x56110f8b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.163+0000 7f8003e00640 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.163+0000 7f8003e00640 1 -- 172.21.15.18:0/1387440986 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110f8b2730 con 0x56110f8adad0 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.163+0000 7f8009e00640 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.164+0000 7f8003e00640 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8c2390 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009a30 tx=0x7f7ff4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e6655c387a1b08 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.164+0000 7f7ffaa00640 1 -- 172.21.15.18:0/1387440986 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff402c070 con 0x56110f8adad0 2024-04-16T17:18:58.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.164+0000 7f7ffaa00640 1 -- 172.21.15.18:0/1387440986 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff400ee70 con 0x56110f8adad0 2024-04-16T17:18:58.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.164+0000 7f7ffaa00640 1 -- 172.21.15.18:0/1387440986 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4018af0 con 0x56110f8adad0 2024-04-16T17:18:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.165+0000 7f8013b89740 1 -- 172.21.15.18:0/1387440986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 msgr2=0x56110f8c2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.165+0000 7f8013b89740 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8c2390 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009a30 tx=0x7f7ff4009260 comp rx=0 tx=0).stop 2024-04-16T17:18:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.166+0000 7f8013b89740 1 -- 172.21.15.18:0/1387440986 shutdown_connections 2024-04-16T17:18:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.166+0000 7f8013b89740 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 0x56110f8b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.166+0000 7f8013b89740 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8c2390 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.166+0000 7f8013b89740 1 --2- 172.21.15.18:0/1387440986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.166+0000 7f8013b89740 1 -- 172.21.15.18:0/1387440986 >> 172.21.15.18:0/1387440986 conn(0x56110f8a78e0 msgr2=0x56110f8a8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:58.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.166+0000 7f8013b89740 1 -- 172.21.15.18:0/1387440986 shutdown_connections 2024-04-16T17:18:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.166+0000 7f8013b89740 1 -- 172.21.15.18:0/1387440986 wait complete. 2024-04-16T17:18:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.167+0000 7f8013b89740 1 -- start start 2024-04-16T17:18:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.167+0000 7f8013b89740 1 Processor -- start 2024-04-16T17:18:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.168+0000 7f8013b89740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8ba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.168+0000 7f8013b89740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.169+0000 7f8013b89740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 0x56110f8bace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.169+0000 7f8009e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.169+0000 7f8009e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8ba260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49292/0 (socket says 172.21.15.18:49292) 2024-04-16T17:18:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.169+0000 7f8009e00640 1 -- 172.21.15.18:0/253621435 learned_addr learned my addr 172.21.15.18:0/253621435 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.169+0000 7f800a800640 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 0x56110f8bace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8003e00640 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.169+0000 7f8013b89740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1c5470 con 0x56110f8ad1b0 2024-04-16T17:18:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1c68e0 con 0x56110f8adad0 2024-04-16T17:18:58.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1c5b40 con 0x56110f8b66b0 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8013b89740 10 client.0 _ll_register_callbacks cb 0x7f7ffc0bd480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8009e00640 1 -- 172.21.15.18:0/253621435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 msgr2=0x56110f8bace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8009e00640 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 0x56110f8bace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8009e00640 1 -- 172.21.15.18:0/253621435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 msgr2=0x56110f8ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8009e00640 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.171+0000 7f8009e00640 1 -- 172.21.15.18:0/253621435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110f8ad900 con 0x56110f8ad1b0 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.172+0000 7f8009e00640 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8ba260 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009830 tx=0x7f7ffc02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30eb10276d379765 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.172+0000 7f7fefe00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.172+0000 7f7fefe00640 1 -- 172.21.15.18:0/253621435 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0c60a0 con 0x56110f8ad1b0 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.173+0000 7f7fefe00640 1 -- 172.21.15.18:0/253621435 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f7ffc034800 con 0x56110f8ad1b0 2024-04-16T17:18:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.173+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff400ee70 con 0x56110f8ad1b0 2024-04-16T17:18:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.173+0000 7f7fefe00640 1 -- 172.21.15.18:0/253621435 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7ffc0bd6b0 con 0x56110f8ad1b0 2024-04-16T17:18:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.173+0000 7f8013b89740 10 client.18993 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.173+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56110f8ad900 con 0x56110f8ad1b0 2024-04-16T17:18:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.174+0000 7f7fefe00640 1 -- 172.21.15.18:0/253621435 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff400ee70 con 0x56110f8ad1b0 2024-04-16T17:18:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.174+0000 7f7fefe00640 1 -- 172.21.15.18:0/253621435 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7ffc0bc570 con 0x56110f8ad1b0 2024-04-16T17:18:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.174+0000 7f8013b89740 10 client.18993 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.174+0000 7f8013b89740 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:18:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.174+0000 7f8013b89740 20 client.18993 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:58.177 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:57 smithi022 ceph-mon[36023]: pgmap v451: 129 pgs: 129 active+clean; 317 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-04-16T17:18:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.174+0000 7f8013b89740 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56110f92d430 0x56110f92f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.174+0000 7f8013b89740 4 client.18993 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:18:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.174+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56110f81a9b0 con 0x56110f92d430 2024-04-16T17:18:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.176+0000 7f8003e00640 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56110f92d430 0x56110f92f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.179+0000 7f8003e00640 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56110f92d430 0x56110f92f850 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea797e0bf2a7b8c server_cookie=6711317f174dfdad in_seq=0 out_seq=0 2024-04-16T17:18:58.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.179+0000 7f7fefe00640 10 client.18993 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.187+0000 7f7fefe00640 1 -- 172.21.15.18:0/253621435 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56110f81a9b0 con 0x56110f92d430 2024-04-16T17:18:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.188+0000 7f7fefe00640 10 client.18993 handle_command_reply: tid=0 2024-04-16T17:18:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.188+0000 7f8013b89740 1 client.18993 shutdown 2024-04-16T17:18:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 msgr2=0x56110f8ba260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f8013b89740 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8ba260 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009830 tx=0x7f7ffc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:18:58.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 shutdown_connections 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f8013b89740 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56110f92d430 0x56110f92f850 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f8013b89740 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56110f8b66b0 0x56110f8bace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f8013b89740 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56110f8adad0 0x56110f8ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f8013b89740 1 --2- 172.21.15.18:0/253621435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56110f8ad1b0 0x56110f8ba260 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 >> 172.21.15.18:0/253621435 conn(0x56110f8a78e0 msgr2=0x56110f8a95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.189+0000 7f7fefe00640 0 client.18993 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.190+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 shutdown_connections 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.190+0000 7f8013b89740 1 -- 172.21.15.18:0/253621435 wait complete. 2024-04-16T17:18:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.190+0000 7f8013b89740 20 client.18993 trim_cache size 0 max 16384 2024-04-16T17:18:58.192 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:18:58.192 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:18:58.192 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:18:58.192 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:18:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.191+0000 7f8010e00640 1 -- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7ff00a3e20 msgr2=0x7f7ff00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.191+0000 7f8010e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7ff00a3e20 0x7f7ff00a62e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8004009f60 tx=0x7f800400d040 comp rx=0 tx=0).stop 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.191+0000 7f8010e00640 1 -- 172.21.15.18:0/1868287862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f800c104450 msgr2=0x7f800c111000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.191+0000 7f8010e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f800c104450 0x7f800c111000 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc037040 tx=0x7f7ffc0323b0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.192+0000 7f8010e00640 1 -- 172.21.15.18:0/1868287862 shutdown_connections 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.192+0000 7f8010e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7ff00a3e20 0x7f7ff00a62e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.192+0000 7f8010e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c10e630 0x7f800c10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.192+0000 7f8010e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f800c109520 0x7f800c10e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.192+0000 7f8010e00640 1 --2- 172.21.15.18:0/1868287862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f800c104450 0x7f800c111000 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.192+0000 7f8010e00640 1 -- 172.21.15.18:0/1868287862 >> 172.21.15.18:0/1868287862 conn(0x7f800c0ffc90 msgr2=0x7f800c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.192+0000 7f8010e00640 1 -- 172.21.15.18:0/1868287862 shutdown_connections 2024-04-16T17:18:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.192+0000 7f8010e00640 1 -- 172.21.15.18:0/1868287862 wait complete. 2024-04-16T17:18:58.228 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:18:58.228 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:18:58.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:57 smithi043 ceph-mon[36901]: pgmap v451: 129 pgs: 129 active+clean; 317 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-04-16T17:18:58.338 INFO:tasks.workunit.client.0.smithi018.stdout: 10 20637 18.18 MB/sec warmup 70 sec latency 826.931 ms 2024-04-16T17:18:58.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.577+0000 7f4964a00640 1 -- 172.21.15.18:0/2368066355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49580ac100 msgr2=0x7f49580b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.577+0000 7f4964a00640 1 --2- 172.21.15.18:0/2368066355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49580ac100 0x7f49580b3c60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f494c009900 tx=0x7f494c02bcb0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.581+0000 7f4964a00640 1 -- 172.21.15.18:0/2368066355 shutdown_connections 2024-04-16T17:18:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.581+0000 7f4964a00640 1 --2- 172.21.15.18:0/2368066355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49580ac100 0x7f49580b3c60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.581+0000 7f4964a00640 1 --2- 172.21.15.18:0/2368066355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49580a5970 0x7f49580ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.581+0000 7f4964a00640 1 --2- 172.21.15.18:0/2368066355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49580a4fc0 0x7f49580a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.581+0000 7f4964a00640 1 -- 172.21.15.18:0/2368066355 >> 172.21.15.18:0/2368066355 conn(0x7f495801a640 msgr2=0x7f495801aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:58.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.585+0000 7f4964a00640 1 -- 172.21.15.18:0/2368066355 shutdown_connections 2024-04-16T17:18:58.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.585+0000 7f4964a00640 1 -- 172.21.15.18:0/2368066355 wait complete. 2024-04-16T17:18:58.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.587+0000 7f4964a00640 1 Processor -- start 2024-04-16T17:18:58.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.589+0000 7f4964a00640 1 -- start start 2024-04-16T17:18:58.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.589+0000 7f4964a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49580a4fc0 0x7f49581455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.589+0000 7f4964a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49580ac100 0x7f4958145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.589+0000 7f4964a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f495813f6b0 0x7f495813fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.589+0000 7f4964a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f49580b7260 con 0x7f49580a4fc0 2024-04-16T17:18:58.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.589+0000 7f4964a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49580b70e0 con 0x7f49580ac100 2024-04-16T17:18:58.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.589+0000 7f4964a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49580b73e0 con 0x7f495813f6b0 2024-04-16T17:18:58.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.590+0000 7f495ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49580ac100 0x7f4958145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.590+0000 7f495ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49580ac100 0x7f4958145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54320/0 (socket says 172.21.15.18:54320) 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.590+0000 7f495ea00640 1 -- 172.21.15.18:0/1080858329 learned_addr learned my addr 172.21.15.18:0/1080858329 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.590+0000 7f495ea00640 1 -- 172.21.15.18:0/1080858329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f495813f6b0 msgr2=0x7f495813fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.590+0000 7f495ea00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f495813f6b0 0x7f495813fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.590+0000 7f495ea00640 1 -- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49580a4fc0 msgr2=0x7f49581455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.590+0000 7f495ea00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49580a4fc0 0x7f49581455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.590+0000 7f495ea00640 1 -- 172.21.15.18:0/1080858329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49581402b0 con 0x7f49580ac100 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.591+0000 7f495ea00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49580ac100 0x7f4958145b30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f495400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.591+0000 7f4953e00640 1 -- 172.21.15.18:0/1080858329 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954011830 con 0x7f49580ac100 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.591+0000 7f4953e00640 1 -- 172.21.15.18:0/1080858329 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4954016070 con 0x7f49580ac100 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.591+0000 7f4953e00640 1 -- 172.21.15.18:0/1080858329 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495400f2e0 con 0x7f49580ac100 2024-04-16T17:18:58.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.591+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495814c390 con 0x7f49580ac100 2024-04-16T17:18:58.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.591+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495814c740 con 0x7f49580ac100 2024-04-16T17:18:58.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.593+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49580a6230 con 0x7f49580ac100 2024-04-16T17:18:58.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.597+0000 7f4953e00640 1 -- 172.21.15.18:0/1080858329 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49540119d0 con 0x7f49580ac100 2024-04-16T17:18:58.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.599+0000 7f4953e00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49480a3d20 0x7f49480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:58.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.599+0000 7f4953e00640 1 -- 172.21.15.18:0/1080858329 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f49540d01b0 con 0x7f49580ac100 2024-04-16T17:18:58.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.600+0000 7f495f400640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49480a3d20 0x7f49480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:58.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.601+0000 7f495f400640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49480a3d20 0x7f49480a61e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49600677d0 tx=0x7f4960067800 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:58.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:58.614+0000 7f4953e00640 1 -- 172.21.15.18:0/1080858329 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f495409c560 con 0x7f49580ac100 2024-04-16T17:18:59.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.142+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49580050c0 con 0x7f49580ac100 2024-04-16T17:18:59.148 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:18:59.148 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:17:53.822480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:18:59.149 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 34 2024-04-16T17:18:59.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.144+0000 7f4953e00640 1 -- 172.21.15.18:0/1080858329 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11857 (secure 0 0 0) 0x7f495409be60 con 0x7f49580ac100 2024-04-16T17:18:59.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.149+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49480a3d20 msgr2=0x7f49480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.149+0000 7f4964a00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49480a3d20 0x7f49480a61e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49600677d0 tx=0x7f4960067800 comp rx=0 tx=0).stop 2024-04-16T17:18:59.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.149+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49580ac100 msgr2=0x7f4958145b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.149+0000 7f4964a00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49580ac100 0x7f4958145b30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f495400b8e0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.150+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 shutdown_connections 2024-04-16T17:18:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.150+0000 7f4964a00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49480a3d20 0x7f49480a61e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.150+0000 7f4964a00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f495813f6b0 0x7f495813fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.150+0000 7f4964a00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49580ac100 0x7f4958145b30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.150+0000 7f4964a00640 1 --2- 172.21.15.18:0/1080858329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49580a4fc0 0x7f49581455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.150+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 >> 172.21.15.18:0/1080858329 conn(0x7f495801a640 msgr2=0x7f49580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.151+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 shutdown_connections 2024-04-16T17:18:59.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.152+0000 7f4964a00640 1 -- 172.21.15.18:0/1080858329 wait complete. 2024-04-16T17:18:59.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:18:59.339 INFO:tasks.workunit.client.0.smithi018.stdout: 10 20924 18.27 MB/sec warmup 71 sec latency 97.762 ms 2024-04-16T17:18:59.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.533+0000 7f49eda00640 1 -- 172.21.15.18:0/546484061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 msgr2=0x7f49e8115270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.533+0000 7f49eda00640 1 --2- 172.21.15.18:0/546484061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 0x7f49e8115270 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009cd0 tx=0x7f49e002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.533+0000 7f49eda00640 1 -- 172.21.15.18:0/546484061 shutdown_connections 2024-04-16T17:18:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.533+0000 7f49eda00640 1 --2- 172.21.15.18:0/546484061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 0x7f49e8115270 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.533+0000 7f49eda00640 1 --2- 172.21.15.18:0/546484061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e8076e50 0x7f49e8112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.533+0000 7f49eda00640 1 --2- 172.21.15.18:0/546484061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49e8076530 0x7f49e8076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.533+0000 7f49eda00640 1 -- 172.21.15.18:0/546484061 >> 172.21.15.18:0/546484061 conn(0x7f49e80ffc90 msgr2=0x7f49e81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.534+0000 7f49eda00640 1 -- 172.21.15.18:0/546484061 shutdown_connections 2024-04-16T17:18:59.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.534+0000 7f49eda00640 1 -- 172.21.15.18:0/546484061 wait complete. 2024-04-16T17:18:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.535+0000 7f49eda00640 1 Processor -- start 2024-04-16T17:18:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.535+0000 7f49eda00640 1 -- start start 2024-04-16T17:18:59.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49eda00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49e8076530 0x7f49e819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49eda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e8076e50 0x7f49e819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49eda00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 0x7f49e81a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49eda00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81189e0 con 0x7f49e8076530 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49eda00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8118860 con 0x7f49e8112e80 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49eda00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8118b60 con 0x7f49e8076e50 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e8076e50 0x7f49e819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e8076e50 0x7f49e819f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54674/0 (socket says 172.21.15.18:54674) 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49e6000640 1 -- 172.21.15.18:0/1447078069 learned_addr learned my addr 172.21.15.18:0/1447078069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.536+0000 7f49e7400640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 0x7f49e81a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.537+0000 7f49e6a00640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49e8076530 0x7f49e819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.537+0000 7f49e6000640 1 -- 172.21.15.18:0/1447078069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 msgr2=0x7f49e81a37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.537+0000 7f49e6000640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 0x7f49e81a37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.537+0000 7f49e6000640 1 -- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49e8076530 msgr2=0x7f49e819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.537+0000 7f49e6000640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49e8076530 0x7f49e819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.537+0000 7f49e6000640 1 -- 172.21.15.18:0/1447078069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e81a3f00 con 0x7f49e8076e50 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.537+0000 7f49e7400640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 0x7f49e81a37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.537+0000 7f49e6000640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e8076e50 0x7f49e819f4a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f49d400e9f0 tx=0x7f49d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.538+0000 7f49df400640 1 -- 172.21.15.18:0/1447078069 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d400cd80 con 0x7f49e8076e50 2024-04-16T17:18:59.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.538+0000 7f49eda00640 1 -- 172.21.15.18:0/1447078069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e81a41f0 con 0x7f49e8076e50 2024-04-16T17:18:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.538+0000 7f49df400640 1 -- 172.21.15.18:0/1447078069 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d4010910 con 0x7f49e8076e50 2024-04-16T17:18:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.538+0000 7f49df400640 1 -- 172.21.15.18:0/1447078069 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4017930 con 0x7f49e8076e50 2024-04-16T17:18:59.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.538+0000 7f49eda00640 1 -- 172.21.15.18:0/1447078069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e81aba80 con 0x7f49e8076e50 2024-04-16T17:18:59.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.542+0000 7f49df400640 1 -- 172.21.15.18:0/1447078069 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49d4010ac0 con 0x7f49e8076e50 2024-04-16T17:18:59.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49f0771740 1 Processor -- start 2024-04-16T17:18:59.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49f0771740 1 -- start start 2024-04-16T17:18:59.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49f0771740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc57130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49f0771740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc57670 0x55f11bc5fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49f0771740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 0x55f11bc62640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49f0771740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f11bbbc020 con 0x55f11bc60250 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49f0771740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f11bb62460 con 0x55f11bc56d50 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49f0771740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f11bbb7fe0 con 0x55f11bc57670 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.543+0000 7f49df400640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49cc0a3d60 0x7f49cc0a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc57670 0x55f11bc5fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49df400640 1 -- 172.21.15.18:0/1447078069 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f49d40d12b0 con 0x7f49e8076e50 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc57670 0x55f11bc5fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54690/0 (socket says 172.21.15.18:54690) 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6a00640 1 -- 172.21.15.18:0/679091441 learned_addr learned my addr 172.21.15.18:0/679091441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e7400640 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc57130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6000640 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 0x55f11bc62640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6a00640 1 -- 172.21.15.18:0/679091441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 msgr2=0x55f11bc57130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6a00640 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc57130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6a00640 1 -- 172.21.15.18:0/679091441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 msgr2=0x55f11bc62640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6a00640 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 0x55f11bc62640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e6a00640 1 -- 172.21.15.18:0/679091441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11bb6a040 con 0x55f11bc57670 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.544+0000 7f49e7400640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49cc0a3d60 0x7f49cc0a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.545+0000 7f49e6a00640 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc57670 0x55f11bc5fd10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009c50 tx=0x7f49d8010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22e8fe1e516375c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.545+0000 7f49d3e00640 1 -- 172.21.15.18:0/679091441 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8004020 con 0x55f11bc57670 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.545+0000 7f49d3e00640 1 -- 172.21.15.18:0/679091441 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d80041c0 con 0x55f11bc57670 2024-04-16T17:18:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49d3e00640 1 -- 172.21.15.18:0/679091441 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80126b0 con 0x55f11bc57670 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49e7400640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49cc0a3d60 0x7f49cc0a6220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009bf0 tx=0x7f49e002c0f0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49f0771740 1 -- 172.21.15.18:0/679091441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc57670 msgr2=0x55f11bc5fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49f0771740 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc57670 0x55f11bc5fd10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009c50 tx=0x7f49d8010740 comp rx=0 tx=0).stop 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49f0771740 1 -- 172.21.15.18:0/679091441 shutdown_connections 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49f0771740 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 0x55f11bc62640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49f0771740 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc57670 0x55f11bc5fd10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49f0771740 1 --2- 172.21.15.18:0/679091441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc57130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.546+0000 7f49f0771740 1 -- 172.21.15.18:0/679091441 >> 172.21.15.18:0/679091441 conn(0x55f11bc51480 msgr2=0x55f11bc51860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.547+0000 7f49f0771740 1 -- 172.21.15.18:0/679091441 shutdown_connections 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.547+0000 7f49f0771740 1 -- 172.21.15.18:0/679091441 wait complete. 2024-04-16T17:18:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.547+0000 7f49f0771740 1 -- start start 2024-04-16T17:18:59.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.547+0000 7f49f0771740 1 Processor -- start 2024-04-16T17:18:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.548+0000 7f49f0771740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc5d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.548+0000 7f49f0771740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc57670 0x55f11bc5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.548+0000 7f49f0771740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 0x55f11bc5dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.548+0000 7f49f0771740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f11bc63090 con 0x55f11bc60250 2024-04-16T17:18:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.548+0000 7f49f0771740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f11bb75830 con 0x55f11bc57670 2024-04-16T17:18:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.548+0000 7f49f0771740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f11bc63210 con 0x55f11bc56d50 2024-04-16T17:18:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.548+0000 7f49f0771740 10 client.0 _ll_register_callbacks cb 0x55f11bb7d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc57670 0x55f11bc5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc57670 0x55f11bc5d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54358/0 (socket says 172.21.15.18:54358) 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 -- 172.21.15.18:0/1096758781 learned_addr learned my addr 172.21.15.18:0/1096758781 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e7400640 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 0x55f11bc5dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6a00640 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc5d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 -- 172.21.15.18:0/1096758781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 msgr2=0x55f11bc5d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc5d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 -- 172.21.15.18:0/1096758781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 msgr2=0x55f11bc5dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 0x55f11bc5dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 -- 172.21.15.18:0/1096758781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11bb79920 con 0x55f11bc57670 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49e6000640 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc57670 0x55f11bc5d690 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f49d4099250 tx=0x7f49d409d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809a6ea0334229b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.549+0000 7f49d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.550+0000 7f49d0c00640 1 -- 172.21.15.18:0/1096758781 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d40b5020 con 0x55f11bc57670 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.550+0000 7f49f0771740 1 -- 172.21.15.18:0/1096758781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d80041c0 con 0x55f11bc57670 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.550+0000 7f49d0c00640 1 -- 172.21.15.18:0/1096758781 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f49d40ce820 con 0x55f11bc57670 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.550+0000 7f49d0c00640 1 -- 172.21.15.18:0/1096758781 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f49d40cea20 con 0x55f11bc57670 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.550+0000 7f49f0771740 10 client.29432 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.550+0000 7f49f0771740 1 -- 172.21.15.18:0/1096758781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f11bb477f0 con 0x55f11bc57670 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.550+0000 7f49d0c00640 1 -- 172.21.15.18:0/1096758781 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d80041c0 con 0x55f11bc57670 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.550+0000 7f49d0c00640 1 -- 172.21.15.18:0/1096758781 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f49d409c3b0 con 0x55f11bc57670 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.551+0000 7f49f0771740 10 client.29432 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.551+0000 7f49f0771740 10 client.29432 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.551+0000 7f49f0771740 20 client.29432 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.551+0000 7f49f0771740 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f11bc70490 0x55f11bc728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.551+0000 7f49f0771740 4 client.29432 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.551+0000 7f49f0771740 1 -- 172.21.15.18:0/1096758781 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f11bbc9a40 con 0x55f11bc70490 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.551+0000 7f49e6a00640 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f11bc70490 0x55f11bc728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.552+0000 7f49d0c00640 10 client.29432 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.552+0000 7f49e6a00640 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f11bc70490 0x55f11bc728c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c6131dbd09e593 server_cookie=36697b7234aa01b6 in_seq=0 out_seq=0 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49d0c00640 1 -- 172.21.15.18:0/1096758781 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f11bbc9a40 con 0x55f11bc70490 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49d0c00640 10 client.29432 handle_command_reply: tid=0 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 client.29432 shutdown 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 -- 172.21.15.18:0/1096758781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc57670 msgr2=0x55f11bc5d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc57670 0x55f11bc5d690 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f49d4099250 tx=0x7f49d409d630 comp rx=0 tx=0).stop 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 -- 172.21.15.18:0/1096758781 shutdown_connections 2024-04-16T17:18:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f11bc70490 0x55f11bc728c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc60250 0x55f11bc5dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49d0c00640 0 client.29432 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:18:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc57670 0x55f11bc5d690 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 --2- 172.21.15.18:0/1096758781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc5d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 -- 172.21.15.18:0/1096758781 >> 172.21.15.18:0/1096758781 conn(0x55f11bc51480 msgr2=0x55f11bc531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 -- 172.21.15.18:0/1096758781 shutdown_connections 2024-04-16T17:18:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.553+0000 7f49f0771740 1 -- 172.21.15.18:0/1096758781 wait complete. 2024-04-16T17:18:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.554+0000 7f49f0771740 20 client.29432 trim_cache size 0 max 16384 2024-04-16T17:18:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.580+0000 7f49f0771740 1 Processor -- start 2024-04-16T17:18:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.580+0000 7f49f0771740 1 -- start start 2024-04-16T17:18:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49f0771740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49f0771740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 0x55f11bc6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49f0771740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc60250 0x55f11bc5c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49f0771740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81b8630 con 0x55f11bc57670 2024-04-16T17:18:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49f0771740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81b5ea0 con 0x55f11bc56d50 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49f0771740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81b3d10 con 0x55f11bc60250 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc60250 0x55f11bc5c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc60250 0x55f11bc5c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54700/0 (socket says 172.21.15.18:54700) 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49e7400640 1 -- 172.21.15.18:0/3385564331 learned_addr learned my addr 172.21.15.18:0/3385564331 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49e6a00640 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.581+0000 7f49e6000640 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 0x55f11bc6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49e6a00640 1 -- 172.21.15.18:0/3385564331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc60250 msgr2=0x55f11bc5c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49e6a00640 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc60250 0x55f11bc5c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49e6a00640 1 -- 172.21.15.18:0/3385564331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 msgr2=0x55f11bc6bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49e6a00640 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 0x55f11bc6bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49e6a00640 1 -- 172.21.15.18:0/3385564331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11b8ce3f0 con 0x55f11bc56d50 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49e6a00640 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc5f880 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000c00 tx=0x7f49d8003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8e7bb7b481bad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49d3400640 1 -- 172.21.15.18:0/3385564331 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d802c020 con 0x55f11bc56d50 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49d3400640 1 -- 172.21.15.18:0/3385564331 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d8009d40 con 0x55f11bc56d50 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.582+0000 7f49d3400640 1 -- 172.21.15.18:0/3385564331 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80052c0 con 0x55f11bc56d50 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 -- 172.21.15.18:0/3385564331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 msgr2=0x55f11bc5f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc5f880 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000c00 tx=0x7f49d8003ec0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 -- 172.21.15.18:0/3385564331 shutdown_connections 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc60250 0x55f11bc5c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 0x55f11bc6bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 --2- 172.21.15.18:0/3385564331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc56d50 0x55f11bc5f880 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 -- 172.21.15.18:0/3385564331 >> 172.21.15.18:0/3385564331 conn(0x55f11bc51480 msgr2=0x55f11bc4fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 -- 172.21.15.18:0/3385564331 shutdown_connections 2024-04-16T17:18:59.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 -- 172.21.15.18:0/3385564331 wait complete. 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.583+0000 7f49f0771740 1 -- start start 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49f0771740 1 Processor -- start 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49f0771740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc68830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49f0771740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 0x55f11bc63ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49f0771740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc60250 0x55f11bc640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49f0771740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81b4360 con 0x55f11bc57670 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49f0771740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81b57d0 con 0x55f11bc60250 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49f0771740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81b4a30 con 0x55f11bc56d50 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49f0771740 10 client.0 _ll_register_callbacks cb 0x7f49d4010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc68830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49e6000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 0x55f11bc63ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc68830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54714/0 (socket says 172.21.15.18:54714) 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49e6a00640 1 -- 172.21.15.18:0/1821130276 learned_addr learned my addr 172.21.15.18:0/1821130276 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.584+0000 7f49e7400640 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc60250 0x55f11bc640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49e6a00640 1 -- 172.21.15.18:0/1821130276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc60250 msgr2=0x55f11bc640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49e6a00640 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc60250 0x55f11bc640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49e6a00640 1 -- 172.21.15.18:0/1821130276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 msgr2=0x55f11bc63ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49e6a00640 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 0x55f11bc63ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49e6a00640 1 -- 172.21.15.18:0/1821130276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11b9987d0 con 0x55f11bc56d50 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49e6a00640 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc68830 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f49d8003e60 tx=0x7f49d8010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9814997682dca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:18:59.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49d0c00640 1 -- 172.21.15.18:0/1821130276 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d802c050 con 0x55f11bc56d50 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49d0c00640 1 -- 172.21.15.18:0/1821130276 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f49d80035f0 con 0x55f11bc56d50 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.585+0000 7f49f0771740 1 -- 172.21.15.18:0/1821130276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d8009d40 con 0x55f11bc56d50 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49d0c00640 1 -- 172.21.15.18:0/1821130276 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f49d8042050 con 0x55f11bc56d50 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49f0771740 10 client.29275 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49f0771740 1 -- 172.21.15.18:0/1821130276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f11bcd6800 con 0x55f11bc56d50 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49d0c00640 1 -- 172.21.15.18:0/1821130276 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d8009d40 con 0x55f11bc56d50 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49d0c00640 1 -- 172.21.15.18:0/1821130276 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f49d800f040 con 0x55f11bc56d50 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49f0771740 10 client.29275 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49f0771740 10 client.29275 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49f0771740 20 client.29275 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49f0771740 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f11bc70490 0x55f11bcd8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49f0771740 4 client.29275 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49f0771740 1 -- 172.21.15.18:0/1821130276 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f11bbc38b0 con 0x55f11bc70490 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.586+0000 7f49e6000640 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f11bc70490 0x55f11bcd8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49e6000640 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f11bc70490 0x55f11bcd8c80 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d4a8003f70c6309 server_cookie=7708efc37b481b5b in_seq=0 out_seq=0 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49d0c00640 10 client.29275 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49d0c00640 1 -- 172.21.15.18:0/1821130276 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f11bbc38b0 con 0x55f11bc70490 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49d0c00640 10 client.29275 handle_command_reply: tid=0 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 client.29275 shutdown 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 -- 172.21.15.18:0/1821130276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 msgr2=0x55f11bc68830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc68830 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f49d8003e60 tx=0x7f49d8010d80 comp rx=0 tx=0).stop 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 -- 172.21.15.18:0/1821130276 shutdown_connections 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f11bc70490 0x55f11bcd8c80 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f11bc60250 0x55f11bc640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f11bc57670 0x55f11bc63ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 --2- 172.21.15.18:0/1821130276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f11bc56d50 0x55f11bc68830 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49f0771740 1 -- 172.21.15.18:0/1821130276 >> 172.21.15.18:0/1821130276 conn(0x55f11bc51480 msgr2=0x55f11bc5c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.587+0000 7f49d0c00640 0 client.29275 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:18:59.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49f0771740 1 -- 172.21.15.18:0/1821130276 shutdown_connections 2024-04-16T17:18:59.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49f0771740 1 -- 172.21.15.18:0/1821130276 wait complete. 2024-04-16T17:18:59.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49f0771740 20 client.29275 trim_cache size 0 max 16384 2024-04-16T17:18:59.596 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:18:59.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49eda00640 1 -- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49cc0a3d60 msgr2=0x7f49cc0a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49eda00640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49cc0a3d60 0x7f49cc0a6220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009bf0 tx=0x7f49e002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49eda00640 1 -- 172.21.15.18:0/1447078069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e8076e50 msgr2=0x7f49e819f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49eda00640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e8076e50 0x7f49e819f4a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f49d400e9f0 tx=0x7f49d400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49eda00640 1 -- 172.21.15.18:0/1447078069 shutdown_connections 2024-04-16T17:18:59.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.588+0000 7f49eda00640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f49cc0a3d60 0x7f49cc0a6220 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.589+0000 7f49eda00640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49e8112e80 0x7f49e81a37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.589+0000 7f49eda00640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e8076e50 0x7f49e819f4a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.589+0000 7f49eda00640 1 --2- 172.21.15.18:0/1447078069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49e8076530 0x7f49e819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.589+0000 7f49eda00640 1 -- 172.21.15.18:0/1447078069 >> 172.21.15.18:0/1447078069 conn(0x7f49e80ffc90 msgr2=0x7f49e8114610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.589+0000 7f49eda00640 1 -- 172.21.15.18:0/1447078069 shutdown_connections 2024-04-16T17:18:59.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.589+0000 7f49eda00640 1 -- 172.21.15.18:0/1447078069 wait complete. 2024-04-16T17:18:59.616 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 -- 172.21.15.18:0/3357532059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103270 msgr2=0x7fab98103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 --2- 172.21.15.18:0/3357532059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103270 0x7fab98103650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fab88009900 tx=0x7fab8802bce0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 -- 172.21.15.18:0/3357532059 shutdown_connections 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 --2- 172.21.15.18:0/3357532059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9810c5b0 0x7fab9810ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 --2- 172.21.15.18:0/3357532059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab98103b90 0x7fab9810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 --2- 172.21.15.18:0/3357532059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103270 0x7fab98103650 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 -- 172.21.15.18:0/3357532059 >> 172.21.15.18:0/3357532059 conn(0x7fab980780b0 msgr2=0x7fab980784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 -- 172.21.15.18:0/3357532059 shutdown_connections 2024-04-16T17:18:59.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.894+0000 7fab9e600640 1 -- 172.21.15.18:0/3357532059 wait complete. 2024-04-16T17:18:59.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab9e600640 1 Processor -- start 2024-04-16T17:18:59.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab9e600640 1 -- start start 2024-04-16T17:18:59.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab9e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab98103270 0x7fab980737d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab9e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103b90 0x7fab9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab9e600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9810c5b0 0x7fab9806ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab9e600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fab98114640 con 0x7fab98103270 2024-04-16T17:18:59.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab9e600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fab981144c0 con 0x7fab9810c5b0 2024-04-16T17:18:59.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab9e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab981147c0 con 0x7fab98103b90 2024-04-16T17:18:59.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9810c5b0 0x7fab9806ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103b90 0x7fab9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9810c5b0 0x7fab9806ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54390/0 (socket says 172.21.15.18:54390) 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103b90 0x7fab9806c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54734/0 (socket says 172.21.15.18:54734) 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.896+0000 7fab97e00640 1 -- 172.21.15.18:0/3526970893 learned_addr learned my addr 172.21.15.18:0/3526970893 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab97400640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab98103270 0x7fab980737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab97e00640 1 -- 172.21.15.18:0/3526970893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103b90 msgr2=0x7fab9806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab97e00640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103b90 0x7fab9806c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab97e00640 1 -- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab98103270 msgr2=0x7fab980737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab97e00640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab98103270 0x7fab980737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab97e00640 1 -- 172.21.15.18:0/3526970893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9806d3f0 con 0x7fab9810c5b0 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab96a00640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103b90 0x7fab9806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:59.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab97400640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab98103270 0x7fab980737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:59.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.897+0000 7fab97e00640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9810c5b0 0x7fab9806ce30 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fab9000ea60 tx=0x7fab9000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.898+0000 7fab8fe00640 1 -- 172.21.15.18:0/3526970893 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9000ce40 con 0x7fab9810c5b0 2024-04-16T17:18:59.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.898+0000 7fab8fe00640 1 -- 172.21.15.18:0/3526970893 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab90005020 con 0x7fab9810c5b0 2024-04-16T17:18:59.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.898+0000 7fab8fe00640 1 -- 172.21.15.18:0/3526970893 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90010690 con 0x7fab9810c5b0 2024-04-16T17:18:59.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.898+0000 7fab9e600640 1 -- 172.21.15.18:0/3526970893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9806d6e0 con 0x7fab9810c5b0 2024-04-16T17:18:59.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.898+0000 7fab9e600640 1 -- 172.21.15.18:0/3526970893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab98071620 con 0x7fab9810c5b0 2024-04-16T17:18:59.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.901+0000 7faba13b9740 1 Processor -- start 2024-04-16T17:18:59.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.902+0000 7fab8fe00640 1 -- 172.21.15.18:0/3526970893 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab90005520 con 0x7fab9810c5b0 2024-04-16T17:18:59.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.902+0000 7faba13b9740 1 -- start start 2024-04-16T17:18:59.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7faba13b9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fc9d50 0x55e543fca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7faba13b9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fca670 0x55e543fd2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7faba13b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd3250 0x55e543fd5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7faba13b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e543f2f020 con 0x55e543fd3250 2024-04-16T17:18:59.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7faba13b9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e543ed5460 con 0x55e543fc9d50 2024-04-16T17:18:59.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7faba13b9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e543f2afe0 con 0x55e543fca670 2024-04-16T17:18:59.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7fab96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fc9d50 0x55e543fca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7fab96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fc9d50 0x55e543fca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54392/0 (socket says 172.21.15.18:54392) 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7fab96a00640 1 -- 172.21.15.18:0/2233547444 learned_addr learned my addr 172.21.15.18:0/2233547444 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.903+0000 7fab97400640 1 --2- 172.21.15.18:0/2233547444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fca670 0x55e543fd2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.904+0000 7fab8fe00640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab7c0ac5b0 0x7fab7c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.904+0000 7fab97400640 1 -- 172.21.15.18:0/2233547444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fc9d50 msgr2=0x55e543fca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.904+0000 7fab97400640 1 --2- 172.21.15.18:0/2233547444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fc9d50 0x55e543fca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.904+0000 7fab97400640 1 -- 172.21.15.18:0/2233547444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd3250 msgr2=0x55e543fd5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.904+0000 7fab97400640 1 --2- 172.21.15.18:0/2233547444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd3250 0x55e543fd5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.904+0000 7fab97400640 1 -- 172.21.15.18:0/2233547444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e543edd040 con 0x55e543fca670 2024-04-16T17:18:59.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.904+0000 7fab8fe00640 1 -- 172.21.15.18:0/3526970893 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fab900d10f0 con 0x7fab9810c5b0 2024-04-16T17:18:59.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.904+0000 7fab97400640 1 --2- 172.21.15.18:0/2233547444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fca670 0x55e543fd2d10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fab8802c220 tx=0x7fab88009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9323ef33ebcd2381 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.905+0000 7fab8cc00640 1 -- 172.21.15.18:0/2233547444 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88042070 con 0x55e543fca670 2024-04-16T17:18:59.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.905+0000 7fab8cc00640 1 -- 172.21.15.18:0/2233547444 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8802cdd0 con 0x55e543fca670 2024-04-16T17:18:59.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.905+0000 7fab8cc00640 1 -- 172.21.15.18:0/2233547444 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab880046c0 con 0x55e543fca670 2024-04-16T17:18:59.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.906+0000 7faba13b9740 1 -- 172.21.15.18:0/2233547444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fca670 msgr2=0x55e543fd2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.906+0000 7faba13b9740 1 --2- 172.21.15.18:0/2233547444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fca670 0x55e543fd2d10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fab8802c220 tx=0x7fab88009420 comp rx=0 tx=0).stop 2024-04-16T17:18:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.907+0000 7faba13b9740 1 -- 172.21.15.18:0/2233547444 shutdown_connections 2024-04-16T17:18:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.907+0000 7faba13b9740 1 --2- 172.21.15.18:0/2233547444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd3250 0x55e543fd5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.907+0000 7faba13b9740 1 --2- 172.21.15.18:0/2233547444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fca670 0x55e543fd2d10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.907+0000 7faba13b9740 1 --2- 172.21.15.18:0/2233547444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fc9d50 0x55e543fca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.907+0000 7faba13b9740 1 -- 172.21.15.18:0/2233547444 >> 172.21.15.18:0/2233547444 conn(0x55e543fc4480 msgr2=0x55e543fc4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.907+0000 7fab97e00640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab7c0ac5b0 0x7fab7c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.908+0000 7faba13b9740 1 -- 172.21.15.18:0/2233547444 shutdown_connections 2024-04-16T17:18:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.908+0000 7faba13b9740 1 -- 172.21.15.18:0/2233547444 wait complete. 2024-04-16T17:18:59.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.908+0000 7faba13b9740 1 -- start start 2024-04-16T17:18:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.908+0000 7fab97e00640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab7c0ac5b0 0x7fab7c0aea70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fab90099090 tx=0x7fab90005920 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.908+0000 7faba13b9740 1 Processor -- start 2024-04-16T17:18:59.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7faba13b9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fca670 0x55e543fd0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7faba13b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd09c0 0x55e543fd0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7faba13b9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 0x55e543fdf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7faba13b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e543fd5fe0 con 0x55e543fd09c0 2024-04-16T17:18:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7faba13b9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e543ee8830 con 0x55e543fca670 2024-04-16T17:18:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7faba13b9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e543fd6160 con 0x55e543fdd480 2024-04-16T17:18:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7faba13b9740 10 client.0 _ll_register_callbacks cb 0x55e543ef0150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7fab96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fca670 0x55e543fd0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7fab96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fca670 0x55e543fd0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54400/0 (socket says 172.21.15.18:54400) 2024-04-16T17:18:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.909+0000 7fab96a00640 1 -- 172.21.15.18:0/3520844702 learned_addr learned my addr 172.21.15.18:0/3520844702 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.910+0000 7fab96a00640 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 0x55e543fdf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.910+0000 7fab97400640 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd09c0 0x55e543fd0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.910+0000 7fab96a00640 1 -- 172.21.15.18:0/3520844702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 msgr2=0x55e543fdf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.910+0000 7fab96a00640 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 0x55e543fdf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.910+0000 7fab96a00640 1 -- 172.21.15.18:0/3520844702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd09c0 msgr2=0x55e543fd0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.910+0000 7fab96a00640 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd09c0 0x55e543fd0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.910+0000 7fab96a00640 1 -- 172.21.15.18:0/3520844702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e543eec920 con 0x55e543fca670 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.911+0000 7fab96a00640 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fca670 0x55e543fd0480 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x55e543fc9280 tx=0x7fab84004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4d7a78121226a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.911+0000 7fab81600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.911+0000 7fab81600640 1 -- 172.21.15.18:0/3520844702 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84010e70 con 0x55e543fca670 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.911+0000 7fab81600640 1 -- 172.21.15.18:0/3520844702 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fab840140d0 con 0x55e543fca670 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.911+0000 7faba13b9740 1 -- 172.21.15.18:0/3520844702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8802cdd0 con 0x55e543fca670 2024-04-16T17:18:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.912+0000 7fab81600640 1 -- 172.21.15.18:0/3520844702 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab840142d0 con 0x55e543fca670 2024-04-16T17:18:59.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.915+0000 7faba13b9740 10 client.29450 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:59.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.915+0000 7faba13b9740 1 -- 172.21.15.18:0/3520844702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e543eba7f0 con 0x55e543fca670 2024-04-16T17:18:59.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.915+0000 7fab81600640 1 -- 172.21.15.18:0/3520844702 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8802cdd0 con 0x55e543fca670 2024-04-16T17:18:59.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.915+0000 7fab81600640 1 -- 172.21.15.18:0/3520844702 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fab84012480 con 0x55e543fca670 2024-04-16T17:18:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.915+0000 7faba13b9740 10 client.29450 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.916+0000 7faba13b9740 10 client.29450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:18:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.916+0000 7faba13b9740 20 client.29450 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.916+0000 7faba13b9740 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e543fe4000 0x55e543fe6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.916+0000 7faba13b9740 4 client.29450 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:18:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.916+0000 7faba13b9740 1 -- 172.21.15.18:0/3520844702 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e543f3ca40 con 0x55e543fe4000 2024-04-16T17:18:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.916+0000 7fab97400640 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e543fe4000 0x55e543fe6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.917+0000 7fab81600640 10 client.29450 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.917+0000 7fab97400640 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e543fe4000 0x55e543fe6450 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de09a7809f84e289 server_cookie=fe0c0e169ab5ae60 in_seq=0 out_seq=0 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.919+0000 7fab81600640 1 -- 172.21.15.18:0/3520844702 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e543f3ca40 con 0x55e543fe4000 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.919+0000 7fab81600640 10 client.29450 handle_command_reply: tid=0 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.919+0000 7faba13b9740 1 client.29450 shutdown 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.919+0000 7faba13b9740 1 -- 172.21.15.18:0/3520844702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fca670 msgr2=0x55e543fd0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.919+0000 7faba13b9740 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fca670 0x55e543fd0480 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x55e543fc9280 tx=0x7fab84004730 comp rx=0 tx=0).stop 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7faba13b9740 1 -- 172.21.15.18:0/3520844702 shutdown_connections 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7faba13b9740 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e543fe4000 0x55e543fe6450 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7faba13b9740 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 0x55e543fdf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7faba13b9740 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fd09c0 0x55e543fd0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7faba13b9740 1 --2- 172.21.15.18:0/3520844702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fca670 0x55e543fd0480 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7faba13b9740 1 -- 172.21.15.18:0/3520844702 >> 172.21.15.18:0/3520844702 conn(0x55e543fc4480 msgr2=0x55e543fc61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7fab81600640 0 client.29450 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:18:59.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7faba13b9740 1 -- 172.21.15.18:0/3520844702 shutdown_connections 2024-04-16T17:18:59.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.920+0000 7faba13b9740 1 -- 172.21.15.18:0/3520844702 wait complete. 2024-04-16T17:18:59.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.921+0000 7faba13b9740 20 client.29450 trim_cache size 0 max 16384 2024-04-16T17:18:59.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.944+0000 7faba13b9740 1 Processor -- start 2024-04-16T17:18:59.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.944+0000 7faba13b9740 1 -- start start 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.944+0000 7faba13b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fcab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.944+0000 7faba13b9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fd09c0 0x55e543febd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.944+0000 7faba13b9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 0x55e543fec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.944+0000 7faba13b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fab981b85f0 con 0x55e543fca670 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.944+0000 7faba13b9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fab981b5e60 con 0x55e543fd09c0 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.944+0000 7faba13b9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab981b3d30 con 0x55e543fdd480 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fd09c0 0x55e543febd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fd09c0 0x55e543febd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54402/0 (socket says 172.21.15.18:54402) 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab97400640 1 -- 172.21.15.18:0/925489143 learned_addr learned my addr 172.21.15.18:0/925489143 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab97400640 1 -- 172.21.15.18:0/925489143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 msgr2=0x55e543fec260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:18:59.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab97400640 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 0x55e543fec260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab97400640 1 -- 172.21.15.18:0/925489143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 msgr2=0x55e543fcab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab96a00640 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fcab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab97400640 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fcab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab97400640 1 -- 172.21.15.18:0/925489143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e543c413f0 con 0x55e543fd09c0 2024-04-16T17:18:59.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.945+0000 7fab96a00640 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fcab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:18:59.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.946+0000 7fab97400640 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fd09c0 0x55e543febd20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fab88009900 tx=0x7fab880052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da11b8171a56ad78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:57 smithi018 ceph-mon[31776]: pgmap v451: 129 pgs: 129 active+clean; 317 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-04-16T17:18:59.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.946+0000 7fab83e00640 1 -- 172.21.15.18:0/925489143 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88042040 con 0x55e543fd09c0 2024-04-16T17:18:59.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.947+0000 7fab83e00640 1 -- 172.21.15.18:0/925489143 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab88061dc0 con 0x55e543fd09c0 2024-04-16T17:18:59.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.947+0000 7fab83e00640 1 -- 172.21.15.18:0/925489143 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88061330 con 0x55e543fd09c0 2024-04-16T17:18:59.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 -- 172.21.15.18:0/925489143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fd09c0 msgr2=0x55e543febd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fd09c0 0x55e543febd20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fab88009900 tx=0x7fab880052a0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 -- 172.21.15.18:0/925489143 shutdown_connections 2024-04-16T17:18:59.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fdd480 0x55e543fec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fd09c0 0x55e543febd20 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 --2- 172.21.15.18:0/925489143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fcab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 -- 172.21.15.18:0/925489143 >> 172.21.15.18:0/925489143 conn(0x55e543fc4480 msgr2=0x55e543fc4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 -- 172.21.15.18:0/925489143 shutdown_connections 2024-04-16T17:18:59.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.948+0000 7faba13b9740 1 -- 172.21.15.18:0/925489143 wait complete. 2024-04-16T17:18:59.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.949+0000 7faba13b9740 1 -- start start 2024-04-16T17:18:59.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.949+0000 7faba13b9740 1 Processor -- start 2024-04-16T17:18:59.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7faba13b9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fdc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7faba13b9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fdd480 0x55e543fdcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7faba13b9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fd7a40 0x55e543fd7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7faba13b9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fab981b4380 con 0x55e543fca670 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7faba13b9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fab981b5790 con 0x55e543fdd480 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7faba13b9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab981b49f0 con 0x55e543fd7a40 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7faba13b9740 10 client.0 _ll_register_callbacks cb 0x7fab8400bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fdc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fdc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49412/0 (socket says 172.21.15.18:49412) 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 -- 172.21.15.18:0/2244317864 learned_addr learned my addr 172.21.15.18:0/2244317864 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fd7a40 0x55e543fd7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab97400640 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fdd480 0x55e543fdcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 -- 172.21.15.18:0/2244317864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fd7a40 msgr2=0x55e543fd7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fd7a40 0x55e543fd7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 -- 172.21.15.18:0/2244317864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fdd480 msgr2=0x55e543fdcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fdd480 0x55e543fdcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab96a00640 1 -- 172.21.15.18:0/2244317864 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e543d0b7d0 con 0x55e543fca670 2024-04-16T17:18:59.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.950+0000 7fab97400640 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fdd480 0x55e543fdcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:18:59.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.951+0000 7fab96a00640 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fdc6e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fab84017040 tx=0x7fab84000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee702c8f533e16b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:59.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.951+0000 7fab81600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:18:59.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.951+0000 7fab81600640 1 -- 172.21.15.18:0/2244317864 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab840140d0 con 0x55e543fca670 2024-04-16T17:18:59.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.951+0000 7fab81600640 1 -- 172.21.15.18:0/2244317864 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fab84012a90 con 0x55e543fca670 2024-04-16T17:18:59.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.951+0000 7faba13b9740 1 -- 172.21.15.18:0/2244317864 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab88061dc0 con 0x55e543fca670 2024-04-16T17:18:59.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.951+0000 7fab81600640 1 -- 172.21.15.18:0/2244317864 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab84034050 con 0x55e543fca670 2024-04-16T17:18:59.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.951+0000 7faba13b9740 10 client.19017 fetch_fsmap learned FSMap version 34 2024-04-16T17:18:59.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.952+0000 7faba13b9740 1 -- 172.21.15.18:0/2244317864 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e54404a1a0 con 0x55e543fca670 2024-04-16T17:18:59.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.952+0000 7fab81600640 1 -- 172.21.15.18:0/2244317864 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab88061dc0 con 0x55e543fca670 2024-04-16T17:18:59.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.953+0000 7fab81600640 1 -- 172.21.15.18:0/2244317864 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fab840048b0 con 0x55e543fca670 2024-04-16T17:18:59.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.953+0000 7faba13b9740 10 client.19017 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:18:59.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.953+0000 7faba13b9740 10 client.19017 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:18:59.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.953+0000 7faba13b9740 20 client.19017 populate_metadata read hostname 'smithi018' 2024-04-16T17:18:59.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.953+0000 7faba13b9740 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e54404a870 0x55e54404cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:59.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.953+0000 7faba13b9740 4 client.19017 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:18:59.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.953+0000 7faba13b9740 1 -- 172.21.15.18:0/2244317864 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e543f368b0 con 0x55e54404a870 2024-04-16T17:18:59.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.953+0000 7fab97400640 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e54404a870 0x55e54404cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:59.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.955+0000 7fab97400640 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e54404a870 0x55e54404cc60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1ddec0879f15798 server_cookie=b8083304e47d1868 in_seq=0 out_seq=0 2024-04-16T17:18:59.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.955+0000 7fab81600640 10 client.19017 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:18:59.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.956+0000 7fab81600640 1 -- 172.21.15.18:0/2244317864 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e543f368b0 con 0x55e54404a870 2024-04-16T17:18:59.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.956+0000 7fab81600640 10 client.19017 handle_command_reply: tid=0 2024-04-16T17:18:59.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.956+0000 7faba13b9740 1 client.19017 shutdown 2024-04-16T17:18:59.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.957+0000 7faba13b9740 1 -- 172.21.15.18:0/2244317864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 msgr2=0x55e543fdc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.957+0000 7faba13b9740 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fdc6e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fab84017040 tx=0x7fab84000bc0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.958+0000 7faba13b9740 1 -- 172.21.15.18:0/2244317864 shutdown_connections 2024-04-16T17:18:59.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.958+0000 7faba13b9740 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e54404a870 0x55e54404cc60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.958+0000 7faba13b9740 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e543fd7a40 0x55e543fd7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.958+0000 7faba13b9740 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e543fdd480 0x55e543fdcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.958+0000 7faba13b9740 1 --2- 172.21.15.18:0/2244317864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e543fca670 0x55e543fdc6e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.958+0000 7faba13b9740 1 -- 172.21.15.18:0/2244317864 >> 172.21.15.18:0/2244317864 conn(0x55e543fc4480 msgr2=0x55e543fcab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.958+0000 7fab81600640 0 client.19017 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:18:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.958+0000 7faba13b9740 1 -- 172.21.15.18:0/2244317864 shutdown_connections 2024-04-16T17:18:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.959+0000 7faba13b9740 1 -- 172.21.15.18:0/2244317864 wait complete. 2024-04-16T17:18:59.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.960+0000 7faba13b9740 20 client.19017 trim_cache size 0 max 16384 2024-04-16T17:18:59.961 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:18:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.960+0000 7fab9e600640 1 -- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab7c0ac5b0 msgr2=0x7fab7c0aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.960+0000 7fab9e600640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab7c0ac5b0 0x7fab7c0aea70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fab90099090 tx=0x7fab90005920 comp rx=0 tx=0).stop 2024-04-16T17:18:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.960+0000 7fab9e600640 1 -- 172.21.15.18:0/3526970893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9810c5b0 msgr2=0x7fab9806ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.960+0000 7fab9e600640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9810c5b0 0x7fab9806ce30 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fab9000ea60 tx=0x7fab9000ef20 comp rx=0 tx=0).stop 2024-04-16T17:18:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.961+0000 7fab9e600640 1 -- 172.21.15.18:0/3526970893 shutdown_connections 2024-04-16T17:18:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.961+0000 7fab9e600640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fab7c0ac5b0 0x7fab7c0aea70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.961+0000 7fab9e600640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9810c5b0 0x7fab9806ce30 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.961+0000 7fab9e600640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab98103b90 0x7fab9806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.961+0000 7fab9e600640 1 --2- 172.21.15.18:0/3526970893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab98103270 0x7fab980737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.961+0000 7fab9e600640 1 -- 172.21.15.18:0/3526970893 >> 172.21.15.18:0/3526970893 conn(0x7fab980780b0 msgr2=0x7fab980fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:59.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.962+0000 7fab9e600640 1 -- 172.21.15.18:0/3526970893 shutdown_connections 2024-04-16T17:18:59.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:18:59.963+0000 7fab9e600640 1 -- 172.21.15.18:0/3526970893 wait complete. 2024-04-16T17:18:59.987 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:19:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:59 smithi022 ceph-mon[36023]: pgmap v452: 129 pgs: 129 active+clean; 317 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 303 op/s 2024-04-16T17:19:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:18:59 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1080858329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 -- 172.21.15.18:0/163409303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 msgr2=0x7f18b01011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 --2- 172.21.15.18:0/163409303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 0x7f18b01011b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009900 tx=0x7f18a002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 -- 172.21.15.18:0/163409303 shutdown_connections 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 --2- 172.21.15.18:0/163409303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b01061d0 0x7f18b0106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 --2- 172.21.15.18:0/163409303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18b01016f0 0x7f18b0105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 --2- 172.21.15.18:0/163409303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 0x7f18b01011b0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 -- 172.21.15.18:0/163409303 >> 172.21.15.18:0/163409303 conn(0x7f18b0077ab0 msgr2=0x7f18b00fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 -- 172.21.15.18:0/163409303 shutdown_connections 2024-04-16T17:19:00.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.219+0000 7f18afe00640 1 -- 172.21.15.18:0/163409303 wait complete. 2024-04-16T17:19:00.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.221+0000 7f18afe00640 1 Processor -- start 2024-04-16T17:19:00.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.221+0000 7f18afe00640 1 -- start start 2024-04-16T17:19:00.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.221+0000 7f18afe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 0x7f18b01a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.221+0000 7f18afe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18b01016f0 0x7f18b01a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.221+0000 7f18afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b01061d0 0x7f18b019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18afe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01187c0 con 0x7f18b01016f0 2024-04-16T17:19:00.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18afe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0118640 con 0x7f18b0100dd0 2024-04-16T17:19:00.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18afe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0118940 con 0x7f18b01061d0 2024-04-16T17:19:00.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 0x7f18b01a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 0x7f18b01a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54434/0 (socket says 172.21.15.18:54434) 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18aea00640 1 -- 172.21.15.18:0/520835459 learned_addr learned my addr 172.21.15.18:0/520835459 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18ae000640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18b01016f0 0x7f18b01a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18af400640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b01061d0 0x7f18b019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18aea00640 1 -- 172.21.15.18:0/520835459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b01061d0 msgr2=0x7f18b019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18aea00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b01061d0 0x7f18b019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18aea00640 1 -- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18b01016f0 msgr2=0x7f18b01a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18aea00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18b01016f0 0x7f18b01a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.222+0000 7f18aea00640 1 -- 172.21.15.18:0/520835459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b019f3e0 con 0x7f18b0100dd0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.223+0000 7f18aea00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 0x7f18b01a4a60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006010 tx=0x7f18a0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.223+0000 7f18a7400640 1 -- 172.21.15.18:0/520835459 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0042020 con 0x7f18b0100dd0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.223+0000 7f18a7400640 1 -- 172.21.15.18:0/520835459 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a002ce30 con 0x7f18b0100dd0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.223+0000 7f18a7400640 1 -- 172.21.15.18:0/520835459 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0003af0 con 0x7f18b0100dd0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.223+0000 7f18afe00640 1 -- 172.21.15.18:0/520835459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b019f670 con 0x7f18b0100dd0 2024-04-16T17:19:00.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.223+0000 7f18afe00640 1 -- 172.21.15.18:0/520835459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b01ab9e0 con 0x7f18b0100dd0 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.226+0000 7f18b857c740 1 Processor -- start 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.226+0000 7f18b857c740 1 -- start start 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.226+0000 7f18a7400640 1 -- 172.21.15.18:0/520835459 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18a0045020 con 0x7f18b0100dd0 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18b857c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e04f10 0x563d51e052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18b857c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e0ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18b857c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e0e410 0x563d51e10800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18b857c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563d51d6a020 con 0x563d51e05830 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18b857c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563d51d10460 con 0x563d51e0e410 2024-04-16T17:19:00.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18b857c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d51d65fe0 con 0x563d51e04f10 2024-04-16T17:19:00.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e04f10 0x563d51e052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18aea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e0ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e04f10 0x563d51e052f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54800/0 (socket says 172.21.15.18:54800) 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18af400640 1 -- 172.21.15.18:0/21691748 learned_addr learned my addr 172.21.15.18:0/21691748 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.227+0000 7f18ae000640 1 --2- 172.21.15.18:0/21691748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e0e410 0x563d51e10800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f18af400640 1 -- 172.21.15.18:0/21691748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e0e410 msgr2=0x563d51e10800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f18af400640 1 --2- 172.21.15.18:0/21691748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e0e410 0x563d51e10800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f18af400640 1 -- 172.21.15.18:0/21691748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 msgr2=0x563d51e0ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f18af400640 1 --2- 172.21.15.18:0/21691748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e0ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f18af400640 1 -- 172.21.15.18:0/21691748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d51d18040 con 0x563d51e04f10 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f18af400640 1 --2- 172.21.15.18:0/21691748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e04f10 0x563d51e052f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009f50 tx=0x7f18a80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc5ea660e28b418 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f189be00640 1 -- 172.21.15.18:0/21691748 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004d20 con 0x563d51e04f10 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f189be00640 1 -- 172.21.15.18:0/21691748 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8004050 con 0x563d51e04f10 2024-04-16T17:19:00.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.228+0000 7f189be00640 1 -- 172.21.15.18:0/21691748 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8005240 con 0x563d51e04f10 2024-04-16T17:19:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 -- 172.21.15.18:0/21691748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e04f10 msgr2=0x563d51e052f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 --2- 172.21.15.18:0/21691748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e04f10 0x563d51e052f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009f50 tx=0x7f18a80118d0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18a7400640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18940a3d20 0x7f18940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 -- 172.21.15.18:0/21691748 shutdown_connections 2024-04-16T17:19:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 --2- 172.21.15.18:0/21691748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e0e410 0x563d51e10800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 --2- 172.21.15.18:0/21691748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e0ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 --2- 172.21.15.18:0/21691748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e04f10 0x563d51e052f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 -- 172.21.15.18:0/21691748 >> 172.21.15.18:0/21691748 conn(0x563d51dff640 msgr2=0x563d51dffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18a7400640 1 -- 172.21.15.18:0/520835459 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f18a00f2950 con 0x7f18b0100dd0 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 -- 172.21.15.18:0/21691748 shutdown_connections 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18aea00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18940a3d20 0x7f18940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.229+0000 7f18b857c740 1 -- 172.21.15.18:0/21691748 wait complete. 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.230+0000 7f18b857c740 1 -- start start 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.230+0000 7f18aea00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18940a3d20 0x7f18940a61e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x563d51e049e0 tx=0x7f18a00f8530 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.230+0000 7f18b857c740 1 Processor -- start 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18b857c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e04f10 0x563d51e0b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18b857c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e05830 0x563d51e0b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18b857c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 0x563d51e0bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18b857c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563d51e11200 con 0x563d51e04f10 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18b857c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563d51d23830 con 0x563d51e05830 2024-04-16T17:19:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18b857c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d51e11380 con 0x563d51e0e410 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18b857c740 10 client.0 _ll_register_callbacks cb 0x563d51d2b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18af400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e04f10 0x563d51e0b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18af400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e04f10 0x563d51e0b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49446/0 (socket says 172.21.15.18:49446) 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18af400640 1 -- 172.21.15.18:0/1557575954 learned_addr learned my addr 172.21.15.18:0/1557575954 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18ae000640 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e05830 0x563d51e0b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.231+0000 7f18af400640 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 0x563d51e0bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f18ae000640 1 -- 172.21.15.18:0/1557575954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 msgr2=0x563d51e0bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f18ae000640 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 0x563d51e0bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f18ae000640 1 -- 172.21.15.18:0/1557575954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e04f10 msgr2=0x563d51e0b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f18ae000640 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e04f10 0x563d51e0b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f18ae000640 1 -- 172.21.15.18:0/1557575954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d51d27920 con 0x563d51e05830 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f18af400640 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e04f10 0x563d51e0b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f18ae000640 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e05830 0x563d51e0b9b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f189c0126d0 tx=0x7f189c012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a66c898aa5c4596f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f1898c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f1898c00640 1 -- 172.21.15.18:0/1557575954 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c00f920 con 0x563d51e05830 2024-04-16T17:19:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f18b857c740 1 -- 172.21.15.18:0/1557575954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a8004050 con 0x563d51e05830 2024-04-16T17:19:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.232+0000 7f1898c00640 1 -- 172.21.15.18:0/1557575954 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f189c00fac0 con 0x563d51e05830 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f1898c00640 1 -- 172.21.15.18:0/1557575954 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f189c00fcc0 con 0x563d51e05830 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f18b857c740 10 client.29468 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f18b857c740 1 -- 172.21.15.18:0/1557575954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563d51cf57f0 con 0x563d51e05830 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f1898c00640 1 -- 172.21.15.18:0/1557575954 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8004050 con 0x563d51e05830 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f1898c00640 1 -- 172.21.15.18:0/1557575954 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f189c01c420 con 0x563d51e05830 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f18b857c740 10 client.29468 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f18b857c740 10 client.29468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f18b857c740 20 client.29468 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f18b857c740 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d51e1e8c0 0x563d51e20d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f18b857c740 4 client.29468 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.233+0000 7f18b857c740 1 -- 172.21.15.18:0/1557575954 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d51d77a40 con 0x563d51e1e8c0 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.234+0000 7f18af400640 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d51e1e8c0 0x563d51e20d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.234+0000 7f18af400640 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d51e1e8c0 0x563d51e20d10 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbb5529c0be3bf6 server_cookie=45586244c5d6c307 in_seq=0 out_seq=0 2024-04-16T17:19:00.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.234+0000 7f1898c00640 10 client.29468 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f1898c00640 1 -- 172.21.15.18:0/1557575954 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d51d77a40 con 0x563d51e1e8c0 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f1898c00640 10 client.29468 handle_command_reply: tid=0 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 client.29468 shutdown 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 -- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e05830 msgr2=0x563d51e0b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e05830 0x563d51e0b9b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f189c0126d0 tx=0x7f189c012b90 comp rx=0 tx=0).stop 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 -- 172.21.15.18:0/1557575954 shutdown_connections 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d51e1e8c0 0x563d51e20d10 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 0x563d51e0bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e05830 0x563d51e0b9b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 --2- 172.21.15.18:0/1557575954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e04f10 0x563d51e0b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.235+0000 7f18b857c740 1 -- 172.21.15.18:0/1557575954 >> 172.21.15.18:0/1557575954 conn(0x563d51dff640 msgr2=0x563d51e01390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.236+0000 7f1898c00640 0 client.29468 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.236+0000 7f18b857c740 1 -- 172.21.15.18:0/1557575954 shutdown_connections 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.236+0000 7f18b857c740 1 -- 172.21.15.18:0/1557575954 wait complete. 2024-04-16T17:19:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.236+0000 7f18b857c740 20 client.29468 trim_cache size 0 max 16384 2024-04-16T17:19:00.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.250+0000 7f18b857c740 1 Processor -- start 2024-04-16T17:19:00.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.250+0000 7f18b857c740 1 -- start start 2024-04-16T17:19:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:59 smithi043 ceph-mon[36901]: pgmap v452: 129 pgs: 129 active+clean; 317 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 303 op/s 2024-04-16T17:19:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:18:59 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1080858329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18b857c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e0dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18b857c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e05830 0x563d51e1a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18b857c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e0e410 0x563d51dfe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18b857c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b83d0 con 0x563d51e0e410 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18b857c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b5c40 con 0x563d51e04f10 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18b857c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b3ab0 con 0x563d51e05830 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e0dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18ae000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e05830 0x563d51e1a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18ae000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e05830 0x563d51e1a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54828/0 (socket says 172.21.15.18:54828) 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e0dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54454/0 (socket says 172.21.15.18:54454) 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18ae000640 1 -- 172.21.15.18:0/116384193 learned_addr learned my addr 172.21.15.18:0/116384193 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.251+0000 7f18af400640 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e0e410 0x563d51dfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f18af400640 1 -- 172.21.15.18:0/116384193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e05830 msgr2=0x563d51e1a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f18af400640 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e05830 0x563d51e1a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f18af400640 1 -- 172.21.15.18:0/116384193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e0e410 msgr2=0x563d51dfe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f18af400640 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e0e410 0x563d51dfe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f18af400640 1 -- 172.21.15.18:0/116384193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d51e05660 con 0x563d51e04f10 2024-04-16T17:19:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f18ae000640 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e05830 0x563d51e1a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f18af400640 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e0dba0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009250 tx=0x7f189c009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86edd460b49929b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f189b400640 1 -- 172.21.15.18:0/116384193 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004750 con 0x563d51e04f10 2024-04-16T17:19:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f189b400640 1 -- 172.21.15.18:0/116384193 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8004dd0 con 0x563d51e04f10 2024-04-16T17:19:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.252+0000 7f189b400640 1 -- 172.21.15.18:0/116384193 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a802c830 con 0x563d51e04f10 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.253+0000 7f18b857c740 1 -- 172.21.15.18:0/116384193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 msgr2=0x563d51e0dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.253+0000 7f18b857c740 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e0dba0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009250 tx=0x7f189c009e30 comp rx=0 tx=0).stop 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.253+0000 7f18b857c740 1 -- 172.21.15.18:0/116384193 shutdown_connections 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.253+0000 7f18b857c740 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e0e410 0x563d51dfe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.253+0000 7f18b857c740 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e05830 0x563d51e1a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.253+0000 7f18b857c740 1 --2- 172.21.15.18:0/116384193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e0dba0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.253+0000 7f18b857c740 1 -- 172.21.15.18:0/116384193 >> 172.21.15.18:0/116384193 conn(0x563d51dff640 msgr2=0x563d51dfde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.254+0000 7f18b857c740 1 -- 172.21.15.18:0/116384193 shutdown_connections 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.254+0000 7f18b857c740 1 -- 172.21.15.18:0/116384193 wait complete. 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.254+0000 7f18b857c740 1 -- start start 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.254+0000 7f18b857c740 1 Processor -- start 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18b857c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e16a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18b857c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e07e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18b857c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 0x563d51e083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18b857c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b4100 con 0x563d51e05830 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18b857c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b5570 con 0x563d51e04f10 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18b857c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b47d0 con 0x563d51e0e410 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18b857c740 10 client.0 _ll_register_callbacks cb 0x7f189c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18ae000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e07e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e16a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e16a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54458/0 (socket says 172.21.15.18:54458) 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18af400640 1 -- 172.21.15.18:0/202122697 learned_addr learned my addr 172.21.15.18:0/202122697 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.255+0000 7f18af400640 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 0x563d51e083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f18ae000640 1 -- 172.21.15.18:0/202122697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 msgr2=0x563d51e083e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f18ae000640 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 0x563d51e083e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f18ae000640 1 -- 172.21.15.18:0/202122697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 msgr2=0x563d51e16a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f18ae000640 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e16a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f18ae000640 1 -- 172.21.15.18:0/202122697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d51a7c3f0 con 0x563d51e05830 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f18ae000640 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e07e60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f189c01c910 tx=0x7f189c01cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e916a3caf65ffae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f1898c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f1898c00640 1 -- 172.21.15.18:0/202122697 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c005c50 con 0x563d51e05830 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f1898c00640 1 -- 172.21.15.18:0/202122697 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f189c01c630 con 0x563d51e05830 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.256+0000 7f18b857c740 1 -- 172.21.15.18:0/202122697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a8004dd0 con 0x563d51e05830 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f1898c00640 1 -- 172.21.15.18:0/202122697 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f189c00fe60 con 0x563d51e05830 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f18b857c740 10 client.19035 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f18b857c740 1 -- 172.21.15.18:0/202122697 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563d51b467d0 con 0x563d51e05830 2024-04-16T17:19:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f1898c00640 1 -- 172.21.15.18:0/202122697 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8004dd0 con 0x563d51e05830 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f1898c00640 1 -- 172.21.15.18:0/202122697 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f189c003bc0 con 0x563d51e05830 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f18b857c740 10 client.19035 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f18b857c740 10 client.19035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f18b857c740 20 client.19035 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f18b857c740 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d51e853c0 0x563d51e877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f18b857c740 4 client.19035 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.257+0000 7f18b857c740 1 -- 172.21.15.18:0/202122697 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d51e210a0 con 0x563d51e853c0 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.258+0000 7f18af400640 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d51e853c0 0x563d51e877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.258+0000 7f18af400640 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d51e853c0 0x563d51e877e0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62b63977c84ac0e server_cookie=cebdf771f88c4182 in_seq=0 out_seq=0 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.258+0000 7f1898c00640 10 client.19035 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.258+0000 7f1898c00640 1 -- 172.21.15.18:0/202122697 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d51e210a0 con 0x563d51e853c0 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.258+0000 7f1898c00640 10 client.19035 handle_command_reply: tid=0 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.258+0000 7f18b857c740 1 client.19035 shutdown 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 -- 172.21.15.18:0/202122697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 msgr2=0x563d51e07e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e07e60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f189c01c910 tx=0x7f189c01cd30 comp rx=0 tx=0).stop 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 -- 172.21.15.18:0/202122697 shutdown_connections 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d51e853c0 0x563d51e877e0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d51e0e410 0x563d51e083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d51e05830 0x563d51e07e60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 --2- 172.21.15.18:0/202122697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d51e04f10 0x563d51e16a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 -- 172.21.15.18:0/202122697 >> 172.21.15.18:0/202122697 conn(0x563d51dff640 msgr2=0x563d51e01220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f1898c00640 0 client.19035 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 -- 172.21.15.18:0/202122697 shutdown_connections 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 1 -- 172.21.15.18:0/202122697 wait complete. 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.259+0000 7f18b857c740 20 client.19035 trim_cache size 0 max 16384 2024-04-16T17:19:00.262 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 -- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18940a3d20 msgr2=0x7f18940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18940a3d20 0x7f18940a61e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x563d51e049e0 tx=0x7f18a00f8530 comp rx=0 tx=0).stop 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 -- 172.21.15.18:0/520835459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 msgr2=0x7f18b01a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 0x7f18b01a4a60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006010 tx=0x7f18a0004270 comp rx=0 tx=0).stop 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 -- 172.21.15.18:0/520835459 shutdown_connections 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18940a3d20 0x7f18940a61e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b01061d0 0x7f18b019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18b01016f0 0x7f18b01a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 --2- 172.21.15.18:0/520835459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18b0100dd0 0x7f18b01a4a60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:00.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 -- 172.21.15.18:0/520835459 >> 172.21.15.18:0/520835459 conn(0x7f18b0077ab0 msgr2=0x7f18b0103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:00.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 -- 172.21.15.18:0/520835459 shutdown_connections 2024-04-16T17:19:00.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:00.260+0000 7f18afe00640 1 -- 172.21.15.18:0/520835459 wait complete. 2024-04-16T17:19:00.273 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:59 smithi018 ceph-mon[31776]: pgmap v452: 129 pgs: 129 active+clean; 317 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 303 op/s 2024-04-16T17:19:00.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:18:59 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1080858329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:19:00.339 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21184 18.30 MB/sec warmup 72 sec latency 192.786 ms 2024-04-16T17:19:01.276 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:19:01.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:19:01.277 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:19:01.339 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21184 18.05 MB/sec warmup 73 sec latency 1192.831 ms 2024-04-16T17:19:01.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 -- 172.21.15.18:0/46424050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa06010e4f0 msgr2=0x7fa06010e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 --2- 172.21.15.18:0/46424050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa06010e4f0 0x7fa06010e8d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa050009930 tx=0x7fa05002bcb0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 -- 172.21.15.18:0/46424050 shutdown_connections 2024-04-16T17:19:01.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 --2- 172.21.15.18:0/46424050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa060108ee0 0x7fa0601152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 --2- 172.21.15.18:0/46424050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa060108540 0x7fa0601089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 --2- 172.21.15.18:0/46424050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa06010e4f0 0x7fa06010e8d0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 -- 172.21.15.18:0/46424050 >> 172.21.15.18:0/46424050 conn(0x7fa060077e10 msgr2=0x7fa060106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:01.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 -- 172.21.15.18:0/46424050 shutdown_connections 2024-04-16T17:19:01.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.488+0000 7fa05fe00640 1 -- 172.21.15.18:0/46424050 wait complete. 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.489+0000 7fa05fe00640 1 Processor -- start 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.489+0000 7fa05fe00640 1 -- start start 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa060108540 0x7fa06019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa060108ee0 0x7fa06019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa06010e4f0 0x7fa0601a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05fe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601187c0 con 0x7fa06010e4f0 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05fe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa060118640 con 0x7fa060108540 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa060118940 con 0x7fa060108ee0 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa060108540 0x7fa06019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa060108540 0x7fa06019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54482/0 (socket says 172.21.15.18:54482) 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05ea00640 1 -- 172.21.15.18:0/934110600 learned_addr learned my addr 172.21.15.18:0/934110600 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05e000640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa060108ee0 0x7fa06019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.490+0000 7fa05f400640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa06010e4f0 0x7fa0601a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa05ea00640 1 -- 172.21.15.18:0/934110600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa060108ee0 msgr2=0x7fa06019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa05ea00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa060108ee0 0x7fa06019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa05ea00640 1 -- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa06010e4f0 msgr2=0x7fa0601a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa05ea00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa06010e4f0 0x7fa0601a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa05ea00640 1 -- 172.21.15.18:0/934110600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0601a3de0 con 0x7fa060108540 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa05e000640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa060108ee0 0x7fa06019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:01.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa05ea00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa060108540 0x7fa06019edf0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa050005940 tx=0x7fa05002cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:01.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa064a00640 1 -- 172.21.15.18:0/934110600 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050043020 con 0x7fa060108540 2024-04-16T17:19:01.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa05fe00640 1 -- 172.21.15.18:0/934110600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0601a4070 con 0x7fa060108540 2024-04-16T17:19:01.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa064a00640 1 -- 172.21.15.18:0/934110600 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0500054e0 con 0x7fa060108540 2024-04-16T17:19:01.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.491+0000 7fa064a00640 1 -- 172.21.15.18:0/934110600 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050003a10 con 0x7fa060108540 2024-04-16T17:19:01.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.492+0000 7fa05fe00640 1 -- 172.21.15.18:0/934110600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601ab960 con 0x7fa060108540 2024-04-16T17:19:01.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.495+0000 7fa064a00640 1 -- 172.21.15.18:0/934110600 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa050046020 con 0x7fa060108540 2024-04-16T17:19:01.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.495+0000 7fa068a5c740 1 Processor -- start 2024-04-16T17:19:01.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.495+0000 7fa068a5c740 1 -- start start 2024-04-16T17:19:01.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a01270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a09e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d98a0c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa068a5c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561d98965cb0 con 0x561d98a00e90 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa068a5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d9890c370 con 0x561d98a0a390 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa068a5c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561d98961c70 con 0x561d98a017b0 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa05e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d98a0c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa05f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a01270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa05e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d98a0c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54484/0 (socket says 172.21.15.18:54484) 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa05e000640 1 -- 172.21.15.18:0/896984760 learned_addr learned my addr 172.21.15.18:0/896984760 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.496+0000 7fa05f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a01270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49496/0 (socket says 172.21.15.18:49496) 2024-04-16T17:19:01.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.497+0000 7fa05ea00640 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a09e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.497+0000 7fa05f400640 1 -- 172.21.15.18:0/896984760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 msgr2=0x561d98a09e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:01.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.497+0000 7fa05f400640 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a09e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.497+0000 7fa05f400640 1 -- 172.21.15.18:0/896984760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 msgr2=0x561d98a0c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:01.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.497+0000 7fa05f400640 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d98a0c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.497+0000 7fa05f400640 1 -- 172.21.15.18:0/896984760 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9892b910 con 0x561d98a00e90 2024-04-16T17:19:01.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.497+0000 7fa05e000640 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d98a0c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:01.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.497+0000 7fa05f400640 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a01270 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa058009b90 tx=0x7fa058011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9739bb3042652d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:01.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.498+0000 7fa04fe00640 1 -- 172.21.15.18:0/896984760 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05801e070 con 0x561d98a00e90 2024-04-16T17:19:01.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.498+0000 7fa04fe00640 1 -- 172.21.15.18:0/896984760 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0580035f0 con 0x561d98a00e90 2024-04-16T17:19:01.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.498+0000 7fa04fe00640 1 -- 172.21.15.18:0/896984760 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058019630 con 0x561d98a00e90 2024-04-16T17:19:01.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.499+0000 7fa068a5c740 1 -- 172.21.15.18:0/896984760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 msgr2=0x561d98a01270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.499+0000 7fa068a5c740 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a01270 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa058009b90 tx=0x7fa058011810 comp rx=0 tx=0).stop 2024-04-16T17:19:01.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.499+0000 7fa068a5c740 1 -- 172.21.15.18:0/896984760 shutdown_connections 2024-04-16T17:19:01.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.499+0000 7fa068a5c740 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d98a0c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.499+0000 7fa068a5c740 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a09e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.499+0000 7fa068a5c740 1 --2- 172.21.15.18:0/896984760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a01270 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.499+0000 7fa068a5c740 1 -- 172.21.15.18:0/896984760 >> 172.21.15.18:0/896984760 conn(0x561d989fb5c0 msgr2=0x561d989fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:01.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.500+0000 7fa068a5c740 1 -- 172.21.15.18:0/896984760 shutdown_connections 2024-04-16T17:19:01.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.500+0000 7fa068a5c740 1 -- 172.21.15.18:0/896984760 wait complete. 2024-04-16T17:19:01.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.500+0000 7fa068a5c740 1 -- start start 2024-04-16T17:19:01.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.501+0000 7fa068a5c740 1 Processor -- start 2024-04-16T17:19:01.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.501+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a07400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.501+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.501+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a0a390 0x561d98a07e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.501+0000 7fa068a5c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561d98a0d180 con 0x561d98a0a390 2024-04-16T17:19:01.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.501+0000 7fa068a5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d9891f730 con 0x561d98a00e90 2024-04-16T17:19:01.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.501+0000 7fa068a5c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561d98a0d300 con 0x561d98a017b0 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.502+0000 7fa05f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a07400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.502+0000 7fa05f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a07400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54500/0 (socket says 172.21.15.18:54500) 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.502+0000 7fa05f400640 1 -- 172.21.15.18:0/2983744300 learned_addr learned my addr 172.21.15.18:0/2983744300 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.501+0000 7fa068a5c740 10 client.0 _ll_register_callbacks cb 0x561d988ee1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.502+0000 7fa05e000640 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a0a390 0x561d98a07e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.502+0000 7fa05f400640 1 -- 172.21.15.18:0/2983744300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 msgr2=0x561d98a07940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.502+0000 7fa05f400640 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.502+0000 7fa05f400640 1 -- 172.21.15.18:0/2983744300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a0a390 msgr2=0x561d98a07e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.502+0000 7fa05f400640 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a0a390 0x561d98a07e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.503+0000 7fa05f400640 1 -- 172.21.15.18:0/2983744300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d987407e0 con 0x561d98a00e90 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.503+0000 7fa05f400640 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a07400 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa058009f70 tx=0x7fa058003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab37013ca38b0dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.503+0000 7fa04cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.503+0000 7fa064a00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0440a3c30 0x7fa0440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.503+0000 7fa04cc00640 1 -- 172.21.15.18:0/2983744300 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05801e050 con 0x561d98a00e90 2024-04-16T17:19:01.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.504+0000 7fa04cc00640 1 -- 172.21.15.18:0/2983744300 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fa058005040 con 0x561d98a00e90 2024-04-16T17:19:01.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.504+0000 7fa068a5c740 1 -- 172.21.15.18:0/2983744300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0580035f0 con 0x561d98a00e90 2024-04-16T17:19:01.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.504+0000 7fa064a00640 1 -- 172.21.15.18:0/934110600 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fa0500f2ab0 con 0x7fa060108540 2024-04-16T17:19:01.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.504+0000 7fa05ea00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0440a3c30 0x7fa0440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.504+0000 7fa04cc00640 1 -- 172.21.15.18:0/2983744300 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa058033050 con 0x561d98a00e90 2024-04-16T17:19:01.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.505+0000 7fa05ea00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0440a3c30 0x7fa0440a60f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x561d98a00960 tx=0x7fa05002ce30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:01.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.506+0000 7fa068a5c740 10 client.29486 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:01.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.506+0000 7fa068a5c740 1 -- 172.21.15.18:0/2983744300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561d987a0f70 con 0x561d98a00e90 2024-04-16T17:19:01.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.506+0000 7fa04cc00640 1 -- 172.21.15.18:0/2983744300 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0580035f0 con 0x561d98a00e90 2024-04-16T17:19:01.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.506+0000 7fa04cc00640 1 -- 172.21.15.18:0/2983744300 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa058025450 con 0x561d98a00e90 2024-04-16T17:19:01.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.507+0000 7fa068a5c740 10 client.29486 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:01.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.507+0000 7fa068a5c740 10 client.29486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:01.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.507+0000 7fa068a5c740 20 client.29486 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:01.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.507+0000 7fa068a5c740 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d98a1a690 0x561d98a1ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.507+0000 7fa068a5c740 4 client.29486 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:01.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.507+0000 7fa068a5c740 1 -- 172.21.15.18:0/2983744300 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d989736d0 con 0x561d98a1a690 2024-04-16T17:19:01.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.507+0000 7fa05e000640 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d98a1a690 0x561d98a1ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.509+0000 7fa05e000640 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d98a1a690 0x561d98a1ca80 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad8899351000564 server_cookie=4392dc3caf27eae6 in_seq=0 out_seq=0 2024-04-16T17:19:01.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.509+0000 7fa04cc00640 10 client.29486 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:01.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.509+0000 7fa04cc00640 1 -- 172.21.15.18:0/2983744300 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d989736d0 con 0x561d98a1a690 2024-04-16T17:19:01.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa04cc00640 10 client.29486 handle_command_reply: tid=0 2024-04-16T17:19:01.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 client.29486 shutdown 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 -- 172.21.15.18:0/2983744300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 msgr2=0x561d98a07400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a07400 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa058009f70 tx=0x7fa058003c40 comp rx=0 tx=0).stop 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 -- 172.21.15.18:0/2983744300 shutdown_connections 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d98a1a690 0x561d98a1ca80 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a0a390 0x561d98a07e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 --2- 172.21.15.18:0/2983744300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a07400 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa04cc00640 0 client.29486 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.510+0000 7fa068a5c740 1 -- 172.21.15.18:0/2983744300 >> 172.21.15.18:0/2983744300 conn(0x561d989fb5c0 msgr2=0x561d989fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:01.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.511+0000 7fa068a5c740 1 -- 172.21.15.18:0/2983744300 shutdown_connections 2024-04-16T17:19:01.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.511+0000 7fa068a5c740 1 -- 172.21.15.18:0/2983744300 wait complete. 2024-04-16T17:19:01.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.511+0000 7fa068a5c740 20 client.29486 trim_cache size 0 max 16384 2024-04-16T17:19:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.550+0000 7fa068a5c740 1 Processor -- start 2024-04-16T17:19:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.550+0000 7fa068a5c740 1 -- start start 2024-04-16T17:19:01.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a09b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d989fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa068a5c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b8410 con 0x561d98a00e90 2024-04-16T17:19:01.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa068a5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b5c80 con 0x561d98a0a390 2024-04-16T17:19:01.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa068a5c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b3af0 con 0x561d98a017b0 2024-04-16T17:19:01.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54902/0 (socket says 172.21.15.18:54902) 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 -- 172.21.15.18:0/127579275 learned_addr learned my addr 172.21.15.18:0/127579275 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05f400640 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d989fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05f400640 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a09b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 -- 172.21.15.18:0/127579275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 msgr2=0x561d989fd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d989fd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 -- 172.21.15.18:0/127579275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 msgr2=0x561d98a09b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a09b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 -- 172.21.15.18:0/127579275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d98a015e0 con 0x561d98a017b0 2024-04-16T17:19:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.551+0000 7fa05e000640 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07ca0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa04800f980 tx=0x7fa048009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9abf2b638e78df89 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.554+0000 7fa04f400640 1 -- 172.21.15.18:0/127579275 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048003970 con 0x561d98a017b0 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.554+0000 7fa04f400640 1 -- 172.21.15.18:0/127579275 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa048004030 con 0x561d98a017b0 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.554+0000 7fa068a5c740 1 -- 172.21.15.18:0/127579275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 msgr2=0x561d98a07ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.554+0000 7fa068a5c740 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07ca0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa04800f980 tx=0x7fa048009500 comp rx=0 tx=0).stop 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 -- 172.21.15.18:0/127579275 shutdown_connections 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a0a390 0x561d989fd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a017b0 0x561d98a07ca0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 --2- 172.21.15.18:0/127579275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a00e90 0x561d98a09b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 -- 172.21.15.18:0/127579275 >> 172.21.15.18:0/127579275 conn(0x561d989fb5c0 msgr2=0x561d989fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 -- 172.21.15.18:0/127579275 shutdown_connections 2024-04-16T17:19:01.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 -- 172.21.15.18:0/127579275 wait complete. 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 -- start start 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 Processor -- start 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a0e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a017b0 0x561d98a0e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a0a390 0x561d98a7fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b4140 con 0x561d98a017b0 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b55b0 con 0x561d98a00e90 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b4810 con 0x561d98a0a390 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa068a5c740 10 client.0 _ll_register_callbacks cb 0x7fa058009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a0e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a0e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54508/0 (socket says 172.21.15.18:54508) 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 -- 172.21.15.18:0/1365220915 learned_addr learned my addr 172.21.15.18:0/1365220915 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05e000640 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a017b0 0x561d98a0e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a0a390 0x561d98a7fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 -- 172.21.15.18:0/1365220915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a0a390 msgr2=0x561d98a7fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:01.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a0a390 0x561d98a7fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 -- 172.21.15.18:0/1365220915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a017b0 msgr2=0x561d98a0e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:01.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a017b0 0x561d98a0e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.555+0000 7fa05f400640 1 -- 172.21.15.18:0/1365220915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d98638170 con 0x561d98a00e90 2024-04-16T17:19:01.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.570+0000 7fa05f400640 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a0e2f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa058003cd0 tx=0x7fa058003ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b52578908dcbc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:01.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.570+0000 7fa04cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:01.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.570+0000 7fa04cc00640 1 -- 172.21.15.18:0/1365220915 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05801e040 con 0x561d98a00e90 2024-04-16T17:19:01.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.570+0000 7fa04cc00640 1 -- 172.21.15.18:0/1365220915 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7fa058025450 con 0x561d98a00e90 2024-04-16T17:19:01.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.571+0000 7fa068a5c740 1 -- 172.21.15.18:0/1365220915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa048004030 con 0x561d98a00e90 2024-04-16T17:19:01.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.571+0000 7fa04cc00640 1 -- 172.21.15.18:0/1365220915 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa058025670 con 0x561d98a00e90 2024-04-16T17:19:01.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.572+0000 7fa068a5c740 10 client.29489 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:01.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.572+0000 7fa068a5c740 1 -- 172.21.15.18:0/1365220915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561d98a80ef0 con 0x561d98a00e90 2024-04-16T17:19:01.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.572+0000 7fa04cc00640 1 -- 172.21.15.18:0/1365220915 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa048004030 con 0x561d98a00e90 2024-04-16T17:19:01.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.572+0000 7fa04cc00640 1 -- 172.21.15.18:0/1365220915 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa058019990 con 0x561d98a00e90 2024-04-16T17:19:01.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.573+0000 7fa068a5c740 10 client.29489 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:01.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.573+0000 7fa068a5c740 10 client.29489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:01.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.573+0000 7fa068a5c740 20 client.29489 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:01.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.573+0000 7fa068a5c740 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d98a815f0 0x561d98a83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:01.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.573+0000 7fa068a5c740 4 client.29489 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:19:01.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.573+0000 7fa068a5c740 1 -- 172.21.15.18:0/1365220915 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d9896d540 con 0x561d98a815f0 2024-04-16T17:19:01.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.591+0000 7fa05e000640 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d98a815f0 0x561d98a83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.685+0000 7fa05e000640 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d98a815f0 0x561d98a83a10 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9864066f3c375297 server_cookie=c7334b5a546632d5 in_seq=0 out_seq=0 2024-04-16T17:19:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.685+0000 7fa04cc00640 10 client.29489 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.693+0000 7fa04cc00640 1 -- 172.21.15.18:0/1365220915 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d9896d540 con 0x561d98a815f0 2024-04-16T17:19:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.693+0000 7fa04cc00640 10 client.29489 handle_command_reply: tid=0 2024-04-16T17:19:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.693+0000 7fa068a5c740 1 client.29489 shutdown 2024-04-16T17:19:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa068a5c740 1 -- 172.21.15.18:0/1365220915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 msgr2=0x561d98a0e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa068a5c740 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a0e2f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa058003cd0 tx=0x7fa058003ab0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa068a5c740 1 -- 172.21.15.18:0/1365220915 shutdown_connections 2024-04-16T17:19:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa068a5c740 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561d98a815f0 0x561d98a83a10 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa068a5c740 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d98a0a390 0x561d98a7fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa068a5c740 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d98a017b0 0x561d98a0e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa068a5c740 1 --2- 172.21.15.18:0/1365220915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d98a00e90 0x561d98a0e2f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa04cc00640 0 client.29489 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:01.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.705+0000 7fa068a5c740 1 -- 172.21.15.18:0/1365220915 >> 172.21.15.18:0/1365220915 conn(0x561d989fb5c0 msgr2=0x561d989fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:01.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.712+0000 7fa068a5c740 1 -- 172.21.15.18:0/1365220915 shutdown_connections 2024-04-16T17:19:01.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.713+0000 7fa068a5c740 1 -- 172.21.15.18:0/1365220915 wait complete. 2024-04-16T17:19:01.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.713+0000 7fa068a5c740 20 client.29489 trim_cache size 0 max 16384 2024-04-16T17:19:01.714 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:19:01.714 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:19:01.714 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "d13bc435-6afc-4e53-8a4d-b106ba9d0262", 2024-04-16T17:19:01.714 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:19:01.714 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 -- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0440a3c30 msgr2=0x7fa0440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0440a3c30 0x7fa0440a60f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x561d98a00960 tx=0x7fa05002ce30 comp rx=0 tx=0).stop 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 -- 172.21.15.18:0/934110600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa060108540 msgr2=0x7fa06019edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa060108540 0x7fa06019edf0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa050005940 tx=0x7fa05002cbb0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 -- 172.21.15.18:0/934110600 shutdown_connections 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0440a3c30 0x7fa0440a60f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa06010e4f0 0x7fa0601a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa060108ee0 0x7fa06019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 --2- 172.21.15.18:0/934110600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa060108540 0x7fa06019edf0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:01.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.714+0000 7fa05fe00640 1 -- 172.21.15.18:0/934110600 >> 172.21.15.18:0/934110600 conn(0x7fa060077e10 msgr2=0x7fa060078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:01.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.715+0000 7fa05fe00640 1 -- 172.21.15.18:0/934110600 shutdown_connections 2024-04-16T17:19:01.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:01.715+0000 7fa05fe00640 1 -- 172.21.15.18:0/934110600 wait complete. 2024-04-16T17:19:01.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:19:01.740 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:19:02.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 -- 172.21.15.18:0/3451823624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c810ec00 msgr2=0x7f11c811d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:02.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3451823624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c810ec00 0x7f11c811d7f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009be0 tx=0x7f11c002bf30 comp rx=0 tx=0).stop 2024-04-16T17:19:02.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 -- 172.21.15.18:0/3451823624 shutdown_connections 2024-04-16T17:19:02.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3451823624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c810ec00 0x7f11c811d7f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3451823624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11c810e0e0 0x7f11c810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3451823624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8110580 0x7f11c8110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 -- 172.21.15.18:0/3451823624 >> 172.21.15.18:0/3451823624 conn(0x7f11c806cf10 msgr2=0x7f11c806d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:02.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 -- 172.21.15.18:0/3451823624 shutdown_connections 2024-04-16T17:19:02.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.040+0000 7f11c7e00640 1 -- 172.21.15.18:0/3451823624 wait complete. 2024-04-16T17:19:02.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.041+0000 7f11c7e00640 1 Processor -- start 2024-04-16T17:19:02.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.042+0000 7f11c7e00640 1 -- start start 2024-04-16T17:19:02.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.042+0000 7f11c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11c810e0e0 0x7f11c81a7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.042+0000 7f11c7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c810ec00 0x7f11c81a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.042+0000 7f11c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8110580 0x7f11c81abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.042+0000 7f11c7e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8121030 con 0x7f11c810ec00 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.042+0000 7f11c7e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8120eb0 con 0x7f11c810e0e0 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.042+0000 7f11c7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81211b0 con 0x7f11c8110580 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8110580 0x7f11c81abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8110580 0x7f11c81abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54930/0 (socket says 172.21.15.18:54930) 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11c7400640 1 -- 172.21.15.18:0/3423605689 learned_addr learned my addr 172.21.15.18:0/3423605689 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11bfe00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c810ec00 0x7f11c81a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11c7400640 1 -- 172.21.15.18:0/3423605689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11c810e0e0 msgr2=0x7f11c81a7510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11c7400640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11c810e0e0 0x7f11c81a7510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11c7400640 1 -- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c810ec00 msgr2=0x7f11c81a7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11c7400640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c810ec00 0x7f11c81a7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.043+0000 7f11c7400640 1 -- 172.21.15.18:0/3423605689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c81ac500 con 0x7f11c8110580 2024-04-16T17:19:02.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.044+0000 7f11c7400640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8110580 0x7f11c81abdd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f11c0006010 tx=0x7f11c0033ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:02.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.044+0000 7f11bf400640 1 -- 172.21.15.18:0/3423605689 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0043020 con 0x7f11c8110580 2024-04-16T17:19:02.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.044+0000 7f11bf400640 1 -- 172.21.15.18:0/3423605689 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c0003cc0 con 0x7f11c8110580 2024-04-16T17:19:02.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.044+0000 7f11bf400640 1 -- 172.21.15.18:0/3423605689 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c003e980 con 0x7f11c8110580 2024-04-16T17:19:02.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.044+0000 7f11c7e00640 1 -- 172.21.15.18:0/3423605689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c81ac790 con 0x7f11c8110580 2024-04-16T17:19:02.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.045+0000 7f11c7e00640 1 -- 172.21.15.18:0/3423605689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c81b4080 con 0x7f11c8110580 2024-04-16T17:19:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.049+0000 7f11d0586740 1 Processor -- start 2024-04-16T17:19:02.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.051+0000 7f11bf400640 1 -- 172.21.15.18:0/3423605689 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11c0046020 con 0x7f11c8110580 2024-04-16T17:19:02.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.051+0000 7f11d0586740 1 -- start start 2024-04-16T17:19:02.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.051+0000 7f11d0586740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5390 0x55aab49e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.051+0000 7f11d0586740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.051+0000 7f11d0586740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49ee890 0x55aab49f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.051+0000 7f11d0586740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55aab494b120 con 0x55aab49e5390 2024-04-16T17:19:02.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.051+0000 7f11d0586740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aab48f1340 con 0x55aab49e5cb0 2024-04-16T17:19:02.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.051+0000 7f11d0586740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aab49470e0 con 0x55aab49ee890 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.052+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49ee890 0x55aab49f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.052+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49ee890 0x55aab49f0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54934/0 (socket says 172.21.15.18:54934) 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.052+0000 7f11bfe00640 1 -- 172.21.15.18:0/1393242057 learned_addr learned my addr 172.21.15.18:0/1393242057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.053+0000 7f11bfe00640 1 --2- 172.21.15.18:0/1393242057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5390 0x55aab49e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.053+0000 7f11bfe00640 1 -- 172.21.15.18:0/1393242057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 msgr2=0x55aab49ee350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.053+0000 7f11bfe00640 1 --2- 172.21.15.18:0/1393242057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ee350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.053+0000 7f11bfe00640 1 -- 172.21.15.18:0/1393242057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5390 msgr2=0x55aab49e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.053+0000 7f11bfe00640 1 --2- 172.21.15.18:0/1393242057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5390 0x55aab49e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.053+0000 7f11bfe00640 1 -- 172.21.15.18:0/1393242057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aab47277d0 con 0x55aab49ee890 2024-04-16T17:19:02.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.053+0000 7f11bfe00640 1 --2- 172.21.15.18:0/1393242057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49ee890 0x55aab49f0c80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f11b800aaa0 tx=0x7f11b800a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba6819cd278923d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:02.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.054+0000 7f11b7e00640 1 -- 172.21.15.18:0/1393242057 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b800fae0 con 0x55aab49ee890 2024-04-16T17:19:02.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.054+0000 7f11b7e00640 1 -- 172.21.15.18:0/1393242057 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b8019070 con 0x55aab49ee890 2024-04-16T17:19:02.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.054+0000 7f11b7e00640 1 -- 172.21.15.18:0/1393242057 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b80135d0 con 0x55aab49ee890 2024-04-16T17:19:02.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.054+0000 7f11bf400640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11ac0a3e60 0x7f11ac0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.054+0000 7f11bf400640 1 -- 172.21.15.18:0/3423605689 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f11c00f3580 con 0x7f11c8110580 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.055+0000 7f11d0586740 1 -- 172.21.15.18:0/1393242057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49ee890 msgr2=0x55aab49f0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.055+0000 7f11d0586740 1 --2- 172.21.15.18:0/1393242057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49ee890 0x55aab49f0c80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f11b800aaa0 tx=0x7f11b800a490 comp rx=0 tx=0).stop 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.055+0000 7f11d0586740 1 -- 172.21.15.18:0/1393242057 shutdown_connections 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.055+0000 7f11d0586740 1 --2- 172.21.15.18:0/1393242057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49ee890 0x55aab49f0c80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.055+0000 7f11d0586740 1 --2- 172.21.15.18:0/1393242057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.055+0000 7f11d0586740 1 --2- 172.21.15.18:0/1393242057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5390 0x55aab49e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.055+0000 7f11d0586740 1 -- 172.21.15.18:0/1393242057 >> 172.21.15.18:0/1393242057 conn(0x55aab49dfac0 msgr2=0x55aab49dfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.056+0000 7f11d0586740 1 -- 172.21.15.18:0/1393242057 shutdown_connections 2024-04-16T17:19:02.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.057+0000 7f11d0586740 1 -- 172.21.15.18:0/1393242057 wait complete. 2024-04-16T17:19:02.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.058+0000 7f11d0586740 1 -- start start 2024-04-16T17:19:02.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.058+0000 7f11d0586740 1 Processor -- start 2024-04-16T17:19:02.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11c6a00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11ac0a3e60 0x7f11ac0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11d0586740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11d0586740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11d0586740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 0x55aab49ec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11d0586740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55aab49f1680 con 0x55aab49ee890 2024-04-16T17:19:02.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ebf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54526/0 (socket says 172.21.15.18:54526) 2024-04-16T17:19:02.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11bfe00640 1 -- 172.21.15.18:0/1855308205 learned_addr learned my addr 172.21.15.18:0/1855308205 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:02.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11c6a00640 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.059+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aab4904710 con 0x55aab49e5cb0 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.060+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aab49f1800 con 0x55aab49e5390 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.060+0000 7f11d0586740 10 client.0 _ll_register_callbacks cb 0x55aab48ffaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.060+0000 7f11c7400640 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 0x55aab49ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.060+0000 7f11bfe00640 1 -- 172.21.15.18:0/1855308205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 msgr2=0x55aab49eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.060+0000 7f11bfe00640 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.060+0000 7f11bfe00640 1 -- 172.21.15.18:0/1855308205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 msgr2=0x55aab49ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.060+0000 7f11bfe00640 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 0x55aab49ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.060+0000 7f11bfe00640 1 -- 172.21.15.18:0/1855308205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aab465d3f0 con 0x55aab49e5cb0 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.061+0000 7f11bfe00640 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ebf20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f11b8014df0 tx=0x7f11b8014640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57195f1ba9023209 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:02.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.061+0000 7f11b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:02.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.061+0000 7f11c6a00640 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:02.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.061+0000 7f11b4c00640 1 -- 172.21.15.18:0/1855308205 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8025600 con 0x55aab49e5cb0 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.061+0000 7f11b4c00640 1 -- 172.21.15.18:0/1855308205 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f11b801d3f0 con 0x55aab49e5cb0 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.061+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b8019070 con 0x55aab49e5cb0 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11b4c00640 1 -- 172.21.15.18:0/1855308205 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f11b8014a70 con 0x55aab49e5cb0 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11d0586740 10 client.29492 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55aab461d0f0 con 0x55aab49e5cb0 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11b4c00640 1 -- 172.21.15.18:0/1855308205 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b8019070 con 0x55aab49e5cb0 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11b4c00640 1 -- 172.21.15.18:0/1855308205 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f11b801dae0 con 0x55aab49e5cb0 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11d0586740 10 client.29492 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11d0586740 10 client.29492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11d0586740 20 client.29492 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11d0586740 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aab4a0fd10 0x55aab4a12100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11d0586740 4 client.29492 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:02.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.062+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aab4958b40 con 0x55aab4a0fd10 2024-04-16T17:19:02.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.065+0000 7f11c6a00640 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aab4a0fd10 0x55aab4a12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.066+0000 7f11c6a00640 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aab4a0fd10 0x55aab4a12100 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc71acd5e450ea78 server_cookie=e88c085c60c580e9 in_seq=0 out_seq=0 2024-04-16T17:19:02.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.066+0000 7f11b4c00640 10 client.29492 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:02.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.066+0000 7f11c6a00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11ac0a3e60 0x7f11ac0a6320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f11b00058c0 tx=0x7f11b000b550 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:02.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.072+0000 7f11b4c00640 1 -- 172.21.15.18:0/1855308205 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aab4958b40 con 0x55aab4a0fd10 2024-04-16T17:19:02.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.072+0000 7f11b4c00640 10 client.29492 handle_command_reply: tid=0 2024-04-16T17:19:02.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.072+0000 7f11d0586740 1 client.29492 shutdown 2024-04-16T17:19:02.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 msgr2=0x55aab49ebf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:02.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11d0586740 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ebf20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f11b8014df0 tx=0x7f11b8014640 comp rx=0 tx=0).stop 2024-04-16T17:19:02.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 shutdown_connections 2024-04-16T17:19:02.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11d0586740 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aab4a0fd10 0x55aab4a12100 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11d0586740 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 0x55aab49ec460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11d0586740 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49ebf20 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11d0586740 1 --2- 172.21.15.18:0/1855308205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11b4c00640 0 client.29492 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:02.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.073+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 >> 172.21.15.18:0/1855308205 conn(0x55aab49dfac0 msgr2=0x55aab49e1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:02.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.074+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 shutdown_connections 2024-04-16T17:19:02.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.074+0000 7f11d0586740 1 -- 172.21.15.18:0/1855308205 wait complete. 2024-04-16T17:19:02.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.074+0000 7f11d0586740 20 client.29492 trim_cache size 0 max 16384 2024-04-16T17:19:02.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.113+0000 7f11d0586740 1 Processor -- start 2024-04-16T17:19:02.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.113+0000 7f11d0586740 1 -- start start 2024-04-16T17:19:02.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.113+0000 7f11d0586740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.113+0000 7f11d0586740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.113+0000 7f11d0586740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 0x55aab49dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.113+0000 7f11d0586740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b83a0 con 0x55aab49ee890 2024-04-16T17:19:02.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.113+0000 7f11d0586740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b5c10 con 0x55aab49e5cb0 2024-04-16T17:19:02.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.113+0000 7f11d0586740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81a8860 con 0x55aab49e5390 2024-04-16T17:19:02.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.114+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.114+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49fa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54530/0 (socket says 172.21.15.18:54530) 2024-04-16T17:19:02.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.114+0000 7f11bfe00640 1 -- 172.21.15.18:0/3695554149 learned_addr learned my addr 172.21.15.18:0/3695554149 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:02.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.114+0000 7f11bfe00640 1 -- 172.21.15.18:0/3695554149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 msgr2=0x55aab49ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:02.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.114+0000 7f11bfe00640 1 --2- 172.21.15.18:0/3695554149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.114+0000 7f11bfe00640 1 -- 172.21.15.18:0/3695554149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 msgr2=0x55aab49dfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:02.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.114+0000 7f11bfe00640 1 --2- 172.21.15.18:0/3695554149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 0x55aab49dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.114+0000 7f11bfe00640 1 -- 172.21.15.18:0/3695554149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aab49e5ad0 con 0x55aab49e5cb0 2024-04-16T17:19:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.115+0000 7f11bfe00640 1 --2- 172.21.15.18:0/3695554149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49fa540 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f11b800a9f0 tx=0x7f11b800fca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47b53d7c7908ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.115+0000 7f11b7400640 1 -- 172.21.15.18:0/3695554149 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b80146b0 con 0x55aab49e5cb0 2024-04-16T17:19:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.115+0000 7f11b7400640 1 -- 172.21.15.18:0/3695554149 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b8019070 con 0x55aab49e5cb0 2024-04-16T17:19:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.115+0000 7f11b7400640 1 -- 172.21.15.18:0/3695554149 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8024890 con 0x55aab49e5cb0 2024-04-16T17:19:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 -- 172.21.15.18:0/3695554149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 msgr2=0x55aab49fa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:02.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 --2- 172.21.15.18:0/3695554149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49fa540 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f11b800a9f0 tx=0x7f11b800fca0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 -- 172.21.15.18:0/3695554149 shutdown_connections 2024-04-16T17:19:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 --2- 172.21.15.18:0/3695554149 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49ee890 0x55aab49dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 --2- 172.21.15.18:0/3695554149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49e5cb0 0x55aab49fa540 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 --2- 172.21.15.18:0/3695554149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 -- 172.21.15.18:0/3695554149 >> 172.21.15.18:0/3695554149 conn(0x55aab49dfac0 msgr2=0x55aab49e0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 -- 172.21.15.18:0/3695554149 shutdown_connections 2024-04-16T17:19:02.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.116+0000 7f11d0586740 1 -- 172.21.15.18:0/3695554149 wait complete. 2024-04-16T17:19:02.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.117+0000 7f11d0586740 1 -- start start 2024-04-16T17:19:02.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.117+0000 7f11d0586740 1 Processor -- start 2024-04-16T17:19:02.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11d0586740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11d0586740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5cb0 0x55aab49f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11d0586740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49ee890 0x55aab49f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11d0586740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b4250 con 0x55aab49e5cb0 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11d0586740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b5540 con 0x55aab49ee890 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11d0586740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b47a0 con 0x55aab49e5390 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11d0586740 10 client.0 _ll_register_callbacks cb 0x7f11b800a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49ee890 0x55aab49f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.118+0000 7f11c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49ee890 0x55aab49f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54544/0 (socket says 172.21.15.18:54544) 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.119+0000 7f11c7400640 1 -- 172.21.15.18:0/3053540379 learned_addr learned my addr 172.21.15.18:0/3053540379 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:02.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.119+0000 7f11c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.119+0000 7f11bfe00640 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5cb0 0x55aab49f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.119+0000 7f11c6a00640 1 -- 172.21.15.18:0/3053540379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49ee890 msgr2=0x55aab49f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:02.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.119+0000 7f11c6a00640 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49ee890 0x55aab49f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.119+0000 7f11c6a00640 1 -- 172.21.15.18:0/3053540379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5cb0 msgr2=0x55aab49f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:02.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.119+0000 7f11c6a00640 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5cb0 0x55aab49f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.119+0000 7f11c6a00640 1 -- 172.21.15.18:0/3053540379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aab4a75320 con 0x55aab49e5390 2024-04-16T17:19:02.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.120+0000 7f11b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:19:02.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.120+0000 7f11c6a00640 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49f2630 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55aab49f18d0 tx=0x7f11b0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f76834fed7419ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:02.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.120+0000 7f11b4c00640 1 -- 172.21.15.18:0/3053540379 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b002b070 con 0x55aab49e5390 2024-04-16T17:19:02.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.120+0000 7f11d0586740 1 -- 172.21.15.18:0/3053540379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b8019070 con 0x55aab49e5390 2024-04-16T17:19:02.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.120+0000 7f11b4c00640 1 -- 172.21.15.18:0/3053540379 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9538+0+0 (secure 0 0 0) 0x7f11b000f450 con 0x55aab49e5390 2024-04-16T17:19:02.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.121+0000 7f11b4c00640 1 -- 172.21.15.18:0/3053540379 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f11b0035050 con 0x55aab49e5390 2024-04-16T17:19:02.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.121+0000 7f11d0586740 10 client.29335 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:02.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.121+0000 7f11d0586740 1 -- 172.21.15.18:0/3053540379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55aab4a766a0 con 0x55aab49e5390 2024-04-16T17:19:02.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.121+0000 7f11b4c00640 1 -- 172.21.15.18:0/3053540379 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b8019070 con 0x55aab49e5390 2024-04-16T17:19:02.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.121+0000 7f11b4c00640 1 -- 172.21.15.18:0/3053540379 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f11b0030740 con 0x55aab49e5390 2024-04-16T17:19:02.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.122+0000 7f11d0586740 10 client.29335 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:02.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.122+0000 7f11d0586740 10 client.29335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:02.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.122+0000 7f11d0586740 20 client.29335 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:02.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.122+0000 7f11d0586740 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aab4a76e00 0x55aab4a79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:02.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.122+0000 7f11d0586740 4 client.29335 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:19:02.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.122+0000 7f11d0586740 1 -- 172.21.15.18:0/3053540379 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aab49529b0 con 0x55aab4a76e00 2024-04-16T17:19:02.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.122+0000 7f11bfe00640 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aab4a76e00 0x55aab4a79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:02.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.123+0000 7f11b4c00640 10 client.29335 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:02.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.123+0000 7f11bfe00640 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aab4a76e00 0x55aab4a79220 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1385606ae4e4f59b server_cookie=a219ab91fe3a01ab in_seq=0 out_seq=0 2024-04-16T17:19:02.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.124+0000 7f11b4c00640 1 -- 172.21.15.18:0/3053540379 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55aab49529b0 con 0x55aab4a76e00 2024-04-16T17:19:02.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.124+0000 7f11b4c00640 10 client.29335 handle_command_reply: tid=0 2024-04-16T17:19:02.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.124+0000 7f11d0586740 1 client.29335 shutdown 2024-04-16T17:19:02.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11d0586740 1 -- 172.21.15.18:0/3053540379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 msgr2=0x55aab49f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:02.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11d0586740 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49f2630 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55aab49f18d0 tx=0x7f11b0004730 comp rx=0 tx=0).stop 2024-04-16T17:19:02.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11d0586740 1 -- 172.21.15.18:0/3053540379 shutdown_connections 2024-04-16T17:19:02.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11d0586740 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aab4a76e00 0x55aab4a79220 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11d0586740 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aab49ee890 0x55aab49f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11d0586740 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aab49e5cb0 0x55aab49f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11d0586740 1 --2- 172.21.15.18:0/3053540379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aab49e5390 0x55aab49f2630 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11d0586740 1 -- 172.21.15.18:0/3053540379 >> 172.21.15.18:0/3053540379 conn(0x55aab49dfac0 msgr2=0x55aab49e1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:02.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.126+0000 7f11b4c00640 0 client.29335 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:02.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.127+0000 7f11d0586740 1 -- 172.21.15.18:0/3053540379 shutdown_connections 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (273 inodes in the stack)", 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout: "d13bc435-6afc-4e53-8a4d-b106ba9d0262": { 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "d13bc435-6afc-4e53-8a4d-b106ba9d0262", 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:19:02.128 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:19:02.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.127+0000 7f11d0586740 1 -- 172.21.15.18:0/3053540379 wait complete. 2024-04-16T17:19:02.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.127+0000 7f11d0586740 20 client.29335 trim_cache size 0 max 16384 2024-04-16T17:19:02.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 -- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11ac0a3e60 msgr2=0x7f11ac0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:02.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11ac0a3e60 0x7f11ac0a6320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f11b00058c0 tx=0x7f11b000b550 comp rx=0 tx=0).stop 2024-04-16T17:19:02.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 -- 172.21.15.18:0/3423605689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8110580 msgr2=0x7f11c81abdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:02.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8110580 0x7f11c81abdd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f11c0006010 tx=0x7f11c0033ac0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 -- 172.21.15.18:0/3423605689 shutdown_connections 2024-04-16T17:19:02.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11ac0a3e60 0x7f11ac0a6320 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8110580 0x7f11c81abdd0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c810ec00 0x7f11c81a7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 --2- 172.21.15.18:0/3423605689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11c810e0e0 0x7f11c81a7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:02.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 -- 172.21.15.18:0/3423605689 >> 172.21.15.18:0/3423605689 conn(0x7f11c806cf10 msgr2=0x7f11c80713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:02.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 -- 172.21.15.18:0/3423605689 shutdown_connections 2024-04-16T17:19:02.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:02.128+0000 7f11c7e00640 1 -- 172.21.15.18:0/3423605689 wait complete. 2024-04-16T17:19:02.157 INFO:tasks.cephfs.filesystem:scrub status for tag:d13bc435-6afc-4e53-8a4d-b106ba9d0262 - {'path': '/', 'tag': 'd13bc435-6afc-4e53-8a4d-b106ba9d0262', 'options': 'recursive,force'} 2024-04-16T17:19:02.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:01 smithi022 ceph-mon[36023]: pgmap v453: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 610 op/s 2024-04-16T17:19:02.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:01 smithi043 ceph-mon[36901]: pgmap v453: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 610 op/s 2024-04-16T17:19:02.339 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21323 17.88 MB/sec warmup 74 sec latency 1391.200 ms 2024-04-16T17:19:03.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:19:03.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:03.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:19:03.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:19:03.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:03.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:19:03.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:03.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:19:03.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:02 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:19:03.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:02 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:03.340 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21464 17.69 MB/sec warmup 75 sec latency 641.405 ms 2024-04-16T17:19:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:03 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:03 smithi022 ceph-mon[36023]: pgmap v454: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 570 op/s 2024-04-16T17:19:04.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:03 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:04.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:03 smithi043 ceph-mon[36901]: pgmap v454: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 570 op/s 2024-04-16T17:19:04.340 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21563 17.48 MB/sec warmup 76 sec latency 822.805 ms 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:01 smithi018 ceph-mon[31776]: pgmap v453: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.5 MiB/s wr, 610 op/s 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:04.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:19:04.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:02 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:04.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:03 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:04.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:03 smithi018 ceph-mon[31776]: pgmap v454: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 570 op/s 2024-04-16T17:19:05.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:04 smithi043 ceph-mon[36901]: osdmap e117: 12 total, 12 up, 12 in 2024-04-16T17:19:05.340 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21615 17.28 MB/sec warmup 77 sec latency 1332.231 ms 2024-04-16T17:19:05.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:04 smithi018 ceph-mon[31776]: osdmap e117: 12 total, 12 up, 12 in 2024-04-16T17:19:05.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:04 smithi022 ceph-mon[36023]: osdmap e117: 12 total, 12 up, 12 in 2024-04-16T17:19:06.136 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:05 smithi018 ceph-mon[31776]: pgmap v456: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 654 op/s 2024-04-16T17:19:06.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:05 smithi043 ceph-mon[36901]: pgmap v456: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 654 op/s 2024-04-16T17:19:06.340 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21676 17.08 MB/sec warmup 78 sec latency 1122.498 ms 2024-04-16T17:19:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:05 smithi022 ceph-mon[36023]: pgmap v456: 129 pgs: 129 active+clean; 301 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 654 op/s 2024-04-16T17:19:07.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21684 16.88 MB/sec warmup 79 sec latency 1365.919 ms 2024-04-16T17:19:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:07 smithi043 ceph-mon[36901]: pgmap v457: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 304 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 704 op/s 2024-04-16T17:19:08.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21685 16.68 MB/sec warmup 80 sec latency 1491.693 ms 2024-04-16T17:19:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:07 smithi022 ceph-mon[36023]: pgmap v457: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 304 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 704 op/s 2024-04-16T17:19:08.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:07 smithi018 ceph-mon[31776]: pgmap v457: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 304 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 704 op/s 2024-04-16T17:19:09.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21717 16.53 MB/sec warmup 81 sec latency 1907.750 ms 2024-04-16T17:19:09.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:09 smithi018 ceph-mon[31776]: pgmap v458: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 304 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 690 op/s 2024-04-16T17:19:09.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:09 smithi022 ceph-mon[36023]: pgmap v458: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 304 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 690 op/s 2024-04-16T17:19:10.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:09 smithi043 ceph-mon[36901]: pgmap v458: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 304 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 690 op/s 2024-04-16T17:19:10.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21756 16.35 MB/sec warmup 82 sec latency 1052.614 ms 2024-04-16T17:19:11.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21894 16.32 MB/sec warmup 83 sec latency 730.040 ms 2024-04-16T17:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.368+0000 7fa66ac00640 1 -- 172.21.15.18:0/1736643752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664101030 msgr2=0x7fa664101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.368+0000 7fa66ac00640 1 --2- 172.21.15.18:0/1736643752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664101030 0x7fa664101410 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009ca0 tx=0x7fa65c02c1a0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.368+0000 7fa66ac00640 1 -- 172.21.15.18:0/1736643752 shutdown_connections 2024-04-16T17:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.368+0000 7fa66ac00640 1 --2- 172.21.15.18:0/1736643752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa664106430 0x7fa6641068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.368+0000 7fa66ac00640 1 --2- 172.21.15.18:0/1736643752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa664101950 0x7fa664105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.368+0000 7fa66ac00640 1 --2- 172.21.15.18:0/1736643752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664101030 0x7fa664101410 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.368+0000 7fa66ac00640 1 -- 172.21.15.18:0/1736643752 >> 172.21.15.18:0/1736643752 conn(0x7fa6640fca20 msgr2=0x7fa6640fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.369+0000 7fa66ac00640 1 -- 172.21.15.18:0/1736643752 shutdown_connections 2024-04-16T17:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.369+0000 7fa66ac00640 1 -- 172.21.15.18:0/1736643752 wait complete. 2024-04-16T17:19:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.370+0000 7fa66ac00640 1 Processor -- start 2024-04-16T17:19:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.370+0000 7fa66ac00640 1 -- start start 2024-04-16T17:19:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa66ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa664101030 0x7fa66419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa66ac00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa664101950 0x7fa66419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa66ac00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664106430 0x7fa6641a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa66ac00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa664118ac0 con 0x7fa664101950 2024-04-16T17:19:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa66ac00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa664118940 con 0x7fa664106430 2024-04-16T17:19:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa66ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa664118c40 con 0x7fa664101030 2024-04-16T17:19:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa663e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa664101030 0x7fa66419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa663e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa664101030 0x7fa66419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39098/0 (socket says 172.21.15.18:39098) 2024-04-16T17:19:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.371+0000 7fa663e00640 1 -- 172.21.15.18:0/3175514402 learned_addr learned my addr 172.21.15.18:0/3175514402 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.372+0000 7fa663e00640 1 -- 172.21.15.18:0/3175514402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664106430 msgr2=0x7fa6641a38c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.372+0000 7fa663e00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664106430 0x7fa6641a38c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.372+0000 7fa663e00640 1 -- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa664101950 msgr2=0x7fa66419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.372+0000 7fa663e00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa664101950 0x7fa66419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.372+0000 7fa663e00640 1 -- 172.21.15.18:0/3175514402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6641a3ff0 con 0x7fa664101030 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.372+0000 7fa663e00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa664101030 0x7fa66419f000 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005e30 tx=0x7fa65c003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.373+0000 7fa660c00640 1 -- 172.21.15.18:0/3175514402 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c039040 con 0x7fa664101030 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.373+0000 7fa660c00640 1 -- 172.21.15.18:0/3175514402 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c044070 con 0x7fa664101030 2024-04-16T17:19:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.373+0000 7fa660c00640 1 -- 172.21.15.18:0/3175514402 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0329f0 con 0x7fa664101030 2024-04-16T17:19:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.373+0000 7fa66ac00640 1 -- 172.21.15.18:0/3175514402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6641a4280 con 0x7fa664101030 2024-04-16T17:19:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.373+0000 7fa66ac00640 1 -- 172.21.15.18:0/3175514402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641abb70 con 0x7fa664101030 2024-04-16T17:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.378+0000 7fa66d85d740 1 Processor -- start 2024-04-16T17:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.378+0000 7fa660c00640 1 -- 172.21.15.18:0/3175514402 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa65c032b90 con 0x7fa664101030 2024-04-16T17:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.378+0000 7fa66d85d740 1 -- start start 2024-04-16T17:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364eb610 0x55c4364ea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 0x55c4364ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364f3e20 0x55c4364f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa66d85d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c43644f780 con 0x55c4364f3e20 2024-04-16T17:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa66d85d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c4363f5f40 con 0x55c4364eb610 2024-04-16T17:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa66d85d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c43644b740 con 0x55c4364ea8a0 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa668c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364eb610 0x55c4364ea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa663400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364f3e20 0x55c4364f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa668c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364eb610 0x55c4364ea360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53116/0 (socket says 172.21.15.18:53116) 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.379+0000 7fa668c00640 1 -- 172.21.15.18:0/3606712088 learned_addr learned my addr 172.21.15.18:0/3606712088 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.380+0000 7fa668c00640 1 -- 172.21.15.18:0/3606712088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 msgr2=0x55c4364ead20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.380+0000 7fa668c00640 1 --2- 172.21.15.18:0/3606712088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 0x55c4364ead20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.380+0000 7fa668c00640 1 -- 172.21.15.18:0/3606712088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364f3e20 msgr2=0x55c4364f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.380+0000 7fa668c00640 1 --2- 172.21.15.18:0/3606712088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364f3e20 0x55c4364f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.380+0000 7fa668c00640 1 -- 172.21.15.18:0/3606712088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c436162210 con 0x55c4364eb610 2024-04-16T17:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.381+0000 7fa668c00640 1 --2- 172.21.15.18:0/3606712088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364eb610 0x55c4364ea360 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa65000ba00 tx=0x7fa65000bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125c656f169d7392 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.381+0000 7fa659200640 1 -- 172.21.15.18:0/3606712088 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650016070 con 0x55c4364eb610 2024-04-16T17:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.381+0000 7fa659200640 1 -- 172.21.15.18:0/3606712088 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa650005500 con 0x55c4364eb610 2024-04-16T17:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.381+0000 7fa659200640 1 -- 172.21.15.18:0/3606712088 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650016070 con 0x55c4364eb610 2024-04-16T17:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.382+0000 7fa660c00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa6480a3e10 0x7fa6480a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.382+0000 7fa663e00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa6480a3e10 0x7fa6480a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.382+0000 7fa66d85d740 1 -- 172.21.15.18:0/3606712088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364eb610 msgr2=0x55c4364ea360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.382+0000 7fa66d85d740 1 --2- 172.21.15.18:0/3606712088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364eb610 0x55c4364ea360 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa65000ba00 tx=0x7fa65000bec0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.383+0000 7fa660c00640 1 -- 172.21.15.18:0/3175514402 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9642+0+0 (secure 0 0 0) 0x7fa65c0f2f30 con 0x7fa664101030 2024-04-16T17:19:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.384+0000 7fa66d85d740 1 -- 172.21.15.18:0/3606712088 shutdown_connections 2024-04-16T17:19:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.384+0000 7fa66d85d740 1 --2- 172.21.15.18:0/3606712088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364f3e20 0x55c4364f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.384+0000 7fa66d85d740 1 --2- 172.21.15.18:0/3606712088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 0x55c4364ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.384+0000 7fa66d85d740 1 --2- 172.21.15.18:0/3606712088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364eb610 0x55c4364ea360 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.384+0000 7fa66d85d740 1 -- 172.21.15.18:0/3606712088 >> 172.21.15.18:0/3606712088 conn(0x55c4364855d0 msgr2=0x55c4364e5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.385+0000 7fa663e00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa6480a3e10 0x7fa6480a62d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55c4364ebd70 tx=0x7fa65c0f6590 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.386+0000 7fa66d85d740 1 -- 172.21.15.18:0/3606712088 shutdown_connections 2024-04-16T17:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.386+0000 7fa66d85d740 1 -- 172.21.15.18:0/3606712088 wait complete. 2024-04-16T17:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.387+0000 7fa66d85d740 1 -- start start 2024-04-16T17:19:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.387+0000 7fa66d85d740 1 Processor -- start 2024-04-16T17:19:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.388+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa668c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa668c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364f0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36254/0 (socket says 172.21.15.18:36254) 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa663400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa663400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364f11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39114/0 (socket says 172.21.15.18:39114) 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa668c00640 1 -- 172.21.15.18:0/1684333921 learned_addr learned my addr 172.21.15.18:0/1684333921 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c4364f6b50 con 0x55c4364ea8a0 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c436409390 con 0x55c4364f3e20 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4364f6cd0 con 0x55c4364eb610 2024-04-16T17:19:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.389+0000 7fa66d85d740 10 client.0 _ll_register_callbacks cb 0x55c4363f2f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.390+0000 7fa668c00640 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.390+0000 7fa668c00640 1 -- 172.21.15.18:0/1684333921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 msgr2=0x55c4364f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.390+0000 7fa668c00640 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.390+0000 7fa668c00640 1 -- 172.21.15.18:0/1684333921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 msgr2=0x55c4364f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.390+0000 7fa668c00640 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.390+0000 7fa668c00640 1 -- 172.21.15.18:0/1684333921 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43622e620 con 0x55c4364ea8a0 2024-04-16T17:19:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.390+0000 7fa668c00640 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364f0cb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa650004a90 tx=0x7fa650004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59360acaf12661c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.391+0000 7fa64d600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:19:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.391+0000 7fa64d600640 1 -- 172.21.15.18:0/1684333921 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650016030 con 0x55c4364ea8a0 2024-04-16T17:19:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.391+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa650005500 con 0x55c4364ea8a0 2024-04-16T17:19:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.392+0000 7fa64d600640 1 -- 172.21.15.18:0/1684333921 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9642+0+0 (secure 0 0 0) 0x7fa65000d9e0 con 0x55c4364ea8a0 2024-04-16T17:19:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.392+0000 7fa64d600640 1 -- 172.21.15.18:0/1684333921 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa65000dc00 con 0x55c4364ea8a0 2024-04-16T17:19:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.392+0000 7fa66d85d740 10 client.19074 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.392+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c436502ce0 con 0x55c4364ea8a0 2024-04-16T17:19:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.393+0000 7fa64d600640 1 -- 172.21.15.18:0/1684333921 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa650005500 con 0x55c4364ea8a0 2024-04-16T17:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.393+0000 7fa64d600640 1 -- 172.21.15.18:0/1684333921 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa650003920 con 0x55c4364ea8a0 2024-04-16T17:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.393+0000 7fa66d85d740 10 client.19074 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.393+0000 7fa66d85d740 10 client.19074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.393+0000 7fa66d85d740 20 client.19074 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.393+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c436504140 0x55c436506530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.393+0000 7fa66d85d740 4 client.19074 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.393+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c43645d1a0 con 0x55c436504140 2024-04-16T17:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.394+0000 7fa663400640 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c436504140 0x55c436506530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.396+0000 7fa663400640 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c436504140 0x55c436506530 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f8f035e6e15a6a server_cookie=49c1166da04bd667 in_seq=0 out_seq=0 2024-04-16T17:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.396+0000 7fa64d600640 10 client.19074 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.400+0000 7fa64d600640 1 -- 172.21.15.18:0/1684333921 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c43645d1a0 con 0x55c436504140 2024-04-16T17:19:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.400+0000 7fa64d600640 10 client.19074 handle_command_reply: tid=0 2024-04-16T17:19:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.400+0000 7fa66d85d740 1 client.19074 shutdown 2024-04-16T17:19:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.400+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 msgr2=0x55c4364f0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.400+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364f0cb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa650004a90 tx=0x7fa650004b90 comp rx=0 tx=0).stop 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.401+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 shutdown_connections 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.401+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c436504140 0x55c436506530 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.401+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.401+0000 7fa64d600640 0 client.19074 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.401+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.401+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1684333921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364f0cb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.401+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 >> 172.21.15.18:0/1684333921 conn(0x55c4364855d0 msgr2=0x55c4364e72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.401+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 shutdown_connections 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.402+0000 7fa66d85d740 1 -- 172.21.15.18:0/1684333921 wait complete. 2024-04-16T17:19:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.402+0000 7fa66d85d740 20 client.19074 trim_cache size 0 max 16384 2024-04-16T17:19:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.434+0000 7fa66d85d740 1 Processor -- start 2024-04-16T17:19:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.435+0000 7fa66d85d740 1 -- start start 2024-04-16T17:19:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.435+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 0x55c4364ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.435+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364eb610 0x55c4364f1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.435+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364e4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.435+0000 7fa66d85d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b8710 con 0x55c4364eb610 2024-04-16T17:19:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.435+0000 7fa66d85d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b5f80 con 0x55c4364f3e20 2024-04-16T17:19:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.435+0000 7fa66d85d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b3df0 con 0x55c4364ea8a0 2024-04-16T17:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.436+0000 7fa668c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.436+0000 7fa668c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364e4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53136/0 (socket says 172.21.15.18:53136) 2024-04-16T17:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.436+0000 7fa668c00640 1 -- 172.21.15.18:0/3853471892 learned_addr learned my addr 172.21.15.18:0/3853471892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.436+0000 7fa668c00640 1 --2- 172.21.15.18:0/3853471892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 0x55c4364ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa668c00640 1 -- 172.21.15.18:0/3853471892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 msgr2=0x55c4364ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa668c00640 1 --2- 172.21.15.18:0/3853471892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 0x55c4364ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa668c00640 1 -- 172.21.15.18:0/3853471892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364eb610 msgr2=0x55c4364f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa668c00640 1 --2- 172.21.15.18:0/3853471892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364eb610 0x55c4364f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa668c00640 1 -- 172.21.15.18:0/3853471892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4364e9b70 con 0x55c4364f3e20 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa668c00640 1 --2- 172.21.15.18:0/3853471892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364e4530 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa650004ff0 tx=0x7fa650001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e051436b6d00f628 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa64fe00640 1 -- 172.21.15.18:0/3853471892 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650016040 con 0x55c4364f3e20 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa64fe00640 1 -- 172.21.15.18:0/3853471892 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65000d6b0 con 0x55c4364f3e20 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.437+0000 7fa64fe00640 1 -- 172.21.15.18:0/3853471892 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500041e0 con 0x55c4364f3e20 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 -- 172.21.15.18:0/3853471892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 msgr2=0x55c4364e4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 --2- 172.21.15.18:0/3853471892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364e4530 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa650004ff0 tx=0x7fa650001e70 comp rx=0 tx=0).stop 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 -- 172.21.15.18:0/3853471892 shutdown_connections 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 --2- 172.21.15.18:0/3853471892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364e4530 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 --2- 172.21.15.18:0/3853471892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364eb610 0x55c4364f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 --2- 172.21.15.18:0/3853471892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364ea8a0 0x55c4364ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 -- 172.21.15.18:0/3853471892 >> 172.21.15.18:0/3853471892 conn(0x55c4364855d0 msgr2=0x55c4364e3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 -- 172.21.15.18:0/3853471892 shutdown_connections 2024-04-16T17:19:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.438+0000 7fa66d85d740 1 -- 172.21.15.18:0/3853471892 wait complete. 2024-04-16T17:19:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.439+0000 7fa66d85d740 1 -- start start 2024-04-16T17:19:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.439+0000 7fa66d85d740 1 Processor -- start 2024-04-16T17:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.440+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.440+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364ed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa66d85d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa66d85d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b4440 con 0x55c4364ea8a0 2024-04-16T17:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa66d85d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b58b0 con 0x55c4364f3e20 2024-04-16T17:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa66d85d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b4b10 con 0x55c4364eb610 2024-04-16T17:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa66d85d740 10 client.0 _ll_register_callbacks cb 0x7fa66406b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa663400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa663400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364ed730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39128/0 (socket says 172.21.15.18:39128) 2024-04-16T17:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa663400640 1 -- 172.21.15.18:0/1174202158 learned_addr learned my addr 172.21.15.18:0/1174202158 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa668c00640 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa663400640 1 -- 172.21.15.18:0/1174202158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 msgr2=0x55c4364edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa663400640 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa663400640 1 -- 172.21.15.18:0/1174202158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 msgr2=0x55c4364fc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa663400640 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.441+0000 7fa663400640 1 -- 172.21.15.18:0/1174202158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4364e7890 con 0x55c4364eb610 2024-04-16T17:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.442+0000 7fa663400640 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364ed730 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55c4364eb340 tx=0x7fa654005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfec7bd9bdbfd132 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.442+0000 7fa64d600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.443+0000 7fa64d600640 1 -- 172.21.15.18:0/1174202158 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654006490 con 0x55c4364eb610 2024-04-16T17:19:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.443+0000 7fa64d600640 1 -- 172.21.15.18:0/1174202158 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9642+0+0 (secure 0 0 0) 0x7fa65400f040 con 0x55c4364eb610 2024-04-16T17:19:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.444+0000 7fa66d85d740 1 -- 172.21.15.18:0/1174202158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65000d6b0 con 0x55c4364eb610 2024-04-16T17:19:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.445+0000 7fa64d600640 1 -- 172.21.15.18:0/1174202158 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa65402b050 con 0x55c4364eb610 2024-04-16T17:19:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.445+0000 7fa66d85d740 10 client.29347 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.446+0000 7fa66d85d740 1 -- 172.21.15.18:0/1174202158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c43656a490 con 0x55c4364eb610 2024-04-16T17:19:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.446+0000 7fa64d600640 1 -- 172.21.15.18:0/1174202158 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65000d6b0 con 0x55c4364eb610 2024-04-16T17:19:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.447+0000 7fa64d600640 1 -- 172.21.15.18:0/1174202158 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa654023380 con 0x55c4364eb610 2024-04-16T17:19:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.447+0000 7fa66d85d740 10 client.29347 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.447+0000 7fa66d85d740 10 client.29347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.447+0000 7fa66d85d740 20 client.29347 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.447+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c436504140 0x55c43656cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.447+0000 7fa66d85d740 4 client.29347 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:19:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.447+0000 7fa66d85d740 1 -- 172.21.15.18:0/1174202158 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c436457010 con 0x55c436504140 2024-04-16T17:19:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.448+0000 7fa668c00640 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c436504140 0x55c43656cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.448+0000 7fa668c00640 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c436504140 0x55c43656cc60 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1c1605a9658a60 server_cookie=d12e14f78c101d19 in_seq=0 out_seq=0 2024-04-16T17:19:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:11.449+0000 7fa64d600640 10 client.29347 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:11 smithi022 ceph-mon[36023]: pgmap v459: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 478 op/s 2024-04-16T17:19:12.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:11 smithi043 ceph-mon[36901]: pgmap v459: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 478 op/s 2024-04-16T17:19:12.184 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:11 smithi018 ceph-mon[31776]: pgmap v459: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 478 op/s 2024-04-16T17:19:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.189+0000 7fa64d600640 1 -- 172.21.15.18:0/1174202158 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c436457010 con 0x55c436504140 2024-04-16T17:19:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.189+0000 7fa64d600640 10 client.29347 handle_command_reply: tid=0 2024-04-16T17:19:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.190+0000 7fa66d85d740 1 client.29347 shutdown 2024-04-16T17:19:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.190+0000 7fa66d85d740 1 -- 172.21.15.18:0/1174202158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 msgr2=0x55c4364ed730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.190+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364ed730 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55c4364eb340 tx=0x7fa654005e70 comp rx=0 tx=0).stop 2024-04-16T17:19:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.191+0000 7fa66d85d740 1 -- 172.21.15.18:0/1174202158 shutdown_connections 2024-04-16T17:19:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.191+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c436504140 0x55c43656cc60 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.191+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4364f3e20 0x55c4364edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.191+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4364eb610 0x55c4364ed730 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.191+0000 7fa66d85d740 1 --2- 172.21.15.18:0/1174202158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4364ea8a0 0x55c4364fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.191+0000 7fa66d85d740 1 -- 172.21.15.18:0/1174202158 >> 172.21.15.18:0/1174202158 conn(0x55c4364855d0 msgr2=0x55c4364e70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.191+0000 7fa64d600640 0 client.29347 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.191+0000 7fa66d85d740 1 -- 172.21.15.18:0/1174202158 shutdown_connections 2024-04-16T17:19:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.192+0000 7fa66d85d740 1 -- 172.21.15.18:0/1174202158 wait complete. 2024-04-16T17:19:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.192+0000 7fa66d85d740 20 client.29347 trim_cache size 0 max 16384 2024-04-16T17:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 -- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa6480a3e10 msgr2=0x7fa6480a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa6480a3e10 0x7fa6480a62d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55c4364ebd70 tx=0x7fa65c0f6590 comp rx=0 tx=0).stop 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 -- 172.21.15.18:0/3175514402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa664101030 msgr2=0x7fa66419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa664101030 0x7fa66419f000 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005e30 tx=0x7fa65c003c20 comp rx=0 tx=0).stop 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 -- 172.21.15.18:0/3175514402 shutdown_connections 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa6480a3e10 0x7fa6480a62d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa664106430 0x7fa6641a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa664101950 0x7fa66419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 --2- 172.21.15.18:0/3175514402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa664101030 0x7fa66419f000 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.194+0000 7fa66ac00640 1 -- 172.21.15.18:0/3175514402 >> 172.21.15.18:0/3175514402 conn(0x7fa6640fca20 msgr2=0x7fa6640fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.195+0000 7fa66ac00640 1 -- 172.21.15.18:0/3175514402 shutdown_connections 2024-04-16T17:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:12.195+0000 7fa66ac00640 1 -- 172.21.15.18:0/3175514402 wait complete. 2024-04-16T17:19:12.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 21992 16.19 MB/sec warmup 84 sec latency 915.012 ms 2024-04-16T17:19:13.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 22177 16.06 MB/sec warmup 85 sec latency 246.598 ms 2024-04-16T17:19:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:13 smithi022 ceph-mon[36023]: pgmap v460: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-04-16T17:19:14.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:13 smithi043 ceph-mon[36901]: pgmap v460: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-04-16T17:19:14.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 22355 15.99 MB/sec warmup 86 sec latency 271.894 ms 2024-04-16T17:19:14.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:14 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:19:14.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:14 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:19:14.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:14 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:19:15.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:14 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:19:15.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:14 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:19:15.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:14 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:19:15.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 22632 15.92 MB/sec warmup 87 sec latency 123.765 ms 2024-04-16T17:19:15.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:15 smithi022 ceph-mon[36023]: pgmap v461: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-04-16T17:19:16.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:15 smithi043 ceph-mon[36901]: pgmap v461: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-04-16T17:19:16.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 23050 15.95 MB/sec warmup 88 sec latency 172.230 ms 2024-04-16T17:19:17.341 INFO:tasks.workunit.client.0.smithi018.stdout: 10 23502 16.11 MB/sec warmup 89 sec latency 190.945 ms 2024-04-16T17:19:17.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:13 smithi018 ceph-mon[31776]: pgmap v460: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-04-16T17:19:17.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:14 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:19:17.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:14 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:19:17.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:14 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:19:17.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:15 smithi018 ceph-mon[31776]: pgmap v461: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-04-16T17:19:18.344 INFO:tasks.workunit.client.0.smithi018.stdout: 10 23585 15.94 MB/sec warmup 90 sec latency 904.475 ms 2024-04-16T17:19:18.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:18 smithi022 ceph-mon[36023]: pgmap v462: 129 pgs: 129 active+clean; 306 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 652 op/s 2024-04-16T17:19:18.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:18 smithi043 ceph-mon[36901]: pgmap v462: 129 pgs: 129 active+clean; 306 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 652 op/s 2024-04-16T17:19:18.566 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:18 smithi018 ceph-mon[31776]: pgmap v462: 129 pgs: 129 active+clean; 306 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 652 op/s 2024-04-16T17:19:19.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:19 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:19:19.030 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:19 smithi018 ceph-mon[31776]: osdmap e118: 12 total, 12 up, 12 in 2024-04-16T17:19:19.345 INFO:tasks.workunit.client.0.smithi018.stdout: 10 23708 15.84 MB/sec warmup 91 sec latency 834.580 ms 2024-04-16T17:19:19.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:19 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:19:19.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:19 smithi022 ceph-mon[36023]: osdmap e118: 12 total, 12 up, 12 in 2024-04-16T17:19:19.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:19 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:19:19.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:19 smithi043 ceph-mon[36901]: osdmap e118: 12 total, 12 up, 12 in 2024-04-16T17:19:20.345 INFO:tasks.workunit.client.0.smithi018.stdout: 10 24020 15.89 MB/sec warmup 92 sec latency 154.694 ms 2024-04-16T17:19:20.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:20 smithi022 ceph-mon[36023]: pgmap v464: 129 pgs: 129 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-04-16T17:19:20.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:20 smithi043 ceph-mon[36901]: pgmap v464: 129 pgs: 129 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-04-16T17:19:21.345 INFO:tasks.workunit.client.0.smithi018.stdout: 10 24328 15.95 MB/sec warmup 93 sec latency 92.826 ms 2024-04-16T17:19:21.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:21 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:19:21.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:21 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:19:21.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:21 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:19:21.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:21 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:19:21.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:21 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:19:21.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:21 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:19:21.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:20 smithi018 ceph-mon[31776]: pgmap v464: 129 pgs: 129 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-04-16T17:19:21.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:21 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:19:21.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:21 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:19:21.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:21 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:19:22.345 INFO:tasks.workunit.client.0.smithi018.stdout: 10 24685 15.97 MB/sec warmup 94 sec latency 220.814 ms 2024-04-16T17:19:22.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:22 smithi022 ceph-mon[36023]: pgmap v465: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 543 op/s 2024-04-16T17:19:22.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:22 smithi043 ceph-mon[36901]: pgmap v465: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 543 op/s 2024-04-16T17:19:23.345 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25003 16.02 MB/sec warmup 95 sec latency 117.999 ms 2024-04-16T17:19:24.345 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25188 15.98 MB/sec warmup 96 sec latency 216.553 ms 2024-04-16T17:19:24.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:24 smithi022 ceph-mon[36023]: pgmap v466: 129 pgs: 129 active+clean; 333 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-04-16T17:19:24.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:24 smithi043 ceph-mon[36901]: pgmap v466: 129 pgs: 129 active+clean; 333 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-04-16T17:19:25.346 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25414 15.94 MB/sec warmup 97 sec latency 127.701 ms 2024-04-16T17:19:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.419+0000 7f3ce4400640 1 -- 172.21.15.18:0/3059113599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076e50 msgr2=0x7f3cdc112960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.419+0000 7f3ce4400640 1 --2- 172.21.15.18:0/3059113599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076e50 0x7f3cdc112960 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009890 tx=0x7f3cc802bb70 comp rx=0 tx=0).stop 2024-04-16T17:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.422+0000 7f3ce4400640 1 -- 172.21.15.18:0/3059113599 shutdown_connections 2024-04-16T17:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.422+0000 7f3ce4400640 1 --2- 172.21.15.18:0/3059113599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cdc112ea0 0x7f3cdc115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.422+0000 7f3ce4400640 1 --2- 172.21.15.18:0/3059113599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076e50 0x7f3cdc112960 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.422+0000 7f3ce4400640 1 --2- 172.21.15.18:0/3059113599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cdc076530 0x7f3cdc076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.422+0000 7f3ce4400640 1 -- 172.21.15.18:0/3059113599 >> 172.21.15.18:0/3059113599 conn(0x7f3cdc0ffcd0 msgr2=0x7f3cdc1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.423+0000 7f3ce4400640 1 -- 172.21.15.18:0/3059113599 shutdown_connections 2024-04-16T17:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.423+0000 7f3ce4400640 1 -- 172.21.15.18:0/3059113599 wait complete. 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.425+0000 7f3ce4400640 1 Processor -- start 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.425+0000 7f3ce4400640 1 -- start start 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.426+0000 7f3ce4400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076530 0x7f3cdc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.426+0000 7f3ce1a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076530 0x7f3cdc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.426+0000 7f3ce1a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076530 0x7f3cdc19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37020/0 (socket says 172.21.15.18:37020) 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.426+0000 7f3ce1a00640 1 -- 172.21.15.18:0/2822578249 learned_addr learned my addr 172.21.15.18:0/2822578249 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.426+0000 7f3ce4400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cdc076e50 0x7f3cdc19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce4400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cdc112ea0 0x7f3cdc1a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc118a00 con 0x7f3cdc076e50 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc118880 con 0x7f3cdc076530 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc118b80 con 0x7f3cdc112ea0 2024-04-16T17:19:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce2400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cdc112ea0 0x7f3cdc1a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce1a00640 1 -- 172.21.15.18:0/2822578249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cdc112ea0 msgr2=0x7f3cdc1a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cdc112ea0 0x7f3cdc1a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce1a00640 1 -- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cdc076e50 msgr2=0x7f3cdc19f2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:19:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cdc076e50 0x7f3cdc19f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.427+0000 7f3ce1a00640 1 -- 172.21.15.18:0/2822578249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc1a3d40 con 0x7f3cdc076530 2024-04-16T17:19:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.428+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076530 0x7f3cdc19eda0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000efb0 tx=0x7f3cd000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.432+0000 7f3cda000640 1 -- 172.21.15.18:0/2822578249 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd001b070 con 0x7f3cdc076530 2024-04-16T17:19:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.432+0000 7f3cda000640 1 -- 172.21.15.18:0/2822578249 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd000ba50 con 0x7f3cdc076530 2024-04-16T17:19:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.432+0000 7f3cda000640 1 -- 172.21.15.18:0/2822578249 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0016c00 con 0x7f3cdc076530 2024-04-16T17:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.432+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cdc1a4030 con 0x7f3cdc076530 2024-04-16T17:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.432+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc104110 con 0x7f3cdc076530 2024-04-16T17:19:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.441+0000 7f3cda000640 1 -- 172.21.15.18:0/2822578249 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3cd001e070 con 0x7f3cdc076530 2024-04-16T17:19:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.441+0000 7f3ce71da740 1 Processor -- start 2024-04-16T17:19:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.442+0000 7f3ce71da740 1 -- start start 2024-04-16T17:19:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.443+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744640610 0x55f74463f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.443+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f74463fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.443+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f74465c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.443+0000 7f3ce71da740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f7445a4780 con 0x55f744640610 2024-04-16T17:19:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.443+0000 7f3ce71da740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f74454af40 con 0x55f74463f8a0 2024-04-16T17:19:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.443+0000 7f3ce71da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f7445a0740 con 0x55f744659e30 2024-04-16T17:19:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce1a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f74463fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce2400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744640610 0x55f74463f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce2400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744640610 0x55f74463f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43158/0 (socket says 172.21.15.18:43158) 2024-04-16T17:19:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f74465c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce2400640 1 -- 172.21.15.18:0/1992763055 learned_addr learned my addr 172.21.15.18:0/1992763055 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3cda000640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3cc40a3e60 0x7f3cc40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce1000640 1 -- 172.21.15.18:0/1992763055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 msgr2=0x55f74463fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce1000640 1 --2- 172.21.15.18:0/1992763055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f74463fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce1000640 1 -- 172.21.15.18:0/1992763055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744640610 msgr2=0x55f74463f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce1000640 1 --2- 172.21.15.18:0/1992763055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744640610 0x55f74463f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3ce1000640 1 -- 172.21.15.18:0/1992763055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7442b7210 con 0x55f744659e30 2024-04-16T17:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.444+0000 7f3cda000640 1 -- 172.21.15.18:0/2822578249 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f3cd00d09f0 con 0x7f3cdc076530 2024-04-16T17:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.445+0000 7f3ce2400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3cc40a3e60 0x7f3cc40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.446+0000 7f3ce1000640 1 --2- 172.21.15.18:0/1992763055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f74465c220 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3cc802c0b0 tx=0x7f3cc8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=636d115d8d6b8bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.446+0000 7f3ccea00640 1 -- 172.21.15.18:0/1992763055 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8041020 con 0x55f744659e30 2024-04-16T17:19:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.447+0000 7f3ccea00640 1 -- 172.21.15.18:0/1992763055 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc8004040 con 0x55f744659e30 2024-04-16T17:19:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.447+0000 7f3ccea00640 1 -- 172.21.15.18:0/1992763055 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8039040 con 0x55f744659e30 2024-04-16T17:19:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.447+0000 7f3ce71da740 1 -- 172.21.15.18:0/1992763055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 msgr2=0x55f74465c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.447+0000 7f3ce71da740 1 --2- 172.21.15.18:0/1992763055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f74465c220 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3cc802c0b0 tx=0x7f3cc8005fd0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.450+0000 7f3ce2400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3cc40a3e60 0x7f3cc40a6320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400b7a0 tx=0x7f3cd40046c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.450+0000 7f3ce71da740 1 -- 172.21.15.18:0/1992763055 shutdown_connections 2024-04-16T17:19:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.450+0000 7f3ce71da740 1 --2- 172.21.15.18:0/1992763055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f74465c220 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.450+0000 7f3ce71da740 1 --2- 172.21.15.18:0/1992763055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f74463fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.450+0000 7f3ce71da740 1 --2- 172.21.15.18:0/1992763055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744640610 0x55f74463f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.450+0000 7f3ce71da740 1 -- 172.21.15.18:0/1992763055 >> 172.21.15.18:0/1992763055 conn(0x55f7445da5d0 msgr2=0x55f74463a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.451+0000 7f3ce71da740 1 -- 172.21.15.18:0/1992763055 shutdown_connections 2024-04-16T17:19:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.451+0000 7f3ce71da740 1 -- 172.21.15.18:0/1992763055 wait complete. 2024-04-16T17:19:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.452+0000 7f3ce71da740 1 -- start start 2024-04-16T17:19:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.453+0000 7f3ce71da740 1 Processor -- start 2024-04-16T17:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f744659810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744656c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744659e30 0x55f744657290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce71da740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f74465cb00 con 0x55f744659e30 2024-04-16T17:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce71da740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f74455e390 con 0x55f74463f8a0 2024-04-16T17:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce71da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f74465cc80 con 0x55f744640610 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce71da740 10 client.0 _ll_register_callbacks cb 0x55f744547f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744656c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce1a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f744659810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744656c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47250/0 (socket says 172.21.15.18:47250) 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.455+0000 7f3ce1000640 1 -- 172.21.15.18:0/3598281729 learned_addr learned my addr 172.21.15.18:0/3598281729 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.456+0000 7f3ce1a00640 1 -- 172.21.15.18:0/3598281729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 msgr2=0x55f744656c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.456+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744656c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.456+0000 7f3ce1a00640 1 -- 172.21.15.18:0/3598281729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744659e30 msgr2=0x55f744657290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.456+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744659e30 0x55f744657290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.456+0000 7f3ce1a00640 1 -- 172.21.15.18:0/3598281729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f744383620 con 0x55f74463f8a0 2024-04-16T17:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.456+0000 7f3ce1000640 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744656c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.456+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f744659810 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00cdd30 tx=0x7f3cd00a51f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f092f9b6c0489cce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.456+0000 7f3cc3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.457+0000 7f3cc3400640 1 -- 172.21.15.18:0/3598281729 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00a4d60 con 0x55f74463f8a0 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.457+0000 7f3cc3400640 1 -- 172.21.15.18:0/3598281729 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f3cd00a4f00 con 0x55f74463f8a0 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.457+0000 7f3ce71da740 1 -- 172.21.15.18:0/3598281729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc8004040 con 0x55f74463f8a0 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.457+0000 7f3cc3400640 1 -- 172.21.15.18:0/3598281729 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3cd009ba80 con 0x55f74463f8a0 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.457+0000 7f3ce71da740 10 client.29525 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.457+0000 7f3ce71da740 1 -- 172.21.15.18:0/3598281729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f744668c30 con 0x55f74463f8a0 2024-04-16T17:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.458+0000 7f3cc3400640 1 -- 172.21.15.18:0/3598281729 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc8004040 con 0x55f74463f8a0 2024-04-16T17:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.459+0000 7f3cc3400640 1 -- 172.21.15.18:0/3598281729 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3cd000b040 con 0x55f74463f8a0 2024-04-16T17:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.459+0000 7f3ce71da740 10 client.29525 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.459+0000 7f3ce71da740 10 client.29525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.459+0000 7f3ce71da740 20 client.29525 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.459+0000 7f3ce71da740 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f74466a540 0x55f74466c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.460+0000 7f3ce71da740 4 client.29525 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.460+0000 7f3ce71da740 1 -- 172.21.15.18:0/3598281729 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7445b21a0 con 0x55f74466a540 2024-04-16T17:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.462+0000 7f3ce1000640 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f74466a540 0x55f74466c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.462+0000 7f3ce1000640 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f74466a540 0x55f74466c9e0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a828719b4df216d server_cookie=14b7aa20f9266771 in_seq=0 out_seq=0 2024-04-16T17:19:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.462+0000 7f3cc3400640 10 client.29525 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.466+0000 7f3cc3400640 1 -- 172.21.15.18:0/3598281729 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7445b21a0 con 0x55f74466a540 2024-04-16T17:19:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.466+0000 7f3cc3400640 10 client.29525 handle_command_reply: tid=0 2024-04-16T17:19:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.466+0000 7f3ce71da740 1 client.29525 shutdown 2024-04-16T17:19:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.467+0000 7f3ce71da740 1 -- 172.21.15.18:0/3598281729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 msgr2=0x55f744659810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.467+0000 7f3ce71da740 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f744659810 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00cdd30 tx=0x7f3cd00a51f0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.468+0000 7f3ce71da740 1 -- 172.21.15.18:0/3598281729 shutdown_connections 2024-04-16T17:19:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.468+0000 7f3ce71da740 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f74466a540 0x55f74466c9e0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.468+0000 7f3ce71da740 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744659e30 0x55f744657290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.468+0000 7f3ce71da740 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744656c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.468+0000 7f3ce71da740 1 --2- 172.21.15.18:0/3598281729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f744659810 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.468+0000 7f3ce71da740 1 -- 172.21.15.18:0/3598281729 >> 172.21.15.18:0/3598281729 conn(0x55f7445da5d0 msgr2=0x55f74463c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.468+0000 7f3cc3400640 0 client.29525 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.468+0000 7f3ce71da740 1 -- 172.21.15.18:0/3598281729 shutdown_connections 2024-04-16T17:19:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.469+0000 7f3ce71da740 1 -- 172.21.15.18:0/3598281729 wait complete. 2024-04-16T17:19:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.469+0000 7f3ce71da740 20 client.29525 trim_cache size 0 max 16384 2024-04-16T17:19:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.492+0000 7f3ce71da740 1 Processor -- start 2024-04-16T17:19:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.493+0000 7f3ce71da740 1 -- start start 2024-04-16T17:19:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.493+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f7446564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.493+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744665650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.493+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744659e30 0x55f744659340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.493+0000 7f3ce71da740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc012a70 con 0x55f744659e30 2024-04-16T17:19:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.493+0000 7f3ce71da740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc0102e0 con 0x55f74463f8a0 2024-04-16T17:19:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.493+0000 7f3ce71da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc00e150 con 0x55f744640610 2024-04-16T17:19:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.494+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744665650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.494+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744665650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47262/0 (socket says 172.21.15.18:47262) 2024-04-16T17:19:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.494+0000 7f3ce1000640 1 -- 172.21.15.18:0/4032733117 learned_addr learned my addr 172.21.15.18:0/4032733117 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.494+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/4032733117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f7446564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.494+0000 7f3ce1000640 1 -- 172.21.15.18:0/4032733117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 msgr2=0x55f7446564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.494+0000 7f3ce1000640 1 --2- 172.21.15.18:0/4032733117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f7446564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.495+0000 7f3ce1000640 1 -- 172.21.15.18:0/4032733117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744659e30 msgr2=0x55f744659340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.495+0000 7f3ce1000640 1 --2- 172.21.15.18:0/4032733117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744659e30 0x55f744659340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.495+0000 7f3ce1000640 1 -- 172.21.15.18:0/4032733117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74463eb70 con 0x55f744640610 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.495+0000 7f3ce1000640 1 --2- 172.21.15.18:0/4032733117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744665650 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005b60 tx=0x7f3cc8005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66eb78a1becd34ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.495+0000 7f3cce000640 1 -- 172.21.15.18:0/4032733117 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8041040 con 0x55f744640610 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.495+0000 7f3cce000640 1 -- 172.21.15.18:0/4032733117 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc804b070 con 0x55f744640610 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.495+0000 7f3cce000640 1 -- 172.21.15.18:0/4032733117 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8041040 con 0x55f744640610 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.497+0000 7f3ce71da740 1 -- 172.21.15.18:0/4032733117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 msgr2=0x55f744665650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.497+0000 7f3ce71da740 1 --2- 172.21.15.18:0/4032733117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744665650 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005b60 tx=0x7f3cc8005ad0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.498+0000 7f3ce71da740 1 -- 172.21.15.18:0/4032733117 shutdown_connections 2024-04-16T17:19:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.498+0000 7f3ce71da740 1 --2- 172.21.15.18:0/4032733117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f744659e30 0x55f744659340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.498+0000 7f3ce71da740 1 --2- 172.21.15.18:0/4032733117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744640610 0x55f744665650 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.498+0000 7f3ce71da740 1 --2- 172.21.15.18:0/4032733117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f74463f8a0 0x55f7446564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.498+0000 7f3ce71da740 1 -- 172.21.15.18:0/4032733117 >> 172.21.15.18:0/4032733117 conn(0x55f7445da5d0 msgr2=0x55f744664f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.498+0000 7f3ce71da740 1 -- 172.21.15.18:0/4032733117 shutdown_connections 2024-04-16T17:19:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.498+0000 7f3ce71da740 1 -- 172.21.15.18:0/4032733117 wait complete. 2024-04-16T17:19:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.499+0000 7f3ce71da740 1 -- start start 2024-04-16T17:19:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.500+0000 7f3ce71da740 1 Processor -- start 2024-04-16T17:19:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.501+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f74463f8a0 0x55f74466f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.502+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f744640610 0x55f74466f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.502+0000 7f3ce71da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f7446ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.502+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f744640610 0x55f74466f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.502+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f744640610 0x55f74466f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37058/0 (socket says 172.21.15.18:37058) 2024-04-16T17:19:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.502+0000 7f3ce2400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f7446ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.502+0000 7f3ce71da740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc00e7a0 con 0x55f74463f8a0 2024-04-16T17:19:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.503+0000 7f3ce1000640 1 -- 172.21.15.18:0/327540763 learned_addr learned my addr 172.21.15.18:0/327540763 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.503+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc00fc10 con 0x55f744640610 2024-04-16T17:19:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.503+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc00ee70 con 0x55f744659e30 2024-04-16T17:19:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.503+0000 7f3ce71da740 10 client.0 _ll_register_callbacks cb 0x7f3cd0098950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.503+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f74463f8a0 0x55f74466f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3ce1000640 1 -- 172.21.15.18:0/327540763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 msgr2=0x55f7446ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3ce1000640 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f7446ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3ce1000640 1 -- 172.21.15.18:0/327540763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f74463f8a0 msgr2=0x55f74466f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3ce1000640 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f74463f8a0 0x55f74466f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3ce1000640 1 -- 172.21.15.18:0/327540763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7446cee40 con 0x55f744640610 2024-04-16T17:19:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f74463f8a0 0x55f74466f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:19:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3ce1000640 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f744640610 0x55f74466f6b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55f744668b30 tx=0x7f3cc8034c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed71d50de2f185be server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3cc3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.504+0000 7f3cc3400640 1 -- 172.21.15.18:0/327540763 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8041050 con 0x55f744640610 2024-04-16T17:19:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.505+0000 7f3cc3400640 1 -- 172.21.15.18:0/327540763 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f3cc8034850 con 0x55f744640610 2024-04-16T17:19:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.505+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc804b070 con 0x55f744640610 2024-04-16T17:19:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.505+0000 7f3cc3400640 1 -- 172.21.15.18:0/327540763 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3cc8039040 con 0x55f744640610 2024-04-16T17:19:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.506+0000 7f3ce71da740 10 client.29531 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.506+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f7446d00f0 con 0x55f744640610 2024-04-16T17:19:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.506+0000 7f3cc3400640 1 -- 172.21.15.18:0/327540763 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc804b070 con 0x55f744640610 2024-04-16T17:19:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.507+0000 7f3cc3400640 1 -- 172.21.15.18:0/327540763 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3cc803c030 con 0x55f744640610 2024-04-16T17:19:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.507+0000 7f3ce71da740 10 client.29531 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.507+0000 7f3ce71da740 10 client.29531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.508+0000 7f3ce71da740 20 client.29531 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.508+0000 7f3ce71da740 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7446d0310 0x55f7446d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.508+0000 7f3ce71da740 4 client.29531 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:19:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.508+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7445ac010 con 0x55f7446d0310 2024-04-16T17:19:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.508+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7446d0310 0x55f7446d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.509+0000 7f3ce1a00640 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7446d0310 0x55f7446d2700 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8895a8f710bb940b server_cookie=f397b2868a383a61 in_seq=0 out_seq=0 2024-04-16T17:19:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:25.509+0000 7f3cc3400640 10 client.29531 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:25.842 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:22 smithi018 ceph-mon[31776]: pgmap v465: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 543 op/s 2024-04-16T17:19:25.842 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:24 smithi018 ceph-mon[31776]: pgmap v466: 129 pgs: 129 active+clean; 333 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-04-16T17:19:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.340+0000 7f3cc3400640 1 -- 172.21.15.18:0/327540763 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7445ac010 con 0x55f7446d0310 2024-04-16T17:19:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.340+0000 7f3cc3400640 10 client.29531 handle_command_reply: tid=0 2024-04-16T17:19:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 client.29531 shutdown 2024-04-16T17:19:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f744640610 msgr2=0x55f74466f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f744640610 0x55f74466f6b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55f744668b30 tx=0x7f3cc8034c80 comp rx=0 tx=0).stop 2024-04-16T17:19:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 shutdown_connections 2024-04-16T17:19:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f7446d0310 0x55f7446d2700 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f744659e30 0x55f7446ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f744640610 0x55f74466f6b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 --2- 172.21.15.18:0/327540763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f74463f8a0 0x55f74466f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.341+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 >> 172.21.15.18:0/327540763 conn(0x55f7445da5d0 msgr2=0x55f7446657d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.342+0000 7f3cc3400640 0 client.29531 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.344+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 shutdown_connections 2024-04-16T17:19:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.344+0000 7f3ce71da740 1 -- 172.21.15.18:0/327540763 wait complete. 2024-04-16T17:19:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.345+0000 7f3ce71da740 20 client.29531 trim_cache size 0 max 16384 2024-04-16T17:19:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:19:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:19:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:19:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:19:26.346 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25506 15.85 MB/sec warmup 98 sec latency 621.370 ms 2024-04-16T17:19:26.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.347+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3cc40a3e60 msgr2=0x7f3cc40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:26.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.347+0000 7f3ce4400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3cc40a3e60 0x7f3cc40a6320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400b7a0 tx=0x7f3cd40046c0 comp rx=0 tx=0).stop 2024-04-16T17:19:26.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.347+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076530 msgr2=0x7f3cdc19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:26.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.347+0000 7f3ce4400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076530 0x7f3cdc19eda0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000efb0 tx=0x7f3cd000c800 comp rx=0 tx=0).stop 2024-04-16T17:19:26.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.348+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 shutdown_connections 2024-04-16T17:19:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.348+0000 7f3ce4400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3cc40a3e60 0x7f3cc40a6320 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.348+0000 7f3ce4400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cdc112ea0 0x7f3cdc1a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.348+0000 7f3ce4400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cdc076e50 0x7f3cdc19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.348+0000 7f3ce4400640 1 --2- 172.21.15.18:0/2822578249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cdc076530 0x7f3cdc19eda0 secure :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000efb0 tx=0x7f3cd000c800 comp rx=0 tx=0).stop 2024-04-16T17:19:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.348+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 >> 172.21.15.18:0/2822578249 conn(0x7f3cdc0ffcd0 msgr2=0x7f3cdc114890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.349+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 shutdown_connections 2024-04-16T17:19:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:26.350+0000 7f3ce4400640 1 -- 172.21.15.18:0/2822578249 wait complete. 2024-04-16T17:19:26.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:26 smithi022 ceph-mon[36023]: pgmap v467: 129 pgs: 129 active+clean; 333 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-04-16T17:19:26.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:26 smithi043 ceph-mon[36901]: pgmap v467: 129 pgs: 129 active+clean; 333 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-04-16T17:19:26.581 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:26 smithi018 ceph-mon[31776]: pgmap v467: 129 pgs: 129 active+clean; 333 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-04-16T17:19:27.347 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25854 15.81 MB/sec warmup 99 sec latency 85.779 ms 2024-04-16T17:19:28.347 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25891 15.66 MB/sec warmup 100 sec latency 881.379 ms 2024-04-16T17:19:28.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:28 smithi022 ceph-mon[36023]: pgmap v468: 129 pgs: 129 active+clean; 335 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 386 op/s 2024-04-16T17:19:28.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:28 smithi018 ceph-mon[31776]: pgmap v468: 129 pgs: 129 active+clean; 335 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 386 op/s 2024-04-16T17:19:28.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:28 smithi043 ceph-mon[36901]: pgmap v468: 129 pgs: 129 active+clean; 335 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 386 op/s 2024-04-16T17:19:29.347 INFO:tasks.workunit.client.0.smithi018.stdout: 10 25917 15.51 MB/sec warmup 101 sec latency 1609.921 ms 2024-04-16T17:19:30.347 INFO:tasks.workunit.client.0.smithi018.stdout: 10 26152 15.47 MB/sec warmup 102 sec latency 420.111 ms 2024-04-16T17:19:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:30 smithi022 ceph-mon[36023]: pgmap v469: 129 pgs: 129 active+clean; 334 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 400 op/s 2024-04-16T17:19:30.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:30 smithi043 ceph-mon[36901]: pgmap v469: 129 pgs: 129 active+clean; 334 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 400 op/s 2024-04-16T17:19:31.347 INFO:tasks.workunit.client.0.smithi018.stdout: 10 26595 15.54 MB/sec warmup 103 sec latency 57.083 ms 2024-04-16T17:19:32.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:19:32.159 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:19:32.348 INFO:tasks.workunit.client.0.smithi018.stdout: 10 26957 15.60 MB/sec warmup 104 sec latency 137.953 ms 2024-04-16T17:19:32.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.472+0000 7f172e400640 1 -- 172.21.15.18:0/576002615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728110750 msgr2=0x7f172810e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.472+0000 7f172e400640 1 --2- 172.21.15.18:0/576002615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728110750 0x7f172810e310 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1714009900 tx=0x7f171402bc60 comp rx=0 tx=0).stop 2024-04-16T17:19:32.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.473+0000 7f172e400640 1 -- 172.21.15.18:0/576002615 shutdown_connections 2024-04-16T17:19:32.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.473+0000 7f172e400640 1 --2- 172.21.15.18:0/576002615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f172811b7e0 0x7f172811dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.473+0000 7f172e400640 1 --2- 172.21.15.18:0/576002615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1728110bc0 0x7f172810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.473+0000 7f172e400640 1 --2- 172.21.15.18:0/576002615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728110750 0x7f172810e310 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.473+0000 7f172e400640 1 -- 172.21.15.18:0/576002615 >> 172.21.15.18:0/576002615 conn(0x7f172806d0a0 msgr2=0x7f172806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:32.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.474+0000 7f172e400640 1 -- 172.21.15.18:0/576002615 shutdown_connections 2024-04-16T17:19:32.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.474+0000 7f172e400640 1 -- 172.21.15.18:0/576002615 wait complete. 2024-04-16T17:19:32.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.476+0000 7f172e400640 1 Processor -- start 2024-04-16T17:19:32.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.476+0000 7f172e400640 1 -- start start 2024-04-16T17:19:32.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f172e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1728110750 0x7f17281a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f172e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1728110bc0 0x7f17281a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f172e400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f172811b7e0 0x7f17281abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f172e400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1728121270 con 0x7f172811b7e0 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f172e400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17281210f0 con 0x7f1728110750 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f172e400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17281213f0 con 0x7f1728110bc0 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f1727400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1728110750 0x7f17281a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f1727400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1728110750 0x7f17281a76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54766/0 (socket says 172.21.15.18:54766) 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f1726a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1728110bc0 0x7f17281a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.477+0000 7f1727400640 1 -- 172.21.15.18:0/2010481617 learned_addr learned my addr 172.21.15.18:0/2010481617 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.478+0000 7f1727e00640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f172811b7e0 0x7f17281abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.478+0000 7f1727400640 1 -- 172.21.15.18:0/2010481617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1728110bc0 msgr2=0x7f17281a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.478+0000 7f1727400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1728110bc0 0x7f17281a7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.478+0000 7f1727400640 1 -- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f172811b7e0 msgr2=0x7f17281abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.478+0000 7f1727400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f172811b7e0 0x7f17281abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.478+0000 7f1727400640 1 -- 172.21.15.18:0/2010481617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17281ac6e0 con 0x7f1728110750 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.478+0000 7f1727400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1728110750 0x7f17281a76f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f171402efd0 tx=0x7f17140043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:32.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.479+0000 7f171fe00640 1 -- 172.21.15.18:0/2010481617 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714041070 con 0x7f1728110750 2024-04-16T17:19:32.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.479+0000 7f171fe00640 1 -- 172.21.15.18:0/2010481617 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f171402ccc0 con 0x7f1728110750 2024-04-16T17:19:32.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.479+0000 7f171fe00640 1 -- 172.21.15.18:0/2010481617 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714003db0 con 0x7f1728110750 2024-04-16T17:19:32.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.483+0000 7f172e400640 1 -- 172.21.15.18:0/2010481617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17281ac970 con 0x7f1728110750 2024-04-16T17:19:32.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.483+0000 7f172e400640 1 -- 172.21.15.18:0/2010481617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17281acde0 con 0x7f1728110750 2024-04-16T17:19:32.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.489+0000 7f171fe00640 1 -- 172.21.15.18:0/2010481617 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1714044070 con 0x7f1728110750 2024-04-16T17:19:32.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.491+0000 7f17311df740 1 Processor -- start 2024-04-16T17:19:32.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.491+0000 7f17311df740 1 -- start start 2024-04-16T17:19:32.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.492+0000 7f17311df740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d57590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.492+0000 7f17311df740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d57ad0 0x55fb00d60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.492+0000 7f17311df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d62aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.492+0000 7f17311df740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fb00cbd120 con 0x55fb00d571b0 2024-04-16T17:19:32.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.492+0000 7f17311df740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb00c63340 con 0x55fb00d57ad0 2024-04-16T17:19:32.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.492+0000 7f17311df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb00cb90e0 con 0x55fb00d606b0 2024-04-16T17:19:32.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.492+0000 7f171fe00640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f170c0a3e70 0x7f170c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.492+0000 7f171fe00640 1 -- 172.21.15.18:0/2010481617 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f17140f2180 con 0x7f1728110750 2024-04-16T17:19:32.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.493+0000 7f1727e00640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f170c0a3e70 0x7f170c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.493+0000 7f1727400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d57ad0 0x55fb00d60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.493+0000 7f1727400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d57ad0 0x55fb00d60170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54780/0 (socket says 172.21.15.18:54780) 2024-04-16T17:19:32.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.493+0000 7f1727400640 1 -- 172.21.15.18:0/1534346243 learned_addr learned my addr 172.21.15.18:0/1534346243 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:32.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.493+0000 7f1727e00640 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d57590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.493+0000 7f1726a00640 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d62aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.494+0000 7f1726a00640 1 -- 172.21.15.18:0/1534346243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d57ad0 msgr2=0x55fb00d60170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:32.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.494+0000 7f1726a00640 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d57ad0 0x55fb00d60170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.494+0000 7f1726a00640 1 -- 172.21.15.18:0/1534346243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 msgr2=0x55fb00d57590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:32.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.494+0000 7f1726a00640 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d57590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.494+0000 7f1726a00640 1 -- 172.21.15.18:0/1534346243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb00a997d0 con 0x55fb00d606b0 2024-04-16T17:19:32.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.494+0000 7f1727400640 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d57ad0 0x55fb00d60170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:19:32.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.494+0000 7f1727e00640 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d57590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:32.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.495+0000 7f1726a00640 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d62aa0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1718006fd0 tx=0x7f1718009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd070a0dde6541e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:32.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.495+0000 7f171cc00640 1 -- 172.21.15.18:0/1534346243 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718017070 con 0x55fb00d606b0 2024-04-16T17:19:32.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.495+0000 7f171cc00640 1 -- 172.21.15.18:0/1534346243 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1718013470 con 0x55fb00d606b0 2024-04-16T17:19:32.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.495+0000 7f171cc00640 1 -- 172.21.15.18:0/1534346243 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718012630 con 0x55fb00d606b0 2024-04-16T17:19:32.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 -- 172.21.15.18:0/1534346243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 msgr2=0x55fb00d62aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d62aa0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1718006fd0 tx=0x7f1718009e10 comp rx=0 tx=0).stop 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f1727e00640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f170c0a3e70 0x7f170c0a6330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f172806b380 tx=0x7f1720004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 -- 172.21.15.18:0/1534346243 shutdown_connections 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d62aa0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d57ad0 0x55fb00d60170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 --2- 172.21.15.18:0/1534346243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d57590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 -- 172.21.15.18:0/1534346243 >> 172.21.15.18:0/1534346243 conn(0x55fb00d518e0 msgr2=0x55fb00d51cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 -- 172.21.15.18:0/1534346243 shutdown_connections 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.496+0000 7f17311df740 1 -- 172.21.15.18:0/1534346243 wait complete. 2024-04-16T17:19:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.497+0000 7f17311df740 1 -- start start 2024-04-16T17:19:32.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.498+0000 7f17311df740 1 Processor -- start 2024-04-16T17:19:32.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.498+0000 7f17311df740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.498+0000 7f17311df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d57ad0 0x55fb00d5dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.498+0000 7f17311df740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d606b0 0x55fb00d5e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.498+0000 7f17311df740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fb00d634a0 con 0x55fb00d571b0 2024-04-16T17:19:32.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.498+0000 7f17311df740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb00c76710 con 0x55fb00d606b0 2024-04-16T17:19:32.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.498+0000 7f17311df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb00d63620 con 0x55fb00d57ad0 2024-04-16T17:19:32.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.499+0000 7f17311df740 10 client.0 _ll_register_callbacks cb 0x55fb00c71af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:32.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.499+0000 7f1727e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d606b0 0x55fb00d5e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.499+0000 7f1727e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d606b0 0x55fb00d5e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54786/0 (socket says 172.21.15.18:54786) 2024-04-16T17:19:32.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.499+0000 7f1727e00640 1 -- 172.21.15.18:0/1599232207 learned_addr learned my addr 172.21.15.18:0/1599232207 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:32.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.499+0000 7f1727400640 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.499+0000 7f1726a00640 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d57ad0 0x55fb00d5dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.500+0000 7f1726a00640 1 -- 172.21.15.18:0/1599232207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d606b0 msgr2=0x55fb00d5e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:32.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.500+0000 7f1726a00640 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d606b0 0x55fb00d5e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.500+0000 7f1726a00640 1 -- 172.21.15.18:0/1599232207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 msgr2=0x55fb00d5d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.500+0000 7f1726a00640 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d5d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.500+0000 7f1726a00640 1 -- 172.21.15.18:0/1599232207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb009cf3f0 con 0x55fb00d57ad0 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.501+0000 7f1726a00640 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d57ad0 0x55fb00d5dbd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1718000c00 tx=0x7f1718013850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b81ee29306fdb6e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.501+0000 7f1711600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.501+0000 7f1711600640 1 -- 172.21.15.18:0/1599232207 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718017050 con 0x55fb00d57ad0 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.501+0000 7f17311df740 1 -- 172.21.15.18:0/1599232207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1718013470 con 0x55fb00d57ad0 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.501+0000 7f1711600640 1 -- 172.21.15.18:0/1599232207 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f171801c650 con 0x55fb00d57ad0 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.501+0000 7f1711600640 1 -- 172.21.15.18:0/1599232207 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f171802e050 con 0x55fb00d57ad0 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.501+0000 7f17311df740 10 client.29374 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:32.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.501+0000 7f17311df740 1 -- 172.21.15.18:0/1599232207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fb0098f0f0 con 0x55fb00d57ad0 2024-04-16T17:19:32.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:32 smithi043 ceph-mon[36901]: pgmap v470: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-04-16T17:19:32.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f1711600640 1 -- 172.21.15.18:0/1599232207 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1718013470 con 0x55fb00d57ad0 2024-04-16T17:19:32.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f1711600640 1 -- 172.21.15.18:0/1599232207 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1718025530 con 0x55fb00d57ad0 2024-04-16T17:19:32.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f17311df740 10 client.29374 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f17311df740 10 client.29374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f17311df740 20 client.29374 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f17311df740 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb00d70870 0x55fb00d72c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f17311df740 4 client.29374 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f17311df740 1 -- 172.21.15.18:0/1599232207 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb00ccab40 con 0x55fb00d70870 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.502+0000 7f1727400640 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb00d70870 0x55fb00d72c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.503+0000 7f1727400640 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb00d70870 0x55fb00d72c60 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a1a94a4e5f5bf server_cookie=41afba66692f5d75 in_seq=0 out_seq=0 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.503+0000 7f1711600640 10 client.29374 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.503+0000 7f1711600640 1 -- 172.21.15.18:0/1599232207 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb00ccab40 con 0x55fb00d70870 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.503+0000 7f1711600640 10 client.29374 handle_command_reply: tid=0 2024-04-16T17:19:32.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 client.29374 shutdown 2024-04-16T17:19:32.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 -- 172.21.15.18:0/1599232207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d57ad0 msgr2=0x55fb00d5dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d57ad0 0x55fb00d5dbd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1718000c00 tx=0x7f1718013850 comp rx=0 tx=0).stop 2024-04-16T17:19:32.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 -- 172.21.15.18:0/1599232207 shutdown_connections 2024-04-16T17:19:32.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb00d70870 0x55fb00d72c60 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d606b0 0x55fb00d5e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d57ad0 0x55fb00d5dbd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 --2- 172.21.15.18:0/1599232207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d571b0 0x55fb00d5d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f1711600640 0 client.29374 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:32.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 -- 172.21.15.18:0/1599232207 >> 172.21.15.18:0/1599232207 conn(0x55fb00d518e0 msgr2=0x55fb00d53630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:32.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 -- 172.21.15.18:0/1599232207 shutdown_connections 2024-04-16T17:19:32.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.504+0000 7f17311df740 1 -- 172.21.15.18:0/1599232207 wait complete. 2024-04-16T17:19:32.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.505+0000 7f17311df740 20 client.29374 trim_cache size 0 max 16384 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.546+0000 7f17311df740 1 Processor -- start 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.546+0000 7f17311df740 1 -- start start 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.547+0000 7f17311df740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d5fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.547+0000 7f17311df740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 0x55fb00d5c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.547+0000 7f17311df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d6c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.548+0000 7f17311df740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17281c0d20 con 0x55fb00d57ad0 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.548+0000 7f17311df740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17281be590 con 0x55fb00d571b0 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.548+0000 7f17311df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17281bc400 con 0x55fb00d606b0 2024-04-16T17:19:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.548+0000 7f1727e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d6c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.548+0000 7f1727e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d6c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42854/0 (socket says 172.21.15.18:42854) 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.548+0000 7f1727e00640 1 -- 172.21.15.18:0/3700652341 learned_addr learned my addr 172.21.15.18:0/3700652341 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.548+0000 7f1727400640 1 --2- 172.21.15.18:0/3700652341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d5fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.549+0000 7f1727e00640 1 -- 172.21.15.18:0/3700652341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 msgr2=0x55fb00d5fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.549+0000 7f1727e00640 1 --2- 172.21.15.18:0/3700652341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d5fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.549+0000 7f1727e00640 1 -- 172.21.15.18:0/3700652341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 msgr2=0x55fb00d5c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.549+0000 7f1727e00640 1 --2- 172.21.15.18:0/3700652341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 0x55fb00d5c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.549+0000 7f1727e00640 1 -- 172.21.15.18:0/3700652341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb00d63380 con 0x55fb00d606b0 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.550+0000 7f1727e00640 1 --2- 172.21.15.18:0/3700652341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d6c360 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1720009950 tx=0x7f1720010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed83456b547bd957 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.550+0000 7f1713e00640 1 -- 172.21.15.18:0/3700652341 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720013a40 con 0x55fb00d606b0 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.550+0000 7f1713e00640 1 -- 172.21.15.18:0/3700652341 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1720004410 con 0x55fb00d606b0 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.550+0000 7f1713e00640 1 -- 172.21.15.18:0/3700652341 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720010b40 con 0x55fb00d606b0 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.551+0000 7f17311df740 1 -- 172.21.15.18:0/3700652341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 msgr2=0x55fb00d6c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.551+0000 7f17311df740 1 --2- 172.21.15.18:0/3700652341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d6c360 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1720009950 tx=0x7f1720010000 comp rx=0 tx=0).stop 2024-04-16T17:19:32.562 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:30 smithi018 ceph-mon[31776]: pgmap v469: 129 pgs: 129 active+clean; 334 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 400 op/s 2024-04-16T17:19:32.563 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:32 smithi018 ceph-mon[31776]: pgmap v470: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-04-16T17:19:32.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.563+0000 7f17311df740 1 -- 172.21.15.18:0/3700652341 shutdown_connections 2024-04-16T17:19:32.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.563+0000 7f17311df740 1 --2- 172.21.15.18:0/3700652341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d6c360 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.563+0000 7f17311df740 1 --2- 172.21.15.18:0/3700652341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 0x55fb00d5c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.563+0000 7f17311df740 1 --2- 172.21.15.18:0/3700652341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d5fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.563+0000 7f17311df740 1 -- 172.21.15.18:0/3700652341 >> 172.21.15.18:0/3700652341 conn(0x55fb00d518e0 msgr2=0x55fb00d5dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:32.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.563+0000 7f17311df740 1 -- 172.21.15.18:0/3700652341 shutdown_connections 2024-04-16T17:19:32.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.564+0000 7f17311df740 1 -- 172.21.15.18:0/3700652341 wait complete. 2024-04-16T17:19:32.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.564+0000 7f17311df740 1 -- start start 2024-04-16T17:19:32.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.565+0000 7f17311df740 1 Processor -- start 2024-04-16T17:19:32.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f17311df740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f17311df740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 0x55fb00d59f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f17311df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d5a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f1727400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f1727400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d68a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54798/0 (socket says 172.21.15.18:54798) 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f1727400640 1 -- 172.21.15.18:0/1204844081 learned_addr learned my addr 172.21.15.18:0/1204844081 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17281bca50 con 0x55fb00d57ad0 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17281bdec0 con 0x55fb00d571b0 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17281bd120 con 0x55fb00d606b0 2024-04-16T17:19:32.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.566+0000 7f17311df740 10 client.0 _ll_register_callbacks cb 0x7f1720009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:32.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.568+0000 7f1727400640 1 -- 172.21.15.18:0/1204844081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 msgr2=0x55fb00d5a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:32.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.567+0000 7f1726a00640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 0x55fb00d59f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.567+0000 7f1727e00640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d5a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.570+0000 7f1727400640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d5a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.570+0000 7f1727400640 1 -- 172.21.15.18:0/1204844081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 msgr2=0x55fb00d59f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:32.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.570+0000 7f1727400640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 0x55fb00d59f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.570+0000 7f1727400640 1 -- 172.21.15.18:0/1204844081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb00d5a9a0 con 0x55fb00d571b0 2024-04-16T17:19:32.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.570+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1720004410 con 0x55fb00d571b0 2024-04-16T17:19:32.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.571+0000 7f1727e00640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d5a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:32.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.571+0000 7f1726a00640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 0x55fb00d59f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:32.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.571+0000 7f1727400640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d68a60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f17140bddb0 tx=0x7f17140038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfc91457f5bf158 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:32.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.571+0000 7f1711600640 10 client.29549 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:32.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.572+0000 7f1711600640 1 -- 172.21.15.18:0/1204844081 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17140cb060 con 0x55fb00d571b0 2024-04-16T17:19:32.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.572+0000 7f1711600640 1 -- 172.21.15.18:0/1204844081 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f17140d7240 con 0x55fb00d571b0 2024-04-16T17:19:32.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.573+0000 7f1711600640 1 -- 172.21.15.18:0/1204844081 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17140c21d0 con 0x55fb00d571b0 2024-04-16T17:19:32.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.573+0000 7f17311df740 10 client.29549 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:32.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.573+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55fb00dd68b0 con 0x55fb00d571b0 2024-04-16T17:19:32.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.573+0000 7f1711600640 1 -- 172.21.15.18:0/1204844081 <== mon.2 v2:172.21.15.22:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f17140c6ce0 con 0x55fb00d571b0 2024-04-16T17:19:32.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.574+0000 7f17311df740 10 client.29549 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:32.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.574+0000 7f17311df740 10 client.29549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:32.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.574+0000 7f17311df740 20 client.29549 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:32.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.574+0000 7f17311df740 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb00dd6f00 0x55fb00dd9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.574+0000 7f17311df740 4 client.29549 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:19:32.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.574+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb00cc49b0 con 0x55fb00dd6f00 2024-04-16T17:19:32.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.574+0000 7f1726a00640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb00dd6f00 0x55fb00dd9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.576+0000 7f1726a00640 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb00dd6f00 0x55fb00dd9320 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bef207370bce761 server_cookie=312cefe535c03cef in_seq=0 out_seq=0 2024-04-16T17:19:32.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.576+0000 7f1711600640 10 client.29549 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:32.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.576+0000 7f1711600640 1 -- 172.21.15.18:0/1204844081 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb00cc49b0 con 0x55fb00dd6f00 2024-04-16T17:19:32.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.576+0000 7f1711600640 10 client.29549 handle_command_reply: tid=0 2024-04-16T17:19:32.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.576+0000 7f17311df740 1 client.29549 shutdown 2024-04-16T17:19:32.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.577+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 msgr2=0x55fb00d68a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.577+0000 7f17311df740 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d68a60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f17140bddb0 tx=0x7f17140038b0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.577+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 shutdown_connections 2024-04-16T17:19:32.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.577+0000 7f17311df740 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb00dd6f00 0x55fb00dd9320 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.577+0000 7f17311df740 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb00d606b0 0x55fb00d5a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.578+0000 7f17311df740 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb00d57ad0 0x55fb00d59f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.578+0000 7f17311df740 1 --2- 172.21.15.18:0/1204844081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb00d571b0 0x55fb00d68a60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.578+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 >> 172.21.15.18:0/1204844081 conn(0x55fb00d518e0 msgr2=0x55fb00d50040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:32.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.579+0000 7f1711600640 0 client.29549 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:32.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.579+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 shutdown_connections 2024-04-16T17:19:32.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.579+0000 7f17311df740 1 -- 172.21.15.18:0/1204844081 wait complete. 2024-04-16T17:19:32.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.579+0000 7f17311df740 20 client.29549 trim_cache size 0 max 16384 2024-04-16T17:19:32.580 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:19:32.580 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:19:32.580 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:19:32.580 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.580+0000 7f172e400640 1 -- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f170c0a3e70 msgr2=0x7f170c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.580+0000 7f172e400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f170c0a3e70 0x7f170c0a6330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f172806b380 tx=0x7f1720004a40 comp rx=0 tx=0).stop 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.580+0000 7f172e400640 1 -- 172.21.15.18:0/2010481617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1728110750 msgr2=0x7f17281a76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.580+0000 7f172e400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1728110750 0x7f17281a76f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f171402efd0 tx=0x7f17140043c0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.581+0000 7f172e400640 1 -- 172.21.15.18:0/2010481617 shutdown_connections 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.581+0000 7f172e400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f170c0a3e70 0x7f170c0a6330 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.581+0000 7f172e400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f172811b7e0 0x7f17281abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.581+0000 7f172e400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1728110bc0 0x7f17281a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.581+0000 7f172e400640 1 --2- 172.21.15.18:0/2010481617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1728110750 0x7f17281a76f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.581+0000 7f172e400640 1 -- 172.21.15.18:0/2010481617 >> 172.21.15.18:0/2010481617 conn(0x7f172806d0a0 msgr2=0x7f172811a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:32.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.582+0000 7f172e400640 1 -- 172.21.15.18:0/2010481617 shutdown_connections 2024-04-16T17:19:32.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.582+0000 7f172e400640 1 -- 172.21.15.18:0/2010481617 wait complete. 2024-04-16T17:19:32.607 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:19:32.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:19:32.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:32 smithi022 ceph-mon[36023]: pgmap v470: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-04-16T17:19:32.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.947+0000 7f5db2200640 1 -- 172.21.15.18:0/34052115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac10c460 msgr2=0x7f5dac10e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.947+0000 7f5db2200640 1 --2- 172.21.15.18:0/34052115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac10c460 0x7f5dac10e8b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009900 tx=0x7f5d9802bc30 comp rx=0 tx=0).stop 2024-04-16T17:19:32.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.950+0000 7f5db2200640 1 -- 172.21.15.18:0/34052115 shutdown_connections 2024-04-16T17:19:32.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.950+0000 7f5db2200640 1 --2- 172.21.15.18:0/34052115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac10c460 0x7f5dac10e8b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.950+0000 7f5db2200640 1 --2- 172.21.15.18:0/34052115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac103910 0x7f5dac10bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.950+0000 7f5db2200640 1 --2- 172.21.15.18:0/34052115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5dac102ff0 0x7f5dac1033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.950+0000 7f5db2200640 1 -- 172.21.15.18:0/34052115 >> 172.21.15.18:0/34052115 conn(0x7f5dac0fc810 msgr2=0x7f5dac0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:32.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.951+0000 7f5db2200640 1 -- 172.21.15.18:0/34052115 shutdown_connections 2024-04-16T17:19:32.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.952+0000 7f5db2200640 1 -- 172.21.15.18:0/34052115 wait complete. 2024-04-16T17:19:32.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.955+0000 7f5db2200640 1 Processor -- start 2024-04-16T17:19:32.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.955+0000 7f5db2200640 1 -- start start 2024-04-16T17:19:32.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5db2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac102ff0 0x7f5dac1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5db2200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac103910 0x7f5dac1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5db2200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5dac10c460 0x7f5dac19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5db2200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac112260 con 0x7f5dac10c460 2024-04-16T17:19:32.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5db2200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1120e0 con 0x7f5dac102ff0 2024-04-16T17:19:32.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5db2200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1123e0 con 0x7f5dac103910 2024-04-16T17:19:32.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5daaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac103910 0x7f5dac1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5daaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac103910 0x7f5dac1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42866/0 (socket says 172.21.15.18:42866) 2024-04-16T17:19:32.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.956+0000 7f5daaa00640 1 -- 172.21.15.18:0/1653442378 learned_addr learned my addr 172.21.15.18:0/1653442378 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:32.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.957+0000 7f5dabe00640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5dac10c460 0x7f5dac19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.957+0000 7f5dab400640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac102ff0 0x7f5dac1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.957+0000 7f5daaa00640 1 -- 172.21.15.18:0/1653442378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac102ff0 msgr2=0x7f5dac1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.957+0000 7f5daaa00640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac102ff0 0x7f5dac1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.957+0000 7f5daaa00640 1 -- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5dac10c460 msgr2=0x7f5dac19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:32.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.957+0000 7f5daaa00640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5dac10c460 0x7f5dac19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:32.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.957+0000 7f5daaa00640 1 -- 172.21.15.18:0/1653442378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac19b020 con 0x7f5dac103910 2024-04-16T17:19:32.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.958+0000 7f5daaa00640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac103910 0x7f5dac1a0be0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00e9c0 tx=0x7f5d9c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:32.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.958+0000 7f5dabe00640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5dac10c460 0x7f5dac19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:19:32.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.958+0000 7f5dab400640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac102ff0 0x7f5dac1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:32.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.959+0000 7f5da3e00640 1 -- 172.21.15.18:0/1653442378 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00cd80 con 0x7f5dac103910 2024-04-16T17:19:32.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.959+0000 7f5da3e00640 1 -- 172.21.15.18:0/1653442378 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c0035f0 con 0x7f5dac103910 2024-04-16T17:19:32.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.959+0000 7f5da3e00640 1 -- 172.21.15.18:0/1653442378 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c010690 con 0x7f5dac103910 2024-04-16T17:19:32.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.959+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac19b310 con 0x7f5dac103910 2024-04-16T17:19:32.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.960+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a7620 con 0x7f5dac103910 2024-04-16T17:19:32.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.961+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac06ad80 con 0x7f5dac103910 2024-04-16T17:19:32.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.965+0000 7f5da3e00640 1 -- 172.21.15.18:0/1653442378 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d9c0037a0 con 0x7f5dac103910 2024-04-16T17:19:32.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.967+0000 7f5da3e00640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d900a3f20 0x7f5d900a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:32.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.967+0000 7f5da3e00640 1 -- 172.21.15.18:0/1653442378 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f5d9c0d0fc0 con 0x7f5dac103910 2024-04-16T17:19:32.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.968+0000 7f5dab400640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d900a3f20 0x7f5d900a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:32.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.970+0000 7f5dab400640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d900a3f20 0x7f5d900a63e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5da400a7b0 tx=0x7f5da400d040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:32.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:32.971+0000 7f5da3e00640 1 -- 172.21.15.18:0/1653442378 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d9c09d1f0 con 0x7f5dac103910 2024-04-16T17:19:33.348 INFO:tasks.workunit.client.0.smithi018.stdout: 10 27311 15.59 MB/sec warmup 105 sec latency 204.857 ms 2024-04-16T17:19:33.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.408+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dac1a77f0 con 0x7f5dac103910 2024-04-16T17:19:33.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.409+0000 7f5da3e00640 1 -- 172.21.15.18:0/1653442378 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11857 (secure 0 0 0) 0x7f5d9c09caf0 con 0x7f5dac103910 2024-04-16T17:19:33.413 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:19:33.413 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:17:53.822480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:19:33.413 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 34 2024-04-16T17:19:33.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.420+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d900a3f20 msgr2=0x7f5d900a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.420+0000 7f5db2200640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d900a3f20 0x7f5d900a63e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5da400a7b0 tx=0x7f5da400d040 comp rx=0 tx=0).stop 2024-04-16T17:19:33.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.420+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac103910 msgr2=0x7f5dac1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.420+0000 7f5db2200640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac103910 0x7f5dac1a0be0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00e9c0 tx=0x7f5d9c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:19:33.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.421+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 shutdown_connections 2024-04-16T17:19:33.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.421+0000 7f5db2200640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d900a3f20 0x7f5d900a63e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.421+0000 7f5db2200640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5dac10c460 0x7f5dac19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.421+0000 7f5db2200640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac103910 0x7f5dac1a0be0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.421+0000 7f5db2200640 1 --2- 172.21.15.18:0/1653442378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac102ff0 0x7f5dac1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.421+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 >> 172.21.15.18:0/1653442378 conn(0x7f5dac0fc810 msgr2=0x7f5dac100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.421+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 shutdown_connections 2024-04-16T17:19:33.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.421+0000 7f5db2200640 1 -- 172.21.15.18:0/1653442378 wait complete. 2024-04-16T17:19:33.449 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:19:33.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.730+0000 7f1fdc000640 1 -- 172.21.15.18:0/539692266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 msgr2=0x7f1fd4115470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.730+0000 7f1fdc000640 1 --2- 172.21.15.18:0/539692266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 0x7f1fd4115470 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009900 tx=0x7f1fc002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.730+0000 7f1fdc000640 1 -- 172.21.15.18:0/539692266 shutdown_connections 2024-04-16T17:19:33.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.730+0000 7f1fdc000640 1 --2- 172.21.15.18:0/539692266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 0x7f1fd4115470 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.730+0000 7f1fdc000640 1 --2- 172.21.15.18:0/539692266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fd4108720 0x7f1fd4108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.731+0000 7f1fdc000640 1 --2- 172.21.15.18:0/539692266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fd410e6b0 0x7f1fd410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.731+0000 7f1fdc000640 1 -- 172.21.15.18:0/539692266 >> 172.21.15.18:0/539692266 conn(0x7f1fd4077e70 msgr2=0x7f1fd4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:33.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.732+0000 7f1fdc000640 1 -- 172.21.15.18:0/539692266 shutdown_connections 2024-04-16T17:19:33.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.732+0000 7f1fdc000640 1 -- 172.21.15.18:0/539692266 wait complete. 2024-04-16T17:19:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.734+0000 7f1fdc000640 1 Processor -- start 2024-04-16T17:19:33.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.734+0000 7f1fdc000640 1 -- start start 2024-04-16T17:19:33.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.734+0000 7f1fdc000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fd4108720 0x7f1fd419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.735+0000 7f1fdc000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 0x7f1fd419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.735+0000 7f1fdc000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fd410e6b0 0x7f1fd41a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.735+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 0x7f1fd419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.735+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 0x7f1fd419f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42882/0 (socket says 172.21.15.18:42882) 2024-04-16T17:19:33.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.735+0000 7f1fd8c00640 1 -- 172.21.15.18:0/3825725498 learned_addr learned my addr 172.21.15.18:0/3825725498 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:33.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4118a30 con 0x7f1fd4108720 2024-04-16T17:19:33.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41188b0 con 0x7f1fd410e6b0 2024-04-16T17:19:33.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4118bb0 con 0x7f1fd41090c0 2024-04-16T17:19:33.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fda000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fd410e6b0 0x7f1fd41a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fd8c00640 1 -- 172.21.15.18:0/3825725498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fd410e6b0 msgr2=0x7f1fd41a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fd410e6b0 0x7f1fd41a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fd8c00640 1 -- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fd4108720 msgr2=0x7f1fd419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fd4108720 0x7f1fd419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.736+0000 7f1fd8c00640 1 -- 172.21.15.18:0/3825725498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd41a3e60 con 0x7f1fd41090c0 2024-04-16T17:19:33.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.737+0000 7f1fda000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fd410e6b0 0x7f1fd41a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:33.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.737+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 0x7f1fd419f3b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800e9f0 tx=0x7f1fc800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:33.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.737+0000 7f1fd2000640 1 -- 172.21.15.18:0/3825725498 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800cd80 con 0x7f1fd41090c0 2024-04-16T17:19:33.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.737+0000 7f1fd2000640 1 -- 172.21.15.18:0/3825725498 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc8010910 con 0x7f1fd41090c0 2024-04-16T17:19:33.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.737+0000 7f1fd2000640 1 -- 172.21.15.18:0/3825725498 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8018990 con 0x7f1fd41090c0 2024-04-16T17:19:33.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.738+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd41a4150 con 0x7f1fd41090c0 2024-04-16T17:19:33.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.738+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd41ab9e0 con 0x7f1fd41090c0 2024-04-16T17:19:33.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.743+0000 7f1fd2000640 1 -- 172.21.15.18:0/3825725498 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1fc8010ac0 con 0x7f1fd41090c0 2024-04-16T17:19:33.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.744+0000 7f1fdee1c740 1 Processor -- start 2024-04-16T17:19:33.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.746+0000 7f1fd2000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1fbc0a3e60 0x7f1fbc0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.746+0000 7f1fd2000640 1 -- 172.21.15.18:0/3825725498 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f1fc80d1050 con 0x7f1fd41090c0 2024-04-16T17:19:33.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.748+0000 7f1fdee1c740 1 -- start start 2024-04-16T17:19:33.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.749+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6e9d50 0x5600ce6ea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.749+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.749+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 0x5600ce6f5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.749+0000 7f1fda000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1fbc0a3e60 0x7f1fbc0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.750+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5600ce64f020 con 0x5600ce6f3250 2024-04-16T17:19:33.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.750+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5600ce5f5460 con 0x5600ce6ea670 2024-04-16T17:19:33.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.750+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5600ce64afe0 con 0x5600ce6e9d50 2024-04-16T17:19:33.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.751+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.751+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54818/0 (socket says 172.21.15.18:54818) 2024-04-16T17:19:33.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.751+0000 7f1fd8c00640 1 -- 172.21.15.18:0/99115872 learned_addr learned my addr 172.21.15.18:0/99115872 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:33.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.751+0000 7f1fd9600640 1 --2- 172.21.15.18:0/99115872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6e9d50 0x5600ce6ea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.751+0000 7f1fd8c00640 1 -- 172.21.15.18:0/99115872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6e9d50 msgr2=0x5600ce6ea130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:33.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.751+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/99115872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6e9d50 0x5600ce6ea130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.751+0000 7f1fd8c00640 1 -- 172.21.15.18:0/99115872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 msgr2=0x5600ce6f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:33.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.751+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/99115872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 0x5600ce6f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.752+0000 7f1fd8c00640 1 -- 172.21.15.18:0/99115872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ce5fd040 con 0x5600ce6ea670 2024-04-16T17:19:33.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.752+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/99115872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2d10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80a52c0 tx=0x7f1fc809d340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9666ce521580845 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:33.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.753+0000 7f1fc6600640 1 -- 172.21.15.18:0/99115872 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc80a57f0 con 0x5600ce6ea670 2024-04-16T17:19:33.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.753+0000 7f1fda000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1fbc0a3e60 0x7f1fbc0a6320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0036040 tx=0x7f1fc0009640 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:33.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.753+0000 7f1fc6600640 1 -- 172.21.15.18:0/99115872 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc80bd070 con 0x5600ce6ea670 2024-04-16T17:19:33.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.753+0000 7f1fc6600640 1 -- 172.21.15.18:0/99115872 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc80a57f0 con 0x5600ce6ea670 2024-04-16T17:19:33.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.754+0000 7f1fdee1c740 1 -- 172.21.15.18:0/99115872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 msgr2=0x5600ce6f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.754+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/99115872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2d10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80a52c0 tx=0x7f1fc809d340 comp rx=0 tx=0).stop 2024-04-16T17:19:33.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.755+0000 7f1fdee1c740 1 -- 172.21.15.18:0/99115872 shutdown_connections 2024-04-16T17:19:33.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.755+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/99115872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 0x5600ce6f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.755+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/99115872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2d10 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.755+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/99115872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6e9d50 0x5600ce6ea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.755+0000 7f1fdee1c740 1 -- 172.21.15.18:0/99115872 >> 172.21.15.18:0/99115872 conn(0x5600ce6e4480 msgr2=0x5600ce6e4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:33.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.755+0000 7f1fdee1c740 1 -- 172.21.15.18:0/99115872 shutdown_connections 2024-04-16T17:19:33.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.756+0000 7f1fdee1c740 1 -- 172.21.15.18:0/99115872 wait complete. 2024-04-16T17:19:33.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.756+0000 7f1fdee1c740 1 -- start start 2024-04-16T17:19:33.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.757+0000 7f1fdee1c740 1 Processor -- start 2024-04-16T17:19:33.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.758+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6e9d50 0x5600ce6f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.758+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6ea670 0x5600ce6f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.759+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce6f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.759+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5600ce6f6040 con 0x5600ce6ea670 2024-04-16T17:19:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.759+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5600ce608830 con 0x5600ce6e9d50 2024-04-16T17:19:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.759+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5600ce6f61c0 con 0x5600ce6f3250 2024-04-16T17:19:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.759+0000 7f1fda000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce6f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.759+0000 7f1fd9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6e9d50 0x5600ce6f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.759+0000 7f1fda000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce6f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42902/0 (socket says 172.21.15.18:42902) 2024-04-16T17:19:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.759+0000 7f1fda000640 1 -- 172.21.15.18:0/1377817673 learned_addr learned my addr 172.21.15.18:0/1377817673 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.760+0000 7f1fdee1c740 10 client.0 _ll_register_callbacks cb 0x5600ce610150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.760+0000 7f1fd9600640 1 -- 172.21.15.18:0/1377817673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 msgr2=0x5600ce6f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.760+0000 7f1fd9600640 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce6f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.760+0000 7f1fd9600640 1 -- 172.21.15.18:0/1377817673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6ea670 msgr2=0x5600ce6f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.760+0000 7f1fd9600640 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6ea670 0x5600ce6f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.760+0000 7f1fd9600640 1 -- 172.21.15.18:0/1377817673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ce60c920 con 0x5600ce6e9d50 2024-04-16T17:19:33.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.761+0000 7f1fd9600640 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6e9d50 0x5600ce6f02d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc010800 tx=0x7f1fcc010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72beae1cbe38e954 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:33.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.761+0000 7f1fbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:33.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.763+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1377817673 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc017070 con 0x5600ce6e9d50 2024-04-16T17:19:33.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.763+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1377817673 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f1fcc004d20 con 0x5600ce6e9d50 2024-04-16T17:19:33.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.763+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1377817673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fc80bd070 con 0x5600ce6e9d50 2024-04-16T17:19:33.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.764+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1377817673 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1fcc023050 con 0x5600ce6e9d50 2024-04-16T17:19:33.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.764+0000 7f1fdee1c740 10 client.29564 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:33.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.764+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1377817673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5600ce5da7f0 con 0x5600ce6e9d50 2024-04-16T17:19:33.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.764+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1377817673 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc80bd070 con 0x5600ce6e9d50 2024-04-16T17:19:33.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.765+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1377817673 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1fcc004300 con 0x5600ce6e9d50 2024-04-16T17:19:33.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.765+0000 7f1fdee1c740 10 client.29564 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:33.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.765+0000 7f1fdee1c740 10 client.29564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:19:33.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.765+0000 7f1fdee1c740 20 client.29564 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:33.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.765+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5600ce703700 0x5600ce705b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.765+0000 7f1fdee1c740 4 client.29564 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:33.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.765+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1377817673 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600ce65ca40 con 0x5600ce703700 2024-04-16T17:19:33.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.766+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5600ce703700 0x5600ce705b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.767+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5600ce703700 0x5600ce705b70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c735c6e01a6c9b2 server_cookie=12414800aeb76605 in_seq=0 out_seq=0 2024-04-16T17:19:33.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.767+0000 7f1fbaa00640 10 client.29564 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:19:33.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.768+0000 7f1fbaa00640 1 -- 172.21.15.18:0/1377817673 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600ce65ca40 con 0x5600ce703700 2024-04-16T17:19:33.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.768+0000 7f1fbaa00640 10 client.29564 handle_command_reply: tid=0 2024-04-16T17:19:33.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.768+0000 7f1fdee1c740 1 client.29564 shutdown 2024-04-16T17:19:33.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.769+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1377817673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6e9d50 msgr2=0x5600ce6f02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.769+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6e9d50 0x5600ce6f02d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc010800 tx=0x7f1fcc010cc0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.769+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1377817673 shutdown_connections 2024-04-16T17:19:33.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.769+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5600ce703700 0x5600ce705b70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.770+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce6f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.770+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6ea670 0x5600ce6f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.770+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1377817673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6e9d50 0x5600ce6f02d0 secure :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc010800 tx=0x7f1fcc010cc0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.770+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1377817673 >> 172.21.15.18:0/1377817673 conn(0x5600ce6e4480 msgr2=0x5600ce6e61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:33.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.770+0000 7f1fbaa00640 0 client.29564 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:19:33.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.770+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1377817673 shutdown_connections 2024-04-16T17:19:33.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.770+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1377817673 wait complete. 2024-04-16T17:19:33.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.770+0000 7f1fdee1c740 20 client.29564 trim_cache size 0 max 16384 2024-04-16T17:19:33.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.800+0000 7f1fdee1c740 1 Processor -- start 2024-04-16T17:19:33.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.801+0000 7f1fdee1c740 1 -- start start 2024-04-16T17:19:33.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.801+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.801+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce70b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.801+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce70b760 0x5600ce6ed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.801+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41b8640 con 0x5600ce70b760 2024-04-16T17:19:33.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.801+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41b5eb0 con 0x5600ce6ea670 2024-04-16T17:19:33.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.802+0000 7f1fdee1c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41b3d20 con 0x5600ce6f3250 2024-04-16T17:19:33.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.802+0000 7f1fd9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.802+0000 7f1fda000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce70b760 0x5600ce6ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.802+0000 7f1fda000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce70b760 0x5600ce6ed530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46714/0 (socket says 172.21.15.18:46714) 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.802+0000 7f1fda000640 1 -- 172.21.15.18:0/1001500416 learned_addr learned my addr 172.21.15.18:0/1001500416 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.802+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce70b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.803+0000 7f1fd9600640 1 -- 172.21.15.18:0/1001500416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 msgr2=0x5600ce70b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.803+0000 7f1fd9600640 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce70b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.803+0000 7f1fd9600640 1 -- 172.21.15.18:0/1001500416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce70b760 msgr2=0x5600ce6ed530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.803+0000 7f1fd9600640 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce70b760 0x5600ce6ed530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.803+0000 7f1fd9600640 1 -- 172.21.15.18:0/1001500416 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ce3613f0 con 0x5600ce6ea670 2024-04-16T17:19:33.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.803+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce70b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:33.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.804+0000 7f1fd9600640 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2a00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc002230 tx=0x7f1fcc009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=251bb6c26888c370 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:33.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.804+0000 7f1fc5c00640 1 -- 172.21.15.18:0/1001500416 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc02b070 con 0x5600ce6ea670 2024-04-16T17:19:33.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.805+0000 7f1fc5c00640 1 -- 172.21.15.18:0/1001500416 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fcc020040 con 0x5600ce6ea670 2024-04-16T17:19:33.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.805+0000 7f1fc5c00640 1 -- 172.21.15.18:0/1001500416 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc017050 con 0x5600ce6ea670 2024-04-16T17:19:33.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.806+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1001500416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 msgr2=0x5600ce6f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.806+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2a00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc002230 tx=0x7f1fcc009fd0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.806+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1001500416 shutdown_connections 2024-04-16T17:19:33.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.806+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce70b760 0x5600ce6ed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.806+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce6f3250 0x5600ce70b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.806+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/1001500416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce6f2a00 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.806+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1001500416 >> 172.21.15.18:0/1001500416 conn(0x5600ce6e4480 msgr2=0x5600ce6e34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:33.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.807+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1001500416 shutdown_connections 2024-04-16T17:19:33.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.807+0000 7f1fdee1c740 1 -- 172.21.15.18:0/1001500416 wait complete. 2024-04-16T17:19:33.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.808+0000 7f1fdee1c740 1 -- start start 2024-04-16T17:19:33.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.808+0000 7f1fdee1c740 1 Processor -- start 2024-04-16T17:19:33.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.809+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce763310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.809+0000 7f1fdee1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 0x5600ce7061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.809+0000 7f1fd9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce763310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.809+0000 7f1fd9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce763310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54854/0 (socket says 172.21.15.18:54854) 2024-04-16T17:19:33.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.810+0000 7f1fd9600640 1 -- 172.21.15.18:0/2000736001 learned_addr learned my addr 172.21.15.18:0/2000736001 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:33.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.810+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce70b760 0x5600ce706730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.810+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41b4370 con 0x5600ce6f3250 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.810+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41b57e0 con 0x5600ce6ea670 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.810+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41b4a40 con 0x5600ce70b760 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.810+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 0x5600ce7061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.810+0000 7f1fdee1c740 10 client.0 _ll_register_callbacks cb 0x7f1fcc0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.810+0000 7f1fda000640 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce70b760 0x5600ce706730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.811+0000 7f1fd9600640 1 -- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce70b760 msgr2=0x5600ce706730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.811+0000 7f1fd9600640 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce70b760 0x5600ce706730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.811+0000 7f1fd9600640 1 -- 172.21.15.18:0/2000736001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 msgr2=0x5600ce7061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:33.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.811+0000 7f1fd9600640 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 0x5600ce7061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.811+0000 7f1fd9600640 1 -- 172.21.15.18:0/2000736001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ce42b7d0 con 0x5600ce6ea670 2024-04-16T17:19:33.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.811+0000 7f1fda000640 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce70b760 0x5600ce706730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:33.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.811+0000 7f1fd9600640 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce763310 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc000f80 tx=0x7f1fcc0104b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bfae30ab8b50301 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:33.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.811+0000 7f1fbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:33.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.812+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2000736001 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc02b070 con 0x5600ce6ea670 2024-04-16T17:19:33.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.812+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2000736001 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f1fcc004d30 con 0x5600ce6ea670 2024-04-16T17:19:33.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.812+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fcc020040 con 0x5600ce6ea670 2024-04-16T17:19:33.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.813+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2000736001 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1fcc017040 con 0x5600ce6ea670 2024-04-16T17:19:33.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.813+0000 7f1fdee1c740 10 client.29570 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:33.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.813+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5600ce77b830 con 0x5600ce6ea670 2024-04-16T17:19:33.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.813+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2000736001 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fcc020040 con 0x5600ce6ea670 2024-04-16T17:19:33.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.814+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2000736001 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1fcc030450 con 0x5600ce6ea670 2024-04-16T17:19:33.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.814+0000 7f1fdee1c740 10 client.29570 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:33.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.814+0000 7f1fdee1c740 10 client.29570 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:19:33.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.814+0000 7f1fdee1c740 20 client.29570 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:33.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.814+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5600ce77be30 0x5600ce77e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:33.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.815+0000 7f1fdee1c740 4 client.29570 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:19:33.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.816+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600ce6568b0 con 0x5600ce77be30 2024-04-16T17:19:33.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.816+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5600ce77be30 0x5600ce77e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:33.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.816+0000 7f1fd8c00640 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5600ce77be30 0x5600ce77e220 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcdefce7b3e2a1fd server_cookie=cbe6291db7955caa in_seq=0 out_seq=0 2024-04-16T17:19:33.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.816+0000 7f1fbaa00640 10 client.29570 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:19:33.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fbaa00640 1 -- 172.21.15.18:0/2000736001 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600ce6568b0 con 0x5600ce77be30 2024-04-16T17:19:33.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fbaa00640 10 client.29570 handle_command_reply: tid=0 2024-04-16T17:19:33.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 client.29570 shutdown 2024-04-16T17:19:33.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 msgr2=0x5600ce763310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce763310 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc000f80 tx=0x7f1fcc0104b0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 shutdown_connections 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5600ce77be30 0x5600ce77e220 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ce70b760 0x5600ce706730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5600ce6f3250 0x5600ce7061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 --2- 172.21.15.18:0/2000736001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600ce6ea670 0x5600ce763310 secure :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc000f80 tx=0x7f1fcc0104b0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.817+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 >> 172.21.15.18:0/2000736001 conn(0x5600ce6e4480 msgr2=0x5600ce6f0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.818+0000 7f1fbaa00640 0 client.29570 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.818+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 shutdown_connections 2024-04-16T17:19:33.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.818+0000 7f1fdee1c740 1 -- 172.21.15.18:0/2000736001 wait complete. 2024-04-16T17:19:33.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.819+0000 7f1fdee1c740 20 client.29570 trim_cache size 0 max 16384 2024-04-16T17:19:33.819 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.819+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1fbc0a3e60 msgr2=0x7f1fbc0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.819+0000 7f1fdc000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1fbc0a3e60 0x7f1fbc0a6320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0036040 tx=0x7f1fc0009640 comp rx=0 tx=0).stop 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 msgr2=0x7f1fd419f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 0x7f1fd419f3b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800e9f0 tx=0x7f1fc800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 shutdown_connections 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1fbc0a3e60 0x7f1fbc0a6320 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fd410e6b0 0x7f1fd41a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fd41090c0 0x7f1fd419f3b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 --2- 172.21.15.18:0/3825725498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fd4108720 0x7f1fd419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:33.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 >> 172.21.15.18:0/3825725498 conn(0x7f1fd4077e70 msgr2=0x7f1fd4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:33.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 shutdown_connections 2024-04-16T17:19:33.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:33.820+0000 7f1fdc000640 1 -- 172.21.15.18:0/3825725498 wait complete. 2024-04-16T17:19:33.846 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:19:34.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.136+0000 7f69ea200640 1 -- 172.21.15.18:0/3341828823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e406dc60 msgr2=0x7f69e41159a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.136+0000 7f69ea200640 1 --2- 172.21.15.18:0/3341828823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e406dc60 0x7f69e41159a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009ca0 tx=0x7f69dc02c140 comp rx=0 tx=0).stop 2024-04-16T17:19:34.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.137+0000 7f69ea200640 1 -- 172.21.15.18:0/3341828823 shutdown_connections 2024-04-16T17:19:34.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.137+0000 7f69ea200640 1 --2- 172.21.15.18:0/3341828823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e406dc60 0x7f69e41159a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.137+0000 7f69ea200640 1 --2- 172.21.15.18:0/3341828823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e4109c20 0x7f69e410d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.137+0000 7f69ea200640 1 --2- 172.21.15.18:0/3341828823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4109270 0x7f69e4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.137+0000 7f69ea200640 1 -- 172.21.15.18:0/3341828823 >> 172.21.15.18:0/3341828823 conn(0x7f69e406d0a0 msgr2=0x7f69e406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.137+0000 7f69ea200640 1 -- 172.21.15.18:0/3341828823 shutdown_connections 2024-04-16T17:19:34.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.137+0000 7f69ea200640 1 -- 172.21.15.18:0/3341828823 wait complete. 2024-04-16T17:19:34.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.141+0000 7f69ea200640 1 Processor -- start 2024-04-16T17:19:34.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.160+0000 7f69ea200640 1 -- start start 2024-04-16T17:19:34.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.160+0000 7f69ea200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e406dc60 0x7f69e41ac450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69ea200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e4109270 0x7f69e41ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69ea200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4112970 0x7f69e4112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69ea200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f69e411acb0 con 0x7f69e4112970 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69ea200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f69e411ab30 con 0x7f69e406dc60 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69ea200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69e411ae30 con 0x7f69e4109270 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e406dc60 0x7f69e41ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e406dc60 0x7f69e41ac450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54866/0 (socket says 172.21.15.18:54866) 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69e3400640 1 -- 172.21.15.18:0/455062789 learned_addr learned my addr 172.21.15.18:0/455062789 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.161+0000 7f69e2a00640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e4109270 0x7f69e41ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.162+0000 7f69e2a00640 1 -- 172.21.15.18:0/455062789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e406dc60 msgr2=0x7f69e41ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.162+0000 7f69e2a00640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e406dc60 0x7f69e41ac450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.162+0000 7f69e2a00640 1 -- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4112970 msgr2=0x7f69e4112e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.162+0000 7f69e2a00640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4112970 0x7f69e4112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.162+0000 7f69e2a00640 1 -- 172.21.15.18:0/455062789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e4113550 con 0x7f69e4109270 2024-04-16T17:19:34.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.163+0000 7f69e2a00640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e4109270 0x7f69e41ac990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f69d000e9c0 tx=0x7f69d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.165+0000 7f69dbe00640 1 -- 172.21.15.18:0/455062789 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d000cd80 con 0x7f69e4109270 2024-04-16T17:19:34.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.165+0000 7f69ea200640 1 -- 172.21.15.18:0/455062789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e41b4370 con 0x7f69e4109270 2024-04-16T17:19:34.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.165+0000 7f69ea200640 1 -- 172.21.15.18:0/455062789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e41b4720 con 0x7f69e4109270 2024-04-16T17:19:34.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.169+0000 7f69dbe00640 1 -- 172.21.15.18:0/455062789 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d00035f0 con 0x7f69e4109270 2024-04-16T17:19:34.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.169+0000 7f69dbe00640 1 -- 172.21.15.18:0/455062789 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0010670 con 0x7f69e4109270 2024-04-16T17:19:34.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.175+0000 7f69ecfb9740 1 Processor -- start 2024-04-16T17:19:34.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.175+0000 7f69ecfb9740 1 -- start start 2024-04-16T17:19:34.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.176+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 0x5565fc265130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.176+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc26dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.176+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 0x5565fc270640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.176+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5565fc1ca020 con 0x5565fc264d50 2024-04-16T17:19:34.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.176+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5565fc170460 con 0x5565fc265670 2024-04-16T17:19:34.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.177+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565fc1c5fe0 con 0x5565fc26e250 2024-04-16T17:19:34.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.177+0000 7f69e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc26dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.177+0000 7f69e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc26dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54876/0 (socket says 172.21.15.18:54876) 2024-04-16T17:19:34.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.177+0000 7f69e3400640 1 -- 172.21.15.18:0/3480124407 learned_addr learned my addr 172.21.15.18:0/3480124407 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.178+0000 7f69e3400640 1 -- 172.21.15.18:0/3480124407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 msgr2=0x5565fc270640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:19:34.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.178+0000 7f69e3400640 1 --2- 172.21.15.18:0/3480124407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 0x5565fc270640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.178+0000 7f69e3400640 1 -- 172.21.15.18:0/3480124407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 msgr2=0x5565fc265130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.178+0000 7f69e3400640 1 --2- 172.21.15.18:0/3480124407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 0x5565fc265130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.178+0000 7f69e3400640 1 -- 172.21.15.18:0/3480124407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565fc178040 con 0x5565fc265670 2024-04-16T17:19:34.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.179+0000 7f69e3400640 1 --2- 172.21.15.18:0/3480124407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc26dd10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009b20 tx=0x7f69d400e1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a786510a284f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.181+0000 7f69cfe00640 1 -- 172.21.15.18:0/3480124407 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4017070 con 0x5565fc265670 2024-04-16T17:19:34.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.181+0000 7f69cfe00640 1 -- 172.21.15.18:0/3480124407 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d400ec10 con 0x5565fc265670 2024-04-16T17:19:34.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.181+0000 7f69cfe00640 1 -- 172.21.15.18:0/3480124407 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4012610 con 0x5565fc265670 2024-04-16T17:19:34.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.181+0000 7f69dbe00640 1 -- 172.21.15.18:0/455062789 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f69d0003fa0 con 0x7f69e4109270 2024-04-16T17:19:34.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.182+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3480124407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 msgr2=0x5565fc26dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.182+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3480124407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc26dd10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009b20 tx=0x7f69d400e1d0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.185+0000 7f69dbe00640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69c80a3ea0 0x7f69c80a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.186+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3480124407 shutdown_connections 2024-04-16T17:19:34.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.186+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3480124407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 0x5565fc270640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.186+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3480124407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc26dd10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.186+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3480124407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 0x5565fc265130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.186+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3480124407 >> 172.21.15.18:0/3480124407 conn(0x5565fc25f480 msgr2=0x5565fc25f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.186+0000 7f69dbe00640 1 -- 172.21.15.18:0/455062789 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f69d009a970 con 0x7f69e4109270 2024-04-16T17:19:34.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.187+0000 7f69e3e00640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69c80a3ea0 0x7f69c80a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.187+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3480124407 shutdown_connections 2024-04-16T17:19:34.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.187+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3480124407 wait complete. 2024-04-16T17:19:34.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.187+0000 7f69ecfb9740 1 -- start start 2024-04-16T17:19:34.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.188+0000 7f69ecfb9740 1 Processor -- start 2024-04-16T17:19:34.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 0x5565fc26b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc26bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5565fc271040 con 0x5565fc265670 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5565fc183830 con 0x5565fc26e250 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565fc2711c0 con 0x5565fc264d50 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69ecfb9740 10 client.0 _ll_register_callbacks cb 0x5565fc18b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69e3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69e3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42944/0 (socket says 172.21.15.18:42944) 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.189+0000 7f69e3400640 1 -- 172.21.15.18:0/3811454428 learned_addr learned my addr 172.21.15.18:0/3811454428 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.190+0000 7f69e3400640 1 -- 172.21.15.18:0/3811454428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 msgr2=0x5565fc26bd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.190+0000 7f69e3400640 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc26bd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.190+0000 7f69e3400640 1 -- 172.21.15.18:0/3811454428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 msgr2=0x5565fc26b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.190+0000 7f69e3400640 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 0x5565fc26b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.190+0000 7f69e3400640 1 -- 172.21.15.18:0/3811454428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565fc187920 con 0x5565fc264d50 2024-04-16T17:19:34.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.192+0000 7f69e3400640 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26b280 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009890 tx=0x7f69d4013c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30d054ecb7ae473b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.192+0000 7f69ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:19:34.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.193+0000 7f69ccc00640 1 -- 172.21.15.18:0/3811454428 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4017050 con 0x5565fc264d50 2024-04-16T17:19:34.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.193+0000 7f69ccc00640 1 -- 172.21.15.18:0/3811454428 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f69d401c920 con 0x5565fc264d50 2024-04-16T17:19:34.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.194+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3811454428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d400ec10 con 0x5565fc264d50 2024-04-16T17:19:34.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.194+0000 7f69e3e00640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69c80a3ea0 0x7f69c80a6360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f69dc02c750 tx=0x7f69dc009800 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.194+0000 7f69ccc00640 1 -- 172.21.15.18:0/3811454428 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69d401cb20 con 0x5565fc264d50 2024-04-16T17:19:34.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.194+0000 7f69ecfb9740 10 client.29410 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:34.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.194+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3811454428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5565fc1557f0 con 0x5565fc264d50 2024-04-16T17:19:34.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.195+0000 7f69ccc00640 1 -- 172.21.15.18:0/3811454428 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d400ec10 con 0x5565fc264d50 2024-04-16T17:19:34.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.196+0000 7f69ccc00640 1 -- 172.21.15.18:0/3811454428 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f69d4030420 con 0x5565fc264d50 2024-04-16T17:19:34.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.197+0000 7f69ecfb9740 10 client.29410 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:34.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.197+0000 7f69ecfb9740 10 client.29410 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:19:34.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.197+0000 7f69ecfb9740 20 client.29410 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:34.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.197+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5565fc27e950 0x5565fc280df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.197+0000 7f69ecfb9740 4 client.29410 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:34.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.197+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3811454428 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565fc1d7a40 con 0x5565fc27e950 2024-04-16T17:19:34.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.197+0000 7f69e2a00640 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5565fc27e950 0x5565fc280df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.199+0000 7f69e2a00640 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5565fc27e950 0x5565fc280df0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9e1f54e3bdb9496 server_cookie=e7ba891d70197adc in_seq=0 out_seq=0 2024-04-16T17:19:34.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.199+0000 7f69ccc00640 10 client.29410 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:19:34.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.202+0000 7f69ccc00640 1 -- 172.21.15.18:0/3811454428 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565fc1d7a40 con 0x5565fc27e950 2024-04-16T17:19:34.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.202+0000 7f69ccc00640 10 client.29410 handle_command_reply: tid=0 2024-04-16T17:19:34.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.203+0000 7f69ecfb9740 1 client.29410 shutdown 2024-04-16T17:19:34.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3811454428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 msgr2=0x5565fc26b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26b280 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009890 tx=0x7f69d4013c60 comp rx=0 tx=0).stop 2024-04-16T17:19:34.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3811454428 shutdown_connections 2024-04-16T17:19:34.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5565fc27e950 0x5565fc280df0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc26bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 0x5565fc26b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/3811454428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26b280 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3811454428 >> 172.21.15.18:0/3811454428 conn(0x5565fc25f480 msgr2=0x5565fc2611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ccc00640 0 client.29410 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:19:34.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3811454428 shutdown_connections 2024-04-16T17:19:34.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.204+0000 7f69ecfb9740 1 -- 172.21.15.18:0/3811454428 wait complete. 2024-04-16T17:19:34.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.205+0000 7f69ecfb9740 20 client.29410 trim_cache size 0 max 16384 2024-04-16T17:19:34.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.234+0000 7f69ecfb9740 1 Processor -- start 2024-04-16T17:19:34.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.235+0000 7f69ecfb9740 1 -- start start 2024-04-16T17:19:34.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.235+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.235+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 0x5565fc279f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.235+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc286700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.235+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f69c40129f0 con 0x5565fc265670 2024-04-16T17:19:34.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.235+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4010260 con 0x5565fc26e250 2024-04-16T17:19:34.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.235+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69c400e090 con 0x5565fc264d50 2024-04-16T17:19:34.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.235+0000 7f69e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc286700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.236+0000 7f69e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc286700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54894/0 (socket says 172.21.15.18:54894) 2024-04-16T17:19:34.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.236+0000 7f69e3e00640 1 -- 172.21.15.18:0/167943501 learned_addr learned my addr 172.21.15.18:0/167943501 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.236+0000 7f69e2a00640 1 --2- 172.21.15.18:0/167943501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 0x5565fc279f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.236+0000 7f69e3e00640 1 -- 172.21.15.18:0/167943501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 msgr2=0x5565fc26d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.236+0000 7f69e3e00640 1 --2- 172.21.15.18:0/167943501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.236+0000 7f69e3e00640 1 -- 172.21.15.18:0/167943501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 msgr2=0x5565fc279f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.236+0000 7f69e3e00640 1 --2- 172.21.15.18:0/167943501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 0x5565fc279f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.236+0000 7f69e3e00640 1 -- 172.21.15.18:0/167943501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565fbedc3f0 con 0x5565fc26e250 2024-04-16T17:19:34.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.237+0000 7f69e3e00640 1 --2- 172.21.15.18:0/167943501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc286700 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009910 tx=0x7f69dc03b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6178955ee1820f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.237+0000 7f69cf400640 1 -- 172.21.15.18:0/167943501 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc04d020 con 0x5565fc26e250 2024-04-16T17:19:34.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.237+0000 7f69cf400640 1 -- 172.21.15.18:0/167943501 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc034920 con 0x5565fc26e250 2024-04-16T17:19:34.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.238+0000 7f69cf400640 1 -- 172.21.15.18:0/167943501 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc045040 con 0x5565fc26e250 2024-04-16T17:19:34.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.239+0000 7f69ecfb9740 1 -- 172.21.15.18:0/167943501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 msgr2=0x5565fc286700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.239+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/167943501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc286700 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009910 tx=0x7f69dc03b000 comp rx=0 tx=0).stop 2024-04-16T17:19:34.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.239+0000 7f69ecfb9740 1 -- 172.21.15.18:0/167943501 shutdown_connections 2024-04-16T17:19:34.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.239+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/167943501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc26e250 0x5565fc286700 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.239+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/167943501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc265670 0x5565fc279f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.239+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/167943501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc264d50 0x5565fc26d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.239+0000 7f69ecfb9740 1 -- 172.21.15.18:0/167943501 >> 172.21.15.18:0/167943501 conn(0x5565fc25f480 msgr2=0x5565fc2665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.241+0000 7f69ecfb9740 1 -- 172.21.15.18:0/167943501 shutdown_connections 2024-04-16T17:19:34.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.241+0000 7f69ecfb9740 1 -- 172.21.15.18:0/167943501 wait complete. 2024-04-16T17:19:34.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.242+0000 7f69ecfb9740 1 -- start start 2024-04-16T17:19:34.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.242+0000 7f69ecfb9740 1 Processor -- start 2024-04-16T17:19:34.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.243+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 0x5565fc276980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.243+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc271cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.243+0000 7f69ecfb9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 0x5565fc272300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.243+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f69c400e720 con 0x5565fc264d50 2024-04-16T17:19:34.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.243+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f69c400fb90 con 0x5565fc265670 2024-04-16T17:19:34.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.243+0000 7f69ecfb9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69c400edf0 con 0x5565fc26e250 2024-04-16T17:19:34.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.243+0000 7f69ecfb9740 10 client.0 _ll_register_callbacks cb 0x7f69d40099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:34.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.244+0000 7f69e2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc271cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.244+0000 7f69e3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 0x5565fc276980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.244+0000 7f69e3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 0x5565fc276980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46792/0 (socket says 172.21.15.18:46792) 2024-04-16T17:19:34.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.244+0000 7f69e3400640 1 -- 172.21.15.18:0/870292858 learned_addr learned my addr 172.21.15.18:0/870292858 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.244+0000 7f69e3e00640 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 0x5565fc272300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.245+0000 7f69e2a00640 1 -- 172.21.15.18:0/870292858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 msgr2=0x5565fc272300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.245+0000 7f69e2a00640 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 0x5565fc272300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.245+0000 7f69e2a00640 1 -- 172.21.15.18:0/870292858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 msgr2=0x5565fc276980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.245+0000 7f69e2a00640 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 0x5565fc276980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.245+0000 7f69e2a00640 1 -- 172.21.15.18:0/870292858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565fbfa67d0 con 0x5565fc265670 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.245+0000 7f69e2a00640 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc271cf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f69d00ce080 tx=0x7f69d00a3f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc50337d798ae921 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.245+0000 7f69ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.245+0000 7f69ccc00640 1 -- 172.21.15.18:0/870292858 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0099300 con 0x5565fc265670 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.246+0000 7f69ccc00640 1 -- 172.21.15.18:0/870292858 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f69d0009280 con 0x5565fc265670 2024-04-16T17:19:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.246+0000 7f69ecfb9740 1 -- 172.21.15.18:0/870292858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69dc034920 con 0x5565fc265670 2024-04-16T17:19:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.246+0000 7f69ccc00640 1 -- 172.21.15.18:0/870292858 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69d00994a0 con 0x5565fc265670 2024-04-16T17:19:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.246+0000 7f69ecfb9740 10 client.29585 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.246+0000 7f69ecfb9740 1 -- 172.21.15.18:0/870292858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5565fc2e4f40 con 0x5565fc265670 2024-04-16T17:19:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.246+0000 7f69ccc00640 1 -- 172.21.15.18:0/870292858 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc034920 con 0x5565fc265670 2024-04-16T17:19:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.247+0000 7f69ccc00640 1 -- 172.21.15.18:0/870292858 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f69d00ce300 con 0x5565fc265670 2024-04-16T17:19:34.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.247+0000 7f69ecfb9740 10 client.29585 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:34.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.247+0000 7f69ecfb9740 10 client.29585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:19:34.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.247+0000 7f69ecfb9740 20 client.29585 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:34.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.247+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5565fc2e56c0 0x5565fc2e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.247+0000 7f69ecfb9740 4 client.29585 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:19:34.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.247+0000 7f69ecfb9740 1 -- 172.21.15.18:0/870292858 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565fc1d18b0 con 0x5565fc2e56c0 2024-04-16T17:19:34.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.248+0000 7f69e3400640 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5565fc2e56c0 0x5565fc2e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.249+0000 7f69e3400640 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5565fc2e56c0 0x5565fc2e7ae0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd55c84cc04e6eb9 server_cookie=985f2f1d40a21ede in_seq=0 out_seq=0 2024-04-16T17:19:34.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.249+0000 7f69ccc00640 10 client.29585 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:19:34.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.251+0000 7f69ccc00640 1 -- 172.21.15.18:0/870292858 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565fc1d18b0 con 0x5565fc2e56c0 2024-04-16T17:19:34.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.251+0000 7f69ccc00640 10 client.29585 handle_command_reply: tid=0 2024-04-16T17:19:34.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.251+0000 7f69ecfb9740 1 client.29585 shutdown 2024-04-16T17:19:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ecfb9740 1 -- 172.21.15.18:0/870292858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 msgr2=0x5565fc271cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc271cf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f69d00ce080 tx=0x7f69d00a3f90 comp rx=0 tx=0).stop 2024-04-16T17:19:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ecfb9740 1 -- 172.21.15.18:0/870292858 shutdown_connections 2024-04-16T17:19:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5565fc2e56c0 0x5565fc2e7ae0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565fc26e250 0x5565fc272300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565fc265670 0x5565fc271cf0 secure :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f69d00ce080 tx=0x7f69d00a3f90 comp rx=0 tx=0).stop 2024-04-16T17:19:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ccc00640 0 client.29585 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:19:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ecfb9740 1 --2- 172.21.15.18:0/870292858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565fc264d50 0x5565fc276980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.252+0000 7f69ecfb9740 1 -- 172.21.15.18:0/870292858 >> 172.21.15.18:0/870292858 conn(0x5565fc25f480 msgr2=0x5565fc26b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.253+0000 7f69ecfb9740 1 -- 172.21.15.18:0/870292858 shutdown_connections 2024-04-16T17:19:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.253+0000 7f69ecfb9740 1 -- 172.21.15.18:0/870292858 wait complete. 2024-04-16T17:19:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.253+0000 7f69ecfb9740 20 client.29585 trim_cache size 0 max 16384 2024-04-16T17:19:34.254 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:19:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.254+0000 7f69ea200640 1 -- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69c80a3ea0 msgr2=0x7f69c80a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.254+0000 7f69ea200640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69c80a3ea0 0x7f69c80a6360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f69dc02c750 tx=0x7f69dc009800 comp rx=0 tx=0).stop 2024-04-16T17:19:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.254+0000 7f69ea200640 1 -- 172.21.15.18:0/455062789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e4109270 msgr2=0x7f69e41ac990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.254+0000 7f69ea200640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e4109270 0x7f69e41ac990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f69d000e9c0 tx=0x7f69d000ee80 comp rx=0 tx=0).stop 2024-04-16T17:19:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.255+0000 7f69ea200640 1 -- 172.21.15.18:0/455062789 shutdown_connections 2024-04-16T17:19:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.255+0000 7f69ea200640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f69c80a3ea0 0x7f69c80a6360 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.255+0000 7f69ea200640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69e4112970 0x7f69e4112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.255+0000 7f69ea200640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e4109270 0x7f69e41ac990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.255+0000 7f69ea200640 1 --2- 172.21.15.18:0/455062789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e406dc60 0x7f69e41ac450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.255+0000 7f69ea200640 1 -- 172.21.15.18:0/455062789 >> 172.21.15.18:0/455062789 conn(0x7f69e406d0a0 msgr2=0x7f69e4071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.255+0000 7f69ea200640 1 -- 172.21.15.18:0/455062789 shutdown_connections 2024-04-16T17:19:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.255+0000 7f69ea200640 1 -- 172.21.15.18:0/455062789 wait complete. 2024-04-16T17:19:34.276 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:19:34.348 INFO:tasks.workunit.client.0.smithi018.stdout: 10 27650 15.67 MB/sec warmup 106 sec latency 160.051 ms 2024-04-16T17:19:34.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:34 smithi043 ceph-mon[36901]: pgmap v471: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-04-16T17:19:34.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:34 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1653442378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:19:34.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.521+0000 7f3fffc00640 1 -- 172.21.15.18:0/1997197714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 msgr2=0x7f3ff8101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.521+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1997197714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 0x7f3ff8101260 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009900 tx=0x7f3fec02bc40 comp rx=0 tx=0).stop 2024-04-16T17:19:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.522+0000 7f3fffc00640 1 -- 172.21.15.18:0/1997197714 shutdown_connections 2024-04-16T17:19:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.522+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1997197714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff810e9a0 0x7f3ff8110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.522+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1997197714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff8101830 0x7f3ff810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.522+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1997197714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 0x7f3ff8101260 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.522+0000 7f3fffc00640 1 -- 172.21.15.18:0/1997197714 >> 172.21.15.18:0/1997197714 conn(0x7f3ff8078020 msgr2=0x7f3ff8078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.522+0000 7f3fffc00640 1 -- 172.21.15.18:0/1997197714 shutdown_connections 2024-04-16T17:19:34.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.522+0000 7f3fffc00640 1 -- 172.21.15.18:0/1997197714 wait complete. 2024-04-16T17:19:34.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.523+0000 7f3fffc00640 1 Processor -- start 2024-04-16T17:19:34.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.524+0000 7f3fffc00640 1 -- start start 2024-04-16T17:19:34.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.524+0000 7f3fffc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 0x7f3ff810e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.524+0000 7f3fffc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff8101830 0x7f3ff810b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.524+0000 7f3fffc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff810e9a0 0x7f3ff810b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.524+0000 7f3fffc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8114720 con 0x7f3ff8101830 2024-04-16T17:19:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.524+0000 7f3fffc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81145a0 con 0x7f3ff810e9a0 2024-04-16T17:19:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.524+0000 7f3fffc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81148a0 con 0x7f3ff8100e80 2024-04-16T17:19:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.524+0000 7f3ffdc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff810e9a0 0x7f3ff810b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffdc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff810e9a0 0x7f3ff810b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54908/0 (socket says 172.21.15.18:54908) 2024-04-16T17:19:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffdc00640 1 -- 172.21.15.18:0/1852194515 learned_addr learned my addr 172.21.15.18:0/1852194515 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff8101830 0x7f3ff810b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffd200640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 0x7f3ff810e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffdc00640 1 -- 172.21.15.18:0/1852194515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 msgr2=0x7f3ff810e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffdc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 0x7f3ff810e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffdc00640 1 -- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff8101830 msgr2=0x7f3ff810b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffdc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff8101830 0x7f3ff810b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffdc00640 1 -- 172.21.15.18:0/1852194515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff81ab460 con 0x7f3ff810e9a0 2024-04-16T17:19:34.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ffd200640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 0x7f3ff810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:34.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.525+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff8101830 0x7f3ff810b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:34.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.526+0000 7f3ffdc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff810e9a0 0x7f3ff810b9b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000eb10 tx=0x7f3ff000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.526+0000 7f3ff5600640 1 -- 172.21.15.18:0/1852194515 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0004b20 con 0x7f3ff810e9a0 2024-04-16T17:19:34.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.526+0000 7f3ff5600640 1 -- 172.21.15.18:0/1852194515 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff0004cc0 con 0x7f3ff810e9a0 2024-04-16T17:19:34.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.526+0000 7f3ff5600640 1 -- 172.21.15.18:0/1852194515 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0010790 con 0x7f3ff810e9a0 2024-04-16T17:19:34.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.526+0000 7f3fffc00640 1 -- 172.21.15.18:0/1852194515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff81ab630 con 0x7f3ff810e9a0 2024-04-16T17:19:34.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.526+0000 7f3fffc00640 1 -- 172.21.15.18:0/1852194515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff81abaa0 con 0x7f3ff810e9a0 2024-04-16T17:19:34.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.530+0000 7f40027ea740 1 Processor -- start 2024-04-16T17:19:34.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.531+0000 7f3ff5600640 1 -- 172.21.15.18:0/1852194515 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ff0010930 con 0x7f3ff810e9a0 2024-04-16T17:19:34.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.532+0000 7f40027ea740 1 -- start start 2024-04-16T17:19:34.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f40027ea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb47d50 0x56476cb48130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f40027ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb50d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f40027ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb51250 0x56476cb53640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f40027ea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56476caad020 con 0x56476cb47d50 2024-04-16T17:19:34.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f40027ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56476ca53460 con 0x56476cb51250 2024-04-16T17:19:34.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f40027ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56476caa8fe0 con 0x56476cb48670 2024-04-16T17:19:34.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f3ffd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb50d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f3ff5600640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3fe00a3ea0 0x7f3fe00a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb47d50 0x56476cb48130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f3ffd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb50d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42992/0 (socket says 172.21.15.18:42992) 2024-04-16T17:19:34.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.533+0000 7f3ffd200640 1 -- 172.21.15.18:0/3975553537 learned_addr learned my addr 172.21.15.18:0/3975553537 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.534+0000 7f3ff5600640 1 -- 172.21.15.18:0/1852194515 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f3ff00d1240 con 0x7f3ff810e9a0 2024-04-16T17:19:34.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.534+0000 7f3ffdc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3fe00a3ea0 0x7f3fe00a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.534+0000 7f3ff7e00640 1 -- 172.21.15.18:0/3975553537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 msgr2=0x56476cb50d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.534+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/3975553537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb50d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.534+0000 7f3ff7e00640 1 -- 172.21.15.18:0/3975553537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb51250 msgr2=0x56476cb53640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:19:34.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.534+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/3975553537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb51250 0x56476cb53640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.534+0000 7f3ff7e00640 1 -- 172.21.15.18:0/3975553537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56476ca5b040 con 0x56476cb47d50 2024-04-16T17:19:34.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.535+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/3975553537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb47d50 0x56476cb48130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005fd0 tx=0x7f3fe400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cff355d3a150765 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.535+0000 7f3ffdc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3fe00a3ea0 0x7f3fe00a6360 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3ff009cad0 tx=0x7f3ff00bbf90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.535+0000 7f3fea000640 1 -- 172.21.15.18:0/3975553537 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4016070 con 0x56476cb47d50 2024-04-16T17:19:34.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.535+0000 7f3fea000640 1 -- 172.21.15.18:0/3975553537 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe4005500 con 0x56476cb47d50 2024-04-16T17:19:34.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.535+0000 7f3fea000640 1 -- 172.21.15.18:0/3975553537 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4011470 con 0x56476cb47d50 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.536+0000 7f40027ea740 1 -- 172.21.15.18:0/3975553537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb47d50 msgr2=0x56476cb48130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.536+0000 7f40027ea740 1 --2- 172.21.15.18:0/3975553537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb47d50 0x56476cb48130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005fd0 tx=0x7f3fe400e2f0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.537+0000 7f40027ea740 1 -- 172.21.15.18:0/3975553537 shutdown_connections 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.537+0000 7f40027ea740 1 --2- 172.21.15.18:0/3975553537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb51250 0x56476cb53640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.537+0000 7f40027ea740 1 --2- 172.21.15.18:0/3975553537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb50d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.537+0000 7f40027ea740 1 --2- 172.21.15.18:0/3975553537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb47d50 0x56476cb48130 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.537+0000 7f40027ea740 1 -- 172.21.15.18:0/3975553537 >> 172.21.15.18:0/3975553537 conn(0x56476cb42480 msgr2=0x56476cb42860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.537+0000 7f40027ea740 1 -- 172.21.15.18:0/3975553537 shutdown_connections 2024-04-16T17:19:34.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.537+0000 7f40027ea740 1 -- 172.21.15.18:0/3975553537 wait complete. 2024-04-16T17:19:34.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.537+0000 7f40027ea740 1 -- start start 2024-04-16T17:19:34.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.538+0000 7f40027ea740 1 Processor -- start 2024-04-16T17:19:34.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.538+0000 7f40027ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb47d50 0x56476cb4e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.538+0000 7f40027ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb48670 0x56476cb4e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.538+0000 7f40027ea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb4ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.538+0000 7f40027ea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56476cb540e0 con 0x56476cb51250 2024-04-16T17:19:34.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.538+0000 7f40027ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56476ca66830 con 0x56476cb48670 2024-04-16T17:19:34.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.538+0000 7f40027ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56476cb54260 con 0x56476cb47d50 2024-04-16T17:19:34.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.538+0000 7f40027ea740 10 client.0 _ll_register_callbacks cb 0x56476ca6e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:34.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.539+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb47d50 0x56476cb4e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.540+0000 7f3ffd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb48670 0x56476cb4e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.540+0000 7f3ffd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb48670 0x56476cb4e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54910/0 (socket says 172.21.15.18:54910) 2024-04-16T17:19:34.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.540+0000 7f3ffd200640 1 -- 172.21.15.18:0/4116361963 learned_addr learned my addr 172.21.15.18:0/4116361963 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.540+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb4ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.541+0000 7f3ff7e00640 1 -- 172.21.15.18:0/4116361963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb48670 msgr2=0x56476cb4e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.541+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb48670 0x56476cb4e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.541+0000 7f3ff7e00640 1 -- 172.21.15.18:0/4116361963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 msgr2=0x56476cb4ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.541+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb4ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.541+0000 7f3ff7e00640 1 -- 172.21.15.18:0/4116361963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56476ca6a920 con 0x56476cb47d50 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.541+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb47d50 0x56476cb4e1d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005fd0 tx=0x7f3fe4004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e6642b41db413b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.541+0000 7f3fdea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.542+0000 7f3fdea00640 1 -- 172.21.15.18:0/4116361963 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4016040 con 0x56476cb47d50 2024-04-16T17:19:34.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.542+0000 7f3fdea00640 1 -- 172.21.15.18:0/4116361963 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f3fe400eaf0 con 0x56476cb47d50 2024-04-16T17:19:34.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.543+0000 7f40027ea740 1 -- 172.21.15.18:0/4116361963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe4005500 con 0x56476cb47d50 2024-04-16T17:19:34.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.543+0000 7f3fdea00640 1 -- 172.21.15.18:0/4116361963 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3fe402e050 con 0x56476cb47d50 2024-04-16T17:19:34.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.543+0000 7f40027ea740 10 client.29422 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:34.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.543+0000 7f40027ea740 1 -- 172.21.15.18:0/4116361963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56476ca387f0 con 0x56476cb47d50 2024-04-16T17:19:34.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.544+0000 7f3fdea00640 1 -- 172.21.15.18:0/4116361963 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe4005500 con 0x56476cb47d50 2024-04-16T17:19:34.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.545+0000 7f3fdea00640 1 -- 172.21.15.18:0/4116361963 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3fe400f040 con 0x56476cb47d50 2024-04-16T17:19:34.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.545+0000 7f40027ea740 10 client.29422 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:34.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.545+0000 7f40027ea740 10 client.29422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:34.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.545+0000 7f40027ea740 20 client.29422 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:34.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.545+0000 7f40027ea740 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56476cb61a30 0x56476cb63e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.545+0000 7f40027ea740 4 client.29422 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:34.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.545+0000 7f40027ea740 1 -- 172.21.15.18:0/4116361963 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56476cabaa40 con 0x56476cb61a30 2024-04-16T17:19:34.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.545+0000 7f3ffd200640 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56476cb61a30 0x56476cb63e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.546+0000 7f3ffd200640 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56476cb61a30 0x56476cb63e70 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431c641bed704de2 server_cookie=c2d826836e338337 in_seq=0 out_seq=0 2024-04-16T17:19:34.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.546+0000 7f3fdea00640 10 client.29422 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:34.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.548+0000 7f3fdea00640 1 -- 172.21.15.18:0/4116361963 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56476cabaa40 con 0x56476cb61a30 2024-04-16T17:19:34.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.548+0000 7f3fdea00640 10 client.29422 handle_command_reply: tid=0 2024-04-16T17:19:34.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.548+0000 7f40027ea740 1 client.29422 shutdown 2024-04-16T17:19:34.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.550+0000 7f40027ea740 1 -- 172.21.15.18:0/4116361963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb47d50 msgr2=0x56476cb4e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.550+0000 7f40027ea740 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb47d50 0x56476cb4e1d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005fd0 tx=0x7f3fe4004ba0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.551+0000 7f40027ea740 1 -- 172.21.15.18:0/4116361963 shutdown_connections 2024-04-16T17:19:34.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.551+0000 7f40027ea740 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56476cb61a30 0x56476cb63e70 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.551+0000 7f40027ea740 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb4ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.551+0000 7f40027ea740 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb48670 0x56476cb4e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.551+0000 7f40027ea740 1 --2- 172.21.15.18:0/4116361963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb47d50 0x56476cb4e1d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.551+0000 7f40027ea740 1 -- 172.21.15.18:0/4116361963 >> 172.21.15.18:0/4116361963 conn(0x56476cb42480 msgr2=0x56476cb441d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.551+0000 7f3fdea00640 0 client.29422 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:34.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.552+0000 7f40027ea740 1 -- 172.21.15.18:0/4116361963 shutdown_connections 2024-04-16T17:19:34.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.552+0000 7f40027ea740 1 -- 172.21.15.18:0/4116361963 wait complete. 2024-04-16T17:19:34.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.552+0000 7f40027ea740 20 client.29422 trim_cache size 0 max 16384 2024-04-16T17:19:34.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.573+0000 7f40027ea740 1 Processor -- start 2024-04-16T17:19:34.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.573+0000 7f40027ea740 1 -- start start 2024-04-16T17:19:34.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f40027ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb50900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f40027ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb5da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f40027ea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb5d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f40027ea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8012a80 con 0x56476cb51250 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f40027ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd80102f0 con 0x56476cb47d50 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f40027ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd800e160 con 0x56476cb48670 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb50900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb50900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54922/0 (socket says 172.21.15.18:54922) 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f3ff7e00640 1 -- 172.21.15.18:0/551324724 learned_addr learned my addr 172.21.15.18:0/551324724 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.574+0000 7f3ffd200640 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb5da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.575+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb5d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.575+0000 7f3ff7e00640 1 -- 172.21.15.18:0/551324724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 msgr2=0x56476cb5da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.575+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb5da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.575+0000 7f3ff7e00640 1 -- 172.21.15.18:0/551324724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 msgr2=0x56476cb5d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.575+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb5d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.575+0000 7f3ff7e00640 1 -- 172.21.15.18:0/551324724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56476c7bf3f0 con 0x56476cb47d50 2024-04-16T17:19:34.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.575+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb50900 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009960 tx=0x7f3fe4011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b3a9dfe9dd66c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.576+0000 7f3fe9600640 1 -- 172.21.15.18:0/551324724 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4004c10 con 0x56476cb47d50 2024-04-16T17:19:34.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.576+0000 7f3fe9600640 1 -- 172.21.15.18:0/551324724 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe4004fe0 con 0x56476cb47d50 2024-04-16T17:19:34.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.576+0000 7f3fe9600640 1 -- 172.21.15.18:0/551324724 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4004280 con 0x56476cb47d50 2024-04-16T17:19:34.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.577+0000 7f40027ea740 1 -- 172.21.15.18:0/551324724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 msgr2=0x56476cb50900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.577+0000 7f40027ea740 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb50900 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009960 tx=0x7f3fe4011f90 comp rx=0 tx=0).stop 2024-04-16T17:19:34.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.577+0000 7f40027ea740 1 -- 172.21.15.18:0/551324724 shutdown_connections 2024-04-16T17:19:34.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.577+0000 7f40027ea740 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb5d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.577+0000 7f40027ea740 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb5da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.577+0000 7f40027ea740 1 --2- 172.21.15.18:0/551324724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb50900 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.577+0000 7f40027ea740 1 -- 172.21.15.18:0/551324724 >> 172.21.15.18:0/551324724 conn(0x56476cb42480 msgr2=0x56476cb49490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.577+0000 7f40027ea740 1 -- 172.21.15.18:0/551324724 shutdown_connections 2024-04-16T17:19:34.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.578+0000 7f40027ea740 1 -- 172.21.15.18:0/551324724 wait complete. 2024-04-16T17:19:34.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.578+0000 7f40027ea740 1 -- start start 2024-04-16T17:19:34.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.578+0000 7f40027ea740 1 Processor -- start 2024-04-16T17:19:34.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f40027ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb54250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f40027ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb54790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f40027ea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb54cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f40027ea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd800e7b0 con 0x56476cb51250 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f40027ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd800fc20 con 0x56476cb47d50 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f40027ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd800ee80 con 0x56476cb48670 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f40027ea740 10 client.0 _ll_register_callbacks cb 0x7f3ff00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb54250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb54250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54930/0 (socket says 172.21.15.18:54930) 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f3ff7e00640 1 -- 172.21.15.18:0/1547960282 learned_addr learned my addr 172.21.15.18:0/1547960282 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f3ffd200640 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb54790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb54cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f3ff7e00640 1 -- 172.21.15.18:0/1547960282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 msgr2=0x56476cb54790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.579+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb54790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.580+0000 7f3ff7e00640 1 -- 172.21.15.18:0/1547960282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 msgr2=0x56476cb54cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:34.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.580+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb54cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.580+0000 7f3ff7e00640 1 -- 172.21.15.18:0/1547960282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56476c8897d0 con 0x56476cb47d50 2024-04-16T17:19:34.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.582+0000 7f3ff7e00640 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb54250 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005820 tx=0x7f3fe40055c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7436e7b1b0f5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:34.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.582+0000 7f3fdea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:34.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.582+0000 7f3fdea00640 1 -- 172.21.15.18:0/1547960282 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4016030 con 0x56476cb47d50 2024-04-16T17:19:34.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.582+0000 7f3fdea00640 1 -- 172.21.15.18:0/1547960282 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f3fe4004aa0 con 0x56476cb47d50 2024-04-16T17:19:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.582+0000 7f40027ea740 1 -- 172.21.15.18:0/1547960282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56476cab48b0 con 0x56476cb47d50 2024-04-16T17:19:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.582+0000 7f3fdea00640 1 -- 172.21.15.18:0/1547960282 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3fe4042050 con 0x56476cb47d50 2024-04-16T17:19:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.583+0000 7f40027ea740 10 client.29600 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.583+0000 7f40027ea740 1 -- 172.21.15.18:0/1547960282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56476cbc7270 con 0x56476cb47d50 2024-04-16T17:19:34.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.584+0000 7f3fdea00640 1 -- 172.21.15.18:0/1547960282 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56476cab48b0 con 0x56476cb47d50 2024-04-16T17:19:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.584+0000 7f3fdea00640 1 -- 172.21.15.18:0/1547960282 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3fe4029ac0 con 0x56476cb47d50 2024-04-16T17:19:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.584+0000 7f40027ea740 10 client.29600 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.584+0000 7f40027ea740 10 client.29600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.584+0000 7f40027ea740 20 client.29600 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.585+0000 7f40027ea740 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56476cbc78b0 0x56476cbc9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.585+0000 7f40027ea740 4 client.29600 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:19:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.585+0000 7f40027ea740 1 -- 172.21.15.18:0/1547960282 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56476ca30570 con 0x56476cbc78b0 2024-04-16T17:19:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.585+0000 7f3ffd200640 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56476cbc78b0 0x56476cbc9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.586+0000 7f3ffd200640 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56476cbc78b0 0x56476cbc9cd0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00ca25efac0a04c server_cookie=1d254b9e027a1538 in_seq=0 out_seq=0 2024-04-16T17:19:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.586+0000 7f3fdea00640 10 client.29600 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.587+0000 7f3fdea00640 1 -- 172.21.15.18:0/1547960282 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56476ca30570 con 0x56476cbc78b0 2024-04-16T17:19:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.587+0000 7f3fdea00640 10 client.29600 handle_command_reply: tid=0 2024-04-16T17:19:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.587+0000 7f40027ea740 1 client.29600 shutdown 2024-04-16T17:19:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f40027ea740 1 -- 172.21.15.18:0/1547960282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 msgr2=0x56476cb54250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f40027ea740 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb54250 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005820 tx=0x7f3fe40055c0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f40027ea740 1 -- 172.21.15.18:0/1547960282 shutdown_connections 2024-04-16T17:19:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f40027ea740 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56476cbc78b0 0x56476cbc9cd0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f3fdea00640 0 client.29600 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f40027ea740 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56476cb51250 0x56476cb54cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f40027ea740 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56476cb48670 0x56476cb54790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f40027ea740 1 --2- 172.21.15.18:0/1547960282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56476cb47d50 0x56476cb54250 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.588+0000 7f40027ea740 1 -- 172.21.15.18:0/1547960282 >> 172.21.15.18:0/1547960282 conn(0x56476cb42480 msgr2=0x56476cb49490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.589+0000 7f40027ea740 1 -- 172.21.15.18:0/1547960282 shutdown_connections 2024-04-16T17:19:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.589+0000 7f40027ea740 1 -- 172.21.15.18:0/1547960282 wait complete. 2024-04-16T17:19:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.589+0000 7f40027ea740 20 client.29600 trim_cache size 0 max 16384 2024-04-16T17:19:34.590 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 -- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3fe00a3ea0 msgr2=0x7f3fe00a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3fe00a3ea0 0x7f3fe00a6360 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3ff009cad0 tx=0x7f3ff00bbf90 comp rx=0 tx=0).stop 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 -- 172.21.15.18:0/1852194515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff810e9a0 msgr2=0x7f3ff810b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff810e9a0 0x7f3ff810b9b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000eb10 tx=0x7f3ff000c2d0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 -- 172.21.15.18:0/1852194515 shutdown_connections 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3fe00a3ea0 0x7f3fe00a6360 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ff810e9a0 0x7f3ff810b9b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff8101830 0x7f3ff810b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 --2- 172.21.15.18:0/1852194515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8100e80 0x7f3ff810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.590+0000 7f3fffc00640 1 -- 172.21.15.18:0/1852194515 >> 172.21.15.18:0/1852194515 conn(0x7f3ff8078020 msgr2=0x7f3ff810fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.591+0000 7f3fffc00640 1 -- 172.21.15.18:0/1852194515 shutdown_connections 2024-04-16T17:19:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:34.591+0000 7f3fffc00640 1 -- 172.21.15.18:0/1852194515 wait complete. 2024-04-16T17:19:34.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:34 smithi022 ceph-mon[36023]: pgmap v471: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-04-16T17:19:34.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:34 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1653442378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:19:35.348 INFO:tasks.workunit.client.0.smithi018.stdout: 10 28055 15.70 MB/sec warmup 107 sec latency 187.893 ms 2024-04-16T17:19:35.618 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:19:35.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:19:35.618 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:19:35.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.912+0000 7fdc69800640 1 -- 172.21.15.18:0/1142061557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc640743b0 msgr2=0x7fdc64074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.912+0000 7fdc69800640 1 --2- 172.21.15.18:0/1142061557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc640743b0 0x7fdc64074790 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402bce0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.913+0000 7fdc69800640 1 -- 172.21.15.18:0/1142061557 shutdown_connections 2024-04-16T17:19:35.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.913+0000 7fdc69800640 1 --2- 172.21.15.18:0/1142061557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc6410e6c0 0x7fdc64110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.913+0000 7fdc69800640 1 --2- 172.21.15.18:0/1142061557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc64074cd0 0x7fdc6410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.913+0000 7fdc69800640 1 --2- 172.21.15.18:0/1142061557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc640743b0 0x7fdc64074790 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.913+0000 7fdc69800640 1 -- 172.21.15.18:0/1142061557 >> 172.21.15.18:0/1142061557 conn(0x7fdc640fd950 msgr2=0x7fdc640ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:35.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.914+0000 7fdc69800640 1 -- 172.21.15.18:0/1142061557 shutdown_connections 2024-04-16T17:19:35.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.914+0000 7fdc69800640 1 -- 172.21.15.18:0/1142061557 wait complete. 2024-04-16T17:19:35.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.916+0000 7fdc69800640 1 Processor -- start 2024-04-16T17:19:35.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.916+0000 7fdc69800640 1 -- start start 2024-04-16T17:19:35.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.916+0000 7fdc69800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc640743b0 0x7fdc641a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.916+0000 7fdc69800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc64074cd0 0x7fdc641a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc69800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6410e6c0 0x7fdc6419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc63400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc64074cd0 0x7fdc641a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc640743b0 0x7fdc641a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc63400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc64074cd0 0x7fdc641a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54946/0 (socket says 172.21.15.18:54946) 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc640743b0 0x7fdc641a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46864/0 (socket says 172.21.15.18:46864) 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc63400640 1 -- 172.21.15.18:0/2598271338 learned_addr learned my addr 172.21.15.18:0/2598271338 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64114470 con 0x7fdc640743b0 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641142f0 con 0x7fdc64074cd0 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641145f0 con 0x7fdc6410e6c0 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.917+0000 7fdc68e00640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6410e6c0 0x7fdc6419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc68e00640 1 -- 172.21.15.18:0/2598271338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc64074cd0 msgr2=0x7fdc641a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc68e00640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc64074cd0 0x7fdc641a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc68e00640 1 -- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc640743b0 msgr2=0x7fdc641a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc68e00640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc640743b0 0x7fdc641a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc68e00640 1 -- 172.21.15.18:0/2598271338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6419afc0 con 0x7fdc6410e6c0 2024-04-16T17:19:35.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc68e00640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6410e6c0 0x7fdc6419a7f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009fd0 tx=0x7fdc5c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:35.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc60c00640 1 -- 172.21.15.18:0/2598271338 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c00ce20 con 0x7fdc6410e6c0 2024-04-16T17:19:35.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc60c00640 1 -- 172.21.15.18:0/2598271338 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c005020 con 0x7fdc6410e6c0 2024-04-16T17:19:35.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc60c00640 1 -- 172.21.15.18:0/2598271338 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c010720 con 0x7fdc6410e6c0 2024-04-16T17:19:35.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6419b2b0 con 0x7fdc6410e6c0 2024-04-16T17:19:35.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.918+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641a74d0 con 0x7fdc6410e6c0 2024-04-16T17:19:35.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.923+0000 7fdc6dbd6740 1 Processor -- start 2024-04-16T17:19:35.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.923+0000 7fdc60c00640 1 -- 172.21.15.18:0/2598271338 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc5c005540 con 0x7fdc6410e6c0 2024-04-16T17:19:35.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.923+0000 7fdc6dbd6740 1 -- start start 2024-04-16T17:19:35.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a2798e90 0x5603a2799270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a27a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a27a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5603a26fdcb0 con 0x5603a27a2390 2024-04-16T17:19:35.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5603a26a4370 con 0x5603a2798e90 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603a26f9c70 con 0x5603a27997b0 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc63400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a2798e90 0x5603a2799270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a27a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.924+0000 7fdc63400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a2798e90 0x5603a2799270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54948/0 (socket says 172.21.15.18:54948) 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a27a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43042/0 (socket says 172.21.15.18:43042) 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc63400640 1 -- 172.21.15.18:0/2571019122 learned_addr learned my addr 172.21.15.18:0/2571019122 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc63400640 1 --2- 172.21.15.18:0/2571019122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a27a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc60c00640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc480a3d20 0x7fdc480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc60c00640 1 -- 172.21.15.18:0/2598271338 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fdc5c0d1270 con 0x7fdc6410e6c0 2024-04-16T17:19:35.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc63e00640 1 -- 172.21.15.18:0/2571019122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a2798e90 msgr2=0x5603a2799270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:35.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc63e00640 1 --2- 172.21.15.18:0/2571019122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a2798e90 0x5603a2799270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc63e00640 1 -- 172.21.15.18:0/2571019122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 msgr2=0x5603a27a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:35.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc63e00640 1 --2- 172.21.15.18:0/2571019122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a27a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.925+0000 7fdc63e00640 1 -- 172.21.15.18:0/2571019122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a26c3910 con 0x5603a27997b0 2024-04-16T17:19:35.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.926+0000 7fdc68e00640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc480a3d20 0x7fdc480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.926+0000 7fdc63e00640 1 --2- 172.21.15.18:0/2571019122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a27a1e50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdc54000c00 tx=0x7fdc54005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e31d4995a5dc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:35.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.927+0000 7fdc68e00640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc480a3d20 0x7fdc480a61e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0a54e0 tx=0x7fdc5c005f70 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:35.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.927+0000 7fdc59600640 1 -- 172.21.15.18:0/2571019122 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54042070 con 0x5603a27997b0 2024-04-16T17:19:35.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.927+0000 7fdc59600640 1 -- 172.21.15.18:0/2571019122 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5402cb30 con 0x5603a27997b0 2024-04-16T17:19:35.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.927+0000 7fdc59600640 1 -- 172.21.15.18:0/2571019122 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54004f10 con 0x5603a27997b0 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.927+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/2571019122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 msgr2=0x5603a27a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.927+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/2571019122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a27a1e50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdc54000c00 tx=0x7fdc54005e00 comp rx=0 tx=0).stop 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.928+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/2571019122 shutdown_connections 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.928+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/2571019122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a27a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.928+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/2571019122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a27a1e50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.928+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/2571019122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a2798e90 0x5603a2799270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.928+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/2571019122 >> 172.21.15.18:0/2571019122 conn(0x5603a27935c0 msgr2=0x5603a27939a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.928+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/2571019122 shutdown_connections 2024-04-16T17:19:35.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.928+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/2571019122 wait complete. 2024-04-16T17:19:35.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.928+0000 7fdc6dbd6740 1 -- start start 2024-04-16T17:19:35.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc6dbd6740 1 Processor -- start 2024-04-16T17:19:35.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a279f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a279f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a279fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5603a27a5120 con 0x5603a27a2390 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5603a26b7730 con 0x5603a27997b0 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603a27a52a0 con 0x5603a2798e90 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc6dbd6740 10 client.0 _ll_register_callbacks cb 0x5603a26861e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a279f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a279f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54960/0 (socket says 172.21.15.18:54960) 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.929+0000 7fdc63e00640 1 -- 172.21.15.18:0/1105459109 learned_addr learned my addr 172.21.15.18:0/1105459109 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:35.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63400640 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a279f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63400640 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a279fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63e00640 1 -- 172.21.15.18:0/1105459109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 msgr2=0x5603a279f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63e00640 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a279f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63e00640 1 -- 172.21.15.18:0/1105459109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 msgr2=0x5603a279fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63e00640 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a279fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63e00640 1 -- 172.21.15.18:0/1105459109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a24d87e0 con 0x5603a27997b0 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63400640 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a279fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc63e00640 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a279f9b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdc54036040 tx=0x7fdc5402c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3f5d32830224ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc4e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:35.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc4e000640 1 -- 172.21.15.18:0/1105459109 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54042070 con 0x5603a27997b0 2024-04-16T17:19:35.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.930+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/1105459109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5402cb30 con 0x5603a27997b0 2024-04-16T17:19:35.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.931+0000 7fdc4e000640 1 -- 172.21.15.18:0/1105459109 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fdc54005110 con 0x5603a27997b0 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.931+0000 7fdc4e000640 1 -- 172.21.15.18:0/1105459109 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdc54058050 con 0x5603a27997b0 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc6dbd6740 10 client.29612 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/1105459109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5603a2538f70 con 0x5603a27997b0 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc4e000640 1 -- 172.21.15.18:0/1105459109 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5402cb30 con 0x5603a27997b0 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc4e000640 1 -- 172.21.15.18:0/1105459109 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fdc540344d0 con 0x5603a27997b0 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc6dbd6740 10 client.29612 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc6dbd6740 10 client.29612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc6dbd6740 20 client.29612 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5603a27b25f0 0x5603a27b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc6dbd6740 4 client.29612 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:35.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/1105459109 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603a270b6d0 con 0x5603a27b25f0 2024-04-16T17:19:35.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.932+0000 7fdc63400640 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5603a27b25f0 0x5603a27b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.933+0000 7fdc63400640 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5603a27b25f0 0x5603a27b49e0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0c1787c18c325e server_cookie=17bee2be8b356093 in_seq=0 out_seq=0 2024-04-16T17:19:35.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.933+0000 7fdc4e000640 10 client.29612 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.935+0000 7fdc4e000640 1 -- 172.21.15.18:0/1105459109 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603a270b6d0 con 0x5603a27b25f0 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.935+0000 7fdc4e000640 10 client.29612 handle_command_reply: tid=0 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.935+0000 7fdc6dbd6740 1 client.29612 shutdown 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/1105459109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 msgr2=0x5603a279f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a279f9b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdc54036040 tx=0x7fdc5402c930 comp rx=0 tx=0).stop 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/1105459109 shutdown_connections 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5603a27b25f0 0x5603a27b49e0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc4e000640 0 client.29612 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:35.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a279fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a279f9b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/1105459109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a279f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/1105459109 >> 172.21.15.18:0/1105459109 conn(0x5603a27935c0 msgr2=0x5603a2795310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:35.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/1105459109 shutdown_connections 2024-04-16T17:19:35.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/1105459109 wait complete. 2024-04-16T17:19:35.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.936+0000 7fdc6dbd6740 20 client.29612 trim_cache size 0 max 16384 2024-04-16T17:19:35.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.956+0000 7fdc6dbd6740 1 Processor -- start 2024-04-16T17:19:35.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.956+0000 7fdc6dbd6740 1 -- start start 2024-04-16T17:19:35.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a2798e90 0x5603a27adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a279ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27a2390 0x5603a27b9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b4050 con 0x5603a2798e90 2024-04-16T17:19:35.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b18c0 con 0x5603a27a2390 2024-04-16T17:19:35.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641af730 con 0x5603a27997b0 2024-04-16T17:19:35.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a279ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a279ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43070/0 (socket says 172.21.15.18:43070) 2024-04-16T17:19:35.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63e00640 1 -- 172.21.15.18:0/3593440520 learned_addr learned my addr 172.21.15.18:0/3593440520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63400640 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a2798e90 0x5603a27adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63400640 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27a2390 0x5603a27b9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63e00640 1 -- 172.21.15.18:0/3593440520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27a2390 msgr2=0x5603a27b9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63e00640 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27a2390 0x5603a27b9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63e00640 1 -- 172.21.15.18:0/3593440520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a2798e90 msgr2=0x5603a27adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.957+0000 7fdc63e00640 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a2798e90 0x5603a27adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.958+0000 7fdc63e00640 1 -- 172.21.15.18:0/3593440520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a23d0170 con 0x5603a27997b0 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.958+0000 7fdc63400640 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a2798e90 0x5603a27adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.958+0000 7fdc63e00640 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a279ed50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdc5402f020 tx=0x7fdc5402c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fcf820ca18a0a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.958+0000 7fdc58c00640 1 -- 172.21.15.18:0/3593440520 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54055070 con 0x5603a27997b0 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.958+0000 7fdc58c00640 1 -- 172.21.15.18:0/3593440520 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc54005520 con 0x5603a27997b0 2024-04-16T17:19:35.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.958+0000 7fdc58c00640 1 -- 172.21.15.18:0/3593440520 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54042070 con 0x5603a27997b0 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3593440520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 msgr2=0x5603a279ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a279ed50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdc5402f020 tx=0x7fdc5402c890 comp rx=0 tx=0).stop 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3593440520 shutdown_connections 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27a2390 0x5603a27b9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a27997b0 0x5603a279ed50 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3593440520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a2798e90 0x5603a27adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3593440520 >> 172.21.15.18:0/3593440520 conn(0x5603a27935c0 msgr2=0x5603a279fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3593440520 shutdown_connections 2024-04-16T17:19:35.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.959+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3593440520 wait complete. 2024-04-16T17:19:35.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.960+0000 7fdc6dbd6740 1 -- start start 2024-04-16T17:19:35.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.960+0000 7fdc6dbd6740 1 Processor -- start 2024-04-16T17:19:35.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a27a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a27a6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc6dbd6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a27a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641afd80 con 0x5603a27a2390 2024-04-16T17:19:35.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b11f0 con 0x5603a27997b0 2024-04-16T17:19:35.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc6dbd6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b0450 con 0x5603a2798e90 2024-04-16T17:19:35.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc6dbd6740 10 client.0 _ll_register_callbacks cb 0x7fdc5402c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc63400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a27a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a27a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc63400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a27a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a27a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54972/0 (socket says 172.21.15.18:54972) 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc63400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a27a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43072/0 (socket says 172.21.15.18:43072) 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.962+0000 7fdc63e00640 1 -- 172.21.15.18:0/3942831255 learned_addr learned my addr 172.21.15.18:0/3942831255 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc63e00640 1 -- 172.21.15.18:0/3942831255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 msgr2=0x5603a27a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc63e00640 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a27a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc63e00640 1 -- 172.21.15.18:0/3942831255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 msgr2=0x5603a27a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc63e00640 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a27a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc63e00640 1 -- 172.21.15.18:0/3942831255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a2820130 con 0x5603a27997b0 2024-04-16T17:19:35.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc63e00640 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a27a6560 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc54034cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29380bbd98360adb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:35.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc4e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc4e000640 1 -- 172.21.15.18:0/3942831255 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54055070 con 0x5603a27997b0 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3942831255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc54005520 con 0x5603a27997b0 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.963+0000 7fdc4e000640 1 -- 172.21.15.18:0/3942831255 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fdc5402c930 con 0x5603a27997b0 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.964+0000 7fdc4e000640 1 -- 172.21.15.18:0/3942831255 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdc5406c050 con 0x5603a27997b0 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.964+0000 7fdc6dbd6740 10 client.29618 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.964+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3942831255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5603a2821450 con 0x5603a27997b0 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.964+0000 7fdc4e000640 1 -- 172.21.15.18:0/3942831255 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc54034440 con 0x5603a27997b0 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.964+0000 7fdc4e000640 1 -- 172.21.15.18:0/3942831255 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fdc54047c10 con 0x5603a27997b0 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.965+0000 7fdc6dbd6740 10 client.29618 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.965+0000 7fdc6dbd6740 10 client.29618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.965+0000 7fdc6dbd6740 20 client.29618 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:35.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.965+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5603a2821b70 0x5603a2823f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:35.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.965+0000 7fdc6dbd6740 4 client.29618 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:19:35.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.965+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3942831255 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fdc54005520 con 0x5603a2821b70 2024-04-16T17:19:35.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.965+0000 7fdc63400640 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5603a2821b70 0x5603a2823f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:35.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.966+0000 7fdc63400640 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5603a2821b70 0x5603a2823f90 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a0a1af432b84b22 server_cookie=69a0c4ec4d3f64cf in_seq=0 out_seq=0 2024-04-16T17:19:35.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.966+0000 7fdc4e000640 10 client.29618 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:35.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc4e000640 1 -- 172.21.15.18:0/3942831255 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fdc54005520 con 0x5603a2821b70 2024-04-16T17:19:35.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc4e000640 10 client.29618 handle_command_reply: tid=0 2024-04-16T17:19:35.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 client.29618 shutdown 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3942831255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 msgr2=0x5603a27a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a27a6560 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc54034cb0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3942831255 shutdown_connections 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5603a2821b70 0x5603a2823f90 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5603a27a2390 0x5603a27a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603a27997b0 0x5603a27a6560 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 --2- 172.21.15.18:0/3942831255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603a2798e90 0x5603a27a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc4e000640 0 client.29618 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3942831255 >> 172.21.15.18:0/3942831255 conn(0x5603a27935c0 msgr2=0x5603a2794780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3942831255 shutdown_connections 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.970+0000 7fdc6dbd6740 1 -- 172.21.15.18:0/3942831255 wait complete. 2024-04-16T17:19:35.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc6dbd6740 20 client.29618 trim_cache size 0 max 16384 2024-04-16T17:19:35.972 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:19:35.972 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:19:35.972 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "6388226a-5c44-4c78-9e3f-25a0a6c524f9", 2024-04-16T17:19:35.972 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:19:35.972 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:19:35.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc480a3d20 msgr2=0x7fdc480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc480a3d20 0x7fdc480a61e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0a54e0 tx=0x7fdc5c005f70 comp rx=0 tx=0).stop 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6410e6c0 msgr2=0x7fdc6419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6410e6c0 0x7fdc6419a7f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009fd0 tx=0x7fdc5c00ef10 comp rx=0 tx=0).stop 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 shutdown_connections 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc480a3d20 0x7fdc480a61e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6410e6c0 0x7fdc6419a7f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc64074cd0 0x7fdc641a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 --2- 172.21.15.18:0/2598271338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc640743b0 0x7fdc641a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.971+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 >> 172.21.15.18:0/2598271338 conn(0x7fdc640fd950 msgr2=0x7fdc640fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.972+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 shutdown_connections 2024-04-16T17:19:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:35.972+0000 7fdc69800640 1 -- 172.21.15.18:0/2598271338 wait complete. 2024-04-16T17:19:35.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:19:35.996 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:19:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.225+0000 7f34e3a00640 1 -- 172.21.15.18:0/2500569774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc102be0 msgr2=0x7f34dc106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.225+0000 7f34e3a00640 1 --2- 172.21.15.18:0/2500569774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc102be0 0x7f34dc106c20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009900 tx=0x7f34c802bc40 comp rx=0 tx=0).stop 2024-04-16T17:19:36.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.226+0000 7f34e3a00640 1 -- 172.21.15.18:0/2500569774 shutdown_connections 2024-04-16T17:19:36.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.226+0000 7f34e3a00640 1 --2- 172.21.15.18:0/2500569774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc107300 0x7f34dc10eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.226+0000 7f34e3a00640 1 --2- 172.21.15.18:0/2500569774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc102be0 0x7f34dc106c20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.226+0000 7f34e3a00640 1 --2- 172.21.15.18:0/2500569774 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34dc102230 0x7f34dc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.226+0000 7f34e3a00640 1 -- 172.21.15.18:0/2500569774 >> 172.21.15.18:0/2500569774 conn(0x7f34dc0fdab0 msgr2=0x7f34dc0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:36.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.228+0000 7f34e3a00640 1 -- 172.21.15.18:0/2500569774 shutdown_connections 2024-04-16T17:19:36.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.228+0000 7f34e3a00640 1 -- 172.21.15.18:0/2500569774 wait complete. 2024-04-16T17:19:36.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.230+0000 7f34e3a00640 1 Processor -- start 2024-04-16T17:19:36.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.230+0000 7f34e3a00640 1 -- start start 2024-04-16T17:19:36.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.230+0000 7f34e3a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc102230 0x7f34dc10ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e3a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34dc107300 0x7f34dc10bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e3a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc10c190 0x7f34dc10c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e3a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc075a70 con 0x7f34dc107300 2024-04-16T17:19:36.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e3a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc0758f0 con 0x7f34dc102230 2024-04-16T17:19:36.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e3a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc075bf0 con 0x7f34dc10c190 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc10c190 0x7f34dc10c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc102230 0x7f34dc10ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc10c190 0x7f34dc10c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43100/0 (socket says 172.21.15.18:43100) 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34e1a00640 1 -- 172.21.15.18:0/3695780007 learned_addr learned my addr 172.21.15.18:0/3695780007 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.231+0000 7f34dbe00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34dc107300 0x7f34dc10bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.232+0000 7f34e1000640 1 -- 172.21.15.18:0/3695780007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc10c190 msgr2=0x7f34dc10c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.232+0000 7f34e1000640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc10c190 0x7f34dc10c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.232+0000 7f34e1000640 1 -- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34dc107300 msgr2=0x7f34dc10bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.232+0000 7f34e1000640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34dc107300 0x7f34dc10bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.232+0000 7f34e1000640 1 -- 172.21.15.18:0/3695780007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc1ab560 con 0x7f34dc102230 2024-04-16T17:19:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.232+0000 7f34dbe00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34dc107300 0x7f34dc10bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.232+0000 7f34e1000640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc102230 0x7f34dc10ebb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f34d400ea20 tx=0x7f34d400eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.232+0000 7f34d9600640 1 -- 172.21.15.18:0/3695780007 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d400cde0 con 0x7f34dc102230 2024-04-16T17:19:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.233+0000 7f34d9600640 1 -- 172.21.15.18:0/3695780007 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4011950 con 0x7f34dc102230 2024-04-16T17:19:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.233+0000 7f34d9600640 1 -- 172.21.15.18:0/3695780007 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d401aa80 con 0x7f34dc102230 2024-04-16T17:19:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.233+0000 7f34e3a00640 1 -- 172.21.15.18:0/3695780007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc1ab850 con 0x7f34dc102230 2024-04-16T17:19:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.233+0000 7f34e3a00640 1 -- 172.21.15.18:0/3695780007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc1abd20 con 0x7f34dc102230 2024-04-16T17:19:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.238+0000 7f34e6676740 1 Processor -- start 2024-04-16T17:19:36.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.238+0000 7f34e6676740 1 -- start start 2024-04-16T17:19:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.238+0000 7f34e6676740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91022c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.238+0000 7f34d9600640 1 -- 172.21.15.18:0/3695780007 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f34d401ac20 con 0x7f34dc102230 2024-04-16T17:19:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.238+0000 7f34e6676740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910235170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.238+0000 7f34e6676740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910237aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.239+0000 7f34e6676740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c910192120 con 0x55c91022cad0 2024-04-16T17:19:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.239+0000 7f34e6676740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c910138340 con 0x55c91022c1b0 2024-04-16T17:19:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.239+0000 7f34e6676740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c91018e0e0 con 0x55c9102356b0 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.239+0000 7f34dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910237aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.239+0000 7f34dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910237aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43104/0 (socket says 172.21.15.18:43104) 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.239+0000 7f34dbe00640 1 -- 172.21.15.18:0/1600279525 learned_addr learned my addr 172.21.15.18:0/1600279525 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.239+0000 7f34e1a00640 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91022c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.239+0000 7f34e1000640 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910235170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34dbe00640 1 -- 172.21.15.18:0/1600279525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 msgr2=0x55c91022c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34dbe00640 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91022c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34dbe00640 1 -- 172.21.15.18:0/1600279525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 msgr2=0x55c910235170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:36.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34dbe00640 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910235170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34dbe00640 1 -- 172.21.15.18:0/1600279525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c90ff6e7d0 con 0x55c9102356b0 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34e1000640 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910235170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34d9600640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34c00a3da0 0x7f34c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34dbe00640 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910237aa0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f34dc06b790 tx=0x7f34c8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af85bc8b2c266269 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.240+0000 7f34d9600640 1 -- 172.21.15.18:0/3695780007 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f34d40d1050 con 0x7f34dc102230 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.241+0000 7f34d2000640 1 -- 172.21.15.18:0/1600279525 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8042070 con 0x55c9102356b0 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.242+0000 7f34d2000640 1 -- 172.21.15.18:0/1600279525 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c802c6a0 con 0x55c9102356b0 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.242+0000 7f34d2000640 1 -- 172.21.15.18:0/1600279525 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8005180 con 0x55c9102356b0 2024-04-16T17:19:36.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.242+0000 7f34e1a00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34c00a3da0 0x7f34c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.243+0000 7f34e6676740 1 -- 172.21.15.18:0/1600279525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 msgr2=0x55c910237aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.243+0000 7f34e6676740 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910237aa0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f34dc06b790 tx=0x7f34c8000bc0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.243+0000 7f34e6676740 1 -- 172.21.15.18:0/1600279525 shutdown_connections 2024-04-16T17:19:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.243+0000 7f34e6676740 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910237aa0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.243+0000 7f34e6676740 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910235170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.243+0000 7f34e6676740 1 --2- 172.21.15.18:0/1600279525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91022c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.243+0000 7f34e6676740 1 -- 172.21.15.18:0/1600279525 >> 172.21.15.18:0/1600279525 conn(0x55c9102268e0 msgr2=0x55c910226cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.244+0000 7f34e6676740 1 -- 172.21.15.18:0/1600279525 shutdown_connections 2024-04-16T17:19:36.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.244+0000 7f34e6676740 1 -- 172.21.15.18:0/1600279525 wait complete. 2024-04-16T17:19:36.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.244+0000 7f34e1a00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34c00a3da0 0x7f34c00a6260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009bc0 tx=0x7f34cc005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:36.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.245+0000 7f34e6676740 1 -- start start 2024-04-16T17:19:36.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.245+0000 7f34e6676740 1 Processor -- start 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.246+0000 7f34e6676740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91022c1b0 0x55c910232680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.246+0000 7f34e6676740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910232bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.246+0000 7f34e6676740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9102356b0 0x55c910233100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91022c1b0 0x55c910232680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910232bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910232bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55014/0 (socket says 172.21.15.18:55014) 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91022c1b0 0x55c910232680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43108/0 (socket says 172.21.15.18:43108) 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c910238440 con 0x55c9102356b0 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34dbe00640 1 -- 172.21.15.18:0/1194232414 learned_addr learned my addr 172.21.15.18:0/1194232414 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c91014b710 con 0x55c91022cad0 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c9102385c0 con 0x55c91022c1b0 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.247+0000 7f34e6676740 10 client.0 _ll_register_callbacks cb 0x55c910146af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.248+0000 7f34dbe00640 1 -- 172.21.15.18:0/1194232414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91022c1b0 msgr2=0x55c910232680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.248+0000 7f34dbe00640 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91022c1b0 0x55c910232680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.248+0000 7f34dbe00640 1 -- 172.21.15.18:0/1194232414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9102356b0 msgr2=0x55c910233100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:36.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.248+0000 7f34dbe00640 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9102356b0 0x55c910233100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.248+0000 7f34dbe00640 1 -- 172.21.15.18:0/1194232414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c90fea43f0 con 0x55c91022cad0 2024-04-16T17:19:36.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.250+0000 7f34dbe00640 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910232bc0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f34c802cc70 tx=0x7f34c8005190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8d5bd36cd9fa847 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:36.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.251+0000 7f34c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:36.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.251+0000 7f34c6a00640 1 -- 172.21.15.18:0/1194232414 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c805c070 con 0x55c91022cad0 2024-04-16T17:19:36.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.251+0000 7f34c6a00640 1 -- 172.21.15.18:0/1194232414 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f34c8047460 con 0x55c91022cad0 2024-04-16T17:19:36.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.251+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c91019fb40 con 0x55c91022cad0 2024-04-16T17:19:36.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.252+0000 7f34c6a00640 1 -- 172.21.15.18:0/1194232414 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f34c8042070 con 0x55c91022cad0 2024-04-16T17:19:36.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.252+0000 7f34e6676740 10 client.29627 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:36.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.252+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c90fe640f0 con 0x55c91022cad0 2024-04-16T17:19:36.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.252+0000 7f34c6a00640 1 -- 172.21.15.18:0/1194232414 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c91019fb40 con 0x55c91022cad0 2024-04-16T17:19:36.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.253+0000 7f34c6a00640 1 -- 172.21.15.18:0/1194232414 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f34c8061610 con 0x55c91022cad0 2024-04-16T17:19:36.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.254+0000 7f34e6676740 10 client.29627 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:36.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.254+0000 7f34e6676740 10 client.29627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:36.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.254+0000 7f34e6676740 20 client.29627 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:36.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.254+0000 7f34e6676740 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c910245800 0x55c910247bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.254+0000 7f34e6676740 4 client.29627 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:36.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.254+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9101999b0 con 0x55c910245800 2024-04-16T17:19:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.255+0000 7f34e1000640 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c910245800 0x55c910247bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.255+0000 7f34e1000640 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c910245800 0x55c910247bf0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8e544de048a884 server_cookie=d3e4d253d7c9321f in_seq=0 out_seq=0 2024-04-16T17:19:36.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.256+0000 7f34c6a00640 10 client.29627 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.258+0000 7f34c6a00640 1 -- 172.21.15.18:0/1194232414 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9101999b0 con 0x55c910245800 2024-04-16T17:19:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.258+0000 7f34c6a00640 10 client.29627 handle_command_reply: tid=0 2024-04-16T17:19:36.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.258+0000 7f34e6676740 1 client.29627 shutdown 2024-04-16T17:19:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.259+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 msgr2=0x55c910232bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.259+0000 7f34e6676740 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910232bc0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f34c802cc70 tx=0x7f34c8005190 comp rx=0 tx=0).stop 2024-04-16T17:19:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.260+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 shutdown_connections 2024-04-16T17:19:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.260+0000 7f34e6676740 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c910245800 0x55c910247bf0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.260+0000 7f34e6676740 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c9102356b0 0x55c910233100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.260+0000 7f34e6676740 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910232bc0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.260+0000 7f34e6676740 1 --2- 172.21.15.18:0/1194232414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c91022c1b0 0x55c910232680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.260+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 >> 172.21.15.18:0/1194232414 conn(0x55c9102268e0 msgr2=0x55c910228630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.260+0000 7f34c6a00640 0 client.29627 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.261+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 shutdown_connections 2024-04-16T17:19:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.261+0000 7f34e6676740 1 -- 172.21.15.18:0/1194232414 wait complete. 2024-04-16T17:19:36.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.262+0000 7f34e6676740 20 client.29627 trim_cache size 0 max 16384 2024-04-16T17:19:36.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.300+0000 7f34e6676740 1 Processor -- start 2024-04-16T17:19:36.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.300+0000 7f34e6676740 1 -- start start 2024-04-16T17:19:36.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e6676740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022c1b0 0x55c910234db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e6676740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910241390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e6676740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910221760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e6676740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc012a80 con 0x55c91022c1b0 2024-04-16T17:19:36.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e6676740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc0102f0 con 0x55c91022cad0 2024-04-16T17:19:36.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e6676740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc00e160 con 0x55c9102356b0 2024-04-16T17:19:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910241390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e1000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022c1b0 0x55c910234db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e1000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022c1b0 0x55c910234db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46944/0 (socket says 172.21.15.18:46944) 2024-04-16T17:19:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.301+0000 7f34e1000640 1 -- 172.21.15.18:0/811207223 learned_addr learned my addr 172.21.15.18:0/811207223 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:36.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.302+0000 7f34e1a00640 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910221760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.302+0000 7f34e1000640 1 -- 172.21.15.18:0/811207223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 msgr2=0x55c910221760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.302+0000 7f34e1000640 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910221760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.302+0000 7f34e1000640 1 -- 172.21.15.18:0/811207223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 msgr2=0x55c910241390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.302+0000 7f34e1000640 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910241390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.302+0000 7f34e1000640 1 -- 172.21.15.18:0/811207223 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c910231630 con 0x55c91022c1b0 2024-04-16T17:19:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.302+0000 7f34dbe00640 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910241390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:19:36.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.303+0000 7f34e1000640 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022c1b0 0x55c910234db0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f34d401d000 tx=0x7f34d40d3420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788ac7545c1f9fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.303+0000 7f34d1600640 1 -- 172.21.15.18:0/811207223 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4066070 con 0x55c91022c1b0 2024-04-16T17:19:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.303+0000 7f34d1600640 1 -- 172.21.15.18:0/811207223 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4088040 con 0x55c91022c1b0 2024-04-16T17:19:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.303+0000 7f34d1600640 1 -- 172.21.15.18:0/811207223 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40bd240 con 0x55c91022c1b0 2024-04-16T17:19:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 -- 172.21.15.18:0/811207223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022c1b0 msgr2=0x55c910234db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022c1b0 0x55c910234db0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f34d401d000 tx=0x7f34d40d3420 comp rx=0 tx=0).stop 2024-04-16T17:19:36.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 -- 172.21.15.18:0/811207223 shutdown_connections 2024-04-16T17:19:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910221760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022cad0 0x55c910241390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 --2- 172.21.15.18:0/811207223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022c1b0 0x55c910234db0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 -- 172.21.15.18:0/811207223 >> 172.21.15.18:0/811207223 conn(0x55c9102268e0 msgr2=0x55c910227d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 -- 172.21.15.18:0/811207223 shutdown_connections 2024-04-16T17:19:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.304+0000 7f34e6676740 1 -- 172.21.15.18:0/811207223 wait complete. 2024-04-16T17:19:36.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.305+0000 7f34e6676740 1 -- start start 2024-04-16T17:19:36.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.305+0000 7f34e6676740 1 Processor -- start 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.306+0000 7f34e6676740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91023db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.306+0000 7f34e6676740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910238ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.306+0000 7f34e6676740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910239400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.306+0000 7f34e6676740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc00e7b0 con 0x55c91022cad0 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.306+0000 7f34e6676740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc00fc20 con 0x55c91022c1b0 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.306+0000 7f34e6676740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc00ee80 con 0x55c9102356b0 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.306+0000 7f34e6676740 10 client.0 _ll_register_callbacks cb 0x7f34c8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.306+0000 7f34e1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91023db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34e1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91023db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55030/0 (socket says 172.21.15.18:55030) 2024-04-16T17:19:36.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34e1000640 1 -- 172.21.15.18:0/3415878823 learned_addr learned my addr 172.21.15.18:0/3415878823 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:36.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34e1a00640 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910239400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34dbe00640 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910238ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34e1000640 1 -- 172.21.15.18:0/3415878823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 msgr2=0x55c910239400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:36.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34e1000640 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910239400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34e1000640 1 -- 172.21.15.18:0/3415878823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 msgr2=0x55c910238ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:36.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34e1000640 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910238ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.307+0000 7f34e1000640 1 -- 172.21.15.18:0/3415878823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c910239af0 con 0x55c91022c1b0 2024-04-16T17:19:36.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.308+0000 7f34e1000640 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91023db50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f34d4011f60 tx=0x7f34d409c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49aa21635a7d8fc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:36.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.308+0000 7f34c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.308+0000 7f34c6a00640 1 -- 172.21.15.18:0/3415878823 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4066050 con 0x55c91022c1b0 2024-04-16T17:19:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.308+0000 7f34c6a00640 1 -- 172.21.15.18:0/3415878823 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f34d40ce5b0 con 0x55c91022c1b0 2024-04-16T17:19:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.308+0000 7f34e6676740 1 -- 172.21.15.18:0/3415878823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d4088040 con 0x55c91022c1b0 2024-04-16T17:19:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.309+0000 7f34c6a00640 1 -- 172.21.15.18:0/3415878823 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f34d40bd040 con 0x55c91022c1b0 2024-04-16T17:19:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.309+0000 7f34e6676740 10 client.29633 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.309+0000 7f34e6676740 1 -- 172.21.15.18:0/3415878823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c9102aba00 con 0x55c91022c1b0 2024-04-16T17:19:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.309+0000 7f34c6a00640 1 -- 172.21.15.18:0/3415878823 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4088040 con 0x55c91022c1b0 2024-04-16T17:19:36.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.310+0000 7f34c6a00640 1 -- 172.21.15.18:0/3415878823 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f34d4022030 con 0x55c91022c1b0 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.310+0000 7f34e6676740 10 client.29633 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.310+0000 7f34e6676740 10 client.29633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.310+0000 7f34e6676740 20 client.29633 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.310+0000 7f34e6676740 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9102ac130 0x55c9102ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.310+0000 7f34e6676740 4 client.29633 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.310+0000 7f34e6676740 1 -- 172.21.15.18:0/3415878823 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c910115420 con 0x55c9102ac130 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.310+0000 7f34dbe00640 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9102ac130 0x55c9102ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.311+0000 7f34dbe00640 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9102ac130 0x55c9102ae550 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac43f7749380058 server_cookie=7804c678a2f30d55 in_seq=0 out_seq=0 2024-04-16T17:19:36.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.311+0000 7f34c6a00640 10 client.29633 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:36.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34c6a00640 1 -- 172.21.15.18:0/3415878823 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55c910115420 con 0x55c9102ac130 2024-04-16T17:19:36.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34c6a00640 10 client.29633 handle_command_reply: tid=0 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 client.29633 shutdown 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 -- 172.21.15.18:0/3415878823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 msgr2=0x55c91023db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91023db50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f34d4011f60 tx=0x7f34d409c270 comp rx=0 tx=0).stop 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 -- 172.21.15.18:0/3415878823 shutdown_connections 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c9102ac130 0x55c9102ae550 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c9102356b0 0x55c910239400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c91022cad0 0x55c910238ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 --2- 172.21.15.18:0/3415878823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c91022c1b0 0x55c91023db50 secure :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f34d4011f60 tx=0x7f34d409c270 comp rx=0 tx=0).stop 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 -- 172.21.15.18:0/3415878823 >> 172.21.15.18:0/3415878823 conn(0x55c9102268e0 msgr2=0x55c9102258d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34c6a00640 0 client.29633 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 -- 172.21.15.18:0/3415878823 shutdown_connections 2024-04-16T17:19:36.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.313+0000 7f34e6676740 1 -- 172.21.15.18:0/3415878823 wait complete. 2024-04-16T17:19:36.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.314+0000 7f34e6676740 20 client.29633 trim_cache size 0 max 16384 2024-04-16T17:19:36.315 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:19:36.315 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (427 inodes in the stack)", 2024-04-16T17:19:36.315 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:19:36.315 INFO:teuthology.orchestra.run.smithi018.stdout: "6388226a-5c44-4c78-9e3f-25a0a6c524f9": { 2024-04-16T17:19:36.315 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:19:36.316 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "6388226a-5c44-4c78-9e3f-25a0a6c524f9", 2024-04-16T17:19:36.316 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:19:36.316 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:19:36.316 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:19:36.316 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:19:36.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.315+0000 7f34e3a00640 1 -- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34c00a3da0 msgr2=0x7f34c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.315+0000 7f34e3a00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34c00a3da0 0x7f34c00a6260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009bc0 tx=0x7f34cc005f90 comp rx=0 tx=0).stop 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.315+0000 7f34e3a00640 1 -- 172.21.15.18:0/3695780007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc102230 msgr2=0x7f34dc10ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.315+0000 7f34e3a00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc102230 0x7f34dc10ebb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f34d400ea20 tx=0x7f34d400eee0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.316+0000 7f34e3a00640 1 -- 172.21.15.18:0/3695780007 shutdown_connections 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.316+0000 7f34e3a00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34c00a3da0 0x7f34c00a6260 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.316+0000 7f34e3a00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34dc10c190 0x7f34dc10c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.316+0000 7f34e3a00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34dc107300 0x7f34dc10bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.316+0000 7f34e3a00640 1 --2- 172.21.15.18:0/3695780007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34dc102230 0x7f34dc10ebb0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.316+0000 7f34e3a00640 1 -- 172.21.15.18:0/3695780007 >> 172.21.15.18:0/3695780007 conn(0x7f34dc0fdab0 msgr2=0x7f34dc0fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.316+0000 7f34e3a00640 1 -- 172.21.15.18:0/3695780007 shutdown_connections 2024-04-16T17:19:36.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:19:36.316+0000 7f34e3a00640 1 -- 172.21.15.18:0/3695780007 wait complete. 2024-04-16T17:19:36.340 INFO:tasks.cephfs.filesystem:scrub status for tag:6388226a-5c44-4c78-9e3f-25a0a6c524f9 - {'path': '/', 'tag': '6388226a-5c44-4c78-9e3f-25a0a6c524f9', 'options': 'recursive,force'} 2024-04-16T17:19:36.348 INFO:tasks.workunit.client.0.smithi018.stdout: 10 28348 15.72 MB/sec warmup 108 sec latency 259.832 ms 2024-04-16T17:19:36.474 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:36 smithi022 ceph-mon[36023]: pgmap v472: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 473 op/s 2024-04-16T17:19:36.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:36 smithi043 ceph-mon[36901]: pgmap v472: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 473 op/s 2024-04-16T17:19:37.352 INFO:tasks.workunit.client.0.smithi018.stdout: 10 28678 15.76 MB/sec warmup 109 sec latency 182.476 ms 2024-04-16T17:19:37.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:19:37.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:19:37.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:19:37.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:19:37.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:19:37.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:37 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:19:37.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:19:37.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:37 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:34 smithi018 ceph-mon[31776]: pgmap v471: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:34 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1653442378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:36 smithi018 ceph-mon[31776]: pgmap v472: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 473 op/s 2024-04-16T17:19:37.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:19:37.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:19:37.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:19:37.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:19:37.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:37.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:19:37.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:37 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:38.499 INFO:tasks.workunit.client.0.smithi018.stdout: 10 28745 15.63 MB/sec warmup 110 sec latency 726.246 ms 2024-04-16T17:19:38.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:38 smithi022 ceph-mon[36023]: pgmap v473: 129 pgs: 129 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 567 op/s 2024-04-16T17:19:38.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:38 smithi043 ceph-mon[36901]: pgmap v473: 129 pgs: 129 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 567 op/s 2024-04-16T17:19:39.129 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:38 smithi018 ceph-mon[31776]: pgmap v473: 129 pgs: 129 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 567 op/s 2024-04-16T17:19:39.458 INFO:tasks.workunit.client.0.smithi018.stdout: 10 29104 15.67 MB/sec warmup 111 sec latency 340.804 ms 2024-04-16T17:19:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 -- 172.21.15.18:0/2797322954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac074d50 msgr2=0x7fb2ac10e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 --2- 172.21.15.18:0/2797322954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac074d50 0x7fb2ac10e310 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb294009900 tx=0x7fb29402bce0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 -- 172.21.15.18:0/2797322954 shutdown_connections 2024-04-16T17:19:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 --2- 172.21.15.18:0/2797322954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ac10e850 0x7fb2ac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 --2- 172.21.15.18:0/2797322954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac074d50 0x7fb2ac10e310 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 --2- 172.21.15.18:0/2797322954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ac074430 0x7fb2ac074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 -- 172.21.15.18:0/2797322954 >> 172.21.15.18:0/2797322954 conn(0x7fb2ac0fdac0 msgr2=0x7fb2ac0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 -- 172.21.15.18:0/2797322954 shutdown_connections 2024-04-16T17:19:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.540+0000 7fb2b1800640 1 -- 172.21.15.18:0/2797322954 wait complete. 2024-04-16T17:19:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.541+0000 7fb2b1800640 1 Processor -- start 2024-04-16T17:19:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2b1800640 1 -- start start 2024-04-16T17:19:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2b1800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ac074430 0x7fb2ac1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2b1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ac074d50 0x7fb2ac1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2b1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac10e850 0x7fb2ac19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2b1800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1146a0 con 0x7fb2ac074430 2024-04-16T17:19:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2b1800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac114520 con 0x7fb2ac074d50 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2b1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac114820 con 0x7fb2ac10e850 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ac074d50 0x7fb2ac1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2aaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ac074430 0x7fb2ac1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.542+0000 7fb2ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac10e850 0x7fb2ac19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.543+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ac074d50 0x7fb2ac1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45158/0 (socket says 172.21.15.18:45158) 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.543+0000 7fb2a3e00640 1 -- 172.21.15.18:0/1302775861 learned_addr learned my addr 172.21.15.18:0/1302775861 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.543+0000 7fb2ab400640 1 -- 172.21.15.18:0/1302775861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ac074d50 msgr2=0x7fb2ac1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.543+0000 7fb2ab400640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ac074d50 0x7fb2ac1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.543+0000 7fb2ab400640 1 -- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ac074430 msgr2=0x7fb2ac1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.543+0000 7fb2ab400640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ac074430 0x7fb2ac1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.543+0000 7fb2ab400640 1 -- 172.21.15.18:0/1302775861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ac19b110 con 0x7fb2ac10e850 2024-04-16T17:19:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.543+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ac074d50 0x7fb2ac1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.544+0000 7fb2ab400640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac10e850 0x7fb2ac19a9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009f20 tx=0x7fb2a4009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.544+0000 7fb2a3400640 1 -- 172.21.15.18:0/1302775861 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a403b070 con 0x7fb2ac10e850 2024-04-16T17:19:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.544+0000 7fb2a3400640 1 -- 172.21.15.18:0/1302775861 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2a4036920 con 0x7fb2ac10e850 2024-04-16T17:19:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.544+0000 7fb2a3400640 1 -- 172.21.15.18:0/1302775861 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4034950 con 0x7fb2ac10e850 2024-04-16T17:19:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.544+0000 7fb2b1800640 1 -- 172.21.15.18:0/1302775861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ac19b400 con 0x7fb2ac10e850 2024-04-16T17:19:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.545+0000 7fb2b1800640 1 -- 172.21.15.18:0/1302775861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ac06ba20 con 0x7fb2ac10e850 2024-04-16T17:19:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.549+0000 7fb2b4470740 1 Processor -- start 2024-04-16T17:19:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.550+0000 7fb2a3400640 1 -- 172.21.15.18:0/1302775861 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2a403e070 con 0x7fb2ac10e850 2024-04-16T17:19:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.550+0000 7fb2b4470740 1 -- start start 2024-04-16T17:19:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.550+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x558391543b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583915440b0 0x558391544530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55839154d630 0x55839154fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2b4470740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5583914a0780 con 0x5583915440b0 2024-04-16T17:19:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2b4470740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558391446f40 con 0x55839154d630 2024-04-16T17:19:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2b4470740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55839149c740 con 0x558391544e20 2024-04-16T17:19:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55839154d630 0x55839154fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55839154d630 0x55839154fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45170/0 (socket says 172.21.15.18:45170) 2024-04-16T17:19:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2a3e00640 1 -- 172.21.15.18:0/610140916 learned_addr learned my addr 172.21.15.18:0/610140916 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.551+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/610140916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x558391543b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.552+0000 7fb2a3400640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2900a3d90 0x7fb2900a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.552+0000 7fb2a3e00640 1 -- 172.21.15.18:0/610140916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 msgr2=0x558391543b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.552+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/610140916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x558391543b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.552+0000 7fb2a3e00640 1 -- 172.21.15.18:0/610140916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583915440b0 msgr2=0x558391544530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.552+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/610140916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583915440b0 0x558391544530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.552+0000 7fb2a3e00640 1 -- 172.21.15.18:0/610140916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583911b3210 con 0x55839154d630 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.552+0000 7fb2a3400640 1 -- 172.21.15.18:0/1302775861 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fb2a40f2500 con 0x7fb2ac10e850 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.552+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/610140916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55839154d630 0x55839154fa20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb294004410 tx=0x7fb294000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35ae4fe63746c67e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.553+0000 7fb2ab400640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2900a3d90 0x7fb2900a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.553+0000 7fb29be00640 1 -- 172.21.15.18:0/610140916 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294048070 con 0x55839154d630 2024-04-16T17:19:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.553+0000 7fb29be00640 1 -- 172.21.15.18:0/610140916 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29402ce10 con 0x55839154d630 2024-04-16T17:19:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.553+0000 7fb29be00640 1 -- 172.21.15.18:0/610140916 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294004920 con 0x55839154d630 2024-04-16T17:19:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.554+0000 7fb2b4470740 1 -- 172.21.15.18:0/610140916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55839154d630 msgr2=0x55839154fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.554+0000 7fb2b4470740 1 --2- 172.21.15.18:0/610140916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55839154d630 0x55839154fa20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb294004410 tx=0x7fb294000bc0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.554+0000 7fb2ab400640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2900a3d90 0x7fb2900a6250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40ef950 tx=0x7fb2a4030040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.554+0000 7fb2b4470740 1 -- 172.21.15.18:0/610140916 shutdown_connections 2024-04-16T17:19:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.554+0000 7fb2b4470740 1 --2- 172.21.15.18:0/610140916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55839154d630 0x55839154fa20 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.554+0000 7fb2b4470740 1 --2- 172.21.15.18:0/610140916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5583915440b0 0x558391544530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.554+0000 7fb2b4470740 1 --2- 172.21.15.18:0/610140916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x558391543b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.554+0000 7fb2b4470740 1 -- 172.21.15.18:0/610140916 >> 172.21.15.18:0/610140916 conn(0x5583914d65d0 msgr2=0x55839153eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.555+0000 7fb2b4470740 1 -- 172.21.15.18:0/610140916 shutdown_connections 2024-04-16T17:19:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.555+0000 7fb2b4470740 1 -- 172.21.15.18:0/610140916 wait complete. 2024-04-16T17:19:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.556+0000 7fb2b4470740 1 -- start start 2024-04-16T17:19:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.556+0000 7fb2b4470740 1 Processor -- start 2024-04-16T17:19:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x55839154a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x55839154aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 0x55839154afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2b4470740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558391550300 con 0x55839154d630 2024-04-16T17:19:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2b4470740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55839145a390 con 0x5583915440b0 2024-04-16T17:19:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2b4470740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558391550480 con 0x558391544e20 2024-04-16T17:19:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2b4470740 10 client.0 _ll_register_callbacks cb 0x558391443f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x55839154aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x55839154aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42490/0 (socket says 172.21.15.18:42490) 2024-04-16T17:19:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.557+0000 7fb2aaa00640 1 -- 172.21.15.18:0/4047836547 learned_addr learned my addr 172.21.15.18:0/4047836547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.558+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x55839154a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.558+0000 7fb2aaa00640 1 -- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 msgr2=0x55839154a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.558+0000 7fb2aaa00640 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x55839154a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.558+0000 7fb2aaa00640 1 -- 172.21.15.18:0/4047836547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 msgr2=0x55839154afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.558+0000 7fb2aaa00640 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 0x55839154afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.558+0000 7fb2aaa00640 1 -- 172.21.15.18:0/4047836547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55839127f620 con 0x558391544e20 2024-04-16T17:19:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.558+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x55839154a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.559+0000 7fb2aaa00640 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x55839154aa70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0116c0 tx=0x7fb29c011b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231fe9587ba26469 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.559+0000 7fb298c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:19:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.560+0000 7fb298c00640 1 -- 172.21.15.18:0/4047836547 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00fd30 con 0x558391544e20 2024-04-16T17:19:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.560+0000 7fb298c00640 1 -- 172.21.15.18:0/4047836547 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fb29c004020 con 0x558391544e20 2024-04-16T17:19:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.560+0000 7fb2b4470740 1 -- 172.21.15.18:0/4047836547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29402ce10 con 0x558391544e20 2024-04-16T17:19:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.560+0000 7fb298c00640 1 -- 172.21.15.18:0/4047836547 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb29c004220 con 0x558391544e20 2024-04-16T17:19:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.560+0000 7fb2b4470740 10 client.29473 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.560+0000 7fb2b4470740 1 -- 172.21.15.18:0/4047836547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55839155c550 con 0x558391544e20 2024-04-16T17:19:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.561+0000 7fb298c00640 1 -- 172.21.15.18:0/4047836547 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29402ce10 con 0x558391544e20 2024-04-16T17:19:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.561+0000 7fb298c00640 1 -- 172.21.15.18:0/4047836547 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb29c004b80 con 0x558391544e20 2024-04-16T17:19:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.562+0000 7fb2b4470740 10 client.29473 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.562+0000 7fb2b4470740 10 client.29473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.562+0000 7fb2b4470740 20 client.29473 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.562+0000 7fb2b4470740 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55839155da80 0x55839155fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.562+0000 7fb2b4470740 4 client.29473 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.562+0000 7fb2b4470740 1 -- 172.21.15.18:0/4047836547 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583914ae1a0 con 0x55839155da80 2024-04-16T17:19:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.562+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55839155da80 0x55839155fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.564+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55839155da80 0x55839155fe70 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=757d88d933529c22 server_cookie=3e5f8670e96972f1 in_seq=0 out_seq=0 2024-04-16T17:19:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.564+0000 7fb298c00640 10 client.29473 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.569+0000 7fb298c00640 1 -- 172.21.15.18:0/4047836547 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583914ae1a0 con 0x55839155da80 2024-04-16T17:19:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.572+0000 7fb298c00640 10 client.29473 handle_command_reply: tid=0 2024-04-16T17:19:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.573+0000 7fb2b4470740 1 client.29473 shutdown 2024-04-16T17:19:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.573+0000 7fb2b4470740 1 -- 172.21.15.18:0/4047836547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 msgr2=0x55839154aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.573+0000 7fb2b4470740 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x55839154aa70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0116c0 tx=0x7fb29c011b80 comp rx=0 tx=0).stop 2024-04-16T17:19:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.574+0000 7fb2b4470740 1 -- 172.21.15.18:0/4047836547 shutdown_connections 2024-04-16T17:19:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.574+0000 7fb2b4470740 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55839155da80 0x55839155fe70 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.574+0000 7fb2b4470740 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 0x55839154afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.574+0000 7fb2b4470740 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558391544e20 0x55839154aa70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.574+0000 7fb2b4470740 1 --2- 172.21.15.18:0/4047836547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x55839154a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.574+0000 7fb298c00640 0 client.29473 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.574+0000 7fb2b4470740 1 -- 172.21.15.18:0/4047836547 >> 172.21.15.18:0/4047836547 conn(0x5583914d65d0 msgr2=0x558391540af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.575+0000 7fb2b4470740 1 -- 172.21.15.18:0/4047836547 shutdown_connections 2024-04-16T17:19:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.578+0000 7fb2b4470740 1 -- 172.21.15.18:0/4047836547 wait complete. 2024-04-16T17:19:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.578+0000 7fb2b4470740 20 client.29473 trim_cache size 0 max 16384 2024-04-16T17:19:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.608+0000 7fb2b4470740 1 Processor -- start 2024-04-16T17:19:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.610+0000 7fb2b4470740 1 -- start start 2024-04-16T17:19:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.610+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583915440b0 0x558391559200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.610+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558391544e20 0x558391535530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.610+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 0x5583915657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.610+0000 7fb2b4470740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1a7720 con 0x55839154d630 2024-04-16T17:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.610+0000 7fb2b4470740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac071ae0 con 0x558391544e20 2024-04-16T17:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.610+0000 7fb2b4470740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac06f950 con 0x5583915440b0 2024-04-16T17:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.610+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583915440b0 0x558391559200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583915440b0 0x558391559200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42506/0 (socket says 172.21.15.18:42506) 2024-04-16T17:19:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 -- 172.21.15.18:0/427522352 learned_addr learned my addr 172.21.15.18:0/427522352 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/427522352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 0x5583915657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 -- 172.21.15.18:0/427522352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558391544e20 msgr2=0x558391535530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/427522352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558391544e20 0x558391535530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 -- 172.21.15.18:0/427522352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 msgr2=0x5583915657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/427522352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 0x5583915657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 -- 172.21.15.18:0/427522352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583915601e0 con 0x5583915440b0 2024-04-16T17:19:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.611+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/427522352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583915440b0 0x558391559200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb294004440 tx=0x7fb2940041f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8004837d50ea10 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.612+0000 7fb29b400640 1 -- 172.21.15.18:0/427522352 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294048040 con 0x5583915440b0 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.612+0000 7fb29b400640 1 -- 172.21.15.18:0/427522352 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2940056e0 con 0x5583915440b0 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.612+0000 7fb29b400640 1 -- 172.21.15.18:0/427522352 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29402ce50 con 0x5583915440b0 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.612+0000 7fb2b4470740 1 -- 172.21.15.18:0/427522352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583915440b0 msgr2=0x558391559200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.612+0000 7fb2b4470740 1 --2- 172.21.15.18:0/427522352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583915440b0 0x558391559200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb294004440 tx=0x7fb2940041f0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 -- 172.21.15.18:0/427522352 shutdown_connections 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 --2- 172.21.15.18:0/427522352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55839154d630 0x5583915657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 --2- 172.21.15.18:0/427522352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558391544e20 0x558391535530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 --2- 172.21.15.18:0/427522352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5583915440b0 0x558391559200 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 -- 172.21.15.18:0/427522352 >> 172.21.15.18:0/427522352 conn(0x5583914d65d0 msgr2=0x558391534e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 -- 172.21.15.18:0/427522352 shutdown_connections 2024-04-16T17:19:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 -- 172.21.15.18:0/427522352 wait complete. 2024-04-16T17:19:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 -- start start 2024-04-16T17:19:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.613+0000 7fb2b4470740 1 Processor -- start 2024-04-16T17:19:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.614+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x558391555710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.614+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558391544e20 0x558391555c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.614+0000 7fb2b4470740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55839154d630 0x5583915526e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.614+0000 7fb2b4470740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac06ffa0 con 0x558391544e20 2024-04-16T17:19:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.614+0000 7fb2b4470740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac071410 con 0x5583915440b0 2024-04-16T17:19:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.614+0000 7fb2b4470740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac070670 con 0x55839154d630 2024-04-16T17:19:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.614+0000 7fb2b4470740 10 client.0 _ll_register_callbacks cb 0x7fb29c009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.615+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x558391555710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.615+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x558391555710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45206/0 (socket says 172.21.15.18:45206) 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.615+0000 7fb2a3e00640 1 -- 172.21.15.18:0/465745967 learned_addr learned my addr 172.21.15.18:0/465745967 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.615+0000 7fb2aaa00640 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558391544e20 0x558391555c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.615+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55839154d630 0x5583915526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.616+0000 7fb2a3e00640 1 -- 172.21.15.18:0/465745967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55839154d630 msgr2=0x5583915526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.616+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55839154d630 0x5583915526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.616+0000 7fb2a3e00640 1 -- 172.21.15.18:0/465745967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558391544e20 msgr2=0x558391555c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.616+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558391544e20 0x558391555c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.616+0000 7fb2a3e00640 1 -- 172.21.15.18:0/465745967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558391552ca0 con 0x5583915440b0 2024-04-16T17:19:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.616+0000 7fb2a3e00640 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x558391555710 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb294036040 tx=0x7fb29402c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e58c497d244555f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.616+0000 7fb298c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:19:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.616+0000 7fb298c00640 1 -- 172.21.15.18:0/465745967 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294048070 con 0x5583915440b0 2024-04-16T17:19:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.617+0000 7fb298c00640 1 -- 172.21.15.18:0/465745967 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fb2940524d0 con 0x5583915440b0 2024-04-16T17:19:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.617+0000 7fb2b4470740 1 -- 172.21.15.18:0/465745967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2940056e0 con 0x5583915440b0 2024-04-16T17:19:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.617+0000 7fb298c00640 1 -- 172.21.15.18:0/465745967 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb2940526d0 con 0x5583915440b0 2024-04-16T17:19:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb2b4470740 10 client.29639 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb2b4470740 1 -- 172.21.15.18:0/465745967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5583915c48c0 con 0x5583915440b0 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb298c00640 1 -- 172.21.15.18:0/465745967 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2940056e0 con 0x5583915440b0 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb298c00640 1 -- 172.21.15.18:0/465745967 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb294034550 con 0x5583915440b0 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb2b4470740 10 client.29639 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb2b4470740 10 client.29639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb2b4470740 20 client.29639 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb2b4470740 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5583915c4b40 0x5583915c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb2b4470740 4 client.29639 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.620+0000 7fb2b4470740 1 -- 172.21.15.18:0/465745967 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583914a8010 con 0x5583915c4b40 2024-04-16T17:19:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.621+0000 7fb2aaa00640 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5583915c4b40 0x5583915c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.624+0000 7fb298c00640 10 client.29639 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:39.625+0000 7fb2aaa00640 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5583915c4b40 0x5583915c6f30 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e200818796bce7 server_cookie=d7ecb43d8aab0040 in_seq=0 out_seq=0 2024-04-16T17:19:40.458 INFO:tasks.workunit.client.0.smithi018.stdout: 10 29246 15.57 MB/sec warmup 112 sec latency 369.765 ms 2024-04-16T17:19:40.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:40 smithi022 ceph-mon[36023]: pgmap v474: 129 pgs: 129 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 421 op/s 2024-04-16T17:19:40.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:40 smithi043 ceph-mon[36901]: pgmap v474: 129 pgs: 129 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 421 op/s 2024-04-16T17:19:41.458 INFO:tasks.workunit.client.0.smithi018.stdout: 10 29608 15.62 MB/sec warmup 113 sec latency 307.224 ms 2024-04-16T17:19:42.459 INFO:tasks.workunit.client.0.smithi018.stdout: 10 29973 15.63 MB/sec warmup 114 sec latency 1045.946 ms 2024-04-16T17:19:42.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:42 smithi022 ceph-mon[36023]: pgmap v475: 129 pgs: 129 active+clean; 342 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 612 op/s 2024-04-16T17:19:42.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:42 smithi043 ceph-mon[36901]: pgmap v475: 129 pgs: 129 active+clean; 342 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 612 op/s 2024-04-16T17:19:43.459 INFO:tasks.workunit.client.0.smithi018.stdout: 10 30399 15.70 MB/sec warmup 115 sec latency 2046.052 ms 2024-04-16T17:19:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.907+0000 7fb298c00640 1 -- 172.21.15.18:0/465745967 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583914a8010 con 0x5583915c4b40 2024-04-16T17:19:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.907+0000 7fb298c00640 10 client.29639 handle_command_reply: tid=0 2024-04-16T17:19:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.907+0000 7fb2b4470740 1 client.29639 shutdown 2024-04-16T17:19:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.910+0000 7fb2b4470740 1 -- 172.21.15.18:0/465745967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 msgr2=0x558391555710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.910+0000 7fb2b4470740 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x558391555710 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb294036040 tx=0x7fb29402c360 comp rx=0 tx=0).stop 2024-04-16T17:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.910+0000 7fb2b4470740 1 -- 172.21.15.18:0/465745967 shutdown_connections 2024-04-16T17:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.910+0000 7fb2b4470740 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5583915c4b40 0x5583915c6f30 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.910+0000 7fb2b4470740 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55839154d630 0x5583915526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.910+0000 7fb2b4470740 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558391544e20 0x558391555c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.910+0000 7fb2b4470740 1 --2- 172.21.15.18:0/465745967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583915440b0 0x558391555710 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.910+0000 7fb2b4470740 1 -- 172.21.15.18:0/465745967 >> 172.21.15.18:0/465745967 conn(0x5583914d65d0 msgr2=0x55839154ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.912+0000 7fb298c00640 0 client.29639 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.913+0000 7fb2b4470740 1 -- 172.21.15.18:0/465745967 shutdown_connections 2024-04-16T17:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.913+0000 7fb2b4470740 1 -- 172.21.15.18:0/465745967 wait complete. 2024-04-16T17:19:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.914+0000 7fb2b4470740 20 client.29639 trim_cache size 0 max 16384 2024-04-16T17:19:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:19:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.916+0000 7fb2b1800640 1 -- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2900a3d90 msgr2=0x7fb2900a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.916+0000 7fb2b1800640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2900a3d90 0x7fb2900a6250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40ef950 tx=0x7fb2a4030040 comp rx=0 tx=0).stop 2024-04-16T17:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.916+0000 7fb2b1800640 1 -- 172.21.15.18:0/1302775861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac10e850 msgr2=0x7fb2ac19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac10e850 0x7fb2ac19a9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009f20 tx=0x7fb2a4009f50 comp rx=0 tx=0).stop 2024-04-16T17:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 -- 172.21.15.18:0/1302775861 shutdown_connections 2024-04-16T17:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb2900a3d90 0x7fb2900a6250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac10e850 0x7fb2ac19a9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2ac074d50 0x7fb2ac1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 --2- 172.21.15.18:0/1302775861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2ac074430 0x7fb2ac1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 -- 172.21.15.18:0/1302775861 >> 172.21.15.18:0/1302775861 conn(0x7fb2ac0fdac0 msgr2=0x7fb2ac0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 -- 172.21.15.18:0/1302775861 shutdown_connections 2024-04-16T17:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:43.917+0000 7fb2b1800640 1 -- 172.21.15.18:0/1302775861 wait complete. 2024-04-16T17:19:44.459 INFO:tasks.workunit.client.0.smithi018.stdout: 10 30822 15.75 MB/sec warmup 116 sec latency 2594.240 ms 2024-04-16T17:19:44.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:44 smithi022 ceph-mon[36023]: pgmap v476: 129 pgs: 129 active+clean; 343 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 475 op/s 2024-04-16T17:19:44.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:44 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:44.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:44 smithi043 ceph-mon[36901]: pgmap v476: 129 pgs: 129 active+clean; 343 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 475 op/s 2024-04-16T17:19:44.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:44 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:45.459 INFO:tasks.workunit.client.0.smithi018.stdout: 10 31101 15.79 MB/sec warmup 117 sec latency 192.449 ms 2024-04-16T17:19:46.459 INFO:tasks.workunit.client.0.smithi018.stdout: 10 31494 15.80 MB/sec warmup 118 sec latency 108.549 ms 2024-04-16T17:19:46.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:46 smithi022 ceph-mon[36023]: pgmap v477: 129 pgs: 129 active+clean; 343 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 415 op/s 2024-04-16T17:19:46.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:46 smithi043 ceph-mon[36901]: pgmap v477: 129 pgs: 129 active+clean; 343 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 415 op/s 2024-04-16T17:19:47.460 INFO:tasks.workunit.client.0.smithi018.stdout: 10 31826 15.88 MB/sec warmup 119 sec latency 50.385 ms 2024-04-16T17:19:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:40 smithi018 ceph-mon[31776]: pgmap v474: 129 pgs: 129 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 421 op/s 2024-04-16T17:19:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:42 smithi018 ceph-mon[31776]: pgmap v475: 129 pgs: 129 active+clean; 342 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 612 op/s 2024-04-16T17:19:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:44 smithi018 ceph-mon[31776]: pgmap v476: 129 pgs: 129 active+clean; 343 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 475 op/s 2024-04-16T17:19:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:44 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:19:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:46 smithi018 ceph-mon[31776]: pgmap v477: 129 pgs: 129 active+clean; 343 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 415 op/s 2024-04-16T17:19:48.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:48 smithi022 ceph-mon[36023]: pgmap v478: 129 pgs: 129 active+clean; 341 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-04-16T17:19:48.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:48 smithi043 ceph-mon[36901]: pgmap v478: 129 pgs: 129 active+clean; 341 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-04-16T17:19:49.460 INFO:tasks.workunit.client.0.smithi018.stdout: 10 32180 11.19 MB/sec execute 1 sec latency 864.510 ms 2024-04-16T17:19:49.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:49 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:19:49.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:49 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:19:50.460 INFO:tasks.workunit.client.0.smithi018.stdout: 10 32384 12.32 MB/sec execute 2 sec latency 249.434 ms 2024-04-16T17:19:50.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:50 smithi022 ceph-mon[36023]: pgmap v479: 129 pgs: 129 active+clean; 337 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 442 op/s 2024-04-16T17:19:50.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:50 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:19:50.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:50 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:19:50.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:50 smithi043 ceph-mon[36901]: pgmap v479: 129 pgs: 129 active+clean; 337 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 442 op/s 2024-04-16T17:19:50.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:50 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:19:50.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:50 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:19:51.460 INFO:tasks.workunit.client.0.smithi018.stdout: 10 32715 14.16 MB/sec execute 3 sec latency 58.693 ms 2024-04-16T17:19:51.871 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:48 smithi018 ceph-mon[31776]: pgmap v478: 129 pgs: 129 active+clean; 341 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-04-16T17:19:51.871 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:49 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:19:51.871 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:50 smithi018 ceph-mon[31776]: pgmap v479: 129 pgs: 129 active+clean; 337 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 442 op/s 2024-04-16T17:19:51.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:50 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:19:51.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:50 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:19:52.460 INFO:tasks.workunit.client.0.smithi018.stdout: 10 33001 13.73 MB/sec execute 4 sec latency 108.240 ms 2024-04-16T17:19:52.546 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:52 smithi018 ceph-mon[31776]: pgmap v480: 129 pgs: 129 active+clean; 345 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 489 op/s 2024-04-16T17:19:52.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:52 smithi022 ceph-mon[36023]: pgmap v480: 129 pgs: 129 active+clean; 345 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 489 op/s 2024-04-16T17:19:52.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:52 smithi043 ceph-mon[36901]: pgmap v480: 129 pgs: 129 active+clean; 345 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 489 op/s 2024-04-16T17:19:53.460 INFO:tasks.workunit.client.0.smithi018.stdout: 10 33273 13.13 MB/sec execute 5 sec latency 309.179 ms 2024-04-16T17:19:54.460 INFO:tasks.workunit.client.0.smithi018.stdout: 10 33648 14.22 MB/sec execute 6 sec latency 186.705 ms 2024-04-16T17:19:54.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:54 smithi022 ceph-mon[36023]: pgmap v481: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 307 op/s 2024-04-16T17:19:54.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:54 smithi043 ceph-mon[36901]: pgmap v481: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 307 op/s 2024-04-16T17:19:55.460 INFO:tasks.workunit.client.0.smithi018.stdout: 10 33938 14.12 MB/sec execute 7 sec latency 388.115 ms 2024-04-16T17:19:56.462 INFO:tasks.workunit.client.0.smithi018.stdout: 10 34294 14.67 MB/sec execute 8 sec latency 204.184 ms 2024-04-16T17:19:56.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:56 smithi022 ceph-mon[36023]: pgmap v482: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-04-16T17:19:56.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:56 smithi043 ceph-mon[36901]: pgmap v482: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-04-16T17:19:56.983 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:54 smithi018 ceph-mon[31776]: pgmap v481: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 307 op/s 2024-04-16T17:19:56.983 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:56 smithi018 ceph-mon[31776]: pgmap v482: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-04-16T17:19:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.095+0000 7f8d9d200640 1 -- 172.21.15.18:0/3794206180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d980743d0 msgr2=0x7f8d980747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.095+0000 7f8d9d200640 1 --2- 172.21.15.18:0/3794206180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d980743d0 0x7f8d980747b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009900 tx=0x7f8d8802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.096+0000 7f8d9d200640 1 -- 172.21.15.18:0/3794206180 shutdown_connections 2024-04-16T17:19:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.096+0000 7f8d9d200640 1 --2- 172.21.15.18:0/3794206180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d9810e6b0 0x7f8d98110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.096+0000 7f8d9d200640 1 --2- 172.21.15.18:0/3794206180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d98074cf0 0x7f8d9810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.096+0000 7f8d9d200640 1 --2- 172.21.15.18:0/3794206180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d980743d0 0x7f8d980747b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.096+0000 7f8d9d200640 1 -- 172.21.15.18:0/3794206180 >> 172.21.15.18:0/3794206180 conn(0x7f8d980fd940 msgr2=0x7f8d980ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.096+0000 7f8d9d200640 1 -- 172.21.15.18:0/3794206180 shutdown_connections 2024-04-16T17:19:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.096+0000 7f8d9d200640 1 -- 172.21.15.18:0/3794206180 wait complete. 2024-04-16T17:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.098+0000 7f8d9d200640 1 Processor -- start 2024-04-16T17:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.098+0000 7f8d9d200640 1 -- start start 2024-04-16T17:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.098+0000 7f8d9d200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d980743d0 0x7f8d9810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.098+0000 7f8d9d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d98074cf0 0x7f8d98104ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d9d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e6b0 0x7f8d98105430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d9d200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8d98103ae0 con 0x7f8d980743d0 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d9d200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d98103960 con 0x7f8d98074cf0 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d9d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d98103c60 con 0x7f8d9810e6b0 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d980743d0 0x7f8d9810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d980743d0 0x7f8d9810dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60582/0 (socket says 172.21.15.18:60582) 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d97400640 1 -- 172.21.15.18:0/195130997 learned_addr learned my addr 172.21.15.18:0/195130997 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d96a00640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d98074cf0 0x7f8d98104ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d97e00640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e6b0 0x7f8d98105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.099+0000 7f8d97400640 1 -- 172.21.15.18:0/195130997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e6b0 msgr2=0x7f8d98105430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.100+0000 7f8d97400640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e6b0 0x7f8d98105430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.100+0000 7f8d97400640 1 -- 172.21.15.18:0/195130997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d98074cf0 msgr2=0x7f8d98104ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.100+0000 7f8d97400640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d98074cf0 0x7f8d98104ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.100+0000 7f8d97400640 1 -- 172.21.15.18:0/195130997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d98105c00 con 0x7f8d980743d0 2024-04-16T17:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.101+0000 7f8d97400640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d980743d0 0x7f8d9810dea0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8d88000c00 tx=0x7f8d880043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.101+0000 7f8d8fe00640 1 -- 172.21.15.18:0/195130997 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88042020 con 0x7f8d980743d0 2024-04-16T17:19:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.102+0000 7f8d9d200640 1 -- 172.21.15.18:0/195130997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d981098e0 con 0x7f8d980743d0 2024-04-16T17:19:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.102+0000 7f8d9d200640 1 -- 172.21.15.18:0/195130997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d98109e80 con 0x7f8d980743d0 2024-04-16T17:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.104+0000 7f8d8fe00640 1 -- 172.21.15.18:0/195130997 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d88004c20 con 0x7f8d980743d0 2024-04-16T17:19:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.104+0000 7f8d8fe00640 1 -- 172.21.15.18:0/195130997 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880035f0 con 0x7f8d980743d0 2024-04-16T17:19:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.107+0000 7f8da1602740 1 Processor -- start 2024-04-16T17:19:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.107+0000 7f8da1602740 1 -- start start 2024-04-16T17:19:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.108+0000 7f8d8fe00640 1 -- 172.21.15.18:0/195130997 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8d88045020 con 0x7f8d980743d0 2024-04-16T17:19:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.108+0000 7f8da1602740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f534540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.109+0000 7f8da1602740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f534a80 0x564c5f534f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.109+0000 7f8da1602740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f5403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.109+0000 7f8d97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f534540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.109+0000 7f8d97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f534540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49602/0 (socket says 172.21.15.18:49602) 2024-04-16T17:19:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.109+0000 7f8d97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f534a80 0x564c5f534f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.109+0000 7f8d97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f534a80 0x564c5f534f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52306/0 (socket says 172.21.15.18:52306) 2024-04-16T17:19:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.110+0000 7f8da1602740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564c5f499780 con 0x564c5f53e000 2024-04-16T17:19:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.110+0000 7f8da1602740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564c5f43ff40 con 0x564c5f534a80 2024-04-16T17:19:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.110+0000 7f8da1602740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c5f495740 con 0x564c5f5357f0 2024-04-16T17:19:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.110+0000 7f8d8fe00640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8d7c0a3de0 0x7f8d7c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.110+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f5403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.110+0000 7f8d97e00640 1 -- 172.21.15.18:0/1233051967 learned_addr learned my addr 172.21.15.18:0/1233051967 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d8fe00640 1 -- 172.21.15.18:0/195130997 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f8d880f1fc0 con 0x7f8d980743d0 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d97400640 1 -- 172.21.15.18:0/1233051967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 msgr2=0x564c5f534540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d97400640 1 --2- 172.21.15.18:0/1233051967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f534540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d97400640 1 -- 172.21.15.18:0/1233051967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 msgr2=0x564c5f5403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d97400640 1 --2- 172.21.15.18:0/1233051967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f5403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d97400640 1 -- 172.21.15.18:0/1233051967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c5f1ac210 con 0x564c5f534a80 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d97400640 1 --2- 172.21.15.18:0/1233051967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f534a80 0x564c5f534f00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8d880ef410 tx=0x7f8d8802bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a54521cc265f6a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d97e00640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8d7c0a3de0 0x7f8d7c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d8cc00640 1 -- 172.21.15.18:0/1233051967 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880bd5c0 con 0x564c5f534a80 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d8cc00640 1 -- 172.21.15.18:0/1233051967 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d880be360 con 0x564c5f534a80 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.111+0000 7f8d8cc00640 1 -- 172.21.15.18:0/1233051967 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880bd5c0 con 0x564c5f534a80 2024-04-16T17:19:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.112+0000 7f8d96a00640 1 --2- 172.21.15.18:0/1233051967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f5403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:19:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.113+0000 7f8da1602740 1 -- 172.21.15.18:0/1233051967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f534a80 msgr2=0x564c5f534f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.113+0000 7f8da1602740 1 --2- 172.21.15.18:0/1233051967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f534a80 0x564c5f534f00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8d880ef410 tx=0x7f8d8802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8d97e00640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8d7c0a3de0 0x7f8d7c0a62a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8d90013940 tx=0x7f8d90009b70 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8da1602740 1 -- 172.21.15.18:0/1233051967 shutdown_connections 2024-04-16T17:19:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8da1602740 1 --2- 172.21.15.18:0/1233051967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f5403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8da1602740 1 --2- 172.21.15.18:0/1233051967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f534a80 0x564c5f534f00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8da1602740 1 --2- 172.21.15.18:0/1233051967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f534540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8da1602740 1 -- 172.21.15.18:0/1233051967 >> 172.21.15.18:0/1233051967 conn(0x564c5f4cf5d0 msgr2=0x564c5f52f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8da1602740 1 -- 172.21.15.18:0/1233051967 shutdown_connections 2024-04-16T17:19:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8da1602740 1 -- 172.21.15.18:0/1233051967 wait complete. 2024-04-16T17:19:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.114+0000 7f8da1602740 1 -- start start 2024-04-16T17:19:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.115+0000 7f8da1602740 1 Processor -- start 2024-04-16T17:19:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.115+0000 7f8da1602740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f53da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.115+0000 7f8da1602740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f53ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.115+0000 7f8da1602740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f53e000 0x564c5f53b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.115+0000 7f8da1602740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564c5f43ff40 con 0x564c5f534a80 2024-04-16T17:19:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.115+0000 7f8da1602740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564c5f453390 con 0x564c5f53e000 2024-04-16T17:19:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.115+0000 7f8da1602740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c5f540af0 con 0x564c5f5357f0 2024-04-16T17:19:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.115+0000 7f8da1602740 10 client.0 _ll_register_callbacks cb 0x564c5f43cf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f53ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f53ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49616/0 (socket says 172.21.15.18:49616) 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d96a00640 1 -- 172.21.15.18:0/3213915302 learned_addr learned my addr 172.21.15.18:0/3213915302 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d97400640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f53da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d97e00640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f53e000 0x564c5f53b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d96a00640 1 -- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f53e000 msgr2=0x564c5f53b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d96a00640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f53e000 0x564c5f53b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d96a00640 1 -- 172.21.15.18:0/3213915302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 msgr2=0x564c5f53da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d96a00640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f53da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.116+0000 7f8d96a00640 1 -- 172.21.15.18:0/3213915302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5f278620 con 0x564c5f5357f0 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.117+0000 7f8d97e00640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f53e000 0x564c5f53b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.117+0000 7f8d97400640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f53da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.117+0000 7f8d96a00640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f53ae10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8d84022650 tx=0x7f8d84022b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b6ef15633926ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.117+0000 7f8d81600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.117+0000 7f8d81600640 1 -- 172.21.15.18:0/3213915302 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84004320 con 0x564c5f5357f0 2024-04-16T17:19:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.117+0000 7f8d81600640 1 -- 172.21.15.18:0/3213915302 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f8d8400e910 con 0x564c5f5357f0 2024-04-16T17:19:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.117+0000 7f8da1602740 1 -- 172.21.15.18:0/3213915302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d880be360 con 0x564c5f5357f0 2024-04-16T17:19:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.119+0000 7f8d81600640 1 -- 172.21.15.18:0/3213915302 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8d840044c0 con 0x564c5f5357f0 2024-04-16T17:19:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.119+0000 7f8da1602740 10 client.29491 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.119+0000 7f8da1602740 1 -- 172.21.15.18:0/3213915302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564c5f54ca90 con 0x564c5f5357f0 2024-04-16T17:19:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.120+0000 7f8d81600640 1 -- 172.21.15.18:0/3213915302 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d880be360 con 0x564c5f5357f0 2024-04-16T17:19:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.121+0000 7f8d81600640 1 -- 172.21.15.18:0/3213915302 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8d84036580 con 0x564c5f5357f0 2024-04-16T17:19:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.122+0000 7f8da1602740 10 client.29491 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.122+0000 7f8da1602740 10 client.29491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.123+0000 7f8da1602740 20 client.29491 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.123+0000 7f8da1602740 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c5f54df70 0x564c5f550380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.123+0000 7f8da1602740 4 client.29491 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:19:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.123+0000 7f8da1602740 1 -- 172.21.15.18:0/3213915302 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c5f4a71a0 con 0x564c5f54df70 2024-04-16T17:19:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.123+0000 7f8d97400640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c5f54df70 0x564c5f550380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.124+0000 7f8d97400640 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c5f54df70 0x564c5f550380 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44fd68719db5e19 server_cookie=bf725989dae25175 in_seq=0 out_seq=0 2024-04-16T17:19:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.124+0000 7f8d81600640 10 client.29491 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8d81600640 1 -- 172.21.15.18:0/3213915302 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c5f4a71a0 con 0x564c5f54df70 2024-04-16T17:19:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8d81600640 10 client.29491 handle_command_reply: tid=0 2024-04-16T17:19:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8da1602740 1 client.29491 shutdown 2024-04-16T17:19:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8da1602740 1 -- 172.21.15.18:0/3213915302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 msgr2=0x564c5f53ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8da1602740 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f53ae10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8d84022650 tx=0x7f8d84022b10 comp rx=0 tx=0).stop 2024-04-16T17:19:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8da1602740 1 -- 172.21.15.18:0/3213915302 shutdown_connections 2024-04-16T17:19:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8da1602740 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c5f54df70 0x564c5f550380 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8da1602740 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f53e000 0x564c5f53b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.129+0000 7f8da1602740 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f5357f0 0x564c5f53ae10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.130+0000 7f8da1602740 1 --2- 172.21.15.18:0/3213915302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f53da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.130+0000 7f8d81600640 0 client.29491 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.130+0000 7f8da1602740 1 -- 172.21.15.18:0/3213915302 >> 172.21.15.18:0/3213915302 conn(0x564c5f4cf5d0 msgr2=0x564c5f5314c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.130+0000 7f8da1602740 1 -- 172.21.15.18:0/3213915302 shutdown_connections 2024-04-16T17:19:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.130+0000 7f8da1602740 1 -- 172.21.15.18:0/3213915302 wait complete. 2024-04-16T17:19:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.130+0000 7f8da1602740 20 client.29491 trim_cache size 0 max 16384 2024-04-16T17:19:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.151+0000 7f8da1602740 1 Processor -- start 2024-04-16T17:19:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.151+0000 7f8da1602740 1 -- start start 2024-04-16T17:19:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.151+0000 7f8da1602740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f534a80 0x564c5f5498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.151+0000 7f8da1602740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f555ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.151+0000 7f8da1602740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f556220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.151+0000 7f8da1602740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8d780129a0 con 0x564c5f53e000 2024-04-16T17:19:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.151+0000 7f8da1602740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78010210 con 0x564c5f5357f0 2024-04-16T17:19:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.151+0000 7f8da1602740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7800e080 con 0x564c5f534a80 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f534a80 0x564c5f5498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f534a80 0x564c5f5498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49626/0 (socket says 172.21.15.18:49626) 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 -- 172.21.15.18:0/2117462390 learned_addr learned my addr 172.21.15.18:0/2117462390 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d96a00640 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f555ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97e00640 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f556220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 -- 172.21.15.18:0/2117462390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 msgr2=0x564c5f555ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f555ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 -- 172.21.15.18:0/2117462390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 msgr2=0x564c5f556220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f556220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 -- 172.21.15.18:0/2117462390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c5f53b800 con 0x564c5f534a80 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d96a00640 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f555ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.152+0000 7f8d97400640 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f534a80 0x564c5f5498c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8d880ba5e0 tx=0x7f8d880bef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4793fd416d186 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.153+0000 7f8d83e00640 1 -- 172.21.15.18:0/2117462390 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880be360 con 0x564c5f534a80 2024-04-16T17:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.153+0000 7f8d83e00640 1 -- 172.21.15.18:0/2117462390 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d880bdc60 con 0x564c5f534a80 2024-04-16T17:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.153+0000 7f8d83e00640 1 -- 172.21.15.18:0/2117462390 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880ef610 con 0x564c5f534a80 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 -- 172.21.15.18:0/2117462390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f534a80 msgr2=0x564c5f5498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f534a80 0x564c5f5498c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8d880ba5e0 tx=0x7f8d880bef90 comp rx=0 tx=0).stop 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 -- 172.21.15.18:0/2117462390 shutdown_connections 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f53e000 0x564c5f556220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f555ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 --2- 172.21.15.18:0/2117462390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f534a80 0x564c5f5498c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 -- 172.21.15.18:0/2117462390 >> 172.21.15.18:0/2117462390 conn(0x564c5f4cf5d0 msgr2=0x564c5f52f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 -- 172.21.15.18:0/2117462390 shutdown_connections 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.154+0000 7f8da1602740 1 -- 172.21.15.18:0/2117462390 wait complete. 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.155+0000 7f8da1602740 1 -- start start 2024-04-16T17:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.155+0000 7f8da1602740 1 Processor -- start 2024-04-16T17:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.157+0000 7f8da1602740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f5462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8da1602740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f541640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8da1602740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f53e000 0x564c5f541d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8da1602740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7800e6d0 con 0x564c5f534a80 2024-04-16T17:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8da1602740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7800fb40 con 0x564c5f5357f0 2024-04-16T17:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8da1602740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7800eda0 con 0x564c5f53e000 2024-04-16T17:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8da1602740 10 client.0 _ll_register_callbacks cb 0x7f8d84009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8d97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f5462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f541640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8d97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f53e000 0x564c5f541d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f541640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52316/0 (socket says 172.21.15.18:52316) 2024-04-16T17:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.158+0000 7f8d96a00640 1 -- 172.21.15.18:0/1976983022 learned_addr learned my addr 172.21.15.18:0/1976983022 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.159+0000 7f8d97e00640 1 -- 172.21.15.18:0/1976983022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 msgr2=0x564c5f541640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.159+0000 7f8d97e00640 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f541640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.159+0000 7f8d97e00640 1 -- 172.21.15.18:0/1976983022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 msgr2=0x564c5f5462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.159+0000 7f8d97e00640 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f5462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.159+0000 7f8d97e00640 1 -- 172.21.15.18:0/1976983022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5f5b3550 con 0x564c5f53e000 2024-04-16T17:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.159+0000 7f8d96a00640 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f541640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.159+0000 7f8d97400640 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f5462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.160+0000 7f8d97e00640 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f53e000 0x564c5f541d80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000f790 tx=0x7f8d9001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=416877bedd7a6241 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:19:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.160+0000 7f8d81600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:19:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.160+0000 7f8d81600640 1 -- 172.21.15.18:0/1976983022 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90004d20 con 0x564c5f53e000 2024-04-16T17:19:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.160+0000 7f8d81600640 1 -- 172.21.15.18:0/1976983022 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7f8d90020930 con 0x564c5f53e000 2024-04-16T17:19:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.160+0000 7f8da1602740 1 -- 172.21.15.18:0/1976983022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d880bdc60 con 0x564c5f53e000 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.160+0000 7f8d81600640 1 -- 172.21.15.18:0/1976983022 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8d90035050 con 0x564c5f53e000 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.160+0000 7f8da1602740 10 client.29497 fetch_fsmap learned FSMap version 34 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.160+0000 7f8da1602740 1 -- 172.21.15.18:0/1976983022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564c5f5b4690 con 0x564c5f53e000 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.161+0000 7f8d81600640 1 -- 172.21.15.18:0/1976983022 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d880bdc60 con 0x564c5f53e000 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.162+0000 7f8d81600640 1 -- 172.21.15.18:0/1976983022 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8d9002a960 con 0x564c5f53e000 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.162+0000 7f8da1602740 10 client.29497 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.162+0000 7f8da1602740 10 client.29497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.162+0000 7f8da1602740 20 client.29497 populate_metadata read hostname 'smithi018' 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.162+0000 7f8da1602740 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c5f5b4ed0 0x564c5f5b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.162+0000 7f8da1602740 4 client.29497 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:19:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.162+0000 7f8da1602740 1 -- 172.21.15.18:0/1976983022 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c5f4a1010 con 0x564c5f5b4ed0 2024-04-16T17:19:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.162+0000 7f8d97400640 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c5f5b4ed0 0x564c5f5b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:19:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.163+0000 7f8d97400640 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c5f5b4ed0 0x564c5f5b72f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c981774b7f91ccf server_cookie=b84317e918a3ee8c in_seq=0 out_seq=0 2024-04-16T17:19:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:57.163+0000 7f8d81600640 10 client.29497 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:57.461 INFO:tasks.workunit.client.0.smithi018.stdout: 10 34591 15.31 MB/sec execute 9 sec latency 331.923 ms 2024-04-16T17:19:58.461 INFO:tasks.workunit.client.0.smithi018.stdout: 10 34742 14.84 MB/sec execute 10 sec latency 982.780 ms 2024-04-16T17:19:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.526+0000 7f8d81600640 1 -- 172.21.15.18:0/1976983022 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c5f4a1010 con 0x564c5f5b4ed0 2024-04-16T17:19:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.526+0000 7f8d81600640 10 client.29497 handle_command_reply: tid=0 2024-04-16T17:19:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.526+0000 7f8da1602740 1 client.29497 shutdown 2024-04-16T17:19:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.526+0000 7f8da1602740 1 -- 172.21.15.18:0/1976983022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f53e000 msgr2=0x564c5f541d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.526+0000 7f8da1602740 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f53e000 0x564c5f541d80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000f790 tx=0x7f8d9001c000 comp rx=0 tx=0).stop 2024-04-16T17:19:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.527+0000 7f8da1602740 1 -- 172.21.15.18:0/1976983022 shutdown_connections 2024-04-16T17:19:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.527+0000 7f8da1602740 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c5f5b4ed0 0x564c5f5b72f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.527+0000 7f8da1602740 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c5f53e000 0x564c5f541d80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.527+0000 7f8da1602740 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c5f5357f0 0x564c5f541640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.527+0000 7f8da1602740 1 --2- 172.21.15.18:0/1976983022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c5f534a80 0x564c5f5462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.527+0000 7f8da1602740 1 -- 172.21.15.18:0/1976983022 >> 172.21.15.18:0/1976983022 conn(0x564c5f4cf5d0 msgr2=0x564c5f5392e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.527+0000 7f8d81600640 0 client.29497 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:19:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.527+0000 7f8da1602740 1 -- 172.21.15.18:0/1976983022 shutdown_connections 2024-04-16T17:19:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.528+0000 7f8da1602740 1 -- 172.21.15.18:0/1976983022 wait complete. 2024-04-16T17:19:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.528+0000 7f8da1602740 20 client.29497 trim_cache size 0 max 16384 2024-04-16T17:19:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:19:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:19:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:19:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:19:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.529+0000 7f8d9d200640 1 -- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8d7c0a3de0 msgr2=0x7f8d7c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.529+0000 7f8d9d200640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8d7c0a3de0 0x7f8d7c0a62a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8d90013940 tx=0x7f8d90009b70 comp rx=0 tx=0).stop 2024-04-16T17:19:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 -- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d980743d0 msgr2=0x7f8d9810dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d980743d0 0x7f8d9810dea0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8d88000c00 tx=0x7f8d880043b0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 -- 172.21.15.18:0/195130997 shutdown_connections 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8d7c0a3de0 0x7f8d7c0a62a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e6b0 0x7f8d98105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d98074cf0 0x7f8d98104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 --2- 172.21.15.18:0/195130997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d980743d0 0x7f8d9810dea0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 -- 172.21.15.18:0/195130997 >> 172.21.15.18:0/195130997 conn(0x7f8d980fd940 msgr2=0x7f8d980fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 -- 172.21.15.18:0/195130997 shutdown_connections 2024-04-16T17:19:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:19:58.530+0000 7f8d9d200640 1 -- 172.21.15.18:0/195130997 wait complete. 2024-04-16T17:19:58.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:19:58 smithi022 ceph-mon[36023]: pgmap v483: 129 pgs: 129 active+clean; 366 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 350 op/s 2024-04-16T17:19:58.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:19:58 smithi043 ceph-mon[36901]: pgmap v483: 129 pgs: 129 active+clean; 366 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 350 op/s 2024-04-16T17:19:59.461 INFO:tasks.workunit.client.0.smithi018.stdout: 10 35151 15.32 MB/sec execute 11 sec latency 55.100 ms 2024-04-16T17:20:00.461 INFO:tasks.workunit.client.0.smithi018.stdout: 10 35442 15.48 MB/sec execute 12 sec latency 191.144 ms 2024-04-16T17:20:00.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:00 smithi022 ceph-mon[36023]: pgmap v484: 129 pgs: 129 active+clean; 366 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-04-16T17:20:00.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:00 smithi022 ceph-mon[36023]: overall HEALTH_OK 2024-04-16T17:20:00.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:00 smithi043 ceph-mon[36901]: pgmap v484: 129 pgs: 129 active+clean; 366 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-04-16T17:20:00.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:00 smithi043 ceph-mon[36901]: overall HEALTH_OK 2024-04-16T17:20:01.461 INFO:tasks.workunit.client.0.smithi018.stdout: 10 35688 15.63 MB/sec execute 13 sec latency 252.458 ms 2024-04-16T17:20:02.461 INFO:tasks.workunit.client.0.smithi018.stdout: 10 35882 14.96 MB/sec execute 14 sec latency 274.027 ms 2024-04-16T17:20:02.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:02 smithi022 ceph-mon[36023]: pgmap v485: 129 pgs: 129 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 543 op/s 2024-04-16T17:20:02.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:02 smithi043 ceph-mon[36901]: pgmap v485: 129 pgs: 129 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 543 op/s 2024-04-16T17:20:03.227 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:19:58 smithi018 ceph-mon[31776]: pgmap v483: 129 pgs: 129 active+clean; 366 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 350 op/s 2024-04-16T17:20:03.227 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:00 smithi018 ceph-mon[31776]: pgmap v484: 129 pgs: 129 active+clean; 366 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-04-16T17:20:03.227 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:00 smithi018 ceph-mon[31776]: overall HEALTH_OK 2024-04-16T17:20:03.227 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:02 smithi018 ceph-mon[31776]: pgmap v485: 129 pgs: 129 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 543 op/s 2024-04-16T17:20:03.462 INFO:tasks.workunit.client.0.smithi018.stdout: 10 35971 14.12 MB/sec execute 15 sec latency 517.880 ms 2024-04-16T17:20:04.462 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36121 13.66 MB/sec execute 16 sec latency 425.265 ms 2024-04-16T17:20:04.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:04 smithi022 ceph-mon[36023]: pgmap v486: 129 pgs: 129 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 562 op/s 2024-04-16T17:20:04.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:04 smithi043 ceph-mon[36901]: pgmap v486: 129 pgs: 129 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 562 op/s 2024-04-16T17:20:04.853 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:04 smithi018 ceph-mon[31776]: pgmap v486: 129 pgs: 129 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 562 op/s 2024-04-16T17:20:05.462 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36229 13.13 MB/sec execute 17 sec latency 1111.119 ms 2024-04-16T17:20:05.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:05 smithi022 ceph-mon[36023]: osdmap e119: 12 total, 12 up, 12 in 2024-04-16T17:20:05.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:05 smithi043 ceph-mon[36901]: osdmap e119: 12 total, 12 up, 12 in 2024-04-16T17:20:06.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:20:06.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:20:06.462 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36302 12.70 MB/sec execute 18 sec latency 2111.179 ms 2024-04-16T17:20:06.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:06 smithi022 ceph-mon[36023]: pgmap v488: 129 pgs: 129 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 646 op/s 2024-04-16T17:20:06.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.684+0000 7f2fc6e00640 1 -- 172.21.15.18:0/4096579760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc0109c20 msgr2=0x7f2fc010d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.684+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/4096579760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc0109c20 0x7f2fc010d6b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0074e0 tx=0x7f2fac02c8d0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.684+0000 7f2fc6e00640 1 -- 172.21.15.18:0/4096579760 shutdown_connections 2024-04-16T17:20:06.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.684+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/4096579760 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fc010dd90 0x7f2fc01159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.684+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/4096579760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc0109c20 0x7f2fc010d6b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.684+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/4096579760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc0109270 0x7f2fc0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.684+0000 7f2fc6e00640 1 -- 172.21.15.18:0/4096579760 >> 172.21.15.18:0/4096579760 conn(0x7f2fc006d0a0 msgr2=0x7f2fc006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:06.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.685+0000 7f2fc6e00640 1 -- 172.21.15.18:0/4096579760 shutdown_connections 2024-04-16T17:20:06.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.686+0000 7f2fc6e00640 1 -- 172.21.15.18:0/4096579760 wait complete. 2024-04-16T17:20:06.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.687+0000 7f2fc6e00640 1 Processor -- start 2024-04-16T17:20:06.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.688+0000 7f2fc6e00640 1 -- start start 2024-04-16T17:20:06.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.689+0000 7f2fc6e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fc0109270 0x7f2fc01a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.689+0000 7f2fc6e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc010dd90 0x7f2fc019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.690+0000 7f2fc6e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc019f360 0x7f2fc019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.690+0000 7f2fc6e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0118a60 con 0x7f2fc0109270 2024-04-16T17:20:06.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.690+0000 7f2fc6e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01188e0 con 0x7f2fc019f360 2024-04-16T17:20:06.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.690+0000 7f2fc6e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0118be0 con 0x7f2fc010dd90 2024-04-16T17:20:06.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.690+0000 7f2fbf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc010dd90 0x7f2fc019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.690+0000 7f2fbf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc010dd90 0x7f2fc019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49664/0 (socket says 172.21.15.18:49664) 2024-04-16T17:20:06.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.691+0000 7f2fbf400640 1 -- 172.21.15.18:0/1925001486 learned_addr learned my addr 172.21.15.18:0/1925001486 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:06.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.691+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc019f360 0x7f2fc019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.692+0000 7f2fbf400640 1 -- 172.21.15.18:0/1925001486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc019f360 msgr2=0x7f2fc019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.692+0000 7f2fbf400640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc019f360 0x7f2fc019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.692+0000 7f2fbf400640 1 -- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fc0109270 msgr2=0x7f2fc01a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.692+0000 7f2fbf400640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fc0109270 0x7f2fc01a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.692+0000 7f2fbf400640 1 -- 172.21.15.18:0/1925001486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc01a38c0 con 0x7f2fc010dd90 2024-04-16T17:20:06.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.692+0000 7f2fbf400640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc010dd90 0x7f2fc019ee20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2fac006010 tx=0x7f2fac007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:06.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.695+0000 7f2fbcc00640 1 -- 172.21.15.18:0/1925001486 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac040070 con 0x7f2fc010dd90 2024-04-16T17:20:06.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.695+0000 7f2fbcc00640 1 -- 172.21.15.18:0/1925001486 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fac009ec0 con 0x7f2fc010dd90 2024-04-16T17:20:06.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.695+0000 7f2fbcc00640 1 -- 172.21.15.18:0/1925001486 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0046c0 con 0x7f2fc010dd90 2024-04-16T17:20:06.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.695+0000 7f2fc6e00640 1 -- 172.21.15.18:0/1925001486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc01a3b50 con 0x7f2fc010dd90 2024-04-16T17:20:06.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.695+0000 7f2fc6e00640 1 -- 172.21.15.18:0/1925001486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc01a4050 con 0x7f2fc010dd90 2024-04-16T17:20:06.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.700+0000 7f2fc9a7e740 1 Processor -- start 2024-04-16T17:20:06.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.701+0000 7f2fc9a7e740 1 -- start start 2024-04-16T17:20:06.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.701+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462ee9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.701+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462ef2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.701+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.701+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564462e4f120 con 0x564462ee9ad0 2024-04-16T17:20:06.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.701+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564462df5340 con 0x564462ee91b0 2024-04-16T17:20:06.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.701+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564462e4b0e0 con 0x564462ef26b0 2024-04-16T17:20:06.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.703+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462ee9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.703+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462ee9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52356/0 (socket says 172.21.15.18:52356) 2024-04-16T17:20:06.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.703+0000 7f2fc4e00640 1 -- 172.21.15.18:0/3097678777 learned_addr learned my addr 172.21.15.18:0/3097678777 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:06.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.703+0000 7f2fc4e00640 1 -- 172.21.15.18:0/3097678777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 msgr2=0x564462ef4aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:06.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.703+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/3097678777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef4aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.703+0000 7f2fc4e00640 1 -- 172.21.15.18:0/3097678777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 msgr2=0x564462ef2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:06.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.703+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/3097678777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462ef2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.703+0000 7f2fc4e00640 1 -- 172.21.15.18:0/3097678777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564462c2b7d0 con 0x564462ee91b0 2024-04-16T17:20:06.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.704+0000 7f2fbcc00640 1 -- 172.21.15.18:0/1925001486 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2fac043070 con 0x7f2fc010dd90 2024-04-16T17:20:06.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.705+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/3097678777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462ee9590 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800a430 tx=0x7f2fb8009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5af73ad14e9198f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:06.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.706+0000 7f2fbcc00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2fa40a3ef0 0x7f2fa40a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.707+0000 7f2fbcc00640 1 -- 172.21.15.18:0/1925001486 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f2fac0f2280 con 0x7f2fc010dd90 2024-04-16T17:20:06.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.707+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2fa40a3ef0 0x7f2fa40a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.707+0000 7f2fb5600640 1 -- 172.21.15.18:0/3097678777 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8013790 con 0x564462ee91b0 2024-04-16T17:20:06.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.707+0000 7f2fb5600640 1 -- 172.21.15.18:0/3097678777 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb8019070 con 0x564462ee91b0 2024-04-16T17:20:06.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.707+0000 7f2fb5600640 1 -- 172.21.15.18:0/3097678777 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8014de0 con 0x564462ee91b0 2024-04-16T17:20:06.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.708+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/3097678777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 msgr2=0x564462ee9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.708+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/3097678777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462ee9590 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800a430 tx=0x7f2fb8009650 comp rx=0 tx=0).stop 2024-04-16T17:20:06.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.709+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/3097678777 shutdown_connections 2024-04-16T17:20:06.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.709+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/3097678777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.709+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/3097678777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462ef2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.709+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/3097678777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462ee9590 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.709+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/3097678777 >> 172.21.15.18:0/3097678777 conn(0x564462ee38e0 msgr2=0x564462ee3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:06.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.709+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/3097678777 shutdown_connections 2024-04-16T17:20:06.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.709+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/3097678777 wait complete. 2024-04-16T17:20:06.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.710+0000 7f2fc9a7e740 1 -- start start 2024-04-16T17:20:06.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.710+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2fa40a3ef0 0x7f2fa40a63b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x564462ee8c80 tx=0x7f2fb0005e50 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:06.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.710+0000 7f2fc9a7e740 1 Processor -- start 2024-04-16T17:20:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee91b0 0x564462eef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee9ad0 0x564462eefc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564462ef54a0 con 0x564462ee91b0 2024-04-16T17:20:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564462e08710 con 0x564462ee9ad0 2024-04-16T17:20:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564462ef5620 con 0x564462ef26b0 2024-04-16T17:20:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee91b0 0x564462eef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49674/0 (socket says 172.21.15.18:49674) 2024-04-16T17:20:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fc4e00640 1 -- 172.21.15.18:0/2710374190 learned_addr learned my addr 172.21.15.18:0/2710374190 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.714+0000 7f2fbf400640 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee9ad0 0x564462eefc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.715+0000 7f2fc9a7e740 10 client.0 _ll_register_callbacks cb 0x564462e03af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.715+0000 7f2fc4e00640 1 -- 172.21.15.18:0/2710374190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee9ad0 msgr2=0x564462eefc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.715+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee9ad0 0x564462eefc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.715+0000 7f2fc4e00640 1 -- 172.21.15.18:0/2710374190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee91b0 msgr2=0x564462eef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.715+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee91b0 0x564462eef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.715+0000 7f2fc4e00640 1 -- 172.21.15.18:0/2710374190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564462b613f0 con 0x564462ef26b0 2024-04-16T17:20:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.715+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee91b0 0x564462eef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.716+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef01c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009bb0 tx=0x7f2fb8014d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701e0428c4cfea1a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.716+0000 7f2faa000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.717+0000 7f2faa000640 1 -- 172.21.15.18:0/2710374190 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80139a0 con 0x564462ef26b0 2024-04-16T17:20:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.718+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/2710374190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb8019070 con 0x564462ef26b0 2024-04-16T17:20:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.718+0000 7f2faa000640 1 -- 172.21.15.18:0/2710374190 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f2fb8013b40 con 0x564462ef26b0 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.719+0000 7f2faa000640 1 -- 172.21.15.18:0/2710374190 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2fb8031050 con 0x564462ef26b0 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.719+0000 7f2fc9a7e740 10 client.29506 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.719+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/2710374190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564462b210f0 con 0x564462ef26b0 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.719+0000 7f2faa000640 1 -- 172.21.15.18:0/2710374190 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb8019070 con 0x564462ef26b0 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.720+0000 7f2faa000640 1 -- 172.21.15.18:0/2710374190 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2fb8028bc0 con 0x564462ef26b0 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.720+0000 7f2fc9a7e740 10 client.29506 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.720+0000 7f2fc9a7e740 10 client.29506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.720+0000 7f2fc9a7e740 20 client.29506 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.720+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564462f029e0 0x564462f04dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.720+0000 7f2fc9a7e740 4 client.29506 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:06.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.720+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/2710374190 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564462e5cb40 con 0x564462f029e0 2024-04-16T17:20:06.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.721+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564462f029e0 0x564462f04dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.722+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564462f029e0 0x564462f04dd0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb1e3f043100a3fe server_cookie=4a983e93f80925e5 in_seq=0 out_seq=0 2024-04-16T17:20:06.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.722+0000 7f2faa000640 10 client.29506 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2faa000640 1 -- 172.21.15.18:0/2710374190 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564462e5cb40 con 0x564462f029e0 2024-04-16T17:20:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2faa000640 10 client.29506 handle_command_reply: tid=0 2024-04-16T17:20:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 client.29506 shutdown 2024-04-16T17:20:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/2710374190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 msgr2=0x564462ef01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef01c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009bb0 tx=0x7f2fb8014d90 comp rx=0 tx=0).stop 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/2710374190 shutdown_connections 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564462f029e0 0x564462f04dd0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef01c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee9ad0 0x564462eefc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/2710374190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee91b0 0x564462eef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2faa000640 0 client.29506 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.731+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/2710374190 >> 172.21.15.18:0/2710374190 conn(0x564462ee38e0 msgr2=0x564462ee5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.732+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/2710374190 shutdown_connections 2024-04-16T17:20:06.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.732+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/2710374190 wait complete. 2024-04-16T17:20:06.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.733+0000 7f2fc9a7e740 20 client.29506 trim_cache size 0 max 16384 2024-04-16T17:20:06.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:06 smithi043 ceph-mon[36901]: pgmap v488: 129 pgs: 129 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 646 op/s 2024-04-16T17:20:06.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.778+0000 7f2fc9a7e740 1 Processor -- start 2024-04-16T17:20:06.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.778+0000 7f2fc9a7e740 1 -- start start 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.778+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ee91b0 0x564462efe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.778+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462eeffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.778+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ef26b0 0x564462ee5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.778+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01c9720 con 0x564462ee9ad0 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.778+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01c6f90 con 0x564462ef26b0 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.778+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01c4e00 con 0x564462ee91b0 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ee91b0 0x564462efe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ee91b0 0x564462efe390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49690/0 (socket says 172.21.15.18:49690) 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbfe00640 1 -- 172.21.15.18:0/30853014 learned_addr learned my addr 172.21.15.18:0/30853014 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:06.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ef26b0 0x564462ee5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbf400640 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462eeffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbfe00640 1 -- 172.21.15.18:0/30853014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ef26b0 msgr2=0x564462ee5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ef26b0 0x564462ee5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbfe00640 1 -- 172.21.15.18:0/30853014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 msgr2=0x564462eeffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462eeffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbfe00640 1 -- 172.21.15.18:0/30853014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564462ee9900 con 0x564462ee91b0 2024-04-16T17:20:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.779+0000 7f2fbf400640 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462eeffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.780+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ee91b0 0x564462efe390 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012040 tx=0x7f2fb0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93fe676576753730 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:06.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.780+0000 7f2fb4c00640 1 -- 172.21.15.18:0/30853014 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0025d50 con 0x564462ee91b0 2024-04-16T17:20:06.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.780+0000 7f2fb4c00640 1 -- 172.21.15.18:0/30853014 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb0016040 con 0x564462ee91b0 2024-04-16T17:20:06.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.780+0000 7f2fb4c00640 1 -- 172.21.15.18:0/30853014 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb001eab0 con 0x564462ee91b0 2024-04-16T17:20:06.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.781+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/30853014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ee91b0 msgr2=0x564462efe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.781+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ee91b0 0x564462efe390 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012040 tx=0x7f2fb0014000 comp rx=0 tx=0).stop 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.781+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/30853014 shutdown_connections 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.781+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ef26b0 0x564462ee5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.781+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462eeffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.781+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/30853014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ee91b0 0x564462efe390 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.781+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/30853014 >> 172.21.15.18:0/30853014 conn(0x564462ee38e0 msgr2=0x564462ee4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.781+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/30853014 shutdown_connections 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.782+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/30853014 wait complete. 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.782+0000 7f2fc9a7e740 1 -- start start 2024-04-16T17:20:06.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.782+0000 7f2fc9a7e740 1 Processor -- start 2024-04-16T17:20:06.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.783+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462efa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.783+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462efae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.783+0000 7f2fc9a7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.783+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01c5450 con 0x564462ee9ad0 2024-04-16T17:20:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.783+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01c68c0 con 0x564462ee91b0 2024-04-16T17:20:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.783+0000 7f2fc9a7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01c5b20 con 0x564462ef26b0 2024-04-16T17:20:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.783+0000 7f2fc9a7e740 10 client.0 _ll_register_callbacks cb 0x7f2fb8009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.784+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462efa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.784+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.784+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49704/0 (socket says 172.21.15.18:49704) 2024-04-16T17:20:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.784+0000 7f2fc4e00640 1 -- 172.21.15.18:0/1683004361 learned_addr learned my addr 172.21.15.18:0/1683004361 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.784+0000 7f2fbf400640 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462efae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.785+0000 7f2fc4e00640 1 -- 172.21.15.18:0/1683004361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 msgr2=0x564462efa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.784+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462efa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52390/0 (socket says 172.21.15.18:52390) 2024-04-16T17:20:06.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.786+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462efa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.786+0000 7f2fc4e00640 1 -- 172.21.15.18:0/1683004361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 msgr2=0x564462efae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.786+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462efae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.786+0000 7f2fc4e00640 1 -- 172.21.15.18:0/1683004361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564462ef6820 con 0x564462ef26b0 2024-04-16T17:20:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.786+0000 7f2fbf400640 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462efae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.786+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462efa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.787+0000 7f2fc4e00640 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef6020 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c010 tx=0x7f2fb801e5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf7e551d003592c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.787+0000 7f2faa000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.788+0000 7f2faa000640 1 -- 172.21.15.18:0/1683004361 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb801ea40 con 0x564462ef26b0 2024-04-16T17:20:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.788+0000 7f2faa000640 1 -- 172.21.15.18:0/1683004361 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f2fb8014e00 con 0x564462ef26b0 2024-04-16T17:20:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.788+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/1683004361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb0016040 con 0x564462ef26b0 2024-04-16T17:20:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.788+0000 7f2faa000640 1 -- 172.21.15.18:0/1683004361 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2fb8042050 con 0x564462ef26b0 2024-04-16T17:20:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.788+0000 7f2fc9a7e740 10 client.29512 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.788+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/1683004361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564462f68940 con 0x564462ef26b0 2024-04-16T17:20:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.789+0000 7f2faa000640 1 -- 172.21.15.18:0/1683004361 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb0016040 con 0x564462ef26b0 2024-04-16T17:20:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.789+0000 7f2faa000640 1 -- 172.21.15.18:0/1683004361 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2fb803d030 con 0x564462ef26b0 2024-04-16T17:20:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.789+0000 7f2fc9a7e740 10 client.29512 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.789+0000 7f2fc9a7e740 10 client.29512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.789+0000 7f2fc9a7e740 20 client.29512 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.789+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564462f69150 0x564462f6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.789+0000 7f2fc9a7e740 4 client.29512 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:20:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.789+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/1683004361 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564462e569b0 con 0x564462f69150 2024-04-16T17:20:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.790+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564462f69150 0x564462f6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:06.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.791+0000 7f2fbfe00640 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564462f69150 0x564462f6b570 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c025f204f5d7f47 server_cookie=fab10e7411975cd0 in_seq=0 out_seq=0 2024-04-16T17:20:06.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.791+0000 7f2faa000640 10 client.29512 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.795+0000 7f2faa000640 1 -- 172.21.15.18:0/1683004361 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564462e569b0 con 0x564462f69150 2024-04-16T17:20:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.795+0000 7f2faa000640 10 client.29512 handle_command_reply: tid=0 2024-04-16T17:20:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.795+0000 7f2fc9a7e740 1 client.29512 shutdown 2024-04-16T17:20:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.795+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/1683004361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 msgr2=0x564462ef6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.795+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef6020 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c010 tx=0x7f2fb801e5e0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/1683004361 shutdown_connections 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564462f69150 0x564462f6b570 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564462ef26b0 0x564462ef6020 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564462ee9ad0 0x564462efae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 1 --2- 172.21.15.18:0/1683004361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564462ee91b0 0x564462efa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/1683004361 >> 172.21.15.18:0/1683004361 conn(0x564462ee38e0 msgr2=0x564462ee5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2faa000640 0 client.29512 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/1683004361 shutdown_connections 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 1 -- 172.21.15.18:0/1683004361 wait complete. 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.796+0000 7f2fc9a7e740 20 client.29512 trim_cache size 0 max 16384 2024-04-16T17:20:06.797 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.797+0000 7f2fc6e00640 1 -- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2fa40a3ef0 msgr2=0x7f2fa40a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.797+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2fa40a3ef0 0x7f2fa40a63b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x564462ee8c80 tx=0x7f2fb0005e50 comp rx=0 tx=0).stop 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.797+0000 7f2fc6e00640 1 -- 172.21.15.18:0/1925001486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc010dd90 msgr2=0x7f2fc019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.797+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc010dd90 0x7f2fc019ee20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2fac006010 tx=0x7f2fac007d00 comp rx=0 tx=0).stop 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.798+0000 7f2fc6e00640 1 -- 172.21.15.18:0/1925001486 shutdown_connections 2024-04-16T17:20:06.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.798+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2fa40a3ef0 0x7f2fa40a63b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.798+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc019f360 0x7f2fc019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.798+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fc010dd90 0x7f2fc019ee20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.798+0000 7f2fc6e00640 1 --2- 172.21.15.18:0/1925001486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fc0109270 0x7f2fc01a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.798+0000 7f2fc6e00640 1 -- 172.21.15.18:0/1925001486 >> 172.21.15.18:0/1925001486 conn(0x7f2fc006d0a0 msgr2=0x7f2fc0071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.799+0000 7f2fc6e00640 1 -- 172.21.15.18:0/1925001486 shutdown_connections 2024-04-16T17:20:06.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:06.799+0000 7f2fc6e00640 1 -- 172.21.15.18:0/1925001486 wait complete. 2024-04-16T17:20:06.831 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:20:06.831 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:20:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.142+0000 7f78cca00640 1 -- 172.21.15.18:0/2478481163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c009ad00 msgr2=0x7f78c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.142+0000 7f78cca00640 1 --2- 172.21.15.18:0/2478481163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c009ad00 0x7f78c0004c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009900 tx=0x7f78bc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:20:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.142+0000 7f78cca00640 1 -- 172.21.15.18:0/2478481163 shutdown_connections 2024-04-16T17:20:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.142+0000 7f78cca00640 1 --2- 172.21.15.18:0/2478481163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78c00986f0 0x7f78c0098b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.142+0000 7f78cca00640 1 --2- 172.21.15.18:0/2478481163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78c00051c0 0x7f78c0005640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.142+0000 7f78cca00640 1 --2- 172.21.15.18:0/2478481163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c009ad00 0x7f78c0004c80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.143+0000 7f78cca00640 1 -- 172.21.15.18:0/2478481163 >> 172.21.15.18:0/2478481163 conn(0x7f78c000a750 msgr2=0x7f78c000ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.143+0000 7f78cca00640 1 -- 172.21.15.18:0/2478481163 shutdown_connections 2024-04-16T17:20:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.143+0000 7f78cca00640 1 -- 172.21.15.18:0/2478481163 wait complete. 2024-04-16T17:20:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.144+0000 7f78cca00640 1 Processor -- start 2024-04-16T17:20:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.145+0000 7f78cca00640 1 -- start start 2024-04-16T17:20:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.146+0000 7f78cca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78c00051c0 0x7f78c01496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.146+0000 7f78cca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00986f0 0x7f78c0149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.146+0000 7f78cca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78c009ad00 0x7f78c01438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.146+0000 7f78c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00986f0 0x7f78c0149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.146+0000 7f78c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00986f0 0x7f78c0149c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48884/0 (socket says 172.21.15.18:48884) 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.146+0000 7f78c6a00640 1 -- 172.21.15.18:0/2572903852 learned_addr learned my addr 172.21.15.18:0/2572903852 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00ac610 con 0x7f78c00051c0 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00ac490 con 0x7f78c009ad00 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78c00ac790 con 0x7f78c00986f0 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78c7e00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78c009ad00 0x7f78c01438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78c7400640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78c00051c0 0x7f78c01496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78c7e00640 1 -- 172.21.15.18:0/2572903852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00986f0 msgr2=0x7f78c0149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78c7e00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00986f0 0x7f78c0149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78c7e00640 1 -- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78c00051c0 msgr2=0x7f78c01496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78c7e00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78c00051c0 0x7f78c01496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78c7e00640 1 -- 172.21.15.18:0/2572903852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c0144020 con 0x7f78c009ad00 2024-04-16T17:20:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.147+0000 7f78c6a00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00986f0 0x7f78c0149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.148+0000 7f78c7e00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78c009ad00 0x7f78c01438a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f78c8067e70 tx=0x7f78c8079e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.148+0000 7f78bbe00640 1 -- 172.21.15.18:0/2572903852 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8063220 con 0x7f78c009ad00 2024-04-16T17:20:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.149+0000 7f78bbe00640 1 -- 172.21.15.18:0/2572903852 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c8077b90 con 0x7f78c009ad00 2024-04-16T17:20:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.149+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c0144310 con 0x7f78c009ad00 2024-04-16T17:20:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.149+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c01505b0 con 0x7f78c009ad00 2024-04-16T17:20:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.149+0000 7f78bbe00640 1 -- 172.21.15.18:0/2572903852 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8075610 con 0x7f78c009ad00 2024-04-16T17:20:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.150+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a8005290 con 0x7f78c009ad00 2024-04-16T17:20:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.155+0000 7f78bbe00640 1 -- 172.21.15.18:0/2572903852 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f78c8062440 con 0x7f78c009ad00 2024-04-16T17:20:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.157+0000 7f78bbe00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b00a3e20 0x7f78b00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.157+0000 7f78bbe00640 1 -- 172.21.15.18:0/2572903852 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f78c8134ab0 con 0x7f78c009ad00 2024-04-16T17:20:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.158+0000 7f78c7400640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b00a3e20 0x7f78b00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.160+0000 7f78c7400640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b00a3e20 0x7f78b00a62e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78c01449e0 tx=0x7f78bc02df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:07.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.164+0000 7f78bbe00640 1 -- 172.21.15.18:0/2572903852 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78c8100c90 con 0x7f78c009ad00 2024-04-16T17:20:07.463 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36451 12.45 MB/sec execute 19 sec latency 2814.408 ms 2024-04-16T17:20:07.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.644+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78a8005850 con 0x7f78c009ad00 2024-04-16T17:20:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.645+0000 7f78bbe00640 1 -- 172.21.15.18:0/2572903852 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11857 (secure 0 0 0) 0x7f78c8100590 con 0x7f78c009ad00 2024-04-16T17:20:07.646 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:20:07.646 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:17:53.822480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24547,24596]},"id":1}]} 2024-04-16T17:20:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 34 2024-04-16T17:20:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b00a3e20 msgr2=0x7f78b00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b00a3e20 0x7f78b00a62e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78c01449e0 tx=0x7f78bc02df90 comp rx=0 tx=0).stop 2024-04-16T17:20:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78c009ad00 msgr2=0x7f78c01438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78c009ad00 0x7f78c01438a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f78c8067e70 tx=0x7f78c8079e80 comp rx=0 tx=0).stop 2024-04-16T17:20:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 shutdown_connections 2024-04-16T17:20:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b00a3e20 0x7f78b00a62e0 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78c01449e0 tx=0x7f78bc02df90 comp rx=0 tx=0).stop 2024-04-16T17:20:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78c009ad00 0x7f78c01438a0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00986f0 0x7f78c0149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 --2- 172.21.15.18:0/2572903852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78c00051c0 0x7f78c01496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.649+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 >> 172.21.15.18:0/2572903852 conn(0x7f78c000a750 msgr2=0x7f78c0092210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:07.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.653+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 shutdown_connections 2024-04-16T17:20:07.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:07.653+0000 7f78cca00640 1 -- 172.21.15.18:0/2572903852 wait complete. 2024-04-16T17:20:07.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:20:07.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:05 smithi018 ceph-mon[31776]: osdmap e119: 12 total, 12 up, 12 in 2024-04-16T17:20:07.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:06 smithi018 ceph-mon[31776]: pgmap v488: 129 pgs: 129 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 646 op/s 2024-04-16T17:20:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- 172.21.15.18:0/1296677533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f68098af0 msgr2=0x7f0f6809cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 --2- 172.21.15.18:0/1296677533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f68098af0 0x7f0f6809cb30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009900 tx=0x7f0f5c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:20:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- 172.21.15.18:0/1296677533 shutdown_connections 2024-04-16T17:20:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 --2- 172.21.15.18:0/1296677533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6809d210 0x7f0f680a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 --2- 172.21.15.18:0/1296677533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f68098af0 0x7f0f6809cb30 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 --2- 172.21.15.18:0/1296677533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f68098140 0x7f0f68098520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- 172.21.15.18:0/1296677533 >> 172.21.15.18:0/1296677533 conn(0x7f0f680939e0 msgr2=0x7f0f68095e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- 172.21.15.18:0/1296677533 shutdown_connections 2024-04-16T17:20:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- 172.21.15.18:0/1296677533 wait complete. 2024-04-16T17:20:08.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 Processor -- start 2024-04-16T17:20:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- start start 2024-04-16T17:20:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f68098140 0x7f0f68145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6809d210 0x7f0f68145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6813f5c0 0x7f0f6813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0f680b92e0 con 0x7f0f6813f5c0 2024-04-16T17:20:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f680b9160 con 0x7f0f6809d210 2024-04-16T17:20:08.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f73200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f680b9460 con 0x7f0f68098140 2024-04-16T17:20:08.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f71400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6809d210 0x7f0f68145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f71400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6809d210 0x7f0f68145950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42642/0 (socket says 172.21.15.18:42642) 2024-04-16T17:20:08.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.070+0000 7f0f71400640 1 -- 172.21.15.18:0/1131834095 learned_addr learned my addr 172.21.15.18:0/1131834095 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.076+0000 7f0f71400640 1 -- 172.21.15.18:0/1131834095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f68098140 msgr2=0x7f0f68145410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:20:08.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.076+0000 7f0f71400640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f68098140 0x7f0f68145410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.076+0000 7f0f71400640 1 -- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6813f5c0 msgr2=0x7f0f6813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.076+0000 7f0f71400640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6813f5c0 0x7f0f6813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.076+0000 7f0f71400640 1 -- 172.21.15.18:0/1131834095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f681400e0 con 0x7f0f6809d210 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.076+0000 7f0f71400640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6809d210 0x7f0f68145950 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c02c250 tx=0x7f0f5c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.078+0000 7f0f66a00640 1 -- 172.21.15.18:0/1131834095 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c02c890 con 0x7f0f6809d210 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.078+0000 7f0f66a00640 1 -- 172.21.15.18:0/1131834095 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f5c0054e0 con 0x7f0f6809d210 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.078+0000 7f0f66a00640 1 -- 172.21.15.18:0/1131834095 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c003a90 con 0x7f0f6809d210 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.078+0000 7f0f73200640 1 -- 172.21.15.18:0/1131834095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f68140370 con 0x7f0f6809d210 2024-04-16T17:20:08.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.078+0000 7f0f73200640 1 -- 172.21.15.18:0/1131834095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f68009140 con 0x7f0f6809d210 2024-04-16T17:20:08.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.083+0000 7f0f66a00640 1 -- 172.21.15.18:0/1131834095 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f5c003c30 con 0x7f0f6809d210 2024-04-16T17:20:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f66a00640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f540a3ef0 0x7f0f540a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f77582740 1 Processor -- start 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f66a00640 1 -- 172.21.15.18:0/1131834095 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f0f5c0fb9b0 con 0x7f0f6809d210 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f77582740 1 -- start start 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f77582740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bc3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f77582740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 0x55d529bcbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f77582740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f77582740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d529b28020 con 0x55d529bc2f50 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f77582740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d529ace460 con 0x55d529bc3870 2024-04-16T17:20:08.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f77582740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d529b23fe0 con 0x55d529bcc450 2024-04-16T17:20:08.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f72800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f72800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48924/0 (socket says 172.21.15.18:48924) 2024-04-16T17:20:08.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f72800640 1 -- 172.21.15.18:0/2207583239 learned_addr learned my addr 172.21.15.18:0/2207583239 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f71400640 1 --2- 172.21.15.18:0/2207583239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 0x55d529bcbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f72800640 1 -- 172.21.15.18:0/2207583239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 msgr2=0x55d529bcbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f72800640 1 --2- 172.21.15.18:0/2207583239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 0x55d529bcbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f72800640 1 -- 172.21.15.18:0/2207583239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 msgr2=0x55d529bc3330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f72800640 1 --2- 172.21.15.18:0/2207583239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bc3330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f72800640 1 -- 172.21.15.18:0/2207583239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d529ad6040 con 0x55d529bcc450 2024-04-16T17:20:08.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.085+0000 7f0f71e00640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f540a3ef0 0x7f0f540a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.091+0000 7f0f72800640 1 --2- 172.21.15.18:0/2207583239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bce840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c051740 tx=0x7f0f6c06aad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=948b12209a7c42ce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.091+0000 7f0f5b000640 1 -- 172.21.15.18:0/2207583239 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c07d070 con 0x55d529bcc450 2024-04-16T17:20:08.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.091+0000 7f0f5b000640 1 -- 172.21.15.18:0/2207583239 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6c077c10 con 0x55d529bcc450 2024-04-16T17:20:08.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.091+0000 7f0f5b000640 1 -- 172.21.15.18:0/2207583239 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c07d070 con 0x55d529bcc450 2024-04-16T17:20:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 -- 172.21.15.18:0/2207583239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 msgr2=0x55d529bce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 --2- 172.21.15.18:0/2207583239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bce840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c051740 tx=0x7f0f6c06aad0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f71e00640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f540a3ef0 0x7f0f540a63b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009a20 tx=0x7f0f60005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 -- 172.21.15.18:0/2207583239 shutdown_connections 2024-04-16T17:20:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 --2- 172.21.15.18:0/2207583239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bce840 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 --2- 172.21.15.18:0/2207583239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 0x55d529bcbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 --2- 172.21.15.18:0/2207583239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bc3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 -- 172.21.15.18:0/2207583239 >> 172.21.15.18:0/2207583239 conn(0x55d529bbd680 msgr2=0x55d529bbda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 -- 172.21.15.18:0/2207583239 shutdown_connections 2024-04-16T17:20:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 -- 172.21.15.18:0/2207583239 wait complete. 2024-04-16T17:20:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 -- start start 2024-04-16T17:20:08.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.093+0000 7f0f77582740 1 Processor -- start 2024-04-16T17:20:08.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.098+0000 7f0f77582740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bc9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.098+0000 7f0f77582740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 0x55d529bc9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.098+0000 7f0f77582740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bc9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.098+0000 7f0f77582740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d529bcf240 con 0x55d529bc2f50 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.098+0000 7f0f77582740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d529ae1830 con 0x55d529bc3870 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.098+0000 7f0f77582740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d529bcf3c0 con 0x55d529bcc450 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.098+0000 7f0f77582740 10 client.0 _ll_register_callbacks cb 0x55d529ae9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.098+0000 7f0f71e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bc9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f72800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bc9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f71e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bc9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58950/0 (socket says 172.21.15.18:58950) 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f71e00640 1 -- 172.21.15.18:0/666007863 learned_addr learned my addr 172.21.15.18:0/666007863 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f71400640 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 0x55d529bc9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f72800640 1 -- 172.21.15.18:0/666007863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 msgr2=0x55d529bc9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f72800640 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 0x55d529bc9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f72800640 1 -- 172.21.15.18:0/666007863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 msgr2=0x55d529bc9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f72800640 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bc9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f72800640 1 -- 172.21.15.18:0/666007863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d529ae5920 con 0x55d529bcc450 2024-04-16T17:20:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f72800640 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bc9dc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c067fb0 tx=0x7f0f6c067760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f777347a8853aae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f4f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:08.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f4f400640 1 -- 172.21.15.18:0/666007863 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c086020 con 0x55d529bcc450 2024-04-16T17:20:08.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.100+0000 7f0f4f400640 1 -- 172.21.15.18:0/666007863 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f0f6c078650 con 0x55d529bcc450 2024-04-16T17:20:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.105+0000 7f0f77582740 1 -- 172.21.15.18:0/666007863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6c077c10 con 0x55d529bcc450 2024-04-16T17:20:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.105+0000 7f0f4f400640 1 -- 172.21.15.18:0/666007863 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0f6c07d070 con 0x55d529bcc450 2024-04-16T17:20:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.105+0000 7f0f77582740 10 client.29527 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.105+0000 7f0f77582740 1 -- 172.21.15.18:0/666007863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d529ab37f0 con 0x55d529bcc450 2024-04-16T17:20:08.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f4f400640 1 -- 172.21.15.18:0/666007863 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6c077c10 con 0x55d529bcc450 2024-04-16T17:20:08.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f4f400640 1 -- 172.21.15.18:0/666007863 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0f6c080040 con 0x55d529bcc450 2024-04-16T17:20:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f77582740 10 client.29527 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f77582740 10 client.29527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:20:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f77582740 20 client.29527 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f77582740 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d529bdc3f0 0x55d529bde7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f77582740 4 client.29527 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f77582740 1 -- 172.21.15.18:0/666007863 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d529b35a40 con 0x55d529bdc3f0 2024-04-16T17:20:08.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f71e00640 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d529bdc3f0 0x55d529bde7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f71e00640 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d529bdc3f0 0x55d529bde7e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c2a70f15c1098da server_cookie=884854dc6857b4a in_seq=0 out_seq=0 2024-04-16T17:20:08.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.108+0000 7f0f4f400640 10 client.29527 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.113+0000 7f0f4f400640 1 -- 172.21.15.18:0/666007863 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d529b35a40 con 0x55d529bdc3f0 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.113+0000 7f0f4f400640 10 client.29527 handle_command_reply: tid=0 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.113+0000 7f0f77582740 1 client.29527 shutdown 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.113+0000 7f0f77582740 1 -- 172.21.15.18:0/666007863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 msgr2=0x55d529bc9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.113+0000 7f0f77582740 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bc9dc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c067fb0 tx=0x7f0f6c067760 comp rx=0 tx=0).stop 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.113+0000 7f0f77582740 1 -- 172.21.15.18:0/666007863 shutdown_connections 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.113+0000 7f0f77582740 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d529bdc3f0 0x55d529bde7e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.115+0000 7f0f77582740 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bcc450 0x55d529bc9dc0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.115+0000 7f0f77582740 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc3870 0x55d529bc9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.115+0000 7f0f77582740 1 --2- 172.21.15.18:0/666007863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bc9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.115+0000 7f0f77582740 1 -- 172.21.15.18:0/666007863 >> 172.21.15.18:0/666007863 conn(0x55d529bbd680 msgr2=0x55d529bbf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.115+0000 7f0f4f400640 0 client.29527 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:20:08.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.115+0000 7f0f77582740 1 -- 172.21.15.18:0/666007863 shutdown_connections 2024-04-16T17:20:08.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.115+0000 7f0f77582740 1 -- 172.21.15.18:0/666007863 wait complete. 2024-04-16T17:20:08.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.115+0000 7f0f77582740 20 client.29527 trim_cache size 0 max 16384 2024-04-16T17:20:08.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f77582740 1 Processor -- start 2024-04-16T17:20:08.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f77582740 1 -- start start 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f77582740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc2f50 0x55d529bcba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f77582740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 0x55d529bd8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f77582740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bcc450 0x55d529bb8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f77582740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68154bd0 con 0x55d529bcc450 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f77582740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68152440 con 0x55d529bc2f50 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f77582740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f681502b0 con 0x55d529bc3870 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc2f50 0x55d529bcba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc2f50 0x55d529bcba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42672/0 (socket says 172.21.15.18:42672) 2024-04-16T17:20:08.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 -- 172.21.15.18:0/2375315878 learned_addr learned my addr 172.21.15.18:0/2375315878 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71400640 1 --2- 172.21.15.18:0/2375315878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 0x55d529bd8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 -- 172.21.15.18:0/2375315878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 msgr2=0x55d529bd8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 --2- 172.21.15.18:0/2375315878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 0x55d529bd8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 -- 172.21.15.18:0/2375315878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bcc450 msgr2=0x55d529bb8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 --2- 172.21.15.18:0/2375315878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bcc450 0x55d529bb8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 -- 172.21.15.18:0/2375315878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d529bb8d60 con 0x55d529bc2f50 2024-04-16T17:20:08.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.145+0000 7f0f71e00640 1 --2- 172.21.15.18:0/2375315878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc2f50 0x55d529bcba70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005ab0 tx=0x7f0f60015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892d1be23c510d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.167+0000 7f0f5a600640 1 -- 172.21.15.18:0/2375315878 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60029030 con 0x55d529bc2f50 2024-04-16T17:20:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.167+0000 7f0f5a600640 1 -- 172.21.15.18:0/2375315878 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6001de40 con 0x55d529bc2f50 2024-04-16T17:20:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.167+0000 7f0f5a600640 1 -- 172.21.15.18:0/2375315878 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60010430 con 0x55d529bc2f50 2024-04-16T17:20:08.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- 172.21.15.18:0/2375315878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc2f50 msgr2=0x55d529bcba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 --2- 172.21.15.18:0/2375315878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc2f50 0x55d529bcba70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005ab0 tx=0x7f0f60015000 comp rx=0 tx=0).stop 2024-04-16T17:20:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- 172.21.15.18:0/2375315878 shutdown_connections 2024-04-16T17:20:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 --2- 172.21.15.18:0/2375315878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bcc450 0x55d529bb8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 --2- 172.21.15.18:0/2375315878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 0x55d529bd8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 --2- 172.21.15.18:0/2375315878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bc2f50 0x55d529bcba70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- 172.21.15.18:0/2375315878 >> 172.21.15.18:0/2375315878 conn(0x55d529bbd680 msgr2=0x55d529bc8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- 172.21.15.18:0/2375315878 shutdown_connections 2024-04-16T17:20:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- 172.21.15.18:0/2375315878 wait complete. 2024-04-16T17:20:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- start start 2024-04-16T17:20:08.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 Processor -- start 2024-04-16T17:20:08.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bd4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 0x55d529bd4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bcc450 0x55d529bc56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68150900 con 0x55d529bc2f50 2024-04-16T17:20:08.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68151d70 con 0x55d529bcc450 2024-04-16T17:20:08.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68150fd0 con 0x55d529bc3870 2024-04-16T17:20:08.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f77582740 10 client.0 _ll_register_callbacks cb 0x7f0f6c069db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:08.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bcc450 0x55d529bc56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bcc450 0x55d529bc56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42682/0 (socket says 172.21.15.18:42682) 2024-04-16T17:20:08.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 -- 172.21.15.18:0/3865379504 learned_addr learned my addr 172.21.15.18:0/3865379504 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 -- 172.21.15.18:0/3865379504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 msgr2=0x55d529bd4b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:20:08.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 0x55d529bd4b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 -- 172.21.15.18:0/3865379504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 msgr2=0x55d529bd4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bd4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 -- 172.21.15.18:0/3865379504 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d52983a3f0 con 0x55d529bcc450 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.168+0000 7f0f72800640 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bcc450 0x55d529bc56d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c065520 tx=0x7f0f6c0819c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94fab657dbca21cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f4f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f4f400640 1 -- 172.21.15.18:0/3865379504 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c086040 con 0x55d529bcc450 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f4f400640 1 -- 172.21.15.18:0/3865379504 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f0f6c077e10 con 0x55d529bcc450 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 1 -- 172.21.15.18:0/3865379504 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d529b2f8b0 con 0x55d529bcc450 2024-04-16T17:20:08.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f4f400640 1 -- 172.21.15.18:0/3865379504 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0f6c077a40 con 0x55d529bcc450 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 10 client.29696 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 1 -- 172.21.15.18:0/3865379504 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d5299047d0 con 0x55d529bcc450 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f4f400640 1 -- 172.21.15.18:0/3865379504 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d529b2f8b0 con 0x55d529bcc450 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f4f400640 1 -- 172.21.15.18:0/3865379504 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0f6c093060 con 0x55d529bcc450 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 10 client.29696 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 10 client.29696 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 20 client.29696 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d529c426d0 0x55d529c44b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 4 client.29696 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:20:08.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f77582740 1 -- 172.21.15.18:0/3865379504 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d529aab570 con 0x55d529c426d0 2024-04-16T17:20:08.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.174+0000 7f0f71e00640 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d529c426d0 0x55d529c44b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.179+0000 7f0f71e00640 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d529c426d0 0x55d529c44b10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7eb0042db106897e server_cookie=9317040acbbb2e9a in_seq=0 out_seq=0 2024-04-16T17:20:08.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.179+0000 7f0f4f400640 10 client.29696 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:20:08.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.179+0000 7f0f4f400640 1 -- 172.21.15.18:0/3865379504 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d529aab570 con 0x55d529c426d0 2024-04-16T17:20:08.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.179+0000 7f0f4f400640 10 client.29696 handle_command_reply: tid=0 2024-04-16T17:20:08.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.179+0000 7f0f77582740 1 client.29696 shutdown 2024-04-16T17:20:08.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.179+0000 7f0f77582740 1 -- 172.21.15.18:0/3865379504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bcc450 msgr2=0x55d529bc56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.179+0000 7f0f77582740 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bcc450 0x55d529bc56d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c065520 tx=0x7f0f6c0819c0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.182+0000 7f0f77582740 1 -- 172.21.15.18:0/3865379504 shutdown_connections 2024-04-16T17:20:08.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.182+0000 7f0f77582740 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55d529c426d0 0x55d529c44b10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.182+0000 7f0f4f400640 0 client.29696 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:20:08.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.182+0000 7f0f77582740 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d529bcc450 0x55d529bc56d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.182+0000 7f0f77582740 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d529bc3870 0x55d529bd4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.182+0000 7f0f77582740 1 --2- 172.21.15.18:0/3865379504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d529bc2f50 0x55d529bd4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.182+0000 7f0f77582740 1 -- 172.21.15.18:0/3865379504 >> 172.21.15.18:0/3865379504 conn(0x55d529bbd680 msgr2=0x55d529bc9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.188+0000 7f0f77582740 1 -- 172.21.15.18:0/3865379504 shutdown_connections 2024-04-16T17:20:08.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.188+0000 7f0f77582740 1 -- 172.21.15.18:0/3865379504 wait complete. 2024-04-16T17:20:08.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.188+0000 7f0f77582740 20 client.29696 trim_cache size 0 max 16384 2024-04-16T17:20:08.189 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:20:08.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 -- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f540a3ef0 msgr2=0x7f0f540a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f540a3ef0 0x7f0f540a63b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009a20 tx=0x7f0f60005f90 comp rx=0 tx=0).stop 2024-04-16T17:20:08.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 -- 172.21.15.18:0/1131834095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6809d210 msgr2=0x7f0f68145950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6809d210 0x7f0f68145950 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c02c250 tx=0x7f0f5c004360 comp rx=0 tx=0).stop 2024-04-16T17:20:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 -- 172.21.15.18:0/1131834095 shutdown_connections 2024-04-16T17:20:08.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f540a3ef0 0x7f0f540a63b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f6813f5c0 0x7f0f6813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6809d210 0x7f0f68145950 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 --2- 172.21.15.18:0/1131834095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f68098140 0x7f0f68145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 -- 172.21.15.18:0/1131834095 >> 172.21.15.18:0/1131834095 conn(0x7f0f680939e0 msgr2=0x7f0f68095cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 -- 172.21.15.18:0/1131834095 shutdown_connections 2024-04-16T17:20:08.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.190+0000 7f0f73200640 1 -- 172.21.15.18:0/1131834095 wait complete. 2024-04-16T17:20:08.233 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:20:08.462 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36489 11.91 MB/sec execute 20 sec latency 3081.631 ms 2024-04-16T17:20:08.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.584+0000 7f7513200640 1 -- 172.21.15.18:0/2774617609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 msgr2=0x7f750c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.584+0000 7f7513200640 1 --2- 172.21.15.18:0/2774617609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 0x7f750c0a51c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7514067e70 tx=0x7f75140a0ab0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.584+0000 7f7513200640 1 -- 172.21.15.18:0/2774617609 shutdown_connections 2024-04-16T17:20:08.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.584+0000 7f7513200640 1 --2- 172.21.15.18:0/2774617609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0b7a70 0x7f750c0b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.584+0000 7f7513200640 1 --2- 172.21.15.18:0/2774617609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750c0a5700 0x7f750c0b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.585+0000 7f7513200640 1 --2- 172.21.15.18:0/2774617609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 0x7f750c0a51c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.585+0000 7f7513200640 1 -- 172.21.15.18:0/2774617609 >> 172.21.15.18:0/2774617609 conn(0x7f750c01a640 msgr2=0x7f750c01aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.588+0000 7f7513200640 1 -- 172.21.15.18:0/2774617609 shutdown_connections 2024-04-16T17:20:08.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.589+0000 7f7513200640 1 -- 172.21.15.18:0/2774617609 wait complete. 2024-04-16T17:20:08.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.590+0000 7f7513200640 1 Processor -- start 2024-04-16T17:20:08.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.591+0000 7f7513200640 1 -- start start 2024-04-16T17:20:08.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.593+0000 7f7513200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 0x7f750c0b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.593+0000 7f7513200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0a5700 0x7f750c0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.593+0000 7f7513200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750c0b7a70 0x7f750c0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.593+0000 7f7511e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 0x7f750c0b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.593+0000 7f7511e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 0x7f750c0b70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42700/0 (socket says 172.21.15.18:42700) 2024-04-16T17:20:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.593+0000 7f7511e00640 1 -- 172.21.15.18:0/2468836886 learned_addr learned my addr 172.21.15.18:0/2468836886 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.593+0000 7f7512800640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750c0b7a70 0x7f750c0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.594+0000 7f7511400640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0a5700 0x7f750c0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.594+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f750c0af670 con 0x7f750c0a5700 2024-04-16T17:20:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.594+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f750c0af4f0 con 0x7f750c0a4de0 2024-04-16T17:20:08.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.594+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f750c0af7f0 con 0x7f750c0b7a70 2024-04-16T17:20:08.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.595+0000 7f7511e00640 1 -- 172.21.15.18:0/2468836886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750c0b7a70 msgr2=0x7f750c0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.595+0000 7f7511e00640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750c0b7a70 0x7f750c0b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.595+0000 7f7511e00640 1 -- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0a5700 msgr2=0x7f750c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.595+0000 7f7511e00640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0a5700 0x7f750c0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.595+0000 7f7511e00640 1 -- 172.21.15.18:0/2468836886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c0b3cf0 con 0x7f750c0a4de0 2024-04-16T17:20:08.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.595+0000 7f7511e00640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 0x7f750c0b70e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7514067ce0 tx=0x7f7514067950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.596+0000 7f750aa00640 1 -- 172.21.15.18:0/2468836886 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75140ba070 con 0x7f750c0a4de0 2024-04-16T17:20:08.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.596+0000 7f750aa00640 1 -- 172.21.15.18:0/2468836886 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75140a3440 con 0x7f750c0a4de0 2024-04-16T17:20:08.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.596+0000 7f750aa00640 1 -- 172.21.15.18:0/2468836886 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75140a7070 con 0x7f750c0a4de0 2024-04-16T17:20:08.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.596+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c0b3f80 con 0x7f750c0a4de0 2024-04-16T17:20:08.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.597+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c147db0 con 0x7f750c0a4de0 2024-04-16T17:20:08.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.608+0000 7f750aa00640 1 -- 172.21.15.18:0/2468836886 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f75140aa070 con 0x7f750c0a4de0 2024-04-16T17:20:08.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.611+0000 7f750aa00640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f74f80a3da0 0x7f74f80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.612+0000 7f750aa00640 1 -- 172.21.15.18:0/2468836886 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f7514166ae0 con 0x7f750c0a4de0 2024-04-16T17:20:08.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.612+0000 7f7511400640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f74f80a3da0 0x7f74f80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.613+0000 7f7511400640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f74f80a3da0 0x7f74f80a6260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f750c0b4930 tx=0x7f7504005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.614+0000 7f751b879740 1 Processor -- start 2024-04-16T17:20:08.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.614+0000 7f751b879740 1 -- start start 2024-04-16T17:20:08.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.614+0000 7f751b879740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c82f40 0x55e058c83320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.614+0000 7f751b879740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c8bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.615+0000 7f751b879740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c8c440 0x55e058c8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.615+0000 7f751b879740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e058be8020 con 0x55e058c82f40 2024-04-16T17:20:08.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.615+0000 7f751b879740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e058b8e460 con 0x55e058c83860 2024-04-16T17:20:08.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.615+0000 7f751b879740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e058be3fe0 con 0x55e058c8c440 2024-04-16T17:20:08.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.615+0000 7f7511400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c8bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.615+0000 7f7511400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c8bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42706/0 (socket says 172.21.15.18:42706) 2024-04-16T17:20:08.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.615+0000 7f7511400640 1 -- 172.21.15.18:0/2682163086 learned_addr learned my addr 172.21.15.18:0/2682163086 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.616+0000 7f7512800640 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c8c440 0x55e058c8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.616+0000 7f7511400640 1 -- 172.21.15.18:0/2682163086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c8c440 msgr2=0x55e058c8e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.617+0000 7f7511400640 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c8c440 0x55e058c8e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.617+0000 7f7511400640 1 -- 172.21.15.18:0/2682163086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c82f40 msgr2=0x55e058c83320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:08.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.617+0000 7f7511400640 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c82f40 0x55e058c83320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.618+0000 7f7511400640 1 -- 172.21.15.18:0/2682163086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e058b96040 con 0x55e058c83860 2024-04-16T17:20:08.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.618+0000 7f7512800640 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c8c440 0x55e058c8e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:08.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.618+0000 7f7511400640 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c8bf00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f750400e630 tx=0x7f7504014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae065d058f8e60e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.619+0000 7f74ff000640 1 -- 172.21.15.18:0/2682163086 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504004020 con 0x55e058c83860 2024-04-16T17:20:08.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.619+0000 7f74ff000640 1 -- 172.21.15.18:0/2682163086 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75040041c0 con 0x55e058c83860 2024-04-16T17:20:08.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.619+0000 7f74ff000640 1 -- 172.21.15.18:0/2682163086 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750401c7a0 con 0x55e058c83860 2024-04-16T17:20:08.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.620+0000 7f751b879740 1 -- 172.21.15.18:0/2682163086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 msgr2=0x55e058c8bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.620+0000 7f751b879740 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c8bf00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f750400e630 tx=0x7f7504014000 comp rx=0 tx=0).stop 2024-04-16T17:20:08.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.624+0000 7f751b879740 1 -- 172.21.15.18:0/2682163086 shutdown_connections 2024-04-16T17:20:08.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.624+0000 7f751b879740 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c8c440 0x55e058c8e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.624+0000 7f751b879740 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c8bf00 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.624+0000 7f751b879740 1 --2- 172.21.15.18:0/2682163086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c82f40 0x55e058c83320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.624+0000 7f751b879740 1 -- 172.21.15.18:0/2682163086 >> 172.21.15.18:0/2682163086 conn(0x55e058c7d670 msgr2=0x55e058c7da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.624+0000 7f751b879740 1 -- 172.21.15.18:0/2682163086 shutdown_connections 2024-04-16T17:20:08.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.625+0000 7f751b879740 1 -- 172.21.15.18:0/2682163086 wait complete. 2024-04-16T17:20:08.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.626+0000 7f751b879740 1 -- start start 2024-04-16T17:20:08.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.626+0000 7f751b879740 1 Processor -- start 2024-04-16T17:20:08.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f751b879740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c89590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f751b879740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c89ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f751b879740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c8c440 0x55e058c8a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f751b879740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e058c8f230 con 0x55e058c8c440 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f751b879740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e058ba1830 con 0x55e058c83860 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f751b879740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e058c8f3b0 con 0x55e058c82f40 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f7511e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c89590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f7511e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c89590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48990/0 (socket says 172.21.15.18:48990) 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.628+0000 7f7511e00640 1 -- 172.21.15.18:0/950422107 learned_addr learned my addr 172.21.15.18:0/950422107 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.629+0000 7f7511400640 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c89ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.629+0000 7f7511e00640 1 -- 172.21.15.18:0/950422107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 msgr2=0x55e058c89ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.629+0000 7f7511e00640 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c89ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.629+0000 7f7511e00640 1 -- 172.21.15.18:0/950422107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c8c440 msgr2=0x55e058c8a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.629+0000 7f7511e00640 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c8c440 0x55e058c8a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.629+0000 7f7511e00640 1 -- 172.21.15.18:0/950422107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e058ba5920 con 0x55e058c82f40 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.630+0000 7f7511e00640 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c89590 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f751412eaa0 tx=0x7f75141325d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ad1279647b8d8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.630+0000 7f74f3400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.630+0000 7f74f3400640 1 -- 172.21.15.18:0/950422107 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514130810 con 0x55e058c82f40 2024-04-16T17:20:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.630+0000 7f74f3400640 1 -- 172.21.15.18:0/950422107 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f751413ae00 con 0x55e058c82f40 2024-04-16T17:20:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.630+0000 7f74f3400640 10 client.0 inactive, discarding osd_map(119..119 src has 1..119) 2024-04-16T17:20:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.631+0000 7f751b879740 10 client.0 _ll_register_callbacks cb 0x55e058ba9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.631+0000 7f751b879740 1 -- 172.21.15.18:0/950422107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75040041c0 con 0x55e058c82f40 2024-04-16T17:20:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.631+0000 7f74f3400640 1 -- 172.21.15.18:0/950422107 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f751406a340 con 0x55e058c82f40 2024-04-16T17:20:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.632+0000 7f751b879740 10 client.29539 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.632+0000 7f751b879740 1 -- 172.21.15.18:0/950422107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55e058b737f0 con 0x55e058c82f40 2024-04-16T17:20:08.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.633+0000 7f74f3400640 1 -- 172.21.15.18:0/950422107 <== mon.1 v2:172.21.15.43:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f75140b4030 con 0x55e058c82f40 2024-04-16T17:20:08.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.633+0000 7f751b879740 10 client.29539 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:08.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.633+0000 7f751b879740 10 client.29539 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:20:08.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.633+0000 7f751b879740 20 client.29539 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:08.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.633+0000 7f751b879740 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e058c9c650 0x55e058c9eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.633+0000 7f751b879740 4 client.29539 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:08.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.633+0000 7f751b879740 1 -- 172.21.15.18:0/950422107 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e058bf5a40 con 0x55e058c9c650 2024-04-16T17:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.636+0000 7f7511400640 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e058c9c650 0x55e058c9eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.640+0000 7f7511400640 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e058c9c650 0x55e058c9eb10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67dee5a615f406c6 server_cookie=201228290fcd3b0c in_seq=0 out_seq=0 2024-04-16T17:20:08.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.640+0000 7f74f3400640 10 client.29539 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:20:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.643+0000 7f74f3400640 1 -- 172.21.15.18:0/950422107 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e058bf5a40 con 0x55e058c9c650 2024-04-16T17:20:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.643+0000 7f74f3400640 10 client.29539 handle_command_reply: tid=0 2024-04-16T17:20:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.643+0000 7f751b879740 1 client.29539 shutdown 2024-04-16T17:20:08.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.645+0000 7f751b879740 1 -- 172.21.15.18:0/950422107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 msgr2=0x55e058c89590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.645+0000 7f751b879740 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c89590 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f751412eaa0 tx=0x7f75141325d0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.645+0000 7f751b879740 1 -- 172.21.15.18:0/950422107 shutdown_connections 2024-04-16T17:20:08.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.645+0000 7f751b879740 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e058c9c650 0x55e058c9eb10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.645+0000 7f751b879740 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c8c440 0x55e058c8a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.645+0000 7f751b879740 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c89ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.645+0000 7f751b879740 1 --2- 172.21.15.18:0/950422107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c89590 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.645+0000 7f751b879740 1 -- 172.21.15.18:0/950422107 >> 172.21.15.18:0/950422107 conn(0x55e058c7d670 msgr2=0x55e058c7f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.646+0000 7f74f3400640 0 client.29539 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:20:08.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.646+0000 7f751b879740 1 -- 172.21.15.18:0/950422107 shutdown_connections 2024-04-16T17:20:08.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.646+0000 7f751b879740 1 -- 172.21.15.18:0/950422107 wait complete. 2024-04-16T17:20:08.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.646+0000 7f751b879740 20 client.29539 trim_cache size 0 max 16384 2024-04-16T17:20:08.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.681+0000 7f751b879740 1 Processor -- start 2024-04-16T17:20:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.682+0000 7f751b879740 1 -- start start 2024-04-16T17:20:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.682+0000 7f751b879740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c98150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.682+0000 7f751b879740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c83860 0x55e058c7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.682+0000 7f751b879740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c8c440 0x55e058ca4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.682+0000 7f751b879740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f750c154ab0 con 0x55e058c83860 2024-04-16T17:20:08.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.682+0000 7f751b879740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f750c152320 con 0x55e058c8c440 2024-04-16T17:20:08.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.682+0000 7f751b879740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f750c150190 con 0x55e058c82f40 2024-04-16T17:20:08.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.683+0000 7f7511e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c98150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.683+0000 7f7512800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c8c440 0x55e058ca4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.683+0000 7f7511e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c98150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49004/0 (socket says 172.21.15.18:49004) 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.683+0000 7f7512800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c8c440 0x55e058ca4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42722/0 (socket says 172.21.15.18:42722) 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.683+0000 7f7512800640 1 -- 172.21.15.18:0/3490545753 learned_addr learned my addr 172.21.15.18:0/3490545753 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.684+0000 7f7511400640 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c83860 0x55e058c7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.684+0000 7f7511e00640 1 -- 172.21.15.18:0/3490545753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c8c440 msgr2=0x55e058ca4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.684+0000 7f7511e00640 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c8c440 0x55e058ca4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.684+0000 7f7511e00640 1 -- 172.21.15.18:0/3490545753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c83860 msgr2=0x55e058c7c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.684+0000 7f7511e00640 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c83860 0x55e058c7c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.684+0000 7f7511e00640 1 -- 172.21.15.18:0/3490545753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0588fa3f0 con 0x55e058c82f40 2024-04-16T17:20:08.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.684+0000 7f7511400640 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c83860 0x55e058c7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.685+0000 7f7511e00640 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c98150 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7514132660 tx=0x7f7514063500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe9469cd890ed6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.686+0000 7f74fe600640 1 -- 172.21.15.18:0/3490545753 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751406a340 con 0x55e058c82f40 2024-04-16T17:20:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.686+0000 7f74fe600640 1 -- 172.21.15.18:0/3490545753 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75040041c0 con 0x55e058c82f40 2024-04-16T17:20:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.686+0000 7f74fe600640 1 -- 172.21.15.18:0/3490545753 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751411bde0 con 0x55e058c82f40 2024-04-16T17:20:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.687+0000 7f751b879740 1 -- 172.21.15.18:0/3490545753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 msgr2=0x55e058c98150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.687+0000 7f751b879740 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c98150 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7514132660 tx=0x7f7514063500 comp rx=0 tx=0).stop 2024-04-16T17:20:08.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.688+0000 7f751b879740 1 -- 172.21.15.18:0/3490545753 shutdown_connections 2024-04-16T17:20:08.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.688+0000 7f751b879740 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c8c440 0x55e058ca4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.688+0000 7f751b879740 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c83860 0x55e058c7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.688+0000 7f751b879740 1 --2- 172.21.15.18:0/3490545753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c98150 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.688+0000 7f751b879740 1 -- 172.21.15.18:0/3490545753 >> 172.21.15.18:0/3490545753 conn(0x55e058c7d670 msgr2=0x55e058c7bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.688+0000 7f751b879740 1 -- 172.21.15.18:0/3490545753 shutdown_connections 2024-04-16T17:20:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.688+0000 7f751b879740 1 -- 172.21.15.18:0/3490545753 wait complete. 2024-04-16T17:20:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.689+0000 7f751b879740 1 -- start start 2024-04-16T17:20:08.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.689+0000 7f751b879740 1 Processor -- start 2024-04-16T17:20:08.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.692+0000 7f751b879740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.692+0000 7f751b879740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.692+0000 7f751b879740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c8c440 0x55e058c85960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.692+0000 7f751b879740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1507e0 con 0x55e058c8c440 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.692+0000 7f751b879740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f750c151c50 con 0x55e058c83860 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.692+0000 7f751b879740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f750c150eb0 con 0x55e058c82f40 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.692+0000 7f751b879740 10 client.0 _ll_register_callbacks cb 0x7f75140677d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.692+0000 7f7511e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c94b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42736/0 (socket says 172.21.15.18:42736) 2024-04-16T17:20:08.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511400640 1 -- 172.21.15.18:0/4290650941 learned_addr learned my addr 172.21.15.18:0/4290650941 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:08.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c94630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49012/0 (socket says 172.21.15.18:49012) 2024-04-16T17:20:08.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511400640 1 -- 172.21.15.18:0/4290650941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 msgr2=0x55e058c94630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511400640 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c94630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511400640 1 -- 172.21.15.18:0/4290650941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c8c440 msgr2=0x55e058c85960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:08.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511400640 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c8c440 0x55e058c85960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.693+0000 7f7511400640 1 -- 172.21.15.18:0/4290650941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0589c47d0 con 0x55e058c83860 2024-04-16T17:20:08.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.694+0000 7f7511400640 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c94b70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f750400e630 tx=0x7f7504005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c663add05ac79aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:08.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.694+0000 7f74f3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:08.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.695+0000 7f74f3400640 1 -- 172.21.15.18:0/4290650941 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504005430 con 0x55e058c83860 2024-04-16T17:20:08.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.695+0000 7f74f3400640 1 -- 172.21.15.18:0/4290650941 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f75040055d0 con 0x55e058c83860 2024-04-16T17:20:08.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.695+0000 7f751b879740 1 -- 172.21.15.18:0/4290650941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75040041c0 con 0x55e058c83860 2024-04-16T17:20:08.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.697+0000 7f74f3400640 1 -- 172.21.15.18:0/4290650941 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7504043050 con 0x55e058c83860 2024-04-16T17:20:08.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.697+0000 7f751b879740 10 client.29714 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:08.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.698+0000 7f751b879740 1 -- 172.21.15.18:0/4290650941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e058d02490 con 0x55e058c83860 2024-04-16T17:20:08.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.698+0000 7f74f3400640 1 -- 172.21.15.18:0/4290650941 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504003ad0 con 0x55e058c83860 2024-04-16T17:20:08.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.699+0000 7f74f3400640 1 -- 172.21.15.18:0/4290650941 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f75040356a0 con 0x55e058c83860 2024-04-16T17:20:08.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.699+0000 7f751b879740 10 client.29714 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:08.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.699+0000 7f751b879740 10 client.29714 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:20:08.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.699+0000 7f751b879740 20 client.29714 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:08.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.699+0000 7f751b879740 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e058d028e0 0x55e058d04cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:08.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.699+0000 7f751b879740 4 client.29714 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:20:08.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.699+0000 7f751b879740 1 -- 172.21.15.18:0/4290650941 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f75040041c0 con 0x55e058d028e0 2024-04-16T17:20:08.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.702+0000 7f7511e00640 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e058d028e0 0x55e058d04cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:08.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.703+0000 7f7511e00640 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e058d028e0 0x55e058d04cd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ac3e1777900b6be server_cookie=30a1e6fc6800afac in_seq=0 out_seq=0 2024-04-16T17:20:08.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.703+0000 7f74f3400640 10 client.29714 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:20:08.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.710+0000 7f74f3400640 1 -- 172.21.15.18:0/4290650941 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f75040041c0 con 0x55e058d028e0 2024-04-16T17:20:08.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.710+0000 7f74f3400640 10 client.29714 handle_command_reply: tid=0 2024-04-16T17:20:08.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.710+0000 7f751b879740 1 client.29714 shutdown 2024-04-16T17:20:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 -- 172.21.15.18:0/4290650941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 msgr2=0x55e058c94b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c94b70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f750400e630 tx=0x7f7504005040 comp rx=0 tx=0).stop 2024-04-16T17:20:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 -- 172.21.15.18:0/4290650941 shutdown_connections 2024-04-16T17:20:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e058d028e0 0x55e058d04cd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e058c8c440 0x55e058c85960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e058c83860 0x55e058c94b70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 --2- 172.21.15.18:0/4290650941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e058c82f40 0x55e058c94630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f74f3400640 0 client.29714 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:20:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 -- 172.21.15.18:0/4290650941 >> 172.21.15.18:0/4290650941 conn(0x55e058c7d670 msgr2=0x55e058c7f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.712+0000 7f751b879740 1 -- 172.21.15.18:0/4290650941 shutdown_connections 2024-04-16T17:20:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.713+0000 7f751b879740 1 -- 172.21.15.18:0/4290650941 wait complete. 2024-04-16T17:20:08.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.713+0000 7f751b879740 20 client.29714 trim_cache size 0 max 16384 2024-04-16T17:20:08.714 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f74f80a3da0 msgr2=0x7f74f80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f74f80a3da0 0x7f74f80a6260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f750c0b4930 tx=0x7f7504005f90 comp rx=0 tx=0).stop 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 msgr2=0x7f750c0b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 0x7f750c0b70e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7514067ce0 tx=0x7f7514067950 comp rx=0 tx=0).stop 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 shutdown_connections 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f74f80a3da0 0x7f74f80a6260 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750c0b7a70 0x7f750c0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f750c0a5700 0x7f750c0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 --2- 172.21.15.18:0/2468836886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f750c0a4de0 0x7f750c0b70e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:08.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.714+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 >> 172.21.15.18:0/2468836886 conn(0x7f750c01a640 msgr2=0x7f750c0b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:08.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.715+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 shutdown_connections 2024-04-16T17:20:08.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:08.715+0000 7f7513200640 1 -- 172.21.15.18:0/2468836886 wait complete. 2024-04-16T17:20:08.743 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:20:08.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:08 smithi043 ceph-mon[36901]: pgmap v489: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 350 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 726 op/s 2024-04-16T17:20:08.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:08 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2572903852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:20:08.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:08 smithi022 ceph-mon[36023]: pgmap v489: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 350 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 726 op/s 2024-04-16T17:20:08.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:08 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2572903852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 -- 172.21.15.18:0/4075277744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109c20 msgr2=0x7f24d410d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 --2- 172.21.15.18:0/4075277744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109c20 0x7f24d410d4a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0074e0 tx=0x7f24bc02c8d0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 -- 172.21.15.18:0/4075277744 shutdown_connections 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 --2- 172.21.15.18:0/4075277744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d410db80 0x7f24d4115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 --2- 172.21.15.18:0/4075277744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109c20 0x7f24d410d4a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 --2- 172.21.15.18:0/4075277744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d4109270 0x7f24d4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 -- 172.21.15.18:0/4075277744 >> 172.21.15.18:0/4075277744 conn(0x7f24d406d360 msgr2=0x7f24d406d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 -- 172.21.15.18:0/4075277744 shutdown_connections 2024-04-16T17:20:09.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.058+0000 7f24d9600640 1 -- 172.21.15.18:0/4075277744 wait complete. 2024-04-16T17:20:09.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.060+0000 7f24d9600640 1 Processor -- start 2024-04-16T17:20:09.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.060+0000 7f24d9600640 1 -- start start 2024-04-16T17:20:09.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109270 0x7f24d41a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d410db80 0x7f24d41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d9600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d41a5a10 0x7f24d41a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d9600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41188b0 con 0x7f24d41a5a10 2024-04-16T17:20:09.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d9600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4118730 con 0x7f24d4109270 2024-04-16T17:20:09.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4118a30 con 0x7f24d410db80 2024-04-16T17:20:09.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109270 0x7f24d41a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109270 0x7f24d41a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42750/0 (socket says 172.21.15.18:42750) 2024-04-16T17:20:09.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d3e00640 1 -- 172.21.15.18:0/787415293 learned_addr learned my addr 172.21.15.18:0/787415293 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:09.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24cbe00640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d410db80 0x7f24d41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.061+0000 7f24d8c00640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d41a5a10 0x7f24d41a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.062+0000 7f24d3e00640 1 -- 172.21.15.18:0/787415293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d410db80 msgr2=0x7f24d41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.062+0000 7f24d3e00640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d410db80 0x7f24d41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.062+0000 7f24d3e00640 1 -- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d41a5a10 msgr2=0x7f24d41a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.062+0000 7f24d3e00640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d41a5a10 0x7f24d41a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.062+0000 7f24d3e00640 1 -- 172.21.15.18:0/787415293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d41a6500 con 0x7f24d4109270 2024-04-16T17:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.063+0000 7f24d3e00640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109270 0x7f24d41a1150 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f24c400b4a0 tx=0x7f24c400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:09.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.064+0000 7f24d1600640 1 -- 172.21.15.18:0/787415293 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4014070 con 0x7f24d4109270 2024-04-16T17:20:09.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.064+0000 7f24d1600640 1 -- 172.21.15.18:0/787415293 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c4010af0 con 0x7f24d4109270 2024-04-16T17:20:09.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.064+0000 7f24d1600640 1 -- 172.21.15.18:0/787415293 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4017d10 con 0x7f24d4109270 2024-04-16T17:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.065+0000 7f24d9600640 1 -- 172.21.15.18:0/787415293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d41a67f0 con 0x7f24d4109270 2024-04-16T17:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.065+0000 7f24d9600640 1 -- 172.21.15.18:0/787415293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d410f9f0 con 0x7f24d4109270 2024-04-16T17:20:09.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.073+0000 7f24d1600640 1 -- 172.21.15.18:0/787415293 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24c4017070 con 0x7f24d4109270 2024-04-16T17:20:09.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.074+0000 7f24dd994740 1 Processor -- start 2024-04-16T17:20:09.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.074+0000 7f24dd994740 1 -- start start 2024-04-16T17:20:09.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.075+0000 7f24dd994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcacf50 0x557e1bcad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.075+0000 7f24dd994740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.075+0000 7f24dd994740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcb6450 0x557e1bcb8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.075+0000 7f24dd994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557e1bc12020 con 0x557e1bcacf50 2024-04-16T17:20:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.075+0000 7f24dd994740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e1bbb8460 con 0x557e1bcb6450 2024-04-16T17:20:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.075+0000 7f24dd994740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557e1bc0dfe0 con 0x557e1bcad870 2024-04-16T17:20:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.076+0000 7f24cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcb6450 0x557e1bcb8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.076+0000 7f24cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcb6450 0x557e1bcb8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42762/0 (socket says 172.21.15.18:42762) 2024-04-16T17:20:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.076+0000 7f24cbe00640 1 -- 172.21.15.18:0/2811742856 learned_addr learned my addr 172.21.15.18:0/2811742856 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.076+0000 7f24d3e00640 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.076+0000 7f24d8c00640 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcacf50 0x557e1bcad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.077+0000 7f24cbe00640 1 -- 172.21.15.18:0/2811742856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 msgr2=0x557e1bcb5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.077+0000 7f24cbe00640 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.077+0000 7f24cbe00640 1 -- 172.21.15.18:0/2811742856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcacf50 msgr2=0x557e1bcad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.077+0000 7f24cbe00640 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcacf50 0x557e1bcad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.077+0000 7f24cbe00640 1 -- 172.21.15.18:0/2811742856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1bbc0040 con 0x557e1bcb6450 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.078+0000 7f24cbe00640 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcb6450 0x557e1bcb8840 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f24bc035000 tx=0x7f24bc00a280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d5742bec27d6c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.080+0000 7f24c9200640 1 -- 172.21.15.18:0/2811742856 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc040070 con 0x557e1bcb6450 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.080+0000 7f24c9200640 1 -- 172.21.15.18:0/2811742856 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24bc00a5d0 con 0x557e1bcb6450 2024-04-16T17:20:09.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.080+0000 7f24c9200640 1 -- 172.21.15.18:0/2811742856 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc004ec0 con 0x557e1bcb6450 2024-04-16T17:20:09.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.081+0000 7f24d1600640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24b80a3e60 0x7f24b80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.081+0000 7f24d1600640 1 -- 172.21.15.18:0/787415293 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f24c40cfe50 con 0x7f24d4109270 2024-04-16T17:20:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.082+0000 7f24dd994740 1 -- 172.21.15.18:0/2811742856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcb6450 msgr2=0x557e1bcb8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.082+0000 7f24dd994740 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcb6450 0x557e1bcb8840 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f24bc035000 tx=0x7f24bc00a280 comp rx=0 tx=0).stop 2024-04-16T17:20:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.082+0000 7f24dd994740 1 -- 172.21.15.18:0/2811742856 shutdown_connections 2024-04-16T17:20:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.082+0000 7f24dd994740 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcb6450 0x557e1bcb8840 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.082+0000 7f24dd994740 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.082+0000 7f24dd994740 1 --2- 172.21.15.18:0/2811742856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcacf50 0x557e1bcad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.082+0000 7f24dd994740 1 -- 172.21.15.18:0/2811742856 >> 172.21.15.18:0/2811742856 conn(0x557e1bca7680 msgr2=0x557e1bca7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.082+0000 7f24d8c00640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24b80a3e60 0x7f24b80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.083+0000 7f24dd994740 1 -- 172.21.15.18:0/2811742856 shutdown_connections 2024-04-16T17:20:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.083+0000 7f24dd994740 1 -- 172.21.15.18:0/2811742856 wait complete. 2024-04-16T17:20:09.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.083+0000 7f24dd994740 1 -- start start 2024-04-16T17:20:09.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.084+0000 7f24dd994740 1 Processor -- start 2024-04-16T17:20:09.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.086+0000 7f24d8c00640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24b80a3e60 0x7f24b80a6320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00b500 tx=0x7f24cc00e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:09.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.087+0000 7f24dd994740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcacf50 0x557e1bcb34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.087+0000 7f24dd994740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.087+0000 7f24dd994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bcb3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.087+0000 7f24dd994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557e1bcb91e0 con 0x557e1bcb6450 2024-04-16T17:20:09.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.087+0000 7f24dd994740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e1bbcb830 con 0x557e1bcacf50 2024-04-16T17:20:09.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.087+0000 7f24dd994740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557e1bcb9360 con 0x557e1bcad870 2024-04-16T17:20:09.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.087+0000 7f24dd994740 10 client.0 _ll_register_callbacks cb 0x557e1bbd3150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49046/0 (socket says 172.21.15.18:49046) 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24cbe00640 1 -- 172.21.15.18:0/126676643 learned_addr learned my addr 172.21.15.18:0/126676643 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24d3e00640 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcacf50 0x557e1bcb34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24cbe00640 1 -- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcacf50 msgr2=0x557e1bcb34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24cbe00640 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcacf50 0x557e1bcb34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24cbe00640 1 -- 172.21.15.18:0/126676643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 msgr2=0x557e1bcb3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24cbe00640 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bcb3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24cbe00640 1 -- 172.21.15.18:0/126676643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1bbcf920 con 0x557e1bcad870 2024-04-16T17:20:09.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.088+0000 7f24d3e00640 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcacf50 0x557e1bcb34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:09.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.089+0000 7f24cbe00640 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb3a00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f24bc004d00 tx=0x7f24bc0048b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb6c4d8538fcd898 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:09.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.089+0000 7f24c1600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:09.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.089+0000 7f24c1600640 1 -- 172.21.15.18:0/126676643 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc04b030 con 0x557e1bcad870 2024-04-16T17:20:09.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.089+0000 7f24c1600640 1 -- 172.21.15.18:0/126676643 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f24bc037040 con 0x557e1bcad870 2024-04-16T17:20:09.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.090+0000 7f24dd994740 1 -- 172.21.15.18:0/126676643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24bc00a5d0 con 0x557e1bcad870 2024-04-16T17:20:09.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.090+0000 7f24c1600640 1 -- 172.21.15.18:0/126676643 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24bc040040 con 0x557e1bcad870 2024-04-16T17:20:09.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.090+0000 7f24dd994740 10 client.29554 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:09.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.090+0000 7f24dd994740 1 -- 172.21.15.18:0/126676643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e1bb9d7f0 con 0x557e1bcad870 2024-04-16T17:20:09.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.091+0000 7f24c1600640 1 -- 172.21.15.18:0/126676643 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24bc00a5d0 con 0x557e1bcad870 2024-04-16T17:20:09.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.091+0000 7f24c1600640 1 -- 172.21.15.18:0/126676643 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f24bc0054c0 con 0x557e1bcad870 2024-04-16T17:20:09.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.091+0000 7f24dd994740 10 client.29554 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:09.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.091+0000 7f24dd994740 10 client.29554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:09.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.091+0000 7f24dd994740 20 client.29554 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:09.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.091+0000 7f24dd994740 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e1bcc66a0 0x557e1bcc8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.091+0000 7f24dd994740 4 client.29554 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:09.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.091+0000 7f24dd994740 1 -- 172.21.15.18:0/126676643 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e1bc1fa40 con 0x557e1bcc66a0 2024-04-16T17:20:09.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.092+0000 7f24d3e00640 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e1bcc66a0 0x557e1bcc8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.093+0000 7f24d3e00640 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e1bcc66a0 0x557e1bcc8a90 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b40da2c1064f8eb server_cookie=dc15ed7cd6e10f60 in_seq=0 out_seq=0 2024-04-16T17:20:09.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.093+0000 7f24c1600640 10 client.29554 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:09.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.093+0000 7f24c1600640 1 -- 172.21.15.18:0/126676643 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e1bc1fa40 con 0x557e1bcc66a0 2024-04-16T17:20:09.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.094+0000 7f24c1600640 10 client.29554 handle_command_reply: tid=0 2024-04-16T17:20:09.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.094+0000 7f24dd994740 1 client.29554 shutdown 2024-04-16T17:20:09.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.094+0000 7f24dd994740 1 -- 172.21.15.18:0/126676643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 msgr2=0x557e1bcb3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.094+0000 7f24dd994740 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb3a00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f24bc004d00 tx=0x7f24bc0048b0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.095+0000 7f24dd994740 1 -- 172.21.15.18:0/126676643 shutdown_connections 2024-04-16T17:20:09.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.095+0000 7f24dd994740 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e1bcc66a0 0x557e1bcc8a90 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.095+0000 7f24dd994740 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bcb3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.095+0000 7f24dd994740 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcad870 0x557e1bcb3a00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.095+0000 7f24c1600640 0 client.29554 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:09.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.095+0000 7f24dd994740 1 --2- 172.21.15.18:0/126676643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcacf50 0x557e1bcb34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.095+0000 7f24dd994740 1 -- 172.21.15.18:0/126676643 >> 172.21.15.18:0/126676643 conn(0x557e1bca7680 msgr2=0x557e1bca93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:09.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.096+0000 7f24dd994740 1 -- 172.21.15.18:0/126676643 shutdown_connections 2024-04-16T17:20:09.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.096+0000 7f24dd994740 1 -- 172.21.15.18:0/126676643 wait complete. 2024-04-16T17:20:09.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.096+0000 7f24dd994740 20 client.29554 trim_cache size 0 max 16384 2024-04-16T17:20:09.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.142+0000 7f24dd994740 1 Processor -- start 2024-04-16T17:20:09.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.142+0000 7f24dd994740 1 -- start start 2024-04-16T17:20:09.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.142+0000 7f24dd994740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcb5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.142+0000 7f24dd994740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcad870 0x557e1bcc20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.143+0000 7f24dd994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bcce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.143+0000 7f24dd994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41c0d80 con 0x557e1bcb6450 2024-04-16T17:20:09.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.143+0000 7f24dd994740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41be5f0 con 0x557e1bcad870 2024-04-16T17:20:09.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.143+0000 7f24dd994740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41bc480 con 0x557e1bcacf50 2024-04-16T17:20:09.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcb5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcb5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49050/0 (socket says 172.21.15.18:49050) 2024-04-16T17:20:09.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24d3e00640 1 -- 172.21.15.18:0/3968876847 learned_addr learned my addr 172.21.15.18:0/3968876847 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:09.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24d8c00640 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bcce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24cbe00640 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcad870 0x557e1bcc20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24cbe00640 1 -- 172.21.15.18:0/3968876847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 msgr2=0x557e1bcb5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:09.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24cbe00640 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcb5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24cbe00640 1 -- 172.21.15.18:0/3968876847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 msgr2=0x557e1bcce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:09.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.144+0000 7f24cbe00640 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bcce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.145+0000 7f24cbe00640 1 -- 172.21.15.18:0/3968876847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1b9243f0 con 0x557e1bcad870 2024-04-16T17:20:09.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.145+0000 7f24d8c00640 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bcce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:09.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.146+0000 7f24cbe00640 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcad870 0x557e1bcc20f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0074e0 tx=0x7f24bc004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6aad870da50c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:09.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.146+0000 7f24c3e00640 1 -- 172.21.15.18:0/3968876847 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc04b070 con 0x557e1bcad870 2024-04-16T17:20:09.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.146+0000 7f24c3e00640 1 -- 172.21.15.18:0/3968876847 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24bc00a5d0 con 0x557e1bcad870 2024-04-16T17:20:09.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.146+0000 7f24c3e00640 1 -- 172.21.15.18:0/3968876847 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0088f0 con 0x557e1bcad870 2024-04-16T17:20:09.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.151+0000 7f24dd994740 1 -- 172.21.15.18:0/3968876847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcad870 msgr2=0x557e1bcc20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.151+0000 7f24dd994740 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcad870 0x557e1bcc20f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0074e0 tx=0x7f24bc004a60 comp rx=0 tx=0).stop 2024-04-16T17:20:09.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.151+0000 7f24dd994740 1 -- 172.21.15.18:0/3968876847 shutdown_connections 2024-04-16T17:20:09.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.151+0000 7f24dd994740 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bcce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.151+0000 7f24dd994740 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bcad870 0x557e1bcc20f0 secure :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0074e0 tx=0x7f24bc004a60 comp rx=0 tx=0).stop 2024-04-16T17:20:09.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.151+0000 7f24dd994740 1 --2- 172.21.15.18:0/3968876847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcb5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.151+0000 7f24dd994740 1 -- 172.21.15.18:0/3968876847 >> 172.21.15.18:0/3968876847 conn(0x557e1bca7680 msgr2=0x557e1bcae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:09.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.153+0000 7f24dd994740 1 -- 172.21.15.18:0/3968876847 shutdown_connections 2024-04-16T17:20:09.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.153+0000 7f24dd994740 1 -- 172.21.15.18:0/3968876847 wait complete. 2024-04-16T17:20:09.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.153+0000 7f24dd994740 1 -- start start 2024-04-16T17:20:09.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.154+0000 7f24dd994740 1 Processor -- start 2024-04-16T17:20:09.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.155+0000 7f24dd994740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcbe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.155+0000 7f24dd994740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bd25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.155+0000 7f24dd994740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bd257b0 0x557e1bcba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.155+0000 7f24dd994740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41bcad0 con 0x557e1bcb6450 2024-04-16T17:20:09.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.156+0000 7f24d8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bd257b0 0x557e1bcba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.156+0000 7f24d8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bd257b0 0x557e1bcba5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42778/0 (socket says 172.21.15.18:42778) 2024-04-16T17:20:09.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.156+0000 7f24d8c00640 1 -- 172.21.15.18:0/1519175686 learned_addr learned my addr 172.21.15.18:0/1519175686 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:09.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.156+0000 7f24d3e00640 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcbe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.156+0000 7f24cbe00640 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bd25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.157+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41bdf20 con 0x557e1bd257b0 2024-04-16T17:20:09.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.157+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41bd180 con 0x557e1bcacf50 2024-04-16T17:20:09.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.158+0000 7f24d8c00640 1 -- 172.21.15.18:0/1519175686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 msgr2=0x557e1bcbe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:09.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.158+0000 7f24d8c00640 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcbe8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.158+0000 7f24d8c00640 1 -- 172.21.15.18:0/1519175686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 msgr2=0x557e1bd25270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:09.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.158+0000 7f24d8c00640 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bd25270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.158+0000 7f24dd994740 10 client.0 _ll_register_callbacks cb 0x7f24cc009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:09.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.158+0000 7f24d8c00640 1 -- 172.21.15.18:0/1519175686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1b9ee7d0 con 0x557e1bd257b0 2024-04-16T17:20:09.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.158+0000 7f24cbe00640 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bd25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:09.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.159+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24bc00a5d0 con 0x557e1bd257b0 2024-04-16T17:20:09.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.159+0000 7f24d8c00640 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bd257b0 0x557e1bcba5d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f24cc018ae0 tx=0x7f24cc004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecac50ab171a9755 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:09.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.159+0000 7f24c1600640 10 client.29729 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:09.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.161+0000 7f24c1600640 1 -- 172.21.15.18:0/1519175686 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc031050 con 0x557e1bd257b0 2024-04-16T17:20:09.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.161+0000 7f24c1600640 1 -- 172.21.15.18:0/1519175686 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f24cc015d20 con 0x557e1bd257b0 2024-04-16T17:20:09.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.161+0000 7f24c1600640 1 -- 172.21.15.18:0/1519175686 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24cc027070 con 0x557e1bd257b0 2024-04-16T17:20:09.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.161+0000 7f24dd994740 10 client.29729 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:09.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.161+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=34}) -- 0x557e1bd3dac0 con 0x557e1bd257b0 2024-04-16T17:20:09.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.163+0000 7f24c1600640 1 -- 172.21.15.18:0/1519175686 <== mon.2 v2:172.21.15.22:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f24cc022480 con 0x557e1bd257b0 2024-04-16T17:20:09.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.163+0000 7f24dd994740 10 client.29729 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:09.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.163+0000 7f24dd994740 10 client.29729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:09.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.163+0000 7f24dd994740 20 client.29729 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:09.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.163+0000 7f24dd994740 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e1bd3e0f0 0x557e1bd405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:09.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.163+0000 7f24dd994740 4 client.29729 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:20:09.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.163+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e1bc198b0 con 0x557e1bd3e0f0 2024-04-16T17:20:09.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.163+0000 7f24d3e00640 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e1bd3e0f0 0x557e1bd405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:09.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.164+0000 7f24d3e00640 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e1bd3e0f0 0x557e1bd405b0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ae7d6a8ea12bf5 server_cookie=f863edd96f9cde4b in_seq=0 out_seq=0 2024-04-16T17:20:09.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.164+0000 7f24c1600640 10 client.29729 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:09.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.167+0000 7f24c1600640 1 -- 172.21.15.18:0/1519175686 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e1bc198b0 con 0x557e1bd3e0f0 2024-04-16T17:20:09.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.167+0000 7f24c1600640 10 client.29729 handle_command_reply: tid=0 2024-04-16T17:20:09.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.167+0000 7f24dd994740 1 client.29729 shutdown 2024-04-16T17:20:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.168+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bd257b0 msgr2=0x557e1bcba5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.168+0000 7f24dd994740 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bd257b0 0x557e1bcba5d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f24cc018ae0 tx=0x7f24cc004a80 comp rx=0 tx=0).stop 2024-04-16T17:20:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.169+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 shutdown_connections 2024-04-16T17:20:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.169+0000 7f24dd994740 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557e1bd3e0f0 0x557e1bd405b0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.169+0000 7f24dd994740 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e1bd257b0 0x557e1bcba5d0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.169+0000 7f24dd994740 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557e1bcb6450 0x557e1bd25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.169+0000 7f24dd994740 1 --2- 172.21.15.18:0/1519175686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557e1bcacf50 0x557e1bcbe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.169+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 >> 172.21.15.18:0/1519175686 conn(0x557e1bca7680 msgr2=0x557e1bca8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.169+0000 7f24c1600640 0 client.29729 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.169+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 shutdown_connections 2024-04-16T17:20:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.170+0000 7f24dd994740 1 -- 172.21.15.18:0/1519175686 wait complete. 2024-04-16T17:20:09.171 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:20:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.170+0000 7f24dd994740 20 client.29729 trim_cache size 0 max 16384 2024-04-16T17:20:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.172+0000 7f24d9600640 1 -- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24b80a3e60 msgr2=0x7f24b80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.172+0000 7f24d9600640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24b80a3e60 0x7f24b80a6320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00b500 tx=0x7f24cc00e040 comp rx=0 tx=0).stop 2024-04-16T17:20:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.172+0000 7f24d9600640 1 -- 172.21.15.18:0/787415293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109270 msgr2=0x7f24d41a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.172+0000 7f24d9600640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109270 0x7f24d41a1150 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f24c400b4a0 tx=0x7f24c400b960 comp rx=0 tx=0).stop 2024-04-16T17:20:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.173+0000 7f24d9600640 1 -- 172.21.15.18:0/787415293 shutdown_connections 2024-04-16T17:20:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.173+0000 7f24d9600640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24b80a3e60 0x7f24b80a6320 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.173+0000 7f24d9600640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d41a5a10 0x7f24d41a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.173+0000 7f24d9600640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d410db80 0x7f24d41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.173+0000 7f24d9600640 1 --2- 172.21.15.18:0/787415293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d4109270 0x7f24d41a1150 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:09.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.173+0000 7f24d9600640 1 -- 172.21.15.18:0/787415293 >> 172.21.15.18:0/787415293 conn(0x7f24d406d360 msgr2=0x7f24d4071e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:09.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.173+0000 7f24d9600640 1 -- 172.21.15.18:0/787415293 shutdown_connections 2024-04-16T17:20:09.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:09.173+0000 7f24d9600640 1 -- 172.21.15.18:0/787415293 wait complete. 2024-04-16T17:20:09.463 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36597 11.69 MB/sec execute 21 sec latency 3097.981 ms 2024-04-16T17:20:10.206 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:20:10.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:20:10.206 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:20:10.466 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36682 11.46 MB/sec execute 22 sec latency 2377.740 ms 2024-04-16T17:20:10.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.529+0000 7f2b20600640 1 -- 172.21.15.18:0/2948727923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 msgr2=0x7f2b181092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.529+0000 7f2b20600640 1 --2- 172.21.15.18:0/2948727923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 0x7f2b181092c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009930 tx=0x7f2b0c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:20:10.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.530+0000 7f2b20600640 1 -- 172.21.15.18:0/2948727923 shutdown_connections 2024-04-16T17:20:10.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.530+0000 7f2b20600640 1 --2- 172.21.15.18:0/2948727923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b18109800 0x7f2b18115470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.530+0000 7f2b20600640 1 --2- 172.21.15.18:0/2948727923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 0x7f2b181092c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.530+0000 7f2b20600640 1 --2- 172.21.15.18:0/2948727923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b1810e660 0x7f2b1810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.530+0000 7f2b20600640 1 -- 172.21.15.18:0/2948727923 >> 172.21.15.18:0/2948727923 conn(0x7f2b18077d60 msgr2=0x7f2b1807a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:10.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.530+0000 7f2b20600640 1 -- 172.21.15.18:0/2948727923 shutdown_connections 2024-04-16T17:20:10.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.530+0000 7f2b20600640 1 -- 172.21.15.18:0/2948727923 wait complete. 2024-04-16T17:20:10.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.531+0000 7f2b20600640 1 Processor -- start 2024-04-16T17:20:10.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b20600640 1 -- start start 2024-04-16T17:20:10.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b20600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 0x7f2b18073930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b20600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18109800 0x7f2b1806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b20600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b1810e660 0x7f2b1806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b20600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1806c430 con 0x7f2b1810e660 2024-04-16T17:20:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b20600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1806c2b0 con 0x7f2b18108e40 2024-04-16T17:20:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b20600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1806c5b0 con 0x7f2b18109800 2024-04-16T17:20:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b1dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 0x7f2b18073930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18109800 0x7f2b1806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b1dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 0x7f2b18073930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42786/0 (socket says 172.21.15.18:42786) 2024-04-16T17:20:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b1dc00640 1 -- 172.21.15.18:0/122600196 learned_addr learned my addr 172.21.15.18:0/122600196 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.532+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18109800 0x7f2b1806d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49070/0 (socket says 172.21.15.18:49070) 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.533+0000 7f2b1e600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b1810e660 0x7f2b1806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.533+0000 7f2b1dc00640 1 -- 172.21.15.18:0/122600196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18109800 msgr2=0x7f2b1806d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.533+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18109800 0x7f2b1806d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.533+0000 7f2b1dc00640 1 -- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b1810e660 msgr2=0x7f2b1806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.533+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b1810e660 0x7f2b1806dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.533+0000 7f2b1dc00640 1 -- 172.21.15.18:0/122600196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1806e610 con 0x7f2b18108e40 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.533+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 0x7f2b18073930 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e9f0 tx=0x7f2b0800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.534+0000 7f2b16000640 1 -- 172.21.15.18:0/122600196 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800cd80 con 0x7f2b18108e40 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.534+0000 7f2b16000640 1 -- 172.21.15.18:0/122600196 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b08005500 con 0x7f2b18108e40 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.534+0000 7f2b16000640 1 -- 172.21.15.18:0/122600196 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08010690 con 0x7f2b18108e40 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.534+0000 7f2b20600640 1 -- 172.21.15.18:0/122600196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b181b3d30 con 0x7f2b18108e40 2024-04-16T17:20:10.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.534+0000 7f2b20600640 1 -- 172.21.15.18:0/122600196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181b4140 con 0x7f2b18108e40 2024-04-16T17:20:10.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.539+0000 7f2b16000640 1 -- 172.21.15.18:0/122600196 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b08004f30 con 0x7f2b18108e40 2024-04-16T17:20:10.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.542+0000 7f2b2326d740 1 Processor -- start 2024-04-16T17:20:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.543+0000 7f2b2326d740 1 -- start start 2024-04-16T17:20:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.544+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.544+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.544+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.544+0000 7f2b2326d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564178431cb0 con 0x5641784ccca0 2024-04-16T17:20:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.544+0000 7f2b2326d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5641783d8370 con 0x5641784cd5c0 2024-04-16T17:20:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.544+0000 7f2b2326d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56417842dc70 con 0x5641784d61a0 2024-04-16T17:20:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.544+0000 7f2b16000640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b000a3da0 0x7f2b000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.545+0000 7f2b16000640 1 -- 172.21.15.18:0/122600196 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f2b080d0ed0 con 0x7f2b18108e40 2024-04-16T17:20:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.545+0000 7f2b1e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.545+0000 7f2b1dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.545+0000 7f2b1e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59054/0 (socket says 172.21.15.18:59054) 2024-04-16T17:20:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.545+0000 7f2b1e600640 1 -- 172.21.15.18:0/4283684987 learned_addr learned my addr 172.21.15.18:0/4283684987 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:10.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.545+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.546+0000 7f2b1e600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b000a3da0 0x7f2b000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.547+0000 7f2b1dc00640 1 -- 172.21.15.18:0/4283684987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 msgr2=0x5641784d8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:10.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.547+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/4283684987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784d8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.547+0000 7f2b1dc00640 1 -- 172.21.15.18:0/4283684987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 msgr2=0x5641784cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.547+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/4283684987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.547+0000 7f2b1dc00640 1 -- 172.21.15.18:0/4283684987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641783f7910 con 0x5641784cd5c0 2024-04-16T17:20:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.549+0000 7f2b1e600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b000a3da0 0x7f2b000a6260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2b10005c40 tx=0x7f2b10007220 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.549+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/4283684987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d5c60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2b080ce1c0 tx=0x7f2b0809c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff842fb800a4cb6b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.549+0000 7f2b06a00640 1 -- 172.21.15.18:0/4283684987 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0801fdb0 con 0x5641784cd5c0 2024-04-16T17:20:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.549+0000 7f2b06a00640 1 -- 172.21.15.18:0/4283684987 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b0809d110 con 0x5641784cd5c0 2024-04-16T17:20:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.549+0000 7f2b06a00640 1 -- 172.21.15.18:0/4283684987 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08005220 con 0x5641784cd5c0 2024-04-16T17:20:10.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.551+0000 7f2b2326d740 1 -- 172.21.15.18:0/4283684987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 msgr2=0x5641784d5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.551+0000 7f2b2326d740 1 --2- 172.21.15.18:0/4283684987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d5c60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2b080ce1c0 tx=0x7f2b0809c8a0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.552+0000 7f2b2326d740 1 -- 172.21.15.18:0/4283684987 shutdown_connections 2024-04-16T17:20:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.552+0000 7f2b2326d740 1 --2- 172.21.15.18:0/4283684987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784d8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.552+0000 7f2b2326d740 1 --2- 172.21.15.18:0/4283684987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d5c60 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.552+0000 7f2b2326d740 1 --2- 172.21.15.18:0/4283684987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.552+0000 7f2b2326d740 1 -- 172.21.15.18:0/4283684987 >> 172.21.15.18:0/4283684987 conn(0x5641784c73d0 msgr2=0x5641784c77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.552+0000 7f2b2326d740 1 -- 172.21.15.18:0/4283684987 shutdown_connections 2024-04-16T17:20:10.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.552+0000 7f2b2326d740 1 -- 172.21.15.18:0/4283684987 wait complete. 2024-04-16T17:20:10.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.553+0000 7f2b2326d740 1 -- start start 2024-04-16T17:20:10.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.553+0000 7f2b2326d740 1 Processor -- start 2024-04-16T17:20:10.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.554+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784ccca0 0x5641784d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.554+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.554+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784d61a0 0x5641784d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.554+0000 7f2b2326d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5641784e9fa0 con 0x5641784d61a0 2024-04-16T17:20:10.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.554+0000 7f2b2326d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5641783eb730 con 0x5641784cd5c0 2024-04-16T17:20:10.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.554+0000 7f2b2326d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5641784ea120 con 0x5641784ccca0 2024-04-16T17:20:10.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.554+0000 7f2b2326d740 10 client.0 _ll_register_callbacks cb 0x5641783ba1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:10.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.555+0000 7f2b1e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784d61a0 0x5641784d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.555+0000 7f2b1dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784ccca0 0x5641784d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.555+0000 7f2b1e600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784d61a0 0x5641784d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59056/0 (socket says 172.21.15.18:59056) 2024-04-16T17:20:10.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.555+0000 7f2b1e600640 1 -- 172.21.15.18:0/1491377079 learned_addr learned my addr 172.21.15.18:0/1491377079 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:10.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.555+0000 7f2b1d200640 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.556+0000 7f2b1e600640 1 -- 172.21.15.18:0/1491377079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784ccca0 msgr2=0x5641784d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:10.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.556+0000 7f2b1e600640 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784ccca0 0x5641784d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.556+0000 7f2b1e600640 1 -- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 msgr2=0x5641784d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:10.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.556+0000 7f2b1e600640 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.556+0000 7f2b1e600640 1 -- 172.21.15.18:0/1491377079 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417820c7e0 con 0x5641784d61a0 2024-04-16T17:20:10.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.556+0000 7f2b1d200640 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:10.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.559+0000 7f2b1e600640 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784d61a0 0x5641784d3cb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2b10013900 tx=0x7f2b10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81ba68fde3c9faa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:10.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.559+0000 7f2afb400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:20:10.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.561+0000 7f2afb400640 1 -- 172.21.15.18:0/1491377079 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1001f660 con 0x5641784d61a0 2024-04-16T17:20:10.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.561+0000 7f2afb400640 1 -- 172.21.15.18:0/1491377079 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f2b1001f800 con 0x5641784d61a0 2024-04-16T17:20:10.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.561+0000 7f2b2326d740 1 -- 172.21.15.18:0/1491377079 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b0809d110 con 0x5641784d61a0 2024-04-16T17:20:10.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.561+0000 7f2afb400640 1 -- 172.21.15.18:0/1491377079 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2b1001fa00 con 0x5641784d61a0 2024-04-16T17:20:10.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.561+0000 7f2b2326d740 10 client.19221 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:10.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.561+0000 7f2b2326d740 1 -- 172.21.15.18:0/1491377079 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56417826cf70 con 0x5641784d61a0 2024-04-16T17:20:10.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.562+0000 7f2afb400640 1 -- 172.21.15.18:0/1491377079 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b0809d110 con 0x5641784d61a0 2024-04-16T17:20:10.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.563+0000 7f2afb400640 1 -- 172.21.15.18:0/1491377079 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2b1001c040 con 0x5641784d61a0 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.563+0000 7f2b2326d740 10 client.19221 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.563+0000 7f2b2326d740 10 client.19221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.563+0000 7f2b2326d740 20 client.19221 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.563+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5641784f75f0 0x5641784f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.563+0000 7f2b2326d740 4 client.19221 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.563+0000 7f2b2326d740 1 -- 172.21.15.18:0/1491377079 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56417843f6d0 con 0x5641784f75f0 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.564+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5641784f75f0 0x5641784f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.564+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5641784f75f0 0x5641784f9a30 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50446c2a374d31 server_cookie=969ed6b451cad3b8 in_seq=0 out_seq=0 2024-04-16T17:20:10.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.564+0000 7f2afb400640 10 client.19221 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:10.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.565+0000 7f2afb400640 1 -- 172.21.15.18:0/1491377079 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56417843f6d0 con 0x5641784f75f0 2024-04-16T17:20:10.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.565+0000 7f2afb400640 10 client.19221 handle_command_reply: tid=0 2024-04-16T17:20:10.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.565+0000 7f2b2326d740 1 client.19221 shutdown 2024-04-16T17:20:10.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.566+0000 7f2b2326d740 1 -- 172.21.15.18:0/1491377079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784d61a0 msgr2=0x5641784d3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.566+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784d61a0 0x5641784d3cb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2b10013900 tx=0x7f2b10004a40 comp rx=0 tx=0).stop 2024-04-16T17:20:10.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2b2326d740 1 -- 172.21.15.18:0/1491377079 shutdown_connections 2024-04-16T17:20:10.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5641784f75f0 0x5641784f9a30 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784d61a0 0x5641784d3cb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1491377079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784ccca0 0x5641784d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2b2326d740 1 -- 172.21.15.18:0/1491377079 >> 172.21.15.18:0/1491377079 conn(0x5641784c73d0 msgr2=0x5641784c9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:10.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2afb400640 0 client.19221 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:10.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2b2326d740 1 -- 172.21.15.18:0/1491377079 shutdown_connections 2024-04-16T17:20:10.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.568+0000 7f2b2326d740 1 -- 172.21.15.18:0/1491377079 wait complete. 2024-04-16T17:20:10.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.569+0000 7f2b2326d740 20 client.19221 trim_cache size 0 max 16384 2024-04-16T17:20:10.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.605+0000 7f2b2326d740 1 Processor -- start 2024-04-16T17:20:10.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.605+0000 7f2b2326d740 1 -- start start 2024-04-16T17:20:10.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.606+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784d5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.606+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.606+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784ff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.606+0000 7f2b2326d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b8670 con 0x5641784ccca0 2024-04-16T17:20:10.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.606+0000 7f2b2326d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b5ee0 con 0x5641784cd5c0 2024-04-16T17:20:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.606+0000 7f2b2326d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18084a00 con 0x5641784d61a0 2024-04-16T17:20:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.607+0000 7f2b1e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.607+0000 7f2b1e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784ff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49090/0 (socket says 172.21.15.18:49090) 2024-04-16T17:20:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.607+0000 7f2b1e600640 1 -- 172.21.15.18:0/2128130697 learned_addr learned my addr 172.21.15.18:0/2128130697 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.607+0000 7f2b1d200640 1 --2- 172.21.15.18:0/2128130697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.608+0000 7f2b1e600640 1 -- 172.21.15.18:0/2128130697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 msgr2=0x5641784f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:10.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.608+0000 7f2b1e600640 1 --2- 172.21.15.18:0/2128130697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.608+0000 7f2b1e600640 1 -- 172.21.15.18:0/2128130697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 msgr2=0x5641784d5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:10.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.608+0000 7f2b1e600640 1 --2- 172.21.15.18:0/2128130697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.608+0000 7f2b1e600640 1 -- 172.21.15.18:0/2128130697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564178104170 con 0x5641784d61a0 2024-04-16T17:20:10.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.609+0000 7f2b1e600640 1 --2- 172.21.15.18:0/2128130697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784ff300 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2b10013ac0 tx=0x7f2b1000bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=613c1408599f8e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:10.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.610+0000 7f2b06000640 1 -- 172.21.15.18:0/2128130697 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1003c030 con 0x5641784d61a0 2024-04-16T17:20:10.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.610+0000 7f2b06000640 1 -- 172.21.15.18:0/2128130697 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1001f660 con 0x5641784d61a0 2024-04-16T17:20:10.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.610+0000 7f2b06000640 1 -- 172.21.15.18:0/2128130697 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1003c030 con 0x5641784d61a0 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.611+0000 7f2b2326d740 1 -- 172.21.15.18:0/2128130697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 msgr2=0x5641784ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.611+0000 7f2b2326d740 1 --2- 172.21.15.18:0/2128130697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784ff300 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2b10013ac0 tx=0x7f2b1000bf00 comp rx=0 tx=0).stop 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.611+0000 7f2b2326d740 1 -- 172.21.15.18:0/2128130697 shutdown_connections 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.611+0000 7f2b2326d740 1 --2- 172.21.15.18:0/2128130697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784d61a0 0x5641784ff300 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.611+0000 7f2b2326d740 1 --2- 172.21.15.18:0/2128130697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784cd5c0 0x5641784f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.611+0000 7f2b2326d740 1 --2- 172.21.15.18:0/2128130697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784d5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.611+0000 7f2b2326d740 1 -- 172.21.15.18:0/2128130697 >> 172.21.15.18:0/2128130697 conn(0x5641784c73d0 msgr2=0x5641784c6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.612+0000 7f2b2326d740 1 -- 172.21.15.18:0/2128130697 shutdown_connections 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.612+0000 7f2b2326d740 1 -- 172.21.15.18:0/2128130697 wait complete. 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.612+0000 7f2b2326d740 1 -- start start 2024-04-16T17:20:10.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.613+0000 7f2b2326d740 1 Processor -- start 2024-04-16T17:20:10.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.614+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.614+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784cd5c0 0x5641784d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.614+0000 7f2b2326d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784d61a0 0x5641784d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.614+0000 7f2b2326d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b43d0 con 0x5641784ccca0 2024-04-16T17:20:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.614+0000 7f2b2326d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b5810 con 0x5641784d61a0 2024-04-16T17:20:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.614+0000 7f2b2326d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b4a70 con 0x5641784cd5c0 2024-04-16T17:20:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.614+0000 7f2b2326d740 10 client.0 _ll_register_callbacks cb 0x7f2b10010cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784cd5c0 0x5641784d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1dc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784cd5c0 0x5641784d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49096/0 (socket says 172.21.15.18:49096) 2024-04-16T17:20:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1d200640 1 -- 172.21.15.18:0/1209792427 learned_addr learned my addr 172.21.15.18:0/1209792427 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1e600640 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784d61a0 0x5641784d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1d200640 1 -- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784d61a0 msgr2=0x5641784d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1d200640 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784d61a0 0x5641784d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1d200640 1 -- 172.21.15.18:0/1209792427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 msgr2=0x5641784d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1d200640 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.615+0000 7f2b1d200640 1 -- 172.21.15.18:0/1209792427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417855cd20 con 0x5641784cd5c0 2024-04-16T17:20:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.616+0000 7f2b1e600640 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784d61a0 0x5641784d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.616+0000 7f2b1d200640 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784cd5c0 0x5641784d0690 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005ab0 tx=0x7f2b0c0045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf0b5c72a9c1f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.616+0000 7f2afb400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.617+0000 7f2afb400640 1 -- 172.21.15.18:0/1209792427 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c005220 con 0x5641784cd5c0 2024-04-16T17:20:10.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.617+0000 7f2b2326d740 1 -- 172.21.15.18:0/1209792427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1001f660 con 0x5641784cd5c0 2024-04-16T17:20:10.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.617+0000 7f2afb400640 1 -- 172.21.15.18:0/1209792427 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f2b0c005020 con 0x5641784cd5c0 2024-04-16T17:20:10.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.618+0000 7f2afb400640 1 -- 172.21.15.18:0/1209792427 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2b0c059050 con 0x5641784cd5c0 2024-04-16T17:20:10.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.618+0000 7f2b2326d740 10 client.29578 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:10.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.618+0000 7f2b2326d740 1 -- 172.21.15.18:0/1209792427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56417855e040 con 0x5641784cd5c0 2024-04-16T17:20:10.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.618+0000 7f2afb400640 1 -- 172.21.15.18:0/1209792427 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b0c050070 con 0x5641784cd5c0 2024-04-16T17:20:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.618+0000 7f2afb400640 1 -- 172.21.15.18:0/1209792427 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2b0c034440 con 0x5641784cd5c0 2024-04-16T17:20:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.618+0000 7f2b2326d740 10 client.29578 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.618+0000 7f2b2326d740 10 client.29578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.618+0000 7f2b2326d740 20 client.29578 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.619+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56417855e8a0 0x564178560cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.619+0000 7f2b2326d740 4 client.29578 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:20:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.619+0000 7f2b2326d740 1 -- 172.21.15.18:0/1209792427 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f2b1001f660 con 0x56417855e8a0 2024-04-16T17:20:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.619+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56417855e8a0 0x564178560cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:10.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.620+0000 7f2afb400640 10 client.29578 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:10.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.620+0000 7f2b1dc00640 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56417855e8a0 0x564178560cc0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca50b1ba4648edc7 server_cookie=943ee57a099e26df in_seq=0 out_seq=0 2024-04-16T17:20:10.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.621+0000 7f2afb400640 1 -- 172.21.15.18:0/1209792427 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f2b1001f660 con 0x56417855e8a0 2024-04-16T17:20:10.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.621+0000 7f2afb400640 10 client.29578 handle_command_reply: tid=0 2024-04-16T17:20:10.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.621+0000 7f2b2326d740 1 client.29578 shutdown 2024-04-16T17:20:10.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.622+0000 7f2b2326d740 1 -- 172.21.15.18:0/1209792427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784cd5c0 msgr2=0x5641784d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.622+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784cd5c0 0x5641784d0690 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005ab0 tx=0x7f2b0c0045f0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.623+0000 7f2b2326d740 1 -- 172.21.15.18:0/1209792427 shutdown_connections 2024-04-16T17:20:10.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.623+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56417855e8a0 0x564178560cc0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.623+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641784d61a0 0x5641784d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.623+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641784cd5c0 0x5641784d0690 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.623+0000 7f2afb400640 0 client.29578 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:10.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.623+0000 7f2b2326d740 1 --2- 172.21.15.18:0/1209792427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641784ccca0 0x5641784d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.623+0000 7f2b2326d740 1 -- 172.21.15.18:0/1209792427 >> 172.21.15.18:0/1209792427 conn(0x5641784c73d0 msgr2=0x5641784d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:10.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.624+0000 7f2b2326d740 1 -- 172.21.15.18:0/1209792427 shutdown_connections 2024-04-16T17:20:10.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.624+0000 7f2b2326d740 1 -- 172.21.15.18:0/1209792427 wait complete. 2024-04-16T17:20:10.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.626+0000 7f2b2326d740 20 client.29578 trim_cache size 0 max 16384 2024-04-16T17:20:10.627 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:20:10.627 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:20:10.627 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "18038043-f815-4918-a3a6-1211cf7f6eec", 2024-04-16T17:20:10.627 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:20:10.627 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:20:10.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 -- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b000a3da0 msgr2=0x7f2b000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b000a3da0 0x7f2b000a6260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2b10005c40 tx=0x7f2b10007220 comp rx=0 tx=0).stop 2024-04-16T17:20:10.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 -- 172.21.15.18:0/122600196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 msgr2=0x7f2b18073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 0x7f2b18073930 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e9f0 tx=0x7f2b0800edf0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 -- 172.21.15.18:0/122600196 shutdown_connections 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2b000a3da0 0x7f2b000a6260 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b1810e660 0x7f2b1806dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18109800 0x7f2b1806d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 --2- 172.21.15.18:0/122600196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b18108e40 0x7f2b18073930 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.628+0000 7f2b20600640 1 -- 172.21.15.18:0/122600196 >> 172.21.15.18:0/122600196 conn(0x7f2b18077d60 msgr2=0x7f2b1807a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.629+0000 7f2b20600640 1 -- 172.21.15.18:0/122600196 shutdown_connections 2024-04-16T17:20:10.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.629+0000 7f2b20600640 1 -- 172.21.15.18:0/122600196 wait complete. 2024-04-16T17:20:10.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:20:10.656 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:20:10.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:10 smithi043 ceph-mon[36901]: pgmap v490: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 350 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 708 op/s 2024-04-16T17:20:10.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:10 smithi022 ceph-mon[36023]: pgmap v490: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 350 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 708 op/s 2024-04-16T17:20:10.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.997+0000 7f58d6a00640 1 -- 172.21.15.18:0/3401514096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0101060 msgr2=0x7f58d0101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:10.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.998+0000 7f58d6a00640 1 --2- 172.21.15.18:0/3401514096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0101060 0x7f58d0101440 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009860 tx=0x7f58c002bc10 comp rx=0 tx=0).stop 2024-04-16T17:20:10.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.998+0000 7f58d6a00640 1 -- 172.21.15.18:0/3401514096 shutdown_connections 2024-04-16T17:20:10.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.998+0000 7f58d6a00640 1 --2- 172.21.15.18:0/3401514096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d0106380 0x7f58d0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.998+0000 7f58d6a00640 1 --2- 172.21.15.18:0/3401514096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0101980 0x7f58d0105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.999+0000 7f58d6a00640 1 --2- 172.21.15.18:0/3401514096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0101060 0x7f58d0101440 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:10.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.999+0000 7f58d6a00640 1 -- 172.21.15.18:0/3401514096 >> 172.21.15.18:0/3401514096 conn(0x7f58d00780b0 msgr2=0x7f58d00784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:10.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.999+0000 7f58d6a00640 1 -- 172.21.15.18:0/3401514096 shutdown_connections 2024-04-16T17:20:11.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:10.999+0000 7f58d6a00640 1 -- 172.21.15.18:0/3401514096 wait complete. 2024-04-16T17:20:11.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.002+0000 7f58d6a00640 1 Processor -- start 2024-04-16T17:20:11.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.002+0000 7f58d6a00640 1 -- start start 2024-04-16T17:20:11.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58d6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0101060 0x7f58d01152a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d0101980 0x7f58d010e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58d6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0106380 0x7f58d010e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58d6a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01189b0 con 0x7f58d0106380 2024-04-16T17:20:11.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58d6a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58d0118830 con 0x7f58d0101060 2024-04-16T17:20:11.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58d6a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58d0118b30 con 0x7f58d0101980 2024-04-16T17:20:11.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0101060 0x7f58d01152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0101060 0x7f58d01152a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42832/0 (socket says 172.21.15.18:42832) 2024-04-16T17:20:11.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58cfe00640 1 -- 172.21.15.18:0/4182904172 learned_addr learned my addr 172.21.15.18:0/4182904172 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.003+0000 7f58cf400640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d0101980 0x7f58d010e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.004+0000 7f58d4a00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0106380 0x7f58d010e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.006+0000 7f58cf400640 1 -- 172.21.15.18:0/4182904172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0101060 msgr2=0x7f58d01152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.006+0000 7f58cf400640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0101060 0x7f58d01152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.006+0000 7f58cf400640 1 -- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0106380 msgr2=0x7f58d010e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.006+0000 7f58cf400640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0106380 0x7f58d010e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.006+0000 7f58cf400640 1 -- 172.21.15.18:0/4182904172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d010eef0 con 0x7f58d0101980 2024-04-16T17:20:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.006+0000 7f58d4a00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0106380 0x7f58d010e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.006+0000 7f58cfe00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0101060 0x7f58d01152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.006+0000 7f58cf400640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d0101980 0x7f58d010e370 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00c9a0 tx=0x7f58bc00ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.007+0000 7f58ccc00640 1 -- 172.21.15.18:0/4182904172 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc007bb0 con 0x7f58d0101980 2024-04-16T17:20:11.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.007+0000 7f58ccc00640 1 -- 172.21.15.18:0/4182904172 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58bc007d50 con 0x7f58d0101980 2024-04-16T17:20:11.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.007+0000 7f58ccc00640 1 -- 172.21.15.18:0/4182904172 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00f450 con 0x7f58d0101980 2024-04-16T17:20:11.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.008+0000 7f58d6a00640 1 -- 172.21.15.18:0/4182904172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d0112dc0 con 0x7f58d0101980 2024-04-16T17:20:11.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.008+0000 7f58d6a00640 1 -- 172.21.15.18:0/4182904172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d01132c0 con 0x7f58d0101980 2024-04-16T17:20:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.019+0000 7f58d9798740 1 Processor -- start 2024-04-16T17:20:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.019+0000 7f58d9798740 1 -- start start 2024-04-16T17:20:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.020+0000 7f58d9798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74db3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.020+0000 7f58d9798740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74dbc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.020+0000 7f58d9798740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74dbeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.020+0000 7f58d9798740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562b74d19120 con 0x562b74dbc6b0 2024-04-16T17:20:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.020+0000 7f58d9798740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562b74cbf340 con 0x562b74db3ad0 2024-04-16T17:20:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.020+0000 7f58d9798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562b74d150e0 con 0x562b74db31b0 2024-04-16T17:20:11.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.021+0000 7f58ccc00640 1 -- 172.21.15.18:0/4182904172 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f58bc004f20 con 0x7f58d0101980 2024-04-16T17:20:11.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.021+0000 7f58d4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74db3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.021+0000 7f58d4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74db3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49128/0 (socket says 172.21.15.18:49128) 2024-04-16T17:20:11.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.021+0000 7f58d4a00640 1 -- 172.21.15.18:0/3262530568 learned_addr learned my addr 172.21.15.18:0/3262530568 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.022+0000 7f58cf400640 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74dbeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.022+0000 7f58d4a00640 1 -- 172.21.15.18:0/3262530568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 msgr2=0x562b74dbc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.022+0000 7f58d4a00640 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74dbc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.022+0000 7f58d4a00640 1 -- 172.21.15.18:0/3262530568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 msgr2=0x562b74dbeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.022+0000 7f58d4a00640 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74dbeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.022+0000 7f58d4a00640 1 -- 172.21.15.18:0/3262530568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b74af57d0 con 0x562b74db31b0 2024-04-16T17:20:11.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.023+0000 7f58cf400640 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74dbeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:11.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.023+0000 7f58d4a00640 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74db3590 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009e80 tx=0x7f58c800f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=871b240f714eb11c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.023+0000 7f58c5200640 1 -- 172.21.15.18:0/3262530568 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c800c610 con 0x562b74db31b0 2024-04-16T17:20:11.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.025+0000 7f58c5200640 1 -- 172.21.15.18:0/3262530568 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c800cc90 con 0x562b74db31b0 2024-04-16T17:20:11.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.025+0000 7f58c5200640 1 -- 172.21.15.18:0/3262530568 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8017db0 con 0x562b74db31b0 2024-04-16T17:20:11.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.027+0000 7f58ccc00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58b40a3e70 0x7f58b40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.027+0000 7f58ccc00640 1 -- 172.21.15.18:0/4182904172 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f58bc0d0b40 con 0x7f58d0101980 2024-04-16T17:20:11.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.028+0000 7f58d9798740 1 -- 172.21.15.18:0/3262530568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 msgr2=0x562b74db3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.029+0000 7f58d9798740 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74db3590 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009e80 tx=0x7f58c800f7d0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.029+0000 7f58cfe00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58b40a3e70 0x7f58b40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.029+0000 7f58d9798740 1 -- 172.21.15.18:0/3262530568 shutdown_connections 2024-04-16T17:20:11.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.029+0000 7f58d9798740 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74dbeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.029+0000 7f58d9798740 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74dbc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.029+0000 7f58d9798740 1 --2- 172.21.15.18:0/3262530568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74db3590 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.029+0000 7f58d9798740 1 -- 172.21.15.18:0/3262530568 >> 172.21.15.18:0/3262530568 conn(0x562b74dad8e0 msgr2=0x562b74dadcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.030+0000 7f58d9798740 1 -- 172.21.15.18:0/3262530568 shutdown_connections 2024-04-16T17:20:11.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.031+0000 7f58d9798740 1 -- 172.21.15.18:0/3262530568 wait complete. 2024-04-16T17:20:11.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.031+0000 7f58cfe00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58b40a3e70 0x7f58b40a6330 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f58c0005db0 tx=0x7f58c002e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.032+0000 7f58d9798740 1 -- start start 2024-04-16T17:20:11.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.032+0000 7f58d9798740 1 Processor -- start 2024-04-16T17:20:11.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.034+0000 7f58d9798740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db31b0 0x562b74db96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.035+0000 7f58d9798740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74db9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.035+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db31b0 0x562b74db96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.035+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db31b0 0x562b74db96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59106/0 (socket says 172.21.15.18:59106) 2024-04-16T17:20:11.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.035+0000 7f58cfe00640 1 -- 172.21.15.18:0/1491656612 learned_addr learned my addr 172.21.15.18:0/1491656612 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.036+0000 7f58d9798740 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74dbc6b0 0x562b74dba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.036+0000 7f58cf400640 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74db9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.036+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562b74dbf4a0 con 0x562b74db31b0 2024-04-16T17:20:11.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.036+0000 7f58d4a00640 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74dbc6b0 0x562b74dba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.037+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562b74cd2710 con 0x562b74db3ad0 2024-04-16T17:20:11.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.037+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562b74dbf620 con 0x562b74dbc6b0 2024-04-16T17:20:11.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.037+0000 7f58d9798740 10 client.0 _ll_register_callbacks cb 0x562b74ccdaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:11.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.039+0000 7f58d4a00640 1 -- 172.21.15.18:0/1491656612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 msgr2=0x562b74db9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.039+0000 7f58d4a00640 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74db9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.039+0000 7f58d4a00640 1 -- 172.21.15.18:0/1491656612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db31b0 msgr2=0x562b74db96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.040+0000 7f58d4a00640 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db31b0 0x562b74db96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.040+0000 7f58d4a00640 1 -- 172.21.15.18:0/1491656612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b74a2b3f0 con 0x562b74dbc6b0 2024-04-16T17:20:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.041+0000 7f58cfe00640 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db31b0 0x562b74db96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.042+0000 7f58d4a00640 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74dbc6b0 0x562b74dba170 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f58c800fcd0 tx=0x7f58c8009bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2e808e0f3bdf65 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.043+0000 7f58b9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.043+0000 7f58b9600640 1 -- 172.21.15.18:0/1491656612 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8027060 con 0x562b74dbc6b0 2024-04-16T17:20:11.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.043+0000 7f58b9600640 1 -- 172.21.15.18:0/1491656612 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f58c8014040 con 0x562b74dbc6b0 2024-04-16T17:20:11.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.044+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c800cc90 con 0x562b74dbc6b0 2024-04-16T17:20:11.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.049+0000 7f58b9600640 1 -- 172.21.15.18:0/1491656612 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f58c8004a40 con 0x562b74dbc6b0 2024-04-16T17:20:11.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.049+0000 7f58d9798740 10 client.29590 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:11.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.049+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562b749eb0f0 con 0x562b74dbc6b0 2024-04-16T17:20:11.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.049+0000 7f58b9600640 1 -- 172.21.15.18:0/1491656612 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c800cc90 con 0x562b74dbc6b0 2024-04-16T17:20:11.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.050+0000 7f58b9600640 1 -- 172.21.15.18:0/1491656612 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f58c800b040 con 0x562b74dbc6b0 2024-04-16T17:20:11.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.053+0000 7f58d9798740 10 client.29590 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:11.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.053+0000 7f58d9798740 10 client.29590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:11.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.053+0000 7f58d9798740 20 client.29590 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:11.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.053+0000 7f58d9798740 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b74dcc9a0 0x562b74dced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.053+0000 7f58d9798740 4 client.29590 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:11.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.053+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b74d26b40 con 0x562b74dcc9a0 2024-04-16T17:20:11.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.055+0000 7f58cfe00640 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b74dcc9a0 0x562b74dced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.057+0000 7f58cfe00640 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b74dcc9a0 0x562b74dced90 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ca24ef555905e2 server_cookie=783b3f9912ad70cc in_seq=0 out_seq=0 2024-04-16T17:20:11.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.057+0000 7f58b9600640 10 client.29590 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:11.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.059+0000 7f58b9600640 1 -- 172.21.15.18:0/1491656612 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b74d26b40 con 0x562b74dcc9a0 2024-04-16T17:20:11.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.059+0000 7f58b9600640 10 client.29590 handle_command_reply: tid=0 2024-04-16T17:20:11.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.060+0000 7f58d9798740 1 client.29590 shutdown 2024-04-16T17:20:11.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.060+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74dbc6b0 msgr2=0x562b74dba170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.060+0000 7f58d9798740 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74dbc6b0 0x562b74dba170 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f58c800fcd0 tx=0x7f58c8009bc0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 shutdown_connections 2024-04-16T17:20:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58d9798740 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b74dcc9a0 0x562b74dced90 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58d9798740 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74dbc6b0 0x562b74dba170 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58d9798740 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74db9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58d9798740 1 --2- 172.21.15.18:0/1491656612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db31b0 0x562b74db96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 >> 172.21.15.18:0/1491656612 conn(0x562b74dad8e0 msgr2=0x562b74daf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58b9600640 0 client.29590 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 shutdown_connections 2024-04-16T17:20:11.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.061+0000 7f58d9798740 1 -- 172.21.15.18:0/1491656612 wait complete. 2024-04-16T17:20:11.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.062+0000 7f58d9798740 20 client.29590 trim_cache size 0 max 16384 2024-04-16T17:20:11.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.123+0000 7f58d9798740 1 Processor -- start 2024-04-16T17:20:11.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.123+0000 7f58d9798740 1 -- start start 2024-04-16T17:20:11.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.124+0000 7f58d9798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dbbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.124+0000 7f58d9798740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db3ad0 0x562b74dc8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.124+0000 7f58d9798740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74dbc6b0 0x562b74dadcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.124+0000 7f58d9798740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01c0fe0 con 0x562b74db3ad0 2024-04-16T17:20:11.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.124+0000 7f58d9798740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01be850 con 0x562b74dbc6b0 2024-04-16T17:20:11.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.124+0000 7f58d9798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01bc6c0 con 0x562b74db31b0 2024-04-16T17:20:11.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.124+0000 7f58d4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74dbc6b0 0x562b74dadcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.125+0000 7f58d4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74dbc6b0 0x562b74dadcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42860/0 (socket says 172.21.15.18:42860) 2024-04-16T17:20:11.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.125+0000 7f58d4a00640 1 -- 172.21.15.18:0/293314177 learned_addr learned my addr 172.21.15.18:0/293314177 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.125+0000 7f58cf400640 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db3ad0 0x562b74dc8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.126+0000 7f58cfe00640 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dbbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.126+0000 7f58d4a00640 1 -- 172.21.15.18:0/293314177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 msgr2=0x562b74dbbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.126+0000 7f58d4a00640 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dbbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.126+0000 7f58d4a00640 1 -- 172.21.15.18:0/293314177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db3ad0 msgr2=0x562b74dc8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.126+0000 7f58d4a00640 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db3ad0 0x562b74dc8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.126+0000 7f58d4a00640 1 -- 172.21.15.18:0/293314177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b74db3900 con 0x562b74dbc6b0 2024-04-16T17:20:11.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.127+0000 7f58cfe00640 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dbbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:11.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.128+0000 7f58d4a00640 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74dbc6b0 0x562b74dadcc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009ff0 tx=0x7f58c8009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b47a5ececa5fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.128+0000 7f58bbe00640 1 -- 172.21.15.18:0/293314177 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8027070 con 0x562b74dbc6b0 2024-04-16T17:20:11.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.129+0000 7f58bbe00640 1 -- 172.21.15.18:0/293314177 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c8017e80 con 0x562b74dbc6b0 2024-04-16T17:20:11.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.129+0000 7f58bbe00640 1 -- 172.21.15.18:0/293314177 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8031080 con 0x562b74dbc6b0 2024-04-16T17:20:11.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.130+0000 7f58d9798740 1 -- 172.21.15.18:0/293314177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74dbc6b0 msgr2=0x562b74dadcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.130+0000 7f58d9798740 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74dbc6b0 0x562b74dadcc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009ff0 tx=0x7f58c8009710 comp rx=0 tx=0).stop 2024-04-16T17:20:11.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.130+0000 7f58d9798740 1 -- 172.21.15.18:0/293314177 shutdown_connections 2024-04-16T17:20:11.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.130+0000 7f58d9798740 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74dbc6b0 0x562b74dadcc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.130+0000 7f58d9798740 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74db3ad0 0x562b74dc8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.130+0000 7f58d9798740 1 --2- 172.21.15.18:0/293314177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dbbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.131+0000 7f58d9798740 1 -- 172.21.15.18:0/293314177 >> 172.21.15.18:0/293314177 conn(0x562b74dad8e0 msgr2=0x562b74db96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.134+0000 7f58d9798740 1 -- 172.21.15.18:0/293314177 shutdown_connections 2024-04-16T17:20:11.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.134+0000 7f58d9798740 1 -- 172.21.15.18:0/293314177 wait complete. 2024-04-16T17:20:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.135+0000 7f58d9798740 1 -- start start 2024-04-16T17:20:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.135+0000 7f58d9798740 1 Processor -- start 2024-04-16T17:20:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.136+0000 7f58d9798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dc4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.136+0000 7f58d9798740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74e2e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.136+0000 7f58d9798740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74e2e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.136+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dc4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.137+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dc4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49140/0 (socket says 172.21.15.18:49140) 2024-04-16T17:20:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.138+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01bcd10 con 0x562b74dbc6b0 2024-04-16T17:20:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.138+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01be180 con 0x562b74db3ad0 2024-04-16T17:20:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.138+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01bd3e0 con 0x562b74db31b0 2024-04-16T17:20:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.138+0000 7f58d9798740 10 client.0 _ll_register_callbacks cb 0x7f58c8009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.138+0000 7f58cfe00640 1 -- 172.21.15.18:0/4186820114 learned_addr learned my addr 172.21.15.18:0/4186820114 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.138+0000 7f58cf400640 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74e2e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.138+0000 7f58d4a00640 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74e2e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.141+0000 7f58cf400640 1 -- 172.21.15.18:0/4186820114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 msgr2=0x562b74dc4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.141+0000 7f58cf400640 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dc4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.141+0000 7f58cf400640 1 -- 172.21.15.18:0/4186820114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 msgr2=0x562b74e2e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.141+0000 7f58cf400640 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74e2e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.141+0000 7f58cf400640 1 -- 172.21.15.18:0/4186820114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b74e2eb00 con 0x562b74db3ad0 2024-04-16T17:20:11.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.141+0000 7f58d4a00640 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74e2e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:11.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.141+0000 7f58cf400640 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74e2e000 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0cde30 tx=0x7f58bc09c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd817a2ec8f6d249 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.142+0000 7f58b9600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:11.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.142+0000 7f58b9600640 1 -- 172.21.15.18:0/4186820114 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc09ce00 con 0x562b74db3ad0 2024-04-16T17:20:11.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.143+0000 7f58b9600640 1 -- 172.21.15.18:0/4186820114 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7f58bc0a4ee0 con 0x562b74db3ad0 2024-04-16T17:20:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.146+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c8017e80 con 0x562b74db3ad0 2024-04-16T17:20:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.146+0000 7f58b9600640 1 -- 172.21.15.18:0/4186820114 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f58bc09c700 con 0x562b74db3ad0 2024-04-16T17:20:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.146+0000 7f58d9798740 10 client.29756 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.146+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562b74e43e40 con 0x562b74db3ad0 2024-04-16T17:20:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.147+0000 7f58b9600640 1 -- 172.21.15.18:0/4186820114 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c8017e80 con 0x562b74db3ad0 2024-04-16T17:20:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.147+0000 7f58b9600640 1 -- 172.21.15.18:0/4186820114 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f58bc0b5020 con 0x562b74db3ad0 2024-04-16T17:20:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.147+0000 7f58d9798740 10 client.29756 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.147+0000 7f58d9798740 10 client.29756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.147+0000 7f58d9798740 20 client.29756 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.147+0000 7f58d9798740 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b74e44620 0x562b74e46a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.147+0000 7f58d9798740 4 client.29756 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:20:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.147+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b74d209b0 con 0x562b74e44620 2024-04-16T17:20:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.148+0000 7f58cfe00640 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b74e44620 0x562b74e46a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.148+0000 7f58cfe00640 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b74e44620 0x562b74e46a40 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e814ac36aa1d6b72 server_cookie=5bf6db9267b8d4c7 in_seq=0 out_seq=0 2024-04-16T17:20:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.148+0000 7f58b9600640 10 client.29756 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:11.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.154+0000 7f58b9600640 1 -- 172.21.15.18:0/4186820114 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x562b74d209b0 con 0x562b74e44620 2024-04-16T17:20:11.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.154+0000 7f58b9600640 10 client.29756 handle_command_reply: tid=0 2024-04-16T17:20:11.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.154+0000 7f58d9798740 1 client.29756 shutdown 2024-04-16T17:20:11.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 msgr2=0x562b74e2e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58d9798740 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74e2e000 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0cde30 tx=0x7f58bc09c240 comp rx=0 tx=0).stop 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 shutdown_connections 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58d9798740 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562b74e44620 0x562b74e46a40 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58d9798740 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b74dbc6b0 0x562b74e2e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58d9798740 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b74db3ad0 0x562b74e2e000 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58d9798740 1 --2- 172.21.15.18:0/4186820114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562b74db31b0 0x562b74dc4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 >> 172.21.15.18:0/4186820114 conn(0x562b74dad8e0 msgr2=0x562b74db9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.161+0000 7f58b9600640 0 client.29756 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.162+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 shutdown_connections 2024-04-16T17:20:11.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.162+0000 7f58d9798740 1 -- 172.21.15.18:0/4186820114 wait complete. 2024-04-16T17:20:11.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.164+0000 7f58d9798740 20 client.29756 trim_cache size 0 max 16384 2024-04-16T17:20:11.164 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:20:11.164 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (502 inodes in the stack)", 2024-04-16T17:20:11.164 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:20:11.164 INFO:teuthology.orchestra.run.smithi018.stdout: "18038043-f815-4918-a3a6-1211cf7f6eec": { 2024-04-16T17:20:11.164 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:20:11.164 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "18038043-f815-4918-a3a6-1211cf7f6eec", 2024-04-16T17:20:11.165 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:20:11.165 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:20:11.165 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:20:11.165 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:20:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.166+0000 7f58d6a00640 1 -- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58b40a3e70 msgr2=0x7f58b40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.166+0000 7f58d6a00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58b40a3e70 0x7f58b40a6330 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f58c0005db0 tx=0x7f58c002e040 comp rx=0 tx=0).stop 2024-04-16T17:20:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.166+0000 7f58d6a00640 1 -- 172.21.15.18:0/4182904172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d0101980 msgr2=0x7f58d010e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.166+0000 7f58d6a00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d0101980 0x7f58d010e370 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00c9a0 tx=0x7f58bc00ce60 comp rx=0 tx=0).stop 2024-04-16T17:20:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.167+0000 7f58d6a00640 1 -- 172.21.15.18:0/4182904172 shutdown_connections 2024-04-16T17:20:11.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.167+0000 7f58d6a00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58b40a3e70 0x7f58b40a6330 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.167+0000 7f58d6a00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58d0106380 0x7f58d010e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.167+0000 7f58d6a00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58d0101980 0x7f58d010e370 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.167+0000 7f58d6a00640 1 --2- 172.21.15.18:0/4182904172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0101060 0x7f58d01152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.167+0000 7f58d6a00640 1 -- 172.21.15.18:0/4182904172 >> 172.21.15.18:0/4182904172 conn(0x7f58d00780b0 msgr2=0x7f58d0103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.167+0000 7f58d6a00640 1 -- 172.21.15.18:0/4182904172 shutdown_connections 2024-04-16T17:20:11.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:11.167+0000 7f58d6a00640 1 -- 172.21.15.18:0/4182904172 wait complete. 2024-04-16T17:20:11.200 INFO:tasks.cephfs.filesystem:scrub status for tag:18038043-f815-4918-a3a6-1211cf7f6eec - {'path': '/', 'tag': '18038043-f815-4918-a3a6-1211cf7f6eec', 'options': 'recursive,force'} 2024-04-16T17:20:11.466 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36768 11.18 MB/sec execute 23 sec latency 2120.145 ms 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.704+0000 7fa179e00640 1 -- 172.21.15.18:0/449041063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 msgr2=0x7fa17410e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.704+0000 7fa179e00640 1 --2- 172.21.15.18:0/449041063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 0x7fa17410e470 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa160009860 tx=0x7fa16002bb70 comp rx=0 tx=0).stop 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.704+0000 7fa179e00640 1 -- 172.21.15.18:0/449041063 shutdown_connections 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.705+0000 7fa179e00640 1 --2- 172.21.15.18:0/449041063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa17410e9b0 0x7fa174110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.705+0000 7fa179e00640 1 --2- 172.21.15.18:0/449041063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 0x7fa17410e470 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.705+0000 7fa179e00640 1 --2- 172.21.15.18:0/449041063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa174100e00 0x7fa1741011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.705+0000 7fa179e00640 1 -- 172.21.15.18:0/449041063 >> 172.21.15.18:0/449041063 conn(0x7fa1740fca20 msgr2=0x7fa1740fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.705+0000 7fa179e00640 1 -- 172.21.15.18:0/449041063 shutdown_connections 2024-04-16T17:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.705+0000 7fa179e00640 1 -- 172.21.15.18:0/449041063 wait complete. 2024-04-16T17:20:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.706+0000 7fa179e00640 1 Processor -- start 2024-04-16T17:20:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.706+0000 7fa179e00640 1 -- start start 2024-04-16T17:20:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.706+0000 7fa179e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa174100e00 0x7fa1741a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa179e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 0x7fa1741a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa179e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa17410e9b0 0x7fa17419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa179e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa174114650 con 0x7fa17410e9b0 2024-04-16T17:20:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa179e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741144d0 con 0x7fa1741017b0 2024-04-16T17:20:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa179e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741147d0 con 0x7fa174100e00 2024-04-16T17:20:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa174100e00 0x7fa1741a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 0x7fa1741a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 0x7fa1741a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42880/0 (socket says 172.21.15.18:42880) 2024-04-16T17:20:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa174100e00 0x7fa1741a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49150/0 (socket says 172.21.15.18:49150) 2024-04-16T17:20:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172000640 1 -- 172.21.15.18:0/2254811318 learned_addr learned my addr 172.21.15.18:0/2254811318 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172a00640 1 -- 172.21.15.18:0/2254811318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 msgr2=0x7fa1741a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172a00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 0x7fa1741a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172a00640 1 -- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa17410e9b0 msgr2=0x7fa17419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172a00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa17410e9b0 0x7fa17419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.707+0000 7fa172a00640 1 -- 172.21.15.18:0/2254811318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17419b1b0 con 0x7fa174100e00 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.708+0000 7fa172a00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa174100e00 0x7fa1741a0830 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa164009fd0 tx=0x7fa16400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.708+0000 7fa178a00640 1 -- 172.21.15.18:0/2254811318 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164003970 con 0x7fa174100e00 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.708+0000 7fa178a00640 1 -- 172.21.15.18:0/2254811318 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa164004020 con 0x7fa174100e00 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.708+0000 7fa178a00640 1 -- 172.21.15.18:0/2254811318 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16400bb30 con 0x7fa174100e00 2024-04-16T17:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.708+0000 7fa179e00640 1 -- 172.21.15.18:0/2254811318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17419b4a0 con 0x7fa174100e00 2024-04-16T17:20:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.708+0000 7fa179e00640 1 -- 172.21.15.18:0/2254811318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1741a76f0 con 0x7fa174100e00 2024-04-16T17:20:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.711+0000 7fa178a00640 1 -- 172.21.15.18:0/2254811318 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa16401d020 con 0x7fa174100e00 2024-04-16T17:20:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa17ca06740 1 Processor -- start 2024-04-16T17:20:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa17ca06740 1 -- start start 2024-04-16T17:20:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa178a00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1580a3d90 0x7fa1580a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d43360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d43d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa17ca06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559d46ca8780 con 0x559d46d44610 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa17ca06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559d46c4ef40 con 0x559d46d438a0 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa17ca06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d46ca4740 con 0x559d46d4ce20 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa178a00640 1 -- 172.21.15.18:0/2254811318 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7fa1640d0ab0 con 0x7fa174100e00 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.713+0000 7fa173400640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1580a3d90 0x7fa1580a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa173400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa173400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49162/0 (socket says 172.21.15.18:49162) 2024-04-16T17:20:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa173400640 1 -- 172.21.15.18:0/2904781468 learned_addr learned my addr 172.21.15.18:0/2904781468 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa172a00640 1 --2- 172.21.15.18:0/2904781468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d43360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa173400640 1 -- 172.21.15.18:0/2904781468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 msgr2=0x559d46d43d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:20:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa173400640 1 --2- 172.21.15.18:0/2904781468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d43d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa173400640 1 -- 172.21.15.18:0/2904781468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 msgr2=0x559d46d43360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa173400640 1 --2- 172.21.15.18:0/2904781468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d43360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.714+0000 7fa173400640 1 -- 172.21.15.18:0/2904781468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d469bb210 con 0x559d46d4ce20 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.715+0000 7fa173400640 1 --2- 172.21.15.18:0/2904781468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4f210 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa16c013530 tx=0x7fa16c0139f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf04d4d410871f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.715+0000 7fa15fe00640 1 -- 172.21.15.18:0/2904781468 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c01d420 con 0x559d46d4ce20 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.715+0000 7fa173400640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1580a3d90 0x7fa1580a6250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa16c013ef0 tx=0x7fa16c01c000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.715+0000 7fa15fe00640 1 -- 172.21.15.18:0/2904781468 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c01daa0 con 0x559d46d4ce20 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.715+0000 7fa15fe00640 1 -- 172.21.15.18:0/2904781468 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c011ae0 con 0x559d46d4ce20 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.716+0000 7fa17ca06740 1 -- 172.21.15.18:0/2904781468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 msgr2=0x559d46d4f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.716+0000 7fa17ca06740 1 --2- 172.21.15.18:0/2904781468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4f210 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa16c013530 tx=0x7fa16c0139f0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.716+0000 7fa17ca06740 1 -- 172.21.15.18:0/2904781468 shutdown_connections 2024-04-16T17:20:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.716+0000 7fa17ca06740 1 --2- 172.21.15.18:0/2904781468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4f210 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.716+0000 7fa17ca06740 1 --2- 172.21.15.18:0/2904781468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d43d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.716+0000 7fa17ca06740 1 --2- 172.21.15.18:0/2904781468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d43360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.717+0000 7fa17ca06740 1 -- 172.21.15.18:0/2904781468 >> 172.21.15.18:0/2904781468 conn(0x559d46cde5d0 msgr2=0x559d46d3e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.717+0000 7fa17ca06740 1 -- 172.21.15.18:0/2904781468 shutdown_connections 2024-04-16T17:20:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.717+0000 7fa17ca06740 1 -- 172.21.15.18:0/2904781468 wait complete. 2024-04-16T17:20:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.717+0000 7fa17ca06740 1 -- start start 2024-04-16T17:20:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.718+0000 7fa17ca06740 1 Processor -- start 2024-04-16T17:20:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.719+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d438a0 0x559d46d49ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.719+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d4a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.719+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 0x559d46d4a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.719+0000 7fa17ca06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559d46d4fa90 con 0x559d46d44610 2024-04-16T17:20:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.719+0000 7fa17ca06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559d46c62390 con 0x559d46d4ce20 2024-04-16T17:20:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.719+0000 7fa17ca06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d46d4fc10 con 0x559d46d438a0 2024-04-16T17:20:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.719+0000 7fa17ca06740 10 client.0 _ll_register_callbacks cb 0x559d46c4bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d438a0 0x559d46d49ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d438a0 0x559d46d49ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49164/0 (socket says 172.21.15.18:49164) 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172a00640 1 -- 172.21.15.18:0/1503714809 learned_addr learned my addr 172.21.15.18:0/1503714809 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa173400640 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 0x559d46d4a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172000640 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d4a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172a00640 1 -- 172.21.15.18:0/1503714809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 msgr2=0x559d46d4a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172a00640 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 0x559d46d4a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172a00640 1 -- 172.21.15.18:0/1503714809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 msgr2=0x559d46d4a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172a00640 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d4a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172a00640 1 -- 172.21.15.18:0/1503714809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d46a87620 con 0x559d46d438a0 2024-04-16T17:20:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.720+0000 7fa172000640 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.721+0000 7fa172a00640 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d438a0 0x559d46d49ca0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa1640989a0 tx=0x7fa164098230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61cb2f66bdfed3af server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.721+0000 7fa15cc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.721+0000 7fa15cc00640 1 -- 172.21.15.18:0/1503714809 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1640a5200 con 0x559d46d438a0 2024-04-16T17:20:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.721+0000 7fa15cc00640 1 -- 172.21.15.18:0/1503714809 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7fa164098c00 con 0x559d46d438a0 2024-04-16T17:20:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.721+0000 7fa17ca06740 1 -- 172.21.15.18:0/1503714809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa16c01daa0 con 0x559d46d438a0 2024-04-16T17:20:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.721+0000 7fa15cc00640 1 -- 172.21.15.18:0/1503714809 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa1640098a0 con 0x559d46d438a0 2024-04-16T17:20:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.722+0000 7fa17ca06740 10 client.29605 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.722+0000 7fa17ca06740 1 -- 172.21.15.18:0/1503714809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559d46d5bc20 con 0x559d46d438a0 2024-04-16T17:20:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.723+0000 7fa15cc00640 1 -- 172.21.15.18:0/1503714809 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c01daa0 con 0x559d46d438a0 2024-04-16T17:20:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.723+0000 7fa15cc00640 1 -- 172.21.15.18:0/1503714809 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa16400b040 con 0x559d46d438a0 2024-04-16T17:20:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.723+0000 7fa17ca06740 10 client.29605 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.724+0000 7fa17ca06740 10 client.29605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.724+0000 7fa17ca06740 20 client.29605 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.724+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d46d5d090 0x559d46d5f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.724+0000 7fa17ca06740 4 client.29605 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.724+0000 7fa17ca06740 1 -- 172.21.15.18:0/1503714809 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d46cb61a0 con 0x559d46d5d090 2024-04-16T17:20:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.724+0000 7fa172000640 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d46d5d090 0x559d46d5f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.725+0000 7fa172000640 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d46d5d090 0x559d46d5f500 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f6e12cf09990b6 server_cookie=bc6c82a87e3cccac in_seq=0 out_seq=0 2024-04-16T17:20:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.725+0000 7fa15cc00640 10 client.29605 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.725+0000 7fa15cc00640 1 -- 172.21.15.18:0/1503714809 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d46cb61a0 con 0x559d46d5d090 2024-04-16T17:20:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.725+0000 7fa15cc00640 10 client.29605 handle_command_reply: tid=0 2024-04-16T17:20:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.725+0000 7fa17ca06740 1 client.29605 shutdown 2024-04-16T17:20:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 -- 172.21.15.18:0/1503714809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d438a0 msgr2=0x559d46d49ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d438a0 0x559d46d49ca0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa1640989a0 tx=0x7fa164098230 comp rx=0 tx=0).stop 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 -- 172.21.15.18:0/1503714809 shutdown_connections 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d46d5d090 0x559d46d5f500 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 0x559d46d4a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1503714809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d438a0 0x559d46d49ca0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 -- 172.21.15.18:0/1503714809 >> 172.21.15.18:0/1503714809 conn(0x559d46cde5d0 msgr2=0x559d46d402e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa15cc00640 0 client.29605 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 -- 172.21.15.18:0/1503714809 shutdown_connections 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.726+0000 7fa17ca06740 1 -- 172.21.15.18:0/1503714809 wait complete. 2024-04-16T17:20:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.727+0000 7fa17ca06740 20 client.29605 trim_cache size 0 max 16384 2024-04-16T17:20:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa17ca06740 1 Processor -- start 2024-04-16T17:20:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa17ca06740 1 -- start start 2024-04-16T17:20:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d58990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d48e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa17ca06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741b4380 con 0x559d46d44610 2024-04-16T17:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa17ca06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741b1bf0 con 0x559d46d438a0 2024-04-16T17:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa17ca06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741afa60 con 0x559d46d4ce20 2024-04-16T17:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa172a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d58990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa172a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d58990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:42920/0 (socket says 172.21.15.18:42920) 2024-04-16T17:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa172a00640 1 -- 172.21.15.18:0/3027539677 learned_addr learned my addr 172.21.15.18:0/3027539677 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa172000640 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d48e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.745+0000 7fa173400640 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.746+0000 7fa172a00640 1 -- 172.21.15.18:0/3027539677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 msgr2=0x559d46d4a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.746+0000 7fa172a00640 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.746+0000 7fa172a00640 1 -- 172.21.15.18:0/3027539677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 msgr2=0x559d46d48e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.746+0000 7fa172a00640 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d48e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.746+0000 7fa172a00640 1 -- 172.21.15.18:0/3027539677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d46d4aa80 con 0x559d46d438a0 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.746+0000 7fa172a00640 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d58990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa16409ce40 tx=0x7fa164098170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5afc231152d57b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.746+0000 7fa15f400640 1 -- 172.21.15.18:0/3027539677 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1640bb2a0 con 0x559d46d438a0 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.746+0000 7fa15f400640 1 -- 172.21.15.18:0/3027539677 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa164064020 con 0x559d46d438a0 2024-04-16T17:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.747+0000 7fa15f400640 1 -- 172.21.15.18:0/3027539677 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164098c00 con 0x559d46d438a0 2024-04-16T17:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.747+0000 7fa17ca06740 1 -- 172.21.15.18:0/3027539677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 msgr2=0x559d46d58990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.747+0000 7fa17ca06740 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d58990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa16409ce40 tx=0x7fa164098170 comp rx=0 tx=0).stop 2024-04-16T17:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.747+0000 7fa17ca06740 1 -- 172.21.15.18:0/3027539677 shutdown_connections 2024-04-16T17:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.747+0000 7fa17ca06740 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d4ce20 0x559d46d4a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.747+0000 7fa17ca06740 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d44610 0x559d46d48e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.747+0000 7fa17ca06740 1 --2- 172.21.15.18:0/3027539677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d438a0 0x559d46d58990 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.747+0000 7fa17ca06740 1 -- 172.21.15.18:0/3027539677 >> 172.21.15.18:0/3027539677 conn(0x559d46cde5d0 msgr2=0x559d46d3f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.748+0000 7fa17ca06740 1 -- 172.21.15.18:0/3027539677 shutdown_connections 2024-04-16T17:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.748+0000 7fa17ca06740 1 -- 172.21.15.18:0/3027539677 wait complete. 2024-04-16T17:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.748+0000 7fa17ca06740 1 -- start start 2024-04-16T17:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.748+0000 7fa17ca06740 1 Processor -- start 2024-04-16T17:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d438a0 0x559d46d55380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d44610 0x559d46d506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa17ca06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 0x559d46d50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa17ca06740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741b00b0 con 0x559d46d438a0 2024-04-16T17:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa17ca06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741b1520 con 0x559d46d4ce20 2024-04-16T17:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa17ca06740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741b0780 con 0x559d46d44610 2024-04-16T17:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa17ca06740 10 client.0 _ll_register_callbacks cb 0x7fa16409c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa172000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d44610 0x559d46d506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa173400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 0x559d46d50d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa172000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d44610 0x559d46d506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49168/0 (socket says 172.21.15.18:49168) 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.749+0000 7fa172000640 1 -- 172.21.15.18:0/1888738843 learned_addr learned my addr 172.21.15.18:0/1888738843 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa172a00640 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d438a0 0x559d46d55380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa172000640 1 -- 172.21.15.18:0/1888738843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 msgr2=0x559d46d50d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa172000640 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 0x559d46d50d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa172000640 1 -- 172.21.15.18:0/1888738843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d438a0 msgr2=0x559d46d55380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa172000640 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d438a0 0x559d46d55380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa172000640 1 -- 172.21.15.18:0/1888738843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d46d51530 con 0x559d46d44610 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa172a00640 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d438a0 0x559d46d55380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa172000640 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d44610 0x559d46d506f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa1600057c0 tx=0x7fa1600040d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3974c7b4ddf4178e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.750+0000 7fa15cc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:11.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:11 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:20:11.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:11 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:11.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:11 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:20:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.751+0000 7fa15cc00640 1 -- 172.21.15.18:0/1888738843 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160039040 con 0x559d46d44610 2024-04-16T17:20:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.751+0000 7fa15cc00640 1 -- 172.21.15.18:0/1888738843 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9642+0+0 (secure 0 0 0) 0x7fa160004480 con 0x559d46d44610 2024-04-16T17:20:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.751+0000 7fa17ca06740 1 -- 172.21.15.18:0/1888738843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa164064020 con 0x559d46d44610 2024-04-16T17:20:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa15cc00640 1 -- 172.21.15.18:0/1888738843 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa160034440 con 0x559d46d44610 2024-04-16T17:20:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa17ca06740 10 client.29611 fetch_fsmap learned FSMap version 34 2024-04-16T17:20:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa17ca06740 1 -- 172.21.15.18:0/1888738843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559d46dc3820 con 0x559d46d44610 2024-04-16T17:20:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa15cc00640 1 -- 172.21.15.18:0/1888738843 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa164064020 con 0x559d46d44610 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa15cc00640 1 -- 172.21.15.18:0/1888738843 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa160044430 con 0x559d46d44610 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa17ca06740 10 client.29611 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa17ca06740 10 client.29611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa17ca06740 20 client.29611 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d46dc3fd0 0x559d46dc63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa17ca06740 4 client.29611 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.752+0000 7fa17ca06740 1 -- 172.21.15.18:0/1888738843 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d46cb0010 con 0x559d46dc3fd0 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.753+0000 7fa172a00640 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d46dc3fd0 0x559d46dc63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.753+0000 7fa172a00640 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d46dc3fd0 0x559d46dc63f0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aefaff2885223be2 server_cookie=4775edcc80c7b283 in_seq=0 out_seq=0 2024-04-16T17:20:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:11.753+0000 7fa15cc00640 10 client.29611 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:11.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:08 smithi018 ceph-mon[31776]: pgmap v489: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 350 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 726 op/s 2024-04-16T17:20:11.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:08 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2572903852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:20:11.792 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:10 smithi018 ceph-mon[31776]: pgmap v490: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 350 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 708 op/s 2024-04-16T17:20:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:11 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:20:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:11 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:11.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:11 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:20:12.043 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:11 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:20:12.043 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:11 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:12.044 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:11 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:20:12.466 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36804 10.74 MB/sec execute 24 sec latency 2818.091 ms 2024-04-16T17:20:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.704+0000 7fa15cc00640 1 -- 172.21.15.18:0/1888738843 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d46cb0010 con 0x559d46dc3fd0 2024-04-16T17:20:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.704+0000 7fa15cc00640 10 client.29611 handle_command_reply: tid=0 2024-04-16T17:20:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.704+0000 7fa17ca06740 1 client.29611 shutdown 2024-04-16T17:20:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.705+0000 7fa17ca06740 1 -- 172.21.15.18:0/1888738843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d44610 msgr2=0x559d46d506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.705+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d44610 0x559d46d506f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa1600057c0 tx=0x7fa1600040d0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.706+0000 7fa17ca06740 1 -- 172.21.15.18:0/1888738843 shutdown_connections 2024-04-16T17:20:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.706+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d46dc3fd0 0x559d46dc63f0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.706+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d46d4ce20 0x559d46d50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.706+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d46d44610 0x559d46d506f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.706+0000 7fa15cc00640 0 client.29611 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.706+0000 7fa17ca06740 1 --2- 172.21.15.18:0/1888738843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d46d438a0 0x559d46d55380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.706+0000 7fa17ca06740 1 -- 172.21.15.18:0/1888738843 >> 172.21.15.18:0/1888738843 conn(0x559d46cde5d0 msgr2=0x559d46d3ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.707+0000 7fa17ca06740 1 -- 172.21.15.18:0/1888738843 shutdown_connections 2024-04-16T17:20:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.707+0000 7fa17ca06740 1 -- 172.21.15.18:0/1888738843 wait complete. 2024-04-16T17:20:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.707+0000 7fa17ca06740 20 client.29611 trim_cache size 0 max 16384 2024-04-16T17:20:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:20:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:20:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:20:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 -- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1580a3d90 msgr2=0x7fa1580a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1580a3d90 0x7fa1580a6250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa16c013ef0 tx=0x7fa16c01c000 comp rx=0 tx=0).stop 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 -- 172.21.15.18:0/2254811318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa174100e00 msgr2=0x7fa1741a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa174100e00 0x7fa1741a0830 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa164009fd0 tx=0x7fa16400c880 comp rx=0 tx=0).stop 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 -- 172.21.15.18:0/2254811318 shutdown_connections 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa1580a3d90 0x7fa1580a6250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa17410e9b0 0x7fa17419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1741017b0 0x7fa1741a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 --2- 172.21.15.18:0/2254811318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa174100e00 0x7fa1741a0830 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.708+0000 7fa179e00640 1 -- 172.21.15.18:0/2254811318 >> 172.21.15.18:0/2254811318 conn(0x7fa1740fca20 msgr2=0x7fa1740fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.709+0000 7fa179e00640 1 -- 172.21.15.18:0/2254811318 shutdown_connections 2024-04-16T17:20:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:12.709+0000 7fa179e00640 1 -- 172.21.15.18:0/2254811318 wait complete. 2024-04-16T17:20:12.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:12 smithi043 ceph-mon[36901]: pgmap v491: 129 pgs: 16 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 60 active+clean; 352 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 489 op/s 2024-04-16T17:20:12.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:12 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:20:12.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:12 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:12.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:12 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:20:12.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:12 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:12.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:12 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:20:12.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:12 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:12.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:12 smithi022 ceph-mon[36023]: pgmap v491: 129 pgs: 16 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 60 active+clean; 352 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 489 op/s 2024-04-16T17:20:12.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:12 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:20:12.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:12 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:12.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:12 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:20:12.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:12 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:12.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:12 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:20:12.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:12 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:13.130 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:12 smithi018 ceph-mon[31776]: pgmap v491: 129 pgs: 16 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 60 active+clean; 352 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 489 op/s 2024-04-16T17:20:13.130 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:12 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:20:13.130 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:12 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:13.130 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:12 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:20:13.130 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:12 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:13.130 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:12 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:20:13.130 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:12 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:13.467 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36849 10.39 MB/sec execute 25 sec latency 3044.451 ms 2024-04-16T17:20:13.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:13 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:13.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:13 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:14.466 INFO:tasks.workunit.client.0.smithi018.stdout: 10 36968 10.31 MB/sec execute 26 sec latency 2059.880 ms 2024-04-16T17:20:14.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:14 smithi043 ceph-mon[36901]: pgmap v492: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-04-16T17:20:14.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:14 smithi022 ceph-mon[36023]: pgmap v492: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-04-16T17:20:15.468 INFO:tasks.workunit.client.0.smithi018.stdout: 10 37148 10.07 MB/sec execute 27 sec latency 1773.409 ms 2024-04-16T17:20:15.750 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:15 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:20:15.751 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:15 smithi043 ceph-mon[36901]: pgmap v493: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 420 op/s 2024-04-16T17:20:15.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:13 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:15.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:14 smithi018 ceph-mon[31776]: pgmap v492: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-04-16T17:20:15.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:15 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:20:15.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:15 smithi022 ceph-mon[36023]: pgmap v493: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 420 op/s 2024-04-16T17:20:16.469 INFO:tasks.workunit.client.0.smithi018.stdout: 10 37297 10.07 MB/sec execute 28 sec latency 1440.173 ms 2024-04-16T17:20:17.469 INFO:tasks.workunit.client.0.smithi018.stdout: 10 37504 10.18 MB/sec execute 29 sec latency 1533.665 ms 2024-04-16T17:20:18.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:17 smithi043 ceph-mon[36901]: pgmap v494: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 705 op/s 2024-04-16T17:20:18.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:15 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:20:18.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:15 smithi018 ceph-mon[31776]: pgmap v493: 129 pgs: 15 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 68 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 420 op/s 2024-04-16T17:20:18.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:17 smithi018 ceph-mon[31776]: pgmap v494: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 705 op/s 2024-04-16T17:20:18.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:17 smithi022 ceph-mon[36023]: pgmap v494: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 705 op/s 2024-04-16T17:20:18.469 INFO:tasks.workunit.client.0.smithi018.stdout: 10 37730 10.42 MB/sec execute 30 sec latency 1017.307 ms 2024-04-16T17:20:19.469 INFO:tasks.workunit.client.0.smithi018.stdout: 10 37983 10.36 MB/sec execute 31 sec latency 470.408 ms 2024-04-16T17:20:19.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:19 smithi022 ceph-mon[36023]: pgmap v495: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 346 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 520 op/s 2024-04-16T17:20:20.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:19 smithi043 ceph-mon[36901]: pgmap v495: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 346 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 520 op/s 2024-04-16T17:20:20.469 INFO:tasks.workunit.client.0.smithi018.stdout: 10 38318 10.62 MB/sec execute 32 sec latency 57.429 ms 2024-04-16T17:20:20.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:20 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:20:21.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:20 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:20:21.469 INFO:tasks.workunit.client.0.smithi018.stdout: 10 38669 10.87 MB/sec execute 33 sec latency 75.382 ms 2024-04-16T17:20:21.938 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:21 smithi022 ceph-mon[36023]: pgmap v496: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 350 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 626 op/s 2024-04-16T17:20:22.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:21 smithi043 ceph-mon[36901]: pgmap v496: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 350 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 626 op/s 2024-04-16T17:20:22.469 INFO:tasks.workunit.client.0.smithi018.stdout: 10 38821 10.73 MB/sec execute 34 sec latency 501.614 ms 2024-04-16T17:20:22.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:19 smithi018 ceph-mon[31776]: pgmap v495: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 346 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 520 op/s 2024-04-16T17:20:22.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:20 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:20:22.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:21 smithi018 ceph-mon[31776]: pgmap v496: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 350 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 626 op/s 2024-04-16T17:20:22.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:22 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:20:22.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:22 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:20:22.726 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:22 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:20:22.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:22 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:20:22.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:22 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:20:22.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:22 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:20:23.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:22 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:20:23.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:22 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:20:23.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:22 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:20:23.469 INFO:tasks.workunit.client.0.smithi018.stdout: 10 39008 10.63 MB/sec execute 35 sec latency 659.208 ms 2024-04-16T17:20:23.954 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:23 smithi022 ceph-mon[36023]: pgmap v497: 129 pgs: 129 active+clean; 348 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 548 op/s 2024-04-16T17:20:23.954 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:23 smithi022 ceph-mon[36023]: osdmap e120: 12 total, 12 up, 12 in 2024-04-16T17:20:24.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:23 smithi043 ceph-mon[36901]: pgmap v497: 129 pgs: 129 active+clean; 348 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 548 op/s 2024-04-16T17:20:24.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:23 smithi043 ceph-mon[36901]: osdmap e120: 12 total, 12 up, 12 in 2024-04-16T17:20:24.470 INFO:tasks.workunit.client.0.smithi018.stdout: 10 39412 11.09 MB/sec execute 36 sec latency 76.522 ms 2024-04-16T17:20:24.941 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:24 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:20:24.941 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:24 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:20:24.941 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:24 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:20:25.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:24 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:20:25.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:24 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:20:25.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:24 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:20:25.470 INFO:tasks.workunit.client.0.smithi018.stdout: 10 39760 11.32 MB/sec execute 37 sec latency 194.674 ms 2024-04-16T17:20:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.916+0000 7f1269c00640 1 -- 172.21.15.18:0/3254284859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126410e880 msgr2=0x7f126410ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.916+0000 7f1269c00640 1 --2- 172.21.15.18:0/3254284859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126410e880 0x7f126410ece0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1250009900 tx=0x7f125002bc60 comp rx=0 tx=0).stop 2024-04-16T17:20:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.917+0000 7f1269c00640 1 -- 172.21.15.18:0/3254284859 shutdown_connections 2024-04-16T17:20:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.917+0000 7f1269c00640 1 --2- 172.21.15.18:0/3254284859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f126411b7e0 0x7f126411dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.917+0000 7f1269c00640 1 --2- 172.21.15.18:0/3254284859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126410e880 0x7f126410ece0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.917+0000 7f1269c00640 1 --2- 172.21.15.18:0/3254284859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12641107a0 0x7f126410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.917+0000 7f1269c00640 1 -- 172.21.15.18:0/3254284859 >> 172.21.15.18:0/3254284859 conn(0x7f126406d0e0 msgr2=0x7f126406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.919+0000 7f1269c00640 1 -- 172.21.15.18:0/3254284859 shutdown_connections 2024-04-16T17:20:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.919+0000 7f1269c00640 1 -- 172.21.15.18:0/3254284859 wait complete. 2024-04-16T17:20:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.920+0000 7f1269c00640 1 Processor -- start 2024-04-16T17:20:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.920+0000 7f1269c00640 1 -- start start 2024-04-16T17:20:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.921+0000 7f1269c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126410e880 0x7f12641a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.921+0000 7f1269c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12641107a0 0x7f12641a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.921+0000 7f1269c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f126411b7e0 0x7f12641abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.921+0000 7f1269c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1264121290 con 0x7f126410e880 2024-04-16T17:20:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.921+0000 7f1269c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1264121110 con 0x7f126411b7e0 2024-04-16T17:20:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.921+0000 7f1269c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1264121410 con 0x7f12641107a0 2024-04-16T17:20:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1263400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f126411b7e0 0x7f12641abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1263400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f126411b7e0 0x7f12641abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40828/0 (socket says 172.21.15.18:40828) 2024-04-16T17:20:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1263400640 1 -- 172.21.15.18:0/2287141918 learned_addr learned my addr 172.21.15.18:0/2287141918 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1263400640 1 -- 172.21.15.18:0/2287141918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12641107a0 msgr2=0x7f12641a7c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:20:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1262a00640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126410e880 0x7f12641a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1263400640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12641107a0 0x7f12641a7c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1263400640 1 -- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126410e880 msgr2=0x7f12641a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1263400640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126410e880 0x7f12641a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.922+0000 7f1263400640 1 -- 172.21.15.18:0/2287141918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12641ac720 con 0x7f126411b7e0 2024-04-16T17:20:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.923+0000 7f1263400640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f126411b7e0 0x7f12641abff0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f125c00e9d0 tx=0x7f125c00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:25.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:25 smithi022 ceph-mon[36023]: pgmap v499: 129 pgs: 129 active+clean; 347 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 607 op/s 2024-04-16T17:20:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.923+0000 7f125b400640 1 -- 172.21.15.18:0/2287141918 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c00cd80 con 0x7f126411b7e0 2024-04-16T17:20:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.923+0000 7f1269c00640 1 -- 172.21.15.18:0/2287141918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12641aca10 con 0x7f126411b7e0 2024-04-16T17:20:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.923+0000 7f1269c00640 1 -- 172.21.15.18:0/2287141918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12641b42a0 con 0x7f126411b7e0 2024-04-16T17:20:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.923+0000 7f125b400640 1 -- 172.21.15.18:0/2287141918 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125c005020 con 0x7f126411b7e0 2024-04-16T17:20:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.925+0000 7f125b400640 1 -- 172.21.15.18:0/2287141918 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c00cd80 con 0x7f126411b7e0 2024-04-16T17:20:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.931+0000 7f126c800740 1 Processor -- start 2024-04-16T17:20:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.931+0000 7f125b400640 1 -- 172.21.15.18:0/2287141918 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f125c0051d0 con 0x7f126411b7e0 2024-04-16T17:20:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.933+0000 7f126c800740 1 -- start start 2024-04-16T17:20:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.933+0000 7f126c800740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 0x562f5ef58360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.933+0000 7f126c800740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef58d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.933+0000 7f126c800740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef64210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.933+0000 7f126c800740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562f5eebd780 con 0x562f5ef59610 2024-04-16T17:20:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.933+0000 7f126c800740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562f5ee63f40 con 0x562f5ef588a0 2024-04-16T17:20:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.933+0000 7f126c800740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f5eeb9740 con 0x562f5ef61e20 2024-04-16T17:20:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.934+0000 7f1262a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef58d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.934+0000 7f1262a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef58d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40836/0 (socket says 172.21.15.18:40836) 2024-04-16T17:20:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.934+0000 7f1262a00640 1 -- 172.21.15.18:0/4095490342 learned_addr learned my addr 172.21.15.18:0/4095490342 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.934+0000 7f1262000640 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 0x562f5ef58360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.934+0000 7f1262000640 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef64210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.935+0000 7f125b400640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12480a41d0 0x7f12480a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.935+0000 7f1262a00640 1 -- 172.21.15.18:0/4095490342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 msgr2=0x562f5ef64210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.935+0000 7f1262a00640 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef64210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.935+0000 7f1262a00640 1 -- 172.21.15.18:0/4095490342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 msgr2=0x562f5ef58360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.935+0000 7f1262a00640 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 0x562f5ef58360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.935+0000 7f1262a00640 1 -- 172.21.15.18:0/4095490342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5ebd0210 con 0x562f5ef588a0 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.935+0000 7f125b400640 1 -- 172.21.15.18:0/2287141918 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9554+0+0 (secure 0 0 0) 0x7f125c00cb80 con 0x7f126411b7e0 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.936+0000 7f1262a00640 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef58d20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f125400a920 tx=0x7f125400ade0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9a9c89accd4481 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.936+0000 7f124fe00640 1 -- 172.21.15.18:0/4095490342 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254015070 con 0x562f5ef588a0 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.936+0000 7f124fe00640 1 -- 172.21.15.18:0/4095490342 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125400cc00 con 0x562f5ef588a0 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.936+0000 7f124fe00640 1 -- 172.21.15.18:0/4095490342 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12540185c0 con 0x562f5ef588a0 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.937+0000 7f126c800740 1 -- 172.21.15.18:0/4095490342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 msgr2=0x562f5ef58d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.937+0000 7f126c800740 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef58d20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f125400a920 tx=0x7f125400ade0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.937+0000 7f1263400640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12480a41d0 0x7f12480a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.938+0000 7f126c800740 1 -- 172.21.15.18:0/4095490342 shutdown_connections 2024-04-16T17:20:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.938+0000 7f126c800740 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef64210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.938+0000 7f126c800740 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef58d20 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.938+0000 7f126c800740 1 --2- 172.21.15.18:0/4095490342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 0x562f5ef58360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.938+0000 7f126c800740 1 -- 172.21.15.18:0/4095490342 >> 172.21.15.18:0/4095490342 conn(0x562f5eef35d0 msgr2=0x562f5ef53590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.938+0000 7f126c800740 1 -- 172.21.15.18:0/4095490342 shutdown_connections 2024-04-16T17:20:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.938+0000 7f126c800740 1 -- 172.21.15.18:0/4095490342 wait complete. 2024-04-16T17:20:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.939+0000 7f1263400640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12480a41d0 0x7f12480a6690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f125c0cd4c0 tx=0x7f125c0980e0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.939+0000 7f126c800740 1 -- start start 2024-04-16T17:20:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.939+0000 7f126c800740 1 Processor -- start 2024-04-16T17:20:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.940+0000 7f126c800740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef588a0 0x562f5ef61760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.940+0000 7f126c800740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef59610 0x562f5ef5eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.940+0000 7f126c800740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef5f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.940+0000 7f126c800740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562f5ef64b50 con 0x562f5ef588a0 2024-04-16T17:20:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.940+0000 7f126c800740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562f5ee77390 con 0x562f5ef59610 2024-04-16T17:20:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.940+0000 7f126c800740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f5ef64cd0 con 0x562f5ef61e20 2024-04-16T17:20:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.940+0000 7f126c800740 10 client.0 _ll_register_callbacks cb 0x562f5ee60f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.941+0000 7f1262a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef59610 0x562f5ef5eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.941+0000 7f1262a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef59610 0x562f5ef5eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40850/0 (socket says 172.21.15.18:40850) 2024-04-16T17:20:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.941+0000 7f1262a00640 1 -- 172.21.15.18:0/2414457891 learned_addr learned my addr 172.21.15.18:0/2414457891 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.942+0000 7f1262000640 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef5f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.942+0000 7f1262000640 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef588a0 0x562f5ef61760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.942+0000 7f1262a00640 1 -- 172.21.15.18:0/2414457891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 msgr2=0x562f5ef5f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.942+0000 7f1262a00640 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef5f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.942+0000 7f1262a00640 1 -- 172.21.15.18:0/2414457891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef588a0 msgr2=0x562f5ef61760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.942+0000 7f1262a00640 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef588a0 0x562f5ef61760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.942+0000 7f1262a00640 1 -- 172.21.15.18:0/2414457891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5ec9c620 con 0x562f5ef59610 2024-04-16T17:20:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.943+0000 7f1262a00640 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef59610 0x562f5ef5eaf0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f125400a8f0 tx=0x7f1254006c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e05ad4c958b08c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.944+0000 7f124cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.944+0000 7f124cc00640 1 -- 172.21.15.18:0/2414457891 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254029060 con 0x562f5ef59610 2024-04-16T17:20:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.944+0000 7f124cc00640 1 -- 172.21.15.18:0/2414457891 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9554+0+0 (secure 0 0 0) 0x7f125401a450 con 0x562f5ef59610 2024-04-16T17:20:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.944+0000 7f126c800740 1 -- 172.21.15.18:0/2414457891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125400cc00 con 0x562f5ef59610 2024-04-16T17:20:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.944+0000 7f124cc00640 1 -- 172.21.15.18:0/2414457891 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1254015040 con 0x562f5ef59610 2024-04-16T17:20:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.944+0000 7f126c800740 10 client.29783 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.944+0000 7f126c800740 1 -- 172.21.15.18:0/2414457891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562f5ef81ba0 con 0x562f5ef59610 2024-04-16T17:20:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.945+0000 7f124cc00640 1 -- 172.21.15.18:0/2414457891 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125400cc00 con 0x562f5ef59610 2024-04-16T17:20:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.946+0000 7f124cc00640 1 -- 172.21.15.18:0/2414457891 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f12540245e0 con 0x562f5ef59610 2024-04-16T17:20:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.946+0000 7f126c800740 10 client.29783 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.946+0000 7f126c800740 10 client.29783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.946+0000 7f126c800740 20 client.29783 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.946+0000 7f126c800740 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f5ef82ef0 0x562f5ef852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.946+0000 7f126c800740 4 client.29783 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.946+0000 7f126c800740 1 -- 172.21.15.18:0/2414457891 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f5eecb1a0 con 0x562f5ef82ef0 2024-04-16T17:20:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.946+0000 7f1262000640 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f5ef82ef0 0x562f5ef852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.947+0000 7f1262000640 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f5ef82ef0 0x562f5ef852e0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cdda874c546f015 server_cookie=5b7a4848845762e3 in_seq=0 out_seq=0 2024-04-16T17:20:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.947+0000 7f124cc00640 10 client.29783 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.952+0000 7f124cc00640 1 -- 172.21.15.18:0/2414457891 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f5eecb1a0 con 0x562f5ef82ef0 2024-04-16T17:20:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.952+0000 7f124cc00640 10 client.29783 handle_command_reply: tid=0 2024-04-16T17:20:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.952+0000 7f126c800740 1 client.29783 shutdown 2024-04-16T17:20:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f126c800740 1 -- 172.21.15.18:0/2414457891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef59610 msgr2=0x562f5ef5eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f126c800740 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef59610 0x562f5ef5eaf0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f125400a8f0 tx=0x7f1254006c20 comp rx=0 tx=0).stop 2024-04-16T17:20:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f126c800740 1 -- 172.21.15.18:0/2414457891 shutdown_connections 2024-04-16T17:20:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f126c800740 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f5ef82ef0 0x562f5ef852e0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f126c800740 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef5f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f126c800740 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef59610 0x562f5ef5eaf0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f126c800740 1 --2- 172.21.15.18:0/2414457891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef588a0 0x562f5ef61760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f126c800740 1 -- 172.21.15.18:0/2414457891 >> 172.21.15.18:0/2414457891 conn(0x562f5eef35d0 msgr2=0x562f5ef552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.953+0000 7f124cc00640 0 client.29783 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.955+0000 7f126c800740 1 -- 172.21.15.18:0/2414457891 shutdown_connections 2024-04-16T17:20:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.955+0000 7f126c800740 1 -- 172.21.15.18:0/2414457891 wait complete. 2024-04-16T17:20:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.956+0000 7f126c800740 20 client.29783 trim_cache size 0 max 16384 2024-04-16T17:20:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.990+0000 7f126c800740 1 Processor -- start 2024-04-16T17:20:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.990+0000 7f126c800740 1 -- start start 2024-04-16T17:20:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.991+0000 7f126c800740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef6db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.991+0000 7f126c800740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 0x562f5ef63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.991+0000 7f126c800740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef61220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.991+0000 7f126c800740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1244012a10 con 0x562f5ef59610 2024-04-16T17:20:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.991+0000 7f126c800740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12440102e0 con 0x562f5ef588a0 2024-04-16T17:20:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.991+0000 7f126c800740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f124400e150 con 0x562f5ef61e20 2024-04-16T17:20:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef61220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef61220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56734/0 (socket says 172.21.15.18:56734) 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 -- 172.21.15.18:0/1517651899 learned_addr learned my addr 172.21.15.18:0/1517651899 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef6db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262a00640 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 0x562f5ef63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 -- 172.21.15.18:0/1517651899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 msgr2=0x562f5ef6db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef6db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 -- 172.21.15.18:0/1517651899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 msgr2=0x562f5ef63210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 0x562f5ef63210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.992+0000 7f1262000640 1 -- 172.21.15.18:0/1517651899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5ef57b70 con 0x562f5ef61e20 2024-04-16T17:20:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.993+0000 7f1262000640 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef61220 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1250000c00 tx=0x7f125002bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f414edde43c96a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.993+0000 7f124f400640 1 -- 172.21.15.18:0/1517651899 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12500043e0 con 0x562f5ef61e20 2024-04-16T17:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.993+0000 7f124f400640 1 -- 172.21.15.18:0/1517651899 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12500054e0 con 0x562f5ef61e20 2024-04-16T17:20:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.993+0000 7f124f400640 1 -- 172.21.15.18:0/1517651899 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12500043e0 con 0x562f5ef61e20 2024-04-16T17:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.994+0000 7f126c800740 1 -- 172.21.15.18:0/1517651899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 msgr2=0x562f5ef61220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.994+0000 7f126c800740 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef61220 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1250000c00 tx=0x7f125002bc60 comp rx=0 tx=0).stop 2024-04-16T17:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.994+0000 7f126c800740 1 -- 172.21.15.18:0/1517651899 shutdown_connections 2024-04-16T17:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.995+0000 7f126c800740 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef61e20 0x562f5ef61220 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.995+0000 7f126c800740 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef59610 0x562f5ef63210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.995+0000 7f126c800740 1 --2- 172.21.15.18:0/1517651899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef6db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.995+0000 7f126c800740 1 -- 172.21.15.18:0/1517651899 >> 172.21.15.18:0/1517651899 conn(0x562f5eef35d0 msgr2=0x562f5ef530c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.997+0000 7f126c800740 1 -- 172.21.15.18:0/1517651899 shutdown_connections 2024-04-16T17:20:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.997+0000 7f126c800740 1 -- 172.21.15.18:0/1517651899 wait complete. 2024-04-16T17:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.998+0000 7f126c800740 1 -- start start 2024-04-16T17:20:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.998+0000 7f126c800740 1 Processor -- start 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f126c800740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef6a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f126c800740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef59610 0x562f5ef5bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f126c800740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef61e20 0x562f5ef5c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f126c800740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f124400e7a0 con 0x562f5ef61e20 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f126c800740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f124400fc10 con 0x562f5ef588a0 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f126c800740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f124400ee70 con 0x562f5ef59610 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f126c800740 10 client.0 _ll_register_callbacks cb 0x7f1250009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f1262a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef59610 0x562f5ef5bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f1262a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef59610 0x562f5ef5bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:56744/0 (socket says 172.21.15.18:56744) 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:25.999+0000 7f1262a00640 1 -- 172.21.15.18:0/1744058418 learned_addr learned my addr 172.21.15.18:0/1744058418 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.000+0000 7f1262000640 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef6a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:26.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:25 smithi043 ceph-mon[36901]: pgmap v499: 129 pgs: 129 active+clean; 347 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 607 op/s 2024-04-16T17:20:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.000+0000 7f1262a00640 1 -- 172.21.15.18:0/1744058418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 msgr2=0x562f5ef6a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.000+0000 7f1262a00640 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef6a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.000+0000 7f1262a00640 1 -- 172.21.15.18:0/1744058418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef61e20 msgr2=0x562f5ef5c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.000+0000 7f1262a00640 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef61e20 0x562f5ef5c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.000+0000 7f1262a00640 1 -- 172.21.15.18:0/1744058418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5ef5c7f0 con 0x562f5ef59610 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.001+0000 7f1262a00640 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef59610 0x562f5ef5bd70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1254004810 tx=0x7f1254006a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d755d19b291a7cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.001+0000 7f124cc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.001+0000 7f124cc00640 1 -- 172.21.15.18:0/1744058418 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125400a020 con 0x562f5ef59610 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.002+0000 7f124cc00640 1 -- 172.21.15.18:0/1744058418 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9554+0+0 (secure 0 0 0) 0x7f125400ca00 con 0x562f5ef59610 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.002+0000 7f126c800740 1 -- 172.21.15.18:0/1744058418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12500054e0 con 0x562f5ef59610 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.002+0000 7f124cc00640 1 -- 172.21.15.18:0/1744058418 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f125403c050 con 0x562f5ef59610 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.002+0000 7f126c800740 10 client.29623 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.002+0000 7f126c800740 1 -- 172.21.15.18:0/1744058418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562f5ef60f40 con 0x562f5ef59610 2024-04-16T17:20:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.003+0000 7f124cc00640 1 -- 172.21.15.18:0/1744058418 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12500054e0 con 0x562f5ef59610 2024-04-16T17:20:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.004+0000 7f124cc00640 1 -- 172.21.15.18:0/1744058418 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1254024420 con 0x562f5ef59610 2024-04-16T17:20:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.004+0000 7f126c800740 10 client.29623 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.005+0000 7f126c800740 10 client.29623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.005+0000 7f126c800740 20 client.29623 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.005+0000 7f126c800740 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f5efe9630 0x562f5efeba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.005+0000 7f126c800740 4 client.29623 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:20:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.005+0000 7f126c800740 1 -- 172.21.15.18:0/1744058418 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f5ef5f3c0 con 0x562f5efe9630 2024-04-16T17:20:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.005+0000 7f1262000640 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f5efe9630 0x562f5efeba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.007+0000 7f1262000640 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f5efe9630 0x562f5efeba20 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb7a854be3ae820 server_cookie=aec4f5accea94b78 in_seq=0 out_seq=0 2024-04-16T17:20:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:26.007+0000 7f124cc00640 10 client.29623 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:26.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:23 smithi018 ceph-mon[31776]: pgmap v497: 129 pgs: 129 active+clean; 348 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 548 op/s 2024-04-16T17:20:26.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:23 smithi018 ceph-mon[31776]: osdmap e120: 12 total, 12 up, 12 in 2024-04-16T17:20:26.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:24 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:20:26.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:24 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:20:26.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:24 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:20:26.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:25 smithi018 ceph-mon[31776]: pgmap v499: 129 pgs: 129 active+clean; 347 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 607 op/s 2024-04-16T17:20:26.470 INFO:tasks.workunit.client.0.smithi018.stdout: 10 40008 11.32 MB/sec execute 38 sec latency 354.968 ms 2024-04-16T17:20:27.471 INFO:tasks.workunit.client.0.smithi018.stdout: 10 40156 11.27 MB/sec execute 39 sec latency 691.753 ms 2024-04-16T17:20:28.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:27 smithi043 ceph-mon[36901]: pgmap v500: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 366 op/s 2024-04-16T17:20:28.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:27 smithi022 ceph-mon[36023]: pgmap v500: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 366 op/s 2024-04-16T17:20:28.471 INFO:tasks.workunit.client.0.smithi018.stdout: 10 40454 11.36 MB/sec execute 40 sec latency 174.504 ms 2024-04-16T17:20:29.472 INFO:tasks.workunit.client.0.smithi018.stdout: 10 40744 11.50 MB/sec execute 41 sec latency 221.538 ms 2024-04-16T17:20:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:29 smithi022 ceph-mon[36023]: osdmap e121: 12 total, 12 up, 12 in 2024-04-16T17:20:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:29 smithi022 ceph-mon[36023]: pgmap v502: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-04-16T17:20:30.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:29 smithi043 ceph-mon[36901]: osdmap e121: 12 total, 12 up, 12 in 2024-04-16T17:20:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:29 smithi043 ceph-mon[36901]: pgmap v502: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-04-16T17:20:30.472 INFO:tasks.workunit.client.0.smithi018.stdout: 10 41115 11.63 MB/sec execute 42 sec latency 180.996 ms 2024-04-16T17:20:31.472 INFO:tasks.workunit.client.0.smithi018.stdout: 10 41503 11.78 MB/sec execute 43 sec latency 135.952 ms 2024-04-16T17:20:32.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:31 smithi043 ceph-mon[36901]: pgmap v503: 129 pgs: 129 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 565 op/s 2024-04-16T17:20:32.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:31 smithi022 ceph-mon[36023]: pgmap v503: 129 pgs: 129 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 565 op/s 2024-04-16T17:20:32.474 INFO:tasks.workunit.client.0.smithi018.stdout: 10 41862 12.10 MB/sec execute 44 sec latency 75.141 ms 2024-04-16T17:20:33.068 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:27 smithi018 ceph-mon[31776]: pgmap v500: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 366 op/s 2024-04-16T17:20:33.068 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:29 smithi018 ceph-mon[31776]: osdmap e121: 12 total, 12 up, 12 in 2024-04-16T17:20:33.068 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:29 smithi018 ceph-mon[31776]: pgmap v502: 129 pgs: 129 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-04-16T17:20:33.068 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:31 smithi018 ceph-mon[31776]: pgmap v503: 129 pgs: 129 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 565 op/s 2024-04-16T17:20:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f124cc00640 1 -- 172.21.15.18:0/1744058418 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f5ef5f3c0 con 0x562f5efe9630 2024-04-16T17:20:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f124cc00640 10 client.29623 handle_command_reply: tid=0 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 client.29623 shutdown 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 -- 172.21.15.18:0/1744058418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef59610 msgr2=0x562f5ef5bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef59610 0x562f5ef5bd70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1254004810 tx=0x7f1254006a70 comp rx=0 tx=0).stop 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 -- 172.21.15.18:0/1744058418 shutdown_connections 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562f5efe9630 0x562f5efeba20 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562f5ef61e20 0x562f5ef5c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f5ef59610 0x562f5ef5bd70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 --2- 172.21.15.18:0/1744058418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f5ef588a0 0x562f5ef6a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f124cc00640 0 client.29623 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.258+0000 7f126c800740 1 -- 172.21.15.18:0/1744058418 >> 172.21.15.18:0/1744058418 conn(0x562f5eef35d0 msgr2=0x562f5ef5cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.259+0000 7f126c800740 1 -- 172.21.15.18:0/1744058418 shutdown_connections 2024-04-16T17:20:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.259+0000 7f126c800740 1 -- 172.21.15.18:0/1744058418 wait complete. 2024-04-16T17:20:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.259+0000 7f126c800740 20 client.29623 trim_cache size 0 max 16384 2024-04-16T17:20:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:20:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:20:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:20:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:20:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 -- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12480a41d0 msgr2=0x7f12480a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12480a41d0 0x7f12480a6690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f125c0cd4c0 tx=0x7f125c0980e0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 -- 172.21.15.18:0/2287141918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f126411b7e0 msgr2=0x7f12641abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f126411b7e0 0x7f12641abff0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f125c00e9d0 tx=0x7f125c00ee90 comp rx=0 tx=0).stop 2024-04-16T17:20:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 -- 172.21.15.18:0/2287141918 shutdown_connections 2024-04-16T17:20:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12480a41d0 0x7f12480a6690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f126411b7e0 0x7f12641abff0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12641107a0 0x7f12641a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 --2- 172.21.15.18:0/2287141918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126410e880 0x7f12641a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 -- 172.21.15.18:0/2287141918 >> 172.21.15.18:0/2287141918 conn(0x7f126406d0e0 msgr2=0x7f1264072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 -- 172.21.15.18:0/2287141918 shutdown_connections 2024-04-16T17:20:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:33.260+0000 7f1269c00640 1 -- 172.21.15.18:0/2287141918 wait complete. 2024-04-16T17:20:33.474 INFO:tasks.workunit.client.0.smithi018.stdout: 10 41893 11.85 MB/sec execute 45 sec latency 483.812 ms 2024-04-16T17:20:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:33 smithi043 ceph-mon[36901]: pgmap v504: 129 pgs: 129 active+clean; 371 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-04-16T17:20:34.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:33 smithi022 ceph-mon[36023]: pgmap v504: 129 pgs: 129 active+clean; 371 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-04-16T17:20:34.476 INFO:tasks.workunit.client.0.smithi018.stdout: 10 42248 11.97 MB/sec execute 46 sec latency 292.086 ms 2024-04-16T17:20:35.474 INFO:tasks.workunit.client.0.smithi018.stdout: 10 42652 12.22 MB/sec execute 47 sec latency 156.157 ms 2024-04-16T17:20:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:35 smithi043 ceph-mon[36901]: pgmap v505: 129 pgs: 129 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-04-16T17:20:36.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:35 smithi022 ceph-mon[36023]: pgmap v505: 129 pgs: 129 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-04-16T17:20:36.474 INFO:tasks.workunit.client.0.smithi018.stdout: 10 43012 12.41 MB/sec execute 48 sec latency 207.738 ms 2024-04-16T17:20:37.480 INFO:tasks.workunit.client.0.smithi018.stdout: 10 43357 12.62 MB/sec execute 49 sec latency 174.320 ms 2024-04-16T17:20:37.984 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:33 smithi018 ceph-mon[31776]: pgmap v504: 129 pgs: 129 active+clean; 371 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-04-16T17:20:37.984 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:35 smithi018 ceph-mon[31776]: pgmap v505: 129 pgs: 129 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-04-16T17:20:37.984 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:37 smithi018 ceph-mon[31776]: pgmap v506: 129 pgs: 129 active+clean; 371 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 499 op/s 2024-04-16T17:20:38.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:37 smithi043 ceph-mon[36901]: pgmap v506: 129 pgs: 129 active+clean; 371 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 499 op/s 2024-04-16T17:20:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:37 smithi022 ceph-mon[36023]: pgmap v506: 129 pgs: 129 active+clean; 371 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 499 op/s 2024-04-16T17:20:38.478 INFO:tasks.workunit.client.0.smithi018.stdout: 10 43498 12.45 MB/sec execute 50 sec latency 312.623 ms 2024-04-16T17:20:39.478 INFO:tasks.workunit.client.0.smithi018.stdout: 10 43794 12.48 MB/sec execute 51 sec latency 184.824 ms 2024-04-16T17:20:40.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:39 smithi043 ceph-mon[36901]: pgmap v507: 129 pgs: 129 active+clean; 371 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-04-16T17:20:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:39 smithi022 ceph-mon[36023]: pgmap v507: 129 pgs: 129 active+clean; 371 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-04-16T17:20:40.478 INFO:tasks.workunit.client.0.smithi018.stdout: 10 44143 12.63 MB/sec execute 52 sec latency 77.419 ms 2024-04-16T17:20:41.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:20:41.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:20:41.478 INFO:tasks.workunit.client.0.smithi018.stdout: 10 44492 12.64 MB/sec execute 53 sec latency 123.355 ms 2024-04-16T17:20:41.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.488+0000 7f434b200640 1 -- 172.21.15.18:0/2108068280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4344107100 msgr2=0x7f43441074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.488+0000 7f434b200640 1 --2- 172.21.15.18:0/2108068280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4344107100 0x7f43441074e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4334009930 tx=0x7f433402bc30 comp rx=0 tx=0).stop 2024-04-16T17:20:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.491+0000 7f434b200640 1 -- 172.21.15.18:0/2108068280 shutdown_connections 2024-04-16T17:20:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.491+0000 7f434b200640 1 --2- 172.21.15.18:0/2108068280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344101260 0x7f434410ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.491+0000 7f434b200640 1 --2- 172.21.15.18:0/2108068280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43441008a0 0x7f4344100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.491+0000 7f434b200640 1 --2- 172.21.15.18:0/2108068280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4344107100 0x7f43441074e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.491+0000 7f434b200640 1 -- 172.21.15.18:0/2108068280 >> 172.21.15.18:0/2108068280 conn(0x7f43440761f0 msgr2=0x7f4344076600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.491+0000 7f434b200640 1 -- 172.21.15.18:0/2108068280 shutdown_connections 2024-04-16T17:20:41.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.492+0000 7f434b200640 1 -- 172.21.15.18:0/2108068280 wait complete. 2024-04-16T17:20:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.494+0000 7f434b200640 1 Processor -- start 2024-04-16T17:20:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.494+0000 7f434b200640 1 -- start start 2024-04-16T17:20:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.495+0000 7f434b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43441008a0 0x7f43441176e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.495+0000 7f434b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344101260 0x7f4344117c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.495+0000 7f434b200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4344107100 0x7f4344111870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.495+0000 7f434b200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f43441101f0 con 0x7f4344107100 2024-04-16T17:20:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.495+0000 7f434b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4344110070 con 0x7f43441008a0 2024-04-16T17:20:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.495+0000 7f434b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4344110370 con 0x7f4344101260 2024-04-16T17:20:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.496+0000 7f4343400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344101260 0x7f4344117c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.496+0000 7f4343400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344101260 0x7f4344117c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49380/0 (socket says 172.21.15.18:49380) 2024-04-16T17:20:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.496+0000 7f4343400640 1 -- 172.21.15.18:0/1255096151 learned_addr learned my addr 172.21.15.18:0/1255096151 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.496+0000 7f4343e00640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43441008a0 0x7f43441176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.496+0000 7f4349200640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4344107100 0x7f4344111870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.497+0000 7f4343400640 1 -- 172.21.15.18:0/1255096151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43441008a0 msgr2=0x7f43441176e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.497+0000 7f4343400640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43441008a0 0x7f43441176e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.497+0000 7f4343400640 1 -- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4344107100 msgr2=0x7f4344111870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.497+0000 7f4343400640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4344107100 0x7f4344111870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.497+0000 7f4343400640 1 -- 172.21.15.18:0/1255096151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4344112130 con 0x7f4344101260 2024-04-16T17:20:41.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.498+0000 7f4343400640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344101260 0x7f4344117c20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:41.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.499+0000 7f4340c00640 1 -- 172.21.15.18:0/1255096151 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433000cd80 con 0x7f4344101260 2024-04-16T17:20:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.499+0000 7f4340c00640 1 -- 172.21.15.18:0/1255096151 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4330010910 con 0x7f4344101260 2024-04-16T17:20:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.499+0000 7f4340c00640 1 -- 172.21.15.18:0/1255096151 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330018990 con 0x7f4344101260 2024-04-16T17:20:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.499+0000 7f434b200640 1 -- 172.21.15.18:0/1255096151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4344112420 con 0x7f4344101260 2024-04-16T17:20:41.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.500+0000 7f434b200640 1 -- 172.21.15.18:0/1255096151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441b4180 con 0x7f4344101260 2024-04-16T17:20:41.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.512+0000 7f434de15740 1 Processor -- start 2024-04-16T17:20:41.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.513+0000 7f4340c00640 1 -- 172.21.15.18:0/1255096151 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4330010ac0 con 0x7f4344101260 2024-04-16T17:20:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.514+0000 7f434de15740 1 -- start start 2024-04-16T17:20:41.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.518+0000 7f434de15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb91b0 0x56034adb9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.518+0000 7f434de15740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb9ad0 0x56034adc2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.518+0000 7f4349200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb91b0 0x56034adb9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.518+0000 7f4349200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb91b0 0x56034adb9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49396/0 (socket says 172.21.15.18:49396) 2024-04-16T17:20:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.518+0000 7f4349200640 1 -- 172.21.15.18:0/1740653917 learned_addr learned my addr 172.21.15.18:0/1740653917 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.519+0000 7f434de15740 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.519+0000 7f434de15740 1 -- 172.21.15.18:0/1740653917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56034ad1f120 con 0x56034adb9ad0 2024-04-16T17:20:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.519+0000 7f434de15740 1 -- 172.21.15.18:0/1740653917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56034acc5340 con 0x56034adc26b0 2024-04-16T17:20:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.519+0000 7f434de15740 1 -- 172.21.15.18:0/1740653917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56034ad1b0e0 con 0x56034adb91b0 2024-04-16T17:20:41.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.519+0000 7f4343400640 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4349200640 1 -- 172.21.15.18:0/1740653917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 msgr2=0x56034adc4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:41.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4349200640 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4349200640 1 -- 172.21.15.18:0/1740653917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb9ad0 msgr2=0x56034adc2170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4349200640 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb9ad0 0x56034adc2170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4349200640 1 -- 172.21.15.18:0/1740653917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56034aafb7d0 con 0x56034adb91b0 2024-04-16T17:20:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4349200640 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb91b0 0x56034adb9590 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f433c009e60 tx=0x7f433c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e21704692209326 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4340c00640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f43280a3e60 0x7f43280a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4340c00640 1 -- 172.21.15.18:0/1255096151 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f43300d0eb0 con 0x7f4344101260 2024-04-16T17:20:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4339600640 1 -- 172.21.15.18:0/1740653917 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c016070 con 0x56034adb91b0 2024-04-16T17:20:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4339600640 1 -- 172.21.15.18:0/1740653917 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c004d20 con 0x56034adb91b0 2024-04-16T17:20:41.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.521+0000 7f4339600640 1 -- 172.21.15.18:0/1740653917 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c012870 con 0x56034adb91b0 2024-04-16T17:20:41.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.523+0000 7f434de15740 1 -- 172.21.15.18:0/1740653917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb91b0 msgr2=0x56034adb9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.523+0000 7f434de15740 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb91b0 0x56034adb9590 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f433c009e60 tx=0x7f433c010740 comp rx=0 tx=0).stop 2024-04-16T17:20:41.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.523+0000 7f434de15740 1 -- 172.21.15.18:0/1740653917 shutdown_connections 2024-04-16T17:20:41.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.523+0000 7f434de15740 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.523+0000 7f434de15740 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb9ad0 0x56034adc2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.524+0000 7f434de15740 1 --2- 172.21.15.18:0/1740653917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb91b0 0x56034adb9590 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.524+0000 7f434de15740 1 -- 172.21.15.18:0/1740653917 >> 172.21.15.18:0/1740653917 conn(0x56034adb38e0 msgr2=0x56034adb3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:41.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.524+0000 7f4343e00640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f43280a3e60 0x7f43280a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.525+0000 7f434de15740 1 -- 172.21.15.18:0/1740653917 shutdown_connections 2024-04-16T17:20:41.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.525+0000 7f434de15740 1 -- 172.21.15.18:0/1740653917 wait complete. 2024-04-16T17:20:41.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.526+0000 7f434de15740 1 -- start start 2024-04-16T17:20:41.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.526+0000 7f434de15740 1 Processor -- start 2024-04-16T17:20:41.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.526+0000 7f4343e00640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f43280a3e60 0x7f43280a6320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4334009900 tx=0x7f433402df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:41.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.528+0000 7f434de15740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb91b0 0x56034adbf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.529+0000 7f434de15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.529+0000 7f434de15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.529+0000 7f4343e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb91b0 0x56034adbf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.529+0000 7f434de15740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56034adc54a0 con 0x56034adb91b0 2024-04-16T17:20:41.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.529+0000 7f4343e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb91b0 0x56034adbf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36544/0 (socket says 172.21.15.18:36544) 2024-04-16T17:20:41.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.529+0000 7f4343e00640 1 -- 172.21.15.18:0/539447239 learned_addr learned my addr 172.21.15.18:0/539447239 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:41.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.529+0000 7f4349200640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.530+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56034acd8710 con 0x56034adc26b0 2024-04-16T17:20:41.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.530+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56034adc5620 con 0x56034adb9ad0 2024-04-16T17:20:41.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.530+0000 7f434de15740 10 client.0 _ll_register_callbacks cb 0x56034acd3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:41.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.530+0000 7f4343400640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.531+0000 7f4343400640 1 -- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 msgr2=0x56034adc0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:41.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.531+0000 7f4343400640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.531+0000 7f4343400640 1 -- 172.21.15.18:0/539447239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb91b0 msgr2=0x56034adbf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:41.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.531+0000 7f4343400640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb91b0 0x56034adbf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.531+0000 7f4343400640 1 -- 172.21.15.18:0/539447239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56034aa313f0 con 0x56034adb9ad0 2024-04-16T17:20:41.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.531+0000 7f4349200640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:41.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.531+0000 7f4343e00640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb91b0 0x56034adbf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:41.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.532+0000 7f4343400640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbfbf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f43300ce220 tx=0x7f433009c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b46b43d90abc6dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:41.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.532+0000 7f432e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:41.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.532+0000 7f432e000640 1 -- 172.21.15.18:0/539447239 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43300a5450 con 0x56034adb9ad0 2024-04-16T17:20:41.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.532+0000 7f432e000640 1 -- 172.21.15.18:0/539447239 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f4330018b30 con 0x56034adb9ad0 2024-04-16T17:20:41.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.532+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c004d20 con 0x56034adb9ad0 2024-04-16T17:20:41.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f432e000640 1 -- 172.21.15.18:0/539447239 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f43300bd0b0 con 0x56034adb9ad0 2024-04-16T17:20:41.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f434de15740 10 client.29635 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:41.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56034a9f10f0 con 0x56034adb9ad0 2024-04-16T17:20:41.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f432e000640 1 -- 172.21.15.18:0/539447239 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c004d20 con 0x56034adb9ad0 2024-04-16T17:20:41.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f432e000640 1 -- 172.21.15.18:0/539447239 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f433009bec0 con 0x56034adb9ad0 2024-04-16T17:20:41.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f434de15740 10 client.29635 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:41.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f434de15740 10 client.29635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:41.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f434de15740 20 client.29635 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:41.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f434de15740 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56034add2a80 0x56034add4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f434de15740 4 client.29635 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:41.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.533+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56034ad2cb40 con 0x56034add2a80 2024-04-16T17:20:41.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.534+0000 7f4343e00640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56034add2a80 0x56034add4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.534+0000 7f4343e00640 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56034add2a80 0x56034add4e70 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eee83affe558241 server_cookie=ff9cb0df11d15770 in_seq=0 out_seq=0 2024-04-16T17:20:41.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.535+0000 7f432e000640 10 client.29635 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:41.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.536+0000 7f432e000640 1 -- 172.21.15.18:0/539447239 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56034ad2cb40 con 0x56034add2a80 2024-04-16T17:20:41.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.536+0000 7f432e000640 10 client.29635 handle_command_reply: tid=0 2024-04-16T17:20:41.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.536+0000 7f434de15740 1 client.29635 shutdown 2024-04-16T17:20:41.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.538+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 msgr2=0x56034adbfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.538+0000 7f434de15740 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbfbf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f43300ce220 tx=0x7f433009c900 comp rx=0 tx=0).stop 2024-04-16T17:20:41.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.539+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 shutdown_connections 2024-04-16T17:20:41.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.539+0000 7f434de15740 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56034add2a80 0x56034add4e70 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.539+0000 7f434de15740 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adc26b0 0x56034adc0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.539+0000 7f434de15740 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbfbf0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.539+0000 7f432e000640 0 client.29635 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:41.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.539+0000 7f434de15740 1 --2- 172.21.15.18:0/539447239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb91b0 0x56034adbf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.539+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 >> 172.21.15.18:0/539447239 conn(0x56034adb38e0 msgr2=0x56034adb5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:41.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.540+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 shutdown_connections 2024-04-16T17:20:41.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.542+0000 7f434de15740 1 -- 172.21.15.18:0/539447239 wait complete. 2024-04-16T17:20:41.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.542+0000 7f434de15740 20 client.29635 trim_cache size 0 max 16384 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.581+0000 7f434de15740 1 Processor -- start 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.581+0000 7f434de15740 1 -- start start 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f434de15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adc1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f434de15740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb9ad0 0x56034adce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f434de15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adc26b0 0x56034adb3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f434de15740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f43441c0d90 con 0x56034adb9ad0 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f434de15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43441be660 con 0x56034adb91b0 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f434de15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43441bc4d0 con 0x56034adc26b0 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f4343e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adc1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f4343e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adc1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51070/0 (socket says 172.21.15.18:51070) 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f4343e00640 1 -- 172.21.15.18:0/67362694 learned_addr learned my addr 172.21.15.18:0/67362694 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:41.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.582+0000 7f4349200640 1 --2- 172.21.15.18:0/67362694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adc26b0 0x56034adb3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.583+0000 7f4343e00640 1 -- 172.21.15.18:0/67362694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adc26b0 msgr2=0x56034adb3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.583+0000 7f4343e00640 1 --2- 172.21.15.18:0/67362694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adc26b0 0x56034adb3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.583+0000 7f4343e00640 1 -- 172.21.15.18:0/67362694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb9ad0 msgr2=0x56034adce3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.583+0000 7f4343e00640 1 --2- 172.21.15.18:0/67362694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb9ad0 0x56034adce3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.583+0000 7f4343e00640 1 -- 172.21.15.18:0/67362694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56034adadd20 con 0x56034adb91b0 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.583+0000 7f4343e00640 1 --2- 172.21.15.18:0/67362694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adc1de0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f433402c240 tx=0x7f4334043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16403b9f4d404ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.583+0000 7f4338c00640 1 -- 172.21.15.18:0/67362694 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433402c940 con 0x56034adb91b0 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.584+0000 7f4338c00640 1 -- 172.21.15.18:0/67362694 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4334046040 con 0x56034adb91b0 2024-04-16T17:20:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.584+0000 7f4338c00640 1 -- 172.21.15.18:0/67362694 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334005380 con 0x56034adb91b0 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 -- 172.21.15.18:0/67362694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 msgr2=0x56034adc1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 --2- 172.21.15.18:0/67362694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adc1de0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f433402c240 tx=0x7f4334043000 comp rx=0 tx=0).stop 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 -- 172.21.15.18:0/67362694 shutdown_connections 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 --2- 172.21.15.18:0/67362694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adc26b0 0x56034adb3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 --2- 172.21.15.18:0/67362694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adb9ad0 0x56034adce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 --2- 172.21.15.18:0/67362694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adc1de0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 -- 172.21.15.18:0/67362694 >> 172.21.15.18:0/67362694 conn(0x56034adb38e0 msgr2=0x56034adb4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 -- 172.21.15.18:0/67362694 shutdown_connections 2024-04-16T17:20:41.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.585+0000 7f434de15740 1 -- 172.21.15.18:0/67362694 wait complete. 2024-04-16T17:20:41.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.587+0000 7f434de15740 1 -- start start 2024-04-16T17:20:41.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.587+0000 7f434de15740 1 Processor -- start 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f434de15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adbc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f434de15740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f434de15740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adc26b0 0x56034ae37f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f434de15740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f43441bcb20 con 0x56034adc26b0 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f434de15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43441bdf90 con 0x56034adb91b0 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f434de15740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43441bd1f0 con 0x56034adb9ad0 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f434de15740 10 client.0 _ll_register_callbacks cb 0x7f433000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f4343e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adbc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f4343400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f4343e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adbc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51074/0 (socket says 172.21.15.18:51074) 2024-04-16T17:20:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.588+0000 7f4343e00640 1 -- 172.21.15.18:0/3323751100 learned_addr learned my addr 172.21.15.18:0/3323751100 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:41.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.589+0000 7f4343400640 1 -- 172.21.15.18:0/3323751100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 msgr2=0x56034adbc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:41.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.589+0000 7f4343400640 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adbc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.589+0000 7f4343400640 1 -- 172.21.15.18:0/3323751100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adc26b0 msgr2=0x56034ae37f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:41.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.589+0000 7f4343400640 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adc26b0 0x56034ae37f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.589+0000 7f4343400640 1 -- 172.21.15.18:0/3323751100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56034ae38470 con 0x56034adb9ad0 2024-04-16T17:20:41.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.590+0000 7f4343400640 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbcd10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x56034add1710 tx=0x7f4330000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64656e2aece0ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:41.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.590+0000 7f432e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:41.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.592+0000 7f432e000640 1 -- 172.21.15.18:0/3323751100 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43300d4630 con 0x56034adb9ad0 2024-04-16T17:20:41.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.592+0000 7f432e000640 1 -- 172.21.15.18:0/3323751100 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f4330098a50 con 0x56034adb9ad0 2024-04-16T17:20:41.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.592+0000 7f434de15740 1 -- 172.21.15.18:0/3323751100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4334046040 con 0x56034adb9ad0 2024-04-16T17:20:41.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.593+0000 7f432e000640 1 -- 172.21.15.18:0/3323751100 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f433009c3d0 con 0x56034adb9ad0 2024-04-16T17:20:41.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.593+0000 7f434de15740 10 client.29641 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:41.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.593+0000 7f434de15740 1 -- 172.21.15.18:0/3323751100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56034ae39640 con 0x56034adb9ad0 2024-04-16T17:20:41.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.593+0000 7f432e000640 1 -- 172.21.15.18:0/3323751100 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4334046040 con 0x56034adb9ad0 2024-04-16T17:20:41.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.593+0000 7f432e000640 1 -- 172.21.15.18:0/3323751100 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f433009b930 con 0x56034adb9ad0 2024-04-16T17:20:41.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.594+0000 7f434de15740 10 client.29641 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:41.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.594+0000 7f434de15740 10 client.29641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:41.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.594+0000 7f434de15740 20 client.29641 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:41.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.594+0000 7f434de15740 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56034ae39d40 0x56034ae3c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.594+0000 7f434de15740 4 client.29641 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:20:41.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.594+0000 7f434de15740 1 -- 172.21.15.18:0/3323751100 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56034ad269b0 con 0x56034ae39d40 2024-04-16T17:20:41.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.595+0000 7f4343e00640 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56034ae39d40 0x56034ae3c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.596+0000 7f4343e00640 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56034ae39d40 0x56034ae3c130 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=869e192150917c2a server_cookie=c8090daf74a220e8 in_seq=0 out_seq=0 2024-04-16T17:20:41.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.596+0000 7f432e000640 10 client.29641 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:41.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.599+0000 7f432e000640 1 -- 172.21.15.18:0/3323751100 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56034ad269b0 con 0x56034ae39d40 2024-04-16T17:20:41.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.599+0000 7f432e000640 10 client.29641 handle_command_reply: tid=0 2024-04-16T17:20:41.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.599+0000 7f434de15740 1 client.29641 shutdown 2024-04-16T17:20:41.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 -- 172.21.15.18:0/3323751100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 msgr2=0x56034adbcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbcd10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x56034add1710 tx=0x7f4330000bc0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 -- 172.21.15.18:0/3323751100 shutdown_connections 2024-04-16T17:20:41.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56034ae39d40 0x56034ae3c130 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56034adc26b0 0x56034ae37f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56034adb9ad0 0x56034adbcd10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 --2- 172.21.15.18:0/3323751100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56034adb91b0 0x56034adbc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 -- 172.21.15.18:0/3323751100 >> 172.21.15.18:0/3323751100 conn(0x56034adb38e0 msgr2=0x56034adb55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:41.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f432e000640 0 client.29641 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:41.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.600+0000 7f434de15740 1 -- 172.21.15.18:0/3323751100 shutdown_connections 2024-04-16T17:20:41.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.601+0000 7f434de15740 1 -- 172.21.15.18:0/3323751100 wait complete. 2024-04-16T17:20:41.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.601+0000 7f434de15740 20 client.29641 trim_cache size 0 max 16384 2024-04-16T17:20:41.602 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:20:41.602 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:20:41.602 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:20:41.602 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 -- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f43280a3e60 msgr2=0x7f43280a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f43280a3e60 0x7f43280a6320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4334009900 tx=0x7f433402df90 comp rx=0 tx=0).stop 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 -- 172.21.15.18:0/1255096151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344101260 msgr2=0x7f4344117c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344101260 0x7f4344117c20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 -- 172.21.15.18:0/1255096151 shutdown_connections 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f43280a3e60 0x7f43280a6320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4344107100 0x7f4344111870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344101260 0x7f4344117c20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 --2- 172.21.15.18:0/1255096151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43441008a0 0x7f43441176e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 -- 172.21.15.18:0/1255096151 >> 172.21.15.18:0/1255096151 conn(0x7f43440761f0 msgr2=0x7f43440ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:41.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 -- 172.21.15.18:0/1255096151 shutdown_connections 2024-04-16T17:20:41.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.602+0000 7f434b200640 1 -- 172.21.15.18:0/1255096151 wait complete. 2024-04-16T17:20:41.631 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:20:41.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:20:41.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.892+0000 7fe117000640 1 -- 172.21.15.18:0/1569109212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 msgr2=0x7fe11010efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.892+0000 7fe117000640 1 --2- 172.21.15.18:0/1569109212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 0x7fe11010efa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00d940 tx=0x7fe0fc02b9f0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.893+0000 7fe117000640 1 -- 172.21.15.18:0/1569109212 shutdown_connections 2024-04-16T17:20:41.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.893+0000 7fe117000640 1 --2- 172.21.15.18:0/1569109212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1101099c0 0x7fe1101158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.893+0000 7fe117000640 1 --2- 172.21.15.18:0/1569109212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe110109000 0x7fe110109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.893+0000 7fe117000640 1 --2- 172.21.15.18:0/1569109212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 0x7fe11010efa0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.893+0000 7fe117000640 1 -- 172.21.15.18:0/1569109212 >> 172.21.15.18:0/1569109212 conn(0x7fe11006e390 msgr2=0x7fe11006e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:41.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.895+0000 7fe117000640 1 -- 172.21.15.18:0/1569109212 shutdown_connections 2024-04-16T17:20:41.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.895+0000 7fe117000640 1 -- 172.21.15.18:0/1569109212 wait complete. 2024-04-16T17:20:41.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.897+0000 7fe117000640 1 Processor -- start 2024-04-16T17:20:41.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.897+0000 7fe117000640 1 -- start start 2024-04-16T17:20:41.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.898+0000 7fe117000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe110109000 0x7fe1101a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.898+0000 7fe117000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1101099c0 0x7fe1101a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.898+0000 7fe117000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 0x7fe1101abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.898+0000 7fe10fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe110109000 0x7fe1101a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.898+0000 7fe10fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe110109000 0x7fe1101a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51086/0 (socket says 172.21.15.18:51086) 2024-04-16T17:20:41.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.898+0000 7fe115000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 0x7fe1101abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.898+0000 7fe115000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 0x7fe1101abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49438/0 (socket says 172.21.15.18:49438) 2024-04-16T17:20:41.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.898+0000 7fe10fe00640 1 -- 172.21.15.18:0/2965237173 learned_addr learned my addr 172.21.15.18:0/2965237173 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:41.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.899+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe110118a00 con 0x7fe1101099c0 2024-04-16T17:20:41.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.899+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe110118880 con 0x7fe110109000 2024-04-16T17:20:41.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.899+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe110118b80 con 0x7fe11010ebc0 2024-04-16T17:20:41.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.900+0000 7fe115000640 1 -- 172.21.15.18:0/2965237173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe110109000 msgr2=0x7fe1101a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:41.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.900+0000 7fe115000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe110109000 0x7fe1101a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.900+0000 7fe115000640 1 -- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1101099c0 msgr2=0x7fe1101a7c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:20:41.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.900+0000 7fe115000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1101099c0 0x7fe1101a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:41.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.901+0000 7fe115000640 1 -- 172.21.15.18:0/2965237173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1101ac720 con 0x7fe11010ebc0 2024-04-16T17:20:41.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.901+0000 7fe115000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 0x7fe1101abff0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe100007830 tx=0x7fe100007cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:41.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.902+0000 7fe10cc00640 1 -- 172.21.15.18:0/2965237173 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100004020 con 0x7fe11010ebc0 2024-04-16T17:20:41.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.902+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1101aca10 con 0x7fe11010ebc0 2024-04-16T17:20:41.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.902+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1101b42a0 con 0x7fe11010ebc0 2024-04-16T17:20:41.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.902+0000 7fe10cc00640 1 -- 172.21.15.18:0/2965237173 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10000ee00 con 0x7fe11010ebc0 2024-04-16T17:20:41.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.903+0000 7fe10cc00640 1 -- 172.21.15.18:0/2965237173 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100004020 con 0x7fe11010ebc0 2024-04-16T17:20:41.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.905+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f0005290 con 0x7fe11010ebc0 2024-04-16T17:20:41.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.909+0000 7fe10cc00640 1 -- 172.21.15.18:0/2965237173 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe10001b030 con 0x7fe11010ebc0 2024-04-16T17:20:41.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.912+0000 7fe10cc00640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0f40a4120 0x7fe0f40a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:41.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.912+0000 7fe10cc00640 1 -- 172.21.15.18:0/2965237173 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7fe100019070 con 0x7fe11010ebc0 2024-04-16T17:20:41.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.913+0000 7fe10fe00640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0f40a4120 0x7fe0f40a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:41.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.915+0000 7fe10fe00640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0f40a4120 0x7fe0f40a65e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0057c0 tx=0x7fe0fc005a00 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:41.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:41.919+0000 7fe10cc00640 1 -- 172.21.15.18:0/2965237173 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe100022020 con 0x7fe11010ebc0 2024-04-16T17:20:42.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:41 smithi043 ceph-mon[36901]: pgmap v508: 129 pgs: 129 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 459 op/s 2024-04-16T17:20:42.007 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:41 smithi022 ceph-mon[36023]: pgmap v508: 129 pgs: 129 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 459 op/s 2024-04-16T17:20:42.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.421+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0f0005850 con 0x7fe11010ebc0 2024-04-16T17:20:42.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.427+0000 7fe10cc00640 1 -- 172.21.15.18:0/2965237173 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11854 (secure 0 0 0) 0x7fe10009c3e0 con 0x7fe11010ebc0 2024-04-16T17:20:42.428 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:20:42.428 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:20:13.845325+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28613":{"gid":28613,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/3815032359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3815032359},{"type":"v1","addr":"172.21.15.22:6837","nonce":3815032359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:20:42.428 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 35 2024-04-16T17:20:42.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0f40a4120 msgr2=0x7fe0f40a65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0f40a4120 0x7fe0f40a65e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0057c0 tx=0x7fe0fc005a00 comp rx=0 tx=0).stop 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 msgr2=0x7fe1101abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 0x7fe1101abff0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe100007830 tx=0x7fe100007cf0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 shutdown_connections 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe0f40a4120 0x7fe0f40a65e0 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0057c0 tx=0x7fe0fc005a00 comp rx=0 tx=0).stop 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe11010ebc0 0x7fe1101abff0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1101099c0 0x7fe1101a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 --2- 172.21.15.18:0/2965237173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe110109000 0x7fe1101a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.431+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 >> 172.21.15.18:0/2965237173 conn(0x7fe11006e390 msgr2=0x7fe1100726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:42.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.433+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 shutdown_connections 2024-04-16T17:20:42.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.434+0000 7fe117000640 1 -- 172.21.15.18:0/2965237173 wait complete. 2024-04-16T17:20:42.459 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:20:42.483 INFO:tasks.workunit.client.0.smithi018.stdout: 10 44860 12.76 MB/sec execute 54 sec latency 152.828 ms 2024-04-16T17:20:42.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.751+0000 7fc87d000640 1 -- 172.21.15.18:0/1733317296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc87009ad10 msgr2=0x7fc87009b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.751+0000 7fc87d000640 1 --2- 172.21.15.18:0/1733317296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc87009ad10 0x7fc87009b0f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc864009960 tx=0x7fc86402bc50 comp rx=0 tx=0).stop 2024-04-16T17:20:42.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.754+0000 7fc87d000640 1 -- 172.21.15.18:0/1733317296 shutdown_connections 2024-04-16T17:20:42.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.754+0000 7fc87d000640 1 --2- 172.21.15.18:0/1733317296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8700052d0 0x7fc8700a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.754+0000 7fc87d000640 1 --2- 172.21.15.18:0/1733317296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc870004910 0x7fc870004d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.754+0000 7fc87d000640 1 --2- 172.21.15.18:0/1733317296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc87009ad10 0x7fc87009b0f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.754+0000 7fc87d000640 1 -- 172.21.15.18:0/1733317296 >> 172.21.15.18:0/1733317296 conn(0x7fc87000a730 msgr2=0x7fc87000ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:42.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.754+0000 7fc87d000640 1 -- 172.21.15.18:0/1733317296 shutdown_connections 2024-04-16T17:20:42.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.754+0000 7fc87d000640 1 -- 172.21.15.18:0/1733317296 wait complete. 2024-04-16T17:20:42.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.755+0000 7fc87d000640 1 Processor -- start 2024-04-16T17:20:42.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.756+0000 7fc87d000640 1 -- start start 2024-04-16T17:20:42.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.756+0000 7fc87d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc870004910 0x7fc8701346b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.756+0000 7fc87d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8700052d0 0x7fc870134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.757+0000 7fc875e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc870004910 0x7fc8701346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.757+0000 7fc875e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc870004910 0x7fc8701346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49458/0 (socket says 172.21.15.18:49458) 2024-04-16T17:20:42.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.757+0000 7fc875e00640 1 -- 172.21.15.18:0/332212410 learned_addr learned my addr 172.21.15.18:0/332212410 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:42.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.757+0000 7fc87d000640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc87009ad10 0x7fc87012e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.757+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8700a6260 con 0x7fc8700052d0 2024-04-16T17:20:42.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.757+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8700a60e0 con 0x7fc87009ad10 2024-04-16T17:20:42.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.757+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8700a63e0 con 0x7fc870004910 2024-04-16T17:20:42.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc877200640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc87009ad10 0x7fc87012e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc875400640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8700052d0 0x7fc870134bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc875e00640 1 -- 172.21.15.18:0/332212410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc87009ad10 msgr2=0x7fc87012e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc875e00640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc87009ad10 0x7fc87012e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc875e00640 1 -- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8700052d0 msgr2=0x7fc870134bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc875e00640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8700052d0 0x7fc870134bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc875e00640 1 -- 172.21.15.18:0/332212410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87012f030 con 0x7fc870004910 2024-04-16T17:20:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc875400640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8700052d0 0x7fc870134bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:42.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.758+0000 7fc875e00640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc870004910 0x7fc8701346b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc864005ab0 tx=0x7fc86402ea00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:42.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.761+0000 7fc86ea00640 1 -- 172.21.15.18:0/332212410 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864041020 con 0x7fc870004910 2024-04-16T17:20:42.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.761+0000 7fc86ea00640 1 -- 172.21.15.18:0/332212410 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86402ee00 con 0x7fc870004910 2024-04-16T17:20:42.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.761+0000 7fc86ea00640 1 -- 172.21.15.18:0/332212410 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864004ef0 con 0x7fc870004910 2024-04-16T17:20:42.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.761+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87012f2c0 con 0x7fc870004910 2024-04-16T17:20:42.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.761+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87013b450 con 0x7fc870004910 2024-04-16T17:20:42.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.773+0000 7fc87fdd2740 1 Processor -- start 2024-04-16T17:20:42.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.773+0000 7fc87fdd2740 1 -- start start 2024-04-16T17:20:42.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.774+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310121f10 0x5593101222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.774+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x55931012aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.774+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x55931012d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.774+0000 7fc87fdd2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559310087020 con 0x55931012b410 2024-04-16T17:20:42.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.774+0000 7fc87fdd2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55931002d460 con 0x559310122830 2024-04-16T17:20:42.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.774+0000 7fc87fdd2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559310082fe0 con 0x559310121f10 2024-04-16T17:20:42.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.774+0000 7fc875400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x55931012d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.775+0000 7fc875400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x55931012d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36608/0 (socket says 172.21.15.18:36608) 2024-04-16T17:20:42.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.775+0000 7fc875400640 1 -- 172.21.15.18:0/2783832615 learned_addr learned my addr 172.21.15.18:0/2783832615 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:42.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.776+0000 7fc877200640 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310121f10 0x5593101222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.776+0000 7fc875400640 1 -- 172.21.15.18:0/2783832615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310121f10 msgr2=0x5593101222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:42.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.776+0000 7fc875400640 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310121f10 0x5593101222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.776+0000 7fc875400640 1 -- 172.21.15.18:0/2783832615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 msgr2=0x55931012aed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:42.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.776+0000 7fc875400640 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x55931012aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.776+0000 7fc875400640 1 -- 172.21.15.18:0/2783832615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559310035040 con 0x55931012b410 2024-04-16T17:20:42.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.776+0000 7fc877200640 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310121f10 0x5593101222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:42.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.777+0000 7fc875400640 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x55931012d800 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc868009c00 tx=0x7fc868010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b5401e1e37acbd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:42.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.783+0000 7fc86ea00640 1 -- 172.21.15.18:0/332212410 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc864044020 con 0x7fc870004910 2024-04-16T17:20:42.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.785+0000 7fc86ea00640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc85c0a3e20 0x7fc85c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.786+0000 7fc86ea00640 1 -- 172.21.15.18:0/332212410 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7fc8640f2230 con 0x7fc870004910 2024-04-16T17:20:42.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.786+0000 7fc877200640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc85c0a3e20 0x7fc85c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.788+0000 7fc877200640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc85c0a3e20 0x7fc85c0a62e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc878067760 tx=0x7fc878050db0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:42.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.791+0000 7fc863400640 1 -- 172.21.15.18:0/2783832615 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868004030 con 0x55931012b410 2024-04-16T17:20:42.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.791+0000 7fc863400640 1 -- 172.21.15.18:0/2783832615 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8680041d0 con 0x55931012b410 2024-04-16T17:20:42.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.791+0000 7fc863400640 1 -- 172.21.15.18:0/2783832615 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868012670 con 0x55931012b410 2024-04-16T17:20:42.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.792+0000 7fc87fdd2740 1 -- 172.21.15.18:0/2783832615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 msgr2=0x55931012d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.792+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x55931012d800 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc868009c00 tx=0x7fc868010740 comp rx=0 tx=0).stop 2024-04-16T17:20:42.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.793+0000 7fc87fdd2740 1 -- 172.21.15.18:0/2783832615 shutdown_connections 2024-04-16T17:20:42.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.793+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x55931012d800 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.793+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x55931012aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.793+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/2783832615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310121f10 0x5593101222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.793+0000 7fc87fdd2740 1 -- 172.21.15.18:0/2783832615 >> 172.21.15.18:0/2783832615 conn(0x55931011c640 msgr2=0x55931011ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:42.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.794+0000 7fc87fdd2740 1 -- 172.21.15.18:0/2783832615 shutdown_connections 2024-04-16T17:20:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.794+0000 7fc87fdd2740 1 -- 172.21.15.18:0/2783832615 wait complete. 2024-04-16T17:20:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.794+0000 7fc87fdd2740 1 -- start start 2024-04-16T17:20:42.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.795+0000 7fc87fdd2740 1 Processor -- start 2024-04-16T17:20:42.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310121f10 0x559310128430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310122830 0x559310128970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x559310128eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc875e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310121f10 0x559310128430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc875e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310121f10 0x559310128430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51110/0 (socket says 172.21.15.18:51110) 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc875400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310122830 0x559310128970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc875400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310122830 0x559310128970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49462/0 (socket says 172.21.15.18:49462) 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55931012e200 con 0x55931012b410 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc875e00640 1 -- 172.21.15.18:0/3613051785 learned_addr learned my addr 172.21.15.18:0/3613051785 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559310040830 con 0x559310121f10 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55931012e380 con 0x559310122830 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.799+0000 7fc87fdd2740 10 client.0 _ll_register_callbacks cb 0x559310048150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.800+0000 7fc875400640 1 -- 172.21.15.18:0/3613051785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310121f10 msgr2=0x559310128430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.800+0000 7fc875400640 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310121f10 0x559310128430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.800+0000 7fc875400640 1 -- 172.21.15.18:0/3613051785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 msgr2=0x559310128eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.800+0000 7fc875400640 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x559310128eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.800+0000 7fc875400640 1 -- 172.21.15.18:0/3613051785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559310044920 con 0x559310122830 2024-04-16T17:20:42.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.800+0000 7fc875400640 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310122830 0x559310128970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc868003e60 tx=0x7fc868003be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9ee651f3cdffa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:42.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.801+0000 7fc857e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.801+0000 7fc857e00640 1 -- 172.21.15.18:0/3613051785 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868004380 con 0x559310122830 2024-04-16T17:20:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.801+0000 7fc857e00640 1 -- 172.21.15.18:0/3613051785 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7fc868012450 con 0x559310122830 2024-04-16T17:20:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.801+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8680041d0 con 0x559310122830 2024-04-16T17:20:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.801+0000 7fc857e00640 1 -- 172.21.15.18:0/3613051785 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc868012650 con 0x559310122830 2024-04-16T17:20:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.801+0000 7fc87fdd2740 10 client.29656 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.801+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5593100127f0 con 0x559310122830 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.802+0000 7fc857e00640 1 -- 172.21.15.18:0/3613051785 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8680041d0 con 0x559310122830 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.802+0000 7fc857e00640 1 -- 172.21.15.18:0/3613051785 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc86800f040 con 0x559310122830 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.802+0000 7fc87fdd2740 10 client.29656 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.802+0000 7fc87fdd2740 10 client.29656 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.802+0000 7fc87fdd2740 20 client.29656 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.802+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55931013b640 0x55931013da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.802+0000 7fc87fdd2740 4 client.29656 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.802+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559310094a40 con 0x55931013b640 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.803+0000 7fc875e00640 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55931013b640 0x55931013da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.803+0000 7fc875e00640 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55931013b640 0x55931013da30 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1386397d54382c75 server_cookie=6abe02db16649a0a in_seq=0 out_seq=0 2024-04-16T17:20:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.803+0000 7fc857e00640 10 client.29656 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:20:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.804+0000 7fc857e00640 1 -- 172.21.15.18:0/3613051785 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559310094a40 con 0x55931013b640 2024-04-16T17:20:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.804+0000 7fc857e00640 10 client.29656 handle_command_reply: tid=0 2024-04-16T17:20:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.804+0000 7fc87fdd2740 1 client.29656 shutdown 2024-04-16T17:20:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310122830 msgr2=0x559310128970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310122830 0x559310128970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc868003e60 tx=0x7fc868003be0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 shutdown_connections 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55931013b640 0x55931013da30 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55931012b410 0x559310128eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559310122830 0x559310128970 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/3613051785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310121f10 0x559310128430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 >> 172.21.15.18:0/3613051785 conn(0x55931011c640 msgr2=0x55931011e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc857e00640 0 client.29656 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 shutdown_connections 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 1 -- 172.21.15.18:0/3613051785 wait complete. 2024-04-16T17:20:42.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.805+0000 7fc87fdd2740 20 client.29656 trim_cache size 0 max 16384 2024-04-16T17:20:42.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.828+0000 7fc87fdd2740 1 Processor -- start 2024-04-16T17:20:42.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.829+0000 7fc87fdd2740 1 -- start start 2024-04-16T17:20:42.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310137150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x559310128cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 0x55931011b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc87fdd2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc858012990 con 0x559310121f10 2024-04-16T17:20:42.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc87fdd2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc858010260 con 0x559310122830 2024-04-16T17:20:42.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc87fdd2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc85800e0d0 con 0x55931012b410 2024-04-16T17:20:42.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc875400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x559310128cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc875400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x559310128cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51122/0 (socket says 172.21.15.18:51122) 2024-04-16T17:20:42.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.830+0000 7fc875400640 1 -- 172.21.15.18:0/1815587810 learned_addr learned my addr 172.21.15.18:0/1815587810 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.831+0000 7fc875400640 1 -- 172.21.15.18:0/1815587810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 msgr2=0x55931011b580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:20:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.831+0000 7fc875400640 1 --2- 172.21.15.18:0/1815587810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 0x55931011b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.831+0000 7fc875400640 1 -- 172.21.15.18:0/1815587810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 msgr2=0x559310137150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.831+0000 7fc875400640 1 --2- 172.21.15.18:0/1815587810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310137150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.831+0000 7fc875400640 1 -- 172.21.15.18:0/1815587810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930fd993f0 con 0x559310122830 2024-04-16T17:20:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.831+0000 7fc875400640 1 --2- 172.21.15.18:0/1815587810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x559310128cd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc868010780 tx=0x7fc8680095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71f4ac5d24ef2704 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.832+0000 7fc862a00640 1 -- 172.21.15.18:0/1815587810 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868003a40 con 0x559310122830 2024-04-16T17:20:42.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.832+0000 7fc862a00640 1 -- 172.21.15.18:0/1815587810 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc868012450 con 0x559310122830 2024-04-16T17:20:42.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.832+0000 7fc862a00640 1 -- 172.21.15.18:0/1815587810 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86802d690 con 0x559310122830 2024-04-16T17:20:42.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.833+0000 7fc87fdd2740 1 -- 172.21.15.18:0/1815587810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 msgr2=0x559310128cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.833+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/1815587810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x559310128cd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc868010780 tx=0x7fc8680095b0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.834+0000 7fc87fdd2740 1 -- 172.21.15.18:0/1815587810 shutdown_connections 2024-04-16T17:20:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.834+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/1815587810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 0x55931011b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.834+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/1815587810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x559310128cd0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.834+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/1815587810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310137150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.834+0000 7fc87fdd2740 1 -- 172.21.15.18:0/1815587810 >> 172.21.15.18:0/1815587810 conn(0x55931011c640 msgr2=0x55931011ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:42.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.834+0000 7fc87fdd2740 1 -- 172.21.15.18:0/1815587810 shutdown_connections 2024-04-16T17:20:42.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.834+0000 7fc87fdd2740 1 -- 172.21.15.18:0/1815587810 wait complete. 2024-04-16T17:20:42.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.835+0000 7fc87fdd2740 1 -- start start 2024-04-16T17:20:42.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.835+0000 7fc87fdd2740 1 Processor -- start 2024-04-16T17:20:42.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.837+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310133730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.837+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x55931012ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.837+0000 7fc87fdd2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 0x55931012f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.837+0000 7fc875e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc875e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310133730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36628/0 (socket says 172.21.15.18:36628) 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc875e00640 1 -- 172.21.15.18:0/747274643 learned_addr learned my addr 172.21.15.18:0/747274643 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc85800e720 con 0x559310121f10 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc85800fb90 con 0x559310122830 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc85800edf0 con 0x55931012b410 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc877200640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 0x55931012f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc875400640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x55931012ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc87fdd2740 10 client.0 _ll_register_callbacks cb 0x7fc868009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.838+0000 7fc875e00640 1 -- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 msgr2=0x55931012f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.839+0000 7fc875e00640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 0x55931012f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.839+0000 7fc875e00640 1 -- 172.21.15.18:0/747274643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 msgr2=0x55931012ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:42.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.839+0000 7fc875e00640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x55931012ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.839+0000 7fc875e00640 1 -- 172.21.15.18:0/747274643 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55930fe637d0 con 0x559310121f10 2024-04-16T17:20:42.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.839+0000 7fc875400640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x55931012ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:42.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.840+0000 7fc877200640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 0x55931012f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:42.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.840+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc868012450 con 0x559310121f10 2024-04-16T17:20:42.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.841+0000 7fc875e00640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310133730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc8640f2010 tx=0x7fc86402c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9036c7b238082eec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:42.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.841+0000 7fc857e00640 10 client.19278 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:20:42.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.842+0000 7fc857e00640 1 -- 172.21.15.18:0/747274643 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8640ef9c0 con 0x559310121f10 2024-04-16T17:20:42.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.842+0000 7fc857e00640 1 -- 172.21.15.18:0/747274643 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7fc86402e0a0 con 0x559310121f10 2024-04-16T17:20:42.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.842+0000 7fc857e00640 1 -- 172.21.15.18:0/747274643 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc86402e2a0 con 0x559310121f10 2024-04-16T17:20:42.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.842+0000 7fc87fdd2740 10 client.19278 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:42.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.842+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=35}) -- 0x5593101a9c10 con 0x559310121f10 2024-04-16T17:20:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.848+0000 7fc857e00640 1 -- 172.21.15.18:0/747274643 <== mon.0 v2:172.21.15.18:3300/0 4 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc8640bacb0 con 0x559310121f10 2024-04-16T17:20:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.848+0000 7fc87fdd2740 10 client.19278 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.848+0000 7fc87fdd2740 10 client.19278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:20:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.848+0000 7fc87fdd2740 20 client.19278 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.848+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5593101aa1d0 0x5593101ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.848+0000 7fc87fdd2740 4 client.19278 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:20:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.848+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55931008e8b0 con 0x5593101aa1d0 2024-04-16T17:20:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.849+0000 7fc875400640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5593101aa1d0 0x5593101ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.850+0000 7fc875400640 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5593101aa1d0 0x5593101ac5f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a95cde957ad6583 server_cookie=51fb6db1dbd12625 in_seq=0 out_seq=0 2024-04-16T17:20:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.850+0000 7fc857e00640 10 client.19278 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:20:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.850+0000 7fc857e00640 1 -- 172.21.15.18:0/747274643 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55931008e8b0 con 0x5593101aa1d0 2024-04-16T17:20:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.850+0000 7fc857e00640 10 client.19278 handle_command_reply: tid=0 2024-04-16T17:20:42.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.850+0000 7fc87fdd2740 1 client.19278 shutdown 2024-04-16T17:20:42.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 msgr2=0x559310133730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310133730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc8640f2010 tx=0x7fc86402c210 comp rx=0 tx=0).stop 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 shutdown_connections 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5593101aa1d0 0x5593101ac5f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55931012b410 0x55931012f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559310122830 0x55931012ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc87fdd2740 1 --2- 172.21.15.18:0/747274643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559310121f10 0x559310133730 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 >> 172.21.15.18:0/747274643 conn(0x55931011c640 msgr2=0x55931011b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.851+0000 7fc857e00640 0 client.19278 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.852+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 shutdown_connections 2024-04-16T17:20:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.852+0000 7fc87fdd2740 1 -- 172.21.15.18:0/747274643 wait complete. 2024-04-16T17:20:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.852+0000 7fc87fdd2740 20 client.19278 trim_cache size 0 max 16384 2024-04-16T17:20:42.853 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:20:42.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.854+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc85c0a3e20 msgr2=0x7fc85c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.854+0000 7fc87d000640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc85c0a3e20 0x7fc85c0a62e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc878067760 tx=0x7fc878050db0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.854+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc870004910 msgr2=0x7fc8701346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:42.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.854+0000 7fc87d000640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc870004910 0x7fc8701346b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc864005ab0 tx=0x7fc86402ea00 comp rx=0 tx=0).stop 2024-04-16T17:20:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.856+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 shutdown_connections 2024-04-16T17:20:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.856+0000 7fc87d000640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc85c0a3e20 0x7fc85c0a62e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.856+0000 7fc87d000640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc87009ad10 0x7fc87012e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.856+0000 7fc87d000640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8700052d0 0x7fc870134bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.856+0000 7fc87d000640 1 --2- 172.21.15.18:0/332212410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc870004910 0x7fc8701346b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:42.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.856+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 >> 172.21.15.18:0/332212410 conn(0x7fc87000a730 msgr2=0x7fc8700a14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:42.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.856+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 shutdown_connections 2024-04-16T17:20:42.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:42.856+0000 7fc87d000640 1 -- 172.21.15.18:0/332212410 wait complete. 2024-04-16T17:20:42.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:20:43.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:42 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2965237173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:20:43.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:42 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2965237173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:20:43.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.256+0000 7f7d4ba00640 1 -- 172.21.15.18:0/4036410337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d44104e00 msgr2=0x7f7d44108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.256+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/4036410337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d44104e00 0x7f7d44108e40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009900 tx=0x7f7d3802bc30 comp rx=0 tx=0).stop 2024-04-16T17:20:43.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.259+0000 7f7d4ba00640 1 -- 172.21.15.18:0/4036410337 shutdown_connections 2024-04-16T17:20:43.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.259+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/4036410337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d44109520 0x7f7d441110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.259+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/4036410337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d44104e00 0x7f7d44108e40 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.259+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/4036410337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d44104450 0x7f7d44104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.259+0000 7f7d4ba00640 1 -- 172.21.15.18:0/4036410337 >> 172.21.15.18:0/4036410337 conn(0x7f7d440ffcb0 msgr2=0x7f7d441020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.265+0000 7f7d4ba00640 1 -- 172.21.15.18:0/4036410337 shutdown_connections 2024-04-16T17:20:43.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.265+0000 7f7d4ba00640 1 -- 172.21.15.18:0/4036410337 wait complete. 2024-04-16T17:20:43.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.266+0000 7f7d4ba00640 1 Processor -- start 2024-04-16T17:20:43.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.267+0000 7f7d4ba00640 1 -- start start 2024-04-16T17:20:43.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.267+0000 7f7d4ba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d44104450 0x7f7d44110d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.268+0000 7f7d4ba00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d44109520 0x7f7d4411c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.268+0000 7f7d4ba00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d4410ee60 0x7f7d4410f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.268+0000 7f7d4ba00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44076a50 con 0x7f7d44109520 2024-04-16T17:20:43.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.268+0000 7f7d4ba00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d440768d0 con 0x7f7d4410ee60 2024-04-16T17:20:43.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.268+0000 7f7d4ba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44076bd0 con 0x7f7d44104450 2024-04-16T17:20:43.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.268+0000 7f7d49a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d4410ee60 0x7f7d4410f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.268+0000 7f7d49a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d4410ee60 0x7f7d4410f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51150/0 (socket says 172.21.15.18:51150) 2024-04-16T17:20:43.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.268+0000 7f7d49a00640 1 -- 172.21.15.18:0/2481213497 learned_addr learned my addr 172.21.15.18:0/2481213497 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.269+0000 7f7d49a00640 1 -- 172.21.15.18:0/2481213497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d44104450 msgr2=0x7f7d44110d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:20:43.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.269+0000 7f7d49a00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d44104450 0x7f7d44110d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.269+0000 7f7d49a00640 1 -- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d44109520 msgr2=0x7f7d4411c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.269+0000 7f7d49a00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d44109520 0x7f7d4411c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.269+0000 7f7d49a00640 1 -- 172.21.15.18:0/2481213497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4410f820 con 0x7f7d4410ee60 2024-04-16T17:20:43.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.270+0000 7f7d49a00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d4410ee60 0x7f7d4410f2e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00b970 tx=0x7f7d3c00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.270+0000 7f7d41600640 1 -- 172.21.15.18:0/2481213497 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c019070 con 0x7f7d4410ee60 2024-04-16T17:20:43.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.271+0000 7f7d41600640 1 -- 172.21.15.18:0/2481213497 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d3c00cb40 con 0x7f7d4410ee60 2024-04-16T17:20:43.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.271+0000 7f7d41600640 1 -- 172.21.15.18:0/2481213497 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c007650 con 0x7f7d4410ee60 2024-04-16T17:20:43.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.271+0000 7f7d4ba00640 1 -- 172.21.15.18:0/2481213497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4410fb10 con 0x7f7d4410ee60 2024-04-16T17:20:43.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.271+0000 7f7d4ba00640 1 -- 172.21.15.18:0/2481213497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d441b4770 con 0x7f7d4410ee60 2024-04-16T17:20:43.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.278+0000 7f7d41600640 1 -- 172.21.15.18:0/2481213497 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d3c01c070 con 0x7f7d4410ee60 2024-04-16T17:20:43.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.279+0000 7f7d4e7b7740 1 Processor -- start 2024-04-16T17:20:43.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.279+0000 7f7d4e7b7740 1 -- start start 2024-04-16T17:20:43.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7bb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bb670 0x55e56a7c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e56a720020 con 0x55e56a7bb670 2024-04-16T17:20:43.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e56a6c6460 con 0x55e56a7c4250 2024-04-16T17:20:43.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e56a71bfe0 con 0x55e56a7bad50 2024-04-16T17:20:43.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7bb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7bb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49502/0 (socket says 172.21.15.18:49502) 2024-04-16T17:20:43.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d43e00640 1 -- 172.21.15.18:0/2511377558 learned_addr learned my addr 172.21.15.18:0/2511377558 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.280+0000 7f7d43e00640 1 --2- 172.21.15.18:0/2511377558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.281+0000 7f7d43e00640 1 -- 172.21.15.18:0/2511377558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 msgr2=0x55e56a7c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.281+0000 7f7d43e00640 1 --2- 172.21.15.18:0/2511377558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.281+0000 7f7d43e00640 1 -- 172.21.15.18:0/2511377558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bb670 msgr2=0x55e56a7c3d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.281+0000 7f7d43e00640 1 --2- 172.21.15.18:0/2511377558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bb670 0x55e56a7c3d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.281+0000 7f7d43e00640 1 -- 172.21.15.18:0/2511377558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56a6ce040 con 0x55e56a7bad50 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.281+0000 7f7d43e00640 1 --2- 172.21.15.18:0/2511377558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7bb130 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7d3802caf0 tx=0x7f7d38005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e272d9ac3c2c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.281+0000 7f7d36000640 1 -- 172.21.15.18:0/2511377558 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38057020 con 0x55e56a7bad50 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.282+0000 7f7d41600640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d2c0a3de0 0x7f7d2c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.282+0000 7f7d41600640 1 -- 172.21.15.18:0/2481213497 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f7d3c0cffe0 con 0x7f7d4410ee60 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.283+0000 7f7d36000640 1 -- 172.21.15.18:0/2511377558 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d38005520 con 0x55e56a7bad50 2024-04-16T17:20:43.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.283+0000 7f7d36000640 1 -- 172.21.15.18:0/2511377558 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38004de0 con 0x55e56a7bad50 2024-04-16T17:20:43.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.284+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/2511377558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 msgr2=0x55e56a7bb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.284+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/2511377558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7bb130 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7d3802caf0 tx=0x7f7d38005fd0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.284+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/2511377558 shutdown_connections 2024-04-16T17:20:43.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.284+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/2511377558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.284+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/2511377558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bb670 0x55e56a7c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.284+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/2511377558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7bb130 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.284+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/2511377558 >> 172.21.15.18:0/2511377558 conn(0x55e56a7b5480 msgr2=0x55e56a7b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.286+0000 7f7d49000640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d2c0a3de0 0x7f7d2c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.286+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/2511377558 shutdown_connections 2024-04-16T17:20:43.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.286+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/2511377558 wait complete. 2024-04-16T17:20:43.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.287+0000 7f7d4e7b7740 1 -- start start 2024-04-16T17:20:43.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.287+0000 7f7d4e7b7740 1 Processor -- start 2024-04-16T17:20:43.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.289+0000 7f7d49000640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d2c0a3de0 0x7f7d2c0a62a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7d30005d00 tx=0x7f7d30005c90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.290+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bad50 0x55e56a7c1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.291+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bb670 0x55e56a7c1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.291+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.291+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e56a7c6fe0 con 0x55e56a7bad50 2024-04-16T17:20:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.291+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e56a6d9830 con 0x55e56a7c4250 2024-04-16T17:20:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.291+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e56a7c7160 con 0x55e56a7bb670 2024-04-16T17:20:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.291+0000 7f7d4e7b7740 10 client.0 _ll_register_callbacks cb 0x55e56a6e1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:43.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.291+0000 7f7d49a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.292+0000 7f7d43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bb670 0x55e56a7c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.292+0000 7f7d43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bb670 0x55e56a7c1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49510/0 (socket says 172.21.15.18:49510) 2024-04-16T17:20:43.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.292+0000 7f7d43e00640 1 -- 172.21.15.18:0/3385998659 learned_addr learned my addr 172.21.15.18:0/3385998659 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.292+0000 7f7d49000640 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bad50 0x55e56a7c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.293+0000 7f7d43e00640 1 -- 172.21.15.18:0/3385998659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 msgr2=0x55e56a7c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.293+0000 7f7d43e00640 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.293+0000 7f7d43e00640 1 -- 172.21.15.18:0/3385998659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bad50 msgr2=0x55e56a7c1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.293+0000 7f7d43e00640 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bad50 0x55e56a7c1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.293+0000 7f7d43e00640 1 -- 172.21.15.18:0/3385998659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e56a6dd920 con 0x55e56a7bb670 2024-04-16T17:20:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.294+0000 7f7d49a00640 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.295+0000 7f7d43e00640 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bb670 0x55e56a7c1790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d380048f0 tx=0x7f7d38004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdf6fe344d7d28b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.295+0000 7f7d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:43.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.296+0000 7f7d2aa00640 1 -- 172.21.15.18:0/3385998659 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38057040 con 0x55e56a7bb670 2024-04-16T17:20:43.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.296+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/3385998659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d38005520 con 0x55e56a7bb670 2024-04-16T17:20:43.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.296+0000 7f7d2aa00640 1 -- 172.21.15.18:0/3385998659 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f7d38005030 con 0x55e56a7bb670 2024-04-16T17:20:43.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.297+0000 7f7d2aa00640 1 -- 172.21.15.18:0/3385998659 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7d3806a050 con 0x55e56a7bb670 2024-04-16T17:20:43.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.300+0000 7f7d4e7b7740 10 client.29665 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:43.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.300+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/3385998659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e56a6ab7f0 con 0x55e56a7bb670 2024-04-16T17:20:43.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.303+0000 7f7d2aa00640 1 -- 172.21.15.18:0/3385998659 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d38005520 con 0x55e56a7bb670 2024-04-16T17:20:43.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.304+0000 7f7d2aa00640 1 -- 172.21.15.18:0/3385998659 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7d38034440 con 0x55e56a7bb670 2024-04-16T17:20:43.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.304+0000 7f7d4e7b7740 10 client.29665 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:43.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.304+0000 7f7d4e7b7740 10 client.29665 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:20:43.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.304+0000 7f7d4e7b7740 20 client.29665 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:43.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.304+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e56a7d46d0 0x55e56a7d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.304+0000 7f7d4e7b7740 4 client.29665 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:43.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.304+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/3385998659 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e56a72da40 con 0x55e56a7d46d0 2024-04-16T17:20:43.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.305+0000 7f7d49000640 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e56a7d46d0 0x55e56a7d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.307+0000 7f7d49000640 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e56a7d46d0 0x55e56a7d6b30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=accc867c9bf353ed server_cookie=d464d53d7e4786a1 in_seq=0 out_seq=0 2024-04-16T17:20:43.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.307+0000 7f7d2aa00640 10 client.29665 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:20:43.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.309+0000 7f7d2aa00640 1 -- 172.21.15.18:0/3385998659 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e56a72da40 con 0x55e56a7d46d0 2024-04-16T17:20:43.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.309+0000 7f7d2aa00640 10 client.29665 handle_command_reply: tid=0 2024-04-16T17:20:43.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.309+0000 7f7d4e7b7740 1 client.29665 shutdown 2024-04-16T17:20:43.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.309+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/3385998659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bb670 msgr2=0x55e56a7c1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.309+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bb670 0x55e56a7c1790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d380048f0 tx=0x7f7d38004920 comp rx=0 tx=0).stop 2024-04-16T17:20:43.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.309+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/3385998659 shutdown_connections 2024-04-16T17:20:43.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.309+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e56a7d46d0 0x55e56a7d6b30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.310+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.310+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bb670 0x55e56a7c1790 secure :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d380048f0 tx=0x7f7d38004920 comp rx=0 tx=0).stop 2024-04-16T17:20:43.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.310+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/3385998659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7bad50 0x55e56a7c1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.310+0000 7f7d2aa00640 0 client.29665 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:20:43.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.310+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/3385998659 >> 172.21.15.18:0/3385998659 conn(0x55e56a7b5480 msgr2=0x55e56a7b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.313+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/3385998659 shutdown_connections 2024-04-16T17:20:43.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.313+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/3385998659 wait complete. 2024-04-16T17:20:43.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.314+0000 7f7d4e7b7740 20 client.29665 trim_cache size 0 max 16384 2024-04-16T17:20:43.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.356+0000 7f7d4e7b7740 1 Processor -- start 2024-04-16T17:20:43.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.360+0000 7f7d4e7b7740 1 -- start start 2024-04-16T17:20:43.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.360+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.360+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.360+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7b4360 0x55e56a7dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.360+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24012990 con 0x55e56a7b4360 2024-04-16T17:20:43.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.360+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24010260 con 0x55e56a7c4250 2024-04-16T17:20:43.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.360+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2400e0d0 con 0x55e56a7bad50 2024-04-16T17:20:43.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.362+0000 7f7d43e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.363+0000 7f7d43e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51174/0 (socket says 172.21.15.18:51174) 2024-04-16T17:20:43.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.363+0000 7f7d43e00640 1 -- 172.21.15.18:0/278297190 learned_addr learned my addr 172.21.15.18:0/278297190 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.363+0000 7f7d43e00640 1 -- 172.21.15.18:0/278297190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 msgr2=0x55e56a7cff30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:43.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.363+0000 7f7d43e00640 1 --2- 172.21.15.18:0/278297190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7cff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.363+0000 7f7d43e00640 1 -- 172.21.15.18:0/278297190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7b4360 msgr2=0x55e56a7dc440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:43.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.363+0000 7f7d43e00640 1 --2- 172.21.15.18:0/278297190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7b4360 0x55e56a7dc440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.363+0000 7f7d43e00640 1 -- 172.21.15.18:0/278297190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56a4323f0 con 0x55e56a7c4250 2024-04-16T17:20:43.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.364+0000 7f7d43e00640 1 --2- 172.21.15.18:0/278297190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1b10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7d38034980 tx=0x7f7d38005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a5ded038f70a064 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.364+0000 7f7d35600640 1 -- 172.21.15.18:0/278297190 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38073070 con 0x55e56a7c4250 2024-04-16T17:20:43.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.365+0000 7f7d35600640 1 -- 172.21.15.18:0/278297190 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d38004a40 con 0x55e56a7c4250 2024-04-16T17:20:43.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.365+0000 7f7d35600640 1 -- 172.21.15.18:0/278297190 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38057040 con 0x55e56a7c4250 2024-04-16T17:20:43.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.366+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/278297190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 msgr2=0x55e56a7c1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.366+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/278297190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1b10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7d38034980 tx=0x7f7d38005fd0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.367+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/278297190 shutdown_connections 2024-04-16T17:20:43.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.367+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/278297190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7b4360 0x55e56a7dc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.367+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/278297190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7c4250 0x55e56a7c1b10 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.367+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/278297190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.367+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/278297190 >> 172.21.15.18:0/278297190 conn(0x55e56a7b5480 msgr2=0x55e56a7b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.367+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/278297190 shutdown_connections 2024-04-16T17:20:43.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.367+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/278297190 wait complete. 2024-04-16T17:20:43.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.368+0000 7f7d4e7b7740 1 -- start start 2024-04-16T17:20:43.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.368+0000 7f7d4e7b7740 1 Processor -- start 2024-04-16T17:20:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.370+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7b4360 0x55e56a7be020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.370+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7be560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.370+0000 7f7d49000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7b4360 0x55e56a7be020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.371+0000 7f7d49000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7b4360 0x55e56a7be020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51186/0 (socket says 172.21.15.18:51186) 2024-04-16T17:20:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.371+0000 7f7d4e7b7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7c4250 0x55e56a7beaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.371+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2400e720 con 0x55e56a7c4250 2024-04-16T17:20:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.371+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2400fb90 con 0x55e56a7b4360 2024-04-16T17:20:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.371+0000 7f7d4e7b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2400edf0 con 0x55e56a7bad50 2024-04-16T17:20:43.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.371+0000 7f7d4e7b7740 10 client.0 _ll_register_callbacks cb 0x7f7d3802c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.372+0000 7f7d49000640 1 -- 172.21.15.18:0/199830211 learned_addr learned my addr 172.21.15.18:0/199830211 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.372+0000 7f7d43e00640 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7be560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.373+0000 7f7d49000640 1 -- 172.21.15.18:0/199830211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 msgr2=0x55e56a7be560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.373+0000 7f7d49000640 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7be560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.373+0000 7f7d49000640 1 -- 172.21.15.18:0/199830211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7c4250 msgr2=0x55e56a7beaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.373+0000 7f7d49000640 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7c4250 0x55e56a7beaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.373+0000 7f7d49000640 1 -- 172.21.15.18:0/199830211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e56a4fc7d0 con 0x55e56a7b4360 2024-04-16T17:20:43.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.373+0000 7f7d43e00640 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7be560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:43.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.374+0000 7f7d49000640 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7b4360 0x55e56a7be020 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000b7e0 tx=0x7f7d300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c52b7fbac0f2b625 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.374+0000 7f7d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:43.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.374+0000 7f7d2aa00640 1 -- 172.21.15.18:0/199830211 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30010ac0 con 0x55e56a7b4360 2024-04-16T17:20:43.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.375+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/199830211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d38004a40 con 0x55e56a7b4360 2024-04-16T17:20:43.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.375+0000 7f7d2aa00640 1 -- 172.21.15.18:0/199830211 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f7d30024070 con 0x55e56a7b4360 2024-04-16T17:20:43.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.375+0000 7f7d2aa00640 1 -- 172.21.15.18:0/199830211 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7d30033050 con 0x55e56a7b4360 2024-04-16T17:20:43.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.375+0000 7f7d4e7b7740 10 client.29846 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:43.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.375+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/199830211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e56a83aee0 con 0x55e56a7b4360 2024-04-16T17:20:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.376+0000 7f7d2aa00640 1 -- 172.21.15.18:0/199830211 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d30016050 con 0x55e56a7b4360 2024-04-16T17:20:43.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.377+0000 7f7d2aa00640 1 -- 172.21.15.18:0/199830211 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7d30005020 con 0x55e56a7b4360 2024-04-16T17:20:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.378+0000 7f7d4e7b7740 10 client.29846 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.378+0000 7f7d4e7b7740 10 client.29846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:20:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.378+0000 7f7d4e7b7740 20 client.29846 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.378+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e56a83b670 0x55e56a83da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.378+0000 7f7d4e7b7740 4 client.29846 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:20:43.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.378+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/199830211 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7d38004a40 con 0x55e56a83b670 2024-04-16T17:20:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.378+0000 7f7d43e00640 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e56a83b670 0x55e56a83da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.379+0000 7f7d43e00640 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e56a83b670 0x55e56a83da90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bfd44203113b7da server_cookie=4bc75a476ad4037f in_seq=0 out_seq=0 2024-04-16T17:20:43.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.379+0000 7f7d2aa00640 10 client.29846 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:20:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.380+0000 7f7d2aa00640 1 -- 172.21.15.18:0/199830211 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7d38004a40 con 0x55e56a83b670 2024-04-16T17:20:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.380+0000 7f7d2aa00640 10 client.29846 handle_command_reply: tid=0 2024-04-16T17:20:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.380+0000 7f7d4e7b7740 1 client.29846 shutdown 2024-04-16T17:20:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.380+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/199830211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7b4360 msgr2=0x55e56a7be020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.380+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7b4360 0x55e56a7be020 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000b7e0 tx=0x7f7d300046c0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.381+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/199830211 shutdown_connections 2024-04-16T17:20:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.381+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55e56a83b670 0x55e56a83da90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.381+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e56a7c4250 0x55e56a7beaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.381+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e56a7bad50 0x55e56a7be560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.381+0000 7f7d4e7b7740 1 --2- 172.21.15.18:0/199830211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e56a7b4360 0x55e56a7be020 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.381+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/199830211 >> 172.21.15.18:0/199830211 conn(0x55e56a7b5480 msgr2=0x55e56a7b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.381+0000 7f7d2aa00640 0 client.29846 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:20:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.382+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/199830211 shutdown_connections 2024-04-16T17:20:43.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.382+0000 7f7d4e7b7740 1 -- 172.21.15.18:0/199830211 wait complete. 2024-04-16T17:20:43.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.385+0000 7f7d4e7b7740 20 client.29846 trim_cache size 0 max 16384 2024-04-16T17:20:43.386 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:20:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 -- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d2c0a3de0 msgr2=0x7f7d2c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d2c0a3de0 0x7f7d2c0a62a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7d30005d00 tx=0x7f7d30005c90 comp rx=0 tx=0).stop 2024-04-16T17:20:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 -- 172.21.15.18:0/2481213497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d4410ee60 msgr2=0x7f7d4410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d4410ee60 0x7f7d4410f2e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00b970 tx=0x7f7d3c00be30 comp rx=0 tx=0).stop 2024-04-16T17:20:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 -- 172.21.15.18:0/2481213497 shutdown_connections 2024-04-16T17:20:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f7d2c0a3de0 0x7f7d2c0a62a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d4410ee60 0x7f7d4410f2e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d44109520 0x7f7d4411c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 --2- 172.21.15.18:0/2481213497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d44104450 0x7f7d44110d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 -- 172.21.15.18:0/2481213497 >> 172.21.15.18:0/2481213497 conn(0x7f7d440ffcb0 msgr2=0x7f7d44101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 -- 172.21.15.18:0/2481213497 shutdown_connections 2024-04-16T17:20:43.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.387+0000 7f7d4ba00640 1 -- 172.21.15.18:0/2481213497 wait complete. 2024-04-16T17:20:43.420 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:20:43.483 INFO:tasks.workunit.client.0.smithi018.stdout: 10 45110 12.79 MB/sec execute 55 sec latency 256.494 ms 2024-04-16T17:20:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.733+0000 7fb640000640 1 -- 172.21.15.18:0/594425049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 msgr2=0x7fb638115260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.733+0000 7fb640000640 1 --2- 172.21.15.18:0/594425049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 0x7fb638115260 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009930 tx=0x7fb62c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:20:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.733+0000 7fb640000640 1 -- 172.21.15.18:0/594425049 shutdown_connections 2024-04-16T17:20:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.733+0000 7fb640000640 1 --2- 172.21.15.18:0/594425049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 0x7fb638115260 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.733+0000 7fb640000640 1 --2- 172.21.15.18:0/594425049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb638076e50 0x7fb638112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.733+0000 7fb640000640 1 --2- 172.21.15.18:0/594425049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb638076530 0x7fb638076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.733+0000 7fb640000640 1 -- 172.21.15.18:0/594425049 >> 172.21.15.18:0/594425049 conn(0x7fb6380ffc80 msgr2=0x7fb6381020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.734+0000 7fb640000640 1 -- 172.21.15.18:0/594425049 shutdown_connections 2024-04-16T17:20:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.734+0000 7fb640000640 1 -- 172.21.15.18:0/594425049 wait complete. 2024-04-16T17:20:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.736+0000 7fb640000640 1 Processor -- start 2024-04-16T17:20:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.736+0000 7fb640000640 1 -- start start 2024-04-16T17:20:43.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.737+0000 7fb640000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb638076530 0x7fb638073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.737+0000 7fb640000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb638076e50 0x7fb63806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.737+0000 7fb640000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 0x7fb63806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.738+0000 7fb640000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381189d0 con 0x7fb638076530 2024-04-16T17:20:43.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.738+0000 7fb640000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb638118850 con 0x7fb638076e50 2024-04-16T17:20:43.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.738+0000 7fb640000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb638118b50 con 0x7fb638112e70 2024-04-16T17:20:43.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.738+0000 7fb63e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 0x7fb63806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.738+0000 7fb63e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 0x7fb63806cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49534/0 (socket says 172.21.15.18:49534) 2024-04-16T17:20:43.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.738+0000 7fb63e000640 1 -- 172.21.15.18:0/2344985369 learned_addr learned my addr 172.21.15.18:0/2344985369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.740+0000 7fb63cc00640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb638076e50 0x7fb63806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.740+0000 7fb63e000640 1 -- 172.21.15.18:0/2344985369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb638076e50 msgr2=0x7fb63806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.740+0000 7fb63e000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb638076e50 0x7fb63806c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.740+0000 7fb63e000640 1 -- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb638076530 msgr2=0x7fb638073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.740+0000 7fb63e000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb638076530 0x7fb638073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.740+0000 7fb63e000640 1 -- 172.21.15.18:0/2344985369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63806d3e0 con 0x7fb638112e70 2024-04-16T17:20:43.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.742+0000 7fb63e000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 0x7fb63806cdf0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb62c02c1d0 tx=0x7fb62c02ea20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.743+0000 7fb636000640 1 -- 172.21.15.18:0/2344985369 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c041020 con 0x7fb638112e70 2024-04-16T17:20:43.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.743+0000 7fb640000640 1 -- 172.21.15.18:0/2344985369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63806d670 con 0x7fb638112e70 2024-04-16T17:20:43.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.743+0000 7fb640000640 1 -- 172.21.15.18:0/2344985369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6380715f0 con 0x7fb638112e70 2024-04-16T17:20:43.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.744+0000 7fb636000640 1 -- 172.21.15.18:0/2344985369 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62c02eda0 con 0x7fb638112e70 2024-04-16T17:20:43.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.745+0000 7fb636000640 1 -- 172.21.15.18:0/2344985369 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c004ef0 con 0x7fb638112e70 2024-04-16T17:20:43.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.749+0000 7fb642def740 1 Processor -- start 2024-04-16T17:20:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.752+0000 7fb636000640 1 -- 172.21.15.18:0/2344985369 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb62c044020 con 0x7fb638112e70 2024-04-16T17:20:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.754+0000 7fb642def740 1 -- start start 2024-04-16T17:20:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.755+0000 7fb642def740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b512a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.755+0000 7fb642def740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5132d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.755+0000 7fb642def740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 0x55e5b5135640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.755+0000 7fb642def740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e5b508f020 con 0x55e5b5129d50 2024-04-16T17:20:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.755+0000 7fb642def740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5b5035460 con 0x55e5b512a670 2024-04-16T17:20:43.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.755+0000 7fb642def740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e5b508afe0 con 0x55e5b5133250 2024-04-16T17:20:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.755+0000 7fb636000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6200a3ea0 0x7fb6200a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.756+0000 7fb636000640 1 -- 172.21.15.18:0/2344985369 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7fb62c0f20e0 con 0x7fb638112e70 2024-04-16T17:20:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.756+0000 7fb63d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5132d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.756+0000 7fb63d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5132d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51226/0 (socket says 172.21.15.18:51226) 2024-04-16T17:20:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.756+0000 7fb63d600640 1 -- 172.21.15.18:0/3239845430 learned_addr learned my addr 172.21.15.18:0/3239845430 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.757+0000 7fb63cc00640 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 0x55e5b5135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.757+0000 7fb63cc00640 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b512a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.757+0000 7fb63e000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6200a3ea0 0x7fb6200a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.757+0000 7fb63d600640 1 -- 172.21.15.18:0/3239845430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 msgr2=0x55e5b5135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.757+0000 7fb63d600640 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 0x55e5b5135640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.757+0000 7fb63d600640 1 -- 172.21.15.18:0/3239845430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 msgr2=0x55e5b512a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.757+0000 7fb63d600640 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b512a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.757+0000 7fb63d600640 1 -- 172.21.15.18:0/3239845430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5b503d040 con 0x55e5b512a670 2024-04-16T17:20:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.758+0000 7fb63d600640 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5132d10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb630007d80 tx=0x7fb63000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6502870efdd96731 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.758+0000 7fb63e000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6200a3ea0 0x7fb6200a6360 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0ef610 tx=0x7fb62c005bb0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.759+0000 7fb62a600640 1 -- 172.21.15.18:0/3239845430 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630014070 con 0x55e5b512a670 2024-04-16T17:20:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.759+0000 7fb62a600640 1 -- 172.21.15.18:0/3239845430 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63000fb30 con 0x55e5b512a670 2024-04-16T17:20:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.759+0000 7fb62a600640 1 -- 172.21.15.18:0/3239845430 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63000eba0 con 0x55e5b512a670 2024-04-16T17:20:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.760+0000 7fb642def740 1 -- 172.21.15.18:0/3239845430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 msgr2=0x55e5b5132d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.760+0000 7fb642def740 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5132d10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb630007d80 tx=0x7fb63000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.760+0000 7fb642def740 1 -- 172.21.15.18:0/3239845430 shutdown_connections 2024-04-16T17:20:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.760+0000 7fb642def740 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 0x55e5b5135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.760+0000 7fb642def740 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5132d10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.760+0000 7fb642def740 1 --2- 172.21.15.18:0/3239845430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b512a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.760+0000 7fb642def740 1 -- 172.21.15.18:0/3239845430 >> 172.21.15.18:0/3239845430 conn(0x55e5b5124480 msgr2=0x55e5b5124860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.762+0000 7fb642def740 1 -- 172.21.15.18:0/3239845430 shutdown_connections 2024-04-16T17:20:43.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.764+0000 7fb642def740 1 -- 172.21.15.18:0/3239845430 wait complete. 2024-04-16T17:20:43.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.765+0000 7fb642def740 1 -- start start 2024-04-16T17:20:43.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.765+0000 7fb642def740 1 Processor -- start 2024-04-16T17:20:43.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.767+0000 7fb642def740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b51301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.767+0000 7fb642def740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5130720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.767+0000 7fb642def740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 0x55e5b5130c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.767+0000 7fb642def740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e5b5136040 con 0x55e5b5129d50 2024-04-16T17:20:43.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.767+0000 7fb642def740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5b5048830 con 0x55e5b512a670 2024-04-16T17:20:43.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.767+0000 7fb642def740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e5b51361c0 con 0x55e5b5133250 2024-04-16T17:20:43.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.767+0000 7fb642def740 10 client.0 _ll_register_callbacks cb 0x55e5b5050150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:43.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.767+0000 7fb63d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5130720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5130720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51230/0 (socket says 172.21.15.18:51230) 2024-04-16T17:20:43.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63d600640 1 -- 172.21.15.18:0/1948993486 learned_addr learned my addr 172.21.15.18:0/1948993486 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63cc00640 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b51301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63d600640 1 -- 172.21.15.18:0/1948993486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 msgr2=0x55e5b5130c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:20:43.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63d600640 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 0x55e5b5130c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63d600640 1 -- 172.21.15.18:0/1948993486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 msgr2=0x55e5b51301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63d600640 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b51301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63d600640 1 -- 172.21.15.18:0/1948993486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5b504c920 con 0x55e5b512a670 2024-04-16T17:20:43.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.768+0000 7fb63cc00640 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b51301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:43.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.769+0000 7fb63d600640 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5130720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb63000cba0 tx=0x7fb630007aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c77b91fee5710dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.769+0000 7fb61ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:43.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.769+0000 7fb61ea00640 1 -- 172.21.15.18:0/1948993486 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630014040 con 0x55e5b512a670 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.769+0000 7fb642def740 1 -- 172.21.15.18:0/1948993486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63000fb30 con 0x55e5b512a670 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.769+0000 7fb61ea00640 1 -- 172.21.15.18:0/1948993486 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7fb63000f860 con 0x55e5b512a670 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.770+0000 7fb61ea00640 1 -- 172.21.15.18:0/1948993486 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb63002f050 con 0x55e5b512a670 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.770+0000 7fb642def740 10 client.29855 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.770+0000 7fb642def740 1 -- 172.21.15.18:0/1948993486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e5b501a7f0 con 0x55e5b512a670 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.770+0000 7fb61ea00640 1 -- 172.21.15.18:0/1948993486 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63000fb30 con 0x55e5b512a670 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.770+0000 7fb61ea00640 1 -- 172.21.15.18:0/1948993486 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb630019980 con 0x55e5b512a670 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb642def740 10 client.29855 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb642def740 10 client.29855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb642def740 20 client.29855 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb642def740 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e5b51439d0 0x55e5b5145e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb642def740 4 client.29855 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:43.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb642def740 1 -- 172.21.15.18:0/1948993486 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5b509ca40 con 0x55e5b51439d0 2024-04-16T17:20:43.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb63cc00640 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e5b51439d0 0x55e5b5145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb63cc00640 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e5b51439d0 0x55e5b5145e50 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e187f519c590f server_cookie=c983feaacb7c3bfa in_seq=0 out_seq=0 2024-04-16T17:20:43.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.771+0000 7fb61ea00640 10 client.29855 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:43.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb61ea00640 1 -- 172.21.15.18:0/1948993486 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5b509ca40 con 0x55e5b51439d0 2024-04-16T17:20:43.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb61ea00640 10 client.29855 handle_command_reply: tid=0 2024-04-16T17:20:43.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 client.29855 shutdown 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 -- 172.21.15.18:0/1948993486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 msgr2=0x55e5b5130720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5130720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb63000cba0 tx=0x7fb630007aa0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 -- 172.21.15.18:0/1948993486 shutdown_connections 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e5b51439d0 0x55e5b5145e50 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5133250 0x55e5b5130c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5130720 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 --2- 172.21.15.18:0/1948993486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5129d50 0x55e5b51301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.774+0000 7fb642def740 1 -- 172.21.15.18:0/1948993486 >> 172.21.15.18:0/1948993486 conn(0x55e5b5124480 msgr2=0x55e5b51261d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.775+0000 7fb61ea00640 0 client.29855 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.775+0000 7fb642def740 1 -- 172.21.15.18:0/1948993486 shutdown_connections 2024-04-16T17:20:43.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.775+0000 7fb642def740 1 -- 172.21.15.18:0/1948993486 wait complete. 2024-04-16T17:20:43.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.775+0000 7fb642def740 20 client.29855 trim_cache size 0 max 16384 2024-04-16T17:20:43.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.809+0000 7fb642def740 1 Processor -- start 2024-04-16T17:20:43.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.809+0000 7fb642def740 1 -- start start 2024-04-16T17:20:43.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.809+0000 7fb642def740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 0x55e5b5132910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.809+0000 7fb642def740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b512a670 0x55e5b513ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.809+0000 7fb642def740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b5133250 0x55e5b514b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.809+0000 7fb642def740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6180128c0 con 0x55e5b512a670 2024-04-16T17:20:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.809+0000 7fb642def740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb618010190 con 0x55e5b5133250 2024-04-16T17:20:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.809+0000 7fb642def740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb61800e000 con 0x55e5b5129d50 2024-04-16T17:20:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.810+0000 7fb63d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b512a670 0x55e5b513ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.810+0000 7fb63d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b512a670 0x55e5b513ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36706/0 (socket says 172.21.15.18:36706) 2024-04-16T17:20:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.810+0000 7fb63d600640 1 -- 172.21.15.18:0/885342209 learned_addr learned my addr 172.21.15.18:0/885342209 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.810+0000 7fb63cc00640 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 0x55e5b5132910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.810+0000 7fb63cc00640 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b5133250 0x55e5b514b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.811+0000 7fb63d600640 1 -- 172.21.15.18:0/885342209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 msgr2=0x55e5b5132910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.811+0000 7fb63d600640 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 0x55e5b5132910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.811+0000 7fb63d600640 1 -- 172.21.15.18:0/885342209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b5133250 msgr2=0x55e5b514b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.811+0000 7fb63d600640 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b5133250 0x55e5b514b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.811+0000 7fb63d600640 1 -- 172.21.15.18:0/885342209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5b4da13f0 con 0x55e5b512a670 2024-04-16T17:20:43.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.812+0000 7fb63d600640 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b512a670 0x55e5b513ef90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb630007b10 tx=0x7fb630004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bcfcb735f3fbed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.812+0000 7fb629c00640 1 -- 172.21.15.18:0/885342209 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630014040 con 0x55e5b512a670 2024-04-16T17:20:43.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.812+0000 7fb629c00640 1 -- 172.21.15.18:0/885342209 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63003a070 con 0x55e5b512a670 2024-04-16T17:20:43.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.812+0000 7fb629c00640 1 -- 172.21.15.18:0/885342209 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630014040 con 0x55e5b512a670 2024-04-16T17:20:43.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.813+0000 7fb642def740 1 -- 172.21.15.18:0/885342209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b512a670 msgr2=0x55e5b513ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.813+0000 7fb642def740 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b512a670 0x55e5b513ef90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb630007b10 tx=0x7fb630004a20 comp rx=0 tx=0).stop 2024-04-16T17:20:43.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.814+0000 7fb642def740 1 -- 172.21.15.18:0/885342209 shutdown_connections 2024-04-16T17:20:43.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.815+0000 7fb642def740 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b5133250 0x55e5b514b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.815+0000 7fb642def740 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b512a670 0x55e5b513ef90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.815+0000 7fb642def740 1 --2- 172.21.15.18:0/885342209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 0x55e5b5132910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.815+0000 7fb642def740 1 -- 172.21.15.18:0/885342209 >> 172.21.15.18:0/885342209 conn(0x55e5b5124480 msgr2=0x55e5b512b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.816+0000 7fb642def740 1 -- 172.21.15.18:0/885342209 shutdown_connections 2024-04-16T17:20:43.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.817+0000 7fb642def740 1 -- 172.21.15.18:0/885342209 wait complete. 2024-04-16T17:20:43.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.818+0000 7fb642def740 1 -- start start 2024-04-16T17:20:43.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.818+0000 7fb642def740 1 Processor -- start 2024-04-16T17:20:43.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb642def740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 0x55e5b5136650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb642def740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5136b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb642def740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5133250 0x55e5b51a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb642def740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb61800e650 con 0x55e5b5133250 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb642def740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb61800fac0 con 0x55e5b512a670 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb642def740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb61800ed20 con 0x55e5b5129d50 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb642def740 10 client.0 _ll_register_callbacks cb 0x7fb624009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 0x55e5b5136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5136b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51242/0 (socket says 172.21.15.18:51242) 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63d600640 1 -- 172.21.15.18:0/1707400247 learned_addr learned my addr 172.21.15.18:0/1707400247 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63cc00640 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5133250 0x55e5b51a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63d600640 1 -- 172.21.15.18:0/1707400247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 msgr2=0x55e5b5136650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63d600640 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 0x55e5b5136650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63d600640 1 -- 172.21.15.18:0/1707400247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5133250 msgr2=0x55e5b51a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63d600640 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5133250 0x55e5b51a7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.819+0000 7fb63d600640 1 -- 172.21.15.18:0/1707400247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5b4e6b7d0 con 0x55e5b512a670 2024-04-16T17:20:43.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.820+0000 7fb63d600640 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5136b90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb630037040 tx=0x7fb63000f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8195d704a259c544 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:43.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.820+0000 7fb61ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:43.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.820+0000 7fb61ea00640 1 -- 172.21.15.18:0/1707400247 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630042030 con 0x55e5b512a670 2024-04-16T17:20:43.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.820+0000 7fb61ea00640 1 -- 172.21.15.18:0/1707400247 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7fb630004b20 con 0x55e5b512a670 2024-04-16T17:20:43.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.821+0000 7fb642def740 1 -- 172.21.15.18:0/1707400247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63003a070 con 0x55e5b512a670 2024-04-16T17:20:43.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.823+0000 7fb61ea00640 1 -- 172.21.15.18:0/1707400247 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb630014040 con 0x55e5b512a670 2024-04-16T17:20:43.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.823+0000 7fb642def740 10 client.29861 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:43.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.823+0000 7fb642def740 1 -- 172.21.15.18:0/1707400247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e5b51a95a0 con 0x55e5b512a670 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.823+0000 7fb61ea00640 1 -- 172.21.15.18:0/1707400247 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63003a070 con 0x55e5b512a670 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.824+0000 7fb61ea00640 1 -- 172.21.15.18:0/1707400247 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb630019450 con 0x55e5b512a670 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.824+0000 7fb642def740 10 client.29861 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.824+0000 7fb642def740 10 client.29861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.824+0000 7fb642def740 20 client.29861 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.824+0000 7fb642def740 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e5b51a9c50 0x55e5b51ac070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.824+0000 7fb642def740 4 client.29861 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.824+0000 7fb642def740 1 -- 172.21.15.18:0/1707400247 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5b50968b0 con 0x55e5b51a9c50 2024-04-16T17:20:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.824+0000 7fb63cc00640 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e5b51a9c50 0x55e5b51ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:43.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.825+0000 7fb63cc00640 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e5b51a9c50 0x55e5b51ac070 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9136f8a41b222be7 server_cookie=77b9769a034ed991 in_seq=0 out_seq=0 2024-04-16T17:20:43.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.825+0000 7fb61ea00640 10 client.29861 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:43.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.829+0000 7fb61ea00640 1 -- 172.21.15.18:0/1707400247 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5b50968b0 con 0x55e5b51a9c50 2024-04-16T17:20:43.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.829+0000 7fb61ea00640 10 client.29861 handle_command_reply: tid=0 2024-04-16T17:20:43.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.829+0000 7fb642def740 1 client.29861 shutdown 2024-04-16T17:20:43.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb642def740 1 -- 172.21.15.18:0/1707400247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 msgr2=0x55e5b5136b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb642def740 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5136b90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb630037040 tx=0x7fb63000f890 comp rx=0 tx=0).stop 2024-04-16T17:20:43.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb642def740 1 -- 172.21.15.18:0/1707400247 shutdown_connections 2024-04-16T17:20:43.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb642def740 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e5b51a9c50 0x55e5b51ac070 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb642def740 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e5b5133250 0x55e5b51a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb642def740 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5b512a670 0x55e5b5136b90 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb642def740 1 --2- 172.21.15.18:0/1707400247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5b5129d50 0x55e5b5136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb61ea00640 0 client.29861 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:43.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.830+0000 7fb642def740 1 -- 172.21.15.18:0/1707400247 >> 172.21.15.18:0/1707400247 conn(0x55e5b5124480 msgr2=0x55e5b5130180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.831+0000 7fb642def740 1 -- 172.21.15.18:0/1707400247 shutdown_connections 2024-04-16T17:20:43.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.831+0000 7fb642def740 1 -- 172.21.15.18:0/1707400247 wait complete. 2024-04-16T17:20:43.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.831+0000 7fb642def740 20 client.29861 trim_cache size 0 max 16384 2024-04-16T17:20:43.832 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:20:43.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 -- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6200a3ea0 msgr2=0x7fb6200a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6200a3ea0 0x7fb6200a6360 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0ef610 tx=0x7fb62c005bb0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 -- 172.21.15.18:0/2344985369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 msgr2=0x7fb63806cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:43.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 0x7fb63806cdf0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb62c02c1d0 tx=0x7fb62c02ea20 comp rx=0 tx=0).stop 2024-04-16T17:20:43.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 -- 172.21.15.18:0/2344985369 shutdown_connections 2024-04-16T17:20:43.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6200a3ea0 0x7fb6200a6360 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb638112e70 0x7fb63806cdf0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb638076e50 0x7fb63806c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 --2- 172.21.15.18:0/2344985369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb638076530 0x7fb638073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:43.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.832+0000 7fb640000640 1 -- 172.21.15.18:0/2344985369 >> 172.21.15.18:0/2344985369 conn(0x7fb6380ffc80 msgr2=0x7fb638114480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:43.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.833+0000 7fb640000640 1 -- 172.21.15.18:0/2344985369 shutdown_connections 2024-04-16T17:20:43.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:43.833+0000 7fb640000640 1 -- 172.21.15.18:0/2344985369 wait complete. 2024-04-16T17:20:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:43 smithi022 ceph-mon[36023]: pgmap v509: 129 pgs: 129 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-04-16T17:20:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:43 smithi043 ceph-mon[36901]: pgmap v509: 129 pgs: 129 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-04-16T17:20:44.483 INFO:tasks.workunit.client.0.smithi018.stdout: 10 45570 13.04 MB/sec execute 56 sec latency 92.908 ms 2024-04-16T17:20:44.867 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:20:44.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:20:44.868 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:20:45.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.195+0000 7f5755800640 1 -- 172.21.15.18:0/3226840530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103270 msgr2=0x7f5750103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.195+0000 7f5755800640 1 --2- 172.21.15.18:0/3226840530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103270 0x7f5750103650 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5740009900 tx=0x7f574002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.195+0000 7f5755800640 1 -- 172.21.15.18:0/3226840530 shutdown_connections 2024-04-16T17:20:45.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.195+0000 7f5755800640 1 --2- 172.21.15.18:0/3226840530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f575010c5b0 0x7f575010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.195+0000 7f5755800640 1 --2- 172.21.15.18:0/3226840530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5750103b90 0x7f575010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.195+0000 7f5755800640 1 --2- 172.21.15.18:0/3226840530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103270 0x7f5750103650 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.195+0000 7f5755800640 1 -- 172.21.15.18:0/3226840530 >> 172.21.15.18:0/3226840530 conn(0x7f57500780b0 msgr2=0x7f57500784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.196+0000 7f5755800640 1 -- 172.21.15.18:0/3226840530 shutdown_connections 2024-04-16T17:20:45.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.196+0000 7f5755800640 1 -- 172.21.15.18:0/3226840530 wait complete. 2024-04-16T17:20:45.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.198+0000 7f5755800640 1 Processor -- start 2024-04-16T17:20:45.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.198+0000 7f5755800640 1 -- start start 2024-04-16T17:20:45.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.199+0000 7f5755800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5750103270 0x7f57500737c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f5755800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103b90 0x7f575006c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f5755800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575010c5b0 0x7f575006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f574ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5750103270 0x7f57500737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f574f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575010c5b0 0x7f575006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f574e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103b90 0x7f575006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f574ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5750103270 0x7f57500737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36728/0 (socket says 172.21.15.18:36728) 2024-04-16T17:20:45.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f574f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575010c5b0 0x7f575006cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49600/0 (socket says 172.21.15.18:49600) 2024-04-16T17:20:45.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f574e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103b90 0x7f575006c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51272/0 (socket says 172.21.15.18:51272) 2024-04-16T17:20:45.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.200+0000 7f574ea00640 1 -- 172.21.15.18:0/2835151926 learned_addr learned my addr 172.21.15.18:0/2835151926 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57501146d0 con 0x7f5750103270 2024-04-16T17:20:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5750114550 con 0x7f5750103b90 2024-04-16T17:20:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5750114850 con 0x7f575010c5b0 2024-04-16T17:20:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f574f400640 1 -- 172.21.15.18:0/2835151926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103b90 msgr2=0x7f575006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f574f400640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103b90 0x7f575006c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f574f400640 1 -- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5750103270 msgr2=0x7f57500737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f574f400640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5750103270 0x7f57500737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f574f400640 1 -- 172.21.15.18:0/2835151926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575006d340 con 0x7f575010c5b0 2024-04-16T17:20:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.201+0000 7f574e000640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103b90 0x7f575006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:45.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.202+0000 7f574f400640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575010c5b0 0x7f575006cdd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f574800be20 tx=0x7f574800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.202+0000 7f5747400640 1 -- 172.21.15.18:0/2835151926 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748019070 con 0x7f575010c5b0 2024-04-16T17:20:45.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.202+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575006d630 con 0x7f575010c5b0 2024-04-16T17:20:45.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.202+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57500715d0 con 0x7f575010c5b0 2024-04-16T17:20:45.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.203+0000 7f5747400640 1 -- 172.21.15.18:0/2835151926 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f574800cd10 con 0x7f575010c5b0 2024-04-16T17:20:45.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.203+0000 7f5747400640 1 -- 172.21.15.18:0/2835151926 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748019070 con 0x7f575010c5b0 2024-04-16T17:20:45.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.208+0000 7f5747400640 1 -- 172.21.15.18:0/2835151926 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f574801c070 con 0x7f575010c5b0 2024-04-16T17:20:45.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.211+0000 7f5747400640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57340a3fd0 0x7f57340a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.211+0000 7f5747400640 1 -- 172.21.15.18:0/2835151926 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f57480cfe20 con 0x7f575010c5b0 2024-04-16T17:20:45.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.212+0000 7f574ea00640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57340a3fd0 0x7f57340a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.212+0000 7f575843e740 1 Processor -- start 2024-04-16T17:20:45.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.212+0000 7f575843e740 1 -- start start 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f575843e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f78ca0 0x560b81f79080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f575843e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f81c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f575843e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f821a0 0x560b81f84590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f575843e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560b81eddcb0 con 0x560b81f78ca0 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f575843e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560b81e84370 con 0x560b81f795c0 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f575843e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560b81ed9c70 con 0x560b81f821a0 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f574e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f81c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f574e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f81c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51286/0 (socket says 172.21.15.18:51286) 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.213+0000 7f574e000640 1 -- 172.21.15.18:0/1884780231 learned_addr learned my addr 172.21.15.18:0/1884780231 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.214+0000 7f574f400640 1 --2- 172.21.15.18:0/1884780231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f821a0 0x560b81f84590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.214+0000 7f574e000640 1 -- 172.21.15.18:0/1884780231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f821a0 msgr2=0x560b81f84590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.214+0000 7f574e000640 1 --2- 172.21.15.18:0/1884780231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f821a0 0x560b81f84590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.214+0000 7f574e000640 1 -- 172.21.15.18:0/1884780231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f78ca0 msgr2=0x560b81f79080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.214+0000 7f574e000640 1 --2- 172.21.15.18:0/1884780231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f78ca0 0x560b81f79080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.214+0000 7f574e000640 1 -- 172.21.15.18:0/1884780231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b81ea3910 con 0x560b81f795c0 2024-04-16T17:20:45.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.214+0000 7f574e000640 1 --2- 172.21.15.18:0/1884780231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f81c60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f573c009de0 tx=0x7f573c00e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ee66c42d815b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.215+0000 7f573be00640 1 -- 172.21.15.18:0/1884780231 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c017070 con 0x560b81f795c0 2024-04-16T17:20:45.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.215+0000 7f573be00640 1 -- 172.21.15.18:0/1884780231 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573c013950 con 0x560b81f795c0 2024-04-16T17:20:45.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.215+0000 7f573be00640 1 -- 172.21.15.18:0/1884780231 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c0129f0 con 0x560b81f795c0 2024-04-16T17:20:45.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.215+0000 7f574ea00640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57340a3fd0 0x7f57340a6490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5740006010 tx=0x7f5740004040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 -- 172.21.15.18:0/1884780231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 msgr2=0x560b81f81c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 --2- 172.21.15.18:0/1884780231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f81c60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f573c009de0 tx=0x7f573c00e5a0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 -- 172.21.15.18:0/1884780231 shutdown_connections 2024-04-16T17:20:45.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 --2- 172.21.15.18:0/1884780231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f821a0 0x560b81f84590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 --2- 172.21.15.18:0/1884780231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f81c60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 --2- 172.21.15.18:0/1884780231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f78ca0 0x560b81f79080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 -- 172.21.15.18:0/1884780231 >> 172.21.15.18:0/1884780231 conn(0x560b81f733d0 msgr2=0x560b81f737b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 -- 172.21.15.18:0/1884780231 shutdown_connections 2024-04-16T17:20:45.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.218+0000 7f575843e740 1 -- 172.21.15.18:0/1884780231 wait complete. 2024-04-16T17:20:45.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.219+0000 7f575843e740 1 -- start start 2024-04-16T17:20:45.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.219+0000 7f575843e740 1 Processor -- start 2024-04-16T17:20:45.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.220+0000 7f575843e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f7f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.220+0000 7f575843e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f7f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.220+0000 7f575843e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 0x560b81f7fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.220+0000 7f575843e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560b81f84f90 con 0x560b81f821a0 2024-04-16T17:20:45.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.220+0000 7f575843e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560b81e97730 con 0x560b81f795c0 2024-04-16T17:20:45.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.220+0000 7f575843e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560b81f85110 con 0x560b81f78ca0 2024-04-16T17:20:45.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.220+0000 7f575843e740 10 client.0 _ll_register_callbacks cb 0x560b81e661e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:45.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.220+0000 7f574e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f7f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f7f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51290/0 (socket says 172.21.15.18:51290) 2024-04-16T17:20:45.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574e000640 1 -- 172.21.15.18:0/2577782486 learned_addr learned my addr 172.21.15.18:0/2577782486 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574f400640 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 0x560b81f7fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574ea00640 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f7f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574f400640 1 -- 172.21.15.18:0/2577782486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 msgr2=0x560b81f7f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574f400640 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f7f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574f400640 1 -- 172.21.15.18:0/2577782486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 msgr2=0x560b81f7f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574f400640 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f7f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.221+0000 7f574f400640 1 -- 172.21.15.18:0/2577782486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b81cb87e0 con 0x560b81f821a0 2024-04-16T17:20:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.222+0000 7f574f400640 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 0x560b81f7fbe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f57480cd240 tx=0x7f57480bc110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3637f2f4cd8054bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.222+0000 7f5738c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:20:45.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.222+0000 7f5738c00640 1 -- 172.21.15.18:0/2577782486 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57480a43d0 con 0x560b81f821a0 2024-04-16T17:20:45.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.223+0000 7f5738c00640 1 -- 172.21.15.18:0/2577782486 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f57480babb0 con 0x560b81f821a0 2024-04-16T17:20:45.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.223+0000 7f575843e740 1 -- 172.21.15.18:0/2577782486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573c013950 con 0x560b81f821a0 2024-04-16T17:20:45.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.223+0000 7f5738c00640 1 -- 172.21.15.18:0/2577782486 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f57480b6030 con 0x560b81f821a0 2024-04-16T17:20:45.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.223+0000 7f575843e740 10 client.19302 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:45.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.223+0000 7f575843e740 1 -- 172.21.15.18:0/2577782486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560b81d18f70 con 0x560b81f821a0 2024-04-16T17:20:45.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.224+0000 7f5738c00640 1 -- 172.21.15.18:0/2577782486 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573c013950 con 0x560b81f821a0 2024-04-16T17:20:45.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.224+0000 7f5738c00640 1 -- 172.21.15.18:0/2577782486 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f574809af10 con 0x560b81f821a0 2024-04-16T17:20:45.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.224+0000 7f575843e740 10 client.19302 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.224+0000 7f575843e740 10 client.19302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.225+0000 7f575843e740 20 client.19302 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.225+0000 7f575843e740 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b81f9abc0 0x560b81f9d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.225+0000 7f575843e740 4 client.19302 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.225+0000 7f575843e740 1 -- 172.21.15.18:0/2577782486 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b81eeb6d0 con 0x560b81f9abc0 2024-04-16T17:20:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.225+0000 7f574ea00640 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b81f9abc0 0x560b81f9d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.228+0000 7f574ea00640 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b81f9abc0 0x560b81f9d020 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f85c09ece3bcf6b server_cookie=a5b5a29bd8b94a9d in_seq=0 out_seq=0 2024-04-16T17:20:45.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.228+0000 7f5738c00640 10 client.19302 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:45.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.229+0000 7f5738c00640 1 -- 172.21.15.18:0/2577782486 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b81eeb6d0 con 0x560b81f9abc0 2024-04-16T17:20:45.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.229+0000 7f5738c00640 10 client.19302 handle_command_reply: tid=0 2024-04-16T17:20:45.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.229+0000 7f575843e740 1 client.19302 shutdown 2024-04-16T17:20:45.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.229+0000 7f575843e740 1 -- 172.21.15.18:0/2577782486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 msgr2=0x560b81f7fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.230+0000 7f575843e740 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 0x560b81f7fbe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f57480cd240 tx=0x7f57480bc110 comp rx=0 tx=0).stop 2024-04-16T17:20:45.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.230+0000 7f575843e740 1 -- 172.21.15.18:0/2577782486 shutdown_connections 2024-04-16T17:20:45.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.230+0000 7f575843e740 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b81f9abc0 0x560b81f9d020 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.230+0000 7f5738c00640 0 client.19302 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:45.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.230+0000 7f575843e740 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 0x560b81f7fbe0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.230+0000 7f575843e740 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f795c0 0x560b81f7f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.230+0000 7f575843e740 1 --2- 172.21.15.18:0/2577782486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f7f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.230+0000 7f575843e740 1 -- 172.21.15.18:0/2577782486 >> 172.21.15.18:0/2577782486 conn(0x560b81f733d0 msgr2=0x560b81f75120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.231+0000 7f575843e740 1 -- 172.21.15.18:0/2577782486 shutdown_connections 2024-04-16T17:20:45.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.231+0000 7f575843e740 1 -- 172.21.15.18:0/2577782486 wait complete. 2024-04-16T17:20:45.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.231+0000 7f575843e740 20 client.19302 trim_cache size 0 max 16384 2024-04-16T17:20:45.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.258+0000 7f575843e740 1 Processor -- start 2024-04-16T17:20:45.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.258+0000 7f575843e740 1 -- start start 2024-04-16T17:20:45.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f575843e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f78ca0 0x560b81f81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f575843e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f795c0 0x560b81f96740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f575843e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 0x560b81fa28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f575843e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5730012870 con 0x560b81f821a0 2024-04-16T17:20:45.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f575843e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5730010140 con 0x560b81f78ca0 2024-04-16T17:20:45.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f575843e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f573000dfb0 con 0x560b81f795c0 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f574ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f78ca0 0x560b81f81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f574e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f795c0 0x560b81f96740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f574e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f795c0 0x560b81f96740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49632/0 (socket says 172.21.15.18:49632) 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f574ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f78ca0 0x560b81f81890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51300/0 (socket says 172.21.15.18:51300) 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.259+0000 7f574ea00640 1 -- 172.21.15.18:0/2897195526 learned_addr learned my addr 172.21.15.18:0/2897195526 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.260+0000 7f574e000640 1 -- 172.21.15.18:0/2897195526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f78ca0 msgr2=0x560b81f81890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.260+0000 7f574e000640 1 --2- 172.21.15.18:0/2897195526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f78ca0 0x560b81f81890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.260+0000 7f574e000640 1 -- 172.21.15.18:0/2897195526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 msgr2=0x560b81fa28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.260+0000 7f574e000640 1 --2- 172.21.15.18:0/2897195526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 0x560b81fa28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.260+0000 7f574e000640 1 -- 172.21.15.18:0/2897195526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b81bb0170 con 0x560b81f795c0 2024-04-16T17:20:45.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.260+0000 7f574e000640 1 --2- 172.21.15.18:0/2897195526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f795c0 0x560b81f96740 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f573c009de0 tx=0x7f573c0125f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d024f8d7160b666 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.261+0000 7f573b400640 1 -- 172.21.15.18:0/2897195526 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c017040 con 0x560b81f795c0 2024-04-16T17:20:45.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.261+0000 7f573b400640 1 -- 172.21.15.18:0/2897195526 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573c013470 con 0x560b81f795c0 2024-04-16T17:20:45.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.261+0000 7f573b400640 1 -- 172.21.15.18:0/2897195526 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c0256d0 con 0x560b81f795c0 2024-04-16T17:20:45.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 -- 172.21.15.18:0/2897195526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f795c0 msgr2=0x560b81f96740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 --2- 172.21.15.18:0/2897195526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f795c0 0x560b81f96740 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f573c009de0 tx=0x7f573c0125f0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 -- 172.21.15.18:0/2897195526 shutdown_connections 2024-04-16T17:20:45.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 --2- 172.21.15.18:0/2897195526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f821a0 0x560b81fa28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 --2- 172.21.15.18:0/2897195526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f795c0 0x560b81f96740 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 --2- 172.21.15.18:0/2897195526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f78ca0 0x560b81f81890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 -- 172.21.15.18:0/2897195526 >> 172.21.15.18:0/2897195526 conn(0x560b81f733d0 msgr2=0x560b81f7e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 -- 172.21.15.18:0/2897195526 shutdown_connections 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.262+0000 7f575843e740 1 -- 172.21.15.18:0/2897195526 wait complete. 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.263+0000 7f575843e740 1 -- start start 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.263+0000 7f575843e740 1 Processor -- start 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.263+0000 7f575843e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f92e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.263+0000 7f575843e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f795c0 0x560b81f8fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.263+0000 7f575843e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f821a0 0x560b81f8d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.263+0000 7f575843e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f573000e600 con 0x560b81f795c0 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.263+0000 7f575843e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f573000fa70 con 0x560b81f821a0 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.263+0000 7f575843e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f573000ecd0 con 0x560b81f78ca0 2024-04-16T17:20:45.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f575843e740 10 client.0 _ll_register_callbacks cb 0x7f574809b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f92e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f92e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49648/0 (socket says 172.21.15.18:49648) 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574ea00640 1 -- 172.21.15.18:0/2902553573 learned_addr learned my addr 172.21.15.18:0/2902553573 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574f400640 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f821a0 0x560b81f8d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574f400640 1 -- 172.21.15.18:0/2902553573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 msgr2=0x560b81f92e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574f400640 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f92e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574f400640 1 -- 172.21.15.18:0/2902553573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f795c0 msgr2=0x560b81f8fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574f400640 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f795c0 0x560b81f8fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.264+0000 7f574f400640 1 -- 172.21.15.18:0/2902553573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b81f93460 con 0x560b81f821a0 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f574f400640 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f821a0 0x560b81f8d450 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5748099ca0 tx=0x7f574809b070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ea51a2b5836d27 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f5738c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f5738c00640 1 -- 172.21.15.18:0/2902553573 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574807f060 con 0x560b81f821a0 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f5738c00640 1 -- 172.21.15.18:0/2902553573 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f57480a43e0 con 0x560b81f821a0 2024-04-16T17:20:45.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f575843e740 1 -- 172.21.15.18:0/2902553573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573c013470 con 0x560b81f821a0 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f5738c00640 1 -- 172.21.15.18:0/2902553573 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5748097c90 con 0x560b81f821a0 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f575843e740 10 client.29879 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f575843e740 1 -- 172.21.15.18:0/2902553573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560b81ff93e0 con 0x560b81f821a0 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.265+0000 7f5738c00640 1 -- 172.21.15.18:0/2902553573 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57480a4660 con 0x560b81f821a0 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.266+0000 7f5738c00640 1 -- 172.21.15.18:0/2902553573 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f57480860e0 con 0x560b81f821a0 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.266+0000 7f575843e740 10 client.29879 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.266+0000 7f575843e740 10 client.29879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.266+0000 7f575843e740 20 client.29879 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.266+0000 7f575843e740 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b81f9abc0 0x560b81ffbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.266+0000 7f575843e740 4 client.29879 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:20:45.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.266+0000 7f575843e740 1 -- 172.21.15.18:0/2902553573 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f573c013470 con 0x560b81f9abc0 2024-04-16T17:20:45.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.268+0000 7f574ea00640 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b81f9abc0 0x560b81ffbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.269+0000 7f5738c00640 10 client.29879 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.269+0000 7f574ea00640 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b81f9abc0 0x560b81ffbc20 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982f65f63ee30467 server_cookie=8d74869b04929672 in_seq=0 out_seq=0 2024-04-16T17:20:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f5738c00640 1 -- 172.21.15.18:0/2902553573 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f573c013470 con 0x560b81f9abc0 2024-04-16T17:20:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f5738c00640 10 client.29879 handle_command_reply: tid=0 2024-04-16T17:20:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 client.29879 shutdown 2024-04-16T17:20:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 -- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f821a0 msgr2=0x560b81f8d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f821a0 0x560b81f8d450 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5748099ca0 tx=0x7f574809b070 comp rx=0 tx=0).stop 2024-04-16T17:20:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 -- 172.21.15.18:0/2902553573 shutdown_connections 2024-04-16T17:20:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560b81f9abc0 0x560b81ffbc20 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b81f821a0 0x560b81f8d450 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560b81f795c0 0x560b81f8fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 --2- 172.21.15.18:0/2902553573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b81f78ca0 0x560b81f92e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f575843e740 1 -- 172.21.15.18:0/2902553573 >> 172.21.15.18:0/2902553573 conn(0x560b81f733d0 msgr2=0x560b81f7f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.275+0000 7f5738c00640 0 client.29879 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:45.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.277+0000 7f575843e740 1 -- 172.21.15.18:0/2902553573 shutdown_connections 2024-04-16T17:20:45.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.277+0000 7f575843e740 1 -- 172.21.15.18:0/2902553573 wait complete. 2024-04-16T17:20:45.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.277+0000 7f575843e740 20 client.29879 trim_cache size 0 max 16384 2024-04-16T17:20:45.277 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:20:45.277 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:20:45.277 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "1a3fbd2c-39df-4f90-b6fa-39c63ed1dd16", 2024-04-16T17:20:45.278 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:20:45.278 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:20:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57340a3fd0 msgr2=0x7f57340a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57340a3fd0 0x7f57340a6490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5740006010 tx=0x7f5740004040 comp rx=0 tx=0).stop 2024-04-16T17:20:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575010c5b0 msgr2=0x7f575006cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575010c5b0 0x7f575006cdd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f574800be20 tx=0x7f574800be50 comp rx=0 tx=0).stop 2024-04-16T17:20:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 shutdown_connections 2024-04-16T17:20:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57340a3fd0 0x7f57340a6490 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575010c5b0 0x7f575006cdd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750103b90 0x7f575006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 --2- 172.21.15.18:0/2835151926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5750103270 0x7f57500737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.279+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 >> 172.21.15.18:0/2835151926 conn(0x7f57500780b0 msgr2=0x7f57500fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.280+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 shutdown_connections 2024-04-16T17:20:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.280+0000 7f5755800640 1 -- 172.21.15.18:0/2835151926 wait complete. 2024-04-16T17:20:45.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:20:45.311 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:20:45.483 INFO:tasks.workunit.client.0.smithi018.stdout: 10 45926 13.18 MB/sec execute 57 sec latency 250.535 ms 2024-04-16T17:20:45.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.605+0000 7f6aedc00640 1 -- 172.21.15.18:0/3294345497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8109520 msgr2=0x7f6ae81110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.605+0000 7f6aedc00640 1 --2- 172.21.15.18:0/3294345497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8109520 0x7f6ae81110f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009ca0 tx=0x7f6ae002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.607+0000 7f6aedc00640 1 -- 172.21.15.18:0/3294345497 shutdown_connections 2024-04-16T17:20:45.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.607+0000 7f6aedc00640 1 --2- 172.21.15.18:0/3294345497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8109520 0x7f6ae81110f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.607+0000 7f6aedc00640 1 --2- 172.21.15.18:0/3294345497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ae8104e00 0x7f6ae8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.607+0000 7f6aedc00640 1 --2- 172.21.15.18:0/3294345497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae8104450 0x7f6ae8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.607+0000 7f6aedc00640 1 -- 172.21.15.18:0/3294345497 >> 172.21.15.18:0/3294345497 conn(0x7f6ae80ffc70 msgr2=0x7f6ae8102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.608+0000 7f6aedc00640 1 -- 172.21.15.18:0/3294345497 shutdown_connections 2024-04-16T17:20:45.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.608+0000 7f6aedc00640 1 -- 172.21.15.18:0/3294345497 wait complete. 2024-04-16T17:20:45.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.610+0000 7f6aedc00640 1 Processor -- start 2024-04-16T17:20:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.610+0000 7f6aedc00640 1 -- start start 2024-04-16T17:20:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.610+0000 7f6aedc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8104450 0x7f6ae8076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.610+0000 7f6aedc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae8109520 0x7f6ae8076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.611+0000 7f6aedc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ae8076f90 0x7f6ae81ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.611+0000 7f6ae6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8104450 0x7f6ae8076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.611+0000 7f6ade000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae8109520 0x7f6ae8076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.611+0000 7f6ae6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8104450 0x7f6ae8076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51330/0 (socket says 172.21.15.18:51330) 2024-04-16T17:20:45.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.611+0000 7f6ade000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae8109520 0x7f6ae8076a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36774/0 (socket says 172.21.15.18:36774) 2024-04-16T17:20:45.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.611+0000 7f6ade000640 1 -- 172.21.15.18:0/319566907 learned_addr learned my addr 172.21.15.18:0/319566907 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6ae7400640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ae8076f90 0x7f6ae81ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8114630 con 0x7f6ae8109520 2024-04-16T17:20:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81144b0 con 0x7f6ae8104450 2024-04-16T17:20:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81147b0 con 0x7f6ae8076f90 2024-04-16T17:20:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6ae7400640 1 -- 172.21.15.18:0/319566907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8104450 msgr2=0x7f6ae8076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6ae7400640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8104450 0x7f6ae8076510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6ae7400640 1 -- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae8109520 msgr2=0x7f6ae8076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6ae7400640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae8109520 0x7f6ae8076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.612+0000 7f6ae7400640 1 -- 172.21.15.18:0/319566907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae81ab6a0 con 0x7f6ae8076f90 2024-04-16T17:20:45.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.613+0000 7f6ae7400640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ae8076f90 0x7f6ae81ab160 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009bf0 tx=0x7f6ae0032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.614+0000 7f6adfe00640 1 -- 172.21.15.18:0/319566907 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0041020 con 0x7f6ae8076f90 2024-04-16T17:20:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.614+0000 7f6adfe00640 1 -- 172.21.15.18:0/319566907 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae0034cd0 con 0x7f6ae8076f90 2024-04-16T17:20:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.614+0000 7f6adfe00640 1 -- 172.21.15.18:0/319566907 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0039040 con 0x7f6ae8076f90 2024-04-16T17:20:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.615+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae81ab930 con 0x7f6ae8076f90 2024-04-16T17:20:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.615+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae81abda0 con 0x7f6ae8076f90 2024-04-16T17:20:45.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.621+0000 7f6adfe00640 1 -- 172.21.15.18:0/319566907 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ae0044020 con 0x7f6ae8076f90 2024-04-16T17:20:45.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.623+0000 7f6af09df740 1 Processor -- start 2024-04-16T17:20:45.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.623+0000 7f6af09df740 1 -- start start 2024-04-16T17:20:45.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.623+0000 7f6af09df740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a8a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.623+0000 7f6af09df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8aad0 0x560280a93170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.623+0000 7f6af09df740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 0x560280a95aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.623+0000 7f6af09df740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5602809f0120 con 0x560280a936b0 2024-04-16T17:20:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.623+0000 7f6af09df740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560280996340 con 0x560280a8a1b0 2024-04-16T17:20:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.623+0000 7f6af09df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5602809ec0e0 con 0x560280a8aad0 2024-04-16T17:20:45.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.624+0000 7f6ae6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8aad0 0x560280a93170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.624+0000 7f6ae6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8aad0 0x560280a93170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49692/0 (socket says 172.21.15.18:49692) 2024-04-16T17:20:45.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.624+0000 7f6ae6a00640 1 -- 172.21.15.18:0/926264122 learned_addr learned my addr 172.21.15.18:0/926264122 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.624+0000 7f6ade000640 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a8a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.624+0000 7f6ae6a00640 1 -- 172.21.15.18:0/926264122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 msgr2=0x560280a8a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.625+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a8a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.625+0000 7f6ae6a00640 1 -- 172.21.15.18:0/926264122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 msgr2=0x560280a95aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.625+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 0x560280a95aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.625+0000 7f6ae6a00640 1 -- 172.21.15.18:0/926264122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602807cc7d0 con 0x560280a8aad0 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.625+0000 7f6ade000640 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a8a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.625+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8aad0 0x560280a93170 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009a70 tx=0x7f6ad0009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27bcc4a71532f955 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.626+0000 7f6ad7e00640 1 -- 172.21.15.18:0/926264122 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0017070 con 0x560280a8aad0 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.626+0000 7f6ad7e00640 1 -- 172.21.15.18:0/926264122 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad0013470 con 0x560280a8aad0 2024-04-16T17:20:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.626+0000 7f6ad7e00640 1 -- 172.21.15.18:0/926264122 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0012630 con 0x560280a8aad0 2024-04-16T17:20:45.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.627+0000 7f6adfe00640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6acc0a3f20 0x7f6acc0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.627+0000 7f6af09df740 1 -- 172.21.15.18:0/926264122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8aad0 msgr2=0x560280a93170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.627+0000 7f6af09df740 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8aad0 0x560280a93170 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009a70 tx=0x7f6ad0009f30 comp rx=0 tx=0).stop 2024-04-16T17:20:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.628+0000 7f6af09df740 1 -- 172.21.15.18:0/926264122 shutdown_connections 2024-04-16T17:20:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.628+0000 7f6af09df740 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 0x560280a95aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.628+0000 7f6af09df740 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8aad0 0x560280a93170 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.628+0000 7f6af09df740 1 --2- 172.21.15.18:0/926264122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a8a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.628+0000 7f6af09df740 1 -- 172.21.15.18:0/926264122 >> 172.21.15.18:0/926264122 conn(0x560280a848e0 msgr2=0x560280a84cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.628+0000 7f6ade000640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6acc0a3f20 0x7f6acc0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.628+0000 7f6adfe00640 1 -- 172.21.15.18:0/319566907 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f6ae00f2620 con 0x7f6ae8076f90 2024-04-16T17:20:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.628+0000 7f6af09df740 1 -- 172.21.15.18:0/926264122 shutdown_connections 2024-04-16T17:20:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.629+0000 7f6af09df740 1 -- 172.21.15.18:0/926264122 wait complete. 2024-04-16T17:20:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.629+0000 7f6ade000640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6acc0a3f20 0x7f6acc0a63e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x560280a85a70 tx=0x7f6ad80046c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.629+0000 7f6af09df740 1 -- start start 2024-04-16T17:20:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.630+0000 7f6af09df740 1 Processor -- start 2024-04-16T17:20:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.630+0000 7f6af09df740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.631+0000 7f6af09df740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a8aad0 0x560280a90c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.631+0000 7f6af09df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a936b0 0x560280a91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.631+0000 7f6ae6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.631+0000 7f6af09df740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560280a964a0 con 0x560280a8aad0 2024-04-16T17:20:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.631+0000 7f6af09df740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5602809a9710 con 0x560280a8a1b0 2024-04-16T17:20:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.631+0000 7f6af09df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560280a96620 con 0x560280a936b0 2024-04-16T17:20:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6af09df740 10 client.0 _ll_register_callbacks cb 0x5602809a4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ade000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a8aad0 0x560280a90c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ade000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a8aad0 0x560280a90c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36788/0 (socket says 172.21.15.18:36788) 2024-04-16T17:20:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ade000640 1 -- 172.21.15.18:0/112253943 learned_addr learned my addr 172.21.15.18:0/112253943 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.631+0000 7f6ae7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a936b0 0x560280a91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ae6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a906d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51358/0 (socket says 172.21.15.18:51358) 2024-04-16T17:20:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ade000640 1 -- 172.21.15.18:0/112253943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a936b0 msgr2=0x560280a91150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ade000640 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a936b0 0x560280a91150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ade000640 1 -- 172.21.15.18:0/112253943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 msgr2=0x560280a906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ade000640 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ade000640 1 -- 172.21.15.18:0/112253943 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602807023f0 con 0x560280a8aad0 2024-04-16T17:20:45.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.632+0000 7f6ae7400640 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a936b0 0x560280a91150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:45.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.633+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:45.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.633+0000 7f6af09df740 1 -- 172.21.15.18:0/112253943 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad0013470 con 0x560280a8aad0 2024-04-16T17:20:45.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.633+0000 7f6ade000640 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a8aad0 0x560280a90c10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6ad801c6d0 tx=0x7f6ad8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f33b82f1c3504e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.633+0000 7f6ad4c00640 10 client.19308 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:20:45.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.635+0000 7f6ad4c00640 1 -- 172.21.15.18:0/112253943 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad802c060 con 0x560280a8aad0 2024-04-16T17:20:45.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.635+0000 7f6ad4c00640 1 -- 172.21.15.18:0/112253943 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f6ad8016030 con 0x560280a8aad0 2024-04-16T17:20:45.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.635+0000 7f6ad4c00640 1 -- 172.21.15.18:0/112253943 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6ad801cd00 con 0x560280a8aad0 2024-04-16T17:20:45.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.635+0000 7f6af09df740 10 client.19308 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:45.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.635+0000 7f6af09df740 1 -- 172.21.15.18:0/112253943 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=35}) -- 0x5602806c20f0 con 0x560280a8aad0 2024-04-16T17:20:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.636+0000 7f6ad4c00640 1 -- 172.21.15.18:0/112253943 <== mon.0 v2:172.21.15.18:3300/0 4 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6ad80303d0 con 0x560280a8aad0 2024-04-16T17:20:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.636+0000 7f6af09df740 10 client.19308 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.636+0000 7f6af09df740 10 client.19308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.636+0000 7f6af09df740 20 client.19308 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:45.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.636+0000 7f6af09df740 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560280aa3840 0x560280aa5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.636+0000 7f6af09df740 4 client.19308 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:45.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.636+0000 7f6af09df740 1 -- 172.21.15.18:0/112253943 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602809fdb40 con 0x560280aa3840 2024-04-16T17:20:45.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.637+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560280aa3840 0x560280aa5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.637+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560280aa3840 0x560280aa5c30 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75422b6cc15b1508 server_cookie=48687aec651bdebc in_seq=0 out_seq=0 2024-04-16T17:20:45.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.637+0000 7f6ad4c00640 10 client.19308 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.645+0000 7f6ad4c00640 1 -- 172.21.15.18:0/112253943 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602809fdb40 con 0x560280aa3840 2024-04-16T17:20:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.645+0000 7f6ad4c00640 10 client.19308 handle_command_reply: tid=0 2024-04-16T17:20:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.645+0000 7f6af09df740 1 client.19308 shutdown 2024-04-16T17:20:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.645+0000 7f6af09df740 1 -- 172.21.15.18:0/112253943 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a8aad0 msgr2=0x560280a90c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.645+0000 7f6af09df740 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a8aad0 0x560280a90c10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6ad801c6d0 tx=0x7f6ad8005f90 comp rx=0 tx=0).stop 2024-04-16T17:20:45.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 1 -- 172.21.15.18:0/112253943 shutdown_connections 2024-04-16T17:20:45.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560280aa3840 0x560280aa5c30 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a936b0 0x560280a91150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a8aad0 0x560280a90c10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 1 --2- 172.21.15.18:0/112253943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 1 -- 172.21.15.18:0/112253943 >> 172.21.15.18:0/112253943 conn(0x560280a848e0 msgr2=0x560280a86630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6ad4c00640 0 client.19308 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 1 -- 172.21.15.18:0/112253943 shutdown_connections 2024-04-16T17:20:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 1 -- 172.21.15.18:0/112253943 wait complete. 2024-04-16T17:20:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.646+0000 7f6af09df740 20 client.19308 trim_cache size 0 max 16384 2024-04-16T17:20:45.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.691+0000 7f6af09df740 1 Processor -- start 2024-04-16T17:20:45.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.691+0000 7f6af09df740 1 -- start start 2024-04-16T17:20:45.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.691+0000 7f6af09df740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a92e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.691+0000 7f6af09df740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 0x560280a8f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.691+0000 7f6af09df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a9f3c0 0x560280a8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.691+0000 7f6af09df740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b8580 con 0x560280a936b0 2024-04-16T17:20:45.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.691+0000 7f6af09df740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b5e50 con 0x560280a8a1b0 2024-04-16T17:20:45.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.691+0000 7f6af09df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b3cc0 con 0x560280a9f3c0 2024-04-16T17:20:45.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.692+0000 7f6ae7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a9f3c0 0x560280a8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.692+0000 7f6ae7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a9f3c0 0x560280a8bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49706/0 (socket says 172.21.15.18:49706) 2024-04-16T17:20:45.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.692+0000 7f6ae7400640 1 -- 172.21.15.18:0/4177518647 learned_addr learned my addr 172.21.15.18:0/4177518647 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.692+0000 7f6ade000640 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 0x560280a8f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.692+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a92e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.693+0000 7f6ae6a00640 1 -- 172.21.15.18:0/4177518647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a9f3c0 msgr2=0x560280a8bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.693+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a9f3c0 0x560280a8bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.693+0000 7f6ae6a00640 1 -- 172.21.15.18:0/4177518647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 msgr2=0x560280a8f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.693+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 0x560280a8f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.693+0000 7f6ae6a00640 1 -- 172.21.15.18:0/4177518647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560280a9f840 con 0x560280a8a1b0 2024-04-16T17:20:45.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.693+0000 7f6ade000640 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 0x560280a8f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:20:45.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.693+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a92e00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0000c00 tx=0x7f6ad0009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a85cad50179fa34 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.694+0000 7f6ad7400640 1 -- 172.21.15.18:0/4177518647 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0017040 con 0x560280a8a1b0 2024-04-16T17:20:45.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.694+0000 7f6ad7400640 1 -- 172.21.15.18:0/4177518647 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad002c070 con 0x560280a8a1b0 2024-04-16T17:20:45.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.694+0000 7f6ad7400640 1 -- 172.21.15.18:0/4177518647 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad001ca10 con 0x560280a8a1b0 2024-04-16T17:20:45.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.696+0000 7f6af09df740 1 -- 172.21.15.18:0/4177518647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 msgr2=0x560280a92e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.696+0000 7f6af09df740 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a92e00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0000c00 tx=0x7f6ad0009110 comp rx=0 tx=0).stop 2024-04-16T17:20:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.696+0000 7f6af09df740 1 -- 172.21.15.18:0/4177518647 shutdown_connections 2024-04-16T17:20:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.696+0000 7f6af09df740 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a9f3c0 0x560280a8bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.696+0000 7f6af09df740 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a936b0 0x560280a8f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.696+0000 7f6af09df740 1 --2- 172.21.15.18:0/4177518647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a8a1b0 0x560280a92e00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.696+0000 7f6af09df740 1 -- 172.21.15.18:0/4177518647 >> 172.21.15.18:0/4177518647 conn(0x560280a848e0 msgr2=0x560280a90f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.697+0000 7f6af09df740 1 -- 172.21.15.18:0/4177518647 shutdown_connections 2024-04-16T17:20:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.697+0000 7f6af09df740 1 -- 172.21.15.18:0/4177518647 wait complete. 2024-04-16T17:20:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.697+0000 7f6af09df740 1 -- start start 2024-04-16T17:20:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.698+0000 7f6af09df740 1 Processor -- start 2024-04-16T17:20:45.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6af09df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8a1b0 0x560280b057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6af09df740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a936b0 0x560280a9b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6af09df740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a9f3c0 0x560280b02b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6af09df740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b4310 con 0x560280a9f3c0 2024-04-16T17:20:45.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6af09df740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b5780 con 0x560280a936b0 2024-04-16T17:20:45.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6af09df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b49e0 con 0x560280a8a1b0 2024-04-16T17:20:45.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6af09df740 10 client.0 _ll_register_callbacks cb 0x7f6ad8009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6ade000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a936b0 0x560280a9b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6ade000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a936b0 0x560280a9b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51374/0 (socket says 172.21.15.18:51374) 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6ade000640 1 -- 172.21.15.18:0/2523143846 learned_addr learned my addr 172.21.15.18:0/2523143846 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6ae6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8a1b0 0x560280b057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.699+0000 7f6ae7400640 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a9f3c0 0x560280b02b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.700+0000 7f6ade000640 1 -- 172.21.15.18:0/2523143846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8a1b0 msgr2=0x560280b057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.700+0000 7f6ade000640 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8a1b0 0x560280b057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.700+0000 7f6ade000640 1 -- 172.21.15.18:0/2523143846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a9f3c0 msgr2=0x560280b02b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.700+0000 7f6ade000640 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a9f3c0 0x560280b02b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.700+0000 7f6ade000640 1 -- 172.21.15.18:0/2523143846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560280b030b0 con 0x560280a936b0 2024-04-16T17:20:45.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.700+0000 7f6ade000640 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a936b0 0x560280a9b8d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8010d80 tx=0x7f6ad8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec452719af1f53b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:45.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.700+0000 7f6ad4c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:20:45.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.701+0000 7f6ad4c00640 1 -- 172.21.15.18:0/2523143846 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad802c040 con 0x560280a936b0 2024-04-16T17:20:45.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.701+0000 7f6ad4c00640 1 -- 172.21.15.18:0/2523143846 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f6ad8005040 con 0x560280a936b0 2024-04-16T17:20:45.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.701+0000 7f6af09df740 1 -- 172.21.15.18:0/2523143846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad002c070 con 0x560280a936b0 2024-04-16T17:20:45.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.702+0000 7f6ad4c00640 1 -- 172.21.15.18:0/2523143846 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6ad801c4b0 con 0x560280a936b0 2024-04-16T17:20:45.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.702+0000 7f6af09df740 10 client.29894 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:45.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.702+0000 7f6af09df740 1 -- 172.21.15.18:0/2523143846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560280b2c4e0 con 0x560280a936b0 2024-04-16T17:20:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.703+0000 7f6ad4c00640 1 -- 172.21.15.18:0/2523143846 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad0013470 con 0x560280a936b0 2024-04-16T17:20:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.703+0000 7f6ad4c00640 1 -- 172.21.15.18:0/2523143846 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6ad8004ae0 con 0x560280a936b0 2024-04-16T17:20:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.703+0000 7f6af09df740 10 client.29894 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.703+0000 7f6af09df740 10 client.29894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.703+0000 7f6af09df740 20 client.29894 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:45.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.703+0000 7f6af09df740 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560280b2cc00 0x560280b2f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:45.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.704+0000 7f6af09df740 4 client.29894 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:20:45.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.704+0000 7f6af09df740 1 -- 172.21.15.18:0/2523143846 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6ad002c070 con 0x560280b2cc00 2024-04-16T17:20:45.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.704+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560280b2cc00 0x560280b2f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.705+0000 7f6ae6a00640 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560280b2cc00 0x560280b2f020 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92105f9499cdfa6 server_cookie=1a5c3583ce0a60d9 in_seq=0 out_seq=0 2024-04-16T17:20:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.705+0000 7f6ad4c00640 10 client.29894 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:45.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.707+0000 7f6ad4c00640 1 -- 172.21.15.18:0/2523143846 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x7f6ad002c070 con 0x560280b2cc00 2024-04-16T17:20:45.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.707+0000 7f6ad4c00640 10 client.29894 handle_command_reply: tid=0 2024-04-16T17:20:45.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.707+0000 7f6af09df740 1 client.29894 shutdown 2024-04-16T17:20:45.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.709+0000 7f6af09df740 1 -- 172.21.15.18:0/2523143846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a936b0 msgr2=0x560280a9b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.709+0000 7f6af09df740 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a936b0 0x560280a9b8d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8010d80 tx=0x7f6ad8017000 comp rx=0 tx=0).stop 2024-04-16T17:20:45.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.711+0000 7f6af09df740 1 -- 172.21.15.18:0/2523143846 shutdown_connections 2024-04-16T17:20:45.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.711+0000 7f6af09df740 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560280b2cc00 0x560280b2f020 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.711+0000 7f6af09df740 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560280a9f3c0 0x560280b02b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.711+0000 7f6ad4c00640 0 client.29894 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.711+0000 7f6af09df740 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560280a936b0 0x560280a9b8d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.711+0000 7f6af09df740 1 --2- 172.21.15.18:0/2523143846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560280a8a1b0 0x560280b057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.712+0000 7f6af09df740 1 -- 172.21.15.18:0/2523143846 >> 172.21.15.18:0/2523143846 conn(0x560280a848e0 msgr2=0x560280a85c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.712+0000 7f6af09df740 1 -- 172.21.15.18:0/2523143846 shutdown_connections 2024-04-16T17:20:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.712+0000 7f6af09df740 1 -- 172.21.15.18:0/2523143846 wait complete. 2024-04-16T17:20:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.712+0000 7f6af09df740 20 client.29894 trim_cache size 0 max 16384 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (345 inodes in the stack)", 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: "1a3fbd2c-39df-4f90-b6fa-39c63ed1dd16": { 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "1a3fbd2c-39df-4f90-b6fa-39c63ed1dd16", 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:20:45.713 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.714+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6acc0a3f20 msgr2=0x7f6acc0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.714+0000 7f6aedc00640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6acc0a3f20 0x7f6acc0a63e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x560280a85a70 tx=0x7f6ad80046c0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.715+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ae8076f90 msgr2=0x7f6ae81ab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.715+0000 7f6aedc00640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ae8076f90 0x7f6ae81ab160 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009bf0 tx=0x7f6ae0032ec0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.715+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 shutdown_connections 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.715+0000 7f6aedc00640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f6acc0a3f20 0x7f6acc0a63e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.715+0000 7f6aedc00640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ae8076f90 0x7f6ae81ab160 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.715+0000 7f6aedc00640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ae8109520 0x7f6ae8076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.715+0000 7f6aedc00640 1 --2- 172.21.15.18:0/319566907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ae8104450 0x7f6ae8076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:45.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.715+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 >> 172.21.15.18:0/319566907 conn(0x7f6ae80ffc70 msgr2=0x7f6ae8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:45.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.718+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 shutdown_connections 2024-04-16T17:20:45.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:20:45.719+0000 7f6aedc00640 1 -- 172.21.15.18:0/319566907 wait complete. 2024-04-16T17:20:45.759 INFO:tasks.cephfs.filesystem:scrub status for tag:1a3fbd2c-39df-4f90-b6fa-39c63ed1dd16 - {'path': '/', 'tag': '1a3fbd2c-39df-4f90-b6fa-39c63ed1dd16', 'options': 'recursive,force'} 2024-04-16T17:20:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:45 smithi022 ceph-mon[36023]: pgmap v510: 129 pgs: 129 active+clean; 382 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2024-04-16T17:20:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:45 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:20:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:45 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:45 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:20:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:45 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:46.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:45 smithi043 ceph-mon[36901]: pgmap v510: 129 pgs: 129 active+clean; 382 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2024-04-16T17:20:46.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:45 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:20:46.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:45 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:46.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:45 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:20:46.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:45 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.452+0000 7f0903e00640 1 -- 172.21.15.18:0/2689662294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0904103900 msgr2=0x7f090410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.452+0000 7f0903e00640 1 --2- 172.21.15.18:0/2689662294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0904103900 0x7f090410bde0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009900 tx=0x7f08f402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.452+0000 7f0903e00640 1 -- 172.21.15.18:0/2689662294 shutdown_connections 2024-04-16T17:20:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.452+0000 7f0903e00640 1 --2- 172.21.15.18:0/2689662294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f090410c400 0x7f090410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.452+0000 7f0903e00640 1 --2- 172.21.15.18:0/2689662294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0904103900 0x7f090410bde0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.452+0000 7f0903e00640 1 --2- 172.21.15.18:0/2689662294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0904102fe0 0x7f09041033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.452+0000 7f0903e00640 1 -- 172.21.15.18:0/2689662294 >> 172.21.15.18:0/2689662294 conn(0x7f09040fc7e0 msgr2=0x7f09040fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.452+0000 7f0903e00640 1 -- 172.21.15.18:0/2689662294 shutdown_connections 2024-04-16T17:20:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.455+0000 7f0903e00640 1 -- 172.21.15.18:0/2689662294 wait complete. 2024-04-16T17:20:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.457+0000 7f0903e00640 1 Processor -- start 2024-04-16T17:20:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.457+0000 7f0903e00640 1 -- start start 2024-04-16T17:20:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0903e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0904102fe0 0x7f09041a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0903e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0904103900 0x7f09041a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0903e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f090410c400 0x7f090419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0903e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0904114490 con 0x7f0904103900 2024-04-16T17:20:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0903e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0904114310 con 0x7f0904102fe0 2024-04-16T17:20:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0903e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0904114610 con 0x7f090410c400 2024-04-16T17:20:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0902000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0904103900 0x7f09041a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0902000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0904103900 0x7f09041a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36826/0 (socket says 172.21.15.18:36826) 2024-04-16T17:20:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0902000640 1 -- 172.21.15.18:0/2499707548 learned_addr learned my addr 172.21.15.18:0/2499707548 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0902a00640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0904102fe0 0x7f09041a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.458+0000 7f0903400640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f090410c400 0x7f090419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.459+0000 7f0902000640 1 -- 172.21.15.18:0/2499707548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f090410c400 msgr2=0x7f090419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.459+0000 7f0902000640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f090410c400 0x7f090419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.459+0000 7f0902000640 1 -- 172.21.15.18:0/2499707548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0904102fe0 msgr2=0x7f09041a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.459+0000 7f0902000640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0904102fe0 0x7f09041a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.459+0000 7f0902000640 1 -- 172.21.15.18:0/2499707548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090419b010 con 0x7f0904103900 2024-04-16T17:20:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.459+0000 7f0902000640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0904103900 0x7f09041a0bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f08f4000c00 tx=0x7f08f40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.460+0000 7f0908a00640 1 -- 172.21.15.18:0/2499707548 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4042020 con 0x7f0904103900 2024-04-16T17:20:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.460+0000 7f0908a00640 1 -- 172.21.15.18:0/2499707548 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f4005000 con 0x7f0904103900 2024-04-16T17:20:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.460+0000 7f0908a00640 1 -- 172.21.15.18:0/2499707548 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4003b20 con 0x7f0904103900 2024-04-16T17:20:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.460+0000 7f0903e00640 1 -- 172.21.15.18:0/2499707548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090419b2a0 con 0x7f0904103900 2024-04-16T17:20:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.460+0000 7f0903e00640 1 -- 172.21.15.18:0/2499707548 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09041a7610 con 0x7f0904103900 2024-04-16T17:20:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.464+0000 7f090cba3740 1 Processor -- start 2024-04-16T17:20:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.465+0000 7f090cba3740 1 -- start start 2024-04-16T17:20:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.465+0000 7f090cba3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa114d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.465+0000 7f090cba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa114df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.465+0000 7f090cba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa1161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.465+0000 7f090cba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cfa10b2780 con 0x55cfa115f810 2024-04-16T17:20:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.465+0000 7f090cba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfa1058f40 con 0x55cfa114da80 2024-04-16T17:20:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.465+0000 7f090cba3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cfa10ae740 con 0x55cfa114e7f0 2024-04-16T17:20:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.466+0000 7f0903400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa114d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.466+0000 7f0903400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa114d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49748/0 (socket says 172.21.15.18:49748) 2024-04-16T17:20:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.466+0000 7f0903400640 1 -- 172.21.15.18:0/60004600 learned_addr learned my addr 172.21.15.18:0/60004600 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.466+0000 7f0902a00640 1 --2- 172.21.15.18:0/60004600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa114df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.467+0000 7f0902a00640 1 -- 172.21.15.18:0/60004600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 msgr2=0x55cfa114d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.467+0000 7f0902a00640 1 --2- 172.21.15.18:0/60004600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa114d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.467+0000 7f0902a00640 1 -- 172.21.15.18:0/60004600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 msgr2=0x55cfa1161c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:20:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.467+0000 7f0902a00640 1 --2- 172.21.15.18:0/60004600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa1161c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.467+0000 7f0902a00640 1 -- 172.21.15.18:0/60004600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa0dc5210 con 0x55cfa114da80 2024-04-16T17:20:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.467+0000 7f0908a00640 1 -- 172.21.15.18:0/2499707548 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f08f4045020 con 0x7f0904103900 2024-04-16T17:20:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.467+0000 7f0902a00640 1 --2- 172.21.15.18:0/60004600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa114df00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009fb0 tx=0x7f08fc010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4624df0d250556b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.468+0000 7f08efe00640 1 -- 172.21.15.18:0/60004600 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc016070 con 0x55cfa114da80 2024-04-16T17:20:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.468+0000 7f08efe00640 1 -- 172.21.15.18:0/60004600 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc004d20 con 0x55cfa114da80 2024-04-16T17:20:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.468+0000 7f08efe00640 1 -- 172.21.15.18:0/60004600 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc005440 con 0x55cfa114da80 2024-04-16T17:20:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.469+0000 7f090cba3740 1 -- 172.21.15.18:0/60004600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 msgr2=0x55cfa114df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.469+0000 7f090cba3740 1 --2- 172.21.15.18:0/60004600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa114df00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009fb0 tx=0x7f08fc010800 comp rx=0 tx=0).stop 2024-04-16T17:20:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.470+0000 7f090cba3740 1 -- 172.21.15.18:0/60004600 shutdown_connections 2024-04-16T17:20:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.470+0000 7f090cba3740 1 --2- 172.21.15.18:0/60004600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa1161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.470+0000 7f090cba3740 1 --2- 172.21.15.18:0/60004600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa114df00 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.470+0000 7f090cba3740 1 --2- 172.21.15.18:0/60004600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa114d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.470+0000 7f090cba3740 1 -- 172.21.15.18:0/60004600 >> 172.21.15.18:0/60004600 conn(0x55cfa10e85d0 msgr2=0x55cfa1148770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.470+0000 7f090cba3740 1 -- 172.21.15.18:0/60004600 shutdown_connections 2024-04-16T17:20:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.470+0000 7f090cba3740 1 -- 172.21.15.18:0/60004600 wait complete. 2024-04-16T17:20:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.471+0000 7f090cba3740 1 -- start start 2024-04-16T17:20:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.471+0000 7f090cba3740 1 Processor -- start 2024-04-16T17:20:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.471+0000 7f0908a00640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08e80a3d70 0x7f08e80a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f0908a00640 1 -- 172.21.15.18:0/2499707548 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f08f40f2710 con 0x7f0904103900 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f090cba3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114da80 0x55cfa115f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f090cba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114e7f0 0x55cfa115c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f0903400640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08e80a3d70 0x7f08e80a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f090cba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa115cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f090cba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cfa1162540 con 0x55cfa115f810 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f090cba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfa106c390 con 0x55cfa114e7f0 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f090cba3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cfa11626c0 con 0x55cfa114da80 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.472+0000 7f090cba3740 10 client.0 _ll_register_callbacks cb 0x55cfa1055f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.473+0000 7f0902a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114da80 0x55cfa115f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.473+0000 7f0902a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114da80 0x55cfa115f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49760/0 (socket says 172.21.15.18:49760) 2024-04-16T17:20:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.473+0000 7f0902a00640 1 -- 172.21.15.18:0/1154722545 learned_addr learned my addr 172.21.15.18:0/1154722545 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.473+0000 7f0902000640 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114e7f0 0x55cfa115c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.473+0000 7f0903400640 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa115cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.474+0000 7f0902a00640 1 -- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114e7f0 msgr2=0x55cfa115c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.474+0000 7f0902a00640 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114e7f0 0x55cfa115c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.474+0000 7f0902a00640 1 -- 172.21.15.18:0/1154722545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 msgr2=0x55cfa115cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.474+0000 7f0902a00640 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa115cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.474+0000 7f0902a00640 1 -- 172.21.15.18:0/1154722545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa0e91620 con 0x55cfa114da80 2024-04-16T17:20:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.475+0000 7f0902a00640 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114da80 0x55cfa115f290 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f08fc012ec0 tx=0x7f08fc005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5fb62779e230ce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.475+0000 7f08ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:20:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.475+0000 7f0902000640 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114e7f0 0x55cfa115c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.475+0000 7f0903400640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08e80a3d70 0x7f08e80a6230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f08f0005fd0 tx=0x7f08f000a730 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.475+0000 7f08ecc00640 1 -- 172.21.15.18:0/1154722545 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc029060 con 0x55cfa114da80 2024-04-16T17:20:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.475+0000 7f08ecc00640 1 -- 172.21.15.18:0/1154722545 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f08fc0051e0 con 0x55cfa114da80 2024-04-16T17:20:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.476+0000 7f090cba3740 1 -- 172.21.15.18:0/1154722545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08fc004d20 con 0x55cfa114da80 2024-04-16T17:20:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.476+0000 7f08ecc00640 1 -- 172.21.15.18:0/1154722545 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f08fc016070 con 0x55cfa114da80 2024-04-16T17:20:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.476+0000 7f090cba3740 10 client.29725 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.476+0000 7f090cba3740 1 -- 172.21.15.18:0/1154722545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cfa116e620 con 0x55cfa114da80 2024-04-16T17:20:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.476+0000 7f08ecc00640 1 -- 172.21.15.18:0/1154722545 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc004d20 con 0x55cfa114da80 2024-04-16T17:20:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.477+0000 7f08ecc00640 1 -- 172.21.15.18:0/1154722545 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f08fc00f040 con 0x55cfa114da80 2024-04-16T17:20:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.478+0000 7f090cba3740 10 client.29725 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.478+0000 7f090cba3740 10 client.29725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.478+0000 7f090cba3740 20 client.29725 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.478+0000 7f090cba3740 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfa116fa20 0x55cfa1171e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.478+0000 7f090cba3740 4 client.29725 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:20:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.478+0000 7f090cba3740 1 -- 172.21.15.18:0/1154722545 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfa10c01a0 con 0x55cfa116fa20 2024-04-16T17:20:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.479+0000 7f0902000640 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfa116fa20 0x55cfa1171e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.480+0000 7f0902000640 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfa116fa20 0x55cfa1171e10 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd529d00807a054 server_cookie=4b7c9e4c1707f9ff in_seq=0 out_seq=0 2024-04-16T17:20:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.480+0000 7f08ecc00640 10 client.29725 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:46.483 INFO:tasks.workunit.client.0.smithi018.stdout: 10 46205 13.25 MB/sec execute 58 sec latency 262.075 ms 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f08ecc00640 1 -- 172.21.15.18:0/1154722545 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfa10c01a0 con 0x55cfa116fa20 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f08ecc00640 10 client.29725 handle_command_reply: tid=0 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 client.29725 shutdown 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 -- 172.21.15.18:0/1154722545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114da80 msgr2=0x55cfa115f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114da80 0x55cfa115f290 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f08fc012ec0 tx=0x7f08fc005c60 comp rx=0 tx=0).stop 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 -- 172.21.15.18:0/1154722545 shutdown_connections 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfa116fa20 0x55cfa1171e10 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f08ecc00640 0 client.29725 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa115cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114e7f0 0x55cfa115c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 --2- 172.21.15.18:0/1154722545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114da80 0x55cfa115f290 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.487+0000 7f090cba3740 1 -- 172.21.15.18:0/1154722545 >> 172.21.15.18:0/1154722545 conn(0x55cfa10e85d0 msgr2=0x55cfa114a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.488+0000 7f090cba3740 1 -- 172.21.15.18:0/1154722545 shutdown_connections 2024-04-16T17:20:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.488+0000 7f090cba3740 1 -- 172.21.15.18:0/1154722545 wait complete. 2024-04-16T17:20:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.488+0000 7f090cba3740 20 client.29725 trim_cache size 0 max 16384 2024-04-16T17:20:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.511+0000 7f090cba3740 1 Processor -- start 2024-04-16T17:20:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.511+0000 7f090cba3740 1 -- start start 2024-04-16T17:20:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.511+0000 7f090cba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa116b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.511+0000 7f090cba3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa115ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.511+0000 7f090cba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa115f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.511+0000 7f090cba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f09041afb90 con 0x55cfa115f810 2024-04-16T17:20:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.511+0000 7f090cba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ad460 con 0x55cfa114da80 2024-04-16T17:20:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.511+0000 7f090cba3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ab2d0 con 0x55cfa114e7f0 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa115ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa115ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49772/0 (socket says 172.21.15.18:49772) 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902000640 1 -- 172.21.15.18:0/2848666957 learned_addr learned my addr 172.21.15.18:0/2848666957 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902000640 1 -- 172.21.15.18:0/2848666957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 msgr2=0x55cfa116b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902a00640 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa116b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902000640 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa116b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902000640 1 -- 172.21.15.18:0/2848666957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 msgr2=0x55cfa115f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902000640 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa115f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902000640 1 -- 172.21.15.18:0/2848666957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa114cd50 con 0x55cfa114e7f0 2024-04-16T17:20:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.512+0000 7f0902a00640 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa116b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:20:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.513+0000 7f0902000640 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa115ed50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f08f40efb90 tx=0x7f08f402c1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff313f9477c93653 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.514+0000 7f08ef400640 1 -- 172.21.15.18:0/2848666957 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40f1040 con 0x55cfa114e7f0 2024-04-16T17:20:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.514+0000 7f08ef400640 1 -- 172.21.15.18:0/2848666957 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f40c2730 con 0x55cfa114e7f0 2024-04-16T17:20:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.514+0000 7f08ef400640 1 -- 172.21.15.18:0/2848666957 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40051b0 con 0x55cfa114e7f0 2024-04-16T17:20:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 -- 172.21.15.18:0/2848666957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 msgr2=0x55cfa115ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa115ed50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f08f40efb90 tx=0x7f08f402c1e0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 -- 172.21.15.18:0/2848666957 shutdown_connections 2024-04-16T17:20:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa115f810 0x55cfa115f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa115ed50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 --2- 172.21.15.18:0/2848666957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa114da80 0x55cfa116b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 -- 172.21.15.18:0/2848666957 >> 172.21.15.18:0/2848666957 conn(0x55cfa10e85d0 msgr2=0x55cfa116a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 -- 172.21.15.18:0/2848666957 shutdown_connections 2024-04-16T17:20:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.515+0000 7f090cba3740 1 -- 172.21.15.18:0/2848666957 wait complete. 2024-04-16T17:20:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.516+0000 7f090cba3740 1 -- start start 2024-04-16T17:20:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.516+0000 7f090cba3740 1 Processor -- start 2024-04-16T17:20:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.517+0000 7f090cba3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa114da80 0x55cfa1170bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.517+0000 7f090cba3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa1171110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.517+0000 7f090cba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa115f810 0x55cfa11d4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.517+0000 7f090cba3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ab920 con 0x55cfa114da80 2024-04-16T17:20:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.517+0000 7f090cba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09041acd90 con 0x55cfa115f810 2024-04-16T17:20:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.517+0000 7f090cba3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09041abff0 con 0x55cfa114e7f0 2024-04-16T17:20:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.517+0000 7f090cba3740 10 client.0 _ll_register_callbacks cb 0x7f08f000a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:20:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.518+0000 7f0903400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa115f810 0x55cfa11d4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.518+0000 7f0903400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa115f810 0x55cfa11d4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51432/0 (socket says 172.21.15.18:51432) 2024-04-16T17:20:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.518+0000 7f0902000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa1171110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.518+0000 7f0903400640 1 -- 172.21.15.18:0/3443519883 learned_addr learned my addr 172.21.15.18:0/3443519883 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:20:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.518+0000 7f0902a00640 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa114da80 0x55cfa1170bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f0902a00640 1 -- 172.21.15.18:0/3443519883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 msgr2=0x55cfa1171110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f0902a00640 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa1171110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f0902a00640 1 -- 172.21.15.18:0/3443519883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa115f810 msgr2=0x55cfa11d4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:20:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f0902a00640 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa115f810 0x55cfa11d4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f0902a00640 1 -- 172.21.15.18:0/3443519883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa1171a10 con 0x55cfa114da80 2024-04-16T17:20:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f0902000640 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa1171110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f0903400640 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa115f810 0x55cfa11d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:20:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f0902a00640 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa114da80 0x55cfa1170bd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f08fc024730 tx=0x7f08fc009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befb1615b0e47573 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:20:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.520+0000 7f08ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:20:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.521+0000 7f08ecc00640 1 -- 172.21.15.18:0/3443519883 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc01e070 con 0x55cfa114da80 2024-04-16T17:20:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.521+0000 7f08ecc00640 1 -- 172.21.15.18:0/3443519883 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f08fc0057f0 con 0x55cfa114da80 2024-04-16T17:20:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.521+0000 7f090cba3740 1 -- 172.21.15.18:0/3443519883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f40c2730 con 0x55cfa114da80 2024-04-16T17:20:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.521+0000 7f08ecc00640 1 -- 172.21.15.18:0/3443519883 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f08fc03f050 con 0x55cfa114da80 2024-04-16T17:20:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.521+0000 7f090cba3740 10 client.19320 fetch_fsmap learned FSMap version 35 2024-04-16T17:20:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.521+0000 7f090cba3740 1 -- 172.21.15.18:0/3443519883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cfa11e71f0 con 0x55cfa114da80 2024-04-16T17:20:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.522+0000 7f08ecc00640 1 -- 172.21.15.18:0/3443519883 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f40c2730 con 0x55cfa114da80 2024-04-16T17:20:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.522+0000 7f08ecc00640 1 -- 172.21.15.18:0/3443519883 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f08fc0247b0 con 0x55cfa114da80 2024-04-16T17:20:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.522+0000 7f090cba3740 10 client.19320 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:20:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.522+0000 7f090cba3740 10 client.19320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:20:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.522+0000 7f090cba3740 20 client.19320 populate_metadata read hostname 'smithi018' 2024-04-16T17:20:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.522+0000 7f090cba3740 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfa11e7960 0x55cfa11e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:20:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.522+0000 7f090cba3740 4 client.19320 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:20:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.522+0000 7f090cba3740 1 -- 172.21.15.18:0/3443519883 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfa10ba010 con 0x55cfa11e7960 2024-04-16T17:20:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.523+0000 7f0902000640 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfa11e7960 0x55cfa11e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:20:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.523+0000 7f0902000640 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfa11e7960 0x55cfa11e9d80 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1f44b0a133e072 server_cookie=19ca60f2041c380f in_seq=0 out_seq=0 2024-04-16T17:20:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:46.523+0000 7f08ecc00640 10 client.19320 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:46.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:39 smithi018 ceph-mon[31776]: pgmap v507: 129 pgs: 129 active+clean; 371 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-04-16T17:20:46.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:41 smithi018 ceph-mon[31776]: pgmap v508: 129 pgs: 129 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 459 op/s 2024-04-16T17:20:46.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:42 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2965237173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:20:46.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:43 smithi018 ceph-mon[31776]: pgmap v509: 129 pgs: 129 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-04-16T17:20:46.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:45 smithi018 ceph-mon[31776]: pgmap v510: 129 pgs: 129 active+clean; 382 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2024-04-16T17:20:46.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:45 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:20:46.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:45 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:46.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:45 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:20:46.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:45 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:46 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:20:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:46 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:46 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:20:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:46 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:20:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:46 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:47.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:46 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:20:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:46 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:46 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:20:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:46 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:20:47.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:46 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:47.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:46 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:20:47.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:46 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:47.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:46 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:20:47.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:46 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:20:47.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:46 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:47.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 46273 13.11 MB/sec execute 59 sec latency 765.450 ms 2024-04-16T17:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.148+0000 7f08ecc00640 1 -- 172.21.15.18:0/3443519883 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cfa10ba010 con 0x55cfa11e7960 2024-04-16T17:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.148+0000 7f08ecc00640 10 client.19320 handle_command_reply: tid=0 2024-04-16T17:20:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.148+0000 7f090cba3740 1 client.19320 shutdown 2024-04-16T17:20:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f090cba3740 1 -- 172.21.15.18:0/3443519883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa114da80 msgr2=0x55cfa1170bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f090cba3740 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa114da80 0x55cfa1170bd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f08fc024730 tx=0x7f08fc009770 comp rx=0 tx=0).stop 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f090cba3740 1 -- 172.21.15.18:0/3443519883 shutdown_connections 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f090cba3740 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cfa11e7960 0x55cfa11e9d80 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f090cba3740 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfa115f810 0x55cfa11d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f090cba3740 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfa114e7f0 0x55cfa1171110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f090cba3740 1 --2- 172.21.15.18:0/3443519883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa114da80 0x55cfa1170bd0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f090cba3740 1 -- 172.21.15.18:0/3443519883 >> 172.21.15.18:0/3443519883 conn(0x55cfa10e85d0 msgr2=0x55cfa116b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.149+0000 7f08ecc00640 0 client.19320 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.150+0000 7f090cba3740 1 -- 172.21.15.18:0/3443519883 shutdown_connections 2024-04-16T17:20:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.150+0000 7f090cba3740 1 -- 172.21.15.18:0/3443519883 wait complete. 2024-04-16T17:20:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.151+0000 7f090cba3740 20 client.19320 trim_cache size 0 max 16384 2024-04-16T17:20:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:20:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:20:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:20:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:20:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 -- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08e80a3d70 msgr2=0x7f08e80a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08e80a3d70 0x7f08e80a6230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f08f0005fd0 tx=0x7f08f000a730 comp rx=0 tx=0).stop 2024-04-16T17:20:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 -- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0904103900 msgr2=0x7f09041a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:20:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0904103900 0x7f09041a0bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f08f4000c00 tx=0x7f08f40043b0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 -- 172.21.15.18:0/2499707548 shutdown_connections 2024-04-16T17:20:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08e80a3d70 0x7f08e80a6230 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f090410c400 0x7f090419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0904103900 0x7f09041a0bd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 --2- 172.21.15.18:0/2499707548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0904102fe0 0x7f09041a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:20:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.155+0000 7f0903e00640 1 -- 172.21.15.18:0/2499707548 >> 172.21.15.18:0/2499707548 conn(0x7f09040fc7e0 msgr2=0x7f09040fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:20:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.156+0000 7f0903e00640 1 -- 172.21.15.18:0/2499707548 shutdown_connections 2024-04-16T17:20:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:20:48.156+0000 7f0903e00640 1 -- 172.21.15.18:0/2499707548 wait complete. 2024-04-16T17:20:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:47 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:47 smithi022 ceph-mon[36023]: pgmap v511: 129 pgs: 129 active+clean; 393 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 357 op/s 2024-04-16T17:20:48.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:47 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:48.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:47 smithi043 ceph-mon[36901]: pgmap v511: 129 pgs: 129 active+clean; 393 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 357 op/s 2024-04-16T17:20:48.484 INFO:tasks.workunit.client.0.smithi018.stdout: 10 46411 12.97 MB/sec execute 60 sec latency 766.838 ms 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3101-g9b42bfbe/rpm/el8/BUILD/ceph-19.0.0-3101-g9b42bfbe/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f0a850cb700 time 2024-04-16T17:20:48.241779+0000 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3101-g9b42bfbe/rpm/el8/BUILD/ceph-19.0.0-3101-g9b42bfbe/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0a972e63cd] 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d1593) [0x7f0a972e6593] 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 3: (MDLog::_replay_thread()+0xd8) [0x55c320f77ff8] 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 4: (MDLog::ReplayThread::entry()+0x11) [0x55c320f81c21] 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f0a960641ca] 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 6: clone() 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.241+0000 7f0a850cb700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3101-g9b42bfbe/rpm/el8/BUILD/ceph-19.0.0-3101-g9b42bfbe/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f0a850cb700 time 2024-04-16T17:20:48.241779+0000 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3101-g9b42bfbe/rpm/el8/BUILD/ceph-19.0.0-3101-g9b42bfbe/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-04-16T17:20:48.674 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0a972e63cd] 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d1593) [0x7f0a972e6593] 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 3: (MDLog::_replay_thread()+0xd8) [0x55c320f77ff8] 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 4: (MDLog::ReplayThread::entry()+0x11) [0x55c320f81c21] 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f0a960641ca] 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 6: clone() 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: *** Caught signal (Aborted) ** 2024-04-16T17:20:48.675 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: in thread 7f0a850cb700 thread_name:md_log_replay 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0a9606ed20] 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2: gsignal() 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 3: abort() 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0a972e6427] 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d1593) [0x7f0a972e6593] 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 6: (MDLog::_replay_thread()+0xd8) [0x55c320f77ff8] 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55c320f81c21] 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f0a960641ca] 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 9: clone() 2024-04-16T17:20:48.676 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.242+0000 7f0a850cb700 -1 *** Caught signal (Aborted) ** 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: in thread 7f0a850cb700 thread_name:md_log_replay 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0a9606ed20] 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2: gsignal() 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 3: abort() 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0a972e6427] 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d1593) [0x7f0a972e6593] 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 6: (MDLog::_replay_thread()+0xd8) [0x55c320f77ff8] 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55c320f81c21] 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f0a960641ca] 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 9: clone() 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.677 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: -1> 2024-04-16T17:20:48.241+0000 7f0a850cb700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3101-g9b42bfbe/rpm/el8/BUILD/ceph-19.0.0-3101-g9b42bfbe/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f0a850cb700 time 2024-04-16T17:20:48.241779+0000 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3101-g9b42bfbe/rpm/el8/BUILD/ceph-19.0.0-3101-g9b42bfbe/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0a972e63cd] 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d1593) [0x7f0a972e6593] 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 3: (MDLog::_replay_thread()+0xd8) [0x55c320f77ff8] 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 4: (MDLog::ReplayThread::entry()+0x11) [0x55c320f81c21] 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f0a960641ca] 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 6: clone() 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 0> 2024-04-16T17:20:48.242+0000 7f0a850cb700 -1 *** Caught signal (Aborted) ** 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: in thread 7f0a850cb700 thread_name:md_log_replay 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T17:20:48.678 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0a9606ed20] 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2: gsignal() 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 3: abort() 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0a972e6427] 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d1593) [0x7f0a972e6593] 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 6: (MDLog::_replay_thread()+0xd8) [0x55c320f77ff8] 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55c320f81c21] 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f0a960641ca] 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 9: clone() 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: -1> 2024-04-16T17:20:48.241+0000 7f0a850cb700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3101-g9b42bfbe/rpm/el8/BUILD/ceph-19.0.0-3101-g9b42bfbe/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f0a850cb700 time 2024-04-16T17:20:48.241779+0000 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3101-g9b42bfbe/rpm/el8/BUILD/ceph-19.0.0-3101-g9b42bfbe/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0a972e63cd] 2024-04-16T17:20:48.679 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d1593) [0x7f0a972e6593] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 3: (MDLog::_replay_thread()+0xd8) [0x55c320f77ff8] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 4: (MDLog::ReplayThread::entry()+0x11) [0x55c320f81c21] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f0a960641ca] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 6: clone() 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 0> 2024-04-16T17:20:48.242+0000 7f0a850cb700 -1 *** Caught signal (Aborted) ** 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: in thread 7f0a850cb700 thread_name:md_log_replay 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0a9606ed20] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2: gsignal() 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 3: abort() 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0a972e6427] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d1593) [0x7f0a972e6593] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 6: (MDLog::_replay_thread()+0xd8) [0x55c320f77ff8] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55c320f81c21] 2024-04-16T17:20:48.680 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f0a960641ca] 2024-04-16T17:20:48.681 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 9: clone() 2024-04-16T17:20:48.681 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T17:20:48.681 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[66061]: 2024-04-16T17:20:48.681 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:48 smithi022 podman[73220]: 2024-04-16 17:20:48.417548886 +0000 UTC m=+0.058179580 container died f0ff3c200e564d60263389e33eb24fb279aaea899ef59d205ce763c354ed1f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-16T17:20:49.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:47 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:20:49.317 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:47 smithi018 ceph-mon[31776]: pgmap v511: 129 pgs: 129 active+clean; 393 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 357 op/s 2024-04-16T17:20:49.484 INFO:tasks.workunit.client.0.smithi018.stdout: 10 46658 12.96 MB/sec execute 61 sec latency 278.504 ms 2024-04-16T17:20:50.165 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:49 smithi022 ceph-mon[36023]: pgmap v512: 129 pgs: 129 active+clean; 390 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-04-16T17:20:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:49 smithi043 ceph-mon[36901]: pgmap v512: 129 pgs: 129 active+clean; 390 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-04-16T17:20:50.423 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:50 smithi022 podman[73220]: 2024-04-16 17:20:50.166740994 +0000 UTC m=+1.807371666 container remove f0ff3c200e564d60263389e33eb24fb279aaea899ef59d205ce763c354ed1f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:20:50.423 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:50 smithi022 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-04-16T17:20:50.484 INFO:tasks.workunit.client.0.smithi018.stdout: 10 46931 12.93 MB/sec execute 62 sec latency 215.713 ms 2024-04-16T17:20:50.923 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:50 smithi022 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-16T17:20:50.923 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:20:50 smithi022 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.h.service: Consumed 33.426s CPU time. 2024-04-16T17:20:51.484 INFO:tasks.workunit.client.0.smithi018.stdout: 10 47297 13.03 MB/sec execute 63 sec latency 193.451 ms 2024-04-16T17:20:52.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:51 smithi022 ceph-mon[36023]: pgmap v513: 129 pgs: 129 active+clean; 377 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 566 op/s 2024-04-16T17:20:52.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:51 smithi043 ceph-mon[36901]: pgmap v513: 129 pgs: 129 active+clean; 377 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 566 op/s 2024-04-16T17:20:52.484 INFO:tasks.workunit.client.0.smithi018.stdout: 10 47750 13.20 MB/sec execute 64 sec latency 48.066 ms 2024-04-16T17:20:53.484 INFO:tasks.workunit.client.0.smithi018.stdout: 10 48022 13.23 MB/sec execute 65 sec latency 476.347 ms 2024-04-16T17:20:54.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:53 smithi022 ceph-mon[36023]: pgmap v514: 129 pgs: 129 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 598 op/s 2024-04-16T17:20:54.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:53 smithi043 ceph-mon[36901]: pgmap v514: 129 pgs: 129 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 598 op/s 2024-04-16T17:20:54.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 48386 13.31 MB/sec execute 66 sec latency 494.940 ms 2024-04-16T17:20:55.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 48740 13.52 MB/sec execute 67 sec latency 183.525 ms 2024-04-16T17:20:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:55 smithi022 ceph-mon[36023]: pgmap v515: 129 pgs: 129 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 568 op/s 2024-04-16T17:20:56.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:55 smithi043 ceph-mon[36901]: pgmap v515: 129 pgs: 129 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 568 op/s 2024-04-16T17:20:56.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 49081 13.63 MB/sec execute 68 sec latency 202.452 ms 2024-04-16T17:20:57.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 49455 13.69 MB/sec execute 69 sec latency 186.575 ms 2024-04-16T17:20:58.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:57 smithi022 ceph-mon[36023]: pgmap v516: 129 pgs: 129 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 670 op/s 2024-04-16T17:20:58.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:57 smithi043 ceph-mon[36901]: pgmap v516: 129 pgs: 129 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 670 op/s 2024-04-16T17:20:58.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 49658 13.58 MB/sec execute 70 sec latency 218.945 ms 2024-04-16T17:20:59.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 49915 13.60 MB/sec execute 71 sec latency 126.570 ms 2024-04-16T17:21:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:59 smithi022 ceph-mon[36023]: pgmap v517: 129 pgs: 129 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 571 op/s 2024-04-16T17:21:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:59 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:21:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:59 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:21:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:20:59 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:21:00.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:59 smithi043 ceph-mon[36901]: pgmap v517: 129 pgs: 129 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 571 op/s 2024-04-16T17:21:00.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:59 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:21:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:59 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:21:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:20:59 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:21:00.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50201 13.53 MB/sec execute 72 sec latency 108.633 ms 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:49 smithi018 ceph-mon[31776]: pgmap v512: 129 pgs: 129 active+clean; 390 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:51 smithi018 ceph-mon[31776]: pgmap v513: 129 pgs: 129 active+clean; 377 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 566 op/s 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:53 smithi018 ceph-mon[31776]: pgmap v514: 129 pgs: 129 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 598 op/s 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:55 smithi018 ceph-mon[31776]: pgmap v515: 129 pgs: 129 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 568 op/s 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:57 smithi018 ceph-mon[31776]: pgmap v516: 129 pgs: 129 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 670 op/s 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:59 smithi018 ceph-mon[31776]: pgmap v517: 129 pgs: 129 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 571 op/s 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:59 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:59 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:21:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:20:59 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:21:01.173 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:00 smithi022 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-04-16T17:21:01.173 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:00 smithi022 systemd[1]: Stopped Ceph mds.h for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:21:01.173 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:00 smithi022 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.h.service: Consumed 33.426s CPU time. 2024-04-16T17:21:01.173 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:00 smithi022 systemd[1]: Starting Ceph mds.h for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 -- 172.21.15.18:0/480168136 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541015a0 msgr2=0x7f125410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 --2- 172.21.15.18:0/480168136 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541015a0 0x7f125410e1d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f123c009900 tx=0x7f123c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 -- 172.21.15.18:0/480168136 shutdown_connections 2024-04-16T17:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 --2- 172.21.15.18:0/480168136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f125410e7a0 0x7f1254110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 --2- 172.21.15.18:0/480168136 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541015a0 0x7f125410e1d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 --2- 172.21.15.18:0/480168136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254100bf0 0x7f1254100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 -- 172.21.15.18:0/480168136 >> 172.21.15.18:0/480168136 conn(0x7f12540fc7e0 msgr2=0x7f12540fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 -- 172.21.15.18:0/480168136 shutdown_connections 2024-04-16T17:21:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.309+0000 7f1253200640 1 -- 172.21.15.18:0/480168136 wait complete. 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.310+0000 7f1253200640 1 Processor -- start 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.310+0000 7f1253200640 1 -- start start 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.310+0000 7f1253200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254100bf0 0x7f12541a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.310+0000 7f1253200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12541015a0 0x7f12541a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.310+0000 7f1253200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f125410e7a0 0x7f125419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.310+0000 7f1253200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1254114490 con 0x7f125410e7a0 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.310+0000 7f1253200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1254114310 con 0x7f12541015a0 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.310+0000 7f1253200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1254114610 con 0x7f1254100bf0 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f125410e7a0 0x7f125419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f125410e7a0 0x7f125419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42604/0 (socket says 172.21.15.18:42604) 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 -- 172.21.15.18:0/1669246615 learned_addr learned my addr 172.21.15.18:0/1669246615 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1251e00640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254100bf0 0x7f12541a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1249400640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12541015a0 0x7f12541a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 -- 172.21.15.18:0/1669246615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254100bf0 msgr2=0x7f12541a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254100bf0 0x7f12541a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 -- 172.21.15.18:0/1669246615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12541015a0 msgr2=0x7f12541a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12541015a0 0x7f12541a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 -- 172.21.15.18:0/1669246615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125419afa0 con 0x7f125410e7a0 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.311+0000 7f1252800640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f125410e7a0 0x7f125419a7d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f124c00dac0 tx=0x7f124c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.312+0000 7f124b400640 1 -- 172.21.15.18:0/1669246615 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c017030 con 0x7f125410e7a0 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.312+0000 7f124b400640 1 -- 172.21.15.18:0/1669246615 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f124c003e50 con 0x7f125410e7a0 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.312+0000 7f1253200640 1 -- 172.21.15.18:0/1669246615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125419b290 con 0x7f125410e7a0 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.312+0000 7f124b400640 1 -- 172.21.15.18:0/1669246615 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c010460 con 0x7f125410e7a0 2024-04-16T17:21:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.312+0000 7f1253200640 1 -- 172.21.15.18:0/1669246615 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12541a74b0 con 0x7f125410e7a0 2024-04-16T17:21:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.316+0000 7f125bd70740 1 Processor -- start 2024-04-16T17:21:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.316+0000 7f124b400640 1 -- 172.21.15.18:0/1669246615 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f124c01a070 con 0x7f125410e7a0 2024-04-16T17:21:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.316+0000 7f125bd70740 1 -- start start 2024-04-16T17:21:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f125bd70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c9536c7f0 0x564c9536b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f125bd70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c9536bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f125bd70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c95375000 0x564c953773f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f125bd70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564c952d0780 con 0x564c95375000 2024-04-16T17:21:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f125bd70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564c95276f40 con 0x564c9536c7f0 2024-04-16T17:21:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f125bd70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c952cc740 con 0x564c9536ba80 2024-04-16T17:21:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1249400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c95375000 0x564c953773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1249400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c95375000 0x564c953773f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42618/0 (socket says 172.21.15.18:42618) 2024-04-16T17:21:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1249400640 1 -- 172.21.15.18:0/3750774491 learned_addr learned my addr 172.21.15.18:0/3750774491 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1251e00640 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c9536bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1249400640 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c9536c7f0 0x564c9536b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1251e00640 1 -- 172.21.15.18:0/3750774491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c9536c7f0 msgr2=0x564c9536b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1251e00640 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c9536c7f0 0x564c9536b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1251e00640 1 -- 172.21.15.18:0/3750774491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c95375000 msgr2=0x564c953773f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.317+0000 7f1251e00640 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c95375000 0x564c953773f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.318+0000 7f1251e00640 1 -- 172.21.15.18:0/3750774491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c94fe3210 con 0x564c9536ba80 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.318+0000 7f124b400640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12380a3d00 0x7f12380a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.318+0000 7f1251e00640 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c9536bf00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1244009c00 tx=0x7f1244011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68212e0a94286197 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.318+0000 7f124b400640 1 -- 172.21.15.18:0/1669246615 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f124c0d08f0 con 0x7f125410e7a0 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.318+0000 7f1249400640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12380a3d00 0x7f12380a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.318+0000 7f1243400640 1 -- 172.21.15.18:0/3750774491 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244004050 con 0x564c9536ba80 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.318+0000 7f1243400640 1 -- 172.21.15.18:0/3750774491 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1244005500 con 0x564c9536ba80 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.318+0000 7f1243400640 1 -- 172.21.15.18:0/3750774491 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244004da0 con 0x564c9536ba80 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 -- 172.21.15.18:0/3750774491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 msgr2=0x564c9536bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f1249400640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12380a3d00 0x7f12380a61c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x564c95367900 tx=0x7f123c005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c9536bf00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1244009c00 tx=0x7f1244011810 comp rx=0 tx=0).stop 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 -- 172.21.15.18:0/3750774491 shutdown_connections 2024-04-16T17:21:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c95375000 0x564c953773f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c9536bf00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 --2- 172.21.15.18:0/3750774491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c9536c7f0 0x564c9536b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 -- 172.21.15.18:0/3750774491 >> 172.21.15.18:0/3750774491 conn(0x564c953065d0 msgr2=0x564c95366770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 -- 172.21.15.18:0/3750774491 shutdown_connections 2024-04-16T17:21:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 -- 172.21.15.18:0/3750774491 wait complete. 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.319+0000 7f125bd70740 1 -- start start 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f125bd70740 1 Processor -- start 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f125bd70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536ba80 0x564c95374a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f125bd70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536c7f0 0x564c95371e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f125bd70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95372540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f125bd70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564c95377d30 con 0x564c9536ba80 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f125bd70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564c9528a390 con 0x564c95375000 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f125bd70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c95377eb0 con 0x564c9536c7f0 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f125bd70740 10 client.0 _ll_register_callbacks cb 0x564c95273f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f1251e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536ba80 0x564c95374a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.320+0000 7f1249400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536c7f0 0x564c95371e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1252800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95372540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1249400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536c7f0 0x564c95371e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36544/0 (socket says 172.21.15.18:36544) 2024-04-16T17:21:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1249400640 1 -- 172.21.15.18:0/3143335168 learned_addr learned my addr 172.21.15.18:0/3143335168 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1252800640 1 -- 172.21.15.18:0/3143335168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536c7f0 msgr2=0x564c95371e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1252800640 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536c7f0 0x564c95371e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1252800640 1 -- 172.21.15.18:0/3143335168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536ba80 msgr2=0x564c95374a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1252800640 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536ba80 0x564c95374a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1252800640 1 -- 172.21.15.18:0/3143335168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c950af620 con 0x564c95375000 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1252800640 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95372540 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f124c0cdc40 tx=0x7f124c09bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ff51d2ecaf587b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1237e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1237e00640 1 -- 172.21.15.18:0/3143335168 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0b5060 con 0x564c95375000 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f1237e00640 1 -- 172.21.15.18:0/3143335168 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f124c098b00 con 0x564c95375000 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.321+0000 7f125bd70740 1 -- 172.21.15.18:0/3143335168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1244005500 con 0x564c95375000 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f1237e00640 1 -- 172.21.15.18:0/3143335168 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f124c0a5100 con 0x564c95375000 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f125bd70740 10 client.29921 fetch_fsmap learned FSMap version 35 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f125bd70740 1 -- 172.21.15.18:0/3143335168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564c95383e30 con 0x564c95375000 2024-04-16T17:21:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f1237e00640 1 -- 172.21.15.18:0/3143335168 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1244005500 con 0x564c95375000 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f1237e00640 1 -- 172.21.15.18:0/3143335168 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f124c024060 con 0x564c95375000 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f125bd70740 10 client.29921 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f125bd70740 10 client.29921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f125bd70740 20 client.29921 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f125bd70740 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c953852a0 0x564c95387690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f125bd70740 4 client.29921 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f125bd70740 1 -- 172.21.15.18:0/3143335168 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c952de1a0 con 0x564c953852a0 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.322+0000 7f1251e00640 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c953852a0 0x564c95387690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.323+0000 7f1251e00640 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c953852a0 0x564c95387690 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723de1c1306ac237 server_cookie=6685666b65fa4efb in_seq=0 out_seq=0 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.323+0000 7f1237e00640 10 client.29921 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f1237e00640 1 -- 172.21.15.18:0/3143335168 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c952de1a0 con 0x564c953852a0 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f1237e00640 10 client.29921 handle_command_reply: tid=0 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 client.29921 shutdown 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 -- 172.21.15.18:0/3143335168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 msgr2=0x564c95372540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95372540 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f124c0cdc40 tx=0x7f124c09bee0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 -- 172.21.15.18:0/3143335168 shutdown_connections 2024-04-16T17:21:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c953852a0 0x564c95387690 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95372540 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f1237e00640 0 client.29921 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536c7f0 0x564c95371e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 --2- 172.21.15.18:0/3143335168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536ba80 0x564c95374a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 -- 172.21.15.18:0/3143335168 >> 172.21.15.18:0/3143335168 conn(0x564c953065d0 msgr2=0x564c953684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.324+0000 7f125bd70740 1 -- 172.21.15.18:0/3143335168 shutdown_connections 2024-04-16T17:21:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.325+0000 7f125bd70740 1 -- 172.21.15.18:0/3143335168 wait complete. 2024-04-16T17:21:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.325+0000 7f125bd70740 20 client.29921 trim_cache size 0 max 16384 2024-04-16T17:21:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.340+0000 7f125bd70740 1 Processor -- start 2024-04-16T17:21:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.340+0000 7f125bd70740 1 -- start start 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f125bd70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95380c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f125bd70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c95370de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f125bd70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c953763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f125bd70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f12541b3f30 con 0x564c9536c7f0 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f125bd70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12541b1800 con 0x564c95375000 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f125bd70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12541af670 con 0x564c9536ba80 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1251e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95380c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1251e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95380c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36556/0 (socket says 172.21.15.18:36556) 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1251e00640 1 -- 172.21.15.18:0/252490635 learned_addr learned my addr 172.21.15.18:0/252490635 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1252800640 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c953763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1249400640 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c95370de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1251e00640 1 -- 172.21.15.18:0/252490635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 msgr2=0x564c953763f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1251e00640 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c953763f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1251e00640 1 -- 172.21.15.18:0/252490635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 msgr2=0x564c95370de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1251e00640 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c95370de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1251e00640 1 -- 172.21.15.18:0/252490635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c9536b8b0 con 0x564c9536ba80 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.341+0000 7f1249400640 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c95370de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.342+0000 7f1251e00640 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95380c60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1244009c00 tx=0x7f12440045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8318344147e87c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.342+0000 7f1242a00640 1 -- 172.21.15.18:0/252490635 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244032020 con 0x564c9536ba80 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.342+0000 7f1242a00640 1 -- 172.21.15.18:0/252490635 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f124402ebd0 con 0x564c9536ba80 2024-04-16T17:21:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.342+0000 7f1242a00640 1 -- 172.21.15.18:0/252490635 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124402e8d0 con 0x564c9536ba80 2024-04-16T17:21:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.342+0000 7f125bd70740 1 -- 172.21.15.18:0/252490635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 msgr2=0x564c95380c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.342+0000 7f125bd70740 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95380c60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1244009c00 tx=0x7f12440045e0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 -- 172.21.15.18:0/252490635 shutdown_connections 2024-04-16T17:21:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c953763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c95370de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 --2- 172.21.15.18:0/252490635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95380c60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 -- 172.21.15.18:0/252490635 >> 172.21.15.18:0/252490635 conn(0x564c953065d0 msgr2=0x564c953706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 -- 172.21.15.18:0/252490635 shutdown_connections 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 -- 172.21.15.18:0/252490635 wait complete. 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 -- start start 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.343+0000 7f125bd70740 1 Processor -- start 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f125bd70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95373c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f125bd70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c953741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f125bd70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95374700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f125bd70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f12541afcc0 con 0x564c9536c7f0 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f125bd70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12541b1130 con 0x564c95375000 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f125bd70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12541b0390 con 0x564c9536ba80 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f125bd70740 10 client.0 _ll_register_callbacks cb 0x7f124c098050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1249400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c953741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1249400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c953741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42654/0 (socket says 172.21.15.18:42654) 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1249400640 1 -- 172.21.15.18:0/3886850133 learned_addr learned my addr 172.21.15.18:0/3886850133 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1251e00640 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95373c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1252800640 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95374700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1249400640 1 -- 172.21.15.18:0/3886850133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 msgr2=0x564c95373c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1249400640 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95373c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1249400640 1 -- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 msgr2=0x564c95374700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1249400640 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95374700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1249400640 1 -- 172.21.15.18:0/3886850133 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c95387a00 con 0x564c9536c7f0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.344+0000 7f1252800640 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95374700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.345+0000 7f1249400640 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c953741c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f123c009900 tx=0x7f123c02df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6225a0fca0436b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.345+0000 7f1237e00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.345+0000 7f1237e00640 1 -- 172.21.15.18:0/3886850133 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c05d060 con 0x564c9536c7f0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.345+0000 7f1237e00640 1 -- 172.21.15.18:0/3886850133 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9510+0+0 (secure 0 0 0) 0x7f123c0050d0 con 0x564c9536c7f0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.345+0000 7f125bd70740 1 -- 172.21.15.18:0/3886850133 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f124402ebd0 con 0x564c9536c7f0 2024-04-16T17:21:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.345+0000 7f1237e00640 1 -- 172.21.15.18:0/3886850133 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f123c064050 con 0x564c9536c7f0 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.345+0000 7f125bd70740 10 client.19338 fetch_fsmap learned FSMap version 35 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.345+0000 7f125bd70740 1 -- 172.21.15.18:0/3886850133 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564c953ebba0 con 0x564c9536c7f0 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f1237e00640 1 -- 172.21.15.18:0/3886850133 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f124402ebd0 con 0x564c9536c7f0 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f1237e00640 1 -- 172.21.15.18:0/3886850133 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f123c04b040 con 0x564c9536c7f0 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f125bd70740 10 client.19338 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f125bd70740 10 client.19338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f125bd70740 20 client.19338 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f125bd70740 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c953ec350 0x564c953ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f125bd70740 4 client.19338 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f125bd70740 1 -- 172.21.15.18:0/3886850133 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c952d8010 con 0x564c953ec350 2024-04-16T17:21:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.346+0000 7f1251e00640 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c953ec350 0x564c953ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.347+0000 7f1251e00640 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c953ec350 0x564c953ee770 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db30a494404b01c0 server_cookie=dfee539fafb40c6 in_seq=0 out_seq=0 2024-04-16T17:21:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:01.347+0000 7f1237e00640 10 client.19338 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:01.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50201 13.34 MB/sec execute 73 sec latency 1057.034 ms 2024-04-16T17:21:02.104 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:01 smithi018 ceph-mon[31776]: pgmap v518: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 644 op/s 2024-04-16T17:21:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f1237e00640 1 -- 172.21.15.18:0/3886850133 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c952d8010 con 0x564c953ec350 2024-04-16T17:21:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f1237e00640 10 client.19338 handle_command_reply: tid=0 2024-04-16T17:21:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 client.19338 shutdown 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 -- 172.21.15.18:0/3886850133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 msgr2=0x564c953741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c953741c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f123c009900 tx=0x7f123c02df90 comp rx=0 tx=0).stop 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 -- 172.21.15.18:0/3886850133 shutdown_connections 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564c953ec350 0x564c953ee770 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c95375000 0x564c95374700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564c9536c7f0 0x564c953741c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 --2- 172.21.15.18:0/3886850133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c9536ba80 0x564c95373c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f125bd70740 1 -- 172.21.15.18:0/3886850133 >> 172.21.15.18:0/3886850133 conn(0x564c953065d0 msgr2=0x564c95370f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.150+0000 7f1237e00640 0 client.19338 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.151+0000 7f125bd70740 1 -- 172.21.15.18:0/3886850133 shutdown_connections 2024-04-16T17:21:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.151+0000 7f125bd70740 1 -- 172.21.15.18:0/3886850133 wait complete. 2024-04-16T17:21:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.151+0000 7f125bd70740 20 client.19338 trim_cache size 0 max 16384 2024-04-16T17:21:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:21:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:21:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:21:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:21:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 -- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12380a3d00 msgr2=0x7f12380a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12380a3d00 0x7f12380a61c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x564c95367900 tx=0x7f123c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 -- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f125410e7a0 msgr2=0x7f125419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f125410e7a0 0x7f125419a7d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f124c00dac0 tx=0x7f124c00df80 comp rx=0 tx=0).stop 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 -- 172.21.15.18:0/1669246615 shutdown_connections 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f12380a3d00 0x7f12380a61c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f125410e7a0 0x7f125419a7d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12541015a0 0x7f12541a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 --2- 172.21.15.18:0/1669246615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254100bf0 0x7f12541a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.153+0000 7f1253200640 1 -- 172.21.15.18:0/1669246615 >> 172.21.15.18:0/1669246615 conn(0x7f12540fc7e0 msgr2=0x7f12540febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.154+0000 7f1253200640 1 -- 172.21.15.18:0/1669246615 shutdown_connections 2024-04-16T17:21:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:02.154+0000 7f1253200640 1 -- 172.21.15.18:0/1669246615 wait complete. 2024-04-16T17:21:02.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:01 smithi022 ceph-mon[36023]: pgmap v518: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 644 op/s 2024-04-16T17:21:02.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:01 smithi043 ceph-mon[36901]: pgmap v518: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 644 op/s 2024-04-16T17:21:02.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50225 13.16 MB/sec execute 74 sec latency 1960.874 ms 2024-04-16T17:21:03.173 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:02 smithi022 podman[73317]: 2024-04-16 17:21:02.641512676 +0000 UTC m=+0.020852168 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:21:03.486 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50456 13.15 MB/sec execute 75 sec latency 314.880 ms 2024-04-16T17:21:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:03 smithi022 ceph-mon[36023]: pgmap v519: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 366 op/s 2024-04-16T17:21:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:03 smithi022 ceph-mon[36023]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-04-16T17:21:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:03 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-04-16T17:21:04.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:03 smithi043 ceph-mon[36901]: pgmap v519: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 366 op/s 2024-04-16T17:21:04.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:03 smithi043 ceph-mon[36901]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-04-16T17:21:04.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:03 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-04-16T17:21:04.487 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50540 13.09 MB/sec execute 76 sec latency 731.855 ms 2024-04-16T17:21:04.855 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:03 smithi018 ceph-mon[31776]: pgmap v519: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 366 op/s 2024-04-16T17:21:04.855 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:03 smithi018 ceph-mon[31776]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-04-16T17:21:04.855 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:03 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-04-16T17:21:05.173 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:04 smithi022 podman[73317]: 2024-04-16 17:21:04.693650473 +0000 UTC m=+2.072989950 container create 790615c7636ee89019078a3e2c4c1b679e84120a44ccc1d61a36fa9b47d13c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-16T17:21:05.487 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50700 13.07 MB/sec execute 77 sec latency 923.401 ms 2024-04-16T17:21:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:05 smithi022 ceph-mon[36023]: osdmap e122: 12 total, 12 up, 12 in 2024-04-16T17:21:05.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:05 smithi022 ceph-mon[36023]: pgmap v521: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-04-16T17:21:06.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:05 smithi043 ceph-mon[36901]: osdmap e122: 12 total, 12 up, 12 in 2024-04-16T17:21:06.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:05 smithi043 ceph-mon[36901]: pgmap v521: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-04-16T17:21:06.013 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:05 smithi018 ceph-mon[31776]: osdmap e122: 12 total, 12 up, 12 in 2024-04-16T17:21:06.013 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:05 smithi018 ceph-mon[31776]: pgmap v521: 129 pgs: 129 active+clean; 390 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-04-16T17:21:06.487 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50782 12.94 MB/sec execute 78 sec latency 1863.162 ms 2024-04-16T17:21:07.487 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50911 12.83 MB/sec execute 79 sec latency 2826.155 ms 2024-04-16T17:21:07.752 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:07 smithi022 ceph-mon[36023]: pgmap v522: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 386 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-04-16T17:21:08.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:07 smithi043 ceph-mon[36901]: pgmap v522: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 386 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-04-16T17:21:08.031 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:07 smithi018 ceph-mon[31776]: pgmap v522: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 386 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-04-16T17:21:08.487 INFO:tasks.workunit.client.0.smithi018.stdout: 10 50973 12.72 MB/sec execute 80 sec latency 3070.872 ms 2024-04-16T17:21:09.487 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51069 12.58 MB/sec execute 81 sec latency 2935.796 ms 2024-04-16T17:21:09.756 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:09 smithi022 podman[73317]: 2024-04-16 17:21:09.455213935 +0000 UTC m=+6.834553427 container init 790615c7636ee89019078a3e2c4c1b679e84120a44ccc1d61a36fa9b47d13c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-16T17:21:09.756 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:09 smithi022 podman[73317]: 2024-04-16 17:21:09.460211803 +0000 UTC m=+6.839551289 container start 790615c7636ee89019078a3e2c4c1b679e84120a44ccc1d61a36fa9b47d13c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-16T17:21:09.756 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:09 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[73346]: starting mds.h at 2024-04-16T17:21:10.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:09 smithi022 ceph-mon[36023]: pgmap v523: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 387 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-04-16T17:21:10.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:09 smithi043 ceph-mon[36901]: pgmap v523: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 387 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-04-16T17:21:10.487 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51192 12.54 MB/sec execute 82 sec latency 3827.775 ms 2024-04-16T17:21:10.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:10 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:21:10.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:10 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6836/456310898,v1:172.21.15.22:6837/456310898] up:boot 2024-04-16T17:21:10.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:10 smithi022 ceph-mon[36023]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:21:10.923 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:10 smithi022 bash[73317]: 790615c7636ee89019078a3e2c4c1b679e84120a44ccc1d61a36fa9b47d13c6e 2024-04-16T17:21:10.923 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:21:10 smithi022 systemd[1]: Started Ceph mds.h for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:21:11.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:10 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:21:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:10 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6836/456310898,v1:172.21.15.22:6837/456310898] up:boot 2024-04-16T17:21:11.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:10 smithi043 ceph-mon[36901]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:21:11.488 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51354 12.46 MB/sec execute 83 sec latency 3979.195 ms 2024-04-16T17:21:12.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:11 smithi022 ceph-mon[36023]: pgmap v524: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 540 op/s 2024-04-16T17:21:12.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:11 smithi043 ceph-mon[36901]: pgmap v524: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 540 op/s 2024-04-16T17:21:12.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51543 12.48 MB/sec execute 84 sec latency 3062.476 ms 2024-04-16T17:21:12.713 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:09 smithi018 ceph-mon[31776]: pgmap v523: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 387 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-04-16T17:21:12.714 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:10 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:21:12.714 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:10 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6836/456310898,v1:172.21.15.22:6837/456310898] up:boot 2024-04-16T17:21:12.714 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:10 smithi018 ceph-mon[31776]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:21:12.714 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:11 smithi018 ceph-mon[31776]: pgmap v524: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 540 op/s 2024-04-16T17:21:13.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51695 12.39 MB/sec execute 85 sec latency 2177.094 ms 2024-04-16T17:21:14.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:13 smithi022 ceph-mon[36023]: pgmap v525: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 390 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-04-16T17:21:14.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:13 smithi043 ceph-mon[36901]: pgmap v525: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 390 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-04-16T17:21:14.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51900 12.41 MB/sec execute 86 sec latency 2004.482 ms 2024-04-16T17:21:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.352+0000 7fa2aa200640 1 -- 172.21.15.18:0/2857423031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 msgr2=0x7fa2a41033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.352+0000 7fa2aa200640 1 --2- 172.21.15.18:0/2857423031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 0x7fa2a41033c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009ca0 tx=0x7fa29c02c180 comp rx=0 tx=0).stop 2024-04-16T17:21:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.354+0000 7fa2aa200640 1 -- 172.21.15.18:0/2857423031 shutdown_connections 2024-04-16T17:21:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.354+0000 7fa2aa200640 1 --2- 172.21.15.18:0/2857423031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a410c400 0x7fa2a410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.354+0000 7fa2aa200640 1 --2- 172.21.15.18:0/2857423031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4103900 0x7fa2a410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.354+0000 7fa2aa200640 1 --2- 172.21.15.18:0/2857423031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 0x7fa2a41033c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.354+0000 7fa2aa200640 1 -- 172.21.15.18:0/2857423031 >> 172.21.15.18:0/2857423031 conn(0x7fa2a40fc7e0 msgr2=0x7fa2a40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.354+0000 7fa2aa200640 1 -- 172.21.15.18:0/2857423031 shutdown_connections 2024-04-16T17:21:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.354+0000 7fa2aa200640 1 -- 172.21.15.18:0/2857423031 wait complete. 2024-04-16T17:21:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.356+0000 7fa2aa200640 1 Processor -- start 2024-04-16T17:21:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.357+0000 7fa2aa200640 1 -- start start 2024-04-16T17:21:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.357+0000 7fa2aa200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 0x7fa2a4100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.357+0000 7fa2aa200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4103900 0x7fa2a41010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.358+0000 7fa2a8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 0x7fa2a4100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.358+0000 7fa2a8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 0x7fa2a4100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48222/0 (socket says 172.21.15.18:48222) 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.358+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4103900 0x7fa2a41010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.358+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4103900 0x7fa2a41010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54280/0 (socket says 172.21.15.18:54280) 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2aa200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a410c400 0x7fa2a41015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2a8e00640 1 -- 172.21.15.18:0/1448702939 learned_addr learned my addr 172.21.15.18:0/1448702939 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4112210 con 0x7fa2a4103900 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4112090 con 0x7fa2a410c400 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4112390 con 0x7fa2a4102fe0 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2a9800640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a410c400 0x7fa2a41015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2a8e00640 1 -- 172.21.15.18:0/1448702939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a410c400 msgr2=0x7fa2a41015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a410c400 0x7fa2a41015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2a8e00640 1 -- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4103900 msgr2=0x7fa2a41010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4103900 0x7fa2a41010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.359+0000 7fa2a8e00640 1 -- 172.21.15.18:0/1448702939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a4072ae0 con 0x7fa2a4102fe0 2024-04-16T17:21:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.364+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 0x7fa2a4100b60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa29c02c710 tx=0x7fa29c035e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.364+0000 7fa2a1600640 1 -- 172.21.15.18:0/1448702939 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c039040 con 0x7fa2a4102fe0 2024-04-16T17:21:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.364+0000 7fa2a1600640 1 -- 172.21.15.18:0/1448702939 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c044070 con 0x7fa2a4102fe0 2024-04-16T17:21:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.366+0000 7fa2a1600640 1 -- 172.21.15.18:0/1448702939 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c032960 con 0x7fa2a4102fe0 2024-04-16T17:21:15.366 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.366+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a4072d70 con 0x7fa2a4102fe0 2024-04-16T17:21:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.367+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a4073210 con 0x7fa2a4102fe0 2024-04-16T17:21:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.374+0000 7fa2a1600640 1 -- 172.21.15.18:0/1448702939 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa29c032b00 con 0x7fa2a4102fe0 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.375+0000 7fa2ae47a740 1 Processor -- start 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.377+0000 7fa2a1600640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2880a3de0 0x7fa2880a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.377+0000 7fa2a1600640 1 -- 172.21.15.18:0/1448702939 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fa29c0f2ff0 con 0x7fa2a4102fe0 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.377+0000 7fa2ae47a740 1 -- start start 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2a9800640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2880a3de0 0x7fa2880a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc7a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 0x55d14bc7af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 0x55d14bc86400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d14bbdf780 con 0x55d14bc84010 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d14bb85f40 con 0x55d14bc7b800 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d14bbdb740 con 0x55d14bc7aa90 2024-04-16T17:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2a8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc7a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2a8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc7a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56044/0 (socket says 172.21.15.18:56044) 2024-04-16T17:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.378+0000 7fa2a8e00640 1 -- 172.21.15.18:0/2613973327 learned_addr learned my addr 172.21.15.18:0/2613973327 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.379+0000 7fa2a8e00640 1 -- 172.21.15.18:0/2613973327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 msgr2=0x55d14bc7af10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.379+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/2613973327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 0x55d14bc7af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.379+0000 7fa2a8e00640 1 -- 172.21.15.18:0/2613973327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 msgr2=0x55d14bc86400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.379+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/2613973327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 0x55d14bc86400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.379+0000 7fa2a8e00640 1 -- 172.21.15.18:0/2613973327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d14b8f2210 con 0x55d14bc7b800 2024-04-16T17:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.379+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/2613973327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc7a550 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0bb610 tx=0x7fa29c0bb010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1536dc2eb4880a00 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.379+0000 7fa29a000640 1 -- 172.21.15.18:0/2613973327 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0bec40 con 0x55d14bc7b800 2024-04-16T17:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.379+0000 7fa29a000640 1 -- 172.21.15.18:0/2613973327 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c0a9040 con 0x55d14bc7b800 2024-04-16T17:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.381+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2613973327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 msgr2=0x55d14bc7a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.381+0000 7fa2a9800640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2880a3de0 0x7fa2880a62a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa290009c00 tx=0x7fa290009740 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.381+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2613973327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc7a550 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0bb610 tx=0x7fa29c0bb010 comp rx=0 tx=0).stop 2024-04-16T17:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.382+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2613973327 shutdown_connections 2024-04-16T17:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.382+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2613973327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 0x55d14bc86400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.382+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2613973327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 0x55d14bc7af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.382+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2613973327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc7a550 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.382+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2613973327 >> 172.21.15.18:0/2613973327 conn(0x55d14bc155d0 msgr2=0x55d14bc75780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.383+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2613973327 shutdown_connections 2024-04-16T17:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.383+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2613973327 wait complete. 2024-04-16T17:21:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.384+0000 7fa2ae47a740 1 -- start start 2024-04-16T17:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.384+0000 7fa2ae47a740 1 Processor -- start 2024-04-16T17:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.385+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 0x55d14bc83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.385+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7b800 0x55d14bc80da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.385+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc84010 0x55d14bc813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.385+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d14bc86b00 con 0x55d14bc7b800 2024-04-16T17:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.385+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d14bb99390 con 0x55d14bc84010 2024-04-16T17:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.385+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d14bc86c80 con 0x55d14bc7aa90 2024-04-16T17:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.385+0000 7fa2ae47a740 10 client.0 _ll_register_callbacks cb 0x55d14bb82f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.386+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7b800 0x55d14bc80da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.386+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7b800 0x55d14bc80da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54288/0 (socket says 172.21.15.18:54288) 2024-04-16T17:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.386+0000 7fa2a3e00640 1 -- 172.21.15.18:0/2369486842 learned_addr learned my addr 172.21.15.18:0/2369486842 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.387+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 0x55d14bc83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.387+0000 7fa2a9800640 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc84010 0x55d14bc813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.387+0000 7fa2a3e00640 1 -- 172.21.15.18:0/2369486842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 msgr2=0x55d14bc83a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.387+0000 7fa2a3e00640 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 0x55d14bc83a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.387+0000 7fa2a3e00640 1 -- 172.21.15.18:0/2369486842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc84010 msgr2=0x55d14bc813b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.387+0000 7fa2a3e00640 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc84010 0x55d14bc813b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.387+0000 7fa2a3e00640 1 -- 172.21.15.18:0/2369486842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d14b9be620 con 0x55d14bc7b800 2024-04-16T17:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.389+0000 7fa2a3e00640 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7b800 0x55d14bc80da0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa29400f650 tx=0x7fa29400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73210e7d31f9e578 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.389+0000 7fa28ea00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:21:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.390+0000 7fa28ea00640 1 -- 172.21.15.18:0/2369486842 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294004220 con 0x55d14bc7b800 2024-04-16T17:21:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.391+0000 7fa28ea00640 1 -- 172.21.15.18:0/2369486842 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fa2940043c0 con 0x55d14bc7b800 2024-04-16T17:21:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.391+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2369486842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29c0a9040 con 0x55d14bc7b800 2024-04-16T17:21:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.392+0000 7fa28ea00640 1 -- 172.21.15.18:0/2369486842 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa29400d8a0 con 0x55d14bc7b800 2024-04-16T17:21:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.392+0000 7fa2ae47a740 10 client.19344 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.392+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2369486842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d14bc92bb0 con 0x55d14bc7b800 2024-04-16T17:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.393+0000 7fa28ea00640 1 -- 172.21.15.18:0/2369486842 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c0a9040 con 0x55d14bc7b800 2024-04-16T17:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.393+0000 7fa28ea00640 1 -- 172.21.15.18:0/2369486842 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fa2940047a0 con 0x55d14bc7b800 2024-04-16T17:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.393+0000 7fa2ae47a740 10 client.19344 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.393+0000 7fa2ae47a740 10 client.19344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.393+0000 7fa2ae47a740 20 client.19344 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.393+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d14bc94130 0x55d14bc96520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.393+0000 7fa2ae47a740 4 client.19344 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.393+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2369486842 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d14bbed1a0 con 0x55d14bc94130 2024-04-16T17:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.394+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d14bc94130 0x55d14bc96520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.395+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d14bc94130 0x55d14bc96520 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1e9b1a5637d204 server_cookie=42f4929e01ba86f3 in_seq=0 out_seq=0 2024-04-16T17:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.395+0000 7fa28ea00640 10 client.19344 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.396+0000 7fa28ea00640 1 -- 172.21.15.18:0/2369486842 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d14bbed1a0 con 0x55d14bc94130 2024-04-16T17:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.397+0000 7fa28ea00640 10 client.19344 handle_command_reply: tid=0 2024-04-16T17:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.397+0000 7fa2ae47a740 1 client.19344 shutdown 2024-04-16T17:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.397+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2369486842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7b800 msgr2=0x55d14bc80da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.397+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7b800 0x55d14bc80da0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa29400f650 tx=0x7fa29400fb10 comp rx=0 tx=0).stop 2024-04-16T17:21:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2369486842 shutdown_connections 2024-04-16T17:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d14bc94130 0x55d14bc96520 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc84010 0x55d14bc813b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7b800 0x55d14bc80da0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa28ea00640 0 client.19344 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2369486842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7aa90 0x55d14bc83a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2369486842 >> 172.21.15.18:0/2369486842 conn(0x55d14bc155d0 msgr2=0x55d14bc74a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2369486842 shutdown_connections 2024-04-16T17:21:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.399+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2369486842 wait complete. 2024-04-16T17:21:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.400+0000 7fa2ae47a740 20 client.19344 trim_cache size 0 max 16384 2024-04-16T17:21:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.438+0000 7fa2ae47a740 1 Processor -- start 2024-04-16T17:21:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.438+0000 7fa2ae47a740 1 -- start start 2024-04-16T17:21:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.438+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7aa90 0x55d14bc8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.438+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc9bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc84010 0x55d14bc9c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41c0c40 con 0x55d14bc7aa90 2024-04-16T17:21:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41be510 con 0x55d14bc7b800 2024-04-16T17:21:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41bc380 con 0x55d14bc84010 2024-04-16T17:21:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2a9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc84010 0x55d14bc9c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2a9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc84010 0x55d14bc9c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48244/0 (socket says 172.21.15.18:48244) 2024-04-16T17:21:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2a9800640 1 -- 172.21.15.18:0/2903874542 learned_addr learned my addr 172.21.15.18:0/2903874542 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2a8e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7aa90 0x55d14bc8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.439+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc9bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.440+0000 7fa2a9800640 1 -- 172.21.15.18:0/2903874542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 msgr2=0x55d14bc9bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.440+0000 7fa2a9800640 1 --2- 172.21.15.18:0/2903874542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc9bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.440+0000 7fa2a9800640 1 -- 172.21.15.18:0/2903874542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7aa90 msgr2=0x55d14bc8fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.440+0000 7fa2a9800640 1 --2- 172.21.15.18:0/2903874542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7aa90 0x55d14bc8fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.440+0000 7fa2a9800640 1 -- 172.21.15.18:0/2903874542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d14bc96850 con 0x55d14bc84010 2024-04-16T17:21:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.441+0000 7fa2a9800640 1 --2- 172.21.15.18:0/2903874542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc84010 0x55d14bc9c320 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa290013d90 tx=0x7fa290016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07bbc5c352b3efe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.443+0000 7fa299600640 1 -- 172.21.15.18:0/2903874542 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290004020 con 0x55d14bc84010 2024-04-16T17:21:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.443+0000 7fa299600640 1 -- 172.21.15.18:0/2903874542 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290025070 con 0x55d14bc84010 2024-04-16T17:21:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.443+0000 7fa299600640 1 -- 172.21.15.18:0/2903874542 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290021660 con 0x55d14bc84010 2024-04-16T17:21:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.444+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2903874542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc84010 msgr2=0x55d14bc9c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.444+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2903874542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc84010 0x55d14bc9c320 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa290013d90 tx=0x7fa290016000 comp rx=0 tx=0).stop 2024-04-16T17:21:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.444+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2903874542 shutdown_connections 2024-04-16T17:21:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.444+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2903874542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc84010 0x55d14bc9c320 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.444+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2903874542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7b800 0x55d14bc9bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.444+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/2903874542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc7aa90 0x55d14bc8fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.444+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2903874542 >> 172.21.15.18:0/2903874542 conn(0x55d14bc155d0 msgr2=0x55d14bc7c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.447+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2903874542 shutdown_connections 2024-04-16T17:21:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.448+0000 7fa2ae47a740 1 -- 172.21.15.18:0/2903874542 wait complete. 2024-04-16T17:21:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.448+0000 7fa2ae47a740 1 -- start start 2024-04-16T17:21:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.448+0000 7fa2ae47a740 1 Processor -- start 2024-04-16T17:21:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.449+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7aa90 0x55d14bc8bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.449+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7b800 0x55d14bc8c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.449+0000 7fa2ae47a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 0x55d14bc88ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.449+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41bc9d0 con 0x55d14bc84010 2024-04-16T17:21:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.449+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41bde40 con 0x55d14bc7aa90 2024-04-16T17:21:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.449+0000 7fa2ae47a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41bd0a0 con 0x55d14bc7b800 2024-04-16T17:21:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.449+0000 7fa2ae47a740 10 client.0 _ll_register_callbacks cb 0x7fa29400a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.450+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7b800 0x55d14bc8c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.450+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7b800 0x55d14bc8c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48254/0 (socket says 172.21.15.18:48254) 2024-04-16T17:21:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.450+0000 7fa2a3e00640 1 -- 172.21.15.18:0/3554189769 learned_addr learned my addr 172.21.15.18:0/3554189769 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.450+0000 7fa2a8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7aa90 0x55d14bc8bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.450+0000 7fa2a9800640 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 0x55d14bc88ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.450+0000 7fa2a3e00640 1 -- 172.21.15.18:0/3554189769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7aa90 msgr2=0x55d14bc8bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.450+0000 7fa2a3e00640 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7aa90 0x55d14bc8bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.451+0000 7fa2a3e00640 1 -- 172.21.15.18:0/3554189769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 msgr2=0x55d14bc88ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.451+0000 7fa2a3e00640 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 0x55d14bc88ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.451+0000 7fa2a3e00640 1 -- 172.21.15.18:0/3554189769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d14bc89560 con 0x55d14bc7b800 2024-04-16T17:21:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.452+0000 7fa2a3e00640 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7b800 0x55d14bc8c460 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x55d14bc82370 tx=0x7fa29400dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8495de3504f245 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.452+0000 7fa28ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.452+0000 7fa28ea00640 1 -- 172.21.15.18:0/3554189769 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294005260 con 0x55d14bc7b800 2024-04-16T17:21:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.452+0000 7fa2ae47a740 1 -- 172.21.15.18:0/3554189769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa290025070 con 0x55d14bc7b800 2024-04-16T17:21:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.452+0000 7fa28ea00640 1 -- 172.21.15.18:0/3554189769 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fa294005400 con 0x55d14bc7b800 2024-04-16T17:21:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.453+0000 7fa28ea00640 1 -- 172.21.15.18:0/3554189769 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa294039050 con 0x55d14bc7b800 2024-04-16T17:21:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.453+0000 7fa2ae47a740 10 client.29767 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.453+0000 7fa2ae47a740 1 -- 172.21.15.18:0/3554189769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d14bcfb060 con 0x55d14bc7b800 2024-04-16T17:21:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.454+0000 7fa28ea00640 1 -- 172.21.15.18:0/3554189769 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290025070 con 0x55d14bc7b800 2024-04-16T17:21:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.454+0000 7fa28ea00640 1 -- 172.21.15.18:0/3554189769 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fa294018920 con 0x55d14bc7b800 2024-04-16T17:21:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.454+0000 7fa2ae47a740 10 client.29767 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.454+0000 7fa2ae47a740 10 client.29767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.454+0000 7fa2ae47a740 20 client.29767 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.454+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d14bcfb7e0 0x55d14bcfdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.454+0000 7fa2ae47a740 4 client.29767 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:21:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.454+0000 7fa2ae47a740 1 -- 172.21.15.18:0/3554189769 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d14bbe7010 con 0x55d14bcfb7e0 2024-04-16T17:21:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.455+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d14bcfb7e0 0x55d14bcfdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.456+0000 7fa2a8e00640 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d14bcfb7e0 0x55d14bcfdc00 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774b7ff01e4c8dfc server_cookie=16cb4bab9e7cfb1b in_seq=0 out_seq=0 2024-04-16T17:21:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:15.456+0000 7fa28ea00640 10 client.29767 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:15.488 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:13 smithi018 ceph-mon[31776]: pgmap v525: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 390 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-04-16T17:21:15.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 51980 12.31 MB/sec execute 87 sec latency 3004.663 ms 2024-04-16T17:21:15.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:21:15.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:21:15.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:15 smithi018 ceph-mon[31776]: pgmap v526: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 480 op/s 2024-04-16T17:21:16.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 -- 172.21.15.18:0/2094524565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba40743d0 msgr2=0x7f1ba40747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/2094524565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba40743d0 0x7f1ba40747b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009ca0 tx=0x7f1b9c02c1b0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 -- 172.21.15.18:0/2094524565 shutdown_connections 2024-04-16T17:21:16.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/2094524565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ba410e870 0x7f1ba4110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/2094524565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4074cf0 0x7f1ba410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/2094524565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba40743d0 0x7f1ba40747b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 -- 172.21.15.18:0/2094524565 >> 172.21.15.18:0/2094524565 conn(0x7f1ba40fdac0 msgr2=0x7f1ba40ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 -- 172.21.15.18:0/2094524565 shutdown_connections 2024-04-16T17:21:16.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.114+0000 7f1ba9e00640 1 -- 172.21.15.18:0/2094524565 wait complete. 2024-04-16T17:21:16.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.116+0000 7f1ba9e00640 1 Processor -- start 2024-04-16T17:21:16.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.116+0000 7f1ba9e00640 1 -- start start 2024-04-16T17:21:16.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.117+0000 7f1ba9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba40743d0 0x7f1ba4107a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.117+0000 7f1ba9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ba4074cf0 0x7f1ba4107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.117+0000 7f1ba9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba410e870 0x7f1ba4102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.117+0000 7f1ba2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba40743d0 0x7f1ba4107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.118+0000 7f1ba2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba40743d0 0x7f1ba4107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48284/0 (socket says 172.21.15.18:48284) 2024-04-16T17:21:16.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.118+0000 7f1ba2a00640 1 -- 172.21.15.18:0/582024277 learned_addr learned my addr 172.21.15.18:0/582024277 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:16.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.118+0000 7f1ba3400640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba410e870 0x7f1ba4102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.118+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41146b0 con 0x7f1ba4074cf0 2024-04-16T17:21:16.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.118+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4114530 con 0x7f1ba410e870 2024-04-16T17:21:16.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.118+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4114830 con 0x7f1ba40743d0 2024-04-16T17:21:16.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.118+0000 7f1ba2000640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ba4074cf0 0x7f1ba4107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.119+0000 7f1ba3400640 1 -- 172.21.15.18:0/582024277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba40743d0 msgr2=0x7f1ba4107a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.119+0000 7f1ba3400640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba40743d0 0x7f1ba4107a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.119+0000 7f1ba3400640 1 -- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ba4074cf0 msgr2=0x7f1ba4107fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.119+0000 7f1ba3400640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ba4074cf0 0x7f1ba4107fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.119+0000 7f1ba3400640 1 -- 172.21.15.18:0/582024277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba4103430 con 0x7f1ba410e870 2024-04-16T17:21:16.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.120+0000 7f1ba3400640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba410e870 0x7f1ba4102b70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4075320 tx=0x7f1b9000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:16.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.120+0000 7f1ba2000640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ba4074cf0 0x7f1ba4107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:21:16.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.120+0000 7f1ba8a00640 1 -- 172.21.15.18:0/582024277 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90015020 con 0x7f1ba410e870 2024-04-16T17:21:16.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.120+0000 7f1ba8a00640 1 -- 172.21.15.18:0/582024277 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b90011d10 con 0x7f1ba410e870 2024-04-16T17:21:16.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.120+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba4103720 con 0x7f1ba410e870 2024-04-16T17:21:16.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.120+0000 7f1ba8a00640 1 -- 172.21.15.18:0/582024277 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90010e40 con 0x7f1ba410e870 2024-04-16T17:21:16.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.120+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba410d950 con 0x7f1ba410e870 2024-04-16T17:21:16.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.127+0000 7f1bacb91740 1 Processor -- start 2024-04-16T17:21:16.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.129+0000 7f1ba8a00640 1 -- 172.21.15.18:0/582024277 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1b90018020 con 0x7f1ba410e870 2024-04-16T17:21:16.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1bacb91740 1 -- start start 2024-04-16T17:21:16.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1ba8a00640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b880a3e20 0x7f1b880a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 0x5562dcffd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1ba8a00640 1 -- 172.21.15.18:0/582024277 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f1b900d0370 con 0x7f1ba410e870 2024-04-16T17:21:16.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd006170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd008aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1bacb91740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5562dcf63120 con 0x5562dcffd1b0 2024-04-16T17:21:16.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1bacb91740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5562dcf09340 con 0x5562dcffdad0 2024-04-16T17:21:16.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.131+0000 7f1bacb91740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5562dcf5f0e0 con 0x5562dd0066b0 2024-04-16T17:21:16.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.132+0000 7f1ba2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd006170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.132+0000 7f1ba2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd006170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56096/0 (socket says 172.21.15.18:56096) 2024-04-16T17:21:16.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.132+0000 7f1ba2000640 1 -- 172.21.15.18:0/4229500542 learned_addr learned my addr 172.21.15.18:0/4229500542 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:16.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.133+0000 7f1ba3400640 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd008aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.133+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 0x5562dcffd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.133+0000 7f1ba2a00640 1 -- 172.21.15.18:0/4229500542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 msgr2=0x5562dd008aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:16.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.133+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd008aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.133+0000 7f1ba2a00640 1 -- 172.21.15.18:0/4229500542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 msgr2=0x5562dd006170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:16.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.133+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd006170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.133+0000 7f1ba2a00640 1 -- 172.21.15.18:0/4229500542 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562dcd3f7d0 con 0x5562dcffd1b0 2024-04-16T17:21:16.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.134+0000 7f1ba2000640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b880a3e20 0x7f1b880a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.136+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 0x5562dcffd590 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000f80 tx=0x7f1b9c032400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db335daaa5ebd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:16.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.136+0000 7f1ba2000640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b880a3e20 0x7f1b880a62e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x5562dcffcc80 tx=0x7f1b940046c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:16.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.136+0000 7f1b8fe00640 1 -- 172.21.15.18:0/4229500542 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c043070 con 0x5562dcffd1b0 2024-04-16T17:21:16.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.137+0000 7f1b8fe00640 1 -- 172.21.15.18:0/4229500542 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9c003b70 con 0x5562dcffd1b0 2024-04-16T17:21:16.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.137+0000 7f1b8fe00640 1 -- 172.21.15.18:0/4229500542 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c039040 con 0x5562dcffd1b0 2024-04-16T17:21:16.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.139+0000 7f1bacb91740 1 -- 172.21.15.18:0/4229500542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 msgr2=0x5562dcffd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.139+0000 7f1bacb91740 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 0x5562dcffd590 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000f80 tx=0x7f1b9c032400 comp rx=0 tx=0).stop 2024-04-16T17:21:16.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.141+0000 7f1bacb91740 1 -- 172.21.15.18:0/4229500542 shutdown_connections 2024-04-16T17:21:16.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.141+0000 7f1bacb91740 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd008aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.141+0000 7f1bacb91740 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd006170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.141+0000 7f1bacb91740 1 --2- 172.21.15.18:0/4229500542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 0x5562dcffd590 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.141+0000 7f1bacb91740 1 -- 172.21.15.18:0/4229500542 >> 172.21.15.18:0/4229500542 conn(0x5562dcff78e0 msgr2=0x5562dcff7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.141+0000 7f1bacb91740 1 -- 172.21.15.18:0/4229500542 shutdown_connections 2024-04-16T17:21:16.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.141+0000 7f1bacb91740 1 -- 172.21.15.18:0/4229500542 wait complete. 2024-04-16T17:21:16.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.142+0000 7f1bacb91740 1 -- start start 2024-04-16T17:21:16.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.142+0000 7f1bacb91740 1 Processor -- start 2024-04-16T17:21:16.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.144+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd003740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.144+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd003c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.144+0000 7f1ba2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd003740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.145+0000 7f1ba2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd003740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48312/0 (socket says 172.21.15.18:48312) 2024-04-16T17:21:16.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.145+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 0x5562dd0041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.145+0000 7f1bacb91740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5562dd009440 con 0x5562dd0066b0 2024-04-16T17:21:16.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.145+0000 7f1bacb91740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5562dcf1c710 con 0x5562dcffdad0 2024-04-16T17:21:16.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.145+0000 7f1bacb91740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5562dd0095c0 con 0x5562dcffd1b0 2024-04-16T17:21:16.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.145+0000 7f1bacb91740 10 client.0 _ll_register_callbacks cb 0x5562dcf17af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:16.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.146+0000 7f1ba2a00640 1 -- 172.21.15.18:0/3735668885 learned_addr learned my addr 172.21.15.18:0/3735668885 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:16.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.146+0000 7f1ba2000640 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd003c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.146+0000 7f1ba3400640 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 0x5562dd0041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.147+0000 7f1ba2a00640 1 -- 172.21.15.18:0/3735668885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 msgr2=0x5562dd003c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:16.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.147+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd003c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.147+0000 7f1ba2a00640 1 -- 172.21.15.18:0/3735668885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 msgr2=0x5562dd0041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:16.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.147+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 0x5562dd0041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.147+0000 7f1ba2a00640 1 -- 172.21.15.18:0/3735668885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562dcc753f0 con 0x5562dcffd1b0 2024-04-16T17:21:16.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.147+0000 7f1ba3400640 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 0x5562dd0041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:16.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.148+0000 7f1b8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:16.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.148+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd003740 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c032ca0 tx=0x7f1b9c032cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6468da382dfb6b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:16.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.149+0000 7f1b8cc00640 1 -- 172.21.15.18:0/3735668885 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c04c070 con 0x5562dcffd1b0 2024-04-16T17:21:16.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.149+0000 7f1b8cc00640 1 -- 172.21.15.18:0/3735668885 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f1b9c051660 con 0x5562dcffd1b0 2024-04-16T17:21:16.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.150+0000 7f1bacb91740 1 -- 172.21.15.18:0/3735668885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9c003b70 con 0x5562dcffd1b0 2024-04-16T17:21:16.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.150+0000 7f1b8cc00640 1 -- 172.21.15.18:0/3735668885 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1b9c043040 con 0x5562dcffd1b0 2024-04-16T17:21:16.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.150+0000 7f1bacb91740 10 client.29776 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:16.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.151+0000 7f1bacb91740 1 -- 172.21.15.18:0/3735668885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5562dcc350f0 con 0x5562dcffd1b0 2024-04-16T17:21:16.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.151+0000 7f1b8cc00640 1 -- 172.21.15.18:0/3735668885 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9c003b70 con 0x5562dcffd1b0 2024-04-16T17:21:16.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.151+0000 7f1b8cc00640 1 -- 172.21.15.18:0/3735668885 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f1b9c048860 con 0x5562dcffd1b0 2024-04-16T17:21:16.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.152+0000 7f1bacb91740 10 client.29776 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:16.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.152+0000 7f1bacb91740 10 client.29776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:16.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.152+0000 7f1bacb91740 20 client.29776 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:16.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.152+0000 7f1bacb91740 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562dd0168c0 0x5562dd018cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.152+0000 7f1bacb91740 4 client.29776 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:16.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.152+0000 7f1bacb91740 1 -- 172.21.15.18:0/3735668885 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562dcf70b40 con 0x5562dd0168c0 2024-04-16T17:21:16.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.152+0000 7f1ba2000640 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562dd0168c0 0x5562dd018cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.153+0000 7f1ba2000640 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562dd0168c0 0x5562dd018cb0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7c5db153e50d2f2 server_cookie=3363e302780a9e3c in_seq=0 out_seq=0 2024-04-16T17:21:16.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.153+0000 7f1b8cc00640 10 client.29776 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:16.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:15 smithi022 ceph-mon[36023]: pgmap v526: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 480 op/s 2024-04-16T17:21:16.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:15 smithi043 ceph-mon[36901]: pgmap v526: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 480 op/s 2024-04-16T17:21:16.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.271+0000 7f1b8cc00640 1 -- 172.21.15.18:0/3735668885 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562dcf70b40 con 0x5562dd0168c0 2024-04-16T17:21:16.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.271+0000 7f1b8cc00640 10 client.29776 handle_command_reply: tid=0 2024-04-16T17:21:16.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.271+0000 7f1bacb91740 1 client.29776 shutdown 2024-04-16T17:21:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.271+0000 7fa28ea00640 1 -- 172.21.15.18:0/3554189769 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d14bbe7010 con 0x55d14bcfb7e0 2024-04-16T17:21:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.271+0000 7fa28ea00640 10 client.29767 handle_command_reply: tid=0 2024-04-16T17:21:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.271+0000 7fa2ae47a740 1 client.29767 shutdown 2024-04-16T17:21:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.272+0000 7f1bacb91740 1 -- 172.21.15.18:0/3735668885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 msgr2=0x5562dd003740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.272+0000 7f1bacb91740 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd003740 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c032ca0 tx=0x7f1b9c032cd0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa2ae47a740 1 -- 172.21.15.18:0/3554189769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7b800 msgr2=0x55d14bc8c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7b800 0x55d14bc8c460 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x55d14bc82370 tx=0x7fa29400dd70 comp rx=0 tx=0).stop 2024-04-16T17:21:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa2ae47a740 1 -- 172.21.15.18:0/3554189769 shutdown_connections 2024-04-16T17:21:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d14bcfb7e0 0x55d14bcfdc00 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d14bc84010 0x55d14bc88ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d14bc7b800 0x55d14bc8c460 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa2ae47a740 1 --2- 172.21.15.18:0/3554189769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d14bc7aa90 0x55d14bc8bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa2ae47a740 1 -- 172.21.15.18:0/3554189769 >> 172.21.15.18:0/3554189769 conn(0x55d14bc155d0 msgr2=0x55d14bc7f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.272+0000 7fa28ea00640 0 client.29767 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.273+0000 7fa2ae47a740 1 -- 172.21.15.18:0/3554189769 shutdown_connections 2024-04-16T17:21:16.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1bacb91740 1 -- 172.21.15.18:0/3735668885 shutdown_connections 2024-04-16T17:21:16.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1bacb91740 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562dd0168c0 0x5562dd018cb0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1bacb91740 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 0x5562dd0041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1bacb91740 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd003c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1bacb91740 1 --2- 172.21.15.18:0/3735668885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd003740 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1bacb91740 1 -- 172.21.15.18:0/3735668885 >> 172.21.15.18:0/3735668885 conn(0x5562dcff78e0 msgr2=0x5562dcff9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1b8cc00640 0 client.29776 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.273+0000 7fa2ae47a740 1 -- 172.21.15.18:0/3554189769 wait complete. 2024-04-16T17:21:16.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1bacb91740 1 -- 172.21.15.18:0/3735668885 shutdown_connections 2024-04-16T17:21:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.273+0000 7fa2ae47a740 20 client.29767 trim_cache size 0 max 16384 2024-04-16T17:21:16.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.273+0000 7f1bacb91740 1 -- 172.21.15.18:0/3735668885 wait complete. 2024-04-16T17:21:16.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.274+0000 7f1bacb91740 20 client.29776 trim_cache size 0 max 16384 2024-04-16T17:21:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:21:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:21:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:21:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:21:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.276+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2880a3de0 msgr2=0x7fa2880a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.276+0000 7fa2aa200640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2880a3de0 0x7fa2880a62a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa290009c00 tx=0x7fa290009740 comp rx=0 tx=0).stop 2024-04-16T17:21:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.276+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 msgr2=0x7fa2a4100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.276+0000 7fa2aa200640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 0x7fa2a4100b60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa29c02c710 tx=0x7fa29c035e50 comp rx=0 tx=0).stop 2024-04-16T17:21:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.278+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 shutdown_connections 2024-04-16T17:21:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.278+0000 7fa2aa200640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa2880a3de0 0x7fa2880a62a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.278+0000 7fa2aa200640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2a410c400 0x7fa2a41015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.278+0000 7fa2aa200640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2a4103900 0x7fa2a41010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.278+0000 7fa2aa200640 1 --2- 172.21.15.18:0/1448702939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2a4102fe0 0x7fa2a4100b60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.278+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 >> 172.21.15.18:0/1448702939 conn(0x7fa2a40fc7e0 msgr2=0x7fa2a40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.281+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 shutdown_connections 2024-04-16T17:21:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:16.281+0000 7fa2aa200640 1 -- 172.21.15.18:0/1448702939 wait complete. 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.319+0000 7f1bacb91740 1 Processor -- start 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.320+0000 7f1bacb91740 1 -- start start 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.320+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd005e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.320+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd012300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.320+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 0x5562dcff9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.320+0000 7f1bacb91740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41c1060 con 0x5562dd0066b0 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.320+0000 7f1bacb91740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41be930 con 0x5562dcffdad0 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.320+0000 7f1bacb91740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41bc7a0 con 0x5562dcffd1b0 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.321+0000 7f1ba2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd005e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.321+0000 7f1ba2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd005e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48322/0 (socket says 172.21.15.18:48322) 2024-04-16T17:21:16.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.321+0000 7f1ba2a00640 1 -- 172.21.15.18:0/2952146574 learned_addr learned my addr 172.21.15.18:0/2952146574 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.321+0000 7f1ba2000640 1 --2- 172.21.15.18:0/2952146574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd012300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.322+0000 7f1ba2000640 1 -- 172.21.15.18:0/2952146574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 msgr2=0x5562dd005e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.322+0000 7f1ba2000640 1 --2- 172.21.15.18:0/2952146574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd005e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.322+0000 7f1ba2000640 1 -- 172.21.15.18:0/2952146574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 msgr2=0x5562dcff9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.322+0000 7f1ba2000640 1 --2- 172.21.15.18:0/2952146574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 0x5562dcff9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.322+0000 7f1ba2000640 1 -- 172.21.15.18:0/2952146574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562dd001c90 con 0x5562dcffdad0 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.322+0000 7f1ba2000640 1 --2- 172.21.15.18:0/2952146574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd012300 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x5562dd0191f0 tx=0x7f1b94012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4244efafee9561cd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.323+0000 7f1b8f400640 1 -- 172.21.15.18:0/2952146574 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9401b040 con 0x5562dcffdad0 2024-04-16T17:21:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.323+0000 7f1b8f400640 1 -- 172.21.15.18:0/2952146574 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9402d070 con 0x5562dcffdad0 2024-04-16T17:21:16.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.324+0000 7f1bacb91740 1 -- 172.21.15.18:0/2952146574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 msgr2=0x5562dd012300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.324+0000 7f1b8f400640 1 -- 172.21.15.18:0/2952146574 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9401b040 con 0x5562dcffdad0 2024-04-16T17:21:16.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.324+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2952146574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd012300 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x5562dd0191f0 tx=0x7f1b94012000 comp rx=0 tx=0).stop 2024-04-16T17:21:16.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.325+0000 7f1bacb91740 1 -- 172.21.15.18:0/2952146574 shutdown_connections 2024-04-16T17:21:16.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.325+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2952146574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dd0066b0 0x5562dcff9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.325+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2952146574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd012300 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.325+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2952146574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dcffd1b0 0x5562dd005e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.325+0000 7f1bacb91740 1 -- 172.21.15.18:0/2952146574 >> 172.21.15.18:0/2952146574 conn(0x5562dcff78e0 msgr2=0x5562dcff8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.326+0000 7f1bacb91740 1 -- 172.21.15.18:0/2952146574 shutdown_connections 2024-04-16T17:21:16.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.327+0000 7f1bacb91740 1 -- 172.21.15.18:0/2952146574 wait complete. 2024-04-16T17:21:16.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.327+0000 7f1bacb91740 1 -- start start 2024-04-16T17:21:16.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.328+0000 7f1bacb91740 1 Processor -- start 2024-04-16T17:21:16.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.329+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 0x5562dd00a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.329+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd00aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.330+0000 7f1bacb91740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd00af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.330+0000 7f1ba2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd00aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.330+0000 7f1ba2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd00aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56116/0 (socket says 172.21.15.18:56116) 2024-04-16T17:21:16.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.330+0000 7f1ba2000640 1 -- 172.21.15.18:0/2034563673 learned_addr learned my addr 172.21.15.18:0/2034563673 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:16.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.330+0000 7f1ba3400640 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd00af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.331+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41bcdf0 con 0x5562dcffd1b0 2024-04-16T17:21:16.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.331+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41be260 con 0x5562dcffdad0 2024-04-16T17:21:16.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.331+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41bd4c0 con 0x5562dd0066b0 2024-04-16T17:21:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.332+0000 7f1ba3400640 1 -- 172.21.15.18:0/2034563673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 msgr2=0x5562dd00aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.332+0000 7f1ba3400640 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd00aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.332+0000 7f1ba3400640 1 -- 172.21.15.18:0/2034563673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 msgr2=0x5562dd00a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.332+0000 7f1ba3400640 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 0x5562dd00a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.332+0000 7f1ba3400640 1 -- 172.21.15.18:0/2034563673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562dd07bd40 con 0x5562dd0066b0 2024-04-16T17:21:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.332+0000 7f1ba2000640 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd00aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.332+0000 7f1ba3400640 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd00af60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1b900cd710 tx=0x7f1b900047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c19741825da019 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.332+0000 7f1bacb91740 10 client.0 _ll_register_callbacks cb 0x7f1b90011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.333+0000 7f1b8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.333+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9402d070 con 0x5562dd0066b0 2024-04-16T17:21:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.333+0000 7f1b8cc00640 1 -- 172.21.15.18:0/2034563673 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900b4070 con 0x5562dd0066b0 2024-04-16T17:21:16.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.333+0000 7f1b8cc00640 1 -- 172.21.15.18:0/2034563673 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f1b90011830 con 0x5562dd0066b0 2024-04-16T17:21:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.336+0000 7f1b8cc00640 1 -- 172.21.15.18:0/2034563673 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1b900a0310 con 0x5562dd0066b0 2024-04-16T17:21:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.336+0000 7f1bacb91740 10 client.29782 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:16.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.336+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5562dd07cf60 con 0x5562dd0066b0 2024-04-16T17:21:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.338+0000 7f1b8cc00640 1 -- 172.21.15.18:0/2034563673 <== mon.1 v2:172.21.15.43:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f1b9009f070 con 0x5562dd0066b0 2024-04-16T17:21:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.338+0000 7f1bacb91740 10 client.29782 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.338+0000 7f1bacb91740 10 client.29782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.338+0000 7f1bacb91740 20 client.29782 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.338+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562dd07d580 0x5562dd07fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.338+0000 7f1bacb91740 4 client.29782 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:21:16.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.338+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562dcf6a9b0 con 0x5562dd07d580 2024-04-16T17:21:16.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.340+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562dd07d580 0x5562dd07fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.342+0000 7f1b8cc00640 10 client.29782 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:16.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.342+0000 7f1ba2a00640 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562dd07d580 0x5562dd07fa00 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313332966bb058c5 server_cookie=1257e2fa468e2061 in_seq=0 out_seq=0 2024-04-16T17:21:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1b8cc00640 1 -- 172.21.15.18:0/2034563673 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5562dcf6a9b0 con 0x5562dd07d580 2024-04-16T17:21:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1b8cc00640 10 client.29782 handle_command_reply: tid=0 2024-04-16T17:21:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 client.29782 shutdown 2024-04-16T17:21:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 msgr2=0x5562dd00af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd00af60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1b900cd710 tx=0x7f1b900047d0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 shutdown_connections 2024-04-16T17:21:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5562dd07d580 0x5562dd07fa00 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562dd0066b0 0x5562dd00af60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5562dcffdad0 0x5562dd00aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 --2- 172.21.15.18:0/2034563673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5562dcffd1b0 0x5562dd00a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1b8cc00640 0 client.29782 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:16.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.350+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 >> 172.21.15.18:0/2034563673 conn(0x5562dcff78e0 msgr2=0x5562dcff68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.352+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 shutdown_connections 2024-04-16T17:21:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.352+0000 7f1bacb91740 1 -- 172.21.15.18:0/2034563673 wait complete. 2024-04-16T17:21:16.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.352+0000 7f1bacb91740 20 client.29782 trim_cache size 0 max 16384 2024-04-16T17:21:16.353 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:21:16.353 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:21:16.353 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:21:16.353 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.353+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b880a3e20 msgr2=0x7f1b880a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.353+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b880a3e20 0x7f1b880a62e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x5562dcffcc80 tx=0x7f1b940046c0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.353+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba410e870 msgr2=0x7f1ba4102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.353+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba410e870 0x7f1ba4102b70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4075320 tx=0x7f1b9000b870 comp rx=0 tx=0).stop 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.354+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 shutdown_connections 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.354+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1b880a3e20 0x7f1b880a62e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.354+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ba410e870 0x7f1ba4102b70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.354+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ba4074cf0 0x7f1ba4107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.354+0000 7f1ba9e00640 1 --2- 172.21.15.18:0/582024277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba40743d0 0x7f1ba4107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.354+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 >> 172.21.15.18:0/582024277 conn(0x7f1ba40fdac0 msgr2=0x7f1ba40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.354+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 shutdown_connections 2024-04-16T17:21:16.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.355+0000 7f1ba9e00640 1 -- 172.21.15.18:0/582024277 wait complete. 2024-04-16T17:21:16.384 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:21:16.384 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:21:16.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 52107 12.28 MB/sec execute 88 sec latency 4004.720 ms 2024-04-16T17:21:16.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.760+0000 7fa965000640 1 -- 172.21.15.18:0/867569298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 msgr2=0x7fa9601068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.760+0000 7fa965000640 1 --2- 172.21.15.18:0/867569298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 0x7fa9601068c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa958009c10 tx=0x7fa95802bfc0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.761+0000 7fa965000640 1 -- 172.21.15.18:0/867569298 shutdown_connections 2024-04-16T17:21:16.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.761+0000 7fa965000640 1 --2- 172.21.15.18:0/867569298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 0x7fa9601068c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.761+0000 7fa965000640 1 --2- 172.21.15.18:0/867569298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa960101930 0x7fa960105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.761+0000 7fa965000640 1 --2- 172.21.15.18:0/867569298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960101010 0x7fa9601013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.761+0000 7fa965000640 1 -- 172.21.15.18:0/867569298 >> 172.21.15.18:0/867569298 conn(0x7fa960078110 msgr2=0x7fa960078520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:16.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.761+0000 7fa965000640 1 -- 172.21.15.18:0/867569298 shutdown_connections 2024-04-16T17:21:16.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.761+0000 7fa965000640 1 -- 172.21.15.18:0/867569298 wait complete. 2024-04-16T17:21:16.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.763+0000 7fa965000640 1 Processor -- start 2024-04-16T17:21:16.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.763+0000 7fa965000640 1 -- start start 2024-04-16T17:21:16.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.764+0000 7fa965000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960101010 0x7fa96019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.764+0000 7fa965000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa960101930 0x7fa96019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.764+0000 7fa965000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 0x7fa9601a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.764+0000 7fa965000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa960118a40 con 0x7fa960101930 2024-04-16T17:21:16.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.764+0000 7fa965000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9601188c0 con 0x7fa960106410 2024-04-16T17:21:16.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.764+0000 7fa965000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa960118bc0 con 0x7fa960101010 2024-04-16T17:21:16.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.765+0000 7fa95e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 0x7fa9601a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.765+0000 7fa95e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 0x7fa9601a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56126/0 (socket says 172.21.15.18:56126) 2024-04-16T17:21:16.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.765+0000 7fa95e800640 1 -- 172.21.15.18:0/3651438744 learned_addr learned my addr 172.21.15.18:0/3651438744 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:16.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.765+0000 7fa95d400640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa960101930 0x7fa96019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.766+0000 7fa95e800640 1 -- 172.21.15.18:0/3651438744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960101010 msgr2=0x7fa96019eea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:21:16.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.767+0000 7fa95e800640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960101010 0x7fa96019eea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.767+0000 7fa95e800640 1 -- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa960101930 msgr2=0x7fa96019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:16.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.767+0000 7fa95e800640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa960101930 0x7fa96019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:16.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.767+0000 7fa95e800640 1 -- 172.21.15.18:0/3651438744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9601a3e90 con 0x7fa960106410 2024-04-16T17:21:16.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.767+0000 7fa95e800640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 0x7fa9601a3760 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa958036040 tx=0x7fa9580043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:16.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.768+0000 7fa956a00640 1 -- 172.21.15.18:0/3651438744 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958043020 con 0x7fa960106410 2024-04-16T17:21:16.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.768+0000 7fa956a00640 1 -- 172.21.15.18:0/3651438744 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9580046f0 con 0x7fa960106410 2024-04-16T17:21:16.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.769+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9601a4120 con 0x7fa960106410 2024-04-16T17:21:16.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.769+0000 7fa956a00640 1 -- 172.21.15.18:0/3651438744 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95803e910 con 0x7fa960106410 2024-04-16T17:21:16.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.769+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601aba10 con 0x7fa960106410 2024-04-16T17:21:16.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.774+0000 7fa956a00640 1 -- 172.21.15.18:0/3651438744 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa958046020 con 0x7fa960106410 2024-04-16T17:21:16.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.776+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96006af10 con 0x7fa960106410 2024-04-16T17:21:16.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.778+0000 7fa956a00640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9440a3da0 0x7fa9440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:16.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.778+0000 7fa956a00640 1 -- 172.21.15.18:0/3651438744 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fa9580f2bd0 con 0x7fa960106410 2024-04-16T17:21:16.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.783+0000 7fa95de00640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9440a3da0 0x7fa9440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:16.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.786+0000 7fa956a00640 1 -- 172.21.15.18:0/3651438744 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa95803d3b0 con 0x7fa960106410 2024-04-16T17:21:16.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:16.789+0000 7fa95de00640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9440a3da0 0x7fa9440a6260 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa9500058c0 tx=0x7fa950005850 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:17.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.331+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9601a01f0 con 0x7fa960106410 2024-04-16T17:21:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.333+0000 7fa956a00640 1 -- 172.21.15.18:0/3651438744 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7fa9580beec0 con 0x7fa960106410 2024-04-16T17:21:17.336 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:21:17.336 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:21:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:21:17.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.349+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9440a3da0 msgr2=0x7fa9440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.349+0000 7fa965000640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9440a3da0 0x7fa9440a6260 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa9500058c0 tx=0x7fa950005850 comp rx=0 tx=0).stop 2024-04-16T17:21:17.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.349+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 msgr2=0x7fa9601a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.349+0000 7fa965000640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 0x7fa9601a3760 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa958036040 tx=0x7fa9580043b0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.350+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 shutdown_connections 2024-04-16T17:21:17.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.350+0000 7fa965000640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa9440a3da0 0x7fa9440a6260 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa9500058c0 tx=0x7fa950005850 comp rx=0 tx=0).stop 2024-04-16T17:21:17.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.350+0000 7fa965000640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa960106410 0x7fa9601a3760 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.350+0000 7fa965000640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa960101930 0x7fa96019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.350+0000 7fa965000640 1 --2- 172.21.15.18:0/3651438744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960101010 0x7fa96019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.350+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 >> 172.21.15.18:0/3651438744 conn(0x7fa960078110 msgr2=0x7fa960103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:17.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.351+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 shutdown_connections 2024-04-16T17:21:17.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.351+0000 7fa965000640 1 -- 172.21.15.18:0/3651438744 wait complete. 2024-04-16T17:21:17.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:21:17.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 52351 12.30 MB/sec execute 89 sec latency 4377.590 ms 2024-04-16T17:21:17.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.715+0000 7f4231200640 1 -- 172.21.15.18:0/201614874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c109b00 msgr2=0x7f422c114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.715+0000 7f4231200640 1 --2- 172.21.15.18:0/201614874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c109b00 0x7f422c114c10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f421c009930 tx=0x7f421c02bc20 comp rx=0 tx=0).stop 2024-04-16T17:21:17.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.715+0000 7f4231200640 1 -- 172.21.15.18:0/201614874 shutdown_connections 2024-04-16T17:21:17.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.715+0000 7f4231200640 1 --2- 172.21.15.18:0/201614874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c115150 0x7f422c117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.715+0000 7f4231200640 1 --2- 172.21.15.18:0/201614874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c109b00 0x7f422c114c10 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.715+0000 7f4231200640 1 --2- 172.21.15.18:0/201614874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f422c1091e0 0x7f422c1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.715+0000 7f4231200640 1 -- 172.21.15.18:0/201614874 >> 172.21.15.18:0/201614874 conn(0x7f422c06d0a0 msgr2=0x7f422c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:17.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.716+0000 7f4231200640 1 -- 172.21.15.18:0/201614874 shutdown_connections 2024-04-16T17:21:17.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.716+0000 7f4231200640 1 -- 172.21.15.18:0/201614874 wait complete. 2024-04-16T17:21:17.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.717+0000 7f4231200640 1 Processor -- start 2024-04-16T17:21:17.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.719+0000 7f4231200640 1 -- start start 2024-04-16T17:21:17.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.719+0000 7f4231200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c1091e0 0x7f422c1a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.720+0000 7f4231200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f422c109b00 0x7f422c1a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.720+0000 7f4229e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c1091e0 0x7f422c1a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.720+0000 7f4229e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c1091e0 0x7f422c1a3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40150/0 (socket says 172.21.15.18:40150) 2024-04-16T17:21:17.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.721+0000 7f4231200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c115150 0x7f422c1a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.721+0000 7f4229e00640 1 -- 172.21.15.18:0/632324621 learned_addr learned my addr 172.21.15.18:0/632324621 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:17.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.721+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10bba0 con 0x7f422c109b00 2024-04-16T17:21:17.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.721+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10ba20 con 0x7f422c1091e0 2024-04-16T17:21:17.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.721+0000 7f422a800640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c115150 0x7f422c1a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.721+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10bd20 con 0x7f422c115150 2024-04-16T17:21:17.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.722+0000 7f422a800640 1 -- 172.21.15.18:0/632324621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c1091e0 msgr2=0x7f422c1a3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.722+0000 7f422a800640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c1091e0 0x7f422c1a3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.722+0000 7f422a800640 1 -- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f422c109b00 msgr2=0x7f422c1a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.722+0000 7f422a800640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f422c109b00 0x7f422c1a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.722+0000 7f422a800640 1 -- 172.21.15.18:0/632324621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c1a8340 con 0x7f422c115150 2024-04-16T17:21:17.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.723+0000 7f422a800640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c115150 0x7f422c1a7c10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f421800e940 tx=0x7f421800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:17.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.724+0000 7f4222a00640 1 -- 172.21.15.18:0/632324621 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421800ccc0 con 0x7f422c115150 2024-04-16T17:21:17.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.724+0000 7f4222a00640 1 -- 172.21.15.18:0/632324621 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4218005020 con 0x7f422c115150 2024-04-16T17:21:17.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.724+0000 7f4222a00640 1 -- 172.21.15.18:0/632324621 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421800ccc0 con 0x7f422c115150 2024-04-16T17:21:17.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.724+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422c1a8630 con 0x7f422c115150 2024-04-16T17:21:17.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.724+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c1afec0 con 0x7f422c115150 2024-04-16T17:21:17.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.735+0000 7f423401d740 1 Processor -- start 2024-04-16T17:21:17.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.738+0000 7f423401d740 1 -- start start 2024-04-16T17:21:17.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.738+0000 7f423401d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 0x5582f6938130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.738+0000 7f4222a00640 1 -- 172.21.15.18:0/632324621 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f42180051d0 con 0x7f422c115150 2024-04-16T17:21:17.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.738+0000 7f423401d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f6940d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.738+0000 7f423401d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f6943640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.738+0000 7f423401d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5582f689d020 con 0x5582f6938670 2024-04-16T17:21:17.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.738+0000 7f423401d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582f6843460 con 0x5582f6937d50 2024-04-16T17:21:17.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.738+0000 7f423401d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5582f6898fe0 con 0x5582f6941250 2024-04-16T17:21:17.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.739+0000 7f4229400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f6943640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.740+0000 7f4229400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f6943640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44372/0 (socket says 172.21.15.18:44372) 2024-04-16T17:21:17.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.740+0000 7f4229400640 1 -- 172.21.15.18:0/1915314897 learned_addr learned my addr 172.21.15.18:0/1915314897 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:17.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.740+0000 7f4229400640 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 0x5582f6938130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.740+0000 7f4229e00640 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f6940d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.741+0000 7f4229400640 1 -- 172.21.15.18:0/1915314897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 msgr2=0x5582f6938130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:17.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.741+0000 7f4229400640 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 0x5582f6938130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.741+0000 7f4229400640 1 -- 172.21.15.18:0/1915314897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 msgr2=0x5582f6940d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:17.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.741+0000 7f4229400640 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f6940d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.742+0000 7f4229400640 1 -- 172.21.15.18:0/1915314897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f684b040 con 0x5582f6941250 2024-04-16T17:21:17.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.742+0000 7f4229400640 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f6943640 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f421c02c160 tx=0x7f421c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b15a9eb2a44749 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:17.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.743+0000 7f4217400640 1 -- 172.21.15.18:0/1915314897 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c02caa0 con 0x5582f6941250 2024-04-16T17:21:17.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.743+0000 7f4217400640 1 -- 172.21.15.18:0/1915314897 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c048070 con 0x5582f6941250 2024-04-16T17:21:17.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.743+0000 7f4217400640 1 -- 172.21.15.18:0/1915314897 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c02caa0 con 0x5582f6941250 2024-04-16T17:21:17.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.744+0000 7f423401d740 1 -- 172.21.15.18:0/1915314897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 msgr2=0x5582f6943640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.744+0000 7f423401d740 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f6943640 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f421c02c160 tx=0x7f421c009260 comp rx=0 tx=0).stop 2024-04-16T17:21:17.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.744+0000 7f4222a00640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f42100a4020 0x7f42100a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.744+0000 7f4222a00640 1 -- 172.21.15.18:0/632324621 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f42180d0870 con 0x7f422c115150 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.748+0000 7f4229400640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f42100a4020 0x7f42100a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.748+0000 7f4229400640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f42100a4020 0x7f42100a64e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f421c045040 tx=0x7f421c009640 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.750+0000 7f423401d740 1 -- 172.21.15.18:0/1915314897 shutdown_connections 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.750+0000 7f423401d740 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f6943640 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.750+0000 7f423401d740 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f6940d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.750+0000 7f423401d740 1 --2- 172.21.15.18:0/1915314897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 0x5582f6938130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.750+0000 7f423401d740 1 -- 172.21.15.18:0/1915314897 >> 172.21.15.18:0/1915314897 conn(0x5582f6932480 msgr2=0x5582f6932860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.750+0000 7f423401d740 1 -- 172.21.15.18:0/1915314897 shutdown_connections 2024-04-16T17:21:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.750+0000 7f423401d740 1 -- 172.21.15.18:0/1915314897 wait complete. 2024-04-16T17:21:17.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.750+0000 7f423401d740 1 -- start start 2024-04-16T17:21:17.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f423401d740 1 Processor -- start 2024-04-16T17:21:17.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f423401d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6937d50 0x5582f693e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f423401d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6938670 0x5582f693e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f423401d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f693ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f423401d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5582f6943fe0 con 0x5582f6937d50 2024-04-16T17:21:17.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f423401d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582f6856830 con 0x5582f6938670 2024-04-16T17:21:17.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f423401d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5582f6944160 con 0x5582f6941250 2024-04-16T17:21:17.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f423401d740 10 client.0 _ll_register_callbacks cb 0x5582f685e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:17.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f422a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f693ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f422a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f693ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44384/0 (socket says 172.21.15.18:44384) 2024-04-16T17:21:17.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f422a800640 1 -- 172.21.15.18:0/4115214767 learned_addr learned my addr 172.21.15.18:0/4115214767 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:17.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f4229400640 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6938670 0x5582f693e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f4229400640 1 -- 172.21.15.18:0/4115214767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 msgr2=0x5582f693ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:17.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f4229400640 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f693ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f4229400640 1 -- 172.21.15.18:0/4115214767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6937d50 msgr2=0x5582f693e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:17.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f4229400640 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6937d50 0x5582f693e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f4229400640 1 -- 172.21.15.18:0/4115214767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f685a920 con 0x5582f6938670 2024-04-16T17:21:17.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f4229400640 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6938670 0x5582f693e7b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f421c005ec0 tx=0x7f421c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a5f32f7169a2138 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:17.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f420be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:17.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f420be00640 1 -- 172.21.15.18:0/4115214767 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c052070 con 0x5582f6938670 2024-04-16T17:21:17.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.751+0000 7f420be00640 1 -- 172.21.15.18:0/4115214767 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f421c043860 con 0x5582f6938670 2024-04-16T17:21:17.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.757+0000 7f423401d740 1 -- 172.21.15.18:0/4115214767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421c048070 con 0x5582f6938670 2024-04-16T17:21:17.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.757+0000 7f420be00640 1 -- 172.21.15.18:0/4115214767 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f421c02caa0 con 0x5582f6938670 2024-04-16T17:21:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.758+0000 7f423401d740 10 client.29987 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.758+0000 7f423401d740 1 -- 172.21.15.18:0/4115214767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5582f68287f0 con 0x5582f6938670 2024-04-16T17:21:17.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f420be00640 1 -- 172.21.15.18:0/4115214767 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c048070 con 0x5582f6938670 2024-04-16T17:21:17.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f420be00640 1 -- 172.21.15.18:0/4115214767 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f421c034a90 con 0x5582f6938670 2024-04-16T17:21:17.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 10 client.29987 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:17.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 10 client.29987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:21:17.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 20 client.29987 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:17.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5582f69621e0 0x5582f69645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 4 client.29987 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:17.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 -- 172.21.15.18:0/4115214767 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582f68aaa40 con 0x5582f69621e0 2024-04-16T17:21:17.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f4229e00640 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5582f69621e0 0x5582f69645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f4229e00640 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5582f69621e0 0x5582f69645d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78a07aa87d29722f server_cookie=957324029eb81510 in_seq=0 out_seq=0 2024-04-16T17:21:17.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f420be00640 10 client.29987 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:21:17.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f420be00640 1 -- 172.21.15.18:0/4115214767 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582f68aaa40 con 0x5582f69621e0 2024-04-16T17:21:17.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f420be00640 10 client.29987 handle_command_reply: tid=0 2024-04-16T17:21:17.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 client.29987 shutdown 2024-04-16T17:21:17.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 -- 172.21.15.18:0/4115214767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6938670 msgr2=0x5582f693e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6938670 0x5582f693e7b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f421c005ec0 tx=0x7f421c009260 comp rx=0 tx=0).stop 2024-04-16T17:21:17.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 -- 172.21.15.18:0/4115214767 shutdown_connections 2024-04-16T17:21:17.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5582f69621e0 0x5582f69645d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6941250 0x5582f693ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6938670 0x5582f693e7b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 --2- 172.21.15.18:0/4115214767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6937d50 0x5582f693e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 -- 172.21.15.18:0/4115214767 >> 172.21.15.18:0/4115214767 conn(0x5582f6932480 msgr2=0x5582f69341d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:17.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f420be00640 0 client.29987 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:21:17.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 -- 172.21.15.18:0/4115214767 shutdown_connections 2024-04-16T17:21:17.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.760+0000 7f423401d740 1 -- 172.21.15.18:0/4115214767 wait complete. 2024-04-16T17:21:17.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.766+0000 7f423401d740 20 client.29987 trim_cache size 0 max 16384 2024-04-16T17:21:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.801+0000 7f423401d740 1 Processor -- start 2024-04-16T17:21:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.801+0000 7f423401d740 1 -- start start 2024-04-16T17:21:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.802+0000 7f423401d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 0x5582f69409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.802+0000 7f423401d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6938670 0x5582f694cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.802+0000 7f423401d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6941250 0x5582f693eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.802+0000 7f423401d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b8590 con 0x5582f6941250 2024-04-16T17:21:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.802+0000 7f423401d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b5e60 con 0x5582f6937d50 2024-04-16T17:21:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.802+0000 7f423401d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b3cf0 con 0x5582f6938670 2024-04-16T17:21:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.803+0000 7f4229400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6938670 0x5582f694cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.803+0000 7f4229400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6938670 0x5582f694cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44396/0 (socket says 172.21.15.18:44396) 2024-04-16T17:21:17.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.803+0000 7f4229400640 1 -- 172.21.15.18:0/3539121141 learned_addr learned my addr 172.21.15.18:0/3539121141 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:17.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.803+0000 7f4229e00640 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 0x5582f69409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.804+0000 7f422a800640 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6941250 0x5582f693eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.805+0000 7f4229400640 1 -- 172.21.15.18:0/3539121141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 msgr2=0x5582f69409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:17.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.805+0000 7f4229400640 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 0x5582f69409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.805+0000 7f4229400640 1 -- 172.21.15.18:0/3539121141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6941250 msgr2=0x5582f693eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:17.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.805+0000 7f4229400640 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6941250 0x5582f693eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.805+0000 7f4229400640 1 -- 172.21.15.18:0/3539121141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f69384a0 con 0x5582f6938670 2024-04-16T17:21:17.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.806+0000 7f422a800640 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6941250 0x5582f693eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:21:17.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.806+0000 7f4229400640 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6938670 0x5582f694cc40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f421c034e30 tx=0x7f421c02c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5862628364db25c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.807+0000 7f4216a00640 1 -- 172.21.15.18:0/3539121141 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c052040 con 0x5582f6938670 2024-04-16T17:21:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.807+0000 7f4216a00640 1 -- 172.21.15.18:0/3539121141 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c048070 con 0x5582f6938670 2024-04-16T17:21:17.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.807+0000 7f4216a00640 1 -- 172.21.15.18:0/3539121141 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c0346a0 con 0x5582f6938670 2024-04-16T17:21:17.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.808+0000 7f423401d740 1 -- 172.21.15.18:0/3539121141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6938670 msgr2=0x5582f694cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.808+0000 7f423401d740 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6938670 0x5582f694cc40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f421c034e30 tx=0x7f421c02c9c0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.809+0000 7f423401d740 1 -- 172.21.15.18:0/3539121141 shutdown_connections 2024-04-16T17:21:17.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.809+0000 7f423401d740 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6941250 0x5582f693eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.809+0000 7f423401d740 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6938670 0x5582f694cc40 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.809+0000 7f423401d740 1 --2- 172.21.15.18:0/3539121141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6937d50 0x5582f69409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.809+0000 7f423401d740 1 -- 172.21.15.18:0/3539121141 >> 172.21.15.18:0/3539121141 conn(0x5582f6932480 msgr2=0x5582f6933830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:17.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.810+0000 7f423401d740 1 -- 172.21.15.18:0/3539121141 shutdown_connections 2024-04-16T17:21:17.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.810+0000 7f423401d740 1 -- 172.21.15.18:0/3539121141 wait complete. 2024-04-16T17:21:17.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.811+0000 7f423401d740 1 -- start start 2024-04-16T17:21:17.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.812+0000 7f423401d740 1 Processor -- start 2024-04-16T17:21:17.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.812+0000 7f423401d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6937d50 0x5582f6964460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.812+0000 7f423401d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f69649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.812+0000 7f423401d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6941250 0x5582f69c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.812+0000 7f423401d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b4340 con 0x5582f6938670 2024-04-16T17:21:17.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.812+0000 7f423401d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b5790 con 0x5582f6941250 2024-04-16T17:21:17.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.812+0000 7f423401d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b49f0 con 0x5582f6937d50 2024-04-16T17:21:17.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.812+0000 7f423401d740 10 client.0 _ll_register_callbacks cb 0x7f421c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:17.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.813+0000 7f422a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6941250 0x5582f69c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.813+0000 7f422a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6941250 0x5582f69c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40200/0 (socket says 172.21.15.18:40200) 2024-04-16T17:21:17.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.814+0000 7f422a800640 1 -- 172.21.15.18:0/3788950082 learned_addr learned my addr 172.21.15.18:0/3788950082 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:17.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.813+0000 7f4229e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6937d50 0x5582f6964460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.814+0000 7f4229400640 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f69649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.815+0000 7f422a800640 1 -- 172.21.15.18:0/3788950082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6937d50 msgr2=0x5582f6964460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:17.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.815+0000 7f422a800640 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6937d50 0x5582f6964460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.815+0000 7f422a800640 1 -- 172.21.15.18:0/3788950082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 msgr2=0x5582f69649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:17.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.815+0000 7f422a800640 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f69649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.815+0000 7f422a800640 1 -- 172.21.15.18:0/3788950082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f65af3f0 con 0x5582f6941250 2024-04-16T17:21:17.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.816+0000 7f4229400640 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f69649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:17.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.817+0000 7f422a800640 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6941250 0x5582f69c63b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f421809c2e0 tx=0x7f421809cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3b263903a98b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:17.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.817+0000 7f420be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:17.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.817+0000 7f420be00640 1 -- 172.21.15.18:0/3788950082 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421801c040 con 0x5582f6941250 2024-04-16T17:21:17.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.818+0000 7f420be00640 1 -- 172.21.15.18:0/3788950082 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f42180cdd80 con 0x5582f6941250 2024-04-16T17:21:17.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.818+0000 7f423401d740 1 -- 172.21.15.18:0/3788950082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421c048070 con 0x5582f6941250 2024-04-16T17:21:17.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.819+0000 7f420be00640 1 -- 172.21.15.18:0/3788950082 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f42180cdf80 con 0x5582f6941250 2024-04-16T17:21:17.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.819+0000 7f423401d740 10 client.29993 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:17.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.819+0000 7f423401d740 1 -- 172.21.15.18:0/3788950082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5582f66797d0 con 0x5582f6941250 2024-04-16T17:21:17.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.819+0000 7f420be00640 1 -- 172.21.15.18:0/3788950082 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c048070 con 0x5582f6941250 2024-04-16T17:21:17.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.820+0000 7f420be00640 1 -- 172.21.15.18:0/3788950082 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f421809b7a0 con 0x5582f6941250 2024-04-16T17:21:17.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.820+0000 7f423401d740 10 client.29993 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:17.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.820+0000 7f423401d740 10 client.29993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:21:17.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.820+0000 7f423401d740 20 client.29993 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:17.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.820+0000 7f423401d740 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5582f69c80b0 0x5582f69ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:17.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.820+0000 7f423401d740 4 client.29993 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:21:17.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.820+0000 7f423401d740 1 -- 172.21.15.18:0/3788950082 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582f68a48b0 con 0x5582f69c80b0 2024-04-16T17:21:17.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.823+0000 7f4229e00640 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5582f69c80b0 0x5582f69ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.824+0000 7f4229e00640 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5582f69c80b0 0x5582f69ca4d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4a88277b37c8950 server_cookie=7c04608516a9fe8c in_seq=0 out_seq=0 2024-04-16T17:21:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.824+0000 7f420be00640 10 client.29993 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:21:17.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.824+0000 7f420be00640 1 -- 172.21.15.18:0/3788950082 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582f68a48b0 con 0x5582f69c80b0 2024-04-16T17:21:17.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.824+0000 7f420be00640 10 client.29993 handle_command_reply: tid=0 2024-04-16T17:21:17.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 client.29993 shutdown 2024-04-16T17:21:17.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 -- 172.21.15.18:0/3788950082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6941250 msgr2=0x5582f69c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6941250 0x5582f69c63b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f421809c2e0 tx=0x7f421809cb10 comp rx=0 tx=0).stop 2024-04-16T17:21:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 -- 172.21.15.18:0/3788950082 shutdown_connections 2024-04-16T17:21:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5582f69c80b0 0x5582f69ca4d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582f6941250 0x5582f69c63b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582f6938670 0x5582f69649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 --2- 172.21.15.18:0/3788950082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5582f6937d50 0x5582f6964460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f423401d740 1 -- 172.21.15.18:0/3788950082 >> 172.21.15.18:0/3788950082 conn(0x5582f6932480 msgr2=0x5582f694cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.825+0000 7f420be00640 0 client.29993 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:21:17.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.826+0000 7f423401d740 1 -- 172.21.15.18:0/3788950082 shutdown_connections 2024-04-16T17:21:17.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.826+0000 7f423401d740 1 -- 172.21.15.18:0/3788950082 wait complete. 2024-04-16T17:21:17.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.826+0000 7f423401d740 20 client.29993 trim_cache size 0 max 16384 2024-04-16T17:21:17.827 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:21:17.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.828+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f42100a4020 msgr2=0x7f42100a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.828+0000 7f4231200640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f42100a4020 0x7f42100a64e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f421c045040 tx=0x7f421c009640 comp rx=0 tx=0).stop 2024-04-16T17:21:17.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.828+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c115150 msgr2=0x7f422c1a7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:17.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.828+0000 7f4231200640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c115150 0x7f422c1a7c10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f421800e940 tx=0x7f421800ee00 comp rx=0 tx=0).stop 2024-04-16T17:21:17.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.839+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 shutdown_connections 2024-04-16T17:21:17.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.839+0000 7f4231200640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f42100a4020 0x7f42100a64e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.839+0000 7f4231200640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c115150 0x7f422c1a7c10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.839+0000 7f4231200640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f422c109b00 0x7f422c1a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.839+0000 7f4231200640 1 --2- 172.21.15.18:0/632324621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f422c1091e0 0x7f422c1a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:17.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.839+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 >> 172.21.15.18:0/632324621 conn(0x7f422c06d0a0 msgr2=0x7f422c070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:17.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.839+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 shutdown_connections 2024-04-16T17:21:17.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:17.839+0000 7f4231200640 1 -- 172.21.15.18:0/632324621 wait complete. 2024-04-16T17:21:17.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:21:18.170 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:17 smithi018 ceph-mon[31776]: pgmap v527: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 398 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 511 op/s 2024-04-16T17:21:18.170 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:17 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3651438744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:21:18.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:17 smithi022 ceph-mon[36023]: pgmap v527: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 398 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 511 op/s 2024-04-16T17:21:18.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:17 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3651438744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:21:18.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.212+0000 7f57aaa00640 1 -- 172.21.15.18:0/1562767928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 msgr2=0x7f57a4101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.212+0000 7f57aaa00640 1 --2- 172.21.15.18:0/1562767928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 0x7f57a4101340 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5798009900 tx=0x7f579802bc50 comp rx=0 tx=0).stop 2024-04-16T17:21:18.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.212+0000 7f57aaa00640 1 -- 172.21.15.18:0/1562767928 shutdown_connections 2024-04-16T17:21:18.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.212+0000 7f57aaa00640 1 --2- 172.21.15.18:0/1562767928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a4106030 0x7f57a4069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.212+0000 7f57aaa00640 1 --2- 172.21.15.18:0/1562767928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57a4101910 0x7f57a4105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.212+0000 7f57aaa00640 1 --2- 172.21.15.18:0/1562767928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 0x7f57a4101340 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.212+0000 7f57aaa00640 1 -- 172.21.15.18:0/1562767928 >> 172.21.15.18:0/1562767928 conn(0x7f57a4077ab0 msgr2=0x7f57a40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.213+0000 7f57aaa00640 1 -- 172.21.15.18:0/1562767928 shutdown_connections 2024-04-16T17:21:18.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.214+0000 7f57aaa00640 1 -- 172.21.15.18:0/1562767928 wait complete. 2024-04-16T17:21:18.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.215+0000 7f57aaa00640 1 Processor -- start 2024-04-16T17:21:18.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.216+0000 7f57aaa00640 1 -- start start 2024-04-16T17:21:18.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.216+0000 7f57aaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 0x7f57a41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.216+0000 7f57aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a4106030 0x7f57a41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.216+0000 7f57aaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57a419a750 0x7f57a419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.216+0000 7f57aaa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4114440 con 0x7f57a419a750 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.216+0000 7f57aaa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41142c0 con 0x7f57a4100f60 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.216+0000 7f57aaa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41145c0 con 0x7f57a4106030 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.217+0000 7f57a9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 0x7f57a41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.217+0000 7f57a9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 0x7f57a41a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40212/0 (socket says 172.21.15.18:40212) 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.217+0000 7f57a9600640 1 -- 172.21.15.18:0/3986717858 learned_addr learned my addr 172.21.15.18:0/3986717858 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.217+0000 7f57a8c00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a4106030 0x7f57a41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.217+0000 7f57aa000640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57a419a750 0x7f57a419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.218+0000 7f57a8c00640 1 -- 172.21.15.18:0/3986717858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 msgr2=0x7f57a41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.218+0000 7f57a8c00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 0x7f57a41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.218+0000 7f57a8c00640 1 -- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57a419a750 msgr2=0x7f57a419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.218+0000 7f57a8c00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57a419a750 0x7f57a419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.218+0000 7f57a8c00640 1 -- 172.21.15.18:0/3986717858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a419b320 con 0x7f57a4106030 2024-04-16T17:21:18.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.218+0000 7f57a8c00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a4106030 0x7f57a41a0bd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f579000d6d0 tx=0x7f579000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.218+0000 7f57aa000640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57a419a750 0x7f57a419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:18.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.219+0000 7f57a2000640 1 -- 172.21.15.18:0/3986717858 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790017070 con 0x7f57a4106030 2024-04-16T17:21:18.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.219+0000 7f57a2000640 1 -- 172.21.15.18:0/3986717858 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5790009820 con 0x7f57a4106030 2024-04-16T17:21:18.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.219+0000 7f57a2000640 1 -- 172.21.15.18:0/3986717858 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790004ce0 con 0x7f57a4106030 2024-04-16T17:21:18.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.219+0000 7f57aaa00640 1 -- 172.21.15.18:0/3986717858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a406ba20 con 0x7f57a4106030 2024-04-16T17:21:18.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.219+0000 7f57aaa00640 1 -- 172.21.15.18:0/3986717858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a406bf20 con 0x7f57a4106030 2024-04-16T17:21:18.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.224+0000 7f57aed75740 1 Processor -- start 2024-04-16T17:21:18.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.226+0000 7f57a2000640 1 -- 172.21.15.18:0/3986717858 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f57900099d0 con 0x7f57a4106030 2024-04-16T17:21:18.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.228+0000 7f57aed75740 1 -- start start 2024-04-16T17:21:18.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57aed75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb6022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57aed75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 0x559ceb60aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57aed75740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb60b410 0x559ceb60d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57aed75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559ceb567020 con 0x559ceb602830 2024-04-16T17:21:18.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57aed75740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559ceb50d460 con 0x559ceb60b410 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57aed75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559ceb562fe0 con 0x559ceb601f10 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57a8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb60b410 0x559ceb60d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57a8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb60b410 0x559ceb60d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40218/0 (socket says 172.21.15.18:40218) 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.229+0000 7f57a8c00640 1 -- 172.21.15.18:0/1123382265 learned_addr learned my addr 172.21.15.18:0/1123382265 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.230+0000 7f57a9600640 1 --2- 172.21.15.18:0/1123382265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 0x559ceb60aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.230+0000 7f57a8c00640 1 -- 172.21.15.18:0/1123382265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 msgr2=0x559ceb6022f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.230+0000 7f57a8c00640 1 --2- 172.21.15.18:0/1123382265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb6022f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.230+0000 7f57a8c00640 1 -- 172.21.15.18:0/1123382265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 msgr2=0x559ceb60aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.230+0000 7f57a8c00640 1 --2- 172.21.15.18:0/1123382265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 0x559ceb60aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.230+0000 7f57a8c00640 1 -- 172.21.15.18:0/1123382265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ceb515040 con 0x559ceb60b410 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.230+0000 7f57a8c00640 1 --2- 172.21.15.18:0/1123382265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb60b410 0x559ceb60d800 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f57900cd6f0 tx=0x7f57900d6b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca41bdc21b1e8008 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.231+0000 7f57a2000640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f578c0a3ea0 0x7f578c0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.231+0000 7f5796600640 1 -- 172.21.15.18:0/1123382265 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900e3070 con 0x559ceb60b410 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.231+0000 7f5796600640 1 -- 172.21.15.18:0/1123382265 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579009c6d0 con 0x559ceb60b410 2024-04-16T17:21:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.231+0000 7f5796600640 1 -- 172.21.15.18:0/1123382265 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900968c0 con 0x559ceb60b410 2024-04-16T17:21:18.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.232+0000 7f57a2000640 1 -- 172.21.15.18:0/3986717858 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f57900d0490 con 0x7f57a4106030 2024-04-16T17:21:18.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.234+0000 7f57aed75740 1 -- 172.21.15.18:0/1123382265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb60b410 msgr2=0x559ceb60d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.234+0000 7f57aed75740 1 --2- 172.21.15.18:0/1123382265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb60b410 0x559ceb60d800 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f57900cd6f0 tx=0x7f57900d6b00 comp rx=0 tx=0).stop 2024-04-16T17:21:18.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.235+0000 7f57aa000640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f578c0a3ea0 0x7f578c0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.235+0000 7f57aed75740 1 -- 172.21.15.18:0/1123382265 shutdown_connections 2024-04-16T17:21:18.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.235+0000 7f57aed75740 1 --2- 172.21.15.18:0/1123382265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb60b410 0x559ceb60d800 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.235+0000 7f57aed75740 1 --2- 172.21.15.18:0/1123382265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 0x559ceb60aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.235+0000 7f57aed75740 1 --2- 172.21.15.18:0/1123382265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb6022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.235+0000 7f57aed75740 1 -- 172.21.15.18:0/1123382265 >> 172.21.15.18:0/1123382265 conn(0x559ceb5fc640 msgr2=0x559ceb5fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.238+0000 7f57aed75740 1 -- 172.21.15.18:0/1123382265 shutdown_connections 2024-04-16T17:21:18.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.238+0000 7f57aed75740 1 -- 172.21.15.18:0/1123382265 wait complete. 2024-04-16T17:21:18.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.238+0000 7f57aa000640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f578c0a3ea0 0x7f578c0a6360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f579c0099a0 tx=0x7f579c009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.239+0000 7f57aed75740 1 -- start start 2024-04-16T17:21:18.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.239+0000 7f57aed75740 1 Processor -- start 2024-04-16T17:21:18.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57aed75740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb601f10 0x559ceb608540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57aed75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 0x559ceb608a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57a9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb601f10 0x559ceb608540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57aed75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb608fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57aed75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559ceb61f210 con 0x559ceb602830 2024-04-16T17:21:18.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57aed75740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559ceb520830 con 0x559ceb601f10 2024-04-16T17:21:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57aed75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559ceb61f390 con 0x559ceb60b410 2024-04-16T17:21:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57a8c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 0x559ceb608a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57a9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb601f10 0x559ceb608540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40228/0 (socket says 172.21.15.18:40228) 2024-04-16T17:21:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.240+0000 7f57a9600640 1 -- 172.21.15.18:0/1155144075 learned_addr learned my addr 172.21.15.18:0/1155144075 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.241+0000 7f57aa000640 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb608fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.241+0000 7f57aed75740 10 client.0 _ll_register_callbacks cb 0x559ceb528150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.241+0000 7f57aa000640 1 -- 172.21.15.18:0/1155144075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb601f10 msgr2=0x559ceb608540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.241+0000 7f57aa000640 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb601f10 0x559ceb608540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.241+0000 7f57aa000640 1 -- 172.21.15.18:0/1155144075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 msgr2=0x559ceb608a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.241+0000 7f57aa000640 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 0x559ceb608a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.241+0000 7f57aa000640 1 -- 172.21.15.18:0/1155144075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ceb524920 con 0x559ceb60b410 2024-04-16T17:21:18.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.242+0000 7f57aa000640 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb608fc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f579c005fd0 tx=0x7f579c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb205e9ee7f4369 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.242+0000 7f578aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:18.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.242+0000 7f578aa00640 1 -- 172.21.15.18:0/1155144075 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c022070 con 0x559ceb60b410 2024-04-16T17:21:18.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.242+0000 7f578aa00640 1 -- 172.21.15.18:0/1155144075 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f579c010980 con 0x559ceb60b410 2024-04-16T17:21:18.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.243+0000 7f57aed75740 1 -- 172.21.15.18:0/1155144075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579009c6d0 con 0x559ceb60b410 2024-04-16T17:21:18.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.243+0000 7f578aa00640 1 -- 172.21.15.18:0/1155144075 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f579c02c050 con 0x559ceb60b410 2024-04-16T17:21:18.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.243+0000 7f57aed75740 10 client.29812 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:18.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.244+0000 7f57aed75740 1 -- 172.21.15.18:0/1155144075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559ceb4f27f0 con 0x559ceb60b410 2024-04-16T17:21:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.246+0000 7f578aa00640 1 -- 172.21.15.18:0/1155144075 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579009c6d0 con 0x559ceb60b410 2024-04-16T17:21:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.246+0000 7f578aa00640 1 -- 172.21.15.18:0/1155144075 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f579c027440 con 0x559ceb60b410 2024-04-16T17:21:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.246+0000 7f57aed75740 10 client.29812 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.246+0000 7f57aed75740 10 client.29812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:21:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.246+0000 7f57aed75740 20 client.29812 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.246+0000 7f57aed75740 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559ceb62c940 0x559ceb62ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.246+0000 7f57aed75740 4 client.29812 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:18.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.246+0000 7f57aed75740 1 -- 172.21.15.18:0/1155144075 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ceb574a40 con 0x559ceb62c940 2024-04-16T17:21:18.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.248+0000 7f57a9600640 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559ceb62c940 0x559ceb62ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:17 smithi043 ceph-mon[36901]: pgmap v527: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 398 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 511 op/s 2024-04-16T17:21:18.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:17 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3651438744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:21:18.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.252+0000 7f57a9600640 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559ceb62c940 0x559ceb62ed70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36c9e9b928fb09d server_cookie=8071ab584b8bdebb in_seq=0 out_seq=0 2024-04-16T17:21:18.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.252+0000 7f578aa00640 10 client.29812 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:21:18.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.257+0000 7f578aa00640 1 -- 172.21.15.18:0/1155144075 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ceb574a40 con 0x559ceb62c940 2024-04-16T17:21:18.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.257+0000 7f578aa00640 10 client.29812 handle_command_reply: tid=0 2024-04-16T17:21:18.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.258+0000 7f57aed75740 1 client.29812 shutdown 2024-04-16T17:21:18.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f57aed75740 1 -- 172.21.15.18:0/1155144075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 msgr2=0x559ceb608fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f57aed75740 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb608fc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f579c005fd0 tx=0x7f579c004ab0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f57aed75740 1 -- 172.21.15.18:0/1155144075 shutdown_connections 2024-04-16T17:21:18.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f57aed75740 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559ceb62c940 0x559ceb62ed70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f57aed75740 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb608fc0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f57aed75740 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb602830 0x559ceb608a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f578aa00640 0 client.29812 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:21:18.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f57aed75740 1 --2- 172.21.15.18:0/1155144075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb601f10 0x559ceb608540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.259+0000 7f57aed75740 1 -- 172.21.15.18:0/1155144075 >> 172.21.15.18:0/1155144075 conn(0x559ceb5fc640 msgr2=0x559ceb5fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.260+0000 7f57aed75740 1 -- 172.21.15.18:0/1155144075 shutdown_connections 2024-04-16T17:21:18.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.260+0000 7f57aed75740 1 -- 172.21.15.18:0/1155144075 wait complete. 2024-04-16T17:21:18.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.261+0000 7f57aed75740 20 client.29812 trim_cache size 0 max 16384 2024-04-16T17:21:18.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.293+0000 7f57aed75740 1 Processor -- start 2024-04-16T17:21:18.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.293+0000 7f57aed75740 1 -- start start 2024-04-16T17:21:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57aed75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb60ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57aed75740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6280d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57aed75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb60b410 0x559ceb634640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57aed75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41b8620 con 0x559ceb60b410 2024-04-16T17:21:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57aed75740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4071ce0 con 0x559ceb602830 2024-04-16T17:21:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57aed75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57a406fb90 con 0x559ceb601f10 2024-04-16T17:21:18.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57a9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb60ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57a9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb60ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44420/0 (socket says 172.21.15.18:44420) 2024-04-16T17:21:18.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.294+0000 7f57a9600640 1 -- 172.21.15.18:0/989663513 learned_addr learned my addr 172.21.15.18:0/989663513 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.295+0000 7f57a8c00640 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6280d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.295+0000 7f57aa000640 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb60b410 0x559ceb634640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.295+0000 7f57a9600640 1 -- 172.21.15.18:0/989663513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 msgr2=0x559ceb6280d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.295+0000 7f57a9600640 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6280d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.295+0000 7f57a9600640 1 -- 172.21.15.18:0/989663513 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb60b410 msgr2=0x559ceb634640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.295+0000 7f57a9600640 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb60b410 0x559ceb634640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.295+0000 7f57a9600640 1 -- 172.21.15.18:0/989663513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ceb2793f0 con 0x559ceb601f10 2024-04-16T17:21:18.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.296+0000 7f57a9600640 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb60ac70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5798000c00 tx=0x7f579802df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f454b37c7858643f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.297+0000 7f5795c00640 1 -- 172.21.15.18:0/989663513 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579804e070 con 0x559ceb601f10 2024-04-16T17:21:18.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.299+0000 7f5795c00640 1 -- 172.21.15.18:0/989663513 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579802c880 con 0x559ceb601f10 2024-04-16T17:21:18.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.299+0000 7f5795c00640 1 -- 172.21.15.18:0/989663513 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57980046c0 con 0x559ceb601f10 2024-04-16T17:21:18.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.300+0000 7f57aed75740 1 -- 172.21.15.18:0/989663513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 msgr2=0x559ceb60ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.300+0000 7f57aed75740 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb60ac70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5798000c00 tx=0x7f579802df90 comp rx=0 tx=0).stop 2024-04-16T17:21:18.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.300+0000 7f57aed75740 1 -- 172.21.15.18:0/989663513 shutdown_connections 2024-04-16T17:21:18.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.300+0000 7f57aed75740 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb60b410 0x559ceb634640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.301+0000 7f57aed75740 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6280d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.301+0000 7f57aed75740 1 --2- 172.21.15.18:0/989663513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb601f10 0x559ceb60ac70 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.301+0000 7f57aed75740 1 -- 172.21.15.18:0/989663513 >> 172.21.15.18:0/989663513 conn(0x559ceb5fc640 msgr2=0x559ceb5fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.301+0000 7f57aed75740 1 -- 172.21.15.18:0/989663513 shutdown_connections 2024-04-16T17:21:18.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.301+0000 7f57aed75740 1 -- 172.21.15.18:0/989663513 wait complete. 2024-04-16T17:21:18.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.302+0000 7f57aed75740 1 -- start start 2024-04-16T17:21:18.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.302+0000 7f57aed75740 1 Processor -- start 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.303+0000 7f57aed75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb601f10 0x559ceb6203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.303+0000 7f57aed75740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.303+0000 7f57aed75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb691eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.303+0000 7f57aed75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40701e0 con 0x559ceb601f10 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.303+0000 7f57aed75740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4071610 con 0x559ceb602830 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.303+0000 7f57aed75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4070870 con 0x559ceb60b410 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.303+0000 7f57aed75740 10 client.0 _ll_register_callbacks cb 0x7f579c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.304+0000 7f57a9600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb601f10 0x559ceb6203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.304+0000 7f57a9600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb601f10 0x559ceb6203a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54820/0 (socket says 172.21.15.18:54820) 2024-04-16T17:21:18.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.304+0000 7f57a9600640 1 -- 172.21.15.18:0/2825650836 learned_addr learned my addr 172.21.15.18:0/2825650836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.304+0000 7f57a8c00640 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.304+0000 7f57aa000640 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb691eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f57a8c00640 1 -- 172.21.15.18:0/2825650836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 msgr2=0x559ceb691eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f57a8c00640 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb691eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f57a8c00640 1 -- 172.21.15.18:0/2825650836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb601f10 msgr2=0x559ceb6203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f57a8c00640 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb601f10 0x559ceb6203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f57a8c00640 1 -- 172.21.15.18:0/2825650836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ceb3437d0 con 0x559ceb602830 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f57a8c00640 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6208e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f579009bc80 tx=0x7f57900cd720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ffb96f85d26b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f578aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f578aa00640 1 -- 172.21.15.18:0/2825650836 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790020040 con 0x559ceb602830 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.305+0000 7f578aa00640 1 -- 172.21.15.18:0/2825650836 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7f579009a2d0 con 0x559ceb602830 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.306+0000 7f57aed75740 1 -- 172.21.15.18:0/2825650836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579802c880 con 0x559ceb602830 2024-04-16T17:21:18.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.306+0000 7f578aa00640 1 -- 172.21.15.18:0/2825650836 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f579009bda0 con 0x559ceb602830 2024-04-16T17:21:18.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.306+0000 7f57aed75740 10 client.30011 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:18.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.306+0000 7f57aed75740 1 -- 172.21.15.18:0/2825650836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559ceb693360 con 0x559ceb602830 2024-04-16T17:21:18.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.306+0000 7f578aa00640 1 -- 172.21.15.18:0/2825650836 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579802c880 con 0x559ceb602830 2024-04-16T17:21:18.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.307+0000 7f578aa00640 1 -- 172.21.15.18:0/2825650836 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f57900cda00 con 0x559ceb602830 2024-04-16T17:21:18.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.307+0000 7f57aed75740 10 client.30011 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:18.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.307+0000 7f57aed75740 10 client.30011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:21:18.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.307+0000 7f57aed75740 20 client.30011 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:18.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.307+0000 7f57aed75740 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559ceb693a60 0x559ceb695e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.307+0000 7f57aed75740 4 client.30011 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:21:18.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.307+0000 7f57aed75740 1 -- 172.21.15.18:0/2825650836 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ceb56e8b0 con 0x559ceb693a60 2024-04-16T17:21:18.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.308+0000 7f57a9600640 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559ceb693a60 0x559ceb695e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.309+0000 7f578aa00640 10 client.30011 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:21:18.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.311+0000 7f57a9600640 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559ceb693a60 0x559ceb695e80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25948e6a61d2127c server_cookie=1a2933a7a558430d in_seq=0 out_seq=0 2024-04-16T17:21:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.314+0000 7f578aa00640 1 -- 172.21.15.18:0/2825650836 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ceb56e8b0 con 0x559ceb693a60 2024-04-16T17:21:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.314+0000 7f578aa00640 10 client.30011 handle_command_reply: tid=0 2024-04-16T17:21:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.314+0000 7f57aed75740 1 client.30011 shutdown 2024-04-16T17:21:18.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.315+0000 7f57aed75740 1 -- 172.21.15.18:0/2825650836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 msgr2=0x559ceb6208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.315+0000 7f57aed75740 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6208e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f579009bc80 tx=0x7f57900cd720 comp rx=0 tx=0).stop 2024-04-16T17:21:18.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.316+0000 7f57aed75740 1 -- 172.21.15.18:0/2825650836 shutdown_connections 2024-04-16T17:21:18.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.316+0000 7f57aed75740 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559ceb693a60 0x559ceb695e80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.316+0000 7f57aed75740 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ceb60b410 0x559ceb691eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.316+0000 7f57aed75740 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559ceb602830 0x559ceb6208e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.316+0000 7f57aed75740 1 --2- 172.21.15.18:0/2825650836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559ceb601f10 0x559ceb6203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.316+0000 7f578aa00640 0 client.30011 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:21:18.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.316+0000 7f57aed75740 1 -- 172.21.15.18:0/2825650836 >> 172.21.15.18:0/2825650836 conn(0x559ceb5fc640 msgr2=0x559ceb5fb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.317+0000 7f57aed75740 1 -- 172.21.15.18:0/2825650836 shutdown_connections 2024-04-16T17:21:18.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.317+0000 7f57aed75740 1 -- 172.21.15.18:0/2825650836 wait complete. 2024-04-16T17:21:18.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.318+0000 7f57aed75740 20 client.30011 trim_cache size 0 max 16384 2024-04-16T17:21:18.318 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:21:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 -- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f578c0a3ea0 msgr2=0x7f578c0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f578c0a3ea0 0x7f578c0a6360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f579c0099a0 tx=0x7f579c009500 comp rx=0 tx=0).stop 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 -- 172.21.15.18:0/3986717858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a4106030 msgr2=0x7f57a41a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a4106030 0x7f57a41a0bd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f579000d6d0 tx=0x7f579000db90 comp rx=0 tx=0).stop 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 -- 172.21.15.18:0/3986717858 shutdown_connections 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f578c0a3ea0 0x7f578c0a6360 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57a419a750 0x7f57a419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a4106030 0x7f57a41a0bd0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 --2- 172.21.15.18:0/3986717858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57a4100f60 0x7f57a41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 -- 172.21.15.18:0/3986717858 >> 172.21.15.18:0/3986717858 conn(0x7f57a4077ab0 msgr2=0x7f57a410e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.319+0000 7f57aaa00640 1 -- 172.21.15.18:0/3986717858 shutdown_connections 2024-04-16T17:21:18.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.321+0000 7f57aaa00640 1 -- 172.21.15.18:0/3986717858 wait complete. 2024-04-16T17:21:18.363 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:21:18.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 52549 12.28 MB/sec execute 90 sec latency 1486.510 ms 2024-04-16T17:21:18.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.706+0000 7fed0dc00640 1 -- 172.21.15.18:0/1488560542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 msgr2=0x7fed0810e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.706+0000 7fed0dc00640 1 --2- 172.21.15.18:0/1488560542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 0x7fed0810e310 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fed00009ca0 tx=0x7fed0002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.707+0000 7fed0dc00640 1 -- 172.21.15.18:0/1488560542 shutdown_connections 2024-04-16T17:21:18.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.707+0000 7fed0dc00640 1 --2- 172.21.15.18:0/1488560542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0811b7e0 0x7fed0811dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.707+0000 7fed0dc00640 1 --2- 172.21.15.18:0/1488560542 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08110bc0 0x7fed0810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.707+0000 7fed0dc00640 1 --2- 172.21.15.18:0/1488560542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 0x7fed0810e310 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.707+0000 7fed0dc00640 1 -- 172.21.15.18:0/1488560542 >> 172.21.15.18:0/1488560542 conn(0x7fed0806d0a0 msgr2=0x7fed0806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.716+0000 7fed0dc00640 1 -- 172.21.15.18:0/1488560542 shutdown_connections 2024-04-16T17:21:18.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.716+0000 7fed0dc00640 1 -- 172.21.15.18:0/1488560542 wait complete. 2024-04-16T17:21:18.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.718+0000 7fed0dc00640 1 Processor -- start 2024-04-16T17:21:18.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.721+0000 7fed0dc00640 1 -- start start 2024-04-16T17:21:18.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed0dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 0x7fed08118d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed0dc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08110bc0 0x7fed08119290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed0dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0811b7e0 0x7fed08111fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed0dc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed08121190 con 0x7fed08110bc0 2024-04-16T17:21:18.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed06a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 0x7fed08118d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed06a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 0x7fed08118d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40262/0 (socket says 172.21.15.18:40262) 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed06a00640 1 -- 172.21.15.18:0/2457312703 learned_addr learned my addr 172.21.15.18:0/2457312703 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed07400640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0811b7e0 0x7fed08111fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed08121010 con 0x7fed08110750 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.722+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed08121310 con 0x7fed0811b7e0 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.723+0000 7fed07400640 1 -- 172.21.15.18:0/2457312703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 msgr2=0x7fed08118d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.723+0000 7fed07400640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 0x7fed08118d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.723+0000 7fed07400640 1 -- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08110bc0 msgr2=0x7fed08119290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.723+0000 7fed07400640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08110bc0 0x7fed08119290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.723+0000 7fed07400640 1 -- 172.21.15.18:0/2457312703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed08112870 con 0x7fed0811b7e0 2024-04-16T17:21:18.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.723+0000 7fed06a00640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 0x7fed08118d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:18.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.726+0000 7fed07400640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0811b7e0 0x7fed08111fb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fecf400e940 tx=0x7fecf400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.726+0000 7fecff400640 1 -- 172.21.15.18:0/2457312703 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf400ccc0 con 0x7fed0811b7e0 2024-04-16T17:21:18.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.726+0000 7fecff400640 1 -- 172.21.15.18:0/2457312703 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4005020 con 0x7fed0811b7e0 2024-04-16T17:21:18.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.726+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed08112b60 con 0x7fed0811b7e0 2024-04-16T17:21:18.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.726+0000 7fecff400640 1 -- 172.21.15.18:0/2457312703 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4004a00 con 0x7fed0811b7e0 2024-04-16T17:21:18.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.726+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed08116b60 con 0x7fed0811b7e0 2024-04-16T17:21:18.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.738+0000 7fecff400640 1 -- 172.21.15.18:0/2457312703 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fecf40051d0 con 0x7fed0811b7e0 2024-04-16T17:21:18.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.740+0000 7fed105fd740 1 Processor -- start 2024-04-16T17:21:18.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.742+0000 7fecff400640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fecec0a3da0 0x7fecec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.742+0000 7fecff400640 1 -- 172.21.15.18:0/2457312703 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fecf40d0b80 con 0x7fed0811b7e0 2024-04-16T17:21:18.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.743+0000 7fed105fd740 1 -- start start 2024-04-16T17:21:18.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.744+0000 7fed105fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d43d50 0x5585b0d44130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.744+0000 7fed105fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.744+0000 7fed105fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d4f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.744+0000 7fed105fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5585b0ca9020 con 0x5585b0d43d50 2024-04-16T17:21:18.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.744+0000 7fed105fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5585b0c4f460 con 0x5585b0d4d250 2024-04-16T17:21:18.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.744+0000 7fed105fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5585b0ca4fe0 con 0x5585b0d44670 2024-04-16T17:21:18.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.745+0000 7fed06a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d43d50 0x5585b0d44130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.745+0000 7fed06a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d43d50 0x5585b0d44130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54856/0 (socket says 172.21.15.18:54856) 2024-04-16T17:21:18.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.745+0000 7fed06a00640 1 -- 172.21.15.18:0/982032546 learned_addr learned my addr 172.21.15.18:0/982032546 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.745+0000 7fed06000640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fecec0a3da0 0x7fecec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.745+0000 7fed07400640 1 --2- 172.21.15.18:0/982032546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d4f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.746+0000 7fed06a00640 1 -- 172.21.15.18:0/982032546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 msgr2=0x5585b0d4cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:18.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.746+0000 7fed06a00640 1 --2- 172.21.15.18:0/982032546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.746+0000 7fed06a00640 1 -- 172.21.15.18:0/982032546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 msgr2=0x5585b0d4f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.746+0000 7fed06a00640 1 --2- 172.21.15.18:0/982032546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d4f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.746+0000 7fed06a00640 1 -- 172.21.15.18:0/982032546 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b0c57040 con 0x5585b0d43d50 2024-04-16T17:21:18.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.747+0000 7fed06a00640 1 --2- 172.21.15.18:0/982032546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d43d50 0x5585b0d44130 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fed00037040 tx=0x7fed000099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f31fce898f921d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.747+0000 7fed06000640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fecec0a3da0 0x7fecec0a6260 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed08113590 tx=0x7fecf8005bf0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.748+0000 7fecf3e00640 1 -- 172.21.15.18:0/982032546 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00043070 con 0x5585b0d43d50 2024-04-16T17:21:18.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.748+0000 7fecf3e00640 1 -- 172.21.15.18:0/982032546 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00032c90 con 0x5585b0d43d50 2024-04-16T17:21:18.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.748+0000 7fecf3e00640 1 -- 172.21.15.18:0/982032546 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00039040 con 0x5585b0d43d50 2024-04-16T17:21:18.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.751+0000 7fed105fd740 1 -- 172.21.15.18:0/982032546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d43d50 msgr2=0x5585b0d44130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.753+0000 7fed105fd740 1 --2- 172.21.15.18:0/982032546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d43d50 0x5585b0d44130 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fed00037040 tx=0x7fed000099b0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.755+0000 7fed105fd740 1 -- 172.21.15.18:0/982032546 shutdown_connections 2024-04-16T17:21:18.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.755+0000 7fed105fd740 1 --2- 172.21.15.18:0/982032546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d4f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.755+0000 7fed105fd740 1 --2- 172.21.15.18:0/982032546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.755+0000 7fed105fd740 1 --2- 172.21.15.18:0/982032546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d43d50 0x5585b0d44130 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.755+0000 7fed105fd740 1 -- 172.21.15.18:0/982032546 >> 172.21.15.18:0/982032546 conn(0x5585b0d3e480 msgr2=0x5585b0d3e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.755+0000 7fed105fd740 1 -- 172.21.15.18:0/982032546 shutdown_connections 2024-04-16T17:21:18.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.755+0000 7fed105fd740 1 -- 172.21.15.18:0/982032546 wait complete. 2024-04-16T17:21:18.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.756+0000 7fed105fd740 1 -- start start 2024-04-16T17:21:18.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.756+0000 7fed105fd740 1 Processor -- start 2024-04-16T17:21:18.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.757+0000 7fed105fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d43d50 0x5585b0d4a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.757+0000 7fed105fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.757+0000 7fed105fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d4d250 0x5585b0d4ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.758+0000 7fed06a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d43d50 0x5585b0d4a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.758+0000 7fed06000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.759+0000 7fed06000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44446/0 (socket says 172.21.15.18:44446) 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.760+0000 7fed07400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d4d250 0x5585b0d4ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.760+0000 7fed105fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5585b0d4ffe0 con 0x5585b0d4d250 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.760+0000 7fed105fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5585b0c62830 con 0x5585b0d43d50 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.760+0000 7fed105fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5585b0d50160 con 0x5585b0d44670 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.760+0000 7fed07400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d4d250 0x5585b0d4ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54870/0 (socket says 172.21.15.18:54870) 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.760+0000 7fed07400640 1 -- 172.21.15.18:0/1177491442 learned_addr learned my addr 172.21.15.18:0/1177491442 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.760+0000 7fed105fd740 10 client.0 _ll_register_callbacks cb 0x5585b0c6a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:18.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.759+0000 7fed06a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d43d50 0x5585b0d4a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40270/0 (socket says 172.21.15.18:40270) 2024-04-16T17:21:18.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.761+0000 7fed06a00640 1 -- 172.21.15.18:0/1177491442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 msgr2=0x5585b0d4a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.761+0000 7fed06a00640 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.761+0000 7fed06a00640 1 -- 172.21.15.18:0/1177491442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d4d250 msgr2=0x5585b0d4ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.761+0000 7fed06a00640 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d4d250 0x5585b0d4ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.762+0000 7fed06a00640 1 -- 172.21.15.18:0/1177491442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b0c66920 con 0x5585b0d43d50 2024-04-16T17:21:18.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.765+0000 7fed06a00640 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d43d50 0x5585b0d4a310 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fed00005e30 tx=0x7fed00032a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c121b0b4912cc845 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.765+0000 7fecf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:18.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.766+0000 7fecf0c00640 1 -- 172.21.15.18:0/1177491442 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0004c030 con 0x5585b0d43d50 2024-04-16T17:21:18.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.766+0000 7fecf0c00640 1 -- 172.21.15.18:0/1177491442 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fed0004d070 con 0x5585b0d43d50 2024-04-16T17:21:18.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.767+0000 7fed105fd740 1 -- 172.21.15.18:0/1177491442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed00032c90 con 0x5585b0d43d50 2024-04-16T17:21:18.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.767+0000 7fecf0c00640 1 -- 172.21.15.18:0/1177491442 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fed00057050 con 0x5585b0d43d50 2024-04-16T17:21:18.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.768+0000 7fed105fd740 10 client.30023 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:18.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.768+0000 7fed105fd740 1 -- 172.21.15.18:0/1177491442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5585b0c347f0 con 0x5585b0d43d50 2024-04-16T17:21:18.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.768+0000 7fecf0c00640 1 -- 172.21.15.18:0/1177491442 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00032c90 con 0x5585b0d43d50 2024-04-16T17:21:18.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.768+0000 7fecf0c00640 1 -- 172.21.15.18:0/1177491442 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fed000484e0 con 0x5585b0d43d50 2024-04-16T17:21:18.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.769+0000 7fed105fd740 10 client.30023 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:18.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.769+0000 7fed105fd740 10 client.30023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:18.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.769+0000 7fed105fd740 20 client.30023 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:18.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.769+0000 7fed105fd740 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5585b0d5d490 0x5585b0d5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.769+0000 7fed105fd740 4 client.30023 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:18.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.769+0000 7fed105fd740 1 -- 172.21.15.18:0/1177491442 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585b0cb6a40 con 0x5585b0d5d490 2024-04-16T17:21:18.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.770+0000 7fed06000640 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5585b0d5d490 0x5585b0d5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.771+0000 7fed06000640 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5585b0d5d490 0x5585b0d5f880 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e80c865d7c6d27c server_cookie=8849476664ba6e78 in_seq=0 out_seq=0 2024-04-16T17:21:18.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.772+0000 7fecf0c00640 10 client.30023 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:18.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.777+0000 7fecf0c00640 1 -- 172.21.15.18:0/1177491442 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585b0cb6a40 con 0x5585b0d5d490 2024-04-16T17:21:18.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.777+0000 7fecf0c00640 10 client.30023 handle_command_reply: tid=0 2024-04-16T17:21:18.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.778+0000 7fed105fd740 1 client.30023 shutdown 2024-04-16T17:21:18.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fed105fd740 1 -- 172.21.15.18:0/1177491442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d43d50 msgr2=0x5585b0d4a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fed105fd740 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d43d50 0x5585b0d4a310 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fed00005e30 tx=0x7fed00032a60 comp rx=0 tx=0).stop 2024-04-16T17:21:18.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fed105fd740 1 -- 172.21.15.18:0/1177491442 shutdown_connections 2024-04-16T17:21:18.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fed105fd740 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5585b0d5d490 0x5585b0d5f880 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fed105fd740 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d4d250 0x5585b0d4ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fed105fd740 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fed105fd740 1 --2- 172.21.15.18:0/1177491442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d43d50 0x5585b0d4a310 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fed105fd740 1 -- 172.21.15.18:0/1177491442 >> 172.21.15.18:0/1177491442 conn(0x5585b0d3e480 msgr2=0x5585b0d401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.780+0000 7fecf0c00640 0 client.30023 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:18.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.781+0000 7fed105fd740 1 -- 172.21.15.18:0/1177491442 shutdown_connections 2024-04-16T17:21:18.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.781+0000 7fed105fd740 1 -- 172.21.15.18:0/1177491442 wait complete. 2024-04-16T17:21:18.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.781+0000 7fed105fd740 20 client.30023 trim_cache size 0 max 16384 2024-04-16T17:21:18.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.815+0000 7fed105fd740 1 Processor -- start 2024-04-16T17:21:18.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.816+0000 7fed105fd740 1 -- start start 2024-04-16T17:21:18.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.816+0000 7fed105fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.816+0000 7fed105fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d3ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.816+0000 7fed105fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.816+0000 7fed105fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed081c96b0 con 0x5585b0d40520 2024-04-16T17:21:18.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.816+0000 7fed105fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed081c6f80 con 0x5585b0d4d250 2024-04-16T17:21:18.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.816+0000 7fed105fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed081c4df0 con 0x5585b0d44670 2024-04-16T17:21:18.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.817+0000 7fed07400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.817+0000 7fed06000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d3ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.817+0000 7fed06000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d3ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40280/0 (socket says 172.21.15.18:40280) 2024-04-16T17:21:18.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.817+0000 7fed06000640 1 -- 172.21.15.18:0/268579396 learned_addr learned my addr 172.21.15.18:0/268579396 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.817+0000 7fed07400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54884/0 (socket says 172.21.15.18:54884) 2024-04-16T17:21:18.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.817+0000 7fed06a00640 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.817+0000 7fed06000640 1 -- 172.21.15.18:0/268579396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 msgr2=0x5585b0d4ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.817+0000 7fed06000640 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.818+0000 7fed06000640 1 -- 172.21.15.18:0/268579396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 msgr2=0x5585b0d628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.818+0000 7fed06000640 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.818+0000 7fed06000640 1 -- 172.21.15.18:0/268579396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b09bb3f0 con 0x5585b0d4d250 2024-04-16T17:21:18.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.818+0000 7fed07400640 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:18.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.818+0000 7fed06000640 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d3ffe0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fecf8011040 tx=0x7fecf800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=288731dde599b10b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.819+0000 7fecf3400640 1 -- 172.21.15.18:0/268579396 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf801ed30 con 0x5585b0d4d250 2024-04-16T17:21:18.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.819+0000 7fecf3400640 1 -- 172.21.15.18:0/268579396 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf8016040 con 0x5585b0d4d250 2024-04-16T17:21:18.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.820+0000 7fed105fd740 1 -- 172.21.15.18:0/268579396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 msgr2=0x5585b0d3ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.820+0000 7fed105fd740 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d3ffe0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fecf8011040 tx=0x7fecf800f000 comp rx=0 tx=0).stop 2024-04-16T17:21:18.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.820+0000 7fed105fd740 1 -- 172.21.15.18:0/268579396 shutdown_connections 2024-04-16T17:21:18.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.821+0000 7fed105fd740 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.821+0000 7fed105fd740 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0d3ffe0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.821+0000 7fed105fd740 1 --2- 172.21.15.18:0/268579396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d4ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.821+0000 7fed105fd740 1 -- 172.21.15.18:0/268579396 >> 172.21.15.18:0/268579396 conn(0x5585b0d3e480 msgr2=0x5585b0d3f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.821+0000 7fed105fd740 1 -- 172.21.15.18:0/268579396 shutdown_connections 2024-04-16T17:21:18.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.821+0000 7fed105fd740 1 -- 172.21.15.18:0/268579396 wait complete. 2024-04-16T17:21:18.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.822+0000 7fed105fd740 1 -- start start 2024-04-16T17:21:18.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.823+0000 7fed105fd740 1 Processor -- start 2024-04-16T17:21:18.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.823+0000 7fed105fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.823+0000 7fed105fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d60f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed105fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0dc2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed06000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d60f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed06000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d60f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44460/0 (socket says 172.21.15.18:44460) 2024-04-16T17:21:18.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed06000640 1 -- 172.21.15.18:0/1107877132 learned_addr learned my addr 172.21.15.18:0/1107877132 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:18.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed081c5440 con 0x5585b0d40520 2024-04-16T17:21:18.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed081c68b0 con 0x5585b0d4d250 2024-04-16T17:21:18.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed081c5b10 con 0x5585b0d44670 2024-04-16T17:21:18.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed06a00640 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed105fd740 10 client.0 _ll_register_callbacks cb 0x7fed00009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:18.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.824+0000 7fed07400640 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0dc2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.825+0000 7fed06000640 1 -- 172.21.15.18:0/1107877132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 msgr2=0x5585b0dc2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.825+0000 7fed06000640 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0dc2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.825+0000 7fed06000640 1 -- 172.21.15.18:0/1107877132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 msgr2=0x5585b0d609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.825+0000 7fed06000640 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.825+0000 7fed06000640 1 -- 172.21.15.18:0/1107877132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b0a857d0 con 0x5585b0d44670 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.825+0000 7fed06000640 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d60f30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fecf8000c00 tx=0x7fecf800c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62309ab9c13b5315 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.825+0000 7fecf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.826+0000 7fecf0c00640 1 -- 172.21.15.18:0/1107877132 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf80037f0 con 0x5585b0d44670 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.826+0000 7fecf0c00640 1 -- 172.21.15.18:0/1107877132 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fecf8003ad0 con 0x5585b0d44670 2024-04-16T17:21:18.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.826+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf8016040 con 0x5585b0d44670 2024-04-16T17:21:18.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.826+0000 7fecf0c00640 1 -- 172.21.15.18:0/1107877132 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fecf8003cd0 con 0x5585b0d44670 2024-04-16T17:21:18.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.826+0000 7fed105fd740 10 client.29833 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:18.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.826+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5585b0dc3c70 con 0x5585b0d44670 2024-04-16T17:21:18.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.827+0000 7fecf0c00640 1 -- 172.21.15.18:0/1107877132 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf8016040 con 0x5585b0d44670 2024-04-16T17:21:18.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.828+0000 7fecf0c00640 1 -- 172.21.15.18:0/1107877132 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fecf800fae0 con 0x5585b0d44670 2024-04-16T17:21:18.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.828+0000 7fed105fd740 10 client.29833 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:18.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.828+0000 7fed105fd740 10 client.29833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:18.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.828+0000 7fed105fd740 20 client.29833 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:18.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.828+0000 7fed105fd740 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5585b0dc43c0 0x5585b0dc67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:18.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.828+0000 7fed105fd740 4 client.29833 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:21:18.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.828+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585b0cb08b0 con 0x5585b0dc43c0 2024-04-16T17:21:18.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.828+0000 7fed06a00640 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5585b0dc43c0 0x5585b0dc67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:18.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.829+0000 7fed06a00640 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5585b0dc43c0 0x5585b0dc67e0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28710fd20bc7b57c server_cookie=35d94736404033ca in_seq=0 out_seq=0 2024-04-16T17:21:18.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.829+0000 7fecf0c00640 10 client.29833 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:18.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.829+0000 7fecf0c00640 1 -- 172.21.15.18:0/1107877132 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585b0cb08b0 con 0x5585b0dc43c0 2024-04-16T17:21:18.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.829+0000 7fecf0c00640 10 client.29833 handle_command_reply: tid=0 2024-04-16T17:21:18.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.830+0000 7fed105fd740 1 client.29833 shutdown 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 msgr2=0x5585b0d60f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d60f30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fecf8000c00 tx=0x7fecf800c990 comp rx=0 tx=0).stop 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 shutdown_connections 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5585b0dc43c0 0x5585b0dc67e0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585b0d4d250 0x5585b0dc2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585b0d44670 0x5585b0d60f30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 --2- 172.21.15.18:0/1107877132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5585b0d40520 0x5585b0d609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 >> 172.21.15.18:0/1107877132 conn(0x5585b0d3e480 msgr2=0x5585b0d3cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fecf0c00640 0 client.29833 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:18.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 shutdown_connections 2024-04-16T17:21:18.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.831+0000 7fed105fd740 1 -- 172.21.15.18:0/1107877132 wait complete. 2024-04-16T17:21:18.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.832+0000 7fed105fd740 20 client.29833 trim_cache size 0 max 16384 2024-04-16T17:21:18.832 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:21:18.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.833+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fecec0a3da0 msgr2=0x7fecec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.833+0000 7fed0dc00640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fecec0a3da0 0x7fecec0a6260 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed08113590 tx=0x7fecf8005bf0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.833+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0811b7e0 msgr2=0x7fed08111fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.833+0000 7fed0dc00640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0811b7e0 0x7fed08111fb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fecf400e940 tx=0x7fecf400ee00 comp rx=0 tx=0).stop 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.834+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 shutdown_connections 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.834+0000 7fed0dc00640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fecec0a3da0 0x7fecec0a6260 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.834+0000 7fed0dc00640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0811b7e0 0x7fed08111fb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.834+0000 7fed0dc00640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed08110bc0 0x7fed08119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.834+0000 7fed0dc00640 1 --2- 172.21.15.18:0/2457312703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed08110750 0x7fed08118d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:18.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.834+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 >> 172.21.15.18:0/2457312703 conn(0x7fed0806d0a0 msgr2=0x7fed0811a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:18.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.834+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 shutdown_connections 2024-04-16T17:21:18.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:18.834+0000 7fed0dc00640 1 -- 172.21.15.18:0/2457312703 wait complete. 2024-04-16T17:21:19.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 52780 12.24 MB/sec execute 91 sec latency 1052.680 ms 2024-04-16T17:21:19.866 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:21:19.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:21:19.866 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:21:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:19 smithi022 ceph-mon[36023]: pgmap v528: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 398 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 318 op/s 2024-04-16T17:21:20.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.214+0000 7ffa1a600640 1 -- 172.21.15.18:0/3502898463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 msgr2=0x7ffa1410b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.214+0000 7ffa1a600640 1 --2- 172.21.15.18:0/3502898463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 0x7ffa1410b090 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009930 tx=0x7ffa0002bc30 comp rx=0 tx=0).stop 2024-04-16T17:21:20.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.214+0000 7ffa1a600640 1 -- 172.21.15.18:0/3502898463 shutdown_connections 2024-04-16T17:21:20.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.214+0000 7ffa1a600640 1 --2- 172.21.15.18:0/3502898463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 0x7ffa1410b090 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.214+0000 7ffa1a600640 1 --2- 172.21.15.18:0/3502898463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa1410a1b0 0x7ffa1410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.214+0000 7ffa1a600640 1 --2- 172.21.15.18:0/3502898463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa14109890 0x7ffa14109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.214+0000 7ffa1a600640 1 -- 172.21.15.18:0/3502898463 >> 172.21.15.18:0/3502898463 conn(0x7ffa1406d0a0 msgr2=0x7ffa1406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.215+0000 7ffa1a600640 1 -- 172.21.15.18:0/3502898463 shutdown_connections 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.215+0000 7ffa1a600640 1 -- 172.21.15.18:0/3502898463 wait complete. 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.216+0000 7ffa1a600640 1 Processor -- start 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.216+0000 7ffa1a600640 1 -- start start 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.217+0000 7ffa1a600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa14109890 0x7ffa140791a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.217+0000 7ffa1a600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa1410a1b0 0x7ffa140796e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.217+0000 7ffa1a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 0x7ffa14074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.217+0000 7ffa1a600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14118af0 con 0x7ffa14109890 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.217+0000 7ffa1a600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14118970 con 0x7ffa1410a1b0 2024-04-16T17:21:20.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.217+0000 7ffa1a600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14118c70 con 0x7ffa1410abe0 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.218+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa1410a1b0 0x7ffa140796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.218+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa1410a1b0 0x7ffa140796e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40312/0 (socket says 172.21.15.18:40312) 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.218+0000 7ffa12a00640 1 -- 172.21.15.18:0/516235828 learned_addr learned my addr 172.21.15.18:0/516235828 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.218+0000 7ffa13e00640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 0x7ffa14074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.219+0000 7ffa13e00640 1 -- 172.21.15.18:0/516235828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa1410a1b0 msgr2=0x7ffa140796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.219+0000 7ffa13e00640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa1410a1b0 0x7ffa140796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.219+0000 7ffa13e00640 1 -- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa14109890 msgr2=0x7ffa140791a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.219+0000 7ffa13e00640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa14109890 0x7ffa140791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.219+0000 7ffa13e00640 1 -- 172.21.15.18:0/516235828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa14074bc0 con 0x7ffa1410abe0 2024-04-16T17:21:20.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.220+0000 7ffa13e00640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 0x7ffa14074300 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002c170 tx=0x7ffa000045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.220+0000 7ffa0be00640 1 -- 172.21.15.18:0/516235828 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00043020 con 0x7ffa1410abe0 2024-04-16T17:21:20.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.220+0000 7ffa1a600640 1 -- 172.21.15.18:0/516235828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa14074e50 con 0x7ffa1410abe0 2024-04-16T17:21:20.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.220+0000 7ffa1a600640 1 -- 172.21.15.18:0/516235828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa140810a0 con 0x7ffa1410abe0 2024-04-16T17:21:20.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.220+0000 7ffa0be00640 1 -- 172.21.15.18:0/516235828 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa00004c10 con 0x7ffa1410abe0 2024-04-16T17:21:20.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.220+0000 7ffa0be00640 1 -- 172.21.15.18:0/516235828 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000038e0 con 0x7ffa1410abe0 2024-04-16T17:21:20.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.226+0000 7ffa0be00640 1 -- 172.21.15.18:0/516235828 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa00046020 con 0x7ffa1410abe0 2024-04-16T17:21:20.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.226+0000 7ffa1d3a9740 1 Processor -- start 2024-04-16T17:21:20.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.226+0000 7ffa1d3a9740 1 -- start start 2024-04-16T17:21:20.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409ee4b0 0x55a6409ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a64094acb0 con 0x55a6409ee4b0 2024-04-16T17:21:20.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a6408f1370 con 0x55a6409eedd0 2024-04-16T17:21:20.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a640946c70 con 0x55a6409f79b0 2024-04-16T17:21:20.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa13400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa13400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40314/0 (socket says 172.21.15.18:40314) 2024-04-16T17:21:20.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa13400640 1 -- 172.21.15.18:0/330572707 learned_addr learned my addr 172.21.15.18:0/330572707 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.229+0000 7ffa12a00640 1 --2- 172.21.15.18:0/330572707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.230+0000 7ffa0be00640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9f80a3d10 0x7ff9f80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.230+0000 7ffa0be00640 1 -- 172.21.15.18:0/516235828 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7ffa000f2be0 con 0x7ffa1410abe0 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.230+0000 7ffa12a00640 1 -- 172.21.15.18:0/330572707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 msgr2=0x55a6409f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.230+0000 7ffa12a00640 1 --2- 172.21.15.18:0/330572707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.230+0000 7ffa12a00640 1 -- 172.21.15.18:0/330572707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409ee4b0 msgr2=0x55a6409ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.230+0000 7ffa12a00640 1 --2- 172.21.15.18:0/330572707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409ee4b0 0x55a6409ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.230+0000 7ffa12a00640 1 -- 172.21.15.18:0/330572707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a640910910 con 0x55a6409f79b0 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.231+0000 7ffa12a00640 1 --2- 172.21.15.18:0/330572707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f9da0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ffa14075b00 tx=0x7ffa04009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ffb1d601edb333 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.231+0000 7ffa13e00640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9f80a3d10 0x7ff9f80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.231+0000 7ffa08c00640 1 -- 172.21.15.18:0/330572707 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04018020 con 0x55a6409f79b0 2024-04-16T17:21:20.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.232+0000 7ffa13e00640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9f80a3d10 0x7ff9f80a61d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffa000bac90 tx=0x7ffa0002bc30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.233+0000 7ffa08c00640 1 -- 172.21.15.18:0/330572707 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04010c20 con 0x55a6409f79b0 2024-04-16T17:21:20.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.233+0000 7ffa08c00640 1 -- 172.21.15.18:0/330572707 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04013700 con 0x55a6409f79b0 2024-04-16T17:21:20.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.234+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/330572707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 msgr2=0x55a6409f9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.234+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/330572707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f9da0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ffa14075b00 tx=0x7ffa04009f40 comp rx=0 tx=0).stop 2024-04-16T17:21:20.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.234+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/330572707 shutdown_connections 2024-04-16T17:21:20.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.234+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/330572707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f9da0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.234+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/330572707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.234+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/330572707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409ee4b0 0x55a6409ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.234+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/330572707 >> 172.21.15.18:0/330572707 conn(0x55a6409e8be0 msgr2=0x55a6409e8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.235+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/330572707 shutdown_connections 2024-04-16T17:21:20.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.235+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/330572707 wait complete. 2024-04-16T17:21:20.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.235+0000 7ffa1d3a9740 1 -- start start 2024-04-16T17:21:20.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.236+0000 7ffa1d3a9740 1 Processor -- start 2024-04-16T17:21:20.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.237+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409ee4b0 0x55a6409f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409eedd0 0x55a6409f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409ee4b0 0x55a6409f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409ee4b0 0x55a6409f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40326/0 (socket says 172.21.15.18:40326) 2024-04-16T17:21:20.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a6409fa7a0 con 0x55a6409eedd0 2024-04-16T17:21:20.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a640904730 con 0x55a6409ee4b0 2024-04-16T17:21:20.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a6409fa920 con 0x55a6409f79b0 2024-04-16T17:21:20.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa1d3a9740 10 client.0 _ll_register_callbacks cb 0x55a6408d31e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.238+0000 7ffa12a00640 1 -- 172.21.15.18:0/3253015429 learned_addr learned my addr 172.21.15.18:0/3253015429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.239+0000 7ffa12a00640 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.239+0000 7ffa12a00640 1 -- 172.21.15.18:0/3253015429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 msgr2=0x55a6409f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.239+0000 7ffa12a00640 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.239+0000 7ffa12a00640 1 -- 172.21.15.18:0/3253015429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409eedd0 msgr2=0x55a6409f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.239+0000 7ffa12a00640 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409eedd0 0x55a6409f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.239+0000 7ffa12a00640 1 -- 172.21.15.18:0/3253015429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6407257e0 con 0x55a6409ee4b0 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.240+0000 7ffa12a00640 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409ee4b0 0x55a6409f4a10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ffa04014f30 tx=0x7ffa04009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2721b54b8f84b5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.240+0000 7ff9fd600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:20.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.240+0000 7ff9fd600640 1 -- 172.21.15.18:0/3253015429 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04018030 con 0x55a6409ee4b0 2024-04-16T17:21:20.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.241+0000 7ff9fd600640 1 -- 172.21.15.18:0/3253015429 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7ffa0401d630 con 0x55a6409ee4b0 2024-04-16T17:21:20.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.241+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/3253015429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa04010c20 con 0x55a6409ee4b0 2024-04-16T17:21:20.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.241+0000 7ff9fd600640 1 -- 172.21.15.18:0/3253015429 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffa04010e50 con 0x55a6409ee4b0 2024-04-16T17:21:20.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.241+0000 7ffa1d3a9740 10 client.30038 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:20.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.242+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/3253015429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a640785f70 con 0x55a6409ee4b0 2024-04-16T17:21:20.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.244+0000 7ff9fd600640 1 -- 172.21.15.18:0/3253015429 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04010c20 con 0x55a6409ee4b0 2024-04-16T17:21:20.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.244+0000 7ff9fd600640 1 -- 172.21.15.18:0/3253015429 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ffa04013a10 con 0x55a6409ee4b0 2024-04-16T17:21:20.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.245+0000 7ffa1d3a9740 10 client.30038 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:20.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.245+0000 7ffa1d3a9740 10 client.30038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:20.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.245+0000 7ffa1d3a9740 20 client.30038 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:20.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.245+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a640a07c80 0x55a640a0a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.245+0000 7ffa1d3a9740 4 client.30038 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:20.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.245+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/3253015429 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6409586d0 con 0x55a640a07c80 2024-04-16T17:21:20.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.245+0000 7ffa13400640 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a640a07c80 0x55a640a0a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.246+0000 7ffa13400640 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a640a07c80 0x55a640a0a070 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfceada3b49bef58 server_cookie=c26362ca0c4b4446 in_seq=0 out_seq=0 2024-04-16T17:21:20.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.246+0000 7ff9fd600640 10 client.30038 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:20.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.249+0000 7ff9fd600640 1 -- 172.21.15.18:0/3253015429 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6409586d0 con 0x55a640a07c80 2024-04-16T17:21:20.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.249+0000 7ff9fd600640 10 client.30038 handle_command_reply: tid=0 2024-04-16T17:21:20.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.250+0000 7ffa1d3a9740 1 client.30038 shutdown 2024-04-16T17:21:20.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:19 smithi043 ceph-mon[36901]: pgmap v528: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 398 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 318 op/s 2024-04-16T17:21:20.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/3253015429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409ee4b0 msgr2=0x55a6409f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409ee4b0 0x55a6409f4a10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ffa04014f30 tx=0x7ffa04009470 comp rx=0 tx=0).stop 2024-04-16T17:21:20.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/3253015429 shutdown_connections 2024-04-16T17:21:20.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a640a07c80 0x55a640a0a070 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409f79b0 0x55a6409f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409eedd0 0x55a6409f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/3253015429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409ee4b0 0x55a6409f4a10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/3253015429 >> 172.21.15.18:0/3253015429 conn(0x55a6409e8be0 msgr2=0x55a6409ea930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.251+0000 7ff9fd600640 0 client.30038 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:20.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.252+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/3253015429 shutdown_connections 2024-04-16T17:21:20.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.252+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/3253015429 wait complete. 2024-04-16T17:21:20.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.253+0000 7ffa1d3a9740 20 client.30038 trim_cache size 0 max 16384 2024-04-16T17:21:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.316+0000 7ffa1d3a9740 1 Processor -- start 2024-04-16T17:21:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.317+0000 7ffa1d3a9740 1 -- start start 2024-04-16T17:21:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.317+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409ee4b0 0x55a640a036f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.317+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.317+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409f79b0 0x55a640a0f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.317+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f4012a70 con 0x55a6409f79b0 2024-04-16T17:21:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.317+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f4010340 con 0x55a6409eedd0 2024-04-16T17:21:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.317+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f400e1b0 con 0x55a6409ee4b0 2024-04-16T17:21:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.319+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409ee4b0 0x55a640a036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.319+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409ee4b0 0x55a640a036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44502/0 (socket says 172.21.15.18:44502) 2024-04-16T17:21:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.319+0000 7ffa12a00640 1 -- 172.21.15.18:0/2770504433 learned_addr learned my addr 172.21.15.18:0/2770504433 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.320+0000 7ffa13400640 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.320+0000 7ffa12a00640 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409f79b0 0x55a640a0f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.321+0000 7ffa12a00640 1 -- 172.21.15.18:0/2770504433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 msgr2=0x55a6409f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.321+0000 7ffa12a00640 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.322+0000 7ffa12a00640 1 -- 172.21.15.18:0/2770504433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409f79b0 msgr2=0x55a640a0f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.323+0000 7ffa12a00640 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409f79b0 0x55a640a0f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.323+0000 7ffa12a00640 1 -- 172.21.15.18:0/2770504433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64061d170 con 0x55a6409ee4b0 2024-04-16T17:21:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.323+0000 7ffa12a00640 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409ee4b0 0x55a640a036f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffa04014a10 tx=0x7ffa04006a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d1540e068595a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.324+0000 7ff9ffe00640 1 -- 172.21.15.18:0/2770504433 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0403b070 con 0x55a6409ee4b0 2024-04-16T17:21:20.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.325+0000 7ff9ffe00640 1 -- 172.21.15.18:0/2770504433 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa0401c470 con 0x55a6409ee4b0 2024-04-16T17:21:20.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.325+0000 7ff9ffe00640 1 -- 172.21.15.18:0/2770504433 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04010e80 con 0x55a6409ee4b0 2024-04-16T17:21:20.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.328+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/2770504433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409ee4b0 msgr2=0x55a640a036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.328+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409ee4b0 0x55a640a036f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffa04014a10 tx=0x7ffa04006a30 comp rx=0 tx=0).stop 2024-04-16T17:21:20.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.328+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/2770504433 shutdown_connections 2024-04-16T17:21:20.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.328+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409f79b0 0x55a640a0f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.328+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409eedd0 0x55a6409f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.328+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/2770504433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409ee4b0 0x55a640a036f0 secure :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffa04014a10 tx=0x7ffa04006a30 comp rx=0 tx=0).stop 2024-04-16T17:21:20.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.328+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/2770504433 >> 172.21.15.18:0/2770504433 conn(0x55a6409e8be0 msgr2=0x55a6409df430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.329+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/2770504433 shutdown_connections 2024-04-16T17:21:20.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.329+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/2770504433 wait complete. 2024-04-16T17:21:20.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.330+0000 7ffa1d3a9740 1 -- start start 2024-04-16T17:21:20.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.330+0000 7ffa1d3a9740 1 Processor -- start 2024-04-16T17:21:20.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.332+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409eedd0 0x55a6409fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.332+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409f79b0 0x55a6409fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.332+0000 7ffa1d3a9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409fc380 0x55a640a6d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.332+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f400e800 con 0x55a6409fc380 2024-04-16T17:21:20.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.332+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f400fc70 con 0x55a6409f79b0 2024-04-16T17:21:20.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.332+0000 7ffa1d3a9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f400eed0 con 0x55a6409eedd0 2024-04-16T17:21:20.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.332+0000 7ffa1d3a9740 10 client.0 _ll_register_callbacks cb 0x7ffa04014900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:20.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.334+0000 7ffa13400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409f79b0 0x55a6409fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.334+0000 7ffa13400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409f79b0 0x55a6409fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40342/0 (socket says 172.21.15.18:40342) 2024-04-16T17:21:20.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.334+0000 7ffa13400640 1 -- 172.21.15.18:0/1492265500 learned_addr learned my addr 172.21.15.18:0/1492265500 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.335+0000 7ffa12a00640 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409fc380 0x55a640a6d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.335+0000 7ffa12a00640 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409eedd0 0x55a6409fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.335+0000 7ffa13400640 1 -- 172.21.15.18:0/1492265500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409eedd0 msgr2=0x55a6409fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.335+0000 7ffa13400640 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409eedd0 0x55a6409fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.335+0000 7ffa13400640 1 -- 172.21.15.18:0/1492265500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409fc380 msgr2=0x55a640a6d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.335+0000 7ffa13400640 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409fc380 0x55a640a6d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.335+0000 7ffa13400640 1 -- 172.21.15.18:0/1492265500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a640a6d710 con 0x55a6409f79b0 2024-04-16T17:21:20.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.335+0000 7ffa12a00640 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409eedd0 0x55a6409fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.336+0000 7ffa13400640 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409f79b0 0x55a6409fbdf0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x55a6409f4270 tx=0x7ffa0c016b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54836bd5ae69903 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.336+0000 7ff9fd600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.336+0000 7ff9fd600640 1 -- 172.21.15.18:0/1492265500 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c008040 con 0x55a6409f79b0 2024-04-16T17:21:20.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.336+0000 7ff9fd600640 1 -- 172.21.15.18:0/1492265500 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7ffa0c012c40 con 0x55a6409f79b0 2024-04-16T17:21:20.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.337+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/1492265500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa0401c470 con 0x55a6409f79b0 2024-04-16T17:21:20.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.340+0000 7ff9fd600640 1 -- 172.21.15.18:0/1492265500 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffa0c016e60 con 0x55a6409f79b0 2024-04-16T17:21:20.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.340+0000 7ffa1d3a9740 10 client.30044 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:20.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.340+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/1492265500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a640a6eab0 con 0x55a6409f79b0 2024-04-16T17:21:20.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ff9fd600640 1 -- 172.21.15.18:0/1492265500 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa0401c470 con 0x55a6409f79b0 2024-04-16T17:21:20.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ff9fd600640 1 -- 172.21.15.18:0/1492265500 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ffa0c022690 con 0x55a6409f79b0 2024-04-16T17:21:20.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ffa1d3a9740 10 client.30044 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:20.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ffa1d3a9740 10 client.30044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:20.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ffa1d3a9740 20 client.30044 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:20.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a640a6f1c0 0x55a640a715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ffa1d3a9740 4 client.30044 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:21:20.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/1492265500 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a640952540 con 0x55a640a6f1c0 2024-04-16T17:21:20.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.341+0000 7ffa12a00640 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a640a6f1c0 0x55a640a715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.346+0000 7ffa12a00640 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a640a6f1c0 0x55a640a715e0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87bde140672974ed server_cookie=e17c5022caf48ae6 in_seq=0 out_seq=0 2024-04-16T17:21:20.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.346+0000 7ff9fd600640 10 client.30044 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.352+0000 7ff9fd600640 1 -- 172.21.15.18:0/1492265500 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a640952540 con 0x55a640a6f1c0 2024-04-16T17:21:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.352+0000 7ff9fd600640 10 client.30044 handle_command_reply: tid=0 2024-04-16T17:21:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.352+0000 7ffa1d3a9740 1 client.30044 shutdown 2024-04-16T17:21:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.352+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/1492265500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409f79b0 msgr2=0x55a6409fbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.352+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409f79b0 0x55a6409fbdf0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x55a6409f4270 tx=0x7ffa0c016b50 comp rx=0 tx=0).stop 2024-04-16T17:21:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/1492265500 shutdown_connections 2024-04-16T17:21:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a640a6f1c0 0x55a640a715e0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6409fc380 0x55a640a6d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6409f79b0 0x55a6409fbdf0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ff9fd600640 0 client.30044 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ffa1d3a9740 1 --2- 172.21.15.18:0/1492265500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6409eedd0 0x55a6409fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/1492265500 >> 172.21.15.18:0/1492265500 conn(0x55a6409e8be0 msgr2=0x55a6409e9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/1492265500 shutdown_connections 2024-04-16T17:21:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.354+0000 7ffa1d3a9740 1 -- 172.21.15.18:0/1492265500 wait complete. 2024-04-16T17:21:20.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.356+0000 7ffa1d3a9740 20 client.30044 trim_cache size 0 max 16384 2024-04-16T17:21:20.357 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:21:20.357 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:21:20.357 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "a2cb7ec5-17f0-45b7-b2e2-cb1a75e28e98", 2024-04-16T17:21:20.357 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:21:20.357 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:21:20.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 -- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9f80a3d10 msgr2=0x7ff9f80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9f80a3d10 0x7ff9f80a61d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffa000bac90 tx=0x7ffa0002bc30 comp rx=0 tx=0).stop 2024-04-16T17:21:20.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 -- 172.21.15.18:0/516235828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 msgr2=0x7ffa14074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 0x7ffa14074300 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002c170 tx=0x7ffa000045e0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 -- 172.21.15.18:0/516235828 shutdown_connections 2024-04-16T17:21:20.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9f80a3d10 0x7ff9f80a61d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa1410abe0 0x7ffa14074300 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa1410a1b0 0x7ffa140796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 --2- 172.21.15.18:0/516235828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa14109890 0x7ffa140791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 -- 172.21.15.18:0/516235828 >> 172.21.15.18:0/516235828 conn(0x7ffa1406d0a0 msgr2=0x7ffa1410d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 -- 172.21.15.18:0/516235828 shutdown_connections 2024-04-16T17:21:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.358+0000 7ffa1a600640 1 -- 172.21.15.18:0/516235828 wait complete. 2024-04-16T17:21:20.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:21:20.388 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:21:20.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 52959 12.24 MB/sec execute 92 sec latency 691.003 ms 2024-04-16T17:21:20.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.671+0000 7fba64600640 1 -- 172.21.15.18:0/3869403677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c105ed0 msgr2=0x7fba5c115080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.671+0000 7fba64600640 1 --2- 172.21.15.18:0/3869403677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c105ed0 0x7fba5c115080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009900 tx=0x7fba4c02bce0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.671+0000 7fba64600640 1 -- 172.21.15.18:0/3869403677 shutdown_connections 2024-04-16T17:21:20.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.671+0000 7fba64600640 1 --2- 172.21.15.18:0/3869403677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c105ed0 0x7fba5c115080 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.671+0000 7fba64600640 1 --2- 172.21.15.18:0/3869403677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba5c074cf0 0x7fba5c112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.671+0000 7fba64600640 1 --2- 172.21.15.18:0/3869403677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba5c0743d0 0x7fba5c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.671+0000 7fba64600640 1 -- 172.21.15.18:0/3869403677 >> 172.21.15.18:0/3869403677 conn(0x7fba5c0fdab0 msgr2=0x7fba5c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.672+0000 7fba64600640 1 -- 172.21.15.18:0/3869403677 shutdown_connections 2024-04-16T17:21:20.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.672+0000 7fba64600640 1 -- 172.21.15.18:0/3869403677 wait complete. 2024-04-16T17:21:20.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.674+0000 7fba64600640 1 Processor -- start 2024-04-16T17:21:20.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.674+0000 7fba64600640 1 -- start start 2024-04-16T17:21:20.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.674+0000 7fba64600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba5c0743d0 0x7fba5c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.675+0000 7fba64600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c074cf0 0x7fba5c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.675+0000 7fba64600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba5c1a3820 0x7fba5c1a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.675+0000 7fba64600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c118a60 con 0x7fba5c1a3820 2024-04-16T17:21:20.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.675+0000 7fba64600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1188e0 con 0x7fba5c0743d0 2024-04-16T17:21:20.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.675+0000 7fba64600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c118be0 con 0x7fba5c074cf0 2024-04-16T17:21:20.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.675+0000 7fba61200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c074cf0 0x7fba5c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.675+0000 7fba61200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c074cf0 0x7fba5c19f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44516/0 (socket says 172.21.15.18:44516) 2024-04-16T17:21:20.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.676+0000 7fba61200640 1 -- 172.21.15.18:0/2136676266 learned_addr learned my addr 172.21.15.18:0/2136676266 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.676+0000 7fba62600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba5c1a3820 0x7fba5c1a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.676+0000 7fba61200640 1 -- 172.21.15.18:0/2136676266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba5c0743d0 msgr2=0x7fba5c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.676+0000 7fba61200640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba5c0743d0 0x7fba5c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.676+0000 7fba61200640 1 -- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba5c1a3820 msgr2=0x7fba5c1a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.676+0000 7fba61200640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba5c1a3820 0x7fba5c1a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.676+0000 7fba61200640 1 -- 172.21.15.18:0/2136676266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c1a4310 con 0x7fba5c074cf0 2024-04-16T17:21:20.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.677+0000 7fba61200640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c074cf0 0x7fba5c19f4a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fba5000ba50 tx=0x7fba5000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.677+0000 7fba5a000640 1 -- 172.21.15.18:0/2136676266 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50019070 con 0x7fba5c074cf0 2024-04-16T17:21:20.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.677+0000 7fba64600640 1 -- 172.21.15.18:0/2136676266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c1a4600 con 0x7fba5c074cf0 2024-04-16T17:21:20.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.677+0000 7fba64600640 1 -- 172.21.15.18:0/2136676266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c1abe40 con 0x7fba5c074cf0 2024-04-16T17:21:20.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.677+0000 7fba5a000640 1 -- 172.21.15.18:0/2136676266 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba5000cbe0 con 0x7fba5c074cf0 2024-04-16T17:21:20.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.677+0000 7fba5a000640 1 -- 172.21.15.18:0/2136676266 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500076e0 con 0x7fba5c074cf0 2024-04-16T17:21:20.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.683+0000 7fba673cb740 1 Processor -- start 2024-04-16T17:21:20.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.683+0000 7fba673cb740 1 -- start start 2024-04-16T17:21:20.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.684+0000 7fba673cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 0x55df1e5b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.684+0000 7fba673cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.684+0000 7fba673cb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5c26b0 0x55df1e5c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.684+0000 7fba673cb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55df1e51f120 con 0x55df1e5c26b0 2024-04-16T17:21:20.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.684+0000 7fba673cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55df1e4c5340 con 0x55df1e5b91b0 2024-04-16T17:21:20.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.684+0000 7fba673cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55df1e51b0e0 con 0x55df1e5b9ad0 2024-04-16T17:21:20.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.684+0000 7fba61c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.685+0000 7fba5a000640 1 -- 172.21.15.18:0/2136676266 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba5001c070 con 0x7fba5c074cf0 2024-04-16T17:21:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.685+0000 7fba61c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44526/0 (socket says 172.21.15.18:44526) 2024-04-16T17:21:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.685+0000 7fba61c00640 1 -- 172.21.15.18:0/3109253438 learned_addr learned my addr 172.21.15.18:0/3109253438 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.685+0000 7fba62600640 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 0x55df1e5b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.686+0000 7fba61200640 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5c26b0 0x55df1e5c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.686+0000 7fba62600640 1 -- 172.21.15.18:0/3109253438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 msgr2=0x55df1e5c2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.686+0000 7fba62600640 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.686+0000 7fba62600640 1 -- 172.21.15.18:0/3109253438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5c26b0 msgr2=0x55df1e5c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.686+0000 7fba62600640 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5c26b0 0x55df1e5c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.686+0000 7fba62600640 1 -- 172.21.15.18:0/3109253438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df1e2fb7d0 con 0x55df1e5b91b0 2024-04-16T17:21:20.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.687+0000 7fba61c00640 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:20.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.687+0000 7fba62600640 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 0x55df1e5b9590 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fba4c02c220 tx=0x7fba4c02c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10adffb65c26a104 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.687+0000 7fba4aa00640 1 -- 172.21.15.18:0/3109253438 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c042070 con 0x55df1e5b91b0 2024-04-16T17:21:20.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.688+0000 7fba4aa00640 1 -- 172.21.15.18:0/3109253438 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c02ce10 con 0x55df1e5b91b0 2024-04-16T17:21:20.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.688+0000 7fba4aa00640 1 -- 172.21.15.18:0/3109253438 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0046c0 con 0x55df1e5b91b0 2024-04-16T17:21:20.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.690+0000 7fba673cb740 1 -- 172.21.15.18:0/3109253438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 msgr2=0x55df1e5b9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.690+0000 7fba673cb740 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 0x55df1e5b9590 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fba4c02c220 tx=0x7fba4c02c750 comp rx=0 tx=0).stop 2024-04-16T17:21:20.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.690+0000 7fba5a000640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba440a3da0 0x7fba440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.690+0000 7fba5a000640 1 -- 172.21.15.18:0/2136676266 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fba500cffe0 con 0x7fba5c074cf0 2024-04-16T17:21:20.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.691+0000 7fba62600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba440a3da0 0x7fba440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.693+0000 7fba62600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba440a3da0 0x7fba440a6260 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009230 tx=0x7fba4c0048c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.694+0000 7fba673cb740 1 -- 172.21.15.18:0/3109253438 shutdown_connections 2024-04-16T17:21:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.694+0000 7fba673cb740 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5c26b0 0x55df1e5c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.694+0000 7fba673cb740 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.694+0000 7fba673cb740 1 --2- 172.21.15.18:0/3109253438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 0x55df1e5b9590 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.694+0000 7fba673cb740 1 -- 172.21.15.18:0/3109253438 >> 172.21.15.18:0/3109253438 conn(0x55df1e5b38e0 msgr2=0x55df1e5b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.694+0000 7fba673cb740 1 -- 172.21.15.18:0/3109253438 shutdown_connections 2024-04-16T17:21:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.694+0000 7fba673cb740 1 -- 172.21.15.18:0/3109253438 wait complete. 2024-04-16T17:21:20.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.695+0000 7fba673cb740 1 -- start start 2024-04-16T17:21:20.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.696+0000 7fba673cb740 1 Processor -- start 2024-04-16T17:21:20.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.697+0000 7fba673cb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.698+0000 7fba673cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.698+0000 7fba673cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5c26b0 0x55df1e5c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.699+0000 7fba673cb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55df1e5c5440 con 0x55df1e5b91b0 2024-04-16T17:21:20.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.699+0000 7fba673cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55df1e4d8710 con 0x55df1e5c26b0 2024-04-16T17:21:20.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.699+0000 7fba673cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55df1e5c55c0 con 0x55df1e5b9ad0 2024-04-16T17:21:20.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.699+0000 7fba673cb740 10 client.0 _ll_register_callbacks cb 0x55df1e4d3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:20.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.699+0000 7fba61c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.699+0000 7fba61c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5bf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54984/0 (socket says 172.21.15.18:54984) 2024-04-16T17:21:20.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.699+0000 7fba61c00640 1 -- 172.21.15.18:0/3687465142 learned_addr learned my addr 172.21.15.18:0/3687465142 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.699+0000 7fba62600640 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5c26b0 0x55df1e5c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.700+0000 7fba61200640 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.700+0000 7fba61c00640 1 -- 172.21.15.18:0/3687465142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 msgr2=0x55df1e5bfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.700+0000 7fba61c00640 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5bfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.700+0000 7fba61c00640 1 -- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5c26b0 msgr2=0x55df1e5c0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.700+0000 7fba61c00640 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5c26b0 0x55df1e5c0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.700+0000 7fba61c00640 1 -- 172.21.15.18:0/3687465142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df1e2313f0 con 0x55df1e5b91b0 2024-04-16T17:21:20.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.700+0000 7fba62600640 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5c26b0 0x55df1e5c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:20.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.701+0000 7fba61c00640 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5bf6b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fba5400f640 tx=0x7fba5400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b698c360f74daf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.701+0000 7fba3f400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:21:20.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.701+0000 7fba3f400640 1 -- 172.21.15.18:0/3687465142 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54004220 con 0x55df1e5b91b0 2024-04-16T17:21:20.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.701+0000 7fba3f400640 1 -- 172.21.15.18:0/3687465142 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fba540043c0 con 0x55df1e5b91b0 2024-04-16T17:21:20.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.701+0000 7fba673cb740 1 -- 172.21.15.18:0/3687465142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba4c02ce10 con 0x55df1e5b91b0 2024-04-16T17:21:20.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.702+0000 7fba3f400640 1 -- 172.21.15.18:0/3687465142 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fba5400d880 con 0x55df1e5b91b0 2024-04-16T17:21:20.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.702+0000 7fba673cb740 10 client.19395 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:20.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.702+0000 7fba673cb740 1 -- 172.21.15.18:0/3687465142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55df1e1f10f0 con 0x55df1e5b91b0 2024-04-16T17:21:20.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.704+0000 7fba3f400640 1 -- 172.21.15.18:0/3687465142 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c02ce10 con 0x55df1e5b91b0 2024-04-16T17:21:20.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.704+0000 7fba3f400640 1 -- 172.21.15.18:0/3687465142 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fba54005130 con 0x55df1e5b91b0 2024-04-16T17:21:20.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.704+0000 7fba673cb740 10 client.19395 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:20.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.704+0000 7fba673cb740 10 client.19395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:20.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.704+0000 7fba673cb740 20 client.19395 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:20.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.704+0000 7fba673cb740 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55df1e5d2af0 0x55df1e5d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.704+0000 7fba673cb740 4 client.19395 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:20.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.704+0000 7fba673cb740 1 -- 172.21.15.18:0/3687465142 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df1e52cb40 con 0x55df1e5d2af0 2024-04-16T17:21:20.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.705+0000 7fba61200640 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55df1e5d2af0 0x55df1e5d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.707+0000 7fba61200640 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55df1e5d2af0 0x55df1e5d4f40 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce5cf9213a6678e server_cookie=5f339e3c6ee61b3d in_seq=0 out_seq=0 2024-04-16T17:21:20.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.707+0000 7fba3f400640 10 client.19395 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:20.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.715+0000 7fba3f400640 1 -- 172.21.15.18:0/3687465142 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df1e52cb40 con 0x55df1e5d2af0 2024-04-16T17:21:20.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.715+0000 7fba3f400640 10 client.19395 handle_command_reply: tid=0 2024-04-16T17:21:20.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.716+0000 7fba673cb740 1 client.19395 shutdown 2024-04-16T17:21:20.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 -- 172.21.15.18:0/3687465142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 msgr2=0x55df1e5bf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5bf6b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fba5400f640 tx=0x7fba5400fb00 comp rx=0 tx=0).stop 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 -- 172.21.15.18:0/3687465142 shutdown_connections 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55df1e5d2af0 0x55df1e5d4f40 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5c26b0 0x55df1e5c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5b9ad0 0x55df1e5bfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 --2- 172.21.15.18:0/3687465142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5bf6b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba3f400640 0 client.19395 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 -- 172.21.15.18:0/3687465142 >> 172.21.15.18:0/3687465142 conn(0x55df1e5b38e0 msgr2=0x55df1e5b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 -- 172.21.15.18:0/3687465142 shutdown_connections 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.718+0000 7fba673cb740 1 -- 172.21.15.18:0/3687465142 wait complete. 2024-04-16T17:21:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.719+0000 7fba673cb740 20 client.19395 trim_cache size 0 max 16384 2024-04-16T17:21:20.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.772+0000 7fba673cb740 1 Processor -- start 2024-04-16T17:21:20.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.772+0000 7fba673cb740 1 -- start start 2024-04-16T17:21:20.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.772+0000 7fba673cb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba673cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b9ad0 0x55df1e5ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba673cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba673cb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba40012ac0 con 0x55df1e5b91b0 2024-04-16T17:21:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba673cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba40010390 con 0x55df1e5b9ad0 2024-04-16T17:21:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba673cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba4000e200 con 0x55df1e5c26b0 2024-04-16T17:21:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba61200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b9ad0 0x55df1e5ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba61200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b9ad0 0x55df1e5ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40392/0 (socket says 172.21.15.18:40392) 2024-04-16T17:21:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba62600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba62600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5be810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44556/0 (socket says 172.21.15.18:44556) 2024-04-16T17:21:20.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.773+0000 7fba62600640 1 -- 172.21.15.18:0/2989738352 learned_addr learned my addr 172.21.15.18:0/2989738352 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.774+0000 7fba61c00640 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.774+0000 7fba62600640 1 -- 172.21.15.18:0/2989738352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b9ad0 msgr2=0x55df1e5ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.774+0000 7fba62600640 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b9ad0 0x55df1e5ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.774+0000 7fba62600640 1 -- 172.21.15.18:0/2989738352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 msgr2=0x55df1e5c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.774+0000 7fba62600640 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5c1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.774+0000 7fba62600640 1 -- 172.21.15.18:0/2989738352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df1e5bed50 con 0x55df1e5c26b0 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.774+0000 7fba61c00640 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.775+0000 7fba62600640 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5be810 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fba4c006010 tx=0x7fba4c04d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf861e299e3759da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.775+0000 7fba4a000640 1 -- 172.21.15.18:0/2989738352 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c05d070 con 0x55df1e5c26b0 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.775+0000 7fba4a000640 1 -- 172.21.15.18:0/2989738352 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c0046c0 con 0x55df1e5c26b0 2024-04-16T17:21:20.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.775+0000 7fba4a000640 1 -- 172.21.15.18:0/2989738352 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c05d070 con 0x55df1e5c26b0 2024-04-16T17:21:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 -- 172.21.15.18:0/2989738352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 msgr2=0x55df1e5be810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5be810 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fba4c006010 tx=0x7fba4c04d000 comp rx=0 tx=0).stop 2024-04-16T17:21:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 -- 172.21.15.18:0/2989738352 shutdown_connections 2024-04-16T17:21:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5be810 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b9ad0 0x55df1e5ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 --2- 172.21.15.18:0/2989738352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b91b0 0x55df1e5c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 -- 172.21.15.18:0/2989738352 >> 172.21.15.18:0/2989738352 conn(0x55df1e5b38e0 msgr2=0x55df1e5b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 -- 172.21.15.18:0/2989738352 shutdown_connections 2024-04-16T17:21:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.777+0000 7fba673cb740 1 -- 172.21.15.18:0/2989738352 wait complete. 2024-04-16T17:21:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.778+0000 7fba673cb740 1 -- start start 2024-04-16T17:21:20.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.778+0000 7fba673cb740 1 Processor -- start 2024-04-16T17:21:20.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.779+0000 7fba673cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 0x55df1e5ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.780+0000 7fba673cb740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.780+0000 7fba673cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5c6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.780+0000 7fba62600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.780+0000 7fba62600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44568/0 (socket says 172.21.15.18:44568) 2024-04-16T17:21:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.780+0000 7fba62600640 1 -- 172.21.15.18:0/253647140 learned_addr learned my addr 172.21.15.18:0/253647140 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.780+0000 7fba673cb740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba4000e850 con 0x55df1e5b9ad0 2024-04-16T17:21:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.780+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba4000fcc0 con 0x55df1e5b91b0 2024-04-16T17:21:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.780+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba4000ef20 con 0x55df1e5c26b0 2024-04-16T17:21:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.781+0000 7fba673cb740 10 client.0 _ll_register_callbacks cb 0x7fba5400a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:20.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.781+0000 7fba62600640 1 -- 172.21.15.18:0/253647140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 msgr2=0x55df1e5ca8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.781+0000 7fba62600640 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 0x55df1e5ca8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.781+0000 7fba62600640 1 -- 172.21.15.18:0/253647140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b9ad0 msgr2=0x55df1e5c5e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.781+0000 7fba62600640 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.781+0000 7fba62600640 1 -- 172.21.15.18:0/253647140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df1e5c68d0 con 0x55df1e5c26b0 2024-04-16T17:21:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.782+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55df1e5269b0 con 0x55df1e5c26b0 2024-04-16T17:21:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.782+0000 7fba62600640 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5c6390 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0096c0 tx=0x7fba4c02c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391268db6afb5fca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:20.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.782+0000 7fba3f400640 10 client.29863 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:20.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.783+0000 7fba3f400640 1 -- 172.21.15.18:0/253647140 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c066060 con 0x55df1e5c26b0 2024-04-16T17:21:20.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.783+0000 7fba3f400640 1 -- 172.21.15.18:0/253647140 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fba4c02c9d0 con 0x55df1e5c26b0 2024-04-16T17:21:20.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.783+0000 7fba3f400640 1 -- 172.21.15.18:0/253647140 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fba4c05d050 con 0x55df1e5c26b0 2024-04-16T17:21:20.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba673cb740 10 client.29863 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:20.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55df1e638c10 con 0x55df1e5c26b0 2024-04-16T17:21:20.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba3f400640 1 -- 172.21.15.18:0/253647140 <== mon.1 v2:172.21.15.43:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fba4c02cbd0 con 0x55df1e5c26b0 2024-04-16T17:21:20.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba673cb740 10 client.29863 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba673cb740 10 client.29863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba673cb740 20 client.29863 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba673cb740 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55df1e638de0 0x55df1e63b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba673cb740 4 client.29863 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:21:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.784+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df1e4a2420 con 0x55df1e638de0 2024-04-16T17:21:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.785+0000 7fba61c00640 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55df1e638de0 0x55df1e63b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:20.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.786+0000 7fba61c00640 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55df1e638de0 0x55df1e63b1d0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c8bf98281249837 server_cookie=77aff40122bb0ff3 in_seq=0 out_seq=0 2024-04-16T17:21:20.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.786+0000 7fba3f400640 10 client.29863 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:20.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.788+0000 7fba3f400640 1 -- 172.21.15.18:0/253647140 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55df1e4a2420 con 0x55df1e638de0 2024-04-16T17:21:20.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.788+0000 7fba3f400640 10 client.29863 handle_command_reply: tid=0 2024-04-16T17:21:20.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.788+0000 7fba673cb740 1 client.29863 shutdown 2024-04-16T17:21:20.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.790+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 msgr2=0x55df1e5c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.790+0000 7fba673cb740 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5c6390 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0096c0 tx=0x7fba4c02c730 comp rx=0 tx=0).stop 2024-04-16T17:21:20.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.791+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 shutdown_connections 2024-04-16T17:21:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.791+0000 7fba673cb740 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55df1e638de0 0x55df1e63b1d0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.791+0000 7fba673cb740 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df1e5c26b0 0x55df1e5c6390 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.791+0000 7fba673cb740 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55df1e5b9ad0 0x55df1e5c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.791+0000 7fba673cb740 1 --2- 172.21.15.18:0/253647140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df1e5b91b0 0x55df1e5ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.791+0000 7fba3f400640 0 client.29863 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.792+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 >> 172.21.15.18:0/253647140 conn(0x55df1e5b38e0 msgr2=0x55df1e5b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.792+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 shutdown_connections 2024-04-16T17:21:20.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.792+0000 7fba673cb740 1 -- 172.21.15.18:0/253647140 wait complete. 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.793+0000 7fba673cb740 20 client.29863 trim_cache size 0 max 16384 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (277 inodes in the stack)", 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout: "a2cb7ec5-17f0-45b7-b2e2-cb1a75e28e98": { 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "a2cb7ec5-17f0-45b7-b2e2-cb1a75e28e98", 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:21:20.793 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:21:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 -- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba440a3da0 msgr2=0x7fba440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba440a3da0 0x7fba440a6260 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009230 tx=0x7fba4c0048c0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 -- 172.21.15.18:0/2136676266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c074cf0 msgr2=0x7fba5c19f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c074cf0 0x7fba5c19f4a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fba5000ba50 tx=0x7fba5000bf10 comp rx=0 tx=0).stop 2024-04-16T17:21:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 -- 172.21.15.18:0/2136676266 shutdown_connections 2024-04-16T17:21:20.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba440a3da0 0x7fba440a6260 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba5c1a3820 0x7fba5c1a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5c074cf0 0x7fba5c19f4a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 --2- 172.21.15.18:0/2136676266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba5c0743d0 0x7fba5c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 -- 172.21.15.18:0/2136676266 >> 172.21.15.18:0/2136676266 conn(0x7fba5c0fdab0 msgr2=0x7fba5c113c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 -- 172.21.15.18:0/2136676266 shutdown_connections 2024-04-16T17:21:20.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:20.794+0000 7fba64600640 1 -- 172.21.15.18:0/2136676266 wait complete. 2024-04-16T17:21:20.824 INFO:tasks.cephfs.filesystem:scrub status for tag:a2cb7ec5-17f0-45b7-b2e2-cb1a75e28e98 - {'path': '/', 'tag': 'a2cb7ec5-17f0-45b7-b2e2-cb1a75e28e98', 'options': 'recursive,force'} 2024-04-16T17:21:21.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:19 smithi018 ceph-mon[31776]: pgmap v528: 129 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 96 active+clean; 398 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 318 op/s 2024-04-16T17:21:21.491 INFO:tasks.workunit.client.0.smithi018.stdout: 10 53094 12.17 MB/sec execute 93 sec latency 997.515 ms 2024-04-16T17:21:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:21:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:21:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: pgmap v529: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 470 op/s 2024-04-16T17:21:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:21:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:21:22.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:22.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:22.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:21:22.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:21 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: pgmap v529: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 470 op/s 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:21:22.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:21 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:22.492 INFO:tasks.workunit.client.0.smithi018.stdout: 10 53406 12.27 MB/sec execute 94 sec latency 690.440 ms 2024-04-16T17:21:22.722 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:21:22.722 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:23.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:22 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:23.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:22 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:21:23.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:22 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:23.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:22 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:21:23.493 INFO:tasks.workunit.client.0.smithi018.stdout: 10 53610 12.28 MB/sec execute 95 sec latency 444.973 ms 2024-04-16T17:21:23.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:21:23.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: pgmap v529: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 470 op/s 2024-04-16T17:21:23.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:21:23.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:21:23.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:23.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:23.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:21:23.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:21 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:23.613 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:22 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:23.613 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:22 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:21:24.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:23 smithi022 ceph-mon[36023]: pgmap v530: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 381 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-04-16T17:21:24.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:23 smithi043 ceph-mon[36901]: pgmap v530: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 381 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-04-16T17:21:24.492 INFO:tasks.workunit.client.0.smithi018.stdout: 10 53900 12.38 MB/sec execute 96 sec latency 104.244 ms 2024-04-16T17:21:25.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:24 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:21:25.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:24 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:21:25.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:24 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:21:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:24 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:21:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:24 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:21:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:24 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:21:25.492 INFO:tasks.workunit.client.0.smithi018.stdout: 10 54117 12.39 MB/sec execute 97 sec latency 326.423 ms 2024-04-16T17:21:26.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:26 smithi022 ceph-mon[36023]: pgmap v531: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 445 op/s 2024-04-16T17:21:26.492 INFO:tasks.workunit.client.0.smithi018.stdout: 10 54272 12.33 MB/sec execute 98 sec latency 322.505 ms 2024-04-16T17:21:26.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:26 smithi043 ceph-mon[36901]: pgmap v531: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 445 op/s 2024-04-16T17:21:27.492 INFO:tasks.workunit.client.0.smithi018.stdout: 10 54474 12.27 MB/sec execute 99 sec latency 230.766 ms 2024-04-16T17:21:28.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:28 smithi022 ceph-mon[36023]: pgmap v532: 129 pgs: 129 active+clean; 391 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-04-16T17:21:28.492 INFO:tasks.workunit.client.0.smithi018.stdout: 10 54759 12.23 MB/sec execute 100 sec latency 227.140 ms 2024-04-16T17:21:28.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:28 smithi043 ceph-mon[36901]: pgmap v532: 129 pgs: 129 active+clean; 391 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-04-16T17:21:29.492 INFO:tasks.workunit.client.0.smithi018.stdout: 10 55110 12.29 MB/sec execute 101 sec latency 189.566 ms 2024-04-16T17:21:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.509+0000 7fd087800640 1 -- 172.21.15.18:0/2075426987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080115150 msgr2=0x7fd080117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.509+0000 7fd087800640 1 --2- 172.21.15.18:0/2075426987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080115150 0x7fd080117540 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009900 tx=0x7fd06c02bcf0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.510+0000 7fd087800640 1 -- 172.21.15.18:0/2075426987 shutdown_connections 2024-04-16T17:21:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.510+0000 7fd087800640 1 --2- 172.21.15.18:0/2075426987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080115150 0x7fd080117540 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.510+0000 7fd087800640 1 --2- 172.21.15.18:0/2075426987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd080109b30 0x7fd080114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.510+0000 7fd087800640 1 --2- 172.21.15.18:0/2075426987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd080109210 0x7fd0801095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.510+0000 7fd087800640 1 -- 172.21.15.18:0/2075426987 >> 172.21.15.18:0/2075426987 conn(0x7fd08006d0e0 msgr2=0x7fd08006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.510+0000 7fd087800640 1 -- 172.21.15.18:0/2075426987 shutdown_connections 2024-04-16T17:21:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.510+0000 7fd087800640 1 -- 172.21.15.18:0/2075426987 wait complete. 2024-04-16T17:21:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.512+0000 7fd087800640 1 Processor -- start 2024-04-16T17:21:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.512+0000 7fd087800640 1 -- start start 2024-04-16T17:21:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.512+0000 7fd087800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080109210 0x7fd0801a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.512+0000 7fd087800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd080109b30 0x7fd0801a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.512+0000 7fd087800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd080115150 0x7fd0801a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.513+0000 7fd087800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd08011acb0 con 0x7fd080109b30 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.513+0000 7fd087800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd08011ab30 con 0x7fd080109210 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.513+0000 7fd087800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd08011ae30 con 0x7fd080115150 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.513+0000 7fd085800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd080115150 0x7fd0801a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.513+0000 7fd084e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080109210 0x7fd0801a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.513+0000 7fd085800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd080115150 0x7fd0801a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34834/0 (socket says 172.21.15.18:34834) 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.513+0000 7fd085800640 1 -- 172.21.15.18:0/4064635714 learned_addr learned my addr 172.21.15.18:0/4064635714 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.514+0000 7fd084e00640 1 -- 172.21.15.18:0/4064635714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd080115150 msgr2=0x7fd0801a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.514+0000 7fd084e00640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd080115150 0x7fd0801a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.514+0000 7fd084e00640 1 -- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd080109b30 msgr2=0x7fd0801a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.514+0000 7fd084e00640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd080109b30 0x7fd0801a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.514+0000 7fd084e00640 1 -- 172.21.15.18:0/4064635714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0801a61e0 con 0x7fd080109210 2024-04-16T17:21:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.515+0000 7fd084e00640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080109210 0x7fd0801a11f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd07800bd70 tx=0x7fd07800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.515+0000 7fd07d600640 1 -- 172.21.15.18:0/4064635714 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078019070 con 0x7fd080109210 2024-04-16T17:21:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.515+0000 7fd07d600640 1 -- 172.21.15.18:0/4064635714 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd078014920 con 0x7fd080109210 2024-04-16T17:21:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.515+0000 7fd07d600640 1 -- 172.21.15.18:0/4064635714 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0780129b0 con 0x7fd080109210 2024-04-16T17:21:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.515+0000 7fd087800640 1 -- 172.21.15.18:0/4064635714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0801a64d0 con 0x7fd080109210 2024-04-16T17:21:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.515+0000 7fd087800640 1 -- 172.21.15.18:0/4064635714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08007a260 con 0x7fd080109210 2024-04-16T17:21:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.521+0000 7fd08a445740 1 Processor -- start 2024-04-16T17:21:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.524+0000 7fd07d600640 1 -- 172.21.15.18:0/4064635714 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd07801c070 con 0x7fd080109210 2024-04-16T17:21:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.526+0000 7fd07d600640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0640a3de0 0x7fd0640a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.527+0000 7fd08a445740 1 -- start start 2024-04-16T17:21:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd08a445740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305e310 0x55d77305e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd08a445740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305ec30 0x55d77305f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd08a445740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773072a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd08a445740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d772fc3780 con 0x55d77305f6d0 2024-04-16T17:21:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd08a445740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d772f69f40 con 0x55d77305e310 2024-04-16T17:21:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd08a445740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d772fbf740 con 0x55d77305ec30 2024-04-16T17:21:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd084e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305e310 0x55d77305e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd084e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305e310 0x55d77305e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:58426/0 (socket says 172.21.15.18:58426) 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd084e00640 1 -- 172.21.15.18:0/34676083 learned_addr learned my addr 172.21.15.18:0/34676083 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.528+0000 7fd07d600640 1 -- 172.21.15.18:0/4064635714 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fd07809a0c0 con 0x7fd080109210 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.529+0000 7fd084e00640 1 -- 172.21.15.18:0/34676083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305ec30 msgr2=0x55d77305f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.529+0000 7fd084e00640 1 --2- 172.21.15.18:0/34676083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305ec30 0x55d77305f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.529+0000 7fd084e00640 1 -- 172.21.15.18:0/34676083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 msgr2=0x55d773072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.529+0000 7fd084e00640 1 --2- 172.21.15.18:0/34676083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.529+0000 7fd084e00640 1 -- 172.21.15.18:0/34676083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d772cd6210 con 0x55d77305e310 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.529+0000 7fd084e00640 1 --2- 172.21.15.18:0/34676083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305e310 0x55d77305e6f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd07800bd40 tx=0x7fd0780a14f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03ff0a7aaa50f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.529+0000 7fd076000640 1 -- 172.21.15.18:0/34676083 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078098a50 con 0x55d77305e310 2024-04-16T17:21:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.530+0000 7fd076000640 1 -- 172.21.15.18:0/34676083 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd078014c70 con 0x55d77305e310 2024-04-16T17:21:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.530+0000 7fd076000640 1 -- 172.21.15.18:0/34676083 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078098bf0 con 0x55d77305e310 2024-04-16T17:21:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.531+0000 7fd085800640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0640a3de0 0x7fd0640a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.531+0000 7fd08a445740 1 -- 172.21.15.18:0/34676083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305e310 msgr2=0x55d77305e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.531+0000 7fd08a445740 1 --2- 172.21.15.18:0/34676083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305e310 0x55d77305e6f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd07800bd40 tx=0x7fd0780a14f0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.532+0000 7fd08a445740 1 -- 172.21.15.18:0/34676083 shutdown_connections 2024-04-16T17:21:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.532+0000 7fd08a445740 1 --2- 172.21.15.18:0/34676083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.532+0000 7fd08a445740 1 --2- 172.21.15.18:0/34676083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305ec30 0x55d77305f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.532+0000 7fd08a445740 1 --2- 172.21.15.18:0/34676083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305e310 0x55d77305e6f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.532+0000 7fd08a445740 1 -- 172.21.15.18:0/34676083 >> 172.21.15.18:0/34676083 conn(0x55d772ff95d0 msgr2=0x55d773059590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.532+0000 7fd08a445740 1 -- 172.21.15.18:0/34676083 shutdown_connections 2024-04-16T17:21:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.532+0000 7fd08a445740 1 -- 172.21.15.18:0/34676083 wait complete. 2024-04-16T17:21:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.533+0000 7fd085800640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0640a3de0 0x7fd0640a62a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0096c0 tx=0x7fd06c005a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.533+0000 7fd08a445740 1 -- start start 2024-04-16T17:21:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.535+0000 7fd08a445740 1 Processor -- start 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd08a445740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305e310 0x55d773072a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd08a445740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 0x55d77306fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd08a445740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd08a445740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d773073380 con 0x55d77305f6d0 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd08a445740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d772f7d390 con 0x55d77305ec30 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd08a445740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d773073500 con 0x55d77305e310 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd08a445740 10 client.0 _ll_register_callbacks cb 0x55d772f66f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd084e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305e310 0x55d773072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd084e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305e310 0x55d773072a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34844/0 (socket says 172.21.15.18:34844) 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.536+0000 7fd084e00640 1 -- 172.21.15.18:0/2864839368 learned_addr learned my addr 172.21.15.18:0/2864839368 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd084e00640 1 -- 172.21.15.18:0/2864839368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 msgr2=0x55d77306fe20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd084e00640 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 0x55d77306fe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd084e00640 1 -- 172.21.15.18:0/2864839368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 msgr2=0x55d773070560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd084e00640 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773070560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd084e00640 1 -- 172.21.15.18:0/2864839368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d772da2620 con 0x55d77305e310 2024-04-16T17:21:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd084e00640 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305e310 0x55d773072a90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd078014c40 tx=0x7fd078004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d6163c694377561 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd06aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd06aa00640 1 -- 172.21.15.18:0/2864839368 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0780ba060 con 0x55d77305e310 2024-04-16T17:21:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.537+0000 7fd06aa00640 1 -- 172.21.15.18:0/2864839368 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fd0780980f0 con 0x55d77305e310 2024-04-16T17:21:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.538+0000 7fd08a445740 1 -- 172.21.15.18:0/2864839368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd078014c70 con 0x55d77305e310 2024-04-16T17:21:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.538+0000 7fd06aa00640 1 -- 172.21.15.18:0/2864839368 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd0780a9050 con 0x55d77305e310 2024-04-16T17:21:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.540+0000 7fd08a445740 10 client.29872 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.540+0000 7fd08a445740 1 -- 172.21.15.18:0/2864839368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d773073380 con 0x55d77305e310 2024-04-16T17:21:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd06aa00640 1 -- 172.21.15.18:0/2864839368 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd078014c70 con 0x55d77305e310 2024-04-16T17:21:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd06aa00640 1 -- 172.21.15.18:0/2864839368 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fd0780ac080 con 0x55d77305e310 2024-04-16T17:21:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd08a445740 10 client.29872 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd08a445740 10 client.29872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd08a445740 20 client.29872 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd08a445740 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d773080430 0x55d773082820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd08a445740 4 client.29872 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd08a445740 1 -- 172.21.15.18:0/2864839368 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d772fd11a0 con 0x55d773080430 2024-04-16T17:21:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.541+0000 7fd07fe00640 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d773080430 0x55d773082820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.544+0000 7fd06aa00640 10 client.29872 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.545+0000 7fd07fe00640 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d773080430 0x55d773082820 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b602331fbce165e server_cookie=75643d923a7d8482 in_seq=0 out_seq=0 2024-04-16T17:21:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.550+0000 7fd06aa00640 1 -- 172.21.15.18:0/2864839368 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d772fd11a0 con 0x55d773080430 2024-04-16T17:21:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.550+0000 7fd06aa00640 10 client.29872 handle_command_reply: tid=0 2024-04-16T17:21:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.550+0000 7fd08a445740 1 client.29872 shutdown 2024-04-16T17:21:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd08a445740 1 -- 172.21.15.18:0/2864839368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305e310 msgr2=0x55d773072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd08a445740 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305e310 0x55d773072a90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd078014c40 tx=0x7fd078004b50 comp rx=0 tx=0).stop 2024-04-16T17:21:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd08a445740 1 -- 172.21.15.18:0/2864839368 shutdown_connections 2024-04-16T17:21:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd08a445740 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d773080430 0x55d773082820 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd08a445740 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd08a445740 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 0x55d77306fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd08a445740 1 --2- 172.21.15.18:0/2864839368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305e310 0x55d773072a90 secure :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd078014c40 tx=0x7fd078004b50 comp rx=0 tx=0).stop 2024-04-16T17:21:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd08a445740 1 -- 172.21.15.18:0/2864839368 >> 172.21.15.18:0/2864839368 conn(0x55d772ff95d0 msgr2=0x55d77305b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.551+0000 7fd06aa00640 0 client.29872 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.552+0000 7fd08a445740 1 -- 172.21.15.18:0/2864839368 shutdown_connections 2024-04-16T17:21:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.552+0000 7fd08a445740 1 -- 172.21.15.18:0/2864839368 wait complete. 2024-04-16T17:21:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.552+0000 7fd08a445740 20 client.29872 trim_cache size 0 max 16384 2024-04-16T17:21:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.594+0000 7fd08a445740 1 Processor -- start 2024-04-16T17:21:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.595+0000 7fd08a445740 1 -- start start 2024-04-16T17:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.595+0000 7fd08a445740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305ec30 0x55d77307c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.595+0000 7fd08a445740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305f6d0 0x55d77306d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.595+0000 7fd08a445740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77306ee20 0x55d773087f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.595+0000 7fd08a445740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd080084aa0 con 0x55d77305ec30 2024-04-16T17:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.595+0000 7fd08a445740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd080082370 con 0x55d77306ee20 2024-04-16T17:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.595+0000 7fd08a445740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800801e0 con 0x55d77305f6d0 2024-04-16T17:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.596+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305f6d0 0x55d77306d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.597+0000 7fd085800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77306ee20 0x55d773087f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.597+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305f6d0 0x55d77306d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34854/0 (socket says 172.21.15.18:34854) 2024-04-16T17:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.597+0000 7fd07fe00640 1 -- 172.21.15.18:0/1299163635 learned_addr learned my addr 172.21.15.18:0/1299163635 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.598+0000 7fd07fe00640 1 -- 172.21.15.18:0/1299163635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77306ee20 msgr2=0x55d773087f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.598+0000 7fd07fe00640 1 --2- 172.21.15.18:0/1299163635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77306ee20 0x55d773087f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.598+0000 7fd07fe00640 1 -- 172.21.15.18:0/1299163635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305ec30 msgr2=0x55d77307c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.598+0000 7fd07fe00640 1 --2- 172.21.15.18:0/1299163635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305ec30 0x55d77307c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.598+0000 7fd07fe00640 1 -- 172.21.15.18:0/1299163635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7730708d0 con 0x55d77305f6d0 2024-04-16T17:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.599+0000 7fd07fe00640 1 --2- 172.21.15.18:0/1299163635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305f6d0 0x55d77306d9f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd0800736a0 tx=0x7fd070009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=534f391e1f2238b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.600+0000 7fd075600640 1 -- 172.21.15.18:0/1299163635 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070010880 con 0x55d77305f6d0 2024-04-16T17:21:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.601+0000 7fd075600640 1 -- 172.21.15.18:0/1299163635 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070005020 con 0x55d77305f6d0 2024-04-16T17:21:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.601+0000 7fd075600640 1 -- 172.21.15.18:0/1299163635 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070017690 con 0x55d77305f6d0 2024-04-16T17:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.603+0000 7fd08a445740 1 -- 172.21.15.18:0/1299163635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305f6d0 msgr2=0x55d77306d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.603+0000 7fd08a445740 1 --2- 172.21.15.18:0/1299163635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305f6d0 0x55d77306d9f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd0800736a0 tx=0x7fd070009570 comp rx=0 tx=0).stop 2024-04-16T17:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.603+0000 7fd08a445740 1 -- 172.21.15.18:0/1299163635 shutdown_connections 2024-04-16T17:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.603+0000 7fd08a445740 1 --2- 172.21.15.18:0/1299163635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77306ee20 0x55d773087f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.603+0000 7fd08a445740 1 --2- 172.21.15.18:0/1299163635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77305f6d0 0x55d77306d9f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.603+0000 7fd08a445740 1 --2- 172.21.15.18:0/1299163635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305ec30 0x55d77307c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.603+0000 7fd08a445740 1 -- 172.21.15.18:0/1299163635 >> 172.21.15.18:0/1299163635 conn(0x55d772ff95d0 msgr2=0x55d77306e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.604+0000 7fd08a445740 1 -- 172.21.15.18:0/1299163635 shutdown_connections 2024-04-16T17:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.604+0000 7fd08a445740 1 -- 172.21.15.18:0/1299163635 wait complete. 2024-04-16T17:21:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.604+0000 7fd08a445740 1 -- start start 2024-04-16T17:21:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.605+0000 7fd08a445740 1 Processor -- start 2024-04-16T17:21:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.606+0000 7fd08a445740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 0x55d773078980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.606+0000 7fd08a445740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773075760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.606+0000 7fd08a445740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77306ee20 0x55d773075ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.607+0000 7fd08a445740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd080080830 con 0x55d77305f6d0 2024-04-16T17:21:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.607+0000 7fd08a445740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd080081ca0 con 0x55d77305ec30 2024-04-16T17:21:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.607+0000 7fd08a445740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd080080f00 con 0x55d77306ee20 2024-04-16T17:21:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.607+0000 7fd08a445740 10 client.0 _ll_register_callbacks cb 0x7fd078017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.607+0000 7fd085800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77306ee20 0x55d773075ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.607+0000 7fd085800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77306ee20 0x55d773075ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34858/0 (socket says 172.21.15.18:34858) 2024-04-16T17:21:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.607+0000 7fd085800640 1 -- 172.21.15.18:0/1019224987 learned_addr learned my addr 172.21.15.18:0/1019224987 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.608+0000 7fd084e00640 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 0x55d773078980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.608+0000 7fd085800640 1 -- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 msgr2=0x55d773078980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.608+0000 7fd085800640 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 0x55d773078980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.608+0000 7fd085800640 1 -- 172.21.15.18:0/1019224987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 msgr2=0x55d773075760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.609+0000 7fd085800640 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.610+0000 7fd085800640 1 -- 172.21.15.18:0/1019224987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d773076260 con 0x55d77306ee20 2024-04-16T17:21:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.610+0000 7fd084e00640 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 0x55d773078980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.611+0000 7fd085800640 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77306ee20 0x55d773075ca0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd06c036040 tx=0x7fd06c02cd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9aca68fa53a2f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.611+0000 7fd06aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.612+0000 7fd06aa00640 1 -- 172.21.15.18:0/1019224987 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c050020 con 0x55d77306ee20 2024-04-16T17:21:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.612+0000 7fd06aa00640 1 -- 172.21.15.18:0/1019224987 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fd06c0048c0 con 0x55d77306ee20 2024-04-16T17:21:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.612+0000 7fd08a445740 1 -- 172.21.15.18:0/1019224987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd070005020 con 0x55d77306ee20 2024-04-16T17:21:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.613+0000 7fd06aa00640 1 -- 172.21.15.18:0/1019224987 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd06c046040 con 0x55d77306ee20 2024-04-16T17:21:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.613+0000 7fd08a445740 10 client.29878 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.613+0000 7fd08a445740 1 -- 172.21.15.18:0/1019224987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d7730e7580 con 0x55d77306ee20 2024-04-16T17:21:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.614+0000 7fd06aa00640 1 -- 172.21.15.18:0/1019224987 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070005020 con 0x55d77306ee20 2024-04-16T17:21:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.614+0000 7fd06aa00640 1 -- 172.21.15.18:0/1019224987 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fd06c040470 con 0x55d77306ee20 2024-04-16T17:21:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.615+0000 7fd08a445740 10 client.29878 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.615+0000 7fd08a445740 10 client.29878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.615+0000 7fd08a445740 20 client.29878 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.615+0000 7fd08a445740 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d7730e7db0 0x55d7730ea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.615+0000 7fd08a445740 4 client.29878 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:21:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.615+0000 7fd08a445740 1 -- 172.21.15.18:0/1019224987 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d772fcb010 con 0x55d7730e7db0 2024-04-16T17:21:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.615+0000 7fd084e00640 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d7730e7db0 0x55d7730ea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.616+0000 7fd084e00640 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d7730e7db0 0x55d7730ea1d0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3c911785c4ae6a server_cookie=598ef15088cf553b in_seq=0 out_seq=0 2024-04-16T17:21:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:29.616+0000 7fd06aa00640 10 client.29878 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:29.794 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:23 smithi018 ceph-mon[31776]: pgmap v530: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 381 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-04-16T17:21:29.794 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:24 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:21:29.794 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:24 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:21:29.794 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:24 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:21:29.794 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:26 smithi018 ceph-mon[31776]: pgmap v531: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 445 op/s 2024-04-16T17:21:29.794 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:28 smithi018 ceph-mon[31776]: pgmap v532: 129 pgs: 129 active+clean; 391 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-04-16T17:21:29.794 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:29 smithi018 ceph-mon[31776]: osdmap e123: 12 total, 12 up, 12 in 2024-04-16T17:21:29.794 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:29 smithi018 ceph-mon[31776]: pgmap v534: 129 pgs: 129 active+clean; 392 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 523 op/s 2024-04-16T17:21:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:29 smithi022 ceph-mon[36023]: osdmap e123: 12 total, 12 up, 12 in 2024-04-16T17:21:29.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:29 smithi022 ceph-mon[36023]: pgmap v534: 129 pgs: 129 active+clean; 392 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 523 op/s 2024-04-16T17:21:30.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:29 smithi043 ceph-mon[36901]: osdmap e123: 12 total, 12 up, 12 in 2024-04-16T17:21:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:29 smithi043 ceph-mon[36901]: pgmap v534: 129 pgs: 129 active+clean; 392 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 523 op/s 2024-04-16T17:21:30.492 INFO:tasks.workunit.client.0.smithi018.stdout: 10 55210 12.24 MB/sec execute 102 sec latency 794.203 ms 2024-04-16T17:21:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.813+0000 7fd06aa00640 1 -- 172.21.15.18:0/1019224987 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d772fcb010 con 0x55d7730e7db0 2024-04-16T17:21:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.813+0000 7fd06aa00640 10 client.29878 handle_command_reply: tid=0 2024-04-16T17:21:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.813+0000 7fd08a445740 1 client.29878 shutdown 2024-04-16T17:21:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.813+0000 7fd08a445740 1 -- 172.21.15.18:0/1019224987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77306ee20 msgr2=0x55d773075ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.813+0000 7fd08a445740 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77306ee20 0x55d773075ca0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd06c036040 tx=0x7fd06c02cd00 comp rx=0 tx=0).stop 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.813+0000 7fd08a445740 1 -- 172.21.15.18:0/1019224987 shutdown_connections 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.813+0000 7fd08a445740 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d7730e7db0 0x55d7730ea1d0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.814+0000 7fd08a445740 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d77306ee20 0x55d773075ca0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.814+0000 7fd08a445740 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d77305f6d0 0x55d773075760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.814+0000 7fd08a445740 1 --2- 172.21.15.18:0/1019224987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d77305ec30 0x55d773078980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.814+0000 7fd08a445740 1 -- 172.21.15.18:0/1019224987 >> 172.21.15.18:0/1019224987 conn(0x55d772ff95d0 msgr2=0x55d773057e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.814+0000 7fd06aa00640 0 client.29878 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.814+0000 7fd08a445740 1 -- 172.21.15.18:0/1019224987 shutdown_connections 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.814+0000 7fd08a445740 1 -- 172.21.15.18:0/1019224987 wait complete. 2024-04-16T17:21:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.814+0000 7fd08a445740 20 client.29878 trim_cache size 0 max 16384 2024-04-16T17:21:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:21:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:21:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:21:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 -- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0640a3de0 msgr2=0x7fd0640a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0640a3de0 0x7fd0640a62a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0096c0 tx=0x7fd06c005a40 comp rx=0 tx=0).stop 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 -- 172.21.15.18:0/4064635714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080109210 msgr2=0x7fd0801a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080109210 0x7fd0801a11f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd07800bd70 tx=0x7fd07800be60 comp rx=0 tx=0).stop 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 -- 172.21.15.18:0/4064635714 shutdown_connections 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd0640a3de0 0x7fd0640a62a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd080115150 0x7fd0801a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd080109b30 0x7fd0801a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 --2- 172.21.15.18:0/4064635714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd080109210 0x7fd0801a11f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 -- 172.21.15.18:0/4064635714 >> 172.21.15.18:0/4064635714 conn(0x7fd08006d0e0 msgr2=0x7fd080070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.818+0000 7fd087800640 1 -- 172.21.15.18:0/4064635714 shutdown_connections 2024-04-16T17:21:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:30.819+0000 7fd087800640 1 -- 172.21.15.18:0/4064635714 wait complete. 2024-04-16T17:21:31.499 INFO:tasks.workunit.client.0.smithi018.stdout: 10 55448 12.21 MB/sec execute 103 sec latency 1098.430 ms 2024-04-16T17:21:32.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:31 smithi043 ceph-mon[36901]: pgmap v535: 129 pgs: 129 active+clean; 400 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-04-16T17:21:32.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:31 smithi022 ceph-mon[36023]: pgmap v535: 129 pgs: 129 active+clean; 400 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-04-16T17:21:32.499 INFO:tasks.workunit.client.0.smithi018.stdout: 10 55831 12.28 MB/sec execute 104 sec latency 106.684 ms 2024-04-16T17:21:33.463 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:31 smithi018 ceph-mon[31776]: pgmap v535: 129 pgs: 129 active+clean; 400 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-04-16T17:21:33.498 INFO:tasks.workunit.client.0.smithi018.stdout: 10 56245 12.39 MB/sec execute 105 sec latency 122.680 ms 2024-04-16T17:21:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:33 smithi043 ceph-mon[36901]: pgmap v536: 129 pgs: 129 active+clean; 401 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 583 op/s 2024-04-16T17:21:34.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:33 smithi022 ceph-mon[36023]: pgmap v536: 129 pgs: 129 active+clean; 401 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 583 op/s 2024-04-16T17:21:34.499 INFO:tasks.workunit.client.0.smithi018.stdout: 10 56714 12.50 MB/sec execute 106 sec latency 165.452 ms 2024-04-16T17:21:35.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:34 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:21:35.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:34 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:21:35.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:34 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:21:35.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:34 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:21:35.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:34 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:21:35.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:34 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:21:35.499 INFO:tasks.workunit.client.0.smithi018.stdout: 10 57110 12.69 MB/sec execute 107 sec latency 186.340 ms 2024-04-16T17:21:36.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:35 smithi022 ceph-mon[36023]: pgmap v537: 129 pgs: 129 active+clean; 397 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 578 op/s 2024-04-16T17:21:36.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:35 smithi043 ceph-mon[36901]: pgmap v537: 129 pgs: 129 active+clean; 397 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 578 op/s 2024-04-16T17:21:36.499 INFO:tasks.workunit.client.0.smithi018.stdout: 10 57359 12.73 MB/sec execute 108 sec latency 183.436 ms 2024-04-16T17:21:37.499 INFO:tasks.workunit.client.0.smithi018.stdout: 10 57610 12.77 MB/sec execute 109 sec latency 175.124 ms 2024-04-16T17:21:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:37 smithi022 ceph-mon[36023]: pgmap v538: 129 pgs: 129 active+clean; 390 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 527 op/s 2024-04-16T17:21:38.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:37 smithi043 ceph-mon[36901]: pgmap v538: 129 pgs: 129 active+clean; 390 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 527 op/s 2024-04-16T17:21:38.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:33 smithi018 ceph-mon[31776]: pgmap v536: 129 pgs: 129 active+clean; 401 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 583 op/s 2024-04-16T17:21:38.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:34 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:21:38.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:34 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:21:38.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:34 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:21:38.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:35 smithi018 ceph-mon[31776]: pgmap v537: 129 pgs: 129 active+clean; 397 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 578 op/s 2024-04-16T17:21:38.499 INFO:tasks.workunit.client.0.smithi018.stdout: 10 57820 12.69 MB/sec execute 110 sec latency 160.134 ms 2024-04-16T17:21:39.502 INFO:tasks.workunit.client.0.smithi018.stdout: 10 58051 12.65 MB/sec execute 111 sec latency 97.860 ms 2024-04-16T17:21:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:39 smithi022 ceph-mon[36023]: pgmap v539: 129 pgs: 129 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-04-16T17:21:40.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:39 smithi043 ceph-mon[36901]: pgmap v539: 129 pgs: 129 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-04-16T17:21:40.501 INFO:tasks.workunit.client.0.smithi018.stdout: 10 58401 12.67 MB/sec execute 112 sec latency 150.644 ms 2024-04-16T17:21:41.502 INFO:tasks.workunit.client.0.smithi018.stdout: 10 58689 12.68 MB/sec execute 113 sec latency 190.816 ms 2024-04-16T17:21:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:41 smithi022 ceph-mon[36023]: pgmap v540: 129 pgs: 129 active+clean; 402 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-04-16T17:21:42.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:41 smithi043 ceph-mon[36901]: pgmap v540: 129 pgs: 129 active+clean; 402 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-04-16T17:21:42.502 INFO:tasks.workunit.client.0.smithi018.stdout: 10 59154 12.77 MB/sec execute 114 sec latency 76.189 ms 2024-04-16T17:21:43.502 INFO:tasks.workunit.client.0.smithi018.stdout: 10 59634 12.91 MB/sec execute 115 sec latency 143.866 ms 2024-04-16T17:21:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 -- 172.21.15.18:0/2540033564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 msgr2=0x7ff798074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 --2- 172.21.15.18:0/2540033564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 0x7ff798074810 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff790009ca0 tx=0x7ff79002c1b0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 -- 172.21.15.18:0/2540033564 shutdown_connections 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 --2- 172.21.15.18:0/2540033564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff79810e820 0x7ff798110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 --2- 172.21.15.18:0/2540033564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff798074d50 0x7ff79810e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 --2- 172.21.15.18:0/2540033564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 0x7ff798074810 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 -- 172.21.15.18:0/2540033564 >> 172.21.15.18:0/2540033564 conn(0x7ff7980fdad0 msgr2=0x7ff7980ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 -- 172.21.15.18:0/2540033564 shutdown_connections 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.024+0000 7ff79ee00640 1 -- 172.21.15.18:0/2540033564 wait complete. 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.026+0000 7ff79ee00640 1 Processor -- start 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.026+0000 7ff79ee00640 1 -- start start 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.027+0000 7ff79ee00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 0x7ff798073820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.027+0000 7ff79ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798074d50 0x7ff79806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.027+0000 7ff797e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 0x7ff798073820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.027+0000 7ff797e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 0x7ff798073820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34372/0 (socket says 172.21.15.18:34372) 2024-04-16T17:21:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.028+0000 7ff797400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798074d50 0x7ff79806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.028+0000 7ff797400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798074d50 0x7ff79806c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:33180/0 (socket says 172.21.15.18:33180) 2024-04-16T17:21:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.028+0000 7ff79ee00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79810e820 0x7ff79806ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.028+0000 7ff797e00640 1 -- 172.21.15.18:0/527427544 learned_addr learned my addr 172.21.15.18:0/527427544 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.029+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff798114660 con 0x7ff79810e820 2024-04-16T17:21:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.029+0000 7ff79ce00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79810e820 0x7ff79806ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.029+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981144e0 con 0x7ff798074430 2024-04-16T17:21:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.029+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981147e0 con 0x7ff798074d50 2024-04-16T17:21:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.032+0000 7ff79ce00640 1 -- 172.21.15.18:0/527427544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798074d50 msgr2=0x7ff79806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.032+0000 7ff79ce00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798074d50 0x7ff79806c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.032+0000 7ff79ce00640 1 -- 172.21.15.18:0/527427544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 msgr2=0x7ff798073820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.032+0000 7ff79ce00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 0x7ff798073820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.032+0000 7ff79ce00640 1 -- 172.21.15.18:0/527427544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79806d420 con 0x7ff79810e820 2024-04-16T17:21:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.032+0000 7ff797400640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798074d50 0x7ff79806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.033+0000 7ff79ce00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79810e820 0x7ff79806ce30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e970 tx=0x7ff78000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.034+0000 7ff794c00640 1 -- 172.21.15.18:0/527427544 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78000ccc0 con 0x7ff79810e820 2024-04-16T17:21:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.034+0000 7ff794c00640 1 -- 172.21.15.18:0/527427544 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff780009e40 con 0x7ff79810e820 2024-04-16T17:21:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.034+0000 7ff794c00640 1 -- 172.21.15.18:0/527427544 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780004a00 con 0x7ff79810e820 2024-04-16T17:21:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.036+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79806d710 con 0x7ff79810e820 2024-04-16T17:21:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.036+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff798071670 con 0x7ff79810e820 2024-04-16T17:21:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.041+0000 7ff7a1ba5740 1 Processor -- start 2024-04-16T17:21:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.045+0000 7ff794c00640 1 -- 172.21.15.18:0/527427544 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff780004ba0 con 0x7ff79810e820 2024-04-16T17:21:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.045+0000 7ff7a1ba5740 1 -- start start 2024-04-16T17:21:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.046+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9221610 0x55f8c9220360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.046+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 0x55f8c9220d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.047+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c922c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.047+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f8c9185780 con 0x55f8c92208a0 2024-04-16T17:21:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.047+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f8c912bf40 con 0x55f8c9221610 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.047+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f8c9181740 con 0x55f8c9229e20 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.047+0000 7ff797400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9221610 0x55f8c9220360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.047+0000 7ff797400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9221610 0x55f8c9220360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34380/0 (socket says 172.21.15.18:34380) 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.047+0000 7ff797400640 1 -- 172.21.15.18:0/3751230568 learned_addr learned my addr 172.21.15.18:0/3751230568 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.048+0000 7ff797400640 1 --2- 172.21.15.18:0/3751230568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c922c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.048+0000 7ff797400640 1 -- 172.21.15.18:0/3751230568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 msgr2=0x55f8c922c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.048+0000 7ff797400640 1 --2- 172.21.15.18:0/3751230568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c922c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.048+0000 7ff797400640 1 -- 172.21.15.18:0/3751230568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 msgr2=0x55f8c9220d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.048+0000 7ff797400640 1 --2- 172.21.15.18:0/3751230568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 0x55f8c9220d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.050+0000 7ff797400640 1 -- 172.21.15.18:0/3751230568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c8e98210 con 0x55f8c9221610 2024-04-16T17:21:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.050+0000 7ff794c00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff77c0a3ea0 0x7ff77c0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.050+0000 7ff794c00640 1 -- 172.21.15.18:0/527427544 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7ff7800d11b0 con 0x7ff79810e820 2024-04-16T17:21:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.051+0000 7ff797e00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff77c0a3ea0 0x7ff77c0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.051+0000 7ff797400640 1 --2- 172.21.15.18:0/3751230568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9221610 0x55f8c9220360 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff788009f30 tx=0x7ff788009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c2aea649a6bc23 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.052+0000 7ff78d600640 1 -- 172.21.15.18:0/3751230568 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788022070 con 0x55f8c9221610 2024-04-16T17:21:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.052+0000 7ff78d600640 1 -- 172.21.15.18:0/3751230568 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78801eb80 con 0x55f8c9221610 2024-04-16T17:21:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.052+0000 7ff78d600640 1 -- 172.21.15.18:0/3751230568 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78800e700 con 0x55f8c9221610 2024-04-16T17:21:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.052+0000 7ff797e00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff77c0a3ea0 0x7ff77c0a6360 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55f8c9221d70 tx=0x7ff79002e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.054+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3751230568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9221610 msgr2=0x55f8c9220360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.054+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3751230568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9221610 0x55f8c9220360 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff788009f30 tx=0x7ff788009a90 comp rx=0 tx=0).stop 2024-04-16T17:21:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.055+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3751230568 shutdown_connections 2024-04-16T17:21:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.055+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3751230568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c922c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.055+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3751230568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 0x55f8c9220d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.055+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3751230568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9221610 0x55f8c9220360 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.055+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3751230568 >> 172.21.15.18:0/3751230568 conn(0x55f8c91bb5d0 msgr2=0x55f8c921b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.056+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3751230568 shutdown_connections 2024-04-16T17:21:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.056+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3751230568 wait complete. 2024-04-16T17:21:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.056+0000 7ff7a1ba5740 1 -- start start 2024-04-16T17:21:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.057+0000 7ff7a1ba5740 1 Processor -- start 2024-04-16T17:21:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.058+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c92208a0 0x55f8c9226d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.058+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 0x55f8c92272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.058+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c92277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.058+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f8c922cb50 con 0x55f8c9221610 2024-04-16T17:21:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.058+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f8c913f390 con 0x55f8c9229e20 2024-04-16T17:21:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.058+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f8c922ccd0 con 0x55f8c92208a0 2024-04-16T17:21:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.058+0000 7ff7a1ba5740 10 client.0 _ll_register_callbacks cb 0x55f8c9128f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.059+0000 7ff797e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c92208a0 0x55f8c9226d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.059+0000 7ff797e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c92208a0 0x55f8c9226d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:33200/0 (socket says 172.21.15.18:33200) 2024-04-16T17:21:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.059+0000 7ff797e00640 1 -- 172.21.15.18:0/3474486466 learned_addr learned my addr 172.21.15.18:0/3474486466 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.059+0000 7ff79ce00640 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c92277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.060+0000 7ff797400640 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 0x55f8c92272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.060+0000 7ff79ce00640 1 -- 172.21.15.18:0/3474486466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c92208a0 msgr2=0x55f8c9226d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.060+0000 7ff79ce00640 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c92208a0 0x55f8c9226d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.060+0000 7ff79ce00640 1 -- 172.21.15.18:0/3474486466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 msgr2=0x55f8c92272a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.060+0000 7ff79ce00640 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 0x55f8c92272a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.060+0000 7ff79ce00640 1 -- 172.21.15.18:0/3474486466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c8f64620 con 0x55f8c9229e20 2024-04-16T17:21:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.061+0000 7ff79ce00640 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c92277e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff7800ce490 tx=0x7ff78009c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a17f421aceec7c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.061+0000 7ff786000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.062+0000 7ff786000640 1 -- 172.21.15.18:0/3474486466 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7800b5020 con 0x55f8c9229e20 2024-04-16T17:21:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.062+0000 7ff786000640 1 -- 172.21.15.18:0/3474486466 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7ff780099350 con 0x55f8c9229e20 2024-04-16T17:21:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.062+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3474486466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78801eb80 con 0x55f8c9229e20 2024-04-16T17:21:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.063+0000 7ff786000640 1 -- 172.21.15.18:0/3474486466 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff7800b5020 con 0x55f8c9229e20 2024-04-16T17:21:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.063+0000 7ff7a1ba5740 10 client.30080 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.063+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3474486466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f8c9249dc0 con 0x55f8c9229e20 2024-04-16T17:21:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.063+0000 7ff786000640 1 -- 172.21.15.18:0/3474486466 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78801eb80 con 0x55f8c9229e20 2024-04-16T17:21:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.064+0000 7ff786000640 1 -- 172.21.15.18:0/3474486466 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff78001f070 con 0x55f8c9229e20 2024-04-16T17:21:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.064+0000 7ff7a1ba5740 10 client.30080 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.064+0000 7ff7a1ba5740 10 client.30080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.064+0000 7ff7a1ba5740 20 client.30080 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.064+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f8c924b2a0 0x55f8c924d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.064+0000 7ff7a1ba5740 4 client.30080 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.064+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3474486466 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8c91931a0 con 0x55f8c924b2a0 2024-04-16T17:21:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.065+0000 7ff797e00640 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f8c924b2a0 0x55f8c924d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.066+0000 7ff797e00640 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f8c924b2a0 0x55f8c924d690 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cdf8213582c4577 server_cookie=a9f335349489fa31 in_seq=0 out_seq=0 2024-04-16T17:21:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.066+0000 7ff786000640 10 client.30080 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.070+0000 7ff786000640 1 -- 172.21.15.18:0/3474486466 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8c91931a0 con 0x55f8c924b2a0 2024-04-16T17:21:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.070+0000 7ff786000640 10 client.30080 handle_command_reply: tid=0 2024-04-16T17:21:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.070+0000 7ff7a1ba5740 1 client.30080 shutdown 2024-04-16T17:21:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 msgr2=0x55f8c92277e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c92277e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff7800ce490 tx=0x7ff78009c730 comp rx=0 tx=0).stop 2024-04-16T17:21:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3474486466 shutdown_connections 2024-04-16T17:21:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f8c924b2a0 0x55f8c924d690 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c92277e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 0x55f8c92272a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3474486466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c92208a0 0x55f8c9226d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3474486466 >> 172.21.15.18:0/3474486466 conn(0x55f8c91bb5d0 msgr2=0x55f8c921d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.071+0000 7ff786000640 0 client.30080 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.072+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3474486466 shutdown_connections 2024-04-16T17:21:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.072+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3474486466 wait complete. 2024-04-16T17:21:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.072+0000 7ff7a1ba5740 20 client.30080 trim_cache size 0 max 16384 2024-04-16T17:21:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.104+0000 7ff7a1ba5740 1 Processor -- start 2024-04-16T17:21:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.104+0000 7ff7a1ba5740 1 -- start start 2024-04-16T17:21:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.105+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 0x55f8c9235a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.105+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9221610 0x55f8c9225ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.105+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c9228b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.105+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b8570 con 0x55f8c92208a0 2024-04-16T17:21:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.105+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b5e40 con 0x55f8c9229e20 2024-04-16T17:21:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.105+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b3d50 con 0x55f8c9221610 2024-04-16T17:21:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.106+0000 7ff79ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c9228b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.106+0000 7ff79ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c9228b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:34400/0 (socket says 172.21.15.18:34400) 2024-04-16T17:21:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.106+0000 7ff79ce00640 1 -- 172.21.15.18:0/2727588838 learned_addr learned my addr 172.21.15.18:0/2727588838 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.106+0000 7ff797400640 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9221610 0x55f8c9225ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.106+0000 7ff797e00640 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 0x55f8c9235a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.107+0000 7ff79ce00640 1 -- 172.21.15.18:0/2727588838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9221610 msgr2=0x55f8c9225ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.107+0000 7ff79ce00640 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9221610 0x55f8c9225ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.107+0000 7ff79ce00640 1 -- 172.21.15.18:0/2727588838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 msgr2=0x55f8c9235a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.107+0000 7ff79ce00640 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 0x55f8c9235a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.107+0000 7ff79ce00640 1 -- 172.21.15.18:0/2727588838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c924dcf0 con 0x55f8c9229e20 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.108+0000 7ff79ce00640 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c9228b90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff780005250 tx=0x7ff78009b1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ab312f1fa7eb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.108+0000 7ff78cc00640 1 -- 172.21.15.18:0/2727588838 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7800b6220 con 0x55f8c9229e20 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.108+0000 7ff78cc00640 1 -- 172.21.15.18:0/2727588838 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78009d3e0 con 0x55f8c9229e20 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.108+0000 7ff78cc00640 1 -- 172.21.15.18:0/2727588838 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7800b6220 con 0x55f8c9229e20 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.109+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/2727588838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 msgr2=0x55f8c9228b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.109+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c9228b90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff780005250 tx=0x7ff78009b1f0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.111+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/2727588838 shutdown_connections 2024-04-16T17:21:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.111+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c9229e20 0x55f8c9228b90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.111+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9221610 0x55f8c9225ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.111+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/2727588838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c92208a0 0x55f8c9235a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.112+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/2727588838 >> 172.21.15.18:0/2727588838 conn(0x55f8c91bb5d0 msgr2=0x55f8c9228480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.112+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/2727588838 shutdown_connections 2024-04-16T17:21:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.112+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/2727588838 wait complete. 2024-04-16T17:21:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.113+0000 7ff7a1ba5740 1 -- start start 2024-04-16T17:21:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.113+0000 7ff7a1ba5740 1 Processor -- start 2024-04-16T17:21:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c92208a0 0x55f8c9231f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 0x55f8c92324a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff7a1ba5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c9222fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b43a0 con 0x55f8c9221610 2024-04-16T17:21:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b5770 con 0x55f8c92208a0 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff7a1ba5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b49f0 con 0x55f8c9229e20 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff7a1ba5740 10 client.0 _ll_register_callbacks cb 0x7ff780004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff79ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c9222fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff79ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c9222fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:33216/0 (socket says 172.21.15.18:33216) 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff79ce00640 1 -- 172.21.15.18:0/3910407421 learned_addr learned my addr 172.21.15.18:0/3910407421 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.114+0000 7ff797e00640 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c92208a0 0x55f8c9231f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.115+0000 7ff797400640 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 0x55f8c92324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.115+0000 7ff79ce00640 1 -- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c92208a0 msgr2=0x55f8c9231f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.115+0000 7ff79ce00640 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c92208a0 0x55f8c9231f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.115+0000 7ff79ce00640 1 -- 172.21.15.18:0/3910407421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 msgr2=0x55f8c92324a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.115+0000 7ff79ce00640 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 0x55f8c92324a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.115+0000 7ff79ce00640 1 -- 172.21.15.18:0/3910407421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c921fb70 con 0x55f8c9229e20 2024-04-16T17:21:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.115+0000 7ff797e00640 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c92208a0 0x55f8c9231f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.115+0000 7ff79ce00640 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c9222fd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e940 tx=0x7ff780098c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=620e8ba449aaf63b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.116+0000 7ff786000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.116+0000 7ff786000640 1 -- 172.21.15.18:0/3910407421 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7800b6220 con 0x55f8c9229e20 2024-04-16T17:21:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.116+0000 7ff786000640 1 -- 172.21.15.18:0/3910407421 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7ff7800a5db0 con 0x55f8c9229e20 2024-04-16T17:21:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.116+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3910407421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78009d3e0 con 0x55f8c9229e20 2024-04-16T17:21:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.116+0000 7ff786000640 1 -- 172.21.15.18:0/3910407421 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff7800a5950 con 0x55f8c9229e20 2024-04-16T17:21:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.116+0000 7ff7a1ba5740 10 client.29896 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.116+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3910407421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f8c921fb70 con 0x55f8c9229e20 2024-04-16T17:21:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.117+0000 7ff786000640 1 -- 172.21.15.18:0/3910407421 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78009d3e0 con 0x55f8c9229e20 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.118+0000 7ff786000640 1 -- 172.21.15.18:0/3910407421 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff78007bb80 con 0x55f8c9229e20 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.118+0000 7ff7a1ba5740 10 client.29896 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.118+0000 7ff7a1ba5740 10 client.29896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.118+0000 7ff7a1ba5740 20 client.29896 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.118+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f8c9224a10 0x55f8c92b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.118+0000 7ff7a1ba5740 4 client.29896 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.118+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3910407421 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8c918d010 con 0x55f8c9224a10 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.118+0000 7ff797e00640 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f8c9224a10 0x55f8c92b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.119+0000 7ff797e00640 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f8c9224a10 0x55f8c92b4810 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf22bf96881877e6 server_cookie=34e8213714197877 in_seq=0 out_seq=0 2024-04-16T17:21:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:44.119+0000 7ff786000640 10 client.29896 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:43 smithi022 ceph-mon[36023]: pgmap v541: 129 pgs: 129 active+clean; 404 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-04-16T17:21:44.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:43 smithi043 ceph-mon[36901]: pgmap v541: 129 pgs: 129 active+clean; 404 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-04-16T17:21:44.503 INFO:tasks.workunit.client.0.smithi018.stdout: 10 59915 12.92 MB/sec execute 116 sec latency 266.996 ms 2024-04-16T17:21:44.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:37 smithi018 ceph-mon[31776]: pgmap v538: 129 pgs: 129 active+clean; 390 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 527 op/s 2024-04-16T17:21:44.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:39 smithi018 ceph-mon[31776]: pgmap v539: 129 pgs: 129 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-04-16T17:21:44.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:41 smithi018 ceph-mon[31776]: pgmap v540: 129 pgs: 129 active+clean; 402 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-04-16T17:21:44.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:43 smithi018 ceph-mon[31776]: pgmap v541: 129 pgs: 129 active+clean; 404 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-04-16T17:21:45.502 INFO:tasks.workunit.client.0.smithi018.stdout: 10 60127 12.90 MB/sec execute 117 sec latency 1096.266 ms 2024-04-16T17:21:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.513+0000 7ff786000640 1 -- 172.21.15.18:0/3910407421 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8c918d010 con 0x55f8c9224a10 2024-04-16T17:21:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.513+0000 7ff786000640 10 client.29896 handle_command_reply: tid=0 2024-04-16T17:21:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.513+0000 7ff7a1ba5740 1 client.29896 shutdown 2024-04-16T17:21:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.515+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3910407421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 msgr2=0x55f8c9222fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.515+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c9222fd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e940 tx=0x7ff780098c40 comp rx=0 tx=0).stop 2024-04-16T17:21:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.519+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3910407421 shutdown_connections 2024-04-16T17:21:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.519+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f8c9224a10 0x55f8c92b4810 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.519+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8c9229e20 0x55f8c9222fd0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.519+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f8c9221610 0x55f8c92324a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.519+0000 7ff7a1ba5740 1 --2- 172.21.15.18:0/3910407421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8c92208a0 0x55f8c9231f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.519+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3910407421 >> 172.21.15.18:0/3910407421 conn(0x55f8c91bb5d0 msgr2=0x55f8c921a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.519+0000 7ff786000640 0 client.29896 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.521+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3910407421 shutdown_connections 2024-04-16T17:21:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.522+0000 7ff7a1ba5740 1 -- 172.21.15.18:0/3910407421 wait complete. 2024-04-16T17:21:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.522+0000 7ff7a1ba5740 20 client.29896 trim_cache size 0 max 16384 2024-04-16T17:21:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:21:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:21:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:21:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:21:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff77c0a3ea0 msgr2=0x7ff77c0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff77c0a3ea0 0x7ff77c0a6360 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55f8c9221d70 tx=0x7ff79002e040 comp rx=0 tx=0).stop 2024-04-16T17:21:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79810e820 msgr2=0x7ff79806ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79810e820 0x7ff79806ce30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e970 tx=0x7ff78000ed70 comp rx=0 tx=0).stop 2024-04-16T17:21:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 shutdown_connections 2024-04-16T17:21:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff77c0a3ea0 0x7ff77c0a6360 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff79810e820 0x7ff79806ce30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff798074d50 0x7ff79806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 --2- 172.21.15.18:0/527427544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff798074430 0x7ff798073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.523+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 >> 172.21.15.18:0/527427544 conn(0x7ff7980fdad0 msgr2=0x7ff7980ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.524+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 shutdown_connections 2024-04-16T17:21:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:45.524+0000 7ff79ee00640 1 -- 172.21.15.18:0/527427544 wait complete. 2024-04-16T17:21:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:45 smithi022 ceph-mon[36023]: pgmap v542: 129 pgs: 129 active+clean; 405 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-04-16T17:21:46.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:45 smithi043 ceph-mon[36901]: pgmap v542: 129 pgs: 129 active+clean; 405 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-04-16T17:21:46.502 INFO:tasks.workunit.client.0.smithi018.stdout: 10 60423 12.98 MB/sec execute 118 sec latency 362.823 ms 2024-04-16T17:21:47.502 INFO:tasks.workunit.client.0.smithi018.stdout: 10 60663 13.04 MB/sec execute 119 sec latency 271.615 ms 2024-04-16T17:21:47.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:45 smithi018 ceph-mon[31776]: pgmap v542: 129 pgs: 129 active+clean; 405 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-04-16T17:21:47.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:47 smithi018 ceph-mon[31776]: pgmap v543: 129 pgs: 129 active+clean; 411 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 471 op/s 2024-04-16T17:21:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:47 smithi022 ceph-mon[36023]: pgmap v543: 129 pgs: 129 active+clean; 411 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 471 op/s 2024-04-16T17:21:48.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:47 smithi043 ceph-mon[36901]: pgmap v543: 129 pgs: 129 active+clean; 411 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 471 op/s 2024-04-16T17:21:48.503 INFO:tasks.workunit.client.0.smithi018.stdout: 10 60702 12.94 MB/sec execute 120 sec latency 928.534 ms 2024-04-16T17:21:49.503 INFO:tasks.workunit.client.0.smithi018.stdout: 10 60865 12.91 MB/sec execute 121 sec latency 1026.011 ms 2024-04-16T17:21:50.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:49 smithi022 ceph-mon[36023]: pgmap v544: 129 pgs: 129 active+clean; 413 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 458 op/s 2024-04-16T17:21:50.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:49 smithi043 ceph-mon[36901]: pgmap v544: 129 pgs: 129 active+clean; 413 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 458 op/s 2024-04-16T17:21:50.503 INFO:tasks.workunit.client.0.smithi018.stdout: 10 61135 12.89 MB/sec execute 122 sec latency 152.036 ms 2024-04-16T17:21:50.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:21:50.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:21:51.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.154+0000 7fb0f8400640 1 -- 172.21.15.18:0/3230681283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f0102bd0 msgr2=0x7fb0f0106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.154+0000 7fb0f8400640 1 --2- 172.21.15.18:0/3230681283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f0102bd0 0x7fb0f0106c10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc02bc40 comp rx=0 tx=0).stop 2024-04-16T17:21:51.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.155+0000 7fb0f8400640 1 -- 172.21.15.18:0/3230681283 shutdown_connections 2024-04-16T17:21:51.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.155+0000 7fb0f8400640 1 --2- 172.21.15.18:0/3230681283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0f01072f0 0x7fb0f010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.155+0000 7fb0f8400640 1 --2- 172.21.15.18:0/3230681283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f0102bd0 0x7fb0f0106c10 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.155+0000 7fb0f8400640 1 --2- 172.21.15.18:0/3230681283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0f0102220 0x7fb0f0102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.155+0000 7fb0f8400640 1 -- 172.21.15.18:0/3230681283 >> 172.21.15.18:0/3230681283 conn(0x7fb0f00fdac0 msgr2=0x7fb0f00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:51.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.155+0000 7fb0f8400640 1 -- 172.21.15.18:0/3230681283 shutdown_connections 2024-04-16T17:21:51.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.156+0000 7fb0f8400640 1 -- 172.21.15.18:0/3230681283 wait complete. 2024-04-16T17:21:51.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.157+0000 7fb0f8400640 1 Processor -- start 2024-04-16T17:21:51.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.157+0000 7fb0f8400640 1 -- start start 2024-04-16T17:21:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.158+0000 7fb0f8400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0f0102220 0x7fb0f01166a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.158+0000 7fb0f8400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0f01072f0 0x7fb0f0116be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.158+0000 7fb0f8400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f011c3b0 0x7fb0f011c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.158+0000 7fb0f8400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0075f50 con 0x7fb0f0102220 2024-04-16T17:21:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.158+0000 7fb0f8400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0075dd0 con 0x7fb0f011c3b0 2024-04-16T17:21:51.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.158+0000 7fb0f8400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f00760d0 con 0x7fb0f01072f0 2024-04-16T17:21:51.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.159+0000 7fb0f5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0f01072f0 0x7fb0f0116be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.159+0000 7fb0f5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0f01072f0 0x7fb0f0116be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52242/0 (socket says 172.21.15.18:52242) 2024-04-16T17:21:51.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.159+0000 7fb0f5000640 1 -- 172.21.15.18:0/990937153 learned_addr learned my addr 172.21.15.18:0/990937153 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.159+0000 7fb0f6400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f011c3b0 0x7fb0f011c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.160+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0f0102220 0x7fb0f01166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.160+0000 7fb0f5000640 1 -- 172.21.15.18:0/990937153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f011c3b0 msgr2=0x7fb0f011c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.160+0000 7fb0f5000640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f011c3b0 0x7fb0f011c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.160+0000 7fb0f5000640 1 -- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0f0102220 msgr2=0x7fb0f01166a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.160+0000 7fb0f5000640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0f0102220 0x7fb0f01166a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.160+0000 7fb0f5000640 1 -- 172.21.15.18:0/990937153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f01117b0 con 0x7fb0f01072f0 2024-04-16T17:21:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.160+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0f0102220 0x7fb0f01166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:51.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.161+0000 7fb0f5000640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0f01072f0 0x7fb0f0116be0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc02c250 tx=0x7fb0dc0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:51.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.162+0000 7fb0ee000640 1 -- 172.21.15.18:0/990937153 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc041070 con 0x7fb0f01072f0 2024-04-16T17:21:51.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.162+0000 7fb0ee000640 1 -- 172.21.15.18:0/990937153 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc02cca0 con 0x7fb0f01072f0 2024-04-16T17:21:51.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.162+0000 7fb0ee000640 1 -- 172.21.15.18:0/990937153 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc003db0 con 0x7fb0f01072f0 2024-04-16T17:21:51.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.162+0000 7fb0f8400640 1 -- 172.21.15.18:0/990937153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f0111a40 con 0x7fb0f01072f0 2024-04-16T17:21:51.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.162+0000 7fb0f8400640 1 -- 172.21.15.18:0/990937153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f0111f30 con 0x7fb0f01072f0 2024-04-16T17:21:51.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.169+0000 7fb0ee000640 1 -- 172.21.15.18:0/990937153 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb0dc044070 con 0x7fb0f01072f0 2024-04-16T17:21:51.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.171+0000 7fb0fb06d740 1 Processor -- start 2024-04-16T17:21:51.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.172+0000 7fb0fb06d740 1 -- start start 2024-04-16T17:21:51.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.172+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.172+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.172+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.172+0000 7fb0fb06d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a853120 con 0x55fb6a8f66b0 2024-04-16T17:21:51.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.172+0000 7fb0fb06d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a7f9340 con 0x55fb6a8edad0 2024-04-16T17:21:51.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.172+0000 7fb0fb06d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a84f0e0 con 0x55fb6a8ed1b0 2024-04-16T17:21:51.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.173+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.173+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45636/0 (socket says 172.21.15.18:45636) 2024-04-16T17:21:51.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.173+0000 7fb0f5a00640 1 -- 172.21.15.18:0/817606133 learned_addr learned my addr 172.21.15.18:0/817606133 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.173+0000 7fb0ee000640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb0d80a3f20 0x7fb0d80a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.173+0000 7fb0ee000640 1 -- 172.21.15.18:0/990937153 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fb0dc0f3690 con 0x7fb0f01072f0 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.174+0000 7fb0f5a00640 1 -- 172.21.15.18:0/817606133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 msgr2=0x55fb6a8ed590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.174+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/817606133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8ed590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.174+0000 7fb0f5a00640 1 -- 172.21.15.18:0/817606133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8f66b0 msgr2=0x55fb6a8f8aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.174+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/817606133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f8aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.174+0000 7fb0f5a00640 1 -- 172.21.15.18:0/817606133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6a62f7d0 con 0x55fb6a8edad0 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.175+0000 7fb0f6400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb0d80a3f20 0x7fb0d80a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.177+0000 7fb0f6400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb0d80a3f20 0x7fb0d80a63e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005fd0 tx=0x7fb0e8009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.177+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/817606133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f6170 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4005fd0 tx=0x7fb0e400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bac8581483039f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.177+0000 7fb0e2a00640 1 -- 172.21.15.18:0/817606133 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4016070 con 0x55fb6a8edad0 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.177+0000 7fb0e2a00640 1 -- 172.21.15.18:0/817606133 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e4005500 con 0x55fb6a8edad0 2024-04-16T17:21:51.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.177+0000 7fb0e2a00640 1 -- 172.21.15.18:0/817606133 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40116d0 con 0x55fb6a8edad0 2024-04-16T17:21:51.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.178+0000 7fb0fb06d740 1 -- 172.21.15.18:0/817606133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8edad0 msgr2=0x55fb6a8f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.178+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/817606133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f6170 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4005fd0 tx=0x7fb0e400e6c0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.179+0000 7fb0fb06d740 1 -- 172.21.15.18:0/817606133 shutdown_connections 2024-04-16T17:21:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.179+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/817606133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.179+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/817606133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f6170 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.179+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/817606133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.179+0000 7fb0fb06d740 1 -- 172.21.15.18:0/817606133 >> 172.21.15.18:0/817606133 conn(0x55fb6a8e78e0 msgr2=0x55fb6a8e7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.180+0000 7fb0fb06d740 1 -- 172.21.15.18:0/817606133 shutdown_connections 2024-04-16T17:21:51.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.180+0000 7fb0fb06d740 1 -- 172.21.15.18:0/817606133 wait complete. 2024-04-16T17:21:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.180+0000 7fb0fb06d740 1 -- start start 2024-04-16T17:21:51.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.181+0000 7fb0fb06d740 1 Processor -- start 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.181+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.181+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.181+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.182+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.182+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52250/0 (socket says 172.21.15.18:52250) 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.182+0000 7fb0f5a00640 1 -- 172.21.15.18:0/2239588253 learned_addr learned my addr 172.21.15.18:0/2239588253 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.182+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a8f94a0 con 0x55fb6a8edad0 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.182+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.182+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a80c710 con 0x55fb6a8f66b0 2024-04-16T17:21:51.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.182+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a8f9620 con 0x55fb6a8ed1b0 2024-04-16T17:21:51.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.184+0000 7fb0fb06d740 10 client.0 _ll_register_callbacks cb 0x55fb6a807af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:51.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.184+0000 7fb0f6400640 1 -- 172.21.15.18:0/2239588253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 msgr2=0x55fb6a8f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:51.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.184+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.184+0000 7fb0f6400640 1 -- 172.21.15.18:0/2239588253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8edad0 msgr2=0x55fb6a8f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:51.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.184+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.184+0000 7fb0f6400640 1 -- 172.21.15.18:0/2239588253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6a5653f0 con 0x55fb6a8f66b0 2024-04-16T17:21:51.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.184+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e4005500 con 0x55fb6a8f66b0 2024-04-16T17:21:51.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.184+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f4240 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80042d0 tx=0x7fb0e8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9655438f163a1a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:51.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.185+0000 7fb0d7400640 10 client.30098 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:51.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.185+0000 7fb0d7400640 1 -- 172.21.15.18:0/2239588253 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8029060 con 0x55fb6a8f66b0 2024-04-16T17:21:51.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.185+0000 7fb0d7400640 1 -- 172.21.15.18:0/2239588253 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fb0e801cb90 con 0x55fb6a8f66b0 2024-04-16T17:21:51.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.185+0000 7fb0d7400640 1 -- 172.21.15.18:0/2239588253 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb0e801cdb0 con 0x55fb6a8f66b0 2024-04-16T17:21:51.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.185+0000 7fb0fb06d740 10 client.30098 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:51.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.185+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55fb6a5250f0 con 0x55fb6a8f66b0 2024-04-16T17:21:51.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.186+0000 7fb0d7400640 1 -- 172.21.15.18:0/2239588253 <== mon.2 v2:172.21.15.22:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fb0e8018040 con 0x55fb6a8f66b0 2024-04-16T17:21:51.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.186+0000 7fb0fb06d740 10 client.30098 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:51.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.186+0000 7fb0fb06d740 10 client.30098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:51.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.186+0000 7fb0fb06d740 20 client.30098 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:51.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.186+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb6a906950 0x55fb6a908d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.186+0000 7fb0fb06d740 4 client.30098 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:51.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.186+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb6a860b40 con 0x55fb6a906950 2024-04-16T17:21:51.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.187+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb6a906950 0x55fb6a908d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.187+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb6a906950 0x55fb6a908d60 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27943a26ca9b2e86 server_cookie=f617ccabd0a0fc49 in_seq=0 out_seq=0 2024-04-16T17:21:51.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.188+0000 7fb0d7400640 10 client.30098 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:51.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.188+0000 7fb0d7400640 1 -- 172.21.15.18:0/2239588253 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb6a860b40 con 0x55fb6a906950 2024-04-16T17:21:51.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.188+0000 7fb0d7400640 10 client.30098 handle_command_reply: tid=0 2024-04-16T17:21:51.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.189+0000 7fb0fb06d740 1 client.30098 shutdown 2024-04-16T17:21:51.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.190+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 msgr2=0x55fb6a8f4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.190+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f4240 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80042d0 tx=0x7fb0e8004ab0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.191+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 shutdown_connections 2024-04-16T17:21:51.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.191+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb6a906950 0x55fb6a908d60 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.191+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f4240 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.191+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.191+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2239588253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.191+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 >> 172.21.15.18:0/2239588253 conn(0x55fb6a8e78e0 msgr2=0x55fb6a8e9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:51.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.191+0000 7fb0d7400640 0 client.30098 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:51.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.192+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 shutdown_connections 2024-04-16T17:21:51.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.192+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2239588253 wait complete. 2024-04-16T17:21:51.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.192+0000 7fb0fb06d740 20 client.30098 trim_cache size 0 max 16384 2024-04-16T17:21:51.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.231+0000 7fb0fb06d740 1 Processor -- start 2024-04-16T17:21:51.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.233+0000 7fb0fb06d740 1 -- start start 2024-04-16T17:21:51.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8edad0 0x55fb6a902390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0fb06d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01c0d60 con 0x55fb6a8ed1b0 2024-04-16T17:21:51.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0fb06d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01be630 con 0x55fb6a8f66b0 2024-04-16T17:21:51.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0fb06d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01bc4a0 con 0x55fb6a8edad0 2024-04-16T17:21:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0f6400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0f6400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45652/0 (socket says 172.21.15.18:45652) 2024-04-16T17:21:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0f6400640 1 -- 172.21.15.18:0/2535357964 learned_addr learned my addr 172.21.15.18:0/2535357964 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0f5000640 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8edad0 0x55fb6a902390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0f6400640 1 -- 172.21.15.18:0/2535357964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8edad0 msgr2=0x55fb6a902390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8edad0 0x55fb6a902390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.234+0000 7fb0f6400640 1 -- 172.21.15.18:0/2535357964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8ed1b0 msgr2=0x55fb6a8f5ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:51.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.235+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.235+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.235+0000 7fb0f6400640 1 -- 172.21.15.18:0/2535357964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6a8f8e10 con 0x55fb6a8f66b0 2024-04-16T17:21:51.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.236+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f2740 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009270 tx=0x7fb0e801e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f713fb81a57e0928 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.237+0000 7fb0e2000640 1 -- 172.21.15.18:0/2535357964 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8029040 con 0x55fb6a8f66b0 2024-04-16T17:21:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.237+0000 7fb0e2000640 1 -- 172.21.15.18:0/2535357964 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e4005500 con 0x55fb6a8f66b0 2024-04-16T17:21:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.237+0000 7fb0e2000640 1 -- 172.21.15.18:0/2535357964 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8005040 con 0x55fb6a8f66b0 2024-04-16T17:21:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.238+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2535357964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 msgr2=0x55fb6a8f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.238+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f2740 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009270 tx=0x7fb0e801e040 comp rx=0 tx=0).stop 2024-04-16T17:21:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.238+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2535357964 shutdown_connections 2024-04-16T17:21:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.238+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f2740 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.238+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8edad0 0x55fb6a902390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.238+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2535357964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.239+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2535357964 >> 172.21.15.18:0/2535357964 conn(0x55fb6a8e78e0 msgr2=0x55fb6a8e95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:51.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.239+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2535357964 shutdown_connections 2024-04-16T17:21:51.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.239+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2535357964 wait complete. 2024-04-16T17:21:51.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.239+0000 7fb0fb06d740 1 -- start start 2024-04-16T17:21:51.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.240+0000 7fb0fb06d740 1 Processor -- start 2024-04-16T17:21:51.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.241+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.241+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.241+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.241+0000 7fb0fb06d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52276/0 (socket says 172.21.15.18:52276) 2024-04-16T17:21:51.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0f5a00640 1 -- 172.21.15.18:0/2530333392 learned_addr learned my addr 172.21.15.18:0/2530333392 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:51.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01bcaf0 con 0x55fb6a8edad0 2024-04-16T17:21:51.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01bdf60 con 0x55fb6a8f66b0 2024-04-16T17:21:51.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01bd1c0 con 0x55fb6a8ed1b0 2024-04-16T17:21:51.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0fb06d740 10 client.0 _ll_register_callbacks cb 0x7fb0e8003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:51.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0f6400640 1 -- 172.21.15.18:0/2530333392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 msgr2=0x55fb6a8f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:51.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0f6400640 1 -- 172.21.15.18:0/2530333392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8edad0 msgr2=0x55fb6a8f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:51.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.242+0000 7fb0f6400640 1 -- 172.21.15.18:0/2530333392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6a96c170 con 0x55fb6a8f66b0 2024-04-16T17:21:51.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.243+0000 7fb0f6400640 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f1230 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80094d0 tx=0x7fb0e8011d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b85b94422776240 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:51.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.243+0000 7fb0d7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:51.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.247+0000 7fb0d7400640 1 -- 172.21.15.18:0/2530333392 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8029040 con 0x55fb6a8f66b0 2024-04-16T17:21:51.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.247+0000 7fb0d7400640 1 -- 172.21.15.18:0/2530333392 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fb0e801a5c0 con 0x55fb6a8f66b0 2024-04-16T17:21:51.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.247+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e4005500 con 0x55fb6a8f66b0 2024-04-16T17:21:51.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.247+0000 7fb0d7400640 1 -- 172.21.15.18:0/2530333392 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb0e801a7e0 con 0x55fb6a8f66b0 2024-04-16T17:21:51.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0fb06d740 10 client.30104 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:51.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fb6a96d440 con 0x55fb6a8f66b0 2024-04-16T17:21:51.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0d7400640 1 -- 172.21.15.18:0/2530333392 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e4005500 con 0x55fb6a8f66b0 2024-04-16T17:21:51.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0d7400640 1 -- 172.21.15.18:0/2530333392 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fb0e8018040 con 0x55fb6a8f66b0 2024-04-16T17:21:51.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0fb06d740 10 client.30104 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:51.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0fb06d740 10 client.30104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:51.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0fb06d740 20 client.30104 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:51.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb6a96da50 0x55fb6a96fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0fb06d740 4 client.30104 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:21:51.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.248+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb6a85a9b0 con 0x55fb6a96da50 2024-04-16T17:21:51.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.249+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb6a96da50 0x55fb6a96fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.250+0000 7fb0d7400640 10 client.30104 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:51.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.250+0000 7fb0f5a00640 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb6a96da50 0x55fb6a96fe40 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cea9d2cea3fd552 server_cookie=b489404e6b978891 in_seq=0 out_seq=0 2024-04-16T17:21:51.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0d7400640 1 -- 172.21.15.18:0/2530333392 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb6a85a9b0 con 0x55fb6a96da50 2024-04-16T17:21:51.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0d7400640 10 client.30104 handle_command_reply: tid=0 2024-04-16T17:21:51.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 client.30104 shutdown 2024-04-16T17:21:51.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 msgr2=0x55fb6a8f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f1230 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80094d0 tx=0x7fb0e8011d20 comp rx=0 tx=0).stop 2024-04-16T17:21:51.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 shutdown_connections 2024-04-16T17:21:51.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fb6a96da50 0x55fb6a96fe40 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb6a8f66b0 0x55fb6a8f1230 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fb6a8edad0 0x55fb6a8f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 --2- 172.21.15.18:0/2530333392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb6a8ed1b0 0x55fb6a8f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 >> 172.21.15.18:0/2530333392 conn(0x55fb6a8e78e0 msgr2=0x55fb6a8f3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:51.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.256+0000 7fb0d7400640 0 client.30104 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:51.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.258+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 shutdown_connections 2024-04-16T17:21:51.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.258+0000 7fb0fb06d740 1 -- 172.21.15.18:0/2530333392 wait complete. 2024-04-16T17:21:51.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.259+0000 7fb0fb06d740 20 client.30104 trim_cache size 0 max 16384 2024-04-16T17:21:51.260 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:21:51.260 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:21:51.260 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:21:51.261 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:21:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.261+0000 7fb0f8400640 1 -- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb0d80a3f20 msgr2=0x7fb0d80a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.261+0000 7fb0f8400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb0d80a3f20 0x7fb0d80a63e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005fd0 tx=0x7fb0e8009500 comp rx=0 tx=0).stop 2024-04-16T17:21:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.261+0000 7fb0f8400640 1 -- 172.21.15.18:0/990937153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0f01072f0 msgr2=0x7fb0f0116be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.261+0000 7fb0f8400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0f01072f0 0x7fb0f0116be0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc02c250 tx=0x7fb0dc0043b0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.262+0000 7fb0f8400640 1 -- 172.21.15.18:0/990937153 shutdown_connections 2024-04-16T17:21:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.262+0000 7fb0f8400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb0d80a3f20 0x7fb0d80a63e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.262+0000 7fb0f8400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0f011c3b0 0x7fb0f011c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.262+0000 7fb0f8400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0f01072f0 0x7fb0f0116be0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.262+0000 7fb0f8400640 1 --2- 172.21.15.18:0/990937153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0f0102220 0x7fb0f01166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.262+0000 7fb0f8400640 1 -- 172.21.15.18:0/990937153 >> 172.21.15.18:0/990937153 conn(0x7fb0f00fdac0 msgr2=0x7fb0f00feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.262+0000 7fb0f8400640 1 -- 172.21.15.18:0/990937153 shutdown_connections 2024-04-16T17:21:51.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.262+0000 7fb0f8400640 1 -- 172.21.15.18:0/990937153 wait complete. 2024-04-16T17:21:51.300 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:21:51.301 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:21:51.503 INFO:tasks.workunit.client.0.smithi018.stdout: 10 61316 12.85 MB/sec execute 123 sec latency 270.343 ms 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.622+0000 7fb13fe00640 1 -- 172.21.15.18:0/3798710590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb140106ea0 msgr2=0x7fb140107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.622+0000 7fb13fe00640 1 --2- 172.21.15.18:0/3798710590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb140106ea0 0x7fb140107280 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb138009ca0 tx=0x7fb13802c190 comp rx=0 tx=0).stop 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.623+0000 7fb13fe00640 1 -- 172.21.15.18:0/3798710590 shutdown_connections 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.624+0000 7fb13fe00640 1 --2- 172.21.15.18:0/3798710590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1401008b0 0x7fb14010ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.624+0000 7fb13fe00640 1 --2- 172.21.15.18:0/3798710590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1400fff10 0x7fb140100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.624+0000 7fb13fe00640 1 --2- 172.21.15.18:0/3798710590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb140106ea0 0x7fb140107280 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.624+0000 7fb13fe00640 1 -- 172.21.15.18:0/3798710590 >> 172.21.15.18:0/3798710590 conn(0x7fb140076e60 msgr2=0x7fb1400fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.624+0000 7fb13fe00640 1 -- 172.21.15.18:0/3798710590 shutdown_connections 2024-04-16T17:21:51.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.624+0000 7fb13fe00640 1 -- 172.21.15.18:0/3798710590 wait complete. 2024-04-16T17:21:51.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.628+0000 7fb13fe00640 1 Processor -- start 2024-04-16T17:21:51.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.628+0000 7fb13fe00640 1 -- start start 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.628+0000 7fb13fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400fff10 0x7fb140103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.629+0000 7fb13fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1401008b0 0x7fb140101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.629+0000 7fb13fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140106ea0 0x7fb140101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.629+0000 7fb13fe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb140112210 con 0x7fb140106ea0 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.629+0000 7fb13fe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb140112090 con 0x7fb1401008b0 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.629+0000 7fb13fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb140112390 con 0x7fb1400fff10 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.629+0000 7fb13ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400fff10 0x7fb140103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.629+0000 7fb13ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400fff10 0x7fb140103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52306/0 (socket says 172.21.15.18:52306) 2024-04-16T17:21:51.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.629+0000 7fb13ea00640 1 -- 172.21.15.18:0/558804192 learned_addr learned my addr 172.21.15.18:0/558804192 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:51.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.630+0000 7fb13ea00640 1 -- 172.21.15.18:0/558804192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1401008b0 msgr2=0x7fb140101980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:21:51.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.630+0000 7fb13ea00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1401008b0 0x7fb140101980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.630+0000 7fb13ea00640 1 -- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140106ea0 msgr2=0x7fb140101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:51.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.630+0000 7fb13ea00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140106ea0 0x7fb140101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:51.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.630+0000 7fb13ea00640 1 -- 172.21.15.18:0/558804192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14019c830 con 0x7fb1400fff10 2024-04-16T17:21:51.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.630+0000 7fb13ea00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400fff10 0x7fb140103580 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb13802c720 tx=0x7fb138035e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:51.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.631+0000 7fb137400640 1 -- 172.21.15.18:0/558804192 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138039040 con 0x7fb1400fff10 2024-04-16T17:21:51.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.632+0000 7fb137400640 1 -- 172.21.15.18:0/558804192 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb138044070 con 0x7fb1400fff10 2024-04-16T17:21:51.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.632+0000 7fb13fe00640 1 -- 172.21.15.18:0/558804192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14019cac0 con 0x7fb1400fff10 2024-04-16T17:21:51.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.632+0000 7fb137400640 1 -- 172.21.15.18:0/558804192 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1380328d0 con 0x7fb1400fff10 2024-04-16T17:21:51.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.632+0000 7fb13fe00640 1 -- 172.21.15.18:0/558804192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14019d060 con 0x7fb1400fff10 2024-04-16T17:21:51.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.634+0000 7fb134c00640 1 -- 172.21.15.18:0/558804192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb140101330 con 0x7fb1400fff10 2024-04-16T17:21:51.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.639+0000 7fb137400640 1 -- 172.21.15.18:0/558804192 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb138004040 con 0x7fb1400fff10 2024-04-16T17:21:51.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.643+0000 7fb137400640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1240a3de0 0x7fb1240a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:51.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.644+0000 7fb13e000640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1240a3de0 0x7fb1240a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:51.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.649+0000 7fb137400640 1 -- 172.21.15.18:0/558804192 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fb1380f31b0 con 0x7fb1400fff10 2024-04-16T17:21:51.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.649+0000 7fb13e000640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1240a3de0 0x7fb1240a62a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb1401030f0 tx=0x7fb128006c20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:51.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:51.655+0000 7fb137400640 1 -- 172.21.15.18:0/558804192 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1380bf550 con 0x7fb1400fff10 2024-04-16T17:21:52.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:51 smithi022 ceph-mon[36023]: pgmap v545: 129 pgs: 129 active+clean; 399 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 538 op/s 2024-04-16T17:21:52.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.181+0000 7fb134c00640 1 -- 172.21.15.18:0/558804192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb140062c00 con 0x7fb1400fff10 2024-04-16T17:21:52.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.182+0000 7fb137400640 1 -- 172.21.15.18:0/558804192 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7fb1380bee50 con 0x7fb1400fff10 2024-04-16T17:21:52.183 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:21:52.184 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:21:52.184 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:21:52.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 -- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1240a3de0 msgr2=0x7fb1240a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1240a3de0 0x7fb1240a62a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb1401030f0 tx=0x7fb128006c20 comp rx=0 tx=0).stop 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 -- 172.21.15.18:0/558804192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400fff10 msgr2=0x7fb140103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400fff10 0x7fb140103580 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb13802c720 tx=0x7fb138035e50 comp rx=0 tx=0).stop 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 -- 172.21.15.18:0/558804192 shutdown_connections 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1240a3de0 0x7fb1240a62a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140106ea0 0x7fb140101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1401008b0 0x7fb140101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 --2- 172.21.15.18:0/558804192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400fff10 0x7fb140103580 secure :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb13802c720 tx=0x7fb138035e50 comp rx=0 tx=0).stop 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 -- 172.21.15.18:0/558804192 >> 172.21.15.18:0/558804192 conn(0x7fb140076e60 msgr2=0x7fb140104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.187+0000 7fb134c00640 1 -- 172.21.15.18:0/558804192 shutdown_connections 2024-04-16T17:21:52.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.188+0000 7fb134c00640 1 -- 172.21.15.18:0/558804192 wait complete. 2024-04-16T17:21:52.216 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:21:52.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:51 smithi043 ceph-mon[36901]: pgmap v545: 129 pgs: 129 active+clean; 399 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 538 op/s 2024-04-16T17:21:52.503 INFO:tasks.workunit.client.0.smithi018.stdout: 10 61434 12.77 MB/sec execute 124 sec latency 141.725 ms 2024-04-16T17:21:52.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.524+0000 7f8723200640 1 -- 172.21.15.18:0/982512063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410e0e0 msgr2=0x7f872410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.524+0000 7f8723200640 1 --2- 172.21.15.18:0/982512063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410e0e0 0x7f872410e560 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f871c009ca0 tx=0x7f871c02c100 comp rx=0 tx=0).stop 2024-04-16T17:21:52.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.527+0000 7f8723200640 1 -- 172.21.15.18:0/982512063 shutdown_connections 2024-04-16T17:21:52.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.527+0000 7f8723200640 1 --2- 172.21.15.18:0/982512063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872410ec00 0x7f872411d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.527+0000 7f8723200640 1 --2- 172.21.15.18:0/982512063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410e0e0 0x7f872410e560 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.527+0000 7f8723200640 1 --2- 172.21.15.18:0/982512063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8724110580 0x7f8724110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.527+0000 7f8723200640 1 -- 172.21.15.18:0/982512063 >> 172.21.15.18:0/982512063 conn(0x7f872406cf10 msgr2=0x7f872406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:52.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.528+0000 7f8723200640 1 -- 172.21.15.18:0/982512063 shutdown_connections 2024-04-16T17:21:52.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.528+0000 7f8723200640 1 -- 172.21.15.18:0/982512063 wait complete. 2024-04-16T17:21:52.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.530+0000 7f8723200640 1 Processor -- start 2024-04-16T17:21:52.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.531+0000 7f8723200640 1 -- start start 2024-04-16T17:21:52.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.532+0000 7f8723200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872410e0e0 0x7f87241a7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8723200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410ec00 0x7f87241a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8723200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8724110580 0x7f87241abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8723200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8724121030 con 0x7f8724110580 2024-04-16T17:21:52.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8723200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8724120eb0 con 0x7f872410ec00 2024-04-16T17:21:52.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8723200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87241211b0 con 0x7f872410e0e0 2024-04-16T17:21:52.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8721e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872410e0e0 0x7f87241a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8721e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872410e0e0 0x7f87241a7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52316/0 (socket says 172.21.15.18:52316) 2024-04-16T17:21:52.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8721e00640 1 -- 172.21.15.18:0/2508296255 learned_addr learned my addr 172.21.15.18:0/2508296255 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:52.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.533+0000 7f8721400640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410ec00 0x7f87241a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.534+0000 7f8721e00640 1 -- 172.21.15.18:0/2508296255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410ec00 msgr2=0x7f87241a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.534+0000 7f8721e00640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410ec00 0x7f87241a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.534+0000 7f8721e00640 1 -- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8724110580 msgr2=0x7f87241abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.534+0000 7f8721e00640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8724110580 0x7f87241abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.534+0000 7f8721e00640 1 -- 172.21.15.18:0/2508296255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87241ac520 con 0x7f872410e0e0 2024-04-16T17:21:52.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.534+0000 7f8721400640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410ec00 0x7f87241a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:52.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.535+0000 7f8721e00640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872410e0e0 0x7f87241a7530 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f871400b4a0 tx=0x7f871400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:52.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.536+0000 7f871aa00640 1 -- 172.21.15.18:0/2508296255 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714014070 con 0x7f872410e0e0 2024-04-16T17:21:52.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.536+0000 7f871aa00640 1 -- 172.21.15.18:0/2508296255 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8714010af0 con 0x7f872410e0e0 2024-04-16T17:21:52.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.536+0000 7f8723200640 1 -- 172.21.15.18:0/2508296255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87241ac810 con 0x7f872410e0e0 2024-04-16T17:21:52.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.536+0000 7f8723200640 1 -- 172.21.15.18:0/2508296255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87241b40a0 con 0x7f872410e0e0 2024-04-16T17:21:52.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.536+0000 7f871aa00640 1 -- 172.21.15.18:0/2508296255 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714017d10 con 0x7f872410e0e0 2024-04-16T17:21:52.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.541+0000 7f872b966740 1 Processor -- start 2024-04-16T17:21:52.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.542+0000 7f871aa00640 1 -- 172.21.15.18:0/2508296255 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8714017070 con 0x7f872410e0e0 2024-04-16T17:21:52.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.543+0000 7f872b966740 1 -- start start 2024-04-16T17:21:52.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.544+0000 7f872b966740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491cf50 0x56225491d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.545+0000 7f872b966740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491d870 0x562254925f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.545+0000 7f8722800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491cf50 0x56225491d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.545+0000 7f8722800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491cf50 0x56225491d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45706/0 (socket says 172.21.15.18:45706) 2024-04-16T17:21:52.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.545+0000 7f8722800640 1 -- 172.21.15.18:0/1878722564 learned_addr learned my addr 172.21.15.18:0/1878722564 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:52.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.545+0000 7f872b966740 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562254926450 0x562254928840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.545+0000 7f872b966740 1 -- 172.21.15.18:0/1878722564 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562254882020 con 0x562254926450 2024-04-16T17:21:52.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.545+0000 7f872b966740 1 -- 172.21.15.18:0/1878722564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562254828460 con 0x56225491cf50 2024-04-16T17:21:52.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.545+0000 7f872b966740 1 -- 172.21.15.18:0/1878722564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56225487dfe0 con 0x56225491d870 2024-04-16T17:21:52.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.546+0000 7f8721e00640 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491d870 0x562254925f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.546+0000 7f871aa00640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87080a3e60 0x7f87080a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.546+0000 7f871aa00640 1 -- 172.21.15.18:0/2508296255 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f87140cfc70 con 0x7f872410e0e0 2024-04-16T17:21:52.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.547+0000 7f8722800640 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562254926450 0x562254928840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.547+0000 7f8722800640 1 -- 172.21.15.18:0/1878722564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491d870 msgr2=0x562254925f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:52.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.547+0000 7f8722800640 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491d870 0x562254925f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.547+0000 7f8722800640 1 -- 172.21.15.18:0/1878722564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562254926450 msgr2=0x562254928840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:52.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.547+0000 7f8722800640 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562254926450 0x562254928840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.547+0000 7f8722800640 1 -- 172.21.15.18:0/1878722564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562254830040 con 0x56225491cf50 2024-04-16T17:21:52.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.548+0000 7f8721400640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87080a3e60 0x7f87080a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.549+0000 7f8721400640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87080a3e60 0x7f87080a6320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f871c000f80 tx=0x7f871c005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:52.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.550+0000 7f8722800640 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491cf50 0x56225491d330 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8710009eb0 tx=0x7f8710009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8614f2277d442e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:52.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.550+0000 7f8721e00640 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491d870 0x562254925f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:52.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.550+0000 7f870f000640 1 -- 172.21.15.18:0/1878722564 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710005020 con 0x56225491cf50 2024-04-16T17:21:52.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.550+0000 7f870f000640 1 -- 172.21.15.18:0/1878722564 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8710016070 con 0x56225491cf50 2024-04-16T17:21:52.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.550+0000 7f870f000640 1 -- 172.21.15.18:0/1878722564 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100126b0 con 0x56225491cf50 2024-04-16T17:21:52.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.551+0000 7f872b966740 1 -- 172.21.15.18:0/1878722564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491cf50 msgr2=0x56225491d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.551+0000 7f872b966740 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491cf50 0x56225491d330 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8710009eb0 tx=0x7f8710009820 comp rx=0 tx=0).stop 2024-04-16T17:21:52.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.553+0000 7f872b966740 1 -- 172.21.15.18:0/1878722564 shutdown_connections 2024-04-16T17:21:52.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.553+0000 7f872b966740 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562254926450 0x562254928840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.553+0000 7f872b966740 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491d870 0x562254925f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.553+0000 7f872b966740 1 --2- 172.21.15.18:0/1878722564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491cf50 0x56225491d330 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.553+0000 7f872b966740 1 -- 172.21.15.18:0/1878722564 >> 172.21.15.18:0/1878722564 conn(0x562254917680 msgr2=0x562254917a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.556+0000 7f872b966740 1 -- 172.21.15.18:0/1878722564 shutdown_connections 2024-04-16T17:21:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.556+0000 7f872b966740 1 -- 172.21.15.18:0/1878722564 wait complete. 2024-04-16T17:21:52.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.557+0000 7f872b966740 1 -- start start 2024-04-16T17:21:52.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.558+0000 7f872b966740 1 Processor -- start 2024-04-16T17:21:52.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.559+0000 7f872b966740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 0x5622549233b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.559+0000 7f872b966740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x5622549238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.559+0000 7f872b966740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x562254923e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.559+0000 7f872b966740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562254929240 con 0x56225491d870 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.559+0000 7f872b966740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56225483b830 con 0x562254926450 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.559+0000 7f872b966740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5622549293c0 con 0x56225491cf50 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.559+0000 7f872b966740 10 client.0 _ll_register_callbacks cb 0x562254843150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.560+0000 7f8721400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x5622549238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.560+0000 7f8721400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x5622549238f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44352/0 (socket says 172.21.15.18:44352) 2024-04-16T17:21:52.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.560+0000 7f8721400640 1 -- 172.21.15.18:0/2044874951 learned_addr learned my addr 172.21.15.18:0/2044874951 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:52.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.560+0000 7f8722800640 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x562254923e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.560+0000 7f8721400640 1 -- 172.21.15.18:0/2044874951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 msgr2=0x5622549233b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:52.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.561+0000 7f8721400640 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 0x5622549233b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.562+0000 7f8721400640 1 -- 172.21.15.18:0/2044874951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 msgr2=0x562254923e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:52.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.562+0000 7f8721400640 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x562254923e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.562+0000 7f8721400640 1 -- 172.21.15.18:0/2044874951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56225483f920 con 0x56225491d870 2024-04-16T17:21:52.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.562+0000 7f8721400640 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x5622549238f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f871c036000 tx=0x7f871c02c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56d10c3411daa4d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:52.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.562+0000 7f8703400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:21:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.564+0000 7f8703400640 1 -- 172.21.15.18:0/2044874951 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c044070 con 0x56225491d870 2024-04-16T17:21:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.564+0000 7f8703400640 1 -- 172.21.15.18:0/2044874951 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f871c0349f0 con 0x56225491d870 2024-04-16T17:21:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.564+0000 7f872b966740 1 -- 172.21.15.18:0/2044874951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8710016070 con 0x56225491d870 2024-04-16T17:21:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.564+0000 7f8703400640 1 -- 172.21.15.18:0/2044874951 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f871c059050 con 0x56225491d870 2024-04-16T17:21:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.565+0000 7f872b966740 10 client.19413 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:52.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.565+0000 7f872b966740 1 -- 172.21.15.18:0/2044874951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56225480d7f0 con 0x56225491d870 2024-04-16T17:21:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.565+0000 7f8703400640 1 -- 172.21.15.18:0/2044874951 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8710016070 con 0x56225491d870 2024-04-16T17:21:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.566+0000 7f8703400640 1 -- 172.21.15.18:0/2044874951 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f871c003e70 con 0x56225491d870 2024-04-16T17:21:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.566+0000 7f872b966740 10 client.19413 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.566+0000 7f872b966740 10 client.19413 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:21:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.566+0000 7f872b966740 20 client.19413 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.566+0000 7f872b966740 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x562254936850 0x562254938c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.566+0000 7f872b966740 4 client.19413 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.566+0000 7f872b966740 1 -- 172.21.15.18:0/2044874951 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56225488fa40 con 0x562254936850 2024-04-16T17:21:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.566+0000 7f8721e00640 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x562254936850 0x562254938c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.567+0000 7f8721e00640 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x562254936850 0x562254938c90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8407c05b21ae7e47 server_cookie=7754bbce5a9ab7d7 in_seq=0 out_seq=0 2024-04-16T17:21:52.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.567+0000 7f8703400640 10 client.19413 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:21:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.568+0000 7f8703400640 1 -- 172.21.15.18:0/2044874951 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56225488fa40 con 0x562254936850 2024-04-16T17:21:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.568+0000 7f8703400640 10 client.19413 handle_command_reply: tid=0 2024-04-16T17:21:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.568+0000 7f872b966740 1 client.19413 shutdown 2024-04-16T17:21:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.569+0000 7f872b966740 1 -- 172.21.15.18:0/2044874951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 msgr2=0x5622549238f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.569+0000 7f872b966740 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x5622549238f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f871c036000 tx=0x7f871c02c100 comp rx=0 tx=0).stop 2024-04-16T17:21:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.570+0000 7f872b966740 1 -- 172.21.15.18:0/2044874951 shutdown_connections 2024-04-16T17:21:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.570+0000 7f872b966740 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x562254936850 0x562254938c90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.570+0000 7f8703400640 0 client.19413 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:21:52.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.570+0000 7f872b966740 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x562254923e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.570+0000 7f872b966740 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x5622549238f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.570+0000 7f872b966740 1 --2- 172.21.15.18:0/2044874951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 0x5622549233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.570+0000 7f872b966740 1 -- 172.21.15.18:0/2044874951 >> 172.21.15.18:0/2044874951 conn(0x562254917680 msgr2=0x5622549193d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:52.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.571+0000 7f872b966740 1 -- 172.21.15.18:0/2044874951 shutdown_connections 2024-04-16T17:21:52.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.571+0000 7f872b966740 1 -- 172.21.15.18:0/2044874951 wait complete. 2024-04-16T17:21:52.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.571+0000 7f872b966740 20 client.19413 trim_cache size 0 max 16384 2024-04-16T17:21:52.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.601+0000 7f872b966740 1 Processor -- start 2024-04-16T17:21:52.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.601+0000 7f872b966740 1 -- start start 2024-04-16T17:21:52.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.602+0000 7f872b966740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491cf50 0x562254932160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.602+0000 7f872b966740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491d870 0x562254923c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.602+0000 7f872b966740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562254926450 0x56225493e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.602+0000 7f872b966740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8704012950 con 0x56225491cf50 2024-04-16T17:21:52.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.602+0000 7f872b966740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8704010220 con 0x56225491d870 2024-04-16T17:21:52.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.602+0000 7f872b966740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f870400e090 con 0x562254926450 2024-04-16T17:21:52.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.603+0000 7f8721400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491d870 0x562254923c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.603+0000 7f8721400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491d870 0x562254923c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45726/0 (socket says 172.21.15.18:45726) 2024-04-16T17:21:52.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.603+0000 7f8721400640 1 -- 172.21.15.18:0/1428669023 learned_addr learned my addr 172.21.15.18:0/1428669023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:52.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.603+0000 7f8721400640 1 -- 172.21.15.18:0/1428669023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562254926450 msgr2=0x56225493e5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:52.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.603+0000 7f8721400640 1 --2- 172.21.15.18:0/1428669023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562254926450 0x56225493e5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.603+0000 7f8721400640 1 -- 172.21.15.18:0/1428669023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491cf50 msgr2=0x562254932160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:52.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.603+0000 7f8721400640 1 --2- 172.21.15.18:0/1428669023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491cf50 0x562254932160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.603+0000 7f8721400640 1 -- 172.21.15.18:0/1428669023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622545943f0 con 0x56225491d870 2024-04-16T17:21:52.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.604+0000 7f8721400640 1 --2- 172.21.15.18:0/1428669023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491d870 0x562254923c50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f871c02c710 tx=0x7f871c032fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbe591b649f056c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:52.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.605+0000 7f870e600640 1 -- 172.21.15.18:0/1428669023 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c044040 con 0x56225491d870 2024-04-16T17:21:52.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.606+0000 7f870e600640 1 -- 172.21.15.18:0/1428669023 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f871c050070 con 0x56225491d870 2024-04-16T17:21:52.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.606+0000 7f870e600640 1 -- 172.21.15.18:0/1428669023 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c039040 con 0x56225491d870 2024-04-16T17:21:52.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.607+0000 7f872b966740 1 -- 172.21.15.18:0/1428669023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491d870 msgr2=0x562254923c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.608+0000 7f872b966740 1 --2- 172.21.15.18:0/1428669023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491d870 0x562254923c50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f871c02c710 tx=0x7f871c032fc0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.620+0000 7f872b966740 1 -- 172.21.15.18:0/1428669023 shutdown_connections 2024-04-16T17:21:52.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.620+0000 7f872b966740 1 --2- 172.21.15.18:0/1428669023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562254926450 0x56225493e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.620+0000 7f872b966740 1 --2- 172.21.15.18:0/1428669023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56225491d870 0x562254923c50 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.620+0000 7f872b966740 1 --2- 172.21.15.18:0/1428669023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491cf50 0x562254932160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.621+0000 7f872b966740 1 -- 172.21.15.18:0/1428669023 >> 172.21.15.18:0/1428669023 conn(0x562254917680 msgr2=0x562254916700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:52.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.621+0000 7f872b966740 1 -- 172.21.15.18:0/1428669023 shutdown_connections 2024-04-16T17:21:52.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.622+0000 7f872b966740 1 -- 172.21.15.18:0/1428669023 wait complete. 2024-04-16T17:21:52.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.623+0000 7f872b966740 1 -- start start 2024-04-16T17:21:52.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.623+0000 7f872b966740 1 Processor -- start 2024-04-16T17:21:52.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f872b966740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 0x56225492a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f872b966740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x56225492a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f872b966740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x56225492aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f872b966740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f870400e6e0 con 0x56225491d870 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f872b966740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f870400fb50 con 0x562254926450 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f872b966740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f870400edb0 con 0x56225491cf50 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f872b966740 10 client.0 _ll_register_callbacks cb 0x7f8710005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f8722800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x56225492aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.624+0000 7f8722800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x56225492aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45732/0 (socket says 172.21.15.18:45732) 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.625+0000 7f8722800640 1 -- 172.21.15.18:0/2362014059 learned_addr learned my addr 172.21.15.18:0/2362014059 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:52.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.625+0000 7f8721e00640 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 0x56225492a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.625+0000 7f8722800640 1 -- 172.21.15.18:0/2362014059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 msgr2=0x56225492a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:52.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.625+0000 7f8722800640 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 0x56225492a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.625+0000 7f8722800640 1 -- 172.21.15.18:0/2362014059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 msgr2=0x56225492a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:52.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.625+0000 7f8722800640 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x56225492a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.625+0000 7f8722800640 1 -- 172.21.15.18:0/2362014059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56225465e7d0 con 0x562254926450 2024-04-16T17:21:52.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.625+0000 7f8722800640 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x56225492aad0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8710005940 tx=0x7f8710012b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f3054617f270ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:52.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.626+0000 7f8703400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:52.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.626+0000 7f8703400640 1 -- 172.21.15.18:0/2362014059 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710010dc0 con 0x562254926450 2024-04-16T17:21:52.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.626+0000 7f8703400640 1 -- 172.21.15.18:0/2362014059 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f8710012c70 con 0x562254926450 2024-04-16T17:21:52.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.627+0000 7f872b966740 1 -- 172.21.15.18:0/2362014059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f871c050070 con 0x562254926450 2024-04-16T17:21:52.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.628+0000 7f8703400640 1 -- 172.21.15.18:0/2362014059 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8710004030 con 0x562254926450 2024-04-16T17:21:52.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.628+0000 7f872b966740 10 client.30122 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:52.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.628+0000 7f872b966740 1 -- 172.21.15.18:0/2362014059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56225499cef0 con 0x562254926450 2024-04-16T17:21:52.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.629+0000 7f8703400640 1 -- 172.21.15.18:0/2362014059 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f871c050070 con 0x562254926450 2024-04-16T17:21:52.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.629+0000 7f8703400640 1 -- 172.21.15.18:0/2362014059 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f8710003a80 con 0x562254926450 2024-04-16T17:21:52.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.629+0000 7f872b966740 10 client.30122 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:52.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.646+0000 7f872b966740 10 client.30122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:21:52.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.646+0000 7f872b966740 20 client.30122 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:52.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.646+0000 7f872b966740 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56225499d3e0 0x56225499f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:52.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.646+0000 7f872b966740 4 client.30122 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:21:52.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.646+0000 7f872b966740 1 -- 172.21.15.18:0/2362014059 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622548898b0 con 0x56225499d3e0 2024-04-16T17:21:52.647 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:49 smithi018 ceph-mon[31776]: pgmap v544: 129 pgs: 129 active+clean; 413 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 458 op/s 2024-04-16T17:21:52.647 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:51 smithi018 ceph-mon[31776]: pgmap v545: 129 pgs: 129 active+clean; 399 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 538 op/s 2024-04-16T17:21:52.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.646+0000 7f8721e00640 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56225499d3e0 0x56225499f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:52.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.647+0000 7f8721e00640 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56225499d3e0 0x56225499f800 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34fe86f010a08b08 server_cookie=d19569488a13efed in_seq=0 out_seq=0 2024-04-16T17:21:52.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.647+0000 7f8703400640 10 client.30122 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:21:52.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.647+0000 7f8703400640 1 -- 172.21.15.18:0/2362014059 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622548898b0 con 0x56225499d3e0 2024-04-16T17:21:52.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.647+0000 7f8703400640 10 client.30122 handle_command_reply: tid=0 2024-04-16T17:21:52.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.648+0000 7f872b966740 1 client.30122 shutdown 2024-04-16T17:21:52.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.649+0000 7f872b966740 1 -- 172.21.15.18:0/2362014059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 msgr2=0x56225492aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.649+0000 7f872b966740 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x56225492aad0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8710005940 tx=0x7f8710012b80 comp rx=0 tx=0).stop 2024-04-16T17:21:52.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.649+0000 7f872b966740 1 -- 172.21.15.18:0/2362014059 shutdown_connections 2024-04-16T17:21:52.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.649+0000 7f872b966740 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56225499d3e0 0x56225499f800 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.650+0000 7f8703400640 0 client.30122 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:21:52.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.650+0000 7f872b966740 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562254926450 0x56225492aad0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.650+0000 7f872b966740 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56225491d870 0x56225492a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.650+0000 7f872b966740 1 --2- 172.21.15.18:0/2362014059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56225491cf50 0x56225492a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.650+0000 7f872b966740 1 -- 172.21.15.18:0/2362014059 >> 172.21.15.18:0/2362014059 conn(0x562254917680 msgr2=0x562254918a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:52.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.651+0000 7f872b966740 1 -- 172.21.15.18:0/2362014059 shutdown_connections 2024-04-16T17:21:52.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.651+0000 7f872b966740 1 -- 172.21.15.18:0/2362014059 wait complete. 2024-04-16T17:21:52.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.651+0000 7f872b966740 20 client.30122 trim_cache size 0 max 16384 2024-04-16T17:21:52.652 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:21:52.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 -- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87080a3e60 msgr2=0x7f87080a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87080a3e60 0x7f87080a6320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f871c000f80 tx=0x7f871c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 -- 172.21.15.18:0/2508296255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872410e0e0 msgr2=0x7f87241a7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872410e0e0 0x7f87241a7530 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f871400b4a0 tx=0x7f871400b960 comp rx=0 tx=0).stop 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 -- 172.21.15.18:0/2508296255 shutdown_connections 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f87080a3e60 0x7f87080a6320 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8724110580 0x7f87241abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f872410ec00 0x7f87241a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 --2- 172.21.15.18:0/2508296255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872410e0e0 0x7f87241a7530 secure :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f871400b4a0 tx=0x7f871400b960 comp rx=0 tx=0).stop 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.653+0000 7f8723200640 1 -- 172.21.15.18:0/2508296255 >> 172.21.15.18:0/2508296255 conn(0x7f872406cf10 msgr2=0x7f8724071390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.654+0000 7f8723200640 1 -- 172.21.15.18:0/2508296255 shutdown_connections 2024-04-16T17:21:52.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:52.654+0000 7f8723200640 1 -- 172.21.15.18:0/2508296255 wait complete. 2024-04-16T17:21:52.718 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:21:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.016+0000 7f1886e00640 1 -- 172.21.15.18:0/253064889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188010a590 msgr2=0x7f188010a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.016+0000 7f1886e00640 1 --2- 172.21.15.18:0/253064889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188010a590 0x7f188010a970 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1870009900 tx=0x7f187002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.017+0000 7f1886e00640 1 -- 172.21.15.18:0/253064889 shutdown_connections 2024-04-16T17:21:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.017+0000 7f1886e00640 1 --2- 172.21.15.18:0/253064889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18801090c0 0x7f1880115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.017+0000 7f1886e00640 1 --2- 172.21.15.18:0/253064889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1880108720 0x7f1880108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.017+0000 7f1886e00640 1 --2- 172.21.15.18:0/253064889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188010a590 0x7f188010a970 secure :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1870009900 tx=0x7f187002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.017+0000 7f1886e00640 1 -- 172.21.15.18:0/253064889 >> 172.21.15.18:0/253064889 conn(0x7f1880077e70 msgr2=0x7f1880106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.018+0000 7f1886e00640 1 -- 172.21.15.18:0/253064889 shutdown_connections 2024-04-16T17:21:53.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.018+0000 7f1886e00640 1 -- 172.21.15.18:0/253064889 wait complete. 2024-04-16T17:21:53.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.019+0000 7f1886e00640 1 Processor -- start 2024-04-16T17:21:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.020+0000 7f1886e00640 1 -- start start 2024-04-16T17:21:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.020+0000 7f1886e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880108720 0x7f1880073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.020+0000 7f1886e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18801090c0 0x7f188006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.020+0000 7f1886e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188006e0f0 0x7f188006e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.021+0000 7f1886e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f188006c430 con 0x7f18801090c0 2024-04-16T17:21:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.021+0000 7f1884e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188006e0f0 0x7f188006e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.021+0000 7f1886e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f188006c2b0 con 0x7f1880108720 2024-04-16T17:21:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.021+0000 7f1886e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f188006c5b0 con 0x7f188006e0f0 2024-04-16T17:21:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.021+0000 7f1884e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188006e0f0 0x7f188006e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52374/0 (socket says 172.21.15.18:52374) 2024-04-16T17:21:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.021+0000 7f1884e00640 1 -- 172.21.15.18:0/3125583837 learned_addr learned my addr 172.21.15.18:0/3125583837 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.021+0000 7f187f400640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18801090c0 0x7f188006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.022+0000 7f1884e00640 1 -- 172.21.15.18:0/3125583837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880108720 msgr2=0x7f1880073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.022+0000 7f1884e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880108720 0x7f1880073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.022+0000 7f1884e00640 1 -- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18801090c0 msgr2=0x7f188006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.022+0000 7f1884e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18801090c0 0x7f188006dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.022+0000 7f1884e00640 1 -- 172.21.15.18:0/3125583837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18801b4040 con 0x7f188006e0f0 2024-04-16T17:21:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.023+0000 7f1884e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188006e0f0 0x7f188006e5a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f187800ea60 tx=0x7f187800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.024+0000 7f187cc00640 1 -- 172.21.15.18:0/3125583837 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187800ce40 con 0x7f188006e0f0 2024-04-16T17:21:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.024+0000 7f187cc00640 1 -- 172.21.15.18:0/3125583837 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1878005020 con 0x7f188006e0f0 2024-04-16T17:21:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.024+0000 7f1886e00640 1 -- 172.21.15.18:0/3125583837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18801b4330 con 0x7f188006e0f0 2024-04-16T17:21:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.024+0000 7f187cc00640 1 -- 172.21.15.18:0/3125583837 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878010690 con 0x7f188006e0f0 2024-04-16T17:21:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.024+0000 7f1886e00640 1 -- 172.21.15.18:0/3125583837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18801b48d0 con 0x7f188006e0f0 2024-04-16T17:21:53.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.028+0000 7f1889a7c740 1 Processor -- start 2024-04-16T17:21:53.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.031+0000 7f187cc00640 1 -- 172.21.15.18:0/3125583837 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18780051d0 con 0x7f188006e0f0 2024-04-16T17:21:53.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.033+0000 7f1889a7c740 1 -- start start 2024-04-16T17:21:53.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8b92130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f187cc00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18640a3ea0 0x7f18640a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b92670 0x55efe8b9ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b9d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f187cc00640 1 -- 172.21.15.18:0/3125583837 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f18780d11a0 con 0x7f188006e0f0 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f1889a7c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55efe8af7020 con 0x55efe8b92670 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f1889a7c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55efe8a9d460 con 0x55efe8b91d50 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f1889a7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55efe8af2fe0 con 0x55efe8b9b250 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f187f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8b92130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f187f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8b92130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45756/0 (socket says 172.21.15.18:45756) 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.034+0000 7f187f400640 1 -- 172.21.15.18:0/3464613664 learned_addr learned my addr 172.21.15.18:0/3464613664 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.036+0000 7f1884e00640 1 --2- 172.21.15.18:0/3464613664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b9d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.036+0000 7f1884e00640 1 -- 172.21.15.18:0/3464613664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 msgr2=0x55efe8b92130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.036+0000 7f1884e00640 1 --2- 172.21.15.18:0/3464613664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8b92130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.036+0000 7f1884e00640 1 -- 172.21.15.18:0/3464613664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b92670 msgr2=0x55efe8b9ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.036+0000 7f1884e00640 1 --2- 172.21.15.18:0/3464613664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b92670 0x55efe8b9ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.036+0000 7f1884e00640 1 -- 172.21.15.18:0/3464613664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe8aa5040 con 0x55efe8b9b250 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.037+0000 7f1884e00640 1 --2- 172.21.15.18:0/3464613664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b9d640 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f18780ce480 tx=0x7f187809d490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4a78469d16f080 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.038+0000 7f1875600640 1 -- 172.21.15.18:0/3464613664 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18780b6070 con 0x55efe8b9b250 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.038+0000 7f1875600640 1 -- 172.21.15.18:0/3464613664 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18780b6210 con 0x55efe8b9b250 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.039+0000 7f1889a7c740 1 -- 172.21.15.18:0/3464613664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 msgr2=0x55efe8b9d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.039+0000 7f1889a7c740 1 --2- 172.21.15.18:0/3464613664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b9d640 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f18780ce480 tx=0x7f187809d490 comp rx=0 tx=0).stop 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.039+0000 7f1889a7c740 1 -- 172.21.15.18:0/3464613664 shutdown_connections 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.039+0000 7f1889a7c740 1 --2- 172.21.15.18:0/3464613664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b9d640 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.039+0000 7f1889a7c740 1 --2- 172.21.15.18:0/3464613664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b92670 0x55efe8b9ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.039+0000 7f1889a7c740 1 --2- 172.21.15.18:0/3464613664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8b92130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.039+0000 7f1889a7c740 1 -- 172.21.15.18:0/3464613664 >> 172.21.15.18:0/3464613664 conn(0x55efe8b8c480 msgr2=0x55efe8b8c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.040+0000 7f187fe00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18640a3ea0 0x7f18640a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.040+0000 7f1889a7c740 1 -- 172.21.15.18:0/3464613664 shutdown_connections 2024-04-16T17:21:53.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.040+0000 7f1889a7c740 1 -- 172.21.15.18:0/3464613664 wait complete. 2024-04-16T17:21:53.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.041+0000 7f1889a7c740 1 -- start start 2024-04-16T17:21:53.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.041+0000 7f1889a7c740 1 Processor -- start 2024-04-16T17:21:53.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.042+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8b98290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.042+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b92670 0x55efe8b987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.042+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b98d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.042+0000 7f1884e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b98d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.042+0000 7f1884e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b98d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52386/0 (socket says 172.21.15.18:52386) 2024-04-16T17:21:53.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.042+0000 7f1889a7c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55efe8b9e090 con 0x55efe8b92670 2024-04-16T17:21:53.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1889a7c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55efe8ab0830 con 0x55efe8b91d50 2024-04-16T17:21:53.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1889a7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55efe8b9e210 con 0x55efe8b9b250 2024-04-16T17:21:53.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1889a7c740 10 client.0 _ll_register_callbacks cb 0x55efe8ab8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:53.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1884e00640 1 -- 172.21.15.18:0/766273598 learned_addr learned my addr 172.21.15.18:0/766273598 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1884e00640 1 -- 172.21.15.18:0/766273598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 msgr2=0x55efe8b98290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:53.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1884e00640 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8b98290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1884e00640 1 -- 172.21.15.18:0/766273598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b92670 msgr2=0x55efe8b987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1884e00640 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b92670 0x55efe8b987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.043+0000 7f1884e00640 1 -- 172.21.15.18:0/766273598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe8ab4920 con 0x55efe8b9b250 2024-04-16T17:21:53.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.044+0000 7f1884e00640 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b98d10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f187809cc60 tx=0x7f1878005820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=682d9a4d7325f602 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.045+0000 7f187fe00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18640a3ea0 0x7f18640a6360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1870009230 tx=0x7f1870000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.045+0000 7f186a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:53.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.045+0000 7f186a000640 1 -- 172.21.15.18:0/766273598 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18780b6040 con 0x55efe8b9b250 2024-04-16T17:21:53.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.045+0000 7f186a000640 1 -- 172.21.15.18:0/766273598 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f18780cebd0 con 0x55efe8b9b250 2024-04-16T17:21:53.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.045+0000 7f1889a7c740 1 -- 172.21.15.18:0/766273598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18780b6210 con 0x55efe8b9b250 2024-04-16T17:21:53.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.045+0000 7f186a000640 1 -- 172.21.15.18:0/766273598 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f18780cedd0 con 0x55efe8b9b250 2024-04-16T17:21:53.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.045+0000 7f1889a7c740 10 client.29941 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:53.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.045+0000 7f1889a7c740 1 -- 172.21.15.18:0/766273598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55efe8a827f0 con 0x55efe8b9b250 2024-04-16T17:21:53.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.046+0000 7f186a000640 1 -- 172.21.15.18:0/766273598 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18780bd030 con 0x55efe8b9b250 2024-04-16T17:21:53.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.046+0000 7f186a000640 1 -- 172.21.15.18:0/766273598 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f1878098a20 con 0x55efe8b9b250 2024-04-16T17:21:53.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.046+0000 7f1889a7c740 10 client.29941 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:53.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.046+0000 7f1889a7c740 10 client.29941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:21:53.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.046+0000 7f1889a7c740 20 client.29941 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:53.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.046+0000 7f1889a7c740 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55efe8bab520 0x55efe8bad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.047+0000 7f1889a7c740 4 client.29941 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:53.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.047+0000 7f1889a7c740 1 -- 172.21.15.18:0/766273598 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18780b6210 con 0x55efe8bab520 2024-04-16T17:21:53.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.048+0000 7f187fe00640 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55efe8bab520 0x55efe8bad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.051+0000 7f186a000640 10 client.29941 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:21:53.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.053+0000 7f187fe00640 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55efe8bab520 0x55efe8bad910 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d1fb881e697a42b server_cookie=ccd9f0c83d39546d in_seq=0 out_seq=0 2024-04-16T17:21:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.056+0000 7f186a000640 1 -- 172.21.15.18:0/766273598 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f18780b6210 con 0x55efe8bab520 2024-04-16T17:21:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.056+0000 7f186a000640 10 client.29941 handle_command_reply: tid=0 2024-04-16T17:21:53.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.056+0000 7f1889a7c740 1 client.29941 shutdown 2024-04-16T17:21:53.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f1889a7c740 1 -- 172.21.15.18:0/766273598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 msgr2=0x55efe8b98d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f1889a7c740 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b98d10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f187809cc60 tx=0x7f1878005820 comp rx=0 tx=0).stop 2024-04-16T17:21:53.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f1889a7c740 1 -- 172.21.15.18:0/766273598 shutdown_connections 2024-04-16T17:21:53.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f1889a7c740 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55efe8bab520 0x55efe8bad910 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f1889a7c740 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b9b250 0x55efe8b98d10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f1889a7c740 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b92670 0x55efe8b987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f1889a7c740 1 --2- 172.21.15.18:0/766273598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8b98290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f186a000640 0 client.29941 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:21:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.057+0000 7f1889a7c740 1 -- 172.21.15.18:0/766273598 >> 172.21.15.18:0/766273598 conn(0x55efe8b8c480 msgr2=0x55efe8b8e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.058+0000 7f1889a7c740 1 -- 172.21.15.18:0/766273598 shutdown_connections 2024-04-16T17:21:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.058+0000 7f1889a7c740 1 -- 172.21.15.18:0/766273598 wait complete. 2024-04-16T17:21:53.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.058+0000 7f1889a7c740 20 client.29941 trim_cache size 0 max 16384 2024-04-16T17:21:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.090+0000 7f1889a7c740 1 Processor -- start 2024-04-16T17:21:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.090+0000 7f1889a7c740 1 -- start start 2024-04-16T17:21:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.091+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b91d50 0x55efe8b9a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.091+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 0x55efe8b954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.091+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b9b250 0x55efe8b95a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.091+0000 7f1889a7c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18801c1510 con 0x55efe8b91d50 2024-04-16T17:21:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.091+0000 7f1889a7c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18801bede0 con 0x55efe8b9b250 2024-04-16T17:21:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.091+0000 7f1889a7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18801bcc50 con 0x55efe8b92670 2024-04-16T17:21:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.092+0000 7f187fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b91d50 0x55efe8b9a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.092+0000 7f187fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b91d50 0x55efe8b9a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44432/0 (socket says 172.21.15.18:44432) 2024-04-16T17:21:53.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.092+0000 7f187fe00640 1 -- 172.21.15.18:0/800378835 learned_addr learned my addr 172.21.15.18:0/800378835 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.093+0000 7f187fe00640 1 -- 172.21.15.18:0/800378835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 msgr2=0x55efe8b954f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.093+0000 7f187fe00640 1 --2- 172.21.15.18:0/800378835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 0x55efe8b954f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.093+0000 7f187fe00640 1 -- 172.21.15.18:0/800378835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b9b250 msgr2=0x55efe8b95a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.093+0000 7f187fe00640 1 --2- 172.21.15.18:0/800378835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b9b250 0x55efe8b95a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.093+0000 7f187fe00640 1 -- 172.21.15.18:0/800378835 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe88093f0 con 0x55efe8b91d50 2024-04-16T17:21:53.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.094+0000 7f187fe00640 1 --2- 172.21.15.18:0/800378835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b91d50 0x55efe8b9a9c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f188007a770 tx=0x7f187003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96d706dfa2e8b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.100+0000 7f1874c00640 1 -- 172.21.15.18:0/800378835 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870051030 con 0x55efe8b91d50 2024-04-16T17:21:53.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.100+0000 7f1874c00640 1 -- 172.21.15.18:0/800378835 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18780b6210 con 0x55efe8b91d50 2024-04-16T17:21:53.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.100+0000 7f1874c00640 1 -- 172.21.15.18:0/800378835 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870045040 con 0x55efe8b91d50 2024-04-16T17:21:53.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.102+0000 7f1889a7c740 1 -- 172.21.15.18:0/800378835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b91d50 msgr2=0x55efe8b9a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.102+0000 7f1889a7c740 1 --2- 172.21.15.18:0/800378835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b91d50 0x55efe8b9a9c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f188007a770 tx=0x7f187003f000 comp rx=0 tx=0).stop 2024-04-16T17:21:53.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.106+0000 7f1889a7c740 1 -- 172.21.15.18:0/800378835 shutdown_connections 2024-04-16T17:21:53.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.106+0000 7f1889a7c740 1 --2- 172.21.15.18:0/800378835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b9b250 0x55efe8b95a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.106+0000 7f1889a7c740 1 --2- 172.21.15.18:0/800378835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 0x55efe8b954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.106+0000 7f1889a7c740 1 --2- 172.21.15.18:0/800378835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b91d50 0x55efe8b9a9c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.106+0000 7f1889a7c740 1 -- 172.21.15.18:0/800378835 >> 172.21.15.18:0/800378835 conn(0x55efe8b8c480 msgr2=0x55efe8b93570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.107+0000 7f1889a7c740 1 -- 172.21.15.18:0/800378835 shutdown_connections 2024-04-16T17:21:53.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.107+0000 7f1889a7c740 1 -- 172.21.15.18:0/800378835 wait complete. 2024-04-16T17:21:53.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.108+0000 7f1889a7c740 1 -- start start 2024-04-16T17:21:53.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.109+0000 7f1889a7c740 1 Processor -- start 2024-04-16T17:21:53.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8bb0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 0x55efe8c09c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f1889a7c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b9b250 0x55efe8c0a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f1889a7c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18801bd2a0 con 0x55efe8b9b250 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f1889a7c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18801be710 con 0x55efe8b91d50 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f1889a7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18801bd970 con 0x55efe8b92670 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f187fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8bb0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f187fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8bb0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45774/0 (socket says 172.21.15.18:45774) 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f1889a7c740 10 client.0 _ll_register_callbacks cb 0x7f18780b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.111+0000 7f187fe00640 1 -- 172.21.15.18:0/2596357308 learned_addr learned my addr 172.21.15.18:0/2596357308 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.112+0000 7f187f400640 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 0x55efe8c09c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.112+0000 7f187fe00640 1 -- 172.21.15.18:0/2596357308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 msgr2=0x55efe8c09c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.112+0000 7f187fe00640 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 0x55efe8c09c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.112+0000 7f187fe00640 1 -- 172.21.15.18:0/2596357308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b9b250 msgr2=0x55efe8c0a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.112+0000 7f187fe00640 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b9b250 0x55efe8c0a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.112+0000 7f187fe00640 1 -- 172.21.15.18:0/2596357308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe88d37d0 con 0x55efe8b91d50 2024-04-16T17:21:53.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.112+0000 7f187f400640 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 0x55efe8c09c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:21:53.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.113+0000 7f187fe00640 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8bb0bb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1870005c40 tx=0x7f1870004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65259a657b0e5d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.113+0000 7f186a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:53.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.114+0000 7f186a000640 1 -- 172.21.15.18:0/2596357308 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187005c070 con 0x55efe8b91d50 2024-04-16T17:21:53.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.114+0000 7f186a000640 1 -- 172.21.15.18:0/2596357308 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f187002c910 con 0x55efe8b91d50 2024-04-16T17:21:53.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.114+0000 7f1889a7c740 1 -- 172.21.15.18:0/2596357308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18780b6210 con 0x55efe8b91d50 2024-04-16T17:21:53.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.115+0000 7f186a000640 1 -- 172.21.15.18:0/2596357308 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1870066050 con 0x55efe8b91d50 2024-04-16T17:21:53.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.115+0000 7f1889a7c740 10 client.30131 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:53.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.115+0000 7f1889a7c740 1 -- 172.21.15.18:0/2596357308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55efe8bb05d0 con 0x55efe8b91d50 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.115+0000 7f186a000640 1 -- 172.21.15.18:0/2596357308 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18780b6210 con 0x55efe8b91d50 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.116+0000 7f186a000640 1 -- 172.21.15.18:0/2596357308 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f187004d040 con 0x55efe8b91d50 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.116+0000 7f1889a7c740 10 client.30131 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.116+0000 7f1889a7c740 10 client.30131 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.116+0000 7f1889a7c740 20 client.30131 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.116+0000 7f1889a7c740 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55efe8c22b70 0x55efe8c24f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.116+0000 7f1889a7c740 4 client.30131 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.116+0000 7f1889a7c740 1 -- 172.21.15.18:0/2596357308 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efe8b04a40 con 0x55efe8c22b70 2024-04-16T17:21:53.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.116+0000 7f187f400640 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55efe8c22b70 0x55efe8c24f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.121+0000 7f187f400640 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55efe8c22b70 0x55efe8c24f90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=801bc4e63dcaeb6f server_cookie=efdb428f08743281 in_seq=0 out_seq=0 2024-04-16T17:21:53.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.122+0000 7f186a000640 10 client.30131 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:21:53.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.127+0000 7f186a000640 1 -- 172.21.15.18:0/2596357308 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efe8b04a40 con 0x55efe8c22b70 2024-04-16T17:21:53.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.127+0000 7f186a000640 10 client.30131 handle_command_reply: tid=0 2024-04-16T17:21:53.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.127+0000 7f1889a7c740 1 client.30131 shutdown 2024-04-16T17:21:53.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.127+0000 7f1889a7c740 1 -- 172.21.15.18:0/2596357308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 msgr2=0x55efe8bb0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.127+0000 7f1889a7c740 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8bb0bb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1870005c40 tx=0x7f1870004320 comp rx=0 tx=0).stop 2024-04-16T17:21:53.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 1 -- 172.21.15.18:0/2596357308 shutdown_connections 2024-04-16T17:21:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55efe8c22b70 0x55efe8c24f90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55efe8b9b250 0x55efe8c0a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55efe8b92670 0x55efe8c09c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 1 --2- 172.21.15.18:0/2596357308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efe8b91d50 0x55efe8bb0bb0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 1 -- 172.21.15.18:0/2596357308 >> 172.21.15.18:0/2596357308 conn(0x55efe8b8c480 msgr2=0x55efe8ba64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f186a000640 0 client.30131 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:21:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 1 -- 172.21.15.18:0/2596357308 shutdown_connections 2024-04-16T17:21:53.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 1 -- 172.21.15.18:0/2596357308 wait complete. 2024-04-16T17:21:53.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.128+0000 7f1889a7c740 20 client.30131 trim_cache size 0 max 16384 2024-04-16T17:21:53.129 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:21:53.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.129+0000 7f1886e00640 1 -- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18640a3ea0 msgr2=0x7f18640a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.129+0000 7f1886e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18640a3ea0 0x7f18640a6360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1870009230 tx=0x7f1870000bc0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.129+0000 7f1886e00640 1 -- 172.21.15.18:0/3125583837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188006e0f0 msgr2=0x7f188006e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.129+0000 7f1886e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188006e0f0 0x7f188006e5a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f187800ea60 tx=0x7f187800ef20 comp rx=0 tx=0).stop 2024-04-16T17:21:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.130+0000 7f1886e00640 1 -- 172.21.15.18:0/3125583837 shutdown_connections 2024-04-16T17:21:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.130+0000 7f1886e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18640a3ea0 0x7f18640a6360 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.130+0000 7f1886e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188006e0f0 0x7f188006e5a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.130+0000 7f1886e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18801090c0 0x7f188006dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.130+0000 7f1886e00640 1 --2- 172.21.15.18:0/3125583837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880108720 0x7f1880073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.130+0000 7f1886e00640 1 -- 172.21.15.18:0/3125583837 >> 172.21.15.18:0/3125583837 conn(0x7f1880077e70 msgr2=0x7f1880078530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.130+0000 7f1886e00640 1 -- 172.21.15.18:0/3125583837 shutdown_connections 2024-04-16T17:21:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.130+0000 7f1886e00640 1 -- 172.21.15.18:0/3125583837 wait complete. 2024-04-16T17:21:53.167 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:21:53.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:52 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/558804192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:21:53.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:52 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/558804192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:21:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.460+0000 7fa0eb400640 1 -- 172.21.15.18:0/667775709 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4032990 con 0x7fa0ec112cb0 2024-04-16T17:21:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.461+0000 7fa0f2400640 1 -- 172.21.15.18:0/667775709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0ec112cb0 msgr2=0x7fa0ec1150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.461+0000 7fa0f2400640 1 --2- 172.21.15.18:0/667775709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0ec112cb0 0x7fa0ec1150a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009cd0 tx=0x7fa0e402c100 comp rx=0 tx=0).stop 2024-04-16T17:21:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.461+0000 7fa0f2400640 1 -- 172.21.15.18:0/667775709 shutdown_connections 2024-04-16T17:21:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.461+0000 7fa0f2400640 1 --2- 172.21.15.18:0/667775709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0ec112cb0 0x7fa0ec1150a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.461+0000 7fa0f2400640 1 --2- 172.21.15.18:0/667775709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ec076e50 0x7fa0ec112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.462+0000 7fa0f2400640 1 --2- 172.21.15.18:0/667775709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0ec076530 0x7fa0ec076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.462+0000 7fa0f2400640 1 -- 172.21.15.18:0/667775709 >> 172.21.15.18:0/667775709 conn(0x7fa0ec0ffae0 msgr2=0x7fa0ec101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.462+0000 7fa0f2400640 1 -- 172.21.15.18:0/667775709 shutdown_connections 2024-04-16T17:21:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.462+0000 7fa0f2400640 1 -- 172.21.15.18:0/667775709 wait complete. 2024-04-16T17:21:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.465+0000 7fa0f2400640 1 Processor -- start 2024-04-16T17:21:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.465+0000 7fa0f2400640 1 -- start start 2024-04-16T17:21:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0ec076530 0x7fa0ec19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ec076e50 0x7fa0ec19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0f2400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0ec112cb0 0x7fa0ec1a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0f2400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec118940 con 0x7fa0ec112cb0 2024-04-16T17:21:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0f2400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1187c0 con 0x7fa0ec076530 2024-04-16T17:21:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0f2400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec118ac0 con 0x7fa0ec076e50 2024-04-16T17:21:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ec076e50 0x7fa0ec19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ec076e50 0x7fa0ec19f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52426/0 (socket says 172.21.15.18:52426) 2024-04-16T17:21:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.466+0000 7fa0ebe00640 1 -- 172.21.15.18:0/422857853 learned_addr learned my addr 172.21.15.18:0/422857853 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.468+0000 7fa0ebe00640 1 -- 172.21.15.18:0/422857853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0ec076530 msgr2=0x7fa0ec19eea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:21:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.468+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0ec076530 0x7fa0ec19eea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.468+0000 7fa0ebe00640 1 -- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0ec112cb0 msgr2=0x7fa0ec1a3760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:21:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.468+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0ec112cb0 0x7fa0ec1a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.468+0000 7fa0ebe00640 1 -- 172.21.15.18:0/422857853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec1a3e60 con 0x7fa0ec076e50 2024-04-16T17:21:53.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.468+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ec076e50 0x7fa0ec19f3e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000b480 tx=0x7fa0e000b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.469+0000 7fa0e9600640 1 -- 172.21.15.18:0/422857853 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0014070 con 0x7fa0ec076e50 2024-04-16T17:21:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.469+0000 7fa0e9600640 1 -- 172.21.15.18:0/422857853 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e0010d50 con 0x7fa0ec076e50 2024-04-16T17:21:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.469+0000 7fa0e9600640 1 -- 172.21.15.18:0/422857853 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0017360 con 0x7fa0ec076e50 2024-04-16T17:21:53.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.470+0000 7fa0f2400640 1 -- 172.21.15.18:0/422857853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec1a4150 con 0x7fa0ec076e50 2024-04-16T17:21:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.473+0000 7fa0f2400640 1 -- 172.21.15.18:0/422857853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec06ba20 con 0x7fa0ec076e50 2024-04-16T17:21:53.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.477+0000 7fa0e9600640 1 -- 172.21.15.18:0/422857853 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0e0017070 con 0x7fa0ec076e50 2024-04-16T17:21:53.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.479+0000 7fa0f6608740 1 Processor -- start 2024-04-16T17:21:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.479+0000 7fa0f6608740 1 -- start start 2024-04-16T17:21:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.481+0000 7fa0e9600640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0d40a3e70 0x7fa0d40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.481+0000 7fa0e9600640 1 -- 172.21.15.18:0/422857853 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fa0e00cf340 con 0x7fa0ec076e50 2024-04-16T17:21:53.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b3302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b338ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0f6608740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558e4b295020 con 0x558e4b339410 2024-04-16T17:21:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0f6608740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558e4b23b460 con 0x558e4b330830 2024-04-16T17:21:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0f6608740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558e4b290fe0 con 0x558e4b32ff10 2024-04-16T17:21:53.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b338ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b338ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45796/0 (socket says 172.21.15.18:45796) 2024-04-16T17:21:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.482+0000 7fa0ebe00640 1 -- 172.21.15.18:0/2001588537 learned_addr learned my addr 172.21.15.18:0/2001588537 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.483+0000 7fa0f1000640 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b3302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.483+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.483+0000 7fa0ebe00640 1 -- 172.21.15.18:0/2001588537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 msgr2=0x558e4b3302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.483+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b3302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.483+0000 7fa0ebe00640 1 -- 172.21.15.18:0/2001588537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 msgr2=0x558e4b33b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.483+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.483+0000 7fa0ebe00640 1 -- 172.21.15.18:0/2001588537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e4b243040 con 0x558e4b330830 2024-04-16T17:21:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.483+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:21:53.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.484+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0d40a3e70 0x7fa0d40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.484+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b338ed0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa0e009b300 tx=0x7fa0e009b250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff38473212b74407 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.484+0000 7fa0de000640 1 -- 172.21.15.18:0/2001588537 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e00a3f50 con 0x558e4b330830 2024-04-16T17:21:53.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.485+0000 7fa0de000640 1 -- 172.21.15.18:0/2001588537 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e00b4070 con 0x558e4b330830 2024-04-16T17:21:53.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.486+0000 7fa0f6608740 1 -- 172.21.15.18:0/2001588537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 msgr2=0x558e4b338ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.489+0000 7fa0de000640 1 -- 172.21.15.18:0/2001588537 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e00979e0 con 0x558e4b330830 2024-04-16T17:21:53.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.490+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b338ed0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa0e009b300 tx=0x7fa0e009b250 comp rx=0 tx=0).stop 2024-04-16T17:21:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.490+0000 7fa0f6608740 1 -- 172.21.15.18:0/2001588537 shutdown_connections 2024-04-16T17:21:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.490+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.490+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b338ed0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.490+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2001588537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b3302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.490+0000 7fa0f6608740 1 -- 172.21.15.18:0/2001588537 >> 172.21.15.18:0/2001588537 conn(0x558e4b32a640 msgr2=0x558e4b32aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.491+0000 7fa0f6608740 1 -- 172.21.15.18:0/2001588537 shutdown_connections 2024-04-16T17:21:53.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.491+0000 7fa0f6608740 1 -- 172.21.15.18:0/2001588537 wait complete. 2024-04-16T17:21:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.491+0000 7fa0f6608740 1 -- start start 2024-04-16T17:21:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.492+0000 7fa0f6608740 1 Processor -- start 2024-04-16T17:21:53.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.493+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b32ff10 0x558e4b336360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.493+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b3368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b339410 0x558e4b336de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0f6608740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558e4b33c1f0 con 0x558e4b32ff10 2024-04-16T17:21:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0f6608740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558e4b24e830 con 0x558e4b330830 2024-04-16T17:21:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0f6608740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558e4b33c370 con 0x558e4b339410 2024-04-16T17:21:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0f6608740 10 client.0 _ll_register_callbacks cb 0x558e4b256150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:53.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b3368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b3368a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45810/0 (socket says 172.21.15.18:45810) 2024-04-16T17:21:53.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0ebe00640 1 -- 172.21.15.18:0/2389146277 learned_addr learned my addr 172.21.15.18:0/2389146277 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.494+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b339410 0x558e4b336de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.495+0000 7fa0f1000640 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b32ff10 0x558e4b336360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.496+0000 7fa0f1a00640 1 -- 172.21.15.18:0/2389146277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 msgr2=0x558e4b3368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.496+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b3368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.496+0000 7fa0f1a00640 1 -- 172.21.15.18:0/2389146277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b32ff10 msgr2=0x558e4b336360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.496+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b32ff10 0x558e4b336360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.496+0000 7fa0f1a00640 1 -- 172.21.15.18:0/2389146277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e4b252920 con 0x558e4b339410 2024-04-16T17:21:53.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.496+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0d40a3e70 0x7fa0d40a6330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa0e402c710 tx=0x7fa0e402e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.496+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b339410 0x558e4b336de0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005cc0 tx=0x7fa0e40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4c885475e36a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.496+0000 7fa0d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:53.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.497+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2389146277 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e404e030 con 0x558e4b339410 2024-04-16T17:21:53.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.497+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2389146277 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fa0e4034910 con 0x558e4b339410 2024-04-16T17:21:53.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.498+0000 7fa0f6608740 1 -- 172.21.15.18:0/2389146277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e00b4070 con 0x558e4b339410 2024-04-16T17:21:53.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.498+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2389146277 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa0e4039040 con 0x558e4b339410 2024-04-16T17:21:53.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.498+0000 7fa0f6608740 10 client.29953 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:53.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.498+0000 7fa0f6608740 1 -- 172.21.15.18:0/2389146277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558e4b2207f0 con 0x558e4b339410 2024-04-16T17:21:53.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.499+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2389146277 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e00b4070 con 0x558e4b339410 2024-04-16T17:21:53.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.499+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2389146277 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fa0e405dca0 con 0x558e4b339410 2024-04-16T17:21:53.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.500+0000 7fa0f6608740 10 client.29953 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:53.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.500+0000 7fa0f6608740 10 client.29953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:53.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.500+0000 7fa0f6608740 20 client.29953 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:53.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.500+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558e4b349580 0x558e4b34b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.500+0000 7fa0f6608740 4 client.29953 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:53.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.500+0000 7fa0f6608740 1 -- 172.21.15.18:0/2389146277 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e4b2a2a40 con 0x558e4b349580 2024-04-16T17:21:53.503 INFO:tasks.workunit.client.0.smithi018.stdout: 10 61697 12.76 MB/sec execute 125 sec latency 110.521 ms 2024-04-16T17:21:53.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.504+0000 7fa0f1000640 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558e4b349580 0x558e4b34b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.506+0000 7fa0f1000640 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558e4b349580 0x558e4b34b970 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ca8fd114262f7a server_cookie=72f2742e4b8b1f18 in_seq=0 out_seq=0 2024-04-16T17:21:53.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.506+0000 7fa0d2a00640 10 client.29953 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:53.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.512+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2389146277 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e4b2a2a40 con 0x558e4b349580 2024-04-16T17:21:53.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.512+0000 7fa0d2a00640 10 client.29953 handle_command_reply: tid=0 2024-04-16T17:21:53.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.512+0000 7fa0f6608740 1 client.29953 shutdown 2024-04-16T17:21:53.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0f6608740 1 -- 172.21.15.18:0/2389146277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b339410 msgr2=0x558e4b336de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b339410 0x558e4b336de0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005cc0 tx=0x7fa0e40095c0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0f6608740 1 -- 172.21.15.18:0/2389146277 shutdown_connections 2024-04-16T17:21:53.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558e4b349580 0x558e4b34b970 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b339410 0x558e4b336de0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b3368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2389146277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b32ff10 0x558e4b336360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0f6608740 1 -- 172.21.15.18:0/2389146277 >> 172.21.15.18:0/2389146277 conn(0x558e4b32a640 msgr2=0x558e4b32c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.514+0000 7fa0d2a00640 0 client.29953 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.515+0000 7fa0f6608740 1 -- 172.21.15.18:0/2389146277 shutdown_connections 2024-04-16T17:21:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.515+0000 7fa0f6608740 1 -- 172.21.15.18:0/2389146277 wait complete. 2024-04-16T17:21:53.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.515+0000 7fa0f6608740 20 client.29953 trim_cache size 0 max 16384 2024-04-16T17:21:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.560+0000 7fa0f6608740 1 Processor -- start 2024-04-16T17:21:53.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.561+0000 7fa0f6608740 1 -- start start 2024-04-16T17:21:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b32ff10 0x558e4b345100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b330830 0x558e4b336c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b329580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b32ff10 0x558e4b345100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b32ff10 0x558e4b345100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45824/0 (socket says 172.21.15.18:45824) 2024-04-16T17:21:53.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f1000640 1 -- 172.21.15.18:0/4063980648 learned_addr learned my addr 172.21.15.18:0/4063980648 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f6608740 1 -- 172.21.15.18:0/4063980648 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1c0dc0 con 0x558e4b339410 2024-04-16T17:21:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f6608740 1 -- 172.21.15.18:0/4063980648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1be690 con 0x558e4b32ff10 2024-04-16T17:21:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0f6608740 1 -- 172.21.15.18:0/4063980648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec073d10 con 0x558e4b330830 2024-04-16T17:21:53.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.562+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b330830 0x558e4b336c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.563+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b329580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.563+0000 7fa0ebe00640 1 -- 172.21.15.18:0/4063980648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b32ff10 msgr2=0x558e4b345100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.563+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b32ff10 0x558e4b345100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.563+0000 7fa0ebe00640 1 -- 172.21.15.18:0/4063980648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 msgr2=0x558e4b329580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.563+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b329580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.563+0000 7fa0ebe00640 1 -- 172.21.15.18:0/4063980648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e4afa73f0 con 0x558e4b330830 2024-04-16T17:21:53.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.564+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b330830 0x558e4b336c00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa0e009b070 tx=0x7fa0e009f8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8549e8f159cd332 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.564+0000 7fa0dd600640 1 -- 172.21.15.18:0/4063980648 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e00b8030 con 0x558e4b330830 2024-04-16T17:21:53.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.565+0000 7fa0dd600640 1 -- 172.21.15.18:0/4063980648 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e00b4040 con 0x558e4b330830 2024-04-16T17:21:53.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.565+0000 7fa0dd600640 1 -- 172.21.15.18:0/4063980648 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e009a7c0 con 0x558e4b330830 2024-04-16T17:21:53.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.566+0000 7fa0f6608740 1 -- 172.21.15.18:0/4063980648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b330830 msgr2=0x558e4b336c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.566+0000 7fa0f6608740 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b330830 0x558e4b336c00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa0e009b070 tx=0x7fa0e009f8a0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.567+0000 7fa0f6608740 1 -- 172.21.15.18:0/4063980648 shutdown_connections 2024-04-16T17:21:53.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.567+0000 7fa0f6608740 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b329580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.567+0000 7fa0f6608740 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b330830 0x558e4b336c00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.567+0000 7fa0f6608740 1 --2- 172.21.15.18:0/4063980648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b32ff10 0x558e4b345100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.567+0000 7fa0f6608740 1 -- 172.21.15.18:0/4063980648 >> 172.21.15.18:0/4063980648 conn(0x558e4b32a640 msgr2=0x558e4b328e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.567+0000 7fa0f6608740 1 -- 172.21.15.18:0/4063980648 shutdown_connections 2024-04-16T17:21:53.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.567+0000 7fa0f6608740 1 -- 172.21.15.18:0/4063980648 wait complete. 2024-04-16T17:21:53.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.568+0000 7fa0f6608740 1 -- start start 2024-04-16T17:21:53.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.569+0000 7fa0f6608740 1 Processor -- start 2024-04-16T17:21:53.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.570+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b341890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.571+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b33cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.571+0000 7fa0f6608740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.571+0000 7fa0f1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b341890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.571+0000 7fa0f1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b341890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52454/0 (socket says 172.21.15.18:52454) 2024-04-16T17:21:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.571+0000 7fa0ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b33cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.571+0000 7fa0ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b33cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45840/0 (socket says 172.21.15.18:45840) 2024-04-16T17:21:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.572+0000 7fa0f1000640 1 -- 172.21.15.18:0/2990989423 learned_addr learned my addr 172.21.15.18:0/2990989423 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.572+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1bcb70 con 0x558e4b339410 2024-04-16T17:21:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.572+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1bdfc0 con 0x558e4b330830 2024-04-16T17:21:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.572+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1bd220 con 0x558e4b32ff10 2024-04-16T17:21:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.572+0000 7fa0f6608740 10 client.0 _ll_register_callbacks cb 0x7fa0e4032de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.573+0000 7fa0ebe00640 1 -- 172.21.15.18:0/2990989423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 msgr2=0x558e4b341890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.573+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b341890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.573+0000 7fa0ebe00640 1 -- 172.21.15.18:0/2990989423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 msgr2=0x558e4b33d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.573+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.574+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.574+0000 7fa0ebe00640 1 -- 172.21.15.18:0/2990989423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e4b0717d0 con 0x558e4b330830 2024-04-16T17:21:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.574+0000 7fa0f1000640 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b341890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.574+0000 7fa0f1a00640 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.575+0000 7fa0ebe00640 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b33cc50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00ccb40 tx=0x7fa0e009b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd012818b9de37e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:53.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.575+0000 7fa0d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:21:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.576+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2990989423 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e00b8030 con 0x558e4b330830 2024-04-16T17:21:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.576+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2990989423 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fa0e00b9070 con 0x558e4b330830 2024-04-16T17:21:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.576+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e00b4040 con 0x558e4b330830 2024-04-16T17:21:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.576+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2990989423 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa0e00a3f50 con 0x558e4b330830 2024-04-16T17:21:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.576+0000 7fa0f6608740 10 client.30146 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.576+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558e4b3af760 con 0x558e4b330830 2024-04-16T17:21:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.577+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2990989423 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e00b4040 con 0x558e4b330830 2024-04-16T17:21:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.577+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2990989423 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fa0e009a2c0 con 0x558e4b330830 2024-04-16T17:21:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.577+0000 7fa0f6608740 10 client.30146 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.577+0000 7fa0f6608740 10 client.30146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.577+0000 7fa0f6608740 20 client.30146 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.578+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558e4b3af760 0x558e4b3b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.578+0000 7fa0f6608740 4 client.30146 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:21:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.578+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e4b331f60 con 0x558e4b3af760 2024-04-16T17:21:53.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.578+0000 7fa0f1000640 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558e4b3af760 0x558e4b3b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:53.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.579+0000 7fa0f1000640 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558e4b3af760 0x558e4b3b1b50 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3eb8597e773d99f server_cookie=9b7732846f50d2d0 in_seq=0 out_seq=0 2024-04-16T17:21:53.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.579+0000 7fa0d2a00640 10 client.30146 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:53.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.581+0000 7fa0d2a00640 1 -- 172.21.15.18:0/2990989423 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e4b331f60 con 0x558e4b3af760 2024-04-16T17:21:53.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.581+0000 7fa0d2a00640 10 client.30146 handle_command_reply: tid=0 2024-04-16T17:21:53.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.582+0000 7fa0f6608740 1 client.30146 shutdown 2024-04-16T17:21:53.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.582+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 msgr2=0x558e4b33cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.582+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b33cc50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00ccb40 tx=0x7fa0e009b4f0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 shutdown_connections 2024-04-16T17:21:53.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558e4b3af760 0x558e4b3b1b50 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558e4b339410 0x558e4b33d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e4b330830 0x558e4b33cc50 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 1 --2- 172.21.15.18:0/2990989423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e4b32ff10 0x558e4b341890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0d2a00640 0 client.30146 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:53.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 >> 172.21.15.18:0/2990989423 conn(0x558e4b32a640 msgr2=0x558e4b328e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 shutdown_connections 2024-04-16T17:21:53.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 1 -- 172.21.15.18:0/2990989423 wait complete. 2024-04-16T17:21:53.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.583+0000 7fa0f6608740 20 client.30146 trim_cache size 0 max 16384 2024-04-16T17:21:53.587 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:21:53.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.586+0000 7fa0f2400640 1 -- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0d40a3e70 msgr2=0x7fa0d40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.586+0000 7fa0f2400640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0d40a3e70 0x7fa0d40a6330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa0e402c710 tx=0x7fa0e402e040 comp rx=0 tx=0).stop 2024-04-16T17:21:53.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.586+0000 7fa0f2400640 1 -- 172.21.15.18:0/422857853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ec076e50 msgr2=0x7fa0ec19f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:53.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.586+0000 7fa0f2400640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ec076e50 0x7fa0ec19f3e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000b480 tx=0x7fa0e000b940 comp rx=0 tx=0).stop 2024-04-16T17:21:53.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.587+0000 7fa0f2400640 1 -- 172.21.15.18:0/422857853 shutdown_connections 2024-04-16T17:21:53.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.587+0000 7fa0f2400640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0d40a3e70 0x7fa0d40a6330 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.587+0000 7fa0f2400640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0ec112cb0 0x7fa0ec1a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.587+0000 7fa0f2400640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0ec076e50 0x7fa0ec19f3e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.587+0000 7fa0f2400640 1 --2- 172.21.15.18:0/422857853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0ec076530 0x7fa0ec19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:53.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.587+0000 7fa0f2400640 1 -- 172.21.15.18:0/422857853 >> 172.21.15.18:0/422857853 conn(0x7fa0ec0ffae0 msgr2=0x7fa0ec101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:53.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.588+0000 7fa0f2400640 1 -- 172.21.15.18:0/422857853 shutdown_connections 2024-04-16T17:21:53.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:53.588+0000 7fa0f2400640 1 -- 172.21.15.18:0/422857853 wait complete. 2024-04-16T17:21:54.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:53 smithi022 ceph-mon[36023]: pgmap v546: 129 pgs: 129 active+clean; 401 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 483 op/s 2024-04-16T17:21:54.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:53 smithi043 ceph-mon[36901]: pgmap v546: 129 pgs: 129 active+clean; 401 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 483 op/s 2024-04-16T17:21:54.504 INFO:tasks.workunit.client.0.smithi018.stdout: 10 62038 12.85 MB/sec execute 126 sec latency 118.444 ms 2024-04-16T17:21:54.631 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:21:54.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:21:54.632 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:21:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.912+0000 7efe02600640 1 -- 172.21.15.18:0/2447890265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104c30 msgr2=0x7efdfc108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.912+0000 7efe02600640 1 --2- 172.21.15.18:0/2447890265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104c30 0x7efdfc108c70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7efde8009900 tx=0x7efde802bc30 comp rx=0 tx=0).stop 2024-04-16T17:21:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.913+0000 7efe02600640 1 -- 172.21.15.18:0/2447890265 shutdown_connections 2024-04-16T17:21:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.913+0000 7efe02600640 1 --2- 172.21.15.18:0/2447890265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc109350 0x7efdfc110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.913+0000 7efe02600640 1 --2- 172.21.15.18:0/2447890265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104c30 0x7efdfc108c70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.913+0000 7efe02600640 1 --2- 172.21.15.18:0/2447890265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdfc104280 0x7efdfc104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.913+0000 7efe02600640 1 -- 172.21.15.18:0/2447890265 >> 172.21.15.18:0/2447890265 conn(0x7efdfc0ffae0 msgr2=0x7efdfc101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.913+0000 7efe02600640 1 -- 172.21.15.18:0/2447890265 shutdown_connections 2024-04-16T17:21:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.913+0000 7efe02600640 1 -- 172.21.15.18:0/2447890265 wait complete. 2024-04-16T17:21:54.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.915+0000 7efe02600640 1 Processor -- start 2024-04-16T17:21:54.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.915+0000 7efe02600640 1 -- start start 2024-04-16T17:21:54.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.915+0000 7efe02600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104280 0x7efdfc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe02600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc109350 0x7efdfc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe02600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdfc19a770 0x7efdfc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe02600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc114430 con 0x7efdfc19a770 2024-04-16T17:21:54.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe02600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1142b0 con 0x7efdfc104280 2024-04-16T17:21:54.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe02600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1145b0 con 0x7efdfc109350 2024-04-16T17:21:54.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104280 0x7efdfc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104280 0x7efdfc1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45852/0 (socket says 172.21.15.18:45852) 2024-04-16T17:21:54.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01200640 1 -- 172.21.15.18:0/1776735741 learned_addr learned my addr 172.21.15.18:0/1776735741 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:54.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efdfbe00640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc109350 0x7efdfc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01c00640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdfc19a770 0x7efdfc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01200640 1 -- 172.21.15.18:0/1776735741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc109350 msgr2=0x7efdfc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01200640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc109350 0x7efdfc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01200640 1 -- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdfc19a770 msgr2=0x7efdfc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01200640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdfc19a770 0x7efdfc19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.916+0000 7efe01200640 1 -- 172.21.15.18:0/1776735741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc19b370 con 0x7efdfc104280 2024-04-16T17:21:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.917+0000 7efe01200640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104280 0x7efdfc1a06b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c920 tx=0x7efdf000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.917+0000 7efdf9600640 1 -- 172.21.15.18:0/1776735741 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0007c70 con 0x7efdfc104280 2024-04-16T17:21:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.917+0000 7efdf9600640 1 -- 172.21.15.18:0/1776735741 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf0007e10 con 0x7efdfc104280 2024-04-16T17:21:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.917+0000 7efdf9600640 1 -- 172.21.15.18:0/1776735741 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0018890 con 0x7efdfc104280 2024-04-16T17:21:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.917+0000 7efe02600640 1 -- 172.21.15.18:0/1776735741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc1a7450 con 0x7efdfc104280 2024-04-16T17:21:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.918+0000 7efe02600640 1 -- 172.21.15.18:0/1776735741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc1a7800 con 0x7efdfc104280 2024-04-16T17:21:54.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.922+0000 7efdf9600640 1 -- 172.21.15.18:0/1776735741 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efdf0018a70 con 0x7efdfc104280 2024-04-16T17:21:54.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.923+0000 7efe06819740 1 Processor -- start 2024-04-16T17:21:54.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.924+0000 7efe06819740 1 -- start start 2024-04-16T17:21:54.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.924+0000 7efe06819740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cce90 0x563d937cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.924+0000 7efe06819740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cd7b0 0x563d937d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.924+0000 7efe06819740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937d6390 0x563d937d8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.924+0000 7efe06819740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563d93731cb0 con 0x563d937cd7b0 2024-04-16T17:21:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.924+0000 7efe06819740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563d936d8370 con 0x563d937cce90 2024-04-16T17:21:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.924+0000 7efe06819740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d9372dc70 con 0x563d937d6390 2024-04-16T17:21:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efe01c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cce90 0x563d937cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efe01c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cce90 0x563d937cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45858/0 (socket says 172.21.15.18:45858) 2024-04-16T17:21:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efe01c00640 1 -- 172.21.15.18:0/4285740126 learned_addr learned my addr 172.21.15.18:0/4285740126 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:54.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efdfbe00640 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937d6390 0x563d937d8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efdf9600640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efde40a3da0 0x7efde40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efdf9600640 1 -- 172.21.15.18:0/1776735741 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7efdf00d0d60 con 0x7efdfc104280 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efe01200640 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cd7b0 0x563d937d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efe01c00640 1 -- 172.21.15.18:0/4285740126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937d6390 msgr2=0x563d937d8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efe01c00640 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937d6390 0x563d937d8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.925+0000 7efe01c00640 1 -- 172.21.15.18:0/4285740126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cd7b0 msgr2=0x563d937d5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.926+0000 7efe01c00640 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cd7b0 0x563d937d5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.926+0000 7efe01c00640 1 -- 172.21.15.18:0/4285740126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d936f7910 con 0x563d937cce90 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.926+0000 7efe01200640 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cd7b0 0x563d937d5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.926+0000 7efe01c00640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efde40a3da0 0x7efde40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.927+0000 7efe01c00640 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cce90 0x563d937cd270 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009eb0 tx=0x7efdf400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b3c53fcfe937f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:54.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.927+0000 7efdee000640 1 -- 172.21.15.18:0/4285740126 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf401a410 con 0x563d937cce90 2024-04-16T17:21:54.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.927+0000 7efdee000640 1 -- 172.21.15.18:0/4285740126 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf401aa90 con 0x563d937cce90 2024-04-16T17:21:54.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.927+0000 7efdee000640 1 -- 172.21.15.18:0/4285740126 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4010980 con 0x563d937cce90 2024-04-16T17:21:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.929+0000 7efe06819740 1 -- 172.21.15.18:0/4285740126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cce90 msgr2=0x563d937cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.929+0000 7efe06819740 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cce90 0x563d937cd270 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009eb0 tx=0x7efdf400f730 comp rx=0 tx=0).stop 2024-04-16T17:21:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.929+0000 7efe01c00640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efde40a3da0 0x7efde40a6260 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdf400fc30 tx=0x7efdf4019000 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:54.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.930+0000 7efe06819740 1 -- 172.21.15.18:0/4285740126 shutdown_connections 2024-04-16T17:21:54.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.930+0000 7efe06819740 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937d6390 0x563d937d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.930+0000 7efe06819740 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cd7b0 0x563d937d5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.930+0000 7efe06819740 1 --2- 172.21.15.18:0/4285740126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cce90 0x563d937cd270 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.930+0000 7efe06819740 1 -- 172.21.15.18:0/4285740126 >> 172.21.15.18:0/4285740126 conn(0x563d937c75c0 msgr2=0x563d937c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:54.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.930+0000 7efe06819740 1 -- 172.21.15.18:0/4285740126 shutdown_connections 2024-04-16T17:21:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.930+0000 7efe06819740 1 -- 172.21.15.18:0/4285740126 wait complete. 2024-04-16T17:21:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.931+0000 7efe06819740 1 -- start start 2024-04-16T17:21:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.931+0000 7efe06819740 1 Processor -- start 2024-04-16T17:21:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efe06819740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d937d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efe06819740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efe06819740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 0x563d937d3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efe06819740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563d937d9180 con 0x563d937cce90 2024-04-16T17:21:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efe06819740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563d936eb730 con 0x563d937d6390 2024-04-16T17:21:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efe06819740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d937d9300 con 0x563d937cd7b0 2024-04-16T17:21:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efe06819740 10 client.0 _ll_register_callbacks cb 0x563d936ba1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:54.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937d38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52482/0 (socket says 172.21.15.18:52482) 2024-04-16T17:21:54.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.932+0000 7efdfbe00640 1 -- 172.21.15.18:0/3710832449 learned_addr learned my addr 172.21.15.18:0/3710832449 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efe01200640 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d937d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efdfbe00640 1 -- 172.21.15.18:0/3710832449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 msgr2=0x563d937d3df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efdfbe00640 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 0x563d937d3df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efdfbe00640 1 -- 172.21.15.18:0/3710832449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 msgr2=0x563d937d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efdfbe00640 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d937d3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efdfbe00640 1 -- 172.21.15.18:0/3710832449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9350c7e0 con 0x563d937cd7b0 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efe01200640 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d937d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efdfbe00640 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937d38b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efdfc069e30 tx=0x7efde802c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d282dffdd32e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.933+0000 7efde2a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.934+0000 7efde2a00640 1 -- 172.21.15.18:0/3710832449 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde802cc50 con 0x563d937cd7b0 2024-04-16T17:21:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.934+0000 7efde2a00640 1 -- 172.21.15.18:0/3710832449 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7efde80046c0 con 0x563d937cd7b0 2024-04-16T17:21:54.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.934+0000 7efe06819740 1 -- 172.21.15.18:0/3710832449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf401aa90 con 0x563d937cd7b0 2024-04-16T17:21:54.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.935+0000 7efde2a00640 1 -- 172.21.15.18:0/3710832449 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efde80048c0 con 0x563d937cd7b0 2024-04-16T17:21:54.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.935+0000 7efe06819740 10 client.29971 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:54.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.935+0000 7efe06819740 1 -- 172.21.15.18:0/3710832449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563d9356cf70 con 0x563d937cd7b0 2024-04-16T17:21:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.936+0000 7efde2a00640 1 -- 172.21.15.18:0/3710832449 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf401aa90 con 0x563d937cd7b0 2024-04-16T17:21:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.936+0000 7efde2a00640 1 -- 172.21.15.18:0/3710832449 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7efde80035f0 con 0x563d937cd7b0 2024-04-16T17:21:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.936+0000 7efe06819740 10 client.29971 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.936+0000 7efe06819740 10 client.29971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.936+0000 7efe06819740 20 client.29971 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.937+0000 7efe06819740 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d937eede0 0x563d937f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.937+0000 7efe06819740 4 client.29971 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.937+0000 7efe06819740 1 -- 172.21.15.18:0/3710832449 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d9373f6d0 con 0x563d937eede0 2024-04-16T17:21:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.937+0000 7efe01200640 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d937eede0 0x563d937f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.938+0000 7efe01200640 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d937eede0 0x563d937f11d0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13ec0fe78e91a21 server_cookie=f2adb181880e5eff in_seq=0 out_seq=0 2024-04-16T17:21:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.938+0000 7efde2a00640 10 client.29971 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:54.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.943+0000 7efde2a00640 1 -- 172.21.15.18:0/3710832449 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d9373f6d0 con 0x563d937eede0 2024-04-16T17:21:54.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.943+0000 7efde2a00640 10 client.29971 handle_command_reply: tid=0 2024-04-16T17:21:54.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.944+0000 7efe06819740 1 client.29971 shutdown 2024-04-16T17:21:54.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.944+0000 7efe06819740 1 -- 172.21.15.18:0/3710832449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 msgr2=0x563d937d38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.944+0000 7efe06819740 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937d38b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efdfc069e30 tx=0x7efde802c7e0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.944+0000 7efe06819740 1 -- 172.21.15.18:0/3710832449 shutdown_connections 2024-04-16T17:21:54.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.944+0000 7efe06819740 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d937eede0 0x563d937f11d0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.945+0000 7efe06819740 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 0x563d937d3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.945+0000 7efe06819740 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937d38b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.945+0000 7efe06819740 1 --2- 172.21.15.18:0/3710832449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d937d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.945+0000 7efe06819740 1 -- 172.21.15.18:0/3710832449 >> 172.21.15.18:0/3710832449 conn(0x563d937c75c0 msgr2=0x563d937c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:54.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.945+0000 7efde2a00640 0 client.29971 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:54.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.945+0000 7efe06819740 1 -- 172.21.15.18:0/3710832449 shutdown_connections 2024-04-16T17:21:54.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.945+0000 7efe06819740 1 -- 172.21.15.18:0/3710832449 wait complete. 2024-04-16T17:21:54.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.945+0000 7efe06819740 20 client.29971 trim_cache size 0 max 16384 2024-04-16T17:21:54.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.972+0000 7efe06819740 1 Processor -- start 2024-04-16T17:21:54.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.972+0000 7efe06819740 1 -- start start 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.972+0000 7efe06819740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cce90 0x563d937d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe06819740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cd7b0 0x563d937e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe06819740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937d6390 0x563d937f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe06819740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1afb60 con 0x563d937d6390 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe06819740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1ad430 con 0x563d937cd7b0 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe06819740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1ab2a0 con 0x563d937cce90 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cd7b0 0x563d937e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cd7b0 0x563d937e2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45876/0 (socket says 172.21.15.18:45876) 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe01200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cce90 0x563d937d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efdfbe00640 1 -- 172.21.15.18:0/4137708491 learned_addr learned my addr 172.21.15.18:0/4137708491 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:54.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe01c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937d6390 0x563d937f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe01200640 1 -- 172.21.15.18:0/4137708491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cd7b0 msgr2=0x563d937e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe01200640 1 --2- 172.21.15.18:0/4137708491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cd7b0 0x563d937e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe01200640 1 -- 172.21.15.18:0/4137708491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937d6390 msgr2=0x563d937f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe01200640 1 --2- 172.21.15.18:0/4137708491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937d6390 0x563d937f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.973+0000 7efe01200640 1 -- 172.21.15.18:0/4137708491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d93404170 con 0x563d937cce90 2024-04-16T17:21:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.974+0000 7efe01200640 1 --2- 172.21.15.18:0/4137708491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cce90 0x563d937d5aa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efdf00ce040 tx=0x7efdf00d0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc56bc9c13c042e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.974+0000 7efded600640 1 -- 172.21.15.18:0/4137708491 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00a5500 con 0x563d937cce90 2024-04-16T17:21:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.974+0000 7efded600640 1 -- 172.21.15.18:0/4137708491 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf009ab90 con 0x563d937cce90 2024-04-16T17:21:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.974+0000 7efded600640 1 -- 172.21.15.18:0/4137708491 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00a5070 con 0x563d937cce90 2024-04-16T17:21:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.975+0000 7efe06819740 1 -- 172.21.15.18:0/4137708491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cce90 msgr2=0x563d937d5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.975+0000 7efe06819740 1 --2- 172.21.15.18:0/4137708491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cce90 0x563d937d5aa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efdf00ce040 tx=0x7efdf00d0040 comp rx=0 tx=0).stop 2024-04-16T17:21:54.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.975+0000 7efe06819740 1 -- 172.21.15.18:0/4137708491 shutdown_connections 2024-04-16T17:21:54.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.975+0000 7efe06819740 1 --2- 172.21.15.18:0/4137708491 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937d6390 0x563d937f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.975+0000 7efe06819740 1 --2- 172.21.15.18:0/4137708491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937cd7b0 0x563d937e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.975+0000 7efe06819740 1 --2- 172.21.15.18:0/4137708491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cce90 0x563d937d5aa0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.975+0000 7efe06819740 1 -- 172.21.15.18:0/4137708491 >> 172.21.15.18:0/4137708491 conn(0x563d937c75c0 msgr2=0x563d937c9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:54.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.975+0000 7efe06819740 1 -- 172.21.15.18:0/4137708491 shutdown_connections 2024-04-16T17:21:54.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.976+0000 7efe06819740 1 -- 172.21.15.18:0/4137708491 wait complete. 2024-04-16T17:21:54.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.976+0000 7efe06819740 1 -- start start 2024-04-16T17:21:54.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.976+0000 7efe06819740 1 Processor -- start 2024-04-16T17:21:54.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efe06819740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d93850b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efe06819740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efe06819740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 0x563d937deae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efe06819740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1ab8f0 con 0x563d937cce90 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efe06819740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1acd60 con 0x563d937d6390 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efe06819740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1abfc0 con 0x563d937cd7b0 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efe06819740 10 client.0 _ll_register_callbacks cb 0x7efde8004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937de5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52510/0 (socket says 172.21.15.18:52510) 2024-04-16T17:21:54.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.977+0000 7efdfbe00640 1 -- 172.21.15.18:0/481817548 learned_addr learned my addr 172.21.15.18:0/481817548 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.978+0000 7efdfbe00640 1 -- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 msgr2=0x563d937deae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.978+0000 7efe01c00640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 0x563d937deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.978+0000 7efdfbe00640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 0x563d937deae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.978+0000 7efdfbe00640 1 -- 172.21.15.18:0/481817548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 msgr2=0x563d93850b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.978+0000 7efe01200640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d93850b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.978+0000 7efdfbe00640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d93850b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.978+0000 7efdfbe00640 1 -- 172.21.15.18:0/481817548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d937d0490 con 0x563d937cd7b0 2024-04-16T17:21:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.978+0000 7efe01c00640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 0x563d937deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.979+0000 7efe01200640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d93850b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.979+0000 7efdfbe00640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937de5a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efde802c240 tx=0x7efde802ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f4b36d5c8cf26ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.979+0000 7efde2a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.980+0000 7efde2a00640 1 -- 172.21.15.18:0/481817548 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8003df0 con 0x563d937cd7b0 2024-04-16T17:21:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.980+0000 7efde2a00640 1 -- 172.21.15.18:0/481817548 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7efde8004a60 con 0x563d937cd7b0 2024-04-16T17:21:54.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.980+0000 7efe06819740 1 -- 172.21.15.18:0/481817548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf009ab90 con 0x563d937cd7b0 2024-04-16T17:21:54.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.981+0000 7efde2a00640 1 -- 172.21.15.18:0/481817548 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efde8062050 con 0x563d937cd7b0 2024-04-16T17:21:54.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.981+0000 7efe06819740 10 client.29977 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:54.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.981+0000 7efe06819740 1 -- 172.21.15.18:0/481817548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563d937e2bf0 con 0x563d937cd7b0 2024-04-16T17:21:54.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.981+0000 7efde2a00640 1 -- 172.21.15.18:0/481817548 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf009ab90 con 0x563d937cd7b0 2024-04-16T17:21:54.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.983+0000 7efde2a00640 1 -- 172.21.15.18:0/481817548 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7efde8004ee0 con 0x563d937cd7b0 2024-04-16T17:21:54.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.983+0000 7efe06819740 10 client.29977 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:54.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.983+0000 7efe06819740 10 client.29977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:54.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.983+0000 7efe06819740 20 client.29977 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:54.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.983+0000 7efe06819740 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d937e33f0 0x563d937e5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:54.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.983+0000 7efe06819740 4 client.29977 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:21:54.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.983+0000 7efe06819740 1 -- 172.21.15.18:0/481817548 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563d93739540 con 0x563d937e33f0 2024-04-16T17:21:54.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.984+0000 7efe01200640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d937e33f0 0x563d937e5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:54.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.985+0000 7efe01200640 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d937e33f0 0x563d937e5810 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93fa5eefe4f6a4b5 server_cookie=2761b09d5f9e1013 in_seq=0 out_seq=0 2024-04-16T17:21:54.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.985+0000 7efde2a00640 10 client.29977 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:54.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.992+0000 7efde2a00640 1 -- 172.21.15.18:0/481817548 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563d93739540 con 0x563d937e33f0 2024-04-16T17:21:54.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.992+0000 7efde2a00640 10 client.29977 handle_command_reply: tid=0 2024-04-16T17:21:54.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.992+0000 7efe06819740 1 client.29977 shutdown 2024-04-16T17:21:54.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.992+0000 7efe06819740 1 -- 172.21.15.18:0/481817548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 msgr2=0x563d937de5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.992+0000 7efe06819740 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937de5a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efde802c240 tx=0x7efde802ce60 comp rx=0 tx=0).stop 2024-04-16T17:21:54.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.993+0000 7efe06819740 1 -- 172.21.15.18:0/481817548 shutdown_connections 2024-04-16T17:21:54.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.993+0000 7efe06819740 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563d937e33f0 0x563d937e5810 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.993+0000 7efe06819740 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563d937d6390 0x563d937deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.993+0000 7efe06819740 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d937cd7b0 0x563d937de5a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.993+0000 7efe06819740 1 --2- 172.21.15.18:0/481817548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563d937cce90 0x563d93850b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.993+0000 7efde2a00640 0 client.29977 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:54.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.993+0000 7efe06819740 1 -- 172.21.15.18:0/481817548 >> 172.21.15.18:0/481817548 conn(0x563d937c75c0 msgr2=0x563d937c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:54.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.993+0000 7efe06819740 1 -- 172.21.15.18:0/481817548 shutdown_connections 2024-04-16T17:21:54.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.994+0000 7efe06819740 1 -- 172.21.15.18:0/481817548 wait complete. 2024-04-16T17:21:54.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.994+0000 7efe06819740 20 client.29977 trim_cache size 0 max 16384 2024-04-16T17:21:54.995 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:21:54.995 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:21:54.995 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "45479b20-da40-49f5-8768-bc9488599637", 2024-04-16T17:21:54.995 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:21:54.995 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:21:54.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 -- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efde40a3da0 msgr2=0x7efde40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efde40a3da0 0x7efde40a6260 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdf400fc30 tx=0x7efdf4019000 comp rx=0 tx=0).stop 2024-04-16T17:21:54.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 -- 172.21.15.18:0/1776735741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104280 msgr2=0x7efdfc1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:54.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104280 0x7efdfc1a06b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c920 tx=0x7efdf000cde0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 -- 172.21.15.18:0/1776735741 shutdown_connections 2024-04-16T17:21:54.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efde40a3da0 0x7efde40a6260 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdfc19a770 0x7efdfc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc109350 0x7efdfc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 --2- 172.21.15.18:0/1776735741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104280 0x7efdfc1a06b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:54.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.996+0000 7efe02600640 1 -- 172.21.15.18:0/1776735741 >> 172.21.15.18:0/1776735741 conn(0x7efdfc0ffae0 msgr2=0x7efdfc100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:54.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.997+0000 7efe02600640 1 -- 172.21.15.18:0/1776735741 shutdown_connections 2024-04-16T17:21:54.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:54.997+0000 7efe02600640 1 -- 172.21.15.18:0/1776735741 wait complete. 2024-04-16T17:21:55.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:21:55.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:21:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.318+0000 7f9d0de00640 1 -- 172.21.15.18:0/3123080702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d08106210 msgr2=0x7f9d0810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.318+0000 7f9d0de00640 1 --2- 172.21.15.18:0/3123080702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d08106210 0x7f9d0810dde0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9d0001acb0 tx=0x7f9d0003d170 comp rx=0 tx=0).stop 2024-04-16T17:21:55.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.318+0000 7f9d0de00640 1 -- 172.21.15.18:0/3123080702 shutdown_connections 2024-04-16T17:21:55.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.318+0000 7f9d0de00640 1 --2- 172.21.15.18:0/3123080702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d08106210 0x7f9d0810dde0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.318+0000 7f9d0de00640 1 --2- 172.21.15.18:0/3123080702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d08101af0 0x7f9d08105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.318+0000 7f9d0de00640 1 --2- 172.21.15.18:0/3123080702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d08101140 0x7f9d08101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.318+0000 7f9d0de00640 1 -- 172.21.15.18:0/3123080702 >> 172.21.15.18:0/3123080702 conn(0x7f9d080780b0 msgr2=0x7f9d080784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:55.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.318+0000 7f9d0de00640 1 -- 172.21.15.18:0/3123080702 shutdown_connections 2024-04-16T17:21:55.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.319+0000 7f9d0de00640 1 -- 172.21.15.18:0/3123080702 wait complete. 2024-04-16T17:21:55.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.320+0000 7f9d0de00640 1 Processor -- start 2024-04-16T17:21:55.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.320+0000 7f9d0de00640 1 -- start start 2024-04-16T17:21:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.321+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d08101140 0x7f9d0810dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d08106210 0x7f9d08108ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d081091e0 0x7f9d08109690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d0de00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0806e4b0 con 0x7f9d08101140 2024-04-16T17:21:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d0de00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0806e330 con 0x7f9d08106210 2024-04-16T17:21:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d0de00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0806e630 con 0x7f9d081091e0 2024-04-16T17:21:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d06000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d08106210 0x7f9d08108ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d06000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d08106210 0x7f9d08108ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45898/0 (socket says 172.21.15.18:45898) 2024-04-16T17:21:55.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d06000640 1 -- 172.21.15.18:0/1212447608 learned_addr learned my addr 172.21.15.18:0/1212447608 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:55.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.322+0000 7f9d07400640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d081091e0 0x7f9d08109690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.323+0000 7f9d07400640 1 -- 172.21.15.18:0/1212447608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d08106210 msgr2=0x7f9d08108ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.323+0000 7f9d07400640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d08106210 0x7f9d08108ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.323+0000 7f9d07400640 1 -- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d08101140 msgr2=0x7f9d0810dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.323+0000 7f9d06a00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d08101140 0x7f9d0810dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.323+0000 7f9d07400640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d08101140 0x7f9d0810dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.323+0000 7f9d07400640 1 -- 172.21.15.18:0/1212447608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0819a7e0 con 0x7f9d081091e0 2024-04-16T17:21:55.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.323+0000 7f9d06a00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d08101140 0x7f9d0810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:55.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.326+0000 7f9d07400640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d081091e0 0x7f9d08109690 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9d00048040 tx=0x7f9d000445d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:55.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.327+0000 7f9d0ca00640 1 -- 172.21.15.18:0/1212447608 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00052020 con 0x7f9d081091e0 2024-04-16T17:21:55.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.327+0000 7f9d0ca00640 1 -- 172.21.15.18:0/1212447608 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d00044ca0 con 0x7f9d081091e0 2024-04-16T17:21:55.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.327+0000 7f9d0ca00640 1 -- 172.21.15.18:0/1212447608 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0004a040 con 0x7f9d081091e0 2024-04-16T17:21:55.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.327+0000 7f9d0de00640 1 -- 172.21.15.18:0/1212447608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0819aa70 con 0x7f9d081091e0 2024-04-16T17:21:55.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.327+0000 7f9d0de00640 1 -- 172.21.15.18:0/1212447608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0819aee0 con 0x7f9d081091e0 2024-04-16T17:21:55.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.332+0000 7f9d10baa740 1 Processor -- start 2024-04-16T17:21:55.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.334+0000 7f9d0ca00640 1 -- 172.21.15.18:0/1212447608 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d00055020 con 0x7f9d081091e0 2024-04-16T17:21:55.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.334+0000 7f9d10baa740 1 -- start start 2024-04-16T17:21:55.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.334+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 0x55d7256e0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d10baa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d72563d120 con 0x55d7256d7ad0 2024-04-16T17:21:55.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d10baa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d7255e3340 con 0x55d7256e06b0 2024-04-16T17:21:55.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d10baa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d7256390e0 con 0x55d7256d71b0 2024-04-16T17:21:55.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256d7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52538/0 (socket says 172.21.15.18:52538) 2024-04-16T17:21:55.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 -- 172.21.15.18:0/192695457 learned_addr learned my addr 172.21.15.18:0/192695457 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:55.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 --2- 172.21.15.18:0/192695457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 -- 172.21.15.18:0/192695457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 msgr2=0x55d7256e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 --2- 172.21.15.18:0/192695457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 -- 172.21.15.18:0/192695457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 msgr2=0x55d7256e0170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 --2- 172.21.15.18:0/192695457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 0x55d7256e0170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.335+0000 7f9d06000640 1 -- 172.21.15.18:0/192695457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7254197d0 con 0x55d7256d71b0 2024-04-16T17:21:55.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.336+0000 7f9d06000640 1 --2- 172.21.15.18:0/192695457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256d7590 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8017780 tx=0x7f9cf8009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=615fae1b9bc21293 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:55.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.336+0000 7f9cf3e00640 1 -- 172.21.15.18:0/192695457 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8015760 con 0x55d7256d71b0 2024-04-16T17:21:55.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.336+0000 7f9cf3e00640 1 -- 172.21.15.18:0/192695457 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf801d070 con 0x55d7256d71b0 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.336+0000 7f9cf3e00640 1 -- 172.21.15.18:0/192695457 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8018c00 con 0x55d7256d71b0 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.336+0000 7f9d0ca00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cec0a3da0 0x7f9cec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.336+0000 7f9d0ca00640 1 -- 172.21.15.18:0/1212447608 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f9d001038a0 con 0x7f9d081091e0 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.337+0000 7f9d10baa740 1 -- 172.21.15.18:0/192695457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 msgr2=0x55d7256d7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.337+0000 7f9d10baa740 1 --2- 172.21.15.18:0/192695457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256d7590 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8017780 tx=0x7f9cf8009550 comp rx=0 tx=0).stop 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.337+0000 7f9d10baa740 1 -- 172.21.15.18:0/192695457 shutdown_connections 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.337+0000 7f9d10baa740 1 --2- 172.21.15.18:0/192695457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.337+0000 7f9d10baa740 1 --2- 172.21.15.18:0/192695457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 0x55d7256e0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.337+0000 7f9d10baa740 1 --2- 172.21.15.18:0/192695457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256d7590 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.337+0000 7f9d10baa740 1 -- 172.21.15.18:0/192695457 >> 172.21.15.18:0/192695457 conn(0x55d7256d18e0 msgr2=0x55d7256d1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:55.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.337+0000 7f9d06a00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cec0a3da0 0x7f9cec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.340+0000 7f9d10baa740 1 -- 172.21.15.18:0/192695457 shutdown_connections 2024-04-16T17:21:55.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.341+0000 7f9d06a00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cec0a3da0 0x7f9cec0a6260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55d7256d6c80 tx=0x7f9cf4008040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:55.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.341+0000 7f9d10baa740 1 -- 172.21.15.18:0/192695457 wait complete. 2024-04-16T17:21:55.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.342+0000 7f9d10baa740 1 -- start start 2024-04-16T17:21:55.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.342+0000 7f9d10baa740 1 Processor -- start 2024-04-16T17:21:55.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.343+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d71b0 0x55d7256dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.343+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d7ad0 0x55d7256ddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.343+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.343+0000 7f9d10baa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d7256e3440 con 0x55d7256d71b0 2024-04-16T17:21:55.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.343+0000 7f9d10baa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d7255f6710 con 0x55d7256e06b0 2024-04-16T17:21:55.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.343+0000 7f9d10baa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d7256e35c0 con 0x55d7256d7ad0 2024-04-16T17:21:55.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.343+0000 7f9d10baa740 10 client.0 _ll_register_callbacks cb 0x55d7255f1af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:55.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d7ad0 0x55d7256ddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d7ad0 0x55d7256ddbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52554/0 (socket says 172.21.15.18:52554) 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06000640 1 -- 172.21.15.18:0/2301149579 learned_addr learned my addr 172.21.15.18:0/2301149579 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06a00640 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d71b0 0x55d7256dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06000640 1 -- 172.21.15.18:0/2301149579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 msgr2=0x55d7256de130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06000640 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06000640 1 -- 172.21.15.18:0/2301149579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d71b0 msgr2=0x55d7256dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06000640 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d71b0 0x55d7256dd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.344+0000 7f9d06000640 1 -- 172.21.15.18:0/2301149579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72534f3f0 con 0x55d7256d7ad0 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.345+0000 7f9d06000640 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d7ad0 0x55d7256ddbf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8004810 tx=0x7f9cf8017f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0bb86e38c5a601 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:55.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.345+0000 7f9cf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.345+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2301149579 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8018d90 con 0x55d7256d7ad0 2024-04-16T17:21:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.345+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2301149579 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f9cf8015760 con 0x55d7256d7ad0 2024-04-16T17:21:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.345+0000 7f9d10baa740 1 -- 172.21.15.18:0/2301149579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cf801d070 con 0x55d7256d7ad0 2024-04-16T17:21:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.346+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2301149579 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9cf8033050 con 0x55d7256d7ad0 2024-04-16T17:21:55.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.346+0000 7f9d10baa740 10 client.29989 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:55.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.346+0000 7f9d10baa740 1 -- 172.21.15.18:0/2301149579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d72530f0f0 con 0x55d7256d7ad0 2024-04-16T17:21:55.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.346+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2301149579 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf801d070 con 0x55d7256d7ad0 2024-04-16T17:21:55.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.347+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2301149579 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f9cf80219d0 con 0x55d7256d7ad0 2024-04-16T17:21:55.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.347+0000 7f9d10baa740 10 client.29989 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.347+0000 7f9d10baa740 10 client.29989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.347+0000 7f9d10baa740 20 client.29989 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.347+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d7256f0960 0x55d7256f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.347+0000 7f9d10baa740 4 client.29989 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.347+0000 7f9d10baa740 1 -- 172.21.15.18:0/2301149579 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d72564ab40 con 0x55d7256f0960 2024-04-16T17:21:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.347+0000 7f9d06a00640 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d7256f0960 0x55d7256f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.348+0000 7f9d06a00640 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d7256f0960 0x55d7256f2d50 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c16588e0dc6a182 server_cookie=3015d3ea11c0b443 in_seq=0 out_seq=0 2024-04-16T17:21:55.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.348+0000 7f9cf0c00640 10 client.29989 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:55.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.352+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2301149579 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d72564ab40 con 0x55d7256f0960 2024-04-16T17:21:55.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.352+0000 7f9cf0c00640 10 client.29989 handle_command_reply: tid=0 2024-04-16T17:21:55.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 client.29989 shutdown 2024-04-16T17:21:55.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 -- 172.21.15.18:0/2301149579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d7ad0 msgr2=0x55d7256ddbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d7ad0 0x55d7256ddbf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8004810 tx=0x7f9cf8017f10 comp rx=0 tx=0).stop 2024-04-16T17:21:55.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 -- 172.21.15.18:0/2301149579 shutdown_connections 2024-04-16T17:21:55.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d7256f0960 0x55d7256f2d50 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256de130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d7ad0 0x55d7256ddbf0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2301149579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d71b0 0x55d7256dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 -- 172.21.15.18:0/2301149579 >> 172.21.15.18:0/2301149579 conn(0x55d7256d18e0 msgr2=0x55d7256d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:55.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9cf0c00640 0 client.29989 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:55.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 -- 172.21.15.18:0/2301149579 shutdown_connections 2024-04-16T17:21:55.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.353+0000 7f9d10baa740 1 -- 172.21.15.18:0/2301149579 wait complete. 2024-04-16T17:21:55.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.354+0000 7f9d10baa740 20 client.29989 trim_cache size 0 max 16384 2024-04-16T17:21:55.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.391+0000 7f9d10baa740 1 Processor -- start 2024-04-16T17:21:55.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.391+0000 7f9d10baa740 1 -- start start 2024-04-16T17:21:55.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.391+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256dfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256d7ad0 0x55d7256ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256e06b0 0x55d7256f8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d10baa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b8550 con 0x55d7256e06b0 2024-04-16T17:21:55.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d10baa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b5e20 con 0x55d7256d7ad0 2024-04-16T17:21:55.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d10baa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b3c90 con 0x55d7256d71b0 2024-04-16T17:21:55.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d07400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256e06b0 0x55d7256f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d06a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256dfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d06a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256dfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52562/0 (socket says 172.21.15.18:52562) 2024-04-16T17:21:55.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.392+0000 7f9d06a00640 1 -- 172.21.15.18:0/1080628473 learned_addr learned my addr 172.21.15.18:0/1080628473 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:55.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.393+0000 7f9d06a00640 1 -- 172.21.15.18:0/1080628473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256d7ad0 msgr2=0x55d7256ec330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:21:55.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.393+0000 7f9d06a00640 1 --2- 172.21.15.18:0/1080628473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256d7ad0 0x55d7256ec330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.393+0000 7f9d06a00640 1 -- 172.21.15.18:0/1080628473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256e06b0 msgr2=0x55d7256f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:55.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.393+0000 7f9d06a00640 1 --2- 172.21.15.18:0/1080628473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256e06b0 0x55d7256f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.393+0000 7f9d06a00640 1 -- 172.21.15.18:0/1080628473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7256e2aa0 con 0x55d7256d71b0 2024-04-16T17:21:55.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.393+0000 7f9d06a00640 1 --2- 172.21.15.18:0/1080628473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256dfde0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400c9e0 tx=0x7f9cf400f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d2a588279bd800 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:55.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.394+0000 7f9cf3400640 1 -- 172.21.15.18:0/1080628473 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf402b070 con 0x55d7256d71b0 2024-04-16T17:21:55.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.394+0000 7f9cf3400640 1 -- 172.21.15.18:0/1080628473 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4025030 con 0x55d7256d71b0 2024-04-16T17:21:55.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.394+0000 7f9cf3400640 1 -- 172.21.15.18:0/1080628473 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4025640 con 0x55d7256d71b0 2024-04-16T17:21:55.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 -- 172.21.15.18:0/1080628473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 msgr2=0x55d7256dfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 --2- 172.21.15.18:0/1080628473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256dfde0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400c9e0 tx=0x7f9cf400f000 comp rx=0 tx=0).stop 2024-04-16T17:21:55.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 -- 172.21.15.18:0/1080628473 shutdown_connections 2024-04-16T17:21:55.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 --2- 172.21.15.18:0/1080628473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256e06b0 0x55d7256f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 --2- 172.21.15.18:0/1080628473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256d7ad0 0x55d7256ec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 --2- 172.21.15.18:0/1080628473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256dfde0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 -- 172.21.15.18:0/1080628473 >> 172.21.15.18:0/1080628473 conn(0x55d7256d18e0 msgr2=0x55d7256dbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:55.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 -- 172.21.15.18:0/1080628473 shutdown_connections 2024-04-16T17:21:55.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.395+0000 7f9d10baa740 1 -- 172.21.15.18:0/1080628473 wait complete. 2024-04-16T17:21:55.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.396+0000 7f9d10baa740 1 -- start start 2024-04-16T17:21:55.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.396+0000 7f9d10baa740 1 Processor -- start 2024-04-16T17:21:55.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.399+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.399+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 0x55d7256e8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.399+0000 7f9d10baa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256d9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.399+0000 7f9d10baa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b42e0 con 0x55d7256d7ad0 2024-04-16T17:21:55.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.399+0000 7f9d10baa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b5750 con 0x55d7256e06b0 2024-04-16T17:21:55.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.399+0000 7f9d10baa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b49b0 con 0x55d7256d71b0 2024-04-16T17:21:55.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.399+0000 7f9d10baa740 10 client.0 _ll_register_callbacks cb 0x7f9cf8009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:55.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.399+0000 7f9d06a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256db390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52570/0 (socket says 172.21.15.18:52570) 2024-04-16T17:21:55.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06a00640 1 -- 172.21.15.18:0/2678933887 learned_addr learned my addr 172.21.15.18:0/2678933887 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:55.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06000640 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 0x55d7256e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06a00640 1 -- 172.21.15.18:0/2678933887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 msgr2=0x55d7256d9920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:55.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06a00640 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256d9920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06a00640 1 -- 172.21.15.18:0/2678933887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 msgr2=0x55d7256e8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:55.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06a00640 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 0x55d7256e8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06a00640 1 -- 172.21.15.18:0/2678933887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7256d9e60 con 0x55d7256d71b0 2024-04-16T17:21:55.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.400+0000 7f9d06000640 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 0x55d7256e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:55.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.403+0000 7f9d06a00640 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256db390 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400afe0 tx=0x7f9cf4015340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c092ab25b7fcff5c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:55.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.403+0000 7f9cf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:55.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.403+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2678933887 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf402b070 con 0x55d7256d71b0 2024-04-16T17:21:55.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.403+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2678933887 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f9cf4015db0 con 0x55d7256d71b0 2024-04-16T17:21:55.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.404+0000 7f9d10baa740 1 -- 172.21.15.18:0/2678933887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cf4025030 con 0x55d7256d71b0 2024-04-16T17:21:55.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.406+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2678933887 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9cf4005990 con 0x55d7256d71b0 2024-04-16T17:21:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.408+0000 7f9d10baa740 10 client.29995 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:55.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.408+0000 7f9d10baa740 1 -- 172.21.15.18:0/2678933887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d725756310 con 0x55d7256d71b0 2024-04-16T17:21:55.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.409+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2678933887 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4025030 con 0x55d7256d71b0 2024-04-16T17:21:55.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.410+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2678933887 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f9cf4006140 con 0x55d7256d71b0 2024-04-16T17:21:55.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.410+0000 7f9d10baa740 10 client.29995 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:55.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.410+0000 7f9d10baa740 10 client.29995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:55.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.410+0000 7f9d10baa740 20 client.29995 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.410+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d725756920 0x55d725758d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.410+0000 7f9d10baa740 4 client.29995 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:21:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.410+0000 7f9d10baa740 1 -- 172.21.15.18:0/2678933887 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7256449b0 con 0x55d725756920 2024-04-16T17:21:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.411+0000 7f9d06000640 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d725756920 0x55d725758d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:55.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.412+0000 7f9d06000640 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d725756920 0x55d725758d10 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a15c6c1af1afd0 server_cookie=6e34e543ec9dcaf2 in_seq=0 out_seq=0 2024-04-16T17:21:55.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.412+0000 7f9cf0c00640 10 client.29995 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.413+0000 7f9cf0c00640 1 -- 172.21.15.18:0/2678933887 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d7256449b0 con 0x55d725756920 2024-04-16T17:21:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.413+0000 7f9cf0c00640 10 client.29995 handle_command_reply: tid=0 2024-04-16T17:21:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.413+0000 7f9d10baa740 1 client.29995 shutdown 2024-04-16T17:21:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.413+0000 7f9d10baa740 1 -- 172.21.15.18:0/2678933887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 msgr2=0x55d7256db390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.413+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256db390 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400afe0 tx=0x7f9cf4015340 comp rx=0 tx=0).stop 2024-04-16T17:21:55.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.414+0000 7f9d10baa740 1 -- 172.21.15.18:0/2678933887 shutdown_connections 2024-04-16T17:21:55.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.414+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d725756920 0x55d725758d10 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.414+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7256e06b0 0x55d7256d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.414+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d7256d7ad0 0x55d7256e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.414+0000 7f9d10baa740 1 --2- 172.21.15.18:0/2678933887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7256d71b0 0x55d7256db390 secure :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400afe0 tx=0x7f9cf4015340 comp rx=0 tx=0).stop 2024-04-16T17:21:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.414+0000 7f9d10baa740 1 -- 172.21.15.18:0/2678933887 >> 172.21.15.18:0/2678933887 conn(0x55d7256d18e0 msgr2=0x55d7256d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.414+0000 7f9cf0c00640 0 client.29995 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.414+0000 7f9d10baa740 1 -- 172.21.15.18:0/2678933887 shutdown_connections 2024-04-16T17:21:55.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.415+0000 7f9d10baa740 1 -- 172.21.15.18:0/2678933887 wait complete. 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.415+0000 7f9d10baa740 20 client.29995 trim_cache size 0 max 16384 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (246 inodes in the stack)", 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: "45479b20-da40-49f5-8768-bc9488599637": { 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "45479b20-da40-49f5-8768-bc9488599637", 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:21:55.416 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:21:55.417 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:21:55.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 -- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cec0a3da0 msgr2=0x7f9cec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cec0a3da0 0x7f9cec0a6260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55d7256d6c80 tx=0x7f9cf4008040 comp rx=0 tx=0).stop 2024-04-16T17:21:55.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 -- 172.21.15.18:0/1212447608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d081091e0 msgr2=0x7f9d08109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:55.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d081091e0 0x7f9d08109690 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9d00048040 tx=0x7f9d000445d0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 -- 172.21.15.18:0/1212447608 shutdown_connections 2024-04-16T17:21:55.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cec0a3da0 0x7f9cec0a6260 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d081091e0 0x7f9d08109690 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d08106210 0x7f9d08108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 --2- 172.21.15.18:0/1212447608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d08101140 0x7f9d0810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:55.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.419+0000 7f9d0de00640 1 -- 172.21.15.18:0/1212447608 >> 172.21.15.18:0/1212447608 conn(0x7f9d080780b0 msgr2=0x7f9d080ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:55.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.420+0000 7f9d0de00640 1 -- 172.21.15.18:0/1212447608 shutdown_connections 2024-04-16T17:21:55.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:21:55.421+0000 7f9d0de00640 1 -- 172.21.15.18:0/1212447608 wait complete. 2024-04-16T17:21:55.454 INFO:tasks.cephfs.filesystem:scrub status for tag:45479b20-da40-49f5-8768-bc9488599637 - {'path': '/', 'tag': '45479b20-da40-49f5-8768-bc9488599637', 'options': 'recursive,force'} 2024-04-16T17:21:55.505 INFO:tasks.workunit.client.0.smithi018.stdout: 10 62381 12.88 MB/sec execute 127 sec latency 194.495 ms 2024-04-16T17:21:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:55 smithi022 ceph-mon[36023]: pgmap v547: 129 pgs: 129 active+clean; 401 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 466 op/s 2024-04-16T17:21:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:55 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:21:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:55 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:55 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:21:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:55 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:56.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:55 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:21:56.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:55 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:56.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:55 smithi043 ceph-mon[36901]: pgmap v547: 129 pgs: 129 active+clean; 401 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 466 op/s 2024-04-16T17:21:56.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:55 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:21:56.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:55 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:56.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:55 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:21:56.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:55 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:56.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:55 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:21:56.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:55 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:56.504 INFO:tasks.workunit.client.0.smithi018.stdout: 10 62718 12.87 MB/sec execute 128 sec latency 269.272 ms 2024-04-16T17:21:57.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:56 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:21:57.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:56 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:57.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:56 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:21:57.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:56 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:57.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:56 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:21:57.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:56 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:57.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:56 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:21:57.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:56 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:21:57.504 INFO:tasks.workunit.client.0.smithi018.stdout: 10 63059 12.95 MB/sec execute 129 sec latency 155.625 ms 2024-04-16T17:21:58.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:57 smithi022 ceph-mon[36023]: pgmap v548: 129 pgs: 129 active+clean; 406 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 558 op/s 2024-04-16T17:21:58.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:57 smithi043 ceph-mon[36901]: pgmap v548: 129 pgs: 129 active+clean; 406 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 558 op/s 2024-04-16T17:21:58.504 INFO:tasks.workunit.client.0.smithi018.stdout: 10 63489 13.08 MB/sec execute 130 sec latency 57.257 ms 2024-04-16T17:21:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.741+0000 7f1e19c00640 1 -- 172.21.15.18:0/3227916621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e1410e640 msgr2=0x7f1e1410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.741+0000 7f1e19c00640 1 --2- 172.21.15.18:0/3227916621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e1410e640 0x7f1e1410ea20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009cd0 tx=0x7f1e0c02c0e0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.742+0000 7f1e19c00640 1 -- 172.21.15.18:0/3227916621 shutdown_connections 2024-04-16T17:21:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.742+0000 7f1e19c00640 1 --2- 172.21.15.18:0/3227916621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e14108ff0 0x7f1e14115500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.742+0000 7f1e19c00640 1 --2- 172.21.15.18:0/3227916621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e14108630 0x7f1e14108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.742+0000 7f1e19c00640 1 --2- 172.21.15.18:0/3227916621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e1410e640 0x7f1e1410ea20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.742+0000 7f1e19c00640 1 -- 172.21.15.18:0/3227916621 >> 172.21.15.18:0/3227916621 conn(0x7f1e14076510 msgr2=0x7f1e14078930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.742+0000 7f1e19c00640 1 -- 172.21.15.18:0/3227916621 shutdown_connections 2024-04-16T17:21:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.743+0000 7f1e19c00640 1 -- 172.21.15.18:0/3227916621 wait complete. 2024-04-16T17:21:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.745+0000 7f1e19c00640 1 Processor -- start 2024-04-16T17:21:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.745+0000 7f1e19c00640 1 -- start start 2024-04-16T17:21:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e19c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e14108630 0x7f1e1419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e19c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e14108ff0 0x7f1e1419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e19c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e1410e640 0x7f1e141a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e19c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14118a10 con 0x7f1e14108630 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e19c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14118890 con 0x7f1e1410e640 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e19c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14118b90 con 0x7f1e14108ff0 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e12000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e14108ff0 0x7f1e1419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e13400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e1410e640 0x7f1e141a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e12000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e14108ff0 0x7f1e1419f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42896/0 (socket says 172.21.15.18:42896) 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.747+0000 7f1e12000640 1 -- 172.21.15.18:0/2400348594 learned_addr learned my addr 172.21.15.18:0/2400348594 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.748+0000 7f1e12000640 1 -- 172.21.15.18:0/2400348594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e1410e640 msgr2=0x7f1e141a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.748+0000 7f1e12000640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e1410e640 0x7f1e141a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.748+0000 7f1e12000640 1 -- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e14108630 msgr2=0x7f1e1419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.748+0000 7f1e12000640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e14108630 0x7f1e1419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.748+0000 7f1e12000640 1 -- 172.21.15.18:0/2400348594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e141a3e70 con 0x7f1e14108ff0 2024-04-16T17:21:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.748+0000 7f1e13400640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e1410e640 0x7f1e141a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.748+0000 7f1e12000640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e14108ff0 0x7f1e1419f410 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c920 tx=0x7f1e0000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.749+0000 7f1e0b400640 1 -- 172.21.15.18:0/2400348594 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00007c70 con 0x7f1e14108ff0 2024-04-16T17:21:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.749+0000 7f1e0b400640 1 -- 172.21.15.18:0/2400348594 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e00007e10 con 0x7f1e14108ff0 2024-04-16T17:21:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.749+0000 7f1e0b400640 1 -- 172.21.15.18:0/2400348594 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00018890 con 0x7f1e14108ff0 2024-04-16T17:21:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.749+0000 7f1e19c00640 1 -- 172.21.15.18:0/2400348594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e141a4160 con 0x7f1e14108ff0 2024-04-16T17:21:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.750+0000 7f1e19c00640 1 -- 172.21.15.18:0/2400348594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e141ab9f0 con 0x7f1e14108ff0 2024-04-16T17:21:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.756+0000 7f1e1c963740 1 Processor -- start 2024-04-16T17:21:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.757+0000 7f1e1c963740 1 -- start start 2024-04-16T17:21:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.757+0000 7f1e0b400640 1 -- 172.21.15.18:0/2400348594 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e00018a70 con 0x7f1e14108ff0 2024-04-16T17:21:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.757+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754dd610 0x5589754dc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.758+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.758+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754e5e20 0x5589754e8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.758+0000 7f1e1c963740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558975441780 con 0x5589754dd610 2024-04-16T17:21:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.758+0000 7f1e1c963740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5589753e7f40 con 0x5589754dc8a0 2024-04-16T17:21:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.758+0000 7f1e1c963740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55897543d740 con 0x5589754e5e20 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.758+0000 7f1e12000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754e5e20 0x5589754e8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.758+0000 7f1e12000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754e5e20 0x5589754e8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42904/0 (socket says 172.21.15.18:42904) 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.758+0000 7f1e12000640 1 -- 172.21.15.18:0/2602506201 learned_addr learned my addr 172.21.15.18:0/2602506201 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.759+0000 7f1e12000640 1 -- 172.21.15.18:0/2602506201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 msgr2=0x5589754dcd20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.759+0000 7f1e12000640 1 --2- 172.21.15.18:0/2602506201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754dcd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.759+0000 7f1e12000640 1 -- 172.21.15.18:0/2602506201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754dd610 msgr2=0x5589754dc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.759+0000 7f1e12000640 1 --2- 172.21.15.18:0/2602506201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754dd610 0x5589754dc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.759+0000 7f1e12000640 1 -- 172.21.15.18:0/2602506201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558975154210 con 0x5589754e5e20 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.759+0000 7f1e0b400640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1df80a3e60 0x7f1df80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.759+0000 7f1e0b400640 1 -- 172.21.15.18:0/2400348594 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f1e000d0d30 con 0x7f1e14108ff0 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.759+0000 7f1e12000640 1 --2- 172.21.15.18:0/2602506201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754e5e20 0x5589754e8210 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1e000cdfe0 tx=0x7f1e000d74e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3737d2cd873afce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.760+0000 7f1dffe00640 1 -- 172.21.15.18:0/2602506201 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0009c690 con 0x5589754e5e20 2024-04-16T17:21:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.760+0000 7f1dffe00640 1 -- 172.21.15.18:0/2602506201 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0009cfc0 con 0x5589754e5e20 2024-04-16T17:21:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.760+0000 7f1dffe00640 1 -- 172.21.15.18:0/2602506201 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0009c220 con 0x5589754e5e20 2024-04-16T17:21:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.761+0000 7f1e13400640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1df80a3e60 0x7f1df80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.762+0000 7f1e1c963740 1 -- 172.21.15.18:0/2602506201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754e5e20 msgr2=0x5589754e8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.762+0000 7f1e1c963740 1 --2- 172.21.15.18:0/2602506201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754e5e20 0x5589754e8210 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1e000cdfe0 tx=0x7f1e000d74e0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.763+0000 7f1e13400640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1df80a3e60 0x7f1df80a6320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005d80 tx=0x7f1e04005e50 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.763+0000 7f1e1c963740 1 -- 172.21.15.18:0/2602506201 shutdown_connections 2024-04-16T17:21:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.763+0000 7f1e1c963740 1 --2- 172.21.15.18:0/2602506201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754e5e20 0x5589754e8210 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.763+0000 7f1e1c963740 1 --2- 172.21.15.18:0/2602506201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.763+0000 7f1e1c963740 1 --2- 172.21.15.18:0/2602506201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754dd610 0x5589754dc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.763+0000 7f1e1c963740 1 -- 172.21.15.18:0/2602506201 >> 172.21.15.18:0/2602506201 conn(0x5589754775d0 msgr2=0x5589754d7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.764+0000 7f1e1c963740 1 -- 172.21.15.18:0/2602506201 shutdown_connections 2024-04-16T17:21:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.765+0000 7f1e1c963740 1 -- 172.21.15.18:0/2602506201 wait complete. 2024-04-16T17:21:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.765+0000 7f1e1c963740 1 -- start start 2024-04-16T17:21:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.766+0000 7f1e1c963740 1 Processor -- start 2024-04-16T17:21:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.767+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dc8a0 0x5589754e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.767+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dd610 0x5589754e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.767+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754e3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e1c963740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5589754e8b50 con 0x5589754e5e20 2024-04-16T17:21:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e1c963740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5589753fb390 con 0x5589754dd610 2024-04-16T17:21:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e1c963740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589754e8cd0 con 0x5589754dc8a0 2024-04-16T17:21:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e1c963740 10 client.0 _ll_register_callbacks cb 0x5589753e4f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e12000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dd610 0x5589754e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e13400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e12000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dd610 0x5589754e2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44542/0 (socket says 172.21.15.18:44542) 2024-04-16T17:21:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e12000640 1 -- 172.21.15.18:0/4271935790 learned_addr learned my addr 172.21.15.18:0/4271935790 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.768+0000 7f1e12a00640 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dc8a0 0x5589754e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.770+0000 7f1e13400640 1 -- 172.21.15.18:0/4271935790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dc8a0 msgr2=0x5589754e58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.770+0000 7f1e13400640 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dc8a0 0x5589754e58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.770+0000 7f1e13400640 1 -- 172.21.15.18:0/4271935790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dd610 msgr2=0x5589754e2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.770+0000 7f1e13400640 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dd610 0x5589754e2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.770+0000 7f1e13400640 1 -- 172.21.15.18:0/4271935790 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558975220620 con 0x5589754e5e20 2024-04-16T17:21:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.770+0000 7f1e12a00640 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dc8a0 0x5589754e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:21:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.771+0000 7f1e12000640 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dd610 0x5589754e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.772+0000 7f1dfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:21:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.772+0000 7f1e13400640 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754e3340 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1e04004700 tx=0x7f1e04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1c97d975fbf7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.773+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4271935790 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04022020 con 0x5589754e5e20 2024-04-16T17:21:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.774+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4271935790 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f1e040107d0 con 0x5589754e5e20 2024-04-16T17:21:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.774+0000 7f1e1c963740 1 -- 172.21.15.18:0/4271935790 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0009cfc0 con 0x5589754e5e20 2024-04-16T17:21:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.775+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4271935790 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1e040109d0 con 0x5589754e5e20 2024-04-16T17:21:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.775+0000 7f1e1c963740 10 client.19443 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.775+0000 7f1e1c963740 1 -- 172.21.15.18:0/4271935790 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5589754f4d00 con 0x5589754e5e20 2024-04-16T17:21:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.776+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4271935790 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0009cfc0 con 0x5589754e5e20 2024-04-16T17:21:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.776+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4271935790 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f1e0400f040 con 0x5589754e5e20 2024-04-16T17:21:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.776+0000 7f1e1c963740 10 client.19443 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.776+0000 7f1e1c963740 10 client.19443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.777+0000 7f1e1c963740 20 client.19443 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.777+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5589754f61c0 0x5589754f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.777+0000 7f1e1c963740 4 client.19443 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:21:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.777+0000 7f1e1c963740 1 -- 172.21.15.18:0/4271935790 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55897544f1a0 con 0x5589754f61c0 2024-04-16T17:21:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.777+0000 7f1e12a00640 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5589754f61c0 0x5589754f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.778+0000 7f1e12a00640 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5589754f61c0 0x5589754f85b0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee9211d3b59202b server_cookie=9d8d0724b56e6cd9 in_seq=0 out_seq=0 2024-04-16T17:21:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.778+0000 7f1dfcc00640 10 client.19443 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.782+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4271935790 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55897544f1a0 con 0x5589754f61c0 2024-04-16T17:21:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.782+0000 7f1dfcc00640 10 client.19443 handle_command_reply: tid=0 2024-04-16T17:21:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.782+0000 7f1e1c963740 1 client.19443 shutdown 2024-04-16T17:21:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 -- 172.21.15.18:0/4271935790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 msgr2=0x5589754e3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754e3340 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1e04004700 tx=0x7f1e04004730 comp rx=0 tx=0).stop 2024-04-16T17:21:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 -- 172.21.15.18:0/4271935790 shutdown_connections 2024-04-16T17:21:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5589754f61c0 0x5589754f85b0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754e3340 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dd610 0x5589754e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4271935790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dc8a0 0x5589754e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 -- 172.21.15.18:0/4271935790 >> 172.21.15.18:0/4271935790 conn(0x5589754775d0 msgr2=0x5589754d92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1dfcc00640 0 client.19443 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.783+0000 7f1e1c963740 1 -- 172.21.15.18:0/4271935790 shutdown_connections 2024-04-16T17:21:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.785+0000 7f1e1c963740 1 -- 172.21.15.18:0/4271935790 wait complete. 2024-04-16T17:21:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.785+0000 7f1e1c963740 20 client.19443 trim_cache size 0 max 16384 2024-04-16T17:21:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.814+0000 7f1e1c963740 1 Processor -- start 2024-04-16T17:21:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.815+0000 7f1e1c963740 1 -- start start 2024-04-16T17:21:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.815+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.815+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.815+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.815+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.815+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:44544/0 (socket says 172.21.15.18:44544) 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.815+0000 7f1e12000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.815+0000 7f1e12000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754fdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42930/0 (socket says 172.21.15.18:42930) 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.816+0000 7f1e13400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.816+0000 7f1e12a00640 1 -- 172.21.15.18:0/2001841181 learned_addr learned my addr 172.21.15.18:0/2001841181 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.816+0000 7f1e1c963740 1 -- 172.21.15.18:0/2001841181 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1e141b85c0 con 0x5589754e5e20 2024-04-16T17:21:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.816+0000 7f1e1c963740 1 -- 172.21.15.18:0/2001841181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e141b5e90 con 0x5589754dc8a0 2024-04-16T17:21:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.816+0000 7f1e1c963740 1 -- 172.21.15.18:0/2001841181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e141b3d00 con 0x5589754dd610 2024-04-16T17:21:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.817+0000 7f1e12a00640 1 -- 172.21.15.18:0/2001841181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 msgr2=0x5589754fdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.817+0000 7f1e12a00640 1 --2- 172.21.15.18:0/2001841181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754fdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.817+0000 7f1e12a00640 1 -- 172.21.15.18:0/2001841181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 msgr2=0x5589754fe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.817+0000 7f1e12a00640 1 --2- 172.21.15.18:0/2001841181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754fe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.817+0000 7f1e12a00640 1 -- 172.21.15.18:0/2001841181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589754e8540 con 0x5589754dc8a0 2024-04-16T17:21:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.817+0000 7f1e12000640 1 --2- 172.21.15.18:0/2001841181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754fdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:21:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.817+0000 7f1e12a00640 1 --2- 172.21.15.18:0/2001841181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754f1ad0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c02c6f0 tx=0x7f1e0c004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f7331fa09f9a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.818+0000 7f1dff400640 1 -- 172.21.15.18:0/2001841181 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c039040 con 0x5589754dc8a0 2024-04-16T17:21:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.818+0000 7f1dff400640 1 -- 172.21.15.18:0/2001841181 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0c048030 con 0x5589754dc8a0 2024-04-16T17:21:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.818+0000 7f1dff400640 1 -- 172.21.15.18:0/2001841181 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0042a0 con 0x5589754dc8a0 2024-04-16T17:21:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 -- 172.21.15.18:0/2001841181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 msgr2=0x5589754f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 --2- 172.21.15.18:0/2001841181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754f1ad0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c02c6f0 tx=0x7f1e0c004a80 comp rx=0 tx=0).stop 2024-04-16T17:21:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 -- 172.21.15.18:0/2001841181 shutdown_connections 2024-04-16T17:21:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 --2- 172.21.15.18:0/2001841181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5589754e5e20 0x5589754fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 --2- 172.21.15.18:0/2001841181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754fdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 --2- 172.21.15.18:0/2001841181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754dc8a0 0x5589754f1ad0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 -- 172.21.15.18:0/2001841181 >> 172.21.15.18:0/2001841181 conn(0x5589754775d0 msgr2=0x5589754d70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 -- 172.21.15.18:0/2001841181 shutdown_connections 2024-04-16T17:21:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.819+0000 7f1e1c963740 1 -- 172.21.15.18:0/2001841181 wait complete. 2024-04-16T17:21:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.820+0000 7f1e1c963740 1 -- start start 2024-04-16T17:21:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.820+0000 7f1e1c963740 1 Processor -- start 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754dff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754e5e20 0x5589754e04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e1c963740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558975559720 0x55897555bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e1c963740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1e141b4350 con 0x558975559720 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e1c963740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e141b57c0 con 0x5589754e5e20 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e1c963740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e141b4a20 con 0x5589754dd610 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e1c963740 10 client.0 _ll_register_callbacks cb 0x7f1e040092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754dff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42944/0 (socket says 172.21.15.18:42944) 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e12a00640 1 -- 172.21.15.18:0/4287961359 learned_addr learned my addr 172.21.15.18:0/4287961359 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e12000640 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754e5e20 0x5589754e04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e12a00640 1 -- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754e5e20 msgr2=0x5589754e04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e12a00640 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754e5e20 0x5589754e04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e12a00640 1 -- 172.21.15.18:0/4287961359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558975559720 msgr2=0x55897555bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:21:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.821+0000 7f1e12a00640 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558975559720 0x55897555bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.822+0000 7f1e12a00640 1 -- 172.21.15.18:0/4287961359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897556d060 con 0x5589754dd610 2024-04-16T17:21:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.822+0000 7f1e12000640 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754e5e20 0x5589754e04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.822+0000 7f1e12a00640 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754dff70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c005e30 tx=0x7f1e0c032c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4caaa671fc81c831 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.822+0000 7f1dfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.822+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4287961359 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c039040 con 0x5589754dd610 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.823+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4287961359 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f1e0c032790 con 0x5589754dd610 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.823+0000 7f1e1c963740 1 -- 172.21.15.18:0/4287961359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0c048030 con 0x5589754dd610 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.823+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4287961359 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1e0c032e60 con 0x5589754dd610 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.823+0000 7f1e1c963740 10 client.30010 fetch_fsmap learned FSMap version 37 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.823+0000 7f1e1c963740 1 -- 172.21.15.18:0/4287961359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55897555c050 con 0x5589754dd610 2024-04-16T17:21:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.824+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4287961359 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0c048030 con 0x5589754dd610 2024-04-16T17:21:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.824+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4287961359 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f1e0c0046d0 con 0x5589754dd610 2024-04-16T17:21:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.825+0000 7f1e1c963740 10 client.30010 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:21:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.825+0000 7f1e1c963740 10 client.30010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:21:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.825+0000 7f1e1c963740 20 client.30010 populate_metadata read hostname 'smithi018' 2024-04-16T17:21:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.825+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55897555c630 0x55897555ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:21:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.825+0000 7f1e1c963740 4 client.30010 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:21:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.825+0000 7f1e1c963740 1 -- 172.21.15.18:0/4287961359 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558975449010 con 0x55897555c630 2024-04-16T17:21:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.825+0000 7f1e12000640 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55897555c630 0x55897555ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:21:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.826+0000 7f1e12000640 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55897555c630 0x55897555ea90 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82dab6a9bedd665 server_cookie=9259a1102b562049 in_seq=0 out_seq=0 2024-04-16T17:21:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:58.826+0000 7f1dfcc00640 10 client.30010 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:59.505 INFO:tasks.workunit.client.0.smithi018.stdout: 10 63660 13.04 MB/sec execute 131 sec latency 598.249 ms 2024-04-16T17:21:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.845+0000 7f1dfcc00640 1 -- 172.21.15.18:0/4287961359 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558975449010 con 0x55897555c630 2024-04-16T17:21:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.847+0000 7f1dfcc00640 10 client.30010 handle_command_reply: tid=0 2024-04-16T17:21:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.847+0000 7f1e1c963740 1 client.30010 shutdown 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 -- 172.21.15.18:0/4287961359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 msgr2=0x5589754dff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754dff70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c005e30 tx=0x7f1e0c032c70 comp rx=0 tx=0).stop 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 -- 172.21.15.18:0/4287961359 shutdown_connections 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55897555c630 0x55897555ea90 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558975559720 0x55897555bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589754e5e20 0x5589754e04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1dfcc00640 0 client.30010 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 --2- 172.21.15.18:0/4287961359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589754dd610 0x5589754dff70 secure :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c005e30 tx=0x7f1e0c032c70 comp rx=0 tx=0).stop 2024-04-16T17:21:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 -- 172.21.15.18:0/4287961359 >> 172.21.15.18:0/4287961359 conn(0x5589754775d0 msgr2=0x5589754e14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.852+0000 7f1e1c963740 1 -- 172.21.15.18:0/4287961359 shutdown_connections 2024-04-16T17:21:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.853+0000 7f1e1c963740 1 -- 172.21.15.18:0/4287961359 wait complete. 2024-04-16T17:21:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.854+0000 7f1e1c963740 20 client.30010 trim_cache size 0 max 16384 2024-04-16T17:21:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:21:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 -- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1df80a3e60 msgr2=0x7f1df80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1df80a3e60 0x7f1df80a6320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005d80 tx=0x7f1e04005e50 comp rx=0 tx=0).stop 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 -- 172.21.15.18:0/2400348594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e14108ff0 msgr2=0x7f1e1419f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e14108ff0 0x7f1e1419f410 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c920 tx=0x7f1e0000cde0 comp rx=0 tx=0).stop 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 -- 172.21.15.18:0/2400348594 shutdown_connections 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1df80a3e60 0x7f1df80a6320 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e1410e640 0x7f1e141a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e14108ff0 0x7f1e1419f410 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 --2- 172.21.15.18:0/2400348594 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e14108630 0x7f1e1419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 -- 172.21.15.18:0/2400348594 >> 172.21.15.18:0/2400348594 conn(0x7f1e14076510 msgr2=0x7f1e14076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.855+0000 7f1e19c00640 1 -- 172.21.15.18:0/2400348594 shutdown_connections 2024-04-16T17:21:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:21:59.856+0000 7f1e19c00640 1 -- 172.21.15.18:0/2400348594 wait complete. 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:52 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/558804192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:53 smithi018 ceph-mon[31776]: pgmap v546: 129 pgs: 129 active+clean; 401 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 483 op/s 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:55 smithi018 ceph-mon[31776]: pgmap v547: 129 pgs: 129 active+clean; 401 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 466 op/s 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:55 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:55 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:55 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:55 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:55 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:22:00.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:55 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:00.084 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:56 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:22:00.084 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:56 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:00.084 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:56 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:22:00.084 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:56 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:00.084 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:57 smithi018 ceph-mon[31776]: pgmap v548: 129 pgs: 129 active+clean; 406 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 558 op/s 2024-04-16T17:22:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:21:59 smithi022 ceph-mon[36023]: pgmap v549: 129 pgs: 129 active+clean; 409 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 349 op/s 2024-04-16T17:22:00.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:21:59 smithi043 ceph-mon[36901]: pgmap v549: 129 pgs: 129 active+clean; 409 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 349 op/s 2024-04-16T17:22:00.506 INFO:tasks.workunit.client.0.smithi018.stdout: 10 63920 13.06 MB/sec execute 132 sec latency 724.370 ms 2024-04-16T17:22:01.506 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64040 12.99 MB/sec execute 133 sec latency 589.087 ms 2024-04-16T17:22:01.588 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:21:59 smithi018 ceph-mon[31776]: pgmap v549: 129 pgs: 129 active+clean; 409 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 349 op/s 2024-04-16T17:22:02.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:01 smithi018 ceph-mon[31776]: pgmap v550: 129 pgs: 129 active+clean; 400 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-04-16T17:22:02.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:01 smithi022 ceph-mon[36023]: pgmap v550: 129 pgs: 129 active+clean; 400 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-04-16T17:22:02.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:01 smithi043 ceph-mon[36901]: pgmap v550: 129 pgs: 129 active+clean; 400 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-04-16T17:22:02.507 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64159 12.96 MB/sec execute 134 sec latency 1260.070 ms 2024-04-16T17:22:03.508 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64340 12.93 MB/sec execute 135 sec latency 510.460 ms 2024-04-16T17:22:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:03 smithi022 ceph-mon[36023]: pgmap v551: 129 pgs: 129 active+clean; 400 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-04-16T17:22:04.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:03 smithi043 ceph-mon[36901]: pgmap v551: 129 pgs: 129 active+clean; 400 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-04-16T17:22:04.508 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64501 12.86 MB/sec execute 136 sec latency 937.873 ms 2024-04-16T17:22:05.510 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64582 12.78 MB/sec execute 137 sec latency 1034.324 ms 2024-04-16T17:22:05.751 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:03 smithi018 ceph-mon[31776]: pgmap v551: 129 pgs: 129 active+clean; 400 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-04-16T17:22:06.131 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:05 smithi018 ceph-mon[31776]: pgmap v552: 129 pgs: 129 active+clean; 405 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 528 op/s 2024-04-16T17:22:06.131 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:05 smithi018 ceph-mon[31776]: osdmap e124: 12 total, 12 up, 12 in 2024-04-16T17:22:06.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:05 smithi043 ceph-mon[36901]: pgmap v552: 129 pgs: 129 active+clean; 405 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 528 op/s 2024-04-16T17:22:06.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:05 smithi043 ceph-mon[36901]: osdmap e124: 12 total, 12 up, 12 in 2024-04-16T17:22:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:05 smithi022 ceph-mon[36023]: pgmap v552: 129 pgs: 129 active+clean; 405 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 528 op/s 2024-04-16T17:22:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:05 smithi022 ceph-mon[36023]: osdmap e124: 12 total, 12 up, 12 in 2024-04-16T17:22:06.508 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64611 12.69 MB/sec execute 138 sec latency 1347.015 ms 2024-04-16T17:22:07.509 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64634 12.63 MB/sec execute 139 sec latency 1474.632 ms 2024-04-16T17:22:08.224 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:07 smithi022 ceph-mon[36023]: pgmap v554: 129 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 87 active+clean; 406 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-04-16T17:22:08.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:07 smithi043 ceph-mon[36901]: pgmap v554: 129 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 87 active+clean; 406 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-04-16T17:22:08.298 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:07 smithi018 ceph-mon[31776]: pgmap v554: 129 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 87 active+clean; 406 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-04-16T17:22:08.508 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64660 12.55 MB/sec execute 140 sec latency 1662.536 ms 2024-04-16T17:22:09.510 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64699 12.47 MB/sec execute 141 sec latency 2181.446 ms 2024-04-16T17:22:10.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:09 smithi043 ceph-mon[36901]: pgmap v555: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 403 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 662 op/s 2024-04-16T17:22:10.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:09 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:22:10.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:09 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:22:10.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:09 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:22:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:09 smithi022 ceph-mon[36023]: pgmap v555: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 403 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 662 op/s 2024-04-16T17:22:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:09 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:22:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:09 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:22:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:09 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:22:10.509 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64773 12.39 MB/sec execute 142 sec latency 2494.492 ms 2024-04-16T17:22:10.645 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:09 smithi018 ceph-mon[31776]: pgmap v555: 129 pgs: 19 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 61 active+clean; 403 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 662 op/s 2024-04-16T17:22:10.645 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:09 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:22:10.645 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:09 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:22:10.645 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:09 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:22:11.510 INFO:tasks.workunit.client.0.smithi018.stdout: 10 64919 12.41 MB/sec execute 143 sec latency 277.249 ms 2024-04-16T17:22:12.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:11 smithi043 ceph-mon[36901]: pgmap v556: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 434 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 518 op/s 2024-04-16T17:22:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:11 smithi022 ceph-mon[36023]: pgmap v556: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 434 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 518 op/s 2024-04-16T17:22:12.510 INFO:tasks.workunit.client.0.smithi018.stdout: 10 65070 12.40 MB/sec execute 144 sec latency 1049.019 ms 2024-04-16T17:22:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.069+0000 7f54b7a00640 1 -- 172.21.15.18:0/3420842574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 msgr2=0x7f54b0104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.069+0000 7f54b7a00640 1 --2- 172.21.15.18:0/3420842574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 0x7f54b0104850 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f549c009900 tx=0x7f549c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:22:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.070+0000 7f54b7a00640 1 -- 172.21.15.18:0/3420842574 shutdown_connections 2024-04-16T17:22:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.070+0000 7f54b7a00640 1 --2- 172.21.15.18:0/3420842574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54b0109540 0x7f54b0111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.070+0000 7f54b7a00640 1 --2- 172.21.15.18:0/3420842574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b0104e20 0x7f54b0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.070+0000 7f54b7a00640 1 --2- 172.21.15.18:0/3420842574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 0x7f54b0104850 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.070+0000 7f54b7a00640 1 -- 172.21.15.18:0/3420842574 >> 172.21.15.18:0/3420842574 conn(0x7f54b00ffcf0 msgr2=0x7f54b0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.072+0000 7f54b7a00640 1 -- 172.21.15.18:0/3420842574 shutdown_connections 2024-04-16T17:22:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.072+0000 7f54b7a00640 1 -- 172.21.15.18:0/3420842574 wait complete. 2024-04-16T17:22:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.074+0000 7f54b7a00640 1 Processor -- start 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.074+0000 7f54b7a00640 1 -- start start 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.074+0000 7f54b7a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 0x7f54b0076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.074+0000 7f54b7a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54b0109540 0x7f54b0076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 0x7f54b0076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b7a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b0076f90 0x7f54b01ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 0x7f54b0076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:58482/0 (socket says 172.21.15.18:58482) 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b5000640 1 -- 172.21.15.18:0/512159673 learned_addr learned my addr 172.21.15.18:0/512159673 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b7a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01146c0 con 0x7f54b0109540 2024-04-16T17:22:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54b0114540 con 0x7f54b0076f90 2024-04-16T17:22:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54b0114840 con 0x7f54b0104470 2024-04-16T17:22:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b5a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b0076f90 0x7f54b01ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b5000640 1 -- 172.21.15.18:0/512159673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b0076f90 msgr2=0x7f54b01ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b5000640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b0076f90 0x7f54b01ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b5000640 1 -- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54b0109540 msgr2=0x7f54b0076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.075+0000 7f54b5000640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54b0109540 0x7f54b0076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.076+0000 7f54b5000640 1 -- 172.21.15.18:0/512159673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b01ab690 con 0x7f54b0104470 2024-04-16T17:22:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.078+0000 7f54b5000640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 0x7f54b0076510 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f549c02c240 tx=0x7f549c004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.079+0000 7f54ad600640 1 -- 172.21.15.18:0/512159673 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c043020 con 0x7f54b0104470 2024-04-16T17:22:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.079+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b01ab920 con 0x7f54b0104470 2024-04-16T17:22:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.079+0000 7f54ad600640 1 -- 172.21.15.18:0/512159673 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549c0054e0 con 0x7f54b0104470 2024-04-16T17:22:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.079+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b01abe90 con 0x7f54b0104470 2024-04-16T17:22:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.079+0000 7f54ad600640 1 -- 172.21.15.18:0/512159673 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c003c00 con 0x7f54b0104470 2024-04-16T17:22:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.088+0000 7f54ba790740 1 Processor -- start 2024-04-16T17:22:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.088+0000 7f54ba790740 1 -- start start 2024-04-16T17:22:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.089+0000 7f54ba790740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c75360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.089+0000 7f54ba790740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.089+0000 7f54ba790740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c81210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.089+0000 7f54ba790740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d306bda780 con 0x55d306c758a0 2024-04-16T17:22:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.089+0000 7f54ba790740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d306b80f40 con 0x55d306c7ee20 2024-04-16T17:22:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.089+0000 7f54ba790740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d306bd6740 con 0x55d306c76610 2024-04-16T17:22:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.089+0000 7f54ad600640 1 -- 172.21.15.18:0/512159673 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f549c046020 con 0x7f54b0104470 2024-04-16T17:22:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.091+0000 7f54ad600640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54980a3de0 0x7f54980a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.091+0000 7f54ad600640 1 -- 172.21.15.18:0/512159673 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f549c02cc20 con 0x7f54b0104470 2024-04-16T17:22:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.092+0000 7f54b5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c75360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.092+0000 7f54b5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c75360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:58490/0 (socket says 172.21.15.18:58490) 2024-04-16T17:22:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.092+0000 7f54b5a00640 1 -- 172.21.15.18:0/674441456 learned_addr learned my addr 172.21.15.18:0/674441456 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.093+0000 7f54afe00640 1 --2- 172.21.15.18:0/674441456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c81210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.093+0000 7f54b5a00640 1 -- 172.21.15.18:0/674441456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 msgr2=0x55d306c81210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.093+0000 7f54b5a00640 1 --2- 172.21.15.18:0/674441456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c81210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.093+0000 7f54b5a00640 1 -- 172.21.15.18:0/674441456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 msgr2=0x55d306c75d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.093+0000 7f54b5a00640 1 --2- 172.21.15.18:0/674441456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.093+0000 7f54b5a00640 1 -- 172.21.15.18:0/674441456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3068ed210 con 0x55d306c76610 2024-04-16T17:22:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.093+0000 7f54b5a00640 1 --2- 172.21.15.18:0/674441456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c75360 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b590 tx=0x7f54a8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd838e31b9cdd21a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.094+0000 7f54a2000640 1 -- 172.21.15.18:0/674441456 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a801b070 con 0x55d306c76610 2024-04-16T17:22:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.094+0000 7f54a2000640 1 -- 172.21.15.18:0/674441456 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a8008dd0 con 0x55d306c76610 2024-04-16T17:22:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.094+0000 7f54a2000640 1 -- 172.21.15.18:0/674441456 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a80153a0 con 0x55d306c76610 2024-04-16T17:22:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 -- 172.21.15.18:0/674441456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 msgr2=0x55d306c75360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 --2- 172.21.15.18:0/674441456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c75360 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b590 tx=0x7f54a8009f90 comp rx=0 tx=0).stop 2024-04-16T17:22:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54b5a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54980a3de0 0x7f54980a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 -- 172.21.15.18:0/674441456 shutdown_connections 2024-04-16T17:22:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 --2- 172.21.15.18:0/674441456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c81210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 --2- 172.21.15.18:0/674441456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 --2- 172.21.15.18:0/674441456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c75360 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 -- 172.21.15.18:0/674441456 >> 172.21.15.18:0/674441456 conn(0x55d306c105d0 msgr2=0x55d306c70590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 -- 172.21.15.18:0/674441456 shutdown_connections 2024-04-16T17:22:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.095+0000 7f54ba790740 1 -- 172.21.15.18:0/674441456 wait complete. 2024-04-16T17:22:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.096+0000 7f54ba790740 1 -- start start 2024-04-16T17:22:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.096+0000 7f54ba790740 1 Processor -- start 2024-04-16T17:22:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.097+0000 7f54ba790740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.097+0000 7f54ba790740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c7bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c7bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c7bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:58498/0 (socket says 172.21.15.18:58498) 2024-04-16T17:22:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54ba790740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c7c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54afe00640 1 -- 172.21.15.18:0/2376220945 learned_addr learned my addr 172.21.15.18:0/2376220945 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d306c81b50 con 0x55d306c758a0 2024-04-16T17:22:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d306b94390 con 0x55d306c7ee20 2024-04-16T17:22:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d306c81cd0 con 0x55d306c76610 2024-04-16T17:22:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54ba790740 10 client.0 _ll_register_callbacks cb 0x55d306b7df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.098+0000 7f54b5000640 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.099+0000 7f54b5a00640 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c7c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.099+0000 7f54b5a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54980a3de0 0x7f54980a62a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f54a8016c40 tx=0x7f54a8009c30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.099+0000 7f54afe00640 1 -- 172.21.15.18:0/2376220945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 msgr2=0x55d306c7c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.099+0000 7f54afe00640 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c7c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.099+0000 7f54afe00640 1 -- 172.21.15.18:0/2376220945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 msgr2=0x55d306c7e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.099+0000 7f54afe00640 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c7e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.099+0000 7f54afe00640 1 -- 172.21.15.18:0/2376220945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3069b9620 con 0x55d306c76610 2024-04-16T17:22:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.100+0000 7f54afe00640 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c7bc60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f54b006b790 tx=0x7f54a400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e51fd7c186a01f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.100+0000 7f5496a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.100+0000 7f5496a00640 1 -- 172.21.15.18:0/2376220945 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4015070 con 0x55d306c76610 2024-04-16T17:22:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.100+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a8008dd0 con 0x55d306c76610 2024-04-16T17:22:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.100+0000 7f5496a00640 1 -- 172.21.15.18:0/2376220945 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f54a401a460 con 0x55d306c76610 2024-04-16T17:22:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.102+0000 7f5496a00640 1 -- 172.21.15.18:0/2376220945 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f54a4015070 con 0x55d306c76610 2024-04-16T17:22:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.102+0000 7f54ba790740 10 client.30022 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.102+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d306c8dd10 con 0x55d306c76610 2024-04-16T17:22:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.103+0000 7f5496a00640 1 -- 172.21.15.18:0/2376220945 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a8008dd0 con 0x55d306c76610 2024-04-16T17:22:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.103+0000 7f5496a00640 1 -- 172.21.15.18:0/2376220945 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f54a4011770 con 0x55d306c76610 2024-04-16T17:22:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.103+0000 7f54ba790740 10 client.30022 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.103+0000 7f54ba790740 10 client.30022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.103+0000 7f54ba790740 20 client.30022 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.103+0000 7f54ba790740 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d306c8f1d0 0x55d306c91600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.103+0000 7f54ba790740 4 client.30022 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.103+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d306be81a0 con 0x55d306c8f1d0 2024-04-16T17:22:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.104+0000 7f54b5000640 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d306c8f1d0 0x55d306c91600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.105+0000 7f54b5000640 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d306c8f1d0 0x55d306c91600 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1790100e07dc27de server_cookie=29b85e1a67737141 in_seq=0 out_seq=0 2024-04-16T17:22:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.105+0000 7f5496a00640 10 client.30022 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f5496a00640 1 -- 172.21.15.18:0/2376220945 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d306be81a0 con 0x55d306c8f1d0 2024-04-16T17:22:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f5496a00640 10 client.30022 handle_command_reply: tid=0 2024-04-16T17:22:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f54ba790740 1 client.30022 shutdown 2024-04-16T17:22:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 msgr2=0x55d306c7bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f54ba790740 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c7bc60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f54b006b790 tx=0x7f54a400f730 comp rx=0 tx=0).stop 2024-04-16T17:22:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 shutdown_connections 2024-04-16T17:22:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f54ba790740 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d306c8f1d0 0x55d306c91600 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f54ba790740 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c7c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.107+0000 7f54ba790740 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c7bc60 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.107+0000 7f54ba790740 1 --2- 172.21.15.18:0/2376220945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c7e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.107+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 >> 172.21.15.18:0/2376220945 conn(0x55d306c105d0 msgr2=0x55d306c722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.106+0000 7f5496a00640 0 client.30022 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.107+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 shutdown_connections 2024-04-16T17:22:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.107+0000 7f54ba790740 1 -- 172.21.15.18:0/2376220945 wait complete. 2024-04-16T17:22:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.107+0000 7f54ba790740 20 client.30022 trim_cache size 0 max 16384 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.142+0000 7f54ba790740 1 Processor -- start 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.142+0000 7f54ba790740 1 -- start start 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.142+0000 7f54ba790740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c8aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.142+0000 7f54ba790740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c96f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.142+0000 7f54ba790740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c97450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.142+0000 7f54ba790740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b8710 con 0x55d306c758a0 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.142+0000 7f54ba790740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b5fe0 con 0x55d306c7ee20 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.142+0000 7f54ba790740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b3df0 con 0x55d306c76610 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.143+0000 7f54afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c96f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.143+0000 7f54afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c96f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:58510/0 (socket says 172.21.15.18:58510) 2024-04-16T17:22:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.143+0000 7f54afe00640 1 -- 172.21.15.18:0/519907268 learned_addr learned my addr 172.21.15.18:0/519907268 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:13.143 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:11 smithi018 ceph-mon[31776]: pgmap v556: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 434 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 518 op/s 2024-04-16T17:22:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.144+0000 7f54b5a00640 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c97450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.145+0000 7f54afe00640 1 -- 172.21.15.18:0/519907268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 msgr2=0x55d306c97450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.145+0000 7f54afe00640 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c97450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.145+0000 7f54afe00640 1 -- 172.21.15.18:0/519907268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 msgr2=0x55d306c8aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.145+0000 7f54afe00640 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c8aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.145+0000 7f54afe00640 1 -- 172.21.15.18:0/519907268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d306c7a1b0 con 0x55d306c76610 2024-04-16T17:22:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.145+0000 7f54b5a00640 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c97450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:22:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.145+0000 7f54afe00640 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c96f10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f54a400c010 tx=0x7f54a400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237545c9429caec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.146+0000 7f54a1600640 1 -- 172.21.15.18:0/519907268 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4029040 con 0x55d306c76610 2024-04-16T17:22:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.146+0000 7f54a1600640 1 -- 172.21.15.18:0/519907268 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a4004520 con 0x55d306c76610 2024-04-16T17:22:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.146+0000 7f54a1600640 1 -- 172.21.15.18:0/519907268 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a40248e0 con 0x55d306c76610 2024-04-16T17:22:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.147+0000 7f54ba790740 1 -- 172.21.15.18:0/519907268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 msgr2=0x55d306c96f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.147+0000 7f54ba790740 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c96f10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f54a400c010 tx=0x7f54a400b8a0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.147+0000 7f54ba790740 1 -- 172.21.15.18:0/519907268 shutdown_connections 2024-04-16T17:22:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.147+0000 7f54ba790740 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c7ee20 0x55d306c97450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.147+0000 7f54ba790740 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c76610 0x55d306c96f10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.147+0000 7f54ba790740 1 --2- 172.21.15.18:0/519907268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c758a0 0x55d306c8aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.147+0000 7f54ba790740 1 -- 172.21.15.18:0/519907268 >> 172.21.15.18:0/519907268 conn(0x55d306c105d0 msgr2=0x55d306c7a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.148+0000 7f54ba790740 1 -- 172.21.15.18:0/519907268 shutdown_connections 2024-04-16T17:22:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.148+0000 7f54ba790740 1 -- 172.21.15.18:0/519907268 wait complete. 2024-04-16T17:22:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.149+0000 7f54ba790740 1 -- start start 2024-04-16T17:22:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.149+0000 7f54ba790740 1 Processor -- start 2024-04-16T17:22:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.150+0000 7f54ba790740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c758a0 0x55d306c829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.150+0000 7f54ba790740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c76610 0x55d306c82ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.151+0000 7f54b5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c758a0 0x55d306c829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.151+0000 7f54b5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c758a0 0x55d306c829a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45710/0 (socket says 172.21.15.18:45710) 2024-04-16T17:22:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.151+0000 7f54b5000640 1 -- 172.21.15.18:0/2609294762 learned_addr learned my addr 172.21.15.18:0/2609294762 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.151+0000 7f54ba790740 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c7ee20 0x55d306c83420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.151+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b44a0 con 0x55d306c76610 2024-04-16T17:22:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.151+0000 7f54b5a00640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c7ee20 0x55d306c83420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.152+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b5910 con 0x55d306c758a0 2024-04-16T17:22:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.152+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b4b70 con 0x55d306c7ee20 2024-04-16T17:22:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.152+0000 7f54ba790740 10 client.0 _ll_register_callbacks cb 0x7f54a4005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.152+0000 7f54afe00640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c76610 0x55d306c82ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.153+0000 7f54b5a00640 1 -- 172.21.15.18:0/2609294762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c758a0 msgr2=0x55d306c829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.153+0000 7f54b5a00640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c758a0 0x55d306c829a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.153+0000 7f54b5a00640 1 -- 172.21.15.18:0/2609294762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c76610 msgr2=0x55d306c82ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.153+0000 7f54b5a00640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c76610 0x55d306c82ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.153+0000 7f54b5a00640 1 -- 172.21.15.18:0/2609294762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d306d05790 con 0x55d306c7ee20 2024-04-16T17:22:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.153+0000 7f54b5000640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c758a0 0x55d306c829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:22:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.153+0000 7f54afe00640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c76610 0x55d306c82ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:22:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.155+0000 7f54b5a00640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c7ee20 0x55d306c83420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f54a800bfd0 tx=0x7f54a8004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e090ee7b164863 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.155+0000 7f5496a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.155+0000 7f5496a00640 1 -- 172.21.15.18:0/2609294762 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8032070 con 0x55d306c7ee20 2024-04-16T17:22:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.155+0000 7f5496a00640 1 -- 172.21.15.18:0/2609294762 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f54a801f600 con 0x55d306c7ee20 2024-04-16T17:22:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.156+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a4004520 con 0x55d306c7ee20 2024-04-16T17:22:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.156+0000 7f5496a00640 1 -- 172.21.15.18:0/2609294762 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f54a801b040 con 0x55d306c7ee20 2024-04-16T17:22:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.156+0000 7f54ba790740 10 client.30028 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.156+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d306d06a50 con 0x55d306c7ee20 2024-04-16T17:22:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.157+0000 7f5496a00640 1 -- 172.21.15.18:0/2609294762 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a4004520 con 0x55d306c7ee20 2024-04-16T17:22:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.157+0000 7f5496a00640 1 -- 172.21.15.18:0/2609294762 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f54a801e040 con 0x55d306c7ee20 2024-04-16T17:22:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.157+0000 7f54ba790740 10 client.30028 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.157+0000 7f54ba790740 10 client.30028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.157+0000 7f54ba790740 20 client.30028 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.157+0000 7f54ba790740 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d306d07020 0x55d306d09410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.157+0000 7f54ba790740 4 client.30028 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:22:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.157+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d306be2010 con 0x55d306d07020 2024-04-16T17:22:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.160+0000 7f54b5000640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d306d07020 0x55d306d09410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.161+0000 7f54b5000640 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d306d07020 0x55d306d09410 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b8aad79ed11067 server_cookie=626318d6a65c9eab in_seq=0 out_seq=0 2024-04-16T17:22:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:13.161+0000 7f5496a00640 10 client.30028 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:13.512 INFO:tasks.workunit.client.0.smithi018.stdout: 10 65181 12.36 MB/sec execute 145 sec latency 1349.343 ms 2024-04-16T17:22:14.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:13 smithi043 ceph-mon[36901]: pgmap v557: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 443 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 497 op/s 2024-04-16T17:22:14.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:13 smithi022 ceph-mon[36023]: pgmap v557: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 443 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 497 op/s 2024-04-16T17:22:14.513 INFO:tasks.workunit.client.0.smithi018.stdout: 10 65279 12.28 MB/sec execute 146 sec latency 1702.942 ms 2024-04-16T17:22:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.646+0000 7f5496a00640 1 -- 172.21.15.18:0/2609294762 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d306be2010 con 0x55d306d07020 2024-04-16T17:22:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.646+0000 7f5496a00640 10 client.30028 handle_command_reply: tid=0 2024-04-16T17:22:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.646+0000 7f54ba790740 1 client.30028 shutdown 2024-04-16T17:22:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.650+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c7ee20 msgr2=0x55d306c83420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.650+0000 7f54ba790740 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c7ee20 0x55d306c83420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f54a800bfd0 tx=0x7f54a8004b50 comp rx=0 tx=0).stop 2024-04-16T17:22:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.650+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 shutdown_connections 2024-04-16T17:22:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.650+0000 7f54ba790740 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d306d07020 0x55d306d09410 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.650+0000 7f54ba790740 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d306c7ee20 0x55d306c83420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.650+0000 7f5496a00640 0 client.30028 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.650+0000 7f54ba790740 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d306c76610 0x55d306c82ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.650+0000 7f54ba790740 1 --2- 172.21.15.18:0/2609294762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d306c758a0 0x55d306c829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.651+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 >> 172.21.15.18:0/2609294762 conn(0x55d306c105d0 msgr2=0x55d306c6ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.651+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 shutdown_connections 2024-04-16T17:22:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.651+0000 7f54ba790740 1 -- 172.21.15.18:0/2609294762 wait complete. 2024-04-16T17:22:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.651+0000 7f54ba790740 20 client.30028 trim_cache size 0 max 16384 2024-04-16T17:22:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:22:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:22:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:22:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.652+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54980a3de0 msgr2=0x7f54980a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.652+0000 7f54b7a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54980a3de0 0x7f54980a62a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f54a8016c40 tx=0x7f54a8009c30 comp rx=0 tx=0).stop 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.652+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 msgr2=0x7f54b0076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.652+0000 7f54b7a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 0x7f54b0076510 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f549c02c240 tx=0x7f549c004390 comp rx=0 tx=0).stop 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.653+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 shutdown_connections 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.653+0000 7f54b7a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f54980a3de0 0x7f54980a62a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.653+0000 7f54b7a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b0076f90 0x7f54b01ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.653+0000 7f54b7a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54b0109540 0x7f54b0076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.653+0000 7f54b7a00640 1 --2- 172.21.15.18:0/512159673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54b0104470 0x7f54b0076510 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.653+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 >> 172.21.15.18:0/512159673 conn(0x7f54b00ffcf0 msgr2=0x7f54b0101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.653+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 shutdown_connections 2024-04-16T17:22:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:14.653+0000 7f54b7a00640 1 -- 172.21.15.18:0/512159673 wait complete. 2024-04-16T17:22:15.000 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:13 smithi018 ceph-mon[31776]: pgmap v557: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 443 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 497 op/s 2024-04-16T17:22:15.513 INFO:tasks.workunit.client.0.smithi018.stdout: 10 65504 12.23 MB/sec execute 147 sec latency 1707.154 ms 2024-04-16T17:22:16.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:15 smithi043 ceph-mon[36901]: pgmap v558: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.9 MiB/s wr, 449 op/s 2024-04-16T17:22:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:15 smithi022 ceph-mon[36023]: pgmap v558: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.9 MiB/s wr, 449 op/s 2024-04-16T17:22:16.513 INFO:tasks.workunit.client.0.smithi018.stdout: 10 65692 12.21 MB/sec execute 148 sec latency 639.045 ms 2024-04-16T17:22:17.515 INFO:tasks.workunit.client.0.smithi018.stdout: 10 65853 12.18 MB/sec execute 149 sec latency 167.743 ms 2024-04-16T17:22:17.735 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:15 smithi018 ceph-mon[31776]: pgmap v558: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.9 MiB/s wr, 449 op/s 2024-04-16T17:22:18.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:18 smithi022 ceph-mon[36023]: pgmap v559: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 424 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.7 MiB/s wr, 527 op/s 2024-04-16T17:22:18.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:18 smithi018 ceph-mon[31776]: pgmap v559: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 424 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.7 MiB/s wr, 527 op/s 2024-04-16T17:22:18.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:18 smithi043 ceph-mon[36901]: pgmap v559: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 424 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.7 MiB/s wr, 527 op/s 2024-04-16T17:22:18.515 INFO:tasks.workunit.client.0.smithi018.stdout: 10 65878 12.10 MB/sec execute 150 sec latency 870.931 ms 2024-04-16T17:22:19.515 INFO:tasks.workunit.client.0.smithi018.stdout: 10 66109 12.15 MB/sec execute 151 sec latency 1013.191 ms 2024-04-16T17:22:20.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:20 smithi022 ceph-mon[36023]: pgmap v560: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 413 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 425 op/s 2024-04-16T17:22:20.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:20 smithi043 ceph-mon[36901]: pgmap v560: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 413 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 425 op/s 2024-04-16T17:22:20.517 INFO:tasks.workunit.client.0.smithi018.stdout: 10 66425 12.20 MB/sec execute 152 sec latency 271.443 ms 2024-04-16T17:22:21.517 INFO:tasks.workunit.client.0.smithi018.stdout: 10 66731 12.24 MB/sec execute 153 sec latency 158.652 ms 2024-04-16T17:22:22.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:22 smithi022 ceph-mon[36023]: pgmap v561: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 402 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 514 op/s 2024-04-16T17:22:22.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:22 smithi043 ceph-mon[36901]: pgmap v561: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 402 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 514 op/s 2024-04-16T17:22:22.517 INFO:tasks.workunit.client.0.smithi018.stdout: 10 66953 12.26 MB/sec execute 154 sec latency 244.740 ms 2024-04-16T17:22:22.952 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:20 smithi018 ceph-mon[31776]: pgmap v560: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 413 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 425 op/s 2024-04-16T17:22:22.953 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:22 smithi018 ceph-mon[31776]: pgmap v561: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 402 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 514 op/s 2024-04-16T17:22:23.517 INFO:tasks.workunit.client.0.smithi018.stdout: 10 66980 12.18 MB/sec execute 155 sec latency 965.850 ms 2024-04-16T17:22:24.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:24 smithi022 ceph-mon[36023]: pgmap v562: 129 pgs: 7 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 116 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-04-16T17:22:24.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:24 smithi043 ceph-mon[36901]: pgmap v562: 129 pgs: 7 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 116 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-04-16T17:22:24.517 INFO:tasks.workunit.client.0.smithi018.stdout: 10 67184 12.17 MB/sec execute 156 sec latency 1141.963 ms 2024-04-16T17:22:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:25 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:22:25.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:22:25.455 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:22:25.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:25 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:22:25.517 INFO:tasks.workunit.client.0.smithi018.stdout: 10 67421 12.18 MB/sec execute 157 sec latency 193.049 ms 2024-04-16T17:22:25.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.823+0000 7f4ddb600640 1 -- 172.21.15.18:0/2824869064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd41091e0 msgr2=0x7f4dd41095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:25.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.823+0000 7f4ddb600640 1 --2- 172.21.15.18:0/2824869064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd41091e0 0x7f4dd41095c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009900 tx=0x7f4dc002bc80 comp rx=0 tx=0).stop 2024-04-16T17:22:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.823+0000 7f4ddb600640 1 -- 172.21.15.18:0/2824869064 shutdown_connections 2024-04-16T17:22:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.823+0000 7f4ddb600640 1 --2- 172.21.15.18:0/2824869064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd4115150 0x7f4dd4117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.823+0000 7f4ddb600640 1 --2- 172.21.15.18:0/2824869064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd4109b00 0x7f4dd4114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.823+0000 7f4ddb600640 1 --2- 172.21.15.18:0/2824869064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd41091e0 0x7f4dd41095c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.823+0000 7f4ddb600640 1 -- 172.21.15.18:0/2824869064 >> 172.21.15.18:0/2824869064 conn(0x7f4dd406d0a0 msgr2=0x7f4dd406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.823+0000 7f4ddb600640 1 -- 172.21.15.18:0/2824869064 shutdown_connections 2024-04-16T17:22:25.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.824+0000 7f4ddb600640 1 -- 172.21.15.18:0/2824869064 wait complete. 2024-04-16T17:22:25.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.825+0000 7f4ddb600640 1 Processor -- start 2024-04-16T17:22:25.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4ddb600640 1 -- start start 2024-04-16T17:22:25.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4ddb600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd4109b00 0x7f4dd41149e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4ddb600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd4115150 0x7f4dd410ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4ddb600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd410ef90 0x7f4dd410f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4ddb600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4118ae0 con 0x7f4dd4115150 2024-04-16T17:22:25.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4ddb600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4118960 con 0x7f4dd410ef90 2024-04-16T17:22:25.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4ddb600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4118c60 con 0x7f4dd4109b00 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4dd8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd4109b00 0x7f4dd41149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.826+0000 7f4dd8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd4109b00 0x7f4dd41149e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51998/0 (socket says 172.21.15.18:51998) 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.827+0000 7f4dd8c00640 1 -- 172.21.15.18:0/1993771298 learned_addr learned my addr 172.21.15.18:0/1993771298 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.827+0000 7f4dd8c00640 1 -- 172.21.15.18:0/1993771298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd410ef90 msgr2=0x7f4dd410f440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.827+0000 7f4dd8c00640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd410ef90 0x7f4dd410f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.827+0000 7f4dd8c00640 1 -- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd4115150 msgr2=0x7f4dd410ea50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.827+0000 7f4dd8c00640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd4115150 0x7f4dd410ea50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.827+0000 7f4dd8c00640 1 -- 172.21.15.18:0/1993771298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd41bc900 con 0x7f4dd4109b00 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.828+0000 7f4dd8c00640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd4109b00 0x7f4dd41149e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4dc002c210 tx=0x7f4dc00045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:25.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.829+0000 7f4dd1600640 1 -- 172.21.15.18:0/1993771298 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0042020 con 0x7f4dd4109b00 2024-04-16T17:22:25.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.829+0000 7f4dd1600640 1 -- 172.21.15.18:0/1993771298 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc002cd90 con 0x7f4dd4109b00 2024-04-16T17:22:25.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.829+0000 7f4dd1600640 1 -- 172.21.15.18:0/1993771298 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0003de0 con 0x7f4dd4109b00 2024-04-16T17:22:25.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.829+0000 7f4ddb600640 1 -- 172.21.15.18:0/1993771298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd41bcb90 con 0x7f4dd4109b00 2024-04-16T17:22:25.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.829+0000 7f4ddb600640 1 -- 172.21.15.18:0/1993771298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd41bd060 con 0x7f4dd4109b00 2024-04-16T17:22:25.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.840+0000 7f4dde3e7740 1 Processor -- start 2024-04-16T17:22:25.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.840+0000 7f4dde3e7740 1 -- start start 2024-04-16T17:22:25.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.841+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f7e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.841+0000 7f4dd9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f7e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.841+0000 7f4dd9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f7e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33354/0 (socket says 172.21.15.18:33354) 2024-04-16T17:22:25.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.841+0000 7f4dd9600640 1 -- 172.21.15.18:0/1070578892 learned_addr learned my addr 172.21.15.18:0/1070578892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:25.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.841+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f87170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.841+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f89aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.842+0000 7f4dd3e00640 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f89aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.842+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1070578892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f899ee4120 con 0x55f899f7ead0 2024-04-16T17:22:25.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.842+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1070578892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f899e8a340 con 0x55f899f7e1b0 2024-04-16T17:22:25.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.842+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1070578892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f899ee00e0 con 0x55f899f876b0 2024-04-16T17:22:25.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.842+0000 7f4dd9600640 1 -- 172.21.15.18:0/1070578892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 msgr2=0x55f899f89aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:25.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.842+0000 7f4dd9600640 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f89aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.843+0000 7f4dd9600640 1 -- 172.21.15.18:0/1070578892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 msgr2=0x55f899f87170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:25.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.843+0000 7f4dd9600640 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f87170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.843+0000 7f4dd9600640 1 -- 172.21.15.18:0/1070578892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f899cc07d0 con 0x55f899f7e1b0 2024-04-16T17:22:25.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.843+0000 7f4dd9600640 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f7e590 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009d20 tx=0x7f4dcc00b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439c265a72c4b19c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:25.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.844+0000 7f4dd1600640 1 -- 172.21.15.18:0/1993771298 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4dc0045020 con 0x7f4dd4109b00 2024-04-16T17:22:25.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.844+0000 7f4dc5c00640 1 -- 172.21.15.18:0/1070578892 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc018070 con 0x55f899f7e1b0 2024-04-16T17:22:25.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.844+0000 7f4dc5c00640 1 -- 172.21.15.18:0/1070578892 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc011a00 con 0x55f899f7e1b0 2024-04-16T17:22:25.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.844+0000 7f4dc5c00640 1 -- 172.21.15.18:0/1070578892 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc013b60 con 0x55f899f7e1b0 2024-04-16T17:22:25.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.845+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1070578892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 msgr2=0x55f899f7e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:25.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.845+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f7e590 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009d20 tx=0x7f4dcc00b820 comp rx=0 tx=0).stop 2024-04-16T17:22:25.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.848+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1070578892 shutdown_connections 2024-04-16T17:22:25.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.848+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f89aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.848+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f87170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.848+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1070578892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f7e590 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.848+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1070578892 >> 172.21.15.18:0/1070578892 conn(0x55f899f788e0 msgr2=0x55f899f78cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:25.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.849+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1070578892 shutdown_connections 2024-04-16T17:22:25.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.849+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1070578892 wait complete. 2024-04-16T17:22:25.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.849+0000 7f4dd1600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4dbc0a3df0 0x7f4dbc0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.850+0000 7f4dde3e7740 1 -- start start 2024-04-16T17:22:25.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.850+0000 7f4dd1600640 1 -- 172.21.15.18:0/1993771298 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f4dc00f2b60 con 0x7f4dd4109b00 2024-04-16T17:22:25.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.850+0000 7f4dd9600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4dbc0a3df0 0x7f4dbc0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.850+0000 7f4dde3e7740 1 Processor -- start 2024-04-16T17:22:25.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.851+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7e1b0 0x55f899f84800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.851+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f84d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.851+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f876b0 0x55f899f85280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.851+0000 7f4dde3e7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f899f8a4a0 con 0x55f899f7ead0 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.851+0000 7f4dde3e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f899e9d710 con 0x55f899f876b0 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.851+0000 7f4dde3e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f899f8a620 con 0x55f899f7e1b0 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.851+0000 7f4dde3e7740 10 client.0 _ll_register_callbacks cb 0x55f899e98af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.852+0000 7f4dd8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7e1b0 0x55f899f84800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.852+0000 7f4dd8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7e1b0 0x55f899f84800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52010/0 (socket says 172.21.15.18:52010) 2024-04-16T17:22:25.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.852+0000 7f4dd8c00640 1 -- 172.21.15.18:0/1722265775 learned_addr learned my addr 172.21.15.18:0/1722265775 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.853+0000 7f4dd8c00640 1 -- 172.21.15.18:0/1722265775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f876b0 msgr2=0x55f899f85280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.853+0000 7f4dd8c00640 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f876b0 0x55f899f85280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.853+0000 7f4dd8c00640 1 -- 172.21.15.18:0/1722265775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 msgr2=0x55f899f84d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.853+0000 7f4dd8c00640 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f84d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.853+0000 7f4dd8c00640 1 -- 172.21.15.18:0/1722265775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f899bf63f0 con 0x55f899f7e1b0 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.853+0000 7f4dd8c00640 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7e1b0 0x55f899f84800 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00f0450 tx=0x7f4dc00be6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e67ea9f4f1993e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.853+0000 7f4dba000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.854+0000 7f4dba000640 1 -- 172.21.15.18:0/1722265775 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00333a0 con 0x55f899f7e1b0 2024-04-16T17:22:25.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.854+0000 7f4dba000640 1 -- 172.21.15.18:0/1722265775 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f4dc00bdce0 con 0x55f899f7e1b0 2024-04-16T17:22:25.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.854+0000 7f4dd9600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4dbc0a3df0 0x7f4dbc0a62b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00a010 tx=0x7f4dcc00bf90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:25.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.854+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1722265775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dcc011a00 con 0x55f899f7e1b0 2024-04-16T17:22:25.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.854+0000 7f4dba000640 1 -- 172.21.15.18:0/1722265775 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4dc00c2b60 con 0x55f899f7e1b0 2024-04-16T17:22:25.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.854+0000 7f4dde3e7740 10 client.30037 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:25.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.854+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1722265775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f899bb60f0 con 0x55f899f7e1b0 2024-04-16T17:22:25.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.855+0000 7f4dba000640 1 -- 172.21.15.18:0/1722265775 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc011a00 con 0x55f899f7e1b0 2024-04-16T17:22:25.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.855+0000 7f4dba000640 1 -- 172.21.15.18:0/1722265775 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f4dc00bd750 con 0x55f899f7e1b0 2024-04-16T17:22:25.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.855+0000 7f4dde3e7740 10 client.30037 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:25.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.855+0000 7f4dde3e7740 10 client.30037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:25.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.855+0000 7f4dde3e7740 20 client.30037 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:25.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.855+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f899f97ab0 0x55f899f99f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.855+0000 7f4dde3e7740 4 client.30037 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:25.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.855+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1722265775 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f899ef1b40 con 0x55f899f97ab0 2024-04-16T17:22:25.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.856+0000 7f4dd3e00640 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f899f97ab0 0x55f899f99f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.856+0000 7f4dd3e00640 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f899f97ab0 0x55f899f99f00 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2ea3e2035a7113 server_cookie=e709995bb243e612 in_seq=0 out_seq=0 2024-04-16T17:22:25.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.858+0000 7f4dba000640 10 client.30037 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:25.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.863+0000 7f4dba000640 1 -- 172.21.15.18:0/1722265775 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f899ef1b40 con 0x55f899f97ab0 2024-04-16T17:22:25.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.863+0000 7f4dba000640 10 client.30037 handle_command_reply: tid=0 2024-04-16T17:22:25.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.863+0000 7f4dde3e7740 1 client.30037 shutdown 2024-04-16T17:22:25.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1722265775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7e1b0 msgr2=0x55f899f84800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:25.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7e1b0 0x55f899f84800 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00f0450 tx=0x7f4dc00be6e0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1722265775 shutdown_connections 2024-04-16T17:22:25.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f899f97ab0 0x55f899f99f00 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f876b0 0x55f899f85280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f84d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/1722265775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7e1b0 0x55f899f84800 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1722265775 >> 172.21.15.18:0/1722265775 conn(0x55f899f788e0 msgr2=0x55f899f7a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:25.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.864+0000 7f4dba000640 0 client.30037 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:25.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.866+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1722265775 shutdown_connections 2024-04-16T17:22:25.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.867+0000 7f4dde3e7740 1 -- 172.21.15.18:0/1722265775 wait complete. 2024-04-16T17:22:25.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.867+0000 7f4dde3e7740 20 client.30037 trim_cache size 0 max 16384 2024-04-16T17:22:25.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.932+0000 7f4dde3e7740 1 Processor -- start 2024-04-16T17:22:25.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.932+0000 7f4dde3e7740 1 -- start start 2024-04-16T17:22:25.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.933+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.933+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7ead0 0x55f899f7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.933+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f876b0 0x55f899f850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.933+0000 7f4dde3e7740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41c9bf0 con 0x55f899f876b0 2024-04-16T17:22:25.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.933+0000 7f4dde3e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41c74c0 con 0x55f899f7e1b0 2024-04-16T17:22:25.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.933+0000 7f4dde3e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41c5330 con 0x55f899f7ead0 2024-04-16T17:22:25.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f93360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33368/0 (socket says 172.21.15.18:33368) 2024-04-16T17:22:25.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd8c00640 1 -- 172.21.15.18:0/3387735551 learned_addr learned my addr 172.21.15.18:0/3387735551 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:25.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd3e00640 1 --2- 172.21.15.18:0/3387735551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7ead0 0x55f899f7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd3e00640 1 -- 172.21.15.18:0/3387735551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 msgr2=0x55f899f93360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:25.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd3e00640 1 --2- 172.21.15.18:0/3387735551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f93360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd3e00640 1 -- 172.21.15.18:0/3387735551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f876b0 msgr2=0x55f899f850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:25.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd3e00640 1 --2- 172.21.15.18:0/3387735551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f876b0 0x55f899f850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.934+0000 7f4dd3e00640 1 -- 172.21.15.18:0/3387735551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f899f7e900 con 0x55f899f7ead0 2024-04-16T17:22:25.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.935+0000 7f4dd3e00640 1 --2- 172.21.15.18:0/3387735551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7ead0 0x55f899f7a420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55f899f84060 tx=0x7f4dc8009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f950b46c5dbefc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:25.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.936+0000 7f4dc5200640 1 -- 172.21.15.18:0/3387735551 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8015a60 con 0x55f899f7ead0 2024-04-16T17:22:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.936+0000 7f4dc5200640 1 -- 172.21.15.18:0/3387735551 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc801d070 con 0x55f899f7ead0 2024-04-16T17:22:25.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.936+0000 7f4dc5200640 1 -- 172.21.15.18:0/3387735551 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8022760 con 0x55f899f7ead0 2024-04-16T17:22:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.941+0000 7f4dde3e7740 1 -- 172.21.15.18:0/3387735551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7ead0 msgr2=0x55f899f7a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.941+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/3387735551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7ead0 0x55f899f7a420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55f899f84060 tx=0x7f4dc8009e40 comp rx=0 tx=0).stop 2024-04-16T17:22:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.942+0000 7f4dde3e7740 1 -- 172.21.15.18:0/3387735551 shutdown_connections 2024-04-16T17:22:25.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.942+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/3387735551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f876b0 0x55f899f850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.942+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/3387735551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f7ead0 0x55f899f7a420 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.942+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/3387735551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.942+0000 7f4dde3e7740 1 -- 172.21.15.18:0/3387735551 >> 172.21.15.18:0/3387735551 conn(0x55f899f788e0 msgr2=0x55f899f79d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.943+0000 7f4dde3e7740 1 -- 172.21.15.18:0/3387735551 shutdown_connections 2024-04-16T17:22:25.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.943+0000 7f4dde3e7740 1 -- 172.21.15.18:0/3387735551 wait complete. 2024-04-16T17:22:25.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.944+0000 7f4dde3e7740 1 -- start start 2024-04-16T17:22:25.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.944+0000 7f4dde3e7740 1 Processor -- start 2024-04-16T17:22:25.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.946+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f8fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.946+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f81180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.946+0000 7f4dd8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f8fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.946+0000 7f4dd8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f8fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:33372/0 (socket says 172.21.15.18:33372) 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.946+0000 7f4dde3e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.947+0000 7f4dd9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.947+0000 7f4dd9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f816c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52016/0 (socket says 172.21.15.18:52016) 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.947+0000 7f4dd9600640 1 -- 172.21.15.18:0/457982842 learned_addr learned my addr 172.21.15.18:0/457982842 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.947+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41c5980 con 0x55f899f7ead0 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.947+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41c6df0 con 0x55f899f7e1b0 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.947+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41c6050 con 0x55f899f876b0 2024-04-16T17:22:25.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.947+0000 7f4dd3e00640 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f81180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.947+0000 7f4dde3e7740 10 client.0 _ll_register_callbacks cb 0x7f4dc00bac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.948+0000 7f4dd9600640 1 -- 172.21.15.18:0/457982842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 msgr2=0x55f899f8fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.948+0000 7f4dd9600640 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f8fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.948+0000 7f4dd9600640 1 -- 172.21.15.18:0/457982842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 msgr2=0x55f899f81180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.948+0000 7f4dd9600640 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f81180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.948+0000 7f4dd9600640 1 -- 172.21.15.18:0/457982842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f899f81ef0 con 0x55f899f876b0 2024-04-16T17:22:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.949+0000 7f4dba000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:25.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.949+0000 7f4dd9600640 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f816c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009ab0 tx=0x7f4dcc0136c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da18488b782caa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:25.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.950+0000 7f4dba000640 1 -- 172.21.15.18:0/457982842 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc02e060 con 0x55f899f876b0 2024-04-16T17:22:25.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.950+0000 7f4dba000640 1 -- 172.21.15.18:0/457982842 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f4dcc010050 con 0x55f899f876b0 2024-04-16T17:22:25.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.951+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc801d070 con 0x55f899f876b0 2024-04-16T17:22:25.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.951+0000 7f4dba000640 1 -- 172.21.15.18:0/457982842 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4dcc018040 con 0x55f899f876b0 2024-04-16T17:22:25.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.951+0000 7f4dde3e7740 10 client.30043 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:25.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.951+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f899ffe060 con 0x55f899f876b0 2024-04-16T17:22:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.952+0000 7f4dba000640 1 -- 172.21.15.18:0/457982842 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc801d070 con 0x55f899f876b0 2024-04-16T17:22:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.954+0000 7f4dba000640 1 -- 172.21.15.18:0/457982842 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f4dcc008030 con 0x55f899f876b0 2024-04-16T17:22:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.954+0000 7f4dde3e7740 10 client.30043 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.954+0000 7f4dde3e7740 10 client.30043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.954+0000 7f4dde3e7740 20 client.30043 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.954+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f899ffe590 0x55f89a000980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.954+0000 7f4dde3e7740 4 client.30043 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:22:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.954+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f899eeb9b0 con 0x55f899ffe590 2024-04-16T17:22:25.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.956+0000 7f4dd8c00640 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f899ffe590 0x55f89a000980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:25.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.959+0000 7f4dd8c00640 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f899ffe590 0x55f89a000980 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bcda328c3f0899e server_cookie=7d19e9f56c380a08 in_seq=0 out_seq=0 2024-04-16T17:22:25.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.959+0000 7f4dba000640 10 client.30043 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:25.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dba000640 1 -- 172.21.15.18:0/457982842 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f899eeb9b0 con 0x55f899ffe590 2024-04-16T17:22:25.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dba000640 10 client.30043 handle_command_reply: tid=0 2024-04-16T17:22:25.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 client.30043 shutdown 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 msgr2=0x55f899f816c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f816c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009ab0 tx=0x7f4dcc0136c0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 shutdown_connections 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f899ffe590 0x55f89a000980 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f899f876b0 0x55f899f816c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f899f7ead0 0x55f899f81180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 --2- 172.21.15.18:0/457982842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f899f7e1b0 0x55f899f8fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.968+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 >> 172.21.15.18:0/457982842 conn(0x55f899f788e0 msgr2=0x55f899f7a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:25.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.969+0000 7f4dba000640 0 client.30043 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:25.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.969+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 shutdown_connections 2024-04-16T17:22:25.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.969+0000 7f4dde3e7740 1 -- 172.21.15.18:0/457982842 wait complete. 2024-04-16T17:22:25.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.970+0000 7f4dde3e7740 20 client.30043 trim_cache size 0 max 16384 2024-04-16T17:22:25.975 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:22:25.975 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:22:25.975 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:22:25.975 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:22:25.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 -- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4dbc0a3df0 msgr2=0x7f4dbc0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:25.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4dbc0a3df0 0x7f4dbc0a62b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00a010 tx=0x7f4dcc00bf90 comp rx=0 tx=0).stop 2024-04-16T17:22:25.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 -- 172.21.15.18:0/1993771298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd4109b00 msgr2=0x7f4dd41149e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:25.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd4109b00 0x7f4dd41149e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4dc002c210 tx=0x7f4dc00045e0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 -- 172.21.15.18:0/1993771298 shutdown_connections 2024-04-16T17:22:25.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4dbc0a3df0 0x7f4dbc0a62b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd410ef90 0x7f4dd410f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd4115150 0x7f4dd410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 --2- 172.21.15.18:0/1993771298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd4109b00 0x7f4dd41149e0 secure :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4dc002c210 tx=0x7f4dc00045e0 comp rx=0 tx=0).stop 2024-04-16T17:22:25.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.978+0000 7f4ddb600640 1 -- 172.21.15.18:0/1993771298 >> 172.21.15.18:0/1993771298 conn(0x7f4dd406d0a0 msgr2=0x7f4dd4072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:25.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.980+0000 7f4ddb600640 1 -- 172.21.15.18:0/1993771298 shutdown_connections 2024-04-16T17:22:25.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:25.982+0000 7f4ddb600640 1 -- 172.21.15.18:0/1993771298 wait complete. 2024-04-16T17:22:26.030 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:22:26.030 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:22:26.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:26 smithi022 ceph-mon[36023]: pgmap v563: 129 pgs: 6 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 117 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2024-04-16T17:22:26.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 -- 172.21.15.18:0/3534155407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d400743d0 msgr2=0x7f3d400747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:26.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 --2- 172.21.15.18:0/3534155407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d400743d0 0x7f3d400747b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009ca0 tx=0x7f3d3802c1b0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 -- 172.21.15.18:0/3534155407 shutdown_connections 2024-04-16T17:22:26.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 --2- 172.21.15.18:0/3534155407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d4010e870 0x7f3d40110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 --2- 172.21.15.18:0/3534155407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d40074cf0 0x7f3d4010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 --2- 172.21.15.18:0/3534155407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d400743d0 0x7f3d400747b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 -- 172.21.15.18:0/3534155407 >> 172.21.15.18:0/3534155407 conn(0x7f3d400fdae0 msgr2=0x7f3d400fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:26.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 -- 172.21.15.18:0/3534155407 shutdown_connections 2024-04-16T17:22:26.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.445+0000 7f3d45e00640 1 -- 172.21.15.18:0/3534155407 wait complete. 2024-04-16T17:22:26.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.449+0000 7f3d45e00640 1 Processor -- start 2024-04-16T17:22:26.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.449+0000 7f3d45e00640 1 -- start start 2024-04-16T17:22:26.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.449+0000 7f3d45e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d400743d0 0x7f3d401a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:26.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d45e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d40074cf0 0x7f3d401a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:26.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d45e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4010e870 0x7f3d4019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:26.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d45e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40114700 con 0x7f3d400743d0 2024-04-16T17:22:26.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d45e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40114580 con 0x7f3d4010e870 2024-04-16T17:22:26.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d45e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40114880 con 0x7f3d40074cf0 2024-04-16T17:22:26.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d3ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d400743d0 0x7f3d401a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d3ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d400743d0 0x7f3d401a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58356/0 (socket says 172.21.15.18:58356) 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d3ea00640 1 -- 172.21.15.18:0/4136676111 learned_addr learned my addr 172.21.15.18:0/4136676111 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d3e000640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d40074cf0 0x7f3d401a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.450+0000 7f3d3f400640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4010e870 0x7f3d4019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.451+0000 7f3d3e000640 1 -- 172.21.15.18:0/4136676111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4010e870 msgr2=0x7f3d4019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.451+0000 7f3d3e000640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4010e870 0x7f3d4019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.451+0000 7f3d3e000640 1 -- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d400743d0 msgr2=0x7f3d401a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.451+0000 7f3d3e000640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d400743d0 0x7f3d401a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.451+0000 7f3d3e000640 1 -- 172.21.15.18:0/4136676111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4019b220 con 0x7f3d40074cf0 2024-04-16T17:22:26.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.451+0000 7f3d3e000640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d40074cf0 0x7f3d401a0e30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000b3f0 tx=0x7f3d3000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:26.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.452+0000 7f3d44a00640 1 -- 172.21.15.18:0/4136676111 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30015020 con 0x7f3d40074cf0 2024-04-16T17:22:26.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.452+0000 7f3d44a00640 1 -- 172.21.15.18:0/4136676111 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d30011d10 con 0x7f3d40074cf0 2024-04-16T17:22:26.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.452+0000 7f3d44a00640 1 -- 172.21.15.18:0/4136676111 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3000f2e0 con 0x7f3d40074cf0 2024-04-16T17:22:26.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.452+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4019b510 con 0x7f3d40074cf0 2024-04-16T17:22:26.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.454+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d40101ec0 con 0x7f3d40074cf0 2024-04-16T17:22:26.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.457+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d40075a80 con 0x7f3d40074cf0 2024-04-16T17:22:26.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.459+0000 7f3d44a00640 1 -- 172.21.15.18:0/4136676111 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3d30018020 con 0x7f3d40074cf0 2024-04-16T17:22:26.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.462+0000 7f3d44a00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3d240a3de0 0x7f3d240a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:26.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.462+0000 7f3d44a00640 1 -- 172.21.15.18:0/4136676111 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f3d300d01b0 con 0x7f3d40074cf0 2024-04-16T17:22:26.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.464+0000 7f3d3ea00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3d240a3de0 0x7f3d240a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:26.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.466+0000 7f3d44a00640 1 -- 172.21.15.18:0/4136676111 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d3009c500 con 0x7f3d40074cf0 2024-04-16T17:22:26.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.468+0000 7f3d3ea00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3d240a3de0 0x7f3d240a62a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3d4019bbe0 tx=0x7f3d3802c6d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:26.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:26 smithi043 ceph-mon[36901]: pgmap v563: 129 pgs: 6 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 117 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2024-04-16T17:22:26.520 INFO:tasks.workunit.client.0.smithi018.stdout: 10 67631 12.16 MB/sec execute 158 sec latency 176.383 ms 2024-04-16T17:22:26.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.991+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d40069300 con 0x7f3d40074cf0 2024-04-16T17:22:26.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.992+0000 7f3d44a00640 1 -- 172.21.15.18:0/4136676111 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7f3d3009be00 con 0x7f3d40074cf0 2024-04-16T17:22:26.992 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:22:26.992 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:22:26.993 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:22:26.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3d240a3de0 msgr2=0x7f3d240a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3d240a3de0 0x7f3d240a62a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3d4019bbe0 tx=0x7f3d3802c6d0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d40074cf0 msgr2=0x7f3d401a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d40074cf0 0x7f3d401a0e30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000b3f0 tx=0x7f3d3000b8b0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 shutdown_connections 2024-04-16T17:22:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3d240a3de0 0x7f3d240a62a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d4010e870 0x7f3d4019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d40074cf0 0x7f3d401a0e30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 --2- 172.21.15.18:0/4136676111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d400743d0 0x7f3d401a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:26.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 >> 172.21.15.18:0/4136676111 conn(0x7f3d400fdae0 msgr2=0x7f3d400ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:26.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.995+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 shutdown_connections 2024-04-16T17:22:26.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:26.996+0000 7f3d45e00640 1 -- 172.21.15.18:0/4136676111 wait complete. 2024-04-16T17:22:27.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:22:27.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.286+0000 7f4c2fe00640 1 -- 172.21.15.18:0/237924502 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280337a0 con 0x7f4c30104450 2024-04-16T17:22:27.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.287+0000 7f4c37e00640 1 -- 172.21.15.18:0/237924502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 msgr2=0x7f4c30104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.287+0000 7f4c37e00640 1 --2- 172.21.15.18:0/237924502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 0x7f4c30104830 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009be0 tx=0x7f4c2802bf10 comp rx=0 tx=0).stop 2024-04-16T17:22:27.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.288+0000 7f4c37e00640 1 -- 172.21.15.18:0/237924502 shutdown_connections 2024-04-16T17:22:27.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.288+0000 7f4c37e00640 1 --2- 172.21.15.18:0/237924502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30109520 0x7f4c301110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.288+0000 7f4c37e00640 1 --2- 172.21.15.18:0/237924502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c30104e00 0x7f4c30108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.288+0000 7f4c37e00640 1 --2- 172.21.15.18:0/237924502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 0x7f4c30104830 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.288+0000 7f4c37e00640 1 -- 172.21.15.18:0/237924502 >> 172.21.15.18:0/237924502 conn(0x7f4c300ffc90 msgr2=0x7f4c301020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.288+0000 7f4c37e00640 1 -- 172.21.15.18:0/237924502 shutdown_connections 2024-04-16T17:22:27.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.288+0000 7f4c37e00640 1 -- 172.21.15.18:0/237924502 wait complete. 2024-04-16T17:22:27.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.289+0000 7f4c37e00640 1 Processor -- start 2024-04-16T17:22:27.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.290+0000 7f4c37e00640 1 -- start start 2024-04-16T17:22:27.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.290+0000 7f4c37e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 0x7f4c30076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.290+0000 7f4c37e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c30109520 0x7f4c30076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.290+0000 7f4c37e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30076f90 0x7f4c301ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.290+0000 7f4c37e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c30114630 con 0x7f4c30109520 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.290+0000 7f4c37e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301144b0 con 0x7f4c30076f90 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.290+0000 7f4c37e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301147b0 con 0x7f4c30104450 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c34a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c30109520 0x7f4c30076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c34a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c30109520 0x7f4c30076a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58772/0 (socket says 172.21.15.18:58772) 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c34a00640 1 -- 172.21.15.18:0/3131398152 learned_addr learned my addr 172.21.15.18:0/3131398152 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c35400640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 0x7f4c30076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c34a00640 1 -- 172.21.15.18:0/3131398152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 msgr2=0x7f4c30076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c34a00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 0x7f4c30076510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c34a00640 1 -- 172.21.15.18:0/3131398152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30076f90 msgr2=0x7f4c301ab160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:22:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c34a00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30076f90 0x7f4c301ab160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.291+0000 7f4c34a00640 1 -- 172.21.15.18:0/3131398152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c301ab6a0 con 0x7f4c30109520 2024-04-16T17:22:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.292+0000 7f4c35400640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 0x7f4c30076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:22:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.292+0000 7f4c34a00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c30109520 0x7f4c30076a50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000d770 tx=0x7f4c2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.292+0000 7f4c2e000640 1 -- 172.21.15.18:0/3131398152 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20017070 con 0x7f4c30109520 2024-04-16T17:22:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.292+0000 7f4c2e000640 1 -- 172.21.15.18:0/3131398152 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c20009c30 con 0x7f4c30109520 2024-04-16T17:22:27.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.292+0000 7f4c2e000640 1 -- 172.21.15.18:0/3131398152 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000f620 con 0x7f4c30109520 2024-04-16T17:22:27.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.294+0000 7f4c37e00640 1 -- 172.21.15.18:0/3131398152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c301ab930 con 0x7f4c30109520 2024-04-16T17:22:27.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.294+0000 7f4c37e00640 1 -- 172.21.15.18:0/3131398152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c301abe00 con 0x7f4c30109520 2024-04-16T17:22:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.302+0000 7f4c2e000640 1 -- 172.21.15.18:0/3131398152 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c2001a070 con 0x7f4c30109520 2024-04-16T17:22:27.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.302+0000 7f4c3ac03740 1 Processor -- start 2024-04-16T17:22:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.305+0000 7f4c3ac03740 1 -- start start 2024-04-16T17:22:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.305+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d91d50 0x55f2c7d92130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.305+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d9ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.305+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d9b250 0x55f2c7d9d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.305+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f2c7cf7020 con 0x55f2c7d91d50 2024-04-16T17:22:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.305+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f2c7c9d460 con 0x55f2c7d9b250 2024-04-16T17:22:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f2c7cf2fe0 con 0x55f2c7d92670 2024-04-16T17:22:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c35400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d9ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c35400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d9ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39642/0 (socket says 172.21.15.18:39642) 2024-04-16T17:22:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c35400640 1 -- 172.21.15.18:0/2752881709 learned_addr learned my addr 172.21.15.18:0/2752881709 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c35400640 1 -- 172.21.15.18:0/2752881709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d9b250 msgr2=0x55f2c7d9d640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c35400640 1 --2- 172.21.15.18:0/2752881709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d9b250 0x55f2c7d9d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c35400640 1 -- 172.21.15.18:0/2752881709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d91d50 msgr2=0x55f2c7d92130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c35400640 1 --2- 172.21.15.18:0/2752881709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d91d50 0x55f2c7d92130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.306+0000 7f4c35400640 1 -- 172.21.15.18:0/2752881709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c7ca5040 con 0x55f2c7d92670 2024-04-16T17:22:27.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.310+0000 7f4c35400640 1 --2- 172.21.15.18:0/2752881709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d9ad10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4c28036040 tx=0x7f4c28005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da69e389520e92c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.310+0000 7f4c1e600640 1 -- 172.21.15.18:0/2752881709 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28043020 con 0x55f2c7d92670 2024-04-16T17:22:27.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.310+0000 7f4c1e600640 1 -- 172.21.15.18:0/2752881709 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c28033c70 con 0x55f2c7d92670 2024-04-16T17:22:27.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.310+0000 7f4c1e600640 1 -- 172.21.15.18:0/2752881709 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2803e5d0 con 0x55f2c7d92670 2024-04-16T17:22:27.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.311+0000 7f4c3ac03740 1 -- 172.21.15.18:0/2752881709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 msgr2=0x55f2c7d9ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.311+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/2752881709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d9ad10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4c28036040 tx=0x7f4c28005ec0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.312+0000 7f4c3ac03740 1 -- 172.21.15.18:0/2752881709 shutdown_connections 2024-04-16T17:22:27.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.312+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/2752881709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d9b250 0x55f2c7d9d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.312+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/2752881709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d9ad10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.312+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/2752881709 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d91d50 0x55f2c7d92130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.312+0000 7f4c3ac03740 1 -- 172.21.15.18:0/2752881709 >> 172.21.15.18:0/2752881709 conn(0x55f2c7d8c480 msgr2=0x55f2c7d8c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.314+0000 7f4c2e000640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c180a3e20 0x7f4c180a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.314+0000 7f4c2e000640 1 -- 172.21.15.18:0/3131398152 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f4c200d0840 con 0x7f4c30109520 2024-04-16T17:22:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.314+0000 7f4c35400640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c180a3e20 0x7f4c180a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.314+0000 7f4c3ac03740 1 -- 172.21.15.18:0/2752881709 shutdown_connections 2024-04-16T17:22:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.315+0000 7f4c3ac03740 1 -- 172.21.15.18:0/2752881709 wait complete. 2024-04-16T17:22:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.315+0000 7f4c3ac03740 1 -- start start 2024-04-16T17:22:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.316+0000 7f4c3ac03740 1 Processor -- start 2024-04-16T17:22:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.317+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d98300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.317+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d98840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.317+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 0x55f2c7d98d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.317+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f2c7daeff0 con 0x55f2c7d9b250 2024-04-16T17:22:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.317+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f2c7cb0830 con 0x55f2c7d91d50 2024-04-16T17:22:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.317+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f2c7daf170 con 0x55f2c7d92670 2024-04-16T17:22:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.317+0000 7f4c3ac03740 10 client.0 _ll_register_callbacks cb 0x55f2c7cb8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.318+0000 7f4c35400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d98300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.318+0000 7f4c35400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d98300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36794/0 (socket says 172.21.15.18:36794) 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.318+0000 7f4c35400640 1 -- 172.21.15.18:0/1345387429 learned_addr learned my addr 172.21.15.18:0/1345387429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.319+0000 7f4c35400640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c180a3e20 0x7f4c180a62e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4c28005cc0 tx=0x7f4c2802bf10 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.319+0000 7f4c35400640 1 -- 172.21.15.18:0/1345387429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 msgr2=0x55f2c7d98840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.319+0000 7f4c35400640 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d98840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.319+0000 7f4c35400640 1 -- 172.21.15.18:0/1345387429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 msgr2=0x55f2c7d98d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.319+0000 7f4c35400640 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 0x55f2c7d98d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.319+0000 7f4c35400640 1 -- 172.21.15.18:0/1345387429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c7cb4920 con 0x55f2c7d91d50 2024-04-16T17:22:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.319+0000 7f4c35400640 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d98300 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x55f2c7d91820 tx=0x7f4c28005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fce41425d059fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.320+0000 7f4c12a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:22:27.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.320+0000 7f4c12a00640 1 -- 172.21.15.18:0/1345387429 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2804e070 con 0x55f2c7d91d50 2024-04-16T17:22:27.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.320+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1345387429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c28033c70 con 0x55f2c7d91d50 2024-04-16T17:22:27.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.320+0000 7f4c12a00640 1 -- 172.21.15.18:0/1345387429 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f4c28054650 con 0x55f2c7d91d50 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.321+0000 7f4c12a00640 1 -- 172.21.15.18:0/1345387429 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4c28054850 con 0x55f2c7d91d50 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.321+0000 7f4c3ac03740 10 client.30206 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.321+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1345387429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f2c7c827f0 con 0x55f2c7d91d50 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.321+0000 7f4c12a00640 1 -- 172.21.15.18:0/1345387429 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c28033c70 con 0x55f2c7d91d50 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.323+0000 7f4c12a00640 1 -- 172.21.15.18:0/1345387429 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f4c28048b20 con 0x55f2c7d91d50 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.323+0000 7f4c3ac03740 10 client.30206 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.323+0000 7f4c3ac03740 10 client.30206 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.323+0000 7f4c3ac03740 20 client.30206 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.323+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f2c7dbc4b0 0x55f2c7dbe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.323+0000 7f4c3ac03740 4 client.30206 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.323+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1345387429 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2c7d04a40 con 0x55f2c7dbc4b0 2024-04-16T17:22:27.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.324+0000 7f4c34a00640 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f2c7dbc4b0 0x55f2c7dbe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.327+0000 7f4c34a00640 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f2c7dbc4b0 0x55f2c7dbe8a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ca7a9aa744ba69b server_cookie=bf29a24790169217 in_seq=0 out_seq=0 2024-04-16T17:22:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.327+0000 7f4c12a00640 10 client.30206 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:22:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.327+0000 7f4c12a00640 1 -- 172.21.15.18:0/1345387429 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2c7d04a40 con 0x55f2c7dbc4b0 2024-04-16T17:22:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.327+0000 7f4c12a00640 10 client.30206 handle_command_reply: tid=0 2024-04-16T17:22:27.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.327+0000 7f4c3ac03740 1 client.30206 shutdown 2024-04-16T17:22:27.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1345387429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 msgr2=0x55f2c7d98300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d98300 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x55f2c7d91820 tx=0x7f4c28005ec0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1345387429 shutdown_connections 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f2c7dbc4b0 0x55f2c7dbe8a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 0x55f2c7d98d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d92670 0x55f2c7d98840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1345387429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d98300 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1345387429 >> 172.21.15.18:0/1345387429 conn(0x55f2c7d8c480 msgr2=0x55f2c7d8e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c12a00640 0 client.30206 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.328+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1345387429 shutdown_connections 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.329+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1345387429 wait complete. 2024-04-16T17:22:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.329+0000 7f4c3ac03740 20 client.30206 trim_cache size 0 max 16384 2024-04-16T17:22:27.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.364+0000 7f4c3ac03740 1 Processor -- start 2024-04-16T17:22:27.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.365+0000 7f4c3ac03740 1 -- start start 2024-04-16T17:22:27.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.365+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d9aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.365+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d92670 0x55f2c7d98ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.366+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d9b250 0x55f2c7dc3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.366+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301afea0 con 0x55f2c7d92670 2024-04-16T17:22:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.366+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301ad770 con 0x55f2c7d91d50 2024-04-16T17:22:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.366+0000 7f4c3ac03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301a2740 con 0x55f2c7d9b250 2024-04-16T17:22:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.366+0000 7f4c34a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d92670 0x55f2c7d98ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.367+0000 7f4c35e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d9b250 0x55f2c7dc3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.367+0000 7f4c34a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d92670 0x55f2c7d98ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58790/0 (socket says 172.21.15.18:58790) 2024-04-16T17:22:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.367+0000 7f4c34a00640 1 -- 172.21.15.18:0/929028403 learned_addr learned my addr 172.21.15.18:0/929028403 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.367+0000 7f4c35400640 1 --2- 172.21.15.18:0/929028403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d9aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.368+0000 7f4c34a00640 1 -- 172.21.15.18:0/929028403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d9b250 msgr2=0x55f2c7dc3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.368+0000 7f4c34a00640 1 --2- 172.21.15.18:0/929028403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d9b250 0x55f2c7dc3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.368+0000 7f4c34a00640 1 -- 172.21.15.18:0/929028403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 msgr2=0x55f2c7d9aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.368+0000 7f4c34a00640 1 --2- 172.21.15.18:0/929028403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d9aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.368+0000 7f4c34a00640 1 -- 172.21.15.18:0/929028403 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c7a093f0 con 0x55f2c7d92670 2024-04-16T17:22:27.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.369+0000 7f4c34a00640 1 --2- 172.21.15.18:0/929028403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d92670 0x55f2c7d98ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4c20098cf0 tx=0x7f4c2009bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5af15307c78744 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.371+0000 7f4c1dc00640 1 -- 172.21.15.18:0/929028403 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c200b0070 con 0x55f2c7d92670 2024-04-16T17:22:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.372+0000 7f4c1dc00640 1 -- 172.21.15.18:0/929028403 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c200a4f90 con 0x55f2c7d92670 2024-04-16T17:22:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.372+0000 7f4c1dc00640 1 -- 172.21.15.18:0/929028403 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c200b0070 con 0x55f2c7d92670 2024-04-16T17:22:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.372+0000 7f4c3ac03740 1 -- 172.21.15.18:0/929028403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d92670 msgr2=0x55f2c7d98ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.372+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/929028403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d92670 0x55f2c7d98ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4c20098cf0 tx=0x7f4c2009bf40 comp rx=0 tx=0).stop 2024-04-16T17:22:27.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.373+0000 7f4c3ac03740 1 -- 172.21.15.18:0/929028403 shutdown_connections 2024-04-16T17:22:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.373+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/929028403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d9b250 0x55f2c7dc3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.373+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/929028403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d92670 0x55f2c7d98ad0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.373+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/929028403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d91d50 0x55f2c7d9aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.374+0000 7f4c3ac03740 1 -- 172.21.15.18:0/929028403 >> 172.21.15.18:0/929028403 conn(0x55f2c7d8c480 msgr2=0x55f2c7d8d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.375+0000 7f4c3ac03740 1 -- 172.21.15.18:0/929028403 shutdown_connections 2024-04-16T17:22:27.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.376+0000 7f4c3ac03740 1 -- 172.21.15.18:0/929028403 wait complete. 2024-04-16T17:22:27.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.376+0000 7f4c3ac03740 1 -- start start 2024-04-16T17:22:27.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.376+0000 7f4c3ac03740 1 Processor -- start 2024-04-16T17:22:27.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.377+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d91d50 0x55f2c7db4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.377+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d92670 0x55f2c7db1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.377+0000 7f4c3ac03740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 0x55f2c7db1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.378+0000 7f4c35400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d91d50 0x55f2c7db4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.378+0000 7f4c34a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d92670 0x55f2c7db1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.378+0000 7f4c35e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 0x55f2c7db1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.378+0000 7f4c35400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d91d50 0x55f2c7db4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39656/0 (socket says 172.21.15.18:39656) 2024-04-16T17:22:27.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.378+0000 7f4c35400640 1 -- 172.21.15.18:0/1896893391 learned_addr learned my addr 172.21.15.18:0/1896893391 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.378+0000 7f4c34a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d92670 0x55f2c7db1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36812/0 (socket says 172.21.15.18:36812) 2024-04-16T17:22:27.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301a2d00 con 0x55f2c7d9b250 2024-04-16T17:22:27.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301ad0a0 con 0x55f2c7d92670 2024-04-16T17:22:27.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301ac300 con 0x55f2c7d91d50 2024-04-16T17:22:27.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c3ac03740 10 client.0 _ll_register_callbacks cb 0x7f4c28009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:27.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c35400640 1 -- 172.21.15.18:0/1896893391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d92670 msgr2=0x55f2c7db1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c35400640 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d92670 0x55f2c7db1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c35400640 1 -- 172.21.15.18:0/1896893391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 msgr2=0x55f2c7db1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c35400640 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 0x55f2c7db1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.379+0000 7f4c35400640 1 -- 172.21.15.18:0/1896893391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c7ad37d0 con 0x55f2c7d91d50 2024-04-16T17:22:27.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.380+0000 7f4c35400640 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d91d50 0x55f2c7db4710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4c28048e90 tx=0x7f4c28009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f68bec18de479e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.380+0000 7f4c12a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:27.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.381+0000 7f4c12a00640 1 -- 172.21.15.18:0/1896893391 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28034ba0 con 0x55f2c7d91d50 2024-04-16T17:22:27.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.382+0000 7f4c12a00640 1 -- 172.21.15.18:0/1896893391 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f4c28034d40 con 0x55f2c7d91d50 2024-04-16T17:22:27.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.382+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c200a4f90 con 0x55f2c7d91d50 2024-04-16T17:22:27.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.382+0000 7f4c12a00640 1 -- 172.21.15.18:0/1896893391 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4c28033c70 con 0x55f2c7d91d50 2024-04-16T17:22:27.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.382+0000 7f4c3ac03740 10 client.30061 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:27.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.382+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f2c7e22520 con 0x55f2c7d91d50 2024-04-16T17:22:27.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.383+0000 7f4c12a00640 1 -- 172.21.15.18:0/1896893391 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c200a4f90 con 0x55f2c7d91d50 2024-04-16T17:22:27.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.384+0000 7f4c12a00640 1 -- 172.21.15.18:0/1896893391 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f4c2805d400 con 0x55f2c7d91d50 2024-04-16T17:22:27.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.384+0000 7f4c3ac03740 10 client.30061 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:27.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.384+0000 7f4c3ac03740 10 client.30061 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:22:27.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.384+0000 7f4c3ac03740 20 client.30061 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:27.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.385+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f2c7e229a0 0x55f2c7e24d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.385+0000 7f4c3ac03740 4 client.30061 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:22:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.385+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2c7cfe8b0 con 0x55f2c7e229a0 2024-04-16T17:22:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.385+0000 7f4c34a00640 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f2c7e229a0 0x55f2c7e24d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.390+0000 7f4c34a00640 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f2c7e229a0 0x55f2c7e24d90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4c716f5e24fefdb server_cookie=d81b2bc95f4632e8 in_seq=0 out_seq=0 2024-04-16T17:22:27.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.390+0000 7f4c12a00640 10 client.30061 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:22:27.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.390+0000 7f4c12a00640 1 -- 172.21.15.18:0/1896893391 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2c7cfe8b0 con 0x55f2c7e229a0 2024-04-16T17:22:27.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.390+0000 7f4c12a00640 10 client.30061 handle_command_reply: tid=0 2024-04-16T17:22:27.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.390+0000 7f4c3ac03740 1 client.30061 shutdown 2024-04-16T17:22:27.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.391+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d91d50 msgr2=0x55f2c7db4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.391+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d91d50 0x55f2c7db4710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4c28048e90 tx=0x7f4c28009280 comp rx=0 tx=0).stop 2024-04-16T17:22:27.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 shutdown_connections 2024-04-16T17:22:27.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f2c7e229a0 0x55f2c7e24d90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f2c7d9b250 0x55f2c7db1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2c7d92670 0x55f2c7db1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 1 --2- 172.21.15.18:0/1896893391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2c7d91d50 0x55f2c7db4710 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 >> 172.21.15.18:0/1896893391 conn(0x55f2c7d8c480 msgr2=0x55f2c7d8dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c12a00640 0 client.30061 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:22:27.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 shutdown_connections 2024-04-16T17:22:27.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 1 -- 172.21.15.18:0/1896893391 wait complete. 2024-04-16T17:22:27.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.393+0000 7f4c3ac03740 20 client.30061 trim_cache size 0 max 16384 2024-04-16T17:22:27.395 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:22:27.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 -- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c180a3e20 msgr2=0x7f4c180a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c180a3e20 0x7f4c180a62e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4c28005cc0 tx=0x7f4c2802bf10 comp rx=0 tx=0).stop 2024-04-16T17:22:27.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 -- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c30109520 msgr2=0x7f4c30076a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c30109520 0x7f4c30076a50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000d770 tx=0x7f4c2000dc30 comp rx=0 tx=0).stop 2024-04-16T17:22:27.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 -- 172.21.15.18:0/3131398152 shutdown_connections 2024-04-16T17:22:27.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4c180a3e20 0x7f4c180a62e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30076f90 0x7f4c301ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c30109520 0x7f4c30076a50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 --2- 172.21.15.18:0/3131398152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c30104450 0x7f4c30076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.400+0000 7f4c37e00640 1 -- 172.21.15.18:0/3131398152 >> 172.21.15.18:0/3131398152 conn(0x7f4c300ffc90 msgr2=0x7f4c30100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.403+0000 7f4c37e00640 1 -- 172.21.15.18:0/3131398152 shutdown_connections 2024-04-16T17:22:27.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.403+0000 7f4c37e00640 1 -- 172.21.15.18:0/3131398152 wait complete. 2024-04-16T17:22:27.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:27 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:22:27.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:27 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:22:27.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:27 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:22:27.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:27 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4136676111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:22:27.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:22:27.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:27 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:22:27.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:27 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:22:27.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:27 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:22:27.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:27 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4136676111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:22:27.520 INFO:tasks.workunit.client.0.smithi018.stdout: 10 67921 12.15 MB/sec execute 159 sec latency 278.163 ms 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 -- 172.21.15.18:0/2205090913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85ec0b7a70 msgr2=0x7f85ec0b9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 --2- 172.21.15.18:0/2205090913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85ec0b7a70 0x7f85ec0b9e60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f85f4067e70 tx=0x7f85f40a0a70 comp rx=0 tx=0).stop 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 -- 172.21.15.18:0/2205090913 shutdown_connections 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 --2- 172.21.15.18:0/2205090913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85ec0b7a70 0x7f85ec0b9e60 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 --2- 172.21.15.18:0/2205090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ec0a5700 0x7f85ec0b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 --2- 172.21.15.18:0/2205090913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85ec0a4de0 0x7f85ec0a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 -- 172.21.15.18:0/2205090913 >> 172.21.15.18:0/2205090913 conn(0x7f85ec01a640 msgr2=0x7f85ec01aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 -- 172.21.15.18:0/2205090913 shutdown_connections 2024-04-16T17:22:27.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.759+0000 7f85fa400640 1 -- 172.21.15.18:0/2205090913 wait complete. 2024-04-16T17:22:27.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.760+0000 7f85fa400640 1 Processor -- start 2024-04-16T17:22:27.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.761+0000 7f85fa400640 1 -- start start 2024-04-16T17:22:27.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.761+0000 7f85fa400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85ec0a4de0 0x7f85ec0b7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.761+0000 7f85fa400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85ec0a5700 0x7f85ec0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.761+0000 7f85fa400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ec0b7a70 0x7f85ec0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85ec0a4de0 0x7f85ec0b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ec0b7a70 0x7f85ec0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ec0b7a70 0x7f85ec0b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39682/0 (socket says 172.21.15.18:39682) 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9a00640 1 -- 172.21.15.18:0/3119488765 learned_addr learned my addr 172.21.15.18:0/3119488765 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0af670 con 0x7f85ec0a5700 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0af4f0 con 0x7f85ec0a4de0 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0af7f0 con 0x7f85ec0b7a70 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9a00640 1 -- 172.21.15.18:0/3119488765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85ec0a4de0 msgr2=0x7f85ec0b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9a00640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85ec0a4de0 0x7f85ec0b7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9a00640 1 -- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85ec0a5700 msgr2=0x7f85ec0b30a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9a00640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85ec0a5700 0x7f85ec0b30a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.762+0000 7f85f9a00640 1 -- 172.21.15.18:0/3119488765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec0b3b20 con 0x7f85ec0b7a70 2024-04-16T17:22:27.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.763+0000 7f85f9a00640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ec0b7a70 0x7f85ec0b35e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f85f4065520 tx=0x7f85f40a2860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.763+0000 7f85f1600640 1 -- 172.21.15.18:0/3119488765 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f40a7050 con 0x7f85ec0b7a70 2024-04-16T17:22:27.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.763+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec0b3db0 con 0x7f85ec0b7a70 2024-04-16T17:22:27.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.763+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec147da0 con 0x7f85ec0b7a70 2024-04-16T17:22:27.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.763+0000 7f85f1600640 1 -- 172.21.15.18:0/3119488765 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f40a2420 con 0x7f85ec0b7a70 2024-04-16T17:22:27.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.763+0000 7f85f1600640 1 -- 172.21.15.18:0/3119488765 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f40a7050 con 0x7f85ec0b7a70 2024-04-16T17:22:27.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.771+0000 7f85fe7d0740 1 Processor -- start 2024-04-16T17:22:27.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.771+0000 7f85f1600640 1 -- 172.21.15.18:0/3119488765 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f85f40aa070 con 0x7f85ec0b7a70 2024-04-16T17:22:27.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.772+0000 7f85fe7d0740 1 -- start start 2024-04-16T17:22:27.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.772+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d16232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.772+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1623830 0x5619d162bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.772+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d162c410 0x5619d162e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.772+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5619d1588020 con 0x5619d162c410 2024-04-16T17:22:27.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.772+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5619d152e460 con 0x5619d1623830 2024-04-16T17:22:27.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.772+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5619d1583fe0 con 0x5619d1622f10 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.773+0000 7f85f1600640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f85dc0a3fa0 0x7f85dc0a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.774+0000 7f85f1600640 1 -- 172.21.15.18:0/3119488765 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f85f4166de0 con 0x7f85ec0b7a70 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.774+0000 7f85f9a00640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f85dc0a3fa0 0x7f85dc0a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.774+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d16232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.774+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d16232f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39692/0 (socket says 172.21.15.18:39692) 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.774+0000 7f85f3e00640 1 -- 172.21.15.18:0/1689077520 learned_addr learned my addr 172.21.15.18:0/1689077520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.775+0000 7f85f9000640 1 --2- 172.21.15.18:0/1689077520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1623830 0x5619d162bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.775+0000 7f85f3e00640 1 -- 172.21.15.18:0/1689077520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1623830 msgr2=0x5619d162bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.775+0000 7f85f3e00640 1 --2- 172.21.15.18:0/1689077520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1623830 0x5619d162bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.775+0000 7f85f3e00640 1 -- 172.21.15.18:0/1689077520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d162c410 msgr2=0x5619d162e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.775+0000 7f85f3e00640 1 --2- 172.21.15.18:0/1689077520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d162c410 0x5619d162e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.775+0000 7f85f3e00640 1 -- 172.21.15.18:0/1689077520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d1536040 con 0x5619d1622f10 2024-04-16T17:22:27.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.775+0000 7f85f9a00640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f85dc0a3fa0 0x7f85dc0a6460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f85f402dbb0 tx=0x7f85f4153040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.776+0000 7f85f3e00640 1 --2- 172.21.15.18:0/1689077520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d16232f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009c40 tx=0x7f85e80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c79f14321d3a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.776+0000 7f85e6000640 1 -- 172.21.15.18:0/1689077520 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8005020 con 0x5619d1622f10 2024-04-16T17:22:27.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.776+0000 7f85e6000640 1 -- 172.21.15.18:0/1689077520 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e8015070 con 0x5619d1622f10 2024-04-16T17:22:27.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.776+0000 7f85e6000640 1 -- 172.21.15.18:0/1689077520 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e80116a0 con 0x5619d1622f10 2024-04-16T17:22:27.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.777+0000 7f85fe7d0740 1 -- 172.21.15.18:0/1689077520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 msgr2=0x5619d16232f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.777+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/1689077520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d16232f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009c40 tx=0x7f85e80107a0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.778+0000 7f85fe7d0740 1 -- 172.21.15.18:0/1689077520 shutdown_connections 2024-04-16T17:22:27.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.778+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/1689077520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d162c410 0x5619d162e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.778+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/1689077520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1623830 0x5619d162bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.778+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/1689077520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d16232f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.778+0000 7f85fe7d0740 1 -- 172.21.15.18:0/1689077520 >> 172.21.15.18:0/1689077520 conn(0x5619d161d640 msgr2=0x5619d161da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.778+0000 7f85fe7d0740 1 -- 172.21.15.18:0/1689077520 shutdown_connections 2024-04-16T17:22:27.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.778+0000 7f85fe7d0740 1 -- 172.21.15.18:0/1689077520 wait complete. 2024-04-16T17:22:27.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.779+0000 7f85fe7d0740 1 -- start start 2024-04-16T17:22:27.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.779+0000 7f85fe7d0740 1 Processor -- start 2024-04-16T17:22:27.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.780+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d1629450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.780+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1629990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.780+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 0x5619d1629ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.780+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5619d162f200 con 0x5619d1623830 2024-04-16T17:22:27.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.780+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5619d1541830 con 0x5619d162c410 2024-04-16T17:22:27.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.780+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5619d162f380 con 0x5619d1622f10 2024-04-16T17:22:27.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.780+0000 7f85fe7d0740 10 client.0 _ll_register_callbacks cb 0x5619d1549150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:27.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.781+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d1629450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.781+0000 7f85f9000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1629990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.781+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d1629450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39704/0 (socket says 172.21.15.18:39704) 2024-04-16T17:22:27.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.781+0000 7f85f3e00640 1 -- 172.21.15.18:0/526044146 learned_addr learned my addr 172.21.15.18:0/526044146 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.781+0000 7f85f3e00640 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 0x5619d1629ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.782+0000 7f85f3e00640 1 -- 172.21.15.18:0/526044146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 msgr2=0x5619d1629ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.782+0000 7f85f3e00640 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 0x5619d1629ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.782+0000 7f85f3e00640 1 -- 172.21.15.18:0/526044146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 msgr2=0x5619d1629990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.782+0000 7f85f3e00640 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1629990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.783+0000 7f85f3e00640 1 -- 172.21.15.18:0/526044146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d1545920 con 0x5619d1622f10 2024-04-16T17:22:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.783+0000 7f85daa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:27.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.783+0000 7f85f3e00640 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d1629450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f85e80095b0 tx=0x7f85e8004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75215343e93f9e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.784+0000 7f85daa00640 1 -- 172.21.15.18:0/526044146 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8009820 con 0x5619d1622f10 2024-04-16T17:22:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.784+0000 7f85daa00640 1 -- 172.21.15.18:0/526044146 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f85e8011440 con 0x5619d1622f10 2024-04-16T17:22:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.784+0000 7f85fe7d0740 1 -- 172.21.15.18:0/526044146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e8015070 con 0x5619d1622f10 2024-04-16T17:22:27.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.785+0000 7f85daa00640 1 -- 172.21.15.18:0/526044146 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f85e8011640 con 0x5619d1622f10 2024-04-16T17:22:27.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.785+0000 7f85fe7d0740 10 client.30073 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:27.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.785+0000 7f85fe7d0740 1 -- 172.21.15.18:0/526044146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5619d15137f0 con 0x5619d1622f10 2024-04-16T17:22:27.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.785+0000 7f85daa00640 1 -- 172.21.15.18:0/526044146 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e8015070 con 0x5619d1622f10 2024-04-16T17:22:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.787+0000 7f85daa00640 1 -- 172.21.15.18:0/526044146 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f85e80199e0 con 0x5619d1622f10 2024-04-16T17:22:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.787+0000 7f85fe7d0740 10 client.30073 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.787+0000 7f85fe7d0740 10 client.30073 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:22:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.787+0000 7f85fe7d0740 20 client.30073 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.787+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5619d163c8b0 0x5619d163ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.787+0000 7f85fe7d0740 4 client.30073 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:27.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.787+0000 7f85fe7d0740 1 -- 172.21.15.18:0/526044146 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619d1595a40 con 0x5619d163c8b0 2024-04-16T17:22:27.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.788+0000 7f85f9000640 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5619d163c8b0 0x5619d163ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.789+0000 7f85f9000640 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5619d163c8b0 0x5619d163ed10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=724dc832d7efb53c server_cookie=1efb572e9244a205 in_seq=0 out_seq=0 2024-04-16T17:22:27.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.790+0000 7f85daa00640 10 client.30073 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:22:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.790+0000 7f85daa00640 1 -- 172.21.15.18:0/526044146 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619d1595a40 con 0x5619d163c8b0 2024-04-16T17:22:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.791+0000 7f85daa00640 10 client.30073 handle_command_reply: tid=0 2024-04-16T17:22:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.791+0000 7f85fe7d0740 1 client.30073 shutdown 2024-04-16T17:22:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.791+0000 7f85fe7d0740 1 -- 172.21.15.18:0/526044146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 msgr2=0x5619d1629450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.791+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d1629450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f85e80095b0 tx=0x7f85e8004c10 comp rx=0 tx=0).stop 2024-04-16T17:22:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.792+0000 7f85fe7d0740 1 -- 172.21.15.18:0/526044146 shutdown_connections 2024-04-16T17:22:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.792+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5619d163c8b0 0x5619d163ed10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.792+0000 7f85daa00640 0 client.30073 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:22:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.792+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 0x5619d1629ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.792+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1629990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.792+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/526044146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d1629450 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.792+0000 7f85fe7d0740 1 -- 172.21.15.18:0/526044146 >> 172.21.15.18:0/526044146 conn(0x5619d161d640 msgr2=0x5619d161f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.792+0000 7f85fe7d0740 1 -- 172.21.15.18:0/526044146 shutdown_connections 2024-04-16T17:22:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.793+0000 7f85fe7d0740 1 -- 172.21.15.18:0/526044146 wait complete. 2024-04-16T17:22:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.793+0000 7f85fe7d0740 20 client.30073 trim_cache size 0 max 16384 2024-04-16T17:22:27.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.823+0000 7f85fe7d0740 1 Processor -- start 2024-04-16T17:22:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.823+0000 7f85fe7d0740 1 -- start start 2024-04-16T17:22:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.823+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d162bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.823+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1638150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.823+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 0x5619d1628480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.823+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f85d4012be0 con 0x5619d1623830 2024-04-16T17:22:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.823+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85d40104b0 con 0x5619d162c410 2024-04-16T17:22:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.823+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85d400e2e0 con 0x5619d1622f10 2024-04-16T17:22:27.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.824+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d162bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.824+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d162bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39710/0 (socket says 172.21.15.18:39710) 2024-04-16T17:22:27.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.824+0000 7f85f3e00640 1 -- 172.21.15.18:0/4114429337 learned_addr learned my addr 172.21.15.18:0/4114429337 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.825+0000 7f85f3e00640 1 -- 172.21.15.18:0/4114429337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 msgr2=0x5619d1628480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.825+0000 7f85f3e00640 1 --2- 172.21.15.18:0/4114429337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 0x5619d1628480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.825+0000 7f85f3e00640 1 -- 172.21.15.18:0/4114429337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 msgr2=0x5619d1638150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.825+0000 7f85f3e00640 1 --2- 172.21.15.18:0/4114429337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1638150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.825+0000 7f85f3e00640 1 -- 172.21.15.18:0/4114429337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d1623660 con 0x5619d1622f10 2024-04-16T17:22:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.826+0000 7f85f3e00640 1 --2- 172.21.15.18:0/4114429337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d162bb80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f85e8019750 tx=0x7f85e8019820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f929525a2b3fe36 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.826+0000 7f85e5600640 1 -- 172.21.15.18:0/4114429337 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8011640 con 0x5619d1622f10 2024-04-16T17:22:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.827+0000 7f85e5600640 1 -- 172.21.15.18:0/4114429337 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e8015070 con 0x5619d1622f10 2024-04-16T17:22:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.827+0000 7f85e5600640 1 -- 172.21.15.18:0/4114429337 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8004b70 con 0x5619d1622f10 2024-04-16T17:22:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.828+0000 7f85fe7d0740 1 -- 172.21.15.18:0/4114429337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 msgr2=0x5619d162bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.828+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/4114429337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d162bb80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f85e8019750 tx=0x7f85e8019820 comp rx=0 tx=0).stop 2024-04-16T17:22:27.831 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:24 smithi018 ceph-mon[31776]: pgmap v562: 129 pgs: 7 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 116 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-04-16T17:22:27.831 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:25 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:22:27.831 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:26 smithi018 ceph-mon[31776]: pgmap v563: 129 pgs: 6 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 117 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2024-04-16T17:22:27.831 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:27 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:22:27.831 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:27 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:22:27.831 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:27 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:22:27.831 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:27 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4136676111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:22:27.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.832+0000 7f85fe7d0740 1 -- 172.21.15.18:0/4114429337 shutdown_connections 2024-04-16T17:22:27.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.832+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/4114429337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d162c410 0x5619d1628480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.832+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/4114429337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1638150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.832+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/4114429337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d1622f10 0x5619d162bb80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.832+0000 7f85fe7d0740 1 -- 172.21.15.18:0/4114429337 >> 172.21.15.18:0/4114429337 conn(0x5619d161d640 msgr2=0x5619d161da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.832+0000 7f85fe7d0740 1 -- 172.21.15.18:0/4114429337 shutdown_connections 2024-04-16T17:22:27.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.833+0000 7f85fe7d0740 1 -- 172.21.15.18:0/4114429337 wait complete. 2024-04-16T17:22:27.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.833+0000 7f85fe7d0740 1 -- start start 2024-04-16T17:22:27.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.834+0000 7f85fe7d0740 1 Processor -- start 2024-04-16T17:22:27.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1622f10 0x5619d1634580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1634ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85fe7d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d162c410 0x5619d162fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f85d400e970 con 0x5619d1623830 2024-04-16T17:22:27.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85d400fde0 con 0x5619d1622f10 2024-04-16T17:22:27.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85fe7d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85d400f040 con 0x5619d162c410 2024-04-16T17:22:27.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85fe7d0740 10 client.0 _ll_register_callbacks cb 0x7f85e8009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:27.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d162c410 0x5619d162fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f9000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1634ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d162c410 0x5619d162fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39718/0 (socket says 172.21.15.18:39718) 2024-04-16T17:22:27.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 -- 172.21.15.18:0/3742842372 learned_addr learned my addr 172.21.15.18:0/3742842372 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1622f10 0x5619d1634580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 -- 172.21.15.18:0/3742842372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1622f10 msgr2=0x5619d1634580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1622f10 0x5619d1634580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 -- 172.21.15.18:0/3742842372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 msgr2=0x5619d1634ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1634ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 -- 172.21.15.18:0/3742842372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d129a3f0 con 0x5619d162c410 2024-04-16T17:22:27.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85f3e00640 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d162c410 0x5619d162fa70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f85e80175f0 tx=0x7f85e80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b65368d362c34fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85daa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:27.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.836+0000 7f85daa00640 1 -- 172.21.15.18:0/3742842372 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8004b70 con 0x5619d162c410 2024-04-16T17:22:27.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85daa00640 1 -- 172.21.15.18:0/3742842372 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f85e8024d10 con 0x5619d162c410 2024-04-16T17:22:27.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 1 -- 172.21.15.18:0/3742842372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e8015070 con 0x5619d162c410 2024-04-16T17:22:27.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85daa00640 1 -- 172.21.15.18:0/3742842372 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f85e80095b0 con 0x5619d162c410 2024-04-16T17:22:27.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 10 client.30079 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:27.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 1 -- 172.21.15.18:0/3742842372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5619d13647d0 con 0x5619d162c410 2024-04-16T17:22:27.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85daa00640 1 -- 172.21.15.18:0/3742842372 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e8041020 con 0x5619d162c410 2024-04-16T17:22:27.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85daa00640 1 -- 172.21.15.18:0/3742842372 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f85e80199e0 con 0x5619d162c410 2024-04-16T17:22:27.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 10 client.30079 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:27.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 10 client.30079 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:22:27.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 20 client.30079 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:27.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5619d16a2ae0 0x5619d16a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 4 client.30079 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:22:27.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.841+0000 7f85fe7d0740 1 -- 172.21.15.18:0/3742842372 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f85e8015070 con 0x5619d16a2ae0 2024-04-16T17:22:27.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.843+0000 7f85f9000640 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5619d16a2ae0 0x5619d16a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.844+0000 7f85f9000640 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5619d16a2ae0 0x5619d16a4ed0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3aad12d786b0511 server_cookie=458e498feac74192 in_seq=0 out_seq=0 2024-04-16T17:22:27.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.844+0000 7f85daa00640 10 client.30079 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:22:27.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.846+0000 7f85daa00640 1 -- 172.21.15.18:0/3742842372 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f85e8015070 con 0x5619d16a2ae0 2024-04-16T17:22:27.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.846+0000 7f85daa00640 10 client.30079 handle_command_reply: tid=0 2024-04-16T17:22:27.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.846+0000 7f85fe7d0740 1 client.30079 shutdown 2024-04-16T17:22:27.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85fe7d0740 1 -- 172.21.15.18:0/3742842372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d162c410 msgr2=0x5619d162fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d162c410 0x5619d162fa70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f85e80175f0 tx=0x7f85e80107a0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85fe7d0740 1 -- 172.21.15.18:0/3742842372 shutdown_connections 2024-04-16T17:22:27.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5619d16a2ae0 0x5619d16a4ed0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619d162c410 0x5619d162fa70 secure :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f85e80175f0 tx=0x7f85e80107a0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5619d1623830 0x5619d1634ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85fe7d0740 1 --2- 172.21.15.18:0/3742842372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619d1622f10 0x5619d1634580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85fe7d0740 1 -- 172.21.15.18:0/3742842372 >> 172.21.15.18:0/3742842372 conn(0x5619d161d640 msgr2=0x5619d161c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.847+0000 7f85daa00640 0 client.30079 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:22:27.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.850+0000 7f85fe7d0740 1 -- 172.21.15.18:0/3742842372 shutdown_connections 2024-04-16T17:22:27.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.850+0000 7f85fe7d0740 1 -- 172.21.15.18:0/3742842372 wait complete. 2024-04-16T17:22:27.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.850+0000 7f85fe7d0740 20 client.30079 trim_cache size 0 max 16384 2024-04-16T17:22:27.850 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:22:27.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.855+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f85dc0a3fa0 msgr2=0x7f85dc0a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.855+0000 7f85fa400640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f85dc0a3fa0 0x7f85dc0a6460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f85f402dbb0 tx=0x7f85f4153040 comp rx=0 tx=0).stop 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.855+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ec0b7a70 msgr2=0x7f85ec0b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.855+0000 7f85fa400640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ec0b7a70 0x7f85ec0b35e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f85f4065520 tx=0x7f85f40a2860 comp rx=0 tx=0).stop 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.856+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 shutdown_connections 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.856+0000 7f85fa400640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f85dc0a3fa0 0x7f85dc0a6460 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.856+0000 7f85fa400640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ec0b7a70 0x7f85ec0b35e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.856+0000 7f85fa400640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85ec0a5700 0x7f85ec0b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.856+0000 7f85fa400640 1 --2- 172.21.15.18:0/3119488765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85ec0a4de0 0x7f85ec0b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.856+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 >> 172.21.15.18:0/3119488765 conn(0x7f85ec01a640 msgr2=0x7f85ec0b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.856+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 shutdown_connections 2024-04-16T17:22:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:27.856+0000 7f85fa400640 1 -- 172.21.15.18:0/3119488765 wait complete. 2024-04-16T17:22:27.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:22:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.912+0000 7f78da400640 1 -- 172.21.15.18:0/994585769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d4117130 msgr2=0x7f78d4119520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.912+0000 7f78da400640 1 --2- 172.21.15.18:0/994585769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d4117130 0x7f78d4119520 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009ca0 tx=0x7f78cc02c120 comp rx=0 tx=0).stop 2024-04-16T17:22:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.913+0000 7f78da400640 1 -- 172.21.15.18:0/994585769 shutdown_connections 2024-04-16T17:22:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.913+0000 7f78da400640 1 --2- 172.21.15.18:0/994585769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d4117130 0x7f78d4119520 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.913+0000 7f78da400640 1 --2- 172.21.15.18:0/994585769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78d41097e0 0x7f78d4109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.913+0000 7f78da400640 1 --2- 172.21.15.18:0/994585769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78d410c760 0x7f78d4109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.913+0000 7f78da400640 1 -- 172.21.15.18:0/994585769 >> 172.21.15.18:0/994585769 conn(0x7f78d406d0e0 msgr2=0x7f78d406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.913+0000 7f78da400640 1 -- 172.21.15.18:0/994585769 shutdown_connections 2024-04-16T17:22:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.913+0000 7f78da400640 1 -- 172.21.15.18:0/994585769 wait complete. 2024-04-16T17:22:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.915+0000 7f78da400640 1 Processor -- start 2024-04-16T17:22:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.915+0000 7f78da400640 1 -- start start 2024-04-16T17:22:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78da400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78d41097e0 0x7f78d4115010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78da400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78d410c760 0x7f78d4115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78da400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d4117130 0x7f78d41b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78da400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410d9b0 con 0x7f78d41097e0 2024-04-16T17:22:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78da400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410d830 con 0x7f78d4117130 2024-04-16T17:22:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78da400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410db30 con 0x7f78d410c760 2024-04-16T17:22:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78d3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78d41097e0 0x7f78d4115010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78d410c760 0x7f78d4115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78d410c760 0x7f78d4115550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39724/0 (socket says 172.21.15.18:39724) 2024-04-16T17:22:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78d3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78d41097e0 0x7f78d4115010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58860/0 (socket says 172.21.15.18:58860) 2024-04-16T17:22:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78d2a00640 1 -- 172.21.15.18:0/3511977563 learned_addr learned my addr 172.21.15.18:0/3511977563 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.916+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d4117130 0x7f78d41b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78d2a00640 1 -- 172.21.15.18:0/3511977563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d4117130 msgr2=0x7f78d41b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78d2a00640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d4117130 0x7f78d41b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78d2a00640 1 -- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78d41097e0 msgr2=0x7f78d4115010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78d2a00640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78d41097e0 0x7f78d4115010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78d2a00640 1 -- 172.21.15.18:0/3511977563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d41b3b90 con 0x7f78d410c760 2024-04-16T17:22:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78d2a00640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78d410c760 0x7f78d4115550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00b470 tx=0x7f78bc00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78cbe00640 1 -- 172.21.15.18:0/3511977563 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc015020 con 0x7f78d410c760 2024-04-16T17:22:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78cbe00640 1 -- 172.21.15.18:0/3511977563 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78bc011d10 con 0x7f78d410c760 2024-04-16T17:22:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.917+0000 7f78cbe00640 1 -- 172.21.15.18:0/3511977563 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00f2e0 con 0x7f78d410c760 2024-04-16T17:22:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.918+0000 7f78da400640 1 -- 172.21.15.18:0/3511977563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d41b3dc0 con 0x7f78d410c760 2024-04-16T17:22:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.919+0000 7f78da400640 1 -- 172.21.15.18:0/3511977563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d41b4230 con 0x7f78d410c760 2024-04-16T17:22:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.924+0000 7f78dd232740 1 Processor -- start 2024-04-16T17:22:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.925+0000 7f78cbe00640 1 -- 172.21.15.18:0/3511977563 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f78bc018020 con 0x7f78d410c760 2024-04-16T17:22:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.925+0000 7f78dd232740 1 -- start start 2024-04-16T17:22:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.928+0000 7f78dd232740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c6610 0x55de299c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.928+0000 7f78dd232740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.928+0000 7f78cbe00640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b80a3d90 0x7f78b80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.928+0000 7f78cbe00640 1 -- 172.21.15.18:0/3511977563 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f78bc0cf7b0 con 0x7f78d410c760 2024-04-16T17:22:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.929+0000 7f78dd232740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299cee20 0x55de299d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.929+0000 7f78dd232740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55de2992a780 con 0x55de299cee20 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.929+0000 7f78dd232740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55de298d0f40 con 0x55de299c6610 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.929+0000 7f78dd232740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55de29926740 con 0x55de299c58a0 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.929+0000 7f78d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.929+0000 7f78d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39738/0 (socket says 172.21.15.18:39738) 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.929+0000 7f78d3400640 1 -- 172.21.15.18:0/3931327044 learned_addr learned my addr 172.21.15.18:0/3931327044 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.929+0000 7f78d3400640 1 -- 172.21.15.18:0/3931327044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c6610 msgr2=0x55de299c5360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.930+0000 7f78d3400640 1 --2- 172.21.15.18:0/3931327044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c6610 0x55de299c5360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.930+0000 7f78d3400640 1 -- 172.21.15.18:0/3931327044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299cee20 msgr2=0x55de299d1210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:22:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.930+0000 7f78d3400640 1 --2- 172.21.15.18:0/3931327044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299cee20 0x55de299d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.930+0000 7f78d3400640 1 -- 172.21.15.18:0/3931327044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2963d210 con 0x55de299c58a0 2024-04-16T17:22:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.930+0000 7f78d2a00640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b80a3d90 0x7f78b80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.930+0000 7f78d3400640 1 --2- 172.21.15.18:0/3931327044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299c5d20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009a70 tx=0x7f78c4009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2ad446d9041e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.931+0000 7f78c8c00640 1 -- 172.21.15.18:0/3931327044 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4017070 con 0x55de299c58a0 2024-04-16T17:22:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.931+0000 7f78c8c00640 1 -- 172.21.15.18:0/3931327044 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c4013470 con 0x55de299c58a0 2024-04-16T17:22:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.931+0000 7f78c8c00640 1 -- 172.21.15.18:0/3931327044 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c40125f0 con 0x55de299c58a0 2024-04-16T17:22:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.931+0000 7f78d2a00640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b80a3d90 0x7f78b80a6250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0ccc60 tx=0x7f78bc097040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.932+0000 7f78dd232740 1 -- 172.21.15.18:0/3931327044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 msgr2=0x55de299c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.932+0000 7f78dd232740 1 --2- 172.21.15.18:0/3931327044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299c5d20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009a70 tx=0x7f78c4009f30 comp rx=0 tx=0).stop 2024-04-16T17:22:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.933+0000 7f78dd232740 1 -- 172.21.15.18:0/3931327044 shutdown_connections 2024-04-16T17:22:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.933+0000 7f78dd232740 1 --2- 172.21.15.18:0/3931327044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299cee20 0x55de299d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.933+0000 7f78dd232740 1 --2- 172.21.15.18:0/3931327044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299c5d20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.933+0000 7f78dd232740 1 --2- 172.21.15.18:0/3931327044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c6610 0x55de299c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.933+0000 7f78dd232740 1 -- 172.21.15.18:0/3931327044 >> 172.21.15.18:0/3931327044 conn(0x55de299605d0 msgr2=0x55de299c0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.933+0000 7f78dd232740 1 -- 172.21.15.18:0/3931327044 shutdown_connections 2024-04-16T17:22:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.933+0000 7f78dd232740 1 -- 172.21.15.18:0/3931327044 wait complete. 2024-04-16T17:22:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.934+0000 7f78dd232740 1 -- start start 2024-04-16T17:22:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.934+0000 7f78dd232740 1 Processor -- start 2024-04-16T17:22:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.934+0000 7f78dd232740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c58a0 0x55de299cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.934+0000 7f78dd232740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 0x55de299cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.934+0000 7f78dd232740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299cee20 0x55de299cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.934+0000 7f78dd232740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55de299d1b50 con 0x55de299c6610 2024-04-16T17:22:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.935+0000 7f78dd232740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55de298e4390 con 0x55de299c58a0 2024-04-16T17:22:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.935+0000 7f78dd232740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55de299d1cd0 con 0x55de299cee20 2024-04-16T17:22:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.935+0000 7f78dd232740 10 client.0 _ll_register_callbacks cb 0x55de298cdf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.935+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c58a0 0x55de299cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.935+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c58a0 0x55de299cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36890/0 (socket says 172.21.15.18:36890) 2024-04-16T17:22:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.935+0000 7f78d3e00640 1 -- 172.21.15.18:0/3988900122 learned_addr learned my addr 172.21.15.18:0/3988900122 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.935+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299cee20 0x55de299cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.935+0000 7f78d3400640 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 0x55de299cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.936+0000 7f78d3400640 1 -- 172.21.15.18:0/3988900122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299cee20 msgr2=0x55de299cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.936+0000 7f78d3400640 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299cee20 0x55de299cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.936+0000 7f78d3400640 1 -- 172.21.15.18:0/3988900122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c58a0 msgr2=0x55de299cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.936+0000 7f78d3400640 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c58a0 0x55de299cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.936+0000 7f78d3400640 1 -- 172.21.15.18:0/3988900122 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de29709620 con 0x55de299c6610 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.936+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c58a0 0x55de299cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.936+0000 7f78d3400640 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 0x55de299cc1d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f78c4004810 tx=0x7f78c40097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c466144281e33cab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.936+0000 7f78c1600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.937+0000 7f78c1600640 1 -- 172.21.15.18:0/3988900122 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4017050 con 0x55de299c6610 2024-04-16T17:22:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.937+0000 7f78c1600640 1 -- 172.21.15.18:0/3988900122 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f78c401c450 con 0x55de299c6610 2024-04-16T17:22:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.937+0000 7f78dd232740 1 -- 172.21.15.18:0/3988900122 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78c4013470 con 0x55de299c6610 2024-04-16T17:22:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.937+0000 7f78c1600640 1 -- 172.21.15.18:0/3988900122 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f78c401c670 con 0x55de299c6610 2024-04-16T17:22:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.937+0000 7f78dd232740 10 client.19479 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.937+0000 7f78dd232740 1 -- 172.21.15.18:0/3988900122 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55de299ddc80 con 0x55de299c6610 2024-04-16T17:22:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.937+0000 7f78c1600640 1 -- 172.21.15.18:0/3988900122 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c4013470 con 0x55de299c6610 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.938+0000 7f78c1600640 1 -- 172.21.15.18:0/3988900122 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f78c40269b0 con 0x55de299c6610 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.938+0000 7f78dd232740 10 client.19479 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.938+0000 7f78dd232740 10 client.19479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.938+0000 7f78dd232740 20 client.19479 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.938+0000 7f78dd232740 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55de299df030 0x55de299e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.938+0000 7f78dd232740 4 client.19479 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.938+0000 7f78dd232740 1 -- 172.21.15.18:0/3988900122 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de299381a0 con 0x55de299df030 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.938+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55de299df030 0x55de299e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.939+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55de299df030 0x55de299e1420 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73822686fd57f6c0 server_cookie=12f8e6ad59fdcbc4 in_seq=0 out_seq=0 2024-04-16T17:22:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.939+0000 7f78c1600640 10 client.19479 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78c1600640 1 -- 172.21.15.18:0/3988900122 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de299381a0 con 0x55de299df030 2024-04-16T17:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78c1600640 10 client.19479 handle_command_reply: tid=0 2024-04-16T17:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78dd232740 1 client.19479 shutdown 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78dd232740 1 -- 172.21.15.18:0/3988900122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 msgr2=0x55de299cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78dd232740 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 0x55de299cc1d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f78c4004810 tx=0x7f78c40097a0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78dd232740 1 -- 172.21.15.18:0/3988900122 shutdown_connections 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78dd232740 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55de299df030 0x55de299e1420 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78dd232740 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299cee20 0x55de299cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78dd232740 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 0x55de299cc1d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.942+0000 7f78dd232740 1 --2- 172.21.15.18:0/3988900122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299c58a0 0x55de299cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.943+0000 7f78dd232740 1 -- 172.21.15.18:0/3988900122 >> 172.21.15.18:0/3988900122 conn(0x55de299605d0 msgr2=0x55de299c22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.943+0000 7f78c1600640 0 client.19479 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.943+0000 7f78dd232740 1 -- 172.21.15.18:0/3988900122 shutdown_connections 2024-04-16T17:22:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.943+0000 7f78dd232740 1 -- 172.21.15.18:0/3988900122 wait complete. 2024-04-16T17:22:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.943+0000 7f78dd232740 20 client.19479 trim_cache size 0 max 16384 2024-04-16T17:22:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.976+0000 7f78dd232740 1 Processor -- start 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.977+0000 7f78dd232740 1 -- start start 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.977+0000 7f78dd232740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.977+0000 7f78dd232740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 0x55de299d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.977+0000 7f78dd232740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de299d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.977+0000 7f78dd232740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f78d41c1090 con 0x55de299c6610 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.977+0000 7f78dd232740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f78d41be960 con 0x55de299cee20 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.977+0000 7f78dd232740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78d41bc7d0 con 0x55de299c58a0 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39754/0 (socket says 172.21.15.18:39754) 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 -- 172.21.15.18:0/3957328695 learned_addr learned my addr 172.21.15.18:0/3957328695 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3957328695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de299d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 -- 172.21.15.18:0/3957328695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 msgr2=0x55de299d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3957328695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de299d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 -- 172.21.15.18:0/3957328695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 msgr2=0x55de299d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3957328695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 0x55de299d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.978+0000 7f78d3e00640 1 -- 172.21.15.18:0/3957328695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de299c4b70 con 0x55de299c58a0 2024-04-16T17:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.979+0000 7f78d3e00640 1 --2- 172.21.15.18:0/3957328695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299ce3c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f78cc02c730 tx=0x7f78cc009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f41eba154a1e9742 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.980+0000 7f78c3e00640 1 -- 172.21.15.18:0/3957328695 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc039040 con 0x55de299c58a0 2024-04-16T17:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.980+0000 7f78c3e00640 1 -- 172.21.15.18:0/3957328695 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc034b20 con 0x55de299c58a0 2024-04-16T17:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.980+0000 7f78c3e00640 1 -- 172.21.15.18:0/3957328695 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc044a20 con 0x55de299c58a0 2024-04-16T17:22:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.981+0000 7f78dd232740 1 -- 172.21.15.18:0/3957328695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 msgr2=0x55de299ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.981+0000 7f78dd232740 1 --2- 172.21.15.18:0/3957328695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299ce3c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f78cc02c730 tx=0x7f78cc009360 comp rx=0 tx=0).stop 2024-04-16T17:22:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.981+0000 7f78dd232740 1 -- 172.21.15.18:0/3957328695 shutdown_connections 2024-04-16T17:22:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.982+0000 7f78dd232740 1 --2- 172.21.15.18:0/3957328695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de299d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.982+0000 7f78dd232740 1 --2- 172.21.15.18:0/3957328695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c6610 0x55de299d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.982+0000 7f78dd232740 1 --2- 172.21.15.18:0/3957328695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c58a0 0x55de299ce3c0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.982+0000 7f78dd232740 1 -- 172.21.15.18:0/3957328695 >> 172.21.15.18:0/3957328695 conn(0x55de299605d0 msgr2=0x55de299c00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.983+0000 7f78dd232740 1 -- 172.21.15.18:0/3957328695 shutdown_connections 2024-04-16T17:22:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.983+0000 7f78dd232740 1 -- 172.21.15.18:0/3957328695 wait complete. 2024-04-16T17:22:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.983+0000 7f78dd232740 1 -- start start 2024-04-16T17:22:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.984+0000 7f78dd232740 1 Processor -- start 2024-04-16T17:22:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.984+0000 7f78dd232740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c58a0 0x55de299e4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.984+0000 7f78dd232740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c6610 0x55de299e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.984+0000 7f78dd232740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de29a401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.984+0000 7f78dd232740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f78d41bce20 con 0x55de299c58a0 2024-04-16T17:22:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.984+0000 7f78dd232740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f78d41be290 con 0x55de299cee20 2024-04-16T17:22:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.984+0000 7f78dd232740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78d41bd4f0 con 0x55de299c6610 2024-04-16T17:22:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.984+0000 7f78dd232740 10 client.0 _ll_register_callbacks cb 0x55de299ced70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.985+0000 7f78d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c6610 0x55de299e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.985+0000 7f78d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c6610 0x55de299e45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39762/0 (socket says 172.21.15.18:39762) 2024-04-16T17:22:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.985+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c58a0 0x55de299e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.985+0000 7f78d3400640 1 -- 172.21.15.18:0/1087947831 learned_addr learned my addr 172.21.15.18:0/1087947831 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.985+0000 7f78d3e00640 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de29a401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.985+0000 7f78d3400640 1 -- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 msgr2=0x55de29a401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.985+0000 7f78d3400640 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de29a401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.986+0000 7f78d3400640 1 -- 172.21.15.18:0/1087947831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c58a0 msgr2=0x55de299e4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.986+0000 7f78d3400640 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c58a0 0x55de299e4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.986+0000 7f78d3400640 1 -- 172.21.15.18:0/1087947831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de299c2890 con 0x55de299c6610 2024-04-16T17:22:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.986+0000 7f78d3e00640 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de29a401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:22:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.986+0000 7f78d3400640 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c6610 0x55de299e45b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f78c4001eb0 tx=0x7f78c4004640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e24826005450b63e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.986+0000 7f78c1600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.986+0000 7f78c1600640 1 -- 172.21.15.18:0/1087947831 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4017040 con 0x55de299c6610 2024-04-16T17:22:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.986+0000 7f78dd232740 1 -- 172.21.15.18:0/1087947831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78cc034b20 con 0x55de299c6610 2024-04-16T17:22:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.987+0000 7f78c1600640 1 -- 172.21.15.18:0/1087947831 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f78c401c810 con 0x55de299c6610 2024-04-16T17:22:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.987+0000 7f78c1600640 1 -- 172.21.15.18:0/1087947831 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f78c402f030 con 0x55de299c6610 2024-04-16T17:22:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.987+0000 7f78dd232740 10 client.30094 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.987+0000 7f78dd232740 1 -- 172.21.15.18:0/1087947831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55de299c84c0 con 0x55de299c6610 2024-04-16T17:22:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.987+0000 7f78c1600640 1 -- 172.21.15.18:0/1087947831 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc034b20 con 0x55de299c6610 2024-04-16T17:22:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.988+0000 7f78c1600640 1 -- 172.21.15.18:0/1087947831 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f78c4005ae0 con 0x55de299c6610 2024-04-16T17:22:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.988+0000 7f78dd232740 10 client.30094 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.988+0000 7f78dd232740 10 client.30094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.988+0000 7f78dd232740 20 client.30094 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.988+0000 7f78dd232740 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55de29a56210 0x55de29a58600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.988+0000 7f78dd232740 4 client.30094 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:22:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.988+0000 7f78dd232740 1 -- 172.21.15.18:0/1087947831 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de29932010 con 0x55de29a56210 2024-04-16T17:22:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.988+0000 7f78d3e00640 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55de29a56210 0x55de29a58600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.989+0000 7f78c1600640 10 client.30094 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:27.990+0000 7f78d3e00640 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55de29a56210 0x55de29a58600 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce810d95bb5f3bd server_cookie=78b51f60936b3774 in_seq=0 out_seq=0 2024-04-16T17:22:28.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1251753277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ed4115150 msgr2=0x7f2ed4117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:28.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1251753277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ed4115150 0x7f2ed4117540 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009cd0 tx=0x7f2ecc02c100 comp rx=0 tx=0).stop 2024-04-16T17:22:28.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1251753277 shutdown_connections 2024-04-16T17:22:28.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1251753277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ed4115150 0x7f2ed4117540 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1251753277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4109b00 0x7f2ed4114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1251753277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ed41091e0 0x7f2ed41095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1251753277 >> 172.21.15.18:0/1251753277 conn(0x7f2ed406d0a0 msgr2=0x7f2ed406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:28.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1251753277 shutdown_connections 2024-04-16T17:22:28.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.210+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1251753277 wait complete. 2024-04-16T17:22:28.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.213+0000 7f2ed9e00640 1 Processor -- start 2024-04-16T17:22:28.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.213+0000 7f2ed9e00640 1 -- start start 2024-04-16T17:22:28.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed41091e0 0x7f2ed41a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ed4109b00 0x7f2ed41a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ed4115150 0x7f2ed41a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed9e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed411abe0 con 0x7f2ed4115150 2024-04-16T17:22:28.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed9e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed411aa60 con 0x7f2ed4109b00 2024-04-16T17:22:28.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed9e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed411ad60 con 0x7f2ed41091e0 2024-04-16T17:22:28.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed41091e0 0x7f2ed41a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed41091e0 0x7f2ed41a3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39772/0 (socket says 172.21.15.18:39772) 2024-04-16T17:22:28.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.214+0000 7f2ed2a00640 1 -- 172.21.15.18:0/1380062642 learned_addr learned my addr 172.21.15.18:0/1380062642 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:28.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.215+0000 7f2ed2a00640 1 -- 172.21.15.18:0/1380062642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ed4109b00 msgr2=0x7f2ed41a38c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:22:28.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.215+0000 7f2ed2a00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ed4109b00 0x7f2ed41a38c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.215+0000 7f2ed2a00640 1 -- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ed4115150 msgr2=0x7f2ed41a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:28.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.215+0000 7f2ed2a00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ed4115150 0x7f2ed41a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.215+0000 7f2ed2a00640 1 -- 172.21.15.18:0/1380062642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed41a8320 con 0x7f2ed41091e0 2024-04-16T17:22:28.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.215+0000 7f2ed2a00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed41091e0 0x7f2ed41a3380 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400b4a0 tx=0x7f2ec400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:28.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.216+0000 7f2ed8a00640 1 -- 172.21.15.18:0/1380062642 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4014070 con 0x7f2ed41091e0 2024-04-16T17:22:28.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.216+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1380062642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed41a8610 con 0x7f2ed41091e0 2024-04-16T17:22:28.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.216+0000 7f2ed8a00640 1 -- 172.21.15.18:0/1380062642 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec4010af0 con 0x7f2ed41091e0 2024-04-16T17:22:28.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.216+0000 7f2ed8a00640 1 -- 172.21.15.18:0/1380062642 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4017c40 con 0x7f2ed41091e0 2024-04-16T17:22:28.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.216+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1380062642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed41aff20 con 0x7f2ed41091e0 2024-04-16T17:22:28.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.221+0000 7f2edcb7b740 1 Processor -- start 2024-04-16T17:22:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.221+0000 7f2edcb7b740 1 -- start start 2024-04-16T17:22:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.222+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac17b60 0x56546ac17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.222+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 0x56546ac29520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac29a60 0x56546ac2be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56546ab7d020 con 0x56546ac18510 2024-04-16T17:22:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56546ab23460 con 0x56546ac29a60 2024-04-16T17:22:28.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56546ab78fe0 con 0x56546ac17b60 2024-04-16T17:22:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac17b60 0x56546ac17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2ed8a00640 1 -- 172.21.15.18:0/1380062642 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ec4017070 con 0x7f2ed41091e0 2024-04-16T17:22:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2ed2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac29a60 0x56546ac2be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2ed2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac29a60 0x56546ac2be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36948/0 (socket says 172.21.15.18:36948) 2024-04-16T17:22:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.223+0000 7f2ed2000640 1 -- 172.21.15.18:0/1576446806 learned_addr learned my addr 172.21.15.18:0/1576446806 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.224+0000 7f2ed3400640 1 -- 172.21.15.18:0/1576446806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac29a60 msgr2=0x56546ac2be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.224+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1576446806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac29a60 0x56546ac2be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.224+0000 7f2ed3400640 1 -- 172.21.15.18:0/1576446806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 msgr2=0x56546ac29520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.224+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1576446806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 0x56546ac29520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.224+0000 7f2ed3400640 1 -- 172.21.15.18:0/1576446806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546ab2b040 con 0x56546ac17b60 2024-04-16T17:22:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.225+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1576446806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac17b60 0x56546ac17f40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009ca0 tx=0x7f2ecc032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae711aa84d6bea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.225+0000 7f2ebfe00640 1 -- 172.21.15.18:0/1576446806 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc039040 con 0x56546ac17b60 2024-04-16T17:22:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.226+0000 7f2ebfe00640 1 -- 172.21.15.18:0/1576446806 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc042070 con 0x56546ac17b60 2024-04-16T17:22:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.226+0000 7f2ebfe00640 1 -- 172.21.15.18:0/1576446806 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc034940 con 0x56546ac17b60 2024-04-16T17:22:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.226+0000 7f2ed8a00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2eb80a3e60 0x7f2eb80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.227+0000 7f2ed8a00640 1 -- 172.21.15.18:0/1380062642 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f2ec40cf8b0 con 0x7f2ed41091e0 2024-04-16T17:22:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.227+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1576446806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac17b60 msgr2=0x56546ac17f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.227+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1576446806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac17b60 0x56546ac17f40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009ca0 tx=0x7f2ecc032340 comp rx=0 tx=0).stop 2024-04-16T17:22:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.228+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1576446806 shutdown_connections 2024-04-16T17:22:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.228+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1576446806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac29a60 0x56546ac2be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.228+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1576446806 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 0x56546ac29520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.228+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1576446806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac17b60 0x56546ac17f40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.228+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1576446806 >> 172.21.15.18:0/1576446806 conn(0x56546ac12480 msgr2=0x56546ac12860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:28.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.228+0000 7f2ed2a00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2eb80a3e60 0x7f2eb80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.228+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1576446806 shutdown_connections 2024-04-16T17:22:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.229+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1576446806 wait complete. 2024-04-16T17:22:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.229+0000 7f2ed2a00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2eb80a3e60 0x7f2eb80a6320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x56546ac13710 tx=0x7f2ec40047d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.229+0000 7f2edcb7b740 1 -- start start 2024-04-16T17:22:28.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.229+0000 7f2edcb7b740 1 Processor -- start 2024-04-16T17:22:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.231+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac26b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.232+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.232+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.232+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac26b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.232+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac26b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58934/0 (socket says 172.21.15.18:58934) 2024-04-16T17:22:28.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.232+0000 7f2ed2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.232+0000 7f2ed2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac270b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36958/0 (socket says 172.21.15.18:36958) 2024-04-16T17:22:28.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.232+0000 7f2ed3400640 1 -- 172.21.15.18:0/2608994686 learned_addr learned my addr 172.21.15.18:0/2608994686 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:28.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.233+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56546ac2c7c0 con 0x56546ac17b60 2024-04-16T17:22:28.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.233+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56546ab36830 con 0x56546ac18510 2024-04-16T17:22:28.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.233+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56546ac2c940 con 0x56546ac29a60 2024-04-16T17:22:28.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.233+0000 7f2edcb7b740 10 client.0 _ll_register_callbacks cb 0x56546ab3e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:28.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.233+0000 7f2ed3400640 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.234+0000 7f2ed3400640 1 -- 172.21.15.18:0/2608994686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 msgr2=0x56546ac275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:28.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.234+0000 7f2ed3400640 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.234+0000 7f2ed3400640 1 -- 172.21.15.18:0/2608994686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 msgr2=0x56546ac270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:28.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.234+0000 7f2ed3400640 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.234+0000 7f2ed3400640 1 -- 172.21.15.18:0/2608994686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546ab3a920 con 0x56546ac17b60 2024-04-16T17:22:28.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.234+0000 7f2ed2000640 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:22:28.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.234+0000 7f2ed3400640 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac26b70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc034620 tx=0x7f2ecc02c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e7020518cf776f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:28.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.234+0000 7f2ebcc00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:22:28.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.235+0000 7f2ebcc00640 1 -- 172.21.15.18:0/2608994686 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc039040 con 0x56546ac17b60 2024-04-16T17:22:28.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.235+0000 7f2ebcc00640 1 -- 172.21.15.18:0/2608994686 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9642+0+0 (secure 0 0 0) 0x7f2ecc003d80 con 0x56546ac17b60 2024-04-16T17:22:28.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.235+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ecc042070 con 0x56546ac17b60 2024-04-16T17:22:28.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.236+0000 7f2ebcc00640 1 -- 172.21.15.18:0/2608994686 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2ecc05a050 con 0x56546ac17b60 2024-04-16T17:22:28.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.236+0000 7f2edcb7b740 10 client.19485 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:28.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.236+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56546ab087f0 con 0x56546ac17b60 2024-04-16T17:22:28.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.237+0000 7f2ebcc00640 1 -- 172.21.15.18:0/2608994686 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc042070 con 0x56546ac17b60 2024-04-16T17:22:28.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.237+0000 7f2ebcc00640 1 -- 172.21.15.18:0/2608994686 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2ecc03c070 con 0x56546ac17b60 2024-04-16T17:22:28.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.237+0000 7f2edcb7b740 10 client.19485 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:28.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.237+0000 7f2edcb7b740 10 client.19485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:28.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.237+0000 7f2edcb7b740 20 client.19485 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:28.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.237+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56546ac39d00 0x56546ac3c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.238+0000 7f2edcb7b740 4 client.19485 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:28.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.238+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56546ab8aa40 con 0x56546ac39d00 2024-04-16T17:22:28.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.243+0000 7f2ed2000640 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56546ac39d00 0x56546ac3c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.246+0000 7f2ed2000640 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56546ac39d00 0x56546ac3c0f0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151c29e38395ead server_cookie=2a86f06cd809f56e in_seq=0 out_seq=0 2024-04-16T17:22:28.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.246+0000 7f2ebcc00640 10 client.19485 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:28.519 INFO:tasks.workunit.client.0.smithi018.stdout: 10 68038 12.12 MB/sec execute 160 sec latency 395.551 ms 2024-04-16T17:22:28.664 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:28 smithi018 ceph-mon[31776]: pgmap v564: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 407 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 383 op/s 2024-04-16T17:22:28.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:28 smithi022 ceph-mon[36023]: pgmap v564: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 407 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 383 op/s 2024-04-16T17:22:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.959+0000 7f78c1600640 1 -- 172.21.15.18:0/1087947831 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de29932010 con 0x55de29a56210 2024-04-16T17:22:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.959+0000 7f78c1600640 10 client.30094 handle_command_reply: tid=0 2024-04-16T17:22:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.959+0000 7f78dd232740 1 client.30094 shutdown 2024-04-16T17:22:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.959+0000 7f78dd232740 1 -- 172.21.15.18:0/1087947831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c6610 msgr2=0x55de299e45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78dd232740 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c6610 0x55de299e45b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f78c4001eb0 tx=0x7f78c4004640 comp rx=0 tx=0).stop 2024-04-16T17:22:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78dd232740 1 -- 172.21.15.18:0/1087947831 shutdown_connections 2024-04-16T17:22:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78dd232740 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55de29a56210 0x55de29a58600 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78dd232740 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55de299cee20 0x55de29a401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78c1600640 0 client.30094 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78dd232740 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55de299c6610 0x55de299e45b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78dd232740 1 --2- 172.21.15.18:0/1087947831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de299c58a0 0x55de299e4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78dd232740 1 -- 172.21.15.18:0/1087947831 >> 172.21.15.18:0/1087947831 conn(0x55de299605d0 msgr2=0x55de299ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:28.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.961+0000 7f2ebcc00640 1 -- 172.21.15.18:0/2608994686 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56546ab8aa40 con 0x56546ac39d00 2024-04-16T17:22:28.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.961+0000 7f2ebcc00640 10 client.19485 handle_command_reply: tid=0 2024-04-16T17:22:28.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.961+0000 7f2edcb7b740 1 client.19485 shutdown 2024-04-16T17:22:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.960+0000 7f78dd232740 1 -- 172.21.15.18:0/1087947831 shutdown_connections 2024-04-16T17:22:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.962+0000 7f78dd232740 1 -- 172.21.15.18:0/1087947831 wait complete. 2024-04-16T17:22:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.962+0000 7f78dd232740 20 client.30094 trim_cache size 0 max 16384 2024-04-16T17:22:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:22:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:22:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:22:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.962+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 msgr2=0x56546ac26b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.962+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac26b70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc034620 tx=0x7f2ecc02c620 comp rx=0 tx=0).stop 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 shutdown_connections 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56546ac39d00 0x56546ac3c0f0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/2608994686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac26b70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 >> 172.21.15.18:0/2608994686 conn(0x56546ac12480 msgr2=0x56546ac141d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2ebcc00640 0 client.19485 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:28.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 shutdown_connections 2024-04-16T17:22:28.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.963+0000 7f2edcb7b740 1 -- 172.21.15.18:0/2608994686 wait complete. 2024-04-16T17:22:28.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f2edcb7b740 20 client.19485 trim_cache size 0 max 16384 2024-04-16T17:22:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 -- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b80a3d90 msgr2=0x7f78b80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b80a3d90 0x7f78b80a6250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0ccc60 tx=0x7f78bc097040 comp rx=0 tx=0).stop 2024-04-16T17:22:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 -- 172.21.15.18:0/3511977563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78d410c760 msgr2=0x7f78d4115550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78d410c760 0x7f78d4115550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00b470 tx=0x7f78bc00b870 comp rx=0 tx=0).stop 2024-04-16T17:22:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 -- 172.21.15.18:0/3511977563 shutdown_connections 2024-04-16T17:22:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f78b80a3d90 0x7f78b80a6250 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d4117130 0x7f78d41b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78d410c760 0x7f78d4115550 secure :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00b470 tx=0x7f78bc00b870 comp rx=0 tx=0).stop 2024-04-16T17:22:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 --2- 172.21.15.18:0/3511977563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78d41097e0 0x7f78d4115010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.964+0000 7f78da400640 1 -- 172.21.15.18:0/3511977563 >> 172.21.15.18:0/3511977563 conn(0x7f78d406d0e0 msgr2=0x7f78d4117510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.967+0000 7f78da400640 1 -- 172.21.15.18:0/3511977563 shutdown_connections 2024-04-16T17:22:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:28.967+0000 7f78da400640 1 -- 172.21.15.18:0/3511977563 wait complete. 2024-04-16T17:22:28.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.995+0000 7f2edcb7b740 1 Processor -- start 2024-04-16T17:22:28.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.995+0000 7f2edcb7b740 1 -- start start 2024-04-16T17:22:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac25d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed41b8660 con 0x56546ac17b60 2024-04-16T17:22:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed41b5f30 con 0x56546ac18510 2024-04-16T17:22:28.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed41b3e00 con 0x56546ac29a60 2024-04-16T17:22:28.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac292a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58940/0 (socket says 172.21.15.18:58940) 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2ed3400640 1 -- 172.21.15.18:0/1592756498 learned_addr learned my addr 172.21.15.18:0/1592756498 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.996+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac25d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.997+0000 7f2ed2000640 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.997+0000 7f2ed3400640 1 -- 172.21.15.18:0/1592756498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 msgr2=0x56546ac25d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.997+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac25d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.997+0000 7f2ed3400640 1 -- 172.21.15.18:0/1592756498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 msgr2=0x56546ac356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.997+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.997+0000 7f2ed3400640 1 -- 172.21.15.18:0/1592756498 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546a88f3f0 con 0x56546ac17b60 2024-04-16T17:22:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.998+0000 7f2ed2000640 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:22:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.998+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac292a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc02f020 tx=0x7f2ecc0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9606b3704bc4d574 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.998+0000 7f2ebf400640 1 -- 172.21.15.18:0/1592756498 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc039070 con 0x56546ac17b60 2024-04-16T17:22:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.998+0000 7f2ebf400640 1 -- 172.21.15.18:0/1592756498 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc042040 con 0x56546ac17b60 2024-04-16T17:22:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.998+0000 7f2ebf400640 1 -- 172.21.15.18:0/1592756498 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0348b0 con 0x56546ac17b60 2024-04-16T17:22:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:28.999+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1592756498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 msgr2=0x56546ac292a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac292a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc02f020 tx=0x7f2ecc0021d0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1592756498 shutdown_connections 2024-04-16T17:22:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac25d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac18510 0x56546ac356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1592756498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac17b60 0x56546ac292a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1592756498 >> 172.21.15.18:0/1592756498 conn(0x56546ac12480 msgr2=0x56546ac138d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1592756498 shutdown_connections 2024-04-16T17:22:29.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:28 smithi043 ceph-mon[36901]: pgmap v564: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 407 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 383 op/s 2024-04-16T17:22:29.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1592756498 wait complete. 2024-04-16T17:22:29.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.000+0000 7f2edcb7b740 1 -- start start 2024-04-16T17:22:29.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.001+0000 7f2edcb7b740 1 Processor -- start 2024-04-16T17:22:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.001+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac17b60 0x56546ac31bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.001+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 0x56546ac32110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.001+0000 7f2edcb7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac23250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.001+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed41b4450 con 0x56546ac18510 2024-04-16T17:22:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.001+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed41b5860 con 0x56546ac17b60 2024-04-16T17:22:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.002+0000 7f2edcb7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed41b4ac0 con 0x56546ac29a60 2024-04-16T17:22:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.002+0000 7f2edcb7b740 10 client.0 _ll_register_callbacks cb 0x7f2ecc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:29.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.002+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac17b60 0x56546ac31bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:29.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.002+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac17b60 0x56546ac31bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36978/0 (socket says 172.21.15.18:36978) 2024-04-16T17:22:29.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.002+0000 7f2ed3400640 1 -- 172.21.15.18:0/1106905340 learned_addr learned my addr 172.21.15.18:0/1106905340 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:29.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.002+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac23250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.003+0000 7f2ed3400640 1 -- 172.21.15.18:0/1106905340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 msgr2=0x56546ac23250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.003+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac23250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.003+0000 7f2ed3400640 1 -- 172.21.15.18:0/1106905340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 msgr2=0x56546ac32110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.003+0000 7f2ed2000640 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 0x56546ac32110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.003+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 0x56546ac32110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.003+0000 7f2ed3400640 1 -- 172.21.15.18:0/1106905340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546a9597d0 con 0x56546ac17b60 2024-04-16T17:22:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.003+0000 7f2ed2000640 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 0x56546ac32110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:22:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.005+0000 7f2ebcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:22:29.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.005+0000 7f2ed3400640 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac17b60 0x56546ac31bd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009a60 tx=0x7f2ecc032980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e808798c68a9c842 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:29.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.005+0000 7f2ebcc00640 1 -- 172.21.15.18:0/1106905340 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc039040 con 0x56546ac17b60 2024-04-16T17:22:29.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.005+0000 7f2ebcc00640 1 -- 172.21.15.18:0/1106905340 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9554+0+0 (secure 0 0 0) 0x7f2ecc0046b0 con 0x56546ac17b60 2024-04-16T17:22:29.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.005+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1106905340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ecc042040 con 0x56546ac17b60 2024-04-16T17:22:29.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.006+0000 7f2ebcc00640 1 -- 172.21.15.18:0/1106905340 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2ecc039040 con 0x56546ac17b60 2024-04-16T17:22:29.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.006+0000 7f2edcb7b740 10 client.30257 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:29.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.006+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1106905340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56546ac19670 con 0x56546ac17b60 2024-04-16T17:22:29.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.007+0000 7f2ebcc00640 1 -- 172.21.15.18:0/1106905340 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc057070 con 0x56546ac17b60 2024-04-16T17:22:29.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.008+0000 7f2ebcc00640 1 -- 172.21.15.18:0/1106905340 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2ecc03c070 con 0x56546ac17b60 2024-04-16T17:22:29.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.008+0000 7f2edcb7b740 10 client.30257 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:29.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.008+0000 7f2edcb7b740 10 client.30257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:29.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.008+0000 7f2edcb7b740 20 client.30257 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:29.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.008+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56546ac19c90 0x56546ac1c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:29.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.008+0000 7f2edcb7b740 4 client.30257 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:22:29.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.008+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1106905340 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2ecc042040 con 0x56546ac19c90 2024-04-16T17:22:29.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.009+0000 7f2ed2000640 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56546ac19c90 0x56546ac1c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:29.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.010+0000 7f2ed2000640 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56546ac19c90 0x56546ac1c080 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e7b76625d586dd server_cookie=ed982c618cb67e88 in_seq=0 out_seq=0 2024-04-16T17:22:29.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.010+0000 7f2ebcc00640 10 client.30257 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:29.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.011+0000 7f2ebcc00640 1 -- 172.21.15.18:0/1106905340 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2ecc042040 con 0x56546ac19c90 2024-04-16T17:22:29.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.011+0000 7f2ebcc00640 10 client.30257 handle_command_reply: tid=0 2024-04-16T17:22:29.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.011+0000 7f2edcb7b740 1 client.30257 shutdown 2024-04-16T17:22:29.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1106905340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac17b60 msgr2=0x56546ac31bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:29.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac17b60 0x56546ac31bd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009a60 tx=0x7f2ecc032980 comp rx=0 tx=0).stop 2024-04-16T17:22:29.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1106905340 shutdown_connections 2024-04-16T17:22:29.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56546ac19c90 0x56546ac1c080 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56546ac29a60 0x56546ac23250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56546ac18510 0x56546ac32110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2edcb7b740 1 --2- 172.21.15.18:0/1106905340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56546ac17b60 0x56546ac31bd0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1106905340 >> 172.21.15.18:0/1106905340 conn(0x56546ac12480 msgr2=0x56546ac114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:29.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.012+0000 7f2ebcc00640 0 client.30257 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:29.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.013+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1106905340 shutdown_connections 2024-04-16T17:22:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.013+0000 7f2edcb7b740 1 -- 172.21.15.18:0/1106905340 wait complete. 2024-04-16T17:22:29.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.014+0000 7f2edcb7b740 20 client.30257 trim_cache size 0 max 16384 2024-04-16T17:22:29.014 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:22:29.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.015+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2eb80a3e60 msgr2=0x7f2eb80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:29.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.015+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2eb80a3e60 0x7f2eb80a6320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x56546ac13710 tx=0x7f2ec40047d0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.015+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1380062642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed41091e0 msgr2=0x7f2ed41a3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:29.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.015+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed41091e0 0x7f2ed41a3380 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400b4a0 tx=0x7f2ec400b960 comp rx=0 tx=0).stop 2024-04-16T17:22:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.016+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1380062642 shutdown_connections 2024-04-16T17:22:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.016+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2eb80a3e60 0x7f2eb80a6320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.016+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ed4115150 0x7f2ed41a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.016+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ed4109b00 0x7f2ed41a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.016+0000 7f2ed9e00640 1 --2- 172.21.15.18:0/1380062642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed41091e0 0x7f2ed41a3380 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:29.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.016+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1380062642 >> 172.21.15.18:0/1380062642 conn(0x7f2ed406d0a0 msgr2=0x7f2ed410aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.019+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1380062642 shutdown_connections 2024-04-16T17:22:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:29.019+0000 7f2ed9e00640 1 -- 172.21.15.18:0/1380062642 wait complete. 2024-04-16T17:22:29.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 68221 12.11 MB/sec execute 161 sec latency 585.438 ms 2024-04-16T17:22:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:29 smithi043 ceph-mon[36901]: osdmap e125: 12 total, 12 up, 12 in 2024-04-16T17:22:30.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:29 smithi043 ceph-mon[36901]: pgmap v566: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 407 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-04-16T17:22:30.042 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:22:30.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:22:30.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:22:30.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:29 smithi022 ceph-mon[36023]: osdmap e125: 12 total, 12 up, 12 in 2024-04-16T17:22:30.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:29 smithi022 ceph-mon[36023]: pgmap v566: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 407 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-04-16T17:22:30.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 -- 172.21.15.18:0/2502817241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103270 msgr2=0x7ff8a4103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 --2- 172.21.15.18:0/2502817241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103270 0x7ff8a4103650 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff898009900 tx=0x7ff89802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 -- 172.21.15.18:0/2502817241 shutdown_connections 2024-04-16T17:22:30.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 --2- 172.21.15.18:0/2502817241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a410c5b0 0x7ff8a410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 --2- 172.21.15.18:0/2502817241 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a4103b90 0x7ff8a410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 --2- 172.21.15.18:0/2502817241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103270 0x7ff8a4103650 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 -- 172.21.15.18:0/2502817241 >> 172.21.15.18:0/2502817241 conn(0x7ff8a40780b0 msgr2=0x7ff8a40784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 -- 172.21.15.18:0/2502817241 shutdown_connections 2024-04-16T17:22:30.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.303+0000 7ff8ab400640 1 -- 172.21.15.18:0/2502817241 wait complete. 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.304+0000 7ff8ab400640 1 Processor -- start 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.304+0000 7ff8ab400640 1 -- start start 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a4103270 0x7ff8a41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8ab400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103b90 0x7ff8a41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8ab400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a410c5b0 0x7ff8a419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8ab400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41012c0 con 0x7ff8a410c5b0 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8ab400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4101140 con 0x7ff8a4103b90 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8ab400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4101440 con 0x7ff8a4103270 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a4103270 0x7ff8a41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a4103270 0x7ff8a41a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39820/0 (socket says 172.21.15.18:39820) 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8a8a00640 1 -- 172.21.15.18:0/86657957 learned_addr learned my addr 172.21.15.18:0/86657957 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103b90 0x7ff8a41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8a9400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a410c5b0 0x7ff8a419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8a3e00640 1 -- 172.21.15.18:0/86657957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a4103270 msgr2=0x7ff8a41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.305+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a4103270 0x7ff8a41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8a3e00640 1 -- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a410c5b0 msgr2=0x7ff8a419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a410c5b0 0x7ff8a419f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8a3e00640 1 -- 172.21.15.18:0/86657957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a419fa70 con 0x7ff8a4103b90 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8a9400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a410c5b0 0x7ff8a419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103b90 0x7ff8a41a0b50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e9f0 tx=0x7ff89000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8a1600640 1 -- 172.21.15.18:0/86657957 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89000cd80 con 0x7ff8a4103b90 2024-04-16T17:22:30.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8a1600640 1 -- 172.21.15.18:0/86657957 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8900035f0 con 0x7ff8a4103b90 2024-04-16T17:22:30.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8a1600640 1 -- 172.21.15.18:0/86657957 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890010690 con 0x7ff8a4103b90 2024-04-16T17:22:30.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.306+0000 7ff8ab400640 1 -- 172.21.15.18:0/86657957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a419fd60 con 0x7ff8a4103b90 2024-04-16T17:22:30.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.307+0000 7ff8ab400640 1 -- 172.21.15.18:0/86657957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a41a75f0 con 0x7ff8a4103b90 2024-04-16T17:22:30.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.311+0000 7ff8ae1a3740 1 Processor -- start 2024-04-16T17:22:30.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.311+0000 7ff8ae1a3740 1 -- start start 2024-04-16T17:22:30.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.311+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.311+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7a35c0 0x5587ae7abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.311+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 0x5587ae7ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.311+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5587ae707cb0 con 0x5587ae7a35c0 2024-04-16T17:22:30.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.311+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587ae6ae370 con 0x5587ae7ac1a0 2024-04-16T17:22:30.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.311+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587ae703c70 con 0x5587ae7a2ca0 2024-04-16T17:22:30.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7a35c0 0x5587ae7abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39828/0 (socket says 172.21.15.18:39828) 2024-04-16T17:22:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a9400640 1 -- 172.21.15.18:0/2521080478 learned_addr learned my addr 172.21.15.18:0/2521080478 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a1600640 1 -- 172.21.15.18:0/86657957 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff890003b30 con 0x7ff8a4103b90 2024-04-16T17:22:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a8a00640 1 -- 172.21.15.18:0/2521080478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 msgr2=0x5587ae7a3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/2521080478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a8a00640 1 -- 172.21.15.18:0/2521080478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 msgr2=0x5587ae7ae590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:22:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/2521080478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 0x5587ae7ae590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.312+0000 7ff8a8a00640 1 -- 172.21.15.18:0/2521080478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587ae6cd910 con 0x5587ae7a35c0 2024-04-16T17:22:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.313+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/2521080478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7a35c0 0x5587ae7abc60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff898005ab0 tx=0x7ff898005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dfc2a3354ab26f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.314+0000 7ff895c00640 1 -- 172.21.15.18:0/2521080478 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898043070 con 0x5587ae7a35c0 2024-04-16T17:22:30.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.314+0000 7ff895c00640 1 -- 172.21.15.18:0/2521080478 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89802c910 con 0x5587ae7a35c0 2024-04-16T17:22:30.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.314+0000 7ff895c00640 1 -- 172.21.15.18:0/2521080478 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898004b60 con 0x5587ae7a35c0 2024-04-16T17:22:30.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.314+0000 7ff8a1600640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff88c0a3da0 0x7ff88c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.314+0000 7ff8a1600640 1 -- 172.21.15.18:0/86657957 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9554+0+0 (secure 0 0 0) 0x7ff8900d0f20 con 0x7ff8a4103b90 2024-04-16T17:22:30.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.315+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/2521080478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7a35c0 msgr2=0x5587ae7abc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.315+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/2521080478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7a35c0 0x5587ae7abc60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff898005ab0 tx=0x7ff898005be0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.315+0000 7ff8a9400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff88c0a3da0 0x7ff88c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.316+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/2521080478 shutdown_connections 2024-04-16T17:22:30.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.316+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/2521080478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 0x5587ae7ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.316+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/2521080478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7a35c0 0x5587ae7abc60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.316+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/2521080478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.316+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/2521080478 >> 172.21.15.18:0/2521080478 conn(0x5587ae79d3d0 msgr2=0x5587ae79d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.316+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/2521080478 shutdown_connections 2024-04-16T17:22:30.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.316+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/2521080478 wait complete. 2024-04-16T17:22:30.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.317+0000 7ff8a9400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff88c0a3da0 0x7ff88c0a6260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff89c005c40 tx=0x7ff89c009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.317+0000 7ff8ae1a3740 1 -- start start 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.317+0000 7ff8ae1a3740 1 Processor -- start 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a35c0 0x5587ae7a9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5587ae7aef30 con 0x5587ae7ac1a0 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587ae6c1730 con 0x5587ae7a35c0 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587ae7af0b0 con 0x5587ae7a2ca0 2024-04-16T17:22:30.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8ae1a3740 10 client.0 _ll_register_callbacks cb 0x5587ae6901e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:30.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a35c0 0x5587ae7a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39838/0 (socket says 172.21.15.18:39838) 2024-04-16T17:22:30.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8a8a00640 1 -- 172.21.15.18:0/3274308155 learned_addr learned my addr 172.21.15.18:0/3274308155 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.318+0000 7ff8a9400640 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.319+0000 7ff8a3e00640 1 -- 172.21.15.18:0/3274308155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 msgr2=0x5587ae7a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.319+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.319+0000 7ff8a3e00640 1 -- 172.21.15.18:0/3274308155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7ac1a0 msgr2=0x5587ae7a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.319+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.319+0000 7ff8a3e00640 1 -- 172.21.15.18:0/3274308155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ae4e27e0 con 0x5587ae7a35c0 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.319+0000 7ff8a9400640 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.320+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a35c0 0x5587ae7a9790 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff8900ce260 tx=0x7ff89009d270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c16481aa5581e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.320+0000 7ff88a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:22:30.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.320+0000 7ff88a000640 1 -- 172.21.15.18:0/3274308155 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8900a5720 con 0x5587ae7a35c0 2024-04-16T17:22:30.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.320+0000 7ff88a000640 1 -- 172.21.15.18:0/3274308155 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9554+0+0 (secure 0 0 0) 0x7ff8900037a0 con 0x5587ae7a35c0 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.320+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/3274308155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff89802c910 con 0x5587ae7a35c0 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.320+0000 7ff88a000640 1 -- 172.21.15.18:0/3274308155 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff8900ce4e0 con 0x5587ae7a35c0 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.320+0000 7ff8ae1a3740 10 client.30266 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.320+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/3274308155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5587ae542f70 con 0x5587ae7a35c0 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.321+0000 7ff88a000640 1 -- 172.21.15.18:0/3274308155 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89802c910 con 0x5587ae7a35c0 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.321+0000 7ff88a000640 1 -- 172.21.15.18:0/3274308155 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff89009b970 con 0x5587ae7a35c0 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.321+0000 7ff8ae1a3740 10 client.30266 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.321+0000 7ff8ae1a3740 10 client.30266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.321+0000 7ff8ae1a3740 20 client.30266 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.321+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587ae7bc5a0 0x5587ae7be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.321+0000 7ff8ae1a3740 4 client.30266 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.321+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/3274308155 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587ae7156d0 con 0x5587ae7bc5a0 2024-04-16T17:22:30.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.322+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587ae7bc5a0 0x5587ae7be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.323+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587ae7bc5a0 0x5587ae7be9c0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b359896929353882 server_cookie=74fae900b5c49d87 in_seq=0 out_seq=0 2024-04-16T17:22:30.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.323+0000 7ff88a000640 10 client.30266 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:30.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.326+0000 7ff88a000640 1 -- 172.21.15.18:0/3274308155 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587ae7156d0 con 0x5587ae7bc5a0 2024-04-16T17:22:30.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.326+0000 7ff88a000640 10 client.30266 handle_command_reply: tid=0 2024-04-16T17:22:30.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.327+0000 7ff8ae1a3740 1 client.30266 shutdown 2024-04-16T17:22:30.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.327+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/3274308155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a35c0 msgr2=0x5587ae7a9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.327+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a35c0 0x5587ae7a9790 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff8900ce260 tx=0x7ff89009d270 comp rx=0 tx=0).stop 2024-04-16T17:22:30.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.327+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/3274308155 shutdown_connections 2024-04-16T17:22:30.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.327+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587ae7bc5a0 0x5587ae7be9c0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.327+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.327+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a35c0 0x5587ae7a9790 secure :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff8900ce260 tx=0x7ff89009d270 comp rx=0 tx=0).stop 2024-04-16T17:22:30.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.328+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/3274308155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.328+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/3274308155 >> 172.21.15.18:0/3274308155 conn(0x5587ae79d3d0 msgr2=0x5587ae79f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.328+0000 7ff88a000640 0 client.30266 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:30.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.328+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/3274308155 shutdown_connections 2024-04-16T17:22:30.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.328+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/3274308155 wait complete. 2024-04-16T17:22:30.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.329+0000 7ff8ae1a3740 20 client.30266 trim_cache size 0 max 16384 2024-04-16T17:22:30.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.367+0000 7ff8ae1a3740 1 Processor -- start 2024-04-16T17:22:30.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.368+0000 7ff8ae1a3740 1 -- start start 2024-04-16T17:22:30.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 0x5587ae7c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41b4150 con 0x5587ae7c4290 2024-04-16T17:22:30.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41b1a20 con 0x5587ae7a2ca0 2024-04-16T17:22:30.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41af890 con 0x5587ae7ac1a0 2024-04-16T17:22:30.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37030/0 (socket says 172.21.15.18:37030) 2024-04-16T17:22:30.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.369+0000 7ff8a8a00640 1 -- 172.21.15.18:0/1817040224 learned_addr learned my addr 172.21.15.18:0/1817040224 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.370+0000 7ff8a8a00640 1 -- 172.21.15.18:0/1817040224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7ac1a0 msgr2=0x5587ae7a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.370+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/1817040224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.370+0000 7ff8a8a00640 1 -- 172.21.15.18:0/1817040224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 msgr2=0x5587ae7c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.370+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/1817040224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 0x5587ae7c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.370+0000 7ff8a8a00640 1 -- 172.21.15.18:0/1817040224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587ae3da170 con 0x5587ae7a2ca0 2024-04-16T17:22:30.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.371+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/1817040224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b7e40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff898036040 tx=0x7ff898005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eed43ab5a36b373 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.371+0000 7ff895200640 1 -- 172.21.15.18:0/1817040224 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898043040 con 0x5587ae7a2ca0 2024-04-16T17:22:30.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.371+0000 7ff895200640 1 -- 172.21.15.18:0/1817040224 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5587ae7156d0 con 0x5587ae7a2ca0 2024-04-16T17:22:30.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.371+0000 7ff895200640 1 -- 172.21.15.18:0/1817040224 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89804f670 con 0x5587ae7a2ca0 2024-04-16T17:22:30.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.373+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/1817040224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a2ca0 msgr2=0x5587ae7b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.373+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/1817040224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b7e40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff898036040 tx=0x7ff898005a20 comp rx=0 tx=0).stop 2024-04-16T17:22:30.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.373+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/1817040224 shutdown_connections 2024-04-16T17:22:30.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.373+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/1817040224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 0x5587ae7c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.373+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/1817040224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7ac1a0 0x5587ae7a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.373+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/1817040224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b7e40 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.373+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/1817040224 >> 172.21.15.18:0/1817040224 conn(0x5587ae79d3d0 msgr2=0x5587ae79c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.374+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/1817040224 shutdown_connections 2024-04-16T17:22:30.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.374+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/1817040224 wait complete. 2024-04-16T17:22:30.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.375+0000 7ff8ae1a3740 1 -- start start 2024-04-16T17:22:30.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.375+0000 7ff8ae1a3740 1 Processor -- start 2024-04-16T17:22:30.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.376+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.377+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 0x5587ae7b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.377+0000 7ff8ae1a3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 0x5587ae7b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.377+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41afee0 con 0x5587ae7c4290 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.377+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41b1350 con 0x5587ae7ac1a0 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.377+0000 7ff8ae1a3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41b05b0 con 0x5587ae7a2ca0 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.377+0000 7ff8ae1a3740 10 client.0 _ll_register_callbacks cb 0x7ff89009c680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.377+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 0x5587ae7b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.378+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.378+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 0x5587ae7b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58990/0 (socket says 172.21.15.18:58990) 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.378+0000 7ff8a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39850/0 (socket says 172.21.15.18:39850) 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.378+0000 7ff8a8a00640 1 -- 172.21.15.18:0/731866995 learned_addr learned my addr 172.21.15.18:0/731866995 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.378+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 0x5587ae7b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.378+0000 7ff8a8a00640 1 -- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 msgr2=0x5587ae7b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.379+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 0x5587ae7b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.379+0000 7ff8a8a00640 1 -- 172.21.15.18:0/731866995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 msgr2=0x5587ae7b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.379+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 0x5587ae7b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.379+0000 7ff8a8a00640 1 -- 172.21.15.18:0/731866995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ae7b1f20 con 0x5587ae7a2ca0 2024-04-16T17:22:30.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.379+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 0x5587ae7b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:22:30.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.380+0000 7ff8a8a00640 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b4620 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff898005940 tx=0x7ff898009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca4f1014f03269f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.380+0000 7ff88a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:30.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.381+0000 7ff88a000640 1 -- 172.21.15.18:0/731866995 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898043040 con 0x5587ae7a2ca0 2024-04-16T17:22:30.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.381+0000 7ff88a000640 1 -- 172.21.15.18:0/731866995 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9554+0+0 (secure 0 0 0) 0x7ff898004240 con 0x5587ae7a2ca0 2024-04-16T17:22:30.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.381+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/731866995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587ae7156d0 con 0x5587ae7a2ca0 2024-04-16T17:22:30.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.382+0000 7ff88a000640 1 -- 172.21.15.18:0/731866995 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff89805d050 con 0x5587ae7a2ca0 2024-04-16T17:22:30.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.382+0000 7ff8ae1a3740 10 client.30130 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:30.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.382+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/731866995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5587ae82c250 con 0x5587ae7a2ca0 2024-04-16T17:22:30.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.383+0000 7ff88a000640 1 -- 172.21.15.18:0/731866995 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5587ae7156d0 con 0x5587ae7a2ca0 2024-04-16T17:22:30.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.384+0000 7ff88a000640 1 -- 172.21.15.18:0/731866995 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff8980049e0 con 0x5587ae7a2ca0 2024-04-16T17:22:30.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.384+0000 7ff8ae1a3740 10 client.30130 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:30.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.384+0000 7ff8ae1a3740 10 client.30130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.384+0000 7ff8ae1a3740 20 client.30130 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.385+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587ae82c480 0x5587ae82e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.385+0000 7ff8ae1a3740 4 client.30130 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:22:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.385+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/731866995 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587ae70f540 con 0x5587ae82c480 2024-04-16T17:22:30.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.385+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587ae82c480 0x5587ae82e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.386+0000 7ff8a3e00640 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587ae82c480 0x5587ae82e870 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff5f28078334073 server_cookie=3722696a0be4727e in_seq=0 out_seq=0 2024-04-16T17:22:30.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.386+0000 7ff88a000640 10 client.30130 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.402+0000 7ff88a000640 1 -- 172.21.15.18:0/731866995 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587ae70f540 con 0x5587ae82c480 2024-04-16T17:22:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.402+0000 7ff88a000640 10 client.30130 handle_command_reply: tid=0 2024-04-16T17:22:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.402+0000 7ff8ae1a3740 1 client.30130 shutdown 2024-04-16T17:22:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/731866995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 msgr2=0x5587ae7b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b4620 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff898005940 tx=0x7ff898009260 comp rx=0 tx=0).stop 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/731866995 shutdown_connections 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5587ae82c480 0x5587ae82e870 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5587ae7c4290 0x5587ae7b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587ae7ac1a0 0x5587ae7b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 --2- 172.21.15.18:0/731866995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587ae7a2ca0 0x5587ae7b4620 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/731866995 >> 172.21.15.18:0/731866995 conn(0x5587ae79d3d0 msgr2=0x5587ae7c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff88a000640 0 client.30130 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/731866995 shutdown_connections 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.403+0000 7ff8ae1a3740 1 -- 172.21.15.18:0/731866995 wait complete. 2024-04-16T17:22:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.404+0000 7ff8ae1a3740 20 client.30130 trim_cache size 0 max 16384 2024-04-16T17:22:30.406 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:22:30.406 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:22:30.406 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "0db4c055-c85c-4da6-9fb6-3f237145695c", 2024-04-16T17:22:30.406 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:22:30.406 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:22:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 -- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff88c0a3da0 msgr2=0x7ff88c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff88c0a3da0 0x7ff88c0a6260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff89c005c40 tx=0x7ff89c009500 comp rx=0 tx=0).stop 2024-04-16T17:22:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 -- 172.21.15.18:0/86657957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103b90 msgr2=0x7ff8a41a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103b90 0x7ff8a41a0b50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e9f0 tx=0x7ff89000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 -- 172.21.15.18:0/86657957 shutdown_connections 2024-04-16T17:22:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff88c0a3da0 0x7ff88c0a6260 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8a410c5b0 0x7ff8a419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8a4103b90 0x7ff8a41a0b50 secure :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e9f0 tx=0x7ff89000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 --2- 172.21.15.18:0/86657957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a4103270 0x7ff8a41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 -- 172.21.15.18:0/86657957 >> 172.21.15.18:0/86657957 conn(0x7ff8a40780b0 msgr2=0x7ff8a410d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 -- 172.21.15.18:0/86657957 shutdown_connections 2024-04-16T17:22:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.407+0000 7ff8ab400640 1 -- 172.21.15.18:0/86657957 wait complete. 2024-04-16T17:22:30.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:22:30.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:22:30.519 INFO:tasks.workunit.client.0.smithi018.stdout: 10 68498 12.15 MB/sec execute 162 sec latency 133.709 ms 2024-04-16T17:22:30.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.720+0000 7f58f3200640 1 -- 172.21.15.18:0/3670177691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 msgr2=0x7f58f4110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.720+0000 7f58f3200640 1 --2- 172.21.15.18:0/3670177691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 0x7f58f4110f20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009ca0 tx=0x7f58ec02c100 comp rx=0 tx=0).stop 2024-04-16T17:22:30.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.721+0000 7f58f3200640 1 -- 172.21.15.18:0/3670177691 shutdown_connections 2024-04-16T17:22:30.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.721+0000 7f58f3200640 1 --2- 172.21.15.18:0/3670177691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 0x7f58f4110f20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.721+0000 7f58f3200640 1 --2- 172.21.15.18:0/3670177691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f4104c30 0x7f58f4108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.721+0000 7f58f3200640 1 --2- 172.21.15.18:0/3670177691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f4104280 0x7f58f4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.721+0000 7f58f3200640 1 -- 172.21.15.18:0/3670177691 >> 172.21.15.18:0/3670177691 conn(0x7f58f40ffae0 msgr2=0x7f58f4101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.721+0000 7f58f3200640 1 -- 172.21.15.18:0/3670177691 shutdown_connections 2024-04-16T17:22:30.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.721+0000 7f58f3200640 1 -- 172.21.15.18:0/3670177691 wait complete. 2024-04-16T17:22:30.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.723+0000 7f58f3200640 1 Processor -- start 2024-04-16T17:22:30.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.723+0000 7f58f3200640 1 -- start start 2024-04-16T17:22:30.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f4104280 0x7f58f4110e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 0x7f58f410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f3200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f410e460 0x7f58f410e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f3200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4114460 con 0x7f58f410e460 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41142e0 con 0x7f58f4104280 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41145e0 con 0x7f58f4109350 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 0x7f58f410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f4104280 0x7f58f4110e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 0x7f58f410df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39872/0 (socket says 172.21.15.18:39872) 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.724+0000 7f58f1400640 1 -- 172.21.15.18:0/831072463 learned_addr learned my addr 172.21.15.18:0/831072463 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.725+0000 7f58f1400640 1 -- 172.21.15.18:0/831072463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f4104280 msgr2=0x7f58f4110e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.725+0000 7f58f1400640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f4104280 0x7f58f4110e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.725+0000 7f58f1400640 1 -- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f410e460 msgr2=0x7f58f410e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.725+0000 7f58f1400640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f410e460 0x7f58f410e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.725+0000 7f58f1400640 1 -- 172.21.15.18:0/831072463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f41a0c10 con 0x7f58f4109350 2024-04-16T17:22:30.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.726+0000 7f58f1400640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 0x7f58f410df20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b450 tx=0x7f58e000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.726+0000 7f58eaa00640 1 -- 172.21.15.18:0/831072463 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0015020 con 0x7f58f4109350 2024-04-16T17:22:30.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.727+0000 7f58eaa00640 1 -- 172.21.15.18:0/831072463 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e0011d70 con 0x7f58f4109350 2024-04-16T17:22:30.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.727+0000 7f58eaa00640 1 -- 172.21.15.18:0/831072463 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e000f340 con 0x7f58f4109350 2024-04-16T17:22:30.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.727+0000 7f58f3200640 1 -- 172.21.15.18:0/831072463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f41a0f00 con 0x7f58f4109350 2024-04-16T17:22:30.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.727+0000 7f58f3200640 1 -- 172.21.15.18:0/831072463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f41a1400 con 0x7f58f4109350 2024-04-16T17:22:30.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.733+0000 7f58eaa00640 1 -- 172.21.15.18:0/831072463 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f58e0018020 con 0x7f58f4109350 2024-04-16T17:22:30.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.734+0000 7f58fb67a740 1 Processor -- start 2024-04-16T17:22:30.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.734+0000 7f58fb67a740 1 -- start start 2024-04-16T17:22:30.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd512770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd51b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd51dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58fb67a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c4cd478120 con 0x55c4cd51b890 2024-04-16T17:22:30.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58fb67a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c4cd41e340 con 0x55c4cd512cb0 2024-04-16T17:22:30.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58fb67a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4cd4740e0 con 0x55c4cd512390 2024-04-16T17:22:30.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58f2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd512770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58f2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd512770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39886/0 (socket says 172.21.15.18:39886) 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58f2800640 1 -- 172.21.15.18:0/3944256944 learned_addr learned my addr 172.21.15.18:0/3944256944 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58f1e00640 1 --2- 172.21.15.18:0/3944256944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd51b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58f2800640 1 -- 172.21.15.18:0/3944256944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 msgr2=0x55c4cd51b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58f2800640 1 --2- 172.21.15.18:0/3944256944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd51b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.735+0000 7f58f2800640 1 -- 172.21.15.18:0/3944256944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 msgr2=0x55c4cd51dc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.737+0000 7f58eaa00640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58d80a3ef0 0x7f58d80a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.737+0000 7f58eaa00640 1 -- 172.21.15.18:0/831072463 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9554+0+0 (secure 0 0 0) 0x7f58e00d0500 con 0x7f58f4109350 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.737+0000 7f58f1e00640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58d80a3ef0 0x7f58d80a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.738+0000 7f58f2800640 1 --2- 172.21.15.18:0/3944256944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd51dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.738+0000 7f58f2800640 1 -- 172.21.15.18:0/3944256944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4cd2547d0 con 0x55c4cd512390 2024-04-16T17:22:30.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.738+0000 7f58f2800640 1 --2- 172.21.15.18:0/3944256944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd512770 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f58ec02c710 tx=0x7f58ec032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=708f1ffa46be299d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.738+0000 7f58f1e00640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58d80a3ef0 0x7f58d80a63b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55c4cd511e60 tx=0x7f58e400b390 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.739+0000 7f58df000640 1 -- 172.21.15.18:0/3944256944 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec041020 con 0x55c4cd512390 2024-04-16T17:22:30.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.739+0000 7f58df000640 1 -- 172.21.15.18:0/3944256944 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec032c90 con 0x55c4cd512390 2024-04-16T17:22:30.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.739+0000 7f58df000640 1 -- 172.21.15.18:0/3944256944 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec039040 con 0x55c4cd512390 2024-04-16T17:22:30.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.740+0000 7f58fb67a740 1 -- 172.21.15.18:0/3944256944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 msgr2=0x55c4cd512770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.740+0000 7f58fb67a740 1 --2- 172.21.15.18:0/3944256944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd512770 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f58ec02c710 tx=0x7f58ec032340 comp rx=0 tx=0).stop 2024-04-16T17:22:30.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.740+0000 7f58fb67a740 1 -- 172.21.15.18:0/3944256944 shutdown_connections 2024-04-16T17:22:30.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.740+0000 7f58fb67a740 1 --2- 172.21.15.18:0/3944256944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd51dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.740+0000 7f58fb67a740 1 --2- 172.21.15.18:0/3944256944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd51b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.740+0000 7f58fb67a740 1 --2- 172.21.15.18:0/3944256944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd512770 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.740+0000 7f58fb67a740 1 -- 172.21.15.18:0/3944256944 >> 172.21.15.18:0/3944256944 conn(0x55c4cd50cac0 msgr2=0x55c4cd50cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.741+0000 7f58fb67a740 1 -- 172.21.15.18:0/3944256944 shutdown_connections 2024-04-16T17:22:30.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.741+0000 7f58fb67a740 1 -- 172.21.15.18:0/3944256944 wait complete. 2024-04-16T17:22:30.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.741+0000 7f58fb67a740 1 -- start start 2024-04-16T17:22:30.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.742+0000 7f58fb67a740 1 Processor -- start 2024-04-16T17:22:30.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.742+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5189b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.742+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd518ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd519430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58fb67a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c4cd51e620 con 0x55c4cd51b890 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58fb67a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c4cd431710 con 0x55c4cd512cb0 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58fb67a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4cd51e7a0 con 0x55c4cd512390 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58fb67a740 10 client.0 _ll_register_callbacks cb 0x55c4cd42caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd518ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd518ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:37080/0 (socket says 172.21.15.18:37080) 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5189b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39890/0 (socket says 172.21.15.18:39890) 2024-04-16T17:22:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1400640 1 -- 172.21.15.18:0/1983959784 learned_addr learned my addr 172.21.15.18:0/1983959784 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1400640 1 -- 172.21.15.18:0/1983959784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 msgr2=0x55c4cd5189b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1400640 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5189b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1400640 1 -- 172.21.15.18:0/1983959784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 msgr2=0x55c4cd519430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1400640 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd519430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.743+0000 7f58f1400640 1 -- 172.21.15.18:0/1983959784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4cd18a3f0 con 0x55c4cd512cb0 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.744+0000 7f58f1400640 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd518ef0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f58e000be10 tx=0x7f58e009bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746b347482d70a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.744+0000 7f58d3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.744+0000 7f58d3400640 1 -- 172.21.15.18:0/1983959784 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e009c140 con 0x55c4cd512cb0 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.744+0000 7f58d3400640 1 -- 172.21.15.18:0/1983959784 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9554+0+0 (secure 0 0 0) 0x7f58e00cda60 con 0x55c4cd512cb0 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.744+0000 7f58fb67a740 1 -- 172.21.15.18:0/1983959784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58ec032c90 con 0x55c4cd512cb0 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.744+0000 7f58d3400640 1 -- 172.21.15.18:0/1983959784 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f58e00a04f0 con 0x55c4cd512cb0 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.744+0000 7f58fb67a740 10 client.30284 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:30.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.744+0000 7f58fb67a740 1 -- 172.21.15.18:0/1983959784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c4cd14a0f0 con 0x55c4cd512cb0 2024-04-16T17:22:30.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.745+0000 7f58d3400640 1 -- 172.21.15.18:0/1983959784 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec032c90 con 0x55c4cd512cb0 2024-04-16T17:22:30.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.745+0000 7f58d3400640 1 -- 172.21.15.18:0/1983959784 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f58e00b0be0 con 0x55c4cd512cb0 2024-04-16T17:22:30.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.745+0000 7f58fb67a740 10 client.30284 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:30.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.745+0000 7f58fb67a740 10 client.30284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:30.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.745+0000 7f58fb67a740 20 client.30284 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:30.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.746+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4cd52bbf0 0x55c4cd52e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.746+0000 7f58fb67a740 4 client.30284 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:30.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.746+0000 7f58fb67a740 1 -- 172.21.15.18:0/1983959784 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4cd485b40 con 0x55c4cd52bbf0 2024-04-16T17:22:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.746+0000 7f58f1e00640 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4cd52bbf0 0x55c4cd52e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.747+0000 7f58f1e00640 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4cd52bbf0 0x55c4cd52e030 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7220138e67495000 server_cookie=252c69907914cc in_seq=0 out_seq=0 2024-04-16T17:22:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.747+0000 7f58d3400640 10 client.30284 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:30.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.749+0000 7f58d3400640 1 -- 172.21.15.18:0/1983959784 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4cd485b40 con 0x55c4cd52bbf0 2024-04-16T17:22:30.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.749+0000 7f58d3400640 10 client.30284 handle_command_reply: tid=0 2024-04-16T17:22:30.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.749+0000 7f58fb67a740 1 client.30284 shutdown 2024-04-16T17:22:30.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.750+0000 7f58fb67a740 1 -- 172.21.15.18:0/1983959784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 msgr2=0x55c4cd518ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.750+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd518ef0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f58e000be10 tx=0x7f58e009bf90 comp rx=0 tx=0).stop 2024-04-16T17:22:30.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.753+0000 7f58fb67a740 1 -- 172.21.15.18:0/1983959784 shutdown_connections 2024-04-16T17:22:30.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.753+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4cd52bbf0 0x55c4cd52e030 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.753+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd519430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.753+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd518ef0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.753+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1983959784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5189b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.753+0000 7f58fb67a740 1 -- 172.21.15.18:0/1983959784 >> 172.21.15.18:0/1983959784 conn(0x55c4cd50cac0 msgr2=0x55c4cd50e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.754+0000 7f58d3400640 0 client.30284 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:30.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.755+0000 7f58fb67a740 1 -- 172.21.15.18:0/1983959784 shutdown_connections 2024-04-16T17:22:30.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.755+0000 7f58fb67a740 1 -- 172.21.15.18:0/1983959784 wait complete. 2024-04-16T17:22:30.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.755+0000 7f58fb67a740 20 client.30284 trim_cache size 0 max 16384 2024-04-16T17:22:30.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.795+0000 7f58fb67a740 1 Processor -- start 2024-04-16T17:22:30.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.795+0000 7f58fb67a740 1 -- start start 2024-04-16T17:22:30.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd513cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd519080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58fb67a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41c0b20 con 0x55c4cd51b890 2024-04-16T17:22:30.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58fb67a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41be3f0 con 0x55c4cd512cb0 2024-04-16T17:22:30.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58fb67a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41bc260 con 0x55c4cd512390 2024-04-16T17:22:30.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5274e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39898/0 (socket says 172.21.15.18:39898) 2024-04-16T17:22:30.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58f1e00640 1 -- 172.21.15.18:0/3220473818 learned_addr learned my addr 172.21.15.18:0/3220473818 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.796+0000 7f58f1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd513cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.797+0000 7f58f1e00640 1 -- 172.21.15.18:0/3220473818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 msgr2=0x55c4cd513cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.797+0000 7f58f1e00640 1 --2- 172.21.15.18:0/3220473818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd513cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.797+0000 7f58f1e00640 1 -- 172.21.15.18:0/3220473818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 msgr2=0x55c4cd519080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.797+0000 7f58f1e00640 1 --2- 172.21.15.18:0/3220473818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd519080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.797+0000 7f58f1e00640 1 -- 172.21.15.18:0/3220473818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4cd516f40 con 0x55c4cd512390 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.797+0000 7f58f1e00640 1 --2- 172.21.15.18:0/3220473818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5274e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f58e40043f0 tx=0x7f58e4004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a41be41cb5cd3b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.798+0000 7f58de600640 1 -- 172.21.15.18:0/3220473818 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e402f070 con 0x55c4cd512390 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.798+0000 7f58de600640 1 -- 172.21.15.18:0/3220473818 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e402a420 con 0x55c4cd512390 2024-04-16T17:22:30.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.798+0000 7f58de600640 1 -- 172.21.15.18:0/3220473818 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e401d8f0 con 0x55c4cd512390 2024-04-16T17:22:30.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.799+0000 7f58fb67a740 1 -- 172.21.15.18:0/3220473818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 msgr2=0x55c4cd5274e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.799+0000 7f58fb67a740 1 --2- 172.21.15.18:0/3220473818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5274e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f58e40043f0 tx=0x7f58e4004610 comp rx=0 tx=0).stop 2024-04-16T17:22:30.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.800+0000 7f58fb67a740 1 -- 172.21.15.18:0/3220473818 shutdown_connections 2024-04-16T17:22:30.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.800+0000 7f58fb67a740 1 --2- 172.21.15.18:0/3220473818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd51b890 0x55c4cd519080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.800+0000 7f58fb67a740 1 --2- 172.21.15.18:0/3220473818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512cb0 0x55c4cd513cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.800+0000 7f58fb67a740 1 --2- 172.21.15.18:0/3220473818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd512390 0x55c4cd5274e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.800+0000 7f58fb67a740 1 -- 172.21.15.18:0/3220473818 >> 172.21.15.18:0/3220473818 conn(0x55c4cd50cac0 msgr2=0x55c4cd518970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.800+0000 7f58fb67a740 1 -- 172.21.15.18:0/3220473818 shutdown_connections 2024-04-16T17:22:30.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.800+0000 7f58fb67a740 1 -- 172.21.15.18:0/3220473818 wait complete. 2024-04-16T17:22:30.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.801+0000 7f58fb67a740 1 -- start start 2024-04-16T17:22:30.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.801+0000 7f58fb67a740 1 Processor -- start 2024-04-16T17:22:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.802+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512390 0x55c4cd5239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.802+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd512cb0 0x55c4cd523f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.802+0000 7f58fb67a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd51b890 0x55c4cd5209c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.802+0000 7f58fb67a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41bc8b0 con 0x55c4cd512cb0 2024-04-16T17:22:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.802+0000 7f58fb67a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41bdd20 con 0x55c4cd512390 2024-04-16T17:22:30.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.802+0000 7f58fb67a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41bcf80 con 0x55c4cd51b890 2024-04-16T17:22:30.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.803+0000 7f58fb67a740 10 client.0 _ll_register_callbacks cb 0x7f58e00983e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:30.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.803+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512390 0x55c4cd5239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.803+0000 7f58f2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd51b890 0x55c4cd5209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.803+0000 7f58f2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd51b890 0x55c4cd5209c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39902/0 (socket says 172.21.15.18:39902) 2024-04-16T17:22:30.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.803+0000 7f58f2800640 1 -- 172.21.15.18:0/1236476713 learned_addr learned my addr 172.21.15.18:0/1236476713 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.804+0000 7f58f1400640 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd512cb0 0x55c4cd523f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.804+0000 7f58f1e00640 1 -- 172.21.15.18:0/1236476713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd51b890 msgr2=0x55c4cd5209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.804+0000 7f58f1e00640 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd51b890 0x55c4cd5209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.804+0000 7f58f1e00640 1 -- 172.21.15.18:0/1236476713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd512cb0 msgr2=0x55c4cd523f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.804+0000 7f58f1e00640 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd512cb0 0x55c4cd523f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.804+0000 7f58f1e00640 1 -- 172.21.15.18:0/1236476713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4cd5210b0 con 0x55c4cd512390 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.804+0000 7f58f1e00640 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512390 0x55c4cd5239f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f58e401d4f0 tx=0x7f58e400be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0bfca13dbdbba56 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.805+0000 7f58d3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.805+0000 7f58d3400640 1 -- 172.21.15.18:0/1236476713 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4038020 con 0x55c4cd512390 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.805+0000 7f58d3400640 1 -- 172.21.15.18:0/1236476713 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9554+0+0 (secure 0 0 0) 0x7f58e402a890 con 0x55c4cd512390 2024-04-16T17:22:30.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.805+0000 7f58fb67a740 1 -- 172.21.15.18:0/1236476713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e402a420 con 0x55c4cd512390 2024-04-16T17:22:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.805+0000 7f58d3400640 1 -- 172.21.15.18:0/1236476713 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f58e402f050 con 0x55c4cd512390 2024-04-16T17:22:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.806+0000 7f58fb67a740 10 client.30290 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.806+0000 7f58fb67a740 1 -- 172.21.15.18:0/1236476713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c4cd592b30 con 0x55c4cd512390 2024-04-16T17:22:30.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.806+0000 7f58d3400640 1 -- 172.21.15.18:0/1236476713 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e402a420 con 0x55c4cd512390 2024-04-16T17:22:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.806+0000 7f58d3400640 1 -- 172.21.15.18:0/1236476713 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f58e4004680 con 0x55c4cd512390 2024-04-16T17:22:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.806+0000 7f58fb67a740 10 client.30290 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.806+0000 7f58fb67a740 10 client.30290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.806+0000 7f58fb67a740 20 client.30290 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.807+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4cd593140 0x55c4cd595560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.807+0000 7f58fb67a740 4 client.30290 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:22:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.807+0000 7f58fb67a740 1 -- 172.21.15.18:0/1236476713 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c4cd47f9b0 con 0x55c4cd593140 2024-04-16T17:22:30.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.807+0000 7f58f1400640 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4cd593140 0x55c4cd595560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:30.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.809+0000 7f58f1400640 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4cd593140 0x55c4cd595560 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682c17ff2d081fcf server_cookie=593bf47e7972ca32 in_seq=0 out_seq=0 2024-04-16T17:22:30.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.809+0000 7f58d3400640 10 client.30290 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:30.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.815+0000 7f58d3400640 1 -- 172.21.15.18:0/1236476713 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55c4cd47f9b0 con 0x55c4cd593140 2024-04-16T17:22:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.815+0000 7f58d3400640 10 client.30290 handle_command_reply: tid=0 2024-04-16T17:22:30.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.816+0000 7f58fb67a740 1 client.30290 shutdown 2024-04-16T17:22:30.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.822+0000 7f58fb67a740 1 -- 172.21.15.18:0/1236476713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512390 msgr2=0x55c4cd5239f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.822+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512390 0x55c4cd5239f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f58e401d4f0 tx=0x7f58e400be00 comp rx=0 tx=0).stop 2024-04-16T17:22:30.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.825+0000 7f58fb67a740 1 -- 172.21.15.18:0/1236476713 shutdown_connections 2024-04-16T17:22:30.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.825+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55c4cd593140 0x55c4cd595560 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.825+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4cd51b890 0x55c4cd5209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.825+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c4cd512cb0 0x55c4cd523f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.825+0000 7f58fb67a740 1 --2- 172.21.15.18:0/1236476713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4cd512390 0x55c4cd5239f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.825+0000 7f58fb67a740 1 -- 172.21.15.18:0/1236476713 >> 172.21.15.18:0/1236476713 conn(0x55c4cd50cac0 msgr2=0x55c4cd519200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.825+0000 7f58d3400640 0 client.30290 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:30.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.827+0000 7f58fb67a740 1 -- 172.21.15.18:0/1236476713 shutdown_connections 2024-04-16T17:22:30.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.827+0000 7f58fb67a740 1 -- 172.21.15.18:0/1236476713 wait complete. 2024-04-16T17:22:30.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.827+0000 7f58fb67a740 20 client.30290 trim_cache size 0 max 16384 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (330 inodes in the stack)", 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout: "0db4c055-c85c-4da6-9fb6-3f237145695c": { 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "0db4c055-c85c-4da6-9fb6-3f237145695c", 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:22:30.828 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:22:30.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.828+0000 7f58f3200640 1 -- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58d80a3ef0 msgr2=0x7f58d80a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.828+0000 7f58f3200640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58d80a3ef0 0x7f58d80a63b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55c4cd511e60 tx=0x7f58e400b390 comp rx=0 tx=0).stop 2024-04-16T17:22:30.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.828+0000 7f58f3200640 1 -- 172.21.15.18:0/831072463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 msgr2=0x7f58f410df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:30.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.828+0000 7f58f3200640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 0x7f58f410df20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b450 tx=0x7f58e000b910 comp rx=0 tx=0).stop 2024-04-16T17:22:30.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.829+0000 7f58f3200640 1 -- 172.21.15.18:0/831072463 shutdown_connections 2024-04-16T17:22:30.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.829+0000 7f58f3200640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58d80a3ef0 0x7f58d80a63b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.829+0000 7f58f3200640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f410e460 0x7f58f410e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.829+0000 7f58f3200640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58f4109350 0x7f58f410df20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.829+0000 7f58f3200640 1 --2- 172.21.15.18:0/831072463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58f4104280 0x7f58f4110e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:30.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.829+0000 7f58f3200640 1 -- 172.21.15.18:0/831072463 >> 172.21.15.18:0/831072463 conn(0x7f58f40ffae0 msgr2=0x7f58f4101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:30.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.831+0000 7f58f3200640 1 -- 172.21.15.18:0/831072463 shutdown_connections 2024-04-16T17:22:30.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:22:30.831+0000 7f58f3200640 1 -- 172.21.15.18:0/831072463 wait complete. 2024-04-16T17:22:30.863 INFO:tasks.cephfs.filesystem:scrub status for tag:0db4c055-c85c-4da6-9fb6-3f237145695c - {'path': '/', 'tag': '0db4c055-c85c-4da6-9fb6-3f237145695c', 'options': 'recursive,force'} 2024-04-16T17:22:31.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:30 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:22:31.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:30 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:31.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:30 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:22:31.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:30 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:31.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 68871 12.16 MB/sec execute 163 sec latency 180.446 ms 2024-04-16T17:22:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:31 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:22:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:31 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:31 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:22:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:31 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:22:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:31 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:32.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:31 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:22:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:31 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:32.002 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:31 smithi043 ceph-mon[36901]: pgmap v567: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 325 op/s 2024-04-16T17:22:32.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:31 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:22:32.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:31 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:32.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:31 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:22:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:31 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:22:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:31 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:31 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:22:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:31 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:32.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:31 smithi022 ceph-mon[36023]: pgmap v567: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 325 op/s 2024-04-16T17:22:32.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 69298 12.23 MB/sec execute 164 sec latency 155.391 ms 2024-04-16T17:22:33.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:32 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:33.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:32 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:33.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 69665 12.31 MB/sec execute 165 sec latency 172.800 ms 2024-04-16T17:22:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:33 smithi043 ceph-mon[36901]: pgmap v568: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-04-16T17:22:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:33 smithi043 ceph-mon[36901]: osdmap e126: 12 total, 12 up, 12 in 2024-04-16T17:22:34.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:33 smithi022 ceph-mon[36023]: pgmap v568: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-04-16T17:22:34.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:33 smithi022 ceph-mon[36023]: osdmap e126: 12 total, 12 up, 12 in 2024-04-16T17:22:34.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 70033 12.36 MB/sec execute 166 sec latency 179.267 ms 2024-04-16T17:22:35.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 70347 12.38 MB/sec execute 167 sec latency 157.568 ms 2024-04-16T17:22:36.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:35 smithi043 ceph-mon[36901]: pgmap v570: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 412 op/s 2024-04-16T17:22:36.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:35 smithi022 ceph-mon[36023]: pgmap v570: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 412 op/s 2024-04-16T17:22:36.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 70644 12.38 MB/sec execute 168 sec latency 181.457 ms 2024-04-16T17:22:37.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 70967 12.43 MB/sec execute 169 sec latency 182.570 ms 2024-04-16T17:22:37.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:29 smithi018 ceph-mon[31776]: osdmap e125: 12 total, 12 up, 12 in 2024-04-16T17:22:37.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:29 smithi018 ceph-mon[31776]: pgmap v566: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 407 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-04-16T17:22:37.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:30 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:22:37.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:30 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:37.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:31 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:22:37.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:31 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:37.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:31 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:31 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:31 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:31 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:31 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:31 smithi018 ceph-mon[31776]: pgmap v567: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 325 op/s 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:32 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:33 smithi018 ceph-mon[31776]: pgmap v568: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:33 smithi018 ceph-mon[31776]: osdmap e126: 12 total, 12 up, 12 in 2024-04-16T17:22:37.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:35 smithi018 ceph-mon[31776]: pgmap v570: 129 pgs: 129 active+clean; 401 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 412 op/s 2024-04-16T17:22:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:37 smithi022 ceph-mon[36023]: pgmap v571: 129 pgs: 129 active+clean; 414 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 568 op/s 2024-04-16T17:22:38.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:37 smithi043 ceph-mon[36901]: pgmap v571: 129 pgs: 129 active+clean; 414 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 568 op/s 2024-04-16T17:22:38.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:37 smithi018 ceph-mon[31776]: pgmap v571: 129 pgs: 129 active+clean; 414 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 568 op/s 2024-04-16T17:22:38.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 71114 12.42 MB/sec execute 170 sec latency 381.829 ms 2024-04-16T17:22:39.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 71380 12.41 MB/sec execute 171 sec latency 201.104 ms 2024-04-16T17:22:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:39 smithi022 ceph-mon[36023]: pgmap v572: 129 pgs: 129 active+clean; 418 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-16T17:22:40.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:39 smithi043 ceph-mon[36901]: pgmap v572: 129 pgs: 129 active+clean; 418 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-16T17:22:40.522 INFO:tasks.workunit.client.0.smithi018.stdout: 10 71758 12.46 MB/sec execute 172 sec latency 174.495 ms 2024-04-16T17:22:41.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 72097 12.48 MB/sec execute 173 sec latency 174.866 ms 2024-04-16T17:22:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.157+0000 7ff9f2800640 1 -- 172.21.15.18:0/2186047258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 msgr2=0x7ff9ec10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.157+0000 7ff9f2800640 1 --2- 172.21.15.18:0/2186047258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 0x7ff9ec10ef00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009900 tx=0x7ff9e402bc30 comp rx=0 tx=0).stop 2024-04-16T17:22:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.157+0000 7ff9f2800640 1 -- 172.21.15.18:0/2186047258 shutdown_connections 2024-04-16T17:22:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.157+0000 7ff9f2800640 1 --2- 172.21.15.18:0/2186047258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 0x7ff9ec10ef00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.157+0000 7ff9f2800640 1 --2- 172.21.15.18:0/2186047258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9ec102c10 0x7ff9ec106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.157+0000 7ff9f2800640 1 --2- 172.21.15.18:0/2186047258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9ec102260 0x7ff9ec102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.157+0000 7ff9f2800640 1 -- 172.21.15.18:0/2186047258 >> 172.21.15.18:0/2186047258 conn(0x7ff9ec0fdac0 msgr2=0x7ff9ec0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.157+0000 7ff9f2800640 1 -- 172.21.15.18:0/2186047258 shutdown_connections 2024-04-16T17:22:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.158+0000 7ff9f2800640 1 -- 172.21.15.18:0/2186047258 wait complete. 2024-04-16T17:22:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.159+0000 7ff9f2800640 1 Processor -- start 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.159+0000 7ff9f2800640 1 -- start start 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.159+0000 7ff9f2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9ec102260 0x7ff9ec19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.159+0000 7ff9f2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 0x7ff9ec19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.159+0000 7ff9f2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9ec198710 0x7ff9ec198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9f2800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1123d0 con 0x7ff9ec102260 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9f2800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec112250 con 0x7ff9ec198710 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9f2800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec112550 con 0x7ff9ec107330 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 0x7ff9ec19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 0x7ff9ec19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52250/0 (socket says 172.21.15.18:52250) 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9e2a00640 1 -- 172.21.15.18:0/436901862 learned_addr learned my addr 172.21.15.18:0/436901862 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9eb400640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9ec102260 0x7ff9ec19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9ec198710 0x7ff9ec198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9e2a00640 1 -- 172.21.15.18:0/436901862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9ec198710 msgr2=0x7ff9ec198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9e2a00640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9ec198710 0x7ff9ec198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9e2a00640 1 -- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9ec102260 msgr2=0x7ff9ec19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9e2a00640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9ec102260 0x7ff9ec19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.160+0000 7ff9e2a00640 1 -- 172.21.15.18:0/436901862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec199310 con 0x7ff9ec107330 2024-04-16T17:22:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.161+0000 7ff9e2a00640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 0x7ff9ec19eb40 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400d770 tx=0x7ff9d400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.161+0000 7ff9e8c00640 1 -- 172.21.15.18:0/436901862 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4017070 con 0x7ff9ec107330 2024-04-16T17:22:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.161+0000 7ff9f2800640 1 -- 172.21.15.18:0/436901862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec1a53a0 con 0x7ff9ec107330 2024-04-16T17:22:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.161+0000 7ff9e8c00640 1 -- 172.21.15.18:0/436901862 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4009c30 con 0x7ff9ec107330 2024-04-16T17:22:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.161+0000 7ff9e8c00640 1 -- 172.21.15.18:0/436901862 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400f620 con 0x7ff9ec107330 2024-04-16T17:22:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.161+0000 7ff9f2800640 1 -- 172.21.15.18:0/436901862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec1a57b0 con 0x7ff9ec107330 2024-04-16T17:22:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.166+0000 7ff9f55f6740 1 Processor -- start 2024-04-16T17:22:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.166+0000 7ff9f55f6740 1 -- start start 2024-04-16T17:22:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.166+0000 7ff9e8c00640 1 -- 172.21.15.18:0/436901862 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff9d401a070 con 0x7ff9ec107330 2024-04-16T17:22:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61310 0x5606dea616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea626d0 0x5606dea75a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5606de9c6780 con 0x5606dea626d0 2024-04-16T17:22:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5606de96cf40 con 0x5606dea61310 2024-04-16T17:22:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5606de9c2740 con 0x5606dea61c30 2024-04-16T17:22:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61310 0x5606dea616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61310 0x5606dea616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51002/0 (socket says 172.21.15.18:51002) 2024-04-16T17:22:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9ebe00640 1 -- 172.21.15.18:0/4011285401 learned_addr learned my addr 172.21.15.18:0/4011285401 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9eb400640 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.167+0000 7ff9e2a00640 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea626d0 0x5606dea75a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.168+0000 7ff9ebe00640 1 -- 172.21.15.18:0/4011285401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 msgr2=0x5606dea620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.168+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.168+0000 7ff9ebe00640 1 -- 172.21.15.18:0/4011285401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea626d0 msgr2=0x5606dea75a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.168+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea626d0 0x5606dea75a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.168+0000 7ff9ebe00640 1 -- 172.21.15.18:0/4011285401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606de6d9210 con 0x5606dea61310 2024-04-16T17:22:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.168+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61310 0x5606dea616f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff9e402cfd0 tx=0x7ff9e4005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660b3cc63f5a9426 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.168+0000 7ff9e8c00640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9d00a3e10 0x7ff9d00a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.169+0000 7ff9e8c00640 1 -- 172.21.15.18:0/436901862 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7ff9d40d05a0 con 0x7ff9ec107330 2024-04-16T17:22:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.169+0000 7ff9e0c00640 1 -- 172.21.15.18:0/4011285401 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4042070 con 0x5606dea61310 2024-04-16T17:22:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.169+0000 7ff9e0c00640 1 -- 172.21.15.18:0/4011285401 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e402ca80 con 0x5606dea61310 2024-04-16T17:22:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.169+0000 7ff9e0c00640 1 -- 172.21.15.18:0/4011285401 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4004f10 con 0x5606dea61310 2024-04-16T17:22:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.169+0000 7ff9eb400640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9d00a3e10 0x7ff9d00a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.170+0000 7ff9f55f6740 1 -- 172.21.15.18:0/4011285401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61310 msgr2=0x5606dea616f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.170+0000 7ff9eb400640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9d00a3e10 0x7ff9d00a62d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5606dea62d70 tx=0x7ff9dc009760 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.170+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61310 0x5606dea616f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff9e402cfd0 tx=0x7ff9e4005dd0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.171+0000 7ff9f55f6740 1 -- 172.21.15.18:0/4011285401 shutdown_connections 2024-04-16T17:22:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.171+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea626d0 0x5606dea75a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.171+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.171+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/4011285401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61310 0x5606dea616f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.171+0000 7ff9f55f6740 1 -- 172.21.15.18:0/4011285401 >> 172.21.15.18:0/4011285401 conn(0x5606de9fc5d0 msgr2=0x5606dea5c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.171+0000 7ff9f55f6740 1 -- 172.21.15.18:0/4011285401 shutdown_connections 2024-04-16T17:22:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.171+0000 7ff9f55f6740 1 -- 172.21.15.18:0/4011285401 wait complete. 2024-04-16T17:22:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.172+0000 7ff9f55f6740 1 -- start start 2024-04-16T17:22:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.172+0000 7ff9f55f6740 1 Processor -- start 2024-04-16T17:22:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:41 smithi022 ceph-mon[36023]: pgmap v573: 129 pgs: 129 active+clean; 427 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 342 op/s 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.173+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea75a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.173+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea72dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.173+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea73510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.173+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5606dea76320 con 0x5606dea61310 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.173+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5606de980390 con 0x5606dea61c30 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.173+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5606dea764a0 con 0x5606dea626d0 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.173+0000 7ff9f55f6740 10 client.0 _ll_register_callbacks cb 0x5606de969f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.173+0000 7ff9ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea73510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea73510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52272/0 (socket says 172.21.15.18:52272) 2024-04-16T17:22:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9ebe00640 1 -- 172.21.15.18:0/1753241669 learned_addr learned my addr 172.21.15.18:0/1753241669 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9ebe00640 1 -- 172.21.15.18:0/1753241669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 msgr2=0x5606dea72dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea72dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9ebe00640 1 -- 172.21.15.18:0/1753241669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 msgr2=0x5606dea75a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea75a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9ebe00640 1 -- 172.21.15.18:0/1753241669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606de7a5620 con 0x5606dea626d0 2024-04-16T17:22:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea73510 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40096c0 tx=0x7ff9e402c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a79ea200a26d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.174+0000 7ff9d9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.175+0000 7ff9d9600640 1 -- 172.21.15.18:0/1753241669 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4042040 con 0x5606dea626d0 2024-04-16T17:22:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.175+0000 7ff9d9600640 1 -- 172.21.15.18:0/1753241669 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7ff9e4004450 con 0x5606dea626d0 2024-04-16T17:22:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.175+0000 7ff9f55f6740 1 -- 172.21.15.18:0/1753241669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e402ca80 con 0x5606dea626d0 2024-04-16T17:22:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.175+0000 7ff9d9600640 1 -- 172.21.15.18:0/1753241669 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff9e4004f10 con 0x5606dea626d0 2024-04-16T17:22:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.175+0000 7ff9f55f6740 10 client.30160 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.175+0000 7ff9f55f6740 1 -- 172.21.15.18:0/1753241669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5606dea76320 con 0x5606dea626d0 2024-04-16T17:22:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.175+0000 7ff9d9600640 1 -- 172.21.15.18:0/1753241669 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e402ca80 con 0x5606dea626d0 2024-04-16T17:22:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.176+0000 7ff9d9600640 1 -- 172.21.15.18:0/1753241669 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff9e4034440 con 0x5606dea626d0 2024-04-16T17:22:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.177+0000 7ff9f55f6740 10 client.30160 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.177+0000 7ff9f55f6740 10 client.30160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.177+0000 7ff9f55f6740 20 client.30160 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.177+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606dea835e0 0x5606dea859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.177+0000 7ff9f55f6740 4 client.30160 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.177+0000 7ff9f55f6740 1 -- 172.21.15.18:0/1753241669 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606de9d41a0 con 0x5606dea835e0 2024-04-16T17:22:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.177+0000 7ff9eb400640 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606dea835e0 0x5606dea859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.178+0000 7ff9eb400640 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606dea835e0 0x5606dea859d0 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39f3598a475f124 server_cookie=67611a8192a3a038 in_seq=0 out_seq=0 2024-04-16T17:22:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.178+0000 7ff9d9600640 10 client.30160 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.179+0000 7ff9d9600640 1 -- 172.21.15.18:0/1753241669 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606de9d41a0 con 0x5606dea835e0 2024-04-16T17:22:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.179+0000 7ff9d9600640 10 client.30160 handle_command_reply: tid=0 2024-04-16T17:22:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.180+0000 7ff9f55f6740 1 client.30160 shutdown 2024-04-16T17:22:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9f55f6740 1 -- 172.21.15.18:0/1753241669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 msgr2=0x5606dea73510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea73510 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40096c0 tx=0x7ff9e402c9c0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9f55f6740 1 -- 172.21.15.18:0/1753241669 shutdown_connections 2024-04-16T17:22:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606dea835e0 0x5606dea859d0 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea73510 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea72dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/1753241669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea75a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9d9600640 0 client.30160 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.181+0000 7ff9f55f6740 1 -- 172.21.15.18:0/1753241669 >> 172.21.15.18:0/1753241669 conn(0x5606de9fc5d0 msgr2=0x5606dea5e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.182+0000 7ff9f55f6740 1 -- 172.21.15.18:0/1753241669 shutdown_connections 2024-04-16T17:22:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.182+0000 7ff9f55f6740 1 -- 172.21.15.18:0/1753241669 wait complete. 2024-04-16T17:22:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.182+0000 7ff9f55f6740 20 client.30160 trim_cache size 0 max 16384 2024-04-16T17:22:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.222+0000 7ff9f55f6740 1 Processor -- start 2024-04-16T17:22:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.222+0000 7ff9f55f6740 1 -- start start 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.223+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea7f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.223+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.223+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea8b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.223+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1afc80 con 0x5606dea61310 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.223+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1ad550 con 0x5606dea61c30 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.223+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1ab3c0 con 0x5606dea626d0 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.224+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.224+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51010/0 (socket says 172.21.15.18:51010) 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.224+0000 7ff9e2a00640 1 -- 172.21.15.18:0/3558103105 learned_addr learned my addr 172.21.15.18:0/3558103105 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.224+0000 7ff9e2a00640 1 -- 172.21.15.18:0/3558103105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 msgr2=0x5606dea8b0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.224+0000 7ff9e2a00640 1 --2- 172.21.15.18:0/3558103105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea8b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.224+0000 7ff9e2a00640 1 -- 172.21.15.18:0/3558103105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 msgr2=0x5606dea7f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.224+0000 7ff9e2a00640 1 --2- 172.21.15.18:0/3558103105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea7f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.224+0000 7ff9e2a00640 1 -- 172.21.15.18:0/3558103105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606dea61a60 con 0x5606dea61c30 2024-04-16T17:22:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.225+0000 7ff9e2a00640 1 --2- 172.21.15.18:0/3558103105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea709f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff9d409c140 tx=0x7ff9d40d3b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4e6b5f399f656b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.225+0000 7ff9dbe00640 1 -- 172.21.15.18:0/3558103105 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4021040 con 0x5606dea61c30 2024-04-16T17:22:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.225+0000 7ff9dbe00640 1 -- 172.21.15.18:0/3558103105 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d40d3e20 con 0x5606dea61c30 2024-04-16T17:22:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.225+0000 7ff9dbe00640 1 -- 172.21.15.18:0/3558103105 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40a0120 con 0x5606dea61c30 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.227+0000 7ff9f55f6740 1 -- 172.21.15.18:0/3558103105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 msgr2=0x5606dea709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.227+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/3558103105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea709f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff9d409c140 tx=0x7ff9d40d3b00 comp rx=0 tx=0).stop 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.229+0000 7ff9f55f6740 1 -- 172.21.15.18:0/3558103105 shutdown_connections 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.229+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/3558103105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea626d0 0x5606dea8b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.229+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/3558103105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea61c30 0x5606dea709f0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.229+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/3558103105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea7f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.229+0000 7ff9f55f6740 1 -- 172.21.15.18:0/3558103105 >> 172.21.15.18:0/3558103105 conn(0x5606de9fc5d0 msgr2=0x5606dea649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.230+0000 7ff9f55f6740 1 -- 172.21.15.18:0/3558103105 shutdown_connections 2024-04-16T17:22:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.230+0000 7ff9f55f6740 1 -- 172.21.15.18:0/3558103105 wait complete. 2024-04-16T17:22:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.231+0000 7ff9f55f6740 1 -- start start 2024-04-16T17:22:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.231+0000 7ff9f55f6740 1 Processor -- start 2024-04-16T17:22:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.232+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.232+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea74890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.232+0000 7ff9f55f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea626d0 0x5606dea74dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.233+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea74890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.233+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea74890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52298/0 (socket says 172.21.15.18:52298) 2024-04-16T17:22:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.233+0000 7ff9f55f6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1aba10 con 0x5606dea61310 2024-04-16T17:22:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.233+0000 7ff9e2a00640 1 -- 172.21.15.18:0/2076590894 learned_addr learned my addr 172.21.15.18:0/2076590894 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.233+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1ace80 con 0x5606dea626d0 2024-04-16T17:22:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.233+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1ac0e0 con 0x5606dea61c30 2024-04-16T17:22:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.233+0000 7ff9f55f6740 10 client.0 _ll_register_callbacks cb 0x7ff9e40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.233+0000 7ff9ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea626d0 0x5606dea74dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.234+0000 7ff9ebe00640 1 -- 172.21.15.18:0/2076590894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 msgr2=0x5606dea74890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.234+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea74890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.234+0000 7ff9ebe00640 1 -- 172.21.15.18:0/2076590894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 msgr2=0x5606dea7bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.234+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.234+0000 7ff9ebe00640 1 -- 172.21.15.18:0/2076590894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606dea755e0 con 0x5606dea626d0 2024-04-16T17:22:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.234+0000 7ff9ebe00640 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea626d0 0x5606dea74dd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff9e402c170 tx=0x7ff9e402c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d348eaf6e353b99d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.234+0000 7ff9d9600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:22:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.235+0000 7ff9d9600640 1 -- 172.21.15.18:0/2076590894 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4042040 con 0x5606dea626d0 2024-04-16T17:22:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.235+0000 7ff9d9600640 1 -- 172.21.15.18:0/2076590894 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7ff9e4004250 con 0x5606dea626d0 2024-04-16T17:22:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.236+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d40d3e20 con 0x5606dea626d0 2024-04-16T17:22:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9d9600640 1 -- 172.21.15.18:0/2076590894 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff9e4034b70 con 0x5606dea626d0 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9f55f6740 10 client.30305 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5606deae99f0 con 0x5606dea626d0 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9d9600640 1 -- 172.21.15.18:0/2076590894 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d40d3e20 con 0x5606dea626d0 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9d9600640 1 -- 172.21.15.18:0/2076590894 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff9e4003710 con 0x5606dea626d0 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9f55f6740 10 client.30305 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9f55f6740 10 client.30305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9f55f6740 20 client.30305 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606deae9f30 0x5606deaec320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9f55f6740 4 client.30305 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.237+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606de9ce010 con 0x5606deae9f30 2024-04-16T17:22:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.238+0000 7ff9eb400640 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606deae9f30 0x5606deaec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.238+0000 7ff9d9600640 10 client.30305 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:42.239+0000 7ff9eb400640 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606deae9f30 0x5606deaec320 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d77d819583aea58 server_cookie=219956342d702ccb in_seq=0 out_seq=0 2024-04-16T17:22:42.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:41 smithi043 ceph-mon[36901]: pgmap v573: 129 pgs: 129 active+clean; 427 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 342 op/s 2024-04-16T17:22:42.348 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:39 smithi018 ceph-mon[31776]: pgmap v572: 129 pgs: 129 active+clean; 418 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-04-16T17:22:42.348 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:41 smithi018 ceph-mon[31776]: pgmap v573: 129 pgs: 129 active+clean; 427 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 342 op/s 2024-04-16T17:22:42.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 72390 12.46 MB/sec execute 174 sec latency 313.097 ms 2024-04-16T17:22:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.423+0000 7ff9d9600640 1 -- 172.21.15.18:0/2076590894 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606de9ce010 con 0x5606deae9f30 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.424+0000 7ff9d9600640 10 client.30305 handle_command_reply: tid=0 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.424+0000 7ff9f55f6740 1 client.30305 shutdown 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.424+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea626d0 msgr2=0x5606dea74dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.424+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea626d0 0x5606dea74dd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff9e402c170 tx=0x7ff9e402c7e0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.424+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 shutdown_connections 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.424+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5606deae9f30 0x5606deaec320 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.424+0000 7ff9d9600640 0 client.30305 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.424+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5606dea626d0 0x5606dea74dd0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.425+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606dea61c30 0x5606dea74890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.425+0000 7ff9f55f6740 1 --2- 172.21.15.18:0/2076590894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5606dea61310 0x5606dea7bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.425+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 >> 172.21.15.18:0/2076590894 conn(0x5606de9fc5d0 msgr2=0x5606dea5d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.425+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 shutdown_connections 2024-04-16T17:22:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.425+0000 7ff9f55f6740 1 -- 172.21.15.18:0/2076590894 wait complete. 2024-04-16T17:22:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.425+0000 7ff9f55f6740 20 client.30305 trim_cache size 0 max 16384 2024-04-16T17:22:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:22:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:22:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:22:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:22:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.433+0000 7ff9f2800640 1 -- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9d00a3e10 msgr2=0x7ff9d00a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.433+0000 7ff9f2800640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9d00a3e10 0x7ff9d00a62d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5606dea62d70 tx=0x7ff9dc009760 comp rx=0 tx=0).stop 2024-04-16T17:22:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.433+0000 7ff9f2800640 1 -- 172.21.15.18:0/436901862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 msgr2=0x7ff9ec19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.433+0000 7ff9f2800640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 0x7ff9ec19eb40 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400d770 tx=0x7ff9d400dc30 comp rx=0 tx=0).stop 2024-04-16T17:22:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.434+0000 7ff9f2800640 1 -- 172.21.15.18:0/436901862 shutdown_connections 2024-04-16T17:22:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.434+0000 7ff9f2800640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff9d00a3e10 0x7ff9d00a62d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.434+0000 7ff9f2800640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9ec198710 0x7ff9ec198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.434+0000 7ff9f2800640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9ec107330 0x7ff9ec19eb40 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.434+0000 7ff9f2800640 1 --2- 172.21.15.18:0/436901862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9ec102260 0x7ff9ec19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.434+0000 7ff9f2800640 1 -- 172.21.15.18:0/436901862 >> 172.21.15.18:0/436901862 conn(0x7ff9ec0fdac0 msgr2=0x7ff9ec0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.434+0000 7ff9f2800640 1 -- 172.21.15.18:0/436901862 shutdown_connections 2024-04-16T17:22:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:43.434+0000 7ff9f2800640 1 -- 172.21.15.18:0/436901862 wait complete. 2024-04-16T17:22:43.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 72420 12.39 MB/sec execute 175 sec latency 1032.935 ms 2024-04-16T17:22:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:43 smithi022 ceph-mon[36023]: pgmap v574: 129 pgs: 129 active+clean; 431 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 291 op/s 2024-04-16T17:22:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:43 smithi043 ceph-mon[36901]: pgmap v574: 129 pgs: 129 active+clean; 431 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 291 op/s 2024-04-16T17:22:44.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 72642 12.41 MB/sec execute 176 sec latency 1073.867 ms 2024-04-16T17:22:45.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:43 smithi018 ceph-mon[31776]: pgmap v574: 129 pgs: 129 active+clean; 431 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 291 op/s 2024-04-16T17:22:45.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:44 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:22:45.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:44 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:22:45.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:44 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:22:45.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:44 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:22:45.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:44 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:22:45.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:44 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:22:45.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 73049 12.49 MB/sec execute 177 sec latency 191.861 ms 2024-04-16T17:22:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:45 smithi022 ceph-mon[36023]: pgmap v575: 129 pgs: 129 active+clean; 432 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-04-16T17:22:46.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:45 smithi043 ceph-mon[36901]: pgmap v575: 129 pgs: 129 active+clean; 432 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-04-16T17:22:46.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 73515 12.59 MB/sec execute 178 sec latency 96.428 ms 2024-04-16T17:22:47.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 73772 12.61 MB/sec execute 179 sec latency 56.765 ms 2024-04-16T17:22:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:47 smithi022 ceph-mon[36023]: pgmap v576: 129 pgs: 129 active+clean; 422 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-04-16T17:22:48.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:47 smithi043 ceph-mon[36901]: pgmap v576: 129 pgs: 129 active+clean; 422 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-04-16T17:22:48.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 74040 12.61 MB/sec execute 180 sec latency 146.803 ms 2024-04-16T17:22:49.523 INFO:tasks.workunit.client.0.smithi018.stdout: 10 74244 12.58 MB/sec execute 181 sec latency 205.450 ms 2024-04-16T17:22:50.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:49 smithi022 ceph-mon[36023]: pgmap v577: 129 pgs: 129 active+clean; 415 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 416 op/s 2024-04-16T17:22:50.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:49 smithi043 ceph-mon[36901]: pgmap v577: 129 pgs: 129 active+clean; 415 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 416 op/s 2024-04-16T17:22:50.524 INFO:tasks.workunit.client.0.smithi018.stdout: 10 74467 12.57 MB/sec execute 182 sec latency 263.253 ms 2024-04-16T17:22:51.525 INFO:tasks.workunit.client.0.smithi018.stdout: 10 74729 12.58 MB/sec execute 183 sec latency 159.695 ms 2024-04-16T17:22:51.892 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:44 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:22:51.892 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:44 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:22:51.892 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:44 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:22:51.892 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:45 smithi018 ceph-mon[31776]: pgmap v575: 129 pgs: 129 active+clean; 432 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-04-16T17:22:51.892 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:47 smithi018 ceph-mon[31776]: pgmap v576: 129 pgs: 129 active+clean; 422 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-04-16T17:22:51.892 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:49 smithi018 ceph-mon[31776]: pgmap v577: 129 pgs: 129 active+clean; 415 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 416 op/s 2024-04-16T17:22:52.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:51 smithi022 ceph-mon[36023]: pgmap v578: 129 pgs: 129 active+clean; 426 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 481 op/s 2024-04-16T17:22:52.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:51 smithi043 ceph-mon[36901]: pgmap v578: 129 pgs: 129 active+clean; 426 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 481 op/s 2024-04-16T17:22:52.524 INFO:tasks.workunit.client.0.smithi018.stdout: 10 75089 12.64 MB/sec execute 184 sec latency 162.564 ms 2024-04-16T17:22:53.315 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:51 smithi018 ceph-mon[31776]: pgmap v578: 129 pgs: 129 active+clean; 426 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 481 op/s 2024-04-16T17:22:53.524 INFO:tasks.workunit.client.0.smithi018.stdout: 10 75325 12.64 MB/sec execute 185 sec latency 311.526 ms 2024-04-16T17:22:54.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:53 smithi022 ceph-mon[36023]: pgmap v579: 129 pgs: 129 active+clean; 423 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2024-04-16T17:22:54.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:53 smithi043 ceph-mon[36901]: pgmap v579: 129 pgs: 129 active+clean; 423 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2024-04-16T17:22:54.525 INFO:tasks.workunit.client.0.smithi018.stdout: 10 75525 12.59 MB/sec execute 186 sec latency 321.247 ms 2024-04-16T17:22:55.502 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:53 smithi018 ceph-mon[31776]: pgmap v579: 129 pgs: 129 active+clean; 423 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2024-04-16T17:22:55.526 INFO:tasks.workunit.client.0.smithi018.stdout: 10 75711 12.56 MB/sec execute 187 sec latency 246.725 ms 2024-04-16T17:22:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:55 smithi022 ceph-mon[36023]: pgmap v580: 129 pgs: 129 active+clean; 427 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2024-04-16T17:22:56.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:55 smithi043 ceph-mon[36901]: pgmap v580: 129 pgs: 129 active+clean; 427 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2024-04-16T17:22:56.526 INFO:tasks.workunit.client.0.smithi018.stdout: 10 76071 12.60 MB/sec execute 188 sec latency 177.512 ms 2024-04-16T17:22:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 -- 172.21.15.18:0/3061719488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f740743a0 msgr2=0x7f1f74074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 --2- 172.21.15.18:0/3061719488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f740743a0 0x7f1f74074780 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009900 tx=0x7f1f6402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 -- 172.21.15.18:0/3061719488 shutdown_connections 2024-04-16T17:22:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 --2- 172.21.15.18:0/3061719488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f74112e30 0x7f1f74115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 --2- 172.21.15.18:0/3061719488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f74074cc0 0x7f1f741128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 --2- 172.21.15.18:0/3061719488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f740743a0 0x7f1f74074780 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 -- 172.21.15.18:0/3061719488 >> 172.21.15.18:0/3061719488 conn(0x7f1f740fd910 msgr2=0x7f1f740ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 -- 172.21.15.18:0/3061719488 shutdown_connections 2024-04-16T17:22:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.605+0000 7f1f73200640 1 -- 172.21.15.18:0/3061719488 wait complete. 2024-04-16T17:22:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.606+0000 7f1f73200640 1 Processor -- start 2024-04-16T17:22:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.607+0000 7f1f73200640 1 -- start start 2024-04-16T17:22:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.607+0000 7f1f73200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740743a0 0x7f1f7419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.607+0000 7f1f73200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f74074cc0 0x7f1f7419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f73200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f74112e30 0x7f1f741a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f73200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1f74118870 con 0x7f1f74074cc0 2024-04-16T17:22:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f73200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741186f0 con 0x7f1f74112e30 2024-04-16T17:22:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f73200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741189f0 con 0x7f1f740743a0 2024-04-16T17:22:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740743a0 0x7f1f7419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740743a0 0x7f1f7419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37182/0 (socket says 172.21.15.18:37182) 2024-04-16T17:22:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f71e00640 1 -- 172.21.15.18:0/1481423403 learned_addr learned my addr 172.21.15.18:0/1481423403 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f72800640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f74112e30 0x7f1f741a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f71e00640 1 -- 172.21.15.18:0/1481423403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f74112e30 msgr2=0x7f1f741a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f71e00640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f74112e30 0x7f1f741a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f71e00640 1 -- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f74074cc0 msgr2=0x7f1f7419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f71e00640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f74074cc0 0x7f1f7419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.608+0000 7f1f71e00640 1 -- 172.21.15.18:0/1481423403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f741a3d20 con 0x7f1f740743a0 2024-04-16T17:22:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.609+0000 7f1f71e00640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740743a0 0x7f1f7419ed30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1f64000c00 tx=0x7f1f64004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.609+0000 7f1f6aa00640 1 -- 172.21.15.18:0/1481423403 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64042020 con 0x7f1f740743a0 2024-04-16T17:22:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.609+0000 7f1f6aa00640 1 -- 172.21.15.18:0/1481423403 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f64005000 con 0x7f1f740743a0 2024-04-16T17:22:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.609+0000 7f1f6aa00640 1 -- 172.21.15.18:0/1481423403 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64003b60 con 0x7f1f740743a0 2024-04-16T17:22:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.609+0000 7f1f73200640 1 -- 172.21.15.18:0/1481423403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f741a3fb0 con 0x7f1f740743a0 2024-04-16T17:22:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.609+0000 7f1f73200640 1 -- 172.21.15.18:0/1481423403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f741ab8a0 con 0x7f1f740743a0 2024-04-16T17:22:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.616+0000 7f1f6aa00640 1 -- 172.21.15.18:0/1481423403 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1f64045020 con 0x7f1f740743a0 2024-04-16T17:22:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.618+0000 7f1f6aa00640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f580a3e00 0x7f1f580a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.618+0000 7f1f6aa00640 1 -- 172.21.15.18:0/1481423403 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f1f640f37a0 con 0x7f1f740743a0 2024-04-16T17:22:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.620+0000 7f1f71400640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f580a3e00 0x7f1f580a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.621+0000 7f1f7be67740 1 Processor -- start 2024-04-16T17:22:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.622+0000 7f1f7be67740 1 -- start start 2024-04-16T17:22:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.622+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 0x55a667314540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.622+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 0x55a667314f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.622+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a667331400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.622+0000 7f1f71400640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f580a3e00 0x7f1f580a62c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1f600058c0 tx=0x7f1f60005850 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f7be67740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a667279780 con 0x55a667314a80 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f7be67740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a66721ff40 con 0x55a66732f010 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f7be67740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a667275740 con 0x55a6673157f0 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f72800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a667331400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f72800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a667331400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50646/0 (socket says 172.21.15.18:50646) 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f72800640 1 -- 172.21.15.18:0/2844179304 learned_addr learned my addr 172.21.15.18:0/2844179304 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 0x55a667314540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f72800640 1 -- 172.21.15.18:0/2844179304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 msgr2=0x55a667314540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f72800640 1 --2- 172.21.15.18:0/2844179304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 0x55a667314540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f72800640 1 -- 172.21.15.18:0/2844179304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 msgr2=0x55a667314f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f72800640 1 --2- 172.21.15.18:0/2844179304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 0x55a667314f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.623+0000 7f1f72800640 1 -- 172.21.15.18:0/2844179304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a666f8c210 con 0x55a66732f010 2024-04-16T17:22:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.624+0000 7f1f72800640 1 --2- 172.21.15.18:0/2844179304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a667331400 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009f00 tx=0x7f1f6c0327b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e827656d4468429 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.624+0000 7f1f5f400640 1 -- 172.21.15.18:0/2844179304 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c004d20 con 0x55a66732f010 2024-04-16T17:22:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.624+0000 7f1f5f400640 1 -- 172.21.15.18:0/2844179304 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c037070 con 0x55a66732f010 2024-04-16T17:22:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.624+0000 7f1f5f400640 1 -- 172.21.15.18:0/2844179304 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0339b0 con 0x55a66732f010 2024-04-16T17:22:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 -- 172.21.15.18:0/2844179304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 msgr2=0x55a667331400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2844179304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a667331400 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009f00 tx=0x7f1f6c0327b0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 -- 172.21.15.18:0/2844179304 shutdown_connections 2024-04-16T17:22:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2844179304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a667331400 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2844179304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 0x55a667314f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2844179304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 0x55a667314540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 -- 172.21.15.18:0/2844179304 >> 172.21.15.18:0/2844179304 conn(0x55a6672af5d0 msgr2=0x55a66730f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 -- 172.21.15.18:0/2844179304 shutdown_connections 2024-04-16T17:22:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.625+0000 7f1f7be67740 1 -- 172.21.15.18:0/2844179304 wait complete. 2024-04-16T17:22:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.626+0000 7f1f7be67740 1 -- start start 2024-04-16T17:22:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.627+0000 7f1f7be67740 1 Processor -- start 2024-04-16T17:22:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 0x55a667320100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 0x55a667320640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a66731b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f7be67740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a6673161e0 con 0x55a667314a80 2024-04-16T17:22:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f7be67740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a667233390 con 0x55a66732f010 2024-04-16T17:22:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f7be67740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a667316360 con 0x55a6673157f0 2024-04-16T17:22:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f7be67740 10 client.0 _ll_register_callbacks cb 0x55a66721cf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 0x55a667320100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 0x55a667320100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46276/0 (socket says 172.21.15.18:46276) 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.628+0000 7f1f71e00640 1 -- 172.21.15.18:0/2817716496 learned_addr learned my addr 172.21.15.18:0/2817716496 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.629+0000 7f1f72800640 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a66731b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.629+0000 7f1f71400640 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 0x55a667320640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.629+0000 7f1f72800640 1 -- 172.21.15.18:0/2817716496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 msgr2=0x55a667320640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.629+0000 7f1f72800640 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 0x55a667320640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.629+0000 7f1f72800640 1 -- 172.21.15.18:0/2817716496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 msgr2=0x55a667320100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.629+0000 7f1f72800640 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 0x55a667320100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.629+0000 7f1f72800640 1 -- 172.21.15.18:0/2817716496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a667058620 con 0x55a66732f010 2024-04-16T17:22:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.630+0000 7f1f72800640 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a66731b500 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c032cb0 tx=0x7f1f6c033e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=208c191899576f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.630+0000 7f1f53e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:22:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.630+0000 7f1f53e00640 1 -- 172.21.15.18:0/2817716496 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c03e690 con 0x55a66732f010 2024-04-16T17:22:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.630+0000 7f1f53e00640 1 -- 172.21.15.18:0/2817716496 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f1f6c03e830 con 0x55a66732f010 2024-04-16T17:22:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.632+0000 7f1f7be67740 1 -- 172.21.15.18:0/2817716496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f6c037070 con 0x55a66732f010 2024-04-16T17:22:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.632+0000 7f1f53e00640 1 -- 172.21.15.18:0/2817716496 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1f6c03ea30 con 0x55a66732f010 2024-04-16T17:22:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.632+0000 7f1f7be67740 10 client.30317 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.632+0000 7f1f7be67740 1 -- 172.21.15.18:0/2817716496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a66732a350 con 0x55a66732f010 2024-04-16T17:22:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.632+0000 7f1f53e00640 1 -- 172.21.15.18:0/2817716496 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c037070 con 0x55a66732f010 2024-04-16T17:22:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.633+0000 7f1f53e00640 1 -- 172.21.15.18:0/2817716496 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f1f6c03c070 con 0x55a66732f010 2024-04-16T17:22:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.634+0000 7f1f7be67740 10 client.30317 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.634+0000 7f1f7be67740 10 client.30317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.634+0000 7f1f7be67740 20 client.30317 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.634+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a66732b760 0x55a66732db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.634+0000 7f1f7be67740 4 client.30317 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:22:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.634+0000 7f1f7be67740 1 -- 172.21.15.18:0/2817716496 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6672871a0 con 0x55a66732b760 2024-04-16T17:22:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.634+0000 7f1f71e00640 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a66732b760 0x55a66732db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.635+0000 7f1f71e00640 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a66732b760 0x55a66732db80 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f01e061a45d11b99 server_cookie=1a2be62c8779aadc in_seq=0 out_seq=0 2024-04-16T17:22:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.635+0000 7f1f53e00640 10 client.30317 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.637+0000 7f1f53e00640 1 -- 172.21.15.18:0/2817716496 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6672871a0 con 0x55a66732b760 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.637+0000 7f1f53e00640 10 client.30317 handle_command_reply: tid=0 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 client.30317 shutdown 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 -- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 msgr2=0x55a66731b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a66731b500 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c032cb0 tx=0x7f1f6c033e80 comp rx=0 tx=0).stop 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 -- 172.21.15.18:0/2817716496 shutdown_connections 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a66732b760 0x55a66732db80 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a66732f010 0x55a66731b500 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6673157f0 0x55a667320640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2817716496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a667314a80 0x55a667320100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 -- 172.21.15.18:0/2817716496 >> 172.21.15.18:0/2817716496 conn(0x55a6672af5d0 msgr2=0x55a6673114c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f53e00640 0 client.30317 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 -- 172.21.15.18:0/2817716496 shutdown_connections 2024-04-16T17:22:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.638+0000 7f1f7be67740 1 -- 172.21.15.18:0/2817716496 wait complete. 2024-04-16T17:22:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.639+0000 7f1f7be67740 20 client.30317 trim_cache size 0 max 16384 2024-04-16T17:22:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.674+0000 7f1f7be67740 1 Processor -- start 2024-04-16T17:22:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.674+0000 7f1f7be67740 1 -- start start 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a667314a80 0x55a66732e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6673157f0 0x55a667318780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a66732f010 0x55a667318cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f7be67740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1f540128b0 con 0x55a6673157f0 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f7be67740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54010180 con 0x55a667314a80 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f7be67740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400dff0 con 0x55a66732f010 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a667314a80 0x55a66732e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a667314a80 0x55a66732e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50674/0 (socket says 172.21.15.18:50674) 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f71e00640 1 -- 172.21.15.18:0/3836084083 learned_addr learned my addr 172.21.15.18:0/3836084083 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f72800640 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a66732f010 0x55a667318cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f72800640 1 -- 172.21.15.18:0/3836084083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a667314a80 msgr2=0x55a66732e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f72800640 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a667314a80 0x55a66732e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.675+0000 7f1f72800640 1 -- 172.21.15.18:0/3836084083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6673157f0 msgr2=0x55a667318780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:22:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.676+0000 7f1f71400640 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6673157f0 0x55a667318780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.677+0000 7f1f72800640 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6673157f0 0x55a667318780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.677+0000 7f1f72800640 1 -- 172.21.15.18:0/3836084083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6673163e0 con 0x55a66732f010 2024-04-16T17:22:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.678+0000 7f1f71400640 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6673157f0 0x55a667318780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:22:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.678+0000 7f1f72800640 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a66732f010 0x55a667318cc0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c049460 tx=0x7f1f6c009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a823751ac0da9f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.679+0000 7f1f5ea00640 1 -- 172.21.15.18:0/3836084083 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c032ce0 con 0x55a66732f010 2024-04-16T17:22:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.679+0000 7f1f5ea00640 1 -- 172.21.15.18:0/3836084083 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c037040 con 0x55a66732f010 2024-04-16T17:22:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.679+0000 7f1f5ea00640 1 -- 172.21.15.18:0/3836084083 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c032ce0 con 0x55a66732f010 2024-04-16T17:22:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.680+0000 7f1f7be67740 1 -- 172.21.15.18:0/3836084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a66732f010 msgr2=0x55a667318cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.680+0000 7f1f7be67740 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a66732f010 0x55a667318cc0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c049460 tx=0x7f1f6c009720 comp rx=0 tx=0).stop 2024-04-16T17:22:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.684+0000 7f1f7be67740 1 -- 172.21.15.18:0/3836084083 shutdown_connections 2024-04-16T17:22:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.684+0000 7f1f7be67740 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a66732f010 0x55a667318cc0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.684+0000 7f1f7be67740 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a6673157f0 0x55a667318780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.684+0000 7f1f7be67740 1 --2- 172.21.15.18:0/3836084083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a667314a80 0x55a66732e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.684+0000 7f1f7be67740 1 -- 172.21.15.18:0/3836084083 >> 172.21.15.18:0/3836084083 conn(0x55a6672af5d0 msgr2=0x55a66732e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.684+0000 7f1f7be67740 1 -- 172.21.15.18:0/3836084083 shutdown_connections 2024-04-16T17:22:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.684+0000 7f1f7be67740 1 -- 172.21.15.18:0/3836084083 wait complete. 2024-04-16T17:22:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.685+0000 7f1f7be67740 1 -- start start 2024-04-16T17:22:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.685+0000 7f1f7be67740 1 Processor -- start 2024-04-16T17:22:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a667314a80 0x55a6673355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6673157f0 0x55a667335b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f7be67740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a66732f010 0x55a667336040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f7be67740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400e640 con 0x55a66732f010 2024-04-16T17:22:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f7be67740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400fab0 con 0x55a6673157f0 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f7be67740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400ed10 con 0x55a667314a80 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f7be67740 10 client.0 _ll_register_callbacks cb 0x7f1f64004f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a667314a80 0x55a6673355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f71e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a667314a80 0x55a6673355c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37218/0 (socket says 172.21.15.18:37218) 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.686+0000 7f1f71e00640 1 -- 172.21.15.18:0/2653511585 learned_addr learned my addr 172.21.15.18:0/2653511585 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f71e00640 1 -- 172.21.15.18:0/2653511585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6673157f0 msgr2=0x55a667335b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f71e00640 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6673157f0 0x55a667335b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f71e00640 1 -- 172.21.15.18:0/2653511585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a66732f010 msgr2=0x55a667336040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f71e00640 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a66732f010 0x55a667336040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f71e00640 1 -- 172.21.15.18:0/2653511585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6673933e0 con 0x55a667314a80 2024-04-16T17:22:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f71e00640 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a667314a80 0x55a6673355c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1f64006010 tx=0x7f1f640bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce77d82082407f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f53e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f53e00640 1 -- 172.21.15.18:0/2653511585 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f640d7070 con 0x55a667314a80 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.687+0000 7f1f53e00640 1 -- 172.21.15.18:0/2653511585 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f1f64003d00 con 0x55a667314a80 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.688+0000 7f1f7be67740 1 -- 172.21.15.18:0/2653511585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f6c037040 con 0x55a667314a80 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.688+0000 7f1f53e00640 1 -- 172.21.15.18:0/2653511585 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1f640c37c0 con 0x55a667314a80 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.688+0000 7f1f7be67740 10 client.30181 fetch_fsmap learned FSMap version 37 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.688+0000 7f1f7be67740 1 -- 172.21.15.18:0/2653511585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a6673946c0 con 0x55a667314a80 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.688+0000 7f1f53e00640 1 -- 172.21.15.18:0/2653511585 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c037040 con 0x55a667314a80 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.688+0000 7f1f53e00640 1 -- 172.21.15.18:0/2653511585 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f1f640f0d30 con 0x55a667314a80 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.688+0000 7f1f7be67740 10 client.30181 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.689+0000 7f1f7be67740 10 client.30181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.689+0000 7f1f7be67740 20 client.30181 populate_metadata read hostname 'smithi018' 2024-04-16T17:22:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.689+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a667394dc0 0x55a667397200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:22:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.689+0000 7f1f7be67740 4 client.30181 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:22:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.689+0000 7f1f7be67740 1 -- 172.21.15.18:0/2653511585 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a667281010 con 0x55a667394dc0 2024-04-16T17:22:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.689+0000 7f1f71400640 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a667394dc0 0x55a667397200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:22:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.690+0000 7f1f71400640 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a667394dc0 0x55a667397200 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78803cc7f42dada9 server_cookie=6bcdd2ddfc6844ff in_seq=0 out_seq=0 2024-04-16T17:22:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:56.690+0000 7f1f53e00640 10 client.30181 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:57.190 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:55 smithi018 ceph-mon[31776]: pgmap v580: 129 pgs: 129 active+clean; 427 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2024-04-16T17:22:57.531 INFO:tasks.workunit.client.0.smithi018.stdout: 10 76270 12.59 MB/sec execute 189 sec latency 689.706 ms 2024-04-16T17:22:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.874+0000 7f1f53e00640 1 -- 172.21.15.18:0/2653511585 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a667281010 con 0x55a667394dc0 2024-04-16T17:22:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.874+0000 7f1f53e00640 10 client.30181 handle_command_reply: tid=0 2024-04-16T17:22:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.874+0000 7f1f7be67740 1 client.30181 shutdown 2024-04-16T17:22:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.876+0000 7f1f7be67740 1 -- 172.21.15.18:0/2653511585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a667314a80 msgr2=0x55a6673355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.876+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a667314a80 0x55a6673355c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1f64006010 tx=0x7f1f640bb040 comp rx=0 tx=0).stop 2024-04-16T17:22:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.878+0000 7f1f7be67740 1 -- 172.21.15.18:0/2653511585 shutdown_connections 2024-04-16T17:22:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.878+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a667394dc0 0x55a667397200 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.878+0000 7f1f53e00640 0 client.30181 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:22:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.878+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a66732f010 0x55a667336040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.878+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6673157f0 0x55a667335b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.878+0000 7f1f7be67740 1 --2- 172.21.15.18:0/2653511585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a667314a80 0x55a6673355c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.878+0000 7f1f7be67740 1 -- 172.21.15.18:0/2653511585 >> 172.21.15.18:0/2653511585 conn(0x55a6672af5d0 msgr2=0x55a667319dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.879+0000 7f1f7be67740 1 -- 172.21.15.18:0/2653511585 shutdown_connections 2024-04-16T17:22:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.879+0000 7f1f7be67740 1 -- 172.21.15.18:0/2653511585 wait complete. 2024-04-16T17:22:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.879+0000 7f1f7be67740 20 client.30181 trim_cache size 0 max 16384 2024-04-16T17:22:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:22:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:22:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:22:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:22:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.890+0000 7f1f73200640 1 -- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f580a3e00 msgr2=0x7f1f580a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.890+0000 7f1f73200640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f580a3e00 0x7f1f580a62c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1f600058c0 tx=0x7f1f60005850 comp rx=0 tx=0).stop 2024-04-16T17:22:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.890+0000 7f1f73200640 1 -- 172.21.15.18:0/1481423403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740743a0 msgr2=0x7f1f7419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:22:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.890+0000 7f1f73200640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740743a0 0x7f1f7419ed30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1f64000c00 tx=0x7f1f64004360 comp rx=0 tx=0).stop 2024-04-16T17:22:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.891+0000 7f1f73200640 1 -- 172.21.15.18:0/1481423403 shutdown_connections 2024-04-16T17:22:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.891+0000 7f1f73200640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f1f580a3e00 0x7f1f580a62c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.891+0000 7f1f73200640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f74112e30 0x7f1f741a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.891+0000 7f1f73200640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f74074cc0 0x7f1f7419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.892+0000 7f1f73200640 1 --2- 172.21.15.18:0/1481423403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740743a0 0x7f1f7419ed30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:22:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.892+0000 7f1f73200640 1 -- 172.21.15.18:0/1481423403 >> 172.21.15.18:0/1481423403 conn(0x7f1f740fd910 msgr2=0x7f1f740fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:22:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.893+0000 7f1f73200640 1 -- 172.21.15.18:0/1481423403 shutdown_connections 2024-04-16T17:22:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:22:57.894+0000 7f1f73200640 1 -- 172.21.15.18:0/1481423403 wait complete. 2024-04-16T17:22:58.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:57 smithi022 ceph-mon[36023]: pgmap v581: 129 pgs: 129 active+clean; 434 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-04-16T17:22:58.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:57 smithi043 ceph-mon[36901]: pgmap v581: 129 pgs: 129 active+clean; 434 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-04-16T17:22:58.526 INFO:tasks.workunit.client.0.smithi018.stdout: 10 76554 12.60 MB/sec execute 190 sec latency 857.588 ms 2024-04-16T17:22:58.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:57 smithi018 ceph-mon[31776]: pgmap v581: 129 pgs: 129 active+clean; 434 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-04-16T17:22:59.527 INFO:tasks.workunit.client.0.smithi018.stdout: 10 76554 12.53 MB/sec execute 191 sec latency 1399.427 ms 2024-04-16T17:23:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:22:59 smithi022 ceph-mon[36023]: pgmap v582: 129 pgs: 129 active+clean; 434 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 325 op/s 2024-04-16T17:23:00.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:22:59 smithi043 ceph-mon[36901]: pgmap v582: 129 pgs: 129 active+clean; 434 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 325 op/s 2024-04-16T17:23:00.527 INFO:tasks.workunit.client.0.smithi018.stdout: 10 76792 12.54 MB/sec execute 192 sec latency 1524.026 ms 2024-04-16T17:23:00.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:23:00.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:23:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.201+0000 7f0841e00640 1 -- 172.21.15.18:0/1159498426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c076530 msgr2=0x7f083c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.201+0000 7f0841e00640 1 --2- 172.21.15.18:0/1159498426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c076530 0x7f083c076910 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0824009930 tx=0x7f082402bc30 comp rx=0 tx=0).stop 2024-04-16T17:23:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.204+0000 7f0841e00640 1 -- 172.21.15.18:0/1159498426 shutdown_connections 2024-04-16T17:23:01.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.204+0000 7f0841e00640 1 --2- 172.21.15.18:0/1159498426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f083c112e60 0x7f083c115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.204+0000 7f0841e00640 1 --2- 172.21.15.18:0/1159498426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c076e50 0x7f083c112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.204+0000 7f0841e00640 1 --2- 172.21.15.18:0/1159498426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c076530 0x7f083c076910 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.204+0000 7f0841e00640 1 -- 172.21.15.18:0/1159498426 >> 172.21.15.18:0/1159498426 conn(0x7f083c0ffc90 msgr2=0x7f083c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:01.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.206+0000 7f0841e00640 1 -- 172.21.15.18:0/1159498426 shutdown_connections 2024-04-16T17:23:01.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.206+0000 7f0841e00640 1 -- 172.21.15.18:0/1159498426 wait complete. 2024-04-16T17:23:01.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.207+0000 7f0841e00640 1 Processor -- start 2024-04-16T17:23:01.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.207+0000 7f0841e00640 1 -- start start 2024-04-16T17:23:01.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.208+0000 7f0841e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c076530 0x7f083c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.208+0000 7f0841e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c076e50 0x7f083c19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.208+0000 7f0841e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f083c112e60 0x7f083c1a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.208+0000 7f0841e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f083c118a20 con 0x7f083c076530 2024-04-16T17:23:01.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.208+0000 7f0841e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1188a0 con 0x7f083c112e60 2024-04-16T17:23:01.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.208+0000 7f0841e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f083c118ba0 con 0x7f083c076e50 2024-04-16T17:23:01.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.209+0000 7f083aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c076530 0x7f083c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.209+0000 7f083a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c076e50 0x7f083c19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.209+0000 7f083aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c076530 0x7f083c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49178/0 (socket says 172.21.15.18:49178) 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.209+0000 7f083aa00640 1 -- 172.21.15.18:0/1016031730 learned_addr learned my addr 172.21.15.18:0/1016031730 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.209+0000 7f083b400640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f083c112e60 0x7f083c1a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.210+0000 7f083b400640 1 -- 172.21.15.18:0/1016031730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c076e50 msgr2=0x7f083c19f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.210+0000 7f083b400640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c076e50 0x7f083c19f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.210+0000 7f083b400640 1 -- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c076530 msgr2=0x7f083c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.210+0000 7f083b400640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c076530 0x7f083c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.210+0000 7f083b400640 1 -- 172.21.15.18:0/1016031730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c1a3d90 con 0x7f083c112e60 2024-04-16T17:23:01.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.210+0000 7f083b400640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f083c112e60 0x7f083c1a3660 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f083400bdf0 tx=0x7f083400bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:01.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.211+0000 7f0840a00640 1 -- 172.21.15.18:0/1016031730 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834019070 con 0x7f083c112e60 2024-04-16T17:23:01.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.211+0000 7f0840a00640 1 -- 172.21.15.18:0/1016031730 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0834014920 con 0x7f083c112e60 2024-04-16T17:23:01.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.211+0000 7f0840a00640 1 -- 172.21.15.18:0/1016031730 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834012900 con 0x7f083c112e60 2024-04-16T17:23:01.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.211+0000 7f0841e00640 1 -- 172.21.15.18:0/1016031730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c1a4080 con 0x7f083c112e60 2024-04-16T17:23:01.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.213+0000 7f0841e00640 1 -- 172.21.15.18:0/1016031730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c1ab910 con 0x7f083c112e60 2024-04-16T17:23:01.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.218+0000 7f0840a00640 1 -- 172.21.15.18:0/1016031730 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0834014ad0 con 0x7f083c112e60 2024-04-16T17:23:01.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.219+0000 7f0844bfd740 1 Processor -- start 2024-04-16T17:23:01.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.219+0000 7f0844bfd740 1 -- start start 2024-04-16T17:23:01.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580c1b0 0x55ee1580c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15815170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 0x55ee15817aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f0844bfd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ee15772120 con 0x55ee1580cad0 2024-04-16T17:23:01.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f0844bfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ee15718340 con 0x55ee158156b0 2024-04-16T17:23:01.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f0844bfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ee1576e0e0 con 0x55ee1580c1b0 2024-04-16T17:23:01.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f083a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580c1b0 0x55ee1580c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f083a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580c1b0 0x55ee1580c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43584/0 (socket says 172.21.15.18:43584) 2024-04-16T17:23:01.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.220+0000 7f083a000640 1 -- 172.21.15.18:0/1408991624 learned_addr learned my addr 172.21.15.18:0/1408991624 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:01.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.221+0000 7f083aa00640 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15815170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.221+0000 7f0840a00640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08200a3e20 0x7f08200a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.221+0000 7f083a000640 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 0x55ee15817aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.221+0000 7f083a000640 1 -- 172.21.15.18:0/1408991624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 msgr2=0x55ee15817aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.221+0000 7f083a000640 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 0x55ee15817aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.221+0000 7f083a000640 1 -- 172.21.15.18:0/1408991624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 msgr2=0x55ee15815170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.221+0000 7f083a000640 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15815170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.221+0000 7f083a000640 1 -- 172.21.15.18:0/1408991624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee1554e7d0 con 0x55ee1580c1b0 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.222+0000 7f0840a00640 1 -- 172.21.15.18:0/1016031730 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f08340d0200 con 0x7f083c112e60 2024-04-16T17:23:01.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.222+0000 7f083b400640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08200a3e20 0x7f08200a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.222+0000 7f083a000640 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580c1b0 0x55ee1580c590 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f082c013610 tx=0x7f082c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=371196c5eb3da305 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:01.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.223+0000 7f082be00640 1 -- 172.21.15.18:0/1408991624 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c004050 con 0x55ee1580c1b0 2024-04-16T17:23:01.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.223+0000 7f082be00640 1 -- 172.21.15.18:0/1408991624 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082c0041f0 con 0x55ee1580c1b0 2024-04-16T17:23:01.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.223+0000 7f082be00640 1 -- 172.21.15.18:0/1408991624 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c00f7e0 con 0x55ee1580c1b0 2024-04-16T17:23:01.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.224+0000 7f0844bfd740 1 -- 172.21.15.18:0/1408991624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580c1b0 msgr2=0x55ee1580c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.224+0000 7f083b400640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08200a3e20 0x7f08200a62e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0834000f80 tx=0x7f0834004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:01.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.224+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580c1b0 0x55ee1580c590 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f082c013610 tx=0x7f082c009a20 comp rx=0 tx=0).stop 2024-04-16T17:23:01.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.224+0000 7f0844bfd740 1 -- 172.21.15.18:0/1408991624 shutdown_connections 2024-04-16T17:23:01.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.224+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 0x55ee15817aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.224+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15815170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.224+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1408991624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580c1b0 0x55ee1580c590 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.224+0000 7f0844bfd740 1 -- 172.21.15.18:0/1408991624 >> 172.21.15.18:0/1408991624 conn(0x55ee158068e0 msgr2=0x55ee15806cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:01.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.225+0000 7f0844bfd740 1 -- 172.21.15.18:0/1408991624 shutdown_connections 2024-04-16T17:23:01.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.225+0000 7f0844bfd740 1 -- 172.21.15.18:0/1408991624 wait complete. 2024-04-16T17:23:01.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.226+0000 7f0844bfd740 1 -- start start 2024-04-16T17:23:01.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.226+0000 7f0844bfd740 1 Processor -- start 2024-04-16T17:23:01.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee158127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15812d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 0x55ee15813260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f0844bfd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ee158184a0 con 0x55ee1580cad0 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f0844bfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ee1572b710 con 0x55ee1580c1b0 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f0844bfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ee15818620 con 0x55ee158156b0 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f0844bfd740 10 client.0 _ll_register_callbacks cb 0x55ee15726af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f083a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 0x55ee15813260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f083a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 0x55ee15813260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43596/0 (socket says 172.21.15.18:43596) 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f083a000640 1 -- 172.21.15.18:0/4119384407 learned_addr learned my addr 172.21.15.18:0/4119384407 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.227+0000 7f083a000640 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee158127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.228+0000 7f083aa00640 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15812d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.229+0000 7f083aa00640 1 -- 172.21.15.18:0/4119384407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 msgr2=0x55ee15813260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:01.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.229+0000 7f083aa00640 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 0x55ee15813260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.229+0000 7f083aa00640 1 -- 172.21.15.18:0/4119384407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 msgr2=0x55ee158127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:01.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.229+0000 7f083aa00640 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee158127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.229+0000 7f083aa00640 1 -- 172.21.15.18:0/4119384407 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee154843f0 con 0x55ee1580cad0 2024-04-16T17:23:01.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.230+0000 7f083aa00640 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15812d20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f082402cfd0 tx=0x7f0824004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82ee6883d8a4dab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:01.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.233+0000 7f0828c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:23:01.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.233+0000 7f0828c00640 1 -- 172.21.15.18:0/4119384407 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08240046c0 con 0x55ee1580cad0 2024-04-16T17:23:01.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.233+0000 7f0828c00640 1 -- 172.21.15.18:0/4119384407 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f0824004860 con 0x55ee1580cad0 2024-04-16T17:23:01.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.233+0000 7f0844bfd740 1 -- 172.21.15.18:0/4119384407 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f082c0041f0 con 0x55ee1580cad0 2024-04-16T17:23:01.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.233+0000 7f0828c00640 1 -- 172.21.15.18:0/4119384407 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0824004a60 con 0x55ee1580cad0 2024-04-16T17:23:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.233+0000 7f0844bfd740 10 client.19530 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.233+0000 7f0844bfd740 1 -- 172.21.15.18:0/4119384407 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ee154440f0 con 0x55ee1580cad0 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.234+0000 7f0828c00640 1 -- 172.21.15.18:0/4119384407 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082c0041f0 con 0x55ee1580cad0 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.234+0000 7f0828c00640 1 -- 172.21.15.18:0/4119384407 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f0824003790 con 0x55ee1580cad0 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.234+0000 7f0844bfd740 10 client.19530 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.234+0000 7f0844bfd740 10 client.19530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.234+0000 7f0844bfd740 20 client.19530 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.235+0000 7f0844bfd740 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ee15836ba0 0x55ee15838f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.235+0000 7f0844bfd740 4 client.19530 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.235+0000 7f0844bfd740 1 -- 172.21.15.18:0/4119384407 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee1577fb40 con 0x55ee15836ba0 2024-04-16T17:23:01.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.235+0000 7f083a000640 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ee15836ba0 0x55ee15838f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.236+0000 7f083a000640 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ee15836ba0 0x55ee15838f90 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98dc0d61ca776cab server_cookie=471748a476e87c92 in_seq=0 out_seq=0 2024-04-16T17:23:01.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.236+0000 7f0828c00640 10 client.19530 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.241+0000 7f0828c00640 1 -- 172.21.15.18:0/4119384407 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee1577fb40 con 0x55ee15836ba0 2024-04-16T17:23:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.241+0000 7f0828c00640 10 client.19530 handle_command_reply: tid=0 2024-04-16T17:23:01.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 client.19530 shutdown 2024-04-16T17:23:01.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 -- 172.21.15.18:0/4119384407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 msgr2=0x55ee15812d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15812d20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f082402cfd0 tx=0x7f0824004270 comp rx=0 tx=0).stop 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 -- 172.21.15.18:0/4119384407 shutdown_connections 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ee15836ba0 0x55ee15838f90 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 0x55ee15813260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15812d20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 --2- 172.21.15.18:0/4119384407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee158127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 -- 172.21.15.18:0/4119384407 >> 172.21.15.18:0/4119384407 conn(0x55ee158068e0 msgr2=0x55ee15808630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0828c00640 0 client.19530 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.242+0000 7f0844bfd740 1 -- 172.21.15.18:0/4119384407 shutdown_connections 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.243+0000 7f0844bfd740 1 -- 172.21.15.18:0/4119384407 wait complete. 2024-04-16T17:23:01.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.243+0000 7f0844bfd740 20 client.19530 trim_cache size 0 max 16384 2024-04-16T17:23:01.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.286+0000 7f0844bfd740 1 Processor -- start 2024-04-16T17:23:01.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.286+0000 7f0844bfd740 1 -- start start 2024-04-16T17:23:01.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.286+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580c1b0 0x55ee15814f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.287+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580cad0 0x55ee15805750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.287+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 0x55ee1583e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.287+0000 7f0844bfd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1afb60 con 0x55ee1580c1b0 2024-04-16T17:23:01.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.287+0000 7f0844bfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1ad430 con 0x55ee158156b0 2024-04-16T17:23:01.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.287+0000 7f0844bfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f083c069340 con 0x55ee1580cad0 2024-04-16T17:23:01.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580cad0 0x55ee15805750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580cad0 0x55ee15805750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43602/0 (socket says 172.21.15.18:43602) 2024-04-16T17:23:01.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083aa00640 1 -- 172.21.15.18:0/1685285783 learned_addr learned my addr 172.21.15.18:0/1685285783 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:01.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083a000640 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 0x55ee1583e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083a000640 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580c1b0 0x55ee15814f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083aa00640 1 -- 172.21.15.18:0/1685285783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 msgr2=0x55ee1583e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:01.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083aa00640 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 0x55ee1583e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083aa00640 1 -- 172.21.15.18:0/1685285783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580c1b0 msgr2=0x55ee15814f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:01.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083aa00640 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580c1b0 0x55ee15814f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.289+0000 7f083aa00640 1 -- 172.21.15.18:0/1685285783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee15800d20 con 0x55ee1580cad0 2024-04-16T17:23:01.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.290+0000 7f083aa00640 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580cad0 0x55ee15805750 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f082402f020 tx=0x7f08240058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e25591313617d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:01.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.293+0000 7f082b400640 1 -- 172.21.15.18:0/1685285783 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082402cd70 con 0x55ee1580cad0 2024-04-16T17:23:01.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.293+0000 7f082b400640 1 -- 172.21.15.18:0/1685285783 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0824003da0 con 0x55ee1580cad0 2024-04-16T17:23:01.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.293+0000 7f082b400640 1 -- 172.21.15.18:0/1685285783 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824004eb0 con 0x55ee1580cad0 2024-04-16T17:23:01.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.294+0000 7f0844bfd740 1 -- 172.21.15.18:0/1685285783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580cad0 msgr2=0x55ee15805750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.294+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580cad0 0x55ee15805750 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f082402f020 tx=0x7f08240058f0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.295+0000 7f0844bfd740 1 -- 172.21.15.18:0/1685285783 shutdown_connections 2024-04-16T17:23:01.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.295+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee158156b0 0x55ee1583e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.295+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee1580cad0 0x55ee15805750 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.295+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1685285783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580c1b0 0x55ee15814f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.295+0000 7f0844bfd740 1 -- 172.21.15.18:0/1685285783 >> 172.21.15.18:0/1685285783 conn(0x55ee158068e0 msgr2=0x55ee15805040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:01.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.295+0000 7f0844bfd740 1 -- 172.21.15.18:0/1685285783 shutdown_connections 2024-04-16T17:23:01.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.295+0000 7f0844bfd740 1 -- 172.21.15.18:0/1685285783 wait complete. 2024-04-16T17:23:01.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.296+0000 7f0844bfd740 1 -- start start 2024-04-16T17:23:01.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.296+0000 7f0844bfd740 1 Processor -- start 2024-04-16T17:23:01.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.298+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee1582ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.298+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15829fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.298+0000 7f0844bfd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 0x55ee1582a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.298+0000 7f0844bfd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f083c10d800 con 0x55ee1580cad0 2024-04-16T17:23:01.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.298+0000 7f0844bfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1acd60 con 0x55ee1580c1b0 2024-04-16T17:23:01.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.298+0000 7f0844bfd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1abfc0 con 0x55ee158156b0 2024-04-16T17:23:01.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.298+0000 7f0844bfd740 10 client.0 _ll_register_callbacks cb 0x7f0824009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:01.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.299+0000 7f083aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15829fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.299+0000 7f083aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15829fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49206/0 (socket says 172.21.15.18:49206) 2024-04-16T17:23:01.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.299+0000 7f083aa00640 1 -- 172.21.15.18:0/1118392142 learned_addr learned my addr 172.21.15.18:0/1118392142 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:01.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.299+0000 7f083a000640 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee1582ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.301+0000 7f083aa00640 1 -- 172.21.15.18:0/1118392142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 msgr2=0x55ee1582a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:23:01.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.301+0000 7f083aa00640 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 0x55ee1582a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.301+0000 7f083aa00640 1 -- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 msgr2=0x55ee1582ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:01.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.301+0000 7f083aa00640 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee1582ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.301+0000 7f083aa00640 1 -- 172.21.15.18:0/1118392142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee1582ac60 con 0x55ee1580cad0 2024-04-16T17:23:01.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.301+0000 7f083a000640 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee1582ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:01.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.302+0000 7f083aa00640 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15829fa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0824003ca0 tx=0x7f0824005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b6e6571ebfe1bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:01.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.302+0000 7f0828c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:23:01.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.303+0000 7f0828c00640 1 -- 172.21.15.18:0/1118392142 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824064060 con 0x55ee1580cad0 2024-04-16T17:23:01.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.303+0000 7f0828c00640 1 -- 172.21.15.18:0/1118392142 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f08240035f0 con 0x55ee1580cad0 2024-04-16T17:23:01.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.303+0000 7f0844bfd740 1 -- 172.21.15.18:0/1118392142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0824003da0 con 0x55ee1580cad0 2024-04-16T17:23:01.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.305+0000 7f0828c00640 1 -- 172.21.15.18:0/1118392142 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f08240037f0 con 0x55ee1580cad0 2024-04-16T17:23:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.306+0000 7f0844bfd740 10 client.19536 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:01.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.306+0000 7f0844bfd740 1 -- 172.21.15.18:0/1118392142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ee158ae060 con 0x55ee1580cad0 2024-04-16T17:23:01.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.307+0000 7f0828c00640 1 -- 172.21.15.18:0/1118392142 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0824003da0 con 0x55ee1580cad0 2024-04-16T17:23:01.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.307+0000 7f0828c00640 1 -- 172.21.15.18:0/1118392142 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f0824055470 con 0x55ee1580cad0 2024-04-16T17:23:01.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.308+0000 7f0844bfd740 10 client.19536 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:01.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.308+0000 7f0844bfd740 10 client.19536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:01.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.308+0000 7f0844bfd740 20 client.19536 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:01.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.308+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ee158ae650 0x55ee158b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.308+0000 7f0844bfd740 4 client.19536 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:23:01.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.308+0000 7f0844bfd740 1 -- 172.21.15.18:0/1118392142 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee157799b0 con 0x55ee158ae650 2024-04-16T17:23:01.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.309+0000 7f083a000640 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ee158ae650 0x55ee158b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.310+0000 7f083a000640 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ee158ae650 0x55ee158b0a90 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b2c38cffbdb39a6 server_cookie=bc496ce3fff4f43b in_seq=0 out_seq=0 2024-04-16T17:23:01.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.310+0000 7f0828c00640 10 client.19536 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.318+0000 7f0828c00640 1 -- 172.21.15.18:0/1118392142 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ee157799b0 con 0x55ee158ae650 2024-04-16T17:23:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.318+0000 7f0828c00640 10 client.19536 handle_command_reply: tid=0 2024-04-16T17:23:01.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.318+0000 7f0844bfd740 1 client.19536 shutdown 2024-04-16T17:23:01.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.319+0000 7f0844bfd740 1 -- 172.21.15.18:0/1118392142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 msgr2=0x55ee15829fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.319+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15829fa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0824003ca0 tx=0x7f0824005c90 comp rx=0 tx=0).stop 2024-04-16T17:23:01.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.319+0000 7f0844bfd740 1 -- 172.21.15.18:0/1118392142 shutdown_connections 2024-04-16T17:23:01.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.319+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ee158ae650 0x55ee158b0a90 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.320+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee158156b0 0x55ee1582a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.320+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ee1580cad0 0x55ee15829fa0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.320+0000 7f0844bfd740 1 --2- 172.21.15.18:0/1118392142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee1580c1b0 0x55ee1582ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.320+0000 7f0844bfd740 1 -- 172.21.15.18:0/1118392142 >> 172.21.15.18:0/1118392142 conn(0x55ee158068e0 msgr2=0x55ee15813030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:01.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.320+0000 7f0828c00640 0 client.19536 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:01.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.321+0000 7f0844bfd740 1 -- 172.21.15.18:0/1118392142 shutdown_connections 2024-04-16T17:23:01.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.321+0000 7f0844bfd740 1 -- 172.21.15.18:0/1118392142 wait complete. 2024-04-16T17:23:01.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.321+0000 7f0844bfd740 20 client.19536 trim_cache size 0 max 16384 2024-04-16T17:23:01.322 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:23:01.322 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:23:01.322 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:23:01.322 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:23:01.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.322+0000 7f0841e00640 1 -- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08200a3e20 msgr2=0x7f08200a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.322+0000 7f0841e00640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08200a3e20 0x7f08200a62e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0834000f80 tx=0x7f0834004a40 comp rx=0 tx=0).stop 2024-04-16T17:23:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.322+0000 7f0841e00640 1 -- 172.21.15.18:0/1016031730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f083c112e60 msgr2=0x7f083c1a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.322+0000 7f0841e00640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f083c112e60 0x7f083c1a3660 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f083400bdf0 tx=0x7f083400bee0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.324+0000 7f0841e00640 1 -- 172.21.15.18:0/1016031730 shutdown_connections 2024-04-16T17:23:01.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.324+0000 7f0841e00640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f08200a3e20 0x7f08200a62e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.324+0000 7f0841e00640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f083c112e60 0x7f083c1a3660 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.324+0000 7f0841e00640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c076e50 0x7f083c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.324+0000 7f0841e00640 1 --2- 172.21.15.18:0/1016031730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c076530 0x7f083c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.324+0000 7f0841e00640 1 -- 172.21.15.18:0/1016031730 >> 172.21.15.18:0/1016031730 conn(0x7f083c0ffc90 msgr2=0x7f083c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:01.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.324+0000 7f0841e00640 1 -- 172.21.15.18:0/1016031730 shutdown_connections 2024-04-16T17:23:01.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.324+0000 7f0841e00640 1 -- 172.21.15.18:0/1016031730 wait complete. 2024-04-16T17:23:01.360 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:23:01.360 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:23:01.527 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77025 12.56 MB/sec execute 193 sec latency 595.814 ms 2024-04-16T17:23:01.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 -- 172.21.15.18:0/585222443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62010a4b0 msgr2=0x7fe62010db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 --2- 172.21.15.18:0/585222443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62010a4b0 0x7fe62010db30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe610009be0 tx=0x7fe61002bf10 comp rx=0 tx=0).stop 2024-04-16T17:23:01.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 -- 172.21.15.18:0/585222443 shutdown_connections 2024-04-16T17:23:01.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 --2- 172.21.15.18:0/585222443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62010e260 0x7fe620115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 --2- 172.21.15.18:0/585222443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62010a4b0 0x7fe62010db30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 --2- 172.21.15.18:0/585222443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620109b00 0x7fe620109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 -- 172.21.15.18:0/585222443 >> 172.21.15.18:0/585222443 conn(0x7fe62006e5f0 msgr2=0x7fe62006ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:01.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 -- 172.21.15.18:0/585222443 shutdown_connections 2024-04-16T17:23:01.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.688+0000 7fe624a00640 1 -- 172.21.15.18:0/585222443 wait complete. 2024-04-16T17:23:01.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.690+0000 7fe624a00640 1 Processor -- start 2024-04-16T17:23:01.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.692+0000 7fe624a00640 1 -- start start 2024-04-16T17:23:01.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.692+0000 7fe624a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe620109b00 0x7fe62011ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.692+0000 7fe624a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62010e260 0x7fe620118db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62010e260 0x7fe620118db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62010e260 0x7fe620118db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59110/0 (socket says 172.21.15.18:59110) 2024-04-16T17:23:01.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.692+0000 7fe624a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6201192f0 0x7fe6201197a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61d400640 1 -- 172.21.15.18:0/2335536274 learned_addr learned my addr 172.21.15.18:0/2335536274 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:01.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe620117780 con 0x7fe620109b00 2024-04-16T17:23:01.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe620117600 con 0x7fe62010e260 2024-04-16T17:23:01.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe620117900 con 0x7fe6201192f0 2024-04-16T17:23:01.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61d400640 1 -- 172.21.15.18:0/2335536274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6201192f0 msgr2=0x7fe6201197a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61e800640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6201192f0 0x7fe6201197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61d400640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6201192f0 0x7fe6201197a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61d400640 1 -- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe620109b00 msgr2=0x7fe62011ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:01.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61d400640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe620109b00 0x7fe62011ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:01.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.693+0000 7fe61d400640 1 -- 172.21.15.18:0/2335536274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6201bc650 con 0x7fe62010e260 2024-04-16T17:23:01.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.694+0000 7fe61e800640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6201192f0 0x7fe6201197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:01.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.694+0000 7fe61d400640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62010e260 0x7fe620118db0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe610036040 tx=0x7fe610033870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:01.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.694+0000 7fe616a00640 1 -- 172.21.15.18:0/2335536274 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610043020 con 0x7fe62010e260 2024-04-16T17:23:01.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.695+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6201bc8e0 con 0x7fe62010e260 2024-04-16T17:23:01.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.695+0000 7fe616a00640 1 -- 172.21.15.18:0/2335536274 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe610003c40 con 0x7fe62010e260 2024-04-16T17:23:01.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.695+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6201bcdc0 con 0x7fe62010e260 2024-04-16T17:23:01.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.695+0000 7fe616a00640 1 -- 172.21.15.18:0/2335536274 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61003e910 con 0x7fe62010e260 2024-04-16T17:23:01.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.698+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe620073f30 con 0x7fe62010e260 2024-04-16T17:23:01.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.702+0000 7fe616a00640 1 -- 172.21.15.18:0/2335536274 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe610046020 con 0x7fe62010e260 2024-04-16T17:23:01.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.706+0000 7fe616a00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6040a3d80 0x7fe6040a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:01.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.707+0000 7fe616a00640 1 -- 172.21.15.18:0/2335536274 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7fe6100f2ff0 con 0x7fe62010e260 2024-04-16T17:23:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.707+0000 7fe616a00640 1 -- 172.21.15.18:0/2335536274 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6100f3510 con 0x7fe62010e260 2024-04-16T17:23:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.707+0000 7fe61de00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6040a3d80 0x7fe6040a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:01.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:01.709+0000 7fe61de00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6040a3d80 0x7fe6040a6240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe6080058c0 tx=0x7fe608005850 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:02.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:01 smithi022 ceph-mon[36023]: pgmap v583: 129 pgs: 129 active+clean; 426 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 496 op/s 2024-04-16T17:23:02.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.220+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe620068eb0 con 0x7fe62010e260 2024-04-16T17:23:02.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.221+0000 7fe616a00640 1 -- 172.21.15.18:0/2335536274 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7fe620068eb0 con 0x7fe62010e260 2024-04-16T17:23:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:23:02.221 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:23:02.221 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:23:02.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6040a3d80 msgr2=0x7fe6040a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6040a3d80 0x7fe6040a6240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe6080058c0 tx=0x7fe608005850 comp rx=0 tx=0).stop 2024-04-16T17:23:02.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62010e260 msgr2=0x7fe620118db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62010e260 0x7fe620118db0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe610036040 tx=0x7fe610033870 comp rx=0 tx=0).stop 2024-04-16T17:23:02.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 shutdown_connections 2024-04-16T17:23:02.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe6040a3d80 0x7fe6040a6240 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6201192f0 0x7fe6201197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe62010e260 0x7fe620118db0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 --2- 172.21.15.18:0/2335536274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe620109b00 0x7fe62011ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.225+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 >> 172.21.15.18:0/2335536274 conn(0x7fe62006e5f0 msgr2=0x7fe620071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:02.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.226+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 shutdown_connections 2024-04-16T17:23:02.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.226+0000 7fe624a00640 1 -- 172.21.15.18:0/2335536274 wait complete. 2024-04-16T17:23:02.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:01 smithi043 ceph-mon[36901]: pgmap v583: 129 pgs: 129 active+clean; 426 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 496 op/s 2024-04-16T17:23:02.259 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:23:02.528 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77254 12.53 MB/sec execute 194 sec latency 1033.439 ms 2024-04-16T17:23:02.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.585+0000 7fba56c00640 1 -- 172.21.15.18:0/2732600735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba50104df0 msgr2=0x7fba50108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.585+0000 7fba56c00640 1 --2- 172.21.15.18:0/2732600735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba50104df0 0x7fba50108e30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fba40009900 tx=0x7fba4002bc40 comp rx=0 tx=0).stop 2024-04-16T17:23:02.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.586+0000 7fba56c00640 1 -- 172.21.15.18:0/2732600735 shutdown_connections 2024-04-16T17:23:02.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.586+0000 7fba56c00640 1 --2- 172.21.15.18:0/2732600735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba50109510 0x7fba501110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.586+0000 7fba56c00640 1 --2- 172.21.15.18:0/2732600735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba50104df0 0x7fba50108e30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.586+0000 7fba56c00640 1 --2- 172.21.15.18:0/2732600735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba50104440 0x7fba50104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.586+0000 7fba56c00640 1 -- 172.21.15.18:0/2732600735 >> 172.21.15.18:0/2732600735 conn(0x7fba500ffc80 msgr2=0x7fba501020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:02.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.586+0000 7fba56c00640 1 -- 172.21.15.18:0/2732600735 shutdown_connections 2024-04-16T17:23:02.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.586+0000 7fba56c00640 1 -- 172.21.15.18:0/2732600735 wait complete. 2024-04-16T17:23:02.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.587+0000 7fba56c00640 1 Processor -- start 2024-04-16T17:23:02.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.588+0000 7fba56c00640 1 -- start start 2024-04-16T17:23:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.588+0000 7fba56c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba50104440 0x7fba501a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.588+0000 7fba4fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba50104440 0x7fba501a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.589+0000 7fba4fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba50104440 0x7fba501a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49248/0 (socket says 172.21.15.18:49248) 2024-04-16T17:23:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.589+0000 7fba4fe00640 1 -- 172.21.15.18:0/572149084 learned_addr learned my addr 172.21.15.18:0/572149084 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.591+0000 7fba56c00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba50109510 0x7fba501a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.591+0000 7fba56c00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5019a990 0x7fba5019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.591+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba501145f0 con 0x7fba50104440 2024-04-16T17:23:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.591+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba50114470 con 0x7fba50109510 2024-04-16T17:23:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.591+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba50114770 con 0x7fba5019a990 2024-04-16T17:23:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.592+0000 7fba4fe00640 1 -- 172.21.15.18:0/572149084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5019a990 msgr2=0x7fba5019ae40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:23:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.592+0000 7fba4fe00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5019a990 0x7fba5019ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.592+0000 7fba4fe00640 1 -- 172.21.15.18:0/572149084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba50109510 msgr2=0x7fba501a0d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:23:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.592+0000 7fba4fe00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba50109510 0x7fba501a0d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.592+0000 7fba4fe00640 1 -- 172.21.15.18:0/572149084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5019b4b0 con 0x7fba50104440 2024-04-16T17:23:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.593+0000 7fba4fe00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba50104440 0x7fba501a07e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00e9f0 tx=0x7fba3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:02.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.593+0000 7fba4cc00640 1 -- 172.21.15.18:0/572149084 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00cd80 con 0x7fba50104440 2024-04-16T17:23:02.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.594+0000 7fba4cc00640 1 -- 172.21.15.18:0/572149084 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba3c005500 con 0x7fba50104440 2024-04-16T17:23:02.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.594+0000 7fba4cc00640 1 -- 172.21.15.18:0/572149084 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c010690 con 0x7fba50104440 2024-04-16T17:23:02.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.594+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5019b7a0 con 0x7fba50104440 2024-04-16T17:23:02.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.594+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba501a79f0 con 0x7fba50104440 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.602+0000 7fba4cc00640 1 -- 172.21.15.18:0/572149084 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba3c0046c0 con 0x7fba50104440 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.603+0000 7fba5982f740 1 Processor -- start 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.604+0000 7fba5982f740 1 -- start start 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.604+0000 7fba5982f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70dd50 0x5586ac70e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.604+0000 7fba5982f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac716d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.604+0000 7fba5982f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 0x5586ac719640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.604+0000 7fba5982f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5586ac673020 con 0x5586ac70e670 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.604+0000 7fba5982f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5586ac619460 con 0x5586ac717250 2024-04-16T17:23:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.604+0000 7fba5982f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586ac66efe0 con 0x5586ac70dd50 2024-04-16T17:23:02.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.604+0000 7fba54c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70dd50 0x5586ac70e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba4f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 0x5586ac719640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba54c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70dd50 0x5586ac70e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43658/0 (socket says 172.21.15.18:43658) 2024-04-16T17:23:02.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba54c00640 1 -- 172.21.15.18:0/427012494 learned_addr learned my addr 172.21.15.18:0/427012494 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:02.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba4fe00640 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac716d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba54c00640 1 -- 172.21.15.18:0/427012494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 msgr2=0x5586ac719640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:02.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba54c00640 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 0x5586ac719640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba54c00640 1 -- 172.21.15.18:0/427012494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 msgr2=0x5586ac716d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:02.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba54c00640 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac716d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba54c00640 1 -- 172.21.15.18:0/427012494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ac621040 con 0x5586ac70dd50 2024-04-16T17:23:02.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba4fe00640 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac716d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:02.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.605+0000 7fba54c00640 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70dd50 0x5586ac70e130 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fba48009c80 tx=0x7fba4800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a14b023101d067cd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:02.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.606+0000 7fba45200640 1 -- 172.21.15.18:0/427012494 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48016070 con 0x5586ac70dd50 2024-04-16T17:23:02.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.606+0000 7fba45200640 1 -- 172.21.15.18:0/427012494 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba48005020 con 0x5586ac70dd50 2024-04-16T17:23:02.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.606+0000 7fba45200640 1 -- 172.21.15.18:0/427012494 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba480116d0 con 0x5586ac70dd50 2024-04-16T17:23:02.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.606+0000 7fba4cc00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba340a3da0 0x7fba340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.606+0000 7fba4cc00640 1 -- 172.21.15.18:0/572149084 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7fba3c0d0ea0 con 0x7fba50104440 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.606+0000 7fba5982f740 1 -- 172.21.15.18:0/427012494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70dd50 msgr2=0x5586ac70e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.607+0000 7fba5982f740 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70dd50 0x5586ac70e130 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fba48009c80 tx=0x7fba4800e6c0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.607+0000 7fba4fe00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba340a3da0 0x7fba340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.607+0000 7fba5982f740 1 -- 172.21.15.18:0/427012494 shutdown_connections 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.607+0000 7fba5982f740 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 0x5586ac719640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.607+0000 7fba5982f740 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac716d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.607+0000 7fba5982f740 1 --2- 172.21.15.18:0/427012494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70dd50 0x5586ac70e130 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.607+0000 7fba5982f740 1 -- 172.21.15.18:0/427012494 >> 172.21.15.18:0/427012494 conn(0x5586ac708480 msgr2=0x5586ac708860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:02.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.608+0000 7fba4fe00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba340a3da0 0x7fba340a6260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0ce2c0 tx=0x7fba3c004d00 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:02.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.608+0000 7fba5982f740 1 -- 172.21.15.18:0/427012494 shutdown_connections 2024-04-16T17:23:02.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.608+0000 7fba5982f740 1 -- 172.21.15.18:0/427012494 wait complete. 2024-04-16T17:23:02.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.609+0000 7fba5982f740 1 -- start start 2024-04-16T17:23:02.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.609+0000 7fba5982f740 1 Processor -- start 2024-04-16T17:23:02.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba5982f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70dd50 0x5586ac714270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba5982f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70e670 0x5586ac7147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba5982f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 0x5586ac714cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba54c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70dd50 0x5586ac714270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba54c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70dd50 0x5586ac714270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49276/0 (socket says 172.21.15.18:49276) 2024-04-16T17:23:02.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba54c00640 1 -- 172.21.15.18:0/4239036837 learned_addr learned my addr 172.21.15.18:0/4239036837 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5586ac71a040 con 0x5586ac70dd50 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5586ac62c830 con 0x5586ac717250 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586ac71a1c0 con 0x5586ac70e670 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba4f400640 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70e670 0x5586ac7147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.610+0000 7fba54c00640 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 0x5586ac714cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.611+0000 7fba5982f740 10 client.0 _ll_register_callbacks cb 0x5586ac634150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.611+0000 7fba4f400640 1 -- 172.21.15.18:0/4239036837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 msgr2=0x5586ac714cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.611+0000 7fba4f400640 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 0x5586ac714cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.611+0000 7fba4f400640 1 -- 172.21.15.18:0/4239036837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70dd50 msgr2=0x5586ac714270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:02.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.611+0000 7fba4f400640 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70dd50 0x5586ac714270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.611+0000 7fba4f400640 1 -- 172.21.15.18:0/4239036837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ac630920 con 0x5586ac70e670 2024-04-16T17:23:02.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.613+0000 7fba4f400640 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70e670 0x5586ac7147b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fba50100230 tx=0x7fba40005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75f2519297cb4d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:02.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.613+0000 7fba39600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:02.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.614+0000 7fba39600640 1 -- 172.21.15.18:0/4239036837 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40042030 con 0x5586ac70e670 2024-04-16T17:23:02.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.614+0000 7fba39600640 1 -- 172.21.15.18:0/4239036837 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7fba4002caa0 con 0x5586ac70e670 2024-04-16T17:23:02.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.615+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba48005020 con 0x5586ac70e670 2024-04-16T17:23:02.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.615+0000 7fba39600640 1 -- 172.21.15.18:0/4239036837 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fba4002ccc0 con 0x5586ac70e670 2024-04-16T17:23:02.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.616+0000 7fba5982f740 10 client.30208 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:02.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.616+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5586ac5fe7f0 con 0x5586ac70e670 2024-04-16T17:23:02.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.616+0000 7fba39600640 1 -- 172.21.15.18:0/4239036837 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba48005020 con 0x5586ac70e670 2024-04-16T17:23:02.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.617+0000 7fba39600640 1 -- 172.21.15.18:0/4239036837 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fba40004b10 con 0x5586ac70e670 2024-04-16T17:23:02.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.617+0000 7fba5982f740 10 client.30208 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:02.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.617+0000 7fba5982f740 10 client.30208 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:23:02.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.617+0000 7fba5982f740 20 client.30208 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:02.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.617+0000 7fba5982f740 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5586ac727580 0x5586ac729a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.617+0000 7fba5982f740 4 client.30208 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:02.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.617+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586ac680a40 con 0x5586ac727580 2024-04-16T17:23:02.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.617+0000 7fba54c00640 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5586ac727580 0x5586ac729a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.618+0000 7fba39600640 10 client.30208 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:23:02.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.618+0000 7fba54c00640 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5586ac727580 0x5586ac729a00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fe9244ae6f93dd7 server_cookie=99c5bbd7a24a3db7 in_seq=0 out_seq=0 2024-04-16T17:23:02.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.619+0000 7fba39600640 1 -- 172.21.15.18:0/4239036837 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586ac680a40 con 0x5586ac727580 2024-04-16T17:23:02.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.619+0000 7fba39600640 10 client.30208 handle_command_reply: tid=0 2024-04-16T17:23:02.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.619+0000 7fba5982f740 1 client.30208 shutdown 2024-04-16T17:23:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.621+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70e670 msgr2=0x5586ac7147b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.621+0000 7fba5982f740 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70e670 0x5586ac7147b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fba50100230 tx=0x7fba40005d50 comp rx=0 tx=0).stop 2024-04-16T17:23:02.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.621+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 shutdown_connections 2024-04-16T17:23:02.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.621+0000 7fba5982f740 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5586ac727580 0x5586ac729a00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.621+0000 7fba5982f740 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac717250 0x5586ac714cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.621+0000 7fba5982f740 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac70e670 0x5586ac7147b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.621+0000 7fba5982f740 1 --2- 172.21.15.18:0/4239036837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70dd50 0x5586ac714270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.622+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 >> 172.21.15.18:0/4239036837 conn(0x5586ac708480 msgr2=0x5586ac70a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:02.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.622+0000 7fba39600640 0 client.30208 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:23:02.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.622+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 shutdown_connections 2024-04-16T17:23:02.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.622+0000 7fba5982f740 1 -- 172.21.15.18:0/4239036837 wait complete. 2024-04-16T17:23:02.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.622+0000 7fba5982f740 20 client.30208 trim_cache size 0 max 16384 2024-04-16T17:23:02.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.652+0000 7fba5982f740 1 Processor -- start 2024-04-16T17:23:02.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.652+0000 7fba5982f740 1 -- start start 2024-04-16T17:23:02.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.653+0000 7fba5982f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac7169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.653+0000 7fba5982f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac722f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.653+0000 7fba5982f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 0x5586ac72f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.653+0000 7fba5982f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba30012870 con 0x5586ac70e670 2024-04-16T17:23:02.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.653+0000 7fba5982f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba30010140 con 0x5586ac70dd50 2024-04-16T17:23:02.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba5982f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba3000dfb0 con 0x5586ac717250 2024-04-16T17:23:02.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac7169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac7169a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59150/0 (socket says 172.21.15.18:59150) 2024-04-16T17:23:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 -- 172.21.15.18:0/3712488762 learned_addr learned my addr 172.21.15.18:0/3712488762 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 --2- 172.21.15.18:0/3712488762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 0x5586ac72f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 -- 172.21.15.18:0/3712488762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 msgr2=0x5586ac72f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 --2- 172.21.15.18:0/3712488762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 0x5586ac72f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 -- 172.21.15.18:0/3712488762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 msgr2=0x5586ac722f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 --2- 172.21.15.18:0/3712488762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac722f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.654+0000 7fba54c00640 1 -- 172.21.15.18:0/3712488762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ac3853f0 con 0x5586ac70dd50 2024-04-16T17:23:02.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.657+0000 7fba54c00640 1 --2- 172.21.15.18:0/3712488762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac7169a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fba48009c80 tx=0x7fba480057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4668eb2a8aac089b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:02.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.657+0000 7fba3be00640 1 -- 172.21.15.18:0/3712488762 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48023040 con 0x5586ac70dd50 2024-04-16T17:23:02.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.657+0000 7fba3be00640 1 -- 172.21.15.18:0/3712488762 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba48024040 con 0x5586ac70dd50 2024-04-16T17:23:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.658+0000 7fba3be00640 1 -- 172.21.15.18:0/3712488762 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48016040 con 0x5586ac70dd50 2024-04-16T17:23:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.659+0000 7fba5982f740 1 -- 172.21.15.18:0/3712488762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 msgr2=0x5586ac7169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.659+0000 7fba5982f740 1 --2- 172.21.15.18:0/3712488762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac7169a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fba48009c80 tx=0x7fba480057a0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.659+0000 7fba5982f740 1 -- 172.21.15.18:0/3712488762 shutdown_connections 2024-04-16T17:23:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.659+0000 7fba5982f740 1 --2- 172.21.15.18:0/3712488762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 0x5586ac72f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.660+0000 7fba5982f740 1 --2- 172.21.15.18:0/3712488762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac722f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.660+0000 7fba5982f740 1 --2- 172.21.15.18:0/3712488762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac7169a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.660+0000 7fba5982f740 1 -- 172.21.15.18:0/3712488762 >> 172.21.15.18:0/3712488762 conn(0x5586ac708480 msgr2=0x5586ac70f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.661+0000 7fba5982f740 1 -- 172.21.15.18:0/3712488762 shutdown_connections 2024-04-16T17:23:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.661+0000 7fba5982f740 1 -- 172.21.15.18:0/3712488762 wait complete. 2024-04-16T17:23:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.662+0000 7fba5982f740 1 -- start start 2024-04-16T17:23:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.662+0000 7fba5982f740 1 Processor -- start 2024-04-16T17:23:02.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.663+0000 7fba5982f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac785ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.664+0000 7fba54c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac785ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.664+0000 7fba54c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac785ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59158/0 (socket says 172.21.15.18:59158) 2024-04-16T17:23:02.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.664+0000 7fba5982f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac786400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.664+0000 7fba5982f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 0x5586ac71a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.664+0000 7fba54c00640 1 -- 172.21.15.18:0/239675189 learned_addr learned my addr 172.21.15.18:0/239675189 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:02.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.664+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba3000e600 con 0x5586ac70e670 2024-04-16T17:23:02.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.665+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba3000fa70 con 0x5586ac70dd50 2024-04-16T17:23:02.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.665+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba3000ecd0 con 0x5586ac717250 2024-04-16T17:23:02.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.665+0000 7fba4f400640 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac786400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.666+0000 7fba54c00640 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 0x5586ac71a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba54c00640 1 -- 172.21.15.18:0/239675189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 msgr2=0x5586ac71a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba54c00640 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 0x5586ac71a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba54c00640 1 -- 172.21.15.18:0/239675189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 msgr2=0x5586ac786400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba54c00640 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac786400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba5982f740 10 client.0 _ll_register_callbacks cb 0x7fba40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba54c00640 1 -- 172.21.15.18:0/239675189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ac44f7d0 con 0x5586ac70dd50 2024-04-16T17:23:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba48024040 con 0x5586ac70dd50 2024-04-16T17:23:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba54c00640 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac785ec0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fba480058f0 tx=0x7fba4802c630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0d31ce324e8599 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.669+0000 7fba39600640 10 client.30350 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.670+0000 7fba39600640 1 -- 172.21.15.18:0/239675189 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48016050 con 0x5586ac70dd50 2024-04-16T17:23:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.670+0000 7fba39600640 1 -- 172.21.15.18:0/239675189 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7fba480052f0 con 0x5586ac70dd50 2024-04-16T17:23:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.670+0000 7fba39600640 1 -- 172.21.15.18:0/239675189 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fba48005570 con 0x5586ac70dd50 2024-04-16T17:23:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.670+0000 7fba5982f740 10 client.30350 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.670+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5586ac78d3f0 con 0x5586ac70dd50 2024-04-16T17:23:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.671+0000 7fba39600640 1 -- 172.21.15.18:0/239675189 <== mon.2 v2:172.21.15.22:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fba4800f040 con 0x5586ac70dd50 2024-04-16T17:23:02.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.671+0000 7fba5982f740 10 client.30350 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:02.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.671+0000 7fba5982f740 10 client.30350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:23:02.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.671+0000 7fba5982f740 20 client.30350 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:02.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.671+0000 7fba5982f740 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5586ac78d720 0x5586ac78fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:02.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.671+0000 7fba5982f740 4 client.30350 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:23:02.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.671+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586ac67a8b0 con 0x5586ac78d720 2024-04-16T17:23:02.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.671+0000 7fba4f400640 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5586ac78d720 0x5586ac78fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:02.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.672+0000 7fba4f400640 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5586ac78d720 0x5586ac78fb10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13d94032e8a6fdc1 server_cookie=80ed2c49a23ab989 in_seq=0 out_seq=0 2024-04-16T17:23:02.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.672+0000 7fba39600640 10 client.30350 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:23:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.672+0000 7fba39600640 1 -- 172.21.15.18:0/239675189 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586ac67a8b0 con 0x5586ac78d720 2024-04-16T17:23:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.672+0000 7fba39600640 10 client.30350 handle_command_reply: tid=0 2024-04-16T17:23:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.672+0000 7fba5982f740 1 client.30350 shutdown 2024-04-16T17:23:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 msgr2=0x5586ac785ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac785ec0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fba480058f0 tx=0x7fba4802c630 comp rx=0 tx=0).stop 2024-04-16T17:23:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 shutdown_connections 2024-04-16T17:23:02.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5586ac78d720 0x5586ac78fb10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ac717250 0x5586ac71a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5586ac70e670 0x5586ac786400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 --2- 172.21.15.18:0/239675189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586ac70dd50 0x5586ac785ec0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba39600640 0 client.30350 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 >> 172.21.15.18:0/239675189 conn(0x5586ac708480 msgr2=0x5586ac714210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 shutdown_connections 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 1 -- 172.21.15.18:0/239675189 wait complete. 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.673+0000 7fba5982f740 20 client.30350 trim_cache size 0 max 16384 2024-04-16T17:23:02.674 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba340a3da0 msgr2=0x7fba340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba340a3da0 0x7fba340a6260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0ce2c0 tx=0x7fba3c004d00 comp rx=0 tx=0).stop 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba50104440 msgr2=0x7fba501a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba50104440 0x7fba501a07e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00e9f0 tx=0x7fba3c00edf0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 shutdown_connections 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fba340a3da0 0x7fba340a6260 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba5019a990 0x7fba5019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba50109510 0x7fba501a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 --2- 172.21.15.18:0/572149084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba50104440 0x7fba501a07e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 >> 172.21.15.18:0/572149084 conn(0x7fba500ffc80 msgr2=0x7fba50102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:02.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 shutdown_connections 2024-04-16T17:23:02.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:02.677+0000 7fba56c00640 1 -- 172.21.15.18:0/572149084 wait complete. 2024-04-16T17:23:02.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:23:02.737 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:22:59 smithi018 ceph-mon[31776]: pgmap v582: 129 pgs: 129 active+clean; 434 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 325 op/s 2024-04-16T17:23:02.738 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:01 smithi018 ceph-mon[31776]: pgmap v583: 129 pgs: 129 active+clean; 426 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 496 op/s 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.040+0000 7fe217e00640 1 -- 172.21.15.18:0/1870927831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe210109b00 msgr2=0x7fe210114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.040+0000 7fe217e00640 1 --2- 172.21.15.18:0/1870927831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe210109b00 0x7fe210114c10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009930 tx=0x7fe1fc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.040+0000 7fe217e00640 1 -- 172.21.15.18:0/1870927831 shutdown_connections 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.040+0000 7fe217e00640 1 --2- 172.21.15.18:0/1870927831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe210115150 0x7fe210117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.040+0000 7fe217e00640 1 --2- 172.21.15.18:0/1870927831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe210109b00 0x7fe210114c10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.040+0000 7fe217e00640 1 --2- 172.21.15.18:0/1870927831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2101091e0 0x7fe2101095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.040+0000 7fe217e00640 1 -- 172.21.15.18:0/1870927831 >> 172.21.15.18:0/1870927831 conn(0x7fe21006d0a0 msgr2=0x7fe21006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.041+0000 7fe217e00640 1 -- 172.21.15.18:0/1870927831 shutdown_connections 2024-04-16T17:23:03.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.041+0000 7fe217e00640 1 -- 172.21.15.18:0/1870927831 wait complete. 2024-04-16T17:23:03.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.042+0000 7fe217e00640 1 Processor -- start 2024-04-16T17:23:03.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.043+0000 7fe217e00640 1 -- start start 2024-04-16T17:23:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.044+0000 7fe217e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2101091e0 0x7fe2101a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.044+0000 7fe217e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe210109b00 0x7fe2101a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.044+0000 7fe217e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe210115150 0x7fe2101abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.044+0000 7fe217e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe210121210 con 0x7fe2101091e0 2024-04-16T17:23:03.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.044+0000 7fe217e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe210121090 con 0x7fe210109b00 2024-04-16T17:23:03.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.044+0000 7fe217e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe210121390 con 0x7fe210115150 2024-04-16T17:23:03.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.045+0000 7fe215e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe210115150 0x7fe2101abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.045+0000 7fe215e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe210115150 0x7fe2101abfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43702/0 (socket says 172.21.15.18:43702) 2024-04-16T17:23:03.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.045+0000 7fe215e00640 1 -- 172.21.15.18:0/3845723069 learned_addr learned my addr 172.21.15.18:0/3845723069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.045+0000 7fe214a00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe210109b00 0x7fe2101a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.046+0000 7fe215400640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2101091e0 0x7fe2101a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.046+0000 7fe215e00640 1 -- 172.21.15.18:0/3845723069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe210109b00 msgr2=0x7fe2101a7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.046+0000 7fe215e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe210109b00 0x7fe2101a7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.046+0000 7fe215e00640 1 -- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2101091e0 msgr2=0x7fe2101a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.046+0000 7fe215e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2101091e0 0x7fe2101a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.046+0000 7fe215e00640 1 -- 172.21.15.18:0/3845723069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2101ac710 con 0x7fe210115150 2024-04-16T17:23:03.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.047+0000 7fe215e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe210115150 0x7fe2101abfe0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe20800ccb0 tx=0x7fe20800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.047+0000 7fe20e000640 1 -- 172.21.15.18:0/3845723069 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20800f450 con 0x7fe210115150 2024-04-16T17:23:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.047+0000 7fe20e000640 1 -- 172.21.15.18:0/3845723069 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20800fad0 con 0x7fe210115150 2024-04-16T17:23:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.047+0000 7fe20e000640 1 -- 172.21.15.18:0/3845723069 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208018ca0 con 0x7fe210115150 2024-04-16T17:23:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.049+0000 7fe217e00640 1 -- 172.21.15.18:0/3845723069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2101aca00 con 0x7fe210115150 2024-04-16T17:23:03.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.049+0000 7fe217e00640 1 -- 172.21.15.18:0/3845723069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2101b4290 con 0x7fe210115150 2024-04-16T17:23:03.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.056+0000 7fe21aa59740 1 Processor -- start 2024-04-16T17:23:03.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.059+0000 7fe20e000640 1 -- 172.21.15.18:0/3845723069 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe20800f5f0 con 0x7fe210115150 2024-04-16T17:23:03.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.060+0000 7fe21aa59740 1 -- start start 2024-04-16T17:23:03.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.062+0000 7fe20e000640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1f80a3df0 0x7fe1f80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.062+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919a6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.062+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.062+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc919b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.062+0000 7fe215400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.063+0000 7fe21aa59740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fc9190b020 con 0x55fc919af250 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.063+0000 7fe21aa59740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fc918b1460 con 0x55fc919a5d50 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.063+0000 7fe21aa59740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc91906fe0 con 0x55fc919a6670 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.063+0000 7fe215e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc919b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.063+0000 7fe214a00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1f80a3df0 0x7fe1f80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.063+0000 7fe215e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc919b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49330/0 (socket says 172.21.15.18:49330) 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.063+0000 7fe215e00640 1 -- 172.21.15.18:0/3064815766 learned_addr learned my addr 172.21.15.18:0/3064815766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.063+0000 7fe20e000640 1 -- 172.21.15.18:0/3845723069 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7fe2080d13a0 con 0x7fe210115150 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.064+0000 7fe215e00640 1 -- 172.21.15.18:0/3064815766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 msgr2=0x55fc919aed10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.064+0000 7fe215e00640 1 --2- 172.21.15.18:0/3064815766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919aed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.064+0000 7fe215e00640 1 -- 172.21.15.18:0/3064815766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 msgr2=0x55fc919a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.064+0000 7fe215e00640 1 --2- 172.21.15.18:0/3064815766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.064+0000 7fe215e00640 1 -- 172.21.15.18:0/3064815766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc918b9040 con 0x55fc919af250 2024-04-16T17:23:03.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.064+0000 7fe215400640 1 --2- 172.21.15.18:0/3064815766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:03.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.064+0000 7fe215e00640 1 --2- 172.21.15.18:0/3064815766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc919b1640 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe20809cc60 tx=0x7fe2080923f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929f5103e884d919 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.065+0000 7fe202600640 1 -- 172.21.15.18:0/3064815766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2080cee60 con 0x55fc919af250 2024-04-16T17:23:03.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.065+0000 7fe202600640 1 -- 172.21.15.18:0/3064815766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe208018e40 con 0x55fc919af250 2024-04-16T17:23:03.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.065+0000 7fe202600640 1 -- 172.21.15.18:0/3064815766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20801d240 con 0x55fc919af250 2024-04-16T17:23:03.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.066+0000 7fe21aa59740 1 -- 172.21.15.18:0/3064815766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 msgr2=0x55fc919b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.066+0000 7fe21aa59740 1 --2- 172.21.15.18:0/3064815766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc919b1640 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe20809cc60 tx=0x7fe2080923f0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.067+0000 7fe214a00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1f80a3df0 0x7fe1f80a62b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009900 tx=0x7fe1fc02df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.067+0000 7fe21aa59740 1 -- 172.21.15.18:0/3064815766 shutdown_connections 2024-04-16T17:23:03.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.067+0000 7fe21aa59740 1 --2- 172.21.15.18:0/3064815766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc919b1640 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.067+0000 7fe21aa59740 1 --2- 172.21.15.18:0/3064815766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.067+0000 7fe21aa59740 1 --2- 172.21.15.18:0/3064815766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.067+0000 7fe21aa59740 1 -- 172.21.15.18:0/3064815766 >> 172.21.15.18:0/3064815766 conn(0x55fc919a0480 msgr2=0x55fc919a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.067+0000 7fe21aa59740 1 -- 172.21.15.18:0/3064815766 shutdown_connections 2024-04-16T17:23:03.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.068+0000 7fe21aa59740 1 -- 172.21.15.18:0/3064815766 wait complete. 2024-04-16T17:23:03.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.069+0000 7fe21aa59740 1 -- start start 2024-04-16T17:23:03.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.069+0000 7fe21aa59740 1 Processor -- start 2024-04-16T17:23:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919ac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 0x55fc919ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919af250 0x55fc919acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe21aa59740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fc919b1fe0 con 0x55fc919a6670 2024-04-16T17:23:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe21aa59740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fc918c4830 con 0x55fc919a5d50 2024-04-16T17:23:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe21aa59740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc919b2160 con 0x55fc919af250 2024-04-16T17:23:03.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe21aa59740 10 client.0 _ll_register_callbacks cb 0x55fc918cc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe215400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe215400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59186/0 (socket says 172.21.15.18:59186) 2024-04-16T17:23:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.070+0000 7fe215400640 1 -- 172.21.15.18:0/5943662 learned_addr learned my addr 172.21.15.18:0/5943662 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.071+0000 7fe215e00640 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919af250 0x55fc919acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.071+0000 7fe214a00640 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 0x55fc919ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.071+0000 7fe215e00640 1 -- 172.21.15.18:0/5943662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 msgr2=0x55fc919ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.071+0000 7fe215e00640 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.071+0000 7fe215e00640 1 -- 172.21.15.18:0/5943662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 msgr2=0x55fc919ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.071+0000 7fe215e00640 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 0x55fc919ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.071+0000 7fe215e00640 1 -- 172.21.15.18:0/5943662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc918c8920 con 0x55fc919af250 2024-04-16T17:23:03.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.072+0000 7fe215400640 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.072+0000 7fe215e00640 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919af250 0x55fc919acdf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe20809cc60 tx=0x7fe2080ce700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fbf64a57dd62755 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.073+0000 7fe1f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.073+0000 7fe1f6a00640 1 -- 172.21.15.18:0/5943662 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20804b070 con 0x55fc919af250 2024-04-16T17:23:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.073+0000 7fe1f6a00640 1 -- 172.21.15.18:0/5943662 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7fe2080995d0 con 0x55fc919af250 2024-04-16T17:23:03.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.073+0000 7fe21aa59740 1 -- 172.21.15.18:0/5943662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc91918a40 con 0x55fc919af250 2024-04-16T17:23:03.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.073+0000 7fe1f6a00640 1 -- 172.21.15.18:0/5943662 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe2080cee60 con 0x55fc919af250 2024-04-16T17:23:03.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.073+0000 7fe21aa59740 10 client.30223 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:03.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.073+0000 7fe21aa59740 1 -- 172.21.15.18:0/5943662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fc918967f0 con 0x55fc919af250 2024-04-16T17:23:03.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.074+0000 7fe1f6a00640 1 -- 172.21.15.18:0/5943662 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fc91918a40 con 0x55fc919af250 2024-04-16T17:23:03.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.074+0000 7fe1f6a00640 1 -- 172.21.15.18:0/5943662 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fe208020070 con 0x55fc919af250 2024-04-16T17:23:03.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.074+0000 7fe21aa59740 10 client.30223 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:03.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.074+0000 7fe21aa59740 10 client.30223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:23:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.074+0000 7fe21aa59740 20 client.30223 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.074+0000 7fe21aa59740 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fc919bf460 0x55fc919c1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.074+0000 7fe21aa59740 4 client.30223 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.074+0000 7fe21aa59740 1 -- 172.21.15.18:0/5943662 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc919128b0 con 0x55fc919bf460 2024-04-16T17:23:03.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.076+0000 7fe215400640 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fc919bf460 0x55fc919c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.077+0000 7fe215400640 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fc919bf460 0x55fc919c1850 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ac59d82bcbaa284 server_cookie=9af6fb452dfb9d17 in_seq=0 out_seq=0 2024-04-16T17:23:03.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.077+0000 7fe1f6a00640 10 client.30223 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:23:03.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.079+0000 7fe1f6a00640 1 -- 172.21.15.18:0/5943662 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc919128b0 con 0x55fc919bf460 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.079+0000 7fe1f6a00640 10 client.30223 handle_command_reply: tid=0 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.080+0000 7fe21aa59740 1 client.30223 shutdown 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.080+0000 7fe21aa59740 1 -- 172.21.15.18:0/5943662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919af250 msgr2=0x55fc919acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.080+0000 7fe21aa59740 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919af250 0x55fc919acdf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe20809cc60 tx=0x7fe2080ce700 comp rx=0 tx=0).stop 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.080+0000 7fe21aa59740 1 -- 172.21.15.18:0/5943662 shutdown_connections 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.080+0000 7fe21aa59740 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fc919bf460 0x55fc919c1850 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.081+0000 7fe21aa59740 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919af250 0x55fc919acdf0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.081+0000 7fe21aa59740 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 0x55fc919ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.081+0000 7fe21aa59740 1 --2- 172.21.15.18:0/5943662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.081+0000 7fe21aa59740 1 -- 172.21.15.18:0/5943662 >> 172.21.15.18:0/5943662 conn(0x55fc919a0480 msgr2=0x55fc919a21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.081+0000 7fe1f6a00640 0 client.30223 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:23:03.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.081+0000 7fe21aa59740 1 -- 172.21.15.18:0/5943662 shutdown_connections 2024-04-16T17:23:03.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.081+0000 7fe21aa59740 1 -- 172.21.15.18:0/5943662 wait complete. 2024-04-16T17:23:03.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.082+0000 7fe21aa59740 20 client.30223 trim_cache size 0 max 16384 2024-04-16T17:23:03.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.112+0000 7fe21aa59740 1 Processor -- start 2024-04-16T17:23:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.112+0000 7fe21aa59740 1 -- start start 2024-04-16T17:23:03.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a5d50 0x55fc919aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 0x55fc919bae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919af250 0x55fc919acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe21aa59740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101c0e00 con 0x55fc919a6670 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe21aa59740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101be6d0 con 0x55fc919af250 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe21aa59740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101bc500 con 0x55fc919a5d50 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe215400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a5d50 0x55fc919aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe215400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a5d50 0x55fc919aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43742/0 (socket says 172.21.15.18:43742) 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe215400640 1 -- 172.21.15.18:0/2328892672 learned_addr learned my addr 172.21.15.18:0/2328892672 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.113+0000 7fe215e00640 1 --2- 172.21.15.18:0/2328892672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919af250 0x55fc919acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.114+0000 7fe215400640 1 -- 172.21.15.18:0/2328892672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919af250 msgr2=0x55fc919acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.114+0000 7fe215400640 1 --2- 172.21.15.18:0/2328892672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919af250 0x55fc919acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.114+0000 7fe215400640 1 -- 172.21.15.18:0/2328892672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 msgr2=0x55fc919bae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.114+0000 7fe215400640 1 --2- 172.21.15.18:0/2328892672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 0x55fc919bae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.114+0000 7fe215400640 1 -- 172.21.15.18:0/2328892672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc9161d3f0 con 0x55fc919a5d50 2024-04-16T17:23:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.114+0000 7fe215400640 1 --2- 172.21.15.18:0/2328892672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a5d50 0x55fc919aeaa0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x55fc919b2120 tx=0x7fe20400b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe50328a8d3b6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.115+0000 7fe201c00640 1 -- 172.21.15.18:0/2328892672 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204004c60 con 0x55fc919a5d50 2024-04-16T17:23:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.115+0000 7fe201c00640 1 -- 172.21.15.18:0/2328892672 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe204004e00 con 0x55fc919a5d50 2024-04-16T17:23:03.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.115+0000 7fe201c00640 1 -- 172.21.15.18:0/2328892672 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2040136e0 con 0x55fc919a5d50 2024-04-16T17:23:03.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.116+0000 7fe21aa59740 1 -- 172.21.15.18:0/2328892672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a5d50 msgr2=0x55fc919aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.116+0000 7fe21aa59740 1 --2- 172.21.15.18:0/2328892672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a5d50 0x55fc919aeaa0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x55fc919b2120 tx=0x7fe20400b540 comp rx=0 tx=0).stop 2024-04-16T17:23:03.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.117+0000 7fe21aa59740 1 -- 172.21.15.18:0/2328892672 shutdown_connections 2024-04-16T17:23:03.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.117+0000 7fe21aa59740 1 --2- 172.21.15.18:0/2328892672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919af250 0x55fc919acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.117+0000 7fe21aa59740 1 --2- 172.21.15.18:0/2328892672 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919a6670 0x55fc919bae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.117+0000 7fe21aa59740 1 --2- 172.21.15.18:0/2328892672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a5d50 0x55fc919aeaa0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.117+0000 7fe21aa59740 1 -- 172.21.15.18:0/2328892672 >> 172.21.15.18:0/2328892672 conn(0x55fc919a0480 msgr2=0x55fc919ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.117+0000 7fe21aa59740 1 -- 172.21.15.18:0/2328892672 shutdown_connections 2024-04-16T17:23:03.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.117+0000 7fe21aa59740 1 -- 172.21.15.18:0/2328892672 wait complete. 2024-04-16T17:23:03.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.118+0000 7fe21aa59740 1 -- start start 2024-04-16T17:23:03.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.118+0000 7fe21aa59740 1 Processor -- start 2024-04-16T17:23:03.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe21aa59740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc91a23630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe215400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe21aa59740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101bcb90 con 0x55fc919af250 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe21aa59740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101be000 con 0x55fc919a5d50 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe21aa59740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2101bd260 con 0x55fc919a6670 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe214a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe21aa59740 10 client.0 _ll_register_callbacks cb 0x7fe20809cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe214a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919c1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43758/0 (socket says 172.21.15.18:43758) 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.120+0000 7fe215400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919c1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59200/0 (socket says 172.21.15.18:59200) 2024-04-16T17:23:03.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe214a00640 1 -- 172.21.15.18:0/1357518525 learned_addr learned my addr 172.21.15.18:0/1357518525 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe215e00640 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc91a23630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe214a00640 1 -- 172.21.15.18:0/1357518525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 msgr2=0x55fc919c1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe214a00640 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919c1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe214a00640 1 -- 172.21.15.18:0/1357518525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 msgr2=0x55fc91a23630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe214a00640 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc91a23630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe214a00640 1 -- 172.21.15.18:0/1357518525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc916e77d0 con 0x55fc919a6670 2024-04-16T17:23:03.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.121+0000 7fe215400640 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:03.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.122+0000 7fe214a00640 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919c1f40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc005c40 tx=0x7fe1fc02cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4368eb78acd40613 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.122+0000 7fe1f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:03.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.123+0000 7fe1f6a00640 1 -- 172.21.15.18:0/1357518525 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc050070 con 0x55fc919a6670 2024-04-16T17:23:03.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.123+0000 7fe1f6a00640 1 -- 172.21.15.18:0/1357518525 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7fe1fc0048d0 con 0x55fc919a6670 2024-04-16T17:23:03.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.124+0000 7fe21aa59740 1 -- 172.21.15.18:0/1357518525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe204004e00 con 0x55fc919a6670 2024-04-16T17:23:03.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.124+0000 7fe1f6a00640 1 -- 172.21.15.18:0/1357518525 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe1fc05a050 con 0x55fc919a6670 2024-04-16T17:23:03.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.124+0000 7fe21aa59740 10 client.30229 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:03.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.124+0000 7fe21aa59740 1 -- 172.21.15.18:0/1357518525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fc91a24ca0 con 0x55fc919a6670 2024-04-16T17:23:03.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.124+0000 7fe1f6a00640 1 -- 172.21.15.18:0/1357518525 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe204004e00 con 0x55fc919a6670 2024-04-16T17:23:03.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.125+0000 7fe1f6a00640 1 -- 172.21.15.18:0/1357518525 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fe1fc003cc0 con 0x55fc919a6670 2024-04-16T17:23:03.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.127+0000 7fe21aa59740 10 client.30229 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:03.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.127+0000 7fe21aa59740 10 client.30229 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:23:03.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.127+0000 7fe21aa59740 20 client.30229 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:03.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.127+0000 7fe21aa59740 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fc91a254a0 0x55fc91a278c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.127+0000 7fe21aa59740 4 client.30229 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:23:03.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.127+0000 7fe21aa59740 1 -- 172.21.15.18:0/1357518525 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc9188e570 con 0x55fc91a254a0 2024-04-16T17:23:03.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.129+0000 7fe215400640 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fc91a254a0 0x55fc91a278c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.132+0000 7fe215400640 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fc91a254a0 0x55fc91a278c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98b12c8f4d06b826 server_cookie=276a7b66e1ee0c9 in_seq=0 out_seq=0 2024-04-16T17:23:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.132+0000 7fe1f6a00640 10 client.30229 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:23:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.133+0000 7fe1f6a00640 1 -- 172.21.15.18:0/1357518525 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc9188e570 con 0x55fc91a254a0 2024-04-16T17:23:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.133+0000 7fe1f6a00640 10 client.30229 handle_command_reply: tid=0 2024-04-16T17:23:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.133+0000 7fe21aa59740 1 client.30229 shutdown 2024-04-16T17:23:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.134+0000 7fe21aa59740 1 -- 172.21.15.18:0/1357518525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 msgr2=0x55fc919c1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.134+0000 7fe21aa59740 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919c1f40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc005c40 tx=0x7fe1fc02cd50 comp rx=0 tx=0).stop 2024-04-16T17:23:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.134+0000 7fe21aa59740 1 -- 172.21.15.18:0/1357518525 shutdown_connections 2024-04-16T17:23:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.134+0000 7fe21aa59740 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fc91a254a0 0x55fc91a278c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.134+0000 7fe21aa59740 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fc919af250 0x55fc91a23630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.134+0000 7fe1f6a00640 0 client.30229 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:23:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.135+0000 7fe21aa59740 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc919a6670 0x55fc919c1f40 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.135+0000 7fe21aa59740 1 --2- 172.21.15.18:0/1357518525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc919a5d50 0x55fc919c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.135+0000 7fe21aa59740 1 -- 172.21.15.18:0/1357518525 >> 172.21.15.18:0/1357518525 conn(0x55fc919a0480 msgr2=0x55fc919acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.135+0000 7fe21aa59740 1 -- 172.21.15.18:0/1357518525 shutdown_connections 2024-04-16T17:23:03.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.135+0000 7fe21aa59740 1 -- 172.21.15.18:0/1357518525 wait complete. 2024-04-16T17:23:03.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.136+0000 7fe21aa59740 20 client.30229 trim_cache size 0 max 16384 2024-04-16T17:23:03.136 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:23:03.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.139+0000 7fe217e00640 1 -- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1f80a3df0 msgr2=0x7fe1f80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.139+0000 7fe217e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1f80a3df0 0x7fe1f80a62b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009900 tx=0x7fe1fc02df90 comp rx=0 tx=0).stop 2024-04-16T17:23:03.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.139+0000 7fe217e00640 1 -- 172.21.15.18:0/3845723069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe210115150 msgr2=0x7fe2101abfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.139+0000 7fe217e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe210115150 0x7fe2101abfe0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe20800ccb0 tx=0x7fe20800a620 comp rx=0 tx=0).stop 2024-04-16T17:23:03.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.140+0000 7fe217e00640 1 -- 172.21.15.18:0/3845723069 shutdown_connections 2024-04-16T17:23:03.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.140+0000 7fe217e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe1f80a3df0 0x7fe1f80a62b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.140+0000 7fe217e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe210115150 0x7fe2101abfe0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.140+0000 7fe217e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe210109b00 0x7fe2101a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.140+0000 7fe217e00640 1 --2- 172.21.15.18:0/3845723069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2101091e0 0x7fe2101a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.140+0000 7fe217e00640 1 -- 172.21.15.18:0/3845723069 >> 172.21.15.18:0/3845723069 conn(0x7fe21006d0a0 msgr2=0x7fe210071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.143+0000 7fe217e00640 1 -- 172.21.15.18:0/3845723069 shutdown_connections 2024-04-16T17:23:03.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.143+0000 7fe217e00640 1 -- 172.21.15.18:0/3845723069 wait complete. 2024-04-16T17:23:03.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:23:03.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:02 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2335536274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:23:03.182 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:02 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2335536274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:23:03.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:02 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2335536274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:23:03.527 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77295 12.48 MB/sec execute 195 sec latency 1096.171 ms 2024-04-16T17:23:03.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.546+0000 7f9e49400640 1 -- 172.21.15.18:0/2219638280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e3c0b7a70 msgr2=0x7f9e3c0b9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.546+0000 7f9e49400640 1 --2- 172.21.15.18:0/2219638280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e3c0b7a70 0x7f9e3c0b9e60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009900 tx=0x7f9e3002bce0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.546+0000 7f9e49400640 1 -- 172.21.15.18:0/2219638280 shutdown_connections 2024-04-16T17:23:03.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.546+0000 7f9e49400640 1 --2- 172.21.15.18:0/2219638280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e3c0b7a70 0x7f9e3c0b9e60 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.546+0000 7f9e49400640 1 --2- 172.21.15.18:0/2219638280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c0a5700 0x7f9e3c0b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.546+0000 7f9e49400640 1 --2- 172.21.15.18:0/2219638280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e3c0a4de0 0x7f9e3c0a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.546+0000 7f9e49400640 1 -- 172.21.15.18:0/2219638280 >> 172.21.15.18:0/2219638280 conn(0x7f9e3c01a640 msgr2=0x7f9e3c01aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 -- 172.21.15.18:0/2219638280 shutdown_connections 2024-04-16T17:23:03.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 -- 172.21.15.18:0/2219638280 wait complete. 2024-04-16T17:23:03.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 Processor -- start 2024-04-16T17:23:03.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 -- start start 2024-04-16T17:23:03.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c0a4de0 0x7f9e3c0b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e3c0a5700 0x7f9e3c0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c0a4de0 0x7f9e3c0b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e3c0a5700 0x7f9e3c0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e3c0a5700 0x7f9e3c0b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43772/0 (socket says 172.21.15.18:43772) 2024-04-16T17:23:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43400640 1 -- 172.21.15.18:0/2586899032 learned_addr learned my addr 172.21.15.18:0/2586899032 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e3c0b7a70 0x7f9e3c0aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c0bd690 con 0x7f9e3c0a4de0 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c0bd510 con 0x7f9e3c0b7a70 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c0bd810 con 0x7f9e3c0a5700 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e48a00640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e3c0b7a70 0x7f9e3c0aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43400640 1 -- 172.21.15.18:0/2586899032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e3c0b7a70 msgr2=0x7f9e3c0aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e3c0b7a70 0x7f9e3c0aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43400640 1 -- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c0a4de0 msgr2=0x7f9e3c0b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c0a4de0 0x7f9e3c0b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43400640 1 -- 172.21.15.18:0/2586899032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3c0af4f0 con 0x7f9e3c0a5700 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.549+0000 7f9e43e00640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c0a4de0 0x7f9e3c0b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:23:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.557+0000 7f9e43400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e3c0a5700 0x7f9e3c0b5f80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c890 tx=0x7f9e3800cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.557+0000 7f9e40c00640 1 -- 172.21.15.18:0/2586899032 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38004220 con 0x7f9e3c0a5700 2024-04-16T17:23:03.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.557+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3c0af7e0 con 0x7f9e3c0a5700 2024-04-16T17:23:03.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.557+0000 7f9e40c00640 1 -- 172.21.15.18:0/2586899032 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e380043c0 con 0x7f9e3c0a5700 2024-04-16T17:23:03.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.557+0000 7f9e40c00640 1 -- 172.21.15.18:0/2586899032 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38004a00 con 0x7f9e3c0a5700 2024-04-16T17:23:03.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.557+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3c0b37f0 con 0x7f9e3c0a5700 2024-04-16T17:23:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.567+0000 7f9e40c00640 1 -- 172.21.15.18:0/2586899032 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e380079b0 con 0x7f9e3c0a5700 2024-04-16T17:23:03.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.571+0000 7f9e4d793740 1 Processor -- start 2024-04-16T17:23:03.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.571+0000 7f9e40c00640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e2c0a3e60 0x7f9e2c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.572+0000 7f9e40c00640 1 -- 172.21.15.18:0/2586899032 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f9e380cff10 con 0x7f9e3c0a5700 2024-04-16T17:23:03.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.574+0000 7f9e48a00640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e2c0a3e60 0x7f9e2c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.575+0000 7f9e4d793740 1 -- start start 2024-04-16T17:23:03.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.576+0000 7f9e48a00640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e2c0a3e60 0x7f9e2c0a6320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e3002c270 tx=0x7f9e3002c200 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.576+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e2579362f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.576+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 0x55e25793eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.577+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257941800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.577+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e2579362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.577+0000 7f9e4d793740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e25789b020 con 0x55e25793f410 2024-04-16T17:23:03.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.577+0000 7f9e4d793740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e257841460 con 0x55e257936830 2024-04-16T17:23:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.577+0000 7f9e4d793740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e257896fe0 con 0x55e257935f10 2024-04-16T17:23:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.577+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e2579362f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43784/0 (socket says 172.21.15.18:43784) 2024-04-16T17:23:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.577+0000 7f9e43e00640 1 -- 172.21.15.18:0/2042667151 learned_addr learned my addr 172.21.15.18:0/2042667151 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.577+0000 7f9e43400640 1 --2- 172.21.15.18:0/2042667151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 0x55e25793eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.578+0000 7f9e43e00640 1 -- 172.21.15.18:0/2042667151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 msgr2=0x55e25793eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.578+0000 7f9e43e00640 1 --2- 172.21.15.18:0/2042667151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 0x55e25793eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.578+0000 7f9e43e00640 1 -- 172.21.15.18:0/2042667151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 msgr2=0x55e257941800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.578+0000 7f9e43e00640 1 --2- 172.21.15.18:0/2042667151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257941800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.578+0000 7f9e43e00640 1 -- 172.21.15.18:0/2042667151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e257849040 con 0x55e257935f10 2024-04-16T17:23:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.579+0000 7f9e43e00640 1 --2- 172.21.15.18:0/2042667151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e2579362f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9e44000b60 tx=0x7f9e44069e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae127a82460aaf52 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.580+0000 7f9e35200640 1 -- 172.21.15.18:0/2042667151 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44075c20 con 0x55e257935f10 2024-04-16T17:23:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.580+0000 7f9e35200640 1 -- 172.21.15.18:0/2042667151 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4407a070 con 0x55e257935f10 2024-04-16T17:23:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.580+0000 7f9e35200640 1 -- 172.21.15.18:0/2042667151 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44068e00 con 0x55e257935f10 2024-04-16T17:23:03.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.581+0000 7f9e4d793740 1 -- 172.21.15.18:0/2042667151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 msgr2=0x55e2579362f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.582+0000 7f9e4d793740 1 --2- 172.21.15.18:0/2042667151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e2579362f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9e44000b60 tx=0x7f9e44069e50 comp rx=0 tx=0).stop 2024-04-16T17:23:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.582+0000 7f9e4d793740 1 -- 172.21.15.18:0/2042667151 shutdown_connections 2024-04-16T17:23:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.582+0000 7f9e4d793740 1 --2- 172.21.15.18:0/2042667151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257941800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.582+0000 7f9e4d793740 1 --2- 172.21.15.18:0/2042667151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 0x55e25793eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.582+0000 7f9e4d793740 1 --2- 172.21.15.18:0/2042667151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e2579362f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.582+0000 7f9e4d793740 1 -- 172.21.15.18:0/2042667151 >> 172.21.15.18:0/2042667151 conn(0x55e257930640 msgr2=0x55e257930a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.582+0000 7f9e4d793740 1 -- 172.21.15.18:0/2042667151 shutdown_connections 2024-04-16T17:23:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.582+0000 7f9e4d793740 1 -- 172.21.15.18:0/2042667151 wait complete. 2024-04-16T17:23:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.583+0000 7f9e4d793740 1 -- start start 2024-04-16T17:23:03.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.583+0000 7f9e4d793740 1 Processor -- start 2024-04-16T17:23:03.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.584+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e257935f10 0x55e25793c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25793ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e25793f410 0x55e25793cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e4d793740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e257942200 con 0x55e257935f10 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e4d793740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e257854830 con 0x55e25793f410 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e4d793740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e257942380 con 0x55e257936830 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e4d793740 10 client.0 _ll_register_callbacks cb 0x55e25785c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25793ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25793ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43796/0 (socket says 172.21.15.18:43796) 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e43400640 1 -- 172.21.15.18:0/552665537 learned_addr learned my addr 172.21.15.18:0/552665537 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e43400640 1 -- 172.21.15.18:0/552665537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e25793f410 msgr2=0x55e25793cf70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e43400640 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e25793f410 0x55e25793cf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e43400640 1 -- 172.21.15.18:0/552665537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e257935f10 msgr2=0x55e25793c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.585+0000 7f9e43e00640 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e257935f10 0x55e25793c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.586+0000 7f9e43400640 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e257935f10 0x55e25793c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.586+0000 7f9e43400640 1 -- 172.21.15.18:0/552665537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e257858920 con 0x55e257936830 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.586+0000 7f9e43400640 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25793ca30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9e38097ed0 tx=0x7f9e3809c230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4481ff2053226c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.586+0000 7f9e29600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.586+0000 7f9e29600640 1 -- 172.21.15.18:0/552665537 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3809fe20 con 0x55e257936830 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.587+0000 7f9e29600640 1 -- 172.21.15.18:0/552665537 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f9e380cd4a0 con 0x55e257936830 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.587+0000 7f9e4d793740 1 -- 172.21.15.18:0/552665537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4407a070 con 0x55e257936830 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.587+0000 7f9e29600640 1 -- 172.21.15.18:0/552665537 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9e380cd6a0 con 0x55e257936830 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.587+0000 7f9e4d793740 10 client.30241 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.587+0000 7f9e4d793740 1 -- 172.21.15.18:0/552665537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e2578267f0 con 0x55e257936830 2024-04-16T17:23:03.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.588+0000 7f9e29600640 1 -- 172.21.15.18:0/552665537 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4407a070 con 0x55e257936830 2024-04-16T17:23:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.588+0000 7f9e29600640 1 -- 172.21.15.18:0/552665537 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f9e3809b040 con 0x55e257936830 2024-04-16T17:23:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.588+0000 7f9e4d793740 10 client.30241 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.588+0000 7f9e4d793740 10 client.30241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.588+0000 7f9e4d793740 20 client.30241 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.588+0000 7f9e4d793740 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e25794f900 0x55e257951dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.588+0000 7f9e4d793740 4 client.30241 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:03.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.588+0000 7f9e4d793740 1 -- 172.21.15.18:0/552665537 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2578a8a40 con 0x55e25794f900 2024-04-16T17:23:03.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.603+0000 7f9e43e00640 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e25794f900 0x55e257951dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.604+0000 7f9e43e00640 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e25794f900 0x55e257951dc0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d14d88f02d7fcc8e server_cookie=1786719218fae3fc in_seq=0 out_seq=0 2024-04-16T17:23:03.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.604+0000 7f9e29600640 10 client.30241 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:03.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.605+0000 7f9e29600640 1 -- 172.21.15.18:0/552665537 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2578a8a40 con 0x55e25794f900 2024-04-16T17:23:03.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.605+0000 7f9e29600640 10 client.30241 handle_command_reply: tid=0 2024-04-16T17:23:03.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.605+0000 7f9e4d793740 1 client.30241 shutdown 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.606+0000 7f9e4d793740 1 -- 172.21.15.18:0/552665537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 msgr2=0x55e25793ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.606+0000 7f9e4d793740 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25793ca30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9e38097ed0 tx=0x7f9e3809c230 comp rx=0 tx=0).stop 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.606+0000 7f9e4d793740 1 -- 172.21.15.18:0/552665537 shutdown_connections 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.606+0000 7f9e4d793740 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e25794f900 0x55e257951dc0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.607+0000 7f9e4d793740 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e25793f410 0x55e25793cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.607+0000 7f9e4d793740 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25793ca30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.607+0000 7f9e4d793740 1 --2- 172.21.15.18:0/552665537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e257935f10 0x55e25793c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.607+0000 7f9e4d793740 1 -- 172.21.15.18:0/552665537 >> 172.21.15.18:0/552665537 conn(0x55e257930640 msgr2=0x55e257932390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.607+0000 7f9e29600640 0 client.30241 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.607+0000 7f9e4d793740 1 -- 172.21.15.18:0/552665537 shutdown_connections 2024-04-16T17:23:03.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.607+0000 7f9e4d793740 1 -- 172.21.15.18:0/552665537 wait complete. 2024-04-16T17:23:03.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.607+0000 7f9e4d793740 20 client.30241 trim_cache size 0 max 16384 2024-04-16T17:23:03.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.638+0000 7f9e4d793740 1 Processor -- start 2024-04-16T17:23:03.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.638+0000 7f9e4d793740 1 -- start start 2024-04-16T17:23:03.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.638+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257935f10 0x55e25793ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.638+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25794b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.638+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257957690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.638+0000 7f9e4d793740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c165960 con 0x55e25793f410 2024-04-16T17:23:03.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.638+0000 7f9e4d793740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c163230 con 0x55e257935f10 2024-04-16T17:23:03.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.638+0000 7f9e4d793740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1610a0 con 0x55e257936830 2024-04-16T17:23:03.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.639+0000 7f9e43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25794b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.639+0000 7f9e43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25794b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43810/0 (socket says 172.21.15.18:43810) 2024-04-16T17:23:03.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.639+0000 7f9e43400640 1 -- 172.21.15.18:0/4046202998 learned_addr learned my addr 172.21.15.18:0/4046202998 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.639+0000 7f9e43e00640 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257935f10 0x55e25793ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.639+0000 7f9e48a00640 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257957690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.641+0000 7f9e48a00640 1 -- 172.21.15.18:0/4046202998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 msgr2=0x55e25794b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.641+0000 7f9e48a00640 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25794b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.641+0000 7f9e48a00640 1 -- 172.21.15.18:0/4046202998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257935f10 msgr2=0x55e25793ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.641+0000 7f9e48a00640 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257935f10 0x55e25793ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.641+0000 7f9e48a00640 1 -- 172.21.15.18:0/4046202998 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2575ad3f0 con 0x55e25793f410 2024-04-16T17:23:03.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.642+0000 7f9e48a00640 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257957690 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9e30036040 tx=0x7f9e3002bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3303b40da78ded server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.643+0000 7f9e2be00640 1 -- 172.21.15.18:0/4046202998 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30051070 con 0x55e25793f410 2024-04-16T17:23:03.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.643+0000 7f9e2be00640 1 -- 172.21.15.18:0/4046202998 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e30048040 con 0x55e25793f410 2024-04-16T17:23:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.644+0000 7f9e4d793740 1 -- 172.21.15.18:0/4046202998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 msgr2=0x55e257957690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.645+0000 7f9e2be00640 1 -- 172.21.15.18:0/4046202998 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30004e90 con 0x55e25793f410 2024-04-16T17:23:03.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.645+0000 7f9e4d793740 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257957690 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9e30036040 tx=0x7f9e3002bce0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.645+0000 7f9e4d793740 1 -- 172.21.15.18:0/4046202998 shutdown_connections 2024-04-16T17:23:03.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.645+0000 7f9e4d793740 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257957690 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.645+0000 7f9e4d793740 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257936830 0x55e25794b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.645+0000 7f9e4d793740 1 --2- 172.21.15.18:0/4046202998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257935f10 0x55e25793ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.645+0000 7f9e4d793740 1 -- 172.21.15.18:0/4046202998 >> 172.21.15.18:0/4046202998 conn(0x55e257930640 msgr2=0x55e257936c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.645+0000 7f9e4d793740 1 -- 172.21.15.18:0/4046202998 shutdown_connections 2024-04-16T17:23:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.646+0000 7f9e4d793740 1 -- 172.21.15.18:0/4046202998 wait complete. 2024-04-16T17:23:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.646+0000 7f9e4d793740 1 -- start start 2024-04-16T17:23:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.646+0000 7f9e4d793740 1 Processor -- start 2024-04-16T17:23:03.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.649+0000 7f9e4d793740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e257943040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.649+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e257943040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.649+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e257943040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43818/0 (socket says 172.21.15.18:43818) 2024-04-16T17:23:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.649+0000 7f9e43e00640 1 -- 172.21.15.18:0/3513754736 learned_addr learned my addr 172.21.15.18:0/3513754736 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.649+0000 7f9e4d793740 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 0x55e257943580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.649+0000 7f9e4d793740 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257943ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.650+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1616f0 con 0x55e25793f410 2024-04-16T17:23:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.650+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c162b60 con 0x55e257936830 2024-04-16T17:23:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.650+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c161dc0 con 0x55e257935f10 2024-04-16T17:23:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.650+0000 7f9e43400640 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 0x55e257943580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.650+0000 7f9e4d793740 10 client.0 _ll_register_callbacks cb 0x7f9e380058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:03.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.651+0000 7f9e43e00640 1 -- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 msgr2=0x55e257943580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.651+0000 7f9e43e00640 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 0x55e257943580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.651+0000 7f9e43e00640 1 -- 172.21.15.18:0/3513754736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 msgr2=0x55e257943ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.651+0000 7f9e43e00640 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257943ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.652+0000 7f9e43e00640 1 -- 172.21.15.18:0/3513754736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2576777d0 con 0x55e257935f10 2024-04-16T17:23:03.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.653+0000 7f9e43e00640 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e257943040 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9e44051690 tx=0x7f9e440681c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fac5ccb7b4f3288e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:03.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.653+0000 7f9e29600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:03.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.653+0000 7f9e29600640 1 -- 172.21.15.18:0/3513754736 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44075e30 con 0x55e257935f10 2024-04-16T17:23:03.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.654+0000 7f9e29600640 1 -- 172.21.15.18:0/3513754736 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9510+0+0 (secure 0 0 0) 0x7f9e4408c030 con 0x55e257935f10 2024-04-16T17:23:03.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.654+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e30048040 con 0x55e257935f10 2024-04-16T17:23:03.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.655+0000 7f9e29600640 1 -- 172.21.15.18:0/3513754736 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9e44050c80 con 0x55e257935f10 2024-04-16T17:23:03.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.655+0000 7f9e4d793740 10 client.30247 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:03.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.655+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e2579b6080 con 0x55e257935f10 2024-04-16T17:23:03.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.656+0000 7f9e29600640 1 -- 172.21.15.18:0/3513754736 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e30048040 con 0x55e257935f10 2024-04-16T17:23:03.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.656+0000 7f9e29600640 1 -- 172.21.15.18:0/3513754736 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f9e44097720 con 0x55e257935f10 2024-04-16T17:23:03.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.656+0000 7f9e4d793740 10 client.30247 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:03.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.656+0000 7f9e4d793740 10 client.30247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.656+0000 7f9e4d793740 20 client.30247 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.656+0000 7f9e4d793740 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2579b6690 0x55e2579b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.656+0000 7f9e4d793740 4 client.30247 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:23:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.656+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2578a28b0 con 0x55e2579b6690 2024-04-16T17:23:03.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.657+0000 7f9e43400640 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2579b6690 0x55e2579b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.658+0000 7f9e43400640 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2579b6690 0x55e2579b8ab0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beade9839cc6dcf0 server_cookie=c1087566208877fa in_seq=0 out_seq=0 2024-04-16T17:23:03.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.658+0000 7f9e29600640 10 client.30247 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:03.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.658+0000 7f9e29600640 1 -- 172.21.15.18:0/3513754736 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2578a28b0 con 0x55e2579b6690 2024-04-16T17:23:03.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.658+0000 7f9e29600640 10 client.30247 handle_command_reply: tid=0 2024-04-16T17:23:03.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.658+0000 7f9e4d793740 1 client.30247 shutdown 2024-04-16T17:23:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 msgr2=0x55e257943040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e4d793740 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e257943040 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9e44051690 tx=0x7f9e440681c0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 shutdown_connections 2024-04-16T17:23:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e4d793740 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2579b6690 0x55e2579b8ab0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e4d793740 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e25793f410 0x55e257943ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e4d793740 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e257936830 0x55e257943580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e4d793740 1 --2- 172.21.15.18:0/3513754736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e257935f10 0x55e257943040 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e29600640 0 client.30247 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:03.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.666+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 >> 172.21.15.18:0/3513754736 conn(0x55e257930640 msgr2=0x55e25792ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.672+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 shutdown_connections 2024-04-16T17:23:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.672+0000 7f9e4d793740 1 -- 172.21.15.18:0/3513754736 wait complete. 2024-04-16T17:23:03.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.681+0000 7f9e4d793740 20 client.30247 trim_cache size 0 max 16384 2024-04-16T17:23:03.692 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:23:03.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.689+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e2c0a3e60 msgr2=0x7f9e2c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.689+0000 7f9e49400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e2c0a3e60 0x7f9e2c0a6320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e3002c270 tx=0x7f9e3002c200 comp rx=0 tx=0).stop 2024-04-16T17:23:03.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.689+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e3c0a5700 msgr2=0x7f9e3c0b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:03.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.689+0000 7f9e49400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e3c0a5700 0x7f9e3c0b5f80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c890 tx=0x7f9e3800cd50 comp rx=0 tx=0).stop 2024-04-16T17:23:03.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.694+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 shutdown_connections 2024-04-16T17:23:03.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.694+0000 7f9e49400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e2c0a3e60 0x7f9e2c0a6320 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.694+0000 7f9e49400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e3c0b7a70 0x7f9e3c0aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.694+0000 7f9e49400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e3c0a5700 0x7f9e3c0b5f80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.694+0000 7f9e49400640 1 --2- 172.21.15.18:0/2586899032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e3c0a4de0 0x7f9e3c0b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:03.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.694+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 >> 172.21.15.18:0/2586899032 conn(0x7f9e3c01a640 msgr2=0x7f9e3c0a1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:03.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.696+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 shutdown_connections 2024-04-16T17:23:03.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:03.696+0000 7f9e49400640 1 -- 172.21.15.18:0/2586899032 wait complete. 2024-04-16T17:23:03.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:03 smithi018 ceph-mon[31776]: pgmap v584: 129 pgs: 129 active+clean; 422 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-16T17:23:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:03 smithi022 ceph-mon[36023]: pgmap v584: 129 pgs: 129 active+clean; 422 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-16T17:23:04.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:03 smithi043 ceph-mon[36901]: pgmap v584: 129 pgs: 129 active+clean; 422 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-16T17:23:04.529 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77323 12.42 MB/sec execute 196 sec latency 1109.983 ms 2024-04-16T17:23:04.726 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:23:04.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:23:04.727 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:23:05.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.033+0000 7f45e9a00640 1 -- 172.21.15.18:0/139652028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e4104450 msgr2=0x7f45e4104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.033+0000 7f45e9a00640 1 --2- 172.21.15.18:0/139652028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e4104450 0x7f45e4104830 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009ca0 tx=0x7f45dc02c120 comp rx=0 tx=0).stop 2024-04-16T17:23:05.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.034+0000 7f45e9a00640 1 -- 172.21.15.18:0/139652028 shutdown_connections 2024-04-16T17:23:05.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.034+0000 7f45e9a00640 1 --2- 172.21.15.18:0/139652028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45e4109520 0x7f45e41110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.034+0000 7f45e9a00640 1 --2- 172.21.15.18:0/139652028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45e4104e00 0x7f45e4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.034+0000 7f45e9a00640 1 --2- 172.21.15.18:0/139652028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e4104450 0x7f45e4104830 secure :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009ca0 tx=0x7f45dc02c120 comp rx=0 tx=0).stop 2024-04-16T17:23:05.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.034+0000 7f45e9a00640 1 -- 172.21.15.18:0/139652028 >> 172.21.15.18:0/139652028 conn(0x7f45e40ffc90 msgr2=0x7f45e41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.038+0000 7f45e9a00640 1 -- 172.21.15.18:0/139652028 shutdown_connections 2024-04-16T17:23:05.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.040+0000 7f45e9a00640 1 -- 172.21.15.18:0/139652028 wait complete. 2024-04-16T17:23:05.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.043+0000 7f45e9a00640 1 Processor -- start 2024-04-16T17:23:05.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.043+0000 7f45e9a00640 1 -- start start 2024-04-16T17:23:05.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45e4109520 0x7f45e419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e419b3e0 0x7f45e419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e9a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45e419fca0 0x7f45e41a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e9a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f45e41146d0 con 0x7f45e419fca0 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e9a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45e4114550 con 0x7f45e4109520 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e9a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45e4114850 con 0x7f45e419b3e0 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e419b3e0 0x7f45e419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45e4109520 0x7f45e419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e419b3e0 0x7f45e419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43840/0 (socket says 172.21.15.18:43840) 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e2000640 1 -- 172.21.15.18:0/3938706612 learned_addr learned my addr 172.21.15.18:0/3938706612 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.044+0000 7f45e3400640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45e419fca0 0x7f45e41a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.045+0000 7f45e2a00640 1 -- 172.21.15.18:0/3938706612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e419b3e0 msgr2=0x7f45e419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.045+0000 7f45e2a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e419b3e0 0x7f45e419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.045+0000 7f45e2a00640 1 -- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45e419fca0 msgr2=0x7f45e41a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.045+0000 7f45e2a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45e419fca0 0x7f45e41a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.045+0000 7f45e2a00640 1 -- 172.21.15.18:0/3938706612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e41a7a10 con 0x7f45e4109520 2024-04-16T17:23:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.048+0000 7f45e2a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45e4109520 0x7f45e419aea0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f45dc008000 tx=0x7f45dc035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.051+0000 7f45db400640 1 -- 172.21.15.18:0/3938706612 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc039050 con 0x7f45e4109520 2024-04-16T17:23:05.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.051+0000 7f45db400640 1 -- 172.21.15.18:0/3938706612 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45dc03a040 con 0x7f45e4109520 2024-04-16T17:23:05.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.051+0000 7f45e9a00640 1 -- 172.21.15.18:0/3938706612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e41a7be0 con 0x7f45e4109520 2024-04-16T17:23:05.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.051+0000 7f45db400640 1 -- 172.21.15.18:0/3938706612 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc004580 con 0x7f45e4109520 2024-04-16T17:23:05.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.051+0000 7f45e9a00640 1 -- 172.21.15.18:0/3938706612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e41a8050 con 0x7f45e4109520 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.055+0000 7f45ec7c3740 1 Processor -- start 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.056+0000 7f45ec7c3740 1 -- start start 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.056+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1d8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.060+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.060+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1e3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.060+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55eb6d13ccb0 con 0x55eb6d1d85c0 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.060+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55eb6d0e3370 con 0x55eb6d1e11a0 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.060+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eb6d138c70 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.061+0000 7f45db400640 1 -- 172.21.15.18:0/3938706612 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45dc047070 con 0x7f45e4109520 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.061+0000 7f45e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.061+0000 7f45e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1e3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59276/0 (socket says 172.21.15.18:59276) 2024-04-16T17:23:05.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.061+0000 7f45e2000640 1 -- 172.21.15.18:0/3532044377 learned_addr learned my addr 172.21.15.18:0/3532044377 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.062+0000 7f45e2000640 1 -- 172.21.15.18:0/3532044377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 msgr2=0x55eb6d1d8080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.062+0000 7f45e2000640 1 --2- 172.21.15.18:0/3532044377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1d8080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.062+0000 7f45e2000640 1 -- 172.21.15.18:0/3532044377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d85c0 msgr2=0x55eb6d1e0c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.062+0000 7f45e2000640 1 --2- 172.21.15.18:0/3532044377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1e0c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.062+0000 7f45e2000640 1 -- 172.21.15.18:0/3532044377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb6d102910 con 0x55eb6d1e11a0 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.062+0000 7f45e2000640 1 --2- 172.21.15.18:0/3532044377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1e3590 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009af0 tx=0x7f45d400e1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc042e001cc2adc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.063+0000 7f45cfe00640 1 -- 172.21.15.18:0/3532044377 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4017070 con 0x55eb6d1e11a0 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.063+0000 7f45cfe00640 1 -- 172.21.15.18:0/3532044377 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d400ec10 con 0x55eb6d1e11a0 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.063+0000 7f45cfe00640 1 -- 172.21.15.18:0/3532044377 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d40126c0 con 0x55eb6d1e11a0 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.063+0000 7f45db400640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45c80a3ef0 0x7f45c80a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.064+0000 7f45db400640 1 -- 172.21.15.18:0/3938706612 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f45dc0f2900 con 0x7f45e4109520 2024-04-16T17:23:05.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.064+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3532044377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 msgr2=0x55eb6d1e3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.064+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3532044377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1e3590 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009af0 tx=0x7f45d400e1d0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.065+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3532044377 shutdown_connections 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.065+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3532044377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1e3590 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.065+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3532044377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1e0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.065+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3532044377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.065+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3532044377 >> 172.21.15.18:0/3532044377 conn(0x55eb6d1d23d0 msgr2=0x55eb6d1d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.065+0000 7f45e3400640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45c80a3ef0 0x7f45c80a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.067+0000 7f45e3400640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45c80a3ef0 0x7f45c80a63b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f45d000b7b0 tx=0x7f45d0008040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.067+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3532044377 shutdown_connections 2024-04-16T17:23:05.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.067+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3532044377 wait complete. 2024-04-16T17:23:05.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.071+0000 7f45ec7c3740 1 -- start start 2024-04-16T17:23:05.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.076+0000 7f45ec7c3740 1 Processor -- start 2024-04-16T17:23:05.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.080+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1de220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.081+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.081+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.081+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55eb6d1e3f90 con 0x55eb6d1e11a0 2024-04-16T17:23:05.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.081+0000 7f45e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1de220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.081+0000 7f45e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1de220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43858/0 (socket says 172.21.15.18:43858) 2024-04-16T17:23:05.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.083+0000 7f45e2a00640 1 -- 172.21.15.18:0/3055818860 learned_addr learned my addr 172.21.15.18:0/3055818860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.081+0000 7f45e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.083+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55eb6d0f6730 con 0x55eb6d1d85c0 2024-04-16T17:23:05.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.082+0000 7f45e3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.083+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eb6d1e4110 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.084+0000 7f45ec7c3740 10 client.0 _ll_register_callbacks cb 0x55eb6d0c51e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:05.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.084+0000 7f45e2a00640 1 -- 172.21.15.18:0/3055818860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 msgr2=0x55eb6d1de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.084+0000 7f45e2a00640 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.084+0000 7f45e2a00640 1 -- 172.21.15.18:0/3055818860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1e11a0 msgr2=0x55eb6d1deca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.084+0000 7f45e2a00640 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.084+0000 7f45e2a00640 1 -- 172.21.15.18:0/3055818860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb6cf177e0 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.084+0000 7f45e2000640 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.085+0000 7f45e2a00640 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1de220 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f45dc033000 tx=0x7f45dc0be030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbfba4fc3c2e0691 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.085+0000 7f45ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.085+0000 7f45ccc00640 1 -- 172.21.15.18:0/3055818860 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0d7060 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.085+0000 7f45ccc00640 1 -- 172.21.15.18:0/3055818860 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f45dc0efe10 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.085+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d400ec10 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.085+0000 7f45ccc00640 1 -- 172.21.15.18:0/3055818860 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f45dc0bebf0 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.085+0000 7f45ec7c3740 10 client.30256 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.085+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eb6cf77f70 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.086+0000 7f45ccc00640 1 -- 172.21.15.18:0/3055818860 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d400ec10 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.087+0000 7f45ccc00640 1 -- 172.21.15.18:0/3055818860 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f45dc02dce0 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.087+0000 7f45ec7c3740 10 client.30256 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:05.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.087+0000 7f45ec7c3740 10 client.30256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:05.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.087+0000 7f45ec7c3740 20 client.30256 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:05.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.087+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55eb6d1f14f0 0x55eb6d1f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.087+0000 7f45ec7c3740 4 client.30256 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:05.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.087+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb6d14a6d0 con 0x55eb6d1f14f0 2024-04-16T17:23:05.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.088+0000 7f45e2000640 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55eb6d1f14f0 0x55eb6d1f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.089+0000 7f45e2000640 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55eb6d1f14f0 0x55eb6d1f3960 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3e0cd988871cf8 server_cookie=7067c2839689400e in_seq=0 out_seq=0 2024-04-16T17:23:05.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.089+0000 7f45ccc00640 10 client.30256 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.117+0000 7f45ccc00640 1 -- 172.21.15.18:0/3055818860 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb6d14a6d0 con 0x55eb6d1f14f0 2024-04-16T17:23:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.117+0000 7f45ccc00640 10 client.30256 handle_command_reply: tid=0 2024-04-16T17:23:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.117+0000 7f45ec7c3740 1 client.30256 shutdown 2024-04-16T17:23:05.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.117+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 msgr2=0x55eb6d1de220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.117+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1de220 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f45dc033000 tx=0x7f45dc0be030 comp rx=0 tx=0).stop 2024-04-16T17:23:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 shutdown_connections 2024-04-16T17:23:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55eb6d1f14f0 0x55eb6d1f3960 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/3055818860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1de220 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 >> 172.21.15.18:0/3055818860 conn(0x55eb6d1d23d0 msgr2=0x55eb6d1d4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ccc00640 0 client.30256 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 shutdown_connections 2024-04-16T17:23:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 1 -- 172.21.15.18:0/3055818860 wait complete. 2024-04-16T17:23:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.118+0000 7f45ec7c3740 20 client.30256 trim_cache size 0 max 16384 2024-04-16T17:23:05.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.165+0000 7f45ec7c3740 1 Processor -- start 2024-04-16T17:23:05.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.165+0000 7f45ec7c3740 1 -- start start 2024-04-16T17:23:05.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.165+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.165+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1dd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.166+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.166+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f45e41b4760 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.167+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45e41b2030 con 0x55eb6d1d85c0 2024-04-16T17:23:05.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.167+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45e41afea0 con 0x55eb6d1e11a0 2024-04-16T17:23:05.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.167+0000 7f45e3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.167+0000 7f45e3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43874/0 (socket says 172.21.15.18:43874) 2024-04-16T17:23:05.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.167+0000 7f45e3400640 1 -- 172.21.15.18:0/504908965 learned_addr learned my addr 172.21.15.18:0/504908965 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.167+0000 7f45e2a00640 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.167+0000 7f45e2000640 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1dd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.169+0000 7f45e2000640 1 -- 172.21.15.18:0/504908965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1e11a0 msgr2=0x55eb6d1deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.169+0000 7f45e2000640 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.169+0000 7f45e2000640 1 -- 172.21.15.18:0/504908965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 msgr2=0x55eb6d1ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.169+0000 7f45e2000640 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.169+0000 7f45e2000640 1 -- 172.21.15.18:0/504908965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb6d1d83f0 con 0x55eb6d1d85c0 2024-04-16T17:23:05.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.169+0000 7f45e2000640 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1dd3c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f45d4004700 tx=0x7f45d4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=225c9e3f53c0a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.170+0000 7f45cf400640 1 -- 172.21.15.18:0/504908965 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4017040 con 0x55eb6d1d85c0 2024-04-16T17:23:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.170+0000 7f45cf400640 1 -- 172.21.15.18:0/504908965 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d4009420 con 0x55eb6d1d85c0 2024-04-16T17:23:05.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.170+0000 7f45cf400640 1 -- 172.21.15.18:0/504908965 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d401c670 con 0x55eb6d1d85c0 2024-04-16T17:23:05.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 -- 172.21.15.18:0/504908965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 msgr2=0x55eb6d1dd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1dd3c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f45d4004700 tx=0x7f45d4008040 comp rx=0 tx=0).stop 2024-04-16T17:23:05.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 -- 172.21.15.18:0/504908965 shutdown_connections 2024-04-16T17:23:05.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1dd3c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/504908965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 -- 172.21.15.18:0/504908965 >> 172.21.15.18:0/504908965 conn(0x55eb6d1d23d0 msgr2=0x55eb6d1d37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 -- 172.21.15.18:0/504908965 shutdown_connections 2024-04-16T17:23:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.171+0000 7f45ec7c3740 1 -- 172.21.15.18:0/504908965 wait complete. 2024-04-16T17:23:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.172+0000 7f45ec7c3740 1 -- start start 2024-04-16T17:23:05.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.172+0000 7f45ec7c3740 1 Processor -- start 2024-04-16T17:23:05.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.173+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1dad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1db2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45ec7c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f45e41b04f0 con 0x55eb6d1d7ca0 2024-04-16T17:23:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45e41b1960 con 0x55eb6d1e11a0 2024-04-16T17:23:05.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45ec7c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45e41b0bc0 con 0x55eb6d1d85c0 2024-04-16T17:23:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45ec7c3740 10 client.0 _ll_register_callbacks cb 0x7f45dc0ba860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1db7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59300/0 (socket says 172.21.15.18:59300) 2024-04-16T17:23:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.174+0000 7f45e3400640 1 -- 172.21.15.18:0/858905724 learned_addr learned my addr 172.21.15.18:0/858905724 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.175+0000 7f45e3400640 1 -- 172.21.15.18:0/858905724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d85c0 msgr2=0x55eb6d1db2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:23:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.175+0000 7f45e3400640 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1db2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.175+0000 7f45e3400640 1 -- 172.21.15.18:0/858905724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 msgr2=0x55eb6d1dad70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:05.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.176+0000 7f45e3400640 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1dad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.176+0000 7f45e3400640 1 -- 172.21.15.18:0/858905724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb6ce0f170 con 0x55eb6d1e11a0 2024-04-16T17:23:05.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.176+0000 7f45ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:05.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.176+0000 7f45e3400640 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1db7f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f45d000df20 tx=0x7f45d0004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1012f0e656f1a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.177+0000 7f45ccc00640 1 -- 172.21.15.18:0/858905724 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0026020 con 0x55eb6d1e11a0 2024-04-16T17:23:05.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.178+0000 7f45ccc00640 1 -- 172.21.15.18:0/858905724 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f45d000f040 con 0x55eb6d1e11a0 2024-04-16T17:23:05.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.178+0000 7f45ec7c3740 1 -- 172.21.15.18:0/858905724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d4009420 con 0x55eb6d1e11a0 2024-04-16T17:23:05.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.179+0000 7f45ccc00640 1 -- 172.21.15.18:0/858905724 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f45d0014e40 con 0x55eb6d1e11a0 2024-04-16T17:23:05.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.179+0000 7f45ec7c3740 10 client.30395 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:05.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.179+0000 7f45ec7c3740 1 -- 172.21.15.18:0/858905724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eb6d260310 con 0x55eb6d1e11a0 2024-04-16T17:23:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.180+0000 7f45ccc00640 1 -- 172.21.15.18:0/858905724 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d4009420 con 0x55eb6d1e11a0 2024-04-16T17:23:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.181+0000 7f45ccc00640 1 -- 172.21.15.18:0/858905724 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f45d0013070 con 0x55eb6d1e11a0 2024-04-16T17:23:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.181+0000 7f45ec7c3740 10 client.30395 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:05.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.181+0000 7f45ec7c3740 10 client.30395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:05.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.181+0000 7f45ec7c3740 20 client.30395 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:05.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.181+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55eb6d260a40 0x55eb6d262ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.181+0000 7f45ec7c3740 4 client.30395 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:23:05.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.181+0000 7f45ec7c3740 1 -- 172.21.15.18:0/858905724 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eb6d144540 con 0x55eb6d260a40 2024-04-16T17:23:05.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.181+0000 7f45e2a00640 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55eb6d260a40 0x55eb6d262ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.182+0000 7f45e2a00640 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55eb6d260a40 0x55eb6d262ee0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7391f9f81b1de87a server_cookie=a15f0998476440a7 in_seq=0 out_seq=0 2024-04-16T17:23:05.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.182+0000 7f45ccc00640 10 client.30395 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:05.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.187+0000 7f45ccc00640 1 -- 172.21.15.18:0/858905724 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eb6d144540 con 0x55eb6d260a40 2024-04-16T17:23:05.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.187+0000 7f45ccc00640 10 client.30395 handle_command_reply: tid=0 2024-04-16T17:23:05.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.187+0000 7f45ec7c3740 1 client.30395 shutdown 2024-04-16T17:23:05.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 -- 172.21.15.18:0/858905724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 msgr2=0x55eb6d1db7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1db7f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f45d000df20 tx=0x7f45d0004700 comp rx=0 tx=0).stop 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 -- 172.21.15.18:0/858905724 shutdown_connections 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55eb6d260a40 0x55eb6d262ee0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb6d1e11a0 0x55eb6d1db7f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eb6d1d85c0 0x55eb6d1db2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 --2- 172.21.15.18:0/858905724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55eb6d1d7ca0 0x55eb6d1dad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 -- 172.21.15.18:0/858905724 >> 172.21.15.18:0/858905724 conn(0x55eb6d1d23d0 msgr2=0x55eb6d1dd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ccc00640 0 client.30395 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 -- 172.21.15.18:0/858905724 shutdown_connections 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.188+0000 7f45ec7c3740 1 -- 172.21.15.18:0/858905724 wait complete. 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.189+0000 7f45ec7c3740 20 client.30395 trim_cache size 0 max 16384 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:23:05.189 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "6879e586-846e-4af6-b8e0-3f57c43c50fa", 2024-04-16T17:23:05.190 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:23:05.190 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 -- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45c80a3ef0 msgr2=0x7f45c80a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45c80a3ef0 0x7f45c80a63b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f45d000b7b0 tx=0x7f45d0008040 comp rx=0 tx=0).stop 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 -- 172.21.15.18:0/3938706612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45e4109520 msgr2=0x7f45e419aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45e4109520 0x7f45e419aea0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f45dc008000 tx=0x7f45dc035d90 comp rx=0 tx=0).stop 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 -- 172.21.15.18:0/3938706612 shutdown_connections 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f45c80a3ef0 0x7f45c80a63b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45e419fca0 0x7f45e41a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e419b3e0 0x7f45e419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 --2- 172.21.15.18:0/3938706612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45e4109520 0x7f45e419aea0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.190+0000 7f45e9a00640 1 -- 172.21.15.18:0/3938706612 >> 172.21.15.18:0/3938706612 conn(0x7f45e40ffc90 msgr2=0x7f45e41007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.191+0000 7f45e9a00640 1 -- 172.21.15.18:0/3938706612 shutdown_connections 2024-04-16T17:23:05.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.191+0000 7f45e9a00640 1 -- 172.21.15.18:0/3938706612 wait complete. 2024-04-16T17:23:05.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:23:05.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:23:05.530 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77355 12.37 MB/sec execute 197 sec latency 869.181 ms 2024-04-16T17:23:05.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.539+0000 7febb5000640 1 -- 172.21.15.18:0/873607210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 msgr2=0x7febb0076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.539+0000 7febb5000640 1 --2- 172.21.15.18:0/873607210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 0x7febb0076910 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7feba0009930 tx=0x7feba002bc40 comp rx=0 tx=0).stop 2024-04-16T17:23:05.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.539+0000 7febb5000640 1 -- 172.21.15.18:0/873607210 shutdown_connections 2024-04-16T17:23:05.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.539+0000 7febb5000640 1 --2- 172.21.15.18:0/873607210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febb0112e80 0x7febb0115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.539+0000 7febb5000640 1 --2- 172.21.15.18:0/873607210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb0076e50 0x7febb0112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.540+0000 7febb5000640 1 --2- 172.21.15.18:0/873607210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 0x7febb0076910 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.540+0000 7febb5000640 1 -- 172.21.15.18:0/873607210 >> 172.21.15.18:0/873607210 conn(0x7febb00ffc90 msgr2=0x7febb01020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.540+0000 7febb5000640 1 -- 172.21.15.18:0/873607210 shutdown_connections 2024-04-16T17:23:05.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.540+0000 7febb5000640 1 -- 172.21.15.18:0/873607210 wait complete. 2024-04-16T17:23:05.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.542+0000 7febb5000640 1 Processor -- start 2024-04-16T17:23:05.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.542+0000 7febb5000640 1 -- start start 2024-04-16T17:23:05.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.544+0000 7febb5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 0x7febb019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.544+0000 7febb5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb0076e50 0x7febb019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.544+0000 7febade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 0x7febb019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.544+0000 7febade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 0x7febb019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59324/0 (socket says 172.21.15.18:59324) 2024-04-16T17:23:05.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.545+0000 7febb5000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febb0112e80 0x7febb01a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.545+0000 7febb5000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7febb0118ab0 con 0x7febb0112e80 2024-04-16T17:23:05.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.545+0000 7febb5000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7febb0118930 con 0x7febb0076530 2024-04-16T17:23:05.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.545+0000 7febb5000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7febb0118c30 con 0x7febb0076e50 2024-04-16T17:23:05.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.545+0000 7febad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb0076e50 0x7febb019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.545+0000 7febad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb0076e50 0x7febb019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43910/0 (socket says 172.21.15.18:43910) 2024-04-16T17:23:05.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.545+0000 7febad400640 1 -- 172.21.15.18:0/3925387147 learned_addr learned my addr 172.21.15.18:0/3925387147 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.546+0000 7febad400640 1 -- 172.21.15.18:0/3925387147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 msgr2=0x7febb019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.546+0000 7febad400640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 0x7febb019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.546+0000 7febad400640 1 -- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febb0112e80 msgr2=0x7febb01a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.546+0000 7febad400640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febb0112e80 0x7febb01a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.546+0000 7febad400640 1 -- 172.21.15.18:0/3925387147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb01a4020 con 0x7febb0076e50 2024-04-16T17:23:05.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.547+0000 7febade00640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 0x7febb019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-16T17:23:05.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.547+0000 7febad400640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb0076e50 0x7febb019f570 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00b450 tx=0x7feb9c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.548+0000 7feba6a00640 1 -- 172.21.15.18:0/3925387147 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c015020 con 0x7febb0076e50 2024-04-16T17:23:05.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.548+0000 7feba6a00640 1 -- 172.21.15.18:0/3925387147 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c011d70 con 0x7febb0076e50 2024-04-16T17:23:05.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.548+0000 7febb5000640 1 -- 172.21.15.18:0/3925387147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb01a4310 con 0x7febb0076e50 2024-04-16T17:23:05.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.548+0000 7febb5000640 1 -- 172.21.15.18:0/3925387147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb01abba0 con 0x7febb0076e50 2024-04-16T17:23:05.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.548+0000 7feba6a00640 1 -- 172.21.15.18:0/3925387147 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00f340 con 0x7febb0076e50 2024-04-16T17:23:05.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.558+0000 7febb7dc5740 1 Processor -- start 2024-04-16T17:23:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.560+0000 7febb7dc5740 1 -- start start 2024-04-16T17:23:05.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.561+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2e1b0 0x5631c3c2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.561+0000 7feba6a00640 1 -- 172.21.15.18:0/3925387147 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb9c018020 con 0x7febb0076e50 2024-04-16T17:23:05.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.561+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c37170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.561+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c39aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.561+0000 7febb7dc5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5631c3b94120 con 0x5631c3c2e1b0 2024-04-16T17:23:05.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.561+0000 7febb7dc5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5631c3b3a340 con 0x5631c3c376b0 2024-04-16T17:23:05.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.561+0000 7febb7dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5631c3b900e0 con 0x5631c3c2ead0 2024-04-16T17:23:05.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.562+0000 7febade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c37170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.562+0000 7febade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c37170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43916/0 (socket says 172.21.15.18:43916) 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.562+0000 7febade00640 1 -- 172.21.15.18:0/1902721248 learned_addr learned my addr 172.21.15.18:0/1902721248 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.562+0000 7febad400640 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c39aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.563+0000 7febae800640 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2e1b0 0x5631c3c2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.563+0000 7febad400640 1 -- 172.21.15.18:0/1902721248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 msgr2=0x5631c3c37170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.563+0000 7febad400640 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c37170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.563+0000 7febad400640 1 -- 172.21.15.18:0/1902721248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2e1b0 msgr2=0x5631c3c2e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.563+0000 7febad400640 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2e1b0 0x5631c3c2e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.563+0000 7febad400640 1 -- 172.21.15.18:0/1902721248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c39707d0 con 0x5631c3c376b0 2024-04-16T17:23:05.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.563+0000 7febae800640 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2e1b0 0x5631c3c2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:05.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.564+0000 7feba6a00640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7feb940a3de0 0x7feb940a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.564+0000 7febad400640 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c39aa0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0cd940 tx=0x7feb9c0d6c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650bbf2eb83bbed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.564+0000 7feba6a00640 1 -- 172.21.15.18:0/3925387147 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7feb9c0d0540 con 0x7febb0076e50 2024-04-16T17:23:05.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.564+0000 7febae800640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7feb940a3de0 0x7feb940a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.564+0000 7feb9b400640 1 -- 172.21.15.18:0/1902721248 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c098430 con 0x5631c3c376b0 2024-04-16T17:23:05.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.565+0000 7feb9b400640 1 -- 172.21.15.18:0/1902721248 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c09c890 con 0x5631c3c376b0 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 -- 172.21.15.18:0/1902721248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 msgr2=0x5631c3c39aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febae800640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7feb940a3de0 0x7feb940a62a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feba8009270 tx=0x7feba8009480 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7feb9b400640 1 -- 172.21.15.18:0/1902721248 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c098430 con 0x5631c3c376b0 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c39aa0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0cd940 tx=0x7feb9c0d6c90 comp rx=0 tx=0).stop 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 -- 172.21.15.18:0/1902721248 shutdown_connections 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c39aa0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c37170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 --2- 172.21.15.18:0/1902721248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2e1b0 0x5631c3c2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 -- 172.21.15.18:0/1902721248 >> 172.21.15.18:0/1902721248 conn(0x5631c3c288e0 msgr2=0x5631c3c28cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 -- 172.21.15.18:0/1902721248 shutdown_connections 2024-04-16T17:23:05.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.566+0000 7febb7dc5740 1 -- 172.21.15.18:0/1902721248 wait complete. 2024-04-16T17:23:05.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.567+0000 7febb7dc5740 1 -- start start 2024-04-16T17:23:05.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.567+0000 7febb7dc5740 1 Processor -- start 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 0x5631c3c345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2ead0 0x5631c3c34af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c35030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febb7dc5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5631c3c3a4f0 con 0x5631c3c2ead0 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febb7dc5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5631c3b4d710 con 0x5631c3c376b0 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febb7dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5631c3c3a670 con 0x5631c3c2e1b0 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febae800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c35030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febae800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c35030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59338/0 (socket says 172.21.15.18:59338) 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.568+0000 7febae800640 1 -- 172.21.15.18:0/2381009160 learned_addr learned my addr 172.21.15.18:0/2381009160 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febad400640 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2ead0 0x5631c3c34af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febb7dc5740 10 client.0 _ll_register_callbacks cb 0x5631c3b48af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febae800640 1 -- 172.21.15.18:0/2381009160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 msgr2=0x5631c3c345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febae800640 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 0x5631c3c345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febae800640 1 -- 172.21.15.18:0/2381009160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2ead0 msgr2=0x5631c3c34af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febae800640 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2ead0 0x5631c3c34af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febae800640 1 -- 172.21.15.18:0/2381009160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c38a63f0 con 0x5631c3c376b0 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febad400640 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2ead0 0x5631c3c34af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.569+0000 7febb7dc5740 1 -- 172.21.15.18:0/2381009160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9c09c890 con 0x5631c3c376b0 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.570+0000 7febae800640 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c35030 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7feba8011790 tx=0x7feba8009200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=babdfab732392c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.570+0000 7feb8fe00640 10 client.30404 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.570+0000 7feb8fe00640 1 -- 172.21.15.18:0/2381009160 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8022070 con 0x5631c3c376b0 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.570+0000 7feb8fe00640 1 -- 172.21.15.18:0/2381009160 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7feba8003b70 con 0x5631c3c376b0 2024-04-16T17:23:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.571+0000 7feb8fe00640 1 -- 172.21.15.18:0/2381009160 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feba8003d70 con 0x5631c3c376b0 2024-04-16T17:23:05.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.571+0000 7febb7dc5740 10 client.30404 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:05.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.571+0000 7febb7dc5740 1 -- 172.21.15.18:0/2381009160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5631c38660f0 con 0x5631c3c376b0 2024-04-16T17:23:05.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.572+0000 7feb8fe00640 1 -- 172.21.15.18:0/2381009160 <== mon.2 v2:172.21.15.22:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7feba802e020 con 0x5631c3c376b0 2024-04-16T17:23:05.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.572+0000 7febb7dc5740 10 client.30404 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:05.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.572+0000 7febb7dc5740 10 client.30404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:05.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.572+0000 7febb7dc5740 20 client.30404 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:05.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.572+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5631c3c47620 0x5631c3c49ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.572+0000 7febb7dc5740 4 client.30404 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:05.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.572+0000 7febb7dc5740 1 -- 172.21.15.18:0/2381009160 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631c3ba1b40 con 0x5631c3c47620 2024-04-16T17:23:05.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.573+0000 7febade00640 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5631c3c47620 0x5631c3c49ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.574+0000 7febade00640 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5631c3c47620 0x5631c3c49ac0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98ea83ba3572777 server_cookie=7c76cc3744d93675 in_seq=0 out_seq=0 2024-04-16T17:23:05.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.574+0000 7feb8fe00640 10 client.30404 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:05.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.575+0000 7feb8fe00640 1 -- 172.21.15.18:0/2381009160 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631c3ba1b40 con 0x5631c3c47620 2024-04-16T17:23:05.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.575+0000 7feb8fe00640 10 client.30404 handle_command_reply: tid=0 2024-04-16T17:23:05.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.575+0000 7febb7dc5740 1 client.30404 shutdown 2024-04-16T17:23:05.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7febb7dc5740 1 -- 172.21.15.18:0/2381009160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 msgr2=0x5631c3c35030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c35030 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7feba8011790 tx=0x7feba8009200 comp rx=0 tx=0).stop 2024-04-16T17:23:05.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7febb7dc5740 1 -- 172.21.15.18:0/2381009160 shutdown_connections 2024-04-16T17:23:05.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5631c3c47620 0x5631c3c49ac0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c376b0 0x5631c3c35030 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7feb8fe00640 0 client.30404 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:05.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c2ead0 0x5631c3c34af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2381009160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 0x5631c3c345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.576+0000 7febb7dc5740 1 -- 172.21.15.18:0/2381009160 >> 172.21.15.18:0/2381009160 conn(0x5631c3c288e0 msgr2=0x5631c3c2a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.577+0000 7febb7dc5740 1 -- 172.21.15.18:0/2381009160 shutdown_connections 2024-04-16T17:23:05.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.580+0000 7febb7dc5740 1 -- 172.21.15.18:0/2381009160 wait complete. 2024-04-16T17:23:05.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.581+0000 7febb7dc5740 20 client.30404 trim_cache size 0 max 16384 2024-04-16T17:23:05.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.618+0000 7febb7dc5740 1 Processor -- start 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.618+0000 7febb7dc5740 1 -- start start 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.618+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2e1b0 0x5631c3c36ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.618+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c43460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.618+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c33710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.618+0000 7febb7dc5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feb90012aa0 con 0x5631c3c376b0 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.618+0000 7febb7dc5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb90010370 con 0x5631c3c2e1b0 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.618+0000 7febb7dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feb9000e1e0 con 0x5631c3c2ead0 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.619+0000 7febade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2e1b0 0x5631c3c36ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.619+0000 7febad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c43460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.619+0000 7febade00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2e1b0 0x5631c3c36ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59346/0 (socket says 172.21.15.18:59346) 2024-04-16T17:23:05.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.619+0000 7febade00640 1 -- 172.21.15.18:0/4203685268 learned_addr learned my addr 172.21.15.18:0/4203685268 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.619+0000 7febae800640 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c33710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.620+0000 7febade00640 1 -- 172.21.15.18:0/4203685268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 msgr2=0x5631c3c43460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.622+0000 7febade00640 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c43460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.623+0000 7febade00640 1 -- 172.21.15.18:0/4203685268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 msgr2=0x5631c3c33710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.623+0000 7febade00640 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c33710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.623+0000 7febade00640 1 -- 172.21.15.18:0/4203685268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c3c22d20 con 0x5631c3c2e1b0 2024-04-16T17:23:05.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.623+0000 7febae800640 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c33710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:23:05.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.623+0000 7febad400640 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c43460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:05.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.623+0000 7febade00640 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2e1b0 0x5631c3c36ce0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7feba0009a20 tx=0x7feba00041c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e8cc350bcb739a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.624+0000 7feb9aa00640 1 -- 172.21.15.18:0/4203685268 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba002c890 con 0x5631c3c2e1b0 2024-04-16T17:23:05.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.625+0000 7feb9aa00640 1 -- 172.21.15.18:0/4203685268 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba0005500 con 0x5631c3c2e1b0 2024-04-16T17:23:05.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.625+0000 7feb9aa00640 1 -- 172.21.15.18:0/4203685268 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0004b30 con 0x5631c3c2e1b0 2024-04-16T17:23:05.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.625+0000 7febb7dc5740 1 -- 172.21.15.18:0/4203685268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2e1b0 msgr2=0x5631c3c36ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.626+0000 7febb7dc5740 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2e1b0 0x5631c3c36ce0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7feba0009a20 tx=0x7feba00041c0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.628+0000 7febb7dc5740 1 -- 172.21.15.18:0/4203685268 shutdown_connections 2024-04-16T17:23:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.628+0000 7febb7dc5740 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c33710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.628+0000 7febb7dc5740 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2ead0 0x5631c3c43460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.628+0000 7febb7dc5740 1 --2- 172.21.15.18:0/4203685268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2e1b0 0x5631c3c36ce0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.628+0000 7febb7dc5740 1 -- 172.21.15.18:0/4203685268 >> 172.21.15.18:0/4203685268 conn(0x5631c3c288e0 msgr2=0x5631c3c27040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.629+0000 7febb7dc5740 1 -- 172.21.15.18:0/4203685268 shutdown_connections 2024-04-16T17:23:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.629+0000 7febb7dc5740 1 -- 172.21.15.18:0/4203685268 wait complete. 2024-04-16T17:23:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.629+0000 7febb7dc5740 1 -- start start 2024-04-16T17:23:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.630+0000 7febb7dc5740 1 Processor -- start 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.630+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 0x5631c3ca94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.630+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2ead0 0x5631c3c3f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.630+0000 7febb7dc5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c3feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febad400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2ead0 0x5631c3c3f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febad400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2ead0 0x5631c3c3f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59358/0 (socket says 172.21.15.18:59358) 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 0x5631c3ca94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 0x5631c3ca94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43960/0 (socket says 172.21.15.18:43960) 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febb7dc5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feb9000e830 con 0x5631c3c376b0 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febb7dc5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb9000fca0 con 0x5631c3c2ead0 2024-04-16T17:23:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febb7dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feb9000ef00 con 0x5631c3c2e1b0 2024-04-16T17:23:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febad400640 1 -- 172.21.15.18:0/2146570476 learned_addr learned my addr 172.21.15.18:0/2146570476 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febb7dc5740 10 client.0 _ll_register_callbacks cb 0x7feba8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.631+0000 7febae800640 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c3feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.632+0000 7febad400640 1 -- 172.21.15.18:0/2146570476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 msgr2=0x5631c3ca94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.632+0000 7febad400640 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 0x5631c3ca94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.632+0000 7febad400640 1 -- 172.21.15.18:0/2146570476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 msgr2=0x5631c3c3feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:05.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.632+0000 7febad400640 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c3feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.632+0000 7febad400640 1 -- 172.21.15.18:0/2146570476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c3c31790 con 0x5631c3c2ead0 2024-04-16T17:23:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.633+0000 7febad400640 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2ead0 0x5631c3c3f970 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0cd9f0 tx=0x7feb9c098a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecce1ef36d6f51ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.633+0000 7feb8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.633+0000 7feb8fe00640 1 -- 172.21.15.18:0/2146570476 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0b5060 con 0x5631c3c2ead0 2024-04-16T17:23:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.634+0000 7feb8fe00640 1 -- 172.21.15.18:0/2146570476 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7feb9c0a0120 con 0x5631c3c2ead0 2024-04-16T17:23:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.634+0000 7febb7dc5740 1 -- 172.21.15.18:0/2146570476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba0005500 con 0x5631c3c2ead0 2024-04-16T17:23:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.634+0000 7feb8fe00640 1 -- 172.21.15.18:0/2146570476 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feb9c09baf0 con 0x5631c3c2ead0 2024-04-16T17:23:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.634+0000 7febb7dc5740 10 client.30410 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.634+0000 7febb7dc5740 1 -- 172.21.15.18:0/2146570476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5631c3cbf0c0 con 0x5631c3c2ead0 2024-04-16T17:23:05.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.634+0000 7feb8fe00640 1 -- 172.21.15.18:0/2146570476 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba0005500 con 0x5631c3c2ead0 2024-04-16T17:23:05.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.635+0000 7feb8fe00640 1 -- 172.21.15.18:0/2146570476 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7feb9c0103e0 con 0x5631c3c2ead0 2024-04-16T17:23:05.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.639+0000 7febb7dc5740 10 client.30410 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:05.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.639+0000 7febb7dc5740 10 client.30410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:05.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.639+0000 7febb7dc5740 20 client.30410 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:05.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.639+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5631c3cbf3b0 0x5631c3cc17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:05.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.639+0000 7febb7dc5740 4 client.30410 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:23:05.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.639+0000 7febb7dc5740 1 -- 172.21.15.18:0/2146570476 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631c3b9b9b0 con 0x5631c3cbf3b0 2024-04-16T17:23:05.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.640+0000 7febade00640 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5631c3cbf3b0 0x5631c3cc17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:05.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.642+0000 7feb8fe00640 10 client.30410 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:05.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.642+0000 7febade00640 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5631c3cbf3b0 0x5631c3cc17a0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642344a27e52ad6f server_cookie=8881760834d48de0 in_seq=0 out_seq=0 2024-04-16T17:23:05.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7feb8fe00640 1 -- 172.21.15.18:0/2146570476 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x5631c3b9b9b0 con 0x5631c3cbf3b0 2024-04-16T17:23:05.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7feb8fe00640 10 client.30410 handle_command_reply: tid=0 2024-04-16T17:23:05.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 client.30410 shutdown 2024-04-16T17:23:05.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 -- 172.21.15.18:0/2146570476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2ead0 msgr2=0x5631c3c3f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2ead0 0x5631c3c3f970 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0cd9f0 tx=0x7feb9c098a10 comp rx=0 tx=0).stop 2024-04-16T17:23:05.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 -- 172.21.15.18:0/2146570476 shutdown_connections 2024-04-16T17:23:05.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5631c3cbf3b0 0x5631c3cc17a0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5631c3c376b0 0x5631c3c3feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631c3c2ead0 0x5631c3c3f970 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 --2- 172.21.15.18:0/2146570476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c3c2e1b0 0x5631c3ca94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7febb7dc5740 1 -- 172.21.15.18:0/2146570476 >> 172.21.15.18:0/2146570476 conn(0x5631c3c288e0 msgr2=0x5631c3c29d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.644+0000 7feb8fe00640 0 client.30410 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:05.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.648+0000 7febb7dc5740 1 -- 172.21.15.18:0/2146570476 shutdown_connections 2024-04-16T17:23:05.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.648+0000 7febb7dc5740 1 -- 172.21.15.18:0/2146570476 wait complete. 2024-04-16T17:23:05.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.648+0000 7febb7dc5740 20 client.30410 trim_cache size 0 max 16384 2024-04-16T17:23:05.648 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (87 inodes in the stack)", 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "6879e586-846e-4af6-b8e0-3f57c43c50fa": { 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "6879e586-846e-4af6-b8e0-3f57c43c50fa", 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:23:05.649 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:23:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.649+0000 7febb5000640 1 -- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7feb940a3de0 msgr2=0x7feb940a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.649+0000 7febb5000640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7feb940a3de0 0x7feb940a62a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feba8009270 tx=0x7feba8009480 comp rx=0 tx=0).stop 2024-04-16T17:23:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.649+0000 7febb5000640 1 -- 172.21.15.18:0/3925387147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb0076e50 msgr2=0x7febb019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.649+0000 7febb5000640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb0076e50 0x7febb019f570 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00b450 tx=0x7feb9c00b910 comp rx=0 tx=0).stop 2024-04-16T17:23:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.650+0000 7febb5000640 1 -- 172.21.15.18:0/3925387147 shutdown_connections 2024-04-16T17:23:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.650+0000 7febb5000640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7feb940a3de0 0x7feb940a62a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.650+0000 7febb5000640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febb0112e80 0x7febb01a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.650+0000 7febb5000640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb0076e50 0x7febb019f570 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.650+0000 7febb5000640 1 --2- 172.21.15.18:0/3925387147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febb0076530 0x7febb019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:05.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.650+0000 7febb5000640 1 -- 172.21.15.18:0/3925387147 >> 172.21.15.18:0/3925387147 conn(0x7febb00ffc90 msgr2=0x7febb0114540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:05.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.650+0000 7febb5000640 1 -- 172.21.15.18:0/3925387147 shutdown_connections 2024-04-16T17:23:05.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:05.650+0000 7febb5000640 1 -- 172.21.15.18:0/3925387147 wait complete. 2024-04-16T17:23:05.682 INFO:tasks.cephfs.filesystem:scrub status for tag:6879e586-846e-4af6-b8e0-3f57c43c50fa - {'path': '/', 'tag': '6879e586-846e-4af6-b8e0-3f57c43c50fa', 'options': 'recursive,force'} 2024-04-16T17:23:06.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:05 smithi022 ceph-mon[36023]: pgmap v585: 129 pgs: 129 active+clean; 422 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 439 op/s 2024-04-16T17:23:06.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:05 smithi022 ceph-mon[36023]: osdmap e127: 12 total, 12 up, 12 in 2024-04-16T17:23:06.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:05 smithi043 ceph-mon[36901]: pgmap v585: 129 pgs: 129 active+clean; 422 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 439 op/s 2024-04-16T17:23:06.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:05 smithi043 ceph-mon[36901]: osdmap e127: 12 total, 12 up, 12 in 2024-04-16T17:23:06.299 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:05 smithi018 ceph-mon[31776]: pgmap v585: 129 pgs: 129 active+clean; 422 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 439 op/s 2024-04-16T17:23:06.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:05 smithi018 ceph-mon[31776]: osdmap e127: 12 total, 12 up, 12 in 2024-04-16T17:23:06.530 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77386 12.33 MB/sec execute 198 sec latency 1838.867 ms 2024-04-16T17:23:07.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:06 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:23:07.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:06 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:06 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:23:07.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:06 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:23:07.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:06 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:06 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:23:07.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:06 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:06 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:23:07.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:06 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:06 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:23:07.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:06 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:23:07.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:06 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:06 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:23:07.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:06 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:06 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:23:07.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:06 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:06 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:23:07.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:06 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:23:07.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:06 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:06 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:23:07.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:06 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:07.530 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77442 12.28 MB/sec execute 199 sec latency 2540.125 ms 2024-04-16T17:23:08.084 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:07 smithi018 ceph-mon[31776]: pgmap v587: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 428 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 577 op/s 2024-04-16T17:23:08.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:07 smithi043 ceph-mon[36901]: pgmap v587: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 428 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 577 op/s 2024-04-16T17:23:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:07 smithi022 ceph-mon[36023]: pgmap v587: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 428 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 577 op/s 2024-04-16T17:23:08.530 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77506 12.24 MB/sec execute 200 sec latency 1833.926 ms 2024-04-16T17:23:09.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:08 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:09.323 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:08 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:09.531 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77583 12.19 MB/sec execute 201 sec latency 2152.450 ms 2024-04-16T17:23:10.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:09 smithi043 ceph-mon[36901]: pgmap v588: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 429 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 554 op/s 2024-04-16T17:23:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:09 smithi022 ceph-mon[36023]: pgmap v588: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 429 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 554 op/s 2024-04-16T17:23:10.532 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77712 12.15 MB/sec execute 202 sec latency 2292.264 ms 2024-04-16T17:23:10.585 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:08 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.125+0000 7f3245600640 1 -- 172.21.15.18:0/3931365360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3240101020 msgr2=0x7f324010c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.125+0000 7f3245600640 1 --2- 172.21.15.18:0/3931365360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3240101020 0x7f324010c0a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f322c009900 tx=0x7f322c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:23:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.125+0000 7f3245600640 1 -- 172.21.15.18:0/3931365360 shutdown_connections 2024-04-16T17:23:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.125+0000 7f3245600640 1 --2- 172.21.15.18:0/3931365360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f324010c670 0x7f324010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.125+0000 7f3245600640 1 --2- 172.21.15.18:0/3931365360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3240101020 0x7f324010c0a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.125+0000 7f3245600640 1 --2- 172.21.15.18:0/3931365360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3240100700 0x7f3240100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.125+0000 7f3245600640 1 -- 172.21.15.18:0/3931365360 >> 172.21.15.18:0/3931365360 conn(0x7f3240075e10 msgr2=0x7f3240076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.126+0000 7f3245600640 1 -- 172.21.15.18:0/3931365360 shutdown_connections 2024-04-16T17:23:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.126+0000 7f3245600640 1 -- 172.21.15.18:0/3931365360 wait complete. 2024-04-16T17:23:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.127+0000 7f3245600640 1 Processor -- start 2024-04-16T17:23:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.127+0000 7f3245600640 1 -- start start 2024-04-16T17:23:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f3245600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100700 0x7f32401a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f3245600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3240101020 0x7f32401a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f3245600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324010c670 0x7f32401abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f3245600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3240110210 con 0x7f3240100700 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f3245600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3240110090 con 0x7f324010c670 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f3245600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3240110390 con 0x7f3240101020 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f323f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324010c670 0x7f32401abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f323e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3240101020 0x7f32401a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f323f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324010c670 0x7f32401abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54072/0 (socket says 172.21.15.18:54072) 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.128+0000 7f323f400640 1 -- 172.21.15.18:0/1045333856 learned_addr learned my addr 172.21.15.18:0/1045333856 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.129+0000 7f323e000640 1 -- 172.21.15.18:0/1045333856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324010c670 msgr2=0x7f32401abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.129+0000 7f323e000640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324010c670 0x7f32401abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.129+0000 7f323e000640 1 -- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100700 msgr2=0x7f32401a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.129+0000 7f323e000640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100700 0x7f32401a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.129+0000 7f323e000640 1 -- 172.21.15.18:0/1045333856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32401ac660 con 0x7f3240101020 2024-04-16T17:23:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.129+0000 7f323f400640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324010c670 0x7f32401abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.130+0000 7f323e000640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3240101020 0x7f32401a7c00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f322c02c170 tx=0x7f322c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.130+0000 7f3237400640 1 -- 172.21.15.18:0/1045333856 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c043020 con 0x7f3240101020 2024-04-16T17:23:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.130+0000 7f3237400640 1 -- 172.21.15.18:0/1045333856 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f322c0054e0 con 0x7f3240101020 2024-04-16T17:23:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.131+0000 7f3245600640 1 -- 172.21.15.18:0/1045333856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32401ac8f0 con 0x7f3240101020 2024-04-16T17:23:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.131+0000 7f3237400640 1 -- 172.21.15.18:0/1045333856 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c003b50 con 0x7f3240101020 2024-04-16T17:23:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.131+0000 7f3245600640 1 -- 172.21.15.18:0/1045333856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32401b41e0 con 0x7f3240101020 2024-04-16T17:23:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.135+0000 7f3248208740 1 Processor -- start 2024-04-16T17:23:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.136+0000 7f3248208740 1 -- start start 2024-04-16T17:23:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f3248208740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f412d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f3248208740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f412dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f3248208740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f4136e20 0x5575f4139210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f3248208740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5575f4092780 con 0x5575f4136e20 2024-04-16T17:23:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f3248208740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5575f4038f40 con 0x5575f412d8a0 2024-04-16T17:23:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f3248208740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5575f408e740 con 0x5575f412e610 2024-04-16T17:23:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f323f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f412d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f323f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f412d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43842/0 (socket says 172.21.15.18:43842) 2024-04-16T17:23:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.137+0000 7f323f400640 1 -- 172.21.15.18:0/23155094 learned_addr learned my addr 172.21.15.18:0/23155094 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.138+0000 7f323ea00640 1 --2- 172.21.15.18:0/23155094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f412dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.138+0000 7f323f400640 1 -- 172.21.15.18:0/23155094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 msgr2=0x5575f412dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.138+0000 7f323f400640 1 --2- 172.21.15.18:0/23155094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f412dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.138+0000 7f323f400640 1 -- 172.21.15.18:0/23155094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f4136e20 msgr2=0x5575f4139210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.138+0000 7f323f400640 1 --2- 172.21.15.18:0/23155094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f4136e20 0x5575f4139210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.138+0000 7f323f400640 1 -- 172.21.15.18:0/23155094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f3da5210 con 0x5575f412e610 2024-04-16T17:23:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.139+0000 7f3237400640 1 -- 172.21.15.18:0/1045333856 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f322c046020 con 0x7f3240101020 2024-04-16T17:23:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.141+0000 7f3237400640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f32240a3d10 0x7f32240a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.141+0000 7f3237400640 1 -- 172.21.15.18:0/1045333856 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f322c0f2cc0 con 0x7f3240101020 2024-04-16T17:23:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.142+0000 7f323f400640 1 --2- 172.21.15.18:0/23155094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f412d360 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f323800f790 tx=0x7f323800fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=562752d706c3561e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.143+0000 7f323ea00640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f32240a3d10 0x7f32240a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.143+0000 7f322b400640 1 -- 172.21.15.18:0/23155094 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323800db30 con 0x5575f412e610 2024-04-16T17:23:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.143+0000 7f322b400640 1 -- 172.21.15.18:0/23155094 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3238003e50 con 0x5575f412e610 2024-04-16T17:23:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.143+0000 7f322b400640 1 -- 172.21.15.18:0/23155094 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238010430 con 0x5575f412e610 2024-04-16T17:23:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.144+0000 7f3248208740 1 -- 172.21.15.18:0/23155094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 msgr2=0x5575f412d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.144+0000 7f323ea00640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f32240a3d10 0x7f32240a61d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5575f412ed70 tx=0x7f3230009530 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.144+0000 7f3248208740 1 --2- 172.21.15.18:0/23155094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f412d360 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f323800f790 tx=0x7f323800fc50 comp rx=0 tx=0).stop 2024-04-16T17:23:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.149+0000 7f3248208740 1 -- 172.21.15.18:0/23155094 shutdown_connections 2024-04-16T17:23:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.149+0000 7f3248208740 1 --2- 172.21.15.18:0/23155094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f4136e20 0x5575f4139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.149+0000 7f3248208740 1 --2- 172.21.15.18:0/23155094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f412dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.149+0000 7f3248208740 1 --2- 172.21.15.18:0/23155094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f412d360 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.149+0000 7f3248208740 1 -- 172.21.15.18:0/23155094 >> 172.21.15.18:0/23155094 conn(0x5575f40c85d0 msgr2=0x5575f4128590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.149+0000 7f3248208740 1 -- 172.21.15.18:0/23155094 shutdown_connections 2024-04-16T17:23:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.150+0000 7f3248208740 1 -- 172.21.15.18:0/23155094 wait complete. 2024-04-16T17:23:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.150+0000 7f3248208740 1 -- start start 2024-04-16T17:23:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.151+0000 7f3248208740 1 Processor -- start 2024-04-16T17:23:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.152+0000 7f3248208740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f412d8a0 0x5575f41368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.152+0000 7f3248208740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412e610 0x5575f4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.152+0000 7f3248208740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f4136e20 0x5575f41343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.152+0000 7f3248208740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5575f4139b50 con 0x5575f412d8a0 2024-04-16T17:23:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.152+0000 7f3248208740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5575f404c390 con 0x5575f412e610 2024-04-16T17:23:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.152+0000 7f3248208740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5575f4139cd0 con 0x5575f4136e20 2024-04-16T17:23:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.153+0000 7f3248208740 10 client.0 _ll_register_callbacks cb 0x5575f4035f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.153+0000 7f323f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f4136e20 0x5575f41343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.153+0000 7f323f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f4136e20 0x5575f41343b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43854/0 (socket says 172.21.15.18:43854) 2024-04-16T17:23:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.153+0000 7f323f400640 1 -- 172.21.15.18:0/3712342849 learned_addr learned my addr 172.21.15.18:0/3712342849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.154+0000 7f323f400640 1 -- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412e610 msgr2=0x5575f4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.153+0000 7f323e000640 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412e610 0x5575f4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.154+0000 7f323f400640 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412e610 0x5575f4133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.154+0000 7f323f400640 1 -- 172.21.15.18:0/3712342849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f412d8a0 msgr2=0x5575f41368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.155+0000 7f323f400640 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f412d8a0 0x5575f41368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.155+0000 7f323f400640 1 -- 172.21.15.18:0/3712342849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f3e71620 con 0x5575f4136e20 2024-04-16T17:23:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.155+0000 7f323e000640 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412e610 0x5575f4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.155+0000 7f323f400640 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f4136e20 0x5575f41343b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3238000f80 tx=0x7f323800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38bac0c5f6ca4e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.155+0000 7f321fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.157+0000 7f321fe00640 1 -- 172.21.15.18:0/3712342849 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238003970 con 0x5575f4136e20 2024-04-16T17:23:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.157+0000 7f321fe00640 1 -- 172.21.15.18:0/3712342849 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f3238003b10 con 0x5575f4136e20 2024-04-16T17:23:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.157+0000 7f3248208740 1 -- 172.21.15.18:0/3712342849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3238003e50 con 0x5575f4136e20 2024-04-16T17:23:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.158+0000 7f321fe00640 1 -- 172.21.15.18:0/3712342849 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f323800db30 con 0x5575f4136e20 2024-04-16T17:23:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.159+0000 7f3248208740 10 client.30280 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.159+0000 7f3248208740 1 -- 172.21.15.18:0/3712342849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5575f4145c40 con 0x5575f4136e20 2024-04-16T17:23:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.160+0000 7f321fe00640 1 -- 172.21.15.18:0/3712342849 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3238003e50 con 0x5575f4136e20 2024-04-16T17:23:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.160+0000 7f321fe00640 1 -- 172.21.15.18:0/3712342849 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f3238005470 con 0x5575f4136e20 2024-04-16T17:23:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.160+0000 7f3248208740 10 client.30280 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.160+0000 7f3248208740 10 client.30280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.160+0000 7f3248208740 20 client.30280 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.160+0000 7f3248208740 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5575f41473f0 0x5575f4149820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.160+0000 7f3248208740 4 client.30280 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.160+0000 7f3248208740 1 -- 172.21.15.18:0/3712342849 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575f40a01a0 con 0x5575f41473f0 2024-04-16T17:23:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.161+0000 7f323ea00640 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5575f41473f0 0x5575f4149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.165+0000 7f321fe00640 10 client.30280 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.165+0000 7f323ea00640 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5575f41473f0 0x5575f4149820 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0715293de060ae6 server_cookie=4a6a43b2635d3c07 in_seq=0 out_seq=0 2024-04-16T17:23:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.168+0000 7f321fe00640 1 -- 172.21.15.18:0/3712342849 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575f40a01a0 con 0x5575f41473f0 2024-04-16T17:23:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.168+0000 7f321fe00640 10 client.30280 handle_command_reply: tid=0 2024-04-16T17:23:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.168+0000 7f3248208740 1 client.30280 shutdown 2024-04-16T17:23:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.168+0000 7f3248208740 1 -- 172.21.15.18:0/3712342849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f4136e20 msgr2=0x5575f41343b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.168+0000 7f3248208740 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f4136e20 0x5575f41343b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3238000f80 tx=0x7f323800f3c0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f3248208740 1 -- 172.21.15.18:0/3712342849 shutdown_connections 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f3248208740 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5575f41473f0 0x5575f4149820 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f3248208740 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f4136e20 0x5575f41343b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f3248208740 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412e610 0x5575f4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f3248208740 1 --2- 172.21.15.18:0/3712342849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f412d8a0 0x5575f41368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f3248208740 1 -- 172.21.15.18:0/3712342849 >> 172.21.15.18:0/3712342849 conn(0x5575f40c85d0 msgr2=0x5575f412a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f321fe00640 0 client.30280 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f3248208740 1 -- 172.21.15.18:0/3712342849 shutdown_connections 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.169+0000 7f3248208740 1 -- 172.21.15.18:0/3712342849 wait complete. 2024-04-16T17:23:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.170+0000 7f3248208740 20 client.30280 trim_cache size 0 max 16384 2024-04-16T17:23:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.214+0000 7f3248208740 1 Processor -- start 2024-04-16T17:23:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.214+0000 7f3248208740 1 -- start start 2024-04-16T17:23:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.215+0000 7f3248208740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f4142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.215+0000 7f3248208740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f41363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.215+0000 7f3248208740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f4136e20 0x5575f41368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.215+0000 7f3248208740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32401c0fb0 con 0x5575f4136e20 2024-04-16T17:23:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.215+0000 7f3248208740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32401be880 con 0x5575f412d8a0 2024-04-16T17:23:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.215+0000 7f3248208740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32401bc6b0 con 0x5575f412e610 2024-04-16T17:23:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f4142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f4142a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54098/0 (socket says 172.21.15.18:54098) 2024-04-16T17:23:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323ea00640 1 -- 172.21.15.18:0/2636756292 learned_addr learned my addr 172.21.15.18:0/2636756292 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323e000640 1 --2- 172.21.15.18:0/2636756292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f41363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323ea00640 1 -- 172.21.15.18:0/2636756292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 msgr2=0x5575f41363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323ea00640 1 --2- 172.21.15.18:0/2636756292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f41363a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323ea00640 1 -- 172.21.15.18:0/2636756292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f4136e20 msgr2=0x5575f41368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323ea00640 1 --2- 172.21.15.18:0/2636756292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f4136e20 0x5575f41368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.216+0000 7f323ea00640 1 -- 172.21.15.18:0/2636756292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f412cb70 con 0x5575f412d8a0 2024-04-16T17:23:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.217+0000 7f323ea00640 1 --2- 172.21.15.18:0/2636756292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f4142a50 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3230009a60 tx=0x7f3230014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8beaf363af5481 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.218+0000 7f322aa00640 1 -- 172.21.15.18:0/2636756292 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230012950 con 0x5575f412d8a0 2024-04-16T17:23:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.218+0000 7f322aa00640 1 -- 172.21.15.18:0/2636756292 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3230005020 con 0x5575f412d8a0 2024-04-16T17:23:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.218+0000 7f322aa00640 1 -- 172.21.15.18:0/2636756292 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230027610 con 0x5575f412d8a0 2024-04-16T17:23:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.219+0000 7f3248208740 1 -- 172.21.15.18:0/2636756292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 msgr2=0x5575f4142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.219+0000 7f3248208740 1 --2- 172.21.15.18:0/2636756292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f4142a50 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3230009a60 tx=0x7f3230014000 comp rx=0 tx=0).stop 2024-04-16T17:23:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.219+0000 7f3248208740 1 -- 172.21.15.18:0/2636756292 shutdown_connections 2024-04-16T17:23:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.219+0000 7f3248208740 1 --2- 172.21.15.18:0/2636756292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f4136e20 0x5575f41368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.219+0000 7f3248208740 1 --2- 172.21.15.18:0/2636756292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412e610 0x5575f41363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.219+0000 7f3248208740 1 --2- 172.21.15.18:0/2636756292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f412d8a0 0x5575f4142a50 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.219+0000 7f3248208740 1 -- 172.21.15.18:0/2636756292 >> 172.21.15.18:0/2636756292 conn(0x5575f40c85d0 msgr2=0x5575f41280c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.220+0000 7f3248208740 1 -- 172.21.15.18:0/2636756292 shutdown_connections 2024-04-16T17:23:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.220+0000 7f3248208740 1 -- 172.21.15.18:0/2636756292 wait complete. 2024-04-16T17:23:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.220+0000 7f3248208740 1 -- start start 2024-04-16T17:23:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.220+0000 7f3248208740 1 Processor -- start 2024-04-16T17:23:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.221+0000 7f3248208740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412d8a0 0x5575f413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.221+0000 7f3248208740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f412e610 0x5575f4131290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.221+0000 7f3248208740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f4136e20 0x5575f41318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.222+0000 7f323ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412d8a0 0x5575f413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.222+0000 7f323ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412d8a0 0x5575f413f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43872/0 (socket says 172.21.15.18:43872) 2024-04-16T17:23:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f323ea00640 1 -- 172.21.15.18:0/756903201 learned_addr learned my addr 172.21.15.18:0/756903201 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32401bcd40 con 0x5575f412e610 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32401be1b0 con 0x5575f4136e20 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32401bd410 con 0x5575f412d8a0 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f3248208740 10 client.0 _ll_register_callbacks cb 0x7f323800f5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f323f400640 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f4136e20 0x5575f41318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f323f400640 1 -- 172.21.15.18:0/756903201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412d8a0 msgr2=0x5575f413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f323f400640 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412d8a0 0x5575f413f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f323f400640 1 -- 172.21.15.18:0/756903201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f412e610 msgr2=0x5575f4131290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f323f400640 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f412e610 0x5575f4131290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.223+0000 7f323f400640 1 -- 172.21.15.18:0/756903201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f4133a40 con 0x5575f4136e20 2024-04-16T17:23:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.224+0000 7f323ea00640 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412d8a0 0x5575f413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.224+0000 7f321fe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.224+0000 7f323f400640 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f4136e20 0x5575f41318a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f32380283d0 tx=0x7f323800fec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125543786e2c7eca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.225+0000 7f321fe00640 1 -- 172.21.15.18:0/756903201 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323800db30 con 0x5575f4136e20 2024-04-16T17:23:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.225+0000 7f321fe00640 1 -- 172.21.15.18:0/756903201 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f3238007c10 con 0x5575f4136e20 2024-04-16T17:23:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.226+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3230005020 con 0x5575f4136e20 2024-04-16T17:23:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.226+0000 7f321fe00640 1 -- 172.21.15.18:0/756903201 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3238029350 con 0x5575f4136e20 2024-04-16T17:23:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.226+0000 7f3248208740 10 client.30425 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.226+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5575f41acc70 con 0x5575f4136e20 2024-04-16T17:23:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.227+0000 7f321fe00640 1 -- 172.21.15.18:0/756903201 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3230005020 con 0x5575f4136e20 2024-04-16T17:23:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.227+0000 7f321fe00640 1 -- 172.21.15.18:0/756903201 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f3238005470 con 0x5575f4136e20 2024-04-16T17:23:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.228+0000 7f3248208740 10 client.30425 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.228+0000 7f3248208740 10 client.30425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.228+0000 7f3248208740 20 client.30425 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.228+0000 7f3248208740 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5575f41ad150 0x5575f41af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.228+0000 7f3248208740 4 client.30425 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:23:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.228+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575f409a010 con 0x5575f41ad150 2024-04-16T17:23:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.228+0000 7f323ea00640 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5575f41ad150 0x5575f41af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.229+0000 7f323ea00640 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5575f41ad150 0x5575f41af540 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=839de4015752bdd6 server_cookie=7d2a5318d6c5e7c in_seq=0 out_seq=0 2024-04-16T17:23:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:11.229+0000 7f321fe00640 10 client.30425 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:11.534 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77836 12.15 MB/sec execute 203 sec latency 1925.597 ms 2024-04-16T17:23:11.596 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:09 smithi018 ceph-mon[31776]: pgmap v588: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 429 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 554 op/s 2024-04-16T17:23:12.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:11 smithi018 ceph-mon[31776]: pgmap v589: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 425 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 446 op/s 2024-04-16T17:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.168+0000 7f321fe00640 1 -- 172.21.15.18:0/756903201 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575f409a010 con 0x5575f41ad150 2024-04-16T17:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.168+0000 7f321fe00640 10 client.30425 handle_command_reply: tid=0 2024-04-16T17:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.168+0000 7f3248208740 1 client.30425 shutdown 2024-04-16T17:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.168+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f4136e20 msgr2=0x5575f41318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.168+0000 7f3248208740 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f4136e20 0x5575f41318a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f32380283d0 tx=0x7f323800fec0 comp rx=0 tx=0).stop 2024-04-16T17:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.168+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 shutdown_connections 2024-04-16T17:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.169+0000 7f3248208740 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5575f41ad150 0x5575f41af540 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.169+0000 7f3248208740 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5575f4136e20 0x5575f41318a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.169+0000 7f3248208740 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5575f412e610 0x5575f4131290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.169+0000 7f321fe00640 0 client.30425 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.169+0000 7f3248208740 1 --2- 172.21.15.18:0/756903201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5575f412d8a0 0x5575f413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.169+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 >> 172.21.15.18:0/756903201 conn(0x5575f40c85d0 msgr2=0x5575f41324b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.169+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 shutdown_connections 2024-04-16T17:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.170+0000 7f3248208740 1 -- 172.21.15.18:0/756903201 wait complete. 2024-04-16T17:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.170+0000 7f3248208740 20 client.30425 trim_cache size 0 max 16384 2024-04-16T17:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:23:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:23:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:23:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 -- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f32240a3d10 msgr2=0x7f32240a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f32240a3d10 0x7f32240a61d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5575f412ed70 tx=0x7f3230009530 comp rx=0 tx=0).stop 2024-04-16T17:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 -- 172.21.15.18:0/1045333856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3240101020 msgr2=0x7f32401a7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3240101020 0x7f32401a7c00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f322c02c170 tx=0x7f322c004360 comp rx=0 tx=0).stop 2024-04-16T17:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 -- 172.21.15.18:0/1045333856 shutdown_connections 2024-04-16T17:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f32240a3d10 0x7f32240a61d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324010c670 0x7f32401abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3240101020 0x7f32401a7c00 secure :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f322c02c170 tx=0x7f322c004360 comp rx=0 tx=0).stop 2024-04-16T17:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 --2- 172.21.15.18:0/1045333856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3240100700 0x7f32401a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 -- 172.21.15.18:0/1045333856 >> 172.21.15.18:0/1045333856 conn(0x7f3240075e10 msgr2=0x7f32400fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 -- 172.21.15.18:0/1045333856 shutdown_connections 2024-04-16T17:23:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:12.175+0000 7f3245600640 1 -- 172.21.15.18:0/1045333856 wait complete. 2024-04-16T17:23:12.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:11 smithi043 ceph-mon[36901]: pgmap v589: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 425 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 446 op/s 2024-04-16T17:23:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:11 smithi022 ceph-mon[36023]: pgmap v589: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 425 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 446 op/s 2024-04-16T17:23:12.535 INFO:tasks.workunit.client.0.smithi018.stdout: 10 77964 12.13 MB/sec execute 204 sec latency 2927.543 ms 2024-04-16T17:23:13.535 INFO:tasks.workunit.client.0.smithi018.stdout: 10 78098 12.11 MB/sec execute 205 sec latency 3581.047 ms 2024-04-16T17:23:14.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:13 smithi043 ceph-mon[36901]: pgmap v590: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 425 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-04-16T17:23:14.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:13 smithi022 ceph-mon[36023]: pgmap v590: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 425 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-04-16T17:23:14.535 INFO:tasks.workunit.client.0.smithi018.stdout: 10 78187 12.08 MB/sec execute 206 sec latency 1879.937 ms 2024-04-16T17:23:14.743 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:13 smithi018 ceph-mon[31776]: pgmap v590: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 425 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-04-16T17:23:15.535 INFO:tasks.workunit.client.0.smithi018.stdout: 10 78267 12.05 MB/sec execute 207 sec latency 2228.148 ms 2024-04-16T17:23:16.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:15 smithi043 ceph-mon[36901]: pgmap v591: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 424 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-04-16T17:23:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:15 smithi022 ceph-mon[36023]: pgmap v591: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 424 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-04-16T17:23:16.535 INFO:tasks.workunit.client.0.smithi018.stdout: 10 78393 12.01 MB/sec execute 208 sec latency 1786.642 ms 2024-04-16T17:23:17.535 INFO:tasks.workunit.client.0.smithi018.stdout: 10 78630 12.06 MB/sec execute 209 sec latency 2261.349 ms 2024-04-16T17:23:18.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:17 smithi043 ceph-mon[36901]: pgmap v592: 129 pgs: 18 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 64 active+clean; 417 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 550 op/s 2024-04-16T17:23:18.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:17 smithi022 ceph-mon[36023]: pgmap v592: 129 pgs: 18 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 64 active+clean; 417 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 550 op/s 2024-04-16T17:23:18.472 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:15 smithi018 ceph-mon[31776]: pgmap v591: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 424 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-04-16T17:23:18.542 INFO:tasks.workunit.client.0.smithi018.stdout: 10 78806 12.04 MB/sec execute 210 sec latency 3267.984 ms 2024-04-16T17:23:18.804 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:17 smithi018 ceph-mon[31776]: pgmap v592: 129 pgs: 18 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 64 active+clean; 417 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 550 op/s 2024-04-16T17:23:19.542 INFO:tasks.workunit.client.0.smithi018.stdout: 10 78916 12.01 MB/sec execute 211 sec latency 3429.250 ms 2024-04-16T17:23:20.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:20 smithi022 ceph-mon[36023]: pgmap v593: 129 pgs: 18 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 64 active+clean; 417 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 415 op/s 2024-04-16T17:23:20.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:20 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:23:20.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:20 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:23:20.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:20 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:23:20.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:20 smithi043 ceph-mon[36901]: pgmap v593: 129 pgs: 18 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 64 active+clean; 417 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 415 op/s 2024-04-16T17:23:20.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:20 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:23:20.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:20 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:23:20.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:20 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:23:20.543 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79072 11.99 MB/sec execute 212 sec latency 2348.710 ms 2024-04-16T17:23:21.542 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79152 11.96 MB/sec execute 213 sec latency 3348.771 ms 2024-04-16T17:23:22.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:22 smithi022 ceph-mon[36023]: pgmap v594: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 421 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-04-16T17:23:22.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:22 smithi043 ceph-mon[36901]: pgmap v594: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 421 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-04-16T17:23:22.542 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79276 11.92 MB/sec execute 214 sec latency 4348.840 ms 2024-04-16T17:23:23.542 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79414 11.90 MB/sec execute 215 sec latency 5348.897 ms 2024-04-16T17:23:23.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:20 smithi018 ceph-mon[31776]: pgmap v593: 129 pgs: 18 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 64 active+clean; 417 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 415 op/s 2024-04-16T17:23:23.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:20 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:23:23.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:20 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:23:23.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:20 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:23:23.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:22 smithi018 ceph-mon[31776]: pgmap v594: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 421 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-04-16T17:23:24.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:24 smithi022 ceph-mon[36023]: pgmap v595: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 421 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-04-16T17:23:24.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:24 smithi043 ceph-mon[36901]: pgmap v595: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 421 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-04-16T17:23:24.542 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79582 11.86 MB/sec execute 216 sec latency 6233.494 ms 2024-04-16T17:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.384+0000 7f66a3c00640 1 -- 172.21.15.18:0/4069295070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c10e640 msgr2=0x7f669c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.384+0000 7f66a3c00640 1 --2- 172.21.15.18:0/4069295070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c10e640 0x7f669c10ea20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6690009930 tx=0x7f669002bc30 comp rx=0 tx=0).stop 2024-04-16T17:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.387+0000 7f66a3c00640 1 -- 172.21.15.18:0/4069295070 shutdown_connections 2024-04-16T17:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.387+0000 7f66a3c00640 1 --2- 172.21.15.18:0/4069295070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669c108ff0 0x7f669c115500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.387+0000 7f66a3c00640 1 --2- 172.21.15.18:0/4069295070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669c108630 0x7f669c108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.387+0000 7f66a3c00640 1 --2- 172.21.15.18:0/4069295070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c10e640 0x7f669c10ea20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.387+0000 7f66a3c00640 1 -- 172.21.15.18:0/4069295070 >> 172.21.15.18:0/4069295070 conn(0x7f669c076510 msgr2=0x7f669c078930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.388+0000 7f66a3c00640 1 -- 172.21.15.18:0/4069295070 shutdown_connections 2024-04-16T17:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.388+0000 7f66a3c00640 1 -- 172.21.15.18:0/4069295070 wait complete. 2024-04-16T17:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.390+0000 7f66a3c00640 1 Processor -- start 2024-04-16T17:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.390+0000 7f66a3c00640 1 -- start start 2024-04-16T17:23:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.391+0000 7f66a3c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669c108630 0x7f669c0825e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.391+0000 7f66a3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c108ff0 0x7f669c082b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.391+0000 7f66a3c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669c10e640 0x7f669c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.391+0000 7f66a3c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f669c118aa0 con 0x7f669c108630 2024-04-16T17:23:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.391+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669c10e640 0x7f669c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.391+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669c10e640 0x7f669c07b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48654/0 (socket says 172.21.15.18:48654) 2024-04-16T17:23:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.391+0000 7f66a1c00640 1 -- 172.21.15.18:0/2592430071 learned_addr learned my addr 172.21.15.18:0/2592430071 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f669c118920 con 0x7f669c10e640 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f669c118c20 con 0x7f669c108ff0 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f66a1200640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669c108630 0x7f669c0825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f669be00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c108ff0 0x7f669c082b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f66a1c00640 1 -- 172.21.15.18:0/2592430071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c108ff0 msgr2=0x7f669c082b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c108ff0 0x7f669c082b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f66a1c00640 1 -- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669c108630 msgr2=0x7f669c0825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669c108630 0x7f669c0825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.392+0000 7f66a1c00640 1 -- 172.21.15.18:0/2592430071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c07c1a0 con 0x7f669c10e640 2024-04-16T17:23:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.393+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669c10e640 0x7f669c07b8e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f669400b970 tx=0x7f669400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.393+0000 7f6699600640 1 -- 172.21.15.18:0/2592430071 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694019070 con 0x7f669c10e640 2024-04-16T17:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.393+0000 7f6699600640 1 -- 172.21.15.18:0/2592430071 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669400cb40 con 0x7f669c10e640 2024-04-16T17:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.393+0000 7f6699600640 1 -- 172.21.15.18:0/2592430071 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694007660 con 0x7f669c10e640 2024-04-16T17:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.393+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669c07c490 con 0x7f669c10e640 2024-04-16T17:23:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.394+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669c0803a0 con 0x7f669c10e640 2024-04-16T17:23:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.399+0000 7f6699600640 1 -- 172.21.15.18:0/2592430071 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f669401c070 con 0x7f669c10e640 2024-04-16T17:23:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.403+0000 7f6699600640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3d90 0x7f66840a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.403+0000 7f6699600640 1 -- 172.21.15.18:0/2592430071 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f66940d00b0 con 0x7f669c10e640 2024-04-16T17:23:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.404+0000 7f66a1200640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3d90 0x7f66840a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.405+0000 7f66a69f5740 1 Processor -- start 2024-04-16T17:23:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.405+0000 7f66a69f5740 1 -- start start 2024-04-16T17:23:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.406+0000 7f66a1200640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3d90 0x7f66840a6250 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6690009900 tx=0x7f669002df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.406+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f71610 0x55b440f70360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.406+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f70d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.406+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f7c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.406+0000 7f669be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f70d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f7c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f669be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f70d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58050/0 (socket says 172.21.15.18:58050) 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f7c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48662/0 (socket says 172.21.15.18:48662) 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f669be00640 1 -- 172.21.15.18:0/703898469 learned_addr learned my addr 172.21.15.18:0/703898469 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a69f5740 1 -- 172.21.15.18:0/703898469 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b440ed5780 con 0x55b440f708a0 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a69f5740 1 -- 172.21.15.18:0/703898469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b440e7bf40 con 0x55b440f79e20 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a69f5740 1 -- 172.21.15.18:0/703898469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b440ed1740 con 0x55b440f71610 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a1200640 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f71610 0x55b440f70360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a1c00640 1 -- 172.21.15.18:0/703898469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f71610 msgr2=0x55b440f70360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a1c00640 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f71610 0x55b440f70360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a1c00640 1 -- 172.21.15.18:0/703898469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 msgr2=0x55b440f70d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a1c00640 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f70d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f66a1c00640 1 -- 172.21.15.18:0/703898469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b440be8210 con 0x55b440f79e20 2024-04-16T17:23:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.407+0000 7f669be00640 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f70d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.409+0000 7f66a1c00640 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f7c210 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f66940cd450 tx=0x7f669409c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740e38f3d34311d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.409+0000 7f668e000640 1 -- 172.21.15.18:0/703898469 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66940bae60 con 0x55b440f79e20 2024-04-16T17:23:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.409+0000 7f668e000640 1 -- 172.21.15.18:0/703898469 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6694099fa0 con 0x55b440f79e20 2024-04-16T17:23:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.410+0000 7f66a69f5740 1 -- 172.21.15.18:0/703898469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 msgr2=0x55b440f7c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:25.410 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.410+0000 7f66a69f5740 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f7c210 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f66940cd450 tx=0x7f669409c4e0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.411+0000 7f66a69f5740 1 -- 172.21.15.18:0/703898469 shutdown_connections 2024-04-16T17:23:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.411+0000 7f66a69f5740 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f7c210 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.411+0000 7f66a69f5740 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f70d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.411+0000 7f66a69f5740 1 --2- 172.21.15.18:0/703898469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f71610 0x55b440f70360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.411+0000 7f66a69f5740 1 -- 172.21.15.18:0/703898469 >> 172.21.15.18:0/703898469 conn(0x55b440f0b5d0 msgr2=0x55b440f6b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.412+0000 7f66a69f5740 1 -- 172.21.15.18:0/703898469 shutdown_connections 2024-04-16T17:23:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.412+0000 7f66a69f5740 1 -- 172.21.15.18:0/703898469 wait complete. 2024-04-16T17:23:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.413+0000 7f66a69f5740 1 -- start start 2024-04-16T17:23:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.413+0000 7f66a69f5740 1 Processor -- start 2024-04-16T17:23:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.414+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.414+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f71610 0x55b440f76b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.414+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f79e20 0x55b440f77190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.414+0000 7f66a69f5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b440f7cbf0 con 0x55b440f708a0 2024-04-16T17:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.414+0000 7f66a69f5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b440e8f390 con 0x55b440f71610 2024-04-16T17:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.414+0000 7f66a69f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b440f7cd70 con 0x55b440f79e20 2024-04-16T17:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.414+0000 7f66a69f5740 10 client.0 _ll_register_callbacks cb 0x55b440e78f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.415+0000 7f669be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f71610 0x55b440f76b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.415+0000 7f669be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f71610 0x55b440f76b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48674/0 (socket says 172.21.15.18:48674) 2024-04-16T17:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.415+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f79e20 0x55b440f77190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.415+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f79e20 0x55b440f77190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37582/0 (socket says 172.21.15.18:37582) 2024-04-16T17:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.415+0000 7f66a1c00640 1 -- 172.21.15.18:0/2517062059 learned_addr learned my addr 172.21.15.18:0/2517062059 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.416+0000 7f66a1c00640 1 -- 172.21.15.18:0/2517062059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f71610 msgr2=0x55b440f76b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.416+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f71610 0x55b440f76b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.416+0000 7f66a1c00640 1 -- 172.21.15.18:0/2517062059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 msgr2=0x55b440f797f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.416+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.416+0000 7f66a1c00640 1 -- 172.21.15.18:0/2517062059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b440cb4620 con 0x55b440f79e20 2024-04-16T17:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.416+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f79e20 0x55b440f77190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f66940980b0 tx=0x7f6694098540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10eeb32b0ae8d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.416+0000 7f6682a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.417+0000 7f6682a00640 1 -- 172.21.15.18:0/2517062059 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66940ba060 con 0x55b440f79e20 2024-04-16T17:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.417+0000 7f6682a00640 1 -- 172.21.15.18:0/2517062059 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f66940a4680 con 0x55b440f79e20 2024-04-16T17:23:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.418+0000 7f66a69f5740 1 -- 172.21.15.18:0/2517062059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6694099fa0 con 0x55b440f79e20 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.418+0000 7f6682a00640 1 -- 172.21.15.18:0/2517062059 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f66940cd7c0 con 0x55b440f79e20 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.418+0000 7f66a69f5740 10 client.30295 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.418+0000 7f66a69f5740 1 -- 172.21.15.18:0/2517062059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b440f7cbf0 con 0x55b440f79e20 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.418+0000 7f6682a00640 1 -- 172.21.15.18:0/2517062059 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66940cd7c0 con 0x55b440f79e20 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.419+0000 7f6682a00640 1 -- 172.21.15.18:0/2517062059 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f669409abf0 con 0x55b440f79e20 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.419+0000 7f66a69f5740 10 client.30295 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.419+0000 7f66a69f5740 10 client.30295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.419+0000 7f66a69f5740 20 client.30295 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.419+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b440f9ae10 0x55b440f9d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.419+0000 7f66a69f5740 4 client.30295 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.419+0000 7f66a69f5740 1 -- 172.21.15.18:0/2517062059 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b440ee31a0 con 0x55b440f9ae10 2024-04-16T17:23:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.419+0000 7f66a1200640 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b440f9ae10 0x55b440f9d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.420+0000 7f66a1200640 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b440f9ae10 0x55b440f9d200 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7102383bcd6cb182 server_cookie=4532d4fd83b0975b in_seq=0 out_seq=0 2024-04-16T17:23:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.420+0000 7f6682a00640 10 client.30295 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.421+0000 7f6682a00640 1 -- 172.21.15.18:0/2517062059 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b440ee31a0 con 0x55b440f9ae10 2024-04-16T17:23:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.421+0000 7f6682a00640 10 client.30295 handle_command_reply: tid=0 2024-04-16T17:23:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.421+0000 7f66a69f5740 1 client.30295 shutdown 2024-04-16T17:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.421+0000 7f66a69f5740 1 -- 172.21.15.18:0/2517062059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f79e20 msgr2=0x55b440f77190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f79e20 0x55b440f77190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f66940980b0 tx=0x7f6694098540 comp rx=0 tx=0).stop 2024-04-16T17:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 -- 172.21.15.18:0/2517062059 shutdown_connections 2024-04-16T17:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b440f9ae10 0x55b440f9d200 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f79e20 0x55b440f77190 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f71610 0x55b440f76b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2517062059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f708a0 0x55b440f797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 -- 172.21.15.18:0/2517062059 >> 172.21.15.18:0/2517062059 conn(0x55b440f0b5d0 msgr2=0x55b440f6d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f6682a00640 0 client.30295 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 -- 172.21.15.18:0/2517062059 shutdown_connections 2024-04-16T17:23:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 1 -- 172.21.15.18:0/2517062059 wait complete. 2024-04-16T17:23:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.422+0000 7f66a69f5740 20 client.30295 trim_cache size 0 max 16384 2024-04-16T17:23:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.465+0000 7f66a69f5740 1 Processor -- start 2024-04-16T17:23:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.466+0000 7f66a69f5740 1 -- start start 2024-04-16T17:23:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.467+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 0x55b440f85b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.467+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 0x55b440f792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.467+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.467+0000 7f66a69f5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f667c012a90 con 0x55b440f71610 2024-04-16T17:23:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.467+0000 7f66a69f5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f667c010360 con 0x55b440f79e20 2024-04-16T17:23:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.467+0000 7f66a69f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f667c00e190 con 0x55b440f708a0 2024-04-16T17:23:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.468+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.468+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f797f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48688/0 (socket says 172.21.15.18:48688) 2024-04-16T17:23:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.468+0000 7f66a1c00640 1 -- 172.21.15.18:0/4069641731 learned_addr learned my addr 172.21.15.18:0/4069641731 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.468+0000 7f669be00640 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 0x55b440f792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f66a1c00640 1 -- 172.21.15.18:0/4069641731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 msgr2=0x55b440f85b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f66a1200640 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 0x55b440f85b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f66a1c00640 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 0x55b440f85b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f66a1c00640 1 -- 172.21.15.18:0/4069641731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 msgr2=0x55b440f792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f66a1c00640 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 0x55b440f792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f66a1c00640 1 -- 172.21.15.18:0/4069641731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b440f6fb70 con 0x55b440f79e20 2024-04-16T17:23:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f66a1200640 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 0x55b440f85b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f669be00640 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 0x55b440f792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.469+0000 7f66a1c00640 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f797f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f669400b940 tx=0x7f66940cdb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f315d812748ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.472+0000 7f668d600640 1 -- 172.21.15.18:0/4069641731 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669409ba70 con 0x55b440f79e20 2024-04-16T17:23:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.472+0000 7f668d600640 1 -- 172.21.15.18:0/4069641731 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6694099fa0 con 0x55b440f79e20 2024-04-16T17:23:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.472+0000 7f668d600640 1 -- 172.21.15.18:0/4069641731 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669409ab60 con 0x55b440f79e20 2024-04-16T17:23:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.474+0000 7f66a69f5740 1 -- 172.21.15.18:0/4069641731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 msgr2=0x55b440f797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.474+0000 7f66a69f5740 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f797f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f669400b940 tx=0x7f66940cdb90 comp rx=0 tx=0).stop 2024-04-16T17:23:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.479+0000 7f66a69f5740 1 -- 172.21.15.18:0/4069641731 shutdown_connections 2024-04-16T17:23:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.479+0000 7f66a69f5740 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f797f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.479+0000 7f66a69f5740 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 0x55b440f792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.479+0000 7f66a69f5740 1 --2- 172.21.15.18:0/4069641731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 0x55b440f85b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.479+0000 7f66a69f5740 1 -- 172.21.15.18:0/4069641731 >> 172.21.15.18:0/4069641731 conn(0x55b440f0b5d0 msgr2=0x55b440f75450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.481+0000 7f66a69f5740 1 -- 172.21.15.18:0/4069641731 shutdown_connections 2024-04-16T17:23:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.481+0000 7f66a69f5740 1 -- 172.21.15.18:0/4069641731 wait complete. 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.482+0000 7f66a69f5740 1 -- start start 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.482+0000 7f66a69f5740 1 Processor -- start 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.482+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 0x55b440f82080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.482+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 0x55b440f825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.483+0000 7f66a69f5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f73e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.483+0000 7f66a69f5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f667c00e820 con 0x55b440f71610 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.483+0000 7f66a69f5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f667c00fc90 con 0x55b440f79e20 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.483+0000 7f66a69f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f667c00eef0 con 0x55b440f708a0 2024-04-16T17:23:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.483+0000 7f66a69f5740 10 client.0 _ll_register_callbacks cb 0x7f668800b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.483+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f73e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.483+0000 7f66a1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f73e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:48692/0 (socket says 172.21.15.18:48692) 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.483+0000 7f66a1c00640 1 -- 172.21.15.18:0/2370374791 learned_addr learned my addr 172.21.15.18:0/2370374791 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.484+0000 7f66a1c00640 1 -- 172.21.15.18:0/2370374791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 msgr2=0x55b440f82080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.484+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 0x55b440f82080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.484+0000 7f66a1c00640 1 -- 172.21.15.18:0/2370374791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 msgr2=0x55b440f825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.484+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 0x55b440f825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.484+0000 7f66a1c00640 1 -- 172.21.15.18:0/2370374791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b440f74630 con 0x55b440f79e20 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.484+0000 7f66a1c00640 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f73e00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6694014e90 tx=0x7f6694004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abae6b95e1a0861 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.484+0000 7f6682a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.484+0000 7f6682a00640 1 -- 172.21.15.18:0/2370374791 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694097f40 con 0x55b440f79e20 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.485+0000 7f6682a00640 1 -- 172.21.15.18:0/2370374791 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f66940a4680 con 0x55b440f79e20 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.485+0000 7f66a69f5740 1 -- 172.21.15.18:0/2370374791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6694099fa0 con 0x55b440f79e20 2024-04-16T17:23:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.485+0000 7f6682a00640 1 -- 172.21.15.18:0/2370374791 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f669409ad70 con 0x55b440f79e20 2024-04-16T17:23:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.485+0000 7f66a69f5740 10 client.30443 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.485+0000 7f66a69f5740 1 -- 172.21.15.18:0/2370374791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b440f74630 con 0x55b440f79e20 2024-04-16T17:23:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.486+0000 7f6682a00640 1 -- 172.21.15.18:0/2370374791 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6694099fa0 con 0x55b440f79e20 2024-04-16T17:23:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.486+0000 7f6682a00640 1 -- 172.21.15.18:0/2370374791 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f66940a6c10 con 0x55b440f79e20 2024-04-16T17:23:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.486+0000 7f66a69f5740 10 client.30443 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.486+0000 7f66a69f5740 10 client.30443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.486+0000 7f66a69f5740 20 client.30443 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.486+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b4410231c0 0x55b4410255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.486+0000 7f66a69f5740 4 client.30443 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:23:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.486+0000 7f66a69f5740 1 -- 172.21.15.18:0/2370374791 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b440edd010 con 0x55b4410231c0 2024-04-16T17:23:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.487+0000 7f66a1200640 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b4410231c0 0x55b4410255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.491+0000 7f66a1200640 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b4410231c0 0x55b4410255b0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82bcfeec81170783 server_cookie=41d8300f2916db79 in_seq=0 out_seq=0 2024-04-16T17:23:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:25.491+0000 7f6682a00640 10 client.30443 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:25.546 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79762 11.85 MB/sec execute 217 sec latency 148.424 ms 2024-04-16T17:23:25.797 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:24 smithi018 ceph-mon[31776]: pgmap v595: 129 pgs: 15 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 75 active+clean; 421 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-04-16T17:23:26.048 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:26 smithi018 ceph-mon[31776]: pgmap v596: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 419 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 345 op/s 2024-04-16T17:23:26.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:26 smithi022 ceph-mon[36023]: pgmap v596: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 419 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 345 op/s 2024-04-16T17:23:26.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:26 smithi043 ceph-mon[36901]: pgmap v596: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 419 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 345 op/s 2024-04-16T17:23:26.546 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79795 11.81 MB/sec execute 218 sec latency 812.145 ms 2024-04-16T17:23:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.881+0000 7f6682a00640 1 -- 172.21.15.18:0/2370374791 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b440edd010 con 0x55b4410231c0 2024-04-16T17:23:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.881+0000 7f6682a00640 10 client.30443 handle_command_reply: tid=0 2024-04-16T17:23:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.881+0000 7f66a69f5740 1 client.30443 shutdown 2024-04-16T17:23:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f66a69f5740 1 -- 172.21.15.18:0/2370374791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 msgr2=0x55b440f73e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f73e00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6694014e90 tx=0x7f6694004a40 comp rx=0 tx=0).stop 2024-04-16T17:23:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f66a69f5740 1 -- 172.21.15.18:0/2370374791 shutdown_connections 2024-04-16T17:23:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b4410231c0 0x55b4410255b0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f6682a00640 0 client.30443 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b440f79e20 0x55b440f73e00 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b440f71610 0x55b440f825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f66a69f5740 1 --2- 172.21.15.18:0/2370374791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b440f708a0 0x55b440f82080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.884+0000 7f66a69f5740 1 -- 172.21.15.18:0/2370374791 >> 172.21.15.18:0/2370374791 conn(0x55b440f0b5d0 msgr2=0x55b440f6c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.885+0000 7f66a69f5740 1 -- 172.21.15.18:0/2370374791 shutdown_connections 2024-04-16T17:23:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.885+0000 7f66a69f5740 1 -- 172.21.15.18:0/2370374791 wait complete. 2024-04-16T17:23:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.885+0000 7f66a69f5740 20 client.30443 trim_cache size 0 max 16384 2024-04-16T17:23:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:23:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:23:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:23:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3d90 msgr2=0x7f66840a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3d90 0x7f66840a6250 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6690009900 tx=0x7f669002df90 comp rx=0 tx=0).stop 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669c10e640 msgr2=0x7f669c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669c10e640 0x7f669c07b8e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f669400b970 tx=0x7f669400be30 comp rx=0 tx=0).stop 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 shutdown_connections 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f66840a3d90 0x7f66840a6250 secure :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6690009900 tx=0x7f669002df90 comp rx=0 tx=0).stop 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669c10e640 0x7f669c07b8e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c108ff0 0x7f669c082b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 --2- 172.21.15.18:0/2592430071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f669c108630 0x7f669c0825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.888+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 >> 172.21.15.18:0/2592430071 conn(0x7f669c076510 msgr2=0x7f669c076c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.889+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 shutdown_connections 2024-04-16T17:23:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:26.889+0000 7f66a3c00640 1 -- 172.21.15.18:0/2592430071 wait complete. 2024-04-16T17:23:27.183 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:27 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:23:27.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:27 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:23:27.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:27 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:23:27.550 INFO:tasks.workunit.client.0.smithi018.stdout: 10 79926 11.78 MB/sec execute 219 sec latency 838.188 ms 2024-04-16T17:23:28.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:28 smithi022 ceph-mon[36023]: pgmap v597: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 422 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 384 op/s 2024-04-16T17:23:28.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:28 smithi043 ceph-mon[36901]: pgmap v597: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 422 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 384 op/s 2024-04-16T17:23:28.548 INFO:tasks.workunit.client.0.smithi018.stdout: 10 80086 11.75 MB/sec execute 220 sec latency 225.346 ms 2024-04-16T17:23:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:28 smithi018 ceph-mon[31776]: pgmap v597: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 422 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 384 op/s 2024-04-16T17:23:29.548 INFO:tasks.workunit.client.0.smithi018.stdout: 10 80088 11.70 MB/sec execute 221 sec latency 1207.636 ms 2024-04-16T17:23:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:30 smithi022 ceph-mon[36023]: pgmap v598: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 421 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 258 op/s 2024-04-16T17:23:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:30 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:23:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:30 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:23:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:30 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:23:30.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:30 smithi043 ceph-mon[36901]: pgmap v598: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 421 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 258 op/s 2024-04-16T17:23:30.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:30 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:23:30.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:30 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:23:30.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:30 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:23:30.548 INFO:tasks.workunit.client.0.smithi018.stdout: 10 80268 11.68 MB/sec execute 222 sec latency 1595.517 ms 2024-04-16T17:23:31.549 INFO:tasks.workunit.client.0.smithi018.stdout: 10 80492 11.72 MB/sec execute 223 sec latency 346.346 ms 2024-04-16T17:23:32.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:32 smithi022 ceph-mon[36023]: pgmap v599: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 420 op/s 2024-04-16T17:23:32.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:32 smithi043 ceph-mon[36901]: pgmap v599: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 420 op/s 2024-04-16T17:23:32.549 INFO:tasks.workunit.client.0.smithi018.stdout: 10 80782 11.72 MB/sec execute 224 sec latency 218.538 ms 2024-04-16T17:23:33.549 INFO:tasks.workunit.client.0.smithi018.stdout: 10 81008 11.71 MB/sec execute 225 sec latency 191.772 ms 2024-04-16T17:23:33.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:30 smithi018 ceph-mon[31776]: pgmap v598: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 421 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 258 op/s 2024-04-16T17:23:33.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:30 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:23:33.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:30 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:23:33.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:30 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:23:33.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:32 smithi018 ceph-mon[31776]: pgmap v599: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 420 op/s 2024-04-16T17:23:34.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:34 smithi022 ceph-mon[36023]: pgmap v600: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-16T17:23:34.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:34 smithi022 ceph-mon[36023]: osdmap e128: 12 total, 12 up, 12 in 2024-04-16T17:23:34.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:34 smithi043 ceph-mon[36901]: pgmap v600: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-16T17:23:34.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:34 smithi043 ceph-mon[36901]: osdmap e128: 12 total, 12 up, 12 in 2024-04-16T17:23:34.549 INFO:tasks.workunit.client.0.smithi018.stdout: 10 81208 11.71 MB/sec execute 226 sec latency 684.579 ms 2024-04-16T17:23:35.550 INFO:tasks.workunit.client.0.smithi018.stdout: 10 81594 11.76 MB/sec execute 227 sec latency 165.431 ms 2024-04-16T17:23:35.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:23:35.682 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:23:36.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.076+0000 7fbe34000640 1 -- 172.21.15.18:0/3370339639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 msgr2=0x7fbe2c114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.076+0000 7fbe34000640 1 --2- 172.21.15.18:0/3370339639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 0x7fbe2c114c10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009c10 tx=0x7fbe2002bef0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.081+0000 7fbe34000640 1 -- 172.21.15.18:0/3370339639 shutdown_connections 2024-04-16T17:23:36.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.081+0000 7fbe34000640 1 --2- 172.21.15.18:0/3370339639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbe2c115150 0x7fbe2c117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.081+0000 7fbe34000640 1 --2- 172.21.15.18:0/3370339639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 0x7fbe2c114c10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.081+0000 7fbe34000640 1 --2- 172.21.15.18:0/3370339639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe2c1091e0 0x7fbe2c1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.081+0000 7fbe34000640 1 -- 172.21.15.18:0/3370339639 >> 172.21.15.18:0/3370339639 conn(0x7fbe2c06d0a0 msgr2=0x7fbe2c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:36.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.082+0000 7fbe34000640 1 -- 172.21.15.18:0/3370339639 shutdown_connections 2024-04-16T17:23:36.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.082+0000 7fbe34000640 1 -- 172.21.15.18:0/3370339639 wait complete. 2024-04-16T17:23:36.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.087+0000 7fbe34000640 1 Processor -- start 2024-04-16T17:23:36.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.090+0000 7fbe34000640 1 -- start start 2024-04-16T17:23:36.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe34000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe2c1091e0 0x7fbe2c1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe34000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 0x7fbe2c1a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe34000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbe2c115150 0x7fbe2c1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe34000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c11abf0 con 0x7fbe2c115150 2024-04-16T17:23:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe34000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c11aa70 con 0x7fbe2c1091e0 2024-04-16T17:23:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe34000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c11ad70 con 0x7fbe2c109b00 2024-04-16T17:23:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 0x7fbe2c1a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 0x7fbe2c1a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37794/0 (socket says 172.21.15.18:37794) 2024-04-16T17:23:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.093+0000 7fbe30c00640 1 -- 172.21.15.18:0/3840655112 learned_addr learned my addr 172.21.15.18:0/3840655112 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.094+0000 7fbe31600640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe2c1091e0 0x7fbe2c1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.094+0000 7fbe30c00640 1 -- 172.21.15.18:0/3840655112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe2c1091e0 msgr2=0x7fbe2c1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.094+0000 7fbe30c00640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe2c1091e0 0x7fbe2c1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.094+0000 7fbe30c00640 1 -- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbe2c115150 msgr2=0x7fbe2c1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.094+0000 7fbe30c00640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbe2c115150 0x7fbe2c1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.094+0000 7fbe30c00640 1 -- 172.21.15.18:0/3840655112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c1a60a0 con 0x7fbe2c109b00 2024-04-16T17:23:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.095+0000 7fbe30c00640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 0x7fbe2c1a15f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000f80 tx=0x7fbe200336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.096+0000 7fbe2a000640 1 -- 172.21.15.18:0/3840655112 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20043020 con 0x7fbe2c109b00 2024-04-16T17:23:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.096+0000 7fbe2a000640 1 -- 172.21.15.18:0/3840655112 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe200338a0 con 0x7fbe2c109b00 2024-04-16T17:23:36.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.096+0000 7fbe2a000640 1 -- 172.21.15.18:0/3840655112 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2003e6b0 con 0x7fbe2c109b00 2024-04-16T17:23:36.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.097+0000 7fbe34000640 1 -- 172.21.15.18:0/3840655112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c1a6330 con 0x7fbe2c109b00 2024-04-16T17:23:36.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.097+0000 7fbe34000640 1 -- 172.21.15.18:0/3840655112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c10a9a0 con 0x7fbe2c109b00 2024-04-16T17:23:36.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.104+0000 7fbe2a000640 1 -- 172.21.15.18:0/3840655112 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbe20046020 con 0x7fbe2c109b00 2024-04-16T17:23:36.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.105+0000 7fbe36d87740 1 Processor -- start 2024-04-16T17:23:36.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.105+0000 7fbe36d87740 1 -- start start 2024-04-16T17:23:36.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f201b0 0x560692f20590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f20ad0 0x560692f29170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f296b0 0x560692f2baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe36d87740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560692e86120 con 0x560692f20ad0 2024-04-16T17:23:36.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe36d87740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560692e2c340 con 0x560692f296b0 2024-04-16T17:23:36.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe36d87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560692e820e0 con 0x560692f201b0 2024-04-16T17:23:36.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f201b0 0x560692f20590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f201b0 0x560692f20590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37810/0 (socket says 172.21.15.18:37810) 2024-04-16T17:23:36.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.106+0000 7fbe32000640 1 -- 172.21.15.18:0/3580849812 learned_addr learned my addr 172.21.15.18:0/3580849812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:36.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.107+0000 7fbe32000640 1 -- 172.21.15.18:0/3580849812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f296b0 msgr2=0x560692f2baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:36.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.107+0000 7fbe32000640 1 --2- 172.21.15.18:0/3580849812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f296b0 0x560692f2baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.107+0000 7fbe32000640 1 -- 172.21.15.18:0/3580849812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f20ad0 msgr2=0x560692f29170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:36.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.107+0000 7fbe32000640 1 --2- 172.21.15.18:0/3580849812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f20ad0 0x560692f29170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.107+0000 7fbe32000640 1 -- 172.21.15.18:0/3580849812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560692c627d0 con 0x560692f201b0 2024-04-16T17:23:36.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.107+0000 7fbe32000640 1 --2- 172.21.15.18:0/3580849812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f201b0 0x560692f20590 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400a9c0 tx=0x7fbe2400ae80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1246ec13934e8c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:36.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.108+0000 7fbe1e600640 1 -- 172.21.15.18:0/3580849812 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24014070 con 0x560692f201b0 2024-04-16T17:23:36.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.108+0000 7fbe1e600640 1 -- 172.21.15.18:0/3580849812 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2400cde0 con 0x560692f201b0 2024-04-16T17:23:36.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.108+0000 7fbe1e600640 1 -- 172.21.15.18:0/3580849812 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2400f6b0 con 0x560692f201b0 2024-04-16T17:23:36.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.109+0000 7fbe2a000640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbe140a81f0 0x7fbe140aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.109+0000 7fbe36d87740 1 -- 172.21.15.18:0/3580849812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f201b0 msgr2=0x560692f20590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.109+0000 7fbe36d87740 1 --2- 172.21.15.18:0/3580849812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f201b0 0x560692f20590 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400a9c0 tx=0x7fbe2400ae80 comp rx=0 tx=0).stop 2024-04-16T17:23:36.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.109+0000 7fbe2a000640 1 -- 172.21.15.18:0/3840655112 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fbe200f2960 con 0x7fbe2c109b00 2024-04-16T17:23:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.111+0000 7fbe36d87740 1 -- 172.21.15.18:0/3580849812 shutdown_connections 2024-04-16T17:23:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.111+0000 7fbe36d87740 1 --2- 172.21.15.18:0/3580849812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f296b0 0x560692f2baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.111+0000 7fbe36d87740 1 --2- 172.21.15.18:0/3580849812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f20ad0 0x560692f29170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.111+0000 7fbe36d87740 1 --2- 172.21.15.18:0/3580849812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f201b0 0x560692f20590 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.111+0000 7fbe36d87740 1 -- 172.21.15.18:0/3580849812 >> 172.21.15.18:0/3580849812 conn(0x560692f1a8e0 msgr2=0x560692f1acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.112+0000 7fbe31600640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbe140a81f0 0x7fbe140aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.113+0000 7fbe36d87740 1 -- 172.21.15.18:0/3580849812 shutdown_connections 2024-04-16T17:23:36.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.113+0000 7fbe36d87740 1 -- 172.21.15.18:0/3580849812 wait complete. 2024-04-16T17:23:36.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.114+0000 7fbe36d87740 1 -- start start 2024-04-16T17:23:36.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.114+0000 7fbe36d87740 1 Processor -- start 2024-04-16T17:23:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.116+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 0x560692f266c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f26c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f296b0 0x560692f27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe36d87740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560692f2c4a0 con 0x560692f201b0 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe36d87740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560692e3f710 con 0x560692f20ad0 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe36d87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560692f2c620 con 0x560692f296b0 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe36d87740 10 client.0 _ll_register_callbacks cb 0x560692e3aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f296b0 0x560692f27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f296b0 0x560692f27140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37816/0 (socket says 172.21.15.18:37816) 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe32000640 1 -- 172.21.15.18:0/1800877649 learned_addr learned my addr 172.21.15.18:0/1800877649 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.117+0000 7fbe31600640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbe140a81f0 0x7fbe140aa6b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x560692f1fc80 tx=0x7fbe18009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.118+0000 7fbe32000640 1 -- 172.21.15.18:0/1800877649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 msgr2=0x560692f26c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.118+0000 7fbe32000640 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f26c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.118+0000 7fbe32000640 1 -- 172.21.15.18:0/1800877649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 msgr2=0x560692f266c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:36.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.118+0000 7fbe31600640 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 0x560692f266c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.119+0000 7fbe32000640 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 0x560692f266c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.119+0000 7fbe32000640 1 -- 172.21.15.18:0/1800877649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560692b983f0 con 0x560692f296b0 2024-04-16T17:23:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.119+0000 7fbe32000640 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f296b0 0x560692f27140 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbe24004030 tx=0x7fbe24004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e9f5d38c6cf6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.119+0000 7fbe12a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.119+0000 7fbe31600640 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 0x560692f266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.120+0000 7fbe12a00640 1 -- 172.21.15.18:0/1800877649 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24014040 con 0x560692f296b0 2024-04-16T17:23:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.120+0000 7fbe12a00640 1 -- 172.21.15.18:0/1800877649 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fbe2400cbe0 con 0x560692f296b0 2024-04-16T17:23:36.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.120+0000 7fbe36d87740 1 -- 172.21.15.18:0/1800877649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe2400cde0 con 0x560692f296b0 2024-04-16T17:23:36.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.123+0000 7fbe12a00640 1 -- 172.21.15.18:0/1800877649 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbe24004490 con 0x560692f296b0 2024-04-16T17:23:36.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.123+0000 7fbe36d87740 10 client.30307 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:36.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.124+0000 7fbe36d87740 1 -- 172.21.15.18:0/1800877649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560692b580f0 con 0x560692f296b0 2024-04-16T17:23:36.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.128+0000 7fbe12a00640 1 -- 172.21.15.18:0/1800877649 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2400cde0 con 0x560692f296b0 2024-04-16T17:23:36.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.128+0000 7fbe12a00640 1 -- 172.21.15.18:0/1800877649 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fbe2400f450 con 0x560692f296b0 2024-04-16T17:23:36.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.129+0000 7fbe36d87740 10 client.30307 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:36.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.129+0000 7fbe36d87740 10 client.30307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:36.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.129+0000 7fbe36d87740 20 client.30307 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:36.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.129+0000 7fbe36d87740 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560692f425a0 0x560692f449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.129+0000 7fbe36d87740 4 client.30307 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:36.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.129+0000 7fbe36d87740 1 -- 172.21.15.18:0/1800877649 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560692e93b40 con 0x560692f425a0 2024-04-16T17:23:36.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.129+0000 7fbe31600640 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560692f425a0 0x560692f449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.130+0000 7fbe12a00640 10 client.30307 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:36.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.132+0000 7fbe31600640 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560692f425a0 0x560692f449d0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182aa63fcf6728f1 server_cookie=9479fc4ed761bfae in_seq=0 out_seq=0 2024-04-16T17:23:36.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.135+0000 7fbe12a00640 1 -- 172.21.15.18:0/1800877649 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560692e93b40 con 0x560692f425a0 2024-04-16T17:23:36.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.136+0000 7fbe12a00640 10 client.30307 handle_command_reply: tid=0 2024-04-16T17:23:36.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.136+0000 7fbe36d87740 1 client.30307 shutdown 2024-04-16T17:23:36.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.137+0000 7fbe36d87740 1 -- 172.21.15.18:0/1800877649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f296b0 msgr2=0x560692f27140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.137+0000 7fbe36d87740 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f296b0 0x560692f27140 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbe24004030 tx=0x7fbe24004060 comp rx=0 tx=0).stop 2024-04-16T17:23:36.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.140+0000 7fbe36d87740 1 -- 172.21.15.18:0/1800877649 shutdown_connections 2024-04-16T17:23:36.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.140+0000 7fbe36d87740 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560692f425a0 0x560692f449d0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.140+0000 7fbe36d87740 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f296b0 0x560692f27140 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.140+0000 7fbe36d87740 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f26c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.140+0000 7fbe36d87740 1 --2- 172.21.15.18:0/1800877649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 0x560692f266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.140+0000 7fbe36d87740 1 -- 172.21.15.18:0/1800877649 >> 172.21.15.18:0/1800877649 conn(0x560692f1a8e0 msgr2=0x560692f1c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:36.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.140+0000 7fbe12a00640 0 client.30307 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.141+0000 7fbe36d87740 1 -- 172.21.15.18:0/1800877649 shutdown_connections 2024-04-16T17:23:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.141+0000 7fbe36d87740 1 -- 172.21.15.18:0/1800877649 wait complete. 2024-04-16T17:23:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.142+0000 7fbe36d87740 20 client.30307 trim_cache size 0 max 16384 2024-04-16T17:23:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.183+0000 7fbe36d87740 1 Processor -- start 2024-04-16T17:23:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.183+0000 7fbe36d87740 1 -- start start 2024-04-16T17:23:36.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f1a8e0 0x560692f25680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f201b0 0x560692f3dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f20ad0 0x560692f26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe36d87740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1b8600 con 0x560692f1a8e0 2024-04-16T17:23:36.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe36d87740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1b5ed0 con 0x560692f201b0 2024-04-16T17:23:36.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe36d87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1b3d20 con 0x560692f20ad0 2024-04-16T17:23:36.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe30c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f201b0 0x560692f3dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe31600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f1a8e0 0x560692f25680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.184+0000 7fbe32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f20ad0 0x560692f26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe31600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f1a8e0 0x560692f25680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59718/0 (socket says 172.21.15.18:59718) 2024-04-16T17:23:36.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f20ad0 0x560692f26fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:37820/0 (socket says 172.21.15.18:37820) 2024-04-16T17:23:36.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe31600640 1 -- 172.21.15.18:0/4100657985 learned_addr learned my addr 172.21.15.18:0/4100657985 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:36.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe30c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f201b0 0x560692f3dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55822/0 (socket says 172.21.15.18:55822) 2024-04-16T17:23:36.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe31600640 1 -- 172.21.15.18:0/4100657985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f20ad0 msgr2=0x560692f26fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:36.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe31600640 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f20ad0 0x560692f26fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe31600640 1 -- 172.21.15.18:0/4100657985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f201b0 msgr2=0x560692f3dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:36.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe31600640 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f201b0 0x560692f3dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe31600640 1 -- 172.21.15.18:0/4100657985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560692f45050 con 0x560692f1a8e0 2024-04-16T17:23:36.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.185+0000 7fbe30c00640 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f201b0 0x560692f3dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:36.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.186+0000 7fbe32000640 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f20ad0 0x560692f26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:36.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.186+0000 7fbe31600640 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f1a8e0 0x560692f25680 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x560692f45250 tx=0x7fbe18013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb26f0ef0697775 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:36.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.186+0000 7fbe1dc00640 1 -- 172.21.15.18:0/4100657985 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18011db0 con 0x560692f1a8e0 2024-04-16T17:23:36.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.186+0000 7fbe1dc00640 1 -- 172.21.15.18:0/4100657985 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1802d070 con 0x560692f1a8e0 2024-04-16T17:23:36.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.186+0000 7fbe1dc00640 1 -- 172.21.15.18:0/4100657985 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1801ec70 con 0x560692f1a8e0 2024-04-16T17:23:36.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.187+0000 7fbe36d87740 1 -- 172.21.15.18:0/4100657985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f1a8e0 msgr2=0x560692f25680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.187+0000 7fbe36d87740 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f1a8e0 0x560692f25680 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x560692f45250 tx=0x7fbe18013000 comp rx=0 tx=0).stop 2024-04-16T17:23:36.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.189+0000 7fbe36d87740 1 -- 172.21.15.18:0/4100657985 shutdown_connections 2024-04-16T17:23:36.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.189+0000 7fbe36d87740 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f20ad0 0x560692f26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.189+0000 7fbe36d87740 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f201b0 0x560692f3dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.189+0000 7fbe36d87740 1 --2- 172.21.15.18:0/4100657985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f1a8e0 0x560692f25680 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.189+0000 7fbe36d87740 1 -- 172.21.15.18:0/4100657985 >> 172.21.15.18:0/4100657985 conn(0x560692f22110 msgr2=0x560692f1c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:36.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.190+0000 7fbe36d87740 1 -- 172.21.15.18:0/4100657985 shutdown_connections 2024-04-16T17:23:36.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.190+0000 7fbe36d87740 1 -- 172.21.15.18:0/4100657985 wait complete. 2024-04-16T17:23:36.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.191+0000 7fbe36d87740 1 -- start start 2024-04-16T17:23:36.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.191+0000 7fbe36d87740 1 Processor -- start 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f1a8e0 0x560692f3a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 0x560692f35810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe36d87740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f35d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe36d87740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1b4390 con 0x560692f201b0 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe36d87740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1b5800 con 0x560692f20ad0 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe36d87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1b4a60 con 0x560692f1a8e0 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe36d87740 10 client.0 _ll_register_callbacks cb 0x7fbe2400a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe32000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f35d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe32000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f35d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55828/0 (socket says 172.21.15.18:55828) 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe32000640 1 -- 172.21.15.18:0/2029671497 learned_addr learned my addr 172.21.15.18:0/2029671497 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.192+0000 7fbe31600640 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f1a8e0 0x560692f3a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.193+0000 7fbe32000640 1 -- 172.21.15.18:0/2029671497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f1a8e0 msgr2=0x560692f3a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.193+0000 7fbe32000640 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f1a8e0 0x560692f3a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.193+0000 7fbe32000640 1 -- 172.21.15.18:0/2029671497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 msgr2=0x560692f35810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.193+0000 7fbe32000640 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 0x560692f35810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.193+0000 7fbe32000640 1 -- 172.21.15.18:0/2029671497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560692f36580 con 0x560692f20ad0 2024-04-16T17:23:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.196+0000 7fbe32000640 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f35d50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbe2402a960 tx=0x7fbe2402afd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9153eb3f73a5df78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.196+0000 7fbe12a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.196+0000 7fbe12a00640 1 -- 172.21.15.18:0/2029671497 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24014040 con 0x560692f20ad0 2024-04-16T17:23:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.196+0000 7fbe12a00640 1 -- 172.21.15.18:0/2029671497 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fbe24018c90 con 0x560692f20ad0 2024-04-16T17:23:36.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.196+0000 7fbe36d87740 1 -- 172.21.15.18:0/2029671497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1802d070 con 0x560692f20ad0 2024-04-16T17:23:36.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.197+0000 7fbe12a00640 1 -- 172.21.15.18:0/2029671497 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbe240040d0 con 0x560692f20ad0 2024-04-16T17:23:36.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.197+0000 7fbe36d87740 10 client.30455 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:36.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.197+0000 7fbe36d87740 1 -- 172.21.15.18:0/2029671497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560692fa8b00 con 0x560692f20ad0 2024-04-16T17:23:36.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.197+0000 7fbe12a00640 1 -- 172.21.15.18:0/2029671497 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe24018e90 con 0x560692f20ad0 2024-04-16T17:23:36.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.198+0000 7fbe12a00640 1 -- 172.21.15.18:0/2029671497 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fbe2400f450 con 0x560692f20ad0 2024-04-16T17:23:36.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.198+0000 7fbe36d87740 10 client.30455 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:36.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.199+0000 7fbe36d87740 10 client.30455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:36.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.199+0000 7fbe36d87740 20 client.30455 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:36.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.199+0000 7fbe36d87740 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560692f296b0 0x560692fab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.199+0000 7fbe36d87740 4 client.30455 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:23:36.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.199+0000 7fbe36d87740 1 -- 172.21.15.18:0/2029671497 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbe1802d070 con 0x560692f296b0 2024-04-16T17:23:36.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.199+0000 7fbe31600640 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560692f296b0 0x560692fab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.200+0000 7fbe31600640 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560692f296b0 0x560692fab100 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ebc3d5635ea185 server_cookie=50185c111e28a8ab in_seq=0 out_seq=0 2024-04-16T17:23:36.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.201+0000 7fbe12a00640 10 client.30455 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.202+0000 7fbe12a00640 1 -- 172.21.15.18:0/2029671497 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fbe1802d070 con 0x560692f296b0 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.202+0000 7fbe12a00640 10 client.30455 handle_command_reply: tid=0 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.202+0000 7fbe36d87740 1 client.30455 shutdown 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.202+0000 7fbe36d87740 1 -- 172.21.15.18:0/2029671497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 msgr2=0x560692f35d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.202+0000 7fbe36d87740 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f35d50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbe2402a960 tx=0x7fbe2402afd0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 1 -- 172.21.15.18:0/2029671497 shutdown_connections 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560692f296b0 0x560692fab100 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560692f20ad0 0x560692f35d50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560692f201b0 0x560692f35810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 1 --2- 172.21.15.18:0/2029671497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560692f1a8e0 0x560692f3a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 1 -- 172.21.15.18:0/2029671497 >> 172.21.15.18:0/2029671497 conn(0x560692f22110 msgr2=0x560692f19040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:36.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe12a00640 0 client.30455 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:36.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 1 -- 172.21.15.18:0/2029671497 shutdown_connections 2024-04-16T17:23:36.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 1 -- 172.21.15.18:0/2029671497 wait complete. 2024-04-16T17:23:36.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.203+0000 7fbe36d87740 20 client.30455 trim_cache size 0 max 16384 2024-04-16T17:23:36.204 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:23:36.204 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:23:36.204 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:23:36.204 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 -- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbe140a81f0 msgr2=0x7fbe140aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbe140a81f0 0x7fbe140aa6b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x560692f1fc80 tx=0x7fbe18009260 comp rx=0 tx=0).stop 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 -- 172.21.15.18:0/3840655112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 msgr2=0x7fbe2c1a15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 0x7fbe2c1a15f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000f80 tx=0x7fbe200336b0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 -- 172.21.15.18:0/3840655112 shutdown_connections 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbe140a81f0 0x7fbe140aa6b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbe2c115150 0x7fbe2c1a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c109b00 0x7fbe2c1a15f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 --2- 172.21.15.18:0/3840655112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe2c1091e0 0x7fbe2c1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 -- 172.21.15.18:0/3840655112 >> 172.21.15.18:0/3840655112 conn(0x7fbe2c06d0a0 msgr2=0x7fbe2c070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:36.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 -- 172.21.15.18:0/3840655112 shutdown_connections 2024-04-16T17:23:36.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.205+0000 7fbe34000640 1 -- 172.21.15.18:0/3840655112 wait complete. 2024-04-16T17:23:36.232 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:23:36.232 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:23:36.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:36 smithi022 ceph-mon[36023]: pgmap v602: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 408 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 391 op/s 2024-04-16T17:23:36.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:36 smithi043 ceph-mon[36901]: pgmap v602: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 408 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 391 op/s 2024-04-16T17:23:36.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.523+0000 7fb667200640 1 -- 172.21.15.18:0/1633653657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6601085d0 msgr2=0x7fb660108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.523+0000 7fb667200640 1 --2- 172.21.15.18:0/1633653657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6601085d0 0x7fb660108a50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb658009ca0 tx=0x7fb65802c100 comp rx=0 tx=0).stop 2024-04-16T17:23:36.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.524+0000 7fb667200640 1 -- 172.21.15.18:0/1633653657 shutdown_connections 2024-04-16T17:23:36.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.524+0000 7fb667200640 1 --2- 172.21.15.18:0/1633653657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb660108f90 0x7fb6601154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.525+0000 7fb667200640 1 --2- 172.21.15.18:0/1633653657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6601085d0 0x7fb660108a50 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.525+0000 7fb667200640 1 --2- 172.21.15.18:0/1633653657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb66010e670 0x7fb66010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.525+0000 7fb667200640 1 -- 172.21.15.18:0/1633653657 >> 172.21.15.18:0/1633653657 conn(0x7fb660076590 msgr2=0x7fb6600789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:36.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.529+0000 7fb667200640 1 -- 172.21.15.18:0/1633653657 shutdown_connections 2024-04-16T17:23:36.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.530+0000 7fb667200640 1 -- 172.21.15.18:0/1633653657 wait complete. 2024-04-16T17:23:36.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.532+0000 7fb667200640 1 Processor -- start 2024-04-16T17:23:36.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.532+0000 7fb667200640 1 -- start start 2024-04-16T17:23:36.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.532+0000 7fb667200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6601085d0 0x7fb660082560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb65fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6601085d0 0x7fb660082560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb65fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6601085d0 0x7fb660082560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59732/0 (socket says 172.21.15.18:59732) 2024-04-16T17:23:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb65fe00640 1 -- 172.21.15.18:0/2259434980 learned_addr learned my addr 172.21.15.18:0/2259434980 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb667200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb660108f90 0x7fb660082aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb667200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb66010e670 0x7fb66007b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb660118a60 con 0x7fb6601085d0 2024-04-16T17:23:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6601188e0 con 0x7fb66010e670 2024-04-16T17:23:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb660118be0 con 0x7fb660108f90 2024-04-16T17:23:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb65f400640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb660108f90 0x7fb660082aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.533+0000 7fb665200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb66010e670 0x7fb66007b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.534+0000 7fb65fe00640 1 -- 172.21.15.18:0/2259434980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb660108f90 msgr2=0x7fb660082aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.534+0000 7fb65fe00640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb660108f90 0x7fb660082aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.534+0000 7fb65fe00640 1 -- 172.21.15.18:0/2259434980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb66010e670 msgr2=0x7fb66007b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.534+0000 7fb65fe00640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb66010e670 0x7fb66007b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.534+0000 7fb65fe00640 1 -- 172.21.15.18:0/2259434980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66007c080 con 0x7fb6601085d0 2024-04-16T17:23:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.534+0000 7fb665200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb66010e670 0x7fb66007b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.535+0000 7fb65f400640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb660108f90 0x7fb660082aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:23:36.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.536+0000 7fb65fe00640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6601085d0 0x7fb660082560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b450 tx=0x7fb64c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:36.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.536+0000 7fb65cc00640 1 -- 172.21.15.18:0/2259434980 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c015020 con 0x7fb6601085d0 2024-04-16T17:23:36.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.536+0000 7fb65cc00640 1 -- 172.21.15.18:0/2259434980 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64c011d70 con 0x7fb6601085d0 2024-04-16T17:23:36.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.536+0000 7fb65cc00640 1 -- 172.21.15.18:0/2259434980 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c00f340 con 0x7fb6601085d0 2024-04-16T17:23:36.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.537+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66007c370 con 0x7fb6601085d0 2024-04-16T17:23:36.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.537+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6600802c0 con 0x7fb6601085d0 2024-04-16T17:23:36.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.541+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb640005290 con 0x7fb6601085d0 2024-04-16T17:23:36.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.547+0000 7fb65cc00640 1 -- 172.21.15.18:0/2259434980 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb64c018020 con 0x7fb6601085d0 2024-04-16T17:23:36.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.549+0000 7fb65cc00640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6440a3de0 0x7fb6440a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:36.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.549+0000 7fb65cc00640 1 -- 172.21.15.18:0/2259434980 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fb64c0d0d00 con 0x7fb6601085d0 2024-04-16T17:23:36.549 INFO:tasks.workunit.client.0.smithi018.stdout: 10 81894 11.79 MB/sec execute 228 sec latency 235.567 ms 2024-04-16T17:23:36.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.550+0000 7fb65f400640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6440a3de0 0x7fb6440a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:36.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.552+0000 7fb65f400640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6440a3de0 0x7fb6440a62a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb658000f80 tx=0x7fb658032340 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:36.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.559+0000 7fb65cc00640 1 -- 172.21.15.18:0/2259434980 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb64c01f020 con 0x7fb6601085d0 2024-04-16T17:23:36.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.993+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb640005580 con 0x7fb6601085d0 2024-04-16T17:23:36.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:36.996+0000 7fb65cc00640 1 -- 172.21.15.18:0/2259434980 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7fb64c00d040 con 0x7fb6601085d0 2024-04-16T17:23:36.996 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:23:36.997 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:23:36.997 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:23:37.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.004+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6440a3de0 msgr2=0x7fb6440a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.004+0000 7fb667200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6440a3de0 0x7fb6440a62a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb658000f80 tx=0x7fb658032340 comp rx=0 tx=0).stop 2024-04-16T17:23:37.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.004+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6601085d0 msgr2=0x7fb660082560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.004+0000 7fb667200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6601085d0 0x7fb660082560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b450 tx=0x7fb64c00b910 comp rx=0 tx=0).stop 2024-04-16T17:23:37.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.005+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 shutdown_connections 2024-04-16T17:23:37.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.005+0000 7fb667200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6440a3de0 0x7fb6440a62a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.005+0000 7fb667200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb66010e670 0x7fb66007b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.005+0000 7fb667200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb660108f90 0x7fb660082aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.005+0000 7fb667200640 1 --2- 172.21.15.18:0/2259434980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6601085d0 0x7fb660082560 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.005+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 >> 172.21.15.18:0/2259434980 conn(0x7fb660076590 msgr2=0x7fb660077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.005+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 shutdown_connections 2024-04-16T17:23:37.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.005+0000 7fb667200640 1 -- 172.21.15.18:0/2259434980 wait complete. 2024-04-16T17:23:37.036 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:23:37.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.396+0000 7fbfdb200640 1 -- 172.21.15.18:0/279457502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401b6f0 msgr2=0x7fbfd40b7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.396+0000 7fbfdb200640 1 --2- 172.21.15.18:0/279457502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401b6f0 0x7fbfd40b7670 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009930 tx=0x7fbfcc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:23:37.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.396+0000 7fbfdb200640 1 -- 172.21.15.18:0/279457502 shutdown_connections 2024-04-16T17:23:37.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.396+0000 7fbfdb200640 1 --2- 172.21.15.18:0/279457502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd40b7bb0 0x7fbfd40b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.396+0000 7fbfdb200640 1 --2- 172.21.15.18:0/279457502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401b6f0 0x7fbfd40b7670 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.396+0000 7fbfdb200640 1 --2- 172.21.15.18:0/279457502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd401add0 0x7fbfd401b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.396+0000 7fbfdb200640 1 -- 172.21.15.18:0/279457502 >> 172.21.15.18:0/279457502 conn(0x7fbfd40a49e0 msgr2=0x7fbfd40a6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.397+0000 7fbfdb200640 1 -- 172.21.15.18:0/279457502 shutdown_connections 2024-04-16T17:23:37.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.397+0000 7fbfdb200640 1 -- 172.21.15.18:0/279457502 wait complete. 2024-04-16T17:23:37.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.399+0000 7fbfdb200640 1 Processor -- start 2024-04-16T17:23:37.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.399+0000 7fbfdb200640 1 -- start start 2024-04-16T17:23:37.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.400+0000 7fbfdb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401add0 0x7fbfd40b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.400+0000 7fbfdb200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd401b6f0 0x7fbfd40b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.400+0000 7fbfdb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd40b7bb0 0x7fbfd40b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.400+0000 7fbfdb200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd40a9870 con 0x7fbfd401b6f0 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.400+0000 7fbfdb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd40a96f0 con 0x7fbfd401add0 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.400+0000 7fbfdb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd40a99f0 con 0x7fbfd40b7bb0 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401add0 0x7fbfd40b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401add0 0x7fbfd40b73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53504/0 (socket says 172.21.15.18:53504) 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9e00640 1 -- 172.21.15.18:0/654761776 learned_addr learned my addr 172.21.15.18:0/654761776 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfda800640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd40b7bb0 0x7fbfd40b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9400640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd401b6f0 0x7fbfd40b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9e00640 1 -- 172.21.15.18:0/654761776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd40b7bb0 msgr2=0x7fbfd40b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9e00640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd40b7bb0 0x7fbfd40b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9e00640 1 -- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd401b6f0 msgr2=0x7fbfd40b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9e00640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd401b6f0 0x7fbfd40b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.401+0000 7fbfd9e00640 1 -- 172.21.15.18:0/654761776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd40b2120 con 0x7fbfd401add0 2024-04-16T17:23:37.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.402+0000 7fbfd9e00640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401add0 0x7fbfd40b73a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400c920 tx=0x7fbfc400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.402+0000 7fbfd2a00640 1 -- 172.21.15.18:0/654761776 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4007c70 con 0x7fbfd401add0 2024-04-16T17:23:37.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.403+0000 7fbfd2a00640 1 -- 172.21.15.18:0/654761776 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc4007e10 con 0x7fbfd401add0 2024-04-16T17:23:37.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.403+0000 7fbfd2a00640 1 -- 172.21.15.18:0/654761776 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4018890 con 0x7fbfd401add0 2024-04-16T17:23:37.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.403+0000 7fbfdb200640 1 -- 172.21.15.18:0/654761776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd4158a10 con 0x7fbfd401add0 2024-04-16T17:23:37.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.403+0000 7fbfdb200640 1 -- 172.21.15.18:0/654761776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd4158ee0 con 0x7fbfd401add0 2024-04-16T17:23:37.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.408+0000 7fbfe386d740 1 Processor -- start 2024-04-16T17:23:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.410+0000 7fbfe386d740 1 -- start start 2024-04-16T17:23:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.410+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3ef10 0x55adbdc3f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.410+0000 7fbfd2a00640 1 -- 172.21.15.18:0/654761776 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbfc4018a70 con 0x7fbfd401add0 2024-04-16T17:23:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 0x55adbdc47ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc48410 0x55adbdc4a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfe386d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55adbdba4020 con 0x55adbdc48410 2024-04-16T17:23:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfe386d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55adbdb4a460 con 0x55adbdc3ef10 2024-04-16T17:23:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfe386d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adbdb9ffe0 con 0x55adbdc3f830 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfda800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3ef10 0x55adbdc3f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfda800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3ef10 0x55adbdc3f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53516/0 (socket says 172.21.15.18:53516) 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfda800640 1 -- 172.21.15.18:0/2714803472 learned_addr learned my addr 172.21.15.18:0/2714803472 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.412+0000 7fbfd9400640 1 --2- 172.21.15.18:0/2714803472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc48410 0x55adbdc4a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.413+0000 7fbfda800640 1 -- 172.21.15.18:0/2714803472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 msgr2=0x55adbdc47ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.413+0000 7fbfda800640 1 --2- 172.21.15.18:0/2714803472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 0x55adbdc47ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.413+0000 7fbfda800640 1 -- 172.21.15.18:0/2714803472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc48410 msgr2=0x55adbdc4a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.413+0000 7fbfda800640 1 --2- 172.21.15.18:0/2714803472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc48410 0x55adbdc4a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.413+0000 7fbfda800640 1 -- 172.21.15.18:0/2714803472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adbdb52040 con 0x55adbdc3ef10 2024-04-16T17:23:37.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.413+0000 7fbfda800640 1 --2- 172.21.15.18:0/2714803472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3ef10 0x55adbdc3f2f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc050b40 tx=0x7fbfdc069e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1bdaad2af564c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.413+0000 7fbfd2a00640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfc00a3f20 0x7fbfc00a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.413+0000 7fbfcb000640 1 -- 172.21.15.18:0/2714803472 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc063220 con 0x55adbdc3ef10 2024-04-16T17:23:37.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.414+0000 7fbfcb000640 1 -- 172.21.15.18:0/2714803472 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfdc062440 con 0x55adbdc3ef10 2024-04-16T17:23:37.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.414+0000 7fbfd2a00640 1 -- 172.21.15.18:0/654761776 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fbfc40d0cb0 con 0x7fbfd401add0 2024-04-16T17:23:37.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.414+0000 7fbfcb000640 1 -- 172.21.15.18:0/2714803472 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc073640 con 0x55adbdc3ef10 2024-04-16T17:23:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.415+0000 7fbfe386d740 1 -- 172.21.15.18:0/2714803472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3ef10 msgr2=0x55adbdc3f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.415+0000 7fbfe386d740 1 --2- 172.21.15.18:0/2714803472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3ef10 0x55adbdc3f2f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc050b40 tx=0x7fbfdc069e80 comp rx=0 tx=0).stop 2024-04-16T17:23:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.415+0000 7fbfd9e00640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfc00a3f20 0x7fbfc00a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.416+0000 7fbfe386d740 1 -- 172.21.15.18:0/2714803472 shutdown_connections 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.416+0000 7fbfe386d740 1 --2- 172.21.15.18:0/2714803472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc48410 0x55adbdc4a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.416+0000 7fbfe386d740 1 --2- 172.21.15.18:0/2714803472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 0x55adbdc47ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.416+0000 7fbfe386d740 1 --2- 172.21.15.18:0/2714803472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3ef10 0x55adbdc3f2f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.416+0000 7fbfe386d740 1 -- 172.21.15.18:0/2714803472 >> 172.21.15.18:0/2714803472 conn(0x55adbdc39640 msgr2=0x55adbdc39a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.416+0000 7fbfe386d740 1 -- 172.21.15.18:0/2714803472 shutdown_connections 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.416+0000 7fbfe386d740 1 -- 172.21.15.18:0/2714803472 wait complete. 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.417+0000 7fbfe386d740 1 -- start start 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.417+0000 7fbfd9e00640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfc00a3f20 0x7fbfc00a63e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55adbdc3e9e0 tx=0x7fbfc4000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.417+0000 7fbfe386d740 1 Processor -- start 2024-04-16T17:23:37.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 0x55adbdc45930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 0x55adbdc45e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfda800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfda800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53584/0 (socket says 172.21.15.18:53584) 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfda800640 1 -- 172.21.15.18:0/1338146323 learned_addr learned my addr 172.21.15.18:0/1338146323 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55adbdc4b200 con 0x55adbdc3ef10 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfd9400640 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 0x55adbdc45930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55adbdb5d830 con 0x55adbdc48410 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adbdc4b380 con 0x55adbdc3f830 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.418+0000 7fbfda800640 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 0x55adbdc45e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.419+0000 7fbfe386d740 10 client.0 _ll_register_callbacks cb 0x55adbdb65150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:37.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.419+0000 7fbfda800640 1 -- 172.21.15.18:0/1338146323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 msgr2=0x55adbdc45930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.419+0000 7fbfda800640 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 0x55adbdc45930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.419+0000 7fbfda800640 1 -- 172.21.15.18:0/1338146323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 msgr2=0x55adbdc45e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.419+0000 7fbfda800640 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 0x55adbdc45e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.419+0000 7fbfda800640 1 -- 172.21.15.18:0/1338146323 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adbdb61920 con 0x55adbdc3ef10 2024-04-16T17:23:37.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.419+0000 7fbfda800640 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc453f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc065520 tx=0x7fbfdc067880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4469098d8672f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.419+0000 7fbfbf400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:23:37.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.420+0000 7fbfbf400640 1 -- 172.21.15.18:0/1338146323 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc084020 con 0x55adbdc3ef10 2024-04-16T17:23:37.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.420+0000 7fbfbf400640 1 -- 172.21.15.18:0/1338146323 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fbfdc0735f0 con 0x55adbdc3ef10 2024-04-16T17:23:37.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.420+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfdc062440 con 0x55adbdc3ef10 2024-04-16T17:23:37.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.420+0000 7fbfbf400640 1 -- 172.21.15.18:0/1338146323 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbfdc097050 con 0x55adbdc3ef10 2024-04-16T17:23:37.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.420+0000 7fbfe386d740 10 client.19608 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:37.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.420+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55adbdb2f7f0 con 0x55adbdc3ef10 2024-04-16T17:23:37.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.421+0000 7fbfbf400640 1 -- 172.21.15.18:0/1338146323 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfdc062440 con 0x55adbdc3ef10 2024-04-16T17:23:37.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.421+0000 7fbfbf400640 1 -- 172.21.15.18:0/1338146323 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fbfdc0737f0 con 0x55adbdc3ef10 2024-04-16T17:23:37.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.422+0000 7fbfe386d740 10 client.19608 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:37.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.422+0000 7fbfe386d740 10 client.19608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:23:37.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.422+0000 7fbfe386d740 20 client.19608 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:37.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.422+0000 7fbfe386d740 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55adbdc584a0 0x55adbdc5a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.422+0000 7fbfe386d740 4 client.19608 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:37.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.422+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adbdbb1a40 con 0x55adbdc584a0 2024-04-16T17:23:37.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.422+0000 7fbfd9400640 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55adbdc584a0 0x55adbdc5a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:37 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2259434980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:23:37.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.423+0000 7fbfd9400640 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55adbdc584a0 0x55adbdc5a890 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=511dae1c22ae01e5 server_cookie=f442bad4c0eb9f7 in_seq=0 out_seq=0 2024-04-16T17:23:37.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.423+0000 7fbfbf400640 10 client.19608 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:23:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfbf400640 1 -- 172.21.15.18:0/1338146323 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55adbdbb1a40 con 0x55adbdc584a0 2024-04-16T17:23:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfbf400640 10 client.19608 handle_command_reply: tid=0 2024-04-16T17:23:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 client.19608 shutdown 2024-04-16T17:23:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 msgr2=0x55adbdc453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc453f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc065520 tx=0x7fbfdc067880 comp rx=0 tx=0).stop 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 shutdown_connections 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55adbdc584a0 0x55adbdc5a890 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 0x55adbdc45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3f830 0x55adbdc45930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 --2- 172.21.15.18:0/1338146323 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc453f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 >> 172.21.15.18:0/1338146323 conn(0x55adbdc39640 msgr2=0x55adbdc3b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfbf400640 0 client.19608 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.425+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 shutdown_connections 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.426+0000 7fbfe386d740 1 -- 172.21.15.18:0/1338146323 wait complete. 2024-04-16T17:23:37.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.426+0000 7fbfe386d740 20 client.19608 trim_cache size 0 max 16384 2024-04-16T17:23:37.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.447+0000 7fbfe386d740 1 Processor -- start 2024-04-16T17:23:37.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.447+0000 7fbfe386d740 1 -- start start 2024-04-16T17:23:37.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.447+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3ef10 0x55adbdc540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.447+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3f830 0x55adbdc44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 0x55adbdc453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfe386d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4165a70 con 0x55adbdc3f830 2024-04-16T17:23:37.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfe386d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4163340 con 0x55adbdc48410 2024-04-16T17:23:37.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfe386d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4161190 con 0x55adbdc3ef10 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfda800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3ef10 0x55adbdc540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfd9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3f830 0x55adbdc44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfda800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3ef10 0x55adbdc540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51364/0 (socket says 172.21.15.18:51364) 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfda800640 1 -- 172.21.15.18:0/4200562824 learned_addr learned my addr 172.21.15.18:0/4200562824 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfda800640 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 0x55adbdc453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfd9400640 1 -- 172.21.15.18:0/4200562824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3ef10 msgr2=0x55adbdc540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfd9400640 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3ef10 0x55adbdc540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfd9400640 1 -- 172.21.15.18:0/4200562824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 msgr2=0x55adbdc453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfd9400640 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 0x55adbdc453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfd9400640 1 -- 172.21.15.18:0/4200562824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adbdc45a90 con 0x55adbdc3f830 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.448+0000 7fbfda800640 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3ef10 0x55adbdc540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:37.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.449+0000 7fbfd9400640 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3f830 0x55adbdc44cd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc036040 tx=0x7fbfcc02df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ec698b3c5b481a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.449+0000 7fbfca600640 1 -- 172.21.15.18:0/4200562824 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc04abd0 con 0x55adbdc3f830 2024-04-16T17:23:37.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.449+0000 7fbfca600640 1 -- 172.21.15.18:0/4200562824 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55adbdbb1a40 con 0x55adbdc3f830 2024-04-16T17:23:37.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.449+0000 7fbfca600640 1 -- 172.21.15.18:0/4200562824 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc005040 con 0x55adbdc3f830 2024-04-16T17:23:37.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.450+0000 7fbfe386d740 1 -- 172.21.15.18:0/4200562824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3f830 msgr2=0x55adbdc44cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.450+0000 7fbfe386d740 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3f830 0x55adbdc44cd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc036040 tx=0x7fbfcc02df90 comp rx=0 tx=0).stop 2024-04-16T17:23:37.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.451+0000 7fbfe386d740 1 -- 172.21.15.18:0/4200562824 shutdown_connections 2024-04-16T17:23:37.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.451+0000 7fbfe386d740 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc48410 0x55adbdc453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.451+0000 7fbfe386d740 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3f830 0x55adbdc44cd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.451+0000 7fbfe386d740 1 --2- 172.21.15.18:0/4200562824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc3ef10 0x55adbdc540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.451+0000 7fbfe386d740 1 -- 172.21.15.18:0/4200562824 >> 172.21.15.18:0/4200562824 conn(0x55adbdc39640 msgr2=0x55adbdc38700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.452+0000 7fbfe386d740 1 -- 172.21.15.18:0/4200562824 shutdown_connections 2024-04-16T17:23:37.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.452+0000 7fbfe386d740 1 -- 172.21.15.18:0/4200562824 wait complete. 2024-04-16T17:23:37.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.452+0000 7fbfe386d740 1 -- start start 2024-04-16T17:23:37.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.453+0000 7fbfe386d740 1 Processor -- start 2024-04-16T17:23:37.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.453+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.453+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3f830 0x55adbdc41d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.453+0000 7fbfe386d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc48410 0x55adbdc42280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.453+0000 7fbfe386d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4161800 con 0x55adbdc3ef10 2024-04-16T17:23:37.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.453+0000 7fbfe386d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4162c70 con 0x55adbdc3f830 2024-04-16T17:23:37.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.453+0000 7fbfe386d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4161ed0 con 0x55adbdc48410 2024-04-16T17:23:37.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.453+0000 7fbfe386d740 10 client.0 _ll_register_callbacks cb 0x7fbfdc0683f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.454+0000 7fbfda800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.454+0000 7fbfda800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53588/0 (socket says 172.21.15.18:53588) 2024-04-16T17:23:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.454+0000 7fbfda800640 1 -- 172.21.15.18:0/14124781 learned_addr learned my addr 172.21.15.18:0/14124781 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.454+0000 7fbfda800640 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc48410 0x55adbdc42280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.455+0000 7fbfda800640 1 -- 172.21.15.18:0/14124781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc48410 msgr2=0x55adbdc42280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.455+0000 7fbfda800640 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc48410 0x55adbdc42280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.455+0000 7fbfda800640 1 -- 172.21.15.18:0/14124781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3f830 msgr2=0x55adbdc41d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.455+0000 7fbfda800640 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3f830 0x55adbdc41d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.455+0000 7fbfda800640 1 -- 172.21.15.18:0/14124781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adbd8b63f0 con 0x55adbdc3ef10 2024-04-16T17:23:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.455+0000 7fbfda800640 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc508f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc065520 tx=0x7fbfdc067880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428fd1bad5e65a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.455+0000 7fbfbf400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:23:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.456+0000 7fbfbf400640 1 -- 172.21.15.18:0/14124781 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc084040 con 0x55adbdc3ef10 2024-04-16T17:23:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.456+0000 7fbfbf400640 1 -- 172.21.15.18:0/14124781 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fbfdc073800 con 0x55adbdc3ef10 2024-04-16T17:23:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.456+0000 7fbfe386d740 1 -- 172.21.15.18:0/14124781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55adbdbb1a40 con 0x55adbdc3ef10 2024-04-16T17:23:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.456+0000 7fbfbf400640 1 -- 172.21.15.18:0/14124781 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbfdc089070 con 0x55adbdc3ef10 2024-04-16T17:23:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.456+0000 7fbfe386d740 10 client.19614 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.456+0000 7fbfe386d740 1 -- 172.21.15.18:0/14124781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55adbd9807d0 con 0x55adbdc3ef10 2024-04-16T17:23:37.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.456+0000 7fbfbf400640 1 -- 172.21.15.18:0/14124781 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55adbdbb1a40 con 0x55adbdc3ef10 2024-04-16T17:23:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.458+0000 7fbfbf400640 1 -- 172.21.15.18:0/14124781 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fbfdc069040 con 0x55adbdc3ef10 2024-04-16T17:23:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.458+0000 7fbfe386d740 10 client.19614 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.458+0000 7fbfe386d740 10 client.19614 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:23:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.458+0000 7fbfe386d740 20 client.19614 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.458+0000 7fbfe386d740 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55adbdcbe950 0x55adbdcc0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.458+0000 7fbfe386d740 4 client.19614 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:23:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.458+0000 7fbfd9400640 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55adbdcbe950 0x55adbdcc0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.458+0000 7fbfe386d740 1 -- 172.21.15.18:0/14124781 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adbdbab8b0 con 0x55adbdcbe950 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.459+0000 7fbfd9400640 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55adbdcbe950 0x55adbdcc0d70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f50211ece1e3acc server_cookie=1847b58b36a13762 in_seq=0 out_seq=0 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.459+0000 7fbfbf400640 10 client.19614 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.459+0000 7fbfbf400640 1 -- 172.21.15.18:0/14124781 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adbdbab8b0 con 0x55adbdcbe950 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.459+0000 7fbfbf400640 10 client.19614 handle_command_reply: tid=0 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.459+0000 7fbfe386d740 1 client.19614 shutdown 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.459+0000 7fbfe386d740 1 -- 172.21.15.18:0/14124781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 msgr2=0x55adbdc508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.459+0000 7fbfe386d740 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc508f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc065520 tx=0x7fbfdc067880 comp rx=0 tx=0).stop 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 1 -- 172.21.15.18:0/14124781 shutdown_connections 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55adbdcbe950 0x55adbdcc0d70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adbdc48410 0x55adbdc42280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adbdc3f830 0x55adbdc41d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfbf400640 0 client.19614 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:23:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 1 --2- 172.21.15.18:0/14124781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55adbdc3ef10 0x55adbdc508f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 1 -- 172.21.15.18:0/14124781 >> 172.21.15.18:0/14124781 conn(0x55adbdc39640 msgr2=0x55adbdc3a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 1 -- 172.21.15.18:0/14124781 shutdown_connections 2024-04-16T17:23:37.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 1 -- 172.21.15.18:0/14124781 wait complete. 2024-04-16T17:23:37.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.460+0000 7fbfe386d740 20 client.19614 trim_cache size 0 max 16384 2024-04-16T17:23:37.461 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:23:37.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.461+0000 7fbfdb200640 1 -- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfc00a3f20 msgr2=0x7fbfc00a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.461+0000 7fbfdb200640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfc00a3f20 0x7fbfc00a63e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55adbdc3e9e0 tx=0x7fbfc4000bc0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.461+0000 7fbfdb200640 1 -- 172.21.15.18:0/654761776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401add0 msgr2=0x7fbfd40b73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.461+0000 7fbfdb200640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401add0 0x7fbfd40b73a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400c920 tx=0x7fbfc400cde0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.462+0000 7fbfdb200640 1 -- 172.21.15.18:0/654761776 shutdown_connections 2024-04-16T17:23:37.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.462+0000 7fbfdb200640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fbfc00a3f20 0x7fbfc00a63e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.462+0000 7fbfdb200640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd40b7bb0 0x7fbfd40b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.462+0000 7fbfdb200640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbfd401b6f0 0x7fbfd40b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.462+0000 7fbfdb200640 1 --2- 172.21.15.18:0/654761776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfd401add0 0x7fbfd40b73a0 secure :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400c920 tx=0x7fbfc400cde0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.462+0000 7fbfdb200640 1 -- 172.21.15.18:0/654761776 >> 172.21.15.18:0/654761776 conn(0x7fbfd40a49e0 msgr2=0x7fbfd40b8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.462+0000 7fbfdb200640 1 -- 172.21.15.18:0/654761776 shutdown_connections 2024-04-16T17:23:37.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.462+0000 7fbfdb200640 1 -- 172.21.15.18:0/654761776 wait complete. 2024-04-16T17:23:37.481 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:23:37.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:37 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2259434980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:23:37.549 INFO:tasks.workunit.client.0.smithi018.stdout: 10 82144 11.79 MB/sec execute 229 sec latency 542.728 ms 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.713+0000 7f24e0600640 1 -- 172.21.15.18:0/2962349218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8100690 msgr2=0x7f24d8100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.713+0000 7f24e0600640 1 --2- 172.21.15.18:0/2962349218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8100690 0x7f24d8100a70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009930 tx=0x7f24cc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.713+0000 7f24e0600640 1 -- 172.21.15.18:0/2962349218 shutdown_connections 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.713+0000 7f24e0600640 1 --2- 172.21.15.18:0/2962349218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d81055b0 0x7f24d810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.714+0000 7f24e0600640 1 --2- 172.21.15.18:0/2962349218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d8101040 0x7f24d8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.714+0000 7f24e0600640 1 --2- 172.21.15.18:0/2962349218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8100690 0x7f24d8100a70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.714+0000 7f24e0600640 1 -- 172.21.15.18:0/2962349218 >> 172.21.15.18:0/2962349218 conn(0x7f24d8075da0 msgr2=0x7f24d80761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.714+0000 7f24e0600640 1 -- 172.21.15.18:0/2962349218 shutdown_connections 2024-04-16T17:23:37.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.714+0000 7f24e0600640 1 -- 172.21.15.18:0/2962349218 wait complete. 2024-04-16T17:23:37.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.715+0000 7f24e0600640 1 Processor -- start 2024-04-16T17:23:37.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.715+0000 7f24e0600640 1 -- start start 2024-04-16T17:23:37.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.715+0000 7f24e0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d8100690 0x7f24d8101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.715+0000 7f24e0600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81055b0 0x7f24d81023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.715+0000 7f24e0600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d8103b30 0x7f24d8102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.715+0000 7f24e0600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8110130 con 0x7f24d8103b30 2024-04-16T17:23:37.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.716+0000 7f24e0600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24d810ffb0 con 0x7f24d81055b0 2024-04-16T17:23:37.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.716+0000 7f24e0600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81102b0 con 0x7f24d8100690 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.716+0000 7f24dd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81055b0 0x7f24d81023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.716+0000 7f24dd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81055b0 0x7f24d81023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53560/0 (socket says 172.21.15.18:53560) 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.716+0000 7f24de600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d8103b30 0x7f24d8102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.716+0000 7f24dd200640 1 -- 172.21.15.18:0/2264046399 learned_addr learned my addr 172.21.15.18:0/2264046399 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.716+0000 7f24ddc00640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d8100690 0x7f24d8101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.717+0000 7f24de600640 1 -- 172.21.15.18:0/2264046399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d8100690 msgr2=0x7f24d8101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.717+0000 7f24de600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d8100690 0x7f24d8101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.717+0000 7f24de600640 1 -- 172.21.15.18:0/2264046399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81055b0 msgr2=0x7f24d81023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.717+0000 7f24de600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81055b0 0x7f24d81023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.717+0000 7f24de600640 1 -- 172.21.15.18:0/2264046399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d81ab5a0 con 0x7f24d8103b30 2024-04-16T17:23:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.717+0000 7f24dd200640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81055b0 0x7f24d81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:23:37.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.717+0000 7f24de600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d8103b30 0x7f24d8102910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f24d000da10 tx=0x7f24d000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.718+0000 7f24d6000640 1 -- 172.21.15.18:0/2264046399 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d000bd70 con 0x7f24d8103b30 2024-04-16T17:23:37.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.718+0000 7f24d6000640 1 -- 172.21.15.18:0/2264046399 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d0003e50 con 0x7f24d8103b30 2024-04-16T17:23:37.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.718+0000 7f24d6000640 1 -- 172.21.15.18:0/2264046399 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d00106a0 con 0x7f24d8103b30 2024-04-16T17:23:37.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.718+0000 7f24e0600640 1 -- 172.21.15.18:0/2264046399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d81ab890 con 0x7f24d8103b30 2024-04-16T17:23:37.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.718+0000 7f24e0600640 1 -- 172.21.15.18:0/2264046399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d81abd90 con 0x7f24d8103b30 2024-04-16T17:23:37.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.722+0000 7f24e3416740 1 Processor -- start 2024-04-16T17:23:37.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.725+0000 7f24d6000640 1 -- 172.21.15.18:0/2264046399 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24d0003970 con 0x7f24d8103b30 2024-04-16T17:23:37.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.725+0000 7f24e3416740 1 -- start start 2024-04-16T17:23:37.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.725+0000 7f24e3416740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db770d50 0x5567db771130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24e3416740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db779d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24e3416740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db77a250 0x5567db77c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24e3416740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5567db6d6020 con 0x5567db771670 2024-04-16T17:23:37.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24e3416740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5567db67c460 con 0x5567db770d50 2024-04-16T17:23:37.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24e3416740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5567db6d1fe0 con 0x5567db77a250 2024-04-16T17:23:37.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db770d50 0x5567db771130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db770d50 0x5567db771130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53574/0 (socket says 172.21.15.18:53574) 2024-04-16T17:23:37.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 -- 172.21.15.18:0/2206570062 learned_addr learned my addr 172.21.15.18:0/2206570062 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db77a250 0x5567db77c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 -- 172.21.15.18:0/2206570062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db77a250 msgr2=0x5567db77c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24ddc00640 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db779d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db77a250 0x5567db77c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 -- 172.21.15.18:0/2206570062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 msgr2=0x5567db779d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db779d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24dd200640 1 -- 172.21.15.18:0/2206570062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567db684040 con 0x5567db770d50 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.726+0000 7f24ddc00640 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db779d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.727+0000 7f24dd200640 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db770d50 0x5567db771130 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f24c400af80 tx=0x7f24c4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2e0af360eb8a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.727+0000 7f24d6000640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24c00a3da0 0x7f24c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.727+0000 7f24d6000640 1 -- 172.21.15.18:0/2264046399 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7f24d00d1020 con 0x7f24d8103b30 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.727+0000 7f24caa00640 1 -- 172.21.15.18:0/2206570062 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4017e00 con 0x5567db770d50 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.727+0000 7f24caa00640 1 -- 172.21.15.18:0/2206570062 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c4005500 con 0x5567db770d50 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.727+0000 7f24caa00640 1 -- 172.21.15.18:0/2206570062 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c40095e0 con 0x5567db770d50 2024-04-16T17:23:37.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.727+0000 7f24ddc00640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24c00a3da0 0x7f24c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.728+0000 7f24e3416740 1 -- 172.21.15.18:0/2206570062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db770d50 msgr2=0x5567db771130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.728+0000 7f24e3416740 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db770d50 0x5567db771130 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f24c400af80 tx=0x7f24c4005f90 comp rx=0 tx=0).stop 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.728+0000 7f24ddc00640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24c00a3da0 0x7f24c00a6260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24cc006010 tx=0x7f24cc005b50 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.728+0000 7f24e3416740 1 -- 172.21.15.18:0/2206570062 shutdown_connections 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.728+0000 7f24e3416740 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db77a250 0x5567db77c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.728+0000 7f24e3416740 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db779d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.728+0000 7f24e3416740 1 --2- 172.21.15.18:0/2206570062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db770d50 0x5567db771130 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.728+0000 7f24e3416740 1 -- 172.21.15.18:0/2206570062 >> 172.21.15.18:0/2206570062 conn(0x5567db76b480 msgr2=0x5567db76b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.729+0000 7f24e3416740 1 -- 172.21.15.18:0/2206570062 shutdown_connections 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.729+0000 7f24e3416740 1 -- 172.21.15.18:0/2206570062 wait complete. 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.729+0000 7f24e3416740 1 -- start start 2024-04-16T17:23:37.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.729+0000 7f24e3416740 1 Processor -- start 2024-04-16T17:23:37.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.730+0000 7f24e3416740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db777370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.730+0000 7f24e3416740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db7778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.730+0000 7f24e3416740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db777df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.730+0000 7f24ddc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db777370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24ddc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db777370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51410/0 (socket says 172.21.15.18:51410) 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24ddc00640 1 -- 172.21.15.18:0/187628280 learned_addr learned my addr 172.21.15.18:0/187628280 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24dd200640 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db7778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.730+0000 7f24e3416740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5567db77cfe0 con 0x5567db771670 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5567db68f830 con 0x5567db77a250 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5567db77d160 con 0x5567db770d50 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24e3416740 10 client.0 _ll_register_callbacks cb 0x5567db697150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24de600640 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db777df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24ddc00640 1 -- 172.21.15.18:0/187628280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 msgr2=0x5567db777df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24ddc00640 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db777df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24ddc00640 1 -- 172.21.15.18:0/187628280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 msgr2=0x5567db7778b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24ddc00640 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db7778b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24ddc00640 1 -- 172.21.15.18:0/187628280 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567db693920 con 0x5567db770d50 2024-04-16T17:23:37.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.731+0000 7f24de600640 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db777df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:37.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.732+0000 7f24ddc00640 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db777370 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f24cc035000 tx=0x7f24cc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c90698687e496f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.732+0000 7f24bf400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.732+0000 7f24bf400640 1 -- 172.21.15.18:0/187628280 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc04d020 con 0x5567db770d50 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.732+0000 7f24bf400640 1 -- 172.21.15.18:0/187628280 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7f24cc0048c0 con 0x5567db770d50 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.732+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c4005500 con 0x5567db770d50 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.732+0000 7f24bf400640 1 -- 172.21.15.18:0/187628280 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f24cc058050 con 0x5567db770d50 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.732+0000 7f24e3416740 10 client.30343 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.732+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5567db6617f0 con 0x5567db770d50 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.733+0000 7f24bf400640 1 -- 172.21.15.18:0/187628280 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c4005500 con 0x5567db770d50 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.733+0000 7f24bf400640 1 -- 172.21.15.18:0/187628280 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f24cc003ae0 con 0x5567db770d50 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.733+0000 7f24e3416740 10 client.30343 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:37.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.733+0000 7f24e3416740 10 client.30343 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:23:37.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.733+0000 7f24e3416740 20 client.30343 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:37.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.733+0000 7f24e3416740 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567db78a670 0x5567db78cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.733+0000 7f24e3416740 4 client.30343 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:37.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.733+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567db6e3a40 con 0x5567db78a670 2024-04-16T17:23:37.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.734+0000 7f24dd200640 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567db78a670 0x5567db78cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.736+0000 7f24bf400640 10 client.30343 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:23:37.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.736+0000 7f24dd200640 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567db78a670 0x5567db78cad0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72906d8c85d04ddf server_cookie=6494ce6fb8a86a25 in_seq=0 out_seq=0 2024-04-16T17:23:37.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.738+0000 7f24bf400640 1 -- 172.21.15.18:0/187628280 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567db6e3a40 con 0x5567db78a670 2024-04-16T17:23:37.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.738+0000 7f24bf400640 10 client.30343 handle_command_reply: tid=0 2024-04-16T17:23:37.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.738+0000 7f24e3416740 1 client.30343 shutdown 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.738+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 msgr2=0x5567db777370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.738+0000 7f24e3416740 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db777370 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f24cc035000 tx=0x7f24cc009260 comp rx=0 tx=0).stop 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.739+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 shutdown_connections 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.739+0000 7f24e3416740 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567db78a670 0x5567db78cad0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.739+0000 7f24e3416740 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db777df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.739+0000 7f24e3416740 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db771670 0x5567db7778b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.739+0000 7f24e3416740 1 --2- 172.21.15.18:0/187628280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db777370 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.739+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 >> 172.21.15.18:0/187628280 conn(0x5567db76b480 msgr2=0x5567db76d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.739+0000 7f24bf400640 0 client.30343 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:23:37.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.743+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 shutdown_connections 2024-04-16T17:23:37.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.743+0000 7f24e3416740 1 -- 172.21.15.18:0/187628280 wait complete. 2024-04-16T17:23:37.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.743+0000 7f24e3416740 20 client.30343 trim_cache size 0 max 16384 2024-04-16T17:23:37.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.772+0000 7f24e3416740 1 Processor -- start 2024-04-16T17:23:37.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.772+0000 7f24e3416740 1 -- start start 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.772+0000 7f24e3416740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db770d50 0x5567db779aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.772+0000 7f24e3416740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db771670 0x5567db785ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.772+0000 7f24e3416740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db7923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.772+0000 7f24e3416740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81b8680 con 0x5567db770d50 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.772+0000 7f24e3416740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81b5f50 con 0x5567db77a250 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.772+0000 7f24e3416740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81b3da0 con 0x5567db771670 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24de600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db7923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24de600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db7923a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53596/0 (socket says 172.21.15.18:53596) 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24de600640 1 -- 172.21.15.18:0/2495058197 learned_addr learned my addr 172.21.15.18:0/2495058197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24dd200640 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db771670 0x5567db785ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24ddc00640 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db770d50 0x5567db779aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24de600640 1 -- 172.21.15.18:0/2495058197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db771670 msgr2=0x5567db785ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24de600640 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db771670 0x5567db785ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24de600640 1 -- 172.21.15.18:0/2495058197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db770d50 msgr2=0x5567db779aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24de600640 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db770d50 0x5567db779aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24de600640 1 -- 172.21.15.18:0/2495058197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567db3e83f0 con 0x5567db77a250 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.773+0000 7f24dd200640 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db771670 0x5567db785ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.774+0000 7f24ddc00640 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db770d50 0x5567db779aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.774+0000 7f24de600640 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db7923a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f24d000b630 tx=0x7f24d009d350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b865ecd575c4671 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.774+0000 7f24ca000640 1 -- 172.21.15.18:0/2495058197 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d00b6070 con 0x5567db77a250 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.774+0000 7f24ca000640 1 -- 172.21.15.18:0/2495058197 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d00a5800 con 0x5567db77a250 2024-04-16T17:23:37.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.774+0000 7f24ca000640 1 -- 172.21.15.18:0/2495058197 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0014070 con 0x5567db77a250 2024-04-16T17:23:37.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.775+0000 7f24e3416740 1 -- 172.21.15.18:0/2495058197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 msgr2=0x5567db7923a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.775+0000 7f24e3416740 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db7923a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f24d000b630 tx=0x7f24d009d350 comp rx=0 tx=0).stop 2024-04-16T17:23:37.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.776+0000 7f24e3416740 1 -- 172.21.15.18:0/2495058197 shutdown_connections 2024-04-16T17:23:37.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.776+0000 7f24e3416740 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db77a250 0x5567db7923a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.776+0000 7f24e3416740 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db771670 0x5567db785ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.776+0000 7f24e3416740 1 --2- 172.21.15.18:0/2495058197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db770d50 0x5567db779aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.776+0000 7f24e3416740 1 -- 172.21.15.18:0/2495058197 >> 172.21.15.18:0/2495058197 conn(0x5567db76b480 msgr2=0x5567db76d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.776+0000 7f24e3416740 1 -- 172.21.15.18:0/2495058197 shutdown_connections 2024-04-16T17:23:37.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.776+0000 7f24e3416740 1 -- 172.21.15.18:0/2495058197 wait complete. 2024-04-16T17:23:37.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.776+0000 7f24e3416740 1 -- start start 2024-04-16T17:23:37.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.777+0000 7f24e3416740 1 Processor -- start 2024-04-16T17:23:37.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.777+0000 7f24e3416740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db7741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24e3416740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db771670 0x5567db774700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24e3416740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db77a250 0x5567db774c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24e3416740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81b4410 con 0x5567db77a250 2024-04-16T17:23:37.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24e3416740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81b5880 con 0x5567db771670 2024-04-16T17:23:37.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24e3416740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81b4ae0 con 0x5567db770d50 2024-04-16T17:23:37.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24e3416740 10 client.0 _ll_register_callbacks cb 0x7f24c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:37.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24dd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db771670 0x5567db774700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24dd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db771670 0x5567db774700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53600/0 (socket says 172.21.15.18:53600) 2024-04-16T17:23:37.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24dd200640 1 -- 172.21.15.18:0/3329257724 learned_addr learned my addr 172.21.15.18:0/3329257724 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:37.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24ddc00640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db7741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24de600640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db77a250 0x5567db774c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24dd200640 1 -- 172.21.15.18:0/3329257724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 msgr2=0x5567db7741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24dd200640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db7741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.778+0000 7f24dd200640 1 -- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db77a250 msgr2=0x5567db774c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24dd200640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db77a250 0x5567db774c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24dd200640 1 -- 172.21.15.18:0/3329257724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567db4b27d0 con 0x5567db771670 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24ddc00640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db7741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24de600640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db77a250 0x5567db774c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24dd200640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db771670 0x5567db774700 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c010 tx=0x7f24c400a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56799f83bbca0775 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24bf400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24bf400640 1 -- 172.21.15.18:0/3329257724 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4017e80 con 0x5567db771670 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24bf400640 1 -- 172.21.15.18:0/3329257724 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7f24c4004680 con 0x5567db771670 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.779+0000 7f24e3416740 1 -- 172.21.15.18:0/3329257724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d00a5800 con 0x5567db771670 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24bf400640 1 -- 172.21.15.18:0/3329257724 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f24c4004880 con 0x5567db771670 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24e3416740 10 client.30491 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24e3416740 1 -- 172.21.15.18:0/3329257724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5567db7f0ed0 con 0x5567db771670 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24bf400640 1 -- 172.21.15.18:0/3329257724 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d00a5800 con 0x5567db771670 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24bf400640 1 -- 172.21.15.18:0/3329257724 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f24c4009810 con 0x5567db771670 2024-04-16T17:23:37.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24e3416740 10 client.30491 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:37.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24e3416740 10 client.30491 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:23:37.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24e3416740 20 client.30491 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:37.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24e3416740 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567db7f1220 0x5567db7f3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:37.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24e3416740 4 client.30491 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:23:37.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.780+0000 7f24e3416740 1 -- 172.21.15.18:0/3329257724 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567db6dd8b0 con 0x5567db7f1220 2024-04-16T17:23:37.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.781+0000 7f24ddc00640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567db7f1220 0x5567db7f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:37.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.782+0000 7f24ddc00640 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567db7f1220 0x5567db7f3610 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3b6774831cc0e86 server_cookie=dc558602f44fb85d in_seq=0 out_seq=0 2024-04-16T17:23:37.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.782+0000 7f24bf400640 10 client.30491 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:23:37.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24bf400640 1 -- 172.21.15.18:0/3329257724 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567db6dd8b0 con 0x5567db7f1220 2024-04-16T17:23:37.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24bf400640 10 client.30491 handle_command_reply: tid=0 2024-04-16T17:23:37.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 client.30491 shutdown 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 -- 172.21.15.18:0/3329257724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db771670 msgr2=0x5567db774700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db771670 0x5567db774700 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c010 tx=0x7f24c400a780 comp rx=0 tx=0).stop 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 -- 172.21.15.18:0/3329257724 shutdown_connections 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5567db7f1220 0x5567db7f3610 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5567db77a250 0x5567db774c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567db771670 0x5567db774700 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 --2- 172.21.15.18:0/3329257724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567db770d50 0x5567db7741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24bf400640 0 client.30491 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 -- 172.21.15.18:0/3329257724 >> 172.21.15.18:0/3329257724 conn(0x5567db76b480 msgr2=0x5567db76c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.783+0000 7f24e3416740 1 -- 172.21.15.18:0/3329257724 shutdown_connections 2024-04-16T17:23:37.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.784+0000 7f24e3416740 1 -- 172.21.15.18:0/3329257724 wait complete. 2024-04-16T17:23:37.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.784+0000 7f24e3416740 20 client.30491 trim_cache size 0 max 16384 2024-04-16T17:23:37.785 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:23:37.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 -- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24c00a3da0 msgr2=0x7f24c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24c00a3da0 0x7f24c00a6260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24cc006010 tx=0x7f24cc005b50 comp rx=0 tx=0).stop 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 -- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d8103b30 msgr2=0x7f24d8102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d8103b30 0x7f24d8102910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f24d000da10 tx=0x7f24d000ded0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 -- 172.21.15.18:0/2264046399 shutdown_connections 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f24c00a3da0 0x7f24c00a6260 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24d8103b30 0x7f24d8102910 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81055b0 0x7f24d81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 --2- 172.21.15.18:0/2264046399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d8100690 0x7f24d8101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 -- 172.21.15.18:0/2264046399 >> 172.21.15.18:0/2264046399 conn(0x7f24d8075da0 msgr2=0x7f24d81044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 -- 172.21.15.18:0/2264046399 shutdown_connections 2024-04-16T17:23:37.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:37.785+0000 7f24e0600640 1 -- 172.21.15.18:0/2264046399 wait complete. 2024-04-16T17:23:37.805 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:23:38.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 -- 172.21.15.18:0/567329619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 msgr2=0x7fb1c41011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/567329619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 0x7fb1c41011d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009900 tx=0x7fb1b402bc30 comp rx=0 tx=0).stop 2024-04-16T17:23:38.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 -- 172.21.15.18:0/567329619 shutdown_connections 2024-04-16T17:23:38.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/567329619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1c410e9a0 0x7fb1c4110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/567329619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1c4101710 0x7fb1c410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/567329619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 0x7fb1c41011d0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 -- 172.21.15.18:0/567329619 >> 172.21.15.18:0/567329619 conn(0x7fb1c40780b0 msgr2=0x7fb1c40784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:38.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 -- 172.21.15.18:0/567329619 shutdown_connections 2024-04-16T17:23:38.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.081+0000 7fb1c8a00640 1 -- 172.21.15.18:0/567329619 wait complete. 2024-04-16T17:23:38.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.082+0000 7fb1c8a00640 1 Processor -- start 2024-04-16T17:23:38.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c8a00640 1 -- start start 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 0x7fb1c41a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1c4101710 0x7fb1c41a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c8a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1c410e9a0 0x7fb1c419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c8a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c4114610 con 0x7fb1c410e9a0 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c8a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c4114490 con 0x7fb1c4100df0 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c4114790 con 0x7fb1c4101710 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 0x7fb1c41a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 0x7fb1c41a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53620/0 (socket says 172.21.15.18:53620) 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.083+0000 7fb1c1e00640 1 -- 172.21.15.18:0/1717289255 learned_addr learned my addr 172.21.15.18:0/1717289255 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:38.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.084+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1c4101710 0x7fb1c41a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.084+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1c410e9a0 0x7fb1c419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.084+0000 7fb1c1e00640 1 -- 172.21.15.18:0/1717289255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1c4101710 msgr2=0x7fb1c41a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.084+0000 7fb1c1e00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1c4101710 0x7fb1c41a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.084+0000 7fb1c1e00640 1 -- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1c410e9a0 msgr2=0x7fb1c419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.084+0000 7fb1c1e00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1c410e9a0 0x7fb1c419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.084+0000 7fb1c1e00640 1 -- 172.21.15.18:0/1717289255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c419b170 con 0x7fb1c4100df0 2024-04-16T17:23:38.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.084+0000 7fb1c1e00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 0x7fb1c41a0840 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb1b402c240 tx=0x7fb1b40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:38.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.085+0000 7fb1baa00640 1 -- 172.21.15.18:0/1717289255 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4043020 con 0x7fb1c4100df0 2024-04-16T17:23:38.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.085+0000 7fb1baa00640 1 -- 172.21.15.18:0/1717289255 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b40054e0 con 0x7fb1c4100df0 2024-04-16T17:23:38.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.085+0000 7fb1baa00640 1 -- 172.21.15.18:0/1717289255 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4003b30 con 0x7fb1c4100df0 2024-04-16T17:23:38.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.085+0000 7fb1c8a00640 1 -- 172.21.15.18:0/1717289255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c419b400 con 0x7fb1c4100df0 2024-04-16T17:23:38.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.085+0000 7fb1c8a00640 1 -- 172.21.15.18:0/1717289255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c41a7700 con 0x7fb1c4100df0 2024-04-16T17:23:38.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.088+0000 7fb1cb78e740 1 Processor -- start 2024-04-16T17:23:38.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1cb78e740 1 -- start start 2024-04-16T17:23:38.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688df670 0x55d1688e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d1688ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d168844020 con 0x55d1688ded50 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d1687ea460 con 0x55d1688e8250 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d16883ffe0 con 0x55d1688df670 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1c1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d1688ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1c1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d1688ea640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53626/0 (socket says 172.21.15.18:53626) 2024-04-16T17:23:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.089+0000 7fb1c1400640 1 -- 172.21.15.18:0/3946906922 learned_addr learned my addr 172.21.15.18:0/3946906922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:38.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.090+0000 7fb1baa00640 1 -- 172.21.15.18:0/1717289255 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb1b4046020 con 0x7fb1c4100df0 2024-04-16T17:23:38.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.090+0000 7fb1c2800640 1 -- 172.21.15.18:0/3946906922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688df670 msgr2=0x55d1688e7d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:38.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.090+0000 7fb1c2800640 1 --2- 172.21.15.18:0/3946906922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688df670 0x55d1688e7d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.090+0000 7fb1c2800640 1 -- 172.21.15.18:0/3946906922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 msgr2=0x55d1688ea640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:38.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.090+0000 7fb1c2800640 1 --2- 172.21.15.18:0/3946906922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d1688ea640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.090+0000 7fb1c2800640 1 -- 172.21.15.18:0/3946906922 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1687f2040 con 0x55d1688ded50 2024-04-16T17:23:38.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.090+0000 7fb1c1400640 1 --2- 172.21.15.18:0/3946906922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d1688ea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:38.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.091+0000 7fb1c2800640 1 --2- 172.21.15.18:0/3946906922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688df130 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc005fd0 tx=0x7fb1bc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b46080fd85f3450 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:38.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.091+0000 7fb1af000640 1 -- 172.21.15.18:0/3946906922 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc003b70 con 0x55d1688ded50 2024-04-16T17:23:38.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.091+0000 7fb1af000640 1 -- 172.21.15.18:0/3946906922 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc003d10 con 0x55d1688ded50 2024-04-16T17:23:38.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.091+0000 7fb1af000640 1 -- 172.21.15.18:0/3946906922 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc012940 con 0x55d1688ded50 2024-04-16T17:23:38.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1baa00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1a80a3da0 0x7fb1a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1baa00640 1 -- 172.21.15.18:0/1717289255 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fb1b40f29a0 con 0x7fb1c4100df0 2024-04-16T17:23:38.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1cb78e740 1 -- 172.21.15.18:0/3946906922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 msgr2=0x55d1688df130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/3946906922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688df130 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc005fd0 tx=0x7fb1bc0117b0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1cb78e740 1 -- 172.21.15.18:0/3946906922 shutdown_connections 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/3946906922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d1688ea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/3946906922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688df670 0x55d1688e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/3946906922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688df130 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1cb78e740 1 -- 172.21.15.18:0/3946906922 >> 172.21.15.18:0/3946906922 conn(0x55d1688d9480 msgr2=0x55d1688d9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.092+0000 7fb1c1e00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1a80a3da0 0x7fb1a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.093+0000 7fb1cb78e740 1 -- 172.21.15.18:0/3946906922 shutdown_connections 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.093+0000 7fb1cb78e740 1 -- 172.21.15.18:0/3946906922 wait complete. 2024-04-16T17:23:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.094+0000 7fb1c1e00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1a80a3da0 0x7fb1a80a6260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55d1688de820 tx=0x7fb1b40035f0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.094+0000 7fb1cb78e740 1 -- start start 2024-04-16T17:23:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.094+0000 7fb1cb78e740 1 Processor -- start 2024-04-16T17:23:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688e52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688df670 0x55d1688e5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 0x55d1688e5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d1688eb040 con 0x55d1688ded50 2024-04-16T17:23:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d1687fd830 con 0x55d1688df670 2024-04-16T17:23:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1688eb1c0 con 0x55d1688e8250 2024-04-16T17:23:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1cb78e740 10 client.0 _ll_register_callbacks cb 0x55d168805150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688e52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1c2800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688e52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53660/0 (socket says 172.21.15.18:53660) 2024-04-16T17:23:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1c2800640 1 -- 172.21.15.18:0/1918008643 learned_addr learned my addr 172.21.15.18:0/1918008643 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:38.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.096+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688df670 0x55d1688e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.097+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 0x55d1688e5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.097+0000 7fb1c1400640 1 -- 172.21.15.18:0/1918008643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 msgr2=0x55d1688e5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:38.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.097+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 0x55d1688e5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.097+0000 7fb1c1400640 1 -- 172.21.15.18:0/1918008643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 msgr2=0x55d1688e52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:38.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.097+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688e52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.097+0000 7fb1c1400640 1 -- 172.21.15.18:0/1918008643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d168801920 con 0x55d1688df670 2024-04-16T17:23:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.098+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688df670 0x55d1688e5820 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00126d0 tx=0x7fb1b0012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc78f3b49faf83f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.098+0000 7fb1a3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.098+0000 7fb1a3400640 1 -- 172.21.15.18:0/1918008643 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000f920 con 0x55d1688df670 2024-04-16T17:23:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.098+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1918008643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1bc003d10 con 0x55d1688df670 2024-04-16T17:23:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.099+0000 7fb1a3400640 1 -- 172.21.15.18:0/1918008643 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fb1b0018070 con 0x55d1688df670 2024-04-16T17:23:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.099+0000 7fb1a3400640 1 -- 172.21.15.18:0/1918008643 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb1b000f920 con 0x55d1688df670 2024-04-16T17:23:38.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.100+0000 7fb1cb78e740 10 client.30503 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:38.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.100+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1918008643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d1687cf7f0 con 0x55d1688df670 2024-04-16T17:23:38.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.101+0000 7fb1a3400640 1 -- 172.21.15.18:0/1918008643 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc003d10 con 0x55d1688df670 2024-04-16T17:23:38.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.101+0000 7fb1a3400640 1 -- 172.21.15.18:0/1918008643 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fb1b001cc80 con 0x55d1688df670 2024-04-16T17:23:38.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.102+0000 7fb1cb78e740 10 client.30503 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:38.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.102+0000 7fb1cb78e740 10 client.30503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:38.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.102+0000 7fb1cb78e740 20 client.30503 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:38.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.102+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d168900f50 0x55d168903380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.102+0000 7fb1cb78e740 4 client.30503 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:38.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.102+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1918008643 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d168851a40 con 0x55d168900f50 2024-04-16T17:23:38.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.102+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d168900f50 0x55d168903380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.103+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d168900f50 0x55d168903380 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea577fc503b08075 server_cookie=b09d04206682953f in_seq=0 out_seq=0 2024-04-16T17:23:38.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.103+0000 7fb1a3400640 10 client.30503 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:38.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.105+0000 7fb1a3400640 1 -- 172.21.15.18:0/1918008643 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d168851a40 con 0x55d168900f50 2024-04-16T17:23:38.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.105+0000 7fb1a3400640 10 client.30503 handle_command_reply: tid=0 2024-04-16T17:23:38.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.106+0000 7fb1cb78e740 1 client.30503 shutdown 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.106+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688df670 msgr2=0x55d1688e5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.106+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688df670 0x55d1688e5820 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00126d0 tx=0x7fb1b0012b90 comp rx=0 tx=0).stop 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1918008643 shutdown_connections 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d168900f50 0x55d168903380 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1a3400640 0 client.30503 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 0x55d1688e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688df670 0x55d1688e5820 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1918008643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688ded50 0x55d1688e52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1918008643 >> 172.21.15.18:0/1918008643 conn(0x55d1688d9480 msgr2=0x55d1688db1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:38.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1918008643 shutdown_connections 2024-04-16T17:23:38.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1918008643 wait complete. 2024-04-16T17:23:38.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.107+0000 7fb1cb78e740 20 client.30503 trim_cache size 0 max 16384 2024-04-16T17:23:38.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.131+0000 7fb1cb78e740 1 Processor -- start 2024-04-16T17:23:38.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.131+0000 7fb1cb78e740 1 -- start start 2024-04-16T17:23:38.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.132+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688ded50 0x55d1688e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.132+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 0x55d1688f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.132+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d168908c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.132+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4012a50 con 0x55d1688df670 2024-04-16T17:23:38.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.132+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4010320 con 0x55d1688e8250 2024-04-16T17:23:38.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.132+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a400e170 con 0x55d1688ded50 2024-04-16T17:23:38.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.132+0000 7fb1c2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688ded50 0x55d1688e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.132+0000 7fb1c2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688ded50 0x55d1688e7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51476/0 (socket says 172.21.15.18:51476) 2024-04-16T17:23:38.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.133+0000 7fb1c2800640 1 -- 172.21.15.18:0/1299296749 learned_addr learned my addr 172.21.15.18:0/1299296749 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:38.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.133+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d168908c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.133+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 0x55d1688f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.133+0000 7fb1c2800640 1 -- 172.21.15.18:0/1299296749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 msgr2=0x55d168908c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:38.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.133+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d168908c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.133+0000 7fb1c2800640 1 -- 172.21.15.18:0/1299296749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 msgr2=0x55d1688f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:38.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.133+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 0x55d1688f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.133+0000 7fb1c2800640 1 -- 172.21.15.18:0/1299296749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1688df4a0 con 0x55d1688ded50 2024-04-16T17:23:38.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.134+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688ded50 0x55d1688e7a10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc012530 tx=0x7fb1bc012470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f3cf7cc5a6dfa5e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:38.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.137+0000 7fb1ae600640 1 -- 172.21.15.18:0/1299296749 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc030020 con 0x55d1688ded50 2024-04-16T17:23:38.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.137+0000 7fb1ae600640 1 -- 172.21.15.18:0/1299296749 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc003fa0 con 0x55d1688ded50 2024-04-16T17:23:38.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.137+0000 7fb1ae600640 1 -- 172.21.15.18:0/1299296749 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc01f040 con 0x55d1688ded50 2024-04-16T17:23:38.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.138+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1299296749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688ded50 msgr2=0x55d1688e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.138+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688ded50 0x55d1688e7a10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc012530 tx=0x7fb1bc012470 comp rx=0 tx=0).stop 2024-04-16T17:23:38.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.139+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1299296749 shutdown_connections 2024-04-16T17:23:38.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.139+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688e8250 0x55d168908c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.139+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 0x55d1688f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.139+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1299296749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688ded50 0x55d1688e7a10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.139+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1299296749 >> 172.21.15.18:0/1299296749 conn(0x55d1688d9480 msgr2=0x55d1688e5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:38.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.142+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1299296749 shutdown_connections 2024-04-16T17:23:38.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.142+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1299296749 wait complete. 2024-04-16T17:23:38.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.142+0000 7fb1cb78e740 1 -- start start 2024-04-16T17:23:38.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.143+0000 7fb1cb78e740 1 Processor -- start 2024-04-16T17:23:38.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.143+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688ded50 0x55d1688e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.143+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 0x55d1688e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.143+0000 7fb1cb78e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 0x55d1688e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.143+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a400e7e0 con 0x55d1688df670 2024-04-16T17:23:38.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.143+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a400fc50 con 0x55d1688ded50 2024-04-16T17:23:38.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.143+0000 7fb1cb78e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a400eeb0 con 0x55d1688e8250 2024-04-16T17:23:38.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.143+0000 7fb1cb78e740 10 client.0 _ll_register_callbacks cb 0x7fb1b0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:38.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688ded50 0x55d1688e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688ded50 0x55d1688e2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53658/0 (socket says 172.21.15.18:53658) 2024-04-16T17:23:38.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c2800640 1 -- 172.21.15.18:0/1883509827 learned_addr learned my addr 172.21.15.18:0/1883509827 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:38.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 0x55d1688e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c2800640 1 -- 172.21.15.18:0/1883509827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 msgr2=0x55d1688e2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:38.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 0x55d1688e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c2800640 1 -- 172.21.15.18:0/1883509827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 msgr2=0x55d1688e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:38.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 0x55d1688e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.144+0000 7fb1c2800640 1 -- 172.21.15.18:0/1883509827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1685563f0 con 0x55d1688ded50 2024-04-16T17:23:38.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.145+0000 7fb1c2800640 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688ded50 0x55d1688e2220 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009dd0 tx=0x7fb1bc003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5cf7553170520db server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:38.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.145+0000 7fb1a3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:38.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.145+0000 7fb1a3400640 1 -- 172.21.15.18:0/1883509827 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc039060 con 0x55d1688ded50 2024-04-16T17:23:38.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.145+0000 7fb1a3400640 1 -- 172.21.15.18:0/1883509827 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9566+0+0 (secure 0 0 0) 0x7fb1bc004150 con 0x55d1688ded50 2024-04-16T17:23:38.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.146+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1883509827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1bc003fa0 con 0x55d1688ded50 2024-04-16T17:23:38.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.146+0000 7fb1a3400640 1 -- 172.21.15.18:0/1883509827 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb1bc030050 con 0x55d1688ded50 2024-04-16T17:23:38.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.146+0000 7fb1cb78e740 10 client.30509 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:38.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.146+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1883509827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d1686207d0 con 0x55d1688ded50 2024-04-16T17:23:38.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.146+0000 7fb1a3400640 1 -- 172.21.15.18:0/1883509827 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc003fa0 con 0x55d1688ded50 2024-04-16T17:23:38.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.146+0000 7fb1a3400640 1 -- 172.21.15.18:0/1883509827 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fb1bc025040 con 0x55d1688ded50 2024-04-16T17:23:38.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.147+0000 7fb1cb78e740 10 client.30509 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:38.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.147+0000 7fb1cb78e740 10 client.30509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:38.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.147+0000 7fb1cb78e740 20 client.30509 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:38.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.147+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d168967a80 0x55d168969e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:38.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.147+0000 7fb1cb78e740 4 client.30509 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:23:38.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.147+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1883509827 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d16884b8b0 con 0x55d168967a80 2024-04-16T17:23:38.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.148+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d168967a80 0x55d168969e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:38.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.148+0000 7fb1c1400640 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d168967a80 0x55d168969e70 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b91273e7f20e02 server_cookie=df6bbb0035d8c54f in_seq=0 out_seq=0 2024-04-16T17:23:38.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.148+0000 7fb1a3400640 10 client.30509 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:38.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.152+0000 7fb1a3400640 1 -- 172.21.15.18:0/1883509827 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d16884b8b0 con 0x55d168967a80 2024-04-16T17:23:38.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.152+0000 7fb1a3400640 10 client.30509 handle_command_reply: tid=0 2024-04-16T17:23:38.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.152+0000 7fb1cb78e740 1 client.30509 shutdown 2024-04-16T17:23:38.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.152+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1883509827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688ded50 msgr2=0x55d1688e2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.152+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688ded50 0x55d1688e2220 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009dd0 tx=0x7fb1bc003d40 comp rx=0 tx=0).stop 2024-04-16T17:23:38.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1883509827 shutdown_connections 2024-04-16T17:23:38.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d168967a80 0x55d168969e70 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1688e8250 0x55d1688e2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1688df670 0x55d1688e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1cb78e740 1 --2- 172.21.15.18:0/1883509827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1688ded50 0x55d1688e2220 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1883509827 >> 172.21.15.18:0/1883509827 conn(0x55d1688d9480 msgr2=0x55d1688d7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1a3400640 0 client.30509 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1883509827 shutdown_connections 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.154+0000 7fb1cb78e740 1 -- 172.21.15.18:0/1883509827 wait complete. 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.155+0000 7fb1cb78e740 20 client.30509 trim_cache size 0 max 16384 2024-04-16T17:23:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:23:38.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 -- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1a80a3da0 msgr2=0x7fb1a80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1a80a3da0 0x7fb1a80a6260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55d1688de820 tx=0x7fb1b40035f0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 -- 172.21.15.18:0/1717289255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 msgr2=0x7fb1c41a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:38.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 0x7fb1c41a0840 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb1b402c240 tx=0x7fb1b40042f0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 -- 172.21.15.18:0/1717289255 shutdown_connections 2024-04-16T17:23:38.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb1a80a3da0 0x7fb1a80a6260 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1c410e9a0 0x7fb1c419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1c4101710 0x7fb1c41a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 --2- 172.21.15.18:0/1717289255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c4100df0 0x7fb1c41a0840 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 -- 172.21.15.18:0/1717289255 >> 172.21.15.18:0/1717289255 conn(0x7fb1c40780b0 msgr2=0x7fb1c410fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 -- 172.21.15.18:0/1717289255 shutdown_connections 2024-04-16T17:23:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:38.156+0000 7fb1c8a00640 1 -- 172.21.15.18:0/1717289255 wait complete. 2024-04-16T17:23:38.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:38 smithi022 ceph-mon[36023]: pgmap v603: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-04-16T17:23:38.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:38 smithi043 ceph-mon[36901]: pgmap v603: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-04-16T17:23:38.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 82451 11.82 MB/sec execute 230 sec latency 1543.506 ms 2024-04-16T17:23:39.177 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:23:39.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:23:39.178 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:23:39.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.503+0000 7f8fbe600640 1 --2- 172.21.15.18:0/237333092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 0x7f8fb8115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:39.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.504+0000 7f8fc0600640 1 -- 172.21.15.18:0/237333092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb8076530 msgr2=0x7f8fb8076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.504+0000 7f8fc0600640 1 --2- 172.21.15.18:0/237333092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb8076530 0x7f8fb8076910 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009930 tx=0x7f8fac02bc40 comp rx=0 tx=0).stop 2024-04-16T17:23:39.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.506+0000 7f8fc0600640 1 -- 172.21.15.18:0/237333092 shutdown_connections 2024-04-16T17:23:39.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.506+0000 7f8fc0600640 1 --2- 172.21.15.18:0/237333092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 0x7f8fb8115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.506+0000 7f8fc0600640 1 --2- 172.21.15.18:0/237333092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb8076e50 0x7f8fb8112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.506+0000 7f8fc0600640 1 --2- 172.21.15.18:0/237333092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb8076530 0x7f8fb8076910 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.506+0000 7f8fc0600640 1 -- 172.21.15.18:0/237333092 >> 172.21.15.18:0/237333092 conn(0x7f8fb80ffc80 msgr2=0x7f8fb81020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:39.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.507+0000 7f8fc0600640 1 -- 172.21.15.18:0/237333092 shutdown_connections 2024-04-16T17:23:39.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.507+0000 7f8fc0600640 1 -- 172.21.15.18:0/237333092 wait complete. 2024-04-16T17:23:39.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.508+0000 7f8fc0600640 1 Processor -- start 2024-04-16T17:23:39.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.509+0000 7f8fc0600640 1 -- start start 2024-04-16T17:23:39.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.509+0000 7f8fc0600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb8076530 0x7f8fb819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.509+0000 7f8fc0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb8076e50 0x7f8fb819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.509+0000 7f8fc0600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 0x7f8fb81a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.509+0000 7f8fc0600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8118ac0 con 0x7f8fb8076530 2024-04-16T17:23:39.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.509+0000 7f8fc0600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8118940 con 0x7f8fb8112e70 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.509+0000 7f8fc0600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8118c40 con 0x7f8fb8076e50 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbe600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 0x7f8fb81a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbe600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 0x7f8fb81a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53674/0 (socket says 172.21.15.18:53674) 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbe600640 1 -- 172.21.15.18:0/4015212364 learned_addr learned my addr 172.21.15.18:0/4015212364 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb8076530 0x7f8fb819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbd200640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb8076e50 0x7f8fb819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbe600640 1 -- 172.21.15.18:0/4015212364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb8076e50 msgr2=0x7f8fb819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbe600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb8076e50 0x7f8fb819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbe600640 1 -- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb8076530 msgr2=0x7f8fb819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbe600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb8076530 0x7f8fb819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.510+0000 7f8fbe600640 1 -- 172.21.15.18:0/4015212364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb81a3fc0 con 0x7f8fb8112e70 2024-04-16T17:23:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.511+0000 7f8fbe600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 0x7f8fb81a3890 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000ccb0 tx=0x7f8fb000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.511+0000 7f8fb6000640 1 -- 172.21.15.18:0/4015212364 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000f450 con 0x7f8fb8112e70 2024-04-16T17:23:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.511+0000 7f8fc0600640 1 -- 172.21.15.18:0/4015212364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81a42b0 con 0x7f8fb8112e70 2024-04-16T17:23:39.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.511+0000 7f8fb6000640 1 -- 172.21.15.18:0/4015212364 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb000fad0 con 0x7f8fb8112e70 2024-04-16T17:23:39.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.511+0000 7f8fb6000640 1 -- 172.21.15.18:0/4015212364 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0018ac0 con 0x7f8fb8112e70 2024-04-16T17:23:39.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.511+0000 7f8fc0600640 1 -- 172.21.15.18:0/4015212364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81abb40 con 0x7f8fb8112e70 2024-04-16T17:23:39.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.520+0000 7f8fb6000640 1 -- 172.21.15.18:0/4015212364 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8fb00213f0 con 0x7f8fb8112e70 2024-04-16T17:23:39.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.520+0000 7f8fc33c2740 1 Processor -- start 2024-04-16T17:23:39.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.520+0000 7f8fc33c2740 1 -- start start 2024-04-16T17:23:39.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.521+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61329080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.521+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613295c0 0x55da61331c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.521+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 0x55da61334590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.521+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55da6128dcb0 con 0x55da613321a0 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.521+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55da61234370 con 0x55da61328ca0 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.521+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da61289c70 con 0x55da613295c0 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.521+0000 7f8fbd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61329080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fbdc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613295c0 0x55da61331c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fbdc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613295c0 0x55da61331c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51536/0 (socket says 172.21.15.18:51536) 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fbdc00640 1 -- 172.21.15.18:0/2815151926 learned_addr learned my addr 172.21.15.18:0/2815151926 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fb6000640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8fa00a3e20 0x7f8fa00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fb6000640 1 -- 172.21.15.18:0/4015212364 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f8fb00d0f20 con 0x7f8fb8112e70 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fbdc00640 1 -- 172.21.15.18:0/2815151926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 msgr2=0x55da61329080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:39.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2815151926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61329080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fbdc00640 1 -- 172.21.15.18:0/2815151926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 msgr2=0x55da61334590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2815151926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 0x55da61334590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.522+0000 7f8fbdc00640 1 -- 172.21.15.18:0/2815151926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da61253910 con 0x55da613295c0 2024-04-16T17:23:39.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.523+0000 7f8fbe600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8fa00a3e20 0x7f8fa00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.524+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2815151926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613295c0 0x55da61331c60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8fac02c250 tx=0x7f8fac02c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f1470e971b1881f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.524+0000 7f8faaa00640 1 -- 172.21.15.18:0/2815151926 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac053070 con 0x55da613295c0 2024-04-16T17:23:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.525+0000 7f8faaa00640 1 -- 172.21.15.18:0/2815151926 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fac004400 con 0x55da613295c0 2024-04-16T17:23:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.525+0000 7f8faaa00640 1 -- 172.21.15.18:0/2815151926 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac004920 con 0x55da613295c0 2024-04-16T17:23:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.526+0000 7f8fbe600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8fa00a3e20 0x7f8fa00a62e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0098ec0 tx=0x7f8fb00077e0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.528+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2815151926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613295c0 msgr2=0x55da61331c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.528+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2815151926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613295c0 0x55da61331c60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8fac02c250 tx=0x7f8fac02c730 comp rx=0 tx=0).stop 2024-04-16T17:23:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.529+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2815151926 shutdown_connections 2024-04-16T17:23:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.529+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2815151926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 0x55da61334590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.529+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2815151926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613295c0 0x55da61331c60 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.529+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2815151926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61329080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.529+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2815151926 >> 172.21.15.18:0/2815151926 conn(0x55da613233d0 msgr2=0x55da613237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.529+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2815151926 shutdown_connections 2024-04-16T17:23:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.529+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2815151926 wait complete. 2024-04-16T17:23:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.530+0000 7f8fc33c2740 1 -- start start 2024-04-16T17:23:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.530+0000 7f8fc33c2740 1 Processor -- start 2024-04-16T17:23:39.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da61328ca0 0x55da6132f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 0x55da6132fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55da61334f30 con 0x55da613321a0 2024-04-16T17:23:39.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55da61247730 con 0x55da613295c0 2024-04-16T17:23:39.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da613350b0 con 0x55da61328ca0 2024-04-16T17:23:39.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fbd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da61328ca0 0x55da6132f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fbd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da61328ca0 0x55da6132f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51548/0 (socket says 172.21.15.18:51548) 2024-04-16T17:23:39.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fbd200640 1 -- 172.21.15.18:0/2160227841 learned_addr learned my addr 172.21.15.18:0/2160227841 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:39.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fbd200640 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 0x55da6132fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.531+0000 7f8fc33c2740 10 client.0 _ll_register_callbacks cb 0x55da612161e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:39.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.532+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.532+0000 7f8fbd200640 1 -- 172.21.15.18:0/2160227841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da61328ca0 msgr2=0x55da6132f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:39.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.532+0000 7f8fbd200640 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da61328ca0 0x55da6132f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.532+0000 7f8fbd200640 1 -- 172.21.15.18:0/2160227841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 msgr2=0x55da6132f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:39.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.532+0000 7f8fbd200640 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.532+0000 7f8fbd200640 1 -- 172.21.15.18:0/2160227841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da610687e0 con 0x55da613321a0 2024-04-16T17:23:39.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.532+0000 7f8fbd200640 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 0x55da6132fbe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4018eb0 tx=0x7f8fa40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cf7921b8f83183 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:39.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.533+0000 7f8f9f400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:23:39.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.534+0000 7f8f9f400640 1 -- 172.21.15.18:0/2160227841 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa401d060 con 0x55da613321a0 2024-04-16T17:23:39.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.534+0000 7f8f9f400640 1 -- 172.21.15.18:0/2160227841 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f8fa4005bc0 con 0x55da613321a0 2024-04-16T17:23:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.534+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2160227841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fac004400 con 0x55da613321a0 2024-04-16T17:23:39.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.536+0000 7f8f9f400640 1 -- 172.21.15.18:0/2160227841 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8fa4005dc0 con 0x55da613321a0 2024-04-16T17:23:39.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.536+0000 7f8fc33c2740 10 client.19650 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.536+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2160227841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55da610c8f70 con 0x55da613321a0 2024-04-16T17:23:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.537+0000 7f8f9f400640 1 -- 172.21.15.18:0/2160227841 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fac004400 con 0x55da613321a0 2024-04-16T17:23:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.537+0000 7f8f9f400640 1 -- 172.21.15.18:0/2160227841 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f8fa4016430 con 0x55da613321a0 2024-04-16T17:23:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.537+0000 7f8fc33c2740 10 client.19650 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.537+0000 7f8fc33c2740 10 client.19650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.537+0000 7f8fc33c2740 20 client.19650 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.537+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55da613424a0 0x55da61344890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.537+0000 7f8fc33c2740 4 client.19650 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.537+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2160227841 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da6129b6d0 con 0x55da613424a0 2024-04-16T17:23:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.539+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55da613424a0 0x55da61344890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.543+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55da613424a0 0x55da61344890 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=646235ebc5fcc7c9 server_cookie=2db91fa672b60bd4 in_seq=0 out_seq=0 2024-04-16T17:23:39.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.543+0000 7f8f9f400640 10 client.19650 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:39.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.548+0000 7f8f9f400640 1 -- 172.21.15.18:0/2160227841 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da6129b6d0 con 0x55da613424a0 2024-04-16T17:23:39.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.549+0000 7f8f9f400640 10 client.19650 handle_command_reply: tid=0 2024-04-16T17:23:39.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.549+0000 7f8fc33c2740 1 client.19650 shutdown 2024-04-16T17:23:39.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.549+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2160227841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 msgr2=0x55da6132fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.549+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 0x55da6132fbe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4018eb0 tx=0x7f8fa40047d0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.549+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2160227841 shutdown_connections 2024-04-16T17:23:39.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.549+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55da613424a0 0x55da61344890 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.549+0000 7f8f9f400640 0 client.19650 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:39.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.550+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613321a0 0x55da6132fbe0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.550+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.550+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2160227841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da61328ca0 0x55da6132f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.550+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2160227841 >> 172.21.15.18:0/2160227841 conn(0x55da613233d0 msgr2=0x55da61325120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:39.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.550+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2160227841 shutdown_connections 2024-04-16T17:23:39.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.550+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2160227841 wait complete. 2024-04-16T17:23:39.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.550+0000 7f8fc33c2740 20 client.19650 trim_cache size 0 max 16384 2024-04-16T17:23:39.552 INFO:tasks.workunit.client.0.smithi018.stdout: 10 82774 11.83 MB/sec execute 231 sec latency 2248.853 ms 2024-04-16T17:23:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.593+0000 7f8fc33c2740 1 Processor -- start 2024-04-16T17:23:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.594+0000 7f8fc33c2740 1 -- start start 2024-04-16T17:23:39.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.594+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61331890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613295c0 0x55da61322430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 0x55da6132fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8f980128f0 con 0x55da613295c0 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8f980101c0 con 0x55da61328ca0 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9800e030 con 0x55da613321a0 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fbd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61331890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fbd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61331890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53698/0 (socket says 172.21.15.18:53698) 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fbd200640 1 -- 172.21.15.18:0/1029516930 learned_addr learned my addr 172.21.15.18:0/1029516930 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:39.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.595+0000 7f8fbd200640 1 --2- 172.21.15.18:0/1029516930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 0x55da6132fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.596+0000 7f8fbd200640 1 -- 172.21.15.18:0/1029516930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 msgr2=0x55da6132fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.596+0000 7f8fbd200640 1 --2- 172.21.15.18:0/1029516930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 0x55da6132fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.596+0000 7f8fbd200640 1 -- 172.21.15.18:0/1029516930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613295c0 msgr2=0x55da61322430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.596+0000 7f8fbd200640 1 --2- 172.21.15.18:0/1029516930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613295c0 0x55da61322430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.596+0000 7f8fbd200640 1 -- 172.21.15.18:0/1029516930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da61329f70 con 0x55da61328ca0 2024-04-16T17:23:39.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.596+0000 7f8fbd200640 1 --2- 172.21.15.18:0/1029516930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61331890 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4015010 tx=0x7f8fa4018b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1d7f568eb1c89f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:39.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.597+0000 7f8faa000640 1 -- 172.21.15.18:0/1029516930 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa401d040 con 0x55da61328ca0 2024-04-16T17:23:39.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.597+0000 7f8faa000640 1 -- 172.21.15.18:0/1029516930 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa40093a0 con 0x55da61328ca0 2024-04-16T17:23:39.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.597+0000 7f8faa000640 1 -- 172.21.15.18:0/1029516930 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4027080 con 0x55da61328ca0 2024-04-16T17:23:39.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.599+0000 7f8fc33c2740 1 -- 172.21.15.18:0/1029516930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 msgr2=0x55da61331890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.600+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/1029516930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61331890 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4015010 tx=0x7f8fa4018b40 comp rx=0 tx=0).stop 2024-04-16T17:23:39.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.601+0000 7f8fc33c2740 1 -- 172.21.15.18:0/1029516930 shutdown_connections 2024-04-16T17:23:39.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.601+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/1029516930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 0x55da6132fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.601+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/1029516930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da613295c0 0x55da61322430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.601+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/1029516930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da61328ca0 0x55da61331890 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.601+0000 7f8fc33c2740 1 -- 172.21.15.18:0/1029516930 >> 172.21.15.18:0/1029516930 conn(0x55da613233d0 msgr2=0x55da61324600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:39.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.601+0000 7f8fc33c2740 1 -- 172.21.15.18:0/1029516930 shutdown_connections 2024-04-16T17:23:39.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.602+0000 7f8fc33c2740 1 -- 172.21.15.18:0/1029516930 wait complete. 2024-04-16T17:23:39.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.603+0000 7f8fc33c2740 1 -- start start 2024-04-16T17:23:39.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.603+0000 7f8fc33c2740 1 Processor -- start 2024-04-16T17:23:39.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.604+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da61328ca0 0x55da6132c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.604+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.604+0000 7f8fc33c2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 0x55da613a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.604+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9800e680 con 0x55da61328ca0 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.604+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9800faf0 con 0x55da613295c0 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.604+0000 7f8fc33c2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9800ed50 con 0x55da613321a0 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.604+0000 7f8fc33c2740 10 client.0 _ll_register_callbacks cb 0x7f8fa4018a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.605+0000 7f8fbdc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.605+0000 7f8fbdc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53708/0 (socket says 172.21.15.18:53708) 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.605+0000 7f8fbdc00640 1 -- 172.21.15.18:0/2285479961 learned_addr learned my addr 172.21.15.18:0/2285479961 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.606+0000 7f8fbdc00640 1 -- 172.21.15.18:0/2285479961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 msgr2=0x55da613a7a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.606+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 0x55da613a7a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.606+0000 7f8fbdc00640 1 -- 172.21.15.18:0/2285479961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da61328ca0 msgr2=0x55da6132c1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:39.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.606+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da61328ca0 0x55da6132c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.606+0000 7f8fbdc00640 1 -- 172.21.15.18:0/2285479961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da60f60170 con 0x55da613295c0 2024-04-16T17:23:39.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.607+0000 7f8fbdc00640 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132c710 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8fac02bc80 tx=0x7f8fac005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f035a9e365764ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:39.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.607+0000 7f8f9f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:39.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.607+0000 7f8f9f400640 1 -- 172.21.15.18:0/2285479961 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac066060 con 0x55da613295c0 2024-04-16T17:23:39.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.607+0000 7f8f9f400640 1 -- 172.21.15.18:0/2285479961 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f8fac02cc50 con 0x55da613295c0 2024-04-16T17:23:39.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.607+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2285479961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa40093a0 con 0x55da613295c0 2024-04-16T17:23:39.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.608+0000 7f8f9f400640 1 -- 172.21.15.18:0/2285479961 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8fac053040 con 0x55da613295c0 2024-04-16T17:23:39.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.608+0000 7f8fc33c2740 10 client.30521 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:39.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.608+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2285479961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55da613a8f30 con 0x55da613295c0 2024-04-16T17:23:39.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.609+0000 7f8f9f400640 1 -- 172.21.15.18:0/2285479961 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa40093a0 con 0x55da613295c0 2024-04-16T17:23:39.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.609+0000 7f8f9f400640 1 -- 172.21.15.18:0/2285479961 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f8fac058450 con 0x55da613295c0 2024-04-16T17:23:39.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.609+0000 7f8fc33c2740 10 client.30521 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:39.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.609+0000 7f8fc33c2740 10 client.30521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:39.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.609+0000 7f8fc33c2740 20 client.30521 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:39.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.609+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55da613424a0 0x55da613ab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:39.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.609+0000 7f8fc33c2740 4 client.30521 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:23:39.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.609+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2285479961 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da61295540 con 0x55da613424a0 2024-04-16T17:23:39.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.646+0000 7f8fbd200640 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55da613424a0 0x55da613ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:39.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.647+0000 7f8fbd200640 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55da613424a0 0x55da613ab280 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=571f92c2d73b9555 server_cookie=f679531b549ce768 in_seq=0 out_seq=0 2024-04-16T17:23:39.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.647+0000 7f8f9f400640 10 client.30521 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:39.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.654+0000 7f8f9f400640 1 -- 172.21.15.18:0/2285479961 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da61295540 con 0x55da613424a0 2024-04-16T17:23:39.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.654+0000 7f8f9f400640 10 client.30521 handle_command_reply: tid=0 2024-04-16T17:23:39.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.654+0000 7f8fc33c2740 1 client.30521 shutdown 2024-04-16T17:23:39.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.654+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2285479961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 msgr2=0x55da6132c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.654+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132c710 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8fac02bc80 tx=0x7f8fac005fd0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.654+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2285479961 shutdown_connections 2024-04-16T17:23:39.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.654+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55da613424a0 0x55da613ab280 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.655+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da613321a0 0x55da613a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.655+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da613295c0 0x55da6132c710 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.655+0000 7f8fc33c2740 1 --2- 172.21.15.18:0/2285479961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da61328ca0 0x55da6132c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.655+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2285479961 >> 172.21.15.18:0/2285479961 conn(0x55da613233d0 msgr2=0x55da613222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:39.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.655+0000 7f8f9f400640 0 client.30521 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:39.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.655+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2285479961 shutdown_connections 2024-04-16T17:23:39.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.655+0000 7f8fc33c2740 1 -- 172.21.15.18:0/2285479961 wait complete. 2024-04-16T17:23:39.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.656+0000 7f8fc33c2740 20 client.30521 trim_cache size 0 max 16384 2024-04-16T17:23:39.657 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:23:39.657 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:23:39.657 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "3a1e6ae0-f683-47a2-9838-7a42e9f0bbd3", 2024-04-16T17:23:39.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:23:39.657 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:23:39.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.659+0000 7f8fc0600640 1 -- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8fa00a3e20 msgr2=0x7f8fa00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.659+0000 7f8fc0600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8fa00a3e20 0x7f8fa00a62e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0098ec0 tx=0x7f8fb00077e0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.659+0000 7f8fc0600640 1 -- 172.21.15.18:0/4015212364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 msgr2=0x7f8fb81a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.659+0000 7f8fc0600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 0x7f8fb81a3890 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000ccb0 tx=0x7f8fb000a620 comp rx=0 tx=0).stop 2024-04-16T17:23:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.660+0000 7f8fc0600640 1 -- 172.21.15.18:0/4015212364 shutdown_connections 2024-04-16T17:23:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.660+0000 7f8fc0600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8fa00a3e20 0x7f8fa00a62e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.660+0000 7f8fc0600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fb8112e70 0x7f8fb81a3890 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.660+0000 7f8fc0600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb8076e50 0x7f8fb819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.660+0000 7f8fc0600640 1 --2- 172.21.15.18:0/4015212364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8fb8076530 0x7f8fb819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.660+0000 7f8fc0600640 1 -- 172.21.15.18:0/4015212364 >> 172.21.15.18:0/4015212364 conn(0x7f8fb80ffc80 msgr2=0x7f8fb8114530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.660+0000 7f8fc0600640 1 -- 172.21.15.18:0/4015212364 shutdown_connections 2024-04-16T17:23:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.660+0000 7f8fc0600640 1 -- 172.21.15.18:0/4015212364 wait complete. 2024-04-16T17:23:39.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:23:39.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:23:39.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.999+0000 7fd1e0000640 1 -- 172.21.15.18:0/4086626882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d811b7e0 msgr2=0x7fd1d811dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:39.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.999+0000 7fd1e0000640 1 --2- 172.21.15.18:0/4086626882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d811b7e0 0x7fd1d811dbd0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009900 tx=0x7fd1c402bc50 comp rx=0 tx=0).stop 2024-04-16T17:23:39.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.999+0000 7fd1e0000640 1 -- 172.21.15.18:0/4086626882 shutdown_connections 2024-04-16T17:23:40.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.999+0000 7fd1e0000640 1 --2- 172.21.15.18:0/4086626882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d811b7e0 0x7fd1d811dbd0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.999+0000 7fd1e0000640 1 --2- 172.21.15.18:0/4086626882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1d8110bc0 0x7fd1d810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:39.999+0000 7fd1e0000640 1 --2- 172.21.15.18:0/4086626882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1d8110750 0x7fd1d810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.000+0000 7fd1e0000640 1 -- 172.21.15.18:0/4086626882 >> 172.21.15.18:0/4086626882 conn(0x7fd1d806d0a0 msgr2=0x7fd1d806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:39 smithi043 ceph-mon[36901]: osdmap e129: 12 total, 12 up, 12 in 2024-04-16T17:23:40.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:39 smithi043 ceph-mon[36901]: pgmap v605: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 161 op/s 2024-04-16T17:23:40.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.001+0000 7fd1e0000640 1 -- 172.21.15.18:0/4086626882 shutdown_connections 2024-04-16T17:23:40.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.001+0000 7fd1e0000640 1 -- 172.21.15.18:0/4086626882 wait complete. 2024-04-16T17:23:40.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.003+0000 7fd1e0000640 1 Processor -- start 2024-04-16T17:23:40.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.003+0000 7fd1e0000640 1 -- start start 2024-04-16T17:23:40.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.004+0000 7fd1e0000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1d8110750 0x7fd1d81a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.004+0000 7fd1e0000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1d8110bc0 0x7fd1d81a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.005+0000 7fd1e0000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d811b7e0 0x7fd1d81abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.005+0000 7fd1dd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1d8110750 0x7fd1d81a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.005+0000 7fd1dd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1d8110750 0x7fd1d81a76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53718/0 (socket says 172.21.15.18:53718) 2024-04-16T17:23:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.005+0000 7fd1dd600640 1 -- 172.21.15.18:0/3907671759 learned_addr learned my addr 172.21.15.18:0/3907671759 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.005+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81212c0 con 0x7fd1d8110bc0 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.005+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8121140 con 0x7fd1d8110750 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.005+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8121440 con 0x7fd1d811b7e0 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.005+0000 7fd1dcc00640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1d8110bc0 0x7fd1d81a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.006+0000 7fd1dcc00640 1 -- 172.21.15.18:0/3907671759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d811b7e0 msgr2=0x7fd1d81abf80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.006+0000 7fd1dcc00640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d811b7e0 0x7fd1d81abf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.006+0000 7fd1dcc00640 1 -- 172.21.15.18:0/3907671759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1d8110750 msgr2=0x7fd1d81a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.006+0000 7fd1dcc00640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1d8110750 0x7fd1d81a76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.006+0000 7fd1dcc00640 1 -- 172.21.15.18:0/3907671759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d81ac6b0 con 0x7fd1d8110bc0 2024-04-16T17:23:40.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.006+0000 7fd1dd600640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1d8110750 0x7fd1d81a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:40.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.007+0000 7fd1dcc00640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1d8110bc0 0x7fd1d81a7c00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009fd0 tx=0x7fd1cc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.011+0000 7fd1d6000640 1 -- 172.21.15.18:0/3907671759 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc015020 con 0x7fd1d8110bc0 2024-04-16T17:23:40.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.012+0000 7fd1d6000640 1 -- 172.21.15.18:0/3907671759 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1cc011d10 con 0x7fd1d8110bc0 2024-04-16T17:23:40.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.012+0000 7fd1d6000640 1 -- 172.21.15.18:0/3907671759 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc015020 con 0x7fd1d8110bc0 2024-04-16T17:23:40.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.013+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d81ac9a0 con 0x7fd1d8110bc0 2024-04-16T17:23:40.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.013+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d81b4230 con 0x7fd1d8110bc0 2024-04-16T17:23:40.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.018+0000 7fd1d6000640 1 -- 172.21.15.18:0/3907671759 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd1cc018020 con 0x7fd1d8110bc0 2024-04-16T17:23:40.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.021+0000 7fd1d6000640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd1c00a3fd0 0x7fd1c00a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.021+0000 7fd1d6000640 1 -- 172.21.15.18:0/3907671759 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7fd1cc0cfe80 con 0x7fd1d8110bc0 2024-04-16T17:23:40.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.022+0000 7fd1e2da6740 1 Processor -- start 2024-04-16T17:23:40.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.022+0000 7fd1e2da6740 1 -- start start 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.022+0000 7fd1dd600640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd1c00a3fd0 0x7fd1c00a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.022+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45923590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.022+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4592c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.022+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 0x55ab4592eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.023+0000 7fd1dd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45923590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.023+0000 7fd1e2da6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ab45889120 con 0x55ab4592c6b0 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.023+0000 7fd1e2da6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ab4582f340 con 0x55ab459231b0 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.023+0000 7fd1e2da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ab458850e0 con 0x55ab45923ad0 2024-04-16T17:23:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.023+0000 7fd1dcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4592c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.023+0000 7fd1de000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 0x55ab4592eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.023+0000 7fd1dcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4592c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51582/0 (socket says 172.21.15.18:51582) 2024-04-16T17:23:40.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.023+0000 7fd1dcc00640 1 -- 172.21.15.18:0/1039084663 learned_addr learned my addr 172.21.15.18:0/1039084663 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.024+0000 7fd1de000640 1 -- 172.21.15.18:0/1039084663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 msgr2=0x55ab4592c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.024+0000 7fd1de000640 1 --2- 172.21.15.18:0/1039084663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4592c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.024+0000 7fd1de000640 1 -- 172.21.15.18:0/1039084663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 msgr2=0x55ab45923590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.024+0000 7fd1de000640 1 --2- 172.21.15.18:0/1039084663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45923590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.024+0000 7fd1de000640 1 -- 172.21.15.18:0/1039084663 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab456657d0 con 0x55ab4592c6b0 2024-04-16T17:23:40.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.028+0000 7fd1dd600640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd1c00a3fd0 0x7fd1c00a6490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0010eb0 tx=0x7fd1d0010160 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.029+0000 7fd1de000640 1 --2- 172.21.15.18:0/1039084663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 0x55ab4592eaa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4036040 tx=0x7fd1c40095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73dcb97ef0bb5acd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.030+0000 7fd1ca600640 1 -- 172.21.15.18:0/1039084663 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4041070 con 0x55ab4592c6b0 2024-04-16T17:23:40.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.032+0000 7fd1ca600640 1 -- 172.21.15.18:0/1039084663 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c402caa0 con 0x55ab4592c6b0 2024-04-16T17:23:40.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.032+0000 7fd1ca600640 1 -- 172.21.15.18:0/1039084663 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40046c0 con 0x55ab4592c6b0 2024-04-16T17:23:40.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.033+0000 7fd1e2da6740 1 -- 172.21.15.18:0/1039084663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 msgr2=0x55ab4592eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.033+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/1039084663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 0x55ab4592eaa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4036040 tx=0x7fd1c40095d0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.033+0000 7fd1e2da6740 1 -- 172.21.15.18:0/1039084663 shutdown_connections 2024-04-16T17:23:40.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.033+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/1039084663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 0x55ab4592eaa0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.033+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/1039084663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4592c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.033+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/1039084663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45923590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.033+0000 7fd1e2da6740 1 -- 172.21.15.18:0/1039084663 >> 172.21.15.18:0/1039084663 conn(0x55ab4591d8e0 msgr2=0x55ab4591dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.034+0000 7fd1e2da6740 1 -- 172.21.15.18:0/1039084663 shutdown_connections 2024-04-16T17:23:40.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.035+0000 7fd1e2da6740 1 -- 172.21.15.18:0/1039084663 wait complete. 2024-04-16T17:23:40.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.035+0000 7fd1e2da6740 1 -- start start 2024-04-16T17:23:40.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.035+0000 7fd1e2da6740 1 Processor -- start 2024-04-16T17:23:40.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.038+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45929670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.038+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 0x55ab45929bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.038+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab4592c6b0 0x55ab4592a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.041+0000 7fd1e2da6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ab4592f440 con 0x55ab45923ad0 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.041+0000 7fd1de000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab4592c6b0 0x55ab4592a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.041+0000 7fd1de000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab4592c6b0 0x55ab4592a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51584/0 (socket says 172.21.15.18:51584) 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.041+0000 7fd1de000640 1 -- 172.21.15.18:0/2282392607 learned_addr learned my addr 172.21.15.18:0/2282392607 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ab45842710 con 0x55ab459231b0 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ab4592f5c0 con 0x55ab4592c6b0 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1dd600640 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45929670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1e2da6740 10 client.0 _ll_register_callbacks cb 0x55ab4583daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1de000640 1 -- 172.21.15.18:0/2282392607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 msgr2=0x55ab45929670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1de000640 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45929670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1de000640 1 -- 172.21.15.18:0/2282392607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 msgr2=0x55ab45929bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1de000640 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 0x55ab45929bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.042+0000 7fd1de000640 1 -- 172.21.15.18:0/2282392607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab4559b3f0 con 0x55ab4592c6b0 2024-04-16T17:23:40.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.043+0000 7fd1de000640 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab4592c6b0 0x55ab4592a0f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009850 tx=0x7fd1c4003880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d0b12162f7b13d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.043+0000 7fd1bea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:40.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.044+0000 7fd1bea00640 1 -- 172.21.15.18:0/2282392607 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c404d020 con 0x55ab4592c6b0 2024-04-16T17:23:40.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.044+0000 7fd1bea00640 1 -- 172.21.15.18:0/2282392607 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7fd1c402c8a0 con 0x55ab4592c6b0 2024-04-16T17:23:40.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.045+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c402caa0 con 0x55ab4592c6b0 2024-04-16T17:23:40.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.045+0000 7fd1bea00640 1 -- 172.21.15.18:0/2282392607 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd1c4041070 con 0x55ab4592c6b0 2024-04-16T17:23:40.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.046+0000 7fd1e2da6740 10 client.30382 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:40.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.046+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ab4555b0f0 con 0x55ab4592c6b0 2024-04-16T17:23:40.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.046+0000 7fd1bea00640 1 -- 172.21.15.18:0/2282392607 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c402caa0 con 0x55ab4592c6b0 2024-04-16T17:23:40.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.047+0000 7fd1bea00640 1 -- 172.21.15.18:0/2282392607 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fd1c4046450 con 0x55ab4592c6b0 2024-04-16T17:23:40.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.047+0000 7fd1e2da6740 10 client.30382 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:40.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.047+0000 7fd1e2da6740 10 client.30382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:40.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.047+0000 7fd1e2da6740 20 client.30382 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:40.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.048+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab4593c5d0 0x55ab4593e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.048+0000 7fd1e2da6740 4 client.30382 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:40.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.048+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab45896b40 con 0x55ab4593c5d0 2024-04-16T17:23:40.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.048+0000 7fd1dd600640 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab4593c5d0 0x55ab4593e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.049+0000 7fd1dd600640 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab4593c5d0 0x55ab4593e9c0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1b531f877295d1 server_cookie=f7854f95444651bf in_seq=0 out_seq=0 2024-04-16T17:23:40.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.050+0000 7fd1bea00640 10 client.30382 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:40.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.051+0000 7fd1bea00640 1 -- 172.21.15.18:0/2282392607 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab45896b40 con 0x55ab4593c5d0 2024-04-16T17:23:40.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.052+0000 7fd1bea00640 10 client.30382 handle_command_reply: tid=0 2024-04-16T17:23:40.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.052+0000 7fd1e2da6740 1 client.30382 shutdown 2024-04-16T17:23:40.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.052+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab4592c6b0 msgr2=0x55ab4592a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.052+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab4592c6b0 0x55ab4592a0f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009850 tx=0x7fd1c4003880 comp rx=0 tx=0).stop 2024-04-16T17:23:40.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.053+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 shutdown_connections 2024-04-16T17:23:40.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.053+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab4593c5d0 0x55ab4593e9c0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.053+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab4592c6b0 0x55ab4592a0f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.053+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 0x55ab45929bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.053+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2282392607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45929670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.053+0000 7fd1bea00640 0 client.30382 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:40.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.053+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 >> 172.21.15.18:0/2282392607 conn(0x55ab4591d8e0 msgr2=0x55ab4591f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.054+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 shutdown_connections 2024-04-16T17:23:40.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.055+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2282392607 wait complete. 2024-04-16T17:23:40.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.055+0000 7fd1e2da6740 20 client.30382 trim_cache size 0 max 16384 2024-04-16T17:23:40.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.121+0000 7fd1e2da6740 1 Processor -- start 2024-04-16T17:23:40.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.121+0000 7fd1e2da6740 1 -- start start 2024-04-16T17:23:40.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.121+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab459231b0 0x55ab4592bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 0x55ab4591c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab4592c6b0 0x55ab459419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1e2da6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81c0d10 con 0x55ab45923ad0 2024-04-16T17:23:40.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1e2da6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81be5e0 con 0x55ab4592c6b0 2024-04-16T17:23:40.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1e2da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81bc450 con 0x55ab459231b0 2024-04-16T17:23:40.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1de000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab4592c6b0 0x55ab459419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1de000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab4592c6b0 0x55ab459419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53728/0 (socket says 172.21.15.18:53728) 2024-04-16T17:23:40.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1de000640 1 -- 172.21.15.18:0/2559753654 learned_addr learned my addr 172.21.15.18:0/2559753654 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.122+0000 7fd1dd600640 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab459231b0 0x55ab4592bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.123+0000 7fd1dcc00640 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 0x55ab4591c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.124+0000 7fd1de000640 1 -- 172.21.15.18:0/2559753654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab459231b0 msgr2=0x55ab4592bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.124+0000 7fd1de000640 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab459231b0 0x55ab4592bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.124+0000 7fd1de000640 1 -- 172.21.15.18:0/2559753654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 msgr2=0x55ab4591c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.124+0000 7fd1de000640 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 0x55ab4591c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.124+0000 7fd1de000640 1 -- 172.21.15.18:0/2559753654 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab45922450 con 0x55ab4592c6b0 2024-04-16T17:23:40.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.125+0000 7fd1de000640 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab4592c6b0 0x55ab459419e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4046a10 tx=0x7fd1c402c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2830fbe2af0e637f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.127+0000 7fd1c9c00640 1 -- 172.21.15.18:0/2559753654 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c405f070 con 0x55ab4592c6b0 2024-04-16T17:23:40.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.127+0000 7fd1c9c00640 1 -- 172.21.15.18:0/2559753654 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c4009520 con 0x55ab4592c6b0 2024-04-16T17:23:40.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.127+0000 7fd1c9c00640 1 -- 172.21.15.18:0/2559753654 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c404d030 con 0x55ab4592c6b0 2024-04-16T17:23:40.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.129+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2559753654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab4592c6b0 msgr2=0x55ab459419e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.129+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab4592c6b0 0x55ab459419e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4046a10 tx=0x7fd1c402c2d0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.130+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2559753654 shutdown_connections 2024-04-16T17:23:40.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.130+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab4592c6b0 0x55ab459419e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.131+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab45923ad0 0x55ab4591c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.131+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2559753654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab459231b0 0x55ab4592bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.131+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2559753654 >> 172.21.15.18:0/2559753654 conn(0x55ab4591d8e0 msgr2=0x55ab459285b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.132+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2559753654 shutdown_connections 2024-04-16T17:23:40.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.132+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2559753654 wait complete. 2024-04-16T17:23:40.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.133+0000 7fd1e2da6740 1 -- start start 2024-04-16T17:23:40.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.133+0000 7fd1e2da6740 1 Processor -- start 2024-04-16T17:23:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.133+0000 7f324a200640 1 -- 172.21.15.18:0/1231178093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3244105380 msgr2=0x7f324410cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.133+0000 7f324a200640 1 --2- 172.21.15.18:0/1231178093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3244105380 0x7f324410cf50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f323c009ca0 tx=0x7f323c02c100 comp rx=0 tx=0).stop 2024-04-16T17:23:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.134+0000 7f324a200640 1 -- 172.21.15.18:0/1231178093 shutdown_connections 2024-04-16T17:23:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.134+0000 7f324a200640 1 --2- 172.21.15.18:0/1231178093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3244105380 0x7f324410cf50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.134+0000 7f324a200640 1 --2- 172.21.15.18:0/1231178093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3244100c60 0x7f3244104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.134+0000 7f324a200640 1 --2- 172.21.15.18:0/1231178093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32441002b0 0x7f3244100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.134+0000 7f324a200640 1 -- 172.21.15.18:0/1231178093 >> 172.21.15.18:0/1231178093 conn(0x7f3244076f40 msgr2=0x7f32440fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.134+0000 7f324a200640 1 -- 172.21.15.18:0/1231178093 shutdown_connections 2024-04-16T17:23:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.134+0000 7f324a200640 1 -- 172.21.15.18:0/1231178093 wait complete. 2024-04-16T17:23:40.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.135+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45941cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.135+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4593f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.135+0000 7fd1e2da6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 0x55ab4593f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.135+0000 7fd1dcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4593f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.136+0000 7fd1dcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4593f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51594/0 (socket says 172.21.15.18:51594) 2024-04-16T17:23:40.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.136+0000 7fd1e2da6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81bcaa0 con 0x55ab4592c6b0 2024-04-16T17:23:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.137+0000 7f324a200640 1 Processor -- start 2024-04-16T17:23:40.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.137+0000 7fd1dcc00640 1 -- 172.21.15.18:0/2536921967 learned_addr learned my addr 172.21.15.18:0/2536921967 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.137+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81bdf10 con 0x55ab459231b0 2024-04-16T17:23:40.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.137+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81bd170 con 0x55ab45923ad0 2024-04-16T17:23:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.137+0000 7f324a200640 1 -- start start 2024-04-16T17:23:40.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.137+0000 7fd1e2da6740 10 client.0 _ll_register_callbacks cb 0x7fd1d00104f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:40.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.137+0000 7fd1dd600640 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45941cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.138+0000 7fd1dcc00640 1 -- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 msgr2=0x55ab45941cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.138+0000 7fd1dcc00640 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45941cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.138+0000 7fd1dcc00640 1 -- 172.21.15.18:0/2536921967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 msgr2=0x55ab4593f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.138+0000 7fd1dcc00640 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 0x55ab4593f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.138+0000 7fd1dcc00640 1 -- 172.21.15.18:0/2536921967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab4593fce0 con 0x55ab45923ad0 2024-04-16T17:23:40.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.138+0000 7fd1dd600640 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45941cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:23:40.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.139+0000 7fd1dcc00640 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4593f060 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc09bac0 tx=0x7fd1cc09c230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a075679a783e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.139+0000 7f324a200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32441002b0 0x7f3244198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.139+0000 7f324a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3244105380 0x7f3244198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.139+0000 7f324a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324419cf10 0x7f324419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.139+0000 7f324a200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32441121f0 con 0x7f32441002b0 2024-04-16T17:23:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.139+0000 7f324a200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3244112070 con 0x7f324419cf10 2024-04-16T17:23:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.139+0000 7f324a200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3244112370 con 0x7f3244105380 2024-04-16T17:23:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3243e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3244105380 0x7f3244198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3243e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3244105380 0x7f3244198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51600/0 (socket says 172.21.15.18:51600) 2024-04-16T17:23:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3243e00640 1 -- 172.21.15.18:0/658529625 learned_addr learned my addr 172.21.15.18:0/658529625 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3248e00640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32441002b0 0x7f3244198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3243e00640 1 -- 172.21.15.18:0/658529625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324419cf10 msgr2=0x7f324419d3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:23:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3243e00640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324419cf10 0x7f324419d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3243e00640 1 -- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32441002b0 msgr2=0x7f3244198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3243e00640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32441002b0 0x7f3244198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.140+0000 7f3243e00640 1 -- 172.21.15.18:0/658529625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324419da00 con 0x7f3244105380 2024-04-16T17:23:40.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.139+0000 7fd1bea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:40.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.139+0000 7fd1bea00640 1 -- 172.21.15.18:0/2536921967 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc09ce60 con 0x55ab45923ad0 2024-04-16T17:23:40.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.139+0000 7fd1bea00640 1 -- 172.21.15.18:0/2536921967 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7fd1cc0981a0 con 0x55ab45923ad0 2024-04-16T17:23:40.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.140+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c4009520 con 0x55ab45923ad0 2024-04-16T17:23:40.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.141+0000 7fd1bea00640 1 -- 172.21.15.18:0/2536921967 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd1cc09fee0 con 0x55ab45923ad0 2024-04-16T17:23:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.141+0000 7f3243e00640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3244105380 0x7f3244198b90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f323000c970 tx=0x7f323000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.141+0000 7fd1e2da6740 10 client.30388 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:40.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.141+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ab459b3700 con 0x55ab45923ad0 2024-04-16T17:23:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.141+0000 7f3241600640 1 -- 172.21.15.18:0/658529625 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230007bb0 con 0x7f3244105380 2024-04-16T17:23:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.141+0000 7f324a200640 1 -- 172.21.15.18:0/658529625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324419dcf0 con 0x7f3244105380 2024-04-16T17:23:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.141+0000 7f3241600640 1 -- 172.21.15.18:0/658529625 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3230007d50 con 0x7f3244105380 2024-04-16T17:23:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.141+0000 7f3241600640 1 -- 172.21.15.18:0/658529625 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323000f450 con 0x7f3244105380 2024-04-16T17:23:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.142+0000 7f324a200640 1 -- 172.21.15.18:0/658529625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32441a5530 con 0x7f3244105380 2024-04-16T17:23:40.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.143+0000 7fd1bea00640 1 -- 172.21.15.18:0/2536921967 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c4009520 con 0x55ab45923ad0 2024-04-16T17:23:40.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.143+0000 7fd1bea00640 1 -- 172.21.15.18:0/2536921967 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fd1cc022060 con 0x55ab45923ad0 2024-04-16T17:23:40.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.143+0000 7fd1e2da6740 10 client.30388 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:40.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.143+0000 7fd1e2da6740 10 client.30388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:40.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.143+0000 7fd1e2da6740 20 client.30388 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:40.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.143+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab459b3eb0 0x55ab459b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.143+0000 7fd1e2da6740 4 client.30388 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:23:40.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.143+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab458909b0 con 0x55ab459b3eb0 2024-04-16T17:23:40.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.144+0000 7fd1dd600640 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab459b3eb0 0x55ab459b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.145+0000 7fd1dd600640 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab459b3eb0 0x55ab459b6310 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac2b1c33d40853e server_cookie=6450c8ec562c875c in_seq=0 out_seq=0 2024-04-16T17:23:40.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.146+0000 7fd1bea00640 10 client.30388 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:40.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.150+0000 7fd1bea00640 1 -- 172.21.15.18:0/2536921967 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55ab458909b0 con 0x55ab459b3eb0 2024-04-16T17:23:40.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.150+0000 7fd1bea00640 10 client.30388 handle_command_reply: tid=0 2024-04-16T17:23:40.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.150+0000 7fd1e2da6740 1 client.30388 shutdown 2024-04-16T17:23:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.150+0000 7f3241600640 1 -- 172.21.15.18:0/658529625 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3230004f20 con 0x7f3244105380 2024-04-16T17:23:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.151+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 msgr2=0x55ab4593f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.151+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4593f060 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc09bac0 tx=0x7fd1cc09c230 comp rx=0 tx=0).stop 2024-04-16T17:23:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.151+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 shutdown_connections 2024-04-16T17:23:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.152+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ab459b3eb0 0x55ab459b6310 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.152+0000 7fd1bea00640 0 client.30388 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.152+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ab4592c6b0 0x55ab4593f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.152+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ab45923ad0 0x55ab4593f060 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.152+0000 7f324e636740 1 Processor -- start 2024-04-16T17:23:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.153+0000 7f324e636740 1 -- start start 2024-04-16T17:23:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.153+0000 7f324e636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f808c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.153+0000 7f324e636740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f807d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.153+0000 7fd1e2da6740 1 --2- 172.21.15.18:0/2536921967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ab459231b0 0x55ab45941cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.153+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 >> 172.21.15.18:0/2536921967 conn(0x55ab4591d8e0 msgr2=0x55ab45929ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.153+0000 7f324e636740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807d680 0x55d1f8093ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.153+0000 7f324e636740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d1f7fe4780 con 0x55d1f807d680 2024-04-16T17:23:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.153+0000 7f324e636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d1f7f8af40 con 0x55d1f808c260 2024-04-16T17:23:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.153+0000 7f324e636740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1f7fe0740 con 0x55d1f807ccc0 2024-04-16T17:23:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.154+0000 7f3249800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f808c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.154+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 shutdown_connections 2024-04-16T17:23:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.154+0000 7f3241600640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f322c0a3e60 0x7f322c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.154+0000 7fd1e2da6740 1 -- 172.21.15.18:0/2536921967 wait complete. 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.154+0000 7f3248e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f807d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.155+0000 7f3248e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f807d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51608/0 (socket says 172.21.15.18:51608) 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.155+0000 7f3248e00640 1 -- 172.21.15.18:0/3843862 learned_addr learned my addr 172.21.15.18:0/3843862 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.155+0000 7f3241600640 1 -- 172.21.15.18:0/658529625 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f32300d0c50 con 0x7f3244105380 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.155+0000 7f3248e00640 1 -- 172.21.15.18:0/3843862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 msgr2=0x55d1f808c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.155+0000 7f3248e00640 1 --2- 172.21.15.18:0/3843862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f808c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.155+0000 7f3248e00640 1 -- 172.21.15.18:0/3843862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807d680 msgr2=0x55d1f8093ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.155+0000 7f3248e00640 1 --2- 172.21.15.18:0/3843862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807d680 0x55d1f8093ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.155+0000 7f3248e00640 1 -- 172.21.15.18:0/3843862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f7cf7210 con 0x55d1f807ccc0 2024-04-16T17:23:40.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.156+0000 7fd1e2da6740 20 client.30388 trim_cache size 0 max 16384 2024-04-16T17:23:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.156+0000 7f3248e00640 1 --2- 172.21.15.18:0/3843862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f807d140 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f323800cc60 tx=0x7f323800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21318953f19811fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.156+0000 7f3249800640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f322c0a3e60 0x7f322c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.156+0000 7f3236000640 1 -- 172.21.15.18:0/3843862 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238014070 con 0x55d1f807ccc0 2024-04-16T17:23:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.156+0000 7f3236000640 1 -- 172.21.15.18:0/3843862 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32380047a0 con 0x55d1f807ccc0 2024-04-16T17:23:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.156+0000 7f3236000640 1 -- 172.21.15.18:0/3843862 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238010790 con 0x55d1f807ccc0 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (239 inodes in the stack)", 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout: "3a1e6ae0-f683-47a2-9838-7a42e9f0bbd3": { 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "3a1e6ae0-f683-47a2-9838-7a42e9f0bbd3", 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:23:40.158 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:23:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.157+0000 7f324e636740 1 -- 172.21.15.18:0/3843862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 msgr2=0x55d1f807d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.157+0000 7f324e636740 1 --2- 172.21.15.18:0/3843862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f807d140 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f323800cc60 tx=0x7f323800e720 comp rx=0 tx=0).stop 2024-04-16T17:23:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.158+0000 7f3249800640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f322c0a3e60 0x7f322c0a6320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f323c02c710 tx=0x7f323c0095c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.158+0000 7f324e636740 1 -- 172.21.15.18:0/3843862 shutdown_connections 2024-04-16T17:23:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.158+0000 7f324e636740 1 --2- 172.21.15.18:0/3843862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807d680 0x55d1f8093ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.158+0000 7f324e636740 1 --2- 172.21.15.18:0/3843862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f807d140 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.158+0000 7f324e636740 1 --2- 172.21.15.18:0/3843862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f808c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.158+0000 7f324e636740 1 -- 172.21.15.18:0/3843862 >> 172.21.15.18:0/3843862 conn(0x55d1f801a5d0 msgr2=0x55d1f807bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.158+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd1c00a3fd0 msgr2=0x7fd1c00a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.158+0000 7f324e636740 1 -- 172.21.15.18:0/3843862 shutdown_connections 2024-04-16T17:23:40.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.158+0000 7fd1e0000640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd1c00a3fd0 0x7fd1c00a6490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0010eb0 tx=0x7fd1d0010160 comp rx=0 tx=0).stop 2024-04-16T17:23:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.158+0000 7f324e636740 1 -- 172.21.15.18:0/3843862 wait complete. 2024-04-16T17:23:40.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.158+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1d8110bc0 msgr2=0x7fd1d81a7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.158+0000 7fd1e0000640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1d8110bc0 0x7fd1d81a7c00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009fd0 tx=0x7fd1cc00b870 comp rx=0 tx=0).stop 2024-04-16T17:23:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.159+0000 7f324e636740 1 -- start start 2024-04-16T17:23:40.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.159+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 shutdown_connections 2024-04-16T17:23:40.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.159+0000 7fd1e0000640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd1c00a3fd0 0x7fd1c00a6490 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.159+0000 7fd1e0000640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1d811b7e0 0x7fd1d81abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.159+0000 7fd1e0000640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1d8110bc0 0x7fd1d81a7c00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.159+0000 7fd1e0000640 1 --2- 172.21.15.18:0/3907671759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1d8110750 0x7fd1d81a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.159+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 >> 172.21.15.18:0/3907671759 conn(0x7fd1d806d0a0 msgr2=0x7fd1d811a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.159+0000 7f324e636740 1 Processor -- start 2024-04-16T17:23:40.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.159+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 shutdown_connections 2024-04-16T17:23:40.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:23:40.159+0000 7fd1e0000640 1 -- 172.21.15.18:0/3907671759 wait complete. 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.160+0000 7f324e636740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f8091280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.160+0000 7f324e636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 0x55d1f80917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.160+0000 7f324e636740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8091d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.160+0000 7f324e636740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d1f80945c0 con 0x55d1f808c260 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.160+0000 7f324e636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d1f7f9e390 con 0x55d1f807d680 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.160+0000 7f324e636740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1f8094740 con 0x55d1f807ccc0 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.160+0000 7f324e636740 10 client.0 _ll_register_callbacks cb 0x55d1f7f87f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.161+0000 7f3249800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8091d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.161+0000 7f3248e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f8091280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.161+0000 7f3249800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8091d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53804/0 (socket says 172.21.15.18:53804) 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.161+0000 7f3249800640 1 -- 172.21.15.18:0/62126761 learned_addr learned my addr 172.21.15.18:0/62126761 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.161+0000 7f3243e00640 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 0x55d1f80917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.161+0000 7f3249800640 1 -- 172.21.15.18:0/62126761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 msgr2=0x55d1f8091280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.162+0000 7f3249800640 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f8091280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.162+0000 7f3249800640 1 -- 172.21.15.18:0/62126761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 msgr2=0x55d1f80917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.162+0000 7f3249800640 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 0x55d1f80917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.162+0000 7f3249800640 1 -- 172.21.15.18:0/62126761 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f7dc3620 con 0x55d1f808c260 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.162+0000 7f3249800640 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8091d00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f323c005f60 tx=0x7f323c032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3242c38bb15d9fad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.162+0000 7f322aa00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.163+0000 7f322aa00640 1 -- 172.21.15.18:0/62126761 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c045050 con 0x55d1f808c260 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.163+0000 7f324e636740 1 -- 172.21.15.18:0/62126761 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32380047a0 con 0x55d1f808c260 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.163+0000 7f322aa00640 1 -- 172.21.15.18:0/62126761 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f323c004040 con 0x55d1f808c260 2024-04-16T17:23:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.164+0000 7f322aa00640 1 -- 172.21.15.18:0/62126761 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f323c039040 con 0x55d1f808c260 2024-04-16T17:23:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.167+0000 7f324e636740 10 client.19665 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.167+0000 7f324e636740 1 -- 172.21.15.18:0/62126761 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d1f80a06c0 con 0x55d1f808c260 2024-04-16T17:23:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.168+0000 7f322aa00640 1 -- 172.21.15.18:0/62126761 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32380047a0 con 0x55d1f808c260 2024-04-16T17:23:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.168+0000 7f322aa00640 1 -- 172.21.15.18:0/62126761 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f323c03e9f0 con 0x55d1f808c260 2024-04-16T17:23:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.168+0000 7f324e636740 10 client.19665 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.168+0000 7f324e636740 10 client.19665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.168+0000 7f324e636740 20 client.19665 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.168+0000 7f324e636740 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d1f80a1f10 0x55d1f80a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.168+0000 7f324e636740 4 client.19665 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.168+0000 7f324e636740 1 -- 172.21.15.18:0/62126761 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1f7ff21a0 con 0x55d1f80a1f10 2024-04-16T17:23:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.169+0000 7f3248e00640 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d1f80a1f10 0x55d1f80a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.171+0000 7f322aa00640 10 client.19665 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.171+0000 7f3248e00640 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d1f80a1f10 0x55d1f80a43b0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fbe77422416e988 server_cookie=edc499cc558be0a4 in_seq=0 out_seq=0 2024-04-16T17:23:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:39 smithi022 ceph-mon[36023]: osdmap e129: 12 total, 12 up, 12 in 2024-04-16T17:23:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:39 smithi022 ceph-mon[36023]: pgmap v605: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 161 op/s 2024-04-16T17:23:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.177+0000 7f322aa00640 1 -- 172.21.15.18:0/62126761 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1f7ff21a0 con 0x55d1f80a1f10 2024-04-16T17:23:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.177+0000 7f322aa00640 10 client.19665 handle_command_reply: tid=0 2024-04-16T17:23:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.178+0000 7f324e636740 1 client.19665 shutdown 2024-04-16T17:23:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.178+0000 7f324e636740 1 -- 172.21.15.18:0/62126761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 msgr2=0x55d1f8091d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.178+0000 7f324e636740 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8091d00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f323c005f60 tx=0x7f323c032590 comp rx=0 tx=0).stop 2024-04-16T17:23:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.179+0000 7f324e636740 1 -- 172.21.15.18:0/62126761 shutdown_connections 2024-04-16T17:23:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.179+0000 7f324e636740 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d1f80a1f10 0x55d1f80a43b0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.179+0000 7f324e636740 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8091d00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.179+0000 7f324e636740 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 0x55d1f80917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.179+0000 7f324e636740 1 --2- 172.21.15.18:0/62126761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807ccc0 0x55d1f8091280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.179+0000 7f324e636740 1 -- 172.21.15.18:0/62126761 >> 172.21.15.18:0/62126761 conn(0x55d1f801a5d0 msgr2=0x55d1f807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.179+0000 7f322aa00640 0 client.19665 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:40.185 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.185+0000 7f324e636740 1 -- 172.21.15.18:0/62126761 shutdown_connections 2024-04-16T17:23:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.190+0000 7f324e636740 1 -- 172.21.15.18:0/62126761 wait complete. 2024-04-16T17:23:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.190+0000 7f324e636740 20 client.19665 trim_cache size 0 max 16384 2024-04-16T17:23:40.195 INFO:tasks.cephfs.filesystem:scrub status for tag:3a1e6ae0-f683-47a2-9838-7a42e9f0bbd3 - {'path': '/', 'tag': '3a1e6ae0-f683-47a2-9838-7a42e9f0bbd3', 'options': 'recursive,force'} 2024-04-16T17:23:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.222+0000 7f324e636740 1 Processor -- start 2024-04-16T17:23:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.223+0000 7f324e636740 1 -- start start 2024-04-16T17:23:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.224+0000 7f324e636740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807ccc0 0x55d1f809d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.224+0000 7f324e636740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807d680 0x55d1f80756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.224+0000 7f324e636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f80903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.224+0000 7f324e636740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32441afc50 con 0x55d1f807ccc0 2024-04-16T17:23:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.224+0000 7f324e636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32441ad520 con 0x55d1f808c260 2024-04-16T17:23:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.224+0000 7f324e636740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32441ab310 con 0x55d1f807d680 2024-04-16T17:23:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3249800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f80903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3249800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f80903e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:53790/0 (socket says 172.21.15.18:53790) 2024-04-16T17:23:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3249800640 1 -- 172.21.15.18:0/3220503329 learned_addr learned my addr 172.21.15.18:0/3220503329 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3248e00640 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807ccc0 0x55d1f809d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3243e00640 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807d680 0x55d1f80756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3248e00640 1 -- 172.21.15.18:0/3220503329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807d680 msgr2=0x55d1f80756a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3248e00640 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807d680 0x55d1f80756a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3248e00640 1 -- 172.21.15.18:0/3220503329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 msgr2=0x55d1f80903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3248e00640 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f80903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.225+0000 7f3248e00640 1 -- 172.21.15.18:0/3220503329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f8076040 con 0x55d1f807ccc0 2024-04-16T17:23:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.226+0000 7f3249800640 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f80903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.227+0000 7f3248e00640 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807ccc0 0x55d1f809d480 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3238000c00 tx=0x7f3238004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57188ceebd62fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.229+0000 7f3235600640 1 -- 172.21.15.18:0/3220503329 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238014040 con 0x55d1f807ccc0 2024-04-16T17:23:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.230+0000 7f3235600640 1 -- 172.21.15.18:0/3220503329 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3238005500 con 0x55d1f807ccc0 2024-04-16T17:23:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.230+0000 7f3235600640 1 -- 172.21.15.18:0/3220503329 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238004e20 con 0x55d1f807ccc0 2024-04-16T17:23:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.231+0000 7f324e636740 1 -- 172.21.15.18:0/3220503329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807ccc0 msgr2=0x55d1f809d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.231+0000 7f324e636740 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807ccc0 0x55d1f809d480 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3238000c00 tx=0x7f3238004280 comp rx=0 tx=0).stop 2024-04-16T17:23:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.232+0000 7f324e636740 1 -- 172.21.15.18:0/3220503329 shutdown_connections 2024-04-16T17:23:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.232+0000 7f324e636740 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f808c260 0x55d1f80903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.232+0000 7f324e636740 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f807d680 0x55d1f80756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.232+0000 7f324e636740 1 --2- 172.21.15.18:0/3220503329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f807ccc0 0x55d1f809d480 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.232+0000 7f324e636740 1 -- 172.21.15.18:0/3220503329 >> 172.21.15.18:0/3220503329 conn(0x55d1f801a5d0 msgr2=0x55d1f807a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.232+0000 7f324e636740 1 -- 172.21.15.18:0/3220503329 shutdown_connections 2024-04-16T17:23:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.232+0000 7f324e636740 1 -- 172.21.15.18:0/3220503329 wait complete. 2024-04-16T17:23:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.233+0000 7f324e636740 1 -- start start 2024-04-16T17:23:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.233+0000 7f324e636740 1 Processor -- start 2024-04-16T17:23:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f324e636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 0x55d1f80999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f324e636740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8099ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f3248e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 0x55d1f80999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f324e636740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f8094950 0x55d1f8094dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f324e636740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32441ab9e0 con 0x55d1f808c260 2024-04-16T17:23:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f324e636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32441ace50 con 0x55d1f807d680 2024-04-16T17:23:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f324e636740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32441ac0b0 con 0x55d1f8094950 2024-04-16T17:23:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f324e636740 10 client.0 _ll_register_callbacks cb 0x55d1f8094360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f3243e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8099ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f3243e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8099ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53812/0 (socket says 172.21.15.18:53812) 2024-04-16T17:23:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.234+0000 7f3243e00640 1 -- 172.21.15.18:0/488689019 learned_addr learned my addr 172.21.15.18:0/488689019 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.235+0000 7f3249800640 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f8094950 0x55d1f8094dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.235+0000 7f3249800640 1 -- 172.21.15.18:0/488689019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 msgr2=0x55d1f80999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.235+0000 7f3249800640 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 0x55d1f80999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.235+0000 7f3249800640 1 -- 172.21.15.18:0/488689019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 msgr2=0x55d1f8099ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.235+0000 7f3249800640 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8099ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.235+0000 7f3249800640 1 -- 172.21.15.18:0/488689019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f8095440 con 0x55d1f8094950 2024-04-16T17:23:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.235+0000 7f3249800640 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f8094950 0x55d1f8094dd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f323c032a90 tx=0x7f323c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c198f5a63bee4b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.235+0000 7f322aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.236+0000 7f322aa00640 1 -- 172.21.15.18:0/488689019 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c061060 con 0x55d1f8094950 2024-04-16T17:23:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.236+0000 7f322aa00640 1 -- 172.21.15.18:0/488689019 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f323c003d80 con 0x55d1f8094950 2024-04-16T17:23:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.236+0000 7f324e636740 1 -- 172.21.15.18:0/488689019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3238005500 con 0x55d1f8094950 2024-04-16T17:23:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.236+0000 7f322aa00640 1 -- 172.21.15.18:0/488689019 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f323c068050 con 0x55d1f8094950 2024-04-16T17:23:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.236+0000 7f324e636740 10 client.30403 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.236+0000 7f324e636740 1 -- 172.21.15.18:0/488689019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d1f8109330 con 0x55d1f8094950 2024-04-16T17:23:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.237+0000 7f322aa00640 1 -- 172.21.15.18:0/488689019 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3238005500 con 0x55d1f8094950 2024-04-16T17:23:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.237+0000 7f322aa00640 1 -- 172.21.15.18:0/488689019 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f323c03e9f0 con 0x55d1f8094950 2024-04-16T17:23:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.237+0000 7f324e636740 10 client.30403 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.237+0000 7f324e636740 10 client.30403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.237+0000 7f324e636740 20 client.30403 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.237+0000 7f324e636740 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d1f81099f0 0x55d1f810be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.237+0000 7f324e636740 4 client.30403 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:23:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.237+0000 7f324e636740 1 -- 172.21.15.18:0/488689019 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1f7fec010 con 0x55d1f81099f0 2024-04-16T17:23:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.238+0000 7f3248e00640 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d1f81099f0 0x55d1f810be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.239+0000 7f3248e00640 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d1f81099f0 0x55d1f810be10 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6244dffda6e784 server_cookie=428ba254b06f74c8 in_seq=0 out_seq=0 2024-04-16T17:23:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:40.239+0000 7f322aa00640 10 client.30403 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:40.550 INFO:tasks.workunit.client.0.smithi018.stdout: 10 82980 11.84 MB/sec execute 232 sec latency 201.363 ms 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:34 smithi018 ceph-mon[31776]: pgmap v600: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:34 smithi018 ceph-mon[31776]: osdmap e128: 12 total, 12 up, 12 in 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:36 smithi018 ceph-mon[31776]: pgmap v602: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 408 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 391 op/s 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:37 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2259434980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:38 smithi018 ceph-mon[31776]: pgmap v603: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:39 smithi018 ceph-mon[31776]: osdmap e129: 12 total, 12 up, 12 in 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:39 smithi018 ceph-mon[31776]: pgmap v605: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 161 op/s 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:40.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:23:40.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:40.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:23:40.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:40 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:23:41.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:23:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:23:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:23:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:23:41.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:40 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:23:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:40 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.356+0000 7f322aa00640 1 -- 172.21.15.18:0/488689019 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1f7fec010 con 0x55d1f81099f0 2024-04-16T17:23:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.356+0000 7f322aa00640 10 client.30403 handle_command_reply: tid=0 2024-04-16T17:23:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.356+0000 7f324e636740 1 client.30403 shutdown 2024-04-16T17:23:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.356+0000 7f324e636740 1 -- 172.21.15.18:0/488689019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f8094950 msgr2=0x55d1f8094dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.356+0000 7f324e636740 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f8094950 0x55d1f8094dd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f323c032a90 tx=0x7f323c009b20 comp rx=0 tx=0).stop 2024-04-16T17:23:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.357+0000 7f324e636740 1 -- 172.21.15.18:0/488689019 shutdown_connections 2024-04-16T17:23:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.357+0000 7f324e636740 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d1f81099f0 0x55d1f810be10 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.357+0000 7f324e636740 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1f8094950 0x55d1f8094dd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.357+0000 7f324e636740 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d1f808c260 0x55d1f8099ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.357+0000 7f324e636740 1 --2- 172.21.15.18:0/488689019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1f807d680 0x55d1f80999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.357+0000 7f324e636740 1 -- 172.21.15.18:0/488689019 >> 172.21.15.18:0/488689019 conn(0x55d1f801a5d0 msgr2=0x55d1f807a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.357+0000 7f322aa00640 0 client.30403 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.358+0000 7f324e636740 1 -- 172.21.15.18:0/488689019 shutdown_connections 2024-04-16T17:23:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.358+0000 7f324e636740 1 -- 172.21.15.18:0/488689019 wait complete. 2024-04-16T17:23:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.358+0000 7f324e636740 20 client.30403 trim_cache size 0 max 16384 2024-04-16T17:23:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:23:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:23:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:23:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:23:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 -- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f322c0a3e60 msgr2=0x7f322c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f322c0a3e60 0x7f322c0a6320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f323c02c710 tx=0x7f323c0095c0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 -- 172.21.15.18:0/658529625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3244105380 msgr2=0x7f3244198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3244105380 0x7f3244198b90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f323000c970 tx=0x7f323000ce30 comp rx=0 tx=0).stop 2024-04-16T17:23:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 -- 172.21.15.18:0/658529625 shutdown_connections 2024-04-16T17:23:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f322c0a3e60 0x7f322c0a6320 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f324419cf10 0x7f324419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3244105380 0x7f3244198b90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 --2- 172.21.15.18:0/658529625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32441002b0 0x7f3244198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.362+0000 7f324a200640 1 -- 172.21.15.18:0/658529625 >> 172.21.15.18:0/658529625 conn(0x7f3244076f40 msgr2=0x7f3244102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.363+0000 7f324a200640 1 -- 172.21.15.18:0/658529625 shutdown_connections 2024-04-16T17:23:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:41.363+0000 7f324a200640 1 -- 172.21.15.18:0/658529625 wait complete. 2024-04-16T17:23:41.550 INFO:tasks.workunit.client.0.smithi018.stdout: 10 83035 11.80 MB/sec execute 233 sec latency 932.039 ms 2024-04-16T17:23:42.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:41 smithi043 ceph-mon[36901]: pgmap v606: 129 pgs: 129 active+clean; 434 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 409 op/s 2024-04-16T17:23:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:41 smithi022 ceph-mon[36023]: pgmap v606: 129 pgs: 129 active+clean; 434 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 409 op/s 2024-04-16T17:23:42.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 83280 11.82 MB/sec execute 234 sec latency 198.840 ms 2024-04-16T17:23:43.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:42 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:43.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:42 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:43.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 83582 11.85 MB/sec execute 235 sec latency 173.616 ms 2024-04-16T17:23:44.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:43 smithi043 ceph-mon[36901]: pgmap v607: 129 pgs: 129 active+clean; 434 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-04-16T17:23:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:43 smithi022 ceph-mon[36023]: pgmap v607: 129 pgs: 129 active+clean; 434 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-04-16T17:23:44.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 83854 11.84 MB/sec execute 236 sec latency 221.454 ms 2024-04-16T17:23:45.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 84117 11.85 MB/sec execute 237 sec latency 196.812 ms 2024-04-16T17:23:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:45 smithi022 ceph-mon[36023]: pgmap v608: 129 pgs: 129 active+clean; 436 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 394 op/s 2024-04-16T17:23:46.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:45 smithi043 ceph-mon[36901]: pgmap v608: 129 pgs: 129 active+clean; 436 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 394 op/s 2024-04-16T17:23:46.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 84337 11.84 MB/sec execute 238 sec latency 266.969 ms 2024-04-16T17:23:47.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 84626 11.86 MB/sec execute 239 sec latency 216.272 ms 2024-04-16T17:23:48.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:47 smithi022 ceph-mon[36023]: pgmap v609: 129 pgs: 129 active+clean; 421 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 519 op/s 2024-04-16T17:23:48.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:47 smithi043 ceph-mon[36901]: pgmap v609: 129 pgs: 129 active+clean; 421 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 519 op/s 2024-04-16T17:23:48.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 84895 11.85 MB/sec execute 240 sec latency 254.767 ms 2024-04-16T17:23:48.708 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:41 smithi018 ceph-mon[31776]: pgmap v606: 129 pgs: 129 active+clean; 434 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 409 op/s 2024-04-16T17:23:48.708 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:42 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:23:48.709 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:43 smithi018 ceph-mon[31776]: pgmap v607: 129 pgs: 129 active+clean; 434 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-04-16T17:23:48.709 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:45 smithi018 ceph-mon[31776]: pgmap v608: 129 pgs: 129 active+clean; 436 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 394 op/s 2024-04-16T17:23:48.709 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:47 smithi018 ceph-mon[31776]: pgmap v609: 129 pgs: 129 active+clean; 421 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 519 op/s 2024-04-16T17:23:49.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 85036 11.84 MB/sec execute 241 sec latency 612.124 ms 2024-04-16T17:23:50.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:49 smithi022 ceph-mon[36023]: pgmap v610: 129 pgs: 129 active+clean; 423 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 524 op/s 2024-04-16T17:23:50.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:49 smithi043 ceph-mon[36901]: pgmap v610: 129 pgs: 129 active+clean; 423 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 524 op/s 2024-04-16T17:23:50.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 85357 11.87 MB/sec execute 242 sec latency 227.624 ms 2024-04-16T17:23:51.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 85666 11.89 MB/sec execute 243 sec latency 289.011 ms 2024-04-16T17:23:52.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:51 smithi022 ceph-mon[36023]: pgmap v611: 129 pgs: 129 active+clean; 429 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-04-16T17:23:52.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:51 smithi043 ceph-mon[36901]: pgmap v611: 129 pgs: 129 active+clean; 429 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-04-16T17:23:52.552 INFO:tasks.workunit.client.0.smithi018.stdout: 10 85931 11.91 MB/sec execute 244 sec latency 205.055 ms 2024-04-16T17:23:53.552 INFO:tasks.workunit.client.0.smithi018.stdout: 10 86232 11.92 MB/sec execute 245 sec latency 183.859 ms 2024-04-16T17:23:54.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:53 smithi022 ceph-mon[36023]: pgmap v612: 129 pgs: 129 active+clean; 429 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-04-16T17:23:54.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:53 smithi043 ceph-mon[36901]: pgmap v612: 129 pgs: 129 active+clean; 429 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-04-16T17:23:54.552 INFO:tasks.workunit.client.0.smithi018.stdout: 10 86607 11.98 MB/sec execute 246 sec latency 188.031 ms 2024-04-16T17:23:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 -- 172.21.15.18:0/3667655140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc102250 msgr2=0x7f99dc102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3667655140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc102250 0x7f99dc102630 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009900 tx=0x7f99cc02bc50 comp rx=0 tx=0).stop 2024-04-16T17:23:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 -- 172.21.15.18:0/3667655140 shutdown_connections 2024-04-16T17:23:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3667655140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99dc107320 0x7f99dc10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3667655140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99dc102c00 0x7f99dc106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3667655140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc102250 0x7f99dc102630 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 -- 172.21.15.18:0/3667655140 >> 172.21.15.18:0/3667655140 conn(0x7f99dc0fda90 msgr2=0x7f99dc0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 -- 172.21.15.18:0/3667655140 shutdown_connections 2024-04-16T17:23:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.558+0000 7f99e0e00640 1 -- 172.21.15.18:0/3667655140 wait complete. 2024-04-16T17:23:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.559+0000 7f99e0e00640 1 Processor -- start 2024-04-16T17:23:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.559+0000 7f99e0e00640 1 -- start start 2024-04-16T17:23:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99e0e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99dc102250 0x7f99dc108f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99e0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc107320 0x7f99dc109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99e0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99dc109990 0x7f99dc1ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99e0e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1123b0 con 0x7f99dc102250 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99e0e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc112230 con 0x7f99dc109990 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99e0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc112530 con 0x7f99dc107320 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc107320 0x7f99dc109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99da800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99dc109990 0x7f99dc1ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc107320 0x7f99dc109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36036/0 (socket says 172.21.15.18:36036) 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99d3e00640 1 -- 172.21.15.18:0/3110224530 learned_addr learned my addr 172.21.15.18:0/3110224530 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99d9e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99dc102250 0x7f99dc108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99d3e00640 1 -- 172.21.15.18:0/3110224530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99dc109990 msgr2=0x7f99dc1ab200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.560+0000 7f99d3e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99dc109990 0x7f99dc1ab200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.561+0000 7f99d3e00640 1 -- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99dc102250 msgr2=0x7f99dc108f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.561+0000 7f99d3e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99dc102250 0x7f99dc108f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.561+0000 7f99d3e00640 1 -- 172.21.15.18:0/3110224530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc1ab740 con 0x7f99dc107320 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.561+0000 7f99d9e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99dc102250 0x7f99dc108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.561+0000 7f99d3e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc107320 0x7f99dc109450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f99c400e9c0 tx=0x7f99c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.561+0000 7f99d2a00640 1 -- 172.21.15.18:0/3110224530 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c400cd80 con 0x7f99dc107320 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.561+0000 7f99e0e00640 1 -- 172.21.15.18:0/3110224530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc1aba30 con 0x7f99dc107320 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.562+0000 7f99d2a00640 1 -- 172.21.15.18:0/3110224530 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c4005500 con 0x7f99dc107320 2024-04-16T17:23:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.562+0000 7f99d2a00640 1 -- 172.21.15.18:0/3110224530 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4010690 con 0x7f99dc107320 2024-04-16T17:23:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.562+0000 7f99e0e00640 1 -- 172.21.15.18:0/3110224530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc1abfa0 con 0x7f99dc107320 2024-04-16T17:23:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.567+0000 7f99d2a00640 1 -- 172.21.15.18:0/3110224530 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99c40046c0 con 0x7f99dc107320 2024-04-16T17:23:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.570+0000 7f99e3722740 1 Processor -- start 2024-04-16T17:23:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.571+0000 7f99d2a00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99c00a3e10 0x7f99c00a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.571+0000 7f99d2a00640 1 -- 172.21.15.18:0/3110224530 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f99c40d0fc0 con 0x7f99dc107320 2024-04-16T17:23:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.572+0000 7f99e3722740 1 -- start start 2024-04-16T17:23:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.572+0000 7f99e3722740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e6a610 0x55e2f3e69360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.572+0000 7f99e3722740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e69d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.572+0000 7f99e3722740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e72e20 0x55e2f3e75210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.572+0000 7f99e3722740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e2f3dce780 con 0x55e2f3e698a0 2024-04-16T17:23:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.572+0000 7f99e3722740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e2f3d74f40 con 0x55e2f3e72e20 2024-04-16T17:23:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.572+0000 7f99e3722740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e2f3dca740 con 0x55e2f3e6a610 2024-04-16T17:23:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.573+0000 7f99d9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e6a610 0x55e2f3e69360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.573+0000 7f99d9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e6a610 0x55e2f3e69360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36052/0 (socket says 172.21.15.18:36052) 2024-04-16T17:23:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.573+0000 7f99d9e00640 1 -- 172.21.15.18:0/534808342 learned_addr learned my addr 172.21.15.18:0/534808342 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.573+0000 7f99d9e00640 1 -- 172.21.15.18:0/534808342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e72e20 msgr2=0x55e2f3e75210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.573+0000 7f99d9e00640 1 --2- 172.21.15.18:0/534808342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e72e20 0x55e2f3e75210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.573+0000 7f99d9e00640 1 -- 172.21.15.18:0/534808342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 msgr2=0x55e2f3e69d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:23:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.573+0000 7f99d9e00640 1 --2- 172.21.15.18:0/534808342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e69d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.573+0000 7f99d9e00640 1 -- 172.21.15.18:0/534808342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f3ae1210 con 0x55e2f3e6a610 2024-04-16T17:23:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.574+0000 7f99da800640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99c00a3e10 0x7f99c00a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.574+0000 7f99d9e00640 1 --2- 172.21.15.18:0/534808342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e6a610 0x55e2f3e69360 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f99cc036040 tx=0x7f99cc005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09372693e2e756f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.575+0000 7f99cb400640 1 -- 172.21.15.18:0/534808342 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc02caa0 con 0x55e2f3e6a610 2024-04-16T17:23:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.575+0000 7f99da800640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99c00a3e10 0x7f99c00a62d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55e2f3e65720 tx=0x7f99d4009600 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.575+0000 7f99cb400640 1 -- 172.21.15.18:0/534808342 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc02cc40 con 0x55e2f3e6a610 2024-04-16T17:23:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.575+0000 7f99cb400640 1 -- 172.21.15.18:0/534808342 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc004920 con 0x55e2f3e6a610 2024-04-16T17:23:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.576+0000 7f99e3722740 1 -- 172.21.15.18:0/534808342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e6a610 msgr2=0x55e2f3e69360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.576+0000 7f99e3722740 1 --2- 172.21.15.18:0/534808342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e6a610 0x55e2f3e69360 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f99cc036040 tx=0x7f99cc005e50 comp rx=0 tx=0).stop 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.576+0000 7f99e3722740 1 -- 172.21.15.18:0/534808342 shutdown_connections 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.576+0000 7f99e3722740 1 --2- 172.21.15.18:0/534808342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e72e20 0x55e2f3e75210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.576+0000 7f99e3722740 1 --2- 172.21.15.18:0/534808342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e69d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.576+0000 7f99e3722740 1 --2- 172.21.15.18:0/534808342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e6a610 0x55e2f3e69360 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.576+0000 7f99e3722740 1 -- 172.21.15.18:0/534808342 >> 172.21.15.18:0/534808342 conn(0x55e2f3e045d0 msgr2=0x55e2f3e64590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.576+0000 7f99e3722740 1 -- 172.21.15.18:0/534808342 shutdown_connections 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.577+0000 7f99e3722740 1 -- 172.21.15.18:0/534808342 wait complete. 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.577+0000 7f99e3722740 1 -- start start 2024-04-16T17:23:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.577+0000 7f99e3722740 1 Processor -- start 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.578+0000 7f99e3722740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.578+0000 7f99e3722740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.578+0000 7f99e3722740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e7efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.578+0000 7f99e3722740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e2f3e75af0 con 0x55e2f3e698a0 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.578+0000 7f99e3722740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e2f3d88390 con 0x55e2f3e70810 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.578+0000 7f99e3722740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e2f3e75c70 con 0x55e2f3e72e20 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.578+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.578+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36062/0 (socket says 172.21.15.18:36062) 2024-04-16T17:23:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99d3e00640 1 -- 172.21.15.18:0/3608212477 learned_addr learned my addr 172.21.15.18:0/3608212477 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99e3722740 10 client.0 _ll_register_callbacks cb 0x55e2f3d71f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99da800640 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e7efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99d9e00640 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99d3e00640 1 -- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 msgr2=0x55e2f3e7efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99d3e00640 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e7efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99d3e00640 1 -- 172.21.15.18:0/3608212477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 msgr2=0x55e2f3e6fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99d3e00640 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99d3e00640 1 -- 172.21.15.18:0/3608212477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f3bad620 con 0x55e2f3e72e20 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.579+0000 7f99da800640 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e7efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.580+0000 7f99d3e00640 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e702d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f99c400c6a0 tx=0x7f99c409c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d3010c0a88187d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.580+0000 7f99bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:23:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.580+0000 7f99bfe00640 1 -- 172.21.15.18:0/3608212477 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c40b6060 con 0x55e2f3e72e20 2024-04-16T17:23:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.580+0000 7f99e3722740 1 -- 172.21.15.18:0/3608212477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99cc02cc40 con 0x55e2f3e72e20 2024-04-16T17:23:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.580+0000 7f99bfe00640 1 -- 172.21.15.18:0/3608212477 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f99c40a52e0 con 0x55e2f3e72e20 2024-04-16T17:23:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.581+0000 7f99bfe00640 1 -- 172.21.15.18:0/3608212477 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f99c40ce730 con 0x55e2f3e72e20 2024-04-16T17:23:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.582+0000 7f99e3722740 10 client.30415 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.582+0000 7f99e3722740 1 -- 172.21.15.18:0/3608212477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e2f3e82260 con 0x55e2f3e72e20 2024-04-16T17:23:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.582+0000 7f99bfe00640 1 -- 172.21.15.18:0/3608212477 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc02cc40 con 0x55e2f3e72e20 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.582+0000 7f99bfe00640 1 -- 172.21.15.18:0/3608212477 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f99c409b9c0 con 0x55e2f3e72e20 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.582+0000 7f99e3722740 10 client.30415 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.583+0000 7f99e3722740 10 client.30415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.583+0000 7f99e3722740 20 client.30415 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.583+0000 7f99e3722740 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2f3e836d0 0x55e2f3e85ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.583+0000 7f99e3722740 4 client.30415 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.583+0000 7f99e3722740 1 -- 172.21.15.18:0/3608212477 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2f3ddc1a0 con 0x55e2f3e836d0 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.583+0000 7f99d9e00640 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2f3e836d0 0x55e2f3e85ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.584+0000 7f99d9e00640 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2f3e836d0 0x55e2f3e85ac0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8759fec82da10db server_cookie=e78c708b4c3368d3 in_seq=0 out_seq=0 2024-04-16T17:23:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.584+0000 7f99bfe00640 10 client.30415 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.588+0000 7f99bfe00640 1 -- 172.21.15.18:0/3608212477 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2f3ddc1a0 con 0x55e2f3e836d0 2024-04-16T17:23:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.588+0000 7f99bfe00640 10 client.30415 handle_command_reply: tid=0 2024-04-16T17:23:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.588+0000 7f99e3722740 1 client.30415 shutdown 2024-04-16T17:23:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.588+0000 7f99e3722740 1 -- 172.21.15.18:0/3608212477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 msgr2=0x55e2f3e702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.588+0000 7f99e3722740 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e702d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f99c400c6a0 tx=0x7f99c409c520 comp rx=0 tx=0).stop 2024-04-16T17:23:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.589+0000 7f99e3722740 1 -- 172.21.15.18:0/3608212477 shutdown_connections 2024-04-16T17:23:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.589+0000 7f99e3722740 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2f3e836d0 0x55e2f3e85ac0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.589+0000 7f99e3722740 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e7efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.589+0000 7f99e3722740 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e702d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.589+0000 7f99e3722740 1 --2- 172.21.15.18:0/3608212477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.589+0000 7f99e3722740 1 -- 172.21.15.18:0/3608212477 >> 172.21.15.18:0/3608212477 conn(0x55e2f3e045d0 msgr2=0x55e2f3e662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.589+0000 7f99bfe00640 0 client.30415 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.589+0000 7f99e3722740 1 -- 172.21.15.18:0/3608212477 shutdown_connections 2024-04-16T17:23:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.590+0000 7f99e3722740 1 -- 172.21.15.18:0/3608212477 wait complete. 2024-04-16T17:23:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.590+0000 7f99e3722740 20 client.30415 trim_cache size 0 max 16384 2024-04-16T17:23:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.616+0000 7f99e3722740 1 Processor -- start 2024-04-16T17:23:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.616+0000 7f99e3722740 1 -- start start 2024-04-16T17:23:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.616+0000 7f99e3722740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e8b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.616+0000 7f99e3722740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e8b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.616+0000 7f99e3722740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e7ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.616+0000 7f99e3722740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1b86a0 con 0x55e2f3e698a0 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.616+0000 7f99e3722740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1b5f70 con 0x55e2f3e70810 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.616+0000 7f99e3722740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1b3de0 con 0x55e2f3e72e20 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e7ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e7ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36072/0 (socket says 172.21.15.18:36072) 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 -- 172.21.15.18:0/3598475432 learned_addr learned my addr 172.21.15.18:0/3598475432 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99d9e00640 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e8b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99d3e00640 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e8b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 -- 172.21.15.18:0/3598475432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 msgr2=0x55e2f3e8b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e8b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 -- 172.21.15.18:0/3598475432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 msgr2=0x55e2f3e8b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e8b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 -- 172.21.15.18:0/3598475432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f3e75580 con 0x55e2f3e72e20 2024-04-16T17:23:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.617+0000 7f99da800640 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e7ea20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f99d4012690 tx=0x7f99d4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afdac0bdc5fa5df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.618+0000 7f99caa00640 1 -- 172.21.15.18:0/3598475432 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4009bc0 con 0x55e2f3e72e20 2024-04-16T17:23:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.618+0000 7f99caa00640 1 -- 172.21.15.18:0/3598475432 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d4003b70 con 0x55e2f3e72e20 2024-04-16T17:23:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.618+0000 7f99caa00640 1 -- 172.21.15.18:0/3598475432 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4011b10 con 0x55e2f3e72e20 2024-04-16T17:23:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 -- 172.21.15.18:0/3598475432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 msgr2=0x55e2f3e7ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e7ea20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f99d4012690 tx=0x7f99d4013000 comp rx=0 tx=0).stop 2024-04-16T17:23:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 -- 172.21.15.18:0/3598475432 shutdown_connections 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e72e20 0x55e2f3e7ea20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e70810 0x55e2f3e8b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 --2- 172.21.15.18:0/3598475432 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e698a0 0x55e2f3e8b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 -- 172.21.15.18:0/3598475432 >> 172.21.15.18:0/3598475432 conn(0x55e2f3e045d0 msgr2=0x55e2f3e856e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 -- 172.21.15.18:0/3598475432 shutdown_connections 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 -- 172.21.15.18:0/3598475432 wait complete. 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 -- start start 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.619+0000 7f99e3722740 1 Processor -- start 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99e3722740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99e3722740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e70810 0x55e2f3e6cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99e3722740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e72e20 0x55e2f3e6d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99e3722740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1b4430 con 0x55e2f3e72e20 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99e3722740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1b58a0 con 0x55e2f3e698a0 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99e3722740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1b4b00 con 0x55e2f3e70810 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99e3722740 10 client.0 _ll_register_callbacks cb 0x7f99c4009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99d9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99da800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e72e20 0x55e2f3e6d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.620+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e70810 0x55e2f3e6cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99da800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e72e20 0x55e2f3e6d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45008/0 (socket says 172.21.15.18:45008) 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99d9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:50830/0 (socket says 172.21.15.18:50830) 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99da800640 1 -- 172.21.15.18:0/1875164578 learned_addr learned my addr 172.21.15.18:0/1875164578 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e70810 0x55e2f3e6cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:36076/0 (socket says 172.21.15.18:36076) 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99d9e00640 1 -- 172.21.15.18:0/1875164578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e70810 msgr2=0x55e2f3e6cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99d9e00640 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e70810 0x55e2f3e6cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99d9e00640 1 -- 172.21.15.18:0/1875164578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e72e20 msgr2=0x55e2f3e6d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99d9e00640 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e72e20 0x55e2f3e6d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99d9e00640 1 -- 172.21.15.18:0/1875164578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f3ee8bd0 con 0x55e2f3e698a0 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99d3e00640 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e70810 0x55e2f3e6cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.621+0000 7f99da800640 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e72e20 0x55e2f3e6d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.622+0000 7f99d9e00640 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6caa0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x55e2f3e65490 tx=0x7f99cc005440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f42c12362212b234 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.622+0000 7f99bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.622+0000 7f99bfe00640 1 -- 172.21.15.18:0/1875164578 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc055060 con 0x55e2f3e698a0 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.622+0000 7f99e3722740 1 -- 172.21.15.18:0/1875164578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d4003b70 con 0x55e2f3e698a0 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.622+0000 7f99bfe00640 1 -- 172.21.15.18:0/1875164578 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9538+0+0 (secure 0 0 0) 0x7f99cc02cab0 con 0x55e2f3e698a0 2024-04-16T17:23:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.623+0000 7f99bfe00640 1 -- 172.21.15.18:0/1875164578 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f99cc05c050 con 0x55e2f3e698a0 2024-04-16T17:23:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.623+0000 7f99e3722740 10 client.30560 fetch_fsmap learned FSMap version 37 2024-04-16T17:23:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.623+0000 7f99e3722740 1 -- 172.21.15.18:0/1875164578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e2f3ee9ec0 con 0x55e2f3e698a0 2024-04-16T17:23:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.623+0000 7f99bfe00640 1 -- 172.21.15.18:0/1875164578 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d4003b70 con 0x55e2f3e698a0 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.624+0000 7f99bfe00640 1 -- 172.21.15.18:0/1875164578 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f99cc03e460 con 0x55e2f3e698a0 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.624+0000 7f99e3722740 10 client.30560 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.624+0000 7f99e3722740 10 client.30560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.624+0000 7f99e3722740 20 client.30560 populate_metadata read hostname 'smithi018' 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.624+0000 7f99e3722740 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2f3eea1b0 0x55e2f3eec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.624+0000 7f99e3722740 4 client.30560 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.624+0000 7f99e3722740 1 -- 172.21.15.18:0/1875164578 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2f3e70660 con 0x55e2f3eea1b0 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.625+0000 7f99d3e00640 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2f3eea1b0 0x55e2f3eec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.625+0000 7f99bfe00640 10 client.30560 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:54.625+0000 7f99d3e00640 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2f3eea1b0 0x55e2f3eec5a0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea994d1a42cdbcf server_cookie=37ed985ebfa76521 in_seq=0 out_seq=0 2024-04-16T17:23:54.913 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:49 smithi018 ceph-mon[31776]: pgmap v610: 129 pgs: 129 active+clean; 423 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 524 op/s 2024-04-16T17:23:54.913 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:51 smithi018 ceph-mon[31776]: pgmap v611: 129 pgs: 129 active+clean; 429 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-04-16T17:23:54.913 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:53 smithi018 ceph-mon[31776]: pgmap v612: 129 pgs: 129 active+clean; 429 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-04-16T17:23:55.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:54 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:23:55.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:54 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:23:55.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:54 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:23:55.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:54 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:23:55.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:54 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:23:55.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:54 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:23:55.551 INFO:tasks.workunit.client.0.smithi018.stdout: 10 86689 11.94 MB/sec execute 247 sec latency 744.289 ms 2024-04-16T17:23:56.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:55 smithi022 ceph-mon[36023]: pgmap v613: 129 pgs: 129 active+clean; 432 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 364 op/s 2024-04-16T17:23:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.173+0000 7f99bfe00640 1 -- 172.21.15.18:0/1875164578 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2f3e70660 con 0x55e2f3eea1b0 2024-04-16T17:23:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.173+0000 7f99bfe00640 10 client.30560 handle_command_reply: tid=0 2024-04-16T17:23:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.173+0000 7f99e3722740 1 client.30560 shutdown 2024-04-16T17:23:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.174+0000 7f99e3722740 1 -- 172.21.15.18:0/1875164578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e698a0 msgr2=0x55e2f3e6caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.174+0000 7f99e3722740 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6caa0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x55e2f3e65490 tx=0x7f99cc005440 comp rx=0 tx=0).stop 2024-04-16T17:23:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.174+0000 7f99e3722740 1 -- 172.21.15.18:0/1875164578 shutdown_connections 2024-04-16T17:23:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.174+0000 7f99e3722740 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55e2f3eea1b0 0x55e2f3eec5a0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.174+0000 7f99e3722740 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e2f3e72e20 0x55e2f3e6d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.174+0000 7f99e3722740 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2f3e70810 0x55e2f3e6cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.174+0000 7f99e3722740 1 --2- 172.21.15.18:0/1875164578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e2f3e698a0 0x55e2f3e6caa0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.174+0000 7f99e3722740 1 -- 172.21.15.18:0/1875164578 >> 172.21.15.18:0/1875164578 conn(0x55e2f3e045d0 msgr2=0x55e2f3e65970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.175+0000 7f99bfe00640 0 client.30560 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.175+0000 7f99e3722740 1 -- 172.21.15.18:0/1875164578 shutdown_connections 2024-04-16T17:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.175+0000 7f99e3722740 1 -- 172.21.15.18:0/1875164578 wait complete. 2024-04-16T17:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.175+0000 7f99e3722740 20 client.30560 trim_cache size 0 max 16384 2024-04-16T17:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:23:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.176+0000 7f99e0e00640 1 -- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99c00a3e10 msgr2=0x7f99c00a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.176+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99c00a3e10 0x7f99c00a62d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55e2f3e65720 tx=0x7f99d4009600 comp rx=0 tx=0).stop 2024-04-16T17:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.177+0000 7f99e0e00640 1 -- 172.21.15.18:0/3110224530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc107320 msgr2=0x7f99dc109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:23:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.177+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc107320 0x7f99dc109450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f99c400e9c0 tx=0x7f99c400ee80 comp rx=0 tx=0).stop 2024-04-16T17:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.178+0000 7f99e0e00640 1 -- 172.21.15.18:0/3110224530 shutdown_connections 2024-04-16T17:23:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.178+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f99c00a3e10 0x7f99c00a62d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.178+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99dc109990 0x7f99dc1ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.178+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc107320 0x7f99dc109450 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.178+0000 7f99e0e00640 1 --2- 172.21.15.18:0/3110224530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99dc102250 0x7f99dc108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.178+0000 7f99e0e00640 1 -- 172.21.15.18:0/3110224530 >> 172.21.15.18:0/3110224530 conn(0x7f99dc0fda90 msgr2=0x7f99dc0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.179+0000 7f99e0e00640 1 -- 172.21.15.18:0/3110224530 shutdown_connections 2024-04-16T17:23:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:23:56.179+0000 7f99e0e00640 1 -- 172.21.15.18:0/3110224530 wait complete. 2024-04-16T17:23:56.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:55 smithi043 ceph-mon[36901]: pgmap v613: 129 pgs: 129 active+clean; 432 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 364 op/s 2024-04-16T17:23:56.552 INFO:tasks.workunit.client.0.smithi018.stdout: 10 86916 11.93 MB/sec execute 248 sec latency 760.985 ms 2024-04-16T17:23:56.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:54 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:23:56.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:54 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:23:56.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:54 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:23:56.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:55 smithi018 ceph-mon[31776]: pgmap v613: 129 pgs: 129 active+clean; 432 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 364 op/s 2024-04-16T17:23:57.552 INFO:tasks.workunit.client.0.smithi018.stdout: 10 87138 11.93 MB/sec execute 249 sec latency 491.520 ms 2024-04-16T17:23:58.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:57 smithi022 ceph-mon[36023]: pgmap v614: 129 pgs: 129 active+clean; 443 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-04-16T17:23:58.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:57 smithi043 ceph-mon[36901]: pgmap v614: 129 pgs: 129 active+clean; 443 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-04-16T17:23:58.559 INFO:tasks.workunit.client.0.smithi018.stdout: 10 87374 11.93 MB/sec execute 250 sec latency 248.487 ms 2024-04-16T17:23:59.559 INFO:tasks.workunit.client.0.smithi018.stdout: 10 87695 11.97 MB/sec execute 251 sec latency 323.315 ms 2024-04-16T17:23:59.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:57 smithi018 ceph-mon[31776]: pgmap v614: 129 pgs: 129 active+clean; 443 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-04-16T17:24:00.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:23:59 smithi022 ceph-mon[36023]: pgmap v615: 129 pgs: 129 active+clean; 439 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-04-16T17:24:00.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:23:59 smithi043 ceph-mon[36901]: pgmap v615: 129 pgs: 129 active+clean; 439 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-04-16T17:24:00.559 INFO:tasks.workunit.client.0.smithi018.stdout: 10 87955 11.97 MB/sec execute 252 sec latency 267.680 ms 2024-04-16T17:24:01.560 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88155 11.94 MB/sec execute 253 sec latency 232.816 ms 2024-04-16T17:24:02.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:01 smithi022 ceph-mon[36023]: pgmap v616: 129 pgs: 129 active+clean; 433 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 520 op/s 2024-04-16T17:24:02.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:01 smithi043 ceph-mon[36901]: pgmap v616: 129 pgs: 129 active+clean; 433 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 520 op/s 2024-04-16T17:24:02.560 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88357 11.92 MB/sec execute 254 sec latency 630.975 ms 2024-04-16T17:24:03.560 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88533 11.89 MB/sec execute 255 sec latency 716.454 ms 2024-04-16T17:24:03.588 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:23:59 smithi018 ceph-mon[31776]: pgmap v615: 129 pgs: 129 active+clean; 439 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-04-16T17:24:03.588 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:01 smithi018 ceph-mon[31776]: pgmap v616: 129 pgs: 129 active+clean; 433 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 520 op/s 2024-04-16T17:24:04.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:03 smithi022 ceph-mon[36023]: pgmap v617: 129 pgs: 129 active+clean; 433 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 458 op/s 2024-04-16T17:24:04.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:03 smithi043 ceph-mon[36901]: pgmap v617: 129 pgs: 129 active+clean; 433 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 458 op/s 2024-04-16T17:24:04.560 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88558 11.85 MB/sec execute 256 sec latency 604.567 ms 2024-04-16T17:24:04.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:03 smithi018 ceph-mon[31776]: pgmap v617: 129 pgs: 129 active+clean; 433 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 458 op/s 2024-04-16T17:24:05.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88581 11.81 MB/sec execute 257 sec latency 790.028 ms 2024-04-16T17:24:06.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:05 smithi022 ceph-mon[36023]: pgmap v618: 129 pgs: 129 active+clean; 432 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 471 op/s 2024-04-16T17:24:06.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:05 smithi022 ceph-mon[36023]: osdmap e130: 12 total, 12 up, 12 in 2024-04-16T17:24:06.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:05 smithi043 ceph-mon[36901]: pgmap v618: 129 pgs: 129 active+clean; 432 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 471 op/s 2024-04-16T17:24:06.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:05 smithi043 ceph-mon[36901]: osdmap e130: 12 total, 12 up, 12 in 2024-04-16T17:24:06.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88615 11.78 MB/sec execute 258 sec latency 1623.524 ms 2024-04-16T17:24:07.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:05 smithi018 ceph-mon[31776]: pgmap v618: 129 pgs: 129 active+clean; 432 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 471 op/s 2024-04-16T17:24:07.027 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:05 smithi018 ceph-mon[31776]: osdmap e130: 12 total, 12 up, 12 in 2024-04-16T17:24:07.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88631 11.74 MB/sec execute 259 sec latency 2271.403 ms 2024-04-16T17:24:08.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:07 smithi043 ceph-mon[36901]: pgmap v620: 129 pgs: 15 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 79 active+clean; 439 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-04-16T17:24:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:07 smithi022 ceph-mon[36023]: pgmap v620: 129 pgs: 15 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 79 active+clean; 439 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-04-16T17:24:08.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88676 11.70 MB/sec execute 260 sec latency 2693.022 ms 2024-04-16T17:24:08.584 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:07 smithi018 ceph-mon[31776]: pgmap v620: 129 pgs: 15 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 79 active+clean; 439 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-04-16T17:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.398+0000 7ff28be00640 1 -- 172.21.15.18:0/2136008928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 msgr2=0x7ff28c100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.398+0000 7ff28be00640 1 --2- 172.21.15.18:0/2136008928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 0x7ff28c100fd0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff278009900 tx=0x7ff27802bc80 comp rx=0 tx=0).stop 2024-04-16T17:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.399+0000 7ff28be00640 1 -- 172.21.15.18:0/2136008928 shutdown_connections 2024-04-16T17:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.399+0000 7ff28be00640 1 --2- 172.21.15.18:0/2136008928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff28c10e7a0 0x7ff28c110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.399+0000 7ff28be00640 1 --2- 172.21.15.18:0/2136008928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff28c1015a0 0x7ff28c10e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.399+0000 7ff28be00640 1 --2- 172.21.15.18:0/2136008928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 0x7ff28c100fd0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.399+0000 7ff28be00640 1 -- 172.21.15.18:0/2136008928 >> 172.21.15.18:0/2136008928 conn(0x7ff28c0fc7e0 msgr2=0x7ff28c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.399+0000 7ff28be00640 1 -- 172.21.15.18:0/2136008928 shutdown_connections 2024-04-16T17:24:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.399+0000 7ff28be00640 1 -- 172.21.15.18:0/2136008928 wait complete. 2024-04-16T17:24:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.400+0000 7ff28be00640 1 Processor -- start 2024-04-16T17:24:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.403+0000 7ff28be00640 1 -- start start 2024-04-16T17:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.404+0000 7ff28be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 0x7ff28c107700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.404+0000 7ff28be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff28c1015a0 0x7ff28c107c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.404+0000 7ff28be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff28c10e7a0 0x7ff28c102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c114560 con 0x7ff28c1015a0 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 0x7ff28c107700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 0x7ff28c107700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40530/0 (socket says 172.21.15.18:40530) 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28aa00640 1 -- 172.21.15.18:0/1496483247 learned_addr learned my addr 172.21.15.18:0/1496483247 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1143e0 con 0x7ff28c10e7a0 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1146e0 con 0x7ff28c100bf0 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28b400640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff28c10e7a0 0x7ff28c102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28aa00640 1 -- 172.21.15.18:0/1496483247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff28c10e7a0 msgr2=0x7ff28c102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.405+0000 7ff28aa00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff28c10e7a0 0x7ff28c102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.406+0000 7ff28aa00640 1 -- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff28c1015a0 msgr2=0x7ff28c107c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.406+0000 7ff28aa00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff28c1015a0 0x7ff28c107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.406+0000 7ff28aa00640 1 -- 172.21.15.18:0/1496483247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c1035a0 con 0x7ff28c100bf0 2024-04-16T17:24:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.407+0000 7ff28aa00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 0x7ff28c107700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff278005ab0 tx=0x7ff27802c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.407+0000 7ff283400640 1 -- 172.21.15.18:0/1496483247 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278041070 con 0x7ff28c100bf0 2024-04-16T17:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.408+0000 7ff283400640 1 -- 172.21.15.18:0/1496483247 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2780054e0 con 0x7ff28c100bf0 2024-04-16T17:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.408+0000 7ff283400640 1 -- 172.21.15.18:0/1496483247 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278034440 con 0x7ff28c100bf0 2024-04-16T17:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.408+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c103830 con 0x7ff28c100bf0 2024-04-16T17:24:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.408+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c10d610 con 0x7ff28c100bf0 2024-04-16T17:24:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.412+0000 7ff29482d740 1 Processor -- start 2024-04-16T17:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.414+0000 7ff29482d740 1 -- start start 2024-04-16T17:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.414+0000 7ff29482d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc777f0 0x55637fc76540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.414+0000 7ff29482d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc76f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.414+0000 7ff29482d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 0x55637fc823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.414+0000 7ff29482d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55637fbdb780 con 0x55637fc80000 2024-04-16T17:24:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.414+0000 7ff29482d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55637fb81f40 con 0x55637fc76a80 2024-04-16T17:24:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.414+0000 7ff29482d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55637fbd7740 con 0x55637fc777f0 2024-04-16T17:24:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc777f0 0x55637fc76540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc777f0 0x55637fc76540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40534/0 (socket says 172.21.15.18:40534) 2024-04-16T17:24:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28b400640 1 -- 172.21.15.18:0/2972355575 learned_addr learned my addr 172.21.15.18:0/2972355575 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28a000640 1 --2- 172.21.15.18:0/2972355575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 0x55637fc823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28b400640 1 -- 172.21.15.18:0/2972355575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 msgr2=0x55637fc76f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28b400640 1 --2- 172.21.15.18:0/2972355575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc76f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28b400640 1 -- 172.21.15.18:0/2972355575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 msgr2=0x55637fc823f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28b400640 1 --2- 172.21.15.18:0/2972355575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 0x55637fc823f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.416+0000 7ff28b400640 1 -- 172.21.15.18:0/2972355575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637f8ee210 con 0x55637fc777f0 2024-04-16T17:24:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.417+0000 7ff283400640 1 -- 172.21.15.18:0/1496483247 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff278044070 con 0x7ff28c100bf0 2024-04-16T17:24:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.419+0000 7ff283400640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2700a3d00 0x7ff2700a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.419+0000 7ff28b400640 1 --2- 172.21.15.18:0/2972355575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc777f0 0x55637fc76540 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff28400b740 tx=0x7ff28400bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3f6502e50628b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.420+0000 7ff283400640 1 -- 172.21.15.18:0/1496483247 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7ff2780f2870 con 0x7ff28c100bf0 2024-04-16T17:24:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.420+0000 7ff277e00640 1 -- 172.21.15.18:0/2972355575 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28400be50 con 0x55637fc777f0 2024-04-16T17:24:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.421+0000 7ff277e00640 1 -- 172.21.15.18:0/2972355575 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff284019070 con 0x55637fc777f0 2024-04-16T17:24:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.421+0000 7ff277e00640 1 -- 172.21.15.18:0/2972355575 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284014e20 con 0x55637fc777f0 2024-04-16T17:24:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.422+0000 7ff29482d740 1 -- 172.21.15.18:0/2972355575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc777f0 msgr2=0x55637fc76540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.422+0000 7ff29482d740 1 --2- 172.21.15.18:0/2972355575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc777f0 0x55637fc76540 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff28400b740 tx=0x7ff28400bc00 comp rx=0 tx=0).stop 2024-04-16T17:24:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.422+0000 7ff29482d740 1 -- 172.21.15.18:0/2972355575 shutdown_connections 2024-04-16T17:24:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.422+0000 7ff29482d740 1 --2- 172.21.15.18:0/2972355575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 0x55637fc823f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.422+0000 7ff29482d740 1 --2- 172.21.15.18:0/2972355575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc76f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.422+0000 7ff29482d740 1 --2- 172.21.15.18:0/2972355575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc777f0 0x55637fc76540 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.422+0000 7ff29482d740 1 -- 172.21.15.18:0/2972355575 >> 172.21.15.18:0/2972355575 conn(0x55637fc115d0 msgr2=0x55637fc71770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.422+0000 7ff28aa00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2700a3d00 0x7ff2700a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.423+0000 7ff29482d740 1 -- 172.21.15.18:0/2972355575 shutdown_connections 2024-04-16T17:24:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.423+0000 7ff29482d740 1 -- 172.21.15.18:0/2972355575 wait complete. 2024-04-16T17:24:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.424+0000 7ff29482d740 1 -- start start 2024-04-16T17:24:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.424+0000 7ff29482d740 1 Processor -- start 2024-04-16T17:24:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.424+0000 7ff28aa00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2700a3d00 0x7ff2700a61c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55637fc77f50 tx=0x7ff278004f10 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff29482d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc76a80 0x55637fc7cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff29482d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc777f0 0x55637fc7d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff29482d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 0x55637fc7d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff29482d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55637fc82d30 con 0x55637fc80000 2024-04-16T17:24:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff29482d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55637fb95390 con 0x55637fc777f0 2024-04-16T17:24:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff29482d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55637fc82eb0 con 0x55637fc76a80 2024-04-16T17:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff28a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc777f0 0x55637fc7d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff28b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc76a80 0x55637fc7cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff28a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc777f0 0x55637fc7d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54798/0 (socket says 172.21.15.18:54798) 2024-04-16T17:24:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.425+0000 7ff29482d740 10 client.0 _ll_register_callbacks cb 0x55637fb7ef60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.426+0000 7ff28b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc76a80 0x55637fc7cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40538/0 (socket says 172.21.15.18:40538) 2024-04-16T17:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.426+0000 7ff28a000640 1 -- 172.21.15.18:0/2649727004 learned_addr learned my addr 172.21.15.18:0/2649727004 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.426+0000 7ff28b400640 1 -- 172.21.15.18:0/2649727004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc777f0 msgr2=0x55637fc7d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.426+0000 7ff28b400640 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc777f0 0x55637fc7d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.426+0000 7ff28b400640 1 -- 172.21.15.18:0/2649727004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 msgr2=0x55637fc7d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.426+0000 7ff28b400640 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 0x55637fc7d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.426+0000 7ff28b400640 1 -- 172.21.15.18:0/2649727004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637f9ba620 con 0x55637fc76a80 2024-04-16T17:24:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.427+0000 7ff28b400640 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc76a80 0x55637fc7cec0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff28400b740 tx=0x7ff2840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c23bfe9cd8b4ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.427+0000 7ff274c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.427+0000 7ff274c00640 1 -- 172.21.15.18:0/2649727004 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28400db50 con 0x55637fc76a80 2024-04-16T17:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.427+0000 7ff29482d740 1 -- 172.21.15.18:0/2649727004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff284019070 con 0x55637fc76a80 2024-04-16T17:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.427+0000 7ff274c00640 1 -- 172.21.15.18:0/2649727004 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7ff28400dcf0 con 0x55637fc76a80 2024-04-16T17:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.428+0000 7ff274c00640 1 -- 172.21.15.18:0/2649727004 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff28400be50 con 0x55637fc76a80 2024-04-16T17:24:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.428+0000 7ff29482d740 10 client.30433 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.428+0000 7ff29482d740 1 -- 172.21.15.18:0/2649727004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55637fc8eea0 con 0x55637fc76a80 2024-04-16T17:24:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.428+0000 7ff274c00640 1 -- 172.21.15.18:0/2649727004 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff284022070 con 0x55637fc76a80 2024-04-16T17:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.430+0000 7ff274c00640 1 -- 172.21.15.18:0/2649727004 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff284013070 con 0x55637fc76a80 2024-04-16T17:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.430+0000 7ff29482d740 10 client.30433 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.430+0000 7ff29482d740 10 client.30433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.430+0000 7ff29482d740 20 client.30433 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.430+0000 7ff29482d740 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55637fc902b0 0x55637fc926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.430+0000 7ff29482d740 4 client.30433 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.430+0000 7ff29482d740 1 -- 172.21.15.18:0/2649727004 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff284019070 con 0x55637fc902b0 2024-04-16T17:24:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.431+0000 7ff28a000640 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55637fc902b0 0x55637fc926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.432+0000 7ff28a000640 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55637fc902b0 0x55637fc926a0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5658044001d8a6 server_cookie=a1a91cd944a814bd in_seq=0 out_seq=0 2024-04-16T17:24:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.434+0000 7ff274c00640 10 client.30433 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.440+0000 7ff274c00640 1 -- 172.21.15.18:0/2649727004 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff284019070 con 0x55637fc902b0 2024-04-16T17:24:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.440+0000 7ff274c00640 10 client.30433 handle_command_reply: tid=0 2024-04-16T17:24:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.441+0000 7ff29482d740 1 client.30433 shutdown 2024-04-16T17:24:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.441+0000 7ff29482d740 1 -- 172.21.15.18:0/2649727004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc76a80 msgr2=0x55637fc7cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.441+0000 7ff29482d740 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc76a80 0x55637fc7cec0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff28400b740 tx=0x7ff2840046c0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.441+0000 7ff29482d740 1 -- 172.21.15.18:0/2649727004 shutdown_connections 2024-04-16T17:24:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.441+0000 7ff29482d740 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55637fc902b0 0x55637fc926a0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.441+0000 7ff29482d740 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc80000 0x55637fc7d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.442+0000 7ff29482d740 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc777f0 0x55637fc7d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.442+0000 7ff29482d740 1 --2- 172.21.15.18:0/2649727004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc76a80 0x55637fc7cec0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.442+0000 7ff29482d740 1 -- 172.21.15.18:0/2649727004 >> 172.21.15.18:0/2649727004 conn(0x55637fc115d0 msgr2=0x55637fc734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.442+0000 7ff274c00640 0 client.30433 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.442+0000 7ff29482d740 1 -- 172.21.15.18:0/2649727004 shutdown_connections 2024-04-16T17:24:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.442+0000 7ff29482d740 1 -- 172.21.15.18:0/2649727004 wait complete. 2024-04-16T17:24:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.443+0000 7ff29482d740 20 client.30433 trim_cache size 0 max 16384 2024-04-16T17:24:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.481+0000 7ff29482d740 1 Processor -- start 2024-04-16T17:24:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.483+0000 7ff29482d740 1 -- start start 2024-04-16T17:24:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.483+0000 7ff29482d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc7f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.483+0000 7ff29482d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 0x55637fc8bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.483+0000 7ff29482d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 0x55637fc7d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.483+0000 7ff29482d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1c0d90 con 0x55637fc777f0 2024-04-16T17:24:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.483+0000 7ff29482d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1be660 con 0x55637fc76a80 2024-04-16T17:24:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.483+0000 7ff29482d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1bc4d0 con 0x55637fc80000 2024-04-16T17:24:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.483+0000 7ff28b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc7f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc7f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54800/0 (socket says 172.21.15.18:54800) 2024-04-16T17:24:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 -- 172.21.15.18:0/2777186132 learned_addr learned my addr 172.21.15.18:0/2777186132 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 --2- 172.21.15.18:0/2777186132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 0x55637fc7d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 -- 172.21.15.18:0/2777186132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 msgr2=0x55637fc7d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 --2- 172.21.15.18:0/2777186132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 0x55637fc7d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 -- 172.21.15.18:0/2777186132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 msgr2=0x55637fc8bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 --2- 172.21.15.18:0/2777186132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 0x55637fc8bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 -- 172.21.15.18:0/2777186132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637fc82850 con 0x55637fc76a80 2024-04-16T17:24:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.484+0000 7ff28b400640 1 --2- 172.21.15.18:0/2777186132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc7f5f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff28400b1e0 tx=0x7ff284014a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1de82f8278e08e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.489+0000 7ff277400640 1 -- 172.21.15.18:0/2777186132 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284014790 con 0x55637fc76a80 2024-04-16T17:24:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.489+0000 7ff277400640 1 -- 172.21.15.18:0/2777186132 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff284022040 con 0x55637fc76a80 2024-04-16T17:24:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.489+0000 7ff277400640 1 -- 172.21.15.18:0/2777186132 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284027bd0 con 0x55637fc76a80 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.490+0000 7ff29482d740 1 -- 172.21.15.18:0/2777186132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 msgr2=0x55637fc7f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.490+0000 7ff29482d740 1 --2- 172.21.15.18:0/2777186132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc7f5f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff28400b1e0 tx=0x7ff284014a00 comp rx=0 tx=0).stop 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.491+0000 7ff29482d740 1 -- 172.21.15.18:0/2777186132 shutdown_connections 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.491+0000 7ff29482d740 1 --2- 172.21.15.18:0/2777186132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 0x55637fc7d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.491+0000 7ff29482d740 1 --2- 172.21.15.18:0/2777186132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 0x55637fc8bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.491+0000 7ff29482d740 1 --2- 172.21.15.18:0/2777186132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc7f5f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.491+0000 7ff29482d740 1 -- 172.21.15.18:0/2777186132 >> 172.21.15.18:0/2777186132 conn(0x55637fc115d0 msgr2=0x55637fc7bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.491+0000 7ff29482d740 1 -- 172.21.15.18:0/2777186132 shutdown_connections 2024-04-16T17:24:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.491+0000 7ff29482d740 1 -- 172.21.15.18:0/2777186132 wait complete. 2024-04-16T17:24:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.491+0000 7ff29482d740 1 -- start start 2024-04-16T17:24:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.492+0000 7ff29482d740 1 Processor -- start 2024-04-16T17:24:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.492+0000 7ff29482d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.492+0000 7ff29482d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 0x55637fc798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.493+0000 7ff29482d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 0x55637fc79df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.493+0000 7ff29482d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1bcb20 con 0x55637fc777f0 2024-04-16T17:24:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.493+0000 7ff29482d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1bdf90 con 0x55637fc76a80 2024-04-16T17:24:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.493+0000 7ff29482d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1bd1f0 con 0x55637fc80000 2024-04-16T17:24:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.493+0000 7ff29482d740 10 client.0 _ll_register_callbacks cb 0x7ff28400b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.493+0000 7ff28b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.493+0000 7ff28b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc884f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54816/0 (socket says 172.21.15.18:54816) 2024-04-16T17:24:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.493+0000 7ff28b400640 1 -- 172.21.15.18:0/1045746311 learned_addr learned my addr 172.21.15.18:0/1045746311 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28a000640 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 0x55637fc798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28b400640 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 0x55637fc79df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28b400640 1 -- 172.21.15.18:0/1045746311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 msgr2=0x55637fc79df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28b400640 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 0x55637fc79df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28b400640 1 -- 172.21.15.18:0/1045746311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 msgr2=0x55637fc798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28b400640 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 0x55637fc798b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28b400640 1 -- 172.21.15.18:0/1045746311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637fc7a620 con 0x55637fc76a80 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28a000640 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 0x55637fc798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff28b400640 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc884f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x55637fc76a50 tx=0x7ff284014a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef78ddba126f703 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.494+0000 7ff274c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.495+0000 7ff274c00640 1 -- 172.21.15.18:0/1045746311 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28403a060 con 0x55637fc76a80 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.495+0000 7ff274c00640 1 -- 172.21.15.18:0/1045746311 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7ff284027680 con 0x55637fc76a80 2024-04-16T17:24:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.495+0000 7ff29482d740 1 -- 172.21.15.18:0/1045746311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff284022040 con 0x55637fc76a80 2024-04-16T17:24:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.495+0000 7ff274c00640 1 -- 172.21.15.18:0/1045746311 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff284041050 con 0x55637fc76a80 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.495+0000 7ff29482d740 10 client.30572 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.495+0000 7ff29482d740 1 -- 172.21.15.18:0/1045746311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55637fcf6490 con 0x55637fc76a80 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.496+0000 7ff274c00640 1 -- 172.21.15.18:0/1045746311 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff284022040 con 0x55637fc76a80 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.496+0000 7ff274c00640 1 -- 172.21.15.18:0/1045746311 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff2840356e0 con 0x55637fc76a80 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.496+0000 7ff29482d740 10 client.30572 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.496+0000 7ff29482d740 10 client.30572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.496+0000 7ff29482d740 20 client.30572 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.496+0000 7ff29482d740 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55637fcf6ab0 0x55637fcf8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.496+0000 7ff29482d740 4 client.30572 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:24:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.496+0000 7ff29482d740 1 -- 172.21.15.18:0/1045746311 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55637fbe91a0 con 0x55637fcf6ab0 2024-04-16T17:24:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.497+0000 7ff28a000640 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55637fcf6ab0 0x55637fcf8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.498+0000 7ff28a000640 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55637fcf6ab0 0x55637fcf8ed0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fbc4b5286faa7cb server_cookie=23dfc30f5f7fb76e in_seq=0 out_seq=0 2024-04-16T17:24:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:09.498+0000 7ff274c00640 10 client.30572 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:09.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88699 11.66 MB/sec execute 261 sec latency 2919.778 ms 2024-04-16T17:24:10.076 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:09 smithi018 ceph-mon[31776]: pgmap v621: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-04-16T17:24:10.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:24:10.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:24:10.224 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:09 smithi022 ceph-mon[36023]: pgmap v621: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-04-16T17:24:10.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:09 smithi043 ceph-mon[36901]: pgmap v621: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-04-16T17:24:10.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.530+0000 7f7680a00640 1 -- 172.21.15.18:0/4158498371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 msgr2=0x7f767c10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.530+0000 7f7680a00640 1 --2- 172.21.15.18:0/4158498371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 0x7f767c10e560 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f766c009900 tx=0x7f766c02bc60 comp rx=0 tx=0).stop 2024-04-16T17:24:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.533+0000 7f7680a00640 1 -- 172.21.15.18:0/4158498371 shutdown_connections 2024-04-16T17:24:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.533+0000 7f7680a00640 1 --2- 172.21.15.18:0/4158498371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f767c10ec00 0x7f767c11d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.533+0000 7f7680a00640 1 --2- 172.21.15.18:0/4158498371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 0x7f767c10e560 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.533+0000 7f7680a00640 1 --2- 172.21.15.18:0/4158498371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767c110580 0x7f767c110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.533+0000 7f7680a00640 1 -- 172.21.15.18:0/4158498371 >> 172.21.15.18:0/4158498371 conn(0x7f767c06cf10 msgr2=0x7f767c06d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:10.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.533+0000 7f7680a00640 1 -- 172.21.15.18:0/4158498371 shutdown_connections 2024-04-16T17:24:10.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.540+0000 7f7680a00640 1 -- 172.21.15.18:0/4158498371 wait complete. 2024-04-16T17:24:10.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.542+0000 7f7680a00640 1 Processor -- start 2024-04-16T17:24:10.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.548+0000 7f7680a00640 1 -- start start 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f7680a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 0x7f767c1a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f7680a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f767c10ec00 0x7f767c1a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f7680a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767c110580 0x7f767c1abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f7680a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f767c121080 con 0x7f767c10ec00 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f7680a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f767c120f00 con 0x7f767c10e0e0 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f7680a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f767c121200 con 0x7f767c110580 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f767a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 0x7f767c1a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f767a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 0x7f767c1a7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54826/0 (socket says 172.21.15.18:54826) 2024-04-16T17:24:10.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.549+0000 7f767a800640 1 -- 172.21.15.18:0/3800202119 learned_addr learned my addr 172.21.15.18:0/3800202119 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:10.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.550+0000 7f7679e00640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f767c10ec00 0x7f767c1a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.550+0000 7f767a800640 1 -- 172.21.15.18:0/3800202119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767c110580 msgr2=0x7f767c1abdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:24:10.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.550+0000 7f767a800640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767c110580 0x7f767c1abdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.550+0000 7f767a800640 1 -- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f767c10ec00 msgr2=0x7f767c1a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.550+0000 7f767a800640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f767c10ec00 0x7f767c1a7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.550+0000 7f767a800640 1 -- 172.21.15.18:0/3800202119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c1ac4f0 con 0x7f767c10e0e0 2024-04-16T17:24:10.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.550+0000 7f767a800640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 0x7f767c1a7500 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f767400cce0 tx=0x7f767400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:10.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.551+0000 7f7673400640 1 -- 172.21.15.18:0/3800202119 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767400f650 con 0x7f767c10e0e0 2024-04-16T17:24:10.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.551+0000 7f7673400640 1 -- 172.21.15.18:0/3800202119 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f767400fcd0 con 0x7f767c10e0e0 2024-04-16T17:24:10.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.551+0000 7f7673400640 1 -- 172.21.15.18:0/3800202119 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674018c70 con 0x7f767c10e0e0 2024-04-16T17:24:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.552+0000 7f7680a00640 1 -- 172.21.15.18:0/3800202119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c1ac7e0 con 0x7f767c10e0e0 2024-04-16T17:24:10.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.552+0000 7f7680a00640 1 -- 172.21.15.18:0/3800202119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c1b4070 con 0x7f767c10e0e0 2024-04-16T17:24:10.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.560+0000 7f76841b3740 1 Processor -- start 2024-04-16T17:24:10.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88705 11.62 MB/sec execute 262 sec latency 2578.565 ms 2024-04-16T17:24:10.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.574+0000 7f7673400640 1 -- 172.21.15.18:0/3800202119 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f767400f7f0 con 0x7f767c10e0e0 2024-04-16T17:24:10.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.576+0000 7f7673400640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76600a3e10 0x7f76600a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.576+0000 7f7673400640 1 -- 172.21.15.18:0/3800202119 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f76740d11b0 con 0x7f767c10e0e0 2024-04-16T17:24:10.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.576+0000 7f76841b3740 1 -- start start 2024-04-16T17:24:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.582+0000 7f76841b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a4917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.582+0000 7f76841b3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a491ce0 0x559d0a49a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.582+0000 7f76841b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a49a8c0 0x559d0a49ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.582+0000 7f76841b3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559d0a3f7120 con 0x559d0a491ce0 2024-04-16T17:24:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.582+0000 7f76841b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559d0a39d340 con 0x559d0a4913c0 2024-04-16T17:24:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.582+0000 7f76841b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d0a3f30e0 con 0x559d0a49a8c0 2024-04-16T17:24:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.582+0000 7f767b200640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76600a3e10 0x7f76600a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a4917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a4917a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54828/0 (socket says 172.21.15.18:54828) 2024-04-16T17:24:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 -- 172.21.15.18:0/3499120902 learned_addr learned my addr 172.21.15.18:0/3499120902 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 -- 172.21.15.18:0/3499120902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a49a8c0 msgr2=0x559d0a49ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 --2- 172.21.15.18:0/3499120902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a49a8c0 0x559d0a49ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 -- 172.21.15.18:0/3499120902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a491ce0 msgr2=0x559d0a49a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 --2- 172.21.15.18:0/3499120902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a491ce0 0x559d0a49a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 -- 172.21.15.18:0/3499120902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0a1d37d0 con 0x559d0a4913c0 2024-04-16T17:24:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f767a800640 1 --2- 172.21.15.18:0/3499120902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a4917a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f76740a53d0 tx=0x7f767409cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0bf1e75c3d7a08a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:10.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f7667400640 1 -- 172.21.15.18:0/3499120902 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76740b6070 con 0x559d0a4913c0 2024-04-16T17:24:10.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f7667400640 1 -- 172.21.15.18:0/3499120902 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76740bbe50 con 0x559d0a4913c0 2024-04-16T17:24:10.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f7667400640 1 -- 172.21.15.18:0/3499120902 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674018e10 con 0x559d0a4913c0 2024-04-16T17:24:10.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f76841b3740 1 -- 172.21.15.18:0/3499120902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 msgr2=0x559d0a4917a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.584+0000 7f76841b3740 1 --2- 172.21.15.18:0/3499120902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a4917a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f76740a53d0 tx=0x7f767409cb90 comp rx=0 tx=0).stop 2024-04-16T17:24:10.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.593+0000 7f767b200640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76600a3e10 0x7f76600a62d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f76680097a0 tx=0x7f7668009120 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:10.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.593+0000 7f76841b3740 1 -- 172.21.15.18:0/3499120902 shutdown_connections 2024-04-16T17:24:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.593+0000 7f76841b3740 1 --2- 172.21.15.18:0/3499120902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a49a8c0 0x559d0a49ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.593+0000 7f76841b3740 1 --2- 172.21.15.18:0/3499120902 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a491ce0 0x559d0a49a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.593+0000 7f76841b3740 1 --2- 172.21.15.18:0/3499120902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a4917a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.593+0000 7f76841b3740 1 -- 172.21.15.18:0/3499120902 >> 172.21.15.18:0/3499120902 conn(0x559d0a48baf0 msgr2=0x559d0a48bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 -- 172.21.15.18:0/3499120902 shutdown_connections 2024-04-16T17:24:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 -- 172.21.15.18:0/3499120902 wait complete. 2024-04-16T17:24:10.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 -- start start 2024-04-16T17:24:10.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 Processor -- start 2024-04-16T17:24:10.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a4913c0 0x559d0a4979a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 0x559d0a497ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a498420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559d0a49d6b0 con 0x559d0a4913c0 2024-04-16T17:24:10.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559d0a3b0710 con 0x559d0a49a8c0 2024-04-16T17:24:10.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d0a49d830 con 0x559d0a491ce0 2024-04-16T17:24:10.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.594+0000 7f76841b3740 10 client.0 _ll_register_callbacks cb 0x559d0a3abaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:10.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.600+0000 7f767a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a4913c0 0x559d0a4979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.600+0000 7f767a800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a4913c0 0x559d0a4979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37304/0 (socket says 172.21.15.18:37304) 2024-04-16T17:24:10.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.600+0000 7f767a800640 1 -- 172.21.15.18:0/2193554378 learned_addr learned my addr 172.21.15.18:0/2193554378 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:10.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.600+0000 7f767a800640 1 -- 172.21.15.18:0/2193554378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 msgr2=0x559d0a497ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:10.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.600+0000 7f767a800640 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 0x559d0a497ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.600+0000 7f767a800640 1 -- 172.21.15.18:0/2193554378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 msgr2=0x559d0a498420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:10.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.600+0000 7f767a800640 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a498420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f767a800640 1 -- 172.21.15.18:0/2193554378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0a1093f0 con 0x559d0a4913c0 2024-04-16T17:24:10.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f767a800640 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a4913c0 0x559d0a4979a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f767409af40 tx=0x7f76740184f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94eb04554f65674 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:10.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f765be00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:24:10.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f765be00640 1 -- 172.21.15.18:0/2193554378 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76740ba020 con 0x559d0a4913c0 2024-04-16T17:24:10.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f765be00640 1 -- 172.21.15.18:0/2193554378 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f76740bd0f0 con 0x559d0a4913c0 2024-04-16T17:24:10.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f76841b3740 1 -- 172.21.15.18:0/2193554378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76740bbe50 con 0x559d0a4913c0 2024-04-16T17:24:10.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f765be00640 1 -- 172.21.15.18:0/2193554378 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f76740b6040 con 0x559d0a4913c0 2024-04-16T17:24:10.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f76841b3740 10 client.19689 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:10.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f76841b3740 1 -- 172.21.15.18:0/2193554378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559d0a0c90f0 con 0x559d0a4913c0 2024-04-16T17:24:10.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f765be00640 1 -- 172.21.15.18:0/2193554378 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76740bbe50 con 0x559d0a4913c0 2024-04-16T17:24:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.602+0000 7f765be00640 1 -- 172.21.15.18:0/2193554378 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f767409cef0 con 0x559d0a4913c0 2024-04-16T17:24:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.607+0000 7f76841b3740 10 client.19689 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.607+0000 7f76841b3740 10 client.19689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.607+0000 7f76841b3740 20 client.19689 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.607+0000 7f76841b3740 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d0a4aab90 0x559d0a4acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.607+0000 7f76841b3740 4 client.19689 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:10.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.607+0000 7f76841b3740 1 -- 172.21.15.18:0/2193554378 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d0a404b40 con 0x559d0a4aab90 2024-04-16T17:24:10.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.611+0000 7f7679e00640 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d0a4aab90 0x559d0a4acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.612+0000 7f7679e00640 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d0a4aab90 0x559d0a4acf80 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49f6f43cc09c2b7 server_cookie=221aee35a3aee23f in_seq=0 out_seq=0 2024-04-16T17:24:10.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.612+0000 7f765be00640 10 client.19689 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.859+0000 7ff274c00640 1 -- 172.21.15.18:0/1045746311 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55637fbe91a0 con 0x55637fcf6ab0 2024-04-16T17:24:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.859+0000 7ff274c00640 10 client.30572 handle_command_reply: tid=0 2024-04-16T17:24:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.859+0000 7ff29482d740 1 client.30572 shutdown 2024-04-16T17:24:10.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.860+0000 7f765be00640 1 -- 172.21.15.18:0/2193554378 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d0a404b40 con 0x559d0a4aab90 2024-04-16T17:24:10.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.860+0000 7f765be00640 10 client.19689 handle_command_reply: tid=0 2024-04-16T17:24:10.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.860+0000 7f76841b3740 1 client.19689 shutdown 2024-04-16T17:24:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.861+0000 7ff29482d740 1 -- 172.21.15.18:0/1045746311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 msgr2=0x55637fc884f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.861+0000 7ff29482d740 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc884f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x55637fc76a50 tx=0x7ff284014a70 comp rx=0 tx=0).stop 2024-04-16T17:24:10.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.861+0000 7f76841b3740 1 -- 172.21.15.18:0/2193554378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a4913c0 msgr2=0x559d0a4979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.861+0000 7f76841b3740 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a4913c0 0x559d0a4979a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f767409af40 tx=0x7f76740184f0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.869+0000 7f76841b3740 1 -- 172.21.15.18:0/2193554378 shutdown_connections 2024-04-16T17:24:10.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.869+0000 7f76841b3740 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d0a4aab90 0x559d0a4acf80 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.869+0000 7f765be00640 0 client.19689 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.870+0000 7ff29482d740 1 -- 172.21.15.18:0/1045746311 shutdown_connections 2024-04-16T17:24:10.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.870+0000 7f76841b3740 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a498420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.870+0000 7ff29482d740 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55637fcf6ab0 0x55637fcf8ed0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.870+0000 7ff29482d740 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55637fc80000 0x55637fc79df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.870+0000 7ff29482d740 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55637fc777f0 0x55637fc798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.870+0000 7ff29482d740 1 --2- 172.21.15.18:0/1045746311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55637fc76a80 0x55637fc884f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.870+0000 7f76841b3740 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 0x559d0a497ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.870+0000 7f76841b3740 1 --2- 172.21.15.18:0/2193554378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a4913c0 0x559d0a4979a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.870+0000 7f76841b3740 1 -- 172.21.15.18:0/2193554378 >> 172.21.15.18:0/2193554378 conn(0x559d0a48baf0 msgr2=0x559d0a48d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.870+0000 7ff29482d740 1 -- 172.21.15.18:0/1045746311 >> 172.21.15.18:0/1045746311 conn(0x55637fc115d0 msgr2=0x55637fc6ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.870+0000 7ff274c00640 0 client.30572 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.871+0000 7ff29482d740 1 -- 172.21.15.18:0/1045746311 shutdown_connections 2024-04-16T17:24:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.871+0000 7ff29482d740 1 -- 172.21.15.18:0/1045746311 wait complete. 2024-04-16T17:24:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.872+0000 7ff29482d740 20 client.30572 trim_cache size 0 max 16384 2024-04-16T17:24:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:24:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:24:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:24:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2700a3d00 msgr2=0x7ff2700a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2700a3d00 0x7ff2700a61c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55637fc77f50 tx=0x7ff278004f10 comp rx=0 tx=0).stop 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 msgr2=0x7ff28c107700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 0x7ff28c107700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff278005ab0 tx=0x7ff27802c760 comp rx=0 tx=0).stop 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 shutdown_connections 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2700a3d00 0x7ff2700a61c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff28c10e7a0 0x7ff28c102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff28c1015a0 0x7ff28c107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 --2- 172.21.15.18:0/1496483247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c100bf0 0x7ff28c107700 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.874+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 >> 172.21.15.18:0/1496483247 conn(0x7ff28c0fc7e0 msgr2=0x7ff28c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.875+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 shutdown_connections 2024-04-16T17:24:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:10.875+0000 7ff28be00640 1 -- 172.21.15.18:0/1496483247 wait complete. 2024-04-16T17:24:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.875+0000 7f76841b3740 1 -- 172.21.15.18:0/2193554378 shutdown_connections 2024-04-16T17:24:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.875+0000 7f76841b3740 1 -- 172.21.15.18:0/2193554378 wait complete. 2024-04-16T17:24:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.875+0000 7f76841b3740 20 client.19689 trim_cache size 0 max 16384 2024-04-16T17:24:10.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.922+0000 7f76841b3740 1 Processor -- start 2024-04-16T17:24:10.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.922+0000 7f76841b3740 1 -- start start 2024-04-16T17:24:10.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.923+0000 7f76841b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a49a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.923+0000 7f76841b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 0x559d0a4a6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.923+0000 7f76841b3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a49a8c0 0x559d0a4affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.924+0000 7f76841b3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f767c1c0d80 con 0x559d0a49a8c0 2024-04-16T17:24:10.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.924+0000 7f76841b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f767c1be650 con 0x559d0a4913c0 2024-04-16T17:24:10.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.924+0000 7f7679e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 0x559d0a4a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.924+0000 7f767a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a49a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.924+0000 7f76841b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f767c1bc4c0 con 0x559d0a491ce0 2024-04-16T17:24:10.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.924+0000 7f7679e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 0x559d0a4a6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40598/0 (socket says 172.21.15.18:40598) 2024-04-16T17:24:10.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.924+0000 7f7679e00640 1 -- 172.21.15.18:0/1136670652 learned_addr learned my addr 172.21.15.18:0/1136670652 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:10.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.924+0000 7f767b200640 1 --2- 172.21.15.18:0/1136670652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a49a8c0 0x559d0a4affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.927+0000 7f767b200640 1 -- 172.21.15.18:0/1136670652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 msgr2=0x559d0a4a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:10.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.927+0000 7f767b200640 1 --2- 172.21.15.18:0/1136670652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 0x559d0a4a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.927+0000 7f767b200640 1 -- 172.21.15.18:0/1136670652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 msgr2=0x559d0a49a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:10.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.927+0000 7f767b200640 1 --2- 172.21.15.18:0/1136670652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a49a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.927+0000 7f767b200640 1 -- 172.21.15.18:0/1136670652 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0a495ef0 con 0x559d0a49a8c0 2024-04-16T17:24:10.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.928+0000 7f767b200640 1 --2- 172.21.15.18:0/1136670652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a49a8c0 0x559d0a4affa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7668006ca0 tx=0x7f7668010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205103f44c71a4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:10.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.970+0000 7f7666a00640 1 -- 172.21.15.18:0/1136670652 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668026070 con 0x559d0a49a8c0 2024-04-16T17:24:10.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.970+0000 7f7666a00640 1 -- 172.21.15.18:0/1136670652 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668016d90 con 0x559d0a49a8c0 2024-04-16T17:24:10.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.970+0000 7f7666a00640 1 -- 172.21.15.18:0/1136670652 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766800f040 con 0x559d0a49a8c0 2024-04-16T17:24:10.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.971+0000 7f76841b3740 1 -- 172.21.15.18:0/1136670652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a49a8c0 msgr2=0x559d0a4affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.971+0000 7f76841b3740 1 --2- 172.21.15.18:0/1136670652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a49a8c0 0x559d0a4affa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7668006ca0 tx=0x7f7668010000 comp rx=0 tx=0).stop 2024-04-16T17:24:10.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.972+0000 7f76841b3740 1 -- 172.21.15.18:0/1136670652 shutdown_connections 2024-04-16T17:24:10.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.972+0000 7f76841b3740 1 --2- 172.21.15.18:0/1136670652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a49a8c0 0x559d0a4affa0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.972+0000 7f76841b3740 1 --2- 172.21.15.18:0/1136670652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a491ce0 0x559d0a4a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.972+0000 7f76841b3740 1 --2- 172.21.15.18:0/1136670652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a4913c0 0x559d0a49a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.972+0000 7f76841b3740 1 -- 172.21.15.18:0/1136670652 >> 172.21.15.18:0/1136670652 conn(0x559d0a48baf0 msgr2=0x559d0a4a5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:10.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.978+0000 7f76841b3740 1 -- 172.21.15.18:0/1136670652 shutdown_connections 2024-04-16T17:24:10.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.978+0000 7f76841b3740 1 -- 172.21.15.18:0/1136670652 wait complete. 2024-04-16T17:24:10.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.978+0000 7f76841b3740 1 -- start start 2024-04-16T17:24:10.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.979+0000 7f76841b3740 1 Processor -- start 2024-04-16T17:24:10.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.979+0000 7f76841b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a4913c0 0x559d0a4b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f76841b3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a491ce0 0x559d0a4ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f76841b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a4ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f76841b3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f767c1bcb10 con 0x559d0a491ce0 2024-04-16T17:24:10.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f76841b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f767c1bdf80 con 0x559d0a49a8c0 2024-04-16T17:24:10.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f76841b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f767c1bd1e0 con 0x559d0a4913c0 2024-04-16T17:24:10.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f76841b3740 10 client.0 _ll_register_callbacks cb 0x7f76740b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:10.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f767b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a4ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f767b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a4ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54846/0 (socket says 172.21.15.18:54846) 2024-04-16T17:24:10.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.980+0000 7f767b200640 1 -- 172.21.15.18:0/684239939 learned_addr learned my addr 172.21.15.18:0/684239939 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:10.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.981+0000 7f767a800640 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a4913c0 0x559d0a4b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.981+0000 7f767a800640 1 -- 172.21.15.18:0/684239939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 msgr2=0x559d0a4ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:10.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.981+0000 7f767a800640 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a4ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.981+0000 7f767a800640 1 -- 172.21.15.18:0/684239939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a491ce0 msgr2=0x559d0a4ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:10.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.981+0000 7f767a800640 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a491ce0 0x559d0a4ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:10.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.981+0000 7f767a800640 1 -- 172.21.15.18:0/684239939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0a4adf40 con 0x559d0a4913c0 2024-04-16T17:24:10.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.981+0000 7f767b200640 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a4ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:10.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.982+0000 7f767a800640 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a4913c0 0x559d0a4b0090 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f767400fef0 tx=0x7f76740cedc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad7ecfa38aeb6eea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:10.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.982+0000 7f765be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:10.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.983+0000 7f765be00640 1 -- 172.21.15.18:0/684239939 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76740ba070 con 0x559d0a4913c0 2024-04-16T17:24:10.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.983+0000 7f765be00640 1 -- 172.21.15.18:0/684239939 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f76740b3cf0 con 0x559d0a4913c0 2024-04-16T17:24:10.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.983+0000 7f76841b3740 1 -- 172.21.15.18:0/684239939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7668016d90 con 0x559d0a4913c0 2024-04-16T17:24:10.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.984+0000 7f765be00640 1 -- 172.21.15.18:0/684239939 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f76740ce8d0 con 0x559d0a4913c0 2024-04-16T17:24:10.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.984+0000 7f76841b3740 10 client.30445 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:10.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.984+0000 7f76841b3740 1 -- 172.21.15.18:0/684239939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559d0a4adf40 con 0x559d0a4913c0 2024-04-16T17:24:10.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.984+0000 7f765be00640 1 -- 172.21.15.18:0/684239939 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668016d90 con 0x559d0a4913c0 2024-04-16T17:24:10.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.985+0000 7f765be00640 1 -- 172.21.15.18:0/684239939 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f767409c0e0 con 0x559d0a4913c0 2024-04-16T17:24:10.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.985+0000 7f76841b3740 10 client.30445 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:10.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.985+0000 7f76841b3740 10 client.30445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:10.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.985+0000 7f76841b3740 20 client.30445 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:10.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.985+0000 7f76841b3740 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d0a4aab90 0x559d0a523e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:10.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.985+0000 7f76841b3740 4 client.30445 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:24:10.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.986+0000 7f7679e00640 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d0a4aab90 0x559d0a523e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:10.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.986+0000 7f76841b3740 1 -- 172.21.15.18:0/684239939 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d0a3fe9b0 con 0x559d0a4aab90 2024-04-16T17:24:10.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.987+0000 7f7679e00640 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d0a4aab90 0x559d0a523e80 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04bd10581e3a23d server_cookie=b466536482430d3 in_seq=0 out_seq=0 2024-04-16T17:24:10.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.987+0000 7f765be00640 10 client.30445 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:10.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.987+0000 7f765be00640 1 -- 172.21.15.18:0/684239939 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d0a3fe9b0 con 0x559d0a4aab90 2024-04-16T17:24:10.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.987+0000 7f765be00640 10 client.30445 handle_command_reply: tid=0 2024-04-16T17:24:10.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.988+0000 7f76841b3740 1 client.30445 shutdown 2024-04-16T17:24:10.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.990+0000 7f76841b3740 1 -- 172.21.15.18:0/684239939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a4913c0 msgr2=0x559d0a4b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:10.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:10.990+0000 7f76841b3740 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a4913c0 0x559d0a4b0090 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f767400fef0 tx=0x7f76740cedc0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.011+0000 7f76841b3740 1 -- 172.21.15.18:0/684239939 shutdown_connections 2024-04-16T17:24:11.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.011+0000 7f76841b3740 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559d0a4aab90 0x559d0a523e80 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.011+0000 7f76841b3740 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d0a49a8c0 0x559d0a4ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.011+0000 7f76841b3740 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d0a491ce0 0x559d0a4ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.011+0000 7f76841b3740 1 --2- 172.21.15.18:0/684239939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d0a4913c0 0x559d0a4b0090 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.011+0000 7f76841b3740 1 -- 172.21.15.18:0/684239939 >> 172.21.15.18:0/684239939 conn(0x559d0a48baf0 msgr2=0x559d0a4981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:11.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.016+0000 7f765be00640 0 client.30445 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:11.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.016+0000 7f76841b3740 1 -- 172.21.15.18:0/684239939 shutdown_connections 2024-04-16T17:24:11.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.016+0000 7f76841b3740 1 -- 172.21.15.18:0/684239939 wait complete. 2024-04-16T17:24:11.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.017+0000 7f76841b3740 20 client.30445 trim_cache size 0 max 16384 2024-04-16T17:24:11.017 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:24:11.017 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:24:11.017 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:24:11.017 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 -- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76600a3e10 msgr2=0x7f76600a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76600a3e10 0x7f76600a62d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f76680097a0 tx=0x7f7668009120 comp rx=0 tx=0).stop 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 -- 172.21.15.18:0/3800202119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 msgr2=0x7f767c1a7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 0x7f767c1a7500 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f767400cce0 tx=0x7f767400a620 comp rx=0 tx=0).stop 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 -- 172.21.15.18:0/3800202119 shutdown_connections 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f76600a3e10 0x7f76600a62d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767c110580 0x7f767c1abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f767c10ec00 0x7f767c1a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 --2- 172.21.15.18:0/3800202119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f767c10e0e0 0x7f767c1a7500 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 -- 172.21.15.18:0/3800202119 >> 172.21.15.18:0/3800202119 conn(0x7f767c06cf10 msgr2=0x7f767c0712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:11.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 -- 172.21.15.18:0/3800202119 shutdown_connections 2024-04-16T17:24:11.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.018+0000 7f7680a00640 1 -- 172.21.15.18:0/3800202119 wait complete. 2024-04-16T17:24:11.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:24:11.051 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:24:11.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.374+0000 7f80a3000640 1 -- 172.21.15.18:0/1316631901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c104430 msgr2=0x7f809c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:11.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.374+0000 7f80a3000640 1 --2- 172.21.15.18:0/1316631901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c104430 0x7f809c104810 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8088009900 tx=0x7f808802bc50 comp rx=0 tx=0).stop 2024-04-16T17:24:11.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.374+0000 7f80a3000640 1 -- 172.21.15.18:0/1316631901 shutdown_connections 2024-04-16T17:24:11.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.374+0000 7f80a3000640 1 --2- 172.21.15.18:0/1316631901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f809c109500 0x7f809c1110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.374+0000 7f80a3000640 1 --2- 172.21.15.18:0/1316631901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809c104de0 0x7f809c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.374+0000 7f80a3000640 1 --2- 172.21.15.18:0/1316631901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c104430 0x7f809c104810 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.374+0000 7f80a3000640 1 -- 172.21.15.18:0/1316631901 >> 172.21.15.18:0/1316631901 conn(0x7f809c0ffcb0 msgr2=0x7f809c1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:11.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.375+0000 7f80a3000640 1 -- 172.21.15.18:0/1316631901 shutdown_connections 2024-04-16T17:24:11.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.375+0000 7f80a3000640 1 -- 172.21.15.18:0/1316631901 wait complete. 2024-04-16T17:24:11.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.376+0000 7f80a3000640 1 Processor -- start 2024-04-16T17:24:11.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.377+0000 7f80a3000640 1 -- start start 2024-04-16T17:24:11.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.377+0000 7f80a3000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c104430 0x7f809c0738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:11.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.377+0000 7f80a3000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f809c109500 0x7f809c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:11.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.377+0000 7f80a3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809c06cf00 0x7f809c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:11.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.378+0000 7f80a3000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f809c114590 con 0x7f809c109500 2024-04-16T17:24:11.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.378+0000 7f80a3000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f809c114410 con 0x7f809c104430 2024-04-16T17:24:11.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.378+0000 7f80a3000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f809c114710 con 0x7f809c06cf00 2024-04-16T17:24:11.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.379+0000 7f809b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f809c109500 0x7f809c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:11.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.379+0000 7f809b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f809c109500 0x7f809c06c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37344/0 (socket says 172.21.15.18:37344) 2024-04-16T17:24:11.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.379+0000 7f809b400640 1 -- 172.21.15.18:0/4223920359 learned_addr learned my addr 172.21.15.18:0/4223920359 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:11.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.380+0000 7f809b400640 1 -- 172.21.15.18:0/4223920359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809c06cf00 msgr2=0x7f809c06d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:24:11.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.380+0000 7f809b400640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809c06cf00 0x7f809c06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.380+0000 7f809b400640 1 -- 172.21.15.18:0/4223920359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c104430 msgr2=0x7f809c0738f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:24:11.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.380+0000 7f809b400640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c104430 0x7f809c0738f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.380+0000 7f809b400640 1 -- 172.21.15.18:0/4223920359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c071410 con 0x7f809c109500 2024-04-16T17:24:11.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.380+0000 7f809b400640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f809c109500 0x7f809c06c9c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f808c00d770 tx=0x7f808c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:11.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.381+0000 7f8098c00640 1 -- 172.21.15.18:0/4223920359 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c017070 con 0x7f809c109500 2024-04-16T17:24:11.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.381+0000 7f8098c00640 1 -- 172.21.15.18:0/4223920359 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c009c30 con 0x7f809c109500 2024-04-16T17:24:11.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.381+0000 7f8098c00640 1 -- 172.21.15.18:0/4223920359 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c00f620 con 0x7f809c109500 2024-04-16T17:24:11.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.381+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c071700 con 0x7f809c109500 2024-04-16T17:24:11.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.381+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c071bd0 con 0x7f809c109500 2024-04-16T17:24:11.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.383+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809c105900 con 0x7f809c109500 2024-04-16T17:24:11.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.387+0000 7f8098c00640 1 -- 172.21.15.18:0/4223920359 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f808c01a070 con 0x7f809c109500 2024-04-16T17:24:11.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.390+0000 7f8098c00640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80800a3e20 0x7f80800a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:11.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.390+0000 7f8098c00640 1 -- 172.21.15.18:0/4223920359 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f808c0d0bb0 con 0x7f809c109500 2024-04-16T17:24:11.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.396+0000 7f809be00640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80800a3e20 0x7f80800a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:11.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.398+0000 7f809be00640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80800a3e20 0x7f80800a62e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f80880096c0 tx=0x7f8088005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:11.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.403+0000 7f8098c00640 1 -- 172.21.15.18:0/4223920359 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f808c09cd90 con 0x7f809c109500 2024-04-16T17:24:11.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88732 11.58 MB/sec execute 263 sec latency 2790.123 ms 2024-04-16T17:24:11.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.924+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f809c069300 con 0x7f809c109500 2024-04-16T17:24:11.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.926+0000 7f8098c00640 1 -- 172.21.15.18:0/4223920359 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7f808c009de0 con 0x7f809c109500 2024-04-16T17:24:11.933 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:24:11.933 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:24:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80800a3e20 msgr2=0x7f80800a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80800a3e20 0x7f80800a62e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f80880096c0 tx=0x7f8088005fd0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f809c109500 msgr2=0x7f809c06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f809c109500 0x7f809c06c9c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f808c00d770 tx=0x7f808c00dc30 comp rx=0 tx=0).stop 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 shutdown_connections 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f80800a3e20 0x7f80800a62e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809c06cf00 0x7f809c06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f809c109500 0x7f809c06c9c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 --2- 172.21.15.18:0/4223920359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c104430 0x7f809c0738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.934+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 >> 172.21.15.18:0/4223920359 conn(0x7f809c0ffcb0 msgr2=0x7f809c101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:11.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.938+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 shutdown_connections 2024-04-16T17:24:11.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:11.938+0000 7f80a3000640 1 -- 172.21.15.18:0/4223920359 wait complete. 2024-04-16T17:24:11.967 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:24:12.084 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:11 smithi018 ceph-mon[31776]: pgmap v622: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 444 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 432 op/s 2024-04-16T17:24:12.085 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:11 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/4223920359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:24:12.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:11 smithi043 ceph-mon[36901]: pgmap v622: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 444 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 432 op/s 2024-04-16T17:24:12.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:11 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/4223920359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:24:12.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.282+0000 7fc189600640 1 -- 172.21.15.18:0/977919516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1841008a0 msgr2=0x7fc184100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.282+0000 7fc189600640 1 --2- 172.21.15.18:0/977919516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1841008a0 0x7fc184100d20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc17400aa70 tx=0x7fc17402cf50 comp rx=0 tx=0).stop 2024-04-16T17:24:12.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.283+0000 7fc189600640 1 -- 172.21.15.18:0/977919516 shutdown_connections 2024-04-16T17:24:12.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.283+0000 7fc189600640 1 --2- 172.21.15.18:0/977919516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc184101260 0x7fc18410d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.283+0000 7fc189600640 1 --2- 172.21.15.18:0/977919516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1841008a0 0x7fc184100d20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.283+0000 7fc189600640 1 --2- 172.21.15.18:0/977919516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc184105650 0x7fc184105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.283+0000 7fc189600640 1 -- 172.21.15.18:0/977919516 >> 172.21.15.18:0/977919516 conn(0x7fc1840761f0 msgr2=0x7fc184076600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.283+0000 7fc189600640 1 -- 172.21.15.18:0/977919516 shutdown_connections 2024-04-16T17:24:12.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.284+0000 7fc189600640 1 -- 172.21.15.18:0/977919516 wait complete. 2024-04-16T17:24:12.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.286+0000 7fc189600640 1 Processor -- start 2024-04-16T17:24:12.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.286+0000 7fc189600640 1 -- start start 2024-04-16T17:24:12.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc189600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1841008a0 0x7fc18419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc189600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc184101260 0x7fc18419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc189600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc184105650 0x7fc1841a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc189600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841189c0 con 0x7fc184105650 2024-04-16T17:24:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc189600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc184118840 con 0x7fc1841008a0 2024-04-16T17:24:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc189600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc184118b40 con 0x7fc184101260 2024-04-16T17:24:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc182000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc184101260 0x7fc18419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc182000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc184101260 0x7fc18419f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40648/0 (socket says 172.21.15.18:40648) 2024-04-16T17:24:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.287+0000 7fc182000640 1 -- 172.21.15.18:0/2507279612 learned_addr learned my addr 172.21.15.18:0/2507279612 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.288+0000 7fc182000640 1 -- 172.21.15.18:0/2507279612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1841008a0 msgr2=0x7fc18419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.288+0000 7fc182000640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1841008a0 0x7fc18419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.288+0000 7fc182000640 1 -- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc184105650 msgr2=0x7fc1841a3800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:24:12.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.288+0000 7fc182000640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc184105650 0x7fc1841a3800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.288+0000 7fc182000640 1 -- 172.21.15.18:0/2507279612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1841a3f30 con 0x7fc184101260 2024-04-16T17:24:12.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.289+0000 7fc182000640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc184101260 0x7fc18419f480 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc174037040 tx=0x7fc174035c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.289+0000 7fc17b400640 1 -- 172.21.15.18:0/2507279612 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174004450 con 0x7fc184101260 2024-04-16T17:24:12.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.290+0000 7fc17b400640 1 -- 172.21.15.18:0/2507279612 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc174043070 con 0x7fc184101260 2024-04-16T17:24:12.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.290+0000 7fc17b400640 1 -- 172.21.15.18:0/2507279612 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174003850 con 0x7fc184101260 2024-04-16T17:24:12.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.290+0000 7fc189600640 1 -- 172.21.15.18:0/2507279612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1841a41c0 con 0x7fc184101260 2024-04-16T17:24:12.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.290+0000 7fc189600640 1 -- 172.21.15.18:0/2507279612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1841abab0 con 0x7fc184101260 2024-04-16T17:24:12.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.295+0000 7fc18c400740 1 Processor -- start 2024-04-16T17:24:12.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.295+0000 7fc18c400740 1 -- start start 2024-04-16T17:24:12.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.295+0000 7fc18c400740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5bd50 0x55f95ec5c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.296+0000 7fc18c400740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5c670 0x55f95ec64d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.296+0000 7fc18c400740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 0x55f95ec67640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.296+0000 7fc18c400740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f95ebc1020 con 0x55f95ec5c670 2024-04-16T17:24:12.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.296+0000 7fc18c400740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f95eb67460 con 0x55f95ec5bd50 2024-04-16T17:24:12.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.296+0000 7fc18c400740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f95ebbcfe0 con 0x55f95ec65250 2024-04-16T17:24:12.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc183400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5bd50 0x55f95ec5c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc183400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5bd50 0x55f95ec5c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54886/0 (socket says 172.21.15.18:54886) 2024-04-16T17:24:12.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc183400640 1 -- 172.21.15.18:0/4152249556 learned_addr learned my addr 172.21.15.18:0/4152249556 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc182a00640 1 --2- 172.21.15.18:0/4152249556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5c670 0x55f95ec64d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc183400640 1 -- 172.21.15.18:0/4152249556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 msgr2=0x55f95ec67640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:12.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc183400640 1 --2- 172.21.15.18:0/4152249556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 0x55f95ec67640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc183400640 1 -- 172.21.15.18:0/4152249556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5c670 msgr2=0x55f95ec64d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc183400640 1 --2- 172.21.15.18:0/4152249556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5c670 0x55f95ec64d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.297+0000 7fc183400640 1 -- 172.21.15.18:0/4152249556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95eb6f040 con 0x55f95ec5bd50 2024-04-16T17:24:12.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.298+0000 7fc17b400640 1 -- 172.21.15.18:0/2507279612 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc1740046c0 con 0x7fc184101260 2024-04-16T17:24:12.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.299+0000 7fc183400640 1 --2- 172.21.15.18:0/4152249556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5bd50 0x55f95ec5c130 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009e30 tx=0x7fc17c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25aeedece85175d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.301+0000 7fc16f400640 1 -- 172.21.15.18:0/4152249556 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c004d20 con 0x55f95ec5bd50 2024-04-16T17:24:12.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.301+0000 7fc16f400640 1 -- 172.21.15.18:0/4152249556 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c00ee70 con 0x55f95ec5bd50 2024-04-16T17:24:12.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.301+0000 7fc16f400640 1 -- 172.21.15.18:0/4152249556 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0107f0 con 0x55f95ec5bd50 2024-04-16T17:24:12.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.301+0000 7fc17b400640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc1680a3e20 0x7fc1680a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.301+0000 7fc17b400640 1 -- 172.21.15.18:0/2507279612 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7fc1740f2770 con 0x7fc184101260 2024-04-16T17:24:12.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.302+0000 7fc18c400740 1 -- 172.21.15.18:0/4152249556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5bd50 msgr2=0x55f95ec5c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.302+0000 7fc18c400740 1 --2- 172.21.15.18:0/4152249556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5bd50 0x55f95ec5c130 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009e30 tx=0x7fc17c00e720 comp rx=0 tx=0).stop 2024-04-16T17:24:12.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.302+0000 7fc18c400740 1 -- 172.21.15.18:0/4152249556 shutdown_connections 2024-04-16T17:24:12.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.302+0000 7fc18c400740 1 --2- 172.21.15.18:0/4152249556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 0x55f95ec67640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.302+0000 7fc18c400740 1 --2- 172.21.15.18:0/4152249556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5c670 0x55f95ec64d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.302+0000 7fc18c400740 1 --2- 172.21.15.18:0/4152249556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5bd50 0x55f95ec5c130 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.302+0000 7fc18c400740 1 -- 172.21.15.18:0/4152249556 >> 172.21.15.18:0/4152249556 conn(0x55f95ec56480 msgr2=0x55f95ec56860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.303+0000 7fc182a00640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc1680a3e20 0x7fc1680a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.306+0000 7fc18c400740 1 -- 172.21.15.18:0/4152249556 shutdown_connections 2024-04-16T17:24:12.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.306+0000 7fc18c400740 1 -- 172.21.15.18:0/4152249556 wait complete. 2024-04-16T17:24:12.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.307+0000 7fc18c400740 1 -- start start 2024-04-16T17:24:12.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.307+0000 7fc18c400740 1 Processor -- start 2024-04-16T17:24:12.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.309+0000 7fc18c400740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5bd50 0x55f95ec62370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.309+0000 7fc18c400740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.309+0000 7fc18c400740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec65250 0x55f95ec62df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.310+0000 7fc182000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.310+0000 7fc182000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54892/0 (socket says 172.21.15.18:54892) 2024-04-16T17:24:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.311+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f95ec68040 con 0x55f95ec65250 2024-04-16T17:24:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.311+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f95eb7a830 con 0x55f95ec5c670 2024-04-16T17:24:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.311+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f95ec681c0 con 0x55f95ec5bd50 2024-04-16T17:24:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.311+0000 7fc183400640 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec65250 0x55f95ec62df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.311+0000 7fc182000640 1 -- 172.21.15.18:0/705980303 learned_addr learned my addr 172.21.15.18:0/705980303 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.311+0000 7fc18c400740 10 client.0 _ll_register_callbacks cb 0x55f95eb82150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.312+0000 7fc182000640 1 -- 172.21.15.18:0/705980303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5bd50 msgr2=0x55f95ec62370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.312+0000 7fc182000640 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5bd50 0x55f95ec62370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.312+0000 7fc182000640 1 -- 172.21.15.18:0/705980303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec65250 msgr2=0x55f95ec62df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.312+0000 7fc182000640 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec65250 0x55f95ec62df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.312+0000 7fc182000640 1 -- 172.21.15.18:0/705980303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95eb7e920 con 0x55f95ec5c670 2024-04-16T17:24:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.312+0000 7fc182a00640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc1680a3e20 0x7fc1680a62e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55f95ec5b820 tx=0x7fc170009d30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.312+0000 7fc183400640 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec65250 0x55f95ec62df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:12.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.313+0000 7fc163e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:12.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.313+0000 7fc182000640 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec628b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc1740be2f0 tx=0x7fc1740bdea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ec37fc8ef0c5b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.313+0000 7fc163e00640 1 -- 172.21.15.18:0/705980303 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740efe90 con 0x55f95ec5c670 2024-04-16T17:24:12.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.314+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c00ee70 con 0x55f95ec5c670 2024-04-16T17:24:12.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.314+0000 7fc163e00640 1 -- 172.21.15.18:0/705980303 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7fc1740d2e00 con 0x55f95ec5c670 2024-04-16T17:24:12.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.315+0000 7fc163e00640 1 -- 172.21.15.18:0/705980303 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc1740efe90 con 0x55f95ec5c670 2024-04-16T17:24:12.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.315+0000 7fc18c400740 10 client.30596 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:12.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.315+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f95eb4c7f0 con 0x55f95ec5c670 2024-04-16T17:24:12.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.315+0000 7fc163e00640 1 -- 172.21.15.18:0/705980303 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c00ee70 con 0x55f95ec5c670 2024-04-16T17:24:12.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.316+0000 7fc163e00640 1 -- 172.21.15.18:0/705980303 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fc17400b040 con 0x55f95ec5c670 2024-04-16T17:24:12.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.316+0000 7fc18c400740 10 client.30596 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:12.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.316+0000 7fc18c400740 10 client.30596 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:24:12.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.316+0000 7fc18c400740 20 client.30596 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:12.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.316+0000 7fc18c400740 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f95ec75600 0x55f95ec779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.317+0000 7fc18c400740 4 client.30596 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:12.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.318+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f95ebcea40 con 0x55f95ec75600 2024-04-16T17:24:12.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.318+0000 7fc182a00640 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f95ec75600 0x55f95ec779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.319+0000 7fc182a00640 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f95ec75600 0x55f95ec779f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bfea065e3b0fb0a server_cookie=9c2f931f7e0c115b in_seq=0 out_seq=0 2024-04-16T17:24:12.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.319+0000 7fc163e00640 10 client.30596 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:24:12.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.323+0000 7fc163e00640 1 -- 172.21.15.18:0/705980303 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f95ebcea40 con 0x55f95ec75600 2024-04-16T17:24:12.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.323+0000 7fc163e00640 10 client.30596 handle_command_reply: tid=0 2024-04-16T17:24:12.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.323+0000 7fc18c400740 1 client.30596 shutdown 2024-04-16T17:24:12.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.324+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 msgr2=0x55f95ec628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.324+0000 7fc18c400740 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec628b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc1740be2f0 tx=0x7fc1740bdea0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.324+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 shutdown_connections 2024-04-16T17:24:12.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.325+0000 7fc18c400740 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f95ec75600 0x55f95ec779f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.325+0000 7fc18c400740 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec65250 0x55f95ec62df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.325+0000 7fc163e00640 0 client.30596 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:24:12.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.325+0000 7fc18c400740 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec628b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.325+0000 7fc18c400740 1 --2- 172.21.15.18:0/705980303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5bd50 0x55f95ec62370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.325+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 >> 172.21.15.18:0/705980303 conn(0x55f95ec56480 msgr2=0x55f95ec581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.325+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 shutdown_connections 2024-04-16T17:24:12.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.325+0000 7fc18c400740 1 -- 172.21.15.18:0/705980303 wait complete. 2024-04-16T17:24:12.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.326+0000 7fc18c400740 20 client.30596 trim_cache size 0 max 16384 2024-04-16T17:24:12.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.359+0000 7fc18c400740 1 Processor -- start 2024-04-16T17:24:12.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.359+0000 7fc18c400740 1 -- start start 2024-04-16T17:24:12.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.359+0000 7fc18c400740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec64aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.359+0000 7fc18c400740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec70f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.359+0000 7fc18c400740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 0x55f95ec57fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.360+0000 7fc18c400740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841b8780 con 0x55f95ec5bd50 2024-04-16T17:24:12.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.360+0000 7fc18c400740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841b6050 con 0x55f95ec5c670 2024-04-16T17:24:12.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.360+0000 7fc18c400740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841b3e60 con 0x55f95ec65250 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.360+0000 7fc182000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec70f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.360+0000 7fc182000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec70f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54908/0 (socket says 172.21.15.18:54908) 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.360+0000 7fc182000640 1 -- 172.21.15.18:0/3341606665 learned_addr learned my addr 172.21.15.18:0/3341606665 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.360+0000 7fc183400640 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 0x55f95ec57fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.361+0000 7fc183400640 1 -- 172.21.15.18:0/3341606665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 msgr2=0x55f95ec70f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.361+0000 7fc182a00640 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec64aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.361+0000 7fc183400640 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec70f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.361+0000 7fc183400640 1 -- 172.21.15.18:0/3341606665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 msgr2=0x55f95ec64aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.361+0000 7fc183400640 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec64aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.361+0000 7fc183400640 1 -- 172.21.15.18:0/3341606665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95e8d33f0 con 0x55f95ec65250 2024-04-16T17:24:12.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.361+0000 7fc183400640 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 0x55f95ec57fc0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc17c006010 tx=0x7fc17c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35807f84386663fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.362+0000 7fc16ea00640 1 -- 172.21.15.18:0/3341606665 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c01d040 con 0x55f95ec65250 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.362+0000 7fc16ea00640 1 -- 172.21.15.18:0/3341606665 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c003e50 con 0x55f95ec65250 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.362+0000 7fc16ea00640 1 -- 172.21.15.18:0/3341606665 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0236e0 con 0x55f95ec65250 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 -- 172.21.15.18:0/3341606665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 msgr2=0x55f95ec57fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 0x55f95ec57fc0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc17c006010 tx=0x7fc17c005b00 comp rx=0 tx=0).stop 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 -- 172.21.15.18:0/3341606665 shutdown_connections 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec65250 0x55f95ec57fc0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec5c670 0x55f95ec70f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 --2- 172.21.15.18:0/3341606665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 -- 172.21.15.18:0/3341606665 >> 172.21.15.18:0/3341606665 conn(0x55f95ec56480 msgr2=0x55f95ec578b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 -- 172.21.15.18:0/3341606665 shutdown_connections 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.363+0000 7fc18c400740 1 -- 172.21.15.18:0/3341606665 wait complete. 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.364+0000 7fc18c400740 1 -- start start 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.364+0000 7fc18c400740 1 Processor -- start 2024-04-16T17:24:12.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc18c400740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec6d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc18c400740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5c670 0x55f95ec5eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc18c400740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec65250 0x55f95ec5f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc182000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5c670 0x55f95ec5eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc182000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5c670 0x55f95ec5eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40680/0 (socket says 172.21.15.18:40680) 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc182a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec6d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc182000640 1 -- 172.21.15.18:0/1051363961 learned_addr learned my addr 172.21.15.18:0/1051363961 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841b4510 con 0x55f95ec5bd50 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc183400640 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec65250 0x55f95ec5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.365+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841b5980 con 0x55f95ec65250 2024-04-16T17:24:12.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841b4be0 con 0x55f95ec5c670 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc18c400740 10 client.0 _ll_register_callbacks cb 0x7fc1740ba690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc182000640 1 -- 172.21.15.18:0/1051363961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec65250 msgr2=0x55f95ec5f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc182000640 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec65250 0x55f95ec5f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc182000640 1 -- 172.21.15.18:0/1051363961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 msgr2=0x55f95ec6d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc182000640 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec6d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc182000640 1 -- 172.21.15.18:0/1051363961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95e99d7d0 con 0x55f95ec5c670 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc163e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.366+0000 7fc182000640 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5c670 0x55f95ec5eec0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc1740ba7c0 tx=0x7fc174000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365847dfe24d21cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.367+0000 7fc163e00640 1 -- 172.21.15.18:0/1051363961 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740bdcc0 con 0x55f95ec5c670 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.367+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c003e50 con 0x55f95ec5c670 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.367+0000 7fc182a00640 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec6d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.367+0000 7fc163e00640 1 -- 172.21.15.18:0/1051363961 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7fc1740d2e00 con 0x55f95ec5c670 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.367+0000 7fc183400640 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec65250 0x55f95ec5f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.368+0000 7fc163e00640 1 -- 172.21.15.18:0/1051363961 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc1740bdcc0 con 0x55f95ec5c670 2024-04-16T17:24:12.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.368+0000 7fc18c400740 10 client.30460 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:12.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.368+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f95ecdbb00 con 0x55f95ec5c670 2024-04-16T17:24:12.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.369+0000 7fc163e00640 1 -- 172.21.15.18:0/1051363961 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c003e50 con 0x55f95ec5c670 2024-04-16T17:24:12.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.369+0000 7fc163e00640 1 -- 172.21.15.18:0/1051363961 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fc1740bd7b0 con 0x55f95ec5c670 2024-04-16T17:24:12.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.369+0000 7fc18c400740 10 client.30460 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:12.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.369+0000 7fc18c400740 10 client.30460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:24:12.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.369+0000 7fc18c400740 20 client.30460 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:12.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.369+0000 7fc18c400740 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f95ec75600 0x55f95ecde1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.369+0000 7fc18c400740 4 client.30460 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:24:12.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.369+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f95ebc88b0 con 0x55f95ec75600 2024-04-16T17:24:12.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.370+0000 7fc182a00640 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f95ec75600 0x55f95ecde1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.371+0000 7fc182a00640 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f95ec75600 0x55f95ecde1b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fe62c69c144e014 server_cookie=5f3809f0334a594a in_seq=0 out_seq=0 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.371+0000 7fc163e00640 10 client.30460 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.371+0000 7fc163e00640 1 -- 172.21.15.18:0/1051363961 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f95ebc88b0 con 0x55f95ec75600 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.371+0000 7fc163e00640 10 client.30460 handle_command_reply: tid=0 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.371+0000 7fc18c400740 1 client.30460 shutdown 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.371+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5c670 msgr2=0x55f95ec5eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.371+0000 7fc18c400740 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5c670 0x55f95ec5eec0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc1740ba7c0 tx=0x7fc174000bc0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.372+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 shutdown_connections 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.372+0000 7fc18c400740 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55f95ec75600 0x55f95ecde1b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.372+0000 7fc18c400740 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f95ec65250 0x55f95ec5f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.372+0000 7fc18c400740 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f95ec5c670 0x55f95ec5eec0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.372+0000 7fc18c400740 1 --2- 172.21.15.18:0/1051363961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f95ec5bd50 0x55f95ec6d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.372+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 >> 172.21.15.18:0/1051363961 conn(0x55f95ec56480 msgr2=0x55f95ec58140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.372+0000 7fc163e00640 0 client.30460 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:24:12.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.374+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 shutdown_connections 2024-04-16T17:24:12.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.374+0000 7fc18c400740 1 -- 172.21.15.18:0/1051363961 wait complete. 2024-04-16T17:24:12.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.374+0000 7fc18c400740 20 client.30460 trim_cache size 0 max 16384 2024-04-16T17:24:12.375 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:24:12.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 -- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc1680a3e20 msgr2=0x7fc1680a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc1680a3e20 0x7fc1680a62e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55f95ec5b820 tx=0x7fc170009d30 comp rx=0 tx=0).stop 2024-04-16T17:24:12.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 -- 172.21.15.18:0/2507279612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc184101260 msgr2=0x7fc18419f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc184101260 0x7fc18419f480 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc174037040 tx=0x7fc174035c20 comp rx=0 tx=0).stop 2024-04-16T17:24:12.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 -- 172.21.15.18:0/2507279612 shutdown_connections 2024-04-16T17:24:12.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc1680a3e20 0x7fc1680a62e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc184105650 0x7fc1841a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc184101260 0x7fc18419f480 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 --2- 172.21.15.18:0/2507279612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1841008a0 0x7fc18419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.376+0000 7fc189600640 1 -- 172.21.15.18:0/2507279612 >> 172.21.15.18:0/2507279612 conn(0x7fc1840761f0 msgr2=0x7fc1840fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.377+0000 7fc189600640 1 -- 172.21.15.18:0/2507279612 shutdown_connections 2024-04-16T17:24:12.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.377+0000 7fc189600640 1 -- 172.21.15.18:0/2507279612 wait complete. 2024-04-16T17:24:12.406 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:24:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:11 smithi022 ceph-mon[36023]: pgmap v622: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 444 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 432 op/s 2024-04-16T17:24:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:11 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/4223920359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:24:12.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88746 11.53 MB/sec execute 264 sec latency 3510.803 ms 2024-04-16T17:24:12.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.729+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4002510012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101980 msgr2=0x7f9e28105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.729+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4002510012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101980 0x7f9e28105e40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009900 tx=0x7f9e1002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.730+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4002510012 shutdown_connections 2024-04-16T17:24:12.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.730+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4002510012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e28106380 0x7f9e28106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.730+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4002510012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101980 0x7f9e28105e40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.730+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4002510012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e28101060 0x7f9e28101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.730+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4002510012 >> 172.21.15.18:0/4002510012 conn(0x7f9e280780b0 msgr2=0x7f9e280784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.731+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4002510012 shutdown_connections 2024-04-16T17:24:12.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.731+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4002510012 wait complete. 2024-04-16T17:24:12.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.732+0000 7f9e2ce00640 1 Processor -- start 2024-04-16T17:24:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.733+0000 7f9e2ce00640 1 -- start start 2024-04-16T17:24:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e2ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101060 0x7f9e2819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e2ce00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e28101980 0x7f9e2819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e2ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e28106380 0x7f9e281a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e2ce00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e28118a00 con 0x7f9e28101980 2024-04-16T17:24:12.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e2ce00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e28118880 con 0x7f9e28106380 2024-04-16T17:24:12.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e2ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e28118b80 con 0x7f9e28101060 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e25e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101060 0x7f9e2819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e26800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e28106380 0x7f9e281a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e25e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101060 0x7f9e2819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40702/0 (socket says 172.21.15.18:40702) 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e25e00640 1 -- 172.21.15.18:0/4068295606 learned_addr learned my addr 172.21.15.18:0/4068295606 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.734+0000 7f9e26800640 1 -- 172.21.15.18:0/4068295606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101060 msgr2=0x7f9e2819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e26800640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101060 0x7f9e2819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e26800640 1 -- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e28101980 msgr2=0x7f9e2819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e26800640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e28101980 0x7f9e2819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e26800640 1 -- 172.21.15.18:0/4068295606 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e281a3e80 con 0x7f9e28106380 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e26800640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e28106380 0x7f9e281a3750 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005fd0 tx=0x7f9e2000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e1f400640 1 -- 172.21.15.18:0/4068295606 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000cd80 con 0x7f9e28106380 2024-04-16T17:24:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e1f400640 1 -- 172.21.15.18:0/4068295606 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e20005020 con 0x7f9e28106380 2024-04-16T17:24:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e1f400640 1 -- 172.21.15.18:0/4068295606 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20010720 con 0x7f9e28106380 2024-04-16T17:24:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.735+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4068295606 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e281a4170 con 0x7f9e28106380 2024-04-16T17:24:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.736+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4068295606 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e281aba00 con 0x7f9e28106380 2024-04-16T17:24:12.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.747+0000 7f9e2f9ff740 1 Processor -- start 2024-04-16T17:24:12.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.749+0000 7f9e1f400640 1 -- 172.21.15.18:0/4068295606 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e200051d0 con 0x7f9e28106380 2024-04-16T17:24:12.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.750+0000 7f9e2f9ff740 1 -- start start 2024-04-16T17:24:12.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.750+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.750+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.750+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9d3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.750+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5592ad92d020 con 0x5592ad9c8670 2024-04-16T17:24:12.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.750+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5592ad8d3460 con 0x5592ad9c7d50 2024-04-16T17:24:12.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.750+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5592ad928fe0 con 0x5592ad9d1250 2024-04-16T17:24:12.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.751+0000 7f9e1d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.751+0000 7f9e1d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54944/0 (socket says 172.21.15.18:54944) 2024-04-16T17:24:12.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.751+0000 7f9e1d400640 1 -- 172.21.15.18:0/2029818067 learned_addr learned my addr 172.21.15.18:0/2029818067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.751+0000 7f9e1d400640 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.752+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.752+0000 7f9e1f400640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e0c0a3d20 0x7f9e0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.752+0000 7f9e1f400640 1 -- 172.21.15.18:0/4068295606 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f9e200d1220 con 0x7f9e28106380 2024-04-16T17:24:12.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.752+0000 7f9e25e00640 1 -- 172.21.15.18:0/2029818067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 msgr2=0x5592ad9d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.752+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9d3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.752+0000 7f9e25e00640 1 -- 172.21.15.18:0/2029818067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 msgr2=0x5592ad9c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.752+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.752+0000 7f9e25e00640 1 -- 172.21.15.18:0/2029818067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ad8db040 con 0x5592ad9c8670 2024-04-16T17:24:12.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.754+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9d0d10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009c80 tx=0x7f9e18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c262a466aac1378 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.755+0000 7f9e17400640 1 -- 172.21.15.18:0/2029818067 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18004230 con 0x5592ad9c8670 2024-04-16T17:24:12.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.755+0000 7f9e17400640 1 -- 172.21.15.18:0/2029818067 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e180043d0 con 0x5592ad9c8670 2024-04-16T17:24:12.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.755+0000 7f9e17400640 1 -- 172.21.15.18:0/2029818067 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18004d80 con 0x5592ad9c8670 2024-04-16T17:24:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.756+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2029818067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 msgr2=0x5592ad9d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.756+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9d0d10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009c80 tx=0x7f9e18011810 comp rx=0 tx=0).stop 2024-04-16T17:24:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.756+0000 7f9e26800640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e0c0a3d20 0x7f9e0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.756+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2029818067 shutdown_connections 2024-04-16T17:24:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.756+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.756+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9d0d10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.756+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2029818067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.756+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2029818067 >> 172.21.15.18:0/2029818067 conn(0x5592ad9c2480 msgr2=0x5592ad9c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.757+0000 7f9e26800640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e0c0a3d20 0x7f9e0c0a61e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9e200991c0 tx=0x7f9e20005840 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.759+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2029818067 shutdown_connections 2024-04-16T17:24:12.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.759+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2029818067 wait complete. 2024-04-16T17:24:12.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.760+0000 7f9e2f9ff740 1 -- start start 2024-04-16T17:24:12.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.760+0000 7f9e2f9ff740 1 Processor -- start 2024-04-16T17:24:12.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.763+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c7d50 0x5592ad9ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.763+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9c8670 0x5592ad9ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.763+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9d1250 0x5592ad9ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.763+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5592ad9d4040 con 0x5592ad9c7d50 2024-04-16T17:24:12.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.763+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5592ad8e6830 con 0x5592ad9d1250 2024-04-16T17:24:12.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.763+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5592ad9d41c0 con 0x5592ad9c8670 2024-04-16T17:24:12.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.763+0000 7f9e2f9ff740 10 client.0 _ll_register_callbacks cb 0x5592ad8ee150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:12.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.764+0000 7f9e1d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9d1250 0x5592ad9ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.764+0000 7f9e1d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9d1250 0x5592ad9ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54954/0 (socket says 172.21.15.18:54954) 2024-04-16T17:24:12.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.764+0000 7f9e1d400640 1 -- 172.21.15.18:0/2102821601 learned_addr learned my addr 172.21.15.18:0/2102821601 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.764+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9c8670 0x5592ad9ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.765+0000 7f9e1d400640 1 -- 172.21.15.18:0/2102821601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9c8670 msgr2=0x5592ad9ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.765+0000 7f9e1d400640 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9c8670 0x5592ad9ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.765+0000 7f9e1d400640 1 -- 172.21.15.18:0/2102821601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c7d50 msgr2=0x5592ad9ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.765+0000 7f9e1d400640 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c7d50 0x5592ad9ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.765+0000 7f9e1d400640 1 -- 172.21.15.18:0/2102821601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ad8ea920 con 0x5592ad9d1250 2024-04-16T17:24:12.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.765+0000 7f9e1d400640 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9d1250 0x5592ad9ced00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9e1002c8c0 tx=0x7f9e100095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=239fe2091ce29971 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.768+0000 7f9e0be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:12.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.768+0000 7f9e0be00640 1 -- 172.21.15.18:0/2102821601 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10046020 con 0x5592ad9d1250 2024-04-16T17:24:12.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.768+0000 7f9e0be00640 1 -- 172.21.15.18:0/2102821601 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f9e10005070 con 0x5592ad9d1250 2024-04-16T17:24:12.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.768+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2102821601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e180043d0 con 0x5592ad9d1250 2024-04-16T17:24:12.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.769+0000 7f9e0be00640 1 -- 172.21.15.18:0/2102821601 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9e10051050 con 0x5592ad9d1250 2024-04-16T17:24:12.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.770+0000 7f9e2f9ff740 10 client.30611 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:12.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.770+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2102821601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5592ad8b87f0 con 0x5592ad9d1250 2024-04-16T17:24:12.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.771+0000 7f9e0be00640 1 -- 172.21.15.18:0/2102821601 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e180043d0 con 0x5592ad9d1250 2024-04-16T17:24:12.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.771+0000 7f9e0be00640 1 -- 172.21.15.18:0/2102821601 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f9e100035f0 con 0x5592ad9d1250 2024-04-16T17:24:12.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.771+0000 7f9e2f9ff740 10 client.30611 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:12.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.771+0000 7f9e2f9ff740 10 client.30611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:24:12.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.771+0000 7f9e2f9ff740 20 client.30611 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:12.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.771+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5592ad9e1620 0x5592ad9e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.773+0000 7f9e2f9ff740 4 client.30611 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:12.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.773+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2102821601 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592ad93aa40 con 0x5592ad9e1620 2024-04-16T17:24:12.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.774+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5592ad9e1620 0x5592ad9e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.778+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5592ad9e1620 0x5592ad9e3a10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f86d5380c158f740 server_cookie=8328cbe04a20e0d4 in_seq=0 out_seq=0 2024-04-16T17:24:12.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.778+0000 7f9e0be00640 10 client.30611 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:24:12.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.783+0000 7f9e0be00640 1 -- 172.21.15.18:0/2102821601 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592ad93aa40 con 0x5592ad9e1620 2024-04-16T17:24:12.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.783+0000 7f9e0be00640 10 client.30611 handle_command_reply: tid=0 2024-04-16T17:24:12.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.784+0000 7f9e2f9ff740 1 client.30611 shutdown 2024-04-16T17:24:12.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.784+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2102821601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9d1250 msgr2=0x5592ad9ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.785+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9d1250 0x5592ad9ced00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9e1002c8c0 tx=0x7f9e100095c0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.786+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2102821601 shutdown_connections 2024-04-16T17:24:12.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.786+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5592ad9e1620 0x5592ad9e3a10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.786+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9d1250 0x5592ad9ced00 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.786+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9c8670 0x5592ad9ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.786+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2102821601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c7d50 0x5592ad9ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.786+0000 7f9e0be00640 0 client.30611 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:24:12.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.786+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2102821601 >> 172.21.15.18:0/2102821601 conn(0x5592ad9c2480 msgr2=0x5592ad9c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.788+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2102821601 shutdown_connections 2024-04-16T17:24:12.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.788+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2102821601 wait complete. 2024-04-16T17:24:12.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.788+0000 7f9e2f9ff740 20 client.30611 trim_cache size 0 max 16384 2024-04-16T17:24:12.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.825+0000 7f9e2f9ff740 1 Processor -- start 2024-04-16T17:24:12.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.825+0000 7f9e2f9ff740 1 -- start start 2024-04-16T17:24:12.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c7d50 0x5592ad9d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c8670 0x5592ad9ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e281b8780 con 0x5592ad9c7d50 2024-04-16T17:24:12.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e281b6050 con 0x5592ad9c8670 2024-04-16T17:24:12.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e281b3ec0 con 0x5592ad9d1250 2024-04-16T17:24:12.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e25e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c8670 0x5592ad9ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e25e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c8670 0x5592ad9ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54968/0 (socket says 172.21.15.18:54968) 2024-04-16T17:24:12.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.826+0000 7f9e25e00640 1 -- 172.21.15.18:0/65760377 learned_addr learned my addr 172.21.15.18:0/65760377 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.827+0000 7f9e25e00640 1 -- 172.21.15.18:0/65760377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 msgr2=0x5592ad9c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.827+0000 7f9e1d400640 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.827+0000 7f9e25e00640 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.827+0000 7f9e25e00640 1 -- 172.21.15.18:0/65760377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c7d50 msgr2=0x5592ad9d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.827+0000 7f9e25e00640 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c7d50 0x5592ad9d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.827+0000 7f9e25e00640 1 -- 172.21.15.18:0/65760377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ad63f3f0 con 0x5592ad9c8670 2024-04-16T17:24:12.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.827+0000 7f9e1d400640 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:12.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.828+0000 7f9e25e00640 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c8670 0x5592ad9ceb20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009c80 tx=0x7f9e18009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635f150be142a5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.833+0000 7f9e16a00640 1 -- 172.21.15.18:0/65760377 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18032070 con 0x5592ad9c8670 2024-04-16T17:24:12.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.833+0000 7f9e16a00640 1 -- 172.21.15.18:0/65760377 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e1802a070 con 0x5592ad9c8670 2024-04-16T17:24:12.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.833+0000 7f9e16a00640 1 -- 172.21.15.18:0/65760377 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1801e040 con 0x5592ad9c8670 2024-04-16T17:24:12.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.834+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/65760377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c8670 msgr2=0x5592ad9ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.834+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c8670 0x5592ad9ceb20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009c80 tx=0x7f9e18009ef0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.834+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/65760377 shutdown_connections 2024-04-16T17:24:12.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.834+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.834+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c8670 0x5592ad9ceb20 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.834+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/65760377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c7d50 0x5592ad9d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.834+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/65760377 >> 172.21.15.18:0/65760377 conn(0x5592ad9c2480 msgr2=0x5592ad9c38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.835+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/65760377 shutdown_connections 2024-04-16T17:24:12.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.835+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/65760377 wait complete. 2024-04-16T17:24:12.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.836+0000 7f9e2f9ff740 1 -- start start 2024-04-16T17:24:12.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.836+0000 7f9e2f9ff740 1 Processor -- start 2024-04-16T17:24:12.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.837+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.837+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.838+0000 7f9e2f9ff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.838+0000 7f9e1d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.838+0000 7f9e1d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:54984/0 (socket says 172.21.15.18:54984) 2024-04-16T17:24:12.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.838+0000 7f9e2f9ff740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9e281b4510 con 0x5592ad9c8670 2024-04-16T17:24:12.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.838+0000 7f9e1d400640 1 -- 172.21.15.18:0/2876642849 learned_addr learned my addr 172.21.15.18:0/2876642849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:12.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.839+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e281b5980 con 0x5592ad9c7d50 2024-04-16T17:24:12.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.839+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e281b4be0 con 0x5592ad9d1250 2024-04-16T17:24:12.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.839+0000 7f9e2f9ff740 10 client.0 _ll_register_callbacks cb 0x7f9e10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:12.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.839+0000 7f9e1d400640 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.840+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.840+0000 7f9e25e00640 1 -- 172.21.15.18:0/2876642849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 msgr2=0x5592ad9cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.840+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.840+0000 7f9e25e00640 1 -- 172.21.15.18:0/2876642849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 msgr2=0x5592ad9cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:12.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.840+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.840+0000 7f9e25e00640 1 -- 172.21.15.18:0/2876642849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ad7097d0 con 0x5592ad9c8670 2024-04-16T17:24:12.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.840+0000 7f9e1d400640 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:12.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.841+0000 7f9e25e00640 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9cb700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9e18000c00 tx=0x7f9e18005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64c7dd3d682a377 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:12.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.841+0000 7f9e0be00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:24:12.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.846+0000 7f9e0be00640 1 -- 172.21.15.18:0/2876642849 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1803b060 con 0x5592ad9c8670 2024-04-16T17:24:12.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.846+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e1802a070 con 0x5592ad9c8670 2024-04-16T17:24:12.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.846+0000 7f9e0be00640 1 -- 172.21.15.18:0/2876642849 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f9e18003870 con 0x5592ad9c8670 2024-04-16T17:24:12.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.847+0000 7f9e0be00640 1 -- 172.21.15.18:0/2876642849 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9e18032070 con 0x5592ad9c8670 2024-04-16T17:24:12.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.847+0000 7f9e2f9ff740 10 client.19713 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:12.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.847+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5592ada47da0 con 0x5592ad9c8670 2024-04-16T17:24:12.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.848+0000 7f9e0be00640 1 -- 172.21.15.18:0/2876642849 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e1802a070 con 0x5592ad9c8670 2024-04-16T17:24:12.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.852+0000 7f9e0be00640 1 -- 172.21.15.18:0/2876642849 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f9e18024040 con 0x5592ad9c8670 2024-04-16T17:24:12.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.852+0000 7f9e2f9ff740 10 client.19713 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:12.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.852+0000 7f9e2f9ff740 10 client.19713 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:24:12.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.852+0000 7f9e2f9ff740 20 client.19713 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:12.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.852+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5592ada48190 0x5592ada4a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:12.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.852+0000 7f9e2f9ff740 4 client.19713 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:24:12.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.852+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592ad9348b0 con 0x5592ada48190 2024-04-16T17:24:12.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.853+0000 7f9e1d400640 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5592ada48190 0x5592ada4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:12.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.854+0000 7f9e1d400640 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5592ada48190 0x5592ada4a580 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a76dfd9fae96a789 server_cookie=c94cfc2d154f4afb in_seq=0 out_seq=0 2024-04-16T17:24:12.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.854+0000 7f9e0be00640 10 client.19713 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:24:12.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.859+0000 7f9e0be00640 1 -- 172.21.15.18:0/2876642849 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592ad9348b0 con 0x5592ada48190 2024-04-16T17:24:12.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.859+0000 7f9e0be00640 10 client.19713 handle_command_reply: tid=0 2024-04-16T17:24:12.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.859+0000 7f9e2f9ff740 1 client.19713 shutdown 2024-04-16T17:24:12.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 msgr2=0x5592ad9cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9cb700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9e18000c00 tx=0x7f9e18005a70 comp rx=0 tx=0).stop 2024-04-16T17:24:12.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 shutdown_connections 2024-04-16T17:24:12.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x5592ada48190 0x5592ada4a580 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592ad9d1250 0x5592ad9cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592ad9c8670 0x5592ad9cb700 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e2f9ff740 1 --2- 172.21.15.18:0/2876642849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ad9c7d50 0x5592ad9cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 >> 172.21.15.18:0/2876642849 conn(0x5592ad9c2480 msgr2=0x5592ad9c38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.860+0000 7f9e0be00640 0 client.19713 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:24:12.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.862+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 shutdown_connections 2024-04-16T17:24:12.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.862+0000 7f9e2f9ff740 1 -- 172.21.15.18:0/2876642849 wait complete. 2024-04-16T17:24:12.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.862+0000 7f9e2f9ff740 20 client.19713 trim_cache size 0 max 16384 2024-04-16T17:24:12.863 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e0c0a3d20 msgr2=0x7f9e0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e0c0a3d20 0x7f9e0c0a61e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9e200991c0 tx=0x7f9e20005840 comp rx=0 tx=0).stop 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4068295606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e28106380 msgr2=0x7f9e281a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e28106380 0x7f9e281a3750 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005fd0 tx=0x7f9e2000ee60 comp rx=0 tx=0).stop 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4068295606 shutdown_connections 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9e0c0a3d20 0x7f9e0c0a61e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e28106380 0x7f9e281a3750 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e28101980 0x7f9e2819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 --2- 172.21.15.18:0/4068295606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e28101060 0x7f9e2819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:12.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4068295606 >> 172.21.15.18:0/4068295606 conn(0x7f9e280780b0 msgr2=0x7f9e28103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:12.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.864+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4068295606 shutdown_connections 2024-04-16T17:24:12.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:12.865+0000 7f9e2ce00640 1 -- 172.21.15.18:0/4068295606 wait complete. 2024-04-16T17:24:12.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:24:13.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.216+0000 7f4d59e00640 1 -- 172.21.15.18:0/962276344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110bc0 msgr2=0x7f4d5410e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.216+0000 7f4d59e00640 1 --2- 172.21.15.18:0/962276344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110bc0 0x7f4d5410e850 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009900 tx=0x7f4d4402bc30 comp rx=0 tx=0).stop 2024-04-16T17:24:13.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.216+0000 7f4d59e00640 1 -- 172.21.15.18:0/962276344 shutdown_connections 2024-04-16T17:24:13.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.217+0000 7f4d59e00640 1 --2- 172.21.15.18:0/962276344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d5411b7e0 0x7f4d5411dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.217+0000 7f4d59e00640 1 --2- 172.21.15.18:0/962276344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110bc0 0x7f4d5410e850 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.217+0000 7f4d59e00640 1 --2- 172.21.15.18:0/962276344 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d54110750 0x7f4d5410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.217+0000 7f4d59e00640 1 -- 172.21.15.18:0/962276344 >> 172.21.15.18:0/962276344 conn(0x7f4d5406d0a0 msgr2=0x7f4d5406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:13.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.217+0000 7f4d59e00640 1 -- 172.21.15.18:0/962276344 shutdown_connections 2024-04-16T17:24:13.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.217+0000 7f4d59e00640 1 -- 172.21.15.18:0/962276344 wait complete. 2024-04-16T17:24:13.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.219+0000 7f4d59e00640 1 Processor -- start 2024-04-16T17:24:13.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.220+0000 7f4d59e00640 1 -- start start 2024-04-16T17:24:13.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.221+0000 7f4d59e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110750 0x7f4d541a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.221+0000 7f4d59e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d54110bc0 0x7f4d541a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.221+0000 7f4d59e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d5411b7e0 0x7f4d541abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.222+0000 7f4d59e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54121240 con 0x7f4d54110bc0 2024-04-16T17:24:13.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.222+0000 7f4d59e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541210c0 con 0x7f4d5411b7e0 2024-04-16T17:24:13.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.222+0000 7f4d59e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541213c0 con 0x7f4d54110750 2024-04-16T17:24:13.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.222+0000 7f4d52a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110750 0x7f4d541a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.222+0000 7f4d52a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110750 0x7f4d541a76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40754/0 (socket says 172.21.15.18:40754) 2024-04-16T17:24:13.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.222+0000 7f4d52a00640 1 -- 172.21.15.18:0/1483034076 learned_addr learned my addr 172.21.15.18:0/1483034076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:13.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.222+0000 7f4d53400640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d5411b7e0 0x7f4d541abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.222+0000 7f4d52000640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d54110bc0 0x7f4d541a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.223+0000 7f4d52a00640 1 -- 172.21.15.18:0/1483034076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d5411b7e0 msgr2=0x7f4d541abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.223+0000 7f4d52a00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d5411b7e0 0x7f4d541abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.223+0000 7f4d52a00640 1 -- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d54110bc0 msgr2=0x7f4d541a7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.223+0000 7f4d52a00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d54110bc0 0x7f4d541a7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.223+0000 7f4d52a00640 1 -- 172.21.15.18:0/1483034076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d541ac6d0 con 0x7f4d54110750 2024-04-16T17:24:13.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.223+0000 7f4d52000640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d54110bc0 0x7f4d541a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:13.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.224+0000 7f4d52a00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110750 0x7f4d541a76e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00ebc0 tx=0x7f4d4c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:13.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.226+0000 7f4d58a00640 1 -- 172.21.15.18:0/1483034076 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c010630 con 0x7f4d54110750 2024-04-16T17:24:13.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.226+0000 7f4d59e00640 1 -- 172.21.15.18:0/1483034076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d541ac9c0 con 0x7f4d54110750 2024-04-16T17:24:13.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.226+0000 7f4d58a00640 1 -- 172.21.15.18:0/1483034076 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d4c010cb0 con 0x7f4d54110750 2024-04-16T17:24:13.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.226+0000 7f4d58a00640 1 -- 172.21.15.18:0/1483034076 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c010630 con 0x7f4d54110750 2024-04-16T17:24:13.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.226+0000 7f4d59e00640 1 -- 172.21.15.18:0/1483034076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541b4250 con 0x7f4d54110750 2024-04-16T17:24:13.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.231+0000 7f4d5cc38740 1 Processor -- start 2024-04-16T17:24:13.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.232+0000 7f4d5cc38740 1 -- start start 2024-04-16T17:24:13.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.233+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f4d50 0x5595c13f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.233+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f5670 0x5595c13fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.233+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13fe250 0x5595c1400640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.233+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5595c135a020 con 0x5595c13fe250 2024-04-16T17:24:13.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.233+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5595c1300460 con 0x5595c13f4d50 2024-04-16T17:24:13.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.233+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5595c1355fe0 con 0x5595c13f5670 2024-04-16T17:24:13.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.233+0000 7f4d58a00640 1 -- 172.21.15.18:0/1483034076 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4d4c0107d0 con 0x7f4d54110750 2024-04-16T17:24:13.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.235+0000 7f4d52000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13fe250 0x5595c1400640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.235+0000 7f4d52000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13fe250 0x5595c1400640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37454/0 (socket says 172.21.15.18:37454) 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.238+0000 7f4d53400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f4d50 0x5595c13f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.238+0000 7f4d58a00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4d380a4020 0x7f4d380a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.238+0000 7f4d53400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f4d50 0x5595c13f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55002/0 (socket says 172.21.15.18:55002) 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.238+0000 7f4d53400640 1 -- 172.21.15.18:0/4134410195 learned_addr learned my addr 172.21.15.18:0/4134410195 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.238+0000 7f4d52a00640 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f5670 0x5595c13fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.238+0000 7f4d52a00640 1 -- 172.21.15.18:0/4134410195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f4d50 msgr2=0x5595c13f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.238+0000 7f4d52a00640 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f4d50 0x5595c13f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.238+0000 7f4d52a00640 1 -- 172.21.15.18:0/4134410195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13fe250 msgr2=0x5595c1400640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.239+0000 7f4d58a00640 1 -- 172.21.15.18:0/1483034076 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f4d4c0d1150 con 0x7f4d54110750 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.239+0000 7f4d52a00640 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13fe250 0x5595c1400640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.239+0000 7f4d52a00640 1 -- 172.21.15.18:0/4134410195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c1308040 con 0x5595c13f5670 2024-04-16T17:24:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.239+0000 7f4d52000640 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13fe250 0x5595c1400640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.239+0000 7f4d52a00640 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f5670 0x5595c13fdd10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0ce3e0 tx=0x7f4d4c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c39e0168e6683ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.239+0000 7f4d43e00640 1 -- 172.21.15.18:0/4134410195 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c087070 con 0x5595c13f5670 2024-04-16T17:24:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.240+0000 7f4d43e00640 1 -- 172.21.15.18:0/4134410195 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d4c09d330 con 0x5595c13f5670 2024-04-16T17:24:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.240+0000 7f4d43e00640 1 -- 172.21.15.18:0/4134410195 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0a58a0 con 0x5595c13f5670 2024-04-16T17:24:13.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.240+0000 7f4d53400640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4d380a4020 0x7f4d380a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.241+0000 7f4d5cc38740 1 -- 172.21.15.18:0/4134410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f5670 msgr2=0x5595c13fdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.241+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f5670 0x5595c13fdd10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0ce3e0 tx=0x7f4d4c00cf90 comp rx=0 tx=0).stop 2024-04-16T17:24:13.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.242+0000 7f4d5cc38740 1 -- 172.21.15.18:0/4134410195 shutdown_connections 2024-04-16T17:24:13.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.242+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13fe250 0x5595c1400640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.242+0000 7f4d53400640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4d380a4020 0x7f4d380a64e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009be0 tx=0x7f4d3c008040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:13.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.242+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f5670 0x5595c13fdd10 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.242+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/4134410195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f4d50 0x5595c13f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.242+0000 7f4d5cc38740 1 -- 172.21.15.18:0/4134410195 >> 172.21.15.18:0/4134410195 conn(0x5595c13ef480 msgr2=0x5595c13ef860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:13.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.242+0000 7f4d5cc38740 1 -- 172.21.15.18:0/4134410195 shutdown_connections 2024-04-16T17:24:13.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.242+0000 7f4d5cc38740 1 -- 172.21.15.18:0/4134410195 wait complete. 2024-04-16T17:24:13.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.243+0000 7f4d5cc38740 1 -- start start 2024-04-16T17:24:13.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.243+0000 7f4d5cc38740 1 Processor -- start 2024-04-16T17:24:13.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.244+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c13fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.245+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13fb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.245+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.245+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5595c1401040 con 0x5595c13f4d50 2024-04-16T17:24:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.245+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5595c1313830 con 0x5595c13f5670 2024-04-16T17:24:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.245+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5595c14011c0 con 0x5595c13fe250 2024-04-16T17:24:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.245+0000 7f4d5cc38740 10 client.0 _ll_register_callbacks cb 0x5595c131b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:13.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.246+0000 7f4d52000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13fb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.246+0000 7f4d52000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13fb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55008/0 (socket says 172.21.15.18:55008) 2024-04-16T17:24:13.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.246+0000 7f4d52000640 1 -- 172.21.15.18:0/2984400976 learned_addr learned my addr 172.21.15.18:0/2984400976 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:13.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.246+0000 7f4d53400640 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.246+0000 7f4d52a00640 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c13fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.247+0000 7f4d52000640 1 -- 172.21.15.18:0/2984400976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 msgr2=0x5595c13fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:13.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.247+0000 7f4d52000640 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.247+0000 7f4d52000640 1 -- 172.21.15.18:0/2984400976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 msgr2=0x5595c13fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:13.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.247+0000 7f4d52000640 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c13fb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.247+0000 7f4d52000640 1 -- 172.21.15.18:0/2984400976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c1317920 con 0x5595c13f5670 2024-04-16T17:24:13.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.249+0000 7f4d52000640 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13fb850 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009230 tx=0x7f4d44005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97f30688489a245d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:13.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.249+0000 7f4d40c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:13.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.249+0000 7f4d40c00640 1 -- 172.21.15.18:0/2984400976 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44005000 con 0x5595c13f5670 2024-04-16T17:24:13.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.249+0000 7f4d40c00640 1 -- 172.21.15.18:0/2984400976 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f4d4402cc50 con 0x5595c13f5670 2024-04-16T17:24:13.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.250+0000 7f4d5cc38740 1 -- 172.21.15.18:0/2984400976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d4c09d330 con 0x5595c13f5670 2024-04-16T17:24:13.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.250+0000 7f4d40c00640 1 -- 172.21.15.18:0/2984400976 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4d44005000 con 0x5595c13f5670 2024-04-16T17:24:13.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.250+0000 7f4d5cc38740 10 client.30626 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:13.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.250+0000 7f4d5cc38740 1 -- 172.21.15.18:0/2984400976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5595c12e57f0 con 0x5595c13f5670 2024-04-16T17:24:13.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.251+0000 7f4d40c00640 1 -- 172.21.15.18:0/2984400976 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d440046c0 con 0x5595c13f5670 2024-04-16T17:24:13.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.252+0000 7f4d40c00640 1 -- 172.21.15.18:0/2984400976 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f4d440035f0 con 0x5595c13f5670 2024-04-16T17:24:13.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.252+0000 7f4d5cc38740 10 client.30626 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:13.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.252+0000 7f4d5cc38740 10 client.30626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:13.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.252+0000 7f4d5cc38740 20 client.30626 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:13.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.252+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595c140e620 0x5595c1410a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.252+0000 7f4d5cc38740 4 client.30626 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:13.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.252+0000 7f4d5cc38740 1 -- 172.21.15.18:0/2984400976 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4d4c09d330 con 0x5595c140e620 2024-04-16T17:24:13.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.253+0000 7f4d52a00640 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595c140e620 0x5595c1410a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.253+0000 7f4d52a00640 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595c140e620 0x5595c1410a10 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e0e669c6ead1567 server_cookie=6d26e2f6736376f6 in_seq=0 out_seq=0 2024-04-16T17:24:13.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.254+0000 7f4d40c00640 10 client.30626 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:13.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.255+0000 7f4d40c00640 1 -- 172.21.15.18:0/2984400976 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f4d4c09d330 con 0x5595c140e620 2024-04-16T17:24:13.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.255+0000 7f4d40c00640 10 client.30626 handle_command_reply: tid=0 2024-04-16T17:24:13.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.255+0000 7f4d5cc38740 1 client.30626 shutdown 2024-04-16T17:24:13.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 -- 172.21.15.18:0/2984400976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 msgr2=0x5595c13fb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13fb850 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009230 tx=0x7f4d44005df0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 -- 172.21.15.18:0/2984400976 shutdown_connections 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595c140e620 0x5595c1410a10 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13fb850 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/2984400976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c13fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 -- 172.21.15.18:0/2984400976 >> 172.21.15.18:0/2984400976 conn(0x5595c13ef480 msgr2=0x5595c13f11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d40c00640 0 client.30626 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.256+0000 7f4d5cc38740 1 -- 172.21.15.18:0/2984400976 shutdown_connections 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.257+0000 7f4d5cc38740 1 -- 172.21.15.18:0/2984400976 wait complete. 2024-04-16T17:24:13.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.257+0000 7f4d5cc38740 20 client.30626 trim_cache size 0 max 16384 2024-04-16T17:24:13.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.284+0000 7f4d5cc38740 1 Processor -- start 2024-04-16T17:24:13.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.285+0000 7f4d5cc38740 1 -- start start 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.285+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f4d50 0x5595c1409f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.285+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f5670 0x5595c13fbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.285+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13fe250 0x5595c13ee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.286+0000 7f4d53400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13fe250 0x5595c13ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.286+0000 7f4d53400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13fe250 0x5595c13ee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55014/0 (socket says 172.21.15.18:55014) 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.286+0000 7f4d53400640 1 -- 172.21.15.18:0/1590027759 learned_addr learned my addr 172.21.15.18:0/1590027759 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.286+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541c0e30 con 0x5595c13f5670 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.286+0000 7f4d5cc38740 1 -- 172.21.15.18:0/1590027759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541be700 con 0x5595c13fe250 2024-04-16T17:24:13.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.286+0000 7f4d5cc38740 1 -- 172.21.15.18:0/1590027759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541bc570 con 0x5595c13f4d50 2024-04-16T17:24:13.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.286+0000 7f4d52000640 1 --2- 172.21.15.18:0/1590027759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f5670 0x5595c13fbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.287+0000 7f4d53400640 1 -- 172.21.15.18:0/1590027759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f4d50 msgr2=0x5595c1409f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:13.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.287+0000 7f4d53400640 1 --2- 172.21.15.18:0/1590027759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f4d50 0x5595c1409f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.287+0000 7f4d53400640 1 -- 172.21.15.18:0/1590027759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f5670 msgr2=0x5595c13fbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:13.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.287+0000 7f4d53400640 1 --2- 172.21.15.18:0/1590027759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f5670 0x5595c13fbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.288+0000 7f4d53400640 1 -- 172.21.15.18:0/1590027759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c106c3f0 con 0x5595c13fe250 2024-04-16T17:24:13.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.288+0000 7f4d53400640 1 --2- 172.21.15.18:0/1590027759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13fe250 0x5595c13ee360 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c012f10 tx=0x7f4d3c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae7cbd50452ada9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:13.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.288+0000 7f4d43400640 1 -- 172.21.15.18:0/1590027759 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c027070 con 0x5595c13fe250 2024-04-16T17:24:13.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.289+0000 7f4d43400640 1 -- 172.21.15.18:0/1590027759 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c0238e0 con 0x5595c13fe250 2024-04-16T17:24:13.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.289+0000 7f4d43400640 1 -- 172.21.15.18:0/1590027759 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c022990 con 0x5595c13fe250 2024-04-16T17:24:13.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 -- 172.21.15.18:0/1590027759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13fe250 msgr2=0x5595c13ee360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/1590027759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13fe250 0x5595c13ee360 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c012f10 tx=0x7f4d3c0046c0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 -- 172.21.15.18:0/1590027759 shutdown_connections 2024-04-16T17:24:13.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/1590027759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13fe250 0x5595c13ee360 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/1590027759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f5670 0x5595c13fbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/1590027759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13f4d50 0x5595c1409f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 -- 172.21.15.18:0/1590027759 >> 172.21.15.18:0/1590027759 conn(0x5595c13ef480 msgr2=0x5595c13edc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:13.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 -- 172.21.15.18:0/1590027759 shutdown_connections 2024-04-16T17:24:13.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.290+0000 7f4d5cc38740 1 -- 172.21.15.18:0/1590027759 wait complete. 2024-04-16T17:24:13.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.291+0000 7f4d5cc38740 1 -- start start 2024-04-16T17:24:13.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.291+0000 7f4d5cc38740 1 Processor -- start 2024-04-16T17:24:13.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c1406600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d5cc38740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541bcbc0 con 0x5595c13f4d50 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541be030 con 0x5595c13f5670 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d5cc38740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541bd290 con 0x5595c13fe250 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d5cc38740 10 client.0 _ll_register_callbacks cb 0x7f4d4c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13f80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40790/0 (socket says 172.21.15.18:40790) 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d53400640 1 -- 172.21.15.18:0/3970706698 learned_addr learned my addr 172.21.15.18:0/3970706698 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.293+0000 7f4d52a00640 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c1406600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.294+0000 7f4d53400640 1 -- 172.21.15.18:0/3970706698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 msgr2=0x5595c13f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:13.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.294+0000 7f4d53400640 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.294+0000 7f4d53400640 1 -- 172.21.15.18:0/3970706698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 msgr2=0x5595c1406600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:13.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.294+0000 7f4d53400640 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c1406600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.294+0000 7f4d53400640 1 -- 172.21.15.18:0/3970706698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c11367d0 con 0x5595c13fe250 2024-04-16T17:24:13.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.294+0000 7f4d52a00640 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c1406600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:13.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.295+0000 7f4d53400640 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13f80c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c008010 tx=0x7f4d3c023630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d9cab32fb158f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:13.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.295+0000 7f4d40c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:13.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.295+0000 7f4d40c00640 1 -- 172.21.15.18:0/3970706698 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c039060 con 0x5595c13fe250 2024-04-16T17:24:13.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.295+0000 7f4d5cc38740 1 -- 172.21.15.18:0/3970706698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c0238e0 con 0x5595c13fe250 2024-04-16T17:24:13.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.296+0000 7f4d40c00640 1 -- 172.21.15.18:0/3970706698 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f4d3c023ca0 con 0x5595c13fe250 2024-04-16T17:24:13.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.298+0000 7f4d40c00640 1 -- 172.21.15.18:0/3970706698 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4d3c027050 con 0x5595c13fe250 2024-04-16T17:24:13.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.298+0000 7f4d5cc38740 10 client.30490 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:13.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.298+0000 7f4d5cc38740 1 -- 172.21.15.18:0/3970706698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5595c1474650 con 0x5595c13fe250 2024-04-16T17:24:13.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.299+0000 7f4d40c00640 1 -- 172.21.15.18:0/3970706698 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c0238e0 con 0x5595c13fe250 2024-04-16T17:24:13.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.299+0000 7f4d40c00640 1 -- 172.21.15.18:0/3970706698 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f4d3c02c560 con 0x5595c13fe250 2024-04-16T17:24:13.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.299+0000 7f4d5cc38740 10 client.30490 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:13.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.299+0000 7f4d5cc38740 10 client.30490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:13.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.299+0000 7f4d5cc38740 20 client.30490 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:13.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.299+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595c1474c50 0x5595c1477070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:13.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.299+0000 7f4d5cc38740 4 client.30490 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:24:13.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.299+0000 7f4d5cc38740 1 -- 172.21.15.18:0/3970706698 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595c1367a40 con 0x5595c1474c50 2024-04-16T17:24:13.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.301+0000 7f4d52a00640 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595c1474c50 0x5595c1477070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:13.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.303+0000 7f4d40c00640 10 client.30490 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:13.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.303+0000 7f4d52a00640 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595c1474c50 0x5595c1477070 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5697e1e9c0836b10 server_cookie=d8ee7c433e89e505 in_seq=0 out_seq=0 2024-04-16T17:24:13.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.303+0000 7f4d40c00640 1 -- 172.21.15.18:0/3970706698 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595c1367a40 con 0x5595c1474c50 2024-04-16T17:24:13.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.303+0000 7f4d40c00640 10 client.30490 handle_command_reply: tid=0 2024-04-16T17:24:13.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.303+0000 7f4d5cc38740 1 client.30490 shutdown 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.310+0000 7f4d5cc38740 1 -- 172.21.15.18:0/3970706698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 msgr2=0x5595c13f80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13f80c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c008010 tx=0x7f4d3c023630 comp rx=0 tx=0).stop 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 -- 172.21.15.18:0/3970706698 shutdown_connections 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5595c1474c50 0x5595c1477070 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5595c13fe250 0x5595c13f80c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595c13f5670 0x5595c13f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 --2- 172.21.15.18:0/3970706698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5595c13f4d50 0x5595c1406600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 -- 172.21.15.18:0/3970706698 >> 172.21.15.18:0/3970706698 conn(0x5595c13ef480 msgr2=0x5595c13edc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:13.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d40c00640 0 client.30490 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:13.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 -- 172.21.15.18:0/3970706698 shutdown_connections 2024-04-16T17:24:13.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.311+0000 7f4d5cc38740 1 -- 172.21.15.18:0/3970706698 wait complete. 2024-04-16T17:24:13.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.312+0000 7f4d5cc38740 20 client.30490 trim_cache size 0 max 16384 2024-04-16T17:24:13.315 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 -- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4d380a4020 msgr2=0x7f4d380a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4d380a4020 0x7f4d380a64e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009be0 tx=0x7f4d3c008040 comp rx=0 tx=0).stop 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 -- 172.21.15.18:0/1483034076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110750 msgr2=0x7f4d541a76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110750 0x7f4d541a76e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00ebc0 tx=0x7f4d4c00c6a0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 -- 172.21.15.18:0/1483034076 shutdown_connections 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4d380a4020 0x7f4d380a64e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d5411b7e0 0x7f4d541abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d54110bc0 0x7f4d541a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 --2- 172.21.15.18:0/1483034076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d54110750 0x7f4d541a76e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 -- 172.21.15.18:0/1483034076 >> 172.21.15.18:0/1483034076 conn(0x7f4d5406d0a0 msgr2=0x7f4d5411a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:13.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 -- 172.21.15.18:0/1483034076 shutdown_connections 2024-04-16T17:24:13.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:13.316+0000 7f4d59e00640 1 -- 172.21.15.18:0/1483034076 wait complete. 2024-04-16T17:24:13.562 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88804 11.51 MB/sec execute 265 sec latency 4510.870 ms 2024-04-16T17:24:14.084 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:14 smithi018 ceph-mon[31776]: pgmap v623: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 444 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 461 op/s 2024-04-16T17:24:14.353 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:24:14.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:24:14.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:24:14.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:14 smithi022 ceph-mon[36023]: pgmap v623: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 444 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 461 op/s 2024-04-16T17:24:14.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:14 smithi043 ceph-mon[36901]: pgmap v623: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 444 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 461 op/s 2024-04-16T17:24:14.564 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88829 11.47 MB/sec execute 266 sec latency 4674.880 ms 2024-04-16T17:24:14.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.665+0000 7f3e92600640 1 -- 172.21.15.18:0/464752702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 msgr2=0x7f3e8c0747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.665+0000 7f3e92600640 1 --2- 172.21.15.18:0/464752702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 0x7f3e8c0747b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009900 tx=0x7f3e7c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.666+0000 7f3e92600640 1 -- 172.21.15.18:0/464752702 shutdown_connections 2024-04-16T17:24:14.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.666+0000 7f3e92600640 1 --2- 172.21.15.18:0/464752702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e8c10e870 0x7f3e8c110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.666+0000 7f3e92600640 1 --2- 172.21.15.18:0/464752702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c074cf0 0x7f3e8c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.666+0000 7f3e92600640 1 --2- 172.21.15.18:0/464752702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 0x7f3e8c0747b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.666+0000 7f3e92600640 1 -- 172.21.15.18:0/464752702 >> 172.21.15.18:0/464752702 conn(0x7f3e8c0fdac0 msgr2=0x7f3e8c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:14.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.667+0000 7f3e92600640 1 -- 172.21.15.18:0/464752702 shutdown_connections 2024-04-16T17:24:14.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.667+0000 7f3e92600640 1 -- 172.21.15.18:0/464752702 wait complete. 2024-04-16T17:24:14.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.669+0000 7f3e92600640 1 Processor -- start 2024-04-16T17:24:14.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.669+0000 7f3e92600640 1 -- start start 2024-04-16T17:24:14.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.670+0000 7f3e92600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 0x7f3e8c101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.670+0000 7f3e92600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e8c074cf0 0x7f3e8c1024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.670+0000 7f3e92600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c10e870 0x7f3e8c102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.670+0000 7f3e92600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c114650 con 0x7f3e8c074cf0 2024-04-16T17:24:14.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.670+0000 7f3e92600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1144d0 con 0x7f3e8c0743d0 2024-04-16T17:24:14.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.670+0000 7f3e92600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1147d0 con 0x7f3e8c10e870 2024-04-16T17:24:14.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.672+0000 7f3e8b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 0x7f3e8c101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.672+0000 7f3e8b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 0x7f3e8c101f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55034/0 (socket says 172.21.15.18:55034) 2024-04-16T17:24:14.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.672+0000 7f3e8b400640 1 -- 172.21.15.18:0/1244829788 learned_addr learned my addr 172.21.15.18:0/1244829788 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:14.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.672+0000 7f3e8be00640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c10e870 0x7f3e8c102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.672+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e8c074cf0 0x7f3e8c1024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.673+0000 7f3e8aa00640 1 -- 172.21.15.18:0/1244829788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c10e870 msgr2=0x7f3e8c102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.673+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c10e870 0x7f3e8c102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.673+0000 7f3e8aa00640 1 -- 172.21.15.18:0/1244829788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 msgr2=0x7f3e8c101f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.673+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 0x7f3e8c101f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.673+0000 7f3e8aa00640 1 -- 172.21.15.18:0/1244829788 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8c1ab210 con 0x7f3e8c074cf0 2024-04-16T17:24:14.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.673+0000 7f3e8b400640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 0x7f3e8c101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:14.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.673+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e8c074cf0 0x7f3e8c1024d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9f0 tx=0x7f3e7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:14.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.674+0000 7f3e83e00640 1 -- 172.21.15.18:0/1244829788 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800ccc0 con 0x7f3e8c074cf0 2024-04-16T17:24:14.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.674+0000 7f3e83e00640 1 -- 172.21.15.18:0/1244829788 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e78005020 con 0x7f3e8c074cf0 2024-04-16T17:24:14.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.674+0000 7f3e92600640 1 -- 172.21.15.18:0/1244829788 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8c1ab500 con 0x7f3e8c074cf0 2024-04-16T17:24:14.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.674+0000 7f3e83e00640 1 -- 172.21.15.18:0/1244829788 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78004b20 con 0x7f3e8c074cf0 2024-04-16T17:24:14.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.674+0000 7f3e92600640 1 -- 172.21.15.18:0/1244829788 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e8c1aba50 con 0x7f3e8c074cf0 2024-04-16T17:24:14.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.683+0000 7f3e83e00640 1 -- 172.21.15.18:0/1244829788 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e78004cc0 con 0x7f3e8c074cf0 2024-04-16T17:24:14.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.684+0000 7f3e9538d740 1 Processor -- start 2024-04-16T17:24:14.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.686+0000 7f3e9538d740 1 -- start start 2024-04-16T17:24:14.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.686+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0180080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.686+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01805c0 0x563ed0188c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.686+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01891a0 0x563ed018b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.686+0000 7f3e9538d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563ed00e4cb0 con 0x563ed01891a0 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.686+0000 7f3e9538d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563ed008b370 con 0x563ed01805c0 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.686+0000 7f3e9538d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ed00e0c70 con 0x563ed017fca0 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.687+0000 7f3e8b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01805c0 0x563ed0188c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.687+0000 7f3e8b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01805c0 0x563ed0188c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55050/0 (socket says 172.21.15.18:55050) 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.687+0000 7f3e8b400640 1 -- 172.21.15.18:0/157994261 learned_addr learned my addr 172.21.15.18:0/157994261 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:14.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.687+0000 7f3e83e00640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3e700a3da0 0x7f3e700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.688+0000 7f3e8be00640 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0180080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.688+0000 7f3e83e00640 1 -- 172.21.15.18:0/1244829788 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f3e780d0b60 con 0x7f3e8c074cf0 2024-04-16T17:24:14.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.688+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01891a0 0x563ed018b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.689+0000 7f3e8be00640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3e700a3da0 0x7f3e700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.690+0000 7f3e8aa00640 1 -- 172.21.15.18:0/157994261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 msgr2=0x563ed0180080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:14.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.690+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0180080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.690+0000 7f3e8aa00640 1 -- 172.21.15.18:0/157994261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01805c0 msgr2=0x563ed0188c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:14.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.690+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01805c0 0x563ed0188c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.690+0000 7f3e8aa00640 1 -- 172.21.15.18:0/157994261 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed00aa910 con 0x563ed01891a0 2024-04-16T17:24:14.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.691+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01891a0 0x563ed018b590 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3e780cde50 tx=0x7f3e7809cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d80d6fa757b2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:14.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.691+0000 7f3e8be00640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3e700a3da0 0x7f3e700a6260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c068f00 tx=0x7f3e84004ab0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:14.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.692+0000 7f3e80c00640 1 -- 172.21.15.18:0/157994261 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7801c030 con 0x563ed01891a0 2024-04-16T17:24:14.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.692+0000 7f3e80c00640 1 -- 172.21.15.18:0/157994261 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e780a0a50 con 0x563ed01891a0 2024-04-16T17:24:14.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.692+0000 7f3e80c00640 1 -- 172.21.15.18:0/157994261 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e780051d0 con 0x563ed01891a0 2024-04-16T17:24:14.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.696+0000 7f3e9538d740 1 -- 172.21.15.18:0/157994261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01891a0 msgr2=0x563ed018b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.696+0000 7f3e9538d740 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01891a0 0x563ed018b590 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3e780cde50 tx=0x7f3e7809cda0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.696+0000 7f3e9538d740 1 -- 172.21.15.18:0/157994261 shutdown_connections 2024-04-16T17:24:14.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.696+0000 7f3e9538d740 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01891a0 0x563ed018b590 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.696+0000 7f3e9538d740 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01805c0 0x563ed0188c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.697+0000 7f3e9538d740 1 --2- 172.21.15.18:0/157994261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0180080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.697+0000 7f3e9538d740 1 -- 172.21.15.18:0/157994261 >> 172.21.15.18:0/157994261 conn(0x563ed017a3d0 msgr2=0x563ed017a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.698+0000 7f3e9538d740 1 -- 172.21.15.18:0/157994261 shutdown_connections 2024-04-16T17:24:14.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.698+0000 7f3e9538d740 1 -- 172.21.15.18:0/157994261 wait complete. 2024-04-16T17:24:14.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.699+0000 7f3e9538d740 1 -- start start 2024-04-16T17:24:14.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.699+0000 7f3e9538d740 1 Processor -- start 2024-04-16T17:24:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.700+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0186230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.700+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed0186770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.700+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0186cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.701+0000 7f3e9538d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563ed018bf90 con 0x563ed01805c0 2024-04-16T17:24:14.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.701+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0186cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.701+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0186cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55056/0 (socket says 172.21.15.18:55056) 2024-04-16T17:24:14.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.701+0000 7f3e8be00640 1 -- 172.21.15.18:0/1428080892 learned_addr learned my addr 172.21.15.18:0/1428080892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:14.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.701+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563ed009e730 con 0x563ed01891a0 2024-04-16T17:24:14.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.701+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ed018c110 con 0x563ed017fca0 2024-04-16T17:24:14.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.701+0000 7f3e8b400640 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0186230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e9538d740 10 client.0 _ll_register_callbacks cb 0x563ed006d1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e8b400640 1 -- 172.21.15.18:0/1428080892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 msgr2=0x563ed0186cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e8b400640 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0186cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e8b400640 1 -- 172.21.15.18:0/1428080892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 msgr2=0x563ed0186770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e8b400640 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed0186770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e8b400640 1 -- 172.21.15.18:0/1428080892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ecfebf7e0 con 0x563ed017fca0 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e8be00640 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0186cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e780a0a50 con 0x563ed017fca0 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.702+0000 7f3e8b400640 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0186230 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c02c2e0 tx=0x7f3e7c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b9de174860289a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:14.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.703+0000 7f3e75600640 10 client.30502 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:14.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.704+0000 7f3e75600640 1 -- 172.21.15.18:0/1428080892 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c044070 con 0x563ed017fca0 2024-04-16T17:24:14.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.706+0000 7f3e75600640 1 -- 172.21.15.18:0/1428080892 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f3e7c02c920 con 0x563ed017fca0 2024-04-16T17:24:14.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.706+0000 7f3e75600640 1 -- 172.21.15.18:0/1428080892 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3e7c02cb60 con 0x563ed017fca0 2024-04-16T17:24:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.708+0000 7f3e9538d740 10 client.30502 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:14.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.708+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=37}) -- 0x563ecff1ff70 con 0x563ed017fca0 2024-04-16T17:24:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.709+0000 7f3e75600640 1 -- 172.21.15.18:0/1428080892 <== mon.1 v2:172.21.15.43:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f3e7c004f40 con 0x563ed017fca0 2024-04-16T17:24:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.709+0000 7f3e9538d740 10 client.30502 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.709+0000 7f3e9538d740 10 client.30502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.709+0000 7f3e9538d740 20 client.30502 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.709+0000 7f3e9538d740 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ed0199360 0x563ed019b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.709+0000 7f3e9538d740 4 client.30502 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.709+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed00f26d0 con 0x563ed0199360 2024-04-16T17:24:14.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.710+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ed0199360 0x563ed019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.710+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ed0199360 0x563ed019b750 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10829cef9c103e51 server_cookie=965b691c3d385d9 in_seq=0 out_seq=0 2024-04-16T17:24:14.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.718+0000 7f3e75600640 10 client.30502 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:14.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.718+0000 7f3e75600640 1 -- 172.21.15.18:0/1428080892 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ed00f26d0 con 0x563ed0199360 2024-04-16T17:24:14.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.718+0000 7f3e75600640 10 client.30502 handle_command_reply: tid=0 2024-04-16T17:24:14.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.718+0000 7f3e9538d740 1 client.30502 shutdown 2024-04-16T17:24:14.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.718+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 msgr2=0x563ed0186230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.718+0000 7f3e9538d740 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0186230 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c02c2e0 tx=0x7f3e7c004040 comp rx=0 tx=0).stop 2024-04-16T17:24:14.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.719+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 shutdown_connections 2024-04-16T17:24:14.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.719+0000 7f3e9538d740 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ed0199360 0x563ed019b750 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.719+0000 7f3e75600640 0 client.30502 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:14.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.719+0000 7f3e9538d740 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0186cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.719+0000 7f3e9538d740 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed0186770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.719+0000 7f3e9538d740 1 --2- 172.21.15.18:0/1428080892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0186230 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.719+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 >> 172.21.15.18:0/1428080892 conn(0x563ed017a3d0 msgr2=0x563ed017c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.727+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 shutdown_connections 2024-04-16T17:24:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.727+0000 7f3e9538d740 1 -- 172.21.15.18:0/1428080892 wait complete. 2024-04-16T17:24:14.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.727+0000 7f3e9538d740 20 client.30502 trim_cache size 0 max 16384 2024-04-16T17:24:14.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.773+0000 7f3e9538d740 1 Processor -- start 2024-04-16T17:24:14.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.774+0000 7f3e9538d740 1 -- start start 2024-04-16T17:24:14.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.775+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0188960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.775+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed0185170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.775+0000 7f3e8b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0188960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.776+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0194e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.776+0000 7f3e9538d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b87e0 con 0x563ed01805c0 2024-04-16T17:24:14.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.776+0000 7f3e9538d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b60b0 con 0x563ed01891a0 2024-04-16T17:24:14.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.776+0000 7f3e9538d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b3f20 con 0x563ed017fca0 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.776+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.776+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0194e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55064/0 (socket says 172.21.15.18:55064) 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.776+0000 7f3e8be00640 1 -- 172.21.15.18:0/4266187842 learned_addr learned my addr 172.21.15.18:0/4266187842 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.777+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed0185170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.777+0000 7f3e8be00640 1 -- 172.21.15.18:0/4266187842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 msgr2=0x563ed0188960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.778+0000 7f3e8be00640 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0188960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.778+0000 7f3e8be00640 1 -- 172.21.15.18:0/4266187842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 msgr2=0x563ed0185170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.778+0000 7f3e8be00640 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed0185170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.778+0000 7f3e8be00640 1 -- 172.21.15.18:0/4266187842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ecfdb7170 con 0x563ed01891a0 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.779+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed0185170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:14.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.779+0000 7f3e8b400640 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0188960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:14.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.779+0000 7f3e8be00640 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0194e50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3e84012710 tx=0x7f3e84009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6112690bfb14eba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:14.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.779+0000 7f3e77e00640 1 -- 172.21.15.18:0/4266187842 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84004b20 con 0x563ed01891a0 2024-04-16T17:24:14.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.780+0000 7f3e77e00640 1 -- 172.21.15.18:0/4266187842 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e84004cc0 con 0x563ed01891a0 2024-04-16T17:24:14.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.780+0000 7f3e77e00640 1 -- 172.21.15.18:0/4266187842 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8401f830 con 0x563ed01891a0 2024-04-16T17:24:14.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 -- 172.21.15.18:0/4266187842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 msgr2=0x563ed0194e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0194e50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3e84012710 tx=0x7f3e84009840 comp rx=0 tx=0).stop 2024-04-16T17:24:14.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 -- 172.21.15.18:0/4266187842 shutdown_connections 2024-04-16T17:24:14.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed01891a0 0x563ed0194e50 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed0185170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 --2- 172.21.15.18:0/4266187842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed017fca0 0x563ed0188960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 -- 172.21.15.18:0/4266187842 >> 172.21.15.18:0/4266187842 conn(0x563ed017a3d0 msgr2=0x563ed0186a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 -- 172.21.15.18:0/4266187842 shutdown_connections 2024-04-16T17:24:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.781+0000 7f3e9538d740 1 -- 172.21.15.18:0/4266187842 wait complete. 2024-04-16T17:24:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.782+0000 7f3e9538d740 1 -- start start 2024-04-16T17:24:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.782+0000 7f3e9538d740 1 Processor -- start 2024-04-16T17:24:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.782+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed017fca0 0x563ed018cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed018d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e9538d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed01891a0 0x563ed018d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e9538d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b4570 con 0x563ed01805c0 2024-04-16T17:24:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e9538d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b59e0 con 0x563ed017fca0 2024-04-16T17:24:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e9538d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b4c40 con 0x563ed01891a0 2024-04-16T17:24:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e9538d740 10 client.0 _ll_register_callbacks cb 0x7f3e7c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e8b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed017fca0 0x563ed018cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e8b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed017fca0 0x563ed018cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55078/0 (socket says 172.21.15.18:55078) 2024-04-16T17:24:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed01891a0 0x563ed018d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e8aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed018d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed01891a0 0x563ed018d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40874/0 (socket says 172.21.15.18:40874) 2024-04-16T17:24:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.783+0000 7f3e8be00640 1 -- 172.21.15.18:0/728959193 learned_addr learned my addr 172.21.15.18:0/728959193 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.784+0000 7f3e8b400640 1 -- 172.21.15.18:0/728959193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed01891a0 msgr2=0x563ed018d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.784+0000 7f3e8b400640 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed01891a0 0x563ed018d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.784+0000 7f3e8b400640 1 -- 172.21.15.18:0/728959193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 msgr2=0x563ed018d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.784+0000 7f3e8b400640 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed018d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.784+0000 7f3e8b400640 1 -- 172.21.15.18:0/728959193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed01fe860 con 0x563ed017fca0 2024-04-16T17:24:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.784+0000 7f3e8b400640 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed017fca0 0x563ed018cce0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c006010 tx=0x7f3e7c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32714ace8e0843f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.784+0000 7f3e75600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.785+0000 7f3e75600640 1 -- 172.21.15.18:0/728959193 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c003a00 con 0x563ed017fca0 2024-04-16T17:24:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.785+0000 7f3e75600640 1 -- 172.21.15.18:0/728959193 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f3e7c003800 con 0x563ed017fca0 2024-04-16T17:24:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.785+0000 7f3e9538d740 1 -- 172.21.15.18:0/728959193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e84004cc0 con 0x563ed017fca0 2024-04-16T17:24:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.786+0000 7f3e75600640 1 -- 172.21.15.18:0/728959193 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3e7c003a00 con 0x563ed017fca0 2024-04-16T17:24:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.786+0000 7f3e9538d740 10 client.30644 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.786+0000 7f3e9538d740 1 -- 172.21.15.18:0/728959193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563ed01ffa50 con 0x563ed017fca0 2024-04-16T17:24:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.787+0000 7f3e75600640 1 -- 172.21.15.18:0/728959193 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e84004cc0 con 0x563ed017fca0 2024-04-16T17:24:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.787+0000 7f3e75600640 1 -- 172.21.15.18:0/728959193 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f3e7c034820 con 0x563ed017fca0 2024-04-16T17:24:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.787+0000 7f3e9538d740 10 client.30644 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.787+0000 7f3e9538d740 10 client.30644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.787+0000 7f3e9538d740 20 client.30644 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.787+0000 7f3e9538d740 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ed0200170 0x563ed0202560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.787+0000 7f3e9538d740 4 client.30644 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:24:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.787+0000 7f3e9538d740 1 -- 172.21.15.18:0/728959193 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ed00ec540 con 0x563ed0200170 2024-04-16T17:24:14.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.788+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ed0200170 0x563ed0202560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:14.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.789+0000 7f3e8aa00640 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ed0200170 0x563ed0202560 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ec702961bf021a server_cookie=97599ec69c29dfab in_seq=0 out_seq=0 2024-04-16T17:24:14.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.789+0000 7f3e75600640 10 client.30644 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:14.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.790+0000 7f3e75600640 1 -- 172.21.15.18:0/728959193 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ed00ec540 con 0x563ed0200170 2024-04-16T17:24:14.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.790+0000 7f3e75600640 10 client.30644 handle_command_reply: tid=0 2024-04-16T17:24:14.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.790+0000 7f3e9538d740 1 client.30644 shutdown 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 -- 172.21.15.18:0/728959193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed017fca0 msgr2=0x563ed018cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed017fca0 0x563ed018cce0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c006010 tx=0x7f3e7c009260 comp rx=0 tx=0).stop 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 -- 172.21.15.18:0/728959193 shutdown_connections 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x563ed0200170 0x563ed0202560 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e75600640 0 client.30644 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ed01891a0 0x563ed018d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ed01805c0 0x563ed018d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 --2- 172.21.15.18:0/728959193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ed017fca0 0x563ed018cce0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 -- 172.21.15.18:0/728959193 >> 172.21.15.18:0/728959193 conn(0x563ed017a3d0 msgr2=0x563ed0178ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:14.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 -- 172.21.15.18:0/728959193 shutdown_connections 2024-04-16T17:24:14.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.791+0000 7f3e9538d740 1 -- 172.21.15.18:0/728959193 wait complete. 2024-04-16T17:24:14.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.792+0000 7f3e9538d740 20 client.30644 trim_cache size 0 max 16384 2024-04-16T17:24:14.792 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:24:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:24:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "851ae21c-698a-4b92-9598-729c17c9510e", 2024-04-16T17:24:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:24:14.793 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:24:14.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.793+0000 7f3e92600640 1 -- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3e700a3da0 msgr2=0x7f3e700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.793+0000 7f3e92600640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3e700a3da0 0x7f3e700a6260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c068f00 tx=0x7f3e84004ab0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.793+0000 7f3e92600640 1 -- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e8c074cf0 msgr2=0x7f3e8c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:14.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.793+0000 7f3e92600640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e8c074cf0 0x7f3e8c1024d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9f0 tx=0x7f3e7800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.793+0000 7f3e92600640 1 -- 172.21.15.18:0/1244829788 shutdown_connections 2024-04-16T17:24:14.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.793+0000 7f3e92600640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f3e700a3da0 0x7f3e700a6260 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.794+0000 7f3e92600640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c10e870 0x7f3e8c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.794+0000 7f3e92600640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e8c074cf0 0x7f3e8c1024d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.794+0000 7f3e92600640 1 --2- 172.21.15.18:0/1244829788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e8c0743d0 0x7f3e8c101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:14.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.794+0000 7f3e92600640 1 -- 172.21.15.18:0/1244829788 >> 172.21.15.18:0/1244829788 conn(0x7f3e8c0fdac0 msgr2=0x7f3e8c0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:14.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.797+0000 7f3e92600640 1 -- 172.21.15.18:0/1244829788 shutdown_connections 2024-04-16T17:24:14.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:14.797+0000 7f3e92600640 1 -- 172.21.15.18:0/1244829788 wait complete. 2024-04-16T17:24:14.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:24:14.828 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.161+0000 7f47f8400640 1 -- 172.21.15.18:0/451140228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f0104450 msgr2=0x7f47f0104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.161+0000 7f47f8400640 1 --2- 172.21.15.18:0/451140228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f0104450 0x7f47f0104830 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009900 tx=0x7f47e402bc60 comp rx=0 tx=0).stop 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.164+0000 7f47f8400640 1 -- 172.21.15.18:0/451140228 shutdown_connections 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.164+0000 7f47f8400640 1 --2- 172.21.15.18:0/451140228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47f0109520 0x7f47f01110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.164+0000 7f47f8400640 1 --2- 172.21.15.18:0/451140228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f0104e00 0x7f47f0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.164+0000 7f47f8400640 1 --2- 172.21.15.18:0/451140228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f0104450 0x7f47f0104830 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.164+0000 7f47f8400640 1 -- 172.21.15.18:0/451140228 >> 172.21.15.18:0/451140228 conn(0x7f47f00ffc90 msgr2=0x7f47f01020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.164+0000 7f47f8400640 1 -- 172.21.15.18:0/451140228 shutdown_connections 2024-04-16T17:24:15.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.164+0000 7f47f8400640 1 -- 172.21.15.18:0/451140228 wait complete. 2024-04-16T17:24:15.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.165+0000 7f47f8400640 1 Processor -- start 2024-04-16T17:24:15.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.166+0000 7f47f8400640 1 -- start start 2024-04-16T17:24:15.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.166+0000 7f47f8400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47f0104450 0x7f47f01a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.166+0000 7f47f8400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f0109520 0x7f47f01a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.166+0000 7f47f8400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f019a9e0 0x7f47f019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.166+0000 7f47f8400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0114680 con 0x7f47f0109520 2024-04-16T17:24:15.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.166+0000 7f47f8400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0114500 con 0x7f47f0104450 2024-04-16T17:24:15.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.166+0000 7f47f8400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0114800 con 0x7f47f019a9e0 2024-04-16T17:24:15.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.167+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47f0104450 0x7f47f01a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.167+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47f0104450 0x7f47f01a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55090/0 (socket says 172.21.15.18:55090) 2024-04-16T17:24:15.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.167+0000 7f47f5a00640 1 -- 172.21.15.18:0/3291303990 learned_addr learned my addr 172.21.15.18:0/3291303990 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:15.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.167+0000 7f47f6400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f019a9e0 0x7f47f019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.167+0000 7f47f6400640 1 -- 172.21.15.18:0/3291303990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47f0104450 msgr2=0x7f47f01a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.167+0000 7f47f6400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47f0104450 0x7f47f01a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.168+0000 7f47f6400640 1 -- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f0109520 msgr2=0x7f47f01a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.168+0000 7f47f5000640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f0109520 0x7f47f01a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.168+0000 7f47f6400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f0109520 0x7f47f01a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.168+0000 7f47f6400640 1 -- 172.21.15.18:0/3291303990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f019b500 con 0x7f47f019a9e0 2024-04-16T17:24:15.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.168+0000 7f47f5000640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f0109520 0x7f47f01a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:15.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.169+0000 7f47f6400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f019a9e0 0x7f47f019ae90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f47e800ec50 tx=0x7f47e800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:15.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.169+0000 7f47ee000640 1 -- 172.21.15.18:0/3291303990 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8003b70 con 0x7f47f019a9e0 2024-04-16T17:24:15.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.169+0000 7f47f8400640 1 -- 172.21.15.18:0/3291303990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f019b7f0 con 0x7f47f019a9e0 2024-04-16T17:24:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.169+0000 7f47ee000640 1 -- 172.21.15.18:0/3291303990 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8003d10 con 0x7f47f019a9e0 2024-04-16T17:24:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.169+0000 7f47ee000640 1 -- 172.21.15.18:0/3291303990 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8010890 con 0x7f47f019a9e0 2024-04-16T17:24:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.169+0000 7f47f8400640 1 -- 172.21.15.18:0/3291303990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01a7a40 con 0x7f47f019a9e0 2024-04-16T17:24:15.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.179+0000 7f47ee000640 1 -- 172.21.15.18:0/3291303990 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f47e8004b40 con 0x7f47f019a9e0 2024-04-16T17:24:15.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.179+0000 7f47fb009740 1 Processor -- start 2024-04-16T17:24:15.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.179+0000 7f47fb009740 1 -- start start 2024-04-16T17:24:15.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47fb009740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd9616b0 0x560edd961a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47fb009740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd961fd0 0x560edd95b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47fb009740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 0x560edd975d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47fb009740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560edd8bf120 con 0x560edd9616b0 2024-04-16T17:24:15.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47fb009740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560edd865340 con 0x560edd973930 2024-04-16T17:24:15.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47fb009740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560edd8bb0e0 con 0x560edd961fd0 2024-04-16T17:24:15.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd961fd0 0x560edd95b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd961fd0 0x560edd95b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40904/0 (socket says 172.21.15.18:40904) 2024-04-16T17:24:15.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.180+0000 7f47f5a00640 1 -- 172.21.15.18:0/3123680619 learned_addr learned my addr 172.21.15.18:0/3123680619 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.181+0000 7f47f5a00640 1 -- 172.21.15.18:0/3123680619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 msgr2=0x560edd975d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.181+0000 7f47f5a00640 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 0x560edd975d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.181+0000 7f47f5a00640 1 -- 172.21.15.18:0/3123680619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd9616b0 msgr2=0x560edd961a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.181+0000 7f47f5000640 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd9616b0 0x560edd961a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.181+0000 7f47f5a00640 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd9616b0 0x560edd961a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.182+0000 7f47f5a00640 1 -- 172.21.15.18:0/3123680619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560edd69b7d0 con 0x560edd961fd0 2024-04-16T17:24:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.182+0000 7f47f5000640 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd9616b0 0x560edd961a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.182+0000 7f47f5a00640 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd961fd0 0x560edd95b1f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f47e4036040 tx=0x7f47e4009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c80a8836b6c2a0ef server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.183+0000 7f47ee000640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47d80a3d20 0x7f47d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.183+0000 7f47ee000640 1 -- 172.21.15.18:0/3291303990 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f47e80d0cb0 con 0x7f47f019a9e0 2024-04-16T17:24:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.183+0000 7f47e2a00640 1 -- 172.21.15.18:0/3123680619 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4045070 con 0x560edd961fd0 2024-04-16T17:24:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.183+0000 7f47e2a00640 1 -- 172.21.15.18:0/3123680619 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4004040 con 0x560edd961fd0 2024-04-16T17:24:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.183+0000 7f47e2a00640 1 -- 172.21.15.18:0/3123680619 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4045070 con 0x560edd961fd0 2024-04-16T17:24:15.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.184+0000 7f47fb009740 1 -- 172.21.15.18:0/3123680619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd961fd0 msgr2=0x560edd95b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.184+0000 7f47fb009740 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd961fd0 0x560edd95b1f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f47e4036040 tx=0x7f47e4009510 comp rx=0 tx=0).stop 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.185+0000 7f47fb009740 1 -- 172.21.15.18:0/3123680619 shutdown_connections 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.185+0000 7f47fb009740 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 0x560edd975d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.185+0000 7f47fb009740 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd961fd0 0x560edd95b1f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.185+0000 7f47fb009740 1 --2- 172.21.15.18:0/3123680619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd9616b0 0x560edd961a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.185+0000 7f47fb009740 1 -- 172.21.15.18:0/3123680619 >> 172.21.15.18:0/3123680619 conn(0x560edd95c0f0 msgr2=0x560edd95c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.185+0000 7f47fb009740 1 -- 172.21.15.18:0/3123680619 shutdown_connections 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.185+0000 7f47fb009740 1 -- 172.21.15.18:0/3123680619 wait complete. 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.186+0000 7f47fb009740 1 -- start start 2024-04-16T17:24:15.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.186+0000 7f47fb009740 1 Processor -- start 2024-04-16T17:24:15.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.187+0000 7f47fb009740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd958710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.187+0000 7f47fb009740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd958c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.187+0000 7f47f5000640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47d80a3d20 0x7f47d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.187+0000 7f47fb009740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd959190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.187+0000 7f47f5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd958c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.187+0000 7f47f5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd958c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55098/0 (socket says 172.21.15.18:55098) 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.188+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd958710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.188+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd958710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40912/0 (socket says 172.21.15.18:40912) 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.188+0000 7f47f5000640 1 -- 172.21.15.18:0/616456373 learned_addr learned my addr 172.21.15.18:0/616456373 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.188+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560edd9764a0 con 0x560edd973930 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.188+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560edd878710 con 0x560edd961fd0 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.188+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560edd976620 con 0x560edd9616b0 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.188+0000 7f47fb009740 10 client.0 _ll_register_callbacks cb 0x560edd873af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:15.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.188+0000 7f47f6400640 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd959190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.189+0000 7f47f5a00640 1 -- 172.21.15.18:0/616456373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 msgr2=0x560edd958c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.189+0000 7f47f5a00640 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd958c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.189+0000 7f47f5a00640 1 -- 172.21.15.18:0/616456373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 msgr2=0x560edd959190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.189+0000 7f47f5a00640 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd959190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.189+0000 7f47f5a00640 1 -- 172.21.15.18:0/616456373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560edd5d13f0 con 0x560edd9616b0 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.189+0000 7f47f6400640 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd959190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.189+0000 7f47f5000640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47d80a3d20 0x7f47d80a61e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00bd20 tx=0x7f47dc0146c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.190+0000 7f47f5a00640 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd958710 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f47e4036040 tx=0x7f47e402bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6f1dab7250cbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:15.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.190+0000 7f47d7400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.191+0000 7f47d7400640 1 -- 172.21.15.18:0/616456373 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e404e070 con 0x560edd9616b0 2024-04-16T17:24:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.191+0000 7f47d7400640 1 -- 172.21.15.18:0/616456373 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f47e4005280 con 0x560edd9616b0 2024-04-16T17:24:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.191+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e4004040 con 0x560edd9616b0 2024-04-16T17:24:15.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.191+0000 7f47d7400640 1 -- 172.21.15.18:0/616456373 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f47e4058050 con 0x560edd9616b0 2024-04-16T17:24:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.192+0000 7f47fb009740 10 client.30517 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.192+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560edd5910f0 con 0x560edd9616b0 2024-04-16T17:24:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.192+0000 7f47d7400640 1 -- 172.21.15.18:0/616456373 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4004040 con 0x560edd9616b0 2024-04-16T17:24:15.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.192+0000 7f47d7400640 1 -- 172.21.15.18:0/616456373 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f47e4003c90 con 0x560edd9616b0 2024-04-16T17:24:15.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.192+0000 7f47fb009740 10 client.30517 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:15.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.193+0000 7f47fb009740 10 client.30517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:15.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.193+0000 7f47fb009740 20 client.30517 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:15.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.193+0000 7f47fb009740 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560edd9835d0 0x560edd9859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.193+0000 7f47fb009740 4 client.30517 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:15.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.193+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560edd8ccb40 con 0x560edd9835d0 2024-04-16T17:24:15.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.193+0000 7f47f5000640 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560edd9835d0 0x560edd9859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.194+0000 7f47f5000640 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560edd9835d0 0x560edd9859c0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a346e084776326e server_cookie=49516e37dcf1d5fe in_seq=0 out_seq=0 2024-04-16T17:24:15.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.194+0000 7f47d7400640 10 client.30517 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:15.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.195+0000 7f47d7400640 1 -- 172.21.15.18:0/616456373 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560edd8ccb40 con 0x560edd9835d0 2024-04-16T17:24:15.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.195+0000 7f47d7400640 10 client.30517 handle_command_reply: tid=0 2024-04-16T17:24:15.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.195+0000 7f47fb009740 1 client.30517 shutdown 2024-04-16T17:24:15.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.195+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 msgr2=0x560edd958710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.195+0000 7f47fb009740 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd958710 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f47e4036040 tx=0x7f47e402bc60 comp rx=0 tx=0).stop 2024-04-16T17:24:15.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 shutdown_connections 2024-04-16T17:24:15.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560edd9835d0 0x560edd9859c0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd959190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd958c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 1 --2- 172.21.15.18:0/616456373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd958710 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47d7400640 0 client.30517 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:15.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 >> 172.21.15.18:0/616456373 conn(0x560edd95c0f0 msgr2=0x560edd95de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:15.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 shutdown_connections 2024-04-16T17:24:15.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 1 -- 172.21.15.18:0/616456373 wait complete. 2024-04-16T17:24:15.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.196+0000 7f47fb009740 20 client.30517 trim_cache size 0 max 16384 2024-04-16T17:24:15.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.255+0000 7f47fb009740 1 Processor -- start 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.255+0000 7f47fb009740 1 -- start start 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.255+0000 7f47fb009740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd95ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.255+0000 7f47fb009740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd961fd0 0x560edd9889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.255+0000 7f47fb009740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 0x560edd988f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.255+0000 7f47fb009740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01aff70 con 0x560edd961fd0 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.255+0000 7f47fb009740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01ad840 con 0x560edd973930 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47fb009740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01ab6b0 con 0x560edd9616b0 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd95ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd95ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40928/0 (socket says 172.21.15.18:40928) 2024-04-16T17:24:15.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f5a00640 1 -- 172.21.15.18:0/2078541118 learned_addr learned my addr 172.21.15.18:0/2078541118 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f6400640 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 0x560edd988f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f5a00640 1 -- 172.21.15.18:0/2078541118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 msgr2=0x560edd988f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f5a00640 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 0x560edd988f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f5a00640 1 -- 172.21.15.18:0/2078541118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd961fd0 msgr2=0x560edd9889e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f5a00640 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd961fd0 0x560edd9889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f5a00640 1 -- 172.21.15.18:0/2078541118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560edd962730 con 0x560edd9616b0 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.256+0000 7f47f6400640 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 0x560edd988f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.257+0000 7f47f5a00640 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd95ae40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f47e4000c00 tx=0x7f47e4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d24c4b8b54b9a107 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.257+0000 7f47e2000640 1 -- 172.21.15.18:0/2078541118 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4061070 con 0x560edd9616b0 2024-04-16T17:24:15.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.257+0000 7f47e2000640 1 -- 172.21.15.18:0/2078541118 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4043040 con 0x560edd9616b0 2024-04-16T17:24:15.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.257+0000 7f47e2000640 1 -- 172.21.15.18:0/2078541118 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e404e040 con 0x560edd9616b0 2024-04-16T17:24:15.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.258+0000 7f47fb009740 1 -- 172.21.15.18:0/2078541118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 msgr2=0x560edd95ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.258+0000 7f47fb009740 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd95ae40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f47e4000c00 tx=0x7f47e4005fd0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.259+0000 7f47fb009740 1 -- 172.21.15.18:0/2078541118 shutdown_connections 2024-04-16T17:24:15.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.259+0000 7f47fb009740 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd973930 0x560edd988f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.259+0000 7f47fb009740 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd961fd0 0x560edd9889e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.259+0000 7f47fb009740 1 --2- 172.21.15.18:0/2078541118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd95ae40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.259+0000 7f47fb009740 1 -- 172.21.15.18:0/2078541118 >> 172.21.15.18:0/2078541118 conn(0x560edd95c0f0 msgr2=0x560edd957740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:15.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.260+0000 7f47fb009740 1 -- 172.21.15.18:0/2078541118 shutdown_connections 2024-04-16T17:24:15.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.260+0000 7f47fb009740 1 -- 172.21.15.18:0/2078541118 wait complete. 2024-04-16T17:24:15.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.260+0000 7f47fb009740 1 -- start start 2024-04-16T17:24:15.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.261+0000 7f47fb009740 1 Processor -- start 2024-04-16T17:24:15.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.262+0000 7f47fb009740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd988a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.262+0000 7f47fb009740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd9839b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.262+0000 7f47fb009740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd983ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.262+0000 7f47fb009740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01abd00 con 0x560edd973930 2024-04-16T17:24:15.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.262+0000 7f47fb009740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01ad170 con 0x560edd961fd0 2024-04-16T17:24:15.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.262+0000 7f47fb009740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01ac3d0 con 0x560edd9616b0 2024-04-16T17:24:15.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.262+0000 7f47fb009740 10 client.0 _ll_register_callbacks cb 0x7f47e4009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:15.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.263+0000 7f47f6400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd983ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.263+0000 7f47f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd988a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.263+0000 7f47f6400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd983ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37618/0 (socket says 172.21.15.18:37618) 2024-04-16T17:24:15.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.263+0000 7f47f6400640 1 -- 172.21.15.18:0/2336998414 learned_addr learned my addr 172.21.15.18:0/2336998414 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:15.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.263+0000 7f47f5000640 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd9839b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.264+0000 7f47f5000640 1 -- 172.21.15.18:0/2336998414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 msgr2=0x560edd988a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:15.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.264+0000 7f47f5000640 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd988a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.264+0000 7f47f5000640 1 -- 172.21.15.18:0/2336998414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 msgr2=0x560edd983ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:15.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.264+0000 7f47f5000640 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd983ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.264+0000 7f47f5000640 1 -- 172.21.15.18:0/2336998414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560edd95b950 con 0x560edd961fd0 2024-04-16T17:24:15.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.264+0000 7f47f6400640 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd983ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:15.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.265+0000 7f47f5000640 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd9839b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f47dc015010 tx=0x7f47dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2318379b09750b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:15.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.265+0000 7f47d7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:15.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.266+0000 7f47d7400640 1 -- 172.21.15.18:0/2336998414 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0254e0 con 0x560edd961fd0 2024-04-16T17:24:15.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.266+0000 7f47fb009740 1 -- 172.21.15.18:0/2336998414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e4043040 con 0x560edd961fd0 2024-04-16T17:24:15.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.266+0000 7f47d7400640 1 -- 172.21.15.18:0/2336998414 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f47dc025680 con 0x560edd961fd0 2024-04-16T17:24:15.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.267+0000 7f47d7400640 1 -- 172.21.15.18:0/2336998414 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f47dc035050 con 0x560edd961fd0 2024-04-16T17:24:15.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.275+0000 7f47fb009740 10 client.30659 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:15.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.275+0000 7f47fb009740 1 -- 172.21.15.18:0/2336998414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560edd9faed0 con 0x560edd961fd0 2024-04-16T17:24:15.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.275+0000 7f47d7400640 1 -- 172.21.15.18:0/2336998414 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4043040 con 0x560edd961fd0 2024-04-16T17:24:15.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.276+0000 7f47d7400640 1 -- 172.21.15.18:0/2336998414 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f47dc015040 con 0x560edd961fd0 2024-04-16T17:24:15.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.276+0000 7f47fb009740 10 client.30659 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:15.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.276+0000 7f47fb009740 10 client.30659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:15.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.276+0000 7f47fb009740 20 client.30659 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:15.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.276+0000 7f47fb009740 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560edd9fb5f0 0x560edd9fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:15.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.276+0000 7f47fb009740 4 client.30659 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:24:15.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.276+0000 7f47fb009740 1 -- 172.21.15.18:0/2336998414 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560edd8c69b0 con 0x560edd9fb5f0 2024-04-16T17:24:15.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.276+0000 7f47f5a00640 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560edd9fb5f0 0x560edd9fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:15.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.277+0000 7f47f5a00640 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560edd9fb5f0 0x560edd9fda10 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8563096dbc761684 server_cookie=1ef1af421e8c121e in_seq=0 out_seq=0 2024-04-16T17:24:15.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.277+0000 7f47d7400640 10 client.30659 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:15.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.277+0000 7f47d7400640 1 -- 172.21.15.18:0/2336998414 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x560edd8c69b0 con 0x560edd9fb5f0 2024-04-16T17:24:15.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.277+0000 7f47d7400640 10 client.30659 handle_command_reply: tid=0 2024-04-16T17:24:15.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.277+0000 7f47fb009740 1 client.30659 shutdown 2024-04-16T17:24:15.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 -- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 msgr2=0x560edd9839b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd9839b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f47dc015010 tx=0x7f47dc0046c0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 -- 172.21.15.18:0/2336998414 shutdown_connections 2024-04-16T17:24:15.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560edd9fb5f0 0x560edd9fda10 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560edd973930 0x560edd983ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560edd961fd0 0x560edd9839b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 --2- 172.21.15.18:0/2336998414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560edd9616b0 0x560edd988a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 -- 172.21.15.18:0/2336998414 >> 172.21.15.18:0/2336998414 conn(0x560edd95c0f0 msgr2=0x560edd97e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:15.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47d7400640 0 client.30659 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:15.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.278+0000 7f47fb009740 1 -- 172.21.15.18:0/2336998414 shutdown_connections 2024-04-16T17:24:15.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.279+0000 7f47fb009740 1 -- 172.21.15.18:0/2336998414 wait complete. 2024-04-16T17:24:15.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.279+0000 7f47fb009740 20 client.30659 trim_cache size 0 max 16384 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (257 inodes in the stack)", 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "851ae21c-698a-4b92-9598-729c17c9510e": { 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "851ae21c-698a-4b92-9598-729c17c9510e", 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:24:15.284 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:24:15.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.284+0000 7f47f8400640 1 -- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47d80a3d20 msgr2=0x7f47d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.284+0000 7f47f8400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47d80a3d20 0x7f47d80a61e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00bd20 tx=0x7f47dc0146c0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 -- 172.21.15.18:0/3291303990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f019a9e0 msgr2=0x7f47f019ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:15.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f019a9e0 0x7f47f019ae90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f47e800ec50 tx=0x7f47e800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 -- 172.21.15.18:0/3291303990 shutdown_connections 2024-04-16T17:24:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47d80a3d20 0x7f47d80a61e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f019a9e0 0x7f47f019ae90 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47f0109520 0x7f47f01a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 --2- 172.21.15.18:0/3291303990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47f0104450 0x7f47f01a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 -- 172.21.15.18:0/3291303990 >> 172.21.15.18:0/3291303990 conn(0x7f47f00ffc90 msgr2=0x7f47f0102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 -- 172.21.15.18:0/3291303990 shutdown_connections 2024-04-16T17:24:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:15.285+0000 7f47f8400640 1 -- 172.21.15.18:0/3291303990 wait complete. 2024-04-16T17:24:15.313 INFO:tasks.cephfs.filesystem:scrub status for tag:851ae21c-698a-4b92-9598-729c17c9510e - {'path': '/', 'tag': '851ae21c-698a-4b92-9598-729c17c9510e', 'options': 'recursive,force'} 2024-04-16T17:24:15.565 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88853 11.43 MB/sec execute 267 sec latency 2589.986 ms 2024-04-16T17:24:16.056 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:16 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:24:16.056 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:16 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:16.056 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:16 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:24:16.057 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:16 smithi018 ceph-mon[31776]: pgmap v624: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 508 op/s 2024-04-16T17:24:16.057 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:16 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:24:16.057 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:16 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:24:16.057 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:16 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:16 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:24:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:16 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:16 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:24:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:16 smithi022 ceph-mon[36023]: pgmap v624: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 508 op/s 2024-04-16T17:24:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:16 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:24:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:16 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:24:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:16 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:16.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:16 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:24:16.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:16 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:16.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:16 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:24:16.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:16 smithi043 ceph-mon[36901]: pgmap v624: 129 pgs: 20 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 54 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 508 op/s 2024-04-16T17:24:16.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:16 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:24:16.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:16 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:24:16.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:16 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:16.567 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88901 11.39 MB/sec execute 268 sec latency 2152.494 ms 2024-04-16T17:24:17.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:17 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:24:17.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:17 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:17.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:17 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:17.467 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:17 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:24:17.467 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:17 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:17.467 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:17 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:17.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:17 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:24:17.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:17 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:17.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:17 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:17.567 INFO:tasks.workunit.client.0.smithi018.stdout: 10 88992 11.38 MB/sec execute 269 sec latency 2110.366 ms 2024-04-16T17:24:18.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:18 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:18.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:18 smithi022 ceph-mon[36023]: pgmap v625: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 421 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 492 op/s 2024-04-16T17:24:18.474 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:18 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:18.475 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:18 smithi018 ceph-mon[31776]: pgmap v625: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 421 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 492 op/s 2024-04-16T17:24:18.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:18 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:18.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:18 smithi043 ceph-mon[36901]: pgmap v625: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 421 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 492 op/s 2024-04-16T17:24:18.567 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89020 11.34 MB/sec execute 270 sec latency 1290.555 ms 2024-04-16T17:24:19.567 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89048 11.30 MB/sec execute 271 sec latency 2088.289 ms 2024-04-16T17:24:19.839 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:19 smithi018 ceph-mon[31776]: pgmap v626: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 424 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 3.5 MiB/s wr, 383 op/s 2024-04-16T17:24:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:19 smithi022 ceph-mon[36023]: pgmap v626: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 424 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 3.5 MiB/s wr, 383 op/s 2024-04-16T17:24:20.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:19 smithi043 ceph-mon[36901]: pgmap v626: 129 pgs: 18 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 60 active+clean; 424 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 3.5 MiB/s wr, 383 op/s 2024-04-16T17:24:20.574 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89076 11.27 MB/sec execute 272 sec latency 2670.107 ms 2024-04-16T17:24:21.575 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89282 11.29 MB/sec execute 273 sec latency 1425.702 ms 2024-04-16T17:24:22.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:21 smithi022 ceph-mon[36023]: pgmap v627: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 425 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-04-16T17:24:22.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:21 smithi043 ceph-mon[36901]: pgmap v627: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 425 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-04-16T17:24:22.578 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89542 11.31 MB/sec execute 274 sec latency 1950.183 ms 2024-04-16T17:24:23.579 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89691 11.30 MB/sec execute 275 sec latency 892.039 ms 2024-04-16T17:24:23.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:21 smithi018 ceph-mon[31776]: pgmap v627: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 425 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-04-16T17:24:23.986 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:23 smithi018 ceph-mon[31776]: pgmap v628: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 427 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-04-16T17:24:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.095+0000 7fe227200640 1 -- 172.21.15.18:0/3061756468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe22009cbc0 msgr2=0x7fe22009cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.095+0000 7fe227200640 1 --2- 172.21.15.18:0/3061756468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe22009cbc0 0x7fe22009cfa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe214009900 tx=0x7fe21402bc80 comp rx=0 tx=0).stop 2024-04-16T17:24:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.095+0000 7fe227200640 1 -- 172.21.15.18:0/3061756468 shutdown_connections 2024-04-16T17:24:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.095+0000 7fe227200640 1 --2- 172.21.15.18:0/3061756468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2200a16d0 0x7fe2200a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.095+0000 7fe227200640 1 --2- 172.21.15.18:0/3061756468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe22009d570 0x7fe2200a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.095+0000 7fe227200640 1 --2- 172.21.15.18:0/3061756468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe22009cbc0 0x7fe22009cfa0 secure :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe214009900 tx=0x7fe21402bc80 comp rx=0 tx=0).stop 2024-04-16T17:24:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.095+0000 7fe227200640 1 -- 172.21.15.18:0/3061756468 >> 172.21.15.18:0/3061756468 conn(0x7fe220009290 msgr2=0x7fe2200096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.110+0000 7fe227200640 1 -- 172.21.15.18:0/3061756468 shutdown_connections 2024-04-16T17:24:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.110+0000 7fe227200640 1 -- 172.21.15.18:0/3061756468 wait complete. 2024-04-16T17:24:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.112+0000 7fe227200640 1 Processor -- start 2024-04-16T17:24:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.112+0000 7fe227200640 1 -- start start 2024-04-16T17:24:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.112+0000 7fe227200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2200a16d0 0x7fe2200a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe227200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2200a31d0 0x7fe2200a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe227200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2200a3b70 0x7fe220147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe225400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2200a31d0 0x7fe2200a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe227200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe22000ddb0 con 0x7fe2200a16d0 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe227200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe22000dc30 con 0x7fe2200a3b70 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe227200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe22000df30 con 0x7fe2200a31d0 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe226800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2200a3b70 0x7fe220147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe226800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2200a3b70 0x7fe220147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51722/0 (socket says 172.21.15.18:51722) 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe226800640 1 -- 172.21.15.18:0/2521593337 learned_addr learned my addr 172.21.15.18:0/2521593337 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.113+0000 7fe225e00640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2200a16d0 0x7fe2200a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.114+0000 7fe226800640 1 -- 172.21.15.18:0/2521593337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2200a31d0 msgr2=0x7fe2200a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.114+0000 7fe226800640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2200a31d0 0x7fe2200a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.114+0000 7fe226800640 1 -- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2200a16d0 msgr2=0x7fe2200a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.114+0000 7fe226800640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2200a16d0 0x7fe2200a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.114+0000 7fe226800640 1 -- 172.21.15.18:0/2521593337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe220148060 con 0x7fe2200a3b70 2024-04-16T17:24:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.115+0000 7fe226800640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2200a3b70 0x7fe220147a10 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe228051740 tx=0x7fe228078810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.115+0000 7fe21ea00640 1 -- 172.21.15.18:0/2521593337 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228087070 con 0x7fe2200a3b70 2024-04-16T17:24:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.115+0000 7fe227200640 1 -- 172.21.15.18:0/2521593337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe220148350 con 0x7fe2200a3b70 2024-04-16T17:24:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.116+0000 7fe227200640 1 -- 172.21.15.18:0/2521593337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe220148820 con 0x7fe2200a3b70 2024-04-16T17:24:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.119+0000 7fe21ea00640 1 -- 172.21.15.18:0/2521593337 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe228076900 con 0x7fe2200a3b70 2024-04-16T17:24:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.119+0000 7fe21ea00640 1 -- 172.21.15.18:0/2521593337 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22807f540 con 0x7fe2200a3b70 2024-04-16T17:24:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.121+0000 7fe22f787740 1 Processor -- start 2024-04-16T17:24:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.123+0000 7fe21ea00640 1 -- 172.21.15.18:0/2521593337 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe228076420 con 0x7fe2200a3b70 2024-04-16T17:24:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.125+0000 7fe22f787740 1 -- start start 2024-04-16T17:24:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.126+0000 7fe21ea00640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe20c0a3da0 0x7fe20c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.126+0000 7fe21ea00640 1 -- 172.21.15.18:0/2521593337 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7fe22813c500 con 0x7fe2200a3b70 2024-04-16T17:24:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.127+0000 7fe225400640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe20c0a3da0 0x7fe20c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.127+0000 7fe22f787740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75c7f0 0x56309e75b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.127+0000 7fe22f787740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e75bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.128+0000 7fe225400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e75bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.128+0000 7fe225400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e75bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51738/0 (socket says 172.21.15.18:51738) 2024-04-16T17:24:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.128+0000 7fe225400640 1 -- 172.21.15.18:0/2986230652 learned_addr learned my addr 172.21.15.18:0/2986230652 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.128+0000 7fe22f787740 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e765000 0x56309e7673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.128+0000 7fe225e00640 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75c7f0 0x56309e75b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.129+0000 7fe22f787740 1 -- 172.21.15.18:0/2986230652 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56309e6c0780 con 0x56309e75c7f0 2024-04-16T17:24:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.129+0000 7fe226800640 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e765000 0x56309e7673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.129+0000 7fe22f787740 1 -- 172.21.15.18:0/2986230652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56309e666f40 con 0x56309e75ba80 2024-04-16T17:24:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.129+0000 7fe22f787740 1 -- 172.21.15.18:0/2986230652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56309e6bc740 con 0x56309e765000 2024-04-16T17:24:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.130+0000 7fe225400640 1 -- 172.21.15.18:0/2986230652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e765000 msgr2=0x56309e7673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.130+0000 7fe225400640 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e765000 0x56309e7673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.130+0000 7fe225400640 1 -- 172.21.15.18:0/2986230652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75c7f0 msgr2=0x56309e75b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.130+0000 7fe225400640 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75c7f0 0x56309e75b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.130+0000 7fe225400640 1 -- 172.21.15.18:0/2986230652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309e3d3210 con 0x56309e75ba80 2024-04-16T17:24:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.130+0000 7fe225e00640 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75c7f0 0x56309e75b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.131+0000 7fe225400640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe20c0a3da0 0x7fe20c0a6260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe218013570 tx=0x7fe218013970 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.132+0000 7fe225400640 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e75bf00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe2180118a0 tx=0x7fe218019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a100e3abfffe62 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.132+0000 7fe213000640 1 -- 172.21.15.18:0/2986230652 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218023070 con 0x56309e75ba80 2024-04-16T17:24:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.133+0000 7fe213000640 1 -- 172.21.15.18:0/2986230652 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21800dc20 con 0x56309e75ba80 2024-04-16T17:24:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.133+0000 7fe213000640 1 -- 172.21.15.18:0/2986230652 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218023070 con 0x56309e75ba80 2024-04-16T17:24:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.133+0000 7fe22f787740 1 -- 172.21.15.18:0/2986230652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 msgr2=0x56309e75bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.133+0000 7fe22f787740 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e75bf00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe2180118a0 tx=0x7fe218019000 comp rx=0 tx=0).stop 2024-04-16T17:24:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.134+0000 7fe22f787740 1 -- 172.21.15.18:0/2986230652 shutdown_connections 2024-04-16T17:24:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.134+0000 7fe22f787740 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e765000 0x56309e7673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.134+0000 7fe22f787740 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e75bf00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.134+0000 7fe22f787740 1 --2- 172.21.15.18:0/2986230652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75c7f0 0x56309e75b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.134+0000 7fe22f787740 1 -- 172.21.15.18:0/2986230652 >> 172.21.15.18:0/2986230652 conn(0x56309e6f65d0 msgr2=0x56309e756770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.135+0000 7fe22f787740 1 -- 172.21.15.18:0/2986230652 shutdown_connections 2024-04-16T17:24:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.135+0000 7fe22f787740 1 -- 172.21.15.18:0/2986230652 wait complete. 2024-04-16T17:24:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.136+0000 7fe22f787740 1 -- start start 2024-04-16T17:24:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.136+0000 7fe22f787740 1 Processor -- start 2024-04-16T17:24:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.138+0000 7fe22f787740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e7649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.138+0000 7fe22f787740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e761d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.138+0000 7fe22f787740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e762380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.138+0000 7fe22f787740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56309e767cd0 con 0x56309e765000 2024-04-16T17:24:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.138+0000 7fe22f787740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56309e67a390 con 0x56309e75ba80 2024-04-16T17:24:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.138+0000 7fe22f787740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56309e767e50 con 0x56309e75c7f0 2024-04-16T17:24:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.138+0000 7fe22f787740 10 client.0 _ll_register_callbacks cb 0x56309e663f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e7649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e7649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51742/0 (socket says 172.21.15.18:51742) 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225e00640 1 -- 172.21.15.18:0/2952080567 learned_addr learned my addr 172.21.15.18:0/2952080567 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225400640 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e761d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225e00640 1 -- 172.21.15.18:0/2952080567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 msgr2=0x56309e761d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225e00640 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e761d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225e00640 1 -- 172.21.15.18:0/2952080567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 msgr2=0x56309e762380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225e00640 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e762380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.139+0000 7fe225e00640 1 -- 172.21.15.18:0/2952080567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309e49f620 con 0x56309e75ba80 2024-04-16T17:24:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.140+0000 7fe225e00640 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e7649e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe21402c290 tx=0x7fe2140045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7d9b30f0b91391c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.140+0000 7fe207400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.141+0000 7fe207400640 1 -- 172.21.15.18:0/2952080567 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214043070 con 0x56309e75ba80 2024-04-16T17:24:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.141+0000 7fe207400640 1 -- 172.21.15.18:0/2952080567 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7fe21402cab0 con 0x56309e75ba80 2024-04-16T17:24:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.141+0000 7fe22f787740 1 -- 172.21.15.18:0/2952080567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21800dc20 con 0x56309e75ba80 2024-04-16T17:24:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.142+0000 7fe207400640 1 -- 172.21.15.18:0/2952080567 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe214043070 con 0x56309e75ba80 2024-04-16T17:24:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.142+0000 7fe22f787740 10 client.30671 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.142+0000 7fe22f787740 1 -- 172.21.15.18:0/2952080567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56309e773d80 con 0x56309e75ba80 2024-04-16T17:24:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.143+0000 7fe207400640 1 -- 172.21.15.18:0/2952080567 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21800dc20 con 0x56309e75ba80 2024-04-16T17:24:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.143+0000 7fe207400640 1 -- 172.21.15.18:0/2952080567 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fe214005430 con 0x56309e75ba80 2024-04-16T17:24:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.145+0000 7fe22f787740 10 client.30671 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.145+0000 7fe22f787740 10 client.30671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.145+0000 7fe22f787740 20 client.30671 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.145+0000 7fe22f787740 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56309e7752b0 0x56309e7776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.145+0000 7fe22f787740 4 client.30671 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.145+0000 7fe22f787740 1 -- 172.21.15.18:0/2952080567 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56309e6ce1a0 con 0x56309e7752b0 2024-04-16T17:24:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.146+0000 7fe225400640 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56309e7752b0 0x56309e7776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.147+0000 7fe225400640 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56309e7752b0 0x56309e7776e0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c4dccc3fe78e37 server_cookie=5ce53e8c29476000 in_seq=0 out_seq=0 2024-04-16T17:24:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.147+0000 7fe207400640 10 client.30671 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.148+0000 7fe207400640 1 -- 172.21.15.18:0/2952080567 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56309e6ce1a0 con 0x56309e7752b0 2024-04-16T17:24:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.148+0000 7fe207400640 10 client.30671 handle_command_reply: tid=0 2024-04-16T17:24:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.148+0000 7fe22f787740 1 client.30671 shutdown 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.150+0000 7fe22f787740 1 -- 172.21.15.18:0/2952080567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 msgr2=0x56309e7649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.150+0000 7fe22f787740 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e7649e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe21402c290 tx=0x7fe2140045b0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.150+0000 7fe22f787740 1 -- 172.21.15.18:0/2952080567 shutdown_connections 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.150+0000 7fe22f787740 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56309e7752b0 0x56309e7776e0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.151+0000 7fe22f787740 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e762380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.151+0000 7fe22f787740 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e761d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.151+0000 7fe207400640 0 client.30671 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.151+0000 7fe22f787740 1 --2- 172.21.15.18:0/2952080567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75ba80 0x56309e7649e0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.151+0000 7fe22f787740 1 -- 172.21.15.18:0/2952080567 >> 172.21.15.18:0/2952080567 conn(0x56309e6f65d0 msgr2=0x56309e7584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.151+0000 7fe22f787740 1 -- 172.21.15.18:0/2952080567 shutdown_connections 2024-04-16T17:24:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.151+0000 7fe22f787740 1 -- 172.21.15.18:0/2952080567 wait complete. 2024-04-16T17:24:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.151+0000 7fe22f787740 20 client.30671 trim_cache size 0 max 16384 2024-04-16T17:24:24.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:23 smithi022 ceph-mon[36023]: pgmap v628: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 427 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-04-16T17:24:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.205+0000 7fe22f787740 1 Processor -- start 2024-04-16T17:24:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.208+0000 7fe22f787740 1 -- start start 2024-04-16T17:24:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.209+0000 7fe22f787740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75ba80 0x56309e770c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.209+0000 7fe22f787740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75c7f0 0x56309e760d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.209+0000 7fe22f787740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e7644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.209+0000 7fe22f787740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe208012ad0 con 0x56309e765000 2024-04-16T17:24:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.209+0000 7fe22f787740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2080103a0 con 0x56309e75c7f0 2024-04-16T17:24:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.209+0000 7fe22f787740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe20800e210 con 0x56309e75ba80 2024-04-16T17:24:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.211+0000 7fe226800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e7644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.212+0000 7fe226800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e7644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40710/0 (socket says 172.21.15.18:40710) 2024-04-16T17:24:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.212+0000 7fe226800640 1 -- 172.21.15.18:0/3370195270 learned_addr learned my addr 172.21.15.18:0/3370195270 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.212+0000 7fe225e00640 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75ba80 0x56309e770c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.213+0000 7fe225400640 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75c7f0 0x56309e760d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.213+0000 7fe226800640 1 -- 172.21.15.18:0/3370195270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75ba80 msgr2=0x56309e770c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.213+0000 7fe226800640 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75ba80 0x56309e770c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.213+0000 7fe226800640 1 -- 172.21.15.18:0/3370195270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75c7f0 msgr2=0x56309e760d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.213+0000 7fe226800640 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75c7f0 0x56309e760d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.213+0000 7fe226800640 1 -- 172.21.15.18:0/3370195270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309e752040 con 0x56309e765000 2024-04-16T17:24:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.213+0000 7fe225400640 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75c7f0 0x56309e760d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.214+0000 7fe226800640 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e7644a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe228107fd0 tx=0x7fe22808e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca852656ad220e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.215+0000 7fe212600640 1 -- 172.21.15.18:0/3370195270 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2281079a0 con 0x56309e765000 2024-04-16T17:24:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.215+0000 7fe212600640 1 -- 172.21.15.18:0/3370195270 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe228108740 con 0x56309e765000 2024-04-16T17:24:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.215+0000 7fe212600640 1 -- 172.21.15.18:0/3370195270 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2281046b0 con 0x56309e765000 2024-04-16T17:24:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.216+0000 7fe22f787740 1 -- 172.21.15.18:0/3370195270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 msgr2=0x56309e7644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.216+0000 7fe22f787740 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e7644a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe228107fd0 tx=0x7fe22808e000 comp rx=0 tx=0).stop 2024-04-16T17:24:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.217+0000 7fe22f787740 1 -- 172.21.15.18:0/3370195270 shutdown_connections 2024-04-16T17:24:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.217+0000 7fe22f787740 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e765000 0x56309e7644a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.217+0000 7fe22f787740 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e75c7f0 0x56309e760d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.217+0000 7fe22f787740 1 --2- 172.21.15.18:0/3370195270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75ba80 0x56309e770c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.217+0000 7fe22f787740 1 -- 172.21.15.18:0/3370195270 >> 172.21.15.18:0/3370195270 conn(0x56309e6f65d0 msgr2=0x56309e760640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.218+0000 7fe22f787740 1 -- 172.21.15.18:0/3370195270 shutdown_connections 2024-04-16T17:24:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.218+0000 7fe22f787740 1 -- 172.21.15.18:0/3370195270 wait complete. 2024-04-16T17:24:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.218+0000 7fe22f787740 1 -- start start 2024-04-16T17:24:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.218+0000 7fe22f787740 1 Processor -- start 2024-04-16T17:24:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.223+0000 7fe22f787740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75ba80 0x56309e768c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.223+0000 7fe22f787740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e7691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.223+0000 7fe22f787740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e765000 0x56309e769710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.224+0000 7fe225400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e7691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.224+0000 7fe225400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e7691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:44454/0 (socket says 172.21.15.18:44454) 2024-04-16T17:24:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.224+0000 7fe226800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e765000 0x56309e769710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.225+0000 7fe226800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e765000 0x56309e769710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:51766/0 (socket says 172.21.15.18:51766) 2024-04-16T17:24:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.225+0000 7fe22f787740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe20800e860 con 0x56309e75ba80 2024-04-16T17:24:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.225+0000 7fe22f787740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe20800fcd0 con 0x56309e765000 2024-04-16T17:24:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.225+0000 7fe22f787740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe20800ef30 con 0x56309e75c7f0 2024-04-16T17:24:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.225+0000 7fe22f787740 10 client.0 _ll_register_callbacks cb 0x7fe214004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.225+0000 7fe225400640 1 -- 172.21.15.18:0/3300399886 learned_addr learned my addr 172.21.15.18:0/3300399886 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.226+0000 7fe226800640 1 -- 172.21.15.18:0/3300399886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 msgr2=0x56309e7691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.226+0000 7fe226800640 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e7691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.226+0000 7fe226800640 1 -- 172.21.15.18:0/3300399886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75ba80 msgr2=0x56309e768c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.226+0000 7fe226800640 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75ba80 0x56309e768c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.226+0000 7fe226800640 1 -- 172.21.15.18:0/3300399886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309e7da930 con 0x56309e765000 2024-04-16T17:24:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.226+0000 7fe226800640 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e765000 0x56309e769710 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe228108200 tx=0x7fe22810c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56559692c0cc1ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.226+0000 7fe207400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.227+0000 7fe207400640 1 -- 172.21.15.18:0/3300399886 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2280e6e60 con 0x56309e765000 2024-04-16T17:24:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.227+0000 7fe207400640 1 -- 172.21.15.18:0/3300399886 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7fe2281046b0 con 0x56309e765000 2024-04-16T17:24:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.227+0000 7fe22f787740 1 -- 172.21.15.18:0/3300399886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe228108740 con 0x56309e765000 2024-04-16T17:24:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.228+0000 7fe207400640 1 -- 172.21.15.18:0/3300399886 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe228125070 con 0x56309e765000 2024-04-16T17:24:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.228+0000 7fe22f787740 10 client.30677 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.228+0000 7fe22f787740 1 -- 172.21.15.18:0/3300399886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56309e7dbc20 con 0x56309e765000 2024-04-16T17:24:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.228+0000 7fe207400640 1 -- 172.21.15.18:0/3300399886 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe228108740 con 0x56309e765000 2024-04-16T17:24:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.228+0000 7fe207400640 1 -- 172.21.15.18:0/3300399886 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fe22813c080 con 0x56309e765000 2024-04-16T17:24:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.231+0000 7fe22f787740 10 client.30677 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.231+0000 7fe22f787740 10 client.30677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.231+0000 7fe22f787740 20 client.30677 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.231+0000 7fe22f787740 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56309e7dc0e0 0x56309e7de500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.231+0000 7fe22f787740 4 client.30677 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:24:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.231+0000 7fe22f787740 1 -- 172.21.15.18:0/3300399886 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56309e6c8010 con 0x56309e7dc0e0 2024-04-16T17:24:24.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.231+0000 7fe225e00640 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56309e7dc0e0 0x56309e7de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.232+0000 7fe225e00640 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56309e7dc0e0 0x56309e7de500 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8959a6e814554f7e server_cookie=b26880e18c2b69e0 in_seq=0 out_seq=0 2024-04-16T17:24:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:24.232+0000 7fe207400640 10 client.30677 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:24.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:23 smithi043 ceph-mon[36901]: pgmap v628: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 427 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-04-16T17:24:24.579 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89750 11.27 MB/sec execute 276 sec latency 741.554 ms 2024-04-16T17:24:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.154+0000 7fe207400640 1 -- 172.21.15.18:0/3300399886 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56309e6c8010 con 0x56309e7dc0e0 2024-04-16T17:24:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.155+0000 7fe207400640 10 client.30677 handle_command_reply: tid=0 2024-04-16T17:24:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.155+0000 7fe22f787740 1 client.30677 shutdown 2024-04-16T17:24:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.155+0000 7fe22f787740 1 -- 172.21.15.18:0/3300399886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e765000 msgr2=0x56309e769710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.155+0000 7fe22f787740 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e765000 0x56309e769710 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe228108200 tx=0x7fe22810c320 comp rx=0 tx=0).stop 2024-04-16T17:24:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.156+0000 7fe22f787740 1 -- 172.21.15.18:0/3300399886 shutdown_connections 2024-04-16T17:24:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.156+0000 7fe22f787740 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56309e7dc0e0 0x56309e7de500 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.156+0000 7fe22f787740 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309e765000 0x56309e769710 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.156+0000 7fe22f787740 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56309e75c7f0 0x56309e7691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.156+0000 7fe22f787740 1 --2- 172.21.15.18:0/3300399886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56309e75ba80 0x56309e768c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.156+0000 7fe22f787740 1 -- 172.21.15.18:0/3300399886 >> 172.21.15.18:0/3300399886 conn(0x56309e6f65d0 msgr2=0x56309e757a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.156+0000 7fe207400640 0 client.30677 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.156+0000 7fe22f787740 1 -- 172.21.15.18:0/3300399886 shutdown_connections 2024-04-16T17:24:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.157+0000 7fe22f787740 1 -- 172.21.15.18:0/3300399886 wait complete. 2024-04-16T17:24:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.157+0000 7fe22f787740 20 client.30677 trim_cache size 0 max 16384 2024-04-16T17:24:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:24:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:24:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:24:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.159+0000 7fe227200640 1 -- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe20c0a3da0 msgr2=0x7fe20c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.159+0000 7fe227200640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe20c0a3da0 0x7fe20c0a6260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe218013570 tx=0x7fe218013970 comp rx=0 tx=0).stop 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.159+0000 7fe227200640 1 -- 172.21.15.18:0/2521593337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2200a3b70 msgr2=0x7fe220147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.159+0000 7fe227200640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2200a3b70 0x7fe220147a10 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe228051740 tx=0x7fe228078810 comp rx=0 tx=0).stop 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.160+0000 7fe227200640 1 -- 172.21.15.18:0/2521593337 shutdown_connections 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.160+0000 7fe227200640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe20c0a3da0 0x7fe20c0a6260 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.160+0000 7fe227200640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2200a3b70 0x7fe220147a10 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.160+0000 7fe227200640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2200a31d0 0x7fe2200a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.160+0000 7fe227200640 1 --2- 172.21.15.18:0/2521593337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2200a16d0 0x7fe2200a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.160+0000 7fe227200640 1 -- 172.21.15.18:0/2521593337 >> 172.21.15.18:0/2521593337 conn(0x7fe220009290 msgr2=0x7fe22009bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.160+0000 7fe227200640 1 -- 172.21.15.18:0/2521593337 shutdown_connections 2024-04-16T17:24:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:25.160+0000 7fe227200640 1 -- 172.21.15.18:0/2521593337 wait complete. 2024-04-16T17:24:25.581 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89852 11.26 MB/sec execute 277 sec latency 706.631 ms 2024-04-16T17:24:26.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:25 smithi043 ceph-mon[36901]: pgmap v629: 129 pgs: 12 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 81 active+clean; 426 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-16T17:24:26.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:25 smithi022 ceph-mon[36023]: pgmap v629: 129 pgs: 12 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 81 active+clean; 426 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-16T17:24:26.582 INFO:tasks.workunit.client.0.smithi018.stdout: 10 89977 11.24 MB/sec execute 278 sec latency 191.538 ms 2024-04-16T17:24:26.889 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:25 smithi018 ceph-mon[31776]: pgmap v629: 129 pgs: 12 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 81 active+clean; 426 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-16T17:24:27.582 INFO:tasks.workunit.client.0.smithi018.stdout: 10 90145 11.24 MB/sec execute 279 sec latency 219.447 ms 2024-04-16T17:24:28.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:27 smithi043 ceph-mon[36901]: pgmap v630: 129 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 104 active+clean; 422 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 342 op/s 2024-04-16T17:24:28.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:27 smithi022 ceph-mon[36023]: pgmap v630: 129 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 104 active+clean; 422 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 342 op/s 2024-04-16T17:24:28.582 INFO:tasks.workunit.client.0.smithi018.stdout: 10 90350 11.25 MB/sec execute 280 sec latency 217.897 ms 2024-04-16T17:24:29.582 INFO:tasks.workunit.client.0.smithi018.stdout: 10 90496 11.24 MB/sec execute 281 sec latency 694.213 ms 2024-04-16T17:24:30.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:29 smithi043 ceph-mon[36901]: pgmap v631: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 421 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 320 op/s 2024-04-16T17:24:30.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:29 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:24:30.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:29 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:24:30.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:29 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:24:30.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:29 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:24:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:29 smithi022 ceph-mon[36023]: pgmap v631: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 421 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 320 op/s 2024-04-16T17:24:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:29 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:24:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:29 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:24:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:29 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:24:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:29 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:24:30.582 INFO:tasks.workunit.client.0.smithi018.stdout: 10 90641 11.23 MB/sec execute 282 sec latency 1694.303 ms 2024-04-16T17:24:30.711 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:27 smithi018 ceph-mon[31776]: pgmap v630: 129 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 104 active+clean; 422 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 342 op/s 2024-04-16T17:24:30.711 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:29 smithi018 ceph-mon[31776]: pgmap v631: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 421 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 320 op/s 2024-04-16T17:24:30.711 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:29 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:24:30.711 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:29 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:24:30.712 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:29 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:24:30.712 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:29 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:24:31.583 INFO:tasks.workunit.client.0.smithi018.stdout: 10 90825 11.21 MB/sec execute 283 sec latency 2694.380 ms 2024-04-16T17:24:32.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:31 smithi043 ceph-mon[36901]: pgmap v632: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 420 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 359 op/s 2024-04-16T17:24:32.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:31 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:24:32.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:31 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:24:32.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:31 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:24:32.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:31 smithi022 ceph-mon[36023]: pgmap v632: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 420 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 359 op/s 2024-04-16T17:24:32.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:31 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:24:32.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:31 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:24:32.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:31 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:24:32.584 INFO:tasks.workunit.client.0.smithi018.stdout: 10 91074 11.22 MB/sec execute 284 sec latency 3695.616 ms 2024-04-16T17:24:33.583 INFO:tasks.workunit.client.0.smithi018.stdout: 10 91279 11.23 MB/sec execute 285 sec latency 4695.676 ms 2024-04-16T17:24:34.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:31 smithi018 ceph-mon[31776]: pgmap v632: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 420 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 359 op/s 2024-04-16T17:24:34.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:31 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:24:34.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:31 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:24:34.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:31 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:24:34.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:34 smithi018 ceph-mon[31776]: pgmap v633: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 422 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-04-16T17:24:34.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:34 smithi022 ceph-mon[36023]: pgmap v633: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 422 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-04-16T17:24:34.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:34 smithi043 ceph-mon[36901]: pgmap v633: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 422 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-04-16T17:24:34.589 INFO:tasks.workunit.client.0.smithi018.stdout: 10 91303 11.19 MB/sec execute 286 sec latency 5700.884 ms 2024-04-16T17:24:35.589 INFO:tasks.workunit.client.0.smithi018.stdout: 10 91355 11.15 MB/sec execute 287 sec latency 6473.761 ms 2024-04-16T17:24:36.302 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:36 smithi043 ceph-mon[36901]: pgmap v634: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 425 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 310 op/s 2024-04-16T17:24:36.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:36 smithi022 ceph-mon[36023]: pgmap v634: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 425 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 310 op/s 2024-04-16T17:24:36.589 INFO:tasks.workunit.client.0.smithi018.stdout: 10 91559 11.14 MB/sec execute 288 sec latency 179.369 ms 2024-04-16T17:24:37.590 INFO:tasks.workunit.client.0.smithi018.stdout: 10 91830 11.12 MB/sec execute 289 sec latency 149.353 ms 2024-04-16T17:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.354+0000 7f213ea00640 1 -- 172.21.15.18:0/211375234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2138109270 msgr2=0x7f2138109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.354+0000 7f213ea00640 1 --2- 172.21.15.18:0/211375234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2138109270 0x7f2138109650 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f212c008570 tx=0x7f212c00ac00 comp rx=0 tx=0).stop 2024-04-16T17:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.354+0000 7f213ea00640 1 -- 172.21.15.18:0/211375234 shutdown_connections 2024-04-16T17:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.354+0000 7f213ea00640 1 --2- 172.21.15.18:0/211375234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213810dbb0 0x7f2138115780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.355+0000 7f213ea00640 1 --2- 172.21.15.18:0/211375234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2138109c20 0x7f213810d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.355+0000 7f213ea00640 1 --2- 172.21.15.18:0/211375234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2138109270 0x7f2138109650 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.355+0000 7f213ea00640 1 -- 172.21.15.18:0/211375234 >> 172.21.15.18:0/211375234 conn(0x7f213806cf80 msgr2=0x7f213806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.355+0000 7f213ea00640 1 -- 172.21.15.18:0/211375234 shutdown_connections 2024-04-16T17:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.355+0000 7f213ea00640 1 -- 172.21.15.18:0/211375234 wait complete. 2024-04-16T17:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.357+0000 7f213ea00640 1 Processor -- start 2024-04-16T17:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.357+0000 7f213ea00640 1 -- start start 2024-04-16T17:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.358+0000 7f213ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2138109270 0x7f21381a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.358+0000 7f213ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f213810dbb0 0x7f21381a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.358+0000 7f213ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21381a77d0 0x7f21381a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.358+0000 7f213ea00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2138118800 con 0x7f213810dbb0 2024-04-16T17:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.358+0000 7f213ea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2138118680 con 0x7f21381a77d0 2024-04-16T17:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.358+0000 7f213ea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2138118980 con 0x7f2138109270 2024-04-16T17:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21381a77d0 0x7f21381a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21381a77d0 0x7f21381a7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40398/0 (socket says 172.21.15.18:40398) 2024-04-16T17:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213e000640 1 -- 172.21.15.18:0/4225713861 learned_addr learned my addr 172.21.15.18:0/4225713861 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213d600640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2138109270 0x7f21381a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213cc00640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f213810dbb0 0x7f21381a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213e000640 1 -- 172.21.15.18:0/4225713861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2138109270 msgr2=0x7f21381a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213e000640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2138109270 0x7f21381a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213e000640 1 -- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f213810dbb0 msgr2=0x7f21381a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213e000640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f213810dbb0 0x7f21381a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.359+0000 7f213e000640 1 -- 172.21.15.18:0/4225713861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21381a82c0 con 0x7f21381a77d0 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.360+0000 7f213cc00640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f213810dbb0 0x7f21381a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.361+0000 7f213e000640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21381a77d0 0x7f21381a7c80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f213000abd0 tx=0x7f2130007590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.361+0000 7f2136000640 1 -- 172.21.15.18:0/4225713861 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130007ca0 con 0x7f21381a77d0 2024-04-16T17:24:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.361+0000 7f213ea00640 1 -- 172.21.15.18:0/4225713861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21381a85b0 con 0x7f21381a77d0 2024-04-16T17:24:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.361+0000 7f213ea00640 1 -- 172.21.15.18:0/4225713861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21381afda0 con 0x7f21381a77d0 2024-04-16T17:24:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.361+0000 7f2136000640 1 -- 172.21.15.18:0/4225713861 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2130007e40 con 0x7f21381a77d0 2024-04-16T17:24:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.361+0000 7f2136000640 1 -- 172.21.15.18:0/4225713861 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130014670 con 0x7f21381a77d0 2024-04-16T17:24:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.367+0000 7f2142d6c740 1 Processor -- start 2024-04-16T17:24:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.369+0000 7f2136000640 1 -- 172.21.15.18:0/4225713861 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2130014ae0 con 0x7f21381a77d0 2024-04-16T17:24:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.369+0000 7f2142d6c740 1 -- start start 2024-04-16T17:24:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.369+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 0x5652a5f20540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.369+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f20f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.369+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 0x5652a5f34c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.370+0000 7f2142d6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5652a5e85780 con 0x5652a5f217f0 2024-04-16T17:24:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.370+0000 7f2142d6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5652a5e2bf40 con 0x5652a5f20a80 2024-04-16T17:24:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.370+0000 7f2142d6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5652a5e81740 con 0x5652a5f32810 2024-04-16T17:24:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.371+0000 7f213d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f20f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.371+0000 7f213d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f20f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40410/0 (socket says 172.21.15.18:40410) 2024-04-16T17:24:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.371+0000 7f213d600640 1 -- 172.21.15.18:0/3846004165 learned_addr learned my addr 172.21.15.18:0/3846004165 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.371+0000 7f213cc00640 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 0x5652a5f34c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.372+0000 7f213cc00640 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 0x5652a5f20540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.372+0000 7f2136000640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f21200a3e60 0x7f21200a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.372+0000 7f2136000640 1 -- 172.21.15.18:0/4225713861 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f21300d05e0 con 0x7f21381a77d0 2024-04-16T17:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.372+0000 7f213d600640 1 -- 172.21.15.18:0/3846004165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 msgr2=0x5652a5f34c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.372+0000 7f213d600640 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 0x5652a5f34c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.372+0000 7f213d600640 1 -- 172.21.15.18:0/3846004165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 msgr2=0x5652a5f20540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.372+0000 7f213d600640 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 0x5652a5f20540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.372+0000 7f213d600640 1 -- 172.21.15.18:0/3846004165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a5b98210 con 0x5652a5f20a80 2024-04-16T17:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.373+0000 7f213d600640 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f20f00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f212c02bdb0 tx=0x7f212c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed929118aca7680 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.374+0000 7f213e000640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f21200a3e60 0x7f21200a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.374+0000 7f212a600640 1 -- 172.21.15.18:0/3846004165 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c042030 con 0x5652a5f20a80 2024-04-16T17:24:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.374+0000 7f212a600640 1 -- 172.21.15.18:0/3846004165 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f212c004040 con 0x5652a5f20a80 2024-04-16T17:24:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.374+0000 7f212a600640 1 -- 172.21.15.18:0/3846004165 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c003770 con 0x5652a5f20a80 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.375+0000 7f2142d6c740 1 -- 172.21.15.18:0/3846004165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 msgr2=0x5652a5f20f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.375+0000 7f2142d6c740 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f20f00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f212c02bdb0 tx=0x7f212c000bc0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.376+0000 7f213e000640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f21200a3e60 0x7f21200a6320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2130098520 tx=0x7f21300983e0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.376+0000 7f2142d6c740 1 -- 172.21.15.18:0/3846004165 shutdown_connections 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.376+0000 7f2142d6c740 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 0x5652a5f34c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.376+0000 7f2142d6c740 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f20f00 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.376+0000 7f2142d6c740 1 --2- 172.21.15.18:0/3846004165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 0x5652a5f20540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.376+0000 7f2142d6c740 1 -- 172.21.15.18:0/3846004165 >> 172.21.15.18:0/3846004165 conn(0x5652a5ebb5d0 msgr2=0x5652a5f1b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.377+0000 7f2142d6c740 1 -- 172.21.15.18:0/3846004165 shutdown_connections 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.377+0000 7f2142d6c740 1 -- 172.21.15.18:0/3846004165 wait complete. 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.378+0000 7f2142d6c740 1 -- start start 2024-04-16T17:24:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.378+0000 7f2142d6c740 1 Processor -- start 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.379+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f20a80 0x5652a5f32270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.379+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f217f0 0x5652a5f2f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f213d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f217f0 0x5652a5f2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f213d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f217f0 0x5652a5f2f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40490/0 (socket says 172.21.15.18:40490) 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f213d600640 1 -- 172.21.15.18:0/1984671972 learned_addr learned my addr 172.21.15.18:0/1984671972 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f2142d6c740 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f32810 0x5652a5f2fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5652a5f354e0 con 0x5652a5f20a80 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5652a5e3f390 con 0x5652a5f32810 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5652a5f35660 con 0x5652a5f217f0 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f213d600640 1 -- 172.21.15.18:0/1984671972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f32810 msgr2=0x5652a5f2fd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f213d600640 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f32810 0x5652a5f2fd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f213d600640 1 -- 172.21.15.18:0/1984671972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f20a80 msgr2=0x5652a5f32270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f213cc00640 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f20a80 0x5652a5f32270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.380+0000 7f2142d6c740 10 client.0 _ll_register_callbacks cb 0x5652a5e28f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.381+0000 7f213d600640 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f20a80 0x5652a5f32270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.381+0000 7f213d600640 1 -- 172.21.15.18:0/1984671972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a5c64620 con 0x5652a5f217f0 2024-04-16T17:24:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.381+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f212c004040 con 0x5652a5f217f0 2024-04-16T17:24:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.381+0000 7f213cc00640 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f20a80 0x5652a5f32270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.381+0000 7f213d600640 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f217f0 0x5652a5f2f600 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f212c005ab0 tx=0x7f212c00a3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db6e3372d2581dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.381+0000 7f211ea00640 10 client.30550 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.382+0000 7f211ea00640 1 -- 172.21.15.18:0/1984671972 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c04b020 con 0x5652a5f217f0 2024-04-16T17:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.382+0000 7f211ea00640 1 -- 172.21.15.18:0/1984671972 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f212c0036f0 con 0x5652a5f217f0 2024-04-16T17:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.382+0000 7f211ea00640 1 -- 172.21.15.18:0/1984671972 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f212c042040 con 0x5652a5f217f0 2024-04-16T17:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.382+0000 7f2142d6c740 10 client.30550 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.382+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5652a5f354e0 con 0x5652a5f217f0 2024-04-16T17:24:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.383+0000 7f211ea00640 1 -- 172.21.15.18:0/1984671972 <== mon.1 v2:172.21.15.43:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f212c032030 con 0x5652a5f217f0 2024-04-16T17:24:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.383+0000 7f2142d6c740 10 client.30550 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.383+0000 7f2142d6c740 10 client.30550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.383+0000 7f2142d6c740 20 client.30550 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.383+0000 7f2142d6c740 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5652a5f42680 0x5652a5f44a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.383+0000 7f2142d6c740 4 client.30550 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.383+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652a5e931a0 con 0x5652a5f42680 2024-04-16T17:24:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.384+0000 7f213cc00640 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5652a5f42680 0x5652a5f44a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.385+0000 7f213cc00640 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5652a5f42680 0x5652a5f44a70 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7acce5a0d8bd89bf server_cookie=6c0c57868c5cd774 in_seq=0 out_seq=0 2024-04-16T17:24:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.385+0000 7f211ea00640 10 client.30550 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.386+0000 7f211ea00640 1 -- 172.21.15.18:0/1984671972 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652a5e931a0 con 0x5652a5f42680 2024-04-16T17:24:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.386+0000 7f211ea00640 10 client.30550 handle_command_reply: tid=0 2024-04-16T17:24:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.386+0000 7f2142d6c740 1 client.30550 shutdown 2024-04-16T17:24:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.388+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f217f0 msgr2=0x5652a5f2f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.388+0000 7f2142d6c740 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f217f0 0x5652a5f2f600 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f212c005ab0 tx=0x7f212c00a3a0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.390+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 shutdown_connections 2024-04-16T17:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.390+0000 7f2142d6c740 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5652a5f42680 0x5652a5f44a70 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.390+0000 7f2142d6c740 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f32810 0x5652a5f2fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.390+0000 7f2142d6c740 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f217f0 0x5652a5f2f600 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.390+0000 7f2142d6c740 1 --2- 172.21.15.18:0/1984671972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f20a80 0x5652a5f32270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.390+0000 7f211ea00640 0 client.30550 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.390+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 >> 172.21.15.18:0/1984671972 conn(0x5652a5ebb5d0 msgr2=0x5652a5f1d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.391+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 shutdown_connections 2024-04-16T17:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.391+0000 7f2142d6c740 1 -- 172.21.15.18:0/1984671972 wait complete. 2024-04-16T17:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.391+0000 7f2142d6c740 20 client.30550 trim_cache size 0 max 16384 2024-04-16T17:24:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.419+0000 7f2142d6c740 1 Processor -- start 2024-04-16T17:24:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.419+0000 7f2142d6c740 1 -- start start 2024-04-16T17:24:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.419+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f3e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.419+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 0x5652a5f33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.419+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 0x5652a5f4a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.419+0000 7f2142d6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f21180129d0 con 0x5652a5f217f0 2024-04-16T17:24:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.419+0000 7f2142d6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f21180102a0 con 0x5652a5f20a80 2024-04-16T17:24:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.419+0000 7f2142d6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f211800e110 con 0x5652a5f32810 2024-04-16T17:24:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f3e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f3e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40426/0 (socket says 172.21.15.18:40426) 2024-04-16T17:24:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 -- 172.21.15.18:0/640935269 learned_addr learned my addr 172.21.15.18:0/640935269 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 0x5652a5f4a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 -- 172.21.15.18:0/640935269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 msgr2=0x5652a5f4a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 0x5652a5f4a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213d600640 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 0x5652a5f33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 -- 172.21.15.18:0/640935269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 msgr2=0x5652a5f33c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 0x5652a5f33c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.420+0000 7f213cc00640 1 -- 172.21.15.18:0/640935269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a5f208b0 con 0x5652a5f20a80 2024-04-16T17:24:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.421+0000 7f213cc00640 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f3e470 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2124000c00 tx=0x7f2124005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a38d39728eaac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.422+0000 7f2129c00640 1 -- 172.21.15.18:0/640935269 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124012a20 con 0x5652a5f20a80 2024-04-16T17:24:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.422+0000 7f2129c00640 1 -- 172.21.15.18:0/640935269 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2124012bc0 con 0x5652a5f20a80 2024-04-16T17:24:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.422+0000 7f2129c00640 1 -- 172.21.15.18:0/640935269 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124010df0 con 0x5652a5f20a80 2024-04-16T17:24:38.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:38 smithi022 ceph-mon[36023]: pgmap v635: 129 pgs: 129 active+clean; 428 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-04-16T17:24:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.423+0000 7f2142d6c740 1 -- 172.21.15.18:0/640935269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 msgr2=0x5652a5f3e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.423+0000 7f2142d6c740 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f3e470 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2124000c00 tx=0x7f2124005b70 comp rx=0 tx=0).stop 2024-04-16T17:24:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.423+0000 7f2142d6c740 1 -- 172.21.15.18:0/640935269 shutdown_connections 2024-04-16T17:24:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.423+0000 7f2142d6c740 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f32810 0x5652a5f4a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.423+0000 7f2142d6c740 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f217f0 0x5652a5f33c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.423+0000 7f2142d6c740 1 --2- 172.21.15.18:0/640935269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f20a80 0x5652a5f3e470 secure :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2124000c00 tx=0x7f2124005b70 comp rx=0 tx=0).stop 2024-04-16T17:24:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.423+0000 7f2142d6c740 1 -- 172.21.15.18:0/640935269 >> 172.21.15.18:0/640935269 conn(0x5652a5ebb5d0 msgr2=0x5652a5f1b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.425+0000 7f2142d6c740 1 -- 172.21.15.18:0/640935269 shutdown_connections 2024-04-16T17:24:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.425+0000 7f2142d6c740 1 -- 172.21.15.18:0/640935269 wait complete. 2024-04-16T17:24:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.426+0000 7f2142d6c740 1 -- start start 2024-04-16T17:24:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.426+0000 7f2142d6c740 1 Processor -- start 2024-04-16T17:24:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.427+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f217f0 0x5652a5f3adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.427+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f32810 0x5652a5f339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.427+0000 7f2142d6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f33f00 0x5652a5f34380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.427+0000 7f2142d6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f211800e760 con 0x5652a5f32810 2024-04-16T17:24:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.427+0000 7f2142d6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f211800fbd0 con 0x5652a5f217f0 2024-04-16T17:24:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.427+0000 7f2142d6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f211800ee30 con 0x5652a5f33f00 2024-04-16T17:24:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.427+0000 7f2142d6c740 10 client.0 _ll_register_callbacks cb 0x7f212c02d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.428+0000 7f213d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f32810 0x5652a5f339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.427+0000 7f213cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f217f0 0x5652a5f3adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.428+0000 7f213d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f32810 0x5652a5f339c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56022/0 (socket says 172.21.15.18:56022) 2024-04-16T17:24:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.428+0000 7f213d600640 1 -- 172.21.15.18:0/4139692740 learned_addr learned my addr 172.21.15.18:0/4139692740 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.428+0000 7f213cc00640 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f33f00 0x5652a5f34380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.430+0000 7f213d600640 1 -- 172.21.15.18:0/4139692740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f33f00 msgr2=0x5652a5f34380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.430+0000 7f213d600640 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f33f00 0x5652a5f34380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.430+0000 7f213d600640 1 -- 172.21.15.18:0/4139692740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f217f0 msgr2=0x5652a5f3adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.430+0000 7f213d600640 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f217f0 0x5652a5f3adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.430+0000 7f213d600640 1 -- 172.21.15.18:0/4139692740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a5f39b80 con 0x5652a5f32810 2024-04-16T17:24:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.430+0000 7f213cc00640 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f33f00 0x5652a5f34380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.431+0000 7f213d600640 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f32810 0x5652a5f339c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f212c006010 tx=0x7f212c00a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4c37eec2a8df69 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.431+0000 7f211ea00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:24:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.431+0000 7f211ea00640 1 -- 172.21.15.18:0/4139692740 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c04b070 con 0x5652a5f32810 2024-04-16T17:24:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.431+0000 7f2142d6c740 1 -- 172.21.15.18:0/4139692740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2124012bc0 con 0x5652a5f32810 2024-04-16T17:24:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.432+0000 7f211ea00640 1 -- 172.21.15.18:0/4139692740 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9642+0+0 (secure 0 0 0) 0x7f212c03e440 con 0x5652a5f32810 2024-04-16T17:24:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.432+0000 7f211ea00640 1 -- 172.21.15.18:0/4139692740 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f212c06a050 con 0x5652a5f32810 2024-04-16T17:24:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.433+0000 7f2142d6c740 10 client.19761 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.433+0000 7f2142d6c740 1 -- 172.21.15.18:0/4139692740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5652a5f22410 con 0x5652a5f32810 2024-04-16T17:24:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.433+0000 7f211ea00640 1 -- 172.21.15.18:0/4139692740 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2124012bc0 con 0x5652a5f32810 2024-04-16T17:24:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.433+0000 7f211ea00640 1 -- 172.21.15.18:0/4139692740 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f212c032030 con 0x5652a5f32810 2024-04-16T17:24:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.434+0000 7f2142d6c740 10 client.19761 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.434+0000 7f2142d6c740 10 client.19761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.434+0000 7f2142d6c740 20 client.19761 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.434+0000 7f2142d6c740 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5652a5f22b70 0x5652a5f24f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.434+0000 7f2142d6c740 4 client.19761 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:24:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.434+0000 7f2142d6c740 1 -- 172.21.15.18:0/4139692740 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652a5e8d010 con 0x5652a5f22b70 2024-04-16T17:24:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.434+0000 7f213cc00640 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5652a5f22b70 0x5652a5f24f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.438+0000 7f213cc00640 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5652a5f22b70 0x5652a5f24f90 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3723c30b3feba server_cookie=65d98c5243b2366b in_seq=0 out_seq=0 2024-04-16T17:24:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:38.438+0000 7f211ea00640 10 client.19761 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:38.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:38 smithi043 ceph-mon[36901]: pgmap v635: 129 pgs: 129 active+clean; 428 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-04-16T17:24:38.590 INFO:tasks.workunit.client.0.smithi018.stdout: 10 92165 11.14 MB/sec execute 290 sec latency 194.348 ms 2024-04-16T17:24:39.062 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:36 smithi018 ceph-mon[31776]: pgmap v634: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 425 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 310 op/s 2024-04-16T17:24:39.062 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:38 smithi018 ceph-mon[31776]: pgmap v635: 129 pgs: 129 active+clean; 428 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-04-16T17:24:39.598 INFO:tasks.workunit.client.0.smithi018.stdout: 10 92216 11.10 MB/sec execute 291 sec latency 954.260 ms 2024-04-16T17:24:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:39 smithi022 ceph-mon[36023]: osdmap e131: 12 total, 12 up, 12 in 2024-04-16T17:24:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:39 smithi022 ceph-mon[36023]: pgmap v637: 129 pgs: 129 active+clean; 431 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-04-16T17:24:40.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:39 smithi043 ceph-mon[36901]: osdmap e131: 12 total, 12 up, 12 in 2024-04-16T17:24:40.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:39 smithi043 ceph-mon[36901]: pgmap v637: 129 pgs: 129 active+clean; 431 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-04-16T17:24:40.598 INFO:tasks.workunit.client.0.smithi018.stdout: 10 92469 11.11 MB/sec execute 292 sec latency 947.186 ms 2024-04-16T17:24:41.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 92894 11.18 MB/sec execute 293 sec latency 147.371 ms 2024-04-16T17:24:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:41 smithi022 ceph-mon[36023]: pgmap v638: 129 pgs: 129 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-04-16T17:24:42.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:41 smithi043 ceph-mon[36901]: pgmap v638: 129 pgs: 129 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-04-16T17:24:42.598 INFO:tasks.workunit.client.0.smithi018.stdout: 10 93285 11.24 MB/sec execute 294 sec latency 164.513 ms 2024-04-16T17:24:43.598 INFO:tasks.workunit.client.0.smithi018.stdout: 10 93568 11.26 MB/sec execute 295 sec latency 224.098 ms 2024-04-16T17:24:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.056+0000 7f211ea00640 1 -- 172.21.15.18:0/4139692740 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652a5e8d010 con 0x5652a5f22b70 2024-04-16T17:24:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.056+0000 7f211ea00640 10 client.19761 handle_command_reply: tid=0 2024-04-16T17:24:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.056+0000 7f2142d6c740 1 client.19761 shutdown 2024-04-16T17:24:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f2142d6c740 1 -- 172.21.15.18:0/4139692740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f32810 msgr2=0x5652a5f339c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f2142d6c740 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f32810 0x5652a5f339c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f212c006010 tx=0x7f212c00a0d0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f2142d6c740 1 -- 172.21.15.18:0/4139692740 shutdown_connections 2024-04-16T17:24:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f2142d6c740 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5652a5f22b70 0x5652a5f24f90 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f2142d6c740 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652a5f33f00 0x5652a5f34380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f2142d6c740 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5652a5f32810 0x5652a5f339c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f2142d6c740 1 --2- 172.21.15.18:0/4139692740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652a5f217f0 0x5652a5f3adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f2142d6c740 1 -- 172.21.15.18:0/4139692740 >> 172.21.15.18:0/4139692740 conn(0x5652a5ebb5d0 msgr2=0x5652a5f19f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.059+0000 7f211ea00640 0 client.19761 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.060+0000 7f2142d6c740 1 -- 172.21.15.18:0/4139692740 shutdown_connections 2024-04-16T17:24:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.060+0000 7f2142d6c740 1 -- 172.21.15.18:0/4139692740 wait complete. 2024-04-16T17:24:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.060+0000 7f2142d6c740 20 client.19761 trim_cache size 0 max 16384 2024-04-16T17:24:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:24:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:24:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:24:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:24:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.063+0000 7f213ea00640 1 -- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f21200a3e60 msgr2=0x7f21200a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.063+0000 7f213ea00640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f21200a3e60 0x7f21200a6320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2130098520 tx=0x7f21300983e0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.063+0000 7f213ea00640 1 -- 172.21.15.18:0/4225713861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21381a77d0 msgr2=0x7f21381a7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.063+0000 7f213ea00640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21381a77d0 0x7f21381a7c80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f213000abd0 tx=0x7f2130007590 comp rx=0 tx=0).stop 2024-04-16T17:24:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.064+0000 7f213ea00640 1 -- 172.21.15.18:0/4225713861 shutdown_connections 2024-04-16T17:24:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.064+0000 7f213ea00640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f21200a3e60 0x7f21200a6320 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.064+0000 7f213ea00640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21381a77d0 0x7f21381a7c80 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.064+0000 7f213ea00640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f213810dbb0 0x7f21381a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.064+0000 7f213ea00640 1 --2- 172.21.15.18:0/4225713861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2138109270 0x7f21381a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.064+0000 7f213ea00640 1 -- 172.21.15.18:0/4225713861 >> 172.21.15.18:0/4225713861 conn(0x7f213806cf80 msgr2=0x7f2138071830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.064+0000 7f213ea00640 1 -- 172.21.15.18:0/4225713861 shutdown_connections 2024-04-16T17:24:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:44.065+0000 7f213ea00640 1 -- 172.21.15.18:0/4225713861 wait complete. 2024-04-16T17:24:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:43 smithi022 ceph-mon[36023]: pgmap v639: 129 pgs: 129 active+clean; 442 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 399 op/s 2024-04-16T17:24:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:43 smithi043 ceph-mon[36901]: pgmap v639: 129 pgs: 129 active+clean; 442 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 399 op/s 2024-04-16T17:24:44.598 INFO:tasks.workunit.client.0.smithi018.stdout: 10 93782 11.27 MB/sec execute 296 sec latency 258.900 ms 2024-04-16T17:24:45.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:24:45.314 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:24:45.598 INFO:tasks.workunit.client.0.smithi018.stdout: 10 94022 11.28 MB/sec execute 297 sec latency 188.211 ms 2024-04-16T17:24:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.624+0000 7fe41be00640 1 -- 172.21.15.18:0/704657495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41c076530 msgr2=0x7fe41c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.624+0000 7fe41be00640 1 --2- 172.21.15.18:0/704657495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41c076530 0x7fe41c076910 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009930 tx=0x7fe40c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:24:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.625+0000 7fe41be00640 1 -- 172.21.15.18:0/704657495 shutdown_connections 2024-04-16T17:24:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.625+0000 7fe41be00640 1 --2- 172.21.15.18:0/704657495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c112cb0 0x7fe41c1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.625+0000 7fe41be00640 1 --2- 172.21.15.18:0/704657495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe41c076e50 0x7fe41c112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.625+0000 7fe41be00640 1 --2- 172.21.15.18:0/704657495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41c076530 0x7fe41c076910 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.625+0000 7fe41be00640 1 -- 172.21.15.18:0/704657495 >> 172.21.15.18:0/704657495 conn(0x7fe41c0ffae0 msgr2=0x7fe41c101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:45.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.625+0000 7fe41be00640 1 -- 172.21.15.18:0/704657495 shutdown_connections 2024-04-16T17:24:45.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.626+0000 7fe41be00640 1 -- 172.21.15.18:0/704657495 wait complete. 2024-04-16T17:24:45.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.627+0000 7fe41be00640 1 Processor -- start 2024-04-16T17:24:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.627+0000 7fe41be00640 1 -- start start 2024-04-16T17:24:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.628+0000 7fe41be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe41c076530 0x7fe41c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.628+0000 7fe41be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41c076e50 0x7fe41c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.628+0000 7fe41be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c112cb0 0x7fe41c1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.629+0000 7fe41aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe41c076530 0x7fe41c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.629+0000 7fe41b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c112cb0 0x7fe41c1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.629+0000 7fe41b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c112cb0 0x7fe41c1a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40468/0 (socket says 172.21.15.18:40468) 2024-04-16T17:24:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.629+0000 7fe41b400640 1 -- 172.21.15.18:0/1581248098 learned_addr learned my addr 172.21.15.18:0/1581248098 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.629+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c118850 con 0x7fe41c076530 2024-04-16T17:24:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.629+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1186d0 con 0x7fe41c112cb0 2024-04-16T17:24:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.629+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1189d0 con 0x7fe41c076e50 2024-04-16T17:24:45.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.629+0000 7fe41a000640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41c076e50 0x7fe41c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.630+0000 7fe41b400640 1 -- 172.21.15.18:0/1581248098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41c076e50 msgr2=0x7fe41c19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.630+0000 7fe41b400640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41c076e50 0x7fe41c19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.630+0000 7fe41b400640 1 -- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe41c076530 msgr2=0x7fe41c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.630+0000 7fe41b400640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe41c076530 0x7fe41c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.630+0000 7fe41b400640 1 -- 172.21.15.18:0/1581248098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c1a3d60 con 0x7fe41c112cb0 2024-04-16T17:24:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.630+0000 7fe41aa00640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe41c076530 0x7fe41c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:45.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.630+0000 7fe41b400640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c112cb0 0x7fe41c1a3630 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe41400eb40 tx=0x7fe41400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.631+0000 7fe413400640 1 -- 172.21.15.18:0/1581248098 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414004b20 con 0x7fe41c112cb0 2024-04-16T17:24:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.631+0000 7fe413400640 1 -- 172.21.15.18:0/1581248098 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414004cc0 con 0x7fe41c112cb0 2024-04-16T17:24:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.631+0000 7fe413400640 1 -- 172.21.15.18:0/1581248098 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414010850 con 0x7fe41c112cb0 2024-04-16T17:24:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.631+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41c1a4050 con 0x7fe41c112cb0 2024-04-16T17:24:45.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.631+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41c1ab8e0 con 0x7fe41c112cb0 2024-04-16T17:24:45.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.636+0000 7fe42497e740 1 Processor -- start 2024-04-16T17:24:45.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.638+0000 7fe413400640 1 -- 172.21.15.18:0/1581248098 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe414005020 con 0x7fe41c112cb0 2024-04-16T17:24:45.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.639+0000 7fe42497e740 1 -- start start 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe42497e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 0x55682d27c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe42497e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27ccb0 0x55682d285350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe42497e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d287c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe42497e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55682d1e2120 con 0x55682d27c390 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe42497e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55682d188340 con 0x55682d27ccb0 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe42497e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55682d1de0e0 con 0x55682d285890 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe41aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27ccb0 0x55682d285350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe41aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27ccb0 0x55682d285350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40484/0 (socket says 172.21.15.18:40484) 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.640+0000 7fe41aa00640 1 -- 172.21.15.18:0/274516903 learned_addr learned my addr 172.21.15.18:0/274516903 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:45.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.641+0000 7fe41aa00640 1 -- 172.21.15.18:0/274516903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 msgr2=0x55682d287c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:45.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.641+0000 7fe41aa00640 1 --2- 172.21.15.18:0/274516903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d287c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.641+0000 7fe41aa00640 1 -- 172.21.15.18:0/274516903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 msgr2=0x55682d27c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:45.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.641+0000 7fe41aa00640 1 --2- 172.21.15.18:0/274516903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 0x55682d27c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.641+0000 7fe41aa00640 1 -- 172.21.15.18:0/274516903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682cfbe7d0 con 0x55682d27ccb0 2024-04-16T17:24:45.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.641+0000 7fe41aa00640 1 --2- 172.21.15.18:0/274516903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27ccb0 0x55682d285350 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe40c036040 tx=0x7fe40c005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e6e13a9dfe37712 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:45.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.642+0000 7fe413400640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4000a3da0 0x7fe4000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.642+0000 7fe413400640 1 -- 172.21.15.18:0/1581248098 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fe4140d1480 con 0x7fe41c112cb0 2024-04-16T17:24:45.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.642+0000 7fe407e00640 1 -- 172.21.15.18:0/274516903 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c044020 con 0x55682d27ccb0 2024-04-16T17:24:45.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.642+0000 7fe407e00640 1 -- 172.21.15.18:0/274516903 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c005040 con 0x55682d27ccb0 2024-04-16T17:24:45.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.642+0000 7fe407e00640 1 -- 172.21.15.18:0/274516903 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c004a20 con 0x55682d27ccb0 2024-04-16T17:24:45.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.644+0000 7fe42497e740 1 -- 172.21.15.18:0/274516903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27ccb0 msgr2=0x55682d285350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.644+0000 7fe42497e740 1 --2- 172.21.15.18:0/274516903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27ccb0 0x55682d285350 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe40c036040 tx=0x7fe40c005e50 comp rx=0 tx=0).stop 2024-04-16T17:24:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.644+0000 7fe41a000640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4000a3da0 0x7fe4000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.644+0000 7fe42497e740 1 -- 172.21.15.18:0/274516903 shutdown_connections 2024-04-16T17:24:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.644+0000 7fe42497e740 1 --2- 172.21.15.18:0/274516903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d287c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.644+0000 7fe42497e740 1 --2- 172.21.15.18:0/274516903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27ccb0 0x55682d285350 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.644+0000 7fe42497e740 1 --2- 172.21.15.18:0/274516903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 0x55682d27c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.644+0000 7fe42497e740 1 -- 172.21.15.18:0/274516903 >> 172.21.15.18:0/274516903 conn(0x55682d276ac0 msgr2=0x55682d276ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:45.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.646+0000 7fe42497e740 1 -- 172.21.15.18:0/274516903 shutdown_connections 2024-04-16T17:24:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.646+0000 7fe42497e740 1 -- 172.21.15.18:0/274516903 wait complete. 2024-04-16T17:24:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.646+0000 7fe41a000640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4000a3da0 0x7fe4000a6260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55682d277c50 tx=0x7fe408005f30 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.646+0000 7fe42497e740 1 -- start start 2024-04-16T17:24:45.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.647+0000 7fe42497e740 1 Processor -- start 2024-04-16T17:24:45.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.647+0000 7fe42497e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 0x55682d282970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.647+0000 7fe42497e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d27ccb0 0x55682d282eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.647+0000 7fe42497e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d285890 0x55682d2833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.647+0000 7fe42497e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55682d288620 con 0x55682d27c390 2024-04-16T17:24:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.647+0000 7fe42497e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55682d19b710 con 0x55682d285890 2024-04-16T17:24:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.647+0000 7fe42497e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55682d2887a0 con 0x55682d27ccb0 2024-04-16T17:24:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.647+0000 7fe42497e740 10 client.0 _ll_register_callbacks cb 0x55682d196af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.648+0000 7fe41a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d27ccb0 0x55682d282eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.648+0000 7fe41a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d27ccb0 0x55682d282eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40556/0 (socket says 172.21.15.18:40556) 2024-04-16T17:24:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.648+0000 7fe41a000640 1 -- 172.21.15.18:0/2080310274 learned_addr learned my addr 172.21.15.18:0/2080310274 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.648+0000 7fe41b400640 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d285890 0x55682d2833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.648+0000 7fe41aa00640 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 0x55682d282970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.648+0000 7fe41b400640 1 -- 172.21.15.18:0/2080310274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d27ccb0 msgr2=0x55682d282eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.648+0000 7fe41b400640 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d27ccb0 0x55682d282eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe41b400640 1 -- 172.21.15.18:0/2080310274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 msgr2=0x55682d282970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe41b400640 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 0x55682d282970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe41b400640 1 -- 172.21.15.18:0/2080310274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682cef43f0 con 0x55682d285890 2024-04-16T17:24:45.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe41b400640 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d285890 0x55682d2833f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe4140ce760 tx=0x7fe41409c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=266839a16f9ef7e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:45.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe404c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:45.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe404c00640 1 -- 172.21.15.18:0/2080310274 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4140b5020 con 0x55682d285890 2024-04-16T17:24:45.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe404c00640 1 -- 172.21.15.18:0/2080310274 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fe414005310 con 0x55682d285890 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe42497e740 1 -- 172.21.15.18:0/2080310274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe40c005040 con 0x55682d285890 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.649+0000 7fe404c00640 1 -- 172.21.15.18:0/2080310274 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe4140bd040 con 0x55682d285890 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe42497e740 10 client.30704 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe42497e740 1 -- 172.21.15.18:0/2080310274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55682ceb40f0 con 0x55682d285890 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe404c00640 1 -- 172.21.15.18:0/2080310274 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c005040 con 0x55682d285890 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe404c00640 1 -- 172.21.15.18:0/2080310274 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fe4140a5790 con 0x55682d285890 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe42497e740 10 client.30704 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe42497e740 10 client.30704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe42497e740 20 client.30704 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe42497e740 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55682d295ca0 0x55682d298100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe42497e740 4 client.30704 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.650+0000 7fe42497e740 1 -- 172.21.15.18:0/2080310274 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55682d1efb40 con 0x55682d295ca0 2024-04-16T17:24:45.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.651+0000 7fe41aa00640 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55682d295ca0 0x55682d298100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.651+0000 7fe41aa00640 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55682d295ca0 0x55682d298100 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0e14bc64d174 server_cookie=4d0d896db33e17c9 in_seq=0 out_seq=0 2024-04-16T17:24:45.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.651+0000 7fe404c00640 10 client.30704 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.654+0000 7fe404c00640 1 -- 172.21.15.18:0/2080310274 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55682d1efb40 con 0x55682d295ca0 2024-04-16T17:24:45.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.654+0000 7fe404c00640 10 client.30704 handle_command_reply: tid=0 2024-04-16T17:24:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.654+0000 7fe42497e740 1 client.30704 shutdown 2024-04-16T17:24:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.654+0000 7fe42497e740 1 -- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d285890 msgr2=0x55682d2833f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.654+0000 7fe42497e740 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d285890 0x55682d2833f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe4140ce760 tx=0x7fe41409c9d0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 1 -- 172.21.15.18:0/2080310274 shutdown_connections 2024-04-16T17:24:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55682d295ca0 0x55682d298100 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d285890 0x55682d2833f0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d27ccb0 0x55682d282eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 1 --2- 172.21.15.18:0/2080310274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27c390 0x55682d282970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 1 -- 172.21.15.18:0/2080310274 >> 172.21.15.18:0/2080310274 conn(0x55682d276ac0 msgr2=0x55682d278810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe404c00640 0 client.30704 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:45.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 1 -- 172.21.15.18:0/2080310274 shutdown_connections 2024-04-16T17:24:45.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 1 -- 172.21.15.18:0/2080310274 wait complete. 2024-04-16T17:24:45.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.655+0000 7fe42497e740 20 client.30704 trim_cache size 0 max 16384 2024-04-16T17:24:45.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.694+0000 7fe42497e740 1 Processor -- start 2024-04-16T17:24:45.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.694+0000 7fe42497e740 1 -- start start 2024-04-16T17:24:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.695+0000 7fe42497e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d2818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.695+0000 7fe42497e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d291510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.695+0000 7fe42497e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d27dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.695+0000 7fe42497e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1b8490 con 0x55682d27ccb0 2024-04-16T17:24:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.695+0000 7fe42497e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1b5d60 con 0x55682d27c390 2024-04-16T17:24:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.695+0000 7fe42497e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1b3bd0 con 0x55682d285890 2024-04-16T17:24:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.695+0000 7fe41aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d2818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d291510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d2818f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40492/0 (socket says 172.21.15.18:40492) 2024-04-16T17:24:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41aa00640 1 -- 172.21.15.18:0/586119894 learned_addr learned my addr 172.21.15.18:0/586119894 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41b400640 1 --2- 172.21.15.18:0/586119894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d27dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41b400640 1 -- 172.21.15.18:0/586119894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 msgr2=0x55682d2818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41b400640 1 --2- 172.21.15.18:0/586119894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d2818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41b400640 1 -- 172.21.15.18:0/586119894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 msgr2=0x55682d291510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41b400640 1 --2- 172.21.15.18:0/586119894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d291510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.696+0000 7fe41b400640 1 -- 172.21.15.18:0/586119894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682d27cae0 con 0x55682d285890 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.697+0000 7fe41b400640 1 --2- 172.21.15.18:0/586119894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d27dc50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe414099980 tx=0x7fe41409ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab94316a6e29720 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.697+0000 7fe407400640 1 -- 172.21.15.18:0/586119894 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4140b5070 con 0x55682d285890 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.697+0000 7fe407400640 1 -- 172.21.15.18:0/586119894 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41409cfb0 con 0x55682d285890 2024-04-16T17:24:45.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.697+0000 7fe407400640 1 -- 172.21.15.18:0/586119894 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4140b6530 con 0x55682d285890 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 -- 172.21.15.18:0/586119894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 msgr2=0x55682d27dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 --2- 172.21.15.18:0/586119894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d27dc50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe414099980 tx=0x7fe41409ca40 comp rx=0 tx=0).stop 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 -- 172.21.15.18:0/586119894 shutdown_connections 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 --2- 172.21.15.18:0/586119894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d27dc50 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 --2- 172.21.15.18:0/586119894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d291510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 --2- 172.21.15.18:0/586119894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d2818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 -- 172.21.15.18:0/586119894 >> 172.21.15.18:0/586119894 conn(0x55682d276ac0 msgr2=0x55682d278700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 -- 172.21.15.18:0/586119894 shutdown_connections 2024-04-16T17:24:45.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.699+0000 7fe42497e740 1 -- 172.21.15.18:0/586119894 wait complete. 2024-04-16T17:24:45.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.700+0000 7fe42497e740 1 -- start start 2024-04-16T17:24:45.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.700+0000 7fe42497e740 1 Processor -- start 2024-04-16T17:24:45.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.701+0000 7fe42497e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d27f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.701+0000 7fe42497e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d27fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.701+0000 7fe42497e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d280000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.701+0000 7fe42497e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1b4220 con 0x55682d27ccb0 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.701+0000 7fe42497e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1b5690 con 0x55682d27c390 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.701+0000 7fe42497e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1b48f0 con 0x55682d285890 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.701+0000 7fe42497e740 10 client.0 _ll_register_callbacks cb 0x7fe414005920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.701+0000 7fe41a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d27fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d27fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56066/0 (socket says 172.21.15.18:56066) 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41a000640 1 -- 172.21.15.18:0/3950463848 learned_addr learned my addr 172.21.15.18:0/3950463848 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41aa00640 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d27f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41b400640 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d280000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41aa00640 1 -- 172.21.15.18:0/3950463848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 msgr2=0x55682d280000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:45.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41aa00640 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d280000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41aa00640 1 -- 172.21.15.18:0/3950463848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 msgr2=0x55682d27fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41aa00640 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d27fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41aa00640 1 -- 172.21.15.18:0/3950463848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682d30c250 con 0x55682d27c390 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe41aa00640 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d27f580 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005ec0 tx=0x7fe40c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e05c55821a75f5da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.702+0000 7fe404c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.703+0000 7fe404c00640 1 -- 172.21.15.18:0/3950463848 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c044040 con 0x55682d27c390 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.703+0000 7fe404c00640 1 -- 172.21.15.18:0/3950463848 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fe40c005270 con 0x55682d27c390 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.703+0000 7fe42497e740 1 -- 172.21.15.18:0/3950463848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41409cfb0 con 0x55682d27c390 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.703+0000 7fe404c00640 1 -- 172.21.15.18:0/3950463848 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe40c02c9a0 con 0x55682d27c390 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.703+0000 7fe42497e740 10 client.30710 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.703+0000 7fe42497e740 1 -- 172.21.15.18:0/3950463848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55682d30d450 con 0x55682d27c390 2024-04-16T17:24:45.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.704+0000 7fe404c00640 1 -- 172.21.15.18:0/3950463848 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41409cfb0 con 0x55682d27c390 2024-04-16T17:24:45.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.704+0000 7fe404c00640 1 -- 172.21.15.18:0/3950463848 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fe40c0516b0 con 0x55682d27c390 2024-04-16T17:24:45.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.705+0000 7fe42497e740 10 client.30710 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:45.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.705+0000 7fe42497e740 10 client.30710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:45.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.705+0000 7fe42497e740 20 client.30710 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:45.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.705+0000 7fe42497e740 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55682d30d730 0x55682d30fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:45.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.705+0000 7fe42497e740 4 client.30710 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:24:45.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.705+0000 7fe42497e740 1 -- 172.21.15.18:0/3950463848 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55682d1e99b0 con 0x55682d30d730 2024-04-16T17:24:45.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.705+0000 7fe41a000640 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55682d30d730 0x55682d30fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:45.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.706+0000 7fe41a000640 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55682d30d730 0x55682d30fb50 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22aacd89ec8d6d1a server_cookie=31bda1fd6ab64219 in_seq=0 out_seq=0 2024-04-16T17:24:45.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.706+0000 7fe404c00640 10 client.30710 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:45.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.709+0000 7fe404c00640 1 -- 172.21.15.18:0/3950463848 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55682d1e99b0 con 0x55682d30d730 2024-04-16T17:24:45.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.710+0000 7fe404c00640 10 client.30710 handle_command_reply: tid=0 2024-04-16T17:24:45.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.710+0000 7fe42497e740 1 client.30710 shutdown 2024-04-16T17:24:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.711+0000 7fe42497e740 1 -- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 msgr2=0x55682d27f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.711+0000 7fe42497e740 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d27f580 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005ec0 tx=0x7fe40c0046c0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe42497e740 1 -- 172.21.15.18:0/3950463848 shutdown_connections 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe42497e740 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55682d30d730 0x55682d30fb50 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe42497e740 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55682d285890 0x55682d280000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe42497e740 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55682d27ccb0 0x55682d27fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe42497e740 1 --2- 172.21.15.18:0/3950463848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55682d27c390 0x55682d27f580 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe42497e740 1 -- 172.21.15.18:0/3950463848 >> 172.21.15.18:0/3950463848 conn(0x55682d276ac0 msgr2=0x55682d275290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe404c00640 0 client.30710 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe42497e740 1 -- 172.21.15.18:0/3950463848 shutdown_connections 2024-04-16T17:24:45.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.712+0000 7fe42497e740 1 -- 172.21.15.18:0/3950463848 wait complete. 2024-04-16T17:24:45.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.713+0000 7fe42497e740 20 client.30710 trim_cache size 0 max 16384 2024-04-16T17:24:45.713 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:24:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:24:45.713 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:24:45.713 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.713+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4000a3da0 msgr2=0x7fe4000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.713+0000 7fe41be00640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4000a3da0 0x7fe4000a6260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55682d277c50 tx=0x7fe408005f30 comp rx=0 tx=0).stop 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.713+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c112cb0 msgr2=0x7fe41c1a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.713+0000 7fe41be00640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c112cb0 0x7fe41c1a3630 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe41400eb40 tx=0x7fe41400c2d0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.714+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 shutdown_connections 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.714+0000 7fe41be00640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4000a3da0 0x7fe4000a6260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.714+0000 7fe41be00640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c112cb0 0x7fe41c1a3630 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.714+0000 7fe41be00640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41c076e50 0x7fe41c19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.714+0000 7fe41be00640 1 --2- 172.21.15.18:0/1581248098 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe41c076530 0x7fe41c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.714+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 >> 172.21.15.18:0/1581248098 conn(0x7fe41c0ffae0 msgr2=0x7fe41c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.714+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 shutdown_connections 2024-04-16T17:24:45.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:45.714+0000 7fe41be00640 1 -- 172.21.15.18:0/1581248098 wait complete. 2024-04-16T17:24:45.745 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:24:45.745 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:24:46.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 -- 172.21.15.18:0/1153780285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f10112ed0 msgr2=0x7f8f101152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 --2- 172.21.15.18:0/1153780285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f10112ed0 0x7f8f101152c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009ca0 tx=0x7f8f0802c180 comp rx=0 tx=0).stop 2024-04-16T17:24:46.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 -- 172.21.15.18:0/1153780285 shutdown_connections 2024-04-16T17:24:46.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 --2- 172.21.15.18:0/1153780285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f10112ed0 0x7f8f101152c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 --2- 172.21.15.18:0/1153780285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f10074cc0 0x7f8f10112990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 --2- 172.21.15.18:0/1153780285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f100743a0 0x7f8f10074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 -- 172.21.15.18:0/1153780285 >> 172.21.15.18:0/1153780285 conn(0x7f8f100fd9f0 msgr2=0x7f8f100ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:46.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 -- 172.21.15.18:0/1153780285 shutdown_connections 2024-04-16T17:24:46.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.045+0000 7f8f16000640 1 -- 172.21.15.18:0/1153780285 wait complete. 2024-04-16T17:24:46.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.046+0000 7f8f16000640 1 Processor -- start 2024-04-16T17:24:46.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.047+0000 7f8f16000640 1 -- start start 2024-04-16T17:24:46.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.047+0000 7f8f16000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f100743a0 0x7f8f1019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.047+0000 7f8f16000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f10074cc0 0x7f8f1019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.047+0000 7f8f16000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f10112ed0 0x7f8f101a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.048+0000 7f8f0fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f10074cc0 0x7f8f1019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.048+0000 7f8f0fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f10074cc0 0x7f8f1019f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56088/0 (socket says 172.21.15.18:56088) 2024-04-16T17:24:46.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.048+0000 7f8f0fe00640 1 -- 172.21.15.18:0/358271309 learned_addr learned my addr 172.21.15.18:0/358271309 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:46.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.048+0000 7f8f15600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f10112ed0 0x7f8f101a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.048+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101188f0 con 0x7f8f10074cc0 2024-04-16T17:24:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.048+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10118770 con 0x7f8f10112ed0 2024-04-16T17:24:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.048+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10118a70 con 0x7f8f100743a0 2024-04-16T17:24:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.048+0000 7f8f14c00640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f100743a0 0x7f8f1019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.049+0000 7f8f14c00640 1 -- 172.21.15.18:0/358271309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f10112ed0 msgr2=0x7f8f101a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.049+0000 7f8f14c00640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f10112ed0 0x7f8f101a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.049+0000 7f8f14c00640 1 -- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f10074cc0 msgr2=0x7f8f1019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.049+0000 7f8f14c00640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f10074cc0 0x7f8f1019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.049+0000 7f8f14c00640 1 -- 172.21.15.18:0/358271309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f101a3df0 con 0x7f8f100743a0 2024-04-16T17:24:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.049+0000 7f8f0fe00640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f10074cc0 0x7f8f1019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.049+0000 7f8f14c00640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f100743a0 0x7f8f1019ee50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b3f0 tx=0x7f8f0400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.050+0000 7f8f0d600640 1 -- 172.21.15.18:0/358271309 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04015020 con 0x7f8f100743a0 2024-04-16T17:24:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.050+0000 7f8f0d600640 1 -- 172.21.15.18:0/358271309 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f04011d10 con 0x7f8f100743a0 2024-04-16T17:24:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.050+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f101a40e0 con 0x7f8f100743a0 2024-04-16T17:24:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.050+0000 7f8f0d600640 1 -- 172.21.15.18:0/358271309 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400f2e0 con 0x7f8f100743a0 2024-04-16T17:24:46.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.050+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101ab970 con 0x7f8f100743a0 2024-04-16T17:24:46.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.052+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1004fe00 con 0x7f8f100743a0 2024-04-16T17:24:46.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.055+0000 7f8f0d600640 1 -- 172.21.15.18:0/358271309 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f04018020 con 0x7f8f100743a0 2024-04-16T17:24:46.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.058+0000 7f8f0d600640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ef80a3ee0 0x7f8ef80a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.059+0000 7f8f0d600640 1 -- 172.21.15.18:0/358271309 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f8f040d0290 con 0x7f8f100743a0 2024-04-16T17:24:46.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.059+0000 7f8f0fe00640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ef80a3ee0 0x7f8ef80a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.061+0000 7f8f0fe00640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ef80a3ee0 0x7f8ef80a63a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8f10105cc0 tx=0x7f8efc005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:46.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.062+0000 7f8f0d600640 1 -- 172.21.15.18:0/358271309 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f0401f020 con 0x7f8f100743a0 2024-04-16T17:24:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:45 smithi022 ceph-mon[36023]: pgmap v640: 129 pgs: 129 active+clean; 439 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 393 op/s 2024-04-16T17:24:46.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:45 smithi043 ceph-mon[36901]: pgmap v640: 129 pgs: 129 active+clean; 439 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 393 op/s 2024-04-16T17:24:46.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.551+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f10105360 con 0x7f8f100743a0 2024-04-16T17:24:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.554+0000 7f8f0d600640 1 -- 172.21.15.18:0/358271309 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7f8f0409c5d0 con 0x7f8f100743a0 2024-04-16T17:24:46.555 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:24:46.555 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:24:46.555 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:24:46.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.560+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ef80a3ee0 msgr2=0x7f8ef80a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.560+0000 7f8f16000640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ef80a3ee0 0x7f8ef80a63a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8f10105cc0 tx=0x7f8efc005f90 comp rx=0 tx=0).stop 2024-04-16T17:24:46.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.560+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f100743a0 msgr2=0x7f8f1019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.560+0000 7f8f16000640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f100743a0 0x7f8f1019ee50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b3f0 tx=0x7f8f0400b8b0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.561+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 shutdown_connections 2024-04-16T17:24:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.561+0000 7f8f16000640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8ef80a3ee0 0x7f8ef80a63a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.561+0000 7f8f16000640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f10112ed0 0x7f8f101a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.561+0000 7f8f16000640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f10074cc0 0x7f8f1019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.561+0000 7f8f16000640 1 --2- 172.21.15.18:0/358271309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f100743a0 0x7f8f1019ee50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.561+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 >> 172.21.15.18:0/358271309 conn(0x7f8f100fd9f0 msgr2=0x7f8f100fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.561+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 shutdown_connections 2024-04-16T17:24:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.561+0000 7f8f16000640 1 -- 172.21.15.18:0/358271309 wait complete. 2024-04-16T17:24:46.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:24:46.599 INFO:tasks.workunit.client.0.smithi018.stdout: 10 94235 11.27 MB/sec execute 298 sec latency 134.402 ms 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.925+0000 7f4889c00640 1 -- 172.21.15.18:0/1516474832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48841029e0 msgr2=0x7f488410ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.925+0000 7f4889c00640 1 --2- 172.21.15.18:0/1516474832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48841029e0 0x7f488410ae30 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4870009900 tx=0x7f487002bc30 comp rx=0 tx=0).stop 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.926+0000 7f4889c00640 1 -- 172.21.15.18:0/1516474832 shutdown_connections 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.926+0000 7f4889c00640 1 --2- 172.21.15.18:0/1516474832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488410b4a0 0x7f488410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.926+0000 7f4889c00640 1 --2- 172.21.15.18:0/1516474832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48841029e0 0x7f488410ae30 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.926+0000 7f4889c00640 1 --2- 172.21.15.18:0/1516474832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4884102030 0x7f4884102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.926+0000 7f4889c00640 1 -- 172.21.15.18:0/1516474832 >> 172.21.15.18:0/1516474832 conn(0x7f4884075da0 msgr2=0x7f48840761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.926+0000 7f4889c00640 1 -- 172.21.15.18:0/1516474832 shutdown_connections 2024-04-16T17:24:46.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.926+0000 7f4889c00640 1 -- 172.21.15.18:0/1516474832 wait complete. 2024-04-16T17:24:46.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.927+0000 7f4889c00640 1 Processor -- start 2024-04-16T17:24:46.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.927+0000 7f4889c00640 1 -- start start 2024-04-16T17:24:46.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4889c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4884102030 0x7f48840737c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4889c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488410b4a0 0x7f488406c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4889c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4884073e10 0x7f488406ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4889c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f48841101c0 con 0x7f4884073e10 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4889c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4884110040 con 0x7f488410b4a0 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4889c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4884110340 con 0x7f4884102030 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4882a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4884102030 0x7f48840737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4882a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4884102030 0x7f48840737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39836/0 (socket says 172.21.15.18:39836) 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.928+0000 7f4882a00640 1 -- 172.21.15.18:0/1551257033 learned_addr learned my addr 172.21.15.18:0/1551257033 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.929+0000 7f4882000640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488410b4a0 0x7f488406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.929+0000 7f4883400640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4884073e10 0x7f488406ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.929+0000 7f4882000640 1 -- 172.21.15.18:0/1551257033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4884102030 msgr2=0x7f48840737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.929+0000 7f4882000640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4884102030 0x7f48840737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.929+0000 7f4882000640 1 -- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4884073e10 msgr2=0x7f488406ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.929+0000 7f4882000640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4884073e10 0x7f488406ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.929+0000 7f4882000640 1 -- 172.21.15.18:0/1551257033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48840712e0 con 0x7f488410b4a0 2024-04-16T17:24:46.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.929+0000 7f4883400640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4884073e10 0x7f488406ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:46.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.930+0000 7f4882000640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488410b4a0 0x7f488406c840 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4870036040 tx=0x7f4870004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:46.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.930+0000 7f487b400640 1 -- 172.21.15.18:0/1551257033 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870043020 con 0x7f488410b4a0 2024-04-16T17:24:46.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.930+0000 7f487b400640 1 -- 172.21.15.18:0/1551257033 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48700054e0 con 0x7f488410b4a0 2024-04-16T17:24:46.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.930+0000 7f487b400640 1 -- 172.21.15.18:0/1551257033 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870003b80 con 0x7f488410b4a0 2024-04-16T17:24:46.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.930+0000 7f4889c00640 1 -- 172.21.15.18:0/1551257033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4884071570 con 0x7f488410b4a0 2024-04-16T17:24:46.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.930+0000 7f4889c00640 1 -- 172.21.15.18:0/1551257033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4884071a40 con 0x7f488410b4a0 2024-04-16T17:24:46.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.935+0000 7f487b400640 1 -- 172.21.15.18:0/1551257033 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4870046020 con 0x7f488410b4a0 2024-04-16T17:24:46.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.937+0000 7f487b400640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f48680a3da0 0x7f48680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.937+0000 7f487b400640 1 -- 172.21.15.18:0/1551257033 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f48700f2b30 con 0x7f488410b4a0 2024-04-16T17:24:46.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.938+0000 7f4882a00640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f48680a3da0 0x7f48680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.939+0000 7f4882a00640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f48680a3da0 0x7f48680a6260 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4874005fd0 tx=0x7f487400b4a0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:46.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.940+0000 7f488c883740 1 Processor -- start 2024-04-16T17:24:46.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.940+0000 7f488c883740 1 -- start start 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f488c883740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca4f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f488c883740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca57d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f488c883740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f488c883740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55753c9b4020 con 0x55753ca58250 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f488c883740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55753c95a460 con 0x55753ca4ed50 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f488c883740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55753c9affe0 con 0x55753ca4f670 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f4882a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca4f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f4882a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca4f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40396/0 (socket says 172.21.15.18:40396) 2024-04-16T17:24:46.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f4882a00640 1 -- 172.21.15.18:0/2892264870 learned_addr learned my addr 172.21.15.18:0/2892264870 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f4882000640 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca57d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.941+0000 7f4883400640 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.942+0000 7f4882a00640 1 -- 172.21.15.18:0/2892264870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 msgr2=0x55753ca57d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:46.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.942+0000 7f4882a00640 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca57d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.942+0000 7f4882a00640 1 -- 172.21.15.18:0/2892264870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 msgr2=0x55753ca5a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.942+0000 7f4882a00640 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.942+0000 7f4882a00640 1 -- 172.21.15.18:0/2892264870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55753c962040 con 0x55753ca4ed50 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.942+0000 7f4882000640 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca57d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.942+0000 7f4883400640 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.942+0000 7f4882a00640 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca4f130 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4874005d00 tx=0x7f4874013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d3c2bb25dc1aacc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.943+0000 7f486fe00640 1 -- 172.21.15.18:0/2892264870 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487401e420 con 0x55753ca4ed50 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.943+0000 7f486fe00640 1 -- 172.21.15.18:0/2892264870 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487401eaa0 con 0x55753ca4ed50 2024-04-16T17:24:46.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.943+0000 7f486fe00640 1 -- 172.21.15.18:0/2892264870 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874026aa0 con 0x55753ca4ed50 2024-04-16T17:24:46.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.944+0000 7f488c883740 1 -- 172.21.15.18:0/2892264870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 msgr2=0x55753ca4f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.944+0000 7f488c883740 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca4f130 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4874005d00 tx=0x7f4874013000 comp rx=0 tx=0).stop 2024-04-16T17:24:46.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.945+0000 7f488c883740 1 -- 172.21.15.18:0/2892264870 shutdown_connections 2024-04-16T17:24:46.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.945+0000 7f488c883740 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.945+0000 7f488c883740 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca57d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.945+0000 7f488c883740 1 --2- 172.21.15.18:0/2892264870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca4f130 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.945+0000 7f488c883740 1 -- 172.21.15.18:0/2892264870 >> 172.21.15.18:0/2892264870 conn(0x55753ca49480 msgr2=0x55753ca49860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:46.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.945+0000 7f488c883740 1 -- 172.21.15.18:0/2892264870 shutdown_connections 2024-04-16T17:24:46.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.945+0000 7f488c883740 1 -- 172.21.15.18:0/2892264870 wait complete. 2024-04-16T17:24:46.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.946+0000 7f488c883740 1 -- start start 2024-04-16T17:24:46.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.946+0000 7f488c883740 1 Processor -- start 2024-04-16T17:24:46.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.946+0000 7f488c883740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4ed50 0x55753ca552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f488c883740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4f670 0x55753ca557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f488c883740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f488c883740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55753ca5b040 con 0x55753ca58250 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f488c883740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55753c96d830 con 0x55753ca4f670 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f488c883740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55753ca5b1c0 con 0x55753ca4ed50 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f4882a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4ed50 0x55753ca552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f4882a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4ed50 0x55753ca552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39858/0 (socket says 172.21.15.18:39858) 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f4882a00640 1 -- 172.21.15.18:0/4181804793 learned_addr learned my addr 172.21.15.18:0/4181804793 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f4882a00640 1 -- 172.21.15.18:0/4181804793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4f670 msgr2=0x55753ca557f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f4882a00640 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4f670 0x55753ca557f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f4882a00640 1 -- 172.21.15.18:0/4181804793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 msgr2=0x55753ca55d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f4882a00640 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f4882a00640 1 -- 172.21.15.18:0/4181804793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55753c971920 con 0x55753ca4ed50 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.947+0000 7f488c883740 10 client.0 _ll_register_callbacks cb 0x55753c975150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:46.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.948+0000 7f488c883740 1 -- 172.21.15.18:0/4181804793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f487401eaa0 con 0x55753ca4ed50 2024-04-16T17:24:46.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.948+0000 7f4882a00640 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4ed50 0x55753ca552b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4874026680 tx=0x7f4874004030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82838a1f89b65d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:46.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.948+0000 7f486cc00640 10 client.30586 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:46.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.948+0000 7f486cc00640 1 -- 172.21.15.18:0/4181804793 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487403e060 con 0x55753ca4ed50 2024-04-16T17:24:46.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.948+0000 7f486cc00640 1 -- 172.21.15.18:0/4181804793 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f487401e630 con 0x55753ca4ed50 2024-04-16T17:24:46.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.948+0000 7f486cc00640 1 -- 172.21.15.18:0/4181804793 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f487401e870 con 0x55753ca4ed50 2024-04-16T17:24:46.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.948+0000 7f488c883740 10 client.30586 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:46.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.948+0000 7f488c883740 1 -- 172.21.15.18:0/4181804793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55753c93f7f0 con 0x55753ca4ed50 2024-04-16T17:24:46.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.949+0000 7f486cc00640 1 -- 172.21.15.18:0/4181804793 <== mon.1 v2:172.21.15.43:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f48740308d0 con 0x55753ca4ed50 2024-04-16T17:24:46.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.949+0000 7f488c883740 10 client.30586 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.949+0000 7f488c883740 10 client.30586 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.949+0000 7f488c883740 20 client.30586 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.949+0000 7f488c883740 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55753ca682b0 0x55753ca6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.949+0000 7f488c883740 4 client.30586 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.949+0000 7f488c883740 1 -- 172.21.15.18:0/4181804793 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55753c9c1a40 con 0x55753ca682b0 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.949+0000 7f4882000640 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55753ca682b0 0x55753ca6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.950+0000 7f4882000640 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55753ca682b0 0x55753ca6a6a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=979365ccbdb5bb48 server_cookie=b17b86b283a50a57 in_seq=0 out_seq=0 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.950+0000 7f486cc00640 10 client.30586 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.950+0000 7f486cc00640 1 -- 172.21.15.18:0/4181804793 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55753c9c1a40 con 0x55753ca682b0 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.950+0000 7f486cc00640 10 client.30586 handle_command_reply: tid=0 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 client.30586 shutdown 2024-04-16T17:24:46.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 -- 172.21.15.18:0/4181804793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4ed50 msgr2=0x55753ca552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4ed50 0x55753ca552b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4874026680 tx=0x7f4874004030 comp rx=0 tx=0).stop 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 -- 172.21.15.18:0/4181804793 shutdown_connections 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55753ca682b0 0x55753ca6a6a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca55d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4f670 0x55753ca557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 --2- 172.21.15.18:0/4181804793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4ed50 0x55753ca552b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 -- 172.21.15.18:0/4181804793 >> 172.21.15.18:0/4181804793 conn(0x55753ca49480 msgr2=0x55753ca4b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f486cc00640 0 client.30586 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 -- 172.21.15.18:0/4181804793 shutdown_connections 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 1 -- 172.21.15.18:0/4181804793 wait complete. 2024-04-16T17:24:46.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.951+0000 7f488c883740 20 client.30586 trim_cache size 0 max 16384 2024-04-16T17:24:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.972+0000 7f488c883740 1 Processor -- start 2024-04-16T17:24:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.973+0000 7f488c883740 1 -- start start 2024-04-16T17:24:46.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.973+0000 7f488c883740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca4ed50 0x55753ca63f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.973+0000 7f488c883740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca54b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.973+0000 7f488c883740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca58250 0x55753ca6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.973+0000 7f488c883740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f48841b8480 con 0x55753ca4ed50 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.973+0000 7f488c883740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f48841b5d50 con 0x55753ca58250 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.973+0000 7f488c883740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48841b3bc0 con 0x55753ca4f670 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.973+0000 7f4882000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca54b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca54b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39862/0 (socket says 172.21.15.18:39862) 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882000640 1 -- 172.21.15.18:0/3660779440 learned_addr learned my addr 172.21.15.18:0/3660779440 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4883400640 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca58250 0x55753ca6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882a00640 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca4ed50 0x55753ca63f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882a00640 1 -- 172.21.15.18:0/3660779440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 msgr2=0x55753ca54b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882a00640 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca54b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882a00640 1 -- 172.21.15.18:0/3660779440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca58250 msgr2=0x55753ca6fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882a00640 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca58250 0x55753ca6fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882a00640 1 -- 172.21.15.18:0/3660779440 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55753c6c63f0 con 0x55753ca4ed50 2024-04-16T17:24:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4882000640 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca54b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.974+0000 7f4883400640 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca58250 0x55753ca6fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.975+0000 7f4882a00640 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca4ed50 0x55753ca63f00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f48740039c0 tx=0x7f4874032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cb7ccb21afa664 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:46.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.975+0000 7f486f400640 1 -- 172.21.15.18:0/3660779440 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400bd30 con 0x55753ca4ed50 2024-04-16T17:24:46.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.976+0000 7f486f400640 1 -- 172.21.15.18:0/3660779440 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487401eaa0 con 0x55753ca4ed50 2024-04-16T17:24:46.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.976+0000 7f486f400640 1 -- 172.21.15.18:0/3660779440 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487403a650 con 0x55753ca4ed50 2024-04-16T17:24:46.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 -- 172.21.15.18:0/3660779440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca4ed50 msgr2=0x55753ca63f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca4ed50 0x55753ca63f00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f48740039c0 tx=0x7f4874032000 comp rx=0 tx=0).stop 2024-04-16T17:24:46.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 -- 172.21.15.18:0/3660779440 shutdown_connections 2024-04-16T17:24:46.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca58250 0x55753ca6fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca54b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 --2- 172.21.15.18:0/3660779440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca4ed50 0x55753ca63f00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 -- 172.21.15.18:0/3660779440 >> 172.21.15.18:0/3660779440 conn(0x55753ca49480 msgr2=0x55753ca55b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:46.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 -- 172.21.15.18:0/3660779440 shutdown_connections 2024-04-16T17:24:46.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.977+0000 7f488c883740 1 -- 172.21.15.18:0/3660779440 wait complete. 2024-04-16T17:24:46.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.978+0000 7f488c883740 1 -- start start 2024-04-16T17:24:46.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.978+0000 7f488c883740 1 Processor -- start 2024-04-16T17:24:46.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.979+0000 7f488c883740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca60770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.979+0000 7f488c883740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca5bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.979+0000 7f488c883740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.979+0000 7f488c883740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f48841b4210 con 0x55753ca58250 2024-04-16T17:24:46.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.979+0000 7f488c883740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f48841b5680 con 0x55753ca4ed50 2024-04-16T17:24:46.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.979+0000 7f488c883740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48841b48e0 con 0x55753ca4f670 2024-04-16T17:24:46.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.979+0000 7f488c883740 10 client.0 _ll_register_callbacks cb 0x7f487c00bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:46.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.979+0000 7f4882a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca60770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.980+0000 7f4882000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca5bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.980+0000 7f4882a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca60770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40438/0 (socket says 172.21.15.18:40438) 2024-04-16T17:24:46.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.980+0000 7f4882a00640 1 -- 172.21.15.18:0/432311516 learned_addr learned my addr 172.21.15.18:0/432311516 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:46.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.980+0000 7f4883400640 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.980+0000 7f4882a00640 1 -- 172.21.15.18:0/432311516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 msgr2=0x55753ca5bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:46.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.981+0000 7f4882a00640 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca5bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.981+0000 7f4882a00640 1 -- 172.21.15.18:0/432311516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 msgr2=0x55753ca5c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:46.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.981+0000 7f4882a00640 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.981+0000 7f4882a00640 1 -- 172.21.15.18:0/432311516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55753c7907d0 con 0x55753ca4ed50 2024-04-16T17:24:46.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.981+0000 7f4883400640 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:46.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.981+0000 7f4882000640 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca5bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:46.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.982+0000 7f4882a00640 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca60770 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4874005860 tx=0x7f4874011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ccb91c6fe7364e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:46.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.982+0000 7f486cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:46.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.982+0000 7f486cc00640 1 -- 172.21.15.18:0/432311516 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400bd30 con 0x55753ca4ed50 2024-04-16T17:24:46.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.982+0000 7f488c883740 1 -- 172.21.15.18:0/432311516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f487401eaa0 con 0x55753ca4ed50 2024-04-16T17:24:46.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.982+0000 7f486cc00640 1 -- 172.21.15.18:0/432311516 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f48740266b0 con 0x55753ca4ed50 2024-04-16T17:24:46.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.982+0000 7f486cc00640 1 -- 172.21.15.18:0/432311516 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4874059050 con 0x55753ca4ed50 2024-04-16T17:24:46.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f488c883740 10 client.30731 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:46.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f488c883740 1 -- 172.21.15.18:0/432311516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55753cace3d0 con 0x55753ca4ed50 2024-04-16T17:24:46.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f486cc00640 1 -- 172.21.15.18:0/432311516 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487401eaa0 con 0x55753ca4ed50 2024-04-16T17:24:46.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f486cc00640 1 -- 172.21.15.18:0/432311516 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f487404cc60 con 0x55753ca4ed50 2024-04-16T17:24:46.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f488c883740 10 client.30731 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:46.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f488c883740 10 client.30731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:24:46.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f488c883740 20 client.30731 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:46.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f488c883740 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55753cacea30 0x55753cad0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:46.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f488c883740 4 client.30731 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:24:46.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.983+0000 7f488c883740 1 -- 172.21.15.18:0/432311516 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55753c9bb8b0 con 0x55753cacea30 2024-04-16T17:24:46.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.984+0000 7f4882000640 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55753cacea30 0x55753cad0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:46.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.985+0000 7f486cc00640 10 client.30731 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:24:46.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.985+0000 7f4882000640 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55753cacea30 0x55753cad0e20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52256f4266c28b95 server_cookie=f2c8a1fa58e5cd15 in_seq=0 out_seq=0 2024-04-16T17:24:46.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.985+0000 7f486cc00640 1 -- 172.21.15.18:0/432311516 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55753c9bb8b0 con 0x55753cacea30 2024-04-16T17:24:46.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.985+0000 7f486cc00640 10 client.30731 handle_command_reply: tid=0 2024-04-16T17:24:46.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.985+0000 7f488c883740 1 client.30731 shutdown 2024-04-16T17:24:46.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 -- 172.21.15.18:0/432311516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 msgr2=0x55753ca60770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca60770 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4874005860 tx=0x7f4874011d90 comp rx=0 tx=0).stop 2024-04-16T17:24:46.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 -- 172.21.15.18:0/432311516 shutdown_connections 2024-04-16T17:24:46.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x55753cacea30 0x55753cad0e20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55753ca58250 0x55753ca5c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55753ca4f670 0x55753ca5bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 --2- 172.21.15.18:0/432311516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55753ca4ed50 0x55753ca60770 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 -- 172.21.15.18:0/432311516 >> 172.21.15.18:0/432311516 conn(0x55753ca49480 msgr2=0x55753ca55250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:46.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f486cc00640 0 client.30731 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:24:46.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.986+0000 7f488c883740 1 -- 172.21.15.18:0/432311516 shutdown_connections 2024-04-16T17:24:46.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.987+0000 7f488c883740 1 -- 172.21.15.18:0/432311516 wait complete. 2024-04-16T17:24:46.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.987+0000 7f488c883740 20 client.30731 trim_cache size 0 max 16384 2024-04-16T17:24:46.988 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:24:46.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 -- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f48680a3da0 msgr2=0x7f48680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f48680a3da0 0x7f48680a6260 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4874005fd0 tx=0x7f487400b4a0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 -- 172.21.15.18:0/1551257033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488410b4a0 msgr2=0x7f488406c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488410b4a0 0x7f488406c840 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4870036040 tx=0x7f4870004360 comp rx=0 tx=0).stop 2024-04-16T17:24:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 -- 172.21.15.18:0/1551257033 shutdown_connections 2024-04-16T17:24:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f48680a3da0 0x7f48680a6260 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4884073e10 0x7f488406ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488410b4a0 0x7f488406c840 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 --2- 172.21.15.18:0/1551257033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4884102030 0x7f48840737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:46.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 -- 172.21.15.18:0/1551257033 >> 172.21.15.18:0/1551257033 conn(0x7f4884075da0 msgr2=0x7f48840fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:46.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.989+0000 7f4889c00640 1 -- 172.21.15.18:0/1551257033 shutdown_connections 2024-04-16T17:24:46.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:46.990+0000 7f4889c00640 1 -- 172.21.15.18:0/1551257033 wait complete. 2024-04-16T17:24:47.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:24:47.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:46 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/358271309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:24:47.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:46 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/358271309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:24:47.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.315+0000 7f8b97800640 1 -- 172.21.15.18:0/3560706876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9010c600 msgr2=0x7f8b9010ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.315+0000 7f8b97800640 1 --2- 172.21.15.18:0/3560706876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9010c600 0x7f8b9010ea30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009ca0 tx=0x7f8b8802c120 comp rx=0 tx=0).stop 2024-04-16T17:24:47.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.316+0000 7f8b97800640 1 -- 172.21.15.18:0/3560706876 shutdown_connections 2024-04-16T17:24:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.316+0000 7f8b97800640 1 --2- 172.21.15.18:0/3560706876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9010c600 0x7f8b9010ea30 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.317+0000 7f8b97800640 1 --2- 172.21.15.18:0/3560706876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b90100fb0 0x7f8b9010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.317+0000 7f8b97800640 1 --2- 172.21.15.18:0/3560706876 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b90100690 0x7f8b90100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.317+0000 7f8b97800640 1 -- 172.21.15.18:0/3560706876 >> 172.21.15.18:0/3560706876 conn(0x7f8b90075da0 msgr2=0x7f8b900761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.318+0000 7f8b97800640 1 -- 172.21.15.18:0/3560706876 shutdown_connections 2024-04-16T17:24:47.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.318+0000 7f8b97800640 1 -- 172.21.15.18:0/3560706876 wait complete. 2024-04-16T17:24:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.320+0000 7f8b97800640 1 Processor -- start 2024-04-16T17:24:47.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.320+0000 7f8b97800640 1 -- start start 2024-04-16T17:24:47.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.321+0000 7f8b97800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b90100690 0x7f8b900737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.321+0000 7f8b97800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b90100fb0 0x7f8b9006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.321+0000 7f8b97800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9010c600 0x7f8b9006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.321+0000 7f8b97800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901123f0 con 0x7f8b90100690 2024-04-16T17:24:47.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.321+0000 7f8b97800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90112270 con 0x7f8b90100fb0 2024-04-16T17:24:47.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.321+0000 7f8b97800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90112570 con 0x7f8b9010c600 2024-04-16T17:24:47.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.321+0000 7f8b94e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b90100690 0x7f8b900737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b94e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b90100690 0x7f8b900737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52646/0 (socket says 172.21.15.18:52646) 2024-04-16T17:24:47.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b94e00640 1 -- 172.21.15.18:0/2083900268 learned_addr learned my addr 172.21.15.18:0/2083900268 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b94e00640 1 -- 172.21.15.18:0/2083900268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9010c600 msgr2=0x7f8b9006cdb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:24:47.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b94e00640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9010c600 0x7f8b9006cdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b90100fb0 0x7f8b9006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b94e00640 1 -- 172.21.15.18:0/2083900268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b90100fb0 msgr2=0x7f8b9006c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b94e00640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b90100fb0 0x7f8b9006c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b94e00640 1 -- 172.21.15.18:0/2083900268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9006d2f0 con 0x7f8b90100690 2024-04-16T17:24:47.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.322+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b90100fb0 0x7f8b9006c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:47.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.324+0000 7f8b94e00640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b90100690 0x7f8b900737f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00d950 tx=0x7f8b7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.326+0000 7f8b8d600640 1 -- 172.21.15.18:0/2083900268 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c017030 con 0x7f8b90100690 2024-04-16T17:24:47.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.326+0000 7f8b8d600640 1 -- 172.21.15.18:0/2083900268 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b7c004020 con 0x7f8b90100690 2024-04-16T17:24:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.326+0000 7f8b8d600640 1 -- 172.21.15.18:0/2083900268 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0047a0 con 0x7f8b90100690 2024-04-16T17:24:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.327+0000 7f8b97800640 1 -- 172.21.15.18:0/2083900268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9006d5e0 con 0x7f8b90100690 2024-04-16T17:24:47.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.327+0000 7f8b97800640 1 -- 172.21.15.18:0/2083900268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b900714f0 con 0x7f8b90100690 2024-04-16T17:24:47.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.332+0000 7f8b9a58f740 1 Processor -- start 2024-04-16T17:24:47.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.332+0000 7f8b9a58f740 1 -- start start 2024-04-16T17:24:47.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.332+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b6cd50 0x562ea0b6d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.332+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 0x562ea0b75d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.332+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b76250 0x562ea0b78640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.333+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562ea0ad2020 con 0x562ea0b6d670 2024-04-16T17:24:47.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.333+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562ea0a78460 con 0x562ea0b6cd50 2024-04-16T17:24:47.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.333+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ea0acdfe0 con 0x562ea0b76250 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.333+0000 7f8b95800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b6cd50 0x562ea0b6d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.333+0000 7f8b95800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b6cd50 0x562ea0b6d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40456/0 (socket says 172.21.15.18:40456) 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.333+0000 7f8b95800640 1 -- 172.21.15.18:0/2929813508 learned_addr learned my addr 172.21.15.18:0/2929813508 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.334+0000 7f8b95800640 1 -- 172.21.15.18:0/2929813508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b76250 msgr2=0x562ea0b78640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.334+0000 7f8b95800640 1 --2- 172.21.15.18:0/2929813508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b76250 0x562ea0b78640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.334+0000 7f8b95800640 1 -- 172.21.15.18:0/2929813508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 msgr2=0x562ea0b75d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.334+0000 7f8b95800640 1 --2- 172.21.15.18:0/2929813508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 0x562ea0b75d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.334+0000 7f8b95800640 1 -- 172.21.15.18:0/2929813508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea0a80040 con 0x562ea0b6cd50 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.334+0000 7f8b95800640 1 --2- 172.21.15.18:0/2929813508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b6cd50 0x562ea0b6d130 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005e30 tx=0x7f8b88005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad568c2364d95f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.335+0000 7f8b82000640 1 -- 172.21.15.18:0/2929813508 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88041020 con 0x562ea0b6cd50 2024-04-16T17:24:47.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.335+0000 7f8b82000640 1 -- 172.21.15.18:0/2929813508 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b88032ce0 con 0x562ea0b6cd50 2024-04-16T17:24:47.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.335+0000 7f8b82000640 1 -- 172.21.15.18:0/2929813508 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88039040 con 0x562ea0b6cd50 2024-04-16T17:24:47.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.336+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2929813508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b6cd50 msgr2=0x562ea0b6d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.336+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/2929813508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b6cd50 0x562ea0b6d130 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005e30 tx=0x7f8b88005fd0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.337+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2929813508 shutdown_connections 2024-04-16T17:24:47.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.337+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/2929813508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b76250 0x562ea0b78640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.337+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/2929813508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 0x562ea0b75d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.337+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/2929813508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b6cd50 0x562ea0b6d130 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.337+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2929813508 >> 172.21.15.18:0/2929813508 conn(0x562ea0b67480 msgr2=0x562ea0b67860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.337+0000 7f8b8d600640 1 -- 172.21.15.18:0/2083900268 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b7c0041d0 con 0x7f8b90100690 2024-04-16T17:24:47.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.339+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2929813508 shutdown_connections 2024-04-16T17:24:47.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.340+0000 7f8b8d600640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8b780a3d30 0x7f8b780a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.340+0000 7f8b8d600640 1 -- 172.21.15.18:0/2083900268 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f8b7c0d1e20 con 0x7f8b90100690 2024-04-16T17:24:47.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.341+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2929813508 wait complete. 2024-04-16T17:24:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.342+0000 7f8b9a58f740 1 -- start start 2024-04-16T17:24:47.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.342+0000 7f8b9a58f740 1 Processor -- start 2024-04-16T17:24:47.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.343+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8b780a3d30 0x7f8b780a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.346+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6cd50 0x562ea0b731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.346+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.346+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b73c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.347+0000 7f8b8fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.347+0000 7f8b8fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b73720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39894/0 (socket says 172.21.15.18:39894) 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.348+0000 7f8b8fe00640 1 -- 172.21.15.18:0/895252860 learned_addr learned my addr 172.21.15.18:0/895252860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.348+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562ea0b78fe0 con 0x562ea0b6cd50 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562ea0a8b830 con 0x562ea0b76250 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ea0b79160 con 0x562ea0b6d670 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b9a58f740 10 client.0 _ll_register_callbacks cb 0x562ea0a93150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:47.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8b780a3d30 0x7f8b780a61f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x562ea0b68610 tx=0x7f8b84008040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b8fe00640 1 -- 172.21.15.18:0/895252860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 msgr2=0x562ea0b73c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b73c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b8fe00640 1 -- 172.21.15.18:0/895252860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6cd50 msgr2=0x562ea0b731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6cd50 0x562ea0b731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.349+0000 7f8b8fe00640 1 -- 172.21.15.18:0/895252860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea0a8f920 con 0x562ea0b6d670 2024-04-16T17:24:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.352+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b73720 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400cd00 tx=0x7f8b840047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d7692f4b1c8b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.352+0000 7f8b76a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:47.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.352+0000 7f8b76a00640 1 -- 172.21.15.18:0/895252860 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84021020 con 0x562ea0b6d670 2024-04-16T17:24:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.352+0000 7f8b76a00640 1 -- 172.21.15.18:0/895252860 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f8b8401d3f0 con 0x562ea0b6d670 2024-04-16T17:24:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.353+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b88032ce0 con 0x562ea0b6d670 2024-04-16T17:24:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.354+0000 7f8b76a00640 1 -- 172.21.15.18:0/895252860 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8b84019040 con 0x562ea0b6d670 2024-04-16T17:24:47.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.354+0000 7f8b9a58f740 10 client.30598 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:47.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.354+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562ea0a5d7f0 con 0x562ea0b6d670 2024-04-16T17:24:47.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.355+0000 7f8b76a00640 1 -- 172.21.15.18:0/895252860 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8401dad0 con 0x562ea0b6d670 2024-04-16T17:24:47.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.355+0000 7f8b76a00640 1 -- 172.21.15.18:0/895252860 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f8b8402e020 con 0x562ea0b6d670 2024-04-16T17:24:47.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.355+0000 7f8b9a58f740 10 client.30598 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:47.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.355+0000 7f8b9a58f740 10 client.30598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:24:47.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.355+0000 7f8b9a58f740 20 client.30598 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:47.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.355+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x562ea0b86640 0x562ea0b88b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.357+0000 7f8b9a58f740 4 client.30598 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:47.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.357+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8b88032ce0 con 0x562ea0b86640 2024-04-16T17:24:47.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.358+0000 7f8b94e00640 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x562ea0b86640 0x562ea0b88b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.363+0000 7f8b76a00640 10 client.30598 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:24:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.363+0000 7f8b94e00640 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x562ea0b86640 0x562ea0b88b00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=961fee68bc3d0ae3 server_cookie=ba30e56ad78ade1b in_seq=0 out_seq=0 2024-04-16T17:24:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.365+0000 7f8b76a00640 1 -- 172.21.15.18:0/895252860 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f8b88032ce0 con 0x562ea0b86640 2024-04-16T17:24:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.365+0000 7f8b76a00640 10 client.30598 handle_command_reply: tid=0 2024-04-16T17:24:47.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.366+0000 7f8b9a58f740 1 client.30598 shutdown 2024-04-16T17:24:47.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.367+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 msgr2=0x562ea0b73720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.367+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b73720 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400cd00 tx=0x7f8b840047d0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.368+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 shutdown_connections 2024-04-16T17:24:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.368+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x562ea0b86640 0x562ea0b88b00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.368+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b73c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.368+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b73720 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.368+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/895252860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6cd50 0x562ea0b731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.368+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 >> 172.21.15.18:0/895252860 conn(0x562ea0b67480 msgr2=0x562ea0b691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.368+0000 7f8b76a00640 0 client.30598 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:24:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.369+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 shutdown_connections 2024-04-16T17:24:47.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.369+0000 7f8b9a58f740 1 -- 172.21.15.18:0/895252860 wait complete. 2024-04-16T17:24:47.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.370+0000 7f8b9a58f740 20 client.30598 trim_cache size 0 max 16384 2024-04-16T17:24:47.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.409+0000 7f8b9a58f740 1 Processor -- start 2024-04-16T17:24:47.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.409+0000 7f8b9a58f740 1 -- start start 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.409+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6cd50 0x562ea0b75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.409+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b81f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.409+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b68fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.409+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901c0e60 con 0x562ea0b6cd50 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.409+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901be730 con 0x562ea0b76250 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.409+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901bc5a0 con 0x562ea0b6d670 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.410+0000 7f8b95800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b68fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.410+0000 7f8b95800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b68fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40472/0 (socket says 172.21.15.18:40472) 2024-04-16T17:24:47.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.410+0000 7f8b95800640 1 -- 172.21.15.18:0/2464990103 learned_addr learned my addr 172.21.15.18:0/2464990103 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.411+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/2464990103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b81f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.411+0000 7f8b95800640 1 -- 172.21.15.18:0/2464990103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 msgr2=0x562ea0b81f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.411+0000 7f8b95800640 1 --2- 172.21.15.18:0/2464990103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b81f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.411+0000 7f8b95800640 1 -- 172.21.15.18:0/2464990103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6cd50 msgr2=0x562ea0b75910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.411+0000 7f8b95800640 1 --2- 172.21.15.18:0/2464990103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6cd50 0x562ea0b75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.411+0000 7f8b95800640 1 -- 172.21.15.18:0/2464990103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea07e43f0 con 0x562ea0b76250 2024-04-16T17:24:47.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.412+0000 7f8b95800640 1 --2- 172.21.15.18:0/2464990103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b68fc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8b88000f80 tx=0x7f8b88032580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d74dec531c889fe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.413+0000 7f8b81600640 1 -- 172.21.15.18:0/2464990103 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8804b070 con 0x562ea0b76250 2024-04-16T17:24:47.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.414+0000 7f8b81600640 1 -- 172.21.15.18:0/2464990103 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b88003970 con 0x562ea0b76250 2024-04-16T17:24:47.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.416+0000 7f8b81600640 1 -- 172.21.15.18:0/2464990103 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8804b070 con 0x562ea0b76250 2024-04-16T17:24:47.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.416+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2464990103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 msgr2=0x562ea0b68fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.416+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/2464990103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b68fc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8b88000f80 tx=0x7f8b88032580 comp rx=0 tx=0).stop 2024-04-16T17:24:47.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.416+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2464990103 shutdown_connections 2024-04-16T17:24:47.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.416+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/2464990103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b68fc0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.416+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/2464990103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6d670 0x562ea0b81f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.416+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/2464990103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6cd50 0x562ea0b75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.416+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2464990103 >> 172.21.15.18:0/2464990103 conn(0x562ea0b67480 msgr2=0x562ea0b688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.417+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2464990103 shutdown_connections 2024-04-16T17:24:47.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.417+0000 7f8b9a58f740 1 -- 172.21.15.18:0/2464990103 wait complete. 2024-04-16T17:24:47.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.418+0000 7f8b9a58f740 1 -- start start 2024-04-16T17:24:47.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.418+0000 7f8b9a58f740 1 Processor -- start 2024-04-16T17:24:47.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.419+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6cd50 0x562ea0b7e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.419+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 0x562ea0b6fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.419+0000 7f8b9a58f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.419+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901bcbf0 con 0x562ea0b6d670 2024-04-16T17:24:47.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.419+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901be060 con 0x562ea0b76250 2024-04-16T17:24:47.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.419+0000 7f8b9a58f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901bd2c0 con 0x562ea0b6cd50 2024-04-16T17:24:47.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.419+0000 7f8b9a58f740 10 client.0 _ll_register_callbacks cb 0x7f8b8400ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b94e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6cd50 0x562ea0b7e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b94e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6cd50 0x562ea0b7e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39916/0 (socket says 172.21.15.18:39916) 2024-04-16T17:24:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b94e00640 1 -- 172.21.15.18:0/1542037520 learned_addr learned my addr 172.21.15.18:0/1542037520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b95800640 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 0x562ea0b6fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b94e00640 1 -- 172.21.15.18:0/1542037520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 msgr2=0x562ea0b701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b94e00640 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b94e00640 1 -- 172.21.15.18:0/1542037520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 msgr2=0x562ea0b6fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b94e00640 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 0x562ea0b6fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.420+0000 7f8b94e00640 1 -- 172.21.15.18:0/1542037520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea08ae7d0 con 0x562ea0b6cd50 2024-04-16T17:24:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.421+0000 7f8b94e00640 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6cd50 0x562ea0b7e8e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c09d8e0 tx=0x7f8b7c099d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b308f52bd60f278e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.421+0000 7f8b76a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:24:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.421+0000 7f8b76a00640 1 -- 172.21.15.18:0/1542037520 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0a65c0 con 0x562ea0b6cd50 2024-04-16T17:24:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.421+0000 7f8b76a00640 1 -- 172.21.15.18:0/1542037520 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f8b7c0cf380 con 0x562ea0b6cd50 2024-04-16T17:24:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.422+0000 7f8b9a58f740 1 -- 172.21.15.18:0/1542037520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b88003970 con 0x562ea0b6cd50 2024-04-16T17:24:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.422+0000 7f8b76a00640 1 -- 172.21.15.18:0/1542037520 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8b7c0cf580 con 0x562ea0b6cd50 2024-04-16T17:24:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.422+0000 7f8b9a58f740 10 client.30604 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.422+0000 7f8b9a58f740 1 -- 172.21.15.18:0/1542037520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562ea0becb10 con 0x562ea0b6cd50 2024-04-16T17:24:47.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.423+0000 7f8b76a00640 1 -- 172.21.15.18:0/1542037520 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b88003970 con 0x562ea0b6cd50 2024-04-16T17:24:47.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.423+0000 7f8b76a00640 1 -- 172.21.15.18:0/1542037520 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f8b7c099770 con 0x562ea0b6cd50 2024-04-16T17:24:47.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.423+0000 7f8b9a58f740 10 client.30604 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:47.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.423+0000 7f8b9a58f740 10 client.30604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:24:47.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.423+0000 7f8b9a58f740 20 client.30604 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:47.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.423+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x562ea0bed280 0x562ea0bef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.423+0000 7f8b9a58f740 4 client.30604 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:24:47.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.423+0000 7f8b9a58f740 1 -- 172.21.15.18:0/1542037520 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ea0adfa40 con 0x562ea0bed280 2024-04-16T17:24:47.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.426+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x562ea0bed280 0x562ea0bef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.428+0000 7f8b8fe00640 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x562ea0bed280 0x562ea0bef670 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b161aab640997f65 server_cookie=dca1e879a0c71412 in_seq=0 out_seq=0 2024-04-16T17:24:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.428+0000 7f8b76a00640 10 client.30604 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:24:47.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.429+0000 7f8b76a00640 1 -- 172.21.15.18:0/1542037520 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ea0adfa40 con 0x562ea0bed280 2024-04-16T17:24:47.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.429+0000 7f8b76a00640 10 client.30604 handle_command_reply: tid=0 2024-04-16T17:24:47.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.429+0000 7f8b9a58f740 1 client.30604 shutdown 2024-04-16T17:24:47.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.429+0000 7f8b9a58f740 1 -- 172.21.15.18:0/1542037520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6cd50 msgr2=0x562ea0b7e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.429+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6cd50 0x562ea0b7e8e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c09d8e0 tx=0x7f8b7c099d10 comp rx=0 tx=0).stop 2024-04-16T17:24:47.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b9a58f740 1 -- 172.21.15.18:0/1542037520 shutdown_connections 2024-04-16T17:24:47.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x562ea0bed280 0x562ea0bef670 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ea0b76250 0x562ea0b701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562ea0b6d670 0x562ea0b6fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b9a58f740 1 --2- 172.21.15.18:0/1542037520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ea0b6cd50 0x562ea0b7e8e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b76a00640 0 client.30604 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:24:47.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b9a58f740 1 -- 172.21.15.18:0/1542037520 >> 172.21.15.18:0/1542037520 conn(0x562ea0b67480 msgr2=0x562ea0b73a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b9a58f740 1 -- 172.21.15.18:0/1542037520 shutdown_connections 2024-04-16T17:24:47.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.430+0000 7f8b9a58f740 1 -- 172.21.15.18:0/1542037520 wait complete. 2024-04-16T17:24:47.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.431+0000 7f8b9a58f740 20 client.30604 trim_cache size 0 max 16384 2024-04-16T17:24:47.432 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.432+0000 7f8b97800640 1 -- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8b780a3d30 msgr2=0x7f8b780a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.432+0000 7f8b97800640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8b780a3d30 0x7f8b780a61f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x562ea0b68610 tx=0x7f8b84008040 comp rx=0 tx=0).stop 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 -- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b90100690 msgr2=0x7f8b900737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b90100690 0x7f8b900737f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00d950 tx=0x7f8b7c00de10 comp rx=0 tx=0).stop 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 -- 172.21.15.18:0/2083900268 shutdown_connections 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f8b780a3d30 0x7f8b780a61f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9010c600 0x7f8b9006cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b90100fb0 0x7f8b9006c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 --2- 172.21.15.18:0/2083900268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b90100690 0x7f8b900737f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 -- 172.21.15.18:0/2083900268 >> 172.21.15.18:0/2083900268 conn(0x7f8b90075da0 msgr2=0x7f8b900fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 -- 172.21.15.18:0/2083900268 shutdown_connections 2024-04-16T17:24:47.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.433+0000 7f8b97800640 1 -- 172.21.15.18:0/2083900268 wait complete. 2024-04-16T17:24:47.464 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:24:47.599 INFO:tasks.workunit.client.0.smithi018.stdout: 10 94471 11.26 MB/sec execute 299 sec latency 184.166 ms 2024-04-16T17:24:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.775+0000 7f5877000640 1 -- 172.21.15.18:0/1252880205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f587010e860 msgr2=0x7f5870110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.775+0000 7f5877000640 1 --2- 172.21.15.18:0/1252880205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f587010e860 0x7f5870110c70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5868009ca0 tx=0x7f586802c190 comp rx=0 tx=0).stop 2024-04-16T17:24:47.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.775+0000 7f5877000640 1 -- 172.21.15.18:0/1252880205 shutdown_connections 2024-04-16T17:24:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.775+0000 7f5877000640 1 --2- 172.21.15.18:0/1252880205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f587010e860 0x7f5870110c70 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.775+0000 7f5877000640 1 --2- 172.21.15.18:0/1252880205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5870074cf0 0x7f587010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.775+0000 7f5877000640 1 --2- 172.21.15.18:0/1252880205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58700743d0 0x7f58700747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.775+0000 7f5877000640 1 -- 172.21.15.18:0/1252880205 >> 172.21.15.18:0/1252880205 conn(0x7f58700fdab0 msgr2=0x7f58700ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.775+0000 7f5877000640 1 -- 172.21.15.18:0/1252880205 shutdown_connections 2024-04-16T17:24:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.777+0000 7f5877000640 1 -- 172.21.15.18:0/1252880205 wait complete. 2024-04-16T17:24:47.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.778+0000 7f5877000640 1 Processor -- start 2024-04-16T17:24:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.778+0000 7f5877000640 1 -- start start 2024-04-16T17:24:47.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.779+0000 7f5877000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58700743d0 0x7f5870101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.779+0000 7f5877000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5870074cf0 0x7f58701024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.779+0000 7f5877000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f587010e860 0x7f5870102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.779+0000 7f5877000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5870114600 con 0x7f587010e860 2024-04-16T17:24:47.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.779+0000 7f5877000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5870114480 con 0x7f58700743d0 2024-04-16T17:24:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.779+0000 7f5877000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5870114780 con 0x7f5870074cf0 2024-04-16T17:24:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.779+0000 7f586f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5870074cf0 0x7f58701024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.780+0000 7f586f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5870074cf0 0x7f58701024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39932/0 (socket says 172.21.15.18:39932) 2024-04-16T17:24:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.780+0000 7f586fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58700743d0 0x7f5870101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.780+0000 7f586f400640 1 -- 172.21.15.18:0/3086304122 learned_addr learned my addr 172.21.15.18:0/3086304122 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.780+0000 7f586f400640 1 -- 172.21.15.18:0/3086304122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58700743d0 msgr2=0x7f5870101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.780+0000 7f586f400640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58700743d0 0x7f5870101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.780+0000 7f586f400640 1 -- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f587010e860 msgr2=0x7f5870102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.780+0000 7f586f400640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f587010e860 0x7f5870102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.780+0000 7f586f400640 1 -- 172.21.15.18:0/3086304122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58701ab250 con 0x7f5870074cf0 2024-04-16T17:24:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.781+0000 7f586f400640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5870074cf0 0x7f58701024c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f585c00b4a0 tx=0x7f585c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.781+0000 7f586cc00640 1 -- 172.21.15.18:0/3086304122 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c014070 con 0x7f5870074cf0 2024-04-16T17:24:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.781+0000 7f586cc00640 1 -- 172.21.15.18:0/3086304122 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c010af0 con 0x7f5870074cf0 2024-04-16T17:24:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.781+0000 7f586cc00640 1 -- 172.21.15.18:0/3086304122 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c017d00 con 0x7f5870074cf0 2024-04-16T17:24:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.781+0000 7f5877000640 1 -- 172.21.15.18:0/3086304122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58701ab540 con 0x7f5870074cf0 2024-04-16T17:24:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.782+0000 7f5877000640 1 -- 172.21.15.18:0/3086304122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701aba40 con 0x7f5870074cf0 2024-04-16T17:24:47.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.785+0000 7f5879d72740 1 Processor -- start 2024-04-16T17:24:47.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.786+0000 7f5879d72740 1 -- start start 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.786+0000 7f5879d72740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff10e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.786+0000 7f5879d72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff116d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.786+0000 7f5879d72740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff117250 0x560bff119640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.786+0000 7f5879d72740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560bff073020 con 0x560bff117250 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.786+0000 7f5879d72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560bff019460 con 0x560bff10e670 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.786+0000 7f5879d72740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560bff06efe0 con 0x560bff10dd50 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.787+0000 7f5875000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff10e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.787+0000 7f5875000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff10e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39942/0 (socket says 172.21.15.18:39942) 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.787+0000 7f5875000640 1 -- 172.21.15.18:0/3641126706 learned_addr learned my addr 172.21.15.18:0/3641126706 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.787+0000 7f5875000640 1 -- 172.21.15.18:0/3641126706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 msgr2=0x560bff116d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.787+0000 7f5875000640 1 --2- 172.21.15.18:0/3641126706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff116d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.787+0000 7f5875000640 1 -- 172.21.15.18:0/3641126706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff117250 msgr2=0x560bff119640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.787+0000 7f5875000640 1 --2- 172.21.15.18:0/3641126706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff117250 0x560bff119640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.787+0000 7f5875000640 1 -- 172.21.15.18:0/3641126706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bff021040 con 0x560bff10dd50 2024-04-16T17:24:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.789+0000 7f5875000640 1 --2- 172.21.15.18:0/3641126706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff10e130 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f586802c6d0 tx=0x7f5868009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec87a0aa466f33e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.789+0000 7f586cc00640 1 -- 172.21.15.18:0/3086304122 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f585c017070 con 0x7f5870074cf0 2024-04-16T17:24:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.789+0000 7f5865600640 1 -- 172.21.15.18:0/3641126706 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868052020 con 0x560bff10dd50 2024-04-16T17:24:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.789+0000 7f5865600640 1 -- 172.21.15.18:0/3641126706 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5868003e50 con 0x560bff10dd50 2024-04-16T17:24:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.789+0000 7f5865600640 1 -- 172.21.15.18:0/3641126706 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868039040 con 0x560bff10dd50 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.790+0000 7f5879d72740 1 -- 172.21.15.18:0/3641126706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 msgr2=0x560bff10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.790+0000 7f5879d72740 1 --2- 172.21.15.18:0/3641126706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff10e130 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f586802c6d0 tx=0x7f5868009800 comp rx=0 tx=0).stop 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.790+0000 7f5879d72740 1 -- 172.21.15.18:0/3641126706 shutdown_connections 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.790+0000 7f5879d72740 1 --2- 172.21.15.18:0/3641126706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff117250 0x560bff119640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.790+0000 7f5879d72740 1 --2- 172.21.15.18:0/3641126706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff116d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.790+0000 7f5879d72740 1 --2- 172.21.15.18:0/3641126706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff10e130 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.790+0000 7f5879d72740 1 -- 172.21.15.18:0/3641126706 >> 172.21.15.18:0/3641126706 conn(0x560bff108480 msgr2=0x560bff108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.791+0000 7f5879d72740 1 -- 172.21.15.18:0/3641126706 shutdown_connections 2024-04-16T17:24:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.791+0000 7f5879d72740 1 -- 172.21.15.18:0/3641126706 wait complete. 2024-04-16T17:24:47.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.791+0000 7f5879d72740 1 -- start start 2024-04-16T17:24:47.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.792+0000 7f5879d72740 1 Processor -- start 2024-04-16T17:24:47.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.793+0000 7f5879d72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10dd50 0x560bff114390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.793+0000 7f5879d72740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 0x560bff1148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.793+0000 7f5879d72740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff117250 0x560bff114e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.793+0000 7f5879d72740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560bff119fe0 con 0x560bff10e670 2024-04-16T17:24:47.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.793+0000 7f5879d72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560bff02c830 con 0x560bff10dd50 2024-04-16T17:24:47.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.793+0000 7f5879d72740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560bff11a160 con 0x560bff117250 2024-04-16T17:24:47.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.793+0000 7f5879d72740 10 client.0 _ll_register_callbacks cb 0x560bff034150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:47.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f5875000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10dd50 0x560bff114390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f586f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff117250 0x560bff114e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f5875000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10dd50 0x560bff114390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40526/0 (socket says 172.21.15.18:40526) 2024-04-16T17:24:47.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f5875000640 1 -- 172.21.15.18:0/2853703794 learned_addr learned my addr 172.21.15.18:0/2853703794 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f5875000640 1 -- 172.21.15.18:0/2853703794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff117250 msgr2=0x560bff114e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f5875000640 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff117250 0x560bff114e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f5875000640 1 -- 172.21.15.18:0/2853703794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 msgr2=0x560bff1148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f5875000640 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 0x560bff1148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.794+0000 7f5875000640 1 -- 172.21.15.18:0/2853703794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bff030920 con 0x560bff10dd50 2024-04-16T17:24:47.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.795+0000 7f5875000640 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10dd50 0x560bff114390 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5868009430 tx=0x7f5868003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2b251df6774d430 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.795+0000 7f585a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:47.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.795+0000 7f586cc00640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58540a3d30 0x7f58540a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.795+0000 7f585a000640 1 -- 172.21.15.18:0/2853703794 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868052030 con 0x560bff10dd50 2024-04-16T17:24:47.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.795+0000 7f585a000640 1 -- 172.21.15.18:0/2853703794 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f5868032670 con 0x560bff10dd50 2024-04-16T17:24:47.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.795+0000 7f586cc00640 1 -- 172.21.15.18:0/3086304122 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f585c0cfe20 con 0x7f5870074cf0 2024-04-16T17:24:47.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.795+0000 7f5879d72740 1 -- 172.21.15.18:0/2853703794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5868003e50 con 0x560bff10dd50 2024-04-16T17:24:47.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.796+0000 7f585a000640 1 -- 172.21.15.18:0/2853703794 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f586805b070 con 0x560bff10dd50 2024-04-16T17:24:47.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.796+0000 7f5879d72740 10 client.30749 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:47.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.796+0000 7f5879d72740 1 -- 172.21.15.18:0/2853703794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560bfeffe7f0 con 0x560bff10dd50 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.796+0000 7f586fe00640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58540a3d30 0x7f58540a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.796+0000 7f585a000640 1 -- 172.21.15.18:0/2853703794 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5868003e50 con 0x560bff10dd50 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.797+0000 7f585a000640 1 -- 172.21.15.18:0/2853703794 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f5868034890 con 0x560bff10dd50 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.797+0000 7f5879d72740 10 client.30749 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.797+0000 7f5879d72740 10 client.30749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.797+0000 7f5879d72740 20 client.30749 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.797+0000 7f5879d72740 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bff127530 0x560bff129920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.797+0000 7f5879d72740 4 client.30749 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.797+0000 7f5879d72740 1 -- 172.21.15.18:0/2853703794 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bff080a40 con 0x560bff127530 2024-04-16T17:24:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.797+0000 7f586f400640 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bff127530 0x560bff129920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.798+0000 7f586f400640 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bff127530 0x560bff129920 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59b6d7de7f89c6e server_cookie=b518687cd688f215 in_seq=0 out_seq=0 2024-04-16T17:24:47.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.798+0000 7f585a000640 10 client.30749 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:47.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.798+0000 7f586fe00640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58540a3d30 0x7f58540a61f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x560bff115960 tx=0x7f586000b6b0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.799+0000 7f585a000640 1 -- 172.21.15.18:0/2853703794 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bff080a40 con 0x560bff127530 2024-04-16T17:24:47.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.799+0000 7f585a000640 10 client.30749 handle_command_reply: tid=0 2024-04-16T17:24:47.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.799+0000 7f5879d72740 1 client.30749 shutdown 2024-04-16T17:24:47.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f5879d72740 1 -- 172.21.15.18:0/2853703794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10dd50 msgr2=0x560bff114390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f5879d72740 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10dd50 0x560bff114390 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5868009430 tx=0x7f5868003ba0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f5879d72740 1 -- 172.21.15.18:0/2853703794 shutdown_connections 2024-04-16T17:24:47.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f5879d72740 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bff127530 0x560bff129920 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f5879d72740 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff117250 0x560bff114e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f5879d72740 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 0x560bff1148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f5879d72740 1 --2- 172.21.15.18:0/2853703794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10dd50 0x560bff114390 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f5879d72740 1 -- 172.21.15.18:0/2853703794 >> 172.21.15.18:0/2853703794 conn(0x560bff108480 msgr2=0x560bff10a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.800+0000 7f585a000640 0 client.30749 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:47.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.801+0000 7f5879d72740 1 -- 172.21.15.18:0/2853703794 shutdown_connections 2024-04-16T17:24:47.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.801+0000 7f5879d72740 1 -- 172.21.15.18:0/2853703794 wait complete. 2024-04-16T17:24:47.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.801+0000 7f5879d72740 20 client.30749 trim_cache size 0 max 16384 2024-04-16T17:24:47.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.831+0000 7f5879d72740 1 Processor -- start 2024-04-16T17:24:47.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.832+0000 7f5879d72740 1 -- start start 2024-04-16T17:24:47.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.832+0000 7f5879d72740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.832+0000 7f5879d72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff122af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.832+0000 7f5879d72740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff117250 0x560bff12c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.832+0000 7f5879d72740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58701afe30 con 0x560bff117250 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.832+0000 7f5879d72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58701ad700 con 0x560bff10e670 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.832+0000 7f5879d72740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58701a2930 con 0x560bff10dd50 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff122af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40540/0 (socket says 172.21.15.18:40540) 2024-04-16T17:24:47.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586f400640 1 -- 172.21.15.18:0/4279151910 learned_addr learned my addr 172.21.15.18:0/4279151910 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586f400640 1 -- 172.21.15.18:0/4279151910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 msgr2=0x560bff116ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586f400640 1 --2- 172.21.15.18:0/4279151910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff116ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586f400640 1 -- 172.21.15.18:0/4279151910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff117250 msgr2=0x560bff12c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586f400640 1 --2- 172.21.15.18:0/4279151910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff117250 0x560bff12c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.833+0000 7f586f400640 1 -- 172.21.15.18:0/4279151910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfed853f0 con 0x560bff10e670 2024-04-16T17:24:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.834+0000 7f586f400640 1 --2- 172.21.15.18:0/4279151910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff122af0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f585c0093d0 tx=0x7f585c00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0feb30cdf42e009 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.834+0000 7f5864c00640 1 -- 172.21.15.18:0/4279151910 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c01e040 con 0x560bff10e670 2024-04-16T17:24:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.835+0000 7f5864c00640 1 -- 172.21.15.18:0/4279151910 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c0bc0b0 con 0x560bff10e670 2024-04-16T17:24:47.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.835+0000 7f5864c00640 1 -- 172.21.15.18:0/4279151910 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c0cd380 con 0x560bff10e670 2024-04-16T17:24:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.837+0000 7f5879d72740 1 -- 172.21.15.18:0/4279151910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 msgr2=0x560bff122af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.837+0000 7f5879d72740 1 --2- 172.21.15.18:0/4279151910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff122af0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f585c0093d0 tx=0x7f585c00bf90 comp rx=0 tx=0).stop 2024-04-16T17:24:47.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.839+0000 7f5879d72740 1 -- 172.21.15.18:0/4279151910 shutdown_connections 2024-04-16T17:24:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.839+0000 7f5879d72740 1 --2- 172.21.15.18:0/4279151910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff117250 0x560bff12c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.839+0000 7f5879d72740 1 --2- 172.21.15.18:0/4279151910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff10e670 0x560bff122af0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.839+0000 7f5879d72740 1 --2- 172.21.15.18:0/4279151910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.839+0000 7f5879d72740 1 -- 172.21.15.18:0/4279151910 >> 172.21.15.18:0/4279151910 conn(0x560bff108480 msgr2=0x560bff1223e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.840+0000 7f5879d72740 1 -- 172.21.15.18:0/4279151910 shutdown_connections 2024-04-16T17:24:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.840+0000 7f5879d72740 1 -- 172.21.15.18:0/4279151910 wait complete. 2024-04-16T17:24:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.840+0000 7f5879d72740 1 -- start start 2024-04-16T17:24:47.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.841+0000 7f5879d72740 1 Processor -- start 2024-04-16T17:24:47.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.841+0000 7f5879d72740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff12ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.841+0000 7f5879d72740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 0x560bff12a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f5879d72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff117250 0x560bff12a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f5879d72740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58701a2f80 con 0x560bff10e670 2024-04-16T17:24:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f5879d72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58701ad030 con 0x560bff117250 2024-04-16T17:24:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f5879d72740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58701ac290 con 0x560bff10dd50 2024-04-16T17:24:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f5879d72740 10 client.0 _ll_register_callbacks cb 0x7f5868009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:47.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f586fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff12ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f586f400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 0x560bff12a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f586fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff12ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39962/0 (socket says 172.21.15.18:39962) 2024-04-16T17:24:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f586fe00640 1 -- 172.21.15.18:0/2980287628 learned_addr learned my addr 172.21.15.18:0/2980287628 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.842+0000 7f5875000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff117250 0x560bff12a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f5875000640 1 -- 172.21.15.18:0/2980287628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 msgr2=0x560bff12ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f5875000640 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff12ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f5875000640 1 -- 172.21.15.18:0/2980287628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 msgr2=0x560bff12a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:47.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f5875000640 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 0x560bff12a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f5875000640 1 -- 172.21.15.18:0/2980287628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfee4f7d0 con 0x560bff117250 2024-04-16T17:24:47.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f5875000640 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff117250 0x560bff12a940 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5868034ea0 tx=0x7f586802c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e669632e883b356c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:47.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f585a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:47.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f585a000640 1 -- 172.21.15.18:0/2980287628 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868052070 con 0x560bff117250 2024-04-16T17:24:47.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.843+0000 7f585a000640 1 -- 172.21.15.18:0/2980287628 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f5868034440 con 0x560bff117250 2024-04-16T17:24:47.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.845+0000 7f5879d72740 1 -- 172.21.15.18:0/2980287628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585c0bc0b0 con 0x560bff117250 2024-04-16T17:24:47.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.847+0000 7f585a000640 1 -- 172.21.15.18:0/2980287628 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5868039040 con 0x560bff117250 2024-04-16T17:24:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.847+0000 7f5879d72740 10 client.30755 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.847+0000 7f5879d72740 1 -- 172.21.15.18:0/2980287628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560bff18d6a0 con 0x560bff117250 2024-04-16T17:24:47.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.848+0000 7f585a000640 1 -- 172.21.15.18:0/2980287628 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c0bc0b0 con 0x560bff117250 2024-04-16T17:24:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.848+0000 7f585a000640 1 -- 172.21.15.18:0/2980287628 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f586806c420 con 0x560bff117250 2024-04-16T17:24:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.848+0000 7f5879d72740 10 client.30755 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.848+0000 7f5879d72740 10 client.30755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.848+0000 7f5879d72740 20 client.30755 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.848+0000 7f5879d72740 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bff18dd60 0x560bff190180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.848+0000 7f5879d72740 4 client.30755 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:24:47.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.848+0000 7f5879d72740 1 -- 172.21.15.18:0/2980287628 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bff07a8b0 con 0x560bff18dd60 2024-04-16T17:24:47.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.849+0000 7f586fe00640 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bff18dd60 0x560bff190180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.850+0000 7f586fe00640 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bff18dd60 0x560bff190180 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3377529fefc67a57 server_cookie=4f46d2b828517cf in_seq=0 out_seq=0 2024-04-16T17:24:47.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.850+0000 7f585a000640 10 client.30755 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:47.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.853+0000 7f585a000640 1 -- 172.21.15.18:0/2980287628 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bff07a8b0 con 0x560bff18dd60 2024-04-16T17:24:47.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.853+0000 7f585a000640 10 client.30755 handle_command_reply: tid=0 2024-04-16T17:24:47.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.853+0000 7f5879d72740 1 client.30755 shutdown 2024-04-16T17:24:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.853+0000 7f5879d72740 1 -- 172.21.15.18:0/2980287628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff117250 msgr2=0x560bff12a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.853+0000 7f5879d72740 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff117250 0x560bff12a940 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5868034ea0 tx=0x7f586802c890 comp rx=0 tx=0).stop 2024-04-16T17:24:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 1 -- 172.21.15.18:0/2980287628 shutdown_connections 2024-04-16T17:24:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560bff18dd60 0x560bff190180 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560bff117250 0x560bff12a940 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560bff10e670 0x560bff12a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 1 --2- 172.21.15.18:0/2980287628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bff10dd50 0x560bff12ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f585a000640 0 client.30755 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:47.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 1 -- 172.21.15.18:0/2980287628 >> 172.21.15.18:0/2980287628 conn(0x560bff108480 msgr2=0x560bff1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 1 -- 172.21.15.18:0/2980287628 shutdown_connections 2024-04-16T17:24:47.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 1 -- 172.21.15.18:0/2980287628 wait complete. 2024-04-16T17:24:47.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.854+0000 7f5879d72740 20 client.30755 trim_cache size 0 max 16384 2024-04-16T17:24:47.855 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 -- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58540a3d30 msgr2=0x7f58540a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58540a3d30 0x7f58540a61f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x560bff115960 tx=0x7f586000b6b0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 -- 172.21.15.18:0/3086304122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5870074cf0 msgr2=0x7f58701024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5870074cf0 0x7f58701024c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f585c00b4a0 tx=0x7f585c00b960 comp rx=0 tx=0).stop 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 -- 172.21.15.18:0/3086304122 shutdown_connections 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f58540a3d30 0x7f58540a61f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f587010e860 0x7f5870102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5870074cf0 0x7f58701024c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 --2- 172.21.15.18:0/3086304122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58700743d0 0x7f5870101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:47.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 -- 172.21.15.18:0/3086304122 >> 172.21.15.18:0/3086304122 conn(0x7f58700fdab0 msgr2=0x7f58700ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:47.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.855+0000 7f5877000640 1 -- 172.21.15.18:0/3086304122 shutdown_connections 2024-04-16T17:24:47.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:47.856+0000 7f5877000640 1 -- 172.21.15.18:0/3086304122 wait complete. 2024-04-16T17:24:48.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:47 smithi043 ceph-mon[36901]: pgmap v641: 129 pgs: 129 active+clean; 440 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 388 op/s 2024-04-16T17:24:48.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:47 smithi022 ceph-mon[36023]: pgmap v641: 129 pgs: 129 active+clean; 440 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 388 op/s 2024-04-16T17:24:48.599 INFO:tasks.workunit.client.0.smithi018.stdout: 10 94802 11.29 MB/sec execute 300 sec latency 206.309 ms 2024-04-16T17:24:48.880 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:24:48.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:24:48.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:24:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.088+0000 7f57b3200640 1 -- 172.21.15.18:0/601497859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac10d380 msgr2=0x7f57ac119890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.088+0000 7f57b3200640 1 --2- 172.21.15.18:0/601497859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac10d380 0x7f57ac119890 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009ca0 tx=0x7f57a402c120 comp rx=0 tx=0).stop 2024-04-16T17:24:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.088+0000 7f57b3200640 1 -- 172.21.15.18:0/601497859 shutdown_connections 2024-04-16T17:24:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.088+0000 7f57b3200640 1 --2- 172.21.15.18:0/601497859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac10d380 0x7f57ac119890 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.088+0000 7f57b3200640 1 --2- 172.21.15.18:0/601497859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57ac10c9c0 0x7f57ac10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.088+0000 7f57b3200640 1 --2- 172.21.15.18:0/601497859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57ac1129d0 0x7f57ac112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.088+0000 7f57b3200640 1 -- 172.21.15.18:0/601497859 >> 172.21.15.18:0/601497859 conn(0x7f57ac06d1b0 msgr2=0x7f57ac10b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.089+0000 7f57b3200640 1 -- 172.21.15.18:0/601497859 shutdown_connections 2024-04-16T17:24:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.089+0000 7f57b3200640 1 -- 172.21.15.18:0/601497859 wait complete. 2024-04-16T17:24:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.090+0000 7f57b3200640 1 Processor -- start 2024-04-16T17:24:49.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.091+0000 7f57b3200640 1 -- start start 2024-04-16T17:24:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.092+0000 7f57b3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57ac10c9c0 0x7f57ac1a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.092+0000 7f57b3200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57ac10d380 0x7f57ac1a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.092+0000 7f57b3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac1129d0 0x7f57ac1a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.092+0000 7f57b3200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac11ce80 con 0x7f57ac10d380 2024-04-16T17:24:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.092+0000 7f57b3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac11cd00 con 0x7f57ac1129d0 2024-04-16T17:24:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.092+0000 7f57b3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac11d000 con 0x7f57ac10c9c0 2024-04-16T17:24:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.093+0000 7f57ab400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57ac10d380 0x7f57ac1a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.094+0000 7f57ab400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57ac10d380 0x7f57ac1a3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52776/0 (socket says 172.21.15.18:52776) 2024-04-16T17:24:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.094+0000 7f57ab400640 1 -- 172.21.15.18:0/4232353464 learned_addr learned my addr 172.21.15.18:0/4232353464 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.094+0000 7f57b1200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac1129d0 0x7f57ac1a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.095+0000 7f57ab400640 1 -- 172.21.15.18:0/4232353464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57ac10c9c0 msgr2=0x7f57ac1a31e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:24:49.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.095+0000 7f57ab400640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57ac10c9c0 0x7f57ac1a31e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.095+0000 7f57ab400640 1 -- 172.21.15.18:0/4232353464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac1129d0 msgr2=0x7f57ac1a7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.095+0000 7f57ab400640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac1129d0 0x7f57ac1a7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.095+0000 7f57ab400640 1 -- 172.21.15.18:0/4232353464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac1a81d0 con 0x7f57ac10d380 2024-04-16T17:24:49.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.096+0000 7f57b1200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac1129d0 0x7f57ac1a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:49.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.098+0000 7f57ab400640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57ac10d380 0x7f57ac1a3720 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f579800b450 tx=0x7f579800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.100+0000 7f57a8c00640 1 -- 172.21.15.18:0/4232353464 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798015020 con 0x7f57ac10d380 2024-04-16T17:24:49.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.101+0000 7f57a8c00640 1 -- 172.21.15.18:0/4232353464 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5798011d70 con 0x7f57ac10d380 2024-04-16T17:24:49.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.101+0000 7f57a8c00640 1 -- 172.21.15.18:0/4232353464 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579800f340 con 0x7f57ac10d380 2024-04-16T17:24:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.101+0000 7f57b3200640 1 -- 172.21.15.18:0/4232353464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ac1a84c0 con 0x7f57ac10d380 2024-04-16T17:24:49.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.104+0000 7f57b3200640 1 -- 172.21.15.18:0/4232353464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ac1afd50 con 0x7f57ac10d380 2024-04-16T17:24:49.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.114+0000 7f57b6000740 1 Processor -- start 2024-04-16T17:24:49.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.116+0000 7f57a8c00640 1 -- 172.21.15.18:0/4232353464 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5798018020 con 0x7f57ac10d380 2024-04-16T17:24:49.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.119+0000 7f57a8c00640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57900a3ef0 0x7f57900a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.119+0000 7f57b6000740 1 -- start start 2024-04-16T17:24:49.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.120+0000 7f57b6000740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 0x55fe96328080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.120+0000 7f57b6000740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 0x55fe96330c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.120+0000 7f57b6000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96333590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.120+0000 7f57b6000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fe9628ccb0 con 0x55fe963311a0 2024-04-16T17:24:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.120+0000 7f57b6000740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe96233370 con 0x55fe963285c0 2024-04-16T17:24:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.120+0000 7f57b6000740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fe96288c70 con 0x55fe96327ca0 2024-04-16T17:24:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.120+0000 7f57a8c00640 1 -- 172.21.15.18:0/4232353464 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f57980d04a0 con 0x7f57ac10d380 2024-04-16T17:24:49.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.121+0000 7f57b1200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96333590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.122+0000 7f57b1200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96333590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52788/0 (socket says 172.21.15.18:52788) 2024-04-16T17:24:49.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.122+0000 7f57b1200640 1 -- 172.21.15.18:0/2762735490 learned_addr learned my addr 172.21.15.18:0/2762735490 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.123+0000 7f57b1200640 1 -- 172.21.15.18:0/2762735490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 msgr2=0x55fe96328080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:49.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.123+0000 7f57b1200640 1 --2- 172.21.15.18:0/2762735490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 0x55fe96328080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.123+0000 7f57b1200640 1 -- 172.21.15.18:0/2762735490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 msgr2=0x55fe96330c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:24:49.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.123+0000 7f57b1200640 1 --2- 172.21.15.18:0/2762735490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 0x55fe96330c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.123+0000 7f57b1200640 1 -- 172.21.15.18:0/2762735490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe96252910 con 0x55fe963311a0 2024-04-16T17:24:49.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.124+0000 7f57b1200640 1 --2- 172.21.15.18:0/2762735490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96333590 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005fa0 tx=0x7f57a4009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886b68fd65c86941 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.124+0000 7f57a1600640 1 -- 172.21.15.18:0/2762735490 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4039040 con 0x55fe963311a0 2024-04-16T17:24:49.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.125+0000 7f57a1600640 1 -- 172.21.15.18:0/2762735490 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a4003970 con 0x55fe963311a0 2024-04-16T17:24:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.125+0000 7f57a1600640 1 -- 172.21.15.18:0/2762735490 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4004600 con 0x55fe963311a0 2024-04-16T17:24:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.125+0000 7f57b1200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57900a3ef0 0x7f57900a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.126+0000 7f57b6000740 1 -- 172.21.15.18:0/2762735490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 msgr2=0x55fe96333590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.126+0000 7f57b6000740 1 --2- 172.21.15.18:0/2762735490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96333590 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005fa0 tx=0x7f57a4009940 comp rx=0 tx=0).stop 2024-04-16T17:24:49.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.128+0000 7f57b6000740 1 -- 172.21.15.18:0/2762735490 shutdown_connections 2024-04-16T17:24:49.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.128+0000 7f57b6000740 1 --2- 172.21.15.18:0/2762735490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96333590 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.128+0000 7f57b6000740 1 --2- 172.21.15.18:0/2762735490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 0x55fe96330c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.128+0000 7f57b6000740 1 --2- 172.21.15.18:0/2762735490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 0x55fe96328080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.128+0000 7f57b6000740 1 -- 172.21.15.18:0/2762735490 >> 172.21.15.18:0/2762735490 conn(0x55fe963223d0 msgr2=0x55fe963227b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.128+0000 7f57b1200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57900a3ef0 0x7f57900a63b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55fe96323560 tx=0x7f57a4009940 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.128+0000 7f57b6000740 1 -- 172.21.15.18:0/2762735490 shutdown_connections 2024-04-16T17:24:49.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.136+0000 7f57b6000740 1 -- 172.21.15.18:0/2762735490 wait complete. 2024-04-16T17:24:49.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.136+0000 7f57b6000740 1 -- start start 2024-04-16T17:24:49.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.137+0000 7f57b6000740 1 Processor -- start 2024-04-16T17:24:49.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.139+0000 7f57b6000740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe9632e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.139+0000 7f57b6000740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9632e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.139+0000 7f57b6000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.139+0000 7f57abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe9632e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.139+0000 7f57abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe9632e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40578/0 (socket says 172.21.15.18:40578) 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.140+0000 7f57ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9632e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.140+0000 7f57ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9632e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39988/0 (socket says 172.21.15.18:39988) 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.140+0000 7f57b1200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.140+0000 7f57abe00640 1 -- 172.21.15.18:0/1561270374 learned_addr learned my addr 172.21.15.18:0/1561270374 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.140+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fe96333f30 con 0x55fe963311a0 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.141+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe96246730 con 0x55fe96327ca0 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.141+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fe963340b0 con 0x55fe963285c0 2024-04-16T17:24:49.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.141+0000 7f57b6000740 10 client.0 _ll_register_callbacks cb 0x55fe962151e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.141+0000 7f57b1200640 1 -- 172.21.15.18:0/1561270374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 msgr2=0x55fe9632e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.142+0000 7f57b1200640 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9632e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.142+0000 7f57b1200640 1 -- 172.21.15.18:0/1561270374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 msgr2=0x55fe9632e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.142+0000 7f57b1200640 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe9632e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.142+0000 7f57b1200640 1 -- 172.21.15.18:0/1561270374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe960677e0 con 0x55fe963311a0 2024-04-16T17:24:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.142+0000 7f57ab400640 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9632e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:49.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.143+0000 7f57b1200640 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632ec30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005fa0 tx=0x7f57a404a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c47d5b0ef210c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.143+0000 7f5796000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:24:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.143+0000 7f5796000640 1 -- 172.21.15.18:0/1561270374 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4039040 con 0x55fe963311a0 2024-04-16T17:24:49.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.143+0000 7f5796000640 1 -- 172.21.15.18:0/1561270374 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f57a4034700 con 0x55fe963311a0 2024-04-16T17:24:49.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.144+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a4003970 con 0x55fe963311a0 2024-04-16T17:24:49.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.146+0000 7f5796000640 1 -- 172.21.15.18:0/1561270374 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f57a4064050 con 0x55fe963311a0 2024-04-16T17:24:49.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.152+0000 7f57b6000740 10 client.19815 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:49.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.152+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fe960c7f70 con 0x55fe963311a0 2024-04-16T17:24:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.154+0000 7f5796000640 1 -- 172.21.15.18:0/1561270374 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a4003970 con 0x55fe963311a0 2024-04-16T17:24:49.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.154+0000 7f5796000640 1 -- 172.21.15.18:0/1561270374 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f57a403f040 con 0x55fe963311a0 2024-04-16T17:24:49.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.156+0000 7f57b6000740 10 client.19815 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:49.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.156+0000 7f57b6000740 10 client.19815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:49.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.156+0000 7f57b6000740 20 client.19815 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:49.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.156+0000 7f57b6000740 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fe96341360 0x55fe96343750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.156+0000 7f57b6000740 4 client.19815 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:49.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.156+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe9629a6d0 con 0x55fe96341360 2024-04-16T17:24:49.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.158+0000 7f57abe00640 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fe96341360 0x55fe96343750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.159+0000 7f57abe00640 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fe96341360 0x55fe96343750 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda7eef921d844e3 server_cookie=7abc884e74415855 in_seq=0 out_seq=0 2024-04-16T17:24:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.160+0000 7f5796000640 10 client.19815 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.160+0000 7f5796000640 1 -- 172.21.15.18:0/1561270374 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe9629a6d0 con 0x55fe96341360 2024-04-16T17:24:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.160+0000 7f5796000640 10 client.19815 handle_command_reply: tid=0 2024-04-16T17:24:49.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.160+0000 7f57b6000740 1 client.19815 shutdown 2024-04-16T17:24:49.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.162+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 msgr2=0x55fe9632ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.162+0000 7f57b6000740 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632ec30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005fa0 tx=0x7f57a404a040 comp rx=0 tx=0).stop 2024-04-16T17:24:49.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.168+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 shutdown_connections 2024-04-16T17:24:49.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.168+0000 7f57b6000740 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fe96341360 0x55fe96343750 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.168+0000 7f57b6000740 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632ec30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.168+0000 7f57b6000740 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9632e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.168+0000 7f57b6000740 1 --2- 172.21.15.18:0/1561270374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe9632e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.168+0000 7f5796000640 0 client.19815 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:49.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.168+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 >> 172.21.15.18:0/1561270374 conn(0x55fe963223d0 msgr2=0x55fe96324120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.169+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 shutdown_connections 2024-04-16T17:24:49.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.169+0000 7f57b6000740 1 -- 172.21.15.18:0/1561270374 wait complete. 2024-04-16T17:24:49.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.177+0000 7f57b6000740 20 client.19815 trim_cache size 0 max 16384 2024-04-16T17:24:49.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b6000740 1 Processor -- start 2024-04-16T17:24:49.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b6000740 1 -- start start 2024-04-16T17:24:49.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b6000740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe963308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b6000740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9633ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b6000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b6000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac1bc710 con 0x55fe963311a0 2024-04-16T17:24:49.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b6000740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac1b9fe0 con 0x55fe96327ca0 2024-04-16T17:24:49.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b6000740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac1b7e50 con 0x55fe963285c0 2024-04-16T17:24:49.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe963308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe963308e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40588/0 (socket says 172.21.15.18:40588) 2024-04-16T17:24:49.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57abe00640 1 -- 172.21.15.18:0/1386737997 learned_addr learned my addr 172.21.15.18:0/1386737997 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b1200640 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57abe00640 1 -- 172.21.15.18:0/1386737997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 msgr2=0x55fe9633ce20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:49.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57abe00640 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9633ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57abe00640 1 -- 172.21.15.18:0/1386737997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 msgr2=0x55fe9632d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57abe00640 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57abe00640 1 -- 172.21.15.18:0/1386737997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe963283f0 con 0x55fe96327ca0 2024-04-16T17:24:49.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.224+0000 7f57b1200640 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:49.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.230+0000 7f57abe00640 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe963308e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f579c006c10 tx=0x7f579c009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b7e62deff36cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57a0c00640 1 -- 172.21.15.18:0/1386737997 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c015af0 con 0x55fe96327ca0 2024-04-16T17:24:49.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57a0c00640 1 -- 172.21.15.18:0/1386737997 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579c01d070 con 0x55fe96327ca0 2024-04-16T17:24:49.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57a0c00640 1 -- 172.21.15.18:0/1386737997 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c015af0 con 0x55fe96327ca0 2024-04-16T17:24:49.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 -- 172.21.15.18:0/1386737997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 msgr2=0x55fe963308e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe963308e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f579c006c10 tx=0x7f579c009eb0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 -- 172.21.15.18:0/1386737997 shutdown_connections 2024-04-16T17:24:49.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe9632d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe963285c0 0x55fe9633ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 --2- 172.21.15.18:0/1386737997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe96327ca0 0x55fe963308e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 -- 172.21.15.18:0/1386737997 >> 172.21.15.18:0/1386737997 conn(0x55fe963223d0 msgr2=0x55fe9632c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 -- 172.21.15.18:0/1386737997 shutdown_connections 2024-04-16T17:24:49.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 -- 172.21.15.18:0/1386737997 wait complete. 2024-04-16T17:24:49.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 -- start start 2024-04-16T17:24:49.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.232+0000 7f57b6000740 1 Processor -- start 2024-04-16T17:24:49.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b6000740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 0x55fe96339520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b6000740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 0x55fe96336300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b6000740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96336840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b6000740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac1b84a0 con 0x55fe963311a0 2024-04-16T17:24:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b6000740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac1b9910 con 0x55fe963285c0 2024-04-16T17:24:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b6000740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac1b8b70 con 0x55fe96327ca0 2024-04-16T17:24:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b6000740 10 client.0 _ll_register_callbacks cb 0x7f57a4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 0x55fe96339520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b1200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96336840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 0x55fe96339520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:40000/0 (socket says 172.21.15.18:40000) 2024-04-16T17:24:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57abe00640 1 -- 172.21.15.18:0/3102696214 learned_addr learned my addr 172.21.15.18:0/3102696214 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57ab400640 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 0x55fe96336300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b1200640 1 -- 172.21.15.18:0/3102696214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 msgr2=0x55fe96339520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b1200640 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 0x55fe96339520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b1200640 1 -- 172.21.15.18:0/3102696214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 msgr2=0x55fe96336300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b1200640 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 0x55fe96336300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b1200640 1 -- 172.21.15.18:0/3102696214 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe95f5f170 con 0x55fe963311a0 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b1200640 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96336840 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009890 tx=0x7f57a4032600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c9ee1c0b3ae6f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f5796000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f5796000640 1 -- 172.21.15.18:0/3102696214 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4034b10 con 0x55fe963311a0 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f5796000640 1 -- 172.21.15.18:0/3102696214 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f57a4034cb0 con 0x55fe963311a0 2024-04-16T17:24:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.239+0000 7f57b6000740 1 -- 172.21.15.18:0/3102696214 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579c01d070 con 0x55fe963311a0 2024-04-16T17:24:49.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.245+0000 7f5796000640 1 -- 172.21.15.18:0/3102696214 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f57a4003970 con 0x55fe963311a0 2024-04-16T17:24:49.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.245+0000 7f57b6000740 10 client.19818 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:49.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.245+0000 7f57b6000740 1 -- 172.21.15.18:0/3102696214 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fe963b0a00 con 0x55fe963311a0 2024-04-16T17:24:49.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f5796000640 1 -- 172.21.15.18:0/3102696214 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579c01d070 con 0x55fe963311a0 2024-04-16T17:24:49.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f5796000640 1 -- 172.21.15.18:0/3102696214 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f57a4070050 con 0x55fe963311a0 2024-04-16T17:24:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f57b6000740 10 client.19818 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f57b6000740 10 client.19818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f57b6000740 20 client.19818 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f57b6000740 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fe963b1120 0x55fe963b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f57b6000740 4 client.19818 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:24:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f57b6000740 1 -- 172.21.15.18:0/3102696214 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fe96294540 con 0x55fe963b1120 2024-04-16T17:24:49.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.247+0000 7f57abe00640 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fe963b1120 0x55fe963b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.254+0000 7f57abe00640 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fe963b1120 0x55fe963b3510 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e07d4269fd34e766 server_cookie=640a7e2bc99a69ef in_seq=0 out_seq=0 2024-04-16T17:24:49.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.254+0000 7f5796000640 10 client.19818 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:49.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.254+0000 7f5796000640 1 -- 172.21.15.18:0/3102696214 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe96294540 con 0x55fe963b1120 2024-04-16T17:24:49.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.254+0000 7f5796000640 10 client.19818 handle_command_reply: tid=0 2024-04-16T17:24:49.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.254+0000 7f57b6000740 1 client.19818 shutdown 2024-04-16T17:24:49.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 -- 172.21.15.18:0/3102696214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 msgr2=0x55fe96336840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96336840 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009890 tx=0x7f57a4032600 comp rx=0 tx=0).stop 2024-04-16T17:24:49.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 -- 172.21.15.18:0/3102696214 shutdown_connections 2024-04-16T17:24:49.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55fe963b1120 0x55fe963b3510 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe963311a0 0x55fe96336840 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe963285c0 0x55fe96336300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 --2- 172.21.15.18:0/3102696214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe96327ca0 0x55fe96339520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 -- 172.21.15.18:0/3102696214 >> 172.21.15.18:0/3102696214 conn(0x55fe963223d0 msgr2=0x55fe963240b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f5796000640 0 client.19818 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:49.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 -- 172.21.15.18:0/3102696214 shutdown_connections 2024-04-16T17:24:49.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 1 -- 172.21.15.18:0/3102696214 wait complete. 2024-04-16T17:24:49.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.259+0000 7f57b6000740 20 client.19818 trim_cache size 0 max 16384 2024-04-16T17:24:49.261 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:24:49.261 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:24:49.261 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "25e3051a-476a-4fad-b616-300e0c6f75c0", 2024-04-16T17:24:49.261 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:24:49.262 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:24:49.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 -- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57900a3ef0 msgr2=0x7f57900a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57900a3ef0 0x7f57900a63b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55fe96323560 tx=0x7f57a4009940 comp rx=0 tx=0).stop 2024-04-16T17:24:49.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 -- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57ac10d380 msgr2=0x7f57ac1a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57ac10d380 0x7f57ac1a3720 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f579800b450 tx=0x7f579800b910 comp rx=0 tx=0).stop 2024-04-16T17:24:49.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 -- 172.21.15.18:0/4232353464 shutdown_connections 2024-04-16T17:24:49.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f57900a3ef0 0x7f57900a63b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ac1129d0 0x7f57ac1a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57ac10d380 0x7f57ac1a3720 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 --2- 172.21.15.18:0/4232353464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57ac10c9c0 0x7f57ac1a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 -- 172.21.15.18:0/4232353464 >> 172.21.15.18:0/4232353464 conn(0x7f57ac06d1b0 msgr2=0x7f57ac10f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 -- 172.21.15.18:0/4232353464 shutdown_connections 2024-04-16T17:24:49.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.262+0000 7f57b3200640 1 -- 172.21.15.18:0/4232353464 wait complete. 2024-04-16T17:24:49.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:39 smithi018 ceph-mon[31776]: osdmap e131: 12 total, 12 up, 12 in 2024-04-16T17:24:49.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:39 smithi018 ceph-mon[31776]: pgmap v637: 129 pgs: 129 active+clean; 431 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-04-16T17:24:49.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:41 smithi018 ceph-mon[31776]: pgmap v638: 129 pgs: 129 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-04-16T17:24:49.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:43 smithi018 ceph-mon[31776]: pgmap v639: 129 pgs: 129 active+clean; 442 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 399 op/s 2024-04-16T17:24:49.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:45 smithi018 ceph-mon[31776]: pgmap v640: 129 pgs: 129 active+clean; 439 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 393 op/s 2024-04-16T17:24:49.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:46 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/358271309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:24:49.272 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:47 smithi018 ceph-mon[31776]: pgmap v641: 129 pgs: 129 active+clean; 440 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 388 op/s 2024-04-16T17:24:49.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:24:49.291 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:24:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.545+0000 7f20e3e00640 1 -- 172.21.15.18:0/198973867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4103900 msgr2=0x7f20e410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.545+0000 7f20e3e00640 1 --2- 172.21.15.18:0/198973867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4103900 0x7f20e410bde0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009860 tx=0x7f20d002bc20 comp rx=0 tx=0).stop 2024-04-16T17:24:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.546+0000 7f20e3e00640 1 -- 172.21.15.18:0/198973867 shutdown_connections 2024-04-16T17:24:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.546+0000 7f20e3e00640 1 --2- 172.21.15.18:0/198973867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e410c400 0x7f20e410e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.546+0000 7f20e3e00640 1 --2- 172.21.15.18:0/198973867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4103900 0x7f20e410bde0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.546+0000 7f20e3e00640 1 --2- 172.21.15.18:0/198973867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20e4102fe0 0x7f20e41033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.546+0000 7f20e3e00640 1 -- 172.21.15.18:0/198973867 >> 172.21.15.18:0/198973867 conn(0x7f20e4077ab0 msgr2=0x7f20e40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.546+0000 7f20e3e00640 1 -- 172.21.15.18:0/198973867 shutdown_connections 2024-04-16T17:24:49.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.546+0000 7f20e3e00640 1 -- 172.21.15.18:0/198973867 wait complete. 2024-04-16T17:24:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.547+0000 7f20e3e00640 1 Processor -- start 2024-04-16T17:24:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.547+0000 7f20e3e00640 1 -- start start 2024-04-16T17:24:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.547+0000 7f20e3e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4102fe0 0x7f20e41a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e4103900 0x7f20e41a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20e410c400 0x7f20e419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e3e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4112230 con 0x7f20e4102fe0 2024-04-16T17:24:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41120b0 con 0x7f20e4103900 2024-04-16T17:24:49.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41123b0 con 0x7f20e410c400 2024-04-16T17:24:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4102fe0 0x7f20e41a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4102fe0 0x7f20e41a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52836/0 (socket says 172.21.15.18:52836) 2024-04-16T17:24:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2a00640 1 -- 172.21.15.18:0/2324605509 learned_addr learned my addr 172.21.15.18:0/2324605509 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2000640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e4103900 0x7f20e41a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2000640 1 -- 172.21.15.18:0/2324605509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20e410c400 msgr2=0x7f20e419a850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2000640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20e410c400 0x7f20e419a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2000640 1 -- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4102fe0 msgr2=0x7f20e41a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2000640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4102fe0 0x7f20e41a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2000640 1 -- 172.21.15.18:0/2324605509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e419b110 con 0x7f20e4103900 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.548+0000 7f20e2a00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4102fe0 0x7f20e41a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.549+0000 7f20e2000640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e4103900 0x7f20e41a0c40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f20d0037040 tx=0x7f20d00346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.549+0000 7f20e8a00640 1 -- 172.21.15.18:0/2324605509 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0041020 con 0x7f20e4103900 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.549+0000 7f20e3e00640 1 -- 172.21.15.18:0/2324605509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e419b3a0 con 0x7f20e4103900 2024-04-16T17:24:49.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.549+0000 7f20e8a00640 1 -- 172.21.15.18:0/2324605509 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d002ce50 con 0x7f20e4103900 2024-04-16T17:24:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.549+0000 7f20e3e00640 1 -- 172.21.15.18:0/2324605509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e41a76f0 con 0x7f20e4103900 2024-04-16T17:24:49.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.549+0000 7f20e8a00640 1 -- 172.21.15.18:0/2324605509 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0039040 con 0x7f20e4103900 2024-04-16T17:24:49.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.553+0000 7f20ec9fd740 1 Processor -- start 2024-04-16T17:24:49.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.553+0000 7f20ec9fd740 1 -- start start 2024-04-16T17:24:49.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa70c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558a0a9cb120 con 0x558a0aa65390 2024-04-16T17:24:49.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a0a971340 con 0x558a0aa65cb0 2024-04-16T17:24:49.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558a0a9c70e0 con 0x558a0aa6e890 2024-04-16T17:24:49.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20e3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20e8a00640 1 -- 172.21.15.18:0/2324605509 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f20d0044020 con 0x7f20e4103900 2024-04-16T17:24:49.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20e3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa65770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52850/0 (socket says 172.21.15.18:52850) 2024-04-16T17:24:49.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.554+0000 7f20e3400640 1 -- 172.21.15.18:0/722019051 learned_addr learned my addr 172.21.15.18:0/722019051 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20e3400640 1 -- 172.21.15.18:0/722019051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 msgr2=0x558a0aa70c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:49.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20e3400640 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa70c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20e3400640 1 -- 172.21.15.18:0/722019051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 msgr2=0x558a0aa6e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20e2a00640 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20e3400640 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20e3400640 1 -- 172.21.15.18:0/722019051 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a0a7a77d0 con 0x558a0aa65390 2024-04-16T17:24:49.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20e2a00640 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:24:49.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20e3400640 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa65770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009d20 tx=0x7f20dc00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af785079c9f87272 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.555+0000 7f20cfe00640 1 -- 172.21.15.18:0/722019051 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00bd40 con 0x558a0aa65390 2024-04-16T17:24:49.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20cfe00640 1 -- 172.21.15.18:0/722019051 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20dc017070 con 0x558a0aa65390 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20cfe00640 1 -- 172.21.15.18:0/722019051 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc011790 con 0x558a0aa65390 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20ec9fd740 1 -- 172.21.15.18:0/722019051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 msgr2=0x558a0aa65770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa65770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009d20 tx=0x7f20dc00b420 comp rx=0 tx=0).stop 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20e8a00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20c80a3da0 0x7f20c80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20ec9fd740 1 -- 172.21.15.18:0/722019051 shutdown_connections 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa70c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/722019051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa65770 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20e8a00640 1 -- 172.21.15.18:0/2324605509 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f20d00f1960 con 0x7f20e4103900 2024-04-16T17:24:49.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.556+0000 7f20ec9fd740 1 -- 172.21.15.18:0/722019051 >> 172.21.15.18:0/722019051 conn(0x558a0aa5fac0 msgr2=0x558a0aa5fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.557+0000 7f20ec9fd740 1 -- 172.21.15.18:0/722019051 shutdown_connections 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.557+0000 7f20ec9fd740 1 -- 172.21.15.18:0/722019051 wait complete. 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.557+0000 7f20e2a00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20c80a3da0 0x7f20c80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.557+0000 7f20ec9fd740 1 -- start start 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.557+0000 7f20ec9fd740 1 Processor -- start 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20e2a00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20c80a3da0 0x7f20c80a6260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x558a0aa64e60 tx=0x7f20d4009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa6b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa6c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558a0aa71680 con 0x558a0aa65390 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a0a984710 con 0x558a0aa65cb0 2024-04-16T17:24:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558a0aa71800 con 0x558a0aa6e890 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20ec9fd740 10 client.0 _ll_register_callbacks cb 0x558a0a97faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20e2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa6b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40650/0 (socket says 172.21.15.18:40650) 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.558+0000 7f20e2000640 1 -- 172.21.15.18:0/4045387865 learned_addr learned my addr 172.21.15.18:0/4045387865 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.559+0000 7f20e2000640 1 -- 172.21.15.18:0/4045387865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 msgr2=0x558a0aa6c320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.559+0000 7f20e2000640 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa6c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.559+0000 7f20e2000640 1 -- 172.21.15.18:0/4045387865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 msgr2=0x558a0aa6b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.559+0000 7f20e2000640 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa6b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.559+0000 7f20e2000640 1 -- 172.21.15.18:0/4045387865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a0a6dd3f0 con 0x558a0aa65cb0 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.559+0000 7f20e2000640 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6bde0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005940 tx=0x7f20d00bd490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed42eca2547353fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.559+0000 7f20ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.559+0000 7f20ccc00640 1 -- 172.21.15.18:0/4045387865 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00b9040 con 0x558a0aa65cb0 2024-04-16T17:24:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.560+0000 7f20ccc00640 1 -- 172.21.15.18:0/4045387865 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f20d00eef20 con 0x558a0aa65cb0 2024-04-16T17:24:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.560+0000 7f20ec9fd740 1 -- 172.21.15.18:0/4045387865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20dc017070 con 0x558a0aa65cb0 2024-04-16T17:24:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.560+0000 7f20ccc00640 1 -- 172.21.15.18:0/4045387865 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f20d00db050 con 0x558a0aa65cb0 2024-04-16T17:24:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.560+0000 7f20ec9fd740 10 client.30776 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.560+0000 7f20ec9fd740 1 -- 172.21.15.18:0/4045387865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558a0a69d0f0 con 0x558a0aa65cb0 2024-04-16T17:24:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.560+0000 7f20ccc00640 1 -- 172.21.15.18:0/4045387865 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d00d5030 con 0x558a0aa65cb0 2024-04-16T17:24:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.561+0000 7f20ccc00640 1 -- 172.21.15.18:0/4045387865 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f20d003c030 con 0x558a0aa65cb0 2024-04-16T17:24:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.561+0000 7f20ec9fd740 10 client.30776 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.561+0000 7f20ec9fd740 10 client.30776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.561+0000 7f20ec9fd740 20 client.30776 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.561+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a0aa7eb90 0x558a0aa80ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.561+0000 7f20ec9fd740 4 client.30776 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.561+0000 7f20ec9fd740 1 -- 172.21.15.18:0/4045387865 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a0a9d8b40 con 0x558a0aa7eb90 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.562+0000 7f20e2a00640 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a0aa7eb90 0x558a0aa80ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.562+0000 7f20e2a00640 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a0aa7eb90 0x558a0aa80ff0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e95c8a69cf9e6b6 server_cookie=ca72d0c146067af0 in_seq=0 out_seq=0 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.562+0000 7f20ccc00640 10 client.30776 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ccc00640 1 -- 172.21.15.18:0/4045387865 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a0a9d8b40 con 0x558a0aa7eb90 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ccc00640 10 client.30776 handle_command_reply: tid=0 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 client.30776 shutdown 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 -- 172.21.15.18:0/4045387865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 msgr2=0x558a0aa6bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6bde0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005940 tx=0x7f20d00bd490 comp rx=0 tx=0).stop 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 -- 172.21.15.18:0/4045387865 shutdown_connections 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a0aa7eb90 0x558a0aa80ff0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ccc00640 0 client.30776 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa6c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa6bde0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/4045387865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa6b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.563+0000 7f20ec9fd740 1 -- 172.21.15.18:0/4045387865 >> 172.21.15.18:0/4045387865 conn(0x558a0aa5fac0 msgr2=0x558a0aa61810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.564+0000 7f20ec9fd740 1 -- 172.21.15.18:0/4045387865 shutdown_connections 2024-04-16T17:24:49.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.564+0000 7f20ec9fd740 1 -- 172.21.15.18:0/4045387865 wait complete. 2024-04-16T17:24:49.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.564+0000 7f20ec9fd740 20 client.30776 trim_cache size 0 max 16384 2024-04-16T17:24:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.585+0000 7f20ec9fd740 1 Processor -- start 2024-04-16T17:24:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.585+0000 7f20ec9fd740 1 -- start start 2024-04-16T17:24:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.585+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65390 0x558a0aa7a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.585+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65cb0 0x558a0aa6c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.585+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa61620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.585+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f20c40128b0 con 0x558a0aa65cb0 2024-04-16T17:24:49.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.585+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f20c4010180 con 0x558a0aa65390 2024-04-16T17:24:49.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.585+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20c400dff0 con 0x558a0aa6e890 2024-04-16T17:24:49.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65cb0 0x558a0aa6c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65cb0 0x558a0aa6c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52876/0 (socket says 172.21.15.18:52876) 2024-04-16T17:24:49.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 -- 172.21.15.18:0/1772842333 learned_addr learned my addr 172.21.15.18:0/1772842333 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2a00640 1 --2- 172.21.15.18:0/1772842333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65390 0x558a0aa7a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 -- 172.21.15.18:0/1772842333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 msgr2=0x558a0aa61620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 --2- 172.21.15.18:0/1772842333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa61620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 -- 172.21.15.18:0/1772842333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65390 msgr2=0x558a0aa7a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 --2- 172.21.15.18:0/1772842333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65390 0x558a0aa7a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 -- 172.21.15.18:0/1772842333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a0aa5f2f0 con 0x558a0aa65cb0 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20e2000640 1 --2- 172.21.15.18:0/1772842333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65cb0 0x558a0aa6c140 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f20d00bd590 tx=0x7f20d0009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61239ec94ec1f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20cf400640 1 -- 172.21.15.18:0/1772842333 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00b9040 con 0x558a0aa65cb0 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.586+0000 7f20cf400640 1 -- 172.21.15.18:0/1772842333 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d00d5050 con 0x558a0aa65cb0 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20cf400640 1 -- 172.21.15.18:0/1772842333 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00c5de0 con 0x558a0aa65cb0 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1772842333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65cb0 msgr2=0x558a0aa6c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1772842333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65cb0 0x558a0aa6c140 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f20d00bd590 tx=0x7f20d0009490 comp rx=0 tx=0).stop 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1772842333 shutdown_connections 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1772842333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa61620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1772842333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65cb0 0x558a0aa6c140 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1772842333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65390 0x558a0aa7a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1772842333 >> 172.21.15.18:0/1772842333 conn(0x558a0aa5fac0 msgr2=0x558a0aa60f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1772842333 shutdown_connections 2024-04-16T17:24:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.587+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1772842333 wait complete. 2024-04-16T17:24:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 1 -- start start 2024-04-16T17:24:49.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 1 Processor -- start 2024-04-16T17:24:49.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa76d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa72180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f20c400e640 con 0x558a0aa65390 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f20c400fab0 con 0x558a0aa65cb0 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20c400ed10 con 0x558a0aa6e890 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20ec9fd740 10 client.0 _ll_register_callbacks cb 0x7f20d0009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.588+0000 7f20e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa72180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa72180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40668/0 (socket says 172.21.15.18:40668) 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2000640 1 -- 172.21.15.18:0/1918517994 learned_addr learned my addr 172.21.15.18:0/1918517994 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2a00640 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa76d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e3400640 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2000640 1 -- 172.21.15.18:0/1918517994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 msgr2=0x558a0aa726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2000640 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2000640 1 -- 172.21.15.18:0/1918517994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 msgr2=0x558a0aa76d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2000640 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa76d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2000640 1 -- 172.21.15.18:0/1918517994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a0aa72db0 con 0x558a0aa65cb0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20e2000640 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa72180 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009230 tx=0x7f20d00c6190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=669d261b13ee864e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.589+0000 7f20ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ccc00640 1 -- 172.21.15.18:0/1918517994 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00a1060 con 0x558a0aa65cb0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1918517994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d00d5050 con 0x558a0aa65cb0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ccc00640 1 -- 172.21.15.18:0/1918517994 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f20d00a6b40 con 0x558a0aa65cb0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ccc00640 1 -- 172.21.15.18:0/1918517994 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f20d00b9040 con 0x558a0aa65cb0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 10 client.30782 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1918517994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558a0aae5040 con 0x558a0aa65cb0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ccc00640 1 -- 172.21.15.18:0/1918517994 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d00d5050 con 0x558a0aa65cb0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ccc00640 1 -- 172.21.15.18:0/1918517994 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f20d00bca50 con 0x558a0aa65cb0 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 10 client.30782 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 10 client.30782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 20 client.30782 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a0aa7eb90 0x558a0aae7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 4 client.30782 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:24:49.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.590+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1918517994 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a0a9d29b0 con 0x558a0aa7eb90 2024-04-16T17:24:49.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.591+0000 7f20e2a00640 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a0aa7eb90 0x558a0aae7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:49.599 INFO:tasks.workunit.client.0.smithi018.stdout: 10 94879 11.27 MB/sec execute 301 sec latency 854.673 ms 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.604+0000 7f20e2a00640 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a0aa7eb90 0x558a0aae7570 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2894ec576ffdc4b8 server_cookie=561a9df102855995 in_seq=0 out_seq=0 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.604+0000 7f20ccc00640 10 client.30782 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.604+0000 7f20ccc00640 1 -- 172.21.15.18:0/1918517994 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558a0a9d29b0 con 0x558a0aa7eb90 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.604+0000 7f20ccc00640 10 client.30782 handle_command_reply: tid=0 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.604+0000 7f20ec9fd740 1 client.30782 shutdown 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1918517994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 msgr2=0x558a0aa72180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa72180 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009230 tx=0x7f20d00c6190 comp rx=0 tx=0).stop 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1918517994 shutdown_connections 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x558a0aa7eb90 0x558a0aae7570 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a0aa6e890 0x558a0aa726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a0aa65cb0 0x558a0aa72180 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 --2- 172.21.15.18:0/1918517994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a0aa65390 0x558a0aa76d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1918517994 >> 172.21.15.18:0/1918517994 conn(0x558a0aa5fac0 msgr2=0x558a0aa60ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ccc00640 0 client.30782 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1918517994 shutdown_connections 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 1 -- 172.21.15.18:0/1918517994 wait complete. 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.605+0000 7f20ec9fd740 20 client.30782 trim_cache size 0 max 16384 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:24:49.606 INFO:teuthology.orchestra.run.smithi018.stdout: "25e3051a-476a-4fad-b616-300e0c6f75c0": { 2024-04-16T17:24:49.607 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:24:49.607 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "25e3051a-476a-4fad-b616-300e0c6f75c0", 2024-04-16T17:24:49.607 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:24:49.607 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:24:49.607 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:24:49.607 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:24:49.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 -- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20c80a3da0 msgr2=0x7f20c80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20c80a3da0 0x7f20c80a6260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x558a0aa64e60 tx=0x7f20d4009260 comp rx=0 tx=0).stop 2024-04-16T17:24:49.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 -- 172.21.15.18:0/2324605509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e4103900 msgr2=0x7f20e41a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:49.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e4103900 0x7f20e41a0c40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f20d0037040 tx=0x7f20d00346f0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 -- 172.21.15.18:0/2324605509 shutdown_connections 2024-04-16T17:24:49.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f20c80a3da0 0x7f20c80a6260 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20e410c400 0x7f20e419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e4103900 0x7f20e41a0c40 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 --2- 172.21.15.18:0/2324605509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20e4102fe0 0x7f20e41a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:49.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 -- 172.21.15.18:0/2324605509 >> 172.21.15.18:0/2324605509 conn(0x7f20e4077ab0 msgr2=0x7f20e4100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:49.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 -- 172.21.15.18:0/2324605509 shutdown_connections 2024-04-16T17:24:49.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:24:49.606+0000 7f20e3e00640 1 -- 172.21.15.18:0/2324605509 wait complete. 2024-04-16T17:24:49.621 INFO:tasks.cephfs.filesystem:scrub status for tag:25e3051a-476a-4fad-b616-300e0c6f75c0 - {'path': '/', 'tag': '25e3051a-476a-4fad-b616-300e0c6f75c0', 'options': 'recursive,force'} 2024-04-16T17:24:50.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:49 smithi018 ceph-mon[31776]: pgmap v642: 129 pgs: 129 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-04-16T17:24:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:49 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:24:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:49 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:24:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:49 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:49 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:50.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:49 smithi043 ceph-mon[36901]: pgmap v642: 129 pgs: 129 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-04-16T17:24:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:49 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:24:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:49 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:24:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:49 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:49 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:49 smithi022 ceph-mon[36023]: pgmap v642: 129 pgs: 129 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-04-16T17:24:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:49 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:24:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:49 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:24:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:49 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:49 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:50.599 INFO:tasks.workunit.client.0.smithi018.stdout: 10 94980 11.26 MB/sec execute 302 sec latency 1521.862 ms 2024-04-16T17:24:51.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:50 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:24:51.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:50 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:51.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:50 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:24:51.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:50 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:51.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:50 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:24:51.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:50 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:51.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:50 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:24:51.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:50 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:51.599 INFO:tasks.workunit.client.0.smithi018.stdout: 10 95171 11.25 MB/sec execute 303 sec latency 147.336 ms 2024-04-16T17:24:52.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:51 smithi043 ceph-mon[36901]: pgmap v643: 129 pgs: 129 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-16T17:24:52.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:51 smithi022 ceph-mon[36023]: pgmap v643: 129 pgs: 129 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-16T17:24:52.599 INFO:tasks.workunit.client.0.smithi018.stdout: 10 95363 11.22 MB/sec execute 304 sec latency 158.098 ms 2024-04-16T17:24:53.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 95614 11.21 MB/sec execute 305 sec latency 232.722 ms 2024-04-16T17:24:54.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:53 smithi043 ceph-mon[36901]: pgmap v644: 129 pgs: 129 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2024-04-16T17:24:54.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:53 smithi022 ceph-mon[36023]: pgmap v644: 129 pgs: 129 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2024-04-16T17:24:54.599 INFO:tasks.workunit.client.0.smithi018.stdout: 10 95913 11.22 MB/sec execute 306 sec latency 178.686 ms 2024-04-16T17:24:55.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 96115 11.21 MB/sec execute 307 sec latency 226.992 ms 2024-04-16T17:24:56.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:55 smithi043 ceph-mon[36901]: pgmap v645: 129 pgs: 129 active+clean; 450 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-04-16T17:24:56.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:55 smithi022 ceph-mon[36023]: pgmap v645: 129 pgs: 129 active+clean; 450 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-04-16T17:24:56.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 96540 11.27 MB/sec execute 308 sec latency 173.165 ms 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.213+0000 7fd42d600640 1 -- 172.21.15.18:0/101940977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 msgr2=0x7fd42810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.213+0000 7fd42d600640 1 --2- 172.21.15.18:0/101940977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 0x7fd42810bde0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd410009900 tx=0x7fd41002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.214+0000 7fd42d600640 1 -- 172.21.15.18:0/101940977 shutdown_connections 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.214+0000 7fd42d600640 1 --2- 172.21.15.18:0/101940977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd42810c400 0x7fd42810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.214+0000 7fd42d600640 1 --2- 172.21.15.18:0/101940977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 0x7fd42810bde0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.214+0000 7fd42d600640 1 --2- 172.21.15.18:0/101940977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd428102fe0 0x7fd4281033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.214+0000 7fd42d600640 1 -- 172.21.15.18:0/101940977 >> 172.21.15.18:0/101940977 conn(0x7fd4280fc7e0 msgr2=0x7fd4280fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.214+0000 7fd42d600640 1 -- 172.21.15.18:0/101940977 shutdown_connections 2024-04-16T17:24:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.214+0000 7fd42d600640 1 -- 172.21.15.18:0/101940977 wait complete. 2024-04-16T17:24:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd42d600640 1 Processor -- start 2024-04-16T17:24:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd42d600640 1 -- start start 2024-04-16T17:24:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd42d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd428102fe0 0x7fd4281a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd42d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 0x7fd4281a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd42d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd42810c400 0x7fd42819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd42d600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd428112230 con 0x7fd428103900 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd42d600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281120b0 con 0x7fd42810c400 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd42d600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281123b0 con 0x7fd428102fe0 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd41fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 0x7fd4281a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd41fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 0x7fd4281a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48800/0 (socket says 172.21.15.18:48800) 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd41fe00640 1 -- 172.21.15.18:0/1071130613 learned_addr learned my addr 172.21.15.18:0/1071130613 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.215+0000 7fd427e00640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd428102fe0 0x7fd4281a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd42cc00640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd42810c400 0x7fd42819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd41fe00640 1 -- 172.21.15.18:0/1071130613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd428102fe0 msgr2=0x7fd4281a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd41fe00640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd428102fe0 0x7fd4281a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd41fe00640 1 -- 172.21.15.18:0/1071130613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd42810c400 msgr2=0x7fd42819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd41fe00640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd42810c400 0x7fd42819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd41fe00640 1 -- 172.21.15.18:0/1071130613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42819b030 con 0x7fd428103900 2024-04-16T17:24:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd42cc00640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd42810c400 0x7fd42819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:24:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd41fe00640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 0x7fd4281a0bf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd410000c00 tx=0x7fd4100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd425600640 1 -- 172.21.15.18:0/1071130613 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410042020 con 0x7fd428103900 2024-04-16T17:24:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd425600640 1 -- 172.21.15.18:0/1071130613 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd410005000 con 0x7fd428103900 2024-04-16T17:24:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd42d600640 1 -- 172.21.15.18:0/1071130613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42819b2c0 con 0x7fd428103900 2024-04-16T17:24:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.216+0000 7fd425600640 1 -- 172.21.15.18:0/1071130613 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410003a10 con 0x7fd428103900 2024-04-16T17:24:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.217+0000 7fd42d600640 1 -- 172.21.15.18:0/1071130613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4281a7630 con 0x7fd428103900 2024-04-16T17:24:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.219+0000 7fd431a17740 1 Processor -- start 2024-04-16T17:24:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.219+0000 7fd425600640 1 -- 172.21.15.18:0/1071130613 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd410045020 con 0x7fd428103900 2024-04-16T17:24:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd431a17740 1 -- start start 2024-04-16T17:24:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd431a17740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595fed7f0 0x55f595fec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd431a17740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595fecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd431a17740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595ff6000 0x55f595ff83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd431a17740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f595f51780 con 0x55f595feca80 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd431a17740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f595ef7f40 con 0x55f595ff6000 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd431a17740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f595f4d740 con 0x55f595fed7f0 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd427e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595fecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd42cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595fed7f0 0x55f595fec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd427e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595fecf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48812/0 (socket says 172.21.15.18:48812) 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd427e00640 1 -- 172.21.15.18:0/2241184436 learned_addr learned my addr 172.21.15.18:0/2241184436 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.220+0000 7fd41fe00640 1 --2- 172.21.15.18:0/2241184436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595ff6000 0x55f595ff83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd42cc00640 1 -- 172.21.15.18:0/2241184436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595ff6000 msgr2=0x55f595ff83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd42cc00640 1 --2- 172.21.15.18:0/2241184436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595ff6000 0x55f595ff83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd42cc00640 1 -- 172.21.15.18:0/2241184436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 msgr2=0x55f595fecf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd42cc00640 1 --2- 172.21.15.18:0/2241184436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595fecf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd42cc00640 1 -- 172.21.15.18:0/2241184436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f595c64210 con 0x55f595fed7f0 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd425600640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd40c0a3d00 0x7fd40c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd42cc00640 1 --2- 172.21.15.18:0/2241184436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595fed7f0 0x55f595fec540 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd420009f10 tx=0x7fd4200107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e4617013e445a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd425600640 1 -- 172.21.15.18:0/1071130613 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fd4100f26d0 con 0x7fd428103900 2024-04-16T17:24:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd427e00640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd40c0a3d00 0x7fd40c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd41d200640 1 -- 172.21.15.18:0/2241184436 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42001c680 con 0x55f595fed7f0 2024-04-16T17:24:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd41d200640 1 -- 172.21.15.18:0/2241184436 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd42001cd00 con 0x55f595fed7f0 2024-04-16T17:24:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.221+0000 7fd41d200640 1 -- 172.21.15.18:0/2241184436 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42001a380 con 0x55f595fed7f0 2024-04-16T17:24:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 -- 172.21.15.18:0/2241184436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595fed7f0 msgr2=0x55f595fec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd427e00640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd40c0a3d00 0x7fd40c0a61c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55f595fedf50 tx=0x7fd418005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 --2- 172.21.15.18:0/2241184436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595fed7f0 0x55f595fec540 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd420009f10 tx=0x7fd4200107a0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 -- 172.21.15.18:0/2241184436 shutdown_connections 2024-04-16T17:24:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 --2- 172.21.15.18:0/2241184436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595ff6000 0x55f595ff83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 --2- 172.21.15.18:0/2241184436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595fecf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 --2- 172.21.15.18:0/2241184436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595fed7f0 0x55f595fec540 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 -- 172.21.15.18:0/2241184436 >> 172.21.15.18:0/2241184436 conn(0x55f595f875d0 msgr2=0x55f595fe7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 -- 172.21.15.18:0/2241184436 shutdown_connections 2024-04-16T17:24:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.222+0000 7fd431a17740 1 -- 172.21.15.18:0/2241184436 wait complete. 2024-04-16T17:24:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.223+0000 7fd431a17740 1 -- start start 2024-04-16T17:24:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.223+0000 7fd431a17740 1 Processor -- start 2024-04-16T17:24:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.223+0000 7fd431a17740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ff5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.223+0000 7fd431a17740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ff2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.223+0000 7fd431a17740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.223+0000 7fd431a17740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f595ff8d30 con 0x55f595fed7f0 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.223+0000 7fd431a17740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f595f0b390 con 0x55f595feca80 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.223+0000 7fd431a17740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f595ff8eb0 con 0x55f595ff6000 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd431a17740 10 client.0 _ll_register_callbacks cb 0x55f595ef4f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ff5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ff5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52688/0 (socket says 172.21.15.18:52688) 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 -- 172.21.15.18:0/2901386090 learned_addr learned my addr 172.21.15.18:0/2901386090 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd42cc00640 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd41fe00640 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ff2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 -- 172.21.15.18:0/2901386090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 msgr2=0x55f595ff3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 -- 172.21.15.18:0/2901386090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 msgr2=0x55f595ff2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ff2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 -- 172.21.15.18:0/2901386090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f595d30620 con 0x55f595feca80 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd42cc00640 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd427e00640 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ff5aa0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd41801e9d0 tx=0x7fd418004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=737b634485787e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.224+0000 7fd415600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd415600640 1 -- 172.21.15.18:0/2901386090 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41802e060 con 0x55f595feca80 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd415600640 1 -- 172.21.15.18:0/2901386090 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fd418018030 con 0x55f595feca80 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd431a17740 1 -- 172.21.15.18:0/2901386090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd42001cd00 con 0x55f595feca80 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd415600640 1 -- 172.21.15.18:0/2901386090 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd418035050 con 0x55f595feca80 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd431a17740 10 client.30794 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd431a17740 1 -- 172.21.15.18:0/2901386090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f596004e60 con 0x55f595feca80 2024-04-16T17:24:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd415600640 1 -- 172.21.15.18:0/2901386090 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd42001cd00 con 0x55f595feca80 2024-04-16T17:24:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd415600640 1 -- 172.21.15.18:0/2901386090 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fd41800f3c0 con 0x55f595feca80 2024-04-16T17:24:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd431a17740 10 client.30794 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.225+0000 7fd431a17740 10 client.30794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.226+0000 7fd431a17740 20 client.30794 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.226+0000 7fd431a17740 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f596006320 0x55f596008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.226+0000 7fd431a17740 4 client.30794 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:24:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.226+0000 7fd431a17740 1 -- 172.21.15.18:0/2901386090 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f595f5f1a0 con 0x55f596006320 2024-04-16T17:24:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.226+0000 7fd41fe00640 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f596006320 0x55f596008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.226+0000 7fd415600640 10 client.30794 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.226+0000 7fd41fe00640 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f596006320 0x55f596008710 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cac35d4895504a4 server_cookie=a873abbfe2559f7b in_seq=0 out_seq=0 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.227+0000 7fd415600640 1 -- 172.21.15.18:0/2901386090 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f595f5f1a0 con 0x55f596006320 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.227+0000 7fd415600640 10 client.30794 handle_command_reply: tid=0 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.227+0000 7fd431a17740 1 client.30794 shutdown 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.227+0000 7fd431a17740 1 -- 172.21.15.18:0/2901386090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 msgr2=0x55f595ff5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.227+0000 7fd431a17740 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ff5aa0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd41801e9d0 tx=0x7fd418004730 comp rx=0 tx=0).stop 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 1 -- 172.21.15.18:0/2901386090 shutdown_connections 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f596006320 0x55f596008710 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ff2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd415600640 0 client.30794 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 1 --2- 172.21.15.18:0/2901386090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ff5aa0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 1 -- 172.21.15.18:0/2901386090 >> 172.21.15.18:0/2901386090 conn(0x55f595f875d0 msgr2=0x55f595fe94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 1 -- 172.21.15.18:0/2901386090 shutdown_connections 2024-04-16T17:24:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 1 -- 172.21.15.18:0/2901386090 wait complete. 2024-04-16T17:24:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.228+0000 7fd431a17740 20 client.30794 trim_cache size 0 max 16384 2024-04-16T17:24:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.243+0000 7fd431a17740 1 Processor -- start 2024-04-16T17:24:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.243+0000 7fd431a17740 1 -- start start 2024-04-16T17:24:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd431a17740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595ff2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd431a17740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595fed7f0 0x55f596001c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd431a17740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd431a17740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b3fa0 con 0x55f595feca80 2024-04-16T17:24:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd431a17740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b1870 con 0x55f595fed7f0 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd431a17740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281af6e0 con 0x55f595ff6000 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd427e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595ff2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd427e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595ff2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48836/0 (socket says 172.21.15.18:48836) 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd427e00640 1 -- 172.21.15.18:0/2656638413 learned_addr learned my addr 172.21.15.18:0/2656638413 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd41fe00640 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595fed7f0 0x55f596001c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd42cc00640 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd427e00640 1 -- 172.21.15.18:0/2656638413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 msgr2=0x55f595ff5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd427e00640 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.244+0000 7fd427e00640 1 -- 172.21.15.18:0/2656638413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595fed7f0 msgr2=0x55f596001c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.245+0000 7fd427e00640 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595fed7f0 0x55f596001c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.245+0000 7fd427e00640 1 -- 172.21.15.18:0/2656638413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f595fe8770 con 0x55f595feca80 2024-04-16T17:24:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.245+0000 7fd41fe00640 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595fed7f0 0x55f596001c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.245+0000 7fd427e00640 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595ff2dd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd418019040 tx=0x7fd418012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe3325312711b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.245+0000 7fd417e00640 1 -- 172.21.15.18:0/2656638413 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41803e070 con 0x55f595feca80 2024-04-16T17:24:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.245+0000 7fd417e00640 1 -- 172.21.15.18:0/2656638413 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd418009b50 con 0x55f595feca80 2024-04-16T17:24:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.245+0000 7fd417e00640 1 -- 172.21.15.18:0/2656638413 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418029c80 con 0x55f595feca80 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 -- 172.21.15.18:0/2656638413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 msgr2=0x55f595ff2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595ff2dd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd418019040 tx=0x7fd418012d00 comp rx=0 tx=0).stop 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 -- 172.21.15.18:0/2656638413 shutdown_connections 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ff5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595fed7f0 0x55f596001c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 --2- 172.21.15.18:0/2656638413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595feca80 0x55f595ff2dd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 -- 172.21.15.18:0/2656638413 >> 172.21.15.18:0/2656638413 conn(0x55f595f875d0 msgr2=0x55f595ff26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 -- 172.21.15.18:0/2656638413 shutdown_connections 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 -- 172.21.15.18:0/2656638413 wait complete. 2024-04-16T17:24:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 -- start start 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.246+0000 7fd431a17740 1 Processor -- start 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd431a17740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ffe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd431a17740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ffe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd431a17740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ffb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd431a17740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281afd30 con 0x55f595fed7f0 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd431a17740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b11a0 con 0x55f595feca80 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd431a17740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b0400 con 0x55f595ff6000 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd431a17740 10 client.0 _ll_register_callbacks cb 0x7fd418012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd41fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ffe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd427e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ffe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd427e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ffe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:52698/0 (socket says 172.21.15.18:52698) 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd427e00640 1 -- 172.21.15.18:0/1024209719 learned_addr learned my addr 172.21.15.18:0/1024209719 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:24:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.247+0000 7fd42cc00640 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ffb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd41fe00640 1 -- 172.21.15.18:0/1024209719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 msgr2=0x55f595ffb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd41fe00640 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ffb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd41fe00640 1 -- 172.21.15.18:0/1024209719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 msgr2=0x55f595ffe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd41fe00640 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ffe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd41fe00640 1 -- 172.21.15.18:0/1024209719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f595ffb750 con 0x55f595fed7f0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd427e00640 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ffe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd41fe00640 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ffe680 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd4100efac0 tx=0x7fd4100045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddbbb7d3276e210 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd415600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd415600640 1 -- 172.21.15.18:0/1024209719 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100f2040 con 0x55f595fed7f0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd415600640 1 -- 172.21.15.18:0/1024209719 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fd410005540 con 0x55f595fed7f0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd431a17740 1 -- 172.21.15.18:0/1024209719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd418009b50 con 0x55f595fed7f0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd415600640 1 -- 172.21.15.18:0/1024209719 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd4100c6f80 con 0x55f595fed7f0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd431a17740 10 client.19854 fetch_fsmap learned FSMap version 37 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.248+0000 7fd431a17740 1 -- 172.21.15.18:0/1024209719 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f59606d210 con 0x55f595fed7f0 2024-04-16T17:24:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd415600640 1 -- 172.21.15.18:0/1024209719 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd418009b50 con 0x55f595fed7f0 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd415600640 1 -- 172.21.15.18:0/1024209719 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fd4100051b0 con 0x55f595fed7f0 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd431a17740 10 client.19854 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd431a17740 10 client.19854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd431a17740 20 client.19854 populate_metadata read hostname 'smithi018' 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd431a17740 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f59606d990 0x55f59606fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd431a17740 4 client.19854 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd431a17740 1 -- 172.21.15.18:0/1024209719 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f595f59010 con 0x55f59606d990 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.249+0000 7fd427e00640 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f59606d990 0x55f59606fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.250+0000 7fd427e00640 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f59606d990 0x55f59606fdb0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39c4fb5c7fb9ffa server_cookie=a848b17a57d77512 in_seq=0 out_seq=0 2024-04-16T17:24:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:57.250+0000 7fd415600640 10 client.19854 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:57.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 96838 11.30 MB/sec execute 309 sec latency 515.470 ms 2024-04-16T17:24:57.688 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:50 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:24:57.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:50 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:57.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:50 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:24:57.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:50 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:24:57.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:51 smithi018 ceph-mon[31776]: pgmap v643: 129 pgs: 129 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-16T17:24:57.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:53 smithi018 ceph-mon[31776]: pgmap v644: 129 pgs: 129 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2024-04-16T17:24:57.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:55 smithi018 ceph-mon[31776]: pgmap v645: 129 pgs: 129 active+clean; 450 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-04-16T17:24:58.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:57 smithi043 ceph-mon[36901]: pgmap v646: 129 pgs: 129 active+clean; 460 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 272 op/s 2024-04-16T17:24:58.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:57 smithi022 ceph-mon[36023]: pgmap v646: 129 pgs: 129 active+clean; 460 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 272 op/s 2024-04-16T17:24:58.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 96993 11.31 MB/sec execute 310 sec latency 745.494 ms 2024-04-16T17:24:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.641+0000 7fd415600640 1 -- 172.21.15.18:0/1024209719 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f595f59010 con 0x55f59606d990 2024-04-16T17:24:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.641+0000 7fd415600640 10 client.19854 handle_command_reply: tid=0 2024-04-16T17:24:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.642+0000 7fd431a17740 1 client.19854 shutdown 2024-04-16T17:24:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.642+0000 7fd431a17740 1 -- 172.21.15.18:0/1024209719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 msgr2=0x55f595ffe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.642+0000 7fd431a17740 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ffe680 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd4100efac0 tx=0x7fd4100045a0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd431a17740 1 -- 172.21.15.18:0/1024209719 shutdown_connections 2024-04-16T17:24:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd431a17740 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f59606d990 0x55f59606fdb0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd431a17740 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f595ff6000 0x55f595ffb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd431a17740 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f595fed7f0 0x55f595ffe680 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd431a17740 1 --2- 172.21.15.18:0/1024209719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f595feca80 0x55f595ffe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd431a17740 1 -- 172.21.15.18:0/1024209719 >> 172.21.15.18:0/1024209719 conn(0x55f595f875d0 msgr2=0x55f595fe5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd415600640 0 client.19854 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:24:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd431a17740 1 -- 172.21.15.18:0/1024209719 shutdown_connections 2024-04-16T17:24:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.643+0000 7fd431a17740 1 -- 172.21.15.18:0/1024209719 wait complete. 2024-04-16T17:24:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.644+0000 7fd431a17740 20 client.19854 trim_cache size 0 max 16384 2024-04-16T17:24:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:24:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:24:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:24:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:24:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.648+0000 7fd42d600640 1 -- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd40c0a3d00 msgr2=0x7fd40c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.648+0000 7fd42d600640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd40c0a3d00 0x7fd40c0a61c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55f595fedf50 tx=0x7fd418005f90 comp rx=0 tx=0).stop 2024-04-16T17:24:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.648+0000 7fd42d600640 1 -- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 msgr2=0x7fd4281a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:24:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.648+0000 7fd42d600640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 0x7fd4281a0bf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd410000c00 tx=0x7fd4100042f0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.649+0000 7fd42d600640 1 -- 172.21.15.18:0/1071130613 shutdown_connections 2024-04-16T17:24:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.649+0000 7fd42d600640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd40c0a3d00 0x7fd40c0a61c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.649+0000 7fd42d600640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd42810c400 0x7fd42819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.649+0000 7fd42d600640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd428103900 0x7fd4281a0bf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.649+0000 7fd42d600640 1 --2- 172.21.15.18:0/1071130613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd428102fe0 0x7fd4281a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:24:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.649+0000 7fd42d600640 1 -- 172.21.15.18:0/1071130613 >> 172.21.15.18:0/1071130613 conn(0x7fd4280fc7e0 msgr2=0x7fd428101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:24:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.650+0000 7fd42d600640 1 -- 172.21.15.18:0/1071130613 shutdown_connections 2024-04-16T17:24:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:24:58.651+0000 7fd42d600640 1 -- 172.21.15.18:0/1071130613 wait complete. 2024-04-16T17:24:59.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:57 smithi018 ceph-mon[31776]: pgmap v646: 129 pgs: 129 active+clean; 460 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 272 op/s 2024-04-16T17:24:59.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97030 11.28 MB/sec execute 311 sec latency 996.395 ms 2024-04-16T17:25:00.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:24:59 smithi043 ceph-mon[36901]: pgmap v647: 129 pgs: 129 active+clean; 460 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-16T17:25:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:24:59 smithi022 ceph-mon[36023]: pgmap v647: 129 pgs: 129 active+clean; 460 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-16T17:25:00.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97214 11.28 MB/sec execute 312 sec latency 1229.786 ms 2024-04-16T17:25:01.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97437 11.29 MB/sec execute 313 sec latency 234.683 ms 2024-04-16T17:25:02.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:01 smithi043 ceph-mon[36901]: pgmap v648: 129 pgs: 129 active+clean; 449 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-04-16T17:25:02.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:01 smithi022 ceph-mon[36023]: pgmap v648: 129 pgs: 129 active+clean; 449 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-04-16T17:25:02.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97602 11.27 MB/sec execute 314 sec latency 437.243 ms 2024-04-16T17:25:03.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97743 11.25 MB/sec execute 315 sec latency 757.095 ms 2024-04-16T17:25:04.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:24:59 smithi018 ceph-mon[31776]: pgmap v647: 129 pgs: 129 active+clean; 460 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-16T17:25:04.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:01 smithi018 ceph-mon[31776]: pgmap v648: 129 pgs: 129 active+clean; 449 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-04-16T17:25:04.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:03 smithi018 ceph-mon[31776]: pgmap v649: 129 pgs: 129 active+clean; 441 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-04-16T17:25:04.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:03 smithi043 ceph-mon[36901]: pgmap v649: 129 pgs: 129 active+clean; 441 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-04-16T17:25:04.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:03 smithi022 ceph-mon[36023]: pgmap v649: 129 pgs: 129 active+clean; 441 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-04-16T17:25:04.600 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97789 11.22 MB/sec execute 316 sec latency 891.581 ms 2024-04-16T17:25:05.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:05 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:25:05.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:05 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:25:05.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:05 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:25:05.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:05 smithi022 ceph-mon[36023]: osdmap e132: 12 total, 12 up, 12 in 2024-04-16T17:25:05.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:05 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:25:05.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:05 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:25:05.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:05 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:25:05.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:05 smithi043 ceph-mon[36901]: osdmap e132: 12 total, 12 up, 12 in 2024-04-16T17:25:05.601 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97842 11.19 MB/sec execute 317 sec latency 1486.808 ms 2024-04-16T17:25:06.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:06 smithi043 ceph-mon[36901]: pgmap v651: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 441 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 486 op/s 2024-04-16T17:25:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:06 smithi022 ceph-mon[36023]: pgmap v651: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 441 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 486 op/s 2024-04-16T17:25:06.601 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97895 11.16 MB/sec execute 318 sec latency 1986.562 ms 2024-04-16T17:25:06.668 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:05 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:25:06.668 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:05 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:25:06.668 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:05 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:25:06.668 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:05 smithi018 ceph-mon[31776]: osdmap e132: 12 total, 12 up, 12 in 2024-04-16T17:25:06.668 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:06 smithi018 ceph-mon[31776]: pgmap v651: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 441 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 486 op/s 2024-04-16T17:25:07.601 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97957 11.14 MB/sec execute 319 sec latency 2706.474 ms 2024-04-16T17:25:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:08 smithi022 ceph-mon[36023]: pgmap v652: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 444 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 523 op/s 2024-04-16T17:25:08.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:08 smithi043 ceph-mon[36901]: pgmap v652: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 444 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 523 op/s 2024-04-16T17:25:08.601 INFO:tasks.workunit.client.0.smithi018.stdout: 10 97999 11.10 MB/sec execute 320 sec latency 1448.507 ms 2024-04-16T17:25:08.882 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:08 smithi018 ceph-mon[31776]: pgmap v652: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 444 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 523 op/s 2024-04-16T17:25:09.605 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98032 11.07 MB/sec execute 321 sec latency 2452.994 ms 2024-04-16T17:25:10.204 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:10 smithi018 ceph-mon[31776]: pgmap v653: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 443 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 507 op/s 2024-04-16T17:25:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:10 smithi022 ceph-mon[36023]: pgmap v653: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 443 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 507 op/s 2024-04-16T17:25:10.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:10 smithi043 ceph-mon[36901]: pgmap v653: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 443 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 507 op/s 2024-04-16T17:25:10.605 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98062 11.05 MB/sec execute 322 sec latency 3453.057 ms 2024-04-16T17:25:11.605 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98109 11.02 MB/sec execute 323 sec latency 4453.116 ms 2024-04-16T17:25:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 -- 172.21.15.18:0/1946069919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 msgr2=0x7f2bd8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 --2- 172.21.15.18:0/1946069919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 0x7f2bd8074790 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009900 tx=0x7f2bc802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 -- 172.21.15.18:0/1946069919 shutdown_connections 2024-04-16T17:25:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 --2- 172.21.15.18:0/1946069919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd810e8b0 0x7f2bd8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 --2- 172.21.15.18:0/1946069919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cd0 0x7f2bd810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 --2- 172.21.15.18:0/1946069919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 0x7f2bd8074790 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 -- 172.21.15.18:0/1946069919 >> 172.21.15.18:0/1946069919 conn(0x7f2bd80fdac0 msgr2=0x7f2bd80ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 -- 172.21.15.18:0/1946069919 shutdown_connections 2024-04-16T17:25:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.862+0000 7f2bdce00640 1 -- 172.21.15.18:0/1946069919 wait complete. 2024-04-16T17:25:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.863+0000 7f2bdce00640 1 Processor -- start 2024-04-16T17:25:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.864+0000 7f2bdce00640 1 -- start start 2024-04-16T17:25:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.864+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 0x7f2bd81a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.864+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cd0 0x7f2bd81a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.864+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd810e8b0 0x7f2bd819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.864+0000 7f2bdce00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8114680 con 0x7f2bd810e8b0 2024-04-16T17:25:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.864+0000 7f2bdce00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8114500 con 0x7f2bd80743b0 2024-04-16T17:25:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.864+0000 7f2bdce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8114800 con 0x7f2bd8074cd0 2024-04-16T17:25:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 0x7f2bd81a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 0x7f2bd81a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56248/0 (socket says 172.21.15.18:56248) 2024-04-16T17:25:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5e00640 1 -- 172.21.15.18:0/491519389 learned_addr learned my addr 172.21.15.18:0/491519389 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5400640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cd0 0x7f2bd81a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd6800640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd810e8b0 0x7f2bd819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5e00640 1 -- 172.21.15.18:0/491519389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cd0 msgr2=0x7f2bd81a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cd0 0x7f2bd81a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5e00640 1 -- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd810e8b0 msgr2=0x7f2bd819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd810e8b0 0x7f2bd819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5e00640 1 -- 172.21.15.18:0/491519389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd819b140 con 0x7f2bd80743b0 2024-04-16T17:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd5400640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cd0 0x7f2bd81a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.865+0000 7f2bd6800640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd810e8b0 0x7f2bd819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.866+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 0x7f2bd81a07c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8000c00 tx=0x7f2bc80043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.866+0000 7f2bcea00640 1 -- 172.21.15.18:0/491519389 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8042020 con 0x7f2bd80743b0 2024-04-16T17:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.866+0000 7f2bdce00640 1 -- 172.21.15.18:0/491519389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd819b3d0 con 0x7f2bd80743b0 2024-04-16T17:25:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.866+0000 7f2bcea00640 1 -- 172.21.15.18:0/491519389 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc80054e0 con 0x7f2bd80743b0 2024-04-16T17:25:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.866+0000 7f2bdce00640 1 -- 172.21.15.18:0/491519389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81a7680 con 0x7f2bd80743b0 2024-04-16T17:25:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.866+0000 7f2bcea00640 1 -- 172.21.15.18:0/491519389 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8003af0 con 0x7f2bd80743b0 2024-04-16T17:25:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.875+0000 7f2bdfb8c740 1 Processor -- start 2024-04-16T17:25:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.875+0000 7f2bdfb8c740 1 -- start start 2024-04-16T17:25:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.875+0000 7f2bcea00640 1 -- 172.21.15.18:0/491519389 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bc8045020 con 0x7f2bd80743b0 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 0x55aa0b2d9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b23e780 con 0x55aa0b2e2e20 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b1e4f40 con 0x55aa0b2da610 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b23a740 con 0x55aa0b2d98a0 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39966/0 (socket says 172.21.15.18:39966) 2024-04-16T17:25:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.876+0000 7f2bd5e00640 1 -- 172.21.15.18:0/1951632522 learned_addr learned my addr 172.21.15.18:0/1951632522 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.877+0000 7f2bd5e00640 1 -- 172.21.15.18:0/1951632522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 msgr2=0x55aa0b2d9360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:25:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.877+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/1951632522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 0x55aa0b2d9360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.877+0000 7f2bd5e00640 1 -- 172.21.15.18:0/1951632522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 msgr2=0x55aa0b2e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.877+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/1951632522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.877+0000 7f2bd5e00640 1 -- 172.21.15.18:0/1951632522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0af51210 con 0x55aa0b2d98a0 2024-04-16T17:25:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.877+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/1951632522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d9d20 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80efb70 tx=0x7f2bc80f8b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=473115c4b25527ec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.878+0000 7f2bc3400640 1 -- 172.21.15.18:0/1951632522 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8101070 con 0x55aa0b2d98a0 2024-04-16T17:25:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.878+0000 7f2bc3400640 1 -- 172.21.15.18:0/1951632522 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc80fd440 con 0x55aa0b2d98a0 2024-04-16T17:25:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.878+0000 7f2bc3400640 1 -- 172.21.15.18:0/1951632522 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80f5e60 con 0x55aa0b2d98a0 2024-04-16T17:25:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.878+0000 7f2bcea00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3d90 0x7f2bbc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.878+0000 7f2bcea00640 1 -- 172.21.15.18:0/491519389 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f2bc80f2800 con 0x7f2bd80743b0 2024-04-16T17:25:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.879+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1951632522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 msgr2=0x55aa0b2d9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.879+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/1951632522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d9d20 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80efb70 tx=0x7f2bc80f8b40 comp rx=0 tx=0).stop 2024-04-16T17:25:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.880+0000 7f2bd6800640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3d90 0x7f2bbc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.880+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1951632522 shutdown_connections 2024-04-16T17:25:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.880+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/1951632522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.880+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/1951632522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d9d20 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.880+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/1951632522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 0x55aa0b2d9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.880+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1951632522 >> 172.21.15.18:0/1951632522 conn(0x55aa0b2745d0 msgr2=0x55aa0b2d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.881+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1951632522 shutdown_connections 2024-04-16T17:25:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.881+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1951632522 wait complete. 2024-04-16T17:25:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.881+0000 7f2bd6800640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3d90 0x7f2bbc0a6250 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007d90 tx=0x7f2bd0007320 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.882+0000 7f2bdfb8c740 1 -- start start 2024-04-16T17:25:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.882+0000 7f2bdfb8c740 1 Processor -- start 2024-04-16T17:25:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.884+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 0x55aa0b2dfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b2e5910 con 0x55aa0b2e2e20 2024-04-16T17:25:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b1f8390 con 0x55aa0b2da610 2024-04-16T17:25:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b2e5a90 con 0x55aa0b2d98a0 2024-04-16T17:25:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:39970/0 (socket says 172.21.15.18:39970) 2024-04-16T17:25:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bd5e00640 1 -- 172.21.15.18:0/2273075674 learned_addr learned my addr 172.21.15.18:0/2273075674 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bd5400640 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 0x55aa0b2dfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bdfb8c740 10 client.0 _ll_register_callbacks cb 0x55aa0b1e1f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.885+0000 7f2bd6800640 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.886+0000 7f2bd5400640 1 -- 172.21.15.18:0/2273075674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 msgr2=0x55aa0b2e2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.886+0000 7f2bd5400640 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.886+0000 7f2bd5400640 1 -- 172.21.15.18:0/2273075674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 msgr2=0x55aa0b2e0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.886+0000 7f2bd5400640 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.886+0000 7f2bd5400640 1 -- 172.21.15.18:0/2273075674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa0b01d620 con 0x55aa0b2da610 2024-04-16T17:25:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.886+0000 7f2bd5400640 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 0x55aa0b2dfc20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x55aa0b2d5720 tx=0x7f2bc400fdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67fc18d20f94559c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.887+0000 7f2bb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.887+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2273075674 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400ead0 con 0x55aa0b2da610 2024-04-16T17:25:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.887+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2273075674 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f2bc400ec70 con 0x55aa0b2da610 2024-04-16T17:25:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.889+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/2273075674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc80fd440 con 0x55aa0b2da610 2024-04-16T17:25:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.889+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2273075674 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2bc400ee90 con 0x55aa0b2da610 2024-04-16T17:25:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.889+0000 7f2bdfb8c740 10 client.30809 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.889+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/2273075674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55aa0b2f1a50 con 0x55aa0b2da610 2024-04-16T17:25:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.890+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2273075674 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc80fd440 con 0x55aa0b2da610 2024-04-16T17:25:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.890+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2273075674 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2bc4021410 con 0x55aa0b2da610 2024-04-16T17:25:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.891+0000 7f2bdfb8c740 10 client.30809 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.891+0000 7f2bdfb8c740 10 client.30809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.892+0000 7f2bdfb8c740 20 client.30809 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.892+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aa0b2f2f10 0x55aa0b2f5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.892+0000 7f2bdfb8c740 4 client.30809 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.892+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/2273075674 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa0b24c1a0 con 0x55aa0b2f2f10 2024-04-16T17:25:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.892+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aa0b2f2f10 0x55aa0b2f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.893+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aa0b2f2f10 0x55aa0b2f5300 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332198d1f8ea5a73 server_cookie=b6912f591135032f in_seq=0 out_seq=0 2024-04-16T17:25:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.894+0000 7f2bb7e00640 10 client.30809 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.895+0000 7f2bb7e00640 1 -- 172.21.15.18:0/2273075674 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa0b24c1a0 con 0x55aa0b2f2f10 2024-04-16T17:25:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.895+0000 7f2bb7e00640 10 client.30809 handle_command_reply: tid=0 2024-04-16T17:25:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.895+0000 7f2bdfb8c740 1 client.30809 shutdown 2024-04-16T17:25:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.895+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 msgr2=0x55aa0b2dfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.895+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 0x55aa0b2dfc20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x55aa0b2d5720 tx=0x7f2bc400fdc0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.897+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/2273075674 shutdown_connections 2024-04-16T17:25:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.897+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aa0b2f2f10 0x55aa0b2f5300 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.897+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.897+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2da610 0x55aa0b2dfc20 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.897+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/2273075674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.897+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/2273075674 >> 172.21.15.18:0/2273075674 conn(0x55aa0b2745d0 msgr2=0x55aa0b2d62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.897+0000 7f2bb7e00640 0 client.30809 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.897+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/2273075674 shutdown_connections 2024-04-16T17:25:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.898+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/2273075674 wait complete. 2024-04-16T17:25:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.898+0000 7f2bdfb8c740 20 client.30809 trim_cache size 0 max 16384 2024-04-16T17:25:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.943+0000 7f2bdfb8c740 1 Processor -- start 2024-04-16T17:25:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.943+0000 7f2bdfb8c740 1 -- start start 2024-04-16T17:25:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.943+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.943+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.943+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.943+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8012b80 con 0x55aa0b2da610 2024-04-16T17:25:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.943+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8010450 con 0x55aa0b2d98a0 2024-04-16T17:25:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.943+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb800e2c0 con 0x55aa0b2e2e20 2024-04-16T17:25:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.944+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.944+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2d3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41010/0 (socket says 172.21.15.18:41010) 2024-04-16T17:25:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.944+0000 7f2bd5400640 1 -- 172.21.15.18:0/1535029405 learned_addr learned my addr 172.21.15.18:0/1535029405 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.946+0000 7f2bd5400640 1 -- 172.21.15.18:0/1535029405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 msgr2=0x55aa0b2e2350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:25:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.947+0000 7f2bd6800640 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.947+0000 7f2bd5400640 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.947+0000 7f2bd5400640 1 -- 172.21.15.18:0/1535029405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 msgr2=0x55aa0b2d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.947+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.947+0000 7f2bd5400640 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.947+0000 7f2bd5400640 1 -- 172.21.15.18:0/1535029405 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0b2d8b70 con 0x55aa0b2da610 2024-04-16T17:25:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.947+0000 7f2bd6800640 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.947+0000 7f2bd5400640 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2d3a70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4021830 tx=0x7f2bc4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d811f8cdab06c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.951+0000 7f2bc2a00640 1 -- 172.21.15.18:0/1535029405 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4021b60 con 0x55aa0b2da610 2024-04-16T17:25:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.951+0000 7f2bc2a00640 1 -- 172.21.15.18:0/1535029405 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc4005510 con 0x55aa0b2da610 2024-04-16T17:25:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.951+0000 7f2bc2a00640 1 -- 172.21.15.18:0/1535029405 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40048e0 con 0x55aa0b2da610 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1535029405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 msgr2=0x55aa0b2d3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2d3a70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4021830 tx=0x7f2bc4005f90 comp rx=0 tx=0).stop 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1535029405 shutdown_connections 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b2e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2d3a70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/1535029405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1535029405 >> 172.21.15.18:0/1535029405 conn(0x55aa0b2745d0 msgr2=0x55aa0b2d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1535029405 shutdown_connections 2024-04-16T17:25:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.952+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/1535029405 wait complete. 2024-04-16T17:25:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.953+0000 7f2bdfb8c740 1 -- start start 2024-04-16T17:25:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.953+0000 7f2bdfb8c740 1 Processor -- start 2024-04-16T17:25:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bdfb8c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b358440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bdfb8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb800e910 con 0x55aa0b2da610 2024-04-16T17:25:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41024/0 (socket says 172.21.15.18:41024) 2024-04-16T17:25:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bd5400640 1 -- 172.21.15.18:0/821973814 learned_addr learned my addr 172.21.15.18:0/821973814 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bd6800640 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b358440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb800fd80 con 0x55aa0b2d98a0 2024-04-16T17:25:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb800efe0 con 0x55aa0b2e2e20 2024-04-16T17:25:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.954+0000 7f2bdfb8c740 10 client.0 _ll_register_callbacks cb 0x7f2bc4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.956+0000 7f2bd5e00640 1 -- 172.21.15.18:0/821973814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 msgr2=0x55aa0b358440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.956+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b358440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.956+0000 7f2bd5e00640 1 -- 172.21.15.18:0/821973814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 msgr2=0x55aa0b2e6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.956+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2e6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.956+0000 7f2bd5e00640 1 -- 172.21.15.18:0/821973814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa0b2e7850 con 0x55aa0b2d98a0 2024-04-16T17:25:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.956+0000 7f2bd5400640 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.957+0000 7f2bd5e00640 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e6a70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80bae20 tx=0x7f2bc80fdc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9375044a8107bfd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.957+0000 7f2bb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.958+0000 7f2bb7e00640 1 -- 172.21.15.18:0/821973814 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8101040 con 0x55aa0b2d98a0 2024-04-16T17:25:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.958+0000 7f2bb7e00640 1 -- 172.21.15.18:0/821973814 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f2bc80efd70 con 0x55aa0b2d98a0 2024-04-16T17:25:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.958+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc4005510 con 0x55aa0b2d98a0 2024-04-16T17:25:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.958+0000 7f2bb7e00640 1 -- 172.21.15.18:0/821973814 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2bc80f5e60 con 0x55aa0b2d98a0 2024-04-16T17:25:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.958+0000 7f2bdfb8c740 10 client.30812 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.958+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55aa0b3599d0 con 0x55aa0b2d98a0 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bb7e00640 1 -- 172.21.15.18:0/821973814 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc4005510 con 0x55aa0b2d98a0 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bb7e00640 1 -- 172.21.15.18:0/821973814 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2bc80baac0 con 0x55aa0b2d98a0 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bdfb8c740 10 client.30812 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bdfb8c740 10 client.30812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bdfb8c740 20 client.30812 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aa0b359f80 0x55aa0b35c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bdfb8c740 4 client.30812 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa0b246010 con 0x55aa0b359f80 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.959+0000 7f2bd5400640 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aa0b359f80 0x55aa0b35c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.960+0000 7f2bd5400640 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aa0b359f80 0x55aa0b35c3c0 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fae1f2c1248d03ea server_cookie=12517f757fc84ce8 in_seq=0 out_seq=0 2024-04-16T17:25:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:11.960+0000 7f2bb7e00640 10 client.30812 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:12.150 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:12 smithi018 ceph-mon[31776]: pgmap v654: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 440 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-04-16T17:25:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:12 smithi022 ceph-mon[36023]: pgmap v654: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 440 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-04-16T17:25:12.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:12 smithi043 ceph-mon[36901]: pgmap v654: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 440 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-04-16T17:25:12.605 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98125 10.99 MB/sec execute 324 sec latency 4468.858 ms 2024-04-16T17:25:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.356+0000 7f2bb7e00640 1 -- 172.21.15.18:0/821973814 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa0b246010 con 0x55aa0b359f80 2024-04-16T17:25:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.356+0000 7f2bb7e00640 10 client.30812 handle_command_reply: tid=0 2024-04-16T17:25:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.356+0000 7f2bdfb8c740 1 client.30812 shutdown 2024-04-16T17:25:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.358+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 msgr2=0x55aa0b2e6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.358+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e6a70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80bae20 tx=0x7f2bc80fdc10 comp rx=0 tx=0).stop 2024-04-16T17:25:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 shutdown_connections 2024-04-16T17:25:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55aa0b359f80 0x55aa0b35c3c0 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bb7e00640 0 client.30812 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa0b2e2e20 0x55aa0b358440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55aa0b2da610 0x55aa0b2e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bdfb8c740 1 --2- 172.21.15.18:0/821973814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa0b2d98a0 0x55aa0b2e6a70 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 >> 172.21.15.18:0/821973814 conn(0x55aa0b2745d0 msgr2=0x55aa0b2d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 shutdown_connections 2024-04-16T17:25:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.359+0000 7f2bdfb8c740 1 -- 172.21.15.18:0/821973814 wait complete. 2024-04-16T17:25:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.360+0000 7f2bdfb8c740 20 client.30812 trim_cache size 0 max 16384 2024-04-16T17:25:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:25:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:25:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:25:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:25:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 -- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3d90 msgr2=0x7f2bbc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3d90 0x7f2bbc0a6250 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007d90 tx=0x7f2bd0007320 comp rx=0 tx=0).stop 2024-04-16T17:25:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 -- 172.21.15.18:0/491519389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 msgr2=0x7f2bd81a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 0x7f2bd81a07c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8000c00 tx=0x7f2bc80043c0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 -- 172.21.15.18:0/491519389 shutdown_connections 2024-04-16T17:25:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2bbc0a3d90 0x7f2bbc0a6250 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2bd810e8b0 0x7f2bd819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8074cd0 0x7f2bd81a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 --2- 172.21.15.18:0/491519389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bd80743b0 0x7f2bd81a07c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.361+0000 7f2bdce00640 1 -- 172.21.15.18:0/491519389 >> 172.21.15.18:0/491519389 conn(0x7f2bd80fdac0 msgr2=0x7f2bd80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.363+0000 7f2bdce00640 1 -- 172.21.15.18:0/491519389 shutdown_connections 2024-04-16T17:25:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:13.363+0000 7f2bdce00640 1 -- 172.21.15.18:0/491519389 wait complete. 2024-04-16T17:25:13.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98166 10.96 MB/sec execute 325 sec latency 3773.004 ms 2024-04-16T17:25:14.244 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:14 smithi018 ceph-mon[31776]: pgmap v655: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 440 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-04-16T17:25:14.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:14 smithi022 ceph-mon[36023]: pgmap v655: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 440 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-04-16T17:25:14.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:14 smithi043 ceph-mon[36901]: pgmap v655: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 440 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-04-16T17:25:14.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98195 10.93 MB/sec execute 326 sec latency 2984.614 ms 2024-04-16T17:25:15.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98245 10.90 MB/sec execute 327 sec latency 3984.676 ms 2024-04-16T17:25:16.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:16 smithi018 ceph-mon[31776]: pgmap v656: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 438 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-04-16T17:25:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:16 smithi022 ceph-mon[36023]: pgmap v656: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 438 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-04-16T17:25:16.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:16 smithi043 ceph-mon[36901]: pgmap v656: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 438 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-04-16T17:25:16.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98280 10.87 MB/sec execute 328 sec latency 4063.707 ms 2024-04-16T17:25:17.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98313 10.84 MB/sec execute 329 sec latency 2071.556 ms 2024-04-16T17:25:18.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:18 smithi022 ceph-mon[36023]: pgmap v657: 129 pgs: 18 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 59 active+clean; 435 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-04-16T17:25:18.561 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:18 smithi043 ceph-mon[36901]: pgmap v657: 129 pgs: 18 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 59 active+clean; 435 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-04-16T17:25:18.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98379 10.82 MB/sec execute 330 sec latency 2677.753 ms 2024-04-16T17:25:19.063 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:18 smithi018 ceph-mon[31776]: pgmap v657: 129 pgs: 18 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 59 active+clean; 435 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 391 op/s 2024-04-16T17:25:19.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98442 10.81 MB/sec execute 331 sec latency 2043.281 ms 2024-04-16T17:25:19.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:25:19.623 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.942+0000 7f14e4400640 1 -- 172.21.15.18:0/2307238854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14dc11b7e0 msgr2=0x7f14dc11dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.942+0000 7f14e4400640 1 --2- 172.21.15.18:0/2307238854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14dc11b7e0 0x7f14dc11dbd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009900 tx=0x7f14d002bc40 comp rx=0 tx=0).stop 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.943+0000 7f14e4400640 1 -- 172.21.15.18:0/2307238854 shutdown_connections 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.943+0000 7f14e4400640 1 --2- 172.21.15.18:0/2307238854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14dc11b7e0 0x7f14dc11dbd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.943+0000 7f14e4400640 1 --2- 172.21.15.18:0/2307238854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc110bc0 0x7f14dc10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.943+0000 7f14e4400640 1 --2- 172.21.15.18:0/2307238854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14dc110750 0x7f14dc10e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.943+0000 7f14e4400640 1 -- 172.21.15.18:0/2307238854 >> 172.21.15.18:0/2307238854 conn(0x7f14dc06d0a0 msgr2=0x7f14dc06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.943+0000 7f14e4400640 1 -- 172.21.15.18:0/2307238854 shutdown_connections 2024-04-16T17:25:19.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.943+0000 7f14e4400640 1 -- 172.21.15.18:0/2307238854 wait complete. 2024-04-16T17:25:19.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.944+0000 7f14e4400640 1 Processor -- start 2024-04-16T17:25:19.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.946+0000 7f14e4400640 1 -- start start 2024-04-16T17:25:19.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.946+0000 7f14e4400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14dc110750 0x7f14dc1a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.946+0000 7f14e4400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14dc110bc0 0x7f14dc1a7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.946+0000 7f14e4400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc11b7e0 0x7f14dc1abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.946+0000 7f14e4400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1211b0 con 0x7f14dc110750 2024-04-16T17:25:19.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.946+0000 7f14e4400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc121030 con 0x7f14dc110bc0 2024-04-16T17:25:19.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.946+0000 7f14e4400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc121330 con 0x7f14dc11b7e0 2024-04-16T17:25:19.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.947+0000 7f14e1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14dc110bc0 0x7f14dc1a7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.947+0000 7f14e1000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14dc110bc0 0x7f14dc1a7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59862/0 (socket says 172.21.15.18:59862) 2024-04-16T17:25:19.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.947+0000 7f14e1000640 1 -- 172.21.15.18:0/2365487611 learned_addr learned my addr 172.21.15.18:0/2365487611 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:19.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.947+0000 7f14e2400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc11b7e0 0x7f14dc1abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.947+0000 7f14e1a00640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14dc110750 0x7f14dc1a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.948+0000 7f14e1000640 1 -- 172.21.15.18:0/2365487611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc11b7e0 msgr2=0x7f14dc1abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:19.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.948+0000 7f14e1000640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc11b7e0 0x7f14dc1abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.948+0000 7f14e1000640 1 -- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14dc110750 msgr2=0x7f14dc1a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:19.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.948+0000 7f14e1000640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14dc110750 0x7f14dc1a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.948+0000 7f14e1000640 1 -- 172.21.15.18:0/2365487611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc1ac620 con 0x7f14dc110bc0 2024-04-16T17:25:19.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.948+0000 7f14e1000640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14dc110bc0 0x7f14dc1a7b70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009d50 tx=0x7f14d400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.949+0000 7f14da000640 1 -- 172.21.15.18:0/2365487611 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4003b70 con 0x7f14dc110bc0 2024-04-16T17:25:19.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.950+0000 7f14da000640 1 -- 172.21.15.18:0/2365487611 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d4003d10 con 0x7f14dc110bc0 2024-04-16T17:25:19.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.950+0000 7f14e4400640 1 -- 172.21.15.18:0/2365487611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc1ac910 con 0x7f14dc110bc0 2024-04-16T17:25:19.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.950+0000 7f14e4400640 1 -- 172.21.15.18:0/2365487611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc1b41a0 con 0x7f14dc110bc0 2024-04-16T17:25:19.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.950+0000 7f14da000640 1 -- 172.21.15.18:0/2365487611 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4010a20 con 0x7f14dc110bc0 2024-04-16T17:25:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.959+0000 7f14e702e740 1 Processor -- start 2024-04-16T17:25:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.960+0000 7f14da000640 1 -- 172.21.15.18:0/2365487611 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14d4010bc0 con 0x7f14dc110bc0 2024-04-16T17:25:19.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.960+0000 7f14e702e740 1 -- start start 2024-04-16T17:25:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.960+0000 7f14e702e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343af1b0 0x55b8343af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.960+0000 7f14e702e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 0x55b8343b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.960+0000 7f14e702e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.960+0000 7f14e702e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b834315120 con 0x55b8343afad0 2024-04-16T17:25:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.960+0000 7f14e702e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b8342bb340 con 0x55b8343af1b0 2024-04-16T17:25:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.960+0000 7f14e702e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b8343110e0 con 0x55b8343b86b0 2024-04-16T17:25:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.961+0000 7f14e2400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343af1b0 0x55b8343af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.961+0000 7f14e2400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343af1b0 0x55b8343af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59878/0 (socket says 172.21.15.18:59878) 2024-04-16T17:25:19.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.961+0000 7f14e2400640 1 -- 172.21.15.18:0/2040856619 learned_addr learned my addr 172.21.15.18:0/2040856619 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:19.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.961+0000 7f14e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.962+0000 7f14e1a00640 1 --2- 172.21.15.18:0/2040856619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 0x55b8343b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.963+0000 7f14da000640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14c40a3e70 0x7f14c40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.963+0000 7f14e1a00640 1 -- 172.21.15.18:0/2040856619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 msgr2=0x55b8343baaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:19.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.963+0000 7f14e1a00640 1 --2- 172.21.15.18:0/2040856619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343baaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.963+0000 7f14e1a00640 1 -- 172.21.15.18:0/2040856619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343af1b0 msgr2=0x55b8343af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:19.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.963+0000 7f14e1a00640 1 --2- 172.21.15.18:0/2040856619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343af1b0 0x55b8343af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.963+0000 7f14e1a00640 1 -- 172.21.15.18:0/2040856619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8340f17d0 con 0x55b8343afad0 2024-04-16T17:25:19.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.964+0000 7f14da000640 1 -- 172.21.15.18:0/2365487611 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f14d40d11c0 con 0x7f14dc110bc0 2024-04-16T17:25:19.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.964+0000 7f14e1a00640 1 --2- 172.21.15.18:0/2040856619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 0x55b8343b8170 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009ac0 tx=0x7f14c8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601b01f519d09aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:19.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.964+0000 7f14e2400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14c40a3e70 0x7f14c40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.964+0000 7f14cea00640 1 -- 172.21.15.18:0/2040856619 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8016070 con 0x55b8343afad0 2024-04-16T17:25:19.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.964+0000 7f14cea00640 1 -- 172.21.15.18:0/2040856619 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8005500 con 0x55b8343afad0 2024-04-16T17:25:19.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.964+0000 7f14cea00640 1 -- 172.21.15.18:0/2040856619 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8011470 con 0x55b8343afad0 2024-04-16T17:25:19.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.966+0000 7f14e702e740 1 -- 172.21.15.18:0/2040856619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 msgr2=0x55b8343b8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:19.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.966+0000 7f14e702e740 1 --2- 172.21.15.18:0/2040856619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 0x55b8343b8170 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009ac0 tx=0x7f14c8009f80 comp rx=0 tx=0).stop 2024-04-16T17:25:19.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.967+0000 7f14e2400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14c40a3e70 0x7f14c40a6330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f14d00096c0 tx=0x7f14d002e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:19.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.967+0000 7f14e702e740 1 -- 172.21.15.18:0/2040856619 shutdown_connections 2024-04-16T17:25:19.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.967+0000 7f14e702e740 1 --2- 172.21.15.18:0/2040856619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343baaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.967+0000 7f14e702e740 1 --2- 172.21.15.18:0/2040856619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 0x55b8343b8170 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.967+0000 7f14e702e740 1 --2- 172.21.15.18:0/2040856619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343af1b0 0x55b8343af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.967+0000 7f14e702e740 1 -- 172.21.15.18:0/2040856619 >> 172.21.15.18:0/2040856619 conn(0x55b8343a98e0 msgr2=0x55b8343a9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:19.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.967+0000 7f14e702e740 1 -- 172.21.15.18:0/2040856619 shutdown_connections 2024-04-16T17:25:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.967+0000 7f14e702e740 1 -- 172.21.15.18:0/2040856619 wait complete. 2024-04-16T17:25:19.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.968+0000 7f14e702e740 1 -- start start 2024-04-16T17:25:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.968+0000 7f14e702e740 1 Processor -- start 2024-04-16T17:25:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.969+0000 7f14e702e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343af1b0 0x55b8343b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.969+0000 7f14e702e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.969+0000 7f14e702e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.969+0000 7f14e702e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b8343bb4a0 con 0x55b8343af1b0 2024-04-16T17:25:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.969+0000 7f14e702e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b8342ce710 con 0x55b8343afad0 2024-04-16T17:25:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.969+0000 7f14e702e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b8343bb620 con 0x55b8343b86b0 2024-04-16T17:25:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.969+0000 7f14e702e740 10 client.0 _ll_register_callbacks cb 0x55b8342c9af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:19.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.970+0000 7f14e2400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.971+0000 7f14e2400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49300/0 (socket says 172.21.15.18:49300) 2024-04-16T17:25:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.971+0000 7f14e2400640 1 -- 172.21.15.18:0/1759393288 learned_addr learned my addr 172.21.15.18:0/1759393288 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.971+0000 7f14e1000640 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.971+0000 7f14e1a00640 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343af1b0 0x55b8343b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.972+0000 7f14e2400640 1 -- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 msgr2=0x55b8343b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:19.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.972+0000 7f14e2400640 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.972+0000 7f14e2400640 1 -- 172.21.15.18:0/1759393288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343af1b0 msgr2=0x55b8343b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.972+0000 7f14e2400640 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343af1b0 0x55b8343b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.972+0000 7f14e2400640 1 -- 172.21.15.18:0/1759393288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8340273f0 con 0x55b8343b86b0 2024-04-16T17:25:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.972+0000 7f14e1000640 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:19.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.972+0000 7f14e2400640 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343b6120 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f14d002c180 tx=0x7f14d0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=294bab0628642f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.972+0000 7f14c3400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.973+0000 7f14c3400640 1 -- 172.21.15.18:0/1759393288 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d004f070 con 0x55b8343b86b0 2024-04-16T17:25:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.973+0000 7f14c3400640 1 -- 172.21.15.18:0/1759393288 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f14d0004ad0 con 0x55b8343b86b0 2024-04-16T17:25:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.973+0000 7f14e702e740 1 -- 172.21.15.18:0/1759393288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c8005500 con 0x55b8343b86b0 2024-04-16T17:25:19.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.973+0000 7f14c3400640 1 -- 172.21.15.18:0/1759393288 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f14d002cd50 con 0x55b8343b86b0 2024-04-16T17:25:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.973+0000 7f14e702e740 10 client.30676 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.973+0000 7f14e702e740 1 -- 172.21.15.18:0/1759393288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b833fe70f0 con 0x55b8343b86b0 2024-04-16T17:25:19.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.974+0000 7f14c3400640 1 -- 172.21.15.18:0/1759393288 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8005500 con 0x55b8343b86b0 2024-04-16T17:25:19.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.975+0000 7f14c3400640 1 -- 172.21.15.18:0/1759393288 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f14d00038f0 con 0x55b8343b86b0 2024-04-16T17:25:19.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.975+0000 7f14e702e740 10 client.30676 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:19.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.975+0000 7f14e702e740 10 client.30676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:19.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.975+0000 7f14e702e740 20 client.30676 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:19.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.975+0000 7f14e702e740 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b8343c89e0 0x55b8343cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:19.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.976+0000 7f14e702e740 4 client.30676 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:19.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.976+0000 7f14e702e740 1 -- 172.21.15.18:0/1759393288 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b834322b40 con 0x55b8343c89e0 2024-04-16T17:25:19.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.976+0000 7f14e1a00640 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b8343c89e0 0x55b8343cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:19.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.977+0000 7f14e1a00640 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b8343c89e0 0x55b8343cae60 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc19ea8e428b700 server_cookie=b3e9bbda1cb3a0c0 in_seq=0 out_seq=0 2024-04-16T17:25:19.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.977+0000 7f14c3400640 10 client.30676 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:19.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.978+0000 7f14c3400640 1 -- 172.21.15.18:0/1759393288 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b834322b40 con 0x55b8343c89e0 2024-04-16T17:25:19.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.978+0000 7f14c3400640 10 client.30676 handle_command_reply: tid=0 2024-04-16T17:25:19.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.978+0000 7f14e702e740 1 client.30676 shutdown 2024-04-16T17:25:19.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 -- 172.21.15.18:0/1759393288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 msgr2=0x55b8343b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:19.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343b6120 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f14d002c180 tx=0x7f14d0009260 comp rx=0 tx=0).stop 2024-04-16T17:25:19.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 -- 172.21.15.18:0/1759393288 shutdown_connections 2024-04-16T17:25:19.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b8343c89e0 0x55b8343cae60 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343b86b0 0x55b8343b6120 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 --2- 172.21.15.18:0/1759393288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343af1b0 0x55b8343b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:19.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 -- 172.21.15.18:0/1759393288 >> 172.21.15.18:0/1759393288 conn(0x55b8343a98e0 msgr2=0x55b8343ab630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:19.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14c3400640 0 client.30676 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:19.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.979+0000 7f14e702e740 1 -- 172.21.15.18:0/1759393288 shutdown_connections 2024-04-16T17:25:19.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.980+0000 7f14e702e740 1 -- 172.21.15.18:0/1759393288 wait complete. 2024-04-16T17:25:19.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:19.980+0000 7f14e702e740 20 client.30676 trim_cache size 0 max 16384 2024-04-16T17:25:20.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.019+0000 7f14e702e740 1 Processor -- start 2024-04-16T17:25:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.020+0000 7f14e702e740 1 -- start start 2024-04-16T17:25:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e702e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e702e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e702e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343b86b0 0x55b8343c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e702e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc012a70 con 0x55b8343b86b0 2024-04-16T17:25:20.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e702e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc010340 con 0x55b8343afad0 2024-04-16T17:25:20.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e702e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc00e1b0 con 0x55b8343af1b0 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49310/0 (socket says 172.21.15.18:49310) 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.021+0000 7f14e1a00640 1 -- 172.21.15.18:0/3058338195 learned_addr learned my addr 172.21.15.18:0/3058338195 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.022+0000 7f14e1000640 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.022+0000 7f14e1a00640 1 -- 172.21.15.18:0/3058338195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 msgr2=0x55b8343b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.022+0000 7f14e1a00640 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.022+0000 7f14e1a00640 1 -- 172.21.15.18:0/3058338195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343b86b0 msgr2=0x55b8343c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.022+0000 7f14e1a00640 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343b86b0 0x55b8343c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.022+0000 7f14e1a00640 1 -- 172.21.15.18:0/3058338195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8343ae4d0 con 0x55b8343af1b0 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.022+0000 7f14e1000640 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.022+0000 7f14e1a00640 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343b7dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x55b8343aec80 tx=0x7f14c8004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ae398075cd2fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:20.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.023+0000 7f14ce000640 1 -- 172.21.15.18:0/3058338195 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8016040 con 0x55b8343af1b0 2024-04-16T17:25:20.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.023+0000 7f14ce000640 1 -- 172.21.15.18:0/3058338195 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8009690 con 0x55b8343af1b0 2024-04-16T17:25:20.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.023+0000 7f14ce000640 1 -- 172.21.15.18:0/3058338195 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80116d0 con 0x55b8343af1b0 2024-04-16T17:25:20.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 -- 172.21.15.18:0/3058338195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 msgr2=0x55b8343b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:20.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343b7dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x55b8343aec80 tx=0x7f14c8004af0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 -- 172.21.15.18:0/3058338195 shutdown_connections 2024-04-16T17:25:20.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343b86b0 0x55b8343c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343afad0 0x55b8343b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 --2- 172.21.15.18:0/3058338195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343b7dd0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 -- 172.21.15.18:0/3058338195 >> 172.21.15.18:0/3058338195 conn(0x55b8343a98e0 msgr2=0x55b8343a88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:20.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 -- 172.21.15.18:0/3058338195 shutdown_connections 2024-04-16T17:25:20.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 -- 172.21.15.18:0/3058338195 wait complete. 2024-04-16T17:25:20.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.024+0000 7f14e702e740 1 -- start start 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.025+0000 7f14e702e740 1 Processor -- start 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.025+0000 7f14e702e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.025+0000 7f14e702e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 0x55b8343c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.025+0000 7f14e702e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343b86b0 0x55b8343bd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.025+0000 7f14e702e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc00e800 con 0x55b8343afad0 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.025+0000 7f14e702e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc00fc70 con 0x55b8343b86b0 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.025+0000 7f14e702e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc00eed0 con 0x55b8343af1b0 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e702e740 10 client.0 _ll_register_callbacks cb 0x7f14d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49314/0 (socket says 172.21.15.18:49314) 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e1a00640 1 -- 172.21.15.18:0/4134151058 learned_addr learned my addr 172.21.15.18:0/4134151058 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:20.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e2400640 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343b86b0 0x55b8343bd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:20.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e1a00640 1 -- 172.21.15.18:0/4134151058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343b86b0 msgr2=0x55b8343bd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:20.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e1a00640 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343b86b0 0x55b8343bd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e1a00640 1 -- 172.21.15.18:0/4134151058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 msgr2=0x55b8343c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:20.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e1a00640 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 0x55b8343c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.026+0000 7f14e1a00640 1 -- 172.21.15.18:0/4134151058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8343bdf40 con 0x55b8343af1b0 2024-04-16T17:25:20.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.027+0000 7f14c3400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:20.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.027+0000 7f14e1a00640 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343c08d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f14c8006010 tx=0x7f14c8005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc405f0bfb41ff78 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:20.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.027+0000 7f14c3400640 1 -- 172.21.15.18:0/4134151058 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8016040 con 0x55b8343af1b0 2024-04-16T17:25:20.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.027+0000 7f14c3400640 1 -- 172.21.15.18:0/4134151058 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f14c800ecc0 con 0x55b8343af1b0 2024-04-16T17:25:20.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.028+0000 7f14e702e740 1 -- 172.21.15.18:0/4134151058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c8009690 con 0x55b8343af1b0 2024-04-16T17:25:20.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.028+0000 7f14c3400640 1 -- 172.21.15.18:0/4134151058 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f14c8016040 con 0x55b8343af1b0 2024-04-16T17:25:20.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.028+0000 7f14e702e740 10 client.30682 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:20.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.028+0000 7f14e702e740 1 -- 172.21.15.18:0/4134151058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b83442f8b0 con 0x55b8343af1b0 2024-04-16T17:25:20.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.028+0000 7f14c3400640 1 -- 172.21.15.18:0/4134151058 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8028600 con 0x55b8343af1b0 2024-04-16T17:25:20.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.029+0000 7f14c3400640 1 -- 172.21.15.18:0/4134151058 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f14c800f040 con 0x55b8343af1b0 2024-04-16T17:25:20.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.030+0000 7f14e702e740 10 client.30682 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:20.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.030+0000 7f14e702e740 10 client.30682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:20.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.030+0000 7f14e702e740 20 client.30682 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:20.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.030+0000 7f14e702e740 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b83442fd70 0x55b834432190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.030+0000 7f14e702e740 4 client.30682 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:25:20.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.030+0000 7f14e702e740 1 -- 172.21.15.18:0/4134151058 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f14c8009690 con 0x55b83442fd70 2024-04-16T17:25:20.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.030+0000 7f14e1000640 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b83442fd70 0x55b834432190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:20.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.031+0000 7f14e1000640 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b83442fd70 0x55b834432190 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ffd95de632ba20 server_cookie=5efcdf305b4fbd8d in_seq=0 out_seq=0 2024-04-16T17:25:20.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.031+0000 7f14c3400640 10 client.30682 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:20.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.032+0000 7f14c3400640 1 -- 172.21.15.18:0/4134151058 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f14c8009690 con 0x55b83442fd70 2024-04-16T17:25:20.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.032+0000 7f14c3400640 10 client.30682 handle_command_reply: tid=0 2024-04-16T17:25:20.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.032+0000 7f14e702e740 1 client.30682 shutdown 2024-04-16T17:25:20.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.032+0000 7f14e702e740 1 -- 172.21.15.18:0/4134151058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 msgr2=0x55b8343c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:20.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.032+0000 7f14e702e740 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343c08d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f14c8006010 tx=0x7f14c8005b00 comp rx=0 tx=0).stop 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14e702e740 1 -- 172.21.15.18:0/4134151058 shutdown_connections 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14e702e740 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55b83442fd70 0x55b834432190 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14e702e740 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8343b86b0 0x55b8343bd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14e702e740 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b8343afad0 0x55b8343c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14e702e740 1 --2- 172.21.15.18:0/4134151058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8343af1b0 0x55b8343c08d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14c3400640 0 client.30682 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14e702e740 1 -- 172.21.15.18:0/4134151058 >> 172.21.15.18:0/4134151058 conn(0x55b8343a98e0 msgr2=0x55b8343b5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14e702e740 1 -- 172.21.15.18:0/4134151058 shutdown_connections 2024-04-16T17:25:20.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.033+0000 7f14e702e740 1 -- 172.21.15.18:0/4134151058 wait complete. 2024-04-16T17:25:20.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.035+0000 7f14e702e740 20 client.30682 trim_cache size 0 max 16384 2024-04-16T17:25:20.034 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:25:20.034 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:25:20.034 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:25:20.034 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 -- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14c40a3e70 msgr2=0x7f14c40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14c40a3e70 0x7f14c40a6330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f14d00096c0 tx=0x7f14d002e040 comp rx=0 tx=0).stop 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 -- 172.21.15.18:0/2365487611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14dc110bc0 msgr2=0x7f14dc1a7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14dc110bc0 0x7f14dc1a7b70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009d50 tx=0x7f14d400c6a0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 -- 172.21.15.18:0/2365487611 shutdown_connections 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f14c40a3e70 0x7f14c40a6330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc11b7e0 0x7f14dc1abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14dc110bc0 0x7f14dc1a7b70 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 --2- 172.21.15.18:0/2365487611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14dc110750 0x7f14dc1a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.036+0000 7f14e4400640 1 -- 172.21.15.18:0/2365487611 >> 172.21.15.18:0/2365487611 conn(0x7f14dc06d0a0 msgr2=0x7f14dc11a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.037+0000 7f14e4400640 1 -- 172.21.15.18:0/2365487611 shutdown_connections 2024-04-16T17:25:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.037+0000 7f14e4400640 1 -- 172.21.15.18:0/2365487611 wait complete. 2024-04-16T17:25:20.064 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:25:20.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:25:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:19 smithi022 ceph-mon[36023]: pgmap v658: 129 pgs: 18 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 59 active+clean; 435 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-04-16T17:25:20.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:19 smithi043 ceph-mon[36901]: pgmap v658: 129 pgs: 18 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 59 active+clean; 435 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-04-16T17:25:20.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:19 smithi018 ceph-mon[31776]: pgmap v658: 129 pgs: 18 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 59 active+clean; 435 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-04-16T17:25:20.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.371+0000 7f9f04000640 1 -- 172.21.15.18:0/2230811001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 msgr2=0x7f9efc117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:20.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.371+0000 7f9f04000640 1 --2- 172.21.15.18:0/2230811001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 0x7f9efc117540 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009900 tx=0x7f9ef002bc30 comp rx=0 tx=0).stop 2024-04-16T17:25:20.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.372+0000 7f9f04000640 1 -- 172.21.15.18:0/2230811001 shutdown_connections 2024-04-16T17:25:20.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.372+0000 7f9f04000640 1 --2- 172.21.15.18:0/2230811001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 0x7f9efc117540 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.372+0000 7f9f04000640 1 --2- 172.21.15.18:0/2230811001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9efc109920 0x7f9efc114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.372+0000 7f9f04000640 1 --2- 172.21.15.18:0/2230811001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9efc109000 0x7f9efc1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.372+0000 7f9f04000640 1 -- 172.21.15.18:0/2230811001 >> 172.21.15.18:0/2230811001 conn(0x7f9efc06e390 msgr2=0x7f9efc06e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:20.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.372+0000 7f9f04000640 1 -- 172.21.15.18:0/2230811001 shutdown_connections 2024-04-16T17:25:20.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.372+0000 7f9f04000640 1 -- 172.21.15.18:0/2230811001 wait complete. 2024-04-16T17:25:20.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.373+0000 7f9f04000640 1 Processor -- start 2024-04-16T17:25:20.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.374+0000 7f9f04000640 1 -- start start 2024-04-16T17:25:20.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.374+0000 7f9f04000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9efc109000 0x7f9efc10aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.374+0000 7f9f04000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9efc109920 0x7f9efc10afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.374+0000 7f9f04000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 0x7f9efc10b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.374+0000 7f9f04000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc11ac60 con 0x7f9efc109920 2024-04-16T17:25:20.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.374+0000 7f9f04000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc11aae0 con 0x7f9efc115150 2024-04-16T17:25:20.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.374+0000 7f9f04000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc11ade0 con 0x7f9efc109000 2024-04-16T17:25:20.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.375+0000 7f9f02000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 0x7f9efc10b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:20.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.375+0000 7f9f02000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 0x7f9efc10b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59914/0 (socket says 172.21.15.18:59914) 2024-04-16T17:25:20.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.375+0000 7f9f02000640 1 -- 172.21.15.18:0/1359781629 learned_addr learned my addr 172.21.15.18:0/1359781629 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:20.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.375+0000 7f9f02000640 1 -- 172.21.15.18:0/1359781629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9efc109000 msgr2=0x7f9efc10aa70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:25:20.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.375+0000 7f9f02000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9efc109000 0x7f9efc10aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.375+0000 7f9f02000640 1 -- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9efc109920 msgr2=0x7f9efc10afb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:25:20.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.378+0000 7f9f00c00640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9efc109920 0x7f9efc10afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:20.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.378+0000 7f9f02000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9efc109920 0x7f9efc10afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.378+0000 7f9f02000640 1 -- 172.21.15.18:0/1359781629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc1b3a00 con 0x7f9efc115150 2024-04-16T17:25:20.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.378+0000 7f9f02000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 0x7f9efc10b4f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9ef002c170 tx=0x7f9ef00045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:20.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.379+0000 7f9efa000640 1 -- 172.21.15.18:0/1359781629 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0041070 con 0x7f9efc115150 2024-04-16T17:25:20.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.379+0000 7f9efa000640 1 -- 172.21.15.18:0/1359781629 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef002cd40 con 0x7f9efc115150 2024-04-16T17:25:20.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.379+0000 7f9efa000640 1 -- 172.21.15.18:0/1359781629 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0003de0 con 0x7f9efc115150 2024-04-16T17:25:20.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.380+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9efc1b3bd0 con 0x7f9efc115150 2024-04-16T17:25:20.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.380+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9efc1b4070 con 0x7f9efc115150 2024-04-16T17:25:20.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.383+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9efc10a6f0 con 0x7f9efc115150 2024-04-16T17:25:20.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.387+0000 7f9efa000640 1 -- 172.21.15.18:0/1359781629 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9ef0044070 con 0x7f9efc115150 2024-04-16T17:25:20.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.389+0000 7f9efa000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9ee40a3ef0 0x7f9ee40a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:20.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.389+0000 7f9efa000640 1 -- 172.21.15.18:0/1359781629 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f9ef00f2380 con 0x7f9efc115150 2024-04-16T17:25:20.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.391+0000 7f9f01600640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9ee40a3ef0 0x7f9ee40a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:20.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.391+0000 7f9f01600640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9ee40a3ef0 0x7f9ee40a63b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4007c10 tx=0x7f9ef40075f0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:20.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.395+0000 7f9efa000640 1 -- 172.21.15.18:0/1359781629 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ef00be670 con 0x7f9efc115150 2024-04-16T17:25:20.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98519 10.79 MB/sec execute 332 sec latency 2304.252 ms 2024-04-16T17:25:20.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.898+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9efc1b4360 con 0x7f9efc115150 2024-04-16T17:25:20.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.899+0000 7f9efa000640 1 -- 172.21.15.18:0/1359781629 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7f9ef00bdf70 con 0x7f9efc115150 2024-04-16T17:25:20.900 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:25:20.903 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:25:20.904 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:25:20.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.904+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9ee40a3ef0 msgr2=0x7f9ee40a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:20.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.904+0000 7f9f04000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9ee40a3ef0 0x7f9ee40a63b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4007c10 tx=0x7f9ef40075f0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 msgr2=0x7f9efc10b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:20.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 0x7f9efc10b4f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9ef002c170 tx=0x7f9ef00045e0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 shutdown_connections 2024-04-16T17:25:20.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9ee40a3ef0 0x7f9ee40a63b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9efc115150 0x7f9efc10b4f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9efc109920 0x7f9efc10afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 --2- 172.21.15.18:0/1359781629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9efc109000 0x7f9efc10aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:20.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 >> 172.21.15.18:0/1359781629 conn(0x7f9efc06e390 msgr2=0x7f9efc070b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:20.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 shutdown_connections 2024-04-16T17:25:20.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:20.905+0000 7f9f04000640 1 -- 172.21.15.18:0/1359781629 wait complete. 2024-04-16T17:25:20.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:25:21.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:20 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1359781629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.292+0000 7fb4cd000640 1 -- 172.21.15.18:0/3783130414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c8102fe0 msgr2=0x7fb4c81033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.292+0000 7fb4cd000640 1 --2- 172.21.15.18:0/3783130414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c8102fe0 0x7fb4c81033c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009ca0 tx=0x7fb4c002c180 comp rx=0 tx=0).stop 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.292+0000 7fb4cd000640 1 -- 172.21.15.18:0/3783130414 shutdown_connections 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.292+0000 7fb4cd000640 1 --2- 172.21.15.18:0/3783130414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4c810c400 0x7fb4c810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.292+0000 7fb4cd000640 1 --2- 172.21.15.18:0/3783130414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4c8103900 0x7fb4c810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.292+0000 7fb4cd000640 1 --2- 172.21.15.18:0/3783130414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c8102fe0 0x7fb4c81033c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.292+0000 7fb4cd000640 1 -- 172.21.15.18:0/3783130414 >> 172.21.15.18:0/3783130414 conn(0x7fb4c8077ab0 msgr2=0x7fb4c80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.293+0000 7fb4cd000640 1 -- 172.21.15.18:0/3783130414 shutdown_connections 2024-04-16T17:25:21.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.293+0000 7fb4cd000640 1 -- 172.21.15.18:0/3783130414 wait complete. 2024-04-16T17:25:21.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.294+0000 7fb4cd000640 1 Processor -- start 2024-04-16T17:25:21.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.294+0000 7fb4cd000640 1 -- start start 2024-04-16T17:25:21.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4cd000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4c8102fe0 0x7fb4c8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4cd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4c8103900 0x7fb4c8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4cd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c810c400 0x7fb4c819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4cd000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81121b0 con 0x7fb4c8102fe0 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4cd000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8112030 con 0x7fb4c810c400 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4cd000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8112330 con 0x7fb4c8103900 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4c8102fe0 0x7fb4c8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4c8102fe0 0x7fb4c8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50368/0 (socket says 172.21.15.18:50368) 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.295+0000 7fb4c7400640 1 -- 172.21.15.18:0/186964626 learned_addr learned my addr 172.21.15.18:0/186964626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.297+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c810c400 0x7fb4c819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.298+0000 7fb4c7e00640 1 -- 172.21.15.18:0/186964626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4c8103900 msgr2=0x7fb4c8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.298+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4c8103900 0x7fb4c8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.298+0000 7fb4c7e00640 1 -- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4c8102fe0 msgr2=0x7fb4c8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.298+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4c8102fe0 0x7fb4c8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.298+0000 7fb4c7e00640 1 -- 172.21.15.18:0/186964626 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c819d6d0 con 0x7fb4c810c400 2024-04-16T17:25:21.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.298+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c810c400 0x7fb4c819cfd0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400d6b0 tx=0x7fb4b400db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.299+0000 7fb4bfe00640 1 -- 172.21.15.18:0/186964626 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4017070 con 0x7fb4c810c400 2024-04-16T17:25:21.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.299+0000 7fb4bfe00640 1 -- 172.21.15.18:0/186964626 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b4009820 con 0x7fb4c810c400 2024-04-16T17:25:21.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.299+0000 7fb4bfe00640 1 -- 172.21.15.18:0/186964626 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4004d00 con 0x7fb4c810c400 2024-04-16T17:25:21.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.300+0000 7fb4cd000640 1 -- 172.21.15.18:0/186964626 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c819d9c0 con 0x7fb4c810c400 2024-04-16T17:25:21.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.300+0000 7fb4cd000640 1 -- 172.21.15.18:0/186964626 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c806ba20 con 0x7fb4c810c400 2024-04-16T17:25:21.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.310+0000 7fb4d13cf740 1 Processor -- start 2024-04-16T17:25:21.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.312+0000 7fb4bfe00640 1 -- 172.21.15.18:0/186964626 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb4b40099d0 con 0x7fb4c810c400 2024-04-16T17:25:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.313+0000 7fb4d13cf740 1 -- start start 2024-04-16T17:25:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.315+0000 7fb4bfe00640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4ac0a3ee0 0x7fb4ac0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.316+0000 7fb4bfe00640 1 -- 172.21.15.18:0/186964626 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fb4b40d04c0 con 0x7fb4c810c400 2024-04-16T17:25:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.317+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c06330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.318+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c06870 0x558590c0ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.318+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c11840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.318+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558590b6b020 con 0x558590c05f50 2024-04-16T17:25:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.318+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558590b11460 con 0x558590c0f450 2024-04-16T17:25:21.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.318+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558590b66fe0 con 0x558590c06870 2024-04-16T17:25:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c11840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c11840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59934/0 (socket says 172.21.15.18:59934) 2024-04-16T17:25:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c7e00640 1 -- 172.21.15.18:0/1795676792 learned_addr learned my addr 172.21.15.18:0/1795676792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4ac0a3ee0 0x7fb4ac0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c06870 0x558590c0ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c7400640 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c06330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c7e00640 1 -- 172.21.15.18:0/1795676792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c06870 msgr2=0x558590c0ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c06870 0x558590c0ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c7e00640 1 -- 172.21.15.18:0/1795676792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 msgr2=0x558590c06330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.319+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c06330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.320+0000 7fb4c7e00640 1 -- 172.21.15.18:0/1795676792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558590b19040 con 0x558590c0f450 2024-04-16T17:25:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.320+0000 7fb4c7400640 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c06330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.320+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c11840 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4098400 tx=0x7fb4b409bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd837cfe5114b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.320+0000 7fb4bcc00640 1 -- 172.21.15.18:0/1795676792 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40b5070 con 0x558590c0f450 2024-04-16T17:25:21.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.320+0000 7fb4bcc00640 1 -- 172.21.15.18:0/1795676792 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b409bfa0 con 0x558590c0f450 2024-04-16T17:25:21.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.320+0000 7fb4bcc00640 1 -- 172.21.15.18:0/1795676792 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4020040 con 0x558590c0f450 2024-04-16T17:25:21.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.321+0000 7fb4d13cf740 1 -- 172.21.15.18:0/1795676792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 msgr2=0x558590c11840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.322+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c11840 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4098400 tx=0x7fb4b409bc00 comp rx=0 tx=0).stop 2024-04-16T17:25:21.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.322+0000 7fb4d13cf740 1 -- 172.21.15.18:0/1795676792 shutdown_connections 2024-04-16T17:25:21.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.322+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c11840 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.322+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c06870 0x558590c0ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.322+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/1795676792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c06330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.322+0000 7fb4d13cf740 1 -- 172.21.15.18:0/1795676792 >> 172.21.15.18:0/1795676792 conn(0x558590c00680 msgr2=0x558590c00a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.323+0000 7fb4d13cf740 1 -- 172.21.15.18:0/1795676792 shutdown_connections 2024-04-16T17:25:21.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.323+0000 7fb4d13cf740 1 -- 172.21.15.18:0/1795676792 wait complete. 2024-04-16T17:25:21.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.324+0000 7fb4d13cf740 1 -- start start 2024-04-16T17:25:21.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.324+0000 7fb4d13cf740 1 Processor -- start 2024-04-16T17:25:21.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.325+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4ac0a3ee0 0x7fb4ac0a63a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009460 tx=0x7fb4b8009a10 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.325+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c05f50 0x558590c0edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.325+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.325+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.325+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558590c165f0 con 0x558590c06870 2024-04-16T17:25:21.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.325+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558590b24830 con 0x558590c0f450 2024-04-16T17:25:21.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.325+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558590c16770 con 0x558590c05f50 2024-04-16T17:25:21.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.325+0000 7fb4d13cf740 10 client.0 _ll_register_callbacks cb 0x558590b2c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:21.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c05f50 0x558590c0edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c05f50 0x558590c0edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49364/0 (socket says 172.21.15.18:49364) 2024-04-16T17:25:21.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c7400640 1 -- 172.21.15.18:0/2211082979 learned_addr learned my addr 172.21.15.18:0/2211082979 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c7400640 1 -- 172.21.15.18:0/2211082979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 msgr2=0x558590c086a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:25:21.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c7400640 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c7400640 1 -- 172.21.15.18:0/2211082979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 msgr2=0x558590c08120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c7400640 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c08120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.326+0000 7fb4c7400640 1 -- 172.21.15.18:0/2211082979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558590b28920 con 0x558590c05f50 2024-04-16T17:25:21.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.328+0000 7fb4b1600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:21.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.328+0000 7fb4c7400640 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c05f50 0x558590c0edd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0037040 tx=0x7fb4c0032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a7b992b6734c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.328+0000 7fb4b1600640 1 -- 172.21.15.18:0/2211082979 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0044070 con 0x558590c05f50 2024-04-16T17:25:21.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.328+0000 7fb4d13cf740 1 -- 172.21.15.18:0/2211082979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b409bfa0 con 0x558590c05f50 2024-04-16T17:25:21.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.329+0000 7fb4b1600640 1 -- 172.21.15.18:0/2211082979 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fb4c0003b80 con 0x558590c05f50 2024-04-16T17:25:21.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.329+0000 7fb4b1600640 1 -- 172.21.15.18:0/2211082979 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb4c004e050 con 0x558590c05f50 2024-04-16T17:25:21.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.329+0000 7fb4d13cf740 10 client.30691 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:21.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.329+0000 7fb4d13cf740 1 -- 172.21.15.18:0/2211082979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558590af67f0 con 0x558590c05f50 2024-04-16T17:25:21.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.329+0000 7fb4b1600640 1 -- 172.21.15.18:0/2211082979 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b409bfa0 con 0x558590c05f50 2024-04-16T17:25:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.331+0000 7fb4b1600640 1 -- 172.21.15.18:0/2211082979 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fb4c00328a0 con 0x558590c05f50 2024-04-16T17:25:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.331+0000 7fb4d13cf740 10 client.30691 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.331+0000 7fb4d13cf740 10 client.30691 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:25:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.331+0000 7fb4d13cf740 20 client.30691 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.331+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558590c27dd0 0x558590c2a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.332+0000 7fb4d13cf740 4 client.30691 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.332+0000 7fb4d13cf740 1 -- 172.21.15.18:0/2211082979 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558590b78a40 con 0x558590c27dd0 2024-04-16T17:25:21.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.332+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558590c27dd0 0x558590c2a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.334+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558590c27dd0 0x558590c2a240 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1928f1fd39e892e server_cookie=5e5c2e2f72490f9e in_seq=0 out_seq=0 2024-04-16T17:25:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.335+0000 7fb4b1600640 10 client.30691 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:25:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.336+0000 7fb4b1600640 1 -- 172.21.15.18:0/2211082979 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558590b78a40 con 0x558590c27dd0 2024-04-16T17:25:21.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.336+0000 7fb4b1600640 10 client.30691 handle_command_reply: tid=0 2024-04-16T17:25:21.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.336+0000 7fb4d13cf740 1 client.30691 shutdown 2024-04-16T17:25:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.337+0000 7fb4d13cf740 1 -- 172.21.15.18:0/2211082979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c05f50 msgr2=0x558590c0edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.337+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c05f50 0x558590c0edd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0037040 tx=0x7fb4c0032340 comp rx=0 tx=0).stop 2024-04-16T17:25:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.337+0000 7fb4d13cf740 1 -- 172.21.15.18:0/2211082979 shutdown_connections 2024-04-16T17:25:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.337+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558590c27dd0 0x558590c2a240 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.337+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c0f450 0x558590c086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.337+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c08120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.337+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/2211082979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c05f50 0x558590c0edd0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.337+0000 7fb4d13cf740 1 -- 172.21.15.18:0/2211082979 >> 172.21.15.18:0/2211082979 conn(0x558590c00680 msgr2=0x558590c023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.338+0000 7fb4b1600640 0 client.30691 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:25:21.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.339+0000 7fb4d13cf740 1 -- 172.21.15.18:0/2211082979 shutdown_connections 2024-04-16T17:25:21.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.339+0000 7fb4d13cf740 1 -- 172.21.15.18:0/2211082979 wait complete. 2024-04-16T17:25:21.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.339+0000 7fb4d13cf740 20 client.30691 trim_cache size 0 max 16384 2024-04-16T17:25:21.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.371+0000 7fb4d13cf740 1 Processor -- start 2024-04-16T17:25:21.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.371+0000 7fb4d13cf740 1 -- start start 2024-04-16T17:25:21.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.371+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c05f50 0x558590c0e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.371+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c2fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.371+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.371+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81b8530 con 0x558590c06870 2024-04-16T17:25:21.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.371+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8073da0 con 0x558590c05f50 2024-04-16T17:25:21.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.371+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8071bb0 con 0x558590c0f450 2024-04-16T17:25:21.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.372+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c05f50 0x558590c0e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.372+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c05f50 0x558590c0e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59966/0 (socket says 172.21.15.18:59966) 2024-04-16T17:25:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.372+0000 7fb4c7400640 1 -- 172.21.15.18:0/506888280 learned_addr learned my addr 172.21.15.18:0/506888280 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.372+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.372+0000 7fb4c7400640 1 -- 172.21.15.18:0/506888280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 msgr2=0x558590c30050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.372+0000 7fb4c7400640 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c30050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.372+0000 7fb4c7400640 1 -- 172.21.15.18:0/506888280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 msgr2=0x558590c2fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.373+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c2fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.373+0000 7fb4c7400640 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c2fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.373+0000 7fb4c7400640 1 -- 172.21.15.18:0/506888280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55859087d3f0 con 0x558590c05f50 2024-04-16T17:25:21.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.373+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:21.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.374+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c2fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:21.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.374+0000 7fb4c7400640 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c05f50 0x558590c0e8e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0000f80 tx=0x7fb4c0032eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad2009a77050263 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.375+0000 7fb4b3e00640 1 -- 172.21.15.18:0/506888280 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0057020 con 0x558590c05f50 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.375+0000 7fb4b3e00640 1 -- 172.21.15.18:0/506888280 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c00323b0 con 0x558590c05f50 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.375+0000 7fb4b3e00640 1 -- 172.21.15.18:0/506888280 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0044070 con 0x558590c05f50 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.376+0000 7fb4d13cf740 1 -- 172.21.15.18:0/506888280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c05f50 msgr2=0x558590c0e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.376+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c05f50 0x558590c0e8e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0000f80 tx=0x7fb4c0032eb0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.376+0000 7fb4d13cf740 1 -- 172.21.15.18:0/506888280 shutdown_connections 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.376+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.376+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c06870 0x558590c2fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.376+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/506888280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c05f50 0x558590c0e8e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.376+0000 7fb4d13cf740 1 -- 172.21.15.18:0/506888280 >> 172.21.15.18:0/506888280 conn(0x558590c00680 msgr2=0x558590c00a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.376+0000 7fb4d13cf740 1 -- 172.21.15.18:0/506888280 shutdown_connections 2024-04-16T17:25:21.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.377+0000 7fb4d13cf740 1 -- 172.21.15.18:0/506888280 wait complete. 2024-04-16T17:25:21.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.377+0000 7fb4d13cf740 1 -- start start 2024-04-16T17:25:21.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.377+0000 7fb4d13cf740 1 Processor -- start 2024-04-16T17:25:21.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.379+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c24960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.379+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c06870 0x558590c1fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4d13cf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c20210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8072260 con 0x558590c05f50 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c80736d0 con 0x558590c06870 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4d13cf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8072930 con 0x558590c0f450 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4d13cf740 10 client.0 _ll_register_callbacks cb 0x7fb4b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c24960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c20210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c20210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49388/0 (socket says 172.21.15.18:49388) 2024-04-16T17:25:21.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7e00640 1 -- 172.21.15.18:0/3323900331 learned_addr learned my addr 172.21.15.18:0/3323900331 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c06870 0x558590c1fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7e00640 1 -- 172.21.15.18:0/3323900331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c06870 msgr2=0x558590c1fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c06870 0x558590c1fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7e00640 1 -- 172.21.15.18:0/3323900331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 msgr2=0x558590c24960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c24960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.380+0000 7fb4c7e00640 1 -- 172.21.15.18:0/3323900331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585909477d0 con 0x558590c0f450 2024-04-16T17:25:21.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.381+0000 7fb4c6a00640 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c06870 0x558590c1fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:21.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.381+0000 7fb4c7e00640 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c20210 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400d680 tx=0x7fb4b4098840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd710b85db965a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.381+0000 7fb4b1600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.381+0000 7fb4b1600640 1 -- 172.21.15.18:0/3323900331 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40a4e60 con 0x558590c0f450 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.382+0000 7fb4b1600640 1 -- 172.21.15.18:0/3323900331 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fb4b409b3f0 con 0x558590c0f450 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.382+0000 7fb4d13cf740 1 -- 172.21.15.18:0/3323900331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c00323b0 con 0x558590c0f450 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.382+0000 7fb4b1600640 1 -- 172.21.15.18:0/3323900331 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb4b40b9070 con 0x558590c0f450 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.382+0000 7fb4d13cf740 10 client.30697 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.382+0000 7fb4d13cf740 1 -- 172.21.15.18:0/3323900331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558590c8df50 con 0x558590c0f450 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.382+0000 7fb4b1600640 1 -- 172.21.15.18:0/3323900331 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c00323b0 con 0x558590c0f450 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.383+0000 7fb4b1600640 1 -- 172.21.15.18:0/3323900331 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fb4b4096b10 con 0x558590c0f450 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.383+0000 7fb4d13cf740 10 client.30697 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.383+0000 7fb4d13cf740 10 client.30697 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.383+0000 7fb4d13cf740 20 client.30697 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.383+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558590c8e1e0 0x558590c905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.383+0000 7fb4d13cf740 4 client.30697 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:25:21.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.383+0000 7fb4d13cf740 1 -- 172.21.15.18:0/3323900331 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558590b728b0 con 0x558590c8e1e0 2024-04-16T17:25:21.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.383+0000 7fb4c7400640 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558590c8e1e0 0x558590c905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.384+0000 7fb4b1600640 10 client.30697 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:25:21.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.384+0000 7fb4c7400640 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558590c8e1e0 0x558590c905d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6174bf9fe9cfaed0 server_cookie=7f93f6bb18090fcf in_seq=0 out_seq=0 2024-04-16T17:25:21.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.384+0000 7fb4b1600640 1 -- 172.21.15.18:0/3323900331 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558590b728b0 con 0x558590c8e1e0 2024-04-16T17:25:21.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.384+0000 7fb4b1600640 10 client.30697 handle_command_reply: tid=0 2024-04-16T17:25:21.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.384+0000 7fb4d13cf740 1 client.30697 shutdown 2024-04-16T17:25:21.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4d13cf740 1 -- 172.21.15.18:0/3323900331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 msgr2=0x558590c20210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c20210 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400d680 tx=0x7fb4b4098840 comp rx=0 tx=0).stop 2024-04-16T17:25:21.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4d13cf740 1 -- 172.21.15.18:0/3323900331 shutdown_connections 2024-04-16T17:25:21.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x558590c8e1e0 0x558590c905d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558590c0f450 0x558590c20210 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558590c06870 0x558590c1fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4d13cf740 1 --2- 172.21.15.18:0/3323900331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558590c05f50 0x558590c24960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4b1600640 0 client.30697 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:25:21.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.386+0000 7fb4d13cf740 1 -- 172.21.15.18:0/3323900331 >> 172.21.15.18:0/3323900331 conn(0x558590c00680 msgr2=0x558590c079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.387+0000 7fb4d13cf740 1 -- 172.21.15.18:0/3323900331 shutdown_connections 2024-04-16T17:25:21.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.387+0000 7fb4d13cf740 1 -- 172.21.15.18:0/3323900331 wait complete. 2024-04-16T17:25:21.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.388+0000 7fb4d13cf740 20 client.30697 trim_cache size 0 max 16384 2024-04-16T17:25:21.387 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:25:21.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.389+0000 7fb4cd000640 1 -- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4ac0a3ee0 msgr2=0x7fb4ac0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.389+0000 7fb4cd000640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4ac0a3ee0 0x7fb4ac0a63a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009460 tx=0x7fb4b8009a10 comp rx=0 tx=0).stop 2024-04-16T17:25:21.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.390+0000 7fb4cd000640 1 -- 172.21.15.18:0/186964626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c810c400 msgr2=0x7fb4c819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.390+0000 7fb4cd000640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c810c400 0x7fb4c819cfd0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400d6b0 tx=0x7fb4b400db70 comp rx=0 tx=0).stop 2024-04-16T17:25:21.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.390+0000 7fb4cd000640 1 -- 172.21.15.18:0/186964626 shutdown_connections 2024-04-16T17:25:21.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.390+0000 7fb4cd000640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb4ac0a3ee0 0x7fb4ac0a63a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.390+0000 7fb4cd000640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4c810c400 0x7fb4c819cfd0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.390+0000 7fb4cd000640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4c8103900 0x7fb4c8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.390+0000 7fb4cd000640 1 --2- 172.21.15.18:0/186964626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4c8102fe0 0x7fb4c8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.390+0000 7fb4cd000640 1 -- 172.21.15.18:0/186964626 >> 172.21.15.18:0/186964626 conn(0x7fb4c8077ab0 msgr2=0x7fb4c80fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.391+0000 7fb4cd000640 1 -- 172.21.15.18:0/186964626 shutdown_connections 2024-04-16T17:25:21.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.391+0000 7fb4cd000640 1 -- 172.21.15.18:0/186964626 wait complete. 2024-04-16T17:25:21.418 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:25:21.422 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:20 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1359781629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:25:21.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98598 10.77 MB/sec execute 333 sec latency 2238.235 ms 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.739+0000 7f714d000640 1 -- 172.21.15.18:0/2211929444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7148104450 msgr2=0x7f7148104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.739+0000 7f714d000640 1 --2- 172.21.15.18:0/2211929444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7148104450 0x7f7148104830 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7138009900 tx=0x7f713802bc30 comp rx=0 tx=0).stop 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.740+0000 7f714d000640 1 -- 172.21.15.18:0/2211929444 shutdown_connections 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.740+0000 7f714d000640 1 --2- 172.21.15.18:0/2211929444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7148109520 0x7f71481110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.740+0000 7f714d000640 1 --2- 172.21.15.18:0/2211929444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7148104e00 0x7f7148108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.740+0000 7f714d000640 1 --2- 172.21.15.18:0/2211929444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7148104450 0x7f7148104830 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.740+0000 7f714d000640 1 -- 172.21.15.18:0/2211929444 >> 172.21.15.18:0/2211929444 conn(0x7f71480ffc90 msgr2=0x7f71481020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.740+0000 7f714d000640 1 -- 172.21.15.18:0/2211929444 shutdown_connections 2024-04-16T17:25:21.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.740+0000 7f714d000640 1 -- 172.21.15.18:0/2211929444 wait complete. 2024-04-16T17:25:21.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.741+0000 7f714d000640 1 Processor -- start 2024-04-16T17:25:21.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.741+0000 7f714d000640 1 -- start start 2024-04-16T17:25:21.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f714d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7148104450 0x7f7148110fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f714d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7148109520 0x7f714810e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f714d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714810e5d0 0x7f714810ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f714d000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7148114680 con 0x7f7148104450 2024-04-16T17:25:21.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f714d000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7148114500 con 0x7f7148109520 2024-04-16T17:25:21.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f714d000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7148114800 con 0x7f714810e5d0 2024-04-16T17:25:21.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f7145400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7148109520 0x7f714810e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f7146800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714810e5d0 0x7f714810ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f7145400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7148109520 0x7f714810e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59994/0 (socket says 172.21.15.18:59994) 2024-04-16T17:25:21.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f7145400640 1 -- 172.21.15.18:0/4020563723 learned_addr learned my addr 172.21.15.18:0/4020563723 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.742+0000 7f7146800640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714810e5d0 0x7f714810ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49410/0 (socket says 172.21.15.18:49410) 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.743+0000 7f7145e00640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7148104450 0x7f7148110fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.743+0000 7f7145400640 1 -- 172.21.15.18:0/4020563723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714810e5d0 msgr2=0x7f714810ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.743+0000 7f7145400640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714810e5d0 0x7f714810ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.743+0000 7f7145400640 1 -- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7148104450 msgr2=0x7f7148110fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.743+0000 7f7145400640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7148104450 0x7f7148110fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.743+0000 7f7145400640 1 -- 172.21.15.18:0/4020563723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71481ab5d0 con 0x7f7148109520 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.743+0000 7f7145e00640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7148104450 0x7f7148110fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.743+0000 7f7145400640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7148109520 0x7f714810e090 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f713400e9a0 tx=0x7f713400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.744+0000 7f713ea00640 1 -- 172.21.15.18:0/4020563723 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713400cd80 con 0x7f7148109520 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.744+0000 7f713ea00640 1 -- 172.21.15.18:0/4020563723 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71340035f0 con 0x7f7148109520 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.744+0000 7f713ea00640 1 -- 172.21.15.18:0/4020563723 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134010670 con 0x7f7148109520 2024-04-16T17:25:21.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.744+0000 7f714d000640 1 -- 172.21.15.18:0/4020563723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71481ab8c0 con 0x7f7148109520 2024-04-16T17:25:21.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.744+0000 7f714d000640 1 -- 172.21.15.18:0/4020563723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71481abdc0 con 0x7f7148109520 2024-04-16T17:25:21.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.749+0000 7f714fd76740 1 Processor -- start 2024-04-16T17:25:21.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.749+0000 7f714fd76740 1 -- start start 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.749+0000 7f713ea00640 1 -- 172.21.15.18:0/4020563723 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7134010810 con 0x7f7148109520 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.749+0000 7f714fd76740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f714fd76740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 0x559d20b6bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f714fd76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b6e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f714fd76740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559d20ac8020 con 0x559d20b63670 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f714fd76740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559d20a6e460 con 0x559d20b62d50 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f714fd76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d20ac3fe0 con 0x559d20b6c250 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7146800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7146800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b63130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:59996/0 (socket says 172.21.15.18:59996) 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7146800640 1 -- 172.21.15.18:0/3050027748 learned_addr learned my addr 172.21.15.18:0/3050027748 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7145e00640 1 --2- 172.21.15.18:0/3050027748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 0x559d20b6bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7146800640 1 -- 172.21.15.18:0/3050027748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 msgr2=0x559d20b6e640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:25:21.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7146800640 1 --2- 172.21.15.18:0/3050027748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b6e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7146800640 1 -- 172.21.15.18:0/3050027748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 msgr2=0x559d20b6bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7146800640 1 --2- 172.21.15.18:0/3050027748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 0x559d20b6bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.750+0000 7f7146800640 1 -- 172.21.15.18:0/3050027748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d20a76040 con 0x559d20b62d50 2024-04-16T17:25:21.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.751+0000 7f7146800640 1 --2- 172.21.15.18:0/3050027748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b63130 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7140009f20 tx=0x7f714000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c7c79e2ae5b25f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.751+0000 7f7133400640 1 -- 172.21.15.18:0/3050027748 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714001a070 con 0x559d20b62d50 2024-04-16T17:25:21.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.752+0000 7f7133400640 1 -- 172.21.15.18:0/3050027748 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71400158f0 con 0x559d20b62d50 2024-04-16T17:25:21.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.752+0000 7f713ea00640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f712c0a3e50 0x7f712c0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f714fd76740 1 -- 172.21.15.18:0/3050027748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 msgr2=0x559d20b63130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f714fd76740 1 --2- 172.21.15.18:0/3050027748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b63130 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7140009f20 tx=0x7f714000c8b0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f713ea00640 1 -- 172.21.15.18:0/4020563723 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f71340d0fe0 con 0x7f7148109520 2024-04-16T17:25:21.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f714fd76740 1 -- 172.21.15.18:0/3050027748 shutdown_connections 2024-04-16T17:25:21.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f7145e00640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f712c0a3e50 0x7f712c0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f714fd76740 1 --2- 172.21.15.18:0/3050027748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b6e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f714fd76740 1 --2- 172.21.15.18:0/3050027748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 0x559d20b6bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f714fd76740 1 --2- 172.21.15.18:0/3050027748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b63130 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.753+0000 7f714fd76740 1 -- 172.21.15.18:0/3050027748 >> 172.21.15.18:0/3050027748 conn(0x559d20b5d480 msgr2=0x559d20b5d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.754+0000 7f714fd76740 1 -- 172.21.15.18:0/3050027748 shutdown_connections 2024-04-16T17:25:21.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.754+0000 7f714fd76740 1 -- 172.21.15.18:0/3050027748 wait complete. 2024-04-16T17:25:21.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.755+0000 7f714fd76740 1 -- start start 2024-04-16T17:25:21.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.755+0000 7f7145e00640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f712c0a3e50 0x7f712c0a6310 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7138006010 tx=0x7f713802df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.756+0000 7f714fd76740 1 Processor -- start 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f714fd76740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b69230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f714fd76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b63670 0x559d20b69770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f714fd76740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 0x559d20b69cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f714fd76740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559d20b6f0e0 con 0x559d20b6c250 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f714fd76740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559d20a81830 con 0x559d20b62d50 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f714fd76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d20b6f260 con 0x559d20b63670 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f714fd76740 10 client.0 _ll_register_callbacks cb 0x559d20a89150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f7145e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b69230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f7145e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b69230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60008/0 (socket says 172.21.15.18:60008) 2024-04-16T17:25:21.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f7145e00640 1 -- 172.21.15.18:0/3549483595 learned_addr learned my addr 172.21.15.18:0/3549483595 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.757+0000 7f7145400640 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b63670 0x559d20b69770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.758+0000 7f7145e00640 1 -- 172.21.15.18:0/3549483595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b63670 msgr2=0x559d20b69770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.758+0000 7f7145e00640 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b63670 0x559d20b69770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.758+0000 7f7145e00640 1 -- 172.21.15.18:0/3549483595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 msgr2=0x559d20b69cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.758+0000 7f7145e00640 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 0x559d20b69cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.758+0000 7f7145e00640 1 -- 172.21.15.18:0/3549483595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d20a85920 con 0x559d20b62d50 2024-04-16T17:25:21.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.758+0000 7f7145e00640 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b69230 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7138035000 tx=0x7f713802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=342a62f1c4e328c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.758+0000 7f7127e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:21.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.759+0000 7f7127e00640 1 -- 172.21.15.18:0/3549483595 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713804e030 con 0x559d20b62d50 2024-04-16T17:25:21.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.759+0000 7f7127e00640 1 -- 172.21.15.18:0/3549483595 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f7138005550 con 0x559d20b62d50 2024-04-16T17:25:21.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.759+0000 7f714fd76740 1 -- 172.21.15.18:0/3549483595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71400158f0 con 0x559d20b62d50 2024-04-16T17:25:21.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.760+0000 7f7127e00640 1 -- 172.21.15.18:0/3549483595 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f71380046c0 con 0x559d20b62d50 2024-04-16T17:25:21.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.760+0000 7f714fd76740 10 client.30860 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:21.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.760+0000 7f714fd76740 1 -- 172.21.15.18:0/3549483595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559d20a537f0 con 0x559d20b62d50 2024-04-16T17:25:21.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.760+0000 7f7127e00640 1 -- 172.21.15.18:0/3549483595 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71400158f0 con 0x559d20b62d50 2024-04-16T17:25:21.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.761+0000 7f7127e00640 1 -- 172.21.15.18:0/3549483595 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f7138003bc0 con 0x559d20b62d50 2024-04-16T17:25:21.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.761+0000 7f714fd76740 10 client.30860 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:21.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.761+0000 7f714fd76740 10 client.30860 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:25:21.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.761+0000 7f714fd76740 20 client.30860 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.761+0000 7f714fd76740 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559d20b7c6c0 0x559d20b7eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.761+0000 7f714fd76740 4 client.30860 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.761+0000 7f714fd76740 1 -- 172.21.15.18:0/3549483595 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d20ad5a40 con 0x559d20b7c6c0 2024-04-16T17:25:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.762+0000 7f7145400640 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559d20b7c6c0 0x559d20b7eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.771+0000 7f7145400640 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559d20b7c6c0 0x559d20b7eaf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=290cf5f57f9c66df server_cookie=6f26f71111b5271 in_seq=0 out_seq=0 2024-04-16T17:25:21.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.771+0000 7f7127e00640 10 client.30860 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:25:21.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.778+0000 7f7127e00640 1 -- 172.21.15.18:0/3549483595 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d20ad5a40 con 0x559d20b7c6c0 2024-04-16T17:25:21.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.778+0000 7f7127e00640 10 client.30860 handle_command_reply: tid=0 2024-04-16T17:25:21.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 client.30860 shutdown 2024-04-16T17:25:21.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 -- 172.21.15.18:0/3549483595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 msgr2=0x559d20b69230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b69230 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7138035000 tx=0x7f713802bc30 comp rx=0 tx=0).stop 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 -- 172.21.15.18:0/3549483595 shutdown_connections 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559d20b7c6c0 0x559d20b7eaf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 0x559d20b69cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b63670 0x559d20b69770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 --2- 172.21.15.18:0/3549483595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b69230 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f7127e00640 0 client.30860 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.779+0000 7f714fd76740 1 -- 172.21.15.18:0/3549483595 >> 172.21.15.18:0/3549483595 conn(0x559d20b5d480 msgr2=0x559d20b5f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.780+0000 7f714fd76740 1 -- 172.21.15.18:0/3549483595 shutdown_connections 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.780+0000 7f714fd76740 1 -- 172.21.15.18:0/3549483595 wait complete. 2024-04-16T17:25:21.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.780+0000 7f714fd76740 20 client.30860 trim_cache size 0 max 16384 2024-04-16T17:25:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.813+0000 7f714fd76740 1 Processor -- start 2024-04-16T17:25:21.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.815+0000 7f714fd76740 1 -- start start 2024-04-16T17:25:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.815+0000 7f714fd76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b62d50 0x559d20b6b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.815+0000 7f714fd76740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b63670 0x559d20b77ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.815+0000 7f714fd76740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 0x559d20b5efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.816+0000 7f7145e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b62d50 0x559d20b6b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.816+0000 7f7145400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b63670 0x559d20b77ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.816+0000 7f7145400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b63670 0x559d20b77ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60022/0 (socket says 172.21.15.18:60022) 2024-04-16T17:25:21.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.816+0000 7f714fd76740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b8830 con 0x559d20b6c250 2024-04-16T17:25:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.816+0000 7f714fd76740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b6100 con 0x559d20b63670 2024-04-16T17:25:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.816+0000 7f714fd76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b3f70 con 0x559d20b62d50 2024-04-16T17:25:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.816+0000 7f7145400640 1 -- 172.21.15.18:0/1964408921 learned_addr learned my addr 172.21.15.18:0/1964408921 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.816+0000 7f7146800640 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 0x559d20b5efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.817+0000 7f7145e00640 1 -- 172.21.15.18:0/1964408921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b63670 msgr2=0x559d20b77ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.817+0000 7f7145e00640 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b63670 0x559d20b77ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.817+0000 7f7145e00640 1 -- 172.21.15.18:0/1964408921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 msgr2=0x559d20b5efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.817+0000 7f7145e00640 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 0x559d20b5efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.817+0000 7f7145e00640 1 -- 172.21.15.18:0/1964408921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d207da3f0 con 0x559d20b62d50 2024-04-16T17:25:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.817+0000 7f7146800640 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 0x559d20b5efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.818+0000 7f7145e00640 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b62d50 0x559d20b6b960 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7138009900 tx=0x7f7138004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7e4c9f29b9dc90 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.819+0000 7f7132a00640 1 -- 172.21.15.18:0/1964408921 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713804e070 con 0x559d20b62d50 2024-04-16T17:25:21.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.819+0000 7f7132a00640 1 -- 172.21.15.18:0/1964408921 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7138061070 con 0x559d20b62d50 2024-04-16T17:25:21.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.819+0000 7f7132a00640 1 -- 172.21.15.18:0/1964408921 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71380038c0 con 0x559d20b62d50 2024-04-16T17:25:21.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.820+0000 7f714fd76740 1 -- 172.21.15.18:0/1964408921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b62d50 msgr2=0x559d20b6b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.820+0000 7f714fd76740 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b62d50 0x559d20b6b960 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7138009900 tx=0x7f7138004170 comp rx=0 tx=0).stop 2024-04-16T17:25:21.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.821+0000 7f714fd76740 1 -- 172.21.15.18:0/1964408921 shutdown_connections 2024-04-16T17:25:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.821+0000 7f714fd76740 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b6c250 0x559d20b5efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.821+0000 7f714fd76740 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b63670 0x559d20b77ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.821+0000 7f714fd76740 1 --2- 172.21.15.18:0/1964408921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b62d50 0x559d20b6b960 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.821+0000 7f714fd76740 1 -- 172.21.15.18:0/1964408921 >> 172.21.15.18:0/1964408921 conn(0x559d20b5d480 msgr2=0x559d20b5e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.821+0000 7f714fd76740 1 -- 172.21.15.18:0/1964408921 shutdown_connections 2024-04-16T17:25:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.821+0000 7f714fd76740 1 -- 172.21.15.18:0/1964408921 wait complete. 2024-04-16T17:25:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.821+0000 7f714fd76740 1 -- start start 2024-04-16T17:25:21.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.822+0000 7f714fd76740 1 Processor -- start 2024-04-16T17:25:21.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.822+0000 7f714fd76740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b6ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.822+0000 7f714fd76740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 0x559d20b704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.822+0000 7f714fd76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b70a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f7145e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b6ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f714fd76740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b45c0 con 0x559d20b63670 2024-04-16T17:25:21.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f714fd76740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b5a30 con 0x559d20b62d50 2024-04-16T17:25:21.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f714fd76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b4c90 con 0x559d20b6c250 2024-04-16T17:25:21.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f714fd76740 10 client.0 _ll_register_callbacks cb 0x7f7138042050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f7145e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b6ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60028/0 (socket says 172.21.15.18:60028) 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f7146800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b70a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f7146800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b70a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49430/0 (socket says 172.21.15.18:49430) 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f7145400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 0x559d20b704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.823+0000 7f7146800640 1 -- 172.21.15.18:0/3024057725 learned_addr learned my addr 172.21.15.18:0/3024057725 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.824+0000 7f7146800640 1 -- 172.21.15.18:0/3024057725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 msgr2=0x559d20b6ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.824+0000 7f7146800640 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b6ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.824+0000 7f7146800640 1 -- 172.21.15.18:0/3024057725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 msgr2=0x559d20b704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.824+0000 7f7146800640 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 0x559d20b704f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.824+0000 7f7146800640 1 -- 172.21.15.18:0/3024057725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d208a47d0 con 0x559d20b6c250 2024-04-16T17:25:21.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.824+0000 7f7145e00640 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b6ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:21.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.824+0000 7f7146800640 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b70a30 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7140017000 tx=0x7f71400096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7121383cb394b32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:21.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.824+0000 7f7127e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:21.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.825+0000 7f7127e00640 1 -- 172.21.15.18:0/3024057725 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714001a040 con 0x559d20b6c250 2024-04-16T17:25:21.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.826+0000 7f7127e00640 1 -- 172.21.15.18:0/3024057725 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f714000acb0 con 0x559d20b6c250 2024-04-16T17:25:21.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.826+0000 7f714fd76740 1 -- 172.21.15.18:0/3024057725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7138061070 con 0x559d20b6c250 2024-04-16T17:25:21.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.826+0000 7f7127e00640 1 -- 172.21.15.18:0/3024057725 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f714000a500 con 0x559d20b6c250 2024-04-16T17:25:21.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.826+0000 7f714fd76740 10 client.30712 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:21.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.826+0000 7f714fd76740 1 -- 172.21.15.18:0/3024057725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559d20be2dd0 con 0x559d20b6c250 2024-04-16T17:25:21.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.827+0000 7f7127e00640 1 -- 172.21.15.18:0/3024057725 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7138061070 con 0x559d20b6c250 2024-04-16T17:25:21.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.827+0000 7f7127e00640 1 -- 172.21.15.18:0/3024057725 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f7140032020 con 0x559d20b6c250 2024-04-16T17:25:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.827+0000 7f714fd76740 10 client.30712 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.828+0000 7f714fd76740 10 client.30712 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:25:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.828+0000 7f714fd76740 20 client.30712 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.828+0000 7f714fd76740 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559d20be3210 0x559d20be5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.828+0000 7f714fd76740 4 client.30712 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:25:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.828+0000 7f714fd76740 1 -- 172.21.15.18:0/3024057725 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d20acf8b0 con 0x559d20be3210 2024-04-16T17:25:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.828+0000 7f7145e00640 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559d20be3210 0x559d20be5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.830+0000 7f7145e00640 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559d20be3210 0x559d20be5600 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=843c7895ab31aee6 server_cookie=1333b81b04295e27 in_seq=0 out_seq=0 2024-04-16T17:25:21.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.830+0000 7f7127e00640 10 client.30712 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:25:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.831+0000 7f7127e00640 1 -- 172.21.15.18:0/3024057725 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d20acf8b0 con 0x559d20be3210 2024-04-16T17:25:21.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.831+0000 7f7127e00640 10 client.30712 handle_command_reply: tid=0 2024-04-16T17:25:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 client.30712 shutdown 2024-04-16T17:25:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 -- 172.21.15.18:0/3024057725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 msgr2=0x559d20b70a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b70a30 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7140017000 tx=0x7f71400096b0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 -- 172.21.15.18:0/3024057725 shutdown_connections 2024-04-16T17:25:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x559d20be3210 0x559d20be5600 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f7127e00640 0 client.30712 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:25:21.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d20b6c250 0x559d20b70a30 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559d20b63670 0x559d20b704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 --2- 172.21.15.18:0/3024057725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d20b62d50 0x559d20b6ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.832+0000 7f714fd76740 1 -- 172.21.15.18:0/3024057725 >> 172.21.15.18:0/3024057725 conn(0x559d20b5d480 msgr2=0x559d20b5f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.833+0000 7f714fd76740 1 -- 172.21.15.18:0/3024057725 shutdown_connections 2024-04-16T17:25:21.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.833+0000 7f714fd76740 1 -- 172.21.15.18:0/3024057725 wait complete. 2024-04-16T17:25:21.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.833+0000 7f714fd76740 20 client.30712 trim_cache size 0 max 16384 2024-04-16T17:25:21.833 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:25:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 -- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f712c0a3e50 msgr2=0x7f712c0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f712c0a3e50 0x7f712c0a6310 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7138006010 tx=0x7f713802df90 comp rx=0 tx=0).stop 2024-04-16T17:25:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 -- 172.21.15.18:0/4020563723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7148109520 msgr2=0x7f714810e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7148109520 0x7f714810e090 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f713400e9a0 tx=0x7f713400ee60 comp rx=0 tx=0).stop 2024-04-16T17:25:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 -- 172.21.15.18:0/4020563723 shutdown_connections 2024-04-16T17:25:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f712c0a3e50 0x7f712c0a6310 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714810e5d0 0x7f714810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7148109520 0x7f714810e090 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.834+0000 7f714d000640 1 --2- 172.21.15.18:0/4020563723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7148104450 0x7f7148110fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.835+0000 7f714d000640 1 -- 172.21.15.18:0/4020563723 >> 172.21.15.18:0/4020563723 conn(0x7f71480ffc90 msgr2=0x7f7148101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.836+0000 7f714d000640 1 -- 172.21.15.18:0/4020563723 shutdown_connections 2024-04-16T17:25:21.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:21.836+0000 7f714d000640 1 -- 172.21.15.18:0/4020563723 wait complete. 2024-04-16T17:25:21.865 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:25:22.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.191+0000 7fd29ca00640 1 -- 172.21.15.18:0/1828326824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd298074cc0 msgr2=0x7fd298112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.191+0000 7fd29ca00640 1 --2- 172.21.15.18:0/1828326824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd298074cc0 0x7fd298112aa0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd28802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.193+0000 7fd29ca00640 1 -- 172.21.15.18:0/1828326824 shutdown_connections 2024-04-16T17:25:22.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.193+0000 7fd29ca00640 1 --2- 172.21.15.18:0/1828326824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd298112fe0 0x7fd2981153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.193+0000 7fd29ca00640 1 --2- 172.21.15.18:0/1828326824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd298074cc0 0x7fd298112aa0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.193+0000 7fd29ca00640 1 --2- 172.21.15.18:0/1828326824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2980743a0 0x7fd298074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.193+0000 7fd29ca00640 1 -- 172.21.15.18:0/1828326824 >> 172.21.15.18:0/1828326824 conn(0x7fd2980fdac0 msgr2=0x7fd2980ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:22.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.194+0000 7fd29ca00640 1 -- 172.21.15.18:0/1828326824 shutdown_connections 2024-04-16T17:25:22.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.194+0000 7fd29ca00640 1 -- 172.21.15.18:0/1828326824 wait complete. 2024-04-16T17:25:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.196+0000 7fd29ca00640 1 Processor -- start 2024-04-16T17:25:22.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.196+0000 7fd29ca00640 1 -- start start 2024-04-16T17:25:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.197+0000 7fd29ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2980743a0 0x7fd298106910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.197+0000 7fd29ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd298074cc0 0x7fd298106e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.197+0000 7fd29ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd298112fe0 0x7fd298107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.197+0000 7fd295e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2980743a0 0x7fd298106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.197+0000 7fd295e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2980743a0 0x7fd298106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60036/0 (socket says 172.21.15.18:60036) 2024-04-16T17:25:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd295e00640 1 -- 172.21.15.18:0/2894522995 learned_addr learned my addr 172.21.15.18:0/2894522995 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd298118a30 con 0x7fd298074cc0 2024-04-16T17:25:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981188b0 con 0x7fd2980743a0 2024-04-16T17:25:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd298118bb0 con 0x7fd298112fe0 2024-04-16T17:25:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd296800640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd298112fe0 0x7fd298107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd295400640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd298074cc0 0x7fd298106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd295e00640 1 -- 172.21.15.18:0/2894522995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd298112fe0 msgr2=0x7fd298107390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd295e00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd298112fe0 0x7fd298107390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd295e00640 1 -- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd298074cc0 msgr2=0x7fd298106e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd295e00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd298074cc0 0x7fd298106e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.198+0000 7fd295e00640 1 -- 172.21.15.18:0/2894522995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2981b39d0 con 0x7fd2980743a0 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.199+0000 7fd295400640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd298074cc0 0x7fd298106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.199+0000 7fd295e00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2980743a0 0x7fd298106910 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd29000eab0 tx=0x7fd29000ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.199+0000 7fd28ea00640 1 -- 172.21.15.18:0/2894522995 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29000ce40 con 0x7fd2980743a0 2024-04-16T17:25:22.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.199+0000 7fd28ea00640 1 -- 172.21.15.18:0/2894522995 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd290011950 con 0x7fd2980743a0 2024-04-16T17:25:22.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.199+0000 7fd28ea00640 1 -- 172.21.15.18:0/2894522995 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290010b40 con 0x7fd2980743a0 2024-04-16T17:25:22.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.199+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2981b3cc0 con 0x7fd2980743a0 2024-04-16T17:25:22.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.200+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2981b41c0 con 0x7fd2980743a0 2024-04-16T17:25:22.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.204+0000 7fd29f64c740 1 Processor -- start 2024-04-16T17:25:22.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.205+0000 7fd29f64c740 1 -- start start 2024-04-16T17:25:22.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.205+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d237432130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23743ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23743d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd29f64c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d237397020 con 0x55d237431d50 2024-04-16T17:25:22.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd29f64c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d23733d460 con 0x55d237432670 2024-04-16T17:25:22.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd29f64c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d237392fe0 con 0x55d23743b250 2024-04-16T17:25:22.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd28ea00640 1 -- 172.21.15.18:0/2894522995 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd290011b00 con 0x7fd2980743a0 2024-04-16T17:25:22.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd295e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23743ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd295e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23743ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60046/0 (socket says 172.21.15.18:60046) 2024-04-16T17:25:22.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.206+0000 7fd295e00640 1 -- 172.21.15.18:0/408581450 learned_addr learned my addr 172.21.15.18:0/408581450 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:22.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.207+0000 7fd295400640 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23743d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.207+0000 7fd296800640 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d237432130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.207+0000 7fd295e00640 1 -- 172.21.15.18:0/408581450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 msgr2=0x55d23743d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:22.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.207+0000 7fd295e00640 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23743d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.207+0000 7fd295e00640 1 -- 172.21.15.18:0/408581450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 msgr2=0x55d237432130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:22.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.207+0000 7fd295e00640 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d237432130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.207+0000 7fd295e00640 1 -- 172.21.15.18:0/408581450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d237345040 con 0x55d237432670 2024-04-16T17:25:22.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.207+0000 7fd296800640 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d237432130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:22.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.208+0000 7fd295e00640 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23743ad10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd2900cd820 tx=0x7fd2900d6bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5da4dcaa70b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:22.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.208+0000 7fd287000640 1 -- 172.21.15.18:0/408581450 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900d0c90 con 0x55d237432670 2024-04-16T17:25:22.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.209+0000 7fd287000640 1 -- 172.21.15.18:0/408581450 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2900d0e30 con 0x55d237432670 2024-04-16T17:25:22.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.209+0000 7fd287000640 1 -- 172.21.15.18:0/408581450 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900d2750 con 0x55d237432670 2024-04-16T17:25:22.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.209+0000 7fd28ea00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd27c0a3e60 0x7fd27c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.211+0000 7fd28ea00640 1 -- 172.21.15.18:0/2894522995 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fd2900d0650 con 0x7fd2980743a0 2024-04-16T17:25:22.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.211+0000 7fd296800640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd27c0a3e60 0x7fd27c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.211+0000 7fd29f64c740 1 -- 172.21.15.18:0/408581450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 msgr2=0x55d23743ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.211+0000 7fd29f64c740 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23743ad10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd2900cd820 tx=0x7fd2900d6bb0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.211+0000 7fd29f64c740 1 -- 172.21.15.18:0/408581450 shutdown_connections 2024-04-16T17:25:22.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.212+0000 7fd29f64c740 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23743d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.212+0000 7fd29f64c740 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23743ad10 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.212+0000 7fd29f64c740 1 --2- 172.21.15.18:0/408581450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d237432130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.212+0000 7fd29f64c740 1 -- 172.21.15.18:0/408581450 >> 172.21.15.18:0/408581450 conn(0x55d23742c480 msgr2=0x55d23742c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:22.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.212+0000 7fd29f64c740 1 -- 172.21.15.18:0/408581450 shutdown_connections 2024-04-16T17:25:22.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.212+0000 7fd29f64c740 1 -- 172.21.15.18:0/408581450 wait complete. 2024-04-16T17:25:22.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.212+0000 7fd296800640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd27c0a3e60 0x7fd27c0a6320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd29810ea50 tx=0x7fd280005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:22.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.213+0000 7fd29f64c740 1 -- start start 2024-04-16T17:25:22.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.213+0000 7fd29f64c740 1 Processor -- start 2024-04-16T17:25:22.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237431d50 0x55d237438290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237432670 0x55d2374387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d237438d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd29f64c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d23743e040 con 0x55d237432670 2024-04-16T17:25:22.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd29f64c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d237350830 con 0x55d237431d50 2024-04-16T17:25:22.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd29f64c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d23743e1c0 con 0x55d23743b250 2024-04-16T17:25:22.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd29f64c740 10 client.0 _ll_register_callbacks cb 0x55d237358150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:22.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd295e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237431d50 0x55d237438290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd295e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237431d50 0x55d237438290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60056/0 (socket says 172.21.15.18:60056) 2024-04-16T17:25:22.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd295e00640 1 -- 172.21.15.18:0/1525834731 learned_addr learned my addr 172.21.15.18:0/1525834731 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:22.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.214+0000 7fd296800640 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d237438d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.215+0000 7fd295e00640 1 -- 172.21.15.18:0/1525834731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 msgr2=0x55d237438d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:22.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.215+0000 7fd295e00640 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d237438d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.215+0000 7fd295e00640 1 -- 172.21.15.18:0/1525834731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237432670 msgr2=0x55d2374387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:22.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.215+0000 7fd295e00640 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237432670 0x55d2374387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.215+0000 7fd295e00640 1 -- 172.21.15.18:0/1525834731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d237354920 con 0x55d237431d50 2024-04-16T17:25:22.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.215+0000 7fd295e00640 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237431d50 0x55d237438290 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd290098b00 tx=0x7fd2900d3f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed84ab468c025ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:22.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.215+0000 7fd27b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:22.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.218+0000 7fd27b400640 1 -- 172.21.15.18:0/1525834731 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900983d0 con 0x55d237431d50 2024-04-16T17:25:22.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.218+0000 7fd27b400640 1 -- 172.21.15.18:0/1525834731 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fd2900987a0 con 0x55d237431d50 2024-04-16T17:25:22.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.218+0000 7fd29f64c740 1 -- 172.21.15.18:0/1525834731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2900d0e30 con 0x55d237431d50 2024-04-16T17:25:22.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.218+0000 7fd27b400640 1 -- 172.21.15.18:0/1525834731 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd2900d0c90 con 0x55d237431d50 2024-04-16T17:25:22.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.218+0000 7fd29f64c740 10 client.30878 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:22.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.219+0000 7fd29f64c740 1 -- 172.21.15.18:0/1525834731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d2373227f0 con 0x55d237431d50 2024-04-16T17:25:22.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.219+0000 7fd27b400640 1 -- 172.21.15.18:0/1525834731 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2900d0e30 con 0x55d237431d50 2024-04-16T17:25:22.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.219+0000 7fd27b400640 1 -- 172.21.15.18:0/1525834731 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fd2900bc040 con 0x55d237431d50 2024-04-16T17:25:22.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.220+0000 7fd29f64c740 10 client.30878 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:22.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.220+0000 7fd29f64c740 10 client.30878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:22.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.220+0000 7fd29f64c740 20 client.30878 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:22.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.220+0000 7fd29f64c740 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d23744b4b0 0x55d23744d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.220+0000 7fd29f64c740 4 client.30878 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:22.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.220+0000 7fd29f64c740 1 -- 172.21.15.18:0/1525834731 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2373a4a40 con 0x55d23744b4b0 2024-04-16T17:25:22.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.220+0000 7fd295400640 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d23744b4b0 0x55d23744d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.227+0000 7fd295400640 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d23744b4b0 0x55d23744d8a0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae212f4950d4776 server_cookie=953c2caeee44dc35 in_seq=0 out_seq=0 2024-04-16T17:25:22.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.227+0000 7fd27b400640 10 client.30878 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:22.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.239+0000 7fd27b400640 1 -- 172.21.15.18:0/1525834731 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2373a4a40 con 0x55d23744b4b0 2024-04-16T17:25:22.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.239+0000 7fd27b400640 10 client.30878 handle_command_reply: tid=0 2024-04-16T17:25:22.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.241+0000 7fd29f64c740 1 client.30878 shutdown 2024-04-16T17:25:22.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.241+0000 7fd29f64c740 1 -- 172.21.15.18:0/1525834731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237431d50 msgr2=0x55d237438290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.241+0000 7fd29f64c740 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237431d50 0x55d237438290 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd290098b00 tx=0x7fd2900d3f60 comp rx=0 tx=0).stop 2024-04-16T17:25:22.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.241+0000 7fd29f64c740 1 -- 172.21.15.18:0/1525834731 shutdown_connections 2024-04-16T17:25:22.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd29f64c740 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d23744b4b0 0x55d23744d8a0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd29f64c740 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d237438d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd29f64c740 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237432670 0x55d2374387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd29f64c740 1 --2- 172.21.15.18:0/1525834731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237431d50 0x55d237438290 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd27b400640 0 client.30878 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:22.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd29f64c740 1 -- 172.21.15.18:0/1525834731 >> 172.21.15.18:0/1525834731 conn(0x55d23742c480 msgr2=0x55d23742e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:22.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd29f64c740 1 -- 172.21.15.18:0/1525834731 shutdown_connections 2024-04-16T17:25:22.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd29f64c740 1 -- 172.21.15.18:0/1525834731 wait complete. 2024-04-16T17:25:22.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.242+0000 7fd29f64c740 20 client.30878 trim_cache size 0 max 16384 2024-04-16T17:25:22.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:21 smithi043 ceph-mon[36901]: pgmap v659: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 423 op/s 2024-04-16T17:25:22.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.272+0000 7fd29f64c740 1 Processor -- start 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.272+0000 7fd29f64c740 1 -- start start 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.272+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d23743a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.272+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d237432670 0x55d2374508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.272+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23743b250 0x55d237450e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.273+0000 7fd29f64c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981c1050 con 0x55d237431d50 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.273+0000 7fd29f64c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981be920 con 0x55d23743b250 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.273+0000 7fd29f64c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981bc790 con 0x55d237432670 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.273+0000 7fd295400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d237432670 0x55d2374508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.273+0000 7fd295400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d237432670 0x55d2374508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49482/0 (socket says 172.21.15.18:49482) 2024-04-16T17:25:22.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.273+0000 7fd295400640 1 -- 172.21.15.18:0/4246807051 learned_addr learned my addr 172.21.15.18:0/4246807051 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:22.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.273+0000 7fd295e00640 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d23743a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.274+0000 7fd296800640 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23743b250 0x55d237450e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.274+0000 7fd295400640 1 -- 172.21.15.18:0/4246807051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23743b250 msgr2=0x55d237450e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:22.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.274+0000 7fd295400640 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23743b250 0x55d237450e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.274+0000 7fd295400640 1 -- 172.21.15.18:0/4246807051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 msgr2=0x55d23743a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:22.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.274+0000 7fd295400640 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d23743a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.274+0000 7fd295400640 1 -- 172.21.15.18:0/4246807051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2370a93f0 con 0x55d237432670 2024-04-16T17:25:22.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.274+0000 7fd295400640 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d237432670 0x55d2374508c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd288005ab0 tx=0x7fd288000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e89ed40685c81a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:22.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.275+0000 7fd286600640 1 -- 172.21.15.18:0/4246807051 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288049070 con 0x55d237432670 2024-04-16T17:25:22.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.275+0000 7fd286600640 1 -- 172.21.15.18:0/4246807051 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd28802ce00 con 0x55d237432670 2024-04-16T17:25:22.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.275+0000 7fd286600640 1 -- 172.21.15.18:0/4246807051 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880441d0 con 0x55d237432670 2024-04-16T17:25:22.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.276+0000 7fd29f64c740 1 -- 172.21.15.18:0/4246807051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d237432670 msgr2=0x55d2374508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.276+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d237432670 0x55d2374508c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd288005ab0 tx=0x7fd288000bc0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.276+0000 7fd29f64c740 1 -- 172.21.15.18:0/4246807051 shutdown_connections 2024-04-16T17:25:22.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.276+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23743b250 0x55d237450e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.276+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d237432670 0x55d2374508c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.276+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4246807051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d23743a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.276+0000 7fd29f64c740 1 -- 172.21.15.18:0/4246807051 >> 172.21.15.18:0/4246807051 conn(0x55d23742c480 msgr2=0x55d237446cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:22.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.277+0000 7fd29f64c740 1 -- 172.21.15.18:0/4246807051 shutdown_connections 2024-04-16T17:25:22.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.277+0000 7fd29f64c740 1 -- 172.21.15.18:0/4246807051 wait complete. 2024-04-16T17:25:22.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.277+0000 7fd29f64c740 1 -- start start 2024-04-16T17:25:22.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.278+0000 7fd29f64c740 1 Processor -- start 2024-04-16T17:25:22.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.278+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d2374509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.278+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23744dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.278+0000 7fd29f64c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23744e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.278+0000 7fd29f64c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981bcde0 con 0x55d237431d50 2024-04-16T17:25:22.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.278+0000 7fd29f64c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981be250 con 0x55d237432670 2024-04-16T17:25:22.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.278+0000 7fd29f64c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981bd4b0 con 0x55d23743b250 2024-04-16T17:25:22.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.279+0000 7fd29f64c740 10 client.0 _ll_register_callbacks cb 0x7fd280005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:22.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.279+0000 7fd296800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23744e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.279+0000 7fd296800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23744e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49498/0 (socket says 172.21.15.18:49498) 2024-04-16T17:25:22.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.279+0000 7fd296800640 1 -- 172.21.15.18:0/4254828146 learned_addr learned my addr 172.21.15.18:0/4254828146 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:22.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.280+0000 7fd296800640 1 -- 172.21.15.18:0/4254828146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 msgr2=0x55d23744dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:22.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.280+0000 7fd296800640 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23744dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.280+0000 7fd296800640 1 -- 172.21.15.18:0/4254828146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 msgr2=0x55d2374509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:22.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.280+0000 7fd296800640 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d2374509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.280+0000 7fd296800640 1 -- 172.21.15.18:0/4254828146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2371737d0 con 0x55d23743b250 2024-04-16T17:25:22.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.280+0000 7fd296800640 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23744e3e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd2800045f0 tx=0x7fd2800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=995a617494013205 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:22.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.280+0000 7fd27b400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:22.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.281+0000 7fd27b400640 1 -- 172.21.15.18:0/4254828146 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28002e060 con 0x55d23743b250 2024-04-16T17:25:22.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.281+0000 7fd27b400640 1 -- 172.21.15.18:0/4254828146 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fd28001f640 con 0x55d23743b250 2024-04-16T17:25:22.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.281+0000 7fd29f64c740 1 -- 172.21.15.18:0/4254828146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd28802ce00 con 0x55d23743b250 2024-04-16T17:25:22.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.281+0000 7fd27b400640 1 -- 172.21.15.18:0/4254828146 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd28001f840 con 0x55d23743b250 2024-04-16T17:25:22.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.281+0000 7fd29f64c740 10 client.30730 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:22.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.281+0000 7fd29f64c740 1 -- 172.21.15.18:0/4254828146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d2374b12a0 con 0x55d23743b250 2024-04-16T17:25:22.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.282+0000 7fd27b400640 1 -- 172.21.15.18:0/4254828146 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd28802ce00 con 0x55d23743b250 2024-04-16T17:25:22.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.282+0000 7fd27b400640 1 -- 172.21.15.18:0/4254828146 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fd280029ae0 con 0x55d23743b250 2024-04-16T17:25:22.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.282+0000 7fd29f64c740 10 client.30730 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:22.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.282+0000 7fd29f64c740 10 client.30730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:22.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.282+0000 7fd29f64c740 20 client.30730 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:22.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.282+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d2374b1a80 0x55d2374b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:22.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.282+0000 7fd29f64c740 4 client.30730 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:25:22.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.282+0000 7fd29f64c740 1 -- 172.21.15.18:0/4254828146 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d23739e8b0 con 0x55d2374b1a80 2024-04-16T17:25:22.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.283+0000 7fd295e00640 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d2374b1a80 0x55d2374b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:22.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.284+0000 7fd295e00640 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d2374b1a80 0x55d2374b3f20 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e1c65dafc94193 server_cookie=942ebadc3f233cc8 in_seq=0 out_seq=0 2024-04-16T17:25:22.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.284+0000 7fd27b400640 10 client.30730 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:22.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.284+0000 7fd27b400640 1 -- 172.21.15.18:0/4254828146 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d23739e8b0 con 0x55d2374b1a80 2024-04-16T17:25:22.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.285+0000 7fd27b400640 10 client.30730 handle_command_reply: tid=0 2024-04-16T17:25:22.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.285+0000 7fd29f64c740 1 client.30730 shutdown 2024-04-16T17:25:22.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.285+0000 7fd29f64c740 1 -- 172.21.15.18:0/4254828146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 msgr2=0x55d23744e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.285+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23744e3e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd2800045f0 tx=0x7fd2800046c0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd29f64c740 1 -- 172.21.15.18:0/4254828146 shutdown_connections 2024-04-16T17:25:22.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d2374b1a80 0x55d2374b3f20 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d23743b250 0x55d23744e3e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d237432670 0x55d23744dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd29f64c740 1 --2- 172.21.15.18:0/4254828146 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d237431d50 0x55d2374509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd29f64c740 1 -- 172.21.15.18:0/4254828146 >> 172.21.15.18:0/4254828146 conn(0x55d23742c480 msgr2=0x55d23742ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:22.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd27b400640 0 client.30730 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:22.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd29f64c740 1 -- 172.21.15.18:0/4254828146 shutdown_connections 2024-04-16T17:25:22.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.286+0000 7fd29f64c740 1 -- 172.21.15.18:0/4254828146 wait complete. 2024-04-16T17:25:22.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.287+0000 7fd29f64c740 20 client.30730 trim_cache size 0 max 16384 2024-04-16T17:25:22.286 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:25:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.288+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd27c0a3e60 msgr2=0x7fd27c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.288+0000 7fd29ca00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd27c0a3e60 0x7fd27c0a6320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd29810ea50 tx=0x7fd280005f90 comp rx=0 tx=0).stop 2024-04-16T17:25:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2980743a0 msgr2=0x7fd298106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2980743a0 0x7fd298106910 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd29000eab0 tx=0x7fd29000ef70 comp rx=0 tx=0).stop 2024-04-16T17:25:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 shutdown_connections 2024-04-16T17:25:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd27c0a3e60 0x7fd27c0a6320 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd298112fe0 0x7fd298107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd298074cc0 0x7fd298106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 --2- 172.21.15.18:0/2894522995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2980743a0 0x7fd298106910 secure :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd29000eab0 tx=0x7fd29000ef70 comp rx=0 tx=0).stop 2024-04-16T17:25:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 >> 172.21.15.18:0/2894522995 conn(0x7fd2980fdac0 msgr2=0x7fd2980ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.289+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 shutdown_connections 2024-04-16T17:25:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:22.290+0000 7fd29ca00640 1 -- 172.21.15.18:0/2894522995 wait complete. 2024-04-16T17:25:22.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:21 smithi022 ceph-mon[36023]: pgmap v659: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 423 op/s 2024-04-16T17:25:22.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98691 10.75 MB/sec execute 334 sec latency 2024.893 ms 2024-04-16T17:25:23.322 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:25:23.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:25:23.322 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:25:23.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:20 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1359781629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:25:23.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:21 smithi018 ceph-mon[31776]: pgmap v659: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 423 op/s 2024-04-16T17:25:23.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98803 10.74 MB/sec execute 335 sec latency 1653.603 ms 2024-04-16T17:25:23.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.653+0000 7f05bc800640 1 -- 172.21.15.18:0/2340034737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b4104e00 msgr2=0x7f05b4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.653+0000 7f05bc800640 1 --2- 172.21.15.18:0/2340034737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b4104e00 0x7f05b4108e40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009900 tx=0x7f05a402bc30 comp rx=0 tx=0).stop 2024-04-16T17:25:23.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.653+0000 7f05bc800640 1 -- 172.21.15.18:0/2340034737 shutdown_connections 2024-04-16T17:25:23.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.653+0000 7f05bc800640 1 --2- 172.21.15.18:0/2340034737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b4109520 0x7f05b41110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.653+0000 7f05bc800640 1 --2- 172.21.15.18:0/2340034737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b4104e00 0x7f05b4108e40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.653+0000 7f05bc800640 1 --2- 172.21.15.18:0/2340034737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b4104450 0x7f05b4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.654+0000 7f05bc800640 1 -- 172.21.15.18:0/2340034737 >> 172.21.15.18:0/2340034737 conn(0x7f05b40ffc90 msgr2=0x7f05b41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:23.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.654+0000 7f05bc800640 1 -- 172.21.15.18:0/2340034737 shutdown_connections 2024-04-16T17:25:23.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.654+0000 7f05bc800640 1 -- 172.21.15.18:0/2340034737 wait complete. 2024-04-16T17:25:23.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.655+0000 7f05bc800640 1 Processor -- start 2024-04-16T17:25:23.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.656+0000 7f05bc800640 1 -- start start 2024-04-16T17:25:23.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.656+0000 7f05bc800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b4104450 0x7f05b4076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.656+0000 7f05bc800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b4109520 0x7f05b4076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.656+0000 7f05bc800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b4076f90 0x7f05b41ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.656+0000 7f05bc800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f05b41145e0 con 0x7f05b4104450 2024-04-16T17:25:23.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.656+0000 7f05bc800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4114460 con 0x7f05b4076f90 2024-04-16T17:25:23.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.656+0000 7f05bc800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4114760 con 0x7f05b4109520 2024-04-16T17:25:23.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05ba800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b4076f90 0x7f05b41ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05ba800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b4076f90 0x7f05b41ab0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60102/0 (socket says 172.21.15.18:60102) 2024-04-16T17:25:23.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05ba800640 1 -- 172.21.15.18:0/1857193057 learned_addr learned my addr 172.21.15.18:0/1857193057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05b9400640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b4109520 0x7f05b4076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05b9e00640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b4104450 0x7f05b4076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05ba800640 1 -- 172.21.15.18:0/1857193057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b4109520 msgr2=0x7f05b4076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05ba800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b4109520 0x7f05b4076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05ba800640 1 -- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b4104450 msgr2=0x7f05b4076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05ba800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b4104450 0x7f05b4076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.657+0000 7f05ba800640 1 -- 172.21.15.18:0/1857193057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b41ab5f0 con 0x7f05b4076f90 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.658+0000 7f05ba800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b4076f90 0x7f05b41ab0b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f05b000eb10 tx=0x7f05b000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.658+0000 7f05aea00640 1 -- 172.21.15.18:0/1857193057 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0004b20 con 0x7f05b4076f90 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.658+0000 7f05aea00640 1 -- 172.21.15.18:0/1857193057 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b0004cc0 con 0x7f05b4076f90 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.658+0000 7f05aea00640 1 -- 172.21.15.18:0/1857193057 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0010760 con 0x7f05b4076f90 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.658+0000 7f05bc800640 1 -- 172.21.15.18:0/1857193057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b41ab8e0 con 0x7f05b4076f90 2024-04-16T17:25:23.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.659+0000 7f05bc800640 1 -- 172.21.15.18:0/1857193057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b41abdb0 con 0x7f05b4076f90 2024-04-16T17:25:23.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.664+0000 7f05bf568740 1 Processor -- start 2024-04-16T17:25:23.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.667+0000 7f05aea00640 1 -- 172.21.15.18:0/1857193057 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05b0005020 con 0x7f05b4076f90 2024-04-16T17:25:23.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.669+0000 7f05aea00640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f059c0a3ea0 0x7f059c0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.669+0000 7f05aea00640 1 -- 172.21.15.18:0/1857193057 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f05b00d1130 con 0x7f05b4076f90 2024-04-16T17:25:23.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.670+0000 7f05bf568740 1 -- start start 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05bf568740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce0cca0 0x560cbce0d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05bf568740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce15c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05bf568740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce161a0 0x560cbce18590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05bf568740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560cbcd71cb0 con 0x560cbce161a0 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05bf568740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560cbcd18370 con 0x560cbce0cca0 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05bf568740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560cbcd6dc70 con 0x560cbce0d5c0 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce15c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce15c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49532/0 (socket says 172.21.15.18:49532) 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.671+0000 7f05b9400640 1 -- 172.21.15.18:0/799622405 learned_addr learned my addr 172.21.15.18:0/799622405 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:23.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.672+0000 7f05ba800640 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce161a0 0x560cbce18590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.672+0000 7f05b9400640 1 -- 172.21.15.18:0/799622405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce0cca0 msgr2=0x560cbce0d080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:25:23.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.672+0000 7f05b9400640 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce0cca0 0x560cbce0d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.672+0000 7f05b9400640 1 -- 172.21.15.18:0/799622405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce161a0 msgr2=0x560cbce18590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:23.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.672+0000 7f05b9400640 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce161a0 0x560cbce18590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.672+0000 7f05b9400640 1 -- 172.21.15.18:0/799622405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cbcd37910 con 0x560cbce0d5c0 2024-04-16T17:25:23.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.672+0000 7f05ba800640 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce161a0 0x560cbce18590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:23.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.673+0000 7f05b9400640 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce15c60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009900 tx=0x7f05a402c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=932e2d22ac7415fc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:23.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.673+0000 7f05b9400640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f059c0a3ea0 0x7f059c0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.673+0000 7f05a3000640 1 -- 172.21.15.18:0/799622405 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4044070 con 0x560cbce0d5c0 2024-04-16T17:25:23.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.673+0000 7f05a3000640 1 -- 172.21.15.18:0/799622405 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a402c880 con 0x560cbce0d5c0 2024-04-16T17:25:23.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.673+0000 7f05a3000640 1 -- 172.21.15.18:0/799622405 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4004920 con 0x560cbce0d5c0 2024-04-16T17:25:23.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.674+0000 7f05bf568740 1 -- 172.21.15.18:0/799622405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 msgr2=0x560cbce15c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.674+0000 7f05bf568740 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce15c60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009900 tx=0x7f05a402c760 comp rx=0 tx=0).stop 2024-04-16T17:25:23.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.675+0000 7f05bf568740 1 -- 172.21.15.18:0/799622405 shutdown_connections 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.675+0000 7f05bf568740 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce161a0 0x560cbce18590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.675+0000 7f05bf568740 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce15c60 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.675+0000 7f05bf568740 1 --2- 172.21.15.18:0/799622405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce0cca0 0x560cbce0d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.675+0000 7f05bf568740 1 -- 172.21.15.18:0/799622405 >> 172.21.15.18:0/799622405 conn(0x560cbce073d0 msgr2=0x560cbce077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.675+0000 7f05b9400640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f059c0a3ea0 0x7f059c0a6360 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f05a402c210 tx=0x7f05a404b040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.675+0000 7f05bf568740 1 -- 172.21.15.18:0/799622405 shutdown_connections 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.675+0000 7f05bf568740 1 -- 172.21.15.18:0/799622405 wait complete. 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.676+0000 7f05bf568740 1 -- start start 2024-04-16T17:25:23.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.676+0000 7f05bf568740 1 Processor -- start 2024-04-16T17:25:23.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.677+0000 7f05bf568740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce13190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.677+0000 7f05bf568740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.677+0000 7f05bf568740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce13c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.677+0000 7f05b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce136d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49540/0 (socket says 172.21.15.18:49540) 2024-04-16T17:25:23.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05b9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce13190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05b9e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce13190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50600/0 (socket says 172.21.15.18:50600) 2024-04-16T17:25:23.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05bf568740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560cbce29f40 con 0x560cbce0cca0 2024-04-16T17:25:23.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05bf568740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560cbcd2b730 con 0x560cbce161a0 2024-04-16T17:25:23.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05bf568740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560cbce2a0c0 con 0x560cbce0d5c0 2024-04-16T17:25:23.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05bf568740 10 client.0 _ll_register_callbacks cb 0x560cbccfa1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:23.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05b9e00640 1 -- 172.21.15.18:0/448366726 learned_addr learned my addr 172.21.15.18:0/448366726 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.678+0000 7f05ba800640 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce13c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.679+0000 7f05b9e00640 1 -- 172.21.15.18:0/448366726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 msgr2=0x560cbce136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.679+0000 7f05b9e00640 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.679+0000 7f05b9e00640 1 -- 172.21.15.18:0/448366726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 msgr2=0x560cbce13c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.679+0000 7f05b9e00640 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce13c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.679+0000 7f05b9e00640 1 -- 172.21.15.18:0/448366726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cbcb4c7e0 con 0x560cbce0cca0 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.679+0000 7f05b9e00640 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce13190 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f05a800d640 tx=0x7f05a800db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e726e4f5b86b8eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.679+0000 7f0597400640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:25:23.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.680+0000 7f0597400640 1 -- 172.21.15.18:0/448366726 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a800f8f0 con 0x560cbce0cca0 2024-04-16T17:25:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.680+0000 7f0597400640 1 -- 172.21.15.18:0/448366726 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f05a800fa90 con 0x560cbce0cca0 2024-04-16T17:25:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.680+0000 7f05bf568740 1 -- 172.21.15.18:0/448366726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a402c880 con 0x560cbce0cca0 2024-04-16T17:25:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.681+0000 7f0597400640 1 -- 172.21.15.18:0/448366726 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f05a800fc90 con 0x560cbce0cca0 2024-04-16T17:25:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.681+0000 7f05bf568740 10 client.19914 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.681+0000 7f05bf568740 1 -- 172.21.15.18:0/448366726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560cbcbacf70 con 0x560cbce0cca0 2024-04-16T17:25:23.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.682+0000 7f0597400640 1 -- 172.21.15.18:0/448366726 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a402c880 con 0x560cbce0cca0 2024-04-16T17:25:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.683+0000 7f0597400640 1 -- 172.21.15.18:0/448366726 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f05a80204f0 con 0x560cbce0cca0 2024-04-16T17:25:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.683+0000 7f05bf568740 10 client.19914 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.683+0000 7f05bf568740 10 client.19914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.683+0000 7f05bf568740 20 client.19914 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.683+0000 7f05bf568740 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cbce374e0 0x560cbce398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.683+0000 7f05bf568740 4 client.19914 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.683+0000 7f05bf568740 1 -- 172.21.15.18:0/448366726 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cbcd7f6d0 con 0x560cbce374e0 2024-04-16T17:25:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.684+0000 7f05b9400640 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cbce374e0 0x560cbce398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.684+0000 7f05b9400640 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cbce374e0 0x560cbce398d0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724f8420bb9ad20a server_cookie=874296edac7d7f70 in_seq=0 out_seq=0 2024-04-16T17:25:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.684+0000 7f0597400640 10 client.19914 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.685+0000 7f0597400640 1 -- 172.21.15.18:0/448366726 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cbcd7f6d0 con 0x560cbce374e0 2024-04-16T17:25:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.685+0000 7f0597400640 10 client.19914 handle_command_reply: tid=0 2024-04-16T17:25:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.685+0000 7f05bf568740 1 client.19914 shutdown 2024-04-16T17:25:23.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.686+0000 7f05bf568740 1 -- 172.21.15.18:0/448366726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 msgr2=0x560cbce13190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.686+0000 7f05bf568740 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce13190 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f05a800d640 tx=0x7f05a800db00 comp rx=0 tx=0).stop 2024-04-16T17:25:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.687+0000 7f05bf568740 1 -- 172.21.15.18:0/448366726 shutdown_connections 2024-04-16T17:25:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.687+0000 7f05bf568740 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cbce374e0 0x560cbce398d0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.687+0000 7f05bf568740 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce13c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.687+0000 7f05bf568740 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.687+0000 7f05bf568740 1 --2- 172.21.15.18:0/448366726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce13190 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.687+0000 7f0597400640 0 client.19914 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.687+0000 7f05bf568740 1 -- 172.21.15.18:0/448366726 >> 172.21.15.18:0/448366726 conn(0x560cbce073d0 msgr2=0x560cbce09120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.687+0000 7f05bf568740 1 -- 172.21.15.18:0/448366726 shutdown_connections 2024-04-16T17:25:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.688+0000 7f05bf568740 1 -- 172.21.15.18:0/448366726 wait complete. 2024-04-16T17:25:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.688+0000 7f05bf568740 20 client.19914 trim_cache size 0 max 16384 2024-04-16T17:25:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.724+0000 7f05bf568740 1 Processor -- start 2024-04-16T17:25:23.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.724+0000 7f05bf568740 1 -- start start 2024-04-16T17:25:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.725+0000 7f05bf568740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0cca0 0x560cbce158c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.725+0000 7f05bf568740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0d5c0 0x560cbce3f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.725+0000 7f05bf568740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce3f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.725+0000 7f05bf568740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f05b41afe30 con 0x560cbce0d5c0 2024-04-16T17:25:23.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.725+0000 7f05bf568740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05b41ad700 con 0x560cbce161a0 2024-04-16T17:25:23.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.725+0000 7f05bf568740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4073700 con 0x560cbce0cca0 2024-04-16T17:25:23.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.726+0000 7f05b9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0cca0 0x560cbce158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.726+0000 7f05b9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0cca0 0x560cbce158c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49554/0 (socket says 172.21.15.18:49554) 2024-04-16T17:25:23.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.726+0000 7f05b9e00640 1 -- 172.21.15.18:0/2039146405 learned_addr learned my addr 172.21.15.18:0/2039146405 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:23.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.726+0000 7f05b9400640 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0d5c0 0x560cbce3f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.726+0000 7f05ba800640 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce3f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.727+0000 7f05b9e00640 1 -- 172.21.15.18:0/2039146405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 msgr2=0x560cbce3f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:23.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.727+0000 7f05b9e00640 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce3f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.727+0000 7f05b9e00640 1 -- 172.21.15.18:0/2039146405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0d5c0 msgr2=0x560cbce3f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:23.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.727+0000 7f05b9e00640 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0d5c0 0x560cbce3f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.727+0000 7f05b9e00640 1 -- 172.21.15.18:0/2039146405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cbca44170 con 0x560cbce0cca0 2024-04-16T17:25:23.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.727+0000 7f05ba800640 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce3f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:23.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.727+0000 7f05b9e00640 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0cca0 0x560cbce158c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f05a8001eb0 tx=0x7f05a800bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9df85119bb87fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:23.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.728+0000 7f05a2600640 1 -- 172.21.15.18:0/2039146405 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8005b20 con 0x560cbce0cca0 2024-04-16T17:25:23.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.728+0000 7f05a2600640 1 -- 172.21.15.18:0/2039146405 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a800b3b0 con 0x560cbce0cca0 2024-04-16T17:25:23.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.728+0000 7f05a2600640 1 -- 172.21.15.18:0/2039146405 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8005280 con 0x560cbce0cca0 2024-04-16T17:25:23.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.729+0000 7f05bf568740 1 -- 172.21.15.18:0/2039146405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0cca0 msgr2=0x560cbce158c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.729+0000 7f05bf568740 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0cca0 0x560cbce158c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f05a8001eb0 tx=0x7f05a800bed0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.729+0000 7f05bf568740 1 -- 172.21.15.18:0/2039146405 shutdown_connections 2024-04-16T17:25:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.729+0000 7f05bf568740 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce3f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.729+0000 7f05bf568740 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0d5c0 0x560cbce3f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.729+0000 7f05bf568740 1 --2- 172.21.15.18:0/2039146405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0cca0 0x560cbce158c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.730+0000 7f05bf568740 1 -- 172.21.15.18:0/2039146405 >> 172.21.15.18:0/2039146405 conn(0x560cbce073d0 msgr2=0x560cbce05ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.730+0000 7f05bf568740 1 -- 172.21.15.18:0/2039146405 shutdown_connections 2024-04-16T17:25:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.730+0000 7f05bf568740 1 -- 172.21.15.18:0/2039146405 wait complete. 2024-04-16T17:25:23.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.731+0000 7f05bf568740 1 -- start start 2024-04-16T17:25:23.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.731+0000 7f05bf568740 1 Processor -- start 2024-04-16T17:25:23.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.733+0000 7f05bf568740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce2cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.733+0000 7f05bf568740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce2d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.733+0000 7f05bf568740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce2d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05bf568740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4073d50 con 0x560cbce0cca0 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05bf568740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05b41ad030 con 0x560cbce161a0 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05bf568740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05b41ac290 con 0x560cbce0d5c0 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05ba800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05ba800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce2da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60134/0 (socket says 172.21.15.18:60134) 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05ba800640 1 -- 172.21.15.18:0/2284131171 learned_addr learned my addr 172.21.15.18:0/2284131171 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05bf568740 10 client.0 _ll_register_callbacks cb 0x7f05a800bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05ba800640 1 -- 172.21.15.18:0/2284131171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 msgr2=0x560cbce2d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:23.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05ba800640 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce2d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05ba800640 1 -- 172.21.15.18:0/2284131171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 msgr2=0x560cbce2cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:23.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05b9e00640 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce2cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05ba800640 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce2cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.734+0000 7f05b9400640 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce2d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:23.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.735+0000 7f05ba800640 1 -- 172.21.15.18:0/2284131171 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cbce9cbd0 con 0x560cbce161a0 2024-04-16T17:25:23.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.735+0000 7f05b9e00640 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce2cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:23.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.735+0000 7f05ba800640 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce2da00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f05b0099640 tx=0x7f05b009c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2feb5f574c0f471 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:23.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.735+0000 7f0597400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:23.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.736+0000 7f0597400640 1 -- 172.21.15.18:0/2284131171 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b009d0d0 con 0x560cbce161a0 2024-04-16T17:25:23.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.736+0000 7f0597400640 1 -- 172.21.15.18:0/2284131171 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f05b00a5450 con 0x560cbce161a0 2024-04-16T17:25:23.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.736+0000 7f05bf568740 1 -- 172.21.15.18:0/2284131171 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a800b3b0 con 0x560cbce161a0 2024-04-16T17:25:23.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.736+0000 7f0597400640 1 -- 172.21.15.18:0/2284131171 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f05b009cc70 con 0x560cbce161a0 2024-04-16T17:25:23.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.736+0000 7f05bf568740 10 client.30893 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:23.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.737+0000 7f05bf568740 1 -- 172.21.15.18:0/2284131171 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560cbce9df50 con 0x560cbce161a0 2024-04-16T17:25:23.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.737+0000 7f0597400640 1 -- 172.21.15.18:0/2284131171 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a800b3b0 con 0x560cbce161a0 2024-04-16T17:25:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.738+0000 7f0597400640 1 -- 172.21.15.18:0/2284131171 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f05b00ceb70 con 0x560cbce161a0 2024-04-16T17:25:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.738+0000 7f05bf568740 10 client.30893 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.738+0000 7f05bf568740 10 client.30893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.738+0000 7f05bf568740 20 client.30893 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.738+0000 7f05bf568740 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cbce9e5a0 0x560cbcea0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:23.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.738+0000 7f05bf568740 4 client.30893 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:25:23.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.738+0000 7f05bf568740 1 -- 172.21.15.18:0/2284131171 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560cbcd79540 con 0x560cbce9e5a0 2024-04-16T17:25:23.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.739+0000 7f05b9e00640 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cbce9e5a0 0x560cbcea0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:23.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.740+0000 7f05b9e00640 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cbce9e5a0 0x560cbcea0a40 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6844aeedc95fcc59 server_cookie=b9e7d6f560068f59 in_seq=0 out_seq=0 2024-04-16T17:25:23.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.740+0000 7f0597400640 10 client.30893 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:23.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f0597400640 1 -- 172.21.15.18:0/2284131171 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560cbcd79540 con 0x560cbce9e5a0 2024-04-16T17:25:23.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f0597400640 10 client.30893 handle_command_reply: tid=0 2024-04-16T17:25:23.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 client.30893 shutdown 2024-04-16T17:25:23.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 -- 172.21.15.18:0/2284131171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 msgr2=0x560cbce2da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce2da00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f05b0099640 tx=0x7f05b009c7b0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 -- 172.21.15.18:0/2284131171 shutdown_connections 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560cbce9e5a0 0x560cbcea0a40 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560cbce161a0 0x560cbce2da00 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cbce0d5c0 0x560cbce2d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 --2- 172.21.15.18:0/2284131171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560cbce0cca0 0x560cbce2cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 -- 172.21.15.18:0/2284131171 >> 172.21.15.18:0/2284131171 conn(0x560cbce073d0 msgr2=0x560cbce06430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f0597400640 0 client.30893 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.742+0000 7f05bf568740 1 -- 172.21.15.18:0/2284131171 shutdown_connections 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.743+0000 7f05bf568740 1 -- 172.21.15.18:0/2284131171 wait complete. 2024-04-16T17:25:23.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.743+0000 7f05bf568740 20 client.30893 trim_cache size 0 max 16384 2024-04-16T17:25:23.743 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:25:23.743 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:25:23.743 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "5f0e024d-a0b6-449b-87e5-08a3461de802", 2024-04-16T17:25:23.743 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:25:23.743 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 -- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f059c0a3ea0 msgr2=0x7f059c0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f059c0a3ea0 0x7f059c0a6360 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f05a402c210 tx=0x7f05a404b040 comp rx=0 tx=0).stop 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 -- 172.21.15.18:0/1857193057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b4076f90 msgr2=0x7f05b41ab0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b4076f90 0x7f05b41ab0b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f05b000eb10 tx=0x7f05b000c2d0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 -- 172.21.15.18:0/1857193057 shutdown_connections 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f059c0a3ea0 0x7f059c0a6360 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b4076f90 0x7f05b41ab0b0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b4109520 0x7f05b4076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.744+0000 7f05bc800640 1 --2- 172.21.15.18:0/1857193057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b4104450 0x7f05b4076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:23.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.745+0000 7f05bc800640 1 -- 172.21.15.18:0/1857193057 >> 172.21.15.18:0/1857193057 conn(0x7f05b40ffc90 msgr2=0x7f05b4102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:23.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.745+0000 7f05bc800640 1 -- 172.21.15.18:0/1857193057 shutdown_connections 2024-04-16T17:25:23.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:23.748+0000 7f05bc800640 1 -- 172.21.15.18:0/1857193057 wait complete. 2024-04-16T17:25:23.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:25:23.773 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:25:24.039 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:23 smithi018 ceph-mon[31776]: pgmap v660: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 341 op/s 2024-04-16T17:25:24.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 -- 172.21.15.18:0/102230957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bb4102be0 msgr2=0x7f0bb4106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 --2- 172.21.15.18:0/102230957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bb4102be0 0x7f0bb4106c20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009900 tx=0x7f0ba402bc30 comp rx=0 tx=0).stop 2024-04-16T17:25:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 -- 172.21.15.18:0/102230957 shutdown_connections 2024-04-16T17:25:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 --2- 172.21.15.18:0/102230957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bb4107300 0x7f0bb410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 --2- 172.21.15.18:0/102230957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bb4102be0 0x7f0bb4106c20 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 --2- 172.21.15.18:0/102230957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bb4102230 0x7f0bb4102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 -- 172.21.15.18:0/102230957 >> 172.21.15.18:0/102230957 conn(0x7f0bb40fdab0 msgr2=0x7f0bb40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 -- 172.21.15.18:0/102230957 shutdown_connections 2024-04-16T17:25:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.177+0000 7f0bbc800640 1 -- 172.21.15.18:0/102230957 wait complete. 2024-04-16T17:25:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.179+0000 7f0bbc800640 1 Processor -- start 2024-04-16T17:25:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bbc800640 1 -- start start 2024-04-16T17:25:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bbc800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bb4102230 0x7f0bb4116540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bbc800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bb4107300 0x7f0bb4116a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bbc800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bb41116a0 0x7f0bb4111b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bbc800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb40756e0 con 0x7f0bb41116a0 2024-04-16T17:25:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bbc800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4075560 con 0x7f0bb4107300 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bbc800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4075860 con 0x7f0bb4102230 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bb4107300 0x7f0bb4116a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bb4107300 0x7f0bb4116a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60160/0 (socket says 172.21.15.18:60160) 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.180+0000 7f0bb9400640 1 -- 172.21.15.18:0/3535269981 learned_addr learned my addr 172.21.15.18:0/3535269981 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.181+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bb4102230 0x7f0bb4116540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.181+0000 7f0bb9400640 1 -- 172.21.15.18:0/3535269981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bb4102230 msgr2=0x7f0bb4116540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.181+0000 7f0bb9400640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bb4102230 0x7f0bb4116540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.181+0000 7f0bb9400640 1 -- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bb41116a0 msgr2=0x7f0bb4111b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.181+0000 7f0bb9400640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bb41116a0 0x7f0bb4111b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.181+0000 7f0bb9400640 1 -- 172.21.15.18:0/3535269981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb4112410 con 0x7f0bb4107300 2024-04-16T17:25:24.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.181+0000 7f0bb9400640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bb4107300 0x7f0bb4116a80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4036040 tx=0x7f0ba4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:24.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.182+0000 7f0baea00640 1 -- 172.21.15.18:0/3535269981 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4041070 con 0x7f0bb4107300 2024-04-16T17:25:24.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.182+0000 7f0bbc800640 1 -- 172.21.15.18:0/3535269981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb411c250 con 0x7f0bb4107300 2024-04-16T17:25:24.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.182+0000 7f0bbc800640 1 -- 172.21.15.18:0/3535269981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb411c7f0 con 0x7f0bb4107300 2024-04-16T17:25:24.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.182+0000 7f0baea00640 1 -- 172.21.15.18:0/3535269981 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba40054e0 con 0x7f0bb4107300 2024-04-16T17:25:24.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.185+0000 7f0baea00640 1 -- 172.21.15.18:0/3535269981 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4041070 con 0x7f0bb4107300 2024-04-16T17:25:24.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.192+0000 7f0bbf423740 1 Processor -- start 2024-04-16T17:25:24.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.193+0000 7f0baea00640 1 -- 172.21.15.18:0/3535269981 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ba4044070 con 0x7f0bb4107300 2024-04-16T17:25:24.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.193+0000 7f0bbf423740 1 -- start start 2024-04-16T17:25:24.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.193+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a49810f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498118170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bbf423740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a498075120 con 0x55a49810f1b0 2024-04-16T17:25:24.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bbf423740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a49801b340 con 0x55a49810fad0 2024-04-16T17:25:24.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bbf423740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a4980710e0 con 0x55a4981186b0 2024-04-16T17:25:24.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bb9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498118170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bb9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498118170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60162/0 (socket says 172.21.15.18:60162) 2024-04-16T17:25:24.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.194+0000 7f0bb9e00640 1 -- 172.21.15.18:0/2609359033 learned_addr learned my addr 172.21.15.18:0/2609359033 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:24.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.195+0000 7f0bb9e00640 1 -- 172.21.15.18:0/2609359033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 msgr2=0x55a49811aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:24.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.195+0000 7f0bba800640 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a49810f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.195+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.195+0000 7f0bb9e00640 1 -- 172.21.15.18:0/2609359033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 msgr2=0x55a49810f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.195+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a49810f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.195+0000 7f0bb9e00640 1 -- 172.21.15.18:0/2609359033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a497e517d0 con 0x55a49810fad0 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.196+0000 7f0bb9400640 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.196+0000 7f0bba800640 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a49810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.196+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498118170 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009a70 tx=0x7f0ba8009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a219fa8926b93a11 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.196+0000 7f0ba3000640 1 -- 172.21.15.18:0/2609359033 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8017070 con 0x55a49810fad0 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.196+0000 7f0ba3000640 1 -- 172.21.15.18:0/2609359033 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba8013470 con 0x55a49810fad0 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.196+0000 7f0ba3000640 1 -- 172.21.15.18:0/2609359033 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8012630 con 0x55a49810fad0 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.197+0000 7f0baea00640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b9c0a3f20 0x7f0b9c0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.197+0000 7f0baea00640 1 -- 172.21.15.18:0/3535269981 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f0ba40f1970 con 0x7f0bb4107300 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.197+0000 7f0bbf423740 1 -- 172.21.15.18:0/2609359033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 msgr2=0x55a498118170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.197+0000 7f0bbf423740 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498118170 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009a70 tx=0x7f0ba8009f30 comp rx=0 tx=0).stop 2024-04-16T17:25:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.198+0000 7f0bbf423740 1 -- 172.21.15.18:0/2609359033 shutdown_connections 2024-04-16T17:25:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.198+0000 7f0bbf423740 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.198+0000 7f0bbf423740 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498118170 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.198+0000 7f0bbf423740 1 --2- 172.21.15.18:0/2609359033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a49810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.198+0000 7f0bbf423740 1 -- 172.21.15.18:0/2609359033 >> 172.21.15.18:0/2609359033 conn(0x55a4981098e0 msgr2=0x55a498109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.198+0000 7f0bbf423740 1 -- 172.21.15.18:0/2609359033 shutdown_connections 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.198+0000 7f0bbf423740 1 -- 172.21.15.18:0/2609359033 wait complete. 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.199+0000 7f0bbf423740 1 -- start start 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.199+0000 7f0bbf423740 1 Processor -- start 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.200+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810f1b0 0x55a4981156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.201+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810fad0 0x55a498115c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.201+0000 7f0bb9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810f1b0 0x55a4981156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.201+0000 7f0bb9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810f1b0 0x55a4981156e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60178/0 (socket says 172.21.15.18:60178) 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.201+0000 7f0bb9e00640 1 -- 172.21.15.18:0/3405199057 learned_addr learned my addr 172.21.15.18:0/3405199057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.201+0000 7f0bbf423740 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498116160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.201+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a49811b4a0 con 0x55a49810fad0 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a49802e710 con 0x55a49810f1b0 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bba800640 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a49811b620 con 0x55a4981186b0 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bbf423740 10 client.0 _ll_register_callbacks cb 0x55a498029af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bb9e00640 1 -- 172.21.15.18:0/3405199057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 msgr2=0x55a498116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498116160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bb9e00640 1 -- 172.21.15.18:0/3405199057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810fad0 msgr2=0x55a498115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810fad0 0x55a498115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.202+0000 7f0bb9e00640 1 -- 172.21.15.18:0/3405199057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a497d873f0 con 0x55a49810f1b0 2024-04-16T17:25:24.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.203+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810f1b0 0x55a4981156e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8013f30 tx=0x7f0ba80097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2742d3b08bd084b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:24.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.203+0000 7f0b97400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:24.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.204+0000 7f0b97400640 1 -- 172.21.15.18:0/3405199057 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8017050 con 0x55a49810f1b0 2024-04-16T17:25:24.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.204+0000 7f0b97400640 1 -- 172.21.15.18:0/3405199057 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f0ba801c450 con 0x55a49810f1b0 2024-04-16T17:25:24.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.204+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba8013470 con 0x55a49810f1b0 2024-04-16T17:25:24.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.205+0000 7f0b97400640 1 -- 172.21.15.18:0/3405199057 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0ba801c670 con 0x55a49810f1b0 2024-04-16T17:25:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.205+0000 7f0bbf423740 10 client.30905 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.205+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a497d470f0 con 0x55a49810f1b0 2024-04-16T17:25:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.205+0000 7f0b97400640 1 -- 172.21.15.18:0/3405199057 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba8013470 con 0x55a49810f1b0 2024-04-16T17:25:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.205+0000 7f0bba800640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b9c0a3f20 0x7f0b9c0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.206+0000 7f0b97400640 1 -- 172.21.15.18:0/3405199057 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f0ba8030910 con 0x55a49810f1b0 2024-04-16T17:25:24.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.207+0000 7f0bbf423740 10 client.30905 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:24.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.207+0000 7f0bbf423740 10 client.30905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:24.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.207+0000 7f0bbf423740 20 client.30905 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:24.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.207+0000 7f0bbf423740 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a498128d00 0x55a49812b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.207+0000 7f0bbf423740 4 client.30905 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:24.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.207+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a498082b40 con 0x55a498128d00 2024-04-16T17:25:24.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.207+0000 7f0bb9400640 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a498128d00 0x55a49812b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.208+0000 7f0b97400640 10 client.30905 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:24.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.208+0000 7f0bb9400640 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a498128d00 0x55a49812b150 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4e9cd372fc5f02 server_cookie=e2c80132e670eada in_seq=0 out_seq=0 2024-04-16T17:25:24.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.209+0000 7f0b97400640 1 -- 172.21.15.18:0/3405199057 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a498082b40 con 0x55a498128d00 2024-04-16T17:25:24.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.209+0000 7f0b97400640 10 client.30905 handle_command_reply: tid=0 2024-04-16T17:25:24.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.209+0000 7f0bbf423740 1 client.30905 shutdown 2024-04-16T17:25:24.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810f1b0 msgr2=0x55a4981156e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0bbf423740 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810f1b0 0x55a4981156e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8013f30 tx=0x7f0ba80097f0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 shutdown_connections 2024-04-16T17:25:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0bbf423740 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a498128d00 0x55a49812b150 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0b97400640 0 client.30905 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0bbf423740 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0bbf423740 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810fad0 0x55a498115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0bbf423740 1 --2- 172.21.15.18:0/3405199057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810f1b0 0x55a4981156e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.210+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 >> 172.21.15.18:0/3405199057 conn(0x55a4981098e0 msgr2=0x55a49810b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.211+0000 7f0bba800640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b9c0a3f20 0x7f0b9c0a63e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0005c40 tx=0x7f0bb0004a40 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.211+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 shutdown_connections 2024-04-16T17:25:24.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.211+0000 7f0bbf423740 1 -- 172.21.15.18:0/3405199057 wait complete. 2024-04-16T17:25:24.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.212+0000 7f0bbf423740 20 client.30905 trim_cache size 0 max 16384 2024-04-16T17:25:24.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.249+0000 7f0bbf423740 1 Processor -- start 2024-04-16T17:25:24.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.249+0000 7f0bbf423740 1 -- start start 2024-04-16T17:25:24.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.250+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a498117e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.250+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.250+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498124360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.250+0000 7f0bbf423740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41c0d50 con 0x55a49810f1b0 2024-04-16T17:25:24.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.250+0000 7f0bbf423740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41be620 con 0x55a49810fad0 2024-04-16T17:25:24.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.250+0000 7f0bbf423740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bc490 con 0x55a4981186b0 2024-04-16T17:25:24.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.250+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.251+0000 7f0bb9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60192/0 (socket says 172.21.15.18:60192) 2024-04-16T17:25:24.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.251+0000 7f0bb9400640 1 -- 172.21.15.18:0/2247674699 learned_addr learned my addr 172.21.15.18:0/2247674699 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:24.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.251+0000 7f0bba800640 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498124360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.251+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a498117e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:23 smithi043 ceph-mon[36901]: pgmap v660: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 341 op/s 2024-04-16T17:25:24.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.252+0000 7f0bb9400640 1 -- 172.21.15.18:0/2247674699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 msgr2=0x55a498124360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:24.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.252+0000 7f0bb9400640 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498124360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.252+0000 7f0bb9400640 1 -- 172.21.15.18:0/2247674699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 msgr2=0x55a498117e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:24.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.252+0000 7f0bb9400640 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a498117e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.253+0000 7f0bb9400640 1 -- 172.21.15.18:0/2247674699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a498109100 con 0x55a49810fad0 2024-04-16T17:25:24.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.253+0000 7f0bba800640 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498124360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:24.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.253+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a498117e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:24.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.253+0000 7f0bb9400640 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498109cc0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40bd350 tx=0x7f0ba40f3e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29833b822765f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:24.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.254+0000 7f0ba2600640 1 -- 172.21.15.18:0/2247674699 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40bcfb0 con 0x55a49810fad0 2024-04-16T17:25:24.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.254+0000 7f0ba2600640 1 -- 172.21.15.18:0/2247674699 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba40054e0 con 0x55a49810fad0 2024-04-16T17:25:24.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.254+0000 7f0ba2600640 1 -- 172.21.15.18:0/2247674699 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40c6230 con 0x55a49810fad0 2024-04-16T17:25:24.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.254+0000 7f0bbf423740 1 -- 172.21.15.18:0/2247674699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 msgr2=0x55a498109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.254+0000 7f0bbf423740 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498109cc0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40bd350 tx=0x7f0ba40f3e80 comp rx=0 tx=0).stop 2024-04-16T17:25:24.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.255+0000 7f0bbf423740 1 -- 172.21.15.18:0/2247674699 shutdown_connections 2024-04-16T17:25:24.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.255+0000 7f0bbf423740 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a498124360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.255+0000 7f0bbf423740 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498109cc0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.255+0000 7f0bbf423740 1 --2- 172.21.15.18:0/2247674699 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a498117e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.255+0000 7f0bbf423740 1 -- 172.21.15.18:0/2247674699 >> 172.21.15.18:0/2247674699 conn(0x55a4981098e0 msgr2=0x55a498108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:24.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.255+0000 7f0bbf423740 1 -- 172.21.15.18:0/2247674699 shutdown_connections 2024-04-16T17:25:24.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.255+0000 7f0bbf423740 1 -- 172.21.15.18:0/2247674699 wait complete. 2024-04-16T17:25:24.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.255+0000 7f0bbf423740 1 -- start start 2024-04-16T17:25:24.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.256+0000 7f0bbf423740 1 Processor -- start 2024-04-16T17:25:24.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.257+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a498120720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.257+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498120c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.257+0000 7f0bbf423740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.257+0000 7f0bbf423740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bcae0 con 0x55a49810f1b0 2024-04-16T17:25:24.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.257+0000 7f0bbf423740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bdf50 con 0x55a49810fad0 2024-04-16T17:25:24.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.257+0000 7f0bbf423740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bd1b0 con 0x55a4981186b0 2024-04-16T17:25:24.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.257+0000 7f0bbf423740 10 client.0 _ll_register_callbacks cb 0x7f0ba80098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:24.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bba800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bba800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49614/0 (socket says 172.21.15.18:49614) 2024-04-16T17:25:24.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bba800640 1 -- 172.21.15.18:0/1323512831 learned_addr learned my addr 172.21.15.18:0/1323512831 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:24.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bb9400640 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498120c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bb9400640 1 -- 172.21.15.18:0/1323512831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 msgr2=0x55a49811bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:24.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bb9400640 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bb9400640 1 -- 172.21.15.18:0/1323512831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 msgr2=0x55a498120720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:25:24.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bb9400640 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a498120720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.258+0000 7f0bb9400640 1 -- 172.21.15.18:0/1323512831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49811c490 con 0x55a49810fad0 2024-04-16T17:25:24.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.259+0000 7f0b97400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:24.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.259+0000 7f0bb9400640 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498120c60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x55a498124920 tx=0x7f0ba402ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7f405c64ba3c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:24.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.260+0000 7f0b97400640 1 -- 172.21.15.18:0/1323512831 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40bcfb0 con 0x55a49810fad0 2024-04-16T17:25:24.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.260+0000 7f0b97400640 1 -- 172.21.15.18:0/1323512831 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7f0ba40bb950 con 0x55a49810fad0 2024-04-16T17:25:24.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.260+0000 7f0bbf423740 1 -- 172.21.15.18:0/1323512831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba40054e0 con 0x55a49810fad0 2024-04-16T17:25:24.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.261+0000 7f0b97400640 1 -- 172.21.15.18:0/1323512831 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0ba4005690 con 0x55a49810fad0 2024-04-16T17:25:24.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.262+0000 7f0bbf423740 10 client.30911 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:24.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.262+0000 7f0bbf423740 1 -- 172.21.15.18:0/1323512831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a49818ee20 con 0x55a49810fad0 2024-04-16T17:25:24.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.264+0000 7f0b97400640 1 -- 172.21.15.18:0/1323512831 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba40054e0 con 0x55a49810fad0 2024-04-16T17:25:24.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.264+0000 7f0b97400640 1 -- 172.21.15.18:0/1323512831 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f0ba40bd8b0 con 0x55a49810fad0 2024-04-16T17:25:24.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.264+0000 7f0bbf423740 10 client.30911 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:24.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.264+0000 7f0bbf423740 10 client.30911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:24.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.264+0000 7f0bbf423740 20 client.30911 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:24.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.264+0000 7f0bbf423740 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a49818f280 0x55a4981916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:24.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.264+0000 7f0bbf423740 4 client.30911 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:25:24.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.264+0000 7f0bbf423740 1 -- 172.21.15.18:0/1323512831 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a49807c9b0 con 0x55a49818f280 2024-04-16T17:25:24.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.266+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a49818f280 0x55a4981916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:24.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.266+0000 7f0bb9e00640 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a49818f280 0x55a4981916a0 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c0e363d6214e8d server_cookie=e6be158af7fc1a00 in_seq=0 out_seq=0 2024-04-16T17:25:24.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.266+0000 7f0b97400640 10 client.30911 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:24.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.267+0000 7f0b97400640 1 -- 172.21.15.18:0/1323512831 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55a49807c9b0 con 0x55a49818f280 2024-04-16T17:25:24.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.267+0000 7f0b97400640 10 client.30911 handle_command_reply: tid=0 2024-04-16T17:25:24.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.267+0000 7f0bbf423740 1 client.30911 shutdown 2024-04-16T17:25:24.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.270+0000 7f0bbf423740 1 -- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 msgr2=0x55a498120c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498120c60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x55a498124920 tx=0x7f0ba402ce80 comp rx=0 tx=0).stop 2024-04-16T17:25:24.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 -- 172.21.15.18:0/1323512831 shutdown_connections 2024-04-16T17:25:24.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a49818f280 0x55a4981916a0 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4981186b0 0x55a49811bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a49810fad0 0x55a498120c60 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 --2- 172.21.15.18:0/1323512831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a49810f1b0 0x55a498120720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0b97400640 0 client.30911 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 -- 172.21.15.18:0/1323512831 >> 172.21.15.18:0/1323512831 conn(0x55a4981098e0 msgr2=0x55a4981088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 -- 172.21.15.18:0/1323512831 shutdown_connections 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.271+0000 7f0bbf423740 1 -- 172.21.15.18:0/1323512831 wait complete. 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.272+0000 7f0bbf423740 20 client.30911 trim_cache size 0 max 16384 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:25:24.271 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (466 inodes in the stack)", 2024-04-16T17:25:24.272 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:25:24.272 INFO:teuthology.orchestra.run.smithi018.stdout: "5f0e024d-a0b6-449b-87e5-08a3461de802": { 2024-04-16T17:25:24.272 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:25:24.272 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "5f0e024d-a0b6-449b-87e5-08a3461de802", 2024-04-16T17:25:24.272 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:25:24.272 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:25:24.272 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:25:24.272 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:25:24.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 -- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b9c0a3f20 msgr2=0x7f0b9c0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b9c0a3f20 0x7f0b9c0a63e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0005c40 tx=0x7f0bb0004a40 comp rx=0 tx=0).stop 2024-04-16T17:25:24.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 -- 172.21.15.18:0/3535269981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bb4107300 msgr2=0x7f0bb4116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:24.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bb4107300 0x7f0bb4116a80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4036040 tx=0x7f0ba4005dc0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 -- 172.21.15.18:0/3535269981 shutdown_connections 2024-04-16T17:25:24.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0b9c0a3f20 0x7f0b9c0a63e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0bb41116a0 0x7f0bb4111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bb4107300 0x7f0bb4116a80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 --2- 172.21.15.18:0/3535269981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bb4102230 0x7f0bb4116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:24.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.274+0000 7f0bbc800640 1 -- 172.21.15.18:0/3535269981 >> 172.21.15.18:0/3535269981 conn(0x7f0bb40fdab0 msgr2=0x7f0bb40fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:24.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.275+0000 7f0bbc800640 1 -- 172.21.15.18:0/3535269981 shutdown_connections 2024-04-16T17:25:24.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:24.275+0000 7f0bbc800640 1 -- 172.21.15.18:0/3535269981 wait complete. 2024-04-16T17:25:24.312 INFO:tasks.cephfs.filesystem:scrub status for tag:5f0e024d-a0b6-449b-87e5-08a3461de802 - {'path': '/', 'tag': '5f0e024d-a0b6-449b-87e5-08a3461de802', 'options': 'recursive,force'} 2024-04-16T17:25:24.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:23 smithi022 ceph-mon[36023]: pgmap v660: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 341 op/s 2024-04-16T17:25:24.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 98895 10.74 MB/sec execute 336 sec latency 1781.799 ms 2024-04-16T17:25:25.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:25:25.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:25.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:25:25.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:25:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:25:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:25:25.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:25 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:25:25.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:25 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:25.606 INFO:tasks.workunit.client.0.smithi018.stdout: 10 99033 10.73 MB/sec execute 337 sec latency 1460.891 ms 2024-04-16T17:25:26.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:26 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:26.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:26 smithi022 ceph-mon[36023]: pgmap v661: 129 pgs: 12 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 79 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 367 op/s 2024-04-16T17:25:26.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:26 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:26.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:26 smithi043 ceph-mon[36901]: pgmap v661: 129 pgs: 12 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 79 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 367 op/s 2024-04-16T17:25:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.580+0000 7fc965600640 1 -- 172.21.15.18:0/1809132985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 msgr2=0x7fc960076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.580+0000 7fc965600640 1 --2- 172.21.15.18:0/1809132985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 0x7fc960076910 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc958009cd0 tx=0x7fc95802c100 comp rx=0 tx=0).stop 2024-04-16T17:25:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.582+0000 7fc965600640 1 -- 172.21.15.18:0/1809132985 shutdown_connections 2024-04-16T17:25:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.582+0000 7fc965600640 1 --2- 172.21.15.18:0/1809132985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc960112cb0 0x7fc9601150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.582+0000 7fc965600640 1 --2- 172.21.15.18:0/1809132985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc960076e50 0x7fc960112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.582+0000 7fc965600640 1 --2- 172.21.15.18:0/1809132985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 0x7fc960076910 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.582+0000 7fc965600640 1 -- 172.21.15.18:0/1809132985 >> 172.21.15.18:0/1809132985 conn(0x7fc9600ffae0 msgr2=0x7fc960101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.582+0000 7fc965600640 1 -- 172.21.15.18:0/1809132985 shutdown_connections 2024-04-16T17:25:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.582+0000 7fc965600640 1 -- 172.21.15.18:0/1809132985 wait complete. 2024-04-16T17:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.585+0000 7fc965600640 1 Processor -- start 2024-04-16T17:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.586+0000 7fc965600640 1 -- start start 2024-04-16T17:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc965600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 0x7fc96019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc965600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc960076e50 0x7fc96019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc965600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc960112cb0 0x7fc9601a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc965600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc960118870 con 0x7fc960076e50 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc965600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601186f0 con 0x7fc960076530 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc965600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601189f0 con 0x7fc960112cb0 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 0x7fc96019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 0x7fc96019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60208/0 (socket says 172.21.15.18:60208) 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.588+0000 7fc95fe00640 1 -- 172.21.15.18:0/1627301374 learned_addr learned my addr 172.21.15.18:0/1627301374 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.589+0000 7fc95fe00640 1 -- 172.21.15.18:0/1627301374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc960112cb0 msgr2=0x7fc9601a35f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.589+0000 7fc95fe00640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc960112cb0 0x7fc9601a35f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.589+0000 7fc95fe00640 1 -- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc960076e50 msgr2=0x7fc96019f270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.589+0000 7fc95fe00640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc960076e50 0x7fc96019f270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.589+0000 7fc95fe00640 1 -- 172.21.15.18:0/1627301374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9601a3d20 con 0x7fc960076530 2024-04-16T17:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.589+0000 7fc95fe00640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 0x7fc96019ed30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc9580348f0 tx=0x7fc958035d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.591+0000 7fc95cc00640 1 -- 172.21.15.18:0/1627301374 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958039040 con 0x7fc960076530 2024-04-16T17:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.591+0000 7fc95cc00640 1 -- 172.21.15.18:0/1627301374 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc958003970 con 0x7fc960076530 2024-04-16T17:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.591+0000 7fc95cc00640 1 -- 172.21.15.18:0/1627301374 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580045b0 con 0x7fc960076530 2024-04-16T17:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.591+0000 7fc965600640 1 -- 172.21.15.18:0/1627301374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9601a3fb0 con 0x7fc960076530 2024-04-16T17:25:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.592+0000 7fc965600640 1 -- 172.21.15.18:0/1627301374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96006ba20 con 0x7fc960076530 2024-04-16T17:25:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.599+0000 7fc95cc00640 1 -- 172.21.15.18:0/1627301374 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc958003b20 con 0x7fc960076530 2024-04-16T17:25:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.599+0000 7fc96999f740 1 Processor -- start 2024-04-16T17:25:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.601+0000 7fc96999f740 1 -- start start 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc96999f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358c7f0 0x557f0358b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc96999f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358ba80 0x557f0358bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc96999f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f035973f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc964c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358c7f0 0x557f0358b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358ba80 0x557f0358bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc964c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358c7f0 0x557f0358b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49622/0 (socket says 172.21.15.18:49622) 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc964c00640 1 -- 172.21.15.18:0/3592878937 learned_addr learned my addr 172.21.15.18:0/3592878937 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc96999f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557f034f0780 con 0x557f03595000 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc96999f740 1 -- 172.21.15.18:0/3592878937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557f03496f40 con 0x557f0358ba80 2024-04-16T17:25:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.604+0000 7fc96999f740 1 -- 172.21.15.18:0/3592878937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f034ec740 con 0x557f0358c7f0 2024-04-16T17:25:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.605+0000 7fc95cc00640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9440a3e50 0x7fc9440a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.605+0000 7fc95cc00640 1 -- 172.21.15.18:0/1627301374 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fc9580f2940 con 0x7fc960076530 2024-04-16T17:25:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.605+0000 7fc95f400640 1 --2- 172.21.15.18:0/3592878937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f035973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc95fe00640 1 -- 172.21.15.18:0/3592878937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358c7f0 msgr2=0x557f0358b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc95fe00640 1 --2- 172.21.15.18:0/3592878937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358c7f0 0x557f0358b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc95fe00640 1 -- 172.21.15.18:0/3592878937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 msgr2=0x557f035973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc95fe00640 1 --2- 172.21.15.18:0/3592878937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f035973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc95fe00640 1 -- 172.21.15.18:0/3592878937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f03203210 con 0x557f0358ba80 2024-04-16T17:25:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc95fe00640 1 --2- 172.21.15.18:0/3592878937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358ba80 0x557f0358bf00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fc9580efd40 tx=0x7fc95802c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330447216fa36bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc955200640 1 -- 172.21.15.18:0/3592878937 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580bec90 con 0x557f0358ba80 2024-04-16T17:25:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc955200640 1 -- 172.21.15.18:0/3592878937 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9580be590 con 0x557f0358ba80 2024-04-16T17:25:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.606+0000 7fc955200640 1 -- 172.21.15.18:0/3592878937 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580efeb0 con 0x557f0358ba80 2024-04-16T17:25:26.607 INFO:tasks.workunit.client.0.smithi018.stdout: 10 99179 10.71 MB/sec execute 338 sec latency 1015.682 ms 2024-04-16T17:25:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.607+0000 7fc96999f740 1 -- 172.21.15.18:0/3592878937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358ba80 msgr2=0x557f0358bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.607+0000 7fc96999f740 1 --2- 172.21.15.18:0/3592878937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358ba80 0x557f0358bf00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fc9580efd40 tx=0x7fc95802c100 comp rx=0 tx=0).stop 2024-04-16T17:25:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.608+0000 7fc96999f740 1 -- 172.21.15.18:0/3592878937 shutdown_connections 2024-04-16T17:25:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.608+0000 7fc96999f740 1 --2- 172.21.15.18:0/3592878937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f035973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.608+0000 7fc96999f740 1 --2- 172.21.15.18:0/3592878937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358ba80 0x557f0358bf00 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.608+0000 7fc96999f740 1 --2- 172.21.15.18:0/3592878937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358c7f0 0x557f0358b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.608+0000 7fc96999f740 1 -- 172.21.15.18:0/3592878937 >> 172.21.15.18:0/3592878937 conn(0x557f035265d0 msgr2=0x557f03586770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.609+0000 7fc964c00640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9440a3e50 0x7fc9440a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.609+0000 7fc96999f740 1 -- 172.21.15.18:0/3592878937 shutdown_connections 2024-04-16T17:25:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.609+0000 7fc96999f740 1 -- 172.21.15.18:0/3592878937 wait complete. 2024-04-16T17:25:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.610+0000 7fc96999f740 1 -- start start 2024-04-16T17:25:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.610+0000 7fc964c00640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9440a3e50 0x7fc9440a6310 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006bd0 tx=0x7fc94c008040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.610+0000 7fc96999f740 1 Processor -- start 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc96999f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 0x557f03594a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc96999f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03591e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc96999f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f03592550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc96999f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557f03597af0 con 0x557f03595000 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc96999f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557f034aa390 con 0x557f0358c7f0 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc96999f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f03597c70 con 0x557f0358ba80 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc96999f740 10 client.0 _ll_register_callbacks cb 0x557f03493f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc95f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03591e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc95f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03591e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60224/0 (socket says 172.21.15.18:60224) 2024-04-16T17:25:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc95f400640 1 -- 172.21.15.18:0/2670568287 learned_addr learned my addr 172.21.15.18:0/2670568287 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.611+0000 7fc964c00640 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f03592550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.612+0000 7fc964c00640 1 -- 172.21.15.18:0/2670568287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 msgr2=0x557f03594a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:25:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.612+0000 7fc964c00640 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 0x557f03594a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.612+0000 7fc964c00640 1 -- 172.21.15.18:0/2670568287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 msgr2=0x557f03591e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.612+0000 7fc964c00640 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03591e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.612+0000 7fc964c00640 1 -- 172.21.15.18:0/2670568287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f032cf620 con 0x557f03595000 2024-04-16T17:25:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.612+0000 7fc95f400640 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03591e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.615+0000 7fc964c00640 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f03592550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006b60 tx=0x7fc94c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50f27c477bec965 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.615+0000 7fc949600640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:25:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.616+0000 7fc949600640 1 -- 172.21.15.18:0/2670568287 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c025060 con 0x557f03595000 2024-04-16T17:25:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.616+0000 7fc949600640 1 -- 172.21.15.18:0/2670568287 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fc94c013790 con 0x557f03595000 2024-04-16T17:25:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.616+0000 7fc96999f740 1 -- 172.21.15.18:0/2670568287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9580be590 con 0x557f03595000 2024-04-16T17:25:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.617+0000 7fc949600640 1 -- 172.21.15.18:0/2670568287 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc94c015e10 con 0x557f03595000 2024-04-16T17:25:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.619+0000 7fc96999f740 10 client.19923 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.620+0000 7fc96999f740 1 -- 172.21.15.18:0/2670568287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557f035a3b10 con 0x557f03595000 2024-04-16T17:25:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc949600640 1 -- 172.21.15.18:0/2670568287 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9580be590 con 0x557f03595000 2024-04-16T17:25:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc949600640 1 -- 172.21.15.18:0/2670568287 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fc94c029c30 con 0x557f03595000 2024-04-16T17:25:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc96999f740 10 client.19923 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc96999f740 10 client.19923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc96999f740 20 client.19923 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc96999f740 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557f035a4f80 0x557f035a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc96999f740 4 client.19923 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc96999f740 1 -- 172.21.15.18:0/2670568287 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f034fe1a0 con 0x557f035a4f80 2024-04-16T17:25:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.622+0000 7fc95fe00640 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557f035a4f80 0x557f035a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.623+0000 7fc95fe00640 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557f035a4f80 0x557f035a73b0 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea16495d1223335 server_cookie=cb8859e29b6c443c in_seq=0 out_seq=0 2024-04-16T17:25:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.623+0000 7fc949600640 10 client.19923 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.626+0000 7fc949600640 1 -- 172.21.15.18:0/2670568287 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f034fe1a0 con 0x557f035a4f80 2024-04-16T17:25:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.626+0000 7fc949600640 10 client.19923 handle_command_reply: tid=0 2024-04-16T17:25:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.626+0000 7fc96999f740 1 client.19923 shutdown 2024-04-16T17:25:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.626+0000 7fc96999f740 1 -- 172.21.15.18:0/2670568287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 msgr2=0x557f03592550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.626+0000 7fc96999f740 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f03592550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006b60 tx=0x7fc94c004700 comp rx=0 tx=0).stop 2024-04-16T17:25:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 1 -- 172.21.15.18:0/2670568287 shutdown_connections 2024-04-16T17:25:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557f035a4f80 0x557f035a73b0 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f03595000 0x557f03592550 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03591e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 1 --2- 172.21.15.18:0/2670568287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 0x557f03594a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 1 -- 172.21.15.18:0/2670568287 >> 172.21.15.18:0/2670568287 conn(0x557f035265d0 msgr2=0x557f035884c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc949600640 0 client.19923 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 1 -- 172.21.15.18:0/2670568287 shutdown_connections 2024-04-16T17:25:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 1 -- 172.21.15.18:0/2670568287 wait complete. 2024-04-16T17:25:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.627+0000 7fc96999f740 20 client.19923 trim_cache size 0 max 16384 2024-04-16T17:25:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.694+0000 7fc96999f740 1 Processor -- start 2024-04-16T17:25:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.694+0000 7fc96999f740 1 -- start start 2024-04-16T17:25:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.695+0000 7fc96999f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358ba80 0x557f03590fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.695+0000 7fc96999f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03585690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.695+0000 7fc96999f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f03595000 0x557f03594540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.695+0000 7fc96999f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601c0ba0 con 0x557f0358ba80 2024-04-16T17:25:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.695+0000 7fc96999f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601be470 con 0x557f0358c7f0 2024-04-16T17:25:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.695+0000 7fc96999f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc960073c50 con 0x557f03595000 2024-04-16T17:25:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.695+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358ba80 0x557f03590fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.695+0000 7fc964c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f03595000 0x557f03594540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.696+0000 7fc964c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f03595000 0x557f03594540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49636/0 (socket says 172.21.15.18:49636) 2024-04-16T17:25:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.696+0000 7fc964c00640 1 -- 172.21.15.18:0/3073612437 learned_addr learned my addr 172.21.15.18:0/3073612437 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.696+0000 7fc95f400640 1 --2- 172.21.15.18:0/3073612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03585690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.696+0000 7fc964c00640 1 -- 172.21.15.18:0/3073612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 msgr2=0x557f03585690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.696+0000 7fc964c00640 1 --2- 172.21.15.18:0/3073612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03585690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.696+0000 7fc964c00640 1 -- 172.21.15.18:0/3073612437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358ba80 msgr2=0x557f03590fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.696+0000 7fc964c00640 1 --2- 172.21.15.18:0/3073612437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358ba80 0x557f03590fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.696+0000 7fc964c00640 1 -- 172.21.15.18:0/3073612437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0358ad50 con 0x557f03595000 2024-04-16T17:25:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.698+0000 7fc964c00640 1 --2- 172.21.15.18:0/3073612437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f03595000 0x557f03594540 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc94c01f590 tx=0x7fc94c01f820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d49caeb7a2b5d21e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.699+0000 7fc94be00640 1 -- 172.21.15.18:0/3073612437 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c02f080 con 0x557f03595000 2024-04-16T17:25:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.699+0000 7fc94be00640 1 -- 172.21.15.18:0/3073612437 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c010040 con 0x557f03595000 2024-04-16T17:25:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.699+0000 7fc94be00640 1 -- 172.21.15.18:0/3073612437 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c02a420 con 0x557f03595000 2024-04-16T17:25:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 -- 172.21.15.18:0/3073612437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f03595000 msgr2=0x557f03594540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 --2- 172.21.15.18:0/3073612437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f03595000 0x557f03594540 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc94c01f590 tx=0x7fc94c01f820 comp rx=0 tx=0).stop 2024-04-16T17:25:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 -- 172.21.15.18:0/3073612437 shutdown_connections 2024-04-16T17:25:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 --2- 172.21.15.18:0/3073612437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f03595000 0x557f03594540 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 --2- 172.21.15.18:0/3073612437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f0358c7f0 0x557f03585690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 --2- 172.21.15.18:0/3073612437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358ba80 0x557f03590fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 -- 172.21.15.18:0/3073612437 >> 172.21.15.18:0/3073612437 conn(0x557f035265d0 msgr2=0x557f03584f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 -- 172.21.15.18:0/3073612437 shutdown_connections 2024-04-16T17:25:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.700+0000 7fc96999f740 1 -- 172.21.15.18:0/3073612437 wait complete. 2024-04-16T17:25:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.701+0000 7fc96999f740 1 -- start start 2024-04-16T17:25:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.701+0000 7fc96999f740 1 Processor -- start 2024-04-16T17:25:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.702+0000 7fc96999f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 0x557f0359d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.702+0000 7fc96999f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358c7f0 0x557f03606680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.702+0000 7fc96999f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f03595000 0x557f03606be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.702+0000 7fc96999f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601bc930 con 0x557f0358c7f0 2024-04-16T17:25:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.702+0000 7fc96999f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601bdda0 con 0x557f03595000 2024-04-16T17:25:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.702+0000 7fc96999f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601bd000 con 0x557f0358ba80 2024-04-16T17:25:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.702+0000 7fc96999f740 10 client.0 _ll_register_callbacks cb 0x7fc94c00fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.703+0000 7fc964c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f03595000 0x557f03606be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.703+0000 7fc964c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f03595000 0x557f03606be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60244/0 (socket says 172.21.15.18:60244) 2024-04-16T17:25:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.703+0000 7fc964c00640 1 -- 172.21.15.18:0/1447483685 learned_addr learned my addr 172.21.15.18:0/1447483685 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.703+0000 7fc95fe00640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 0x557f0359d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.703+0000 7fc964c00640 1 -- 172.21.15.18:0/1447483685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 msgr2=0x557f0359d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.703+0000 7fc95f400640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358c7f0 0x557f03606680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.704+0000 7fc964c00640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 0x557f0359d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.704+0000 7fc964c00640 1 -- 172.21.15.18:0/1447483685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358c7f0 msgr2=0x557f03606680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.704+0000 7fc964c00640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358c7f0 0x557f03606680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.704+0000 7fc964c00640 1 -- 172.21.15.18:0/1447483685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f03588a70 con 0x557f03595000 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.704+0000 7fc964c00640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f03595000 0x557f03606be0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc94c004740 tx=0x7fc94c013840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec2db135a6e75e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc949600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc949600640 1 -- 172.21.15.18:0/1447483685 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c02f040 con 0x557f03595000 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc949600640 1 -- 172.21.15.18:0/1447483685 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fc94c01fda0 con 0x557f03595000 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc96999f740 1 -- 172.21.15.18:0/1447483685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc94c010040 con 0x557f03595000 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc95f400640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358c7f0 0x557f03606680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc95fe00640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 0x557f0359d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc949600640 1 -- 172.21.15.18:0/1447483685 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc94c025040 con 0x557f03595000 2024-04-16T17:25:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc96999f740 10 client.30929 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.705+0000 7fc96999f740 1 -- 172.21.15.18:0/1447483685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557f0359be90 con 0x557f03595000 2024-04-16T17:25:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.706+0000 7fc949600640 1 -- 172.21.15.18:0/1447483685 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c010040 con 0x557f03595000 2024-04-16T17:25:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.707+0000 7fc949600640 1 -- 172.21.15.18:0/1447483685 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fc94c045c70 con 0x557f03595000 2024-04-16T17:25:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.707+0000 7fc96999f740 10 client.30929 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.707+0000 7fc96999f740 10 client.30929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.707+0000 7fc96999f740 20 client.30929 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.707+0000 7fc96999f740 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557f0359c510 0x557f03620560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.707+0000 7fc96999f740 4 client.30929 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:25:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.707+0000 7fc96999f740 1 -- 172.21.15.18:0/1447483685 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f034f8010 con 0x557f0359c510 2024-04-16T17:25:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.708+0000 7fc95fe00640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557f0359c510 0x557f03620560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.710+0000 7fc95fe00640 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557f0359c510 0x557f03620560 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe59ee450b1a97a server_cookie=f34121c9a521ac66 in_seq=0 out_seq=0 2024-04-16T17:25:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:26.710+0000 7fc949600640 10 client.30929 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:24 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:24 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:24 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:24 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:25 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:25 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:25 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:25 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:25:27.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:25 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:27.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:26 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:25:27.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:26 smithi018 ceph-mon[31776]: pgmap v661: 129 pgs: 12 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 79 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 367 op/s 2024-04-16T17:25:27.607 INFO:tasks.workunit.client.0.smithi018.stdout: 10 99220 10.68 MB/sec execute 339 sec latency 570.530 ms 2024-04-16T17:25:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.715+0000 7fc949600640 1 -- 172.21.15.18:0/1447483685 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f034f8010 con 0x557f0359c510 2024-04-16T17:25:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.716+0000 7fc949600640 10 client.30929 handle_command_reply: tid=0 2024-04-16T17:25:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.716+0000 7fc96999f740 1 client.30929 shutdown 2024-04-16T17:25:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.719+0000 7fc96999f740 1 -- 172.21.15.18:0/1447483685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f03595000 msgr2=0x557f03606be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.720+0000 7fc96999f740 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f03595000 0x557f03606be0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc94c004740 tx=0x7fc94c013840 comp rx=0 tx=0).stop 2024-04-16T17:25:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.720+0000 7fc96999f740 1 -- 172.21.15.18:0/1447483685 shutdown_connections 2024-04-16T17:25:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.720+0000 7fc96999f740 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x557f0359c510 0x557f03620560 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.720+0000 7fc96999f740 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f03595000 0x557f03606be0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.720+0000 7fc96999f740 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557f0358c7f0 0x557f03606680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.720+0000 7fc96999f740 1 --2- 172.21.15.18:0/1447483685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f0358ba80 0x557f0359d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.720+0000 7fc96999f740 1 -- 172.21.15.18:0/1447483685 >> 172.21.15.18:0/1447483685 conn(0x557f035265d0 msgr2=0x557f03585810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.720+0000 7fc949600640 0 client.30929 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.721+0000 7fc96999f740 1 -- 172.21.15.18:0/1447483685 shutdown_connections 2024-04-16T17:25:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.721+0000 7fc96999f740 1 -- 172.21.15.18:0/1447483685 wait complete. 2024-04-16T17:25:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.721+0000 7fc96999f740 20 client.30929 trim_cache size 0 max 16384 2024-04-16T17:25:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:25:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:25:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:25:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:25:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.723+0000 7fc965600640 1 -- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9440a3e50 msgr2=0x7fc9440a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9440a3e50 0x7fc9440a6310 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006bd0 tx=0x7fc94c008040 comp rx=0 tx=0).stop 2024-04-16T17:25:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 -- 172.21.15.18:0/1627301374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 msgr2=0x7fc96019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 0x7fc96019ed30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc9580348f0 tx=0x7fc958035d80 comp rx=0 tx=0).stop 2024-04-16T17:25:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 -- 172.21.15.18:0/1627301374 shutdown_connections 2024-04-16T17:25:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9440a3e50 0x7fc9440a6310 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc960112cb0 0x7fc9601a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc960076e50 0x7fc96019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 --2- 172.21.15.18:0/1627301374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc960076530 0x7fc96019ed30 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.724+0000 7fc965600640 1 -- 172.21.15.18:0/1627301374 >> 172.21.15.18:0/1627301374 conn(0x7fc9600ffae0 msgr2=0x7fc960101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.725+0000 7fc965600640 1 -- 172.21.15.18:0/1627301374 shutdown_connections 2024-04-16T17:25:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:27.725+0000 7fc965600640 1 -- 172.21.15.18:0/1627301374 wait complete. 2024-04-16T17:25:28.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:28 smithi022 ceph-mon[36023]: pgmap v662: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 430 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-04-16T17:25:28.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:28 smithi043 ceph-mon[36901]: pgmap v662: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 430 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-04-16T17:25:28.611 INFO:tasks.workunit.client.0.smithi018.stdout: 10 99392 10.67 MB/sec execute 340 sec latency 562.543 ms 2024-04-16T17:25:28.895 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:28 smithi018 ceph-mon[31776]: pgmap v662: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 430 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-04-16T17:25:29.611 INFO:tasks.workunit.client.0.smithi018.stdout: 10 99537 10.66 MB/sec execute 341 sec latency 513.138 ms 2024-04-16T17:25:30.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:30 smithi022 ceph-mon[36023]: pgmap v663: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 430 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-04-16T17:25:30.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:30 smithi043 ceph-mon[36901]: pgmap v663: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 430 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-04-16T17:25:30.611 INFO:tasks.workunit.client.0.smithi018.stdout: 10 99718 10.65 MB/sec execute 342 sec latency 193.504 ms 2024-04-16T17:25:31.612 INFO:tasks.workunit.client.0.smithi018.stdout: 10 99862 10.65 MB/sec execute 343 sec latency 406.822 ms 2024-04-16T17:25:32.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:32 smithi022 ceph-mon[36023]: pgmap v664: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 428 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 471 op/s 2024-04-16T17:25:32.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:32 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:25:32.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:32 smithi043 ceph-mon[36901]: pgmap v664: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 428 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 471 op/s 2024-04-16T17:25:32.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:32 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:25:32.612 INFO:tasks.workunit.client.0.smithi018.stdout: 10 100040 10.65 MB/sec execute 344 sec latency 474.043 ms 2024-04-16T17:25:33.612 INFO:tasks.workunit.client.0.smithi018.stdout: 10 100316 10.67 MB/sec execute 345 sec latency 190.549 ms 2024-04-16T17:25:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:33 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:33 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:33 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:33 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:33 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:33.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:33 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:33 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:33 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:33 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:33 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:33 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:33 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.539 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:30 smithi018 ceph-mon[31776]: pgmap v663: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 430 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-04-16T17:25:34.539 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:32 smithi018 ceph-mon[31776]: pgmap v664: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 428 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 471 op/s 2024-04-16T17:25:34.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:32 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:25:34.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:33 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:33 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:33 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:33 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:33 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:33 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:34.614 INFO:tasks.workunit.client.0.smithi018.stdout: 10 100508 10.68 MB/sec execute 346 sec latency 405.615 ms 2024-04-16T17:25:34.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:34 smithi022 ceph-mon[36023]: pgmap v665: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 428 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-04-16T17:25:34.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:34 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:25:34.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:34 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:25:35.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:34 smithi043 ceph-mon[36901]: pgmap v665: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 428 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-04-16T17:25:35.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:34 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:25:35.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:34 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:25:35.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:34 smithi018 ceph-mon[31776]: pgmap v665: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 428 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-04-16T17:25:35.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:34 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:25:35.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:34 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:25:35.614 INFO:tasks.workunit.client.0.smithi018.stdout: 10 100698 10.68 MB/sec execute 347 sec latency 202.108 ms 2024-04-16T17:25:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:25:35.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:35.924 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:35 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:35 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:36.615 INFO:tasks.workunit.client.0.smithi018.stdout: 10 100881 10.67 MB/sec execute 348 sec latency 191.334 ms 2024-04-16T17:25:36.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:36 smithi022 ceph-mon[36023]: pgmap v666: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 430 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 353 op/s 2024-04-16T17:25:36.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:36 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:25:36.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:36 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:25:36.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:36 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:36.923 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:36 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:37.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:36 smithi043 ceph-mon[36901]: pgmap v666: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 430 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 353 op/s 2024-04-16T17:25:37.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:36 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:25:37.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:36 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:25:37.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:36 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:37.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:36 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:37.615 INFO:tasks.workunit.client.0.smithi018.stdout: 10 101091 10.66 MB/sec execute 349 sec latency 245.446 ms 2024-04-16T17:25:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:37 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:37 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:37 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:37 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:37 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:37 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:37 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:37 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:37 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:37 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:37 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:37 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:37 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:37 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:37 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:37 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:38.615 INFO:tasks.workunit.client.0.smithi018.stdout: 10 101245 10.64 MB/sec execute 350 sec latency 210.270 ms 2024-04-16T17:25:39.000 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:38 smithi043 ceph-mon[36901]: pgmap v667: 129 pgs: 129 active+clean; 440 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-04-16T17:25:39.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:38 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:25:39.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:38 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:25:39.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:38 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:39.001 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:38 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:39.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:38 smithi022 ceph-mon[36023]: pgmap v667: 129 pgs: 129 active+clean; 440 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-04-16T17:25:39.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:38 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:25:39.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:38 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:25:39.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:38 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:39.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:38 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:39.615 INFO:tasks.workunit.client.0.smithi018.stdout: 10 101456 10.64 MB/sec execute 351 sec latency 187.866 ms 2024-04-16T17:25:40.074 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: osdmap e133: 12 total, 12 up, 12 in 2024-04-16T17:25:40.074 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: pgmap v669: 129 pgs: 129 active+clean; 439 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-16T17:25:40.074 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.074 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.074 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.075 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.075 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.075 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.075 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.075 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.075 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:25:40.075 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:25:40.076 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:39 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: osdmap e133: 12 total, 12 up, 12 in 2024-04-16T17:25:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: pgmap v669: 129 pgs: 129 active+clean; 439 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-16T17:25:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:25:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:25:40.252 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:39 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:40.616 INFO:tasks.workunit.client.0.smithi018.stdout: 10 101755 10.67 MB/sec execute 352 sec latency 137.630 ms 2024-04-16T17:25:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.933+0000 7f8921e00640 1 -- 172.21.15.18:0/4014974234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c074cf0 msgr2=0x7f891c112b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.933+0000 7f8921e00640 1 --2- 172.21.15.18:0/4014974234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c074cf0 0x7f891c112b40 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8908009900 tx=0x7f890802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.934+0000 7f8921e00640 1 -- 172.21.15.18:0/4014974234 shutdown_connections 2024-04-16T17:25:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.934+0000 7f8921e00640 1 --2- 172.21.15.18:0/4014974234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c105f00 0x7f891c1150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.934+0000 7f8921e00640 1 --2- 172.21.15.18:0/4014974234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c074cf0 0x7f891c112b40 secure :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8908009900 tx=0x7f890802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.934+0000 7f8921e00640 1 --2- 172.21.15.18:0/4014974234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c0743d0 0x7f891c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.934+0000 7f8921e00640 1 -- 172.21.15.18:0/4014974234 >> 172.21.15.18:0/4014974234 conn(0x7f891c0fdac0 msgr2=0x7f891c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.935+0000 7f8921e00640 1 -- 172.21.15.18:0/4014974234 shutdown_connections 2024-04-16T17:25:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.935+0000 7f8921e00640 1 -- 172.21.15.18:0/4014974234 wait complete. 2024-04-16T17:25:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.937+0000 7f8921e00640 1 Processor -- start 2024-04-16T17:25:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.937+0000 7f8921e00640 1 -- start start 2024-04-16T17:25:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.937+0000 7f8921e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c0743d0 0x7f891c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.937+0000 7f8921e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19f610 0x7f891c1a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.938+0000 7f8921e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c1a3ed0 0x7f891c1a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.938+0000 7f8921e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f891c118a50 con 0x7f891c0743d0 2024-04-16T17:25:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.938+0000 7f8921e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f891c1188d0 con 0x7f891c1a3ed0 2024-04-16T17:25:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.938+0000 7f8921e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f891c118bd0 con 0x7f891c19f610 2024-04-16T17:25:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.938+0000 7f891a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19f610 0x7f891c1a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.938+0000 7f891a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19f610 0x7f891c1a3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35790/0 (socket says 172.21.15.18:35790) 2024-04-16T17:25:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.938+0000 7f891a000640 1 -- 172.21.15.18:0/804593812 learned_addr learned my addr 172.21.15.18:0/804593812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.938+0000 7f891b400640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c1a3ed0 0x7f891c1a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.939+0000 7f891a000640 1 -- 172.21.15.18:0/804593812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c1a3ed0 msgr2=0x7f891c1a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.939+0000 7f891a000640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c1a3ed0 0x7f891c1a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.939+0000 7f891aa00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c0743d0 0x7f891c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.939+0000 7f891a000640 1 -- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c0743d0 msgr2=0x7f891c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.939+0000 7f891a000640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c0743d0 0x7f891c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.939+0000 7f891a000640 1 -- 172.21.15.18:0/804593812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c1abc40 con 0x7f891c19f610 2024-04-16T17:25:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.939+0000 7f891aa00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c0743d0 0x7f891c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.940+0000 7f891a000640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19f610 0x7f891c1a3990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8908005ab0 tx=0x7f890802c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.940+0000 7f8920a00640 1 -- 172.21.15.18:0/804593812 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908041020 con 0x7f891c19f610 2024-04-16T17:25:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.940+0000 7f8921e00640 1 -- 172.21.15.18:0/804593812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c1abe10 con 0x7f891c19f610 2024-04-16T17:25:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.941+0000 7f8920a00640 1 -- 172.21.15.18:0/804593812 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89080054e0 con 0x7f891c19f610 2024-04-16T17:25:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.941+0000 7f8921e00640 1 -- 172.21.15.18:0/804593812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c1ac2d0 con 0x7f891c19f610 2024-04-16T17:25:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.941+0000 7f8920a00640 1 -- 172.21.15.18:0/804593812 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908003a80 con 0x7f891c19f610 2024-04-16T17:25:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.946+0000 7f8924b77740 1 Processor -- start 2024-04-16T17:25:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.946+0000 7f8924b77740 1 -- start start 2024-04-16T17:25:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f8924b77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 0x561dfbb4f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f8924b77740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb4fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f8924b77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb5b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f8924b77740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561dfbab4780 con 0x561dfbb4f8a0 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f8924b77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561dfba5af40 con 0x561dfbb58e20 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f8924b77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561dfbab0740 con 0x561dfbb50610 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f891a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb5b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f891a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb5b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49924/0 (socket says 172.21.15.18:49924) 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.947+0000 7f891a000640 1 -- 172.21.15.18:0/1934283406 learned_addr learned my addr 172.21.15.18:0/1934283406 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.948+0000 7f891aa00640 1 --2- 172.21.15.18:0/1934283406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb4fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.948+0000 7f891a000640 1 -- 172.21.15.18:0/1934283406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 msgr2=0x561dfbb4f360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:25:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.948+0000 7f891a000640 1 --2- 172.21.15.18:0/1934283406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 0x561dfbb4f360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.948+0000 7f891a000640 1 -- 172.21.15.18:0/1934283406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 msgr2=0x561dfbb4fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.948+0000 7f891a000640 1 --2- 172.21.15.18:0/1934283406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb4fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.948+0000 7f891a000640 1 -- 172.21.15.18:0/1934283406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dfb7c7210 con 0x561dfbb58e20 2024-04-16T17:25:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.948+0000 7f891a000640 1 --2- 172.21.15.18:0/1934283406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb5b210 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f89080ef440 tx=0x7f89080fa270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87da20fe63f21bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.949+0000 7f8907e00640 1 -- 172.21.15.18:0/1934283406 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908116070 con 0x561dfbb58e20 2024-04-16T17:25:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.949+0000 7f8907e00640 1 -- 172.21.15.18:0/1934283406 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89080f42e0 con 0x561dfbb58e20 2024-04-16T17:25:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.949+0000 7f8907e00640 1 -- 172.21.15.18:0/1934283406 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908111410 con 0x561dfbb58e20 2024-04-16T17:25:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.950+0000 7f8920a00640 1 -- 172.21.15.18:0/804593812 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8908044020 con 0x7f891c19f610 2024-04-16T17:25:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.951+0000 7f8924b77740 1 -- 172.21.15.18:0/1934283406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 msgr2=0x561dfbb5b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.951+0000 7f8924b77740 1 --2- 172.21.15.18:0/1934283406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb5b210 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f89080ef440 tx=0x7f89080fa270 comp rx=0 tx=0).stop 2024-04-16T17:25:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.953+0000 7f8920a00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d90 0x7f89000a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.953+0000 7f8924b77740 1 -- 172.21.15.18:0/1934283406 shutdown_connections 2024-04-16T17:25:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.953+0000 7f8920a00640 1 -- 172.21.15.18:0/804593812 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f89080f2110 con 0x7f891c19f610 2024-04-16T17:25:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.953+0000 7f8924b77740 1 --2- 172.21.15.18:0/1934283406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb5b210 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.953+0000 7f8924b77740 1 --2- 172.21.15.18:0/1934283406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb4fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.953+0000 7f8924b77740 1 --2- 172.21.15.18:0/1934283406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 0x561dfbb4f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.953+0000 7f8924b77740 1 -- 172.21.15.18:0/1934283406 >> 172.21.15.18:0/1934283406 conn(0x561dfbaea5d0 msgr2=0x561dfbb4a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.954+0000 7f891b400640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d90 0x7f89000a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.954+0000 7f8924b77740 1 -- 172.21.15.18:0/1934283406 shutdown_connections 2024-04-16T17:25:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.955+0000 7f8924b77740 1 -- 172.21.15.18:0/1934283406 wait complete. 2024-04-16T17:25:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.955+0000 7f8924b77740 1 -- start start 2024-04-16T17:25:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.957+0000 7f891b400640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d90 0x7f89000a6250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f891c06b700 tx=0x7f8914009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.957+0000 7f8924b77740 1 Processor -- start 2024-04-16T17:25:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.958+0000 7f8924b77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb4f8a0 0x561dfbb58870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.959+0000 7f8924b77740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb50610 0x561dfbb55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.959+0000 7f8924b77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb56340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.959+0000 7f8924b77740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561dfbb5b910 con 0x561dfbb50610 2024-04-16T17:25:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.959+0000 7f8924b77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561dfba6e390 con 0x561dfbb58e20 2024-04-16T17:25:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.959+0000 7f8924b77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561dfbb5ba90 con 0x561dfbb4f8a0 2024-04-16T17:25:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.959+0000 7f8924b77740 10 client.0 _ll_register_callbacks cb 0x561dfba57f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.960+0000 7f891aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb4f8a0 0x561dfbb58870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.960+0000 7f891aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb4f8a0 0x561dfbb58870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:35804/0 (socket says 172.21.15.18:35804) 2024-04-16T17:25:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.960+0000 7f891aa00640 1 -- 172.21.15.18:0/4022358077 learned_addr learned my addr 172.21.15.18:0/4022358077 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.960+0000 7f891b400640 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb56340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.960+0000 7f891a000640 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb50610 0x561dfbb55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.961+0000 7f891a000640 1 -- 172.21.15.18:0/4022358077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb4f8a0 msgr2=0x561dfbb58870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.961+0000 7f891a000640 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb4f8a0 0x561dfbb58870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.961+0000 7f891a000640 1 -- 172.21.15.18:0/4022358077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 msgr2=0x561dfbb56340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.961+0000 7f891a000640 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb56340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.961+0000 7f891a000640 1 -- 172.21.15.18:0/4022358077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dfb893620 con 0x561dfbb50610 2024-04-16T17:25:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.962+0000 7f891a000640 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb50610 0x561dfbb55c00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f89080bdce0 tx=0x7f89080035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e507b3432503de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.962+0000 7f8904c00640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:25:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.963+0000 7f8904c00640 1 -- 172.21.15.18:0/4022358077 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89080b9ff0 con 0x561dfbb50610 2024-04-16T17:25:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.963+0000 7f8904c00640 1 -- 172.21.15.18:0/4022358077 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f890810e070 con 0x561dfbb50610 2024-04-16T17:25:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.963+0000 7f8924b77740 1 -- 172.21.15.18:0/4022358077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89080f42e0 con 0x561dfbb50610 2024-04-16T17:25:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.965+0000 7f8904c00640 1 -- 172.21.15.18:0/4022358077 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f89080b9ff0 con 0x561dfbb50610 2024-04-16T17:25:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.965+0000 7f8924b77740 10 client.19932 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.965+0000 7f8924b77740 1 -- 172.21.15.18:0/4022358077 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561dfbb5b910 con 0x561dfbb50610 2024-04-16T17:25:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.966+0000 7f8904c00640 1 -- 172.21.15.18:0/4022358077 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8908005690 con 0x561dfbb50610 2024-04-16T17:25:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.966+0000 7f8904c00640 1 -- 172.21.15.18:0/4022358077 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f89080ef670 con 0x561dfbb50610 2024-04-16T17:25:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.966+0000 7f8924b77740 10 client.19932 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.966+0000 7f8924b77740 10 client.19932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.966+0000 7f8924b77740 20 client.19932 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.966+0000 7f8924b77740 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561dfbb68c80 0x561dfbb6b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.967+0000 7f8924b77740 4 client.19932 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.967+0000 7f8924b77740 1 -- 172.21.15.18:0/4022358077 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f89080f42e0 con 0x561dfbb68c80 2024-04-16T17:25:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.967+0000 7f891aa00640 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561dfbb68c80 0x561dfbb6b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.968+0000 7f891aa00640 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561dfbb68c80 0x561dfbb6b0a0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67d033238c99312 server_cookie=77b3a56a87951350 in_seq=0 out_seq=0 2024-04-16T17:25:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.968+0000 7f8904c00640 10 client.19932 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.976+0000 7f8904c00640 1 -- 172.21.15.18:0/4022358077 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f89080f42e0 con 0x561dfbb68c80 2024-04-16T17:25:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.976+0000 7f8904c00640 10 client.19932 handle_command_reply: tid=0 2024-04-16T17:25:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.977+0000 7f8924b77740 1 client.19932 shutdown 2024-04-16T17:25:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8924b77740 1 -- 172.21.15.18:0/4022358077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb50610 msgr2=0x561dfbb55c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8924b77740 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb50610 0x561dfbb55c00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f89080bdce0 tx=0x7f89080035f0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8924b77740 1 -- 172.21.15.18:0/4022358077 shutdown_connections 2024-04-16T17:25:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8924b77740 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561dfbb68c80 0x561dfbb6b0a0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8924b77740 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8924b77740 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb50610 0x561dfbb55c00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8924b77740 1 --2- 172.21.15.18:0/4022358077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb4f8a0 0x561dfbb58870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8904c00640 0 client.19932 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.978+0000 7f8924b77740 1 -- 172.21.15.18:0/4022358077 >> 172.21.15.18:0/4022358077 conn(0x561dfbaea5d0 msgr2=0x561dfbb4c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.979+0000 7f8924b77740 1 -- 172.21.15.18:0/4022358077 shutdown_connections 2024-04-16T17:25:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.979+0000 7f8924b77740 1 -- 172.21.15.18:0/4022358077 wait complete. 2024-04-16T17:25:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:40.980+0000 7f8924b77740 20 client.19932 trim_cache size 0 max 16384 2024-04-16T17:25:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.020+0000 7f8924b77740 1 Processor -- start 2024-04-16T17:25:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f8924b77740 1 -- start start 2024-04-16T17:25:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f8924b77740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb55ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f8924b77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 0x561dfbb64810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f8924b77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb70970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f8924b77740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc012910 con 0x561dfbb4f8a0 2024-04-16T17:25:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f8924b77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc0101e0 con 0x561dfbb58e20 2024-04-16T17:25:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f8924b77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc00e050 con 0x561dfbb50610 2024-04-16T17:25:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f891a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 0x561dfbb64810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f891aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb55ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f891aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb55ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42846/0 (socket says 172.21.15.18:42846) 2024-04-16T17:25:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.021+0000 7f891aa00640 1 -- 172.21.15.18:0/904733668 learned_addr learned my addr 172.21.15.18:0/904733668 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.022+0000 7f891b400640 1 --2- 172.21.15.18:0/904733668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb70970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.022+0000 7f891aa00640 1 -- 172.21.15.18:0/904733668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 msgr2=0x561dfbb64810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.022+0000 7f891aa00640 1 --2- 172.21.15.18:0/904733668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 0x561dfbb64810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.022+0000 7f891aa00640 1 -- 172.21.15.18:0/904733668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 msgr2=0x561dfbb70970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.022+0000 7f891aa00640 1 --2- 172.21.15.18:0/904733668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb70970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.022+0000 7f891aa00640 1 -- 172.21.15.18:0/904733668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dfbb4eb70 con 0x561dfbb4f8a0 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.022+0000 7f891aa00640 1 --2- 172.21.15.18:0/904733668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb55ba0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f890c000c00 tx=0x7f890c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb0bf9dbb25b03e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.023+0000 7f8907400640 1 -- 172.21.15.18:0/904733668 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c01c050 con 0x561dfbb4f8a0 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.023+0000 7f8907400640 1 -- 172.21.15.18:0/904733668 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c01c6d0 con 0x561dfbb4f8a0 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.023+0000 7f8907400640 1 -- 172.21.15.18:0/904733668 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c010b50 con 0x561dfbb4f8a0 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.023+0000 7f8924b77740 1 -- 172.21.15.18:0/904733668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 msgr2=0x561dfbb55ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.023+0000 7f8924b77740 1 --2- 172.21.15.18:0/904733668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb55ba0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f890c000c00 tx=0x7f890c012820 comp rx=0 tx=0).stop 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.024+0000 7f8924b77740 1 -- 172.21.15.18:0/904733668 shutdown_connections 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.024+0000 7f8924b77740 1 --2- 172.21.15.18:0/904733668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb58e20 0x561dfbb70970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.024+0000 7f8924b77740 1 --2- 172.21.15.18:0/904733668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb50610 0x561dfbb64810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.024+0000 7f8924b77740 1 --2- 172.21.15.18:0/904733668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb55ba0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.024+0000 7f8924b77740 1 -- 172.21.15.18:0/904733668 >> 172.21.15.18:0/904733668 conn(0x561dfbaea5d0 msgr2=0x561dfbb55490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.024+0000 7f8924b77740 1 -- 172.21.15.18:0/904733668 shutdown_connections 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.024+0000 7f8924b77740 1 -- 172.21.15.18:0/904733668 wait complete. 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.024+0000 7f8924b77740 1 -- start start 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.025+0000 7f8924b77740 1 Processor -- start 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f8924b77740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb5cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f8924b77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb50610 0x561dfbb5d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f8924b77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb58e20 0x561dfbbce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f8924b77740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc00e6a0 con 0x561dfbb4f8a0 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f8924b77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc00fb10 con 0x561dfbb50610 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f8924b77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc00ed70 con 0x561dfbb58e20 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f8924b77740 10 client.0 _ll_register_callbacks cb 0x7f8908114070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb5cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb5cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42854/0 (socket says 172.21.15.18:42854) 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891aa00640 1 -- 172.21.15.18:0/806926410 learned_addr learned my addr 172.21.15.18:0/806926410 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891a000640 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb50610 0x561dfbb5d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891b400640 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb58e20 0x561dfbbce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891b400640 1 -- 172.21.15.18:0/806926410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb50610 msgr2=0x561dfbb5d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891b400640 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb50610 0x561dfbb5d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891b400640 1 -- 172.21.15.18:0/806926410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 msgr2=0x561dfbb5cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891b400640 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb5cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.026+0000 7f891b400640 1 -- 172.21.15.18:0/806926410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dfbbce720 con 0x561dfbb58e20 2024-04-16T17:25:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.027+0000 7f891b400640 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb58e20 0x561dfbbce1e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f891c075320 tx=0x7f8914004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff50ab9c1550972 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.027+0000 7f8904c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.027+0000 7f8904c00640 1 -- 172.21.15.18:0/806926410 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891402b060 con 0x561dfbb58e20 2024-04-16T17:25:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.027+0000 7f8904c00640 1 -- 172.21.15.18:0/806926410 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f891401bd50 con 0x561dfbb58e20 2024-04-16T17:25:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.027+0000 7f8924b77740 1 -- 172.21.15.18:0/806926410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890c01c6d0 con 0x561dfbb58e20 2024-04-16T17:25:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.027+0000 7f8904c00640 1 -- 172.21.15.18:0/806926410 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8914032050 con 0x561dfbb58e20 2024-04-16T17:25:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.027+0000 7f8924b77740 10 client.30784 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.027+0000 7f8924b77740 1 -- 172.21.15.18:0/806926410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561dfbbcfa00 con 0x561dfbb58e20 2024-04-16T17:25:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.028+0000 7f8904c00640 1 -- 172.21.15.18:0/806926410 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c01c6d0 con 0x561dfbb58e20 2024-04-16T17:25:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.028+0000 7f8904c00640 1 -- 172.21.15.18:0/806926410 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f8914018040 con 0x561dfbb58e20 2024-04-16T17:25:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.028+0000 7f8924b77740 10 client.30784 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.028+0000 7f8924b77740 10 client.30784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.028+0000 7f8924b77740 20 client.30784 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.028+0000 7f8924b77740 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561dfbbd01b0 0x561dfbbd25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.028+0000 7f8924b77740 4 client.30784 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:25:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.028+0000 7f8924b77740 1 -- 172.21.15.18:0/806926410 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dfbac21a0 con 0x561dfbbd01b0 2024-04-16T17:25:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.167+0000 7f891aa00640 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561dfbbd01b0 0x561dfbbd25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.168+0000 7f891aa00640 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561dfbbd01b0 0x561dfbbd25d0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e36ae4846fd95590 server_cookie=25c9f5f1453c14b9 in_seq=0 out_seq=0 2024-04-16T17:25:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:41.168+0000 7f8904c00640 10 client.30784 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:40 smithi022 ceph-mon[36023]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:25:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:40 smithi022 ceph-mon[36023]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:25:41.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:40 smithi043 ceph-mon[36901]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:25:41.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:40 smithi043 ceph-mon[36901]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:25:41.284 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:35 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:36 smithi018 ceph-mon[31776]: pgmap v666: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 430 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 353 op/s 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:36 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi043 to 175.2M 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:36 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi043 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:36 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:36 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:37 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:37 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:37 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:37 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:37 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:37 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:37 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:37 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:38 smithi018 ceph-mon[31776]: pgmap v667: 129 pgs: 129 active+clean; 440 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:38 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi022 to 175.2M 2024-04-16T17:25:41.286 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:38 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi022 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:38 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:38 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: osdmap e133: 12 total, 12 up, 12 in 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: pgmap v669: 129 pgs: 129 active+clean; 439 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:25:41.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:25:41.288 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:25:41.288 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:39 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:25:41.288 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:40 smithi018 ceph-mon[31776]: Adjusting osd_memory_target on smithi018 to 177.5M 2024-04-16T17:25:41.288 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:40 smithi018 ceph-mon[31776]: Unable to set osd_memory_target on smithi018 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:25:41.617 INFO:tasks.workunit.client.0.smithi018.stdout: 10 101929 10.68 MB/sec execute 353 sec latency 504.559 ms 2024-04-16T17:25:42.058 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:41 smithi018 ceph-mon[31776]: pgmap v670: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-04-16T17:25:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:41 smithi022 ceph-mon[36023]: pgmap v670: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-04-16T17:25:42.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:41 smithi043 ceph-mon[36901]: pgmap v670: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-04-16T17:25:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.308+0000 7f8904c00640 1 -- 172.21.15.18:0/806926410 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561dfbac21a0 con 0x561dfbbd01b0 2024-04-16T17:25:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.308+0000 7f8904c00640 10 client.30784 handle_command_reply: tid=0 2024-04-16T17:25:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.308+0000 7f8924b77740 1 client.30784 shutdown 2024-04-16T17:25:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.308+0000 7f8924b77740 1 -- 172.21.15.18:0/806926410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb58e20 msgr2=0x561dfbbce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.308+0000 7f8924b77740 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb58e20 0x561dfbbce1e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f891c075320 tx=0x7f8914004ab0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.309+0000 7f8924b77740 1 -- 172.21.15.18:0/806926410 shutdown_connections 2024-04-16T17:25:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.309+0000 7f8924b77740 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x561dfbbd01b0 0x561dfbbd25d0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.309+0000 7f8924b77740 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dfbb58e20 0x561dfbbce1e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.309+0000 7f8924b77740 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561dfbb50610 0x561dfbb5d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.309+0000 7f8924b77740 1 --2- 172.21.15.18:0/806926410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561dfbb4f8a0 0x561dfbb5cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.309+0000 7f8924b77740 1 -- 172.21.15.18:0/806926410 >> 172.21.15.18:0/806926410 conn(0x561dfbaea5d0 msgr2=0x561dfbb54960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.309+0000 7f8904c00640 0 client.30784 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.310+0000 7f8924b77740 1 -- 172.21.15.18:0/806926410 shutdown_connections 2024-04-16T17:25:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.310+0000 7f8924b77740 1 -- 172.21.15.18:0/806926410 wait complete. 2024-04-16T17:25:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.310+0000 7f8924b77740 20 client.30784 trim_cache size 0 max 16384 2024-04-16T17:25:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:25:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:25:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:25:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:25:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.313+0000 7f8921e00640 1 -- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d90 msgr2=0x7f89000a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.313+0000 7f8921e00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d90 0x7f89000a6250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f891c06b700 tx=0x7f8914009500 comp rx=0 tx=0).stop 2024-04-16T17:25:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.314+0000 7f8921e00640 1 -- 172.21.15.18:0/804593812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19f610 msgr2=0x7f891c1a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.314+0000 7f8921e00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19f610 0x7f891c1a3990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8908005ab0 tx=0x7f890802c880 comp rx=0 tx=0).stop 2024-04-16T17:25:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.315+0000 7f8921e00640 1 -- 172.21.15.18:0/804593812 shutdown_connections 2024-04-16T17:25:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.315+0000 7f8921e00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f89000a3d90 0x7f89000a6250 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.315+0000 7f8921e00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f891c1a3ed0 0x7f891c1a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.315+0000 7f8921e00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c19f610 0x7f891c1a3990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.315+0000 7f8921e00640 1 --2- 172.21.15.18:0/804593812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f891c0743d0 0x7f891c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.316+0000 7f8921e00640 1 -- 172.21.15.18:0/804593812 >> 172.21.15.18:0/804593812 conn(0x7f891c0fdac0 msgr2=0x7f891c114310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.320+0000 7f8921e00640 1 -- 172.21.15.18:0/804593812 shutdown_connections 2024-04-16T17:25:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:42.320+0000 7f8921e00640 1 -- 172.21.15.18:0/804593812 wait complete. 2024-04-16T17:25:42.617 INFO:tasks.workunit.client.0.smithi018.stdout: 10 102063 10.68 MB/sec execute 354 sec latency 837.024 ms 2024-04-16T17:25:43.617 INFO:tasks.workunit.client.0.smithi018.stdout: 10 102287 10.68 MB/sec execute 355 sec latency 163.712 ms 2024-04-16T17:25:44.174 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:43 smithi022 ceph-mon[36023]: pgmap v671: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-04-16T17:25:44.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:43 smithi043 ceph-mon[36901]: pgmap v671: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-04-16T17:25:44.618 INFO:tasks.workunit.client.0.smithi018.stdout: 10 102461 10.68 MB/sec execute 356 sec latency 233.274 ms 2024-04-16T17:25:45.618 INFO:tasks.workunit.client.0.smithi018.stdout: 10 102652 10.66 MB/sec execute 357 sec latency 200.052 ms 2024-04-16T17:25:46.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:45 smithi022 ceph-mon[36023]: pgmap v672: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-16T17:25:46.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:45 smithi043 ceph-mon[36901]: pgmap v672: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-16T17:25:46.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:43 smithi018 ceph-mon[31776]: pgmap v671: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-04-16T17:25:46.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:45 smithi018 ceph-mon[31776]: pgmap v672: 129 pgs: 129 active+clean; 451 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-16T17:25:46.618 INFO:tasks.workunit.client.0.smithi018.stdout: 10 102902 10.65 MB/sec execute 358 sec latency 302.667 ms 2024-04-16T17:25:47.618 INFO:tasks.workunit.client.0.smithi018.stdout: 10 103129 10.65 MB/sec execute 359 sec latency 202.293 ms 2024-04-16T17:25:48.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:47 smithi043 ceph-mon[36901]: pgmap v673: 129 pgs: 129 active+clean; 440 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-04-16T17:25:48.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:47 smithi022 ceph-mon[36023]: pgmap v673: 129 pgs: 129 active+clean; 440 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-04-16T17:25:48.618 INFO:tasks.workunit.client.0.smithi018.stdout: 10 103513 10.70 MB/sec execute 360 sec latency 174.038 ms 2024-04-16T17:25:49.618 INFO:tasks.workunit.client.0.smithi018.stdout: 10 103900 10.73 MB/sec execute 361 sec latency 169.182 ms 2024-04-16T17:25:50.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:49 smithi043 ceph-mon[36901]: pgmap v674: 129 pgs: 129 active+clean; 440 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-04-16T17:25:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:49 smithi022 ceph-mon[36023]: pgmap v674: 129 pgs: 129 active+clean; 440 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-04-16T17:25:50.620 INFO:tasks.workunit.client.0.smithi018.stdout: 10 104171 10.75 MB/sec execute 362 sec latency 165.651 ms 2024-04-16T17:25:51.620 INFO:tasks.workunit.client.0.smithi018.stdout: 10 104374 10.74 MB/sec execute 363 sec latency 241.582 ms 2024-04-16T17:25:52.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:51 smithi043 ceph-mon[36901]: pgmap v675: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-04-16T17:25:52.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:51 smithi022 ceph-mon[36023]: pgmap v675: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-04-16T17:25:52.623 INFO:tasks.workunit.client.0.smithi018.stdout: 10 104589 10.74 MB/sec execute 364 sec latency 274.027 ms 2024-04-16T17:25:53.623 INFO:tasks.workunit.client.0.smithi018.stdout: 10 104822 10.74 MB/sec execute 365 sec latency 187.015 ms 2024-04-16T17:25:54.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:53 smithi043 ceph-mon[36901]: pgmap v676: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 324 op/s 2024-04-16T17:25:54.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:25:54.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:25:54.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:53 smithi022 ceph-mon[36023]: pgmap v676: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 324 op/s 2024-04-16T17:25:54.623 INFO:tasks.workunit.client.0.smithi018.stdout: 10 105127 10.77 MB/sec execute 366 sec latency 189.836 ms 2024-04-16T17:25:54.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.646+0000 7fb646200640 1 -- 172.21.15.18:0/1311003571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6380986e0 msgr2=0x7fb638098b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.646+0000 7fb646200640 1 --2- 172.21.15.18:0/1311003571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6380986e0 0x7fb638098b60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb640067e70 tx=0x7fb640098090 comp rx=0 tx=0).stop 2024-04-16T17:25:54.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.647+0000 7fb646200640 1 -- 172.21.15.18:0/1311003571 shutdown_connections 2024-04-16T17:25:54.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.647+0000 7fb646200640 1 --2- 172.21.15.18:0/1311003571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6380986e0 0x7fb638098b60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.647+0000 7fb646200640 1 --2- 172.21.15.18:0/1311003571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6380051f0 0x7fb638005670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.647+0000 7fb646200640 1 --2- 172.21.15.18:0/1311003571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63809acf0 0x7fb638004cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.647+0000 7fb646200640 1 -- 172.21.15.18:0/1311003571 >> 172.21.15.18:0/1311003571 conn(0x7fb63800a740 msgr2=0x7fb63800ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:54.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.647+0000 7fb646200640 1 -- 172.21.15.18:0/1311003571 shutdown_connections 2024-04-16T17:25:54.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.647+0000 7fb646200640 1 -- 172.21.15.18:0/1311003571 wait complete. 2024-04-16T17:25:54.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.649+0000 7fb646200640 1 Processor -- start 2024-04-16T17:25:54.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.649+0000 7fb646200640 1 -- start start 2024-04-16T17:25:54.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.650+0000 7fb646200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6380051f0 0x7fb638138640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.650+0000 7fb646200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6380986e0 0x7fb638138b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.650+0000 7fb646200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63809acf0 0x7fb6381327b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.650+0000 7fb646200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6380ac4f0 con 0x7fb6380051f0 2024-04-16T17:25:54.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.650+0000 7fb646200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6380ac370 con 0x7fb6380986e0 2024-04-16T17:25:54.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.650+0000 7fb646200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6380ac670 con 0x7fb63809acf0 2024-04-16T17:25:54.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.651+0000 7fb645800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63809acf0 0x7fb6381327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.651+0000 7fb645800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63809acf0 0x7fb6381327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53258/0 (socket says 172.21.15.18:53258) 2024-04-16T17:25:54.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.651+0000 7fb645800640 1 -- 172.21.15.18:0/853577843 learned_addr learned my addr 172.21.15.18:0/853577843 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:54.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.651+0000 7fb63fe00640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6380986e0 0x7fb638138b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.652+0000 7fb63fe00640 1 -- 172.21.15.18:0/853577843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63809acf0 msgr2=0x7fb6381327b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.652+0000 7fb63fe00640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63809acf0 0x7fb6381327b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.652+0000 7fb63fe00640 1 -- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6380051f0 msgr2=0x7fb638138640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.652+0000 7fb63fe00640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6380051f0 0x7fb638138640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.652+0000 7fb63fe00640 1 -- 172.21.15.18:0/853577843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb638132f80 con 0x7fb6380986e0 2024-04-16T17:25:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.652+0000 7fb63fe00640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6380986e0 0x7fb638138b80 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d6d0 tx=0x7fb62c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:54.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.653+0000 7fb63d600640 1 -- 172.21.15.18:0/853577843 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c017070 con 0x7fb6380986e0 2024-04-16T17:25:54.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.653+0000 7fb63d600640 1 -- 172.21.15.18:0/853577843 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62c010460 con 0x7fb6380986e0 2024-04-16T17:25:54.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.654+0000 7fb646200640 1 -- 172.21.15.18:0/853577843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb638133270 con 0x7fb6380986e0 2024-04-16T17:25:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.655+0000 7fb63d600640 1 -- 172.21.15.18:0/853577843 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00f620 con 0x7fb6380986e0 2024-04-16T17:25:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.655+0000 7fb646200640 1 -- 172.21.15.18:0/853577843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63813f490 con 0x7fb6380986e0 2024-04-16T17:25:54.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.662+0000 7fb63d600640 1 -- 172.21.15.18:0/853577843 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb62c01a070 con 0x7fb6380986e0 2024-04-16T17:25:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.663+0000 7fb64a454740 1 Processor -- start 2024-04-16T17:25:54.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.664+0000 7fb63d600640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6280a3c80 0x7fb6280a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.666+0000 7fb64a454740 1 -- start start 2024-04-16T17:25:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.666+0000 7fb64a454740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe77770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.666+0000 7fb64a454740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 0x5578dbe80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.666+0000 7fb64a454740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe82c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.666+0000 7fb64a454740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5578dbddd120 con 0x5578dbe77cb0 2024-04-16T17:25:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.666+0000 7fb64a454740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5578dbd83340 con 0x5578dbe77390 2024-04-16T17:25:54.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.666+0000 7fb64a454740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578dbdd90e0 con 0x5578dbe80890 2024-04-16T17:25:54.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.667+0000 7fb645800640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6280a3c80 0x7fb6280a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb63d600640 1 -- 172.21.15.18:0/853577843 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fb62c099c50 con 0x7fb6380986e0 2024-04-16T17:25:54.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb644e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe77770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb644e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe77770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56042/0 (socket says 172.21.15.18:56042) 2024-04-16T17:25:54.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb644e00640 1 -- 172.21.15.18:0/3781465452 learned_addr learned my addr 172.21.15.18:0/3781465452 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:54.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb645800640 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe82c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb644e00640 1 -- 172.21.15.18:0/3781465452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 msgr2=0x5578dbe82c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:54.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb644e00640 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe82c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb644e00640 1 -- 172.21.15.18:0/3781465452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 msgr2=0x5578dbe80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:54.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb644e00640 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 0x5578dbe80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.668+0000 7fb644e00640 1 -- 172.21.15.18:0/3781465452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dbbb97d0 con 0x5578dbe77390 2024-04-16T17:25:54.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.669+0000 7fb645800640 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe82c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:54.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.669+0000 7fb644e00640 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe77770 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb634009c40 tx=0x7fb63400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92b7e65c843869d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:54.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.670+0000 7fb645800640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6280a3c80 0x7fb6280a6140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb6400985d0 tx=0x7fb640050650 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.673+0000 7fb632000640 1 -- 172.21.15.18:0/3781465452 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634005020 con 0x5578dbe77390 2024-04-16T17:25:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.673+0000 7fb632000640 1 -- 172.21.15.18:0/3781465452 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6340047a0 con 0x5578dbe77390 2024-04-16T17:25:54.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.673+0000 7fb632000640 1 -- 172.21.15.18:0/3781465452 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634010720 con 0x5578dbe77390 2024-04-16T17:25:54.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.674+0000 7fb64a454740 1 -- 172.21.15.18:0/3781465452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 msgr2=0x5578dbe77770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.674+0000 7fb64a454740 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe77770 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb634009c40 tx=0x7fb63400e720 comp rx=0 tx=0).stop 2024-04-16T17:25:54.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.674+0000 7fb64a454740 1 -- 172.21.15.18:0/3781465452 shutdown_connections 2024-04-16T17:25:54.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.674+0000 7fb64a454740 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe82c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.674+0000 7fb64a454740 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 0x5578dbe80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.674+0000 7fb64a454740 1 --2- 172.21.15.18:0/3781465452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe77770 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.674+0000 7fb64a454740 1 -- 172.21.15.18:0/3781465452 >> 172.21.15.18:0/3781465452 conn(0x5578dbe71ac0 msgr2=0x5578dbe71ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:54.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.675+0000 7fb64a454740 1 -- 172.21.15.18:0/3781465452 shutdown_connections 2024-04-16T17:25:54.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.675+0000 7fb64a454740 1 -- 172.21.15.18:0/3781465452 wait complete. 2024-04-16T17:25:54.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.675+0000 7fb64a454740 1 -- start start 2024-04-16T17:25:54.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.676+0000 7fb64a454740 1 Processor -- start 2024-04-16T17:25:54.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.677+0000 7fb64a454740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe7d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.678+0000 7fb64a454740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 0x5578dbe7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.678+0000 7fb64a454740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe7e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.678+0000 7fb64a454740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5578dbe83680 con 0x5578dbe77cb0 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.678+0000 7fb64a454740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5578dbd96710 con 0x5578dbe77390 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.678+0000 7fb64a454740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578dbe83800 con 0x5578dbe80890 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.678+0000 7fb64a454740 10 client.0 _ll_register_callbacks cb 0x5578dbd91af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.679+0000 7fb644e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe7d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.679+0000 7fb644e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe7d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56046/0 (socket says 172.21.15.18:56046) 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.679+0000 7fb644e00640 1 -- 172.21.15.18:0/4114983283 learned_addr learned my addr 172.21.15.18:0/4114983283 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.679+0000 7fb63fe00640 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 0x5578dbe7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.679+0000 7fb645800640 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe7e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.680+0000 7fb644e00640 1 -- 172.21.15.18:0/4114983283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 msgr2=0x5578dbe7e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:54.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.680+0000 7fb644e00640 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe7e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.680+0000 7fb644e00640 1 -- 172.21.15.18:0/4114983283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 msgr2=0x5578dbe7dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:54.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.680+0000 7fb644e00640 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 0x5578dbe7dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.681+0000 7fb644e00640 1 -- 172.21.15.18:0/4114983283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dbaef3f0 con 0x5578dbe77390 2024-04-16T17:25:54.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.681+0000 7fb645800640 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe7e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:54.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.682+0000 7fb644e00640 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe7d840 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb634009c10 tx=0x7fb634004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa874fb88913f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:54.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.682+0000 7fb626a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.683+0000 7fb626a00640 1 -- 172.21.15.18:0/4114983283 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63401d040 con 0x5578dbe77390 2024-04-16T17:25:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.683+0000 7fb626a00640 1 -- 172.21.15.18:0/4114983283 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fb634004250 con 0x5578dbe77390 2024-04-16T17:25:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.683+0000 7fb64a454740 1 -- 172.21.15.18:0/4114983283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6340047a0 con 0x5578dbe77390 2024-04-16T17:25:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.684+0000 7fb626a00640 1 -- 172.21.15.18:0/4114983283 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb634004dc0 con 0x5578dbe77390 2024-04-16T17:25:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.684+0000 7fb64a454740 10 client.30959 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.684+0000 7fb64a454740 1 -- 172.21.15.18:0/4114983283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5578dbaaf0f0 con 0x5578dbe77390 2024-04-16T17:25:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.685+0000 7fb626a00640 1 -- 172.21.15.18:0/4114983283 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6340047a0 con 0x5578dbe77390 2024-04-16T17:25:54.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.685+0000 7fb626a00640 1 -- 172.21.15.18:0/4114983283 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fb634003a50 con 0x5578dbe77390 2024-04-16T17:25:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.686+0000 7fb64a454740 10 client.30959 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.686+0000 7fb64a454740 10 client.30959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.686+0000 7fb64a454740 20 client.30959 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.686+0000 7fb64a454740 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5578dbe90bb0 0x5578dbe92fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.686+0000 7fb64a454740 4 client.30959 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.686+0000 7fb64a454740 1 -- 172.21.15.18:0/4114983283 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578dbdeab40 con 0x5578dbe90bb0 2024-04-16T17:25:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.687+0000 7fb63fe00640 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5578dbe90bb0 0x5578dbe92fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.689+0000 7fb63fe00640 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5578dbe90bb0 0x5578dbe92fa0 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc43a23987931f6c server_cookie=65fbc459d0423f39 in_seq=0 out_seq=0 2024-04-16T17:25:54.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.689+0000 7fb626a00640 10 client.30959 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:54.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.693+0000 7fb626a00640 1 -- 172.21.15.18:0/4114983283 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578dbdeab40 con 0x5578dbe90bb0 2024-04-16T17:25:54.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.693+0000 7fb626a00640 10 client.30959 handle_command_reply: tid=0 2024-04-16T17:25:54.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.693+0000 7fb64a454740 1 client.30959 shutdown 2024-04-16T17:25:54.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb64a454740 1 -- 172.21.15.18:0/4114983283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 msgr2=0x5578dbe7d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb64a454740 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe7d840 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb634009c10 tx=0x7fb634004bd0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb64a454740 1 -- 172.21.15.18:0/4114983283 shutdown_connections 2024-04-16T17:25:54.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb64a454740 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5578dbe90bb0 0x5578dbe92fa0 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb64a454740 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe80890 0x5578dbe7e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb626a00640 0 client.30959 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:54.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb64a454740 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe77cb0 0x5578dbe7dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb64a454740 1 --2- 172.21.15.18:0/4114983283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77390 0x5578dbe7d840 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.694+0000 7fb64a454740 1 -- 172.21.15.18:0/4114983283 >> 172.21.15.18:0/4114983283 conn(0x5578dbe71ac0 msgr2=0x5578dbe73810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:54.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.695+0000 7fb64a454740 1 -- 172.21.15.18:0/4114983283 shutdown_connections 2024-04-16T17:25:54.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.695+0000 7fb64a454740 1 -- 172.21.15.18:0/4114983283 wait complete. 2024-04-16T17:25:54.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.695+0000 7fb64a454740 20 client.30959 trim_cache size 0 max 16384 2024-04-16T17:25:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.737+0000 7fb64a454740 1 Processor -- start 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.737+0000 7fb64a454740 1 -- start start 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb64a454740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbe7ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb64a454740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbe8c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb64a454740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe71ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb64a454740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb63814be50 con 0x5578dbe80890 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb64a454740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb638149720 con 0x5578dbe77cb0 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb64a454740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb638147590 con 0x5578dbe77390 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb644e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbe7ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb63fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbe8c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb644e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbe7ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53272/0 (socket says 172.21.15.18:53272) 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb645800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe71ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.738+0000 7fb644e00640 1 -- 172.21.15.18:0/191455345 learned_addr learned my addr 172.21.15.18:0/191455345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:54.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.739+0000 7fb644e00640 1 -- 172.21.15.18:0/191455345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 msgr2=0x5578dbe8c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:54.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.739+0000 7fb644e00640 1 --2- 172.21.15.18:0/191455345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbe8c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.739+0000 7fb644e00640 1 -- 172.21.15.18:0/191455345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 msgr2=0x5578dbe71ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:54.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.739+0000 7fb644e00640 1 --2- 172.21.15.18:0/191455345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe71ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.739+0000 7fb644e00640 1 -- 172.21.15.18:0/191455345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dbe718f0 con 0x5578dbe77390 2024-04-16T17:25:54.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.739+0000 7fb645800640 1 --2- 172.21.15.18:0/191455345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe71ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:54.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.741+0000 7fb644e00640 1 --2- 172.21.15.18:0/191455345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbe7ff70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb634000c00 tx=0x7fb634004d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc97f7c487ba9f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:54.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.741+0000 7fb631600640 1 -- 172.21.15.18:0/191455345 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634010670 con 0x5578dbe77390 2024-04-16T17:25:54.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.741+0000 7fb631600640 1 -- 172.21.15.18:0/191455345 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb634027070 con 0x5578dbe77390 2024-04-16T17:25:54.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.741+0000 7fb631600640 1 -- 172.21.15.18:0/191455345 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634004340 con 0x5578dbe77390 2024-04-16T17:25:54.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.742+0000 7fb64a454740 1 -- 172.21.15.18:0/191455345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 msgr2=0x5578dbe7ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.742+0000 7fb64a454740 1 --2- 172.21.15.18:0/191455345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbe7ff70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb634000c00 tx=0x7fb634004d40 comp rx=0 tx=0).stop 2024-04-16T17:25:54.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.743+0000 7fb64a454740 1 -- 172.21.15.18:0/191455345 shutdown_connections 2024-04-16T17:25:54.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.743+0000 7fb64a454740 1 --2- 172.21.15.18:0/191455345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe71ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.743+0000 7fb64a454740 1 --2- 172.21.15.18:0/191455345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbe8c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.743+0000 7fb64a454740 1 --2- 172.21.15.18:0/191455345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbe7ff70 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.743+0000 7fb64a454740 1 -- 172.21.15.18:0/191455345 >> 172.21.15.18:0/191455345 conn(0x5578dbe71ac0 msgr2=0x5578dbe7d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:54.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.743+0000 7fb64a454740 1 -- 172.21.15.18:0/191455345 shutdown_connections 2024-04-16T17:25:54.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.744+0000 7fb64a454740 1 -- 172.21.15.18:0/191455345 wait complete. 2024-04-16T17:25:54.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.744+0000 7fb64a454740 1 -- start start 2024-04-16T17:25:54.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.744+0000 7fb64a454740 1 Processor -- start 2024-04-16T17:25:54.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.745+0000 7fb64a454740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbef23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.745+0000 7fb64a454740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbef2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.745+0000 7fb64a454740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe88c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.745+0000 7fb64a454740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb638147be0 con 0x5578dbe80890 2024-04-16T17:25:54.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.745+0000 7fb64a454740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb638149050 con 0x5578dbe77cb0 2024-04-16T17:25:54.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.745+0000 7fb64a454740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381482b0 con 0x5578dbe77390 2024-04-16T17:25:54.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.745+0000 7fb64a454740 10 client.0 _ll_register_callbacks cb 0x7fb634009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.746+0000 7fb63fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbef2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.746+0000 7fb63fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbef2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56068/0 (socket says 172.21.15.18:56068) 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.746+0000 7fb63fe00640 1 -- 172.21.15.18:0/1861137391 learned_addr learned my addr 172.21.15.18:0/1861137391 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.746+0000 7fb644e00640 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbef23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.746+0000 7fb645800640 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe88c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.747+0000 7fb63fe00640 1 -- 172.21.15.18:0/1861137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 msgr2=0x5578dbef23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.747+0000 7fb63fe00640 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbef23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.747+0000 7fb63fe00640 1 -- 172.21.15.18:0/1861137391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 msgr2=0x5578dbe88c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.747+0000 7fb63fe00640 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe88c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.747+0000 7fb63fe00640 1 -- 172.21.15.18:0/1861137391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dbe7a390 con 0x5578dbe77cb0 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.747+0000 7fb644e00640 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbef23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.748+0000 7fb63fe00640 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbef2900 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d6a0 tx=0x7fb62c0a3270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a0f3385cfcf224 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:54.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.748+0000 7fb626a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:54.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.748+0000 7fb626a00640 1 -- 172.21.15.18:0/1861137391 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0985e0 con 0x5578dbe77cb0 2024-04-16T17:25:54.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.749+0000 7fb626a00640 1 -- 172.21.15.18:0/1861137391 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fb62c0a4760 con 0x5578dbe77cb0 2024-04-16T17:25:54.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.753+0000 7fb64a454740 1 -- 172.21.15.18:0/1861137391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb634027070 con 0x5578dbe77cb0 2024-04-16T17:25:54.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.753+0000 7fb626a00640 1 -- 172.21.15.18:0/1861137391 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb62c098780 con 0x5578dbe77cb0 2024-04-16T17:25:54.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.753+0000 7fb64a454740 10 client.30962 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:54.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.753+0000 7fb64a454740 1 -- 172.21.15.18:0/1861137391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5578dbf08080 con 0x5578dbe77cb0 2024-04-16T17:25:54.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb626a00640 1 -- 172.21.15.18:0/1861137391 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb634027070 con 0x5578dbe77cb0 2024-04-16T17:25:54.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb626a00640 1 -- 172.21.15.18:0/1861137391 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fb62c097900 con 0x5578dbe77cb0 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb64a454740 10 client.30962 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb64a454740 10 client.30962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb64a454740 20 client.30962 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb64a454740 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5578dbf08780 0x5578dbf0aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb64a454740 4 client.30962 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb64a454740 1 -- 172.21.15.18:0/1861137391 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578dbde49b0 con 0x5578dbf08780 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.754+0000 7fb644e00640 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5578dbf08780 0x5578dbf0aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.755+0000 7fb644e00640 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5578dbf08780 0x5578dbf0aba0 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c13ac8e8f12a54 server_cookie=bf28d79a2825336f in_seq=0 out_seq=0 2024-04-16T17:25:54.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.755+0000 7fb626a00640 10 client.30962 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:54.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.757+0000 7fb626a00640 1 -- 172.21.15.18:0/1861137391 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5578dbde49b0 con 0x5578dbf08780 2024-04-16T17:25:54.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.757+0000 7fb626a00640 10 client.30962 handle_command_reply: tid=0 2024-04-16T17:25:54.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.757+0000 7fb64a454740 1 client.30962 shutdown 2024-04-16T17:25:54.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.757+0000 7fb64a454740 1 -- 172.21.15.18:0/1861137391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 msgr2=0x5578dbef2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.757+0000 7fb64a454740 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbef2900 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d6a0 tx=0x7fb62c0a3270 comp rx=0 tx=0).stop 2024-04-16T17:25:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.758+0000 7fb64a454740 1 -- 172.21.15.18:0/1861137391 shutdown_connections 2024-04-16T17:25:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.758+0000 7fb64a454740 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5578dbf08780 0x5578dbf0aba0 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.758+0000 7fb64a454740 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5578dbe80890 0x5578dbe88c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.758+0000 7fb64a454740 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578dbe77cb0 0x5578dbef2900 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.758+0000 7fb626a00640 0 client.30962 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.758+0000 7fb64a454740 1 --2- 172.21.15.18:0/1861137391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578dbe77390 0x5578dbef23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.758+0000 7fb64a454740 1 -- 172.21.15.18:0/1861137391 >> 172.21.15.18:0/1861137391 conn(0x5578dbe71ac0 msgr2=0x5578dbe70b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:54.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.758+0000 7fb64a454740 1 -- 172.21.15.18:0/1861137391 shutdown_connections 2024-04-16T17:25:54.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.759+0000 7fb64a454740 1 -- 172.21.15.18:0/1861137391 wait complete. 2024-04-16T17:25:54.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.759+0000 7fb64a454740 20 client.30962 trim_cache size 0 max 16384 2024-04-16T17:25:54.763 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:25:54.763 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:25:54.763 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:25:54.763 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:25:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.763+0000 7fb646200640 1 -- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6280a3c80 msgr2=0x7fb6280a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.763+0000 7fb646200640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6280a3c80 0x7fb6280a6140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb6400985d0 tx=0x7fb640050650 comp rx=0 tx=0).stop 2024-04-16T17:25:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.763+0000 7fb646200640 1 -- 172.21.15.18:0/853577843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6380986e0 msgr2=0x7fb638138b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.763+0000 7fb646200640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6380986e0 0x7fb638138b80 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d6d0 tx=0x7fb62c00db90 comp rx=0 tx=0).stop 2024-04-16T17:25:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.764+0000 7fb646200640 1 -- 172.21.15.18:0/853577843 shutdown_connections 2024-04-16T17:25:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.764+0000 7fb646200640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb6280a3c80 0x7fb6280a6140 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.764+0000 7fb646200640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63809acf0 0x7fb6381327b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.764+0000 7fb646200640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6380986e0 0x7fb638138b80 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.764+0000 7fb646200640 1 --2- 172.21.15.18:0/853577843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6380051f0 0x7fb638138640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.764+0000 7fb646200640 1 -- 172.21.15.18:0/853577843 >> 172.21.15.18:0/853577843 conn(0x7fb63800a740 msgr2=0x7fb638093e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.764+0000 7fb646200640 1 -- 172.21.15.18:0/853577843 shutdown_connections 2024-04-16T17:25:54.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:54.764+0000 7fb646200640 1 -- 172.21.15.18:0/853577843 wait complete. 2024-04-16T17:25:54.800 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:25:54.800 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:25:55.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.057+0000 7ff614a00640 1 -- 172.21.15.18:0/1006732734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 msgr2=0x7ff610115170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.057+0000 7ff614a00640 1 --2- 172.21.15.18:0/1006732734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 0x7ff610115170 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff608009cd0 tx=0x7ff60802c0e0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.058+0000 7ff614a00640 1 -- 172.21.15.18:0/1006732734 shutdown_connections 2024-04-16T17:25:55.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.058+0000 7ff614a00640 1 --2- 172.21.15.18:0/1006732734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 0x7ff610115170 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.058+0000 7ff614a00640 1 --2- 172.21.15.18:0/1006732734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff610076e50 0x7ff610112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.058+0000 7ff614a00640 1 --2- 172.21.15.18:0/1006732734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff610076530 0x7ff610076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.058+0000 7ff614a00640 1 -- 172.21.15.18:0/1006732734 >> 172.21.15.18:0/1006732734 conn(0x7ff6100ffb70 msgr2=0x7ff610101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.058+0000 7ff614a00640 1 -- 172.21.15.18:0/1006732734 shutdown_connections 2024-04-16T17:25:55.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.058+0000 7ff614a00640 1 -- 172.21.15.18:0/1006732734 wait complete. 2024-04-16T17:25:55.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.060+0000 7ff614a00640 1 Processor -- start 2024-04-16T17:25:55.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.060+0000 7ff614a00640 1 -- start start 2024-04-16T17:25:55.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff614a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff610076530 0x7ff610073810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff614a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff610076e50 0x7ff61006c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff614a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 0x7ff61006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff614a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff6101188a0 con 0x7ff610076530 2024-04-16T17:25:55.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff614a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff610118720 con 0x7ff610076e50 2024-04-16T17:25:55.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff614a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff610118a20 con 0x7ff610112d80 2024-04-16T17:25:55.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 0x7ff61006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 0x7ff61006cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53296/0 (socket says 172.21.15.18:53296) 2024-04-16T17:25:55.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.061+0000 7ff60fe00640 1 -- 172.21.15.18:0/606242070 learned_addr learned my addr 172.21.15.18:0/606242070 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.062+0000 7ff60fe00640 1 -- 172.21.15.18:0/606242070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff610076e50 msgr2=0x7ff61006c890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.062+0000 7ff60fe00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff610076e50 0x7ff61006c890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.062+0000 7ff60fe00640 1 -- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff610076530 msgr2=0x7ff610073810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.062+0000 7ff60fe00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff610076530 0x7ff610073810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.062+0000 7ff60fe00640 1 -- 172.21.15.18:0/606242070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61006d390 con 0x7ff610112d80 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.062+0000 7ff60fe00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 0x7ff61006cdd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff608005e30 tx=0x7ff608032460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.063+0000 7ff607e00640 1 -- 172.21.15.18:0/606242070 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608039040 con 0x7ff610112d80 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.063+0000 7ff607e00640 1 -- 172.21.15.18:0/606242070 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff608044070 con 0x7ff610112d80 2024-04-16T17:25:55.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.063+0000 7ff607e00640 1 -- 172.21.15.18:0/606242070 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6080042a0 con 0x7ff610112d80 2024-04-16T17:25:55.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.065+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61006d620 con 0x7ff610112d80 2024-04-16T17:25:55.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.065+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff610071570 con 0x7ff610112d80 2024-04-16T17:25:55.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.067+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff610077c70 con 0x7ff610112d80 2024-04-16T17:25:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.074+0000 7ff607e00640 1 -- 172.21.15.18:0/606242070 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff608003970 con 0x7ff610112d80 2024-04-16T17:25:55.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.077+0000 7ff607e00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f40a3ee0 0x7ff5f40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.077+0000 7ff607e00640 1 -- 172.21.15.18:0/606242070 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7ff6080f2f70 con 0x7ff610112d80 2024-04-16T17:25:55.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.078+0000 7ff607e00640 1 -- 172.21.15.18:0/606242070 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff608073080 con 0x7ff610112d80 2024-04-16T17:25:55.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.078+0000 7ff60f400640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f40a3ee0 0x7ff5f40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.079+0000 7ff60f400640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f40a3ee0 0x7ff5f40a63a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff6000058c0 tx=0x7ff600005850 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.452+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff61004fe00 con 0x7ff610112d80 2024-04-16T17:25:55.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.453+0000 7ff607e00640 1 -- 172.21.15.18:0/606242070 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7ff61004fe00 con 0x7ff610112d80 2024-04-16T17:25:55.453 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:25:55.453 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:25:55.454 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:25:55.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.457+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f40a3ee0 msgr2=0x7ff5f40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.457+0000 7ff614a00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f40a3ee0 0x7ff5f40a63a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff6000058c0 tx=0x7ff600005850 comp rx=0 tx=0).stop 2024-04-16T17:25:55.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.457+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 msgr2=0x7ff61006cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.457+0000 7ff614a00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 0x7ff61006cdd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff608005e30 tx=0x7ff608032460 comp rx=0 tx=0).stop 2024-04-16T17:25:55.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.458+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 shutdown_connections 2024-04-16T17:25:55.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.458+0000 7ff614a00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff5f40a3ee0 0x7ff5f40a63a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.458+0000 7ff614a00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff610112d80 0x7ff61006cdd0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.458+0000 7ff614a00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff610076e50 0x7ff61006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.458+0000 7ff614a00640 1 --2- 172.21.15.18:0/606242070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff610076530 0x7ff610073810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.458+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 >> 172.21.15.18:0/606242070 conn(0x7ff6100ffb70 msgr2=0x7ff610101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.458+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 shutdown_connections 2024-04-16T17:25:55.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.459+0000 7ff614a00640 1 -- 172.21.15.18:0/606242070 wait complete. 2024-04-16T17:25:55.486 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:25:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.538+0000 7fdc89400640 1 -- 172.21.15.18:0/3208210893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076530 msgr2=0x7fdc84076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.538+0000 7fdc89400640 1 --2- 172.21.15.18:0/3208210893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076530 0x7fdc84076910 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009930 tx=0x7fdc6c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:25:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.539+0000 7fdc89400640 1 -- 172.21.15.18:0/3208210893 shutdown_connections 2024-04-16T17:25:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.539+0000 7fdc89400640 1 --2- 172.21.15.18:0/3208210893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc84112ea0 0x7fdc84115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.539+0000 7fdc89400640 1 --2- 172.21.15.18:0/3208210893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc84076e50 0x7fdc84112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.539+0000 7fdc89400640 1 --2- 172.21.15.18:0/3208210893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076530 0x7fdc84076910 secure :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009930 tx=0x7fdc6c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:25:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.539+0000 7fdc89400640 1 -- 172.21.15.18:0/3208210893 >> 172.21.15.18:0/3208210893 conn(0x7fdc840ffcd0 msgr2=0x7fdc841020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.539+0000 7fdc89400640 1 -- 172.21.15.18:0/3208210893 shutdown_connections 2024-04-16T17:25:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.539+0000 7fdc89400640 1 -- 172.21.15.18:0/3208210893 wait complete. 2024-04-16T17:25:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.542+0000 7fdc89400640 1 Processor -- start 2024-04-16T17:25:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.542+0000 7fdc89400640 1 -- start start 2024-04-16T17:25:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.543+0000 7fdc89400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076e50 0x7fdc8419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.543+0000 7fdc89400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc84112ea0 0x7fdc8419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.543+0000 7fdc82800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076e50 0x7fdc8419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.543+0000 7fdc82800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076e50 0x7fdc8419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53312/0 (socket says 172.21.15.18:53312) 2024-04-16T17:25:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.544+0000 7fdc89400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc841a3a00 0x7fdc841a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.544+0000 7fdc89400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84118a50 con 0x7fdc84112ea0 2024-04-16T17:25:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.544+0000 7fdc89400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841188d0 con 0x7fdc841a3a00 2024-04-16T17:25:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.544+0000 7fdc89400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84118bd0 con 0x7fdc84076e50 2024-04-16T17:25:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.544+0000 7fdc83200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc841a3a00 0x7fdc841a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.544+0000 7fdc83200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc841a3a00 0x7fdc841a3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56106/0 (socket says 172.21.15.18:56106) 2024-04-16T17:25:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.544+0000 7fdc82800640 1 -- 172.21.15.18:0/227701083 learned_addr learned my addr 172.21.15.18:0/227701083 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.545+0000 7fdc82800640 1 -- 172.21.15.18:0/227701083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc841a3a00 msgr2=0x7fdc841a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.546+0000 7fdc82800640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc841a3a00 0x7fdc841a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.546+0000 7fdc82800640 1 -- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc84112ea0 msgr2=0x7fdc8419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.546+0000 7fdc82800640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc84112ea0 0x7fdc8419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.546+0000 7fdc82800640 1 -- 172.21.15.18:0/227701083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc841a44f0 con 0x7fdc84076e50 2024-04-16T17:25:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.546+0000 7fdc83200640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc841a3a00 0x7fdc841a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.548+0000 7fdc82800640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076e50 0x7fdc8419f140 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c000c00 tx=0x7fdc6c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.550+0000 7fdc7be00640 1 -- 172.21.15.18:0/227701083 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c043020 con 0x7fdc84076e50 2024-04-16T17:25:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.550+0000 7fdc7be00640 1 -- 172.21.15.18:0/227701083 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6c004440 con 0x7fdc84076e50 2024-04-16T17:25:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.550+0000 7fdc7be00640 1 -- 172.21.15.18:0/227701083 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c003de0 con 0x7fdc84076e50 2024-04-16T17:25:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.552+0000 7fdc89400640 1 -- 172.21.15.18:0/227701083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc841a4780 con 0x7fdc84076e50 2024-04-16T17:25:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.552+0000 7fdc89400640 1 -- 172.21.15.18:0/227701083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc841abef0 con 0x7fdc84076e50 2024-04-16T17:25:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.559+0000 7fdc8c1e9740 1 Processor -- start 2024-04-16T17:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.560+0000 7fdc8c1e9740 1 -- start start 2024-04-16T17:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.561+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.561+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0cfbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.561+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d07210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.561+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0c60780 con 0x55d0f0cfc610 2024-04-16T17:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.561+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0c06f40 con 0x55d0f0d04e20 2024-04-16T17:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.561+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0c5c740 con 0x55d0f0cfb8a0 2024-04-16T17:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc83200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc79e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d07210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc79e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d07210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56116/0 (socket says 172.21.15.18:56116) 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc79e00640 1 -- 172.21.15.18:0/3466036543 learned_addr learned my addr 172.21.15.18:0/3466036543 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc79e00640 1 -- 172.21.15.18:0/3466036543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 msgr2=0x55d0f0cfbd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc79e00640 1 --2- 172.21.15.18:0/3466036543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0cfbd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc79e00640 1 -- 172.21.15.18:0/3466036543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 msgr2=0x55d0f0cfb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc79e00640 1 --2- 172.21.15.18:0/3466036543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc79e00640 1 -- 172.21.15.18:0/3466036543 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f0973210 con 0x55d0f0d04e20 2024-04-16T17:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.562+0000 7fdc83200640 1 --2- 172.21.15.18:0/3466036543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.564+0000 7fdc79e00640 1 --2- 172.21.15.18:0/3466036543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d07210 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009e90 tx=0x7fdc7400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd2a0a8731a610d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.564+0000 7fdc73400640 1 -- 172.21.15.18:0/3466036543 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74003b70 con 0x55d0f0d04e20 2024-04-16T17:25:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.565+0000 7fdc73400640 1 -- 172.21.15.18:0/3466036543 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc74003d10 con 0x55d0f0d04e20 2024-04-16T17:25:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.565+0000 7fdc73400640 1 -- 172.21.15.18:0/3466036543 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740189d0 con 0x55d0f0d04e20 2024-04-16T17:25:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.566+0000 7fdc7be00640 1 -- 172.21.15.18:0/227701083 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc6c046020 con 0x7fdc84076e50 2024-04-16T17:25:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.567+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3466036543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 msgr2=0x55d0f0d07210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.567+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3466036543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d07210 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009e90 tx=0x7fdc7400e720 comp rx=0 tx=0).stop 2024-04-16T17:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.567+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3466036543 shutdown_connections 2024-04-16T17:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.567+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3466036543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d07210 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.567+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3466036543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0cfbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.567+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3466036543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.567+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3466036543 >> 172.21.15.18:0/3466036543 conn(0x55d0f0c965d0 msgr2=0x55d0f0cf6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.567+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3466036543 shutdown_connections 2024-04-16T17:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.568+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3466036543 wait complete. 2024-04-16T17:25:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.568+0000 7fdc7be00640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc680a3df0 0x7fdc680a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.569+0000 7fdc7be00640 1 -- 172.21.15.18:0/227701083 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fdc6c0f27b0 con 0x7fdc84076e50 2024-04-16T17:25:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.570+0000 7fdc83200640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc680a3df0 0x7fdc680a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.571+0000 7fdc8c1e9740 1 -- start start 2024-04-16T17:25:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.571+0000 7fdc8c1e9740 1 Processor -- start 2024-04-16T17:25:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.572+0000 7fdc83200640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc680a3df0 0x7fdc680a62b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdc8406b520 tx=0x7fdc7c005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.574+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d01d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfc610 0x55d0f0d02280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0d07b50 con 0x55d0f0cfb8a0 2024-04-16T17:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc79e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfc610 0x55d0f0d02280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0c1a390 con 0x55d0f0d04e20 2024-04-16T17:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0d07cd0 con 0x55d0f0cfc610 2024-04-16T17:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc83200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc83200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56132/0 (socket says 172.21.15.18:56132) 2024-04-16T17:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.575+0000 7fdc83200640 1 -- 172.21.15.18:0/3797677869 learned_addr learned my addr 172.21.15.18:0/3797677869 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.576+0000 7fdc82800640 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d01d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.576+0000 7fdc8c1e9740 10 client.0 _ll_register_callbacks cb 0x55d0f0c03f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.577+0000 7fdc83200640 1 -- 172.21.15.18:0/3797677869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfc610 msgr2=0x55d0f0d02280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.577+0000 7fdc83200640 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfc610 0x55d0f0d02280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.577+0000 7fdc83200640 1 -- 172.21.15.18:0/3797677869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfb8a0 msgr2=0x55d0f0d01d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.577+0000 7fdc83200640 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d01d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.577+0000 7fdc83200640 1 -- 172.21.15.18:0/3797677869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f0a3f620 con 0x55d0f0d04e20 2024-04-16T17:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.577+0000 7fdc83200640 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d027c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c003820 tx=0x7fdc7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1b367020bab526 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.577+0000 7fdc67e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.578+0000 7fdc67e00640 1 -- 172.21.15.18:0/3797677869 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c011c20 con 0x55d0f0d04e20 2024-04-16T17:25:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.579+0000 7fdc67e00640 1 -- 172.21.15.18:0/3797677869 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fdc7c011dc0 con 0x55d0f0d04e20 2024-04-16T17:25:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.579+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3797677869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc74003d10 con 0x55d0f0d04e20 2024-04-16T17:25:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.580+0000 7fdc67e00640 1 -- 172.21.15.18:0/3797677869 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdc7c020420 con 0x55d0f0d04e20 2024-04-16T17:25:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.580+0000 7fdc8c1e9740 10 client.30977 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.580+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3797677869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d0f0d13d20 con 0x55d0f0d04e20 2024-04-16T17:25:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.582+0000 7fdc67e00640 1 -- 172.21.15.18:0/3797677869 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc74003d10 con 0x55d0f0d04e20 2024-04-16T17:25:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.582+0000 7fdc67e00640 1 -- 172.21.15.18:0/3797677869 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fdc7c028810 con 0x55d0f0d04e20 2024-04-16T17:25:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.583+0000 7fdc8c1e9740 10 client.30977 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.583+0000 7fdc8c1e9740 10 client.30977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.583+0000 7fdc8c1e9740 20 client.30977 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.583+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0f0d151e0 0x55d0f0d175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.583+0000 7fdc8c1e9740 4 client.30977 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.583+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3797677869 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0f0c6e1a0 con 0x55d0f0d151e0 2024-04-16T17:25:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.584+0000 7fdc82800640 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0f0d151e0 0x55d0f0d175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.585+0000 7fdc82800640 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0f0d151e0 0x55d0f0d175d0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad84b8fd1309af4f server_cookie=e8268fbcd6b6c240 in_seq=0 out_seq=0 2024-04-16T17:25:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.585+0000 7fdc67e00640 10 client.30977 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.588+0000 7fdc67e00640 1 -- 172.21.15.18:0/3797677869 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0f0c6e1a0 con 0x55d0f0d151e0 2024-04-16T17:25:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.588+0000 7fdc67e00640 10 client.30977 handle_command_reply: tid=0 2024-04-16T17:25:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.588+0000 7fdc8c1e9740 1 client.30977 shutdown 2024-04-16T17:25:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.589+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3797677869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 msgr2=0x55d0f0d027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.590+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d027c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c003820 tx=0x7fdc7c0046c0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.590+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3797677869 shutdown_connections 2024-04-16T17:25:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.590+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0f0d151e0 0x55d0f0d175d0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.590+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0d027c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.590+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfc610 0x55d0f0d02280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.590+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/3797677869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d01d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.590+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3797677869 >> 172.21.15.18:0/3797677869 conn(0x55d0f0c965d0 msgr2=0x55d0f0cf82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.591+0000 7fdc67e00640 0 client.30977 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.591+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3797677869 shutdown_connections 2024-04-16T17:25:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.591+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/3797677869 wait complete. 2024-04-16T17:25:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.591+0000 7fdc8c1e9740 20 client.30977 trim_cache size 0 max 16384 2024-04-16T17:25:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.619+0000 7fdc8c1e9740 1 Processor -- start 2024-04-16T17:25:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.620+0000 7fdc8c1e9740 1 -- start start 2024-04-16T17:25:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.620+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d10a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.620+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cf84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.620+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0d04e20 0x55d0f0d01640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.620+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841b8b40 con 0x55d0f0cfc610 2024-04-16T17:25:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.620+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841b6410 con 0x55d0f0cfb8a0 2024-04-16T17:25:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.620+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841b4280 con 0x55d0f0d04e20 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.620+0000 7fdc82800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d10a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc82800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d10a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56136/0 (socket says 172.21.15.18:56136) 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc82800640 1 -- 172.21.15.18:0/1045896794 learned_addr learned my addr 172.21.15.18:0/1045896794 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc79e00640 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cf84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc83200640 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0d04e20 0x55d0f0d01640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc82800640 1 -- 172.21.15.18:0/1045896794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0d04e20 msgr2=0x55d0f0d01640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc82800640 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0d04e20 0x55d0f0d01640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc82800640 1 -- 172.21.15.18:0/1045896794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 msgr2=0x55d0f0cf84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc82800640 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cf84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc82800640 1 -- 172.21.15.18:0/1045896794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f0cf1e60 con 0x55d0f0cfb8a0 2024-04-16T17:25:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc83200640 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0d04e20 0x55d0f0d01640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.621+0000 7fdc79e00640 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cf84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.622+0000 7fdc82800640 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d10a80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0efc00 tx=0x7fdc6c0ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde141436785dc18 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.622+0000 7fdc72a00640 1 -- 172.21.15.18:0/1045896794 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0be3f0 con 0x55d0f0cfb8a0 2024-04-16T17:25:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.622+0000 7fdc72a00640 1 -- 172.21.15.18:0/1045896794 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6c0c27a0 con 0x55d0f0cfb8a0 2024-04-16T17:25:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.622+0000 7fdc72a00640 1 -- 172.21.15.18:0/1045896794 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0efd10 con 0x55d0f0cfb8a0 2024-04-16T17:25:55.623 INFO:tasks.workunit.client.0.smithi018.stdout: 10 105417 10.79 MB/sec execute 367 sec latency 179.326 ms 2024-04-16T17:25:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.623+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1045896794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0cfb8a0 msgr2=0x55d0f0d10a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.623+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d10a80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0efc00 tx=0x7fdc6c0ba040 comp rx=0 tx=0).stop 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.623+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1045896794 shutdown_connections 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.623+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0d04e20 0x55d0f0d01640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.623+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cf84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.623+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1045896794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d10a80 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.623+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1045896794 >> 172.21.15.18:0/1045896794 conn(0x55d0f0c965d0 msgr2=0x55d0f0d02590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.623+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1045896794 shutdown_connections 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.624+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1045896794 wait complete. 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.624+0000 7fdc8c1e9740 1 -- start start 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.624+0000 7fdc8c1e9740 1 Processor -- start 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d0d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc8c1e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0cfeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841b48d0 con 0x55d0f0cfc610 2024-04-16T17:25:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841b5d40 con 0x55d0f0d04e20 2024-04-16T17:25:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc8c1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841b4fa0 con 0x55d0f0cfb8a0 2024-04-16T17:25:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc8c1e9740 10 client.0 _ll_register_callbacks cb 0x7fdc7c013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc82800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d0d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc82800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d0d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53344/0 (socket says 172.21.15.18:53344) 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc82800640 1 -- 172.21.15.18:0/1554531296 learned_addr learned my addr 172.21.15.18:0/1554531296 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.625+0000 7fdc83200640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0cfeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.626+0000 7fdc79e00640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.626+0000 7fdc83200640 1 -- 172.21.15.18:0/1554531296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 msgr2=0x55d0f0d0d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.626+0000 7fdc83200640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d0d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.626+0000 7fdc83200640 1 -- 172.21.15.18:0/1554531296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 msgr2=0x55d0f0cfe5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.626+0000 7fdc83200640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfe5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.626+0000 7fdc83200640 1 -- 172.21.15.18:0/1554531296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f0cff120 con 0x55d0f0d04e20 2024-04-16T17:25:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.626+0000 7fdc82800640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d0d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.626+0000 7fdc79e00640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.627+0000 7fdc83200640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0cfeb30 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x55d0f0cf2130 tx=0x7fdc7c005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f3938b68e90ce0d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.627+0000 7fdc67e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.627+0000 7fdc67e00640 1 -- 172.21.15.18:0/1554531296 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c03b020 con 0x55d0f0d04e20 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.627+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1554531296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc6c0c27a0 con 0x55d0f0d04e20 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.627+0000 7fdc67e00640 1 -- 172.21.15.18:0/1554531296 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fdc7c005040 con 0x55d0f0d04e20 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.628+0000 7fdc67e00640 1 -- 172.21.15.18:0/1554531296 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdc7c005240 con 0x55d0f0d04e20 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.628+0000 7fdc8c1e9740 10 client.30983 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.628+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1554531296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d0f0d7b4a0 con 0x55d0f0d04e20 2024-04-16T17:25:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.628+0000 7fdc67e00640 1 -- 172.21.15.18:0/1554531296 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6c0c27a0 con 0x55d0f0d04e20 2024-04-16T17:25:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.629+0000 7fdc67e00640 1 -- 172.21.15.18:0/1554531296 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fdc7c004a30 con 0x55d0f0d04e20 2024-04-16T17:25:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.629+0000 7fdc8c1e9740 10 client.30983 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.629+0000 7fdc8c1e9740 10 client.30983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.629+0000 7fdc8c1e9740 20 client.30983 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.629+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0f0d7bc40 0x55d0f0d7e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.629+0000 7fdc8c1e9740 4 client.30983 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:25:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.629+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1554531296 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0f0c68010 con 0x55d0f0d7bc40 2024-04-16T17:25:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.633+0000 7fdc82800640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0f0d7bc40 0x55d0f0d7e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.633+0000 7fdc82800640 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0f0d7bc40 0x55d0f0d7e060 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfdae5b318d46470 server_cookie=fc2956728b336963 in_seq=0 out_seq=0 2024-04-16T17:25:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:55.633+0000 7fdc67e00640 10 client.30983 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:55.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.783+0000 7f917ae00640 1 -- 172.21.15.18:0/3613884587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9174104450 msgr2=0x7f9174104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.783+0000 7f917ae00640 1 --2- 172.21.15.18:0/3613884587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9174104450 0x7f9174104830 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f915c009900 tx=0x7f915c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:25:55.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.783+0000 7f917ae00640 1 -- 172.21.15.18:0/3613884587 shutdown_connections 2024-04-16T17:25:55.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.783+0000 7f917ae00640 1 --2- 172.21.15.18:0/3613884587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9174109520 0x7f91741110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.783+0000 7f917ae00640 1 --2- 172.21.15.18:0/3613884587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9174104e00 0x7f9174108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.783+0000 7f917ae00640 1 --2- 172.21.15.18:0/3613884587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9174104450 0x7f9174104830 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.783+0000 7f917ae00640 1 -- 172.21.15.18:0/3613884587 >> 172.21.15.18:0/3613884587 conn(0x7f91740ffc90 msgr2=0x7f91741020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.785+0000 7f917ae00640 1 -- 172.21.15.18:0/3613884587 shutdown_connections 2024-04-16T17:25:55.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.785+0000 7f917ae00640 1 -- 172.21.15.18:0/3613884587 wait complete. 2024-04-16T17:25:55.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.786+0000 7f917ae00640 1 Processor -- start 2024-04-16T17:25:55.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.786+0000 7f917ae00640 1 -- start start 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.787+0000 7f917ae00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9174109520 0x7f917419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.787+0000 7f917ae00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f917419b2b0 0x7f917419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.787+0000 7f917ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917419fb20 0x7f917419ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.787+0000 7f917ae00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9174114680 con 0x7f9174109520 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.787+0000 7f917ae00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9174114500 con 0x7f917419b2b0 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.787+0000 7f917ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9174114800 con 0x7f917419fb20 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.787+0000 7f916be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f917419b2b0 0x7f917419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.787+0000 7f916be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f917419b2b0 0x7f917419f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56150/0 (socket says 172.21.15.18:56150) 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.788+0000 7f916be00640 1 -- 172.21.15.18:0/3679451894 learned_addr learned my addr 172.21.15.18:0/3679451894 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.788+0000 7f9178e00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917419fb20 0x7f917419ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.788+0000 7f916be00640 1 -- 172.21.15.18:0/3679451894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917419fb20 msgr2=0x7f917419ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.788+0000 7f916be00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917419fb20 0x7f917419ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.788+0000 7f916be00640 1 -- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9174109520 msgr2=0x7f917419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.788+0000 7f916be00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9174109520 0x7f917419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.788+0000 7f916be00640 1 -- 172.21.15.18:0/3679451894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91741a7890 con 0x7f917419b2b0 2024-04-16T17:25:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.789+0000 7f916be00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f917419b2b0 0x7f917419f5e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f916400e9a0 tx=0x7f916400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.789+0000 7f9171600640 1 -- 172.21.15.18:0/3679451894 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916400cd80 con 0x7f917419b2b0 2024-04-16T17:25:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.789+0000 7f9171600640 1 -- 172.21.15.18:0/3679451894 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91640035f0 con 0x7f917419b2b0 2024-04-16T17:25:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.789+0000 7f9171600640 1 -- 172.21.15.18:0/3679451894 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164010670 con 0x7f917419b2b0 2024-04-16T17:25:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.789+0000 7f917ae00640 1 -- 172.21.15.18:0/3679451894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91741a7ac0 con 0x7f917419b2b0 2024-04-16T17:25:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.790+0000 7f917ae00640 1 -- 172.21.15.18:0/3679451894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91741a7f90 con 0x7f917419b2b0 2024-04-16T17:25:55.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.801+0000 7f917dbbc740 1 Processor -- start 2024-04-16T17:25:55.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.803+0000 7f9171600640 1 -- 172.21.15.18:0/3679451894 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f91640037a0 con 0x7f917419b2b0 2024-04-16T17:25:55.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.803+0000 7f917dbbc740 1 -- start start 2024-04-16T17:25:55.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.803+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 0x56528e9f0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.804+0000 7f9178e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 0x56528e9f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.804+0000 7f9178e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 0x56528e9f0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44656/0 (socket says 172.21.15.18:44656) 2024-04-16T17:25:55.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.804+0000 7f9178e00640 1 -- 172.21.15.18:0/3577207012 learned_addr learned my addr 172.21.15.18:0/3577207012 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.804+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.804+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.805+0000 7f917dbbc740 1 -- 172.21.15.18:0/3577207012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56528e955020 con 0x56528e9efd50 2024-04-16T17:25:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.805+0000 7f917dbbc740 1 -- 172.21.15.18:0/3577207012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56528e8fb460 con 0x56528e9f0670 2024-04-16T17:25:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.805+0000 7f917dbbc740 1 -- 172.21.15.18:0/3577207012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56528e950fe0 con 0x56528e9f9250 2024-04-16T17:25:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.805+0000 7f916be00640 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.805+0000 7f9173e00640 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.806+0000 7f916be00640 1 -- 172.21.15.18:0/3577207012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 msgr2=0x56528e9f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.806+0000 7f916be00640 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.806+0000 7f916be00640 1 -- 172.21.15.18:0/3577207012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 msgr2=0x56528e9f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.806+0000 7f916be00640 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 0x56528e9f0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.806+0000 7f916be00640 1 -- 172.21.15.18:0/3577207012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56528e903040 con 0x56528e9f9250 2024-04-16T17:25:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.806+0000 7f9171600640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91580a3da0 0x7f91580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.807+0000 7f9173e00640 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:55.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.807+0000 7f9171600640 1 -- 172.21.15.18:0/3679451894 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f91640d0e60 con 0x7f917419b2b0 2024-04-16T17:25:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.808+0000 7f9178e00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91580a3da0 0x7f91580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.808+0000 7f916be00640 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9fb640 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f91640ce1a0 tx=0x7f91640d94b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9375dc14efb2bfdf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.808+0000 7f9169600640 1 -- 172.21.15.18:0/3577207012 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916409c7c0 con 0x56528e9f9250 2024-04-16T17:25:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.809+0000 7f9169600640 1 -- 172.21.15.18:0/3577207012 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916409d0f0 con 0x56528e9f9250 2024-04-16T17:25:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.809+0000 7f9169600640 1 -- 172.21.15.18:0/3577207012 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916409c350 con 0x56528e9f9250 2024-04-16T17:25:55.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.810+0000 7f917dbbc740 1 -- 172.21.15.18:0/3577207012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 msgr2=0x56528e9fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.810+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9fb640 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f91640ce1a0 tx=0x7f91640d94b0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.812+0000 7f917dbbc740 1 -- 172.21.15.18:0/3577207012 shutdown_connections 2024-04-16T17:25:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.812+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9fb640 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.812+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.812+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3577207012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 0x56528e9f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.812+0000 7f917dbbc740 1 -- 172.21.15.18:0/3577207012 >> 172.21.15.18:0/3577207012 conn(0x56528e9ea480 msgr2=0x56528e9ea860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.812+0000 7f917dbbc740 1 -- 172.21.15.18:0/3577207012 shutdown_connections 2024-04-16T17:25:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.812+0000 7f917dbbc740 1 -- 172.21.15.18:0/3577207012 wait complete. 2024-04-16T17:25:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.813+0000 7f917dbbc740 1 -- start start 2024-04-16T17:25:55.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.813+0000 7f917dbbc740 1 Processor -- start 2024-04-16T17:25:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.814+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9efd50 0x56528e9f6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.814+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.814+0000 7f9178e00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91580a3da0 0x7f91580a6260 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f916c009fd0 tx=0x7f916c00d040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.814+0000 7f9173e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9efd50 0x56528e9f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.814+0000 7f9173e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9efd50 0x56528e9f6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53364/0 (socket says 172.21.15.18:53364) 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.814+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f9250 0x56528e9f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.814+0000 7f916be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.814+0000 7f916be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56172/0 (socket says 172.21.15.18:56172) 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.815+0000 7f916be00640 1 -- 172.21.15.18:0/3278160120 learned_addr learned my addr 172.21.15.18:0/3278160120 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.815+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56528e9fc040 con 0x56528e9f9250 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.815+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56528e90e830 con 0x56528e9f0670 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.815+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56528e9fc1c0 con 0x56528e9efd50 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.815+0000 7f917dbbc740 10 client.0 _ll_register_callbacks cb 0x56528e916150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.816+0000 7f916be00640 1 -- 172.21.15.18:0/3278160120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9efd50 msgr2=0x56528e9f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.816+0000 7f916be00640 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9efd50 0x56528e9f6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.816+0000 7f9178e00640 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f9250 0x56528e9f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.816+0000 7f916be00640 1 -- 172.21.15.18:0/3278160120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f9250 msgr2=0x56528e9f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.816+0000 7f916be00640 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f9250 0x56528e9f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.816+0000 7f916be00640 1 -- 172.21.15.18:0/3278160120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56528e912920 con 0x56528e9f0670 2024-04-16T17:25:55.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.816+0000 7f9173e00640 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9efd50 0x56528e9f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-16T17:25:55.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.816+0000 7f9178e00640 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f9250 0x56528e9f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:55.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.818+0000 7f916be00640 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f67b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9164098e50 tx=0x7f9164009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485d7dd8c26d7ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.818+0000 7f9162000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:55.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.821+0000 7f9162000640 1 -- 172.21.15.18:0/3278160120 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916409c530 con 0x56528e9f0670 2024-04-16T17:25:55.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.822+0000 7f9162000640 1 -- 172.21.15.18:0/3278160120 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f916409cc50 con 0x56528e9f0670 2024-04-16T17:25:55.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.822+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f916409d0f0 con 0x56528e9f0670 2024-04-16T17:25:55.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.822+0000 7f9162000640 1 -- 172.21.15.18:0/3278160120 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f91640b7030 con 0x56528e9f0670 2024-04-16T17:25:55.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.822+0000 7f917dbbc740 10 client.30995 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:55.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.823+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56528e8e07f0 con 0x56528e9f0670 2024-04-16T17:25:55.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.823+0000 7f9162000640 1 -- 172.21.15.18:0/3278160120 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916409d0f0 con 0x56528e9f0670 2024-04-16T17:25:55.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.823+0000 7f9162000640 1 -- 172.21.15.18:0/3278160120 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f91640ba020 con 0x56528e9f0670 2024-04-16T17:25:55.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.824+0000 7f917dbbc740 10 client.30995 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:55.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.824+0000 7f917dbbc740 10 client.30995 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:25:55.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.824+0000 7f917dbbc740 20 client.30995 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:55.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.824+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56528ea11c30 0x56528ea14020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.824+0000 7f917dbbc740 4 client.30995 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:55.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.824+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56528e962a40 con 0x56528ea11c30 2024-04-16T17:25:55.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.825+0000 7f9173e00640 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56528ea11c30 0x56528ea14020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.826+0000 7f9173e00640 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56528ea11c30 0x56528ea14020 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54515f37034007fc server_cookie=506bd9ec6fb25cf4 in_seq=0 out_seq=0 2024-04-16T17:25:55.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.826+0000 7f9162000640 10 client.30995 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:25:55.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.830+0000 7f9162000640 1 -- 172.21.15.18:0/3278160120 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56528e962a40 con 0x56528ea11c30 2024-04-16T17:25:55.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.830+0000 7f9162000640 10 client.30995 handle_command_reply: tid=0 2024-04-16T17:25:55.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 client.30995 shutdown 2024-04-16T17:25:55.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 msgr2=0x56528e9f67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f67b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9164098e50 tx=0x7f9164009310 comp rx=0 tx=0).stop 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 shutdown_connections 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56528ea11c30 0x56528ea14020 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f9250 0x56528e9f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f0670 0x56528e9f67b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 --2- 172.21.15.18:0/3278160120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9efd50 0x56528e9f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f9162000640 0 client.30995 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.831+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 >> 172.21.15.18:0/3278160120 conn(0x56528e9ea480 msgr2=0x56528e9ec1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.832+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 shutdown_connections 2024-04-16T17:25:55.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.832+0000 7f917dbbc740 1 -- 172.21.15.18:0/3278160120 wait complete. 2024-04-16T17:25:55.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.833+0000 7f917dbbc740 20 client.30995 trim_cache size 0 max 16384 2024-04-16T17:25:55.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.861+0000 7f917dbbc740 1 Processor -- start 2024-04-16T17:25:55.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.862+0000 7f917dbbc740 1 -- start start 2024-04-16T17:25:55.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.862+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9efd50 0x56528e9f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.862+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f0670 0x56528e9f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.862+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.862+0000 7f917dbbc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91741b0460 con 0x56528e9f0670 2024-04-16T17:25:55.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.862+0000 7f917dbbc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91741add30 con 0x56528e9efd50 2024-04-16T17:25:55.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.862+0000 7f917dbbc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91741abba0 con 0x56528e9f9250 2024-04-16T17:25:55.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f9173e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9efd50 0x56528e9f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f9173e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9efd50 0x56528e9f89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56178/0 (socket says 172.21.15.18:56178) 2024-04-16T17:25:55.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f9173e00640 1 -- 172.21.15.18:0/2985585383 learned_addr learned my addr 172.21.15.18:0/2985585383 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f916be00640 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f0670 0x56528e9f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f9178e00640 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f9173e00640 1 -- 172.21.15.18:0/2985585383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 msgr2=0x56528e9f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f9173e00640 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f9173e00640 1 -- 172.21.15.18:0/2985585383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f0670 msgr2=0x56528e9f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.863+0000 7f9173e00640 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f0670 0x56528e9f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.864+0000 7f9173e00640 1 -- 172.21.15.18:0/2985585383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56528e6673f0 con 0x56528e9efd50 2024-04-16T17:25:55.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.864+0000 7f9173e00640 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9efd50 0x56528e9f89a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f915c02e010 tx=0x7f915c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34753f4a2947e27f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.865+0000 7f9168c00640 1 -- 172.21.15.18:0/2985585383 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c049020 con 0x56528e9efd50 2024-04-16T17:25:55.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.865+0000 7f9168c00640 1 -- 172.21.15.18:0/2985585383 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c004240 con 0x56528e9efd50 2024-04-16T17:25:55.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.865+0000 7f9168c00640 1 -- 172.21.15.18:0/2985585383 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c000e60 con 0x56528e9efd50 2024-04-16T17:25:55.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.866+0000 7f917dbbc740 1 -- 172.21.15.18:0/2985585383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9efd50 msgr2=0x56528e9f89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.866+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9efd50 0x56528e9f89a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f915c02e010 tx=0x7f915c000bc0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.866+0000 7f917dbbc740 1 -- 172.21.15.18:0/2985585383 shutdown_connections 2024-04-16T17:25:55.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.866+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f9250 0x56528e9f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.866+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9f0670 0x56528e9f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.866+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2985585383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9efd50 0x56528e9f89a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.866+0000 7f917dbbc740 1 -- 172.21.15.18:0/2985585383 >> 172.21.15.18:0/2985585383 conn(0x56528e9ea480 msgr2=0x56528e9f15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.866+0000 7f917dbbc740 1 -- 172.21.15.18:0/2985585383 shutdown_connections 2024-04-16T17:25:55.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.867+0000 7f917dbbc740 1 -- 172.21.15.18:0/2985585383 wait complete. 2024-04-16T17:25:55.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.867+0000 7f917dbbc740 1 -- start start 2024-04-16T17:25:55.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.867+0000 7f917dbbc740 1 Processor -- start 2024-04-16T17:25:55.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.868+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 0x56528ea14a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.868+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f0670 0x56528ea14f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.868+0000 7f917dbbc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f9250 0x56528ea154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.868+0000 7f917dbbc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91741ac1f0 con 0x56528e9efd50 2024-04-16T17:25:55.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.868+0000 7f917dbbc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91741ad660 con 0x56528e9f9250 2024-04-16T17:25:55.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.868+0000 7f917dbbc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91741ac8c0 con 0x56528e9f0670 2024-04-16T17:25:55.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.868+0000 7f917dbbc740 10 client.0 _ll_register_callbacks cb 0x7f9164005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:55.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.869+0000 7f9178e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f9250 0x56528ea154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.869+0000 7f9178e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f9250 0x56528ea154c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56194/0 (socket says 172.21.15.18:56194) 2024-04-16T17:25:55.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.869+0000 7f9178e00640 1 -- 172.21.15.18:0/2138208053 learned_addr learned my addr 172.21.15.18:0/2138208053 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:55.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.869+0000 7f916be00640 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f0670 0x56528ea14f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.870+0000 7f916be00640 1 -- 172.21.15.18:0/2138208053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f9250 msgr2=0x56528ea154c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.870+0000 7f916be00640 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f9250 0x56528ea154c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.870+0000 7f916be00640 1 -- 172.21.15.18:0/2138208053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 msgr2=0x56528ea14a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:55.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.870+0000 7f916be00640 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 0x56528ea14a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.870+0000 7f916be00640 1 -- 172.21.15.18:0/2138208053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56528e7317d0 con 0x56528e9f0670 2024-04-16T17:25:55.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.873+0000 7f9162000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:55.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.873+0000 7f916be00640 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f0670 0x56528ea14f80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x56528e9ec920 tx=0x7f91640ce250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c45ac3473449db server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:55.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.873+0000 7f9162000640 1 -- 172.21.15.18:0/2138208053 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916409c530 con 0x56528e9f0670 2024-04-16T17:25:55.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.873+0000 7f9162000640 1 -- 172.21.15.18:0/2138208053 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f91640b8070 con 0x56528e9f0670 2024-04-16T17:25:55.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.874+0000 7f917dbbc740 1 -- 172.21.15.18:0/2138208053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f915c004240 con 0x56528e9f0670 2024-04-16T17:25:55.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.874+0000 7f9162000640 1 -- 172.21.15.18:0/2138208053 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f916409c530 con 0x56528e9f0670 2024-04-16T17:25:55.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.875+0000 7f917dbbc740 10 client.30835 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:55.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.875+0000 7f917dbbc740 1 -- 172.21.15.18:0/2138208053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56528ea77ee0 con 0x56528e9f0670 2024-04-16T17:25:55.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.876+0000 7f9162000640 1 -- 172.21.15.18:0/2138208053 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916409cea0 con 0x56528e9f0670 2024-04-16T17:25:55.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.876+0000 7f9162000640 1 -- 172.21.15.18:0/2138208053 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f91640ba070 con 0x56528e9f0670 2024-04-16T17:25:55.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.876+0000 7f917dbbc740 10 client.30835 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:55.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.876+0000 7f917dbbc740 10 client.30835 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:25:55.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.876+0000 7f917dbbc740 20 client.30835 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:55.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.876+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56528ea78370 0x56528ea7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:55.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.877+0000 7f917dbbc740 4 client.30835 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:25:55.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.877+0000 7f917dbbc740 1 -- 172.21.15.18:0/2138208053 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f915c004240 con 0x56528ea78370 2024-04-16T17:25:55.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.878+0000 7f9173e00640 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56528ea78370 0x56528ea7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:55.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.879+0000 7f9173e00640 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56528ea78370 0x56528ea7a790 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=442940804fef7183 server_cookie=1819c367a6710e70 in_seq=0 out_seq=0 2024-04-16T17:25:55.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.879+0000 7f9162000640 10 client.30835 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:25:55.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.879+0000 7f9162000640 1 -- 172.21.15.18:0/2138208053 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f915c004240 con 0x56528ea78370 2024-04-16T17:25:55.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.879+0000 7f9162000640 10 client.30835 handle_command_reply: tid=0 2024-04-16T17:25:55.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.879+0000 7f917dbbc740 1 client.30835 shutdown 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.879+0000 7f917dbbc740 1 -- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f0670 msgr2=0x56528ea14f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.879+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f0670 0x56528ea14f80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x56528e9ec920 tx=0x7f91640ce250 comp rx=0 tx=0).stop 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.880+0000 7f917dbbc740 1 -- 172.21.15.18:0/2138208053 shutdown_connections 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.880+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x56528ea78370 0x56528ea7a790 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.880+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56528e9f9250 0x56528ea154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.880+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56528e9f0670 0x56528ea14f80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.880+0000 7f917dbbc740 1 --2- 172.21.15.18:0/2138208053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56528e9efd50 0x56528ea14a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.880+0000 7f917dbbc740 1 -- 172.21.15.18:0/2138208053 >> 172.21.15.18:0/2138208053 conn(0x56528e9ea480 msgr2=0x56528ea0cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.880+0000 7f9162000640 0 client.30835 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:25:55.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.880+0000 7f917dbbc740 1 -- 172.21.15.18:0/2138208053 shutdown_connections 2024-04-16T17:25:55.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.881+0000 7f917dbbc740 1 -- 172.21.15.18:0/2138208053 wait complete. 2024-04-16T17:25:55.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.881+0000 7f917dbbc740 20 client.30835 trim_cache size 0 max 16384 2024-04-16T17:25:55.881 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:25:55.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 -- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91580a3da0 msgr2=0x7f91580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91580a3da0 0x7f91580a6260 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f916c009fd0 tx=0x7f916c00d040 comp rx=0 tx=0).stop 2024-04-16T17:25:55.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 -- 172.21.15.18:0/3679451894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f917419b2b0 msgr2=0x7f917419f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:55.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f917419b2b0 0x7f917419f5e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f916400e9a0 tx=0x7f916400ee60 comp rx=0 tx=0).stop 2024-04-16T17:25:55.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 -- 172.21.15.18:0/3679451894 shutdown_connections 2024-04-16T17:25:55.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f91580a3da0 0x7f91580a6260 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f917419fb20 0x7f917419ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f917419b2b0 0x7f917419f5e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 --2- 172.21.15.18:0/3679451894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9174109520 0x7f917419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:55.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.882+0000 7f917ae00640 1 -- 172.21.15.18:0/3679451894 >> 172.21.15.18:0/3679451894 conn(0x7f91740ffc90 msgr2=0x7f9174101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:55.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.883+0000 7f917ae00640 1 -- 172.21.15.18:0/3679451894 shutdown_connections 2024-04-16T17:25:55.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:55.883+0000 7f917ae00640 1 -- 172.21.15.18:0/3679451894 wait complete. 2024-04-16T17:25:55.918 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:25:55.933 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:47 smithi018 ceph-mon[31776]: pgmap v673: 129 pgs: 129 active+clean; 440 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-04-16T17:25:55.934 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:49 smithi018 ceph-mon[31776]: pgmap v674: 129 pgs: 129 active+clean; 440 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-04-16T17:25:55.934 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:51 smithi018 ceph-mon[31776]: pgmap v675: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2024-04-16T17:25:55.934 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:53 smithi018 ceph-mon[31776]: pgmap v676: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 324 op/s 2024-04-16T17:25:56.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:55 smithi043 ceph-mon[36901]: pgmap v677: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-04-16T17:25:56.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:55 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/606242070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:25:56.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.264+0000 7f6801800640 1 -- 172.21.15.18:0/3113827260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc104280 msgr2=0x7f67fc104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.264+0000 7f6801800640 1 --2- 172.21.15.18:0/3113827260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc104280 0x7f67fc104660 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009ca0 tx=0x7f67f402c100 comp rx=0 tx=0).stop 2024-04-16T17:25:56.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.264+0000 7f6801800640 1 -- 172.21.15.18:0/3113827260 shutdown_connections 2024-04-16T17:25:56.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.264+0000 7f6801800640 1 --2- 172.21.15.18:0/3113827260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc109350 0x7f67fc110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.264+0000 7f6801800640 1 --2- 172.21.15.18:0/3113827260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67fc104c30 0x7f67fc108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.264+0000 7f6801800640 1 --2- 172.21.15.18:0/3113827260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc104280 0x7f67fc104660 secure :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009ca0 tx=0x7f67f402c100 comp rx=0 tx=0).stop 2024-04-16T17:25:56.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.264+0000 7f6801800640 1 -- 172.21.15.18:0/3113827260 >> 172.21.15.18:0/3113827260 conn(0x7f67fc0ffae0 msgr2=0x7f67fc101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:56.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.266+0000 7f6801800640 1 -- 172.21.15.18:0/3113827260 shutdown_connections 2024-04-16T17:25:56.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.267+0000 7f6801800640 1 -- 172.21.15.18:0/3113827260 wait complete. 2024-04-16T17:25:56.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.268+0000 7f6801800640 1 Processor -- start 2024-04-16T17:25:56.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.269+0000 7f6801800640 1 -- start start 2024-04-16T17:25:56.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.269+0000 7f6801800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67fc109350 0x7f67fc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.269+0000 7f6801800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1a0270 0x7f67fc1a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.269+0000 7f6801800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc1a06d0 0x7f67fc1ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.269+0000 7f6801800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc078220 con 0x7f67fc109350 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.269+0000 7f6801800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc0780a0 con 0x7f67fc1a0270 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.269+0000 7f6801800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc0783a0 con 0x7f67fc1a06d0 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f67fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1a0270 0x7f67fc1a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f67fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1a0270 0x7f67fc1a8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56212/0 (socket says 172.21.15.18:56212) 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f67fb400640 1 -- 172.21.15.18:0/2162805785 learned_addr learned my addr 172.21.15.18:0/2162805785 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f6800e00640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc1a06d0 0x7f67fc1ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f6800e00640 1 -- 172.21.15.18:0/2162805785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1a0270 msgr2=0x7f67fc1a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f6800e00640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1a0270 0x7f67fc1a8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f6800e00640 1 -- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67fc109350 msgr2=0x7f67fc19fd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f6800e00640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67fc109350 0x7f67fc19fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.270+0000 7f6800e00640 1 -- 172.21.15.18:0/2162805785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67fc06bb50 con 0x7f67fc1a06d0 2024-04-16T17:25:56.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.274+0000 7f6800e00640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc1a06d0 0x7f67fc1ab110 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e970 tx=0x7f67e800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.275+0000 7f67f8c00640 1 -- 172.21.15.18:0/2162805785 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e800ccc0 con 0x7f67fc1a06d0 2024-04-16T17:25:56.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.275+0000 7f67f8c00640 1 -- 172.21.15.18:0/2162805785 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e8005020 con 0x7f67fc1a06d0 2024-04-16T17:25:56.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.275+0000 7f6801800640 1 -- 172.21.15.18:0/2162805785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67fc06be40 con 0x7f67fc1a06d0 2024-04-16T17:25:56.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.275+0000 7f67f8c00640 1 -- 172.21.15.18:0/2162805785 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8004a00 con 0x7f67fc1a06d0 2024-04-16T17:25:56.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.275+0000 7f6801800640 1 -- 172.21.15.18:0/2162805785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67fc06c390 con 0x7f67fc1a06d0 2024-04-16T17:25:56.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.282+0000 7f6805c0a740 1 Processor -- start 2024-04-16T17:25:56.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.285+0000 7f6805c0a740 1 -- start start 2024-04-16T17:25:56.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.285+0000 7f67f8c00640 1 -- 172.21.15.18:0/2162805785 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f67e80051d0 con 0x7f67fc1a06d0 2024-04-16T17:25:56.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.285+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.285+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f30ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.285+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f31410 0x55a346f33800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.285+0000 7f6805c0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a346e8d020 con 0x55a346f27f10 2024-04-16T17:25:56.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.285+0000 7f6805c0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a346e33460 con 0x55a346f31410 2024-04-16T17:25:56.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.285+0000 7f6805c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a346e88fe0 con 0x55a346f28830 2024-04-16T17:25:56.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.286+0000 7f67fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f31410 0x55a346f33800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.286+0000 7f67fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f31410 0x55a346f33800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56222/0 (socket says 172.21.15.18:56222) 2024-04-16T17:25:56.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.286+0000 7f67fb400640 1 -- 172.21.15.18:0/3678984561 learned_addr learned my addr 172.21.15.18:0/3678984561 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:56.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.286+0000 7f67fbe00640 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f30ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.287+0000 7f67fb400640 1 -- 172.21.15.18:0/3678984561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 msgr2=0x55a346f30ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.287+0000 7f67fb400640 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f30ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.287+0000 7f67fb400640 1 -- 172.21.15.18:0/3678984561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 msgr2=0x55a346f282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.287+0000 7f67fb400640 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.287+0000 7f67fb400640 1 -- 172.21.15.18:0/3678984561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a346e3b040 con 0x55a346f31410 2024-04-16T17:25:56.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.287+0000 7f67fbe00640 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f30ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:56.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.290+0000 7f67fb400640 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f31410 0x55a346f33800 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00cdb0 tx=0x7f67ec0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d952c8062fa478 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.290+0000 7f67f8c00640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f67e00a3da0 0x7f67e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.291+0000 7f67f8c00640 1 -- 172.21.15.18:0/2162805785 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f67e80d0c90 con 0x7f67fc1a06d0 2024-04-16T17:25:56.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.291+0000 7f67f1600640 1 -- 172.21.15.18:0/3678984561 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec005220 con 0x55a346f31410 2024-04-16T17:25:56.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.291+0000 7f67f1600640 1 -- 172.21.15.18:0/3678984561 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec016030 con 0x55a346f31410 2024-04-16T17:25:56.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.291+0000 7f67f1600640 1 -- 172.21.15.18:0/3678984561 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec012740 con 0x55a346f31410 2024-04-16T17:25:56.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.291+0000 7f67fbe00640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f67e00a3da0 0x7f67e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.292+0000 7f6805c0a740 1 -- 172.21.15.18:0/3678984561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f31410 msgr2=0x55a346f33800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.292+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f31410 0x55a346f33800 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00cdb0 tx=0x7f67ec0117b0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.292+0000 7f6805c0a740 1 -- 172.21.15.18:0/3678984561 shutdown_connections 2024-04-16T17:25:56.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.292+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f31410 0x55a346f33800 secure :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00cdb0 tx=0x7f67ec0117b0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.292+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f30ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.292+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3678984561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.292+0000 7f6805c0a740 1 -- 172.21.15.18:0/3678984561 >> 172.21.15.18:0/3678984561 conn(0x55a346f22640 msgr2=0x55a346f22a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:56.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.292+0000 7f6805c0a740 1 -- 172.21.15.18:0/3678984561 shutdown_connections 2024-04-16T17:25:56.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.293+0000 7f6805c0a740 1 -- 172.21.15.18:0/3678984561 wait complete. 2024-04-16T17:25:56.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.293+0000 7f6805c0a740 1 -- start start 2024-04-16T17:25:56.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.293+0000 7f6805c0a740 1 Processor -- start 2024-04-16T17:25:56.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.294+0000 7f67fbe00640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f67e00a3da0 0x7f67e00a6260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009bf0 tx=0x7f67f4005d80 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.295+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f2e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.295+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f28830 0x55a346f2ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.295+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f2efb0 0x55a346f3d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f67fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f28830 0x55a346f2ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f67fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f28830 0x55a346f2ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56226/0 (socket says 172.21.15.18:56226) 2024-04-16T17:25:56.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a346f34200 con 0x55a346f27f10 2024-04-16T17:25:56.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f67fbe00640 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f2e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f67fb400640 1 -- 172.21.15.18:0/1165088446 learned_addr learned my addr 172.21.15.18:0/1165088446 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:56.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a346e46830 con 0x55a346f28830 2024-04-16T17:25:56.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a346f34380 con 0x55a346f2efb0 2024-04-16T17:25:56.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f6800e00640 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f2efb0 0x55a346f3d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.296+0000 7f6805c0a740 10 client.0 _ll_register_callbacks cb 0x55a346e4e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:56.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.297+0000 7f67fb400640 1 -- 172.21.15.18:0/1165088446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f2efb0 msgr2=0x55a346f3d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.297+0000 7f67fb400640 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f2efb0 0x55a346f3d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.297+0000 7f67fb400640 1 -- 172.21.15.18:0/1165088446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 msgr2=0x55a346f2e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.297+0000 7f67fb400640 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f2e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.297+0000 7f67fb400640 1 -- 172.21.15.18:0/1165088446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a346e4a920 con 0x55a346f28830 2024-04-16T17:25:56.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.297+0000 7f67fb400640 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f28830 0x55a346f2ea70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00cdb0 tx=0x7f67ec00cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd0d4d2bb484de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.297+0000 7f67e6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:56.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.297+0000 7f67e6000640 1 -- 172.21.15.18:0/1165088446 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec029060 con 0x55a346f28830 2024-04-16T17:25:56.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.298+0000 7f67e6000640 1 -- 172.21.15.18:0/1165088446 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f67ec005230 con 0x55a346f28830 2024-04-16T17:25:56.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.298+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ec016030 con 0x55a346f28830 2024-04-16T17:25:56.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.298+0000 7f67e6000640 1 -- 172.21.15.18:0/1165088446 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f67ec005450 con 0x55a346f28830 2024-04-16T17:25:56.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.298+0000 7f6805c0a740 10 client.31013 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:56.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.298+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a346e187f0 con 0x55a346f28830 2024-04-16T17:25:56.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.299+0000 7f67e6000640 1 -- 172.21.15.18:0/1165088446 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec016030 con 0x55a346f28830 2024-04-16T17:25:56.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.299+0000 7f67e6000640 1 -- 172.21.15.18:0/1165088446 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f67ec0128b0 con 0x55a346f28830 2024-04-16T17:25:56.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.299+0000 7f6805c0a740 10 client.31013 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:56.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.299+0000 7f6805c0a740 10 client.31013 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:25:56.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.299+0000 7f6805c0a740 20 client.31013 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:56.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.299+0000 7f6805c0a740 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a346f41bd0 0x55a346f43fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.299+0000 7f6805c0a740 4 client.31013 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:56.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.299+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a346e9aa40 con 0x55a346f41bd0 2024-04-16T17:25:56.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.302+0000 7f67fbe00640 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a346f41bd0 0x55a346f43fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.303+0000 7f67fbe00640 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a346f41bd0 0x55a346f43fc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b535df38f2f2dd server_cookie=c0b4613d01e9a8ca in_seq=0 out_seq=0 2024-04-16T17:25:56.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.303+0000 7f67e6000640 10 client.31013 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:25:56.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.305+0000 7f67e6000640 1 -- 172.21.15.18:0/1165088446 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a346e9aa40 con 0x55a346f41bd0 2024-04-16T17:25:56.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.305+0000 7f67e6000640 10 client.31013 handle_command_reply: tid=0 2024-04-16T17:25:56.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.305+0000 7f6805c0a740 1 client.31013 shutdown 2024-04-16T17:25:56.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.305+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f28830 msgr2=0x55a346f2ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.305+0000 7f6805c0a740 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f28830 0x55a346f2ea70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00cdb0 tx=0x7f67ec00cf30 comp rx=0 tx=0).stop 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 shutdown_connections 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a346f41bd0 0x55a346f43fc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f2efb0 0x55a346f3d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f28830 0x55a346f2ea70 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 1 --2- 172.21.15.18:0/1165088446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f2e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f67e6000640 0 client.31013 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 >> 172.21.15.18:0/1165088446 conn(0x55a346f22640 msgr2=0x55a346f24390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 shutdown_connections 2024-04-16T17:25:56.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 1 -- 172.21.15.18:0/1165088446 wait complete. 2024-04-16T17:25:56.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.306+0000 7f6805c0a740 20 client.31013 trim_cache size 0 max 16384 2024-04-16T17:25:56.315 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:55 smithi018 ceph-mon[31776]: pgmap v677: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-04-16T17:25:56.315 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:55 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/606242070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:25:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.334+0000 7f6805c0a740 1 Processor -- start 2024-04-16T17:25:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.334+0000 7f6805c0a740 1 -- start start 2024-04-16T17:25:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f49590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f49ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 0x55a346f3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f6805c0a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1c1250 con 0x55a346f27f10 2024-04-16T17:25:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f67fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f49ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f67fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f49ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53436/0 (socket says 172.21.15.18:53436) 2024-04-16T17:25:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f67fb400640 1 -- 172.21.15.18:0/2393005306 learned_addr learned my addr 172.21.15.18:0/2393005306 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f6805c0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1beb20 con 0x55a346f2efb0 2024-04-16T17:25:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.335+0000 7f6805c0a740 1 -- 172.21.15.18:0/2393005306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1bc9b0 con 0x55a346f28830 2024-04-16T17:25:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.336+0000 7f67fb400640 1 -- 172.21.15.18:0/2393005306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 msgr2=0x55a346f3d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.336+0000 7f67fb400640 1 --2- 172.21.15.18:0/2393005306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 0x55a346f3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.336+0000 7f67fb400640 1 -- 172.21.15.18:0/2393005306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 msgr2=0x55a346f49590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.336+0000 7f67fb400640 1 --2- 172.21.15.18:0/2393005306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f49590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.336+0000 7f67fb400640 1 -- 172.21.15.18:0/2393005306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a346b9f3f0 con 0x55a346f28830 2024-04-16T17:25:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.336+0000 7f67fb400640 1 --2- 172.21.15.18:0/2393005306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f49ad0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0117f0 tx=0x7f67ec011de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174ac07cfc1bf4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.337+0000 7f67f0c00640 1 -- 172.21.15.18:0/2393005306 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec029070 con 0x55a346f28830 2024-04-16T17:25:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.337+0000 7f67f0c00640 1 -- 172.21.15.18:0/2393005306 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec016070 con 0x55a346f28830 2024-04-16T17:25:56.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.337+0000 7f67f0c00640 1 -- 172.21.15.18:0/2393005306 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec01b7b0 con 0x55a346f28830 2024-04-16T17:25:56.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.338+0000 7f6805c0a740 1 -- 172.21.15.18:0/2393005306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 msgr2=0x55a346f49ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.338+0000 7f6805c0a740 1 --2- 172.21.15.18:0/2393005306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f49ad0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0117f0 tx=0x7f67ec011de0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.338+0000 7f6805c0a740 1 -- 172.21.15.18:0/2393005306 shutdown_connections 2024-04-16T17:25:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.338+0000 7f6805c0a740 1 --2- 172.21.15.18:0/2393005306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 0x55a346f3d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.338+0000 7f6805c0a740 1 --2- 172.21.15.18:0/2393005306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f28830 0x55a346f49ad0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.338+0000 7f6805c0a740 1 --2- 172.21.15.18:0/2393005306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f27f10 0x55a346f49590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.338+0000 7f6805c0a740 1 -- 172.21.15.18:0/2393005306 >> 172.21.15.18:0/2393005306 conn(0x55a346f22640 msgr2=0x55a346f29810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.339+0000 7f6805c0a740 1 -- 172.21.15.18:0/2393005306 shutdown_connections 2024-04-16T17:25:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.339+0000 7f6805c0a740 1 -- 172.21.15.18:0/2393005306 wait complete. 2024-04-16T17:25:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.339+0000 7f6805c0a740 1 -- start start 2024-04-16T17:25:56.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.340+0000 7f6805c0a740 1 Processor -- start 2024-04-16T17:25:56.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.341+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f27f10 0x55a346f39a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.341+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f28830 0x55a346f34de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.341+0000 7f6805c0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 0x55a346f35360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.341+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f27f10 0x55a346f39a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.341+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f27f10 0x55a346f39a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53440/0 (socket says 172.21.15.18:53440) 2024-04-16T17:25:56.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.342+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1bd000 con 0x55a346f28830 2024-04-16T17:25:56.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.342+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1be450 con 0x55a346f2efb0 2024-04-16T17:25:56.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.342+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1bd6b0 con 0x55a346f27f10 2024-04-16T17:25:56.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.342+0000 7f6805c0a740 10 client.0 _ll_register_callbacks cb 0x7f67f4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:56.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.342+0000 7f67fbe00640 1 -- 172.21.15.18:0/3570428394 learned_addr learned my addr 172.21.15.18:0/3570428394 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:56.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.342+0000 7f6800e00640 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 0x55a346f35360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.342+0000 7f67fb400640 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f28830 0x55a346f34de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.343+0000 7f6800e00640 1 -- 172.21.15.18:0/3570428394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f27f10 msgr2=0x55a346f39a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.343+0000 7f6800e00640 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f27f10 0x55a346f39a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.343+0000 7f6800e00640 1 -- 172.21.15.18:0/3570428394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f28830 msgr2=0x55a346f34de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.343+0000 7f6800e00640 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f28830 0x55a346f34de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.343+0000 7f6800e00640 1 -- 172.21.15.18:0/3570428394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a346c697d0 con 0x55a346f2efb0 2024-04-16T17:25:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.343+0000 7f6800e00640 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 0x55a346f35360 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f67e80cdf70 tx=0x7f67e809c1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f23063d0c51497fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.344+0000 7f67e6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.344+0000 7f67e6000640 1 -- 172.21.15.18:0/3570428394 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e801c040 con 0x55a346f2efb0 2024-04-16T17:25:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.344+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ec016070 con 0x55a346f2efb0 2024-04-16T17:25:56.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.345+0000 7f67e6000640 1 -- 172.21.15.18:0/3570428394 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f67e80ce1f0 con 0x55a346f2efb0 2024-04-16T17:25:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.345+0000 7f67e6000640 1 -- 172.21.15.18:0/3570428394 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f67e801c040 con 0x55a346f2efb0 2024-04-16T17:25:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.345+0000 7f6805c0a740 10 client.31016 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.345+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a346fa7d20 con 0x55a346f2efb0 2024-04-16T17:25:56.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.346+0000 7f67e6000640 1 -- 172.21.15.18:0/3570428394 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec016070 con 0x55a346f2efb0 2024-04-16T17:25:56.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.346+0000 7f67e6000640 1 -- 172.21.15.18:0/3570428394 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f67e809ca20 con 0x55a346f2efb0 2024-04-16T17:25:56.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.346+0000 7f6805c0a740 10 client.31016 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:56.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.346+0000 7f6805c0a740 10 client.31016 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:25:56.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.346+0000 7f6805c0a740 20 client.31016 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:56.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.346+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a346fa83f0 0x55a346faa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.346+0000 7f6805c0a740 4 client.31016 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:25:56.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.346+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a346e948b0 con 0x55a346fa83f0 2024-04-16T17:25:56.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.347+0000 7f67fbe00640 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a346fa83f0 0x55a346faa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.349+0000 7f67fbe00640 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a346fa83f0 0x55a346faa810 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b0f24cc12366af8 server_cookie=620897fdbc445494 in_seq=0 out_seq=0 2024-04-16T17:25:56.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.349+0000 7f67e6000640 10 client.31016 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:25:56.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.349+0000 7f67e6000640 1 -- 172.21.15.18:0/3570428394 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a346e948b0 con 0x55a346fa83f0 2024-04-16T17:25:56.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.350+0000 7f67e6000640 10 client.31016 handle_command_reply: tid=0 2024-04-16T17:25:56.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.350+0000 7f6805c0a740 1 client.31016 shutdown 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 msgr2=0x55a346f35360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 0x55a346f35360 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f67e80cdf70 tx=0x7f67e809c1e0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 shutdown_connections 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55a346fa83f0 0x55a346faa810 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a346f2efb0 0x55a346f35360 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a346f28830 0x55a346f34de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 --2- 172.21.15.18:0/3570428394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a346f27f10 0x55a346f39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f67e6000640 0 client.31016 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:25:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 >> 172.21.15.18:0/3570428394 conn(0x55a346f22640 msgr2=0x55a346f20e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:56.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 shutdown_connections 2024-04-16T17:25:56.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.352+0000 7f6805c0a740 1 -- 172.21.15.18:0/3570428394 wait complete. 2024-04-16T17:25:56.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.353+0000 7f6805c0a740 20 client.31016 trim_cache size 0 max 16384 2024-04-16T17:25:56.353 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 -- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f67e00a3da0 msgr2=0x7f67e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f67e00a3da0 0x7f67e00a6260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009bf0 tx=0x7f67f4005d80 comp rx=0 tx=0).stop 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 -- 172.21.15.18:0/2162805785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc1a06d0 msgr2=0x7f67fc1ab110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc1a06d0 0x7f67fc1ab110 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e970 tx=0x7f67e800ee30 comp rx=0 tx=0).stop 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 -- 172.21.15.18:0/2162805785 shutdown_connections 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f67e00a3da0 0x7f67e00a6260 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc1a06d0 0x7f67fc1ab110 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67fc1a0270 0x7f67fc1a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 --2- 172.21.15.18:0/2162805785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67fc109350 0x7f67fc19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.354+0000 7f6801800640 1 -- 172.21.15.18:0/2162805785 >> 172.21.15.18:0/2162805785 conn(0x7f67fc0ffae0 msgr2=0x7f67fc079a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:56.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.358+0000 7f6801800640 1 -- 172.21.15.18:0/2162805785 shutdown_connections 2024-04-16T17:25:56.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.358+0000 7f6801800640 1 -- 172.21.15.18:0/2162805785 wait complete. 2024-04-16T17:25:56.385 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:25:56.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:55 smithi022 ceph-mon[36023]: pgmap v677: 129 pgs: 129 active+clean; 451 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-04-16T17:25:56.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:55 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/606242070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:25:56.623 INFO:tasks.workunit.client.0.smithi018.stdout: 10 105460 10.77 MB/sec execute 368 sec latency 774.213 ms 2024-04-16T17:25:56.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 -- 172.21.15.18:0/1107646514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa41090c0 msgr2=0x7feaa4115570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 --2- 172.21.15.18:0/1107646514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa41090c0 0x7feaa4115570 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009ca0 tx=0x7fea9c02c1b0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 -- 172.21.15.18:0/1107646514 shutdown_connections 2024-04-16T17:25:56.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 --2- 172.21.15.18:0/1107646514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa41090c0 0x7feaa4115570 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 --2- 172.21.15.18:0/1107646514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feaa4108720 0x7feaa4108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 --2- 172.21.15.18:0/1107646514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaa410a590 0x7feaa410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 -- 172.21.15.18:0/1107646514 >> 172.21.15.18:0/1107646514 conn(0x7feaa4077e70 msgr2=0x7feaa4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:56.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 -- 172.21.15.18:0/1107646514 shutdown_connections 2024-04-16T17:25:56.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.775+0000 7feaa9400640 1 -- 172.21.15.18:0/1107646514 wait complete. 2024-04-16T17:25:56.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.777+0000 7feaa9400640 1 Processor -- start 2024-04-16T17:25:56.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.777+0000 7feaa9400640 1 -- start start 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.777+0000 7feaa9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaa4108720 0x7feaa419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.777+0000 7feaa9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa41090c0 0x7feaa419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.777+0000 7feaa9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feaa410a590 0x7feaa41a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.777+0000 7feaa9400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4118b90 con 0x7feaa4108720 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.777+0000 7feaa9400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4118a10 con 0x7feaa410a590 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.777+0000 7feaa9400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4118d10 con 0x7feaa41090c0 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feaa410a590 0x7feaa41a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feaa410a590 0x7feaa41a38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56252/0 (socket says 172.21.15.18:56252) 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa3200640 1 -- 172.21.15.18:0/716336263 learned_addr learned my addr 172.21.15.18:0/716336263 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa1e00640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa41090c0 0x7feaa419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa2800640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaa4108720 0x7feaa419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa3200640 1 -- 172.21.15.18:0/716336263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa41090c0 msgr2=0x7feaa419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa3200640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa41090c0 0x7feaa419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa3200640 1 -- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaa4108720 msgr2=0x7feaa419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa3200640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaa4108720 0x7feaa419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa3200640 1 -- 172.21.15.18:0/716336263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa41a3fd0 con 0x7feaa410a590 2024-04-16T17:25:56.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.778+0000 7feaa2800640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaa4108720 0x7feaa419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:56.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.780+0000 7feaa3200640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feaa410a590 0x7feaa41a38a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fea9c034670 tx=0x7fea9c0346a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.781+0000 7fea9b400640 1 -- 172.21.15.18:0/716336263 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c042070 con 0x7feaa410a590 2024-04-16T17:25:56.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.781+0000 7fea9b400640 1 -- 172.21.15.18:0/716336263 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c032d20 con 0x7feaa410a590 2024-04-16T17:25:56.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.781+0000 7fea9b400640 1 -- 172.21.15.18:0/716336263 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c039070 con 0x7feaa410a590 2024-04-16T17:25:56.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.782+0000 7feaa9400640 1 -- 172.21.15.18:0/716336263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa41a4260 con 0x7feaa410a590 2024-04-16T17:25:56.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.782+0000 7feaa9400640 1 -- 172.21.15.18:0/716336263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa41abb50 con 0x7feaa410a590 2024-04-16T17:25:56.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.791+0000 7feaac186740 1 Processor -- start 2024-04-16T17:25:56.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.791+0000 7fea9b400640 1 -- 172.21.15.18:0/716336263 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea9c045070 con 0x7feaa410a590 2024-04-16T17:25:56.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.793+0000 7fea9b400640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea880a3e70 0x7fea880a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.794+0000 7feaa1e00640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea880a3e70 0x7fea880a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.794+0000 7feaac186740 1 -- start start 2024-04-16T17:25:56.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.794+0000 7feaac186740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd3c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.794+0000 7feaac186740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd44d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.794+0000 7feaac186740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd47640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.794+0000 7fea9b400640 1 -- 172.21.15.18:0/716336263 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fea9c0348c0 con 0x7feaa410a590 2024-04-16T17:25:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.795+0000 7feaa2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd3c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.795+0000 7feaa2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd3c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53472/0 (socket says 172.21.15.18:53472) 2024-04-16T17:25:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.795+0000 7feaa2800640 1 -- 172.21.15.18:0/206519128 learned_addr learned my addr 172.21.15.18:0/206519128 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.795+0000 7feaac186740 1 -- 172.21.15.18:0/206519128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ea9dca1020 con 0x55ea9dd3c670 2024-04-16T17:25:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.795+0000 7feaa3200640 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd47640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.795+0000 7feaac186740 1 -- 172.21.15.18:0/206519128 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ea9dc47460 con 0x55ea9dd45250 2024-04-16T17:25:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.796+0000 7feaac186740 1 -- 172.21.15.18:0/206519128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ea9dc9cfe0 con 0x55ea9dd3bd50 2024-04-16T17:25:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.796+0000 7feaa1e00640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea880a3e70 0x7fea880a6330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feaa4108080 tx=0x7fea90009520 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.796+0000 7feaa1e00640 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd44d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.796+0000 7feaa3200640 1 -- 172.21.15.18:0/206519128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 msgr2=0x55ea9dd3c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.796+0000 7feaa3200640 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd3c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.796+0000 7feaa3200640 1 -- 172.21.15.18:0/206519128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 msgr2=0x55ea9dd44d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.799+0000 7feaa3200640 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd44d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.799+0000 7feaa3200640 1 -- 172.21.15.18:0/206519128 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9dc4f040 con 0x55ea9dd45250 2024-04-16T17:25:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.800+0000 7feaa1e00640 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd44d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.800+0000 7feaa3200640 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd47640 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0efd70 tx=0x7fea9c0c3a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1365e7aec13b80e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.800+0000 7fea8f400640 1 -- 172.21.15.18:0/206519128 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0baff0 con 0x55ea9dd45250 2024-04-16T17:25:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.800+0000 7fea8f400640 1 -- 172.21.15.18:0/206519128 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0098d0 con 0x55ea9dd45250 2024-04-16T17:25:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.800+0000 7fea8f400640 1 -- 172.21.15.18:0/206519128 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0bb190 con 0x55ea9dd45250 2024-04-16T17:25:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.801+0000 7feaac186740 1 -- 172.21.15.18:0/206519128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 msgr2=0x55ea9dd47640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.801+0000 7feaac186740 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd47640 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0efd70 tx=0x7fea9c0c3a90 comp rx=0 tx=0).stop 2024-04-16T17:25:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.801+0000 7feaac186740 1 -- 172.21.15.18:0/206519128 shutdown_connections 2024-04-16T17:25:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.801+0000 7feaac186740 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd47640 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.801+0000 7feaac186740 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd44d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.801+0000 7feaac186740 1 --2- 172.21.15.18:0/206519128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd3c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.801+0000 7feaac186740 1 -- 172.21.15.18:0/206519128 >> 172.21.15.18:0/206519128 conn(0x55ea9dd36480 msgr2=0x55ea9dd36860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:56.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.802+0000 7feaac186740 1 -- 172.21.15.18:0/206519128 shutdown_connections 2024-04-16T17:25:56.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.804+0000 7feaac186740 1 -- 172.21.15.18:0/206519128 wait complete. 2024-04-16T17:25:56.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.804+0000 7feaac186740 1 -- start start 2024-04-16T17:25:56.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.805+0000 7feaac186740 1 Processor -- start 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.805+0000 7feaac186740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.805+0000 7feaac186740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3c670 0x55ea9dd42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.805+0000 7feaac186740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd42d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.805+0000 7feaac186740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ea9dd47e00 con 0x55ea9dd3bd50 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.805+0000 7feaac186740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ea9dc5a830 con 0x55ea9dd45250 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.805+0000 7feaac186740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ea9dd47f80 con 0x55ea9dd3c670 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaac186740 10 client.0 _ll_register_callbacks cb 0x55ea9dc62150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaa1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3c670 0x55ea9dd42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaa1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3c670 0x55ea9dd42830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:53486/0 (socket says 172.21.15.18:53486) 2024-04-16T17:25:56.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaa1e00640 1 -- 172.21.15.18:0/3213219636 learned_addr learned my addr 172.21.15.18:0/3213219636 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:56.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaa1e00640 1 -- 172.21.15.18:0/3213219636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 msgr2=0x55ea9dd42d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:25:56.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaa1e00640 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd42d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaa1e00640 1 -- 172.21.15.18:0/3213219636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3bd50 msgr2=0x55ea9dd422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:56.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaa1e00640 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:56.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.806+0000 7feaa1e00640 1 -- 172.21.15.18:0/3213219636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9dc5e920 con 0x55ea9dd3c670 2024-04-16T17:25:56.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.807+0000 7feaa1e00640 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3c670 0x55ea9dd42830 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fea90015840 tx=0x7fea900046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b873042eac660cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:56.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.807+0000 7fea83e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:56.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.807+0000 7fea83e00640 1 -- 172.21.15.18:0/3213219636 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90024030 con 0x55ea9dd3c670 2024-04-16T17:25:56.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.808+0000 7fea83e00640 1 -- 172.21.15.18:0/3213219636 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fea90016b90 con 0x55ea9dd3c670 2024-04-16T17:25:56.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.808+0000 7feaac186740 1 -- 172.21.15.18:0/3213219636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ea9dcaea40 con 0x55ea9dd3c670 2024-04-16T17:25:56.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.809+0000 7fea83e00640 1 -- 172.21.15.18:0/3213219636 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fea90016d90 con 0x55ea9dd3c670 2024-04-16T17:25:56.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.810+0000 7feaac186740 10 client.30865 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:56.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.810+0000 7feaac186740 1 -- 172.21.15.18:0/3213219636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ea9dc2c7f0 con 0x55ea9dd3c670 2024-04-16T17:25:56.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.810+0000 7fea83e00640 1 -- 172.21.15.18:0/3213219636 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ea9dcaea40 con 0x55ea9dd3c670 2024-04-16T17:25:56.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.810+0000 7fea83e00640 1 -- 172.21.15.18:0/3213219636 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fea9001f040 con 0x55ea9dd3c670 2024-04-16T17:25:56.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.811+0000 7feaac186740 10 client.30865 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:56.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.811+0000 7feaac186740 10 client.30865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:56.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.811+0000 7feaac186740 20 client.30865 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:56.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.811+0000 7feaac186740 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ea9dd663f0 0x55ea9dd68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:56.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.811+0000 7feaac186740 4 client.30865 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:56.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.811+0000 7feaac186740 1 -- 172.21.15.18:0/3213219636 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea9dca88b0 con 0x55ea9dd663f0 2024-04-16T17:25:56.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.811+0000 7feaa2800640 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ea9dd663f0 0x55ea9dd68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:56.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.812+0000 7feaa2800640 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ea9dd663f0 0x55ea9dd68880 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f9b12975d828e9 server_cookie=139b2d69931872c2 in_seq=0 out_seq=0 2024-04-16T17:25:56.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:56.813+0000 7fea83e00640 10 client.30865 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.602+0000 7fdc67e00640 1 -- 172.21.15.18:0/1554531296 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0f0c68010 con 0x55d0f0d7bc40 2024-04-16T17:25:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.602+0000 7fdc67e00640 10 client.30983 handle_command_reply: tid=0 2024-04-16T17:25:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.602+0000 7fdc8c1e9740 1 client.30983 shutdown 2024-04-16T17:25:57.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.602+0000 7fea83e00640 1 -- 172.21.15.18:0/3213219636 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea9dca88b0 con 0x55ea9dd663f0 2024-04-16T17:25:57.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.602+0000 7fea83e00640 10 client.30865 handle_command_reply: tid=0 2024-04-16T17:25:57.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.603+0000 7feaac186740 1 client.30865 shutdown 2024-04-16T17:25:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.603+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 msgr2=0x55d0f0cfeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:57.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.603+0000 7feaac186740 1 -- 172.21.15.18:0/3213219636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3c670 msgr2=0x55ea9dd42830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:57.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.603+0000 7feaac186740 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3c670 0x55ea9dd42830 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fea90015840 tx=0x7fea900046e0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.604+0000 7feaac186740 1 -- 172.21.15.18:0/3213219636 shutdown_connections 2024-04-16T17:25:57.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.604+0000 7feaac186740 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ea9dd663f0 0x55ea9dd68880 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0cfeb30 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x55d0f0cf2130 tx=0x7fdc7c005bf0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1554531296 shutdown_connections 2024-04-16T17:25:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.604+0000 7feaac186740 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd42d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fea83e00640 0 client.30865 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.604+0000 7feaac186740 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3c670 0x55ea9dd42830 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.604+0000 7feaac186740 1 --2- 172.21.15.18:0/3213219636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.604+0000 7feaac186740 1 -- 172.21.15.18:0/3213219636 >> 172.21.15.18:0/3213219636 conn(0x55ea9dd36480 msgr2=0x55ea9dd381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55d0f0d7bc40 0x55d0f0d7e060 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0f0d04e20 0x55d0f0cfeb30 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d0f0cfc610 0x55d0f0cfe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fdc67e00640 0 client.30983 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fdc8c1e9740 1 --2- 172.21.15.18:0/1554531296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0f0cfb8a0 0x55d0f0d0d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.605+0000 7feaac186740 1 -- 172.21.15.18:0/3213219636 shutdown_connections 2024-04-16T17:25:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.605+0000 7feaac186740 1 -- 172.21.15.18:0/3213219636 wait complete. 2024-04-16T17:25:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.604+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1554531296 >> 172.21.15.18:0/1554531296 conn(0x55d0f0c965d0 msgr2=0x55d0f0cf4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.605+0000 7feaac186740 20 client.30865 trim_cache size 0 max 16384 2024-04-16T17:25:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.605+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1554531296 shutdown_connections 2024-04-16T17:25:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.605+0000 7fdc8c1e9740 1 -- 172.21.15.18:0/1554531296 wait complete. 2024-04-16T17:25:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.606+0000 7fdc8c1e9740 20 client.30983 trim_cache size 0 max 16384 2024-04-16T17:25:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:25:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:25:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:25:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:25:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.613+0000 7fdc89400640 1 -- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc680a3df0 msgr2=0x7fdc680a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.613+0000 7fdc89400640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc680a3df0 0x7fdc680a62b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdc8406b520 tx=0x7fdc7c005f90 comp rx=0 tx=0).stop 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.613+0000 7fdc89400640 1 -- 172.21.15.18:0/227701083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076e50 msgr2=0x7fdc8419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.613+0000 7fdc89400640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076e50 0x7fdc8419f140 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c000c00 tx=0x7fdc6c004040 comp rx=0 tx=0).stop 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.614+0000 7fdc89400640 1 -- 172.21.15.18:0/227701083 shutdown_connections 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.614+0000 7fdc89400640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fdc680a3df0 0x7fdc680a62b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.614+0000 7fdc89400640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc841a3a00 0x7fdc841a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.614+0000 7fdc89400640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc84112ea0 0x7fdc8419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.614+0000 7fdc89400640 1 --2- 172.21.15.18:0/227701083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc84076e50 0x7fdc8419f140 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.614+0000 7fdc89400640 1 -- 172.21.15.18:0/227701083 >> 172.21.15.18:0/227701083 conn(0x7fdc840ffcd0 msgr2=0x7fdc84114600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.614+0000 7fdc89400640 1 -- 172.21.15.18:0/227701083 shutdown_connections 2024-04-16T17:25:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:25:57.614+0000 7fdc89400640 1 -- 172.21.15.18:0/227701083 wait complete. 2024-04-16T17:25:57.623 INFO:tasks.workunit.client.0.smithi018.stdout: 10 105574 10.76 MB/sec execute 369 sec latency 1186.538 ms 2024-04-16T17:25:57.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.642+0000 7feaac186740 1 Processor -- start 2024-04-16T17:25:57.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.642+0000 7feaac186740 1 -- start start 2024-04-16T17:25:57.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaac186740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd44a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:57.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaac186740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd3c670 0x55ea9dd50c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:57.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaac186740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd45250 0x55ea9dd6e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:57.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaac186740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41afca0 con 0x55ea9dd45250 2024-04-16T17:25:57.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaac186740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41ad570 con 0x55ea9dd3c670 2024-04-16T17:25:57.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaac186740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feaa40738e0 con 0x55ea9dd3bd50 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaa3200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd45250 0x55ea9dd6e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaa1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd3c670 0x55ea9dd50c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaa1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd3c670 0x55ea9dd50c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56552/0 (socket says 172.21.15.18:56552) 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.643+0000 7feaa1e00640 1 -- 172.21.15.18:0/2167041982 learned_addr learned my addr 172.21.15.18:0/2167041982 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.644+0000 7feaa1e00640 1 -- 172.21.15.18:0/2167041982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 msgr2=0x55ea9dd44a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.644+0000 7feaa1e00640 1 --2- 172.21.15.18:0/2167041982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd44a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.644+0000 7feaa1e00640 1 -- 172.21.15.18:0/2167041982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd45250 msgr2=0x55ea9dd6e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.644+0000 7feaa1e00640 1 --2- 172.21.15.18:0/2167041982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd45250 0x55ea9dd6e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.644+0000 7feaa1e00640 1 -- 172.21.15.18:0/2167041982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9dd3c4a0 con 0x55ea9dd3c670 2024-04-16T17:25:57.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.645+0000 7feaa1e00640 1 --2- 172.21.15.18:0/2167041982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd3c670 0x55ea9dd50c30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fea90029750 tx=0x7fea900096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d205cc072d17f6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:57.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.645+0000 7fea8ea00640 1 -- 172.21.15.18:0/2167041982 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90024070 con 0x55ea9dd3c670 2024-04-16T17:25:57.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.645+0000 7fea8ea00640 1 -- 172.21.15.18:0/2167041982 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9001a040 con 0x55ea9dd3c670 2024-04-16T17:25:57.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.645+0000 7fea8ea00640 1 -- 172.21.15.18:0/2167041982 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90033420 con 0x55ea9dd3c670 2024-04-16T17:25:57.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.647+0000 7feaac186740 1 -- 172.21.15.18:0/2167041982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd3c670 msgr2=0x55ea9dd50c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:57.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.647+0000 7feaac186740 1 --2- 172.21.15.18:0/2167041982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd3c670 0x55ea9dd50c30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fea90029750 tx=0x7fea900096b0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.647+0000 7feaac186740 1 -- 172.21.15.18:0/2167041982 shutdown_connections 2024-04-16T17:25:57.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.647+0000 7feaac186740 1 --2- 172.21.15.18:0/2167041982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd45250 0x55ea9dd6e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.647+0000 7feaac186740 1 --2- 172.21.15.18:0/2167041982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd3c670 0x55ea9dd50c30 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.647+0000 7feaac186740 1 --2- 172.21.15.18:0/2167041982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9dd44a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.648+0000 7feaac186740 1 -- 172.21.15.18:0/2167041982 >> 172.21.15.18:0/2167041982 conn(0x55ea9dd36480 msgr2=0x55ea9dd354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:57.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.649+0000 7feaac186740 1 -- 172.21.15.18:0/2167041982 shutdown_connections 2024-04-16T17:25:57.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.650+0000 7feaac186740 1 -- 172.21.15.18:0/2167041982 wait complete. 2024-04-16T17:25:57.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.650+0000 7feaac186740 1 -- start start 2024-04-16T17:25:57.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.651+0000 7feaac186740 1 Processor -- start 2024-04-16T17:25:57.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.651+0000 7feaac186740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9ddc8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:57.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.651+0000 7feaac186740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd4d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:57.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.651+0000 7feaac186740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd4d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaac186740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4073f30 con 0x55ea9dd3c670 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaa3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd4d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaa3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd4d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56560/0 (socket says 172.21.15.18:56560) 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaa3200640 1 -- 172.21.15.18:0/1519591336 learned_addr learned my addr 172.21.15.18:0/1519591336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaa1e00640 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd4d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaa2800640 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9ddc8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41acea0 con 0x55ea9dd45250 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41ac100 con 0x55ea9dd3bd50 2024-04-16T17:25:57.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.652+0000 7feaac186740 10 client.0 _ll_register_callbacks cb 0x7fea90009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:57.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.654+0000 7feaa2800640 1 -- 172.21.15.18:0/1519591336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 msgr2=0x55ea9dd4d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:57.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.654+0000 7feaa2800640 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd4d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.654+0000 7feaa2800640 1 -- 172.21.15.18:0/1519591336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 msgr2=0x55ea9dd4d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:57.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.654+0000 7feaa2800640 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd4d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.654+0000 7feaa2800640 1 -- 172.21.15.18:0/1519591336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9d9b33f0 con 0x55ea9dd3bd50 2024-04-16T17:25:57.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.655+0000 7feaa2800640 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9ddc8030 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x55ea9dd69090 tx=0x7fea9400ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89239ef8dea7b731 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:57.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.655+0000 7fea83e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.656+0000 7fea83e00640 1 -- 172.21.15.18:0/1519591336 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea940194a0 con 0x55ea9dd3bd50 2024-04-16T17:25:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.656+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9001a040 con 0x55ea9dd3bd50 2024-04-16T17:25:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.656+0000 7fea83e00640 1 -- 172.21.15.18:0/1519591336 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fea9400f030 con 0x55ea9dd3bd50 2024-04-16T17:25:57.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.657+0000 7fea83e00640 1 -- 172.21.15.18:0/1519591336 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fea94029050 con 0x55ea9dd3bd50 2024-04-16T17:25:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.657+0000 7feaac186740 10 client.30868 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.657+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ea9da7d7d0 con 0x55ea9dd3bd50 2024-04-16T17:25:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.657+0000 7fea83e00640 1 -- 172.21.15.18:0/1519591336 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9001a040 con 0x55ea9dd3bd50 2024-04-16T17:25:57.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.658+0000 7fea83e00640 1 -- 172.21.15.18:0/1519591336 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fea94022410 con 0x55ea9dd3bd50 2024-04-16T17:25:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.658+0000 7feaac186740 10 client.30868 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.658+0000 7feaac186740 10 client.30868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.658+0000 7feaac186740 20 client.30868 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.658+0000 7feaac186740 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ea9dd663f0 0x55ea9dde0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.658+0000 7feaac186740 4 client.30868 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:25:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.658+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea9dc24570 con 0x55ea9dd663f0 2024-04-16T17:25:57.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.658+0000 7feaa1e00640 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ea9dd663f0 0x55ea9dde0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:57.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.659+0000 7feaa1e00640 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ea9dd663f0 0x55ea9dde0080 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79fed9cd2d20e349 server_cookie=d373c4295cdb59e2 in_seq=0 out_seq=0 2024-04-16T17:25:57.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.659+0000 7fea83e00640 10 client.30868 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:57.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.665+0000 7fea83e00640 1 -- 172.21.15.18:0/1519591336 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea9dc24570 con 0x55ea9dd663f0 2024-04-16T17:25:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.665+0000 7fea83e00640 10 client.30868 handle_command_reply: tid=0 2024-04-16T17:25:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.665+0000 7feaac186740 1 client.30868 shutdown 2024-04-16T17:25:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.666+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 msgr2=0x55ea9ddc8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:57.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.666+0000 7feaac186740 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9ddc8030 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x55ea9dd69090 tx=0x7fea9400ad20 comp rx=0 tx=0).stop 2024-04-16T17:25:57.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.667+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 shutdown_connections 2024-04-16T17:25:57.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.667+0000 7feaac186740 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55ea9dd663f0 0x55ea9dde0080 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.667+0000 7feaac186740 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ea9dd45250 0x55ea9dd4d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.667+0000 7feaac186740 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ea9dd3c670 0x55ea9dd4d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.667+0000 7feaac186740 1 --2- 172.21.15.18:0/1519591336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ea9dd3bd50 0x55ea9ddc8030 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.667+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 >> 172.21.15.18:0/1519591336 conn(0x55ea9dd36480 msgr2=0x55ea9dd422b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:57.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.667+0000 7fea83e00640 0 client.30868 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.667+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 shutdown_connections 2024-04-16T17:25:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.668+0000 7feaac186740 1 -- 172.21.15.18:0/1519591336 wait complete. 2024-04-16T17:25:57.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.668+0000 7feaac186740 20 client.30868 trim_cache size 0 max 16384 2024-04-16T17:25:57.668 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:25:57.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.669+0000 7feaa9400640 1 -- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea880a3e70 msgr2=0x7fea880a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:57.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.669+0000 7feaa9400640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea880a3e70 0x7fea880a6330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feaa4108080 tx=0x7fea90009520 comp rx=0 tx=0).stop 2024-04-16T17:25:57.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.669+0000 7feaa9400640 1 -- 172.21.15.18:0/716336263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feaa410a590 msgr2=0x7feaa41a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:57.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.670+0000 7feaa9400640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feaa410a590 0x7feaa41a38a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fea9c034670 tx=0x7fea9c0346a0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.670+0000 7feaa9400640 1 -- 172.21.15.18:0/716336263 shutdown_connections 2024-04-16T17:25:57.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.670+0000 7feaa9400640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea880a3e70 0x7fea880a6330 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.670+0000 7feaa9400640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feaa410a590 0x7feaa41a38a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.670+0000 7feaa9400640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa41090c0 0x7feaa419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.670+0000 7feaa9400640 1 --2- 172.21.15.18:0/716336263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaa4108720 0x7feaa419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:57.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.670+0000 7feaa9400640 1 -- 172.21.15.18:0/716336263 >> 172.21.15.18:0/716336263 conn(0x7feaa4077e70 msgr2=0x7feaa40784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:57.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.670+0000 7feaa9400640 1 -- 172.21.15.18:0/716336263 shutdown_connections 2024-04-16T17:25:57.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:57.671+0000 7feaa9400640 1 -- 172.21.15.18:0/716336263 wait complete. 2024-04-16T17:25:58.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:57 smithi043 ceph-mon[36901]: pgmap v678: 129 pgs: 129 active+clean; 464 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-04-16T17:25:58.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:57 smithi022 ceph-mon[36023]: pgmap v678: 129 pgs: 129 active+clean; 464 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-04-16T17:25:58.623 INFO:tasks.workunit.client.0.smithi018.stdout: 10 105819 10.75 MB/sec execute 370 sec latency 206.937 ms 2024-04-16T17:25:58.699 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:25:58.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:25:58.699 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:25:59.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.071+0000 7fc88d200640 1 -- 172.21.15.18:0/1588379446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc888113c00 msgr2=0x7fc888115ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.071+0000 7fc88d200640 1 --2- 172.21.15.18:0/1588379446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc888113c00 0x7fc888115ff0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ca0 tx=0x7fc88002c190 comp rx=0 tx=0).stop 2024-04-16T17:25:59.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.073+0000 7fc88d200640 1 -- 172.21.15.18:0/1588379446 shutdown_connections 2024-04-16T17:25:59.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.073+0000 7fc88d200640 1 --2- 172.21.15.18:0/1588379446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc888113c00 0x7fc888115ff0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.073+0000 7fc88d200640 1 --2- 172.21.15.18:0/1588379446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc88806c070 0x7fc88806c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.073+0000 7fc88d200640 1 --2- 172.21.15.18:0/1588379446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc888073d50 0x7fc88806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.073+0000 7fc88d200640 1 -- 172.21.15.18:0/1588379446 >> 172.21.15.18:0/1588379446 conn(0x7fc888077ab0 msgr2=0x7fc8880fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.076+0000 7fc88d200640 1 -- 172.21.15.18:0/1588379446 shutdown_connections 2024-04-16T17:25:59.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.077+0000 7fc88d200640 1 -- 172.21.15.18:0/1588379446 wait complete. 2024-04-16T17:25:59.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.080+0000 7fc88d200640 1 Processor -- start 2024-04-16T17:25:59.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.081+0000 7fc88d200640 1 -- start start 2024-04-16T17:25:59.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.086+0000 7fc88d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc88806c070 0x7fc8881a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.086+0000 7fc88d200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc888073d50 0x7fc8881a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.086+0000 7fc88d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc888113c00 0x7fc8881a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.086+0000 7fc88d200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc88811aa10 con 0x7fc888073d50 2024-04-16T17:25:59.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.086+0000 7fc88d200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc88811a890 con 0x7fc88806c070 2024-04-16T17:25:59.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.086+0000 7fc88d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc88811ab90 con 0x7fc888113c00 2024-04-16T17:25:59.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.087+0000 7fc887400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc88806c070 0x7fc8881a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.087+0000 7fc886a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc888073d50 0x7fc8881a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.087+0000 7fc887400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc88806c070 0x7fc8881a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56582/0 (socket says 172.21.15.18:56582) 2024-04-16T17:25:59.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.087+0000 7fc887400640 1 -- 172.21.15.18:0/2600792537 learned_addr learned my addr 172.21.15.18:0/2600792537 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.088+0000 7fc887e00640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc888113c00 0x7fc8881a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.088+0000 7fc886a00640 1 -- 172.21.15.18:0/2600792537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc888113c00 msgr2=0x7fc8881a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.088+0000 7fc886a00640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc888113c00 0x7fc8881a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.088+0000 7fc886a00640 1 -- 172.21.15.18:0/2600792537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc88806c070 msgr2=0x7fc8881a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.088+0000 7fc886a00640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc88806c070 0x7fc8881a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.088+0000 7fc886a00640 1 -- 172.21.15.18:0/2600792537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8881a5ef0 con 0x7fc888073d50 2024-04-16T17:25:59.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.089+0000 7fc886a00640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc888073d50 0x7fc8881a1490 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc878005fd0 tx=0x7fc87800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.089+0000 7fc87fe00640 1 -- 172.21.15.18:0/2600792537 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878018070 con 0x7fc888073d50 2024-04-16T17:25:59.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.089+0000 7fc87fe00640 1 -- 172.21.15.18:0/2600792537 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc878009c30 con 0x7fc888073d50 2024-04-16T17:25:59.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.089+0000 7fc87fe00640 1 -- 172.21.15.18:0/2600792537 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8780035f0 con 0x7fc888073d50 2024-04-16T17:25:59.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.089+0000 7fc88d200640 1 -- 172.21.15.18:0/2600792537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8881a61e0 con 0x7fc888073d50 2024-04-16T17:25:59.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.090+0000 7fc88d200640 1 -- 172.21.15.18:0/2600792537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc888116cb0 con 0x7fc888073d50 2024-04-16T17:25:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.095+0000 7fc8915d6740 1 Processor -- start 2024-04-16T17:25:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.096+0000 7fc8915d6740 1 -- start start 2024-04-16T17:25:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.097+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5e5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.097+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc5ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.097+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5ee390 0x560dfc5f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.097+0000 7fc8915d6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560dfc549cb0 con 0x560dfc5e57b0 2024-04-16T17:25:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.097+0000 7fc8915d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560dfc4f0370 con 0x560dfc5e4e90 2024-04-16T17:25:59.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.097+0000 7fc8915d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560dfc545c70 con 0x560dfc5ee390 2024-04-16T17:25:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.097+0000 7fc887e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.098+0000 7fc887e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5e5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56592/0 (socket says 172.21.15.18:56592) 2024-04-16T17:25:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.098+0000 7fc887e00640 1 -- 172.21.15.18:0/3347628307 learned_addr learned my addr 172.21.15.18:0/3347628307 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.098+0000 7fc887e00640 1 -- 172.21.15.18:0/3347628307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5ee390 msgr2=0x560dfc5f0780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:25:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.098+0000 7fc887e00640 1 --2- 172.21.15.18:0/3347628307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5ee390 0x560dfc5f0780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.098+0000 7fc887e00640 1 -- 172.21.15.18:0/3347628307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 msgr2=0x560dfc5ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.098+0000 7fc887e00640 1 --2- 172.21.15.18:0/3347628307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc5ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.098+0000 7fc887e00640 1 -- 172.21.15.18:0/3347628307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dfc50f910 con 0x560dfc5e4e90 2024-04-16T17:25:59.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.099+0000 7fc87fe00640 1 -- 172.21.15.18:0/2600792537 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc878025440 con 0x7fc888073d50 2024-04-16T17:25:59.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.099+0000 7fc887e00640 1 --2- 172.21.15.18:0/3347628307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5e5270 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ca0 tx=0x7fc880009970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6303c6c38dcf2cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.099+0000 7fc87cc00640 1 -- 172.21.15.18:0/3347628307 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880043070 con 0x560dfc5e4e90 2024-04-16T17:25:59.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.100+0000 7fc87cc00640 1 -- 172.21.15.18:0/3347628307 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc880003b70 con 0x560dfc5e4e90 2024-04-16T17:25:59.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.100+0000 7fc87cc00640 1 -- 172.21.15.18:0/3347628307 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880039040 con 0x560dfc5e4e90 2024-04-16T17:25:59.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.101+0000 7fc8915d6740 1 -- 172.21.15.18:0/3347628307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 msgr2=0x560dfc5e5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.101+0000 7fc8915d6740 1 --2- 172.21.15.18:0/3347628307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5e5270 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ca0 tx=0x7fc880009970 comp rx=0 tx=0).stop 2024-04-16T17:25:59.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.101+0000 7fc87fe00640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc86c0a3da0 0x7fc86c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.101+0000 7fc87fe00640 1 -- 172.21.15.18:0/2600792537 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fc8780d45f0 con 0x7fc888073d50 2024-04-16T17:25:59.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.102+0000 7fc8915d6740 1 -- 172.21.15.18:0/3347628307 shutdown_connections 2024-04-16T17:25:59.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.102+0000 7fc8915d6740 1 --2- 172.21.15.18:0/3347628307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5ee390 0x560dfc5f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.102+0000 7fc8915d6740 1 --2- 172.21.15.18:0/3347628307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc5ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.102+0000 7fc8915d6740 1 --2- 172.21.15.18:0/3347628307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5e5270 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.102+0000 7fc8915d6740 1 -- 172.21.15.18:0/3347628307 >> 172.21.15.18:0/3347628307 conn(0x560dfc5df5c0 msgr2=0x560dfc5df9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.102+0000 7fc887e00640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc86c0a3da0 0x7fc86c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.103+0000 7fc8915d6740 1 -- 172.21.15.18:0/3347628307 shutdown_connections 2024-04-16T17:25:59.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.104+0000 7fc8915d6740 1 -- 172.21.15.18:0/3347628307 wait complete. 2024-04-16T17:25:59.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.104+0000 7fc8915d6740 1 -- start start 2024-04-16T17:25:59.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.105+0000 7fc8915d6740 1 Processor -- start 2024-04-16T17:25:59.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.104+0000 7fc887e00640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc86c0a3da0 0x7fc86c0a6260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc8800049b0 tx=0x7fc88002c190 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.107+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e4e90 0x560dfc5eb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.107+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e57b0 0x560dfc5eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.107+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5ee390 0x560dfc5ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.107+0000 7fc8915d6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560dfc5f1120 con 0x560dfc5e4e90 2024-04-16T17:25:59.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.108+0000 7fc887e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5ee390 0x560dfc5ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.108+0000 7fc887e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5ee390 0x560dfc5ebe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56602/0 (socket says 172.21.15.18:56602) 2024-04-16T17:25:59.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.108+0000 7fc887e00640 1 -- 172.21.15.18:0/2551833492 learned_addr learned my addr 172.21.15.18:0/2551833492 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.108+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560dfc503730 con 0x560dfc5ee390 2024-04-16T17:25:59.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.108+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560dfc5f12a0 con 0x560dfc5e57b0 2024-04-16T17:25:59.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.108+0000 7fc8915d6740 10 client.0 _ll_register_callbacks cb 0x560dfc4d21e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:59.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.108+0000 7fc886a00640 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e57b0 0x560dfc5eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.109+0000 7fc886a00640 1 -- 172.21.15.18:0/2551833492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5ee390 msgr2=0x560dfc5ebe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.109+0000 7fc886a00640 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5ee390 0x560dfc5ebe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.109+0000 7fc886a00640 1 -- 172.21.15.18:0/2551833492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e4e90 msgr2=0x560dfc5eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.109+0000 7fc886a00640 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e4e90 0x560dfc5eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.109+0000 7fc886a00640 1 -- 172.21.15.18:0/2551833492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dfc3247e0 con 0x560dfc5e57b0 2024-04-16T17:25:59.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.110+0000 7fc886a00640 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e57b0 0x560dfc5eb8c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc8780d1a40 tx=0x7fc8780a0120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f671d6a5fdc2cce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.110+0000 7fc871600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:59.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.110+0000 7fc871600640 1 -- 172.21.15.18:0/2551833492 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8780b9060 con 0x560dfc5e57b0 2024-04-16T17:25:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.110+0000 7fc871600640 1 -- 172.21.15.18:0/2551833492 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fc8780d1bb0 con 0x560dfc5e57b0 2024-04-16T17:25:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.111+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc880003b70 con 0x560dfc5e57b0 2024-04-16T17:25:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.111+0000 7fc871600640 1 -- 172.21.15.18:0/2551833492 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc8780be050 con 0x560dfc5e57b0 2024-04-16T17:25:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.111+0000 7fc8915d6740 10 client.30877 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.111+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560dfc384f70 con 0x560dfc5e57b0 2024-04-16T17:25:59.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.112+0000 7fc871600640 1 -- 172.21.15.18:0/2551833492 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc880003b70 con 0x560dfc5e57b0 2024-04-16T17:25:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.112+0000 7fc871600640 1 -- 172.21.15.18:0/2551833492 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fc8780a04f0 con 0x560dfc5e57b0 2024-04-16T17:25:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.112+0000 7fc8915d6740 10 client.30877 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.112+0000 7fc8915d6740 10 client.30877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:59.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.112+0000 7fc8915d6740 20 client.30877 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.112+0000 7fc8915d6740 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560dfc5fe740 0x560dfc600b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.112+0000 7fc8915d6740 4 client.30877 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.112+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dfc5576d0 con 0x560dfc5fe740 2024-04-16T17:25:59.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.113+0000 7fc887400640 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560dfc5fe740 0x560dfc600b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.114+0000 7fc871600640 10 client.30877 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:59.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.114+0000 7fc887400640 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560dfc5fe740 0x560dfc600b30 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5e69472f2e9d2 server_cookie=51f44998b247aa3e in_seq=0 out_seq=0 2024-04-16T17:25:59.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.125+0000 7fc871600640 1 -- 172.21.15.18:0/2551833492 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560dfc5576d0 con 0x560dfc5fe740 2024-04-16T17:25:59.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.125+0000 7fc871600640 10 client.30877 handle_command_reply: tid=0 2024-04-16T17:25:59.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.127+0000 7fc8915d6740 1 client.30877 shutdown 2024-04-16T17:25:59.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.128+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e57b0 msgr2=0x560dfc5eb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.128+0000 7fc8915d6740 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e57b0 0x560dfc5eb8c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc8780d1a40 tx=0x7fc8780a0120 comp rx=0 tx=0).stop 2024-04-16T17:25:59.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.128+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 shutdown_connections 2024-04-16T17:25:59.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.128+0000 7fc8915d6740 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560dfc5fe740 0x560dfc600b30 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.128+0000 7fc8915d6740 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5ee390 0x560dfc5ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.128+0000 7fc8915d6740 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e57b0 0x560dfc5eb8c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.128+0000 7fc8915d6740 1 --2- 172.21.15.18:0/2551833492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e4e90 0x560dfc5eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.128+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 >> 172.21.15.18:0/2551833492 conn(0x560dfc5df5c0 msgr2=0x560dfc5e1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.129+0000 7fc871600640 0 client.30877 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:59.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.130+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 shutdown_connections 2024-04-16T17:25:59.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.130+0000 7fc8915d6740 1 -- 172.21.15.18:0/2551833492 wait complete. 2024-04-16T17:25:59.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.131+0000 7fc8915d6740 20 client.30877 trim_cache size 0 max 16384 2024-04-16T17:25:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.165+0000 7fc8915d6740 1 Processor -- start 2024-04-16T17:25:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.165+0000 7fc8915d6740 1 -- start start 2024-04-16T17:25:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5df5c0 0x560dfc5fa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc6063c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc8915d6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b8370 con 0x560dfc5e57b0 2024-04-16T17:25:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc8915d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b5c40 con 0x560dfc5e4e90 2024-04-16T17:25:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc8915d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b3ab0 con 0x560dfc5df5c0 2024-04-16T17:25:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc887400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5df5c0 0x560dfc5fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc887400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5df5c0 0x560dfc5fa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38294/0 (socket says 172.21.15.18:38294) 2024-04-16T17:25:59.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc887400640 1 -- 172.21.15.18:0/3267821785 learned_addr learned my addr 172.21.15.18:0/3267821785 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.166+0000 7fc886a00640 1 --2- 172.21.15.18:0/3267821785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.167+0000 7fc887400640 1 -- 172.21.15.18:0/3267821785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 msgr2=0x560dfc5ebc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.167+0000 7fc887400640 1 --2- 172.21.15.18:0/3267821785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5ebc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.167+0000 7fc887400640 1 -- 172.21.15.18:0/3267821785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 msgr2=0x560dfc6063c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.167+0000 7fc887400640 1 --2- 172.21.15.18:0/3267821785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc6063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.167+0000 7fc887400640 1 -- 172.21.15.18:0/3267821785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dfc5e55e0 con 0x560dfc5df5c0 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.167+0000 7fc887400640 1 --2- 172.21.15.18:0/3267821785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5df5c0 0x560dfc5fa010 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc874009290 tx=0x7fc874009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d44182d4e0e63ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.167+0000 7fc873e00640 1 -- 172.21.15.18:0/3267821785 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874017370 con 0x560dfc5df5c0 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.168+0000 7fc873e00640 1 -- 172.21.15.18:0/3267821785 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc874014530 con 0x560dfc5df5c0 2024-04-16T17:25:59.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.168+0000 7fc873e00640 1 -- 172.21.15.18:0/3267821785 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874013690 con 0x560dfc5df5c0 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 -- 172.21.15.18:0/3267821785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5df5c0 msgr2=0x560dfc5fa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 --2- 172.21.15.18:0/3267821785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5df5c0 0x560dfc5fa010 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc874009290 tx=0x7fc874009a20 comp rx=0 tx=0).stop 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 -- 172.21.15.18:0/3267821785 shutdown_connections 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 --2- 172.21.15.18:0/3267821785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc6063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 --2- 172.21.15.18:0/3267821785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5e4e90 0x560dfc5ebc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 --2- 172.21.15.18:0/3267821785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5df5c0 0x560dfc5fa010 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 -- 172.21.15.18:0/3267821785 >> 172.21.15.18:0/3267821785 conn(0x560dfc5e6d50 msgr2=0x560dfc5e12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 -- 172.21.15.18:0/3267821785 shutdown_connections 2024-04-16T17:25:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.169+0000 7fc8915d6740 1 -- 172.21.15.18:0/3267821785 wait complete. 2024-04-16T17:25:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.170+0000 7fc8915d6740 1 -- start start 2024-04-16T17:25:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.170+0000 7fc8915d6740 1 Processor -- start 2024-04-16T17:25:59.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.170+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5df5c0 0x560dfc5f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e4e90 0x560dfc5f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc8915d6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc5f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc8915d6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b4100 con 0x560dfc5e57b0 2024-04-16T17:25:59.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc8915d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b5570 con 0x560dfc5df5c0 2024-04-16T17:25:59.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc8915d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b47d0 con 0x560dfc5e4e90 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc8915d6740 10 client.0 _ll_register_callbacks cb 0x7fc87809c530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5df5c0 0x560dfc5f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5df5c0 0x560dfc5f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56630/0 (socket says 172.21.15.18:56630) 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887400640 1 -- 172.21.15.18:0/484209908 learned_addr learned my addr 172.21.15.18:0/484209908 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc886a00640 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e4e90 0x560dfc5f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887400640 1 -- 172.21.15.18:0/484209908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e4e90 msgr2=0x560dfc5f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887400640 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e4e90 0x560dfc5f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887400640 1 -- 172.21.15.18:0/484209908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 msgr2=0x560dfc5f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887e00640 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc5f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887400640 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc5f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887400640 1 -- 172.21.15.18:0/484209908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dfc21c170 con 0x560dfc5df5c0 2024-04-16T17:25:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.171+0000 7fc887e00640 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc5f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc887400640 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5df5c0 0x560dfc5f20d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc874004810 tx=0x7fc874014980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=343190cbf36cf7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc871600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc871600640 1 -- 172.21.15.18:0/484209908 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874024070 con 0x560dfc5df5c0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc8915d6740 1 -- 172.21.15.18:0/484209908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc874014530 con 0x560dfc5df5c0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc871600640 1 -- 172.21.15.18:0/484209908 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fc874017780 con 0x560dfc5df5c0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc871600640 1 -- 172.21.15.18:0/484209908 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc874030050 con 0x560dfc5df5c0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc8915d6740 10 client.31046 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc8915d6740 1 -- 172.21.15.18:0/484209908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560dfc66d740 con 0x560dfc5df5c0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.172+0000 7fc871600640 1 -- 172.21.15.18:0/484209908 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc874014530 con 0x560dfc5df5c0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc871600640 1 -- 172.21.15.18:0/484209908 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fc874013550 con 0x560dfc5df5c0 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc8915d6740 10 client.31046 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc8915d6740 10 client.31046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc8915d6740 20 client.31046 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc8915d6740 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560dfc5ee390 0x560dfc66fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc8915d6740 4 client.31046 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:25:59.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc8915d6740 1 -- 172.21.15.18:0/484209908 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560dfc551540 con 0x560dfc5ee390 2024-04-16T17:25:59.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc886a00640 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560dfc5ee390 0x560dfc66fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.173+0000 7fc886a00640 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560dfc5ee390 0x560dfc66fd70 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b8da7b47402c7d server_cookie=4ae6ee0fd2941268 in_seq=0 out_seq=0 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.174+0000 7fc871600640 10 client.31046 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.174+0000 7fc871600640 1 -- 172.21.15.18:0/484209908 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560dfc551540 con 0x560dfc5ee390 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.174+0000 7fc871600640 10 client.31046 handle_command_reply: tid=0 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 client.31046 shutdown 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 -- 172.21.15.18:0/484209908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5df5c0 msgr2=0x560dfc5f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5df5c0 0x560dfc5f20d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc874004810 tx=0x7fc874014980 comp rx=0 tx=0).stop 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 -- 172.21.15.18:0/484209908 shutdown_connections 2024-04-16T17:25:59.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x560dfc5ee390 0x560dfc66fd70 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560dfc5e57b0 0x560dfc5f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc871600640 0 client.31046 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dfc5e4e90 0x560dfc5f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 --2- 172.21.15.18:0/484209908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560dfc5df5c0 0x560dfc5f20d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 -- 172.21.15.18:0/484209908 >> 172.21.15.18:0/484209908 conn(0x560dfc5e6d50 msgr2=0x560dfc5ddcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.175+0000 7fc8915d6740 1 -- 172.21.15.18:0/484209908 shutdown_connections 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.176+0000 7fc8915d6740 1 -- 172.21.15.18:0/484209908 wait complete. 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.176+0000 7fc8915d6740 20 client.31046 trim_cache size 0 max 16384 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:25:59.177 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:25:59.178 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "07d7b902-7efc-4252-aa96-c0af041c4217", 2024-04-16T17:25:59.178 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:25:59.178 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:25:59.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 -- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc86c0a3da0 msgr2=0x7fc86c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc86c0a3da0 0x7fc86c0a6260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc8800049b0 tx=0x7fc88002c190 comp rx=0 tx=0).stop 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 -- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc888073d50 msgr2=0x7fc8881a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc888073d50 0x7fc8881a1490 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc878005fd0 tx=0x7fc87800dd20 comp rx=0 tx=0).stop 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 -- 172.21.15.18:0/2600792537 shutdown_connections 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc86c0a3da0 0x7fc86c0a6260 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc888113c00 0x7fc8881a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc888073d50 0x7fc8881a1490 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 --2- 172.21.15.18:0/2600792537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc88806c070 0x7fc8881a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 -- 172.21.15.18:0/2600792537 >> 172.21.15.18:0/2600792537 conn(0x7fc888077ab0 msgr2=0x7fc888071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 -- 172.21.15.18:0/2600792537 shutdown_connections 2024-04-16T17:25:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.177+0000 7fc88d200640 1 -- 172.21.15.18:0/2600792537 wait complete. 2024-04-16T17:25:59.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:25:59.204 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:25:59.269 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:57 smithi018 ceph-mon[31776]: pgmap v678: 129 pgs: 129 active+clean; 464 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-04-16T17:25:59.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.542+0000 7f06dac00640 1 -- 172.21.15.18:0/1300495225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4101910 msgr2=0x7f06d4105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.543+0000 7f06dac00640 1 --2- 172.21.15.18:0/1300495225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4101910 0x7f06d4105950 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009ca0 tx=0x7f06c402c100 comp rx=0 tx=0).stop 2024-04-16T17:25:59.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.543+0000 7f06dac00640 1 -- 172.21.15.18:0/1300495225 shutdown_connections 2024-04-16T17:25:59.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.543+0000 7f06dac00640 1 --2- 172.21.15.18:0/1300495225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06d4106030 0x7f06d410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.543+0000 7f06dac00640 1 --2- 172.21.15.18:0/1300495225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4101910 0x7f06d4105950 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.543+0000 7f06dac00640 1 --2- 172.21.15.18:0/1300495225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d4100f60 0x7f06d4101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.543+0000 7f06dac00640 1 -- 172.21.15.18:0/1300495225 >> 172.21.15.18:0/1300495225 conn(0x7f06d4077ab0 msgr2=0x7f06d40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.543+0000 7f06dac00640 1 -- 172.21.15.18:0/1300495225 shutdown_connections 2024-04-16T17:25:59.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.543+0000 7f06dac00640 1 -- 172.21.15.18:0/1300495225 wait complete. 2024-04-16T17:25:59.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.545+0000 7f06dac00640 1 Processor -- start 2024-04-16T17:25:59.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.545+0000 7f06dac00640 1 -- start start 2024-04-16T17:25:59.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06dac00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4100f60 0x7f06d41965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06dac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d4106030 0x7f06d4196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06dac00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06d41971e0 0x7f06d419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06dac00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f06d4110020 con 0x7f06d41971e0 2024-04-16T17:25:59.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06dac00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06d410fea0 con 0x7f06d4100f60 2024-04-16T17:25:59.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06dac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41101a0 con 0x7f06d4106030 2024-04-16T17:25:59.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06d9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4100f60 0x7f06d41965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06d9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4100f60 0x7f06d41965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56648/0 (socket says 172.21.15.18:56648) 2024-04-16T17:25:59.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06d9800640 1 -- 172.21.15.18:0/2025450767 learned_addr learned my addr 172.21.15.18:0/2025450767 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.546+0000 7f06d8e00640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d4106030 0x7f06d4196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.547+0000 7f06d9800640 1 -- 172.21.15.18:0/2025450767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d4106030 msgr2=0x7f06d4196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.547+0000 7f06d9800640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d4106030 0x7f06d4196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.547+0000 7f06d9800640 1 -- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06d41971e0 msgr2=0x7f06d419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.547+0000 7f06d9800640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06d41971e0 0x7f06d419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.547+0000 7f06d9800640 1 -- 172.21.15.18:0/2025450767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d419b9e0 con 0x7f06d4100f60 2024-04-16T17:25:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.547+0000 7f06d8e00640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d4106030 0x7f06d4196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:25:59.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.547+0000 7f06d9800640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4100f60 0x7f06d41965e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00e9f0 tx=0x7f06cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.548+0000 7f06d2000640 1 -- 172.21.15.18:0/2025450767 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00cd80 con 0x7f06d4100f60 2024-04-16T17:25:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.548+0000 7f06d2000640 1 -- 172.21.15.18:0/2025450767 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06cc010910 con 0x7f06d4100f60 2024-04-16T17:25:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.548+0000 7f06dac00640 1 -- 172.21.15.18:0/2025450767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d419bcd0 con 0x7f06d4100f60 2024-04-16T17:25:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.548+0000 7f06d2000640 1 -- 172.21.15.18:0/2025450767 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc018990 con 0x7f06d4100f60 2024-04-16T17:25:59.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.548+0000 7f06dac00640 1 -- 172.21.15.18:0/2025450767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d41a3700 con 0x7f06d4100f60 2024-04-16T17:25:59.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.553+0000 7f06d2000640 1 -- 172.21.15.18:0/2025450767 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f06cc010ac0 con 0x7f06d4100f60 2024-04-16T17:25:59.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.554+0000 7f06df02e740 1 Processor -- start 2024-04-16T17:25:59.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.554+0000 7f06df02e740 1 -- start start 2024-04-16T17:25:59.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.554+0000 7f06df02e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f3d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.555+0000 7f06df02e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f46380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.555+0000 7f06df02e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f48cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.555+0000 7f06df02e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cae2ea3120 con 0x55cae2f3d3c0 2024-04-16T17:25:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.555+0000 7f06df02e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cae2e49340 con 0x55cae2f3dce0 2024-04-16T17:25:59.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.555+0000 7f06df02e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cae2e9f0e0 con 0x55cae2f468c0 2024-04-16T17:25:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.556+0000 7f06d9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f46380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.556+0000 7f06d9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f46380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56656/0 (socket says 172.21.15.18:56656) 2024-04-16T17:25:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.556+0000 7f06d9800640 1 -- 172.21.15.18:0/3518346264 learned_addr learned my addr 172.21.15.18:0/3518346264 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.556+0000 7f06d8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f48cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.556+0000 7f06da200640 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f3d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.556+0000 7f06d2000640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f06b80a3d80 0x7f06b80a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.557+0000 7f06d9800640 1 -- 172.21.15.18:0/3518346264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 msgr2=0x55cae2f48cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.557+0000 7f06d9800640 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f48cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.557+0000 7f06d9800640 1 -- 172.21.15.18:0/3518346264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 msgr2=0x55cae2f3d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.557+0000 7f06d9800640 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f3d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.557+0000 7f06d9800640 1 -- 172.21.15.18:0/3518346264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae2c7f7d0 con 0x55cae2f3dce0 2024-04-16T17:25:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.557+0000 7f06da200640 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f3d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:25:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.558+0000 7f06d2000640 1 -- 172.21.15.18:0/2025450767 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f06cc0d0ff0 con 0x7f06d4100f60 2024-04-16T17:25:59.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.558+0000 7f06d9800640 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f46380 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0ce2a0 tx=0x7f06cc05c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6a0b27645f2efd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.559+0000 7f06caa00640 1 -- 172.21.15.18:0/3518346264 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc026040 con 0x55cae2f3dce0 2024-04-16T17:25:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.559+0000 7f06caa00640 1 -- 172.21.15.18:0/3518346264 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06cc09d280 con 0x55cae2f3dce0 2024-04-16T17:25:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.559+0000 7f06caa00640 1 -- 172.21.15.18:0/3518346264 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0a0f30 con 0x55cae2f3dce0 2024-04-16T17:25:59.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.559+0000 7f06da200640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f06b80a3d80 0x7f06b80a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.560+0000 7f06df02e740 1 -- 172.21.15.18:0/3518346264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 msgr2=0x55cae2f46380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.560+0000 7f06df02e740 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f46380 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0ce2a0 tx=0x7f06cc05c760 comp rx=0 tx=0).stop 2024-04-16T17:25:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.560+0000 7f06da200640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f06b80a3d80 0x7f06b80a6240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f06c0006fd0 tx=0x7f06c00047d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.560+0000 7f06df02e740 1 -- 172.21.15.18:0/3518346264 shutdown_connections 2024-04-16T17:25:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.560+0000 7f06df02e740 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f48cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.560+0000 7f06df02e740 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f46380 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.561+0000 7f06df02e740 1 --2- 172.21.15.18:0/3518346264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f3d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.561+0000 7f06df02e740 1 -- 172.21.15.18:0/3518346264 >> 172.21.15.18:0/3518346264 conn(0x55cae2f37af0 msgr2=0x55cae2f37ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.561+0000 7f06df02e740 1 -- 172.21.15.18:0/3518346264 shutdown_connections 2024-04-16T17:25:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.561+0000 7f06df02e740 1 -- 172.21.15.18:0/3518346264 wait complete. 2024-04-16T17:25:59.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.562+0000 7f06df02e740 1 -- start start 2024-04-16T17:25:59.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.563+0000 7f06df02e740 1 Processor -- start 2024-04-16T17:25:59.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.564+0000 7f06df02e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f43920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.564+0000 7f06df02e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f43e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.564+0000 7f06df02e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f443a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06d8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f43e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06d8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f43e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:56670/0 (socket says 172.21.15.18:56670) 2024-04-16T17:25:59.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06da200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06da200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38340/0 (socket says 172.21.15.18:38340) 2024-04-16T17:25:59.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06df02e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cae2f49650 con 0x55cae2f3d3c0 2024-04-16T17:25:59.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06d8e00640 1 -- 172.21.15.18:0/2344521485 learned_addr learned my addr 172.21.15.18:0/2344521485 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cae2e5c710 con 0x55cae2f3dce0 2024-04-16T17:25:59.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cae2f497d0 con 0x55cae2f468c0 2024-04-16T17:25:59.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.565+0000 7f06df02e740 10 client.0 _ll_register_callbacks cb 0x55cae2e57af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:59.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.566+0000 7f06d9800640 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f43920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.566+0000 7f06da200640 1 -- 172.21.15.18:0/2344521485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 msgr2=0x55cae2f43e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.566+0000 7f06da200640 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f43e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.566+0000 7f06da200640 1 -- 172.21.15.18:0/2344521485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 msgr2=0x55cae2f43920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.566+0000 7f06da200640 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f43920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.567+0000 7f06da200640 1 -- 172.21.15.18:0/2344521485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae2bb53f0 con 0x55cae2f468c0 2024-04-16T17:25:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.567+0000 7f06d8e00640 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f43e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.567+0000 7f06bf400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:59.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.568+0000 7f06da200640 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f443a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f06c000c520 tx=0x7f06c0004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12aa35647e863159 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.568+0000 7f06bf400640 1 -- 172.21.15.18:0/2344521485 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c001c7c0 con 0x55cae2f468c0 2024-04-16T17:25:59.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.568+0000 7f06bf400640 1 -- 172.21.15.18:0/2344521485 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f06c001c5c0 con 0x55cae2f468c0 2024-04-16T17:25:59.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.569+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06cc09d280 con 0x55cae2f468c0 2024-04-16T17:25:59.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.569+0000 7f06bf400640 1 -- 172.21.15.18:0/2344521485 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f06c001c7c0 con 0x55cae2f468c0 2024-04-16T17:25:59.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.569+0000 7f06df02e740 10 client.30895 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:59.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.569+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cae2b750f0 con 0x55cae2f468c0 2024-04-16T17:25:59.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.570+0000 7f06bf400640 1 -- 172.21.15.18:0/2344521485 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06cc09d280 con 0x55cae2f468c0 2024-04-16T17:25:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.570+0000 7f06bf400640 1 -- 172.21.15.18:0/2344521485 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f06c001da50 con 0x55cae2f468c0 2024-04-16T17:25:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.571+0000 7f06df02e740 10 client.30895 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.571+0000 7f06df02e740 10 client.30895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.571+0000 7f06df02e740 20 client.30895 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.571+0000 7f06df02e740 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cae2f56a00 0x55cae2f58e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.571+0000 7f06df02e740 4 client.30895 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:25:59.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.571+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae2eb0b40 con 0x55cae2f56a00 2024-04-16T17:25:59.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.572+0000 7f06d9800640 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cae2f56a00 0x55cae2f58e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.573+0000 7f06d9800640 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cae2f56a00 0x55cae2f58e60 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35940c683f618a1 server_cookie=3cb4ada48116209e in_seq=0 out_seq=0 2024-04-16T17:25:59.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.573+0000 7f06bf400640 10 client.30895 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:59.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.577+0000 7f06bf400640 1 -- 172.21.15.18:0/2344521485 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cae2eb0b40 con 0x55cae2f56a00 2024-04-16T17:25:59.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.577+0000 7f06bf400640 10 client.30895 handle_command_reply: tid=0 2024-04-16T17:25:59.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.577+0000 7f06df02e740 1 client.30895 shutdown 2024-04-16T17:25:59.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 msgr2=0x55cae2f443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06df02e740 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f443a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f06c000c520 tx=0x7f06c0004840 comp rx=0 tx=0).stop 2024-04-16T17:25:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 shutdown_connections 2024-04-16T17:25:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06df02e740 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cae2f56a00 0x55cae2f58e60 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06bf400640 0 client.30895 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06df02e740 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f468c0 0x55cae2f443a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06df02e740 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3dce0 0x55cae2f43e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06df02e740 1 --2- 172.21.15.18:0/2344521485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f43920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.578+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 >> 172.21.15.18:0/2344521485 conn(0x55cae2f37af0 msgr2=0x55cae2f39840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.579+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 shutdown_connections 2024-04-16T17:25:59.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.579+0000 7f06df02e740 1 -- 172.21.15.18:0/2344521485 wait complete. 2024-04-16T17:25:59.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.580+0000 7f06df02e740 20 client.30895 trim_cache size 0 max 16384 2024-04-16T17:25:59.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.619+0000 7f06df02e740 1 Processor -- start 2024-04-16T17:25:59.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.619+0000 7f06df02e740 1 -- start start 2024-04-16T17:25:59.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.619+0000 7f06df02e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3d3c0 0x55cae2f46050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.619+0000 7f06df02e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f52510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06df02e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f468c0 0x55cae2f5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06df02e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41d2320 con 0x55cae2f468c0 2024-04-16T17:25:59.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06df02e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41adbe0 con 0x55cae2f3d3c0 2024-04-16T17:25:59.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06df02e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41aba50 con 0x55cae2f3dce0 2024-04-16T17:25:59.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06d8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f52510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06d8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f52510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38354/0 (socket says 172.21.15.18:38354) 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06d8e00640 1 -- 172.21.15.18:0/3816863375 learned_addr learned my addr 172.21.15.18:0/3816863375 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06d9800640 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3d3c0 0x55cae2f46050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06da200640 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f468c0 0x55cae2f5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06d8e00640 1 -- 172.21.15.18:0/3816863375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3d3c0 msgr2=0x55cae2f46050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06d8e00640 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3d3c0 0x55cae2f46050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06d8e00640 1 -- 172.21.15.18:0/3816863375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f468c0 msgr2=0x55cae2f5e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.620+0000 7f06d8e00640 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f468c0 0x55cae2f5e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.621+0000 7f06d8e00640 1 -- 172.21.15.18:0/3816863375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae2f3f510 con 0x55cae2f3dce0 2024-04-16T17:25:59.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.621+0000 7f06d8e00640 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f52510 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f06d4074de0 tx=0x7f06c4009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9c662d877f0cbf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.622+0000 7f06ca000640 1 -- 172.21.15.18:0/3816863375 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4045070 con 0x55cae2f3dce0 2024-04-16T17:25:59.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.622+0000 7f06ca000640 1 -- 172.21.15.18:0/3816863375 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c4003970 con 0x55cae2f3dce0 2024-04-16T17:25:59.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.622+0000 7f06ca000640 1 -- 172.21.15.18:0/3816863375 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4039040 con 0x55cae2f3dce0 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 -- 172.21.15.18:0/3816863375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 msgr2=0x55cae2f52510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f52510 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f06d4074de0 tx=0x7f06c4009940 comp rx=0 tx=0).stop 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 -- 172.21.15.18:0/3816863375 shutdown_connections 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f468c0 0x55cae2f5e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f52510 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 --2- 172.21.15.18:0/3816863375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f3d3c0 0x55cae2f46050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 -- 172.21.15.18:0/3816863375 >> 172.21.15.18:0/3816863375 conn(0x55cae2f37af0 msgr2=0x55cae2f421f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 -- 172.21.15.18:0/3816863375 shutdown_connections 2024-04-16T17:25:59.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.623+0000 7f06df02e740 1 -- 172.21.15.18:0/3816863375 wait complete. 2024-04-16T17:25:59.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.624+0000 7f06df02e740 1 -- start start 2024-04-16T17:25:59.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.624+0000 7f06df02e740 1 Processor -- start 2024-04-16T17:25:59.624 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106003 10.74 MB/sec execute 371 sec latency 267.997 ms 2024-04-16T17:25:59.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.625+0000 7f06df02e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f4ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.625+0000 7f06df02e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f4ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.625+0000 7f06df02e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f468c0 0x55cae2f40020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.625+0000 7f06df02e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41ac0a0 con 0x55cae2f3d3c0 2024-04-16T17:25:59.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.625+0000 7f06df02e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41ad510 con 0x55cae2f468c0 2024-04-16T17:25:59.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.625+0000 7f06df02e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41ac770 con 0x55cae2f3dce0 2024-04-16T17:25:59.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.625+0000 7f06df02e740 10 client.0 _ll_register_callbacks cb 0x7f06c000a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:25:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.625+0000 7f06d9800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f4ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06d8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f4ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06da200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f468c0 0x55cae2f40020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06d9800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f4ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57768/0 (socket says 172.21.15.18:57768) 2024-04-16T17:25:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06d9800640 1 -- 172.21.15.18:0/2908042215 learned_addr learned my addr 172.21.15.18:0/2908042215 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:25:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06d8e00640 1 -- 172.21.15.18:0/2908042215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f468c0 msgr2=0x55cae2f40020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06d8e00640 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f468c0 0x55cae2f40020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06d8e00640 1 -- 172.21.15.18:0/2908042215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 msgr2=0x55cae2f4ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:25:59.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06d8e00640 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f4ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06d8e00640 1 -- 172.21.15.18:0/2908042215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae2f40850 con 0x55cae2f3dce0 2024-04-16T17:25:59.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.626+0000 7f06da200640 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f468c0 0x55cae2f40020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:25:59.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.627+0000 7f06d8e00640 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f4ef60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f06c402e010 tx=0x7f06c4032590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0deb2f2f6165d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:25:59.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.627+0000 7f06bf400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:25:59.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.627+0000 7f06bf400640 1 -- 172.21.15.18:0/2908042215 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4045070 con 0x55cae2f3dce0 2024-04-16T17:25:59.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.627+0000 7f06bf400640 1 -- 172.21.15.18:0/2908042215 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f06c4003d20 con 0x55cae2f3dce0 2024-04-16T17:25:59.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.628+0000 7f06df02e740 1 -- 172.21.15.18:0/2908042215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c4003970 con 0x55cae2f3dce0 2024-04-16T17:25:59.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.628+0000 7f06bf400640 1 -- 172.21.15.18:0/2908042215 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f06c4039040 con 0x55cae2f3dce0 2024-04-16T17:25:59.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.628+0000 7f06df02e740 10 client.30901 fetch_fsmap learned FSMap version 37 2024-04-16T17:25:59.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.629+0000 7f06df02e740 1 -- 172.21.15.18:0/2908042215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cae2fbcb50 con 0x55cae2f3dce0 2024-04-16T17:25:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.629+0000 7f06bf400640 1 -- 172.21.15.18:0/2908042215 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c4003970 con 0x55cae2f3dce0 2024-04-16T17:25:59.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.630+0000 7f06bf400640 1 -- 172.21.15.18:0/2908042215 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f06c404a7b0 con 0x55cae2f3dce0 2024-04-16T17:25:59.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.630+0000 7f06df02e740 10 client.30901 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:25:59.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.630+0000 7f06df02e740 10 client.30901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:25:59.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.630+0000 7f06df02e740 20 client.30901 populate_metadata read hostname 'smithi018' 2024-04-16T17:25:59.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.630+0000 7f06df02e740 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cae2fbd170 0x55cae2fbf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:25:59.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.630+0000 7f06df02e740 4 client.30901 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:25:59.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.630+0000 7f06df02e740 1 -- 172.21.15.18:0/2908042215 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cae2eaa9b0 con 0x55cae2fbd170 2024-04-16T17:25:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.632+0000 7f06d9800640 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cae2fbd170 0x55cae2fbf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:25:59.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.633+0000 7f06d9800640 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cae2fbd170 0x55cae2fbf590 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7652b95d88c62f24 server_cookie=b8ff81ca3948eefd in_seq=0 out_seq=0 2024-04-16T17:25:59.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.633+0000 7f06bf400640 10 client.30901 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:59.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06bf400640 1 -- 172.21.15.18:0/2908042215 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55cae2eaa9b0 con 0x55cae2fbd170 2024-04-16T17:25:59.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06bf400640 10 client.30901 handle_command_reply: tid=0 2024-04-16T17:25:59.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 client.30901 shutdown 2024-04-16T17:25:59.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 -- 172.21.15.18:0/2908042215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 msgr2=0x55cae2f4ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f4ef60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f06c402e010 tx=0x7f06c4032590 comp rx=0 tx=0).stop 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 -- 172.21.15.18:0/2908042215 shutdown_connections 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55cae2fbd170 0x55cae2fbf590 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cae2f468c0 0x55cae2f40020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cae2f3dce0 0x55cae2f4ef60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 --2- 172.21.15.18:0/2908042215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cae2f3d3c0 0x55cae2f4ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 -- 172.21.15.18:0/2908042215 >> 172.21.15.18:0/2908042215 conn(0x55cae2f37af0 msgr2=0x55cae2f362c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06bf400640 0 client.30901 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.635+0000 7f06df02e740 1 -- 172.21.15.18:0/2908042215 shutdown_connections 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.636+0000 7f06df02e740 1 -- 172.21.15.18:0/2908042215 wait complete. 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.636+0000 7f06df02e740 20 client.30901 trim_cache size 0 max 16384 2024-04-16T17:25:59.636 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (336 inodes in the stack)", 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout: "07d7b902-7efc-4252-aa96-c0af041c4217": { 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "07d7b902-7efc-4252-aa96-c0af041c4217", 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:25:59.637 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:25:59.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.637+0000 7f06dac00640 1 -- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f06b80a3d80 msgr2=0x7f06b80a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.637+0000 7f06dac00640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f06b80a3d80 0x7f06b80a6240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f06c0006fd0 tx=0x7f06c00047d0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.638+0000 7f06dac00640 1 -- 172.21.15.18:0/2025450767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4100f60 msgr2=0x7f06d41965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:25:59.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.638+0000 7f06dac00640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4100f60 0x7f06d41965e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00e9f0 tx=0x7f06cc00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.639+0000 7f06dac00640 1 -- 172.21.15.18:0/2025450767 shutdown_connections 2024-04-16T17:25:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.639+0000 7f06dac00640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f06b80a3d80 0x7f06b80a6240 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.639+0000 7f06dac00640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06d41971e0 0x7f06d419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.639+0000 7f06dac00640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d4106030 0x7f06d4196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.639+0000 7f06dac00640 1 --2- 172.21.15.18:0/2025450767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d4100f60 0x7f06d41965e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:25:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.639+0000 7f06dac00640 1 -- 172.21.15.18:0/2025450767 >> 172.21.15.18:0/2025450767 conn(0x7f06d4077ab0 msgr2=0x7f06d4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:25:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.639+0000 7f06dac00640 1 -- 172.21.15.18:0/2025450767 shutdown_connections 2024-04-16T17:25:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:25:59.639+0000 7f06dac00640 1 -- 172.21.15.18:0/2025450767 wait complete. 2024-04-16T17:25:59.662 INFO:tasks.cephfs.filesystem:scrub status for tag:07d7b902-7efc-4252-aa96-c0af041c4217 - {'path': '/', 'tag': '07d7b902-7efc-4252-aa96-c0af041c4217', 'options': 'recursive,force'} 2024-04-16T17:26:00.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:59 smithi018 ceph-mon[31776]: pgmap v679: 129 pgs: 129 active+clean; 464 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 217 op/s 2024-04-16T17:26:00.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:59 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:26:00.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:59 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:26:00.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:59 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:59 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:59 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:26:00.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:25:59 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:59 smithi043 ceph-mon[36901]: pgmap v679: 129 pgs: 129 active+clean; 464 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 217 op/s 2024-04-16T17:26:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:59 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:26:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:59 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:26:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:59 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:59 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:59 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:26:00.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:25:59 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:59 smithi022 ceph-mon[36023]: pgmap v679: 129 pgs: 129 active+clean; 464 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 217 op/s 2024-04-16T17:26:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:59 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:26:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:59 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:26:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:59 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:59 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:59 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:26:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:25:59 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:00.624 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106060 10.72 MB/sec execute 372 sec latency 856.583 ms 2024-04-16T17:26:01.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:00 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:26:01.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:00 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:01.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:00 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:26:01.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:00 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:01.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:00 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:26:01.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:00 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:01.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:00 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:26:01.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:00 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:01.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:00 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:26:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:00 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:00 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:26:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:00 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:01.624 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106060 10.69 MB/sec execute 373 sec latency 1856.697 ms 2024-04-16T17:26:02.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:01 smithi043 ceph-mon[36901]: pgmap v680: 129 pgs: 129 active+clean; 458 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-04-16T17:26:02.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:01 smithi022 ceph-mon[36023]: pgmap v680: 129 pgs: 129 active+clean; 458 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-04-16T17:26:02.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:01 smithi018 ceph-mon[31776]: pgmap v680: 129 pgs: 129 active+clean; 458 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-04-16T17:26:02.625 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106060 10.66 MB/sec execute 374 sec latency 2858.100 ms 2024-04-16T17:26:03.626 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106131 10.64 MB/sec execute 375 sec latency 3546.899 ms 2024-04-16T17:26:04.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:03 smithi018 ceph-mon[31776]: pgmap v681: 129 pgs: 129 active+clean; 450 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-04-16T17:26:04.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:03 smithi043 ceph-mon[36901]: pgmap v681: 129 pgs: 129 active+clean; 450 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-04-16T17:26:04.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:03 smithi022 ceph-mon[36023]: pgmap v681: 129 pgs: 129 active+clean; 450 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2024-04-16T17:26:04.626 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106197 10.62 MB/sec execute 376 sec latency 774.717 ms 2024-04-16T17:26:05.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:05 smithi022 ceph-mon[36023]: osdmap e134: 12 total, 12 up, 12 in 2024-04-16T17:26:05.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:05 smithi043 ceph-mon[36901]: osdmap e134: 12 total, 12 up, 12 in 2024-04-16T17:26:05.626 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106335 10.62 MB/sec execute 377 sec latency 1544.180 ms 2024-04-16T17:26:05.800 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:05 smithi018 ceph-mon[31776]: osdmap e134: 12 total, 12 up, 12 in 2024-04-16T17:26:06.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:06 smithi043 ceph-mon[36901]: pgmap v683: 129 pgs: 129 active+clean; 452 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 553 op/s 2024-04-16T17:26:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:06 smithi022 ceph-mon[36023]: pgmap v683: 129 pgs: 129 active+clean; 452 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 553 op/s 2024-04-16T17:26:06.626 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106426 10.60 MB/sec execute 378 sec latency 1823.959 ms 2024-04-16T17:26:07.285 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:06 smithi018 ceph-mon[31776]: pgmap v683: 129 pgs: 129 active+clean; 452 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 553 op/s 2024-04-16T17:26:07.626 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106521 10.57 MB/sec execute 379 sec latency 2824.022 ms 2024-04-16T17:26:08.320 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:08 smithi018 ceph-mon[31776]: pgmap v684: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 455 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-04-16T17:26:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:08 smithi022 ceph-mon[36023]: pgmap v684: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 455 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-04-16T17:26:08.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:08 smithi043 ceph-mon[36901]: pgmap v684: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 455 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-04-16T17:26:08.628 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106544 10.55 MB/sec execute 380 sec latency 3090.021 ms 2024-04-16T17:26:09.628 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106591 10.53 MB/sec execute 381 sec latency 2466.350 ms 2024-04-16T17:26:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:10 smithi022 ceph-mon[36023]: pgmap v685: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 455 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-04-16T17:26:10.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:10 smithi043 ceph-mon[36901]: pgmap v685: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 455 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-04-16T17:26:10.629 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106722 10.52 MB/sec execute 382 sec latency 2220.421 ms 2024-04-16T17:26:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.828+0000 7fc7f9a00640 1 -- 172.21.15.18:0/1482596511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f4102c30 msgr2=0x7fc7f4106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.829+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/1482596511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f4102c30 0x7fc7f4106c70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009900 tx=0x7fc7e002bc30 comp rx=0 tx=0).stop 2024-04-16T17:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.829+0000 7fc7f9a00640 1 -- 172.21.15.18:0/1482596511 shutdown_connections 2024-04-16T17:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.829+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/1482596511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f4107350 0x7fc7f410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.829+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/1482596511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f4102c30 0x7fc7f4106c70 secure :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009900 tx=0x7fc7e002bc30 comp rx=0 tx=0).stop 2024-04-16T17:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.829+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/1482596511 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f4102280 0x7fc7f4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.829+0000 7fc7f9a00640 1 -- 172.21.15.18:0/1482596511 >> 172.21.15.18:0/1482596511 conn(0x7fc7f40fdac0 msgr2=0x7fc7f40ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.831+0000 7fc7f9a00640 1 -- 172.21.15.18:0/1482596511 shutdown_connections 2024-04-16T17:26:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.831+0000 7fc7f9a00640 1 -- 172.21.15.18:0/1482596511 wait complete. 2024-04-16T17:26:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.832+0000 7fc7f9a00640 1 Processor -- start 2024-04-16T17:26:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.832+0000 7fc7f9a00640 1 -- start start 2024-04-16T17:26:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f9a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f4102280 0x7fc7f4074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f4107350 0x7fc7f4074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f4075fa0 0x7fc7f4074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f9a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4112430 con 0x7fc7f4102280 2024-04-16T17:26:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f9a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41122b0 con 0x7fc7f4075fa0 2024-04-16T17:26:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f9a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41125b0 con 0x7fc7f4107350 2024-04-16T17:26:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f4107350 0x7fc7f4074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f4107350 0x7fc7f4074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38568/0 (socket says 172.21.15.18:38568) 2024-04-16T17:26:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.833+0000 7fc7f2000640 1 -- 172.21.15.18:0/637494023 learned_addr learned my addr 172.21.15.18:0/637494023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.834+0000 7fc7f2000640 1 -- 172.21.15.18:0/637494023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f4075fa0 msgr2=0x7fc7f4074d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.834+0000 7fc7f2000640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f4075fa0 0x7fc7f4074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.834+0000 7fc7f2000640 1 -- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f4102280 msgr2=0x7fc7f4074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.834+0000 7fc7f2000640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f4102280 0x7fc7f4074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.834+0000 7fc7f2000640 1 -- 172.21.15.18:0/637494023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f419cd80 con 0x7fc7f4107350 2024-04-16T17:26:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.834+0000 7fc7f2000640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f4107350 0x7fc7f4074840 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc7e002c240 tx=0x7fc7e0004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.835+0000 7fc7eb400640 1 -- 172.21.15.18:0/637494023 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0043020 con 0x7fc7f4107350 2024-04-16T17:26:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.835+0000 7fc7eb400640 1 -- 172.21.15.18:0/637494023 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e00054e0 con 0x7fc7f4107350 2024-04-16T17:26:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.835+0000 7fc7eb400640 1 -- 172.21.15.18:0/637494023 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0003bc0 con 0x7fc7f4107350 2024-04-16T17:26:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.835+0000 7fc7f9a00640 1 -- 172.21.15.18:0/637494023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f419d010 con 0x7fc7f4107350 2024-04-16T17:26:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.836+0000 7fc7f9a00640 1 -- 172.21.15.18:0/637494023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f419d510 con 0x7fc7f4107350 2024-04-16T17:26:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.843+0000 7fc7eb400640 1 -- 172.21.15.18:0/637494023 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7e0046020 con 0x7fc7f4107350 2024-04-16T17:26:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.844+0000 7fc7fc7dc740 1 Processor -- start 2024-04-16T17:26:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.846+0000 7fc7eb400640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7d80a3d10 0x7fc7d80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.846+0000 7fc7eb400640 1 -- 172.21.15.18:0/637494023 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7fc7e00f2000 con 0x7fc7f4107350 2024-04-16T17:26:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.846+0000 7fc7fc7dc740 1 -- start start 2024-04-16T17:26:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7f3400640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7d80a3d10 0x7fc7d80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b45360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b45d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b51210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a4b7aaa780 con 0x55a4b7b4ee20 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4b7a50f40 con 0x55a4b7b46610 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a4b7aa6740 con 0x55a4b7b458a0 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7f2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b45360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7f2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b45360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46806/0 (socket says 172.21.15.18:46806) 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7f2a00640 1 -- 172.21.15.18:0/1365292482 learned_addr learned my addr 172.21.15.18:0/1365292482 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.847+0000 7fc7f2000640 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b45d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7f2000640 1 -- 172.21.15.18:0/1365292482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 msgr2=0x55a4b7b45360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7f2000640 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b45360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7f2000640 1 -- 172.21.15.18:0/1365292482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 msgr2=0x55a4b7b51210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7f2000640 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b51210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7f2000640 1 -- 172.21.15.18:0/1365292482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b77bd210 con 0x55a4b7b458a0 2024-04-16T17:26:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b45360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7f2000640 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b45d20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00ef400 tx=0x7fc7e00bda20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e635b582876bfd7f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7dfe00640 1 -- 172.21.15.18:0/1365292482 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0004c80 con 0x55a4b7b458a0 2024-04-16T17:26:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7dfe00640 1 -- 172.21.15.18:0/1365292482 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e0005690 con 0x55a4b7b458a0 2024-04-16T17:26:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.848+0000 7fc7dfe00640 1 -- 172.21.15.18:0/1365292482 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0003e30 con 0x55a4b7b458a0 2024-04-16T17:26:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.849+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1365292482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 msgr2=0x55a4b7b45d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.849+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b45d20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00ef400 tx=0x7fc7e00bda20 comp rx=0 tx=0).stop 2024-04-16T17:26:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.850+0000 7fc7f3400640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7d80a3d10 0x7fc7d80a61d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0049d0 tx=0x7fc7ec009400 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.850+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1365292482 shutdown_connections 2024-04-16T17:26:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.850+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b51210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.850+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b45d20 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.850+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1365292482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b45360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.850+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1365292482 >> 172.21.15.18:0/1365292482 conn(0x55a4b7ae05d0 msgr2=0x55a4b7b40590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.855+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1365292482 shutdown_connections 2024-04-16T17:26:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.856+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1365292482 wait complete. 2024-04-16T17:26:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.856+0000 7fc7fc7dc740 1 -- start start 2024-04-16T17:26:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.857+0000 7fc7fc7dc740 1 Processor -- start 2024-04-16T17:26:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b4c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a4b7aa6740 con 0x55a4b7b4ee20 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4b7a64390 con 0x55a4b7b46610 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a4b7b51910 con 0x55a4b7b458a0 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7fc7dc740 10 client.0 _ll_register_callbacks cb 0x55a4b7a4df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7f2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b4c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7f2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b4c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46816/0 (socket says 172.21.15.18:46816) 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7f2000640 1 -- 172.21.15.18:0/4087166503 learned_addr learned my addr 172.21.15.18:0/4087166503 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.858+0000 7fc7f3400640 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.859+0000 7fc7f2000640 1 -- 172.21.15.18:0/4087166503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 msgr2=0x55a4b7b4bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.859+0000 7fc7f2000640 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.859+0000 7fc7f2000640 1 -- 172.21.15.18:0/4087166503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 msgr2=0x55a4b7b4c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.859+0000 7fc7f2000640 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.859+0000 7fc7f2000640 1 -- 172.21.15.18:0/4087166503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b7889620 con 0x55a4b7b46610 2024-04-16T17:26:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.859+0000 7fc7f3400640 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:26:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.859+0000 7fc7f2000640 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b4c240 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0006010 tx=0x7fc7e0004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa23b5f0746f64bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.859+0000 7fc7dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.860+0000 7fc7dcc00640 1 -- 172.21.15.18:0/4087166503 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00dc020 con 0x55a4b7b46610 2024-04-16T17:26:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.860+0000 7fc7dcc00640 1 -- 172.21.15.18:0/4087166503 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7fc7e00ba2c0 con 0x55a4b7b46610 2024-04-16T17:26:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.860+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/4087166503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a4b7ab81a0 con 0x55a4b7b46610 2024-04-16T17:26:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.860+0000 7fc7dcc00640 1 -- 172.21.15.18:0/4087166503 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc7e00d6070 con 0x55a4b7b46610 2024-04-16T17:26:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.862+0000 7fc7fc7dc740 10 client.31073 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.862+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/4087166503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a4b7b5d860 con 0x55a4b7b46610 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.864+0000 7fc7dcc00640 1 -- 172.21.15.18:0/4087166503 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a4b7ab81a0 con 0x55a4b7b46610 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.864+0000 7fc7dcc00640 1 -- 172.21.15.18:0/4087166503 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fc7e0084c80 con 0x55a4b7b46610 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.864+0000 7fc7fc7dc740 10 client.31073 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.864+0000 7fc7fc7dc740 10 client.31073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.864+0000 7fc7fc7dc740 20 client.31073 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.864+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a4b7b5ebb0 0x55a4b7b60fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.864+0000 7fc7fc7dc740 4 client.31073 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.864+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/4087166503 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4b7ab2010 con 0x55a4b7b5ebb0 2024-04-16T17:26:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.865+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a4b7b5ebb0 0x55a4b7b60fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.866+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a4b7b5ebb0 0x55a4b7b60fa0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c435b1f031a88c9 server_cookie=42325e80e8ec5d24 in_seq=0 out_seq=0 2024-04-16T17:26:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.866+0000 7fc7dcc00640 10 client.31073 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.868+0000 7fc7dcc00640 1 -- 172.21.15.18:0/4087166503 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4b7ab2010 con 0x55a4b7b5ebb0 2024-04-16T17:26:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.868+0000 7fc7dcc00640 10 client.31073 handle_command_reply: tid=0 2024-04-16T17:26:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.868+0000 7fc7fc7dc740 1 client.31073 shutdown 2024-04-16T17:26:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/4087166503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 msgr2=0x55a4b7b4c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b4c240 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0006010 tx=0x7fc7e0004c80 comp rx=0 tx=0).stop 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/4087166503 shutdown_connections 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a4b7b5ebb0 0x55a4b7b60fa0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b46610 0x55a4b7b4c240 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/4087166503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7dcc00640 0 client.31073 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/4087166503 >> 172.21.15.18:0/4087166503 conn(0x55a4b7ae05d0 msgr2=0x55a4b7b422e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.869+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/4087166503 shutdown_connections 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.870+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/4087166503 wait complete. 2024-04-16T17:26:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.870+0000 7fc7fc7dc740 20 client.31073 trim_cache size 0 max 16384 2024-04-16T17:26:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.911+0000 7fc7fc7dc740 1 Processor -- start 2024-04-16T17:26:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.913+0000 7fc7fc7dc740 1 -- start start 2024-04-16T17:26:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.913+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.913+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 0x55a4b7b4ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.913+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.914+0000 7fc7f2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.914+0000 7fc7f2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46828/0 (socket says 172.21.15.18:46828) 2024-04-16T17:26:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.916+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41c1630 con 0x55a4b7b46610 2024-04-16T17:26:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.916+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41bef00 con 0x55a4b7b458a0 2024-04-16T17:26:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.916+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41a5220 con 0x55a4b7b4ee20 2024-04-16T17:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.917+0000 7fc7f2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 0x55a4b7b4ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.917+0000 7fc7f2a00640 1 -- 172.21.15.18:0/2260019727 learned_addr learned my addr 172.21.15.18:0/2260019727 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.917+0000 7fc7f3400640 1 --2- 172.21.15.18:0/2260019727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.917+0000 7fc7f2a00640 1 -- 172.21.15.18:0/2260019727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b4ee20 msgr2=0x55a4b7b4c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.917+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/2260019727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.917+0000 7fc7f2a00640 1 -- 172.21.15.18:0/2260019727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 msgr2=0x55a4b7b4ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.917+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/2260019727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 0x55a4b7b4ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.917+0000 7fc7f2a00640 1 -- 172.21.15.18:0/2260019727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b7b41160 con 0x55a4b7b458a0 2024-04-16T17:26:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.919+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/2260019727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4e430 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x55a4b7b61790 tx=0x7fc7e400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17534b6cddacee16 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.919+0000 7fc7df400640 1 -- 172.21.15.18:0/2260019727 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40139f0 con 0x55a4b7b458a0 2024-04-16T17:26:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.919+0000 7fc7df400640 1 -- 172.21.15.18:0/2260019727 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e401a810 con 0x55a4b7b458a0 2024-04-16T17:26:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.919+0000 7fc7df400640 1 -- 172.21.15.18:0/2260019727 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40119a0 con 0x55a4b7b458a0 2024-04-16T17:26:10.920 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:10 smithi018 ceph-mon[31776]: pgmap v685: 129 pgs: 17 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 68 active+clean; 455 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-04-16T17:26:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.920+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/2260019727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b458a0 msgr2=0x55a4b7b4e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.920+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/2260019727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4e430 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x55a4b7b61790 tx=0x7fc7e400b8c0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.920+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/2260019727 shutdown_connections 2024-04-16T17:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.920+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/2260019727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b4c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.920+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/2260019727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 0x55a4b7b4ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.920+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/2260019727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b458a0 0x55a4b7b4e430 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.920+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/2260019727 >> 172.21.15.18:0/2260019727 conn(0x55a4b7ae05d0 msgr2=0x55a4b7b4bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.921+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/2260019727 shutdown_connections 2024-04-16T17:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.921+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/2260019727 wait complete. 2024-04-16T17:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.921+0000 7fc7fc7dc740 1 -- start start 2024-04-16T17:26:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.921+0000 7fc7fc7dc740 1 Processor -- start 2024-04-16T17:26:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.922+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b56fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.922+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 0x55a4b7b48710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.922+0000 7fc7fc7dc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b48c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.922+0000 7fc7f2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b56fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.923+0000 7fc7f2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b56fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:38586/0 (socket says 172.21.15.18:38586) 2024-04-16T17:26:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f2000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 0x55a4b7b48710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41bd3e0 con 0x55a4b7b46610 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41be830 con 0x55a4b7b4ee20 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7fc7dc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41bda90 con 0x55a4b7b458a0 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7fc7dc740 10 client.0 _ll_register_callbacks cb 0x7fc7e400b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b48c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b48c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46830/0 (socket says 172.21.15.18:46830) 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f3400640 1 -- 172.21.15.18:0/1021676922 learned_addr learned my addr 172.21.15.18:0/1021676922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f3400640 1 -- 172.21.15.18:0/1021676922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 msgr2=0x55a4b7b56fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f3400640 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b56fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f3400640 1 -- 172.21.15.18:0/1021676922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 msgr2=0x55a4b7b48710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f3400640 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 0x55a4b7b48710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.924+0000 7fc7f3400640 1 -- 172.21.15.18:0/1021676922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b7b494c0 con 0x55a4b7b4ee20 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.925+0000 7fc7f3400640 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b48c90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0092c0 tx=0x7fc7ec004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9062aae4b9fab332 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.925+0000 7fc7dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.925+0000 7fc7dcc00640 1 -- 172.21.15.18:0/1021676922 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec023030 con 0x55a4b7b4ee20 2024-04-16T17:26:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.926+0000 7fc7dcc00640 1 -- 172.21.15.18:0/1021676922 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7fc7ec009d20 con 0x55a4b7b4ee20 2024-04-16T17:26:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.926+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1021676922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e401a810 con 0x55a4b7b4ee20 2024-04-16T17:26:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.926+0000 7fc7dcc00640 1 -- 172.21.15.18:0/1021676922 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc7ec02f050 con 0x55a4b7b4ee20 2024-04-16T17:26:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.926+0000 7fc7fc7dc740 10 client.31079 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.926+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1021676922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a4b7bc5140 con 0x55a4b7b4ee20 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.926+0000 7fc7dcc00640 1 -- 172.21.15.18:0/1021676922 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e401a810 con 0x55a4b7b4ee20 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.927+0000 7fc7dcc00640 1 -- 172.21.15.18:0/1021676922 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fc7ec01d040 con 0x55a4b7b4ee20 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.927+0000 7fc7fc7dc740 10 client.31079 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.927+0000 7fc7fc7dc740 10 client.31079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.927+0000 7fc7fc7dc740 20 client.31079 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.927+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a4b7bc5830 0x55a4b7bc7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.927+0000 7fc7fc7dc740 4 client.31079 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.927+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1021676922 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4b7a2e530 con 0x55a4b7bc5830 2024-04-16T17:26:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.928+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a4b7bc5830 0x55a4b7bc7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.928+0000 7fc7f2a00640 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a4b7bc5830 0x55a4b7bc7c50 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f59a63797f8bbbaf server_cookie=e5800ee0dea4ebf0 in_seq=0 out_seq=0 2024-04-16T17:26:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:10.928+0000 7fc7dcc00640 10 client.31079 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:11.629 INFO:tasks.workunit.client.0.smithi018.stdout: 10 106859 10.51 MB/sec execute 383 sec latency 553.694 ms 2024-04-16T17:26:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.279+0000 7fc7dcc00640 1 -- 172.21.15.18:0/1021676922 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4b7a2e530 con 0x55a4b7bc5830 2024-04-16T17:26:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.279+0000 7fc7dcc00640 10 client.31079 handle_command_reply: tid=0 2024-04-16T17:26:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.280+0000 7fc7fc7dc740 1 client.31079 shutdown 2024-04-16T17:26:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1021676922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b4ee20 msgr2=0x55a4b7b48c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b48c90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0092c0 tx=0x7fc7ec004a40 comp rx=0 tx=0).stop 2024-04-16T17:26:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1021676922 shutdown_connections 2024-04-16T17:26:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55a4b7bc5830 0x55a4b7bc7c50 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4b7b4ee20 0x55a4b7b48c90 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a4b7b46610 0x55a4b7b48710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7fc7dc740 1 --2- 172.21.15.18:0/1021676922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4b7b458a0 0x55a4b7b56fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1021676922 >> 172.21.15.18:0/1021676922 conn(0x55a4b7ae05d0 msgr2=0x55a4b7b4ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.283+0000 7fc7dcc00640 0 client.31079 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.284+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1021676922 shutdown_connections 2024-04-16T17:26:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.284+0000 7fc7fc7dc740 1 -- 172.21.15.18:0/1021676922 wait complete. 2024-04-16T17:26:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.284+0000 7fc7fc7dc740 20 client.31079 trim_cache size 0 max 16384 2024-04-16T17:26:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:26:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:26:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:26:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:26:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.285+0000 7fc7f9a00640 1 -- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7d80a3d10 msgr2=0x7fc7d80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.285+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7d80a3d10 0x7fc7d80a61d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0049d0 tx=0x7fc7ec009400 comp rx=0 tx=0).stop 2024-04-16T17:26:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.285+0000 7fc7f9a00640 1 -- 172.21.15.18:0/637494023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f4107350 msgr2=0x7fc7f4074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.285+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f4107350 0x7fc7f4074840 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc7e002c240 tx=0x7fc7e0004430 comp rx=0 tx=0).stop 2024-04-16T17:26:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.286+0000 7fc7f9a00640 1 -- 172.21.15.18:0/637494023 shutdown_connections 2024-04-16T17:26:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.286+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc7d80a3d10 0x7fc7d80a61d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.286+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7f4075fa0 0x7fc7f4074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.286+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f4107350 0x7fc7f4074840 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.286+0000 7fc7f9a00640 1 --2- 172.21.15.18:0/637494023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc7f4102280 0x7fc7f4074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.286+0000 7fc7f9a00640 1 -- 172.21.15.18:0/637494023 >> 172.21.15.18:0/637494023 conn(0x7fc7f40fdac0 msgr2=0x7fc7f40fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.286+0000 7fc7f9a00640 1 -- 172.21.15.18:0/637494023 shutdown_connections 2024-04-16T17:26:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:12.286+0000 7fc7f9a00640 1 -- 172.21.15.18:0/637494023 wait complete. 2024-04-16T17:26:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:12 smithi022 ceph-mon[36023]: pgmap v686: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 463 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 5.2 MiB/s wr, 322 op/s 2024-04-16T17:26:12.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:12 smithi043 ceph-mon[36901]: pgmap v686: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 463 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 5.2 MiB/s wr, 322 op/s 2024-04-16T17:26:12.629 INFO:tasks.workunit.client.0.smithi018.stdout: 10 107087 10.52 MB/sec execute 384 sec latency 439.617 ms 2024-04-16T17:26:13.630 INFO:tasks.workunit.client.0.smithi018.stdout: 10 107329 10.55 MB/sec execute 385 sec latency 494.866 ms 2024-04-16T17:26:14.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:14 smithi022 ceph-mon[36023]: pgmap v687: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 472 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 316 op/s 2024-04-16T17:26:14.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:14 smithi043 ceph-mon[36901]: pgmap v687: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 472 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 316 op/s 2024-04-16T17:26:14.630 INFO:tasks.workunit.client.0.smithi018.stdout: 10 107514 10.56 MB/sec execute 386 sec latency 451.963 ms 2024-04-16T17:26:14.965 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:12 smithi018 ceph-mon[31776]: pgmap v686: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 463 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 5.2 MiB/s wr, 322 op/s 2024-04-16T17:26:15.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:15 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:26:15.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:15 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:26:15.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:15 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:26:15.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:15 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:26:15.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:15 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:26:15.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:15 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:26:15.630 INFO:tasks.workunit.client.0.smithi018.stdout: 10 107680 10.55 MB/sec execute 387 sec latency 336.111 ms 2024-04-16T17:26:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:16 smithi022 ceph-mon[36023]: pgmap v688: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 472 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.6 MiB/s wr, 285 op/s 2024-04-16T17:26:16.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:16 smithi043 ceph-mon[36901]: pgmap v688: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 472 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.6 MiB/s wr, 285 op/s 2024-04-16T17:26:16.631 INFO:tasks.workunit.client.0.smithi018.stdout: 10 107849 10.55 MB/sec execute 388 sec latency 772.691 ms 2024-04-16T17:26:17.228 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:14 smithi018 ceph-mon[31776]: pgmap v687: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 472 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 316 op/s 2024-04-16T17:26:17.228 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:15 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:26:17.228 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:15 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:26:17.228 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:15 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:26:17.228 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:16 smithi018 ceph-mon[31776]: pgmap v688: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 472 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.6 MiB/s wr, 285 op/s 2024-04-16T17:26:17.631 INFO:tasks.workunit.client.0.smithi018.stdout: 10 107941 10.53 MB/sec execute 389 sec latency 883.322 ms 2024-04-16T17:26:18.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:18 smithi022 ceph-mon[36023]: pgmap v689: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 448 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 404 op/s 2024-04-16T17:26:18.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:18 smithi043 ceph-mon[36901]: pgmap v689: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 448 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 404 op/s 2024-04-16T17:26:18.631 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108038 10.51 MB/sec execute 390 sec latency 504.529 ms 2024-04-16T17:26:19.039 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:18 smithi018 ceph-mon[31776]: pgmap v689: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 448 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 404 op/s 2024-04-16T17:26:19.631 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108065 10.49 MB/sec execute 391 sec latency 849.222 ms 2024-04-16T17:26:20.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:20 smithi022 ceph-mon[36023]: pgmap v690: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 448 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 309 op/s 2024-04-16T17:26:20.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:20 smithi043 ceph-mon[36901]: pgmap v690: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 448 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 309 op/s 2024-04-16T17:26:20.631 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108142 10.47 MB/sec execute 392 sec latency 566.187 ms 2024-04-16T17:26:21.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:20 smithi018 ceph-mon[31776]: pgmap v690: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 448 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 309 op/s 2024-04-16T17:26:21.631 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108271 10.46 MB/sec execute 393 sec latency 239.811 ms 2024-04-16T17:26:22.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:22 smithi022 ceph-mon[36023]: pgmap v691: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 441 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-16T17:26:22.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:22 smithi043 ceph-mon[36901]: pgmap v691: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 441 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-16T17:26:22.632 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108308 10.44 MB/sec execute 394 sec latency 620.321 ms 2024-04-16T17:26:22.644 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:22 smithi018 ceph-mon[31776]: pgmap v691: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 441 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-16T17:26:23.632 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108386 10.42 MB/sec execute 395 sec latency 640.589 ms 2024-04-16T17:26:24.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:24 smithi018 ceph-mon[31776]: pgmap v692: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 441 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2024-04-16T17:26:24.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:24 smithi022 ceph-mon[36023]: pgmap v692: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 441 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2024-04-16T17:26:24.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:24 smithi043 ceph-mon[36901]: pgmap v692: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 441 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2024-04-16T17:26:24.633 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108455 10.40 MB/sec execute 396 sec latency 490.002 ms 2024-04-16T17:26:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.582+0000 7ff302000640 1 -- 172.21.15.18:0/3922721750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f40a29b0 msgr2=0x7ff2f40a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.582+0000 7ff302000640 1 --2- 172.21.15.18:0/3922721750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f40a29b0 0x7ff2f40a4da0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009900 tx=0x7ff2e802bc40 comp rx=0 tx=0).stop 2024-04-16T17:26:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.584+0000 7ff302000640 1 -- 172.21.15.18:0/3922721750 shutdown_connections 2024-04-16T17:26:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.584+0000 7ff302000640 1 --2- 172.21.15.18:0/3922721750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f40a29b0 0x7ff2f40a4da0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.584+0000 7ff302000640 1 --2- 172.21.15.18:0/3922721750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f4007f90 0x7ff2f40a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.584+0000 7ff302000640 1 --2- 172.21.15.18:0/3922721750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f4007670 0x7ff2f4007a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.584+0000 7ff302000640 1 -- 172.21.15.18:0/3922721750 >> 172.21.15.18:0/3922721750 conn(0x7ff2f400b830 msgr2=0x7ff2f400bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.584+0000 7ff302000640 1 -- 172.21.15.18:0/3922721750 shutdown_connections 2024-04-16T17:26:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.584+0000 7ff302000640 1 -- 172.21.15.18:0/3922721750 wait complete. 2024-04-16T17:26:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.588+0000 7ff302000640 1 Processor -- start 2024-04-16T17:26:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.588+0000 7ff302000640 1 -- start start 2024-04-16T17:26:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.588+0000 7ff302000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4007670 0x7ff2f4134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.588+0000 7ff302000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f4007f90 0x7ff2f4134940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff302000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f40a29b0 0x7ff2f412e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff302000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f40a82f0 con 0x7ff2f4007f90 2024-04-16T17:26:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff302000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f40a8170 con 0x7ff2f4007670 2024-04-16T17:26:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff302000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f40a8470 con 0x7ff2f40a29b0 2024-04-16T17:26:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff300c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4007670 0x7ff2f4134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff300c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4007670 0x7ff2f4134400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60970/0 (socket says 172.21.15.18:60970) 2024-04-16T17:26:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff300c00640 1 -- 172.21.15.18:0/3748157854 learned_addr learned my addr 172.21.15.18:0/3748157854 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff301600640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f40a29b0 0x7ff2f412e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.589+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f4007f90 0x7ff2f4134940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.590+0000 7ff301600640 1 -- 172.21.15.18:0/3748157854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4007670 msgr2=0x7ff2f4134400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.590+0000 7ff301600640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4007670 0x7ff2f4134400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.590+0000 7ff301600640 1 -- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f4007f90 msgr2=0x7ff2f4134940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.590+0000 7ff301600640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f4007f90 0x7ff2f4134940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.590+0000 7ff301600640 1 -- 172.21.15.18:0/3748157854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f412ed30 con 0x7ff2f40a29b0 2024-04-16T17:26:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.590+0000 7ff301600640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f40a29b0 0x7ff2f412e5b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009230 tx=0x7ff2e80045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.591+0000 7ff2f9600640 1 -- 172.21.15.18:0/3748157854 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8041070 con 0x7ff2f40a29b0 2024-04-16T17:26:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.591+0000 7ff2f9600640 1 -- 172.21.15.18:0/3748157854 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e802cca0 con 0x7ff2f40a29b0 2024-04-16T17:26:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.591+0000 7ff2f9600640 1 -- 172.21.15.18:0/3748157854 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8041070 con 0x7ff2f40a29b0 2024-04-16T17:26:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.591+0000 7ff302000640 1 -- 172.21.15.18:0/3748157854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f412efc0 con 0x7ff2f40a29b0 2024-04-16T17:26:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.591+0000 7ff302000640 1 -- 172.21.15.18:0/3748157854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f412f370 con 0x7ff2f40a29b0 2024-04-16T17:26:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.597+0000 7ff2f9600640 1 -- 172.21.15.18:0/3748157854 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff2e8044070 con 0x7ff2f40a29b0 2024-04-16T17:26:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.599+0000 7ff2f9600640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2e40a3fd0 0x7ff2e40a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.599+0000 7ff2f9600640 1 -- 172.21.15.18:0/3748157854 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7ff2e80f3330 con 0x7ff2f40a29b0 2024-04-16T17:26:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.600+0000 7ff300c00640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2e40a3fd0 0x7ff2e40a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.646 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108585 10.39 MB/sec execute 397 sec latency 259.780 ms 2024-04-16T17:26:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.837+0000 7ff300c00640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2e40a3fd0 0x7ff2e40a6490 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc065170 tx=0x7ff2fc07a040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.838+0000 7ff3063ac740 1 Processor -- start 2024-04-16T17:26:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.838+0000 7ff3063ac740 1 -- start start 2024-04-16T17:26:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.839+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c8a7f0 0x555656c89540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.839+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c89a80 0x555656c89f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.839+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.839+0000 7ff3063ac740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555656bee780 con 0x555656c8a7f0 2024-04-16T17:26:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.839+0000 7ff3063ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555656b94f40 con 0x555656c89a80 2024-04-16T17:26:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.840+0000 7ff2fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c89a80 0x555656c89f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.841+0000 7ff2fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c89a80 0x555656c89f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60974/0 (socket says 172.21.15.18:60974) 2024-04-16T17:26:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.842+0000 7ff2fbe00640 1 -- 172.21.15.18:0/2526426994 learned_addr learned my addr 172.21.15.18:0/2526426994 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.842+0000 7ff3063ac740 1 -- 172.21.15.18:0/2526426994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555656bea740 con 0x555656c93000 2024-04-16T17:26:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.841+0000 7ff301600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.844+0000 7ff2fbe00640 1 -- 172.21.15.18:0/2526426994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 msgr2=0x555656c953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.844+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/2526426994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.844+0000 7ff2fbe00640 1 -- 172.21.15.18:0/2526426994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c8a7f0 msgr2=0x555656c89540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.844+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/2526426994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c8a7f0 0x555656c89540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.844+0000 7ff2fbe00640 1 -- 172.21.15.18:0/2526426994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555656901210 con 0x555656c89a80 2024-04-16T17:26:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.846+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/2526426994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c89a80 0x555656c89f00 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000a8b0 tx=0x7ff2f000ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b3222cdc4851a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.846+0000 7ff2edc00640 1 -- 172.21.15.18:0/2526426994 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0015070 con 0x555656c89a80 2024-04-16T17:26:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.847+0000 7ff2edc00640 1 -- 172.21.15.18:0/2526426994 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f000cb40 con 0x555656c89a80 2024-04-16T17:26:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.847+0000 7ff2edc00640 1 -- 172.21.15.18:0/2526426994 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0015070 con 0x555656c89a80 2024-04-16T17:26:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.848+0000 7ff3063ac740 1 -- 172.21.15.18:0/2526426994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c89a80 msgr2=0x555656c89f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.848+0000 7ff3063ac740 1 --2- 172.21.15.18:0/2526426994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c89a80 0x555656c89f00 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000a8b0 tx=0x7ff2f000ad70 comp rx=0 tx=0).stop 2024-04-16T17:26:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.848+0000 7ff3063ac740 1 -- 172.21.15.18:0/2526426994 shutdown_connections 2024-04-16T17:26:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.848+0000 7ff3063ac740 1 --2- 172.21.15.18:0/2526426994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.848+0000 7ff3063ac740 1 --2- 172.21.15.18:0/2526426994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c89a80 0x555656c89f00 secure :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000a8b0 tx=0x7ff2f000ad70 comp rx=0 tx=0).stop 2024-04-16T17:26:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.848+0000 7ff3063ac740 1 --2- 172.21.15.18:0/2526426994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c8a7f0 0x555656c89540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.848+0000 7ff3063ac740 1 -- 172.21.15.18:0/2526426994 >> 172.21.15.18:0/2526426994 conn(0x555656c245d0 msgr2=0x555656c84770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.850+0000 7ff3063ac740 1 -- 172.21.15.18:0/2526426994 shutdown_connections 2024-04-16T17:26:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.850+0000 7ff3063ac740 1 -- 172.21.15.18:0/2526426994 wait complete. 2024-04-16T17:26:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.851+0000 7ff3063ac740 1 -- start start 2024-04-16T17:26:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.851+0000 7ff3063ac740 1 Processor -- start 2024-04-16T17:26:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.852+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c8a7f0 0x555656c8ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.852+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.852+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cb0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.853+0000 7ff300c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c8a7f0 0x555656c8ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.853+0000 7ff300c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c8a7f0 0x555656c8ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:60984/0 (socket says 172.21.15.18:60984) 2024-04-16T17:26:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.853+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555656ca6d40 con 0x555656c90a10 2024-04-16T17:26:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.853+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555656ba8390 con 0x555656c8a7f0 2024-04-16T17:26:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.853+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555656ca6ec0 con 0x555656c93000 2024-04-16T17:26:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.853+0000 7ff2fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.853+0000 7ff301600640 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cb0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.853+0000 7ff300c00640 1 -- 172.21.15.18:0/3300217036 learned_addr learned my addr 172.21.15.18:0/3300217036 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.854+0000 7ff2fbe00640 1 -- 172.21.15.18:0/3300217036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c8a7f0 msgr2=0x555656c8ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.854+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c8a7f0 0x555656c8ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.854+0000 7ff2fbe00640 1 -- 172.21.15.18:0/3300217036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 msgr2=0x555656cb0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.854+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cb0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.854+0000 7ff3063ac740 10 client.0 _ll_register_callbacks cb 0x555656b91f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.854+0000 7ff2fbe00640 1 -- 172.21.15.18:0/3300217036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556569cd620 con 0x555656c93000 2024-04-16T17:26:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.854+0000 7ff300c00640 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c8a7f0 0x555656c8ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:26:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.854+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f000cb40 con 0x555656c93000 2024-04-16T17:26:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.855+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c904d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x555656c8aec0 tx=0x7ff2f0012ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6297d0c5afb916f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.855+0000 7ff2e2000640 10 client.30931 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.856+0000 7ff2e2000640 1 -- 172.21.15.18:0/3300217036 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0015040 con 0x555656c93000 2024-04-16T17:26:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.858+0000 7ff2e2000640 1 -- 172.21.15.18:0/3300217036 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7ff2f0011600 con 0x555656c93000 2024-04-16T17:26:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.858+0000 7ff2e2000640 1 -- 172.21.15.18:0/3300217036 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff2f0015040 con 0x555656c93000 2024-04-16T17:26:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.858+0000 7ff3063ac740 10 client.30931 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.858+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=37}) -- 0x555656cb32a0 con 0x555656c93000 2024-04-16T17:26:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.859+0000 7ff2e2000640 1 -- 172.21.15.18:0/3300217036 <== mon.1 v2:172.21.15.43:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff2f001a450 con 0x555656c93000 2024-04-16T17:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.860+0000 7ff3063ac740 10 client.30931 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.860+0000 7ff3063ac740 10 client.30931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.860+0000 7ff3063ac740 20 client.30931 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.860+0000 7ff3063ac740 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555656cb48d0 0x555656cb6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.860+0000 7ff3063ac740 4 client.30931 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.860+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555656bfc1a0 con 0x555656cb48d0 2024-04-16T17:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.861+0000 7ff300c00640 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555656cb48d0 0x555656cb6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.861+0000 7ff300c00640 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555656cb48d0 0x555656cb6d90 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba32db4458ea3b7c server_cookie=276287214da19e3c in_seq=0 out_seq=0 2024-04-16T17:26:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.861+0000 7ff2e2000640 10 client.30931 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.863+0000 7ff2e2000640 1 -- 172.21.15.18:0/3300217036 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555656bfc1a0 con 0x555656cb48d0 2024-04-16T17:26:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.864+0000 7ff2e2000640 10 client.30931 handle_command_reply: tid=0 2024-04-16T17:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.864+0000 7ff3063ac740 1 client.30931 shutdown 2024-04-16T17:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.864+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 msgr2=0x555656c904d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.864+0000 7ff3063ac740 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c904d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x555656c8aec0 tx=0x7ff2f0012ff0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 shutdown_connections 2024-04-16T17:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff3063ac740 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555656cb48d0 0x555656cb6d90 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff2e2000640 0 client.30931 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff3063ac740 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cb0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff3063ac740 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c904d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff3063ac740 1 --2- 172.21.15.18:0/3300217036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c8a7f0 0x555656c8ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 >> 172.21.15.18:0/3300217036 conn(0x555656c245d0 msgr2=0x555656c864c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 shutdown_connections 2024-04-16T17:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.865+0000 7ff3063ac740 1 -- 172.21.15.18:0/3300217036 wait complete. 2024-04-16T17:26:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.866+0000 7ff3063ac740 20 client.30931 trim_cache size 0 max 16384 2024-04-16T17:26:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.898+0000 7ff3063ac740 1 Processor -- start 2024-04-16T17:26:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.898+0000 7ff3063ac740 1 -- start start 2024-04-16T17:26:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.899+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c8a7f0 0x555656cbc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.899+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cbcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.899+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c93000 0x555656cafc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.899+0000 7ff3063ac740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc0129f0 con 0x555656c90a10 2024-04-16T17:26:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.899+0000 7ff3063ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc0102c0 con 0x555656c93000 2024-04-16T17:26:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.899+0000 7ff3063ac740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc00e130 con 0x555656c8a7f0 2024-04-16T17:26:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.899+0000 7ff301600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c93000 0x555656cafc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.900+0000 7ff301600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c93000 0x555656cafc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32768/0 (socket says 172.21.15.18:32768) 2024-04-16T17:26:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.900+0000 7ff301600640 1 -- 172.21.15.18:0/572703048 learned_addr learned my addr 172.21.15.18:0/572703048 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.900+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cbcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.900+0000 7ff300c00640 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c8a7f0 0x555656cbc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.900+0000 7ff2fbe00640 1 -- 172.21.15.18:0/572703048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c8a7f0 msgr2=0x555656cbc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.900+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c8a7f0 0x555656cbc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.901+0000 7ff2fbe00640 1 -- 172.21.15.18:0/572703048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c93000 msgr2=0x555656cafc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.901+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c93000 0x555656cafc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.901+0000 7ff2fbe00640 1 -- 172.21.15.18:0/572703048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555656c89830 con 0x555656c90a10 2024-04-16T17:26:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.901+0000 7ff301600640 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c93000 0x555656cafc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.901+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cbcba0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000e010 tx=0x7ff2f001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8abe9893a3556ce server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.902+0000 7ff2ed200640 1 -- 172.21.15.18:0/572703048 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0035020 con 0x555656c90a10 2024-04-16T17:26:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.903+0000 7ff2ed200640 1 -- 172.21.15.18:0/572703048 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f000cb40 con 0x555656c90a10 2024-04-16T17:26:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.903+0000 7ff2ed200640 1 -- 172.21.15.18:0/572703048 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0011860 con 0x555656c90a10 2024-04-16T17:26:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.904+0000 7ff3063ac740 1 -- 172.21.15.18:0/572703048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 msgr2=0x555656cbcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.904+0000 7ff3063ac740 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cbcba0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000e010 tx=0x7ff2f001e000 comp rx=0 tx=0).stop 2024-04-16T17:26:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.905+0000 7ff3063ac740 1 -- 172.21.15.18:0/572703048 shutdown_connections 2024-04-16T17:26:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.905+0000 7ff3063ac740 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c93000 0x555656cafc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.905+0000 7ff3063ac740 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c90a10 0x555656cbcba0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.905+0000 7ff3063ac740 1 --2- 172.21.15.18:0/572703048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c8a7f0 0x555656cbc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.905+0000 7ff3063ac740 1 -- 172.21.15.18:0/572703048 >> 172.21.15.18:0/572703048 conn(0x555656c245d0 msgr2=0x555656c842a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.905+0000 7ff3063ac740 1 -- 172.21.15.18:0/572703048 shutdown_connections 2024-04-16T17:26:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.905+0000 7ff3063ac740 1 -- 172.21.15.18:0/572703048 wait complete. 2024-04-16T17:26:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.906+0000 7ff3063ac740 1 -- start start 2024-04-16T17:26:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.906+0000 7ff3063ac740 1 Processor -- start 2024-04-16T17:26:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.907+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c8a7f0 0x555656cac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.907+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c90a10 0x555656cac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.908+0000 7ff3063ac740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c8d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.908+0000 7ff3063ac740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc00e780 con 0x555656c8a7f0 2024-04-16T17:26:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.908+0000 7ff3063ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc00fbf0 con 0x555656c90a10 2024-04-16T17:26:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.908+0000 7ff3063ac740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc00ee50 con 0x555656c93000 2024-04-16T17:26:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.909+0000 7ff2fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c90a10 0x555656cac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.909+0000 7ff2fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c90a10 0x555656cac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32776/0 (socket says 172.21.15.18:32776) 2024-04-16T17:26:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.909+0000 7ff2fbe00640 1 -- 172.21.15.18:0/1449453693 learned_addr learned my addr 172.21.15.18:0/1449453693 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.909+0000 7ff301600640 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c8d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.909+0000 7ff3063ac740 10 client.0 _ll_register_callbacks cb 0x7ff2e8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.910+0000 7ff301600640 1 -- 172.21.15.18:0/1449453693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c90a10 msgr2=0x555656cac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.910+0000 7ff301600640 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c90a10 0x555656cac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.910+0000 7ff301600640 1 -- 172.21.15.18:0/1449453693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c8a7f0 msgr2=0x555656cac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.910+0000 7ff301600640 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c8a7f0 0x555656cac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.910+0000 7ff301600640 1 -- 172.21.15.18:0/1449453693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555656c8d770 con 0x555656c93000 2024-04-16T17:26:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.910+0000 7ff2fbe00640 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c90a10 0x555656cac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.911+0000 7ff2e2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.911+0000 7ff301600640 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c8d230 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80bb8c0 tx=0x7ff2e8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e3824bef70aca0c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.911+0000 7ff2e2000640 1 -- 172.21.15.18:0/1449453693 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80d7020 con 0x555656c93000 2024-04-16T17:26:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.912+0000 7ff2e2000640 1 -- 172.21.15.18:0/1449453693 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7ff2e80bf1e0 con 0x555656c93000 2024-04-16T17:26:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.912+0000 7ff3063ac740 1 -- 172.21.15.18:0/1449453693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f000cb40 con 0x555656c93000 2024-04-16T17:26:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.912+0000 7ff2e2000640 1 -- 172.21.15.18:0/1449453693 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff2e804b040 con 0x555656c93000 2024-04-16T17:26:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.912+0000 7ff3063ac740 10 client.30937 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.912+0000 7ff3063ac740 1 -- 172.21.15.18:0/1449453693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555656c93c70 con 0x555656c93000 2024-04-16T17:26:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.913+0000 7ff2e2000640 1 -- 172.21.15.18:0/1449453693 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f000cb40 con 0x555656c93000 2024-04-16T17:26:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.917+0000 7ff2e2000640 1 -- 172.21.15.18:0/1449453693 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7ff2e80c7760 con 0x555656c93000 2024-04-16T17:26:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.917+0000 7ff3063ac740 10 client.30937 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.917+0000 7ff3063ac740 10 client.30937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.917+0000 7ff3063ac740 20 client.30937 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.917+0000 7ff3063ac740 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555656d236e0 0x555656d25af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.917+0000 7ff3063ac740 4 client.30937 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:26:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.917+0000 7ff3063ac740 1 -- 172.21.15.18:0/1449453693 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555656bf6010 con 0x555656d236e0 2024-04-16T17:26:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.918+0000 7ff300c00640 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555656d236e0 0x555656d25af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.922+0000 7ff2e2000640 10 client.30937 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:25.922+0000 7ff300c00640 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555656d236e0 0x555656d25af0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b4f6145e3d5313 server_cookie=c5fcd3799b506e3 in_seq=0 out_seq=0 2024-04-16T17:26:26.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:26 smithi022 ceph-mon[36023]: pgmap v693: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 446 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-04-16T17:26:26.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:26 smithi043 ceph-mon[36901]: pgmap v693: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 446 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-04-16T17:26:26.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:26 smithi018 ceph-mon[31776]: pgmap v693: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 446 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-04-16T17:26:26.645 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108624 10.37 MB/sec execute 398 sec latency 610.877 ms 2024-04-16T17:26:27.646 INFO:tasks.workunit.client.0.smithi018.stdout: 10 108837 10.38 MB/sec execute 399 sec latency 467.105 ms 2024-04-16T17:26:28.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:28 smithi022 ceph-mon[36023]: pgmap v694: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 458 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 292 op/s 2024-04-16T17:26:28.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:28 smithi043 ceph-mon[36901]: pgmap v694: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 458 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 292 op/s 2024-04-16T17:26:28.646 INFO:tasks.workunit.client.0.smithi018.stdout: 10 109021 10.37 MB/sec execute 400 sec latency 185.736 ms 2024-04-16T17:26:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.150+0000 7ff2e2000640 1 -- 172.21.15.18:0/1449453693 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555656bf6010 con 0x555656d236e0 2024-04-16T17:26:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.150+0000 7ff2e2000640 10 client.30937 handle_command_reply: tid=0 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.150+0000 7ff3063ac740 1 client.30937 shutdown 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff3063ac740 1 -- 172.21.15.18:0/1449453693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 msgr2=0x555656c8d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff3063ac740 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c8d230 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80bb8c0 tx=0x7ff2e8009260 comp rx=0 tx=0).stop 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff3063ac740 1 -- 172.21.15.18:0/1449453693 shutdown_connections 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff3063ac740 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x555656d236e0 0x555656d25af0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff3063ac740 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555656c93000 0x555656c8d230 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff3063ac740 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555656c90a10 0x555656cac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff3063ac740 1 --2- 172.21.15.18:0/1449453693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555656c8a7f0 0x555656cac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff3063ac740 1 -- 172.21.15.18:0/1449453693 >> 172.21.15.18:0/1449453693 conn(0x555656c245d0 msgr2=0x555656c8e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.151+0000 7ff2e2000640 0 client.30937 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.152+0000 7ff3063ac740 1 -- 172.21.15.18:0/1449453693 shutdown_connections 2024-04-16T17:26:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.152+0000 7ff3063ac740 1 -- 172.21.15.18:0/1449453693 wait complete. 2024-04-16T17:26:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.152+0000 7ff3063ac740 20 client.30937 trim_cache size 0 max 16384 2024-04-16T17:26:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:26:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:26:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:26:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:26:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 -- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2e40a3fd0 msgr2=0x7ff2e40a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2e40a3fd0 0x7ff2e40a6490 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc065170 tx=0x7ff2fc07a040 comp rx=0 tx=0).stop 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 -- 172.21.15.18:0/3748157854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f40a29b0 msgr2=0x7ff2f412e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f40a29b0 0x7ff2f412e5b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009230 tx=0x7ff2e80045e0 comp rx=0 tx=0).stop 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 -- 172.21.15.18:0/3748157854 shutdown_connections 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ff2e40a3fd0 0x7ff2e40a6490 secure :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc065170 tx=0x7ff2fc07a040 comp rx=0 tx=0).stop 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f40a29b0 0x7ff2f412e5b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f4007f90 0x7ff2f4134940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 --2- 172.21.15.18:0/3748157854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4007670 0x7ff2f4134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.167+0000 7ff302000640 1 -- 172.21.15.18:0/3748157854 >> 172.21.15.18:0/3748157854 conn(0x7ff2f400b830 msgr2=0x7ff2f4093220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.170+0000 7ff302000640 1 -- 172.21.15.18:0/3748157854 shutdown_connections 2024-04-16T17:26:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:29.170+0000 7ff302000640 1 -- 172.21.15.18:0/3748157854 wait complete. 2024-04-16T17:26:29.647 INFO:tasks.workunit.client.0.smithi018.stdout: 10 109191 10.36 MB/sec execute 401 sec latency 241.154 ms 2024-04-16T17:26:29.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:26:29.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:26:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.397+0000 7f040a000640 1 -- 172.21.15.18:0/664415380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04041091e0 msgr2=0x7f04041095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.397+0000 7f040a000640 1 --2- 172.21.15.18:0/664415380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04041091e0 0x7f04041095c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009860 tx=0x7f03f402bb90 comp rx=0 tx=0).stop 2024-04-16T17:26:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.397+0000 7f040a000640 1 -- 172.21.15.18:0/664415380 shutdown_connections 2024-04-16T17:26:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.397+0000 7f040a000640 1 --2- 172.21.15.18:0/664415380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0404115150 0x7f0404117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.397+0000 7f040a000640 1 --2- 172.21.15.18:0/664415380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0404109b00 0x7f0404114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.397+0000 7f040a000640 1 --2- 172.21.15.18:0/664415380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04041091e0 0x7f04041095c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.397+0000 7f040a000640 1 -- 172.21.15.18:0/664415380 >> 172.21.15.18:0/664415380 conn(0x7f040406d0a0 msgr2=0x7f040406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:30.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.398+0000 7f040a000640 1 -- 172.21.15.18:0/664415380 shutdown_connections 2024-04-16T17:26:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.398+0000 7f040a000640 1 -- 172.21.15.18:0/664415380 wait complete. 2024-04-16T17:26:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.400+0000 7f040a000640 1 Processor -- start 2024-04-16T17:26:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.402+0000 7f040a000640 1 -- start start 2024-04-16T17:26:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.403+0000 7f040a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04041091e0 0x7f04041145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.403+0000 7f040a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0404109b00 0x7f0404114ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.404+0000 7f040a000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404115150 0x7f040410b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.404+0000 7f040a000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f040411ac20 con 0x7f0404115150 2024-04-16T17:26:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.404+0000 7f040a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f040411aaa0 con 0x7f0404109b00 2024-04-16T17:26:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.404+0000 7f040a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f040411ada0 con 0x7f04041091e0 2024-04-16T17:26:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.404+0000 7f0403400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04041091e0 0x7f04041145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.404+0000 7f0403400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04041091e0 0x7f04041145a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52606/0 (socket says 172.21.15.18:52606) 2024-04-16T17:26:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.405+0000 7f0403400640 1 -- 172.21.15.18:0/2510589334 learned_addr learned my addr 172.21.15.18:0/2510589334 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.405+0000 7f0403e00640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404115150 0x7f040410b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.405+0000 7f0403400640 1 -- 172.21.15.18:0/2510589334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0404109b00 msgr2=0x7f0404114ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.405+0000 7f0403400640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0404109b00 0x7f0404114ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.406+0000 7f0403400640 1 -- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404115150 msgr2=0x7f040410b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.406+0000 7f0403400640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404115150 0x7f040410b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.406+0000 7f0403400640 1 -- 172.21.15.18:0/2510589334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040410bea0 con 0x7f04041091e0 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.407+0000 7f0403400640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04041091e0 0x7f04041145a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f03f40323d0 tx=0x7f03f40344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.408+0000 7f03fbe00640 1 -- 172.21.15.18:0/2510589334 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4041020 con 0x7f04041091e0 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.408+0000 7f040a000640 1 -- 172.21.15.18:0/2510589334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040410c130 con 0x7f04041091e0 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.408+0000 7f040a000640 1 -- 172.21.15.18:0/2510589334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04041101c0 con 0x7f04041091e0 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.410+0000 7f03fbe00640 1 -- 172.21.15.18:0/2510589334 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f40348b0 con 0x7f04041091e0 2024-04-16T17:26:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.410+0000 7f03fbe00640 1 -- 172.21.15.18:0/2510589334 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4041020 con 0x7f04041091e0 2024-04-16T17:26:30.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.414+0000 7f040cbf0740 1 Processor -- start 2024-04-16T17:26:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.415+0000 7f040cbf0740 1 -- start start 2024-04-16T17:26:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.415+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b71b0 0x564a2b3b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.415+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b7ad0 0x564a2b3c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.415+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.415+0000 7f040cbf0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564a2b31d120 con 0x564a2b3b71b0 2024-04-16T17:26:30.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.415+0000 7f040cbf0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564a2b2c3340 con 0x564a2b3b7ad0 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.415+0000 7f040cbf0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564a2b3190e0 con 0x564a2b3c06b0 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.416+0000 7f0402a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.416+0000 7f0402a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52618/0 (socket says 172.21.15.18:52618) 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.416+0000 7f0402a00640 1 -- 172.21.15.18:0/3471481325 learned_addr learned my addr 172.21.15.18:0/3471481325 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.416+0000 7f0402a00640 1 -- 172.21.15.18:0/3471481325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b7ad0 msgr2=0x564a2b3c0170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.416+0000 7f0402a00640 1 --2- 172.21.15.18:0/3471481325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b7ad0 0x564a2b3c0170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.416+0000 7f0402a00640 1 -- 172.21.15.18:0/3471481325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b71b0 msgr2=0x564a2b3b7590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.416+0000 7f0402a00640 1 --2- 172.21.15.18:0/3471481325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b71b0 0x564a2b3b7590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.416+0000 7f0402a00640 1 -- 172.21.15.18:0/3471481325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2b0f97d0 con 0x564a2b3c06b0 2024-04-16T17:26:30.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.417+0000 7f0402a00640 1 --2- 172.21.15.18:0/3471481325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c2aa0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009790 tx=0x7f03f0009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56c5f37e60ff579 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.418+0000 7f03fbe00640 1 -- 172.21.15.18:0/2510589334 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f03f4044020 con 0x7f04041091e0 2024-04-16T17:26:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.418+0000 7f03efe00640 1 -- 172.21.15.18:0/3471481325 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0015020 con 0x564a2b3c06b0 2024-04-16T17:26:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.418+0000 7f03efe00640 1 -- 172.21.15.18:0/3471481325 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0011420 con 0x564a2b3c06b0 2024-04-16T17:26:30.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.418+0000 7f03efe00640 1 -- 172.21.15.18:0/3471481325 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0015020 con 0x564a2b3c06b0 2024-04-16T17:26:30.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.421+0000 7f03fbe00640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f03e80a3fa0 0x7f03e80a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.421+0000 7f0403e00640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f03e80a3fa0 0x7f03e80a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.422+0000 7f03fbe00640 1 -- 172.21.15.18:0/2510589334 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f03f40f20f0 con 0x7f04041091e0 2024-04-16T17:26:30.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.424+0000 7f040cbf0740 1 -- 172.21.15.18:0/3471481325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 msgr2=0x564a2b3c2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.424+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3471481325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c2aa0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009790 tx=0x7f03f0009c50 comp rx=0 tx=0).stop 2024-04-16T17:26:30.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.427+0000 7f0403e00640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f03e80a3fa0 0x7f03e80a6460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f040406b380 tx=0x7f03fc009480 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:30.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.427+0000 7f040cbf0740 1 -- 172.21.15.18:0/3471481325 shutdown_connections 2024-04-16T17:26:30.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.427+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3471481325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c2aa0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.427+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3471481325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b7ad0 0x564a2b3c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.427+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3471481325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b71b0 0x564a2b3b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.427+0000 7f040cbf0740 1 -- 172.21.15.18:0/3471481325 >> 172.21.15.18:0/3471481325 conn(0x564a2b3b18e0 msgr2=0x564a2b3b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:30.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.427+0000 7f040cbf0740 1 -- 172.21.15.18:0/3471481325 shutdown_connections 2024-04-16T17:26:30.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.427+0000 7f040cbf0740 1 -- 172.21.15.18:0/3471481325 wait complete. 2024-04-16T17:26:30.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.428+0000 7f040cbf0740 1 -- start start 2024-04-16T17:26:30.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.428+0000 7f040cbf0740 1 Processor -- start 2024-04-16T17:26:30.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b7ad0 0x564a2b3bdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3be180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f040cbf0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564a2b3c34a0 con 0x564a2b3b7ad0 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f040cbf0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564a2b2d6710 con 0x564a2b3b71b0 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f040cbf0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564a2b3c3620 con 0x564a2b3c06b0 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f0403e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3be180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f0402a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b7ad0 0x564a2b3bdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f0403e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3be180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52630/0 (socket says 172.21.15.18:52630) 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f0403e00640 1 -- 172.21.15.18:0/3204456520 learned_addr learned my addr 172.21.15.18:0/3204456520 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.431+0000 7f0402a00640 1 -- 172.21.15.18:0/3204456520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 msgr2=0x564a2b3be180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.431+0000 7f0402a00640 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3be180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.431+0000 7f0402a00640 1 -- 172.21.15.18:0/3204456520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 msgr2=0x564a2b3bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f040cbf0740 10 client.0 _ll_register_callbacks cb 0x564a2b2d1af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.430+0000 7f0403400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.433+0000 7f0402a00640 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3bd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.433+0000 7f0402a00640 1 -- 172.21.15.18:0/3204456520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2b02f3f0 con 0x564a2b3b7ad0 2024-04-16T17:26:30.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.433+0000 7f040cbf0740 1 -- 172.21.15.18:0/3204456520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f0011420 con 0x564a2b3b7ad0 2024-04-16T17:26:30.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.433+0000 7f0403400640 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:26:30.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.434+0000 7f0402a00640 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b7ad0 0x564a2b3bdc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f03f0016000 tx=0x7f03f0011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e4eadc965d8415 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:30.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.434+0000 7f03ecc00640 10 client.20016 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:26:30.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.435+0000 7f03ecc00640 1 -- 172.21.15.18:0/3204456520 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f000bce0 con 0x564a2b3b7ad0 2024-04-16T17:26:30.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.435+0000 7f03ecc00640 1 -- 172.21.15.18:0/3204456520 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f03f0026480 con 0x564a2b3b7ad0 2024-04-16T17:26:30.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.435+0000 7f03ecc00640 1 -- 172.21.15.18:0/3204456520 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f03f0026720 con 0x564a2b3b7ad0 2024-04-16T17:26:30.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.435+0000 7f040cbf0740 10 client.20016 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:30.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.435+0000 7f040cbf0740 1 -- 172.21.15.18:0/3204456520 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=37}) -- 0x564a2afef0f0 con 0x564a2b3b7ad0 2024-04-16T17:26:30.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.436+0000 7f03ecc00640 1 -- 172.21.15.18:0/3204456520 <== mon.0 v2:172.21.15.18:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f03f00268c0 con 0x564a2b3b7ad0 2024-04-16T17:26:30.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.436+0000 7f040cbf0740 10 client.20016 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:30.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.436+0000 7f040cbf0740 10 client.20016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:30.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.436+0000 7f040cbf0740 20 client.20016 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:30.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.436+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564a2b3d0680 0x564a2b3d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.436+0000 7f040cbf0740 4 client.20016 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:30.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.436+0000 7f040cbf0740 1 -- 172.21.15.18:0/3204456520 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a2b32ab40 con 0x564a2b3d0680 2024-04-16T17:26:30.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.437+0000 7f0403400640 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564a2b3d0680 0x564a2b3d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.437+0000 7f03ecc00640 10 client.20016 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:30.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.437+0000 7f0403400640 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564a2b3d0680 0x564a2b3d2a70 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f98511cd38a0dea server_cookie=d77830084955a977 in_seq=0 out_seq=0 2024-04-16T17:26:30.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f03ecc00640 1 -- 172.21.15.18:0/3204456520 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a2b32ab40 con 0x564a2b3d0680 2024-04-16T17:26:30.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f03ecc00640 10 client.20016 handle_command_reply: tid=0 2024-04-16T17:26:30.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f040cbf0740 1 client.20016 shutdown 2024-04-16T17:26:30.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f040cbf0740 1 -- 172.21.15.18:0/3204456520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b7ad0 msgr2=0x564a2b3bdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b7ad0 0x564a2b3bdc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f03f0016000 tx=0x7f03f0011800 comp rx=0 tx=0).stop 2024-04-16T17:26:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f040cbf0740 1 -- 172.21.15.18:0/3204456520 shutdown_connections 2024-04-16T17:26:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564a2b3d0680 0x564a2b3d2a70 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3be180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b7ad0 0x564a2b3bdc40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f03ecc00640 0 client.20016 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.440+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3204456520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.441+0000 7f040cbf0740 1 -- 172.21.15.18:0/3204456520 >> 172.21.15.18:0/3204456520 conn(0x564a2b3b18e0 msgr2=0x564a2b3b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:30.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.441+0000 7f040cbf0740 1 -- 172.21.15.18:0/3204456520 shutdown_connections 2024-04-16T17:26:30.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.441+0000 7f040cbf0740 1 -- 172.21.15.18:0/3204456520 wait complete. 2024-04-16T17:26:30.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.442+0000 7f040cbf0740 20 client.20016 trim_cache size 0 max 16384 2024-04-16T17:26:30.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:30 smithi043 ceph-mon[36901]: pgmap v695: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 458 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-04-16T17:26:30.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.508+0000 7f040cbf0740 1 Processor -- start 2024-04-16T17:26:30.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.510+0000 7f040cbf0740 1 -- start start 2024-04-16T17:26:30.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b71b0 0x564a2b3bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3c06b0 0x564a2b3cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3b3420 0x564a2b3d8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f040cbf0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f03e4012a60 con 0x564a2b3b71b0 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f040cbf0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f03e4010330 con 0x564a2b3c06b0 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f040cbf0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03e400e1a0 con 0x564a2b3b3420 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f0403e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3b3420 0x564a2b3d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f0403e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3b3420 0x564a2b3d8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52640/0 (socket says 172.21.15.18:52640) 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f0403e00640 1 -- 172.21.15.18:0/2597880460 learned_addr learned my addr 172.21.15.18:0/2597880460 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.511+0000 7f0402a00640 1 --2- 172.21.15.18:0/2597880460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3c06b0 0x564a2b3cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.512+0000 7f0402a00640 1 -- 172.21.15.18:0/2597880460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3b3420 msgr2=0x564a2b3d8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:30.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.512+0000 7f0402a00640 1 --2- 172.21.15.18:0/2597880460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3b3420 0x564a2b3d8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.512+0000 7f0402a00640 1 -- 172.21.15.18:0/2597880460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b71b0 msgr2=0x564a2b3bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:30.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.513+0000 7f0402a00640 1 --2- 172.21.15.18:0/2597880460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b71b0 0x564a2b3bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.513+0000 7f0402a00640 1 -- 172.21.15.18:0/2597880460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2b3c2dd0 con 0x564a2b3c06b0 2024-04-16T17:26:30.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.514+0000 7f0402a00640 1 --2- 172.21.15.18:0/2597880460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3c06b0 0x564a2b3cc3f0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x564a2b3be7c0 tx=0x7f03f001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a99a68650a15351 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:30.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.515+0000 7f03ef400640 1 -- 172.21.15.18:0/2597880460 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0015040 con 0x564a2b3c06b0 2024-04-16T17:26:30.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.515+0000 7f03ef400640 1 -- 172.21.15.18:0/2597880460 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0011420 con 0x564a2b3c06b0 2024-04-16T17:26:30.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.515+0000 7f03ef400640 1 -- 172.21.15.18:0/2597880460 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f001b430 con 0x564a2b3c06b0 2024-04-16T17:26:30.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.516+0000 7f040cbf0740 1 -- 172.21.15.18:0/2597880460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3c06b0 msgr2=0x564a2b3cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.516+0000 7f040cbf0740 1 --2- 172.21.15.18:0/2597880460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3c06b0 0x564a2b3cc3f0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x564a2b3be7c0 tx=0x7f03f001d000 comp rx=0 tx=0).stop 2024-04-16T17:26:30.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.517+0000 7f040cbf0740 1 -- 172.21.15.18:0/2597880460 shutdown_connections 2024-04-16T17:26:30.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.517+0000 7f040cbf0740 1 --2- 172.21.15.18:0/2597880460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3b3420 0x564a2b3d8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.517+0000 7f040cbf0740 1 --2- 172.21.15.18:0/2597880460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3c06b0 0x564a2b3cc3f0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.517+0000 7f040cbf0740 1 --2- 172.21.15.18:0/2597880460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b71b0 0x564a2b3bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.517+0000 7f040cbf0740 1 -- 172.21.15.18:0/2597880460 >> 172.21.15.18:0/2597880460 conn(0x564a2b3b18e0 msgr2=0x564a2b3b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:30.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.517+0000 7f040cbf0740 1 -- 172.21.15.18:0/2597880460 shutdown_connections 2024-04-16T17:26:30.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.517+0000 7f040cbf0740 1 -- 172.21.15.18:0/2597880460 wait complete. 2024-04-16T17:26:30.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.518+0000 7f040cbf0740 1 -- start start 2024-04-16T17:26:30.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.518+0000 7f040cbf0740 1 Processor -- start 2024-04-16T17:26:30.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.519+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b3420 0x564a2b3c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.519+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.519+0000 7f040cbf0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.519+0000 7f040cbf0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f03e400e7f0 con 0x564a2b3b3420 2024-04-16T17:26:30.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.519+0000 7f040cbf0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f03e400fc60 con 0x564a2b3b71b0 2024-04-16T17:26:30.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.519+0000 7f040cbf0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03e400eec0 con 0x564a2b3c06b0 2024-04-16T17:26:30.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.519+0000 7f040cbf0740 10 client.0 _ll_register_callbacks cb 0x7f03f0011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:30.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.522+0000 7f0402a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.522+0000 7f0402a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3c8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55698/0 (socket says 172.21.15.18:55698) 2024-04-16T17:26:30.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.522+0000 7f0402a00640 1 -- 172.21.15.18:0/3727432273 learned_addr learned my addr 172.21.15.18:0/3727432273 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:30.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.522+0000 7f0403e00640 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.522+0000 7f0403400640 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b3420 0x564a2b3c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.523+0000 7f0403e00640 1 -- 172.21.15.18:0/3727432273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 msgr2=0x564a2b3c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:30.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.523+0000 7f0403e00640 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.523+0000 7f0403e00640 1 -- 172.21.15.18:0/3727432273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b3420 msgr2=0x564a2b3c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:30.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.523+0000 7f0403e00640 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b3420 0x564a2b3c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.523+0000 7f0403e00640 1 -- 172.21.15.18:0/3727432273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2b3c5e40 con 0x564a2b3c06b0 2024-04-16T17:26:30.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.524+0000 7f0402a00640 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:30.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.524+0000 7f0403e00640 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c58d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x564a2b3b8450 tx=0x7f03fc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa50b49da7ef329 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:30.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.527+0000 7f03ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:30.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.527+0000 7f03ecc00640 1 -- 172.21.15.18:0/3727432273 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0255f0 con 0x564a2b3c06b0 2024-04-16T17:26:30.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.528+0000 7f03ecc00640 1 -- 172.21.15.18:0/3727432273 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f03fc025790 con 0x564a2b3c06b0 2024-04-16T17:26:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.528+0000 7f040cbf0740 1 -- 172.21.15.18:0/3727432273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f0011420 con 0x564a2b3c06b0 2024-04-16T17:26:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.528+0000 7f03ecc00640 1 -- 172.21.15.18:0/3727432273 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f03fc025990 con 0x564a2b3c06b0 2024-04-16T17:26:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.529+0000 7f040cbf0740 10 client.30952 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:30.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.529+0000 7f040cbf0740 1 -- 172.21.15.18:0/3727432273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564a2b4372f0 con 0x564a2b3c06b0 2024-04-16T17:26:30.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.529+0000 7f03ecc00640 1 -- 172.21.15.18:0/3727432273 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0011420 con 0x564a2b3c06b0 2024-04-16T17:26:30.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.530+0000 7f03ecc00640 1 -- 172.21.15.18:0/3727432273 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f03fc017040 con 0x564a2b3c06b0 2024-04-16T17:26:30.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.530+0000 7f040cbf0740 10 client.30952 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:30.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.530+0000 7f040cbf0740 10 client.30952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:30.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.530+0000 7f040cbf0740 20 client.30952 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:30.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.530+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564a2b437ac0 0x564a2b439ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:30.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.530+0000 7f040cbf0740 4 client.30952 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:26:30.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.530+0000 7f040cbf0740 1 -- 172.21.15.18:0/3727432273 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a2b3249b0 con 0x564a2b437ac0 2024-04-16T17:26:30.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.530+0000 7f0403400640 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564a2b437ac0 0x564a2b439ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:30.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.531+0000 7f03ecc00640 10 client.30952 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:30.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.535+0000 7f0403400640 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564a2b437ac0 0x564a2b439ee0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1c68e5ffe5ba99 server_cookie=f19fa241082ff3c5 in_seq=0 out_seq=0 2024-04-16T17:26:30.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.538+0000 7f03ecc00640 1 -- 172.21.15.18:0/3727432273 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564a2b3249b0 con 0x564a2b437ac0 2024-04-16T17:26:30.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.538+0000 7f03ecc00640 10 client.30952 handle_command_reply: tid=0 2024-04-16T17:26:30.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.538+0000 7f040cbf0740 1 client.30952 shutdown 2024-04-16T17:26:30.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f040cbf0740 1 -- 172.21.15.18:0/3727432273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 msgr2=0x564a2b3c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c58d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x564a2b3b8450 tx=0x7f03fc004ab0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f040cbf0740 1 -- 172.21.15.18:0/3727432273 shutdown_connections 2024-04-16T17:26:30.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x564a2b437ac0 0x564a2b439ee0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564a2b3c06b0 0x564a2b3c58d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a2b3b71b0 0x564a2b3c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f040cbf0740 1 --2- 172.21.15.18:0/3727432273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564a2b3b3420 0x564a2b3c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f040cbf0740 1 -- 172.21.15.18:0/3727432273 >> 172.21.15.18:0/3727432273 conn(0x564a2b3b18e0 msgr2=0x564a2b3bd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:30.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.540+0000 7f03ecc00640 0 client.30952 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:30.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.541+0000 7f040cbf0740 1 -- 172.21.15.18:0/3727432273 shutdown_connections 2024-04-16T17:26:30.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.541+0000 7f040cbf0740 1 -- 172.21.15.18:0/3727432273 wait complete. 2024-04-16T17:26:30.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.542+0000 7f040cbf0740 20 client.30952 trim_cache size 0 max 16384 2024-04-16T17:26:30.545 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:26:30.545 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:26:30.545 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:26:30.545 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:26:30.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.546+0000 7f040a000640 1 -- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f03e80a3fa0 msgr2=0x7f03e80a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.546+0000 7f040a000640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f03e80a3fa0 0x7f03e80a6460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f040406b380 tx=0x7f03fc009480 comp rx=0 tx=0).stop 2024-04-16T17:26:30.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.546+0000 7f040a000640 1 -- 172.21.15.18:0/2510589334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04041091e0 msgr2=0x7f04041145a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:30.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.546+0000 7f040a000640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04041091e0 0x7f04041145a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f03f40323d0 tx=0x7f03f40344c0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.547+0000 7f040a000640 1 -- 172.21.15.18:0/2510589334 shutdown_connections 2024-04-16T17:26:30.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.547+0000 7f040a000640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f03e80a3fa0 0x7f03e80a6460 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.547+0000 7f040a000640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0404115150 0x7f040410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.547+0000 7f040a000640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0404109b00 0x7f0404114ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.547+0000 7f040a000640 1 --2- 172.21.15.18:0/2510589334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04041091e0 0x7f04041145a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:30.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.547+0000 7f040a000640 1 -- 172.21.15.18:0/2510589334 >> 172.21.15.18:0/2510589334 conn(0x7f040406d0a0 msgr2=0x7f0404071f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:30.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.548+0000 7f040a000640 1 -- 172.21.15.18:0/2510589334 shutdown_connections 2024-04-16T17:26:30.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:30.548+0000 7f040a000640 1 -- 172.21.15.18:0/2510589334 wait complete. 2024-04-16T17:26:30.585 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:26:30.586 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:26:30.646 INFO:tasks.workunit.client.0.smithi018.stdout: 10 109342 10.36 MB/sec execute 402 sec latency 406.218 ms 2024-04-16T17:26:30.667 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:28 smithi018 ceph-mon[31776]: pgmap v694: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 458 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 292 op/s 2024-04-16T17:26:30.667 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:30 smithi018 ceph-mon[31776]: pgmap v695: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 458 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-04-16T17:26:30.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:30 smithi022 ceph-mon[36023]: pgmap v695: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 458 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-04-16T17:26:31.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.392+0000 7f35ca400640 1 -- 172.21.15.18:0/3122501671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c410a9d0 msgr2=0x7f35c41165b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:31.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.392+0000 7f35ca400640 1 --2- 172.21.15.18:0/3122501671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c410a9d0 0x7f35c41165b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009c10 tx=0x7f35bc02bef0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.392+0000 7f35ca400640 1 -- 172.21.15.18:0/3122501671 shutdown_connections 2024-04-16T17:26:31.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.392+0000 7f35ca400640 1 --2- 172.21.15.18:0/3122501671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c410a9d0 0x7f35c41165b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.392+0000 7f35ca400640 1 --2- 172.21.15.18:0/3122501671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c410a010 0x7f35c410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.392+0000 7f35ca400640 1 --2- 172.21.15.18:0/3122501671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35c410e720 0x7f35c410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.392+0000 7f35ca400640 1 -- 172.21.15.18:0/3122501671 >> 172.21.15.18:0/3122501671 conn(0x7f35c406e1d0 msgr2=0x7f35c406e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:31.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.393+0000 7f35ca400640 1 -- 172.21.15.18:0/3122501671 shutdown_connections 2024-04-16T17:26:31.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.393+0000 7f35ca400640 1 -- 172.21.15.18:0/3122501671 wait complete. 2024-04-16T17:26:31.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.394+0000 7f35ca400640 1 Processor -- start 2024-04-16T17:26:31.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.394+0000 7f35ca400640 1 -- start start 2024-04-16T17:26:31.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.397+0000 7f35ca400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35c410a010 0x7f35c410b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:31.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.397+0000 7f35ca400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c410a9d0 0x7f35c410b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:31.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.397+0000 7f35ca400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c410e720 0x7f35c410be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:31.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.397+0000 7f35ca400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4119c30 con 0x7f35c410e720 2024-04-16T17:26:31.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.397+0000 7f35ca400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4119ab0 con 0x7f35c410a010 2024-04-16T17:26:31.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.397+0000 7f35ca400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4119db0 con 0x7f35c410a9d0 2024-04-16T17:26:31.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35c410a010 0x7f35c410b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:31.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35c410a010 0x7f35c410b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55716/0 (socket says 172.21.15.18:55716) 2024-04-16T17:26:31.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3400640 1 -- 172.21.15.18:0/1892464362 learned_addr learned my addr 172.21.15.18:0/1892464362 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3e00640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c410e720 0x7f35c410be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3400640 1 -- 172.21.15.18:0/1892464362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c410a9d0 msgr2=0x7f35c410b950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c410a9d0 0x7f35c410b950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3400640 1 -- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c410e720 msgr2=0x7f35c410be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c410e720 0x7f35c410be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.398+0000 7f35c3400640 1 -- 172.21.15.18:0/1892464362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c41af630 con 0x7f35c410a010 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.399+0000 7f35c3400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35c410a010 0x7f35c410b410 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00ea60 tx=0x7f35ac00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.399+0000 7f35c0c00640 1 -- 172.21.15.18:0/1892464362 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00cde0 con 0x7f35c410a010 2024-04-16T17:26:31.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.399+0000 7f35c0c00640 1 -- 172.21.15.18:0/1892464362 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35ac0035f0 con 0x7f35c410a010 2024-04-16T17:26:31.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.399+0000 7f35c0c00640 1 -- 172.21.15.18:0/1892464362 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac010720 con 0x7f35c410a010 2024-04-16T17:26:31.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.399+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c41af920 con 0x7f35c410a010 2024-04-16T17:26:31.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.399+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c41afe20 con 0x7f35c410a010 2024-04-16T17:26:31.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.402+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c4068eb0 con 0x7f35c410a010 2024-04-16T17:26:31.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.407+0000 7f35c0c00640 1 -- 172.21.15.18:0/1892464362 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f35ac004020 con 0x7f35c410a010 2024-04-16T17:26:31.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.409+0000 7f35c0c00640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35a80a3da0 0x7f35a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:31.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.410+0000 7f35bbe00640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35a80a3da0 0x7f35a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:31.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.410+0000 7f35c0c00640 1 -- 172.21.15.18:0/1892464362 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f35ac0d10a0 con 0x7f35c410a010 2024-04-16T17:26:31.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.411+0000 7f35bbe00640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35a80a3da0 0x7f35a80a6260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f35b40058c0 tx=0x7f35b4005850 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.415+0000 7f35c0c00640 1 -- 172.21.15.18:0/1892464362 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35ac09d280 con 0x7f35c410a010 2024-04-16T17:26:31.648 INFO:tasks.workunit.client.0.smithi018.stdout: 10 109571 10.35 MB/sec execute 403 sec latency 233.846 ms 2024-04-16T17:26:31.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.844+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35c4112ba0 con 0x7f35c410a010 2024-04-16T17:26:31.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.847+0000 7f35c0c00640 1 -- 172.21.15.18:0/1892464362 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7f35ac09cb80 con 0x7f35c410a010 2024-04-16T17:26:31.847 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:26:31.847 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:26:31.847 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:26:31.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.852+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35a80a3da0 msgr2=0x7f35a80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:31.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.852+0000 7f35ca400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35a80a3da0 0x7f35a80a6260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f35b40058c0 tx=0x7f35b4005850 comp rx=0 tx=0).stop 2024-04-16T17:26:31.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.852+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35c410a010 msgr2=0x7f35c410b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:31.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.852+0000 7f35ca400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35c410a010 0x7f35c410b410 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00ea60 tx=0x7f35ac00ef20 comp rx=0 tx=0).stop 2024-04-16T17:26:31.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.853+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 shutdown_connections 2024-04-16T17:26:31.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.853+0000 7f35ca400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f35a80a3da0 0x7f35a80a6260 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.853+0000 7f35ca400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35c410e720 0x7f35c410be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.853+0000 7f35ca400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c410a9d0 0x7f35c410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:31.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.853+0000 7f35ca400640 1 --2- 172.21.15.18:0/1892464362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35c410a010 0x7f35c410b410 secure :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00ea60 tx=0x7f35ac00ef20 comp rx=0 tx=0).stop 2024-04-16T17:26:31.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.853+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 >> 172.21.15.18:0/1892464362 conn(0x7f35c406e1d0 msgr2=0x7f35c4072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:31.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.853+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 shutdown_connections 2024-04-16T17:26:31.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:31.853+0000 7f35ca400640 1 -- 172.21.15.18:0/1892464362 wait complete. 2024-04-16T17:26:31.885 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:26:32.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.249+0000 7f39d5600640 1 -- 172.21.15.18:0/3656544595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d01090c0 msgr2=0x7f39d0115570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.249+0000 7f39d5600640 1 --2- 172.21.15.18:0/3656544595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d01090c0 0x7f39d0115570 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009ca0 tx=0x7f39c802c1b0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.249+0000 7f39d5600640 1 -- 172.21.15.18:0/3656544595 shutdown_connections 2024-04-16T17:26:32.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.249+0000 7f39d5600640 1 --2- 172.21.15.18:0/3656544595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d01090c0 0x7f39d0115570 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.249+0000 7f39d5600640 1 --2- 172.21.15.18:0/3656544595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0108720 0x7f39d0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.249+0000 7f39d5600640 1 --2- 172.21.15.18:0/3656544595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d010a590 0x7f39d010a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.249+0000 7f39d5600640 1 -- 172.21.15.18:0/3656544595 >> 172.21.15.18:0/3656544595 conn(0x7f39d0077e70 msgr2=0x7f39d0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.252+0000 7f39d5600640 1 -- 172.21.15.18:0/3656544595 shutdown_connections 2024-04-16T17:26:32.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.252+0000 7f39d5600640 1 -- 172.21.15.18:0/3656544595 wait complete. 2024-04-16T17:26:32.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.253+0000 7f39d5600640 1 Processor -- start 2024-04-16T17:26:32.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.254+0000 7f39d5600640 1 -- start start 2024-04-16T17:26:32.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.255+0000 7f39d5600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0108720 0x7f39d00825e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.256+0000 7f39d5600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d01090c0 0x7f39d0082b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.256+0000 7f39d5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d010a590 0x7f39d007b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.256+0000 7f39d5600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0118b10 con 0x7f39d0108720 2024-04-16T17:26:32.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.256+0000 7f39d5600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0118990 con 0x7f39d01090c0 2024-04-16T17:26:32.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.256+0000 7f39d5600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0118c90 con 0x7f39d010a590 2024-04-16T17:26:32.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.256+0000 7f39cf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d010a590 0x7f39d007b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.256+0000 7f39cf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d010a590 0x7f39d007b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52678/0 (socket says 172.21.15.18:52678) 2024-04-16T17:26:32.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.256+0000 7f39cf400640 1 -- 172.21.15.18:0/1580112306 learned_addr learned my addr 172.21.15.18:0/1580112306 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.257+0000 7f39cea00640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0108720 0x7f39d00825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.257+0000 7f39c7e00640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d01090c0 0x7f39d0082b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.257+0000 7f39cf400640 1 -- 172.21.15.18:0/1580112306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d01090c0 msgr2=0x7f39d0082b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.257+0000 7f39cf400640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d01090c0 0x7f39d0082b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.257+0000 7f39cf400640 1 -- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0108720 msgr2=0x7f39d00825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.257+0000 7f39cf400640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0108720 0x7f39d00825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.257+0000 7f39cf400640 1 -- 172.21.15.18:0/1580112306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d007c100 con 0x7f39d010a590 2024-04-16T17:26:32.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.259+0000 7f39cea00640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0108720 0x7f39d00825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:26:32.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.260+0000 7f39cf400640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d010a590 0x7f39d007b840 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f39c80348f0 tx=0x7f39c8003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.260+0000 7f39c7400640 1 -- 172.21.15.18:0/1580112306 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8039040 con 0x7f39d010a590 2024-04-16T17:26:32.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.260+0000 7f39c7400640 1 -- 172.21.15.18:0/1580112306 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c8044070 con 0x7f39d010a590 2024-04-16T17:26:32.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.260+0000 7f39c7400640 1 -- 172.21.15.18:0/1580112306 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80329c0 con 0x7f39d010a590 2024-04-16T17:26:32.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.260+0000 7f39d5600640 1 -- 172.21.15.18:0/1580112306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d007c390 con 0x7f39d010a590 2024-04-16T17:26:32.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.261+0000 7f39d5600640 1 -- 172.21.15.18:0/1580112306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d00802e0 con 0x7f39d010a590 2024-04-16T17:26:32.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.273+0000 7f39d8244740 1 Processor -- start 2024-04-16T17:26:32.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.274+0000 7f39d8244740 1 -- start start 2024-04-16T17:26:32.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39d8244740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e3a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39d8244740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 0x564f80e42d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39d8244740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e45640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39d8244740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564f80d9f020 con 0x564f80e3a670 2024-04-16T17:26:32.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39d8244740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564f80d45460 con 0x564f80e43250 2024-04-16T17:26:32.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39d8244740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f80d9afe0 con 0x564f80e39d50 2024-04-16T17:26:32.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e3a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e3a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52694/0 (socket says 172.21.15.18:52694) 2024-04-16T17:26:32.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39c7e00640 1 -- 172.21.15.18:0/3483521095 learned_addr learned my addr 172.21.15.18:0/3483521095 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.275+0000 7f39c7e00640 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e45640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.276+0000 7f39c7400640 1 -- 172.21.15.18:0/1580112306 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f39c8032b60 con 0x7f39d010a590 2024-04-16T17:26:32.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.276+0000 7f39c7e00640 1 -- 172.21.15.18:0/3483521095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 msgr2=0x564f80e45640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.276+0000 7f39c7e00640 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e45640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.276+0000 7f39c7e00640 1 -- 172.21.15.18:0/3483521095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 msgr2=0x564f80e42d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:26:32.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.279+0000 7f39cea00640 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 0x564f80e42d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.280+0000 7f39c7e00640 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 0x564f80e42d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.280+0000 7f39c7e00640 1 -- 172.21.15.18:0/3483521095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f80d4d040 con 0x564f80e39d50 2024-04-16T17:26:32.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.280+0000 7f39c7e00640 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e3a130 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f39b80176a0 tx=0x7f39b8009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=141b8109e2fbc12b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.283+0000 7f39c7400640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39b40a3e60 0x7f39b40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.283+0000 7f39bf400640 1 -- 172.21.15.18:0/3483521095 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8013780 con 0x564f80e39d50 2024-04-16T17:26:32.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.283+0000 7f39bf400640 1 -- 172.21.15.18:0/3483521095 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b801d070 con 0x564f80e39d50 2024-04-16T17:26:32.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.283+0000 7f39bf400640 1 -- 172.21.15.18:0/3483521095 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8018cf0 con 0x564f80e39d50 2024-04-16T17:26:32.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.284+0000 7f39cea00640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39b40a3e60 0x7f39b40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.285+0000 7f39c7400640 1 -- 172.21.15.18:0/1580112306 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f39c80f3280 con 0x7f39d010a590 2024-04-16T17:26:32.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.285+0000 7f39d8244740 1 -- 172.21.15.18:0/3483521095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 msgr2=0x564f80e3a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.285+0000 7f39d8244740 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e3a130 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f39b80176a0 tx=0x7f39b8009460 comp rx=0 tx=0).stop 2024-04-16T17:26:32.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.288+0000 7f39d8244740 1 -- 172.21.15.18:0/3483521095 shutdown_connections 2024-04-16T17:26:32.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.288+0000 7f39d8244740 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e45640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.288+0000 7f39d8244740 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 0x564f80e42d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.288+0000 7f39d8244740 1 --2- 172.21.15.18:0/3483521095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e3a130 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.288+0000 7f39d8244740 1 -- 172.21.15.18:0/3483521095 >> 172.21.15.18:0/3483521095 conn(0x564f80e34480 msgr2=0x564f80e34860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.289+0000 7f39cea00640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39b40a3e60 0x7f39b40a6320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f39c0005fd0 tx=0x7f39c0005ec0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.289+0000 7f39d8244740 1 -- 172.21.15.18:0/3483521095 shutdown_connections 2024-04-16T17:26:32.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.289+0000 7f39d8244740 1 -- 172.21.15.18:0/3483521095 wait complete. 2024-04-16T17:26:32.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.289+0000 7f39d8244740 1 -- start start 2024-04-16T17:26:32.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.290+0000 7f39d8244740 1 Processor -- start 2024-04-16T17:26:32.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.291+0000 7f39d8244740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.291+0000 7f39d8244740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 0x564f80e407f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.292+0000 7f39cea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.292+0000 7f39cea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e402b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52702/0 (socket says 172.21.15.18:52702) 2024-04-16T17:26:32.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.292+0000 7f39cea00640 1 -- 172.21.15.18:0/105251204 learned_addr learned my addr 172.21.15.18:0/105251204 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.292+0000 7f39d8244740 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e40d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.292+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564f80e45fe0 con 0x564f80e3a670 2024-04-16T17:26:32.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.292+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564f80d58830 con 0x564f80e43250 2024-04-16T17:26:32.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.292+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f80e46160 con 0x564f80e39d50 2024-04-16T17:26:32.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.292+0000 7f39d8244740 10 client.0 _ll_register_callbacks cb 0x564f80d60150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:32.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.293+0000 7f39cf400640 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e40d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.293+0000 7f39cf400640 1 -- 172.21.15.18:0/105251204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 msgr2=0x564f80e402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.293+0000 7f39cf400640 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.293+0000 7f39cf400640 1 -- 172.21.15.18:0/105251204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 msgr2=0x564f80e407f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.293+0000 7f39cf400640 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 0x564f80e407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.293+0000 7f39cf400640 1 -- 172.21.15.18:0/105251204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f80d5c920 con 0x564f80e43250 2024-04-16T17:26:32.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.297+0000 7f39cf400640 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e40d30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f39c80f05f0 tx=0x7f39c80c7920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bfe472cb5edfef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.297+0000 7f39b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:32.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.297+0000 7f39b3e00640 1 -- 172.21.15.18:0/105251204 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80c72e0 con 0x564f80e43250 2024-04-16T17:26:32.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.297+0000 7f39b3e00640 1 -- 172.21.15.18:0/105251204 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f39c8004b40 con 0x564f80e43250 2024-04-16T17:26:32.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.298+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39b801d070 con 0x564f80e43250 2024-04-16T17:26:32.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.298+0000 7f39b3e00640 1 -- 172.21.15.18:0/105251204 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f39c80be2e0 con 0x564f80e43250 2024-04-16T17:26:32.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.301+0000 7f39d8244740 10 client.31121 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:32.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.301+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564f80d2a7f0 con 0x564f80e43250 2024-04-16T17:26:32.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.301+0000 7f39b3e00640 1 -- 172.21.15.18:0/105251204 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b801d070 con 0x564f80e43250 2024-04-16T17:26:32.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.302+0000 7f39b3e00640 1 -- 172.21.15.18:0/105251204 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f39c803c030 con 0x564f80e43250 2024-04-16T17:26:32.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.302+0000 7f39d8244740 10 client.31121 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:32.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.302+0000 7f39d8244740 10 client.31121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:26:32.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.302+0000 7f39d8244740 20 client.31121 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:32.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.302+0000 7f39d8244740 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x564f80e53420 0x564f80e558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.302+0000 7f39d8244740 4 client.31121 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:32.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.302+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f80daca40 con 0x564f80e53420 2024-04-16T17:26:32.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.303+0000 7f39cea00640 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x564f80e53420 0x564f80e558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.303+0000 7f39b3e00640 10 client.31121 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:26:32.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.304+0000 7f39cea00640 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x564f80e53420 0x564f80e558c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63dec0b8db03cd84 server_cookie=ef0a417c587d6ad2 in_seq=0 out_seq=0 2024-04-16T17:26:32.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.305+0000 7f39b3e00640 1 -- 172.21.15.18:0/105251204 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f80daca40 con 0x564f80e53420 2024-04-16T17:26:32.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.305+0000 7f39b3e00640 10 client.31121 handle_command_reply: tid=0 2024-04-16T17:26:32.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.306+0000 7f39d8244740 1 client.31121 shutdown 2024-04-16T17:26:32.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.306+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 msgr2=0x564f80e40d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.306+0000 7f39d8244740 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e40d30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f39c80f05f0 tx=0x7f39c80c7920 comp rx=0 tx=0).stop 2024-04-16T17:26:32.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.307+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 shutdown_connections 2024-04-16T17:26:32.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.307+0000 7f39d8244740 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x564f80e53420 0x564f80e558c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.307+0000 7f39d8244740 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e43250 0x564f80e40d30 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.307+0000 7f39d8244740 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e3a670 0x564f80e407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.307+0000 7f39d8244740 1 --2- 172.21.15.18:0/105251204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.307+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 >> 172.21.15.18:0/105251204 conn(0x564f80e34480 msgr2=0x564f80e361d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.307+0000 7f39b3e00640 0 client.31121 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:26:32.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.307+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 shutdown_connections 2024-04-16T17:26:32.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.308+0000 7f39d8244740 1 -- 172.21.15.18:0/105251204 wait complete. 2024-04-16T17:26:32.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.308+0000 7f39d8244740 20 client.31121 trim_cache size 0 max 16384 2024-04-16T17:26:32.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.339+0000 7f39d8244740 1 Processor -- start 2024-04-16T17:26:32.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.339+0000 7f39d8244740 1 -- start start 2024-04-16T17:26:32.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.340+0000 7f39d8244740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.340+0000 7f39d8244740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e3a670 0x564f80e4eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.340+0000 7f39d8244740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 0x564f80e3f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.340+0000 7f39d8244740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01c0f40 con 0x564f80e43250 2024-04-16T17:26:32.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.340+0000 7f39d8244740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01be810 con 0x564f80e3a670 2024-04-16T17:26:32.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.340+0000 7f39d8244740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01bc680 con 0x564f80e39d50 2024-04-16T17:26:32.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e3a670 0x564f80e4eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e3a670 0x564f80e4eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55748/0 (socket says 172.21.15.18:55748) 2024-04-16T17:26:32.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39c7e00640 1 -- 172.21.15.18:0/2382391036 learned_addr learned my addr 172.21.15.18:0/2382391036 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39cea00640 1 --2- 172.21.15.18:0/2382391036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39cea00640 1 -- 172.21.15.18:0/2382391036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e3a670 msgr2=0x564f80e4eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39cea00640 1 --2- 172.21.15.18:0/2382391036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e3a670 0x564f80e4eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39cea00640 1 -- 172.21.15.18:0/2382391036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 msgr2=0x564f80e3f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39cea00640 1 --2- 172.21.15.18:0/2382391036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 0x564f80e3f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.341+0000 7f39cea00640 1 -- 172.21.15.18:0/2382391036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f80ab13f0 con 0x564f80e39d50 2024-04-16T17:26:32.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.344+0000 7f39cea00640 1 --2- 172.21.15.18:0/2382391036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e429e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x564f80e560d0 tx=0x7f39c0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4727f42d0380eb98 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.344+0000 7f39bea00640 1 -- 172.21.15.18:0/2382391036 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0020950 con 0x564f80e39d50 2024-04-16T17:26:32.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.344+0000 7f39bea00640 1 -- 172.21.15.18:0/2382391036 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c0026250 con 0x564f80e39d50 2024-04-16T17:26:32.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.344+0000 7f39bea00640 1 -- 172.21.15.18:0/2382391036 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0011730 con 0x564f80e39d50 2024-04-16T17:26:32.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.345+0000 7f39d8244740 1 -- 172.21.15.18:0/2382391036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 msgr2=0x564f80e429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.345+0000 7f39d8244740 1 --2- 172.21.15.18:0/2382391036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e429e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x564f80e560d0 tx=0x7f39c0015000 comp rx=0 tx=0).stop 2024-04-16T17:26:32.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.346+0000 7f39d8244740 1 -- 172.21.15.18:0/2382391036 shutdown_connections 2024-04-16T17:26:32.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.346+0000 7f39d8244740 1 --2- 172.21.15.18:0/2382391036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 0x564f80e3f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.346+0000 7f39d8244740 1 --2- 172.21.15.18:0/2382391036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e3a670 0x564f80e4eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.346+0000 7f39d8244740 1 --2- 172.21.15.18:0/2382391036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e39d50 0x564f80e429e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.346+0000 7f39d8244740 1 -- 172.21.15.18:0/2382391036 >> 172.21.15.18:0/2382391036 conn(0x564f80e34480 msgr2=0x564f80e358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.346+0000 7f39d8244740 1 -- 172.21.15.18:0/2382391036 shutdown_connections 2024-04-16T17:26:32.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.346+0000 7f39d8244740 1 -- 172.21.15.18:0/2382391036 wait complete. 2024-04-16T17:26:32.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.347+0000 7f39d8244740 1 -- start start 2024-04-16T17:26:32.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.347+0000 7f39d8244740 1 Processor -- start 2024-04-16T17:26:32.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.350+0000 7f39d8244740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e39d50 0x564f80eb21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.350+0000 7f39d8244740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e3a670 0x564f80eb26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.350+0000 7f39d8244740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 0x564f80e4a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.350+0000 7f39d8244740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01bccd0 con 0x564f80e43250 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.350+0000 7f39d8244740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01be140 con 0x564f80e39d50 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.350+0000 7f39d8244740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01bd3a0 con 0x564f80e3a670 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.351+0000 7f39d8244740 10 client.0 _ll_register_callbacks cb 0x7f39c8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.351+0000 7f39cea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e39d50 0x564f80eb21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.351+0000 7f39cf400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 0x564f80e4a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.351+0000 7f39cf400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 0x564f80e4a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42334/0 (socket says 172.21.15.18:42334) 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.351+0000 7f39cf400640 1 -- 172.21.15.18:0/37063790 learned_addr learned my addr 172.21.15.18:0/37063790 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.351+0000 7f39c7e00640 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e3a670 0x564f80eb26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.352+0000 7f39c7e00640 1 -- 172.21.15.18:0/37063790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e39d50 msgr2=0x564f80eb21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.352+0000 7f39c7e00640 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e39d50 0x564f80eb21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.352+0000 7f39c7e00640 1 -- 172.21.15.18:0/37063790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 msgr2=0x564f80e4a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.352+0000 7f39c7e00640 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 0x564f80e4a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.352+0000 7f39c7e00640 1 -- 172.21.15.18:0/37063790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f80b7b7d0 con 0x564f80e3a670 2024-04-16T17:26:32.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.355+0000 7f39c7e00640 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e3a670 0x564f80eb26f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006fd0 tx=0x7f39b80137b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d205d91033c7568b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.355+0000 7f39b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.355+0000 7f39b3e00640 1 -- 172.21.15.18:0/37063790 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8025020 con 0x564f80e3a670 2024-04-16T17:26:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.356+0000 7f39b3e00640 1 -- 172.21.15.18:0/37063790 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f39b80185f0 con 0x564f80e3a670 2024-04-16T17:26:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.357+0000 7f39d8244740 1 -- 172.21.15.18:0/37063790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c0026250 con 0x564f80e3a670 2024-04-16T17:26:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.358+0000 7f39b3e00640 1 -- 172.21.15.18:0/37063790 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f39b8018810 con 0x564f80e3a670 2024-04-16T17:26:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.358+0000 7f39d8244740 10 client.30973 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.358+0000 7f39d8244740 1 -- 172.21.15.18:0/37063790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564f80e4baf0 con 0x564f80e3a670 2024-04-16T17:26:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.359+0000 7f39b3e00640 1 -- 172.21.15.18:0/37063790 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c0026250 con 0x564f80e3a670 2024-04-16T17:26:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.359+0000 7f39b3e00640 1 -- 172.21.15.18:0/37063790 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f39b8015070 con 0x564f80e3a670 2024-04-16T17:26:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.359+0000 7f39d8244740 10 client.30973 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.359+0000 7f39d8244740 10 client.30973 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:26:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.359+0000 7f39d8244740 20 client.30973 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.359+0000 7f39d8244740 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x564f80ecaa60 0x564f80ecce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.359+0000 7f39d8244740 4 client.30973 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:26:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.359+0000 7f39d8244740 1 -- 172.21.15.18:0/37063790 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f80da68b0 con 0x564f80ecaa60 2024-04-16T17:26:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.360+0000 7f39cea00640 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x564f80ecaa60 0x564f80ecce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.361+0000 7f39cea00640 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x564f80ecaa60 0x564f80ecce80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49d5579e8382c1a6 server_cookie=8c4193c51994e15b in_seq=0 out_seq=0 2024-04-16T17:26:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39b3e00640 10 client.30973 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:26:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39b3e00640 1 -- 172.21.15.18:0/37063790 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f80da68b0 con 0x564f80ecaa60 2024-04-16T17:26:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39b3e00640 10 client.30973 handle_command_reply: tid=0 2024-04-16T17:26:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 client.30973 shutdown 2024-04-16T17:26:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 -- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e3a670 msgr2=0x564f80eb26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e3a670 0x564f80eb26f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006fd0 tx=0x7f39b80137b0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 -- 172.21.15.18:0/37063790 shutdown_connections 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x564f80ecaa60 0x564f80ecce80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564f80e43250 0x564f80e4a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f80e3a670 0x564f80eb26f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39b3e00640 0 client.30973 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 --2- 172.21.15.18:0/37063790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f80e39d50 0x564f80eb21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 -- 172.21.15.18:0/37063790 >> 172.21.15.18:0/37063790 conn(0x564f80e34480 msgr2=0x564f80e36140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 -- 172.21.15.18:0/37063790 shutdown_connections 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.362+0000 7f39d8244740 1 -- 172.21.15.18:0/37063790 wait complete. 2024-04-16T17:26:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.363+0000 7f39d8244740 20 client.30973 trim_cache size 0 max 16384 2024-04-16T17:26:32.364 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.364+0000 7f39d5600640 1 -- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39b40a3e60 msgr2=0x7f39b40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.364+0000 7f39d5600640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39b40a3e60 0x7f39b40a6320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f39c0005fd0 tx=0x7f39c0005ec0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.364+0000 7f39d5600640 1 -- 172.21.15.18:0/1580112306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d010a590 msgr2=0x7f39d007b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.364+0000 7f39d5600640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d010a590 0x7f39d007b840 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f39c80348f0 tx=0x7f39c8003ba0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.365+0000 7f39d5600640 1 -- 172.21.15.18:0/1580112306 shutdown_connections 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.365+0000 7f39d5600640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f39b40a3e60 0x7f39b40a6320 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.365+0000 7f39d5600640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d010a590 0x7f39d007b840 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.365+0000 7f39d5600640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39d01090c0 0x7f39d0082b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.365+0000 7f39d5600640 1 --2- 172.21.15.18:0/1580112306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0108720 0x7f39d00825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.365+0000 7f39d5600640 1 -- 172.21.15.18:0/1580112306 >> 172.21.15.18:0/1580112306 conn(0x7f39d0077e70 msgr2=0x7f39d00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.365+0000 7f39d5600640 1 -- 172.21.15.18:0/1580112306 shutdown_connections 2024-04-16T17:26:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.365+0000 7f39d5600640 1 -- 172.21.15.18:0/1580112306 wait complete. 2024-04-16T17:26:32.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:26:32.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:32 smithi043 ceph-mon[36901]: pgmap v696: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 461 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 275 op/s 2024-04-16T17:26:32.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:32 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1892464362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:26:32.648 INFO:tasks.workunit.client.0.smithi018.stdout: 10 109785 10.35 MB/sec execute 404 sec latency 209.060 ms 2024-04-16T17:26:32.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:32 smithi022 ceph-mon[36023]: pgmap v696: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 461 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 275 op/s 2024-04-16T17:26:32.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:32 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1892464362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:26:32.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.800+0000 7f9d16800640 1 -- 172.21.15.18:0/753711869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10108720 msgr2=0x7f9d10108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.800+0000 7f9d16800640 1 --2- 172.21.15.18:0/753711869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10108720 0x7f9d10108b80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009860 tx=0x7f9cfc02bc10 comp rx=0 tx=0).stop 2024-04-16T17:26:32.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.800+0000 7f9d16800640 1 -- 172.21.15.18:0/753711869 shutdown_connections 2024-04-16T17:26:32.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.800+0000 7f9d16800640 1 --2- 172.21.15.18:0/753711869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d101090c0 0x7f9d10115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.800+0000 7f9d16800640 1 --2- 172.21.15.18:0/753711869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d10108720 0x7f9d10108b80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.800+0000 7f9d16800640 1 --2- 172.21.15.18:0/753711869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d1010a590 0x7f9d1010a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.800+0000 7f9d16800640 1 -- 172.21.15.18:0/753711869 >> 172.21.15.18:0/753711869 conn(0x7f9d10077e70 msgr2=0x7f9d10106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.801+0000 7f9d16800640 1 -- 172.21.15.18:0/753711869 shutdown_connections 2024-04-16T17:26:32.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.803+0000 7f9d16800640 1 -- 172.21.15.18:0/753711869 wait complete. 2024-04-16T17:26:32.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.804+0000 7f9d16800640 1 Processor -- start 2024-04-16T17:26:32.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.805+0000 7f9d16800640 1 -- start start 2024-04-16T17:26:32.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.805+0000 7f9d16800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d10108720 0x7f9d1019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.805+0000 7f9d16800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d101090c0 0x7f9d1019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.805+0000 7f9d16800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1010a590 0x7f9d101a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.805+0000 7f9d16800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10118af0 con 0x7f9d10108720 2024-04-16T17:26:32.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.805+0000 7f9d16800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10118970 con 0x7f9d101090c0 2024-04-16T17:26:32.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.805+0000 7f9d16800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10118c70 con 0x7f9d1010a590 2024-04-16T17:26:32.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.806+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d101090c0 0x7f9d1019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.806+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1010a590 0x7f9d101a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.806+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d101090c0 0x7f9d1019f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55776/0 (socket says 172.21.15.18:55776) 2024-04-16T17:26:32.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.807+0000 7f9d0ea00640 1 -- 172.21.15.18:0/1803755414 learned_addr learned my addr 172.21.15.18:0/1803755414 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.807+0000 7f9d0f400640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d10108720 0x7f9d1019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.807+0000 7f9d0ea00640 1 -- 172.21.15.18:0/1803755414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1010a590 msgr2=0x7f9d101a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.807+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1010a590 0x7f9d101a38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.807+0000 7f9d0ea00640 1 -- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d10108720 msgr2=0x7f9d1019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.807+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d10108720 0x7f9d1019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.808+0000 7f9d0ea00640 1 -- 172.21.15.18:0/1803755414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d101a3fe0 con 0x7f9d101090c0 2024-04-16T17:26:32.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.808+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d101090c0 0x7f9d1019f530 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc037040 tx=0x7f9cfc0346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.809+0000 7f9d07e00640 1 -- 172.21.15.18:0/1803755414 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc041020 con 0x7f9d101090c0 2024-04-16T17:26:32.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.809+0000 7f9d07e00640 1 -- 172.21.15.18:0/1803755414 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cfc0044d0 con 0x7f9d101090c0 2024-04-16T17:26:32.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.809+0000 7f9d07e00640 1 -- 172.21.15.18:0/1803755414 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc039040 con 0x7f9d101090c0 2024-04-16T17:26:32.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.809+0000 7f9d16800640 1 -- 172.21.15.18:0/1803755414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d101a4270 con 0x7f9d101090c0 2024-04-16T17:26:32.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.809+0000 7f9d16800640 1 -- 172.21.15.18:0/1803755414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101a4650 con 0x7f9d101090c0 2024-04-16T17:26:32.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.818+0000 7f9d19122740 1 Processor -- start 2024-04-16T17:26:32.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.818+0000 7f9d07e00640 1 -- 172.21.15.18:0/1803755414 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9cfc044020 con 0x7f9d101090c0 2024-04-16T17:26:32.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.822+0000 7f9d19122740 1 -- start start 2024-04-16T17:26:32.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.822+0000 7f9d19122740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7fd50 0x55fe49d80130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.822+0000 7f9d19122740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 0x55fe49d88d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.822+0000 7f9d19122740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d89250 0x55fe49d8b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.822+0000 7f9d19122740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fe49ce5020 con 0x55fe49d7fd50 2024-04-16T17:26:32.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.822+0000 7f9d19122740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe49c8b460 con 0x55fe49d80670 2024-04-16T17:26:32.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.822+0000 7f9d19122740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fe49ce0fe0 con 0x55fe49d89250 2024-04-16T17:26:32.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.823+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d89250 0x55fe49d8b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.823+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d89250 0x55fe49d8b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52754/0 (socket says 172.21.15.18:52754) 2024-04-16T17:26:32.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.823+0000 7f9d0ea00640 1 -- 172.21.15.18:0/273093921 learned_addr learned my addr 172.21.15.18:0/273093921 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.824+0000 7f9d0ea00640 1 -- 172.21.15.18:0/273093921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 msgr2=0x55fe49d88d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:26:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.824+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/273093921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 0x55fe49d88d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.824+0000 7f9d0ea00640 1 -- 172.21.15.18:0/273093921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7fd50 msgr2=0x55fe49d80130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.824+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/273093921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7fd50 0x55fe49d80130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.824+0000 7f9d0ea00640 1 -- 172.21.15.18:0/273093921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe49c93040 con 0x55fe49d89250 2024-04-16T17:26:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.824+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/273093921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d89250 0x55fe49d8b640 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0ef560 tx=0x7f9cfc0f7ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b096f30c4bacd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.824+0000 7f9d07e00640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cf40a3da0 0x7f9cf40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.825+0000 7f9d04c00640 1 -- 172.21.15.18:0/273093921 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0ba6e0 con 0x55fe49d89250 2024-04-16T17:26:32.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.825+0000 7f9d04c00640 1 -- 172.21.15.18:0/273093921 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cfc0f8040 con 0x55fe49d89250 2024-04-16T17:26:32.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.825+0000 7f9d04c00640 1 -- 172.21.15.18:0/273093921 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0bdbc0 con 0x55fe49d89250 2024-04-16T17:26:32.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.825+0000 7f9d07e00640 1 -- 172.21.15.18:0/1803755414 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f9cfc0f1c70 con 0x7f9d101090c0 2024-04-16T17:26:32.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.825+0000 7f9d0fe00640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cf40a3da0 0x7f9cf40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.826+0000 7f9d19122740 1 -- 172.21.15.18:0/273093921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d89250 msgr2=0x55fe49d8b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.826+0000 7f9d19122740 1 --2- 172.21.15.18:0/273093921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d89250 0x55fe49d8b640 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0ef560 tx=0x7f9cfc0f7ac0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.826+0000 7f9d19122740 1 -- 172.21.15.18:0/273093921 shutdown_connections 2024-04-16T17:26:32.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.826+0000 7f9d19122740 1 --2- 172.21.15.18:0/273093921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d89250 0x55fe49d8b640 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.826+0000 7f9d19122740 1 --2- 172.21.15.18:0/273093921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 0x55fe49d88d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.826+0000 7f9d19122740 1 --2- 172.21.15.18:0/273093921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7fd50 0x55fe49d80130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.826+0000 7f9d19122740 1 -- 172.21.15.18:0/273093921 >> 172.21.15.18:0/273093921 conn(0x55fe49d7a480 msgr2=0x55fe49d7a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.826+0000 7f9d0fe00640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cf40a3da0 0x7f9cf40a6260 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9d08005fd0 tx=0x7f9d080094e0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.829+0000 7f9d19122740 1 -- 172.21.15.18:0/273093921 shutdown_connections 2024-04-16T17:26:32.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.829+0000 7f9d19122740 1 -- 172.21.15.18:0/273093921 wait complete. 2024-04-16T17:26:32.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.829+0000 7f9d19122740 1 -- start start 2024-04-16T17:26:32.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.830+0000 7f9d19122740 1 Processor -- start 2024-04-16T17:26:32.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.830+0000 7f9d19122740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d7fd50 0x55fe49d86270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d19122740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d80670 0x55fe49d867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d19122740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d86cf0 0x55fe49d95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d0f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d7fd50 0x55fe49d86270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d19122740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55fe49d8be00 con 0x55fe49d80670 2024-04-16T17:26:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d19122740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe49c9e830 con 0x55fe49d7fd50 2024-04-16T17:26:32.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d19122740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fe49d8bf80 con 0x55fe49d86cf0 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d0f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d7fd50 0x55fe49d86270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55784/0 (socket says 172.21.15.18:55784) 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d0f400640 1 -- 172.21.15.18:0/1359524692 learned_addr learned my addr 172.21.15.18:0/1359524692 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.831+0000 7f9d19122740 10 client.0 _ll_register_callbacks cb 0x55fe49ca6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.832+0000 7f9d0fe00640 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d86cf0 0x55fe49d95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.832+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d80670 0x55fe49d867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.832+0000 7f9d0f400640 1 -- 172.21.15.18:0/1359524692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d86cf0 msgr2=0x55fe49d95270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.832+0000 7f9d0f400640 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d86cf0 0x55fe49d95270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.832+0000 7f9d0f400640 1 -- 172.21.15.18:0/1359524692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d80670 msgr2=0x55fe49d867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.832+0000 7f9d0f400640 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d80670 0x55fe49d867b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.832+0000 7f9d0f400640 1 -- 172.21.15.18:0/1359524692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe49ca2920 con 0x55fe49d7fd50 2024-04-16T17:26:32.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.833+0000 7f9d0f400640 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d7fd50 0x55fe49d86270 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010680 tx=0x7f9d00010b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=522c43cce77edb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.833+0000 7f9cf9600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:32.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.833+0000 7f9cf9600640 1 -- 172.21.15.18:0/1359524692 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00017070 con 0x55fe49d7fd50 2024-04-16T17:26:32.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.833+0000 7f9cf9600640 1 -- 172.21.15.18:0/1359524692 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f9d0000eab0 con 0x55fe49d7fd50 2024-04-16T17:26:32.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.833+0000 7f9d19122740 1 -- 172.21.15.18:0/1359524692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cfc0f8040 con 0x55fe49d7fd50 2024-04-16T17:26:32.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.834+0000 7f9cf9600640 1 -- 172.21.15.18:0/1359524692 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9d00026050 con 0x55fe49d7fd50 2024-04-16T17:26:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.836+0000 7f9d19122740 10 client.31136 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:32.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.836+0000 7f9d19122740 1 -- 172.21.15.18:0/1359524692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fe49c707f0 con 0x55fe49d7fd50 2024-04-16T17:26:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.836+0000 7f9cf9600640 1 -- 172.21.15.18:0/1359524692 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cfc0f8040 con 0x55fe49d7fd50 2024-04-16T17:26:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.836+0000 7f9cf9600640 1 -- 172.21.15.18:0/1359524692 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f9d0001b400 con 0x55fe49d7fd50 2024-04-16T17:26:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.837+0000 7f9d19122740 10 client.31136 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.837+0000 7f9d19122740 10 client.31136 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:26:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.837+0000 7f9d19122740 20 client.31136 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.837+0000 7f9d19122740 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fe49d998e0 0x55fe49d9bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.837+0000 7f9d19122740 4 client.31136 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:32.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.837+0000 7f9d19122740 1 -- 172.21.15.18:0/1359524692 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe49cf2a40 con 0x55fe49d998e0 2024-04-16T17:26:32.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.838+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fe49d998e0 0x55fe49d9bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.844+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fe49d998e0 0x55fe49d9bd80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38333473f2a5427a server_cookie=517bf6a8fd0f2208 in_seq=0 out_seq=0 2024-04-16T17:26:32.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.844+0000 7f9cf9600640 10 client.31136 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:26:32.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.848+0000 7f9cf9600640 1 -- 172.21.15.18:0/1359524692 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe49cf2a40 con 0x55fe49d998e0 2024-04-16T17:26:32.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.848+0000 7f9cf9600640 10 client.31136 handle_command_reply: tid=0 2024-04-16T17:26:32.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.848+0000 7f9d19122740 1 client.31136 shutdown 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.848+0000 7f9d19122740 1 -- 172.21.15.18:0/1359524692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d7fd50 msgr2=0x55fe49d86270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.848+0000 7f9d19122740 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d7fd50 0x55fe49d86270 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010680 tx=0x7f9d00010b40 comp rx=0 tx=0).stop 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 1 -- 172.21.15.18:0/1359524692 shutdown_connections 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fe49d998e0 0x55fe49d9bd80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d86cf0 0x55fe49d95270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d80670 0x55fe49d867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 1 --2- 172.21.15.18:0/1359524692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d7fd50 0x55fe49d86270 secure :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010680 tx=0x7f9d00010b40 comp rx=0 tx=0).stop 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 1 -- 172.21.15.18:0/1359524692 >> 172.21.15.18:0/1359524692 conn(0x55fe49d7a480 msgr2=0x55fe49d7c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9cf9600640 0 client.31136 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 1 -- 172.21.15.18:0/1359524692 shutdown_connections 2024-04-16T17:26:32.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 1 -- 172.21.15.18:0/1359524692 wait complete. 2024-04-16T17:26:32.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.849+0000 7f9d19122740 20 client.31136 trim_cache size 0 max 16384 2024-04-16T17:26:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.883+0000 7f9d19122740 1 Processor -- start 2024-04-16T17:26:32.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.884+0000 7f9d19122740 1 -- start start 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.884+0000 7f9d19122740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d80670 0x55fe49d853d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.884+0000 7f9d19122740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d86cf0 0x55fe49d7bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.884+0000 7f9d19122740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7c500 0x55fe49da1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.884+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d86cf0 0x55fe49d7bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d86cf0 0x55fe49d7bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55786/0 (socket says 172.21.15.18:55786) 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d19122740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0012a70 con 0x55fe49d7c500 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d19122740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0010340 con 0x55fe49d86cf0 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d19122740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf000e1b0 con 0x55fe49d80670 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7c500 0x55fe49da1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7c500 0x55fe49da1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42362/0 (socket says 172.21.15.18:42362) 2024-04-16T17:26:32.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d0fe00640 1 -- 172.21.15.18:0/3048466154 learned_addr learned my addr 172.21.15.18:0/3048466154 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d0fe00640 1 -- 172.21.15.18:0/3048466154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d80670 msgr2=0x55fe49d853d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:26:32.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d0fe00640 1 --2- 172.21.15.18:0/3048466154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d80670 0x55fe49d853d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d0fe00640 1 -- 172.21.15.18:0/3048466154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d86cf0 msgr2=0x55fe49d7bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.885+0000 7f9d0fe00640 1 --2- 172.21.15.18:0/3048466154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d86cf0 0x55fe49d7bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.886+0000 7f9d0fe00640 1 -- 172.21.15.18:0/3048466154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe499f73f0 con 0x55fe49d7c500 2024-04-16T17:26:32.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.886+0000 7f9d0fe00640 1 --2- 172.21.15.18:0/3048466154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7c500 0x55fe49da1690 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9d08011570 tx=0x7f9d08011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291c30394ed3c8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.891+0000 7f9cfbe00640 1 -- 172.21.15.18:0/3048466154 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08023070 con 0x55fe49d7c500 2024-04-16T17:26:32.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.891+0000 7f9cfbe00640 1 -- 172.21.15.18:0/3048466154 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d08017040 con 0x55fe49d7c500 2024-04-16T17:26:32.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.892+0000 7f9d19122740 1 -- 172.21.15.18:0/3048466154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7c500 msgr2=0x55fe49da1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.892+0000 7f9cfbe00640 1 -- 172.21.15.18:0/3048466154 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08023070 con 0x55fe49d7c500 2024-04-16T17:26:32.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.892+0000 7f9d19122740 1 --2- 172.21.15.18:0/3048466154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7c500 0x55fe49da1690 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9d08011570 tx=0x7f9d08011000 comp rx=0 tx=0).stop 2024-04-16T17:26:32.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.892+0000 7f9d19122740 1 -- 172.21.15.18:0/3048466154 shutdown_connections 2024-04-16T17:26:32.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.892+0000 7f9d19122740 1 --2- 172.21.15.18:0/3048466154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d7c500 0x55fe49da1690 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.892+0000 7f9d19122740 1 --2- 172.21.15.18:0/3048466154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d86cf0 0x55fe49d7bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.892+0000 7f9d19122740 1 --2- 172.21.15.18:0/3048466154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d80670 0x55fe49d853d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.892+0000 7f9d19122740 1 -- 172.21.15.18:0/3048466154 >> 172.21.15.18:0/3048466154 conn(0x55fe49d7a480 msgr2=0x55fe49d7b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.894+0000 7f9d19122740 1 -- 172.21.15.18:0/3048466154 shutdown_connections 2024-04-16T17:26:32.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.896+0000 7f9d19122740 1 -- 172.21.15.18:0/3048466154 wait complete. 2024-04-16T17:26:32.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.896+0000 7f9d19122740 1 -- start start 2024-04-16T17:26:32.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.896+0000 7f9d19122740 1 Processor -- start 2024-04-16T17:26:32.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.897+0000 7f9d19122740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d7c500 0x55fe49d830f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.897+0000 7f9d19122740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 0x55fe49d83630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.897+0000 7f9d19122740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d86cf0 0x55fe49d83b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d7c500 0x55fe49d830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d86cf0 0x55fe49d83b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d7c500 0x55fe49d830f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52782/0 (socket says 172.21.15.18:52782) 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 0x55fe49d83630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0f400640 1 -- 172.21.15.18:0/4113181250 learned_addr learned my addr 172.21.15.18:0/4113181250 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf000e800 con 0x55fe49d86cf0 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf000fc70 con 0x55fe49d80670 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf000eed0 con 0x55fe49d7c500 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d19122740 10 client.0 _ll_register_callbacks cb 0x7f9d00009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0f400640 1 -- 172.21.15.18:0/4113181250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 msgr2=0x55fe49d83630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0f400640 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 0x55fe49d83630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0f400640 1 -- 172.21.15.18:0/4113181250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d86cf0 msgr2=0x55fe49d83b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.898+0000 7f9d0f400640 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d86cf0 0x55fe49d83b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.899+0000 7f9d0f400640 1 -- 172.21.15.18:0/4113181250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe49ac17d0 con 0x55fe49d7c500 2024-04-16T17:26:32.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.901+0000 7f9d0f400640 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d7c500 0x55fe49d830f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9d0001ba10 tx=0x7f9d0000e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=312b5512e310063a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:32.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.901+0000 7f9cf9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:32.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.901+0000 7f9cf9600640 1 -- 172.21.15.18:0/4113181250 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00017040 con 0x55fe49d7c500 2024-04-16T17:26:32.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.901+0000 7f9cf9600640 1 -- 172.21.15.18:0/4113181250 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f9d0001b400 con 0x55fe49d7c500 2024-04-16T17:26:32.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.902+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d08017040 con 0x55fe49d7c500 2024-04-16T17:26:32.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.902+0000 7f9cf9600640 1 -- 172.21.15.18:0/4113181250 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9d00047050 con 0x55fe49d7c500 2024-04-16T17:26:32.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.902+0000 7f9d19122740 10 client.30985 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:32.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.902+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fe49e001a0 con 0x55fe49d7c500 2024-04-16T17:26:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.903+0000 7f9cf9600640 1 -- 172.21.15.18:0/4113181250 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d08017040 con 0x55fe49d7c500 2024-04-16T17:26:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.903+0000 7f9cf9600640 1 -- 172.21.15.18:0/4113181250 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f9d0000eb00 con 0x55fe49d7c500 2024-04-16T17:26:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.903+0000 7f9d19122740 10 client.30985 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.903+0000 7f9d19122740 10 client.30985 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:26:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.903+0000 7f9d19122740 20 client.30985 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.903+0000 7f9d19122740 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fe49e00890 0x55fe49e02cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.904+0000 7f9d19122740 4 client.30985 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:26:32.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.904+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe49cec8b0 con 0x55fe49e00890 2024-04-16T17:26:32.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.905+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fe49e00890 0x55fe49e02cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.906+0000 7f9cf9600640 10 client.30985 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:26:32.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.906+0000 7f9d0ea00640 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fe49e00890 0x55fe49e02cb0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c228235329b106a3 server_cookie=1a97539d9f88629a in_seq=0 out_seq=0 2024-04-16T17:26:32.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.911+0000 7f9cf9600640 1 -- 172.21.15.18:0/4113181250 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe49cec8b0 con 0x55fe49e00890 2024-04-16T17:26:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.911+0000 7f9cf9600640 10 client.30985 handle_command_reply: tid=0 2024-04-16T17:26:32.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.911+0000 7f9d19122740 1 client.30985 shutdown 2024-04-16T17:26:32.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.912+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d7c500 msgr2=0x55fe49d830f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.912+0000 7f9d19122740 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d7c500 0x55fe49d830f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9d0001ba10 tx=0x7f9d0000e7c0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.913+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 shutdown_connections 2024-04-16T17:26:32.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.913+0000 7f9d19122740 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55fe49e00890 0x55fe49e02cb0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.913+0000 7f9d19122740 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55fe49d86cf0 0x55fe49d83b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.913+0000 7f9d19122740 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe49d80670 0x55fe49d83630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.913+0000 7f9d19122740 1 --2- 172.21.15.18:0/4113181250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe49d7c500 0x55fe49d830f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.913+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 >> 172.21.15.18:0/4113181250 conn(0x55fe49d7a480 msgr2=0x55fe49d853d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.913+0000 7f9cf9600640 0 client.30985 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:26:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.914+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 shutdown_connections 2024-04-16T17:26:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.914+0000 7f9d19122740 1 -- 172.21.15.18:0/4113181250 wait complete. 2024-04-16T17:26:32.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.914+0000 7f9d19122740 20 client.30985 trim_cache size 0 max 16384 2024-04-16T17:26:32.914 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:26:32.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.915+0000 7f9d16800640 1 -- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cf40a3da0 msgr2=0x7f9cf40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.915+0000 7f9d16800640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cf40a3da0 0x7f9cf40a6260 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9d08005fd0 tx=0x7f9d080094e0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.915+0000 7f9d16800640 1 -- 172.21.15.18:0/1803755414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d101090c0 msgr2=0x7f9d1019f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.915+0000 7f9d16800640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d101090c0 0x7f9d1019f530 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc037040 tx=0x7f9cfc0346f0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.916+0000 7f9d16800640 1 -- 172.21.15.18:0/1803755414 shutdown_connections 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.916+0000 7f9d16800640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f9cf40a3da0 0x7f9cf40a6260 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.916+0000 7f9d16800640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1010a590 0x7f9d101a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.916+0000 7f9d16800640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d101090c0 0x7f9d1019f530 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.916+0000 7f9d16800640 1 --2- 172.21.15.18:0/1803755414 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d10108720 0x7f9d1019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:32.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.916+0000 7f9d16800640 1 -- 172.21.15.18:0/1803755414 >> 172.21.15.18:0/1803755414 conn(0x7f9d10077e70 msgr2=0x7f9d100784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.919+0000 7f9d16800640 1 -- 172.21.15.18:0/1803755414 shutdown_connections 2024-04-16T17:26:32.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:32.919+0000 7f9d16800640 1 -- 172.21.15.18:0/1803755414 wait complete. 2024-04-16T17:26:32.965 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:26:33.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.312+0000 7f11b2e00640 1 -- 172.21.15.18:0/3887467099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ac10d860 msgr2=0x7f11ac10dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.312+0000 7f11b2e00640 1 --2- 172.21.15.18:0/3887467099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ac10d860 0x7f11ac10dc40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f119c0085a0 tx=0x7f119c02fbf0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.314+0000 7f11b2e00640 1 -- 172.21.15.18:0/3887467099 shutdown_connections 2024-04-16T17:26:33.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.314+0000 7f11b2e00640 1 --2- 172.21.15.18:0/3887467099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11ac1099a0 0x7f11ac1156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.314+0000 7f11b2e00640 1 --2- 172.21.15.18:0/3887467099 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11ac109000 0x7f11ac109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.314+0000 7f11b2e00640 1 --2- 172.21.15.18:0/3887467099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ac10d860 0x7f11ac10dc40 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.314+0000 7f11b2e00640 1 -- 172.21.15.18:0/3887467099 >> 172.21.15.18:0/3887467099 conn(0x7f11ac06d3a0 msgr2=0x7f11ac06d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:33.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.314+0000 7f11b2e00640 1 -- 172.21.15.18:0/3887467099 shutdown_connections 2024-04-16T17:26:33.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.314+0000 7f11b2e00640 1 -- 172.21.15.18:0/3887467099 wait complete. 2024-04-16T17:26:33.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.315+0000 7f11b2e00640 1 Processor -- start 2024-04-16T17:26:33.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.316+0000 7f11b2e00640 1 -- start start 2024-04-16T17:26:33.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.316+0000 7f11b2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ac109000 0x7f11ac11f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.316+0000 7f11b2e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11ac1099a0 0x7f11ac11f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.316+0000 7f11b2e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11ac10d860 0x7f11ac118390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.316+0000 7f11b2e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac121140 con 0x7f11ac1099a0 2024-04-16T17:26:33.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.316+0000 7f11b2e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac120fc0 con 0x7f11ac10d860 2024-04-16T17:26:33.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.316+0000 7f11b2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1212c0 con 0x7f11ac109000 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ac109000 0x7f11ac11f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11ab400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11ac1099a0 0x7f11ac11f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11b0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11ac10d860 0x7f11ac118390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11b0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11ac10d860 0x7f11ac118390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55810/0 (socket says 172.21.15.18:55810) 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11ab400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11ac1099a0 0x7f11ac11f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42392/0 (socket says 172.21.15.18:42392) 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11b0e00640 1 -- 172.21.15.18:0/1804025528 learned_addr learned my addr 172.21.15.18:0/1804025528 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11b0e00640 1 -- 172.21.15.18:0/1804025528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ac109000 msgr2=0x7f11ac11f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11b0e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ac109000 0x7f11ac11f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11b0e00640 1 -- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11ac1099a0 msgr2=0x7f11ac11f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11b0e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11ac1099a0 0x7f11ac11f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.317+0000 7f11b0e00640 1 -- 172.21.15.18:0/1804025528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac118c50 con 0x7f11ac10d860 2024-04-16T17:26:33.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.318+0000 7f11b0e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11ac10d860 0x7f11ac118390 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1198007800 tx=0x7f1198007cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:33.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.318+0000 7f11a8c00640 1 -- 172.21.15.18:0/1804025528 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198019030 con 0x7f11ac10d860 2024-04-16T17:26:33.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.318+0000 7f11a8c00640 1 -- 172.21.15.18:0/1804025528 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119800fe00 con 0x7f11ac10d860 2024-04-16T17:26:33.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.318+0000 7f11a8c00640 1 -- 172.21.15.18:0/1804025528 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198004ba0 con 0x7f11ac10d860 2024-04-16T17:26:33.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.318+0000 7f11b2e00640 1 -- 172.21.15.18:0/1804025528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac118f40 con 0x7f11ac10d860 2024-04-16T17:26:33.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.319+0000 7f11b2e00640 1 -- 172.21.15.18:0/1804025528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac11cef0 con 0x7f11ac10d860 2024-04-16T17:26:33.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.324+0000 7f11a8c00640 1 -- 172.21.15.18:0/1804025528 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f119801c070 con 0x7f11ac10d860 2024-04-16T17:26:33.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.326+0000 7f11b5bf9740 1 Processor -- start 2024-04-16T17:26:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.327+0000 7f11b5bf9740 1 -- start start 2024-04-16T17:26:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.327+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 0x55f35af672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.327+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af67830 0x55f35af6fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.327+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 0x55f35af72800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.327+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f35aecc020 con 0x55f35af67830 2024-04-16T17:26:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.327+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f35ae72460 con 0x55f35af66f10 2024-04-16T17:26:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.327+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f35aec7fe0 con 0x55f35af70410 2024-04-16T17:26:33.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.328+0000 7f11a8c00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11900a3d90 0x7f11900a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.328+0000 7f11abe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af67830 0x55f35af6fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.329+0000 7f11a8c00640 1 -- 172.21.15.18:0/1804025528 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f11980d0b70 con 0x7f11ac10d860 2024-04-16T17:26:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.329+0000 7f11abe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af67830 0x55f35af6fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42396/0 (socket says 172.21.15.18:42396) 2024-04-16T17:26:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.329+0000 7f11abe00640 1 -- 172.21.15.18:0/1577924562 learned_addr learned my addr 172.21.15.18:0/1577924562 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.329+0000 7f11ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 0x55f35af72800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.329+0000 7f11b0e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11900a3d90 0x7f11900a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.330+0000 7f11ab400640 1 --2- 172.21.15.18:0/1577924562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 0x55f35af672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.331+0000 7f11abe00640 1 -- 172.21.15.18:0/1577924562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 msgr2=0x55f35af72800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.331+0000 7f11abe00640 1 --2- 172.21.15.18:0/1577924562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 0x55f35af72800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.331+0000 7f11abe00640 1 -- 172.21.15.18:0/1577924562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 msgr2=0x55f35af672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.331+0000 7f11abe00640 1 --2- 172.21.15.18:0/1577924562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 0x55f35af672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.331+0000 7f11abe00640 1 -- 172.21.15.18:0/1577924562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f35ae7a040 con 0x55f35af67830 2024-04-16T17:26:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.332+0000 7f11b0e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11900a3d90 0x7f11900a6250 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f119809c3b0 tx=0x7f11980046c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.332+0000 7f11abe00640 1 --2- 172.21.15.18:0/1577924562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af67830 0x55f35af6fed0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f119c005e00 tx=0x7f119c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24874dac4bdc207c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.332+0000 7f11a1600640 1 -- 172.21.15.18:0/1577924562 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c005020 con 0x55f35af67830 2024-04-16T17:26:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.333+0000 7f11a1600640 1 -- 172.21.15.18:0/1577924562 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119c00f1a0 con 0x55f35af67830 2024-04-16T17:26:33.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.333+0000 7f11a1600640 1 -- 172.21.15.18:0/1577924562 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c00a6d0 con 0x55f35af67830 2024-04-16T17:26:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.334+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1577924562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af67830 msgr2=0x55f35af6fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.334+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1577924562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af67830 0x55f35af6fed0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f119c005e00 tx=0x7f119c000bc0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.334+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1577924562 shutdown_connections 2024-04-16T17:26:33.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.335+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1577924562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 0x55f35af72800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.335+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1577924562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af67830 0x55f35af6fed0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.335+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1577924562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 0x55f35af672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.335+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1577924562 >> 172.21.15.18:0/1577924562 conn(0x55f35af61640 msgr2=0x55f35af61a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:33.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.338+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1577924562 shutdown_connections 2024-04-16T17:26:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.338+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1577924562 wait complete. 2024-04-16T17:26:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.338+0000 7f11b5bf9740 1 -- start start 2024-04-16T17:26:33.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.339+0000 7f11b5bf9740 1 Processor -- start 2024-04-16T17:26:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.339+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af66f10 0x55f35af6d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.339+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af6d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.339+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af70410 0x55f35af6def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.339+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f35af73140 con 0x55f35af70410 2024-04-16T17:26:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.339+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f35ae85830 con 0x55f35af67830 2024-04-16T17:26:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.339+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f35af732c0 con 0x55f35af66f10 2024-04-16T17:26:33.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.340+0000 7f11b5bf9740 10 client.0 _ll_register_callbacks cb 0x55f35ae8d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.340+0000 7f11abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af6d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.340+0000 7f11abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af6d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55834/0 (socket says 172.21.15.18:55834) 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.340+0000 7f11abe00640 1 -- 172.21.15.18:0/1627380602 learned_addr learned my addr 172.21.15.18:0/1627380602 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.340+0000 7f11ab400640 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af66f10 0x55f35af6d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.341+0000 7f11abe00640 1 -- 172.21.15.18:0/1627380602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af66f10 msgr2=0x55f35af6d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.341+0000 7f11abe00640 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af66f10 0x55f35af6d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.341+0000 7f11abe00640 1 -- 172.21.15.18:0/1627380602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af70410 msgr2=0x55f35af6def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.341+0000 7f11abe00640 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af70410 0x55f35af6def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.341+0000 7f11abe00640 1 -- 172.21.15.18:0/1627380602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35ae89920 con 0x55f35af67830 2024-04-16T17:26:33.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.341+0000 7f11ab400640 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af66f10 0x55f35af6d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:26:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.341+0000 7f11abe00640 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af6d9b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f119c030130 tx=0x7f119c00aad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e78ee6c7129fab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.342+0000 7f1196000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.342+0000 7f1196000640 1 -- 172.21.15.18:0/1627380602 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c04c030 con 0x55f35af67830 2024-04-16T17:26:33.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.342+0000 7f1196000640 1 -- 172.21.15.18:0/1627380602 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f119c008d70 con 0x55f35af67830 2024-04-16T17:26:33.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.343+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1627380602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119c00f1a0 con 0x55f35af67830 2024-04-16T17:26:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.344+0000 7f1196000640 1 -- 172.21.15.18:0/1627380602 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f119c030740 con 0x55f35af67830 2024-04-16T17:26:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.344+0000 7f11b5bf9740 10 client.31154 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.344+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1627380602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f35ae577f0 con 0x55f35af67830 2024-04-16T17:26:33.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.345+0000 7f1196000640 1 -- 172.21.15.18:0/1627380602 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119c00f1a0 con 0x55f35af67830 2024-04-16T17:26:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.345+0000 7f1196000640 1 -- 172.21.15.18:0/1627380602 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f119c003890 con 0x55f35af67830 2024-04-16T17:26:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.345+0000 7f11b5bf9740 10 client.31154 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.346+0000 7f11b5bf9740 10 client.31154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.346+0000 7f11b5bf9740 20 client.31154 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.346+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f35af80570 0x55f35af82960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.346+0000 7f11b5bf9740 4 client.31154 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:33.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.346+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1627380602 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f35aed9a40 con 0x55f35af80570 2024-04-16T17:26:33.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.347+0000 7f11ab400640 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f35af80570 0x55f35af82960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.348+0000 7f11ab400640 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f35af80570 0x55f35af82960 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118ce7a32d43b1c5 server_cookie=784fddf725ad93b3 in_seq=0 out_seq=0 2024-04-16T17:26:33.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.348+0000 7f1196000640 10 client.31154 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.352+0000 7f1196000640 1 -- 172.21.15.18:0/1627380602 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f35aed9a40 con 0x55f35af80570 2024-04-16T17:26:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.352+0000 7f1196000640 10 client.31154 handle_command_reply: tid=0 2024-04-16T17:26:33.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.352+0000 7f11b5bf9740 1 client.31154 shutdown 2024-04-16T17:26:33.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1627380602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 msgr2=0x55f35af6d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af6d9b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f119c030130 tx=0x7f119c00aad0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1627380602 shutdown_connections 2024-04-16T17:26:33.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f35af80570 0x55f35af82960 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f1196000640 0 client.31154 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:33.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af70410 0x55f35af6def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af6d9b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/1627380602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af66f10 0x55f35af6d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.353+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1627380602 >> 172.21.15.18:0/1627380602 conn(0x55f35af61640 msgr2=0x55f35af63390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:33.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.354+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1627380602 shutdown_connections 2024-04-16T17:26:33.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.354+0000 7f11b5bf9740 1 -- 172.21.15.18:0/1627380602 wait complete. 2024-04-16T17:26:33.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.354+0000 7f11b5bf9740 20 client.31154 trim_cache size 0 max 16384 2024-04-16T17:26:33.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.384+0000 7f11b5bf9740 1 Processor -- start 2024-04-16T17:26:33.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.384+0000 7f11b5bf9740 1 -- start start 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 0x55f35af6fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af67830 0x55f35af7c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af70410 0x55f35af87f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1c95c0 con 0x55f35af70410 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1c6e90 con 0x55f35af66f10 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1c4d00 con 0x55f35af67830 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af67830 0x55f35af7c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af67830 0x55f35af7c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52846/0 (socket says 172.21.15.18:52846) 2024-04-16T17:26:33.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.385+0000 7f11abe00640 1 -- 172.21.15.18:0/3809802009 learned_addr learned my addr 172.21.15.18:0/3809802009 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:33.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11ab400640 1 --2- 172.21.15.18:0/3809802009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 0x55f35af6fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11abe00640 1 -- 172.21.15.18:0/3809802009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 msgr2=0x55f35af6fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:33.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11abe00640 1 --2- 172.21.15.18:0/3809802009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 0x55f35af6fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11abe00640 1 -- 172.21.15.18:0/3809802009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af70410 msgr2=0x55f35af87f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11abe00640 1 --2- 172.21.15.18:0/3809802009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af70410 0x55f35af87f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11abe00640 1 -- 172.21.15.18:0/3809802009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f35af67660 con 0x55f35af67830 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11abe00640 1 --2- 172.21.15.18:0/3809802009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af67830 0x55f35af7c000 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f119c02fc30 tx=0x7f119c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e68ec45bb645f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11a0c00640 1 -- 172.21.15.18:0/3809802009 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c04c070 con 0x55f35af67830 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11a0c00640 1 -- 172.21.15.18:0/3809802009 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119c00f1a0 con 0x55f35af67830 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.386+0000 7f11a0c00640 1 -- 172.21.15.18:0/3809802009 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c003ed0 con 0x55f35af67830 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.388+0000 7f11b5bf9740 1 -- 172.21.15.18:0/3809802009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af67830 msgr2=0x55f35af7c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.388+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/3809802009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af67830 0x55f35af7c000 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f119c02fc30 tx=0x7f119c000bc0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.388+0000 7f11b5bf9740 1 -- 172.21.15.18:0/3809802009 shutdown_connections 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.388+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/3809802009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af70410 0x55f35af87f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.388+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/3809802009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af67830 0x55f35af7c000 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.388+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/3809802009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af66f10 0x55f35af6fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.388+0000 7f11b5bf9740 1 -- 172.21.15.18:0/3809802009 >> 172.21.15.18:0/3809802009 conn(0x55f35af61640 msgr2=0x55f35af61a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.389+0000 7f11b5bf9740 1 -- 172.21.15.18:0/3809802009 shutdown_connections 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.389+0000 7f11b5bf9740 1 -- 172.21.15.18:0/3809802009 wait complete. 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.389+0000 7f11b5bf9740 1 -- start start 2024-04-16T17:26:33.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.390+0000 7f11b5bf9740 1 Processor -- start 2024-04-16T17:26:33.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.390+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af66f10 0x55f35af787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.390+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af73b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.390+0000 7f11b5bf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 0x55f35af74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.390+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1c5350 con 0x55f35af66f10 2024-04-16T17:26:33.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.390+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1c67c0 con 0x55f35af67830 2024-04-16T17:26:33.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.390+0000 7f11b5bf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1c5a20 con 0x55f35af70410 2024-04-16T17:26:33.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.390+0000 7f11b5bf9740 10 client.0 _ll_register_callbacks cb 0x7f11a400a540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.391+0000 7f11ab400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af66f10 0x55f35af787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.391+0000 7f11ab400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af66f10 0x55f35af787a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42440/0 (socket says 172.21.15.18:42440) 2024-04-16T17:26:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.391+0000 7f11ab400640 1 -- 172.21.15.18:0/2524689254 learned_addr learned my addr 172.21.15.18:0/2524689254 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.392+0000 7f11ab400640 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 0x55f35af74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.392+0000 7f11abe00640 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af73b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.392+0000 7f11ab400640 1 -- 172.21.15.18:0/2524689254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 msgr2=0x55f35af74050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.392+0000 7f11ab400640 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 0x55f35af74050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.392+0000 7f11ab400640 1 -- 172.21.15.18:0/2524689254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 msgr2=0x55f35af73b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:33.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.392+0000 7f11ab400640 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af73b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.392+0000 7f11ab400640 1 -- 172.21.15.18:0/2524689254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35abde3f0 con 0x55f35af66f10 2024-04-16T17:26:33.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.393+0000 7f11ab400640 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af66f10 0x55f35af787a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f11ac119970 tx=0x7f11a4009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c702f5fb8abe29d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:33.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.393+0000 7f1196000640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-04-16T17:26:33.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.393+0000 7f1196000640 1 -- 172.21.15.18:0/2524689254 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4013040 con 0x55f35af66f10 2024-04-16T17:26:33.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.393+0000 7f1196000640 1 -- 172.21.15.18:0/2524689254 <== mon.0 v2:172.21.15.18:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9642+0+0 (secure 0 0 0) 0x7f11a40197d0 con 0x55f35af66f10 2024-04-16T17:26:33.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.394+0000 7f11b5bf9740 1 -- 172.21.15.18:0/2524689254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119c00f1a0 con 0x55f35af66f10 2024-04-16T17:26:33.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.395+0000 7f1196000640 1 -- 172.21.15.18:0/2524689254 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f11a4031050 con 0x55f35af66f10 2024-04-16T17:26:33.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.395+0000 7f11b5bf9740 10 client.20052 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:33.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.395+0000 7f11b5bf9740 1 -- 172.21.15.18:0/2524689254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f35aca87d0 con 0x55f35af66f10 2024-04-16T17:26:33.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.396+0000 7f1196000640 1 -- 172.21.15.18:0/2524689254 <== mon.0 v2:172.21.15.18:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119c00f1a0 con 0x55f35af66f10 2024-04-16T17:26:33.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.396+0000 7f1196000640 1 -- 172.21.15.18:0/2524689254 <== mon.0 v2:172.21.15.18:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f11a402cb20 con 0x55f35af66f10 2024-04-16T17:26:33.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.396+0000 7f11b5bf9740 10 client.20052 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:33.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.396+0000 7f11b5bf9740 10 client.20052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:33.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.396+0000 7f11b5bf9740 20 client.20052 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:33.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.396+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f35af80570 0x55f35afe8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:33.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.396+0000 7f11b5bf9740 4 client.20052 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:26:33.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.396+0000 7f11b5bf9740 1 -- 172.21.15.18:0/2524689254 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f35aed38b0 con 0x55f35af80570 2024-04-16T17:26:33.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.397+0000 7f11abe00640 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f35af80570 0x55f35afe8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:33.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.397+0000 7f11abe00640 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f35af80570 0x55f35afe8ae0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f2e19f79ad3e73 server_cookie=ad75ac075b0e4fc9 in_seq=0 out_seq=0 2024-04-16T17:26:33.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.397+0000 7f1196000640 10 client.20052 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:33.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.400+0000 7f1196000640 1 -- 172.21.15.18:0/2524689254 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f35aed38b0 con 0x55f35af80570 2024-04-16T17:26:33.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.400+0000 7f1196000640 10 client.20052 handle_command_reply: tid=0 2024-04-16T17:26:33.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.400+0000 7f11b5bf9740 1 client.20052 shutdown 2024-04-16T17:26:33.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.400+0000 7f11b5bf9740 1 -- 172.21.15.18:0/2524689254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af66f10 msgr2=0x55f35af787a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.400+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af66f10 0x55f35af787a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f11ac119970 tx=0x7f11a4009df0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.401+0000 7f11b5bf9740 1 -- 172.21.15.18:0/2524689254 shutdown_connections 2024-04-16T17:26:33.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.401+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f35af80570 0x55f35afe8ae0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.401+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f35af70410 0x55f35af74050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.401+0000 7f1196000640 0 client.20052 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:33.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.401+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f35af67830 0x55f35af73b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.401+0000 7f11b5bf9740 1 --2- 172.21.15.18:0/2524689254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f35af66f10 0x55f35af787a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.401+0000 7f11b5bf9740 1 -- 172.21.15.18:0/2524689254 >> 172.21.15.18:0/2524689254 conn(0x55f35af61640 msgr2=0x55f35af5fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:33.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.401+0000 7f11b5bf9740 1 -- 172.21.15.18:0/2524689254 shutdown_connections 2024-04-16T17:26:33.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.402+0000 7f11b5bf9740 1 -- 172.21.15.18:0/2524689254 wait complete. 2024-04-16T17:26:33.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.402+0000 7f11b5bf9740 20 client.20052 trim_cache size 0 max 16384 2024-04-16T17:26:33.402 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:26:33.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.403+0000 7f11b2e00640 1 -- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11900a3d90 msgr2=0x7f11900a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.403+0000 7f11b2e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11900a3d90 0x7f11900a6250 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f119809c3b0 tx=0x7f11980046c0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.403+0000 7f11b2e00640 1 -- 172.21.15.18:0/1804025528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11ac10d860 msgr2=0x7f11ac118390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:33.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.403+0000 7f11b2e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11ac10d860 0x7f11ac118390 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1198007800 tx=0x7f1198007cc0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.404+0000 7f11b2e00640 1 -- 172.21.15.18:0/1804025528 shutdown_connections 2024-04-16T17:26:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.404+0000 7f11b2e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f11900a3d90 0x7f11900a6250 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.404+0000 7f11b2e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11ac10d860 0x7f11ac118390 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.404+0000 7f11b2e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11ac1099a0 0x7f11ac11f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.404+0000 7f11b2e00640 1 --2- 172.21.15.18:0/1804025528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ac109000 0x7f11ac11f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.404+0000 7f11b2e00640 1 -- 172.21.15.18:0/1804025528 >> 172.21.15.18:0/1804025528 conn(0x7f11ac06d3a0 msgr2=0x7f11ac10b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:33.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.405+0000 7f11b2e00640 1 -- 172.21.15.18:0/1804025528 shutdown_connections 2024-04-16T17:26:33.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:33.405+0000 7f11b2e00640 1 -- 172.21.15.18:0/1804025528 wait complete. 2024-04-16T17:26:33.648 INFO:tasks.workunit.client.0.smithi018.stdout: 10 110142 10.39 MB/sec execute 405 sec latency 160.392 ms 2024-04-16T17:26:34.438 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:26:34.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:26:34.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:26:34.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:34 smithi043 ceph-mon[36901]: pgmap v697: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 446 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-04-16T17:26:34.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:34 smithi043 ceph-mon[36901]: osdmap e135: 12 total, 12 up, 12 in 2024-04-16T17:26:34.649 INFO:tasks.workunit.client.0.smithi018.stdout: 10 110448 10.41 MB/sec execute 406 sec latency 217.927 ms 2024-04-16T17:26:34.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:34 smithi022 ceph-mon[36023]: pgmap v697: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 446 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-04-16T17:26:34.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:34 smithi022 ceph-mon[36023]: osdmap e135: 12 total, 12 up, 12 in 2024-04-16T17:26:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 -- 172.21.15.18:0/3128462973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410b4a0 msgr2=0x7f474410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 --2- 172.21.15.18:0/3128462973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410b4a0 0x7f474410d920 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f473c009be0 tx=0x7f473c02bef0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 -- 172.21.15.18:0/3128462973 shutdown_connections 2024-04-16T17:26:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 --2- 172.21.15.18:0/3128462973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410b4a0 0x7f474410d920 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 --2- 172.21.15.18:0/3128462973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47441029e0 0x7f474410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 --2- 172.21.15.18:0/3128462973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4744102030 0x7f4744102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 -- 172.21.15.18:0/3128462973 >> 172.21.15.18:0/3128462973 conn(0x7f4744075da0 msgr2=0x7f47440761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 -- 172.21.15.18:0/3128462973 shutdown_connections 2024-04-16T17:26:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.736+0000 7f4749600640 1 -- 172.21.15.18:0/3128462973 wait complete. 2024-04-16T17:26:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.738+0000 7f4749600640 1 Processor -- start 2024-04-16T17:26:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.738+0000 7f4749600640 1 -- start start 2024-04-16T17:26:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4749600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4744102030 0x7f474410d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4749600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f474410b4a0 0x7f474410b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4749600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410be40 0x7f474410c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4749600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4744111280 con 0x7f4744102030 2024-04-16T17:26:34.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4749600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4744111100 con 0x7f474410be40 2024-04-16T17:26:34.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4749600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4744111400 con 0x7f474410b4a0 2024-04-16T17:26:34.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4743400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410be40 0x7f474410c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4742000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f474410b4a0 0x7f474410b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4743400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410be40 0x7f474410c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55862/0 (socket says 172.21.15.18:55862) 2024-04-16T17:26:34.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4743400640 1 -- 172.21.15.18:0/2957596076 learned_addr learned my addr 172.21.15.18:0/2957596076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:34.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.739+0000 7f4742a00640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4744102030 0x7f474410d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.740+0000 7f4743400640 1 -- 172.21.15.18:0/2957596076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f474410b4a0 msgr2=0x7f474410b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.740+0000 7f4743400640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f474410b4a0 0x7f474410b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.740+0000 7f4743400640 1 -- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4744102030 msgr2=0x7f474410d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.740+0000 7f4743400640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4744102030 0x7f474410d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.740+0000 7f4743400640 1 -- 172.21.15.18:0/2957596076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47441af8f0 con 0x7f474410be40 2024-04-16T17:26:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.740+0000 7f4743400640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410be40 0x7f474410c2f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f473c004af0 tx=0x7f473c0336b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.741+0000 7f473b400640 1 -- 172.21.15.18:0/2957596076 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c043020 con 0x7f474410be40 2024-04-16T17:26:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.741+0000 7f473b400640 1 -- 172.21.15.18:0/2957596076 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473c003e50 con 0x7f474410be40 2024-04-16T17:26:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.741+0000 7f473b400640 1 -- 172.21.15.18:0/2957596076 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c03e6b0 con 0x7f474410be40 2024-04-16T17:26:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.742+0000 7f4749600640 1 -- 172.21.15.18:0/2957596076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47441afb80 con 0x7f474410be40 2024-04-16T17:26:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.742+0000 7f4749600640 1 -- 172.21.15.18:0/2957596076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47441b0050 con 0x7f474410be40 2024-04-16T17:26:34.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.748+0000 7f474c26a740 1 Processor -- start 2024-04-16T17:26:34.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.751+0000 7f473b400640 1 -- 172.21.15.18:0/2957596076 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f473c046020 con 0x7f474410be40 2024-04-16T17:26:34.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.752+0000 7f474c26a740 1 -- start start 2024-04-16T17:26:34.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.753+0000 7f473b400640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47280a3ea0 0x7f47280a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.753+0000 7f474c26a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe394b0 0x56087fe39890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.753+0000 7f474c26a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe42470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.753+0000 7f474c26a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe44da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.753+0000 7f474c26a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56087fd95cb0 con 0x56087fe394b0 2024-04-16T17:26:34.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.753+0000 7f474c26a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56087fd3c370 con 0x56087fe39dd0 2024-04-16T17:26:34.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.753+0000 7f474c26a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56087fd91c70 con 0x56087fe429b0 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.754+0000 7f4743400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe44da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.754+0000 7f4743400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe44da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52872/0 (socket says 172.21.15.18:52872) 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.754+0000 7f4743400640 1 -- 172.21.15.18:0/3126375192 learned_addr learned my addr 172.21.15.18:0/3126375192 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.754+0000 7f473b400640 1 -- 172.21.15.18:0/2957596076 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9566+0+0 (secure 0 0 0) 0x7f473c0f2d50 con 0x7f474410be40 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.754+0000 7f4742000640 1 --2- 172.21.15.18:0/3126375192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe42470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f4743400640 1 -- 172.21.15.18:0/3126375192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 msgr2=0x56087fe42470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f4743400640 1 --2- 172.21.15.18:0/3126375192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe42470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f4742000640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47280a3ea0 0x7f47280a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f4743400640 1 -- 172.21.15.18:0/3126375192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe394b0 msgr2=0x56087fe39890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:34.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f4743400640 1 --2- 172.21.15.18:0/3126375192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe394b0 0x56087fe39890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f4743400640 1 -- 172.21.15.18:0/3126375192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087fd5b910 con 0x56087fe429b0 2024-04-16T17:26:34.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f4743400640 1 --2- 172.21.15.18:0/3126375192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe44da0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f473c0be920 tx=0x7f473c0b3e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e03bb0715cb544c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:34.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f472f400640 1 -- 172.21.15.18:0/3126375192 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c0be760 con 0x56087fe429b0 2024-04-16T17:26:34.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f472f400640 1 -- 172.21.15.18:0/3126375192 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473c0bf090 con 0x56087fe429b0 2024-04-16T17:26:34.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.755+0000 7f472f400640 1 -- 172.21.15.18:0/3126375192 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c0be2f0 con 0x56087fe429b0 2024-04-16T17:26:34.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f4742000640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47280a3ea0 0x7f47280a6360 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x56087fe38f80 tx=0x7f4730005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:34.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 -- 172.21.15.18:0/3126375192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 msgr2=0x56087fe44da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 --2- 172.21.15.18:0/3126375192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe44da0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f473c0be920 tx=0x7f473c0b3e20 comp rx=0 tx=0).stop 2024-04-16T17:26:34.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 -- 172.21.15.18:0/3126375192 shutdown_connections 2024-04-16T17:26:34.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 --2- 172.21.15.18:0/3126375192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe44da0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 --2- 172.21.15.18:0/3126375192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe42470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 --2- 172.21.15.18:0/3126375192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe394b0 0x56087fe39890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 -- 172.21.15.18:0/3126375192 >> 172.21.15.18:0/3126375192 conn(0x56087fe33be0 msgr2=0x56087fe33fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:34.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 -- 172.21.15.18:0/3126375192 shutdown_connections 2024-04-16T17:26:34.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.757+0000 7f474c26a740 1 -- 172.21.15.18:0/3126375192 wait complete. 2024-04-16T17:26:34.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.758+0000 7f474c26a740 1 -- start start 2024-04-16T17:26:34.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.758+0000 7f474c26a740 1 Processor -- start 2024-04-16T17:26:34.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.759+0000 7f474c26a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe394b0 0x56087fe3f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.759+0000 7f474c26a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe3ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.759+0000 7f474c26a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe40460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.759+0000 7f474c26a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56087fd91c70 con 0x56087fe394b0 2024-04-16T17:26:34.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.759+0000 7f474c26a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56087fd4f730 con 0x56087fe39dd0 2024-04-16T17:26:34.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.759+0000 7f474c26a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56087fe45560 con 0x56087fe429b0 2024-04-16T17:26:34.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.759+0000 7f474c26a740 10 client.0 _ll_register_callbacks cb 0x56087fd1e1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:34.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.760+0000 7f4742000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe3ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.760+0000 7f4742000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe3ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55882/0 (socket says 172.21.15.18:55882) 2024-04-16T17:26:34.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.760+0000 7f4742000640 1 -- 172.21.15.18:0/1546374926 learned_addr learned my addr 172.21.15.18:0/1546374926 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:34.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.761+0000 7f4743400640 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe40460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.761+0000 7f4742000640 1 -- 172.21.15.18:0/1546374926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 msgr2=0x56087fe40460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:34.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.761+0000 7f4742000640 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe40460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.761+0000 7f4742000640 1 -- 172.21.15.18:0/1546374926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe394b0 msgr2=0x56087fe3f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:34.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.761+0000 7f4742000640 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe394b0 0x56087fe3f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.761+0000 7f4742000640 1 -- 172.21.15.18:0/1546374926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087fb707e0 con 0x56087fe39dd0 2024-04-16T17:26:34.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.762+0000 7f4742000640 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe3ff20 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f473001fc20 tx=0x7f47300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a81f46da9a54ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:34.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.762+0000 7f4723e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:34.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.762+0000 7f4723e00640 1 -- 172.21.15.18:0/1546374926 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473002d060 con 0x56087fe39dd0 2024-04-16T17:26:34.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.762+0000 7f4723e00640 1 -- 172.21.15.18:0/1546374926 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9566+0+0 (secure 0 0 0) 0x7f473001f0d0 con 0x56087fe39dd0 2024-04-16T17:26:34.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.763+0000 7f474c26a740 1 -- 172.21.15.18:0/1546374926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473c0bf090 con 0x56087fe39dd0 2024-04-16T17:26:34.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.765+0000 7f4723e00640 1 -- 172.21.15.18:0/1546374926 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4730005020 con 0x56087fe39dd0 2024-04-16T17:26:34.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.765+0000 7f474c26a740 10 client.31169 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:34.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.765+0000 7f474c26a740 1 -- 172.21.15.18:0/1546374926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56087fbd0f70 con 0x56087fe39dd0 2024-04-16T17:26:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.765+0000 7f4723e00640 1 -- 172.21.15.18:0/1546374926 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473c0bf090 con 0x56087fe39dd0 2024-04-16T17:26:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.766+0000 7f4723e00640 1 -- 172.21.15.18:0/1546374926 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f473000fcb0 con 0x56087fe39dd0 2024-04-16T17:26:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.766+0000 7f474c26a740 10 client.31169 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.766+0000 7f474c26a740 10 client.31169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.766+0000 7f474c26a740 20 client.31169 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.766+0000 7f474c26a740 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56087fe52950 0x56087fe54d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.766+0000 7f474c26a740 4 client.31169 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.766+0000 7f474c26a740 1 -- 172.21.15.18:0/1546374926 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56087fda36d0 con 0x56087fe52950 2024-04-16T17:26:34.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.895+0000 7f4742a00640 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56087fe52950 0x56087fe54d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.896+0000 7f4742a00640 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56087fe52950 0x56087fe54d40 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91d82c21c752564 server_cookie=67a0393df19cf876 in_seq=0 out_seq=0 2024-04-16T17:26:34.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.896+0000 7f4723e00640 10 client.31169 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:34.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.900+0000 7f4723e00640 1 -- 172.21.15.18:0/1546374926 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56087fda36d0 con 0x56087fe52950 2024-04-16T17:26:34.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.900+0000 7f4723e00640 10 client.31169 handle_command_reply: tid=0 2024-04-16T17:26:34.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.900+0000 7f474c26a740 1 client.31169 shutdown 2024-04-16T17:26:34.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 -- 172.21.15.18:0/1546374926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 msgr2=0x56087fe3ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe3ff20 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f473001fc20 tx=0x7f47300046c0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 -- 172.21.15.18:0/1546374926 shutdown_connections 2024-04-16T17:26:34.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56087fe52950 0x56087fe54d40 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe429b0 0x56087fe40460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe3ff20 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 --2- 172.21.15.18:0/1546374926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe394b0 0x56087fe3f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 -- 172.21.15.18:0/1546374926 >> 172.21.15.18:0/1546374926 conn(0x56087fe33be0 msgr2=0x56087fe35930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f4723e00640 0 client.31169 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.901+0000 7f474c26a740 1 -- 172.21.15.18:0/1546374926 shutdown_connections 2024-04-16T17:26:34.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.905+0000 7f474c26a740 1 -- 172.21.15.18:0/1546374926 wait complete. 2024-04-16T17:26:34.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.905+0000 7f474c26a740 20 client.31169 trim_cache size 0 max 16384 2024-04-16T17:26:34.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.952+0000 7f474c26a740 1 Processor -- start 2024-04-16T17:26:34.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.953+0000 7f474c26a740 1 -- start start 2024-04-16T17:26:34.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.953+0000 7f474c26a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe42110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.953+0000 7f474c26a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe4e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.953+0000 7f474c26a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe429b0 0x56087fe2a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.953+0000 7f474c26a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f47441b8620 con 0x56087fe429b0 2024-04-16T17:26:34.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.953+0000 7f474c26a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47441b5ef0 con 0x56087fe39dd0 2024-04-16T17:26:34.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.953+0000 7f474c26a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47441b3d00 con 0x56087fe394b0 2024-04-16T17:26:34.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.954+0000 7f4742000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe4e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.954+0000 7f4742000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe4e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55896/0 (socket says 172.21.15.18:55896) 2024-04-16T17:26:34.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.954+0000 7f4742000640 1 -- 172.21.15.18:0/1005082478 learned_addr learned my addr 172.21.15.18:0/1005082478 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:34.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.955+0000 7f4743400640 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe429b0 0x56087fe2a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.954+0000 7f4742a00640 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe42110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.955+0000 7f4742000640 1 -- 172.21.15.18:0/1005082478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 msgr2=0x56087fe42110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:34.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.955+0000 7f4742000640 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe42110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.955+0000 7f4742000640 1 -- 172.21.15.18:0/1005082478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe429b0 msgr2=0x56087fe2a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:34.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.955+0000 7f4742000640 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe429b0 0x56087fe2a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.955+0000 7f4742000640 1 -- 172.21.15.18:0/1005082478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087fa68170 con 0x56087fe39dd0 2024-04-16T17:26:34.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.956+0000 7f4742000640 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe4e210 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f473001ad30 tx=0x7f473000ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7611e82ecafeebf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:34.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.957+0000 7f472ea00640 1 -- 172.21.15.18:0/1005082478 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730005020 con 0x56087fe39dd0 2024-04-16T17:26:34.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.957+0000 7f472ea00640 1 -- 172.21.15.18:0/1005082478 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473001c3a0 con 0x56087fe39dd0 2024-04-16T17:26:34.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.957+0000 7f472ea00640 1 -- 172.21.15.18:0/1005082478 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730005020 con 0x56087fe39dd0 2024-04-16T17:26:34.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.958+0000 7f4742a00640 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe42110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:26:34.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.958+0000 7f474c26a740 1 -- 172.21.15.18:0/1005082478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 msgr2=0x56087fe4e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.958+0000 7f474c26a740 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe4e210 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f473001ad30 tx=0x7f473000ffd0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.959+0000 7f474c26a740 1 -- 172.21.15.18:0/1005082478 shutdown_connections 2024-04-16T17:26:34.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.959+0000 7f474c26a740 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe429b0 0x56087fe2a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.959+0000 7f474c26a740 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe39dd0 0x56087fe4e210 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.959+0000 7f474c26a740 1 --2- 172.21.15.18:0/1005082478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe42110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.959+0000 7f474c26a740 1 -- 172.21.15.18:0/1005082478 >> 172.21.15.18:0/1005082478 conn(0x56087fe33be0 msgr2=0x56087fe29ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:34.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.959+0000 7f474c26a740 1 -- 172.21.15.18:0/1005082478 shutdown_connections 2024-04-16T17:26:34.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.959+0000 7f474c26a740 1 -- 172.21.15.18:0/1005082478 wait complete. 2024-04-16T17:26:34.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.960+0000 7f474c26a740 1 -- start start 2024-04-16T17:26:34.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.960+0000 7f474c26a740 1 Processor -- start 2024-04-16T17:26:34.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.961+0000 7f474c26a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.961+0000 7f474c26a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe39dd0 0x56087fe45fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.961+0000 7f474c26a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe429b0 0x56087fe466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.961+0000 7f474c26a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f47441b43b0 con 0x56087fe39dd0 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.961+0000 7f474c26a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47441b5820 con 0x56087fe429b0 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.961+0000 7f474c26a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47441b4a80 con 0x56087fe394b0 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.961+0000 7f474c26a740 10 client.0 _ll_register_callbacks cb 0x7f473001f7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.962+0000 7f4743400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe429b0 0x56087fe466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.962+0000 7f4743400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe429b0 0x56087fe466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55904/0 (socket says 172.21.15.18:55904) 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.962+0000 7f4743400640 1 -- 172.21.15.18:0/442793121 learned_addr learned my addr 172.21.15.18:0/442793121 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:34.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.962+0000 7f4742a00640 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.963+0000 7f4742000640 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe39dd0 0x56087fe45fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.966+0000 7f4742a00640 1 -- 172.21.15.18:0/442793121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe429b0 msgr2=0x56087fe466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:34.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.966+0000 7f4742a00640 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe429b0 0x56087fe466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.966+0000 7f4742a00640 1 -- 172.21.15.18:0/442793121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe39dd0 msgr2=0x56087fe45fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:34.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.966+0000 7f4742a00640 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe39dd0 0x56087fe45fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.966+0000 7f4742a00640 1 -- 172.21.15.18:0/442793121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087feb7e60 con 0x56087fe394b0 2024-04-16T17:26:34.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.967+0000 7f4742a00640 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe4ac30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x56087fd4f880 tx=0x7f4734009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27a848089381d7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:34.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.967+0000 7f4723e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:34.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.968+0000 7f4723e00640 1 -- 172.21.15.18:0/442793121 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734012d90 con 0x56087fe394b0 2024-04-16T17:26:34.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.968+0000 7f4723e00640 1 -- 172.21.15.18:0/442793121 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9566+0+0 (secure 0 0 0) 0x7f4734005020 con 0x56087fe394b0 2024-04-16T17:26:34.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.968+0000 7f474c26a740 1 -- 172.21.15.18:0/442793121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473001c3a0 con 0x56087fe394b0 2024-04-16T17:26:34.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.969+0000 7f4723e00640 1 -- 172.21.15.18:0/442793121 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4734028050 con 0x56087fe394b0 2024-04-16T17:26:34.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.970+0000 7f474c26a740 10 client.31021 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:34.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.970+0000 7f474c26a740 1 -- 172.21.15.18:0/442793121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56087fe2ad00 con 0x56087fe394b0 2024-04-16T17:26:34.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.971+0000 7f4723e00640 1 -- 172.21.15.18:0/442793121 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473001c3a0 con 0x56087fe394b0 2024-04-16T17:26:34.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.971+0000 7f4723e00640 1 -- 172.21.15.18:0/442793121 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f4734004850 con 0x56087fe394b0 2024-04-16T17:26:34.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.971+0000 7f474c26a740 10 client.31021 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:34.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.971+0000 7f474c26a740 10 client.31021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:34.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.971+0000 7f474c26a740 20 client.31021 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:34.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.971+0000 7f474c26a740 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56087fe2b4b0 0x56087fe2d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:34.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.971+0000 7f474c26a740 4 client.31021 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:26:34.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.971+0000 7f474c26a740 1 -- 172.21.15.18:0/442793121 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56087fd9d540 con 0x56087fe2b4b0 2024-04-16T17:26:34.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.972+0000 7f4742000640 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56087fe2b4b0 0x56087fe2d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:34.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.973+0000 7f4742000640 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56087fe2b4b0 0x56087fe2d8d0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58742b1b83a4e9c7 server_cookie=28b03b9cf1733f6b in_seq=0 out_seq=0 2024-04-16T17:26:34.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.974+0000 7f4723e00640 10 client.31021 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:34.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.977+0000 7f4723e00640 1 -- 172.21.15.18:0/442793121 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56087fd9d540 con 0x56087fe2b4b0 2024-04-16T17:26:34.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.977+0000 7f4723e00640 10 client.31021 handle_command_reply: tid=0 2024-04-16T17:26:34.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.977+0000 7f474c26a740 1 client.31021 shutdown 2024-04-16T17:26:34.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f474c26a740 1 -- 172.21.15.18:0/442793121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 msgr2=0x56087fe4ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f474c26a740 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe4ac30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x56087fd4f880 tx=0x7f4734009d30 comp rx=0 tx=0).stop 2024-04-16T17:26:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f474c26a740 1 -- 172.21.15.18:0/442793121 shutdown_connections 2024-04-16T17:26:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f474c26a740 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x56087fe2b4b0 0x56087fe2d8d0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f474c26a740 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56087fe429b0 0x56087fe466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f474c26a740 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56087fe39dd0 0x56087fe45fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f474c26a740 1 --2- 172.21.15.18:0/442793121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56087fe394b0 0x56087fe4ac30 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f474c26a740 1 -- 172.21.15.18:0/442793121 >> 172.21.15.18:0/442793121 conn(0x56087fe33be0 msgr2=0x56087fe29ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.978+0000 7f4723e00640 0 client.31021 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:34.979 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:26:34.980 INFO:teuthology.orchestra.run.smithi018.stdout: "return_code": 0, 2024-04-16T17:26:34.980 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_tag": "6d86fd96-e7d6-4056-bc20-ad6c5cdf8bf9", 2024-04-16T17:26:34.980 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": "asynchronous" 2024-04-16T17:26:34.980 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:26:34.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.979+0000 7f474c26a740 1 -- 172.21.15.18:0/442793121 shutdown_connections 2024-04-16T17:26:34.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.979+0000 7f474c26a740 1 -- 172.21.15.18:0/442793121 wait complete. 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.979+0000 7f474c26a740 20 client.31021 trim_cache size 0 max 16384 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 -- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47280a3ea0 msgr2=0x7f47280a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47280a3ea0 0x7f47280a6360 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x56087fe38f80 tx=0x7f4730005f90 comp rx=0 tx=0).stop 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 -- 172.21.15.18:0/2957596076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410be40 msgr2=0x7f474410c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410be40 0x7f474410c2f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f473c004af0 tx=0x7f473c0336b0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 -- 172.21.15.18:0/2957596076 shutdown_connections 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f47280a3ea0 0x7f47280a6360 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474410be40 0x7f474410c2f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f474410b4a0 0x7f474410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 --2- 172.21.15.18:0/2957596076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4744102030 0x7f474410d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.980+0000 7f4749600640 1 -- 172.21.15.18:0/2957596076 >> 172.21.15.18:0/2957596076 conn(0x7f4744075da0 msgr2=0x7f47440fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:34.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.981+0000 7f4749600640 1 -- 172.21.15.18:0/2957596076 shutdown_connections 2024-04-16T17:26:34.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:34.981+0000 7f4749600640 1 -- 172.21.15.18:0/2957596076 wait complete. 2024-04-16T17:26:35.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:26:35.015 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:26:35.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.323+0000 7f170ee00640 1 -- 172.21.15.18:0/77317603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1708110bc0 msgr2=0x7f170810e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.323+0000 7f170ee00640 1 --2- 172.21.15.18:0/77317603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1708110bc0 0x7f170810e850 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009860 tx=0x7f16f402bb70 comp rx=0 tx=0).stop 2024-04-16T17:26:35.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.324+0000 7f170ee00640 1 -- 172.21.15.18:0/77317603 shutdown_connections 2024-04-16T17:26:35.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.324+0000 7f170ee00640 1 --2- 172.21.15.18:0/77317603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f170811b7e0 0x7f170811dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.324+0000 7f170ee00640 1 --2- 172.21.15.18:0/77317603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1708110bc0 0x7f170810e850 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.324+0000 7f170ee00640 1 --2- 172.21.15.18:0/77317603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1708110750 0x7f170810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.324+0000 7f170ee00640 1 -- 172.21.15.18:0/77317603 >> 172.21.15.18:0/77317603 conn(0x7f170806d0a0 msgr2=0x7f170806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:35.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.324+0000 7f170ee00640 1 -- 172.21.15.18:0/77317603 shutdown_connections 2024-04-16T17:26:35.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.324+0000 7f170ee00640 1 -- 172.21.15.18:0/77317603 wait complete. 2024-04-16T17:26:35.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.325+0000 7f170ee00640 1 Processor -- start 2024-04-16T17:26:35.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f170ee00640 1 -- start start 2024-04-16T17:26:35.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f170ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1708110750 0x7f17081a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f170ee00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1708110bc0 0x7f17081a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f170ee00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170811b7e0 0x7f17081abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f170ee00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1708121180 con 0x7f1708110bc0 2024-04-16T17:26:35.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f170ee00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1708121000 con 0x7f170811b7e0 2024-04-16T17:26:35.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f170ee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1708121300 con 0x7f1708110750 2024-04-16T17:26:35.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f1707e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1708110750 0x7f17081a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f1707e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1708110750 0x7f17081a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:52916/0 (socket says 172.21.15.18:52916) 2024-04-16T17:26:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f1707e00640 1 -- 172.21.15.18:0/474322467 learned_addr learned my addr 172.21.15.18:0/474322467 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.326+0000 7f170ce00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170811b7e0 0x7f17081abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.327+0000 7f170ce00640 1 -- 172.21.15.18:0/474322467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1708110750 msgr2=0x7f17081a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.327+0000 7f170ce00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1708110750 0x7f17081a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.327+0000 7f170ce00640 1 -- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1708110bc0 msgr2=0x7f17081a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.327+0000 7f170ce00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1708110bc0 0x7f17081a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.327+0000 7f170ce00640 1 -- 172.21.15.18:0/474322467 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17081ac5b0 con 0x7f170811b7e0 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.327+0000 7f170ce00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170811b7e0 0x7f17081abe80 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f170000ca80 tx=0x7f170000cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.328+0000 7f1704c00640 1 -- 172.21.15.18:0/474322467 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170000ae20 con 0x7f170811b7e0 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.328+0000 7f170ee00640 1 -- 172.21.15.18:0/474322467 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17081ac8a0 con 0x7f170811b7e0 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.328+0000 7f1704c00640 1 -- 172.21.15.18:0/474322467 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f170000f930 con 0x7f170811b7e0 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.328+0000 7f1704c00640 1 -- 172.21.15.18:0/474322467 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700018ad0 con 0x7f170811b7e0 2024-04-16T17:26:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.328+0000 7f170ee00640 1 -- 172.21.15.18:0/474322467 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17081b4130 con 0x7f170811b7e0 2024-04-16T17:26:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.335+0000 7f1711b96740 1 Processor -- start 2024-04-16T17:26:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.335+0000 7f1704c00640 1 -- 172.21.15.18:0/474322467 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f170000fae0 con 0x7f170811b7e0 2024-04-16T17:26:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.335+0000 7f1711b96740 1 -- start start 2024-04-16T17:26:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.336+0000 7f1711b96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe363e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.336+0000 7f1711b96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363ead0 0x55abe3647170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.336+0000 7f1711b96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3649aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.336+0000 7f1711b96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55abe35a4120 con 0x55abe36476b0 2024-04-16T17:26:35.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.336+0000 7f1711b96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55abe354a340 con 0x55abe363e1b0 2024-04-16T17:26:35.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.336+0000 7f1711b96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55abe35a00e0 con 0x55abe363ead0 2024-04-16T17:26:35.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.336+0000 7f1707400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe363e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1707400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe363e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55938/0 (socket says 172.21.15.18:55938) 2024-04-16T17:26:35.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1707400640 1 -- 172.21.15.18:0/3942174600 learned_addr learned my addr 172.21.15.18:0/3942174600 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:35.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1707400640 1 --2- 172.21.15.18:0/3942174600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3649aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1707400640 1 -- 172.21.15.18:0/3942174600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363ead0 msgr2=0x55abe3647170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:26:35.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1707400640 1 --2- 172.21.15.18:0/3942174600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363ead0 0x55abe3647170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1707400640 1 -- 172.21.15.18:0/3942174600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 msgr2=0x55abe3649aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:35.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1707400640 1 --2- 172.21.15.18:0/3942174600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3649aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1707400640 1 -- 172.21.15.18:0/3942174600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abe33807d0 con 0x55abe363e1b0 2024-04-16T17:26:35.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.337+0000 7f1704c00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f16ec0a3e20 0x7f16ec0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.338+0000 7f1704c00640 1 -- 172.21.15.18:0/474322467 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9566+0+0 (secure 0 0 0) 0x7f17000d0fe0 con 0x7f170811b7e0 2024-04-16T17:26:35.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.338+0000 7f1707400640 1 --2- 172.21.15.18:0/3942174600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe363e590 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005940 tx=0x7f16f4005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b18c68f62c72e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:35.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.338+0000 7f1707e00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f16ec0a3e20 0x7f16ec0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.338+0000 7f16fd600640 1 -- 172.21.15.18:0/3942174600 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4043070 con 0x55abe363e1b0 2024-04-16T17:26:35.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.338+0000 7f16fd600640 1 -- 172.21.15.18:0/3942174600 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f402c710 con 0x55abe363e1b0 2024-04-16T17:26:35.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.338+0000 7f16fd600640 1 -- 172.21.15.18:0/3942174600 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4039070 con 0x55abe363e1b0 2024-04-16T17:26:35.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.339+0000 7f1711b96740 1 -- 172.21.15.18:0/3942174600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 msgr2=0x55abe363e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.339+0000 7f1711b96740 1 --2- 172.21.15.18:0/3942174600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe363e590 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005940 tx=0x7f16f4005af0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.340+0000 7f1707e00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f16ec0a3e20 0x7f16ec0a62e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55abe363dc80 tx=0x7f16f800b610 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:35.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.340+0000 7f1711b96740 1 -- 172.21.15.18:0/3942174600 shutdown_connections 2024-04-16T17:26:35.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.340+0000 7f1711b96740 1 --2- 172.21.15.18:0/3942174600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3649aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.340+0000 7f1711b96740 1 --2- 172.21.15.18:0/3942174600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363ead0 0x55abe3647170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.340+0000 7f1711b96740 1 --2- 172.21.15.18:0/3942174600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe363e590 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.340+0000 7f1711b96740 1 -- 172.21.15.18:0/3942174600 >> 172.21.15.18:0/3942174600 conn(0x55abe36388e0 msgr2=0x55abe3638cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:35.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.341+0000 7f1711b96740 1 -- 172.21.15.18:0/3942174600 shutdown_connections 2024-04-16T17:26:35.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.341+0000 7f1711b96740 1 -- 172.21.15.18:0/3942174600 wait complete. 2024-04-16T17:26:35.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.341+0000 7f1711b96740 1 -- start start 2024-04-16T17:26:35.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.341+0000 7f1711b96740 1 Processor -- start 2024-04-16T17:26:35.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.342+0000 7f1711b96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363e1b0 0x55abe3644710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.342+0000 7f1711b96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe3644c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.342+0000 7f1711b96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3645190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.342+0000 7f1711b96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55abe364a440 con 0x55abe36476b0 2024-04-16T17:26:35.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.342+0000 7f1711b96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55abe355d710 con 0x55abe363ead0 2024-04-16T17:26:35.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.342+0000 7f1711b96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55abe364a5c0 con 0x55abe363e1b0 2024-04-16T17:26:35.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.342+0000 7f1711b96740 10 client.0 _ll_register_callbacks cb 0x55abe3558af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe3644c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363e1b0 0x55abe3644710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe3644c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55952/0 (socket says 172.21.15.18:55952) 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707400640 1 -- 172.21.15.18:0/962064955 learned_addr learned my addr 172.21.15.18:0/962064955 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f170ce00640 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3645190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707400640 1 -- 172.21.15.18:0/962064955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363e1b0 msgr2=0x55abe3644710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707400640 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363e1b0 0x55abe3644710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707400640 1 -- 172.21.15.18:0/962064955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 msgr2=0x55abe3645190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707400640 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3645190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.343+0000 7f1707400640 1 -- 172.21.15.18:0/962064955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abe32b63f0 con 0x55abe363ead0 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.344+0000 7f170ce00640 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3645190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.344+0000 7f1707400640 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe3644c50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f16f402c5f0 tx=0x7f16f40349f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5b2f2a6f81d65b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:35.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.344+0000 7f16f2000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:35.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.344+0000 7f16f2000640 1 -- 172.21.15.18:0/962064955 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f404c070 con 0x55abe363ead0 2024-04-16T17:26:35.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.344+0000 7f16f2000640 1 -- 172.21.15.18:0/962064955 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9566+0+0 (secure 0 0 0) 0x7f16f402cd90 con 0x55abe363ead0 2024-04-16T17:26:35.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.345+0000 7f1711b96740 1 -- 172.21.15.18:0/962064955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f402c710 con 0x55abe363ead0 2024-04-16T17:26:35.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.345+0000 7f16f2000640 1 -- 172.21.15.18:0/962064955 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f16f4055050 con 0x55abe363ead0 2024-04-16T17:26:35.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.345+0000 7f1711b96740 10 client.31187 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:35.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.345+0000 7f1711b96740 1 -- 172.21.15.18:0/962064955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55abe32760f0 con 0x55abe363ead0 2024-04-16T17:26:35.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.346+0000 7f16f2000640 1 -- 172.21.15.18:0/962064955 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f402c710 con 0x55abe363ead0 2024-04-16T17:26:35.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.346+0000 7f16f2000640 1 -- 172.21.15.18:0/962064955 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f16f4048750 con 0x55abe363ead0 2024-04-16T17:26:35.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.346+0000 7f1711b96740 10 client.31187 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:35.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.347+0000 7f1711b96740 10 client.31187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:35.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.347+0000 7f1711b96740 20 client.31187 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:35.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.347+0000 7f1711b96740 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55abe3657890 0x55abe3659c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.347+0000 7f1711b96740 4 client.31187 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:35.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.347+0000 7f1711b96740 1 -- 172.21.15.18:0/962064955 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abe35b1b40 con 0x55abe3657890 2024-04-16T17:26:35.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.347+0000 7f1707e00640 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55abe3657890 0x55abe3659c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.348+0000 7f1707e00640 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55abe3657890 0x55abe3659c80 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e82a54984589fc37 server_cookie=75c3f86252c1f504 in_seq=0 out_seq=0 2024-04-16T17:26:35.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.348+0000 7f16f2000640 10 client.31187 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:35.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.350+0000 7f16f2000640 1 -- 172.21.15.18:0/962064955 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abe35b1b40 con 0x55abe3657890 2024-04-16T17:26:35.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.350+0000 7f16f2000640 10 client.31187 handle_command_reply: tid=0 2024-04-16T17:26:35.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.350+0000 7f1711b96740 1 client.31187 shutdown 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 -- 172.21.15.18:0/962064955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 msgr2=0x55abe3644c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe3644c50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f16f402c5f0 tx=0x7f16f40349f0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 -- 172.21.15.18:0/962064955 shutdown_connections 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55abe3657890 0x55abe3659c80 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe36476b0 0x55abe3645190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe3644c50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 --2- 172.21.15.18:0/962064955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe363e1b0 0x55abe3644710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 -- 172.21.15.18:0/962064955 >> 172.21.15.18:0/962064955 conn(0x55abe36388e0 msgr2=0x55abe363a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f16f2000640 0 client.31187 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 -- 172.21.15.18:0/962064955 shutdown_connections 2024-04-16T17:26:35.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 1 -- 172.21.15.18:0/962064955 wait complete. 2024-04-16T17:26:35.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.351+0000 7f1711b96740 20 client.31187 trim_cache size 0 max 16384 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.380+0000 7f1711b96740 1 Processor -- start 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.380+0000 7f1711b96740 1 -- start start 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.380+0000 7f1711b96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe3646e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.380+0000 7f1711b96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363ead0 0x55abe3637750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.380+0000 7f1711b96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.380+0000 7f1711b96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17081c0ca0 con 0x55abe363ead0 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.380+0000 7f1711b96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17081be570 con 0x55abe363e1b0 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.380+0000 7f1711b96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17081bc3c0 con 0x55abe36476b0 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f1707e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe3646e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f1707e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe3646e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55954/0 (socket says 172.21.15.18:55954) 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f1707e00640 1 -- 172.21.15.18:0/3210224441 learned_addr learned my addr 172.21.15.18:0/3210224441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f170ce00640 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f1707400640 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363ead0 0x55abe3637750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f170ce00640 1 -- 172.21.15.18:0/3210224441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 msgr2=0x55abe3646e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:35.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f170ce00640 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe3646e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f170ce00640 1 -- 172.21.15.18:0/3210224441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363ead0 msgr2=0x55abe3637750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:35.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f170ce00640 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363ead0 0x55abe3637750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f170ce00640 1 -- 172.21.15.18:0/3210224441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abe3649fc0 con 0x55abe36476b0 2024-04-16T17:26:35.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.381+0000 7f1707400640 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363ead0 0x55abe3637750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:26:35.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.382+0000 7f170ce00640 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365cca0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f17000ce2c0 tx=0x7f170009c590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4811b843e8f3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:35.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.382+0000 7f16fcc00640 1 -- 172.21.15.18:0/3210224441 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17000a5780 con 0x55abe36476b0 2024-04-16T17:26:35.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.382+0000 7f16fcc00640 1 -- 172.21.15.18:0/3210224441 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f170009ae10 con 0x55abe36476b0 2024-04-16T17:26:35.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.382+0000 7f16fcc00640 1 -- 172.21.15.18:0/3210224441 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170009c1c0 con 0x55abe36476b0 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.383+0000 7f1711b96740 1 -- 172.21.15.18:0/3210224441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 msgr2=0x55abe365cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.383+0000 7f1711b96740 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365cca0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f17000ce2c0 tx=0x7f170009c590 comp rx=0 tx=0).stop 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.383+0000 7f1711b96740 1 -- 172.21.15.18:0/3210224441 shutdown_connections 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.383+0000 7f1711b96740 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365cca0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.383+0000 7f1711b96740 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363ead0 0x55abe3637750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.383+0000 7f1711b96740 1 --2- 172.21.15.18:0/3210224441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363e1b0 0x55abe3646e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.383+0000 7f1711b96740 1 -- 172.21.15.18:0/3210224441 >> 172.21.15.18:0/3210224441 conn(0x55abe36388e0 msgr2=0x55abe3637040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.383+0000 7f1711b96740 1 -- 172.21.15.18:0/3210224441 shutdown_connections 2024-04-16T17:26:35.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.384+0000 7f1711b96740 1 -- 172.21.15.18:0/3210224441 wait complete. 2024-04-16T17:26:35.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.384+0000 7f1711b96740 1 -- start start 2024-04-16T17:26:35.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.385+0000 7f1711b96740 1 Processor -- start 2024-04-16T17:26:35.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.385+0000 7f1711b96740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363e1b0 0x55abe365a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.385+0000 7f1711b96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe365ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.385+0000 7f1711b96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.385+0000 7f1711b96740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f17081bca30 con 0x55abe363e1b0 2024-04-16T17:26:35.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.385+0000 7f1711b96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17081bdea0 con 0x55abe363ead0 2024-04-16T17:26:35.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.385+0000 7f1711b96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17081bd100 con 0x55abe36476b0 2024-04-16T17:26:35.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.385+0000 7f1711b96740 10 client.0 _ll_register_callbacks cb 0x7f16f40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:35.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.386+0000 7f1707400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe365ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.386+0000 7f1707400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe365ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:55966/0 (socket says 172.21.15.18:55966) 2024-04-16T17:26:35.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.386+0000 7f1707400640 1 -- 172.21.15.18:0/3803060796 learned_addr learned my addr 172.21.15.18:0/3803060796 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:35.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.386+0000 7f170ce00640 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.387+0000 7f170ce00640 1 -- 172.21.15.18:0/3803060796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 msgr2=0x55abe365ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.387+0000 7f170ce00640 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe365ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.387+0000 7f170ce00640 1 -- 172.21.15.18:0/3803060796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363e1b0 msgr2=0x55abe365a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.387+0000 7f170ce00640 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363e1b0 0x55abe365a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.387+0000 7f170ce00640 1 -- 172.21.15.18:0/3803060796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abe36bca70 con 0x55abe36476b0 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.387+0000 7f1707400640 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe365ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.387+0000 7f170ce00640 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365b060 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f170009c950 tx=0x7f17000a0f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0078addfe9cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.387+0000 7f16f2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.388+0000 7f16f2000640 1 -- 172.21.15.18:0/3803060796 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17000ce750 con 0x55abe36476b0 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.388+0000 7f16f2000640 1 -- 172.21.15.18:0/3803060796 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9566+0+0 (secure 0 0 0) 0x7f170009c160 con 0x55abe36476b0 2024-04-16T17:26:35.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.388+0000 7f1711b96740 1 -- 172.21.15.18:0/3803060796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f170009ae10 con 0x55abe36476b0 2024-04-16T17:26:35.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.388+0000 7f16f2000640 1 -- 172.21.15.18:0/3803060796 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f17000a5780 con 0x55abe36476b0 2024-04-16T17:26:35.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.388+0000 7f1711b96740 10 client.31033 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:35.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f1711b96740 1 -- 172.21.15.18:0/3803060796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55abe36bdc60 con 0x55abe36476b0 2024-04-16T17:26:35.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f16f2000640 1 -- 172.21.15.18:0/3803060796 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f170009ae10 con 0x55abe36476b0 2024-04-16T17:26:35.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f16f2000640 1 -- 172.21.15.18:0/3803060796 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f1700020030 con 0x55abe36476b0 2024-04-16T17:26:35.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f1711b96740 10 client.31033 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:35.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f1711b96740 10 client.31033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:35.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f1711b96740 20 client.31033 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:35.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f1711b96740 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55abe36be200 0x55abe36c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:35.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f1711b96740 4 client.31033 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:26:35.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.389+0000 7f1711b96740 1 -- 172.21.15.18:0/3803060796 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55abe35ab9b0 con 0x55abe36be200 2024-04-16T17:26:35.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.390+0000 7f1707e00640 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55abe36be200 0x55abe36c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:35.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.391+0000 7f16f2000640 10 client.31033 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:35.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.391+0000 7f1707e00640 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55abe36be200 0x55abe36c0620 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a6eb56acb0f7cb server_cookie=730611d55ae3268 in_seq=0 out_seq=0 2024-04-16T17:26:35.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.394+0000 7f16f2000640 1 -- 172.21.15.18:0/3803060796 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55abe35ab9b0 con 0x55abe36be200 2024-04-16T17:26:35.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.394+0000 7f16f2000640 10 client.31033 handle_command_reply: tid=0 2024-04-16T17:26:35.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.394+0000 7f1711b96740 1 client.31033 shutdown 2024-04-16T17:26:35.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f1711b96740 1 -- 172.21.15.18:0/3803060796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 msgr2=0x55abe365b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f1711b96740 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365b060 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f170009c950 tx=0x7f17000a0f80 comp rx=0 tx=0).stop 2024-04-16T17:26:35.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f1711b96740 1 -- 172.21.15.18:0/3803060796 shutdown_connections 2024-04-16T17:26:35.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f1711b96740 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55abe36be200 0x55abe36c0620 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f1711b96740 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55abe36476b0 0x55abe365b060 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f1711b96740 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abe363ead0 0x55abe365ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f1711b96740 1 --2- 172.21.15.18:0/3803060796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55abe363e1b0 0x55abe365a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f16f2000640 0 client.31033 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:35.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.395+0000 7f1711b96740 1 -- 172.21.15.18:0/3803060796 >> 172.21.15.18:0/3803060796 conn(0x55abe36388e0 msgr2=0x55abe363a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:35.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.396+0000 7f1711b96740 1 -- 172.21.15.18:0/3803060796 shutdown_connections 2024-04-16T17:26:35.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.396+0000 7f1711b96740 1 -- 172.21.15.18:0/3803060796 wait complete. 2024-04-16T17:26:35.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.396+0000 7f1711b96740 20 client.31033 trim_cache size 0 max 16384 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "scrub active (313 inodes in the stack)", 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": { 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout: "6d86fd96-e7d6-4056-bc20-ad6c5cdf8bf9": { 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout: "path": "/", 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout: "tag": "6d86fd96-e7d6-4056-bc20-ad6c5cdf8bf9", 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout: "options": "recursive,force" 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:26:35.397 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:26:35.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.397+0000 7f170ee00640 1 -- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f16ec0a3e20 msgr2=0x7f16ec0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.397+0000 7f170ee00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f16ec0a3e20 0x7f16ec0a62e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55abe363dc80 tx=0x7f16f800b610 comp rx=0 tx=0).stop 2024-04-16T17:26:35.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.397+0000 7f170ee00640 1 -- 172.21.15.18:0/474322467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170811b7e0 msgr2=0x7f17081abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:35.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.397+0000 7f170ee00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170811b7e0 0x7f17081abe80 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f170000ca80 tx=0x7f170000cf40 comp rx=0 tx=0).stop 2024-04-16T17:26:35.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.398+0000 7f170ee00640 1 -- 172.21.15.18:0/474322467 shutdown_connections 2024-04-16T17:26:35.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.398+0000 7f170ee00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f16ec0a3e20 0x7f16ec0a62e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.398+0000 7f170ee00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170811b7e0 0x7f17081abe80 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.398+0000 7f170ee00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1708110bc0 0x7f17081a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.398+0000 7f170ee00640 1 --2- 172.21.15.18:0/474322467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1708110750 0x7f17081a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:35.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.398+0000 7f170ee00640 1 -- 172.21.15.18:0/474322467 >> 172.21.15.18:0/474322467 conn(0x7f170806d0a0 msgr2=0x7f170811a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:35.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.398+0000 7f170ee00640 1 -- 172.21.15.18:0/474322467 shutdown_connections 2024-04-16T17:26:35.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:26:35.398+0000 7f170ee00640 1 -- 172.21.15.18:0/474322467 wait complete. 2024-04-16T17:26:35.420 INFO:tasks.cephfs.filesystem:scrub status for tag:6d86fd96-e7d6-4056-bc20-ad6c5cdf8bf9 - {'path': '/', 'tag': '6d86fd96-e7d6-4056-bc20-ad6c5cdf8bf9', 'options': 'recursive,force'} 2024-04-16T17:26:35.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:35 smithi043 ceph-mon[36901]: scrub queued for path: / 2024-04-16T17:26:35.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:35 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:35.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:35 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:26:35.641 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:32 smithi018 ceph-mon[31776]: pgmap v696: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 461 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 275 op/s 2024-04-16T17:26:35.641 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:32 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1892464362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:26:35.641 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:34 smithi018 ceph-mon[31776]: pgmap v697: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 446 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-04-16T17:26:35.641 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:34 smithi018 ceph-mon[31776]: osdmap e135: 12 total, 12 up, 12 in 2024-04-16T17:26:35.649 INFO:tasks.workunit.client.0.smithi018.stdout: 10 110613 10.41 MB/sec execute 407 sec latency 284.638 ms 2024-04-16T17:26:35.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:35 smithi022 ceph-mon[36023]: scrub queued for path: / 2024-04-16T17:26:35.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:35 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:35.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:35 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:26:36.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:36 smithi043 ceph-mon[36901]: pgmap v699: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 319 op/s 2024-04-16T17:26:36.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:36 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:26:36.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:36 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:26:36.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:36 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:36.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:36 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:36.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:36 smithi043 ceph-mon[36901]: scrub summary: active paths [/] 2024-04-16T17:26:36.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:36 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:36.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:36 smithi043 ceph-mon[36901]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:36.649 INFO:tasks.workunit.client.0.smithi018.stdout: 10 110854 10.43 MB/sec execute 408 sec latency 207.303 ms 2024-04-16T17:26:36.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:36 smithi022 ceph-mon[36023]: pgmap v699: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 319 op/s 2024-04-16T17:26:36.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:36 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:26:36.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:36 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:26:36.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:36 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:36.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:36 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:36.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:36 smithi022 ceph-mon[36023]: scrub summary: active paths [/] 2024-04-16T17:26:36.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:36 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:36.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:36 smithi022 ceph-mon[36023]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:37.649 INFO:tasks.workunit.client.0.smithi018.stdout: 10 110961 10.42 MB/sec execute 409 sec latency 570.293 ms 2024-04-16T17:26:38.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:38 smithi043 ceph-mon[36901]: pgmap v700: 129 pgs: 129 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 374 op/s 2024-04-16T17:26:38.649 INFO:tasks.workunit.client.0.smithi018.stdout: 10 111229 10.43 MB/sec execute 410 sec latency 204.639 ms 2024-04-16T17:26:38.673 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:38 smithi022 ceph-mon[36023]: pgmap v700: 129 pgs: 129 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 374 op/s 2024-04-16T17:26:38.959 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:35 smithi018 ceph-mon[31776]: scrub queued for path: / 2024-04-16T17:26:39.651 INFO:tasks.workunit.client.0.smithi018.stdout: 10 111405 10.43 MB/sec execute 411 sec latency 287.281 ms 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:35 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:35 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:36 smithi018 ceph-mon[31776]: pgmap v699: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 319 op/s 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:36 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:36 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:36 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:36 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:36 smithi018 ceph-mon[31776]: scrub summary: active paths [/] 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:36 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:39.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:36 smithi018 ceph-mon[31776]: scrub summary: idle+waiting paths [/] 2024-04-16T17:26:39.996 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:38 smithi018 ceph-mon[31776]: pgmap v700: 129 pgs: 129 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 374 op/s 2024-04-16T17:26:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:39 smithi022 ceph-mon[36023]: osdmap e136: 12 total, 12 up, 12 in 2024-04-16T17:26:40.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:39 smithi022 ceph-mon[36023]: pgmap v702: 129 pgs: 129 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2024-04-16T17:26:40.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:39 smithi043 ceph-mon[36901]: osdmap e136: 12 total, 12 up, 12 in 2024-04-16T17:26:40.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:39 smithi043 ceph-mon[36901]: pgmap v702: 129 pgs: 129 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2024-04-16T17:26:40.651 INFO:tasks.workunit.client.0.smithi018.stdout: 10 111561 10.41 MB/sec execute 412 sec latency 221.271 ms 2024-04-16T17:26:41.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:40 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:26:41.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:40 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:26:41.651 INFO:tasks.workunit.client.0.smithi018.stdout: 10 111734 10.40 MB/sec execute 413 sec latency 444.104 ms 2024-04-16T17:26:42.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:41 smithi022 ceph-mon[36023]: pgmap v703: 129 pgs: 129 active+clean; 462 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-04-16T17:26:42.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:41 smithi043 ceph-mon[36901]: pgmap v703: 129 pgs: 129 active+clean; 462 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-04-16T17:26:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2643946280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c100490 msgr2=0x7fe99c100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2643946280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c100490 0x7fe99c100870 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009900 tx=0x7fe98c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:26:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2643946280 shutdown_connections 2024-04-16T17:26:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2643946280 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c10e9d0 0x7fe99c110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2643946280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c100db0 0x7fe99c10e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2643946280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c100490 0x7fe99c100870 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2643946280 >> 172.21.15.18:0/2643946280 conn(0x7fe99c075e10 msgr2=0x7fe99c076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2643946280 shutdown_connections 2024-04-16T17:26:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.552+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2643946280 wait complete. 2024-04-16T17:26:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.565+0000 7fe9a2e00640 1 Processor -- start 2024-04-16T17:26:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.566+0000 7fe9a2e00640 1 -- start start 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.567+0000 7fe9a2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c100490 0x7fe99c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.567+0000 7fe9a2e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c100db0 0x7fe99c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe9a2e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c10e9d0 0x7fe99c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe9a2e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1145e0 con 0x7fe99c100db0 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe9a2e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c114460 con 0x7fe99c10e9d0 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe9a2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c114760 con 0x7fe99c100490 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe99be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c100490 0x7fe99c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe99be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c100490 0x7fe99c1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51500/0 (socket says 172.21.15.18:51500) 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe99be00640 1 -- 172.21.15.18:0/2549675178 learned_addr learned my addr 172.21.15.18:0/2549675178 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe99b400640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c100db0 0x7fe99c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.568+0000 7fe99be00640 1 -- 172.21.15.18:0/2549675178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c10e9d0 msgr2=0x7fe99c19a8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.569+0000 7fe9a0e00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c10e9d0 0x7fe99c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.570+0000 7fe99be00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c10e9d0 0x7fe99c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.570+0000 7fe99be00640 1 -- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c100db0 msgr2=0x7fe99c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.570+0000 7fe99be00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c100db0 0x7fe99c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.570+0000 7fe99be00640 1 -- 172.21.15.18:0/2549675178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c19b020 con 0x7fe99c100490 2024-04-16T17:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.570+0000 7fe99b400640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c100db0 0x7fe99c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.570+0000 7fe9a0e00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c10e9d0 0x7fe99c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.570+0000 7fe99be00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c100490 0x7fe99c1a06f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe98c02c250 tx=0x7fe98c0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.571+0000 7fe998c00640 1 -- 172.21.15.18:0/2549675178 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c041070 con 0x7fe99c100490 2024-04-16T17:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.571+0000 7fe998c00640 1 -- 172.21.15.18:0/2549675178 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c02cca0 con 0x7fe99c100490 2024-04-16T17:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.571+0000 7fe998c00640 1 -- 172.21.15.18:0/2549675178 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c003db0 con 0x7fe99c100490 2024-04-16T17:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.571+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2549675178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c19b2b0 con 0x7fe99c100490 2024-04-16T17:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.572+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2549675178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c1a75b0 con 0x7fe99c100490 2024-04-16T17:26:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.577+0000 7fe998c00640 1 -- 172.21.15.18:0/2549675178 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe98c044070 con 0x7fe99c100490 2024-04-16T17:26:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.580+0000 7fe998c00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe9800a3ea0 0x7fe9800a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.581+0000 7fe998c00640 1 -- 172.21.15.18:0/2549675178 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9538+0+0 (secure 0 0 0) 0x7fe98c0f2660 con 0x7fe99c100490 2024-04-16T17:26:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.581+0000 7fe99b400640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe9800a3ea0 0x7fe9800a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.582+0000 7fe99b400640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe9800a3ea0 0x7fe9800a6360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe99c110d60 tx=0x7fe988005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.591+0000 7fe9a5b75740 1 Processor -- start 2024-04-16T17:26:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.591+0000 7fe9a5b75740 1 -- start start 2024-04-16T17:26:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc392a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc392ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 0x55bcc3936210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bcc388f780 con 0x55bcc3933e20 2024-04-16T17:26:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3835f40 con 0x55bcc392a8a0 2024-04-16T17:26:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bcc388b740 con 0x55bcc392b610 2024-04-16T17:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe99be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc392a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe99be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc392a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51512/0 (socket says 172.21.15.18:51512) 2024-04-16T17:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe99be00640 1 -- 172.21.15.18:0/2380668765 learned_addr learned my addr 172.21.15.18:0/2380668765 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.592+0000 7fe99b400640 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc392ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.593+0000 7fe9a0e00640 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 0x55bcc3936210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.593+0000 7fe99be00640 1 -- 172.21.15.18:0/2380668765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 msgr2=0x55bcc392ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.593+0000 7fe99be00640 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc392ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.593+0000 7fe99be00640 1 -- 172.21.15.18:0/2380668765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 msgr2=0x55bcc3936210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.593+0000 7fe99be00640 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 0x55bcc3936210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.593+0000 7fe99be00640 1 -- 172.21.15.18:0/2380668765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc35a2210 con 0x55bcc392b610 2024-04-16T17:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.594+0000 7fe99be00640 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc392a360 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe98c02c830 tx=0x7fe98c02ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab2661b21d203e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.594+0000 7fe991600640 1 -- 172.21.15.18:0/2380668765 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c04b040 con 0x55bcc392b610 2024-04-16T17:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.594+0000 7fe991600640 1 -- 172.21.15.18:0/2380668765 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c0be9b0 con 0x55bcc392b610 2024-04-16T17:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.594+0000 7fe991600640 1 -- 172.21.15.18:0/2380668765 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0ba890 con 0x55bcc392b610 2024-04-16T17:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.595+0000 7fe9a5b75740 1 -- 172.21.15.18:0/2380668765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 msgr2=0x55bcc392a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.595+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc392a360 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe98c02c830 tx=0x7fe98c02ce50 comp rx=0 tx=0).stop 2024-04-16T17:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.596+0000 7fe9a5b75740 1 -- 172.21.15.18:0/2380668765 shutdown_connections 2024-04-16T17:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.596+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 0x55bcc3936210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.596+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc392ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.596+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/2380668765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc392a360 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.596+0000 7fe9a5b75740 1 -- 172.21.15.18:0/2380668765 >> 172.21.15.18:0/2380668765 conn(0x55bcc38c55d0 msgr2=0x55bcc3925590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.596+0000 7fe9a5b75740 1 -- 172.21.15.18:0/2380668765 shutdown_connections 2024-04-16T17:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.596+0000 7fe9a5b75740 1 -- 172.21.15.18:0/2380668765 wait complete. 2024-04-16T17:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.597+0000 7fe9a5b75740 1 -- start start 2024-04-16T17:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.597+0000 7fe9a5b75740 1 Processor -- start 2024-04-16T17:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.598+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc3930cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.598+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc3931200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.598+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 0x55bcc3931740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.598+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3936910 con 0x55bcc3933e20 2024-04-16T17:26:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.598+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3849390 con 0x55bcc392a8a0 2024-04-16T17:26:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.598+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3936a90 con 0x55bcc392b610 2024-04-16T17:26:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.598+0000 7fe9a5b75740 10 client.0 _ll_register_callbacks cb 0x55bcc3832f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc3930cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc3931200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc3931200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51526/0 (socket says 172.21.15.18:51526) 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99b400640 1 -- 172.21.15.18:0/4044564290 learned_addr learned my addr 172.21.15.18:0/4044564290 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99be00640 1 -- 172.21.15.18:0/4044564290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 msgr2=0x55bcc3931200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99be00640 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc3931200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99be00640 1 -- 172.21.15.18:0/4044564290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 msgr2=0x55bcc3931740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99be00640 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 0x55bcc3931740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99be00640 1 -- 172.21.15.18:0/4044564290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc366e620 con 0x55bcc392a8a0 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99b400640 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc3931200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.599+0000 7fe99be00640 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc3930cc0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0be470 tx=0x7fe98c02cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b719b75716434d27 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.600+0000 7fe986000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.600+0000 7fe986000640 1 -- 172.21.15.18:0/4044564290 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c04b070 con 0x55bcc392a8a0 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.600+0000 7fe986000640 1 -- 172.21.15.18:0/4044564290 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9538+0+0 (secure 0 0 0) 0x7fe98c0c6c90 con 0x55bcc392a8a0 2024-04-16T17:26:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.600+0000 7fe9a5b75740 1 -- 172.21.15.18:0/4044564290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98c0be9b0 con 0x55bcc392a8a0 2024-04-16T17:26:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.601+0000 7fe986000640 1 -- 172.21.15.18:0/4044564290 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe98c0c8030 con 0x55bcc392a8a0 2024-04-16T17:26:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.601+0000 7fe9a5b75740 10 client.31202 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.601+0000 7fe9a5b75740 1 -- 172.21.15.18:0/4044564290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bcc3936910 con 0x55bcc392a8a0 2024-04-16T17:26:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.601+0000 7fe986000640 1 -- 172.21.15.18:0/4044564290 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c0be9b0 con 0x55bcc392a8a0 2024-04-16T17:26:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.602+0000 7fe986000640 1 -- 172.21.15.18:0/4044564290 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fe98c0f21c0 con 0x55bcc392a8a0 2024-04-16T17:26:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.602+0000 7fe9a5b75740 10 client.31202 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.602+0000 7fe9a5b75740 10 client.31202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.602+0000 7fe9a5b75740 20 client.31202 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.602+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bcc394c3d0 0x55bcc394e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.602+0000 7fe9a5b75740 4 client.31202 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.602+0000 7fe9a5b75740 1 -- 172.21.15.18:0/4044564290 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcc389d1a0 con 0x55bcc394c3d0 2024-04-16T17:26:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.602+0000 7fe99b400640 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bcc394c3d0 0x55bcc394e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.606+0000 7fe99b400640 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bcc394c3d0 0x55bcc394e7c0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc47fcd143311761 server_cookie=1ac73db6b22cd905 in_seq=0 out_seq=0 2024-04-16T17:26:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.606+0000 7fe986000640 10 client.31202 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.611+0000 7fe986000640 1 -- 172.21.15.18:0/4044564290 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcc389d1a0 con 0x55bcc394c3d0 2024-04-16T17:26:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.611+0000 7fe986000640 10 client.31202 handle_command_reply: tid=0 2024-04-16T17:26:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.611+0000 7fe9a5b75740 1 client.31202 shutdown 2024-04-16T17:26:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.613+0000 7fe9a5b75740 1 -- 172.21.15.18:0/4044564290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 msgr2=0x55bcc3930cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.613+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc3930cc0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0be470 tx=0x7fe98c02cf90 comp rx=0 tx=0).stop 2024-04-16T17:26:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.614+0000 7fe9a5b75740 1 -- 172.21.15.18:0/4044564290 shutdown_connections 2024-04-16T17:26:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.614+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bcc394c3d0 0x55bcc394e7c0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.614+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc3933e20 0x55bcc3931740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.614+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc3931200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.614+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/4044564290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc392a8a0 0x55bcc3930cc0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.614+0000 7fe986000640 0 client.31202 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.614+0000 7fe9a5b75740 1 -- 172.21.15.18:0/4044564290 >> 172.21.15.18:0/4044564290 conn(0x55bcc38c55d0 msgr2=0x55bcc39272e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.614+0000 7fe9a5b75740 1 -- 172.21.15.18:0/4044564290 shutdown_connections 2024-04-16T17:26:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.616+0000 7fe9a5b75740 1 -- 172.21.15.18:0/4044564290 wait complete. 2024-04-16T17:26:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.616+0000 7fe9a5b75740 20 client.31202 trim_cache size 0 max 16384 2024-04-16T17:26:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.648+0000 7fe9a5b75740 1 Processor -- start 2024-04-16T17:26:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.649+0000 7fe9a5b75740 1 -- start start 2024-04-16T17:26:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.649+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392a8a0 0x55bcc39333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.649+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc39540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.649+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc3954610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.649+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1afd20 con 0x55bcc392a8a0 2024-04-16T17:26:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.649+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1ad5f0 con 0x55bcc3933e20 2024-04-16T17:26:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.649+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1ab4a0 con 0x55bcc392b610 2024-04-16T17:26:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.650+0000 7fe9a0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc3954610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.651+0000 7fe9a0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc3954610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32874/0 (socket says 172.21.15.18:32874) 2024-04-16T17:26:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.651+0000 7fe9a0e00640 1 -- 172.21.15.18:0/894882083 learned_addr learned my addr 172.21.15.18:0/894882083 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.651+0000 7fe99b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc39540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.651+0000 7fe99be00640 1 --2- 172.21.15.18:0/894882083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392a8a0 0x55bcc39333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.652 INFO:tasks.workunit.client.0.smithi018.stdout: 10 111934 10.41 MB/sec execute 414 sec latency 199.046 ms 2024-04-16T17:26:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.652+0000 7fe9a0e00640 1 -- 172.21.15.18:0/894882083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 msgr2=0x55bcc39540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.652+0000 7fe9a0e00640 1 --2- 172.21.15.18:0/894882083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc39540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.652+0000 7fe9a0e00640 1 -- 172.21.15.18:0/894882083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392a8a0 msgr2=0x55bcc39333f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.652+0000 7fe9a0e00640 1 --2- 172.21.15.18:0/894882083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392a8a0 0x55bcc39333f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.652+0000 7fe9a0e00640 1 -- 172.21.15.18:0/894882083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc394ee40 con 0x55bcc3933e20 2024-04-16T17:26:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.653+0000 7fe9a0e00640 1 --2- 172.21.15.18:0/894882083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc3954610 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe994008da0 tx=0x7fe994010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b75dff6e1ae3e913 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.653+0000 7fe990c00640 1 -- 172.21.15.18:0/894882083 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994003b90 con 0x55bcc3933e20 2024-04-16T17:26:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.654+0000 7fe990c00640 1 -- 172.21.15.18:0/894882083 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994003d30 con 0x55bcc3933e20 2024-04-16T17:26:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.654+0000 7fe990c00640 1 -- 172.21.15.18:0/894882083 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99400bc10 con 0x55bcc3933e20 2024-04-16T17:26:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.655+0000 7fe9a5b75740 1 -- 172.21.15.18:0/894882083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 msgr2=0x55bcc3954610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.655+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/894882083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc3954610 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe994008da0 tx=0x7fe994010800 comp rx=0 tx=0).stop 2024-04-16T17:26:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.655+0000 7fe9a5b75740 1 -- 172.21.15.18:0/894882083 shutdown_connections 2024-04-16T17:26:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.655+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/894882083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc3954610 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.655+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/894882083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392b610 0x55bcc39540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.655+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/894882083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392a8a0 0x55bcc39333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.655+0000 7fe9a5b75740 1 -- 172.21.15.18:0/894882083 >> 172.21.15.18:0/894882083 conn(0x55bcc38c55d0 msgr2=0x55bcc39246b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.656+0000 7fe9a5b75740 1 -- 172.21.15.18:0/894882083 shutdown_connections 2024-04-16T17:26:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.656+0000 7fe9a5b75740 1 -- 172.21.15.18:0/894882083 wait complete. 2024-04-16T17:26:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.657+0000 7fe9a5b75740 1 -- start start 2024-04-16T17:26:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.657+0000 7fe9a5b75740 1 Processor -- start 2024-04-16T17:26:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.658+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392a8a0 0x55bcc393ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.658+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392b610 0x55bcc393d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.658+0000 7fe9a5b75740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc393f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe99be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392a8a0 0x55bcc393ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe99be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392a8a0 0x55bcc393ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:51542/0 (socket says 172.21.15.18:51542) 2024-04-16T17:26:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe99b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392b610 0x55bcc393d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1abaf0 con 0x55bcc392b610 2024-04-16T17:26:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1acf20 con 0x55bcc3933e20 2024-04-16T17:26:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe9a5b75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1ac180 con 0x55bcc392a8a0 2024-04-16T17:26:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe9a5b75740 10 client.0 _ll_register_callbacks cb 0x7fe994008510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe9a0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc393f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe9a0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc393f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:32880/0 (socket says 172.21.15.18:32880) 2024-04-16T17:26:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.659+0000 7fe9a0e00640 1 -- 172.21.15.18:0/778549189 learned_addr learned my addr 172.21.15.18:0/778549189 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.661+0000 7fe99be00640 1 -- 172.21.15.18:0/778549189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 msgr2=0x55bcc393f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.661+0000 7fe99be00640 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc393f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.661+0000 7fe99be00640 1 -- 172.21.15.18:0/778549189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392b610 msgr2=0x55bcc393d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.661+0000 7fe99be00640 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392b610 0x55bcc393d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.661+0000 7fe99be00640 1 -- 172.21.15.18:0/778549189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc3929b70 con 0x55bcc392a8a0 2024-04-16T17:26:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.661+0000 7fe9a0e00640 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc393f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.662+0000 7fe99be00640 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392a8a0 0x55bcc393ace0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0be240 tx=0x7fe98c02bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a742b5d97c5fff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.662+0000 7fe986000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.662+0000 7fe986000640 1 -- 172.21.15.18:0/778549189 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c04b070 con 0x55bcc392a8a0 2024-04-16T17:26:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.663+0000 7fe986000640 1 -- 172.21.15.18:0/778549189 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9538+0+0 (secure 0 0 0) 0x7fe98c0bd910 con 0x55bcc392a8a0 2024-04-16T17:26:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.664+0000 7fe9a5b75740 1 -- 172.21.15.18:0/778549189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe994003d30 con 0x55bcc392a8a0 2024-04-16T17:26:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.665+0000 7fe986000640 1 -- 172.21.15.18:0/778549189 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe98c0bc3f0 con 0x55bcc392a8a0 2024-04-16T17:26:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.666+0000 7fe9a5b75740 10 client.31051 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.666+0000 7fe9a5b75740 1 -- 172.21.15.18:0/778549189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bcc3929e60 con 0x55bcc392a8a0 2024-04-16T17:26:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.667+0000 7fe986000640 1 -- 172.21.15.18:0/778549189 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994003d30 con 0x55bcc392a8a0 2024-04-16T17:26:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.667+0000 7fe986000640 1 -- 172.21.15.18:0/778549189 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fe98c0bd170 con 0x55bcc392a8a0 2024-04-16T17:26:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.667+0000 7fe9a5b75740 10 client.31051 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.667+0000 7fe9a5b75740 10 client.31051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.667+0000 7fe9a5b75740 20 client.31051 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.667+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bcc3941400 0x55bcc39437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.667+0000 7fe9a5b75740 4 client.31051 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:26:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.667+0000 7fe9a5b75740 1 -- 172.21.15.18:0/778549189 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcc3897010 con 0x55bcc3941400 2024-04-16T17:26:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.668+0000 7fe99b400640 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bcc3941400 0x55bcc39437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.671+0000 7fe99b400640 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bcc3941400 0x55bcc39437f0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8eaae5b6b12fa00 server_cookie=d2c2f5d23028ed61 in_seq=0 out_seq=0 2024-04-16T17:26:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:42.671+0000 7fe986000640 10 client.31051 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:43.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:39 smithi018 ceph-mon[31776]: osdmap e136: 12 total, 12 up, 12 in 2024-04-16T17:26:43.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:39 smithi018 ceph-mon[31776]: pgmap v702: 129 pgs: 129 active+clean; 450 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2024-04-16T17:26:43.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:40 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:26:43.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:41 smithi018 ceph-mon[31776]: pgmap v703: 129 pgs: 129 active+clean; 462 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-04-16T17:26:43.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:42 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:26:43.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:42 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:26:43.016 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:42 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:26:43.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:42 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:26:43.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:42 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:26:43.175 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:42 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:26:43.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:42 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:26:43.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:42 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:26:43.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:42 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:26:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.459+0000 7fe986000640 1 -- 172.21.15.18:0/778549189 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcc3897010 con 0x55bcc3941400 2024-04-16T17:26:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.460+0000 7fe986000640 10 client.31051 handle_command_reply: tid=0 2024-04-16T17:26:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.460+0000 7fe9a5b75740 1 client.31051 shutdown 2024-04-16T17:26:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.460+0000 7fe9a5b75740 1 -- 172.21.15.18:0/778549189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392a8a0 msgr2=0x55bcc393ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.460+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392a8a0 0x55bcc393ace0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0be240 tx=0x7fe98c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:26:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.460+0000 7fe9a5b75740 1 -- 172.21.15.18:0/778549189 shutdown_connections 2024-04-16T17:26:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.460+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55bcc3941400 0x55bcc39437f0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.461+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcc3933e20 0x55bcc393f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.461+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bcc392b610 0x55bcc393d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.461+0000 7fe9a5b75740 1 --2- 172.21.15.18:0/778549189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bcc392a8a0 0x55bcc393ace0 secure :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0be240 tx=0x7fe98c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:26:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.461+0000 7fe9a5b75740 1 -- 172.21.15.18:0/778549189 >> 172.21.15.18:0/778549189 conn(0x55bcc38c55d0 msgr2=0x55bcc3930c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.461+0000 7fe986000640 0 client.31051 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.462+0000 7fe9a5b75740 1 -- 172.21.15.18:0/778549189 shutdown_connections 2024-04-16T17:26:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.462+0000 7fe9a5b75740 1 -- 172.21.15.18:0/778549189 wait complete. 2024-04-16T17:26:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.463+0000 7fe9a5b75740 20 client.31051 trim_cache size 0 max 16384 2024-04-16T17:26:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:26:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:26:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:26:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:26:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.466+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe9800a3ea0 msgr2=0x7fe9800a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.466+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe9800a3ea0 0x7fe9800a6360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe99c110d60 tx=0x7fe988005f90 comp rx=0 tx=0).stop 2024-04-16T17:26:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.466+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2549675178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c100490 msgr2=0x7fe99c1a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.466+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c100490 0x7fe99c1a06f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe98c02c250 tx=0x7fe98c0043b0 comp rx=0 tx=0).stop 2024-04-16T17:26:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.467+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2549675178 shutdown_connections 2024-04-16T17:26:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.467+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe9800a3ea0 0x7fe9800a6360 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.467+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c10e9d0 0x7fe99c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.467+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe99c100db0 0x7fe99c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.467+0000 7fe9a2e00640 1 --2- 172.21.15.18:0/2549675178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c100490 0x7fe99c1a06f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.467+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2549675178 >> 172.21.15.18:0/2549675178 conn(0x7fe99c075e10 msgr2=0x7fe99c10eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.468+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2549675178 shutdown_connections 2024-04-16T17:26:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:43.468+0000 7fe9a2e00640 1 -- 172.21.15.18:0/2549675178 wait complete. 2024-04-16T17:26:43.652 INFO:tasks.workunit.client.0.smithi018.stdout: 10 112104 10.41 MB/sec execute 415 sec latency 562.299 ms 2024-04-16T17:26:44.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:43 smithi022 ceph-mon[36023]: pgmap v704: 129 pgs: 129 active+clean; 476 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 315 op/s 2024-04-16T17:26:44.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:43 smithi043 ceph-mon[36901]: pgmap v704: 129 pgs: 129 active+clean; 476 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 315 op/s 2024-04-16T17:26:44.652 INFO:tasks.workunit.client.0.smithi018.stdout: 10 112321 10.41 MB/sec execute 416 sec latency 166.251 ms 2024-04-16T17:26:45.652 INFO:tasks.workunit.client.0.smithi018.stdout: 10 112573 10.42 MB/sec execute 417 sec latency 196.572 ms 2024-04-16T17:26:46.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:45 smithi043 ceph-mon[36901]: pgmap v705: 129 pgs: 129 active+clean; 470 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-04-16T17:26:46.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:45 smithi022 ceph-mon[36023]: pgmap v705: 129 pgs: 129 active+clean; 470 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-04-16T17:26:46.652 INFO:tasks.workunit.client.0.smithi018.stdout: 10 112733 10.40 MB/sec execute 418 sec latency 308.954 ms 2024-04-16T17:26:47.652 INFO:tasks.workunit.client.0.smithi018.stdout: 10 112988 10.41 MB/sec execute 419 sec latency 155.591 ms 2024-04-16T17:26:48.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:47 smithi043 ceph-mon[36901]: pgmap v706: 129 pgs: 129 active+clean; 458 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 390 op/s 2024-04-16T17:26:48.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:47 smithi022 ceph-mon[36023]: pgmap v706: 129 pgs: 129 active+clean; 458 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 390 op/s 2024-04-16T17:26:48.653 INFO:tasks.workunit.client.0.smithi018.stdout: 10 113277 10.42 MB/sec execute 420 sec latency 155.713 ms 2024-04-16T17:26:49.654 INFO:tasks.workunit.client.0.smithi018.stdout: 10 113498 10.43 MB/sec execute 421 sec latency 233.635 ms 2024-04-16T17:26:50.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:49 smithi043 ceph-mon[36901]: pgmap v707: 129 pgs: 129 active+clean; 458 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 387 op/s 2024-04-16T17:26:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:49 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:26:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:49 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:26:50.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:49 smithi043 ceph-mon[36901]: scrub summary: idle 2024-04-16T17:26:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:49 smithi022 ceph-mon[36023]: pgmap v707: 129 pgs: 129 active+clean; 458 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 387 op/s 2024-04-16T17:26:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:49 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:26:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:49 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:26:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:49 smithi022 ceph-mon[36023]: scrub summary: idle 2024-04-16T17:26:50.653 INFO:tasks.workunit.client.0.smithi018.stdout: 10 113799 10.43 MB/sec execute 422 sec latency 134.128 ms 2024-04-16T17:26:51.653 INFO:tasks.workunit.client.0.smithi018.stdout: 10 114046 10.44 MB/sec execute 423 sec latency 177.976 ms 2024-04-16T17:26:52.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:51 smithi043 ceph-mon[36901]: pgmap v708: 129 pgs: 129 active+clean; 457 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.7 MiB/s wr, 368 op/s 2024-04-16T17:26:52.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:51 smithi022 ceph-mon[36023]: pgmap v708: 129 pgs: 129 active+clean; 457 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.7 MiB/s wr, 368 op/s 2024-04-16T17:26:52.654 INFO:tasks.workunit.client.0.smithi018.stdout: 10 114315 10.45 MB/sec execute 424 sec latency 169.920 ms 2024-04-16T17:26:53.654 INFO:tasks.workunit.client.0.smithi018.stdout: 10 114420 10.44 MB/sec execute 425 sec latency 578.090 ms 2024-04-16T17:26:54.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:43 smithi018 ceph-mon[31776]: pgmap v704: 129 pgs: 129 active+clean; 476 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 315 op/s 2024-04-16T17:26:54.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:45 smithi018 ceph-mon[31776]: pgmap v705: 129 pgs: 129 active+clean; 470 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-04-16T17:26:54.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:54 smithi043 ceph-mon[36901]: pgmap v709: 129 pgs: 129 active+clean; 461 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 362 op/s 2024-04-16T17:26:54.322 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:47 smithi018 ceph-mon[31776]: pgmap v706: 129 pgs: 129 active+clean; 458 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 390 op/s 2024-04-16T17:26:54.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:49 smithi018 ceph-mon[31776]: pgmap v707: 129 pgs: 129 active+clean; 458 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 387 op/s 2024-04-16T17:26:54.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:49 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:26:54.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:49 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:26:54.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:49 smithi018 ceph-mon[31776]: scrub summary: idle 2024-04-16T17:26:54.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:51 smithi018 ceph-mon[31776]: pgmap v708: 129 pgs: 129 active+clean; 457 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.7 MiB/s wr, 368 op/s 2024-04-16T17:26:54.323 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:54 smithi018 ceph-mon[31776]: pgmap v709: 129 pgs: 129 active+clean; 461 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 362 op/s 2024-04-16T17:26:54.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:54 smithi022 ceph-mon[36023]: pgmap v709: 129 pgs: 129 active+clean; 461 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 362 op/s 2024-04-16T17:26:54.657 INFO:tasks.workunit.client.0.smithi018.stdout: 10 114585 10.44 MB/sec execute 426 sec latency 427.848 ms 2024-04-16T17:26:55.655 INFO:tasks.workunit.client.0.smithi018.stdout: 10 114946 10.48 MB/sec execute 427 sec latency 166.643 ms 2024-04-16T17:26:56.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:56 smithi022 ceph-mon[36023]: pgmap v710: 129 pgs: 129 active+clean; 462 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2024-04-16T17:26:56.489 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:56 smithi018 ceph-mon[31776]: pgmap v710: 129 pgs: 129 active+clean; 462 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2024-04-16T17:26:56.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:56 smithi043 ceph-mon[36901]: pgmap v710: 129 pgs: 129 active+clean; 462 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2024-04-16T17:26:56.656 INFO:tasks.workunit.client.0.smithi018.stdout: 10 115252 10.50 MB/sec execute 428 sec latency 213.985 ms 2024-04-16T17:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.685+0000 7fed6f600640 1 -- 172.21.15.18:0/1568602255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed68109000 msgr2=0x7fed68109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.685+0000 7fed6f600640 1 --2- 172.21.15.18:0/1568602255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed68109000 0x7fed68109480 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fed60009ca0 tx=0x7fed6002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.685+0000 7fed6f600640 1 -- 172.21.15.18:0/1568602255 shutdown_connections 2024-04-16T17:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.685+0000 7fed6f600640 1 --2- 172.21.15.18:0/1568602255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed6806dca0 0x7fed681099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.685+0000 7fed6f600640 1 --2- 172.21.15.18:0/1568602255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed68109000 0x7fed68109480 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.685+0000 7fed6f600640 1 --2- 172.21.15.18:0/1568602255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed6810da60 0x7fed6810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.685+0000 7fed6f600640 1 -- 172.21.15.18:0/1568602255 >> 172.21.15.18:0/1568602255 conn(0x7fed6806d0e0 msgr2=0x7fed6806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.686+0000 7fed6f600640 1 -- 172.21.15.18:0/1568602255 shutdown_connections 2024-04-16T17:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.686+0000 7fed6f600640 1 -- 172.21.15.18:0/1568602255 wait complete. 2024-04-16T17:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.688+0000 7fed6f600640 1 Processor -- start 2024-04-16T17:26:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.689+0000 7fed6f600640 1 -- start start 2024-04-16T17:26:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.689+0000 7fed6f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6806dca0 0x7fed681a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.689+0000 7fed6f600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed68109000 0x7fed68112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.689+0000 7fed6f600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed6810da60 0x7fed68112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.689+0000 7fed6f600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed6811ac10 con 0x7fed68109000 2024-04-16T17:26:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.689+0000 7fed6f600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed6811aa90 con 0x7fed6810da60 2024-04-16T17:26:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.689+0000 7fed6f600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed6811ad90 con 0x7fed6806dca0 2024-04-16T17:26:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.694+0000 7fed6cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6806dca0 0x7fed681a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.695+0000 7fed6cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6806dca0 0x7fed681a3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54688/0 (socket says 172.21.15.18:54688) 2024-04-16T17:26:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.695+0000 7fed6cc00640 1 -- 172.21.15.18:0/1613831258 learned_addr learned my addr 172.21.15.18:0/1613831258 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.695+0000 7fed67e00640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed68109000 0x7fed68112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.695+0000 7fed6d600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed6810da60 0x7fed68112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.695+0000 7fed6d600640 1 -- 172.21.15.18:0/1613831258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6806dca0 msgr2=0x7fed681a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.695+0000 7fed6d600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6806dca0 0x7fed681a3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.695+0000 7fed6d600640 1 -- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed68109000 msgr2=0x7fed68112650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.695+0000 7fed6d600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed68109000 0x7fed68112650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.696+0000 7fed6d600640 1 -- 172.21.15.18:0/1613831258 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed68113450 con 0x7fed6810da60 2024-04-16T17:26:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.696+0000 7fed6d600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed6810da60 0x7fed68112b90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fed5800e940 tx=0x7fed5800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.697+0000 7fed65600640 1 -- 172.21.15.18:0/1613831258 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5800ccc0 con 0x7fed6810da60 2024-04-16T17:26:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.697+0000 7fed65600640 1 -- 172.21.15.18:0/1613831258 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed580035f0 con 0x7fed6810da60 2024-04-16T17:26:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.697+0000 7fed65600640 1 -- 172.21.15.18:0/1613831258 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5800ce60 con 0x7fed6810da60 2024-04-16T17:26:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.697+0000 7fed6f600640 1 -- 172.21.15.18:0/1613831258 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed681b3d40 con 0x7fed6810da60 2024-04-16T17:26:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.697+0000 7fed6f600640 1 -- 172.21.15.18:0/1613831258 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed681b41b0 con 0x7fed6810da60 2024-04-16T17:26:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.704+0000 7fed65600640 1 -- 172.21.15.18:0/1613831258 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed580037a0 con 0x7fed6810da60 2024-04-16T17:26:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.704+0000 7fed7241c740 1 Processor -- start 2024-04-16T17:26:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.705+0000 7fed7241c740 1 -- start start 2024-04-16T17:26:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.706+0000 7fed7241c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.706+0000 7fed65600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed4c0a3d90 0x7fed4c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.706+0000 7fed65600640 1 -- 172.21.15.18:0/1613831258 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9538+0+0 (secure 0 0 0) 0x7fed580d0e40 con 0x7fed6810da60 2024-04-16T17:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4ef360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54702/0 (socket says 172.21.15.18:54702) 2024-04-16T17:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed67e00640 1 -- 172.21.15.18:0/664221317 learned_addr learned my addr 172.21.15.18:0/664221317 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed7241c740 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4ef8a0 0x55f3af4efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed7241c740 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f8e20 0x55f3af4fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed7241c740 1 -- 172.21.15.18:0/664221317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f3af454780 con 0x55f3af4f8e20 2024-04-16T17:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed7241c740 1 -- 172.21.15.18:0/664221317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f3af3faf40 con 0x55f3af4ef8a0 2024-04-16T17:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed7241c740 1 -- 172.21.15.18:0/664221317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f3af450740 con 0x55f3af4f0610 2024-04-16T17:26:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.707+0000 7fed6cc00640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed4c0a3d90 0x7fed4c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.708+0000 7fed67e00640 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4ef8a0 0x55f3af4efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.708+0000 7fed6d600640 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f8e20 0x55f3af4fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.709+0000 7fed67e00640 1 -- 172.21.15.18:0/664221317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4ef8a0 msgr2=0x55f3af4efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.709+0000 7fed67e00640 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4ef8a0 0x55f3af4efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.709+0000 7fed67e00640 1 -- 172.21.15.18:0/664221317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f8e20 msgr2=0x55f3af4fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.709+0000 7fed67e00640 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f8e20 0x55f3af4fb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.709+0000 7fed67e00640 1 -- 172.21.15.18:0/664221317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3af167210 con 0x55f3af4f0610 2024-04-16T17:26:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.709+0000 7fed6d600640 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f8e20 0x55f3af4fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:26:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.710+0000 7fed6cc00640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed4c0a3d90 0x7fed4c0a6250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed6810f410 tx=0x7fed540091c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.710+0000 7fed67e00640 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4ef360 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fed60009430 tx=0x7fed60032440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=562ae9365dbfac3d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.711+0000 7fed5dc00640 1 -- 172.21.15.18:0/664221317 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60039040 con 0x55f3af4f0610 2024-04-16T17:26:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.711+0000 7fed5dc00640 1 -- 172.21.15.18:0/664221317 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60003970 con 0x55f3af4f0610 2024-04-16T17:26:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.711+0000 7fed5dc00640 1 -- 172.21.15.18:0/664221317 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60004390 con 0x55f3af4f0610 2024-04-16T17:26:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.712+0000 7fed7241c740 1 -- 172.21.15.18:0/664221317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 msgr2=0x55f3af4ef360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.712+0000 7fed7241c740 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4ef360 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fed60009430 tx=0x7fed60032440 comp rx=0 tx=0).stop 2024-04-16T17:26:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.714+0000 7fed7241c740 1 -- 172.21.15.18:0/664221317 shutdown_connections 2024-04-16T17:26:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.714+0000 7fed7241c740 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f8e20 0x55f3af4fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.714+0000 7fed7241c740 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4ef8a0 0x55f3af4efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.714+0000 7fed7241c740 1 --2- 172.21.15.18:0/664221317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4ef360 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.714+0000 7fed7241c740 1 -- 172.21.15.18:0/664221317 >> 172.21.15.18:0/664221317 conn(0x55f3af48a5d0 msgr2=0x55f3af4ea590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.716+0000 7fed7241c740 1 -- 172.21.15.18:0/664221317 shutdown_connections 2024-04-16T17:26:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.716+0000 7fed7241c740 1 -- 172.21.15.18:0/664221317 wait complete. 2024-04-16T17:26:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.717+0000 7fed7241c740 1 -- start start 2024-04-16T17:26:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.717+0000 7fed7241c740 1 Processor -- start 2024-04-16T17:26:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.718+0000 7fed7241c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4ef8a0 0x55f3af4f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.719+0000 7fed7241c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f0610 0x55f3af4f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.719+0000 7fed7241c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4f6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed6d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed6d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:36158/0 (socket says 172.21.15.18:36158) 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed6d600640 1 -- 172.21.15.18:0/993356552 learned_addr learned my addr 172.21.15.18:0/993356552 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f3af4fbaf0 con 0x55f3af4f0610 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f3af40e390 con 0x55f3af4f8e20 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f3af4fbc70 con 0x55f3af4ef8a0 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed67e00640 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f0610 0x55f3af4f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed6d600640 1 -- 172.21.15.18:0/993356552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4ef8a0 msgr2=0x55f3af4f5d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed6d600640 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4ef8a0 0x55f3af4f5d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed6d600640 1 -- 172.21.15.18:0/993356552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f0610 msgr2=0x55f3af4f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed6d600640 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f0610 0x55f3af4f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.720+0000 7fed6d600640 1 -- 172.21.15.18:0/993356552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3af233620 con 0x55f3af4f8e20 2024-04-16T17:26:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.721+0000 7fed7241c740 10 client.0 _ll_register_callbacks cb 0x55f3af3f7f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.721+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed60003970 con 0x55f3af4f8e20 2024-04-16T17:26:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.721+0000 7fed52000640 10 client.31220 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:26:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.721+0000 7fed6d600640 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4f6780 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fed58098e40 tx=0x7fed58003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f4176e888e065e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.722+0000 7fed52000640 1 -- 172.21.15.18:0/993356552 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed580a53c0 con 0x55f3af4f8e20 2024-04-16T17:26:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.722+0000 7fed52000640 1 -- 172.21.15.18:0/993356552 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9538+0+0 (secure 0 0 0) 0x7fed580a51c0 con 0x55f3af4f8e20 2024-04-16T17:26:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.722+0000 7fed52000640 1 -- 172.21.15.18:0/993356552 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fed580a53c0 con 0x55f3af4f8e20 2024-04-16T17:26:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.723+0000 7fed7241c740 10 client.31220 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.723+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55f3af507c10 con 0x55f3af4f8e20 2024-04-16T17:26:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.723+0000 7fed52000640 1 -- 172.21.15.18:0/993356552 <== mon.2 v2:172.21.15.22:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fed580ba020 con 0x55f3af4f8e20 2024-04-16T17:26:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.723+0000 7fed7241c740 10 client.31220 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.723+0000 7fed7241c740 10 client.31220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.723+0000 7fed7241c740 20 client.31220 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.724+0000 7fed7241c740 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3af508fc0 0x55f3af50b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.724+0000 7fed7241c740 4 client.31220 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:26:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.724+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3af4621a0 con 0x55f3af508fc0 2024-04-16T17:26:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.724+0000 7fed6cc00640 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3af508fc0 0x55f3af50b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.725+0000 7fed6cc00640 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3af508fc0 0x55f3af50b3d0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6bb4616e7037466 server_cookie=a0607ef33cddbcf0 in_seq=0 out_seq=0 2024-04-16T17:26:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.725+0000 7fed52000640 10 client.31220 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.726+0000 7fed52000640 1 -- 172.21.15.18:0/993356552 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3af4621a0 con 0x55f3af508fc0 2024-04-16T17:26:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.727+0000 7fed52000640 10 client.31220 handle_command_reply: tid=0 2024-04-16T17:26:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.727+0000 7fed7241c740 1 client.31220 shutdown 2024-04-16T17:26:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.728+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 msgr2=0x55f3af4f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.728+0000 7fed7241c740 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4f6780 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fed58098e40 tx=0x7fed58003bd0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.728+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 shutdown_connections 2024-04-16T17:26:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.728+0000 7fed7241c740 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3af508fc0 0x55f3af50b3d0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.728+0000 7fed7241c740 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4f6780 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.728+0000 7fed7241c740 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4f0610 0x55f3af4f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.728+0000 7fed7241c740 1 --2- 172.21.15.18:0/993356552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4ef8a0 0x55f3af4f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.729+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 >> 172.21.15.18:0/993356552 conn(0x55f3af48a5d0 msgr2=0x55f3af4ec2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.729+0000 7fed52000640 0 client.31220 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.729+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 shutdown_connections 2024-04-16T17:26:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.729+0000 7fed7241c740 1 -- 172.21.15.18:0/993356552 wait complete. 2024-04-16T17:26:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.729+0000 7fed7241c740 20 client.31220 trim_cache size 0 max 16384 2024-04-16T17:26:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.757+0000 7fed7241c740 1 Processor -- start 2024-04-16T17:26:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.758+0000 7fed7241c740 1 -- start start 2024-04-16T17:26:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.758+0000 7fed7241c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.758+0000 7fed7241c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.758+0000 7fed7241c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.758+0000 7fed7241c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed681c0d00 con 0x55f3af4ef8a0 2024-04-16T17:26:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.758+0000 7fed7241c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed681be5d0 con 0x55f3af4f8e20 2024-04-16T17:26:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.758+0000 7fed7241c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed681bc440 con 0x55f3af4f0610 2024-04-16T17:26:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.759+0000 7fed67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.759+0000 7fed67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54716/0 (socket says 172.21.15.18:54716) 2024-04-16T17:26:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.759+0000 7fed67e00640 1 -- 172.21.15.18:0/2184627895 learned_addr learned my addr 172.21.15.18:0/2184627895 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.759+0000 7fed6d600640 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.760+0000 7fed67e00640 1 -- 172.21.15.18:0/2184627895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 msgr2=0x55f3af4e9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.760+0000 7fed6cc00640 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.761+0000 7fed67e00640 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4e9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.761+0000 7fed67e00640 1 -- 172.21.15.18:0/2184627895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 msgr2=0x55f3af5049f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.761+0000 7fed67e00640 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5049f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.761+0000 7fed67e00640 1 -- 172.21.15.18:0/2184627895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3af4f4250 con 0x55f3af4f0610 2024-04-16T17:26:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.761+0000 7fed6d600640 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4e9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.761+0000 7fed6cc00640 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:26:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.762+0000 7fed67e00640 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4f65a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fed6002c700 tx=0x7fed60032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=145151727c2f7917 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.763+0000 7fed5d200640 1 -- 172.21.15.18:0/2184627895 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60051070 con 0x55f3af4f0610 2024-04-16T17:26:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.763+0000 7fed5d200640 1 -- 172.21.15.18:0/2184627895 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6004c460 con 0x55f3af4f0610 2024-04-16T17:26:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.763+0000 7fed5d200640 1 -- 172.21.15.18:0/2184627895 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60039040 con 0x55f3af4f0610 2024-04-16T17:26:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.764+0000 7fed7241c740 1 -- 172.21.15.18:0/2184627895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 msgr2=0x55f3af4f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.764+0000 7fed7241c740 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4f65a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fed6002c700 tx=0x7fed60032340 comp rx=0 tx=0).stop 2024-04-16T17:26:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.764+0000 7fed7241c740 1 -- 172.21.15.18:0/2184627895 shutdown_connections 2024-04-16T17:26:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.764+0000 7fed7241c740 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4e9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.764+0000 7fed7241c740 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4f65a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.764+0000 7fed7241c740 1 --2- 172.21.15.18:0/2184627895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.764+0000 7fed7241c740 1 -- 172.21.15.18:0/2184627895 >> 172.21.15.18:0/2184627895 conn(0x55f3af48a5d0 msgr2=0x55f3af4e8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.766+0000 7fed7241c740 1 -- 172.21.15.18:0/2184627895 shutdown_connections 2024-04-16T17:26:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.766+0000 7fed7241c740 1 -- 172.21.15.18:0/2184627895 wait complete. 2024-04-16T17:26:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.766+0000 7fed7241c740 1 -- start start 2024-04-16T17:26:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.766+0000 7fed7241c740 1 Processor -- start 2024-04-16T17:26:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.768+0000 7fed7241c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.768+0000 7fed7241c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4fc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.768+0000 7fed7241c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed7241c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed681bca90 con 0x55f3af4ef8a0 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed7241c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed681bdf00 con 0x55f3af4f8e20 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed7241c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed681bd160 con 0x55f3af4f0610 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed7241c740 10 client.0 _ll_register_callbacks cb 0x7fed5809c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed6cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4fc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:54728/0 (socket says 172.21.15.18:54728) 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed67e00640 1 -- 172.21.15.18:0/2287156238 learned_addr learned my addr 172.21.15.18:0/2287156238 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:26:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed6d600640 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.769+0000 7fed67e00640 1 -- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 msgr2=0x55f3af4fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.772+0000 7fed67e00640 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.772+0000 7fed67e00640 1 -- 172.21.15.18:0/2287156238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 msgr2=0x55f3af5013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:26:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.772+0000 7fed67e00640 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.772+0000 7fed67e00640 1 -- 172.21.15.18:0/2287156238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3af4fd540 con 0x55f3af4f0610 2024-04-16T17:26:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.773+0000 7fed67e00640 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4fc730 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fed60009ca0 tx=0x7fed6002c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a008702e13daa8a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:26:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.773+0000 7fed52000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:26:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.773+0000 7fed52000640 1 -- 172.21.15.18:0/2287156238 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60051070 con 0x55f3af4f0610 2024-04-16T17:26:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.773+0000 7fed7241c740 1 -- 172.21.15.18:0/2287156238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed6004c460 con 0x55f3af4f0610 2024-04-16T17:26:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.774+0000 7fed52000640 1 -- 172.21.15.18:0/2287156238 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9538+0+0 (secure 0 0 0) 0x7fed600327a0 con 0x55f3af4f0610 2024-04-16T17:26:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.774+0000 7fed52000640 1 -- 172.21.15.18:0/2287156238 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fed60051070 con 0x55f3af4f0610 2024-04-16T17:26:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.774+0000 7fed7241c740 10 client.31066 fetch_fsmap learned FSMap version 37 2024-04-16T17:26:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.774+0000 7fed7241c740 1 -- 172.21.15.18:0/2287156238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f3af56f5c0 con 0x55f3af4f0610 2024-04-16T17:26:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed6d600640 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:26:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.772+0000 7fed6cc00640 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:26:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed52000640 1 -- 172.21.15.18:0/2287156238 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6004c460 con 0x55f3af4f0610 2024-04-16T17:26:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed52000640 1 -- 172.21.15.18:0/2287156238 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fed600348c0 con 0x55f3af4f0610 2024-04-16T17:26:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed7241c740 10 client.31066 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:26:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed7241c740 10 client.31066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:26:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed7241c740 20 client.31066 populate_metadata read hostname 'smithi018' 2024-04-16T17:26:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed7241c740 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3af56fa90 0x55f3af571e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:26:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed7241c740 4 client.31066 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:26:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.775+0000 7fed7241c740 1 -- 172.21.15.18:0/2287156238 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3af45c010 con 0x55f3af56fa90 2024-04-16T17:26:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.776+0000 7fed6cc00640 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3af56fa90 0x55f3af571e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:26:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.779+0000 7fed6cc00640 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3af56fa90 0x55f3af571e80 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94189aade64623c0 server_cookie=aa7b98ba06e40fd4 in_seq=0 out_seq=0 2024-04-16T17:26:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:56.779+0000 7fed52000640 10 client.31066 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.622+0000 7fed52000640 1 -- 172.21.15.18:0/2287156238 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3af45c010 con 0x55f3af56fa90 2024-04-16T17:26:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.622+0000 7fed52000640 10 client.31066 handle_command_reply: tid=0 2024-04-16T17:26:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.622+0000 7fed7241c740 1 client.31066 shutdown 2024-04-16T17:26:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.624+0000 7fed7241c740 1 -- 172.21.15.18:0/2287156238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 msgr2=0x55f3af4fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.624+0000 7fed7241c740 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4fc730 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fed60009ca0 tx=0x7fed6002c810 comp rx=0 tx=0).stop 2024-04-16T17:26:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.624+0000 7fed7241c740 1 -- 172.21.15.18:0/2287156238 shutdown_connections 2024-04-16T17:26:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.624+0000 7fed7241c740 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x55f3af56fa90 0x55f3af571e80 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.624+0000 7fed7241c740 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3af4f8e20 0x55f3af4fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.624+0000 7fed7241c740 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f3af4f0610 0x55f3af4fc730 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.624+0000 7fed7241c740 1 --2- 172.21.15.18:0/2287156238 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f3af4ef8a0 0x55f3af5013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.624+0000 7fed7241c740 1 -- 172.21.15.18:0/2287156238 >> 172.21.15.18:0/2287156238 conn(0x55f3af48a5d0 msgr2=0x55f3af4eb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.625+0000 7fed52000640 0 client.31066 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:26:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.625+0000 7fed7241c740 1 -- 172.21.15.18:0/2287156238 shutdown_connections 2024-04-16T17:26:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.625+0000 7fed7241c740 1 -- 172.21.15.18:0/2287156238 wait complete. 2024-04-16T17:26:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.625+0000 7fed7241c740 20 client.31066 trim_cache size 0 max 16384 2024-04-16T17:26:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:{ 2024-04-16T17:26:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "message": "", 2024-04-16T17:26:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout: "return_code": 0 2024-04-16T17:26:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stdout:} 2024-04-16T17:26:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 -- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed4c0a3d90 msgr2=0x7fed4c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed4c0a3d90 0x7fed4c0a6250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed6810f410 tx=0x7fed540091c0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 -- 172.21.15.18:0/1613831258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed6810da60 msgr2=0x7fed68112b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:26:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed6810da60 0x7fed68112b90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fed5800e940 tx=0x7fed5800ee00 comp rx=0 tx=0).stop 2024-04-16T17:26:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 -- 172.21.15.18:0/1613831258 shutdown_connections 2024-04-16T17:26:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fed4c0a3d90 0x7fed4c0a6250 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed6810da60 0x7fed68112b90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed68109000 0x7fed68112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 --2- 172.21.15.18:0/1613831258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6806dca0 0x7fed681a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:26:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.627+0000 7fed6f600640 1 -- 172.21.15.18:0/1613831258 >> 172.21.15.18:0/1613831258 conn(0x7fed6806d0e0 msgr2=0x7fed68070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:26:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.628+0000 7fed6f600640 1 -- 172.21.15.18:0/1613831258 shutdown_connections 2024-04-16T17:26:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi018.front.sepia.ceph.com.smithi018.stderr:2024-04-16T17:26:57.628+0000 7fed6f600640 1 -- 172.21.15.18:0/1613831258 wait complete. 2024-04-16T17:26:57.655 INFO:tasks.workunit.client.0.smithi018.stdout: 10 115349 10.49 MB/sec execute 429 sec latency 505.770 ms 2024-04-16T17:26:58.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:26:58 smithi022 ceph-mon[36023]: pgmap v711: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 402 op/s 2024-04-16T17:26:58.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:26:58 smithi043 ceph-mon[36901]: pgmap v711: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 402 op/s 2024-04-16T17:26:58.657 INFO:tasks.workunit.client.0.smithi018.stdout: 10 115551 10.49 MB/sec execute 430 sec latency 169.859 ms 2024-04-16T17:26:59.657 INFO:tasks.workunit.client.0.smithi018.stdout: 10 115667 10.48 MB/sec execute 431 sec latency 322.643 ms 2024-04-16T17:27:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:00 smithi022 ceph-mon[36023]: pgmap v712: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-04-16T17:27:00.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:00 smithi043 ceph-mon[36901]: pgmap v712: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-04-16T17:27:00.657 INFO:tasks.workunit.client.0.smithi018.stdout: 10 115862 10.47 MB/sec execute 432 sec latency 167.206 ms 2024-04-16T17:27:01.056 INFO:tasks.workunit.client.0.smithi018.stdout:[122737] write failed on handle 10494 (Resource temporarily unavailable) 2024-04-16T17:27:01.057 INFO:tasks.workunit.client.0.smithi018.stdout:Child failed with status 1 2024-04-16T17:27:01.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:27:01.087 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-16T17:27:01.088 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-16T17:27:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:26:58 smithi018 ceph-mon[31776]: pgmap v711: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 402 op/s 2024-04-16T17:27:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:00 smithi018 ceph-mon[31776]: pgmap v712: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 213 op/s 2024-04-16T17:27:02.352 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:02 smithi018 ceph-mon[31776]: pgmap v713: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 347 op/s 2024-04-16T17:27:02.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:02 smithi022 ceph-mon[36023]: pgmap v713: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 347 op/s 2024-04-16T17:27:02.445 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi018 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b137e4a675398e295567f9652210865a4d573fac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-16T17:27:02.675 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1c3db1d8239f4a198174c124580123a5 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi018 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b137e4a675398e295567f9652210865a4d573fac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-16T17:27:02.677 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-16T17:27:02.687 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:27:02.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:02 smithi043 ceph-mon[36901]: pgmap v713: 129 pgs: 129 active+clean; 476 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 347 op/s 2024-04-16T17:27:02.707 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-16T17:27:02.718 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-16T17:27:02.718 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T17:27:02.718 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:27:02.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-16T17:27:02.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-16T17:27:02.980 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-16T17:27:03.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 -- 172.21.15.18:0/2111588078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0104470 msgr2=0x7f5db0104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 --2- 172.21.15.18:0/2111588078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0104470 0x7f5db0104850 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009900 tx=0x7f5da402bc50 comp rx=0 tx=0).stop 2024-04-16T17:27:03.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 -- 172.21.15.18:0/2111588078 shutdown_connections 2024-04-16T17:27:03.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 --2- 172.21.15.18:0/2111588078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0109540 0x7f5db0111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 --2- 172.21.15.18:0/2111588078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db0104e20 0x7f5db0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 --2- 172.21.15.18:0/2111588078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0104470 0x7f5db0104850 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 -- 172.21.15.18:0/2111588078 >> 172.21.15.18:0/2111588078 conn(0x7f5db00ffcf0 msgr2=0x7f5db0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:03.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 -- 172.21.15.18:0/2111588078 shutdown_connections 2024-04-16T17:27:03.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.187+0000 7f5db7400640 1 -- 172.21.15.18:0/2111588078 wait complete. 2024-04-16T17:27:03.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.189+0000 7f5db7400640 1 Processor -- start 2024-04-16T17:27:03.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.189+0000 7f5db7400640 1 -- start start 2024-04-16T17:27:03.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.189+0000 7f5db7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0104470 0x7f5db01a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:03.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.189+0000 7f5db7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0109540 0x7f5db01a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:03.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.189+0000 7f5db7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db019aa20 0x7f5db019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:03.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.189+0000 7f5db7400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01145d0 con 0x7f5db0109540 2024-04-16T17:27:03.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.189+0000 7f5db7400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5db0114450 con 0x7f5db019aa20 2024-04-16T17:27:03.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.189+0000 7f5db7400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5db0114750 con 0x7f5db0104470 2024-04-16T17:27:03.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5dafe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0109540 0x7f5db01a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:03.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db019aa20 0x7f5db019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:03.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db019aa20 0x7f5db019aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45950/0 (socket says 172.21.15.18:45950) 2024-04-16T17:27:03.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 -- 172.21.15.18:0/3621304095 learned_addr learned my addr 172.21.15.18:0/3621304095 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:03.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5dafe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0109540 0x7f5db01a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39962/0 (socket says 172.21.15.18:39962) 2024-04-16T17:27:03.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db4a00640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0104470 0x7f5db01a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:03.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 -- 172.21.15.18:0/3621304095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0104470 msgr2=0x7f5db01a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0104470 0x7f5db01a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 -- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0109540 msgr2=0x7f5db01a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0109540 0x7f5db01a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 -- 172.21.15.18:0/3621304095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db019b540 con 0x7f5db019aa20 2024-04-16T17:27:03.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5dafe00640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0109540 0x7f5db01a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:27:03.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.190+0000 7f5db5400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db019aa20 0x7f5db019aed0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009fd0 tx=0x7f5da800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:03.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.191+0000 7f5dad600640 1 -- 172.21.15.18:0/3621304095 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8017070 con 0x7f5db019aa20 2024-04-16T17:27:03.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.191+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db019b830 con 0x7f5db019aa20 2024-04-16T17:27:03.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.191+0000 7f5dad600640 1 -- 172.21.15.18:0/3621304095 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da8003e50 con 0x7f5db019aa20 2024-04-16T17:27:03.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.191+0000 7f5dad600640 1 -- 172.21.15.18:0/3621304095 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da80106c0 con 0x7f5db019aa20 2024-04-16T17:27:03.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.191+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db01a7a30 con 0x7f5db019aa20 2024-04-16T17:27:03.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.192+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db0105940 con 0x7f5db019aa20 2024-04-16T17:27:03.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.194+0000 7f5dad600640 1 -- 172.21.15.18:0/3621304095 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5da8003970 con 0x7f5db019aa20 2024-04-16T17:27:03.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.195+0000 7f5dad600640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d980a3d90 0x7f5d980a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:03.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.195+0000 7f5dad600640 1 -- 172.21.15.18:0/3621304095 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9538+0+0 (secure 0 0 0) 0x7f5da80d0890 con 0x7f5db019aa20 2024-04-16T17:27:03.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.196+0000 7f5db4a00640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d980a3d90 0x7f5d980a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:03.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.196+0000 7f5db4a00640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d980a3d90 0x7f5d980a6250 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009230 tx=0x7f5da4005fd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:03.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.200+0000 7f5dad600640 1 -- 172.21.15.18:0/3621304095 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5da809c480 con 0x7f5db019aa20 2024-04-16T17:27:03.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.425+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5db010c7e0 con 0x7f5d980a3d90 2024-04-16T17:27:03.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.442+0000 7f5dad600640 1 -- 172.21.15.18:0/3621304095 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f5da402c8a0 con 0x7f5d980a3d90 2024-04-16T17:27:03.442 INFO:teuthology.orchestra.run.smithi018.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-16T00:00:00", "created": "2024-04-16T17:04:55", "first": "2024-04-16T17:05:00", "last": "2024-04-16T17:27:00", "last_pruned": "2024-04-16T17:27:00", "created_count": 23, "pruned_count": 17, "active": true} 2024-04-16T17:27:03.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d980a3d90 msgr2=0x7f5d980a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d980a3d90 0x7f5d980a6250 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009230 tx=0x7f5da4005fd0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db019aa20 msgr2=0x7f5db019aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db019aa20 0x7f5db019aed0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009fd0 tx=0x7f5da800b260 comp rx=0 tx=0).stop 2024-04-16T17:27:03.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 shutdown_connections 2024-04-16T17:27:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f5d980a3d90 0x7f5d980a6250 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db019aa20 0x7f5db019aed0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5db0109540 0x7f5db01a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 --2- 172.21.15.18:0/3621304095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db0104470 0x7f5db01a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 >> 172.21.15.18:0/3621304095 conn(0x7f5db00ffcf0 msgr2=0x7f5db0101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 shutdown_connections 2024-04-16T17:27:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.445+0000 7f5db7400640 1 -- 172.21.15.18:0/3621304095 wait complete. 2024-04-16T17:27:03.463 DEBUG:teuthology.orchestra.run.smithi018:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-16T17:27:03.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.662+0000 7fef40a00640 1 -- 172.21.15.18:0/3071837631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 msgr2=0x7fef3c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.662+0000 7fef40a00640 1 --2- 172.21.15.18:0/3071837631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 0x7fef3c10e8a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fef34009be0 tx=0x7fef3402bf80 comp rx=0 tx=0).stop 2024-04-16T17:27:03.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.662+0000 7fef40a00640 1 -- 172.21.15.18:0/3071837631 shutdown_connections 2024-04-16T17:27:03.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.662+0000 7fef40a00640 1 --2- 172.21.15.18:0/3071837631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 0x7fef3c10e8a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.662+0000 7fef40a00640 1 --2- 172.21.15.18:0/3071837631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef3c103900 0x7fef3c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.662+0000 7fef40a00640 1 --2- 172.21.15.18:0/3071837631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3c102fe0 0x7fef3c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.662+0000 7fef40a00640 1 -- 172.21.15.18:0/3071837631 >> 172.21.15.18:0/3071837631 conn(0x7fef3c0fc7e0 msgr2=0x7fef3c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:03.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.662+0000 7fef40a00640 1 -- 172.21.15.18:0/3071837631 shutdown_connections 2024-04-16T17:27:03.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.663+0000 7fef40a00640 1 -- 172.21.15.18:0/3071837631 wait complete. 2024-04-16T17:27:03.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.664+0000 7fef40a00640 1 Processor -- start 2024-04-16T17:27:03.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.664+0000 7fef40a00640 1 -- start start 2024-04-16T17:27:03.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.664+0000 7fef40a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef3c102fe0 0x7fef3c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:03.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.664+0000 7fef40a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3c103900 0x7fef3c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:03.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef40a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 0x7fef3c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:03.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef40a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c112210 con 0x7fef3c102fe0 2024-04-16T17:27:03.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef40a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c112090 con 0x7fef3c103900 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef40a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c112390 con 0x7fef3c10c400 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef39e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3c103900 0x7fef3c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef39e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3c103900 0x7fef3c1a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45970/0 (socket says 172.21.15.18:45970) 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef39e00640 1 -- 172.21.15.18:0/840464859 learned_addr learned my addr 172.21.15.18:0/840464859 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef3a800640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef3c102fe0 0x7fef3c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef3b200640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 0x7fef3c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef39e00640 1 -- 172.21.15.18:0/840464859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 msgr2=0x7fef3c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef39e00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 0x7fef3c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef39e00640 1 -- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef3c102fe0 msgr2=0x7fef3c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef39e00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef3c102fe0 0x7fef3c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef39e00640 1 -- 172.21.15.18:0/840464859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3c19af80 con 0x7fef3c103900 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.665+0000 7fef3b200640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 0x7fef3c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.666+0000 7fef39e00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3c103900 0x7fef3c1a0b40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fef2800e790 tx=0x7fef2800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.666+0000 7fef33400640 1 -- 172.21.15.18:0/840464859 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2800eea0 con 0x7fef3c103900 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.666+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3c19b270 con 0x7fef3c103900 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.666+0000 7fef33400640 1 -- 172.21.15.18:0/840464859 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef28005500 con 0x7fef3c103900 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.666+0000 7fef33400640 1 -- 172.21.15.18:0/840464859 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28010430 con 0x7fef3c103900 2024-04-16T17:27:03.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.666+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3c1a7580 con 0x7fef3c103900 2024-04-16T17:27:03.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.667+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef3c06a100 con 0x7fef3c103900 2024-04-16T17:27:03.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.669+0000 7fef33400640 1 -- 172.21.15.18:0/840464859 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef28005000 con 0x7fef3c103900 2024-04-16T17:27:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.671+0000 7fef33400640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fef200a3d00 0x7fef200a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.671+0000 7fef33400640 1 -- 172.21.15.18:0/840464859 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9538+0+0 (secure 0 0 0) 0x7fef280d0e40 con 0x7fef3c103900 2024-04-16T17:27:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.671+0000 7fef3a800640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fef200a3d00 0x7fef200a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.672+0000 7fef3a800640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fef200a3d00 0x7fef200a61c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009bd0 tx=0x7fef2c005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.675+0000 7fef33400640 1 -- 172.21.15.18:0/840464859 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef2809d080 con 0x7fef3c103900 2024-04-16T17:27:03.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.915+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 --> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fef3c102280 con 0x7fef200a3d00 2024-04-16T17:27:03.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.928+0000 7fef33400640 1 -- 172.21.15.18:0/840464859 <== mgr.25417 v2:172.21.15.18:6800/3776183549 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fef3c102280 con 0x7fef200a3d00 2024-04-16T17:27:03.928 INFO:teuthology.orchestra.run.smithi018.stdout:/ 1m 6m3h 2024-04-16T17:27:03.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fef200a3d00 msgr2=0x7fef200a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fef200a3d00 0x7fef200a61c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009bd0 tx=0x7fef2c005f90 comp rx=0 tx=0).stop 2024-04-16T17:27:03.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3c103900 msgr2=0x7fef3c1a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:03.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3c103900 0x7fef3c1a0b40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fef2800e790 tx=0x7fef2800ec50 comp rx=0 tx=0).stop 2024-04-16T17:27:03.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 shutdown_connections 2024-04-16T17:27:03.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fef200a3d00 0x7fef200a61c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef3c10c400 0x7fef3c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3c103900 0x7fef3c1a0b40 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 --2- 172.21.15.18:0/840464859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef3c102fe0 0x7fef3c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:03.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 >> 172.21.15.18:0/840464859 conn(0x7fef3c0fc7e0 msgr2=0x7fef3c100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:03.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.930+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 shutdown_connections 2024-04-16T17:27:03.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:03.931+0000 7fef40a00640 1 -- 172.21.15.18:0/840464859 wait complete. 2024-04-16T17:27:03.949 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-16T17:27:03.959 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-16T17:27:04.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:04 smithi022 ceph-mon[36023]: pgmap v714: 129 pgs: 129 active+clean; 466 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 390 op/s 2024-04-16T17:27:04.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:04 smithi022 ceph-mon[36023]: from='client.31229 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:27:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:04 smithi018 ceph-mon[31776]: pgmap v714: 129 pgs: 129 active+clean; 466 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 390 op/s 2024-04-16T17:27:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:04 smithi018 ceph-mon[31776]: from='client.31229 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:27:04.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:04 smithi043 ceph-mon[36901]: pgmap v714: 129 pgs: 129 active+clean; 466 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 390 op/s 2024-04-16T17:27:04.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:04 smithi043 ceph-mon[36901]: from='client.31229 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:27:05.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:27:05.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:27:05.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:05 smithi022 ceph-mon[36023]: from='client.31235 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:27:05.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:05 smithi022 ceph-mon[36023]: osdmap e137: 12 total, 12 up, 12 in 2024-04-16T17:27:05.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:05 smithi018 ceph-mon[31776]: from='client.31235 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:27:05.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:05 smithi018 ceph-mon[31776]: osdmap e137: 12 total, 12 up, 12 in 2024-04-16T17:27:05.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:05 smithi043 ceph-mon[36901]: from='client.31235 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:27:05.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:05 smithi043 ceph-mon[36901]: osdmap e137: 12 total, 12 up, 12 in 2024-04-16T17:27:05.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.730+0000 7f2c60e00640 1 -- 172.21.15.18:0/2988794316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c074cc0 msgr2=0x7f2c5c1128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.730+0000 7f2c60e00640 1 --2- 172.21.15.18:0/2988794316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c074cc0 0x7f2c5c1128d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009860 tx=0x7f2c4802bc00 comp rx=0 tx=0).stop 2024-04-16T17:27:05.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.731+0000 7f2c60e00640 1 -- 172.21.15.18:0/2988794316 shutdown_connections 2024-04-16T17:27:05.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.731+0000 7f2c60e00640 1 --2- 172.21.15.18:0/2988794316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c5c112e10 0x7f2c5c115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.731+0000 7f2c60e00640 1 --2- 172.21.15.18:0/2988794316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c074cc0 0x7f2c5c1128d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.731+0000 7f2c60e00640 1 --2- 172.21.15.18:0/2988794316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c5c0743a0 0x7f2c5c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.731+0000 7f2c60e00640 1 -- 172.21.15.18:0/2988794316 >> 172.21.15.18:0/2988794316 conn(0x7f2c5c0fd950 msgr2=0x7f2c5c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:05.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.731+0000 7f2c60e00640 1 -- 172.21.15.18:0/2988794316 shutdown_connections 2024-04-16T17:27:05.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.731+0000 7f2c60e00640 1 -- 172.21.15.18:0/2988794316 wait complete. 2024-04-16T17:27:05.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.732+0000 7f2c60e00640 1 Processor -- start 2024-04-16T17:27:05.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.735+0000 7f2c60e00640 1 -- start start 2024-04-16T17:27:05.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c60e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c5c0743a0 0x7f2c5c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c60e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c074cc0 0x7f2c5c19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c60e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c5c112e10 0x7f2c5c1a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c60e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1187f0 con 0x7f2c5c0743a0 2024-04-16T17:27:05.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c60e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c118670 con 0x7f2c5c112e10 2024-04-16T17:27:05.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c60e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c118970 con 0x7f2c5c074cc0 2024-04-16T17:27:05.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c5be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c5c112e10 0x7f2c5c1a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c5be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c5c112e10 0x7f2c5c1a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:45980/0 (socket says 172.21.15.18:45980) 2024-04-16T17:27:05.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.736+0000 7f2c5be00640 1 -- 172.21.15.18:0/857111666 learned_addr learned my addr 172.21.15.18:0/857111666 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:05.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.737+0000 7f2c5b400640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c5c0743a0 0x7f2c5c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.737+0000 7f2c5be00640 1 -- 172.21.15.18:0/857111666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c074cc0 msgr2=0x7f2c5c19f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.737+0000 7f2c5be00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c074cc0 0x7f2c5c19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.737+0000 7f2c5be00640 1 -- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c5c0743a0 msgr2=0x7f2c5c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.737+0000 7f2c5be00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c5c0743a0 0x7f2c5c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.737+0000 7f2c5be00640 1 -- 172.21.15.18:0/857111666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c1a3d80 con 0x7f2c5c112e10 2024-04-16T17:27:05.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.737+0000 7f2c5b400640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c5c0743a0 0x7f2c5c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:27:05.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.738+0000 7f2c5be00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c5c112e10 0x7f2c5c1a3650 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009fa0 tx=0x7f2c5400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:05.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.738+0000 7f2c53e00640 1 -- 172.21.15.18:0/857111666 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54004b20 con 0x7f2c5c112e10 2024-04-16T17:27:05.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.738+0000 7f2c53e00640 1 -- 172.21.15.18:0/857111666 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c54003e50 con 0x7f2c5c112e10 2024-04-16T17:27:05.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.738+0000 7f2c53e00640 1 -- 172.21.15.18:0/857111666 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400b570 con 0x7f2c5c112e10 2024-04-16T17:27:05.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.739+0000 7f2c60e00640 1 -- 172.21.15.18:0/857111666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5c1a4070 con 0x7f2c5c112e10 2024-04-16T17:27:05.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.739+0000 7f2c60e00640 1 -- 172.21.15.18:0/857111666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5c1ab900 con 0x7f2c5c112e10 2024-04-16T17:27:05.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.747+0000 7f2c53e00640 1 -- 172.21.15.18:0/857111666 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c5401d020 con 0x7f2c5c112e10 2024-04-16T17:27:05.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.750+0000 7f2c53e00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c400a3e20 0x7f2c400a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.750+0000 7f2c53e00640 1 -- 172.21.15.18:0/857111666 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f2c540d0c80 con 0x7f2c5c112e10 2024-04-16T17:27:05.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.751+0000 7f2c5b400640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c400a3e20 0x7f2c400a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.752+0000 7f2c5b400640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c400a3e20 0x7f2c400a62e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009b40 tx=0x7f2c4c005e50 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:05.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.764+0000 7f2c65215740 1 Processor -- start 2024-04-16T17:27:05.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.765+0000 7f2c65215740 1 -- start start 2024-04-16T17:27:05.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.765+0000 7f2c65215740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a040 0x559842f2a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.765+0000 7f2c65215740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f44360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.765+0000 7f2c65215740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f46c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.765+0000 7f2c65215740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559842e90120 con 0x559842f448a0 2024-04-16T17:27:05.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.765+0000 7f2c65215740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559842e36340 con 0x559842f2a960 2024-04-16T17:27:05.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.765+0000 7f2c65215740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559842e8c0e0 con 0x559842f2a040 2024-04-16T17:27:05.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.765+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a040 0x559842f2a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a040 0x559842f2a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47264/0 (socket says 172.21.15.18:47264) 2024-04-16T17:27:05.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5b400640 1 -- 172.21.15.18:0/2058107213 learned_addr learned my addr 172.21.15.18:0/2058107213 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:05.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5be00640 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f46c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5aa00640 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f44360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5be00640 1 -- 172.21.15.18:0/2058107213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a040 msgr2=0x559842f2a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:05.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5be00640 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a040 0x559842f2a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5be00640 1 -- 172.21.15.18:0/2058107213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 msgr2=0x559842f44360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:05.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5be00640 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f44360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.766+0000 7f2c5be00640 1 -- 172.21.15.18:0/2058107213 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559842c6c7d0 con 0x559842f448a0 2024-04-16T17:27:05.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.767+0000 7f2c5be00640 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f46c90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2c540cdf10 tx=0x7f2c5409c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56a6a73c53e96c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:05.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.767+0000 7f2c50c00640 1 -- 172.21.15.18:0/2058107213 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5401a040 con 0x559842f448a0 2024-04-16T17:27:05.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.767+0000 7f2c50c00640 1 -- 172.21.15.18:0/2058107213 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c5409ce60 con 0x559842f448a0 2024-04-16T17:27:05.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.767+0000 7f2c50c00640 1 -- 172.21.15.18:0/2058107213 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5401a040 con 0x559842f448a0 2024-04-16T17:27:05.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.768+0000 7f2c65215740 1 -- 172.21.15.18:0/2058107213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 msgr2=0x559842f46c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.768+0000 7f2c65215740 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f46c90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2c540cdf10 tx=0x7f2c5409c2a0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.768+0000 7f2c65215740 1 -- 172.21.15.18:0/2058107213 shutdown_connections 2024-04-16T17:27:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.768+0000 7f2c65215740 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f46c90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.768+0000 7f2c65215740 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f44360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.768+0000 7f2c65215740 1 --2- 172.21.15.18:0/2058107213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a040 0x559842f2a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.768+0000 7f2c65215740 1 -- 172.21.15.18:0/2058107213 >> 172.21.15.18:0/2058107213 conn(0x559842f24ac0 msgr2=0x559842f24ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.769+0000 7f2c65215740 1 -- 172.21.15.18:0/2058107213 shutdown_connections 2024-04-16T17:27:05.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.769+0000 7f2c65215740 1 -- 172.21.15.18:0/2058107213 wait complete. 2024-04-16T17:27:05.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.769+0000 7f2c65215740 1 -- start start 2024-04-16T17:27:05.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.770+0000 7f2c65215740 1 Processor -- start 2024-04-16T17:27:05.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.770+0000 7f2c65215740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f35b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.770+0000 7f2c65215740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f36080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.770+0000 7f2c65215740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f448a0 0x559842f30f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.771+0000 7f2c65215740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559842f2b4d0 con 0x559842f2a040 2024-04-16T17:27:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.771+0000 7f2c65215740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559842e49710 con 0x559842f448a0 2024-04-16T17:27:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.771+0000 7f2c65215740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559842f2b650 con 0x559842f2a960 2024-04-16T17:27:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.771+0000 7f2c65215740 10 client.0 _ll_register_callbacks cb 0x559842e44af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:27:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.771+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f36080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.771+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f36080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47276/0 (socket says 172.21.15.18:47276) 2024-04-16T17:27:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.771+0000 7f2c5aa00640 1 -- 172.21.15.18:0/3128254568 learned_addr learned my addr 172.21.15.18:0/3128254568 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.771+0000 7f2c5be00640 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f448a0 0x559842f30f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.772+0000 7f2c5b400640 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f35b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.772+0000 7f2c5aa00640 1 -- 172.21.15.18:0/3128254568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f448a0 msgr2=0x559842f30f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.772+0000 7f2c5aa00640 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f448a0 0x559842f30f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.772+0000 7f2c5aa00640 1 -- 172.21.15.18:0/3128254568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 msgr2=0x559842f35b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.772+0000 7f2c5aa00640 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f35b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.772+0000 7f2c5aa00640 1 -- 172.21.15.18:0/3128254568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559842ba23f0 con 0x559842f2a960 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.773+0000 7f2c5aa00640 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f36080 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2c4802c140 tx=0x7f2c48004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78a0754afaa6d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:05.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.773+0000 7f2c45600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:27:05.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.773+0000 7f2c45600640 1 -- 172.21.15.18:0/3128254568 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48039050 con 0x559842f2a960 2024-04-16T17:27:05.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.773+0000 7f2c65215740 1 -- 172.21.15.18:0/3128254568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c5409ce60 con 0x559842f2a960 2024-04-16T17:27:05.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.774+0000 7f2c45600640 1 -- 172.21.15.18:0/3128254568 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f2c48034640 con 0x559842f2a960 2024-04-16T17:27:05.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.774+0000 7f2c45600640 1 -- 172.21.15.18:0/3128254568 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2c48034840 con 0x559842f2a960 2024-04-16T17:27:05.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.774+0000 7f2c65215740 10 client.31087 fetch_fsmap learned FSMap version 37 2024-04-16T17:27:05.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.774+0000 7f2c65215740 1 -- 172.21.15.18:0/3128254568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559842b620f0 con 0x559842f2a960 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.774+0000 7f2c45600640 1 -- 172.21.15.18:0/3128254568 <== mon.1 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c5409ce60 con 0x559842f2a960 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.775+0000 7f2c45600640 1 -- 172.21.15.18:0/3128254568 <== mon.1 v2:172.21.15.43:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2c480035f0 con 0x559842f2a960 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.775+0000 7f2c65215740 10 client.31087 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.775+0000 7f2c65215740 10 client.31087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.775+0000 7f2c65215740 20 client.31087 populate_metadata read hostname 'smithi018' 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.776+0000 7f2c65215740 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559842f40f90 0x559842f433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.776+0000 7f2c65215740 4 client.31087 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.776+0000 7f2c65215740 1 -- 172.21.15.18:0/3128254568 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559842e9db40 con 0x559842f40f90 2024-04-16T17:27:05.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.776+0000 7f2c5b400640 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559842f40f90 0x559842f433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.779+0000 7f2c5b400640 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559842f40f90 0x559842f433a0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309bfb8e5ed157da server_cookie=be0776833fd05ca0 in_seq=0 out_seq=0 2024-04-16T17:27:05.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.779+0000 7f2c45600640 10 client.31087 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:27:05.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.780+0000 7f2c45600640 1 -- 172.21.15.18:0/3128254568 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559842e9db40 con 0x559842f40f90 2024-04-16T17:27:05.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.780+0000 7f2c45600640 10 client.31087 handle_command_reply: tid=0 2024-04-16T17:27:05.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.780+0000 7f2c65215740 1 client.31087 shutdown 2024-04-16T17:27:05.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 -- 172.21.15.18:0/3128254568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 msgr2=0x559842f36080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f36080 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2c4802c140 tx=0x7f2c48004270 comp rx=0 tx=0).stop 2024-04-16T17:27:05.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 -- 172.21.15.18:0/3128254568 shutdown_connections 2024-04-16T17:27:05.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559842f40f90 0x559842f433a0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f448a0 0x559842f30f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f36080 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 --2- 172.21.15.18:0/3128254568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f35b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 -- 172.21.15.18:0/3128254568 >> 172.21.15.18:0/3128254568 conn(0x559842f24ac0 msgr2=0x559842f26810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:05.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c45600640 0 client.31087 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:27:05.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 -- 172.21.15.18:0/3128254568 shutdown_connections 2024-04-16T17:27:05.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.781+0000 7f2c65215740 1 -- 172.21.15.18:0/3128254568 wait complete. 2024-04-16T17:27:05.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.782+0000 7f2c65215740 20 client.31087 trim_cache size 0 max 16384 2024-04-16T17:27:05.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.833+0000 7f2c65215740 1 Processor -- start 2024-04-16T17:27:05.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.836+0000 7f2c65215740 1 -- start start 2024-04-16T17:27:05.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c65215740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f44320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c65215740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f2e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c65215740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f448a0 0x559842f2e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c65215740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c06fce0 con 0x559842f2a040 2024-04-16T17:27:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c65215740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c06d5b0 con 0x559842f2a960 2024-04-16T17:27:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c65215740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c10e2b0 con 0x559842f448a0 2024-04-16T17:27:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f2e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f2e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46002/0 (socket says 172.21.15.18:46002) 2024-04-16T17:27:05.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c5aa00640 1 -- 172.21.15.18:0/966509239 learned_addr learned my addr 172.21.15.18:0/966509239 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c5b400640 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f44320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.837+0000 7f2c5be00640 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f448a0 0x559842f2e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.838+0000 7f2c5b400640 1 -- 172.21.15.18:0/966509239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f448a0 msgr2=0x559842f2e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:05.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.838+0000 7f2c5b400640 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f448a0 0x559842f2e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.838+0000 7f2c5b400640 1 -- 172.21.15.18:0/966509239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 msgr2=0x559842f2e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:05.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.838+0000 7f2c5b400640 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f2e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.838+0000 7f2c5b400640 1 -- 172.21.15.18:0/966509239 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559842f29480 con 0x559842f2a040 2024-04-16T17:27:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.839+0000 7f2c5be00640 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f448a0 0x559842f2e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:27:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.839+0000 7f2c5aa00640 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f2e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:27:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.839+0000 7f2c5b400640 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f44320 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x559842f43b80 tx=0x7f2c4c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70167485641fba4f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.841+0000 7f2c47e00640 1 -- 172.21.15.18:0/966509239 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c04adc0 con 0x559842f2a040 2024-04-16T17:27:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.841+0000 7f2c47e00640 1 -- 172.21.15.18:0/966509239 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c4c005020 con 0x559842f2a040 2024-04-16T17:27:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.841+0000 7f2c47e00640 1 -- 172.21.15.18:0/966509239 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c04a830 con 0x559842f2a040 2024-04-16T17:27:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.842+0000 7f2c65215740 1 -- 172.21.15.18:0/966509239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 msgr2=0x559842f44320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.842+0000 7f2c65215740 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f44320 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x559842f43b80 tx=0x7f2c4c014000 comp rx=0 tx=0).stop 2024-04-16T17:27:05.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.842+0000 7f2c65215740 1 -- 172.21.15.18:0/966509239 shutdown_connections 2024-04-16T17:27:05.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.842+0000 7f2c65215740 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f448a0 0x559842f2e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.842+0000 7f2c65215740 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842f2a960 0x559842f2e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.842+0000 7f2c65215740 1 --2- 172.21.15.18:0/966509239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f2a040 0x559842f44320 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.842+0000 7f2c65215740 1 -- 172.21.15.18:0/966509239 >> 172.21.15.18:0/966509239 conn(0x559842f24ac0 msgr2=0x559842f43c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:05.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.843+0000 7f2c65215740 1 -- 172.21.15.18:0/966509239 shutdown_connections 2024-04-16T17:27:05.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.843+0000 7f2c65215740 1 -- 172.21.15.18:0/966509239 wait complete. 2024-04-16T17:27:05.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.843+0000 7f2c65215740 1 -- start start 2024-04-16T17:27:05.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.844+0000 7f2c65215740 1 Processor -- start 2024-04-16T17:27:05.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.844+0000 7f2c65215740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f4b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.844+0000 7f2c65215740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f4ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.844+0000 7f2c65215740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842fa6d40 0x559842fa9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.844+0000 7f2c65215740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c06ba70 con 0x559842f448a0 2024-04-16T17:27:05.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.844+0000 7f2c65215740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c06cee0 con 0x559842fa6d40 2024-04-16T17:27:05.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.844+0000 7f2c65215740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c06c140 con 0x559842f2a960 2024-04-16T17:27:05.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.844+0000 7f2c65215740 10 client.0 _ll_register_callbacks cb 0x7f2c48037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:27:05.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.845+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f4b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.845+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f4b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47284/0 (socket says 172.21.15.18:47284) 2024-04-16T17:27:05.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.845+0000 7f2c5be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842fa6d40 0x559842fa9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.846+0000 7f2c5be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842fa6d40 0x559842fa9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46006/0 (socket says 172.21.15.18:46006) 2024-04-16T17:27:05.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.846+0000 7f2c5be00640 1 -- 172.21.15.18:0/918823922 learned_addr learned my addr 172.21.15.18:0/918823922 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:05.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.846+0000 7f2c5aa00640 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f4ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.846+0000 7f2c5be00640 1 -- 172.21.15.18:0/918823922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 msgr2=0x559842f4b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.846+0000 7f2c5be00640 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f4b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.846+0000 7f2c5be00640 1 -- 172.21.15.18:0/918823922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 msgr2=0x559842f4ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.846+0000 7f2c5be00640 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f4ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.846+0000 7f2c5be00640 1 -- 172.21.15.18:0/918823922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559842fa9670 con 0x559842fa6d40 2024-04-16T17:27:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.847+0000 7f2c5be00640 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842fa6d40 0x559842fa9130 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f2c54099130 tx=0x7f2c5400cec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d416c79bf6bdf122 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.847+0000 7f2c45600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:27:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.848+0000 7f2c45600640 1 -- 172.21.15.18:0/918823922 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5401a040 con 0x559842fa6d40 2024-04-16T17:27:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.848+0000 7f2c45600640 1 -- 172.21.15.18:0/918823922 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f2c5407b170 con 0x559842fa6d40 2024-04-16T17:27:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.848+0000 7f2c65215740 1 -- 172.21.15.18:0/918823922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c4c005020 con 0x559842fa6d40 2024-04-16T17:27:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.848+0000 7f2c45600640 1 -- 172.21.15.18:0/918823922 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2c540bb050 con 0x559842fa6d40 2024-04-16T17:27:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.848+0000 7f2c65215740 10 client.31250 fetch_fsmap learned FSMap version 37 2024-04-16T17:27:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.848+0000 7f2c65215740 1 -- 172.21.15.18:0/918823922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559842faa8f0 con 0x559842fa6d40 2024-04-16T17:27:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.849+0000 7f2c45600640 1 -- 172.21.15.18:0/918823922 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c4c005020 con 0x559842fa6d40 2024-04-16T17:27:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.849+0000 7f2c45600640 1 -- 172.21.15.18:0/918823922 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2c5400b040 con 0x559842fa6d40 2024-04-16T17:27:05.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.849+0000 7f2c65215740 10 client.31250 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:27:05.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.849+0000 7f2c65215740 10 client.31250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:27:05.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.849+0000 7f2c65215740 20 client.31250 populate_metadata read hostname 'smithi018' 2024-04-16T17:27:05.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.849+0000 7f2c65215740 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559842faabe0 0x559842facfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:05.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.849+0000 7f2c65215740 4 client.31250 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:27:05.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.849+0000 7f2c65215740 1 -- 172.21.15.18:0/918823922 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559842f20180 con 0x559842faabe0 2024-04-16T17:27:05.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.850+0000 7f2c5b400640 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559842faabe0 0x559842facfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:05.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.851+0000 7f2c45600640 10 client.31250 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:27:05.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.851+0000 7f2c5b400640 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559842faabe0 0x559842facfd0 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a481c45b75004ae2 server_cookie=a8b2538d845049a2 in_seq=0 out_seq=0 2024-04-16T17:27:05.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.851+0000 7f2c45600640 1 -- 172.21.15.18:0/918823922 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559842f20180 con 0x559842faabe0 2024-04-16T17:27:05.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.851+0000 7f2c45600640 10 client.31250 handle_command_reply: tid=0 2024-04-16T17:27:05.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.851+0000 7f2c65215740 1 client.31250 shutdown 2024-04-16T17:27:05.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.851+0000 7f2c65215740 1 -- 172.21.15.18:0/918823922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842fa6d40 msgr2=0x559842fa9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.851+0000 7f2c65215740 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842fa6d40 0x559842fa9130 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f2c54099130 tx=0x7f2c5400cec0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 1 -- 172.21.15.18:0/918823922 shutdown_connections 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x559842faabe0 0x559842facfd0 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559842fa6d40 0x559842fa9130 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559842f448a0 0x559842f4ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c45600640 0 client.31250 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 1 --2- 172.21.15.18:0/918823922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559842f2a960 0x559842f4b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 1 -- 172.21.15.18:0/918823922 >> 172.21.15.18:0/918823922 conn(0x559842f24ac0 msgr2=0x559842f23290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 1 -- 172.21.15.18:0/918823922 shutdown_connections 2024-04-16T17:27:05.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 1 -- 172.21.15.18:0/918823922 wait complete. 2024-04-16T17:27:05.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.852+0000 7f2c65215740 20 client.31250 trim_cache size 0 max 16384 2024-04-16T17:27:05.856 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:27:05.856 INFO:teuthology.orchestra.run.smithi018.stdout: "status": "no active scrubs running", 2024-04-16T17:27:05.856 INFO:teuthology.orchestra.run.smithi018.stdout: "scrubs": {} 2024-04-16T17:27:05.857 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:27:05.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.855+0000 7f2c60e00640 1 -- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c400a3e20 msgr2=0x7f2c400a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.855+0000 7f2c60e00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c400a3e20 0x7f2c400a62e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009b40 tx=0x7f2c4c005e50 comp rx=0 tx=0).stop 2024-04-16T17:27:05.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.855+0000 7f2c60e00640 1 -- 172.21.15.18:0/857111666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c5c112e10 msgr2=0x7f2c5c1a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:05.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.855+0000 7f2c60e00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c5c112e10 0x7f2c5c1a3650 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009fa0 tx=0x7f2c5400c800 comp rx=0 tx=0).stop 2024-04-16T17:27:05.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.856+0000 7f2c60e00640 1 -- 172.21.15.18:0/857111666 shutdown_connections 2024-04-16T17:27:05.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.856+0000 7f2c60e00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f2c400a3e20 0x7f2c400a62e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.856+0000 7f2c60e00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c5c112e10 0x7f2c5c1a3650 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.856+0000 7f2c60e00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c074cc0 0x7f2c5c19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.856+0000 7f2c60e00640 1 --2- 172.21.15.18:0/857111666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c5c0743a0 0x7f2c5c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.856+0000 7f2c60e00640 1 -- 172.21.15.18:0/857111666 >> 172.21.15.18:0/857111666 conn(0x7f2c5c0fd950 msgr2=0x7f2c5c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.856+0000 7f2c60e00640 1 -- 172.21.15.18:0/857111666 shutdown_connections 2024-04-16T17:27:05.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:05.856+0000 7f2c60e00640 1 -- 172.21.15.18:0/857111666 wait complete. 2024-04-16T17:27:05.884 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:27:05.885 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:27:06.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:06 smithi018 ceph-mon[31776]: pgmap v716: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 466 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 395 op/s 2024-04-16T17:27:06.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.220+0000 7f65f6600640 1 -- 172.21.15.18:0/629684316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8095800 msgr2=0x7f65e80a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:06.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.220+0000 7f65f6600640 1 --2- 172.21.15.18:0/629684316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8095800 0x7f65e80a1ad0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f65f0067fb0 tx=0x7f65f0099f00 comp rx=0 tx=0).stop 2024-04-16T17:27:06.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.221+0000 7f65f6600640 1 -- 172.21.15.18:0/629684316 shutdown_connections 2024-04-16T17:27:06.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.221+0000 7f65f6600640 1 --2- 172.21.15.18:0/629684316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8095800 0x7f65e80a1ad0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.221+0000 7f65f6600640 1 --2- 172.21.15.18:0/629684316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e8094e60 0x7f65e80952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.221+0000 7f65f6600640 1 --2- 172.21.15.18:0/629684316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e8099bd0 0x7f65e8099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.221+0000 7f65f6600640 1 -- 172.21.15.18:0/629684316 >> 172.21.15.18:0/629684316 conn(0x7f65e800a750 msgr2=0x7f65e800ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:06.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.221+0000 7f65f6600640 1 -- 172.21.15.18:0/629684316 shutdown_connections 2024-04-16T17:27:06.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.221+0000 7f65f6600640 1 -- 172.21.15.18:0/629684316 wait complete. 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.222+0000 7f65f6600640 1 Processor -- start 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.223+0000 7f65f6600640 1 -- start start 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.224+0000 7f65f6600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8094e60 0x7f65e812a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.224+0000 7f65f6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e8095800 0x7f65e812a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.224+0000 7f65f5200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8094e60 0x7f65e812a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.224+0000 7f65f5200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8094e60 0x7f65e812a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:46026/0 (socket says 172.21.15.18:46026) 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.224+0000 7f65f6600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e8099bd0 0x7f65e812eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.225+0000 7f65efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e8095800 0x7f65e812a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:06.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.225+0000 7f65efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e8095800 0x7f65e812a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:47300/0 (socket says 172.21.15.18:47300) 2024-04-16T17:27:06.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.225+0000 7f65f5200640 1 -- 172.21.15.18:0/1907850198 learned_addr learned my addr 172.21.15.18:0/1907850198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:06.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.225+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f65e80a3e30 con 0x7f65e8099bd0 2024-04-16T17:27:06.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.225+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f65e80a3cb0 con 0x7f65e8094e60 2024-04-16T17:27:06.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.225+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65e80a3fb0 con 0x7f65e8095800 2024-04-16T17:27:06.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.226+0000 7f65f5200640 1 -- 172.21.15.18:0/1907850198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e8095800 msgr2=0x7f65e812a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:06.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.226+0000 7f65f5200640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e8095800 0x7f65e812a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.226+0000 7f65f5200640 1 -- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e8099bd0 msgr2=0x7f65e812eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:06.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.226+0000 7f65f5200640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e8099bd0 0x7f65e812eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.226+0000 7f65f5200640 1 -- 172.21.15.18:0/1907850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e812f260 con 0x7f65e8094e60 2024-04-16T17:27:06.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.226+0000 7f65f5200640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8094e60 0x7f65e812a270 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f65e400bf20 tx=0x7f65e400dca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:06.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.227+0000 7f65ed600640 1 -- 172.21.15.18:0/1907850198 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4018070 con 0x7f65e8094e60 2024-04-16T17:27:06.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.227+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e812f550 con 0x7f65e8094e60 2024-04-16T17:27:06.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.227+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e8136de0 con 0x7f65e8094e60 2024-04-16T17:27:06.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.227+0000 7f65ed600640 1 -- 172.21.15.18:0/1907850198 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e400de90 con 0x7f65e8094e60 2024-04-16T17:27:06.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.227+0000 7f65ed600640 1 -- 172.21.15.18:0/1907850198 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e401b870 con 0x7f65e8094e60 2024-04-16T17:27:06.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.230+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d0005290 con 0x7f65e8094e60 2024-04-16T17:27:06.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.233+0000 7f65ed600640 1 -- 172.21.15.18:0/1907850198 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65e4009c30 con 0x7f65e8094e60 2024-04-16T17:27:06.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.237+0000 7f65ed600640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f65d80a3de0 0x7f65d80a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:06.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.237+0000 7f65ed600640 1 -- 172.21.15.18:0/1907850198 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f65e40d4910 con 0x7f65e8094e60 2024-04-16T17:27:06.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.237+0000 7f65efe00640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f65d80a3de0 0x7f65d80a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:06.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.238+0000 7f65efe00640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f65d80a3de0 0x7f65d80a62a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f65e80949a0 tx=0x7f65dc009260 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:06.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.242+0000 7f65ed600640 1 -- 172.21.15.18:0/1907850198 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65e40192b0 con 0x7f65e8094e60 2024-04-16T17:27:06.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:06 smithi043 ceph-mon[36901]: pgmap v716: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 466 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 395 op/s 2024-04-16T17:27:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:06 smithi022 ceph-mon[36023]: pgmap v716: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 466 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 395 op/s 2024-04-16T17:27:06.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.708+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65d0005850 con 0x7f65e8094e60 2024-04-16T17:27:06.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.710+0000 7f65ed600640 1 -- 172.21.15.18:0/1907850198 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7f65e40a0c60 con 0x7f65e8094e60 2024-04-16T17:27:06.710 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:27:06.710 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:27:06.711 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:27:06.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.713+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f65d80a3de0 msgr2=0x7f65d80a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:06.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.713+0000 7f65f6600640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f65d80a3de0 0x7f65d80a62a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f65e80949a0 tx=0x7f65dc009260 comp rx=0 tx=0).stop 2024-04-16T17:27:06.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.713+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8094e60 msgr2=0x7f65e812a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.713+0000 7f65f6600640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8094e60 0x7f65e812a270 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f65e400bf20 tx=0x7f65e400dca0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.714+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 shutdown_connections 2024-04-16T17:27:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.714+0000 7f65f6600640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f65d80a3de0 0x7f65d80a62a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.714+0000 7f65f6600640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65e8099bd0 0x7f65e812eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.714+0000 7f65f6600640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e8095800 0x7f65e812a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.714+0000 7f65f6600640 1 --2- 172.21.15.18:0/1907850198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e8094e60 0x7f65e812a270 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.714+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 >> 172.21.15.18:0/1907850198 conn(0x7f65e800a750 msgr2=0x7f65e8092c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.714+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 shutdown_connections 2024-04-16T17:27:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:06.714+0000 7f65f6600640 1 -- 172.21.15.18:0/1907850198 wait complete. 2024-04-16T17:27:06.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:27:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.052+0000 7fcf11000640 1 -- 172.21.15.18:0/4000514618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c103960 msgr2=0x7fcf0c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.052+0000 7fcf11000640 1 --2- 172.21.15.18:0/4000514618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c103960 0x7fcf0c10be40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009ca0 tx=0x7fcf0402c180 comp rx=0 tx=0).stop 2024-04-16T17:27:07.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.053+0000 7fcf11000640 1 -- 172.21.15.18:0/4000514618 shutdown_connections 2024-04-16T17:27:07.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.053+0000 7fcf11000640 1 --2- 172.21.15.18:0/4000514618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf0c10c410 0x7fcf0c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.053+0000 7fcf11000640 1 --2- 172.21.15.18:0/4000514618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c103960 0x7fcf0c10be40 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.053+0000 7fcf11000640 1 --2- 172.21.15.18:0/4000514618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf0c103040 0x7fcf0c103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.053+0000 7fcf11000640 1 -- 172.21.15.18:0/4000514618 >> 172.21.15.18:0/4000514618 conn(0x7fcf0c077ab0 msgr2=0x7fcf0c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.056+0000 7fcf11000640 1 -- 172.21.15.18:0/4000514618 shutdown_connections 2024-04-16T17:27:07.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.056+0000 7fcf11000640 1 -- 172.21.15.18:0/4000514618 wait complete. 2024-04-16T17:27:07.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.057+0000 7fcf11000640 1 Processor -- start 2024-04-16T17:27:07.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.057+0000 7fcf11000640 1 -- start start 2024-04-16T17:27:07.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf11000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf0c103040 0x7fcf0c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf11000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf0c103960 0x7fcf0c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf11000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10c410 0x7fcf0c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf11000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1121a0 con 0x7fcf0c103040 2024-04-16T17:27:07.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf11000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c112020 con 0x7fcf0c103960 2024-04-16T17:27:07.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf11000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c112320 con 0x7fcf0c10c410 2024-04-16T17:27:07.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf0c103960 0x7fcf0c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf0c103960 0x7fcf0c1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38122/0 (socket says 172.21.15.18:38122) 2024-04-16T17:27:07.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf0aa00640 1 -- 172.21.15.18:0/1858974825 learned_addr learned my addr 172.21.15.18:0/1858974825 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.058+0000 7fcf0be00640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10c410 0x7fcf0c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.059+0000 7fcf0aa00640 1 -- 172.21.15.18:0/1858974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10c410 msgr2=0x7fcf0c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.059+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10c410 0x7fcf0c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.059+0000 7fcf0aa00640 1 -- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf0c103040 msgr2=0x7fcf0c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.059+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf0c103040 0x7fcf0c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.059+0000 7fcf0aa00640 1 -- 172.21.15.18:0/1858974825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c19af50 con 0x7fcf0c103960 2024-04-16T17:27:07.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.059+0000 7fcf0be00640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10c410 0x7fcf0c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:27:07.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.061+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf0c103960 0x7fcf0c1a0bc0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcf0402c790 tx=0x7fcf04035ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.063+0000 7fcf03e00640 1 -- 172.21.15.18:0/1858974825 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04039040 con 0x7fcf0c103960 2024-04-16T17:27:07.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.063+0000 7fcf11000640 1 -- 172.21.15.18:0/1858974825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c19b1e0 con 0x7fcf0c103960 2024-04-16T17:27:07.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.063+0000 7fcf03e00640 1 -- 172.21.15.18:0/1858974825 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf04044070 con 0x7fcf0c103960 2024-04-16T17:27:07.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.063+0000 7fcf11000640 1 -- 172.21.15.18:0/1858974825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c06ba20 con 0x7fcf0c103960 2024-04-16T17:27:07.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.063+0000 7fcf03e00640 1 -- 172.21.15.18:0/1858974825 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04039040 con 0x7fcf0c103960 2024-04-16T17:27:07.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.069+0000 7fcf03e00640 1 -- 172.21.15.18:0/1858974825 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf040327f0 con 0x7fcf0c103960 2024-04-16T17:27:07.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.070+0000 7fcf153f1740 1 Processor -- start 2024-04-16T17:27:07.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.070+0000 7fcf153f1740 1 -- start start 2024-04-16T17:27:07.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174617d20 0x564174618100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x5641746296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174629c20 0x56417462c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf153f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56417457d020 con 0x5641746186d0 2024-04-16T17:27:07.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf153f1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564174523460 con 0x564174629c20 2024-04-16T17:27:07.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf153f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564174578fe0 con 0x564174617d20 2024-04-16T17:27:07.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf0be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174617d20 0x564174618100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf0be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174617d20 0x564174618100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48864/0 (socket says 172.21.15.18:48864) 2024-04-16T17:27:07.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf0be00640 1 -- 172.21.15.18:0/2333140167 learned_addr learned my addr 172.21.15.18:0/2333140167 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174629c20 0x56417462c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.071+0000 7fcf0b400640 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x5641746296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.074+0000 7fcf0be00640 1 -- 172.21.15.18:0/2333140167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174629c20 msgr2=0x56417462c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.074+0000 7fcf0be00640 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174629c20 0x56417462c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.074+0000 7fcf0be00640 1 -- 172.21.15.18:0/2333140167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 msgr2=0x5641746296e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.074+0000 7fcf0be00640 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x5641746296e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.074+0000 7fcf0be00640 1 -- 172.21.15.18:0/2333140167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417452b040 con 0x564174617d20 2024-04-16T17:27:07.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.074+0000 7fcf03e00640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcef00a3fa0 0x7fcef00a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.074+0000 7fcf0be00640 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174617d20 0x564174618100 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcef800bcd0 tx=0x7fcef800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c184e90ef5fe2c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.075+0000 7fcf03e00640 1 -- 172.21.15.18:0/1858974825 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fcf040f2bf0 con 0x7fcf0c103960 2024-04-16T17:27:07.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.075+0000 7fcf00c00640 1 -- 172.21.15.18:0/2333140167 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8015070 con 0x564174617d20 2024-04-16T17:27:07.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.076+0000 7fcf00c00640 1 -- 172.21.15.18:0/2333140167 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef8005500 con 0x564174617d20 2024-04-16T17:27:07.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.076+0000 7fcf00c00640 1 -- 172.21.15.18:0/2333140167 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80116a0 con 0x564174617d20 2024-04-16T17:27:07.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.076+0000 7fcf0be00640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcef00a3fa0 0x7fcef00a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.077+0000 7fcf153f1740 1 -- 172.21.15.18:0/2333140167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174617d20 msgr2=0x564174618100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.077+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174617d20 0x564174618100 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcef800bcd0 tx=0x7fcef800f730 comp rx=0 tx=0).stop 2024-04-16T17:27:07.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.078+0000 7fcf153f1740 1 -- 172.21.15.18:0/2333140167 shutdown_connections 2024-04-16T17:27:07.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.078+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174629c20 0x56417462c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.078+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x5641746296e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.078+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2333140167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174617d20 0x564174618100 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.078+0000 7fcf153f1740 1 -- 172.21.15.18:0/2333140167 >> 172.21.15.18:0/2333140167 conn(0x564174612640 msgr2=0x564174612a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.078+0000 7fcf0be00640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcef00a3fa0 0x7fcef00a6460 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcef8011fd0 tx=0x7fcef80119c0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.079+0000 7fcf153f1740 1 -- 172.21.15.18:0/2333140167 shutdown_connections 2024-04-16T17:27:07.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.079+0000 7fcf153f1740 1 -- 172.21.15.18:0/2333140167 wait complete. 2024-04-16T17:27:07.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.079+0000 7fcf153f1740 1 -- start start 2024-04-16T17:27:07.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.080+0000 7fcf153f1740 1 Processor -- start 2024-04-16T17:27:07.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.080+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564174617d20 0x564174623730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.080+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641746186d0 0x564174623c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.080+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174629450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641746186d0 0x564174623c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf0be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174629450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641746186d0 0x564174623c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38138/0 (socket says 172.21.15.18:38138) 2024-04-16T17:27:07.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf0be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174629450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48878/0 (socket says 172.21.15.18:48878) 2024-04-16T17:27:07.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf153f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5641746191f0 con 0x564174617d20 2024-04-16T17:27:07.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf153f1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564174536830 con 0x5641746186d0 2024-04-16T17:27:07.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf153f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564174619370 con 0x564174629c20 2024-04-16T17:27:07.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf153f1740 10 client.0 _ll_register_callbacks cb 0x56417453e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:27:07.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.081+0000 7fcf0b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564174617d20 0x564174623730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.082+0000 7fcf0aa00640 1 -- 172.21.15.18:0/72443476 learned_addr learned my addr 172.21.15.18:0/72443476 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.083+0000 7fcf0aa00640 1 -- 172.21.15.18:0/72443476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 msgr2=0x564174629450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.083+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174629450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.083+0000 7fcf0aa00640 1 -- 172.21.15.18:0/72443476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564174617d20 msgr2=0x564174623730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.083+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564174617d20 0x564174623730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.083+0000 7fcf0aa00640 1 -- 172.21.15.18:0/72443476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417453a920 con 0x5641746186d0 2024-04-16T17:27:07.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.085+0000 7fcf0be00640 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174629450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:27:07.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.085+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641746186d0 0x564174623c70 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fcf040f0050 tx=0x7fcf04009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5ab49930a42841 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.085+0000 7fcef5600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:27:07.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.085+0000 7fcf0b400640 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564174617d20 0x564174623730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:27:07.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.085+0000 7fcef5600640 1 -- 172.21.15.18:0/72443476 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040bf080 con 0x5641746186d0 2024-04-16T17:27:07.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.085+0000 7fcf153f1740 1 -- 172.21.15.18:0/72443476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef8005500 con 0x5641746186d0 2024-04-16T17:27:07.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.085+0000 7fcef5600640 1 -- 172.21.15.18:0/72443476 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fcf040bbfc0 con 0x5641746186d0 2024-04-16T17:27:07.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcef5600640 1 -- 172.21.15.18:0/72443476 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcf040c2bf0 con 0x5641746186d0 2024-04-16T17:27:07.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcf153f1740 10 client.31268 fetch_fsmap learned FSMap version 37 2024-04-16T17:27:07.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcf153f1740 1 -- 172.21.15.18:0/72443476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5641745087f0 con 0x5641746186d0 2024-04-16T17:27:07.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcef5600640 1 -- 172.21.15.18:0/72443476 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef8005500 con 0x5641746186d0 2024-04-16T17:27:07.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcef5600640 1 -- 172.21.15.18:0/72443476 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fcf0403c030 con 0x5641746186d0 2024-04-16T17:27:07.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcf153f1740 10 client.31268 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:27:07.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcf153f1740 10 client.31268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:27:07.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcf153f1740 20 client.31268 populate_metadata read hostname 'smithi018' 2024-04-16T17:27:07.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.086+0000 7fcf153f1740 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641746426c0 0x564174644b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.087+0000 7fcf153f1740 4 client.31268 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:27:07.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.087+0000 7fcf153f1740 1 -- 172.21.15.18:0/72443476 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56417458aa40 con 0x5641746426c0 2024-04-16T17:27:07.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.088+0000 7fcf0b400640 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641746426c0 0x564174644b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.093+0000 7fcf0b400640 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641746426c0 0x564174644b60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3127d1cd15df41fa server_cookie=312eecf1bad0c80e in_seq=0 out_seq=0 2024-04-16T17:27:07.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.093+0000 7fcef5600640 10 client.31268 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:27:07.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.095+0000 7fcef5600640 1 -- 172.21.15.18:0/72443476 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56417458aa40 con 0x5641746426c0 2024-04-16T17:27:07.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.095+0000 7fcef5600640 10 client.31268 handle_command_reply: tid=0 2024-04-16T17:27:07.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.095+0000 7fcf153f1740 1 client.31268 shutdown 2024-04-16T17:27:07.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 -- 172.21.15.18:0/72443476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641746186d0 msgr2=0x564174623c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641746186d0 0x564174623c70 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fcf040f0050 tx=0x7fcf04009800 comp rx=0 tx=0).stop 2024-04-16T17:27:07.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 -- 172.21.15.18:0/72443476 shutdown_connections 2024-04-16T17:27:07.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641746426c0 0x564174644b60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174629450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5641746186d0 0x564174623c70 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcef5600640 0 client.31268 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:27:07.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 --2- 172.21.15.18:0/72443476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564174617d20 0x564174623730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 -- 172.21.15.18:0/72443476 >> 172.21.15.18:0/72443476 conn(0x564174612640 msgr2=0x564174614390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.096+0000 7fcf153f1740 1 -- 172.21.15.18:0/72443476 shutdown_connections 2024-04-16T17:27:07.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.097+0000 7fcf153f1740 1 -- 172.21.15.18:0/72443476 wait complete. 2024-04-16T17:27:07.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.097+0000 7fcf153f1740 20 client.31268 trim_cache size 0 max 16384 2024-04-16T17:27:07.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.141+0000 7fcf153f1740 1 Processor -- start 2024-04-16T17:27:07.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.141+0000 7fcf153f1740 1 -- start start 2024-04-16T17:27:07.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.141+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x56417461dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.141+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x564174611580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.141+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x56417464a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.141+0000 7fcf153f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fceec012a70 con 0x5641746186d0 2024-04-16T17:27:07.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.141+0000 7fcf153f1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fceec010340 con 0x564174617d20 2024-04-16T17:27:07.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.141+0000 7fcf153f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fceec00e1b0 con 0x564174629c20 2024-04-16T17:27:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x56417461dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x56417461dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38146/0 (socket says 172.21.15.18:38146) 2024-04-16T17:27:07.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0b400640 1 -- 172.21.15.18:0/3200343832 learned_addr learned my addr 172.21.15.18:0/3200343832 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0be00640 1 --2- 172.21.15.18:0/3200343832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x56417464a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0b400640 1 -- 172.21.15.18:0/3200343832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 msgr2=0x56417464a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0b400640 1 --2- 172.21.15.18:0/3200343832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x56417464a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0b400640 1 -- 172.21.15.18:0/3200343832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 msgr2=0x564174611580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0b400640 1 --2- 172.21.15.18:0/3200343832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x564174611580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.142+0000 7fcf0b400640 1 -- 172.21.15.18:0/3200343832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417428f3f0 con 0x564174617d20 2024-04-16T17:27:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.143+0000 7fcf0b400640 1 --2- 172.21.15.18:0/3200343832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x56417461dcb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcefc005d00 tx=0x7fcefc00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5cacb1492be14d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.143+0000 7fcef7e00640 1 -- 172.21.15.18:0/3200343832 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc003de0 con 0x564174617d20 2024-04-16T17:27:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.144+0000 7fcef7e00640 1 -- 172.21.15.18:0/3200343832 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc01d530 con 0x564174617d20 2024-04-16T17:27:07.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.144+0000 7fcef7e00640 1 -- 172.21.15.18:0/3200343832 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc011c00 con 0x564174617d20 2024-04-16T17:27:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 -- 172.21.15.18:0/3200343832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 msgr2=0x56417461dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 --2- 172.21.15.18:0/3200343832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x56417461dcb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcefc005d00 tx=0x7fcefc00bea0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 -- 172.21.15.18:0/3200343832 shutdown_connections 2024-04-16T17:27:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 --2- 172.21.15.18:0/3200343832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x56417464a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 --2- 172.21.15.18:0/3200343832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x564174611580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 --2- 172.21.15.18:0/3200343832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x56417461dcb0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 -- 172.21.15.18:0/3200343832 >> 172.21.15.18:0/3200343832 conn(0x564174612640 msgr2=0x564174610e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 -- 172.21.15.18:0/3200343832 shutdown_connections 2024-04-16T17:27:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.145+0000 7fcf153f1740 1 -- 172.21.15.18:0/3200343832 wait complete. 2024-04-16T17:27:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.146+0000 7fcf153f1740 1 -- start start 2024-04-16T17:27:07.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.148+0000 7fcf153f1740 1 Processor -- start 2024-04-16T17:27:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.148+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x564174655f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.148+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x564174653330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.148+0000 7fcf153f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174653940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf0b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x564174655f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf0b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x564174655f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38148/0 (socket says 172.21.15.18:38148) 2024-04-16T17:27:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf153f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fceec00e800 con 0x5641746186d0 2024-04-16T17:27:07.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf153f1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fceec00fc70 con 0x564174617d20 2024-04-16T17:27:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf153f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fceec00eed0 con 0x564174629c20 2024-04-16T17:27:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf153f1740 10 client.0 _ll_register_callbacks cb 0x7fcf04005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:27:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf0b400640 1 -- 172.21.15.18:0/2675729032 learned_addr learned my addr 172.21.15.18:0/2675729032 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf0be00640 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174653940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.149+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x564174653330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.150+0000 7fcf0b400640 1 -- 172.21.15.18:0/2675729032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 msgr2=0x564174653940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.150+0000 7fcf0b400640 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174653940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.150+0000 7fcf0b400640 1 -- 172.21.15.18:0/2675729032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 msgr2=0x564174653330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.150+0000 7fcf0b400640 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x564174653330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.150+0000 7fcf0b400640 1 -- 172.21.15.18:0/2675729032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641743597d0 con 0x564174617d20 2024-04-16T17:27:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.150+0000 7fcf0b400640 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x564174655f50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcefc011560 tx=0x7fcefc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb52f4d439723a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.150+0000 7fcef5600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:27:07.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.150+0000 7fcef5600640 1 -- 172.21.15.18:0/2675729032 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc01d8e0 con 0x564174617d20 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.151+0000 7fcef5600640 1 -- 172.21.15.18:0/2675729032 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fcefc003be0 con 0x564174617d20 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.151+0000 7fcf153f1740 1 -- 172.21.15.18:0/2675729032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc01d530 con 0x564174617d20 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.151+0000 7fcef5600640 1 -- 172.21.15.18:0/2675729032 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcefc003de0 con 0x564174617d20 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.151+0000 7fcf153f1740 10 client.31274 fetch_fsmap learned FSMap version 37 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.151+0000 7fcf153f1740 1 -- 172.21.15.18:0/2675729032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5641746a8740 con 0x564174617d20 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.151+0000 7fcef5600640 1 -- 172.21.15.18:0/2675729032 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc01d530 con 0x564174617d20 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.151+0000 7fcef5600640 1 -- 172.21.15.18:0/2675729032 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fcefc02d030 con 0x564174617d20 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.152+0000 7fcf153f1740 10 client.31274 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:27:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.152+0000 7fcf153f1740 10 client.31274 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:27:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.152+0000 7fcf153f1740 20 client.31274 populate_metadata read hostname 'smithi018' 2024-04-16T17:27:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.152+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641746a8e30 0x5641746ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.152+0000 7fcf153f1740 4 client.31274 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:27:07.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.152+0000 7fcf153f1740 1 -- 172.21.15.18:0/2675729032 --> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641745848b0 con 0x5641746a8e30 2024-04-16T17:27:07.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.152+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641746a8e30 0x5641746ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.153+0000 7fcef5600640 10 client.31274 ms_handle_connect on v2:172.21.15.43:6838/3993514244 2024-04-16T17:27:07.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.153+0000 7fcf0aa00640 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641746a8e30 0x5641746ab220 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9eaf7e57ac63fbc1 server_cookie=ee7d0cbe6daef155 in_seq=0 out_seq=0 2024-04-16T17:27:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.153+0000 7fcef5600640 1 -- 172.21.15.18:0/2675729032 <== mds.2 v2:172.21.15.43:6838/3993514244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641745848b0 con 0x5641746a8e30 2024-04-16T17:27:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.153+0000 7fcef5600640 10 client.31274 handle_command_reply: tid=0 2024-04-16T17:27:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.153+0000 7fcf153f1740 1 client.31274 shutdown 2024-04-16T17:27:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.154+0000 7fcf153f1740 1 -- 172.21.15.18:0/2675729032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 msgr2=0x564174655f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.154+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x564174655f50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcefc011560 tx=0x7fcefc0046c0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 1 -- 172.21.15.18:0/2675729032 shutdown_connections 2024-04-16T17:27:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.43:6838/3993514244,v1:172.21.15.43:6839/3993514244] conn(0x5641746a8e30 0x5641746ab220 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564174629c20 0x564174653940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5641746186d0 0x564174653330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 1 --2- 172.21.15.18:0/2675729032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564174617d20 0x564174655f50 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcef5600640 0 client.31274 ms_handle_reset on v2:172.21.15.43:6838/3993514244 2024-04-16T17:27:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 1 -- 172.21.15.18:0/2675729032 >> 172.21.15.18:0/2675729032 conn(0x564174612640 msgr2=0x564174611700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 1 -- 172.21.15.18:0/2675729032 shutdown_connections 2024-04-16T17:27:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 1 -- 172.21.15.18:0/2675729032 wait complete. 2024-04-16T17:27:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.155+0000 7fcf153f1740 20 client.31274 trim_cache size 0 max 16384 2024-04-16T17:27:07.158 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:27:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 -- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcef00a3fa0 msgr2=0x7fcef00a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcef00a3fa0 0x7fcef00a6460 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcef8011fd0 tx=0x7fcef80119c0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 -- 172.21.15.18:0/1858974825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf0c103960 msgr2=0x7fcf0c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf0c103960 0x7fcf0c1a0bc0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcf0402c790 tx=0x7fcf04035ed0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 -- 172.21.15.18:0/1858974825 shutdown_connections 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcef00a3fa0 0x7fcef00a6460 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf0c10c410 0x7fcf0c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf0c103960 0x7fcf0c1a0bc0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 --2- 172.21.15.18:0/1858974825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf0c103040 0x7fcf0c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.156+0000 7fcf11000640 1 -- 172.21.15.18:0/1858974825 >> 172.21.15.18:0/1858974825 conn(0x7fcf0c077ab0 msgr2=0x7fcf0c100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.157+0000 7fcf11000640 1 -- 172.21.15.18:0/1858974825 shutdown_connections 2024-04-16T17:27:07.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.157+0000 7fcf11000640 1 -- 172.21.15.18:0/1858974825 wait complete. 2024-04-16T17:27:07.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:27:07.397 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:07 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1907850198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:27:07.415 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:07 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1907850198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:27:07.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:07 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1907850198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:27:07.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.494+0000 7f8629e00640 1 -- 172.21.15.18:0/3458261610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8624102be0 msgr2=0x7f8624106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.494+0000 7f8629e00640 1 --2- 172.21.15.18:0/3458261610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8624102be0 0x7f8624106c20 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8610009900 tx=0x7f861002bc30 comp rx=0 tx=0).stop 2024-04-16T17:27:07.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.496+0000 7f8629e00640 1 -- 172.21.15.18:0/3458261610 shutdown_connections 2024-04-16T17:27:07.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.496+0000 7f8629e00640 1 --2- 172.21.15.18:0/3458261610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8624107300 0x7f862410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.496+0000 7f8629e00640 1 --2- 172.21.15.18:0/3458261610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8624102be0 0x7f8624106c20 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.496+0000 7f8629e00640 1 --2- 172.21.15.18:0/3458261610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8624102230 0x7f8624102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.496+0000 7f8629e00640 1 -- 172.21.15.18:0/3458261610 >> 172.21.15.18:0/3458261610 conn(0x7f86240fdab0 msgr2=0x7f86240ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.496+0000 7f8629e00640 1 -- 172.21.15.18:0/3458261610 shutdown_connections 2024-04-16T17:27:07.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.497+0000 7f8629e00640 1 -- 172.21.15.18:0/3458261610 wait complete. 2024-04-16T17:27:07.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.498+0000 7f8629e00640 1 Processor -- start 2024-04-16T17:27:07.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.498+0000 7f8629e00640 1 -- start start 2024-04-16T17:27:07.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.499+0000 7f8629e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8624102230 0x7f862410e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8629e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8624107300 0x7f862410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8629e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86241a3920 0x7f86241a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8629e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8624112440 con 0x7f8624107300 2024-04-16T17:27:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8629e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86241122c0 con 0x7f86241a3920 2024-04-16T17:27:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8629e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86241125c0 con 0x7f8624102230 2024-04-16T17:27:07.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8622a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8624102230 0x7f862410e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8622a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8624102230 0x7f862410e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48930/0 (socket says 172.21.15.18:48930) 2024-04-16T17:27:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8622a00640 1 -- 172.21.15.18:0/2081081721 learned_addr learned my addr 172.21.15.18:0/2081081721 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.500+0000 7f8623400640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86241a3920 0x7f86241a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.501+0000 7f8622000640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8624107300 0x7f862410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.501+0000 7f8622a00640 1 -- 172.21.15.18:0/2081081721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86241a3920 msgr2=0x7f86241a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.501+0000 7f8622a00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86241a3920 0x7f86241a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.501+0000 7f8622a00640 1 -- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8624107300 msgr2=0x7f862410e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.501+0000 7f8622a00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8624107300 0x7f862410e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.501+0000 7f8622a00640 1 -- 172.21.15.18:0/2081081721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86241a4410 con 0x7f8624102230 2024-04-16T17:27:07.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.505+0000 7f8622a00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8624102230 0x7f862410e180 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f861400d950 tx=0x7f861400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.506+0000 7f8628a00640 1 -- 172.21.15.18:0/2081081721 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614017070 con 0x7f8624102230 2024-04-16T17:27:07.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.506+0000 7f8629e00640 1 -- 172.21.15.18:0/2081081721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86241a4700 con 0x7f8624102230 2024-04-16T17:27:07.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.506+0000 7f8628a00640 1 -- 172.21.15.18:0/2081081721 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8614009c30 con 0x7f8624102230 2024-04-16T17:27:07.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.506+0000 7f8629e00640 1 -- 172.21.15.18:0/2081081721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86241abed0 con 0x7f8624102230 2024-04-16T17:27:07.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.506+0000 7f8628a00640 1 -- 172.21.15.18:0/2081081721 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861400f6b0 con 0x7f8624102230 2024-04-16T17:27:07.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.512+0000 7f862cb90740 1 Processor -- start 2024-04-16T17:27:07.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.512+0000 7f862cb90740 1 -- start start 2024-04-16T17:27:07.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.513+0000 7f862cb90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563ad7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.513+0000 7f862cb90740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad7670 0x55d563adfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.513+0000 7f862cb90740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ae0250 0x55d563ae2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.513+0000 7f862cb90740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d563a3c020 con 0x55d563ae0250 2024-04-16T17:27:07.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.513+0000 7f862cb90740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d5639e2460 con 0x55d563ad7670 2024-04-16T17:27:07.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.513+0000 7f862cb90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d563a37fe0 con 0x55d563ad6d50 2024-04-16T17:27:07.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.514+0000 7f8623400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563ad7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.514+0000 7f8623400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563ad7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48944/0 (socket says 172.21.15.18:48944) 2024-04-16T17:27:07.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.514+0000 7f8623400640 1 -- 172.21.15.18:0/797364830 learned_addr learned my addr 172.21.15.18:0/797364830 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.515+0000 7f8628a00640 1 -- 172.21.15.18:0/2081081721 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f86140109d0 con 0x7f8624102230 2024-04-16T17:27:07.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.515+0000 7f8622a00640 1 --2- 172.21.15.18:0/797364830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad7670 0x55d563adfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.516+0000 7f8622a00640 1 -- 172.21.15.18:0/797364830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 msgr2=0x55d563ad7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.516+0000 7f8622a00640 1 --2- 172.21.15.18:0/797364830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563ad7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.516+0000 7f8622a00640 1 -- 172.21.15.18:0/797364830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ae0250 msgr2=0x55d563ae2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.516+0000 7f8622a00640 1 --2- 172.21.15.18:0/797364830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ae0250 0x55d563ae2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.516+0000 7f8622a00640 1 -- 172.21.15.18:0/797364830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5639ea040 con 0x55d563ad7670 2024-04-16T17:27:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.519+0000 7f8628a00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f86080a3f20 0x7f86080a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.519+0000 7f8628a00640 1 -- 172.21.15.18:0/2081081721 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f861400f450 con 0x7f8624102230 2024-04-16T17:27:07.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.520+0000 7f8623400640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f86080a3f20 0x7f86080a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.521+0000 7f8622a00640 1 --2- 172.21.15.18:0/797364830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad7670 0x55d563adfd10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f86140cd950 tx=0x7f86140d6ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35b7717a6653f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.521+0000 7f860fe00640 1 -- 172.21.15.18:0/797364830 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614098440 con 0x55d563ad7670 2024-04-16T17:27:07.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.521+0000 7f860fe00640 1 -- 172.21.15.18:0/797364830 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8614009de0 con 0x55d563ad7670 2024-04-16T17:27:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.521+0000 7f860fe00640 1 -- 172.21.15.18:0/797364830 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861409bb00 con 0x55d563ad7670 2024-04-16T17:27:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.522+0000 7f862cb90740 1 -- 172.21.15.18:0/797364830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad7670 msgr2=0x55d563adfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.522+0000 7f862cb90740 1 --2- 172.21.15.18:0/797364830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad7670 0x55d563adfd10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f86140cd950 tx=0x7f86140d6ae0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.523+0000 7f8623400640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f86080a3f20 0x7f86080a63e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f861c00a510 tx=0x7f861c00d040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.524+0000 7f862cb90740 1 -- 172.21.15.18:0/797364830 shutdown_connections 2024-04-16T17:27:07.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.524+0000 7f862cb90740 1 --2- 172.21.15.18:0/797364830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ae0250 0x55d563ae2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.524+0000 7f862cb90740 1 --2- 172.21.15.18:0/797364830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad7670 0x55d563adfd10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.524+0000 7f862cb90740 1 --2- 172.21.15.18:0/797364830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563ad7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.524+0000 7f862cb90740 1 -- 172.21.15.18:0/797364830 >> 172.21.15.18:0/797364830 conn(0x55d563ad1480 msgr2=0x55d563ad1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.524+0000 7f862cb90740 1 -- 172.21.15.18:0/797364830 shutdown_connections 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.524+0000 7f862cb90740 1 -- 172.21.15.18:0/797364830 wait complete. 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.525+0000 7f862cb90740 1 -- start start 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.525+0000 7f862cb90740 1 Processor -- start 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.526+0000 7f862cb90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563add380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.526+0000 7f862cb90740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 0x55d563add8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.526+0000 7f8622a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563add380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.526+0000 7f8622a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563add380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48960/0 (socket says 172.21.15.18:48960) 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.526+0000 7f8622000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 0x55d563add8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.526+0000 7f8622000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 0x55d563add8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59038/0 (socket says 172.21.15.18:59038) 2024-04-16T17:27:07.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.527+0000 7f8622a00640 1 -- 172.21.15.18:0/2173944803 learned_addr learned my addr 172.21.15.18:0/2173944803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.527+0000 7f862cb90740 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 0x55d563adde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.527+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d563ae3040 con 0x55d563ad7670 2024-04-16T17:27:07.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.527+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d5639f5830 con 0x55d563ae0250 2024-04-16T17:27:07.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.527+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d563ae31c0 con 0x55d563ad6d50 2024-04-16T17:27:07.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.527+0000 7f8623400640 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 0x55d563adde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.527+0000 7f862cb90740 10 client.0 _ll_register_callbacks cb 0x55d5639fd150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:27:07.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f8622a00640 1 -- 172.21.15.18:0/2173944803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 msgr2=0x55d563adde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f8622a00640 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 0x55d563adde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f8622a00640 1 -- 172.21.15.18:0/2173944803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 msgr2=0x55d563add8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f8622a00640 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 0x55d563add8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f8622a00640 1 -- 172.21.15.18:0/2173944803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5639f9920 con 0x55d563ad6d50 2024-04-16T17:27:07.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8614009de0 con 0x55d563ad6d50 2024-04-16T17:27:07.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f860cc00640 10 client.31129 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-16T17:27:07.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f8622a00640 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563add380 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8614010fd0 tx=0x7f8614010d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c161c6c6ef22883 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f860cc00640 1 -- 172.21.15.18:0/2173944803 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614020040 con 0x55d563ad6d50 2024-04-16T17:27:07.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.528+0000 7f860cc00640 1 -- 172.21.15.18:0/2173944803 <== mon.1 v2:172.21.15.43:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f861409a4a0 con 0x55d563ad6d50 2024-04-16T17:27:07.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.529+0000 7f860cc00640 1 -- 172.21.15.18:0/2173944803 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8614020040 con 0x55d563ad6d50 2024-04-16T17:27:07.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.529+0000 7f862cb90740 10 client.31129 fetch_fsmap learned FSMap version 37 2024-04-16T17:27:07.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.529+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55d5639c77f0 con 0x55d563ad6d50 2024-04-16T17:27:07.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.531+0000 7f860cc00640 1 -- 172.21.15.18:0/2173944803 <== mon.1 v2:172.21.15.43:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f8614098810 con 0x55d563ad6d50 2024-04-16T17:27:07.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.531+0000 7f862cb90740 10 client.31129 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:27:07.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.531+0000 7f862cb90740 10 client.31129 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:27:07.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.531+0000 7f862cb90740 20 client.31129 populate_metadata read hostname 'smithi018' 2024-04-16T17:27:07.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.531+0000 7f862cb90740 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d563b01400 0x55d563b037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.531+0000 7f862cb90740 4 client.31129 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:27:07.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.531+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d563a49a40 con 0x55d563b01400 2024-04-16T17:27:07.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.532+0000 7f8622000640 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d563b01400 0x55d563b037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.533+0000 7f8622000640 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d563b01400 0x55d563b037f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=556cd914e48ddd4b server_cookie=80d9864f0e9d8f8c in_seq=0 out_seq=0 2024-04-16T17:27:07.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.533+0000 7f860cc00640 10 client.31129 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:27:07.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.534+0000 7f860cc00640 1 -- 172.21.15.18:0/2173944803 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d563a49a40 con 0x55d563b01400 2024-04-16T17:27:07.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.534+0000 7f860cc00640 10 client.31129 handle_command_reply: tid=0 2024-04-16T17:27:07.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.534+0000 7f862cb90740 1 client.31129 shutdown 2024-04-16T17:27:07.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 msgr2=0x55d563add380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f862cb90740 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563add380 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8614010fd0 tx=0x7f8614010d70 comp rx=0 tx=0).stop 2024-04-16T17:27:07.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 shutdown_connections 2024-04-16T17:27:07.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f862cb90740 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d563b01400 0x55d563b037f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f862cb90740 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 0x55d563adde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f862cb90740 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 0x55d563add8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f862cb90740 1 --2- 172.21.15.18:0/2173944803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563add380 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f860cc00640 0 client.31129 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:27:07.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.535+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 >> 172.21.15.18:0/2173944803 conn(0x55d563ad1480 msgr2=0x55d563ad31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.536+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 shutdown_connections 2024-04-16T17:27:07.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.536+0000 7f862cb90740 1 -- 172.21.15.18:0/2173944803 wait complete. 2024-04-16T17:27:07.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.536+0000 7f862cb90740 20 client.31129 trim_cache size 0 max 16384 2024-04-16T17:27:07.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.568+0000 7f862cb90740 1 Processor -- start 2024-04-16T17:27:07.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.568+0000 7f862cb90740 1 -- start start 2024-04-16T17:27:07.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.568+0000 7f862cb90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563adfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.568+0000 7f862cb90740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 0x55d563afcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.568+0000 7f862cb90740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 0x55d563addc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8622a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563adfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8622a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563adfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:48970/0 (socket says 172.21.15.18:48970) 2024-04-16T17:27:07.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f862cb90740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f86241b0230 con 0x55d563ad7670 2024-04-16T17:27:07.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f862cb90740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86241adb00 con 0x55d563ae0250 2024-04-16T17:27:07.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f862cb90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8624073950 con 0x55d563ad6d50 2024-04-16T17:27:07.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8622a00640 1 -- 172.21.15.18:0/1560272559 learned_addr learned my addr 172.21.15.18:0/1560272559 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8623400640 1 --2- 172.21.15.18:0/1560272559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 0x55d563addc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8622a00640 1 -- 172.21.15.18:0/1560272559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 msgr2=0x55d563addc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8622a00640 1 --2- 172.21.15.18:0/1560272559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 0x55d563addc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8622a00640 1 -- 172.21.15.18:0/1560272559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 msgr2=0x55d563afcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8622a00640 1 --2- 172.21.15.18:0/1560272559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 0x55d563afcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.569+0000 7f8622a00640 1 -- 172.21.15.18:0/1560272559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d563ad74a0 con 0x55d563ad6d50 2024-04-16T17:27:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.570+0000 7f8622a00640 1 --2- 172.21.15.18:0/1560272559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563adfab0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8614009610 tx=0x7f86140d2000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524ac9890ec0b79c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.570+0000 7f860f400640 1 -- 172.21.15.18:0/1560272559 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614020040 con 0x55d563ad6d50 2024-04-16T17:27:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.570+0000 7f860f400640 1 -- 172.21.15.18:0/1560272559 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8614009de0 con 0x55d563ad6d50 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.570+0000 7f860f400640 1 -- 172.21.15.18:0/1560272559 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861409bce0 con 0x55d563ad6d50 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 -- 172.21.15.18:0/1560272559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 msgr2=0x55d563adfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 --2- 172.21.15.18:0/1560272559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563adfab0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8614009610 tx=0x7f86140d2000 comp rx=0 tx=0).stop 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 -- 172.21.15.18:0/1560272559 shutdown_connections 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 --2- 172.21.15.18:0/1560272559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ae0250 0x55d563addc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 --2- 172.21.15.18:0/1560272559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ad7670 0x55d563afcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 --2- 172.21.15.18:0/1560272559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad6d50 0x55d563adfab0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 -- 172.21.15.18:0/1560272559 >> 172.21.15.18:0/1560272559 conn(0x55d563ad1480 msgr2=0x55d563afc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 -- 172.21.15.18:0/1560272559 shutdown_connections 2024-04-16T17:27:07.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.571+0000 7f862cb90740 1 -- 172.21.15.18:0/1560272559 wait complete. 2024-04-16T17:27:07.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.572+0000 7f862cb90740 1 -- start start 2024-04-16T17:27:07.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.572+0000 7f862cb90740 1 Processor -- start 2024-04-16T17:27:07.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f862cb90740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad6d50 0x55d563b068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f862cb90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad7670 0x55d563b03c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f862cb90740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ae0250 0x55d563b041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f862cb90740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8624074020 con 0x55d563ae0250 2024-04-16T17:27:07.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f862cb90740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86241ad430 con 0x55d563ad6d50 2024-04-16T17:27:07.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f862cb90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86241ac690 con 0x55d563ad7670 2024-04-16T17:27:07.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f862cb90740 10 client.0 _ll_register_callbacks cb 0x7f861c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:27:07.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f8622a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad6d50 0x55d563b068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f8622a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad6d50 0x55d563b068d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38192/0 (socket says 172.21.15.18:38192) 2024-04-16T17:27:07.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.575+0000 7f8622a00640 1 -- 172.21.15.18:0/1442770533 learned_addr learned my addr 172.21.15.18:0/1442770533 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.576+0000 7f8622a00640 1 -- 172.21.15.18:0/1442770533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad7670 msgr2=0x55d563b03c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.576+0000 7f8622a00640 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad7670 0x55d563b03c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.576+0000 7f8622a00640 1 -- 172.21.15.18:0/1442770533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ae0250 msgr2=0x55d563b041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.576+0000 7f8622a00640 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ae0250 0x55d563b041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.576+0000 7f8622a00640 1 -- 172.21.15.18:0/1442770533 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d56374e3f0 con 0x55d563ad6d50 2024-04-16T17:27:07.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.576+0000 7f860cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:27:07.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.576+0000 7f8622a00640 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad6d50 0x55d563b068d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f861409bea0 tx=0x7f8614005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95496d05ba41ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.577+0000 7f860cc00640 1 -- 172.21.15.18:0/1442770533 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861407f070 con 0x55d563ad6d50 2024-04-16T17:27:07.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.577+0000 7f862cb90740 1 -- 172.21.15.18:0/1442770533 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8614009de0 con 0x55d563ad6d50 2024-04-16T17:27:07.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.577+0000 7f860cc00640 1 -- 172.21.15.18:0/1442770533 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f861409a4a0 con 0x55d563ad6d50 2024-04-16T17:27:07.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.577+0000 7f860cc00640 1 -- 172.21.15.18:0/1442770533 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f861407f070 con 0x55d563ad6d50 2024-04-16T17:27:07.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.577+0000 7f862cb90740 10 client.31292 fetch_fsmap learned FSMap version 37 2024-04-16T17:27:07.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.577+0000 7f862cb90740 1 -- 172.21.15.18:0/1442770533 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d5638187d0 con 0x55d563ad6d50 2024-04-16T17:27:07.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.578+0000 7f860cc00640 1 -- 172.21.15.18:0/1442770533 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861409c8a0 con 0x55d563ad6d50 2024-04-16T17:27:07.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.578+0000 7f860cc00640 1 -- 172.21.15.18:0/1442770533 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f86140a4980 con 0x55d563ad6d50 2024-04-16T17:27:07.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.578+0000 7f862cb90740 10 client.31292 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:27:07.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.578+0000 7f862cb90740 10 client.31292 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-16T17:27:07.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.578+0000 7f862cb90740 20 client.31292 populate_metadata read hostname 'smithi018' 2024-04-16T17:27:07.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.578+0000 7f862cb90740 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d563b672b0 0x55d563b696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.578+0000 7f862cb90740 4 client.31292 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:27:07.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.578+0000 7f862cb90740 1 -- 172.21.15.18:0/1442770533 --> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8614009de0 con 0x55d563b672b0 2024-04-16T17:27:07.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.579+0000 7f8622000640 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d563b672b0 0x55d563b696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.579+0000 7f860cc00640 10 client.31292 ms_handle_connect on v2:172.21.15.18:6838/294785837 2024-04-16T17:27:07.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.579+0000 7f8622000640 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d563b672b0 0x55d563b696f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b1cb0498480ef35 server_cookie=74f7d3a6a0f321da in_seq=0 out_seq=0 2024-04-16T17:27:07.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.581+0000 7f860cc00640 1 -- 172.21.15.18:0/1442770533 <== mds.1 v2:172.21.15.18:6838/294785837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8614009de0 con 0x55d563b672b0 2024-04-16T17:27:07.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.581+0000 7f860cc00640 10 client.31292 handle_command_reply: tid=0 2024-04-16T17:27:07.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.581+0000 7f862cb90740 1 client.31292 shutdown 2024-04-16T17:27:07.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.581+0000 7f862cb90740 1 -- 172.21.15.18:0/1442770533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad6d50 msgr2=0x55d563b068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.581+0000 7f862cb90740 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad6d50 0x55d563b068d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f861409bea0 tx=0x7f8614005880 comp rx=0 tx=0).stop 2024-04-16T17:27:07.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 1 -- 172.21.15.18:0/1442770533 shutdown_connections 2024-04-16T17:27:07.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.18:6838/294785837,v1:172.21.15.18:6839/294785837] conn(0x55d563b672b0 0x55d563b696f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d563ae0250 0x55d563b041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d563ad7670 0x55d563b03c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 1 --2- 172.21.15.18:0/1442770533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d563ad6d50 0x55d563b068d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 1 -- 172.21.15.18:0/1442770533 >> 172.21.15.18:0/1442770533 conn(0x55d563ad1480 msgr2=0x55d563afc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f860cc00640 0 client.31292 ms_handle_reset on v2:172.21.15.18:6838/294785837 2024-04-16T17:27:07.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 1 -- 172.21.15.18:0/1442770533 shutdown_connections 2024-04-16T17:27:07.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 1 -- 172.21.15.18:0/1442770533 wait complete. 2024-04-16T17:27:07.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.582+0000 7f862cb90740 20 client.31292 trim_cache size 0 max 16384 2024-04-16T17:27:07.585 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:27:07.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.584+0000 7f8629e00640 1 -- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f86080a3f20 msgr2=0x7f86080a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.584+0000 7f8629e00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f86080a3f20 0x7f86080a63e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f861c00a510 tx=0x7f861c00d040 comp rx=0 tx=0).stop 2024-04-16T17:27:07.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.584+0000 7f8629e00640 1 -- 172.21.15.18:0/2081081721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8624102230 msgr2=0x7f862410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.584+0000 7f8629e00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8624102230 0x7f862410e180 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f861400d950 tx=0x7f861400de10 comp rx=0 tx=0).stop 2024-04-16T17:27:07.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.586+0000 7f8629e00640 1 -- 172.21.15.18:0/2081081721 shutdown_connections 2024-04-16T17:27:07.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.586+0000 7f8629e00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f86080a3f20 0x7f86080a63e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.586+0000 7f8629e00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86241a3920 0x7f86241a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.586+0000 7f8629e00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8624107300 0x7f862410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.586+0000 7f8629e00640 1 --2- 172.21.15.18:0/2081081721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8624102230 0x7f862410e180 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.586+0000 7f8629e00640 1 -- 172.21.15.18:0/2081081721 >> 172.21.15.18:0/2081081721 conn(0x7f86240fdab0 msgr2=0x7f86240fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.586+0000 7f8629e00640 1 -- 172.21.15.18:0/2081081721 shutdown_connections 2024-04-16T17:27:07.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.586+0000 7f8629e00640 1 -- 172.21.15.18:0/2081081721 wait complete. 2024-04-16T17:27:07.612 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:27:07.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.931+0000 7fcb3d200640 1 -- 172.21.15.18:0/464126259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38076530 msgr2=0x7fcb38076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.931+0000 7fcb3d200640 1 --2- 172.21.15.18:0/464126259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38076530 0x7fcb38076910 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009930 tx=0x7fcb2802bc20 comp rx=0 tx=0).stop 2024-04-16T17:27:07.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.931+0000 7fcb3d200640 1 -- 172.21.15.18:0/464126259 shutdown_connections 2024-04-16T17:27:07.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.931+0000 7fcb3d200640 1 --2- 172.21.15.18:0/464126259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb38112e80 0x7fcb38115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.931+0000 7fcb3d200640 1 --2- 172.21.15.18:0/464126259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb38076e50 0x7fcb38112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.931+0000 7fcb3d200640 1 --2- 172.21.15.18:0/464126259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38076530 0x7fcb38076910 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.931+0000 7fcb3d200640 1 -- 172.21.15.18:0/464126259 >> 172.21.15.18:0/464126259 conn(0x7fcb380ffc90 msgr2=0x7fcb381020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.932+0000 7fcb3d200640 1 -- 172.21.15.18:0/464126259 shutdown_connections 2024-04-16T17:27:07.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.932+0000 7fcb3d200640 1 -- 172.21.15.18:0/464126259 wait complete. 2024-04-16T17:27:07.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.933+0000 7fcb3d200640 1 Processor -- start 2024-04-16T17:27:07.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.933+0000 7fcb3d200640 1 -- start start 2024-04-16T17:27:07.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb38076530 0x7fcb3819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb38076e50 0x7fcb3819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38112e80 0x7fcb381a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb3d200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381189e0 con 0x7fcb38076530 2024-04-16T17:27:07.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb3d200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38118860 con 0x7fcb38112e80 2024-04-16T17:27:07.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb3d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38118b60 con 0x7fcb38076e50 2024-04-16T17:27:07.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb36800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38112e80 0x7fcb381a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb36800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38112e80 0x7fcb381a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38210/0 (socket says 172.21.15.18:38210) 2024-04-16T17:27:07.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb36800640 1 -- 172.21.15.18:0/2012704369 learned_addr learned my addr 172.21.15.18:0/2012704369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.934+0000 7fcb35400640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb38076e50 0x7fcb3819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.935+0000 7fcb36800640 1 -- 172.21.15.18:0/2012704369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb38076e50 msgr2=0x7fcb3819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.935+0000 7fcb36800640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb38076e50 0x7fcb3819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.935+0000 7fcb36800640 1 -- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb38076530 msgr2=0x7fcb3819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.935+0000 7fcb36800640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb38076530 0x7fcb3819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.935+0000 7fcb36800640 1 -- 172.21.15.18:0/2012704369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb381a3f30 con 0x7fcb38112e80 2024-04-16T17:27:07.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.935+0000 7fcb36800640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38112e80 0x7fcb381a3800 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000eb40 tx=0x7fcb3000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.936+0000 7fcb2ea00640 1 -- 172.21.15.18:0/2012704369 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30004b20 con 0x7fcb38112e80 2024-04-16T17:27:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.936+0000 7fcb3d200640 1 -- 172.21.15.18:0/2012704369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb381a4220 con 0x7fcb38112e80 2024-04-16T17:27:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.936+0000 7fcb2ea00640 1 -- 172.21.15.18:0/2012704369 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb30004cc0 con 0x7fcb38112e80 2024-04-16T17:27:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.936+0000 7fcb2ea00640 1 -- 172.21.15.18:0/2012704369 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30010850 con 0x7fcb38112e80 2024-04-16T17:27:07.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.936+0000 7fcb3d200640 1 -- 172.21.15.18:0/2012704369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb381abab0 con 0x7fcb38112e80 2024-04-16T17:27:07.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb3ff6d740 1 Processor -- start 2024-04-16T17:27:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb2ea00640 1 -- 172.21.15.18:0/2012704369 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcb30005020 con 0x7fcb38112e80 2024-04-16T17:27:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb3ff6d740 1 -- start start 2024-04-16T17:27:07.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824276d50 0x562824277130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824277670 0x56282427fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824282640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5628241dc020 con 0x562824276d50 2024-04-16T17:27:07.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562824182460 con 0x562824280250 2024-04-16T17:27:07.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.941+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5628241d7fe0 con 0x562824277670 2024-04-16T17:27:07.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.942+0000 7fcb35400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824282640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.942+0000 7fcb35400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824282640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38212/0 (socket says 172.21.15.18:38212) 2024-04-16T17:27:07.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.942+0000 7fcb35400640 1 -- 172.21.15.18:0/4160929423 learned_addr learned my addr 172.21.15.18:0/4160929423 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.942+0000 7fcb35400640 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824276d50 0x562824277130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.943+0000 7fcb35e00640 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824277670 0x56282427fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.943+0000 7fcb35400640 1 -- 172.21.15.18:0/4160929423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824277670 msgr2=0x56282427fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.943+0000 7fcb35400640 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824277670 0x56282427fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.943+0000 7fcb35400640 1 -- 172.21.15.18:0/4160929423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824276d50 msgr2=0x562824277130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.943+0000 7fcb35400640 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824276d50 0x562824277130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.943+0000 7fcb35400640 1 -- 172.21.15.18:0/4160929423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282418a040 con 0x562824280250 2024-04-16T17:27:07.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.944+0000 7fcb35400640 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824282640 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcb240114c0 tx=0x7fcb24009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea0b6b637f4a5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.944+0000 7fcb2ea00640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb1c0a3e20 0x7fcb1c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.944+0000 7fcb23400640 1 -- 172.21.15.18:0/4160929423 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24004030 con 0x562824280250 2024-04-16T17:27:07.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.945+0000 7fcb35e00640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb1c0a3e20 0x7fcb1c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.946+0000 7fcb23400640 1 -- 172.21.15.18:0/4160929423 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb240041d0 con 0x562824280250 2024-04-16T17:27:07.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.946+0000 7fcb23400640 1 -- 172.21.15.18:0/4160929423 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2401e7c0 con 0x562824280250 2024-04-16T17:27:07.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.947+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/4160929423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 msgr2=0x562824282640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.947+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824282640 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcb240114c0 tx=0x7fcb24009570 comp rx=0 tx=0).stop 2024-04-16T17:27:07.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb2ea00640 1 -- 172.21.15.18:0/2012704369 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fcb300d1450 con 0x7fcb38112e80 2024-04-16T17:27:07.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb35e00640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb1c0a3e20 0x7fcb1c0a62e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009230 tx=0x7fcb28000bc0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/4160929423 shutdown_connections 2024-04-16T17:27:07.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824282640 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824277670 0x56282427fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/4160929423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824276d50 0x562824277130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/4160929423 >> 172.21.15.18:0/4160929423 conn(0x562824271480 msgr2=0x562824271860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/4160929423 shutdown_connections 2024-04-16T17:27:07.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.948+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/4160929423 wait complete. 2024-04-16T17:27:07.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.949+0000 7fcb3ff6d740 1 -- start start 2024-04-16T17:27:07.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.949+0000 7fcb3ff6d740 1 Processor -- start 2024-04-16T17:27:07.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282427d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x56282427dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb35400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282427d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb36800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x56282427dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb36800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x56282427dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38214/0 (socket says 172.21.15.18:38214) 2024-04-16T17:27:07.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb36800640 1 -- 172.21.15.18:0/444579686 learned_addr learned my addr 172.21.15.18:0/444579686 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:07.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb35400640 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282427d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59078/0 (socket says 172.21.15.18:59078) 2024-04-16T17:27:07.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb35e00640 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562824294050 con 0x562824277670 2024-04-16T17:27:07.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562824195830 con 0x562824280250 2024-04-16T17:27:07.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5628242941d0 con 0x562824276d50 2024-04-16T17:27:07.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.952+0000 7fcb3ff6d740 10 client.0 _ll_register_callbacks cb 0x56282419d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:27:07.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.953+0000 7fcb36800640 1 -- 172.21.15.18:0/444579686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 msgr2=0x56282427d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.953+0000 7fcb36800640 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.953+0000 7fcb36800640 1 -- 172.21.15.18:0/444579686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 msgr2=0x56282427d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:07.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.953+0000 7fcb36800640 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282427d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.953+0000 7fcb36800640 1 -- 172.21.15.18:0/444579686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562824199920 con 0x562824280250 2024-04-16T17:27:07.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.953+0000 7fcb35e00640 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:27:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.956+0000 7fcb36800640 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x56282427dd60 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcb30099390 tx=0x7fcb3009c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac81d1da4bebf527 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.957+0000 7fcb17e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:27:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.958+0000 7fcb17e00640 1 -- 172.21.15.18:0/444579686 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb300a12e0 con 0x562824280250 2024-04-16T17:27:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.958+0000 7fcb17e00640 1 -- 172.21.15.18:0/444579686 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fcb300ce960 con 0x562824280250 2024-04-16T17:27:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.958+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb240041d0 con 0x562824280250 2024-04-16T17:27:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.958+0000 7fcb17e00640 1 -- 172.21.15.18:0/444579686 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcb300ceb60 con 0x562824280250 2024-04-16T17:27:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.958+0000 7fcb3ff6d740 10 client.31304 fetch_fsmap learned FSMap version 37 2024-04-16T17:27:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.958+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5628241677f0 con 0x562824280250 2024-04-16T17:27:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.958+0000 7fcb17e00640 1 -- 172.21.15.18:0/444579686 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb240041d0 con 0x562824280250 2024-04-16T17:27:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.959+0000 7fcb17e00640 1 -- 172.21.15.18:0/444579686 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fcb300a5710 con 0x562824280250 2024-04-16T17:27:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.961+0000 7fcb3ff6d740 10 client.31304 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:27:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.961+0000 7fcb3ff6d740 10 client.31304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:27:07.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.962+0000 7fcb3ff6d740 20 client.31304 populate_metadata read hostname 'smithi018' 2024-04-16T17:27:07.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.962+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628242a16d0 0x5628242a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:07.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.962+0000 7fcb3ff6d740 4 client.31304 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:27:07.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.962+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628241e9a40 con 0x5628242a16d0 2024-04-16T17:27:07.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.962+0000 7fcb35e00640 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628242a16d0 0x5628242a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:07.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.963+0000 7fcb35e00640 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628242a16d0 0x5628242a3b00 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7750ee3c674f959f server_cookie=1bc8e9c3d7b33ccf in_seq=0 out_seq=0 2024-04-16T17:27:07.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.963+0000 7fcb17e00640 10 client.31304 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:27:07.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.964+0000 7fcb17e00640 1 -- 172.21.15.18:0/444579686 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628241e9a40 con 0x5628242a16d0 2024-04-16T17:27:07.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.964+0000 7fcb17e00640 10 client.31304 handle_command_reply: tid=0 2024-04-16T17:27:07.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.964+0000 7fcb3ff6d740 1 client.31304 shutdown 2024-04-16T17:27:07.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.965+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 msgr2=0x56282427dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:07.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.965+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x56282427dd60 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcb30099390 tx=0x7fcb3009c950 comp rx=0 tx=0).stop 2024-04-16T17:27:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.965+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 shutdown_connections 2024-04-16T17:27:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.965+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x5628242a16d0 0x5628242a3b00 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.965+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x56282427dd60 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.965+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282427d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.965+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/444579686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.965+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 >> 172.21.15.18:0/444579686 conn(0x562824271480 msgr2=0x5628242731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.966+0000 7fcb17e00640 0 client.31304 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:27:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.966+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 shutdown_connections 2024-04-16T17:27:07.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.966+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/444579686 wait complete. 2024-04-16T17:27:07.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:07.966+0000 7fcb3ff6d740 20 client.31304 trim_cache size 0 max 16384 2024-04-16T17:27:08.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.005+0000 7fcb3ff6d740 1 Processor -- start 2024-04-16T17:27:08.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.005+0000 7fcb3ff6d740 1 -- start start 2024-04-16T17:27:08.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.005+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:08.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.005+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282429cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:08.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.005+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824272fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:08.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.005+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381aff20 con 0x562824277670 2024-04-16T17:27:08.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.005+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381ad7f0 con 0x562824280250 2024-04-16T17:27:08.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.005+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3810dbb0 con 0x562824276d50 2024-04-16T17:27:08.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.006+0000 7fcb35e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:08.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.006+0000 7fcb35e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49036/0 (socket says 172.21.15.18:49036) 2024-04-16T17:27:08.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.006+0000 7fcb35e00640 1 -- 172.21.15.18:0/2846153459 learned_addr learned my addr 172.21.15.18:0/2846153459 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.006+0000 7fcb35400640 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282429cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.007+0000 7fcb36800640 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824272fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.007+0000 7fcb35e00640 1 -- 172.21.15.18:0/2846153459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 msgr2=0x562824272fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.007+0000 7fcb35e00640 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824272fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.007+0000 7fcb35e00640 1 -- 172.21.15.18:0/2846153459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 msgr2=0x56282429cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.007+0000 7fcb35e00640 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282429cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.007+0000 7fcb35e00640 1 -- 172.21.15.18:0/2846153459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562823eee3f0 con 0x562824276d50 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.007+0000 7fcb35400640 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282429cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:27:08.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.007+0000 7fcb36800640 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824272fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:27:08.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.008+0000 7fcb35e00640 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427fa10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802c160 tx=0x7fcb2803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b5df1017b86504 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:08.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.008+0000 7fcb22a00640 1 -- 172.21.15.18:0/2846153459 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28056070 con 0x562824276d50 2024-04-16T17:27:08.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.008+0000 7fcb22a00640 1 -- 172.21.15.18:0/2846153459 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb28042040 con 0x562824276d50 2024-04-16T17:27:08.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.008+0000 7fcb22a00640 1 -- 172.21.15.18:0/2846153459 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28004dc0 con 0x562824276d50 2024-04-16T17:27:08.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.009+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/2846153459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 msgr2=0x56282427fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:08.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.009+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427fa10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802c160 tx=0x7fcb2803d000 comp rx=0 tx=0).stop 2024-04-16T17:27:08.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.010+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/2846153459 shutdown_connections 2024-04-16T17:27:08.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.010+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824280250 0x562824272fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.010+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824277670 0x56282429cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.010+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/2846153459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x56282427fa10 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.010+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/2846153459 >> 172.21.15.18:0/2846153459 conn(0x562824271480 msgr2=0x5628242728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:08.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.010+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/2846153459 shutdown_connections 2024-04-16T17:27:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.011+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/2846153459 wait complete. 2024-04-16T17:27:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.012+0000 7fcb3ff6d740 1 -- start start 2024-04-16T17:27:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.012+0000 7fcb3ff6d740 1 Processor -- start 2024-04-16T17:27:08.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x5628242996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:08.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824277670 0x562824294a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:08.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb3ff6d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824280250 0x562824294f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:08.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381abce0 con 0x562824280250 2024-04-16T17:27:08.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381ad120 con 0x562824277670 2024-04-16T17:27:08.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb3ff6d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381ac380 con 0x562824276d50 2024-04-16T17:27:08.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb3ff6d740 10 client.0 _ll_register_callbacks cb 0x7fcb30005920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:27:08.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb35e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x5628242996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:08.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb35400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824277670 0x562824294a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:08.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.013+0000 7fcb35400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824277670 0x562824294a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38226/0 (socket says 172.21.15.18:38226) 2024-04-16T17:27:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.014+0000 7fcb35400640 1 -- 172.21.15.18:0/246887690 learned_addr learned my addr 172.21.15.18:0/246887690 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.014+0000 7fcb36800640 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824280250 0x562824294f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.014+0000 7fcb35400640 1 -- 172.21.15.18:0/246887690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 msgr2=0x5628242996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.014+0000 7fcb35400640 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x5628242996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.014+0000 7fcb35400640 1 -- 172.21.15.18:0/246887690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824280250 msgr2=0x562824294f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:27:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.015+0000 7fcb35400640 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824280250 0x562824294f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.015+0000 7fcb35400640 1 -- 172.21.15.18:0/246887690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562823fb87d0 con 0x562824277670 2024-04-16T17:27:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.015+0000 7fcb35e00640 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x5628242996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:27:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.015+0000 7fcb36800640 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824280250 0x562824294f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:27:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.016+0000 7fcb35400640 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824277670 0x562824294a30 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x562824272610 tx=0x7fcb24005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53c227aa9096e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.016+0000 7fcb17e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-16T17:27:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.017+0000 7fcb17e00640 1 -- 172.21.15.18:0/246887690 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24003a00 con 0x562824277670 2024-04-16T17:27:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.017+0000 7fcb17e00640 1 -- 172.21.15.18:0/246887690 <== mon.2 v2:172.21.15.22:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fcb24003ba0 con 0x562824277670 2024-04-16T17:27:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.017+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/246887690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb28042040 con 0x562824277670 2024-04-16T17:27:08.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.018+0000 7fcb17e00640 1 -- 172.21.15.18:0/246887690 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcb24032050 con 0x562824277670 2024-04-16T17:27:08.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.018+0000 7fcb3ff6d740 10 client.31307 fetch_fsmap learned FSMap version 37 2024-04-16T17:27:08.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.018+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/246887690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562824307890 con 0x562824277670 2024-04-16T17:27:08.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.018+0000 7fcb17e00640 1 -- 172.21.15.18:0/246887690 <== mon.2 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb28042040 con 0x562824277670 2024-04-16T17:27:08.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.019+0000 7fcb17e00640 1 -- 172.21.15.18:0/246887690 <== mon.2 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7fcb2401e460 con 0x562824277670 2024-04-16T17:27:08.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.019+0000 7fcb3ff6d740 10 client.31307 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:27:08.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.019+0000 7fcb3ff6d740 10 client.31307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:27:08.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.019+0000 7fcb3ff6d740 20 client.31307 populate_metadata read hostname 'smithi018' 2024-04-16T17:27:08.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.019+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562824307c70 0x56282430a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:08.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.019+0000 7fcb3ff6d740 4 client.31307 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:27:08.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.019+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/246887690 --> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628241e38b0 con 0x562824307c70 2024-04-16T17:27:08.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.019+0000 7fcb35e00640 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562824307c70 0x56282430a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:08.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.020+0000 7fcb35e00640 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562824307c70 0x56282430a060 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83368628d495e1b server_cookie=aad43ba1fbdfdf3f in_seq=0 out_seq=0 2024-04-16T17:27:08.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.020+0000 7fcb17e00640 10 client.31307 ms_handle_connect on v2:172.21.15.22:6838/836242731 2024-04-16T17:27:08.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.020+0000 7fcb17e00640 1 -- 172.21.15.18:0/246887690 <== mds.0 v2:172.21.15.22:6838/836242731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628241e38b0 con 0x562824307c70 2024-04-16T17:27:08.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.020+0000 7fcb17e00640 10 client.31307 handle_command_reply: tid=0 2024-04-16T17:27:08.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.020+0000 7fcb3ff6d740 1 client.31307 shutdown 2024-04-16T17:27:08.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/246887690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824277670 msgr2=0x562824294a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:08.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824277670 0x562824294a30 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x562824272610 tx=0x7fcb24005c50 comp rx=0 tx=0).stop 2024-04-16T17:27:08.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/246887690 shutdown_connections 2024-04-16T17:27:08.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.22:6838/836242731,v1:172.21.15.22:6839/836242731] conn(0x562824307c70 0x56282430a060 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562824280250 0x562824294f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562824277670 0x562824294a30 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 --2- 172.21.15.18:0/246887690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562824276d50 0x5628242996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb17e00640 0 client.31307 ms_handle_reset on v2:172.21.15.22:6838/836242731 2024-04-16T17:27:08.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/246887690 >> 172.21.15.18:0/246887690 conn(0x562824271480 msgr2=0x562824273140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:08.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/246887690 shutdown_connections 2024-04-16T17:27:08.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.021+0000 7fcb3ff6d740 1 -- 172.21.15.18:0/246887690 wait complete. 2024-04-16T17:27:08.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.022+0000 7fcb3ff6d740 20 client.31307 trim_cache size 0 max 16384 2024-04-16T17:27:08.025 INFO:teuthology.orchestra.run.smithi018.stdout:[] 2024-04-16T17:27:08.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.023+0000 7fcb3d200640 1 -- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb1c0a3e20 msgr2=0x7fcb1c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:08.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.023+0000 7fcb3d200640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb1c0a3e20 0x7fcb1c0a62e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009230 tx=0x7fcb28000bc0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.023+0000 7fcb3d200640 1 -- 172.21.15.18:0/2012704369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38112e80 msgr2=0x7fcb381a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:08.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.023+0000 7fcb3d200640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38112e80 0x7fcb381a3800 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000eb40 tx=0x7fcb3000c2d0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.024+0000 7fcb3d200640 1 -- 172.21.15.18:0/2012704369 shutdown_connections 2024-04-16T17:27:08.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.024+0000 7fcb3d200640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fcb1c0a3e20 0x7fcb1c0a62e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.024+0000 7fcb3d200640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb38112e80 0x7fcb381a3800 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.024+0000 7fcb3d200640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb38076e50 0x7fcb3819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.024+0000 7fcb3d200640 1 --2- 172.21.15.18:0/2012704369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb38076530 0x7fcb3819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:08.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.024+0000 7fcb3d200640 1 -- 172.21.15.18:0/2012704369 >> 172.21.15.18:0/2012704369 conn(0x7fcb380ffc90 msgr2=0x7fcb381145b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:08.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.024+0000 7fcb3d200640 1 -- 172.21.15.18:0/2012704369 shutdown_connections 2024-04-16T17:27:08.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:08.024+0000 7fcb3d200640 1 -- 172.21.15.18:0/2012704369 wait complete. 2024-04-16T17:27:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:08 smithi022 ceph-mon[36023]: pgmap v717: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 462 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-04-16T17:27:08.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:08 smithi018 ceph-mon[31776]: pgmap v717: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 462 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-04-16T17:27:08.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:08 smithi043 ceph-mon[36901]: pgmap v717: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 462 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-04-16T17:27:09.055 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-16T17:27:09.055 INFO:tasks.fwd_scrub:done joining 2024-04-16T17:27:09.055 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-16T17:27:09.066 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:27:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.363+0000 7f00ec000640 1 -- 172.21.15.18:0/719955514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00e4074d50 msgr2=0x7f00e410e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:09.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.363+0000 7f00ec000640 1 --2- 172.21.15.18:0/719955514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00e4074d50 0x7f00e410e2d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009860 tx=0x7f00d002bc10 comp rx=0 tx=0).stop 2024-04-16T17:27:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.365+0000 7f00ec000640 1 -- 172.21.15.18:0/719955514 shutdown_connections 2024-04-16T17:27:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.365+0000 7f00ec000640 1 --2- 172.21.15.18:0/719955514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e410e810 0x7f00e4110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.365+0000 7f00ec000640 1 --2- 172.21.15.18:0/719955514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00e4074d50 0x7f00e410e2d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.365+0000 7f00ec000640 1 --2- 172.21.15.18:0/719955514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e4074430 0x7f00e4074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.365+0000 7f00ec000640 1 -- 172.21.15.18:0/719955514 >> 172.21.15.18:0/719955514 conn(0x7f00e40fdae0 msgr2=0x7f00e40fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.365+0000 7f00ec000640 1 -- 172.21.15.18:0/719955514 shutdown_connections 2024-04-16T17:27:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.365+0000 7f00ec000640 1 -- 172.21.15.18:0/719955514 wait complete. 2024-04-16T17:27:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.366+0000 7f00ec000640 1 Processor -- start 2024-04-16T17:27:09.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.367+0000 7f00ec000640 1 -- start start 2024-04-16T17:27:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.367+0000 7f00ec000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e4074430 0x7f00e41094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.367+0000 7f00ec000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00e4074d50 0x7f00e41099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.367+0000 7f00ec000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e410e810 0x7f00e41045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.367+0000 7f00ec000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4103180 con 0x7f00e4074430 2024-04-16T17:27:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.367+0000 7f00ec000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4103000 con 0x7f00e410e810 2024-04-16T17:27:09.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.367+0000 7f00ec000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4103300 con 0x7f00e4074d50 2024-04-16T17:27:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00ea000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e410e810 0x7f00e41045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00ea000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e410e810 0x7f00e41045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38246/0 (socket says 172.21.15.18:38246) 2024-04-16T17:27:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00ea000640 1 -- 172.21.15.18:0/690658358 learned_addr learned my addr 172.21.15.18:0/690658358 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00e8c00640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00e4074d50 0x7f00e41099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00ea000640 1 -- 172.21.15.18:0/690658358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00e4074d50 msgr2=0x7f00e41099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00ea000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00e4074d50 0x7f00e41099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00ea000640 1 -- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e4074430 msgr2=0x7f00e41094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00ea000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e4074430 0x7f00e41094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.368+0000 7f00ea000640 1 -- 172.21.15.18:0/690658358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e4104e70 con 0x7f00e410e810 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.369+0000 7f00ea000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e410e810 0x7f00e41045b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00eda0 tx=0x7f00dc00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.369+0000 7f00e2000640 1 -- 172.21.15.18:0/690658358 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc003b70 con 0x7f00e410e810 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.369+0000 7f00e2000640 1 -- 172.21.15.18:0/690658358 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00dc003d10 con 0x7f00e410e810 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.369+0000 7f00e2000640 1 -- 172.21.15.18:0/690658358 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc003b70 con 0x7f00e410e810 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.369+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e4105160 con 0x7f00e410e810 2024-04-16T17:27:09.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.369+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e410d1a0 con 0x7f00e410e810 2024-04-16T17:27:09.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.371+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e410e610 con 0x7f00e410e810 2024-04-16T17:27:09.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.374+0000 7f00e2000640 1 -- 172.21.15.18:0/690658358 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f00dc003fc0 con 0x7f00e410e810 2024-04-16T17:27:09.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.376+0000 7f00e2000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00cc0a4050 0x7f00cc0a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:09.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.377+0000 7f00e2000640 1 -- 172.21.15.18:0/690658358 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f00dc0d0a80 con 0x7f00e410e810 2024-04-16T17:27:09.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.378+0000 7f00e9600640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00cc0a4050 0x7f00cc0a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:09.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.379+0000 7f00e9600640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00cc0a4050 0x7f00cc0a6510 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f00d80058c0 tx=0x7f00d8005850 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:09.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.380+0000 7f00e2000640 1 -- 172.21.15.18:0/690658358 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00dc09cd70 con 0x7f00e410e810 2024-04-16T17:27:09.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.765+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00e4105b10 con 0x7f00e410e810 2024-04-16T17:27:09.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.768+0000 7f00e2000640 1 -- 172.21.15.18:0/690658358 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+25451 (secure 0 0 0) 0x7f00e4105b10 con 0x7f00e410e810 2024-04-16T17:27:09.772 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:27:09.772 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":137,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:27:04.050051+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":41,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":41,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3045212308":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/3152065578":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/1033276779":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6801/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/2149549301":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":28,"length":1},{"begin":41,"length":1}]},{"pool":4,"snaps":[{"begin":28,"length":1},{"begin":41,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":28,"length":1},{"begin":41,"length":1}]},{"pool":4,"snaps":[{"begin":28,"length":1},{"begin":41,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:27:09.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00cc0a4050 msgr2=0x7f00cc0a6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00cc0a4050 0x7f00cc0a6510 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f00d80058c0 tx=0x7f00d8005850 comp rx=0 tx=0).stop 2024-04-16T17:27:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e410e810 msgr2=0x7f00e41045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e410e810 0x7f00e41045b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00eda0 tx=0x7f00dc00c6e0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 shutdown_connections 2024-04-16T17:27:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f00cc0a4050 0x7f00cc0a6510 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e410e810 0x7f00e41045b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00e4074d50 0x7f00e41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 --2- 172.21.15.18:0/690658358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e4074430 0x7f00e41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 >> 172.21.15.18:0/690658358 conn(0x7f00e40fdae0 msgr2=0x7f00e40ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 shutdown_connections 2024-04-16T17:27:09.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:09.773+0000 7f00ec000640 1 -- 172.21.15.18:0/690658358 wait complete. 2024-04-16T17:27:09.799 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 137, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 137, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:27:09.800 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:27:10.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:10 smithi018 ceph-mon[31776]: pgmap v718: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 462 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-04-16T17:27:10.119 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:10 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/690658358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:27:10.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.129+0000 7fa0fbe00640 1 -- 172.21.15.18:0/341111641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc112c50 msgr2=0x7fa0fc115040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:10.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.129+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/341111641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc112c50 0x7fa0fc115040 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009ca0 tx=0x7fa0f402c190 comp rx=0 tx=0).stop 2024-04-16T17:27:10.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.130+0000 7fa0fbe00640 1 -- 172.21.15.18:0/341111641 shutdown_connections 2024-04-16T17:27:10.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.130+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/341111641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc112c50 0x7fa0fc115040 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.130+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/341111641 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0fc076e50 0x7fa0fc112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.130+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/341111641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc076530 0x7fa0fc076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.130+0000 7fa0fbe00640 1 -- 172.21.15.18:0/341111641 >> 172.21.15.18:0/341111641 conn(0x7fa0fc0ffae0 msgr2=0x7fa0fc101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:10.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.130+0000 7fa0fbe00640 1 -- 172.21.15.18:0/341111641 shutdown_connections 2024-04-16T17:27:10.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.130+0000 7fa0fbe00640 1 -- 172.21.15.18:0/341111641 wait complete. 2024-04-16T17:27:10.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.131+0000 7fa0fbe00640 1 Processor -- start 2024-04-16T17:27:10.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fbe00640 1 -- start start 2024-04-16T17:27:10.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0fc076530 0x7fa0fc1a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:10.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc076e50 0x7fa0fc1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:10.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc112c50 0x7fa0fc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:10.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fbe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1187d0 con 0x7fa0fc076530 2024-04-16T17:27:10.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc118650 con 0x7fa0fc112c50 2024-04-16T17:27:10.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc118950 con 0x7fa0fc076e50 2024-04-16T17:27:10.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc112c50 0x7fa0fc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:10.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc112c50 0x7fa0fc19ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38254/0 (socket says 172.21.15.18:38254) 2024-04-16T17:27:10.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.132+0000 7fa0fb400640 1 -- 172.21.15.18:0/3964745057 learned_addr learned my addr 172.21.15.18:0/3964745057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:10.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.133+0000 7fa0fa000640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc076e50 0x7fa0fc1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:10.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.133+0000 7fa0faa00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0fc076530 0x7fa0fc1a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:10.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.133+0000 7fa0fb400640 1 -- 172.21.15.18:0/3964745057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc076e50 msgr2=0x7fa0fc1a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.133+0000 7fa0fb400640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc076e50 0x7fa0fc1a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.133+0000 7fa0fb400640 1 -- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0fc076530 msgr2=0x7fa0fc1a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.133+0000 7fa0fb400640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0fc076530 0x7fa0fc1a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.133+0000 7fa0fb400640 1 -- 172.21.15.18:0/3964745057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc19f250 con 0x7fa0fc112c50 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.133+0000 7fa0faa00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0fc076530 0x7fa0fc1a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.134+0000 7fa0fb400640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc112c50 0x7fa0fc19ead0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4037040 tx=0x7fa0f4035e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.134+0000 7fa0f3400640 1 -- 172.21.15.18:0/3964745057 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4039040 con 0x7fa0fc112c50 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.134+0000 7fa0f3400640 1 -- 172.21.15.18:0/3964745057 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f4044070 con 0x7fa0fc112c50 2024-04-16T17:27:10.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.134+0000 7fa0f3400640 1 -- 172.21.15.18:0/3964745057 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4032940 con 0x7fa0fc112c50 2024-04-16T17:27:10.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.134+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc19f4e0 con 0x7fa0fc112c50 2024-04-16T17:27:10.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.135+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1ab7e0 con 0x7fa0fc112c50 2024-04-16T17:27:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.136+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc04fd90 con 0x7fa0fc112c50 2024-04-16T17:27:10.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.139+0000 7fa0f3400640 1 -- 172.21.15.18:0/3964745057 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0f4032ae0 con 0x7fa0fc112c50 2024-04-16T17:27:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.144+0000 7fa0f3400640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0e00a3d90 0x7fa0e00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.144+0000 7fa0f3400640 1 -- 172.21.15.18:0/3964745057 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fa0f40f3110 con 0x7fa0fc112c50 2024-04-16T17:27:10.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.146+0000 7fa0faa00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0e00a3d90 0x7fa0e00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:10.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.146+0000 7fa0f3400640 1 -- 172.21.15.18:0/3964745057 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0f404e020 con 0x7fa0fc112c50 2024-04-16T17:27:10.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.147+0000 7fa0faa00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0e00a3d90 0x7fa0e00a6250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec005d00 tx=0x7fa0ec009550 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:10 smithi022 ceph-mon[36023]: pgmap v718: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 462 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-04-16T17:27:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:10 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/690658358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:27:10.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:10 smithi043 ceph-mon[36901]: pgmap v718: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 462 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-04-16T17:27:10.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:10 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/690658358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:27:10.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.536+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0fc10dde0 con 0x7fa0fc112c50 2024-04-16T17:27:10.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.536+0000 7fa0f3400640 1 -- 172.21.15.18:0/3964745057 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7fa0f40bf460 con 0x7fa0fc112c50 2024-04-16T17:27:10.539 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0e00a3d90 msgr2=0x7fa0e00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0e00a3d90 0x7fa0e00a6250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec005d00 tx=0x7fa0ec009550 comp rx=0 tx=0).stop 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc112c50 msgr2=0x7fa0fc19ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc112c50 0x7fa0fc19ead0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4037040 tx=0x7fa0f4035e50 comp rx=0 tx=0).stop 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 shutdown_connections 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fa0e00a3d90 0x7fa0e00a6250 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0fc112c50 0x7fa0fc19ead0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc076e50 0x7fa0fc1a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 --2- 172.21.15.18:0/3964745057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0fc076530 0x7fa0fc1a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.543+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 >> 172.21.15.18:0/3964745057 conn(0x7fa0fc0ffae0 msgr2=0x7fa0fc114290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.544+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 shutdown_connections 2024-04-16T17:27:10.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.544+0000 7fa0fbe00640 1 -- 172.21.15.18:0/3964745057 wait complete. 2024-04-16T17:27:10.573 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:27:10.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.877+0000 7f70b8e00640 1 -- 172.21.15.18:0/3462504006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4102070 msgr2=0x7f70b4102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.877+0000 7f70b8e00640 1 --2- 172.21.15.18:0/3462504006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4102070 0x7f70b4102450 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009900 tx=0x7f70a002bc40 comp rx=0 tx=0).stop 2024-04-16T17:27:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.877+0000 7f70b8e00640 1 -- 172.21.15.18:0/3462504006 shutdown_connections 2024-04-16T17:27:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.877+0000 7f70b8e00640 1 --2- 172.21.15.18:0/3462504006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b4107140 0x7f70b410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.877+0000 7f70b8e00640 1 --2- 172.21.15.18:0/3462504006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4102a20 0x7f70b4106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.877+0000 7f70b8e00640 1 --2- 172.21.15.18:0/3462504006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4102070 0x7f70b4102450 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.877+0000 7f70b8e00640 1 -- 172.21.15.18:0/3462504006 >> 172.21.15.18:0/3462504006 conn(0x7f70b40fd950 msgr2=0x7f70b40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.877+0000 7f70b8e00640 1 -- 172.21.15.18:0/3462504006 shutdown_connections 2024-04-16T17:27:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.878+0000 7f70b8e00640 1 -- 172.21.15.18:0/3462504006 wait complete. 2024-04-16T17:27:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.879+0000 7f70b8e00640 1 Processor -- start 2024-04-16T17:27:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.879+0000 7f70b8e00640 1 -- start start 2024-04-16T17:27:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4102070 0x7f70b4116420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b8e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b4107140 0x7f70b4116960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4111530 0x7f70b41119e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:10.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b8e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4110030 con 0x7f70b4107140 2024-04-16T17:27:10.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b8e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70b410feb0 con 0x7f70b4102070 2024-04-16T17:27:10.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b8e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41101b0 con 0x7f70b4111530 2024-04-16T17:27:10.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b4107140 0x7f70b4116960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:10.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4102070 0x7f70b4116420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:10.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b4107140 0x7f70b4116960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59142/0 (socket says 172.21.15.18:59142) 2024-04-16T17:27:10.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.880+0000 7f70b2a00640 1 -- 172.21.15.18:0/1693091789 learned_addr learned my addr 172.21.15.18:0/1693091789 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.881+0000 7f70b3400640 1 -- 172.21.15.18:0/1693091789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4111530 msgr2=0x7f70b41119e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:27:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.881+0000 7f70b3400640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4111530 0x7f70b41119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.881+0000 7f70b3400640 1 -- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b4107140 msgr2=0x7f70b4116960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.881+0000 7f70b3400640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b4107140 0x7f70b4116960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.881+0000 7f70b3400640 1 -- 172.21.15.18:0/1693091789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b4111f20 con 0x7f70b4102070 2024-04-16T17:27:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.881+0000 7f70b3400640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4102070 0x7f70b4116420 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f70a002c180 tx=0x7f70a00045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:10.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.882+0000 7f70abe00640 1 -- 172.21.15.18:0/1693091789 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0041070 con 0x7f70b4102070 2024-04-16T17:27:10.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.883+0000 7f70abe00640 1 -- 172.21.15.18:0/1693091789 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a002cd20 con 0x7f70b4102070 2024-04-16T17:27:10.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.883+0000 7f70abe00640 1 -- 172.21.15.18:0/1693091789 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0034440 con 0x7f70b4102070 2024-04-16T17:27:10.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.883+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b41121b0 con 0x7f70b4102070 2024-04-16T17:27:10.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.884+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b41b4390 con 0x7f70b4102070 2024-04-16T17:27:10.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.887+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b4103540 con 0x7f70b4102070 2024-04-16T17:27:10.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.889+0000 7f70abe00640 1 -- 172.21.15.18:0/1693091789 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f70a0044070 con 0x7f70b4102070 2024-04-16T17:27:10.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.892+0000 7f70abe00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3ef0 0x7f70980a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:10.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.893+0000 7f70abe00640 1 -- 172.21.15.18:0/1693091789 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f70a00f2640 con 0x7f70b4102070 2024-04-16T17:27:10.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.894+0000 7f70b2a00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3ef0 0x7f70980a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:10.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.895+0000 7f70b2a00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3ef0 0x7f70980a63b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f70b4112cf0 tx=0x7f70a400a380 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:10.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:10.901+0000 7f70abe00640 1 -- 172.21.15.18:0/1693091789 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70a00be930 con 0x7f70b4102070 2024-04-16T17:27:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.309+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70b410a370 con 0x7f70b4102070 2024-04-16T17:27:11.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.310+0000 7f70abe00640 1 -- 172.21.15.18:0/1693091789 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) ==== 104+0+11 (secure 0 0 0) 0x7f70a00be230 con 0x7f70b4102070 2024-04-16T17:27:11.310 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:27:11.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3ef0 msgr2=0x7f70980a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:11.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3ef0 0x7f70980a63b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f70b4112cf0 tx=0x7f70a400a380 comp rx=0 tx=0).stop 2024-04-16T17:27:11.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4102070 msgr2=0x7f70b4116420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:11.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4102070 0x7f70b4116420 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f70a002c180 tx=0x7f70a00045e0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 shutdown_connections 2024-04-16T17:27:11.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f70980a3ef0 0x7f70980a63b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4111530 0x7f70b41119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70b4107140 0x7f70b4116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 --2- 172.21.15.18:0/1693091789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70b4102070 0x7f70b4116420 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.313+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 >> 172.21.15.18:0/1693091789 conn(0x7f70b40fd950 msgr2=0x7f70b40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:11.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.316+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 shutdown_connections 2024-04-16T17:27:11.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.316+0000 7f70b8e00640 1 -- 172.21.15.18:0/1693091789 wait complete. 2024-04-16T17:27:11.345 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:27:11.422 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:11 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3964745057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:27:11.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:11 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3964745057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:27:11.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:11 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3964745057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:27:11.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 -- 172.21.15.18:0/569903903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 msgr2=0x7fd41810c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:11.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 --2- 172.21.15.18:0/569903903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 0x7fd41810c070 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd400009900 tx=0x7fd4000300d0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 -- 172.21.15.18:0/569903903 shutdown_connections 2024-04-16T17:27:11.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 --2- 172.21.15.18:0/569903903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd41810c640 0x7fd41810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 --2- 172.21.15.18:0/569903903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 0x7fd41810c070 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 --2- 172.21.15.18:0/569903903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd418103270 0x7fd418103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 -- 172.21.15.18:0/569903903 >> 172.21.15.18:0/569903903 conn(0x7fd418077ff0 msgr2=0x7fd418078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:11.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 -- 172.21.15.18:0/569903903 shutdown_connections 2024-04-16T17:27:11.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.648+0000 7fd41f200640 1 -- 172.21.15.18:0/569903903 wait complete. 2024-04-16T17:27:11.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.650+0000 7fd41f200640 1 Processor -- start 2024-04-16T17:27:11.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd41f200640 1 -- start start 2024-04-16T17:27:11.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd41f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd418103270 0x7fd418196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:11.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd41f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 0x7fd418196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:11.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd41f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd41810c640 0x7fd41819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:11.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd41f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd418110230 con 0x7fd418103270 2024-04-16T17:27:11.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd41f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181100b0 con 0x7fd418103b90 2024-04-16T17:27:11.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd41f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181103b0 con 0x7fd41810c640 2024-04-16T17:27:11.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd40f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 0x7fd418196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:11.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd40f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 0x7fd418196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38290/0 (socket says 172.21.15.18:38290) 2024-04-16T17:27:11.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.651+0000 7fd40f400640 1 -- 172.21.15.18:0/1426037559 learned_addr learned my addr 172.21.15.18:0/1426037559 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:11.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.652+0000 7fd41d200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd41810c640 0x7fd41819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:11.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.652+0000 7fd417e00640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd418103270 0x7fd418196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:11.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.652+0000 7fd41d200640 1 -- 172.21.15.18:0/1426037559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 msgr2=0x7fd418196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:11.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.652+0000 7fd41d200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 0x7fd418196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.652+0000 7fd41d200640 1 -- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd418103270 msgr2=0x7fd418196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:11.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.652+0000 7fd41d200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd418103270 0x7fd418196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:11.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.652+0000 7fd41d200640 1 -- 172.21.15.18:0/1426037559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41819b640 con 0x7fd41810c640 2024-04-16T17:27:11.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.653+0000 7fd41d200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd41810c640 0x7fd41819af10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd41000ca80 tx=0x7fd41000cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:11.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.653+0000 7fd415600640 1 -- 172.21.15.18:0/1426037559 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41000ae20 con 0x7fd41810c640 2024-04-16T17:27:11.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.653+0000 7fd415600640 1 -- 172.21.15.18:0/1426037559 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41000f930 con 0x7fd41810c640 2024-04-16T17:27:11.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.653+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41819b930 con 0x7fd41810c640 2024-04-16T17:27:11.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.653+0000 7fd415600640 1 -- 172.21.15.18:0/1426037559 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410018b50 con 0x7fd41810c640 2024-04-16T17:27:11.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.653+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181a31c0 con 0x7fd41810c640 2024-04-16T17:27:11.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.655+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd418068620 con 0x7fd41810c640 2024-04-16T17:27:11.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.659+0000 7fd415600640 1 -- 172.21.15.18:0/1426037559 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd41000fae0 con 0x7fd41810c640 2024-04-16T17:27:11.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.661+0000 7fd415600640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd3fc0a3da0 0x7fd3fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:11.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.662+0000 7fd417e00640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd3fc0a3da0 0x7fd3fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:11.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.662+0000 7fd415600640 1 -- 172.21.15.18:0/1426037559 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fd4100d0e80 con 0x7fd41810c640 2024-04-16T17:27:11.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.665+0000 7fd415600640 1 -- 172.21.15.18:0/1426037559 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd410014030 con 0x7fd41810c640 2024-04-16T17:27:11.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:11.666+0000 7fd417e00640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd3fc0a3da0 0x7fd3fc0a6260 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd408005fd0 tx=0x7fd408009470 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:12.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.064+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd41802d620 con 0x7fd41810c640 2024-04-16T17:27:12.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.065+0000 7fd415600640 1 -- 172.21.15.18:0/1426037559 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) ==== 100+0+11 (secure 0 0 0) 0x7fd41009d0c0 con 0x7fd41810c640 2024-04-16T17:27:12.068 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:27:12.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd3fc0a3da0 msgr2=0x7fd3fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:12.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd3fc0a3da0 0x7fd3fc0a6260 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd408005fd0 tx=0x7fd408009470 comp rx=0 tx=0).stop 2024-04-16T17:27:12.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd41810c640 msgr2=0x7fd41819af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:12.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd41810c640 0x7fd41819af10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd41000ca80 tx=0x7fd41000cf40 comp rx=0 tx=0).stop 2024-04-16T17:27:12.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 shutdown_connections 2024-04-16T17:27:12.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fd3fc0a3da0 0x7fd3fc0a6260 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd41810c640 0x7fd41819af10 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd418103b90 0x7fd418196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 --2- 172.21.15.18:0/1426037559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd418103270 0x7fd418196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.070+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 >> 172.21.15.18:0/1426037559 conn(0x7fd418077ff0 msgr2=0x7fd4180fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:12.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.072+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 shutdown_connections 2024-04-16T17:27:12.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.072+0000 7fd41f200640 1 -- 172.21.15.18:0/1426037559 wait complete. 2024-04-16T17:27:12.098 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:27:12.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.391+0000 7efc36c00640 1 -- 172.21.15.18:0/1927516892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc301067b0 msgr2=0x7efc301111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:12.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.391+0000 7efc36c00640 1 --2- 172.21.15.18:0/1927516892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc301067b0 0x7efc301111b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efc28009cd0 tx=0x7efc2802c0e0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.395+0000 7efc36c00640 1 -- 172.21.15.18:0/1927516892 shutdown_connections 2024-04-16T17:27:12.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.395+0000 7efc36c00640 1 --2- 172.21.15.18:0/1927516892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc301067b0 0x7efc301111b0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.395+0000 7efc36c00640 1 --2- 172.21.15.18:0/1927516892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc30105e10 0x7efc30106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.395+0000 7efc36c00640 1 --2- 172.21.15.18:0/1927516892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc30104530 0x7efc30104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.395+0000 7efc36c00640 1 -- 172.21.15.18:0/1927516892 >> 172.21.15.18:0/1927516892 conn(0x7efc300ffc90 msgr2=0x7efc301020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:12.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.395+0000 7efc36c00640 1 -- 172.21.15.18:0/1927516892 shutdown_connections 2024-04-16T17:27:12.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.395+0000 7efc36c00640 1 -- 172.21.15.18:0/1927516892 wait complete. 2024-04-16T17:27:12.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.396+0000 7efc36c00640 1 Processor -- start 2024-04-16T17:27:12.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.397+0000 7efc36c00640 1 -- start start 2024-04-16T17:27:12.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.397+0000 7efc36c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc30104530 0x7efc3010f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:12.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.397+0000 7efc36c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30105e10 0x7efc30073a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:12.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.397+0000 7efc36c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc301067b0 0x7efc3010fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:12.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.397+0000 7efc36c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efc30077360 con 0x7efc301067b0 2024-04-16T17:27:12.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.397+0000 7efc36c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efc300771e0 con 0x7efc30104530 2024-04-16T17:27:12.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.397+0000 7efc36c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc300774e0 con 0x7efc30105e10 2024-04-16T17:27:12.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.397+0000 7efc2fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc30104530 0x7efc3010f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:12.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc2fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc30104530 0x7efc3010f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:38304/0 (socket says 172.21.15.18:38304) 2024-04-16T17:27:12.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc2fe00640 1 -- 172.21.15.18:0/900927627 learned_addr learned my addr 172.21.15.18:0/900927627 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:12.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc2f400640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30105e10 0x7efc30073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:12.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc34c00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc301067b0 0x7efc3010fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:12.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc2fe00640 1 -- 172.21.15.18:0/900927627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30105e10 msgr2=0x7efc30073a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:12.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc2fe00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30105e10 0x7efc30073a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc2fe00640 1 -- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc301067b0 msgr2=0x7efc3010fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:12.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc2fe00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc301067b0 0x7efc3010fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc2fe00640 1 -- 172.21.15.18:0/900927627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc301104d0 con 0x7efc30104530 2024-04-16T17:27:12.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.398+0000 7efc34c00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc301067b0 0x7efc3010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:27:12.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.399+0000 7efc2fe00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc30104530 0x7efc3010f760 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e9f0 tx=0x7efc1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:12.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.399+0000 7efc2cc00640 1 -- 172.21.15.18:0/900927627 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00cd80 con 0x7efc30104530 2024-04-16T17:27:12.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.399+0000 7efc2cc00640 1 -- 172.21.15.18:0/900927627 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c010910 con 0x7efc30104530 2024-04-16T17:27:12.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.399+0000 7efc2cc00640 1 -- 172.21.15.18:0/900927627 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c018990 con 0x7efc30104530 2024-04-16T17:27:12.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.399+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc301b3ca0 con 0x7efc30104530 2024-04-16T17:27:12.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.399+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301b41a0 con 0x7efc30104530 2024-04-16T17:27:12.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.401+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30069300 con 0x7efc30104530 2024-04-16T17:27:12.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.409+0000 7efc2cc00640 1 -- 172.21.15.18:0/900927627 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc1c010ac0 con 0x7efc30104530 2024-04-16T17:27:12.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.411+0000 7efc2cc00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efc140a3da0 0x7efc140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:12.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.411+0000 7efc2cc00640 1 -- 172.21.15.18:0/900927627 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7efc1c0d1140 con 0x7efc30104530 2024-04-16T17:27:12.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.412+0000 7efc2f400640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efc140a3da0 0x7efc140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:12.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.412+0000 7efc2cc00640 1 -- 172.21.15.18:0/900927627 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc1c0d1660 con 0x7efc30104530 2024-04-16T17:27:12.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.413+0000 7efc2f400640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efc140a3da0 0x7efc140a6260 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efc301111f0 tx=0x7efc2000b390 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:12 smithi022 ceph-mon[36023]: pgmap v719: 129 pgs: 17 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 61 active+clean; 457 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-04-16T17:27:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:12 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1693091789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:27:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:12 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1426037559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:27:12.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:12 smithi018 ceph-mon[31776]: pgmap v719: 129 pgs: 17 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 61 active+clean; 457 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-04-16T17:27:12.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:12 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1693091789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:27:12.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:12 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1426037559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:27:12.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:12 smithi043 ceph-mon[36901]: pgmap v719: 129 pgs: 17 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 61 active+clean; 457 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-04-16T17:27:12.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:12 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1693091789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:27:12.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:12 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1426037559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:27:12.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.795+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7efc3002d620 con 0x7efc30104530 2024-04-16T17:27:12.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.795+0000 7efc2cc00640 1 -- 172.21.15.18:0/900927627 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v137) ==== 103+0+11 (secure 0 0 0) 0x7efc1c09d380 con 0x7efc30104530 2024-04-16T17:27:12.797 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:27:12.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.798+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efc140a3da0 msgr2=0x7efc140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:12.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efc140a3da0 0x7efc140a6260 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efc301111f0 tx=0x7efc2000b390 comp rx=0 tx=0).stop 2024-04-16T17:27:12.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc30104530 msgr2=0x7efc3010f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:12.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc30104530 0x7efc3010f760 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e9f0 tx=0x7efc1c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 shutdown_connections 2024-04-16T17:27:12.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7efc140a3da0 0x7efc140a6260 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc301067b0 0x7efc3010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30105e10 0x7efc30073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 --2- 172.21.15.18:0/900927627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc30104530 0x7efc3010f760 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:12.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.799+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 >> 172.21.15.18:0/900927627 conn(0x7efc300ffc90 msgr2=0x7efc30101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:12.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.800+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 shutdown_connections 2024-04-16T17:27:12.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:12.800+0000 7efc36c00640 1 -- 172.21.15.18:0/900927627 wait complete. 2024-04-16T17:27:12.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-16T17:27:13.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.125+0000 7fea3f200640 1 -- 172.21.15.18:0/367258994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea38095f30 msgr2=0x7fea380963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:13.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.125+0000 7fea3f200640 1 --2- 172.21.15.18:0/367258994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea38095f30 0x7fea380963b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009930 tx=0x7fea2c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:27:13.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.126+0000 7fea3f200640 1 -- 172.21.15.18:0/367258994 shutdown_connections 2024-04-16T17:27:13.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.126+0000 7fea3f200640 1 --2- 172.21.15.18:0/367258994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea380968f0 0x7fea380a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:13.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.126+0000 7fea3f200640 1 --2- 172.21.15.18:0/367258994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea38095f30 0x7fea380963b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:13.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.126+0000 7fea3f200640 1 --2- 172.21.15.18:0/367258994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea3809acf0 0x7fea3809b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:13.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.126+0000 7fea3f200640 1 -- 172.21.15.18:0/367258994 >> 172.21.15.18:0/367258994 conn(0x7fea3800cb40 msgr2=0x7fea3800cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:13.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.126+0000 7fea3f200640 1 -- 172.21.15.18:0/367258994 shutdown_connections 2024-04-16T17:27:13.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.126+0000 7fea3f200640 1 -- 172.21.15.18:0/367258994 wait complete. 2024-04-16T17:27:13.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.128+0000 7fea3f200640 1 Processor -- start 2024-04-16T17:27:13.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.129+0000 7fea3f200640 1 -- start start 2024-04-16T17:27:13.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea38095f30 0x7fea38006e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:13.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea380968f0 0x7fea38007380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:13.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3809acf0 0x7fea380078c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:13.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fea380a60e0 con 0x7fea38095f30 2024-04-16T17:27:13.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fea380a5f60 con 0x7fea380968f0 2024-04-16T17:27:13.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea380a6260 con 0x7fea3809acf0 2024-04-16T17:27:13.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3809acf0 0x7fea380078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:13.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3809acf0 0x7fea380078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:49152/0 (socket says 172.21.15.18:49152) 2024-04-16T17:27:13.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3e800640 1 -- 172.21.15.18:0/2054630423 learned_addr learned my addr 172.21.15.18:0/2054630423 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:27:13.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3de00640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea38095f30 0x7fea38006e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:13.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.130+0000 7fea3d400640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea380968f0 0x7fea38007380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:13.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.131+0000 7fea3e800640 1 -- 172.21.15.18:0/2054630423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea380968f0 msgr2=0x7fea38007380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:13.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.131+0000 7fea3e800640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea380968f0 0x7fea38007380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:13.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.131+0000 7fea3e800640 1 -- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea38095f30 msgr2=0x7fea38006e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:13.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.131+0000 7fea3e800640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea38095f30 0x7fea38006e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:13.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.131+0000 7fea3e800640 1 -- 172.21.15.18:0/2054630423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3813aa80 con 0x7fea3809acf0 2024-04-16T17:27:13.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.131+0000 7fea3de00640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea38095f30 0x7fea38006e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:27:13.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.132+0000 7fea3e800640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3809acf0 0x7fea380078c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fea40067760 tx=0x7fea40077690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:13.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.133+0000 7fea36a00640 1 -- 172.21.15.18:0/2054630423 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40083620 con 0x7fea3809acf0 2024-04-16T17:27:13.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.133+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3813ad70 con 0x7fea3809acf0 2024-04-16T17:27:13.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.133+0000 7fea36a00640 1 -- 172.21.15.18:0/2054630423 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4007b900 con 0x7fea3809acf0 2024-04-16T17:27:13.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.133+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3813b1e0 con 0x7fea3809acf0 2024-04-16T17:27:13.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.133+0000 7fea36a00640 1 -- 172.21.15.18:0/2054630423 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40083620 con 0x7fea3809acf0 2024-04-16T17:27:13.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.136+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea38004830 con 0x7fea3809acf0 2024-04-16T17:27:13.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.143+0000 7fea36a00640 1 -- 172.21.15.18:0/2054630423 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea4008a060 con 0x7fea3809acf0 2024-04-16T17:27:13.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.145+0000 7fea36a00640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea240ac7b0 0x7fea240aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:27:13.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.146+0000 7fea36a00640 1 -- 172.21.15.18:0/2054630423 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7fea4013c9e0 con 0x7fea3809acf0 2024-04-16T17:27:13.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.149+0000 7fea3de00640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea240ac7b0 0x7fea240aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:27:13.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.150+0000 7fea3de00640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea240ac7b0 0x7fea240aec70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fea30009b50 tx=0x7fea30005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:27:13.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.151+0000 7fea36a00640 1 -- 172.21.15.18:0/2054630423 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea40108bc0 con 0x7fea3809acf0 2024-04-16T17:27:13.155 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:13 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/900927627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:27:13.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:13 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/900927627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:27:13.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:13 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/900927627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:27:13.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.639+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fea3812c520 con 0x7fea3809acf0 2024-04-16T17:27:13.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.657+0000 7fea36a00640 1 -- 172.21.15.18:0/2054630423 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746058 (secure 0 0 0) 0x7fea401084c0 con 0x7fea3809acf0 2024-04-16T17:27:13.661 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:27:13.661 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:27:13.662 INFO:teuthology.orchestra.run.smithi018.stdout: "epoch": 26, 2024-04-16T17:27:13.662 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.662 INFO:teuthology.orchestra.run.smithi018.stdout: "active_gid": 25417, 2024-04-16T17:27:13.662 INFO:teuthology.orchestra.run.smithi018.stdout: "active_name": "x", 2024-04-16T17:27:13.662 INFO:teuthology.orchestra.run.smithi018.stdout: "active_addrs": { 2024-04-16T17:27:13.662 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-04-16T17:27:13.662 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.662 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6800", 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3776183549 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6801", 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3776183549 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.663 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: "active_addr": "172.21.15.18:6801/3776183549", 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: "active_change": "2024-04-16T17:04:44.092266+0000", 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: "active_mgr_features": 4540701547738038271, 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: "available": true, 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: "standbys": [ 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: "gid": 25414, 2024-04-16T17:27:13.664 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "z", 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_features": 4540701547738038271, 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: "available_modules": [ 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "alerts", 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:13.665 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T17:27:13.666 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.667 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.668 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:13.669 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.670 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:13.671 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.672 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.673 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.674 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_destination": { 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_destination", 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.675 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email address to send alerts to", 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_from_name": { 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_from_name", 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.676 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "Ceph", 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email From: name", 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.677 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_host": { 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_host", 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.678 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP server", 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_password": { 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_password", 2024-04-16T17:27:13.679 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Password to authenticate with", 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.680 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_port": { 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_port", 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "465", 2024-04-16T17:27:13.681 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP port", 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_sender": { 2024-04-16T17:27:13.682 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_sender", 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP envelope sender", 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.683 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.684 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.684 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.684 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_ssl": { 2024-04-16T17:27:13.684 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_ssl", 2024-04-16T17:27:13.684 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.684 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.684 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.684 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_user": { 2024-04-16T17:27:13.685 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_user", 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "User to authenticate as", 2024-04-16T17:27:13.686 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "balancer", 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:13.687 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "active": { 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "active", 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.688 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_time": { 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_time", 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.689 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0000", 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.690 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.691 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_weekday": { 2024-04-16T17:27:13.691 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_weekday", 2024-04-16T17:27:13.691 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:13.691 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.691 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.691 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:13.691 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-04-16T17:27:13.691 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_max_iterations": { 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_max_iterations", 2024-04-16T17:27:13.692 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:13.693 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.693 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.693 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "25", 2024-04-16T17:27:13.693 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:13.693 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "250", 2024-04-16T17:27:13.693 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.693 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T17:27:13.693 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_metrics": { 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_metrics", 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.694 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.695 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_step": { 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_step", 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.5", 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0.001", 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "0.999", 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.696 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "aggressiveness of optimization", 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: "end_time": { 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_time", 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.697 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2359", 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "end_weekday": { 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_weekday", 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-04-16T17:27:13.699 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.700 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.701 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.702 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.702 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.702 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.702 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.702 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.702 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.702 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.702 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.703 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:13.703 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:13.703 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.703 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.703 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.703 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.703 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.703 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:13.704 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.705 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.707 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: "min_score": { 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_score", 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.708 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.709 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "upmap", 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.710 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-compat", 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: "none", 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: "read", 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap", 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap-read" 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Balancer mode", 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.711 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_ids": { 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_ids", 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.712 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:13.713 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.714 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.714 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.714 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:13.714 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.714 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.714 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.714 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T17:27:13.714 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_deviation": { 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_deviation", 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.715 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.716 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_optimizations": { 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_optimizations", 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T17:27:13.718 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.718 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.718 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.718 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.718 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.718 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.718 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.718 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm", 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_down_multiplier": { 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_down_multiplier", 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.719 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-04-16T17:27:13.720 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.720 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.720 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.720 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T17:27:13.720 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.720 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.720 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.720 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_refresh_rate": { 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_refresh_rate", 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "20", 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.721 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_starting_port": { 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_starting_port", 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.722 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4721", 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.724 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_ptrace": { 2024-04-16T17:27:13.724 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_ptrace", 2024-04-16T17:27:13.724 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.724 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.724 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.724 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.725 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.725 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.725 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.725 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T17:27:13.725 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T17:27:13.725 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.725 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.726 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_interval": { 2024-04-16T17:27:13.726 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_interval", 2024-04-16T17:27:13.726 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.726 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.726 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.726 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:13.726 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.726 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_memory_target_ratio": { 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T17:27:13.727 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.7", 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.728 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm_log_destination": { 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm_log_destination", 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.729 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.730 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.730 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.730 INFO:teuthology.orchestra.run.smithi018.stdout: "file", 2024-04-16T17:27:13.730 INFO:teuthology.orchestra.run.smithi018.stdout: "file,syslog", 2024-04-16T17:27:13.730 INFO:teuthology.orchestra.run.smithi018.stdout: "syslog" 2024-04-16T17:27:13.730 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.730 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T17:27:13.730 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: "cgroups_split": { 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cgroups_split", 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.731 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.732 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.732 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.732 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.732 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T17:27:13.732 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.732 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.732 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.732 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.733 INFO:teuthology.orchestra.run.smithi018.stdout: "config_checks_enabled": { 2024-04-16T17:27:13.733 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_checks_enabled", 2024-04-16T17:27:13.733 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.733 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.733 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.733 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.733 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.733 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.734 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.734 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T17:27:13.734 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.734 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.734 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.734 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.734 INFO:teuthology.orchestra.run.smithi018.stdout: "config_dashboard": { 2024-04-16T17:27:13.734 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_dashboard", 2024-04-16T17:27:13.735 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.735 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.735 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.735 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.735 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.735 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.735 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.735 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_alertmanager": { 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_alertmanager", 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.736 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.737 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_base": { 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_base", 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.738 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.739 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Container image name, without the tag", 2024-04-16T17:27:13.739 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.739 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.739 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.739 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.739 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_elasticsearch": { 2024-04-16T17:27:13.739 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_elasticsearch", 2024-04-16T17:27:13.739 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "elasticsearch container image", 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.740 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_grafana": { 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_grafana", 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T17:27:13.741 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_haproxy": { 2024-04-16T17:27:13.742 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_haproxy", 2024-04-16T17:27:13.743 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.743 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.743 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.743 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T17:27:13.743 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.743 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.743 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.743 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "HAproxy container image", 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_agent": { 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_agent", 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.744 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger agent container image", 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.745 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_collector": { 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_collector", 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.746 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger collector container image", 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_query": { 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_query", 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.747 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger query container image", 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.748 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_keepalived": { 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_keepalived", 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.749 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Keepalived container image", 2024-04-16T17:27:13.750 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.750 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.750 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.750 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_loki": { 2024-04-16T17:27:13.750 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_loki", 2024-04-16T17:27:13.750 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.750 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Loki container image", 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.751 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_node_exporter": { 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_node_exporter", 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.752 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_nvmeof": { 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_nvmeof", 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.753 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.754 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.754 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T17:27:13.754 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.754 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.754 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.754 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Nvme-of container image", 2024-04-16T17:27:13.754 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.754 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_prometheus": { 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_prometheus", 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T17:27:13.755 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_promtail": { 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_promtail", 2024-04-16T17:27:13.756 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Promtail container image", 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.757 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_samba": { 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_samba", 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.758 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Samba/SMB container image", 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_snmp_gateway": { 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_snmp_gateway", 2024-04-16T17:27:13.759 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.760 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.760 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.760 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T17:27:13.760 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.760 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.760 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.760 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SNMP Gateway container image", 2024-04-16T17:27:13.760 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.761 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.761 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.761 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.761 INFO:teuthology.orchestra.run.smithi018.stdout: "container_init": { 2024-04-16T17:27:13.761 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_init", 2024-04-16T17:27:13.761 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.761 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.761 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.762 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.762 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.762 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.762 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.762 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T17:27:13.762 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.762 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.762 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: "daemon_cache_timeout": { 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "daemon_cache_timeout", 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.763 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "default_cephadm_command_timeout": { 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.764 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "900", 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.765 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: "default_registry": { 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_registry", 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io", 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.766 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: "device_cache_timeout": { 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_cache_timeout", 2024-04-16T17:27:13.767 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1800", 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache device inventory", 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.768 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: "device_enhanced_scan": { 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_enhanced_scan", 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.769 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.770 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.770 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T17:27:13.770 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.770 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.770 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.770 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.770 INFO:teuthology.orchestra.run.smithi018.stdout: "facts_cache_timeout": { 2024-04-16T17:27:13.771 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "facts_cache_timeout", 2024-04-16T17:27:13.771 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.771 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.771 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.771 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:13.771 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.771 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.771 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache host facts data", 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "grafana_dashboards_path": { 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "grafana_dashboards_path", 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.772 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.773 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "host_check_interval": { 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "host_check_interval", 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.774 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to perform a host check", 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: "hw_monitoring": { 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hw_monitoring", 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.775 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "inventory_list_all": { 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "inventory_list_all", 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.777 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T17:27:13.778 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.778 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.778 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.778 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:13.778 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:13.778 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.778 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.779 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: "log_refresh_metadata": { 2024-04-16T17:27:13.780 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_refresh_metadata", 2024-04-16T17:27:13.781 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.781 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.781 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.781 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.781 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.781 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.781 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.781 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.782 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.783 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.784 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.785 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.785 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.785 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.785 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.785 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.785 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.785 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.785 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.786 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T17:27:13.787 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.788 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.789 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.789 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.789 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T17:27:13.789 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T17:27:13.789 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.789 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.789 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.789 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "*", 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.790 INFO:teuthology.orchestra.run.smithi018.stdout: "max_count_per_host": { 2024-04-16T17:27:13.791 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_count_per_host", 2024-04-16T17:27:13.791 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.791 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.791 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.791 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:13.791 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.791 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.791 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of daemons per service per host", 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd_draining_count": { 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_osd_draining_count", 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.792 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.793 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.793 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:13.793 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.793 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.793 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.793 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T17:27:13.793 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.793 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.794 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.794 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.794 INFO:teuthology.orchestra.run.smithi018.stdout: "migration_current": { 2024-04-16T17:27:13.794 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "migration_current", 2024-04-16T17:27:13.794 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.794 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.794 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.794 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "internal - do not modify", 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.795 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "root", 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.796 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm-package", 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: "root" 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: "oob_default_addr": { 2024-04-16T17:27:13.797 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "oob_default_addr", 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "169.254.1.1", 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T17:27:13.798 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.799 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.799 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.799 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.799 INFO:teuthology.orchestra.run.smithi018.stdout: "prometheus_alerts_path": { 2024-04-16T17:27:13.799 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus_alerts_path", 2024-04-16T17:27:13.799 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.799 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.799 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.800 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.801 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_insecure": { 2024-04-16T17:27:13.801 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_insecure", 2024-04-16T17:27:13.801 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.801 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.801 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.801 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.801 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.801 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.802 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.802 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T17:27:13.802 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.802 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.802 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.802 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.802 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_password": { 2024-04-16T17:27:13.802 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_password", 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T17:27:13.803 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_url": { 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_url", 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.804 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.805 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.805 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.805 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.805 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T17:27:13.805 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.805 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.805 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.805 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.806 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_username": { 2024-04-16T17:27:13.806 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_username", 2024-04-16T17:27:13.806 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.806 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.806 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.806 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.806 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.806 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: "secure_monitoring_stack": { 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "secure_monitoring_stack", 2024-04-16T17:27:13.807 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.808 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.808 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.808 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.808 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.808 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.808 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.808 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T17:27:13.808 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.809 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.809 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.809 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.809 INFO:teuthology.orchestra.run.smithi018.stdout: "service_discovery_port": { 2024-04-16T17:27:13.809 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "service_discovery_port", 2024-04-16T17:27:13.809 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.809 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.809 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8765", 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "cephadm service discovery port", 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.810 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "ssh_config_file": { 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssh_config_file", 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.811 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.812 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T17:27:13.812 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.812 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.812 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.812 INFO:teuthology.orchestra.run.smithi018.stdout: "use_agent": { 2024-04-16T17:27:13.816 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_agent", 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T17:27:13.817 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.818 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.818 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.818 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.818 INFO:teuthology.orchestra.run.smithi018.stdout: "use_repo_digest": { 2024-04-16T17:27:13.818 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_repo_digest", 2024-04-16T17:27:13.818 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.818 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.818 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.819 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_failed_host_check": { 2024-04-16T17:27:13.820 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_failed_host_check", 2024-04-16T17:27:13.820 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.820 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.820 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.820 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.820 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.820 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.820 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_daemons": { 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_daemons", 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.821 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.822 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_hosts": { 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_hosts", 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.823 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.824 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crash", 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.825 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.826 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:13.827 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.828 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.829 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.830 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.831 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.832 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "retain_interval": { 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retain_interval", 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "31536000", 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.833 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.834 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T17:27:13.834 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.834 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.834 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.834 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_recent_interval": { 2024-04-16T17:27:13.834 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_recent_interval", 2024-04-16T17:27:13.834 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.835 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.835 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.835 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1209600", 2024-04-16T17:27:13.835 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.835 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.835 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.835 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T17:27:13.835 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dashboard", 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:13.836 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.838 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.838 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.838 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.838 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.838 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.838 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T17:27:13.838 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.840 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.841 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.841 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.841 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.841 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.841 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.841 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.841 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.842 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.842 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_ENABLED": { 2024-04-16T17:27:13.842 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T17:27:13.842 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.842 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.842 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.842 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.842 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T17:27:13.843 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.844 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.845 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.846 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.847 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.848 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.849 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.850 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T17:27:13.850 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T17:27:13.850 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.850 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.850 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.850 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.850 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.850 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T17:27:13.851 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.852 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.853 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.854 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.854 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.854 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.854 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.854 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.854 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.854 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T17:27:13.854 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T17:27:13.855 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.855 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.855 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.855 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.855 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.855 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.855 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.856 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.856 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.856 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.856 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.856 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.856 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T17:27:13.856 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T17:27:13.856 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.857 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.858 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.858 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.858 INFO:teuthology.orchestra.run.smithi018.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T17:27:13.858 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T17:27:13.858 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.858 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.858 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.859 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.859 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.859 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.859 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.859 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.859 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.859 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.859 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.860 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.860 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T17:27:13.860 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T17:27:13.860 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.860 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.860 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.860 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:13.860 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.861 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.861 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.861 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.861 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.861 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.861 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.861 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.861 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T17:27:13.862 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T17:27:13.862 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.862 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.862 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.862 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.862 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.862 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_URL": { 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_URL", 2024-04-16T17:27:13.863 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.864 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.864 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.864 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.864 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.864 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.864 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.864 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.864 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.865 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.865 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.865 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.865 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T17:27:13.865 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T17:27:13.865 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.865 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.865 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.866 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:13.866 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.866 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.866 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.866 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.866 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.866 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.866 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.867 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.867 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T17:27:13.867 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T17:27:13.867 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.867 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.867 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.867 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.868 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T17:27:13.869 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T17:27:13.869 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.869 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.869 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.869 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.869 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.869 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.869 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.870 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.870 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.870 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.870 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.870 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.870 INFO:teuthology.orchestra.run.smithi018.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T17:27:13.870 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T17:27:13.871 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.871 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.871 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.871 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.871 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.871 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.871 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.871 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.872 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.872 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.872 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.872 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.872 INFO:teuthology.orchestra.run.smithi018.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T17:27:13.872 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T17:27:13.872 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.872 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.873 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.874 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.874 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.874 INFO:teuthology.orchestra.run.smithi018.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T17:27:13.874 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T17:27:13.874 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.874 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.874 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.874 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "[]", 2024-04-16T17:27:13.875 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.875 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.875 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.875 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.875 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.875 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.875 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.875 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.876 INFO:teuthology.orchestra.run.smithi018.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T17:27:13.876 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T17:27:13.876 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.876 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.876 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.876 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "{}", 2024-04-16T17:27:13.876 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.877 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.877 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.877 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.877 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.877 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.877 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.877 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.878 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T17:27:13.878 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T17:27:13.878 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.878 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.878 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.878 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.878 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.879 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.879 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.879 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.879 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.879 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.879 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.879 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.879 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.880 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.881 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.881 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.881 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.881 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.881 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T17:27:13.881 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T17:27:13.881 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.881 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.882 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.882 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.882 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.882 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.882 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.882 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.882 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.882 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.883 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.884 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.884 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.884 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.884 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.884 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.884 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.884 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.884 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T17:27:13.885 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T17:27:13.885 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.885 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.885 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.885 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.885 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.885 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.885 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.886 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.886 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.886 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.886 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.886 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.886 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T17:27:13.886 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T17:27:13.886 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.887 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.887 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.887 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.887 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.887 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.887 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.887 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.887 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.888 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.888 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.888 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T17:27:13.888 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T17:27:13.888 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.888 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.888 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.889 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.890 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T17:27:13.890 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T17:27:13.890 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.890 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.890 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.890 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.890 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.890 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T17:27:13.891 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.892 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.893 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.893 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.893 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T17:27:13.893 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T17:27:13.893 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.893 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.893 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.893 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.894 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.894 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.894 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.894 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.894 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.894 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.894 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.894 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.895 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.896 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.896 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.896 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.896 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.896 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.896 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T17:27:13.896 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T17:27:13.896 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.897 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.897 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.897 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:13.897 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.897 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.897 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.897 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.897 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.898 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.898 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.898 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.898 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T17:27:13.898 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T17:27:13.898 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.898 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.898 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.899 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8", 2024-04-16T17:27:13.899 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.899 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.899 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.899 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.899 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.899 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.899 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "45", 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.900 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.901 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.901 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.901 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.901 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.901 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.901 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T17:27:13.902 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T17:27:13.902 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.902 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.902 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.902 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.902 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.902 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.902 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.903 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.903 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.903 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.903 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.903 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.903 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T17:27:13.903 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T17:27:13.903 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.904 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.904 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.904 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:13.904 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.904 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.904 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.904 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.904 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.905 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.905 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.905 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.905 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T17:27:13.905 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T17:27:13.905 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.905 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.906 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.906 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.906 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.906 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.906 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.906 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.906 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.906 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.907 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.907 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.907 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T17:27:13.907 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T17:27:13.907 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.907 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.907 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.908 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.908 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.908 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.908 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.908 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.908 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.908 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.908 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.909 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.909 INFO:teuthology.orchestra.run.smithi018.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T17:27:13.909 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T17:27:13.909 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.909 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.909 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.909 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.909 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.910 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.910 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.910 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.910 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.910 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.910 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.910 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.911 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T17:27:13.911 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T17:27:13.911 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.911 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.911 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.911 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:13.911 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.912 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.912 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.912 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.912 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.912 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.912 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.912 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.913 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T17:27:13.913 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T17:27:13.913 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.913 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.913 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.913 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:13.913 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.913 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.914 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.914 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.914 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.914 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.914 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.914 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.914 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T17:27:13.914 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T17:27:13.915 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.915 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.915 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.915 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:13.915 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.915 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.915 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.915 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.916 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.916 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.916 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.916 INFO:teuthology.orchestra.run.smithi018.stdout: "cross_origin_url": { 2024-04-16T17:27:13.916 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cross_origin_url", 2024-04-16T17:27:13.916 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.916 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.917 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.918 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.918 INFO:teuthology.orchestra.run.smithi018.stdout: "crt_file": { 2024-04-16T17:27:13.918 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crt_file", 2024-04-16T17:27:13.918 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.918 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.918 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.918 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.918 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.919 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.919 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.919 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.919 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.919 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.919 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.919 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.920 INFO:teuthology.orchestra.run.smithi018.stdout: "debug": { 2024-04-16T17:27:13.920 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "debug", 2024-04-16T17:27:13.920 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.920 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.920 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.920 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.920 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.920 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.921 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.921 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable/disable debug options", 2024-04-16T17:27:13.921 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.921 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.921 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.921 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.921 INFO:teuthology.orchestra.run.smithi018.stdout: "jwt_token_ttl": { 2024-04-16T17:27:13.921 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "jwt_token_ttl", 2024-04-16T17:27:13.922 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.922 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.922 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.922 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "28800", 2024-04-16T17:27:13.922 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.922 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.922 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.922 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.923 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.923 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.923 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.923 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.923 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-04-16T17:27:13.923 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-04-16T17:27:13.923 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.923 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.924 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.924 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.924 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.924 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.924 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.924 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.924 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.925 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.925 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.925 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.925 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:13.925 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:13.925 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.925 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.926 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.926 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.926 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.926 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.926 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.926 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.926 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.926 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.927 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.928 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:13.928 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:13.928 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.928 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.928 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.928 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.928 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.928 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.929 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.929 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.929 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.929 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.929 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.929 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:13.929 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:13.930 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.930 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.930 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.930 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:13.930 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.930 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.930 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.931 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.931 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.931 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.931 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.931 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.931 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.931 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.931 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.933 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: "motd": { 2024-04-16T17:27:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "motd", 2024-04-16T17:27:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "The message of the day", 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect_resolve_ip_addr": { 2024-04-16T17:27:13.935 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T17:27:13.936 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.936 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.936 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.936 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.936 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.936 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.936 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.937 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.938 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8080", 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.939 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.940 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.940 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.940 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.940 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.940 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.940 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-04-16T17:27:13.940 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-04-16T17:27:13.941 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.941 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.941 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.941 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.941 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.941 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.941 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.941 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.942 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.942 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.942 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.942 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.942 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl_server_port": { 2024-04-16T17:27:13.942 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl_server_port", 2024-04-16T17:27:13.942 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.943 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.943 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.943 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8443", 2024-04-16T17:27:13.943 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.943 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.943 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.943 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.943 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.944 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "redirect", 2024-04-16T17:27:13.945 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.945 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.945 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.945 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.945 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect" 2024-04-16T17:27:13.945 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.945 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.945 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.946 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.946 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.946 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-04-16T17:27:13.946 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-04-16T17:27:13.946 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.946 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.947 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "url_prefix": { 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url_prefix", 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.948 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.949 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "devicehealth", 2024-04-16T17:27:13.950 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:13.950 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:13.950 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:13.950 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_monitoring": { 2024-04-16T17:27:13.950 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_monitoring", 2024-04-16T17:27:13.950 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.950 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.950 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.951 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.951 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.951 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.951 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.951 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "monitor device health metrics", 2024-04-16T17:27:13.951 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.951 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.952 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.953 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.953 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.953 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.953 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.953 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.953 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.953 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.953 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:13.954 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.955 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.956 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.957 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.958 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.959 INFO:teuthology.orchestra.run.smithi018.stdout: "mark_out_threshold": { 2024-04-16T17:27:13.960 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mark_out_threshold", 2024-04-16T17:27:13.960 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.960 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.960 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.960 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2419200", 2024-04-16T17:27:13.960 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.960 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.960 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": { 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_name", 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.961 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "device_health_metrics", 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.962 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.963 INFO:teuthology.orchestra.run.smithi018.stdout: "retention_period": { 2024-04-16T17:27:13.963 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retention_period", 2024-04-16T17:27:13.963 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.963 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.963 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.963 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15552000", 2024-04-16T17:27:13.963 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain device health metrics", 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_frequency": { 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_frequency", 2024-04-16T17:27:13.964 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.965 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.966 INFO:teuthology.orchestra.run.smithi018.stdout: "self_heal": { 2024-04-16T17:27:13.966 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "self_heal", 2024-04-16T17:27:13.966 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.966 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.966 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.966 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:13.966 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:13.967 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.968 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_threshold": { 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_threshold", 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7257600", 2024-04-16T17:27:13.969 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.970 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.970 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.970 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T17:27:13.970 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.970 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.970 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.970 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.970 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.971 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.971 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.971 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "diskprediction_local", 2024-04-16T17:27:13.971 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:13.971 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:13.971 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:13.971 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:13.971 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:13.972 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.972 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.972 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.972 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.972 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.972 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.972 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.973 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.974 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.974 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:13.974 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:13.974 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.974 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.974 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.974 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.975 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.976 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.977 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.977 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.977 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.977 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.977 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.977 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.977 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.977 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.978 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.978 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.978 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.978 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:13.978 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:13.978 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.978 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.978 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.979 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.979 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.979 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.979 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.979 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.980 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.980 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.980 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.980 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.980 INFO:teuthology.orchestra.run.smithi018.stdout: "predict_interval": { 2024-04-16T17:27:13.980 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predict_interval", 2024-04-16T17:27:13.980 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.980 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.981 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: "predictor_model": { 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predictor_model", 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "prophetstor", 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.982 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:13.983 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.984 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "influx", 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": false, 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "influxdb python module not found", 2024-04-16T17:27:13.985 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:13.986 INFO:teuthology.orchestra.run.smithi018.stdout: "batch_size": { 2024-04-16T17:27:13.986 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "batch_size", 2024-04-16T17:27:13.986 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:13.986 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.986 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.986 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5000", 2024-04-16T17:27:13.986 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: "database": { 2024-04-16T17:27:13.987 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "database", 2024-04-16T17:27:13.988 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.988 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.988 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.988 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "ceph", 2024-04-16T17:27:13.988 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.988 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.988 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.988 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": { 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hostname", 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.989 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server hostname", 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.990 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "30", 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "5", 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.991 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.992 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T17:27:13.992 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.992 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.992 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.992 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:13.992 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:13.992 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.993 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.994 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.994 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.994 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.994 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.994 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.994 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.994 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.994 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.995 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:13.996 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:13.997 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:13.998 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:13.999 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: "password": { 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "password", 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.000 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "password of InfluxDB server user", 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.001 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.002 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.002 INFO:teuthology.orchestra.run.smithi018.stdout: "port": { 2024-04-16T17:27:14.002 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "port", 2024-04-16T17:27:14.002 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.002 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.002 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.002 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8086", 2024-04-16T17:27:14.002 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server port", 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-04-16T17:27:14.003 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-04-16T17:27:14.005 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.006 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.006 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.006 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "false", 2024-04-16T17:27:14.006 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.006 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.006 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.006 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:27:14.006 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.007 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.007 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.007 INFO:teuthology.orchestra.run.smithi018.stdout: "threads": { 2024-04-16T17:27:14.007 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threads", 2024-04-16T17:27:14.007 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.007 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.007 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "32", 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.009 INFO:teuthology.orchestra.run.smithi018.stdout: "username": { 2024-04-16T17:27:14.009 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "username", 2024-04-16T17:27:14.009 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.009 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.009 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.009 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.009 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.009 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.010 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.010 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "username of InfluxDB server user", 2024-04-16T17:27:14.010 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.010 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.010 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.010 INFO:teuthology.orchestra.run.smithi018.stdout: "verify_ssl": { 2024-04-16T17:27:14.011 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "verify_ssl", 2024-04-16T17:27:14.011 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.011 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.011 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.011 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "true", 2024-04-16T17:27:14.011 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.011 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.011 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.012 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:27:14.012 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.012 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.012 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.012 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.012 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.012 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "insights", 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.013 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.014 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.015 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.016 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.016 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.016 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.016 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.016 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.016 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.016 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.016 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.017 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.018 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.019 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.019 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.019 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.019 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.019 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.019 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.019 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.020 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.021 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.021 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.021 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.021 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.021 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.021 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.021 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "iostat", 2024-04-16T17:27:14.022 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.022 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.022 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.022 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.022 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.022 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.022 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.022 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.024 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.025 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.026 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.027 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.028 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.029 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "k8sevents", 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.030 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_event_retention_days": { 2024-04-16T17:27:14.031 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ceph_event_retention_days", 2024-04-16T17:27:14.031 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.031 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.031 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.031 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7", 2024-04-16T17:27:14.031 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.031 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.031 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "config_check_secs": { 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_check_secs", 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.032 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "10", 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.034 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.035 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.036 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.037 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.038 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.038 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.038 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.038 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.038 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.038 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.038 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.039 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.040 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.040 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.040 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.040 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.040 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.040 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.040 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.041 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.041 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.041 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.041 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.041 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.041 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.041 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.041 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "localpool", 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.042 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "failure_domain": { 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "failure_domain", 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "host", 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.043 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.044 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "failure domain for any created local pool", 2024-04-16T17:27:14.044 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T17:27:14.044 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.044 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.044 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.044 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.044 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.044 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.045 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.046 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.047 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.047 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.047 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.047 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.047 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.047 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.047 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.048 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.048 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.048 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.048 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.048 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.048 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.049 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.049 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.049 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.049 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.049 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.049 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.049 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.050 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.050 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.050 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.050 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.050 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.050 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.050 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.050 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.051 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.052 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": { 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_size", 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.053 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default min_size for any created local pool", 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: "num_rep": { 2024-04-16T17:27:14.054 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "num_rep", 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3", 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default replica count for any created local pool", 2024-04-16T17:27:14.055 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": { 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_num", 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "128", 2024-04-16T17:27:14.056 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "prefix": { 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prefix", 2024-04-16T17:27:14.057 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.058 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.058 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.058 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.058 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.058 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.058 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.058 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name prefix for any created local pool", 2024-04-16T17:27:14.058 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: "subtree": { 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "subtree", 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.059 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "rack", 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.060 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.061 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.061 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.061 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mds_autoscaler", 2024-04-16T17:27:14.061 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.061 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.061 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.061 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.061 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.062 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.063 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.064 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.065 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.065 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.065 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.065 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.065 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.065 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.065 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.065 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.066 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.066 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.066 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.066 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.066 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.066 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.066 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.066 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.067 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.068 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.069 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.070 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirroring", 2024-04-16T17:27:14.070 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.070 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.070 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.070 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.070 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.070 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.070 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.071 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.072 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.072 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.072 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.072 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.072 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.072 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.072 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.073 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.074 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.074 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.074 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.074 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.074 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.074 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.074 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.074 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.075 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.076 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.077 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.078 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.078 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.078 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.078 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.078 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.078 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "nfs", 2024-04-16T17:27:14.078 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.078 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.079 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.079 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.079 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.079 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.079 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.079 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.079 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.079 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.080 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.080 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.080 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.080 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.080 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.080 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.080 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.081 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.081 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.081 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.081 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.081 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.081 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.081 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.081 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.082 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.082 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.082 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.082 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.082 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.082 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.082 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.082 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.083 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.084 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.085 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.086 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.086 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.086 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.086 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.086 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.086 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.086 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-04-16T17:27:14.087 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "fail_fs": { 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "fail_fs", 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.088 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.089 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.090 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.091 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.091 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.091 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.091 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.091 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.091 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.091 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.091 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.092 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.093 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.094 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.095 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.096 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator": { 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.097 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm", 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: "rook", 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: "test_orchestrator" 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Orchestrator backend", 2024-04-16T17:27:14.098 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_perf_query", 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.099 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.100 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.100 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.100 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.100 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.100 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.100 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.100 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.100 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.101 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.101 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.101 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.101 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.101 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.101 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.101 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.101 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.102 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.103 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.103 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.103 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.103 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.103 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.103 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.103 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.103 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.104 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.105 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.106 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.106 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.106 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.106 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.106 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.106 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.107 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.107 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.107 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.107 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.107 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.107 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.107 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.107 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.108 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_support", 2024-04-16T17:27:14.109 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.109 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.109 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.109 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.109 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.110 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.111 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.112 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.113 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.113 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.113 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.113 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.113 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.113 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.113 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.113 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.114 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.114 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.114 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.114 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.114 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.114 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.114 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.114 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.115 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.115 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.115 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.115 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.115 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.115 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.115 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.116 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.117 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_autoscaler", 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.118 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.119 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.120 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.121 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.122 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.122 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.122 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.122 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.122 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.122 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.122 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.123 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.123 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.123 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.123 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.123 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.123 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.123 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.123 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.124 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.124 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.124 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.124 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.124 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.124 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.124 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.124 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.125 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.126 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.127 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "threshold": { 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threshold", 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1.0", 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.128 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "scaling threshold", 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "progress", 2024-04-16T17:27:14.129 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.130 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.130 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.130 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_pg_recovery_event": { 2024-04-16T17:27:14.130 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_pg_recovery_event", 2024-04-16T17:27:14.130 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.130 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.130 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.130 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.131 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.132 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.133 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.135 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.136 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.137 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.138 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.139 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "max_completed_events": { 2024-04-16T17:27:14.140 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_completed_events", 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "50", 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "number of past completed events to remember", 2024-04-16T17:27:14.141 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.142 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.142 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.142 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.142 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:14.142 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:14.142 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.142 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.142 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long the module is going to sleep", 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.143 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus", 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: "cache": { 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cache", 2024-04-16T17:27:14.144 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.145 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.146 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.146 INFO:teuthology.orchestra.run.smithi018.stdout: "exclude_perf_counters": { 2024-04-16T17:27:14.146 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "exclude_perf_counters", 2024-04-16T17:27:14.146 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.146 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.146 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.146 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.146 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.147 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.148 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.149 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.149 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.149 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.149 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.149 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.149 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.149 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.149 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.150 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.150 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.150 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.150 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.150 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.150 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.150 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.150 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.151 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.151 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.151 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.151 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.151 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.151 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.151 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.151 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.152 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.153 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.153 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.153 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.153 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.153 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.153 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.153 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.153 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.154 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools": { 2024-04-16T17:27:14.155 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools", 2024-04-16T17:27:14.156 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.156 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.156 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.156 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.156 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.156 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.156 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.156 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.157 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300", 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_interval": { 2024-04-16T17:27:14.158 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_interval", 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15.0", 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.159 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.160 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.161 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:14.162 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:14.162 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.162 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.162 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.162 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "9283", 2024-04-16T17:27:14.163 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.163 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.163 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.163 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T17:27:14.163 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.163 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.163 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "stale_cache_strategy": { 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stale_cache_strategy", 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "log", 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.164 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.165 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.166 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "default", 2024-04-16T17:27:14.166 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.166 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.166 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.166 INFO:teuthology.orchestra.run.smithi018.stdout: "default", 2024-04-16T17:27:14.166 INFO:teuthology.orchestra.run.smithi018.stdout: "error" 2024-04-16T17:27:14.166 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.166 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.167 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.168 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_support", 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.169 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.170 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.170 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.170 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.170 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.170 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.170 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.170 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.170 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.171 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.171 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.171 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.171 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.171 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.171 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.171 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.172 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.172 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.172 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.172 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.172 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.172 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.172 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.172 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.173 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.173 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.173 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.173 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.173 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.173 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.173 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.173 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.174 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.175 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.176 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_snap_create": { 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_snap_create", 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.177 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.178 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "mirror_snapshot_schedule": { 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.179 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.180 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.180 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.180 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.180 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.180 INFO:teuthology.orchestra.run.smithi018.stdout: "trash_purge_schedule": { 2024-04-16T17:27:14.180 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "trash_purge_schedule", 2024-04-16T17:27:14.180 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.181 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "restful", 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.182 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_auth": { 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_auth", 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.185 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.185 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.185 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.185 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.185 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.185 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.186 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.187 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.188 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.189 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.190 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.191 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.192 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.193 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.194 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.195 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.196 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rgw", 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.197 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.198 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.199 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.200 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.201 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.202 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.203 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.204 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.205 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.205 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.205 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.205 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.205 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rook", 2024-04-16T17:27:14.205 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.205 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.205 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.206 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.206 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.206 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.206 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.206 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.206 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.206 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.206 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.207 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.208 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.209 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.209 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.209 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.209 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.209 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.209 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.209 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.209 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.210 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.210 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.210 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.210 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.210 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.210 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.210 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.210 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.211 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.212 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "storage_class": { 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "storage_class", 2024-04-16T17:27:14.213 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.214 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.214 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.214 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "local", 2024-04-16T17:27:14.214 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.214 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.214 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.214 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T17:27:14.214 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.215 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.215 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.215 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.215 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.215 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.215 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.215 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "selftest", 2024-04-16T17:27:14.215 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.216 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.217 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.218 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.220 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.220 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.220 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.220 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.220 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.220 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.220 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.220 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.221 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.222 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.223 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.223 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.223 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.223 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.223 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.223 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.223 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.224 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.224 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.224 INFO:teuthology.orchestra.run.smithi018.stdout: "roption1": { 2024-04-16T17:27:14.224 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption1", 2024-04-16T17:27:14.224 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.224 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.224 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.224 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "roption2": { 2024-04-16T17:27:14.225 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption2", 2024-04-16T17:27:14.226 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.226 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.226 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.226 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "xyz", 2024-04-16T17:27:14.226 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.226 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.226 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.226 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption1": { 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption1", 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.227 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.228 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption2": { 2024-04-16T17:27:14.229 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption2", 2024-04-16T17:27:14.229 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.229 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.229 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.229 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.229 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.229 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.229 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.230 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.230 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.230 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.230 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.230 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.230 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption3": { 2024-04-16T17:27:14.230 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption3", 2024-04-16T17:27:14.230 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.231 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption4": { 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption4", 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.232 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption5": { 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption5", 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.233 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.234 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption6": { 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption6", 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.235 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.236 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.236 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.236 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.236 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.236 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.236 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.236 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption7": { 2024-04-16T17:27:14.236 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption7", 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "42", 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.237 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: "testkey": { 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testkey", 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.238 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "testlkey": { 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testlkey", 2024-04-16T17:27:14.239 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.240 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: "testnewline": { 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testnewline", 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.241 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.242 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_m_granularity": { 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_m_granularity", 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.243 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.244 INFO:teuthology.orchestra.run.smithi018.stdout: "dump_on_update": { 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dump_on_update", 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.245 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "dump database to debug log on update", 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.246 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.247 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.248 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.248 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.248 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.248 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.248 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.248 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.248 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.249 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.250 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.251 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.251 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.251 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.251 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.251 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.251 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.251 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.252 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.253 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stats", 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.254 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.255 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.256 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.257 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.258 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.259 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.260 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.261 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.262 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.263 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "status", 2024-04-16T17:27:14.263 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.263 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.263 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.263 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.263 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.263 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.264 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.264 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.264 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.264 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.265 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.266 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.266 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.266 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.267 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.268 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.269 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.270 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.270 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.270 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.270 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.270 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.270 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.270 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.270 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.271 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telegraf", 2024-04-16T17:27:14.272 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "address": { 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "address", 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.273 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15", 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.276 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.277 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.278 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.278 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.278 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.278 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.278 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.278 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.278 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.279 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.280 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.280 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.280 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.280 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.280 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.280 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.280 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.281 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.281 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.281 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.281 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.281 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.281 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.281 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.281 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.282 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.282 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.282 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.282 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.282 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.282 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.282 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.283 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.283 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.283 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.283 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.283 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.283 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.283 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.283 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telemetry", 2024-04-16T17:27:14.284 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_basic": { 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_basic", 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.285 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_crash": { 2024-04-16T17:27:14.286 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_crash", 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.287 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_device": { 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_device", 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.288 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.289 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.289 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.289 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T17:27:14.289 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.289 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.289 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.289 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_ident": { 2024-04-16T17:27:14.290 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_ident", 2024-04-16T17:27:14.290 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.290 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.290 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.290 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.290 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.290 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.290 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_perf": { 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_perf", 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.291 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.292 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.293 INFO:teuthology.orchestra.run.smithi018.stdout: "contact": { 2024-04-16T17:27:14.293 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "contact", 2024-04-16T17:27:14.293 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.293 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.293 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.294 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.294 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.294 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.294 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.295 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.295 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.295 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.295 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.295 INFO:teuthology.orchestra.run.smithi018.stdout: "description": { 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "description", 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.296 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "device_url": { 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_url", 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.297 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.298 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "24", 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "8", 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.301 INFO:teuthology.orchestra.run.smithi018.stdout: "last_opt_revision": { 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "last_opt_revision", 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1", 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.302 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard": { 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard", 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.303 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard_description": { 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard_description", 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.305 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.306 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.307 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.308 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.309 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.310 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.310 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.310 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.310 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.310 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.310 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.310 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.310 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.311 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.312 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.313 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: "organization": { 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "organization", 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.314 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: "proxy": { 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "proxy", 2024-04-16T17:27:14.315 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.316 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "url": { 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url", 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.317 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.318 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "test_orchestrator", 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.319 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.320 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.320 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.320 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.320 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.320 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.320 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.320 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.320 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.321 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.322 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.323 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.324 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.325 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.326 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.326 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.326 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.326 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.326 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.326 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.326 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.327 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.327 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.327 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.327 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.327 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.327 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "volumes", 2024-04-16T17:27:14.327 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.328 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.329 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.330 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.330 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.330 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.330 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.330 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.330 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.330 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.330 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.331 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.331 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.331 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.331 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.331 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.331 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.331 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.331 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.332 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.333 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.334 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.335 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_clones": { 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_clones", 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4", 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.336 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: "periodic_async_work": { 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "periodic_async_work", 2024-04-16T17:27:14.337 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Periodically check for async work", 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.338 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_delay": { 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_delay", 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:14.339 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_no_wait": { 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T17:27:14.340 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.341 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.342 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.342 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.342 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.342 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.342 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix", 2024-04-16T17:27:14.342 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.342 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "discovery_interval": { 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "discovery_interval", 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "100", 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.343 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "identifier": { 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "identifier", 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.344 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.345 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.346 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:14.346 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:14.346 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.346 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.346 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.346 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:14.346 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.349 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.350 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.350 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.350 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.350 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.350 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.350 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.350 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.351 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.352 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.353 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.354 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.355 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.355 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.355 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.355 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.355 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.355 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_host": { 2024-04-16T17:27:14.355 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_host", 2024-04-16T17:27:14.356 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.356 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.356 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.356 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.356 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.356 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.356 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.356 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.357 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.357 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.357 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.357 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_port": { 2024-04-16T17:27:14.357 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_port", 2024-04-16T17:27:14.357 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.357 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10051", 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.358 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_sender": { 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_sender", 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.359 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.360 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: "gid": 25427, 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "y", 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_features": 4540701547738038271, 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: "available_modules": [ 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "alerts", 2024-04-16T17:27:14.361 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:14.362 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.363 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.364 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.366 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.367 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.368 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.369 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.370 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_destination": { 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_destination", 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.371 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email address to send alerts to", 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.372 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_from_name": { 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_from_name", 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "Ceph", 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.373 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email From: name", 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_host": { 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_host", 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.374 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.375 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.375 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.375 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.375 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.376 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP server", 2024-04-16T17:27:14.376 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.376 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.376 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.376 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.376 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_password": { 2024-04-16T17:27:14.376 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_password", 2024-04-16T17:27:14.377 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.377 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.377 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.377 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.377 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.377 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.378 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.378 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Password to authenticate with", 2024-04-16T17:27:14.378 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.378 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.378 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.378 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_port": { 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_port", 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "465", 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.379 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.380 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP port", 2024-04-16T17:27:14.380 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.380 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.380 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.380 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.380 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_sender": { 2024-04-16T17:27:14.380 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_sender", 2024-04-16T17:27:14.380 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP envelope sender", 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.381 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.382 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.382 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.382 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_ssl": { 2024-04-16T17:27:14.382 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_ssl", 2024-04-16T17:27:14.382 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.382 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.382 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.382 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.383 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.383 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.383 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.383 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T17:27:14.383 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.383 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.383 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.383 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.384 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_user": { 2024-04-16T17:27:14.384 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_user", 2024-04-16T17:27:14.384 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.384 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.384 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.384 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.384 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.384 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.385 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.385 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "User to authenticate as", 2024-04-16T17:27:14.385 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.385 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.385 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.385 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.385 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.386 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.386 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.386 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "balancer", 2024-04-16T17:27:14.386 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.386 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.386 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.386 INFO:teuthology.orchestra.run.smithi018.stdout: "active": { 2024-04-16T17:27:14.386 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "active", 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T17:27:14.387 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_time": { 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_time", 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.388 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0000", 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.389 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_weekday": { 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_weekday", 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.390 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T17:27:14.391 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:27:14.391 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.391 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.391 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.391 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_max_iterations": { 2024-04-16T17:27:14.391 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_max_iterations", 2024-04-16T17:27:14.391 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:14.391 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "25", 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "250", 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.392 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_metrics": { 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_metrics", 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.393 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_step": { 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_step", 2024-04-16T17:27:14.394 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.395 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.395 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.395 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.5", 2024-04-16T17:27:14.395 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0.001", 2024-04-16T17:27:14.395 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "0.999", 2024-04-16T17:27:14.395 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.395 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "aggressiveness of optimization", 2024-04-16T17:27:14.396 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T17:27:14.396 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.396 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.397 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.397 INFO:teuthology.orchestra.run.smithi018.stdout: "end_time": { 2024-04-16T17:27:14.397 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_time", 2024-04-16T17:27:14.397 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.397 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.397 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.397 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2359", 2024-04-16T17:27:14.397 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.398 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.398 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.398 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T17:27:14.398 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:27:14.399 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.399 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.399 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.399 INFO:teuthology.orchestra.run.smithi018.stdout: "end_weekday": { 2024-04-16T17:27:14.399 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_weekday", 2024-04-16T17:27:14.399 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:14.399 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.400 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.401 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.401 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.401 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.401 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.401 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.401 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.401 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.401 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.402 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.403 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.404 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.405 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.406 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.406 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.406 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.406 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.406 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.406 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.406 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.406 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.407 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.408 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.409 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.409 INFO:teuthology.orchestra.run.smithi018.stdout: "min_score": { 2024-04-16T17:27:14.409 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_score", 2024-04-16T17:27:14.409 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.409 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.409 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.409 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:14.409 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.410 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.410 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.410 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T17:27:14.410 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.410 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.410 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.410 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.410 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "upmap", 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.411 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-compat", 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: "none", 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: "read", 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap", 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap-read" 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Balancer mode", 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.412 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.413 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_ids": { 2024-04-16T17:27:14.413 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_ids", 2024-04-16T17:27:14.413 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.413 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.413 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.413 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.413 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.413 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.414 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.414 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T17:27:14.414 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.414 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.414 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.414 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.414 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:14.415 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:14.415 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.415 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.415 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.415 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:14.415 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.415 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_deviation": { 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_deviation", 2024-04-16T17:27:14.416 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T17:27:14.417 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.418 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.418 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.418 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_optimizations": { 2024-04-16T17:27:14.418 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_optimizations", 2024-04-16T17:27:14.418 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:14.418 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.418 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.419 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.420 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.420 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.420 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.420 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm", 2024-04-16T17:27:14.420 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.420 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.420 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.420 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_down_multiplier": { 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_down_multiplier", 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.421 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T17:27:14.422 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.422 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.422 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.422 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.422 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_refresh_rate": { 2024-04-16T17:27:14.422 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_refresh_rate", 2024-04-16T17:27:14.422 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.423 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.423 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.423 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "20", 2024-04-16T17:27:14.423 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.423 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.423 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.423 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T17:27:14.423 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_starting_port": { 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_starting_port", 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.424 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4721", 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.425 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_ptrace": { 2024-04-16T17:27:14.426 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_ptrace", 2024-04-16T17:27:14.426 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.426 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.426 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.426 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.426 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.426 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.427 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.427 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T17:27:14.427 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T17:27:14.427 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.428 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.428 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_interval": { 2024-04-16T17:27:14.428 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_interval", 2024-04-16T17:27:14.429 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.429 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.429 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.429 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:14.430 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.430 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.430 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.430 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T17:27:14.430 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.430 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.431 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.431 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_memory_target_ratio": { 2024-04-16T17:27:14.431 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T17:27:14.431 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.431 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.431 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.431 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.7", 2024-04-16T17:27:14.432 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.432 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.432 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.432 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T17:27:14.432 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.432 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.432 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.432 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm_log_destination": { 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm_log_destination", 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.433 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.434 INFO:teuthology.orchestra.run.smithi018.stdout: "file", 2024-04-16T17:27:14.434 INFO:teuthology.orchestra.run.smithi018.stdout: "file,syslog", 2024-04-16T17:27:14.434 INFO:teuthology.orchestra.run.smithi018.stdout: "syslog" 2024-04-16T17:27:14.434 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.434 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T17:27:14.434 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.434 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.435 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.435 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.435 INFO:teuthology.orchestra.run.smithi018.stdout: "cgroups_split": { 2024-04-16T17:27:14.435 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cgroups_split", 2024-04-16T17:27:14.435 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.435 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.435 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.436 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: "config_checks_enabled": { 2024-04-16T17:27:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_checks_enabled", 2024-04-16T17:27:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.437 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: "config_dashboard": { 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_dashboard", 2024-04-16T17:27:14.438 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.439 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_alertmanager": { 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_alertmanager", 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T17:27:14.440 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_base": { 2024-04-16T17:27:14.441 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_base", 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Container image name, without the tag", 2024-04-16T17:27:14.442 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_elasticsearch": { 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_elasticsearch", 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.443 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T17:27:14.444 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.444 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.444 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.444 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "elasticsearch container image", 2024-04-16T17:27:14.444 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.444 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.444 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_grafana": { 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_grafana", 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.445 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_haproxy": { 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_haproxy", 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.446 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "HAproxy container image", 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.447 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_agent": { 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_agent", 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.448 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger agent container image", 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_collector": { 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_collector", 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.449 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.450 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.450 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T17:27:14.450 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.450 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.450 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.450 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger collector container image", 2024-04-16T17:27:14.450 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.450 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_query": { 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_query", 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.451 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger query container image", 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_keepalived": { 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_keepalived", 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.452 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Keepalived container image", 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.453 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_loki": { 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_loki", 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.454 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Loki container image", 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_node_exporter": { 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_node_exporter", 2024-04-16T17:27:14.455 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.456 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.456 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.456 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T17:27:14.456 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.456 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.456 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.456 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:14.457 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.457 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.457 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.457 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.457 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_nvmeof": { 2024-04-16T17:27:14.457 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_nvmeof", 2024-04-16T17:27:14.457 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.457 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Nvme-of container image", 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.458 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.459 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.459 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_prometheus": { 2024-04-16T17:27:14.459 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_prometheus", 2024-04-16T17:27:14.459 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.459 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.459 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.459 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T17:27:14.459 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.460 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.460 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.460 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:14.460 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.460 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.460 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.460 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.460 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_promtail": { 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_promtail", 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.461 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Promtail container image", 2024-04-16T17:27:14.462 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.462 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.462 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.462 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.462 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_samba": { 2024-04-16T17:27:14.462 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_samba", 2024-04-16T17:27:14.462 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.462 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Samba/SMB container image", 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.463 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_snmp_gateway": { 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_snmp_gateway", 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.464 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SNMP Gateway container image", 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: "container_init": { 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_init", 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.465 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.466 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "daemon_cache_timeout": { 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "daemon_cache_timeout", 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.467 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: "default_cephadm_command_timeout": { 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.468 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "900", 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.469 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "default_registry": { 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_registry", 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io", 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: "device_cache_timeout": { 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_cache_timeout", 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.471 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1800", 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache device inventory", 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.472 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "device_enhanced_scan": { 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_enhanced_scan", 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.473 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: "facts_cache_timeout": { 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "facts_cache_timeout", 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.474 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache host facts data", 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.475 INFO:teuthology.orchestra.run.smithi018.stdout: "grafana_dashboards_path": { 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "grafana_dashboards_path", 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T17:27:14.476 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "host_check_interval": { 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "host_check_interval", 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:14.477 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.478 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.478 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.478 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to perform a host check", 2024-04-16T17:27:14.478 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.478 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.478 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.478 INFO:teuthology.orchestra.run.smithi018.stdout: "hw_monitoring": { 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hw_monitoring", 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.479 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T17:27:14.480 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.480 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.480 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.480 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.480 INFO:teuthology.orchestra.run.smithi018.stdout: "inventory_list_all": { 2024-04-16T17:27:14.480 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "inventory_list_all", 2024-04-16T17:27:14.480 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.480 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.481 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.482 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.483 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "log_refresh_metadata": { 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_refresh_metadata", 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.484 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.485 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.486 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.487 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.488 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.489 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.490 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.491 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T17:27:14.492 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T17:27:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "*", 2024-04-16T17:27:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.493 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "max_count_per_host": { 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_count_per_host", 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.494 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of daemons per service per host", 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd_draining_count": { 2024-04-16T17:27:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_osd_draining_count", 2024-04-16T17:27:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: "migration_current": { 2024-04-16T17:27:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "migration_current", 2024-04-16T17:27:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "internal - do not modify", 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "root", 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm-package", 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "root" 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.500 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "oob_default_addr": { 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "oob_default_addr", 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.501 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "169.254.1.1", 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.502 INFO:teuthology.orchestra.run.smithi018.stdout: "prometheus_alerts_path": { 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus_alerts_path", 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.503 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_insecure": { 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_insecure", 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.504 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.505 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_password": { 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_password", 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.506 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_url": { 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_url", 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.507 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.508 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_username": { 2024-04-16T17:27:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_username", 2024-04-16T17:27:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.509 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T17:27:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "secure_monitoring_stack": { 2024-04-16T17:27:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "secure_monitoring_stack", 2024-04-16T17:27:14.510 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.511 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "service_discovery_port": { 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "service_discovery_port", 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8765", 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.512 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "cephadm service discovery port", 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "ssh_config_file": { 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssh_config_file", 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.513 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.514 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "use_agent": { 2024-04-16T17:27:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_agent", 2024-04-16T17:27:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.515 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "use_repo_digest": { 2024-04-16T17:27:14.516 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_repo_digest", 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T17:27:14.517 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_failed_host_check": { 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_failed_host_check", 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.518 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T17:27:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.519 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_daemons": { 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_daemons", 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.520 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_hosts": { 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_hosts", 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.521 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.522 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crash", 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.523 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.524 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.526 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.527 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.528 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.529 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.530 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.531 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "retain_interval": { 2024-04-16T17:27:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retain_interval", 2024-04-16T17:27:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "31536000", 2024-04-16T17:27:14.532 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T17:27:14.533 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_recent_interval": { 2024-04-16T17:27:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_recent_interval", 2024-04-16T17:27:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.534 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1209600", 2024-04-16T17:27:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T17:27:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.535 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dashboard", 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.536 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.537 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.538 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.540 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_ENABLED": { 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.541 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.543 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.544 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.545 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.546 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T17:27:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T17:27:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.547 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.548 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.549 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T17:27:14.550 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.551 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.552 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T17:27:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T17:27:14.553 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea240ac7b0 msgr2=0x7fea240aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea240ac7b0 0x7fea240aec70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fea30009b50 tx=0x7fea30005f90 comp rx=0 tx=0).stop 2024-04-16T17:27:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3809acf0 msgr2=0x7fea380078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:27:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3809acf0 0x7fea380078c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fea40067760 tx=0x7fea40077690 comp rx=0 tx=0).stop 2024-04-16T17:27:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 shutdown_connections 2024-04-16T17:27:14.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fea240ac7b0 0x7fea240aec70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:14.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3809acf0 0x7fea380078c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:14.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea380968f0 0x7fea38007380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:14.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 --2- 172.21.15.18:0/2054630423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fea38095f30 0x7fea38006e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:27:14.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.676+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 >> 172.21.15.18:0/2054630423 conn(0x7fea3800cb40 msgr2=0x7fea38093ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:27:14.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.677+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 shutdown_connections 2024-04-16T17:27:14.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:27:13.677+0000 7fea3f200640 1 -- 172.21.15.18:0/2054630423 wait complete. 2024-04-16T17:27:14.558 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:14 smithi043 ceph-mon[36901]: pgmap v720: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 455 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 233 KiB/s wr, 52 op/s 2024-04-16T17:27:14.558 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:14 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2054630423' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T17:27:14.558 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:14 smithi022 ceph-mon[36023]: pgmap v720: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 455 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 233 KiB/s wr, 52 op/s 2024-04-16T17:27:14.559 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:14 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2054630423' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T17:27:14.560 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.561 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.561 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.561 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.561 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.561 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.561 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.561 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.561 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.562 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T17:27:14.563 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T17:27:14.564 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.564 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.564 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.564 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.564 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.564 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.564 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.564 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.565 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.565 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.565 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.565 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.566 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T17:27:14.566 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T17:27:14.566 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.567 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.567 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.567 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:14.567 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.567 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.567 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.567 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.567 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.568 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.568 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.568 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.568 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T17:27:14.568 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T17:27:14.568 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.568 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.568 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_URL": { 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_URL", 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.570 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.571 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.572 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:14.572 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.572 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.572 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.572 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.572 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.572 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.572 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.573 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T17:27:14.574 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.575 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.576 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.577 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.578 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "[]", 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.579 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.580 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.581 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "{}", 2024-04-16T17:27:14.581 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.581 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.581 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.581 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.581 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.581 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.581 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.582 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.582 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T17:27:14.582 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T17:27:14.582 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.582 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.582 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.582 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.582 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T17:27:14.583 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.584 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.584 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.584 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.584 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.584 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.584 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.584 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.584 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.585 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.585 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.585 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.585 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T17:27:14.585 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T17:27:14.585 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.585 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.585 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.586 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.586 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.586 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.586 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.586 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.586 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.586 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.586 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.587 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T17:27:14.588 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.589 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.590 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T17:27:14.591 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.592 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.593 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T17:27:14.594 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.595 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.596 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T17:27:14.597 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.598 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.599 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T17:27:14.600 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8", 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.601 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "45", 2024-04-16T17:27:14.602 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T17:27:14.603 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.604 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.605 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T17:27:14.606 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.607 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.608 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T17:27:14.609 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.610 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.610 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.610 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.610 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.610 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.610 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.610 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.611 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.612 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.613 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.614 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.614 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.614 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.614 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T17:27:14.614 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T17:27:14.614 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.614 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.615 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.615 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:14.615 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.615 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.615 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.615 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.615 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.615 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "cross_origin_url": { 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cross_origin_url", 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.616 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "crt_file": { 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crt_file", 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.617 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.618 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: "debug": { 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "debug", 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.619 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable/disable debug options", 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "jwt_token_ttl": { 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "jwt_token_ttl", 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.620 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "28800", 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.621 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.622 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.623 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.624 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.625 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.625 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.625 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.625 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.625 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.625 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.625 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.626 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.626 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.626 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.626 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.626 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.626 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.626 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.626 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.627 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.628 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.630 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.630 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.630 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.630 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.630 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.630 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.630 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: "motd": { 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "motd", 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.631 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "The message of the day", 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect_resolve_ip_addr": { 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.633 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.634 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.634 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.634 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.634 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:14.634 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:14.634 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.634 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.635 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8080", 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.636 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.637 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.637 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.637 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.637 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.637 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.637 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.637 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-04-16T17:27:14.637 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.638 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl_server_port": { 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl_server_port", 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.639 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8443", 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-04-16T17:27:14.640 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "redirect", 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.641 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect" 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.642 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.643 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.644 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.644 INFO:teuthology.orchestra.run.smithi018.stdout: "url_prefix": { 2024-04-16T17:27:14.644 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url_prefix", 2024-04-16T17:27:14.644 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.644 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.644 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.644 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.645 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "devicehealth", 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_monitoring": { 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_monitoring", 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.646 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "monitor device health metrics", 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.647 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.648 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.649 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.649 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.649 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.649 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.649 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.649 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.649 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.649 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.650 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.650 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.650 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.650 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.650 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.650 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.650 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.650 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.651 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.652 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.653 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.654 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.654 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.654 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.654 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.654 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.654 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.654 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.655 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.655 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.655 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.655 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.655 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.655 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.655 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.655 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mark_out_threshold": { 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mark_out_threshold", 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.656 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2419200", 2024-04-16T17:27:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T17:27:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.657 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": { 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_name", 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "device_health_metrics", 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.658 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: "retention_period": { 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retention_period", 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.659 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.660 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15552000", 2024-04-16T17:27:14.660 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.660 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.660 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.660 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain device health metrics", 2024-04-16T17:27:14.660 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.660 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.660 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_frequency": { 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_frequency", 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.661 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: "self_heal": { 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "self_heal", 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.662 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.663 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.663 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.663 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.663 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.663 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.663 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T17:27:14.663 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.663 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.664 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.665 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.665 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T17:27:14.665 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.665 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.665 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.665 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.665 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_threshold": { 2024-04-16T17:27:14.665 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_threshold", 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7257600", 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.666 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.667 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.667 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.667 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.667 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.667 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.668 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "diskprediction_local", 2024-04-16T17:27:14.668 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.668 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.668 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.668 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.669 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.669 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.669 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.669 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.669 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.669 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.669 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.670 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.670 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.670 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.670 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.672 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.672 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.672 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.672 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.672 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.672 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.672 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.672 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.673 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.674 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.675 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.676 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "predict_interval": { 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predict_interval", 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.677 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.678 INFO:teuthology.orchestra.run.smithi018.stdout: "predictor_model": { 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predictor_model", 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "prophetstor", 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.679 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.680 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.680 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.680 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.680 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.680 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:14.680 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:14.680 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.681 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.681 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.681 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:14.681 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.681 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.681 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.681 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.681 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "influx", 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": false, 2024-04-16T17:27:14.682 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "influxdb python module not found", 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "batch_size": { 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "batch_size", 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5000", 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.683 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: "database": { 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "database", 2024-04-16T17:27:14.684 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "ceph", 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.685 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.686 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.686 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.686 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": { 2024-04-16T17:27:14.686 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hostname", 2024-04-16T17:27:14.686 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.686 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.686 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server hostname", 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.687 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "30", 2024-04-16T17:27:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "5", 2024-04-16T17:27:14.688 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.689 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.690 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.691 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.692 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.692 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.692 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.692 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.692 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.692 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.692 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.692 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.693 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.693 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.693 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.693 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.693 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.693 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.693 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.694 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.694 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.694 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.694 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.695 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.695 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.695 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.695 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.696 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.697 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "password": { 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "password", 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.698 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "password of InfluxDB server user", 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.699 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "port": { 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "port", 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8086", 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.700 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server port", 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.701 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "false", 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.702 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "threads": { 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threads", 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "32", 2024-04-16T17:27:14.703 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "username": { 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "username", 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.704 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.705 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.705 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.705 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.705 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.705 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "username of InfluxDB server user", 2024-04-16T17:27:14.705 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.705 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.706 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.706 INFO:teuthology.orchestra.run.smithi018.stdout: "verify_ssl": { 2024-04-16T17:27:14.706 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "verify_ssl", 2024-04-16T17:27:14.706 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.706 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.706 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "true", 2024-04-16T17:27:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:27:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.707 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.708 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.708 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.708 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.708 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.708 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "insights", 2024-04-16T17:27:14.708 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.708 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.708 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.709 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.710 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.710 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.710 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.710 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.710 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.710 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.710 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.710 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.711 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.711 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.711 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.711 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.711 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.711 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.711 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.711 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.712 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.712 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.712 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.712 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.712 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.712 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.712 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.712 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.713 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.714 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.714 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.714 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.714 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.714 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.714 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.714 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.715 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.715 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.715 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.715 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.715 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.715 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.715 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.716 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.716 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.716 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.716 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.716 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.716 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.716 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.716 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.717 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.718 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "iostat", 2024-04-16T17:27:14.718 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.718 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.718 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.718 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.718 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.718 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.718 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.719 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.719 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.719 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.719 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.719 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.719 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.719 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.719 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.720 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.721 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.721 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.721 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.721 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.721 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.721 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.721 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.722 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.722 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.722 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.722 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.722 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.722 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.722 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.722 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.723 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.724 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.724 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.724 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.724 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.724 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.724 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.724 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.724 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.725 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.726 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.726 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.726 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.726 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.726 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.726 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.726 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.726 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.727 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.727 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.727 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "k8sevents", 2024-04-16T17:27:14.727 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.727 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.727 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.727 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_event_retention_days": { 2024-04-16T17:27:14.728 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ceph_event_retention_days", 2024-04-16T17:27:14.728 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.728 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.728 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.728 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7", 2024-04-16T17:27:14.728 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.728 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.729 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.729 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T17:27:14.729 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.729 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.729 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.729 INFO:teuthology.orchestra.run.smithi018.stdout: "config_check_secs": { 2024-04-16T17:27:14.730 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_check_secs", 2024-04-16T17:27:14.730 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.730 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.730 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.730 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.730 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "10", 2024-04-16T17:27:14.730 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.731 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.731 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T17:27:14.731 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.731 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.731 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.731 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.732 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.732 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.732 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.732 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.732 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.732 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.732 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.733 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.733 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.733 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.733 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.733 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.733 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.733 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.734 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.734 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.734 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.734 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.734 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.734 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.735 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.735 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.735 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.735 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.735 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.735 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.735 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.736 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.736 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.736 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.736 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.736 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.736 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.737 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.737 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.737 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.737 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.737 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.737 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.737 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.737 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.738 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.738 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.738 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.738 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.738 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.738 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.738 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.738 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.739 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.740 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.740 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.740 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.740 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.740 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.740 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.740 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.740 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "localpool", 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.741 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.742 INFO:teuthology.orchestra.run.smithi018.stdout: "failure_domain": { 2024-04-16T17:27:14.742 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "failure_domain", 2024-04-16T17:27:14.742 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.742 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.742 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.742 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "host", 2024-04-16T17:27:14.742 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.742 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.743 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.743 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "failure domain for any created local pool", 2024-04-16T17:27:14.743 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T17:27:14.743 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.743 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.743 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.743 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.743 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.744 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.745 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.745 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.745 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.745 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.745 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.745 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.745 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.745 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.746 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.746 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.746 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.746 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.746 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.746 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.746 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.746 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.747 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.748 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.748 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.748 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.748 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.748 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.748 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.748 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.748 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.749 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.750 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.750 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.750 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.750 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.750 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.750 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.750 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.751 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.752 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": { 2024-04-16T17:27:14.752 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_size", 2024-04-16T17:27:14.752 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.752 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.752 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.752 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.752 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.752 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default min_size for any created local pool", 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: "num_rep": { 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "num_rep", 2024-04-16T17:27:14.753 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.754 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.754 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.754 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3", 2024-04-16T17:27:14.754 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.754 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.754 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.754 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default replica count for any created local pool", 2024-04-16T17:27:14.754 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.755 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.755 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.755 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.755 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": { 2024-04-16T17:27:14.755 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_num", 2024-04-16T17:27:14.755 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.755 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.755 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "128", 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.756 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.757 INFO:teuthology.orchestra.run.smithi018.stdout: "prefix": { 2024-04-16T17:27:14.757 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prefix", 2024-04-16T17:27:14.757 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.757 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.757 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.757 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.757 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.758 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.758 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.758 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name prefix for any created local pool", 2024-04-16T17:27:14.758 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.758 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.758 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.758 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.758 INFO:teuthology.orchestra.run.smithi018.stdout: "subtree": { 2024-04-16T17:27:14.759 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "subtree", 2024-04-16T17:27:14.759 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.759 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.759 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.759 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "rack", 2024-04-16T17:27:14.759 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.759 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.759 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.760 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mds_autoscaler", 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.761 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.762 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.762 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.762 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.762 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.762 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.762 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.762 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.763 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.764 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.764 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.764 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.764 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.764 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.764 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.764 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.764 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.765 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.765 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.765 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.765 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.765 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.765 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.765 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.765 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.766 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.766 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.766 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.766 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.766 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.766 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.766 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.766 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.767 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.768 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.768 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.768 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.768 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.768 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.768 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.768 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.768 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.769 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.769 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.769 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.769 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.769 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.769 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.769 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirroring", 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.770 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.771 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.771 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.771 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.771 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.771 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.771 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.771 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.771 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.772 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.773 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.774 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.774 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.774 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.774 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.774 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.774 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.774 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.774 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.775 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.776 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.776 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.776 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.776 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.776 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.776 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.776 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.776 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.777 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.777 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.777 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.777 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.777 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.777 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.777 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.777 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.778 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.778 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.778 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.778 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.778 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.778 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.778 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.778 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "nfs", 2024-04-16T17:27:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.781 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.781 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.781 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.781 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.781 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.781 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.781 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.782 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.782 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.782 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.782 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.782 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.782 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.782 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.783 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.783 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.783 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.783 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.783 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.783 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.783 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.783 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.784 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.784 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.784 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.784 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.784 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.784 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.784 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.784 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.785 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.785 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.785 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.785 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.785 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.785 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.785 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.785 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.786 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.786 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.786 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.786 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.786 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.786 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.786 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.786 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.787 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.788 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "fail_fs": { 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "fail_fs", 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.789 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.790 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.790 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.790 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.790 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.790 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T17:27:14.790 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.790 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.790 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.791 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.791 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.791 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.791 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.791 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.791 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.791 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.791 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.792 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.793 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.794 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.794 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.794 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.794 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.794 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.794 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.794 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.795 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.796 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.796 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.796 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.796 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.796 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.796 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.797 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.797 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.797 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.797 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.798 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.798 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.798 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.798 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.798 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.798 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.798 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.799 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.799 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.799 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.799 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.800 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.800 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.800 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.800 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.800 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.800 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.800 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator": { 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.801 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.802 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.802 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm", 2024-04-16T17:27:14.802 INFO:teuthology.orchestra.run.smithi018.stdout: "rook", 2024-04-16T17:27:14.802 INFO:teuthology.orchestra.run.smithi018.stdout: "test_orchestrator" 2024-04-16T17:27:14.802 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.802 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Orchestrator backend", 2024-04-16T17:27:14.802 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.802 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_perf_query", 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.803 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.804 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.805 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.806 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.807 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.807 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.807 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.807 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.807 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.807 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.808 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.808 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.808 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.808 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.808 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.808 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.808 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.808 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.809 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.809 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.809 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.809 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.809 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.809 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.809 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.809 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.810 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.810 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.810 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.810 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.810 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.810 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.810 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.810 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.811 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_support", 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.812 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.813 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.814 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.814 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.814 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.814 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.814 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.814 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.814 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.815 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.815 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.815 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.815 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.815 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.815 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.815 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.815 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.816 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.817 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.818 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.819 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.819 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.819 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.821 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.822 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.822 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.822 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.822 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.822 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.822 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.822 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.822 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.823 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.823 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.823 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.823 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.823 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.823 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.823 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.823 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_autoscaler", 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.824 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.825 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.826 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.826 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.826 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.826 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.826 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.826 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.826 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.826 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.827 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.827 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.827 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.827 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.827 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.827 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.827 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.827 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.828 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.828 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.828 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.828 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.829 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.829 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.829 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.829 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.830 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.830 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.830 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.830 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.830 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.835 INFO:teuthology.orchestra.run.smithi018.stdout: "threshold": { 2024-04-16T17:27:14.836 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threshold", 2024-04-16T17:27:14.836 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.836 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.836 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.836 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-04-16T17:27:14.836 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1.0", 2024-04-16T17:27:14.836 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.836 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.837 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "scaling threshold", 2024-04-16T17:27:14.837 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T17:27:14.837 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.837 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.837 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.837 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.837 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.838 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.838 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "progress", 2024-04-16T17:27:14.838 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.838 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.838 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.838 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_pg_recovery_event": { 2024-04-16T17:27:14.838 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_pg_recovery_event", 2024-04-16T17:27:14.838 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.839 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.839 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.839 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.839 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.839 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.839 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.839 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T17:27:14.839 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.840 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.840 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.840 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-04-16T17:27:14.840 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-04-16T17:27:14.840 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.840 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.840 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.841 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.841 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.841 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.841 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.841 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.841 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.841 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.841 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.842 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.842 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.842 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.842 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.842 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.842 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.842 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.842 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.843 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.844 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.844 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.844 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.844 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.844 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.844 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.844 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.844 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.845 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.846 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.847 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.848 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.849 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.850 INFO:teuthology.orchestra.run.smithi018.stdout: "max_completed_events": { 2024-04-16T17:27:14.850 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_completed_events", 2024-04-16T17:27:14.850 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.850 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.850 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.850 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "50", 2024-04-16T17:27:14.850 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.850 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "number of past completed events to remember", 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:14.851 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long the module is going to sleep", 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.852 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus", 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.853 INFO:teuthology.orchestra.run.smithi018.stdout: "cache": { 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cache", 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.854 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: "exclude_perf_counters": { 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "exclude_perf_counters", 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.855 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.856 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.856 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.856 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.856 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.856 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T17:27:14.856 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T17:27:14.856 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.856 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.857 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.858 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.858 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.858 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.858 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.858 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.858 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.858 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.858 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.859 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.860 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.861 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.862 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.863 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.864 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.864 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.864 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.864 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.864 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.864 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.864 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.864 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools": { 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools", 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.865 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.866 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.866 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.866 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.866 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.866 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.866 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.866 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T17:27:14.867 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T17:27:14.867 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.867 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.867 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.867 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300", 2024-04-16T17:27:14.867 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.867 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.867 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_interval": { 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_interval", 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.868 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15.0", 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.869 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.870 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.870 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:14.870 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:14.870 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.870 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.870 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.870 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-04-16T17:27:14.870 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.871 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.871 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.871 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T17:27:14.871 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.871 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.871 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.871 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.871 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "9283", 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.872 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "stale_cache_strategy": { 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stale_cache_strategy", 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.873 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "log", 2024-04-16T17:27:14.874 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.874 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.874 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.874 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.874 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.874 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.874 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.874 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "default", 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.875 INFO:teuthology.orchestra.run.smithi018.stdout: "default", 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: "error" 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-04-16T17:27:14.876 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.877 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.878 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.878 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.878 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.878 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.878 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.878 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.878 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_support", 2024-04-16T17:27:14.878 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.879 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.879 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.879 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.879 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.879 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.879 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.879 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.879 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.880 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.880 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.880 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.880 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.880 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.880 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.880 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.880 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.881 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.881 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.881 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.881 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.881 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.881 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.881 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.881 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.882 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.882 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.882 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.882 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.882 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.882 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.882 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.882 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.883 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.884 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.885 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.885 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.885 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.885 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.886 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.886 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.886 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.886 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.886 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.887 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.887 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.887 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.887 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.887 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.887 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.887 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.887 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.888 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.888 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.888 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.888 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.888 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_snap_create": { 2024-04-16T17:27:14.888 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_snap_create", 2024-04-16T17:27:14.888 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.889 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: "mirror_snapshot_schedule": { 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.890 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.891 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.891 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.891 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.891 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.891 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.891 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.891 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.891 INFO:teuthology.orchestra.run.smithi018.stdout: "trash_purge_schedule": { 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "trash_purge_schedule", 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.892 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.893 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.893 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.893 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.893 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.893 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.893 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.893 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.893 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "restful", 2024-04-16T17:27:14.894 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.894 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.894 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.894 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_auth": { 2024-04-16T17:27:14.894 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_auth", 2024-04-16T17:27:14.894 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.894 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.894 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.895 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.895 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.895 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.895 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.895 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.895 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.895 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.895 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.896 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.897 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.897 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.897 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.897 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.897 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.897 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.897 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.898 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.898 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.898 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.898 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.898 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.898 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.898 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.898 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.899 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.899 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.899 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.899 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.899 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.899 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.899 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.899 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.900 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.902 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.902 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.902 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.902 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.902 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.902 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.902 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.902 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.903 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.904 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.904 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.904 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.904 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.904 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.904 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.904 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.904 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.905 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.906 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.906 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:14.906 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:14.906 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.906 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.906 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.906 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.906 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.907 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.907 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.907 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.907 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.907 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.907 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.907 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.907 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:14.908 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:14.908 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.908 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.908 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.908 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.908 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.909 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.910 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.910 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.910 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rgw", 2024-04-16T17:27:14.910 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.910 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.910 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.910 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.910 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.911 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.912 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.913 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.914 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.914 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.914 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.914 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.914 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.914 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.914 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.914 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.915 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.915 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.915 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.915 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.915 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.915 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.915 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.916 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.917 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.918 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.918 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.918 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.918 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.918 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.918 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.918 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.918 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rook", 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.919 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.920 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.921 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.922 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.922 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.922 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.922 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.922 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.922 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.922 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.922 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.923 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.924 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.925 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.925 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.925 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.925 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.925 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.925 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.925 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.925 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.926 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.926 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.926 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.926 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.926 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.926 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.926 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.926 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.927 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.927 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.927 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.927 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.927 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.927 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.927 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.927 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "storage_class": { 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "storage_class", 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "local", 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.928 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "selftest", 2024-04-16T17:27:14.929 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.930 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.930 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.930 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.930 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.930 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.930 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.931 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.931 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.931 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.931 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.931 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.931 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.931 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.932 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.932 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.932 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.932 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.932 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.933 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.933 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.933 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.933 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.933 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.933 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.934 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.934 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.934 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.935 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.935 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.935 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.935 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.935 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.935 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.935 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.935 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.936 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.936 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.936 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.936 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.936 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.936 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.936 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.937 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.937 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.937 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.937 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.937 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.937 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.937 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.937 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.938 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.939 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "roption1": { 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption1", 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.940 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.941 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.941 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.941 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.941 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.941 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.941 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.941 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.941 INFO:teuthology.orchestra.run.smithi018.stdout: "roption2": { 2024-04-16T17:27:14.942 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption2", 2024-04-16T17:27:14.942 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.942 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.942 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.942 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "xyz", 2024-04-16T17:27:14.942 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.942 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption1": { 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption1", 2024-04-16T17:27:14.943 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.944 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption2": { 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption2", 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.945 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.946 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.946 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.946 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.946 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.946 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.946 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.946 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption3": { 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption3", 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.947 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption4": { 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption4", 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.948 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.950 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption5": { 2024-04-16T17:27:14.950 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption5", 2024-04-16T17:27:14.950 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.950 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.950 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.950 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.950 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.951 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.951 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.951 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.951 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.951 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.951 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.951 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption6": { 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption6", 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.952 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption7": { 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption7", 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.953 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.954 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.954 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:14.954 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "42", 2024-04-16T17:27:14.954 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.954 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.954 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.954 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.954 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: "testkey": { 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testkey", 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.955 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.956 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.956 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.956 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.956 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.956 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.957 INFO:teuthology.orchestra.run.smithi018.stdout: "testlkey": { 2024-04-16T17:27:14.957 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testlkey", 2024-04-16T17:27:14.957 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.958 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.958 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.958 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.958 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.959 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.959 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.959 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.959 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.959 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.959 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.959 INFO:teuthology.orchestra.run.smithi018.stdout: "testnewline": { 2024-04-16T17:27:14.960 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testnewline", 2024-04-16T17:27:14.960 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.960 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.960 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.960 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.960 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.960 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.960 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.961 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-04-16T17:27:14.962 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.962 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.962 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.962 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_m_granularity": { 2024-04-16T17:27:14.962 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_m_granularity", 2024-04-16T17:27:14.962 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.962 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.962 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.963 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.963 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.963 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.963 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.963 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T17:27:14.963 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.963 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.963 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: "dump_on_update": { 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dump_on_update", 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.964 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.965 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.965 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "dump database to debug log on update", 2024-04-16T17:27:14.965 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.965 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.965 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.965 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.965 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.965 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.966 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.966 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.966 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.966 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.966 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.966 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.966 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.967 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.967 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.967 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.967 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.967 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.967 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.967 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.967 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.968 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.968 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.968 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.968 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.968 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.968 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.969 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.969 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.969 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.969 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.969 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.969 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.969 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.970 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.970 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.970 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.970 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.970 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.970 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.971 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.971 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.971 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.971 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.971 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.971 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.971 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.972 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.972 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.972 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.972 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.972 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.972 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.972 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.972 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.973 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.973 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.973 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.973 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.973 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.973 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.974 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.974 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.974 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.974 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.974 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.974 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.974 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stats", 2024-04-16T17:27:14.975 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.976 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.976 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.976 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.976 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.976 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.976 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.976 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.977 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.977 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.977 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.977 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.977 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.977 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.977 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.977 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.978 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.978 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.978 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.978 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.978 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.978 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.978 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.979 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.979 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.979 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.979 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.979 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.979 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.979 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.980 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.980 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.980 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.980 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.980 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.980 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.980 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.981 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.981 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.981 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.981 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.981 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.981 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.981 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.981 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.982 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.982 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.982 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.982 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.982 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.982 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.982 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.982 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.983 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.983 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.983 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.983 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.983 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.983 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.983 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.984 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.984 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.984 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.984 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.984 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.984 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.984 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.984 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "status", 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.985 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.986 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.986 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.986 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.986 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.986 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.986 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.986 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:14.986 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.987 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.987 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.987 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.987 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.987 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.987 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.987 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.987 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.988 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.988 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.988 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.988 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.988 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.988 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:14.988 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:14.989 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.989 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.989 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.989 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.989 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.989 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.989 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.989 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.990 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.990 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.990 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.990 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.990 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:14.990 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:14.990 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:14.991 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:14.992 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:14.992 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:14.992 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:14.992 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:14.992 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.992 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.992 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.992 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.993 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:14.993 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:14.993 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:14.993 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.993 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:14.993 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:14.994 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.994 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.994 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.994 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.994 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.994 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.994 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.995 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.995 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:14.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.995 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:14.995 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telegraf", 2024-04-16T17:27:14.995 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:14.995 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:14.995 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: "address": { 2024-04-16T17:27:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "address", 2024-04-16T17:27:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T17:27:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.996 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:14.997 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15", 2024-04-16T17:27:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:14.998 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:14.999 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.000 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.002 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.002 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.002 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.002 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.002 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.002 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.002 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.002 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.003 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.004 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.004 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.004 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.004 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.004 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.004 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.004 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.004 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.005 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.005 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.005 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.005 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.005 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.005 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.005 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.005 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.006 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.006 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.006 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.006 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.006 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.006 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.006 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.007 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.007 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.007 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.007 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.007 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.007 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.007 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.008 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.008 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.008 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.008 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telemetry", 2024-04-16T17:27:15.008 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.008 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.008 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.009 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_basic": { 2024-04-16T17:27:15.009 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_basic", 2024-04-16T17:27:15.009 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.009 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.009 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.009 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.009 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.010 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.010 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.010 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T17:27:15.010 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.010 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.010 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.010 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_crash": { 2024-04-16T17:27:15.011 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_crash", 2024-04-16T17:27:15.011 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.011 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.011 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.011 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.011 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.011 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.012 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.012 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T17:27:15.012 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.012 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.012 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.012 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_device": { 2024-04-16T17:27:15.013 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_device", 2024-04-16T17:27:15.013 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.013 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.013 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.013 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.013 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.013 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.013 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.014 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T17:27:15.014 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.014 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.014 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.014 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_ident": { 2024-04-16T17:27:15.014 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_ident", 2024-04-16T17:27:15.015 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.015 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.015 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.015 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.015 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.015 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.015 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.015 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T17:27:15.016 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.016 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.016 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.016 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_perf": { 2024-04-16T17:27:15.016 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_perf", 2024-04-16T17:27:15.016 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.016 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.017 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.018 INFO:teuthology.orchestra.run.smithi018.stdout: "contact": { 2024-04-16T17:27:15.018 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "contact", 2024-04-16T17:27:15.018 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.018 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.018 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.018 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.019 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.019 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.019 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.019 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.019 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.019 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.019 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: "description": { 2024-04-16T17:27:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "description", 2024-04-16T17:27:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.020 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.021 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.021 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.021 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.021 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.021 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.021 INFO:teuthology.orchestra.run.smithi018.stdout: "device_url": { 2024-04-16T17:27:15.021 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_url", 2024-04-16T17:27:15.022 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.022 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.022 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.022 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T17:27:15.022 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.022 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.022 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.022 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.023 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.023 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.023 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.023 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.023 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-04-16T17:27:15.023 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-04-16T17:27:15.023 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.023 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.024 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.024 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.024 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.025 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.025 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.025 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.026 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.026 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.026 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.026 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:15.026 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:15.026 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.026 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "24", 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "8", 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.027 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.028 INFO:teuthology.orchestra.run.smithi018.stdout: "last_opt_revision": { 2024-04-16T17:27:15.028 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "last_opt_revision", 2024-04-16T17:27:15.028 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.028 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.028 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.028 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1", 2024-04-16T17:27:15.028 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard": { 2024-04-16T17:27:15.029 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard", 2024-04-16T17:27:15.030 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.030 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.030 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.030 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.030 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.030 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.030 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.030 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.031 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.031 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.031 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.031 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard_description": { 2024-04-16T17:27:15.031 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard_description", 2024-04-16T17:27:15.031 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.031 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.032 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.033 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.034 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.034 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.034 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.034 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.034 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.034 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.034 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.034 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.035 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.035 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.035 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.035 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.035 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.035 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.035 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.035 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.036 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.037 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.037 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.037 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.037 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.037 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.037 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.037 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.037 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.038 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.038 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.038 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.038 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.038 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.038 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.038 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.038 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.039 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.039 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.039 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.039 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.039 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.039 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.039 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.039 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.040 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.040 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.040 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.040 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.040 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.040 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.040 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.040 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.041 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.041 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.041 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.041 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.041 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.041 INFO:teuthology.orchestra.run.smithi018.stdout: "organization": { 2024-04-16T17:27:15.041 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "organization", 2024-04-16T17:27:15.041 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.042 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.043 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.043 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.043 INFO:teuthology.orchestra.run.smithi018.stdout: "proxy": { 2024-04-16T17:27:15.043 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "proxy", 2024-04-16T17:27:15.043 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.043 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.043 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.043 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.044 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.044 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.044 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.044 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.044 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.045 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.045 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.045 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.045 INFO:teuthology.orchestra.run.smithi018.stdout: "url": { 2024-04-16T17:27:15.045 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url", 2024-04-16T17:27:15.045 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.045 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.045 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.046 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T17:27:15.046 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.046 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.046 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.046 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.046 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.046 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.046 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.047 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.047 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.047 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.047 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.047 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "test_orchestrator", 2024-04-16T17:27:15.047 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.047 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.048 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.048 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.048 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.048 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.048 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.048 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.048 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.048 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.049 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.049 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.049 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.049 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.049 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.049 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.049 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.049 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.050 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.050 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.050 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.050 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.050 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.050 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.050 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.051 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.051 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.051 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.051 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.051 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.051 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.051 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.051 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.052 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.052 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.052 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.052 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.052 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.052 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.052 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.053 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.053 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.053 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.053 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.053 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.053 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.053 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.053 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.054 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.054 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.054 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.054 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.054 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.054 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.054 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.054 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.055 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.055 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.055 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.055 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.055 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.055 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.055 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.055 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.056 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.056 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.056 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.056 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.056 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.056 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.056 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.056 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.057 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.057 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.057 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.057 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.057 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "volumes", 2024-04-16T17:27:15.057 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.057 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.058 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.058 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.058 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.058 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.058 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.058 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.058 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.058 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.059 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.059 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.059 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.059 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.059 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.059 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.059 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.060 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.060 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.060 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.060 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.060 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.060 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.060 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.061 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.061 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.061 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.061 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.061 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.061 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.061 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.062 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.063 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.063 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.063 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.063 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.063 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.063 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.063 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.063 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.064 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.064 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.064 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.064 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.064 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.064 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.064 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.064 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.067 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.067 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.067 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_clones": { 2024-04-16T17:27:15.067 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_clones", 2024-04-16T17:27:15.067 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.067 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.067 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.067 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4", 2024-04-16T17:27:15.068 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.068 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.068 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.068 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T17:27:15.068 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.068 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.068 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.068 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.069 INFO:teuthology.orchestra.run.smithi018.stdout: "periodic_async_work": { 2024-04-16T17:27:15.069 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "periodic_async_work", 2024-04-16T17:27:15.069 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.069 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.069 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.069 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.069 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.069 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.070 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.070 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Periodically check for async work", 2024-04-16T17:27:15.070 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.070 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.070 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.070 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.070 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_delay": { 2024-04-16T17:27:15.071 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_delay", 2024-04-16T17:27:15.071 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.071 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.071 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.071 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:15.071 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.071 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.071 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.072 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T17:27:15.072 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.072 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.072 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.072 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_no_wait": { 2024-04-16T17:27:15.072 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T17:27:15.073 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.073 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.073 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.073 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.073 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.073 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.073 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.073 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T17:27:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix", 2024-04-16T17:27:15.075 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.075 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.075 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.075 INFO:teuthology.orchestra.run.smithi018.stdout: "discovery_interval": { 2024-04-16T17:27:15.075 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "discovery_interval", 2024-04-16T17:27:15.075 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:15.075 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.076 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.076 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "100", 2024-04-16T17:27:15.076 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.076 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.076 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.076 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.076 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.076 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.077 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.077 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.077 INFO:teuthology.orchestra.run.smithi018.stdout: "identifier": { 2024-04-16T17:27:15.077 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "identifier", 2024-04-16T17:27:15.077 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.077 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.077 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.077 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.078 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.078 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.078 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.078 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.078 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.078 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.078 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.079 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.079 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:15.079 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:15.079 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.079 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.079 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.079 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:15.079 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.080 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.080 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.080 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.080 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.080 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.080 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.080 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.080 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.081 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.081 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.081 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.081 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.081 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.081 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.081 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.082 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.082 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.082 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.082 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.082 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.082 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.082 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.082 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.083 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.083 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.083 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.083 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.083 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.083 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.083 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.084 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.084 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.084 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.084 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.084 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.084 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.084 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.085 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.085 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.085 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.085 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.085 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.085 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.085 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.085 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.086 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.086 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.086 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.086 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.086 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.086 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.086 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.087 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.087 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.087 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.087 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.087 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.087 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.087 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.087 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.088 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.088 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.088 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.088 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.088 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.088 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.088 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.088 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.089 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.089 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.089 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.089 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.089 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.089 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.089 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.089 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.090 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.090 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_host": { 2024-04-16T17:27:15.090 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_host", 2024-04-16T17:27:15.090 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.090 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.090 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.090 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.091 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.091 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.091 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.091 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.091 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.091 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.091 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.091 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.092 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_port": { 2024-04-16T17:27:15.092 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_port", 2024-04-16T17:27:15.092 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.092 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.092 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.092 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10051", 2024-04-16T17:27:15.092 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.092 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_sender": { 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_sender", 2024-04-16T17:27:15.093 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.094 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.094 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.094 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T17:27:15.094 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.094 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.094 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.094 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.094 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.096 INFO:teuthology.orchestra.run.smithi018.stdout: "modules": [ 2024-04-16T17:27:15.096 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm", 2024-04-16T17:27:15.096 INFO:teuthology.orchestra.run.smithi018.stdout: "dashboard", 2024-04-16T17:27:15.096 INFO:teuthology.orchestra.run.smithi018.stdout: "iostat", 2024-04-16T17:27:15.096 INFO:teuthology.orchestra.run.smithi018.stdout: "nfs", 2024-04-16T17:27:15.096 INFO:teuthology.orchestra.run.smithi018.stdout: "restful", 2024-04-16T17:27:15.096 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_schedule" 2024-04-16T17:27:15.097 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.097 INFO:teuthology.orchestra.run.smithi018.stdout: "available_modules": [ 2024-04-16T17:27:15.097 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.097 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "alerts", 2024-04-16T17:27:15.097 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.097 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.097 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.098 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:15.098 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:15.098 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.098 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.098 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.098 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:15.098 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.099 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.100 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.100 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.100 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.100 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.100 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.100 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.100 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.101 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.102 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.102 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.102 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.102 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.102 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.102 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.102 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.103 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.103 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.103 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.103 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.103 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.103 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.103 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.103 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.104 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.104 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.104 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.104 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.104 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.104 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.104 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.104 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.105 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.105 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.105 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.105 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.105 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.105 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.105 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.106 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.106 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.106 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.106 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.106 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.106 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.106 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.106 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.107 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.107 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.107 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.107 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.107 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.107 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.107 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.108 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.108 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.108 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.108 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.108 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.108 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.108 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.108 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_destination": { 2024-04-16T17:27:15.109 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_destination", 2024-04-16T17:27:15.109 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.109 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.109 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.109 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.109 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.109 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.109 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.110 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email address to send alerts to", 2024-04-16T17:27:15.110 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.110 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.110 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.110 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.110 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_from_name": { 2024-04-16T17:27:15.110 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_from_name", 2024-04-16T17:27:15.111 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.111 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.111 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.111 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "Ceph", 2024-04-16T17:27:15.111 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.111 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.111 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.112 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Email From: name", 2024-04-16T17:27:15.112 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.112 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.112 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.112 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.112 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_host": { 2024-04-16T17:27:15.112 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_host", 2024-04-16T17:27:15.112 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.113 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.113 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.113 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.113 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.113 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.113 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.113 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP server", 2024-04-16T17:27:15.114 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.114 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.114 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.114 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.114 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_password": { 2024-04-16T17:27:15.114 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_password", 2024-04-16T17:27:15.114 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.114 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.115 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.115 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.115 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.115 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.115 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.115 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Password to authenticate with", 2024-04-16T17:27:15.115 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.115 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.116 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.116 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.116 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_port": { 2024-04-16T17:27:15.116 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_port", 2024-04-16T17:27:15.116 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.116 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.116 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.117 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "465", 2024-04-16T17:27:15.117 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.117 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.117 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.117 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP port", 2024-04-16T17:27:15.117 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.118 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.118 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.118 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.118 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_sender": { 2024-04-16T17:27:15.118 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_sender", 2024-04-16T17:27:15.118 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.118 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.119 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.119 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.119 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.119 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.119 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.119 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SMTP envelope sender", 2024-04-16T17:27:15.119 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.119 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.120 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.120 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.120 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_ssl": { 2024-04-16T17:27:15.120 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_ssl", 2024-04-16T17:27:15.120 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.120 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.120 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.121 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.121 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.121 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.121 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.121 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T17:27:15.121 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.122 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.122 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.122 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.122 INFO:teuthology.orchestra.run.smithi018.stdout: "smtp_user": { 2024-04-16T17:27:15.122 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "smtp_user", 2024-04-16T17:27:15.122 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.122 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.122 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.123 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.123 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.123 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.123 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.123 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "User to authenticate as", 2024-04-16T17:27:15.123 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.123 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.124 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.124 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.124 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.124 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.124 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.124 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "balancer", 2024-04-16T17:27:15.124 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.124 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.125 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.125 INFO:teuthology.orchestra.run.smithi018.stdout: "active": { 2024-04-16T17:27:15.125 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "active", 2024-04-16T17:27:15.125 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.125 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.125 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.125 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.125 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.126 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.126 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.126 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T17:27:15.126 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.126 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.126 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.127 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.127 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_time": { 2024-04-16T17:27:15.127 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_time", 2024-04-16T17:27:15.127 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.127 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.127 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.127 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0000", 2024-04-16T17:27:15.127 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: "begin_weekday": { 2024-04-16T17:27:15.128 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "begin_weekday", 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T17:27:15.129 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:27:15.130 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.130 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.130 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.130 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_max_iterations": { 2024-04-16T17:27:15.130 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_max_iterations", 2024-04-16T17:27:15.130 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:15.130 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.130 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.131 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "25", 2024-04-16T17:27:15.131 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:15.131 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "250", 2024-04-16T17:27:15.131 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.131 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T17:27:15.131 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.131 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.132 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.132 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.132 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_metrics": { 2024-04-16T17:27:15.132 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_metrics", 2024-04-16T17:27:15.132 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.132 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.132 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.132 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T17:27:15.133 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.133 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.133 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.133 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T17:27:15.133 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T17:27:15.133 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.133 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.133 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.134 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_compat_step": { 2024-04-16T17:27:15.134 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crush_compat_step", 2024-04-16T17:27:15.134 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:15.134 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.134 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.134 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.5", 2024-04-16T17:27:15.134 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0.001", 2024-04-16T17:27:15.134 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "0.999", 2024-04-16T17:27:15.135 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.135 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "aggressiveness of optimization", 2024-04-16T17:27:15.135 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T17:27:15.135 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.135 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.135 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.135 INFO:teuthology.orchestra.run.smithi018.stdout: "end_time": { 2024-04-16T17:27:15.135 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_time", 2024-04-16T17:27:15.136 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.136 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.136 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.136 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2359", 2024-04-16T17:27:15.136 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.136 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.136 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.137 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T17:27:15.137 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:27:15.137 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.137 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.137 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.137 INFO:teuthology.orchestra.run.smithi018.stdout: "end_weekday": { 2024-04-16T17:27:15.137 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "end_weekday", 2024-04-16T17:27:15.137 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:15.138 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.138 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.138 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:15.138 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "0", 2024-04-16T17:27:15.138 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "6", 2024-04-16T17:27:15.138 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.138 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T17:27:15.139 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:27:15.139 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.139 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.139 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.139 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.139 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.139 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.139 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.140 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.140 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.140 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.140 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.140 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.140 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.140 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.140 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.141 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.141 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.141 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.141 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.141 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.141 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.141 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.141 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.142 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.142 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.142 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.142 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.142 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.142 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.142 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.142 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.143 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.143 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.143 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.143 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.143 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.143 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.143 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.143 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.144 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.144 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.144 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.144 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.144 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.144 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.144 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.144 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "min_score": { 2024-04-16T17:27:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_score", 2024-04-16T17:27:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T17:27:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-04-16T17:27:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-04-16T17:27:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "upmap", 2024-04-16T17:27:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-compat", 2024-04-16T17:27:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "none", 2024-04-16T17:27:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "read", 2024-04-16T17:27:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap", 2024-04-16T17:27:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap-read" 2024-04-16T17:27:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Balancer mode", 2024-04-16T17:27:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_ids": { 2024-04-16T17:27:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_ids", 2024-04-16T17:27:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T17:27:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_deviation": { 2024-04-16T17:27:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_deviation", 2024-04-16T17:27:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T17:27:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T17:27:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "upmap_max_optimizations": { 2024-04-16T17:27:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "upmap_max_optimizations", 2024-04-16T17:27:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T17:27:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm", 2024-04-16T17:27:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_down_multiplier": { 2024-04-16T17:27:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_down_multiplier", 2024-04-16T17:27:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-04-16T17:27:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T17:27:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_refresh_rate": { 2024-04-16T17:27:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_refresh_rate", 2024-04-16T17:27:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "20", 2024-04-16T17:27:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "agent_starting_port": { 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "agent_starting_port", 2024-04-16T17:27:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4721", 2024-04-16T17:27:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T17:27:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_ptrace": { 2024-04-16T17:27:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_ptrace", 2024-04-16T17:27:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_interval": { 2024-04-16T17:27:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_interval", 2024-04-16T17:27:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T17:27:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "autotune_memory_target_ratio": { 2024-04-16T17:27:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T17:27:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0.7", 2024-04-16T17:27:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm_log_destination": { 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cephadm_log_destination", 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "file", 2024-04-16T17:27:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "file,syslog", 2024-04-16T17:27:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "syslog" 2024-04-16T17:27:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T17:27:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "cgroups_split": { 2024-04-16T17:27:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cgroups_split", 2024-04-16T17:27:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T17:27:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "config_checks_enabled": { 2024-04-16T17:27:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_checks_enabled", 2024-04-16T17:27:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T17:27:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "config_dashboard": { 2024-04-16T17:27:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_dashboard", 2024-04-16T17:27:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T17:27:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_alertmanager": { 2024-04-16T17:27:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_alertmanager", 2024-04-16T17:27:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T17:27:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_base": { 2024-04-16T17:27:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_base", 2024-04-16T17:27:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T17:27:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Container image name, without the tag", 2024-04-16T17:27:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_elasticsearch": { 2024-04-16T17:27:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_elasticsearch", 2024-04-16T17:27:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T17:27:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "elasticsearch container image", 2024-04-16T17:27:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_grafana": { 2024-04-16T17:27:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_grafana", 2024-04-16T17:27:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T17:27:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_haproxy": { 2024-04-16T17:27:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_haproxy", 2024-04-16T17:27:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T17:27:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "HAproxy container image", 2024-04-16T17:27:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_agent": { 2024-04-16T17:27:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_agent", 2024-04-16T17:27:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T17:27:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger agent container image", 2024-04-16T17:27:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_collector": { 2024-04-16T17:27:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_collector", 2024-04-16T17:27:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T17:27:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger collector container image", 2024-04-16T17:27:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_jaeger_query": { 2024-04-16T17:27:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_jaeger_query", 2024-04-16T17:27:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T17:27:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Jaeger query container image", 2024-04-16T17:27:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_keepalived": { 2024-04-16T17:27:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_keepalived", 2024-04-16T17:27:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T17:27:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Keepalived container image", 2024-04-16T17:27:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_loki": { 2024-04-16T17:27:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_loki", 2024-04-16T17:27:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T17:27:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Loki container image", 2024-04-16T17:27:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_node_exporter": { 2024-04-16T17:27:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_node_exporter", 2024-04-16T17:27:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T17:27:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_nvmeof": { 2024-04-16T17:27:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_nvmeof", 2024-04-16T17:27:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T17:27:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Nvme-of container image", 2024-04-16T17:27:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_prometheus": { 2024-04-16T17:27:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_prometheus", 2024-04-16T17:27:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T17:27:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Prometheus container image", 2024-04-16T17:27:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_promtail": { 2024-04-16T17:27:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_promtail", 2024-04-16T17:27:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T17:27:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Promtail container image", 2024-04-16T17:27:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_samba": { 2024-04-16T17:27:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_samba", 2024-04-16T17:27:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T17:27:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Samba/SMB container image", 2024-04-16T17:27:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image_snmp_gateway": { 2024-04-16T17:27:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_image_snmp_gateway", 2024-04-16T17:27:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T17:27:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "SNMP Gateway container image", 2024-04-16T17:27:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "container_init": { 2024-04-16T17:27:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "container_init", 2024-04-16T17:27:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T17:27:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "daemon_cache_timeout": { 2024-04-16T17:27:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "daemon_cache_timeout", 2024-04-16T17:27:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T17:27:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "default_cephadm_command_timeout": { 2024-04-16T17:27:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T17:27:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "900", 2024-04-16T17:27:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "default_registry": { 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "default_registry", 2024-04-16T17:27:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "docker.io", 2024-04-16T17:27:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T17:27:15.215 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.215 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.215 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.215 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.215 INFO:teuthology.orchestra.run.smithi018.stdout: "device_cache_timeout": { 2024-04-16T17:27:15.215 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_cache_timeout", 2024-04-16T17:27:15.215 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.215 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1800", 2024-04-16T17:27:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache device inventory", 2024-04-16T17:27:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "device_enhanced_scan": { 2024-04-16T17:27:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_enhanced_scan", 2024-04-16T17:27:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T17:27:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "facts_cache_timeout": { 2024-04-16T17:27:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "facts_cache_timeout", 2024-04-16T17:27:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "seconds to cache host facts data", 2024-04-16T17:27:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "grafana_dashboards_path": { 2024-04-16T17:27:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "grafana_dashboards_path", 2024-04-16T17:27:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T17:27:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T17:27:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "host_check_interval": { 2024-04-16T17:27:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "host_check_interval", 2024-04-16T17:27:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to perform a host check", 2024-04-16T17:27:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "hw_monitoring": { 2024-04-16T17:27:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hw_monitoring", 2024-04-16T17:27:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T17:27:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "inventory_list_all": { 2024-04-16T17:27:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "inventory_list_all", 2024-04-16T17:27:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "log_refresh_metadata": { 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_refresh_metadata", 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "*", 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "max_count_per_host": { 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_count_per_host", 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of daemons per service per host", 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd_draining_count": { 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_osd_draining_count", 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T17:27:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "migration_current": { 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "migration_current", 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "internal - do not modify", 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "mode": { 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mode", 2024-04-16T17:27:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "root", 2024-04-16T17:27:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm-package", 2024-04-16T17:27:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "root" 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "oob_default_addr": { 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "oob_default_addr", 2024-04-16T17:27:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "169.254.1.1", 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "prometheus_alerts_path": { 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus_alerts_path", 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T17:27:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_insecure": { 2024-04-16T17:27:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_insecure", 2024-04-16T17:27:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_password": { 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_password", 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_url": { 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_url", 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "registry_username": { 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "registry_username", 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T17:27:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "secure_monitoring_stack": { 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "secure_monitoring_stack", 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "service_discovery_port": { 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "service_discovery_port", 2024-04-16T17:27:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8765", 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "cephadm service discovery port", 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "ssh_config_file": { 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssh_config_file", 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "use_agent": { 2024-04-16T17:27:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_agent", 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "use_repo_digest": { 2024-04-16T17:27:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "use_repo_digest", 2024-04-16T17:27:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T17:27:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_failed_host_check": { 2024-04-16T17:27:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_failed_host_check", 2024-04-16T17:27:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_daemons": { 2024-04-16T17:27:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_daemons", 2024-04-16T17:27:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_on_stray_hosts": { 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_on_stray_hosts", 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T17:27:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crash", 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "retain_interval": { 2024-04-16T17:27:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retain_interval", 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "31536000", 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T17:27:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_recent_interval": { 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_recent_interval", 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1209600", 2024-04-16T17:27:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dashboard", 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_ENABLED": { 2024-04-16T17:27:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T17:27:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T17:27:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T17:27:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T17:27:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T17:27:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T17:27:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T17:27:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.309 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.310 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.310 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.310 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.310 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.310 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.310 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.310 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.310 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_URL": { 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_URL", 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.311 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.312 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.312 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.312 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T17:27:15.312 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T17:27:15.312 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.312 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.312 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.313 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.314 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T17:27:15.314 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T17:27:15.314 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.314 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.314 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.314 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.314 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.314 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.315 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.315 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.315 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.315 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.315 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.315 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.315 INFO:teuthology.orchestra.run.smithi018.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T17:27:15.315 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.316 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.317 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.318 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.318 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.318 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.318 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.318 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.318 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.318 INFO:teuthology.orchestra.run.smithi018.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.319 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.320 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "[]", 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.321 INFO:teuthology.orchestra.run.smithi018.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T17:27:15.322 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T17:27:15.322 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.322 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.322 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.322 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "{}", 2024-04-16T17:27:15.322 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.322 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.322 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.323 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.323 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.323 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.323 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.323 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.323 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T17:27:15.323 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T17:27:15.323 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.324 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.324 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.324 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.324 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.324 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.324 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.324 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.324 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.325 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.325 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.325 INFO:teuthology.orchestra.run.smithi018.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T17:27:15.325 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T17:27:15.325 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.325 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.326 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.327 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T17:27:15.328 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.329 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T17:27:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T17:27:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.330 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.331 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.332 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.333 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.335 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.336 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.338 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T17:27:15.338 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T17:27:15.338 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.338 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.338 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.338 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.338 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.338 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.339 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.339 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.339 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.339 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.339 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.339 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T17:27:15.340 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T17:27:15.340 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.340 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.340 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.340 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T17:27:15.340 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.341 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.341 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.341 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.341 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.341 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.341 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.341 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T17:27:15.342 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T17:27:15.342 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.342 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.342 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.342 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:15.342 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.342 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T17:27:15.343 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.344 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.344 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.344 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8", 2024-04-16T17:27:15.344 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.344 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.344 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.344 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.345 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.346 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "45", 2024-04-16T17:27:15.346 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.346 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.346 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.346 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.346 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.346 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.347 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.348 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.348 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.348 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.348 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.348 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.348 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.348 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.348 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T17:27:15.349 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T17:27:15.349 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.349 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.349 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.349 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "admin", 2024-04-16T17:27:15.349 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.349 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.349 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.350 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.351 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.351 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.351 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.351 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.351 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.351 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.351 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.352 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.352 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.352 INFO:teuthology.orchestra.run.smithi018.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T17:27:15.352 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T17:27:15.352 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.352 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.352 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.354 INFO:teuthology.orchestra.run.smithi018.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T17:27:15.354 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T17:27:15.354 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.354 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.354 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.354 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.354 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.354 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.355 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.355 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.355 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.355 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.355 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.355 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T17:27:15.355 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T17:27:15.356 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.356 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.356 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.356 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:15.356 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.356 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.356 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.356 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.357 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.357 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.357 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.357 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T17:27:15.357 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T17:27:15.357 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.357 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.358 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.358 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:15.358 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.358 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.358 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.358 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.358 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.358 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.359 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.359 INFO:teuthology.orchestra.run.smithi018.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T17:27:15.359 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T17:27:15.359 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.359 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.359 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.359 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.361 INFO:teuthology.orchestra.run.smithi018.stdout: "cross_origin_url": { 2024-04-16T17:27:15.361 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cross_origin_url", 2024-04-16T17:27:15.361 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.361 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.361 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.361 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.361 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.361 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.362 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.362 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.362 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.362 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.362 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.362 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.362 INFO:teuthology.orchestra.run.smithi018.stdout: "crt_file": { 2024-04-16T17:27:15.363 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "crt_file", 2024-04-16T17:27:15.363 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.363 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.363 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.363 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.363 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.363 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.363 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.364 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.364 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.364 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.364 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.364 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.364 INFO:teuthology.orchestra.run.smithi018.stdout: "debug": { 2024-04-16T17:27:15.364 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "debug", 2024-04-16T17:27:15.364 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.365 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.365 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.365 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.365 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.365 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.365 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.365 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Enable/disable debug options", 2024-04-16T17:27:15.365 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.366 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.366 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.366 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.366 INFO:teuthology.orchestra.run.smithi018.stdout: "jwt_token_ttl": { 2024-04-16T17:27:15.366 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "jwt_token_ttl", 2024-04-16T17:27:15.366 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.366 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.366 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.367 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "28800", 2024-04-16T17:27:15.367 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.367 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.367 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.367 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.367 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.367 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.367 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.368 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.368 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-04-16T17:27:15.368 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-04-16T17:27:15.368 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.368 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.368 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.368 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.368 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.369 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.369 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.369 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.369 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.369 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.369 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.369 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.370 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.370 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.370 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.370 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.370 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.370 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.370 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.371 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.371 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.371 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.371 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.371 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.371 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.371 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.371 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.372 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.372 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.372 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.372 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.372 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.372 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.372 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.373 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.373 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.373 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.373 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.373 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.373 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.373 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.374 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.374 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.374 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.374 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.374 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.374 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.374 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.374 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.375 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.375 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.375 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.375 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.375 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.375 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.375 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.375 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.376 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.377 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.377 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.377 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.377 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.377 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.377 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.377 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.377 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.378 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.378 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.378 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.378 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.378 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.378 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.378 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.378 INFO:teuthology.orchestra.run.smithi018.stdout: "motd": { 2024-04-16T17:27:15.379 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "motd", 2024-04-16T17:27:15.379 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.379 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.379 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.379 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.379 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.379 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.379 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.380 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "The message of the day", 2024-04-16T17:27:15.380 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.380 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.380 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.380 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.380 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect_resolve_ip_addr": { 2024-04-16T17:27:15.380 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T17:27:15.380 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.381 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.381 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.381 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.381 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.381 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.381 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.381 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.381 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.382 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.382 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.382 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.382 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:15.382 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:15.382 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.382 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.382 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.383 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-04-16T17:27:15.383 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.383 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.383 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.383 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.383 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.383 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.383 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.384 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.384 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:15.384 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:15.384 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.384 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.384 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.384 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8080", 2024-04-16T17:27:15.384 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.385 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.385 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.385 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.385 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.385 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.385 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.385 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.385 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-04-16T17:27:15.386 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-04-16T17:27:15.386 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.386 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.386 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.386 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.386 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.386 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.386 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.387 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.387 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.387 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.387 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.387 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.387 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl_server_port": { 2024-04-16T17:27:15.387 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl_server_port", 2024-04-16T17:27:15.387 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8443", 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.388 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.389 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.389 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.389 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-04-16T17:27:15.389 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-04-16T17:27:15.389 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.389 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.389 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.390 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "redirect", 2024-04-16T17:27:15.390 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.390 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.390 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.390 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.390 INFO:teuthology.orchestra.run.smithi018.stdout: "redirect" 2024-04-16T17:27:15.390 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.391 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.392 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.392 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-04-16T17:27:15.392 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-04-16T17:27:15.392 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-04-16T17:27:15.392 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.392 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.392 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.392 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.393 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.393 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.393 INFO:teuthology.orchestra.run.smithi018.stdout: "url_prefix": { 2024-04-16T17:27:15.393 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url_prefix", 2024-04-16T17:27:15.393 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.393 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.393 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.393 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.394 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.394 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.394 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.394 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.394 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.394 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.394 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.394 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.395 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.395 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.395 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.395 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "devicehealth", 2024-04-16T17:27:15.395 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.395 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.395 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.396 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_monitoring": { 2024-04-16T17:27:15.396 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_monitoring", 2024-04-16T17:27:15.396 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.396 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.396 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.396 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.396 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.396 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.397 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.397 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "monitor device health metrics", 2024-04-16T17:27:15.397 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.397 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.397 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.397 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.397 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.397 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.398 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.398 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.398 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.398 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.398 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.398 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.398 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.398 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.399 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.399 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.399 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.399 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.399 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.399 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.399 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.399 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.400 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.400 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.400 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.400 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.400 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.400 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.400 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.400 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.401 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.401 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.401 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.401 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.401 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.401 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.401 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.401 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.402 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.402 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.402 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.402 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.402 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.402 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.402 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.402 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.403 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.403 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.403 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.403 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.403 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.403 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.403 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.403 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.404 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.404 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.404 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.404 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.404 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.404 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.404 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.404 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.405 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.405 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.405 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.405 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.405 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.405 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.405 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.405 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: "mark_out_threshold": { 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mark_out_threshold", 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.406 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.407 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "2419200", 2024-04-16T17:27:15.407 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.407 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.407 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.407 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T17:27:15.407 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.407 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.407 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.408 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.408 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": { 2024-04-16T17:27:15.408 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pool_name", 2024-04-16T17:27:15.408 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.408 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.408 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.408 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "device_health_metrics", 2024-04-16T17:27:15.408 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.409 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.409 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.409 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T17:27:15.409 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.409 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.409 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.409 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.410 INFO:teuthology.orchestra.run.smithi018.stdout: "retention_period": { 2024-04-16T17:27:15.410 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "retention_period", 2024-04-16T17:27:15.410 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.410 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.410 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.410 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15552000", 2024-04-16T17:27:15.410 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.410 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.411 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.411 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long to retain device health metrics", 2024-04-16T17:27:15.411 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.411 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.411 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.411 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.411 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_frequency": { 2024-04-16T17:27:15.411 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_frequency", 2024-04-16T17:27:15.412 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.412 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.412 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.412 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-04-16T17:27:15.412 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.412 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.412 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.412 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: "self_heal": { 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "self_heal", 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.413 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.414 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.414 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.414 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.414 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.414 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T17:27:15.414 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.414 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.414 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.415 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.415 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:15.415 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:15.415 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.415 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.415 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.415 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:15.415 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.416 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.416 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.416 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T17:27:15.416 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.416 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.416 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.416 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.416 INFO:teuthology.orchestra.run.smithi018.stdout: "warn_threshold": { 2024-04-16T17:27:15.417 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "warn_threshold", 2024-04-16T17:27:15.417 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.417 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.417 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.417 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7257600", 2024-04-16T17:27:15.417 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.417 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.417 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.418 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T17:27:15.418 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.418 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.418 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.418 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.418 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.418 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "diskprediction_local", 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.419 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.420 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.421 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.422 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.423 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.424 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.425 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.426 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.426 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.426 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.426 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.426 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.426 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.426 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: "predict_interval": { 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predict_interval", 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.427 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "86400", 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "predictor_model": { 2024-04-16T17:27:15.428 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "predictor_model", 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "prophetstor", 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.429 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.430 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.430 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.430 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.430 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:15.430 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:15.430 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.430 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "600", 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.431 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.432 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.432 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.432 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.432 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "influx", 2024-04-16T17:27:15.432 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": false, 2024-04-16T17:27:15.432 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "influxdb python module not found", 2024-04-16T17:27:15.432 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.432 INFO:teuthology.orchestra.run.smithi018.stdout: "batch_size": { 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "batch_size", 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5000", 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.433 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: "database": { 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "database", 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.434 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "ceph", 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.435 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": { 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "hostname", 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.436 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server hostname", 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.437 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "30", 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "5", 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.438 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.439 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.439 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.439 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.439 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.439 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.439 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.439 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.439 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.440 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.441 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.441 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.441 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.441 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.441 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.441 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.441 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.441 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.442 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.443 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.443 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.443 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.443 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.443 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.443 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.443 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.443 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.444 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.445 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.446 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.446 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.446 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.446 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.446 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.446 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.446 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.446 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: "password": { 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "password", 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.447 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "password of InfluxDB server user", 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: "port": { 2024-04-16T17:27:15.448 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "port", 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "8086", 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "InfluxDB server port", 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.449 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: "ssl": { 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ssl", 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "false", 2024-04-16T17:27:15.450 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: "threads": { 2024-04-16T17:27:15.451 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threads", 2024-04-16T17:27:15.452 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.452 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.452 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.452 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:15.452 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:15.452 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "32", 2024-04-16T17:27:15.452 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.452 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "username": { 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "username", 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.453 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.454 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.454 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.454 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.454 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "username of InfluxDB server user", 2024-04-16T17:27:15.454 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.454 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.454 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.454 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "verify_ssl": { 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "verify_ssl", 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "true", 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.455 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "insights", 2024-04-16T17:27:15.456 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.457 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.458 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.459 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.460 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.461 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.463 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.464 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "iostat", 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.465 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.466 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.467 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.468 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.468 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.468 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.468 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.468 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.468 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.468 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.468 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.469 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.470 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.470 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.470 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.470 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.470 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.470 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.470 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.472 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.472 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.472 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.472 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.472 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.472 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.472 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.472 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.473 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.474 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "k8sevents", 2024-04-16T17:27:15.474 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.474 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.474 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.474 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_event_retention_days": { 2024-04-16T17:27:15.474 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "ceph_event_retention_days", 2024-04-16T17:27:15.474 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.474 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.475 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.475 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "7", 2024-04-16T17:27:15.475 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.475 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.475 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.475 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T17:27:15.475 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.475 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: "config_check_secs": { 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "config_check_secs", 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:15.476 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "10", 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.477 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.478 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.479 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.480 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.481 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.482 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.482 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.482 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.482 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.482 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.482 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.482 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.482 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.483 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.484 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "localpool", 2024-04-16T17:27:15.485 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.486 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.486 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.486 INFO:teuthology.orchestra.run.smithi018.stdout: "failure_domain": { 2024-04-16T17:27:15.486 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "failure_domain", 2024-04-16T17:27:15.486 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.486 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.486 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.486 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "host", 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "failure domain for any created local pool", 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.487 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.488 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.489 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.489 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.489 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.489 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.489 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.489 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.489 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.489 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.490 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.491 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.492 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.493 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.494 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": { 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "min_size", 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.495 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default min_size for any created local pool", 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.496 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: "num_rep": { 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "num_rep", 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3", 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.497 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default replica count for any created local pool", 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": { 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_num", 2024-04-16T17:27:15.498 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "128", 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.499 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: "prefix": { 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prefix", 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.500 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.501 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.501 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.501 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "name prefix for any created local pool", 2024-04-16T17:27:15.501 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.501 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.501 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.501 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.501 INFO:teuthology.orchestra.run.smithi018.stdout: "subtree": { 2024-04-16T17:27:15.502 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "subtree", 2024-04-16T17:27:15.502 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.502 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.502 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.502 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "rack", 2024-04-16T17:27:15.502 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.502 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.502 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.503 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mds_autoscaler", 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.504 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.505 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.505 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.505 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.505 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.505 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.505 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.505 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.505 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.506 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.507 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.507 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.507 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.507 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.507 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.507 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.507 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.507 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.508 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.509 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.509 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.509 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.509 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.509 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.509 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.509 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.509 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.510 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.510 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.510 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.510 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.510 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.510 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.510 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.510 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.511 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.511 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.511 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.511 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.511 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.511 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.511 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.511 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.512 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.512 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.512 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.512 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.512 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.512 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.513 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.513 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.513 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.513 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirroring", 2024-04-16T17:27:15.513 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.513 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.513 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.514 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.515 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.515 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.515 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.515 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.515 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.515 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.515 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.515 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.516 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.516 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.516 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.516 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.516 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.516 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.516 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.516 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.517 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.518 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.519 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.519 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.519 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.519 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.519 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.519 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.519 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.519 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.520 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "nfs", 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.522 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.523 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.524 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.524 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.524 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.524 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.524 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.524 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.524 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.525 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.525 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.525 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.525 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.525 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.525 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.525 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.525 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.526 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.527 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.528 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.529 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-04-16T17:27:15.530 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "fail_fs": { 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "fail_fs", 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.532 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.532 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.532 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T17:27:15.532 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.532 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.532 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.532 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.532 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.535 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.536 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.537 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.539 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.539 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.539 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.539 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.539 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.539 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.539 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.539 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator": { 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "orchestrator", 2024-04-16T17:27:15.540 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm", 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "rook", 2024-04-16T17:27:15.541 INFO:teuthology.orchestra.run.smithi018.stdout: "test_orchestrator" 2024-04-16T17:27:15.542 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.542 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Orchestrator backend", 2024-04-16T17:27:15.542 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.542 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.542 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.542 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.542 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.543 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.543 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_perf_query", 2024-04-16T17:27:15.543 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.543 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.543 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.543 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.543 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.544 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.545 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.546 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.547 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.549 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.550 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.550 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.550 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.550 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.550 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.550 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.550 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.551 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "osd_support", 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.552 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.553 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.554 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.555 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.556 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.557 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.558 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.559 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "pg_autoscaler", 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.560 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.561 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.562 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.563 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.565 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.566 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.567 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:15.568 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.569 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: "threshold": { 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "threshold", 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "3.0", 2024-04-16T17:27:15.570 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1.0", 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "scaling threshold", 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.571 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "progress", 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_pg_recovery_event": { 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_pg_recovery_event", 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.572 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.573 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.574 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.575 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.576 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.577 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.578 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.578 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.578 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.578 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.578 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.578 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.578 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.579 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.580 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.581 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.582 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: "max_completed_events": { 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_completed_events", 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.583 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "50", 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "number of past completed events to remember", 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.584 INFO:teuthology.orchestra.run.smithi018.stdout: "sleep_interval": { 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "sleep_interval", 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "5", 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.585 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "how long the module is going to sleep", 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "prometheus", 2024-04-16T17:27:15.586 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "cache": { 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "cache", 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.587 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: "exclude_perf_counters": { 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "exclude_perf_counters", 2024-04-16T17:27:15.588 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T17:27:15.589 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.590 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.590 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.590 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.590 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.590 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.590 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.590 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.590 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.591 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.592 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.593 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.594 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.595 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.596 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.597 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools": { 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools", 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.598 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.599 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "300", 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.600 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: "scrape_interval": { 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "scrape_interval", 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15.0", 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.601 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:15.602 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "::", 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.603 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "9283", 2024-04-16T17:27:15.604 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: "stale_cache_strategy": { 2024-04-16T17:27:15.605 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stale_cache_strategy", 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "log", 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.606 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.607 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.607 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.607 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.607 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_behaviour": { 2024-04-16T17:27:15.607 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_behaviour", 2024-04-16T17:27:15.607 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.607 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.607 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "default", 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: "default", 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: "error" 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.608 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.609 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.609 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.609 INFO:teuthology.orchestra.run.smithi018.stdout: "standby_error_status_code": { 2024-04-16T17:27:15.609 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "standby_error_status_code", 2024-04-16T17:27:15.609 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.610 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.610 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.610 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "500", 2024-04-16T17:27:15.610 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "400", 2024-04-16T17:27:15.610 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "599", 2024-04-16T17:27:15.610 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.610 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.610 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_support", 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.611 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.612 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.613 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.614 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.615 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.615 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.615 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.615 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.615 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.615 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.615 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.615 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.616 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.616 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.616 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.616 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.616 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.616 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.616 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.616 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.617 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.619 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.619 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.619 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.619 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.619 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.619 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.619 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.619 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_snap_create": { 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_snap_create", 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10", 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.620 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: "mirror_snapshot_schedule": { 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T17:27:15.621 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.622 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.622 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.622 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.622 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.622 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.622 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.622 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.622 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: "trash_purge_schedule": { 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "trash_purge_schedule", 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.623 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.624 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "restful", 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_auth": { 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enable_auth", 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.625 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.626 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: "key_file": { 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "key_file", 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.627 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.628 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.629 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.630 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.630 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.630 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.630 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.630 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.630 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.630 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.630 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.631 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.631 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.631 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.631 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.631 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.631 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.631 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.631 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.632 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.634 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.634 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.634 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.634 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.634 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.634 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.634 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.634 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.635 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "server_addr": { 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_addr", 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.636 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.637 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: "server_port": { 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "server_port", 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.638 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.639 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.639 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.639 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.639 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.639 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.639 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.639 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.640 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.640 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rgw", 2024-04-16T17:27:15.640 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.640 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.640 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.640 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.640 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.640 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.641 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.643 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.644 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.644 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.644 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.644 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.644 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.644 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.644 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.644 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.645 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.645 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.645 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.645 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.645 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.645 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.645 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.646 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.646 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.646 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.646 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.646 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.646 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.646 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.646 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.647 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.648 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rook", 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.651 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.652 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.653 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.653 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.653 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.653 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.653 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.653 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.653 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.653 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.654 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.654 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.654 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.654 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.654 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.654 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.654 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.654 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.655 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.656 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "storage_class": { 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "storage_class", 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.657 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "local", 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.658 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "selftest", 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.659 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.660 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.661 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.662 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.663 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.664 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.665 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.666 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: "roption1": { 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption1", 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.667 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: "roption2": { 2024-04-16T17:27:15.668 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "roption2", 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "xyz", 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.669 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.670 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.670 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.670 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.670 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption1": { 2024-04-16T17:27:15.670 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption1", 2024-04-16T17:27:15.670 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.670 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.670 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption2": { 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption2", 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.672 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption3": { 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption3", 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "float", 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.673 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.674 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption4": { 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption4", 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.675 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption5": { 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption5", 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.676 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.677 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption6": { 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption6", 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.678 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: "rwoption7": { 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rwoption7", 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.679 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.680 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.680 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.680 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "1", 2024-04-16T17:27:15.680 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "42", 2024-04-16T17:27:15.680 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.680 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.680 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.680 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.681 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.681 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.681 INFO:teuthology.orchestra.run.smithi018.stdout: "testkey": { 2024-04-16T17:27:15.681 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testkey", 2024-04-16T17:27:15.681 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.681 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.681 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.681 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.682 INFO:teuthology.orchestra.run.smithi018.stdout: "testlkey": { 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testlkey", 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.683 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: "testnewline": { 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "testnewline", 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.684 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.685 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: "allow_m_granularity": { 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "allow_m_granularity", 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.686 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.687 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "dump_on_update": { 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "dump_on_update", 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.688 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "dump database to debug log on update", 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.689 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.690 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.690 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.690 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.690 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.690 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.690 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.690 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.691 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.692 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.693 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.694 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.695 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.696 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.697 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.697 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.697 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.697 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.697 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.697 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "stats", 2024-04-16T17:27:15.697 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.697 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.698 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.698 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.698 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.698 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.698 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.698 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.698 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.699 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.699 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.699 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.699 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.699 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.699 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.700 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.700 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.700 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.700 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.700 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.700 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.700 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.701 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.701 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.701 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.701 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.701 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.701 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.702 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.702 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.702 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.702 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.702 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.702 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.704 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.704 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.704 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.704 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.704 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.704 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.704 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.705 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.705 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.705 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.705 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.705 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.705 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.705 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.706 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.706 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.706 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.706 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.706 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.707 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.707 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.707 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.707 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.707 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.707 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.707 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "status", 2024-04-16T17:27:15.708 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.709 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.710 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.710 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.710 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.710 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.710 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.710 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.710 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.711 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.712 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.713 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.714 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.715 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.716 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.716 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.716 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.716 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.716 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.716 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.716 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.716 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telegraf", 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.717 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.718 INFO:teuthology.orchestra.run.smithi018.stdout: "address": { 2024-04-16T17:27:15.718 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "address", 2024-04-16T17:27:15.718 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.718 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.718 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.718 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T17:27:15.718 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.718 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:15.719 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "15", 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.720 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.721 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.722 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.722 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.722 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.722 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.722 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.722 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.722 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.722 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.723 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.724 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.724 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.724 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.724 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.724 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.724 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.724 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.724 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.725 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.725 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.725 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.725 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.725 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.725 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.726 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.727 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.728 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "telemetry", 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.729 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_basic": { 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_basic", 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.730 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.731 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T17:27:15.731 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.731 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.731 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.731 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_crash": { 2024-04-16T17:27:15.731 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_crash", 2024-04-16T17:27:15.731 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.732 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_device": { 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_device", 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.733 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.734 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.734 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T17:27:15.734 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.734 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.734 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.734 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.734 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_ident": { 2024-04-16T17:27:15.734 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_ident", 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T17:27:15.735 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: "channel_perf": { 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "channel_perf", 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.736 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.737 INFO:teuthology.orchestra.run.smithi018.stdout: "contact": { 2024-04-16T17:27:15.738 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "contact", 2024-04-16T17:27:15.738 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.738 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.738 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.738 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.738 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.738 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.738 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.739 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.739 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.739 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.739 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.739 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.739 INFO:teuthology.orchestra.run.smithi018.stdout: "description": { 2024-04-16T17:27:15.739 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "description", 2024-04-16T17:27:15.740 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.740 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.740 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.740 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.740 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.740 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.740 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: "device_url": { 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "device_url", 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.741 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.742 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.742 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T17:27:15.742 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.742 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.742 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.742 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.742 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.743 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.743 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.743 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.743 INFO:teuthology.orchestra.run.smithi018.stdout: "enabled": { 2024-04-16T17:27:15.743 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "enabled", 2024-04-16T17:27:15.743 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.743 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.744 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.745 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:15.745 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:15.745 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.745 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.745 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.745 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "24", 2024-04-16T17:27:15.745 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "8", 2024-04-16T17:27:15.746 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.746 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.746 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.746 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.746 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.746 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: "last_opt_revision": { 2024-04-16T17:27:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "last_opt_revision", 2024-04-16T17:27:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "1", 2024-04-16T17:27:15.747 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.748 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard": { 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard", 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.749 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.750 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.750 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.750 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.750 INFO:teuthology.orchestra.run.smithi018.stdout: "leaderboard_description": { 2024-04-16T17:27:15.750 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "leaderboard_description", 2024-04-16T17:27:15.750 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.750 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.751 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.752 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.753 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.754 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.755 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.756 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.757 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.758 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.758 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.758 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.758 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.758 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.758 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.758 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.758 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.759 INFO:teuthology.orchestra.run.smithi018.stdout: "organization": { 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "organization", 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.760 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: "proxy": { 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "proxy", 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.761 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.762 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.762 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.762 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.762 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.762 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.762 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.762 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.762 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.763 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.763 INFO:teuthology.orchestra.run.smithi018.stdout: "url": { 2024-04-16T17:27:15.763 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "url", 2024-04-16T17:27:15.763 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.763 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.763 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.763 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T17:27:15.763 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.764 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.764 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.764 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.764 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.764 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.764 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.764 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.764 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "test_orchestrator", 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.765 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.766 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.767 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.767 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.767 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.767 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.767 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.767 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.767 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.767 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.768 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.769 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.769 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.769 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.769 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.769 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.769 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.769 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.770 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.771 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.772 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.772 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.772 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.772 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.772 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.772 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.772 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.772 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.773 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.773 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.773 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.773 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.773 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.773 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.773 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.773 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "volumes", 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.774 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.775 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.777 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.778 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.778 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.778 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.778 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.778 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.778 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.778 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.779 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.780 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.780 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.780 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.780 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.780 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.780 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.780 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.780 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.781 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.781 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.781 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.781 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.781 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.781 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.781 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.782 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.782 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.782 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.782 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.782 INFO:teuthology.orchestra.run.smithi018.stdout: "max_concurrent_clones": { 2024-04-16T17:27:15.782 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "max_concurrent_clones", 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "4", 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.783 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: "periodic_async_work": { 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "periodic_async_work", 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.784 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.785 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.785 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.785 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Periodically check for async work", 2024-04-16T17:27:15.785 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.785 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.785 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.785 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_delay": { 2024-04-16T17:27:15.786 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_delay", 2024-04-16T17:27:15.786 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.786 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.786 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.786 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "0", 2024-04-16T17:27:15.786 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.786 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.786 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: "snapshot_clone_no_wait": { 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.787 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "True", 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.788 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix", 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: "can_run": true, 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: "error_string": "", 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: "module_options": { 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: "discovery_interval": { 2024-04-16T17:27:15.789 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "discovery_interval", 2024-04-16T17:27:15.790 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "uint", 2024-04-16T17:27:15.790 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.790 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.790 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "100", 2024-04-16T17:27:15.790 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.790 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.790 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: "identifier": { 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "identifier", 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.791 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.792 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: "interval": { 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "interval", 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "secs", 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "60", 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.793 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: "log_level": { 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_level", 2024-04-16T17:27:15.794 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.795 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.796 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster": { 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster", 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.797 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.798 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.798 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.798 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.798 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.798 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_cluster_level": { 2024-04-16T17:27:15.798 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_cluster_level", 2024-04-16T17:27:15.798 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "info", 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [ 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "", 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "critical", 2024-04-16T17:27:15.799 INFO:teuthology.orchestra.run.smithi018.stdout: "debug", 2024-04-16T17:27:15.800 INFO:teuthology.orchestra.run.smithi018.stdout: "error", 2024-04-16T17:27:15.800 INFO:teuthology.orchestra.run.smithi018.stdout: "info", 2024-04-16T17:27:15.800 INFO:teuthology.orchestra.run.smithi018.stdout: "warning" 2024-04-16T17:27:15.800 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.800 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.800 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.800 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": { 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "log_to_file", 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "bool", 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "False", 2024-04-16T17:27:15.801 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_host": { 2024-04-16T17:27:15.802 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_host", 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "", 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.803 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_port": { 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_port", 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "int", 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "10051", 2024-04-16T17:27:15.804 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.805 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.805 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.805 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.805 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.805 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.805 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.805 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.805 INFO:teuthology.orchestra.run.smithi018.stdout: "zabbix_sender": { 2024-04-16T17:27:15.806 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "zabbix_sender", 2024-04-16T17:27:15.806 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "str", 2024-04-16T17:27:15.806 INFO:teuthology.orchestra.run.smithi018.stdout: "level": "advanced", 2024-04-16T17:27:15.806 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 0, 2024-04-16T17:27:15.806 INFO:teuthology.orchestra.run.smithi018.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T17:27:15.806 INFO:teuthology.orchestra.run.smithi018.stdout: "min": "", 2024-04-16T17:27:15.806 INFO:teuthology.orchestra.run.smithi018.stdout: "max": "", 2024-04-16T17:27:15.806 INFO:teuthology.orchestra.run.smithi018.stdout: "enum_allowed": [], 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: "desc": "", 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: "long_desc": "", 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: "tags": [], 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: "see_also": [] 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.807 INFO:teuthology.orchestra.run.smithi018.stdout: "services": { 2024-04-16T17:27:15.808 INFO:teuthology.orchestra.run.smithi018.stdout: "dashboard": "https://172.21.15.18:8443/" 2024-04-16T17:27:15.808 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.808 INFO:teuthology.orchestra.run.smithi018.stdout: "always_on_modules": { 2024-04-16T17:27:15.808 INFO:teuthology.orchestra.run.smithi018.stdout: "octopus": [ 2024-04-16T17:27:15.808 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-04-16T17:27:15.808 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-16T17:27:15.808 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-04-16T17:27:15.808 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: "pacific": [ 2024-04-16T17:27:15.809 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-04-16T17:27:15.810 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-16T17:27:15.810 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-04-16T17:27:15.810 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-04-16T17:27:15.810 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-04-16T17:27:15.810 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-04-16T17:27:15.810 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: "quincy": [ 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-04-16T17:27:15.811 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "reef": [ 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-04-16T17:27:15.812 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-04-16T17:27:15.813 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-16T17:27:15.814 INFO:teuthology.orchestra.run.smithi018.stdout: "squid": [ 2024-04-16T17:27:15.814 INFO:teuthology.orchestra.run.smithi018.stdout: "balancer", 2024-04-16T17:27:15.814 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-16T17:27:15.814 INFO:teuthology.orchestra.run.smithi018.stdout: "devicehealth", 2024-04-16T17:27:15.814 INFO:teuthology.orchestra.run.smithi018.stdout: "orchestrator", 2024-04-16T17:27:15.814 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscaler", 2024-04-16T17:27:15.814 INFO:teuthology.orchestra.run.smithi018.stdout: "progress", 2024-04-16T17:27:15.814 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_support", 2024-04-16T17:27:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: "status", 2024-04-16T17:27:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: "telemetry", 2024-04-16T17:27:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: "volumes" 2024-04-16T17:27:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: "last_failure_osd_epoch": 94, 2024-04-16T17:27:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: "active_clients": [ 2024-04-16T17:27:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "devicehealth", 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 547628127 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.816 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "libcephsqlite", 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2089186680 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.817 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "rbd_support", 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3219415644 2024-04-16T17:27:15.818 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 707489962 2024-04-16T17:27:15.819 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "snap_schedule", 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-04-16T17:27:15.820 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 69706356 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "volumes", 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-04-16T17:27:15.821 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:0", 2024-04-16T17:27:15.822 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 750294189 2024-04-16T17:27:15.822 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.822 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.822 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:15.822 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-16T17:27:15.822 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:27:15.829 DEBUG:tasks.check_counter:Getting stats from a 2024-04-16T17:27:15.829 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-16T17:27:15.830 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:14 smithi018 ceph-mon[31776]: pgmap v720: 129 pgs: 16 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 62 active+clean; 455 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 233 KiB/s wr, 52 op/s 2024-04-16T17:27:15.830 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:14 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2054630423' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T17:27:16.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:16 smithi022 ceph-mon[36023]: pgmap v721: 129 pgs: 14 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 72 active+clean; 455 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 215 KiB/s wr, 48 op/s 2024-04-16T17:27:16.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:16 smithi018 ceph-mon[31776]: pgmap v721: 129 pgs: 14 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 72 active+clean; 455 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 215 KiB/s wr, 48 op/s 2024-04-16T17:27:16.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:16 smithi043 ceph-mon[36901]: pgmap v721: 129 pgs: 14 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 72 active+clean; 455 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 215 KiB/s wr, 48 op/s 2024-04-16T17:27:16.604 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:27:18.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:18 smithi022 ceph-mon[36023]: pgmap v722: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 210 KiB/s wr, 47 op/s 2024-04-16T17:27:18.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:18 smithi018 ceph-mon[31776]: pgmap v722: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 210 KiB/s wr, 47 op/s 2024-04-16T17:27:18.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:18 smithi043 ceph-mon[36901]: pgmap v722: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 210 KiB/s wr, 47 op/s 2024-04-16T17:27:20.070 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:19 smithi018 ceph-mon[31776]: osdmap e138: 12 total, 12 up, 12 in 2024-04-16T17:27:20.071 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:19 smithi018 ceph-mon[31776]: pgmap v724: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 11 op/s 2024-04-16T17:27:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:19 smithi022 ceph-mon[36023]: osdmap e138: 12 total, 12 up, 12 in 2024-04-16T17:27:20.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:19 smithi022 ceph-mon[36023]: pgmap v724: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 11 op/s 2024-04-16T17:27:20.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:19 smithi043 ceph-mon[36901]: osdmap e138: 12 total, 12 up, 12 in 2024-04-16T17:27:20.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:19 smithi043 ceph-mon[36901]: pgmap v724: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 11 op/s 2024-04-16T17:27:21.430 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:27:21.430 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:21.430 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 0, 2024-04-16T17:27:21.430 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 2, 2024-04-16T17:27:21.430 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:27:21.431 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 0, 2024-04-16T17:27:21.431 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 4, 2024-04-16T17:27:21.431 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 0, 2024-04-16T17:27:21.431 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.004304551, 2024-04-16T17:27:21.431 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:27:21.432 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.000746037, 2024-04-16T17:27:21.432 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:21.432 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:21.432 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:21.432 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:21.432 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:21.432 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.432 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 426, 2024-04-16T17:27:21.433 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 415, 2024-04-16T17:27:21.434 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 1648320, 2024-04-16T17:27:21.434 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 257056, 2024-04-16T17:27:21.434 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 2, 2024-04-16T17:27:21.434 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 1, 2024-04-16T17:27:21.434 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.137023600, 2024-04-16T17:27:21.434 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.058282721, 2024-04-16T17:27:21.434 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.077137224, 2024-04-16T17:27:21.434 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.016188922, 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 414, 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.078158594, 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000188788 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:21.435 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 1648320, 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 257056 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 6, 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 319, 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 3392, 2024-04-16T17:27:21.436 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 212608, 2024-04-16T17:27:21.437 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 5, 2024-04-16T17:27:21.437 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 3, 2024-04-16T17:27:21.437 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.051104524, 2024-04-16T17:27:21.437 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.038934343, 2024-04-16T17:27:21.437 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.005074648, 2024-04-16T17:27:21.437 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:21.437 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:21.437 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 317, 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.072625023, 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000229101 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.438 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 212608 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 0, 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:21.439 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:21.440 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:21.440 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:21.440 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:21.440 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:21.440 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:21.440 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:21.440 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:21.440 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:21.441 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:21.442 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:21.442 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:21.442 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:21.442 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:21.442 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:21.442 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:21.442 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:21.442 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:21.443 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 149745, 2024-04-16T17:27:21.444 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 31, 2024-04-16T17:27:21.444 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:21.444 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:21.444 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-04-16T17:27:21.444 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-04-16T17:27:21.444 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:21.444 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:21.444 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 0, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 0, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 928, 2024-04-16T17:27:21.445 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 2, 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-04-16T17:27:21.446 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 428, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 1588108, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 428, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 428, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 1588108, 2024-04-16T17:27:21.447 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-04-16T17:27:21.448 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:21.448 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:21.448 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:21.448 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:21.448 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:21.448 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:27:22.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:21 smithi043 ceph-mon[36901]: pgmap v725: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-16T17:27:22.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:21 smithi022 ceph-mon[36023]: pgmap v725: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-16T17:27:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:21 smithi018 ceph-mon[31776]: pgmap v725: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-16T17:27:22.693 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-16T17:27:22.694 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-16T17:27:22.694 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-16T17:27:22.694 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-16T17:27:22.694 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-16T17:27:22.694 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-16T17:27:22.694 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-16T17:27:22.694 DEBUG:tasks.check_counter:Getting stats from d 2024-04-16T17:27:22.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-16T17:27:23.157 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:27:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:23 smithi018 ceph-mon[31776]: pgmap v726: 129 pgs: 129 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 10 op/s 2024-04-16T17:27:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:23 smithi018 ceph-mon[31776]: osdmap e139: 12 total, 12 up, 12 in 2024-04-16T17:27:24.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:23 smithi043 ceph-mon[36901]: pgmap v726: 129 pgs: 129 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 10 op/s 2024-04-16T17:27:24.251 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:23 smithi043 ceph-mon[36901]: osdmap e139: 12 total, 12 up, 12 in 2024-04-16T17:27:24.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:23 smithi022 ceph-mon[36023]: pgmap v726: 129 pgs: 129 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 10 op/s 2024-04-16T17:27:24.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:23 smithi022 ceph-mon[36023]: osdmap e139: 12 total, 12 up, 12 in 2024-04-16T17:27:26.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:26 smithi018 ceph-mon[31776]: pgmap v728: 129 pgs: 129 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 82 KiB/s wr, 6 op/s 2024-04-16T17:27:26.338 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:27:26.339 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:26.339 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 4, 2024-04-16T17:27:26.339 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 4, 2024-04-16T17:27:26.339 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 3072, 2024-04-16T17:27:26.339 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 448, 2024-04-16T17:27:26.339 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 4, 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 0, 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.004497969, 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.000077211, 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.001446769, 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 3, 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.001604858, 2024-04-16T17:27:26.340 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000534952 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T17:27:26.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.342 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:26.342 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 2, 2024-04-16T17:27:26.342 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 323, 2024-04-16T17:27:26.342 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 320, 2024-04-16T17:27:26.342 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 213216, 2024-04-16T17:27:26.342 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 5, 2024-04-16T17:27:26.342 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 3, 2024-04-16T17:27:26.342 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.052649796, 2024-04-16T17:27:26.343 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.042319474, 2024-04-16T17:27:26.343 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.003327017, 2024-04-16T17:27:26.343 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:26.343 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:26.343 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 320, 2024-04-16T17:27:26.343 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.072310056, 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000225968 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T17:27:26.344 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 213216 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 433, 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 422, 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 1650224, 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 261536, 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 2, 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 1, 2024-04-16T17:27:26.345 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.170100917, 2024-04-16T17:27:26.346 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.071320674, 2024-04-16T17:27:26.346 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.093753074, 2024-04-16T17:27:26.346 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.022966262, 2024-04-16T17:27:26.346 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:26.346 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 421, 2024-04-16T17:27:26.346 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.075016816, 2024-04-16T17:27:26.346 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000178187 2024-04-16T17:27:26.346 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.347 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:26.347 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:26.347 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:26.347 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:26.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.347 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 1650224, 2024-04-16T17:27:26.347 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 261536 2024-04-16T17:27:26.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 0, 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:26.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:26.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:26.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:26.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:26.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:26.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:26.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:26.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:26.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:26.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:26.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:26.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:26.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:26.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:26.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:26.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:26.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:26.351 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 148731, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 31, 2024-04-16T17:27:26.352 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:26.353 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:26.353 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-04-16T17:27:26.353 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-04-16T17:27:26.353 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:26.353 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:26.353 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:26.353 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:26.353 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 0, 2024-04-16T17:27:26.354 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 0, 2024-04-16T17:27:26.354 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:26.354 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:26.354 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:26.354 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-04-16T17:27:26.354 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 928, 2024-04-16T17:27:26.354 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 2, 2024-04-16T17:27:26.354 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-04-16T17:27:26.355 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 435, 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 1589018, 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 435, 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 435, 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 1589018, 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-04-16T17:27:26.356 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:26.357 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:26.357 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:26.357 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:26.357 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:26.357 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:27:26.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:26 smithi022 ceph-mon[36023]: pgmap v728: 129 pgs: 129 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 82 KiB/s wr, 6 op/s 2024-04-16T17:27:26.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:26 smithi043 ceph-mon[36901]: pgmap v728: 129 pgs: 129 active+clean; 447 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 82 KiB/s wr, 6 op/s 2024-04-16T17:27:27.568 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-16T17:27:27.568 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-16T17:27:27.568 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-16T17:27:27.568 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-16T17:27:27.568 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-16T17:27:27.568 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-16T17:27:27.568 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-16T17:27:27.569 DEBUG:tasks.check_counter:Getting stats from g 2024-04-16T17:27:27.569 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-16T17:27:28.052 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:27:28.314 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:28 smithi018 ceph-mon[31776]: pgmap v729: 129 pgs: 129 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 700 KiB/s wr, 17 op/s 2024-04-16T17:27:28.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:28 smithi022 ceph-mon[36023]: pgmap v729: 129 pgs: 129 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 700 KiB/s wr, 17 op/s 2024-04-16T17:27:28.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:28 smithi043 ceph-mon[36901]: pgmap v729: 129 pgs: 129 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 700 KiB/s wr, 17 op/s 2024-04-16T17:27:30.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:29 smithi022 ceph-mon[36023]: pgmap v730: 129 pgs: 129 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 566 KiB/s wr, 13 op/s 2024-04-16T17:27:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:29 smithi018 ceph-mon[31776]: pgmap v730: 129 pgs: 129 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 566 KiB/s wr, 13 op/s 2024-04-16T17:27:30.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:29 smithi043 ceph-mon[36901]: pgmap v730: 129 pgs: 129 active+clean; 448 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 566 KiB/s wr, 13 op/s 2024-04-16T17:27:31.287 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:27:31.288 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:31.288 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 16187, 2024-04-16T17:27:31.288 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 25204, 2024-04-16T17:27:31.288 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 3362454, 2024-04-16T17:27:31.289 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 54140223, 2024-04-16T17:27:31.289 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 62, 2024-04-16T17:27:31.289 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 6, 2024-04-16T17:27:31.289 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 3.872043381, 2024-04-16T17:27:31.289 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 1.404612183, 2024-04-16T17:27:31.289 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 1.316611337, 2024-04-16T17:27:31.289 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.719922983, 2024-04-16T17:27:31.290 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:31.290 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 25197, 2024-04-16T17:27:31.290 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 12.322554049, 2024-04-16T17:27:31.290 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000489048 2024-04-16T17:27:31.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.290 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:31.290 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 26160, 2024-04-16T17:27:31.291 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.007855752, 2024-04-16T17:27:31.291 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000300 2024-04-16T17:27:31.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.291 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 5696, 2024-04-16T17:27:31.291 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 28112 2024-04-16T17:27:31.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.291 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:31.292 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 386254, 2024-04-16T17:27:31.292 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 260970, 2024-04-16T17:27:31.292 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 155915989, 2024-04-16T17:27:31.292 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 126412861, 2024-04-16T17:27:31.292 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 48, 2024-04-16T17:27:31.292 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 6, 2024-04-16T17:27:31.292 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 74.765532496, 2024-04-16T17:27:31.293 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 47.771688861, 2024-04-16T17:27:31.293 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 37.296367151, 2024-04-16T17:27:31.293 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.447104776, 2024-04-16T17:27:31.293 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:31.293 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 260972, 2024-04-16T17:27:31.294 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 80.327333158, 2024-04-16T17:27:31.294 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000307800 2024-04-16T17:27:31.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.294 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:31.294 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 604845, 2024-04-16T17:27:31.295 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.302007404, 2024-04-16T17:27:31.295 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000499 2024-04-16T17:27:31.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.295 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-16T17:27:31.295 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 145216 2024-04-16T17:27:31.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.295 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:31.296 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 7723, 2024-04-16T17:27:31.296 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 7968, 2024-04-16T17:27:31.296 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 4059885, 2024-04-16T17:27:31.296 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 30359069, 2024-04-16T17:27:31.296 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 7, 2024-04-16T17:27:31.296 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 6, 2024-04-16T17:27:31.297 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 1.934886017, 2024-04-16T17:27:31.297 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.648577369, 2024-04-16T17:27:31.297 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.353344318, 2024-04-16T17:27:31.297 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.741753425, 2024-04-16T17:27:31.297 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:31.297 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 7968, 2024-04-16T17:27:31.297 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 3.237266565, 2024-04-16T17:27:31.298 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000406283 2024-04-16T17:27:31.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.298 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:31.298 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.298 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.298 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.299 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 1774416, 2024-04-16T17:27:31.299 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 867296 2024-04-16T17:27:31.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.299 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-04-16T17:27:31.299 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 1, 2024-04-16T17:27:31.299 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-04-16T17:27:31.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.300 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-MDSRank": { 2024-04-16T17:27:31.300 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-04-16T17:27:31.300 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-04-16T17:27:31.300 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 18036, 2024-04-16T17:27:31.300 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 31.539386894, 2024-04-16T17:27:31.300 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001748690 2024-04-16T17:27:31.300 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.301 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-PurgeQueue": { 2024-04-16T17:27:31.301 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-04-16T17:27:31.301 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-04-16T17:27:31.301 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 7849, 2024-04-16T17:27:31.301 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 2.093304116, 2024-04-16T17:27:31.301 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000266696 2024-04-16T17:27:31.301 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.302 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-16T17:27:31.302 INFO:teuthology.orchestra.run.smithi018.stdout: "request": 56390, 2024-04-16T17:27:31.302 INFO:teuthology.orchestra.run.smithi018.stdout: "reply": 32460, 2024-04-16T17:27:31.302 INFO:teuthology.orchestra.run.smithi018.stdout: "reply_latency": { 2024-04-16T17:27:31.302 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 32460, 2024-04-16T17:27:31.302 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 209.723020194, 2024-04-16T17:27:31.303 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.006460967 2024-04-16T17:27:31.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.303 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_reply": 0, 2024-04-16T17:27:31.303 INFO:teuthology.orchestra.run.smithi018.stdout: "forward": 23930, 2024-04-16T17:27:31.303 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_complete": 0, 2024-04-16T17:27:31.303 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_keys": 0, 2024-04-16T17:27:31.303 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_commit": 20, 2024-04-16T17:27:31.304 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_split": 1, 2024-04-16T17:27:31.304 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_merge": 0, 2024-04-16T17:27:31.304 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes": 643, 2024-04-16T17:27:31.304 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_top": 119, 2024-04-16T17:27:31.304 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_bottom": 524, 2024-04-16T17:27:31.305 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pin_tail": 0, 2024-04-16T17:27:31.305 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pinned": 458, 2024-04-16T17:27:31.305 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_expired": 0, 2024-04-16T17:27:31.305 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_with_caps": 46, 2024-04-16T17:27:31.305 INFO:teuthology.orchestra.run.smithi018.stdout: "caps": 46, 2024-04-16T17:27:31.305 INFO:teuthology.orchestra.run.smithi018.stdout: "subtrees": 6, 2024-04-16T17:27:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse": 73802, 2024-04-16T17:27:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_hit": 32248, 2024-04-16T17:27:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_forward": 23927, 2024-04-16T17:27:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_discover": 104, 2024-04-16T17:27:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:27:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_remote_ino": 0, 2024-04-16T17:27:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_lock": 191, 2024-04-16T17:27:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "load_cent": 0, 2024-04-16T17:27:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "q": 0, 2024-04-16T17:27:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "exported": 0, 2024-04-16T17:27:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "exported_inodes": 0, 2024-04-16T17:27:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "imported": 1, 2024-04-16T17:27:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "imported_inodes": 64, 2024-04-16T17:27:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_dir_fetch": 0, 2024-04-16T17:27:31.308 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_backtrace_fetch": 795, 2024-04-16T17:27:31.308 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_peer_discover": 0, 2024-04-16T17:27:31.308 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rfiles": 762, 2024-04-16T17:27:31.308 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rbytes": 226530171, 2024-04-16T17:27:31.308 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rsnaps": 6, 2024-04-16T17:27:31.308 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_fetch": 795, 2024-04-16T17:27:31.308 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_set_tag": 0, 2024-04-16T17:27:31.309 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:27:31.309 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:27:31.309 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_inodes": 135, 2024-04-16T17:27:31.309 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:27:31.309 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dirfrag_rstats": 135, 2024-04-16T17:27:31.309 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_file_inodes": 660, 2024-04-16T17:27:31.309 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_inode_file_caps": 5, 2024-04-16T17:27:31.310 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_revoke": 2171, 2024-04-16T17:27:31.310 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_grant": 13977, 2024-04-16T17:27:31.310 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:27:31.310 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:27:31.311 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:27:31.311 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps": 9828, 2024-04-16T17:27:31.311 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps_dirty": 3888, 2024-04-16T17:27:31.311 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_cap_release": 1397, 2024-04-16T17:27:31.311 INFO:teuthology.orchestra.run.smithi018.stdout: "process_request_cap_release": 23375 2024-04-16T17:27:31.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.311 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache": { 2024-04-16T17:27:31.311 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update": 5, 2024-04-16T17:27:31.312 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update_receipt": 399, 2024-04-16T17:27:31.312 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_try_discover": 250, 2024-04-16T17:27:31.312 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_send_discover": 304, 2024-04-16T17:27:31.312 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_handle_discover": 156, 2024-04-16T17:27:31.312 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays": 0, 2024-04-16T17:27:31.313 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_delayed": 0, 2024-04-16T17:27:31.313 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:27:31.313 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_created": 4218, 2024-04-16T17:27:31.313 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_enqueued": 4218, 2024-04-16T17:27:31.313 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_reintegrated": 0, 2024-04-16T17:27:31.313 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_migrated": 0, 2024-04-16T17:27:31.314 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_processing": 0, 2024-04-16T17:27:31.314 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:27:31.314 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:27:31.314 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_started": 0, 2024-04-16T17:27:31.315 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_completed": 0, 2024-04-16T17:27:31.315 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:27:31.316 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:27:31.316 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:27:31.316 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_exportdir": 0, 2024-04-16T17:27:31.316 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_flush": 0, 2024-04-16T17:27:31.316 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragmentdir": 1, 2024-04-16T17:27:31.317 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragstats": 0, 2024-04-16T17:27:31.317 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_inodestats": 0 2024-04-16T17:27:31.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.317 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log": { 2024-04-16T17:27:31.317 INFO:teuthology.orchestra.run.smithi018.stdout: "evlrg": 0, 2024-04-16T17:27:31.317 INFO:teuthology.orchestra.run.smithi018.stdout: "evadd": 22460, 2024-04-16T17:27:31.317 INFO:teuthology.orchestra.run.smithi018.stdout: "evex": 0, 2024-04-16T17:27:31.318 INFO:teuthology.orchestra.run.smithi018.stdout: "evtrm": 0, 2024-04-16T17:27:31.318 INFO:teuthology.orchestra.run.smithi018.stdout: "ev": 22460, 2024-04-16T17:27:31.318 INFO:teuthology.orchestra.run.smithi018.stdout: "evexg": 0, 2024-04-16T17:27:31.318 INFO:teuthology.orchestra.run.smithi018.stdout: "evexd": 0, 2024-04-16T17:27:31.318 INFO:teuthology.orchestra.run.smithi018.stdout: "segadd": 35, 2024-04-16T17:27:31.318 INFO:teuthology.orchestra.run.smithi018.stdout: "segex": 0, 2024-04-16T17:27:31.319 INFO:teuthology.orchestra.run.smithi018.stdout: "segtrm": 0, 2024-04-16T17:27:31.319 INFO:teuthology.orchestra.run.smithi018.stdout: "seg": 35, 2024-04-16T17:27:31.319 INFO:teuthology.orchestra.run.smithi018.stdout: "segmjr": 3, 2024-04-16T17:27:31.319 INFO:teuthology.orchestra.run.smithi018.stdout: "segexg": 0, 2024-04-16T17:27:31.319 INFO:teuthology.orchestra.run.smithi018.stdout: "segexd": 0, 2024-04-16T17:27:31.319 INFO:teuthology.orchestra.run.smithi018.stdout: "expos": 4194304, 2024-04-16T17:27:31.320 INFO:teuthology.orchestra.run.smithi018.stdout: "wrpos": 114590097, 2024-04-16T17:27:31.320 INFO:teuthology.orchestra.run.smithi018.stdout: "rdpos": 0, 2024-04-16T17:27:31.320 INFO:teuthology.orchestra.run.smithi018.stdout: "jlat": { 2024-04-16T17:27:31.320 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 8738, 2024-04-16T17:27:31.320 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 174.489683137, 2024-04-16T17:27:31.320 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.019969064 2024-04-16T17:27:31.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.321 INFO:teuthology.orchestra.run.smithi018.stdout: "replayed": 0 2024-04-16T17:27:31.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.321 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_mem": { 2024-04-16T17:27:31.321 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 574, 2024-04-16T17:27:31.322 INFO:teuthology.orchestra.run.smithi018.stdout: "ino+": 4792, 2024-04-16T17:27:31.322 INFO:teuthology.orchestra.run.smithi018.stdout: "ino-": 4218, 2024-04-16T17:27:31.322 INFO:teuthology.orchestra.run.smithi018.stdout: "dir": 260, 2024-04-16T17:27:31.322 INFO:teuthology.orchestra.run.smithi018.stdout: "dir+": 335, 2024-04-16T17:27:31.322 INFO:teuthology.orchestra.run.smithi018.stdout: "dir-": 75, 2024-04-16T17:27:31.323 INFO:teuthology.orchestra.run.smithi018.stdout: "dn": 644, 2024-04-16T17:27:31.323 INFO:teuthology.orchestra.run.smithi018.stdout: "dn+": 8283, 2024-04-16T17:27:31.323 INFO:teuthology.orchestra.run.smithi018.stdout: "dn-": 7639, 2024-04-16T17:27:31.323 INFO:teuthology.orchestra.run.smithi018.stdout: "cap": 46, 2024-04-16T17:27:31.323 INFO:teuthology.orchestra.run.smithi018.stdout: "cap+": 4924, 2024-04-16T17:27:31.323 INFO:teuthology.orchestra.run.smithi018.stdout: "cap-": 4878, 2024-04-16T17:27:31.323 INFO:teuthology.orchestra.run.smithi018.stdout: "rss": 67104, 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: "heap": 223608 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_server": { 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_client_request": 68200, 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_server_request": 2038, 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_request": 56390, 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_session": 696, 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_peer_request": 4077, 2024-04-16T17:27:31.324 INFO:teuthology.orchestra.run.smithi018.stdout: "req_create_latency": { 2024-04-16T17:27:31.325 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4231, 2024-04-16T17:27:31.325 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 37.362932219, 2024-04-16T17:27:31.325 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.008830756 2024-04-16T17:27:31.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.325 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getattr_latency": { 2024-04-16T17:27:31.325 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 3194, 2024-04-16T17:27:31.325 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 78.437908723, 2024-04-16T17:27:31.325 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.024557892 2024-04-16T17:27:31.326 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.326 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getfilelock_latency": { 2024-04-16T17:27:31.326 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.326 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.326 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.326 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.326 INFO:teuthology.orchestra.run.smithi018.stdout: "req_link_latency": { 2024-04-16T17:27:31.326 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookup_latency": { 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 7462, 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 12.117310977, 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001623869 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.327 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookuphash_latency": { 2024-04-16T17:27:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupino_latency": { 2024-04-16T17:27:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupname_latency": { 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupparent_latency": { 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.330 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.330 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.330 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupsnap_latency": { 2024-04-16T17:27:31.330 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.330 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.330 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.331 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lssnap_latency": { 2024-04-16T17:27:31.331 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.331 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.331 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.332 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.332 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mkdir_latency": { 2024-04-16T17:27:31.332 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.332 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.332 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.332 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.332 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mknod_latency": { 2024-04-16T17:27:31.332 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.333 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.333 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.333 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.333 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mksnap_latency": { 2024-04-16T17:27:31.333 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.333 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.333 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.334 INFO:teuthology.orchestra.run.smithi018.stdout: "req_open_latency": { 2024-04-16T17:27:31.334 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.334 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.334 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.334 INFO:teuthology.orchestra.run.smithi018.stdout: "req_readdir_latency": { 2024-04-16T17:27:31.335 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 12520, 2024-04-16T17:27:31.335 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 54.485208512, 2024-04-16T17:27:31.335 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.004351853 2024-04-16T17:27:31.335 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.335 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rename_latency": { 2024-04-16T17:27:31.335 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 651, 2024-04-16T17:27:31.335 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 3.825229237, 2024-04-16T17:27:31.335 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.005875928 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: "req_renamesnap_latency": { 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: "req_snapdiff_latency": { 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.336 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.337 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.337 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmdir_latency": { 2024-04-16T17:27:31.337 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.337 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.337 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.337 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmsnap_latency": { 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmxattr_latency": { 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setattr_latency": { 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 20, 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.816838736, 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.040841936 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setdirlayout_latency": { 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.339 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.340 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.340 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setfilelock_latency": { 2024-04-16T17:27:31.340 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 164, 2024-04-16T17:27:31.340 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.230210918, 2024-04-16T17:27:31.340 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001403725 2024-04-16T17:27:31.340 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.340 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setlayout_latency": { 2024-04-16T17:27:31.340 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setxattr_latency": { 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.341 INFO:teuthology.orchestra.run.smithi018.stdout: "req_symlink_latency": { 2024-04-16T17:27:31.342 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.342 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.342 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.342 INFO:teuthology.orchestra.run.smithi018.stdout: "req_unlink_latency": { 2024-04-16T17:27:31.342 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4218, 2024-04-16T17:27:31.342 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 22.447380872, 2024-04-16T17:27:31.342 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.005321806 2024-04-16T17:27:31.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.343 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:27:31.343 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:27:31.343 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getvxattr_latency": { 2024-04-16T17:27:31.343 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.343 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.343 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.343 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.344 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_sessions": { 2024-04-16T17:27:31.344 INFO:teuthology.orchestra.run.smithi018.stdout: "session_count": 2, 2024-04-16T17:27:31.344 INFO:teuthology.orchestra.run.smithi018.stdout: "session_add": 3, 2024-04-16T17:27:31.344 INFO:teuthology.orchestra.run.smithi018.stdout: "session_remove": 1, 2024-04-16T17:27:31.344 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_open": 2, 2024-04-16T17:27:31.344 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_stale": 0, 2024-04-16T17:27:31.344 INFO:teuthology.orchestra.run.smithi018.stdout: "total_load": 5495, 2024-04-16T17:27:31.345 INFO:teuthology.orchestra.run.smithi018.stdout: "average_load": 2747, 2024-04-16T17:27:31.345 INFO:teuthology.orchestra.run.smithi018.stdout: "avg_session_uptime": 1355, 2024-04-16T17:27:31.345 INFO:teuthology.orchestra.run.smithi018.stdout: "mdthresh_evicted": 0 2024-04-16T17:27:31.345 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.345 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-04-16T17:27:31.345 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:31.346 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:31.346 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:31.346 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:31.346 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:31.346 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:31.346 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:31.346 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:31.346 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:31.347 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:31.347 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:31.347 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:31.347 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:31.347 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:31.347 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:31.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:31.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:31.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:31.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:31.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:31.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:31.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:31.348 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:31.349 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:31.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:31.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:31.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-04-16T17:27:31.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:31.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:31.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:31.350 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:31.350 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 303153, 2024-04-16T17:27:31.354 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 389, 2024-04-16T17:27:31.354 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 528, 2024-04-16T17:27:31.354 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 6, 2024-04-16T17:27:31.354 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-04-16T17:27:31.354 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-04-16T17:27:31.354 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:31.354 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:31.355 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:31.355 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:31.355 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 9688, 2024-04-16T17:27:31.355 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 185, 2024-04-16T17:27:31.355 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:31.355 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:31.355 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:31.356 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-04-16T17:27:31.356 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 2301242, 2024-04-16T17:27:31.356 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 16642, 2024-04-16T17:27:31.356 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:31.356 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-04-16T17:27:31.356 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:31.356 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-04-16T17:27:31.356 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.357 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter": { 2024-04-16T17:27:31.357 INFO:teuthology.orchestra.run.smithi018.stdout: "op_active": 0, 2024-04-16T17:27:31.357 INFO:teuthology.orchestra.run.smithi018.stdout: "op_laggy": 0, 2024-04-16T17:27:31.357 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send": 19617, 2024-04-16T17:27:31.357 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send_bytes": 111518209, 2024-04-16T17:27:31.357 INFO:teuthology.orchestra.run.smithi018.stdout: "op_resend": 33, 2024-04-16T17:27:31.357 INFO:teuthology.orchestra.run.smithi018.stdout: "op_reply": 19617, 2024-04-16T17:27:31.357 INFO:teuthology.orchestra.run.smithi018.stdout: "op_latency": { 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 19617, 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 570.335644720, 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.029073540 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: "op_inflight": 0, 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: "oplen_avg": { 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 19617, 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 20415 2024-04-16T17:27:31.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "op": 19617, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "op_r": 1205, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "op_w": 18412, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "op_rmw": 0, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "op_pg": 0, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_stat": 0, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_create": 10, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_read": 410, 2024-04-16T17:27:31.359 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_write": 9148, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writefull": 249, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writesame": 0, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_append": 0, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_zero": 0, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_truncate": 1, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_delete": 8474, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_mapext": 0, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_sparse_read": 0, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_clonerange": 0, 2024-04-16T17:27:31.360 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_getxattr": 795, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_setxattr": 20, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_rmxattr": 0, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_call": 0, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_watch": 0, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_notify": 0, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:27:31.361 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls": 0, 2024-04-16T17:27:31.362 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:27:31.362 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_other": 0, 2024-04-16T17:27:31.362 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_active": 0, 2024-04-16T17:27:31.362 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_send": 0, 2024-04-16T17:27:31.362 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_resend": 0, 2024-04-16T17:27:31.362 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_ping": 0, 2024-04-16T17:27:31.362 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_active": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_send": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_resend": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_active": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_send": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_resend": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_active": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_send": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_resend": 0, 2024-04-16T17:27:31.363 INFO:teuthology.orchestra.run.smithi018.stdout: "command_active": 0, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "command_send": 0, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "command_resend": 0, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "map_epoch": 139, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "map_full": 0, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "map_inc": 50, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_sessions": 12, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_open": 14, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_close": 2, 2024-04-16T17:27:31.364 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_laggy": 0, 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_wr": 1007, 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_rd": 0, 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_del": 301 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: "oft": { 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_objs": 1, 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_updates": 0, 2024-04-16T17:27:31.365 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_removes": 0 2024-04-16T17:27:31.366 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.366 INFO:teuthology.orchestra.run.smithi018.stdout: "osd-slow-ops": { 2024-04-16T17:27:31.366 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_ops_count": 1 2024-04-16T17:27:31.366 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.366 INFO:teuthology.orchestra.run.smithi018.stdout: "purge_queue": { 2024-04-16T17:27:31.366 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops": 0, 2024-04-16T17:27:31.366 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops_high_water": 16, 2024-04-16T17:27:31.367 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing": 0, 2024-04-16T17:27:31.367 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_high_water": 8, 2024-04-16T17:27:31.367 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed_ops": 8436, 2024-04-16T17:27:31.367 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed": 4218, 2024-04-16T17:27:31.367 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_item_in_journal": 0 2024-04-16T17:27:31.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.368 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:31.368 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-04-16T17:27:31.368 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-04-16T17:27:31.368 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-04-16T17:27:31.368 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 410160, 2024-04-16T17:27:31.368 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 126391534, 2024-04-16T17:27:31.368 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:31.369 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 410160, 2024-04-16T17:27:31.369 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-04-16T17:27:31.369 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-04-16T17:27:31.369 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 410160, 2024-04-16T17:27:31.369 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 126391534, 2024-04-16T17:27:31.370 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-04-16T17:27:31.370 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.370 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.370 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.370 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.370 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.370 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_bytes": { 2024-04-16T17:27:31.371 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-04-16T17:27:31.371 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-04-16T17:27:31.371 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-04-16T17:27:31.371 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-04-16T17:27:31.371 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-04-16T17:27:31.371 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:31.371 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:31.371 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 19617, 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 111978715, 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 19617, 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 111978715, 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.372 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.373 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_ops": { 2024-04-16T17:27:31.373 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-04-16T17:27:31.373 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 1024, 2024-04-16T17:27:31.373 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-04-16T17:27:31.373 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-04-16T17:27:31.373 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-04-16T17:27:31.373 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:31.373 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:31.374 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 19617, 2024-04-16T17:27:31.374 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 19617, 2024-04-16T17:27:31.374 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 19617, 2024-04-16T17:27:31.374 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 19617, 2024-04-16T17:27:31.374 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-04-16T17:27:31.374 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.374 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.374 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.375 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.375 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.375 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:27:31.375 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-04-16T17:27:31.375 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-04-16T17:27:31.375 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-04-16T17:27:31.375 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 4218, 2024-04-16T17:27:31.375 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 460506, 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 4218, 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 410, 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 460506, 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.376 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.377 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.377 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.377 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:31.377 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle-0x561606d6e460": { 2024-04-16T17:27:31.377 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 18685, 2024-04-16T17:27:31.377 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-04-16T17:27:31.377 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-04-16T17:27:31.377 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 22460, 2024-04-16T17:27:31.378 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 110395793, 2024-04-16T17:27:31.378 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:31.378 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 22460, 2024-04-16T17:27:31.378 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-04-16T17:27:31.378 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-04-16T17:27:31.378 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 8738, 2024-04-16T17:27:31.378 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 110377108, 2024-04-16T17:27:31.378 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-04-16T17:27:31.379 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:31.379 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:31.379 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:31.379 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.379 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:31.379 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:27:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:31 smithi018 ceph-mon[31776]: pgmap v731: 129 pgs: 129 active+clean; 449 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 626 KiB/s wr, 15 op/s 2024-04-16T17:27:32.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:31 smithi043 ceph-mon[36901]: pgmap v731: 129 pgs: 129 active+clean; 449 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 626 KiB/s wr, 15 op/s 2024-04-16T17:27:32.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:31 smithi022 ceph-mon[36023]: pgmap v731: 129 pgs: 129 active+clean; 449 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 626 KiB/s wr, 15 op/s 2024-04-16T17:27:32.498 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-16T17:27:32.498 INFO:tasks.check_counter:Daemon mds.g mds.imported=1 2024-04-16T17:27:32.498 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=5 2024-04-16T17:27:32.498 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=399 2024-04-16T17:27:32.498 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-04-16T17:27:32.498 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:27:32.498 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:27:32.499 DEBUG:tasks.check_counter:Getting stats from j 2024-04-16T17:27:32.499 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-16T17:27:32.965 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:27:34.356 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:34 smithi018 ceph-mon[31776]: pgmap v732: 129 pgs: 129 active+clean; 450 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 768 KiB/s wr, 17 op/s 2024-04-16T17:27:34.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:34 smithi022 ceph-mon[36023]: pgmap v732: 129 pgs: 129 active+clean; 450 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 768 KiB/s wr, 17 op/s 2024-04-16T17:27:34.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:34 smithi043 ceph-mon[36901]: pgmap v732: 129 pgs: 129 active+clean; 450 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 768 KiB/s wr, 17 op/s 2024-04-16T17:27:35.942 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-16T17:27:35.943 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:35.943 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 0, 2024-04-16T17:27:35.943 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 2, 2024-04-16T17:27:35.943 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:27:35.944 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 0, 2024-04-16T17:27:35.944 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 4, 2024-04-16T17:27:35.944 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 0, 2024-04-16T17:27:35.944 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.004121291, 2024-04-16T17:27:35.944 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:27:35.944 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.000754369, 2024-04-16T17:27:35.945 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:35.945 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:35.945 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:35.945 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:35.945 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:35.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.946 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:35.946 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:35.946 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:35.946 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:35.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.946 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:27:35.946 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:27:35.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.947 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:35.947 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 7, 2024-04-16T17:27:35.947 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 329, 2024-04-16T17:27:35.947 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 3552, 2024-04-16T17:27:35.947 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 292816, 2024-04-16T17:27:35.947 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 6, 2024-04-16T17:27:35.947 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 3, 2024-04-16T17:27:35.948 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.050846716, 2024-04-16T17:27:35.948 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.039400605, 2024-04-16T17:27:35.948 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.005143243, 2024-04-16T17:27:35.948 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:35.948 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:35.948 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 327, 2024-04-16T17:27:35.948 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.063610814, 2024-04-16T17:27:35.948 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000194528 2024-04-16T17:27:35.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.949 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:35.949 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:35.949 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:35.949 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:35.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.949 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-04-16T17:27:35.950 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 292816 2024-04-16T17:27:35.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.950 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:35.950 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 436, 2024-04-16T17:27:35.950 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 425, 2024-04-16T17:27:35.950 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 1651040, 2024-04-16T17:27:35.950 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 263456, 2024-04-16T17:27:35.951 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 2, 2024-04-16T17:27:35.951 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 1, 2024-04-16T17:27:35.951 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.127969016, 2024-04-16T17:27:35.951 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.034032084, 2024-04-16T17:27:35.951 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.102021457, 2024-04-16T17:27:35.951 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.019382639, 2024-04-16T17:27:35.951 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:35.951 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 424, 2024-04-16T17:27:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.058447385, 2024-04-16T17:27:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000137847 2024-04-16T17:27:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 1651040, 2024-04-16T17:27:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 263456 2024-04-16T17:27:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-04-16T17:27:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 0, 2024-04-16T17:27:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-04-16T17:27:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.954 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-04-16T17:27:35.954 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:35.954 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:35.954 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:35.954 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:35.954 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:35.954 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:35.954 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:35.955 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:35.955 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:35.955 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:35.955 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:35.955 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:35.955 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:35.955 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:35.955 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:35.956 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:35.956 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:35.956 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:35.956 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:35.956 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:35.956 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:35.956 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:35.957 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:35.957 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:35.957 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:35.957 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:35.957 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:35.957 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:35.957 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:35.958 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:35.958 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:35.958 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-04-16T17:27:35.958 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:35.958 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:35.958 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:35.958 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:35.959 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 149745, 2024-04-16T17:27:35.959 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 31, 2024-04-16T17:27:35.959 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:35.959 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:35.959 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-04-16T17:27:35.959 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-04-16T17:27:35.959 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:35.959 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:35.960 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:35.960 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:35.960 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 0, 2024-04-16T17:27:35.960 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 0, 2024-04-16T17:27:35.960 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:35.960 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:35.960 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:35.961 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-04-16T17:27:35.961 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 928, 2024-04-16T17:27:35.961 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 2, 2024-04-16T17:27:35.961 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:35.961 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-04-16T17:27:35.961 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:35.961 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-04-16T17:27:35.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-16T17:27:35.962 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:35.962 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-04-16T17:27:35.962 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-04-16T17:27:35.962 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-04-16T17:27:35.962 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 439, 2024-04-16T17:27:35.962 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 1589435, 2024-04-16T17:27:35.962 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:35.963 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 439, 2024-04-16T17:27:35.963 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-04-16T17:27:35.963 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-04-16T17:27:35.963 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 439, 2024-04-16T17:27:35.963 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 1589435, 2024-04-16T17:27:35.963 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-04-16T17:27:35.963 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-04-16T17:27:35.963 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-04-16T17:27:35.964 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-04-16T17:27:35.964 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:35.964 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-16T17:27:35.964 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-16T17:27:36.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:36 smithi018 ceph-mon[31776]: pgmap v733: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 719 KiB/s wr, 16 op/s 2024-04-16T17:27:36.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:36 smithi043 ceph-mon[36901]: pgmap v733: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 719 KiB/s wr, 16 op/s 2024-04-16T17:27:36.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:36 smithi022 ceph-mon[36023]: pgmap v733: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 719 KiB/s wr, 16 op/s 2024-04-16T17:27:37.017 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-16T17:27:37.017 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-16T17:27:37.017 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-16T17:27:37.017 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-16T17:27:37.017 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-16T17:27:37.017 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-16T17:27:37.018 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-16T17:27:37.018 DEBUG:tasks.check_counter:Getting stats from b 2024-04-16T17:27:37.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-16T17:27:37.446 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:27:38.173 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:38 smithi022 ceph-mon[36023]: pgmap v734: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 669 KiB/s wr, 21 op/s 2024-04-16T17:27:38.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:38 smithi018 ceph-mon[31776]: pgmap v734: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 669 KiB/s wr, 21 op/s 2024-04-16T17:27:38.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:38 smithi043 ceph-mon[36901]: pgmap v734: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 669 KiB/s wr, 21 op/s 2024-04-16T17:27:39.564 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-16T17:27:39.564 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:39.565 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 90488, 2024-04-16T17:27:39.565 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 88968, 2024-04-16T17:27:39.565 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 27782759, 2024-04-16T17:27:39.565 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 498405604, 2024-04-16T17:27:39.565 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 490, 2024-04-16T17:27:39.565 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 7, 2024-04-16T17:27:39.565 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 10.373374238, 2024-04-16T17:27:39.565 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 3.499781500, 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 1.732015210, 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 4.365028983, 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 88962, 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 38.447764009, 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000432181 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:39.566 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2881, 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000430982, 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000149 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 866704, 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 271792 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 233933, 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 381335, 2024-04-16T17:27:39.567 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 54360813, 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 515399903, 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 40, 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 6, 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 19.374746155, 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 6.876552686, 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 7.711743287, 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 4.540549923, 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:39.568 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 381338, 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 62.564634549, 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000164066 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 434306, 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.075407823, 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000173 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1798640, 2024-04-16T17:27:39.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 918368 2024-04-16T17:27:39.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.570 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:39.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 710464, 2024-04-16T17:27:39.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 1153713, 2024-04-16T17:27:39.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 209227261, 2024-04-16T17:27:39.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 1064634736, 2024-04-16T17:27:39.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 23, 2024-04-16T17:27:39.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 6, 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 47.021938759, 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 19.778993948, 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 27.893973522, 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 5.148392668, 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1153711, 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 99.672813868, 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000086393 2024-04-16T17:27:39.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1217175, 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.242115832, 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000198 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 42720, 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 236208 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.572 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-04-16T17:27:39.573 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 1, 2024-04-16T17:27:39.573 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-04-16T17:27:39.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.573 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-MDSRank": { 2024-04-16T17:27:39.573 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2024-04-16T17:27:39.573 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2024-04-16T17:27:39.573 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 122701, 2024-04-16T17:27:39.573 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 242.746705495, 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.001978359 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-PurgeQueue": { 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 99026, 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 22.537028705, 2024-04-16T17:27:39.574 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000227586 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: "request": 320702, 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: "reply": 318164, 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: "reply_latency": { 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 318164, 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 2822.343613587, 2024-04-16T17:27:39.575 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.008870719 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_reply": 0, 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: "forward": 2538, 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_complete": 9, 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_keys": 1, 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_commit": 8124, 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_split": 40, 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_merge": 34, 2024-04-16T17:27:39.576 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes": 7324, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_top": 4351, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_bottom": 2955, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pin_tail": 18, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pinned": 1110, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_expired": 97, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_with_caps": 671, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": 672, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "subtrees": 12, 2024-04-16T17:27:39.577 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse": 427405, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_hit": 281129, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_forward": 2536, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_discover": 6, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_dir_fetch": 5, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_remote_ino": 0, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_lock": 2075, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "load_cent": 0, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "q": 0, 2024-04-16T17:27:39.578 INFO:teuthology.orchestra.run.smithi022.stdout: "exported": 3, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "exported_inodes": 78, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "imported": 0, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "imported_inodes": 0, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_dir_fetch": 0, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_backtrace_fetch": 14326, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_peer_discover": 0, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rfiles": 753, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rbytes": 227017401, 2024-04-16T17:27:39.579 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rsnaps": 6, 2024-04-16T17:27:39.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_fetch": 14326, 2024-04-16T17:27:39.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_set_tag": 0, 2024-04-16T17:27:39.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:27:39.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:27:39.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_inodes": 2270, 2024-04-16T17:27:39.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_base_inodes": 41, 2024-04-16T17:27:39.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dirfrag_rstats": 2229, 2024-04-16T17:27:39.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_file_inodes": 12056, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_inode_file_caps": 3, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_revoke": 35784, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_grant": 240119, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps": 115584, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps_dirty": 46348, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_cap_release": 16859, 2024-04-16T17:27:39.581 INFO:teuthology.orchestra.run.smithi022.stdout: "process_request_cap_release": 212440 2024-04-16T17:27:39.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.582 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache": { 2024-04-16T17:27:39.582 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update": 335, 2024-04-16T17:27:39.582 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update_receipt": 11, 2024-04-16T17:27:39.582 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_try_discover": 7, 2024-04-16T17:27:39.582 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_send_discover": 10, 2024-04-16T17:27:39.582 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_handle_discover": 602, 2024-04-16T17:27:39.582 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays": 0, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_delayed": 0, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_created": 52481, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_enqueued": 52481, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_reintegrated": 0, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_migrated": 0, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_processing": 0, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:27:39.583 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_started": 0, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_completed": 0, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_enqueue_scrub": 41, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_exportdir": 3, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_flush": 0, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragmentdir": 74, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragstats": 0, 2024-04-16T17:27:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_inodestats": 0 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log": { 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: "evlrg": 0, 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: "evadd": 254925, 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: "evex": 253303, 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: "evtrm": 253303, 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: "ev": 1622, 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: "evexg": 0, 2024-04-16T17:27:39.585 INFO:teuthology.orchestra.run.smithi022.stdout: "evexd": 4419, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "segadd": 425, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "segex": 422, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "segtrm": 422, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "seg": 3, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "segmjr": 1, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "segexg": 0, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "segexd": 7, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "expos": 1024027447, 2024-04-16T17:27:39.586 INFO:teuthology.orchestra.run.smithi022.stdout: "wrpos": 1032478349, 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: "rdpos": 0, 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: "jlat": { 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 64451, 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1357.616786209, 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.021064324 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: "replayed": 0 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.587 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_mem": { 2024-04-16T17:27:39.588 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 5469, 2024-04-16T17:27:39.588 INFO:teuthology.orchestra.run.smithi022.stdout: "ino+": 58066, 2024-04-16T17:27:39.588 INFO:teuthology.orchestra.run.smithi022.stdout: "ino-": 52597, 2024-04-16T17:27:39.588 INFO:teuthology.orchestra.run.smithi022.stdout: "dir": 265, 2024-04-16T17:27:39.588 INFO:teuthology.orchestra.run.smithi022.stdout: "dir+": 501, 2024-04-16T17:27:39.588 INFO:teuthology.orchestra.run.smithi022.stdout: "dir-": 236, 2024-04-16T17:27:39.588 INFO:teuthology.orchestra.run.smithi022.stdout: "dn": 7324, 2024-04-16T17:27:39.588 INFO:teuthology.orchestra.run.smithi022.stdout: "dn+": 130455, 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: "dn-": 123131, 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: "cap": 672, 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: "cap+": 55732, 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: "cap-": 55060, 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: "rss": 105032, 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: "heap": 231800 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_server": { 2024-04-16T17:27:39.589 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_client_request": 442365, 2024-04-16T17:27:39.590 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_server_request": 929, 2024-04-16T17:27:39.590 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_request": 320702, 2024-04-16T17:27:39.590 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_session": 5533, 2024-04-16T17:27:39.590 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_peer_request": 3896, 2024-04-16T17:27:39.590 INFO:teuthology.orchestra.run.smithi022.stdout: "req_create_latency": { 2024-04-16T17:27:39.590 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 53223, 2024-04-16T17:27:39.590 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 304.890358881, 2024-04-16T17:27:39.590 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.005728545 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getattr_latency": { 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1253, 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 31.928600948, 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.025481724 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getfilelock_latency": { 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.591 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: "req_link_latency": { 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookup_latency": { 2024-04-16T17:27:39.592 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 80387, 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 478.375522926, 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.005950906 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookuphash_latency": { 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.593 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupino_latency": { 2024-04-16T17:27:39.594 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2, 2024-04-16T17:27:39.594 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000351916, 2024-04-16T17:27:39.594 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000175958 2024-04-16T17:27:39.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.594 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupname_latency": { 2024-04-16T17:27:39.594 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.594 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.594 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.595 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupparent_latency": { 2024-04-16T17:27:39.595 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.595 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.595 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.596 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupsnap_latency": { 2024-04-16T17:27:39.596 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.596 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.596 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.596 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lssnap_latency": { 2024-04-16T17:27:39.596 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 23, 2024-04-16T17:27:39.597 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.025206350, 2024-04-16T17:27:39.597 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.001095928 2024-04-16T17:27:39.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.597 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mkdir_latency": { 2024-04-16T17:27:39.597 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 119, 2024-04-16T17:27:39.597 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.483957665, 2024-04-16T17:27:39.597 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.004066871 2024-04-16T17:27:39.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mknod_latency": { 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mksnap_latency": { 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 23, 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 6.564341604, 2024-04-16T17:27:39.598 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.285406156 2024-04-16T17:27:39.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.599 INFO:teuthology.orchestra.run.smithi022.stdout: "req_open_latency": { 2024-04-16T17:27:39.599 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 10, 2024-04-16T17:27:39.599 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.007657522, 2024-04-16T17:27:39.599 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000765752 2024-04-16T17:27:39.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.599 INFO:teuthology.orchestra.run.smithi022.stdout: "req_readdir_latency": { 2024-04-16T17:27:39.599 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 117424, 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1590.696530675, 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.013546604 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rename_latency": { 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 11240, 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 73.797495630, 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.006565613 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.600 INFO:teuthology.orchestra.run.smithi022.stdout: "req_renamesnap_latency": { 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: "req_snapdiff_latency": { 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.602 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmdir_latency": { 2024-04-16T17:27:39.602 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1, 2024-04-16T17:27:39.602 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.002412599, 2024-04-16T17:27:39.602 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.002412599 2024-04-16T17:27:39.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.602 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmsnap_latency": { 2024-04-16T17:27:39.602 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 17, 2024-04-16T17:27:39.602 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.969292893, 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.057017229 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmxattr_latency": { 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setattr_latency": { 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 272, 2024-04-16T17:27:39.603 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 12.794260590, 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.047037722 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setdirlayout_latency": { 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setfilelock_latency": { 2024-04-16T17:27:39.604 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1680, 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 7.095121871, 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.004223286 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setlayout_latency": { 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setxattr_latency": { 2024-04-16T17:27:39.605 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 9, 2024-04-16T17:27:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.022895696, 2024-04-16T17:27:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.002543966 2024-04-16T17:27:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: "req_symlink_latency": { 2024-04-16T17:27:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "req_unlink_latency": { 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 52481, 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 314.689605821, 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.005996257 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getvxattr_latency": { 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.607 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_sessions": { 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: "session_count": 2, 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: "session_add": 7, 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: "session_remove": 5, 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_open": 2, 2024-04-16T17:27:39.608 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_stale": 0, 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "total_load": 36916, 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "average_load": 18458, 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "avg_session_uptime": 1367, 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "mdthresh_evicted": 0 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:39.609 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:39.610 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:39.610 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:39.610 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:39.610 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:39.610 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:39.610 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:39.610 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:39.610 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:39.611 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:39.611 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:39.611 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:39.611 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:39.611 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:39.611 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:39.611 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:39.611 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:39.612 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 704419, 2024-04-16T17:27:39.613 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 2526, 2024-04-16T17:27:39.614 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 176, 2024-04-16T17:27:39.614 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 2, 2024-04-16T17:27:39.614 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-04-16T17:27:39.614 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-04-16T17:27:39.614 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:39.614 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:39.614 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:39.614 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:39.615 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 9688, 2024-04-16T17:27:39.615 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 185, 2024-04-16T17:27:39.615 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:39.615 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:39.615 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:39.615 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-04-16T17:27:39.615 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 19116368, 2024-04-16T17:27:39.615 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 159408, 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter": { 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: "op_active": 0, 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: "op_laggy": 0, 2024-04-16T17:27:39.616 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send": 252869, 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send_bytes": 1101504687, 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: "op_resend": 32, 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: "op_reply": 252869, 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: "op_latency": { 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 252869, 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 22572.592356160, 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.089265953 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.617 INFO:teuthology.orchestra.run.smithi022.stdout: "op_inflight": 0, 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: "oplen_avg": { 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 252869, 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 357816 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: "op": 252869, 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: "op_r": 15774, 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: "op_w": 237095, 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: "op_rmw": 0, 2024-04-16T17:27:39.618 INFO:teuthology.orchestra.run.smithi022.stdout: "op_pg": 0, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_stat": 7648, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_create": 54403, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_read": 1397, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_write": 65807, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writefull": 1086, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writesame": 0, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_append": 0, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_zero": 0, 2024-04-16T17:27:39.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_truncate": 40, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_delete": 106153, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_mapext": 0, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_sparse_read": 0, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_clonerange": 0, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_getxattr": 14377, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_setxattr": 83115, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_rmxattr": 0, 2024-04-16T17:27:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:27:39.621 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_call": 0, 2024-04-16T17:27:39.621 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_watch": 0, 2024-04-16T17:27:39.621 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_notify": 0, 2024-04-16T17:27:39.621 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:27:39.621 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls": 0, 2024-04-16T17:27:39.621 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:27:39.621 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_other": 0, 2024-04-16T17:27:39.621 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_active": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_send": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_resend": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_ping": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_active": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_send": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_resend": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_active": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_send": 0, 2024-04-16T17:27:39.622 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_resend": 0, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_active": 0, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_send": 0, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_resend": 0, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "command_active": 0, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "command_send": 0, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "command_resend": 0, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "map_epoch": 139, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "map_full": 0, 2024-04-16T17:27:39.623 INFO:teuthology.orchestra.run.smithi022.stdout: "map_inc": 55, 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_sessions": 12, 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_open": 12, 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_close": 0, 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_laggy": 0, 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_wr": 17814, 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_rd": 20, 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_del": 5956 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.624 INFO:teuthology.orchestra.run.smithi022.stdout: "oft": { 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_objs": 1, 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_updates": 0, 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_removes": 0 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: "osd-slow-ops": { 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_ops_count": 4 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.625 INFO:teuthology.orchestra.run.smithi022.stdout: "purge_queue": { 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops": 0, 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops_high_water": 16, 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing": 0, 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_high_water": 8, 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executed_ops": 104962, 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executed": 52481, 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_item_in_journal": 0 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.626 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 1034881, 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 198101251, 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 1034881, 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-16T17:27:39.627 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1034881, 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 198101251, 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_bytes": { 2024-04-16T17:27:39.628 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:39.629 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-16T17:27:39.629 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:39.629 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2024-04-16T17:27:39.629 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2024-04-16T17:27:39.629 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:39.629 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:39.629 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 252869, 2024-04-16T17:27:39.629 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 1107239248, 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 252869, 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1107239248, 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.630 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_ops": { 2024-04-16T17:27:39.631 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:39.631 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 1024, 2024-04-16T17:27:39.631 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:39.631 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2024-04-16T17:27:39.631 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2024-04-16T17:27:39.631 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:39.631 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:39.631 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 252869, 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 252869, 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 252869, 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 252869, 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.632 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 52481, 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 5734709, 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 52481, 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-16T17:27:39.633 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1356, 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 5734709, 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:39.634 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle-0x55c859688460": { 2024-04-16T17:27:39.635 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:39.635 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2024-04-16T17:27:39.635 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:39.635 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 254925, 2024-04-16T17:27:39.635 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1028284045, 2024-04-16T17:27:39.635 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:39.635 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 254925, 2024-04-16T17:27:39.635 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 64451, 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1028284045, 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:39.637 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-16T17:27:40.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:40 smithi022 ceph-mon[36023]: pgmap v735: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 253 KiB/s wr, 13 op/s 2024-04-16T17:27:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:40 smithi018 ceph-mon[31776]: pgmap v735: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 253 KiB/s wr, 13 op/s 2024-04-16T17:27:40.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:40 smithi043 ceph-mon[36901]: pgmap v735: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 253 KiB/s wr, 13 op/s 2024-04-16T17:27:40.708 INFO:tasks.check_counter:Daemon mds.b mds.exported=3 2024-04-16T17:27:40.709 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-16T17:27:40.709 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=335 2024-04-16T17:27:40.709 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=11 2024-04-16T17:27:40.709 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-16T17:27:40.709 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=23 2024-04-16T17:27:40.709 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=17 2024-04-16T17:27:40.709 DEBUG:tasks.check_counter:Getting stats from e 2024-04-16T17:27:40.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-16T17:27:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:27:42.153 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:42 smithi022 ceph-mon[36023]: pgmap v736: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 275 KiB/s wr, 16 op/s 2024-04-16T17:27:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:42 smithi018 ceph-mon[31776]: pgmap v736: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 275 KiB/s wr, 16 op/s 2024-04-16T17:27:42.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:42 smithi043 ceph-mon[36901]: pgmap v736: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 275 KiB/s wr, 16 op/s 2024-04-16T17:27:43.057 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-16T17:27:43.057 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:43.057 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 632, 2024-04-16T17:27:43.057 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 633, 2024-04-16T17:27:43.057 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 172818538, 2024-04-16T17:27:43.057 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 200673, 2024-04-16T17:27:43.058 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 8, 2024-04-16T17:27:43.058 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 4, 2024-04-16T17:27:43.058 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.262422223, 2024-04-16T17:27:43.058 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.011146761, 2024-04-16T17:27:43.058 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.122212827, 2024-04-16T17:27:43.058 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.122836147, 2024-04-16T17:27:43.058 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 631, 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.040597593, 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000064338 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 1359, 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 1348, 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 158280216, 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 553559, 2024-04-16T17:27:43.060 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 4, 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.218062034, 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.044062049, 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.141391885, 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.034109101, 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1347, 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.068310833, 2024-04-16T17:27:43.061 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000050713 2024-04-16T17:27:43.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.062 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:43.062 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.062 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.062 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.062 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1680160, 2024-04-16T17:27:43.062 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 267616 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 454, 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 776, 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 142725203, 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 573347, 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 9, 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 6, 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.129535884, 2024-04-16T17:27:43.063 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.020751203, 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.091272608, 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.011532916, 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 773, 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.051476680, 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000066593 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.064 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 431200 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 1, 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-04-16T17:27:43.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-MDSRank": { 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 3034, 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.350043418, 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000115373 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.066 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-PurgeQueue": { 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2024-04-16T17:27:43.067 INFO:teuthology.orchestra.run.smithi022.stdout: "request": 0, 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: "reply": 0, 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: "reply_latency": { 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_reply": 0, 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: "forward": 0, 2024-04-16T17:27:43.068 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_complete": 0, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_keys": 0, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_commit": 0, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_split": 0, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_merge": 0, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes": 25415, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_top": 17077, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_bottom": 8338, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pin_tail": 0, 2024-04-16T17:27:43.069 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pinned": 1020, 2024-04-16T17:27:43.070 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_expired": 0, 2024-04-16T17:27:43.070 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_with_caps": 0, 2024-04-16T17:27:43.070 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": 0, 2024-04-16T17:27:43.070 INFO:teuthology.orchestra.run.smithi022.stdout: "subtrees": 8, 2024-04-16T17:27:43.070 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse": 0, 2024-04-16T17:27:43.070 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_hit": 0, 2024-04-16T17:27:43.070 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_forward": 0, 2024-04-16T17:27:43.070 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_discover": 0, 2024-04-16T17:27:43.071 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:27:43.071 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_remote_ino": 0, 2024-04-16T17:27:43.071 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_lock": 0, 2024-04-16T17:27:43.071 INFO:teuthology.orchestra.run.smithi022.stdout: "load_cent": 0, 2024-04-16T17:27:43.071 INFO:teuthology.orchestra.run.smithi022.stdout: "q": 0, 2024-04-16T17:27:43.071 INFO:teuthology.orchestra.run.smithi022.stdout: "exported": 0, 2024-04-16T17:27:43.071 INFO:teuthology.orchestra.run.smithi022.stdout: "exported_inodes": 0, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "imported": 0, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "imported_inodes": 0, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_dir_fetch": 0, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_backtrace_fetch": 0, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_peer_discover": 0, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rfiles": 753, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rbytes": 227306179, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rsnaps": 6, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T17:27:43.072 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_set_tag": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_file_inodes": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T17:27:43.073 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_grant": 0, 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps": 0, 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps_dirty": 0, 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_cap_release": 0, 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: "process_request_cap_release": 0 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.074 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache": { 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update": 0, 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update_receipt": 0, 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_try_discover": 0, 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_send_discover": 0, 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_handle_discover": 0, 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays": 0, 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_delayed": 0, 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:27:43.075 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_created": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_enqueued": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_reintegrated": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_migrated": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_processing": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_started": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_completed": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:27:43.076 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_exportdir": 0, 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_flush": 0, 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragstats": 0, 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_inodestats": 0 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log": { 2024-04-16T17:27:43.077 INFO:teuthology.orchestra.run.smithi022.stdout: "evlrg": 0, 2024-04-16T17:27:43.078 INFO:teuthology.orchestra.run.smithi022.stdout: "evadd": 0, 2024-04-16T17:27:43.078 INFO:teuthology.orchestra.run.smithi022.stdout: "evex": 0, 2024-04-16T17:27:43.078 INFO:teuthology.orchestra.run.smithi022.stdout: "evtrm": 0, 2024-04-16T17:27:43.078 INFO:teuthology.orchestra.run.smithi022.stdout: "ev": 104503, 2024-04-16T17:27:43.078 INFO:teuthology.orchestra.run.smithi022.stdout: "evexg": 0, 2024-04-16T17:27:43.078 INFO:teuthology.orchestra.run.smithi022.stdout: "evexd": 0, 2024-04-16T17:27:43.078 INFO:teuthology.orchestra.run.smithi022.stdout: "segadd": 0, 2024-04-16T17:27:43.078 INFO:teuthology.orchestra.run.smithi022.stdout: "segex": 0, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "segtrm": 0, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "seg": 3, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "segmjr": 25, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "segexg": 0, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "segexd": 0, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "expos": 1024027447, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "wrpos": 1032478349, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "rdpos": 1032468491, 2024-04-16T17:27:43.079 INFO:teuthology.orchestra.run.smithi022.stdout: "jlat": { 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: "replayed": 104503 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_mem": { 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 3313, 2024-04-16T17:27:43.080 INFO:teuthology.orchestra.run.smithi022.stdout: "ino+": 24813, 2024-04-16T17:27:43.081 INFO:teuthology.orchestra.run.smithi022.stdout: "ino-": 21500, 2024-04-16T17:27:43.081 INFO:teuthology.orchestra.run.smithi022.stdout: "dir": 247, 2024-04-16T17:27:43.081 INFO:teuthology.orchestra.run.smithi022.stdout: "dir+": 411, 2024-04-16T17:27:43.081 INFO:teuthology.orchestra.run.smithi022.stdout: "dir-": 164, 2024-04-16T17:27:43.081 INFO:teuthology.orchestra.run.smithi022.stdout: "dn": 25415, 2024-04-16T17:27:43.081 INFO:teuthology.orchestra.run.smithi022.stdout: "dn+": 25415, 2024-04-16T17:27:43.081 INFO:teuthology.orchestra.run.smithi022.stdout: "dn-": 0, 2024-04-16T17:27:43.081 INFO:teuthology.orchestra.run.smithi022.stdout: "cap": 0, 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "cap+": 0, 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "cap-": 0, 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "rss": 83432, 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "heap": 215416 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_server": { 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_client_request": 0, 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_server_request": 0, 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_request": 0, 2024-04-16T17:27:43.082 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_session": 0, 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_peer_request": 0, 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: "req_create_latency": { 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getattr_latency": { 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.083 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getfilelock_latency": { 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: "req_link_latency": { 2024-04-16T17:27:43.084 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookup_latency": { 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookuphash_latency": { 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupino_latency": { 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupname_latency": { 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupparent_latency": { 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupsnap_latency": { 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lssnap_latency": { 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.089 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mkdir_latency": { 2024-04-16T17:27:43.089 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.090 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.090 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.090 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mknod_latency": { 2024-04-16T17:27:43.090 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.090 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.090 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.091 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mksnap_latency": { 2024-04-16T17:27:43.091 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.091 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.091 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.091 INFO:teuthology.orchestra.run.smithi022.stdout: "req_open_latency": { 2024-04-16T17:27:43.091 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.091 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: "req_readdir_latency": { 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rename_latency": { 2024-04-16T17:27:43.092 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.093 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.093 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.093 INFO:teuthology.orchestra.run.smithi022.stdout: "req_renamesnap_latency": { 2024-04-16T17:27:43.093 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.093 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.093 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.094 INFO:teuthology.orchestra.run.smithi022.stdout: "req_snapdiff_latency": { 2024-04-16T17:27:43.094 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.094 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.094 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.094 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmdir_latency": { 2024-04-16T17:27:43.094 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.094 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmsnap_latency": { 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmxattr_latency": { 2024-04-16T17:27:43.095 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.096 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.096 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.096 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setattr_latency": { 2024-04-16T17:27:43.096 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.096 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.096 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.097 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setdirlayout_latency": { 2024-04-16T17:27:43.097 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.097 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.097 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.097 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setfilelock_latency": { 2024-04-16T17:27:43.097 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.097 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setlayout_latency": { 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setxattr_latency": { 2024-04-16T17:27:43.098 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: "req_symlink_latency": { 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.099 INFO:teuthology.orchestra.run.smithi022.stdout: "req_unlink_latency": { 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getvxattr_latency": { 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.100 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.101 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.101 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.101 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_sessions": { 2024-04-16T17:27:43.101 INFO:teuthology.orchestra.run.smithi022.stdout: "session_count": 2, 2024-04-16T17:27:43.101 INFO:teuthology.orchestra.run.smithi022.stdout: "session_add": 2, 2024-04-16T17:27:43.101 INFO:teuthology.orchestra.run.smithi022.stdout: "session_remove": 0, 2024-04-16T17:27:43.101 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_open": 0, 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_stale": 0, 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: "total_load": 0, 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: "average_load": 0, 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: "avg_session_uptime": 2998, 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: "mdthresh_evicted": 0 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:43.102 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:43.103 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:43.103 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:43.103 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:43.103 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:43.103 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:43.103 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:43.103 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:43.103 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:43.104 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:43.105 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:43.106 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 378387, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 4075, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:43.107 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 9688, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 185, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 26708152, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 195450, 2024-04-16T17:27:43.108 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter": { 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: "op_active": 0, 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: "op_laggy": 0, 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send": 1977, 2024-04-16T17:27:43.109 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send_bytes": 26, 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "op_resend": 4, 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "op_reply": 1977, 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "op_latency": { 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1977, 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 23.980732734, 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.012129859 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "op_inflight": 0, 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "oplen_avg": { 2024-04-16T17:27:43.110 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1977, 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1978 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: "op": 1977, 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: "op_r": 1977, 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: "op_w": 0, 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: "op_rmw": 0, 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: "op_pg": 0, 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_stat": 1162, 2024-04-16T17:27:43.111 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_create": 0, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_read": 812, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_write": 0, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writefull": 0, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writesame": 0, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_append": 0, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_zero": 0, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_truncate": 0, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_delete": 0, 2024-04-16T17:27:43.112 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_mapext": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_sparse_read": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_clonerange": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_getxattr": 2, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_setxattr": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_rmxattr": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_call": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_watch": 0, 2024-04-16T17:27:43.113 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_notify": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_other": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_active": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_send": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_resend": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_ping": 0, 2024-04-16T17:27:43.114 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_active": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_send": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_resend": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_active": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_send": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_resend": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_active": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_send": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_resend": 0, 2024-04-16T17:27:43.115 INFO:teuthology.orchestra.run.smithi022.stdout: "command_active": 0, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "command_send": 0, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "command_resend": 0, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "map_epoch": 139, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "map_full": 0, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "map_inc": 18, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_sessions": 10, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_open": 10, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_close": 0, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_laggy": 0, 2024-04-16T17:27:43.116 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_wr": 0, 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_rd": 2, 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_del": 0 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: "oft": { 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_objs": 0, 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_updates": 0, 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_removes": 0 2024-04-16T17:27:43.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: "osd-slow-ops": { 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_ops_count": 0 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: "purge_queue": { 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops": 0, 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing": 0, 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_high_water": 0, 2024-04-16T17:27:43.118 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executed_ops": 0, 2024-04-16T17:27:43.119 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executed": 0, 2024-04-16T17:27:43.119 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_item_in_journal": 0 2024-04-16T17:27:43.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.119 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:43.119 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:43.119 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-16T17:27:43.119 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:43.119 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 2441, 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 473577507, 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 2441, 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 2441, 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 473577507, 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:43.120 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.121 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.121 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.121 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.121 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_bytes": { 2024-04-16T17:27:43.121 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:43.121 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-16T17:27:43.121 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 1977, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 471597470, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1977, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 471597470, 2024-04-16T17:27:43.122 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_ops": { 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 1024, 2024-04-16T17:27:43.123 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 1977, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 1977, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1977, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1977, 2024-04-16T17:27:43.124 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2024-04-16T17:27:43.125 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 0, 2024-04-16T17:27:43.126 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle-0x55ea781866e0": { 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2024-04-16T17:27:43.127 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 0, 2024-04-16T17:27:43.128 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:43.129 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:43.129 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:43.129 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:43.129 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.129 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:43.129 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-16T17:27:43.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:43 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:27:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:43 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:27:43.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:43 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:27:43.996 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-16T17:27:43.996 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-16T17:27:43.996 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-16T17:27:43.996 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-16T17:27:43.996 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-16T17:27:43.997 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:27:43.997 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:27:43.997 DEBUG:tasks.check_counter:Getting stats from h 2024-04-16T17:27:43.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-16T17:27:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:27:44.374 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:44 smithi022 ceph-mon[36023]: pgmap v737: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 214 KiB/s wr, 14 op/s 2024-04-16T17:27:44.375 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:27:44.375 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:44 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:27:44.375 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:44 smithi022 ceph-mon[36023]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:27:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:44 smithi018 ceph-mon[31776]: pgmap v737: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 214 KiB/s wr, 14 op/s 2024-04-16T17:27:44.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:27:44.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:44 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:27:44.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:44 smithi018 ceph-mon[31776]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:27:44.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:44 smithi043 ceph-mon[36901]: pgmap v737: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 214 KiB/s wr, 14 op/s 2024-04-16T17:27:44.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:27:44.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:44 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:27:44.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:44 smithi043 ceph-mon[36901]: from='mgr.25417 ' entity='mgr.x' 2024-04-16T17:27:45.980 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-16T17:27:45.980 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:45.980 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 4, 2024-04-16T17:27:45.980 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 4, 2024-04-16T17:27:45.980 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3072, 2024-04-16T17:27:45.981 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 448, 2024-04-16T17:27:45.981 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-04-16T17:27:45.981 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-04-16T17:27:45.981 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.003559826, 2024-04-16T17:27:45.981 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000053748, 2024-04-16T17:27:45.981 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000659601, 2024-04-16T17:27:45.981 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:45.981 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 3, 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.001663313, 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000554437 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:45.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.983 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-16T17:27:45.983 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T17:27:45.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.983 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:45.983 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 110, 2024-04-16T17:27:45.983 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 109, 2024-04-16T17:27:45.983 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 172640, 2024-04-16T17:27:45.983 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 66448, 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.016185264, 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.005510676, 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.007991904, 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.002521207, 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 108, 2024-04-16T17:27:45.984 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.007203121, 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000066695 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 172640, 2024-04-16T17:27:45.985 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 66448 2024-04-16T17:27:45.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.986 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:45.986 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 1, 2024-04-16T17:27:45.986 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 83, 2024-04-16T17:27:45.986 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 160, 2024-04-16T17:27:45.986 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 92704, 2024-04-16T17:27:45.986 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 3, 2024-04-16T17:27:45.986 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 2, 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.004212367, 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.002752600, 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000541772, 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 81, 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.004959170, 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000061224 2024-04-16T17:27:45.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.988 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:45.988 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:45.988 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:45.988 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:45.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.988 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-16T17:27:45.988 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 92704 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:45.989 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:45.990 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:45.990 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:45.990 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:45.990 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:45.990 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:45.990 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:45.990 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:45.990 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:45.991 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:45.992 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:45.992 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:45.992 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:45.992 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:45.992 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:45.992 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:45.992 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:45.993 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:45.994 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148779, 2024-04-16T17:27:45.994 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 27, 2024-04-16T17:27:45.994 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:45.994 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:45.994 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-04-16T17:27:45.994 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-04-16T17:27:45.994 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:45.994 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-04-16T17:27:45.995 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:45.996 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 111, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 157126, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 111, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 111, 2024-04-16T17:27:45.997 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 157126, 2024-04-16T17:27:45.998 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:45.998 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:45.998 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:45.998 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:45.998 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:45.998 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:45.998 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-16T17:27:46.378 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:46 smithi022 ceph-mon[36023]: pgmap v738: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 95 KiB/s wr, 11 op/s 2024-04-16T17:27:46.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:46 smithi018 ceph-mon[31776]: pgmap v738: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 95 KiB/s wr, 11 op/s 2024-04-16T17:27:46.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:46 smithi043 ceph-mon[36901]: pgmap v738: 129 pgs: 129 active+clean; 451 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 95 KiB/s wr, 11 op/s 2024-04-16T17:27:46.597 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.h 2024-04-16T17:27:46.597 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.h 2024-04-16T17:27:46.597 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.h 2024-04-16T17:27:46.597 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.h 2024-04-16T17:27:46.597 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-16T17:27:46.597 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-16T17:27:46.598 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-16T17:27:46.598 DEBUG:tasks.check_counter:Getting stats from k 2024-04-16T17:27:46.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-16T17:27:46.855 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:27:48.312 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:48 smithi022 ceph-mon[36023]: pgmap v739: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 98 KiB/s wr, 15 op/s 2024-04-16T17:27:48.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:48 smithi018 ceph-mon[31776]: pgmap v739: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 98 KiB/s wr, 15 op/s 2024-04-16T17:27:48.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:48 smithi043 ceph-mon[36901]: pgmap v739: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 98 KiB/s wr, 15 op/s 2024-04-16T17:27:48.569 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-16T17:27:48.569 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:48.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-04-16T17:27:48.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-04-16T17:27:48.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:27:48.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-04-16T17:27:48.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-04-16T17:27:48.569 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-04-16T17:27:48.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.001103048, 2024-04-16T17:27:48.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:27:48.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000125846, 2024-04-16T17:27:48.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:48.570 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:48.570 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:48.570 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:48.570 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:27:48.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 442, 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 431, 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1653296, 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 266992, 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.068040844, 2024-04-16T17:27:48.572 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.021764363, 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.043462708, 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.009911478, 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 430, 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.020341439, 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000047305 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:48.573 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1653296, 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 266992 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 7, 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 331, 2024-04-16T17:27:48.574 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3552, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 293168, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 6, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.016810499, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.011665357, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.001953719, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 329, 2024-04-16T17:27:48.575 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.036418874, 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000110695 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-04-16T17:27:48.576 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 293168 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:48.577 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:48.578 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:48.579 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:48.579 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:48.579 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:48.579 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:48.579 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:48.579 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:48.579 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:48.579 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:48.580 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:48.580 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:48.580 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:48.580 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:48.580 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:48.580 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:48.580 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:48.580 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:48.581 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:48.581 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:48.581 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-04-16T17:27:48.581 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:48.581 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:48.581 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:48.581 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:48.581 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148692, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 30, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:48.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-04-16T17:27:48.583 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-16T17:27:48.584 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 445, 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1590852, 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 445, 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 445, 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1590852, 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-16T17:27:48.585 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-16T17:27:48.586 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-16T17:27:48.586 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-16T17:27:48.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:48.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-16T17:27:48.586 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-16T17:27:49.266 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-16T17:27:49.266 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-16T17:27:49.266 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-16T17:27:49.266 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-16T17:27:49.266 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-16T17:27:49.266 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-16T17:27:49.267 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-16T17:27:49.267 DEBUG:tasks.check_counter:Getting stats from c 2024-04-16T17:27:49.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-16T17:27:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:27:50.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:50 smithi043 ceph-mon[36901]: pgmap v740: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-04-16T17:27:50.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:50 smithi022 ceph-mon[36023]: pgmap v740: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-04-16T17:27:50.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:50 smithi018 ceph-mon[31776]: pgmap v740: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-04-16T17:27:51.405 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-16T17:27:51.406 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:51.406 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4061, 2024-04-16T17:27:51.406 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4062, 2024-04-16T17:27:51.406 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 47295757, 2024-04-16T17:27:51.406 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 1291087, 2024-04-16T17:27:51.406 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 8, 2024-04-16T17:27:51.407 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-16T17:27:51.407 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.352383552, 2024-04-16T17:27:51.407 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.100754329, 2024-04-16T17:27:51.407 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.109828631, 2024-04-16T17:27:51.407 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.114111831, 2024-04-16T17:27:51.407 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:51.407 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4060, 2024-04-16T17:27:51.407 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.139670895, 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000034401 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:27:51.408 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 582, 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 904, 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 35127861, 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 915326, 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 9, 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 7, 2024-04-16T17:27:51.409 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.082003706, 2024-04-16T17:27:51.410 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.022715932, 2024-04-16T17:27:51.410 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.037241216, 2024-04-16T17:27:51.410 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.015112443, 2024-04-16T17:27:51.410 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:51.410 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 902, 2024-04-16T17:27:51.410 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.047793944, 2024-04-16T17:27:51.410 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000052986 2024-04-16T17:27:51.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 732096 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:51.412 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 844, 2024-04-16T17:27:51.412 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 821, 2024-04-16T17:27:51.412 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 31200430, 2024-04-16T17:27:51.412 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 381848, 2024-04-16T17:27:51.412 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 6, 2024-04-16T17:27:51.412 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-16T17:27:51.412 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.115457593, 2024-04-16T17:27:51.412 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.032786251, 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.069576393, 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.018165609, 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 820, 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.034015695, 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000041482 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1754128, 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 273728 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-16T17:27:51.414 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-16T17:27:51.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.415 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-16T17:27:51.415 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-16T17:27:51.415 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-16T17:27:51.415 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8522, 2024-04-16T17:27:51.415 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.206084673, 2024-04-16T17:27:51.415 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000024182 2024-04-16T17:27:51.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.416 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-16T17:27:51.416 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-16T17:27:51.416 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-16T17:27:51.416 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.416 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.416 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 0, 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.418 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-16T17:27:51.418 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-04-16T17:27:51.418 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-16T17:27:51.418 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-16T17:27:51.418 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 0, 2024-04-16T17:27:51.418 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-16T17:27:51.418 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-16T17:27:51.418 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 4736, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 4736, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 4736, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 0, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 0, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 6, 2024-04-16T17:27:51.419 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-16T17:27:51.420 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-16T17:27:51.421 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-16T17:27:51.421 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-04-16T17:27:51.421 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-04-16T17:27:51.421 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-16T17:27:51.421 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 0, 2024-04-16T17:27:51.421 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-16T17:27:51.421 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-04-16T17:27:51.421 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-04-16T17:27:51.422 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-04-16T17:27:51.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T17:27:51.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-16T17:27:51.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:27:51.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:27:51.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:27:51.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:27:51.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 0, 2024-04-16T17:27:51.423 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 0, 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 0, 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 0 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-04-16T17:27:51.424 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-16T17:27:51.425 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:27:51.426 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:27:51.426 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-16T17:27:51.426 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-16T17:27:51.426 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:27:51.426 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:27:51.426 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:27:51.426 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-16T17:27:51.426 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 0, 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-16T17:27:51.427 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 22470, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 0, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 35, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 3, 2024-04-16T17:27:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 114670325, 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 114663322, 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 22470 2024-04-16T17:27:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-16T17:27:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 479, 2024-04-16T17:27:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 4697, 2024-04-16T17:27:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 4218, 2024-04-16T17:27:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 37, 2024-04-16T17:27:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 38, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 1, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 4736, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 4736, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 0, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 0, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 0, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 0, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 56328, 2024-04-16T17:27:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215416 2024-04-16T17:27:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-16T17:27:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 0, 2024-04-16T17:27:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-16T17:27:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 0, 2024-04-16T17:27:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 0, 2024-04-16T17:27:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-04-16T17:27:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-16T17:27:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-16T17:27:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-16T17:27:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-16T17:27:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-16T17:27:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-16T17:27:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-16T17:27:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-16T17:27:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-16T17:27:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-16T17:27:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-16T17:27:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-16T17:27:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-16T17:27:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-16T17:27:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-16T17:27:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-16T17:27:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-16T17:27:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:27:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:27:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-16T17:27:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-16T17:27:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 2, 2024-04-16T17:27:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 3, 2024-04-16T17:27:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 1, 2024-04-16T17:27:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 3, 2024-04-16T17:27:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-16T17:27:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 0, 2024-04-16T17:27:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 0, 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 3808, 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 262172, 2024-04-16T17:27:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 655, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 9688, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 185, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 4671378, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 32493, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 4976, 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 21, 2024-04-16T17:27:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 3, 2024-04-16T17:27:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 4976, 2024-04-16T17:27:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-16T17:27:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4976, 2024-04-16T17:27:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 20.326775931, 2024-04-16T17:27:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.004084963 2024-04-16T17:27:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-16T17:27:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-16T17:27:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4976, 2024-04-16T17:27:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 4977 2024-04-16T17:27:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 4976, 2024-04-16T17:27:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 4976, 2024-04-16T17:27:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 0, 2024-04-16T17:27:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-16T17:27:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 2903, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 0, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 2071, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 0, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 0, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 0, 2024-04-16T17:27:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 0, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 1, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 0, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-16T17:27:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-16T17:27:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-16T17:27:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 139, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 50, 2024-04-16T17:27:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 11, 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 12, 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 1, 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 0, 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 2, 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 0 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-16T17:27:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 0, 2024-04-16T17:27:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:27:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-16T17:27:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-16T17:27:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-16T17:27:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-16T17:27:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 0, 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 0, 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 0, 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-16T17:27:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 5483, 2024-04-16T17:27:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 113101820, 2024-04-16T17:27:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 5483, 2024-04-16T17:27:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 5483, 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 113101820, 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.473 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-16T17:27:51.473 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:51.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-16T17:27:51.473 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:51.473 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:51.473 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:51.473 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:51.473 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:51.474 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 4976, 2024-04-16T17:27:51.474 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 110476026, 2024-04-16T17:27:51.474 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 4976, 2024-04-16T17:27:51.474 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 110476026, 2024-04-16T17:27:51.474 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:51.474 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.474 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:51.475 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:51.476 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:51.476 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:51.476 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 4976, 2024-04-16T17:27:51.476 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 4976, 2024-04-16T17:27:51.476 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 4976, 2024-04-16T17:27:51.476 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 4976, 2024-04-16T17:27:51.476 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:51.476 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.477 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.477 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.477 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:27:51.477 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:51.477 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-16T17:27:51.477 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:51.478 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:51.478 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:51.478 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:51.478 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:51.478 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:51.478 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:51.478 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-04-16T17:27:51.478 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x5600b3e106e0": { 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:51.479 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-16T17:27:51.480 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:51.480 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:51.480 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:51.480 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:51.480 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:51.480 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:51.480 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:51.480 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-04-16T17:27:51.481 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-04-16T17:27:51.481 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:51.481 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:51.481 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:51.481 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:51.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:51.481 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-16T17:27:52.242 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-16T17:27:52.242 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-16T17:27:52.243 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-16T17:27:52.243 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-16T17:27:52.243 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-16T17:27:52.243 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:27:52.243 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:27:52.243 DEBUG:tasks.check_counter:Getting stats from f 2024-04-16T17:27:52.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-16T17:27:52.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:52 smithi022 ceph-mon[36023]: pgmap v741: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-04-16T17:27:52.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:52 smithi018 ceph-mon[31776]: pgmap v741: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-04-16T17:27:52.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:52 smithi043 ceph-mon[36901]: pgmap v741: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-04-16T17:27:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:27:54.331 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:54 smithi018 ceph-mon[31776]: pgmap v742: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-16T17:27:54.360 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:54 smithi043 ceph-mon[36901]: pgmap v742: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-16T17:27:54.427 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:54 smithi022 ceph-mon[36023]: pgmap v742: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-16T17:27:54.498 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-16T17:27:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 1511, 2024-04-16T17:27:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 1513, 2024-04-16T17:27:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 4588281, 2024-04-16T17:27:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 480562, 2024-04-16T17:27:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 7, 2024-04-16T17:27:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 2, 2024-04-16T17:27:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.136021120, 2024-04-16T17:27:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.039398943, 2024-04-16T17:27:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.040674708, 2024-04-16T17:27:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.042279860, 2024-04-16T17:27:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1511, 2024-04-16T17:27:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.055436033, 2024-04-16T17:27:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000036688 2024-04-16T17:27:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 508, 2024-04-16T17:27:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 763, 2024-04-16T17:27:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1755074, 2024-04-16T17:27:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 861089, 2024-04-16T17:27:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-16T17:27:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-16T17:27:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.087691389, 2024-04-16T17:27:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.035023172, 2024-04-16T17:27:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.047698262, 2024-04-16T17:27:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.010613808, 2024-04-16T17:27:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 762, 2024-04-16T17:27:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.046469023, 2024-04-16T17:27:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000060982 2024-04-16T17:27:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1754560, 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 860448 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 3181, 2024-04-16T17:27:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 3226, 2024-04-16T17:27:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 2260590, 2024-04-16T17:27:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 1234608, 2024-04-16T17:27:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 8, 2024-04-16T17:27:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 5, 2024-04-16T17:27:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.310030847, 2024-04-16T17:27:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.085239598, 2024-04-16T17:27:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.081972943, 2024-04-16T17:27:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.115363318, 2024-04-16T17:27:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3224, 2024-04-16T17:27:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.118623651, 2024-04-16T17:27:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000036793 2024-04-16T17:27:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 224640 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-16T17:27:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-16T17:27:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-16T17:27:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-16T17:27:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 7889, 2024-04-16T17:27:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.220308845, 2024-04-16T17:27:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000027926 2024-04-16T17:27:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-16T17:27:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-04-16T17:27:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 0, 2024-04-16T17:27:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-16T17:27:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 0, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 507, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-04-16T17:27:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 507, 2024-04-16T17:27:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-16T17:27:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 507, 2024-04-16T17:27:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-04-16T17:27:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 0, 2024-04-16T17:27:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 0, 2024-04-16T17:27:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 4, 2024-04-16T17:27:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 0, 2024-04-16T17:27:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-16T17:27:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-16T17:27:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-04-16T17:27:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-04-16T17:27:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-16T17:27:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 0, 2024-04-16T17:27:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-16T17:27:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-04-16T17:27:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-04-16T17:27:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-04-16T17:27:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T17:27:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-16T17:27:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:27:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:27:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:27:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:27:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:27:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 0, 2024-04-16T17:27:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:27:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T17:27:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 0, 2024-04-16T17:27:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:27:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:27:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:27:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 0, 2024-04-16T17:27:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 0, 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 0, 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 0 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-04-16T17:27:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-04-16T17:27:54.518 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-04-16T17:27:54.518 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-04-16T17:27:54.518 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:27:54.518 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 0, 2024-04-16T17:27:54.518 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 0, 2024-04-16T17:27:54.518 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 0, 2024-04-16T17:27:54.518 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-16T17:27:54.518 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-16T17:27:54.519 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:27:54.519 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:27:54.519 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-16T17:27:54.519 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-16T17:27:54.519 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:27:54.519 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:27:54.519 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:27:54.519 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 0, 2024-04-16T17:27:54.520 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-16T17:27:54.521 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-16T17:27:54.521 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 3526, 2024-04-16T17:27:54.521 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-16T17:27:54.521 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-16T17:27:54.521 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 0, 2024-04-16T17:27:54.521 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-16T17:27:54.521 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-16T17:27:54.521 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 5, 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 2, 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 9722690, 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 9720322, 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.522 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.523 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.523 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 3526 2024-04-16T17:27:54.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.523 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-16T17:27:54.523 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 106, 2024-04-16T17:27:54.523 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 504, 2024-04-16T17:27:54.523 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 398, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 19, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 19, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 0, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 507, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 507, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 0, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 0, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 0, 2024-04-16T17:27:54.524 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 0, 2024-04-16T17:27:54.525 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 53288, 2024-04-16T17:27:54.525 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215416 2024-04-16T17:27:54.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.525 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-16T17:27:54.525 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 0, 2024-04-16T17:27:54.525 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-16T17:27:54.525 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 0, 2024-04-16T17:27:54.525 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 0, 2024-04-16T17:27:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-04-16T17:27:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-16T17:27:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-16T17:27:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.527 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-16T17:27:54.528 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.528 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.528 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.528 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-16T17:27:54.528 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.528 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.528 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.529 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.530 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.530 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-16T17:27:54.530 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.530 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.530 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.530 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-16T17:27:54.531 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.531 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.531 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.531 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-16T17:27:54.531 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.531 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.531 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.532 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-16T17:27:54.532 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.532 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.532 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.532 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-16T17:27:54.532 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.533 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-16T17:27:54.534 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.534 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.534 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.534 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-16T17:27:54.534 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.534 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.534 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.535 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.536 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.536 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-16T17:27:54.536 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.536 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.536 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.537 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-16T17:27:54.537 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.537 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.537 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.537 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-16T17:27:54.537 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.537 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.538 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.538 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-16T17:27:54.538 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.538 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.538 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.538 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-16T17:27:54.539 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.539 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.539 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.539 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-16T17:27:54.539 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.539 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.539 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.541 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.541 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-16T17:27:54.541 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.541 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.541 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.541 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-16T17:27:54.542 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.542 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.542 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.542 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-16T17:27:54.542 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.542 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.542 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:27:54.543 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-16T17:27:54.544 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.544 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.544 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.544 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-16T17:27:54.544 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 1, 2024-04-16T17:27:54.544 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 4, 2024-04-16T17:27:54.545 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 3, 2024-04-16T17:27:54.545 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 4, 2024-04-16T17:27:54.545 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-16T17:27:54.545 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 0, 2024-04-16T17:27:54.545 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 0, 2024-04-16T17:27:54.545 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 7860, 2024-04-16T17:27:54.545 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-16T17:27:54.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:54.546 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:54.547 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-16T17:27:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 203395, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 158, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-16T17:27:54.550 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-16T17:27:54.551 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:54.551 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:54.551 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:54.551 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:54.551 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 9688, 2024-04-16T17:27:54.551 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 185, 2024-04-16T17:27:54.551 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:54.551 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:54.552 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:54.552 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-16T17:27:54.552 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 658101, 2024-04-16T17:27:54.552 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 4505, 2024-04-16T17:27:54.552 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:54.552 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-16T17:27:54.552 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:54.552 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 4689, 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 21, 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 3, 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 4689, 2024-04-16T17:27:54.553 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-16T17:27:54.554 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4689, 2024-04-16T17:27:54.554 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 13.479850716, 2024-04-16T17:27:54.554 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002874781 2024-04-16T17:27:54.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.554 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-16T17:27:54.554 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-16T17:27:54.554 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4689, 2024-04-16T17:27:54.554 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 4690 2024-04-16T17:27:54.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.555 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 4689, 2024-04-16T17:27:54.555 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 4689, 2024-04-16T17:27:54.555 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 0, 2024-04-16T17:27:54.555 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-16T17:27:54.555 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-16T17:27:54.555 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 2919, 2024-04-16T17:27:54.555 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 0, 2024-04-16T17:27:54.556 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 1768, 2024-04-16T17:27:54.556 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 0, 2024-04-16T17:27:54.556 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 0, 2024-04-16T17:27:54.556 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-16T17:27:54.556 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-16T17:27:54.556 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 0, 2024-04-16T17:27:54.556 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-16T17:27:54.556 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 0, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 1, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 0, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:27:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-16T17:27:54.558 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-16T17:27:54.558 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-16T17:27:54.558 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:27:54.558 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-16T17:27:54.558 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:27:54.558 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-16T17:27:54.558 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-16T17:27:54.558 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-16T17:27:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 139, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 50, 2024-04-16T17:27:54.560 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 6, 2024-04-16T17:27:54.561 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 7, 2024-04-16T17:27:54.561 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 1, 2024-04-16T17:27:54.561 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-16T17:27:54.561 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 0, 2024-04-16T17:27:54.561 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 2, 2024-04-16T17:27:54.561 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 0 2024-04-16T17:27:54.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.561 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-16T17:27:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 0, 2024-04-16T17:27:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:27:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-16T17:27:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-16T17:27:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-16T17:27:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-16T17:27:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 0, 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 0, 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 0, 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-16T17:27:54.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.564 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:54.564 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:54.564 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-16T17:27:54.564 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:54.564 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 5196, 2024-04-16T17:27:54.564 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 8107495, 2024-04-16T17:27:54.564 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:54.564 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 5196, 2024-04-16T17:27:54.565 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:54.565 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:54.565 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 5196, 2024-04-16T17:27:54.565 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 8107495, 2024-04-16T17:27:54.565 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:54.565 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.565 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.565 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:54.566 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:54.567 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:54.567 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 4689, 2024-04-16T17:27:54.567 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 5528391, 2024-04-16T17:27:54.567 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 4689, 2024-04-16T17:27:54.567 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 5528391, 2024-04-16T17:27:54.567 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:54.568 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.568 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.568 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.568 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-16T17:27:54.569 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:54.569 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-16T17:27:54.569 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:54.569 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:54.569 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:54.569 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:54.569 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:54.569 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 4689, 2024-04-16T17:27:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 4689, 2024-04-16T17:27:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 4689, 2024-04-16T17:27:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 4689, 2024-04-16T17:27:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.571 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:27:54.571 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:54.571 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-16T17:27:54.571 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:54.571 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:54.571 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:54.571 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.572 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x55ed4373e6e0": { 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-04-16T17:27:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-04-16T17:27:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:54.575 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:54.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:54.575 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-16T17:27:55.331 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-16T17:27:55.331 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-16T17:27:55.331 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-16T17:27:55.332 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-16T17:27:55.332 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-16T17:27:55.332 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:27:55.332 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:27:55.332 DEBUG:tasks.check_counter:Getting stats from i 2024-04-16T17:27:55.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-16T17:27:55.587 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:27:56.250 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:56 smithi043 ceph-mon[36901]: pgmap v743: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-16T17:27:56.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:56 smithi022 ceph-mon[36023]: pgmap v743: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-16T17:27:56.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:56 smithi018 ceph-mon[31776]: pgmap v743: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-16T17:27:57.319 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-16T17:27:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 299676, 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 152458, 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 131689883, 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 30194674, 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 41, 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 7, 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 13.033145550, 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 4.621155218, 2024-04-16T17:27:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 12.066019478, 2024-04-16T17:27:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.085837145, 2024-04-16T17:27:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 152457, 2024-04-16T17:27:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 2.744479596, 2024-04-16T17:27:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000018001 2024-04-16T17:27:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 445401, 2024-04-16T17:27:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.053846026, 2024-04-16T17:27:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000120 2024-04-16T17:27:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1768464, 2024-04-16T17:27:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 294752 2024-04-16T17:27:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:27:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 31168, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 33292, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 9436390, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 9267346, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 17, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 6, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 2.186873099, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.765603569, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 1.841699163, 2024-04-16T17:27:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.023486772, 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 33289, 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.742015205, 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000022290 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 60076, 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.011861731, 2024-04-16T17:27:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000197 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 5696, 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 818640 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 18687, 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 9759, 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 5450540, 2024-04-16T17:27:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 2442580, 2024-04-16T17:27:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 62, 2024-04-16T17:27:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 5, 2024-04-16T17:27:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.931388099, 2024-04-16T17:27:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.317246417, 2024-04-16T17:27:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.826318003, 2024-04-16T17:27:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.019916229, 2024-04-16T17:27:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:27:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 9757, 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.202432347, 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000020747 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 26909, 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.004118580, 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000153 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3440, 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 5648 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-16T17:27:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-16T17:27:57.329 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3412, 2024-04-16T17:27:57.329 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.123681310, 2024-04-16T17:27:57.329 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000329332 2024-04-16T17:27:57.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.329 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-16T17:27:57.329 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-16T17:27:57.329 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1328, 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.065505986, 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000049326 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 28005, 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 2182, 2024-04-16T17:27:57.330 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-16T17:27:57.331 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2182, 2024-04-16T17:27:57.331 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 15.712748920, 2024-04-16T17:27:57.331 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.007201076 2024-04-16T17:27:57.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.331 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-16T17:27:57.331 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 25823, 2024-04-16T17:27:57.331 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-16T17:27:57.331 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-16T17:27:57.332 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 11, 2024-04-16T17:27:57.332 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-16T17:27:57.332 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-16T17:27:57.332 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 224, 2024-04-16T17:27:57.332 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 65, 2024-04-16T17:27:57.332 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 159, 2024-04-16T17:27:57.332 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-16T17:27:57.332 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 131, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 1, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 7, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 7, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 5, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 31829, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 3014, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 25821, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 115, 2024-04-16T17:27:57.333 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:27:57.334 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-16T17:27:57.334 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 132, 2024-04-16T17:27:57.334 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-16T17:27:57.334 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-16T17:27:57.334 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-16T17:27:57.334 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-16T17:27:57.334 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 2, 2024-04-16T17:27:57.334 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 14, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 148, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 762, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 226530171, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 6, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 148, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-16T17:27:57.335 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:27:57.336 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:27:57.336 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:27:57.336 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:27:57.336 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:27:57.336 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 148, 2024-04-16T17:27:57.336 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:27:57.336 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 67, 2024-04-16T17:27:57.336 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 1657, 2024-04-16T17:27:57.337 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:27:57.337 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:27:57.337 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:27:57.337 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 1739, 2024-04-16T17:27:57.337 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 1185, 2024-04-16T17:27:57.337 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 172, 2024-04-16T17:27:57.337 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 2180 2024-04-16T17:27:57.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-16T17:27:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 47, 2024-04-16T17:27:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 570, 2024-04-16T17:27:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 336, 2024-04-16T17:27:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 494, 2024-04-16T17:27:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 50, 2024-04-16T17:27:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-04-16T17:27:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 398, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 398, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 0, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:27:57.339 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-16T17:27:57.340 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-16T17:27:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-16T17:27:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-16T17:27:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 3527, 2024-04-16T17:27:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-16T17:27:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-16T17:27:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 3527, 2024-04-16T17:27:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-16T17:27:57.342 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-16T17:27:57.342 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 5, 2024-04-16T17:27:57.342 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-16T17:27:57.342 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-16T17:27:57.342 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 5, 2024-04-16T17:27:57.342 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 2, 2024-04-16T17:27:57.342 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-16T17:27:57.342 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-16T17:27:57.343 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-16T17:27:57.343 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 9725046, 2024-04-16T17:27:57.343 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 0, 2024-04-16T17:27:57.343 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-16T17:27:57.343 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1498, 2024-04-16T17:27:57.343 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 16.407267036, 2024-04-16T17:27:57.343 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.010952781 2024-04-16T17:27:57.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 0 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 218, 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 616, 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 398, 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 259, 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 428, 2024-04-16T17:27:57.344 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 169, 2024-04-16T17:27:57.345 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 225, 2024-04-16T17:27:57.345 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 682, 2024-04-16T17:27:57.345 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 457, 2024-04-16T17:27:57.345 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 6, 2024-04-16T17:27:57.345 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 445, 2024-04-16T17:27:57.345 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 439, 2024-04-16T17:27:57.345 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 61768, 2024-04-16T17:27:57.345 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 223608 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 30308, 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 28005, 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 227, 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 928, 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-16T17:27:57.346 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 396, 2024-04-16T17:27:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 6.820474890, 2024-04-16T17:27:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.017223421 2024-04-16T17:27:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-16T17:27:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 20, 2024-04-16T17:27:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.032628682, 2024-04-16T17:27:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001631434 2024-04-16T17:27:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.348 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-16T17:27:57.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.348 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.348 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-16T17:27:57.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.348 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.349 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-16T17:27:57.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 644, 2024-04-16T17:27:57.349 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.491585419, 2024-04-16T17:27:57.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002316126 2024-04-16T17:27:57.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.349 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.351 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-16T17:27:57.351 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.351 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.351 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.351 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-16T17:27:57.351 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.351 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.352 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.352 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-16T17:27:57.352 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.352 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.352 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.352 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-16T17:27:57.353 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.353 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.353 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.354 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-16T17:27:57.354 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.354 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.354 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.354 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-16T17:27:57.354 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.355 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.355 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.355 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-16T17:27:57.355 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.355 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.355 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.356 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-16T17:27:57.356 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.356 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.356 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.356 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-16T17:27:57.356 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 591, 2024-04-16T17:27:57.356 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.413571750, 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000699782 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 132, 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 3.031762708, 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.022967899 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-16T17:27:57.357 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.358 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.358 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.358 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-16T17:27:57.358 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.358 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.358 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.359 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-16T17:27:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-16T17:27:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-16T17:27:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.001428567, 2024-04-16T17:27:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001428567 2024-04-16T17:27:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-16T17:27:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-16T17:27:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-16T17:27:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.362 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-16T17:27:57.363 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 398, 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 3.921296904, 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.009852504 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.364 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:27:57.365 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:27:57.365 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-16T17:27:57.365 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.365 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.365 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.365 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-16T17:27:57.366 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 1, 2024-04-16T17:27:57.366 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 4, 2024-04-16T17:27:57.366 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 3, 2024-04-16T17:27:57.366 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 1, 2024-04-16T17:27:57.366 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-16T17:27:57.366 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 249, 2024-04-16T17:27:57.366 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 249, 2024-04-16T17:27:57.366 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 1322, 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:27:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:27:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:27:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-16T17:27:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:27:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-16T17:27:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:27:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:27:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:27:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:27:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-16T17:27:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-16T17:27:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:27:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:27:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:27:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:27:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 319814, 2024-04-16T17:27:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 271, 2024-04-16T17:27:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:27:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 9688, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 185, 2024-04-16T17:27:57.372 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 1044290, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 6108, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-16T17:27:57.373 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-16T17:27:57.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.374 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-16T17:27:57.374 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-16T17:27:57.374 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-16T17:27:57.374 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 3196, 2024-04-16T17:27:57.374 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 5651933, 2024-04-16T17:27:57.374 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 28, 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 3196, 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3196, 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 68.235246274, 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.021350202 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-16T17:27:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3196, 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 3523 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 3196, 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 316, 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 2880, 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 0, 2024-04-16T17:27:57.376 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 10, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 168, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 1666, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 139, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 0, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 809, 2024-04-16T17:27:57.377 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-16T17:27:57.378 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-16T17:27:57.378 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-16T17:27:57.378 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 148, 2024-04-16T17:27:57.378 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 20, 2024-04-16T17:27:57.378 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:27:57.378 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-16T17:27:57.378 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:27:57.378 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-16T17:27:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-16T17:27:57.380 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-16T17:27:57.380 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-16T17:27:57.380 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-16T17:27:57.380 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-16T17:27:57.380 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-16T17:27:57.380 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-16T17:27:57.380 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-16T17:27:57.380 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 139, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 50, 2024-04-16T17:27:57.381 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 12, 2024-04-16T17:27:57.382 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 15, 2024-04-16T17:27:57.382 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 3, 2024-04-16T17:27:57.382 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-16T17:27:57.382 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 405, 2024-04-16T17:27:57.382 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 0, 2024-04-16T17:27:57.382 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 158 2024-04-16T17:27:57.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.382 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 1, 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-16T17:27:57.383 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 16, 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 8, 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 796, 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 398, 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:27:57.384 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 349527, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 115087070, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 349527, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 349527, 2024-04-16T17:27:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 115087070, 2024-04-16T17:27:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 3196, 2024-04-16T17:27:57.387 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 5695467, 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 3196, 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 5695467, 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.388 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 3196, 2024-04-16T17:27:57.389 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 3196, 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 3196, 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 3196, 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 398, 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 43534, 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 398, 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:57.391 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 168, 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 43534, 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:27:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x563f0b288460": { 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 2356, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 3527, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 5530742, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 3527, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:27:57.393 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 1498, 2024-04-16T17:27:57.394 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 5528386, 2024-04-16T17:27:57.394 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:27:57.394 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:27:57.394 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:27:57.394 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:27:57.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:27:57.394 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-16T17:27:58.264 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-16T17:27:58.265 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-04-16T17:27:58.265 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=47 2024-04-16T17:27:58.265 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=570 2024-04-16T17:27:58.265 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-16T17:27:58.265 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:27:58.265 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:27:58.265 DEBUG:tasks.check_counter:Getting stats from l 2024-04-16T17:27:58.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid 872930f8-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-16T17:27:58.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:27:58 smithi022 ceph-mon[36023]: pgmap v744: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-04-16T17:27:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:27:58 smithi018 ceph-mon[31776]: pgmap v744: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-04-16T17:27:58.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:27:58 smithi043 ceph-mon[36901]: pgmap v744: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-04-16T17:27:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:28:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:00 smithi018 ceph-mon[31776]: pgmap v745: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-04-16T17:28:00.294 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:00 smithi043 ceph-mon[36901]: pgmap v745: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-04-16T17:28:00.317 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-16T17:28:00.318 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:28:00.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 0, 2024-04-16T17:28:00.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2, 2024-04-16T17:28:00.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:28:00.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 0, 2024-04-16T17:28:00.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-16T17:28:00.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.001166940, 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000112802, 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:28:00.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 3, 2024-04-16T17:28:00.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 334, 2024-04-16T17:28:00.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 480, 2024-04-16T17:28:00.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 372048, 2024-04-16T17:28:00.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 7, 2024-04-16T17:28:00.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-16T17:28:00.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.019774919, 2024-04-16T17:28:00.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.014009828, 2024-04-16T17:28:00.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.001925815, 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 331, 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.042298563, 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000127790 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:28:00.322 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 372048 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 449, 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 436, 2024-04-16T17:28:00.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1657184, 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 269360, 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.073345079, 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.024033260, 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.044336370, 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.010454110, 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:28:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 436, 2024-04-16T17:28:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.019667651, 2024-04-16T17:28:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000045109 2024-04-16T17:28:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:28:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:28:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:28:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:28:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1657184, 2024-04-16T17:28:00.326 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 269360 2024-04-16T17:28:00.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.326 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-16T17:28:00.326 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-04-16T17:28:00.326 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-16T17:28:00.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.326 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:28:00.327 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-16T17:28:00.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:28:00.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 148692, 2024-04-16T17:28:00.330 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 30, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-16T17:28:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-16T17:28:00.332 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 448, 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1591242, 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-16T17:28:00.333 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 448, 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 448, 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1591242, 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-16T17:28:00.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:28:00.335 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-16T17:28:00.335 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-16T17:28:00.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:00 smithi022 ceph-mon[36023]: pgmap v745: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-04-16T17:28:01.036 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-16T17:28:01.037 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-16T17:28:01.037 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-16T17:28:01.037 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-16T17:28:01.037 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-16T17:28:01.037 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-16T17:28:01.037 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-16T17:28:01.038 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-16T17:28:01.048 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-16T17:28:01.049 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T17:28:01.049 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:28:01.104 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-16T17:28:01.104 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:28:01.148 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:28:01.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 -- 172.21.15.18:0/1399267443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0102150 msgr2=0x7f0de0102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:01.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 --2- 172.21.15.18:0/1399267443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0102150 0x7f0de0102530 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009900 tx=0x7f0dd002bc30 comp rx=0 tx=0).stop 2024-04-16T17:28:01.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 -- 172.21.15.18:0/1399267443 shutdown_connections 2024-04-16T17:28:01.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 --2- 172.21.15.18:0/1399267443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de0107220 0x7f0de010edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 --2- 172.21.15.18:0/1399267443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de0102b00 0x7f0de0106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 --2- 172.21.15.18:0/1399267443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0102150 0x7f0de0102530 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 -- 172.21.15.18:0/1399267443 >> 172.21.15.18:0/1399267443 conn(0x7f0de00fd9f0 msgr2=0x7f0de00ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:01.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 -- 172.21.15.18:0/1399267443 shutdown_connections 2024-04-16T17:28:01.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.396+0000 7f0de6200640 1 -- 172.21.15.18:0/1399267443 wait complete. 2024-04-16T17:28:01.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.397+0000 7f0de6200640 1 Processor -- start 2024-04-16T17:28:01.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.397+0000 7f0de6200640 1 -- start start 2024-04-16T17:28:01.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de0102150 0x7f0de01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:01.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de6200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0107220 0x7f0de01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:01.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de019a880 0x7f0de019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:01.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de6200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0de0112320 con 0x7f0de0107220 2024-04-16T17:28:01.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de6200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0de01121a0 con 0x7f0de0102150 2024-04-16T17:28:01.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de6200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0de01124a0 con 0x7f0de019a880 2024-04-16T17:28:01.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0dd7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0107220 0x7f0de01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:01.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0dd7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0107220 0x7f0de01a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59072/0 (socket says 172.21.15.18:59072) 2024-04-16T17:28:01.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0dd7e00640 1 -- 172.21.15.18:0/1460777636 learned_addr learned my addr 172.21.15.18:0/1460777636 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:01.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de5800640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de019a880 0x7f0de019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:01.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de4e00640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de0102150 0x7f0de01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:01.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de5800640 1 -- 172.21.15.18:0/1460777636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de0102150 msgr2=0x7f0de01a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:01.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de5800640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de0102150 0x7f0de01a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de5800640 1 -- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0107220 msgr2=0x7f0de01a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de5800640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0107220 0x7f0de01a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.398+0000 7f0de5800640 1 -- 172.21.15.18:0/1460777636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de019b3a0 con 0x7f0de019a880 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.399+0000 7f0de5800640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de019a880 0x7f0de019ad30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800ec50 tx=0x7f0dd800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.399+0000 7f0dde000640 1 -- 172.21.15.18:0/1460777636 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8010630 con 0x7f0de019a880 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.399+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de019b690 con 0x7f0de019a880 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.399+0000 7f0dde000640 1 -- 172.21.15.18:0/1460777636 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd8010cb0 con 0x7f0de019a880 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.399+0000 7f0dde000640 1 -- 172.21.15.18:0/1460777636 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8018ce0 con 0x7f0de019a880 2024-04-16T17:28:01.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.399+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de01a78e0 con 0x7f0de019a880 2024-04-16T17:28:01.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.400+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de004fe00 con 0x7f0de019a880 2024-04-16T17:28:01.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.402+0000 7f0dde000640 1 -- 172.21.15.18:0/1460777636 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0dd801f020 con 0x7f0de019a880 2024-04-16T17:28:01.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.404+0000 7f0dde000640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dc40a3d10 0x7f0dc40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:01.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.404+0000 7f0dde000640 1 -- 172.21.15.18:0/1460777636 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7f0dd80d0da0 con 0x7f0de019a880 2024-04-16T17:28:01.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.404+0000 7f0de4e00640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dc40a3d10 0x7f0dc40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:01.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.405+0000 7f0de4e00640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dc40a3d10 0x7f0dc40a61d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0000c00 tx=0x7f0dd002df90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:01.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.408+0000 7f0dde000640 1 -- 172.21.15.18:0/1460777636 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0dd809cfe0 con 0x7f0de019a880 2024-04-16T17:28:01.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.626+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0de0102b00 con 0x7f0de019a880 2024-04-16T17:28:01.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.628+0000 7f0dde000640 1 -- 172.21.15.18:0/1460777636 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+25290 (secure 0 0 0) 0x7f0dd809c8e0 con 0x7f0de019a880 2024-04-16T17:28:01.629 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:28:01.629 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":139,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:27:23.843667+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":41,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":41,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3045212308":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/3152065578":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/1033276779":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6801/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/2149549301":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":41,"length":1}]},{"pool":4,"snaps":[{"begin":28,"length":1},{"begin":41,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:28:01.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dc40a3d10 msgr2=0x7f0dc40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:01.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dc40a3d10 0x7f0dc40a61d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0000c00 tx=0x7f0dd002df90 comp rx=0 tx=0).stop 2024-04-16T17:28:01.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de019a880 msgr2=0x7f0de019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:01.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de019a880 0x7f0de019ad30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800ec50 tx=0x7f0dd800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 shutdown_connections 2024-04-16T17:28:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0dc40a3d10 0x7f0dc40a61d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de019a880 0x7f0de019ad30 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0de0107220 0x7f0de01a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 --2- 172.21.15.18:0/1460777636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de0102150 0x7f0de01a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 >> 172.21.15.18:0/1460777636 conn(0x7f0de00fd9f0 msgr2=0x7f0de00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 shutdown_connections 2024-04-16T17:28:01.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.631+0000 7f0de6200640 1 -- 172.21.15.18:0/1460777636 wait complete. 2024-04-16T17:28:01.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:56:08.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:01:21.974887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:01:23.365582+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 137, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:01:26.987830+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 137, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:28:01.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:28:01.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 -- 172.21.15.18:0/335204453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90104e00 msgr2=0x7f0f90108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:01.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 --2- 172.21.15.18:0/335204453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90104e00 0x7f0f90108e40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009900 tx=0x7f0f7802bc40 comp rx=0 tx=0).stop 2024-04-16T17:28:01.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 -- 172.21.15.18:0/335204453 shutdown_connections 2024-04-16T17:28:01.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 --2- 172.21.15.18:0/335204453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f90109520 0x7f0f901110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 --2- 172.21.15.18:0/335204453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90104e00 0x7f0f90108e40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 --2- 172.21.15.18:0/335204453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f90104450 0x7f0f90104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 -- 172.21.15.18:0/335204453 >> 172.21.15.18:0/335204453 conn(0x7f0f900ffcb0 msgr2=0x7f0f901020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:01.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 -- 172.21.15.18:0/335204453 shutdown_connections 2024-04-16T17:28:01.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.861+0000 7f0f96200640 1 -- 172.21.15.18:0/335204453 wait complete. 2024-04-16T17:28:01.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.862+0000 7f0f96200640 1 Processor -- start 2024-04-16T17:28:01.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.862+0000 7f0f96200640 1 -- start start 2024-04-16T17:28:01.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.862+0000 7f0f96200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f90104450 0x7f0f901a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:01.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.862+0000 7f0f96200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90109520 0x7f0f901a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.862+0000 7f0f96200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f9019a910 0x7f0f9019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.862+0000 7f0f96200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90114590 con 0x7f0f90109520 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.862+0000 7f0f96200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90114410 con 0x7f0f9019a910 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.862+0000 7f0f96200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90114710 con 0x7f0f90104450 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f86a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90109520 0x7f0f901a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f9019a910 0x7f0f9019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f9019a910 0x7f0f9019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49064/0 (socket says 172.21.15.18:49064) 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8fe00640 1 -- 172.21.15.18:0/2955356353 learned_addr learned my addr 172.21.15.18:0/2955356353 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:01.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8f400640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f90104450 0x7f0f901a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8fe00640 1 -- 172.21.15.18:0/2955356353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f90104450 msgr2=0x7f0f901a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8fe00640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f90104450 0x7f0f901a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8fe00640 1 -- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90109520 msgr2=0x7f0f901a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8fe00640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90109520 0x7f0f901a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.863+0000 7f0f8fe00640 1 -- 172.21.15.18:0/2955356353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9019b430 con 0x7f0f9019a910 2024-04-16T17:28:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.864+0000 7f0f8fe00640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f9019a910 0x7f0f9019adc0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800d750 tx=0x7f0f8800dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.864+0000 7f0f8cc00640 1 -- 172.21.15.18:0/2955356353 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88017070 con 0x7f0f9019a910 2024-04-16T17:28:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.864+0000 7f0f8cc00640 1 -- 172.21.15.18:0/2955356353 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f88005500 con 0x7f0f9019a910 2024-04-16T17:28:01.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.864+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9019b720 con 0x7f0f9019a910 2024-04-16T17:28:01.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.864+0000 7f0f8cc00640 1 -- 172.21.15.18:0/2955356353 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88004d90 con 0x7f0f9019a910 2024-04-16T17:28:01.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.864+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f901a7920 con 0x7f0f9019a910 2024-04-16T17:28:01.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.865+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f90069300 con 0x7f0f9019a910 2024-04-16T17:28:01.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.867+0000 7f0f8cc00640 1 -- 172.21.15.18:0/2955356353 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f88004f30 con 0x7f0f9019a910 2024-04-16T17:28:01.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.869+0000 7f0f8cc00640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f740a3d20 0x7f0f740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:01.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.869+0000 7f0f8cc00640 1 -- 172.21.15.18:0/2955356353 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7f0f880d0930 con 0x7f0f9019a910 2024-04-16T17:28:01.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.869+0000 7f0f8f400640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f740a3d20 0x7f0f740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:01.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.870+0000 7f0f8f400640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f740a3d20 0x7f0f740a61e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0f9019ba50 tx=0x7f0f80009500 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:01.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:01.871+0000 7f0f8cc00640 1 -- 172.21.15.18:0/2955356353 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f8809c520 con 0x7f0f9019a910 2024-04-16T17:28:02.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.083+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f90104e00 con 0x7f0f9019a910 2024-04-16T17:28:02.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.083+0000 7f0f8cc00640 1 -- 172.21.15.18:0/2955356353 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) ==== 93+0+10 (secure 0 0 0) 0x7f0f880a0820 con 0x7f0f9019a910 2024-04-16T17:28:02.083 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-16T17:28:02.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f740a3d20 msgr2=0x7f0f740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f740a3d20 0x7f0f740a61e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0f9019ba50 tx=0x7f0f80009500 comp rx=0 tx=0).stop 2024-04-16T17:28:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f9019a910 msgr2=0x7f0f9019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f9019a910 0x7f0f9019adc0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800d750 tx=0x7f0f8800dc10 comp rx=0 tx=0).stop 2024-04-16T17:28:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 shutdown_connections 2024-04-16T17:28:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f0f740a3d20 0x7f0f740a61e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f9019a910 0x7f0f9019adc0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f90109520 0x7f0f901a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 --2- 172.21.15.18:0/2955356353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f90104450 0x7f0f901a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 >> 172.21.15.18:0/2955356353 conn(0x7f0f900ffcb0 msgr2=0x7f0f90100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.086+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 shutdown_connections 2024-04-16T17:28:02.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.087+0000 7f0f96200640 1 -- 172.21.15.18:0/2955356353 wait complete. 2024-04-16T17:28:02.104 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:28:02.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.311+0000 7fb770c00640 1 -- 172.21.15.18:0/1404356176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c0743a0 msgr2=0x7fb76c074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.311+0000 7fb770c00640 1 --2- 172.21.15.18:0/1404356176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c0743a0 0x7fb76c074780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009900 tx=0x7fb75c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.312+0000 7fb770c00640 1 -- 172.21.15.18:0/1404356176 shutdown_connections 2024-04-16T17:28:02.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.312+0000 7fb770c00640 1 --2- 172.21.15.18:0/1404356176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb76c112e10 0x7fb76c115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.312+0000 7fb770c00640 1 --2- 172.21.15.18:0/1404356176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb76c074cc0 0x7fb76c1128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.312+0000 7fb770c00640 1 --2- 172.21.15.18:0/1404356176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c0743a0 0x7fb76c074780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.312+0000 7fb770c00640 1 -- 172.21.15.18:0/1404356176 >> 172.21.15.18:0/1404356176 conn(0x7fb76c0fd950 msgr2=0x7fb76c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:02.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.312+0000 7fb770c00640 1 -- 172.21.15.18:0/1404356176 shutdown_connections 2024-04-16T17:28:02.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.312+0000 7fb770c00640 1 -- 172.21.15.18:0/1404356176 wait complete. 2024-04-16T17:28:02.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.313+0000 7fb770c00640 1 Processor -- start 2024-04-16T17:28:02.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.313+0000 7fb770c00640 1 -- start start 2024-04-16T17:28:02.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.313+0000 7fb770c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb76c0743a0 0x7fb76c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:02.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.313+0000 7fb770c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c074cc0 0x7fb76c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:02.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.313+0000 7fb770c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb76c112e10 0x7fb76c1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:02.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.313+0000 7fb770c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c118830 con 0x7fb76c074cc0 2024-04-16T17:28:02.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.313+0000 7fb770c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1186b0 con 0x7fb76c112e10 2024-04-16T17:28:02.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.313+0000 7fb770c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1189b0 con 0x7fb76c0743a0 2024-04-16T17:28:02.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c074cc0 0x7fb76c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:02.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb76c0743a0 0x7fb76c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:02.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c074cc0 0x7fb76c19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59096/0 (socket says 172.21.15.18:59096) 2024-04-16T17:28:02.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 -- 172.21.15.18:0/1656776723 learned_addr learned my addr 172.21.15.18:0/1656776723 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:02.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76be00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb76c112e10 0x7fb76c1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:02.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 -- 172.21.15.18:0/1656776723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb76c0743a0 msgr2=0x7fb76c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb76c0743a0 0x7fb76c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 -- 172.21.15.18:0/1656776723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb76c112e10 msgr2=0x7fb76c1a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb76c112e10 0x7fb76c1a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 -- 172.21.15.18:0/1656776723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c1a3d60 con 0x7fb76c074cc0 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.314+0000 7fb76aa00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c074cc0 0x7fb76c19f2b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb75800e9f0 tx=0x7fb75800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.315+0000 7fb763e00640 1 -- 172.21.15.18:0/1656776723 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75800ccc0 con 0x7fb76c074cc0 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.315+0000 7fb763e00640 1 -- 172.21.15.18:0/1656776723 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb758010910 con 0x7fb76c074cc0 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.315+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c1a4050 con 0x7fb76c074cc0 2024-04-16T17:28:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.315+0000 7fb763e00640 1 -- 172.21.15.18:0/1656776723 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580188a0 con 0x7fb76c074cc0 2024-04-16T17:28:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.315+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c1ab8e0 con 0x7fb76c074cc0 2024-04-16T17:28:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.316+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c04fd90 con 0x7fb76c074cc0 2024-04-16T17:28:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.319+0000 7fb763e00640 1 -- 172.21.15.18:0/1656776723 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb758010ac0 con 0x7fb76c074cc0 2024-04-16T17:28:02.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.321+0000 7fb763e00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb7500a3da0 0x7fb7500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.321+0000 7fb763e00640 1 -- 172.21.15.18:0/1656776723 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7fb7580d0fe0 con 0x7fb76c074cc0 2024-04-16T17:28:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.321+0000 7fb76b400640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb7500a3da0 0x7fb7500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.322+0000 7fb76b400640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb7500a3da0 0x7fb7500a6260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005ab0 tx=0x7fb75c005a20 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:02.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.326+0000 7fb763e00640 1 -- 172.21.15.18:0/1656776723 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb75809d220 con 0x7fb76c074cc0 2024-04-16T17:28:02.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:02 smithi022 ceph-mon[36023]: pgmap v746: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 10 op/s 2024-04-16T17:28:02.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:02 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1460777636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:28:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:02 smithi018 ceph-mon[31776]: pgmap v746: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 10 op/s 2024-04-16T17:28:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:02 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1460777636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:28:02.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:02 smithi043 ceph-mon[36901]: pgmap v746: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 10 op/s 2024-04-16T17:28:02.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:02 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1460777636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:28:02.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.536+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb76c02d620 con 0x7fb76c074cc0 2024-04-16T17:28:02.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.537+0000 7fb763e00640 1 -- 172.21.15.18:0/1656776723 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) ==== 104+0+11 (secure 0 0 0) 0x7fb75809cb20 con 0x7fb76c074cc0 2024-04-16T17:28:02.537 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:28:02.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb7500a3da0 msgr2=0x7fb7500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb7500a3da0 0x7fb7500a6260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005ab0 tx=0x7fb75c005a20 comp rx=0 tx=0).stop 2024-04-16T17:28:02.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c074cc0 msgr2=0x7fb76c19f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c074cc0 0x7fb76c19f2b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb75800e9f0 tx=0x7fb75800edf0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 shutdown_connections 2024-04-16T17:28:02.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fb7500a3da0 0x7fb7500a6260 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb76c112e10 0x7fb76c1a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb76c074cc0 0x7fb76c19f2b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 --2- 172.21.15.18:0/1656776723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb76c0743a0 0x7fb76c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.540+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 >> 172.21.15.18:0/1656776723 conn(0x7fb76c0fd950 msgr2=0x7fb76c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:02.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.541+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 shutdown_connections 2024-04-16T17:28:02.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.541+0000 7fb770c00640 1 -- 172.21.15.18:0/1656776723 wait complete. 2024-04-16T17:28:02.560 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:28:02.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.772+0000 7fc337e00640 1 -- 172.21.15.18:0/2536308701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3381016f0 msgr2=0x7fc338105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.772+0000 7fc337e00640 1 --2- 172.21.15.18:0/2536308701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3381016f0 0x7fc338105bb0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc320009900 tx=0x7fc32002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.772+0000 7fc337e00640 1 -- 172.21.15.18:0/2536308701 shutdown_connections 2024-04-16T17:28:02.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.772+0000 7fc337e00640 1 --2- 172.21.15.18:0/2536308701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3381061d0 0x7fc338106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.772+0000 7fc337e00640 1 --2- 172.21.15.18:0/2536308701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3381016f0 0x7fc338105bb0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.772+0000 7fc337e00640 1 --2- 172.21.15.18:0/2536308701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc338100dd0 0x7fc3381011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.772+0000 7fc337e00640 1 -- 172.21.15.18:0/2536308701 >> 172.21.15.18:0/2536308701 conn(0x7fc338077ab0 msgr2=0x7fc3380fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:02.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.772+0000 7fc337e00640 1 -- 172.21.15.18:0/2536308701 shutdown_connections 2024-04-16T17:28:02.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.773+0000 7fc337e00640 1 -- 172.21.15.18:0/2536308701 wait complete. 2024-04-16T17:28:02.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.773+0000 7fc337e00640 1 Processor -- start 2024-04-16T17:28:02.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc337e00640 1 -- start start 2024-04-16T17:28:02.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc337e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc338100dd0 0x7fc33819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc337e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3381016f0 0x7fc33819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc337e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3381061d0 0x7fc3381a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc337e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381187c0 con 0x7fc3381061d0 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc337e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc338118640 con 0x7fc3381016f0 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc337e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc338118940 con 0x7fc338100dd0 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc336a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc338100dd0 0x7fc33819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc337400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3381061d0 0x7fc3381a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc32fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3381016f0 0x7fc33819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:02.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc336a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc338100dd0 0x7fc33819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34426/0 (socket says 172.21.15.18:34426) 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.774+0000 7fc336a00640 1 -- 172.21.15.18:0/1490268940 learned_addr learned my addr 172.21.15.18:0/1490268940 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc336a00640 1 -- 172.21.15.18:0/1490268940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3381016f0 msgr2=0x7fc33819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc336a00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3381016f0 0x7fc33819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc336a00640 1 -- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3381061d0 msgr2=0x7fc3381a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc336a00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3381061d0 0x7fc3381a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc336a00640 1 -- 172.21.15.18:0/1490268940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3381a3d50 con 0x7fc338100dd0 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc336a00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc338100dd0 0x7fc33819ed60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc32800e9f0 tx=0x7fc32800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc32f400640 1 -- 172.21.15.18:0/1490268940 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32800cd80 con 0x7fc338100dd0 2024-04-16T17:28:02.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3381a4040 con 0x7fc338100dd0 2024-04-16T17:28:02.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc32f400640 1 -- 172.21.15.18:0/1490268940 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc328005500 con 0x7fc338100dd0 2024-04-16T17:28:02.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc32f400640 1 -- 172.21.15.18:0/1490268940 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328010690 con 0x7fc338100dd0 2024-04-16T17:28:02.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.775+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381ab8d0 con 0x7fc338100dd0 2024-04-16T17:28:02.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.776+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33804fd90 con 0x7fc338100dd0 2024-04-16T17:28:02.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.779+0000 7fc32f400640 1 -- 172.21.15.18:0/1490268940 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3280046c0 con 0x7fc338100dd0 2024-04-16T17:28:02.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.780+0000 7fc32f400640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc31c0a3d20 0x7fc31c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:02.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.780+0000 7fc32f400640 1 -- 172.21.15.18:0/1490268940 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7fc3280d0ff0 con 0x7fc338100dd0 2024-04-16T17:28:02.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.780+0000 7fc32fe00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc31c0a3d20 0x7fc31c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:02.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.781+0000 7fc32fe00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc31c0a3d20 0x7fc31c0a61e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc320006010 tx=0x7fc320005dd0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:02.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.784+0000 7fc32f400640 1 -- 172.21.15.18:0/1490268940 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc32809d290 con 0x7fc338100dd0 2024-04-16T17:28:02.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.991+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3381abb00 con 0x7fc338100dd0 2024-04-16T17:28:02.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.992+0000 7fc32f400640 1 -- 172.21.15.18:0/1490268940 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) ==== 100+0+11 (secure 0 0 0) 0x7fc32809cb90 con 0x7fc338100dd0 2024-04-16T17:28:02.992 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 64 2024-04-16T17:28:02.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc31c0a3d20 msgr2=0x7fc31c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc31c0a3d20 0x7fc31c0a61e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc320006010 tx=0x7fc320005dd0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc338100dd0 msgr2=0x7fc33819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:02.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc338100dd0 0x7fc33819ed60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc32800e9f0 tx=0x7fc32800edf0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 shutdown_connections 2024-04-16T17:28:02.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc31c0a3d20 0x7fc31c0a61e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3381061d0 0x7fc3381a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3381016f0 0x7fc33819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 --2- 172.21.15.18:0/1490268940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc338100dd0 0x7fc33819ed60 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:02.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 >> 172.21.15.18:0/1490268940 conn(0x7fc338077ab0 msgr2=0x7fc338103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:02.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 shutdown_connections 2024-04-16T17:28:02.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:02.995+0000 7fc337e00640 1 -- 172.21.15.18:0/1490268940 wait complete. 2024-04-16T17:28:03.013 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:28:03.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 -- 172.21.15.18:0/2463555720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc104df0 msgr2=0x7fe4cc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/2463555720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc104df0 0x7fe4cc108e30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009860 tx=0x7fe4b802bb70 comp rx=0 tx=0).stop 2024-04-16T17:28:03.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 -- 172.21.15.18:0/2463555720 shutdown_connections 2024-04-16T17:28:03.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/2463555720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4cc109510 0x7fe4cc1110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/2463555720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc104df0 0x7fe4cc108e30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/2463555720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc104440 0x7fe4cc104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 -- 172.21.15.18:0/2463555720 >> 172.21.15.18:0/2463555720 conn(0x7fe4cc0ffc80 msgr2=0x7fe4cc1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:03.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 -- 172.21.15.18:0/2463555720 shutdown_connections 2024-04-16T17:28:03.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.222+0000 7fe4d2a00640 1 -- 172.21.15.18:0/2463555720 wait complete. 2024-04-16T17:28:03.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.223+0000 7fe4d2a00640 1 Processor -- start 2024-04-16T17:28:03.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.223+0000 7fe4d2a00640 1 -- start start 2024-04-16T17:28:03.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.223+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4cc104440 0x7fe4cc1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:03.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.223+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc109510 0x7fe4cc1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc19a920 0x7fe4cc19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4d2a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc114580 con 0x7fe4cc109510 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4d2a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc114400 con 0x7fe4cc104440 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4d2a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc114700 con 0x7fe4cc19a920 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4cc104440 0x7fe4cc1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cb400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc109510 0x7fe4cc1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4cc104440 0x7fe4cc1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49114/0 (socket says 172.21.15.18:49114) 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cbe00640 1 -- 172.21.15.18:0/476606578 learned_addr learned my addr 172.21.15.18:0/476606578 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:03.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4d0a00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc19a920 0x7fe4cc19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:03.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cbe00640 1 -- 172.21.15.18:0/476606578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc19a920 msgr2=0x7fe4cc19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cbe00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc19a920 0x7fe4cc19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cbe00640 1 -- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc109510 msgr2=0x7fe4cc1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cbe00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc109510 0x7fe4cc1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cbe00640 1 -- 172.21.15.18:0/476606578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc19b440 con 0x7fe4cc104440 2024-04-16T17:28:03.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.224+0000 7fe4cb400640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc109510 0x7fe4cc1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:28:03.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.225+0000 7fe4cbe00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4cc104440 0x7fe4cc1a0770 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00e9c0 tx=0x7fe4bc00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:03.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.225+0000 7fe4c8c00640 1 -- 172.21.15.18:0/476606578 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00ccc0 con 0x7fe4cc104440 2024-04-16T17:28:03.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.225+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc19b730 con 0x7fe4cc104440 2024-04-16T17:28:03.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.225+0000 7fe4c8c00640 1 -- 172.21.15.18:0/476606578 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc005500 con 0x7fe4cc104440 2024-04-16T17:28:03.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.225+0000 7fe4c8c00640 1 -- 172.21.15.18:0/476606578 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc010430 con 0x7fe4cc104440 2024-04-16T17:28:03.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.225+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1a79d0 con 0x7fe4cc104440 2024-04-16T17:28:03.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.226+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc19be50 con 0x7fe4cc104440 2024-04-16T17:28:03.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.228+0000 7fe4c8c00640 1 -- 172.21.15.18:0/476606578 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4bc0046c0 con 0x7fe4cc104440 2024-04-16T17:28:03.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.229+0000 7fe4c8c00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4b00a3da0 0x7fe4b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:03.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.230+0000 7fe4c8c00640 1 -- 172.21.15.18:0/476606578 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7fe4bc0d0f00 con 0x7fe4cc104440 2024-04-16T17:28:03.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.230+0000 7fe4cb400640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4b00a3da0 0x7fe4b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:03.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.231+0000 7fe4cb400640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4b00a3da0 0x7fe4b00a6260 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009830 tx=0x7fe4b802e040 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:03.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.234+0000 7fe4c8c00640 1 -- 172.21.15.18:0/476606578 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4bc09d140 con 0x7fe4cc104440 2024-04-16T17:28:03.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:03 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2955356353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:03 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1656776723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:03 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1490268940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.442+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe4cc104df0 con 0x7fe4cc104440 2024-04-16T17:28:03.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.443+0000 7fe4c8c00640 1 -- 172.21.15.18:0/476606578 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v139) ==== 103+0+11 (secure 0 0 0) 0x7fe4bc09ca40 con 0x7fe4cc104440 2024-04-16T17:28:03.443 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-16T17:28:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:03 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2955356353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:03 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1656776723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:03 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1490268940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4b00a3da0 msgr2=0x7fe4b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4b00a3da0 0x7fe4b00a6260 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009830 tx=0x7fe4b802e040 comp rx=0 tx=0).stop 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4cc104440 msgr2=0x7fe4cc1a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4cc104440 0x7fe4cc1a0770 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00e9c0 tx=0x7fe4bc00edc0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 shutdown_connections 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fe4b00a3da0 0x7fe4b00a6260 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc19a920 0x7fe4cc19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4cc109510 0x7fe4cc1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 --2- 172.21.15.18:0/476606578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4cc104440 0x7fe4cc1a0770 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.446+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 >> 172.21.15.18:0/476606578 conn(0x7fe4cc0ffc80 msgr2=0x7fe4cc101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:03.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.447+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 shutdown_connections 2024-04-16T17:28:03.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.447+0000 7fe4d2a00640 1 -- 172.21.15.18:0/476606578 wait complete. 2024-04-16T17:28:03.466 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T17:28:03.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:03 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2955356353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:03 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1656776723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:03 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1490268940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:28:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.673+0000 7fc9f4a00640 1 -- 172.21.15.18:0/3246665169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f0107160 msgr2=0x7fc9f010ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.673+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/3246665169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f0107160 0x7fc9f010ed30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009ca0 tx=0x7fc9e802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.673+0000 7fc9f4a00640 1 -- 172.21.15.18:0/3246665169 shutdown_connections 2024-04-16T17:28:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.673+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/3246665169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f0107160 0x7fc9f010ed30 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.673+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/3246665169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9f0102a40 0x7fc9f0106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.673+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/3246665169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f0102090 0x7fc9f0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.673+0000 7fc9f4a00640 1 -- 172.21.15.18:0/3246665169 >> 172.21.15.18:0/3246665169 conn(0x7fc9f00fd910 msgr2=0x7fc9f00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.674+0000 7fc9f4a00640 1 -- 172.21.15.18:0/3246665169 shutdown_connections 2024-04-16T17:28:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.674+0000 7fc9f4a00640 1 -- 172.21.15.18:0/3246665169 wait complete. 2024-04-16T17:28:03.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.675+0000 7fc9f4a00640 1 Processor -- start 2024-04-16T17:28:03.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.675+0000 7fc9f4a00640 1 -- start start 2024-04-16T17:28:03.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.675+0000 7fc9f4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f0102090 0x7fc9f01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:03.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.675+0000 7fc9f4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f0107160 0x7fc9f01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.675+0000 7fc9f4a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9f019a7c0 0x7fc9f019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.675+0000 7fc9f4a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0112220 con 0x7fc9f019a7c0 2024-04-16T17:28:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.675+0000 7fc9f4a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01120a0 con 0x7fc9f0102090 2024-04-16T17:28:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.675+0000 7fc9f4a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01123a0 con 0x7fc9f0107160 2024-04-16T17:28:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ef200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9f019a7c0 0x7fc9f019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ef200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9f019a7c0 0x7fc9f019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59140/0 (socket says 172.21.15.18:59140) 2024-04-16T17:28:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ef200640 1 -- 172.21.15.18:0/1080533231 learned_addr learned my addr 172.21.15.18:0/1080533231 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ee800640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f0102090 0x7fc9f01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ede00640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f0107160 0x7fc9f01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ef200640 1 -- 172.21.15.18:0/1080533231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f0107160 msgr2=0x7fc9f01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ef200640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f0107160 0x7fc9f01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ef200640 1 -- 172.21.15.18:0/1080533231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f0102090 msgr2=0x7fc9f01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ef200640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f0102090 0x7fc9f01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ef200640 1 -- 172.21.15.18:0/1080533231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f019b3c0 con 0x7fc9f019a7c0 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.676+0000 7fc9ee800640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f0102090 0x7fc9f01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.677+0000 7fc9ef200640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9f019a7c0 0x7fc9f019ac40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc9e802c700 tx=0x7fc9e8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:03.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.677+0000 7fc9e7400640 1 -- 172.21.15.18:0/1080533231 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8041020 con 0x7fc9f019a7c0 2024-04-16T17:28:03.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.677+0000 7fc9e7400640 1 -- 172.21.15.18:0/1080533231 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e8034cd0 con 0x7fc9f019a7c0 2024-04-16T17:28:03.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.677+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f019b650 con 0x7fc9f019a7c0 2024-04-16T17:28:03.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.677+0000 7fc9e7400640 1 -- 172.21.15.18:0/1080533231 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8039040 con 0x7fc9f019a7c0 2024-04-16T17:28:03.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.677+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01a7740 con 0x7fc9f019a7c0 2024-04-16T17:28:03.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.678+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f0103560 con 0x7fc9f019a7c0 2024-04-16T17:28:03.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.680+0000 7fc9e7400640 1 -- 172.21.15.18:0/1080533231 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc9e8044020 con 0x7fc9f019a7c0 2024-04-16T17:28:03.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.682+0000 7fc9e7400640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9d40a3d20 0x7fc9d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:03.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.682+0000 7fc9e7400640 1 -- 172.21.15.18:0/1080533231 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7fc9e80f2be0 con 0x7fc9f019a7c0 2024-04-16T17:28:03.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.682+0000 7fc9ee800640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9d40a3d20 0x7fc9d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:03.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.683+0000 7fc9ee800640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9d40a3d20 0x7fc9d40a61e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009fc0 tx=0x7fc9e0005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:03.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.689+0000 7fc9e7400640 1 -- 172.21.15.18:0/1080533231 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9e804b020 con 0x7fc9f019a7c0 2024-04-16T17:28:03.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.914+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9f0102a40 con 0x7fc9f019a7c0 2024-04-16T17:28:03.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.914+0000 7fc9e7400640 1 -- 172.21.15.18:0/1080533231 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v139) ==== 81+0+946 (secure 0 0 0) 0x7fc9e80bef30 con 0x7fc9f019a7c0 2024-04-16T17:28:03.914 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3045212308 2024-04-17T17:04:44.091692+0000 2024-04-16T17:28:03.915 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/2462414912 2024-04-17T17:04:44.091692+0000 2024-04-16T17:28:03.915 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3152065578 2024-04-17T17:04:44.091692+0000 2024-04-16T17:28:03.915 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2364490471 2024-04-17T16:53:24.732531+0000 2024-04-16T17:28:03.915 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1759186913 2024-04-17T16:53:24.732531+0000 2024-04-16T17:28:03.915 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1033276779 2024-04-17T17:04:44.091692+0000 2024-04-16T17:28:03.915 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/2462414912 2024-04-17T17:04:44.091692+0000 2024-04-16T17:28:03.915 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2887519304 2024-04-17T16:53:24.732531+0000 2024-04-16T17:28:03.915 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2656869540 2024-04-17T16:52:49.432499+0000 2024-04-16T17:28:03.916 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1032645701 2024-04-17T16:52:49.432499+0000 2024-04-16T17:28:03.916 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1116733165 2024-04-17T16:52:49.432499+0000 2024-04-16T17:28:03.916 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1116733165 2024-04-17T16:52:49.432499+0000 2024-04-16T17:28:03.916 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2149549301 2024-04-17T17:04:44.091692+0000 2024-04-16T17:28:03.916 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1759186913 2024-04-17T16:53:24.732531+0000 2024-04-16T17:28:03.916 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2417150453 2024-04-17T16:52:49.432499+0000 2024-04-16T17:28:03.916 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3503805752 2024-04-17T16:53:24.732531+0000 2024-04-16T17:28:03.917 INFO:teuthology.orchestra.run.smithi018.stderr:listed 16 entries 2024-04-16T17:28:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9d40a3d20 msgr2=0x7fc9d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9d40a3d20 0x7fc9d40a61e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009fc0 tx=0x7fc9e0005f90 comp rx=0 tx=0).stop 2024-04-16T17:28:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9f019a7c0 msgr2=0x7fc9f019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9f019a7c0 0x7fc9f019ac40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc9e802c700 tx=0x7fc9e8032f20 comp rx=0 tx=0).stop 2024-04-16T17:28:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 shutdown_connections 2024-04-16T17:28:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7fc9d40a3d20 0x7fc9d40a61e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9f019a7c0 0x7fc9f019ac40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f0107160 0x7fc9f01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 --2- 172.21.15.18:0/1080533231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f0102090 0x7fc9f01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:03.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 >> 172.21.15.18:0/1080533231 conn(0x7fc9f00fd910 msgr2=0x7fc9f00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:03.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 shutdown_connections 2024-04-16T17:28:03.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:03.917+0000 7fc9f4a00640 1 -- 172.21.15.18:0/1080533231 wait complete. 2024-04-16T17:28:03.934 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-16T17:28:03.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:28:03.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-16T17:28:04.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:04 smithi022 ceph-mon[36023]: pgmap v747: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 10 op/s 2024-04-16T17:28:04.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:04 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/476606578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:28:04.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:04 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1080533231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:28:04.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:04 smithi043 ceph-mon[36901]: pgmap v747: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 10 op/s 2024-04-16T17:28:04.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:04 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/476606578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:28:04.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:04 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1080533231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:28:04.728 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:28:04.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:28:04.728 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-16T17:28:04.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:04 smithi018 ceph-mon[31776]: pgmap v747: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 10 op/s 2024-04-16T17:28:04.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:04 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/476606578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:28:04.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:04 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1080533231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:28:04.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:28:04.782 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-04-16T17:28:04.906 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-16T17:28:04.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:28:04.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T17:28:04.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:28:04.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-04-16T17:28:05.104 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-16T17:28:05.115 INFO:tasks.cephadm:Teardown begin 2024-04-16T17:28:05.115 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-16T17:28:05.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:28:05.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.310+0000 7ffa75600640 1 -- 172.21.15.18:0/562951766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa70074cf0 msgr2=0x7ffa7010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:05.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.310+0000 7ffa75600640 1 --2- 172.21.15.18:0/562951766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa70074cf0 0x7ffa7010e240 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009860 tx=0x7ffa5c02bc00 comp rx=0 tx=0).stop 2024-04-16T17:28:05.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.310+0000 7ffa75600640 1 -- 172.21.15.18:0/562951766 shutdown_connections 2024-04-16T17:28:05.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.310+0000 7ffa75600640 1 --2- 172.21.15.18:0/562951766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7010e780 0x7ffa70110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.310+0000 7ffa75600640 1 --2- 172.21.15.18:0/562951766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa70074cf0 0x7ffa7010e240 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.310+0000 7ffa75600640 1 --2- 172.21.15.18:0/562951766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa700743d0 0x7ffa700747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.310+0000 7ffa75600640 1 -- 172.21.15.18:0/562951766 >> 172.21.15.18:0/562951766 conn(0x7ffa700fd9f0 msgr2=0x7ffa700ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:05.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.311+0000 7ffa75600640 1 -- 172.21.15.18:0/562951766 shutdown_connections 2024-04-16T17:28:05.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.311+0000 7ffa75600640 1 -- 172.21.15.18:0/562951766 wait complete. 2024-04-16T17:28:05.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.311+0000 7ffa75600640 1 Processor -- start 2024-04-16T17:28:05.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa75600640 1 -- start start 2024-04-16T17:28:05.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa75600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa700743d0 0x7ffa701a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:05.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa75600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70074cf0 0x7ffa701a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:05.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa75600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7010e780 0x7ffa7019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:05.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa75600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701144e0 con 0x7ffa700743d0 2024-04-16T17:28:05.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa75600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70114360 con 0x7ffa70074cf0 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa75600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70114660 con 0x7ffa7010e780 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa6fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa700743d0 0x7ffa701a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa6fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa700743d0 0x7ffa701a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59164/0 (socket says 172.21.15.18:59164) 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa6fe00640 1 -- 172.21.15.18:0/2136877268 learned_addr learned my addr 172.21.15.18:0/2136877268 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa6f400640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70074cf0 0x7ffa701a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.312+0000 7ffa74c00640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7010e780 0x7ffa7019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6fe00640 1 -- 172.21.15.18:0/2136877268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7010e780 msgr2=0x7ffa7019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6fe00640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7010e780 0x7ffa7019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6fe00640 1 -- 172.21.15.18:0/2136877268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70074cf0 msgr2=0x7ffa701a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:05.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6fe00640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70074cf0 0x7ffa701a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6fe00640 1 -- 172.21.15.18:0/2136877268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7019b050 con 0x7ffa700743d0 2024-04-16T17:28:05.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6fe00640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa700743d0 0x7ffa701a06c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e9c0 tx=0x7ffa6000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:05.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6cc00640 1 -- 172.21.15.18:0/2136877268 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000ccc0 con 0x7ffa700743d0 2024-04-16T17:28:05.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6cc00640 1 -- 172.21.15.18:0/2136877268 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa60005500 con 0x7ffa700743d0 2024-04-16T17:28:05.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7019b340 con 0x7ffa700743d0 2024-04-16T17:28:05.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa6cc00640 1 -- 172.21.15.18:0/2136877268 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60010430 con 0x7ffa700743d0 2024-04-16T17:28:05.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.313+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701a7510 con 0x7ffa700743d0 2024-04-16T17:28:05.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.314+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa70075a80 con 0x7ffa700743d0 2024-04-16T17:28:05.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.316+0000 7ffa6cc00640 1 -- 172.21.15.18:0/2136877268 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa600046c0 con 0x7ffa700743d0 2024-04-16T17:28:05.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.318+0000 7ffa6cc00640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa540a3d10 0x7ffa540a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:05.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.319+0000 7ffa6cc00640 1 -- 172.21.15.18:0/2136877268 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7ffa600d0ff0 con 0x7ffa700743d0 2024-04-16T17:28:05.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.319+0000 7ffa6f400640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa540a3d10 0x7ffa540a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:05.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.320+0000 7ffa6f400640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa540a3d10 0x7ffa540a61d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c02c210 tx=0x7ffa5c0058d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:05.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.324+0000 7ffa6cc00640 1 -- 172.21.15.18:0/2136877268 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa6009d230 con 0x7ffa700743d0 2024-04-16T17:28:05.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.569+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa70109950 con 0x7ffa700743d0 2024-04-16T17:28:05.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.570+0000 7ffa6cc00640 1 -- 172.21.15.18:0/2136877268 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7ffa70109950 con 0x7ffa700743d0 2024-04-16T17:28:05.570 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:28:05.571 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:28:05.571 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:28:05.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa540a3d10 msgr2=0x7ffa540a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:05.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa540a3d10 0x7ffa540a61d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c02c210 tx=0x7ffa5c0058d0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa700743d0 msgr2=0x7ffa701a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:05.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa700743d0 0x7ffa701a06c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e9c0 tx=0x7ffa6000ee80 comp rx=0 tx=0).stop 2024-04-16T17:28:05.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 shutdown_connections 2024-04-16T17:28:05.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7ffa540a3d10 0x7ffa540a61d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7010e780 0x7ffa7019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70074cf0 0x7ffa701a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 --2- 172.21.15.18:0/2136877268 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa700743d0 0x7ffa701a06c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.573+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 >> 172.21.15.18:0/2136877268 conn(0x7ffa700fd9f0 msgr2=0x7ffa700fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:05.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.574+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 shutdown_connections 2024-04-16T17:28:05.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.574+0000 7ffa75600640 1 -- 172.21.15.18:0/2136877268 wait complete. 2024-04-16T17:28:05.591 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:28:05.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 -- 172.21.15.18:0/3913995316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104d00 msgr2=0x7f6694108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:05.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 --2- 172.21.15.18:0/3913995316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104d00 0x7f6694108d40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6680009860 tx=0x7f668002bb70 comp rx=0 tx=0).stop 2024-04-16T17:28:05.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 -- 172.21.15.18:0/3913995316 shutdown_connections 2024-04-16T17:28:05.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 --2- 172.21.15.18:0/3913995316 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6694109420 0x7f6694110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 --2- 172.21.15.18:0/3913995316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104d00 0x7f6694108d40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 --2- 172.21.15.18:0/3913995316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6694104350 0x7f6694104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 -- 172.21.15.18:0/3913995316 >> 172.21.15.18:0/3913995316 conn(0x7f66940ffb70 msgr2=0x7f6694101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:05.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 -- 172.21.15.18:0/3913995316 shutdown_connections 2024-04-16T17:28:05.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.796+0000 7f6699e00640 1 -- 172.21.15.18:0/3913995316 wait complete. 2024-04-16T17:28:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.797+0000 7f6699e00640 1 Processor -- start 2024-04-16T17:28:05.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.797+0000 7f6699e00640 1 -- start start 2024-04-16T17:28:05.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6699e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104350 0x7f66941a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:05.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6699e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6694109420 0x7f66941a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:05.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6699e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669419a890 0x7f669419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:05.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6699e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66941144b0 con 0x7f6694109420 2024-04-16T17:28:05.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6699e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6694114330 con 0x7f669419a890 2024-04-16T17:28:05.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6699e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6694114630 con 0x7f6694104350 2024-04-16T17:28:05.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6698a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104350 0x7f66941a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:05.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6699400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669419a890 0x7f669419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6698a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104350 0x7f66941a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:34520/0 (socket says 172.21.15.18:34520) 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6699400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669419a890 0x7f669419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:49142/0 (socket says 172.21.15.18:49142) 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6698a00640 1 -- 172.21.15.18:0/1559815592 learned_addr learned my addr 172.21.15.18:0/1559815592 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.798+0000 7f6693e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6694109420 0x7f66941a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.799+0000 7f6698a00640 1 -- 172.21.15.18:0/1559815592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669419a890 msgr2=0x7f669419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.799+0000 7f6698a00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669419a890 0x7f669419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.799+0000 7f6698a00640 1 -- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6694109420 msgr2=0x7f66941a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.799+0000 7f6698a00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6694109420 0x7f66941a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:05.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.799+0000 7f6698a00640 1 -- 172.21.15.18:0/1559815592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669419b420 con 0x7f6694104350 2024-04-16T17:28:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.799+0000 7f6698a00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104350 0x7f66941a06d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f668800eb90 tx=0x7f668800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.799+0000 7f6691600640 1 -- 172.21.15.18:0/1559815592 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688004d20 con 0x7f6694104350 2024-04-16T17:28:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.799+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669419b710 con 0x7f6694104350 2024-04-16T17:28:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.800+0000 7f6691600640 1 -- 172.21.15.18:0/1559815592 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f668800ee90 con 0x7f6694104350 2024-04-16T17:28:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.800+0000 7f6691600640 1 -- 172.21.15.18:0/1559815592 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688010830 con 0x7f6694104350 2024-04-16T17:28:05.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.800+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941a7870 con 0x7f6694104350 2024-04-16T17:28:05.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.801+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6694105820 con 0x7f6694104350 2024-04-16T17:28:05.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.805+0000 7f6691600640 1 -- 172.21.15.18:0/1559815592 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6688005020 con 0x7f6694104350 2024-04-16T17:28:05.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.806+0000 7f6691600640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f667c0a3d10 0x7f667c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:05.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.806+0000 7f6691600640 1 -- 172.21.15.18:0/1559815592 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7f66880d1290 con 0x7f6694104350 2024-04-16T17:28:05.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.806+0000 7f6693e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f667c0a3d10 0x7f667c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:05.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.807+0000 7f6693e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f667c0a3d10 0x7f667c0a61d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6680009830 tx=0x7f66800056d0 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:05.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:05.811+0000 7f6691600640 1 -- 172.21.15.18:0/1559815592 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f669419b710 con 0x7f6694104350 2024-04-16T17:28:06.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.048+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f669404fe00 con 0x7f6694104350 2024-04-16T17:28:06.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.049+0000 7f6691600640 1 -- 172.21.15.18:0/1559815592 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11852 (secure 0 0 0) 0x7f668809d4d0 con 0x7f6694104350 2024-04-16T17:28:06.049 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:28:06.050 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/620951660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":620951660},{"type":"v1","addr":"172.21.15.18:6837","nonce":620951660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6841/2108347898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6840","nonce":2108347898},{"type":"v1","addr":"172.21.15.18:6841","nonce":2108347898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3903003819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3903003819},{"type":"v1","addr":"172.21.15.43:6833","nonce":3903003819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/4293504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4293504707},{"type":"v1","addr":"172.21.15.22:6833","nonce":4293504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/1011323908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":1011323908},{"type":"v1","addr":"172.21.15.18:6835","nonce":1011323908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":29933,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/456310898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":456310898},{"type":"v1","addr":"172.21.15.22:6837","nonce":456310898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:01:24.398529+0000","modified":"2024-04-16T17:21:03.619110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24566,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6835/3315767064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3315767064},{"type":"v1","addr":"172.21.15.22:6835","nonce":3315767064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/1265446335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1265446335},{"type":"v1","addr":"172.21.15.43:6835","nonce":1265446335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6837/1790976373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1790976373},{"type":"v1","addr":"172.21.15.43:6837","nonce":1790976373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.43:6839/3993514244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3993514244},{"type":"v1","addr":"172.21.15.43:6839","nonce":3993514244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.18:6839/294785837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":294785837},{"type":"v1","addr":"172.21.15.18:6839","nonce":294785837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6839/836242731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":836242731},{"type":"v1","addr":"172.21.15.22:6839","nonce":836242731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24566,24596,24547]},"id":1}]} 2024-04-16T17:28:06.050 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-16T17:28:06.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f667c0a3d10 msgr2=0x7f667c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f667c0a3d10 0x7f667c0a61d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6680009830 tx=0x7f66800056d0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104350 msgr2=0x7f66941a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104350 0x7f66941a06d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f668800eb90 tx=0x7f668800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 shutdown_connections 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f667c0a3d10 0x7f667c0a61d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f669419a890 0x7f669419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6694109420 0x7f66941a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 --2- 172.21.15.18:0/1559815592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6694104350 0x7f66941a06d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 >> 172.21.15.18:0/1559815592 conn(0x7f66940ffb70 msgr2=0x7f6694100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 shutdown_connections 2024-04-16T17:28:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.053+0000 7f6699e00640 1 -- 172.21.15.18:0/1559815592 wait complete. 2024-04-16T17:28:06.080 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:28:06.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 -- 172.21.15.18:0/701815887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f190010e820 msgr2=0x7f1900110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 --2- 172.21.15.18:0/701815887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f190010e820 0x7f1900110c10 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009ca0 tx=0x7f18f802c100 comp rx=0 tx=0).stop 2024-04-16T17:28:06.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 -- 172.21.15.18:0/701815887 shutdown_connections 2024-04-16T17:28:06.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 --2- 172.21.15.18:0/701815887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f190010e820 0x7f1900110c10 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 --2- 172.21.15.18:0/701815887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1900101620 0x7f190010e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 --2- 172.21.15.18:0/701815887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1900100c70 0x7f1900101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 -- 172.21.15.18:0/701815887 >> 172.21.15.18:0/701815887 conn(0x7f19000fc890 msgr2=0x7f19000fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:06.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 -- 172.21.15.18:0/701815887 shutdown_connections 2024-04-16T17:28:06.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.290+0000 7f1906800640 1 -- 172.21.15.18:0/701815887 wait complete. 2024-04-16T17:28:06.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.291+0000 7f1906800640 1 Processor -- start 2024-04-16T17:28:06.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.291+0000 7f1906800640 1 -- start start 2024-04-16T17:28:06.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1906800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1900100c70 0x7f19001a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:06.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1906800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1900101620 0x7f19001a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:06.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1906800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f190010e820 0x7f190019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:06.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1906800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1900114470 con 0x7f1900101620 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1906800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f19001142f0 con 0x7f190010e820 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1906800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19001145f0 con 0x7f1900100c70 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1904a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1900101620 0x7f19001a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1904a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1900101620 0x7f19001a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59200/0 (socket says 172.21.15.18:59200) 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1904a00640 1 -- 172.21.15.18:0/1803356964 learned_addr learned my addr 172.21.15.18:0/1803356964 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1905400640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1900100c70 0x7f19001a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.292+0000 7f1905e00640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f190010e820 0x7f190019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f1904a00640 1 -- 172.21.15.18:0/1803356964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1900100c70 msgr2=0x7f19001a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f1904a00640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1900100c70 0x7f19001a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f1904a00640 1 -- 172.21.15.18:0/1803356964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f190010e820 msgr2=0x7f190019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f1904a00640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f190010e820 0x7f190019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f1904a00640 1 -- 172.21.15.18:0/1803356964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190019afe0 con 0x7f1900101620 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f1904a00640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1900101620 0x7f19001a0be0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e9f0 tx=0x7f18ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f18fe000640 1 -- 172.21.15.18:0/1803356964 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00cd80 con 0x7f1900101620 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f18fe000640 1 -- 172.21.15.18:0/1803356964 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18ec010910 con 0x7f1900101620 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f18fe000640 1 -- 172.21.15.18:0/1803356964 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec018990 con 0x7f1900101620 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190019b2d0 con 0x7f1900101620 2024-04-16T17:28:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.293+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19001a74f0 con 0x7f1900101620 2024-04-16T17:28:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.295+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1900069b80 con 0x7f1900101620 2024-04-16T17:28:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.297+0000 7f18fe000640 1 -- 172.21.15.18:0/1803356964 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18ec010ac0 con 0x7f1900101620 2024-04-16T17:28:06.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.299+0000 7f18fe000640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18e80a3d90 0x7f18e80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.299+0000 7f18fe000640 1 -- 172.21.15.18:0/1803356964 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7f18ec0d12c0 con 0x7f1900101620 2024-04-16T17:28:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.299+0000 7f1905400640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18e80a3d90 0x7f18e80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:06.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.300+0000 7f1905400640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18e80a3d90 0x7f18e80a6250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009bd0 tx=0x7f18f4005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:06.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.302+0000 7f18fe000640 1 -- 172.21.15.18:0/1803356964 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18ec09ceb0 con 0x7f1900101620 2024-04-16T17:28:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:06 smithi022 ceph-mon[36023]: pgmap v748: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 10 op/s 2024-04-16T17:28:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:06 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2136877268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:28:06.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:06 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1559815592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:28:06.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:06 smithi018 ceph-mon[31776]: pgmap v748: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 10 op/s 2024-04-16T17:28:06.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:06 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2136877268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:28:06.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:06 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1559815592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:28:06.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:06 smithi043 ceph-mon[36901]: pgmap v748: 129 pgs: 129 active+clean; 452 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 10 op/s 2024-04-16T17:28:06.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:06 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2136877268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:28:06.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:06 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1559815592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:28:06.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.513+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19001099b0 con 0x7f1900101620 2024-04-16T17:28:06.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.515+0000 7f18fe000640 1 -- 172.21.15.18:0/1803356964 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+25290 (secure 0 0 0) 0x7f18ec0a11b0 con 0x7f1900101620 2024-04-16T17:28:06.515 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-16T17:28:06.516 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":139,"fsid":"872930f8-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:52:16.225349+0000","modified":"2024-04-16T17:27:23.843667+0000","last_up_change":"2024-04-16T17:00:24.881345+0000","last_in_change":"2024-04-16T17:00:04.874935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:56:08.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:01:21.974887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:01:23.365582+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":41,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:01:26.987830+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":41,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a6d2ab-b5fc-4430-99dc-5074f1d7e51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6803","nonce":2892109649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6805","nonce":2892109649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6809","nonce":2892109649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2892109649},{"type":"v1","addr":"172.21.15.18:6807","nonce":2892109649}]},"public_addr":"172.21.15.18:6803/2892109649","cluster_addr":"172.21.15.18:6805/2892109649","heartbeat_back_addr":"172.21.15.18:6809/2892109649","heartbeat_front_addr":"172.21.15.18:6807/2892109649","state":["exists","up"]},{"osd":1,"uuid":"efef211f-28b9-4f7b-a3c7-59ccb4914bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6801","nonce":1527176746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6803","nonce":1527176746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6807","nonce":1527176746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":1527176746},{"type":"v1","addr":"172.21.15.22:6805","nonce":1527176746}]},"public_addr":"172.21.15.22:6801/1527176746","cluster_addr":"172.21.15.22:6803/1527176746","heartbeat_back_addr":"172.21.15.22:6807/1527176746","heartbeat_front_addr":"172.21.15.22:6805/1527176746","state":["exists","up"]},{"osd":2,"uuid":"0f783df6-43e0-4a3e-b413-af66cf95e18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6801","nonce":3809455013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6803","nonce":3809455013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6807","nonce":3809455013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3809455013},{"type":"v1","addr":"172.21.15.43:6805","nonce":3809455013}]},"public_addr":"172.21.15.43:6801/3809455013","cluster_addr":"172.21.15.43:6803/3809455013","heartbeat_back_addr":"172.21.15.43:6807/3809455013","heartbeat_front_addr":"172.21.15.43:6805/3809455013","state":["exists","up"]},{"osd":3,"uuid":"c0da47d3-ae80-4e06-a98e-887862be0ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6811","nonce":2424709764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6813","nonce":2424709764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6817","nonce":2424709764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2424709764},{"type":"v1","addr":"172.21.15.18:6815","nonce":2424709764}]},"public_addr":"172.21.15.18:6811/2424709764","cluster_addr":"172.21.15.18:6813/2424709764","heartbeat_back_addr":"172.21.15.18:6817/2424709764","heartbeat_front_addr":"172.21.15.18:6815/2424709764","state":["exists","up"]},{"osd":4,"uuid":"2338ca28-bed3-415a-b762-28d2275a3940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6809","nonce":301052168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6811","nonce":301052168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6815","nonce":301052168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":301052168},{"type":"v1","addr":"172.21.15.22:6813","nonce":301052168}]},"public_addr":"172.21.15.22:6809/301052168","cluster_addr":"172.21.15.22:6811/301052168","heartbeat_back_addr":"172.21.15.22:6815/301052168","heartbeat_front_addr":"172.21.15.22:6813/301052168","state":["exists","up"]},{"osd":5,"uuid":"dd247a85-21b7-4f15-bf4b-4f391dd6e543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6809","nonce":3934456075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6811","nonce":3934456075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6815","nonce":3934456075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3934456075},{"type":"v1","addr":"172.21.15.43:6813","nonce":3934456075}]},"public_addr":"172.21.15.43:6809/3934456075","cluster_addr":"172.21.15.43:6811/3934456075","heartbeat_back_addr":"172.21.15.43:6815/3934456075","heartbeat_front_addr":"172.21.15.43:6813/3934456075","state":["exists","up"]},{"osd":6,"uuid":"1d5bb4a0-908c-41b1-9967-a35d4e77d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6819","nonce":3519009309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6821","nonce":3519009309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6825","nonce":3519009309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3519009309},{"type":"v1","addr":"172.21.15.18:6823","nonce":3519009309}]},"public_addr":"172.21.15.18:6819/3519009309","cluster_addr":"172.21.15.18:6821/3519009309","heartbeat_back_addr":"172.21.15.18:6825/3519009309","heartbeat_front_addr":"172.21.15.18:6823/3519009309","state":["exists","up"]},{"osd":7,"uuid":"d3917a58-159e-4672-a8b1-83b0fe54efbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6817","nonce":1748007509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6819","nonce":1748007509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6823","nonce":1748007509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1748007509},{"type":"v1","addr":"172.21.15.22:6821","nonce":1748007509}]},"public_addr":"172.21.15.22:6817/1748007509","cluster_addr":"172.21.15.22:6819/1748007509","heartbeat_back_addr":"172.21.15.22:6823/1748007509","heartbeat_front_addr":"172.21.15.22:6821/1748007509","state":["exists","up"]},{"osd":8,"uuid":"06e72a0a-097f-49cb-9626-d0de66ba5641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6817","nonce":546535747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6819","nonce":546535747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6823","nonce":546535747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":546535747},{"type":"v1","addr":"172.21.15.43:6821","nonce":546535747}]},"public_addr":"172.21.15.43:6817/546535747","cluster_addr":"172.21.15.43:6819/546535747","heartbeat_back_addr":"172.21.15.43:6823/546535747","heartbeat_front_addr":"172.21.15.43:6821/546535747","state":["exists","up"]},{"osd":9,"uuid":"7cf9ee08-5ede-446d-9b67-cebda4d2be28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6827","nonce":3050280223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6829","nonce":3050280223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6833","nonce":3050280223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":3050280223},{"type":"v1","addr":"172.21.15.18:6831","nonce":3050280223}]},"public_addr":"172.21.15.18:6827/3050280223","cluster_addr":"172.21.15.18:6829/3050280223","heartbeat_back_addr":"172.21.15.18:6833/3050280223","heartbeat_front_addr":"172.21.15.18:6831/3050280223","state":["exists","up"]},{"osd":10,"uuid":"1a70d7bc-095f-4be6-9096-7bf5d46cd26d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6825","nonce":3445736292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6827","nonce":3445736292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6831","nonce":3445736292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3445736292},{"type":"v1","addr":"172.21.15.22:6829","nonce":3445736292}]},"public_addr":"172.21.15.22:6825/3445736292","cluster_addr":"172.21.15.22:6827/3445736292","heartbeat_back_addr":"172.21.15.22:6831/3445736292","heartbeat_front_addr":"172.21.15.22:6829/3445736292","state":["exists","up"]},{"osd":11,"uuid":"35fefd2c-28c2-4fd4-bd0b-7b3171fe5a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6825","nonce":504916242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6827","nonce":504916242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6831","nonce":504916242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":504916242},{"type":"v1","addr":"172.21.15.43:6829","nonce":504916242}]},"public_addr":"172.21.15.43:6825/504916242","cluster_addr":"172.21.15.43:6827/504916242","heartbeat_back_addr":"172.21.15.43:6831/504916242","heartbeat_front_addr":"172.21.15.43:6829/504916242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:16.453284+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:55:41.588741+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:05.989084+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:56:38.414490+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:04.446757+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:29.960828+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.424441+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:29.757151+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:55.404449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:29.175569+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:56.665239+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:23.407618+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3045212308":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/3152065578":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2364490471":"2024-04-17T16:53:24.732531+0000","172.21.15.18:6801/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/1033276779":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6801/2462414912":"2024-04-17T17:04:44.091692+0000","172.21.15.18:0/2887519304":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2656869540":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/1032645701":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6801/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:6800/1116733165":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/2149549301":"2024-04-17T17:04:44.091692+0000","172.21.15.18:6800/1759186913":"2024-04-17T16:53:24.732531+0000","172.21.15.18:0/2417150453":"2024-04-17T16:52:49.432499+0000","172.21.15.18:0/3503805752":"2024-04-17T16:53:24.732531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":41,"length":1}]},{"pool":4,"snaps":[{"begin":28,"length":1},{"begin":41,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:28:06.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18e80a3d90 msgr2=0x7f18e80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18e80a3d90 0x7f18e80a6250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009bd0 tx=0x7f18f4005f90 comp rx=0 tx=0).stop 2024-04-16T17:28:06.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1900101620 msgr2=0x7f19001a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1900101620 0x7f19001a0be0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e9f0 tx=0x7f18ec00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 shutdown_connections 2024-04-16T17:28:06.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f18e80a3d90 0x7f18e80a6250 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f190010e820 0x7f190019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1900101620 0x7f19001a0be0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 --2- 172.21.15.18:0/1803356964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1900100c70 0x7f19001a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 >> 172.21.15.18:0/1803356964 conn(0x7f19000fc890 msgr2=0x7f190010fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:06.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 shutdown_connections 2024-04-16T17:28:06.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.518+0000 7f1906800640 1 -- 172.21.15.18:0/1803356964 wait complete. 2024-04-16T17:28:06.539 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-16T17:28:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 -- 172.21.15.18:0/777675625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34f4109520 msgr2=0x7f34f41110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 --2- 172.21.15.18:0/777675625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34f4109520 0x7f34f41110f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009900 tx=0x7f34ec02bc30 comp rx=0 tx=0).stop 2024-04-16T17:28:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 -- 172.21.15.18:0/777675625 shutdown_connections 2024-04-16T17:28:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 --2- 172.21.15.18:0/777675625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34f4109520 0x7f34f41110f0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 --2- 172.21.15.18:0/777675625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34f4104e00 0x7f34f4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 --2- 172.21.15.18:0/777675625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34f4104450 0x7f34f4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 -- 172.21.15.18:0/777675625 >> 172.21.15.18:0/777675625 conn(0x7f34f40ffc90 msgr2=0x7f34f41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 -- 172.21.15.18:0/777675625 shutdown_connections 2024-04-16T17:28:06.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.755+0000 7f34faa00640 1 -- 172.21.15.18:0/777675625 wait complete. 2024-04-16T17:28:06.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.756+0000 7f34faa00640 1 Processor -- start 2024-04-16T17:28:06.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34faa00640 1 -- start start 2024-04-16T17:28:06.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34faa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34f4104450 0x7f34f41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:06.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34faa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34f4109520 0x7f34f41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34faa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34f419a8f0 0x7f34f419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34faa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f34f41145d0 con 0x7f34f4109520 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34faa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34f4114450 con 0x7f34f419a8f0 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34faa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f34f4114750 con 0x7f34f4104450 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34f3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34f4109520 0x7f34f41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34f3400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34f4109520 0x7f34f41a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59228/0 (socket says 172.21.15.18:59228) 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34f3400640 1 -- 172.21.15.18:0/791304029 learned_addr learned my addr 172.21.15.18:0/791304029 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34f3e00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34f4104450 0x7f34f41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:06.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.757+0000 7f34f8a00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34f419a8f0 0x7f34f419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f3400640 1 -- 172.21.15.18:0/791304029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34f4104450 msgr2=0x7f34f41a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f3400640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34f4104450 0x7f34f41a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f3400640 1 -- 172.21.15.18:0/791304029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34f419a8f0 msgr2=0x7f34f419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f3400640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34f419a8f0 0x7f34f419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f3400640 1 -- 172.21.15.18:0/791304029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f419b410 con 0x7f34f4109520 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f3400640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34f4109520 0x7f34f41a0c30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00e9f0 tx=0x7f34dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f0c00640 1 -- 172.21.15.18:0/791304029 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00ccc0 con 0x7f34f4109520 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f0c00640 1 -- 172.21.15.18:0/791304029 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34dc005500 con 0x7f34f4109520 2024-04-16T17:28:06.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f419b700 con 0x7f34f4109520 2024-04-16T17:28:06.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34f0c00640 1 -- 172.21.15.18:0/791304029 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc010690 con 0x7f34f4109520 2024-04-16T17:28:06.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.758+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f41a78b0 con 0x7f34f4109520 2024-04-16T17:28:06.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.760+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f4069300 con 0x7f34f4109520 2024-04-16T17:28:06.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.762+0000 7f34f0c00640 1 -- 172.21.15.18:0/791304029 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f34dc0046c0 con 0x7f34f4109520 2024-04-16T17:28:06.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.764+0000 7f34f0c00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34d80a3d20 0x7f34d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:06.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.764+0000 7f34f0c00640 1 -- 172.21.15.18:0/791304029 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9538+0+0 (secure 0 0 0) 0x7f34dc0d0f20 con 0x7f34f4109520 2024-04-16T17:28:06.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.764+0000 7f34f3e00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34d80a3d20 0x7f34d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:06.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.765+0000 7f34f3e00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34d80a3d20 0x7f34d80a61e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f34f41000e0 tx=0x7f34e4009680 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:06.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:06.769+0000 7f34f0c00640 1 -- 172.21.15.18:0/791304029 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34dc09d160 con 0x7f34f4109520 2024-04-16T17:28:07.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.023+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f34f4104e00 con 0x7f34f4109520 2024-04-16T17:28:07.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.026+0000 7f34f0c00640 1 -- 172.21.15.18:0/791304029 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f34dc09ca60 con 0x7f34f4109520 2024-04-16T17:28:07.026 INFO:teuthology.orchestra.run.smithi018.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-16T17:28:07.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34d80a3d20 msgr2=0x7f34d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:07.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34d80a3d20 0x7f34d80a61e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f34f41000e0 tx=0x7f34e4009680 comp rx=0 tx=0).stop 2024-04-16T17:28:07.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34f4109520 msgr2=0x7f34f41a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:07.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34f4109520 0x7f34f41a0c30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00e9f0 tx=0x7f34dc00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 shutdown_connections 2024-04-16T17:28:07.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f34d80a3d20 0x7f34d80a61e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34f419a8f0 0x7f34f419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34f4109520 0x7f34f41a0c30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 --2- 172.21.15.18:0/791304029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34f4104450 0x7f34f41a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.030+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 >> 172.21.15.18:0/791304029 conn(0x7f34f40ffc90 msgr2=0x7f34f4101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:07.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.031+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 shutdown_connections 2024-04-16T17:28:07.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.031+0000 7f34faa00640 1 -- 172.21.15.18:0/791304029 wait complete. 2024-04-16T17:28:07.050 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-16T17:28:07.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 -- 172.21.15.18:0/4099006097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284103c20 msgr2=0x7f928410c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:07.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 --2- 172.21.15.18:0/4099006097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284103c20 0x7f928410c070 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f926c009900 tx=0x7f926c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:28:07.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 -- 172.21.15.18:0/4099006097 shutdown_connections 2024-04-16T17:28:07.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 --2- 172.21.15.18:0/4099006097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f928410c780 0x7f928410ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 --2- 172.21.15.18:0/4099006097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284103c20 0x7f928410c070 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 --2- 172.21.15.18:0/4099006097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284103270 0x7f9284103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 -- 172.21.15.18:0/4099006097 >> 172.21.15.18:0/4099006097 conn(0x7f9284078020 msgr2=0x7f9284078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:07.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 -- 172.21.15.18:0/4099006097 shutdown_connections 2024-04-16T17:28:07.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.255+0000 7f928a200640 1 -- 172.21.15.18:0/4099006097 wait complete. 2024-04-16T17:28:07.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.256+0000 7f928a200640 1 Processor -- start 2024-04-16T17:28:07.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f928a200640 1 -- start start 2024-04-16T17:28:07.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f928a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9284103270 0x7f928419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f928a200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f928410c780 0x7f928419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f928a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284198940 0x7f9284198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f928a200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9284112580 con 0x7f928410c780 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f928a200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9284112400 con 0x7f9284103270 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f928a200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9284112700 con 0x7f9284198940 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f9283e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284198940 0x7f9284198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f9283400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9284103270 0x7f928419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f9283e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284198940 0x7f9284198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:42980/0 (socket says 172.21.15.18:42980) 2024-04-16T17:28:07.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f9283e00640 1 -- 172.21.15.18:0/2821478632 learned_addr learned my addr 172.21.15.18:0/2821478632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.257+0000 7f927aa00640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f928410c780 0x7f928419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9283e00640 1 -- 172.21.15.18:0/2821478632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9284103270 msgr2=0x7f928419e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9283e00640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9284103270 0x7f928419e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9283e00640 1 -- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f928410c780 msgr2=0x7f928419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9283e00640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f928410c780 0x7f928419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9283e00640 1 -- 172.21.15.18:0/2821478632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9284199460 con 0x7f9284198940 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9283400640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9284103270 0x7f928419e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9283e00640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284198940 0x7f9284198df0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f927c00eb10 tx=0x7f927c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:07.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9280c00640 1 -- 172.21.15.18:0/2821478632 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c004b20 con 0x7f9284198940 2024-04-16T17:28:07.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9280c00640 1 -- 172.21.15.18:0/2821478632 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927c004cc0 con 0x7f9284198940 2024-04-16T17:28:07.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f9280c00640 1 -- 172.21.15.18:0/2821478632 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c010760 con 0x7f9284198940 2024-04-16T17:28:07.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9284199750 con 0x7f9284198940 2024-04-16T17:28:07.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.258+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92841a5900 con 0x7f9284198940 2024-04-16T17:28:07.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.259+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9284069300 con 0x7f9284198940 2024-04-16T17:28:07.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.262+0000 7f9280c00640 1 -- 172.21.15.18:0/2821478632 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f927c005000 con 0x7f9284198940 2024-04-16T17:28:07.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.263+0000 7f9280c00640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92680a3da0 0x7f92680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:07.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.263+0000 7f9280c00640 1 -- 172.21.15.18:0/2821478632 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9724+0+0 (secure 0 0 0) 0x7f927c0d13b0 con 0x7f9284198940 2024-04-16T17:28:07.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.263+0000 7f9283400640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92680a3da0 0x7f92680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:07.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.264+0000 7f9283400640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92680a3da0 0x7f92680a6260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9284078850 tx=0x7f9274005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:07.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.265+0000 7f9280c00640 1 -- 172.21.15.18:0/2821478632 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f927c053ff0 con 0x7f9284198940 2024-04-16T17:28:07.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1803356964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:28:07.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/791304029' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T17:28:07.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mon[36023]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T17:28:07.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mon[36023]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T17:28:07.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/791304029' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T17:28:07.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mon[36023]: osdmap e140: 12 total, 12 up, 12 in 2024-04-16T17:28:07.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mon[36023]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T17:28:07.424 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: ignoring --setuser ceph since I am not root 2024-04-16T17:28:07.425 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: ignoring --setgroup ceph since I am not root 2024-04-16T17:28:07.425 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mds[66375]: -- 172.21.15.22:0/1373330969 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632bf199a40 con 0x5632bf16d000 2024-04-16T17:28:07.425 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: starting mds.b at 2024-04-16T17:28:07.425 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e[65726]: ignoring --setuser ceph since I am not root 2024-04-16T17:28:07.425 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e[65726]: ignoring --setgroup ceph since I am not root 2024-04-16T17:28:07.425 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mds[65730]: -- 172.21.15.22:0/1910585517 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a8b2d764e0 con 0x55a8b2d4c800 2024-04-16T17:28:07.426 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-mds[65730]: -- 172.21.15.22:0/1910585517 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8b2d79a40 con 0x55a8b2d4c800 2024-04-16T17:28:07.426 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:28:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e[65726]: starting mds.e at 2024-04-16T17:28:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a[31772]: 2024-04-16T17:28:07.024+0000 7f7b08400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T17:28:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1803356964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:28:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/791304029' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T17:28:07.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-mon[31776]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T17:28:07.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-mon[31776]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T17:28:07.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/791304029' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T17:28:07.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-mon[31776]: osdmap e140: 12 total, 12 up, 12 in 2024-04-16T17:28:07.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-mon[31776]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T17:28:07.445 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g[70753]: ignoring --setuser ceph since I am not root 2024-04-16T17:28:07.445 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g[70753]: ignoring --setgroup ceph since I am not root 2024-04-16T17:28:07.446 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-mds[70759]: -- 172.21.15.18:0/3569985880 <== mon.2 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597c54f7a40 con 0x5597c54cb000 2024-04-16T17:28:07.446 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:28:07 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g[70753]: starting mds.g at 2024-04-16T17:28:07.501 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c[67328]: ignoring --setuser ceph since I am not root 2024-04-16T17:28:07.501 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c[67328]: ignoring --setgroup ceph since I am not root 2024-04-16T17:28:07.501 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mds[67332]: -- 172.21.15.43:0/217912527 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec2a209a40 con 0x55ec2a1dd000 2024-04-16T17:28:07.501 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c[67328]: starting mds.c at 2024-04-16T17:28:07.501 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f[67009]: ignoring --setuser ceph since I am not root 2024-04-16T17:28:07.502 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f[67009]: ignoring --setgroup ceph since I am not root 2024-04-16T17:28:07.502 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mds[67013]: -- 172.21.15.43:0/774138503 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d93fe3a40 con 0x561d93fb6800 2024-04-16T17:28:07.502 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f[67009]: starting mds.f at 2024-04-16T17:28:07.502 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i[67650]: ignoring --setuser ceph since I am not root 2024-04-16T17:28:07.503 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i[67650]: ignoring --setgroup ceph since I am not root 2024-04-16T17:28:07.503 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mds[67654]: -- 172.21.15.43:0/3707900553 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba5ba0e4e0 con 0x55ba5b9e4800 2024-04-16T17:28:07.503 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mds[67654]: -- 172.21.15.43:0/3707900553 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba5ba11a40 con 0x55ba5b9e4800 2024-04-16T17:28:07.503 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i[67650]: starting mds.i at 2024-04-16T17:28:07.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1803356964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:28:07.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/791304029' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T17:28:07.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mon[36901]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T17:28:07.504 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mon[36901]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T17:28:07.504 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/791304029' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T17:28:07.504 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mon[36901]: osdmap e140: 12 total, 12 up, 12 in 2024-04-16T17:28:07.504 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:07 smithi043 ceph-mon[36901]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T17:28:07.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:07.521+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9284107730 con 0x7f9284198940 2024-04-16T17:28:08.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.095+0000 7f9280c00640 1 -- 172.21.15.18:0/2821478632 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f927c09d540 con 0x7f9284198940 2024-04-16T17:28:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.098+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92680a3da0 msgr2=0x7f92680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.098+0000 7f928a200640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92680a3da0 0x7f92680a6260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9284078850 tx=0x7f9274005f90 comp rx=0 tx=0).stop 2024-04-16T17:28:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.098+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284198940 msgr2=0x7f9284198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.098+0000 7f928a200640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284198940 0x7f9284198df0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f927c00eb10 tx=0x7f927c00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.099+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 shutdown_connections 2024-04-16T17:28:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.099+0000 7f928a200640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f92680a3da0 0x7f92680a6260 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.099+0000 7f928a200640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9284198940 0x7f9284198df0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.099+0000 7f928a200640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f928410c780 0x7f928419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.099+0000 7f928a200640 1 --2- 172.21.15.18:0/2821478632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9284103270 0x7f928419e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.099+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 >> 172.21.15.18:0/2821478632 conn(0x7f9284078020 msgr2=0x7f92840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.099+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 shutdown_connections 2024-04-16T17:28:08.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.100+0000 7f928a200640 1 -- 172.21.15.18:0/2821478632 wait complete. 2024-04-16T17:28:08.126 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-16T17:28:08.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.332+0000 7f4e5ea00640 1 -- 172.21.15.18:0/2464440667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58106010 msgr2=0x7f4e5810dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:08.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.332+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/2464440667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58106010 0x7f4e5810dbe0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009be0 tx=0x7f4e5002bef0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.332+0000 7f4e5ea00640 1 -- 172.21.15.18:0/2464440667 shutdown_connections 2024-04-16T17:28:08.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.332+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/2464440667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58106010 0x7f4e5810dbe0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.332+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/2464440667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e581018f0 0x7f4e58105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.332+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/2464440667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58100f40 0x7f4e58101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.332+0000 7f4e5ea00640 1 -- 172.21.15.18:0/2464440667 >> 172.21.15.18:0/2464440667 conn(0x7f4e58077ab0 msgr2=0x7f4e580fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:08.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.332+0000 7f4e5ea00640 1 -- 172.21.15.18:0/2464440667 shutdown_connections 2024-04-16T17:28:08.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.333+0000 7f4e5ea00640 1 -- 172.21.15.18:0/2464440667 wait complete. 2024-04-16T17:28:08.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.333+0000 7f4e5ea00640 1 Processor -- start 2024-04-16T17:28:08.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5ea00640 1 -- start start 2024-04-16T17:28:08.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58100f40 0x7f4e58198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:08.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58106010 0x7f4e58198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:08.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e5819cf20 0x7f4e5819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:08.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5ea00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581121e0 con 0x7f4e58100f40 2024-04-16T17:28:08.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5ea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58112060 con 0x7f4e58106010 2024-04-16T17:28:08.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5ea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58112360 con 0x7f4e5819cf20 2024-04-16T17:28:08.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58100f40 0x7f4e58198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:08.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58100f40 0x7f4e58198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43410/0 (socket says 172.21.15.18:43410) 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5d600640 1 -- 172.21.15.18:0/1471983342 learned_addr learned my addr 172.21.15.18:0/1471983342 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.334+0000 7f4e5e000640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e5819cf20 0x7f4e5819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.335+0000 7f4e5cc00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58106010 0x7f4e58198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.335+0000 7f4e5d600640 1 -- 172.21.15.18:0/1471983342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e5819cf20 msgr2=0x7f4e5819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.335+0000 7f4e5d600640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e5819cf20 0x7f4e5819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.335+0000 7f4e5d600640 1 -- 172.21.15.18:0/1471983342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58106010 msgr2=0x7f4e58198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.335+0000 7f4e5d600640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58106010 0x7f4e58198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.335+0000 7f4e5d600640 1 -- 172.21.15.18:0/1471983342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5819da10 con 0x7f4e58100f40 2024-04-16T17:28:08.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.335+0000 7f4e5d600640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58100f40 0x7f4e58198700 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00e9c0 tx=0x7f4e4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.336+0000 7f4e56000640 1 -- 172.21.15.18:0/1471983342 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00cde0 con 0x7f4e58100f40 2024-04-16T17:28:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.336+0000 7f4e56000640 1 -- 172.21.15.18:0/1471983342 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c0035f0 con 0x7f4e58100f40 2024-04-16T17:28:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.336+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5819dd00 con 0x7f4e58100f40 2024-04-16T17:28:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.336+0000 7f4e56000640 1 -- 172.21.15.18:0/1471983342 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c010690 con 0x7f4e58100f40 2024-04-16T17:28:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.336+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a54f0 con 0x7f4e58100f40 2024-04-16T17:28:08.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.337+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5804fd90 con 0x7f4e58100f40 2024-04-16T17:28:08.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.339+0000 7f4e56000640 1 -- 172.21.15.18:0/1471983342 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e4c0037a0 con 0x7f4e58100f40 2024-04-16T17:28:08.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.340+0000 7f4e56000640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e400a3d10 0x7f4e400a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:08.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.340+0000 7f4e56000640 1 -- 172.21.15.18:0/1471983342 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9724+0+0 (secure 0 0 0) 0x7f4e4c0d0ff0 con 0x7f4e58100f40 2024-04-16T17:28:08.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.340+0000 7f4e5cc00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e400a3d10 0x7f4e400a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:08.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.341+0000 7f4e5cc00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e400a3d10 0x7f4e400a61d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4e58103cb0 tx=0x7f4e44005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:08.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.344+0000 7f4e56000640 1 -- 172.21.15.18:0/1471983342 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e4c09d1c0 con 0x7f4e58100f40 2024-04-16T17:28:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: pgmap v749: 129 pgs: 129 active+clean; 456 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 364 KiB/s wr, 31 op/s 2024-04-16T17:28:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/2821478632' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:28:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:28:08.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T17:28:08.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T17:28:08.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: Cluster is now healthy 2024-04-16T17:28:08.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T17:28:08.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.43:6834/139205430,v1:172.21.15.43:6835/139205430] up:boot 2024-04-16T17:28:08.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.43:6838/1562229131,v1:172.21.15.43:6839/1562229131] up:boot 2024-04-16T17:28:08.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.43:6836/1279243918,v1:172.21.15.43:6837/1279243918] up:boot 2024-04-16T17:28:08.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6835/1332121044,v1:172.21.15.22:6839/1332121044] up:boot 2024-04-16T17:28:08.424 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.18:6838/1765882812,v1:172.21.15.18:6839/1765882812] up:boot 2024-04-16T17:28:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: mds.? [v2:172.21.15.22:6834/1574508556,v1:172.21.15.22:6838/1574508556] up:boot 2024-04-16T17:28:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: fsmap 12 up:standby 2024-04-16T17:28:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:28:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:28:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:28:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:28:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:28:08.425 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:08 smithi022 ceph-mon[36023]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:28:08.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: pgmap v749: 129 pgs: 129 active+clean; 456 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 364 KiB/s wr, 31 op/s 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/2821478632' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: Cluster is now healthy 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.43:6834/139205430,v1:172.21.15.43:6835/139205430] up:boot 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.43:6838/1562229131,v1:172.21.15.43:6839/1562229131] up:boot 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.43:6836/1279243918,v1:172.21.15.43:6837/1279243918] up:boot 2024-04-16T17:28:08.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6835/1332121044,v1:172.21.15.22:6839/1332121044] up:boot 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.18:6838/1765882812,v1:172.21.15.18:6839/1765882812] up:boot 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: mds.? [v2:172.21.15.22:6834/1574508556,v1:172.21.15.22:6838/1574508556] up:boot 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: fsmap 12 up:standby 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:28:08.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:08 smithi018 ceph-mon[31776]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:28:08.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: pgmap v749: 129 pgs: 129 active+clean; 456 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 364 KiB/s wr, 31 op/s 2024-04-16T17:28:08.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/2821478632' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:28:08.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:28:08.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T17:28:08.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T17:28:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: Cluster is now healthy 2024-04-16T17:28:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T17:28:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.43:6834/139205430,v1:172.21.15.43:6835/139205430] up:boot 2024-04-16T17:28:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.43:6838/1562229131,v1:172.21.15.43:6839/1562229131] up:boot 2024-04-16T17:28:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.43:6836/1279243918,v1:172.21.15.43:6837/1279243918] up:boot 2024-04-16T17:28:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6835/1332121044,v1:172.21.15.22:6839/1332121044] up:boot 2024-04-16T17:28:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.18:6838/1765882812,v1:172.21.15.18:6839/1765882812] up:boot 2024-04-16T17:28:08.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: mds.? [v2:172.21.15.22:6834/1574508556,v1:172.21.15.22:6838/1574508556] up:boot 2024-04-16T17:28:08.506 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: fsmap 12 up:standby 2024-04-16T17:28:08.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:28:08.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:28:08.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:28:08.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:28:08.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:28:08.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:08 smithi043 ceph-mon[36901]: from='mgr.25417 172.21.15.18:0/1192623112' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:28:08.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:08.571+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4e581018f0 con 0x7f4e58100f40 2024-04-16T17:28:09.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.115+0000 7f4e56000640 1 -- 172.21.15.18:0/1471983342 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v141) ==== 181+0+0 (secure 0 0 0) 0x7f4e4c09cac0 con 0x7f4e58100f40 2024-04-16T17:28:09.120 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_metadata' removed 2024-04-16T17:28:09.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.119+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e400a3d10 msgr2=0x7f4e400a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:09.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.119+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e400a3d10 0x7f4e400a61d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4e58103cb0 tx=0x7f4e44005f90 comp rx=0 tx=0).stop 2024-04-16T17:28:09.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.119+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58100f40 msgr2=0x7f4e58198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:09.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.119+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58100f40 0x7f4e58198700 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00e9c0 tx=0x7f4e4c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:28:09.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.120+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 shutdown_connections 2024-04-16T17:28:09.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.120+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f4e400a3d10 0x7f4e400a61d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.120+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e5819cf20 0x7f4e5819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.120+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e58106010 0x7f4e58198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.120+0000 7f4e5ea00640 1 --2- 172.21.15.18:0/1471983342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e58100f40 0x7f4e58198700 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.120+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 >> 172.21.15.18:0/1471983342 conn(0x7f4e58077ab0 msgr2=0x7f4e580fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:09.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.120+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 shutdown_connections 2024-04-16T17:28:09.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.120+0000 7f4e5ea00640 1 -- 172.21.15.18:0/1471983342 wait complete. 2024-04-16T17:28:09.147 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-16T17:28:09.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 -- 172.21.15.18:0/1729486002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42b80743a0 msgr2=0x7f42b8074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 --2- 172.21.15.18:0/1729486002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42b80743a0 0x7f42b8074780 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009900 tx=0x7f42a802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 -- 172.21.15.18:0/1729486002 shutdown_connections 2024-04-16T17:28:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 --2- 172.21.15.18:0/1729486002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42b8112e10 0x7f42b8115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 --2- 172.21.15.18:0/1729486002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42b8074cc0 0x7f42b81128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 --2- 172.21.15.18:0/1729486002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42b80743a0 0x7f42b8074780 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 -- 172.21.15.18:0/1729486002 >> 172.21.15.18:0/1729486002 conn(0x7f42b80fd950 msgr2=0x7f42b80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:09.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 -- 172.21.15.18:0/1729486002 shutdown_connections 2024-04-16T17:28:09.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.355+0000 7f42b7e00640 1 -- 172.21.15.18:0/1729486002 wait complete. 2024-04-16T17:28:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.357+0000 7f42b7e00640 1 Processor -- start 2024-04-16T17:28:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.357+0000 7f42b7e00640 1 -- start start 2024-04-16T17:28:09.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.357+0000 7f42b7e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42b80743a0 0x7f42b819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42b8074cc0 0x7f42b819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42b8112e10 0x7f42b81a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b7e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42b8118800 con 0x7f42b80743a0 2024-04-16T17:28:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b7e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f42b8118680 con 0x7f42b8074cc0 2024-04-16T17:28:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f42b8118980 con 0x7f42b8112e10 2024-04-16T17:28:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42b80743a0 0x7f42b819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42b8112e10 0x7f42b81a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:09.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b6a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42b80743a0 0x7f42b819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43424/0 (socket says 172.21.15.18:43424) 2024-04-16T17:28:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b6a00640 1 -- 172.21.15.18:0/3186377464 learned_addr learned my addr 172.21.15.18:0/3186377464 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b6000640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42b8074cc0 0x7f42b819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b6a00640 1 -- 172.21.15.18:0/3186377464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42b8112e10 msgr2=0x7f42b81a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b6a00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42b8112e10 0x7f42b81a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.358+0000 7f42b6a00640 1 -- 172.21.15.18:0/3186377464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42b8074cc0 msgr2=0x7f42b819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:09.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.359+0000 7f42b6a00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42b8074cc0 0x7f42b819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:09.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.359+0000 7f42b6a00640 1 -- 172.21.15.18:0/3186377464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b81a3d70 con 0x7f42b80743a0 2024-04-16T17:28:09.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.359+0000 7f42b6000640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42b8074cc0 0x7f42b819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:28:09.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.359+0000 7f42b6a00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42b80743a0 0x7f42b819ed80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f42a8036040 tx=0x7f42a802cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:09.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.360+0000 7f42bca00640 1 -- 172.21.15.18:0/3186377464 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8043020 con 0x7f42b80743a0 2024-04-16T17:28:09.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.360+0000 7f42bca00640 1 -- 172.21.15.18:0/3186377464 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42a80054e0 con 0x7f42b80743a0 2024-04-16T17:28:09.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.360+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b81a4000 con 0x7f42b80743a0 2024-04-16T17:28:09.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.360+0000 7f42bca00640 1 -- 172.21.15.18:0/3186377464 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8003c20 con 0x7f42b80743a0 2024-04-16T17:28:09.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.361+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b81ab8f0 con 0x7f42b80743a0 2024-04-16T17:28:09.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.362+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b804fd90 con 0x7f42b80743a0 2024-04-16T17:28:09.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.365+0000 7f42bca00640 1 -- 172.21.15.18:0/3186377464 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f42a8046020 con 0x7f42b80743a0 2024-04-16T17:28:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.367+0000 7f42bca00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f429c0a3d20 0x7f429c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.367+0000 7f42bca00640 1 -- 172.21.15.18:0/3186377464 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9170+0+0 (secure 0 0 0) 0x7f42a80f2760 con 0x7f42b80743a0 2024-04-16T17:28:09.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.368+0000 7f42b6000640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f429c0a3d20 0x7f429c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.369+0000 7f42b6000640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f429c0a3d20 0x7f429c0a61e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f42b8105c20 tx=0x7f42a4005f90 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:09.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.374+0000 7f42bca00640 1 -- 172.21.15.18:0/3186377464 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42a804d070 con 0x7f42b80743a0 2024-04-16T17:28:09.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:09 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1471983342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:09.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:09 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1471983342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:09.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:09 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1471983342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:09.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:09.683+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f42b81052c0 con 0x7f42b80743a0 2024-04-16T17:28:10.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.118+0000 7f42bca00640 1 -- 172.21.15.18:0/3186377464 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v142) ==== 169+0+0 (secure 0 0 0) 0x7f42a80bec20 con 0x7f42b80743a0 2024-04-16T17:28:10.119 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_data' removed 2024-04-16T17:28:10.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.123+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f429c0a3d20 msgr2=0x7f429c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:10.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.123+0000 7f42b7e00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f429c0a3d20 0x7f429c0a61e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f42b8105c20 tx=0x7f42a4005f90 comp rx=0 tx=0).stop 2024-04-16T17:28:10.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.123+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42b80743a0 msgr2=0x7f42b819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:10.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.123+0000 7f42b7e00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42b80743a0 0x7f42b819ed80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f42a8036040 tx=0x7f42a802cbc0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.124+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 shutdown_connections 2024-04-16T17:28:10.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.124+0000 7f42b7e00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f429c0a3d20 0x7f429c0a61e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.124+0000 7f42b7e00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42b8112e10 0x7f42b81a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.124+0000 7f42b7e00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42b8074cc0 0x7f42b819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.124+0000 7f42b7e00640 1 --2- 172.21.15.18:0/3186377464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42b80743a0 0x7f42b819ed80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.124+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 >> 172.21.15.18:0/3186377464 conn(0x7f42b80fd950 msgr2=0x7f42b80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:10.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.124+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 shutdown_connections 2024-04-16T17:28:10.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.124+0000 7f42b7e00640 1 -- 172.21.15.18:0/3186377464 wait complete. 2024-04-16T17:28:10.152 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-16T17:28:10.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 -- 172.21.15.18:0/136900824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37941018f0 msgr2=0x7f3794105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:10.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 --2- 172.21.15.18:0/136900824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37941018f0 0x7f3794105930 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f377c009ca0 tx=0x7f377c02c0f0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 -- 172.21.15.18:0/136900824 shutdown_connections 2024-04-16T17:28:10.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 --2- 172.21.15.18:0/136900824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794106010 0x7f379410dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 --2- 172.21.15.18:0/136900824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37941018f0 0x7f3794105930 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 --2- 172.21.15.18:0/136900824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3794100f40 0x7f3794101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 -- 172.21.15.18:0/136900824 >> 172.21.15.18:0/136900824 conn(0x7f3794077ab0 msgr2=0x7f37940fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:10.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 -- 172.21.15.18:0/136900824 shutdown_connections 2024-04-16T17:28:10.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.344+0000 7f3793e00640 1 -- 172.21.15.18:0/136900824 wait complete. 2024-04-16T17:28:10.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.345+0000 7f3793e00640 1 Processor -- start 2024-04-16T17:28:10.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.345+0000 7f3793e00640 1 -- start start 2024-04-16T17:28:10.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.345+0000 7f3793e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794100f40 0x7f37941a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:10.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.345+0000 7f3793e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3794106010 0x7f37941a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:10.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f379419a700 0x7f379419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:10.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f37941110a0 con 0x7f3794106010 2024-04-16T17:28:10.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3794110f20 con 0x7f379419a700 2024-04-16T17:28:10.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3794111220 con 0x7f3794100f40 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f379419a700 0x7f379419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3792a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794100f40 0x7f37941a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3792000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3794106010 0x7f37941a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f379419a700 0x7f379419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.18:40636/0 (socket says 172.21.15.18:40636) 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3792a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794100f40 0x7f37941a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.18:43022/0 (socket says 172.21.15.18:43022) 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793400640 1 -- 172.21.15.18:0/3685242786 learned_addr learned my addr 172.21.15.18:0/3685242786 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793400640 1 -- 172.21.15.18:0/3685242786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794100f40 msgr2=0x7f37941a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793400640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794100f40 0x7f37941a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793400640 1 -- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3794106010 msgr2=0x7f37941a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:10.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793400640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3794106010 0x7f37941a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:10.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3793400640 1 -- 172.21.15.18:0/3685242786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379419b300 con 0x7f379419a700 2024-04-16T17:28:10.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3792a00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794100f40 0x7f37941a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:28:10.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.346+0000 7f3792000640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3794106010 0x7f37941a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:28:10.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.347+0000 7f3793400640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f379419a700 0x7f379419ab80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f378c00e970 tx=0x7f378c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:10.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.347+0000 7f378b400640 1 -- 172.21.15.18:0/3685242786 <== mon.2 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c00ccc0 con 0x7f379419a700 2024-04-16T17:28:10.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.347+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37941a73e0 con 0x7f379419a700 2024-04-16T17:28:10.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.347+0000 7f378b400640 1 -- 172.21.15.18:0/3685242786 <== mon.2 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f378c00ce60 con 0x7f379419a700 2024-04-16T17:28:10.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.347+0000 7f378b400640 1 -- 172.21.15.18:0/3685242786 <== mon.2 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c010450 con 0x7f379419a700 2024-04-16T17:28:10.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.347+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37941a7790 con 0x7f379419a700 2024-04-16T17:28:10.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.348+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3794102410 con 0x7f379419a700 2024-04-16T17:28:10.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.350+0000 7f378b400640 1 -- 172.21.15.18:0/3685242786 <== mon.2 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f378c005020 con 0x7f379419a700 2024-04-16T17:28:10.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.351+0000 7f378b400640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f37740a3d10 0x7f37740a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:28:10.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.351+0000 7f378b400640 1 -- 172.21.15.18:0/3685242786 <== mon.2 v2:172.21.15.22:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8582+0+0 (secure 0 0 0) 0x7f378c0d0c70 con 0x7f379419a700 2024-04-16T17:28:10.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.352+0000 7f3792a00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f37740a3d10 0x7f37740a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:28:10.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.352+0000 7f3792a00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f37740a3d10 0x7f37740a61d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3784005d00 tx=0x7f378400b410 comp rx=0 tx=0).ready entity=mgr.25417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:28:10.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.356+0000 7f378b400640 1 -- 172.21.15.18:0/3685242786 <== mon.2 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f378c09d270 con 0x7f379419a700 2024-04-16T17:28:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:10 smithi022 ceph-mon[36023]: pgmap v751: 129 pgs: 129 active+clean; 456 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 427 KiB/s wr, 32 op/s 2024-04-16T17:28:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:10 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/1471983342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:10 smithi022 ceph-mon[36023]: osdmap e141: 12 total, 12 up, 12 in 2024-04-16T17:28:10.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:10 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3186377464' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:10 smithi018 ceph-mon[31776]: pgmap v751: 129 pgs: 129 active+clean; 456 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 427 KiB/s wr, 32 op/s 2024-04-16T17:28:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:10 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/1471983342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:10 smithi018 ceph-mon[31776]: osdmap e141: 12 total, 12 up, 12 in 2024-04-16T17:28:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:10 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3186377464' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:10.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:10 smithi043 ceph-mon[36901]: pgmap v751: 129 pgs: 129 active+clean; 456 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 427 KiB/s wr, 32 op/s 2024-04-16T17:28:10.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:10 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/1471983342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:10.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:10 smithi043 ceph-mon[36901]: osdmap e141: 12 total, 12 up, 12 in 2024-04-16T17:28:10.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:10 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3186377464' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:10.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:10.624+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f37940008d0 con 0x7f379419a700 2024-04-16T17:28:11.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.134+0000 7f378b400640 1 -- 172.21.15.18:0/3685242786 <== mon.2 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v143) ==== 178+0+0 (secure 0 0 0) 0x7f378c09cb70 con 0x7f379419a700 2024-04-16T17:28:11.134 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_data_ec' removed 2024-04-16T17:28:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f37740a3d10 msgr2=0x7f37740a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:11.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f37740a3d10 0x7f37740a61d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3784005d00 tx=0x7f378400b410 comp rx=0 tx=0).stop 2024-04-16T17:28:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f379419a700 msgr2=0x7f379419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:28:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f379419a700 0x7f379419ab80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f378c00e970 tx=0x7f378c00ed70 comp rx=0 tx=0).stop 2024-04-16T17:28:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 shutdown_connections 2024-04-16T17:28:11.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:6800/3776183549,v1:172.21.15.18:6801/3776183549] conn(0x7f37740a3d10 0x7f37740a61d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f379419a700 0x7f379419ab80 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3794106010 0x7f37941a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 --2- 172.21.15.18:0/3685242786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3794100f40 0x7f37941a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:28:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 >> 172.21.15.18:0/3685242786 conn(0x7f3794077ab0 msgr2=0x7f3794069320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:28:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 shutdown_connections 2024-04-16T17:28:11.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-16T17:28:11.137+0000 7f3793e00640 1 -- 172.21.15.18:0/3685242786 wait complete. 2024-04-16T17:28:11.159 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:28:11.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:28:11.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:28:11.351 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-16T17:28:11.351 DEBUG:teuthology.orchestra.run.smithi018:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T17:28:11.414 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T17:28:11.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:11 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3186377464' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:11.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:11 smithi022 ceph-mon[36023]: osdmap e142: 12 total, 12 up, 12 in 2024-04-16T17:28:11.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:11 smithi022 ceph-mon[36023]: from='client.? 172.21.15.18:0/3685242786' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:11.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:11 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:11.430 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T17:28:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:11 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3186377464' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:11 smithi018 ceph-mon[31776]: osdmap e142: 12 total, 12 up, 12 in 2024-04-16T17:28:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:11 smithi018 ceph-mon[31776]: from='client.? 172.21.15.18:0/3685242786' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:11 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:11.451 INFO:tasks.cephadm:Stopping all daemons... 2024-04-16T17:28:11.452 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-16T17:28:11.452 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a 2024-04-16T17:28:11.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:11 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3186377464' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:11.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:11 smithi043 ceph-mon[36901]: osdmap e142: 12 total, 12 up, 12 in 2024-04-16T17:28:11.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:11 smithi043 ceph-mon[36901]: from='client.? 172.21.15.18:0/3685242786' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:11.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:11 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:28:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:11 smithi018 systemd[1]: Stopping Ceph mon.a for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:12 smithi022 ceph-mon[36023]: pgmap v754: 33 pgs: 33 active+clean; 330 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:28:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:12 smithi022 ceph-mon[36023]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:12.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:12 smithi022 ceph-mon[36023]: osdmap e143: 12 total, 12 up, 12 in 2024-04-16T17:28:12.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:12 smithi018 ceph-mon[31776]: pgmap v754: 33 pgs: 33 active+clean; 330 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:28:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:12 smithi018 ceph-mon[31776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:12 smithi018 ceph-mon[31776]: osdmap e143: 12 total, 12 up, 12 in 2024-04-16T17:28:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:12 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a[31772]: 2024-04-16T17:28:12.204+0000 7f7b0f200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:28:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:12 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a[31772]: 2024-04-16T17:28:12.204+0000 7f7b0f200700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-16T17:28:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:12 smithi018 podman[115311]: 2024-04-16 17:28:12.288327465 +0000 UTC m=+0.297209237 container died 387238bc74ca4531be73c90bd119996ff34b8b94cee7b79c277dd8f8d969fe49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:28:12.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:12 smithi043 ceph-mon[36901]: pgmap v754: 33 pgs: 33 active+clean; 330 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:28:12.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:12 smithi043 ceph-mon[36901]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:28:12.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:12 smithi043 ceph-mon[36901]: osdmap e143: 12 total, 12 up, 12 in 2024-04-16T17:28:13.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:12 smithi018 podman[115311]: 2024-04-16 17:28:12.935575762 +0000 UTC m=+0.944457586 container cleanup 387238bc74ca4531be73c90bd119996ff34b8b94cee7b79c277dd8f8d969fe49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:28:13.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:12 smithi018 bash[115311]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a 2024-04-16T17:28:13.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Apr 16 17:28:13 smithi018 podman[115321]: 2024-04-16 17:28:13.317382443 +0000 UTC m=+1.019872018 container remove 387238bc74ca4531be73c90bd119996ff34b8b94cee7b79c277dd8f8d969fe49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-16T17:28:13.844 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.a.service' 2024-04-16T17:28:14.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:14.002 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-16T17:28:14.002 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-16T17:28:14.002 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.b 2024-04-16T17:28:14.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:14 smithi022 systemd[1]: Stopping Ceph mon.b for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:15.103 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:14 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-b[36019]: 2024-04-16T17:28:14.687+0000 7f0d21c23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:28:15.103 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:14 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-b[36019]: 2024-04-16T17:28:14.687+0000 7f0d21c23700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-16T17:28:15.103 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:14 smithi022 podman[75746]: 2024-04-16 17:28:14.780340241 +0000 UTC m=+0.486561744 container died a4591baa9f1d26bebcf50b1b60b30ce803393b55e62764d0ca2a03b402c84af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:28:15.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:15 smithi022 podman[75746]: 2024-04-16 17:28:15.238380214 +0000 UTC m=+0.944601711 container cleanup a4591baa9f1d26bebcf50b1b60b30ce803393b55e62764d0ca2a03b402c84af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-b, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409) 2024-04-16T17:28:15.423 INFO:journalctl@ceph.mon.b.smithi022.stdout:Apr 16 17:28:15 smithi022 bash[75746]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-b 2024-04-16T17:28:15.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.b.service' 2024-04-16T17:28:15.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:15.726 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-16T17:28:15.726 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-16T17:28:15.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.c 2024-04-16T17:28:16.053 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:15 smithi043 systemd[1]: Stopping Ceph mon.c for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:16.500 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:16 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-c[36897]: 2024-04-16T17:28:16.053+0000 7f0805c6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:28:16.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:16 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-c[36897]: 2024-04-16T17:28:16.053+0000 7f0805c6b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-16T17:28:16.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:16 smithi043 podman[76277]: 2024-04-16 17:28:16.279527742 +0000 UTC m=+0.342422430 container died 986fbbd02f667355d38ac3e619e74c759457b68a591beb00f0f110b92d970ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-16T17:28:16.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:16 smithi043 podman[76277]: 2024-04-16 17:28:16.688130565 +0000 UTC m=+0.751025253 container cleanup 986fbbd02f667355d38ac3e619e74c759457b68a591beb00f0f110b92d970ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-16T17:28:16.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Apr 16 17:28:16 smithi043 bash[76277]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mon-c 2024-04-16T17:28:17.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mon.c.service' 2024-04-16T17:28:17.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:17.108 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-16T17:28:17.108 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-16T17:28:17.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.x 2024-04-16T17:28:17.444 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:28:17 smithi018 systemd[1]: Stopping Ceph mgr.x for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:18.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:28:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:28:17.712+0000 7f3566400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:28:18.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:28:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x[32013]: 2024-04-16T17:28:17.712+0000 7f3566400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-16T17:28:18.196 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:28:17 smithi018 podman[115428]: 2024-04-16 17:28:17.809284473 +0000 UTC m=+0.307027335 container died 29fb07aec5fd4486c0d03ff8bf8b85a785bd74f5adf57ada3d7853d11517497c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-16T17:28:18.591 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:28:18 smithi018 podman[115428]: 2024-04-16 17:28:18.584895568 +0000 UTC m=+1.082638405 container cleanup 29fb07aec5fd4486c0d03ff8bf8b85a785bd74f5adf57ada3d7853d11517497c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-16T17:28:18.592 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:28:18 smithi018 bash[115428]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x 2024-04-16T17:28:19.194 INFO:journalctl@ceph.mgr.x.smithi018.stdout:Apr 16 17:28:18 smithi018 podman[115442]: 2024-04-16 17:28:18.938460234 +0000 UTC m=+1.120455273 container remove 29fb07aec5fd4486c0d03ff8bf8b85a785bd74f5adf57ada3d7853d11517497c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-x, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-16T17:28:19.458 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.x.service' 2024-04-16T17:28:19.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:19.564 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-16T17:28:19.565 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-16T17:28:19.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.y 2024-04-16T17:28:19.913 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:28:19 smithi022 systemd[1]: Stopping Ceph mgr.y for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:20.173 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:28:19 smithi022 podman[75868]: 2024-04-16 17:28:19.914079985 +0000 UTC m=+0.138880058 container died d5136006e60ccf184225388893b2478f613e78c27060a403fbd7bc527847cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-16T17:28:20.673 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:28:20 smithi022 podman[75868]: 2024-04-16 17:28:20.251718011 +0000 UTC m=+0.476518082 container cleanup d5136006e60ccf184225388893b2478f613e78c27060a403fbd7bc527847cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-16T17:28:20.673 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:28:20 smithi022 bash[75868]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y 2024-04-16T17:28:20.673 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:28:20 smithi022 podman[75882]: 2024-04-16 17:28:20.470126101 +0000 UTC m=+0.548797279 container remove d5136006e60ccf184225388893b2478f613e78c27060a403fbd7bc527847cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-16T17:28:20.673 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Apr 16 17:28:20 smithi022 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-16T17:28:20.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.y.service' 2024-04-16T17:28:20.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:20.763 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-16T17:28:20.763 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-16T17:28:20.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.z 2024-04-16T17:28:21.183 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:28:20 smithi043 systemd[1]: Stopping Ceph mgr.z for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:21.501 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:28:21 smithi043 podman[76403]: 2024-04-16 17:28:21.184772915 +0000 UTC m=+0.217082315 container died 4749a67dbb9e422736ddd1b0a07e9ff26a70c2bb3071575a7c3b3a3b8446ece3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:28:21.919 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:28:21 smithi043 podman[76403]: 2024-04-16 17:28:21.635552248 +0000 UTC m=+0.667861660 container cleanup 4749a67dbb9e422736ddd1b0a07e9ff26a70c2bb3071575a7c3b3a3b8446ece3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-16T17:28:21.919 INFO:journalctl@ceph.mgr.z.smithi043.stdout:Apr 16 17:28:21 smithi043 bash[76403]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mgr-z 2024-04-16T17:28:22.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mgr.z.service' 2024-04-16T17:28:22.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:22.107 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-16T17:28:22.108 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-16T17:28:22.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.0 2024-04-16T17:28:22.445 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:22 smithi018 systemd[1]: Stopping Ceph osd.0 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:23.195 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0[44006]: 2024-04-16T17:28:22.779+0000 7f119de00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:28:23.195 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0[44006]: 2024-04-16T17:28:22.779+0000 7f119de00700 -1 osd.0 143 *** Got signal Terminated *** 2024-04-16T17:28:23.195 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0[44006]: 2024-04-16T17:28:22.779+0000 7f119de00700 -1 osd.0 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:28:28.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:28 smithi018 podman[115545]: 2024-04-16 17:28:28.397656699 +0000 UTC m=+5.856447097 container died 52b5ee851b1788972ecf41a019e71ec0277bd96bd1c3b256f96050ab23c25e01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-16T17:28:29.194 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:29 smithi018 podman[115545]: 2024-04-16 17:28:29.094668051 +0000 UTC m=+6.553458519 container cleanup 52b5ee851b1788972ecf41a019e71ec0277bd96bd1c3b256f96050ab23c25e01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409) 2024-04-16T17:28:29.194 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:29 smithi018 bash[115545]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0 2024-04-16T17:28:29.944 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:29 smithi018 podman[115563]: 2024-04-16 17:28:29.572417275 +0000 UTC m=+1.163680475 container remove 52b5ee851b1788972ecf41a019e71ec0277bd96bd1c3b256f96050ab23c25e01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-16T17:28:30.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:30 smithi018 podman[115647]: 2024-04-16 17:28:30.318148419 +0000 UTC m=+0.037933496 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:28:30.696 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:30 smithi018 podman[115647]: 2024-04-16 17:28:30.490764724 +0000 UTC m=+0.210549841 container create 0650a5290fde8700b470b70a5bdcc871fcd7029b59897f860d41fd05283f7c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-16T17:28:31.137 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:30 smithi018 podman[115647]: 2024-04-16 17:28:30.971714886 +0000 UTC m=+0.691499983 container init 0650a5290fde8700b470b70a5bdcc871fcd7029b59897f860d41fd05283f7c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:28:31.137 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:30 smithi018 podman[115647]: 2024-04-16 17:28:30.983192178 +0000 UTC m=+0.702977275 container start 0650a5290fde8700b470b70a5bdcc871fcd7029b59897f860d41fd05283f7c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=) 2024-04-16T17:28:31.137 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:31 smithi018 podman[115647]: 2024-04-16 17:28:31.137159611 +0000 UTC m=+0.856944741 container attach 0650a5290fde8700b470b70a5bdcc871fcd7029b59897f860d41fd05283f7c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:28:31.944 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 16 17:28:31 smithi018 podman[115647]: 2024-04-16 17:28:31.669640597 +0000 UTC m=+1.389425727 container died 0650a5290fde8700b470b70a5bdcc871fcd7029b59897f860d41fd05283f7c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-16T17:28:32.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.0.service' 2024-04-16T17:28:33.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:33.055 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-16T17:28:33.055 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-16T17:28:33.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.1 2024-04-16T17:28:33.372 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:33 smithi022 systemd[1]: Stopping Ceph osd.1 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:33.673 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:33 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1[40615]: 2024-04-16T17:28:33.372+0000 7f0aed889700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:28:33.673 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:33 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1[40615]: 2024-04-16T17:28:33.372+0000 7f0aed889700 -1 osd.1 143 *** Got signal Terminated *** 2024-04-16T17:28:33.674 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:33 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1[40615]: 2024-04-16T17:28:33.372+0000 7f0aed889700 -1 osd.1 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:28:39.289 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:38 smithi022 podman[75990]: 2024-04-16 17:28:38.991725804 +0000 UTC m=+5.732813239 container died e2558225937c2eb7b2810fd3a2da4bfddb817509ef7ab4b59737f035bd14da30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-16T17:28:39.640 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:39 smithi022 podman[75990]: 2024-04-16 17:28:39.38160946 +0000 UTC m=+6.122696906 container cleanup e2558225937c2eb7b2810fd3a2da4bfddb817509ef7ab4b59737f035bd14da30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:28:39.640 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:39 smithi022 bash[75990]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1 2024-04-16T17:28:39.923 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:39 smithi022 podman[76005]: 2024-04-16 17:28:39.641538405 +0000 UTC m=+0.648289762 container remove e2558225937c2eb7b2810fd3a2da4bfddb817509ef7ab4b59737f035bd14da30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:28:40.270 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:39 smithi022 podman[76101]: 2024-04-16 17:28:39.865945566 +0000 UTC m=+0.018082752 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:28:40.270 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:39 smithi022 podman[76101]: 2024-04-16 17:28:39.99291372 +0000 UTC m=+0.145050972 container create 7cb0027f46b81c0ad264bfd795ec97cd609b0213952808ac65c5c68fbd55c976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-16T17:28:40.270 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:40 smithi022 podman[76101]: 2024-04-16 17:28:40.270619094 +0000 UTC m=+0.422756279 container init 7cb0027f46b81c0ad264bfd795ec97cd609b0213952808ac65c5c68fbd55c976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-16T17:28:40.673 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:40 smithi022 podman[76101]: 2024-04-16 17:28:40.274824516 +0000 UTC m=+0.426961685 container start 7cb0027f46b81c0ad264bfd795ec97cd609b0213952808ac65c5c68fbd55c976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-16T17:28:40.673 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:40 smithi022 podman[76101]: 2024-04-16 17:28:40.387197126 +0000 UTC m=+0.539334320 container attach 7cb0027f46b81c0ad264bfd795ec97cd609b0213952808ac65c5c68fbd55c976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-16T17:28:41.122 INFO:journalctl@ceph.osd.1.smithi022.stdout:Apr 16 17:28:40 smithi022 podman[76101]: 2024-04-16 17:28:40.852047347 +0000 UTC m=+1.004184529 container died 7cb0027f46b81c0ad264bfd795ec97cd609b0213952808ac65c5c68fbd55c976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD) 2024-04-16T17:28:41.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.1.service' 2024-04-16T17:28:41.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:41.482 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-16T17:28:41.482 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-16T17:28:41.482 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.2 2024-04-16T17:28:41.921 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:41 smithi043 systemd[1]: Stopping Ceph osd.2 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:42.204 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2[42083]: 2024-04-16T17:28:41.921+0000 7f3a9a8f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:28:42.204 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2[42083]: 2024-04-16T17:28:41.921+0000 7f3a9a8f0700 -1 osd.2 143 *** Got signal Terminated *** 2024-04-16T17:28:42.204 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2[42083]: 2024-04-16T17:28:41.921+0000 7f3a9a8f0700 -1 osd.2 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:28:47.906 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:47 smithi043 podman[76528]: 2024-04-16 17:28:47.512145094 +0000 UTC m=+5.823488614 container died a1f0e6eb9fb006ee497a77e41c510f86c11d304e153b0f60353ab407929c3157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-16T17:28:48.250 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:48 smithi043 podman[76528]: 2024-04-16 17:28:48.114431337 +0000 UTC m=+6.425774851 container cleanup a1f0e6eb9fb006ee497a77e41c510f86c11d304e153b0f60353ab407929c3157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-16T17:28:48.251 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:48 smithi043 bash[76528]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2 2024-04-16T17:28:48.739 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:48 smithi043 podman[76557]: 2024-04-16 17:28:48.299664625 +0000 UTC m=+0.783535534 container remove a1f0e6eb9fb006ee497a77e41c510f86c11d304e153b0f60353ab407929c3157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:28:49.001 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:48 smithi043 podman[76650]: 2024-04-16 17:28:48.644472432 +0000 UTC m=+0.017806127 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:28:49.001 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:48 smithi043 podman[76650]: 2024-04-16 17:28:48.758834116 +0000 UTC m=+0.132167800 container create 52008c6b53787e3a57d62c92adcf08c4b7c649dfa4ec7547f045f2dac7d19a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-16T17:28:49.345 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:49 smithi043 podman[76650]: 2024-04-16 17:28:49.045024051 +0000 UTC m=+0.418357737 container init 52008c6b53787e3a57d62c92adcf08c4b7c649dfa4ec7547f045f2dac7d19a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0) 2024-04-16T17:28:49.345 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:49 smithi043 podman[76650]: 2024-04-16 17:28:49.04822457 +0000 UTC m=+0.421558250 container start 52008c6b53787e3a57d62c92adcf08c4b7c649dfa4ec7547f045f2dac7d19a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-16T17:28:49.345 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:49 smithi043 podman[76650]: 2024-04-16 17:28:49.153730294 +0000 UTC m=+0.527063996 container attach 52008c6b53787e3a57d62c92adcf08c4b7c649dfa4ec7547f045f2dac7d19a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:28:49.695 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:49 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:49.352+0000 7fb312400700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.836441+0000 front 2024-04-16T17:28:24.836538+0000 (oldest deadline 2024-04-16T17:28:48.936263+0000) 2024-04-16T17:28:49.697 INFO:journalctl@ceph.osd.2.smithi043.stdout:Apr 16 17:28:49 smithi043 podman[76650]: 2024-04-16 17:28:49.34646422 +0000 UTC m=+0.719797909 container died 52008c6b53787e3a57d62c92adcf08c4b7c649dfa4ec7547f045f2dac7d19a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-16T17:28:49.990 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:49.946+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:50.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.2.service' 2024-04-16T17:28:50.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:28:50.064 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-16T17:28:50.064 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-16T17:28:50.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.3 2024-04-16T17:28:50.444 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:50 smithi018 systemd[1]: Stopping Ceph osd.3 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:28:50.445 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:50 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:50.313+0000 7fb312400700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.836441+0000 front 2024-04-16T17:28:24.836538+0000 (oldest deadline 2024-04-16T17:28:48.936263+0000) 2024-04-16T17:28:50.945 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:50 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:50.720+0000 7fb31bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:28:50.945 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:50 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:50.720+0000 7fb31bc00700 -1 osd.3 143 *** Got signal Terminated *** 2024-04-16T17:28:50.945 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:50 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:50.720+0000 7fb31bc00700 -1 osd.3 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:28:50.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:50 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:50.658+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:51.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:50.809+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:51.250 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:50.962+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:51.251 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:50.811+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:51.658 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:51 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:51.355+0000 7fb312400700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.836441+0000 front 2024-04-16T17:28:24.836538+0000 (oldest deadline 2024-04-16T17:28:48.936263+0000) 2024-04-16T17:28:51.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:51 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:51.657+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:52.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:51.781+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:52.250 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:51.856+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:52.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:51.917+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:52.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:28:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:28:51.984+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:28:52.677 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:52 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:52.372+0000 7fb312400700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.836441+0000 front 2024-04-16T17:28:24.836538+0000 (oldest deadline 2024-04-16T17:28:48.936263+0000) 2024-04-16T17:28:52.934 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:52.849+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:52.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:52 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:52.676+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:53.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:52.778+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:53.250 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:52.934+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:53.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:28:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:28:52.946+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:28:53.670 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:53 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:53.403+0000 7fb312400700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.836441+0000 front 2024-04-16T17:28:24.836538+0000 (oldest deadline 2024-04-16T17:28:48.936263+0000) 2024-04-16T17:28:53.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:53 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:53.669+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:54.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:53.749+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:54.251 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:53.876+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:54.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:53.978+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:54.252 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:28:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:28:53.963+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:28:54.684 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:54 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:54.360+0000 7fb312400700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.836441+0000 front 2024-04-16T17:28:24.836538+0000 (oldest deadline 2024-04-16T17:28:48.936263+0000) 2024-04-16T17:28:54.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:54 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:54.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:55.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:54.745+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:55.250 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:54.840+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:55.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:54.967+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:55.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:28:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:28:54.950+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:28:55.694 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3[50857]: 2024-04-16T17:28:55.385+0000 7fb312400700 -1 osd.3 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.836441+0000 front 2024-04-16T17:28:24.836538+0000 (oldest deadline 2024-04-16T17:28:48.936263+0000) 2024-04-16T17:28:56.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:55.708+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:56.194 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:55 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:55.710+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:56.250 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:55.797+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:56.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:55.977+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:56.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:28:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:28:55.992+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:28:56.694 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:56 smithi018 podman[115750]: 2024-04-16 17:28:56.417810771 +0000 UTC m=+5.906955913 container died fe37aec92614149f475eec5a7cdbc5aec811232d8610716bf236cfa72304104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T17:28:56.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:56 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:56.663+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:56.963 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:56 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:56.710+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:57.010 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:56.942+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:57.011 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:56.752+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:57.237 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:57 smithi018 podman[115750]: 2024-04-16 17:28:57.201381042 +0000 UTC m=+6.690526194 container cleanup fe37aec92614149f475eec5a7cdbc5aec811232d8610716bf236cfa72304104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:28:57.238 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:57 smithi018 bash[115750]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3 2024-04-16T17:28:57.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:28:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:28:57.010+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:28:57.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:57.625+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:57.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:57 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:57.733+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:57.945 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:57 smithi018 podman[115768]: 2024-04-16 17:28:57.589184632 +0000 UTC m=+1.099890872 container remove fe37aec92614149f475eec5a7cdbc5aec811232d8610716bf236cfa72304104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-16T17:28:58.251 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:57.765+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:58.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:57.938+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:58.252 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:28:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:28:58.003+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:28:58.694 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:58 smithi018 podman[115852]: 2024-04-16 17:28:58.342186848 +0000 UTC m=+0.036988066 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:28:58.695 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:58 smithi018 podman[115852]: 2024-04-16 17:28:58.549207742 +0000 UTC m=+0.244008996 container create e9e4a42661f2aed2f24d2ff6fe1c227127fc08b4c7031c6699fb002ef4ea60ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-16T17:28:58.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:58 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:58.583+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:28:59.020 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:58 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:58.762+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:28:59.250 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:58.795+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:28:59.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:58.942+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:28:59.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:28:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:28:59.003+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:28:59.445 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:59 smithi018 podman[115852]: 2024-04-16 17:28:59.020865077 +0000 UTC m=+0.715666354 container init e9e4a42661f2aed2f24d2ff6fe1c227127fc08b4c7031c6699fb002ef4ea60ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-04-16T17:28:59.445 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:59 smithi018 podman[115852]: 2024-04-16 17:28:59.033436569 +0000 UTC m=+0.728237853 container start e9e4a42661f2aed2f24d2ff6fe1c227127fc08b4c7031c6699fb002ef4ea60ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:28:59.445 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:59 smithi018 podman[115852]: 2024-04-16 17:28:59.148932071 +0000 UTC m=+0.843733345 container attach e9e4a42661f2aed2f24d2ff6fe1c227127fc08b4c7031c6699fb002ef4ea60ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:28:59.773 INFO:journalctl@ceph.osd.3.smithi018.stdout:Apr 16 17:28:59 smithi018 podman[115852]: 2024-04-16 17:28:59.510480942 +0000 UTC m=+1.205282174 container died e9e4a42661f2aed2f24d2ff6fe1c227127fc08b4c7031c6699fb002ef4ea60ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:28:59.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:28:59 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:28:59.614+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:00.096 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:59 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:59.772+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:00.096 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:28:59 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:28:59.772+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:00.250 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:28:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:28:59.926+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:00.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:00.022+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:00.251 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:28:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:28:59.775+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:00.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.3.service' 2024-04-16T17:29:00.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:29:00.781 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-16T17:29:00.782 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-16T17:29:00.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.4 2024-04-16T17:29:00.923 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:00 smithi022 systemd[1]: Stopping Ceph osd.4 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:29:00.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:00 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:00.607+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:00.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:00 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:00.751+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:00.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:00 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:00.751+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:01.250 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:00.788+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:01.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:00.891+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:01.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:00.987+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:01.423 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:01 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4[47299]: 2024-04-16T17:29:01.117+0000 7ff6ec526700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:29:01.423 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:01 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4[47299]: 2024-04-16T17:29:01.117+0000 7ff6ec526700 -1 osd.4 143 *** Got signal Terminated *** 2024-04-16T17:29:01.423 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:01 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4[47299]: 2024-04-16T17:29:01.117+0000 7ff6ec526700 -1 osd.4 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:29:01.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:01 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:01.583+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:01.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:01 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:01.583+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:01.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:01 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:01.481+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:02.036 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:01.893+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:02.036 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:01.781+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:02.036 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:01.781+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:02.194 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:01 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:01.757+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:02.194 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:01 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:01.757+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:02.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:02.036+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:02.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:02 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:02.528+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:02.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:02 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:02.559+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:02.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:02 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:02.559+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:02.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:02 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:02.726+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:02.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:02 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:02.726+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:02.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:02 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:02.589+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:03.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:03.023+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:03.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:03.023+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:03.251 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:02.804+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:03.251 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:02.804+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:03.252 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:02.905+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:03.252 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:02.905+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:03.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:03 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:03.488+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:03.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:03 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:03.562+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:03.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:03 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:03.562+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:03.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:03 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:03.727+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:03.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:03 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:03.727+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:03.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:03 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:03.614+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:04.250 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:04.035+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:04.251 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:04.035+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:04.251 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:03.792+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:04.251 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:03.792+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:04.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:03.917+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:04.251 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:03.917+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:04.902 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:04 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:04.442+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:04.902 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:04 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:04.521+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:04.902 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:04 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:04.521+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:04.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:04 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:04.720+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:04.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:04 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:04.720+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:04.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:04 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:04.640+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:05.062 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:04.766+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:05.062 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:04.766+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:05.062 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:04.954+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:05.063 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:04.954+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:05.173 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:04 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4[47299]: 2024-04-16T17:29:04.902+0000 7ff6e8b1a700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:33.067982+0000 front 2024-04-16T17:28:33.068023+0000 (oldest deadline 2024-04-16T17:29:04.268914+0000) 2024-04-16T17:29:05.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:05.062+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:05.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:05.062+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:05.673 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:05 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:05.401+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:05.673 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:05 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:05.568+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:05.673 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:05 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:05.568+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:05.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:05 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:05.612+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:05.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:05 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:05.687+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:05.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:05 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:05.687+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:06.084 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:05.992+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:06.084 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:05.992+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:06.084 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:05.781+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:06.084 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:05.781+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:06.173 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:05 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4[47299]: 2024-04-16T17:29:05.909+0000 7ff6e8b1a700 -1 osd.4 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:33.067982+0000 front 2024-04-16T17:28:33.068023+0000 (oldest deadline 2024-04-16T17:29:04.268914+0000) 2024-04-16T17:29:06.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:06 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:06.084+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:06.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:06 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:06.084+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:06.909 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:06 smithi022 podman[76206]: 2024-04-16 17:29:06.607725163 +0000 UTC m=+5.613799535 container died d55c4e4f47157cabcdb19ef94f5059cccf64031ed852a1be3e0b679a75a212ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:29:06.910 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:06 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:06.425+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:06.910 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:06 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:06.572+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:06.910 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:06 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:06.572+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:06.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:06 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:06.594+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:06.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:06 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:06.703+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:06.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:06 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:06.703+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:07.000 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:06 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:06.744+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:07.001 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:06 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:06.744+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:07.173 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:07 smithi022 podman[76206]: 2024-04-16 17:29:07.00167577 +0000 UTC m=+6.007750158 container cleanup d55c4e4f47157cabcdb19ef94f5059cccf64031ed852a1be3e0b679a75a212ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:29:07.173 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:07 smithi022 bash[76206]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4 2024-04-16T17:29:07.500 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:07.013+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:07.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:07.013+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:07.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:07.120+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:07.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:07.120+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:07.559 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:07 smithi022 podman[76225]: 2024-04-16 17:29:07.261421795 +0000 UTC m=+0.651001519 container remove d55c4e4f47157cabcdb19ef94f5059cccf64031ed852a1be3e0b679a75a212ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True, ceph=True) 2024-04-16T17:29:07.559 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:07.444+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:07.833 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:07.565+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:07.834 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:07 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:07.565+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:07.834 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:07 smithi022 podman[76314]: 2024-04-16 17:29:07.463684901 +0000 UTC m=+0.017995427 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:29:07.834 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:07 smithi022 podman[76314]: 2024-04-16 17:29:07.587218081 +0000 UTC m=+0.141528605 container create 7572137ec812a570875a8c112e276b4df5f7318cbf846b2f765cc7876da7d927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:29:07.834 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:07 smithi022 podman[76314]: 2024-04-16 17:29:07.834810424 +0000 UTC m=+0.389120951 container init 7572137ec812a570875a8c112e276b4df5f7318cbf846b2f765cc7876da7d927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=) 2024-04-16T17:29:07.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:07 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:07.600+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:07.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:07 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:07.672+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:07.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:07 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:07.673+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:08.085 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:08 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:08.019+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:08.085 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:08 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:08.019+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:08.085 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:07.789+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:08.085 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:07 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:07.789+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:08.138 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:07 smithi022 podman[76314]: 2024-04-16 17:29:07.838329589 +0000 UTC m=+0.392640108 container start 7572137ec812a570875a8c112e276b4df5f7318cbf846b2f765cc7876da7d927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True) 2024-04-16T17:29:08.138 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:07 smithi022 podman[76314]: 2024-04-16 17:29:07.925500518 +0000 UTC m=+0.479811051 container attach 7572137ec812a570875a8c112e276b4df5f7318cbf846b2f765cc7876da7d927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-16T17:29:08.423 INFO:journalctl@ceph.osd.4.smithi022.stdout:Apr 16 17:29:08 smithi022 podman[76314]: 2024-04-16 17:29:08.14021073 +0000 UTC m=+0.694521256 container died 7572137ec812a570875a8c112e276b4df5f7318cbf846b2f765cc7876da7d927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-16T17:29:08.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:08 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:08.085+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:08.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:08 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:08.085+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:08.770 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:08 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:08.434+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:08.770 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:08 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:08.571+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:08.770 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:08 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:08.571+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:08.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.4.service' 2024-04-16T17:29:08.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:29:08.845 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-16T17:29:08.845 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-16T17:29:08.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.5 2024-04-16T17:29:08.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:08 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:08.564+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:08.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:08 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:08.703+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:08.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:08 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:08.703+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:08.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:08 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:08.703+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:09.125 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:09.019+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:09.125 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:09.019+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:09.125 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:08 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:08.809+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:09.125 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:08 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:08.809+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:09.126 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:08 smithi043 systemd[1]: Stopping Ceph osd.5 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:29:09.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:09.125+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:09.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:09.125+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:09.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:09.125+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:09.501 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:09.176+0000 7f8efac78700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:29:09.501 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:09.176+0000 7f8efac78700 -1 osd.5 143 *** Got signal Terminated *** 2024-04-16T17:29:09.501 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:09.176+0000 7f8efac78700 -1 osd.5 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:29:09.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:09 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:09.474+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:09.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:09 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:09.576+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:09.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:09 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:09.576+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:09.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:09 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:09.576+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:09.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:09 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:09.563+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:09.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:09 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:09.713+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:09.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:09 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:09.713+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:09.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:09 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:09.713+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:10.062 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:09.766+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:10.063 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:09 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:09.766+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:10.500 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:10 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:10.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:10.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:10 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:10.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:10.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:10 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:10.127+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:10.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:10 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:10.127+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:10.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:10 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:10.127+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:10.825 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:10 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:10.511+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:10.825 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:10 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:10.511+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:10.825 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:10 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:10.536+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:10.825 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:10 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:10.536+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:10.826 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:10 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:10.536+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:10.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:10 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:10.579+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:10.945 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:10 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:10.579+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:10.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:10 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:10.736+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:10.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:10 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:10.736+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:10.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:10 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:10.736+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:11.000 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:10 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:10.717+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:11.001 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:10 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:10.717+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:11.001 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:10 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:10.717+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.560790+0000 front 2024-04-16T17:28:46.560818+0000 (oldest deadline 2024-04-16T17:29:10.661121+0000) 2024-04-16T17:29:11.291 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:11 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:11.080+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:11.292 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:11 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:11.080+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:11.292 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:11 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:11.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:11.292 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:11 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:11.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:11.292 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:11 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:11.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:11.838 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:11 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:11.477+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:11.839 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:11 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:11.477+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:11.839 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:11 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:11.526+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:11.839 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:11 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:11.526+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:11.839 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:11 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:11.526+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:11.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:11 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:11.612+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:11.945 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:11 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:11.612+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:11.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:11 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:11.695+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:11.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:11 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:11.695+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:11.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:11 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:11.695+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:12.000 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:11 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:11.708+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:12.000 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:11 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:11.708+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:12.001 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:11 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:11.708+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.560790+0000 front 2024-04-16T17:28:46.560818+0000 (oldest deadline 2024-04-16T17:29:10.661121+0000) 2024-04-16T17:29:12.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:12.184+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:12.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:12.184+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:12.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:12.184+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:12.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:12.056+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:12.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:12.056+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:12.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:12.056+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:12.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:12 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:12.520+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:12.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:12 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:12.520+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:12.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:12 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:12.478+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:12.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:12 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:12.478+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:12.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:12 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:12.478+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:12.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:12 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:12.616+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:12.945 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:12 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:12.616+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:12.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:12 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:12.677+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:12.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:12 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:12.677+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:12.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:12 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:12.677+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:13.071 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:12.754+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:13.071 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:12.754+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:13.071 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:12 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:12.754+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.560790+0000 front 2024-04-16T17:28:46.560818+0000 (oldest deadline 2024-04-16T17:29:10.661121+0000) 2024-04-16T17:29:13.500 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:13.069+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:13.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:13.069+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:13.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:13.069+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:13.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:13.217+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:13.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:13.217+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:13.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:13.217+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:13.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:13 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:13.536+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:13.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:13 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:13.536+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:13.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:13 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:13.494+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:13.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:13 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:13.494+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:13.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:13 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:13.494+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:13.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:13 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:13.589+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:13.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:13 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:13.589+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:13.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:13 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:13.650+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:13.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:13 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:13.650+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:13.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:13 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:13.650+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:14.000 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:13.730+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.156347+0000 front 2024-04-16T17:28:26.156168+0000 (oldest deadline 2024-04-16T17:28:50.256369+0000) 2024-04-16T17:29:14.001 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:13.730+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.058590+0000 front 2024-04-16T17:28:36.058627+0000 (oldest deadline 2024-04-16T17:29:01.358685+0000) 2024-04-16T17:29:14.001 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:13 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5[48917]: 2024-04-16T17:29:13.730+0000 7f8ef3264700 -1 osd.5 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.560790+0000 front 2024-04-16T17:28:46.560818+0000 (oldest deadline 2024-04-16T17:29:10.661121+0000) 2024-04-16T17:29:14.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:14 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:14.066+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:14.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:14 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:14.066+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:14.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:14 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:14.066+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:14.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:14 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:14.235+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:14.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:14 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:14.235+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:14.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:14 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:14.235+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:14.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:14 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:14.576+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:14.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:14 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:14.576+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:14.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:14 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:14.451+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:14.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:14 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:14.451+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:14.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:14 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:14.451+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:14.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:14 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:14.601+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:14.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:14 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:14.601+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:14.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:14 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:14.626+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:14.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:14 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:14.626+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:14.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:14 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:14.626+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:15.052 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:14 smithi043 podman[76758]: 2024-04-16 17:29:14.76216417 +0000 UTC m=+5.709598831 container died 9a4fb67392f57a1271f17cc8720f70a7a9e5c3f63c1f8aab3df1b8ba1f8a456e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240409, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-16T17:29:15.428 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:15 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:15.285+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:15.428 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:15 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:15.285+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:15.429 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:15 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:15.285+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:15.429 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:15 smithi043 podman[76758]: 2024-04-16 17:29:15.16119617 +0000 UTC m=+6.108630823 container cleanup 9a4fb67392f57a1271f17cc8720f70a7a9e5c3f63c1f8aab3df1b8ba1f8a456e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:29:15.429 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:15 smithi043 bash[76758]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5 2024-04-16T17:29:15.429 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:15 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:15.070+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:15.429 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:15 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:15.070+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:15.430 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:15 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:15.070+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:15.750 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:15 smithi043 podman[76787]: 2024-04-16 17:29:15.429107566 +0000 UTC m=+0.666599432 container remove 9a4fb67392f57a1271f17cc8720f70a7a9e5c3f63c1f8aab3df1b8ba1f8a456e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:29:15.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:15 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:15.553+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:15.926 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:15 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:15.553+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:15.926 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:15 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:15.491+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:15.926 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:15 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:15.491+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:15.926 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:15 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:15.491+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:15.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:15 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:15.613+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:15.945 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:15 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:15.613+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:15.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:15 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:15.655+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:15.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:15 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:15.655+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:15.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:15 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:15.655+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:16.066 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:15 smithi043 podman[76877]: 2024-04-16 17:29:15.682157954 +0000 UTC m=+0.018268813 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:29:16.066 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:15 smithi043 podman[76877]: 2024-04-16 17:29:15.78831074 +0000 UTC m=+0.124421593 container create ecd55c5c2bbd532f116c010e071001421b4ad97bece49554c1bb4386b21ffe90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:29:16.400 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:16 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:16.240+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:16.401 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:16 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:16.240+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:16.401 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:16 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:16.240+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:16.402 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:16 smithi043 podman[76877]: 2024-04-16 17:29:16.088277055 +0000 UTC m=+0.424387899 container init ecd55c5c2bbd532f116c010e071001421b4ad97bece49554c1bb4386b21ffe90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-16T17:29:16.402 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:16 smithi043 podman[76877]: 2024-04-16 17:29:16.091600002 +0000 UTC m=+0.427710866 container start ecd55c5c2bbd532f116c010e071001421b4ad97bece49554c1bb4386b21ffe90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, io.buildah.version=1.33.5) 2024-04-16T17:29:16.402 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:16 smithi043 podman[76877]: 2024-04-16 17:29:16.199323655 +0000 UTC m=+0.535434529 container attach ecd55c5c2bbd532f116c010e071001421b4ad97bece49554c1bb4386b21ffe90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409) 2024-04-16T17:29:16.403 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:16 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:16.067+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:16.403 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:16 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:16.067+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:16.403 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:16 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:16.067+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:16.751 INFO:journalctl@ceph.osd.5.smithi043.stdout:Apr 16 17:29:16 smithi043 podman[76877]: 2024-04-16 17:29:16.407062084 +0000 UTC m=+0.743172993 container died ecd55c5c2bbd532f116c010e071001421b4ad97bece49554c1bb4386b21ffe90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:29:16.939 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:16 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:16.508+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:16.940 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:16 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:16.508+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:16.940 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:16 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:16.493+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:16.940 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:16 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:16.493+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:16.940 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:16 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:16.493+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:16.947 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:16 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:16.585+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:16.947 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:16 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:16.585+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:16.947 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:16 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:16.656+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:16.947 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:16 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:16.656+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:16.947 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:16 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:16.656+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:17.235 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:17 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:17.074+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:17.236 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:17 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:17.074+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:17.236 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:17 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:17.074+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:17.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.5.service' 2024-04-16T17:29:17.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:29:17.325 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-16T17:29:17.325 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-16T17:29:17.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.6 2024-04-16T17:29:17.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:17 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:17.249+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:17.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:17 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:17.249+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:17.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:17 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:17.249+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:17.694 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:17 smithi018 systemd[1]: Stopping Ceph osd.6 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:29:17.695 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:17.560+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:17.695 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:17.560+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:17.695 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:17.641+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:17.695 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:17.641+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:17.695 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:17.641+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:17.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:17 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:17.550+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:17.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:17 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:17.550+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:17.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:17 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:17.514+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:17.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:17 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:17.514+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:17.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:17 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:17.514+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:18.194 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:17.929+0000 7f1f2f000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:29:18.194 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:17.929+0000 7f1f2f000700 -1 osd.6 143 *** Got signal Terminated *** 2024-04-16T17:29:18.195 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:17 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:17.929+0000 7f1f2f000700 -1 osd.6 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:29:18.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:18 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:18.251+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:18.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:18 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:18.251+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:18.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:18 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:18.251+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:18.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:18 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:18.077+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:18.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:18 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:18.077+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:18.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:18 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:18.077+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:18.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:18 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:18.077+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:18.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:18 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:18.574+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:18.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:18 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:18.574+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:18.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:18 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:18.478+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:18.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:18 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:18.478+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:18.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:18 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:18.478+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:18.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:18.550+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:18.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:18.550+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:18.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:18.615+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:18.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:18.615+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:18.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:18.615+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:19.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:19 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:19.252+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:19.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:19 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:19.252+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:19.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:19 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:19.252+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:19.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:19 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:19.097+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:19.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:19 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:19.097+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:19.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:19 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:19.097+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:19.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:19 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:19.097+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:19.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:19 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:19.594+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:19.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:19 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:19.594+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:19.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:19 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:19.594+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:19.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:19 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:19.484+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:19.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:19 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:19.484+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:19.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:19 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:19.484+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:19.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:19 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:19.484+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:19.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:19 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:19.534+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:19.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:19 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:19.534+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:19.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:19 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:19.591+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:19.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:19 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:19.591+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:19.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:19 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:19.591+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:20.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:20 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:20.224+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:20.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:20 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:20.224+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:20.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:20 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:20.224+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:20.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:20 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:20.146+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:20.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:20 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:20.146+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:20.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:20 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:20.146+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:20.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:20 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:20.146+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:20.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:20 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:20.639+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:20.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:20 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:20.639+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:20.924 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:20 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:20.639+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:20.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:20 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:20.531+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:20.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:20 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:20.531+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:20.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:20 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:20.531+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:20.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:20 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:20.531+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:20.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:20.556+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:20.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:20.556+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:20.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:20.556+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.999547+0000 front 2024-04-16T17:28:54.999600+0000 (oldest deadline 2024-04-16T17:29:19.699848+0000) 2024-04-16T17:29:20.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:20.641+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:20.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:20.641+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:20.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:20 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:20.641+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:21.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:21 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:21.104+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:21.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:21 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:21.104+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:21.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:21 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:21.104+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:21.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:21 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:21.104+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:21.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:21 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:21.208+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:21.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:21 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:21.208+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:21.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:21 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:21.208+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:21.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:21 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:21.590+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:21.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:21 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:21.590+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:21.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:21 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:21.590+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:21.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:21 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:21.494+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:21.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:21 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:21.494+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:21.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:21 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:21.494+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:21.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:21 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:21.494+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:21.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:21.508+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:21.945 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:21.508+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:21.945 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:21.508+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.999547+0000 front 2024-04-16T17:28:54.999600+0000 (oldest deadline 2024-04-16T17:29:19.699848+0000) 2024-04-16T17:29:21.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:21.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:21.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:21.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:21.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:21.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:21.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:21 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:21.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:22.336 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:22 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:22.242+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:22.337 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:22 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:22.242+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:22.337 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:22 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:22.242+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:22.337 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:22 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:22.083+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:22.337 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:22 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:22.083+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:22.337 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:22 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:22.083+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:22.337 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:22 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:22.083+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:22.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:22 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:22.599+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:22.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:22 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:22.600+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:22.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:22 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:22.600+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:22.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:22 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:22.533+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:22.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:22 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:22.533+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:22.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:22 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:22.533+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:22.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:22 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:22.533+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:22.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:22.516+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:38.092026+0000 front 2024-04-16T17:28:38.092086+0000 (oldest deadline 2024-04-16T17:29:02.192261+0000) 2024-04-16T17:29:22.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:22.516+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:45.596935+0000 front 2024-04-16T17:28:45.597262+0000 (oldest deadline 2024-04-16T17:29:10.296974+0000) 2024-04-16T17:29:22.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6[57436]: 2024-04-16T17:29:22.516+0000 7f1f25800700 -1 osd.6 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.999547+0000 front 2024-04-16T17:28:54.999600+0000 (oldest deadline 2024-04-16T17:29:19.699848+0000) 2024-04-16T17:29:22.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:22.676+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:22.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:22.676+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:22.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:22.676+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:22.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:22 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:22.676+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:23.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:23.258+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:23.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:23.258+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:23.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:23.258+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:23.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:23.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:23.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:23.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:23.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:23.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:23.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:23 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:23.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:23.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:23 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:23.613+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:23.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:23 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:23.613+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:23.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:23 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:23.613+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:23.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:23 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:23.561+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:23.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:23 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:23.561+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:23.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:23 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:23.561+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:23.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:23 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:23.561+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:23.944 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:23 smithi018 podman[115958]: 2024-04-16 17:29:23.724717732 +0000 UTC m=+5.939328944 container died c80c80357d27e3d8e04966531805a9eb3f4b39d2716146b7f1794df76c4f04cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T17:29:23.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:23 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:23.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:23.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:23 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:23.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:23.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:23 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:23.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:23.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:23 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:23.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:24.356 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:24 smithi018 podman[115958]: 2024-04-16 17:29:24.139068636 +0000 UTC m=+6.353679825 container cleanup c80c80357d27e3d8e04966531805a9eb3f4b39d2716146b7f1794df76c4f04cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-16T17:29:24.357 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:24 smithi018 bash[115958]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6 2024-04-16T17:29:24.500 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:24 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:24.083+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:24.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:24 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:24.083+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:24.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:24 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:24.083+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:24.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:24 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:24.083+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:24.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:24 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:24.270+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:24.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:24 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:24.270+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:24.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:24 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:24.270+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:24.613 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:24 smithi018 podman[115978]: 2024-04-16 17:29:24.356780017 +0000 UTC m=+0.604813078 container remove c80c80357d27e3d8e04966531805a9eb3f4b39d2716146b7f1794df76c4f04cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-16T17:29:24.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:24 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:24.633+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:24.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:24 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:24.633+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:24.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:24 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:24.633+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:24.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:24 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:24.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:24.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:24 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:24.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:24.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:24 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:24.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:24.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:24 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:24.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:24.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:24.612+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:24.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:24.612+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:24.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:24.612+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:24.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:24.612+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:25.304 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:25 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:25.043+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:25.305 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:25 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:25.043+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:25.305 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:25 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:25.043+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:25.305 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:25 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:25.043+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:25.444 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:25 smithi018 podman[116062]: 2024-04-16 17:29:25.057106987 +0000 UTC m=+0.041140281 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:29:25.445 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:25 smithi018 podman[116062]: 2024-04-16 17:29:25.16590356 +0000 UTC m=+0.149936862 container create a34deb85efd677598b0c44b573f9058292fea4b396b9db656f679efbed43f68d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-16T17:29:25.750 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:25 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:25.304+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:25.750 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:25 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:25.304+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:25.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:25 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:25.304+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:25.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:25 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:25.589+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:25.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:25 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:25.589+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:25.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:25 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:25.589+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:25.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:25 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:25.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:25.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:25 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:25.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:25.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:25 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:25.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:25.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:25 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:25.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:25.937 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:25 smithi018 podman[116062]: 2024-04-16 17:29:25.453069809 +0000 UTC m=+0.437103163 container init a34deb85efd677598b0c44b573f9058292fea4b396b9db656f679efbed43f68d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-16T17:29:25.937 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:25 smithi018 podman[116062]: 2024-04-16 17:29:25.464609116 +0000 UTC m=+0.448642419 container start a34deb85efd677598b0c44b573f9058292fea4b396b9db656f679efbed43f68d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-16T17:29:25.937 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:25 smithi018 podman[116062]: 2024-04-16 17:29:25.568837596 +0000 UTC m=+0.552871003 container attach a34deb85efd677598b0c44b573f9058292fea4b396b9db656f679efbed43f68d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T17:29:25.938 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:25 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:25.651+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:25.938 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:25 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:25.651+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:25.938 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:25 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:25.651+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:25.938 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:25 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:25.651+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:26.194 INFO:journalctl@ceph.osd.6.smithi018.stdout:Apr 16 17:29:25 smithi018 podman[116088]: 2024-04-16 17:29:25.998074093 +0000 UTC m=+0.041367021 container died a34deb85efd677598b0c44b573f9058292fea4b396b9db656f679efbed43f68d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-16T17:29:26.291 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:26 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:26.037+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:26.291 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:26 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:26.037+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:26.291 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:26 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:26.037+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:26.291 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:26 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:26.037+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:26.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.6.service' 2024-04-16T17:29:26.750 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:26 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:26.291+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:26.750 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:26 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:26.291+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:26.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:26 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:26.291+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:26.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:29:26.820 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-16T17:29:26.820 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-16T17:29:26.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.7 2024-04-16T17:29:26.854 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:26 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:26.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:26.854 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:26 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:26.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:26.855 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:26 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:26.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:26.855 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:26 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:26.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:26.855 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:26 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:26.590+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:26.855 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:26 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:26.590+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:26.855 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:26 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:26.590+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:26.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:26 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:26.647+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:26.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:26 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:26.647+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:26.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:26 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:26.647+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:26.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:26 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:26.647+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:27.148 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:26 smithi022 systemd[1]: Stopping Ceph osd.7 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:29:27.423 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:27.148+0000 7f8cddf5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:29:27.423 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:27.148+0000 7f8cddf5d700 -1 osd.7 143 *** Got signal Terminated *** 2024-04-16T17:29:27.423 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:27.148+0000 7f8cddf5d700 -1 osd.7 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:29:27.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:27 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:27.259+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:27.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:27 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:27.259+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:27.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:27 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:27.259+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:27.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:27 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:27.057+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:27.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:27 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:27.057+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:27.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:27 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:27.057+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:27.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:27 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:27.057+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:27.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:27.584+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:27.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:27.584+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:27.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:27.584+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:27.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:27.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:27.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:27.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:27.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:27.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:27.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:27 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:27.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:27.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:27.692+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:27.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:27.692+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:27.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:27.692+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:27.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:27.692+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:27.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:27 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:27.692+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:28.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:28 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:28.283+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:28.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:28 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:28.283+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:28.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:28 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:28.283+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:28.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:28 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:28.065+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:28.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:28 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:28.065+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:28.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:28 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:28.065+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:28.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:28 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:28.065+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:28.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:28 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:28.584+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:28.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:28 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:28.584+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:28.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:28 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:28.584+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:28.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:28 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:28.605+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:28.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:28 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:28.605+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:28.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:28 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:28.605+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:28.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:28 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:28.605+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:28.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:28 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:28.605+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:28.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:28.643+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:28.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:28.643+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:28.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:28.643+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:28.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:28.643+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:28.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:28 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:28.643+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:29.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:29 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:29.283+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:29.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:29 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:29.283+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:29.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:29 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:29.283+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:29.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:29 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:29.088+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:29.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:29 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:29.088+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:29.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:29 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:29.088+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:29.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:29 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:29.088+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:29.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:29.598+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:29.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:29.598+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:29.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:29.598+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:29.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:29.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:29.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:29.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:29.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:29.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:29.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:29.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:29.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:29 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:29.585+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:29.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:29 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:29.671+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:29.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:29 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:29.671+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:29.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:29 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:29.671+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:29.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:29 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:29.671+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:29.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:29 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:29.671+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:30.310 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:30 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:30.045+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:30.310 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:30 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:30.045+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:30.310 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:30 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:30.045+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:30.310 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:30 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:30.045+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:30.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:30 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:30.310+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:30.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:30 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:30.310+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:30.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:30 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:30.310+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:30.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:30 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:30.310+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:30.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:30.563+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:30.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:30.563+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:30.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:30.563+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:30.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:30.563+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.776195+0000 front 2024-04-16T17:29:05.776257+0000 (oldest deadline 2024-04-16T17:29:29.876348+0000) 2024-04-16T17:29:30.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:30.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:30.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:30.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:30.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:30.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:30.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:30.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:30.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:30.581+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:30.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:30.669+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:30.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:30.669+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:30.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:30.669+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:30.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:30.669+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:30.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:30 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:30.669+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:31.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:31.084+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:31.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:31.084+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:31.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:31.084+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:31.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:31.084+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:31.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:31.331+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:31.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:31.331+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:31.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:31.331+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:31.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:31 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:31.331+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:31.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:31.577+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:34.969692+0000 front 2024-04-16T17:28:34.969658+0000 (oldest deadline 2024-04-16T17:29:01.370441+0000) 2024-04-16T17:29:31.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:31.577+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.772297+0000 front 2024-04-16T17:28:44.772569+0000 (oldest deadline 2024-04-16T17:29:10.072223+0000) 2024-04-16T17:29:31.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:31.577+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.674237+0000 front 2024-04-16T17:28:54.674072+0000 (oldest deadline 2024-04-16T17:29:18.774112+0000) 2024-04-16T17:29:31.924 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7[54176]: 2024-04-16T17:29:31.577+0000 7f8cda551700 -1 osd.7 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.776195+0000 front 2024-04-16T17:29:05.776257+0000 (oldest deadline 2024-04-16T17:29:29.876348+0000) 2024-04-16T17:29:31.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:31.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:31.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:31.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:31.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:31.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:31.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:31.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:31.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:31 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:31.579+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:32.194 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:31.712+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:32.195 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:31.712+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:32.195 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:31.712+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:32.195 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:31.712+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:32.195 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:31 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:31.712+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:32.342 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:32 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:32.043+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:32.342 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:32 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:32.043+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:32.342 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:32 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:32.043+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:32.342 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:32 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:32.043+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:32.750 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:32 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:32.342+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:32.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:32 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:32.342+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:32.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:32 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:32.342+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:32.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:32 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:32.342+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:32.915 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:32 smithi022 podman[76431]: 2024-04-16 17:29:32.661334774 +0000 UTC m=+5.629677178 container died 341a3492349b9aaf62efc25edb7cc4b5ba04d425ea83f192e89213e5eac9a670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-16T17:29:32.915 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:32 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:32.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:32.915 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:32 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:32.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:32.916 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:32 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:32.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:32.916 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:32 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:32.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:32.916 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:32 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:32.590+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:32.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:32 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:32.666+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:32.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:32 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:32.666+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:32.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:32 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:32.666+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:32.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:32 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:32.666+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:32.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:32 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:32.666+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:33.173 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:33 smithi022 podman[76431]: 2024-04-16 17:29:33.006851175 +0000 UTC m=+5.975193573 container cleanup 341a3492349b9aaf62efc25edb7cc4b5ba04d425ea83f192e89213e5eac9a670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:29:33.173 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:33 smithi022 bash[76431]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7 2024-04-16T17:29:33.310 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:33 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:33.006+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:33.310 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:33 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:33.006+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:33.310 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:33 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:33.006+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:33.310 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:33 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:33.006+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:33.644 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:33 smithi022 podman[76448]: 2024-04-16 17:29:33.324886204 +0000 UTC m=+0.663087896 container remove 341a3492349b9aaf62efc25edb7cc4b5ba04d425ea83f192e89213e5eac9a670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-16T17:29:33.645 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:33 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:33.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:33.645 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:33 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:33.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:33.645 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:33 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:33.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:33.645 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:33 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:33.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:33.645 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:33 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:33.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:33.750 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:33 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:33.310+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:33.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:33 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:33.310+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:33.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:33 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:33.310+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:33.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:33 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:33.310+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:33.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:33 smithi022 podman[76539]: 2024-04-16 17:29:33.549011588 +0000 UTC m=+0.018929740 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:29:33.923 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:33 smithi022 podman[76539]: 2024-04-16 17:29:33.675756857 +0000 UTC m=+0.145674997 container create ecdbfc92c5a75e52b881da820856d78c39858642ea7c0ba8eaf00b4631bac25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:29:33.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:33 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:33.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:33.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:33 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:33.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:33.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:33 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:33.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:33.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:33 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:33.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:33.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:33 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:33.635+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:34.228 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:33 smithi022 podman[76539]: 2024-04-16 17:29:33.928515179 +0000 UTC m=+0.398433314 container init ecdbfc92c5a75e52b881da820856d78c39858642ea7c0ba8eaf00b4631bac25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD) 2024-04-16T17:29:34.228 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:33 smithi022 podman[76539]: 2024-04-16 17:29:33.931756577 +0000 UTC m=+0.401674707 container start ecdbfc92c5a75e52b881da820856d78c39858642ea7c0ba8eaf00b4631bac25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:29:34.229 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:34 smithi022 podman[76539]: 2024-04-16 17:29:34.020327584 +0000 UTC m=+0.490245733 container attach ecdbfc92c5a75e52b881da820856d78c39858642ea7c0ba8eaf00b4631bac25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:29:34.489 INFO:journalctl@ceph.osd.7.smithi022.stdout:Apr 16 17:29:34 smithi022 podman[76539]: 2024-04-16 17:29:34.229814816 +0000 UTC m=+0.699732951 container died ecdbfc92c5a75e52b881da820856d78c39858642ea7c0ba8eaf00b4631bac25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-16T17:29:34.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:34.261+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:34.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:34.261+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:34.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:34.261+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:34.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:34.261+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:34.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:34.053+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:34.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:34.053+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:34.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:34.053+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:34.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:34 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:34.053+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:34.773 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:34 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:34.508+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:34.773 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:34 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:34.508+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:34.773 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:34 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:34.508+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:34.773 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:34 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:34.508+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:34.773 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:34 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:34.508+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:34.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.7.service' 2024-04-16T17:29:34.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:29:34.882 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-16T17:29:34.882 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-16T17:29:34.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.8 2024-04-16T17:29:34.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:34.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:34.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:34.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:34.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:34.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:34.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:34.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:34.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:34 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:34.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:35.213 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:34 smithi043 systemd[1]: Stopping Ceph osd.8 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:29:35.213 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:35.028+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:35.213 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:35.028+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:35.214 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:35.028+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:35.214 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:35.028+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:35.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:35.213+0000 7f7122335700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:29:35.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:35.213+0000 7f7122335700 -1 osd.8 143 *** Got signal Terminated *** 2024-04-16T17:29:35.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:35.213+0000 7f7122335700 -1 osd.8 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:29:35.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:35.294+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:35.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:35.294+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:35.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:35.294+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:35.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:35 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:35.294+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:35.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:35.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:35.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:35.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:35.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:35.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:35.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:35.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:35.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:35 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:35.554+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:35.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:35 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:35.636+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:35.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:35 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:35.636+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:35.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:35 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:35.636+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:35.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:35 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:35.636+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:35.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:35 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:35.636+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:36.307 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:36.024+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:36.307 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:36.024+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:36.307 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:36.024+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:36.307 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:36.024+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:36.750 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:36.307+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:36.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:36.307+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:36.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:36.307+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:36.751 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:36 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:36.307+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:36.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:36.547+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:36.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:36.547+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:36.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:36.547+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:36.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:36.547+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:36.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:36.547+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:36.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:36.547+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:36.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:36.647+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:36.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:36.648+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:36.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:36.648+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:36.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:36.648+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:36.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:36 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:36.648+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:37.500 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:37.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:37.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:37.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:37.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:37.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:37.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:37.063+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:37.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:37.295+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:37.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:37.295+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:37.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:37.295+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:37.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:37.295+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:37.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:37 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:37.295+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:37.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:37.562+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:37.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:37.562+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:37.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:37.562+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:37.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:37.562+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:37.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:37.562+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:37.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:37 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:37.562+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:37.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:37.646+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:37.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:37.646+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:37.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:37.646+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:37.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:37.646+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:37.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:37 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:37.646+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:38.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:38.072+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:38.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:38.072+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:38.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:38.072+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:38.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:38.072+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:38.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:38.249+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:38.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:38.249+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:38.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:38.249+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:38.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:38.249+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:38.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:38 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:38.249+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:38.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:38.587+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:38.984 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:38.587+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:38.984 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:38.587+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:38.985 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:38.587+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:38.985 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:38.587+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:38.985 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:38.587+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:38.986 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:38 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:38.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:38.986 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:38 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:38.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:38.986 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:38 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:38.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:38.986 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:38 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:38.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:38.986 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:38 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:38.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:38.986 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:38 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:38.684+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:39.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:39.248+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:39.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:39.248+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:39.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:39.248+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:39.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:39.248+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:39.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:39.248+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:39.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:39.099+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:39.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:39.099+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:39.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:39.099+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:39.502 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:39 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:39.099+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:39.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:39.592+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:39.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:39.592+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:39.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:39.592+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:39.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:39.592+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:39.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:39.592+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:39.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:39 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:39.592+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:39.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:39.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:39.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:39.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:39.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:39.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:39.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:39.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:39.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:39.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:39.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:39 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:39.683+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:40.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:40.148+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:25.231916+0000 front 2024-04-16T17:28:25.232090+0000 (oldest deadline 2024-04-16T17:28:49.332162+0000) 2024-04-16T17:29:40.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:40.148+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:36.835084+0000 front 2024-04-16T17:28:36.834817+0000 (oldest deadline 2024-04-16T17:29:02.134978+0000) 2024-04-16T17:29:40.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:40.148+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:42.135452+0000 front 2024-04-16T17:28:42.135488+0000 (oldest deadline 2024-04-16T17:29:11.536254+0000) 2024-04-16T17:29:40.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:40.148+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.338192+0000 front 2024-04-16T17:28:54.338160+0000 (oldest deadline 2024-04-16T17:29:17.238009+0000) 2024-04-16T17:29:40.501 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8[55674]: 2024-04-16T17:29:40.148+0000 7f711a921700 -1 osd.8 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.942404+0000 front 2024-04-16T17:29:13.942435+0000 (oldest deadline 2024-04-16T17:29:39.842574+0000) 2024-04-16T17:29:40.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:40.232+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:40.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:40.232+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:40.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:40.232+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:40.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:40.232+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:40.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:40 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:40.232+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:40.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:40 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:40.612+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:40.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:40 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:40.612+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:40.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:40 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:40.612+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:40.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:40 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:40.612+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:40.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:40 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:40.612+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:40.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:40 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:40.612+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:40.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:40.639+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:40.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:40.639+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:40.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:40.639+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:40.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:40.639+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:40.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:40.639+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:40.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:40 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:40.639+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:41.000 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:40 smithi043 podman[76982]: 2024-04-16 17:29:40.659654278 +0000 UTC m=+5.572132723 container died 56b999447a87acd7420cd5d539743e36e57ec657527877459a992a73ff397e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-16T17:29:41.372 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:41 smithi043 podman[76982]: 2024-04-16 17:29:41.155772928 +0000 UTC m=+6.068251347 container cleanup 56b999447a87acd7420cd5d539743e36e57ec657527877459a992a73ff397e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-16T17:29:41.373 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:41 smithi043 bash[76982]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8 2024-04-16T17:29:41.373 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:41.187+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:41.373 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:41.187+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:41.373 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:41.187+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:41.373 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:41.187+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:41.374 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:41.187+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:41.750 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:41 smithi043 podman[76999]: 2024-04-16 17:29:41.373837728 +0000 UTC m=+0.709413107 container remove 56b999447a87acd7420cd5d539743e36e57ec657527877459a992a73ff397e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-16T17:29:41.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:41.652+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:41.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:41.652+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:41.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:41.652+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:41.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:41.652+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:41.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:41.652+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:41.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:41 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:41.652+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:41.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:41.602+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:41.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:41.602+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:41.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:41.602+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:41.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:41.602+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:41.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:41.602+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:41.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:41 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:41.602+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:42.117 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:41 smithi043 podman[77089]: 2024-04-16 17:29:41.680237774 +0000 UTC m=+0.018304998 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:29:42.117 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:41 smithi043 podman[77089]: 2024-04-16 17:29:41.7831767 +0000 UTC m=+0.121243928 container create e6eff092c9eb07b39c6e866ed3f5c211217d16e8d78931526e6bd06aa1feeb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:29:42.117 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:42 smithi043 podman[77089]: 2024-04-16 17:29:42.118963974 +0000 UTC m=+0.457031188 container init e6eff092c9eb07b39c6e866ed3f5c211217d16e8d78931526e6bd06aa1feeb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:29:42.398 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:42 smithi043 podman[77089]: 2024-04-16 17:29:42.122473581 +0000 UTC m=+0.460540799 container start e6eff092c9eb07b39c6e866ed3f5c211217d16e8d78931526e6bd06aa1feeb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:29:42.398 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:42 smithi043 podman[77089]: 2024-04-16 17:29:42.227330425 +0000 UTC m=+0.565397640 container attach e6eff092c9eb07b39c6e866ed3f5c211217d16e8d78931526e6bd06aa1feeb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T17:29:42.398 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:42 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:42.176+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:42.399 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:42 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:42.176+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:42.399 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:42 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:42.176+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:42.399 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:42 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:42.176+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:42.399 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:42 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:42.176+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:42.750 INFO:journalctl@ceph.osd.8.smithi043.stdout:Apr 16 17:29:42 smithi043 podman[77089]: 2024-04-16 17:29:42.435595422 +0000 UTC m=+0.773662639 container died e6eff092c9eb07b39c6e866ed3f5c211217d16e8d78931526e6bd06aa1feeb0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-16T17:29:42.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:42 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:42.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:42.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:42 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:42.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:42.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:42 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:42.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:42.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:42 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:42.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:42.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:42 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:42.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:42.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:42 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:42.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:42.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:42.599+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:42.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:42.599+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:42.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:42.599+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:42.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:42.599+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:42.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:42.599+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:42.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:42 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:42.599+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:43.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.8.service' 2024-04-16T17:29:43.117 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:29:43.118 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-16T17:29:43.118 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-16T17:29:43.118 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.9 2024-04-16T17:29:43.375 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:43 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:43.164+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:43.375 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:43 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:43.164+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:43.375 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:43 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:43.164+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:43.375 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:43 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:43.164+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:43.375 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:43 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:43.164+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:43.582 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 systemd[1]: Stopping Ceph osd.9 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:29:43.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:43 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:43.608+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:43.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:43 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:43.609+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:43.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:43 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:43.609+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:43.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:43 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:43.609+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:43.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:43 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:43.609+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:43.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:43 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:43.609+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:43.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.580+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:43.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.580+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:43.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.580+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:43.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.580+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:43.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.580+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:43.946 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.580+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:43.946 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.771+0000 7f0d25c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:29:43.946 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.771+0000 7f0d25c00700 -1 osd.9 143 *** Got signal Terminated *** 2024-04-16T17:29:43.946 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:43 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:43.771+0000 7f0d25c00700 -1 osd.9 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:29:44.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:44.158+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:44.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:44.158+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:44.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:44.158+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:44.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:44.158+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:44.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:44.158+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:44.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:44 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:44.586+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:44.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:44 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:44.586+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:44.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:44 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:44.586+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:44.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:44 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:44.586+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:44.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:44 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:44.586+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:44.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:44 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:44.586+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:44.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:44 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:44.586+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:44.949 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:44.576+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:44.950 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:44.576+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:44.950 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:44.576+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:44.950 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:44.576+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:44.950 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:44.576+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:44.950 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:44 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:44.576+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:45.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:45.136+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:45.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:45.136+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:45.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:45.136+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:45.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:45.136+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:45.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:45 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:45.136+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:45.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:45 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:45.631+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:45.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:45 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:45.631+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:45.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:45 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:45.631+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:45.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:45 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:45.631+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:45.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:45 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:45.631+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:45.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:45 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:45.631+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:45.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:45 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:45.631+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:45.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:45.597+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:45.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:45.597+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:45.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:45.597+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:45.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:45.597+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:45.946 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:45.597+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:45.946 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:45 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:45.597+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:46.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:46.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:46.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:46.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:46.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:46.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:46.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:46.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:46.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:46.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:46.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:46 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:46.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:46.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:46 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:46.616+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:46.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:46 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:46.616+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:46.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:46 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:46.616+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:46.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:46 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:46.616+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:46.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:46 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:46.616+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:46.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:46 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:46.616+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:46.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:46 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:46.616+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:46.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:46.571+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:46.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:46.571+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:46.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:46.571+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:46.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:46.571+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:46.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:46.571+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:46.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:46 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:46.571+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:47.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:47 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:47.199+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:47.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:47 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:47.199+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:47.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:47 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:47.199+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:47.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:47 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:47.199+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:47.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:47 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:47.199+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:47.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:47 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:47.199+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:47.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:47.588+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:47.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:47.588+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:47.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:47.588+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:47.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:47.588+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:47.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:47.588+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:47.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:47.588+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:47.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:47 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:47.588+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:47.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:47 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:47.565+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:47.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:47 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:47.565+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:47.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:47 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:47.565+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:47.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:47 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:47.565+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:47.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:47 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:47.565+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:47.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:47 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:47.565+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:48.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:48.222+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:48.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:48.222+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:48.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:48.222+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:48.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:48.222+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:48.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:48.222+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:48.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:48.222+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:48.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:48.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:48.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:48.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:48.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:48.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:48.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:48.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:48.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:48.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:48.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:48.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:48.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:48 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:48.634+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:48.944 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:48 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:48.585+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:24.527396+0000 front 2024-04-16T17:28:24.518627+0000 (oldest deadline 2024-04-16T17:28:49.818692+0000) 2024-04-16T17:29:48.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:48 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:48.585+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.121964+0000 front 2024-04-16T17:28:35.122045+0000 (oldest deadline 2024-04-16T17:28:59.222222+0000) 2024-04-16T17:29:48.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:48 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:48.585+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:46.826455+0000 front 2024-04-16T17:28:46.826535+0000 (oldest deadline 2024-04-16T17:29:08.428293+0000) 2024-04-16T17:29:48.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:48 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:48.585+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:55.333191+0000 front 2024-04-16T17:28:55.332552+0000 (oldest deadline 2024-04-16T17:29:21.232610+0000) 2024-04-16T17:29:48.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:48 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:48.585+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:05.934855+0000 front 2024-04-16T17:29:05.934964+0000 (oldest deadline 2024-04-16T17:29:27.034978+0000) 2024-04-16T17:29:48.945 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:48 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9[64299]: 2024-04-16T17:29:48.585+0000 7f0d21400700 -1 osd.9 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:13.939818+0000 front 2024-04-16T17:29:13.939852+0000 (oldest deadline 2024-04-16T17:29:38.039680+0000) 2024-04-16T17:29:49.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:49.189+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:49.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:49.189+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:49.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:49.189+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:49.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:49.189+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:49.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:49.189+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:49.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:49 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:49.189+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:49.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:49.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:49.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:49.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:49.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:49.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:49.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:49.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:49.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:49.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:49.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:49.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:49.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:49 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:49.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:49.940 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:49 smithi018 podman[116168]: 2024-04-16 17:29:49.653388945 +0000 UTC m=+6.034425303 container died 332f0d9e208196f6fec03771f0cb90a17c09b6699ca48ffc7baf48243942524b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:29:50.194 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:50 smithi018 podman[116168]: 2024-04-16 17:29:50.056561074 +0000 UTC m=+6.437597425 container cleanup 332f0d9e208196f6fec03771f0cb90a17c09b6699ca48ffc7baf48243942524b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD) 2024-04-16T17:29:50.194 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:50 smithi018 bash[116168]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9 2024-04-16T17:29:50.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:50.203+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:50.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:50.203+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:50.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:50.203+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:50.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:50.203+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:50.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:50.203+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:50.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:50 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:50.203+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:50.694 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:50 smithi018 podman[116187]: 2024-04-16 17:29:50.257599075 +0000 UTC m=+0.557792587 container remove 332f0d9e208196f6fec03771f0cb90a17c09b6699ca48ffc7baf48243942524b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-16T17:29:50.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:50.567+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:50.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:50.567+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:50.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:50.567+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:50.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:50.567+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:50.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:50.567+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:50.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:50.567+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:50.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:50 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:50.567+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:51.405 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:51 smithi018 podman[116277]: 2024-04-16 17:29:51.141498346 +0000 UTC m=+0.122762578 container create e482ae053f47919abaf718fd5f80de8e279bc6f1189ca6719063bbf30ba994e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-16T17:29:51.405 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:51 smithi018 podman[116277]: 2024-04-16 17:29:51.055805673 +0000 UTC m=+0.037069882 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:29:51.405 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:51 smithi018 podman[116277]: 2024-04-16 17:29:51.405573009 +0000 UTC m=+0.386837250 container init e482ae053f47919abaf718fd5f80de8e279bc6f1189ca6719063bbf30ba994e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240409, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-16T17:29:51.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:51.188+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:51.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:51.188+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:51.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:51.188+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:51.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:51.188+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:51.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:51.188+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:51.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:51 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:51.188+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:51.694 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:51 smithi018 podman[116277]: 2024-04-16 17:29:51.417583586 +0000 UTC m=+0.398847792 container start e482ae053f47919abaf718fd5f80de8e279bc6f1189ca6719063bbf30ba994e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-16T17:29:51.694 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:51 smithi018 podman[116277]: 2024-04-16 17:29:51.518791699 +0000 UTC m=+0.500055960 container attach e482ae053f47919abaf718fd5f80de8e279bc6f1189ca6719063bbf30ba994e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T17:29:51.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:51.603+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:51.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:51.603+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:51.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:51.603+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:51.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:51.603+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:51.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:51.603+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:51.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:51.603+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:51.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:51 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:51.603+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:52.191 INFO:journalctl@ceph.osd.9.smithi018.stdout:Apr 16 17:29:51 smithi018 podman[116277]: 2024-04-16 17:29:51.894558033 +0000 UTC m=+0.875822321 container died e482ae053f47919abaf718fd5f80de8e279bc6f1189ca6719063bbf30ba994e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:29:52.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:52.211+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:52.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:52.211+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:52.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:52.211+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:52.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:52.211+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:52.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:52.211+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:52.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:52.211+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:52.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.9.service' 2024-04-16T17:29:52.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:29:52.712 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-16T17:29:52.713 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-16T17:29:52.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.10 2024-04-16T17:29:52.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:52.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:52.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:52.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:52.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:52.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:52.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:52.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:52.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:52.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:52.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:52.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:52.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:52 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:52.594+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:52.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:52 smithi022 systemd[1]: Stopping Ceph osd.10 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:29:53.423 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.018+0000 7f3ad7709700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:29:53.423 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.018+0000 7f3ad7709700 -1 osd.10 143 *** Got signal Terminated *** 2024-04-16T17:29:53.423 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.018+0000 7f3ad7709700 -1 osd.10 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:29:53.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:53.218+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:53.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:53.218+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:53.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:53.218+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:53.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:53.218+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:53.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:53.218+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:53.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:53 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:53.218+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:53.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.613+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:53.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.613+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:53.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.613+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:53.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.613+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:53.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.613+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:53.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.613+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:53.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:53 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:53.613+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:54.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:54.246+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:54.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:54.246+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:54.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:54.246+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:54.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:54.246+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:54.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:54.246+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:54.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:54 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:54.246+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:54.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:54.644+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:54.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:54.644+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:54.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:54.644+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:54.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:54.644+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:54.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:54.644+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:54.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:54.644+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:54.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:54 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:54.644+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:55.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:55.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:55.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:55.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:55.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:55.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:55.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:55.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:55.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:55.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:55.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:55 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:55.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:55.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:55.649+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:55.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:55.649+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:55.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:55.649+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:55.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:55.649+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:55.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:55.649+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:55.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:55.649+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:55.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:55 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:55.649+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:56.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:56.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:56.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:56.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:56.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:56.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:56.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:56.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:56.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:56.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:56.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:56 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:56.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:56.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:56 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:56.629+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:56.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:56 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:56.629+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:56.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:56 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:56.629+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:56.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:56 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:56.629+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:56.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:56 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:56.629+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:56.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:56 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:56.629+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:56.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:56 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:56.629+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:57.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:57.231+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:57.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:57.231+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:57.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:57.231+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:57.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:57.231+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:57.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:57.231+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:57.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:57.231+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:57.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:57 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:57.231+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:29:57.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:57.645+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:26.504026+0000 front 2024-04-16T17:28:26.504227+0000 (oldest deadline 2024-04-16T17:28:50.604051+0000) 2024-04-16T17:29:57.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:57.645+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:35.305249+0000 front 2024-04-16T17:28:35.305217+0000 (oldest deadline 2024-04-16T17:29:01.205428+0000) 2024-04-16T17:29:57.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:57.645+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:41.206262+0000 front 2024-04-16T17:28:41.206091+0000 (oldest deadline 2024-04-16T17:29:08.806803+0000) 2024-04-16T17:29:57.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:57.645+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6814 osd.3 since back 2024-04-16T17:28:54.009076+0000 front 2024-04-16T17:28:54.009269+0000 (oldest deadline 2024-04-16T17:29:19.308968+0000) 2024-04-16T17:29:57.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:57.645+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:02.710609+0000 front 2024-04-16T17:29:02.710628+0000 (oldest deadline 2024-04-16T17:29:28.010722+0000) 2024-04-16T17:29:57.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:57.645+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:12.711902+0000 front 2024-04-16T17:29:12.711693+0000 (oldest deadline 2024-04-16T17:29:36.211649+0000) 2024-04-16T17:29:57.924 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:57 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10[60816]: 2024-04-16T17:29:57.645+0000 7f3acfcf5700 -1 osd.10 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.512879+0000 front 2024-04-16T17:29:21.512966+0000 (oldest deadline 2024-04-16T17:29:44.412691+0000) 2024-04-16T17:29:58.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:58.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:58.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:58.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:58.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:58.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:58.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:58.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:58.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:58.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:58.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:58.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:58.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:58 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:58.185+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:29:58.673 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:58 smithi022 podman[76658]: 2024-04-16 17:29:58.411662449 +0000 UTC m=+5.493213234 container died b5d529b53511fa540da988d80d37578105940f9fbd9fd3f9ff3f5982ad65e2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-16T17:29:59.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:58 smithi022 podman[76658]: 2024-04-16 17:29:58.743044784 +0000 UTC m=+5.824595559 container cleanup b5d529b53511fa540da988d80d37578105940f9fbd9fd3f9ff3f5982ad65e2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:29:59.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:58 smithi022 bash[76658]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10 2024-04-16T17:29:59.173 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:58 smithi022 podman[76675]: 2024-04-16 17:29:58.927437353 +0000 UTC m=+0.511703600 container remove b5d529b53511fa540da988d80d37578105940f9fbd9fd3f9ff3f5982ad65e2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:29:59.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:59.226+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:29:59.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:59.226+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:29:59.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:59.226+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:29:59.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:59.226+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:29:59.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:59.226+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:29:59.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:59.226+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:29:59.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:29:59 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:29:59.226+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:29:59.614 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:59 smithi022 podman[76766]: 2024-04-16 17:29:59.320175324 +0000 UTC m=+0.104737727 container create 3ea95ee62504872e7bc02f5c2fed1808e0ce6d4fb06441dcb063821a029ff196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:29:59.614 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:59 smithi022 podman[76766]: 2024-04-16 17:29:59.233424276 +0000 UTC m=+0.017986663 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:29:59.614 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:59 smithi022 podman[76766]: 2024-04-16 17:29:59.530790173 +0000 UTC m=+0.315352557 container init 3ea95ee62504872e7bc02f5c2fed1808e0ce6d4fb06441dcb063821a029ff196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-16T17:29:59.614 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:59 smithi022 podman[76766]: 2024-04-16 17:29:59.533946957 +0000 UTC m=+0.318509339 container start 3ea95ee62504872e7bc02f5c2fed1808e0ce6d4fb06441dcb063821a029ff196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:29:59.615 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:59 smithi022 podman[76766]: 2024-04-16 17:29:59.614195668 +0000 UTC m=+0.398758070 container attach 3ea95ee62504872e7bc02f5c2fed1808e0ce6d4fb06441dcb063821a029ff196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:29:59.923 INFO:journalctl@ceph.osd.10.smithi022.stdout:Apr 16 17:29:59 smithi022 podman[76766]: 2024-04-16 17:29:59.843842738 +0000 UTC m=+0.628405121 container died 3ea95ee62504872e7bc02f5c2fed1808e0ce6d4fb06441dcb063821a029ff196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-16T17:30:00.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.10.service' 2024-04-16T17:30:00.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:30:00.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:30:00.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:30:00.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:30:00.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:30:00.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:30:00.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.213+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:30:00.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:00.527 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-16T17:30:00.528 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-16T17:30:00.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.11 2024-04-16T17:30:00.832 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 systemd[1]: Stopping Ceph osd.11 for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:01.169 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.832+0000 7fa4df0de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:01.169 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.833+0000 7fa4df0de700 -1 osd.11 143 *** Got signal Terminated *** 2024-04-16T17:30:01.169 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:00 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:00.833+0000 7fa4df0de700 -1 osd.11 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:30:01.169 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:01.169+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:30:01.170 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:01.169+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:30:01.170 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:01.169+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:30:01.170 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:01.169+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:30:01.170 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:01.169+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:30:01.170 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:01.169+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:30:01.170 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:01 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:01.169+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:30:02.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:02.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:30:02.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:02.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:30:02.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:02.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:30:02.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:02.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:30:02.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:02.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:30:02.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:02.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:30:02.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:02.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:30:02.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:02 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:02.138+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6820 osd.8 since back 2024-04-16T17:29:36.664503+0000 front 2024-04-16T17:29:36.664468+0000 (oldest deadline 2024-04-16T17:30:01.964655+0000) 2024-04-16T17:30:03.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:03.183+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:30:03.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:03.183+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:30:03.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:03.183+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:30:03.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:03.183+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:30:03.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:03.183+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:30:03.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:03.183+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:30:03.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:03.183+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:30:03.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:03 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:03.183+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6820 osd.8 since back 2024-04-16T17:29:36.664503+0000 front 2024-04-16T17:29:36.664468+0000 (oldest deadline 2024-04-16T17:30:01.964655+0000) 2024-04-16T17:30:04.500 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:04.207+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:30:04.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:04.207+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:30:04.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:04.207+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:30:04.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:04.207+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:30:04.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:04.207+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:30:04.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:04.207+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:30:04.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:04.207+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:30:04.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:04 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:04.207+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6820 osd.8 since back 2024-04-16T17:29:36.664503+0000 front 2024-04-16T17:29:36.664468+0000 (oldest deadline 2024-04-16T17:30:01.964655+0000) 2024-04-16T17:30:05.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:05.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-16T17:28:23.951836+0000 front 2024-04-16T17:28:23.951727+0000 (oldest deadline 2024-04-16T17:28:51.552370+0000) 2024-04-16T17:30:05.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:05.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6804 osd.1 since back 2024-04-16T17:28:37.954335+0000 front 2024-04-16T17:28:37.954260+0000 (oldest deadline 2024-04-16T17:29:02.654224+0000) 2024-04-16T17:30:05.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:05.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6804 osd.2 since back 2024-04-16T17:28:44.955072+0000 front 2024-04-16T17:28:44.955101+0000 (oldest deadline 2024-04-16T17:29:09.055180+0000) 2024-04-16T17:30:05.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:05.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6812 osd.4 since back 2024-04-16T17:29:04.759206+0000 front 2024-04-16T17:29:04.759292+0000 (oldest deadline 2024-04-16T17:29:30.059120+0000) 2024-04-16T17:30:05.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:05.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6812 osd.5 since back 2024-04-16T17:29:11.160056+0000 front 2024-04-16T17:29:11.159909+0000 (oldest deadline 2024-04-16T17:29:36.460061+0000) 2024-04-16T17:30:05.501 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:05.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.18:6822 osd.6 since back 2024-04-16T17:29:21.761323+0000 front 2024-04-16T17:29:21.761419+0000 (oldest deadline 2024-04-16T17:29:45.861142+0000) 2024-04-16T17:30:05.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:05.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.22:6820 osd.7 since back 2024-04-16T17:29:31.364036+0000 front 2024-04-16T17:29:31.364019+0000 (oldest deadline 2024-04-16T17:29:56.663946+0000) 2024-04-16T17:30:05.502 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:05 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11[62307]: 2024-04-16T17:30:05.239+0000 7fa4db6d2700 -1 osd.11 143 heartbeat_check: no reply from 172.21.15.43:6820 osd.8 since back 2024-04-16T17:29:36.664503+0000 front 2024-04-16T17:29:36.664468+0000 (oldest deadline 2024-04-16T17:30:01.964655+0000) 2024-04-16T17:30:06.558 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:06 smithi043 podman[77207]: 2024-04-16 17:30:06.271950293 +0000 UTC m=+5.539923478 container died 3831aec416085d3ad5a2ff5c6f02b0cab859701adda2087ee7a40ef32c15e9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-16T17:30:06.907 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:06 smithi043 podman[77207]: 2024-04-16 17:30:06.632627756 +0000 UTC m=+5.900600925 container cleanup 3831aec416085d3ad5a2ff5c6f02b0cab859701adda2087ee7a40ef32c15e9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:30:06.907 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:06 smithi043 bash[77207]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11 2024-04-16T17:30:07.250 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:06 smithi043 podman[77224]: 2024-04-16 17:30:06.908681841 +0000 UTC m=+0.630937118 container remove 3831aec416085d3ad5a2ff5c6f02b0cab859701adda2087ee7a40ef32c15e9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:30:07.586 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:07 smithi043 podman[77311]: 2024-04-16 17:30:07.168331855 +0000 UTC m=+0.018597246 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:30:07.586 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:07 smithi043 podman[77311]: 2024-04-16 17:30:07.268159786 +0000 UTC m=+0.118425195 container create 2a10863c9eefb9a32c6e782ddd41dbe4f804c66518670e593d977b98f162d8d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-16T17:30:07.586 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:07 smithi043 podman[77311]: 2024-04-16 17:30:07.504055658 +0000 UTC m=+0.354321049 container init 2a10863c9eefb9a32c6e782ddd41dbe4f804c66518670e593d977b98f162d8d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:30:07.586 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:07 smithi043 podman[77311]: 2024-04-16 17:30:07.507192807 +0000 UTC m=+0.357458200 container start 2a10863c9eefb9a32c6e782ddd41dbe4f804c66518670e593d977b98f162d8d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-16T17:30:07.587 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:07 smithi043 podman[77311]: 2024-04-16 17:30:07.587101296 +0000 UTC m=+0.437366687 container attach 2a10863c9eefb9a32c6e782ddd41dbe4f804c66518670e593d977b98f162d8d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:30:08.000 INFO:journalctl@ceph.osd.11.smithi043.stdout:Apr 16 17:30:07 smithi043 podman[77311]: 2024-04-16 17:30:07.818352454 +0000 UTC m=+0.668617853 container died 2a10863c9eefb9a32c6e782ddd41dbe4f804c66518670e593d977b98f162d8d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-16T17:30:08.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@osd.11.service' 2024-04-16T17:30:08.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:08.361 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-16T17:30:08.362 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-16T17:30:08.362 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.a 2024-04-16T17:30:08.945 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:30:08 smithi018 systemd[1]: Stopping Ceph mds.a for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:09.445 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:30:08 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a[71139]: 2024-04-16T17:30:08.947+0000 7f4e36400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:09.445 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:30:08 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a[71139]: 2024-04-16T17:30:08.947+0000 7f4e36400700 -1 mds.a *** got signal Terminated *** 2024-04-16T17:30:12.721 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:30:12 smithi018 podman[116382]: 2024-04-16 17:30:12.446347538 +0000 UTC m=+3.643441732 container died af5e644397b54022807f5ed05e45fe8d256592f2204aa35f4cac6b381f71e85d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-16T17:30:13.023 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:30:12 smithi018 podman[116382]: 2024-04-16 17:30:12.838902809 +0000 UTC m=+4.035997029 container cleanup af5e644397b54022807f5ed05e45fe8d256592f2204aa35f4cac6b381f71e85d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:30:13.023 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:30:12 smithi018 bash[116382]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a 2024-04-16T17:30:13.444 INFO:journalctl@ceph.mds.a.smithi018.stdout:Apr 16 17:30:13 smithi018 podman[116396]: 2024-04-16 17:30:13.023314458 +0000 UTC m=+0.555888264 container remove af5e644397b54022807f5ed05e45fe8d256592f2204aa35f4cac6b381f71e85d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-16T17:30:13.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.a.service' 2024-04-16T17:30:13.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:13.688 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-16T17:30:13.688 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-16T17:30:13.688 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.d 2024-04-16T17:30:14.195 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:30:13 smithi018 systemd[1]: Stopping Ceph mds.d for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:14.695 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:30:14 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d[70102]: 2024-04-16T17:30:14.315+0000 7fb69fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:14.695 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:30:14 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d[70102]: 2024-04-16T17:30:14.315+0000 7fb69fe00700 -1 mds.d *** got signal Terminated *** 2024-04-16T17:30:16.694 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:30:16 smithi018 podman[116502]: 2024-04-16 17:30:16.433254308 +0000 UTC m=+2.244065494 container died fe06c480a72fb72947438479df9aedeca788014f00a3c0ddc26a9a6a25f4b1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:30:17.101 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:30:16 smithi018 podman[116502]: 2024-04-16 17:30:16.875629544 +0000 UTC m=+2.686440723 container cleanup fe06c480a72fb72947438479df9aedeca788014f00a3c0ddc26a9a6a25f4b1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-16T17:30:17.101 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:30:16 smithi018 bash[116502]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d 2024-04-16T17:30:17.444 INFO:journalctl@ceph.mds.d.smithi018.stdout:Apr 16 17:30:17 smithi018 podman[116516]: 2024-04-16 17:30:17.101658893 +0000 UTC m=+0.650983136 container remove fe06c480a72fb72947438479df9aedeca788014f00a3c0ddc26a9a6a25f4b1ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-d, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-16T17:30:17.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.d.service' 2024-04-16T17:30:17.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:17.776 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-16T17:30:17.776 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-16T17:30:17.776 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.g 2024-04-16T17:30:18.194 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:30:17 smithi018 systemd[1]: Stopping Ceph mds.g for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:18.694 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:30:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g[70753]: 2024-04-16T17:30:18.419+0000 7fcac4800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:18.695 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:30:18 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g[70753]: 2024-04-16T17:30:18.419+0000 7fcac4800700 -1 mds.g *** got signal Terminated *** 2024-04-16T17:30:22.694 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:30:22 smithi018 podman[116623]: 2024-04-16 17:30:22.436825654 +0000 UTC m=+4.141012716 container died fc88423e24a5c3a0951c35a216e01eb3d76346d5edd57526cb64d4e678b52ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-16T17:30:23.029 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:30:22 smithi018 podman[116623]: 2024-04-16 17:30:22.812311964 +0000 UTC m=+4.516499060 container cleanup fc88423e24a5c3a0951c35a216e01eb3d76346d5edd57526cb64d4e678b52ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409) 2024-04-16T17:30:23.030 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:30:22 smithi018 bash[116623]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g 2024-04-16T17:30:23.444 INFO:journalctl@ceph.mds.g.smithi018.stdout:Apr 16 17:30:23 smithi018 podman[116637]: 2024-04-16 17:30:23.030000439 +0000 UTC m=+0.574081909 container remove fc88423e24a5c3a0951c35a216e01eb3d76346d5edd57526cb64d4e678b52ed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-g, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-16T17:30:23.482 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.g.service' 2024-04-16T17:30:23.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:23.604 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-16T17:30:23.604 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-16T17:30:23.604 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.j 2024-04-16T17:30:24.194 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:30:23 smithi018 systemd[1]: Stopping Ceph mds.j for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:24.694 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:30:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j[69745]: 2024-04-16T17:30:24.339+0000 7f37e1600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:24.695 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:30:24 smithi018 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j[69745]: 2024-04-16T17:30:24.339+0000 7f37e1600700 -1 mds.j *** got signal Terminated *** 2024-04-16T17:30:28.627 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:30:28 smithi018 podman[116742]: 2024-04-16 17:30:28.364556727 +0000 UTC m=+4.268586844 container died a65dd7e3a5af0e5a2f1f8c4cd4e2235920d3b9c2afeb1b3c6d0a3e42404315d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T17:30:28.944 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:30:28 smithi018 podman[116742]: 2024-04-16 17:30:28.740318592 +0000 UTC m=+4.644348712 container cleanup a65dd7e3a5af0e5a2f1f8c4cd4e2235920d3b9c2afeb1b3c6d0a3e42404315d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-16T17:30:28.945 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:30:28 smithi018 bash[116742]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j 2024-04-16T17:30:29.444 INFO:journalctl@ceph.mds.j.smithi018.stdout:Apr 16 17:30:28 smithi018 podman[116756]: 2024-04-16 17:30:28.97683698 +0000 UTC m=+0.592133124 container remove a65dd7e3a5af0e5a2f1f8c4cd4e2235920d3b9c2afeb1b3c6d0a3e42404315d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2) 2024-04-16T17:30:29.566 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.j.service' 2024-04-16T17:30:29.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:29.690 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-16T17:30:29.690 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-16T17:30:29.690 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.b 2024-04-16T17:30:30.008 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:30:29 smithi022 systemd[1]: Stopping Ceph mds.b for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:30.423 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:30:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: 2024-04-16T17:30:30.008+0000 7fe623954700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:30.423 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:30:30 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b[66371]: 2024-04-16T17:30:30.008+0000 7fe623954700 -1 mds.b *** got signal Terminated *** 2024-04-16T17:30:31.333 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:30:31 smithi022 podman[76872]: 2024-04-16 17:30:31.129044935 +0000 UTC m=+1.218518431 container died f45263a6338f69dd1d53fe9cdcbc24c0b2477ef91c721b0afffb37baf7868eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:30:31.673 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:30:31 smithi022 podman[76872]: 2024-04-16 17:30:31.408629846 +0000 UTC m=+1.498103335 container cleanup f45263a6338f69dd1d53fe9cdcbc24c0b2477ef91c721b0afffb37baf7868eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T17:30:31.673 INFO:journalctl@ceph.mds.b.smithi022.stdout:Apr 16 17:30:31 smithi022 bash[76872]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-b 2024-04-16T17:30:31.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.b.service' 2024-04-16T17:30:31.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:31.866 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-16T17:30:31.866 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-16T17:30:31.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.e 2024-04-16T17:30:32.423 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:30:31 smithi022 systemd[1]: Stopping Ceph mds.e for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:32.423 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:30:32 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e[65726]: 2024-04-16T17:30:32.196+0000 7ff16dc7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:32.423 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:30:32 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e[65726]: 2024-04-16T17:30:32.196+0000 7ff16dc7f700 -1 mds.e *** got signal Terminated *** 2024-04-16T17:30:35.403 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:30:35 smithi022 podman[77004]: 2024-04-16 17:30:35.13300527 +0000 UTC m=+3.031408563 container died 596e7819f6cb225811acf23bd89e15d21acf167f37b0fe6ff34d7516a34de00f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-16T17:30:35.403 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:30:35 smithi022 podman[77004]: 2024-04-16 17:30:35.40459634 +0000 UTC m=+3.302999631 container cleanup 596e7819f6cb225811acf23bd89e15d21acf167f37b0fe6ff34d7516a34de00f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS) 2024-04-16T17:30:35.673 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:30:35 smithi022 bash[77004]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e 2024-04-16T17:30:35.673 INFO:journalctl@ceph.mds.e.smithi022.stdout:Apr 16 17:30:35 smithi022 podman[77018]: 2024-04-16 17:30:35.555796219 +0000 UTC m=+0.418331534 container remove 596e7819f6cb225811acf23bd89e15d21acf167f37b0fe6ff34d7516a34de00f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:30:35.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.e.service' 2024-04-16T17:30:35.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:35.815 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-16T17:30:35.815 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-16T17:30:35.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.h 2024-04-16T17:30:36.140 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:30:35 smithi022 systemd[1]: Stopping Ceph mds.h for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:36.422 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:30:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[73346]: 2024-04-16T17:30:36.141+0000 7f2cd8cf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:36.423 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:30:36 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h[73346]: 2024-04-16T17:30:36.141+0000 7f2cd8cf0700 -1 mds.h *** got signal Terminated *** 2024-04-16T17:30:37.923 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:30:37 smithi022 podman[77124]: 2024-04-16 17:30:37.608960822 +0000 UTC m=+1.568624993 container died 790615c7636ee89019078a3e2c4c1b679e84120a44ccc1d61a36fa9b47d13c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-16T17:30:38.357 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:30:37 smithi022 podman[77124]: 2024-04-16 17:30:37.924875311 +0000 UTC m=+1.884539509 container cleanup 790615c7636ee89019078a3e2c4c1b679e84120a44ccc1d61a36fa9b47d13c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-16T17:30:38.357 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:30:37 smithi022 bash[77124]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h 2024-04-16T17:30:38.357 INFO:journalctl@ceph.mds.h.smithi022.stdout:Apr 16 17:30:38 smithi022 podman[77138]: 2024-04-16 17:30:38.12599484 +0000 UTC m=+0.510535948 container remove 790615c7636ee89019078a3e2c4c1b679e84120a44ccc1d61a36fa9b47d13c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:30:38.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.h.service' 2024-04-16T17:30:38.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:38.412 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-16T17:30:38.413 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-16T17:30:38.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.k 2024-04-16T17:30:38.673 INFO:journalctl@ceph.mds.k.smithi022.stdout:Apr 16 17:30:38 smithi022 systemd[1]: Stopping Ceph mds.k for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:39.173 INFO:journalctl@ceph.mds.k.smithi022.stdout:Apr 16 17:30:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-k[65405]: 2024-04-16T17:30:38.769+0000 7f39cdba0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:39.173 INFO:journalctl@ceph.mds.k.smithi022.stdout:Apr 16 17:30:38 smithi022 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-k[65405]: 2024-04-16T17:30:38.769+0000 7f39cdba0700 -1 mds.k *** got signal Terminated *** 2024-04-16T17:30:40.412 INFO:journalctl@ceph.mds.k.smithi022.stdout:Apr 16 17:30:40 smithi022 podman[77244]: 2024-04-16 17:30:40.124317191 +0000 UTC m=+1.476626679 container died a99abe97caad0d43d5da02f7d6b6bacd66e805c32f2c147a26f74864bbd32dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T17:30:40.673 INFO:journalctl@ceph.mds.k.smithi022.stdout:Apr 16 17:30:40 smithi022 podman[77244]: 2024-04-16 17:30:40.529280744 +0000 UTC m=+1.881590226 container cleanup a99abe97caad0d43d5da02f7d6b6bacd66e805c32f2c147a26f74864bbd32dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409) 2024-04-16T17:30:40.803 INFO:journalctl@ceph.mds.k.smithi022.stdout:Apr 16 17:30:40 smithi022 bash[77244]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-k 2024-04-16T17:30:40.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.k.service' 2024-04-16T17:30:40.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:40.933 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-16T17:30:40.933 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-16T17:30:40.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.c 2024-04-16T17:30:41.231 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:30:40 smithi043 systemd[1]: Stopping Ceph mds.c for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:41.501 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:30:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c[67328]: 2024-04-16T17:30:41.230+0000 7efd763fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:41.501 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:30:41 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c[67328]: 2024-04-16T17:30:41.230+0000 7efd763fd700 -1 mds.c *** got signal Terminated *** 2024-04-16T17:30:43.372 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:30:43 smithi043 podman[77417]: 2024-04-16 17:30:43.108776369 +0000 UTC m=+1.975410012 container died 66e397f7e4e148a97ea2f05510ea9b6939fb292040d4ee89ac47838725bfc931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5) 2024-04-16T17:30:43.739 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:30:43 smithi043 podman[77417]: 2024-04-16 17:30:43.464562939 +0000 UTC m=+2.331196578 container cleanup 66e397f7e4e148a97ea2f05510ea9b6939fb292040d4ee89ac47838725bfc931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-16T17:30:43.739 INFO:journalctl@ceph.mds.c.smithi043.stdout:Apr 16 17:30:43 smithi043 bash[77417]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-c 2024-04-16T17:30:43.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.c.service' 2024-04-16T17:30:43.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:43.935 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-16T17:30:43.935 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-16T17:30:43.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.f 2024-04-16T17:30:44.267 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:30:44 smithi043 systemd[1]: Stopping Ceph mds.f for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:44.750 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:30:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f[67009]: 2024-04-16T17:30:44.268+0000 7f3c9cd59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:44.751 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:30:44 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f[67009]: 2024-04-16T17:30:44.268+0000 7f3c9cd59700 -1 mds.f *** got signal Terminated *** 2024-04-16T17:30:47.369 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:30:47 smithi043 podman[77551]: 2024-04-16 17:30:47.090542853 +0000 UTC m=+2.926121663 container died b8f38de83cbb10ebe0cf61b1904699445c2b1ab0cf2aff1e39a93cab633af6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:30:47.709 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:30:47 smithi043 podman[77551]: 2024-04-16 17:30:47.476585778 +0000 UTC m=+3.312164573 container cleanup b8f38de83cbb10ebe0cf61b1904699445c2b1ab0cf2aff1e39a93cab633af6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-04-16T17:30:47.709 INFO:journalctl@ceph.mds.f.smithi043.stdout:Apr 16 17:30:47 smithi043 bash[77551]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-f 2024-04-16T17:30:47.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.f.service' 2024-04-16T17:30:47.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:47.898 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-16T17:30:47.898 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-16T17:30:47.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.i 2024-04-16T17:30:48.221 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:47 smithi043 systemd[1]: Stopping Ceph mds.i for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:48.501 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i[67650]: 2024-04-16T17:30:48.221+0000 7f2a2a582700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:48.501 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:48 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i[67650]: 2024-04-16T17:30:48.221+0000 7f2a2a582700 -1 mds.i *** got signal Terminated *** 2024-04-16T17:30:51.371 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:51 smithi043 podman[77672]: 2024-04-16 17:30:51.112347235 +0000 UTC m=+2.989673234 container died 859a77133dd022a6cd8f7958844b21da44d29bd577d49357c5b650052e53a6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-16T17:30:51.629 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:51 smithi043 podman[77672]: 2024-04-16 17:30:51.446794801 +0000 UTC m=+3.324120816 container cleanup 859a77133dd022a6cd8f7958844b21da44d29bd577d49357c5b650052e53a6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-16T17:30:51.630 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:51 smithi043 bash[77672]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i 2024-04-16T17:30:51.880 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:51 smithi043 podman[77686]: 2024-04-16 17:30:51.631158468 +0000 UTC m=+0.516140880 container remove 859a77133dd022a6cd8f7958844b21da44d29bd577d49357c5b650052e53a6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-04-16T17:30:51.880 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:51 smithi043 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-16T17:30:51.880 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:51 smithi043 systemd[1]: Stopped Ceph mds.i for 872930f8-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:30:51.880 INFO:journalctl@ceph.mds.i.smithi043.stdout:Apr 16 17:30:51 smithi043 systemd[1]: ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.i.service: Consumed 1min 22.766s CPU time. 2024-04-16T17:30:51.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.i.service' 2024-04-16T17:30:51.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:51.922 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-16T17:30:51.922 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-16T17:30:51.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.l 2024-04-16T17:30:52.240 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:30:52 smithi043 systemd[1]: Stopping Ceph mds.l for 872930f8-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:30:52.492 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:30:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l[66698]: 2024-04-16T17:30:52.241+0000 7f943e9bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:30:52.492 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:30:52 smithi043 ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l[66698]: 2024-04-16T17:30:52.241+0000 7f943e9bf700 -1 mds.l *** got signal Terminated *** 2024-04-16T17:30:56.501 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:30:56 smithi043 podman[77792]: 2024-04-16 17:30:56.247542881 +0000 UTC m=+4.103998805 container died 83bd4e8f0a300ca3e7007ad18e6c93a98b9d395672734f5cd31d39ab78dc0533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-16T17:30:56.874 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:30:56 smithi043 podman[77792]: 2024-04-16 17:30:56.691581724 +0000 UTC m=+4.548037655 container cleanup 83bd4e8f0a300ca3e7007ad18e6c93a98b9d395672734f5cd31d39ab78dc0533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-16T17:30:56.874 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:30:56 smithi043 bash[77792]: ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l 2024-04-16T17:30:57.129 INFO:journalctl@ceph.mds.l.smithi043.stdout:Apr 16 17:30:56 smithi043 podman[77806]: 2024-04-16 17:30:56.875760211 +0000 UTC m=+0.621417241 container remove 83bd4e8f0a300ca3e7007ad18e6c93a98b9d395672734f5cd31d39ab78dc0533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-872930f8-fc11-11ee-bc90-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:30:57.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-872930f8-fc11-11ee-bc90-c7b262605968@mds.l.service' 2024-04-16T17:30:57.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:30:57.171 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-16T17:30:57.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 872930f8-fc11-11ee-bc90-c7b262605968 --force --keep-logs 2024-04-16T17:30:57.515 INFO:teuthology.orchestra.run.smithi018.stdout:Deleting cluster with fsid: 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T17:31:02.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 872930f8-fc11-11ee-bc90-c7b262605968 --force --keep-logs 2024-04-16T17:31:02.729 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T17:31:05.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 872930f8-fc11-11ee-bc90-c7b262605968 --force --keep-logs 2024-04-16T17:31:05.585 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T17:31:08.268 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:31:08.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:31:08.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:31:08.448 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-16T17:31:08.451 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi018/crash 2024-04-16T17:31:08.452 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash -- . 2024-04-16T17:31:08.564 INFO:teuthology.orchestra.run.smithi018.stderr:tar: /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T17:31:08.564 INFO:teuthology.orchestra.run.smithi018.stderr:tar: Error is not recoverable: exiting now 2024-04-16T17:31:08.570 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi022/crash 2024-04-16T17:31:08.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash -- . 2024-04-16T17:31:08.653 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T17:31:08.653 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2024-04-16T17:31:08.656 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi043/crash 2024-04-16T17:31:08.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash -- . 2024-04-16T17:31:08.700 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/872930f8-fc11-11ee-bc90-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T17:31:08.700 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-04-16T17:31:08.701 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-16T17:31:08.701 DEBUG:teuthology.orchestra.run.smithi018:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-16T17:31:08.845 INFO:tasks.cephadm:Compressing logs... 2024-04-16T17:31:08.845 DEBUG:teuthology.orchestra.run.smithi018:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T17:31:08.850 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T17:31:08.852 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T17:31:08.884 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-04-16T17:31:08.884 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/cephadm.log 2024-04-16T17:31:08.884 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.l.log 2024-04-16T17:31:08.884 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/cephadm.log: /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log 2024-04-16T17:31:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.i.log 2024-04-16T17:31:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.11.log 2024-04-16T17:31:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.z.log 2024-04-16T17:31:08.887 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.8.log 2024-04-16T17:31:08.888 INFO:teuthology.orchestra.run.smithi043.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T17:31:08.888 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.5.log 2024-04-16T17:31:08.888 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.f.log 2024-04-16T17:31:08.888 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log 2024-04-16T17:31:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log.gz 2024-04-16T17:31:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log 2024-04-16T17:31:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log.gz 2024-04-16T17:31:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.2.log 2024-04-16T17:31:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log: 94.4% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.l.log.gz 2024-04-16T17:31:08.893 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.c.log 2024-04-16T17:31:08.896 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.2.log: 87.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log.gz 2024-04-16T17:31:08.897 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log 2024-04-16T17:31:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-16T17:31:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T17:31:08.900 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.c.log: 92.6% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.z.log.gz 2024-04-16T17:31:08.900 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.c.log 2024-04-16T17:31:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.10.log 2024-04-16T17:31:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log 2024-04-16T17:31:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.b.log 2024-04-16T17:31:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.4.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.b.log: -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.1.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.k.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.h.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.7.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.e.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.y.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log 2024-04-16T17:31:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log 2024-04-16T17:31:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.b.log 2024-04-16T17:31:08.905 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log: /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.b.log: 94.3% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.k.log.gz 2024-04-16T17:31:08.909 INFO:teuthology.orchestra.run.smithi022.stderr: 92.5% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.y.log.gz 2024-04-16T17:31:08.913 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log: /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log.gz 2024-04-16T17:31:08.936 INFO:teuthology.orchestra.run.smithi018.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-16T17:31:08.936 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T17:31:08.937 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.6.log 2024-04-16T17:31:08.938 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.a.log 2024-04-16T17:31:08.939 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log 2024-04-16T17:31:08.940 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.g.log 2024-04-16T17:31:08.941 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.j.log 2024-04-16T17:31:08.942 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.3.log 2024-04-16T17:31:08.943 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.x.log 2024-04-16T17:31:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.0.log 2024-04-16T17:31:08.945 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log 2024-04-16T17:31:08.946 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log 2024-04-16T17:31:08.947 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log 2024-04-16T17:31:08.949 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.9.log 2024-04-16T17:31:08.950 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.d.log 2024-04-16T17:31:08.951 INFO:teuthology.orchestra.run.smithi018.stderr: 94.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.j.log.gz 2024-04-16T17:31:08.951 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.a.log 2024-04-16T17:31:08.952 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.9.log: /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-16T17:31:08.953 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.a.log: 87.1% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log.gz 2024-04-16T17:31:08.956 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.admin.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-16T17:31:08.957 INFO:teuthology.orchestra.run.smithi018.stderr: 94.4% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.d.log.gz 2024-04-16T17:31:08.959 INFO:teuthology.orchestra.run.smithi018.stderr: 94.4% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.a.log.gz 2024-04-16T17:31:08.966 INFO:teuthology.orchestra.run.smithi018.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T17:31:09.010 INFO:teuthology.orchestra.run.smithi022.stderr: 84.1% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log.gz 2024-04-16T17:31:09.036 INFO:teuthology.orchestra.run.smithi018.stderr: 84.3% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log.gz 2024-04-16T17:31:09.055 INFO:teuthology.orchestra.run.smithi018.stderr: 90.8% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log.gz 2024-04-16T17:31:09.172 INFO:teuthology.orchestra.run.smithi022.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T17:31:09.183 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.log.gz 2024-04-16T17:31:09.202 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-volume.log.gz 2024-04-16T17:31:09.214 INFO:teuthology.orchestra.run.smithi022.stderr: 90.3% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log.gz 2024-04-16T17:31:09.215 INFO:teuthology.orchestra.run.smithi043.stderr: 94.1% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.f.log.gz 2024-04-16T17:31:09.219 INFO:teuthology.orchestra.run.smithi018.stderr: 90.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph.audit.log.gz 2024-04-16T17:31:09.595 INFO:teuthology.orchestra.run.smithi043.stderr: 91.7% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.c.log.gz 2024-04-16T17:31:09.688 INFO:teuthology.orchestra.run.smithi018.stderr: 90.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mgr.x.log.gz 2024-04-16T17:31:10.647 INFO:teuthology.orchestra.run.smithi043.stderr: 95.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.c.log.gz 2024-04-16T17:31:11.740 INFO:teuthology.orchestra.run.smithi018.stderr: 91.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.a.log.gz 2024-04-16T17:31:24.028 INFO:teuthology.orchestra.run.smithi043.stderr: 94.9% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.i.log.gz 2024-04-16T17:31:26.412 INFO:teuthology.orchestra.run.smithi022.stderr: 91.6% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mon.b.log.gz 2024-04-16T17:31:45.959 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.11.log.gz 2024-04-16T17:31:54.072 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.8.log.gz 2024-04-16T17:31:55.842 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.2.log.gz 2024-04-16T17:32:02.264 INFO:teuthology.orchestra.run.smithi018.stderr: 93.6% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.g.log.gz 2024-04-16T17:32:03.268 INFO:teuthology.orchestra.run.smithi018.stderr: 91.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.0.log.gz 2024-04-16T17:32:11.728 INFO:teuthology.orchestra.run.smithi018.stderr: 91.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.3.log.gz 2024-04-16T17:32:13.959 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.5.log.gz 2024-04-16T17:32:13.961 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-16T17:32:13.961 INFO:teuthology.orchestra.run.smithi043.stderr:real 1m5.093s 2024-04-16T17:32:13.961 INFO:teuthology.orchestra.run.smithi043.stderr:user 3m25.851s 2024-04-16T17:32:13.961 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m5.801s 2024-04-16T17:32:24.531 INFO:teuthology.orchestra.run.smithi018.stderr: 91.1% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.6.log.gz 2024-04-16T17:32:28.448 INFO:teuthology.orchestra.run.smithi018.stderr: 91.1% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.9.log.gz 2024-04-16T17:32:28.453 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-16T17:32:28.454 INFO:teuthology.orchestra.run.smithi018.stderr:real 1m19.567s 2024-04-16T17:32:28.454 INFO:teuthology.orchestra.run.smithi018.stderr:user 4m8.499s 2024-04-16T17:32:28.454 INFO:teuthology.orchestra.run.smithi018.stderr:sys 0m19.197s 2024-04-16T17:32:59.541 INFO:teuthology.orchestra.run.smithi022.stderr: 94.7% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.h.log.gz 2024-04-16T17:33:29.805 INFO:teuthology.orchestra.run.smithi022.stderr: 93.9% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.e.log.gz 2024-04-16T17:35:07.048 INFO:teuthology.orchestra.run.smithi022.stderr: 91.5% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.10.log.gz 2024-04-16T17:35:33.114 INFO:teuthology.orchestra.run.smithi022.stderr: 91.4% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.1.log.gz 2024-04-16T17:36:14.854 INFO:teuthology.orchestra.run.smithi022.stderr: 91.2% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.7.log.gz 2024-04-16T17:37:01.406 INFO:teuthology.orchestra.run.smithi022.stderr: 91.4% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-osd.4.log.gz 2024-04-16T17:41:33.580 INFO:teuthology.orchestra.run.smithi022.stderr: 93.0% -- replaced with /var/log/ceph/872930f8-fc11-11ee-bc90-c7b262605968/ceph-mds.b.log.gz 2024-04-16T17:41:33.590 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-16T17:41:33.590 INFO:teuthology.orchestra.run.smithi022.stderr:real 10m24.723s 2024-04-16T17:41:33.590 INFO:teuthology.orchestra.run.smithi022.stderr:user 7m57.757s 2024-04-16T17:41:33.591 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m17.196s 2024-04-16T17:41:33.591 INFO:tasks.cephadm:Archiving logs... 2024-04-16T17:41:33.592 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi018/log 2024-04-16T17:41:33.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T17:42:08.956 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi022/log 2024-04-16T17:42:08.962 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T17:43:23.702 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi043/log 2024-04-16T17:43:23.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T17:43:57.568 INFO:tasks.cephadm:Removing cluster... 2024-04-16T17:43:57.569 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 872930f8-fc11-11ee-bc90-c7b262605968 --force 2024-04-16T17:43:57.915 INFO:teuthology.orchestra.run.smithi018.stdout:Deleting cluster with fsid: 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T17:43:59.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 872930f8-fc11-11ee-bc90-c7b262605968 --force 2024-04-16T17:43:59.739 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T17:44:00.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 872930f8-fc11-11ee-bc90-c7b262605968 --force 2024-04-16T17:44:00.632 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 872930f8-fc11-11ee-bc90-c7b262605968 2024-04-16T17:44:01.192 INFO:tasks.cephadm:Removing cephadm ... 2024-04-16T17:44:01.192 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T17:44:01.254 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T17:44:01.273 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T17:44:01.289 INFO:tasks.cephadm:Teardown complete 2024-04-16T17:44:01.290 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-16T17:44:01.301 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-16T17:44:01.302 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T17:44:01.308 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T17:44:01.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T17:44:01.483 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-16T17:44:01.494 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-16T17:44:01.494 DEBUG:teuthology.orchestra.run.smithi018:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:44:01.502 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:44:01.505 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:44:01.519 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-16T17:44:01.519 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-04-16T17:44:01.523 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T17:44:01.523 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-16T17:44:01.523 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 +751us[ +744us] +/- 56ms 2024-04-16T17:44:01.523 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -2234us[-2241us] +/- 64ms 2024-04-16T17:44:01.523 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +1911us[+1904us] +/- 89ms 2024-04-16T17:44:01.523 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T17:44:01.553 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-04-16T17:44:01.563 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T17:44:01.564 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-04-16T17:44:01.564 INFO:teuthology.orchestra.run.smithi018.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +750us[ +767us] +/- 56ms 2024-04-16T17:44:01.564 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -2233us[-2217us] +/- 64ms 2024-04-16T17:44:01.564 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +1895us[+1912us] +/- 89ms 2024-04-16T17:44:01.564 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T17:44:01.628 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T17:44:01.628 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-04-16T17:44:01.628 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +726us[ +691us] +/- 56ms 2024-04-16T17:44:01.628 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -2248us[-2282us] +/- 64ms 2024-04-16T17:44:01.628 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +1889us[+1889us] +/- 89ms 2024-04-16T17:44:01.628 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T17:44:01.630 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-16T17:44:01.641 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-16T17:44:01.641 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-16T17:44:01.652 DEBUG:teuthology.orchestra.run.smithi018:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T17:44:01.931 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T17:44:02.074 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T17:44:02.162 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T17:44:02.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:44:02.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T17:44:02.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:44:02.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T17:44:02.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:44:02.347 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-16T17:44:02.359 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-16T17:44:02.370 INFO:teuthology.task.internal:Duration was 3673.085677 seconds 2024-04-16T17:44:02.370 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-16T17:44:02.383 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-16T17:44:02.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T17:44:02.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T17:44:02.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T17:44:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T17:44:02.461 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T17:44:02.578 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T17:44:03.088 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-16T17:44:03.088 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi018.front.sepia.ceph.com 2024-04-16T17:44:03.088 DEBUG:teuthology.orchestra.run.smithi018:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T17:44:03.151 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-04-16T17:44:03.152 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T17:44:03.182 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-04-16T17:44:03.183 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T17:44:03.209 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-16T17:44:03.210 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T17:44:03.217 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T17:44:03.225 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T17:44:03.315 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-16T17:44:03.316 DEBUG:teuthology.orchestra.run.smithi018:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T17:44:03.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T17:44:03.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T17:44:16.852 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-16T17:44:16.864 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-16T17:44:16.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T17:44:16.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T17:44:17.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T17:44:17.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-16T17:44:17.054 DEBUG:teuthology.orchestra.run.smithi018:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:44:17.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:44:17.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:44:17.113 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-04-16T17:44:17.117 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-04-16T17:44:17.173 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = core 2024-04-16T17:44:17.229 DEBUG:teuthology.orchestra.run.smithi018:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:44:17.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:44:17.280 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:44:17.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:44:17.298 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:44:17.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:44:17.314 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-16T17:44:17.326 INFO:teuthology.task.internal:Transferring archived files... 2024-04-16T17:44:17.327 DEBUG:teuthology.misc:Transferring archived files from smithi018:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi018 2024-04-16T17:44:17.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T17:44:17.787 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi022 2024-04-16T17:44:17.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T17:44:17.835 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438/remote/smithi043 2024-04-16T17:44:17.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T17:44:17.883 INFO:teuthology.task.internal:Removing archive directory... 2024-04-16T17:44:17.883 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T17:44:17.891 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T17:44:17.893 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T17:44:17.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-16T17:44:17.969 INFO:teuthology.task.internal:Not uploading archives. 2024-04-16T17:44:17.969 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-16T17:44:17.982 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-16T17:44:17.982 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T17:44:17.990 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T17:44:17.992 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T17:44:18.008 INFO:teuthology.orchestra.run.smithi022.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 17:44 /home/ubuntu/cephtest 2024-04-16T17:44:18.010 INFO:teuthology.orchestra.run.smithi043.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 17:44 /home/ubuntu/cephtest 2024-04-16T17:44:18.042 INFO:teuthology.orchestra.run.smithi018.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 17:44 /home/ubuntu/cephtest 2024-04-16T17:44:18.045 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-16T17:44:18.058 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-16T17:44:18.084 INFO:teuthology.nuke:Checking targets against current locks 2024-04-16T17:44:18.110 DEBUG:teuthology.nuke:shortname: smithi018 2024-04-16T17:44:18.110 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T17:44:18.134 DEBUG:teuthology.nuke:shortname: smithi022 2024-04-16T17:44:18.134 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T17:44:18.139 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.017147', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAo4EezMFHe1z901+6BtI0fVVwcpdTxG4yIKOw6LLTIs95phj17UsQ982onIZxnxa5uJO7iw3QvIgJ8j3M2GjEU='} 2024-04-16T17:44:18.156 DEBUG:teuthology.nuke:shortname: smithi043 2024-04-16T17:44:18.156 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T17:44:18.166 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.015479', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDhzcHsZsvFDokU439Oqm3OxXlUxW+WgmFwyGYD6esZF/hUhBBQBW9Y1ABceWKyM8NjhjqzMDiJg06elct/2WpE='} 2024-04-16T17:44:18.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658438', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.018500', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiririN9/EWV7rRFNXeAQeezT+rGESyk0NhmvQT5TpQZQGQsT3MwV83KrND2u57or8cyOpX2NQsWXfkK1+y25I='} 2024-04-16T17:44:18.212 INFO:teuthology.orchestra.console.smithi022:Power off 2024-04-16T17:44:18.213 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T17:44:18.242 INFO:teuthology.orchestra.console.smithi043:Power off 2024-04-16T17:44:18.243 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T17:44:18.256 INFO:teuthology.orchestra.console.smithi018:Power off 2024-04-16T17:44:18.256 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T17:44:18.269 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2024-04-16T17:44:18.269 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:18.279 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2024-04-16T17:44:18.279 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:18.289 DEBUG:teuthology.orchestra.console.smithi018:power off output: Chassis Power Control: Down/Off 2024-04-16T17:44:18.289 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:18.297 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-04-16T17:44:18.310 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on 2024-04-16T17:44:18.331 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-04-16T17:44:22.299 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:22.311 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:22.332 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:22.410 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-04-16T17:44:22.423 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on 2024-04-16T17:44:22.441 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-04-16T17:44:26.412 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:26.424 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:26.442 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:26.522 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2024-04-16T17:44:26.534 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is off 2024-04-16T17:44:26.550 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-04-16T17:44:26.624 INFO:teuthology.orchestra.console.smithi043:Power off completed 2024-04-16T17:44:26.635 INFO:teuthology.orchestra.console.smithi018:Power off completed 2024-04-16T17:44:30.555 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:44:30.665 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-04-16T17:44:30.766 INFO:teuthology.orchestra.console.smithi022:Power off completed 2024-04-16T17:44:30.872 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3673.0856766700745 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi018 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b137e4a675398e295567f9652210865a4d573fac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1c3db1d8239f4a198174c124580123a5 status: fail success: false 2024-04-16T17:44:30.872 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:44:30.962 INFO:teuthology.run:FAIL